The module implements a runtime threats detection engine.
The module collects Linux kernel system calls and Kubernetes API audit events (by
k8saudit plugin enabled by default), enrich them with metadata from Kubernetes Pods and generate security audit events according to conditional rules.
- Detects threats at runtime by observing the behavior of your applications and containers.
- Helps to detect CVEs exploits and cryptocurrency mining attacks.
- Improves Kubernetes security by detecting:
- Shells running in containers or Pods in Kubernetes.
- Containers running in privileged mode or attempting to mount sensitive paths, such as
/proc, on the host.
- Unauthorized attempts to read confidential files such as
The module is based on the Falco system. Deckhouse deploys Falco agents (which run as a DaemonSet) on every node. The agents then start consuming kernel / kube audit events.
Falco developers recommend deploying Falco as a systemd unit for maximum security. However, a Kubernetes cluster with the autoscaling feature enabled makes it hard to operate. Additional security mechanisms of Deckhouse (implemented by other modules), such as multitenancy and admission policy control, provide the required level of security to mitigate attacks on the Falco DaemonSet.
There are four different containers in a single agent Pod:
falco— collects events, enriches them with metadata and sends them to stdout.
rules-loader— collects (FalcoAuditRules) CRs from Kubernetes and saves them in a shared directory (empty dir).
falcosidekick— it takes a
Falcoevents and forward them to different outputs in a fan-out way. By default, it exports events as metrics on which alerts can be generated. Falcosidekick source code.
kube-rbac-proxy— protects the
The event collection itself is a low-yielding activity because the amount of data coming from the Linux kernel is too large to be analyzed by a human. Rules address this problem by collecting events according to certain pattens that can help in detecting suspicious activities.
The main part of a rule is a conditional expression (which uses the conditions syntax).
There is a built-in set of rules that cannot be disabled. It helps to identify problems with Deckhouse security as well as security problems affecting the
/etc/falco/k8s_audit_rules.yaml— Kubernetes audit rules.
Custom audit rules
Users can use a
FalcoAuditRules CRD to add custom security audit rules.
Each Falco agent Pod has a sidecar container running shell-operator.
This sidecar reads rules from the custom resources, converts them to Falco rules and saves Falco rules in the Pod’s
Falco automatically reloads the configuration when a new rule becomes available.
Such a schema allows the IaC approach to be used to maintain Falco rules.
The module uses the eBPF Falco driver to ingest syscall data. It is better suited for environments where loading a kernel module is prohibited or unsupported, such as GKE, EKS, and other managed Kubernetes solutions. However, there are known limitations to the eBPF driver:
- The eBPF probe may not work for every system.
- At least a Linux kernel version 4.14 is required, although the Falco project suggests an LTS kernel version 4.14/4.19 or higher.
CPU / Memory
Falco agents are running on every node. Therefore, the resource consumption of each Pod depends on the number of rules or ingested events.
Kubernetes Audit Webhook
Webhook audit mode should be configured to collect audit events of
If the control-plane-manager module is enabled, settings will be automatically applied when the
runtime-audit-engine module is enabled.
You can manually configure the webhook for Kubernetes clusters with a control plane that is not controlled by Deckhouse:
Create a webhook kubeconfig file with the
https://127.0.0.1:9765/k8s-auditaddress and the CA (ca.crt) from the
apiVersion: v1 kind: Config clusters: - name: webhook cluster: certificate-authority-data: BASE64_CA server: "https://127.0.0.1:9765/k8s-audit" users: - name: webhook contexts: - context: cluster: webhook user: webhook name: webhook current-context: webhook
--audit-webhook-config-fileflag to the
kube-apiservermanifest. The flag must point to the previously created file.
Remember to configure the audit policy, because Deckhouse only collects Kubernetes audit events from the system namespaces by default. An example of configuration can be found in the control-plane-manager module documentation.
If a number of
runtime-audit-engine pods are not scheduled, then the
D8RuntimeAuditEngineNotScheduledInCluster alert will be generated.