FAQ

How to collect events?

Pods of runtime-audit-engine output all events to stdout. Those events can then be collected by log-shipper-agents and sent to a log storage.

Below is an example ClusterLoggingConfig configuration for the log-shipper module:

apiVersion: deckhouse.io/v1alpha2
kind: ClusterLoggingConfig
metadata:
  name: falco-events
spec:
  destinationRefs:
  - xxxx
  kubernetesPods:
    namespaceSelector:
      labelSelector:
        matchExpressions:
        - key: "kubernetes.io/metadata.name"
          operator: In
          values: [d8-runtime-audit-engine]
  labelFilter:
  - operator: Regex
    values: ["\\{.*"] # To collect only JSON logs.
    field: "message"
  type: KubernetesPods

How to get notifications on critical events?

All event metrics are automatically collected by Prometheus. To enable critical alerts, add a CustomPrometheusRule rule to the cluster.

Example:

apiVersion: deckhouse.io/v1
kind: CustomPrometheusRules
metadata:
  name: falco-critical-alerts
spec:
  groups:
  - name: falco-critical-alerts
    rules:
    - alert: FalcoCriticalAlertsAreFiring
      for: 1m
      annotations:
        description: |
          There is a suspicious activity on a node {{ $labels.node }}. 
          Check you events journal for details.
        summary: Falco detects a critical security incident
      expr: |
        sum by (node) (rate(falcosecurity_falcosidekick_falco_events_total{priority="Critical"}[5m]) > 0)

How to apply Falco rules found on the internet?

The structure of native Falco rules is different from the CRD schema. It is due to limitations of schema validation capabilities in Kubernetes.

The script for converting Falco rules to FalcoAuditRules resources is built into the d8 utility functionality. Using it, you can apply Falco rules in Deckhouse:

d8 tools far-converter /path/to/falco/rule_example.yaml > ./my-rules-cr.yaml

Example of a script output:

# /path/to/falco/rule_example.yaml
- macro: spawned_process
  condition: (evt.type in (execve, execveat) and evt.dir=<)

- rule: Linux Cgroup Container Escape Vulnerability (CVE-2022-0492)
  desc: "This rule detects an attempt to exploit a container escape vulnerability in the Linux Kernel."
  condition: container.id != "" and proc.name = "unshare" and spawned_process and evt.args contains "mount" and evt.args contains "-o rdma" and evt.args contains "/release_agent"
  output: "Detect Linux Cgroup Container Escape Vulnerability (CVE-2022-0492) (user=%user.loginname uid=%user.loginuid command=%proc.cmdline args=%proc.args)"
  priority: CRITICAL
  tags: [process, mitre_privilege_escalation]

# ./my-rules-cr.yaml
apiVersion: deckhouse.io/v1alpha1
kind: FalcoAuditRules
metadata:
  name: rule-example
spec:
  rules:
    - macro:
        name: spawned_process
        condition: (evt.type in (execve, execveat) and evt.dir=<)
    - rule:
        name: Linux Cgroup Container Escape Vulnerability (CVE-2022-0492)
        condition: container.id != "" and proc.name = "unshare" and spawned_process and evt.args contains "mount" and evt.args contains "-o rdma" and evt.args contains "/release_agent"
        desc: This rule detects an attempt to exploit a container escape vulnerability in the Linux Kernel.
        output: Detect Linux Cgroup Container Escape Vulnerability (CVE-2022-0492) (user=%user.loginname uid=%user.loginuid command=%proc.cmdline args=%proc.args)
        priority: Critical
        tags:
          - process
          - mitre_privilege_escalation

How to configure resources for DaemonSet?

The runtime-audit-engine module allows you to flexibly configure resource requests for its DaemonSet. This is especially useful in heterogeneous clusters where nodes have different capacities.

Resource Management Modes

You can choose between two modes for managing resource requests using the resourcesRequests.mode parameter:

• Static: You explicitly specify the resources for the pods. See resourcesRequests.static.
• VPA: Resources are managed automatically by the Vertical Pod Autoscaler. See resourcesRequests.vpa.

Example of static configuration:

apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: runtime-audit-engine
spec:
  version: 1
  enabled: true
  settings:
    resourcesRequests:
      mode: Static
      static:
        cpu: "100m"
        memory: "256Mi"

VPA Scoping (Recommendation Grouping)

VPA Scoping is a mechanism that allows the Vertical Pod Autoscaler to generate independent resource recommendations for different groups of pods within a single DaemonSet. This is crucial for clusters with heterogeneous nodes where the same workload (e.g., Falco) might consume significantly different amounts of resources depending on the node’s characteristics.

How it works

By default, in Deckhouse versions 1.76 and higher, the module uses the resourcesRequests.vpa.scopeLabel parameter with the standard value node.deckhouse.io/group.

VPA monitors the specified label on the nodes and automatically splits the DaemonSet pods into subgroups:

• __absent__: Nodes that do not have the specified label key.
• "" (empty string): Nodes where the label key exists but has no value.
• Specific values: A separate recommendation group is created for every unique value of the label (e.g., worker, system, gpu).

When to change the scope label

You might need to change the scopeLabel if your resource consumption depends on factors other than the standard Deckhouse node groups. For example, in Yandex Cloud, you might want to group recommendations by the physical size (flavor) of the instance or by custom hardware markers.

Example: Grouping by Yandex Cloud instance type using the label node.kubernetes.io/instance-type:

apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: runtime-audit-engine
spec:
  version: 1
  enabled: true
  settings:
    resourcesRequests:
      mode: VPA
      vpa:
        mode: Auto
        scopeLabel: "node.kubernetes.io/instance-type"

With this configuration, a pod running on a c3-m8 instance will receive a different VPA recommendation than a pod running on a s2-m2 instance, ensuring optimal resource allocation across your entire cloud infrastructure.

LimitRange and VPA Interaction

LimitRange is optional and is not used by default.

Use resourcesRequests.limitRange as follows:

• false — explicitly disable LimitRange creation;
• object — create LimitRange with the specified boundaries.

Important: When using VPA, the LimitRange values act as hard boundaries. VPA recommendations will be capped by the max values and padded by the min values defined in the LimitRange.

Minimum Resource Requirements

To ensure that pods can start and function correctly, you must ensure that LimitRange values are not lower than the static resources defined for the containers.

The module uses the following static resources for its components:

• Falco: 50m CPU, 64Mi Memory.
• Falcosidekick: 5m CPU, 10Mi Memory.
• Rules Loader: 10m CPU, 25Mi Memory.

If you set containerMin or containerDefaultRequest in LimitRange higher than these values, the pods will fail to start.

Recommended minimum values for LimitRange:

• Memory: At least 64Mi for containerMin and containerDefaultRequest to accommodate Falco.
• CPU: At least 50m.

Example of a safe configuration:

apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: runtime-audit-engine
spec:
  version: 1
  enabled: true
  settings:
    resourcesRequests:
      limitRange:
        containerMax:
          memory: "1Gi" # Default
        containerMin:
          memory: "64Mi"
        containerDefaultRequest:
          memory: "64Mi"