The operator-trivy module: runtime application and network map

In addition to static scanning (image vulnerabilities, configuration audit, CIS), the module can observe what each workload actually does at runtime and turn those observations into a per-workload map.

What the map contains

When the map is enabled, an agent runs on every node and watches each container’s behaviour using eBPF. The observations are aggregated per workload (Deployment, DaemonSet, StatefulSet, …) into two objects:

• ApplicationProfile: Processes the workload launches, files it opens, and Linux capabilities and system calls it uses (the application fingerprint).
• NetworkNeighborhood: Inbound and outbound connections the workload makes, by peer and port (the network map).

Both objects are built from the behaviour observed in the running cluster, not from workload manifests.

Requirements

The runtime map has the following requirements:

• Node kernels must be Linux 5.4 or newer and expose BTF at /sys/kernel/btf/vmlinux.
• The agent runs privileged with host PID access. It is scheduled on every node by default; restrict it with nodeAgent.nodeSelector if some nodes do not meet the kernel requirement. Optional nodeAgent.tolerations control scheduling on tainted nodes.
• The module stores the map in its own built-in database (security-storage) instead of the cluster’s etcd. Use a StorageClass backed by network or local block devices, or by local paths (localpath), for module data. Do not use a StorageClass backed by network file or object storage, such as NFS, CephFS, or S3. The apiServerStorageClass parameter sets the StorageClass for security-storage. The PVC size is set by securityStoragePVCRequest (default 10 GiB).

Enabling

Enable the runtime map in ModuleConfig operator-trivy:

apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: operator-trivy
spec:
  settings:
    nodeAgent:
      enabled: true
      learningPeriod: "1h"
  version: 2

learningPeriod controls how long each container is observed before its profile is considered complete. During this window the map keeps absorbing newly seen behaviour; afterwards the profile is finalized. Use a longer value (for example 24h) for workloads whose behaviour varies over the day.

Viewing the map

The map objects are namespaced, alongside the workloads they describe.

To list the application fingerprints across the cluster, run:

d8 k get applicationprofiles -A

To list the network maps across the cluster, run:

d8 k get networkneighborhoods -A

To get the full fingerprint of one workload, run:

d8 k -n <NAMESPACE> get applicationprofile <NAME> -o yaml

Where:

• <NAMESPACE>: Namespace of the workload.
• <NAME>: Name of the ApplicationProfile from the output of d8 k get applicationprofiles -A.

The recorded detail (processes, files, capabilities, connections) is returned only by a request for a named object, such as d8 k -n <NAMESPACE> get applicationprofile <NAME> -o yaml. Any list request, with or without -A, returns the objects without those fields, so in a list they always look “empty”.

Data starts appearing within a few minutes of a container starting; the profile keeps growing and is marked complete once the learning period of the container elapses. Give a freshly enabled workload at least learningPeriod before expecting a complete map.

Generating least-privilege policies

Once a workload’s map is learned, the module can turn it into least-privilege policies. Enable either or both:

spec:
  settings:
    nodeAgent:
      enabled: true
      generatePolicies:
        networkPolicy: true
        seccomp: true
  version: 2

Network policies

With networkPolicy: true, a GeneratedNetworkPolicy is computed from each workload’s observed NetworkNeighborhood: a ready-to-review Kubernetes NetworkPolicy describing exactly the traffic the workload was seen to use.

To list the generated policies across the cluster, run:

d8 k get generatednetworkpolicies -A

To get the policy of one workload, run:

d8 k -n <NAMESPACE> get generatednetworkpolicy <NAME> -o yaml

Where:

• <NAMESPACE>: Namespace of the workload.
• <NAME>: Name of the GeneratedNetworkPolicy from the output of d8 k get generatednetworkpolicies -A.

The object embeds a standard NetworkPolicy under .spec. Review it, then extract and apply it to enforce least privilege:

d8 k -n <NAMESPACE> get generatednetworkpolicy <NAME> -o json | jq '.spec' | d8 k apply -f -

Seccomp profiles

With seccomp: true, the module generates a SeccompProfile from each workload’s observed system calls and updates it every 10 minutes. A SeccompProfile holds a separate profile for each container of the workload, in .spec.containers[].spec. The profiles are generated for the x86 architectures only (SCMP_ARCH_X86_64, SCMP_ARCH_X86, SCMP_ARCH_X32), so use them only for pods that run on x86-64 nodes.

To list the generated profiles, run:

d8 k get seccompprofiles.spdx.softwarecomposition.kubescape.io -A

The module does not copy profiles to nodes. To use a profile, do the following:

1. Extract the profile of one container to a file:

   d8 k -n <NAMESPACE> get seccompprofiles.spdx.softwarecomposition.kubescape.io <NAME> -o json | jq '.spec.containers[] | select(.name == "<CONTAINER_NAME>") | .spec' > <PROFILE_FILE>.json

   Where:

   • <NAMESPACE>: Namespace of the workload.
   • <NAME>: Name of the SeccompProfile from the output of the previous command.
   • <CONTAINER_NAME>: Name of the container.
   • <PROFILE_FILE>: Name of the profile file.

2. Copy the file to the kubelet seccomp directory <KUBELET_ROOT>/seccomp/ on every node where the pod can run. <KUBELET_ROOT> is the kubelet root directory, /var/lib/kubelet by default.

3. Reference the profile in the container’s security context. The localhostProfile path is relative to <KUBELET_ROOT>/seccomp/:

   spec:
     containers:
       - name: <CONTAINER_NAME>
         securityContext:
           seccompProfile:
             type: Localhost
             localhostProfile: <PROFILE_FILE>.json