The operator-trivy module: CIS Kubernetes Benchmark checks

The module performs automated cluster security checks according to the CIS Kubernetes Benchmark specification.

Version

The module implements checks according to CIS Kubernetes Benchmark v1.23 specification.

Component versions:

d8 k -n d8-operator-trivy get pods -o json | \
  jq -r '.items[] | .metadata.name as $p | .spec.containers[] | "\($p)\t\(.name)\t\(.image)"'

Check categories

CIS Kubernetes Benchmark checks are grouped into the following categories:

Exclusions

In Deckhouse Kubernetes Platform (DKP), some checks are disabled or excluded from reports. This is due to platform architecture and system component requirements.

Globally disabled checks

The following checks are disabled for all cluster resources:

CIS 1.2.1 — Anonymous auth (AVD-KCV-0001)

Check: Ensure that the --anonymous-auth argument is set to false.

Status: Completely disabled.

Reason: The Trivy/defsec CIS 1.2.1 implementation is based on inspecting the legacy --anonymous-auth flag. On newer Kubernetes versions, anonymous authentication configuration may be managed differently (for example, via AuthenticationConfiguration), therefore a flag-based check may produce false FAIL/WARN results (including in kube-bench). For auditing, rely on the actual kube-apiserver configuration and access controls in your environment.

CIS 1.2.13 — SecurityContextDeny (AVD-KCV-0013)

Check: Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used.

Status: Completely disabled.

Reason: The SecurityContextDeny admission controller was deprecated and completely removed in Kubernetes v1.30. This controller blocked pods with privileged settings, but its approach was too coarse and didn’t allow granular policy configuration.

Alternative in DKP: Deckhouse uses Pod Security Standards implemented in the admission-policy-engine module based on OPA Gatekeeper. This allows:

• Applying Privileged, Baseline, or Restricted policies at namespace level
• Creating exceptions for specific workloads
• Flexible rule configuration per organization requirements

Checks disabled for system namespaces

The following checks are disabled for kube-system and d8-* namespaces (all Deckhouse namespaces start with d8- prefix).

Why do system components need privileges?

A Kubernetes cluster requires components that work at a low level: network management, storage, host resource monitoring. These components by their nature cannot run in an isolated container without privileges.

Partial exclusions

Specific exclusions apply to individual resources:

CIS 5.7.4 — Default namespace (AVD-KSV-0110)

Check: The default namespace should not be used.

Exclusion: Resource default/service-kubernetes.

Reason: The kubernetes service in the default namespace is a built-in Kubernetes API Server service. It’s created automatically during cluster initialization and cannot be moved to another namespace. This is standard Kubernetes behavior documented in official documentation.

CIS 5.7.3 — Security Context (AVD-KSV-0020, AVD-KSV-0021)

Check: Apply Security Context to Your Pods and Containers.

Exclusion: ReplicaSets in d8-* namespaces.

Reason: Checks require runAsUser > 10000 and runAsGroup > 10000 (using unprivileged UID/GID). In DKP there are system workloads that run as root (runAsUser/runAsGroup = 0) or do not set runAsUser/runAsGroup explicitly — this can be required for host filesystem compatibility and low-level operations.

• Compatibility with host file permissions
• Correct operation with volumes where permissions are already configured
• Alignment with standard system service practices

How to check which objects are “below the 5.7.3 threshold” in your cluster (readable format):

Short summary (pod-level runAsUser/runAsGroup) for system namespaces d8-*, kube-system, kube-public, kube-node-lease:

d8 k get deploy,rs,ds,sts -A -o json | jq -r '
  .items[]
  | select(.metadata.namespace | test("^(d8-)|^(kube-system|kube-public|kube-node-lease)$"))
  | . as $o
  | ($o.spec.template.spec.securityContext.runAsUser // null) as $u
  | ($o.spec.template.spec.securityContext.runAsGroup // null) as $g
  | select((($u|tonumber? // -1) <= 10000) or (($g|tonumber? // -1) <= 10000))
  | [
      $o.kind,
      $o.metadata.namespace,
      $o.metadata.name,
      ("runAsUser=" + (($u|tostring) // "null")),
      ("runAsGroup=" + (($g|tostring) // "null"))
    ] | @tsv'

Container-level detail to see what is set on pod vs container level:

d8 k get deploy,rs,ds,sts -A -o json | jq -r '
  .items[]
  | select(.metadata.namespace | test("^(d8-)|^(kube-system|kube-public|kube-node-lease)$"))
  | . as $o
  | ($o.spec.template.spec.securityContext.runAsUser // null) as $pu
  | ($o.spec.template.spec.securityContext.runAsGroup // null) as $pg
  | $o.spec.template.spec.containers[]?
  | . as $c
  | ($c.securityContext.runAsUser // null) as $cu
  | ($c.securityContext.runAsGroup // null) as $cg
  | select((($pu|tonumber? // 999999) <= 10000) or (($pg|tonumber? // 999999) <= 10000) or (($cu|tonumber? // 999999) <= 10000) or (($cg|tonumber? // 999999) <= 10000))
  | [
      $o.kind,
      $o.metadata.namespace,
      $o.metadata.name,
      ("pod.runAsUser=" + (($pu|tostring) // "null")),
      ("pod.runAsGroup=" + (($pg|tostring) // "null")),
      ("container=" + $c.name),
      ("container.runAsUser=" + (($cu|tostring) // "null")),
      ("container.runAsGroup=" + (($cg|tostring) // "null"))
    ] | @tsv'

Checks excluded from metrics

The following checks are performed, but their results are not displayed in Prometheus metrics or on the Grafana dashboard. This is intentional so the dashboard reflects real security issues, not false positives.

CIS 5.1.2 — Access to secrets (AVD-KSV-0041)

Check: Minimize access to secrets.

What it checks: Finds all Roles/ClusterRoles with get, list, or watch permissions on the secrets resource.

Why excluded from metrics:

In the original CIS Benchmark, this check is marked as type: manual and scored: false because:

• It’s impossible to automatically determine if secrets access is legitimate
• Any controller working with TLS or credentials requires such access

Examples of roles with legitimate secrets access in DKP (non-exhaustive):

• d8:ingress-nginx:kruise-role — managing secrets for ingress controllers
• d8:node-manager:machine-controller-manager — access to cloud provider credentials
• d8:operator-prometheus — Prometheus operator ClusterRole reads/creates secrets
• Role/d8-log-shipper/log-shipper — log-shipper Role reads secrets in d8-log-shipper

How to see the full list of roles with secrets access:

# All Roles/ClusterRoles that have secrets access (get/list/watch)
d8 k get clusterroles,roles -A -o json | jq -r '
  .items[]
  | select(any(.rules[]?;
      ((.resources // []) | index("secrets")) != null
      and (
        ((.verbs // []) | index("get")) != null
        or ((.verbs // []) | index("list")) != null
        or ((.verbs // []) | index("watch")) != null
      )
    ))
  | "\(.kind)\t\(.metadata.namespace // "-")\t\(.metadata.name)"'

# Quick lookup of bindings by role name (example)
role_name="d8:operator-prometheus"
d8 k get clusterrolebindings,rolebindings -A | grep -F "$role_name"

How to analyze results:

# Get all roles with secrets access
d8 k get clustercompliancereports.aquasecurity.github.io cis -ojson | \
  jq '.status.detailReport.results | map(select(.id == "5.1.2")) | .[].checks'

Analyze the list and ensure that:

• All roles belong to known system components
• No user roles have excessive access

CIS 5.1.3 — Wildcard use in Roles (AVD-KSV-0044, AVD-KSV-0045, AVD-KSV-0046)

Check: Minimize wildcard use in Roles and ClusterRoles.

What it checks: Finds roles with wildcards (*) in resources, verbs, or apiGroups fields.

Why excluded from metrics:

In the original CIS Benchmark, this check is marked as type: manual and scored: false because:

• cluster-admin — standard Kubernetes role, contains wildcards
• Operators often require broad access to manage diverse resources

Examples of legitimate wildcard use in DKP:

• cluster-admin — full access for administrators
• deckhouse ClusterRole — managing all Deckhouse modules
• CRD operators — access to all resources in their API group

How to analyze results:

# Get all roles with wildcards
d8 k get clustercompliancereports.aquasecurity.github.io cis -ojson | \
  jq '.status.detailReport.results | map(select(.id == "5.1.3")) | .[].checks'

Ensure wildcards are used only in:

• Kubernetes system roles (cluster-admin, admin, edit, view)
• Deckhouse component roles (d8-*)
• Known operator roles

Known issues

Kubelet TLS certificate check (CIS 4.2.10)

Checks AVD-KCV-0088 and AVD-KCV-0089 may show FAIL status. This is expected behavior: Deckhouse uses automatic kubelet certificate rotation (RotateKubeletServerCertificate) instead of static --tls-cert-file and --tls-private-key-file files.

See control-plane-manager module FAQ for details.

Manual checks

The following checks require manual verification and are not automated. This aligns with the original CIS Benchmark specification where they are marked as Manual.

Mapping to kube-bench

DKP uses Trivy Operator for CIS checks instead of the kube-bench utility. Check mapping:

Use CIS Control ID (e.g., 1.2.1, 5.2.2) to compare results.

Viewing results

Grafana

Check results are available on the Security / CIS Kubernetes Benchmark dashboard.

Command line

Get all results:

d8 k get clustercompliancereports.aquasecurity.github.io cis -ojson | \
  jq '.status.detailReport.results'

Get only failed checks:

d8 k get clustercompliancereports.aquasecurity.github.io cis -ojson | \
  jq '.status.detailReport.results | map(select(.checks | map(.success) | all | not))'

Get results for a specific check:

d8 k get clustercompliancereports.aquasecurity.github.io cis -ojson | \
  jq '.status.detailReport.results | map(select(.id == "5.7.3"))'

Check schedule

CIS Benchmark checks run:

• every 6 hours (cron: 0 */6 * * *);
• on operator startup.