The deckhouse module: FAQ

How to run kube-bench in my cluster?

First, you have to exec in Deckhouse Pod:

d8 k -n d8-system exec -ti svc/deckhouse-leader -c deckhouse -- bash

Then you have to select which node you want to run kube-bench.

• Run on random node:

  curl -s https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job.yaml | d8 k create -f -
  

• Run on specific node, e.g. control-plane node:

  curl -s https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job.yaml | d8 k apply -f - --dry-run=client -o json | jq '.spec.template.spec.tolerations=[{"operator": "Exists"}] | .spec.template.spec.nodeSelector={"node-role.kubernetes.io/control-plane": ""}' | d8 k create -f -
  

Then you can check report:

d8 k logs job.batch/kube-bench

How to collect debug info?

1. Create a diagnostic archive with the d8 utility, redirecting its output (stdout) to a file:

   d8 system collect-debug-info > deckhouse-debug-$(date +"%Y_%m_%d").tar.gz
   

2. Send the resulting archive to the Deckhouse team for further debugging.

The --exclude flag omits the specified items from the archive. Example:

  d8 system collect-debug-info --exclude=queue global-values > deckhouse-debug-$(date +"%Y_%m_%d").tar.gz

The --list-exclude flag prints the list of items available for exclusion. Example:

  d8 system collect-debug-info --list-exclude

The following information is produced when creating the archive. Names in the “File in archive” column correspond to top-level items inside the resulting tar.gz archive. Certain sensitive values (e.g., kubeRBACProxyCA and registry.dockercfg) are excluded.

How to debug pod problems with ephemeral containers?

Run the following command:

d8 k -n <namespace_name> debug -it <pod_name> --image=ubuntu <container_name>

More info in official documentation.

An ephemeral container cannot be added to a Pod labeled heritage: deckhouse, see the next section.

Who can run exec into Deckhouse Platform pods and scale Deckhouse Platform objects?

Only Deckhouse Platform system users. For all other users, including those with the ClusterAdmin and SuperAdmin roles, the deckhouse module prohibits the following operations in any namespace:

• kubectl exec, kubectl attach, kubectl cp, kubectl port-forward and kubectl debug into a Pod labeled heritage: deckhouse. The restriction also applies to kubectl port-forward to Services if the request is directed to a protected Pod.
• kubectl scale of Deployment, StatefulSet, ReplicaSet or ReplicationController resources labeled heritage: deckhouse, unless the object is under maintenance, that is, labeled maintenance.deckhouse.io/no-resource-reconciliation.

The heritage: deckhouse label is checked on the object itself. For example, if a Deployment has this label but a Pod created by it does not, the access restrictions do not apply to that Pod.

These restrictions do not apply to the following users and components:

• system:sudouser.
• dhctl and observability.
• Service accounts from d8-* and kube-* namespaces (system:serviceaccount:d8-*, system:serviceaccount:kube-*), including the Deckhouse Platform controllers and the Horizontal Pod Autoscaler from kube-controller-manager.
• Kubernetes control plane components (system:apiserver, system:kube-controller-manager, system:kube-scheduler, system:volume-scheduler) and nodes belonging to the system:nodes group.
• Members of the system:masters group, that is, when using super-admin.conf on a master node. The restrictions still apply when using admin.conf (the kubeadm:cluster-admins group).

Except for system:masters, these are the same users that are allowed to modify resources with the heritage: deckhouse label. The restrictions apply regardless of whether the admission-policy-engine module is enabled in the cluster.

Requests are validated by the webhook-handler Pods in the d8-system namespace. If these Pods are unavailable, requests are rejected. In this case, the following operations are blocked in all namespaces for all users except those listed above, regardless of whether the heritage: deckhouse label is present:

• kubectl exec, kubectl attach, kubectl cp, and kubectl port-forward for any Pod.
• kubectl scale for any Deployment, StatefulSet, ReplicaSet, or ReplicationController resource.

To access a Pod for diagnostics, you can use system:sudouser or super-admin.conf on a master node.

Example of executing a command in a Pod as system:sudouser:

d8 k --as=system:sudouser -n d8-system exec -it <POD_NAME> -- sh

How to debug node problems with ephemeral containers?

Run the following command:

d8 k debug node/mynode -it --image=ubuntu

More info in official documentation.