The module lifecycle stage: General Availability

How to run kube-bench in my cluster?

First, you have to exec in Deckhouse Pod:

d8 k -n d8-system exec -ti svc/deckhouse-leader -c deckhouse -- bash

Then you have to select which node you want to run kube-bench.

  • Run on random node:

    curl -s https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job.yaml | d8 k create -f -

  • Run on specific node, e.g. control-plane node:

    curl -s https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job.yaml | d8 k apply -f - --dry-run=client -o json | jq '.spec.template.spec.tolerations=[{"operator": "Exists"}] | .spec.template.spec.nodeSelector={"node-role.kubernetes.io/control-plane": ""}' | d8 k create -f -

Then you can check report:

d8 k logs job.batch/kube-bench

Deckhouse set the log retention period to 7 days. However, according to the security requirements specified in kube-bench, logs should be retained for at least 30 days. Use separate storage for logs if you need to keep logs for more than 7 days.

How to collect debug info?

  1. Create a diagnostic archive with the d8 utility, redirecting its output (stdout) to a file:

    d8 system collect-debug-info > deckhouse-debug-$(date +"%Y_%m_%d").tar.gz

  2. Send the resulting archive to the Deckhouse team for further debugging.

The --exclude flag omits the specified items from the archive. Example:

  d8 system collect-debug-info --exclude=queue global-values > deckhouse-debug-$(date +"%Y_%m_%d").tar.gz

The --list-exclude flag prints the list of items available for exclusion. Example:

  d8 system collect-debug-info --list-exclude

The following information is produced when creating the archive. Names in the “File in archive” column correspond to top-level items inside the resulting tar.gz archive. Certain sensitive values (e.g., kubeRBACProxyCA and registry.dockercfg) are excluded.

Category Collected data File in archive
Deckhouse Deckhouse queue state queue
Deckhouse values (excluding kubeRBACProxyCA and registry.dockercfg) global-values
Current deckhouse pod version deckhouse-version
All DeckhouseRelease objects deckhouse-releases
Deckhouse pod logs deckhouse-logs
Manifests of controllers and pods from all Deckhouse namespaces d8-all
Cluster objects NodeGroup node-groups
NodeGroupConfiguration node-group-configuration
Node nodes
Machine machines
Instance instances
StaticInstance staticinstances
MachineDeployment cloud-machine-deployment, static-machine-deployment
ClusterAuthorizationRule cluster-authorization-rules
AuthorizationRule authorization-rules
ModuleConfig module-configs
Events (all namespaces) events
Modules and states List of enabled modules deckhouse-enabled-modules
ModuleSource objects in the cluster deckhouse-module-sources
ModulePullOverride objects in the cluster deckhouse-module-pull-overrides
Modules in maintenance mode deckhouse-maintenance-modules
Controller logs and manifests machine-controller-manager logs mcm-logs
cloud-controller-manager logs ccm-logs
csi-controller logs csi-controller-logs
cluster-autoscaler logs cluster-autoscaler-logs
Vertical Pod Autoscaler admission controller logs vpa-admission-controller-logs
Vertical Pod Autoscaler recommender logs vpa-recommender-logs
Vertical Pod Autoscaler updater logs vpa-updater-logs
capi-controller-manager YAML capi-controller-manager
caps-controller-manager YAML caps-controller-manager
machine-controller-manager YAML machine-controller-manager
Monitoring and alerts Prometheus logs prometheus-logs
Active (firing) Prometheus alerts alerts
Pods not in Running (excluding Completed and Evicted) bad-pods
List of Audit Policies audit-policy
Network All objects in the d8-istio namespace d8-istio-resources
All istio custom resources d8-istio-custom-resources
Envoy configuration for istio d8-istio-envoy-config
istio logs d8-istio-system-logs
istio ingress gateway logs d8-istio-ingress-logs
istio users logs d8-istio-users-logs
Cilium connection status (cilium health status) cilium-health-status

How to debug pod problems with ephemeral containers?

Run the following command:

d8 k -n <namespace_name> debug -it <pod_name> --image=ubuntu <container_name>

More info in official documentation.

How to debug node problems with ephemeral containers?

Run the following command:

d8 k debug node/mynode -it --image=ubuntu

More info in official documentation.