The module lifecycle stage: General Availability
How do I configure alternative security policy management solutions?
For DP to work correctly, extended privileges are required to run and operate system component payloads. If you are using some alternative security policy management solution (e.g., Kyverno) instead of the admission-policy-engine module, you have to configure exceptions for the following namespaces:
kube-system;- all namespaces with the
d8-*prefix (e.g.,d8-system).
How do I configure policy selectors?
In OperationPolicy and SecurityPolicy, the spec.match field determines which specific objects (pods) in the cluster the policy will apply to. It must be present in the configuration. Filtering is performed by combining two main criteria: a pod selector (labelSelector) and a namespace selector (namespaceSelector).
If both selectors are specified, the policy is applied only to those pods that simultaneously:
- satisfy the pod selection conditions.
- are located in namespaces that passed the filtering.
If either selector is omitted, the corresponding check is not performed (all pods or namespaces will be used).
spec.match.labelSelector– pod selection
The labelSelector is used to set criteria for selecting pods by their labels. Two mutually exclusive methods are supported:
matchLabels– a simple check for an exact label match (key-value). The pod must have all the specified labels.matchExpressions– flexible expressions with operators. Each expression is defined by an object with the following fields:key(string, required) – the label name.operator(string, required) – one of the values:In,NotIn,Exists,DoesNotExist.values(array of strings) – a list of values for theIn/NotInoperators; not specified forExists/DoesNotExist.
All elements in the matchExpressions list are combined with a logical AND – the pod must satisfy every expression.
Examples:
spec:
match:
labelSelector:
matchLabels:
app: nginx
role: frontend
spec:
match:
labelSelector:
matchExpressions:
- key: tier
operator: In
values:
- production
- staging
- key: monitoring
operator: Exists
spec.match.namespaceSelector– namespace selection
Allows limiting the namespaces in which the policy is active. You can use a combination of three filters:
matchNames– an explicit list of allowed namespaces. If specified, the policy only applies to the listed namespaces.excludeNames– a list of excluded namespaces. The policy will apply to all namespaces except those specified.labelSelector– a selector based on the labels of the Namespace object itself.
If multiple fields are set, they are combined with logical AND:
- Start with the set from
matchNames(or all namespaces ifmatchNamesis omitted). - Apply
labelSelector(if set). - Subtract
excludeNames.
Formula:
result = (base_from_matchNames ∩ selected_by_labelSelector) \ excludeNames
It is recommended not to mix matchNames, excludeNames, and labelSelector without a clear need.
When to use labelSelector
Use labelSelector when the policy should automatically apply to a group of namespaces by attribute rather than by fixed names.
For example:
- “all namespaces with label
env=prod”; - “all
team=backendnamespaces” (with the corresponding label); - “all namespaces with
security.deckhouse.io/pod-policy=restricted”.
labelSelector is especially useful when namespaces are created/deleted dynamically: it is enough to automatically set a label on the namespace when it is created, and the policy will apply without editing the policy.
labelSelector is not required if you have a small static list of namespaces — in that case, using matchNames is simpler and easier to read.
Can matchNames and labelSelector be used together?
Yes, technically they are not mutually exclusive: you can specify both, and then their intersection applies.
But in practice this often hurts readability and complicates maintenance. Therefore, it is recommended to choose one primary selection method:
- either
matchNames + excludeNames; - or
labelSelector + excludeNames.
This makes it easier to understand why a particular namespace is included in or excluded from the policy.
Typical scenarios
- Static environments list →
matchNames + excludeNames. - Dynamic environments/teams →
labelSelector + excludeNames. matchNames + labelSelectorcombination — only when you really need the intersection of two independent conditions.
Examples:
Static namespace list (matchNames + excludeNames)
spec:
match:
namespaceSelector:
matchNames:
- production
- staging
excludeNames:
- staging
Result: the policy applies only in the production namespace.
Dynamic label-based selection (labelSelector + excludeNames)
spec:
match:
namespaceSelector:
labelSelector:
matchLabels:
team: backend
environment: production
excludeNames:
- backend-sandbox
Result: all namespaces with labels team=backend and environment=production, except backend-sandbox.
Flexible filtering with expressions (labelSelector.matchExpressions)
spec:
match:
namespaceSelector:
labelSelector:
matchExpressions:
- key: compliance
operator: In
values:
- pci
- sox
- key: lifecycle
operator: NotIn
values:
- deprecated
Result: only namespaces with labels compliance=pci or compliance=sox and without label lifecycle=deprecated.
matchNames and labelSelector combination (intersection, use carefully)
spec:
match:
namespaceSelector:
matchNames:
- production
- staging
- qa
labelSelector:
matchLabels:
team: backend
Result: applies only to namespaces that simultaneously:
- are in the list
production|staging|qa; - have label
team=backend.
For example, if qa does not have team=backend, it will not be matched.
How do I extend Pod Security Standards policies?
Pod Security Standards respond to the security.deckhouse.io/pod-policy: restricted or security.deckhouse.io/pod-policy: baseline label.
To extend the Pod Security Standards policy by adding your checks to existing checks, you need to:
- Create a constraint template for the check (
ConstraintTemplate). - Bind it to the
restrictedorbaselinepolicy.
Example of the ConstraintTemplate for checking a repository URL of a container image:
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
name: k8sallowedrepos
spec:
crd:
spec:
names:
kind: K8sAllowedRepos
validation:
openAPIV3Schema:
type: object
properties:
repos:
type: array
items:
type: string
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package d8.pod_security_standards.extended
violation[{"msg": msg}] {
container := input.review.object.spec.containers[_]
satisfied := [good | repo = input.parameters.repos[_] ; good = startswith(container.image, repo)]
not any(satisfied)
msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v", [container.name, container.image, input.parameters.repos])
}
violation[{"msg": msg}] {
container := input.review.object.spec.initContainers[_]
satisfied := [good | repo = input.parameters.repos[_] ; good = startswith(container.image, repo)]
not any(satisfied)
msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v", [container.name, container.image, input.parameters.repos])
}
Example of binding a check to the restricted policy:
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sAllowedRepos
metadata:
name: prod-repo
spec:
match:
kinds:
- apiGroups: [""]
kinds: ["Pod"]
namespaceSelector:
matchLabels:
security.deckhouse.io/pod-policy: restricted
parameters:
repos:
- "mycompany.registry.com"
The example demonstrates the configuration of checking the repository address in the image field for all Pods created in the namespace having the security.deckhouse.io/pod-policy: restricted label. A Pod will not be created if the address in the image field of the Pod does not start with mycompany.registry.com.
For more information about templates and the policy language, see the Gatekeeper documentation.
More examples of checks for policy extension are available in the Gatekeeper Library.
How to allow some Pod Security Standards policies without disabling whole list?
To apply only the required security policies without turning off the entire built-in set:
- Add the
security.deckhouse.io/pod-policy: privilegedlabel to your namespace in order to disable built-in policies. - Create a SecurityPolicy that matches the baseline or restricted policy while also editing the list of
policieselements as you see fit. Take the rules from thed8-pod-security-baselineord8-pod-security-restrictedobject the module publishes the standards as, see Rules of the standards. - Add a label to your namespace that matches the
namespaceSelectorin the SecurityPolicy. In the examples below, the label issecurity-policy.deckhouse.io/baseline-enabled: "true"orsecurity-policy.deckhouse.io/restricted-enabled: "true".
The allowedHostPaths field defines the list of allowed path prefixes for hostPath mounts. An empty list ([]) forbids hostPath for all paths. If the field is omitted, no restrictions apply.
SecurityPolicy that matches baseline:
apiVersion: deckhouse.io/v1alpha1
kind: SecurityPolicy
metadata:
name: baseline
spec:
enforcementAction: Deny
policies:
allowHostIPC: false
allowHostNetwork: false
allowHostPID: false
allowPrivilegeEscalation: true
allowPrivileged: false
allowedAppArmor:
- runtime/default
- localhost/*
allowedCapabilities:
- AUDIT_WRITE
- CHOWN
- DAC_OVERRIDE
- FOWNER
- FSETID
- KILL
- MKNOD
- NET_BIND_SERVICE
- SETFCAP
- SETGID
- SETPCAP
- SETUID
- SYS_CHROOT
allowedHostPaths: []
allowedHostPorts:
- max: 0
min: 0
allowedProcMount: Default
allowedUnsafeSysctls:
- kernel.shm_rmid_forced
- net.ipv4.ip_local_port_range
- net.ipv4.ip_unprivileged_port_start
- net.ipv4.tcp_syncookies
- net.ipv4.ping_group_range
- net.ipv4.ip_local_reserved_ports
- net.ipv4.tcp_keepalive_time
- net.ipv4.tcp_fin_timeout
- net.ipv4.tcp_keepalive_intvl
- net.ipv4.tcp_keepalive_probes
seLinux:
- type: ""
- type: container_t
- type: container_init_t
- type: container_kvm_t
- type: container_engine_t
seccompProfiles:
allowedProfiles:
- RuntimeDefault
- Localhost
- undefined
- ''
allowedLocalhostFiles:
- '*'
match:
namespaceSelector:
labelSelector:
matchLabels:
security-policy.deckhouse.io/baseline-enabled: "true"
SecurityPolicy that matches restricted:
apiVersion: deckhouse.io/v1alpha1
kind: SecurityPolicy
metadata:
name: restricted
spec:
enforcementAction: Deny
policies:
allowHostIPC: false
allowHostNetwork: false
allowHostPID: false
allowPrivilegeEscalation: false
allowPrivileged: false
allowedAppArmor:
- runtime/default
- localhost/*
allowedCapabilities:
- NET_BIND_SERVICE
allowedHostPaths: []
allowedHostPorts:
- max: 0
min: 0
allowedProcMount: Default
allowedUnsafeSysctls:
- kernel.shm_rmid_forced
- net.ipv4.ip_local_port_range
- net.ipv4.ip_unprivileged_port_start
- net.ipv4.tcp_syncookies
- net.ipv4.ping_group_range
- net.ipv4.ip_local_reserved_ports
- net.ipv4.tcp_keepalive_time
- net.ipv4.tcp_fin_timeout
- net.ipv4.tcp_keepalive_intvl
- net.ipv4.tcp_keepalive_probes
allowedVolumes:
- configMap
- csi
- downwardAPI
- emptyDir
- ephemeral
- persistentVolumeClaim
- projected
- secret
requiredDropCapabilities:
- ALL
runAsUser:
rule: MustRunAsNonRoot
seLinux:
- type: ""
- type: container_t
- type: container_init_t
- type: container_kvm_t
- type: container_engine_t
seccompProfiles:
allowedProfiles:
- RuntimeDefault
- Localhost
allowedLocalhostFiles:
- '*'
match:
namespaceSelector:
labelSelector:
matchLabels:
security-policy.deckhouse.io/restricted-enabled: "true"
What if there are multiple policies (operational or security) that are applied to the same object?
In that case the object’s specification has to fulfill all the requirements imposed by the policies.
For example, consider the following two security policies:
apiVersion: deckhouse.io/v1alpha1
kind: SecurityPolicy
metadata:
name: foo
spec:
enforcementAction: Deny
match:
namespaceSelector:
labelSelector:
matchLabels:
name: test
policies:
readOnlyRootFilesystem: true
requiredDropCapabilities:
- MKNOD
---
apiVersion: deckhouse.io/v1alpha1
kind: SecurityPolicy
metadata:
name: bar
spec:
enforcementAction: Deny
match:
namespaceSelector:
labelSelector:
matchLabels:
name: test
policies:
requiredDropCapabilities:
- NET_BIND_SERVICE
Then, in order to fulfill the requirements of the above security policies, the following settings must be set in a container specification:
securityContext:
capabilities:
drop:
- MKNOD
- NET_BIND_SERVICE
readOnlyRootFilesystem: true
Why is an Assign or AssignMetadata mutation rejected or silently ignored?
The Assign, AssignMetadata, ModifySet, and AssignImage CRDs are imported from Gatekeeper without modification (see Mutation Custom Resources). However, the Gatekeeper mutation webhook enforces content restrictions that go beyond the CRD schema. The admission-policy-engine module enforces the checkable restrictions with built-in ValidatingAdmissionPolicies. As a result, kubectl apply rejects an invalid resource immediately instead of silently ignoring it or having Gatekeeper reject it later with an unclear parser error.
The module enforces the following restrictions:
deny-invalid-assign-location.deckhouse.io— thespec.locationfield ofAssigncannot target themetadatafield (metadata.name,.namespace,.labels,.annotations). UseAssignMetadatato add labels or annotations.deny-invalid-assignmetadata-location.deckhouse.io— thespec.locationfield ofAssignMetadatamust be exactlymetadata.labels.<KEY>ormetadata.annotations.<KEY>. If<KEY>contains characters other than letters, digits,_, or-(for example, a domain-prefixed key such asapp.kubernetes.io/name), wrap it in quotes:metadata.annotations."app.kubernetes.io/name".deny-invalid-assignmetadata-value.deckhouse.io— thespec.parameters.assign.valuefield ofAssignMetadatamust be a string, because Kubernetes label and annotation values are strings.deny-invalid-mutator-frommetadata-field.deckhouse.io— thespec.parameters.assign.fromMetadata.fieldfield (inAssignandAssignMetadata) is mandatory oncefromMetadatais set, and accepts onlynamespaceorname.deny-invalid-assignmetadata-externaldata-datasource.deckhouse.io— thespec.parameters.assign.externalData.dataSourcefield ofAssignMetadatasupports onlyUsername. This field defaults toValueAtLocationwhen omitted, and that default is rejected too — setdataSource: Usernameexplicitly. UnlikeAssign,AssignMetadatadoesn’t supportValueAtLocation, because a metadata mutation has no source value atlocationto read.deny-mutator-without-match-kinds.deckhouse.io— thespec.match.kindsfield is required and must not be empty inAssign,AssignMetadata,ModifySet, andAssignImageresources, and every entry in it must set non-emptyapiGroupsandkindswithout a*kind. This restriction is specific to Deckhouse: thed8-admission-policy-engine-configMutatingWebhookConfiguration rules are generated only from thematch.kindsfield across all resources of these kinds, resolvingapiGroups/kindsthrough the Kubernetes REST mapper. A resource with an entry that doesn’t resolve to a concrete resource type never triggers a mutation, even though Gatekeeper itself would still treat it as valid.
How to apply a mutation only when a resource is created?
Use the spec.applyTo[].operations field of Assign, ModifySet, or AssignImage
to choose the admission operations that trigger the mutation.
The field accepts CREATE, UPDATE, and *.
If the field is omitted or contains *, the mutation runs on both CREATE and UPDATE.
The following example sets imagePullPolicy only when a pod is created and leaves updates of the pod unchanged:
apiVersion: mutations.gatekeeper.sh/v1
kind: Assign
metadata:
name: always-pull-on-create
spec:
applyTo:
- groups: [""]
kinds: ["Pod"]
versions: ["v1"]
operations: ["CREATE"]
match:
scope: Namespaced
kinds:
- apiGroups: [""]
kinds: ["Pod"]
location: "spec.containers[name:*].imagePullPolicy"
parameters:
assign:
value: Always
The module builds the rules of the d8-admission-policy-engine-config MutatingWebhookConfiguration from these operations.
A resource is sent to Gatekeeper only for the operations that at least one of its mutations requests.
AssignMetadata has no applyTo field and always runs on CREATE and UPDATE.
Verification of image signatures
Available in the following DP editions: SE+, EE, Ultimate.
Cosign versions up to v2 are supported. Versions v3 and above are not supported.
The module implements a function for verifying signatures of container images signed using Cosign. For more details on signing and verifying container images, see the DP documentation.
Signature verification caches the credentials of a private registry read from the dockerCfg field of a SecurityPolicy.
After you change dockerCfg, the new credentials take effect once the cache entry expires.
To set the cache lifetime, use the imageSignatureVerification.registryCredentialsCacheTTLSeconds parameter.
The default is 300 seconds.
How to block deleting a node without a label
DELETE operations are handled by Gatekeeper by default.
You can create your own Gatekeeper policy to block Node deletion unless a special label is present. The example below uses oldObject to check labels on the Node being deleted:
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
name: d8customnodedeleteguard
spec:
crd:
spec:
names:
kind: D8CustomNodeDeleteGuard
validation:
openAPIV3Schema:
type: object
properties:
requiredLabelKey:
type: string
requiredLabelValue:
type: string
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package d8.custom
is_delete { input.review.operation == "DELETE" }
is_node { input.review.kind.kind == "Node" }
has_required_label {
key := input.parameters.requiredLabelKey
val := input.parameters.requiredLabelValue
obj := input.review.oldObject
obj.metadata.labels[key] == val
}
violation[{"msg": msg}] {
is_delete
is_node
not has_required_label
msg := sprintf("Node deletion is blocked. Add label %q=%q to proceed.", [input.parameters.requiredLabelKey, input.parameters.requiredLabelValue])
}
---
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: D8CustomNodeDeleteGuard
metadata:
name: require-node-delete-label
spec:
enforcementAction: warn
match:
kinds:
- apiGroups: [""]
kinds: ["Node"]
parameters:
requiredLabelKey: "admission.deckhouse.io/allow-delete"
requiredLabelValue: "true"
How to deny kubectl exec and kubectl attach to specific Pods?
The admission-policy-engine module webhook routes CONNECT requests for pods/exec and pods/attach in namespaces other than d8-* and kube-* through Gatekeeper. This allows creating custom policies to allow or deny kubectl exec and kubectl attach operations.
The deckhouse module denies the following operations into pods of system components
labeled heritage: deckhouse for all users, but the Deckhouse Platform system users:
kubectl execkubectl attachkubectl cpkubectl port-forwardkubectl debug
The operations are denied regardless of whether the admission-policy-engine module is enabled in the cluster.
For details, refer to the FAQ of the deckhouse module.
Built-in policy for finalizers
To protect objects managed by DP controllers, the admission-policy-engine module includes a built-in ValidatingAdmissionPolicy deny-deckhouse-finalizers.deckhouse.io that forbids removing finalizers containing the deckhouse.io substring on any cluster objects.
This policy doesn’t apply to the following users who are allowed to remove such finalizers:
- Kubernetes system controllers (
system:kube-controller-manager,system:kube-scheduler, etc.); system:sudouser,dhctl,observability;- service accounts from
d8-*namespaces (system:serviceaccount:d8-*); - service accounts from
kube-*namespaces (system:serviceaccount:kube-*).
Custom policy example
You can create your own Gatekeeper policy to deny kubectl exec and kubectl attach operations in specific namespaces. The example below uses input.review.operation and input.review.resource.resource to check for CONNECT operations:
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
name: d8customdenyexec
spec:
crd:
spec:
names:
kind: D8CustomDenyExec
validation:
openAPIV3Schema:
type: object
properties:
forbiddenNamespaces:
type: array
items:
type: string
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package d8.custom
is_connect {
input.review.operation == "CONNECT"
}
# requestSubResource is preferred, but fall back to subResource for older APIs
subresource_is(sub) {
sr := object.get(input.review, "requestSubResource", input.review.subResource)
sr == sub
}
is_exec_or_attach {
input.review.resource.resource == "pods"
subresource_is("exec")
}
is_exec_or_attach {
input.review.resource.resource == "pods"
subresource_is("attach")
}
is_forbidden_namespace {
ns := input.review.namespace
ns == input.parameters.forbiddenNamespaces[_]
}
violation[{"msg": msg}] {
is_connect
is_exec_or_attach
is_forbidden_namespace
msg := sprintf("Exec/attach is forbidden in namespace %q", [input.review.namespace])
}
---
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: D8CustomDenyExec
metadata:
name: deny-exec-in-namespaces
spec:
enforcementAction: deny
match:
kinds:
- apiGroups: ["*"]
kinds: ["*"]
scope: Namespaced
parameters:
forbiddenNamespaces:
- production
- staging
Key data and checks available when validating CONNECT operations:
- Use
input.review.operation == "CONNECT"to check forCONNECToperations. - User information is available in
input.review.userInfo.usernameandinput.review.userInfo.groups. - The namespace is available in
input.review.namespace.
How do I restrict GPU resource usage in namespaces?
The gpuResourceRestriction policy in OperationPolicy
denies the workloads that request GPU resources when the namespace has no label allowing GPU usage.
The check runs on creation and update of a pod, including when an ephemeral container is added,
and on creation and update of the controllers that create pods.
Both resources.requests and resources.limits of every container, init container,
and ephemeral container are inspected. A resource with a quantity of 0 is not treated as a GPU request.
To set up GPU resource restriction:
-
Create an OperationPolicy with the
gpuResourceRestrictionsetting.apiVersion: deckhouse.io/v1alpha1 kind: OperationPolicy metadata: name: gpu-restriction spec: enforcementAction: Deny match: namespaceSelector: labelSelector: matchLabels: operation-policy.deckhouse.io/enabled: "true" policies: gpuResourceRestriction: namespaceLabel: key: "gpu.deckhouse.io/enabled" value: "true" gpuResourcePatterns: - '^nvidia\.com/.*$' - '^amd\.com/gpu$'The
namespaceLabelparameter defines the label key and value that must be present on the namespace. ThegpuResourcePatternsparameter lists the regular expressions matched against GPU resource names. Patterns are not anchored implicitly, so use^and$to match the whole resource name. -
Add the policy label to the namespaces where the policy must be enforced.
d8 k label ns my-namespace operation-policy.deckhouse.io/enabled=true -
Add the GPU label to the namespaces where GPU usage is allowed.
d8 k label ns my-gpu-namespace gpu.deckhouse.io/enabled=true
After that, the pods that request GPU resources are allowed only in the namespaces
with the gpu.deckhouse.io/enabled: "true" label.
The policy is applied to pods and to the pod-creating controllers, so a Deployment whose pod template requests GPU resources is denied on creation. To check this rule on Pods only, add gpuResourceRestriction to the spec.controllerValidation.excludedPolicies parameter of the policy.
The namespace label is read from the Gatekeeper cache. While a namespace is missing from that cache, for example when the namespace and the pod are applied together and the namespace has not been cached yet, the policy denies the pods that request GPU resources in that namespace.
The devices requested through Dynamic Resource Allocation in the spec.resourceClaims field are not extended resources, and the policy does not check them.
What to do if the validating webhook is unavailable?
While the gatekeeper-controller-manager Deployment has no available replicas, the API server rejects every request its webhook intercepts, and the cluster cannot create or delete workloads. The D8AdmissionPolicyEngineWebhookUnavailable alert reports this state. Read why the component is critical for the list of what stops working.
Confirm that the Deployment is the cause and collect the reason:
d8 k -n d8-admission-policy-engine get deploy gatekeeper-controller-manager
d8 k -n d8-admission-policy-engine get pods -l app=gatekeeper,control-plane=controller-manager -o wide
d8 k -n d8-admission-policy-engine describe pods -l app=gatekeeper,control-plane=controller-manager
d8 k -n d8-admission-policy-engine logs deploy/gatekeeper-controller-manager -c manager --all-pods=true --tail=200
The pods of this Deployment are excluded from validation by their gatekeeper.sh/operation: webhook label, so the ReplicaSet creates a replacement pod even while the webhook is down. To get a fresh pod, delete the current one:
d8 k -n d8-admission-policy-engine delete pod -l app=gatekeeper,control-plane=controller-manager
Restarting the Deployment with d8 k rollout restart does not work here. The same label is guarded by the deny-gatekeeper-webhook-operation-label-controllers ValidatingAdmissionPolicy, which rejects a change to a pod template carrying it unless the request comes from a service account of a d8-* or kube-* namespace, or from system:sudouser.
Look for the cause first. Disabling the module unblocks the cluster, but it also removes every policy the cluster relies on, so treat it as a last resort rather than as the first step:
d8 platform module disable admission-policy-engine
Deckhouse Platform keeps the objects of the module removed while it is disabled, so the cluster stays unblocked for as long as the module is off. Return the module as soon as the cause is fixed, since no policy is enforced while it is disabled:
d8 platform module enable admission-policy-engine