Patch release descriptions are generated automatically. To expand a description, click the version number.
Version 1.78
Important
-
The v1 implementation of the container registry management, used up to and including DP 1.77 (the
ProxyandLocalmodes set in theregistrysection of thedeckhouseModuleConfig) has been removed. A cluster can no longer be installed in these modes, and the update to DP 1.78 is blocked while the cluster runsProxyorLocal, orDirectwithout theregistryModuleConfig configured. Migrate the cluster to theregistrymodule on DP 1.77 before updating; the procedures are described in the module FAQ. Update to DP 1.78 only from DP 1.77.1 or later: that is the first release that checks which registry implementation the cluster runs. An update that skips it, for example an LTS release hop or an update constraint from an earlier release, can install DP 1.78 on a cluster still inProxyorLocalmode, which is not supported. -
In Deckhouse Commander, checking, converging and deleting a cluster whose stored resources still set
registry.modetoProxyorLocalin thedeckhouseModuleConfig now stop with an error and change nothing. Before these operations, bring the resources in line with the cluster: replace thedeckhouseModuleConfig in them with the cluster’s own and add the cluster’sregistryModuleConfig, if it has one.
Version 1.77
Important
-
Support for Kubernetes 1.36 has been added, while support for Kubernetes 1.31 has been discontinued. The default Kubernetes version (used when the
kubernetesVersionparameter is set toAutomatic) has been changed to 1.34. -
Control plane components will be restarted during the update.
-
All Ingress NGINX Controller pods will be restarted.
-
If the
enableMultiTenancysetting of theuser-authzmodule is enabled, existing RoleBinding and ClusterRoleBinding objects that were previously ignored because of ClusterAuthorizationRule limitations become effective after the update. Before updating, review the RoleBinding and ClusterRoleBinding objects assigned to users, and modify or remove those that should not grant access. -
If the DP cluster uses custom roles based on the legacy RBACv2 scheme, the
D8UserAuthzLegacyRBACv2CustomRoleFoundalert will fire after the update. Such roles must be migrated to the new scheme before updating to DP 1.78, otherwise the update will be blocked. The migration procedure is described in theuser-authzmodule FAQ. -
If the cluster uses the v1 implementation of the container registry management (the
registrysection of thedeckhouseModuleConfig) in theProxyorLocalmode, or in theDirectmode without theregistryModuleConfig configured, it must be migrated to theregistrymodule before updating to DP 1.78, otherwise the update will be blocked. The migration procedures are described in theregistrymodule FAQ. -
If the
metallbmodule configuration is managed through GitOps, temporarily disable synchronization from your repository before the update. After the update, replace the BGP configuration in your repository from ModuleConfig with the MetalLoadBalancerConfiguration, MetalLoadBalancerBGPPeer, and MetalLoadBalancerPool custom resources, and then re-enable synchronization. -
If the cluster uses NetworkPolicy objects with the
ipBlock: 0.0.0.0/0rule, after the update they will also apply to intra-cluster traffic. It is recommended that you review such policies before the update and adjust them if necessary. -
In the AWS, Azure, and GCP providers, the default CNI has been changed to Cilium. Before creating or updating a cluster, make sure that the Linux kernel version is 5.8 or later and that network policies and firewalls do not block the UDP ports required for Cilium to operate in VXLAN mode. Failure to meet these requirements may result in cluster creation or update failures, or in loss of network connectivity between nodes. For detailed requirements, refer to “Network interaction”.
-
The vSphere, Huawei Cloud, OpenStack, and VCD providers have been migrated from Terraform to OpenTofu. The migration is performed automatically and requires no manual actions.
-
The Yandex Cloud and OpenStack providers have been migrated from Machine Controller Manager (MCM) to Cluster API (CAPI). The management engine is chosen per CloudEphemeral node group. Groups that already exist keep running on MCM, and new groups are created on CAPI by default, so a single cluster can contain groups of both kinds. To move an existing group, refer to “How to migrate node groups to Cluster API (CAPI)”.
-
The following modules are now loaded from external sources and are developed on an independent release cycle (no manual migration actions are required):
-
Monitoring modules:
extended-monitoringlog-shipperlokimonitoring-custommonitoring-deckhousemonitoring-kubernetesmonitoring-pingokmeteroperator-prometheusprometheusprometheus-metrics-adapterprometheus-pushgatewayupmeter
-
The
ingress-nginxmodule.
-
-
The
monitoring-kubernetes-control-planemodule has been merged into thecontrol-plane-managermodule. Monitoring of control plane components is now performed by thecontrol-plane-managermodule.
Deckhouse subsystem
-
A policy has been added to the
deckhousemodule that prevents non-system users from setting or modifying theheritagelabel. This prevents accidental creation of resources that DP may recognize as its own. -
The reliability of updating clusters that use external modules (ModuleSource) has been improved. The update starts only after the modules have been preloaded, preventing their temporary unavailability during the update.
-
Support has been added to the
registrymodule for installing a cluster with the container registry operating inLocalmode. This makes it possible to deploy static clusters without access to an external registry, including in air-gapped environments.
Kubernetes & Scheduling subsystem
-
Changes in the
deschedulermodule:-
Added the
descheduler_pod_evictions_totalmetric with information about the workload to which the evicted pod belongs. -
Added the “Workload Happiness” Grafana dashboard, which displays the workload happiness score, workload distribution across nodes, and the module activity history (evictions, strategy execution, and timeline).
-
-
Changes in the
vertical-pod-autoscalermodule:-
Added the
VerticalPodAutoscalerDeprecatedUpdateModeAutoalert and the “VPA Deprecated Update Mode” Grafana dashboard to detect VerticalPodAutoscaler objects that use the deprecatedupdateMode: Auto mode. -
Improved the reliability of the
vertical-pod-autoscalerwebhook deployment. Fixed a component startup ordering issue that could cause errors during updates.
-
-
Changes in the
control-plane-managermodule:-
The module has been migrated to a new internal implementation for managing control plane components. Added the new ControlPlaneNode and ControlPlaneOperation custom resources to track node state and control plane component update operations.
-
Added automatic etcd defragmentation, which is enabled by default for multi-master clusters and runs when fragmentation exceeds 40%. A defragmentation schedule can be configured in the module settings.
-
Several settings have been moved to the module ModuleConfig. Automatic migration and alerts notifying about the need to update the configuration have also been added:
-
Kubernetes API publishing settings (
publishAPI) from theuser-authnmodule. -
The
encryptionAlgorithmparameter from ClusterConfiguration. -
The
resourcesRequestsparameter for control plane components from theglobalModuleConfig.
-
-
-
Added a patch to
kube-schedulerthat prevents scheduling new pods onto nodes that are undergoing graceful shutdown. -
Accelerated etcd updates in single-master clusters. Eliminated control plane component downtime that occurred during updates.
-
Reduced the risk of
kube-controller-managerthrottling under high controller load. -
Accelerated pod startup by enabling parallel pulling of multiple container images.
-
Enabled the
KubeletEnsureSecretPulledImagesfeature gate by default in the kubelet configuration to comply with CIS Benchmark requirements.
IAM subsystem
-
Changes in the
user-authnmodule:-
Authentication protection in Dex has been strengthened: rate limiting has been added for authentication endpoints, LDAP/Crowd accounts are now automatically blocked after a series of failed sign-in attempts, and external accounts can now be blocked and unblocked manually.
-
The UserOperation resource now supports permanent user locking (
spec.lock.for: permanent). Audit logs for account operations have also been extended: they now include information about the operation initiator, operation type, and target user. -
Added the DexProviderCheck resource for verifying authentication providers (LDAP, OIDC, and others). It allows connection and credential issues to be diagnosed before they affect user sign-in.
-
Added the
Customcomplexity level to the password policy settings, allowing password requirements to be configured flexibly according to corporate standards. -
Added support for specifying custom labels and annotations in the new
spec.podMetadatafield of the DexAuthenticator custom resource, for example, annotations for managing the Istio sidecar container. -
DexAuthenticator now works correctly in namespaces with ResourceQuota and policies that require resource limits for all containers. Fixed an issue that could prevent the authenticator pod from starting.
-
-
Changes in the
user-authzmodule:-
The RBACv2 role model aggregation label scheme changes in DP 1.78. Added the
D8UserAuthzLegacyRBACv2CustomRoleFoundalert, which detects custom roles based on the legacy scheme. Such roles must be migrated to the new scheme before updating to DP 1.78; otherwise, they will stop working and the update will be blocked. The migration procedure is described in the module FAQ. -
If the
enableMultiTenancysetting is enabled, user permissions are now determined by the combination of ClusterAuthorizationRule, AuthorizationRule, and standard RBAC rules (RoleBinding/ClusterRoleBinding). After the update, existing RBAC bindings that were previously ignored because of ClusterAuthorizationRule limitations may become effective. It is recommended to review such bindings before the update. For details, refer to “Using ClusterAuthorizationRule and AuthorizationRule together with RBAC”. -
Added the
D8UserAuthzPermissionBrowserUnavailablealert, which notifies about the unavailability of the user permissions browsing service (permission-browser-apiserver).
-
Security subsystem
-
Changes in the
admission-policy-enginemodule:-
SecurityPolicy exceptions (SecurityPolicyException) can now be configured at the individual container level instead of only for the entire pod. This allows security requirements to be relaxed for a specific sidecar or init container without removing restrictions from the entire pod. For details, see the module documentation.
-
Added the built-in ValidatingAdmissionPolicy
deny-deckhouse-finalizers.deckhouse.io, which prevents manual removal ofdeckhouse.iofinalizers from resources. This prevents incorrect object deletion and the creation of orphan resources.
-
-
Added a mechanism to the
multitenancy-managermodule for granting cluster-wide resources to projects. DP administrators can now define which cluster resources are available to each project and specify default values for them. The mechanism can be used by any DP module to publish its own cluster resources. -
Added the ability to list resources across all accessible namespaces without cluster-wide permissions. The
d8 k get <resource> -A --scope=<accessible|projects|system|project:NAME>command returns only the objects the user has access to instead of returning a403 Forbiddenerror. For details, see “Resource listing with access-based filtering”. -
ACL filtering of the namespace list now supports
watchrequests. Thed8 k get ns -wcommand displays changes in real time without requiring the command to be re-run. -
The cainjector component of the
cert-managermodule is now enabled automatically when resources usingcert-managerCA injection are detected. It no longer needs to be enabled manually or kept running at all times.
Cluster & Infrastructure subsystem
-
Changes in
dhctl(CLI installer):-
Added a preflight check for disk name length in cloud providers. If the name exceeds the allowed limit, the installation is aborted before infrastructure provisioning begins.
-
Improved the interactive mode of
dhctl: added a dedicated area for logs of current operations, and made the output more compact by displaying progress instead of a stream of service messages. Full logs are still available in debug mode and during non-interactive execution. -
Improved the preflight check for the
deckhouseuser. Repeated bootstrap now correctly handles an existing user, and additional checks help prevent permission-related issues. -
Fixed issues with cleaning up temporary processes and SSH connections after running the
convergeanddestroycommands. -
Fixed the
dhctl lock releasecommand in interactive mode. -
Fixed issues in the
dhctl converge-migrationcommand that could cause migration failures in certain scenarios. -
Improved the reliability of control plane
convergeoperations. Checks are now performed in a more appropriate order, and etcd status messages have become more informative.
-
-
Changes in the
node-managermodule:-
Added Instance API v1alpha2. Instance status and error information is now consolidated into a single resource, simplifying diagnostics.
-
Reduced the wait time for MachineDeployment creation for AWS Spot instances. If AWS cannot allocate Spot instances because of insufficient capacity, retries begin sooner.
-
Added the
spec.kubelet.seccompDefaultparameter to the NodeGroup resource, allowing the RuntimeDefault seccomp profile to be enabled by default for all workloads without explicit seccomp settings.
-
-
Reduced the probability of false restarts of the CSI Controller in all cloud provider modules during short delays by increasing the liveness probe timeout.
-
Changes in the
cloud-provider-dvpmodule:-
Added support for live disk migration between StorageClass objects without stopping the virtual machine. Migration is performed automatically when the StorageClass changes.
-
The module configuration has been migrated to the unified ModuleConfig contract. Permanent and Ephemeral nodes are now configured in the same way through InstanceClass and NodeGroup, without a separate ProviderClusterConfiguration. This simplifies cluster configuration, enables schema versioning and conversion without breaking compatibility, and prepares cloud providers for an independent release cycle outside the DP core. For details, refer to “Cluster and Infrastructure” in the DP FAQ.
-
Fixed an issue that could leave LoadBalancer Services in the
Pendingstate without creating the parent load balancer. Improved conflict handling during ServiceWithHealthchecks updates and load balancer status synchronization. -
StorageClass objects created through Cluster API clusters now always use the
WaitForFirstConsumermode regardless of the parent cluster configuration. During updates, managed StorageClass objects with an incompatiblevolumeBindingModeare also recreated automatically. This eliminates an issue that could block cluster updates and synchronization because of the immutablevolumeBindingModefield.
-
-
The
cloud-provider-openstack,cloud-provider-huaweicloud,cloud-provider-vsphere, andcloud-provider-vcdmodules now use OpenTofu instead of Terraform. This makes it possible to use an open infrastructure platform compatible with Terraform, reduces dependency on Terraform licensing changes, and aligns these providers with other cloud provider modules that already use OpenTofu. -
The OpenStack and Yandex Cloud providers now use Cluster API, the industry-standard framework for machine management in Kubernetes. To ensure a smooth transition, the legacy mode based on Machine Controller Manager (MCM) remains supported. Existing node groups keep running on MCM, while new ones are created on Cluster API.
-
Improved virtual machine deletion handling in the
cloud-provider-huaweicloudmodule, increasing resource cleanup reliability and preventing false deletion errors. Virtual machines that remain in the deleting state for more than 24 hours are now marked with the corresponding status, simplifying diagnostics. -
Updated the CSI plugin and CSI Syncer versions in the
cloud-provider-vspheremodule, and fixed known vulnerabilities in Cloud Controller Manager, the CSI plugin, and Terraform Manager. This improves integration security and ensures the use of up-to-date component versions. -
Added support for the
vcd.cpi.flant.com/load-balancer-ipannotation for LoadBalancer Services in thecloud-provider-vcdmodule. It allows assigning a predefined external IP address to the load balancer. If both the annotation and thespec.loadBalancerIPfield are specified, the annotation takes precedence.
Network subsystem
-
Changes in the
istiomodule:-
Added support for Istio 1.27. Users now have access to the new features of the latest Istio release using an operatorless mode.
-
Expanded ambient mesh support. New settings provide more explicit control over ambient mesh, allow using waypoint proxies for Layer 7 traffic processing, and enable the HBONE protocol for encrypting traffic between pods without requiring a sidecar container to run in every pod.
-
-
Changes in the
cni-ciliummodule:-
Changed the behavior of NetworkPolicy objects with the
ipBlock: 0.0.0.0/0rule in the new module version. The rule now also applies to intra-cluster traffic. -
Reduced CPU consumption by Cilium agents. The freed resources become available to user workloads.
-
-
Changes in the
ingress-nginxmodule:-
Added support for IngressNginxController v1.15.
-
Updated the load calculation formula for Ingress NGINX Controller pods used for horizontal scaling of LoadBalancer, LoadBalancerWithProxyProtocol, and LoadBalancerWithSSLPassthrough inlets. The controller now adds and removes replicas in a timely manner, resulting in more stable behavior under load.
-
Updated the IngressNginxController API to v2. The
spec.resourcesRequestsresource consumption settings have been replaced with the extendedspec.resourcesManagementsettings, including support for resource limits. Compatibility with the v1 API is preserved automatically. -
Implemented isolated configuration validation, making it possible to restore full NGINX configuration validation without reducing the security level.
-
-
Added authorization for the Hubble web interface in the
cilium-hubblemodule. Access to cluster network telemetry can now be restricted by user groups to improve security. -
In the
metallbmodule, BGP mode configuration has been moved from ModuleConfig to the dedicated MetalLoadBalancerPool, MetalLoadBalancerBGPPeer, and MetalLoadBalancerConfiguration custom resources. The existing configuration will be migrated automatically, so no manual changes in the cluster are required. All new BGP settings must be configured through these resources.
The complete list of changes, including the updated components, is available in the changelog on GitHub.
1.77.8
Know before update
- Dex pods restart. A client of Dex that expects HTTP 429 on the refresh of a locked user, HTTP 500 for a deleted one, or
invalid_requestfor a claimed or expired refresh token gets HTTP 400invalid_grant. The password and token exchange grants and the sign-in page answer as before. - Existing clusters are unaffected; the default zone set (ru-central1-a, ru-central1-b, ru-central1-d, ru-central1-e) is statically pinned. To use ru-central1-f, list it in
zonesinstead of one of the default zones; the dropped zone’s subnet is replaced by a new subnet in ru-central1-f and the nodes in it are recreated. - containerd restarts on all nodes. On nodes where the old AppArmor profile is already loaded, the fix applies after a node reboot.
Features
- [cloud-provider-yandex] Add ru-central1-f zone support via explicit zone selection with a four-zone limit.
- [cloud-provider-yandex] Add ru-central1-f zone support via explicit zone selection with a four-zone limit.
Existing clusters are unaffected; the default zone set (ru-central1-a, ru-central1-b, ru-central1-d, ru-central1-e) is statically pinned. To use ru-central1-f, list it inzonesinstead of one of the default zones; the dropped zone’s subnet is replaced by a new subnet in ru-central1-f and the nodes in it are recreated.
Fixes
- [admission-policy-engine] Pods with a Localhost AppArmor profile set in
securityContextare no longer denied by the Baseline policy. - [ci] Pass the werf meta repo to the dev site and documentation deploy jobs, matching the builds.
- [common] Updated
golang.org/x/net,golang.org/x/text,google.golang.org/grpcandgithub.com/klauspost/compressin the Kubernetes 1.31 CSI sidecars of CSE to fix CVEs. - [control-plane-manager] Fixed ControlPlaneOperation failures on clusters whose root CAs have a non-default subject.
- [dhctl] The bootstrap terminal no longer shows glued lines, FAILED for bashible step attempts that recovered, or “Unhandled Error” from client-go.
- [docs] Fix highlighting the L2 menu on module pages in the documentation.
- [registrypackages] Fix AppArmor denying signals between processes of one container on Ubuntu kernels 7.0.0-41 and later.
containerd restarts on all nodes. On nodes where the old AppArmor profile is already loaded, the fix applies after a node reboot. - [user-authn] Dex answers the refresh of a locked user, of a user deleted from the local store or LDAP, and of a claimed or expired refresh token with
invalid_grant(HTTP 400), so that applications and a Dex that signs in through this one send the user to sign in.
Dex pods restart. A client of Dex that expects HTTP 429 on the refresh of a locked user, HTTP 500 for a deleted one, orinvalid_requestfor a claimed or expired refresh token gets HTTP 400invalid_grant. The password and token exchange grants and the sign-in page answer as before.
1.77.7
Know before update
- A cluster that uses
ProxyorLocalof the previous registry implementation, orDirectwithout theregistryModuleConfig, is not updated to DP 1.78, as before. This now also applies from the moment such a mode is set in thedeckhouseModuleConfig, before the switch starts. Migrate as described in “Migrating from the previous implementation to the current one” in the registry documentation. Clusters already prepared for the migration are not affected. - A node that joins while a registry-packages-proxy pod of the previous release is still running on a master waits until a pod of this release answers.
- Cilium agent pods will be restarted.
- Flannel pods will be restarted.
- On a NodeGroup with
spec.gpu, a failing GPU check used to abort the bashible run and be retried forever, so the node never configured kubelet and never received updates from any other module. Such a node now converges and only its GPU stays unusable. If/usr/bin/nvidia-container-runtimeis absent, containerd is now configured withruncas the default runtime instead ofnvidia; on a node whose configuration previously selectednvidiathis changes the containerd config and therefore restarts containerd. Previously such a node could not start any container at all. - Pods of the kube-proxy, cni-cilium, cni-flannel, and cni-simple-bridge agent DaemonSets will be restarted because their cni-migration-init-checker init container image is updated.
- The controller and agent pods will be restarted.
- The registry in-cluster proxy and node services pods will be restarted.
- The terraform-manager pods will be restarted.
- istiod, istio-cni, ingress gateways, Kiali, api-proxy and metadata-exporter pods will be restarted.
- node-controller, capi-controller-manager, machine-controller-manager and bashible-apiserver do not run on a master tainted with a key from
global.modules.placement.customTolerationKeys. Remove such taints from the master NodeGroup before the update. - simple-bridge pods will be restarted.
Features
- [docs] Implement glossary with term linking.
- [docs] Added the “System requirements” page with deployment options and resource requirements for containers, virtual machines and combined workloads.
- [node-manager] Alert when a NodeGroup uses the deprecated built-in GPU support while the
gpumodule is disabled.
Fixes
- [candi] A GPU problem no longer blocks node convergence.
On a NodeGroup withspec.gpu, a failing GPU check used to abort the bashible run and be retried forever, so the node never configured kubelet and never received updates from any other module. Such a node now converges and only its GPU stays unusable. If/usr/bin/nvidia-container-runtimeis absent, containerd is now configured withruncas the default runtime instead ofnvidia; on a node whose configuration previously selectednvidiathis changes the containerd config and therefore restarts containerd. Previously such a node could not start any container at all. - [candi] GPU compute capability check no longer requires
bc. - [candi] Persist
net.core.bpf_jit_hardenfor GPU nodes in an own sysctl drop-in instead of editing distribution files that may be absent. - [candi] bashible verifies the rpp-get binary against the image digest from its bundle before running it; a substituted bootstrap proxy on a master can no longer hand a binary to the nodes.
A node that joins while a registry-packages-proxy pod of the previous release is still running on a master waits until a pod of this release answers. - [candi] Fixed an intermittent bootstrap hang where kube-apiserver never started because kubelet read its manifest while it was being written.
- [candi] A new cloud node sets up the provider network before it installs bootstrap packages, so it can download them from a master in another zone.
- [chrony] Fixed known vulnerabilities in chrony-exporter.
- [cloud-provider-dvp] Fixes DVP CSI staging so volumes with filesystems detected by Kubernetes SafeFormatAndMount are preserved and unformatted volumes are formatted without using lsblk.
- [cloud-provider-dvp] Creates ModuleConfig/cloud-provider-dvp automatically when it is missing.
- [cni-cilium] The ipv4-options-guard sidecar no longer writes to the host
/etc, so Cilium agents start on Deckhouse Engine nodes.
Cilium agent pods will be restarted. - [cni-flannel] The ipv4-options-guard sidecar no longer writes to the host
/etc, so Flannel pods start on Deckhouse Engine nodes.
Flannel pods will be restarted. - [cni-simple-bridge] The ipv4-options-guard sidecar no longer writes to the host
/etc, so simple-bridge pods start on Deckhouse Engine nodes.
simple-bridge pods will be restarted. - [common] Fixed known vulnerabilities in the cni-migration-controller and cni-migration-init-checker images.
Pods of the kube-proxy, cni-cilium, cni-flannel, and cni-simple-bridge agent DaemonSets will be restarted because their cni-migration-init-checker init container image is updated. - [common] added mountpoint in ipv4-options-guard
- [common] Updated
golang.org/x/net,golang.org/x/text,google.golang.org/grpcandgithub.com/klauspost/compressin CSI sidecars to fix CVEs. - [control-plane-manager] Preserve custom TLS certificates when enabling a default Gateway.
- [control-plane-manager] The migration hook for the api listener is added.
- [deckhouse] Fixed a memory leak in webhook-handler, which kept the kubernetes monitors of replaced hooks running after every hook reload.
- [deckhouse-controller] Fixed image pulls for modules served by ModulePullOverride when another source updates its registry or the override is not ready during Deckhouse startup.
- [dhctl] Fixed known vulnerabilities in dhctl dependencies.
- [docs] Move the language switch from the header to the footer
- [documentation] Add the language switch to the footer.
- [istio] Fixed CVEs in the istio module images (go-jose, golang.org/x/net).
istiod, istio-cni, ingress gateways, Kiali, api-proxy and metadata-exporter pods will be restarted. - [local-path-provisioner] Updated
golang.org/x/net,golang.org/x/textandgithub.com/klauspost/compressto fix CVEs. - [metallb] Fixed BGP migration to preserve pool allocation flags and allow users to manage migrated resources after switching ModuleConfig to version 3.
- [node-manager] Master controllers leave an unreachable master after about 100 seconds and are not evicted from a single master.
- [node-manager] Master controllers no longer tolerate keys from
customTolerationKeys, so a key that repeats a built-in one no longer blocks pod updates.
node-controller, capi-controller-manager, machine-controller-manager and bashible-apiserver do not run on a master tainted with a key fromglobal.modules.placement.customTolerationKeys. Remove such taints from the master NodeGroup before the update. - [node-manager] A node-manager release no longer waits for capi-controller-manager when neither capi nor its webhooks change.
- [node-manager] Fixed known vulnerabilities in the bashible-apiserver, node-controller, node-group-exporter, kubernetes-api-proxy, d8-shutdown-inhibitor and capi-controller-manager images.
- [node-manager] Fixed constant NodeGroup status updates by node-controller on every Node update.
- [registry] Fixed known vulnerabilities in the docker-auth image.
The registry in-cluster proxy and node services pods will be restarted. - [registry] Fixed known vulnerabilities in the nodeservices-manager, docker-auth, mirrorer and syncer images.
- [registry] The update to DP 1.78 is held as soon as
registry.modeProxyorLocal, or a switch betweenDirectandUnmanaged, is set in thedeckhouseModuleConfig, and the message of the held release lists the migration steps for each mode.
A cluster that usesProxyorLocalof the previous registry implementation, orDirectwithout theregistryModuleConfig, is not updated to DP 1.78, as before. This now also applies from the moment such a mode is set in thedeckhouseModuleConfig, before the switch starts. Migrate as described in “Migrating from the previous implementation to the current one” in the registry documentation. Clusters already prepared for the migration are not affected. - [registry-packages-proxy] The bootstrap port serves the manifest and the layer of the rpp-get image, so a node can verify the binary it installs.
- [registry-packages-proxy] Fixed known vulnerabilities in registry-packages-proxy.
- [registrypackages] Fixed known vulnerabilities in the cfssl and d8-bcheck packages.
- [registrypackages] Bump golang.org/x/net to v0.62.0 in d8-bcheck to fix CVE-2026-78661 and CVE-2026-97045.
- [registrypackages] Fixed known vulnerabilities in the rpp-get package.
- [registrypackages] Fixed known vulnerabilities in auth_server and registry of the docker-registry package.
- [registrypackages] rpp-get tries every registry-packages-proxy endpoint before an attempt fails and retries connection timeouts, so a node that reaches only the master of its own zone during bootstrap joins the cluster.
- [service-with-healthchecks] Go dependencies are updated.
The controller and agent pods will be restarted. - [terraform-manager] Fixed known vulnerabilities in opentofu.
The terraform-manager pods will be restarted. - [terraform-manager] Fixed known vulnerabilities in opentofu and in the terraform-manager-dvp image.
- [tools] The e2e directories of modules no longer reach the Deckhouse image.
- [user-authn] Default DKP gateway requirement is removed from dex-authenticator template.
1.77.6
Know before update
- Applying a discovered address restarts the Deckhouse Pod once.
- Cilium agent pods will be restarted. With DSR, the DSR IPv4 option is stripped before the packet reaches the pod.
- Control plane components (kube-apiserver, kube-controller-manager, kube-scheduler, etcd) and kubelet will be restarted.
- Flannel pods will be restarted.
- In ambient mode with Istio 1.27 or later, on nodes with legacy iptables,
istio-cni-nodeswitches from nftables to legacy iptables. Pods that were added to the mesh before the update keep their nftables rules. Restart these pods after the update. Otherwise, removing such a pod from the mesh without a restart leaves its old redirect rules in place, and the pod loses network connectivity. - The controller and agent pods will be restared.
- The metallb components (controller, speaker, l2lb) will restart after the update.
- containerd will be restarted on all nodes.
- kube-proxy pods will be restarted.
- simple-bridge pods will be restarted.
Features
- [cni-cilium] Added a CVE-2026-98432 mitigation that filters IPv4 TCP SYN packets with IP options in the host and pod network namespaces (
ipv4OptionsGuard;Enforcefor new clusters,Auditfor existing ones).
Cilium agent pods will be restarted. With DSR, the DSR IPv4 option is stripped before the packet reaches the pod. - [cni-flannel] Added a CVE-2026-98432 mitigation that filters IPv4 TCP SYN packets with IP options in the host and pod network namespaces (
ipv4OptionsGuard;Enforcefor new clusters,Auditfor existing ones).
Flannel pods will be restarted. - [cni-simple-bridge] Added a CVE-2026-98432 mitigation that filters IPv4 TCP SYN packets with IP options in the host and pod network namespaces (
ipv4OptionsGuard;Enforcefor new clusters,Auditfor existing ones).
simple-bridge pods will be restarted. - [docs] Add static-routing-manager architecture module section.
- [docs] Add cni-flannel and cni-simple-bridge modules architecture section.
Fixes
- [admission-policy-engine] Fixed CVE-2026-116585, CVE-2026-116586 and CVE-2026-116587 in
github.com/go-jose/go-jose/v4and CVE-2026-78661 and CVE-2026-97045 ingolang.org/x/net. - [admission-policy-engine] Switch ratify to go-jose v4 to fix CVE-2026-116585, CVE-2026-116586, CVE-2026-116587.
- [admission-policy-engine] An unavailable Gatekeeper no longer blocks system components in namespaces that opted into security policy checks, which could deadlock the cluster together with the CNI agent.
- [cert-manager] Bump x/net to v0.62.0 and go-jose to v4.1.6 to fix CVE-2026-78661, CVE-2026-97045, CVE-2026-116585, CVE-2026-116586, CVE-2026-116587.
- [ci] Svace stages are rebuilt in every svace pipeline.
- [cloud-provider-dvp] Fixes x/net, x/text and kin-openapi CVEs in all module images.
- [cloud-provider-openstack] Fixes creation of nodes with
rootDiskSizewhen the node availability zone does not exist in Cinder. - [cloud-provider-openstack] fix CloudEphemeral node creation in non-default availability zones via automatic Cinder-AZ discovery and new OpenStackInstanceClass.volumeTypeMap field
- [cni-cilium] Fixed file descriptor, connection and lock leaks in Cilium found by static analysis.
- [common] Bump x/net in kube-rbac-proxy to v0.62.0 to fix CVE-2026-78661 and CVE-2026-97045.
- [control-plane-manager] The basic audit policy records an exec into a pod (
pods/exec) in all namespaces again, as it did before Deckhouse 1.77. - [control-plane-manager] Fixed a goroutine leak in the helm release deprecation scan when listing releases fails.
- [control-plane-manager] Updated go-jose, x/net and klauspost/compress in control plane components to fix known vulnerabilities.
Control plane components (kube-apiserver, kube-controller-manager, kube-scheduler, etcd) and kubelet will be restarted. - [deckhouse] Resolve
registry.d8-system.svcto a node running the in-cluster registry proxy during bootstrap, instead of always to127.0.0.1.
Applying a discovered address restarts the Deckhouse Pod once. - [deckhouse] Update runtime dependencies and document unaffected vulnerability paths.
- [deckhouse] The
d8-mainTelemetry no longer references thedeckhouse-tracingprovider when the module doesn’t register it. - [deckhouse-controller] Prevent module hooks from running with unavailable snapshots while waiting for Deckhouse to restart after a module update
- [deckhouse-controller] A pending module release no longer loses its Kubernetes, Deckhouse and module requirements, so an update that the cluster does not satisfy stays pending instead of disabling the module.
- [dhctl] Keep the SSH host list and master readiness checks in sync with the infrastructure state during master converge.
- [dhctl] Bump go-jose to v4.1.6 to fix CVE-2026-116585, CVE-2026-116586, CVE-2026-116587.
- [docs] Fix docs build config errors in werf includes.
- [documentation] Fix HTML parser vulnerabilities in docs-builder by updating golang.org/x/net to v0.62.0.
- [go_lib] Package proxy streams the full package body when the cache skips writing it.
- [istio] Fixed ambient mode on nodes with legacy iptables, where
istio-cni-nodecrashed on Istio 1.25 and used nftables instead of legacy iptables on Istio 1.27 and later.
In ambient mode with Istio 1.27 or later, on nodes with legacy iptables,istio-cni-nodeswitches from nftables to legacy iptables. Pods that were added to the mesh before the update keep their nftables rules. Restart these pods after the update. Otherwise, removing such a pod from the mesh without a restart leaves its old redirect rules in place, and the pod loses network connectivity. - [kube-proxy] Fixed known vulnerabilities in the kube-proxy images.
kube-proxy pods will be restarted. - [local-path-provisioner]
config.jsonof the provisioner is serialized from a structure, and the CRD accepts inspec.pathonly an absolute path without quotes, backslashes, whitespace and control characters. A crafted path can no longer add manifests to the module release. - [metallb] Fixed CVE-2026-78661 and CVE-2026-97045 in the metallb and l2lb images.
The metallb components (controller, speaker, l2lb) will restart after the update. - [multitenancy-manager] Fixed CVE-2026-116585, CVE-2026-116586 and CVE-2026-116587 in
github.com/go-jose/go-jose/v4and CVE-2026-78661 and CVE-2026-97045 ingolang.org/x/net. - [node-manager] kube-rbac-proxy in capi-controller-manager and node-controller no longer listens on all addresses of a node that has no addresses yet, so it does not take the controller’s port.
- [node-manager] Fixed known vulnerabilities in the integrity-controller and integrity-containerd-configurator images (CSE).
- [node-manager] Fixes x/net CVEs in the caps-controller-manager image.
- [registry] Require brackets around IPv6 upstream registry URL hosts and accept bracketed addresses with or without a port.
- [registry-packages-proxy] Fixed registry-packages-proxy hanging after a cache error and serving corrupted packages, which blocked node updates.
- [registry-packages-proxy] Bump go-jose to v4.1.6 to fix CVE-2026-116585, CVE-2026-116586, CVE-2026-116587.
- [registrypackages] Updated go-jose, x/net and klauspost/compress in containerd, runc and crictl to fix known vulnerabilities.
containerd will be restarted on all nodes. - [registrypackages] Support rebuilding legacy image layers and preserve verity data in CSE.
- [service-with-healthchecks] Status updates were refactored.
The controller and agent pods will be restared. - [user-authn] Local users keep the passwords they changed in Dex when the cluster updates from 1.76.
- [user-authn] Fixed CVE-2026-116585, CVE-2026-116586 and CVE-2026-116587 in
github.com/go-jose/go-jose/v4and CVE-2026-78661 and CVE-2026-97045 ingolang.org/x/net. - [user-authn] Bump go-oidc in dex-authenticator to v3.11.0 to fix CVE-2026-116585, CVE-2026-116586, CVE-2026-116587.
- [user-authz] Fixed CVE-2026-78661 and CVE-2026-97045 in
golang.org/x/net. - [vertical-pod-autoscaler] Fix a panic in vpa-updater and vpa-admission-controller when a pod with a container without resources is capped to a LimitRange of type Pod.
1.77.5
Know before update
- After a reset, the previous password cannot be reused when passwordHistoryLimit is set; the current password cannot be set again through the change form, and a lowered passwordHistoryLimit applies at the next password change.
- Cilium agent and operator pods will be restarted.
- Control plane components and kubelet will be restarted.
- CoreDNS pods will be restarted.
- Node-local-dns safe updater will be restarted.
- TLS connections from nodes to hosts whose certificates chain to
emSign ECC Root CA - G3or ComodoAAA Certificate Services(eMudhra / emSign issued) will fail. Reissue such certificates from another CA before the update. - The
cert-managercontroller pod restarts. - The controller and agent pods will be restarted.
- cni-flannel pods will be restarted.
- kube-proxy pods will be restarted.
- network-policy-engine pods will be restarted.
- node-local-dns pods will be restarted.
Features
- [control-plane-manager] The basic audit policy logs Pod status updates from kubelet with the request body, so container terminations and their reasons get into the audit log.
- [deckhouse] Migrated package release management to nelm v2 while keeping the v1 install and delete behaviour.
- [user-authn] DexAuthenticator logins, logouts, failed and denied logins, CSRF mismatches and failed session refreshes are delivered as security events when the
security-events-managermodule is enabled.
Fixes
- [admission-policy-engine] Fixed CVE-2026-114437 in
github.com/go-chi/chi/v5and CVE-2026-63209 ingithub.com/klauspost/compress. - [candi] The first master bootstrap waits for
kube-apiserverreadiness instead of failing the control-plane step onconnection refused. - [cert-manager] The metrics proxy of the controller is no longer restarted by its liveness probe while its node is short of CPU.
Thecert-managercontroller pod restarts. - [cilium-hubble] Fixed a known vulnerability in the Hubble relay image.
- [cloud-provider-dvp] Fixed deletion of load balancers backed by ServiceWithHealthchecks, which failed because the child Service is protected by the
heritage - deckhouselabel in the parent cluster. - [cloud-provider-huaweicloud] Recreated the etcd disk when a master VM is recreated.
- [cloud-provider-openstack] CAPI now mirrors MCM —
configDrivehonorsexternalNetworkDHCP, and additional networks always produce ports - [cloud-provider-vcd] Fixes the module getting stuck in Error on upgrade from 1.76 because of the immutable selector of the
capcd-controller-managerDeployment. - [cni-cilium] Fixed a known vulnerability in the Cilium images.
Cilium agent and operator pods will be restarted. - [cni-flannel] Fixed known vulnerabilities in the flanneld image.
cni-flannel pods will be restarted. - [control-plane-manager] Updated Go dependencies of Kubernetes control plane components to fix known vulnerabilities.
Control plane components and kubelet will be restarted. - [control-plane-manager] The cluster-wide controllers of
control-plane-managerrun in a single pod regardless of thehighAvailabilitysetting or an etcd arbiter. - [control-plane-manager] Control plane components are no longer restarted in parallel when a node is NotReady or after a certificate renewal.
- [control-plane-manager] Fixed a Deckhouse crash caused by a helm release
SecretorConfigMapwith too short release data. - [control-plane-manager] removed unused executable files from the etcd-backup image
- [control-plane-manager] In CSE,
availableVersionsind8-cluster-kuberneteslists only Kubernetes versions that can be set. - [deckhouse] Restore ClusterAdmin read access to ModuleSettingsDefinition so the console can show module settings again.
- [deckhouse] Show the Terminating phase while package repositories are being deleted
- [deckhouse] Update d8 CLI and scoped runtime VEX exceptions for known vulnerabilities.
- [deckhouse-controller] A module enabled in the bundle is no longer disabled when its ModuleConfig does not set
enabled. - [deckhouse-controller] Module sources now scan the registry every 30 minutes by default, reducing registry load
- [istio] added limit paralle in common images
- [istio] Fixed CVE-2026-63209 in istio images.
- [istio] Fixed istio module installation with nelm v2 when Istio is managed by the istio operator (versions below 1.27).
- [istio] Fixed
alliance-healthcheckkeeping the sidecar of the previous Istio version after aglobalVersionchange. - [kube-dns] Fixed a known vulnerability in the CoreDNS image.
CoreDNS pods will be restarted. - [kube-proxy] Fixed known vulnerabilities in the kube-proxy init-container image.
kube-proxy pods will be restarted. - [kube-proxy] Fixed known vulnerabilities in the kube-proxy images.
kube-proxy pods will be restarted. - [multitenancy-manager] Fixed CVE-2026-63209 in
github.com/klauspost/compress. - [network-policy-engine] Fixed known vulnerabilities in the kube-router image.
network-policy-engine pods will be restarted. - [node-local-dns] Fixed DNS errors during node-local-dns pod restarts in clusters with Cilium.
- [node-local-dns] Fixed a known vulnerability in the node-local-dns CoreDNS image.
node-local-dns pods will be restarted. - [node-local-dns] Node-local-dns safe updater logic was updated to match the Kubernetes logic.
Node-local-dns safe updater will be restarted. - [openvpn] Fixed known vulnerabilities in the ovpn-admin and easyrsa-migrator images.
- [registry] In the
Proxymode, reuse upstream auth challenges and tokens instead of requesting them before every request to the upstream registry. - [registry] Fix the registry static pod stuck in
ImagePullBackOffafter a registry image update on a single master in theProxyandLocalmodes. - [registrypackages] Nodes stop trusting
emSign ECC Root CA - G3and ComodoAAA Certificate Services, the roots behind the compromised eMudhra intermediate CAs.
TLS connections from nodes to hosts whose certificates chain toemSign ECC Root CA - G3or ComodoAAA Certificate Services(eMudhra / emSign issued) will fail. Reissue such certificates from another CA before the update. - [registrypackages] Update containerdv2 patch
- [service-with-healthchecks] CVEs were fixed.
The controller and agent pods will be restarted. - [user-authn] dex-authenticator no longer writes the request headers, cookies included, to the log when the CSRF cookie is missing.
- [user-authn] A password reset of a local user keeps the replaced password in the password history, and the password change form refuses the current password in every case.
After a reset, the previous password cannot be reused when passwordHistoryLimit is set; the current password cannot be set again through the change form, and a lowered passwordHistoryLimit applies at the next password change.
1.77.4
Know before update
- Cilium agent and operator pods will be restarted.
- CoreDNS pods will be restarted.
- NodeGroups that already have preemptible: true under the CAPI engine keep their current regular VMs until the next rollout (any rollout-triggering InstanceClass change or manual-rollout-id); after that rollout their nodes become preemptible VMs and are rotated every ~20h.
- Upgrade does not recreate any node. NodeGroups that already have preemptible: true under the CAPI engine keep their current regular VMs; Machines created after the upgrade (scale-up, replacement) are preemptible, so such NodeGroups run mixed VMs until a full rollout (any rollout-triggering InstanceClass change or the manual-rollout-id annotation). Preemptible VMs are rotated every ~20h.
- metallb pods will be restarted.
- node-local-dns pods will be restarted.
Features
- [control-plane-manager] The
etcddashboard is reworked to find the cause of slowetcd- disk, network, leader,kube-apiserverload or page cache.
Fixes
- [chrony] Bump golang.org/x/net in chrony_exporter to fix CVE-2026-78669.
- [cilium-hubble] Fixed known vulnerabilities in the Hubble relay and UI images.
- [cloud-provider-aws] Allow bootstrapping clusters in AWS regions unknown to the bundled terraform-provider-aws, such as
ap-southeast-6. - [cloud-provider-dvp] A DVP cluster migrated to the new provider configuration format keeps its
networkPolicy - Noneinstead of silently switching toIsolated. - [cloud-provider-dvp] Clusters with
networkPolicy - Nonein the provider configuration no longer get NetworkPolicies created for LoadBalancer Services. - [cloud-provider-yandex] Preemptible nodes are created as preemptible VMs and rotated before the 24h limit when a NodeGroup is managed by the CAPI engine.
Upgrade does not recreate any node. NodeGroups that already have preemptible: true under the CAPI engine keep their current regular VMs; Machines created after the upgrade (scale-up, replacement) are preemptible, so such NodeGroups run mixed VMs until a full rollout (any rollout-triggering InstanceClass change or the manual-rollout-id annotation). Preemptible VMs are rotated every ~20h. - [cloud-provider-yandex] Preemptible nodes are created as preemptible VMs and rotated before the 24h limit when a NodeGroup is managed by the CAPI engine.
NodeGroups that already have preemptible: true under the CAPI engine keep their current regular VMs until the next rollout (any rollout-triggering InstanceClass change or manual-rollout-id); after that rollout their nodes become preemptible VMs and are rotated every ~20h. - [cni-cilium] Fixed known vulnerabilities in the Cilium images.
Cilium agent and operator pods will be restarted. - [common] Fixed known vulnerabilities in the CNI migration images.
- [deckhouse] Bump containerd to v2.2.10 and dependencies to fix CVEs
- [dhctl] Bump golang.org/x/net to fix CVE-2026-78669.
- [docs] Update terminology in the documentation and SEO improvements.
- [documentation] Update documentation building rules.
- [istio] Added VEX for CVE-2026-85747 in pilot and proxyv2 images.
- [kube-dns] Fixed known vulnerabilities in the CoreDNS and sts-pods-hosts-appender images.
CoreDNS pods will be restarted. - [metallb] Fixed known vulnerabilities in the MetalLB images.
metallb pods will be restarted. - [node-local-dns] Fixed known vulnerabilities in the node-local-dns images.
node-local-dns pods will be restarted. - [node-manager] Updates otel, grpc and x/crypto dependencies of node-controller to fix CVEs.
- [node-manager] fixed private build
- [node-manager] Bump golang.org/x/net to fix CVE-2026-78669, CVE-2026-78663, CVE-2026-97032.
- [registry] Bump golang.org/x/net in nodeservices-manager to fix CVE-2026-78669.
- [registry-packages-proxy] Bump golang.org/x/net to fix CVE-2026-78669.
- [registrypackages] Fixed known vulnerabilities in the kubernetes-cni package.
- [registrypackages] Bump golang.org/x/net, golang.org/x/crypto and OpenTelemetry in rpp-get and the docker-registry auth_server to fix CVE-2026-78669, CVE-2026-56855, CVE-2026-81870.
- [terraform-manager] Bump golang.org/x/net in opentofu to fix CVE-2026-78669.
- [vertical-pod-autoscaler] The admission-controller webhook now serves HTTP/1.1 only and no longer exposes
1.77.3
Know before update
- Before, such an entry stopped the webhook from reloading the rules, so subjects of rules added later, and every subject after a restart of the webhook, were not limited to their namespaces. Subjects of a rule whose only entry is invalid now get no namespace through the entries of that rule.
- Bumping
deckhouse_lib_helmfrom1.72.11to1.72.24also drops thedrbd.linbit.comtaint tolerations fromhelm_lib_tolerations, so pods rendered through it no longer toleratelost-quorum,force-io-errorandignore-fail-over, and aligns the VPAmaxAllowedmemory to the recommender’s 64Mi quantum. - Exec and attach requests in user namespaces reach Gatekeeper again, as in Deckhouse 1.73 and 1.74.
Every constraint that matches them is evaluated again, including a constraint without
match.kindsor withkinds: ["*"]whose policy does not checkinput.review.operation. Before updating, check that such constraints do not denykubectl execandkubectl attachfor users who need them. - In a cluster with the
proxyparameter ofClusterConfigurationset, thecert-managercontroller sends requests to the ACME server, HashiCorp Vault and the DNS provider APIs through the proxy server. TheHTTP-01self-check also requests the domain through the proxy server. If the proxy server cannot reach the issuer or the public address of the cluster, add these addresses tonoProxy. - In single-replica installations a new controller pod reconciles projects only after the previous pod has stopped and released the lease. If the previous pod is lost without a shutdown, for example together with its node, the new pod starts reconciling projects 60 to about 78 seconds after it starts.
- Machines are not recreated on update. Machines created before the update keep the standard network type and no labels until the NodeGroup is rolled out (change a rolling InstanceClass field or set the
manual-rollout-idannotation on the NodeGroup). Machines stuck in provisioning because ofadditionalSubnetsbecome ready without recreation. - Such identities can write ClusterAuthorizationRules, Users and Groups that carry
SuperAdminorcluster-admin. A ClusterAuthorizationRule gives this right only withoutnamespaceSelectorandlimitNamespacesand, with multi-tenancy on, only withallowAccessToSystemNamespaces: true, even if it grantscluster-admin. - The cloud MachineHealthCheck is updated in existing clusters. A node in Ready=False is replaced after 10 minutes instead of 5 (Ready=Unknown still after 5), and remediation stops while more than 50% of the cluster’s CAPI machines are unhealthy.
- The controller and agents will be restarted.
- The grant webhooks are now removed together with the module. On a cluster where
multitenancy-manageris already disabled at the update, thecluster-objects-grants-defaultingandcluster-objects-grants-validatorwebhook configurations left by earlier versions stay, and writes to grantable resources in project namespaces (for example, removing a PVC finalizer) keep failing withservice "multitenancy-manager" not found. Delete them by hand:d8 k delete mutatingwebhookconfiguration cluster-objects-grants-defaultingd8 k delete validatingwebhookconfiguration cluster-objects-grants-validatorNothing has to be done on clusters where the module is enabled.
Features
- [admission-policy-engine] Report the availability of the module components with alerts.
- [cert-manager] Add the
highAvailabilityparameter to control the HA mode of the module independently of the global setting. - [cert-manager]
cert-manageruses the cluster-wide proxy server from theproxyparameter ofClusterConfiguration.
In a cluster with theproxyparameter ofClusterConfigurationset, thecert-managercontroller sends requests to the ACME server, HashiCorp Vault and the DNS provider APIs through the proxy server. TheHTTP-01self-check also requests the domain through the proxy server. If the proxy server cannot reach the issuer or the public address of the cluster, add these addresses tonoProxy. - [cloud-provider-dvp] Makes CCM create NPs on every lb creation while using networkPolicy - Restricted
- [cni-cilium] Added an alert and a release requirement that block the Cilium update while CiliumNetworkPolicy or CiliumClusterwideNetworkPolicy objects use the
fromRequiresortoRequiresfields. - [deckhouse] ModuleSource scans skip unchanged release channels using a cheap registry HEAD request instead of pulling the whole release image every cycle.
- [deckhouse-controller] Added the update settings of applications: the
spec.updatefield of an Application and the globalapplications.updateparameter. Deckhouse merges them with the update settings of thedeckhousemodule but doesn’t apply them to application updates yet. - [docs] Add user-authz module architecture section.
- [docs] Add service-with-healthchecks module architecture section.
- [documentation] The in-cluster documentation now includes the guides.
- [documentation] Documentation updates for DP CSE 1.77.
- [multitenancy-manager] A new alert warns about a user project template named
simple, which the built-in template of the same name overwrites after the update. - [user-authz] The new
D8UserAuthzLegacyRBACv2CapabilityBindingFoundalert lists the bindings to built-in capabilities that the next minor release renames without aliases, and names the new name of each capability. - [user-authz] The new
D8UserAuthzLegacyRBACv2CustomCapabilityFoundalert lists custom capabilities that extend the namespace roles only through labels the next role scheme does not select. - [user-authz] New
D8UserAuthzLimitNamespacesEntryNeedsRewriteandD8UserAuthzLimitNamespacesEntryInvalidalerts report ClusterAuthorizationRulelimitNamespacesentries that are anchored only in part or are not valid regular expressions.
Fixes
- [admission-policy-engine] Fixed Gatekeeper policies for
kubectl execandkubectl attachbeing skipped in user namespaces.
Exec and attach requests in user namespaces reach Gatekeeper again, as in Deckhouse 1.73 and 1.74. Every constraint that matches them is evaluated again, including a constraint withoutmatch.kindsor withkinds: ["*"]whose policy does not checkinput.review.operation. Before updating, check that such constraints do not denykubectl execandkubectl attachfor users who need them. - [admission-policy-engine] Disabling the module deletes the webhook configurations before Gatekeeper, so the uninstall no longer blocks
execand the Deckhouse queue. - [admission-policy-engine] Fixed CVE-2026-78659, CVE-2026-78660, CVE-2026-78663, CVE-2026-78669 and CVE-2026-97032 in
golang.org/x/netand CVE-2026-56851 ingolang.org/x/text. - [candi] Restored the lost
kubelet-disable-k-panic-checkpatch for k8s 1.35 and 1.36, so kubelet no longer enforceskernel.panic=10on these versions. - [candi] New nodes reboot on the first bashible run again when a step requests it. Only the first master skips this reboot.
- [cert-manager] The webhook keeps serving requests during a rollout and during a rotation of its CA, and the controller waits for the API server at startup instead of restarting.
- [cert-manager] The
spec.certificateOwnerReffield of a certificate no longer causes an endless Secret update loop when it differs from the global owner reference setting thatcleanupOrphanSecretsenables in theCertManagerHTTPS mode, and is applied to an already issued Secret. - [cert-manager] Fixed CVE-2026-78659, CVE-2026-78660, CVE-2026-78663, CVE-2026-78669 and CVE-2026-97032 in
golang.org/x/net. - [chrony] Bump golang.org/x/crypto in chrony_exporter to fix CVE-2026-56855, CVE-2026-78662.
- [cloud-provider-dvp] The CSI driver reports a volume as detached from a node only after the disk is released by the node virtual machine, so with virtualization that keeps the VM listed on the disk until it is released, the volume is not attached to two nodes at once.
- [cloud-provider-dvp] The CSI driver reports the volume limit of a node, so the scheduler does not place more volumes on a node than its virtual machine can attach.
- [cloud-provider-dvp] Stop infinite check/converge loop caused by the DVP controller annotation on the cloud-init Secret.
- [cloud-provider-vsphere] Fixed master node creation when the datastore is set as a path with parent folders.
- [cloud-provider-yandex] Restored
additionalSubnets,networkTypeand VM labels (nodes.parameters.labels,additionalLabels) for CAPI-managed Yandex nodes.
Machines are not recreated on update. Machines created before the update keep the standard network type and no labels until the NodeGroup is rolled out (change a rolling InstanceClass field or set themanual-rollout-idannotation on the NodeGroup). Machines stuck in provisioning because ofadditionalSubnetsbecome ready without recreation. - [cloud-provider-zvirt] VMs are now actually removed from zVirt on NodeGroup scale-down and cluster destroy
- [common] Fixed a
kube-apiserverpanic on JSON Patch requests whoseadd/replaceof the whole document ortestoperation lacks thevaluefield; such requests are now rejected with a validation error. - [common] kube-apiserver returns a single short validation error for an invalid
fieldManagerinstead of an error for every invalid character. - [common] Fixed CVE-2026-78659, CVE-2026-78660, CVE-2026-78663, CVE-2026-78669 and CVE-2026-97032 in
golang.org/x/netin kube-rbac-proxy. - [control-plane-manager] Keep
resource.k8s.io/v1beta1enabled inkube-apiserver1.34. - [deckhouse] Fixed CVEs in d8 CLI dependencies by updating d8 to v0.33.25.
- [deckhouse] Restore kubectl plugin lookup in d8 k and d8 kubectl
- [deckhouse] Update runtime libraries and VEX exceptions for known vulnerabilities
- [deckhouse-controller] Deleting an Application now also deletes objects with the
werf.io/ownership: anyoneorwerf.io/resource-policyannotation, such as Jobs deployed only on install, if the package lists their kinds inorphanResources. Onlyhelm.sh/resource-policy: keepprotects an object. - [descheduler] Fixed CVE-2026-81870 in descheduler by bumping OpenTelemetry Go modules to v1.45.0.
- [dhctl] Preserve existing SSH keys when switching to the temporary converge user.
- [dhctl] Destroy and abort of a static cluster now fail instead of reporting success when a control-plane node was not cleaned because of an SSH error or cleanup timeout.
- [dhctl] Bump golang.org/x/crypto and OpenTelemetry to fix CVE-2026-56855, CVE-2026-78662, CVE-2026-81870, CVE-2026-81871, CVE-2026-81872.
- [docs] Correct typo in managed-rabbit key in modules library.
- [docs] Improve documentation clarity and fix typos.
- [docs] Fixed tab switching on pages with several tab sets of the same name and opening hidden tabs by an anchor link.
- [docs] Enhance tab activation logic in the documentation.
- [docs] Enhance tab shortcode functionality.
- [docs] Remove raw tags from usage documentation.
- [docs] Show expand icon on non-collapsible top-level CRD parameters in the documentation.
- [docs] Added experimental module enable note to the lifecycle badge.
- [docs] Added registry module port 5001 to Deckhouse ports documentation
- [docs] Rendered all versions from oss.yaml versions array.
- [docs] Update hugo to 0.168.0 in docs-builder
- [docs] Page with DNS names full list was added in to reference.
- [helm_lib] Size the resources of a module whose
resourcesManagementnames a fractional quantity, instead of rendering a zero limit.
Bumpingdeckhouse_lib_helmfrom1.72.11to1.72.24also drops thedrbd.linbit.comtaint tolerations fromhelm_lib_tolerations, so pods rendered through it no longer toleratelost-quorum,force-io-errorandignore-fail-over, and aligns the VPAmaxAllowedmemory to the recommender’s 64Mi quantum. - [istio] Fixed CVEs in istio, kiali and operator images (golang.org/x/net, golang.org/x/text).
- [multitenancy-manager] Disabling the module now removes the cluster-objects grant webhooks, which used to stay behind and block writes in project namespaces.
The grant webhooks are now removed together with the module. On a cluster wheremultitenancy-manageris already disabled at the update, thecluster-objects-grants-defaultingandcluster-objects-grants-validatorwebhook configurations left by earlier versions stay, and writes to grantable resources in project namespaces (for example, removing a PVC finalizer) keep failing withservice "multitenancy-manager" not found. Delete them by hand:d8 k delete mutatingwebhookconfiguration cluster-objects-grants-defaultingd8 k delete validatingwebhookconfiguration cluster-objects-grants-validatorNothing has to be done on clusters where the module is enabled. - [multitenancy-manager] The controller takes its leader lease with any number of replicas, so a rollout of a single-replica installation no longer has two pods upgrading project Helm releases at once.
In single-replica installations a new controller pod reconciles projects only after the previous pod has stopped and released the lease. If the previous pod is lost without a shutdown, for example together with its node, the new pod starts reconciling projects 60 to about 78 seconds after it starts. - [multitenancy-manager] The Project conversions of the next release are registered in advance, so reads and writes of projects keep working while webhook-handler is updated in HA mode.
- [multitenancy-manager] Fixed CVE-2026-78659, CVE-2026-78660, CVE-2026-78663, CVE-2026-78669 and CVE-2026-97032 in
golang.org/x/net. - [multitenancy-manager] The webhooks no longer leave zombie processes in webhook-handler until it runs out of pids.
- [node-manager] Fixed cluster-autoscaler using node group names without the cluster prefix after a Deckhouse restart, which stopped autoscaling and triggered the
D8ClusterAutoscalerTooManyErrorsalert. - [node-manager] Fixed cluster-autoscaler stopping autoscaling of all node groups and restarting every two hours when an MCM node group has
minPerZoneequal tomaxPerZone. - [node-manager] Fixed CVEs in cluster-autoscaler Go dependencies (OpenTelemetry, and grpc and x/crypto for Kubernetes 1.31).
- [node-manager] A StaticInstance whose StaticMachine was deleted is returned to Pending and becomes available for bootstrapping again instead of staying out of the pool forever.
- [node-manager] Fixed cluster-autoscaler not scaling CAPI node groups up from zero for pods requesting ephemeral storage, and putting the whole scale-up of a multi-zone node group into a single zone.
- [node-manager] Bump golang.org/x/crypto and OpenTelemetry in bashible-apiserver and capi-controller-manager.
- [node-manager] Fixed the CAPI MachineHealthCheck replacing cloud nodes that take longer than 5 minutes to become Ready after registering, and limited remediation to when at most half of the CAPI machines are unhealthy.
The cloud MachineHealthCheck is updated in existing clusters. A node in Ready=False is replaced after 10 minutes instead of 5 (Ready=Unknown still after 5), and remediation stops while more than 50% of the cluster’s CAPI machines are unhealthy. - [node-manager] Fixed node-manager failing to apply its values in the BE, SE, SE+, EE, FE and CSE editions when a CAPI node group has a known boot disk size.
- [registry] Build docker-auth from the new fork repository at flant-1.0.0, which fixes CVE-2026-81870.
- [registrypackages] Fixed CVEs in crictl 1.31.
- [service-with-healthchecks] Go deps are updated.
The controller and agents will be restarted. - [terraform-manager] Bump opentofu to 1.12.7 to fix CVE-2026-56855, CVE-2026-78662.
- [tools] Remove d8-gpu namespace from integrity check.
- [user-authn] The
idTokenTTLdescription and theD8UserAuthnIDTokenTTLTooLongalert say that a value of 6 hours or more is replaced with5h59mby the next settings version, and what lowering it does to DexAuthenticator sessions and issued ID tokens. - [user-authn] Restrict the Job that issues the
basic-auth-proxycertificate to the read-only front-proxy CA files and cover it with a SecurityPolicyException. - [user-authn] A DexClient or a DexAuthenticator in a namespace whose name YAML reads as a number or a boolean no longer breaks the user-authn release.
- [user-authn] A DexAuthenticator with a name longer than 45 characters no longer logs its users out on every run of the hook.
- [user-authn] The DexAuthenticator validating webhook no longer logs a jq error for an object without additionalApplications.
- [user-authn] Errors the webhook scripts of user-authn, user-authz and multitenancy-manager write to stderr are logged as warnings instead of info messages.
- [user-authn] Protected applications keep serving users during a short dex outage and during restarts.
- [user-authn] Fixed CVE-2026-78659, CVE-2026-78660, CVE-2026-78663, CVE-2026-78669 and CVE-2026-97032 in
golang.org/x/net. - [user-authn] The webhooks no longer leave zombie processes in webhook-handler until it runs out of pids.
- [user-authz] The authorization webhook no longer restarts while the API server is unreachable, which could deadlock the control plane on bootstrap or an API server restart with multitenancy enabled.
- [user-authz] a
kube-apiserverhealth probes no longer fail whenauthz-webhookis unavailable. - [user-authz] The identity-assign check treats an identity with every permission across the cluster, such as a subject of
cluster-adminbound by a ClusterRoleBinding, as SuperAdmin.
Such identities can write ClusterAuthorizationRules, Users and Groups that carrySuperAdminorcluster-admin. A ClusterAuthorizationRule gives this right only withoutnamespaceSelectorandlimitNamespacesand, with multi-tenancy on, only withallowAccessToSystemNamespaces: true, even if it grantscluster-admin. - [user-authz] The RBACv2 roles d8:manage:* and d8:use:role:* keep the rights of the modules that have already switched to the capability labels of the next release while the upgrade to it is in progress.
- [user-authz] The authorization webhook keeps applying ClusterAuthorizationRules when a
limitNamespacesentry is not a valid regular expression; the entry is left out, and its rule covers fewer namespaces.
Before, such an entry stopped the webhook from reloading the rules, so subjects of rules added later, and every subject after a restart of the webhook, were not limited to their namespaces. Subjects of a rule whose only entry is invalid now get no namespace through the entries of that rule. - [user-authz] The
D8UserAuthzLegacyRBACv2CustomRoleFoundalert lists every custom role that selects capabilities by therbac.deckhouse.io/kind: manage|uselabel or by the aggregation label of a lineage the next role model does not collect, or that loses its RoleBindings in the namespaces of the modules, and the FAQ describes how to prepare such a role before the update without losing access. - [user-authz] An AuthorizationRule in a namespace whose name YAML reads as a number, a boolean or null no longer breaks the module’s bindings.
- [user-authz] Managers who assign roles by range can again change DexProviders, UserOperations, Users and Groups of identities that hold a ClusterAuthorizationRule.
- [user-authz] The authorization webhook answers the requests in flight when it restarts instead of denying them.
- [user-authz] Fixed CVE-2026-78659, CVE-2026-78660, CVE-2026-78663, CVE-2026-78669 and CVE-2026-97032 in
golang.org/x/netand CVE-2026-56851 ingolang.org/x/text. - [user-authz] The webhooks no longer leave zombie processes in webhook-handler until it runs out of pids.
1.77.2
Know before update
- A StaticInstance whose connectivity check fails now keeps its reservation for the whole bootstrap window instead of returning to the pool on every failed attempt, and the bootstrap (20 min) and cleanup (10 min) timeouts, which previously never fired, are now reachable. If nothing ever ran on the host, the bootstrap timeout returns the instance to the pool after 20 minutes as before. If the host was already bootstrapped in part, the instance returns only through MachineHealthCheck remediation (nodeStartupTimeoutSeconds: 1200) plus the cleanup timeout, so it becomes available again roughly 20 + 10 minutes after the failure, and remediation reboots the host as part of the cleanup.
- A cluster still on 1.76 whose user-authn ModuleConfig sets idTokenTTL to 6h or more updates to this release without Deckhouse crash-looping, and stays on it until idTokenTTL is lowered below 6h. A cluster already crash-looping on 1.77.0 or 1.77.1 does not recover by itself, since the release controller runs in the pod that is down; set failurePolicy Ignore on the module-configs.deckhouse-webhook.deckhouse.io webhook of the deckhouse-webhook ValidatingWebhookConfiguration, lower idTokenTTL below 6h, and restart the deckhouse pod.
- Clusters bootstrapped before v1.55 were switched to the
Baselinedefault PSS policy in 1.77, which denies privileged workloads in user namespaces. After the update, the default policy on these clusters becomes less strict: it is switched back toPrivileged, and privileged pods in user namespaces pass admission again. To keep theBaselinepolicy, setsettings.podSecurityStandards.defaultPolicy: Baselineexplicitly in theadmission-policy-engineModuleConfig. - Clusters whose
d8-system/install-dataConfigMap holds an empty or invalid version and whosepodSecurityStandards.defaultPolicyisn’t set were switched to theBaselinedefault PSS policy in 1.77. After the update, the default policy on these clusters becomes less strict: it is switched back toPrivileged. To keep theBaselinepolicy, setsettings.podSecurityStandards.defaultPolicy: Baselineexplicitly in theadmission-policy-engineModuleConfig. - Deckhouse 1.78 makes
Baselinethe default PSS policy for all clusters. On clusters bootstrapped with a Deckhouse version lower than v1.55 or with an unknown bootstrap version, thePodSecurityStandardsDefaultPolicyNotSetalert fires, and the update to 1.78 is blocked untilsettings.podSecurityStandards.defaultPolicyis set explicitly in theadmission-policy-engineModuleConfig. To keep the current behavior, setdefaultPolicy: Privileged. To switch toBaselineorRestricted, make sure that the workloads in non-system namespaces comply with the chosen policy first. - In Deckhouse CSE, a DexProvider that already has
enableBasicAuth: truekeeps it after the update, and any change to itsoidcorldapsection is rejected untilenableBasicAuthis set tofalse. The same holds for an LDAP provider that combinesinsecureNoSSLwithstartTLS,insecureSkipVerifyorrootCAData. SetenableBasicAuth: falseand resolve the LDAP TLS settings on such providers. - The
d8-kube-proxypods will be restarted on all nodes. - Two Go hooks registered in one file shared a derived name, so the Kubernetes-binding half lost its bindings and the
d8_cluster_objects_grant_violatedseries was recomputed only by the periodic scan. The series itself was never missing; it could lag a policy change by up to two minutes. - dhctl converge no longer deadlocks on master replacement or externalIPAddresses removal in Yandex Cloud; the terraform-manager provider release is patched to detach one-to-one NAT before deleting a reserved address.
Features
- [admission-policy-engine] Added the
PodSecurityStandardsDefaultPolicyNotSetalert that fires when theinstall-dataConfigMap is missing and the default PSS policy must be set explicitly in the ModuleConfig. - [admission-policy-engine] The
PodSecurityStandardsDefaultPolicyNotSetalert now fires whenpodSecurityStandards.defaultPolicyisn’t set and thePrivilegedpolicy is derived from the bootstrap version. The update to Deckhouse 1.78 is blocked until the policy is set explicitly.
Deckhouse 1.78 makesBaselinethe default PSS policy for all clusters. On clusters bootstrapped with a Deckhouse version lower than v1.55 or with an unknown bootstrap version, thePodSecurityStandardsDefaultPolicyNotSetalert fires, and the update to 1.78 is blocked untilsettings.podSecurityStandards.defaultPolicyis set explicitly in theadmission-policy-engineModuleConfig. To keep the current behavior, setdefaultPolicy: Privileged. To switch toBaselineorRestricted, make sure that the workloads in non-system namespaces comply with the chosen policy first. - [deckhouse-controller] Add the
x-deckhouse-immutable,x-deckhouse-ui-group, andx-deckhouse-enum-switch-settingsextensions to Application package settings schemas; the Application webhook rejects changes tox-deckhouse-immutablefields after the settings were applied. - [deckhouse-controller] When an Application is deleted, Deckhouse also deletes the objects of the kinds listed in the package’s
orphanResourcesvalue that the application left in its namespace. - [docs] Add user-authn-controller, user-api and basic-auth-proxy components to user-authn module architecture section.
- [docs] Rename DKP->DP and replace PNG->SVG architecture section
- [docs] Fix links to architecture diagrams
- [docs] Add alb module architecture section.
- [docs] Add managed services modules architecture documentation section.
- [docs] Add sdn module architecture section.
Fixes
- [admission-policy-engine] Restored the
Privilegeddefault PSS policy for clusters bootstrapped before v1.55 (without theinstall-dataConfigMap).
Clusters bootstrapped before v1.55 were switched to theBaselinedefault PSS policy in 1.77, which denies privileged workloads in user namespaces. After the update, the default policy on these clusters becomes less strict: it is switched back toPrivileged, and privileged pods in user namespaces pass admission again. To keep theBaselinepolicy, setsettings.podSecurityStandards.defaultPolicy: Baselineexplicitly in theadmission-policy-engineModuleConfig. - [admission-policy-engine] Fixed CVEs in the
gatekeeperandratifyimages. - [admission-policy-engine] Fixed the default Pod Security Standards (PSS) policy being switched to
Baselinewithout notice when the bootstrap version in thed8-system/install-dataConfigMap is empty or invalid. Such clusters now get thePrivilegedpolicy.
Clusters whosed8-system/install-dataConfigMap holds an empty or invalid version and whosepodSecurityStandards.defaultPolicyisn’t set were switched to theBaselinedefault PSS policy in 1.77. After the update, the default policy on these clusters becomes less strict: it is switched back toPrivileged. To keep theBaselinepolicy, setsettings.podSecurityStandards.defaultPolicy: Baselineexplicitly in theadmission-policy-engineModuleConfig. - [candi] remove d8-bcheck in cleanup_static_node
- [candi] containerd integrity errors are written to
/var/log/containerd/containerd-integrity.logon RED OS 8. - [cert-manager] Fixed CVEs in the cert-manager images.
- [cert-manager] Gateway API integration is now enabled automatically when the Gateway and HTTPRoute CRDs are installed.
- [chrony] Bump golang.org/x/crypto to v0.55.0 in chrony_exporter to fix CVE-2026-56854.
- [ci] update werf to v2.74.2-dk.1
- [ci] Add allowed alerts for D8Monitoring for e2e which uses 1.77+
- [ci] The documentation changes validation resolves symlinked related language files.
- [ci] Fix changelog-generator image tag in changelog-update job to use the shared CI_IMAGES_TAG
- [ci] add skip docs validation label
- [cloud-provider-dvp] Fixed the
D8CloudProviderDVPMigrationPendingalert that kept firing after the migration resources were applied. - [cloud-provider-dvp] Fixes CVE vulnerabilities in cloud-provider-dvp images.
- [cloud-provider-dvp] Fixes cluster bootstrap getting stuck when
capdvp-controller-managerand the validation webhook cannot reach the API server and DNS before the CNI is ready. - [cloud-provider-openstack] Fixes cluster bootstrap getting stuck when
capo-controller-managercannot reach the API server and DNS before the CNI is ready. - [cloud-provider-vcd] Batch VM cache refresh on cache miss
- [cloud-provider-vcd] Fixes cluster bootstrap getting stuck when
capcd-controller-managercannot reach the API server and DNS before the CNI is ready. - [cloud-provider-yandex] fix “Address in use” failures when replacing nodes and when removing external IP addresses
dhctl converge no longer deadlocks on master replacement or externalIPAddresses removal in Yandex Cloud; the terraform-manager provider release is patched to detach one-to-one NAT before deleting a reserved address. - [common] Fix kubelet podSandbox order
- [common] fixed private build cni-migration-*
- [common] Fix CVEs in the
kube-rbac-proxyimage. - [common] Fixed known vulnerabilities in the CoreDNS image used by the node-local-dns module.
- [control-plane-manager] A new etcd
maxDbSizeis applied without adeckhouserestart. - [control-plane-manager] Fix kubectl exec through Kubernetes API published with Gateway API.
- [deckhouse] set releaseChannel in CSE
- [deckhouse] Fix setting Scaled before Installed.
- [deckhouse] Remove leftover symlink-backend entries when deploying packages with the erofs backend.
- [deckhouse] Fix random startup failures with ‘waitid - no child processes’ when restoring erofs modules.
- [deckhouse] Fix CVEs in deckhouse-cli.
- [dhctl] Restoring of handling api_server_url and api_server_token in dhctl
- [dhctl] Bump google.golang.org/grpc to v1.83.2 to fix CVE-2026-84304 and CVE-2026-84445.
- [dhctl] Fixed terraform-auto-converger, terraform-state-exporter and
dhctl convergefailing on cloud clusters whosecloud-provider-<name>ModuleConfig has nospec.enabledfield. dhctl now treatsspec.enabledas optional and a ModuleConfig without it as disabled. - [dhctl] Fixed the terraform-auto-converger init container failing on clusters already migrated to OpenTofu - the migration check now skips planning when no Terraform state is stored in the cluster.
- [docs] Fixed FAQ section for ansible inventory.
- [docs] Register the ports of csi-hitachi and csi-mind-ustor, list the registry storage ports, and drop storage ports that are no longer bound in the host network from the ports reference.
- [docs] Fix search hanging when IndexedDB upgrade is blocked.
- [docs] Removed mentions of Cilium from the inter-cluster cooperation documentation page.
- [istio] Fixed the istio module getting stuck when switching
globalVersionfrom 1.21 while proxies are still connected to the 1.21 control plane. - [istio] Fix leaking istiod control plane after removing an Istio version or disabling the module.
- [istio] Switching back to Istio 1.21 no longer removes the
networking.istio.io/v1API from the Istio CRDs and no longer breaks the istio release. - [istio] Deckhouse update requirements now take into account Istio control planes that are still being removed.
- [istio] The Istio operator now recovers Helm releases stuck in the
pending-*oruninstallingstate. - [kube-dns] Set resource limits for the
render-etc-hosts-with-cluster-domain-aliasesinit container so that pods can be created in namespaces with a ResourceQuota onlimits.cpu/limits.memory. - [kube-proxy] Fixed kube-proxy failing to start on Kubernetes 1.36 and higher because of the removed
TopologyAwareHintsfeature gate.
Thed8-kube-proxypods will be restarted on all nodes. - [metallb] Fixed creation of
BGPPeerobjects on Kubernetes 1.36+, which blocked BGP sessions and LoadBalancer IP announcements in BGP mode. - [multitenancy-manager] The grant-violation hook re-evaluates policies on change again, not only on its two-minute schedule.
Two Go hooks registered in one file shared a derived name, so the Kubernetes-binding half lost its bindings and thed8_cluster_objects_grant_violatedseries was recomputed only by the periodic scan. The series itself was never missing; it could lag a policy change by up to two minutes. - [multitenancy-manager] Fixed vulnerabilities in the
multitenancy-managerimage. - [node-manager] Increase the capi-controller-manager VPA maximum memory recommendation to prevent health-check failures and restarts under memory pressure.
- [node-manager] Stop caps-controller-manager from rewriting StaticInstance objects in a hot loop when a connectivity check fails, which caused a sustained load on etcd.
A StaticInstance whose connectivity check fails now keeps its reservation for the whole bootstrap window instead of returning to the pool on every failed attempt, and the bootstrap (20 min) and cleanup (10 min) timeouts, which previously never fired, are now reachable. If nothing ever ran on the host, the bootstrap timeout returns the instance to the pool after 20 minutes as before. If the host was already bootstrapped in part, the instance returns only through MachineHealthCheck remediation (nodeStartupTimeoutSeconds: 1200) plus the cleanup timeout, so it becomes available again roughly 20 + 10 minutes after the failure, and remediation reboots the host as part of the cleanup. - [node-manager] Fix duplicate NVIDIA GPU dashboards when the gpu module is enabled.
- [node-manager] Bump google.golang.org/grpc to v1.83.2 in bashible-apiserver to fix CVE-2026-84304 and CVE-2026-84445.
- [node-manager] Bump golang.org/x/crypto to v0.55.0 and google.golang.org/grpc to v1.83.2 in capi-controller-manager to fix CVE-2026-56854, CVE-2026-84304 and CVE-2026-84445.
- [node-manager] Fixes CVE vulnerabilities in caps-controller-manager.
- [node-manager] Hooks no longer fail the module run when a NodeGroup, Node or NodeUser is deleted between the snapshot and the patch. Such failures delayed the readiness of the
deckhousepod in clusters that delete NodeGroups often. - [registry] Bump dependencies in registry.
- [registry] Bump golang.org/x/crypto in nodeservices-manager to fix CVE-2026-56854, add VEX statements for x/crypto/ssh findings.
- [registrypackages] Bumped
containerdto 1.7.36 / 2.2.9 and OpenTelemetry to 1.45.0 incontainerdandcrictlto fix CVEs. - [service-with-healthchecks] Fixed metrics endpoints being reachable without going through kube-rbac-proxy authorization.
- [terraform-manager] Bump google.golang.org/grpc to v1.82.2 in opentofu to fix CVE-2026-84445. CVE-2026-84304 is not exploitable in opentofu and is covered by VEX.
- [terraform-manager] Fixed the
to-tofu-migratorinit container ofterraform-auto-convergerfailing to resolve the provider bundle digest, which kept auto-converge from starting. - [user-authn] An idTokenTTL of 6h or more no longer stops Deckhouse from starting; it raises the D8UserAuthnIDTokenTTLTooLong alert and blocks the update to the next minor release until it is lowered.
A cluster still on 1.76 whose user-authn ModuleConfig sets idTokenTTL to 6h or more updates to this release without Deckhouse crash-looping, and stays on it until idTokenTTL is lowered below 6h. A cluster already crash-looping on 1.77.0 or 1.77.1 does not recover by itself, since the release controller runs in the pod that is down; set failurePolicy Ignore on the module-configs.deckhouse-webhook.deckhouse.io webhook of the deckhouse-webhook ValidatingWebhookConfiguration, lower idTokenTTL below 6h, and restart the deckhouse pod. - [user-authn] Fixed CVEs in the module images.
- [user-authn] In Deckhouse CSE, a DexProvider with
enableBasicAuthis rejected, since the edition does not ship the basic authentication proxy; the CSE copy of the CRD also gets the LDAP TLS checks of the module.
In Deckhouse CSE, a DexProvider that already hasenableBasicAuth: truekeeps it after the update, and any change to itsoidcorldapsection is rejected untilenableBasicAuthis set tofalse. The same holds for an LDAP provider that combinesinsecureNoSSLwithstartTLS,insecureSkipVerifyorrootCAData. SetenableBasicAuth: falseand resolve the LDAP TLS settings on such providers. - [user-authz] Fixed CVEs in the
permission-browser-apiserverimage. - [user-authz] The authorization webhook answers 500 and logs the real error instead of terminating when it cannot marshal a response.
1.77.1
Know before update
- A Deckhouse release suspended on its release channel is no longer applied by clusters that are behind and reach it through a step-by-step update. The suspend flag lives only in the release-channel image; previously it was dropped when the target release was built from its per-version image, so lagging clusters updated to a suspended release anyway.
- After the update, the controller adds the annotations and labels of every ServiceWithHealthchecks to the Service created for it.
If a ServiceWithHealthchecks of the
LoadBalancertype carries annotations that configure the load balancer (network.deckhouse.io/load-balancer-ips,network.deckhouse.io/load-balancer-shared-ip-keyand similar), the load balancer controller applies them and may assign a different address to the service, which causes a short interruption of the connections. - CAPI NodeGroups whose MachineDeployment was stuck on an old MachineTemplate are rolled to the current InstanceClass after the update, within maxSurgePerZone/maxUnavailablePerZone. Groups already on the current template are not touched.
- D8UserAuthzRenderSlow requires at least three renders averaging above 30 seconds over 30 minutes, D8UserAuthzReleaseSlow at least two releases averaging above 5 minutes over one hour, and both must hold for 15 minutes. Severities are one step lower (6 and 7). Their descriptions now point at CPU shortage on the node rather than at the number of ClusterAuthorizationRules, which does not affect render time.
- During the first
user-authzrelease after the update the per-role bindings are protected from the release engine, the aggregated ones are created, and the per-role bindings are deleted right after the release. Permissions granted through annotated ClusterRoles stay in place throughout. Theuser-authz.deckhouse.io/access-levellabel is now set automatically on annotated ClusterRoles. In audit logs, access granted through annotated ClusterRoles is attributed touser-authz:<level>:custominstead of the individual ClusterRole. - Endpoints for pods in a terminal phase (Failed/Succeeded) are no longer published. In DVP clusters this prevents traffic from being routed to a VirtualMachine IP that has been reused by another pod. Pods being deleted are now published with the serving and terminating conditions, which enables the graceful shutdown flow for consumers. Pod readiness is derived from the PodReady condition, and stale probe results are reset when a pod becomes not ready, is recreated, or changes its IP.
- The
deschedulerDeployment ind8-deschedulerrestarts once on upgrade to pick up the new argument. Descheduling behavior is unchanged for workloads that do not implement their own eviction policy. - The
service-with-healthcheckscontroller and agent are restarted. AServiceWithHealthchecksthat was left inNotAllEndpointsAreReadywith no EndpointSlice recovers automatically once the updated agent starts, no manual action is needed. - The installer, the deckhouse ServiceAccount, holders of the cluster administrator kubeconfig and the kube-system / d8-system ServiceAccounts can create a User or a Group named by an existing authorization rule without the acknowledgement annotation. Every other requester is checked as before.
Features
- [candi] Base images updated to v3
- [descheduler] Always run the descheduler with the upstream
EvictionsInBackgroundfeature gate enabled.
ThedeschedulerDeployment ind8-deschedulerrestarts once on upgrade to pick up the new argument. Descheduling behavior is unchanged for workloads that do not implement their own eviction policy. - [docs] Add managed-valkey module architecture section.
- [docs] Refactor build pipeline and add CSE edition support.
- [docs] Update virtualization module architecture section according to release v1.10.0
- [docs] Add monitoring-kubernetes module architecture section.
- [docs] Add documentation for Managed PostgreSQL service.
- [docs] Add extended-monitoring module architecture section.
- [docs] Add registry-packages-proxy module architecture section.
- [docs] Expand the documentation for virtualization.
- [docs] Add istio module architecture section.
- [registry] Migration part of new registry implementation.
- [user-authn] Update the Dex authentication page to the new Deckhouse branding, with light and dark themes selected by the user’s system settings.
Fixes
- [candi] Fix hanging sysctl tuner on nodes with many loop and dm devices
- [candi] containerd v2 no longer starts without the
erofssnapshotter, and loads theerofsmodule before start. - [cert-manager] Restore the
cert-managerandcainjectorpermissions that were lost against the upstream chart, including access toListenerSetresources. - [chrony] Fixed the unit-detection condition in the
disable-ntp-on-node.shnode step. - [cloud-provider-aws] fix GetCapacity not implemented error spam in logs
- [cloud-provider-aws] Fixed CVEs in the cloud-provider-aws module.
- [cloud-provider-azure] fix GetCapacity not implemented error spam in logs
- [cloud-provider-azure] Fix multimaster bootstrap by falling back to the
d8-masters-kubernetes-data-device-pathSecret when the master data device LUN is not provided via cloud-init. - [cloud-provider-dvp] fix GetCapacity not implemented error spam in logs
- [cloud-provider-dvp] Fixed CVEs in the cloud-provider-dvp module.
- [cloud-provider-gcp] fix GetCapacity not implemented error spam in logs
- [cloud-provider-gcp] Fixed CVEs in the cloud-provider-gcp module.
- [cloud-provider-huaweicloud] Fixed CVEs in the cloud-provider-huaweicloud module.
- [cloud-provider-openstack] Fixed CVEs in the cloud-provider-openstack module.
- [cloud-provider-yandex] Fixed CVEs in the cloud-provider-yandex module.
- [cloud-provider-zvirt] Resize the boot disk of a recreated VM, let a replaced VM actually power off, and recreate the VM when customNetworkConfig changes.
- [deckhouse] Exclude all Python test files from webhook image.
- [deckhouse] Keep
x-kubernetes-sensitive-dataon CRDs that have a conversion webhook. - [deckhouse] Allow editing and deleting the resources of an application under maintenance.
- [deckhouse-controller] Honor a channel-level release suspend for clusters that reach the suspended version through a step-by-step update.
A Deckhouse release suspended on its release channel is no longer applied by clusters that are behind and reach it through a step-by-step update. The suspend flag lives only in the release-channel image; previously it was dropped when the target release was built from its per-version image, so lagging clusters updated to a suspended release anyway. - [deckhouse-controller] Stop the module source registry fan-out from replaying and flooding the main queue with duplicate moduleRun tasks.
- [deckhouse-controller] Do not commit the package repository registry checksum when no application could be annotated, which left them all on stale registry settings.
- [docs] Fix broken JUnit XML report links on the Kubernetes conformance page.
- [docs] Clarify global storageClass behavior for Prometheus
- [docs] Fix alert rendering in OpenAPI specs.
- [istio] Waypoints are no longer advertised as cross-network east-west gateways.
- [istio] Enable Istio Gateway API deployment and gatewayclass controllers for istiod 1.27 and 1.29.
- [istio] Document that ambient-mode workloads cannot take part in federation or multicluster.
- [istio] Fix istiod RBAC rules that drifted away from upstream.
- [multitenancy-manager] Refresh grant webhook CA bundles after the admission certificate is rotated.
- [node-manager] node-controller reads the CAPI instance-class checksum from a helm-rendered ConfigMap instead of a possibly stale MachineTemplate, so MachineDeployments follow InstanceClass changes and their bootstrap Secrets keep a valid token.
CAPI NodeGroups whose MachineDeployment was stuck on an old MachineTemplate are rolled to the current InstanceClass after the update, within maxSurgePerZone/maxUnavailablePerZone. Groups already on the current template are not touched. - [node-manager] node-controller no longer removes taints set by other components on the first reconcile of Static and CloudPermanent nodes.
- [registry] Fix the order in which the Unmanaged params are stored for the fallback to the Legacy mode.
- [registry] Fix the security context in the registry module for CSE.
- [registry-packages-proxy] Bump golang.org/x/{crypto,net,sys,text} to clear 24 CVEs, including CVE-2026-56854 (CRITICAL) and CVE-2026-25681, CVE-2026-39821, CVE-2026-46600 (HIGH).
- [registrypackages] Fix dm-verity devices that could never be closed for containerd 2.2.7 (CSE only).
- [registrypackages] Bumped containerd to 1.7.35/2.2.8; bumped grpc, x/crypto, and websocket in containerd, crictl, and Kubernetes 1.32–1.36 gomod patches.
- [registrypackages] Fsync the EROFS layer blob in
containerdbefore committing snapshot metadata, so an unclean shutdown can no longer leave an unmountable image layer. - [service-with-healthchecks] Annotations and labels of a ServiceWithHealthchecks are now copied to the Service created for it.
After the update, the controller adds the annotations and labels of every ServiceWithHealthchecks to the Service created for it. If a ServiceWithHealthchecks of theLoadBalancertype carries annotations that configure the load balancer (network.deckhouse.io/load-balancer-ips,network.deckhouse.io/load-balancer-shared-ip-keyand similar), the load balancer controller applies them and may assign a different address to the service, which causes a short interruption of the connections. - [service-with-healthchecks] Stopped publishing terminated pods in EndpointSlices and started publishing pods being deleted as terminating endpoints.
Endpoints for pods in a terminal phase (Failed/Succeeded) are no longer published. In DVP clusters this prevents traffic from being routed to a VirtualMachine IP that has been reused by another pod. Pods being deleted are now published with the serving and terminating conditions, which enables the graceful shutdown flow for consumers. Pod readiness is derived from the PodReady condition, and stale probe results are reset when a pod becomes not ready, is recreated, or changes its IP. - [service-with-healthchecks] Fixed a load balancer losing all its endpoints for good after a target pod was replaced.
Theservice-with-healthcheckscontroller and agent are restarted. AServiceWithHealthchecksthat was left inNotAllEndpointsAreReadywith no EndpointSlice recovers automatically once the updated agent starts, no manual action is needed. - [tools] Bump d8-cli to v0.33.17 -
d8 mirror pushrestores the release channel tags of the installer image, sodocker run <repo>/install:<channel>works against a mirrored registry - [tools] Bump d8-cli to v0.33.19 - closes the CVE findings in the bundled d8 binary (grpc 1.83.2 and x/image 0.45.0) and adds resume of interrupted d8 data and snapshot transfers
- [user-authz] The identity collision check no longer applies to the platform’s own identities, so a retried bootstrap creates the initial User again instead of timing out.
The installer, the deckhouse ServiceAccount, holders of the cluster administrator kubeconfig and the kube-system / d8-system ServiceAccounts can create a User or a Group named by an existing authorization rule without the acknowledgement annotation. Every other requester is checked as before. - [user-authz] Grant custom ClusterRoles (annotated with
user-authz.deckhouse.io/access-level) through one aggregated ClusterRole per access level, so the number of bindings per ClusterAuthorizationRule/AuthorizationRule no longer depends on the number of such roles.
During the firstuser-authzrelease after the update the per-role bindings are protected from the release engine, the aggregated ones are created, and the per-role bindings are deleted right after the release. Permissions granted through annotated ClusterRoles stay in place throughout. Theuser-authz.deckhouse.io/access-levellabel is now set automatically on annotated ClusterRoles. In audit logs, access granted through annotated ClusterRoles is attributed touser-authz:<level>:custominstead of the individual ClusterRole. - [user-authz] Group the module alerts under a group named apart from the alerts themselves, so the incident shelf accepts them.
- [user-authz] The D8UserAuthzRenderSlow and D8UserAuthzReleaseSlow alerts no longer fire on a single slow render or release, such as a cluster bootstrap under CPU pressure or the one-time migration pass.
D8UserAuthzRenderSlow requires at least three renders averaging above 30 seconds over 30 minutes, D8UserAuthzReleaseSlow at least two releases averaging above 5 minutes over one hour, and both must hold for 15 minutes. Severities are one step lower (6 and 7). Their descriptions now point at CPU shortage on the node rather than at the number of ClusterAuthorizationRules, which does not affect render time.
Version 1.76
Important
-
The fencing-agent component of the
node-managermodule now uses a gossip-based protocol (memberlist library) for distributed node health monitoring. This reduces the risk of false worker node reboots when the control plane is unavailable or the API server is under heavy load. -
Fixed an issue where
cluster-autoscalercould get stuck after a failed machine creation in the cloud (for errors other thanResourceExhausted). The cluster now automatically recovers its ability to scale after temporary cloud provider failures.
Deckhouse subsystem
-
Added support for deploying a DP cluster using the
Proxymode for accessing the container registry. -
Kubectlon cluster nodes has been replaced with thed8 kalias (Deckhouse CLI utility). -
Changes in
dhctl(CLI installer):-
Added support for running
dhctlas a standalone binary without downloading the full image with all dependencies. Required components are pulled from the registry during execution, reducing startup time. -
Added validation of public SSH keys. Invalid keys are now detected earlier, before they can cause connection issues.
-
Improved handling of TLS certificates for registries. The system certificate pool is now used together with a custom CA.
-
Added validation of InstanceClass against the selected cloud provider before installation, allowing early detection and installation abort in case of misconfiguration.
-
Fixed namespace configuration during bootstrap.
Dhctlnow updates settings even if the namespace already exists.
-
-
Improved multi-master resiliency. Kubernetes versions are now correctly detected even when some master nodes are unavailable. Kube-apiserver no longer routes requests to etcd nodes in the learner state.
Kubernetes & Scheduling subsystem
-
The
d8-cluster-kubernetesConfigMap now includes supported Kubernetes versions (supportedVersions), versions available for upgrade and downgrade (availableVersions), the current automatic version (automaticVersion). -
Changes in the
deschedulermodule:-
Upgraded to version 0.35.1. Added namespace filtering via label selectors and other improvements (see the module changelog for details).
-
Added the
deschedulingIntervalparameter to control how often descheduling runs:Frequent,Moderate, orRare. -
Added the
RemovePodsHavingTooManyRestartsstrategy, which evicts pods exceeding a restart threshold, freeing node resources from pods inCrashLoopBackOffstate. -
The module now automatically uses the Kubernetes Metrics API when the
metrics.k8s.ioAPI group is available in the cluster. This provides real resource consumption data for strategies to use.
-
-
The
vertical-pod-autoscalermodule updated to version 1.6.0. Key changes:-
The
InPlaceOrRecreatemode is now used by default. In this mode, VPA attempts to adjust pod requests and limits without recreating pods (in-place updates). -
Reduced likelihood of pod restarts when in-place updates are not possible (via the
--in-place-skip-disruption-budgetflag), alsominReplicasvalue is no longer considered for in-place updates.
-
-
VPA memory recommendations are now rounded to 64 MiB, improving readability and reducing unnecessary pod restarts caused by minor recommendation fluctuations.
-
For Kubernetes 1.34 and later, the
DRAExtendedResourcefeature gate is enabled by default. -
Added the
D8EtcdHighRequestsLatencyalert to detect high etcd request latency, which can lead to API server timeouts. A corresponding metric has been added to the “Control Plane Status” dashboard.
IAM subsystem
-
Changes in the
user-authnmodule:-
Disabled the insecure OAuth 2.0 Implicit Flow. In that flow, access tokens are passed in URL fragments, posing leakage risks (for example, via the
Refererheader, browser history, or logs). Any integrations using the Authorization Code Flow are unaffected. -
Added SAML 2.0 support via the DexProvider resource. DP clusters can now be connected to such identity providers as AD FS, Okta, Keycloak, OneLogin, and Shibboleth.
-
Added refresh token support for SAML in Dex. This allows DexAuthenticator и kubeconfig-generator to refresh tokens without a manual user login.
-
Added Single Logout (SLO) support. The identity provider can end user sessions via
/saml/slo/{connector}. -
Fixed stale cache issue on the login page by adding a Dex build-based
cacheparameter to the CSS URL.
-
-
In multitenancy mode, users without permissions defined via ClusterAuthorizationRule no longer see all namespaces in the Deckhouse web UI. The default behavior is now deny-by-default: users get an empty namespace list unless access is explicitly granted. This does not affect privileged groups (
system:masters,kubeadm:cluster-admins). Documentation of theuser-authzmodule now includes an example for granting access to all namespaces. -
Added support for the
service-with-health-checksmodule in isolated projects. A NetworkPolicy exception for thed8-service-with-healthchecksnamespace has been added to the project template, which allows module agents to verify service availability and correctly operate under conditions of network isolation.
Security subsystem
-
Gatekeeper upgraded to 3.22.0 and Ratify to 1.4.0. Key changes include a unified CEL and Rego policy behavior, added namespace context support in policies, and a CLI tool for policy benchmarking.
-
The
denyVulnerableImagessetting has been moved fromadmission-policy-enginetooperator-trivy, making it the single owner of this logic. -
The
cert-managermodule upgraded to 1.20.0. Key changes include added support for Azure DNS Private Zones in the DNS01 challenge, added annotationacme.cert-manager.io/http01-ingress-ingressclassnamefor redefining IngressClasses in HTTP-01 solvers, and security fixes. -
Updated vulnerable Go dependencies in
multitenancy-manager(go-jose,golang.org/x/crypto,grpc, etc.).
Cluster & Infrastructure subsystem
-
Changes in the
node-managermodule:-
Improved draining hook event generation for better observability of the node eviction process.
-
Fixed StaticInstance issue causing infinite infrastructure deletion loops. That prevents infinite cleanup and queueing cycles.
-
Apiserver-proxy rewritten from NGINX to native Go implementation for better stability and reconnection speed.
-
-
Changes in the
cloud-provider-dvpmodule:-
Added hybrid cluster support. Static master nodes and cloud worker nodes can now be used in a single cluster.
-
Improved converge reliability. Reworked the mechanism of resource readiness check and added the fail-fast behavior for possible errors when creating the infrastructure.
-
Added default StorageClass inheritance for Cluster API child clusters.
-
Improved VM lifecycle handling, reducing potential problems when deleting VMs and associated objects.
-
Added support for the
masterNodeGroup.instanceClass.additionalDisksparameter that lets you configure additional disks for master nodes.
-
-
Changes in the
cloud-provider-awsmodule:-
Spot node draining logic was moved into DP, making the spot node handling more unified.
-
Fixed handling of regions without
DescribeInstanceTopology, reducing the number of false IAM errors.
-
-
Changes in the
cloud-provider-openstackmodule:-
Added
ipMode: Proxysupport for LoadBalancer in Kubernetes 1.32 and later. -
Added the
csiDriver.fsGroupPolicyparameter for the CSI driver, which defines whether the volume supports changing owner and permissions before mounting.
-
-
Changes in the
cloud-provider-vspheremodule:-
Enabled the CSI snapshotter, enabling volume snapshotting.
-
Fixed zone and datastore detection. Now the only considered zones are the ones specified in the provider configuration.
-
-
In the
cloud-provider-azuremodule, added support for NVMe disks on Gen2 VMs. -
In the
cloud-provider-gcpmodule, addedenableNestedVirtualizationandadditionalDisksparameters to GCPInstanceClass, which let you configure nodes for workloads using virtualization and storage scenarios. -
In the
cloud-provider-yandexmodule, fixed converge failure when removing externalIPAddresses. -
DVP, Huawei Cloud, and zVirt providers were migrated to Cluster API
v1beta2. -
For Huawei Cloud and VCD providers, added SecurityPolicyException resources and enabled security policy validations.
Network subsystem
-
Added support for handling incoming traffic via Gateway API using the
albmodule. -
Changes in the
cni-ciliummodule:-
Added conntrack table synchronization for live VM migration.
-
Added ICMP (ping) support via ExternalIP for LoadBalancer services with MetalLB.
-
-
Added support for switching between supported CNI plugins in DP clusters.
-
In the
node-local-dnsmodule, addeddisableIPv6parameter to disable IPv6 DNS resolution.
The complete list of changes, including the updated components, is available in the changelog on GitHub.
1.76.17
Know before update
- A diff will appear in the Terraform state for the base infrastructure: the E zone will be added. Because of this, the Terraform check will fail on the first master node. Run
dhctl convergefor the base infrastructure, and everything should get back to normal.
Features
- [cloud-provider-yandex] Add support for the
ru-central1-ezone in Yandex Cloud.
A diff will appear in the Terraform state for the base infrastructure: the E zone will be added. Because of this, the Terraform check will fail on the first master node. Rundhctl convergefor the base infrastructure, and everything should get back to normal. - [documentation] The in-cluster documentation now includes the guides.
Fixes
- [candi] Restored the lost
kubelet-disable-k-panic-checkpatch for k8s 1.35, so kubelet no longer enforceskernel.panic=10on this version. - [ci] add skip docs validation label
- [ci] Delete tests registry images
- [cloud-provider-azure] use .normal.apiserverEndpoints in data device Secret fallback
- [cloud-provider-vsphere] Fixed master node creation when the datastore is set as a path with parent folders.
- [deckhouse] Restore kubectl plugin lookup in d8 k and d8 kubectl
- [dhctl] Prevent dhctl converge from panicking when kubeconfig is used without an SSH host.
- [docs] Removed mentions of Cilium from the inter-cluster cooperation documentation page.
- [docs] Added experimental module enable note to the lifecycle badge.
- [docs] Rendered all versions from oss.yaml versions array.
- [istio] The Istio operator now recovers Helm releases stuck in the
pending-*oruninstallingstate.
1.76.16
Features
- [docs] Add managed services modules architecture documentation section.
Fixes
- [ci] Fix changelog-generator image tag in changelog-update job to use the shared CI_IMAGES_TAG
- [cloud-provider-vcd] Fixes cluster bootstrap getting stuck when
capcd-controller-managercannot reach the API server and DNS before the CNI is ready. - [istio] Aligned CNI templates with upstream and fixed Istio 1.25 compatibility.
- [istio] Fixed the istio module getting stuck when switching
globalVersionfrom 1.21 while proxies are still connected to the 1.21 control plane. - [istio] Fix leaking istiod control plane after removing an Istio version or disabling the module.
- [istio] Switching back to Istio 1.21 no longer removes the
networking.istio.io/v1API from the Istio CRDs and no longer breaks the istio release.
1.76.15
Know before update
- A StaticInstance whose connectivity check fails now keeps its reservation for the whole bootstrap window instead of returning to the pool on every failed attempt, and the bootstrap (20 min) and cleanup (10 min) timeouts, which previously never fired, are now reachable. If nothing ever ran on the host, the bootstrap timeout returns the instance to the pool after 20 minutes as before. If the host was already bootstrapped in part, the instance returns only through MachineHealthCheck remediation (nodeStartupTimeoutSeconds: 1200) plus the cleanup timeout, so it becomes available again roughly 20 + 10 minutes after the failure, and remediation reboots the host as part of the cleanup.
- The
self-signed-generatorinit container now carries the fixed10m/25Miit was always meant to have, the same as the identical init container ofuser-api, instead of the sum of the main container limits. A DexAuthenticator that set large limits no longer gets that sum as burst: generating the self-signed certificate then takes 3 to 4 seconds instead of under a second. It happens once per pod start and needs no action. - log-shipper
Features
- [docs] Add istio module architecture section.
- [docs] Add user-authn-controller, user-api and basic-auth-proxy components to user-authn module architecture section.
Fixes
- [cert-manager] Restore the
cert-managerandcainjectorpermissions that were lost against the upstream chart, including access toListenerSetresources. - [chrony] Fixed the unit-detection condition in the
disable-ntp-on-node.shnode step. - [cloud-provider-aws] fix GetCapacity not implemented error spam in logs
- [cloud-provider-azure] Fix multimaster bootstrap by falling back to the
d8-masters-kubernetes-data-device-pathSecret when the master data device LUN is not provided via cloud-init. - [cloud-provider-azure] fix GetCapacity not implemented error spam in logs
- [cloud-provider-dvp] fix GetCapacity not implemented error spam in logs
- [cloud-provider-gcp] fix GetCapacity not implemented error spam in logs
- [cloud-provider-vcd] Batch VM cache refresh on cache miss
- [cloud-provider-yandex] fix “Address in use” failures when replacing nodes and when removing external IP addresses
- [deckhouse] In upmeter, the
monitoring-and-autoscaling/alertmanagerprobe is disabled by default and is turned on with the newalertmanagerProbe.enabledsetting. - [deckhouse] Fixed the CVEs reported for the v1.76.14 scan slice across all Observability team module images by bumping grpc to v1.83.2, the OpenTelemetry family to v1.45.0, kin-openapi to v0.149.0, cel-go to v0.29.0, golang.org/x/crypto to v0.55.0 and the rest of the golang.org/x/* set, by moving pushgateway to 1.11.3 and image-availability-exporter to 0.16.0, and by building the log-shipper reloader with Go 1.25.13; advisories without a reachable fix are mitigated via VEX.
- [deckhouse-controller] Stop the module source registry fan-out from replaying and flooding the main queue with duplicate moduleRun tasks.
- [deckhouse-controller] Do not commit the package repository registry checksum when no application could be annotated, which left them all on stale registry settings.
- [descheduler] Fix duplicate enabled key in doc-ru-deschedulers.yaml
- [docs] Fixed FAQ section for ansible inventory.
- [istio] Fix istiod RBAC rules that drifted away from upstream.
- [log-shipper] Fixed an invalid VRL script generated by the
replaceValuetransformation.
log-shipper - [node-manager] Stop caps-controller-manager from rewriting StaticInstance objects in a hot loop when a connectivity check fails, which caused a sustained load on etcd.
A StaticInstance whose connectivity check fails now keeps its reservation for the whole bootstrap window instead of returning to the pool on every failed attempt, and the bootstrap (20 min) and cleanup (10 min) timeouts, which previously never fired, are now reachable. If nothing ever ran on the host, the bootstrap timeout returns the instance to the pool after 20 minutes as before. If the host was already bootstrapped in part, the instance returns only through MachineHealthCheck remediation (nodeStartupTimeoutSeconds: 1200) plus the cleanup timeout, so it becomes available again roughly 20 + 10 minutes after the failure, and remediation reboots the host as part of the cleanup. - [tools] Bump d8-cli to v0.32.6 -
d8 mirror pushrestores the release channel tags of the installer image, sodocker run <repo>/install:<channel>works against a mirrored registry - [tools] Bump d8-cli to v0.32.7 - closes the CVE findings in the bundled d8 binary (grpc 1.83.2, x/image 0.45.0, x/crypto 0.55.0 and x/mod 0.40.0)
- [upmeter] Fixed ListenerSet upmeter namespace.
- [user-authn] Render a valid Deployment for a DexAuthenticator whose
resourceslimits are fractional or below the init container requests.
Theself-signed-generatorinit container now carries the fixed10m/25Miit was always meant to have, the same as the identical init container ofuser-api, instead of the sum of the main container limits. A DexAuthenticator that set large limits no longer gets that sum as burst: generating the self-signed certificate then takes 3 to 4 seconds instead of under a second. It happens once per pod start and needs no action.
1.76.14
Know before update
- D8UserAuthzRenderSlow requires at least three renders averaging above 30 seconds over 30 minutes, D8UserAuthzReleaseSlow at least two releases averaging above 5 minutes over one hour, and both must hold for 15 minutes. Severities are one step lower (6 and 7). Their descriptions now point at CPU shortage on the node rather than at the number of ClusterAuthorizationRules, which does not affect render time.
- During the first
user-authzrelease after the update the per-role bindings are protected from the release engine, the aggregated ones are created, and the per-role bindings are deleted right after the release. Permissions granted through annotated ClusterRoles stay in place throughout. Theuser-authz.deckhouse.io/access-levellabel is now set automatically on annotated ClusterRoles. In audit logs, access granted through annotated ClusterRoles is attributed touser-authz:<level>:custominstead of the individual ClusterRole. - The
service-with-healthcheckscontroller and agent are restarted. AServiceWithHealthchecksthat was left inNotAllEndpointsAreReadywith no EndpointSlice recovers automatically once the updated agent starts, no manual action is needed.
Features
- [docs] Add extended-monitoring module architecture section.
- [docs] Add registry-packages-proxy module architecture section.
Fixes
- [service-with-healthchecks] Fixed a load balancer losing all its endpoints for good after a target pod was replaced.
Theservice-with-healthcheckscontroller and agent are restarted. AServiceWithHealthchecksthat was left inNotAllEndpointsAreReadywith no EndpointSlice recovers automatically once the updated agent starts, no manual action is needed. - [user-authn] Fixed DexAuthenticator pod creation under ResourceQuota by setting init container CPU/memory limits to the sum of main container limits.
- [user-authz] Grant custom ClusterRoles (annotated with
user-authz.deckhouse.io/access-level) through one aggregated ClusterRole per access level, so the number of bindings per ClusterAuthorizationRule/AuthorizationRule no longer depends on the number of such roles.
During the firstuser-authzrelease after the update the per-role bindings are protected from the release engine, the aggregated ones are created, and the per-role bindings are deleted right after the release. Permissions granted through annotated ClusterRoles stay in place throughout. Theuser-authz.deckhouse.io/access-levellabel is now set automatically on annotated ClusterRoles. In audit logs, access granted through annotated ClusterRoles is attributed touser-authz:<level>:custominstead of the individual ClusterRole. - [user-authz] Group the module alerts under a group named apart from the alerts themselves, so the incident shelf accepts them.
- [user-authz] The D8UserAuthzRenderSlow and D8UserAuthzReleaseSlow alerts no longer fire on a single slow render or release, such as a cluster bootstrap under CPU pressure or the one-time migration pass.
D8UserAuthzRenderSlow requires at least three renders averaging above 30 seconds over 30 minutes, D8UserAuthzReleaseSlow at least two releases averaging above 5 minutes over one hour, and both must hold for 15 minutes. Severities are one step lower (6 and 7). Their descriptions now point at CPU shortage on the node rather than at the number of ClusterAuthorizationRules, which does not affect render time.
1.76.13
Know before update
- A Deckhouse release suspended on its release channel is no longer applied by clusters that are behind and reach it through a step-by-step update. The suspend flag lives only in the release-channel image; previously it was dropped when the target release was built from its per-version image, so lagging clusters updated to a suspended release anyway.
- After the update, the controller adds the annotations and labels of every ServiceWithHealthchecks to the Service created for it.
If a ServiceWithHealthchecks of the
LoadBalancertype carries annotations that configure the load balancer (network.deckhouse.io/load-balancer-ips,network.deckhouse.io/load-balancer-shared-ip-keyand similar), the load balancer controller applies them and may assign a different address to the service, which causes a short interruption of the connections. - Endpoints for pods in a terminal phase (Failed/Succeeded) are no longer published. In DVP clusters this prevents traffic from being routed to a VirtualMachine IP that has been reused by another pod. Pods being deleted are now published with the serving and terminating conditions, which enables the graceful shutdown flow for consumers. Pod readiness is derived from the PodReady condition, and stale probe results are reset when a pod becomes not ready, is recreated, or changes its IP.
-
The SSH key and sudo password in
SSHCredentials(used by CAPS to reach static nodes) are now returned as<omitted>to callers withoutgeton thesshcredentials/sensitivesubresource, masked in the audit log, and encrypted in etcd whenapiserver.encryptionEnabledis on.d8:manage:infrastructure:viewerand:managerno longer see these values. CAPS still does, and so does every holder of adeckhouse.iowildcard, since*matches subresources:SuperAdmin, thekubeadm:cluster-adminsgroup, and thedeckhouseandwebhook-handlerSAs ofd8-system.For masked readers the
last-applied-configurationannotation disappears fromget -o yaml, and creating anSSHCredentialswith<omitted>fails with422 Invalid— editing still works. Treat previously exposed keys as leaked and rotate them. Withapiserver.encryptionEnabledon, rewrite existing objects to encrypt them:d8 k get sshcredentials -o json | d8 k replace -f -. - The
deschedulerDeployment ind8-deschedulerrestarts once on upgrade to pick up the new argument. Descheduling behavior is unchanged for workloads that do not implement their own eviction policy. - The
service-with-healthchecksstatus logic was heavily refactored to reduce API and etcd load. If you rely onlastProbeTimeobservability on every probe, explicitly enableverboseStatusin the module configuration. - The installer, the deckhouse ServiceAccount, holders of the cluster administrator kubeconfig and the kube-system / d8-system ServiceAccounts can create a User or a Group named by an existing authorization rule without the acknowledgement annotation. Every other requester is checked as before.
Features
- [descheduler] Always run the descheduler with the upstream
EvictionsInBackgroundfeature gate enabled.
ThedeschedulerDeployment ind8-deschedulerrestarts once on upgrade to pick up the new argument. Descheduling behavior is unchanged for workloads that do not implement their own eviction policy. - [docs] Add managed-valkey module architecture section.
- [docs] Add monitoring-kubernetes module architecture section.
- [docs] Add documentation for Managed PostgreSQL service.
Fixes
- [ci] Add REGCTL_CONFIG
- [ci] Fix regctl in 1.76 build.
- [common] Quote user-controlled values in the prometheus, upmeter, prometheus-pushgateway and extended-monitoring templates to prevent YAML injection into the module release.
- [deckhouse] validate the hardlink target when extracting module and package images
- [deckhouse-controller] Honor a channel-level release suspend for clusters that reach the suspended version through a step-by-step update.
A Deckhouse release suspended on its release channel is no longer applied by clusters that are behind and reach it through a step-by-step update. The suspend flag lives only in the release-channel image; previously it was dropped when the target release was built from its per-version image, so lagging clusters updated to a suspended release anyway. - [deckhouse-controller] Fix tests for check_release_tests
- [docs] Documented the identity collision rules for local Group and User objects and the allow-access-to-kubernetes annotation, and corrected the described scope of a security subsystem manage role.
- [docs] add static cluster OS change for master nodes and Stronghold backup alerts
- [node-manager] Fix NodeCapacity calculation.
-
[node-manager] Hide the CAPS SSH key and sudo password in SSHCredentials from users without the
sshcredentials/sensitivesubresource.
The SSH key and sudo password inSSHCredentials(used by CAPS to reach static nodes) are now returned as<omitted>to callers withoutgeton thesshcredentials/sensitivesubresource, masked in the audit log, and encrypted in etcd whenapiserver.encryptionEnabledis on.d8:manage:infrastructure:viewerand:managerno longer see these values. CAPS still does, and so does every holder of adeckhouse.iowildcard, since*matches subresources:SuperAdmin, thekubeadm:cluster-adminsgroup, and thedeckhouseandwebhook-handlerSAs ofd8-system.For masked readers the
last-applied-configurationannotation disappears fromget -o yaml, and creating anSSHCredentialswith<omitted>fails with422 Invalid— editing still works. Treat previously exposed keys as leaked and rotate them. Withapiserver.encryptionEnabledon, rewrite existing objects to encrypt them:d8 k get sshcredentials -o json | d8 k replace -f -. - [service-with-healthchecks] Fixed an API server overload issue (“status storm”), resolved validation errors for ClusterIP services, corrected pod readiness evaluation logic, and improved code quality.
Theservice-with-healthchecksstatus logic was heavily refactored to reduce API and etcd load. If you rely onlastProbeTimeobservability on every probe, explicitly enableverboseStatusin the module configuration. - [service-with-healthchecks] Annotations and labels of a ServiceWithHealthchecks are now copied to the Service created for it.
After the update, the controller adds the annotations and labels of every ServiceWithHealthchecks to the Service created for it. If a ServiceWithHealthchecks of theLoadBalancertype carries annotations that configure the load balancer (network.deckhouse.io/load-balancer-ips,network.deckhouse.io/load-balancer-shared-ip-keyand similar), the load balancer controller applies them and may assign a different address to the service, which causes a short interruption of the connections. - [service-with-healthchecks] Stopped publishing terminated pods in EndpointSlices and started publishing pods being deleted as terminating endpoints.
Endpoints for pods in a terminal phase (Failed/Succeeded) are no longer published. In DVP clusters this prevents traffic from being routed to a VirtualMachine IP that has been reused by another pod. Pods being deleted are now published with the serving and terminating conditions, which enables the graceful shutdown flow for consumers. Pod readiness is derived from the PodReady condition, and stale probe results are reset when a pod becomes not ready, is recreated, or changes its IP. - [user-authz] The identity collision check no longer applies to the platform’s own identities, so a retried bootstrap creates the initial User again instead of timing out.
The installer, the deckhouse ServiceAccount, holders of the cluster administrator kubeconfig and the kube-system / d8-system ServiceAccounts can create a User or a Group named by an existing authorization rule without the acknowledgement annotation. Every other requester is checked as before.
1.76.12
Know before update
- The
vpa-recommenderpod is restarted and recommendations are recalculated with the new granularity. Memory recommendations will generally become lower (less over-provisioning), CPU recommendations become multiples of 10m. VPA objects withupdateMode: Auto/Recreatemay evict and recreate pods to apply the updated requests. - ValidatingAdmissionPolicy reserved-public-hosts-* is removed. Ingresses that were denied because they matched publicDomainTemplate are allowed again. settings.reservedPublicHosts in ModuleConfig deckhouse remains accepted and is ignored. Enforcement returns in a later release.
Fixes
- [deckhouse] Stop reserving platform public hostnames at admission; keep the ModuleConfig field so existing settings still validate.
ValidatingAdmissionPolicy reserved-public-hosts-* is removed. Ingresses that were denied because they matched publicDomainTemplate are allowed again. settings.reservedPublicHosts in ModuleConfig deckhouse remains accepted and is ignored. Enforcement returns in a later release. - [deckhouse] Fixed goroutine and memory leak in upmeter-agent caused by per-request HTTP clients leaving idle keep-alive connections to Prometheus open forever.
- [deckhouse-controller] Fixed a bug when PackageRepository always used a HTTPS scheme.
- [docs] Add InPlaceOrRecreate to VPA modes list and move Auto to a warning alert.
- [local-path-provisioner] Add wildcard tolerations to the helper pod template so PVC provisioning works on tainted nodes after local-path-provisioner v0.0.32+.
- [user-authn] Commander releases older than 1.18 can grant Kubernetes API access on DexClient again.
- [vertical-pod-autoscaler] Reduce VPA memory recommendation rounding from 64Mi to 16Mi and round CPU recommendations up to 10m.
Thevpa-recommenderpod is restarted and recommendations are recalculated with the new granularity. Memory recommendations will generally become lower (less over-provisioning), CPU recommendations become multiples of 10m. VPA objects withupdateMode: Auto/Recreatemay evict and recreate pods to apply the updated requests.
1.76.11
Know before update
-
Containers attached through the
pods/ephemeralcontainerssubresource were not seen by the module’s webhook, so no policy was evaluated for them: SecurityPolicy, OperationPolicy, Pod Security Standards and image signature verification were all bypassed bykubectl debug. After the update these requests are validated.On clusters with restricted Pod Security Standards,
deny-execor strict SecurityPolicy, typical debug images (root user, writable root filesystem, interactive-it) will now be rejected. Adjust the policies or use a SecurityPolicyException for the namespaces where debugging must stay available.The webhook uses
failurePolicy: Fail, so while Gatekeeper is unavailablekubectl debugwill not work in user namespaces. System namespaces are excluded from the webhook and are not affected. - Creating a
Groupor aUser, or renaming an existing one, so thatGroup.spec.namematches a group subject orUser.spec.emailmatches a user subject of an existing AuthorizationRule or ClusterAuthorizationRule is now rejected, preventing an unintended privilege grant. The email is compared after lowercasing, since that is the form that reaches the token. Set theuser-authz.deckhouse.io/allow-authorization-rule-collisionannotation to"true"on the object if the collision is intentional. Already existing objects keep working and only produce a warning. Any declarative flow that manages both the rule and the identity is affected, not only one that deletes and recreates the identity: a first-time apply denies theGrouporUserwhenever the rule happens to be applied first, and nothing guarantees the order. Add the annotation to such objects before upgrading. Deleting aGroupor aUserwhose name is still a subject of a rule now produces a warning, since the rule keeps granting that name. -
On upgrade, every DexAuthenticator that carries the
dexauthenticator.deckhouse.io/allow-access-to-kubernetesannotation gets a freshly generated client secret, because it previously received the client secret of the privilegedkubernetesOAuth2 client instead. Its pods roll once and users of those applications may have to sign in again. Each rotation is written to the Deckhouse log asRotating DexAuthenticator client secret that reused the shared kubernetes client secret, with the authenticator name and namespace.If that annotation was ever used outside the
d8-*andkube-*namespaces, the shared secret must be considered disclosed to anyone who could read Secrets in those namespaces, and has to be rotated by hand. Deleting the Secret alone does not rotate it: while the value is non-empty in memory the owning hook re-renders the same one. Rotate it as follows.- If a GitOps tool syncs the
d8-user-authnnamespace, excludeSecret/kubernetes-dex-client-app-secretfrom that sync first, otherwise it is restored to the old value. - Blank the stored value:
kubectl -n d8-user-authn patch secret kubernetes-dex-client-app-secret --type merge -p '{"data":{"secret":""}}' - Drop the in-memory copy:
kubectl -n d8-system rollout restart deployment/deckhouse - Check that
kubectl -n d8-user-authn get secret kubernetes-dex-client-app-secret -o jsonpath='{.data.secret}'now differs from the previous value. If it does not, the module re-rendered the old value before the restart took effect; repeat steps 2 and 3.
The blast radius of that rotation is wider than the
kubernetesclient alone. The same value is the client secret of thekubeconfig-generator,kubeconfig-publish-apiand everykubeconfig-<slug>OAuth2 client, and it is passed to basic-auth-proxy as--ldap-client-secret. In-cluster consumers are re-rendered and their pods roll automatically, but kubeconfig files already downloaded from the kubeconfig generator carry the old client secret and can no longer refresh their tokens, so their users must download a new kubeconfig. Already issued ID tokens keep working until they expire, at mostsettings.idTokenTTL.Deleting and recreating a DexAuthenticator is always a safe way to force a fresh per-authenticator secret: the credentials Secret is regenerated with the resource. Its content is owned by Deckhouse, so no GitOps change is needed for the per-authenticator rotation itself.
- If a GitOps tool syncs the
- Only a subject allowed to update the user-authn ModuleConfig, or the Deckhouse service account, may add the
dexclient.deckhouse.io/allow-access-to-kubernetesordexauthenticator.deckhouse.io/allow-access-to-kubernetesannotation. Objects that already carry it keep working and stay editable by their owners as long as they are updated in place. A GitOps controller that deletes and recreates such an object instead of patching it will have the recreation denied, and the object will come back without access to the Kubernetes API unless the controller’s service account is allowed to update the user-authn ModuleConfig. - Project administrators (user-authz Admin access level) can no longer create, update or delete SecurityPolicyException resources; the permission now requires ClusterAdmin. Clusters that granted accessLevel Admin to untrusted tenants over RBAC v1 were exposed to node-level privilege escalation. RBAC v2 use/manage roles were never affected.
-
Setting the annotation no longer requires write access to the user-authn ModuleConfig. Module administrators are unaffected, as their roles already carry the new permission, and so are the platform’s own components in
d8-systemandkube-system. Any other subject that sets the annotation, a cluster provisioner in particular, has to be grantedcreateon the subresource, which for a single namespace is:apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: allow-access-to-kubernetes namespace: <namespace> rules: - apiGroups: ["deckhouse.io"] resources: ["dexclients/allow-access-to-kubernetes", "dexauthenticators/allow-access-to-kubernetes"] verbs: ["create"]bound to the subject with a RoleBinding in the same namespace, or with a ClusterRole and a ClusterRoleBinding to grant it cluster-wide.
- The
d8:manage:permission:subsystem:kubernetes:view_resourcesrole no longer grantsnodes/proxy,nodes/log,nodes/metricsandnodes/stats. Users holdingd8:manage:kubernetes:viewer,d8:manage:infrastructure:vieweror any role aggregating them lose access to the kubelet API through/api/v1/nodes/<node>/proxy/....nodesthemselves stay readable. Automation that relied on reaching the kubelet API with one of these roles must be granted a dedicated role explicitly. spec.oidc.scopesitems are now validated against^\S+$in both served versions of DexProvider. A scope containing whitespace could never work, because OAuth2 passes scopes as a space-delimited list, but it was previously admitted and silently broke authentication. Stored objects are not invalidated, however a DexProvider holding such a value is rejected on its next write until the value is corrected, which also affects a GitOps flow that reapplies the object unchanged.
Features
- [docs] Add previous page and next page buttons to modules documentation
Fixes
-
[admission-policy-engine] Admission policies are now evaluated for ephemeral containers added with
kubectl debug.
Containers attached through thepods/ephemeralcontainerssubresource were not seen by the module’s webhook, so no policy was evaluated for them: SecurityPolicy, OperationPolicy, Pod Security Standards and image signature verification were all bypassed bykubectl debug. After the update these requests are validated.On clusters with restricted Pod Security Standards,
deny-execor strict SecurityPolicy, typical debug images (root user, writable root filesystem, interactive-it) will now be rejected. Adjust the policies or use a SecurityPolicyException for the namespaces where debugging must stay available.The webhook uses
failurePolicy: Fail, so while Gatekeeper is unavailablekubectl debugwill not work in user namespaces. System namespaces are excluded from the webhook and are not affected. - [admission-policy-engine] Restrict SecurityPolicyException write access to ClusterAdmin, closing a tenant-to-node privilege escalation.
Project administrators (user-authz Admin access level) can no longer create, update or delete SecurityPolicyException resources; the permission now requires ClusterAdmin. Clusters that granted accessLevel Admin to untrusted tenants over RBAC v1 were exposed to node-level privilege escalation. RBAC v2 use/manage roles were never affected. - [cloud-provider-yandex] A Node that cannot be resolved to a Yandex Instance no longer blocks target group synchronization for the whole cluster.
- [cloud-provider-zvirt] Prevent capz-controller-manager from crashlooping without diagnostics when zVirt tags cannot be created.
- [deckhouse] bump addon-operator to run the module startup phase unconditionally
- [deckhouse] Deleted module documentation when a module is disabled.
- [deckhouse] The reserved public hosts policies no longer make every aggregated apiserver in the cluster watch all ConfigMaps, which left permission-browser-apiserver permanently unready and stopped namespace deletion cluster-wide.
- [docs] Fix the upmeter web UI Ingress name in SLA documentation
- [testing] Copy Helm render capabilities per render instead of mutating the shared defaults, and allow tests to declare available API kinds.
- [user-authn] Quote DexProvider-derived values rendered into the basic-auth-proxy Deployment and the Dex configuration file to prevent YAML injection.
- [user-authn] Reject whitespace in DexProvider OIDC scopes, which the OAuth2 protocol cannot represent.
spec.oidc.scopesitems are now validated against^\S+$in both served versions of DexProvider. A scope containing whitespace could never work, because OAuth2 passes scopes as a space-delimited list, but it was previously admitted and silently broke authentication. Stored objects are not invalidated, however a DexProvider holding such a value is rejected on its next write until the value is corrected, which also affects a GitOps flow that reapplies the object unchanged. -
[user-authn] Issue a dedicated OAuth2 client secret per DexAuthenticator instead of reusing the shared cluster client secret.
On upgrade, every DexAuthenticator that carries thedexauthenticator.deckhouse.io/allow-access-to-kubernetesannotation gets a freshly generated client secret, because it previously received the client secret of the privilegedkubernetesOAuth2 client instead. Its pods roll once and users of those applications may have to sign in again. Each rotation is written to the Deckhouse log asRotating DexAuthenticator client secret that reused the shared kubernetes client secret, with the authenticator name and namespace.If that annotation was ever used outside the
d8-*andkube-*namespaces, the shared secret must be considered disclosed to anyone who could read Secrets in those namespaces, and has to be rotated by hand. Deleting the Secret alone does not rotate it: while the value is non-empty in memory the owning hook re-renders the same one. Rotate it as follows.- If a GitOps tool syncs the
d8-user-authnnamespace, excludeSecret/kubernetes-dex-client-app-secretfrom that sync first, otherwise it is restored to the old value. - Blank the stored value:
kubectl -n d8-user-authn patch secret kubernetes-dex-client-app-secret --type merge -p '{"data":{"secret":""}}' - Drop the in-memory copy:
kubectl -n d8-system rollout restart deployment/deckhouse - Check that
kubectl -n d8-user-authn get secret kubernetes-dex-client-app-secret -o jsonpath='{.data.secret}'now differs from the previous value. If it does not, the module re-rendered the old value before the restart took effect; repeat steps 2 and 3.
The blast radius of that rotation is wider than the
kubernetesclient alone. The same value is the client secret of thekubeconfig-generator,kubeconfig-publish-apiand everykubeconfig-<slug>OAuth2 client, and it is passed to basic-auth-proxy as--ldap-client-secret. In-cluster consumers are re-rendered and their pods roll automatically, but kubeconfig files already downloaded from the kubeconfig generator carry the old client secret and can no longer refresh their tokens, so their users must download a new kubeconfig. Already issued ID tokens keep working until they expire, at mostsettings.idTokenTTL.Deleting and recreating a DexAuthenticator is always a safe way to force a fresh per-authenticator secret: the credentials Secret is regenerated with the resource. Its content is owned by Deckhouse, so no GitOps change is needed for the per-authenticator rotation itself.
- If a GitOps tool syncs the
- [user-authn] Drop the dead per-DexAuthenticator redirect URI from the privileged kubernetes OAuth2 client.
- [user-authn] Adding the allow-access-to-kubernetes annotation to a DexClient or a DexAuthenticator now requires cluster-level authority over the user-authn module.
Only a subject allowed to update the user-authn ModuleConfig, or the Deckhouse service account, may add thedexclient.deckhouse.io/allow-access-to-kubernetesordexauthenticator.deckhouse.io/allow-access-to-kubernetesannotation. Objects that already carry it keep working and stay editable by their owners as long as they are updated in place. A GitOps controller that deletes and recreates such an object instead of patching it will have the recreation denied, and the object will come back without access to the Kubernetes API unless the controller’s service account is allowed to update the user-authn ModuleConfig. -
[user-authn] The policy gating the allow-access-to-kubernetes annotation no longer applies to platform components, and proves authority with a create permission on the dexclients/allow-access-to-kubernetes and dexauthenticators/allow-access-to-kubernetes subresources instead of write access to the user-authn ModuleConfig.
Setting the annotation no longer requires write access to the user-authn ModuleConfig. Module administrators are unaffected, as their roles already carry the new permission, and so are the platform’s own components ind8-systemandkube-system. Any other subject that sets the annotation, a cluster provisioner in particular, has to be grantedcreateon the subresource, which for a single namespace is:apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: allow-access-to-kubernetes namespace: <namespace> rules: - apiGroups: ["deckhouse.io"] resources: ["dexclients/allow-access-to-kubernetes", "dexauthenticators/allow-access-to-kubernetes"] verbs: ["create"]bound to the subject with a RoleBinding in the same namespace, or with a ClusterRole and a ClusterRoleBinding to grant it cluster-wide.
- [user-authz] Reject local Group names and User emails that collide with a subject already granted privileges by an authorization rule.
Creating aGroupor aUser, or renaming an existing one, so thatGroup.spec.namematches a group subject orUser.spec.emailmatches a user subject of an existing AuthorizationRule or ClusterAuthorizationRule is now rejected, preventing an unintended privilege grant. The email is compared after lowercasing, since that is the form that reaches the token. Set theuser-authz.deckhouse.io/allow-authorization-rule-collisionannotation to"true"on the object if the collision is intentional. Already existing objects keep working and only produce a warning. Any declarative flow that manages both the rule and the identity is affected, not only one that deletes and recreates the identity: a first-time apply denies theGrouporUserwhenever the rule happens to be applied first, and nothing guarantees the order. Add the annotation to such objects before upgrading. Deleting aGroupor aUserwhose name is still a subject of a rule now produces a warning, since the rule keeps granting that name. - [user-authz] Remove node subresources from the RBACv2 viewer permission role
Thed8:manage:permission:subsystem:kubernetes:view_resourcesrole no longer grantsnodes/proxy,nodes/log,nodes/metricsandnodes/stats. Users holdingd8:manage:kubernetes:viewer,d8:manage:infrastructure:vieweror any role aggregating them lose access to the kubelet API through/api/v1/nodes/<node>/proxy/....nodesthemselves stay readable. Automation that relied on reaching the kubelet API with one of these roles must be granted a dedicated role explicitly.
1.76.10
Know before update
- Fixes recreation of all CloudEphemeral nodes on upgrade to 1.76.9. The MachineDeployment replica count was dropped and the MachineDeployment was scaled to zero.
- Values taken from
Project.spec.parametersare now quoted where they are substituted into the shipped project templates, and a parameter that changes the structure of the rendered manifests rather than only their values is refused for any template, including custom ones. An administrator name may no longer contain control characters, andclusterLogDestinationNamemust be a Kubernetes object name or empty. Both are checked from the reconcile loop as well as at admission, so a project already carrying such a value goes into an error state on its next reconcile rather than at its next edit; the same holds for a custom template that turns a parameter into several objects or into YAML, which is refused when the project is created, edited, or reconciled with a change to apply. The module’s admission policy now also matches CREATE, so objects labelledheritage: multitenancy-managercan only be created by the module itself. -
d8:manage:permission:module:multitenancy-manager:editno longer grants write access toprojecttemplates. AProjectTemplaterenders into a release applied by a ServiceAccount bound tocluster-admin, and itsspec.resourcesTemplateis arbitrary, so authoring one is equivalent to holdingcluster-adminand is not a module-level permission.Reading templates stays in
d8:manage:permission:module:multitenancy-manager:view, and creating aProjecton an existing template stays in the edit role. The legacyClusterAdminaccess level and thekubeadm:cluster-adminsgroup keep the right. A subject that authored templates through the manager role has to be granted one of those instead. - network-gateway pod’s will be restarted
Features
- [docs] Add gpu module architecture section.
- [docs] Added cilium-hubble module architecture section.
- [docs] Added adaptive-resource-management module architecture section.
- [docs] Added operator-trivy module architecture section.
- [docs] Added managed-postgres and managed-memcached modules architecture sections.
- [docs] Add pod-reloader module architecture section.
- [docs] Add secrets-store-integration module architecture section.
Fixes
- [cloud-provider-vcd] Fixes an intermittent “network still in use” error during cluster deletion with the WithNAT placement strategy.
- [deckhouse] Fix values validation failing on the injected registry
- [deckhouse-controller] The module-documentation build request timeout to docs-builder is configurable via DOCUMENTATION_BUILD_TIMEOUT (default 120s) instead of a fixed 10s that canceled healthy full-site builds.
- [docs] Fix issues in managed-postgres and managed-memcached modules architecture sections.
- [docs] Clarify which integrity control mechanisms are available in which DKP editions and correct the DKP module integrity description.
- [docs] Link GPU driver install examples to gpu module FAQ
- [docs] Updated the Deckhouse Kubernetes Platform GUI installer links on the installation page from the
#gui-installanchor to the/gs/installer/path. - [docs] Corrected the documentation that incorrectly stated DKP supports only Bearer token authentication for container registries.
- [docs] Module documentation no longer gets stuck in Error under load; docs-builder builds are cancelable and no longer pile up redundant full-site rebuilds behind requests whose clients already timed out.
- [istio] Restore Istiod pod anti-affinity for Istio 1.25 installations managed by the Sail Operator.
- [multitenancy-manager] A project parameter can no longer inject objects into the rendered project manifests.
Values taken fromProject.spec.parametersare now quoted where they are substituted into the shipped project templates, and a parameter that changes the structure of the rendered manifests rather than only their values is refused for any template, including custom ones. An administrator name may no longer contain control characters, andclusterLogDestinationNamemust be a Kubernetes object name or empty. Both are checked from the reconcile loop as well as at admission, so a project already carrying such a value goes into an error state on its next reconcile rather than at its next edit; the same holds for a custom template that turns a parameter into several objects or into YAML, which is refused when the project is created, edited, or reconciled with a change to apply. The module’s admission policy now also matches CREATE, so objects labelledheritage: multitenancy-managercan only be created by the module itself. -
[multitenancy-manager] The module manager role no longer grants authoring of project templates.
d8:manage:permission:module:multitenancy-manager:editno longer grants write access toprojecttemplates. AProjectTemplaterenders into a release applied by a ServiceAccount bound tocluster-admin, and itsspec.resourcesTemplateis arbitrary, so authoring one is equivalent to holdingcluster-adminand is not a module-level permission.Reading templates stays in
d8:manage:permission:module:multitenancy-manager:view, and creating aProjecton an existing template stays in the edit role. The legacyClusterAdminaccess level and thekubeadm:cluster-adminsgroup keep the right. A subject that authored templates through the manager role has to be granted one of those instead. - [network-gateway] Fixed user and group in ConfigMap
network-gateway pod’s will be restarted - [node-manager] Pin MachineDeployment minReadySeconds to 1 so the MachineSet apply converges.
- [node-manager] MachineDeployment
spec.replicasis no longer dropped on upgrade, so cloud nodes are not recreated.
Fixes recreation of all CloudEphemeral nodes on upgrade to 1.76.9. The MachineDeployment replica count was dropped and the MachineDeployment was scaled to zero.
1.76.9
Know before update
- The node-local-dns DaemonSet pods will restart after the update.
Features
- [docs] Add upmeter module architecture section.
- [docs] Add descheduler module architecture section.
- [docs] Add cni-cilium module architecture section.
Fixes
- [cloud-provider-aws] Do not fail the node bootstrap or update when
linux-modules-extrais unavailable for the running kernel. - [deckhouse] Locate CRDs by repository contents.
- [deckhouse] stop false drift detection reinstalling packages with helm hooks
- [deckhouse] An unset
settings.update.modenow defaults toAutoPatchinstead of silently running asAuto. - [deckhouse] Fix a minor Deckhouse release being auto-applied as a patch when no Deployed release object is present.
- [deckhouse] limit the impact of a failed documentation build
- [deckhouse] Surface layer read errors instead of extracting a truncated image
- [deckhouse] Do not inject source registry into a still-embedded module
- [deckhouse-controller] Module releases rendered with nelm no longer raise false absent-resource alerts.
- [deckhouse-controller] nelm now takes over fields left by the old Helm 3 engine, so stale fields get removed on module upgrade.
- [docs] Clarify that DKP does not support version downgrades
- [docs] Add nil-safety guards for OpenAPI extraction to prevent panics.
- [docs] Fixed incorrect description of allowedHostPaths in SecurityPolicy documentation.
- [docs] Fix broken documentation links
- [docs] Clarified AutoPatch update mode and external module update settings.
- [docs] Fixed ModuleUpdatePolicy example in release channel FAQ.
- [documentation] Fix a memory leak in the documentation builder so memory is released between builds; add CPU/memory limits and an opt-in profiling endpoint.
- [istio] Add missing tools to proxyv2 images so the application-aware proxy termination hook works correctly.
- [istio] Fix graceful draining of established HTTP connections when application pods terminate.
- [istio] Create the ConfigMaps required for additional JWKS root CA certificates.
- [keepalived] Fixed Python linking in the keepalived module image.
- [network-gateway] Fixed Python linking in the network-gateway module image.
- [node-local-dns] Bump Go dependencies in the coredns helper image to fix known CVEs.
The node-local-dns DaemonSet pods will restart after the update. - [registry] Fix DKP bootstrap retry by keeping the
registry-initsecret. - [registrypackages] Rebuild kubernetes-cni with updated Go dependencies to fix CVEs.
- [tools] Fixed module release validation incorrectly failing for
!optionaldependencies when the required module doesn’t exist in the cluster. - [user-authn] ClusterAdmin can watch the Dex storage CRDs (passwords, offlinesessionses), not only list them.
1.76.8
Know before update
- All Ingress-nginx controller pods will be restarted.
- The coredns and kube-dns components will restart after the update.
- The egress-gateway-agent component will restart after the update.
- The node-local-dns components will restart after the update.
- The service-with-healthchecks components (controller, agent) will restart after the update.
Features
- [docs] Update node-termination-handler behaviour in cloud-provider-aws module architecture section
Fixes
- [cloud-provider-dvp] skip foreign nodes in cloud controller manager
- [cni-cilium] Bump Go dependencies in the egress-gateway-agent image to fix known CVEs.
The egress-gateway-agent component will restart after the update. - [deckhouse] Skip the multiple CNI check in the enable_cni hook while a CNI migration is in progress to avoid a stuck Deckhouse queue.
- [deckhouse-controller] Re-derive the module release chain and align the from-to gap check with the release updater so intermediate releases are created instead of the chain freezing
- [deckhouse-controller] ModuleDocumentation will not be created for embedded modules.
- [ingress-nginx] Nginx version is updated to 1.30.4.
All Ingress-nginx controller pods will be restarted. - [kube-dns] Bump Go dependencies in the sts-pods-hosts-appender-webhook and coredns images to fix known CVEs.
The coredns and kube-dns components will restart after the update. - [node-local-dns] Bump Go dependencies in the safe-updater and stale-dns-connections-cleaner images to fix known CVEs.
The node-local-dns components will restart after the update. - [service-with-healthchecks] Bump Go dependencies in the service-with-healthchecks image to fix known CVEs.
The service-with-healthchecks components (controller, agent) will restart after the update. - [user-authn] Fix Dex token refresh with upstream providers that rotate refresh tokens (GitLab), which logged users out every
idTokenTTL. - [user-authn] backport fix of 21615
- [user-authn] Accessing kubernetes api via gateway api fixed.
1.76.7
Fixes
- [cloud-provider-huaweicloud] Fixed the Machine initialization contract — new nodes could not join the cluster.
1.76.6
Know before update
- All ingress-nginx controller pods of v1.10 will be restarted.
- The cilium-hubble components (hubble-ui, hubble-relay) will restart after the update.
- The cni-cilium components (cilium agent, operator) will restart after the update.
- The metallb components (controller, speaker, l2lb) will restart after the update.
- Users with accessLevel ClusterAdmin in ClusterAuthorizationRule gain read and write access to DVPInstanceClass objects; previously all access was denied.
- Users with accessLevel ClusterAdmin in ClusterAuthorizationRule gain read and write access to DynamixInstanceClass objects; previously all access was denied.
Fixes
- [candi] Add exception for Oracle Linux UEK kernels for fix CVE-2025-37999 on bashible
- [cilium-hubble] Upgrade hubble-ui backend dependencies (cilium v1.17.16, Go 1.25.0) and switch build base image to fix known CVEs.
The cilium-hubble components (hubble-ui, hubble-relay) will restart after the update. - [cloud-provider-dvp] Fix DVPInstanceClass access for users with user-authz ClusterAdmin role
Users with accessLevel ClusterAdmin in ClusterAuthorizationRule gain read and write access to DVPInstanceClass objects; previously all access was denied. - [cloud-provider-dynamix] Fix DynamixInstanceClass access for users with user-authz ClusterAdmin role
Users with accessLevel ClusterAdmin in ClusterAuthorizationRule gain read and write access to DynamixInstanceClass objects; previously all access was denied. - [cloud-provider-vsphere] fix missing default StorageClass annotation when a DatastoreCluster entry sorts before all Datastore entries
- [cni-cilium] Bump Go dependencies and backport upstream cilium security patches to fix known CVEs.
The cni-cilium components (cilium agent, operator) will restart after the update. - [common] Fix curl in debug-container by shipping a static binary
- [common] Bump Go dependencies in the cni-migration-controller and cni-migration-init-checker images to fix known CVEs.
- [deckhouse] atomically install modules and re-download incomplete versions
- [deckhouse] Restore admin access to list moduleconfigs
- [deckhouse] Fix hugo CVE.
- [deckhouse-controller] Fix maintenance state tracking — module no longer gets stuck in Unmanaged after removing maintenance.
- [deckhouse-controller] Force embedded modules back to the “Embedded” source on startup, healing a stale external source that previously stuck until the Module resource was deleted manually.
- [docs] Added a note about the termination of support for Ubuntu 20.04 and replace the link to the Docker container registry.
- [docs] Added a note about changing the name of the Docker container registry.
- [documentation] Bump Hugo version to 0.163.3
- [ingress-nginx] Controller 1.10 image uid and gid are set to deckhouse.
All ingress-nginx controller pods of v1.10 will be restarted. - [metallb] Bump Go dependencies in the metallb and l2lb images to fix known CVEs.
The metallb components (controller, speaker, l2lb) will restart after the update. - [node-manager] Fix create_rbac_and_certificate_for_kubernetes_api_proxy hook behavior.
- [user-authn] User-authn httproutes are refactored.
1.76.5
Know before update
- With enableMultiTenancy, effective access is now the union of CAR (within its limitNamespaces/namespaceSelector), AuthorizationRules, and plain RoleBindings/ClusterRoleBindings. Previously the webhook denied requests outside the CAR scope even when RBAC explicitly granted them: such existing bindings for subjects with a CAR silently become effective after the upgrade — review them. The CAR access level still does not apply outside its namespace limits. AccessibleNamespaces reflects the same union.
- log-shipper
Fixes
- [cert-manager] Additional letsencrypt-staging cluster issuer for Gateway API is provided.
- [ci] This is a manual cherry-pick the script for the CI fix in the release branch
- [cilium-hubble] Gateway API certificate manifest is updated to support any cluster issuer.
- [cloud-provider-dvp] fix dvp kubernetes dependency mismatch
- [cloud-provider-huaweicloud] Adds patches to the upstream version to make it ignore static nodes
- [deckhouse] Bump addon-operator to fix module install with conversion webhooks
- [deckhouse-controller] Bump addon-operator to v1.21.22 — propagate module context to nelm logs.
- [deckhouse-tools] Gateway API certificate manifest is updated to support any cluster issuer.
- [docs] Fix ingressClass description in global settings
- [docs] Corrected the link from direct to relative in the alert about the module requirements.
- [docs] Corrected the link from direct to relative in the alert about the module requirements for external modules.
- [docs] Add missing sidebar title mappings for Istio documentation CR page.
- [documentation] Gateway API certificate manifest is updated to support any cluster issuer.
- [istio] Gateway API certificate manifest is updated to support any cluster issuer.
- [log-shipper] fix daemonset template
log-shipper - [openvpn] Gateway API certificate manifest is updated to support any cluster issuer.
- [prometheus] Gateway API certificate manifest is updated to support any cluster issuer.
- [registry] Gateway API certificate manifest is updated to support any cluster issuer.
- [registry-packages-proxy] Gateway API certificate manifest is updated to support any cluster issuer.
- [upmeter] Gateway API certificate manifest is updated to support any cluster issuer.
- [user-authn] Gateway API certificate manifest is updated to support any cluster issuer. Kubernetes API HTTProute was updated to return 403 instead of 404.
- [user-authz] Honor CAR-independent RBAC in webhook and permission-browser
With enableMultiTenancy, effective access is now the union of CAR (within its limitNamespaces/namespaceSelector), AuthorizationRules, and plain RoleBindings/ClusterRoleBindings. Previously the webhook denied requests outside the CAR scope even when RBAC explicitly granted them: such existing bindings for subjects with a CAR silently become effective after the upgrade — review them. The CAR access level still does not apply outside its namespace limits. AccessibleNamespaces reflects the same union.
1.76.4
Know before update
- kube-apiserver static pod is reconfigured and restarts on the next control-plane sync.
Features
- [cloud-provider-dvp] Add an option to disable the CCM LoadBalancer functionality via ConfigMap.
- [docs] Add operator-argo module architecture section
- [docs] Add module module architecture section
Fixes
- [candi] kube-apiserver no longer caches watches for
ManifestCheckpointContentChunkresources fromstate-snapshotter.
kube-apiserver static pod is reconfigured and restarts on the next control-plane sync. - [ci] Fix extract images_digests.
- [deckhouse] Fix module metadata (
module.yaml/oss.yaml) missing from in-cluster documentation by allowing these files inIsDocsPath(sender) and creating parent directories on extraction in the docs-builder (receiver). - [deckhouse] Fixes an unauthenticated path traversal (CWE-22) in docs-builder by validating moduleName/channels and enforcing base-directory containment before any filesystem operation.
- [deckhouse] Remove documentation networkPolicy.
- [deckhouse] Add log diagnostic command to ModuleIsNotValid alert description.
- [deckhouse] Use metav1.Duration for ModulePullOverride scanInterval
- [deckhouse-controller] Fix deckhouse-controller crash loop and hooks receiving silently empty snapshots
- [deckhouse-controller] Fixed showing warnings while errors during kubectl edit.
- [deckhouse-controller] Fixed ModuleConfig validation.
- [docs] Document self-service password reset
- [documentation] remove stale fox links
- [user-authn] Preserve Dex Password fields (notably groups) when resetting password, locking or unlocking a user, so users are no longer locked out after these operations.
1.76.3
Know before update
- All ingress-nginx controllers pod will be restarted.
- Previously, a transient cluster DNS failure could cause the user-authz-webhook liveness probe to fail and restart the pod, which combined with the fail-closed authorization webhook (failurePolicy: Deny) could deny all API requests, including cluster-admins, until DNS recovered.
Features
- [deckhouse-controller] harden migrated modules check
- [istio] Custom annotations for Ingress Controller
- [istio] Custom extension providers
- [istio] JWKS extra root CA implementation
Fixes
- [ci] fix unlimited upmeter iterations
- [cloud-provider-vcd] add werf deploy-dependency annotations to capcd webhook configurations to fix DMT lint and prevent race on install/upgrade
- [deckhouse] Fix serialize docs builder builds.
- [dhctl] Preflight checks are no longer re-run on every dhctl-server restart when the cluster config is unchanged.
- [docs] Fix broken-link. Module admission
- [docs] Change security events docs
- [ingress-nginx] The config hashing is fixed.
All ingress-nginx controllers pod will be restarted. - [kube-dns] Added deploy dependencies for webhook configuration
- [node-manager] Creating or re-applying an already-existing StaticInstance no longer fails address validation.
- [user-authz] user-authz-webhook now uses the node-local kube-apiserver endpoint for its discovery cache and liveness check, instead of resolving the “kubernetes.default” DNS name.
Previously, a transient cluster DNS failure could cause the user-authz-webhook liveness probe to fail and restart the pod, which combined with the fail-closed authorization webhook (failurePolicy: Deny) could deny all API requests, including cluster-admins, until DNS recovered.
1.76.2
Features
- [docs] Add registry module architecture section.
- [istio] Add deploy dependencies for Istio webhook configurations
Fixes
- [deckhouse] fix libs in the python-based images
- [dhctl] fix grpc stream cancel deadlock
- [docs] Added the ml/ai tag to the ai-models module on the modules homepage.
- [registry] Missing copy_custom_certificate for Gateway API path was added.
- [registry-packages-proxy] Missing Gateway API objects were added.
- [vertical-pod-autoscaler] Ensure VPA mutating webhook is deployed after its admission controller Deployment and Service are ready.
1.76.1
Know before update
- After upgrading to v1.76.0, kubectl logs and exec fail cluster-wide for all users. Manual workaround is available — see PR description.
- All ingress-nginx controller pods using default controller version will be restarted.
Features
- [deckhouse] Ensure conversion hooks.
- [docs] Add auxiliary components (of virtualization module) architecture page.
- [docs] Update csi-scsi-generic module architecture section
- [docs] Add deep-link anchors and auto-expand logic for collapsible sections.
- [docs] Add page about pod security settings
Fixes
- [candi] Fix containerd credential escaping by rendering username/password into auth string.
- [ci] remove CSE in deploy
- [cloud-provider-dvp] CVE fixes
- [control-plane-manager] fix Helm adoption of kubeadm:cluster-admins CRB on upgrade from pre-v1.76
After upgrading to v1.76.0, kubectl logs and exec fail cluster-wide for all users. Manual workaround is available — see PR description. - [deckhouse] Fix stale application conditions during update over a failed version
- [deckhouse-controller] Add alias for legacy bash completion flag.
- [deckhouse-controller] add werf dependency to webhook
- [docs] Deleted the note about the beta version of the graphical installer on the DKP installation page.
- [docs] Update the wording in the FAQ section
- [ingress-nginx] Default ingress controller version is reverted to 1.10.
All ingress-nginx controller pods using default controller version will be restarted. - [user-authn] Fix blank dex error page caused by missing BuildIdentifier in the error template data struct.
Version 1.75
Important
-
Support for Kubernetes 1.35 has been added, while support for Kubernetes 1.30 has been discontinued. In future DKP releases, support for Kubernetes 1.31 will be removed. The default Kubernetes version (used when the
kubernetesVersionparameter is set toAutomatic) has been changed to 1.33. - Update to DKP 1.75 will be blocked if the cluster uses one of the following deprecated component versions (they are no longer supported):
- IngressNginxController v1.9
- Istio v1.19
-
In DKP 1.75.0–1.75.3, the
PodSecurityStandardsViolationalert may fire immediately after cluster installation. This happens when thed8-systemnamespace is missing theheritage=deckhouselabel, causing Pod Security Standards policies to be mistakenly applied to DKP system pods. The issue is fixed in DKP 1.75.4.For clusters installed on DKP 1.75.0–1.75.3, add the required label manually using the following command:
d8 k --as=system:sudouser label namespace d8-system heritage=deckhouse - Istio version support changes:
- Support for Istio 1.19 has been discontinued.
- Istio 1.21 is now considered deprecated, and support for this version will be discontinued in the upcoming DKP releases. To update Istio, follow the instruction.
-
Starting with Kubernetes 1.35, cluster nodes must use cgroup v2. The previous version (cgroup v1) is considered deprecated. If cgroup v2 is not supported on a node, the D8NodeCgroupV2NotSupported alert will be triggered. See the Kubernetes documentation for cgroup v2 migration details.
-
Starting with Kubernetes 1.36, cluster nodes must use containerd v2 as the container runtime. Support for containerd v1 will be discontinued. If containerd v2 is not supported on a node, the D8NodeContainerdV2NotSupported alert will be triggered.
-
The
vertical-pod-autoscalermodule has been updated from version 1.4.1 to 1.5.1. Starting from version 1.5.0, theupdateMode: Automode is considered deprecated. Users are advised to replace it withupdateMode: InPlaceOrRecreate. See the vertical scaling documentation for details. -
When issuing new certificates using the
cert-managermodule, the certificate’s private key is now re-generated as well. This change mitigates potential security risks. -
Pods handling HTTP-01 challenges via Issuer and ClusterIssuer resources now have default resource requests defined. As a result, these pods consume part of the cluster and project resource quotas upon creation.
- The operator-trivy module is now delivered from an external source and follows an independent release cycle. No manual migration steps are required.
Deckhouse subsystem
-
The ModuleSource resource now includes the
scanIntervalparameter to control how frequently the container registry is checked for new modules and versions. -
A dedicated node mode with the
node.deckhouse.io/etcd-arbiterlabel has been introduced. In this mode, only etcd runs on the node (no control plane components). This enables a highly available cluster with two master nodes plus a dedicated etcd node, reducing resource requirements compared to the traditional three full master nodes configuration. See the High Availability documentation for details. -
New operating modes have been introduced in the
registrymodule: Proxy (with image caching) and Local (for isolated environments). These modes are currently supported only for already deployed static clusters and are not yet available during DKP cluster installation. -
Container registry access parameters can now be configured via the deckhouse module during DKP cluster installation. Support for the DKP cluster deployment via Direct and Unmanaged modes has been added. The Legacy Unmanaged mode (via InitConfiguration) is preserved for compatibility. If no explicit configuration is provided, the mode is selected automatically.
-
Added user shell validation: if a shell other than bash is used, the
dhctlutility now displays a clear error message. -
Added StaticInstance validation before DKP installation using the corresponding SSHCredentials, preventing
dhctlfrom reporting readiness before the cluster deployment is actually complete. -
Extended preflight test coverage to improve cluster deployment predictability.
-
Partially reduced usage of netcat in bashible, lowering the number of security system triggers.
-
Fixed an issue where cluster installation could not be resumed after interruption.
-
Fixed Node object deletion logic during
converge. The node is now properly removed from the cluster after the corresponding VM is destroyed. -
Fixed the
--skip-resourcesflag behavior indhctl destroycommand. Specified resources are now correctly skipped.
Kubernetes & Scheduling subsystem
-
A new
update-observercomponent in thecontrol-plane-managermodule enables real-time tracking of control plane component updates (kube-apiserver, kube-scheduler, kube-controller-manager) and kubelet on nodes. This helps detect discrepancies and eliminates “blind spots” during Kubernetes updates. -
For all VPA resources managed by DKP,
updateMode: Autohas been automatically replaced withupdateMode: InPlaceOrRecreate. For clusters running Kubernetes 1.32 or earlier, theInPlacePodVerticalScalingfeature gate is automatically enabled in control plane components (kube-apiserver, kube-scheduler, kube-controller-manager) and kubelet. This allows pod resource changes without recreation. -
The descheduler module has been updated to version 0.34. Key changes:
- Priority-based pod eviction: pods with a higher restart count are evicted first, improving cluster stability during load redistribution.
- DRA support: proper handling of resources managed via Dynamic Resource Allocation.
- Default
cpu,memory, andpodsthresholds removed fromlowNodeUtilizationandhighNodeUtilizationstrategies to align with upstream behavior.
IAM subsystem
-
Namespace listing behavior has been changed. When requesting all cluster namespaces, users now see only the namespaces they have access to, even without permission to list all namespaces. This improves working with multitenancy and UI.
- The ProjectTemplate resource in the
multitenancy-managermodule now supports the unmanaged and skip-heritage options (set via annotations), allowing administrators to adjust the DKP control degree over resources:unmanaged: The module does not manage the lifecycle of objects.skip-heritage: DKP retains management but allows modification by external controllers.
-
Added support for authenticating to the Kubernetes API via LDAP using Basic authentication instead of OIDC tokens (via the
ldap.enableBasicAuthparameter in the DexProvider resource). -
The authorization webhook in the
user-authzmodule now operates in fail-closed mode: if it’s unavailable, authorization is denied. For improved reliability, webhook communication with the apiserver is performed via the cluster node network. -
The
user-authnmodule now supports a global user session lifetime limit (viarefreshTokenAbsoluteLifetime). Re-authentication is required after expiration. -
The
user-authnmodule now supports Kerberos (SPNEGO) authentication in DKP. -
Updated kube-apiserver TLS cipher suites. Removed the following insecure ciphers from the
--tls-cipher-suitesflag: TLS_RSA_WITH_AES_256_GCM_SHA384 and TLS_RSA_WITH_AES_128_GCM_SHA256 (considered unsafe due to RSA key exchange). Added TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 and TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. - The anonymous access restriction mechanism for the API server has been modified.
Instead of using the
--anonymous-auth=falseflag and a sidecar proxy, the AuthenticationConfiguration resource is now used to allow anonymous access only to/livez,/readyz, and/healthz. This improves cluster resource efficiency. Anonymous probe access remains restricted when public API access is enabled viapublishAPI.enabled.
Security subsystem
-
Fixed serialization of empty arrays and objects in SecurityPolicy and OperationPolicy resources of the
admission-policy-enginemodule. Previously, some settings could be applied incorrectly. -
Revised ModSecurity extension implementation. OWASP Core Rule Set (CRS) functionality is now available in all supported IngressNginxController versions.
Cluster & Infrastructure subsystem
- Lifecycle stages updated for cloud provider integration modules:
cloud-provider-dvp(DVP),cloud-provider-vcd(VMware Cloud Director),cloud-provider-huaweicloud(Huawei Cloud), andcloud-provider-zvirt(zVirt) modules are now General Availability (previously Experimental).
-
Added support for swap memory usage by pods. Instead of immediate OOM-kill, a pod can temporarily use swap (if configured and available on the node) and continue running after peak usage. Requires cgroup v2 and sufficient node resources. For details, see the
kubelet.memorySwapparameter in the NodeGroup resource documentation. - In the DVP cloud provider:
- Added the
networkPolicyparameter to define network traffic rules for workloads within a Project resource. - Added
managed-by,cluster-uuid, andvm_namelabels to all infrastructure objects. - Improved CSI driver diagnostics. Diagnostic messages are now displayed directly in PersistentVolumeClaim status.
- Implemented automatic cleanup of VMBDA objects when a VM is deleted.
- Fixed an issue where DeckhouseMachine objects could remain in deleting state after VM removal.
- Fixed an issue where Cloud Controller Manager (CCM) could recreate external load balancers after the associated Kubernetes Service was deleted.
- Added the
-
In Yandex Cloud, fixed a false NAT warning displayed during installation in configurations where NAT is not used.
- In the
node-managermodule:- Cluster API (CAPI) has been updated from 1.10.6 to 1.11.3, improving stability and provider compatibility.
- Added a dedicated controller for NodeGroup hooks (extracted from DKP), reducing load on master nodes and improving responsiveness and UI feedback.
- Added per-GPU MIG configuration support. You can now define a distinct set of MIG partitions for each physical GPU on a node. Previously, MIG configuration was applied at the node level and identical across all GPUs. This enables more flexible workload resource allocation on a single node.
Network subsystem
-
In IngressNginxController, the
acceptClientIPHeadersFromparameter has been added for the following inlets: HostPort, HostPortWithProxyProtocol, LoadBalancer, and LoadBalancerWithProxyProtocol. The parameter defines trusted CIDR ranges forX-Forwarded-*headers that should not be overwritten during proxying. - In the
cni-ciliummodule, network observability capabilities have been extended:- Added extended metrics for monitoring interactions between specific objects (Pods, Services, Namespaces),
configurable via the
extendedMetricsparameter of the HubbleMonitoringConfig resource. - Implemented export of traffic flow logs for analyzing network interactions,
configurable via the
flowLogsparameter of the HubbleMonitoringConfig resource.
- Added extended metrics for monitoring interactions between specific objects (Pods, Services, Namespaces),
configurable via the
-
Fixed an issue where, after VM migration in DVP, a LoadBalancer IP address could stop being announced when using MetalLB in BGP mode.
- IngressNginxController version support changes:
- Support for v1.9 has been discontinued.
- Support for v1.14 has been added.
The complete list of changes, including the updated components, is available in the changelog on GitHub.
1.75.16
Fixes
- [istio] Add missing tools to proxyv2 images so the application-aware proxy termination hook works correctly.
- [istio] Fix graceful draining of established HTTP connections when application pods terminate.
1.75.15
Know before update
- The cilium-hubble components (hubble-ui, hubble-relay) will restart after the update.
- The cni-cilium components (cilium agent, operator) will restart after the update.
- The coredns and kube-dns components will restart after the update.
- The egress-gateway-agent component will restart after the update.
- The metallb components (controller, speaker, l2lb) will restart after the update.
- The node-local-dns DaemonSet pods will restart after the update.
- The node-local-dns components will restart after the update.
- The service-with-healthchecks components (controller, agent) will restart after the update.
Fixes
- [cilium-hubble] Upgrade hubble-ui backend dependencies (cilium v1.17.16, Go 1.25.0) and switch build base image to fix known CVEs.
The cilium-hubble components (hubble-ui, hubble-relay) will restart after the update. - [cni-cilium] Bump Go dependencies in the egress-gateway-agent image to fix known CVEs.
The egress-gateway-agent component will restart after the update. - [cni-cilium] Bump Go dependencies and backport upstream cilium security patches to fix known CVEs.
The cni-cilium components (cilium agent, operator) will restart after the update. - [deckhouse] An unset
settings.update.modenow defaults toAutoPatchinstead of silently running asAuto. - [deckhouse] Fix a minor Deckhouse release being auto-applied as a patch when no Deployed release object is present.
- [deckhouse] Surface layer read errors instead of extracting a truncated image
- [kube-dns] Bump Go dependencies in the sts-pods-hosts-appender-webhook and coredns images to fix known CVEs.
The coredns and kube-dns components will restart after the update. - [metallb] Bump Go dependencies in the metallb and l2lb images to fix known CVEs.
The metallb components (controller, speaker, l2lb) will restart after the update. - [node-local-dns] Bump Go dependencies in the safe-updater and stale-dns-connections-cleaner images to fix known CVEs.
The node-local-dns components will restart after the update. - [node-local-dns] Bump Go dependencies in the coredns helper image to fix known CVEs.
The node-local-dns DaemonSet pods will restart after the update. - [service-with-healthchecks] Bump Go dependencies in the service-with-healthchecks image to fix known CVEs.
The service-with-healthchecks components (controller, agent) will restart after the update. - [tools] Fixed module release validation incorrectly failing for
!optionaldependencies when the required module doesn’t exist in the cluster.
1.75.14
Fixes
- [common] Fix curl in debug-container by shipping a static binary
- [deckhouse] atomically install modules and re-download incomplete versions
- [deckhouse] show the allow-disabling annotation hint when disabling a confirmation-required module
- [deckhouse] Fix CVEs.
- [deckhouse-controller] Fix maintenance state tracking — module no longer gets stuck in Unmanaged after removing maintenance.
- [docs] Corrected the link from direct to relative in the alert about the module requirements.
1.75.13
Fixes
- [deckhouse] Fix nil pointer panic in resource and namespace informers by using each informer’s own logger instead of the possibly-nil Monitor.Logger.
- [user-authn] Preserve Dex Password fields (notably groups) when resetting password, locking or unlocking a user, so users are no longer locked out after these operations.
1.75.12
Know before update
- Previously, a transient cluster DNS failure could cause the user-authz-webhook liveness probe to fail and restart the pod, which combined with the fail-closed authorization webhook (failurePolicy: Deny) could deny all API requests, including cluster-admins, until DNS recovered.
Fixes
- [candi] fix if node has bashible-uninitialized taint in race condition.
- [deckhouse] add cryptography CVE VEX to deckhouse-controller
- [deckhouse] Fix module metadata (
module.yaml/oss.yaml) missing from in-cluster documentation by allowing these files inIsDocsPath(sender) and creating parent directories on extraction in the docs-builder (receiver). - [deckhouse-controller] Don’t create an external module release for a module that is still shipped embedded, so it can’t replace or duplicate the embedded copy.
- [deckhouse-controller] Disabling modules with confirmation will be rejected without using the annotation.
- [docs] Added the ml/ai tag to the ai-models module on the modules homepage.
- [docs] Fix broken-link. Module admission
- [documentation] Add /app/hugo to docs-builder mount points
- [user-authz] user-authz-webhook now uses the node-local kube-apiserver endpoint for its discovery cache and liveness check, instead of resolving the “kubernetes.default” DNS name.
Previously, a transient cluster DNS failure could cause the user-authz-webhook liveness probe to fail and restart the pod, which combined with the fail-closed authorization webhook (failurePolicy: Deny) could deny all API requests, including cluster-admins, until DNS recovered.
1.75.11
Know before update
- All ingress-nginx pods will be restarted.
Fixes
- [common] Fixed CVE-2026-40898 in CoreDNS by updating the quic-go dependency.
- [ingress-nginx] Nginx is updated to 1.30.3.
All ingress-nginx pods will be restarted.
1.75.10
Features
- [docs] Add csi-ceph module architecture section.
- [docs] Add Core component (of virtualization module) architecture section.
- [documentation] Update editions availability info for stronghold module.
Fixes
- [candi] fix cve node-manager and opentofu.
- [cilium-hubble] Fixed CVE-2026-29181 in hubble-ui-backend by bumping OpenTelemetry Go to v1.41.0
- [cilium-hubble] Fixed CVE-2026-41520 in hubble-ui-backend
- [cni-cilium] Fixed CVE-2026-41520 for cilium-bugtool util
- [cni-flannel] Reverted module stage from Deprecated back to General Availability to stop false deprecation alerts.
- [common] fixed CVE-2026-29181 in the CoreDNS
- [deckhouse-controller] A module that conditionally depends on another is no longer disabled when an incompatible version of that dependency is enabled; the enable is rejected instead.
- [deckhouse-controller] fix module config validation for disable warnings
- [docs] Add a general description of container registers DKP
- [docs] Updated the etcd restore documentation.
- [docs] Fix CRD version tabs markup and nesting.
- [istio] Implement graceful metadata secret renewal for multiclusters.
- [network-policy-engine] Reverted module stage from Deprecated back to General Availability to stop false deprecation alerts.
- [node-manager] hook to restore apiVersion on CAPI resources.
- [node-manager] Fix static NodeGroup readiness calculation.
- [node-manager] add rbac policies for persistantvolumes to manage from capi-controller-manager.
- [tools] fix d8 build
1.75.9
Know before update
- All Ingress-nginx controller pods will be restarted.
Fixes
- [admission-policy-engine] Refactor main documentation page
- [ingress-nginx] Nginx was updated to 1.30.2.
All Ingress-nginx controller pods will be restarted.
1.75.8
Know before update
- Registry pods will be restarted.
Features
- [docs] Update deckhouse web modules architecture section.
Fixes
- [csi-vsphere] Fixed the Deckhouse queue getting stuck
- [docs] Change network-policy-engine and cni-flannel to deprecated.
- [monitoring-deckhouse] Fix module-release alerts (
ModuleReleaseIsWaitingManualApproval,ModuleReleaseIsOutdated,ModuleReleaseIsBlockedByRequirements,ModuleIsInMaintenanceMode,D8ModuleOutdatedByMajorVersion) that never fired on 1.75.x clusters due to amoduleName→modulelabel mismatch introduced when addon-operator was bumped to v1.19.7 in v1.75.1. - [registry] Updated auth image Go dependencies to fix Go CVEs.
Registry pods will be restarted. - [user-authn] Improve basic-auth-proxy request handling, cache implementation, and shutdown behavior.
1.75.7
Know before update
- All Ingress-nginx controller pods will be restarted.
Features
- [docs] Add Deckhouse web modules architecture section.
- [docs] Add cert-manager module architecture section.
Fixes
- [cloud-provider-vsphere] normalizes new paths and makes bashible resolve existing paths case-insensitively
- [cni-cilium] Fixed CVE-2026-35469
- [deckhouse] Fix exp modules auto enabling.
- [deckhouse] Fixed global configuration generation.
- [dhctl] Wait for stronghold cluster sync before node deletion
- [docs] Refactored web UI page.
- [docs] New screenshots have been added to the Console documentation.
- [ingress-nginx] Nginx is updated up to 1.30.1.
All Ingress-nginx controller pods will be restarted.
1.75.6
Fixes
- [admission-policy-engine] Fixed enforcementAction for D8ReplicaLimits, added more template tests.
- [dhctl] fix SSH preflight check for StaticInstances with password-only auth.
- [docs] Fix broken links in VPA section.
- [docs] Fix visibility scope for nested details component
- [docs] Add info about kernel requirement for containerdv2 migration.
- [node-manager] CAPI crd served version fix
- [node-manager] Added cleanup for oversized MCM MachineSet revision history annotation
1.75.5
Know before update
- module pods will be restarted
Features
- [docs] Add a Jekyll plugin for tabbed content in the documentation.
- [istio] Allow custom ports in metadataEndpoint URLs for IstioFederation and IstioMulticluster CRDs.
Fixes
- [candi] Added a Netplan override to force the secondary NIC to use the main routing table, fixing cloud-init PBR conflicts.
- [common] Fixed an issue where Trivy could not detect VEX files in coredns images.
- [deckhouse] Overwrite currentReleaseImageName on mismatch.
- [deckhouse] Allow updating scanInterval on the deckhouse ModuleSource.
- [istio] fixed CVEs in module images
module pods will be restarted - [service-with-healthchecks] Correct RBACv2 is provided.
- [terraform-manager] Fix opentofu patches build after pull request 19080.
1.75.4
Know before update
- All Ingress-NGINX controller pods will be restated.
- All pods of Ingress-NGINX controller of 1.10 and 1.12 versions will be restarted.
- Cilium agents will be restarted during the update.
- HPA scaler pods will be restarted.
- Kruise controller will be restarted.
- These changes prevent unnecessary or destructive plan updates that could occur when data sources depend on changing labels and annotations. The behavior of other cloud providers is not affected. If you encounter unexpected converge plans or cluster bootstrap issues when using OpenTofu-based providers (such as dvp, dynamics, zvirt, or yandex), please report them to Deckhouse Technical Support.
- This update triggers a rolling update of the flannel pods.
- This update triggers a rolling update of the kube-proxy pods.
- This update triggers a rolling update of the network-policy-engine pods.
- istio module pods will be restarted
- istio-cni-nodes will be restarted
Features
- [docs] Respect nowrap default for code blocks.
- [docs] Update sds-node-configuration module architecture section.
- [docs] Add geoproxy and failover to ingress-nginx module architecture section
- [docs] Add csi-nfs module architecture section.
- [docs] Add public cloud providers aws, azure, and gcp module architecture section.
- [docs] Add cloud-provider-vsphere module architecture section.
- [docs] Add cloud-provider-openstack module architecture section.
- [terraform-manager] Suppress destructive changes when add/remove/change labels and annotations for cloud resources via opentofu.
These changes prevent unnecessary or destructive plan updates that could occur when data sources depend on changing labels and annotations. The behavior of other cloud providers is not affected. If you encounter unexpected converge plans or cluster bootstrap issues when using OpenTofu-based providers (such as dvp, dynamics, zvirt, or yandex), please report them to Deckhouse Technical Support.
Fixes
- [admission-policy-engine] Fix high resource consumption for constraint d8denyexecheritage
- [candi] Fix containerd unmount for migration if /var/lib/containerd is mounted manually.
- [cert-manager] Disable SecurityPolicyExceptions for cert-manager namespace
- [cloud-provider-aws] add information about AWS security group rules limits
- [cloud-provider-dvp] Suppress destructive changes when add/remove/change labels and annotations for cloud resources via opentofu.
- [cloud-provider-vcd] fix vCD CCM TCP health monitors removal
- [cni-cilium] Updated go-jose dependency to v4.1.4 to fix CVE-2026-34986.
Cilium agents will be restarted during the update. - [cni-flannel] Fixed CVE-2026-33186 by updating google.golang.org/grpc in flanneld.
This update triggers a rolling update of the flannel pods. - [cni-simple-bridge] Refactored python image source and pip exclusion.
- [common] Removed Python completely from the debug-container image as it is no longer needed, resolving corresponding CVEs, and silenced false positives for etcd binaries via VEX.
- [deckhouse] Return setting notified for mr.
- [deckhouse] Remove notified=false annotation reset from runReleaseDeploy in the module release controller.
- [deckhouse] Ensure heritage label on d8-system namespace via hook.
- [deckhouse-controller] Add new discoveryv1 scheme to deckhouse controller.
- [deckhouse-controller] added extra validation for kubernets version multiple downgrades scenario
- [dhctl] Use internal node ip if bastion ip was passed in converge.
- [docs] Fix indented Liquid blocks in lists.
- [docs] Fix broken links.
- [docs] Improved title generation for module documentation pages.
- [ingress-nginx] CVE-2025-15566 is fixed in 1.10 and 1.12 controllers.
All pods of Ingress-NGINX controller of 1.10 and 1.12 versions will be restarted. - [ingress-nginx] Added missing pod security context settings to hpa scaler pods.
HPA scaler pods will be restarted. - [ingress-nginx] Kruise go dependencies are updated to mitigate latest CVEs.
Kruise controller will be restarted. - [ingress-nginx] Added fix for CVE-2026-4342.
All Ingress-NGINX controller pods will be restated. - [istio] fixed CVE-2026-34986
istio module pods will be restarted - [istio] added iptables wrapper in cni-v1x21x6
istio-cni-nodes will be restarted - [keepalived] Excluded vulnerable pip-25.3 from keepalived final image to fix CVE-2026-1703
- [kube-dns] Refactored hook remove_old_coredns_deploy_and_svc — delegated deletion of original kube-dns service to helm.
- [kube-proxy] Fixed CVE-2026-33186 and CVE-2026-24051 in kube-proxy dependencies.
This update triggers a rolling update of the kube-proxy pods. - [network-gateway] Updated python image source and mitigated pip CVE-2026-1703
- [network-policy-engine] Fixed CVE-2026-34040, CVE-2026-33997, and CVE-2026-33186 in network-policy-engine dependencies.
This update triggers a rolling update of the network-policy-engine pods. - [node-manager] rollback inject capi conversion hook in crds as on-before-helm
- [node-manager] optimize node-controller cache resources usage.
- [registrypackages] Add vex for CVE-2026-39882
- [registrypackages] Remediated CVE-2026-32285 in crictl dependencies.
1.75.3
Know before update
- CoreDNS pods will undergo a rolling restart.
Features
- [docs] Add cloud-provider-yandex module architecture section.
Fixes
- [common] Fixed CVE-2026-33186 in the CoreDNS image.
CoreDNS pods will undergo a rolling restart. - [deckhouse-controller] revert nelm to helm
- [dhctl] Skip tmp lock for exporter and auto-converger.
- [node-local-dns] Fix werf manifest
- [node-local-dns] Adapt node-local-dns for air-gapped environments.
- [node-manager] deploy capi controller and webhooks before basic resources to prevent race condition during upgrades.
- [registrypackages] Added vex with CVE-2026-33186.
1.75.2
Know before update
- All Ingress-NGINX controller pods will be restarted.
Features
- [cert-manager] Add alerts for ACME Challenges stuck in pending or error states
- [docs] Add cloud providers dynamix, huaweicloud, and zvirt module architecture section.
Fixes
- [ci] Use GITHUB_TOKEN for backport action and set required workflow permissions
- [cilium-hubble] Fixed CVE-2026-33186 in the hubble-ui image.
- [cni-cilium] Fixed CVE-2026-33186, CVE-2026-27142, and CVE-2026-27139 by updating grpc dependency and Go version, and resolved build compatibility issues.
- [common] Fixed CVE-2026-24051 in the CoreDNS image.
- [dhctl] mitigate CVE-2026-33186
- [ingress-nginx] Nelm fixes are backported.
All Ingress-NGINX controller pods will be restarted.
1.75.1
Know before update
- Registry pods will be restarted.
- ingressgateway, istio-cni-node, istiod, kiali, operator pods will be restarted after update
Features
- [cloud-provider-zvirt] add customNetworkConfig
- [cloud-provider-zvirt] add customNetworkConfig
- [loki] Added generation and usage of a dedicated TLS server certificate for Loki kube-rbac-proxy
Fixes
- [admission-policy-engine] Fix SecurityPolicyException handling for hostPorts-only exceptions
- [cloud-provider-aws] fix CVE in cloud-provider-aws
- [cloud-provider-azure] fix CVEs in cloud-provider-azure
- [cloud-provider-dvp] fix CVEs in cloud-provider-dvp
- [cloud-provider-dvp] fix CVEs in cloud-provider-dvp
- [cloud-provider-dvp] refactored CreateVolume to improve idempotency when disk.status.capacity is not yet reported and standardized gRPC error handling
- [cloud-provider-huaweicloud] fix CVEs in cloud-provider-huaweicloud
- [cloud-provider-huaweicloud] fix CVEs in cloud-provider-huaweicloud
- [cloud-provider-openstack] fix CVE in cloud-provider-openstack module
- [cloud-provider-yandex] fix CVEs in cloud-provider-yandex
- [cloud-provider-zvirt] fix CVEs in cloud-provider-zvirt
- [cloud-provider-zvirt] docs fox
- [cni-cilium] fixed mount-points in agent
- [deckhouse] Bump nelm version with deadlock fix.
- [dhctl] Fix restart bootstrap during creating additional nodes in cloud permanent node groups.
- [dhctl] Аix non-strict unmarshalling for metaconfigs.
- [dhctl] Fix deadlock in converge.
- [docs] Added the display of all used component versions in modules on the OSS info page.
- [docs] Add support for highlighting unknown code block types.
- [docs] Fix alerts rendering.
- [docs] Moving the platform update section to the top in the sidebar.
- [docs] Changing the term metka to label in Russian-language documentation.
- [docs] Escape HTML in OpenAPI descriptions v2
- [docs] Added info about using update-observer according to
- [extended-monitoring] fix typo in image-availability-exporter template
- [istio] cve fixes
ingressgateway, istio-cni-node, istiod, kiali, operator pods will be restarted after update - [registry] fixed path in images
- [registry] Updated auth image Go dependencies to fix Go CVEs.
Registry pods will be restarted. - [registry-packages-proxy] fix possible deadlock in cache retention policy
- [tools] Added HTTP client proxy support
- [tools] Bump lib-helm version to 1.71.2
Version 1.74
Important
- Support for Kubernetes 1.34 has been added, while support for Kubernetes 1.29 has been discontinued.
In future DKP releases, support for Kubernetes 1.30 will be removed.
The default Kubernetes version (used when the
kubernetesVersionparameter is set toAutomatic) has been changed to 1.32.
Major changes
-
Added support for managing experimental and alpha Kubernetes features (feature gates) using the
enabledFeatureGatesparameter of thecontrol-plane-managermodule. For details on managing these features in DKP, refer to the module documentation. -
The
--terminated-pod-gc-thresholdparameter (the termination threshold for evicted pods) is now calculated automatically based on the number of nodes in the cluster. This provides a balance between cluster stability and the ability to perform diagnostics. Threshold values are available in thecontrol-plane-managermodule documentation. -
The maximum number of pods (
maxPods) for a node group is now set automatically based on thepodSubnetNodeCIDRPrefixparameter from ClusterConfiguration (unless explicitly specified). This ensures stable cluster operation. -
Added the
vipAddressparameter to the HuaweiCloudInstanceClass resource of the Huawei Cloud provider, allowing you to specify a virtual IP address for all nodes in the instance class. -
The Huawei Cloud provider configuration now allows overriding the primary network for the main network interface (
mainNetwork) and specifying subnets for additional network interfaces (additionalNetworks) on both CloudPermanent and CloudEphemeral nodes.Warning. When upgrading to DKP 1.74 on Huawei Cloud, a bulk recreation of CloudEphemeral nodes may occur. Due to a temporary node shortage, some Pods may fail to start.
-
The archive with debugging data is now collected using the
d8tool. If you need to collect diagnostic data, follow the instructions. -
Now when hook execution fails, all metrics collected by that hook are preserved. This prevents loss of debugging information and simplifies the following root-cause analysis.
-
Fixed a Deckhouse queue freeze in hybrid clusters that occurred when module configuration parameters were incomplete.
Security
- Implemented module integrity control based on the EROFS filesystem to improve security and prevent unauthorized modifications. The new system requires kernel support for dm-verity and EROFS.
Network
-
Added support for the SCTP (Stream Control Transmission Protocol). Applications using this protocol can now run in a Deckhouse cluster.
- Added a
geoproxyauxiliary microservice to theingress-nginxmodule. This service is aimed at improving the stability of the Ingress NGINX Controller when working with GeoIP databases and provides the following features:- MaxMind license saving (databases are downloaded from a single point once a day).
- Persistent data storage (if components are restarted, it doesn’t require accessing the MaxMind servers).
- Lets you specify a custom mirror for downloading databases.
-
Added the Prometheus metric
bpf_progs_complexity_max_verified_inststo assess the number of instructions in eBPF programs loaded into the kernel on cluster nodes. This metric helps evaluate the compatibility of the networking subsystem with the node kernels. The metric requires Linux kernel version 5.16 or newer. -
Added a new metric (
geoip_version) and dashboards for monitoring GeoIP functionality in the cluster. - Fixed issues with simultaneous updates of the
node-local-dnsmodule and the networking subsystem.
The complete list of changes, including the updated components, is available in the changelog on GitHub.
1.74.18
Fixes
- [deckhouse-controller] Don’t create an external module release for a module that is still shipped embedded, so it can’t replace or duplicate the embedded copy.
- [deckhouse-controller] A module that conditionally depends on another is no longer disabled when an incompatible version of that dependency is enabled; the enable is rejected instead.
- [istio] Implement graceful metadata secret renewal for multiclusters.
- [network-gateway] Updated dnsmasq to v2.92-alt2 to address multiple security vulnerabilities (CVE-2026-*)
- [node-local-dns] Fix name of registry secret in safe-updater deployment
1.74.17
Fixes
- [node-manager] Added cleanup for oversized MCM MachineSet revision history annotation
1.74.16
Know before update
- All Ingress-NGINX controller pods will be restarted.
- All pods of Ingress-NGINX controller will be restarted.
- This update triggers a rolling update of the kube-proxy pods.
- istio module pods will be restarted
- module pods will be restarted
Features
- [docs] Image zooming and mobile view.
- [docs] Add sds-node-configurator module architecture section.
- [docs] Add cloud-provider-vcd module architecture section.
Fixes
- [admission-policy-engine] cve fixes for ratify
- [cloud-provider-azure] fix disk discovery for Ubuntu 22.04 Gen2 VMs with NVMe controllers
- [cloud-provider-openstack] Increase interval and timeout for health monitor
- [cloud-provider-vsphere] Fix vSphere privilege matrix and describe instructions for setting up environment via vSphere Client
- [cni-simple-bridge] Updated base image reference to support new Werf import format.
- [common] Fixed an issue where Trivy could not detect VEX files in coredns images.
- [common] Removed Python completely from the debug-container image as it is no longer needed, resolving corresponding CVEs, and silenced false positives for etcd binaries via VEX.
- [deckhouse] Remove notified=false annotation reset from runReleaseDeploy in the module release controller.
- [deckhouse] Fix webhook-handler CVEs.
- [dhctl] Added validation of the command execution status code
- [docs] Fix vSphere privilege matrix and describe instructions for setting up environment via vSphere Client
- [documentation] Fix search.
- [ingress-nginx] CVE-2025-15566 is backported.
All pods of Ingress-NGINX controller will be restarted. - [ingress-nginx] CVE-2026-4342 fix is backported to Dechkouse 1.74.
All Ingress-NGINX controller pods will be restarted. - [istio] fixed CVEs in module images
module pods will be restarted - [istio] fixed CVE-2026-39882, CVE-2026-39883 and CVE-2026-35206
istio module pods will be restarted - [istio] fixed CVE-2026-34986
istio module pods will be restarted - [kube-proxy] Fixed CVE-2026-33186 and CVE-2026-24051 in kube-proxy dependencies.
This update triggers a rolling update of the kube-proxy pods. - [network-gateway] Fixed werf import syntax for compatibility with older werf versions.
1.74.15
Know before update
- An additional service daemonset will be added.
- CoreDNS pods will undergo a rolling restart.
- pods in d8-istio namespace will be restarted
Features
- [docs] Add snapshot-controller module architecture section.
Fixes
- [ci] Use GITHUB_TOKEN for backport action and set required workflow permissions
- [cilium-hubble] Fixed CVE-2026-33186 in the hubble-ui image.
- [cni-cilium] Fixed CVE-2026-33186, CVE-2026-27142, and CVE-2026-27139 by updating grpc dependency and Go version, and resolved build compatibility issues.
- [common] Fixed CVE-2026-33186 in the CoreDNS image.
CoreDNS pods will undergo a rolling restart. - [dhctl] mitigate CVE-2026-33186
- [istio] fixed cve in module images
pods in d8-istio namespace will be restarted - [node-local-dns] Return stale-dns-connections-cleaner
An additional service daemonset will be added. - [registrypackages] Added vex with CVE-2026-33186.
1.74.14
Features
- [docs] Add cloud-provider-dvp module architecture section.
- [docs] Add local-path-provisioner module architecture section.
- [docs] Update architecture section.
Fixes
- [cloud-provider-zvirt] fix CSI token refresh patch apply
- [docs] Added the display of the d8 utility version on the command reference page.
- [istio] Reduce CPU and RAM for regenerate multicluster JWT token and sort ingressGateway
- [tools] Added HTTP client proxy support
1.74.13
Fixes
- [cloud-provider-openstack] fix LB.enabled flag
1.74.12
Know before update
- All Ingress-Nginx controller pods will be restarted.
Fixes
- [candi] Added a Netplan override to force the secondary NIC to use the main routing table, fixing cloud-init PBR conflicts.
- [ingress-nginx] CVE-2026-3288 fix is backported in all Ingress-Nginx controllers.
All Ingress-Nginx controller pods will be restarted. - [node-manager] Fixed vulnerabilities.
1.74.11
Know before update
- Kruise controller pod will be restarted.
- Registry pods will be restarted.
Fixes
- [admission-policy-engine] Fix template tests
- [ci] Add zVirt testing to e2e tests to DKP 1.74.
- [ci] Backport the provider instance change and the new containerd version in OpenStack tests to release 1.74.
- [cloud-provider-aws] fix getInstancesByIDs to comply with the describeInstanceBatcher.
- [cloud-provider-openstack] Add loadBalancer.enabled flag to prevent CCM crashes on k8s 1.32 without Octavia service
- [docs] Correct the wording in the cloud-provider-vsphere module docs.
- [docs] Updated editions for the modules managed-postgres, managed-valkey and managed-memcached.
- [ingress-nginx] Golang versions are updated.
Kruise controller pod will be restarted. - [registry] Updated auth image Go dependencies to fix Go CVEs.
Registry pods will be restarted.
1.74.10
Know before update
- All ingress-nginx controller pods will be restared.
Fixes
- [ingress-nginx] Nginx and module’s dependencies are updated.
All ingress-nginx controller pods will be restared.
1.74.9
Know before update
- Workload Pods are no longer denied by unrelated SecurityPolicy checks (e.g. hostNetwork/hostPort) when corresponding policy fields are not explicitly set.
Fixes
- [admission-policy-engine] Prevent unintended Gatekeeper constraints from being rendered for SecurityPolicy when boolean fields are omitted.
Workload Pods are no longer denied by unrelated SecurityPolicy checks (e.g. hostNetwork/hostPort) when corresponding policy fields are not explicitly set. - [candi] Bashible script fix to handle multiple GPUs
- [candi] fix if node has bashible-uninitialized taint in race condition.
- [ci] Add ignoring of D8NodeContainerdV2NotSupported and D8NodeCgroupV2NotSupported alerts to release 1.74.
- [cloud-provider-dvp] added fixes an infinite deletion state of DeckhouseMachine.
- [deckhouse] Fix yaml validation in settings.
- [docs] Changing module descriptions
- [docs] Editing FAQ problems applying dexprovider
- [docs] Editing flannel to cillium.
- [docs] Adding information about using images sign policy.
- [istio] Fixing the list of requests from istiod to gateway API
- [registrypackages] added usage GOPROXY in systemctl-power-commands-wrapper
1.74.8
Fixes
- [docs] Escape HTML in OpenAPI descriptions
- [extended-monitoring] Cleanup exporter metrics when the monitored resource has been deleted
1.74.7
Fixes
- [admission-policy-engine] Convert constraints multi-line output to single-line
- [candi] Added a Netplan override to force the secondary NIC to use the main routing table, fixing cloud-init PBR conflicts.
- [dhctl] Fix panic in dhctl config render kubeadm-config command.
- [istio] Correction of an useless error in the Istio CNI workflow
1.74.6
Know before update
- All current connections powered by EgressGateways will be terminated.
Features
- [docs] Added info from
csi-cephmodule about Ceph storage system integration in DKP documentation.
Fixes
- [cni-cilium] Fix issue in generating CiliumEgressGatewayPolicy CR.
All current connections powered by EgressGateways will be terminated. - [ingress-nginx] A false-positive trigger of alert GeoIPDownloadErrorDetectedFromMaxMind is fixed.
- [node-manager] Fix panic in cluster-autoscaler caused by nil pointer dereference during node removal simulation.
1.74.5
Know before update
- The ingress nginx controllers’ pods will be restated.
Features
- [candi] Display lists of third-party (OSS) components in the modules documentation.
- [cloud-provider-dvp] Add validation for VirtualMachineClass and boot images before VM creation
Fixes
- [admission-policy-engine] Refactor constraint templates
- [dhctl] Improved reliability when connecting to dhctl servers by adding retry logic and better error handling during startup
- [docs] Corrections of typos and terms.
- [docs] Small imrovements of control-plane management docs
- [docs] Improve documentation for cluster network configuration
- [ingress-nginx] The CVE-2026-1580, CVE-2026-24512, CVE-2026-24513, CVE-2026-24514 CVEs fixes are backported.
The ingress nginx controllers’ pods will be restated. - [ingress-nginx] The CVE-2026-1580, CVE-2026-24512, CVE-2026-24513, CVE-2026-24514 CVEs are fixed.
The ingress nginx controllers’ pods will be restated. - [monitoring-kubernetes] fix kube_persistentvolume_is_local recording rule when there are more than one kube-state-metrics exporter in cluster
- [prometheus] Make Grafana redirect ingress pass the annotations validation.
- [user-authz] Updated logo on the authentication page.
1.74.4
Know before update
- If the SDN module is used in the cluster, the Cilium agent pods will be restarted.
- Improved documentation clarity for Yandex Cloud NAT instance deployment scenarios
- no impact
Features
- [docs] Add the enableNote display to the external modules configuration page.
- [prometheus] improve redirects from Grafana to the Deckhouse UI when Grafana is disabled
no impact
Fixes
- [cloud-provider-azure] fixed patch in azure
- [cloud-provider-dvp] Fixed VM deletion timeout and improved memory validation error reporting
- [cloud-provider-dvp] Cleanup orphaned resources when VM creation fails
- [cloud-provider-huaweicloud] fix cve
- [cloud-provider-vcd] fix the interaction of legacy and non-legacy components with the API
- [cloud-provider-vsphere] fix cve
- [cni-cilium] Fix hook discovery_cni_exclusive.go
If the SDN module is used in the cluster, the Cilium agent pods will be restarted. - [docs] Fixed typo in the OpenStack integration section.
- [docs] Added Docker Bearer Token Realm configuration steps for Nexus registry setup.
- [docs] Document NAT instance deployment zones for different subnet configurations
Improved documentation clarity for Yandex Cloud NAT instance deployment scenarios - [ingress-nginx] Fixed useless geoip alerts.
- [monitoring-kubernetes] fix kube_persistentvolume_is_local recording rule for not-bound PVCs
1.74.3
Know before update
- Node-local-dns safe-uptader pods will be restarted.
Features
- [prometheus] Add prometheus configuration reload failed alert
Fixes
- [admission-policy-engine] Fix constraint templates, add additional tests
- [cloud-provider-aws] fix cve
- [cloud-provider-vcd] fix cve
- [cloud-provider-yandex] fix cve
- [cni-cilium] Fixed egress-gateway-agent controller logic for deleted resources and disable dev logging.
- [deckhouse] Fix module rerun.
- [docs] Added a link to the ModuleSource to the explanation table of the editions.
- [metallb] Fixed D8MetallbBGPSessionDown alert false positives caused by duplicate metrics without port information.
- [node-local-dns] The safe-updater’s logic of detecting if daemonset pods are up-to-date is improved.
Node-local-dns safe-uptader pods will be restarted. - [registry] Fixed validation of input image list changes in the registry checker.
1.74.2
Know before update
- All pods running kube-rbac-proxy will be restarted.
Features
- [prometheus] Replace PrometheusRules with ClusterObservabilityMetricsRulesGroups or ClusterObservabilityPropagatedMetricsRulesGroups when deployed using helm_lib_prometheus_rules helper and the observability module is enabled
Fixes
- [admission-policy-engine] fix namespaceSelector for constraints, add CRDs
- [candi] Remove duplicate
additional_disks_hashesdefinition in static-node Terraform module. - [candi] Updated the bashible step to include Linux kernel versions that address CVE-2025-37999
- [candi] Allow manually stopping DVP node VirtualMachines in nested clusters by using runPolicy AlwaysOnUnlessStoppedManually.
- [cloud-provider-azure] fixed cve
- [cloud-provider-dvp] this PR gives a lot more informational errors and messages to user
- [cloud-provider-openstack] fix cve
- [common] Latest CVEs are fixed.
All pods running kube-rbac-proxy will be restarted. - [control-plane-manager] Fix validation for resource-quota-overrides labels
- [deckhouse] Fix mpo cleanup.
- [deckhouse] Fix module installer cleanup.
- [deckhouse-controller] Exclude all service accounts from
d8-namespaces ind8ms-prefixValidatingAdmissionPolicy. - [ingress-nginx] Latest CVEs are fixed.
All pods running kube-rbac-proxy will be restarted. - [kube-dns] Reverts
- [log-shipper] Fixed incorrect apiVersion in ClusterLogDestination documentation examples.
- [node-manager] Change NodeGroup maxPods validation from hard rejection to warning.
- [registry] Omitted the auth field in DockerConfig when credentials (username and password) are empty.
1.74.1
Know before update
- Prevents creation of Projects with too-long names that would lead to invalid generated Kubernetes resource names.
- deckhouse queue
- deckhouse queue blocked
Features
- [common] Enrich debug-container image with nsenter and /bin/sh symlink.
- [common] Resource quota ignore mechanism for pvc and pods
- [dhctl] Skipped application edition validation for standalone builds.
Fixes
- [candi] remove excessive netcat calls from d8-shutdown-inhibitor
- [candi] Add pause and kubernetes-api-proxy registry packages to bashible
bb-package-fetchto prevent node failures during containerd major upgrades. - [control-plane-manager] Fixed vulnerabilities
- [deckhouse] Fix module docs rendering.
- [deckhouse] Fix module enabling.
- [deckhouse-controller] fix copy-custom-certificate hook
- [docs] Added link to NVIDIA Container Toolkit docs.
- [docs] Fixed typo.
- [docs] Fixed the logic of generating the revision table.
- [extended-monitoring] Add namespace-scoped overrides
- [ingress-nginx] Improved stability of geoproxy service startup.
- [kube-dns] Refactored hook remove_old_coredns_deploy_and_svc — delegated deletion of original kube-dns service to helm.
- [multitenancy-manager] Add validation to restrict Project name length to 53 characters.
Prevents creation of Projects with too-long names that would lead to invalid generated Kubernetes resource names. - [node-manager] remove excessive netcat calls from d8-shutdown-inhibitor
- [node-manager] It fixes issues in the DaemonSet manifest for fencing module.
- [tools] Add check values not null in prometheusRule or ClusterObservabilityPropagatedMetricsRulesGroup
deckhouse queue - [tools] Add check values not null in prometheusRule or ClusterObservabilityPropagatedMetricsRulesGroup
deckhouse queue blocked - [user-authn] Quote service names to prevent digit-only names from breaking yaml parser
- [user-authz] Allow project-scoped roles to access Cluster-wide objects
Version 1.73
Important
-
This release includes several important security improvements. Multiple known vulnerabilities have been fixed, including one in the user-authn module (CVE-2025-22868) that could potentially allow bypassing authentication checks. It is recommended that you schedule this update. See the Security section for details.
-
The
dashboardmodule will be removed in future DKP versions. Use the Deckhouse web UI instead (requires theconsolemodule to be enabled). -
The
runtime-audit-enginemodule is now loaded from an external source (thedeckhouseModuleSource). -
All DKP components will be restarted during the update.
Major changes
- Added support for changing the registry settings (via the
modeparameter) inUnmanagedmode, similar toDirectmode. A new parametercheckModehas been added to control registry validation behavior:Default: Verifies the availability of all system module images and the deckhouse-controller images.Relax: Verifies only the deckhouse-controller images.
-
Updated requirements for the
emailfield of the User object: the email address must now be in lowercase. Existing users will not be affected. -
Dex updated to v2.44.0. It now allows authentication through available identity providers if one of them is down, and supports authentication via identity providers through a proxy.
-
The User object status now displays the reason for user lockout (controlled by the
lockoutparameter). -
Added the
additionalDisksparameter for the Deckhouse Virtualization Platform integration provider, allowing creation and attachment of additional disks to VMs in a NodeGroup (sizeand StorageClass must be specified). This simplifies data distribution across multiple disks. -
Added support for
additionalMetadatain objects (networks, VMs, disks) for the VMware Cloud Director integration provider. Metadata is merged with existing data, withadditionalMetadatavalues taking precedence. ChangingadditionalMetadatatriggers recreation of CloudEphemeral node groups that use the affected VCDInstanceClass. -
For the VMware vSphere integration provider, you can now specify an SPBM storage policy ID (via the
storagePolicyIDparameter) and configure automatic creation of a StorageClass for each available SPBM policy. You can now explicitly select a policy for master and worker nodes and use the corresponding storage classes. - Added alerts to help plan module deprecation or migration:
ModuleIsDeprecated: Notifies when a module is deprecated and nearing end of support.D8ModuleOutdatedByMajorVersion: Notifies when a module is behind by one or more major versions.
-
Added
GeoIPDownloadErrorDetectedalert to notify about MaxMind GeoIP database download issues. -
The update notification workflow has changed — a release is applied only after the notification is successfully delivered to the configured webhook. If delivery fails, the update is paused until the webhook is restored.
-
Reorganized in-cluster documentation. All module documentation (including connected ones) is now located under the Modules section. Search has been updated.
-
For Ingress NGINX Controller v1.10, added the option to enable the profiler (via the
nginxProfilingEnabledparameter). Enabling the profiler increases resource consumption but may be useful for debugging controller issues. -
Added support for custom HTTP authentication headers (via the
headersparameter of UpmeterRemoteWrite) when sending SLA monitoring metrics via Prometheus Remote Write protocol. -
Optimized DKP core module loading, reducing startup time in clusters with many modules.
-
Audit logs now show which OIDC provider issued the authentication token
- Deckhouse CLI (
d8) updated to v0.23.0:- Added the
d8 statuscommand, which provides a quick cluster summary (nodes, releases, Deckhouse pods, alerts, registry, Deckhouse settings, CNI, queue state). - Added the
d8 k debugcommand, which runs the DKP built-in debug container (image can be overridden via--image). This simplifies interactive pod debugging. - Added the
--watchflag to thed8 system queue listcommand to track queue state changes.
- Added the
Security
-
Improved baseline container security. Updated security profiles for the following modules to restrict privileges and access rights to the minimum required:
cni-cilium,cni-flannel,cni-simple-bridge,ingress-nginx,istio,keepalived,kube-dns,kube-proxy,node-local-dns,network-gateway,network-policy-engine,open-vpn. -
Added the
allowRbacWildcardsflag to the SecurityPolicy, controlling whether wildcards are allowed in Role and RoleBinding definitions (set totrueby default). Security policies can now also restrict interactive connections to pods (CONNECTforpods/execandpods/attach) within namespaces. -
Added support for preventing creation of pods with specific tolerations from a list (
policies.disallowedTolerationsparameter in the operational policy). This helps prevent user workloads from running on nodes reserved for special tasks. -
Enhanced security in Ingress NGINX Controller v1.12 (distroless image, vulnerability fixes, and other improvements).
-
Fixed known vulnerabilities in the following modules:
operator-trivy,registry,user-authn,cloud-provider-dvp,multitenancy-manager,admission-policy-engine,ingress-nginx,alertmanager,metallb,istio,node-local-dns,kube-apiserver.
Network
- For the VMware Cloud Director integration provider, added support for LoadBalancer backed by VMware NSX Advanced Load Balancer (ALB/Avi) when using
NSX-T. Requires the Load Balancer feature to be enabled on the Edge Gateway. If LoadBalancer is enabled after cluster creation, DKP automatically detects and applies the change within an hour. For open ports, DKP creates correspondingPool + Virtual Servicepairs. If there is a firewall, add allow rules for the LoadBalancer’s external IP address and relevant ports.
The complete list of changes, including the updated components, is available in the changelog on GitHub.
1.73.14
Know before update
- All Ingress-Nginx controller pods will be restarted.
- All ingress-nginx controller pods will be restared.
- All modules of the ingress-nginx module will be restarted.
- Registry pods will be restarted.
- The ingress nginx controllers’ pods will be restated.
Fixes
- [admission-policy-engine] added mount-points in ratify
- [admission-policy-engine] deny_vulnerable_images webhook must use module status instead of moduleconfig
- [candi] cve fix for admission-policy-engine, cert-manager, user-authn, multitenancy-manager
- [candi] Updated pwru tool to v1.0.11 to fix CVE-2025-68121.
- [candi] Updated pwru tool to v1.0.11 to fix CVE-2025-68121.
- [chrony] mitigate CVE-2025-58181
- [common] Fixed vulnerability CVE-2026-24051.
- [control-plane-manager] Fixed vulnerabilities.
- [documentation] Fixed HTTP/404 error when opening modules documentation from the Web interface.
- [extended-monitoring] Fixed CVE-2025-47914, CVE-2025-58181
- [ingress-nginx] CVE-2026-3288 fix is backported in all Ingress-Nginx controllers.
All Ingress-Nginx controller pods will be restarted. - [ingress-nginx] Golang and Altlinux updates are reverted.
All modules of the ingress-nginx module will be restarted. - [ingress-nginx] Nginx and module’s dependencies are updated.
All ingress-nginx controller pods will be restared. - [ingress-nginx] The CVE-2026-1580, CVE-2026-24512, CVE-2026-24513, CVE-2026-24514 CVEs fixes are backported.
The ingress nginx controllers’ pods will be restated. - [loki] Fixed CVE-2025-47914, CVE-2025-58181
- [monitoring-kubernetes] Fixed CVE-2025-47914, CVE-2025-58181
- [node-local-dns] Fixed CVEs
- [node-manager] Fixed vulnerabilities.
- [operator-prometheus] Fixed CVE-2025-47914, CVE-2025-58181
- [prometheus] Fixed CVE-2025-47914, CVE-2025-58181, CVE-2025-65637
- [prometheus-metrics-adapter] Fixed CVE-2025-47914, CVE-2025-58181
- [prometheus-pushgateway] Fixed CVE-2025-47914, CVE-2025-58181, CVE-2025-22872, CVE-2025-22868
- [registry] Updated auth image Go dependencies to fix Go CVEs.
Registry pods will be restarted. - [registry] Updated auth image Go dependencies to fix Go CVEs.
Registry pods will be restarted. - [registrypackages] Fixed vulnerability CVE-2026-24051.
- [upmeter] Add proper securityContext to the upmeter probe to meet the restricted security profile constraints.
- [upmeter] Fixed CVE-2025-47914, CVE-2025-58181, CVE-2025-65637
1.73.13
Know before update
- config module loki ↓
Fixes
- [candi] Updated the bashible step to include Linux kernel versions that address CVE-2025-37999
- [cni-cilium] Fixed vulnerabilities.
- [cni-flannel] Bump version and fixed vulnerabilities.
- [istio] Fix CVE for Istio version 1.21 and 1.25
- [kube-dns] Fixed vulnerabilities.
- [kube-proxy] Fixed vulnerabilities.
- [kube-proxy] Fixed vulnerabilities.
- [loki] disable send analytics report to stats.grafana.org
config module loki ↓ - [network-policy-engine] Fixed vulnerabilities.
- [node-local-dns] Bump version and fixed vulnerabilities.
- [openvpn] Fixed vulnerabilities.
- [registry] Omitted the auth field in DockerConfig when credentials (username and password) are empty.
1.73.12
Fixes
- [cloud-provider-vsphere] fix stale session for cloud-data-discoverer
- [istio] fixing the CVE in Kiali
1.73.11
Fixes
- [dashboard] Fixed CVE-2025-30204 by updating dashboard components
- [deckhouse] Fix module enabling.
- [registrypackages] Update integrity patch for containerd (cse only).
1.73.10
Fixes
- [cloud-provider-dvp] Added functionality to wait for a disk to be attached to a VM
- [cloud-provider-huaweicloud] fix CSI unpublishValidation for non exist ECS instance
- [registrypackages] Update integrity patch for containerd (cse only).
- [service-with-healthchecks] Fixed CVEs
- [user-authz] cache namespace label checks in the user-authz webhook via informer to avoid per-request apiserver GETs
1.73.9
Features
- [ci] Add DVP e2e for the cse edition
Fixes
- [deckhouse-controller] Fix conversions for external modules
- [dhctl] Fix parallel bootstrap cloud permanent nodes
1.73.8
Know before update
- Ingress Nginx controller dashboards are fixed
- The MTU will be updated on all interfaces of all pods.
Fixes
- [cloud-provider-yandex] cloud-provider-yandex CVE’s was fixed
- [cni-cilium] The MTU configuration has been updated.
The MTU will be updated on all interfaces of all pods. - [deckhouse-controller] Fix “multiple readiness hooks found” error on hook registration retry after failure.
- [deckhouse-controller] Fix deprecated and experimental module metrics not being reset when module is disabled.
- [dhctl] Fix panic during destroy. Change opentofu log level to INFO.
- [docs] The current editions are listed on the page about installation in a closed environment.
- [docs] Fixed links to the Getting Started in the documentation.
- [prometheus] Fix namespace label value in the Ingress Nginx controller and several other metrics
Ingress Nginx controller dashboards are fixed - [user-authn] Fix BadRequest after the change password redirect when password policy is enabled
- [user-authn] Fix login error 500 with password policy enabled.
1.73.7
Know before update
- Allows scaling control-plane from 1→3 in clusters where ControlPlaneKubeletLocalMode=true.
- The instructions for reducing the number of master nodes have not been updated for a long time, and for a long time now, the conversion to reducing the number of master nodes in clusters in a cloud installation is automatic.
- upmeter check
Features
- [ci] Add Deckhouse Virtualization Platform testing to e2e tests.
- [docs] Added Commander support for DVP EE in the Platform Editions table.
Fixes
- [candi] Add missing debian 13 version to detect_bundle
- [cloud-provider-huaweicloud] fix Provider ID
- [control-plane-manager] Fix “etcd join” phase for control-plane scaling in v1.33.
Allows scaling control-plane from 1→3 in clusters where ControlPlaneKubeletLocalMode=true. - [deckhouse-controller] Fixed verifying migrated modules
- [deckhouse-controller] fix conversion applying for external modules
- [dhctl] Fix StaticInstance readiness check and refactoring readiness check for resources.
- [docs] Outdated instructions for reducing the number of master nodes in cloud clusters have been fixed. Users will correctly perform the converge of cloud clusters.
The instructions for reducing the number of master nodes have not been updated for a long time, and for a long time now, the conversion to reducing the number of master nodes in clusters in a cloud installation is automatic. - [kube-dns] Improved /etc/hosts renderer compatibility with admission-policy-engine Restricted mode.
- [log-shipper] Update documentation.
- [upmeter] fix securityxontext for statefulset
upmeter check
1.73.6
Fixes
- [cloud-provider-vsphere] fix cloud-data-discoverer (SPBM)
1.73.5
Know before update
- node-exporter pods will be rollout restarted during upgrade
Features
- [ci] Add static e2e for the cse edition
- [ci] Add static e2e for the cse edition
- [docs] Added documentation about configuration of the Kubernetes audit policy.
Fixes
- [cloud-provider-azure] fix build image for azure ccm
- [cni-cilium] Some issues have been fixed in the EgressGateway.
- [dhctl] Fix converge manifests for static cluster in commander.
- [monitoring-kubernetes] remove the Docker traces from the module code
node-exporter pods will be rollout restarted during upgrade
1.73.4
Know before update
- Ingress-Nginx controller’s pods of 1.9 version will be restarted.
- erroneous alerts for node disk usage are fixed
Fixes
- [cloud-provider-dvp] add missing field to Cluster
- [cloud-provider-yandex] Terraform auto converger was failed for WithNATInstance layout.
- [deckhouse] Fix validation logic for a disabled module
- [dhctl] Validate WithNATInstance Yandex layout params only in bootstrap.
- [dhctl] Added nil check to dhctl during converge in migrator
- [extended-monitoring] drop metrics when extended monitoring is disabled for node(s)
erroneous alerts for node disk usage are fixed - [ingress-nginx] A symlink to the new opentelemetry config path is added.
Ingress-Nginx controller’s pods of 1.9 version will be restarted. - [ingress-nginx] Fixed CVEs
- [monitoring-kubernetes] Rollout changes for resources metrics kubelet
- [node-manager] Fix panic in registry packages proxy if image not found.
- [operator-trivy] Fix CIS Benchmark report template
- [operator-trivy] fix CVE-2025-52881 CVE-2024-25621 on image trivy
- [registrypackages] Update containerd to 1.7.29 / 2.1.5 and runc to 1.3.3
1.73.3
Know before update
- the extended monitoring will only be enabled when the label is explicitly set on a namespace
Fixes
- [control-plane-manager] Add vex for CVE-2025-31133, CVE-2025-52881 .
- [deckhouse] Add umask before creating first-control-plane-bashible-ran file
- [dhctl] Fix getting passphrase for key from connection config for cli.
- [dhctl] Move yandex withNATInstance layout settings from preflights to preparator.
- [documentation] Fix module’s deployment and configuration.
- [extended-monitoring] Fix extended-monitoring.deckhouse.io/enabled label handling
the extended monitoring will only be enabled when the label is explicitly set on a namespace - [monitoring-kubernetes] fix CVE-2025-52881 for node-exporter
- [prometheus] Fix description for not usable CVE
- [registrypackages] Fixes CVE in kubernetes-cni
1.73.2
Fixes
- [ingress-nginx] CVEs fixed
- [istio] Kiali’s CVE-2024-28180 is fixed.
- [multitenancy-manager] fix CVE-2024-25621 CVE-2025-64329
- [node-manager] fix cve and vex
- [operator-trivy] fix CVE-2025-52881 CVE-2024-25621
- [operator-trivy] Add grep to node-collector and improve error reporting
- [user-authn] Rollback patch for handling insecureSkipEmailVerified condition
1.73.1
Features
- [docs] Added manifest for internal LB VK Cloud.
- [docs] Added info from
csi-netappmodule about NetApp storage integration in DKP documentation. - [istio] Added access log format setting for proxy sidecars.
Fixes
- [admission-policy-engine] Prohibit only creation or modification for objects with vulnerable images
- [admission-policy-engine] Fixed proxy support for trivy-provider
- [candi] Remove proxy-environment for kubelet when proxy disabled
- [ci] Fix e2e upgrade tests.
- [ci] fix grafana vex
- [ci] fix trickster vex
- [ci] Fix missing edition suffixes in prerelease build.
- [cloud-provider-dvp] Correct the calculation of the path to the device
- [cloud-provider-vsphere] fix vSphere storageClass template
- [cni-cilium] fixed connectivity dashboard
- [deckhouse] Fix race conditions in test
- [deckhouse-controller] fix bash completion
- [docs] Small fix on the “Security event audit” page.
- [docs] Fixed the description for the 4239 port in the reference table.
- [docs] Updated installer block in GS.
- [docs] Added a link to the DVP integration page in the virtualization overview.
- [istio] The same owner is specified for the files that are used to run in the operator container.
- [metallb] Disabled unnecessary speaker functionality.
- [monitoring-kubernetes] Fixed CVE-2025-22870, CVE-2025-22872.
- [prometheus] Fix plur in vex for tricster
- [prometheus-pushgateway] Fixed CVE-2025-22869, CVE-2025-22868, CVE-2025-22870.
- [registrypackages] Update runc to 1.3.1.
Version 1.72
Important
- All DKP components will be restarted during the update.
- To use experimental modules in the cluster, you now need to explicitly enable the allowExperimentalModules parameter. By default, experimental modules are disabled. Modules that were enabled before the update will not be automatically disabled. However, if an experimental module enabled prior to the update is manually disabled during the update process, you will need to grant permission to use experimental modules again in order to re-enable it.
- If there are WireGuard interfaces on the cluster nodes, you must update the Linux kernel to version 6.8 or higher.
Major changes
-
Added a new registry module and the ability to adjust container registry parameters without restarting all DKP components. Two modes for working with the container registry are now available in DKP:
Unmanaged(the approach used in previous versions) andDirect(a new mode). InDirectmode, DKP creates a virtual container registry address in the cluster that all DKP components use. Changing the container registry address (for example, switching to a different registry or changing the DKP edition in the cluster) in this mode does not trigger a forced restart of all DKP components. -
Added support for recursive DNS servers (configured via the recursiveSettings section of the
cert-managermodule). They are used to verify the existence of a DNS record before starting the ACME DNS-01 domain ownership validation process. This is useful if the same domain is used both publicly and within the cluster, or if the domain has dedicated authoritative DNS servers. -
Introduced separation of modules into critical and functional using the
criticalflag inmodule.yaml. Critical modules are started first. Functional modules are started after the bootstrap process is complete. Their tasks run in parallel and do not block the queue in case of failure. This speeds up cluster installation and improves fault tolerance when starting modules. -
You can now enable logging of all DNS queries (the enableLogs parameter of the
node-local-dnsmodule). -
In the
cloud-provider-vcdmodule, a new WithNAT layout has been added for cluster deployment. It automatically configures NAT and, if necessary, firewall rules for accessing nodes through a bastion host. It also supports bothNSX-TandNSX-V. This makes it possible to deploy a cluster in VMware Cloud Director without pre-configuring the environment (unlike theStandardlayout).
Security
-
Added the fields
user-authn.deckhouse.io/nameanduser-authn.deckhouse.io/preferred_usernameto Kubernetes audit log events. These fields display user claims from the OIDC provider, improving authentication monitoring and troubleshooting. -
Kubernetes versions 1.30–1.33 have been updated to the latest patch releases.
-
For the AWS provider, added the ability to disable the creation of default security groups (the disableDefaultSecurityGroup parameter). When disabled, security groups must be created manually and explicitly specified in AWSClusterConfiguration, AWSInstanceClass, and NodeGroup. This new feature provides greater control over security settings.
-
Added support for password policies for local users (configured in the passwordPolicy section). You can now enforce a minimum password complexity, set password expiration, require password rotation, prevent reuse of old passwords, and lock accounts after a specified number of failed login attempts. These changes allow administrators to centrally enforce password requirements and improve cluster security.
The complete list of changes, including the updated components, is available in the changelog on GitHub.
1.72.13
Fixes
- [istio] Reduce CPU and RAM for regenerate multicluster JWT token and sort ingressGateway
1.72.12
Fixes
- [deckhouse-controller] Fix conversions for external modules
- [deckhouse-controller] Fixed verifying migrated modules
- [deckhouse-controller] Fixed a crash during external module updates with conversions that caused ModuleRelease to fail validation due to a forbidden property error.
- [istio] Fixed AuthorizationPolicy CRD insufficiency for Istio 1.25.
1.72.11
Fixes
- [cert-manager] Mark module as critical for cluster
- [deckhouse-controller] fix deckhouse update accidentally minor skip
- [istio] Erroneous option in 1.25 control-plane helm template fixed.
- [registrypackages] Fixed permissions for directory with cni-plugins in PCI-DSS clusters
- [tools] update dmt version to 0.1.53
1.72.10
Fixes
- [ci] Delete an extra entry from the changelog.
- [cloud-provider-vsphere] fix hook logick
- [deckhouse-controller] fix possibility deadlock
- [istio] Reduce RAM for regenerate multicluster JWT token
1.72.9
Fixes
- [candi] Switch node hostname in discover_node_ip.sh to locked hostname
- [candi] fixed version of containerd image in bashible step
- [deckhouse] Fixes bug when module`s config values unchanged after enabling.
- [deckhouse-controller] Add more mutexes to shell-operator framework
- [dhctl] Change default ssh mode
- [ingress-nginx] The is reverted.
- [ingress-nginx] Enable ExecuteHookOnEvents on hook set_annotation_validation_suspended.go
1.72.8
Fixes
- [cloud-provider-dvp] fix single-node LoadBalancer bug and add multi-LB per node support
- [cloud-provider-vsphere] cloud-data-discoverer fixes
- [node-manager] Enable SSHLegacyMode if SSHPrivateKey is not empty and fix gossh panic.
1.72.7
Fixes
- [cloud-provider-huaweicloud] add rule to d8:cloud-provider-huaweicloud:cloud-controller-manager account for list pods as needed
- [node-manager] CAPI crd served version fix
- [node-manager] shutdown inhibitor use three-state GracefulShutdownPostpone condition so kubelet waits for the final decision before ending pods
1.72.6
Fixes
- [candi] Fix nodeuser deletion.
- [cloud-provider-vsphere] cloud-data-discoverer fixes
- [deckhouse-controller] Ensure
afterDeleteHelmhooks receive Kubernetes snapshots by stopping monitors after hook execution - [dhctl] Fixing bashible steps when it does not respond to the limit on the number of attempts
- [dhctl] Fix start and keep-alive behavior, do not change host if any kube-proxies has been started.
- [node-manager] CAPI crd served version fix
- [vertical-pod-autoscaler] Disable vpa for k8s-metacollector in module runtime-audit-engine
1.72.5
Fixes
- [cloud-provider-vsphere] cloud-data-discoverer fixes
- [cni-cilium] Fixed 1.14 -> 1.17 migration mechanism
- [cni-cilium] improved migration to 1.17 logic
- [node-manager] capi crd served version fix
1.72.4
Fixes
- [cloud-provider-openstack] fix openstack ccm deployment
- [deckhouse] Fix maintenance mode label ignore rule.
- [ingress-nginx] lowers the severity level of the IngressNginxController alert from 3 to 4
1.72.3
Fixes
- [dhctl] Add timeout to ssh client dial and fix stop.
- [dhctl] Fix ssh client start with bastion in dhctl.
- [dhctl] Fix deadlock while reading dhctl singlethreaded server logs
- [dhctl] Fixed the behaviour of the attach operation when there are no keys in SSHConfig, only a password
- [docs] Containerdv2 migration FAQ fix
- [ingress-nginx] fix set_annotation_validation_suspended.go hook to set correct metric
- [ingress-nginx] fix set_annotation_validation_suspended.go hook to set correct metric
- [loki] Change stage to Preview
1.72.2
Know before update
- The ingress-nginx pods of 1.10 will be restarted.
Fixes
- [docs] Added the Local path provisioner storage section to the documentation sidebar, including Overview, Examples, Reference, and FAQ entries.
- [ingress-nginx] Fixed nginx segfaults when opentelemetry is enabled.
The ingress-nginx pods of 1.10 will be restarted. - [node-manager] Fix node update approval hook for CAPI nodes.
1.72.1
Fixes
- [deckhouse] add check for user-authn in enabled script
- [deckhouse] fix editions list in check deckhouse edition hook
- [dhctl] Fix ssh client initialising in Commander Attach and Commander Detach operations
- [node-manager] Some minor fixes for nvidia gpu templates to improve security.
Version 1.71
Important
-
Prometheus has been replaced with Deckhouse Prom++. If you want to keep using Prometheus, disable the
promppmodule manually before upgrading DKP by running the commandd8 system module disable prompp. -
Support for Kubernetes 1.33 has been added, while support for Kubernetes 1.28 has been discontinued. In future DKP releases, support for Kubernetes 1.29 will be removed. The default Kubernetes version (used when the
kubernetesVersionparameter is set toAutomatic) has been changed to 1.31. -
Upgrading the cluster to Kubernetes 1.31 requires a sequential update of all nodes, with each node drained. You can control how node updates requiring workload disruptions are applied using the
disruptionsparameter section. -
The built-in
snapshot-controllerandstatic-routing-managermodules will now be replaced with their external counterparts of the same name, sourced via ModuleSourcedeckhouse. -
The new version of Cilium requires nodes to run Linux kernel version 5.8 or newer. If any node in the cluster has a kernel older than 5.8, the Deckhouse Kubernetes Platform upgrade will be blocked. Cilium Pods will be restarted.
-
All DKP components will be restarted during the update.
Major changes
-
You can now enforce two-factor authentication for static users. This is configured via the
staticUsers2FAparameter section of theuser-authnmodule. -
Added support for GPUs on nodes. Three GPU resource sharing modes are now available: Exclusive (no sharing), TimeSlicing (time-based sharing), and MIG (a single GPU split into multiple instances). The NodeGroup spec.gpu parameter section is used to configure the GPU resource sharing mode. Using a GPU on a node requires installing the NVIDIA Container Toolkit and the GPU driver.
-
When enabling a module (with
d8 system module enable) or editing a ModuleConfig resource, a warning is now displayed if multiple module sources are found. In such a case, explicitly specify the module source using the source parameter in the module’s configuration. -
Improved error handling for module configuration. Module-related errors no longer block DKP operations. Instead, they are now displayed in the status fields of Module and ModuleRelease objects.
- Improved virtualization support:
- Added a provider for integration with Deckhouse Virtualization Platform (DVP), enabling deployment of DKP clusters on top of DVP.
- Added support for nested virtualization on nodes in the
cni-ciliummodule.
- The
node-managermodule now includes several enhancements for improved node reliability and manageability:- You can now prevent a node from restarting if it still hosts critical Pods (labeled with
pod.deckhouse.io/inhibit-node-shutdown). This can be necessary for workloads with stateful components, such as long-running data migrations. - Introduced API version
v1alpha2for the SSHCredential resource, where thesudoPasswordEncodedparameter allows specifying thesudopassword in Base64 format. - The
capiEmergencyBrakeparameter allows you to disable Cluster API (CAPI) in emergency scenarios, preventing potentially destructive changes. Its behavior is similar to the existingmcmEmergencyBrakesetting.
- You can now prevent a node from restarting if it still hosts critical Pods (labeled with
-
Added a pre-installation check to verify connectivity to the DKP container image registry.
-
Improved the log file rotation mechanism when using short-term log storage (via the
lokimodule). Added theLokiInsufficientDiskForRetentionalert to warn about insufficient disk space for log retention. -
The documentation now includes a reference for the Deckhouse CLI (
d8utility) commands and parameters. -
When using CEF encoding for collecting logs from Apache Kafka or socket sources, you can now configure auxiliary CEF fields such as Device Product, Device Vendor, and Device ID.
-
The
passwordHashfield in the NodeUser resource is no longer required. This allows you to create users without passwords — for example, in clusters that use external authentication systems (such as PAM or LDAP). - Added support for CRI Containerd v2 with CgroupsV2. The new version introduces a different configuration format and includes a mechanism to migrate between Containerd v1 and v2. You can change the CRI type used on nodes via the
cri.typeparameter and configure it usingcri.containerdV2.
Security
-
Container image signature verification is now available in DKP SE+. This feature is now supported in DKP SE+ and EE.
-
The
log-shipper,deckhouse-controller, andIstio(version 1.21) modules have been migrated to distroless builds. This improves security and ensures a more transparent and controlled build process. -
New audit rules have been added to track interactions with containerd. The following are now monitored: access to the
/run/containerd/containerd.socksocket, modifications to the/etc/containerdand/var/lib/containerddirectories and the/opt/deckhouse/bin/containerdfile. -
Known vulnerabilities have been fixed in the following modules:
loki,extended-monitoring,operator-prometheus,prometheus,prometheus-metrics-adapter,user-authn, andcloud-provider-zvirt.
Network
-
Added support for Istio version 1.25.2, which uses the Sail operator instead of the deprecated Istio Operator. Also added support for Kiali version 2.7, without Ambient Mesh support. Istio version 1.19 is now considered deprecated.
-
Added support for encrypting traffic between nodes and Pods using the WireGuard protocol (via the
encryption.modeparameter). -
Fixed the logic for determining service readiness in the ServiceWithHealthcheck resource. Previously, Pods without an IP address (for example, in
Pendingstate) could be mistakenly included in the load balancing list. -
Added support for the least-conn load balancing algorithm. This algorithm directs traffic to the service backend with the fewest active connections, improving performance for connection-heavy applications (such as WebSocket services). To use this algorithm, enable the
extraLoadBalancerAlgorithmsEnabledparameter in thecni-ciliummodule settings and use theservice.cilium.io/lb-algorithmannotation on the service and set it to a supported value: random, maglev, or least-conn. -
Fixed an issue in Cilium 1.17
cilium-operatorwhere IP addresses were not reused after aCiliumEndpointwas deleted. The issue was caused by improper cleanup of priority filters, which could lead to IP pool exhaustion in large clusters. -
Refined the list of ports used for networking:
- Added and updated:
4287/UDP: WireGuard port used for CNI Cilium traffic encryption.4295-4297/UDP: Used by thecni-ciliummodule for VXLAN encapsulation of inter-pod traffic in multiple nested virtualization — when DKP with thevirtualizationmodule enabled is deployed inside virtual machines that are also created in DKP with thevirtualizationmodule enabled.4298/UDP: Used by thecni-ciliummodule for VXLAN encapsulation of traffic between pods if the cluster was deployed on DKP version starting from 1.71 (for clusters deployed on DKP versions up to 1.71, see the note for ports4299/UDP,8469/UDP, and8472/UDP).4299/UDP: Port for clusters deployed on DKP versions 1.64–1.70. Used by thecni-ciliummodule for VXLAN encapsulation of traffic between pods. Updating DKP to newer versions will not change the port used unless thevirtualizationmodule is enabled.8469/UDP: Port for clusters deployed on DKP version 1.63 and below with thevirtualizationmodule enabled prior to DKP version 1.63. Used by thecni-ciliummodule for VXLAN encapsulation of traffic between pods. Updating DKP to newer versions will not change the occupied port8472/UDP: Port for clusters deployed on DKP version 1.63 and below. Used by thecni-ciliummodule for VXLAN encapsulation of traffic between pods. Updating DKP to newer versions will not change the occupied port if thevirtualizationmodule is not enabled. Note that in such clusters, enabling thevirtualizationmodule on DKP before version 1.70 changes the port:- Enabling the
virtualizationmodule on DKP version 1.63 and below will change it to8469/UDPand will not change with subsequent DKP updates - Enabling the
virtualizationmodule on DKP starting from version 1.64 will change it to4298/UDPand will not change with subsequent DKP updates
- Enabling the
- Removed:
49152,49153/TCP: Previously used for live migration of virtual machines (in thevirtualizationmodule). Migration now occurs over the Pod network.
- Added and updated:
The complete list of changes, including the updated components, is available in the changelog on GitHub.
1.71.10
Features
- [deckhouse-controller] ignore migrate module if disabled
Fixes
- [deckhouse-controller] fix deckhouse update accidentally minor skip
- [istio] Reduce CPU and RAM for regenerate multicluster JWT token and sort ingressGateway
- [local-path-provisioner] Fix CVE in local-path-provisioner module
1.71.9
Fixes
- [cni-cilium] Fixed 1.14 -> 1.17 migration mechanism
1.71.8
Fixes
- [deckhouse] Fix module source updating fails.
- [vertical-pod-autoscaler] Disable vpa for k8s-metacollector
1.71.7
Fixes
- [candi] Fix arg
encryption-provider-configfor kubeadm configuration. - [candi] Disable immutable flag on erofs files in cleanup node stage.
1.71.6
Fixes
- [cni-cilium] Fixed the infinite loop in the “cilium migration” bashible step and improved synchronization between bashible and safe-agent-updater.
1.71.5
Know before update
- If wireguard interface is present on nodes, then cilium-agent upgrade will stuck. Upgrading the linux kernel to 6.8 is required.
- main and longterm Prometheuses will be rollout-restarted
Fixes
- [candi] Changed ExecStartPre in d8-shutdown-inhibitor.service
- [candi] Added warnings to the VMware Cloud Director environment documentation about Edge requirements
- [cni-cilium] Add a compatibility check for the Cilium version and the kernel version, if WireGuard is installed on the node
If wireguard interface is present on nodes, then cilium-agent upgrade will stuck. Upgrading the linux kernel to 6.8 is required. - [cni-cilium] Added a migration mechanism, which was implemented through the node group disruptive updates with approval.
- [cni-cilium] fixed invalid annotation name for lb-algorithm in docs
- [deckhouse-controller] fix module config ensure
- [docs] Fix relative links in the multitenancy-manager module documentation.
- [docs] Added steps that patch secret and prevented the image pull fail.
- [docs] Add containerv2 additional registry examples
- [docs] Add new requirement and commands to meet containerdv2
- [docs] Fixed command syntax for Docker container run in documentation.
- [docs] Add new requirement and commands to meet containerdv2 requirements
- [docs] Fix D8KubernetesStaleTokensDetected alert description.
- [docs] There should be one disk in the template.
- [docs] Updates for Observability docs.
- [log-shipper] Documentation updates.
- [loki] Documentation updates.
- [monitoring-custom] Documentation updates.
- [monitoring-kubernetes] Documentation updates.
- [monitoring-ping] Documentation updates.
- [prometheus] fix securityContext indentation in the Prometheus main and longterm resources
main and longterm Prometheuses will be rollout-restarted - [prometheus] Documentation updates.
- [upmeter] Documentation updates.
1.71.4
Know before update
- low
Features
- [docs] Added descriptions of the problem of using Cilium and Elastic together
low
Fixes
- [cloud-provider-vsphere] ignore ept_rvi_mode and hv_mode
- [docs] Fix mentions of ingress-nginx module in docs.
- [ingress-nginx] Fix mentions of ingress-nginx module in docs.
- [prometheus] Fix mentions of ingress-nginx module in docs.
1.71.3
Features
- [candi] Add rosa 12.6.
- [deckhouse-controller] add validation for module source changes
Fixes
- [candi] containerd migration fix
- [cni-cilium] fixed hostPort workability with extraLoadBalancerAlgorithmsEnabled.
- [log-shipper] Fix matchExpressions values validation for ClusterLoggingConfig
- [node-manager] Hide Nvidia GPU dashboards if there is no GPU nodes in a cluster.
- [node-manager] containerd migration fix
1.71.2
Know before update
- Users will know how to configure KeyCloak and dexProvider to get user rights in dashboard and console.
- default
- if there are several ModuleSources with the prompp module available, the “deckhouse” ModuleSource will be used.
Features
- [admission-policy-engine] Added the ability to specify allowedServiceTypes and blockWildcardDomains in the SecurityPolicy object
- [deckhouse-controller] add leader election for non ha clusters
- [user-authn] Add allowedGroup in dexProvider OIDC
Fixes
- [admission-policy-engine] Fix validation policy for pods with skip-pss-check labels
- [ci] fix e2e creds
- [ci] fix e2e creds
- [ci] Run changelog check only on tag pushes to enforce release documentation
- [ci] create resources before deckhouse manifest and mutating webhook for prom rules for e2e tests
- [ci] Run changelog check only on tag pushes to enforce release documentation
- [ci] create resources before deckhouse manifest and mutating webhook for prom rules for e2e tests
- [cloud-provider-aws] Fix root_block_device converge.
- [cni-cilium] Fixed least-conn logs when feature is disabled
- [deckhouse-controller] fix new snapshot logic bag
- [deckhouse-controller] add more verbosity to snapshot error
- [dhctl] Fix false-positive staticinstance ip duplication preflight checks fails.
- [docs] Correction of KeyCloak documentation in d8-user-authn
Users will know how to configure KeyCloak and dexProvider to get user rights in dashboard and console. - [docs] Refactored table of supported OS versions.
- [istio] Added missed documentation about version 1.25
- [metallb] Fixed overwriting of Service
statusfield by module components. - [metallb] avoid double prefixes in the dashboard names.
default - [node-manager] Changed draining logic in update_approval
- [node-manager] fixed render nvidia-mig-manager
- [prometheus] use the “deckhouse” ModuleSource as the default for the prompp ModuleConfig
if there are several ModuleSources with the prompp module available, the “deckhouse” ModuleSource will be used. - [runtime-audit-engine] Improve memory footprint by switching to the stdlib memory allocator instead of jemalloc
1.71.1
Features
- [docs] Added documentation with recommendations for generating self-signed certificates
Fixes
- [ci] add sh symlink to bash to the deckhouse-controller
- [ci] Fix listing of changed modules.
- [control-plane-manager] correct indentation of the ‘certSANs’ in v1beta4 template.
- [monitoring-ping] Reducing PROCESSOR time consumption by changing the method of waiting for incoming packets.
- [user-authn] fix dex oidc connector insecureSkipVerify and rootCAData options
Version 1.70
Important
-
The
ceph-csimodule has been removed. Use thecsi-cephmodule instead. Deckhouse will not be updated as long asceph-csiis enabled in the cluster. Forcsi-cephmigration instructions, refer to the module documentation. -
Version 1.12 of the Ingress NGINX Controller has been added. The default controller version has been changed to 1.10. All Ingress controllers that do not have an explicitly specified version (via the
controllerVersionparameter in the IngressNginxController resource or thedefaultControllerVersionparameter in theingress-nginxmodule) will be restarted. -
The
falco_eventsmetric (from theruntime-audit-enginemodule) has been removed. Thefalco_eventsmetric was considered deprecated since DKP 1.68. Use thefalcosecurity_falcosidekick_falco_events_totalmetric instead. Dashboards and alerts based on thefalco_eventsmetric may stop working. -
All DKP components will be restarted during the update.
Major changes
- In the
Autoupdate mode, patch version updates (for example, fromv1.70.1tov1.70.2) are now applied taking into account the update windows, if they are set. Previously, in this update mode, only minor version updates (for example, fromv1.69.xtov1.70.x) were applied with consideration to update windows, while patch version updates were applied as they appeared on a release channel. - A node can now be rebooted if the corresponding Node object has the
update.node.deckhouse.io/rebootannotation set. - When cleaning up a static node, any local users created by Deckhouse Kubernetes Platform are now also removed.
- Added synchronization monitoring for Istio in multi-cluster configurations. A new alert
D8IstioRemoteClusterNotSyncedhas been introduced and triggers in the following cases:- The remote cluster is offline.
- The remote API endpoint is not reachable.
- The remote
ServiceAccounttoken is invalid or expired. - There is a TLS or certificate issue between the clusters.
- The
deckhouse-controller collect-debug-infocommand now also collects debug information forIstio, including:- Resources in the
d8-istionamespace. - CRDs from the
istio.ioandgateway.networking.k8s.iogroups. Istiologs.Sidecarlogs of a single randomly selected user application.
- Resources in the
- A new monitoring dashboard has been added to display OpenVPN certificate status. Upon expiration, server certificates will now be reissued, and client certificates will be removed. The following alerts have been added:
OpenVPNClientCertificateExpired: Warns about expired client certificates.OpenVPNServerCACertificateExpired: Warns about an expired OpenVPN CA certificate.OpenVPNServerCACertificateExpiringSoonandOpenVPNServerCACertificateExpiringInAWeek: Warn when an OpenVPN CA certificate is expiring in less than 30 or 7 days, respectively.OpenVPNServerCertificateExpired: Warns about an expired OpenVPN server certificate.OpenVPNServerCertificateExpiringSoonandOpenVPNServerCertificateExpiringInAWeek: Warn when an OpenVPN server certificate is expiring in less than 30 or 7 days, respectively.
- Monitoring dashboards have been renamed and updated:
- “L2LoadBalancer” renamed to “MetalLB L2”; pool and column filtering added.
- “Metallb” renamed to “MetalLB BGP”; pool and column filtering added. The ARP request panel has been removed.
- “L2LoadBalancer / Pools” renamed to “MetalLB / Pools”.
-
The
upmetermodule’s PVC size has been increased to accommodate data retention for 13 months. In some cases, the previous PVC size was insufficient. -
The ModuleSource resource status now includes information about module versions in the source.
-
The Module resource status now includes information about the module’s lifecycle stage. A module can move through the following stages in its lifecycle: Experimental, Preview, General Availability, and Deprecated. For details on module lifecycle stages and how to evaluate its stability, refer to the corresponding section in the documentation.
-
It is now possible to use stronger or more modern encryption algorithms (such as
RSA-3072,RSA-4096, orECDSA-P256) for control plane cluster certificates instead of the defaultRSA-2048. You can use theencryptionAlgorithmparameter in the ClusterConfiguration resource to configure this. -
The
deschedulermodule can now be configured to evict pods that are using local storage. Use theevictLocalStoragePodsparameter in the module configuration to adjust this. - You can now adjust the logging level of the Ingress controller using the
controllerLogLevelparameter in the IngressNginxController resource. The default log level isInfo. Controlling the logging level can help prevent log collector overload during Ingress controller restarts.
Security
-
The severity level of alerts indicating security policy violations has been raised from 7 to 3.
-
The configuration for
Yandex Cloud,Zvirt, andBasis Dynamixproviders now usesOpenTofuinstead ofTerraform. This enables easier provider updates, such as applying fixes for known vulnerabilities (CVEs). -
CVE vulnerabilities have been fixed in the following modules:
chrony,descheduler,dhctl,node-manager,registry-packages-proxy,falco,cni-cilium, andvertical-pod-autoscaler.
The complete list of changes, including the updated components, is available in the changelog on GitHub.
1.70.19
Fixes
- [ci] Fix render workflow job.
- [istio] Reduce CPU and RAM for regenerate multicluster JWT token and sort ingressGateway
Chore
- [runtime-audit-engine] Update CI file
1.70.18
Fixes
- [istio] The metrics-exporter’s template is fixed, it blocked the main queue if
controlPlane.nodeSelectorsetting was configured.
1.70.17
Fixes
- [istio] fixed an error in processing custom input gateway addresses using the configuration option of the alliance.ingressGateway.advertise module.
1.70.16
Fixes
- [cloud-provider-aws] fix root_block_device converge
- [runtime-audit-engine] Improve memory footprint by switching to the stdlib memory allocator instead of jemalloc
1.70.15
Fixes
- [control-plane-manager] correct indentation of the ‘certSANs’ in v1beta4 template.
1.70.14
Fixes
- [cloud-provider-dvp] fix logic of work with disks and coreFraction validation
- [cloud-provider-huaweicloud] fix virtualMachineClassName in terraform
- [deckhouse] Filter deleted pvcs in storage class change hook.
- [deckhouse] Filter deleted pvcs in storage class change hook.
- [deckhouse-controller] make metrics more verbose
1.70.13
Know before update
- Users can now restore etcd backups and scale their master nodes efficiently without encountering previous errors.
Fixes
- [control-plane-manager] Fixed issues with etcd backup restoration process, ensuring reliable master node scaling from 1 to 3.
Users can now restore etcd backups and scale their master nodes efficiently without encountering previous errors. - [deckhouse-controller] fix nil annotations map for module fetcher
- [ingress-nginx] Validation is switched off temporarily.
- [prometheus] Fix fake longterm emptydir alert.
1.70.12
Features
- [cloud-provider-huaweicloud] add enterprise_project_id
Fixes
- [dhctl] TLS support on dhctl edition checking step.
1.70.11
Fixes
- [docs] remove Yandex Cloud’s api-gateway.editor and connection-manager.editor roles
1.70.10
Features
- [docs] Add unified manual about edition switch
Fixes
- [dhctl] Fix dhctl output in case of invalid config
- [dhctl] Release infrastructure dependency lock.
1.70.9
Fixes
- [control-plane-manager] Fix default behaviour for etcd backup.
- [ingress-nginx] fixed geoip issues in validator
- [ingress-nginx] Automatic geoip updating feature was reverted.
1.70.8
Fixes
- [docs] Delete the mention that addresses are allocated starting from 10 from internalNetworkCIDR in vcd cloud-provider
1.70.7
Know before update
- When called collect-debug-info command in cluster without Istio, user see errors, but archive create correct
Fixes
- [control-plane-manager] fix kubeadm v1beta4 config
- [deckhouse-controller] fix wrong behavior of Pending ModuleReleases
- [ingress-nginx] fixed
node-selectorandtolerationsin validatorpod - [istio] Fix for collecting debugging resources in collect-debug-info about Istio
When called collect-debug-info command in cluster without Istio, user see errors, but archive create correct - [node-local-dns] Improved stability for clusters with poor node-to-node connection quality by increasing node-local-dns to kube-dns healthcheck timeouts.
1.70.6
Fixes
- [control-plane-manager] audit-log-maxage value to 30 days
- [deckhouse] Fix obsolete config alert stuck after deleting module config.
- [deckhouse] Remove unnecessary modules downloading.
- [docs] Rendered SSHHost and SSHConfig specs.
- [ingress-nginx] fixed an issue with the snapshot retrieval mechanism
- [ingress-nginx] fix missing proxy client cert in validator deployment
- [ingress-nginx] Ingress resource validation in ingress-nginx controller re-enabled.
- [openvpn] Added a migration hook to mitigate false positive
OpenVPNClientCertificateExpiredalerts. - [user-authz] Add patch verb for service accounts
1.70.5
Know before update
- loki statefulset will be rollout restarted
Fixes
- [loki] Fix several bugs introduced while implementing loki disk based retention
loki statefulset will be rollout restarted - [prometheus] Add headers field to values spec
1.70.4
Know before update
- operator-prometheus
Features
- [cloud-provider-dvp] Add vmClassName to the config
- [operator-prometheus] Add to documetation for operator-prometheus CRD.
operator-prometheus - [snapshot-controller] Automatic snapshot-controller MC creation for future switch to external module
Fixes
- [cloud-provider-dvp] Fix cherry-pick conflicts pr
- [ingress-nginx] Ingress resource validation in ingress-nginx controller re-enabled.
1.70.3
Fixes
- [cloud-provider-yandex] Handle hybrid clusters case in the to_tofu_migrate_metric hook
- [cloud-provider-zvirt] fix UUID validation regex
- [deckhouse-controller] fixes default maintenance value which prevents stuck metric
- [deckhouse-tools] deckhouse-tools fix deployment render
- [docs] Updated conversions message.
- [node-local-dns] Changed container liveness port.
- [node-manager] fixed update for static clusters
1.70.2
Fixes
- [candi] memory.available calculation python2 fix
- [openvpn] Fixed condition of alert triggering for client certificate.
- [service-with-healthchecks] change port kube-rbac-proxy in agent DaemonSet
1.70.1
Know before update
- prometheus
Features
- [cloud-provider-dvp] Add vmClassName to the config
- [prometheus] Validation prometheus remotewrite ca webhook
prometheus
Fixes
- [candi] Added a check to the bashible step for the presence of a previously created symlink and where it leads to
- [candi] bashible bb-discover-node-name use node name from kubelet certificate
- [ci] returned the condition when the branch was not found
- [ci] fix kubernetes version info in the release file
- [cloud-provider-vcd] Fix unexpected behavior with legacy API versions in VCD
- [deckhouse-controller] fix deckhouse crds mechanism
- [dhctl] Fix unexpected behavior with legacy API versions in VCD
- [docs] Update docs about Keycloak configuration for IP
- [operator-trivy] Add support anon registry creds for bdu updater.
- [user-authz] fix error with webhook system-roles
Version 1.69
Important
-
Support for Kubernetes 1.32 has been added, while support for Kubernetes 1.27 has been discontinued. The default Kubernetes version has been changed to 1.30. In future DKP releases, support for Kubernetes 1.28 will be removed.
-
All DKP components will be restarted during the update.
Major changes
-
The
ceph-csimodule is now deprecated. Plan to migrate to thecsi-cephmodule instead. For details, refer to the Ceph documentation. -
You can now grant access to Deckhouse web interfaces using user names via the
auth.allowedUserEmailsfield. Access restriction is configured together with theauth.allowedUserGroupsparameter in configuration of the following modules with web interfaces:cilium-hubble,dashboard,deckhouse-tools,documentation,istio,openvpn,prometheus, andupmeter(example forprometheus). -
A new dashboard Cilium Nodes Connectivity Status & Latency has been added to Grafana in the
cni-ciliummodule. It helps monitor node network connectivity issues. The dashboard displays a connectivity matrix similar to thecilium-health statuscommand, using metrics that are already available in Prometheus. -
A new
D8KubernetesStaleTokensDetectedalert has been added in thecontrol-plane-managermodule that is triggered when stale service account tokens are detected in the cluster. -
You can now create a Project from an existing namespace and adopt existing objects into it. To do this, annotate the namespace and its resources with
projects.deckhouse.io/adopt. This lets you switch to using Projects without recreating cluster resources. -
A
Terminatingstatus has been added to ModuleSource and ModuleRelease resources. The new status will be displayed when an attempt to delete one of them fails. -
The installer container now automatically configures cluster access after a successful bootstrap. A
kubeconfigfile is generated in~/.kube/config, and a local TCP proxy is set up through an SSH tunnel. This allows you to use kubectl locally right away without manually connecting to the control-plane node via SSH. -
Changes to Kubernetes resources in multi-cluster and federation setups are now tracked directly via Kubernetes API. This enables faster synchronization between clusters and eliminates the use of outdated certificates. In addition, mounting of ConfigMap and Secret resources into Pods has been removed to eliminate family system compromise risks.
-
A new dynamicforward plugin has been added to CoreDNS, improving DNS query processing in the cluster. It integrates with
node-local-dns, continuously monitorskube-dnsendpoints, and automatically updates the list of DNS forwarders. If the control-plane node is unavailable, DNS queries are still forwarded to available endpoints, improving cluster stability. -
A new log rotation approach has been introduced in the
lokimodule. Now, old logs are automatically removed when disk usage exceeds a threshold: either 95% of PVC size or PVC size minus the size required to store two minutes of log data at the configured ingestion rate (ingestionRateMB). TheretentionPeriodHoursparameter no longer controls the data retention and is used for monitoring alerts only. Iflokibegins removing old logs before the set period is reached, aLokiRetentionPerionViolationalert will be triggered, informing the user that they must reduce the value ofretentionPeriodHoursor increase the PVC size. -
A new
nodeDrainTimeoutSecondparameter lets you set the maximum timeout when attempting to drain a node (in seconds) for each NodeGroup resource. Previously, you could only use the default value (10 minutes) or reduce it to 5 minutes using thequickShutdownparameter, which is now deprecated. -
The
openvpnmodule now includes adefaultClientCertExpirationDaysparameter, allowing you to define the lifetime of client certificates.
Security
- Known vulnerabilities have been addressed in the following modules:
ingress-nginx,istio,prometheus, andlocal-path-provisioner.
The complete list of changes, including the updated components, is available in the changelog on GitHub.
1.69.17
Know before update
- Users can now configure containerd to pull images from private registries, easing deployments in private environments.
Features
- [docs] Add command to view all falco audit rules to docs.
- [istio] Validating jwt token exp time less than 30 days
- [node-manager] Added support for configuring private HTTP container registries in containerd.
Users can now configure containerd to pull images from private registries, easing deployments in private environments.
Fixes
- [docs] Removed port 22322 from setup docs.
- [istio] Fix istiod CPU leak when the order of the ingressGateways array is changed
- [istio] Reduce RAM for regenerate multicluster JWT token
1.69.16
Fixes
- [control-plane-manager] audit-log-maxage value to 30 days
- [deckhouse] Fix obsolete config alert stuck after deleting module config.
1.69.15
Know before update
- Prevent important data loss when using wide retention intervals.
Fixes
- [loki] Fix incorrect indices sort function used in disk-based retention.
Prevent important data loss when using wide retention intervals.
1.69.14
Fixes
- [deckhouse-controller] fixes default maintenance value which prevents stuck metric
- [node-manager] fixed update for static clusters
1.69.13
Fixes
- [istio] The
alliance.ingressGateway.advertiseoption was fixed up.
1.69.12
Fixes
- [dashboard] fix display workload metrics on dashboard
1.69.11
Know before update
- In L2 mode, the Public IP address will not be marked as free in the pool after deleting the LoadBalancer Service. As a temporary fix, you can restart the MetalLB controller pods.
- fix static e2e tests for fe/upgrade
Fixes
- [ci] fix static e2e tests for fe/upgrade
fix static e2e tests for fe/upgrade - [istio] proxy-buffer-size increased in kiali Ingress.
- [metallb] Reverted the fix for handling deleted Services and revising the IP pools in L2 mode. The correct fix is under construction.
In L2 mode, the Public IP address will not be marked as free in the pool after deleting the LoadBalancer Service. As a temporary fix, you can restart the MetalLB controller pods.
1.69.10
Fixes
- [deckhouse-controller] Added validation according to
RFC 1123for module names added toModuleSource. - [docs] Fix apiVersion for s3 storageclass
1.69.9
Fixes
- [metallb] Fixed deprecated CRD addresspools.metallb.io deletion.
- [network-gateway] Fix python path
1.69.8
Fixes
- [cloud-provider-yandex] fix synchronizing TGs
- [deckhouse] Gracefully restore deployed modules on HA installations.
1.69.7
Fixes
- [docs] Fixed new layouts at documentation.
- [kube-dns] Fixed release upgrade issue with removed d8-kube-dns-redirect service.
1.69.6
Fixes
- [cni-cilium] Fixed race condition when deleting element from ipcache map during VM migration.
- [docs] Fix broken links in the Storage overview.
- [metallb] Fixed IP pool exhaustion on LoadBalancer deletion.
- [node-manager] Revert cluster autoscaler
1.69.5
Features
- [candi] Add rosa 12.6.
Fixes
- [cloud-provider-yandex] fix WithNATInstance clusters converge
- [control-plane-manager] Stale service account alert upd
- [node-manager] Autoscaler - remove additional cordon node by mcm provider.
1.69.4
Features
- [deckhouse-controller] sequential processing of module releases
- [deckhouse-controller] sequential processing of module releases
- [testing] Add e2e tests for autoscaler and enable it permanent for GCP.
- [testing] Add e2e tests for autoscaler and enable it permanent for GCP
Fixes
- [candi] Return support
.cri.notManaged.criSocketPath. - [deckhouse] Restore only ready mpo.
- [dhctl] <Disable caching metaconfig during converge and disable converge deckhouse manifests.
- [dhctl] Fix incorrect behavior that fails sudoers preflight check if password contains whitespaces.
- [docs] Fixed Dynamix layout at documentation.
- [docs] Add support of
x-doc-d8Editionsarray instead ofx-doc-d8Revisionfield to JSON schema renderer. - [node-manager] Increase verbosity and revert versions and fix bug with unsupported mcm annotation
- [upmeter] upmeter sa token rotation
- [user-authn] Add separate ServiceAccount to basic-auth-proxy.
- [user-authz] Exclude deckhouse clusterrolebindings from webhook.
1.69.3
Know before update
- Ingress-nginx controller pods of v1.9 will be restated.
Fixes
- [control-plane-manager] Stale service account alert fix.
- [deckhouse] Limit module name.
- [deckhouse] Fix mpo/source requeue.
- [docs] Add info about debug GrafanaDashboardDefenition.
- [extended-monitoring] Fix x509_certificate_exporter_resources deployment
- [ingress-nginx] All necessary shared libraries are added to the container image.
Ingress-nginx controller pods of v1.9 will be restated. - [kube-dns] Expanded pattern for
stubZonesto allow underscores (_) in domain names. - [user-authn] Fix secret generation on empty data field in the dex client app secret.
1.69.2
Know before update
- the grafana deployment will be rollout restarted
Fixes
- [deckhouse] Gracefully restore deployed modules on HA installations.
- [prometheus] enable WAL for the grafana SQLite database to prevent locking errors, thus fixing in-dashboard alerting.
the grafana deployment will be rollout restarted
1.69.1
Know before update
- fixes the bug introduced in v1.69.0
- release upgrade will be blocked on AWS-based clusters where SA doesn’t have DescribeAddressesAttribute and DescribeInstanceTopology roles. They are required for new Terraform AWS Provider version.
Features
- [deckhouse] enable UnmetCloudConditions check
release upgrade will be blocked on AWS-based clusters where SA doesn’t have DescribeAddressesAttribute and DescribeInstanceTopology roles. They are required for new Terraform AWS Provider version. - [deckhouse-controller] rename module flag
managementStatetomaintenance. - [dhctl] add variable time range to request counter
- [node-manager] enable UnmetCloudConditions check
- [node-manager] UnmetCloudConditions requirement and alert
Fixes
- [ci] Use fixed werf version.
- [cloud-provider-vcd] Add a hook to set
legacyModebased on the detectedVCD APIversion - [deckhouse] Clear deployed release`s message.
- [deckhouse] Delete deckhouse epslices.
- [deckhouse-controller] Fix runtime handling for the global config.
- [dhctl] fix bootstrap and abort config preparation
- [docs] Fix d8 install commands.
- [loki] fix storage capacity calculator hook for Loki
fixes the bug introduced in v1.69.0 - [operator-trivy] Add proxy env variables support to the trivy server.
Version 1.68
Important
- After the update, the UID will change for all Grafana data sources created using the GrafanaAdditionalDatasource resource. If a data source was referenced by UID, that reference will no longer be valid.
Major changes
-
A new parameter,
iamNodeRole, has been introduced for the AWS provider. It lets you specify the name of the IAM role to bind to all AWS instances of cluster nodes. This can come in handy if you need to grant additional permissions (for example, access to ECR, etc.). -
Creating nodes of the CloudPermanent type now takes less time. Now, CloudPermanent nodes are created in parallel. Previously, they were created in parallel only within a single group.
- Monitoring changes:
- Support for monitoring certificates in secrets of the
Opaquetype has been added. - Support for monitoring images in Amazon ECR has been added.
- A bug that could cause partial loss of metrics when Prometheus instances were restarted has been fixed.
- Support for monitoring certificates in secrets of the
-
When using a multi-cluster Istio configuration or federation, you can now explicitly specify the list of addresses used for inter-cluster requests. Previously, these addresses were determined automatically; however, in some configurations, they could not be resolved.
-
The DexAuthenticator resource now has a
highAvailabilityparameter that controls high availability mode. In high availability mode, multiple replicas of the authenticator are launched. Previously, high availability mode of all authenticators was determined by a global parameter or by theuser-authnmodule. All authenticators deployed by DKP now inherit the high availability mode of the corresponding module. -
Node labels can now be added, removed, or modified using files stored on the node in the
/var/lib/node_labelsdirectory and its subdirectories. The full set of applied labels is stored in thenode.deckhouse.io/last-applied-local-labelsannotation. -
Support for the Huawei Cloud provider has been added.
-
The new
keepDeletedFilesOpenedForparameter in thelog-shippermodule allows you to configure the period to keep the deleted log files open. This way, you can continue reading logs from deleted pods for some time if log storage is temporarily unavailable. -
TLS encryption for log collectors (Elasticsearch, Vector, Loki, Splunk, Logstash, Socket, Kafka) can now be configured using secrets, rather than by storing certificates in the ClusterLogDestination resources. The secret must reside in the
d8-log-shippernamespace and have thelog-shipper.deckhouse.io/watch-secret: truelabel. -
In the project status under the
resourcessection, you can now see which project resources have been installed. Those resources are marked withinstalled: true. - A new parameter,
--tf-resource-management-timeout, has been added to the installer. It controls the resource creation timeout in cloud environments. By default, the timeout is set to 10 minutes. This parameter applies only to the following clouds: AWS, Azure, GCP, OpenStack.
Security
Known vulnerabilities have been addressed in the following modules:
admission-policy-enginechronycloud-provider-azurecloud-provider-gcpcloud-provider-openstackcloud-provider-yandexcloud-provider-zvirtcni-ciliumcontrol-plane-managerextended-monitoringdeschedulerdocumentationingress-nginxistiolokimetallbmonitoring-kubernetesmonitoring-pingnode-manageroperator-trivypod-reloaderprometheusprometheus-metrics-adapterregistrypackagesruntime-audit-engineterraform-manageruser-authnvertical-pod-autoscalerstatic-routing-manager
Mandatory component restart
The following components will be restarted after updating DKP to 1.68:
- Kubernetes Control Plane
- Ingress controller
- Prometheus, Grafana
admission-policy-enginechronycloud-provider-azurecloud-provider-gcpcloud-provider-openstackcloud-provider-yandexcloud-provider-zvirtcni-ciliumcontrol-plane-managerdeschedulerdocumentationextended-monitoringingress-nginxistiokube-state-metricslog-shipperlokimetallbmonitoring-kubernetesmonitoring-pingnode-manageropenvpnoperator-trivyprometheusprometheus-metrics-adapterpod-reloaderregistrypackagesruntime-audit-engineservice-with-healthchecksstatic-routing-managerterraform-manageruser-authnvertical-pod-autoscaler
The complete list of changes, including the updated components, is available in the changelog on GitHub.
1.68.15
Fixes
- [dhctl] Do not create a local-converger lock for commanderMode
- [docs] Fixed the parameter generator for modules on Hugo.
- [docs] This reverts commit cf9d7590da7c4b23c7eab10da37abc16397faf87.
1.68.14
Fixes
- [deckhouse-controller] fix module release force behaviour
- [deckhouse-controller] fix module release force behaviour
- [runtime-audit-engine] Fix CrashLoopBack due to invalid config.
1.68.13
Know before update
- Grafana deployment will be rollout restarted.
Fixes
- [prometheus] Enables WAL for Grafana SQLite database to prevent locking errors, thus fixing in-dashboard alerting.
Grafana deployment will be rollout restarted.
1.68.12
Fixes
- [ci] Use fixed werf version.
- [cloud-provider-zvirt] fix invalid zvirt csi template
- [node-local-dns] Stale-dns-connection-cleaner file permissions and security context are fixed.
1.68.11
Fixes
- [cloud-provider-zvirt] fix zvirt-csi-driver patching (token refresh fix patch)
- [node-manager] partial rollback of changes in images cluster-autoscaler
1.68.10
Features
- [deckhouse-controller] restore releases only by deckhouse-controller and in deployed state
Fixes
- [deckhouse-controller] fix suspend release
- [node-manager] handle_node_template hook ignore metallb label
- [runtime-audit-engine] copy falcosecurity_falcosidekick_falco_events_total metric to falco_events for backward compatibility
1.68.9
Fixes
- [node-manager] partial rollback of changes in images cluster-autoscaler
1.68.8
Features
- [cert-manager] Bump cert-manager version up to 1.17.1. This fixes acme confirmation for cloudflare provider.
Fixes
- [docs] FIx local docimentation build and run.
- [docs] The information label with the editions has been removed from the global settings page.
- [ingress-nginx] Fixed the display of ports in the examples in the CRDs.
- [istio] Fixed the display of ports in the examples in the CRDs.
1.68.7
Know before update
- Ingress-nginx pods will restart. The validation of the generated NGINX configuration during the validation of
Ingressresources is disabled until finding a way of running validations safely.
Fixes
- [ingress-nginx] latest CVE fixes backported.
Ingress-nginx pods will restart. The validation of the generated NGINX configuration during the validation ofIngressresources is disabled until finding a way of running validations safely.
1.68.6
Fixes
- [candi] Fix errors in bashible steps.
- [deckhouse] Fix erroneous inclusions in 1.68
- [deckhouse] Fix module config ignored.
- [deckhouse] Fix checking automatic kubernetes version in managed clusters.
- [deckhouse] Fix module config ignored.
- [deckhouse] Explicitly set ports in d8-kube-dns deployment.
- [deckhouse] Set EnabledByModuleConfig condition by default for modules without this condition.
- [deckhouse] Fix module condition race.
- [deckhouse-controller] Fix applying releases and their metrics.
- [deckhouse-controller] Fix applying releases and their metrics.
- [deckhouse-tools] d8-cli build fix
- [docs] Corrected a typo in the NFSStorageClass name in the NFS data storage section.
- [docs] Added SE and SE+ restrictions for Istio.
- [log-shipper] Add delete parsed_data to final kafka transform render
1.68.5
Features
- [prometheus] add grafana enable flag
Fixes
- [candi] Fix
disable_unattended_upgradesbashible step - [docs] Correct mistakes in the crds for the Storage section in Deckhouse Kubernetes Platform.
- [docs] Update Deckhouse CLI installation guide
- [docs] Fixed sidebar behaviour with links.
- [docs] Correct mistakes in the documentation for the storage section in Deckhouse Kubernetes Platform.
- [extended-monitoring] Minor certificate alerts improvements
- [metallb] Add clearing of alert metric group D8MetallbObsoleteLayer2PoolsAreUsed
- [operator-trivy] Updated info about databases with vulnerabilities.
1.68.4
Fixes
- [candi] Fix add_node_users.sh search for sudoersd_path.
- [deckhouse-controller] Fix DeckhouseRelease approval in the Manual update mode.
- [dhctl] Escape special shell characters in sudo preflight check.
- [docs] Fixed typos at VMware configuration parameters.
- [upmeter] Add D8UpmeterDiskUsage alert.
1.68.3
Fixes
- [deckhouse] Add initial module config validation.
- [deckhouse-controller] Update addon-operator.
- [dhctl] sudoPassword from the connection-config not used in dhctl CLI and dhctl server
- [metallb] Fixed module startup error with empty Selector value in SVC.
1.68.2
Know before update
- The links in the alerts will be correct when the documentation module is enabled.
Fixes
- [admission-policy-engine] Fix wildcard apparmor profile.
- [deckhouse-controller] fix deckhouse-release check loop
- [deckhouse-tools] Fix tools links pointing to wrong path
- [metallb] Fixed alerts for EE edition.
- [monitoring-applications] Fix module dependencies. The
monitoring-applicationsmodule was affected and disabled by the enabled script. - [node-local-dns] Fix module dependencies. The
monitoring-applicationsmodule was affected and disabled by the enabled script. - [prometheus] Changes the links in alert’s template about alerts from url to relative links
The links in the alerts will be correct when the documentation module is enabled. - [service-with-healthchecks] Fixed several issues.
- [upmeter] Fixed metallb module probe.
1.68.1
Know before update
- kubernetes-api will be restarted.
- okmeter daemonset pods will be rollout restarted
Features
- [candi] Add a new layout for HuaweiCloud provider with support for VPC peering connections.
- [docs] Add new description info for conversions.
Fixes
- [candi] Fix update_node_labels.sh to work on python2.
- [candi] Reduce
request-timeoutin kubernetes-api to 60s.
kubernetes-api will be restarted. - [ci] CI_COMMIT_TAG does not found in deckhouse controller
- [ci] fix standalone-installer build
- [ci] cloud providers CVE fixes, use single images for cloud-provider-zvirt and cloud-provider-yandex
- [cloud-provider-vcd] fix CVE, bump ccm, use go 1.23, use werf images and src-artifact
- [deckhouse] Fix stuck overridden status in ms.
- [deckhouse] Fix module.yaml ignored when weight == 0.
- [node-manager] fix autoscaler RBAC for zvirt and dynamix
- [okmeter] drop service account fields from the okmeter daemonset manifest
okmeter daemonset pods will be rollout restarted - [user-authn] Fix cronjob security context.