Patch release descriptions are generated automatically. To expand a description, click the version number.

Version 1.78

Important

  • The v1 implementation of the container registry management, used up to and including DP 1.77 (the Proxy and Local modes set in the registry section of the deckhouse ModuleConfig) has been removed. A cluster can no longer be installed in these modes, and the update to DP 1.78 is blocked while the cluster runs Proxy or Local, or Direct without the registry ModuleConfig configured. Migrate the cluster to the registry module on DP 1.77 before updating; the procedures are described in the module FAQ. Update to DP 1.78 only from DP 1.77.1 or later: that is the first release that checks which registry implementation the cluster runs. An update that skips it, for example an LTS release hop or an update constraint from an earlier release, can install DP 1.78 on a cluster still in Proxy or Local mode, which is not supported.

  • In Deckhouse Commander, checking, converging and deleting a cluster whose stored resources still set registry.mode to Proxy or Local in the deckhouse ModuleConfig now stop with an error and change nothing. Before these operations, bring the resources in line with the cluster: replace the deckhouse ModuleConfig in them with the cluster’s own and add the cluster’s registry ModuleConfig, if it has one.

Version 1.77

Important

  • Support for Kubernetes 1.36 has been added, while support for Kubernetes 1.31 has been discontinued. The default Kubernetes version (used when the kubernetesVersion parameter is set to Automatic) has been changed to 1.34.

  • Control plane components will be restarted during the update.

  • All Ingress NGINX Controller pods will be restarted.

  • If the enableMultiTenancy setting of the user-authz module is enabled, existing RoleBinding and ClusterRoleBinding objects that were previously ignored because of ClusterAuthorizationRule limitations become effective after the update. Before updating, review the RoleBinding and ClusterRoleBinding objects assigned to users, and modify or remove those that should not grant access.

  • If the DP cluster uses custom roles based on the legacy RBACv2 scheme, the D8UserAuthzLegacyRBACv2CustomRoleFound alert will fire after the update. Such roles must be migrated to the new scheme before updating to DP 1.78, otherwise the update will be blocked. The migration procedure is described in the user-authz module FAQ.

  • If the cluster uses the v1 implementation of the container registry management (the registry section of the deckhouse ModuleConfig) in the Proxy or Local mode, or in the Direct mode without the registry ModuleConfig configured, it must be migrated to the registry module before updating to DP 1.78, otherwise the update will be blocked. The migration procedures are described in the registry module FAQ.

  • If the metallb module configuration is managed through GitOps, temporarily disable synchronization from your repository before the update. After the update, replace the BGP configuration in your repository from ModuleConfig with the MetalLoadBalancerConfiguration, MetalLoadBalancerBGPPeer, and MetalLoadBalancerPool custom resources, and then re-enable synchronization.

  • If the cluster uses NetworkPolicy objects with the ipBlock: 0.0.0.0/0 rule, after the update they will also apply to intra-cluster traffic. It is recommended that you review such policies before the update and adjust them if necessary.

  • In the AWS, Azure, and GCP providers, the default CNI has been changed to Cilium. Before creating or updating a cluster, make sure that the Linux kernel version is 5.8 or later and that network policies and firewalls do not block the UDP ports required for Cilium to operate in VXLAN mode. Failure to meet these requirements may result in cluster creation or update failures, or in loss of network connectivity between nodes. For detailed requirements, refer to “Network interaction”.

  • The vSphere, Huawei Cloud, OpenStack, and VCD providers have been migrated from Terraform to OpenTofu. The migration is performed automatically and requires no manual actions.

  • The Yandex Cloud and OpenStack providers have been migrated from Machine Controller Manager (MCM) to Cluster API (CAPI). The management engine is chosen per CloudEphemeral node group. Groups that already exist keep running on MCM, and new groups are created on CAPI by default, so a single cluster can contain groups of both kinds. To move an existing group, refer to “How to migrate node groups to Cluster API (CAPI)”.

  • The following modules are now loaded from external sources and are developed on an independent release cycle (no manual migration actions are required):

    • Monitoring modules:

      • extended-monitoring
      • log-shipper
      • loki
      • monitoring-custom
      • monitoring-deckhouse
      • monitoring-kubernetes
      • monitoring-ping
      • okmeter
      • operator-prometheus
      • prometheus
      • prometheus-metrics-adapter
      • prometheus-pushgateway
      • upmeter
    • The ingress-nginx module.

  • The monitoring-kubernetes-control-plane module has been merged into the control-plane-manager module. Monitoring of control plane components is now performed by the control-plane-manager module.

Deckhouse subsystem

  • A policy has been added to the deckhouse module that prevents non-system users from setting or modifying the heritage label. This prevents accidental creation of resources that DP may recognize as its own.

  • The reliability of updating clusters that use external modules (ModuleSource) has been improved. The update starts only after the modules have been preloaded, preventing their temporary unavailability during the update.

  • Support has been added to the registry module for installing a cluster with the container registry operating in Local mode. This makes it possible to deploy static clusters without access to an external registry, including in air-gapped environments.

Kubernetes & Scheduling subsystem

  • Changes in the descheduler module:

    • Added the descheduler_pod_evictions_total metric with information about the workload to which the evicted pod belongs.

    • Added the “Workload Happiness” Grafana dashboard, which displays the workload happiness score, workload distribution across nodes, and the module activity history (evictions, strategy execution, and timeline).

  • Changes in the vertical-pod-autoscaler module:

    • Added the VerticalPodAutoscalerDeprecatedUpdateModeAuto alert and the “VPA Deprecated Update Mode” Grafana dashboard to detect VerticalPodAutoscaler objects that use the deprecated updateMode: Auto mode.

    • Improved the reliability of the vertical-pod-autoscaler webhook deployment. Fixed a component startup ordering issue that could cause errors during updates.

  • Changes in the control-plane-manager module:

    • The module has been migrated to a new internal implementation for managing control plane components. Added the new ControlPlaneNode and ControlPlaneOperation custom resources to track node state and control plane component update operations.

    • Added automatic etcd defragmentation, which is enabled by default for multi-master clusters and runs when fragmentation exceeds 40%. A defragmentation schedule can be configured in the module settings.

    • Several settings have been moved to the module ModuleConfig. Automatic migration and alerts notifying about the need to update the configuration have also been added:

  • Added a patch to kube-scheduler that prevents scheduling new pods onto nodes that are undergoing graceful shutdown.

  • Accelerated etcd updates in single-master clusters. Eliminated control plane component downtime that occurred during updates.

  • Reduced the risk of kube-controller-manager throttling under high controller load.

  • Accelerated pod startup by enabling parallel pulling of multiple container images.

  • Enabled the KubeletEnsureSecretPulledImages feature gate by default in the kubelet configuration to comply with CIS Benchmark requirements.

IAM subsystem

  • Changes in the user-authn module:

    • Authentication protection in Dex has been strengthened: rate limiting has been added for authentication endpoints, LDAP/Crowd accounts are now automatically blocked after a series of failed sign-in attempts, and external accounts can now be blocked and unblocked manually.

    • The UserOperation resource now supports permanent user locking (spec.lock.for: permanent). Audit logs for account operations have also been extended: they now include information about the operation initiator, operation type, and target user.

    • Added the DexProviderCheck resource for verifying authentication providers (LDAP, OIDC, and others). It allows connection and credential issues to be diagnosed before they affect user sign-in.

    • Added the Custom complexity level to the password policy settings, allowing password requirements to be configured flexibly according to corporate standards.

    • Added support for specifying custom labels and annotations in the new spec.podMetadata field of the DexAuthenticator custom resource, for example, annotations for managing the Istio sidecar container.

    • DexAuthenticator now works correctly in namespaces with ResourceQuota and policies that require resource limits for all containers. Fixed an issue that could prevent the authenticator pod from starting.

  • Changes in the user-authz module:

    • The RBACv2 role model aggregation label scheme changes in DP 1.78. Added the D8UserAuthzLegacyRBACv2CustomRoleFound alert, which detects custom roles based on the legacy scheme. Such roles must be migrated to the new scheme before updating to DP 1.78; otherwise, they will stop working and the update will be blocked. The migration procedure is described in the module FAQ.

    • If the enableMultiTenancy setting is enabled, user permissions are now determined by the combination of ClusterAuthorizationRule, AuthorizationRule, and standard RBAC rules (RoleBinding/ClusterRoleBinding). After the update, existing RBAC bindings that were previously ignored because of ClusterAuthorizationRule limitations may become effective. It is recommended to review such bindings before the update. For details, refer to “Using ClusterAuthorizationRule and AuthorizationRule together with RBAC”.

    • Added the D8UserAuthzPermissionBrowserUnavailable alert, which notifies about the unavailability of the user permissions browsing service (permission-browser-apiserver).

Security subsystem

  • Changes in the admission-policy-engine module:

    • SecurityPolicy exceptions (SecurityPolicyException) can now be configured at the individual container level instead of only for the entire pod. This allows security requirements to be relaxed for a specific sidecar or init container without removing restrictions from the entire pod. For details, see the module documentation.

    • Added the built-in ValidatingAdmissionPolicy deny-deckhouse-finalizers.deckhouse.io, which prevents manual removal of deckhouse.io finalizers from resources. This prevents incorrect object deletion and the creation of orphan resources.

  • Added a mechanism to the multitenancy-manager module for granting cluster-wide resources to projects. DP administrators can now define which cluster resources are available to each project and specify default values for them. The mechanism can be used by any DP module to publish its own cluster resources.

  • Added the ability to list resources across all accessible namespaces without cluster-wide permissions. The d8 k get <resource> -A --scope=<accessible|projects|system|project:NAME> command returns only the objects the user has access to instead of returning a 403 Forbidden error. For details, see “Resource listing with access-based filtering”.

  • ACL filtering of the namespace list now supports watch requests. The d8 k get ns -w command displays changes in real time without requiring the command to be re-run.

  • The cainjector component of the cert-manager module is now enabled automatically when resources using cert-manager CA injection are detected. It no longer needs to be enabled manually or kept running at all times.

Cluster & Infrastructure subsystem

  • Changes in dhctl (CLI installer):

    • Added a preflight check for disk name length in cloud providers. If the name exceeds the allowed limit, the installation is aborted before infrastructure provisioning begins.

    • Improved the interactive mode of dhctl: added a dedicated area for logs of current operations, and made the output more compact by displaying progress instead of a stream of service messages. Full logs are still available in debug mode and during non-interactive execution.

    • Improved the preflight check for the deckhouse user. Repeated bootstrap now correctly handles an existing user, and additional checks help prevent permission-related issues.

    • Fixed issues with cleaning up temporary processes and SSH connections after running the converge and destroy commands.

    • Fixed the dhctl lock release command in interactive mode.

    • Fixed issues in the dhctl converge-migration command that could cause migration failures in certain scenarios.

    • Improved the reliability of control plane converge operations. Checks are now performed in a more appropriate order, and etcd status messages have become more informative.

  • Changes in the node-manager module:

    • Added Instance API v1alpha2. Instance status and error information is now consolidated into a single resource, simplifying diagnostics.

    • Reduced the wait time for MachineDeployment creation for AWS Spot instances. If AWS cannot allocate Spot instances because of insufficient capacity, retries begin sooner.

    • Added the spec.kubelet.seccompDefault parameter to the NodeGroup resource, allowing the RuntimeDefault seccomp profile to be enabled by default for all workloads without explicit seccomp settings.

  • Reduced the probability of false restarts of the CSI Controller in all cloud provider modules during short delays by increasing the liveness probe timeout.

  • Changes in the cloud-provider-dvp module:

    • Added support for live disk migration between StorageClass objects without stopping the virtual machine. Migration is performed automatically when the StorageClass changes.

    • The module configuration has been migrated to the unified ModuleConfig contract. Permanent and Ephemeral nodes are now configured in the same way through InstanceClass and NodeGroup, without a separate ProviderClusterConfiguration. This simplifies cluster configuration, enables schema versioning and conversion without breaking compatibility, and prepares cloud providers for an independent release cycle outside the DP core. For details, refer to “Cluster and Infrastructure” in the DP FAQ.

    • Fixed an issue that could leave LoadBalancer Services in the Pending state without creating the parent load balancer. Improved conflict handling during ServiceWithHealthchecks updates and load balancer status synchronization.

    • StorageClass objects created through Cluster API clusters now always use the WaitForFirstConsumer mode regardless of the parent cluster configuration. During updates, managed StorageClass objects with an incompatible volumeBindingMode are also recreated automatically. This eliminates an issue that could block cluster updates and synchronization because of the immutable volumeBindingMode field.

  • The cloud-provider-openstack, cloud-provider-huaweicloud, cloud-provider-vsphere, and cloud-provider-vcd modules now use OpenTofu instead of Terraform. This makes it possible to use an open infrastructure platform compatible with Terraform, reduces dependency on Terraform licensing changes, and aligns these providers with other cloud provider modules that already use OpenTofu.

  • The OpenStack and Yandex Cloud providers now use Cluster API, the industry-standard framework for machine management in Kubernetes. To ensure a smooth transition, the legacy mode based on Machine Controller Manager (MCM) remains supported. Existing node groups keep running on MCM, while new ones are created on Cluster API.

  • Improved virtual machine deletion handling in the cloud-provider-huaweicloud module, increasing resource cleanup reliability and preventing false deletion errors. Virtual machines that remain in the deleting state for more than 24 hours are now marked with the corresponding status, simplifying diagnostics.

  • Updated the CSI plugin and CSI Syncer versions in the cloud-provider-vsphere module, and fixed known vulnerabilities in Cloud Controller Manager, the CSI plugin, and Terraform Manager. This improves integration security and ensures the use of up-to-date component versions.

  • Added support for the vcd.cpi.flant.com/load-balancer-ip annotation for LoadBalancer Services in the cloud-provider-vcd module. It allows assigning a predefined external IP address to the load balancer. If both the annotation and the spec.loadBalancerIP field are specified, the annotation takes precedence.

Network subsystem

  • Changes in the istio module:

    • Added support for Istio 1.27. Users now have access to the new features of the latest Istio release using an operatorless mode.

    • Expanded ambient mesh support. New settings provide more explicit control over ambient mesh, allow using waypoint proxies for Layer 7 traffic processing, and enable the HBONE protocol for encrypting traffic between pods without requiring a sidecar container to run in every pod.

  • Changes in the cni-cilium module:

    • Changed the behavior of NetworkPolicy objects with the ipBlock: 0.0.0.0/0 rule in the new module version. The rule now also applies to intra-cluster traffic.

    • Reduced CPU consumption by Cilium agents. The freed resources become available to user workloads.

  • Changes in the ingress-nginx module:

    • Added support for IngressNginxController v1.15.

    • Updated the load calculation formula for Ingress NGINX Controller pods used for horizontal scaling of LoadBalancer, LoadBalancerWithProxyProtocol, and LoadBalancerWithSSLPassthrough inlets. The controller now adds and removes replicas in a timely manner, resulting in more stable behavior under load.

    • Updated the IngressNginxController API to v2. The spec.resourcesRequests resource consumption settings have been replaced with the extended spec.resourcesManagement settings, including support for resource limits. Compatibility with the v1 API is preserved automatically.

    • Implemented isolated configuration validation, making it possible to restore full NGINX configuration validation without reducing the security level.

  • Added authorization for the Hubble web interface in the cilium-hubble module. Access to cluster network telemetry can now be restricted by user groups to improve security.

  • In the metallb module, BGP mode configuration has been moved from ModuleConfig to the dedicated MetalLoadBalancerPool, MetalLoadBalancerBGPPeer, and MetalLoadBalancerConfiguration custom resources. The existing configuration will be migrated automatically, so no manual changes in the cluster are required. All new BGP settings must be configured through these resources.

The complete list of changes, including the updated components, is available in the changelog on GitHub.

1.77.8

Know before update

  • Dex pods restart. A client of Dex that expects HTTP 429 on the refresh of a locked user, HTTP 500 for a deleted one, or invalid_request for a claimed or expired refresh token gets HTTP 400 invalid_grant. The password and token exchange grants and the sign-in page answer as before.
  • Existing clusters are unaffected; the default zone set (ru-central1-a, ru-central1-b, ru-central1-d, ru-central1-e) is statically pinned. To use ru-central1-f, list it in zones instead of one of the default zones; the dropped zone’s subnet is replaced by a new subnet in ru-central1-f and the nodes in it are recreated.
  • containerd restarts on all nodes. On nodes where the old AppArmor profile is already loaded, the fix applies after a node reboot.

Features

  • [cloud-provider-yandex] Add ru-central1-f zone support via explicit zone selection with a four-zone limit.
  • [cloud-provider-yandex] Add ru-central1-f zone support via explicit zone selection with a four-zone limit.
    Existing clusters are unaffected; the default zone set (ru-central1-a, ru-central1-b, ru-central1-d, ru-central1-e) is statically pinned. To use ru-central1-f, list it in zones instead of one of the default zones; the dropped zone’s subnet is replaced by a new subnet in ru-central1-f and the nodes in it are recreated.

Fixes

  • [admission-policy-engine] Pods with a Localhost AppArmor profile set in securityContext are no longer denied by the Baseline policy.
  • [ci] Pass the werf meta repo to the dev site and documentation deploy jobs, matching the builds.
  • [common] Updated golang.org/x/net, golang.org/x/text, google.golang.org/grpc and github.com/klauspost/compress in the Kubernetes 1.31 CSI sidecars of CSE to fix CVEs.
  • [control-plane-manager] Fixed ControlPlaneOperation failures on clusters whose root CAs have a non-default subject.
  • [dhctl] The bootstrap terminal no longer shows glued lines, FAILED for bashible step attempts that recovered, or “Unhandled Error” from client-go.
  • [docs] Fix highlighting the L2 menu on module pages in the documentation.
  • [registrypackages] Fix AppArmor denying signals between processes of one container on Ubuntu kernels 7.0.0-41 and later.
    containerd restarts on all nodes. On nodes where the old AppArmor profile is already loaded, the fix applies after a node reboot.
  • [user-authn] Dex answers the refresh of a locked user, of a user deleted from the local store or LDAP, and of a claimed or expired refresh token with invalid_grant (HTTP 400), so that applications and a Dex that signs in through this one send the user to sign in.
    Dex pods restart. A client of Dex that expects HTTP 429 on the refresh of a locked user, HTTP 500 for a deleted one, or invalid_request for a claimed or expired refresh token gets HTTP 400 invalid_grant. The password and token exchange grants and the sign-in page answer as before.

1.77.7

Know before update

  • A cluster that uses Proxy or Local of the previous registry implementation, or Direct without the registry ModuleConfig, is not updated to DP 1.78, as before. This now also applies from the moment such a mode is set in the deckhouse ModuleConfig, before the switch starts. Migrate as described in “Migrating from the previous implementation to the current one” in the registry documentation. Clusters already prepared for the migration are not affected.
  • A node that joins while a registry-packages-proxy pod of the previous release is still running on a master waits until a pod of this release answers.
  • Cilium agent pods will be restarted.
  • Flannel pods will be restarted.
  • On a NodeGroup with spec.gpu, a failing GPU check used to abort the bashible run and be retried forever, so the node never configured kubelet and never received updates from any other module. Such a node now converges and only its GPU stays unusable. If /usr/bin/nvidia-container-runtime is absent, containerd is now configured with runc as the default runtime instead of nvidia; on a node whose configuration previously selected nvidia this changes the containerd config and therefore restarts containerd. Previously such a node could not start any container at all.
  • Pods of the kube-proxy, cni-cilium, cni-flannel, and cni-simple-bridge agent DaemonSets will be restarted because their cni-migration-init-checker init container image is updated.
  • The controller and agent pods will be restarted.
  • The registry in-cluster proxy and node services pods will be restarted.
  • The terraform-manager pods will be restarted.
  • istiod, istio-cni, ingress gateways, Kiali, api-proxy and metadata-exporter pods will be restarted.
  • node-controller, capi-controller-manager, machine-controller-manager and bashible-apiserver do not run on a master tainted with a key from global.modules.placement.customTolerationKeys. Remove such taints from the master NodeGroup before the update.
  • simple-bridge pods will be restarted.

Features

  • [docs] Implement glossary with term linking.
  • [docs] Added the “System requirements” page with deployment options and resource requirements for containers, virtual machines and combined workloads.
  • [node-manager] Alert when a NodeGroup uses the deprecated built-in GPU support while the gpu module is disabled.

Fixes

  • [candi] A GPU problem no longer blocks node convergence.
    On a NodeGroup with spec.gpu, a failing GPU check used to abort the bashible run and be retried forever, so the node never configured kubelet and never received updates from any other module. Such a node now converges and only its GPU stays unusable. If /usr/bin/nvidia-container-runtime is absent, containerd is now configured with runc as the default runtime instead of nvidia; on a node whose configuration previously selected nvidia this changes the containerd config and therefore restarts containerd. Previously such a node could not start any container at all.
  • [candi] GPU compute capability check no longer requires bc.
  • [candi] Persist net.core.bpf_jit_harden for GPU nodes in an own sysctl drop-in instead of editing distribution files that may be absent.
  • [candi] bashible verifies the rpp-get binary against the image digest from its bundle before running it; a substituted bootstrap proxy on a master can no longer hand a binary to the nodes.
    A node that joins while a registry-packages-proxy pod of the previous release is still running on a master waits until a pod of this release answers.
  • [candi] Fixed an intermittent bootstrap hang where kube-apiserver never started because kubelet read its manifest while it was being written.
  • [candi] A new cloud node sets up the provider network before it installs bootstrap packages, so it can download them from a master in another zone.
  • [chrony] Fixed known vulnerabilities in chrony-exporter.
  • [cloud-provider-dvp] Fixes DVP CSI staging so volumes with filesystems detected by Kubernetes SafeFormatAndMount are preserved and unformatted volumes are formatted without using lsblk.
  • [cloud-provider-dvp] Creates ModuleConfig/cloud-provider-dvp automatically when it is missing.
  • [cni-cilium] The ipv4-options-guard sidecar no longer writes to the host /etc, so Cilium agents start on Deckhouse Engine nodes.
    Cilium agent pods will be restarted.
  • [cni-flannel] The ipv4-options-guard sidecar no longer writes to the host /etc, so Flannel pods start on Deckhouse Engine nodes.
    Flannel pods will be restarted.
  • [cni-simple-bridge] The ipv4-options-guard sidecar no longer writes to the host /etc, so simple-bridge pods start on Deckhouse Engine nodes.
    simple-bridge pods will be restarted.
  • [common] Fixed known vulnerabilities in the cni-migration-controller and cni-migration-init-checker images.
    Pods of the kube-proxy, cni-cilium, cni-flannel, and cni-simple-bridge agent DaemonSets will be restarted because their cni-migration-init-checker init container image is updated.
  • [common] added mountpoint in ipv4-options-guard
  • [common] Updated golang.org/x/net, golang.org/x/text, google.golang.org/grpc and github.com/klauspost/compress in CSI sidecars to fix CVEs.
  • [control-plane-manager] Preserve custom TLS certificates when enabling a default Gateway.
  • [control-plane-manager] The migration hook for the api listener is added.
  • [deckhouse] Fixed a memory leak in webhook-handler, which kept the kubernetes monitors of replaced hooks running after every hook reload.
  • [deckhouse-controller] Fixed image pulls for modules served by ModulePullOverride when another source updates its registry or the override is not ready during Deckhouse startup.
  • [dhctl] Fixed known vulnerabilities in dhctl dependencies.
  • [docs] Move the language switch from the header to the footer
  • [documentation] Add the language switch to the footer.
  • [istio] Fixed CVEs in the istio module images (go-jose, golang.org/x/net).
    istiod, istio-cni, ingress gateways, Kiali, api-proxy and metadata-exporter pods will be restarted.
  • [local-path-provisioner] Updated golang.org/x/net, golang.org/x/text and github.com/klauspost/compress to fix CVEs.
  • [metallb] Fixed BGP migration to preserve pool allocation flags and allow users to manage migrated resources after switching ModuleConfig to version 3.
  • [node-manager] Master controllers leave an unreachable master after about 100 seconds and are not evicted from a single master.
  • [node-manager] Master controllers no longer tolerate keys from customTolerationKeys, so a key that repeats a built-in one no longer blocks pod updates.
    node-controller, capi-controller-manager, machine-controller-manager and bashible-apiserver do not run on a master tainted with a key from global.modules.placement.customTolerationKeys. Remove such taints from the master NodeGroup before the update.
  • [node-manager] A node-manager release no longer waits for capi-controller-manager when neither capi nor its webhooks change.
  • [node-manager] Fixed known vulnerabilities in the bashible-apiserver, node-controller, node-group-exporter, kubernetes-api-proxy, d8-shutdown-inhibitor and capi-controller-manager images.
  • [node-manager] Fixed constant NodeGroup status updates by node-controller on every Node update.
  • [registry] Fixed known vulnerabilities in the docker-auth image.
    The registry in-cluster proxy and node services pods will be restarted.
  • [registry] Fixed known vulnerabilities in the nodeservices-manager, docker-auth, mirrorer and syncer images.
  • [registry] The update to DP 1.78 is held as soon as registry.mode Proxy or Local, or a switch between Direct and Unmanaged, is set in the deckhouse ModuleConfig, and the message of the held release lists the migration steps for each mode.
    A cluster that uses Proxy or Local of the previous registry implementation, or Direct without the registry ModuleConfig, is not updated to DP 1.78, as before. This now also applies from the moment such a mode is set in the deckhouse ModuleConfig, before the switch starts. Migrate as described in “Migrating from the previous implementation to the current one” in the registry documentation. Clusters already prepared for the migration are not affected.
  • [registry-packages-proxy] The bootstrap port serves the manifest and the layer of the rpp-get image, so a node can verify the binary it installs.
  • [registry-packages-proxy] Fixed known vulnerabilities in registry-packages-proxy.
  • [registrypackages] Fixed known vulnerabilities in the cfssl and d8-bcheck packages.
  • [registrypackages] Bump golang.org/x/net to v0.62.0 in d8-bcheck to fix CVE-2026-78661 and CVE-2026-97045.
  • [registrypackages] Fixed known vulnerabilities in the rpp-get package.
  • [registrypackages] Fixed known vulnerabilities in auth_server and registry of the docker-registry package.
  • [registrypackages] rpp-get tries every registry-packages-proxy endpoint before an attempt fails and retries connection timeouts, so a node that reaches only the master of its own zone during bootstrap joins the cluster.
  • [service-with-healthchecks] Go dependencies are updated.
    The controller and agent pods will be restarted.
  • [terraform-manager] Fixed known vulnerabilities in opentofu.
    The terraform-manager pods will be restarted.
  • [terraform-manager] Fixed known vulnerabilities in opentofu and in the terraform-manager-dvp image.
  • [tools] The e2e directories of modules no longer reach the Deckhouse image.
  • [user-authn] Default DKP gateway requirement is removed from dex-authenticator template.

1.77.6

Know before update

  • Applying a discovered address restarts the Deckhouse Pod once.
  • Cilium agent pods will be restarted. With DSR, the DSR IPv4 option is stripped before the packet reaches the pod.
  • Control plane components (kube-apiserver, kube-controller-manager, kube-scheduler, etcd) and kubelet will be restarted.
  • Flannel pods will be restarted.
  • In ambient mode with Istio 1.27 or later, on nodes with legacy iptables, istio-cni-node switches from nftables to legacy iptables. Pods that were added to the mesh before the update keep their nftables rules. Restart these pods after the update. Otherwise, removing such a pod from the mesh without a restart leaves its old redirect rules in place, and the pod loses network connectivity.
  • The controller and agent pods will be restared.
  • The metallb components (controller, speaker, l2lb) will restart after the update.
  • containerd will be restarted on all nodes.
  • kube-proxy pods will be restarted.
  • simple-bridge pods will be restarted.

Features

  • [cni-cilium] Added a CVE-2026-98432 mitigation that filters IPv4 TCP SYN packets with IP options in the host and pod network namespaces (ipv4OptionsGuard; Enforce for new clusters, Audit for existing ones).
    Cilium agent pods will be restarted. With DSR, the DSR IPv4 option is stripped before the packet reaches the pod.
  • [cni-flannel] Added a CVE-2026-98432 mitigation that filters IPv4 TCP SYN packets with IP options in the host and pod network namespaces (ipv4OptionsGuard; Enforce for new clusters, Audit for existing ones).
    Flannel pods will be restarted.
  • [cni-simple-bridge] Added a CVE-2026-98432 mitigation that filters IPv4 TCP SYN packets with IP options in the host and pod network namespaces (ipv4OptionsGuard; Enforce for new clusters, Audit for existing ones).
    simple-bridge pods will be restarted.
  • [docs] Add static-routing-manager architecture module section.
  • [docs] Add cni-flannel and cni-simple-bridge modules architecture section.

Fixes

  • [admission-policy-engine] Fixed CVE-2026-116585, CVE-2026-116586 and CVE-2026-116587 in github.com/go-jose/go-jose/v4 and CVE-2026-78661 and CVE-2026-97045 in golang.org/x/net.
  • [admission-policy-engine] Switch ratify to go-jose v4 to fix CVE-2026-116585, CVE-2026-116586, CVE-2026-116587.
  • [admission-policy-engine] An unavailable Gatekeeper no longer blocks system components in namespaces that opted into security policy checks, which could deadlock the cluster together with the CNI agent.
  • [cert-manager] Bump x/net to v0.62.0 and go-jose to v4.1.6 to fix CVE-2026-78661, CVE-2026-97045, CVE-2026-116585, CVE-2026-116586, CVE-2026-116587.
  • [ci] Svace stages are rebuilt in every svace pipeline.
  • [cloud-provider-dvp] Fixes x/net, x/text and kin-openapi CVEs in all module images.
  • [cloud-provider-openstack] Fixes creation of nodes with rootDiskSize when the node availability zone does not exist in Cinder.
  • [cloud-provider-openstack] fix CloudEphemeral node creation in non-default availability zones via automatic Cinder-AZ discovery and new OpenStackInstanceClass.volumeTypeMap field
  • [cni-cilium] Fixed file descriptor, connection and lock leaks in Cilium found by static analysis.
  • [common] Bump x/net in kube-rbac-proxy to v0.62.0 to fix CVE-2026-78661 and CVE-2026-97045.
  • [control-plane-manager] The basic audit policy records an exec into a pod (pods/exec) in all namespaces again, as it did before Deckhouse 1.77.
  • [control-plane-manager] Fixed a goroutine leak in the helm release deprecation scan when listing releases fails.
  • [control-plane-manager] Updated go-jose, x/net and klauspost/compress in control plane components to fix known vulnerabilities.
    Control plane components (kube-apiserver, kube-controller-manager, kube-scheduler, etcd) and kubelet will be restarted.
  • [deckhouse] Resolve registry.d8-system.svc to a node running the in-cluster registry proxy during bootstrap, instead of always to 127.0.0.1.
    Applying a discovered address restarts the Deckhouse Pod once.
  • [deckhouse] Update runtime dependencies and document unaffected vulnerability paths.
  • [deckhouse] The d8-main Telemetry no longer references the deckhouse-tracing provider when the module doesn’t register it.
  • [deckhouse-controller] Prevent module hooks from running with unavailable snapshots while waiting for Deckhouse to restart after a module update
  • [deckhouse-controller] A pending module release no longer loses its Kubernetes, Deckhouse and module requirements, so an update that the cluster does not satisfy stays pending instead of disabling the module.
  • [dhctl] Keep the SSH host list and master readiness checks in sync with the infrastructure state during master converge.
  • [dhctl] Bump go-jose to v4.1.6 to fix CVE-2026-116585, CVE-2026-116586, CVE-2026-116587.
  • [docs] Fix docs build config errors in werf includes.
  • [documentation] Fix HTML parser vulnerabilities in docs-builder by updating golang.org/x/net to v0.62.0.
  • [go_lib] Package proxy streams the full package body when the cache skips writing it.
  • [istio] Fixed ambient mode on nodes with legacy iptables, where istio-cni-node crashed on Istio 1.25 and used nftables instead of legacy iptables on Istio 1.27 and later.
    In ambient mode with Istio 1.27 or later, on nodes with legacy iptables, istio-cni-node switches from nftables to legacy iptables. Pods that were added to the mesh before the update keep their nftables rules. Restart these pods after the update. Otherwise, removing such a pod from the mesh without a restart leaves its old redirect rules in place, and the pod loses network connectivity.
  • [kube-proxy] Fixed known vulnerabilities in the kube-proxy images.
    kube-proxy pods will be restarted.
  • [local-path-provisioner] config.json of the provisioner is serialized from a structure, and the CRD accepts in spec.path only an absolute path without quotes, backslashes, whitespace and control characters. A crafted path can no longer add manifests to the module release.
  • [metallb] Fixed CVE-2026-78661 and CVE-2026-97045 in the metallb and l2lb images.
    The metallb components (controller, speaker, l2lb) will restart after the update.
  • [multitenancy-manager] Fixed CVE-2026-116585, CVE-2026-116586 and CVE-2026-116587 in github.com/go-jose/go-jose/v4 and CVE-2026-78661 and CVE-2026-97045 in golang.org/x/net.
  • [node-manager] kube-rbac-proxy in capi-controller-manager and node-controller no longer listens on all addresses of a node that has no addresses yet, so it does not take the controller’s port.
  • [node-manager] Fixed known vulnerabilities in the integrity-controller and integrity-containerd-configurator images (CSE).
  • [node-manager] Fixes x/net CVEs in the caps-controller-manager image.
  • [registry] Require brackets around IPv6 upstream registry URL hosts and accept bracketed addresses with or without a port.
  • [registry-packages-proxy] Fixed registry-packages-proxy hanging after a cache error and serving corrupted packages, which blocked node updates.
  • [registry-packages-proxy] Bump go-jose to v4.1.6 to fix CVE-2026-116585, CVE-2026-116586, CVE-2026-116587.
  • [registrypackages] Updated go-jose, x/net and klauspost/compress in containerd, runc and crictl to fix known vulnerabilities.
    containerd will be restarted on all nodes.
  • [registrypackages] Support rebuilding legacy image layers and preserve verity data in CSE.
  • [service-with-healthchecks] Status updates were refactored.
    The controller and agent pods will be restared.
  • [user-authn] Local users keep the passwords they changed in Dex when the cluster updates from 1.76.
  • [user-authn] Fixed CVE-2026-116585, CVE-2026-116586 and CVE-2026-116587 in github.com/go-jose/go-jose/v4 and CVE-2026-78661 and CVE-2026-97045 in golang.org/x/net.
  • [user-authn] Bump go-oidc in dex-authenticator to v3.11.0 to fix CVE-2026-116585, CVE-2026-116586, CVE-2026-116587.
  • [user-authz] Fixed CVE-2026-78661 and CVE-2026-97045 in golang.org/x/net.
  • [vertical-pod-autoscaler] Fix a panic in vpa-updater and vpa-admission-controller when a pod with a container without resources is capped to a LimitRange of type Pod.

1.77.5

Know before update

  • After a reset, the previous password cannot be reused when passwordHistoryLimit is set; the current password cannot be set again through the change form, and a lowered passwordHistoryLimit applies at the next password change.
  • Cilium agent and operator pods will be restarted.
  • Control plane components and kubelet will be restarted.
  • CoreDNS pods will be restarted.
  • Node-local-dns safe updater will be restarted.
  • TLS connections from nodes to hosts whose certificates chain to emSign ECC Root CA - G3 or Comodo AAA Certificate Services (eMudhra / emSign issued) will fail. Reissue such certificates from another CA before the update.
  • The cert-manager controller pod restarts.
  • The controller and agent pods will be restarted.
  • cni-flannel pods will be restarted.
  • kube-proxy pods will be restarted.
  • network-policy-engine pods will be restarted.
  • node-local-dns pods will be restarted.

Features

  • [control-plane-manager] The basic audit policy logs Pod status updates from kubelet with the request body, so container terminations and their reasons get into the audit log.
  • [deckhouse] Migrated package release management to nelm v2 while keeping the v1 install and delete behaviour.
  • [user-authn] DexAuthenticator logins, logouts, failed and denied logins, CSRF mismatches and failed session refreshes are delivered as security events when the security-events-manager module is enabled.

Fixes

  • [admission-policy-engine] Fixed CVE-2026-114437 in github.com/go-chi/chi/v5 and CVE-2026-63209 in github.com/klauspost/compress.
  • [candi] The first master bootstrap waits for kube-apiserver readiness instead of failing the control-plane step on connection refused.
  • [cert-manager] The metrics proxy of the controller is no longer restarted by its liveness probe while its node is short of CPU.
    The cert-manager controller pod restarts.
  • [cilium-hubble] Fixed a known vulnerability in the Hubble relay image.
  • [cloud-provider-dvp] Fixed deletion of load balancers backed by ServiceWithHealthchecks, which failed because the child Service is protected by the heritage - deckhouse label in the parent cluster.
  • [cloud-provider-huaweicloud] Recreated the etcd disk when a master VM is recreated.
  • [cloud-provider-openstack] CAPI now mirrors MCM — configDrive honors externalNetworkDHCP, and additional networks always produce ports
  • [cloud-provider-vcd] Fixes the module getting stuck in Error on upgrade from 1.76 because of the immutable selector of the capcd-controller-manager Deployment.
  • [cni-cilium] Fixed a known vulnerability in the Cilium images.
    Cilium agent and operator pods will be restarted.
  • [cni-flannel] Fixed known vulnerabilities in the flanneld image.
    cni-flannel pods will be restarted.
  • [control-plane-manager] Updated Go dependencies of Kubernetes control plane components to fix known vulnerabilities.
    Control plane components and kubelet will be restarted.
  • [control-plane-manager] The cluster-wide controllers of control-plane-manager run in a single pod regardless of the highAvailability setting or an etcd arbiter.
  • [control-plane-manager] Control plane components are no longer restarted in parallel when a node is NotReady or after a certificate renewal.
  • [control-plane-manager] Fixed a Deckhouse crash caused by a helm release Secret or ConfigMap with too short release data.
  • [control-plane-manager] removed unused executable files from the etcd-backup image
  • [control-plane-manager] In CSE, availableVersions in d8-cluster-kubernetes lists only Kubernetes versions that can be set.
  • [deckhouse] Restore ClusterAdmin read access to ModuleSettingsDefinition so the console can show module settings again.
  • [deckhouse] Show the Terminating phase while package repositories are being deleted
  • [deckhouse] Update d8 CLI and scoped runtime VEX exceptions for known vulnerabilities.
  • [deckhouse-controller] A module enabled in the bundle is no longer disabled when its ModuleConfig does not set enabled.
  • [deckhouse-controller] Module sources now scan the registry every 30 minutes by default, reducing registry load
  • [istio] added limit paralle in common images
  • [istio] Fixed CVE-2026-63209 in istio images.
  • [istio] Fixed istio module installation with nelm v2 when Istio is managed by the istio operator (versions below 1.27).
  • [istio] Fixed alliance-healthcheck keeping the sidecar of the previous Istio version after a globalVersion change.
  • [kube-dns] Fixed a known vulnerability in the CoreDNS image.
    CoreDNS pods will be restarted.
  • [kube-proxy] Fixed known vulnerabilities in the kube-proxy init-container image.
    kube-proxy pods will be restarted.
  • [kube-proxy] Fixed known vulnerabilities in the kube-proxy images.
    kube-proxy pods will be restarted.
  • [multitenancy-manager] Fixed CVE-2026-63209 in github.com/klauspost/compress.
  • [network-policy-engine] Fixed known vulnerabilities in the kube-router image.
    network-policy-engine pods will be restarted.
  • [node-local-dns] Fixed DNS errors during node-local-dns pod restarts in clusters with Cilium.
  • [node-local-dns] Fixed a known vulnerability in the node-local-dns CoreDNS image.
    node-local-dns pods will be restarted.
  • [node-local-dns] Node-local-dns safe updater logic was updated to match the Kubernetes logic.
    Node-local-dns safe updater will be restarted.
  • [openvpn] Fixed known vulnerabilities in the ovpn-admin and easyrsa-migrator images.
  • [registry] In the Proxy mode, reuse upstream auth challenges and tokens instead of requesting them before every request to the upstream registry.
  • [registry] Fix the registry static pod stuck in ImagePullBackOff after a registry image update on a single master in the Proxy and Local modes.
  • [registrypackages] Nodes stop trusting emSign ECC Root CA - G3 and Comodo AAA Certificate Services, the roots behind the compromised eMudhra intermediate CAs.
    TLS connections from nodes to hosts whose certificates chain to emSign ECC Root CA - G3 or Comodo AAA Certificate Services (eMudhra / emSign issued) will fail. Reissue such certificates from another CA before the update.
  • [registrypackages] Update containerdv2 patch
  • [service-with-healthchecks] CVEs were fixed.
    The controller and agent pods will be restarted.
  • [user-authn] dex-authenticator no longer writes the request headers, cookies included, to the log when the CSRF cookie is missing.
  • [user-authn] A password reset of a local user keeps the replaced password in the password history, and the password change form refuses the current password in every case.
    After a reset, the previous password cannot be reused when passwordHistoryLimit is set; the current password cannot be set again through the change form, and a lowered passwordHistoryLimit applies at the next password change.

1.77.4

Know before update

  • Cilium agent and operator pods will be restarted.
  • CoreDNS pods will be restarted.
  • NodeGroups that already have preemptible: true under the CAPI engine keep their current regular VMs until the next rollout (any rollout-triggering InstanceClass change or manual-rollout-id); after that rollout their nodes become preemptible VMs and are rotated every ~20h.
  • Upgrade does not recreate any node. NodeGroups that already have preemptible: true under the CAPI engine keep their current regular VMs; Machines created after the upgrade (scale-up, replacement) are preemptible, so such NodeGroups run mixed VMs until a full rollout (any rollout-triggering InstanceClass change or the manual-rollout-id annotation). Preemptible VMs are rotated every ~20h.
  • metallb pods will be restarted.
  • node-local-dns pods will be restarted.

Features

  • [control-plane-manager] The etcd dashboard is reworked to find the cause of slow etcd - disk, network, leader, kube-apiserver load or page cache.

Fixes

  • [chrony] Bump golang.org/x/net in chrony_exporter to fix CVE-2026-78669.
  • [cilium-hubble] Fixed known vulnerabilities in the Hubble relay and UI images.
  • [cloud-provider-aws] Allow bootstrapping clusters in AWS regions unknown to the bundled terraform-provider-aws, such as ap-southeast-6.
  • [cloud-provider-dvp] A DVP cluster migrated to the new provider configuration format keeps its networkPolicy - None instead of silently switching to Isolated.
  • [cloud-provider-dvp] Clusters with networkPolicy - None in the provider configuration no longer get NetworkPolicies created for LoadBalancer Services.
  • [cloud-provider-yandex] Preemptible nodes are created as preemptible VMs and rotated before the 24h limit when a NodeGroup is managed by the CAPI engine.
    Upgrade does not recreate any node. NodeGroups that already have preemptible: true under the CAPI engine keep their current regular VMs; Machines created after the upgrade (scale-up, replacement) are preemptible, so such NodeGroups run mixed VMs until a full rollout (any rollout-triggering InstanceClass change or the manual-rollout-id annotation). Preemptible VMs are rotated every ~20h.
  • [cloud-provider-yandex] Preemptible nodes are created as preemptible VMs and rotated before the 24h limit when a NodeGroup is managed by the CAPI engine.
    NodeGroups that already have preemptible: true under the CAPI engine keep their current regular VMs until the next rollout (any rollout-triggering InstanceClass change or manual-rollout-id); after that rollout their nodes become preemptible VMs and are rotated every ~20h.
  • [cni-cilium] Fixed known vulnerabilities in the Cilium images.
    Cilium agent and operator pods will be restarted.
  • [common] Fixed known vulnerabilities in the CNI migration images.
  • [deckhouse] Bump containerd to v2.2.10 and dependencies to fix CVEs
  • [dhctl] Bump golang.org/x/net to fix CVE-2026-78669.
  • [docs] Update terminology in the documentation and SEO improvements.
  • [documentation] Update documentation building rules.
  • [istio] Added VEX for CVE-2026-85747 in pilot and proxyv2 images.
  • [kube-dns] Fixed known vulnerabilities in the CoreDNS and sts-pods-hosts-appender images.
    CoreDNS pods will be restarted.
  • [metallb] Fixed known vulnerabilities in the MetalLB images.
    metallb pods will be restarted.
  • [node-local-dns] Fixed known vulnerabilities in the node-local-dns images.
    node-local-dns pods will be restarted.
  • [node-manager] Updates otel, grpc and x/crypto dependencies of node-controller to fix CVEs.
  • [node-manager] fixed private build
  • [node-manager] Bump golang.org/x/net to fix CVE-2026-78669, CVE-2026-78663, CVE-2026-97032.
  • [registry] Bump golang.org/x/net in nodeservices-manager to fix CVE-2026-78669.
  • [registry-packages-proxy] Bump golang.org/x/net to fix CVE-2026-78669.
  • [registrypackages] Fixed known vulnerabilities in the kubernetes-cni package.
  • [registrypackages] Bump golang.org/x/net, golang.org/x/crypto and OpenTelemetry in rpp-get and the docker-registry auth_server to fix CVE-2026-78669, CVE-2026-56855, CVE-2026-81870.
  • [terraform-manager] Bump golang.org/x/net in opentofu to fix CVE-2026-78669.
  • [vertical-pod-autoscaler] The admission-controller webhook now serves HTTP/1.1 only and no longer exposes

1.77.3

Know before update

  • Before, such an entry stopped the webhook from reloading the rules, so subjects of rules added later, and every subject after a restart of the webhook, were not limited to their namespaces. Subjects of a rule whose only entry is invalid now get no namespace through the entries of that rule.
  • Bumping deckhouse_lib_helm from 1.72.11 to 1.72.24 also drops the drbd.linbit.com taint tolerations from helm_lib_tolerations, so pods rendered through it no longer tolerate lost-quorum, force-io-error and ignore-fail-over, and aligns the VPA maxAllowed memory to the recommender’s 64Mi quantum.
  • Exec and attach requests in user namespaces reach Gatekeeper again, as in Deckhouse 1.73 and 1.74. Every constraint that matches them is evaluated again, including a constraint without match.kinds or with kinds: ["*"] whose policy does not check input.review.operation. Before updating, check that such constraints do not deny kubectl exec and kubectl attach for users who need them.
  • In a cluster with the proxy parameter of ClusterConfiguration set, the cert-manager controller sends requests to the ACME server, HashiCorp Vault and the DNS provider APIs through the proxy server. The HTTP-01 self-check also requests the domain through the proxy server. If the proxy server cannot reach the issuer or the public address of the cluster, add these addresses to noProxy.
  • In single-replica installations a new controller pod reconciles projects only after the previous pod has stopped and released the lease. If the previous pod is lost without a shutdown, for example together with its node, the new pod starts reconciling projects 60 to about 78 seconds after it starts.
  • Machines are not recreated on update. Machines created before the update keep the standard network type and no labels until the NodeGroup is rolled out (change a rolling InstanceClass field or set the manual-rollout-id annotation on the NodeGroup). Machines stuck in provisioning because of additionalSubnets become ready without recreation.
  • Such identities can write ClusterAuthorizationRules, Users and Groups that carry SuperAdmin or cluster-admin. A ClusterAuthorizationRule gives this right only without namespaceSelector and limitNamespaces and, with multi-tenancy on, only with allowAccessToSystemNamespaces: true, even if it grants cluster-admin.
  • The cloud MachineHealthCheck is updated in existing clusters. A node in Ready=False is replaced after 10 minutes instead of 5 (Ready=Unknown still after 5), and remediation stops while more than 50% of the cluster’s CAPI machines are unhealthy.
  • The controller and agents will be restarted.
  • The grant webhooks are now removed together with the module. On a cluster where multitenancy-manager is already disabled at the update, the cluster-objects-grants-defaulting and cluster-objects-grants-validator webhook configurations left by earlier versions stay, and writes to grantable resources in project namespaces (for example, removing a PVC finalizer) keep failing with service "multitenancy-manager" not found. Delete them by hand: d8 k delete mutatingwebhookconfiguration cluster-objects-grants-defaulting d8 k delete validatingwebhookconfiguration cluster-objects-grants-validator Nothing has to be done on clusters where the module is enabled.

Features

  • [admission-policy-engine] Report the availability of the module components with alerts.
  • [cert-manager] Add the highAvailability parameter to control the HA mode of the module independently of the global setting.
  • [cert-manager] cert-manager uses the cluster-wide proxy server from the proxy parameter of ClusterConfiguration.
    In a cluster with the proxy parameter of ClusterConfiguration set, the cert-manager controller sends requests to the ACME server, HashiCorp Vault and the DNS provider APIs through the proxy server. The HTTP-01 self-check also requests the domain through the proxy server. If the proxy server cannot reach the issuer or the public address of the cluster, add these addresses to noProxy.
  • [cloud-provider-dvp] Makes CCM create NPs on every lb creation while using networkPolicy - Restricted
  • [cni-cilium] Added an alert and a release requirement that block the Cilium update while CiliumNetworkPolicy or CiliumClusterwideNetworkPolicy objects use the fromRequires or toRequires fields.
  • [deckhouse] ModuleSource scans skip unchanged release channels using a cheap registry HEAD request instead of pulling the whole release image every cycle.
  • [deckhouse-controller] Added the update settings of applications: the spec.update field of an Application and the global applications.update parameter. Deckhouse merges them with the update settings of the deckhouse module but doesn’t apply them to application updates yet.
  • [docs] Add user-authz module architecture section.
  • [docs] Add service-with-healthchecks module architecture section.
  • [documentation] The in-cluster documentation now includes the guides.
  • [documentation] Documentation updates for DP CSE 1.77.
  • [multitenancy-manager] A new alert warns about a user project template named simple, which the built-in template of the same name overwrites after the update.
  • [user-authz] The new D8UserAuthzLegacyRBACv2CapabilityBindingFound alert lists the bindings to built-in capabilities that the next minor release renames without aliases, and names the new name of each capability.
  • [user-authz] The new D8UserAuthzLegacyRBACv2CustomCapabilityFound alert lists custom capabilities that extend the namespace roles only through labels the next role scheme does not select.
  • [user-authz] New D8UserAuthzLimitNamespacesEntryNeedsRewrite and D8UserAuthzLimitNamespacesEntryInvalid alerts report ClusterAuthorizationRule limitNamespaces entries that are anchored only in part or are not valid regular expressions.

Fixes

  • [admission-policy-engine] Fixed Gatekeeper policies for kubectl exec and kubectl attach being skipped in user namespaces.
    Exec and attach requests in user namespaces reach Gatekeeper again, as in Deckhouse 1.73 and 1.74. Every constraint that matches them is evaluated again, including a constraint without match.kinds or with kinds: ["*"] whose policy does not check input.review.operation. Before updating, check that such constraints do not deny kubectl exec and kubectl attach for users who need them.
  • [admission-policy-engine] Disabling the module deletes the webhook configurations before Gatekeeper, so the uninstall no longer blocks exec and the Deckhouse queue.
  • [admission-policy-engine] Fixed CVE-2026-78659, CVE-2026-78660, CVE-2026-78663, CVE-2026-78669 and CVE-2026-97032 in golang.org/x/net and CVE-2026-56851 in golang.org/x/text.
  • [candi] Restored the lost kubelet-disable-k-panic-check patch for k8s 1.35 and 1.36, so kubelet no longer enforces kernel.panic=10 on these versions.
  • [candi] New nodes reboot on the first bashible run again when a step requests it. Only the first master skips this reboot.
  • [cert-manager] The webhook keeps serving requests during a rollout and during a rotation of its CA, and the controller waits for the API server at startup instead of restarting.
  • [cert-manager] The spec.certificateOwnerRef field of a certificate no longer causes an endless Secret update loop when it differs from the global owner reference setting that cleanupOrphanSecrets enables in the CertManager HTTPS mode, and is applied to an already issued Secret.
  • [cert-manager] Fixed CVE-2026-78659, CVE-2026-78660, CVE-2026-78663, CVE-2026-78669 and CVE-2026-97032 in golang.org/x/net.
  • [chrony] Bump golang.org/x/crypto in chrony_exporter to fix CVE-2026-56855, CVE-2026-78662.
  • [cloud-provider-dvp] The CSI driver reports a volume as detached from a node only after the disk is released by the node virtual machine, so with virtualization that keeps the VM listed on the disk until it is released, the volume is not attached to two nodes at once.
  • [cloud-provider-dvp] The CSI driver reports the volume limit of a node, so the scheduler does not place more volumes on a node than its virtual machine can attach.
  • [cloud-provider-dvp] Stop infinite check/converge loop caused by the DVP controller annotation on the cloud-init Secret.
  • [cloud-provider-vsphere] Fixed master node creation when the datastore is set as a path with parent folders.
  • [cloud-provider-yandex] Restored additionalSubnets, networkType and VM labels (nodes.parameters.labels, additionalLabels) for CAPI-managed Yandex nodes.
    Machines are not recreated on update. Machines created before the update keep the standard network type and no labels until the NodeGroup is rolled out (change a rolling InstanceClass field or set the manual-rollout-id annotation on the NodeGroup). Machines stuck in provisioning because of additionalSubnets become ready without recreation.
  • [cloud-provider-zvirt] VMs are now actually removed from zVirt on NodeGroup scale-down and cluster destroy
  • [common] Fixed a kube-apiserver panic on JSON Patch requests whose add/replace of the whole document or test operation lacks the value field; such requests are now rejected with a validation error.
  • [common] kube-apiserver returns a single short validation error for an invalid fieldManager instead of an error for every invalid character.
  • [common] Fixed CVE-2026-78659, CVE-2026-78660, CVE-2026-78663, CVE-2026-78669 and CVE-2026-97032 in golang.org/x/net in kube-rbac-proxy.
  • [control-plane-manager] Keep resource.k8s.io/v1beta1 enabled in kube-apiserver 1.34.
  • [deckhouse] Fixed CVEs in d8 CLI dependencies by updating d8 to v0.33.25.
  • [deckhouse] Restore kubectl plugin lookup in d8 k and d8 kubectl
  • [deckhouse] Update runtime libraries and VEX exceptions for known vulnerabilities
  • [deckhouse-controller] Deleting an Application now also deletes objects with the werf.io/ownership: anyone or werf.io/resource-policy annotation, such as Jobs deployed only on install, if the package lists their kinds in orphanResources. Only helm.sh/resource-policy: keep protects an object.
  • [descheduler] Fixed CVE-2026-81870 in descheduler by bumping OpenTelemetry Go modules to v1.45.0.
  • [dhctl] Preserve existing SSH keys when switching to the temporary converge user.
  • [dhctl] Destroy and abort of a static cluster now fail instead of reporting success when a control-plane node was not cleaned because of an SSH error or cleanup timeout.
  • [dhctl] Bump golang.org/x/crypto and OpenTelemetry to fix CVE-2026-56855, CVE-2026-78662, CVE-2026-81870, CVE-2026-81871, CVE-2026-81872.
  • [docs] Correct typo in managed-rabbit key in modules library.
  • [docs] Improve documentation clarity and fix typos.
  • [docs] Fixed tab switching on pages with several tab sets of the same name and opening hidden tabs by an anchor link.
  • [docs] Enhance tab activation logic in the documentation.
  • [docs] Enhance tab shortcode functionality.
  • [docs] Remove raw tags from usage documentation.
  • [docs] Show expand icon on non-collapsible top-level CRD parameters in the documentation.
  • [docs] Added experimental module enable note to the lifecycle badge.
  • [docs] Added registry module port 5001 to Deckhouse ports documentation
  • [docs] Rendered all versions from oss.yaml versions array.
  • [docs] Update hugo to 0.168.0 in docs-builder
  • [docs] Page with DNS names full list was added in to reference.
  • [helm_lib] Size the resources of a module whose resourcesManagement names a fractional quantity, instead of rendering a zero limit.
    Bumping deckhouse_lib_helm from 1.72.11 to 1.72.24 also drops the drbd.linbit.com taint tolerations from helm_lib_tolerations, so pods rendered through it no longer tolerate lost-quorum, force-io-error and ignore-fail-over, and aligns the VPA maxAllowed memory to the recommender’s 64Mi quantum.
  • [istio] Fixed CVEs in istio, kiali and operator images (golang.org/x/net, golang.org/x/text).
  • [multitenancy-manager] Disabling the module now removes the cluster-objects grant webhooks, which used to stay behind and block writes in project namespaces.
    The grant webhooks are now removed together with the module. On a cluster where multitenancy-manager is already disabled at the update, the cluster-objects-grants-defaulting and cluster-objects-grants-validator webhook configurations left by earlier versions stay, and writes to grantable resources in project namespaces (for example, removing a PVC finalizer) keep failing with service "multitenancy-manager" not found. Delete them by hand: d8 k delete mutatingwebhookconfiguration cluster-objects-grants-defaulting d8 k delete validatingwebhookconfiguration cluster-objects-grants-validator Nothing has to be done on clusters where the module is enabled.
  • [multitenancy-manager] The controller takes its leader lease with any number of replicas, so a rollout of a single-replica installation no longer has two pods upgrading project Helm releases at once.
    In single-replica installations a new controller pod reconciles projects only after the previous pod has stopped and released the lease. If the previous pod is lost without a shutdown, for example together with its node, the new pod starts reconciling projects 60 to about 78 seconds after it starts.
  • [multitenancy-manager] The Project conversions of the next release are registered in advance, so reads and writes of projects keep working while webhook-handler is updated in HA mode.
  • [multitenancy-manager] Fixed CVE-2026-78659, CVE-2026-78660, CVE-2026-78663, CVE-2026-78669 and CVE-2026-97032 in golang.org/x/net.
  • [multitenancy-manager] The webhooks no longer leave zombie processes in webhook-handler until it runs out of pids.
  • [node-manager] Fixed cluster-autoscaler using node group names without the cluster prefix after a Deckhouse restart, which stopped autoscaling and triggered the D8ClusterAutoscalerTooManyErrors alert.
  • [node-manager] Fixed cluster-autoscaler stopping autoscaling of all node groups and restarting every two hours when an MCM node group has minPerZone equal to maxPerZone.
  • [node-manager] Fixed CVEs in cluster-autoscaler Go dependencies (OpenTelemetry, and grpc and x/crypto for Kubernetes 1.31).
  • [node-manager] A StaticInstance whose StaticMachine was deleted is returned to Pending and becomes available for bootstrapping again instead of staying out of the pool forever.
  • [node-manager] Fixed cluster-autoscaler not scaling CAPI node groups up from zero for pods requesting ephemeral storage, and putting the whole scale-up of a multi-zone node group into a single zone.
  • [node-manager] Bump golang.org/x/crypto and OpenTelemetry in bashible-apiserver and capi-controller-manager.
  • [node-manager] Fixed the CAPI MachineHealthCheck replacing cloud nodes that take longer than 5 minutes to become Ready after registering, and limited remediation to when at most half of the CAPI machines are unhealthy.
    The cloud MachineHealthCheck is updated in existing clusters. A node in Ready=False is replaced after 10 minutes instead of 5 (Ready=Unknown still after 5), and remediation stops while more than 50% of the cluster’s CAPI machines are unhealthy.
  • [node-manager] Fixed node-manager failing to apply its values in the BE, SE, SE+, EE, FE and CSE editions when a CAPI node group has a known boot disk size.
  • [registry] Build docker-auth from the new fork repository at flant-1.0.0, which fixes CVE-2026-81870.
  • [registrypackages] Fixed CVEs in crictl 1.31.
  • [service-with-healthchecks] Go deps are updated.
    The controller and agents will be restarted.
  • [terraform-manager] Bump opentofu to 1.12.7 to fix CVE-2026-56855, CVE-2026-78662.
  • [tools] Remove d8-gpu namespace from integrity check.
  • [user-authn] The idTokenTTL description and the D8UserAuthnIDTokenTTLTooLong alert say that a value of 6 hours or more is replaced with 5h59m by the next settings version, and what lowering it does to DexAuthenticator sessions and issued ID tokens.
  • [user-authn] Restrict the Job that issues the basic-auth-proxy certificate to the read-only front-proxy CA files and cover it with a SecurityPolicyException.
  • [user-authn] A DexClient or a DexAuthenticator in a namespace whose name YAML reads as a number or a boolean no longer breaks the user-authn release.
  • [user-authn] A DexAuthenticator with a name longer than 45 characters no longer logs its users out on every run of the hook.
  • [user-authn] The DexAuthenticator validating webhook no longer logs a jq error for an object without additionalApplications.
  • [user-authn] Errors the webhook scripts of user-authn, user-authz and multitenancy-manager write to stderr are logged as warnings instead of info messages.
  • [user-authn] Protected applications keep serving users during a short dex outage and during restarts.
  • [user-authn] Fixed CVE-2026-78659, CVE-2026-78660, CVE-2026-78663, CVE-2026-78669 and CVE-2026-97032 in golang.org/x/net.
  • [user-authn] The webhooks no longer leave zombie processes in webhook-handler until it runs out of pids.
  • [user-authz] The authorization webhook no longer restarts while the API server is unreachable, which could deadlock the control plane on bootstrap or an API server restart with multitenancy enabled.
  • [user-authz] a kube-apiserver health probes no longer fail when authz-webhook is unavailable.
  • [user-authz] The identity-assign check treats an identity with every permission across the cluster, such as a subject of cluster-admin bound by a ClusterRoleBinding, as SuperAdmin.
    Such identities can write ClusterAuthorizationRules, Users and Groups that carry SuperAdmin or cluster-admin. A ClusterAuthorizationRule gives this right only without namespaceSelector and limitNamespaces and, with multi-tenancy on, only with allowAccessToSystemNamespaces: true, even if it grants cluster-admin.
  • [user-authz] The RBACv2 roles d8:manage:* and d8:use:role:* keep the rights of the modules that have already switched to the capability labels of the next release while the upgrade to it is in progress.
  • [user-authz] The authorization webhook keeps applying ClusterAuthorizationRules when a limitNamespaces entry is not a valid regular expression; the entry is left out, and its rule covers fewer namespaces.
    Before, such an entry stopped the webhook from reloading the rules, so subjects of rules added later, and every subject after a restart of the webhook, were not limited to their namespaces. Subjects of a rule whose only entry is invalid now get no namespace through the entries of that rule.
  • [user-authz] The D8UserAuthzLegacyRBACv2CustomRoleFound alert lists every custom role that selects capabilities by the rbac.deckhouse.io/kind: manage|use label or by the aggregation label of a lineage the next role model does not collect, or that loses its RoleBindings in the namespaces of the modules, and the FAQ describes how to prepare such a role before the update without losing access.
  • [user-authz] An AuthorizationRule in a namespace whose name YAML reads as a number, a boolean or null no longer breaks the module’s bindings.
  • [user-authz] Managers who assign roles by range can again change DexProviders, UserOperations, Users and Groups of identities that hold a ClusterAuthorizationRule.
  • [user-authz] The authorization webhook answers the requests in flight when it restarts instead of denying them.
  • [user-authz] Fixed CVE-2026-78659, CVE-2026-78660, CVE-2026-78663, CVE-2026-78669 and CVE-2026-97032 in golang.org/x/net and CVE-2026-56851 in golang.org/x/text.
  • [user-authz] The webhooks no longer leave zombie processes in webhook-handler until it runs out of pids.

1.77.2

Know before update

  • A StaticInstance whose connectivity check fails now keeps its reservation for the whole bootstrap window instead of returning to the pool on every failed attempt, and the bootstrap (20 min) and cleanup (10 min) timeouts, which previously never fired, are now reachable. If nothing ever ran on the host, the bootstrap timeout returns the instance to the pool after 20 minutes as before. If the host was already bootstrapped in part, the instance returns only through MachineHealthCheck remediation (nodeStartupTimeoutSeconds: 1200) plus the cleanup timeout, so it becomes available again roughly 20 + 10 minutes after the failure, and remediation reboots the host as part of the cleanup.
  • A cluster still on 1.76 whose user-authn ModuleConfig sets idTokenTTL to 6h or more updates to this release without Deckhouse crash-looping, and stays on it until idTokenTTL is lowered below 6h. A cluster already crash-looping on 1.77.0 or 1.77.1 does not recover by itself, since the release controller runs in the pod that is down; set failurePolicy Ignore on the module-configs.deckhouse-webhook.deckhouse.io webhook of the deckhouse-webhook ValidatingWebhookConfiguration, lower idTokenTTL below 6h, and restart the deckhouse pod.
  • Clusters bootstrapped before v1.55 were switched to the Baseline default PSS policy in 1.77, which denies privileged workloads in user namespaces. After the update, the default policy on these clusters becomes less strict: it is switched back to Privileged, and privileged pods in user namespaces pass admission again. To keep the Baseline policy, set settings.podSecurityStandards.defaultPolicy: Baseline explicitly in the admission-policy-engine ModuleConfig.
  • Clusters whose d8-system/install-data ConfigMap holds an empty or invalid version and whose podSecurityStandards.defaultPolicy isn’t set were switched to the Baseline default PSS policy in 1.77. After the update, the default policy on these clusters becomes less strict: it is switched back to Privileged. To keep the Baseline policy, set settings.podSecurityStandards.defaultPolicy: Baseline explicitly in the admission-policy-engine ModuleConfig.
  • Deckhouse 1.78 makes Baseline the default PSS policy for all clusters. On clusters bootstrapped with a Deckhouse version lower than v1.55 or with an unknown bootstrap version, the PodSecurityStandardsDefaultPolicyNotSet alert fires, and the update to 1.78 is blocked until settings.podSecurityStandards.defaultPolicy is set explicitly in the admission-policy-engine ModuleConfig. To keep the current behavior, set defaultPolicy: Privileged. To switch to Baseline or Restricted, make sure that the workloads in non-system namespaces comply with the chosen policy first.
  • In Deckhouse CSE, a DexProvider that already has enableBasicAuth: true keeps it after the update, and any change to its oidc or ldap section is rejected until enableBasicAuth is set to false. The same holds for an LDAP provider that combines insecureNoSSL with startTLS, insecureSkipVerify or rootCAData. Set enableBasicAuth: false and resolve the LDAP TLS settings on such providers.
  • The d8-kube-proxy pods will be restarted on all nodes.
  • Two Go hooks registered in one file shared a derived name, so the Kubernetes-binding half lost its bindings and the d8_cluster_objects_grant_violated series was recomputed only by the periodic scan. The series itself was never missing; it could lag a policy change by up to two minutes.
  • dhctl converge no longer deadlocks on master replacement or externalIPAddresses removal in Yandex Cloud; the terraform-manager provider release is patched to detach one-to-one NAT before deleting a reserved address.

Features

  • [admission-policy-engine] Added the PodSecurityStandardsDefaultPolicyNotSet alert that fires when the install-data ConfigMap is missing and the default PSS policy must be set explicitly in the ModuleConfig.
  • [admission-policy-engine] The PodSecurityStandardsDefaultPolicyNotSet alert now fires when podSecurityStandards.defaultPolicy isn’t set and the Privileged policy is derived from the bootstrap version. The update to Deckhouse 1.78 is blocked until the policy is set explicitly.
    Deckhouse 1.78 makes Baseline the default PSS policy for all clusters. On clusters bootstrapped with a Deckhouse version lower than v1.55 or with an unknown bootstrap version, the PodSecurityStandardsDefaultPolicyNotSet alert fires, and the update to 1.78 is blocked until settings.podSecurityStandards.defaultPolicy is set explicitly in the admission-policy-engine ModuleConfig. To keep the current behavior, set defaultPolicy: Privileged. To switch to Baseline or Restricted, make sure that the workloads in non-system namespaces comply with the chosen policy first.
  • [deckhouse-controller] Add the x-deckhouse-immutable, x-deckhouse-ui-group, and x-deckhouse-enum-switch-settings extensions to Application package settings schemas; the Application webhook rejects changes to x-deckhouse-immutable fields after the settings were applied.
  • [deckhouse-controller] When an Application is deleted, Deckhouse also deletes the objects of the kinds listed in the package’s orphanResources value that the application left in its namespace.
  • [docs] Add user-authn-controller, user-api and basic-auth-proxy components to user-authn module architecture section.
  • [docs] Rename DKP->DP and replace PNG->SVG architecture section
  • [docs] Fix links to architecture diagrams
  • [docs] Add alb module architecture section.
  • [docs] Add managed services modules architecture documentation section.
  • [docs] Add sdn module architecture section.

Fixes

  • [admission-policy-engine] Restored the Privileged default PSS policy for clusters bootstrapped before v1.55 (without the install-data ConfigMap).
    Clusters bootstrapped before v1.55 were switched to the Baseline default PSS policy in 1.77, which denies privileged workloads in user namespaces. After the update, the default policy on these clusters becomes less strict: it is switched back to Privileged, and privileged pods in user namespaces pass admission again. To keep the Baseline policy, set settings.podSecurityStandards.defaultPolicy: Baseline explicitly in the admission-policy-engine ModuleConfig.
  • [admission-policy-engine] Fixed CVEs in the gatekeeper and ratify images.
  • [admission-policy-engine] Fixed the default Pod Security Standards (PSS) policy being switched to Baseline without notice when the bootstrap version in the d8-system/install-data ConfigMap is empty or invalid. Such clusters now get the Privileged policy.
    Clusters whose d8-system/install-data ConfigMap holds an empty or invalid version and whose podSecurityStandards.defaultPolicy isn’t set were switched to the Baseline default PSS policy in 1.77. After the update, the default policy on these clusters becomes less strict: it is switched back to Privileged. To keep the Baseline policy, set settings.podSecurityStandards.defaultPolicy: Baseline explicitly in the admission-policy-engine ModuleConfig.
  • [candi] remove d8-bcheck in cleanup_static_node
  • [candi] containerd integrity errors are written to /var/log/containerd/containerd-integrity.log on RED OS 8.
  • [cert-manager] Fixed CVEs in the cert-manager images.
  • [cert-manager] Gateway API integration is now enabled automatically when the Gateway and HTTPRoute CRDs are installed.
  • [chrony] Bump golang.org/x/crypto to v0.55.0 in chrony_exporter to fix CVE-2026-56854.
  • [ci] update werf to v2.74.2-dk.1
  • [ci] Add allowed alerts for D8Monitoring for e2e which uses 1.77+
  • [ci] The documentation changes validation resolves symlinked related language files.
  • [ci] Fix changelog-generator image tag in changelog-update job to use the shared CI_IMAGES_TAG
  • [ci] add skip docs validation label
  • [cloud-provider-dvp] Fixed the D8CloudProviderDVPMigrationPending alert that kept firing after the migration resources were applied.
  • [cloud-provider-dvp] Fixes CVE vulnerabilities in cloud-provider-dvp images.
  • [cloud-provider-dvp] Fixes cluster bootstrap getting stuck when capdvp-controller-manager and the validation webhook cannot reach the API server and DNS before the CNI is ready.
  • [cloud-provider-openstack] Fixes cluster bootstrap getting stuck when capo-controller-manager cannot reach the API server and DNS before the CNI is ready.
  • [cloud-provider-vcd] Batch VM cache refresh on cache miss
  • [cloud-provider-vcd] Fixes cluster bootstrap getting stuck when capcd-controller-manager cannot reach the API server and DNS before the CNI is ready.
  • [cloud-provider-yandex] fix “Address in use” failures when replacing nodes and when removing external IP addresses
    dhctl converge no longer deadlocks on master replacement or externalIPAddresses removal in Yandex Cloud; the terraform-manager provider release is patched to detach one-to-one NAT before deleting a reserved address.
  • [common] Fix kubelet podSandbox order
  • [common] fixed private build cni-migration-*
  • [common] Fix CVEs in the kube-rbac-proxy image.
  • [common] Fixed known vulnerabilities in the CoreDNS image used by the node-local-dns module.
  • [control-plane-manager] A new etcd maxDbSize is applied without a deckhouse restart.
  • [control-plane-manager] Fix kubectl exec through Kubernetes API published with Gateway API.
  • [deckhouse] set releaseChannel in CSE
  • [deckhouse] Fix setting Scaled before Installed.
  • [deckhouse] Remove leftover symlink-backend entries when deploying packages with the erofs backend.
  • [deckhouse] Fix random startup failures with ‘waitid - no child processes’ when restoring erofs modules.
  • [deckhouse] Fix CVEs in deckhouse-cli.
  • [dhctl] Restoring of handling api_server_url and api_server_token in dhctl
  • [dhctl] Bump google.golang.org/grpc to v1.83.2 to fix CVE-2026-84304 and CVE-2026-84445.
  • [dhctl] Fixed terraform-auto-converger, terraform-state-exporter and dhctl converge failing on cloud clusters whose cloud-provider-<name> ModuleConfig has no spec.enabled field. dhctl now treats spec.enabled as optional and a ModuleConfig without it as disabled.
  • [dhctl] Fixed the terraform-auto-converger init container failing on clusters already migrated to OpenTofu - the migration check now skips planning when no Terraform state is stored in the cluster.
  • [docs] Fixed FAQ section for ansible inventory.
  • [docs] Register the ports of csi-hitachi and csi-mind-ustor, list the registry storage ports, and drop storage ports that are no longer bound in the host network from the ports reference.
  • [docs] Fix search hanging when IndexedDB upgrade is blocked.
  • [docs] Removed mentions of Cilium from the inter-cluster cooperation documentation page.
  • [istio] Fixed the istio module getting stuck when switching globalVersion from 1.21 while proxies are still connected to the 1.21 control plane.
  • [istio] Fix leaking istiod control plane after removing an Istio version or disabling the module.
  • [istio] Switching back to Istio 1.21 no longer removes the networking.istio.io/v1 API from the Istio CRDs and no longer breaks the istio release.
  • [istio] Deckhouse update requirements now take into account Istio control planes that are still being removed.
  • [istio] The Istio operator now recovers Helm releases stuck in the pending-* or uninstalling state.
  • [kube-dns] Set resource limits for the render-etc-hosts-with-cluster-domain-aliases init container so that pods can be created in namespaces with a ResourceQuota on limits.cpu/limits.memory.
  • [kube-proxy] Fixed kube-proxy failing to start on Kubernetes 1.36 and higher because of the removed TopologyAwareHints feature gate.
    The d8-kube-proxy pods will be restarted on all nodes.
  • [metallb] Fixed creation of BGPPeer objects on Kubernetes 1.36+, which blocked BGP sessions and LoadBalancer IP announcements in BGP mode.
  • [multitenancy-manager] The grant-violation hook re-evaluates policies on change again, not only on its two-minute schedule.
    Two Go hooks registered in one file shared a derived name, so the Kubernetes-binding half lost its bindings and the d8_cluster_objects_grant_violated series was recomputed only by the periodic scan. The series itself was never missing; it could lag a policy change by up to two minutes.
  • [multitenancy-manager] Fixed vulnerabilities in the multitenancy-manager image.
  • [node-manager] Increase the capi-controller-manager VPA maximum memory recommendation to prevent health-check failures and restarts under memory pressure.
  • [node-manager] Stop caps-controller-manager from rewriting StaticInstance objects in a hot loop when a connectivity check fails, which caused a sustained load on etcd.
    A StaticInstance whose connectivity check fails now keeps its reservation for the whole bootstrap window instead of returning to the pool on every failed attempt, and the bootstrap (20 min) and cleanup (10 min) timeouts, which previously never fired, are now reachable. If nothing ever ran on the host, the bootstrap timeout returns the instance to the pool after 20 minutes as before. If the host was already bootstrapped in part, the instance returns only through MachineHealthCheck remediation (nodeStartupTimeoutSeconds: 1200) plus the cleanup timeout, so it becomes available again roughly 20 + 10 minutes after the failure, and remediation reboots the host as part of the cleanup.
  • [node-manager] Fix duplicate NVIDIA GPU dashboards when the gpu module is enabled.
  • [node-manager] Bump google.golang.org/grpc to v1.83.2 in bashible-apiserver to fix CVE-2026-84304 and CVE-2026-84445.
  • [node-manager] Bump golang.org/x/crypto to v0.55.0 and google.golang.org/grpc to v1.83.2 in capi-controller-manager to fix CVE-2026-56854, CVE-2026-84304 and CVE-2026-84445.
  • [node-manager] Fixes CVE vulnerabilities in caps-controller-manager.
  • [node-manager] Hooks no longer fail the module run when a NodeGroup, Node or NodeUser is deleted between the snapshot and the patch. Such failures delayed the readiness of the deckhouse pod in clusters that delete NodeGroups often.
  • [registry] Bump dependencies in registry.
  • [registry] Bump golang.org/x/crypto in nodeservices-manager to fix CVE-2026-56854, add VEX statements for x/crypto/ssh findings.
  • [registrypackages] Bumped containerd to 1.7.36 / 2.2.9 and OpenTelemetry to 1.45.0 in containerd and crictl to fix CVEs.
  • [service-with-healthchecks] Fixed metrics endpoints being reachable without going through kube-rbac-proxy authorization.
  • [terraform-manager] Bump google.golang.org/grpc to v1.82.2 in opentofu to fix CVE-2026-84445. CVE-2026-84304 is not exploitable in opentofu and is covered by VEX.
  • [terraform-manager] Fixed the to-tofu-migrator init container of terraform-auto-converger failing to resolve the provider bundle digest, which kept auto-converge from starting.
  • [user-authn] An idTokenTTL of 6h or more no longer stops Deckhouse from starting; it raises the D8UserAuthnIDTokenTTLTooLong alert and blocks the update to the next minor release until it is lowered.
    A cluster still on 1.76 whose user-authn ModuleConfig sets idTokenTTL to 6h or more updates to this release without Deckhouse crash-looping, and stays on it until idTokenTTL is lowered below 6h. A cluster already crash-looping on 1.77.0 or 1.77.1 does not recover by itself, since the release controller runs in the pod that is down; set failurePolicy Ignore on the module-configs.deckhouse-webhook.deckhouse.io webhook of the deckhouse-webhook ValidatingWebhookConfiguration, lower idTokenTTL below 6h, and restart the deckhouse pod.
  • [user-authn] Fixed CVEs in the module images.
  • [user-authn] In Deckhouse CSE, a DexProvider with enableBasicAuth is rejected, since the edition does not ship the basic authentication proxy; the CSE copy of the CRD also gets the LDAP TLS checks of the module.
    In Deckhouse CSE, a DexProvider that already has enableBasicAuth: true keeps it after the update, and any change to its oidc or ldap section is rejected until enableBasicAuth is set to false. The same holds for an LDAP provider that combines insecureNoSSL with startTLS, insecureSkipVerify or rootCAData. Set enableBasicAuth: false and resolve the LDAP TLS settings on such providers.
  • [user-authz] Fixed CVEs in the permission-browser-apiserver image.
  • [user-authz] The authorization webhook answers 500 and logs the real error instead of terminating when it cannot marshal a response.

1.77.1

Know before update

  • A Deckhouse release suspended on its release channel is no longer applied by clusters that are behind and reach it through a step-by-step update. The suspend flag lives only in the release-channel image; previously it was dropped when the target release was built from its per-version image, so lagging clusters updated to a suspended release anyway.
  • After the update, the controller adds the annotations and labels of every ServiceWithHealthchecks to the Service created for it. If a ServiceWithHealthchecks of the LoadBalancer type carries annotations that configure the load balancer (network.deckhouse.io/load-balancer-ips, network.deckhouse.io/load-balancer-shared-ip-key and similar), the load balancer controller applies them and may assign a different address to the service, which causes a short interruption of the connections.
  • CAPI NodeGroups whose MachineDeployment was stuck on an old MachineTemplate are rolled to the current InstanceClass after the update, within maxSurgePerZone/maxUnavailablePerZone. Groups already on the current template are not touched.
  • D8UserAuthzRenderSlow requires at least three renders averaging above 30 seconds over 30 minutes, D8UserAuthzReleaseSlow at least two releases averaging above 5 minutes over one hour, and both must hold for 15 minutes. Severities are one step lower (6 and 7). Their descriptions now point at CPU shortage on the node rather than at the number of ClusterAuthorizationRules, which does not affect render time.
  • During the first user-authz release after the update the per-role bindings are protected from the release engine, the aggregated ones are created, and the per-role bindings are deleted right after the release. Permissions granted through annotated ClusterRoles stay in place throughout. The user-authz.deckhouse.io/access-level label is now set automatically on annotated ClusterRoles. In audit logs, access granted through annotated ClusterRoles is attributed to user-authz:<level>:custom instead of the individual ClusterRole.
  • Endpoints for pods in a terminal phase (Failed/Succeeded) are no longer published. In DVP clusters this prevents traffic from being routed to a VirtualMachine IP that has been reused by another pod. Pods being deleted are now published with the serving and terminating conditions, which enables the graceful shutdown flow for consumers. Pod readiness is derived from the PodReady condition, and stale probe results are reset when a pod becomes not ready, is recreated, or changes its IP.
  • The descheduler Deployment in d8-descheduler restarts once on upgrade to pick up the new argument. Descheduling behavior is unchanged for workloads that do not implement their own eviction policy.
  • The service-with-healthchecks controller and agent are restarted. A ServiceWithHealthchecks that was left in NotAllEndpointsAreReady with no EndpointSlice recovers automatically once the updated agent starts, no manual action is needed.
  • The installer, the deckhouse ServiceAccount, holders of the cluster administrator kubeconfig and the kube-system / d8-system ServiceAccounts can create a User or a Group named by an existing authorization rule without the acknowledgement annotation. Every other requester is checked as before.

Features

  • [candi] Base images updated to v3
  • [descheduler] Always run the descheduler with the upstream EvictionsInBackground feature gate enabled.
    The descheduler Deployment in d8-descheduler restarts once on upgrade to pick up the new argument. Descheduling behavior is unchanged for workloads that do not implement their own eviction policy.
  • [docs] Add managed-valkey module architecture section.
  • [docs] Refactor build pipeline and add CSE edition support.
  • [docs] Update virtualization module architecture section according to release v1.10.0
  • [docs] Add monitoring-kubernetes module architecture section.
  • [docs] Add documentation for Managed PostgreSQL service.
  • [docs] Add extended-monitoring module architecture section.
  • [docs] Add registry-packages-proxy module architecture section.
  • [docs] Expand the documentation for virtualization.
  • [docs] Add istio module architecture section.
  • [registry] Migration part of new registry implementation.
  • [user-authn] Update the Dex authentication page to the new Deckhouse branding, with light and dark themes selected by the user’s system settings.

Fixes

  • [candi] Fix hanging sysctl tuner on nodes with many loop and dm devices
  • [candi] containerd v2 no longer starts without the erofs snapshotter, and loads the erofs module before start.
  • [cert-manager] Restore the cert-manager and cainjector permissions that were lost against the upstream chart, including access to ListenerSet resources.
  • [chrony] Fixed the unit-detection condition in the disable-ntp-on-node.sh node step.
  • [cloud-provider-aws] fix GetCapacity not implemented error spam in logs
  • [cloud-provider-aws] Fixed CVEs in the cloud-provider-aws module.
  • [cloud-provider-azure] fix GetCapacity not implemented error spam in logs
  • [cloud-provider-azure] Fix multimaster bootstrap by falling back to the d8-masters-kubernetes-data-device-path Secret when the master data device LUN is not provided via cloud-init.
  • [cloud-provider-dvp] fix GetCapacity not implemented error spam in logs
  • [cloud-provider-dvp] Fixed CVEs in the cloud-provider-dvp module.
  • [cloud-provider-gcp] fix GetCapacity not implemented error spam in logs
  • [cloud-provider-gcp] Fixed CVEs in the cloud-provider-gcp module.
  • [cloud-provider-huaweicloud] Fixed CVEs in the cloud-provider-huaweicloud module.
  • [cloud-provider-openstack] Fixed CVEs in the cloud-provider-openstack module.
  • [cloud-provider-yandex] Fixed CVEs in the cloud-provider-yandex module.
  • [cloud-provider-zvirt] Resize the boot disk of a recreated VM, let a replaced VM actually power off, and recreate the VM when customNetworkConfig changes.
  • [deckhouse] Exclude all Python test files from webhook image.
  • [deckhouse] Keep x-kubernetes-sensitive-data on CRDs that have a conversion webhook.
  • [deckhouse] Allow editing and deleting the resources of an application under maintenance.
  • [deckhouse-controller] Honor a channel-level release suspend for clusters that reach the suspended version through a step-by-step update.
    A Deckhouse release suspended on its release channel is no longer applied by clusters that are behind and reach it through a step-by-step update. The suspend flag lives only in the release-channel image; previously it was dropped when the target release was built from its per-version image, so lagging clusters updated to a suspended release anyway.
  • [deckhouse-controller] Stop the module source registry fan-out from replaying and flooding the main queue with duplicate moduleRun tasks.
  • [deckhouse-controller] Do not commit the package repository registry checksum when no application could be annotated, which left them all on stale registry settings.
  • [docs] Fix broken JUnit XML report links on the Kubernetes conformance page.
  • [docs] Clarify global storageClass behavior for Prometheus
  • [docs] Fix alert rendering in OpenAPI specs.
  • [istio] Waypoints are no longer advertised as cross-network east-west gateways.
  • [istio] Enable Istio Gateway API deployment and gatewayclass controllers for istiod 1.27 and 1.29.
  • [istio] Document that ambient-mode workloads cannot take part in federation or multicluster.
  • [istio] Fix istiod RBAC rules that drifted away from upstream.
  • [multitenancy-manager] Refresh grant webhook CA bundles after the admission certificate is rotated.
  • [node-manager] node-controller reads the CAPI instance-class checksum from a helm-rendered ConfigMap instead of a possibly stale MachineTemplate, so MachineDeployments follow InstanceClass changes and their bootstrap Secrets keep a valid token.
    CAPI NodeGroups whose MachineDeployment was stuck on an old MachineTemplate are rolled to the current InstanceClass after the update, within maxSurgePerZone/maxUnavailablePerZone. Groups already on the current template are not touched.
  • [node-manager] node-controller no longer removes taints set by other components on the first reconcile of Static and CloudPermanent nodes.
  • [registry] Fix the order in which the Unmanaged params are stored for the fallback to the Legacy mode.
  • [registry] Fix the security context in the registry module for CSE.
  • [registry-packages-proxy] Bump golang.org/x/{crypto,net,sys,text} to clear 24 CVEs, including CVE-2026-56854 (CRITICAL) and CVE-2026-25681, CVE-2026-39821, CVE-2026-46600 (HIGH).
  • [registrypackages] Fix dm-verity devices that could never be closed for containerd 2.2.7 (CSE only).
  • [registrypackages] Bumped containerd to 1.7.35/2.2.8; bumped grpc, x/crypto, and websocket in containerd, crictl, and Kubernetes 1.32–1.36 gomod patches.
  • [registrypackages] Fsync the EROFS layer blob in containerd before committing snapshot metadata, so an unclean shutdown can no longer leave an unmountable image layer.
  • [service-with-healthchecks] Annotations and labels of a ServiceWithHealthchecks are now copied to the Service created for it.
    After the update, the controller adds the annotations and labels of every ServiceWithHealthchecks to the Service created for it. If a ServiceWithHealthchecks of the LoadBalancer type carries annotations that configure the load balancer (network.deckhouse.io/load-balancer-ips, network.deckhouse.io/load-balancer-shared-ip-key and similar), the load balancer controller applies them and may assign a different address to the service, which causes a short interruption of the connections.
  • [service-with-healthchecks] Stopped publishing terminated pods in EndpointSlices and started publishing pods being deleted as terminating endpoints.
    Endpoints for pods in a terminal phase (Failed/Succeeded) are no longer published. In DVP clusters this prevents traffic from being routed to a VirtualMachine IP that has been reused by another pod. Pods being deleted are now published with the serving and terminating conditions, which enables the graceful shutdown flow for consumers. Pod readiness is derived from the PodReady condition, and stale probe results are reset when a pod becomes not ready, is recreated, or changes its IP.
  • [service-with-healthchecks] Fixed a load balancer losing all its endpoints for good after a target pod was replaced.
    The service-with-healthchecks controller and agent are restarted. A ServiceWithHealthchecks that was left in NotAllEndpointsAreReady with no EndpointSlice recovers automatically once the updated agent starts, no manual action is needed.
  • [tools] Bump d8-cli to v0.33.17 - d8 mirror push restores the release channel tags of the installer image, so docker run <repo>/install:<channel> works against a mirrored registry
  • [tools] Bump d8-cli to v0.33.19 - closes the CVE findings in the bundled d8 binary (grpc 1.83.2 and x/image 0.45.0) and adds resume of interrupted d8 data and snapshot transfers
  • [user-authz] The identity collision check no longer applies to the platform’s own identities, so a retried bootstrap creates the initial User again instead of timing out.
    The installer, the deckhouse ServiceAccount, holders of the cluster administrator kubeconfig and the kube-system / d8-system ServiceAccounts can create a User or a Group named by an existing authorization rule without the acknowledgement annotation. Every other requester is checked as before.
  • [user-authz] Grant custom ClusterRoles (annotated with user-authz.deckhouse.io/access-level) through one aggregated ClusterRole per access level, so the number of bindings per ClusterAuthorizationRule/AuthorizationRule no longer depends on the number of such roles.
    During the first user-authz release after the update the per-role bindings are protected from the release engine, the aggregated ones are created, and the per-role bindings are deleted right after the release. Permissions granted through annotated ClusterRoles stay in place throughout. The user-authz.deckhouse.io/access-level label is now set automatically on annotated ClusterRoles. In audit logs, access granted through annotated ClusterRoles is attributed to user-authz:<level>:custom instead of the individual ClusterRole.
  • [user-authz] Group the module alerts under a group named apart from the alerts themselves, so the incident shelf accepts them.
  • [user-authz] The D8UserAuthzRenderSlow and D8UserAuthzReleaseSlow alerts no longer fire on a single slow render or release, such as a cluster bootstrap under CPU pressure or the one-time migration pass.
    D8UserAuthzRenderSlow requires at least three renders averaging above 30 seconds over 30 minutes, D8UserAuthzReleaseSlow at least two releases averaging above 5 minutes over one hour, and both must hold for 15 minutes. Severities are one step lower (6 and 7). Their descriptions now point at CPU shortage on the node rather than at the number of ClusterAuthorizationRules, which does not affect render time.

Version 1.76

Important

  • The fencing-agent component of the node-manager module now uses a gossip-based protocol (memberlist library) for distributed node health monitoring. This reduces the risk of false worker node reboots when the control plane is unavailable or the API server is under heavy load.

  • Fixed an issue where cluster-autoscaler could get stuck after a failed machine creation in the cloud (for errors other than ResourceExhausted). The cluster now automatically recovers its ability to scale after temporary cloud provider failures.

Deckhouse subsystem

  • Added support for deploying a DP cluster using the Proxy mode for accessing the container registry.

  • Kubectl on cluster nodes has been replaced with the d8 k alias (Deckhouse CLI utility).

  • Changes in dhctl (CLI installer):

    • Added support for running dhctl as a standalone binary without downloading the full image with all dependencies. Required components are pulled from the registry during execution, reducing startup time.

    • Added validation of public SSH keys. Invalid keys are now detected earlier, before they can cause connection issues.

    • Improved handling of TLS certificates for registries. The system certificate pool is now used together with a custom CA.

    • Added validation of InstanceClass against the selected cloud provider before installation, allowing early detection and installation abort in case of misconfiguration.

    • Fixed namespace configuration during bootstrap. Dhctl now updates settings even if the namespace already exists.

  • Improved multi-master resiliency. Kubernetes versions are now correctly detected even when some master nodes are unavailable. Kube-apiserver no longer routes requests to etcd nodes in the learner state.

Kubernetes & Scheduling subsystem

  • The d8-cluster-kubernetes ConfigMap now includes supported Kubernetes versions (supportedVersions), versions available for upgrade and downgrade (availableVersions), the current automatic version (automaticVersion).

  • Changes in the descheduler module:

  • The vertical-pod-autoscaler module updated to version 1.6.0. Key changes:

    • The InPlaceOrRecreate mode is now used by default. In this mode, VPA attempts to adjust pod requests and limits without recreating pods (in-place updates).

    • Reduced likelihood of pod restarts when in-place updates are not possible (via the --in-place-skip-disruption-budget flag), also minReplicas value is no longer considered for in-place updates.

  • VPA memory recommendations are now rounded to 64 MiB, improving readability and reducing unnecessary pod restarts caused by minor recommendation fluctuations.

  • For Kubernetes 1.34 and later, the DRAExtendedResource feature gate is enabled by default.

  • Added the D8EtcdHighRequestsLatency alert to detect high etcd request latency, which can lead to API server timeouts. A corresponding metric has been added to the “Control Plane Status” dashboard.

IAM subsystem

  • Changes in the user-authn module:

    • Disabled the insecure OAuth 2.0 Implicit Flow. In that flow, access tokens are passed in URL fragments, posing leakage risks (for example, via the Referer header, browser history, or logs). Any integrations using the Authorization Code Flow are unaffected.

    • Added SAML 2.0 support via the DexProvider resource. DP clusters can now be connected to such identity providers as AD FS, Okta, Keycloak, OneLogin, and Shibboleth.

    • Added refresh token support for SAML in Dex. This allows DexAuthenticator и kubeconfig-generator to refresh tokens without a manual user login.

    • Added Single Logout (SLO) support. The identity provider can end user sessions via /saml/slo/{connector}.

    • Fixed stale cache issue on the login page by adding a Dex build-based cache parameter to the CSS URL.

  • In multitenancy mode, users without permissions defined via ClusterAuthorizationRule no longer see all namespaces in the Deckhouse web UI. The default behavior is now deny-by-default: users get an empty namespace list unless access is explicitly granted. This does not affect privileged groups (system:masters, kubeadm:cluster-admins). Documentation of the user-authz module now includes an example for granting access to all namespaces.

  • Added support for the service-with-health-checks module in isolated projects. A NetworkPolicy exception for the d8-service-with-healthchecks namespace has been added to the project template, which allows module agents to verify service availability and correctly operate under conditions of network isolation.

Security subsystem

  • Gatekeeper upgraded to 3.22.0 and Ratify to 1.4.0. Key changes include a unified CEL and Rego policy behavior, added namespace context support in policies, and a CLI tool for policy benchmarking.

  • The denyVulnerableImages setting has been moved from admission-policy-engine to operator-trivy, making it the single owner of this logic.

  • The cert-manager module upgraded to 1.20.0. Key changes include added support for Azure DNS Private Zones in the DNS01 challenge, added annotation acme.cert-manager.io/http01-ingress-ingressclassname for redefining IngressClasses in HTTP-01 solvers, and security fixes.

  • Updated vulnerable Go dependencies in multitenancy-manager (go-jose, golang.org/x/crypto, grpc, etc.).

Cluster & Infrastructure subsystem

  • Changes in the node-manager module:

    • Improved draining hook event generation for better observability of the node eviction process.

    • Fixed StaticInstance issue causing infinite infrastructure deletion loops. That prevents infinite cleanup and queueing cycles.

    • Apiserver-proxy rewritten from NGINX to native Go implementation for better stability and reconnection speed.

  • Changes in the cloud-provider-dvp module:

    • Added hybrid cluster support. Static master nodes and cloud worker nodes can now be used in a single cluster.

    • Improved converge reliability. Reworked the mechanism of resource readiness check and added the fail-fast behavior for possible errors when creating the infrastructure.

    • Added default StorageClass inheritance for Cluster API child clusters.

    • Improved VM lifecycle handling, reducing potential problems when deleting VMs and associated objects.

    • Added support for the masterNodeGroup.instanceClass.additionalDisks parameter that lets you configure additional disks for master nodes.

  • Changes in the cloud-provider-aws module:

    • Spot node draining logic was moved into DP, making the spot node handling more unified.

    • Fixed handling of regions without DescribeInstanceTopology, reducing the number of false IAM errors.

  • Changes in the cloud-provider-openstack module:

    • Added ipMode: Proxy support for LoadBalancer in Kubernetes 1.32 and later.

    • Added the csiDriver.fsGroupPolicy parameter for the CSI driver, which defines whether the volume supports changing owner and permissions before mounting.

  • Changes in the cloud-provider-vsphere module:

    • Enabled the CSI snapshotter, enabling volume snapshotting.

    • Fixed zone and datastore detection. Now the only considered zones are the ones specified in the provider configuration.

  • In the cloud-provider-azure module, added support for NVMe disks on Gen2 VMs.

  • In the cloud-provider-gcp module, added enableNestedVirtualization and additionalDisks parameters to GCPInstanceClass, which let you configure nodes for workloads using virtualization and storage scenarios.

  • In the cloud-provider-yandex module, fixed converge failure when removing externalIPAddresses.

  • DVP, Huawei Cloud, and zVirt providers were migrated to Cluster API v1beta2.

  • For Huawei Cloud and VCD providers, added SecurityPolicyException resources and enabled security policy validations.

Network subsystem

  • Added support for handling incoming traffic via Gateway API using the alb module.

  • Changes in the cni-cilium module:

    • Added conntrack table synchronization for live VM migration.

    • Added ICMP (ping) support via ExternalIP for LoadBalancer services with MetalLB.

  • Added support for switching between supported CNI plugins in DP clusters.

  • In the node-local-dns module, added disableIPv6 parameter to disable IPv6 DNS resolution.

The complete list of changes, including the updated components, is available in the changelog on GitHub.

1.76.17

Know before update

  • A diff will appear in the Terraform state for the base infrastructure: the E zone will be added. Because of this, the Terraform check will fail on the first master node. Run dhctl converge for the base infrastructure, and everything should get back to normal.

Features

  • [cloud-provider-yandex] Add support for the ru-central1-e zone in Yandex Cloud.
    A diff will appear in the Terraform state for the base infrastructure: the E zone will be added. Because of this, the Terraform check will fail on the first master node. Run dhctl converge for the base infrastructure, and everything should get back to normal.
  • [documentation] The in-cluster documentation now includes the guides.

Fixes

  • [candi] Restored the lost kubelet-disable-k-panic-check patch for k8s 1.35, so kubelet no longer enforces kernel.panic=10 on this version.
  • [ci] add skip docs validation label
  • [ci] Delete tests registry images
  • [cloud-provider-azure] use .normal.apiserverEndpoints in data device Secret fallback
  • [cloud-provider-vsphere] Fixed master node creation when the datastore is set as a path with parent folders.
  • [deckhouse] Restore kubectl plugin lookup in d8 k and d8 kubectl
  • [dhctl] Prevent dhctl converge from panicking when kubeconfig is used without an SSH host.
  • [docs] Removed mentions of Cilium from the inter-cluster cooperation documentation page.
  • [docs] Added experimental module enable note to the lifecycle badge.
  • [docs] Rendered all versions from oss.yaml versions array.
  • [istio] The Istio operator now recovers Helm releases stuck in the pending-* or uninstalling state.

1.76.16

Features

  • [docs] Add managed services modules architecture documentation section.

Fixes

  • [ci] Fix changelog-generator image tag in changelog-update job to use the shared CI_IMAGES_TAG
  • [cloud-provider-vcd] Fixes cluster bootstrap getting stuck when capcd-controller-manager cannot reach the API server and DNS before the CNI is ready.
  • [istio] Aligned CNI templates with upstream and fixed Istio 1.25 compatibility.
  • [istio] Fixed the istio module getting stuck when switching globalVersion from 1.21 while proxies are still connected to the 1.21 control plane.
  • [istio] Fix leaking istiod control plane after removing an Istio version or disabling the module.
  • [istio] Switching back to Istio 1.21 no longer removes the networking.istio.io/v1 API from the Istio CRDs and no longer breaks the istio release.

1.76.15

Know before update

  • A StaticInstance whose connectivity check fails now keeps its reservation for the whole bootstrap window instead of returning to the pool on every failed attempt, and the bootstrap (20 min) and cleanup (10 min) timeouts, which previously never fired, are now reachable. If nothing ever ran on the host, the bootstrap timeout returns the instance to the pool after 20 minutes as before. If the host was already bootstrapped in part, the instance returns only through MachineHealthCheck remediation (nodeStartupTimeoutSeconds: 1200) plus the cleanup timeout, so it becomes available again roughly 20 + 10 minutes after the failure, and remediation reboots the host as part of the cleanup.
  • The self-signed-generator init container now carries the fixed 10m / 25Mi it was always meant to have, the same as the identical init container of user-api, instead of the sum of the main container limits. A DexAuthenticator that set large limits no longer gets that sum as burst: generating the self-signed certificate then takes 3 to 4 seconds instead of under a second. It happens once per pod start and needs no action.
  • log-shipper

Features

  • [docs] Add istio module architecture section.
  • [docs] Add user-authn-controller, user-api and basic-auth-proxy components to user-authn module architecture section.

Fixes

  • [cert-manager] Restore the cert-manager and cainjector permissions that were lost against the upstream chart, including access to ListenerSet resources.
  • [chrony] Fixed the unit-detection condition in the disable-ntp-on-node.sh node step.
  • [cloud-provider-aws] fix GetCapacity not implemented error spam in logs
  • [cloud-provider-azure] Fix multimaster bootstrap by falling back to the d8-masters-kubernetes-data-device-path Secret when the master data device LUN is not provided via cloud-init.
  • [cloud-provider-azure] fix GetCapacity not implemented error spam in logs
  • [cloud-provider-dvp] fix GetCapacity not implemented error spam in logs
  • [cloud-provider-gcp] fix GetCapacity not implemented error spam in logs
  • [cloud-provider-vcd] Batch VM cache refresh on cache miss
  • [cloud-provider-yandex] fix “Address in use” failures when replacing nodes and when removing external IP addresses
  • [deckhouse] In upmeter, the monitoring-and-autoscaling/alertmanager probe is disabled by default and is turned on with the new alertmanagerProbe.enabled setting.
  • [deckhouse] Fixed the CVEs reported for the v1.76.14 scan slice across all Observability team module images by bumping grpc to v1.83.2, the OpenTelemetry family to v1.45.0, kin-openapi to v0.149.0, cel-go to v0.29.0, golang.org/x/crypto to v0.55.0 and the rest of the golang.org/x/* set, by moving pushgateway to 1.11.3 and image-availability-exporter to 0.16.0, and by building the log-shipper reloader with Go 1.25.13; advisories without a reachable fix are mitigated via VEX.
  • [deckhouse-controller] Stop the module source registry fan-out from replaying and flooding the main queue with duplicate moduleRun tasks.
  • [deckhouse-controller] Do not commit the package repository registry checksum when no application could be annotated, which left them all on stale registry settings.
  • [descheduler] Fix duplicate enabled key in doc-ru-deschedulers.yaml
  • [docs] Fixed FAQ section for ansible inventory.
  • [istio] Fix istiod RBAC rules that drifted away from upstream.
  • [log-shipper] Fixed an invalid VRL script generated by the replaceValue transformation.
    log-shipper
  • [node-manager] Stop caps-controller-manager from rewriting StaticInstance objects in a hot loop when a connectivity check fails, which caused a sustained load on etcd.
    A StaticInstance whose connectivity check fails now keeps its reservation for the whole bootstrap window instead of returning to the pool on every failed attempt, and the bootstrap (20 min) and cleanup (10 min) timeouts, which previously never fired, are now reachable. If nothing ever ran on the host, the bootstrap timeout returns the instance to the pool after 20 minutes as before. If the host was already bootstrapped in part, the instance returns only through MachineHealthCheck remediation (nodeStartupTimeoutSeconds: 1200) plus the cleanup timeout, so it becomes available again roughly 20 + 10 minutes after the failure, and remediation reboots the host as part of the cleanup.
  • [tools] Bump d8-cli to v0.32.6 - d8 mirror push restores the release channel tags of the installer image, so docker run <repo>/install:<channel> works against a mirrored registry
  • [tools] Bump d8-cli to v0.32.7 - closes the CVE findings in the bundled d8 binary (grpc 1.83.2, x/image 0.45.0, x/crypto 0.55.0 and x/mod 0.40.0)
  • [upmeter] Fixed ListenerSet upmeter namespace.
  • [user-authn] Render a valid Deployment for a DexAuthenticator whose resources limits are fractional or below the init container requests.
    The self-signed-generator init container now carries the fixed 10m / 25Mi it was always meant to have, the same as the identical init container of user-api, instead of the sum of the main container limits. A DexAuthenticator that set large limits no longer gets that sum as burst: generating the self-signed certificate then takes 3 to 4 seconds instead of under a second. It happens once per pod start and needs no action.

1.76.14

Know before update

  • D8UserAuthzRenderSlow requires at least three renders averaging above 30 seconds over 30 minutes, D8UserAuthzReleaseSlow at least two releases averaging above 5 minutes over one hour, and both must hold for 15 minutes. Severities are one step lower (6 and 7). Their descriptions now point at CPU shortage on the node rather than at the number of ClusterAuthorizationRules, which does not affect render time.
  • During the first user-authz release after the update the per-role bindings are protected from the release engine, the aggregated ones are created, and the per-role bindings are deleted right after the release. Permissions granted through annotated ClusterRoles stay in place throughout. The user-authz.deckhouse.io/access-level label is now set automatically on annotated ClusterRoles. In audit logs, access granted through annotated ClusterRoles is attributed to user-authz:<level>:custom instead of the individual ClusterRole.
  • The service-with-healthchecks controller and agent are restarted. A ServiceWithHealthchecks that was left in NotAllEndpointsAreReady with no EndpointSlice recovers automatically once the updated agent starts, no manual action is needed.

Features

  • [docs] Add extended-monitoring module architecture section.
  • [docs] Add registry-packages-proxy module architecture section.

Fixes

  • [service-with-healthchecks] Fixed a load balancer losing all its endpoints for good after a target pod was replaced.
    The service-with-healthchecks controller and agent are restarted. A ServiceWithHealthchecks that was left in NotAllEndpointsAreReady with no EndpointSlice recovers automatically once the updated agent starts, no manual action is needed.
  • [user-authn] Fixed DexAuthenticator pod creation under ResourceQuota by setting init container CPU/memory limits to the sum of main container limits.
  • [user-authz] Grant custom ClusterRoles (annotated with user-authz.deckhouse.io/access-level) through one aggregated ClusterRole per access level, so the number of bindings per ClusterAuthorizationRule/AuthorizationRule no longer depends on the number of such roles.
    During the first user-authz release after the update the per-role bindings are protected from the release engine, the aggregated ones are created, and the per-role bindings are deleted right after the release. Permissions granted through annotated ClusterRoles stay in place throughout. The user-authz.deckhouse.io/access-level label is now set automatically on annotated ClusterRoles. In audit logs, access granted through annotated ClusterRoles is attributed to user-authz:<level>:custom instead of the individual ClusterRole.
  • [user-authz] Group the module alerts under a group named apart from the alerts themselves, so the incident shelf accepts them.
  • [user-authz] The D8UserAuthzRenderSlow and D8UserAuthzReleaseSlow alerts no longer fire on a single slow render or release, such as a cluster bootstrap under CPU pressure or the one-time migration pass.
    D8UserAuthzRenderSlow requires at least three renders averaging above 30 seconds over 30 minutes, D8UserAuthzReleaseSlow at least two releases averaging above 5 minutes over one hour, and both must hold for 15 minutes. Severities are one step lower (6 and 7). Their descriptions now point at CPU shortage on the node rather than at the number of ClusterAuthorizationRules, which does not affect render time.

1.76.13

Know before update

  • A Deckhouse release suspended on its release channel is no longer applied by clusters that are behind and reach it through a step-by-step update. The suspend flag lives only in the release-channel image; previously it was dropped when the target release was built from its per-version image, so lagging clusters updated to a suspended release anyway.
  • After the update, the controller adds the annotations and labels of every ServiceWithHealthchecks to the Service created for it. If a ServiceWithHealthchecks of the LoadBalancer type carries annotations that configure the load balancer (network.deckhouse.io/load-balancer-ips, network.deckhouse.io/load-balancer-shared-ip-key and similar), the load balancer controller applies them and may assign a different address to the service, which causes a short interruption of the connections.
  • Endpoints for pods in a terminal phase (Failed/Succeeded) are no longer published. In DVP clusters this prevents traffic from being routed to a VirtualMachine IP that has been reused by another pod. Pods being deleted are now published with the serving and terminating conditions, which enables the graceful shutdown flow for consumers. Pod readiness is derived from the PodReady condition, and stale probe results are reset when a pod becomes not ready, is recreated, or changes its IP.
  • The SSH key and sudo password in SSHCredentials (used by CAPS to reach static nodes) are now returned as <omitted> to callers without get on the sshcredentials/sensitive subresource, masked in the audit log, and encrypted in etcd when apiserver.encryptionEnabled is on. d8:manage:infrastructure:viewer and :manager no longer see these values. CAPS still does, and so does every holder of a deckhouse.io wildcard, since * matches subresources: SuperAdmin, the kubeadm:cluster-admins group, and the deckhouse and webhook-handler SAs of d8-system.

    For masked readers the last-applied-configuration annotation disappears from get -o yaml, and creating an SSHCredentials with <omitted> fails with 422 Invalid — editing still works. Treat previously exposed keys as leaked and rotate them. With apiserver.encryptionEnabled on, rewrite existing objects to encrypt them: d8 k get sshcredentials -o json | d8 k replace -f -.

  • The descheduler Deployment in d8-descheduler restarts once on upgrade to pick up the new argument. Descheduling behavior is unchanged for workloads that do not implement their own eviction policy.
  • The service-with-healthchecks status logic was heavily refactored to reduce API and etcd load. If you rely on lastProbeTime observability on every probe, explicitly enable verboseStatus in the module configuration.
  • The installer, the deckhouse ServiceAccount, holders of the cluster administrator kubeconfig and the kube-system / d8-system ServiceAccounts can create a User or a Group named by an existing authorization rule without the acknowledgement annotation. Every other requester is checked as before.

Features

  • [descheduler] Always run the descheduler with the upstream EvictionsInBackground feature gate enabled.
    The descheduler Deployment in d8-descheduler restarts once on upgrade to pick up the new argument. Descheduling behavior is unchanged for workloads that do not implement their own eviction policy.
  • [docs] Add managed-valkey module architecture section.
  • [docs] Add monitoring-kubernetes module architecture section.
  • [docs] Add documentation for Managed PostgreSQL service.

Fixes

  • [ci] Add REGCTL_CONFIG
  • [ci] Fix regctl in 1.76 build.
  • [common] Quote user-controlled values in the prometheus, upmeter, prometheus-pushgateway and extended-monitoring templates to prevent YAML injection into the module release.
  • [deckhouse] validate the hardlink target when extracting module and package images
  • [deckhouse-controller] Honor a channel-level release suspend for clusters that reach the suspended version through a step-by-step update.
    A Deckhouse release suspended on its release channel is no longer applied by clusters that are behind and reach it through a step-by-step update. The suspend flag lives only in the release-channel image; previously it was dropped when the target release was built from its per-version image, so lagging clusters updated to a suspended release anyway.
  • [deckhouse-controller] Fix tests for check_release_tests
  • [docs] Documented the identity collision rules for local Group and User objects and the allow-access-to-kubernetes annotation, and corrected the described scope of a security subsystem manage role.
  • [docs] add static cluster OS change for master nodes and Stronghold backup alerts
  • [node-manager] Fix NodeCapacity calculation.
  • [node-manager] Hide the CAPS SSH key and sudo password in SSHCredentials from users without the sshcredentials/sensitive subresource.
    The SSH key and sudo password in SSHCredentials (used by CAPS to reach static nodes) are now returned as <omitted> to callers without get on the sshcredentials/sensitive subresource, masked in the audit log, and encrypted in etcd when apiserver.encryptionEnabled is on. d8:manage:infrastructure:viewer and :manager no longer see these values. CAPS still does, and so does every holder of a deckhouse.io wildcard, since * matches subresources: SuperAdmin, the kubeadm:cluster-admins group, and the deckhouse and webhook-handler SAs of d8-system.

    For masked readers the last-applied-configuration annotation disappears from get -o yaml, and creating an SSHCredentials with <omitted> fails with 422 Invalid — editing still works. Treat previously exposed keys as leaked and rotate them. With apiserver.encryptionEnabled on, rewrite existing objects to encrypt them: d8 k get sshcredentials -o json | d8 k replace -f -.

  • [service-with-healthchecks] Fixed an API server overload issue (“status storm”), resolved validation errors for ClusterIP services, corrected pod readiness evaluation logic, and improved code quality.
    The service-with-healthchecks status logic was heavily refactored to reduce API and etcd load. If you rely on lastProbeTime observability on every probe, explicitly enable verboseStatus in the module configuration.
  • [service-with-healthchecks] Annotations and labels of a ServiceWithHealthchecks are now copied to the Service created for it.
    After the update, the controller adds the annotations and labels of every ServiceWithHealthchecks to the Service created for it. If a ServiceWithHealthchecks of the LoadBalancer type carries annotations that configure the load balancer (network.deckhouse.io/load-balancer-ips, network.deckhouse.io/load-balancer-shared-ip-key and similar), the load balancer controller applies them and may assign a different address to the service, which causes a short interruption of the connections.
  • [service-with-healthchecks] Stopped publishing terminated pods in EndpointSlices and started publishing pods being deleted as terminating endpoints.
    Endpoints for pods in a terminal phase (Failed/Succeeded) are no longer published. In DVP clusters this prevents traffic from being routed to a VirtualMachine IP that has been reused by another pod. Pods being deleted are now published with the serving and terminating conditions, which enables the graceful shutdown flow for consumers. Pod readiness is derived from the PodReady condition, and stale probe results are reset when a pod becomes not ready, is recreated, or changes its IP.
  • [user-authz] The identity collision check no longer applies to the platform’s own identities, so a retried bootstrap creates the initial User again instead of timing out.
    The installer, the deckhouse ServiceAccount, holders of the cluster administrator kubeconfig and the kube-system / d8-system ServiceAccounts can create a User or a Group named by an existing authorization rule without the acknowledgement annotation. Every other requester is checked as before.

1.76.12

Know before update

  • The vpa-recommender pod is restarted and recommendations are recalculated with the new granularity. Memory recommendations will generally become lower (less over-provisioning), CPU recommendations become multiples of 10m. VPA objects with updateMode: Auto/Recreate may evict and recreate pods to apply the updated requests.
  • ValidatingAdmissionPolicy reserved-public-hosts-* is removed. Ingresses that were denied because they matched publicDomainTemplate are allowed again. settings.reservedPublicHosts in ModuleConfig deckhouse remains accepted and is ignored. Enforcement returns in a later release.

Fixes

  • [deckhouse] Stop reserving platform public hostnames at admission; keep the ModuleConfig field so existing settings still validate.
    ValidatingAdmissionPolicy reserved-public-hosts-* is removed. Ingresses that were denied because they matched publicDomainTemplate are allowed again. settings.reservedPublicHosts in ModuleConfig deckhouse remains accepted and is ignored. Enforcement returns in a later release.
  • [deckhouse] Fixed goroutine and memory leak in upmeter-agent caused by per-request HTTP clients leaving idle keep-alive connections to Prometheus open forever.
  • [deckhouse-controller] Fixed a bug when PackageRepository always used a HTTPS scheme.
  • [docs] Add InPlaceOrRecreate to VPA modes list and move Auto to a warning alert.
  • [local-path-provisioner] Add wildcard tolerations to the helper pod template so PVC provisioning works on tainted nodes after local-path-provisioner v0.0.32+.
  • [user-authn] Commander releases older than 1.18 can grant Kubernetes API access on DexClient again.
  • [vertical-pod-autoscaler] Reduce VPA memory recommendation rounding from 64Mi to 16Mi and round CPU recommendations up to 10m.
    The vpa-recommender pod is restarted and recommendations are recalculated with the new granularity. Memory recommendations will generally become lower (less over-provisioning), CPU recommendations become multiples of 10m. VPA objects with updateMode: Auto/Recreate may evict and recreate pods to apply the updated requests.

1.76.11

Know before update

  • Containers attached through the pods/ephemeralcontainers subresource were not seen by the module’s webhook, so no policy was evaluated for them: SecurityPolicy, OperationPolicy, Pod Security Standards and image signature verification were all bypassed by kubectl debug. After the update these requests are validated.

    On clusters with restricted Pod Security Standards, deny-exec or strict SecurityPolicy, typical debug images (root user, writable root filesystem, interactive -it) will now be rejected. Adjust the policies or use a SecurityPolicyException for the namespaces where debugging must stay available.

    The webhook uses failurePolicy: Fail, so while Gatekeeper is unavailable kubectl debug will not work in user namespaces. System namespaces are excluded from the webhook and are not affected.

  • Creating a Group or a User, or renaming an existing one, so that Group.spec.name matches a group subject or User.spec.email matches a user subject of an existing AuthorizationRule or ClusterAuthorizationRule is now rejected, preventing an unintended privilege grant. The email is compared after lowercasing, since that is the form that reaches the token. Set the user-authz.deckhouse.io/allow-authorization-rule-collision annotation to "true" on the object if the collision is intentional. Already existing objects keep working and only produce a warning. Any declarative flow that manages both the rule and the identity is affected, not only one that deletes and recreates the identity: a first-time apply denies the Group or User whenever the rule happens to be applied first, and nothing guarantees the order. Add the annotation to such objects before upgrading. Deleting a Group or a User whose name is still a subject of a rule now produces a warning, since the rule keeps granting that name.
  • On upgrade, every DexAuthenticator that carries the dexauthenticator.deckhouse.io/allow-access-to-kubernetes annotation gets a freshly generated client secret, because it previously received the client secret of the privileged kubernetes OAuth2 client instead. Its pods roll once and users of those applications may have to sign in again. Each rotation is written to the Deckhouse log as Rotating DexAuthenticator client secret that reused the shared kubernetes client secret, with the authenticator name and namespace.

    If that annotation was ever used outside the d8-* and kube-* namespaces, the shared secret must be considered disclosed to anyone who could read Secrets in those namespaces, and has to be rotated by hand. Deleting the Secret alone does not rotate it: while the value is non-empty in memory the owning hook re-renders the same one. Rotate it as follows.

    1. If a GitOps tool syncs the d8-user-authn namespace, exclude Secret/kubernetes-dex-client-app-secret from that sync first, otherwise it is restored to the old value.
    2. Blank the stored value: kubectl -n d8-user-authn patch secret kubernetes-dex-client-app-secret --type merge -p '{"data":{"secret":""}}'
    3. Drop the in-memory copy: kubectl -n d8-system rollout restart deployment/deckhouse
    4. Check that kubectl -n d8-user-authn get secret kubernetes-dex-client-app-secret -o jsonpath='{.data.secret}' now differs from the previous value. If it does not, the module re-rendered the old value before the restart took effect; repeat steps 2 and 3.

    The blast radius of that rotation is wider than the kubernetes client alone. The same value is the client secret of the kubeconfig-generator, kubeconfig-publish-api and every kubeconfig-<slug> OAuth2 client, and it is passed to basic-auth-proxy as --ldap-client-secret. In-cluster consumers are re-rendered and their pods roll automatically, but kubeconfig files already downloaded from the kubeconfig generator carry the old client secret and can no longer refresh their tokens, so their users must download a new kubeconfig. Already issued ID tokens keep working until they expire, at most settings.idTokenTTL.

    Deleting and recreating a DexAuthenticator is always a safe way to force a fresh per-authenticator secret: the credentials Secret is regenerated with the resource. Its content is owned by Deckhouse, so no GitOps change is needed for the per-authenticator rotation itself.

  • Only a subject allowed to update the user-authn ModuleConfig, or the Deckhouse service account, may add the dexclient.deckhouse.io/allow-access-to-kubernetes or dexauthenticator.deckhouse.io/allow-access-to-kubernetes annotation. Objects that already carry it keep working and stay editable by their owners as long as they are updated in place. A GitOps controller that deletes and recreates such an object instead of patching it will have the recreation denied, and the object will come back without access to the Kubernetes API unless the controller’s service account is allowed to update the user-authn ModuleConfig.
  • Project administrators (user-authz Admin access level) can no longer create, update or delete SecurityPolicyException resources; the permission now requires ClusterAdmin. Clusters that granted accessLevel Admin to untrusted tenants over RBAC v1 were exposed to node-level privilege escalation. RBAC v2 use/manage roles were never affected.
  • Setting the annotation no longer requires write access to the user-authn ModuleConfig. Module administrators are unaffected, as their roles already carry the new permission, and so are the platform’s own components in d8-system and kube-system. Any other subject that sets the annotation, a cluster provisioner in particular, has to be granted create on the subresource, which for a single namespace is:

    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
      name: allow-access-to-kubernetes
      namespace: <namespace>
    rules:
    - apiGroups: ["deckhouse.io"]
      resources: ["dexclients/allow-access-to-kubernetes", "dexauthenticators/allow-access-to-kubernetes"]
      verbs: ["create"]
    

    bound to the subject with a RoleBinding in the same namespace, or with a ClusterRole and a ClusterRoleBinding to grant it cluster-wide.

  • The d8:manage:permission:subsystem:kubernetes:view_resources role no longer grants nodes/proxy, nodes/log, nodes/metrics and nodes/stats. Users holding d8:manage:kubernetes:viewer, d8:manage:infrastructure:viewer or any role aggregating them lose access to the kubelet API through /api/v1/nodes/<node>/proxy/.... nodes themselves stay readable. Automation that relied on reaching the kubelet API with one of these roles must be granted a dedicated role explicitly.
  • spec.oidc.scopes items are now validated against ^\S+$ in both served versions of DexProvider. A scope containing whitespace could never work, because OAuth2 passes scopes as a space-delimited list, but it was previously admitted and silently broke authentication. Stored objects are not invalidated, however a DexProvider holding such a value is rejected on its next write until the value is corrected, which also affects a GitOps flow that reapplies the object unchanged.

Features

  • [docs] Add previous page and next page buttons to modules documentation

Fixes

  • [admission-policy-engine] Admission policies are now evaluated for ephemeral containers added with kubectl debug.
    Containers attached through the pods/ephemeralcontainers subresource were not seen by the module’s webhook, so no policy was evaluated for them: SecurityPolicy, OperationPolicy, Pod Security Standards and image signature verification were all bypassed by kubectl debug. After the update these requests are validated.

    On clusters with restricted Pod Security Standards, deny-exec or strict SecurityPolicy, typical debug images (root user, writable root filesystem, interactive -it) will now be rejected. Adjust the policies or use a SecurityPolicyException for the namespaces where debugging must stay available.

    The webhook uses failurePolicy: Fail, so while Gatekeeper is unavailable kubectl debug will not work in user namespaces. System namespaces are excluded from the webhook and are not affected.

  • [admission-policy-engine] Restrict SecurityPolicyException write access to ClusterAdmin, closing a tenant-to-node privilege escalation.
    Project administrators (user-authz Admin access level) can no longer create, update or delete SecurityPolicyException resources; the permission now requires ClusterAdmin. Clusters that granted accessLevel Admin to untrusted tenants over RBAC v1 were exposed to node-level privilege escalation. RBAC v2 use/manage roles were never affected.
  • [cloud-provider-yandex] A Node that cannot be resolved to a Yandex Instance no longer blocks target group synchronization for the whole cluster.
  • [cloud-provider-zvirt] Prevent capz-controller-manager from crashlooping without diagnostics when zVirt tags cannot be created.
  • [deckhouse] bump addon-operator to run the module startup phase unconditionally
  • [deckhouse] Deleted module documentation when a module is disabled.
  • [deckhouse] The reserved public hosts policies no longer make every aggregated apiserver in the cluster watch all ConfigMaps, which left permission-browser-apiserver permanently unready and stopped namespace deletion cluster-wide.
  • [docs] Fix the upmeter web UI Ingress name in SLA documentation
  • [testing] Copy Helm render capabilities per render instead of mutating the shared defaults, and allow tests to declare available API kinds.
  • [user-authn] Quote DexProvider-derived values rendered into the basic-auth-proxy Deployment and the Dex configuration file to prevent YAML injection.
  • [user-authn] Reject whitespace in DexProvider OIDC scopes, which the OAuth2 protocol cannot represent.
    spec.oidc.scopes items are now validated against ^\S+$ in both served versions of DexProvider. A scope containing whitespace could never work, because OAuth2 passes scopes as a space-delimited list, but it was previously admitted and silently broke authentication. Stored objects are not invalidated, however a DexProvider holding such a value is rejected on its next write until the value is corrected, which also affects a GitOps flow that reapplies the object unchanged.
  • [user-authn] Issue a dedicated OAuth2 client secret per DexAuthenticator instead of reusing the shared cluster client secret.
    On upgrade, every DexAuthenticator that carries the dexauthenticator.deckhouse.io/allow-access-to-kubernetes annotation gets a freshly generated client secret, because it previously received the client secret of the privileged kubernetes OAuth2 client instead. Its pods roll once and users of those applications may have to sign in again. Each rotation is written to the Deckhouse log as Rotating DexAuthenticator client secret that reused the shared kubernetes client secret, with the authenticator name and namespace.

    If that annotation was ever used outside the d8-* and kube-* namespaces, the shared secret must be considered disclosed to anyone who could read Secrets in those namespaces, and has to be rotated by hand. Deleting the Secret alone does not rotate it: while the value is non-empty in memory the owning hook re-renders the same one. Rotate it as follows.

    1. If a GitOps tool syncs the d8-user-authn namespace, exclude Secret/kubernetes-dex-client-app-secret from that sync first, otherwise it is restored to the old value.
    2. Blank the stored value: kubectl -n d8-user-authn patch secret kubernetes-dex-client-app-secret --type merge -p '{"data":{"secret":""}}'
    3. Drop the in-memory copy: kubectl -n d8-system rollout restart deployment/deckhouse
    4. Check that kubectl -n d8-user-authn get secret kubernetes-dex-client-app-secret -o jsonpath='{.data.secret}' now differs from the previous value. If it does not, the module re-rendered the old value before the restart took effect; repeat steps 2 and 3.

    The blast radius of that rotation is wider than the kubernetes client alone. The same value is the client secret of the kubeconfig-generator, kubeconfig-publish-api and every kubeconfig-<slug> OAuth2 client, and it is passed to basic-auth-proxy as --ldap-client-secret. In-cluster consumers are re-rendered and their pods roll automatically, but kubeconfig files already downloaded from the kubeconfig generator carry the old client secret and can no longer refresh their tokens, so their users must download a new kubeconfig. Already issued ID tokens keep working until they expire, at most settings.idTokenTTL.

    Deleting and recreating a DexAuthenticator is always a safe way to force a fresh per-authenticator secret: the credentials Secret is regenerated with the resource. Its content is owned by Deckhouse, so no GitOps change is needed for the per-authenticator rotation itself.

  • [user-authn] Drop the dead per-DexAuthenticator redirect URI from the privileged kubernetes OAuth2 client.
  • [user-authn] Adding the allow-access-to-kubernetes annotation to a DexClient or a DexAuthenticator now requires cluster-level authority over the user-authn module.
    Only a subject allowed to update the user-authn ModuleConfig, or the Deckhouse service account, may add the dexclient.deckhouse.io/allow-access-to-kubernetes or dexauthenticator.deckhouse.io/allow-access-to-kubernetes annotation. Objects that already carry it keep working and stay editable by their owners as long as they are updated in place. A GitOps controller that deletes and recreates such an object instead of patching it will have the recreation denied, and the object will come back without access to the Kubernetes API unless the controller’s service account is allowed to update the user-authn ModuleConfig.
  • [user-authn] The policy gating the allow-access-to-kubernetes annotation no longer applies to platform components, and proves authority with a create permission on the dexclients/allow-access-to-kubernetes and dexauthenticators/allow-access-to-kubernetes subresources instead of write access to the user-authn ModuleConfig.
    Setting the annotation no longer requires write access to the user-authn ModuleConfig. Module administrators are unaffected, as their roles already carry the new permission, and so are the platform’s own components in d8-system and kube-system. Any other subject that sets the annotation, a cluster provisioner in particular, has to be granted create on the subresource, which for a single namespace is:

    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
      name: allow-access-to-kubernetes
      namespace: <namespace>
    rules:
    - apiGroups: ["deckhouse.io"]
      resources: ["dexclients/allow-access-to-kubernetes", "dexauthenticators/allow-access-to-kubernetes"]
      verbs: ["create"]
    

    bound to the subject with a RoleBinding in the same namespace, or with a ClusterRole and a ClusterRoleBinding to grant it cluster-wide.

  • [user-authz] Reject local Group names and User emails that collide with a subject already granted privileges by an authorization rule.
    Creating a Group or a User, or renaming an existing one, so that Group.spec.name matches a group subject or User.spec.email matches a user subject of an existing AuthorizationRule or ClusterAuthorizationRule is now rejected, preventing an unintended privilege grant. The email is compared after lowercasing, since that is the form that reaches the token. Set the user-authz.deckhouse.io/allow-authorization-rule-collision annotation to "true" on the object if the collision is intentional. Already existing objects keep working and only produce a warning. Any declarative flow that manages both the rule and the identity is affected, not only one that deletes and recreates the identity: a first-time apply denies the Group or User whenever the rule happens to be applied first, and nothing guarantees the order. Add the annotation to such objects before upgrading. Deleting a Group or a User whose name is still a subject of a rule now produces a warning, since the rule keeps granting that name.
  • [user-authz] Remove node subresources from the RBACv2 viewer permission role
    The d8:manage:permission:subsystem:kubernetes:view_resources role no longer grants nodes/proxy, nodes/log, nodes/metrics and nodes/stats. Users holding d8:manage:kubernetes:viewer, d8:manage:infrastructure:viewer or any role aggregating them lose access to the kubelet API through /api/v1/nodes/<node>/proxy/.... nodes themselves stay readable. Automation that relied on reaching the kubelet API with one of these roles must be granted a dedicated role explicitly.

1.76.10

Know before update

  • Fixes recreation of all CloudEphemeral nodes on upgrade to 1.76.9. The MachineDeployment replica count was dropped and the MachineDeployment was scaled to zero.
  • Values taken from Project.spec.parameters are now quoted where they are substituted into the shipped project templates, and a parameter that changes the structure of the rendered manifests rather than only their values is refused for any template, including custom ones. An administrator name may no longer contain control characters, and clusterLogDestinationName must be a Kubernetes object name or empty. Both are checked from the reconcile loop as well as at admission, so a project already carrying such a value goes into an error state on its next reconcile rather than at its next edit; the same holds for a custom template that turns a parameter into several objects or into YAML, which is refused when the project is created, edited, or reconciled with a change to apply. The module’s admission policy now also matches CREATE, so objects labelled heritage: multitenancy-manager can only be created by the module itself.
  • d8:manage:permission:module:multitenancy-manager:edit no longer grants write access to projecttemplates. A ProjectTemplate renders into a release applied by a ServiceAccount bound to cluster-admin, and its spec.resourcesTemplate is arbitrary, so authoring one is equivalent to holding cluster-admin and is not a module-level permission.

    Reading templates stays in d8:manage:permission:module:multitenancy-manager:view, and creating a Project on an existing template stays in the edit role. The legacy ClusterAdmin access level and the kubeadm:cluster-admins group keep the right. A subject that authored templates through the manager role has to be granted one of those instead.

  • network-gateway pod’s will be restarted

Features

  • [docs] Add gpu module architecture section.
  • [docs] Added cilium-hubble module architecture section.
  • [docs] Added adaptive-resource-management module architecture section.
  • [docs] Added operator-trivy module architecture section.
  • [docs] Added managed-postgres and managed-memcached modules architecture sections.
  • [docs] Add pod-reloader module architecture section.
  • [docs] Add secrets-store-integration module architecture section.

Fixes

  • [cloud-provider-vcd] Fixes an intermittent “network still in use” error during cluster deletion with the WithNAT placement strategy.
  • [deckhouse] Fix values validation failing on the injected registry
  • [deckhouse-controller] The module-documentation build request timeout to docs-builder is configurable via DOCUMENTATION_BUILD_TIMEOUT (default 120s) instead of a fixed 10s that canceled healthy full-site builds.
  • [docs] Fix issues in managed-postgres and managed-memcached modules architecture sections.
  • [docs] Clarify which integrity control mechanisms are available in which DKP editions and correct the DKP module integrity description.
  • [docs] Link GPU driver install examples to gpu module FAQ
  • [docs] Updated the Deckhouse Kubernetes Platform GUI installer links on the installation page from the #gui-install anchor to the /gs/installer/ path.
  • [docs] Corrected the documentation that incorrectly stated DKP supports only Bearer token authentication for container registries.
  • [docs] Module documentation no longer gets stuck in Error under load; docs-builder builds are cancelable and no longer pile up redundant full-site rebuilds behind requests whose clients already timed out.
  • [istio] Restore Istiod pod anti-affinity for Istio 1.25 installations managed by the Sail Operator.
  • [multitenancy-manager] A project parameter can no longer inject objects into the rendered project manifests.
    Values taken from Project.spec.parameters are now quoted where they are substituted into the shipped project templates, and a parameter that changes the structure of the rendered manifests rather than only their values is refused for any template, including custom ones. An administrator name may no longer contain control characters, and clusterLogDestinationName must be a Kubernetes object name or empty. Both are checked from the reconcile loop as well as at admission, so a project already carrying such a value goes into an error state on its next reconcile rather than at its next edit; the same holds for a custom template that turns a parameter into several objects or into YAML, which is refused when the project is created, edited, or reconciled with a change to apply. The module’s admission policy now also matches CREATE, so objects labelled heritage: multitenancy-manager can only be created by the module itself.
  • [multitenancy-manager] The module manager role no longer grants authoring of project templates.
    d8:manage:permission:module:multitenancy-manager:edit no longer grants write access to projecttemplates. A ProjectTemplate renders into a release applied by a ServiceAccount bound to cluster-admin, and its spec.resourcesTemplate is arbitrary, so authoring one is equivalent to holding cluster-admin and is not a module-level permission.

    Reading templates stays in d8:manage:permission:module:multitenancy-manager:view, and creating a Project on an existing template stays in the edit role. The legacy ClusterAdmin access level and the kubeadm:cluster-admins group keep the right. A subject that authored templates through the manager role has to be granted one of those instead.

  • [network-gateway] Fixed user and group in ConfigMap
    network-gateway pod’s will be restarted
  • [node-manager] Pin MachineDeployment minReadySeconds to 1 so the MachineSet apply converges.
  • [node-manager] MachineDeployment spec.replicas is no longer dropped on upgrade, so cloud nodes are not recreated.
    Fixes recreation of all CloudEphemeral nodes on upgrade to 1.76.9. The MachineDeployment replica count was dropped and the MachineDeployment was scaled to zero.

1.76.9

Know before update

  • The node-local-dns DaemonSet pods will restart after the update.

Features

  • [docs] Add upmeter module architecture section.
  • [docs] Add descheduler module architecture section.
  • [docs] Add cni-cilium module architecture section.

Fixes

  • [cloud-provider-aws] Do not fail the node bootstrap or update when linux-modules-extra is unavailable for the running kernel.
  • [deckhouse] Locate CRDs by repository contents.
  • [deckhouse] stop false drift detection reinstalling packages with helm hooks
  • [deckhouse] An unset settings.update.mode now defaults to AutoPatch instead of silently running as Auto.
  • [deckhouse] Fix a minor Deckhouse release being auto-applied as a patch when no Deployed release object is present.
  • [deckhouse] limit the impact of a failed documentation build
  • [deckhouse] Surface layer read errors instead of extracting a truncated image
  • [deckhouse] Do not inject source registry into a still-embedded module
  • [deckhouse-controller] Module releases rendered with nelm no longer raise false absent-resource alerts.
  • [deckhouse-controller] nelm now takes over fields left by the old Helm 3 engine, so stale fields get removed on module upgrade.
  • [docs] Clarify that DKP does not support version downgrades
  • [docs] Add nil-safety guards for OpenAPI extraction to prevent panics.
  • [docs] Fixed incorrect description of allowedHostPaths in SecurityPolicy documentation.
  • [docs] Fix broken documentation links
  • [docs] Clarified AutoPatch update mode and external module update settings.
  • [docs] Fixed ModuleUpdatePolicy example in release channel FAQ.
  • [documentation] Fix a memory leak in the documentation builder so memory is released between builds; add CPU/memory limits and an opt-in profiling endpoint.
  • [istio] Add missing tools to proxyv2 images so the application-aware proxy termination hook works correctly.
  • [istio] Fix graceful draining of established HTTP connections when application pods terminate.
  • [istio] Create the ConfigMaps required for additional JWKS root CA certificates.
  • [keepalived] Fixed Python linking in the keepalived module image.
  • [network-gateway] Fixed Python linking in the network-gateway module image.
  • [node-local-dns] Bump Go dependencies in the coredns helper image to fix known CVEs.
    The node-local-dns DaemonSet pods will restart after the update.
  • [registry] Fix DKP bootstrap retry by keeping the registry-init secret.
  • [registrypackages] Rebuild kubernetes-cni with updated Go dependencies to fix CVEs.
  • [tools] Fixed module release validation incorrectly failing for !optional dependencies when the required module doesn’t exist in the cluster.
  • [user-authn] ClusterAdmin can watch the Dex storage CRDs (passwords, offlinesessionses), not only list them.

1.76.8

Know before update

  • All Ingress-nginx controller pods will be restarted.
  • The coredns and kube-dns components will restart after the update.
  • The egress-gateway-agent component will restart after the update.
  • The node-local-dns components will restart after the update.
  • The service-with-healthchecks components (controller, agent) will restart after the update.

Features

  • [docs] Update node-termination-handler behaviour in cloud-provider-aws module architecture section

Fixes

  • [cloud-provider-dvp] skip foreign nodes in cloud controller manager
  • [cni-cilium] Bump Go dependencies in the egress-gateway-agent image to fix known CVEs.
    The egress-gateway-agent component will restart after the update.
  • [deckhouse] Skip the multiple CNI check in the enable_cni hook while a CNI migration is in progress to avoid a stuck Deckhouse queue.
  • [deckhouse-controller] Re-derive the module release chain and align the from-to gap check with the release updater so intermediate releases are created instead of the chain freezing
  • [deckhouse-controller] ModuleDocumentation will not be created for embedded modules.
  • [ingress-nginx] Nginx version is updated to 1.30.4.
    All Ingress-nginx controller pods will be restarted.
  • [kube-dns] Bump Go dependencies in the sts-pods-hosts-appender-webhook and coredns images to fix known CVEs.
    The coredns and kube-dns components will restart after the update.
  • [node-local-dns] Bump Go dependencies in the safe-updater and stale-dns-connections-cleaner images to fix known CVEs.
    The node-local-dns components will restart after the update.
  • [service-with-healthchecks] Bump Go dependencies in the service-with-healthchecks image to fix known CVEs.
    The service-with-healthchecks components (controller, agent) will restart after the update.
  • [user-authn] Fix Dex token refresh with upstream providers that rotate refresh tokens (GitLab), which logged users out every idTokenTTL.
  • [user-authn] backport fix of 21615
  • [user-authn] Accessing kubernetes api via gateway api fixed.

1.76.7

Fixes

  • [cloud-provider-huaweicloud] Fixed the Machine initialization contract — new nodes could not join the cluster.

1.76.6

Know before update

  • All ingress-nginx controller pods of v1.10 will be restarted.
  • The cilium-hubble components (hubble-ui, hubble-relay) will restart after the update.
  • The cni-cilium components (cilium agent, operator) will restart after the update.
  • The metallb components (controller, speaker, l2lb) will restart after the update.
  • Users with accessLevel ClusterAdmin in ClusterAuthorizationRule gain read and write access to DVPInstanceClass objects; previously all access was denied.
  • Users with accessLevel ClusterAdmin in ClusterAuthorizationRule gain read and write access to DynamixInstanceClass objects; previously all access was denied.

Fixes

  • [candi] Add exception for Oracle Linux UEK kernels for fix CVE-2025-37999 on bashible
  • [cilium-hubble] Upgrade hubble-ui backend dependencies (cilium v1.17.16, Go 1.25.0) and switch build base image to fix known CVEs.
    The cilium-hubble components (hubble-ui, hubble-relay) will restart after the update.
  • [cloud-provider-dvp] Fix DVPInstanceClass access for users with user-authz ClusterAdmin role
    Users with accessLevel ClusterAdmin in ClusterAuthorizationRule gain read and write access to DVPInstanceClass objects; previously all access was denied.
  • [cloud-provider-dynamix] Fix DynamixInstanceClass access for users with user-authz ClusterAdmin role
    Users with accessLevel ClusterAdmin in ClusterAuthorizationRule gain read and write access to DynamixInstanceClass objects; previously all access was denied.
  • [cloud-provider-vsphere] fix missing default StorageClass annotation when a DatastoreCluster entry sorts before all Datastore entries
  • [cni-cilium] Bump Go dependencies and backport upstream cilium security patches to fix known CVEs.
    The cni-cilium components (cilium agent, operator) will restart after the update.
  • [common] Fix curl in debug-container by shipping a static binary
  • [common] Bump Go dependencies in the cni-migration-controller and cni-migration-init-checker images to fix known CVEs.
  • [deckhouse] atomically install modules and re-download incomplete versions
  • [deckhouse] Restore admin access to list moduleconfigs
  • [deckhouse] Fix hugo CVE.
  • [deckhouse-controller] Fix maintenance state tracking — module no longer gets stuck in Unmanaged after removing maintenance.
  • [deckhouse-controller] Force embedded modules back to the “Embedded” source on startup, healing a stale external source that previously stuck until the Module resource was deleted manually.
  • [docs] Added a note about the termination of support for Ubuntu 20.04 and replace the link to the Docker container registry.
  • [docs] Added a note about changing the name of the Docker container registry.
  • [documentation] Bump Hugo version to 0.163.3
  • [ingress-nginx] Controller 1.10 image uid and gid are set to deckhouse.
    All ingress-nginx controller pods of v1.10 will be restarted.
  • [metallb] Bump Go dependencies in the metallb and l2lb images to fix known CVEs.
    The metallb components (controller, speaker, l2lb) will restart after the update.
  • [node-manager] Fix create_rbac_and_certificate_for_kubernetes_api_proxy hook behavior.
  • [user-authn] User-authn httproutes are refactored.

1.76.5

Know before update

  • With enableMultiTenancy, effective access is now the union of CAR (within its limitNamespaces/namespaceSelector), AuthorizationRules, and plain RoleBindings/ClusterRoleBindings. Previously the webhook denied requests outside the CAR scope even when RBAC explicitly granted them: such existing bindings for subjects with a CAR silently become effective after the upgrade — review them. The CAR access level still does not apply outside its namespace limits. AccessibleNamespaces reflects the same union.
  • log-shipper

Fixes

  • [cert-manager] Additional letsencrypt-staging cluster issuer for Gateway API is provided.
  • [ci] This is a manual cherry-pick the script for the CI fix in the release branch
  • [cilium-hubble] Gateway API certificate manifest is updated to support any cluster issuer.
  • [cloud-provider-dvp] fix dvp kubernetes dependency mismatch
  • [cloud-provider-huaweicloud] Adds patches to the upstream version to make it ignore static nodes
  • [deckhouse] Bump addon-operator to fix module install with conversion webhooks
  • [deckhouse-controller] Bump addon-operator to v1.21.22 — propagate module context to nelm logs.
  • [deckhouse-tools] Gateway API certificate manifest is updated to support any cluster issuer.
  • [docs] Fix ingressClass description in global settings
  • [docs] Corrected the link from direct to relative in the alert about the module requirements.
  • [docs] Corrected the link from direct to relative in the alert about the module requirements for external modules.
  • [docs] Add missing sidebar title mappings for Istio documentation CR page.
  • [documentation] Gateway API certificate manifest is updated to support any cluster issuer.
  • [istio] Gateway API certificate manifest is updated to support any cluster issuer.
  • [log-shipper] fix daemonset template
    log-shipper
  • [openvpn] Gateway API certificate manifest is updated to support any cluster issuer.
  • [prometheus] Gateway API certificate manifest is updated to support any cluster issuer.
  • [registry] Gateway API certificate manifest is updated to support any cluster issuer.
  • [registry-packages-proxy] Gateway API certificate manifest is updated to support any cluster issuer.
  • [upmeter] Gateway API certificate manifest is updated to support any cluster issuer.
  • [user-authn] Gateway API certificate manifest is updated to support any cluster issuer. Kubernetes API HTTProute was updated to return 403 instead of 404.
  • [user-authz] Honor CAR-independent RBAC in webhook and permission-browser
    With enableMultiTenancy, effective access is now the union of CAR (within its limitNamespaces/namespaceSelector), AuthorizationRules, and plain RoleBindings/ClusterRoleBindings. Previously the webhook denied requests outside the CAR scope even when RBAC explicitly granted them: such existing bindings for subjects with a CAR silently become effective after the upgrade — review them. The CAR access level still does not apply outside its namespace limits. AccessibleNamespaces reflects the same union.

1.76.4

Know before update

  • kube-apiserver static pod is reconfigured and restarts on the next control-plane sync.

Features

  • [cloud-provider-dvp] Add an option to disable the CCM LoadBalancer functionality via ConfigMap.
  • [docs] Add operator-argo module architecture section
  • [docs] Add module module architecture section

Fixes

  • [candi] kube-apiserver no longer caches watches for ManifestCheckpointContentChunk resources from state-snapshotter.
    kube-apiserver static pod is reconfigured and restarts on the next control-plane sync.
  • [ci] Fix extract images_digests.
  • [deckhouse] Fix module metadata (module.yaml/oss.yaml) missing from in-cluster documentation by allowing these files in IsDocsPath (sender) and creating parent directories on extraction in the docs-builder (receiver).
  • [deckhouse] Fixes an unauthenticated path traversal (CWE-22) in docs-builder by validating moduleName/channels and enforcing base-directory containment before any filesystem operation.
  • [deckhouse] Remove documentation networkPolicy.
  • [deckhouse] Add log diagnostic command to ModuleIsNotValid alert description.
  • [deckhouse] Use metav1.Duration for ModulePullOverride scanInterval
  • [deckhouse-controller] Fix deckhouse-controller crash loop and hooks receiving silently empty snapshots
  • [deckhouse-controller] Fixed showing warnings while errors during kubectl edit.
  • [deckhouse-controller] Fixed ModuleConfig validation.
  • [docs] Document self-service password reset
  • [documentation] remove stale fox links
  • [user-authn] Preserve Dex Password fields (notably groups) when resetting password, locking or unlocking a user, so users are no longer locked out after these operations.

1.76.3

Know before update

  • All ingress-nginx controllers pod will be restarted.
  • Previously, a transient cluster DNS failure could cause the user-authz-webhook liveness probe to fail and restart the pod, which combined with the fail-closed authorization webhook (failurePolicy: Deny) could deny all API requests, including cluster-admins, until DNS recovered.

Features

  • [deckhouse-controller] harden migrated modules check
  • [istio] Custom annotations for Ingress Controller
  • [istio] Custom extension providers
  • [istio] JWKS extra root CA implementation

Fixes

  • [ci] fix unlimited upmeter iterations
  • [cloud-provider-vcd] add werf deploy-dependency annotations to capcd webhook configurations to fix DMT lint and prevent race on install/upgrade
  • [deckhouse] Fix serialize docs builder builds.
  • [dhctl] Preflight checks are no longer re-run on every dhctl-server restart when the cluster config is unchanged.
  • [docs] Fix broken-link. Module admission
  • [docs] Change security events docs
  • [ingress-nginx] The config hashing is fixed.
    All ingress-nginx controllers pod will be restarted.
  • [kube-dns] Added deploy dependencies for webhook configuration
  • [node-manager] Creating or re-applying an already-existing StaticInstance no longer fails address validation.
  • [user-authz] user-authz-webhook now uses the node-local kube-apiserver endpoint for its discovery cache and liveness check, instead of resolving the “kubernetes.default” DNS name.
    Previously, a transient cluster DNS failure could cause the user-authz-webhook liveness probe to fail and restart the pod, which combined with the fail-closed authorization webhook (failurePolicy: Deny) could deny all API requests, including cluster-admins, until DNS recovered.

1.76.2

Features

  • [docs] Add registry module architecture section.
  • [istio] Add deploy dependencies for Istio webhook configurations

Fixes

  • [deckhouse] fix libs in the python-based images
  • [dhctl] fix grpc stream cancel deadlock
  • [docs] Added the ml/ai tag to the ai-models module on the modules homepage.
  • [registry] Missing copy_custom_certificate for Gateway API path was added.
  • [registry-packages-proxy] Missing Gateway API objects were added.
  • [vertical-pod-autoscaler] Ensure VPA mutating webhook is deployed after its admission controller Deployment and Service are ready.

1.76.1

Know before update

  • After upgrading to v1.76.0, kubectl logs and exec fail cluster-wide for all users. Manual workaround is available — see PR description.
  • All ingress-nginx controller pods using default controller version will be restarted.

Features

  • [deckhouse] Ensure conversion hooks.
  • [docs] Add auxiliary components (of virtualization module) architecture page.
  • [docs] Update csi-scsi-generic module architecture section
  • [docs] Add deep-link anchors and auto-expand logic for collapsible sections.
  • [docs] Add page about pod security settings

Fixes

  • [candi] Fix containerd credential escaping by rendering username/password into auth string.
  • [ci] remove CSE in deploy
  • [cloud-provider-dvp] CVE fixes
  • [control-plane-manager] fix Helm adoption of kubeadm:cluster-admins CRB on upgrade from pre-v1.76
    After upgrading to v1.76.0, kubectl logs and exec fail cluster-wide for all users. Manual workaround is available — see PR description.
  • [deckhouse] Fix stale application conditions during update over a failed version
  • [deckhouse-controller] Add alias for legacy bash completion flag.
  • [deckhouse-controller] add werf dependency to webhook
  • [docs] Deleted the note about the beta version of the graphical installer on the DKP installation page.
  • [docs] Update the wording in the FAQ section
  • [ingress-nginx] Default ingress controller version is reverted to 1.10.
    All ingress-nginx controller pods using default controller version will be restarted.
  • [user-authn] Fix blank dex error page caused by missing BuildIdentifier in the error template data struct.

Version 1.75

Important

  • Support for Kubernetes 1.35 has been added, while support for Kubernetes 1.30 has been discontinued. In future DKP releases, support for Kubernetes 1.31 will be removed. The default Kubernetes version (used when the kubernetesVersion parameter is set to Automatic) has been changed to 1.33.

  • Update to DKP 1.75 will be blocked if the cluster uses one of the following deprecated component versions (they are no longer supported):
    • IngressNginxController v1.9
    • Istio v1.19
  • In DKP 1.75.0–1.75.3, the PodSecurityStandardsViolation alert may fire immediately after cluster installation. This happens when the d8-system namespace is missing the heritage=deckhouse label, causing Pod Security Standards policies to be mistakenly applied to DKP system pods. The issue is fixed in DKP 1.75.4.

    For clusters installed on DKP 1.75.0–1.75.3, add the required label manually using the following command:

    d8 k --as=system:sudouser label namespace d8-system heritage=deckhouse
    
  • Istio version support changes:
    • Support for Istio 1.19 has been discontinued.
    • Istio 1.21 is now considered deprecated, and support for this version will be discontinued in the upcoming DKP releases. To update Istio, follow the instruction.
  • Starting with Kubernetes 1.35, cluster nodes must use cgroup v2. The previous version (cgroup v1) is considered deprecated. If cgroup v2 is not supported on a node, the D8NodeCgroupV2NotSupported alert will be triggered. See the Kubernetes documentation for cgroup v2 migration details.

  • Starting with Kubernetes 1.36, cluster nodes must use containerd v2 as the container runtime. Support for containerd v1 will be discontinued. If containerd v2 is not supported on a node, the D8NodeContainerdV2NotSupported alert will be triggered.

  • The vertical-pod-autoscaler module has been updated from version 1.4.1 to 1.5.1. Starting from version 1.5.0, the updateMode: Auto mode is considered deprecated. Users are advised to replace it with updateMode: InPlaceOrRecreate. See the vertical scaling documentation for details.

  • When issuing new certificates using the cert-manager module, the certificate’s private key is now re-generated as well. This change mitigates potential security risks.

  • Pods handling HTTP-01 challenges via Issuer and ClusterIssuer resources now have default resource requests defined. As a result, these pods consume part of the cluster and project resource quotas upon creation.

  • The operator-trivy module is now delivered from an external source and follows an independent release cycle. No manual migration steps are required.

Deckhouse subsystem

  • The ModuleSource resource now includes the scanInterval parameter to control how frequently the container registry is checked for new modules and versions.

  • A dedicated node mode with the node.deckhouse.io/etcd-arbiter label has been introduced. In this mode, only etcd runs on the node (no control plane components). This enables a highly available cluster with two master nodes plus a dedicated etcd node, reducing resource requirements compared to the traditional three full master nodes configuration. See the High Availability documentation for details.

  • New operating modes have been introduced in the registry module: Proxy (with image caching) and Local (for isolated environments). These modes are currently supported only for already deployed static clusters and are not yet available during DKP cluster installation.

  • Container registry access parameters can now be configured via the deckhouse module during DKP cluster installation. Support for the DKP cluster deployment via Direct and Unmanaged modes has been added. The Legacy Unmanaged mode (via InitConfiguration) is preserved for compatibility. If no explicit configuration is provided, the mode is selected automatically.

  • Added user shell validation: if a shell other than bash is used, the dhctl utility now displays a clear error message.

  • Added StaticInstance validation before DKP installation using the corresponding SSHCredentials, preventing dhctl from reporting readiness before the cluster deployment is actually complete.

  • Extended preflight test coverage to improve cluster deployment predictability.

  • Partially reduced usage of netcat in bashible, lowering the number of security system triggers.

  • Fixed an issue where cluster installation could not be resumed after interruption.

  • Fixed Node object deletion logic during converge. The node is now properly removed from the cluster after the corresponding VM is destroyed.

  • Fixed the --skip-resources flag behavior in dhctl destroy command. Specified resources are now correctly skipped.

Kubernetes & Scheduling subsystem

  • A new update-observer component in the control-plane-manager module enables real-time tracking of control plane component updates (kube-apiserver, kube-scheduler, kube-controller-manager) and kubelet on nodes. This helps detect discrepancies and eliminates “blind spots” during Kubernetes updates.

  • For all VPA resources managed by DKP, updateMode: Auto has been automatically replaced with updateMode: InPlaceOrRecreate. For clusters running Kubernetes 1.32 or earlier, the InPlacePodVerticalScaling feature gate is automatically enabled in control plane components (kube-apiserver, kube-scheduler, kube-controller-manager) and kubelet. This allows pod resource changes without recreation.

  • The descheduler module has been updated to version 0.34. Key changes:

    • Priority-based pod eviction: pods with a higher restart count are evicted first, improving cluster stability during load redistribution.
    • DRA support: proper handling of resources managed via Dynamic Resource Allocation.
    • Default cpu, memory, and pods thresholds removed from lowNodeUtilization and highNodeUtilization strategies to align with upstream behavior.

IAM subsystem

  • Namespace listing behavior has been changed. When requesting all cluster namespaces, users now see only the namespaces they have access to, even without permission to list all namespaces. This improves working with multitenancy and UI.

  • The ProjectTemplate resource in the multitenancy-manager module now supports the unmanaged and skip-heritage options (set via annotations), allowing administrators to adjust the DKP control degree over resources:
    • unmanaged: The module does not manage the lifecycle of objects.
    • skip-heritage: DKP retains management but allows modification by external controllers.
  • Added support for authenticating to the Kubernetes API via LDAP using Basic authentication instead of OIDC tokens (via the ldap.enableBasicAuth parameter in the DexProvider resource).

  • The authorization webhook in the user-authz module now operates in fail-closed mode: if it’s unavailable, authorization is denied. For improved reliability, webhook communication with the apiserver is performed via the cluster node network.

  • The user-authn module now supports a global user session lifetime limit (via refreshTokenAbsoluteLifetime). Re-authentication is required after expiration.

  • The user-authn module now supports Kerberos (SPNEGO) authentication in DKP.

  • Updated kube-apiserver TLS cipher suites. Removed the following insecure ciphers from the --tls-cipher-suites flag: TLS_RSA_WITH_AES_256_GCM_SHA384 and TLS_RSA_WITH_AES_128_GCM_SHA256 (considered unsafe due to RSA key exchange). Added TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 and TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256.

  • The anonymous access restriction mechanism for the API server has been modified. Instead of using the --anonymous-auth=false flag and a sidecar proxy, the AuthenticationConfiguration resource is now used to allow anonymous access only to /livez, /readyz, and /healthz. This improves cluster resource efficiency. Anonymous probe access remains restricted when public API access is enabled via publishAPI.enabled.

Security subsystem

  • Fixed serialization of empty arrays and objects in SecurityPolicy and OperationPolicy resources of the admission-policy-engine module. Previously, some settings could be applied incorrectly.

  • Revised ModSecurity extension implementation. OWASP Core Rule Set (CRS) functionality is now available in all supported IngressNginxController versions.

Cluster & Infrastructure subsystem

  • Lifecycle stages updated for cloud provider integration modules:
  • Added support for swap memory usage by pods. Instead of immediate OOM-kill, a pod can temporarily use swap (if configured and available on the node) and continue running after peak usage. Requires cgroup v2 and sufficient node resources. For details, see the kubelet.memorySwap parameter in the NodeGroup resource documentation.

  • In the DVP cloud provider:
    • Added the networkPolicy parameter to define network traffic rules for workloads within a Project resource.
    • Added managed-by, cluster-uuid, and vm_name labels to all infrastructure objects.
    • Improved CSI driver diagnostics. Diagnostic messages are now displayed directly in PersistentVolumeClaim status.
    • Implemented automatic cleanup of VMBDA objects when a VM is deleted.
    • Fixed an issue where DeckhouseMachine objects could remain in deleting state after VM removal.
    • Fixed an issue where Cloud Controller Manager (CCM) could recreate external load balancers after the associated Kubernetes Service was deleted.
  • In Yandex Cloud, fixed a false NAT warning displayed during installation in configurations where NAT is not used.

  • In the node-manager module:
    • Cluster API (CAPI) has been updated from 1.10.6 to 1.11.3, improving stability and provider compatibility.
    • Added a dedicated controller for NodeGroup hooks (extracted from DKP), reducing load on master nodes and improving responsiveness and UI feedback.
    • Added per-GPU MIG configuration support. You can now define a distinct set of MIG partitions for each physical GPU on a node. Previously, MIG configuration was applied at the node level and identical across all GPUs. This enables more flexible workload resource allocation on a single node.

Network subsystem

  • In IngressNginxController, the acceptClientIPHeadersFrom parameter has been added for the following inlets: HostPort, HostPortWithProxyProtocol, LoadBalancer, and LoadBalancerWithProxyProtocol. The parameter defines trusted CIDR ranges for X-Forwarded-* headers that should not be overwritten during proxying.

  • In the cni-cilium module, network observability capabilities have been extended:
    • Added extended metrics for monitoring interactions between specific objects (Pods, Services, Namespaces), configurable via the extendedMetrics parameter of the HubbleMonitoringConfig resource.
    • Implemented export of traffic flow logs for analyzing network interactions, configurable via the flowLogs parameter of the HubbleMonitoringConfig resource.
  • Fixed an issue where, after VM migration in DVP, a LoadBalancer IP address could stop being announced when using MetalLB in BGP mode.

  • IngressNginxController version support changes:
    • Support for v1.9 has been discontinued.
    • Support for v1.14 has been added.

The complete list of changes, including the updated components, is available in the changelog on GitHub.

1.75.16

Fixes

  • [istio] Add missing tools to proxyv2 images so the application-aware proxy termination hook works correctly.
  • [istio] Fix graceful draining of established HTTP connections when application pods terminate.

1.75.15

Know before update

  • The cilium-hubble components (hubble-ui, hubble-relay) will restart after the update.
  • The cni-cilium components (cilium agent, operator) will restart after the update.
  • The coredns and kube-dns components will restart after the update.
  • The egress-gateway-agent component will restart after the update.
  • The metallb components (controller, speaker, l2lb) will restart after the update.
  • The node-local-dns DaemonSet pods will restart after the update.
  • The node-local-dns components will restart after the update.
  • The service-with-healthchecks components (controller, agent) will restart after the update.

Fixes

  • [cilium-hubble] Upgrade hubble-ui backend dependencies (cilium v1.17.16, Go 1.25.0) and switch build base image to fix known CVEs.
    The cilium-hubble components (hubble-ui, hubble-relay) will restart after the update.
  • [cni-cilium] Bump Go dependencies in the egress-gateway-agent image to fix known CVEs.
    The egress-gateway-agent component will restart after the update.
  • [cni-cilium] Bump Go dependencies and backport upstream cilium security patches to fix known CVEs.
    The cni-cilium components (cilium agent, operator) will restart after the update.
  • [deckhouse] An unset settings.update.mode now defaults to AutoPatch instead of silently running as Auto.
  • [deckhouse] Fix a minor Deckhouse release being auto-applied as a patch when no Deployed release object is present.
  • [deckhouse] Surface layer read errors instead of extracting a truncated image
  • [kube-dns] Bump Go dependencies in the sts-pods-hosts-appender-webhook and coredns images to fix known CVEs.
    The coredns and kube-dns components will restart after the update.
  • [metallb] Bump Go dependencies in the metallb and l2lb images to fix known CVEs.
    The metallb components (controller, speaker, l2lb) will restart after the update.
  • [node-local-dns] Bump Go dependencies in the safe-updater and stale-dns-connections-cleaner images to fix known CVEs.
    The node-local-dns components will restart after the update.
  • [node-local-dns] Bump Go dependencies in the coredns helper image to fix known CVEs.
    The node-local-dns DaemonSet pods will restart after the update.
  • [service-with-healthchecks] Bump Go dependencies in the service-with-healthchecks image to fix known CVEs.
    The service-with-healthchecks components (controller, agent) will restart after the update.
  • [tools] Fixed module release validation incorrectly failing for !optional dependencies when the required module doesn’t exist in the cluster.

1.75.14

Fixes

  • [common] Fix curl in debug-container by shipping a static binary
  • [deckhouse] atomically install modules and re-download incomplete versions
  • [deckhouse] show the allow-disabling annotation hint when disabling a confirmation-required module
  • [deckhouse] Fix CVEs.
  • [deckhouse-controller] Fix maintenance state tracking — module no longer gets stuck in Unmanaged after removing maintenance.
  • [docs] Corrected the link from direct to relative in the alert about the module requirements.

1.75.13

Fixes

  • [deckhouse] Fix nil pointer panic in resource and namespace informers by using each informer’s own logger instead of the possibly-nil Monitor.Logger.
  • [user-authn] Preserve Dex Password fields (notably groups) when resetting password, locking or unlocking a user, so users are no longer locked out after these operations.

1.75.12

Know before update

  • Previously, a transient cluster DNS failure could cause the user-authz-webhook liveness probe to fail and restart the pod, which combined with the fail-closed authorization webhook (failurePolicy: Deny) could deny all API requests, including cluster-admins, until DNS recovered.

Fixes

  • [candi] fix if node has bashible-uninitialized taint in race condition.
  • [deckhouse] add cryptography CVE VEX to deckhouse-controller
  • [deckhouse] Fix module metadata (module.yaml/oss.yaml) missing from in-cluster documentation by allowing these files in IsDocsPath (sender) and creating parent directories on extraction in the docs-builder (receiver).
  • [deckhouse-controller] Don’t create an external module release for a module that is still shipped embedded, so it can’t replace or duplicate the embedded copy.
  • [deckhouse-controller] Disabling modules with confirmation will be rejected without using the annotation.
  • [docs] Added the ml/ai tag to the ai-models module on the modules homepage.
  • [docs] Fix broken-link. Module admission
  • [documentation] Add /app/hugo to docs-builder mount points
  • [user-authz] user-authz-webhook now uses the node-local kube-apiserver endpoint for its discovery cache and liveness check, instead of resolving the “kubernetes.default” DNS name.
    Previously, a transient cluster DNS failure could cause the user-authz-webhook liveness probe to fail and restart the pod, which combined with the fail-closed authorization webhook (failurePolicy: Deny) could deny all API requests, including cluster-admins, until DNS recovered.

1.75.11

Know before update

  • All ingress-nginx pods will be restarted.

Fixes

  • [common] Fixed CVE-2026-40898 in CoreDNS by updating the quic-go dependency.
  • [ingress-nginx] Nginx is updated to 1.30.3.
    All ingress-nginx pods will be restarted.

1.75.10

Features

  • [docs] Add csi-ceph module architecture section.
  • [docs] Add Core component (of virtualization module) architecture section.
  • [documentation] Update editions availability info for stronghold module.

Fixes

  • [candi] fix cve node-manager and opentofu.
  • [cilium-hubble] Fixed CVE-2026-29181 in hubble-ui-backend by bumping OpenTelemetry Go to v1.41.0
  • [cilium-hubble] Fixed CVE-2026-41520 in hubble-ui-backend
  • [cni-cilium] Fixed CVE-2026-41520 for cilium-bugtool util
  • [cni-flannel] Reverted module stage from Deprecated back to General Availability to stop false deprecation alerts.
  • [common] fixed CVE-2026-29181 in the CoreDNS
  • [deckhouse-controller] A module that conditionally depends on another is no longer disabled when an incompatible version of that dependency is enabled; the enable is rejected instead.
  • [deckhouse-controller] fix module config validation for disable warnings
  • [docs] Add a general description of container registers DKP
  • [docs] Updated the etcd restore documentation.
  • [docs] Fix CRD version tabs markup and nesting.
  • [istio] Implement graceful metadata secret renewal for multiclusters.
  • [network-policy-engine] Reverted module stage from Deprecated back to General Availability to stop false deprecation alerts.
  • [node-manager] hook to restore apiVersion on CAPI resources.
  • [node-manager] Fix static NodeGroup readiness calculation.
  • [node-manager] add rbac policies for persistantvolumes to manage from capi-controller-manager.
  • [tools] fix d8 build

1.75.9

Know before update

  • All Ingress-nginx controller pods will be restarted.

Fixes

  • [admission-policy-engine] Refactor main documentation page
  • [ingress-nginx] Nginx was updated to 1.30.2.
    All Ingress-nginx controller pods will be restarted.

1.75.8

Know before update

  • Registry pods will be restarted.

Features

  • [docs] Update deckhouse web modules architecture section.

Fixes

  • [csi-vsphere] Fixed the Deckhouse queue getting stuck
  • [docs] Change network-policy-engine and cni-flannel to deprecated.
  • [monitoring-deckhouse] Fix module-release alerts (ModuleReleaseIsWaitingManualApproval, ModuleReleaseIsOutdated, ModuleReleaseIsBlockedByRequirements, ModuleIsInMaintenanceMode, D8ModuleOutdatedByMajorVersion) that never fired on 1.75.x clusters due to a moduleName→module label mismatch introduced when addon-operator was bumped to v1.19.7 in v1.75.1.
  • [registry] Updated auth image Go dependencies to fix Go CVEs.
    Registry pods will be restarted.
  • [user-authn] Improve basic-auth-proxy request handling, cache implementation, and shutdown behavior.

1.75.7

Know before update

  • All Ingress-nginx controller pods will be restarted.

Features

  • [docs] Add Deckhouse web modules architecture section.
  • [docs] Add cert-manager module architecture section.

Fixes

  • [cloud-provider-vsphere] normalizes new paths and makes bashible resolve existing paths case-insensitively
  • [cni-cilium] Fixed CVE-2026-35469
  • [deckhouse] Fix exp modules auto enabling.
  • [deckhouse] Fixed global configuration generation.
  • [dhctl] Wait for stronghold cluster sync before node deletion
  • [docs] Refactored web UI page.
  • [docs] New screenshots have been added to the Console documentation.
  • [ingress-nginx] Nginx is updated up to 1.30.1.
    All Ingress-nginx controller pods will be restarted.

1.75.6

Fixes

  • [admission-policy-engine] Fixed enforcementAction for D8ReplicaLimits, added more template tests.
  • [dhctl] fix SSH preflight check for StaticInstances with password-only auth.
  • [docs] Fix broken links in VPA section.
  • [docs] Fix visibility scope for nested details component
  • [docs] Add info about kernel requirement for containerdv2 migration.
  • [node-manager] CAPI crd served version fix
  • [node-manager] Added cleanup for oversized MCM MachineSet revision history annotation

1.75.5

Know before update

  • module pods will be restarted

Features

  • [docs] Add a Jekyll plugin for tabbed content in the documentation.
  • [istio] Allow custom ports in metadataEndpoint URLs for IstioFederation and IstioMulticluster CRDs.

Fixes

  • [candi] Added a Netplan override to force the secondary NIC to use the main routing table, fixing cloud-init PBR conflicts.
  • [common] Fixed an issue where Trivy could not detect VEX files in coredns images.
  • [deckhouse] Overwrite currentReleaseImageName on mismatch.
  • [deckhouse] Allow updating scanInterval on the deckhouse ModuleSource.
  • [istio] fixed CVEs in module images
    module pods will be restarted
  • [service-with-healthchecks] Correct RBACv2 is provided.
  • [terraform-manager] Fix opentofu patches build after pull request 19080.

1.75.4

Know before update

  • All Ingress-NGINX controller pods will be restated.
  • All pods of Ingress-NGINX controller of 1.10 and 1.12 versions will be restarted.
  • Cilium agents will be restarted during the update.
  • HPA scaler pods will be restarted.
  • Kruise controller will be restarted.
  • These changes prevent unnecessary or destructive plan updates that could occur when data sources depend on changing labels and annotations. The behavior of other cloud providers is not affected. If you encounter unexpected converge plans or cluster bootstrap issues when using OpenTofu-based providers (such as dvp, dynamics, zvirt, or yandex), please report them to Deckhouse Technical Support.
  • This update triggers a rolling update of the flannel pods.
  • This update triggers a rolling update of the kube-proxy pods.
  • This update triggers a rolling update of the network-policy-engine pods.
  • istio module pods will be restarted
  • istio-cni-nodes will be restarted

Features

  • [docs] Respect nowrap default for code blocks.
  • [docs] Update sds-node-configuration module architecture section.
  • [docs] Add geoproxy and failover to ingress-nginx module architecture section
  • [docs] Add csi-nfs module architecture section.
  • [docs] Add public cloud providers aws, azure, and gcp module architecture section.
  • [docs] Add cloud-provider-vsphere module architecture section.
  • [docs] Add cloud-provider-openstack module architecture section.
  • [terraform-manager] Suppress destructive changes when add/remove/change labels and annotations for cloud resources via opentofu.
    These changes prevent unnecessary or destructive plan updates that could occur when data sources depend on changing labels and annotations. The behavior of other cloud providers is not affected. If you encounter unexpected converge plans or cluster bootstrap issues when using OpenTofu-based providers (such as dvp, dynamics, zvirt, or yandex), please report them to Deckhouse Technical Support.

Fixes

  • [admission-policy-engine] Fix high resource consumption for constraint d8denyexecheritage
  • [candi] Fix containerd unmount for migration if /var/lib/containerd is mounted manually.
  • [cert-manager] Disable SecurityPolicyExceptions for cert-manager namespace
  • [cloud-provider-aws] add information about AWS security group rules limits
  • [cloud-provider-dvp] Suppress destructive changes when add/remove/change labels and annotations for cloud resources via opentofu.
  • [cloud-provider-vcd] fix vCD CCM TCP health monitors removal
  • [cni-cilium] Updated go-jose dependency to v4.1.4 to fix CVE-2026-34986.
    Cilium agents will be restarted during the update.
  • [cni-flannel] Fixed CVE-2026-33186 by updating google.golang.org/grpc in flanneld.
    This update triggers a rolling update of the flannel pods.
  • [cni-simple-bridge] Refactored python image source and pip exclusion.
  • [common] Removed Python completely from the debug-container image as it is no longer needed, resolving corresponding CVEs, and silenced false positives for etcd binaries via VEX.
  • [deckhouse] Return setting notified for mr.
  • [deckhouse] Remove notified=false annotation reset from runReleaseDeploy in the module release controller.
  • [deckhouse] Ensure heritage label on d8-system namespace via hook.
  • [deckhouse-controller] Add new discoveryv1 scheme to deckhouse controller.
  • [deckhouse-controller] added extra validation for kubernets version multiple downgrades scenario
  • [dhctl] Use internal node ip if bastion ip was passed in converge.
  • [docs] Fix indented Liquid blocks in lists.
  • [docs] Fix broken links.
  • [docs] Improved title generation for module documentation pages.
  • [ingress-nginx] CVE-2025-15566 is fixed in 1.10 and 1.12 controllers.
    All pods of Ingress-NGINX controller of 1.10 and 1.12 versions will be restarted.
  • [ingress-nginx] Added missing pod security context settings to hpa scaler pods.
    HPA scaler pods will be restarted.
  • [ingress-nginx] Kruise go dependencies are updated to mitigate latest CVEs.
    Kruise controller will be restarted.
  • [ingress-nginx] Added fix for CVE-2026-4342.
    All Ingress-NGINX controller pods will be restated.
  • [istio] fixed CVE-2026-34986
    istio module pods will be restarted
  • [istio] added iptables wrapper in cni-v1x21x6
    istio-cni-nodes will be restarted
  • [keepalived] Excluded vulnerable pip-25.3 from keepalived final image to fix CVE-2026-1703
  • [kube-dns] Refactored hook remove_old_coredns_deploy_and_svc — delegated deletion of original kube-dns service to helm.
  • [kube-proxy] Fixed CVE-2026-33186 and CVE-2026-24051 in kube-proxy dependencies.
    This update triggers a rolling update of the kube-proxy pods.
  • [network-gateway] Updated python image source and mitigated pip CVE-2026-1703
  • [network-policy-engine] Fixed CVE-2026-34040, CVE-2026-33997, and CVE-2026-33186 in network-policy-engine dependencies.
    This update triggers a rolling update of the network-policy-engine pods.
  • [node-manager] rollback inject capi conversion hook in crds as on-before-helm
  • [node-manager] optimize node-controller cache resources usage.
  • [registrypackages] Add vex for CVE-2026-39882
  • [registrypackages] Remediated CVE-2026-32285 in crictl dependencies.

1.75.3

Know before update

  • CoreDNS pods will undergo a rolling restart.

Features

  • [docs] Add cloud-provider-yandex module architecture section.

Fixes

  • [common] Fixed CVE-2026-33186 in the CoreDNS image.
    CoreDNS pods will undergo a rolling restart.
  • [deckhouse-controller] revert nelm to helm
  • [dhctl] Skip tmp lock for exporter and auto-converger.
  • [node-local-dns] Fix werf manifest
  • [node-local-dns] Adapt node-local-dns for air-gapped environments.
  • [node-manager] deploy capi controller and webhooks before basic resources to prevent race condition during upgrades.
  • [registrypackages] Added vex with CVE-2026-33186.

1.75.2

Know before update

  • All Ingress-NGINX controller pods will be restarted.

Features

  • [cert-manager] Add alerts for ACME Challenges stuck in pending or error states
  • [docs] Add cloud providers dynamix, huaweicloud, and zvirt module architecture section.

Fixes

  • [ci] Use GITHUB_TOKEN for backport action and set required workflow permissions
  • [cilium-hubble] Fixed CVE-2026-33186 in the hubble-ui image.
  • [cni-cilium] Fixed CVE-2026-33186, CVE-2026-27142, and CVE-2026-27139 by updating grpc dependency and Go version, and resolved build compatibility issues.
  • [common] Fixed CVE-2026-24051 in the CoreDNS image.
  • [dhctl] mitigate CVE-2026-33186
  • [ingress-nginx] Nelm fixes are backported.
    All Ingress-NGINX controller pods will be restarted.

1.75.1

Know before update

  • Registry pods will be restarted.
  • ingressgateway, istio-cni-node, istiod, kiali, operator pods will be restarted after update

Features

  • [cloud-provider-zvirt] add customNetworkConfig
  • [cloud-provider-zvirt] add customNetworkConfig
  • [loki] Added generation and usage of a dedicated TLS server certificate for Loki kube-rbac-proxy

Fixes

  • [admission-policy-engine] Fix SecurityPolicyException handling for hostPorts-only exceptions
  • [cloud-provider-aws] fix CVE in cloud-provider-aws
  • [cloud-provider-azure] fix CVEs in cloud-provider-azure
  • [cloud-provider-dvp] fix CVEs in cloud-provider-dvp
  • [cloud-provider-dvp] fix CVEs in cloud-provider-dvp
  • [cloud-provider-dvp] refactored CreateVolume to improve idempotency when disk.status.capacity is not yet reported and standardized gRPC error handling
  • [cloud-provider-huaweicloud] fix CVEs in cloud-provider-huaweicloud
  • [cloud-provider-huaweicloud] fix CVEs in cloud-provider-huaweicloud
  • [cloud-provider-openstack] fix CVE in cloud-provider-openstack module
  • [cloud-provider-yandex] fix CVEs in cloud-provider-yandex
  • [cloud-provider-zvirt] fix CVEs in cloud-provider-zvirt
  • [cloud-provider-zvirt] docs fox
  • [cni-cilium] fixed mount-points in agent
  • [deckhouse] Bump nelm version with deadlock fix.
  • [dhctl] Fix restart bootstrap during creating additional nodes in cloud permanent node groups.
  • [dhctl] Аix non-strict unmarshalling for metaconfigs.
  • [dhctl] Fix deadlock in converge.
  • [docs] Added the display of all used component versions in modules on the OSS info page.
  • [docs] Add support for highlighting unknown code block types.
  • [docs] Fix alerts rendering.
  • [docs] Moving the platform update section to the top in the sidebar.
  • [docs] Changing the term metka to label in Russian-language documentation.
  • [docs] Escape HTML in OpenAPI descriptions v2
  • [docs] Added info about using update-observer according to
  • [extended-monitoring] fix typo in image-availability-exporter template
  • [istio] cve fixes
    ingressgateway, istio-cni-node, istiod, kiali, operator pods will be restarted after update
  • [registry] fixed path in images
  • [registry] Updated auth image Go dependencies to fix Go CVEs.
    Registry pods will be restarted.
  • [registry-packages-proxy] fix possible deadlock in cache retention policy
  • [tools] Added HTTP client proxy support
  • [tools] Bump lib-helm version to 1.71.2

Version 1.74

Important

  • Support for Kubernetes 1.34 has been added, while support for Kubernetes 1.29 has been discontinued. In future DKP releases, support for Kubernetes 1.30 will be removed. The default Kubernetes version (used when the kubernetesVersion parameter is set to Automatic) has been changed to 1.32.

Major changes

  • Added support for managing experimental and alpha Kubernetes features (feature gates) using the enabledFeatureGates parameter of the control-plane-manager module. For details on managing these features in DKP, refer to the module documentation.

  • The --terminated-pod-gc-threshold parameter (the termination threshold for evicted pods) is now calculated automatically based on the number of nodes in the cluster. This provides a balance between cluster stability and the ability to perform diagnostics. Threshold values are available in the control-plane-manager module documentation.

  • The maximum number of pods (maxPods) for a node group is now set automatically based on the podSubnetNodeCIDRPrefix parameter from ClusterConfiguration (unless explicitly specified). This ensures stable cluster operation.

  • Added the vipAddress parameter to the HuaweiCloudInstanceClass resource of the Huawei Cloud provider, allowing you to specify a virtual IP address for all nodes in the instance class.

  • The Huawei Cloud provider configuration now allows overriding the primary network for the main network interface (mainNetwork) and specifying subnets for additional network interfaces (additionalNetworks) on both CloudPermanent and CloudEphemeral nodes.

    Warning. When upgrading to DKP 1.74 on Huawei Cloud, a bulk recreation of CloudEphemeral nodes may occur. Due to a temporary node shortage, some Pods may fail to start.

  • The archive with debugging data is now collected using the d8 tool. If you need to collect diagnostic data, follow the instructions.

  • Now when hook execution fails, all metrics collected by that hook are preserved. This prevents loss of debugging information and simplifies the following root-cause analysis.

  • Fixed a Deckhouse queue freeze in hybrid clusters that occurred when module configuration parameters were incomplete.

Security

  • Implemented module integrity control based on the EROFS filesystem to improve security and prevent unauthorized modifications. The new system requires kernel support for dm-verity and EROFS.

Network

  • Added support for the SCTP (Stream Control Transmission Protocol). Applications using this protocol can now run in a Deckhouse cluster.

  • Added a geoproxy auxiliary microservice to the ingress-nginx module. This service is aimed at improving the stability of the Ingress NGINX Controller when working with GeoIP databases and provides the following features:
    • MaxMind license saving (databases are downloaded from a single point once a day).
    • Persistent data storage (if components are restarted, it doesn’t require accessing the MaxMind servers).
    • Lets you specify a custom mirror for downloading databases.
  • Added the Prometheus metric bpf_progs_complexity_max_verified_insts to assess the number of instructions in eBPF programs loaded into the kernel on cluster nodes. This metric helps evaluate the compatibility of the networking subsystem with the node kernels. The metric requires Linux kernel version 5.16 or newer.

  • Added a new metric (geoip_version) and dashboards for monitoring GeoIP functionality in the cluster.

  • Fixed issues with simultaneous updates of the node-local-dns module and the networking subsystem.

The complete list of changes, including the updated components, is available in the changelog on GitHub.

1.74.18

Fixes

  • [deckhouse-controller] Don’t create an external module release for a module that is still shipped embedded, so it can’t replace or duplicate the embedded copy.
  • [deckhouse-controller] A module that conditionally depends on another is no longer disabled when an incompatible version of that dependency is enabled; the enable is rejected instead.
  • [istio] Implement graceful metadata secret renewal for multiclusters.
  • [network-gateway] Updated dnsmasq to v2.92-alt2 to address multiple security vulnerabilities (CVE-2026-*)
  • [node-local-dns] Fix name of registry secret in safe-updater deployment

1.74.17

Fixes

  • [node-manager] Added cleanup for oversized MCM MachineSet revision history annotation

1.74.16

Know before update

  • All Ingress-NGINX controller pods will be restarted.
  • All pods of Ingress-NGINX controller will be restarted.
  • This update triggers a rolling update of the kube-proxy pods.
  • istio module pods will be restarted
  • module pods will be restarted

Features

  • [docs] Image zooming and mobile view.
  • [docs] Add sds-node-configurator module architecture section.
  • [docs] Add cloud-provider-vcd module architecture section.

Fixes

  • [admission-policy-engine] cve fixes for ratify
  • [cloud-provider-azure] fix disk discovery for Ubuntu 22.04 Gen2 VMs with NVMe controllers
  • [cloud-provider-openstack] Increase interval and timeout for health monitor
  • [cloud-provider-vsphere] Fix vSphere privilege matrix and describe instructions for setting up environment via vSphere Client
  • [cni-simple-bridge] Updated base image reference to support new Werf import format.
  • [common] Fixed an issue where Trivy could not detect VEX files in coredns images.
  • [common] Removed Python completely from the debug-container image as it is no longer needed, resolving corresponding CVEs, and silenced false positives for etcd binaries via VEX.
  • [deckhouse] Remove notified=false annotation reset from runReleaseDeploy in the module release controller.
  • [deckhouse] Fix webhook-handler CVEs.
  • [dhctl] Added validation of the command execution status code
  • [docs] Fix vSphere privilege matrix and describe instructions for setting up environment via vSphere Client
  • [documentation] Fix search.
  • [ingress-nginx] CVE-2025-15566 is backported.
    All pods of Ingress-NGINX controller will be restarted.
  • [ingress-nginx] CVE-2026-4342 fix is backported to Dechkouse 1.74.
    All Ingress-NGINX controller pods will be restarted.
  • [istio] fixed CVEs in module images
    module pods will be restarted
  • [istio] fixed CVE-2026-39882, CVE-2026-39883 and CVE-2026-35206
    istio module pods will be restarted
  • [istio] fixed CVE-2026-34986
    istio module pods will be restarted
  • [kube-proxy] Fixed CVE-2026-33186 and CVE-2026-24051 in kube-proxy dependencies.
    This update triggers a rolling update of the kube-proxy pods.
  • [network-gateway] Fixed werf import syntax for compatibility with older werf versions.

1.74.15

Know before update

  • An additional service daemonset will be added.
  • CoreDNS pods will undergo a rolling restart.
  • pods in d8-istio namespace will be restarted

Features

  • [docs] Add snapshot-controller module architecture section.

Fixes

  • [ci] Use GITHUB_TOKEN for backport action and set required workflow permissions
  • [cilium-hubble] Fixed CVE-2026-33186 in the hubble-ui image.
  • [cni-cilium] Fixed CVE-2026-33186, CVE-2026-27142, and CVE-2026-27139 by updating grpc dependency and Go version, and resolved build compatibility issues.
  • [common] Fixed CVE-2026-33186 in the CoreDNS image.
    CoreDNS pods will undergo a rolling restart.
  • [dhctl] mitigate CVE-2026-33186
  • [istio] fixed cve in module images
    pods in d8-istio namespace will be restarted
  • [node-local-dns] Return stale-dns-connections-cleaner
    An additional service daemonset will be added.
  • [registrypackages] Added vex with CVE-2026-33186.

1.74.14

Features

  • [docs] Add cloud-provider-dvp module architecture section.
  • [docs] Add local-path-provisioner module architecture section.
  • [docs] Update architecture section.

Fixes

  • [cloud-provider-zvirt] fix CSI token refresh patch apply
  • [docs] Added the display of the d8 utility version on the command reference page.
  • [istio] Reduce CPU and RAM for regenerate multicluster JWT token and sort ingressGateway
  • [tools] Added HTTP client proxy support

1.74.13

Fixes

  • [cloud-provider-openstack] fix LB.enabled flag

1.74.12

Know before update

  • All Ingress-Nginx controller pods will be restarted.

Fixes

  • [candi] Added a Netplan override to force the secondary NIC to use the main routing table, fixing cloud-init PBR conflicts.
  • [ingress-nginx] CVE-2026-3288 fix is backported in all Ingress-Nginx controllers.
    All Ingress-Nginx controller pods will be restarted.
  • [node-manager] Fixed vulnerabilities.

1.74.11

Know before update

  • Kruise controller pod will be restarted.
  • Registry pods will be restarted.

Fixes

  • [admission-policy-engine] Fix template tests
  • [ci] Add zVirt testing to e2e tests to DKP 1.74.
  • [ci] Backport the provider instance change and the new containerd version in OpenStack tests to release 1.74.
  • [cloud-provider-aws] fix getInstancesByIDs to comply with the describeInstanceBatcher.
  • [cloud-provider-openstack] Add loadBalancer.enabled flag to prevent CCM crashes on k8s 1.32 without Octavia service
  • [docs] Correct the wording in the cloud-provider-vsphere module docs.
  • [docs] Updated editions for the modules managed-postgres, managed-valkey and managed-memcached.
  • [ingress-nginx] Golang versions are updated.
    Kruise controller pod will be restarted.
  • [registry] Updated auth image Go dependencies to fix Go CVEs.
    Registry pods will be restarted.

1.74.10

Know before update

  • All ingress-nginx controller pods will be restared.

Fixes

  • [ingress-nginx] Nginx and module’s dependencies are updated.
    All ingress-nginx controller pods will be restared.

1.74.9

Know before update

  • Workload Pods are no longer denied by unrelated SecurityPolicy checks (e.g. hostNetwork/hostPort) when corresponding policy fields are not explicitly set.

Fixes

  • [admission-policy-engine] Prevent unintended Gatekeeper constraints from being rendered for SecurityPolicy when boolean fields are omitted.
    Workload Pods are no longer denied by unrelated SecurityPolicy checks (e.g. hostNetwork/hostPort) when corresponding policy fields are not explicitly set.
  • [candi] Bashible script fix to handle multiple GPUs
  • [candi] fix if node has bashible-uninitialized taint in race condition.
  • [ci] Add ignoring of D8NodeContainerdV2NotSupported and D8NodeCgroupV2NotSupported alerts to release 1.74.
  • [cloud-provider-dvp] added fixes an infinite deletion state of DeckhouseMachine.
  • [deckhouse] Fix yaml validation in settings.
  • [docs] Changing module descriptions
  • [docs] Editing FAQ problems applying dexprovider
  • [docs] Editing flannel to cillium.
  • [docs] Adding information about using images sign policy.
  • [istio] Fixing the list of requests from istiod to gateway API
  • [registrypackages] added usage GOPROXY in systemctl-power-commands-wrapper

1.74.8

Fixes

  • [docs] Escape HTML in OpenAPI descriptions
  • [extended-monitoring] Cleanup exporter metrics when the monitored resource has been deleted

1.74.7

Fixes

  • [admission-policy-engine] Convert constraints multi-line output to single-line
  • [candi] Added a Netplan override to force the secondary NIC to use the main routing table, fixing cloud-init PBR conflicts.
  • [dhctl] Fix panic in dhctl config render kubeadm-config command.
  • [istio] Correction of an useless error in the Istio CNI workflow

1.74.6

Know before update

  • All current connections powered by EgressGateways will be terminated.

Features

  • [docs] Added info from csi-ceph module about Ceph storage system integration in DKP documentation.

Fixes

  • [cni-cilium] Fix issue in generating CiliumEgressGatewayPolicy CR.
    All current connections powered by EgressGateways will be terminated.
  • [ingress-nginx] A false-positive trigger of alert GeoIPDownloadErrorDetectedFromMaxMind is fixed.
  • [node-manager] Fix panic in cluster-autoscaler caused by nil pointer dereference during node removal simulation.

1.74.5

Know before update

  • The ingress nginx controllers’ pods will be restated.

Features

  • [candi] Display lists of third-party (OSS) components in the modules documentation.
  • [cloud-provider-dvp] Add validation for VirtualMachineClass and boot images before VM creation

Fixes

  • [admission-policy-engine] Refactor constraint templates
  • [dhctl] Improved reliability when connecting to dhctl servers by adding retry logic and better error handling during startup
  • [docs] Corrections of typos and terms.
  • [docs] Small imrovements of control-plane management docs
  • [docs] Improve documentation for cluster network configuration
  • [ingress-nginx] The CVE-2026-1580, CVE-2026-24512, CVE-2026-24513, CVE-2026-24514 CVEs fixes are backported.
    The ingress nginx controllers’ pods will be restated.
  • [ingress-nginx] The CVE-2026-1580, CVE-2026-24512, CVE-2026-24513, CVE-2026-24514 CVEs are fixed.
    The ingress nginx controllers’ pods will be restated.
  • [monitoring-kubernetes] fix kube_persistentvolume_is_local recording rule when there are more than one kube-state-metrics exporter in cluster
  • [prometheus] Make Grafana redirect ingress pass the annotations validation.
  • [user-authz] Updated logo on the authentication page.

1.74.4

Know before update

  • If the SDN module is used in the cluster, the Cilium agent pods will be restarted.
  • Improved documentation clarity for Yandex Cloud NAT instance deployment scenarios
  • no impact

Features

  • [docs] Add the enableNote display to the external modules configuration page.
  • [prometheus] improve redirects from Grafana to the Deckhouse UI when Grafana is disabled
    no impact

Fixes

  • [cloud-provider-azure] fixed patch in azure
  • [cloud-provider-dvp] Fixed VM deletion timeout and improved memory validation error reporting
  • [cloud-provider-dvp] Cleanup orphaned resources when VM creation fails
  • [cloud-provider-huaweicloud] fix cve
  • [cloud-provider-vcd] fix the interaction of legacy and non-legacy components with the API
  • [cloud-provider-vsphere] fix cve
  • [cni-cilium] Fix hook discovery_cni_exclusive.go
    If the SDN module is used in the cluster, the Cilium agent pods will be restarted.
  • [docs] Fixed typo in the OpenStack integration section.
  • [docs] Added Docker Bearer Token Realm configuration steps for Nexus registry setup.
  • [docs] Document NAT instance deployment zones for different subnet configurations
    Improved documentation clarity for Yandex Cloud NAT instance deployment scenarios
  • [ingress-nginx] Fixed useless geoip alerts.
  • [monitoring-kubernetes] fix kube_persistentvolume_is_local recording rule for not-bound PVCs

1.74.3

Know before update

  • Node-local-dns safe-uptader pods will be restarted.

Features

  • [prometheus] Add prometheus configuration reload failed alert

Fixes

  • [admission-policy-engine] Fix constraint templates, add additional tests
  • [cloud-provider-aws] fix cve
  • [cloud-provider-vcd] fix cve
  • [cloud-provider-yandex] fix cve
  • [cni-cilium] Fixed egress-gateway-agent controller logic for deleted resources and disable dev logging.
  • [deckhouse] Fix module rerun.
  • [docs] Added a link to the ModuleSource to the explanation table of the editions.
  • [metallb] Fixed D8MetallbBGPSessionDown alert false positives caused by duplicate metrics without port information.
  • [node-local-dns] The safe-updater’s logic of detecting if daemonset pods are up-to-date is improved.
    Node-local-dns safe-uptader pods will be restarted.
  • [registry] Fixed validation of input image list changes in the registry checker.

1.74.2

Know before update

  • All pods running kube-rbac-proxy will be restarted.

Features

  • [prometheus] Replace PrometheusRules with ClusterObservabilityMetricsRulesGroups or ClusterObservabilityPropagatedMetricsRulesGroups when deployed using helm_lib_prometheus_rules helper and the observability module is enabled

Fixes

  • [admission-policy-engine] fix namespaceSelector for constraints, add CRDs
  • [candi] Remove duplicate additional_disks_hashes definition in static-node Terraform module.
  • [candi] Updated the bashible step to include Linux kernel versions that address CVE-2025-37999
  • [candi] Allow manually stopping DVP node VirtualMachines in nested clusters by using runPolicy AlwaysOnUnlessStoppedManually.
  • [cloud-provider-azure] fixed cve
  • [cloud-provider-dvp] this PR gives a lot more informational errors and messages to user
  • [cloud-provider-openstack] fix cve
  • [common] Latest CVEs are fixed.
    All pods running kube-rbac-proxy will be restarted.
  • [control-plane-manager] Fix validation for resource-quota-overrides labels
  • [deckhouse] Fix mpo cleanup.
  • [deckhouse] Fix module installer cleanup.
  • [deckhouse-controller] Exclude all service accounts from d8- namespaces in d8ms-prefix ValidatingAdmissionPolicy.
  • [ingress-nginx] Latest CVEs are fixed.
    All pods running kube-rbac-proxy will be restarted.
  • [kube-dns] Reverts
  • [log-shipper] Fixed incorrect apiVersion in ClusterLogDestination documentation examples.
  • [node-manager] Change NodeGroup maxPods validation from hard rejection to warning.
  • [registry] Omitted the auth field in DockerConfig when credentials (username and password) are empty.

1.74.1

Know before update

  • Prevents creation of Projects with too-long names that would lead to invalid generated Kubernetes resource names.
  • deckhouse queue
  • deckhouse queue blocked

Features

  • [common] Enrich debug-container image with nsenter and /bin/sh symlink.
  • [common] Resource quota ignore mechanism for pvc and pods
  • [dhctl] Skipped application edition validation for standalone builds.

Fixes

  • [candi] remove excessive netcat calls from d8-shutdown-inhibitor
  • [candi] Add pause and kubernetes-api-proxy registry packages to bashible bb-package-fetch to prevent node failures during containerd major upgrades.
  • [control-plane-manager] Fixed vulnerabilities
  • [deckhouse] Fix module docs rendering.
  • [deckhouse] Fix module enabling.
  • [deckhouse-controller] fix copy-custom-certificate hook
  • [docs] Added link to NVIDIA Container Toolkit docs.
  • [docs] Fixed typo.
  • [docs] Fixed the logic of generating the revision table.
  • [extended-monitoring] Add namespace-scoped overrides
  • [ingress-nginx] Improved stability of geoproxy service startup.
  • [kube-dns] Refactored hook remove_old_coredns_deploy_and_svc — delegated deletion of original kube-dns service to helm.
  • [multitenancy-manager] Add validation to restrict Project name length to 53 characters.
    Prevents creation of Projects with too-long names that would lead to invalid generated Kubernetes resource names.
  • [node-manager] remove excessive netcat calls from d8-shutdown-inhibitor
  • [node-manager] It fixes issues in the DaemonSet manifest for fencing module.
  • [tools] Add check values not null in prometheusRule or ClusterObservabilityPropagatedMetricsRulesGroup
    deckhouse queue
  • [tools] Add check values not null in prometheusRule or ClusterObservabilityPropagatedMetricsRulesGroup
    deckhouse queue blocked
  • [user-authn] Quote service names to prevent digit-only names from breaking yaml parser
  • [user-authz] Allow project-scoped roles to access Cluster-wide objects

Version 1.73

Important

  • This release includes several important security improvements. Multiple known vulnerabilities have been fixed, including one in the user-authn module (CVE-2025-22868) that could potentially allow bypassing authentication checks. It is recommended that you schedule this update. See the Security section for details.

  • The dashboard module will be removed in future DKP versions. Use the Deckhouse web UI instead (requires the console module to be enabled).

  • The runtime-audit-engine module is now loaded from an external source (the deckhouse ModuleSource).

  • All DKP components will be restarted during the update.

Major changes

  • Added support for changing the registry settings (via the mode parameter) in Unmanaged mode, similar to Direct mode. A new parameter checkMode has been added to control registry validation behavior:
    • Default: Verifies the availability of all system module images and the deckhouse-controller images.
    • Relax: Verifies only the deckhouse-controller images.
  • Updated requirements for the email field of the User object: the email address must now be in lowercase. Existing users will not be affected.

  • Dex updated to v2.44.0. It now allows authentication through available identity providers if one of them is down, and supports authentication via identity providers through a proxy.

  • The User object status now displays the reason for user lockout (controlled by the lockout parameter).

  • Added the additionalDisks parameter for the Deckhouse Virtualization Platform integration provider, allowing creation and attachment of additional disks to VMs in a NodeGroup (size and StorageClass must be specified). This simplifies data distribution across multiple disks.

  • Added support for additionalMetadata in objects (networks, VMs, disks) for the VMware Cloud Director integration provider. Metadata is merged with existing data, with additionalMetadata values taking precedence. Changing additionalMetadata triggers recreation of CloudEphemeral node groups that use the affected VCDInstanceClass.

  • For the VMware vSphere integration provider, you can now specify an SPBM storage policy ID (via the storagePolicyID parameter) and configure automatic creation of a StorageClass for each available SPBM policy. You can now explicitly select a policy for master and worker nodes and use the corresponding storage classes.

  • Added alerts to help plan module deprecation or migration:
  • Added GeoIPDownloadErrorDetected alert to notify about MaxMind GeoIP database download issues.

  • The update notification workflow has changed — a release is applied only after the notification is successfully delivered to the configured webhook. If delivery fails, the update is paused until the webhook is restored.

  • Reorganized in-cluster documentation. All module documentation (including connected ones) is now located under the Modules section. Search has been updated.

  • For Ingress NGINX Controller v1.10, added the option to enable the profiler (via the nginxProfilingEnabled parameter). Enabling the profiler increases resource consumption but may be useful for debugging controller issues.

  • Added support for custom HTTP authentication headers (via the headers parameter of UpmeterRemoteWrite) when sending SLA monitoring metrics via Prometheus Remote Write protocol.

  • Optimized DKP core module loading, reducing startup time in clusters with many modules.

  • Audit logs now show which OIDC provider issued the authentication token

  • Deckhouse CLI (d8) updated to v0.23.0:
    • Added the d8 status command, which provides a quick cluster summary (nodes, releases, Deckhouse pods, alerts, registry, Deckhouse settings, CNI, queue state).
    • Added the d8 k debug command, which runs the DKP built-in debug container (image can be overridden via --image). This simplifies interactive pod debugging.
    • Added the --watch flag to the d8 system queue list command to track queue state changes.

Security

  • Improved baseline container security. Updated security profiles for the following modules to restrict privileges and access rights to the minimum required: cni-cilium, cni-flannel, cni-simple-bridge, ingress-nginx, istio, keepalived, kube-dns, kube-proxy, node-local-dns, network-gateway, network-policy-engine, open-vpn.

  • Added the allowRbacWildcards flag to the SecurityPolicy, controlling whether wildcards are allowed in Role and RoleBinding definitions (set to true by default). Security policies can now also restrict interactive connections to pods (CONNECT for pods/exec and pods/attach) within namespaces.

  • Added support for preventing creation of pods with specific tolerations from a list (policies.disallowedTolerations parameter in the operational policy). This helps prevent user workloads from running on nodes reserved for special tasks.

  • Enhanced security in Ingress NGINX Controller v1.12 (distroless image, vulnerability fixes, and other improvements).

  • Fixed known vulnerabilities in the following modules: operator-trivy, registry, user-authn, cloud-provider-dvp, multitenancy-manager, admission-policy-engine, ingress-nginx, alertmanager, metallb, istio, node-local-dns, kube-apiserver.

Network

  • For the VMware Cloud Director integration provider, added support for LoadBalancer backed by VMware NSX Advanced Load Balancer (ALB/Avi) when using NSX-T. Requires the Load Balancer feature to be enabled on the Edge Gateway. If LoadBalancer is enabled after cluster creation, DKP automatically detects and applies the change within an hour. For open ports, DKP creates corresponding Pool + Virtual Service pairs. If there is a firewall, add allow rules for the LoadBalancer’s external IP address and relevant ports.

The complete list of changes, including the updated components, is available in the changelog on GitHub.

1.73.14

Know before update

  • All Ingress-Nginx controller pods will be restarted.
  • All ingress-nginx controller pods will be restared.
  • All modules of the ingress-nginx module will be restarted.
  • Registry pods will be restarted.
  • The ingress nginx controllers’ pods will be restated.

Fixes

  • [admission-policy-engine] added mount-points in ratify
  • [admission-policy-engine] deny_vulnerable_images webhook must use module status instead of moduleconfig
  • [candi] cve fix for admission-policy-engine, cert-manager, user-authn, multitenancy-manager
  • [candi] Updated pwru tool to v1.0.11 to fix CVE-2025-68121.
  • [candi] Updated pwru tool to v1.0.11 to fix CVE-2025-68121.
  • [chrony] mitigate CVE-2025-58181
  • [common] Fixed vulnerability CVE-2026-24051.
  • [control-plane-manager] Fixed vulnerabilities.
  • [documentation] Fixed HTTP/404 error when opening modules documentation from the Web interface.
  • [extended-monitoring] Fixed CVE-2025-47914, CVE-2025-58181
  • [ingress-nginx] CVE-2026-3288 fix is backported in all Ingress-Nginx controllers.
    All Ingress-Nginx controller pods will be restarted.
  • [ingress-nginx] Golang and Altlinux updates are reverted.
    All modules of the ingress-nginx module will be restarted.
  • [ingress-nginx] Nginx and module’s dependencies are updated.
    All ingress-nginx controller pods will be restared.
  • [ingress-nginx] The CVE-2026-1580, CVE-2026-24512, CVE-2026-24513, CVE-2026-24514 CVEs fixes are backported.
    The ingress nginx controllers’ pods will be restated.
  • [loki] Fixed CVE-2025-47914, CVE-2025-58181
  • [monitoring-kubernetes] Fixed CVE-2025-47914, CVE-2025-58181
  • [node-local-dns] Fixed CVEs
  • [node-manager] Fixed vulnerabilities.
  • [operator-prometheus] Fixed CVE-2025-47914, CVE-2025-58181
  • [prometheus] Fixed CVE-2025-47914, CVE-2025-58181, CVE-2025-65637
  • [prometheus-metrics-adapter] Fixed CVE-2025-47914, CVE-2025-58181
  • [prometheus-pushgateway] Fixed CVE-2025-47914, CVE-2025-58181, CVE-2025-22872, CVE-2025-22868
  • [registry] Updated auth image Go dependencies to fix Go CVEs.
    Registry pods will be restarted.
  • [registry] Updated auth image Go dependencies to fix Go CVEs.
    Registry pods will be restarted.
  • [registrypackages] Fixed vulnerability CVE-2026-24051.
  • [upmeter] Add proper securityContext to the upmeter probe to meet the restricted security profile constraints.
  • [upmeter] Fixed CVE-2025-47914, CVE-2025-58181, CVE-2025-65637

1.73.13

Know before update

  • config module loki ↓

Fixes

  • [candi] Updated the bashible step to include Linux kernel versions that address CVE-2025-37999
  • [cni-cilium] Fixed vulnerabilities.
  • [cni-flannel] Bump version and fixed vulnerabilities.
  • [istio] Fix CVE for Istio version 1.21 and 1.25
  • [kube-dns] Fixed vulnerabilities.
  • [kube-proxy] Fixed vulnerabilities.
  • [kube-proxy] Fixed vulnerabilities.
  • [loki] disable send analytics report to stats.grafana.org
    config module loki ↓
  • [network-policy-engine] Fixed vulnerabilities.
  • [node-local-dns] Bump version and fixed vulnerabilities.
  • [openvpn] Fixed vulnerabilities.
  • [registry] Omitted the auth field in DockerConfig when credentials (username and password) are empty.

1.73.12

Fixes

  • [cloud-provider-vsphere] fix stale session for cloud-data-discoverer
  • [istio] fixing the CVE in Kiali

1.73.11

Fixes

  • [dashboard] Fixed CVE-2025-30204 by updating dashboard components
  • [deckhouse] Fix module enabling.
  • [registrypackages] Update integrity patch for containerd (cse only).

1.73.10

Fixes

  • [cloud-provider-dvp] Added functionality to wait for a disk to be attached to a VM
  • [cloud-provider-huaweicloud] fix CSI unpublishValidation for non exist ECS instance
  • [registrypackages] Update integrity patch for containerd (cse only).
  • [service-with-healthchecks] Fixed CVEs
  • [user-authz] cache namespace label checks in the user-authz webhook via informer to avoid per-request apiserver GETs

1.73.9

Features

  • [ci] Add DVP e2e for the cse edition

Fixes

  • [deckhouse-controller] Fix conversions for external modules
  • [dhctl] Fix parallel bootstrap cloud permanent nodes

1.73.8

Know before update

  • Ingress Nginx controller dashboards are fixed
  • The MTU will be updated on all interfaces of all pods.

Fixes

  • [cloud-provider-yandex] cloud-provider-yandex CVE’s was fixed
  • [cni-cilium] The MTU configuration has been updated.
    The MTU will be updated on all interfaces of all pods.
  • [deckhouse-controller] Fix “multiple readiness hooks found” error on hook registration retry after failure.
  • [deckhouse-controller] Fix deprecated and experimental module metrics not being reset when module is disabled.
  • [dhctl] Fix panic during destroy. Change opentofu log level to INFO.
  • [docs] The current editions are listed on the page about installation in a closed environment.
  • [docs] Fixed links to the Getting Started in the documentation.
  • [prometheus] Fix namespace label value in the Ingress Nginx controller and several other metrics
    Ingress Nginx controller dashboards are fixed
  • [user-authn] Fix BadRequest after the change password redirect when password policy is enabled
  • [user-authn] Fix login error 500 with password policy enabled.

1.73.7

Know before update

  • Allows scaling control-plane from 1→3 in clusters where ControlPlaneKubeletLocalMode=true.
  • The instructions for reducing the number of master nodes have not been updated for a long time, and for a long time now, the conversion to reducing the number of master nodes in clusters in a cloud installation is automatic.
  • upmeter check

Features

  • [ci] Add Deckhouse Virtualization Platform testing to e2e tests.
  • [docs] Added Commander support for DVP EE in the Platform Editions table.

Fixes

  • [candi] Add missing debian 13 version to detect_bundle
  • [cloud-provider-huaweicloud] fix Provider ID
  • [control-plane-manager] Fix “etcd join” phase for control-plane scaling in v1.33.
    Allows scaling control-plane from 1→3 in clusters where ControlPlaneKubeletLocalMode=true.
  • [deckhouse-controller] Fixed verifying migrated modules
  • [deckhouse-controller] fix conversion applying for external modules
  • [dhctl] Fix StaticInstance readiness check and refactoring readiness check for resources.
  • [docs] Outdated instructions for reducing the number of master nodes in cloud clusters have been fixed. Users will correctly perform the converge of cloud clusters.
    The instructions for reducing the number of master nodes have not been updated for a long time, and for a long time now, the conversion to reducing the number of master nodes in clusters in a cloud installation is automatic.
  • [kube-dns] Improved /etc/hosts renderer compatibility with admission-policy-engine Restricted mode.
  • [log-shipper] Update documentation.
  • [upmeter] fix securityxontext for statefulset
    upmeter check

1.73.6

Fixes

  • [cloud-provider-vsphere] fix cloud-data-discoverer (SPBM)

1.73.5

Know before update

  • node-exporter pods will be rollout restarted during upgrade

Features

  • [ci] Add static e2e for the cse edition
  • [ci] Add static e2e for the cse edition
  • [docs] Added documentation about configuration of the Kubernetes audit policy.

Fixes

  • [cloud-provider-azure] fix build image for azure ccm
  • [cni-cilium] Some issues have been fixed in the EgressGateway.
  • [dhctl] Fix converge manifests for static cluster in commander.
  • [monitoring-kubernetes] remove the Docker traces from the module code
    node-exporter pods will be rollout restarted during upgrade

1.73.4

Know before update

  • Ingress-Nginx controller’s pods of 1.9 version will be restarted.
  • erroneous alerts for node disk usage are fixed

Fixes

  • [cloud-provider-dvp] add missing field to Cluster
  • [cloud-provider-yandex] Terraform auto converger was failed for WithNATInstance layout.
  • [deckhouse] Fix validation logic for a disabled module
  • [dhctl] Validate WithNATInstance Yandex layout params only in bootstrap.
  • [dhctl] Added nil check to dhctl during converge in migrator
  • [extended-monitoring] drop metrics when extended monitoring is disabled for node(s)
    erroneous alerts for node disk usage are fixed
  • [ingress-nginx] A symlink to the new opentelemetry config path is added.
    Ingress-Nginx controller’s pods of 1.9 version will be restarted.
  • [ingress-nginx] Fixed CVEs
  • [monitoring-kubernetes] Rollout changes for resources metrics kubelet
  • [node-manager] Fix panic in registry packages proxy if image not found.
  • [operator-trivy] Fix CIS Benchmark report template
  • [operator-trivy] fix CVE-2025-52881 CVE-2024-25621 on image trivy
  • [registrypackages] Update containerd to 1.7.29 / 2.1.5 and runc to 1.3.3

1.73.3

Know before update

  • the extended monitoring will only be enabled when the label is explicitly set on a namespace

Fixes

  • [control-plane-manager] Add vex for CVE-2025-31133, CVE-2025-52881 .
  • [deckhouse] Add umask before creating first-control-plane-bashible-ran file
  • [dhctl] Fix getting passphrase for key from connection config for cli.
  • [dhctl] Move yandex withNATInstance layout settings from preflights to preparator.
  • [documentation] Fix module’s deployment and configuration.
  • [extended-monitoring] Fix extended-monitoring.deckhouse.io/enabled label handling
    the extended monitoring will only be enabled when the label is explicitly set on a namespace
  • [monitoring-kubernetes] fix CVE-2025-52881 for node-exporter
  • [prometheus] Fix description for not usable CVE
  • [registrypackages] Fixes CVE in kubernetes-cni

1.73.2

Fixes

  • [ingress-nginx] CVEs fixed
  • [istio] Kiali’s CVE-2024-28180 is fixed.
  • [multitenancy-manager] fix CVE-2024-25621 CVE-2025-64329
  • [node-manager] fix cve and vex
  • [operator-trivy] fix CVE-2025-52881 CVE-2024-25621
  • [operator-trivy] Add grep to node-collector and improve error reporting
  • [user-authn] Rollback patch for handling insecureSkipEmailVerified condition

1.73.1

Features

  • [docs] Added manifest for internal LB VK Cloud.
  • [docs] Added info from csi-netapp module about NetApp storage integration in DKP documentation.
  • [istio] Added access log format setting for proxy sidecars.

Fixes

  • [admission-policy-engine] Prohibit only creation or modification for objects with vulnerable images
  • [admission-policy-engine] Fixed proxy support for trivy-provider
  • [candi] Remove proxy-environment for kubelet when proxy disabled
  • [ci] Fix e2e upgrade tests.
  • [ci] fix grafana vex
  • [ci] fix trickster vex
  • [ci] Fix missing edition suffixes in prerelease build.
  • [cloud-provider-dvp] Correct the calculation of the path to the device
  • [cloud-provider-vsphere] fix vSphere storageClass template
  • [cni-cilium] fixed connectivity dashboard
  • [deckhouse] Fix race conditions in test
  • [deckhouse-controller] fix bash completion
  • [docs] Small fix on the “Security event audit” page.
  • [docs] Fixed the description for the 4239 port in the reference table.
  • [docs] Updated installer block in GS.
  • [docs] Added a link to the DVP integration page in the virtualization overview.
  • [istio] The same owner is specified for the files that are used to run in the operator container.
  • [metallb] Disabled unnecessary speaker functionality.
  • [monitoring-kubernetes] Fixed CVE-2025-22870, CVE-2025-22872.
  • [prometheus] Fix plur in vex for tricster
  • [prometheus-pushgateway] Fixed CVE-2025-22869, CVE-2025-22868, CVE-2025-22870.
  • [registrypackages] Update runc to 1.3.1.

Version 1.72

Important

  • All DKP components will be restarted during the update.
  • To use experimental modules in the cluster, you now need to explicitly enable the allowExperimentalModules parameter. By default, experimental modules are disabled. Modules that were enabled before the update will not be automatically disabled. However, if an experimental module enabled prior to the update is manually disabled during the update process, you will need to grant permission to use experimental modules again in order to re-enable it.
  • If there are WireGuard interfaces on the cluster nodes, you must update the Linux kernel to version 6.8 or higher.

Major changes

  • Added a new registry module and the ability to adjust container registry parameters without restarting all DKP components. Two modes for working with the container registry are now available in DKP: Unmanaged (the approach used in previous versions) and Direct (a new mode). In Direct mode, DKP creates a virtual container registry address in the cluster that all DKP components use. Changing the container registry address (for example, switching to a different registry or changing the DKP edition in the cluster) in this mode does not trigger a forced restart of all DKP components.

  • Added support for recursive DNS servers (configured via the recursiveSettings section of the cert-manager module). They are used to verify the existence of a DNS record before starting the ACME DNS-01 domain ownership validation process. This is useful if the same domain is used both publicly and within the cluster, or if the domain has dedicated authoritative DNS servers.

  • Introduced separation of modules into critical and functional using the critical flag in module.yaml. Critical modules are started first. Functional modules are started after the bootstrap process is complete. Their tasks run in parallel and do not block the queue in case of failure. This speeds up cluster installation and improves fault tolerance when starting modules.

  • You can now enable logging of all DNS queries (the enableLogs parameter of the node-local-dns module).

  • In the cloud-provider-vcd module, a new WithNAT layout has been added for cluster deployment. It automatically configures NAT and, if necessary, firewall rules for accessing nodes through a bastion host. It also supports both NSX-T and NSX-V. This makes it possible to deploy a cluster in VMware Cloud Director without pre-configuring the environment (unlike the Standard layout).

Security

  • Added the fields user-authn.deckhouse.io/name and user-authn.deckhouse.io/preferred_username to Kubernetes audit log events. These fields display user claims from the OIDC provider, improving authentication monitoring and troubleshooting.

  • Kubernetes versions 1.30–1.33 have been updated to the latest patch releases.

  • For the AWS provider, added the ability to disable the creation of default security groups (the disableDefaultSecurityGroup parameter). When disabled, security groups must be created manually and explicitly specified in AWSClusterConfiguration, AWSInstanceClass, and NodeGroup. This new feature provides greater control over security settings.

  • Added support for password policies for local users (configured in the passwordPolicy section). You can now enforce a minimum password complexity, set password expiration, require password rotation, prevent reuse of old passwords, and lock accounts after a specified number of failed login attempts. These changes allow administrators to centrally enforce password requirements and improve cluster security.

The complete list of changes, including the updated components, is available in the changelog on GitHub.

1.72.13

Fixes

  • [istio] Reduce CPU and RAM for regenerate multicluster JWT token and sort ingressGateway

1.72.12

Fixes

  • [deckhouse-controller] Fix conversions for external modules
  • [deckhouse-controller] Fixed verifying migrated modules
  • [deckhouse-controller] Fixed a crash during external module updates with conversions that caused ModuleRelease to fail validation due to a forbidden property error.
  • [istio] Fixed AuthorizationPolicy CRD insufficiency for Istio 1.25.

1.72.11

Fixes

  • [cert-manager] Mark module as critical for cluster
  • [deckhouse-controller] fix deckhouse update accidentally minor skip
  • [istio] Erroneous option in 1.25 control-plane helm template fixed.
  • [registrypackages] Fixed permissions for directory with cni-plugins in PCI-DSS clusters
  • [tools] update dmt version to 0.1.53

1.72.10

Fixes

  • [ci] Delete an extra entry from the changelog.
  • [cloud-provider-vsphere] fix hook logick
  • [deckhouse-controller] fix possibility deadlock
  • [istio] Reduce RAM for regenerate multicluster JWT token

1.72.9

Fixes

  • [candi] Switch node hostname in discover_node_ip.sh to locked hostname
  • [candi] fixed version of containerd image in bashible step
  • [deckhouse] Fixes bug when module`s config values unchanged after enabling.
  • [deckhouse-controller] Add more mutexes to shell-operator framework
  • [dhctl] Change default ssh mode
  • [ingress-nginx] The is reverted.
  • [ingress-nginx] Enable ExecuteHookOnEvents on hook set_annotation_validation_suspended.go

1.72.8

Fixes

  • [cloud-provider-dvp] fix single-node LoadBalancer bug and add multi-LB per node support
  • [cloud-provider-vsphere] cloud-data-discoverer fixes
  • [node-manager] Enable SSHLegacyMode if SSHPrivateKey is not empty and fix gossh panic.

1.72.7

Fixes

  • [cloud-provider-huaweicloud] add rule to d8:cloud-provider-huaweicloud:cloud-controller-manager account for list pods as needed
  • [node-manager] CAPI crd served version fix
  • [node-manager] shutdown inhibitor use three-state GracefulShutdownPostpone condition so kubelet waits for the final decision before ending pods

1.72.6

Fixes

  • [candi] Fix nodeuser deletion.
  • [cloud-provider-vsphere] cloud-data-discoverer fixes
  • [deckhouse-controller] Ensure afterDeleteHelm hooks receive Kubernetes snapshots by stopping monitors after hook execution
  • [dhctl] Fixing bashible steps when it does not respond to the limit on the number of attempts
  • [dhctl] Fix start and keep-alive behavior, do not change host if any kube-proxies has been started.
  • [node-manager] CAPI crd served version fix
  • [vertical-pod-autoscaler] Disable vpa for k8s-metacollector in module runtime-audit-engine

1.72.5

Fixes

  • [cloud-provider-vsphere] cloud-data-discoverer fixes
  • [cni-cilium] Fixed 1.14 -> 1.17 migration mechanism
  • [cni-cilium] improved migration to 1.17 logic
  • [node-manager] capi crd served version fix

1.72.4

Fixes

  • [cloud-provider-openstack] fix openstack ccm deployment
  • [deckhouse] Fix maintenance mode label ignore rule.
  • [ingress-nginx] lowers the severity level of the IngressNginxController alert from 3 to 4

1.72.3

Fixes

  • [dhctl] Add timeout to ssh client dial and fix stop.
  • [dhctl] Fix ssh client start with bastion in dhctl.
  • [dhctl] Fix deadlock while reading dhctl singlethreaded server logs
  • [dhctl] Fixed the behaviour of the attach operation when there are no keys in SSHConfig, only a password
  • [docs] Containerdv2 migration FAQ fix
  • [ingress-nginx] fix set_annotation_validation_suspended.go hook to set correct metric
  • [ingress-nginx] fix set_annotation_validation_suspended.go hook to set correct metric
  • [loki] Change stage to Preview

1.72.2

Know before update

  • The ingress-nginx pods of 1.10 will be restarted.

Fixes

  • [docs] Added the Local path provisioner storage section to the documentation sidebar, including Overview, Examples, Reference, and FAQ entries.
  • [ingress-nginx] Fixed nginx segfaults when opentelemetry is enabled.
    The ingress-nginx pods of 1.10 will be restarted.
  • [node-manager] Fix node update approval hook for CAPI nodes.

1.72.1

Fixes

  • [deckhouse] add check for user-authn in enabled script
  • [deckhouse] fix editions list in check deckhouse edition hook
  • [dhctl] Fix ssh client initialising in Commander Attach and Commander Detach operations
  • [node-manager] Some minor fixes for nvidia gpu templates to improve security.

Version 1.71

Important

  • Prometheus has been replaced with Deckhouse Prom++. If you want to keep using Prometheus, disable the prompp module manually before upgrading DKP by running the command d8 system module disable prompp.

  • Support for Kubernetes 1.33 has been added, while support for Kubernetes 1.28 has been discontinued. In future DKP releases, support for Kubernetes 1.29 will be removed. The default Kubernetes version (used when the kubernetesVersion parameter is set to Automatic) has been changed to 1.31.

  • Upgrading the cluster to Kubernetes 1.31 requires a sequential update of all nodes, with each node drained. You can control how node updates requiring workload disruptions are applied using the disruptions parameter section.

  • The built-in snapshot-controller and static-routing-manager modules will now be replaced with their external counterparts of the same name, sourced via ModuleSource deckhouse.

  • The new version of Cilium requires nodes to run Linux kernel version 5.8 or newer. If any node in the cluster has a kernel older than 5.8, the Deckhouse Kubernetes Platform upgrade will be blocked. Cilium Pods will be restarted.

  • All DKP components will be restarted during the update.

Major changes

  • You can now enforce two-factor authentication for static users. This is configured via the staticUsers2FA parameter section of the user-authn module.

  • Added support for GPUs on nodes. Three GPU resource sharing modes are now available: Exclusive (no sharing), TimeSlicing (time-based sharing), and MIG (a single GPU split into multiple instances). The NodeGroup spec.gpu parameter section is used to configure the GPU resource sharing mode. Using a GPU on a node requires installing the NVIDIA Container Toolkit and the GPU driver.

  • When enabling a module (with d8 system module enable) or editing a ModuleConfig resource, a warning is now displayed if multiple module sources are found. In such a case, explicitly specify the module source using the source parameter in the module’s configuration.

  • Improved error handling for module configuration. Module-related errors no longer block DKP operations. Instead, they are now displayed in the status fields of Module and ModuleRelease objects.

  • Improved virtualization support:
    • Added a provider for integration with Deckhouse Virtualization Platform (DVP), enabling deployment of DKP clusters on top of DVP.
    • Added support for nested virtualization on nodes in the cni-cilium module.
  • The node-manager module now includes several enhancements for improved node reliability and manageability:
    • You can now prevent a node from restarting if it still hosts critical Pods (labeled with pod.deckhouse.io/inhibit-node-shutdown). This can be necessary for workloads with stateful components, such as long-running data migrations.
    • Introduced API version v1alpha2 for the SSHCredential resource, where the sudoPasswordEncoded parameter allows specifying the sudo password in Base64 format.
    • The capiEmergencyBrake parameter allows you to disable Cluster API (CAPI) in emergency scenarios, preventing potentially destructive changes. Its behavior is similar to the existing mcmEmergencyBrake setting.
  • Added a pre-installation check to verify connectivity to the DKP container image registry.

  • Improved the log file rotation mechanism when using short-term log storage (via the loki module). Added the LokiInsufficientDiskForRetention alert to warn about insufficient disk space for log retention.

  • The documentation now includes a reference for the Deckhouse CLI (d8 utility) commands and parameters.

  • When using CEF encoding for collecting logs from Apache Kafka or socket sources, you can now configure auxiliary CEF fields such as Device Product, Device Vendor, and Device ID.

  • The passwordHash field in the NodeUser resource is no longer required. This allows you to create users without passwords — for example, in clusters that use external authentication systems (such as PAM or LDAP).

  • Added support for CRI Containerd v2 with CgroupsV2. The new version introduces a different configuration format and includes a mechanism to migrate between Containerd v1 and v2. You can change the CRI type used on nodes via the cri.type parameter and configure it using cri.containerdV2.

Security

  • Container image signature verification is now available in DKP SE+. This feature is now supported in DKP SE+ and EE.

  • The log-shipper, deckhouse-controller, and Istio (version 1.21) modules have been migrated to distroless builds. This improves security and ensures a more transparent and controlled build process.

  • New audit rules have been added to track interactions with containerd. The following are now monitored: access to the /run/containerd/containerd.sock socket, modifications to the /etc/containerd and /var/lib/containerd directories and the /opt/deckhouse/bin/containerd file.

  • Known vulnerabilities have been fixed in the following modules: loki, extended-monitoring, operator-prometheus, prometheus, prometheus-metrics-adapter, user-authn, and cloud-provider-zvirt.

Network

  • Added support for Istio version 1.25.2, which uses the Sail operator instead of the deprecated Istio Operator. Also added support for Kiali version 2.7, without Ambient Mesh support. Istio version 1.19 is now considered deprecated.

  • Added support for encrypting traffic between nodes and Pods using the WireGuard protocol (via the encryption.mode parameter).

  • Fixed the logic for determining service readiness in the ServiceWithHealthcheck resource. Previously, Pods without an IP address (for example, in Pending state) could be mistakenly included in the load balancing list.

  • Added support for the least-conn load balancing algorithm. This algorithm directs traffic to the service backend with the fewest active connections, improving performance for connection-heavy applications (such as WebSocket services). To use this algorithm, enable the extraLoadBalancerAlgorithmsEnabled parameter in the cni-cilium module settings and use the service.cilium.io/lb-algorithm annotation on the service and set it to a supported value: random, maglev, or least-conn.

  • Fixed an issue in Cilium 1.17 cilium-operator where IP addresses were not reused after a CiliumEndpoint was deleted. The issue was caused by improper cleanup of priority filters, which could lead to IP pool exhaustion in large clusters.

  • Refined the list of ports used for networking:

    • Added and updated:
      • 4287/UDP: WireGuard port used for CNI Cilium traffic encryption.
      • 4295-4297/UDP: Used by the cni-cilium module for VXLAN encapsulation of inter-pod traffic in multiple nested virtualization — when DKP with the virtualization module enabled is deployed inside virtual machines that are also created in DKP with the virtualization module enabled.
      • 4298/UDP: Used by the cni-cilium module for VXLAN encapsulation of traffic between pods if the cluster was deployed on DKP version starting from 1.71 (for clusters deployed on DKP versions up to 1.71, see the note for ports 4299/UDP, 8469/UDP, and 8472/UDP).
      • 4299/UDP: Port for clusters deployed on DKP versions 1.64–1.70. Used by the cni-cilium module for VXLAN encapsulation of traffic between pods. Updating DKP to newer versions will not change the port used unless the virtualization module is enabled.
      • 8469/UDP: Port for clusters deployed on DKP version 1.63 and below with the virtualization module enabled prior to DKP version 1.63. Used by the cni-cilium module for VXLAN encapsulation of traffic between pods. Updating DKP to newer versions will not change the occupied port
      • 8472/UDP: Port for clusters deployed on DKP version 1.63 and below. Used by the cni-cilium module for VXLAN encapsulation of traffic between pods. Updating DKP to newer versions will not change the occupied port if the virtualization module is not enabled. Note that in such clusters, enabling the virtualization module on DKP before version 1.70 changes the port:
        • Enabling the virtualization module on DKP version 1.63 and below will change it to 8469/UDP and will not change with subsequent DKP updates
        • Enabling the virtualization module on DKP starting from version 1.64 will change it to 4298/UDP and will not change with subsequent DKP updates
    • Removed:
      • 49152, 49153/TCP: Previously used for live migration of virtual machines (in the virtualization module). Migration now occurs over the Pod network.

The complete list of changes, including the updated components, is available in the changelog on GitHub.

1.71.10

Features

  • [deckhouse-controller] ignore migrate module if disabled

Fixes

  • [deckhouse-controller] fix deckhouse update accidentally minor skip
  • [istio] Reduce CPU and RAM for regenerate multicluster JWT token and sort ingressGateway
  • [local-path-provisioner] Fix CVE in local-path-provisioner module

1.71.9

Fixes

  • [cni-cilium] Fixed 1.14 -> 1.17 migration mechanism

1.71.8

Fixes

  • [deckhouse] Fix module source updating fails.
  • [vertical-pod-autoscaler] Disable vpa for k8s-metacollector

1.71.7

Fixes

  • [candi] Fix arg encryption-provider-config for kubeadm configuration.
  • [candi] Disable immutable flag on erofs files in cleanup node stage.

1.71.6

Fixes

  • [cni-cilium] Fixed the infinite loop in the “cilium migration” bashible step and improved synchronization between bashible and safe-agent-updater.

1.71.5

Know before update

  • If wireguard interface is present on nodes, then cilium-agent upgrade will stuck. Upgrading the linux kernel to 6.8 is required.
  • main and longterm Prometheuses will be rollout-restarted

Fixes

  • [candi] Changed ExecStartPre in d8-shutdown-inhibitor.service
  • [candi] Added warnings to the VMware Cloud Director environment documentation about Edge requirements
  • [cni-cilium] Add a compatibility check for the Cilium version and the kernel version, if WireGuard is installed on the node
    If wireguard interface is present on nodes, then cilium-agent upgrade will stuck. Upgrading the linux kernel to 6.8 is required.
  • [cni-cilium] Added a migration mechanism, which was implemented through the node group disruptive updates with approval.
  • [cni-cilium] fixed invalid annotation name for lb-algorithm in docs
  • [deckhouse-controller] fix module config ensure
  • [docs] Fix relative links in the multitenancy-manager module documentation.
  • [docs] Added steps that patch secret and prevented the image pull fail.
  • [docs] Add containerv2 additional registry examples
  • [docs] Add new requirement and commands to meet containerdv2
  • [docs] Fixed command syntax for Docker container run in documentation.
  • [docs] Add new requirement and commands to meet containerdv2 requirements
  • [docs] Fix D8KubernetesStaleTokensDetected alert description.
  • [docs] There should be one disk in the template.
  • [docs] Updates for Observability docs.
  • [log-shipper] Documentation updates.
  • [loki] Documentation updates.
  • [monitoring-custom] Documentation updates.
  • [monitoring-kubernetes] Documentation updates.
  • [monitoring-ping] Documentation updates.
  • [prometheus] fix securityContext indentation in the Prometheus main and longterm resources
    main and longterm Prometheuses will be rollout-restarted
  • [prometheus] Documentation updates.
  • [upmeter] Documentation updates.

1.71.4

Know before update

  • low

Features

  • [docs] Added descriptions of the problem of using Cilium and Elastic together
    low

Fixes

  • [cloud-provider-vsphere] ignore ept_rvi_mode and hv_mode
  • [docs] Fix mentions of ingress-nginx module in docs.
  • [ingress-nginx] Fix mentions of ingress-nginx module in docs.
  • [prometheus] Fix mentions of ingress-nginx module in docs.

1.71.3

Features

  • [candi] Add rosa 12.6.
  • [deckhouse-controller] add validation for module source changes

Fixes

  • [candi] containerd migration fix
  • [cni-cilium] fixed hostPort workability with extraLoadBalancerAlgorithmsEnabled.
  • [log-shipper] Fix matchExpressions values validation for ClusterLoggingConfig
  • [node-manager] Hide Nvidia GPU dashboards if there is no GPU nodes in a cluster.
  • [node-manager] containerd migration fix

1.71.2

Know before update

  • Users will know how to configure KeyCloak and dexProvider to get user rights in dashboard and console.
  • default
  • if there are several ModuleSources with the prompp module available, the “deckhouse” ModuleSource will be used.

Features

  • [admission-policy-engine] Added the ability to specify allowedServiceTypes and blockWildcardDomains in the SecurityPolicy object
  • [deckhouse-controller] add leader election for non ha clusters
  • [user-authn] Add allowedGroup in dexProvider OIDC

Fixes

  • [admission-policy-engine] Fix validation policy for pods with skip-pss-check labels
  • [ci] fix e2e creds
  • [ci] fix e2e creds
  • [ci] Run changelog check only on tag pushes to enforce release documentation
  • [ci] create resources before deckhouse manifest and mutating webhook for prom rules for e2e tests
  • [ci] Run changelog check only on tag pushes to enforce release documentation
  • [ci] create resources before deckhouse manifest and mutating webhook for prom rules for e2e tests
  • [cloud-provider-aws] Fix root_block_device converge.
  • [cni-cilium] Fixed least-conn logs when feature is disabled
  • [deckhouse-controller] fix new snapshot logic bag
  • [deckhouse-controller] add more verbosity to snapshot error
  • [dhctl] Fix false-positive staticinstance ip duplication preflight checks fails.
  • [docs] Correction of KeyCloak documentation in d8-user-authn
    Users will know how to configure KeyCloak and dexProvider to get user rights in dashboard and console.
  • [docs] Refactored table of supported OS versions.
  • [istio] Added missed documentation about version 1.25
  • [metallb] Fixed overwriting of Service status field by module components.
  • [metallb] avoid double prefixes in the dashboard names.
    default
  • [node-manager] Changed draining logic in update_approval
  • [node-manager] fixed render nvidia-mig-manager
  • [prometheus] use the “deckhouse” ModuleSource as the default for the prompp ModuleConfig
    if there are several ModuleSources with the prompp module available, the “deckhouse” ModuleSource will be used.
  • [runtime-audit-engine] Improve memory footprint by switching to the stdlib memory allocator instead of jemalloc

1.71.1

Features

  • [docs] Added documentation with recommendations for generating self-signed certificates

Fixes

  • [ci] add sh symlink to bash to the deckhouse-controller
  • [ci] Fix listing of changed modules.
  • [control-plane-manager] correct indentation of the ‘certSANs’ in v1beta4 template.
  • [monitoring-ping] Reducing PROCESSOR time consumption by changing the method of waiting for incoming packets.
  • [user-authn] fix dex oidc connector insecureSkipVerify and rootCAData options

Version 1.70

Important

  • The ceph-csi module has been removed. Use the csi-ceph module instead. Deckhouse will not be updated as long as ceph-csi is enabled in the cluster. For csi-ceph migration instructions, refer to the module documentation.

  • Version 1.12 of the Ingress NGINX Controller has been added. The default controller version has been changed to 1.10. All Ingress controllers that do not have an explicitly specified version (via the controllerVersion parameter in the IngressNginxController resource or the defaultControllerVersion parameter in the ingress-nginx module) will be restarted.

  • The falco_events metric (from the runtime-audit-engine module) has been removed. The falco_events metric was considered deprecated since DKP 1.68. Use the falcosecurity_falcosidekick_falco_events_total metric instead. Dashboards and alerts based on the falco_events metric may stop working.

  • All DKP components will be restarted during the update.

Major changes

  • In the Auto update mode, patch version updates (for example, from v1.70.1 to v1.70.2) are now applied taking into account the update windows, if they are set. Previously, in this update mode, only minor version updates (for example, from v1.69.x to v1.70.x) were applied with consideration to update windows, while patch version updates were applied as they appeared on a release channel.
  • A node can now be rebooted if the corresponding Node object has the update.node.deckhouse.io/reboot annotation set.
  • When cleaning up a static node, any local users created by Deckhouse Kubernetes Platform are now also removed.
  • Added synchronization monitoring for Istio in multi-cluster configurations. A new alert D8IstioRemoteClusterNotSynced has been introduced and triggers in the following cases:
    • The remote cluster is offline.
    • The remote API endpoint is not reachable.
    • The remote ServiceAccount token is invalid or expired.
    • There is a TLS or certificate issue between the clusters.
  • The deckhouse-controller collect-debug-info command now also collects debug information for Istio, including:
    • Resources in the d8-istio namespace.
    • CRDs from the istio.io and gateway.networking.k8s.io groups.
    • Istio logs.
    • Sidecar logs of a single randomly selected user application.
  • A new monitoring dashboard has been added to display OpenVPN certificate status. Upon expiration, server certificates will now be reissued, and client certificates will be removed. The following alerts have been added:
  • Monitoring dashboards have been renamed and updated:
    • “L2LoadBalancer” renamed to “MetalLB L2”; pool and column filtering added.
    • “Metallb” renamed to “MetalLB BGP”; pool and column filtering added. The ARP request panel has been removed.
    • “L2LoadBalancer / Pools” renamed to “MetalLB / Pools”.
  • The upmeter module’s PVC size has been increased to accommodate data retention for 13 months. In some cases, the previous PVC size was insufficient.

  • The ModuleSource resource status now includes information about module versions in the source.

  • The Module resource status now includes information about the module’s lifecycle stage. A module can move through the following stages in its lifecycle: Experimental, Preview, General Availability, and Deprecated. For details on module lifecycle stages and how to evaluate its stability, refer to the corresponding section in the documentation.

  • It is now possible to use stronger or more modern encryption algorithms (such as RSA-3072, RSA-4096, or ECDSA-P256) for control plane cluster certificates instead of the default RSA-2048. You can use the encryptionAlgorithm parameter in the ClusterConfiguration resource to configure this.

  • The descheduler module can now be configured to evict pods that are using local storage. Use the evictLocalStoragePods parameter in the module configuration to adjust this.

  • You can now adjust the logging level of the Ingress controller using the controllerLogLevel parameter in the IngressNginxController resource. The default log level is Info. Controlling the logging level can help prevent log collector overload during Ingress controller restarts.

Security

  • The severity level of alerts indicating security policy violations has been raised from 7 to 3.

  • The configuration for Yandex Cloud, Zvirt, and Basis Dynamix providers now uses OpenTofu instead of Terraform. This enables easier provider updates, such as applying fixes for known vulnerabilities (CVEs).

  • CVE vulnerabilities have been fixed in the following modules: chrony, descheduler, dhctl, node-manager, registry-packages-proxy, falco, cni-cilium, and vertical-pod-autoscaler.

The complete list of changes, including the updated components, is available in the changelog on GitHub.

1.70.19

Fixes

  • [ci] Fix render workflow job.
  • [istio] Reduce CPU and RAM for regenerate multicluster JWT token and sort ingressGateway

Chore

  • [runtime-audit-engine] Update CI file

1.70.18

Fixes

  • [istio] The metrics-exporter’s template is fixed, it blocked the main queue if controlPlane.nodeSelector setting was configured.

1.70.17

Fixes

  • [istio] fixed an error in processing custom input gateway addresses using the configuration option of the alliance.ingressGateway.advertise module.

1.70.16

Fixes

  • [cloud-provider-aws] fix root_block_device converge
  • [runtime-audit-engine] Improve memory footprint by switching to the stdlib memory allocator instead of jemalloc

1.70.15

Fixes

  • [control-plane-manager] correct indentation of the ‘certSANs’ in v1beta4 template.

1.70.14

Fixes

  • [cloud-provider-dvp] fix logic of work with disks and coreFraction validation
  • [cloud-provider-huaweicloud] fix virtualMachineClassName in terraform
  • [deckhouse] Filter deleted pvcs in storage class change hook.
  • [deckhouse] Filter deleted pvcs in storage class change hook.
  • [deckhouse-controller] make metrics more verbose

1.70.13

Know before update

  • Users can now restore etcd backups and scale their master nodes efficiently without encountering previous errors.

Fixes

  • [control-plane-manager] Fixed issues with etcd backup restoration process, ensuring reliable master node scaling from 1 to 3.
    Users can now restore etcd backups and scale their master nodes efficiently without encountering previous errors.
  • [deckhouse-controller] fix nil annotations map for module fetcher
  • [ingress-nginx] Validation is switched off temporarily.
  • [prometheus] Fix fake longterm emptydir alert.

1.70.12

Features

  • [cloud-provider-huaweicloud] add enterprise_project_id

Fixes

  • [dhctl] TLS support on dhctl edition checking step.

1.70.11

Fixes

  • [docs] remove Yandex Cloud’s api-gateway.editor and connection-manager.editor roles

1.70.10

Features

  • [docs] Add unified manual about edition switch

Fixes

  • [dhctl] Fix dhctl output in case of invalid config
  • [dhctl] Release infrastructure dependency lock.

1.70.9

Fixes

  • [control-plane-manager] Fix default behaviour for etcd backup.
  • [ingress-nginx] fixed geoip issues in validator
  • [ingress-nginx] Automatic geoip updating feature was reverted.

1.70.8

Fixes

  • [docs] Delete the mention that addresses are allocated starting from 10 from internalNetworkCIDR in vcd cloud-provider

1.70.7

Know before update

  • When called collect-debug-info command in cluster without Istio, user see errors, but archive create correct

Fixes

  • [control-plane-manager] fix kubeadm v1beta4 config
  • [deckhouse-controller] fix wrong behavior of Pending ModuleReleases
  • [ingress-nginx] fixed node-selector and tolerations in validator pod
  • [istio] Fix for collecting debugging resources in collect-debug-info about Istio
    When called collect-debug-info command in cluster without Istio, user see errors, but archive create correct
  • [node-local-dns] Improved stability for clusters with poor node-to-node connection quality by increasing node-local-dns to kube-dns healthcheck timeouts.

1.70.6

Fixes

  • [control-plane-manager] audit-log-maxage value to 30 days
  • [deckhouse] Fix obsolete config alert stuck after deleting module config.
  • [deckhouse] Remove unnecessary modules downloading.
  • [docs] Rendered SSHHost and SSHConfig specs.
  • [ingress-nginx] fixed an issue with the snapshot retrieval mechanism
  • [ingress-nginx] fix missing proxy client cert in validator deployment
  • [ingress-nginx] Ingress resource validation in ingress-nginx controller re-enabled.
  • [openvpn] Added a migration hook to mitigate false positive OpenVPNClientCertificateExpired alerts.
  • [user-authz] Add patch verb for service accounts

1.70.5

Know before update

  • loki statefulset will be rollout restarted

Fixes

  • [loki] Fix several bugs introduced while implementing loki disk based retention
    loki statefulset will be rollout restarted
  • [prometheus] Add headers field to values spec

1.70.4

Know before update

  • operator-prometheus

Features

  • [cloud-provider-dvp] Add vmClassName to the config
  • [operator-prometheus] Add to documetation for operator-prometheus CRD.
    operator-prometheus
  • [snapshot-controller] Automatic snapshot-controller MC creation for future switch to external module

Fixes

  • [cloud-provider-dvp] Fix cherry-pick conflicts pr
  • [ingress-nginx] Ingress resource validation in ingress-nginx controller re-enabled.

1.70.3

Fixes

  • [cloud-provider-yandex] Handle hybrid clusters case in the to_tofu_migrate_metric hook
  • [cloud-provider-zvirt] fix UUID validation regex
  • [deckhouse-controller] fixes default maintenance value which prevents stuck metric
  • [deckhouse-tools] deckhouse-tools fix deployment render
  • [docs] Updated conversions message.
  • [node-local-dns] Changed container liveness port.
  • [node-manager] fixed update for static clusters

1.70.2

Fixes

  • [candi] memory.available calculation python2 fix
  • [openvpn] Fixed condition of alert triggering for client certificate.
  • [service-with-healthchecks] change port kube-rbac-proxy in agent DaemonSet

1.70.1

Know before update

  • prometheus

Features

  • [cloud-provider-dvp] Add vmClassName to the config
  • [prometheus] Validation prometheus remotewrite ca webhook
    prometheus

Fixes

  • [candi] Added a check to the bashible step for the presence of a previously created symlink and where it leads to
  • [candi] bashible bb-discover-node-name use node name from kubelet certificate
  • [ci] returned the condition when the branch was not found
  • [ci] fix kubernetes version info in the release file
  • [cloud-provider-vcd] Fix unexpected behavior with legacy API versions in VCD
  • [deckhouse-controller] fix deckhouse crds mechanism
  • [dhctl] Fix unexpected behavior with legacy API versions in VCD
  • [docs] Update docs about Keycloak configuration for IP
  • [operator-trivy] Add support anon registry creds for bdu updater.
  • [user-authz] fix error with webhook system-roles

Version 1.69

Important

  • Support for Kubernetes 1.32 has been added, while support for Kubernetes 1.27 has been discontinued. The default Kubernetes version has been changed to 1.30. In future DKP releases, support for Kubernetes 1.28 will be removed.

  • All DKP components will be restarted during the update.

Major changes

  • The ceph-csi module is now deprecated. Plan to migrate to the csi-ceph module instead. For details, refer to the Ceph documentation.

  • You can now grant access to Deckhouse web interfaces using user names via the auth.allowedUserEmails field. Access restriction is configured together with the auth.allowedUserGroups parameter in configuration of the following modules with web interfaces: cilium-hubble, dashboard, deckhouse-tools, documentation, istio, openvpn, prometheus, and upmeter (example for prometheus).

  • A new dashboard Cilium Nodes Connectivity Status & Latency has been added to Grafana in the cni-cilium module. It helps monitor node network connectivity issues. The dashboard displays a connectivity matrix similar to the cilium-health status command, using metrics that are already available in Prometheus.

  • A new D8KubernetesStaleTokensDetected alert has been added in the control-plane-manager module that is triggered when stale service account tokens are detected in the cluster.

  • You can now create a Project from an existing namespace and adopt existing objects into it. To do this, annotate the namespace and its resources with projects.deckhouse.io/adopt. This lets you switch to using Projects without recreating cluster resources.

  • A Terminating status has been added to ModuleSource and ModuleRelease resources. The new status will be displayed when an attempt to delete one of them fails.

  • The installer container now automatically configures cluster access after a successful bootstrap. A kubeconfig file is generated in ~/.kube/config, and a local TCP proxy is set up through an SSH tunnel. This allows you to use kubectl locally right away without manually connecting to the control-plane node via SSH.

  • Changes to Kubernetes resources in multi-cluster and federation setups are now tracked directly via Kubernetes API. This enables faster synchronization between clusters and eliminates the use of outdated certificates. In addition, mounting of ConfigMap and Secret resources into Pods has been removed to eliminate family system compromise risks.

  • A new dynamicforward plugin has been added to CoreDNS, improving DNS query processing in the cluster. It integrates with node-local-dns, continuously monitors kube-dns endpoints, and automatically updates the list of DNS forwarders. If the control-plane node is unavailable, DNS queries are still forwarded to available endpoints, improving cluster stability.

  • A new log rotation approach has been introduced in the loki module. Now, old logs are automatically removed when disk usage exceeds a threshold: either 95% of PVC size or PVC size minus the size required to store two minutes of log data at the configured ingestion rate (ingestionRateMB). The retentionPeriodHours parameter no longer controls the data retention and is used for monitoring alerts only. If loki begins removing old logs before the set period is reached, a LokiRetentionPerionViolation alert will be triggered, informing the user that they must reduce the value of retentionPeriodHours or increase the PVC size.

  • A new nodeDrainTimeoutSecond parameter lets you set the maximum timeout when attempting to drain a node (in seconds) for each NodeGroup resource. Previously, you could only use the default value (10 minutes) or reduce it to 5 minutes using the quickShutdown parameter, which is now deprecated.

  • The openvpn module now includes a defaultClientCertExpirationDays parameter, allowing you to define the lifetime of client certificates.

Security

  • Known vulnerabilities have been addressed in the following modules: ingress-nginx, istio, prometheus, and local-path-provisioner.

The complete list of changes, including the updated components, is available in the changelog on GitHub.

1.69.17

Know before update

  • Users can now configure containerd to pull images from private registries, easing deployments in private environments.

Features

  • [docs] Add command to view all falco audit rules to docs.
  • [istio] Validating jwt token exp time less than 30 days
  • [node-manager] Added support for configuring private HTTP container registries in containerd.
    Users can now configure containerd to pull images from private registries, easing deployments in private environments.

Fixes

  • [docs] Removed port 22322 from setup docs.
  • [istio] Fix istiod CPU leak when the order of the ingressGateways array is changed
  • [istio] Reduce RAM for regenerate multicluster JWT token

1.69.16

Fixes

  • [control-plane-manager] audit-log-maxage value to 30 days
  • [deckhouse] Fix obsolete config alert stuck after deleting module config.

1.69.15

Know before update

  • Prevent important data loss when using wide retention intervals.

Fixes

  • [loki] Fix incorrect indices sort function used in disk-based retention.
    Prevent important data loss when using wide retention intervals.

1.69.14

Fixes

  • [deckhouse-controller] fixes default maintenance value which prevents stuck metric
  • [node-manager] fixed update for static clusters

1.69.13

Fixes

  • [istio] The alliance.ingressGateway.advertise option was fixed up.

1.69.12

Fixes

  • [dashboard] fix display workload metrics on dashboard

1.69.11

Know before update

  • In L2 mode, the Public IP address will not be marked as free in the pool after deleting the LoadBalancer Service. As a temporary fix, you can restart the MetalLB controller pods.
  • fix static e2e tests for fe/upgrade

Fixes

  • [ci] fix static e2e tests for fe/upgrade
    fix static e2e tests for fe/upgrade
  • [istio] proxy-buffer-size increased in kiali Ingress.
  • [metallb] Reverted the fix for handling deleted Services and revising the IP pools in L2 mode. The correct fix is under construction.
    In L2 mode, the Public IP address will not be marked as free in the pool after deleting the LoadBalancer Service. As a temporary fix, you can restart the MetalLB controller pods.

1.69.10

Fixes

  • [deckhouse-controller] Added validation according to RFC 1123 for module names added to ModuleSource.
  • [docs] Fix apiVersion for s3 storageclass

1.69.9

Fixes

  • [metallb] Fixed deprecated CRD addresspools.metallb.io deletion.
  • [network-gateway] Fix python path

1.69.8

Fixes

  • [cloud-provider-yandex] fix synchronizing TGs
  • [deckhouse] Gracefully restore deployed modules on HA installations.

1.69.7

Fixes

  • [docs] Fixed new layouts at documentation.
  • [kube-dns] Fixed release upgrade issue with removed d8-kube-dns-redirect service.

1.69.6

Fixes

  • [cni-cilium] Fixed race condition when deleting element from ipcache map during VM migration.
  • [docs] Fix broken links in the Storage overview.
  • [metallb] Fixed IP pool exhaustion on LoadBalancer deletion.
  • [node-manager] Revert cluster autoscaler

1.69.5

Features

  • [candi] Add rosa 12.6.

Fixes

  • [cloud-provider-yandex] fix WithNATInstance clusters converge
  • [control-plane-manager] Stale service account alert upd
  • [node-manager] Autoscaler - remove additional cordon node by mcm provider.

1.69.4

Features

  • [deckhouse-controller] sequential processing of module releases
  • [deckhouse-controller] sequential processing of module releases
  • [testing] Add e2e tests for autoscaler and enable it permanent for GCP.
  • [testing] Add e2e tests for autoscaler and enable it permanent for GCP

Fixes

  • [candi] Return support .cri.notManaged.criSocketPath.
  • [deckhouse] Restore only ready mpo.
  • [dhctl] <Disable caching metaconfig during converge and disable converge deckhouse manifests.
  • [dhctl] Fix incorrect behavior that fails sudoers preflight check if password contains whitespaces.
  • [docs] Fixed Dynamix layout at documentation.
  • [docs] Add support of x-doc-d8Editions array instead of x-doc-d8Revision field to JSON schema renderer.
  • [node-manager] Increase verbosity and revert versions and fix bug with unsupported mcm annotation
  • [upmeter] upmeter sa token rotation
  • [user-authn] Add separate ServiceAccount to basic-auth-proxy.
  • [user-authz] Exclude deckhouse clusterrolebindings from webhook.

1.69.3

Know before update

  • Ingress-nginx controller pods of v1.9 will be restated.

Fixes

  • [control-plane-manager] Stale service account alert fix.
  • [deckhouse] Limit module name.
  • [deckhouse] Fix mpo/source requeue.
  • [docs] Add info about debug GrafanaDashboardDefenition.
  • [extended-monitoring] Fix x509_certificate_exporter_resources deployment
  • [ingress-nginx] All necessary shared libraries are added to the container image.
    Ingress-nginx controller pods of v1.9 will be restated.
  • [kube-dns] Expanded pattern for stubZones to allow underscores (_) in domain names.
  • [user-authn] Fix secret generation on empty data field in the dex client app secret.

1.69.2

Know before update

  • the grafana deployment will be rollout restarted

Fixes

  • [deckhouse] Gracefully restore deployed modules on HA installations.
  • [prometheus] enable WAL for the grafana SQLite database to prevent locking errors, thus fixing in-dashboard alerting.
    the grafana deployment will be rollout restarted

1.69.1

Know before update

  • fixes the bug introduced in v1.69.0
  • release upgrade will be blocked on AWS-based clusters where SA doesn’t have DescribeAddressesAttribute and DescribeInstanceTopology roles. They are required for new Terraform AWS Provider version.

Features

  • [deckhouse] enable UnmetCloudConditions check
    release upgrade will be blocked on AWS-based clusters where SA doesn’t have DescribeAddressesAttribute and DescribeInstanceTopology roles. They are required for new Terraform AWS Provider version.
  • [deckhouse-controller] rename module flag managementState to maintenance.
  • [dhctl] add variable time range to request counter
  • [node-manager] enable UnmetCloudConditions check
  • [node-manager] UnmetCloudConditions requirement and alert

Fixes

  • [ci] Use fixed werf version.
  • [cloud-provider-vcd] Add a hook to set legacyMode based on the detected VCD API version
  • [deckhouse] Clear deployed release`s message.
  • [deckhouse] Delete deckhouse epslices.
  • [deckhouse-controller] Fix runtime handling for the global config.
  • [dhctl] fix bootstrap and abort config preparation
  • [docs] Fix d8 install commands.
  • [loki] fix storage capacity calculator hook for Loki
    fixes the bug introduced in v1.69.0
  • [operator-trivy] Add proxy env variables support to the trivy server.

Version 1.68

Important

  • After the update, the UID will change for all Grafana data sources created using the GrafanaAdditionalDatasource resource. If a data source was referenced by UID, that reference will no longer be valid.

Major changes

  • A new parameter, iamNodeRole, has been introduced for the AWS provider. It lets you specify the name of the IAM role to bind to all AWS instances of cluster nodes. This can come in handy if you need to grant additional permissions (for example, access to ECR, etc.).

  • Creating nodes of the CloudPermanent type now takes less time. Now, CloudPermanent nodes are created in parallel. Previously, they were created in parallel only within a single group.

  • Monitoring changes:
    • Support for monitoring certificates in secrets of the Opaque type has been added.
    • Support for monitoring images in Amazon ECR has been added.
    • A bug that could cause partial loss of metrics when Prometheus instances were restarted has been fixed.
  • When using a multi-cluster Istio configuration or federation, you can now explicitly specify the list of addresses used for inter-cluster requests. Previously, these addresses were determined automatically; however, in some configurations, they could not be resolved.

  • The DexAuthenticator resource now has a highAvailability parameter that controls high availability mode. In high availability mode, multiple replicas of the authenticator are launched. Previously, high availability mode of all authenticators was determined by a global parameter or by the user-authn module. All authenticators deployed by DKP now inherit the high availability mode of the corresponding module.

  • Node labels can now be added, removed, or modified using files stored on the node in the /var/lib/node_labels directory and its subdirectories. The full set of applied labels is stored in the node.deckhouse.io/last-applied-local-labels annotation.

  • Support for the Huawei Cloud provider has been added.

  • The new keepDeletedFilesOpenedFor parameter in the log-shipper module allows you to configure the period to keep the deleted log files open. This way, you can continue reading logs from deleted pods for some time if log storage is temporarily unavailable.

  • TLS encryption for log collectors (Elasticsearch, Vector, Loki, Splunk, Logstash, Socket, Kafka) can now be configured using secrets, rather than by storing certificates in the ClusterLogDestination resources. The secret must reside in the d8-log-shipper namespace and have the log-shipper.deckhouse.io/watch-secret: true label.

  • In the project status under the resources section, you can now see which project resources have been installed. Those resources are marked with installed: true.

  • A new parameter, --tf-resource-management-timeout, has been added to the installer. It controls the resource creation timeout in cloud environments. By default, the timeout is set to 10 minutes. This parameter applies only to the following clouds: AWS, Azure, GCP, OpenStack.

Security

Known vulnerabilities have been addressed in the following modules:

  • admission-policy-engine
  • chrony
  • cloud-provider-azure
  • cloud-provider-gcp
  • cloud-provider-openstack
  • cloud-provider-yandex
  • cloud-provider-zvirt
  • cni-cilium
  • control-plane-manager
  • extended-monitoring
  • descheduler
  • documentation
  • ingress-nginx
  • istio
  • loki
  • metallb
  • monitoring-kubernetes
  • monitoring-ping
  • node-manager
  • operator-trivy
  • pod-reloader
  • prometheus
  • prometheus-metrics-adapter
  • registrypackages
  • runtime-audit-engine
  • terraform-manager
  • user-authn
  • vertical-pod-autoscaler
  • static-routing-manager

Mandatory component restart

The following components will be restarted after updating DKP to 1.68:

  • Kubernetes Control Plane
  • Ingress controller
  • Prometheus, Grafana
  • admission-policy-engine
  • chrony
  • cloud-provider-azure
  • cloud-provider-gcp
  • cloud-provider-openstack
  • cloud-provider-yandex
  • cloud-provider-zvirt
  • cni-cilium
  • control-plane-manager
  • descheduler
  • documentation
  • extended-monitoring
  • ingress-nginx
  • istio
  • kube-state-metrics
  • log-shipper
  • loki
  • metallb
  • monitoring-kubernetes
  • monitoring-ping
  • node-manager
  • openvpn
  • operator-trivy
  • prometheus
  • prometheus-metrics-adapter
  • pod-reloader
  • registrypackages
  • runtime-audit-engine
  • service-with-healthchecks
  • static-routing-manager
  • terraform-manager
  • user-authn
  • vertical-pod-autoscaler

The complete list of changes, including the updated components, is available in the changelog on GitHub.

1.68.15

Fixes

  • [dhctl] Do not create a local-converger lock for commanderMode
  • [docs] Fixed the parameter generator for modules on Hugo.
  • [docs] This reverts commit cf9d7590da7c4b23c7eab10da37abc16397faf87.

1.68.14

Fixes

  • [deckhouse-controller] fix module release force behaviour
  • [deckhouse-controller] fix module release force behaviour
  • [runtime-audit-engine] Fix CrashLoopBack due to invalid config.

1.68.13

Know before update

  • Grafana deployment will be rollout restarted.

Fixes

  • [prometheus] Enables WAL for Grafana SQLite database to prevent locking errors, thus fixing in-dashboard alerting.
    Grafana deployment will be rollout restarted.

1.68.12

Fixes

  • [ci] Use fixed werf version.
  • [cloud-provider-zvirt] fix invalid zvirt csi template
  • [node-local-dns] Stale-dns-connection-cleaner file permissions and security context are fixed.

1.68.11

Fixes

  • [cloud-provider-zvirt] fix zvirt-csi-driver patching (token refresh fix patch)
  • [node-manager] partial rollback of changes in images cluster-autoscaler

1.68.10

Features

  • [deckhouse-controller] restore releases only by deckhouse-controller and in deployed state

Fixes

  • [deckhouse-controller] fix suspend release
  • [node-manager] handle_node_template hook ignore metallb label
  • [runtime-audit-engine] copy falcosecurity_falcosidekick_falco_events_total metric to falco_events for backward compatibility

1.68.9

Fixes

  • [node-manager] partial rollback of changes in images cluster-autoscaler

1.68.8

Features

  • [cert-manager] Bump cert-manager version up to 1.17.1. This fixes acme confirmation for cloudflare provider.

Fixes

  • [docs] FIx local docimentation build and run.
  • [docs] The information label with the editions has been removed from the global settings page.
  • [ingress-nginx] Fixed the display of ports in the examples in the CRDs.
  • [istio] Fixed the display of ports in the examples in the CRDs.

1.68.7

Know before update

  • Ingress-nginx pods will restart. The validation of the generated NGINX configuration during the validation of Ingress resources is disabled until finding a way of running validations safely.

Fixes

  • [ingress-nginx] latest CVE fixes backported.
    Ingress-nginx pods will restart. The validation of the generated NGINX configuration during the validation of Ingress resources is disabled until finding a way of running validations safely.

1.68.6

Fixes

  • [candi] Fix errors in bashible steps.
  • [deckhouse] Fix erroneous inclusions in 1.68
  • [deckhouse] Fix module config ignored.
  • [deckhouse] Fix checking automatic kubernetes version in managed clusters.
  • [deckhouse] Fix module config ignored.
  • [deckhouse] Explicitly set ports in d8-kube-dns deployment.
  • [deckhouse] Set EnabledByModuleConfig condition by default for modules without this condition.
  • [deckhouse] Fix module condition race.
  • [deckhouse-controller] Fix applying releases and their metrics.
  • [deckhouse-controller] Fix applying releases and their metrics.
  • [deckhouse-tools] d8-cli build fix
  • [docs] Corrected a typo in the NFSStorageClass name in the NFS data storage section.
  • [docs] Added SE and SE+ restrictions for Istio.
  • [log-shipper] Add delete parsed_data to final kafka transform render

1.68.5

Features

  • [prometheus] add grafana enable flag

Fixes

  • [candi] Fix disable_unattended_upgrades bashible step
  • [docs] Correct mistakes in the crds for the Storage section in Deckhouse Kubernetes Platform.
  • [docs] Update Deckhouse CLI installation guide
  • [docs] Fixed sidebar behaviour with links.
  • [docs] Correct mistakes in the documentation for the storage section in Deckhouse Kubernetes Platform.
  • [extended-monitoring] Minor certificate alerts improvements
  • [metallb] Add clearing of alert metric group D8MetallbObsoleteLayer2PoolsAreUsed
  • [operator-trivy] Updated info about databases with vulnerabilities.

1.68.4

Fixes

  • [candi] Fix add_node_users.sh search for sudoersd_path.
  • [deckhouse-controller] Fix DeckhouseRelease approval in the Manual update mode.
  • [dhctl] Escape special shell characters in sudo preflight check.
  • [docs] Fixed typos at VMware configuration parameters.
  • [upmeter] Add D8UpmeterDiskUsage alert.

1.68.3

Fixes

  • [deckhouse] Add initial module config validation.
  • [deckhouse-controller] Update addon-operator.
  • [dhctl] sudoPassword from the connection-config not used in dhctl CLI and dhctl server
  • [metallb] Fixed module startup error with empty Selector value in SVC.

1.68.2

Know before update

  • The links in the alerts will be correct when the documentation module is enabled.

Fixes

  • [admission-policy-engine] Fix wildcard apparmor profile.
  • [deckhouse-controller] fix deckhouse-release check loop
  • [deckhouse-tools] Fix tools links pointing to wrong path
  • [metallb] Fixed alerts for EE edition.
  • [monitoring-applications] Fix module dependencies. The monitoring-applications module was affected and disabled by the enabled script.
  • [node-local-dns] Fix module dependencies. The monitoring-applications module was affected and disabled by the enabled script.
  • [prometheus] Changes the links in alert’s template about alerts from url to relative links
    The links in the alerts will be correct when the documentation module is enabled.
  • [service-with-healthchecks] Fixed several issues.
  • [upmeter] Fixed metallb module probe.

1.68.1

Know before update

  • kubernetes-api will be restarted.
  • okmeter daemonset pods will be rollout restarted

Features

  • [candi] Add a new layout for HuaweiCloud provider with support for VPC peering connections.
  • [docs] Add new description info for conversions.

Fixes

  • [candi] Fix update_node_labels.sh to work on python2.
  • [candi] Reduce request-timeout in kubernetes-api to 60s.
    kubernetes-api will be restarted.
  • [ci] CI_COMMIT_TAG does not found in deckhouse controller
  • [ci] fix standalone-installer build
  • [ci] cloud providers CVE fixes, use single images for cloud-provider-zvirt and cloud-provider-yandex
  • [cloud-provider-vcd] fix CVE, bump ccm, use go 1.23, use werf images and src-artifact
  • [deckhouse] Fix stuck overridden status in ms.
  • [deckhouse] Fix module.yaml ignored when weight == 0.
  • [node-manager] fix autoscaler RBAC for zvirt and dynamix
  • [okmeter] drop service account fields from the okmeter daemonset manifest
    okmeter daemonset pods will be rollout restarted
  • [user-authn] Fix cronjob security context.