Glossary

Active-active balancing

A scheme of balancing incoming traffic in a multicluster in which the external load balancer distributes traffic across several active clusters at once, for example by geographic proximity or current load.

In the active-active scheme all clusters of the multicluster process traffic simultaneously. Traffic goes to the nearest or least loaded cluster, or follows other defined rules. Shared data is replicated between the clusters. When one cluster fails, traffic is automatically rerouted to the others. The balancing is configured by the cluster administrator as part of geo-reserving. The alternative scheme is active-standby.

For details, see Geo-reserving.

Active-standby balancing

A scheme of balancing incoming traffic in a multicluster in which the external load balancer routes all traffic to the currently active cluster, and the standby cluster receives traffic only when the active one becomes unavailable.

In the active-standby scheme the active cluster of the multicluster receives all traffic, and the standby cluster stays idle until the active one fails. Traffic failover to the standby cluster happens automatically when problems are detected. The balancing is configured by the cluster administrator as part of geo-reserving. The alternative scheme is active-active.

For details, see Geo-reserving.

Additional network

A software-defined network for pods and virtual machines on top of the main cluster network, provided by the sdn module. Additional networks are cluster networks or project networks.

A cluster network is publicly available in every project and is managed by the administrator. A project network is available within one namespace and is created by the user. Pods are connected to additional networks with a pod annotation, virtual machines with the list of networks in the machine specification. IP addresses on the additional interfaces are assigned by the IPAM of the sdn module.

For details, see Additional networks for use in application pods.

Additional project namespace

A user namespace created within a project in addition to the main one. It is used only to run the workload.

Additional namespaces let you divide the workload of a project logically and apply separate access policies to each part. Like the main project namespace, an additional project namespace is a user namespace.

Addon-operator queue

A task queue of the Deckhouse controller that processes built-in modules and modules from a source. Each queue is a separate pipeline with one worker that executes tasks from the head until they succeed.

The Deckhouse controller creates three kinds of queues:

• the main queue, which runs global hooks at startup, installs critical modules and configures or deletes modules;
• several parallel queues parallel_queue_0..N for module runs with dependencies taken into account;
• hook queues named in the configuration of a hook.

A task that fails and has no allowFailure blocks its queue, and it is restarted with an exponential delay starting at 5 seconds. Queues do not block each other. The queues can be inspected with d8 system queue list.

For details, see Queuing mechanism.

Admission policy

A rule that is applied to objects such as pods and Services when they are created or modified in the cluster, based on their manifests, and not while they run.

Admission policies formalize the parameters that are allowed or prohibited in object manifests. Deckhouse Platform implements them with the admission-policy-engine module and divides them into three categories: Pod Security Standards, operational policies (OperationPolicy) that validate parameters not related to security, such as allowed image registries, and security policies (SecurityPolicy) that validate security-related parameters, such as privileged mode or host namespaces.

The policies complement each other: if several apply to a namespace, an object must satisfy all of them. When a policy changes, existing pods are validated on the next restart, and Deckhouse Platform raises an alert for running objects that violate a policy. SecurityPolicyException allows fine-grained exceptions for individual pods or containers.

For details, see Security policies.

Advanced load balancer

A load balancer of the service-with-healthchecks module that runs active probes for individual TCP ports instead of relying on container readiness probes, so that each balancer serving the same pod works independently.

The advanced load balancer is built on ServiceWithHealthchecks objects. It consists of a controller that runs on the master nodes and manages these objects, and agents on every node that run the probes and publish only verified addresses. The standard load balancer, a Kubernetes Service, remains available for workloads that need no separate probes.

For details, see Intra-cluster communication.

Air-gapped cluster

A cluster without access to an external container image registry. The only source of images for it is the in-cluster cache.

In the Managed registry mode the cluster is considered isolated when the primary.upstream parameter is not set. The in-cluster cache is then filled with the d8 mirror push command and serves as the registry of Deckhouse Platform components.

For details, see Managing the registry in DP-managed clusters.

ALB (application load balancing)

Load balancing at the application level. HTTP(S) headers, paths and domains are analysed, SSL termination and content-based routing are supported.

Deckhouse Platform provides three ALB implementations: Ingress NGINX Controller (the ingress-nginx module), Kubernetes Gateway API (the alb module) and Istio (the istio module). Balancing at the network level is NLB.

For the criteria of choosing an implementation, see Incoming traffic balancing.

Alert

A notification of the Deckhouse Platform monitoring system about an event in the cluster or its components.

The standard delivery includes a set of basic alerts that cover the state of the cluster and its components. Custom alerts can be added. Alerts are sent to external systems through Alertmanager: by SMTP, to PagerDuty, Slack, Telegram or by webhook.

The list of all alerts of the monitoring system is given in the Alerts reference.

Alert notification channel

A delivery channel through which the monitoring system of Deckhouse Platform sends alerts to recipients, such as Telegram, email or a webhook.

Notification channels, notification policies and silences are configured in the observability module. Alerts are delivered by Alertmanager, which in Deckhouse Platform supports multitenancy: alerts are divided into system (cluster) and project ones, and channels and delivery policies can be configured separately in every project.

For details, see Observability module.

Alert severity

The Severity (S) parameter of an alert that indicates its level of criticality, from S1 for a critical failure that requires immediate action to S4–S9 for a low-level problem that does not impair overall performance.

Every alert description contains the Severity parameter with a value from S1 to S9:

• S1 is the maximum level, a critical failure that requires immediate action;
• S2 is a high level close to the maximum, a possible accident that requires rapid response;
• S3 is a medium level, a potentially serious problem that requires verification;
• S4 to S9 are low levels, a problem that does not impair overall performance.

The severity of every alert is shown in the list of alerts.

Application endpoints

The addresses of the Ingresses of an application that Deckhouse Platform publishes in the status.urls field of the Application. The web interface shows them as links to the application.

To publish an endpoint, the package adds the packages.deckhouse.io/application-endpoint-description annotation to the Ingress: its value is the endpoint description, "true" adds the endpoint without a description, "false" excludes the Ingress. Deckhouse Platform builds a URL for every host and path pair of the Ingress rules, with the https scheme if the host is listed in spec.tls, and updates status.urls of the application instance after every successful apply of the templates.

For details, see Templates.

Application instance

An application installed from a Marketplace package as an Application object in a namespace. The same package can be installed several times with separate names and settings, and the instance name is a part of the names of all application objects.

Each instance is fully independent: two Redis instances, one for caching and one for sessions, can live in the same namespace. Deckhouse Platform processes every instance in a separate queue. The objects of an instance are named d8a-<INSTANCE_NAME>-<SUFFIX>, so the instance name is limited to 24 characters, and every rendered object carries the packages.deckhouse.io/instance label.

For details, see Installing and managing applications.

Application reconciliation run

The configuration, hook initialization and apply steps that Deckhouse Platform performs for an application instance. A new run starts when the settings change.

Deckhouse Platform processes each application instance in a separate queue: it downloads the package bundle, loads it, waits until the package requirements are met, validates and applies the settings, enables the hook bindings and runs the onStartup hooks, then runs beforeHelm hooks, renders the templates and applies them with Nelm, and runs afterHelm hooks. The last three steps form a reconciliation run. The Helm release is upgraded only when needed: on the first installation, if the rendered manifests have changed, or if some resources are missing in the cluster.

A new run starts when spec.settings or spec.maintenance of the Application changes or when a schedule or Kubernetes hook changes values.

For details, see Lifecycle and debugging.

Application suspension

The state of an installed application whose package requirements have stopped being met, for example a mandatory module was disabled. Deckhouse Platform uninstalls the Helm release but keeps the Application in the Suspended state.

Deckhouse Platform checks the package requirements all the time, not only on admission. When they stop being met, it runs the beforeDeleteHelm hooks, uninstalls the Helm release, which deletes the resources of the application instance, and runs the afterDeleteHelm hooks. The orphan resources are not deleted. When the requirements are met again, the application is installed again with the same settings.

The resources are deleted together with the Helm release, including PersistentVolumeClaims. To keep the data, add the helm.sh/resource-policy: keep annotation to them.

For details, see Lifecycle and debugging.

Arbiter node

A node that holds only etcd, without the other control plane components, to provide the etcd quorum in HA mode with two master nodes.

HA mode with two master nodes and an arbiter node meets the high availability requirements with limited resources. The arbiter node must have the label node.deckhouse.io/etcd-arbiter: "" and a taint that keeps user workloads off it. It needs at least 2 CPU cores, 4 GB of RAM and 8 GB of disk space for etcd, and the same network latency as the master nodes.

For details, see Managing HA mode.

Automatic certificate issuance

Obtaining TLS certificates automatically with the cert-manager module, either through a Certificate object that references a ClusterIssuer or through the tls-acme annotation on an Ingress.

With a Certificate object, cert-manager runs the domain ownership challenge by the method defined in the ClusterIssuer, for example HTTP-01 or DNS-01, creates a temporary Ingress for the challenge if needed and stores the certificate in the Secret. The tls-acme annotation on an Ingress requests a certificate without a separate Certificate object. Deckhouse Platform supports certificates from Let’s Encrypt, HashiCorp Vault and Venafi, self-signed certificates, automatic renewal and expiration monitoring.

For details, see Configuration of using TLS certificates.

Availability probe

A check that the upmeter module runs regularly to calculate the availability of Deckhouse Platform components. Probes are grouped, and groups or individual probes can be disabled.

The upmeter-agent component runs on master nodes and executes probe groups: control-plane (API server availability and controller health), deckhouse (cluster health and the deckhouse controller), extensions, load-balancing, monitoring-and-autoscaling, nginx (every Ingress controller has a Ready pod), nodegroups and synthetic (network connectivity between nodes). For example, the control-plane group creates a namespace, a pod and a StatefulSet and checks that Kubernetes processes them.

Probe results are stored by the upmeter controller excluding the downtime intervals defined in Downtime objects, shown on the status page and the availability dashboard, and used for SLA monitoring. Probes or probe groups are disabled with the disabledProbes parameter of the module.

For details, see Upmeter module.

Availability zone

The basic unit of a disaster-resilient architecture: a logical group of one or several data centers located within a few kilometers of each other and viewed as an isolated unit. Availability zones are part of regions.

An availability zone (AZ) is physically distributed infrastructure that can be treated as a virtual data center. The data centers of one zone are separated far enough (usually 10 km or less) to survive local incidents such as floods or fires, keep low network latency between each other, have independent power supply and use different network providers and routes. Several zones form a region.

Deckhouse Platform lets you distribute cluster nodes across availability zones (Multi-AZ) so that applications keep running when one zone of a data center or cloud becomes unavailable. With most cloud providers the zone distribution happens automatically. Outside clouds the administrator labels nodes with their zone. This is one form of geo-distribution.

For details, see Geo-distribution.

bashible

The service that runs on every cluster node and executes the bash scripts, also called bashible, that configure the node according to its node group.

The scripts are Go templates grouped into bundles. An individual script is called a step. They are written so that they can be re-run safely. The service re-runs the steps when the configuration checksum of the node group changes. The steps are stored on the node in /var/lib/bashible/bundle_steps/. NodeGroupConfiguration adds custom steps to the bashible configuration of a group.

bashible is part of the Cluster & Infrastructure subsystem. It runs at the operating system level as a system service and is not part of the node-manager module.

For details, see Bashible.

Bashible bundle

A collection of bashible scripts. Four bundles exist, bashible installation scripts, bootstrap scripts for the first node, node configuration scripts of a cloud provider and common scripts. An individual script of a bundle is a step.

bashible scripts are Go templates, so a bundle is rendered for a specific node group, and every step must be safe to run again after a failure. Rendered bundles are served by bashible-api-server. The steps cover the main node configuration stages: access for NodeUser, CA certificates, binaries in /opt/deckhouse/bin, packages, containerd, kubernetes-api-proxy, kubelet and the bashible service itself.

For details, see Bashible.

bashible-api-server

A Kubernetes extension API server on the master nodes that generates bashible scripts from templates stored in custom resources.

Storing every variant of the scripts for all supported operating systems in etcd is not feasible because of the key size limit and the load, so kube-apiserver forwards requests for resources with bashible bundles to bashible-api-server and returns the generated result. It serves the second phase of the bootstrap script, the bashible.sh script and the rendered bundle of a node group.

For details, see Bashible.

Bastion host

A host through which the master node is reached over SSH when the cluster nodes have no public IP addresses.

A bastion host is needed in layouts where nodes are created without public IP addresses. The installer connects through it when run with the --ssh-bastion-host, --ssh-bastion-user and, if needed, --ssh-bastion-port parameters.

Bootstrap script

The bootstrap.sh script that Deckhouse Platform generates for every NodeGroup to connect a server to the cluster as a node of that group.

The script is stored in the manual-bootstrap-for-<NODE-GROUP> secret in the d8-cloud-instance-manager namespace and is also available in the Deckhouse web interface. For manual connection of a static node, copy the script to the server and run it as root. CAPS runs it over SSH itself. The script installs and configures the operating system settings, containerd, kubelet and the kubernetes-api-proxy component, then registers the node in the cluster.

For details, see Adding and managing bare-metal nodes.

bpfLB

The BPF Load Balancer mode of the cni-cilium module that processes external traffic for Services of the NodePort and LoadBalancer types. The mode is set by the bpfLBMode parameter.

In the SNAT mode, the BPF load balancer fully replicates the logic of kube-proxy: the source IP address of packets sent to pods on other nodes is replaced with the InternalIP of the node. In the DSR mode, incoming traffic passes through the load balancer and outgoing traffic goes directly from the node of the target pod, which increases the overall throughput.

For details, see BpfLB operation modes.

Built-in module

A module included in Deckhouse Platform. Its release cycle is tied to the release cycle of the platform.

Built-in modules are delivered together with a Deckhouse Platform release and are updated with it. Which of them are enabled by default depends on the module bundle and the edition. The other kind of modules are modules from a source, whose releases do not depend on the platform releases.

For details, see Deckhouse Platform module development.

Caching DNS server

A DNS server that the node-local-dns module runs on every cluster node. It caches DNS responses and removes the network round trip to the cluster DNS for repeated requests.

On each node the module configures an interface with the IP address of the kube-dns service, starts a caching CoreDNS on that address and adds an iptables rule that redirects DNS traffic to it while its socket is open. Otherwise the standard routing through the cluster DNS is used. External requests that are not cached yet are still resolved through the chain of internal zones first, but under load the cache alone improves resolution performance noticeably.

For details, see Caching DNS server in a cluster.

Canary deployment

An application deployment strategy that rolls out a new version to a small portion of traffic first and shifts more traffic to it as confidence grows, with a quick rollback to the old version if issues arise.

In Deckhouse Platform a canary deployment is implemented with the ingress-nginx module through Ingress annotations or with the istio module through VirtualService and DestinationRule objects (the Istio way is recommended). The new version runs as a separate Deployment and Service next to the stable one.

For details, see Configuring Canary deployment.

Chaos engineering

Deliberate or random disruption of cluster components to test its resilience. In Deckhouse Platform it is enabled per node group with the spec.chaos parameter, only for CloudEphemeral nodes.

The spec.chaos parameter of a node group of CloudEphemeral nodes sets the mode and the period. In DrainAndDelete mode Chaos Monkey drains and deletes a node of the group once per period. The Disabled mode skips the group. The mode can also be enabled in the web interface in the node group settings.

For details, see Chaos engineering.

CIS compliance check

A check that Deckhouse Platform runs automatically in every cluster against the CIS Kubernetes Benchmark recommendations. The results are available as reports and on the Security / CIS Kubernetes Benchmark dashboard.

Deckhouse Platform follows the CIS Kubernetes Benchmark both at the level of individual components and for the platform as a whole. Access to the reports, like access to vulnerability scanning results, is granted through the role model of the operator-trivy module.

For details, see Configuration of container image vulnerability scanning.

Cloud cluster

A cluster with the clusterType: Cloud. Deckhouse Platform has access to the API of the cloud infrastructure and creates and manages the virtual machines of the nodes.

The installer creates all the required resources, such as virtual machines and networks, from the <PROVIDER>ClusterConfiguration of the cloud provider. The master nodes of a cloud cluster are CloudPermanent nodes, worker nodes are usually CloudEphemeral nodes.

For details, see Platform installation.

Cloud provider

A cloud platform or virtualization system that Deckhouse Platform integrates with through a cloud-provider-* module to create virtual machines, connect them to the cluster as nodes and create base infrastructure resources.

The workflow is the same for all providers, only the preparation steps, such as creating a service account, and the configuration files differ. The integration enables CloudEphemeral nodes, which Deckhouse Platform creates and deletes through the provider API, and the choice of a layout.

For details, see Integration with IaaS.

CloudEphemeral node

A temporary cloud node that Deckhouse Platform orders, creates and deletes automatically through the cloud provider API. It is described by a NodeGroup with the CloudEphemeral node type.

The parameters of the virtual machines are taken from an instance class, the number of nodes and their zones from the cloudInstances section of the node group. The nodes are created by Machine Controller Manager or Cluster API, which are part of the node-manager module.

CloudEphemeral groups support node group autoscaling, standby nodes and chaos engineering.

For details, see Adding and managing cloud nodes.

CloudPermanent node

A permanent cloud node whose configuration is taken from the provider cluster configuration rather than from a NodeGroup. The master nodes of a cloud cluster are CloudPermanent nodes.

CloudPermanent nodes are described in the nodeGroups section of the <PROVIDER>ClusterConfiguration structure, and their number is set by replicas. The nodes are managed by Terraform built into Deckhouse Platform, and configuration changes are applied by running the installer with dhctl converge.

For details, see Adding and managing cloud nodes.

CloudStatic node

A node created manually or by external tools in the cloud for which the provider integration is configured, and connected to the cluster with the bootstrap script. CSI runs on it, and cloud-controller-manager manages its Node object.

The Node object of a CloudStatic node automatically receives the zone and region from the provider data. If the machine is deleted from the cloud, its Node object is deleted from the cluster. Like a static node, a CloudStatic node does not support automatic scaling and updates.

For details, see Adding and managing cloud nodes.

Cluster

A group of interconnected nodes managed by Deckhouse Platform that provides a single environment for deploying, scaling and operating workloads.

A cluster consists of the system, which keeps the cluster running, and the workload, which runs in the cluster.

Cluster API Provider Static (CAPS)

The mechanism that connects, configures and, when needed, cleans up static nodes automatically, based on StaticInstance and SSHCredentials objects.

CAPS is enabled by the staticInstances section of a node group with the Static type: count sets how many nodes the group must have, labelSelector selects the StaticInstance objects to use. CAPS keeps the number of nodes equal to count: it connects to a server over SSH, runs the bootstrap script and adds the node to the cluster, and when a node is removed it cleans the server up and returns its StaticInstance to the Pending state for reuse.

For details, see Adding and managing bare-metal nodes.

Cluster configuration (ClusterConfiguration)

The structure that holds the general parameters of the cluster, such as the Kubernetes version, network settings, CRI parameters and the cluster type.

The cluster configuration is defined at installation and edited in a running cluster with the d8 system edit cluster-configuration command. After the change is saved, Deckhouse Platform reconciles the cluster state with the new configuration. Some of its parameters are protected and cannot be changed in a running cluster by default.

The clusterType parameter defines whether the cluster is static or cloud.

For details, see Deckhouse Platform configuration.

Cluster managed by Deckhouse Platform

A cluster in which Deckhouse Platform manages both the control plane and the nodes. The opposite is a Managed Kubernetes cluster, where the control plane is managed by a cloud provider.

Deckhouse Platform can also be installed into an existing Managed Kubernetes cluster, such as GKE, EKS or AKS. In this case Deckhouse Platform does not manage the control plane, and some features are limited by the API of the cloud provider: for example, node management and the registry module are available only in clusters fully managed by Deckhouse Platform.

For details, see Platform installation.

Cluster network

An additional software-defined network that is publicly available in every project of the cluster and is configured and managed by the administrator.

Examples of a cluster network are a public WAN network or a shared network for traffic exchange between projects. The administrator creates it with a ClusterNetwork object of the sdn module and, if needed, enables IPAM for it with a cluster-wide address pool. To use such a network in application pods, contact the cluster administrator. The other kind of additional network is the project network.

For details, see Software-defined networking (SDN).

ColdStandby

A mechanism that returns a virtual machine to service on another node after the failure of the node it was running on.

The mechanism requires the runPolicy of the virtual machine to be AlwaysOnUnlessStoppedManually or AlwaysOn, and Fencing to be enabled on the nodes that run virtual machines. Without Fencing, an unavailable machine does not move: it stays on the failed node and resumes together with it.

For details, see Virtual machine fault tolerance and balancing.

Conditional module dependency

A dependency on another module that applies only if that module is enabled. The module works on its own, but if the other module is present its version must match the constraint.

A conditional dependency is a kind of module dependency and is available in Deckhouse Platform starting with version 1.73. It may affect the ability to enable, disable and update both modules: the dependent one and the one it depends on. A Marketplace package lists such modules in requirements.modules.conditional.

For details, see Dependencies of the Deckhouse Platform modules.

Consistent snapshot

A snapshot of a virtual machine disk that captures a coherent and integral state of its data. It is created when the disk is not attached to any machine, the machine is powered off, or the guest OS agent freezes the file system while the snapshot is taken.

Whether a consistent snapshot is required is set by the requiredConsistency parameter, which is true by default. With a running machine the file system is frozen by the guest OS agent. Its hook scripts can also prepare applications for the snapshot without stopping services.

An inconsistent snapshot is created when the machine is running and the agent is not installed or not running. Recovery from such a snapshot can lead to file system integrity problems.

For details, see Snapshots of disks and virtual machines.

Control plane

The Kubernetes components that manage the cluster, such as kube-apiserver and etcd. In Deckhouse Platform they run on master nodes, and in HA mode with two master nodes the etcd quorum is provided by an arbiter node.

Deckhouse Platform manages the certificates of the control plane, generates the configuration and manifests of its components, upgrades and downgrades their versions, manages the etcd cluster and its members, runs etcd defragmentation and maintains the kubeconfig files. The control-plane-manager module runs on all master nodes.

The control plane works in the single-master or multi-master mode. In HA mode with two master nodes, the third etcd member runs on an arbiter node: it holds only etcd, without the other control plane components, so that the etcd cluster keeps its quorum.

For details, see General management and configuration of the control plane.

Core fraction

The guaranteed share of the power of each virtual core of a virtual machine, set by the coreFraction parameter. Without it every virtual core gets 100% of a physical one.

A machine with cores: 2 and coreFraction: 20% is guaranteed 0.4 cores in total regardless of the node load and can take both cores in full when the node has free resources. Such a reserve keeps more machines on a node than it has physical cores. This is how CPU oversubscription works. The guaranteed share is taken into account when a node is selected, so a machine does not start where the node cannot provide the guarantees to all machines placed on it.

An administrator can restrict the allowed values in the sizing policy of the VM class. In commercial editions coreFraction: Auto lets Deckhouse Platform pick the share by the actual consumption of the machine.

For details, see Virtual machine CPU and memory.

CPU oversubscription

Giving the virtual machines on a node more virtual cores than the node physically has. The degree of oversubscription is controlled by the core fraction guaranteed to a virtual machine.

The coreFraction parameter of a virtual machine defines the share of a core’s capacity guaranteed to it. The allowed values are set in the sizing policy of the class. With coreFraction: 20%, a machine always gets a fifth of a core and can take a whole core when the node has spare resources. When placing a machine, Deckhouse Platform sums the guaranteed shares of all machines on the node, and if the sum exceeds the number of physical cores, the machine does not start on that node.

For details, see CPU oversubscription for virtual machines.

Critical module

A module marked with critical set to true in module.yaml. It starts during the initial cluster bootstrap, before the cluster is considered fully ready. Other modules start only after that.

The Deckhouse controller puts the tasks of critical modules into the main queue in ascending order of module weight. Non-critical modules are processed in the parallel queues without regard to weight. If critical modules can be processed in parallel, the controller runs them as a batch and waits for completion before continuing the main queue.

For details, see Queuing mechanism.

CSI driver

A plugin that manages persistent storage volumes in Deckhouse Platform. It is part of the cloud-provider-* and csi-* modules, and all implementations share one reference architecture.

Container Storage Interface (CSI) is a standard interface that unifies access to storage systems. Every cloud provider and external storage system has its own implementation of the specification. The implementations differ in the supported capabilities and components, but not in architecture.

For details, see CSI driver.

Dashboard

A collection of charts and tables with data about the state of the cluster and the applications running in it. Deckhouse Platform ships preinstalled dashboards and lets users create custom ones.

Dashboards show CPU usage, memory consumption, disk and network activity, and the status of pods, controllers, nodes and namespaces. They are available in the Deckhouse web UI under “Monitoring” → “Dashboards”.

Preinstalled dashboards are shipped with Deckhouse Platform and cannot be edited. Custom dashboards are created with an ObservabilityDashboard object of the observability module at the namespace level with access control, or with a GrafanaDashboardDefinition object at the cluster level, which is a legacy approach.

For details, see Monitoring dashboards.

Deckhouse CLI (d8)

The command-line interface for working with the products of the Deckhouse ecosystem. It combines commands for Kubernetes, Deckhouse Platform administration, application delivery, image copying, backups, diagnostics, virtualization and other tasks.

Starting from release 1.59, d8 is installed on all Deckhouse Platform cluster nodes. It can also be installed on an administrator machine with trdl (an open-source tool for securely delivering and updating binary files) or from the executable file. Commands are grouped by purpose: d8 k is equivalent to kubectl, d8 system works with platform settings, modules, queues and packages, d8 mirror copies platform images, d8 v manages virtual machines, d8 backup creates backups.

For details, see Deckhouse CLI overview and installation.

Deckhouse controller

The controller of the deckhouse module that implements platform management operations and coordinates them through the queueing mechanism. The deckhouse module is the only module without which Deckhouse Platform cannot run.

The Deckhouse controller manages modules: it runs their hooks, applies module settings and installs module releases. Tasks are processed in the addon-operator queues. In HA mode several controller instances run and elect a leader through the deckhouse-leader-election Lease. The leader performs all management operations.

The controller can run in standard mode or in hook isolation mode, in which shell hooks and module enable scripts run in a chroot environment with a limited set of mounted directories.

For details, see Deckhouse module.

Deckhouse Platform API

The interface through which users and Deckhouse Platform components interact with the cluster. It is fully compatible with the Kubernetes API.

All operations in Deckhouse Platform are performed through the API, which keeps the interaction uniform. The API of every module, its custom resources and parameters, are described in the module documentation.

Deckhouse Platform component registry

The container image registry from which the cluster pulls the images of Deckhouse Platform components.

The registry is specified during cluster bootstrap in the deckhouse parameter of InitConfiguration. How its configuration is managed in a running cluster depends on how the cluster is managed: in a cluster fully managed by Deckhouse Platform, the registry module is used and the registry management mode is set in its ModuleConfig. In a Managed Kubernetes cluster, the helper change-registry is used.

For details, see DP components registry.

Deckhouse Platform image bundle

An archive of Deckhouse Platform container images produced by d8 mirror pull and loaded into a private container registry with d8 mirror push. It is the way to deliver platform images into a private environment.

The bundle is downloaded from the public Deckhouse Platform registry with a license key, usually on the bastion host, and brought into the perimeter by any permitted means, for example on removable media. Its size after d8 mirror push guides the disk capacity of the private registry. The d8 mirror commands are not available in the Community Edition and Basic Edition editions.

For details, see Installing DP in a private environment.

Deckhouse Platform release

Any published version of Deckhouse Platform. Releases are distributed through release channels with defined delays and are of two types, a patch version and a minor version.

A patch version includes bug fixes, a minor version includes new features. Each new release first appears on the Alpha release channel and then moves to the more stable channels. In the cluster a release is represented by a DeckhouseRelease object, which is applied according to the update mode and update windows. Minor releases cannot be skipped, because they contain migrations that must be applied in order.

For details, see Updating.

Deckhouse web UI

The web interface of Deckhouse Platform for cluster management, including monitoring, parameter configuration, node and module management, security and network configuration.

Most operations available from the command line through Deckhouse CLI or kubectl can also be performed in the web UI. It opens at console.<CLUSTER_NAME_TEMPLATE>, where the template is the DNS name template of the cluster, and contains the sections Deckhouse, node management, multitenancy, access, network, storage, security, monitoring and logging.

For details, see Deckhouse Platform web UI.

Deletion confirmation

A confirmation that the user must give in the web interface before an application is deleted or a module is disabled. It is declared in the disable section of package.yaml or module.yaml together with the message shown to the user.

For a Marketplace package the disable section sets confirmation: true and localized messages, for example a warning that deleting the application deletes the data stored in its volumes. Deckhouse Platform publishes the section in the status.packageMetadata.disableOptions field of the package version, and the Deckhouse web UI shows the message in the language of the user. The confirmation is not checked when an Application is deleted through the Kubernetes API.

For a module the disable.confirmation and disable.message parameters of module.yaml work the same way when the module is disabled.

For details, see Application development.

Deployment option

A cluster configuration that defines the number of nodes, the distribution of roles between them, and whether the cluster tolerates a node failure.

The deployment option is chosen before the installation. Examples of options: a single-node cluster, a single-master cluster, a high-availability cluster with three master nodes, and a cluster with role separation, where system, frontend and monitoring nodes are dedicated to the platform components and incoming traffic. The node resource requirements depend on the deployment option.

For details, see the Deployment options section of the “System requirements” page.

Device passthrough

Giving a virtual machine a device of the node it runs on, such as a USB device, a GPU or a PCI device.

Each device type has its own node requirements and its own preparation order.

For details, see Virtualization.

Disaster resilience

The ability of infrastructure based on Deckhouse Platform to remain operational in the event of large-scale failures. It is achieved through distributed deployment, automatic traffic switching and replication of critical components.

Deckhouse Platform implements two approaches to disaster resilience. Geo-distribution spreads the components of one cluster across several availability zones or regions. Geo-reserving combines several independent clusters into a multicluster and redirects traffic to another cluster when one becomes unavailable.

Both approaches require network connectivity between nodes and regions, traffic balancing and a suitable storage organization. The choice of an architecture depends on the applications and the constraints of the infrastructure. Within one cluster, the redundancy of critical components is provided by the HA mode.

For details, see Disaster resilience.

Disruptive update

A node update that requires node downtime and may briefly interrupt system components, for example upgrading containerd or kubelet across several versions. The approval mode is set by the disruptions.approvalMode parameter of the NodeGroup.

Regular node updates are always applied automatically and cause no downtime or reboot. The disruptions section governs only disruptive updates, such as a kernel or containerd update or a significant kubelet version change. Only one node per group is updated at a time.

Three approval modes are available in a node group. In Manual mode the NodeRequiresDisruptionApprovalForUpdate alert is raised, and the update starts after the update.node.deckhouse.io/disruption-approved= annotation is added to the node. The node is not drained automatically. In Automatic mode the node is drained and updated without confirmation. In RollingUpdate mode a new node with the updated settings is created and the old one is removed. This mode applies to cloud nodes only.

Use Manual mode for the groups of master nodes.

For details, see Node management basics.

DNS name template

The cluster DNS name template from the modules.publicDomainTemplate parameter of the global ModuleConfig. Deckhouse Platform modules substitute a reserved name for the %s placeholder to build the DNS names of their service web interfaces.

If the template is not set, the modules do not create Ingress resources. The domain in the template must not match or be a subdomain of the cluster domain from clusterDomain. With a wildcard DNS record one record covers all names. With a non-wildcard template, for example %s-kube.company.my, an A or CNAME record is needed for each reserved name, pointing to the public IP address of the Ingress controller. The template is a part of the global settings.

For details, see List of reserved Deckhouse Platform DNS names.

DSR (Direct Server Return)

A traffic processing method in which incoming packets pass through the load balancer and outgoing packets go directly from the servers that handle the request.

Responses are usually much larger than requests, so DSR unloads the balancer and increases the overall throughput. In Deckhouse Platform, DSR is one of the bpfLB modes of the cni-cilium module and is used instead of SNAT.

For details, see BpfLB operation modes.

DVCR

The internal container image storage in which Deckhouse Platform keeps virtual machine images. It resides on a persistent volume of the cluster.

Images travel from DVCR to virtual machine disks, so the size of the volume determines how many images fit into the cluster. The volume size and storage class are set in the dvcr.storage block of the virtualization ModuleConfig. After the volume is created, its size cannot be reduced and its storage class cannot be changed. Stale data of deleted images and disks is removed by garbage collection on a schedule, daily at 02:00 by default.

For details, see Virtual machine image storage.

Edition

A Deckhouse Platform distribution, such as Open, Core, Ultimate, or CSE Pro, that defines the set of available modules and features.

The edition is selected when the cluster is installed. A running cluster can be switched to another edition.

For details, see Switching between DP editions.

Egress gateway

Centralized management and control of outgoing traffic from the cluster, including encryption, routing and monitoring.

The egress gateway is provided by the cni-cilium module and is configured with two custom resources. EgressGateway describes a group of nodes, one of which is selected as the active gateway while the others stay in standby mode. EgressGatewayPolicy describes which traffic of cluster pods is routed through that gateway.

Two modes are available: the basic mode with pre-configured IP addresses on the egress nodes and the Virtual IP mode, in which additional IP addresses are assigned to nodes dynamically. For details, see Egress Gateway configuration.

End-to-end authentication

Authentication implemented in Deckhouse Platform so that a user is verified when accessing any interface of the platform or any cluster resource. Users can also enable it for their own applications.

Depending on the configuration, authentication uses either the internal user database (static users) or external authentication providers such as LDAP, GitLab or GitHub. An external provider lets users keep their existing credentials and use them in several Deckhouse Platform clusters. For a cluster user or an application developer the interface and the authentication methods for applications stay the same whichever way the administrator configured authentication.

For details, see Configuring authentication for applications.

Extended monitoring

A monitoring mode of the extended-monitoring module that adds monitoring of secrets and TLS certificate expiration, collection of cluster events as metrics, checks of image availability in the registry and monitoring of objects in labelled namespaces.

Objects are monitored in the namespaces that carry the label extended-monitoring.deckhouse.io/enabled="". Image availability is checked for the images used by Deployments, StatefulSets, DaemonSets and CronJobs.

For details, see Monitoring in Deckhouse Platform.

External storage

Storage systems outside the cluster nodes, Ceph, NFS, S3, SCSI and vendor storage systems, connected to Deckhouse Platform through csi-* modules, as opposed to software-defined storage.

Deckhouse Platform ships ready-to-use integrations with distributed Ceph storage, HPE, Huawei and TATLIN.UNIFIED (Yadro) storage systems, NFS, S3 object storage and SCSI-based storage. Each integration is a module with a CSI driver. The second group of storage types is software-defined storage on the disks of cluster nodes.

For details, see Storage.

Federation

An inter-cluster communication model that connects independent clusters. Each cluster keeps its own namespaces and services, and access between clusters is configured explicitly.

In a federation, the administrator defines which services are accessible from outside the cluster. The model is implemented with Istio through the istio module and is built on the service mesh. The model for tightly connected clusters is multicluster.

For details, see Inter-cluster cooperation.

Fencing

A mechanism that isolates a node that has lost connectivity with the cluster. The node is shut down and, if needed, its Node object is deleted, so that no workload keeps running on it in an undefined state.

Fencing works in two stages: the node is first marked as suspicious at the gossip protocol level, then isolated. It is enabled per node group with the spec.fencing.mode parameter of the NodeGroup. Fencing is required for ColdStandby: without it an unavailable virtual machine stays on the failed node and resumes together with it.

For details, see Virtual machine reliability.

Frontend node

A node that balances incoming traffic. Ingress controllers run on frontend nodes.

A cluster needs more than one frontend node, and the nodes must handle the traffic when at least one of them fails. With two frontend nodes each must carry the full load, with three each must withstand a load increase of at least 150%. Frontend nodes usually form the front node group and run the Ingress controllers.

For details, see Recommendations for configuring cluster nodes.

Geo-distribution

An approach to disaster resilience in which the components of a cluster are distributed across several availability zones (Multi-AZ) or regions (Multi-Region) so that the cluster keeps running when a zone or region fails.

Geo-distribution reduces the impact of infrastructure failures on application availability. Distributing nodes across availability zones protects applications from the failure of one zone in a data center or cloud. Distributing them across regions lets applications survive the outage of a whole data center or cloud provider, provided that the application can work independently in each region.

The platform administrator is responsible for the network connectivity between cluster nodes, for the balancing of incoming traffic by the cloud provider or an internal load balancer, and for the organization of storage. Geo-distribution is one of the two approaches to disaster resilience in Deckhouse Platform. The other is geo-reserving.

For details, see Geo-distribution.

Geo-reserving

A method of increasing fault tolerance in which several independent Kubernetes clusters in different geographic locations are combined into a multicluster, and traffic is automatically redirected to a cluster that has an available copy of the application.

Geo-reserving requires one Deckhouse Platform cluster per region, connected with a declarative API into a multicluster: either through the built-in service mesh based on Istio or through the networking features of the platform based on Cilium. The application must be able to run in parallel in different regions, and a stable network channel must exist between the regions. A properly configured system keeps the application running even when a whole region or cloud becomes unavailable.

Incoming traffic in a multicluster is balanced by an external load balancer in one of two schemes, active-active or active-standby. The balancing and the storage organization are configured by the cluster administrator. Geo-reserving is one of the two approaches to disaster resilience. The other is geo-distribution.

For details, see Geo-reserving.

Global settings

Deckhouse Platform settings that apply to the whole cluster. They are stored in the ModuleConfig named global.

Global settings can be thought of as a special global module that cannot be disabled. The settings of other modules are stored in separate ModuleConfig objects named after the module.

For details, see Global settings.

Grantable cluster-wide resource

A cluster-wide resource, for example a StorageClass, that the multitenancy-manager module makes available to a project through a grant. Application settings bound to such a resource get the project default and are validated against the resources available to the project.

A settings field of type string is bound to a grantable resource with the x-deckhouse-grantable-resource extension, which names the resource available to the project, for example storageclasses. When the user leaves the field empty, the resource name configured as the project default is injected into the values. A value that is not in the list of resources available to the project is rejected. The group, version and kind of the resource are defined by the grant.

For details, see Application settings.

Guest OS agent

QEMU Guest Agent installed in the guest system so that Deckhouse Platform can interact with the operating system inside the virtual machine.

The agent is needed for three things: it makes consistent snapshots of disks and machines possible, it reports information about the running system to the .status.guestOSInfo block, and it shows that the operating system has actually booted rather than just the virtual machine having started. The AgentReady condition of a running machine means that the agent is working.

Deckhouse Platform works with qemu-guest-agent version 5.2.0 and later. The agent can be installed and enabled by the VM provisioning script.

For details, see Initial provisioning and the guest OS agent.

HA mode

A mode in which critical Deckhouse Platform components run with redundancy, so that the failure of a single instance does not stop the service.

HA mode is enabled automatically when the cluster has more than one master node, both at the initial deployment and when nodes are added later. It can also be managed manually.

For details, see High reliability and availability.

High-level role

A role of the basic role model of Deckhouse Platform, provided by the user-authz module on top of Kubernetes RBAC. User, PrivilegedUser, Editor, Admin, ClusterEditor, ClusterAdmin and SuperAdmin.

High-level roles belong to the basic role model. The user-authz module creates them as a set of ClusterRole objects and grants them to users and groups with ClusterAuthorizationRule and AuthorizationRule. Each role extends the previous one: User can view pods, logs and Deployments, Admin manages RBAC in a namespace, ClusterAdmin has full access to cluster-wide resources.

The recommended granular role model uses use roles and manage roles instead of high-level roles. The models are not compatible, automatic conversion is impossible, but they can be used at the same time: the permissions of both models are summed up.

For details, see Basic authorization model.

Hook binding

An event on which a hook runs, specified in the hook configuration. A hook can run on Kubernetes events, on schedule, at startup, before or after the Helm templates are applied.

Bindings of a module hook are onStartup, beforeHelm, afterHelm, kubernetes and schedule. Global hooks also have beforeAll and afterAll. The hooks of a Marketplace package use the same kinds of bindings, with the Queue field that sets a separate queue for the Schedule and Kubernetes bindings. Hooks with the same binding run in the order of their Order value.

For details, see Hooks.

Host firewall

A Cilium mode in which network policies apply to the cluster nodes themselves rather than to pods.

A host firewall is configured with CiliumClusterwideNetworkPolicy and the nodeSelector field. Such a policy applies to the host endpoint with the label reserved:host and filters the traffic entering and leaving the node, including hostNetwork pods. It is available only in clusters with the cni-cilium module.

Host policies add a filter layer inside the cluster and do not replace infrastructure-level protection such as physical firewalls or cloud security groups. A mistake in them can break SSH access, the control plane, kubelet or etcd, so they are rolled out through the audit mode first. For details, see Host firewall on nodes.

Hybrid cluster

A static Deckhouse Platform cluster extended with nodes placed in the infrastructure of a supported cloud provider.

The control plane and the initial nodes remain part of the static cluster. Additional nodes are added through the cloud provider module: either created automatically through the provider API as CloudEphemeral nodes, or created in advance and connected with the bootstrap script as CloudStatic nodes.

A hybrid cluster requires L3 connectivity between the data center and the cloud.

For details, see Hybrid clusters.

Hybrid node group

A node group that includes both Static nodes deployed in the cloud and servers in the on-premises data centre, bare metal or virtual machines.

In a hybrid node group the primary workload can run on bare-metal servers while cloud instances serve as a scalable addition during peak loads. All nodes are static nodes, so the node-manager module works as for static nodes. The only requirement is L3 connectivity between the data centre and the cloud. A hybrid node group differs from a hybrid cluster, which combines static nodes with CloudEphemeral nodes.

For details, see Hybrid node groups and clusters.

Image signature verification

Verification of the integrity and authenticity of user application images during Kubernetes API request validation, before a pod is created, against Cosign signatures and the public keys listed in a SecurityPolicy.

At the build stage the image is signed with a private key using Cosign. The signature is published in the registry as a separate tag and does not change the image. In the cluster a SecurityPolicy with the verifyImageSignatures parameter lists the public keys, and the admission-policy-engine module with Ratify verifies the signature of every image when the pod is created. This is the part of integrity control available in all editions. Verification at the container runtime level exists only in Certified Core and Certified Pro.

For details, see Integrity control.

In-cluster image cache

A container image registry deployed on the master nodes of the cluster, from which all nodes pull the images of Deckhouse Platform components.

The cache is enabled by the storage.cache parameter of the registry ModuleConfig in the Managed registry management mode. While the cache is being filled, the upstream registry stays a fallback path. In an isolated cluster, the cache is the only source of images and is filled with the d8 mirror push command.

Infrastructure node

A collective name for the master, frontend, system and monitoring nodes of a cluster, as opposed to worker nodes that run user applications.

Infrastructure nodes run the cluster control plane and Deckhouse Platform components. The recommended resource minimums depend on the role: master nodes and monitoring nodes need fast disks, frontend nodes need the least resources, and the size of system nodes depends on whether the cluster has dedicated monitoring nodes.

For details, see Going to production.

Ingress controller

A controller that routes user requests to applications and services by the rules defined in Ingress resources.

In Deckhouse Platform, the Ingress controller is Ingress NGINX Controller, managed by the ingress-nginx module through IngressNginxController resources. Several Ingress NGINX Controllers can run in one cluster at the same time, one of them is the primary one. Kubernetes Gateway API (the alb module) and Istio Ingress Gateway also balance traffic at the application level, but they work not with Ingress resources but with Gateway API routes and Gateway and VirtualService resources. All three solutions are implementations of ALB.

Ingress controllers are usually placed on frontend nodes. The way traffic reaches a controller is defined by its inlet. For details, see Incoming traffic balancing.

Inlet

The way incoming traffic reaches an application-level load balancer — Ingress NGINX Controller or a gateway of the alb module. It is set in the inlet field of IngressNginxController, ClusterALBInstance and ALBInstance resources.

For the Ingress controller, the inlet takes one of the values LoadBalancer, LoadBalancerWithProxyProtocol, HostPort or HostWithFailover. The LoadBalancer and LoadBalancerWithProxyProtocol inlets use a load balancer of the cloud provider or the built-in metallb module. The HostPort inlet exposes the controller on node ports, for example behind an external load balancer. The HostWithFailover inlet is used on bare metal, where the controller runs on frontend nodes.

For gateways of the alb module, the inlet takes one of the values LoadBalancer, HostPort or ClusterIP. ClusterALBInstance supports LoadBalancer and HostPort, ALBInstance supports LoadBalancer and ClusterIP. The ClusterIP inlet accepts traffic through a service of the ClusterIP type, without an external entry point.

For examples of each inlet, see ALB with Ingress NGINX Controller and the Inlets section of the “ALB with Kubernetes Gateway API” page.

Installation configuration file

A set of YAML documents with Deckhouse Platform settings and manifests of the cluster objects to be created after installation. It is passed to the CLI installer with the –config parameter.

The file contains InitConfiguration (required), ClusterConfiguration (required unless Deckhouse Platform is installed into an existing cluster), StaticClusterConfiguration or the provider cluster configuration, and optionally ModuleConfig, NodeGroup and other objects. The CLI installer reads it during preflight checks and the cluster bootstrap.

For details, see Installation.

Installation into an existing cluster

An installation mode in which Deckhouse Platform runs on top of an existing Kubernetes cluster and installs and updates platform modules, but does not manage the operating system, control plane or node lifecycle.

In this mode the existing infrastructure keeps managing the operating system, kubelet, containerd, etcd, the control plane and Kubernetes updates, the existing CNI provides pod networking, and machines are created and deleted by an external provisioner or the user. The module bundle Managed is selected during installation and cannot be changed afterwards. ClusterConfiguration is not required in the installation configuration file.

For details, see Installing Deckhouse Platform in an existing Talos cluster.

Installer

The tool that deploys a Kubernetes cluster and installs Deckhouse Platform in it. It is available as the dhctl CLI installer in a container image and as the graphical installer.

The dhctl CLI installer deploys a cluster in a supported cloud, on bare-metal servers, including hybrid clusters, or installs Deckhouse Platform into an existing Kubernetes cluster. In a cloud cluster it creates the required infrastructure: virtual machines, networks and other resources. In a running cluster the dhctl converge command applies configuration changes of CloudPermanent nodes and changes of protected parameters.

The second form is the graphical installer available on the Deckhouse Platform website.

For details, see Platform installation.

Instance class

A provider-specific resource, such as OpenStackInstanceClass or AWSInstanceClass, that describes the parameters of the virtual machines Deckhouse Platform creates for a node group.

A node group of CloudEphemeral nodes refers to an instance class in cloudInstances.classReference. The instance class defines the machine type, the image, disks and other provider-specific parameters, while the number of nodes and their zones are defined by the node group.

For details, see Adding and managing cloud nodes.

Integrity control

A set of mechanisms that verify containers to ensure their security and compliance with the specified configuration, including image signature verification at startup, runtime audit of running containers and immutable images of platform modules.

Deckhouse Platform provides three mechanisms: image signature verification of user workloads during Kubernetes API request validation, runtime audit of running containers, and integrity protection of Deckhouse Platform modules. Cryptographic verification of image signatures at the container runtime (CRI) level with DM-Verity protection of unpacked layers is available only in the certified editions.

Starting with version 1.74, modules are installed as read-only EROFS images mounted through DM-Verity, which protects an installed module from being replaced or modified on the node. This mechanism is enabled automatically when the node OS supports the erofs kernel module. Without it Deckhouse Platform keeps working and raises an alert.

For details, see Integrity control.

IPAM

An IP address management mechanism that allocates addresses to the network interfaces of pods and virtual machines and ensures that no address is assigned twice. In Deckhouse Platform the CNI modules, the sdn module and the virtualization module each have their own IPAM.

The cni-cilium, cni-flannel and cni-simple-bridge modules assign pod addresses from the pod address range allocated to the node. This IPAM is not configurable.

In the sdn module, IPAM allocates IPv4 addresses from pools to the additional network interfaces of pods and virtual machines. It is enabled for each network separately by an address pool reference in Network or ClusterNetwork. For cluster networks the pools are created by the administrator, for project networks by users. If a pod is connected to multiple networks that use IPAM, specify the address objects for each interface explicitly.

The addresses of virtual machines in the main cluster network are managed by default by the virtualization module through the VirtualMachineIPAddress and VirtualMachineIPAddressLease. When a pool is configured for the main network in the sdn module, the addresses of virtual machines are managed by the shared IPAM of that module: the address is requested automatically, and existing virtual machines are brought under its management without changing their addresses or restarting.

For details, see Additional networks for use in application pods and IP addresses of virtual machines.

Istio Ingress Gateway

The entry point of an application load balancer built on Istio. The administrator deploys it with IngressIstioController, and application developers publish applications with Gateway and VirtualService objects.

Publishing with Istio has two layers: the cluster administrator deploys the IngressIstioController and the related infrastructure, and developers create Gateway and VirtualService objects in their namespace. This is one of the ALB implementations in Deckhouse Platform, next to Ingress NGINX and the Gateway API with its managed Gateway.

For details, see Publishing applications using Istio.

Kubernetes API audit

Logging of requests to the Kubernetes API server according to an audit policy. By default the log is written to the /var/log/kube-audit/audit.log file on master nodes.

Kubernetes API audit tracks requests to the API server and helps to investigate unexpected behaviour and to meet security requirements. Deckhouse Platform configures a basic audit policy by default: it skips frequent service updates such as Endpoints, Events and leader election and logs changes of pods, nodes, ServiceAccount objects and operations in system namespaces. The policy can be extended with custom rules.

The audit log is a source for runtime audit and for security events.

For details, see Kubernetes API event audit.

Kubernetes conformance

Testing of Deckhouse Platform against the conformance test suite of the CNCF Certified Kubernetes Conformance Program. The tests are run for every supported Kubernetes minor version.

The tests are run with Sonobuoy in the certified-conformance mode, and the JUnit reports are published in the reference. The results are valid only when the multitenancy-manager module is disabled.

For details, see Kubernetes conformance test results.

kubernetes-api-proxy

An NGINX-based proxy on every cluster node that serves requests to the Kubernetes API sent to localhost and forwards them to the master nodes, providing high availability and load balancing of the API.

bashible installs and configures the proxy and keeps its list of upstream master nodes up to date. On master nodes it runs as a static pod: by default it proxies requests to the local kube-apiserver and, if that is unavailable, queries the remaining instances in turn. Requests from kubelet to localhost go through it.

For details, see Control plane management.

Layout

The scheme of node placement in the infrastructure of a cloud provider. It affects networking, availability of public IP addresses, outgoing traffic routing and the way nodes are accessed.

The layout is set in the layout parameter of the ModuleConfig of the cloud provider module. The available layouts depend on the provider, for example Standard, WithoutNAT and WithNATInstance in Yandex Cloud.

In some layouts nodes have no public IP addresses, and the master node is reached through a bastion host.

For details, see Integration with cloud providers.

License key

The credential required for using the commercial editions of Deckhouse Platform.

The license key is required to pull images of the commercial editions from registry.deckhouse.io, for example when updating a cluster in a private environment with d8 mirror pull.

For details, see Updating DP in air-gapped environment.

Live migration

Moving a running virtual machine from one node to another without shutting it down.

Live migration is used for load balancing, to free a node for maintenance or update, and for a virtual machine firmware update that would otherwise require a restart. A node sends the memory of one virtual machine at a time and accepts one incoming migration at a time, and the transfer rate of a single migration is limited to 640 MiB/s. VM rebalancing also moves machines with live migration.

For details, see Maintenance of nodes running virtual machines.

Local storage

Storage on the local disks of a node. It reduces network latency and gives higher performance than storage accessed over the network.

Local storage is one of the software-defined storage types and is provided by the sds-local-volume module, which supports the LVM and LVMThin modes. It is effective in test environments and EDGE clusters.

For details, see Setting up local storage based on LVM.

Log buffering

Temporary accumulation of logs in a buffer before they are sent to the storage system.

Buffering keeps logs during temporary connection issues with the storage and sends them when the connection is restored, smooths load spikes on the storage and reduces the number of network requests by sending logs in batches. It is configured with the buffer parameter of ClusterLogDestination in the log pipeline.

For details, see Log collection and delivery.

Log pipeline

The chain of log collection, filtering and delivery that the log-shipper module builds from ClusterLoggingConfig, PodLoggingConfig and ClusterLogDestination.

ClusterLoggingConfig describes log sources at the cluster level, PodLoggingConfig describes sources within one namespace, and ClusterLogDestination defines the parameters of the log storage. Along the pipeline, logs can be filtered by labels and message content, enriched with Kubernetes metadata, buffered and sent to several storages at once, for example to the short-term log storage and to Elasticsearch.

For details, see Log collection and delivery.

Long-term metrics storage

A separate storage, for example VictoriaMetrics, to which Prometheus sends metrics with the remote_write mechanism.

In Deckhouse Platform, sending metrics to a long-term storage is configured with a PrometheusRemoteWrite object.

For details, see Writing Prometheus data to longterm storage.

Main cluster network

The network in which every virtual machine gets an IP address. It is bound to the main network interface of the pod that runs the machine.

In the list of networks of a virtual machine the main cluster network has the type Main. It does not have to be listed, but if it is, it must come first. Adding or removing the main network requires a reboot of the machine, unlike additional networks, because it cannot be changed on a running pod. The subnets that machines get addresses from are set in virtualMachineCIDRs of the virtualization module. When an address pool is configured for the main network in the sdn module, the addresses are managed by the shared IPAM.

For details, see IP addresses of virtual machines.

Main project namespace

The user namespace that is created together with the project. It is used to manage additional project namespaces and to run the workload.

The main project namespace is a user namespace created as part of a project. Additional project namespaces are managed from it.

Maintenance mode of a module or application

A mode in which Deckhouse Platform stops applying templates and restoring the resources of a module or an application, so that they can be changed manually. It is enabled with spec.maintenance: NoResourceReconciliation and is not related to the node maintenance mode.

In this mode changes of settings and values are not reflected in the resources, deleted resources are not restored, and hooks keep running. The resources rendered from the templates get the maintenance.deckhouse.io/no-resource-reconciliation label, and the admission policies stop protecting them. For an application instance the Managed condition becomes False and the ApplicationIsInMaintenanceMode alert fires. To return to normal operation, remove the spec.maintenance field.

The field is set in ModuleConfig for a module and in Application for an application. The mode is intended for debugging and development, not for production clusters. The node maintenance mode is a different mechanism.

For details, see Lifecycle and debugging.

Manage role

A role of the granular role model for administering Deckhouse Platform or one of its subsystems, that is, cluster-wide resources, the modules of the subsystem and their system namespaces.

Manage roles belong to the granular role model. A manage role grants access to cluster-wide Kubernetes resources, to the ModuleConfig and cluster-wide resources of the modules of its subsystem, and to the system namespaces in which these modules run. It does not grant access to the namespaces of user applications.

The name of a manage role has the form d8:manage:<SUBSYSTEM>:<ACCESS_LEVEL>, for example d8:manage:networking:manager. The subsystem all covers all modules. Permissions within a namespace are granted with use roles.

For details, see Granular authorization model.

Managed cluster

A Deckhouse Platform cluster with the commander-agent module installed. Its configuration and lifecycle are managed from a managing cluster.

The commander-agent module connects the cluster to the commander module of the managing cluster. Not to be confused with a Managed Kubernetes cluster, in which the control plane is provided by a cloud provider: see cluster managed by Deckhouse Platform.

For details, see Cluster management.

Managed Gateway

The Gateway object that the alb module creates from a ClusterALBInstance or an ALBInstance. Applications are published through it with ListenerSet and route resources of the Kubernetes Gateway API.

A cluster-wide gateway is created by the cluster administrator from a ClusterALBInstance, usually in the d8-alb namespace. A dedicated gateway in the application namespace is created from an ALBInstance. The name of the managed Gateway is published in the status.gateway field of the instance. The namespace administrator attaches ListenerSets with hostnames, TLS and ports, and developers attach HTTPRoute and other routes. The listeners d8-http and d8-https serve service tasks such as availability checks and are not meant for application routes.

For details, see Publishing applications using the Kubernetes Gateway API.

Managed service

A ready-to-use application service in the cluster, such as a database or a message queue, whose lifecycle is handled by Deckhouse Platform.

Each managed service adds its own Kubernetes resources and controllers to the cluster. A user describes the desired state of the service in a namespaced resource, and the controller creates and updates the related objects: workload instances, Services, secrets and snapshots. The administrator manages cluster-wide resources, such as service classes, that define the allowed configurations and default values for user resources. Deckhouse Platform handles administration, scaling, backup and updates of the service.

For details, see Managed services.

Managing cluster

A Deckhouse Platform cluster with the commander module installed. It creates standardised Deckhouse Platform clusters and manages their configuration and lifecycle.

The commander module implements a web application for cluster management and needs a PostgreSQL instance, deployed inside or outside the cluster. Clusters under its management run the commander-agent module and are called managed clusters.

For details, see Cluster management.

Marketplace

The package management system of Deckhouse Platform. Administrators connect package repositories, and users install applications from the packages in their namespaces.

A cluster administrator connects a package repository with PackageRepository, monitors the scanning operations in which Marketplace discovers the available packages, and grants users the permissions to view package versions and to create Application objects. An installed application is an Application object in the namespace of the user.

Marketplace is available in Deckhouse Platform starting from version 1.76.

For details, see Marketplace.

Marketplace package

An abstract unit of Marketplace: either an application, a user workload deployed into a namespace, or a module that extends the capabilities of the cluster.

Packages are published in container registries connected to Marketplace as package repositories. For every discovered package version Deckhouse Platform stores its metadata: description, lifecycle stage, requirements, deletion confirmation, changelog and the OpenAPI schemas of settings and values.

For details, see Concepts.

Master node

A node that runs the control plane components. Master nodes carry the label node-role.kubernetes.io/control-plane, and the control-plane-manager module runs on them.

Deckhouse Platform manages the control plane on master nodes: certificates, component configuration, versions, the etcd cluster and kubeconfig files. In a cloud cluster, master nodes are CloudPermanent nodes.

The number of master nodes defines whether the cluster is single-master or multi-master. With more than one master node, HA mode is enabled automatically.

For details, see General management and configuration of the control plane.

Minor version

A Deckhouse Platform release that includes new features.

A minor version is one of the two types of a Deckhouse Platform release, the other being a patch version. Minor versions are installed sequentially: if the cluster lags behind the release channel by several minor versions, Deckhouse Platform reconstructs the intermediate releases, because they contain migrations that must be applied in order. In the Manual update mode a minor version needs approval.

For details, see Updating.

Module

A set of resources and applications that adds new capabilities to Deckhouse Platform.

Modules perform related functions, for example collecting metrics, balancing traffic or working with secrets. The documentation of a module and its API is available in the module library.

Module accessibility

The accessibility settings of module.yaml that define in which Deckhouse Platform editions a module is available and in which module bundles it is enabled by default.

Settings are given per edition with a _default block for editions without their own settings: available says whether the module can be used in the edition, enabledInBundles lists the module bundles (Default, Managed, Minimal) in which the module is enabled without a ModuleConfig.

For details, see Module structure.

Module artifacts

The three kinds of artifacts a module build pushes to the container registry, images of application containers, the module image and the release. Deckhouse Platform pulls and updates a module from them.

Images of application containers are tagged with content-based tags and referenced from the templates. The module image is built by werf.yaml and tagged with the semantic version. The release is the artifact of a module version: it carries a semantic version tag and a tag of the release channel, and Deckhouse Platform decides whether to update the module in the cluster by its data. The registry with the artifacts serves as a module source.

For details, see Build and publish.

Module bundle

A named set of built-in Deckhouse Platform modules that are enabled by default. Default, Minimal or Managed.

The bundle is set in the bundle parameter of the deckhouse module and cannot be changed in a deployed cluster. Default is the recommended set for cluster operation, including monitoring, authorization, networking and other basic modules. Managed is the set for managed clusters of cloud providers. Minimal is the minimal set.

Regardless of the bundle, a module can be enabled or disabled explicitly in its ModuleConfig.

For details, see Deckhouse Platform configuration.

Module dependencies

A set of conditions (requirements) that must be met for Deckhouse Platform to run a module. They are declared in the requirements section of module.yaml.

A module can depend on the Deckhouse Platform version, on the Kubernetes version and on the versions of other modules. Dependencies on other modules affect the module startup order together with the module weight, and a dependency may be conditional. A Marketplace package declares the same kind of requirements in package.yaml, including groups of alternative and incompatible modules.

For details, see Dependencies of the Deckhouse Platform modules.

Module exclusive group

A group of modules that share one value of the exclusiveGroup parameter in module.yaml. Only one module of the group can be active in the cluster at a time.

Exclusive groups prevent conflicts between modules that perform similar or incompatible functions. Enabling a second module of the same group is not possible while the first one is active.

For details, see Module structure.

Module from a source

A module that Deckhouse Platform gets from a module source instead of its own distribution. Its release cycle is not tied to the release cycle of the platform.

The list of available modules is defined by the module sources tracked by Deckhouse Platform. Such a module is updated by its own update policy: it has its own release channels, lifecycle stage and dependencies. Alerts of modules from a source are not included in the alerts reference of the platform.

For details, see Updating DP in air-gapped environment.

Module hook

An executable file of a module or a Marketplace package that Deckhouse Platform runs in response to an event. Hooks prepare values for templates, create and change objects in the cluster and validate settings.

A hook runs when one of its bindings fires: at startup, before or after the Helm templates are applied, on schedule or when watched Kubernetes objects change. The hooks of a module are located in the /hooks/batch directory. The hooks of a Marketplace package are written in Go with module-sdk and compiled into a single binary. Module hooks and global hooks exist. The Deckhouse controller runs them as tasks of the addon-operator queues.

For details, see Module structure.

Module lifecycle stage

The maturity stage of a module or a Marketplace package, Experimental, Preview, General Availability or Deprecated. It is declared in module.yaml or package.yaml and, together with the release channel, defines how stable a version is.

• Experimental: the functionality may change significantly and compatibility with future versions is not guaranteed; such modules cannot be enabled unless the allowExperimentalModules parameter of the deckhouse module is set to true.
• Preview: a preliminary version; the basic features are preserved, compatibility is ensured but may require migration.
• General Availability: ready for production environments.
• Deprecated: development and support are discontinued.

The stage of a Marketplace package is published in the package metadata when Deckhouse Platform scans the repository.

For details, see Versioning modules.

Module settings conversion

A rule in the openapi/conversions directory of a module that converts module parameters from one version of the OpenAPI specification to the next, for example when a parameter is renamed or moved.

Each conversion works between two consecutive versions and is stored in a v<N>.yaml file with its tests. The chain of conversions must cover all versions of the parameter specification without gaps, so that Deckhouse Platform can convert the settings of a module automatically when the version of its settings schema changes.

For details, see Module structure.

Module source

The container registry from which Deckhouse Platform fetches information about modules that are not built into the platform. It is set by ModuleSource with the registry address and authentication parameters.

Modules from a source are published to the registry as module artifacts. Deckhouse Platform synchronizes with the source and creates module releases according to the module update policy. A module is then enabled with ModuleConfig. In an air-gapped cluster the source points to the in-cluster or private registry.

For details, see How to run and verify a module in the DP cluster.

Module update policy

The rules by which Deckhouse Platform updates a module from a source in the cluster. It is set by ModuleUpdatePolicy and defines the update mode, the release channel and the update windows of the module.

A policy sets the update mode (automatic, manual or updates disabled), the release channel used for updates and the update windows in which automatic updates are allowed. If no policy is defined for a module, the module follows the update settings of the platform from the update parameter of the deckhouse module. A module release in the Pending status waits for manual approval.

For details, see How to run and verify a module in the DP cluster.

Module weight

A number from the weight parameter of module.yaml that defines the module startup order. Modules with a lower weight start earlier, the default is 900.

If module.yaml is missing, the weight is taken from the numeric prefix of the module directory, for example 040-node-manager means 40. If it cannot be obtained, 100 is used. The Deckhouse controller considers the weight only for critical modules, whose tasks go to the main queue in ascending order of weight. The startup order is also affected by module dependencies.

For details, see Queuing mechanism.

Monitoring node

A dedicated node for Grafana, Prometheus and other monitoring components. Without such nodes the monitoring components run on system nodes.

Dedicated monitoring nodes are recommended in heavily loaded clusters with many alerts and large volumes of metrics. They need fast disks, at least 400 IOPS. Monitoring nodes usually form the monitoring node group.

For details, see Recommendations for configuring cluster nodes.

Multi-master cluster

A control plane operation mode with several master nodes. kube-apiserver interacts with all etcd instances, and requests are forwarded to other nodes if the local kube-apiserver is unavailable.

The multi-master mode provides high availability of the control plane and supports scaling of master nodes. When the cluster has more than one master node, HA mode is enabled automatically. In a configuration with two master nodes, an arbiter node provides the etcd quorum.

For details, see Scaling and changing master nodes.

Multicluster

An inter-cluster communication model that tightly connects clusters. The clusters share namespaces, and the services of one cluster are available to the others as local ones.

Access to services from other clusters can be restricted by authorization policies. The model is implemented with Istio through the istio module and is built on the service mesh. The model for independent clusters is federation.

For details, see Inter-cluster cooperation.

Namespace

A Kubernetes mechanism for grouping objects logically.

Examples of Kubernetes objects that can be grouped by a namespace: Pod, Deployment, Service, Ingress, ConfigMap.

Namespaces are often used for isolation, but without additional configuration they do not provide it, unlike projects in Deckhouse Platform.

Namespaces are divided into system and user namespaces.

Network policy

Rules that restrict how pods communicate with each other, with external systems and with cluster nodes.

The implementation of network policies in Deckhouse Platform depends on the enabled CNI module. With cni-cilium, policies are enforced by Cilium in three formats: the standard NetworkPolicy at L3 and L4, Cilium network policies with L3–L7 rules within a namespace, and CiliumClusterwideNetworkPolicy for the whole cluster, which also protects nodes as a host firewall. With cni-flannel or another CNI without policy support, the network-policy-engine module enforces the standard NetworkPolicy at L3 and L4 through iptables and ipset rules on every node.

For details, see Network policies.

NLB (network load balancing)

Load balancing at the network level. Traffic is routed by IP addresses and ports without inspecting the contents of requests.

In Deckhouse Platform, NLB is implemented either by an external load balancer of the cloud provider or by the built-in metallb module, which works both in cloud and in bare-metal clusters. Balancing at the application level is ALB.

For details, see Incoming traffic balancing.

Node group

A set of cluster nodes with common parameters, described by a NodeGroup. The NodeGroup defines the node type, the number of nodes and their configuration, and Deckhouse Platform keeps the nodes in line with it.

Node groups divide the cluster infrastructure logically. Typical groups are master for the control plane, front for frontend nodes, monitoring for monitoring nodes, system for system nodes and worker for user applications.

Each group has centralized settings: the Kubernetes version, resources, taints, labels and kubelet parameters. When a NodeGroup is created or changed, the node-manager module brings the nodes of the group to the specified configuration. The node type is one of Static, CloudEphemeral, CloudPermanent or CloudStatic.

For details, see Node management basics.

Node group autoscaling

Automatic change of the number of nodes in a group by the Cluster Autoscaler component of the node-manager module, based on CPU and memory demand. It triggers only when Pending pods cannot be scheduled on the existing nodes.

Autoscaling is available for CloudEphemeral nodes. The limits are set in the cloudInstances section of the node group: minPerZone and maxPerZone bound the number of nodes per zone, maxUnavailablePerZone limits unavailable nodes during changes, priority defines which groups Cluster Autoscaler scales first. Autoscaling does not apply to static nodes: for them CAPS maintains the number set by staticInstances.count.

For details, see Adding and managing cloud nodes.

Node maintenance mode

A node state in which Deckhouse Platform moves the virtual machines to other nodes and does not start new ones on it, so that work on the node does not disrupt them.

A node is switched to maintenance mode with the d8 k drain command or with Cordon + Drain in the web interface, and returned to service with d8 k uncordon. Virtual machines that can be moved are moved by live migration. A machine that cannot be moved, for example one pinned to the node or using a passed-through device, keeps the node busy until it is restarted. The restart is allowed by the virtualization.deckhouse.io/virtualmachines-restart-approved annotation on the node.

For details, see Maintenance of nodes running virtual machines.

Orphan resources of an application

Objects that stay in the namespace after the Helm release of an application is uninstalled, for example objects created by workloads or hooks at runtime. Deckhouse Platform deletes the listed kinds together with the Application object.

The package declares the kinds of orphan resources in the orphanResources field of openapi/values.yaml. When the Application is deleted, Deckhouse Platform deletes the objects of these kinds in the namespace of the application instance that carry the packages.deckhouse.io/instance: <INSTANCE_NAME> label, after the Helm release is uninstalled and the hooks are stopped. Objects created by workloads and hooks do not get the label automatically.

Orphan resources are deleted only when the Application is deleted. On suspension and on a package version change they stay in the cluster.

For details, see Lifecycle and debugging.

Package bundle

The main image of a package version in the container registry. It contains the package manifest, image digests, OpenAPI schemas, Helm templates, the hooks binary, documentation and the changelog.

The bundle is published as <REPOSITORY>/<PACKAGE_NAME>:<PACKAGE_VERSION>, next to the container images of the application (referenced by digest), the version metadata image and the release channel images. Deckhouse Platform downloads the bundle of the requested version as the first step of installing an application instance, then reads package.yaml, the schemas and images_digests.json and discovers the hooks.

For details, see Application development.

Patch version

A Deckhouse Platform release that includes fixes. Patch versions are published when needed.

A patch version is one of the two types of a Deckhouse Platform release, the other being a minor version. In the AutoPatch update mode only patch versions are installed automatically. Starting from Deckhouse Platform 1.70 they are applied with update windows taken into account.

For details, see Updating.

Pod limit per node

The maximum number of pods that can be placed on a node. In Deckhouse Platform it is derived from the node subnet size set by the podSubnetNodeCIDRPrefix parameter when the cluster is deployed.

With a prefix of 24 or larger a node holds 120 pods, with 23 it holds 250, with 22 it holds 500, and with 21 or smaller it holds 1000. The parameter is part of the protected parameters of the cluster, so choose the value at deployment time if many pods per node are expected. The control plane components for high pod density are configured by Deckhouse Platform automatically.

For details, see High pod density.

Pod redistribution

Periodic eviction of pods that match the active scheduling strategies, so that they are scheduled again according to the current state of the cluster. Controlled by the descheduler module.

Deckhouse Platform analyzes the state of the cluster every 15 minutes and evicts the matching pods. The evicted pods go through the standard scheduling process again. The priority class of a pod defines the order of eviction, and the priorityClassThreshold parameter of a Descheduler limits eviction to pods with a priority below the threshold. VM rebalancing is built on the same mechanism.

For details, see Scheduler.

Pod Security Standards

Three levels of pod security policies, privileged, baseline and restricted, that Deckhouse Platform applies to namespaces with the admission-policy-engine module.

Pod Security Standards are one of the categories of admission policies. The level privileged imposes no restrictions, baseline prevents the most common privilege escalation techniques, restricted sets the strictest requirements for pods. Deckhouse Platform takes only the policy descriptions from Kubernetes and enforces them with Gatekeeper rather than with the Kubernetes Pod Security Admission controller.

The default level is baseline. It applies to namespaces without the security.deckhouse.io/pod-policy label, except the system ones. The label assigns a level to one namespace, the admission-policy-engine module setting changes the default.

For details, see Security policies.

Post-bootstrap script

A custom script that the installer runs on one of the master nodes after the cluster installation is complete.

The script is used for additional cluster configuration, collecting diagnostic information or integration with external systems. Its path is passed to the installer with the --post-bootstrap-script-path parameter.

For details, see Installation.

Preflight checks

Checks that the installer runs before installing Deckhouse Platform.

The checks are grouped into general checks, checks for static clusters and checks for cloud clusters. An individual check is skipped with the --preflight-skip-check flag and its name, --preflight-skip-all-checks skips all of them, and dhctl preflight list prints the full list. Cluster subnets and the DNS name template are validated when the installation configuration file is loaded and cannot be skipped.

For details, see Installation.

Priority class

One of the PriorityClass objects that Deckhouse Platform creates in the cluster and assigns to its components. When resources are insufficient, the scheduler preempts pods with a lower priority first.

Applications use a priority class through the priorityClassName field. A pod without it is treated as having the lowest priority. The classes range from system-node-critical and system-cluster-critical, reserved for critical cluster components, through production-high, production-medium, production-low and staging to develop, the default. Additional classes can be created with a name and a numeric priority value. Priority classes also define the order of eviction during pod redistribution.

For details, see Priority classes.

Private environment

An environment without direct access to the public container registry of Deckhouse Platform and to external package repositories. Access goes through a proxy server or through a container registry deployed inside the perimeter.

Proxy server parameters set in the cluster configuration during installation are propagated to the environment variables of cluster nodes and Deckhouse Platform components. User applications do not receive them automatically. Cluster nodes are usually reached through a bastion host, and the private registry runs on a separate machine in the internal network. When access to external resources is disabled completely, no proxy is used, the required artefacts are brought into the perimeter by other means, and the cluster is an air-gapped cluster.

For details, see Installing DP in a private environment.

Project

A mechanism for isolating and managing workloads in Deckhouse Platform. A project logically divides the cluster and gives a team its own area to deploy applications, configure resources and manage access within the granted rights.

Projects provide isolation, support creation from templates, control which platform capabilities are available to the team and are compatible with Kubernetes namespaces. A project includes the main project namespace and may include additional project namespaces.

For details, see Multitenancy.

Project features

The set of platform capabilities available within a project. Features define which APIs and resource kinds can be used in the project.

Project features are not a part of the role model. Examples of features: Kubernetes, Virtualization. The features of a single namespace within the project are limited by the project namespace features.

Project namespace features

A restriction of the project features at the level of a single user namespace within the project.

The set of features of a user namespace cannot exceed the set of features of its project.

Project network

An additional software-defined network available within one namespace. The user creates and manages it with the NetworkClass provided by the administrator. It is also called a user network.

The administrator defines in a NetworkClass the range of VLAN tags and the node network interfaces on which users may create their own networks. The user creates the network with the Network resource in the namespace of the project. Users can allocate an address pool for the project network and assign addresses to pod interfaces through IPAM. The other kind of additional network is the cluster network.

For details, see Additional networks for use in application pods.

Protected parameters

Cluster parameters that are critical for cluster operation and cannot be changed in a running cluster by default. podSubnetCIDR, podSubnetNodeCIDRPrefix and serviceSubnetCIDR.

The protected parameters are part of the cluster configuration. An attempt to change them is blocked by the admission webhook with an error message. The procedure for changing them, with the installer or manually, is described in the documentation.

For details, see Deckhouse Platform configuration.

Quorum

The majority of the members of a distributed system (more than half of their total number) that must be available for the system to keep working and accept changes.

In Deckhouse Platform, the quorum matters most for etcd, the storage of the control plane. The etcd cluster accepts changes only while the majority of its members (N/2 + 1) are available: a cluster of three members tolerates the failure of one member, a cluster of five tolerates the failure of two. An even number of members does not improve fault tolerance: with two members, the failure of either one loses the quorum. For this reason, use an odd number of master nodes, usually three. In a configuration with two master nodes, the third etcd member may run on an arbiter node, which provides the quorum.

For details, see High availability in the control plane recovery guide.

Region

An independent geographic area that includes several, typically at least three, isolated availability zones united into a single logical infrastructure unit with shared management services.

A region is a key element of the global distributed architecture of cloud providers. Regions are widely dispersed geographically, placed outside seismically or politically unstable areas and connected to several power grids. Latency between the availability zones of one region is about 5 ms, between regions it is 50 to 200 ms.

Deckhouse Platform supports distributing cluster nodes across regions (Multi-Region) so that applications keep running even when a whole data center or cloud provider fails. Because of the latency between regions, this is appropriate only for applications that work independently in each region, for example CDN cache servers or distributed monitoring nodes. The administrator labels each node with its region and provides stable communication channels between regions.

For details, see Geo-distribution.

Registry management mode

The mode in which the registry module manages the pull paths of Deckhouse Platform component images, Unmanaged or Managed.

The mode is set by the mode parameter in the registry ModuleConfig:

• Unmanaged (the default). The registry module does not manage the pull paths: the cluster pulls images from the registry it was installed with.
• Managed. The registry module manages the pull paths. The primary.upstream parameter sets the upstream registry, and storage.cache enables the in-cluster image cache on the master nodes. Without primary.upstream, the cluster is considered isolated.

For details, see Managing the registry in DP-managed clusters.

Release channel

One of the five channels, from Alpha to Rock Solid, through which every new Deckhouse Platform version is rolled out gradually. The cluster receives the versions published on the channel it follows.

Each new version is first published to the Alpha channel and then moves to Beta, Early Access, Stable and Rock Solid. Versions appear on Early Access at least 7 days after the release, on Stable at least 14 days and on Rock Solid at least 30 days after the release.

The channel of the cluster is set in the settings.releaseChannel parameter of the deckhouse module. How a new version is applied depends on the update mode and update windows.

For details, see Updating.

Release channel image

The release-channel image in the Deckhouse Platform registry whose tag is the name of a release channel. The tag points to the image of a specific Deckhouse Platform version and is moved to a new image when a new version is released.

The image contains version.json with the current version of the release channel and changelog.yaml. When the version in the image changes, Deckhouse Platform creates a DeckhouseRelease object and starts the update. A rollout can be paused by patching the image with a suspend flag, which makes it a suspended release. In a private environment the image is copied to the private registry together with the image bundle.

For details, see Updating DP in air-gapped environment.

Replicated storage

Storage in which data is replicated across several nodes, so that it stays available when one of the nodes fails.

Replicated block StorageClass objects are created with the sds-replicated-volume module, which is based on Distributed Replicated Block Device (DRBD). Replicated storage is one of the software-defined storage types. Its pools are created on LVM volume groups as thick or thin pools.

For details, see Setting up replicated storage based on DRBD.

Role model

Role-based access control (RBAC), a mechanism that defines which users or services can perform specific actions within projects or the system.

Deckhouse Platform supports two role models: the granular one, which is recommended and is built on use roles and manage roles, and the basic one with high-level roles. Both models are supported by the user-authz module and can be used at the same time: the permissions they grant are summed up.

For details, see Authorization.

Runtime audit

Detection of security threats in a running cluster by analysing Linux kernel events and Kubernetes API audit events.

Runtime audit in Deckhouse Platform uses two sources of events: Linux kernel events, received through the eBPF driver of the Falco threat detection system, and Kubernetes API audit events, received through a webhook. It detects attempts to exploit known vulnerabilities, signs of cryptocurrency miners and Kubernetes-specific threats such as shells launched in containers, privileged containers, mounting of insecure paths and reading of sensitive files.

Capturing kernel events requires Linux kernel 5.8 or later with eBPF support.

For details, see Runtime audit.

Runtime audit rule

A rule of the runtime audit that describes a pattern of suspicious behaviour as a condition expression in the Falco syntax. Deckhouse Platform ships built-in rules and accepts custom ones.

Built-in rules include rules for the Kubernetes audit, which reveal problems with the security of Deckhouse Platform and of the audit mechanism itself, and the fstec regulatory rules that meet the requirements of FSTEC of Russia order No. 118 for containerization tools. Custom rules are defined with a FalcoAuditRules object of the runtime-audit-engine module. The runtime audit matches events against all rules and produces security events.

For details, see Runtime audit architecture.

Runtime profile

A behaviour model of an application or container at runtime, including system calls, executed processes, file access and network interactions, built by the node-agent of the operator-trivy module with eBPF programs.

The node-agent is an optional component that runs on all cluster nodes in privileged mode, observes container behaviour with eBPF programs and stores the profiles in the ApplicationProfile and NetworkNeighborhood objects. It works in profiling mode and does not detect attacks by rules. That is the task of the runtime audit. The profiles can be used to generate network policies from the observed traffic.

For details, see Module operator-trivy.

Scheduler extender

An extension of kube-scheduler provided by a Deckhouse Platform module that places pods according to the module’s own logic instead of the standard scheduler, for example by local volumes, node selectors of a storage class or network interfaces.

The sds-local-volume module places pods with local volumes by the free space on nodes and the space to reserve for the volumes. Its webhook sets spec.schedulerName of such pods to sds-local-volume. The csi-nfs module places pods by the node selectors of NFSStorageClass, and the sdn module suggests nodes according to the network interface settings of pods using software-defined networks. kube-scheduler sends scheduling requests for such pods to the extender’s webhook.

For details, see Storage architecture.

Secret distribution

Automatic copying of secrets from the default namespace that carry the secret-copier.deckhouse.io/enabled label to all namespaces of the cluster.

Deckhouse Platform copies the labelled secrets to every existing and every new namespace, propagates updates of the original, overwrites manual changes of the copies and deletes the copies when the original is deleted. A nightly synchronization keeps the copies up to date. This removes manual copying of secrets in CI/CD pipelines, for example registry credentials.

The mechanism is not compatible with the multitenancy mode: a secret distributed to all namespaces can reach the isolated environment of a project. Shared secrets for projects are defined in the project template instead.

For details, see Automatic distribution of secrets across namespaces.

Secrets store

An external store compatible with the HashiCorp Vault API, or the built-in Deckhouse Stronghold, from which the secrets-store-integration module delivers secrets to applications without storing them in etcd.

The secrets-store-integration module delivers secrets to pods as mounted files or environment variables, refreshes mounted secrets every two minutes when the value in the store changes and injects an entrypoint for applications that cannot read secrets from the store themselves. It connects to an internal Deckhouse Stronghold instance automatically in the DiscoverLocalStronghold mode and works with any Vault-compatible store in the Manual mode.

For details, see Secrets-store-integration module.

Security event

A structured record of an action or fact that is significant from an information security perspective, extracted from the logs of applications and Kubernetes infrastructure components.

Deckhouse Platform collects security events from container sources, that is, application logs in pods, and from cluster sources, such as the Kubernetes API audit log and node authentication logs. The log-shipper module pre-selects log records, and the security-events-manager module parses them, brings them to a unified format with a mandatory set of attributes, enriches them with context and delivers them to storage and analytics systems such as Loki, Elasticsearch, Kafka or Splunk.

For details, see Security events.

Service mesh

An infrastructure layer that manages network communication between the services of an application. Encryption, access control, routing, request retries and metrics collection are configured centrally, without changing the code of the services.

The services of an application exchange requests over the network, and without a service mesh each of them has to implement encryption, authorization, retries and timeouts on its own. A service mesh takes over these tasks: traffic between services passes through proxies that are configured from a single place. Usually the proxy is added to the application pod as a sidecar container. In the experimental ambient mode the proxy runs at the node level, and pods need no sidecars.

In Deckhouse Platform the service mesh is provided by the istio module. Within a single cluster, it is used, for example, for canary deployment and for automatically cutting off a failing service (Circuit Breaker). It also forms the basis for federation and multicluster.

For details, see Inter-cluster cooperation.

Short-term log storage

The built-in in-cluster log storage of Deckhouse Platform based on Grafana Loki. It is integrated with log collection and added to Grafana as a data source.

The storage is provided by the loki module, whose settings define the disk size, the retention period, the StorageClass and the resources. Logs of system components flow into it after the log pipeline is configured. Log collection from user applications is configured separately. The storage does not support the high availability mode, so important logs are kept long-term in an external storage.

For details, see Short-term log storage.

Single-master cluster

A control plane operation mode with one master node. kube-apiserver uses only the local etcd instance and listens only on the address of the master node.

A proxy server on the node handles requests on localhost. When master nodes are added or removed, Deckhouse Platform migrates the control plane between the single-master and multi-master modes.

For details, see Scaling and changing master nodes.

Single-node cluster

A cluster of one node that performs all node roles. It has no fault tolerance and is intended for familiarisation or workloads with low resource requirements.

After the taint is removed from the node, it runs all cluster components of the selected module bundle. The resource requirements for such a node depend on the workload type: containers, virtual machines or both. Do not use single-node clusters in production environments.

For details, see the Resource requirements section of the “System requirements” page.

Sizing policy

The block of a virtual machine class that defines which combinations of cores, core fraction and memory are allowed for virtual machines of this class.

A policy consists of a list of rules, each applying to its own range of cores. The ranges must not overlap. Changes to the policy affect existing virtual machines: a machine that no longer meets the requirements gets the SizingPolicyMatched condition with the False status. The allowed values of the core fraction set in the policy define the degree of CPU oversubscription.

For details, see Virtual machine sizing policy.

Skipping intermediate module releases

The from-to mechanism that lets Deckhouse Platform skip step-by-step module updates. If the installed module version falls into a from-to range declared in module.yaml, Deckhouse Platform installs the latest version within to.

Transition rules are listed in the update.versions section of module.yaml. A release that contains them is called a constrained release. The rule is read from the release to move to, and its to must match the version of that release. If the installed version is lower than from, the update proceeds sequentially. When several rules match, the one with the largest to is chosen.

For details, see Module development and debugging.

SLA monitoring

Collection of availability statistics of cluster components and Deckhouse Platform components by the upmeter module to evaluate SLA compliance.

Availability data is shown in the web interface and can be exported over the Prometheus Remote Write protocol with an UpmeterRemoteWrite object.

For details, see Cluster SLA monitoring.

Software-defined network (SDN)

An additional network segment for pods and virtual machines that the administrator describes declaratively in custom resources, while the sdn module configures the nodes.

Software-defined networks in Deckhouse Platform cover the configuration of network interfaces on nodes (port aggregation, bridges, VLAN interfaces), additional networks for workloads that are available in the whole cluster or within one namespace (project networks), IP address management for these networks, underlay networks for passing physical interfaces to applications, and system networks for service traffic at the node level.

For details, see Software-defined networking.

Software-defined storage

A group of storage types in Deckhouse Platform that run on the disks of the cluster nodes and are managed by the sds-* modules, as opposed to external storage systems.

Software-defined storage (SDS) includes managing logical volumes on cluster nodes, local storage based on Local Path Provisioner or LVM, and replicated storage based on DRBD. The other group is external storage: Ceph, NFS, S3 and storage systems of hardware vendors.

For details, see Storage.

Standby node

A pre-provisioned node kept in the cluster so that new application replicas start without waiting for node creation during horizontal scaling. The number of standby nodes is set by cloudInstances.standby.

Standby nodes are available for CloudEphemeral nodes. The number is set as an absolute value or as a percentage of the maximum number of nodes in the node group. At least one node of the group must already be running. The share of node resources taken by service components outside Deckhouse Platform is set by standbyHolder.overprovisioningRate. Standby nodes increase infrastructure costs.

For details, see Adding and managing cloud nodes.

Static cluster

A cluster with the clusterType: Static. Its nodes are physical servers or virtual machines that Deckhouse Platform does not create or manage through a cloud API.

The administrator prepares the servers, and the installer configures them, deploys Kubernetes and installs Deckhouse Platform. The nodes of such a cluster are static nodes. A static cluster can be extended with nodes of a cloud provider into a hybrid cluster.

For details, see Platform installation.

Static node

A node on a bare-metal server or a virtual machine that is created outside Deckhouse Platform automation and connected to the cluster manually or through CAPS. It is described by a NodeGroup with the Static node type.

Static nodes do not support automatic scaling and updates: kubelet updates, restarts and replacements are performed manually or by external tools. A node is connected either by running the bootstrap script on the server or automatically by CAPS. In a cloud environment, cloud-controller-manager does not manage a Static node even if a cloud provider module is enabled.

A node group can combine static nodes with CloudStatic nodes, which is the basis of a hybrid cluster.

For details, see Adding and managing bare-metal nodes.

Static pod

A pod that kubelet starts directly from a manifest in the /etc/kubernetes/manifests directory of the node, without the Kubernetes API. Control plane components and kubernetes-api-proxy run this way in Deckhouse Platform.

In Deckhouse Platform kubelet processes only files with the .yaml or .yml extension in this directory. Backup copies, editor swap files and other temporary files are ignored. The control plane components kube-apiserver, etcd, kube-controller-manager and kube-scheduler and the kubernetes-api-proxy are static pods.

For details, see Kubelet.

Static user

A user created in the cluster as a User object of local authentication, with an email and a hashed password, without an external authentication provider.

Static users and groups are the basis of local authentication in Deckhouse Platform. A User object stores the user information and the password hash, the password is not stored in plain text. A Group object defines a list of users. Local authentication supports password policies and temporary users with a TTL. Two-factor authentication is also available.

The recommended way to create a static user is the d8 iam user create command.

For details, see Local authentication.

Stronghold lease

Temporary access to secrets and resources in the Deckhouse Stronghold secrets store, identified by a lease ID.

Leases are managed in the “Leases” section of the Stronghold web interface, where a lease is found by its ID. Together with access control policies, authentication methods and entities, leases define who may use the secrets store and for how long.

For details, see Secret management web interface.

Stronghold sealing

The state of the Deckhouse Stronghold secrets store in which it can neither read nor write secrets. The store is sealed and unsealed in the Stronghold web interface.

The “Seal Stronghold” section of the web interface shows the current state of the secrets store and a button that seals or unseals it. While the store is sealed, the stronghold module does not serve requests to read or write secrets.

For details, see Secret management web interface.

Stronghold secrets engine

A component of the Deckhouse Stronghold secrets store that holds and serves secrets of one kind, for example key-value secrets (KV) or certificates (PKI).

Secrets engines are managed in the Stronghold web interface: each engine shows its list of secrets (roles, keys and others), its configuration and a button to add a new secret. The web interface is provided by the stronghold module and opens at stronghold.<CLUSTER_NAME_TEMPLATE>.

For details, see Secret management web interface.

Subsystem

A logical part of the system that groups related modules and their system components.

Subsystems group the components of the system and delimit access to them. A subsystem consists of modules and their system components.

Suspended release

A Deckhouse Platform release whose rollout on a release channel was paused after it had been published. The release channel image is patched with a suspend flag.

While a release is suspended, d8 mirror pull refuses to copy the release channel and reports that the source registry contains a suspended release channel. The platform images can still be downloaded by specifying a version explicitly with the --deckhouse-tag flag.

For details, see Updating DP in air-gapped environment.

System

The part of the cluster that keeps it operational, manages resources and automates processes.

The system consists of subsystems, which in turn consist of modules and system components.

The system runs mostly in system namespaces. The parts of the system that serve the workload of a specific project can run in user namespaces.

System component

A process or a group of processes that run in a container or as a systemd service on a node and keep the platform and its modules running.

Examples of system components are kubelet and containerd.

Some system components run in user namespaces. For example, when a user creates a virtual machine or a managed service, the pods that support this object start in the user namespace.

System namespace

A namespace in which the system runs, that is, modules and system components. Running the workload in system namespaces is prohibited.

System namespaces of Deckhouse Platform have the prefixes d8- and kube-. The system, its modules and system components run there.

System network

An additional service network on the cluster nodes, created by the sdn module on top of an existing underlay network for node-level service traffic.

A system network is created with a SystemNetwork object. Interfaces connected to a system network get cluster-scoped addresses through ClusterIPAddress, like interfaces of cluster networks. It belongs to the node-level functions of software-defined networks, together with the configuration of node interfaces and underlay networks.

For details, see Software-defined networking (SDN).

System node

A node dedicated to running Deckhouse Platform modules so that they do not interfere with user applications.

System nodes are isolated with labels and taints so that user pods are not scheduled on them. Two system nodes are recommended. System nodes can be static or cloud nodes and usually form the system node group. Without dedicated monitoring nodes, the monitoring components also run on system nodes.

For details, see Recommendations for configuring cluster nodes.

The deckhouse system user

The local user and group with UID and GID 64535 that Deckhouse Platform components use to run unprivileged processes on nodes. The name and identifiers are reserved and not configurable.

If the account does not exist, bashible creates it without a password, sudo privileges or a login shell. The account must not be provided through AD or LDAP. Processes owned by deckhouse in ps or top output belong to Deckhouse Platform components. The preflight check accepts an absent user and group or an existing pair with the reserved identifiers and no sudo privileges. The installation fails if the name is used with other identifiers or the identifiers belong to another account.

For details, see Installation.

Thick volume

An LVM volume in a thick pool, that is, a pool of the LVM type: the space for such a volume is allocated in full at creation. A thick pool gives performance comparable to the storage device but does not allow snapshots.

Thick volumes are created in thick pools of local and replicated storage: the LVM mode of the sds-local-volume module and LVM volume groups without a thin pool definition in the sds-replicated-volume module. For volumes that need snapshots, use a thin volume.

Thin volume

An LVM volume in a thin pool, that is, a pool of the LVMThin type: the space for such a volume is allocated as it fills up. A thin pool allows snapshots but has lower performance.

Thin volumes are created in thin pools of local and replicated storage: the LVMThin mode of the sds-local-volume module and LVM volume groups with a thin pool definition in the sds-replicated-volume module. Where snapshots are not needed, a thick volume gives higher performance.

Underlay network

Direct connection of physical functions (PF) and virtual functions (VF) of the network interfaces of a node to pods through Kubernetes Dynamic Resource Allocation. It serves workloads that need direct access to hardware, such as DPDK applications.

Physical network interfaces are connected to pods in one of two modes: Shared, where Virtual Functions are created from a Physical Function with SR-IOV and several pods share the hardware, or Dedicated, where a pod gets the whole Physical Function. The administrator creates an UnderlayNetwork object of the sdn module and marks the namespace that may use it. The pod requests a device with an annotation.

DPDK, Data Plane Development Kit, is a set of libraries for processing network packets in user space, bypassing the kernel network stack.

For details, see Connecting physical network interfaces to DPDK application pods.

Update mode

The setting that defines how Deckhouse Platform applies new versions.

In the Auto mode both patch and minor versions are applied automatically, in the AutoPatch mode only patch versions within the current minor version. In the Manual mode a version is applied only after the administrator approves the corresponding DeckhouseRelease.

The mode is set in the settings.update.mode parameter of the deckhouse module. In the automatic modes updates are applied with regard to update windows.

For details, see Configuring updates.

Update window

A time interval during which Deckhouse Platform is allowed to install updates automatically. Outside the windows new versions are not installed.

Update windows are set in the update.windows parameter of the deckhouse module. If no windows are configured, an update is applied as soon as a new version appears on the release channel.

Disruptive updates of nodes have separate windows in the NodeGroup parameters disruptions.automatic.windows and disruptions.rollingUpdate.windows.

For details, see Configuring updates.

Use role

A role of the granular role model for granting users permissions within one namespace, that is, access to namespaced module resources and standard namespaced Kubernetes resources.

Use roles belong to the granular role model. They are intended for users, typically application developers, who do not manage the cluster or its modules but need to create Ingress resources, configure authentication for applications or collect logs in their namespace.

The user-authz module creates the roles d8:use:role:viewer, d8:use:role:user, d8:use:role:manager and d8:use:role:admin. A use role is granted only with RoleBinding. Administrator permissions are granted with manage roles.

For details, see Granular authorization model.

User namespace

A namespace in which the workload runs. It always belongs to one project.

A user namespace always belongs to one project and cannot have the prefix of a system namespace.

The namespaces of a project are divided into the main project namespace and additional project namespaces.

Virtual machine (VM)

A machine that Deckhouse Platform creates and manages declaratively through the virtualization module. It runs in a pod on a cluster node under QEMU/KVM, managed by KubeVirt.

A machine is described by VirtualMachine in a project or namespace. Its disks and images are attached as block devices, its CPU is set by the number of cores and the core fraction within the limits of the VM class, and its state is kept by the run policy. The guest system is configured on the first boot by provisioning, after which Deckhouse Platform communicates with it through the guest OS agent.

The module core is based on KubeVirt: the QEMU process started by libvirtd in the compute container of the machine pod is the running machine. Machines move between nodes by live migration, are grouped into pools, and go through the phases of the virtual machine lifecycle.

For details, see Virtualization subsystem.

Virtual machine block device

A disk or an image attached to a virtual machine. Images are attached read-only or as a CD-ROM for ISO images, disks are attached read-write.

Devices are attached in two ways. The blockDeviceRefs list of the virtual machine specification sets the boot order by position or by the bootOrder field and is used for manual configuration, for example an ISO for OS installation. A VirtualMachineBlockDeviceAttachment attaches a disk as a separate resource that does not take part in the boot order, over the virtio-scsi bus, and is used for automation and when the user has no rights to edit the machine.

With paravirtualization enabled, both ways attach and detach disks on a running machine without a reboot, if the storage is available on the node where the machine runs. While a live migration prepares the target node, devices can be neither attached nor detached.

For details, see Attaching disks and images to a virtual machine.

Virtual machine clone

A copy of a virtual machine created from an existing machine or from its snapshot. The clone gets a new IP address and new MAC addresses, and labels are not copied to it.

Because a clone copies the machine, the resources of the new machine need unique names: nameReplacements replaces the names of specific resources, and customization adds a prefix or a suffix to the names of all cloned resources, such as disks and IP addresses.

The clone gets a new IP address in the cluster network and new MAC addresses for additional interfaces, so the network settings of the guest OS have to be reconfigured. Labels are not copied, so the clone does not receive Service traffic until the needed labels are added. A snapshot of the machine can be the source of the clone.

For details, see Cloning virtual machines.

Virtual machine disk

Storage of virtual machine data, including the operating system and application files. A disk is described by its storage parameters, the StorageClass and the size, and by a data source, which can be an image, another disk or a snapshot.

A disk is described by VirtualDisk. Without a data source an empty disk of the given size is created. With a source, Deckhouse Platform takes the size from the source and picks the storage class by it. Disk behaviour depends on the storage behind the StorageClass: on file system volumes the disk is created in the qcow2 format, on block devices data is written directly, and the volume binding mode defines whether the disk is created right away or only after it is attached to a virtual machine.

For a ready disk only the size and the storage class can be changed. A disk is attached to a machine read-write as a block device. Its data can be captured in a snapshot or written to a file by disk data export.

For details, see Virtual machine disks.

Virtual machine image

The contents of a disk from which virtual machine disks are created. A project image is available only in the project or namespace where it was created, and a virtual machine accesses an attached image read-only.

An image is described by VirtualImage. Its source can be an HTTP server with the image file, a container image registry, a file uploaded from the command line, another image, a virtual machine disk or a disk snapshot. The downloaded image is stored either in DVCR or in a PVC. An image stored in a PVC can be used to create disks only in the same storage class.

To make an image available to every project of the cluster, an administrator creates a cluster image, ClusterVirtualImage. Images are attached to a virtual machine as block devices: read-only or as a CD-ROM for ISO images.

For details, see Virtual machine images.

Virtual machine lifecycle

The phases a virtual machine goes through from creation to deletion, shown in its status. Conditions explain why the machine is in its current phase.

In the Pending phase the machine waits for its dependent resources to become ready: disks, images, the VM class and the secret with the provisioning script. In the Starting phase Deckhouse Platform starts the machine on one of the nodes, in Running it works, in Migrating it moves to another node by live migration, and the Terminating phase is irreversible.

The .status.conditions block answers why the machine is in its phase: the conditions ending in Ready show what blocks the startup, Running reports the reason from the scheduler, Migratable shows whether the machine can be moved, EvictionRequired warns about eviction from a node. The run policy defines how the machine is kept running.

For details, see Creating a virtual machine.

Virtual machine pool

A set of identical virtual machines, replicas created from one template, whose number Deckhouse Platform maintains. A pool can be scaled through the scale subresource, HPA or KEDA. Available in commercial editions.

A pool is described by VirtualMachinePool. Its template matches a regular virtual machine specification, so a replica is no different from a manually created virtual machine. Replica disks are described once in disk templates with a reclaim policy, a size and a data source, and the template references them in the boot order. Shared images, such as one ISO for all replicas, are attached read-only.

Replicas are differentiated by provisioning, so the Legacy OS type is not supported in a pool.

For details, see Virtual machine pools.

Virtual machine snapshot

The captured state of a disk or of a whole virtual machine at a point in time, kept for recovery or cloning. A disk snapshot saves only the data of the selected disk, a VM snapshot includes the VM parameters and the state of all its disks.

A disk snapshot is described by VirtualDiskSnapshot and serves as a source for new disks and images. A virtual machine snapshot is described by VirtualMachineSnapshot and is the saved state of the machine at a certain moment. Attached images are not saved with it, so they have to be present in the cluster at recovery.

Snapshots are consistent or inconsistent, see consistent snapshot. A snapshot of a machine can also be the source of a virtual machine clone.

For details, see Snapshots of disks and virtual machines.

VM CPU topology

The distribution of the processor cores of a virtual machine across sockets. The user sets only the total number of cores, and Deckhouse Platform calculates the number of sockets.

The more cores, the more sockets they are divided into and the larger the step in which their number can be changed: up to 16 cores use one socket, up to 32 two, up to 64 four and up to 248 eight. The total number of cores has to be a multiple of the number of sockets. The topology matters for applications sensitive to the processor configuration.

The number of cores of a running machine can be changed within the current topology by live migration when the corresponding feature is enabled. A change that requires a different topology needs a restart.

For details, see Virtual machine CPU and memory.

VM disk data export

Writing the contents of a virtual machine disk or of its snapshot to a file so that the data can be moved outside the cluster.

Export requires the enabled storage-volume-data-manager module and, from the command line, the d8 utility version 0.20.7 or later. The export is performed by the d8 data export download command. The disk must not be in use at the moment of export, so a machine it is attached to is stopped first. A snapshot can be exported instead of the disk.

A downloaded file is imported back into the cluster by uploading it as an image or as a disk.

For details, see Exporting virtual machine disk data.

VM eviction

Moving or restarting a virtual machine to free its node. It is triggered when the node is put into maintenance mode, and a machine that can be moved by live migration is not restarted.

The EvictionRequired condition appears when the node with the machine is put into node maintenance mode. A node that was only cordoned does not trigger it. The condition message says what will happen: a live migration without stopping the guest OS, a restart by Deckhouse Platform with the administrator’s permission, or waiting until the machine is restarted. Until the eviction starts, the condition is only a warning, because maintenance can be cancelled.

After a restart the machine starts on another suitable node, and without one it stays in the Pending phase. A user can also evict a machine to another node with d8 v evict or a VirtualMachineOperation of the Evict type.

For details, see Creating a virtual machine.

VM migration policy

The policy that determines when a live migration of a virtual machine uses the AutoConverge mechanism, CPU throttling that guarantees the migration completes. Set by the liveMigrationPolicy parameter of the machine.

AutoConverge gradually lowers the CPU frequency of the source machine so that fewer dirty memory pages appear, and completes the live migration as soon as the transfer rate exceeds the memory change rate. It helps on a slow network at the cost of application performance in the machine.

The allowed values: AlwaysSafe (never throttle), PreferSafe (the default, no throttling, but a migration with force=true may throttle), AlwaysForced (always use AutoConverge) and PreferForced (use AutoConverge, but a migration with force=false runs without it).

For details, see Live migration of virtual machines.

VM placement rules

The rules that control on which node a virtual machine starts, from nodeSelector, node affinity, and affinity and anti-affinity to other machines and pods. Conditions are hard or soft.

A hard condition is mandatory: if no node satisfies it, the machine stays in the Pending phase. A soft condition is taken into account by the scheduler where possible. All rules, including the nodeSelector of the VM class, apply together. Nodes with taints need matching tolerations on the machine.

When the rules of a running machine change and its current node no longer meets them, commercial editions move the machine by live migration, and in Deckhouse Platform Open the change applies after a reboot. The placement rules of the machine, its disks and its class are also checked when a block device is attached. VM rebalancing restores the correct placement when the node stops meeting the rules.

For details, see Placing virtual machines on nodes.

VM pool disk reclaim policy

The reclaim policy of a disk template of a virtual machine pool. It sets what happens to a replica disk when the replica is removed from the pool.

The policy is set in the virtualDiskTemplates block of a virtual machine pool together with the size and the data source of the replica disk. With Delete, the default, the disk is deleted together with the replica. With Retain, the disk is kept and reused by the next replica.

For details, see Virtual machine pools.

VM provisioning

Initial configuration of the guest system of a virtual machine on its first boot by an initialization script, Cloud-Init for Linux or Sysprep for Windows.

The initialization script creates users, sets passwords and SSH keys, installs software, runs commands and enables services, for example the guest OS agent. The Cloud-Init configuration is written in YAML and starts with the #cloud-config header.

Replicas of a virtual machine pool are differentiated by provisioning, so operating systems without it, the Legacy OS type, are not supported in pools.

For details, see Initial provisioning and the guest OS agent.

VM rebalancing

Restoring an even distribution of virtual machines across nodes by moving them with live migration. Performed by the descheduler module.

Rebalancing evens out the load: when a node reserves more than 80% of CPU, some machines are moved to nodes that reserve less than 50%. It also restores correct placement when the current node no longer meets the machine requirements or the rules of mutual placement. With the descheduler module enabled, Deckhouse Platform creates a Descheduler named virtualization and selects in it only the machines that can be moved by live migration.

For details, see Virtual machine fault tolerance and balancing.

VM run policy

The policy that determines how Deckhouse Platform maintains the state of a virtual machine, set by the runPolicy parameter of the machine.

The allowed values: AlwaysOnUnlessStoppedManually (the default, the machine always runs and can be stopped only manually), AlwaysOn (the machine is started again even after a shutdown from the guest OS), Manual (the user manages the state) and AlwaysOff (the machine is always off and cannot be started).

The state of a virtual machine is managed by VirtualMachineOperation or by the d8 v start, d8 v stop, d8 v restart and d8 v evict commands, which create the same operation.

For details, see Connecting to a virtual machine.

Vulnerability database

Databases of known vulnerabilities used by image scanning with Trivy. The operator-trivy module downloads them from the registry every 6 hours. They take about 5 GB on the system nodes.

Scanning uses public vulnerability databases. The database images have fixed names and tags in the registry, so they can be copied into a private registry for an air-gapped cluster. The databases are stored on the system nodes. If a cluster has none, they are placed on a worker node. See also vulnerability scanning.

For details, see Disk layout and size.

Vulnerability scanning

Automatic detection of known vulnerabilities (CVE) and CIS compliance checks for the container images used by pods of the cluster, based on Trivy.

Deckhouse Platform scans the container images of all resources in namespaces labelled security-scanning.deckhouse.io/enabled="". Without such labels only the default namespace is scanned. Scanning starts every 24 hours and when components with new images are deployed. The result of scanning each image is stored in the namespace of the scanned resource.

For details, see Container image vulnerability scanning.

Worker node

A node intended to run user applications. Its resources are dictated by the workload, with a part of them reserved for kubelet and system pods.

Worker nodes are present in every deployment option except the single-node cluster, where the only node takes their role. In a single-master cluster, some Deckhouse Platform components also run on worker nodes. The actual node size depends on the applications running on it.

For details, see the Resource requirements section of the “System requirements” page.

Workload

All projects together with the disk, network and other resources they use.

The workload runs in user namespaces. Running the workload in system namespaces is prohibited.