The following recommendations are essential for a production cluster and may be irrelevant for a testing or development one.

Release channel and update mode

Use Early Access or Stable release channel. Configure the auto-update window or select manual mode.

Select the release channel and update mode in accordance with your requirements. The more stable the release channel is, the later new features appear in it.

If possible, use separate release channels for different clusters. For a development cluster, use a less stable release channel than for a testing or stage (pre-production) cluster.

We recommend using the Early Access or Stable release channel for production clusters. If you have more than one cluster in a production environment, consider using separate release channels for them. For example, you can use Early Access for the first cluster and Stable for the second. If you can’t use separate release channels, we recommend setting update windows so that they don’t overlap.

Even in very busy and critical clusters, it’s recommended that you don’t disable the release channel. The best strategy is to configure scheduled updates. In clusters that haven’t been updated for six months or more, there may be bugs that, as a rule, have already been fixed in newer versions. In such a case, it will be difficult to resolve the issue promptly.

The update windows management lets you schedule automatic platform release updates during periods of low cluster activity.

Kubernetes version

Use the automatic Kubernetes version selection or set the version explicitly.

In most cases, we recommend opting for the automatic selection of the Kubernetes version. In the platform, this behavior is set by default, but it can be changed with the kubernetesVersion parameter. Upgrading the Kubernetes version in the cluster has no effect on applications and is done consistently and securely.

If automatic Kubernetes version selection is enabled, the platform can upgrade the Kubernetes version in the cluster when updating the platform release (when upgrading a minor version).

If the Kubernetes version is explicitly specified in the kubernetesVersion parameter, the platform update may fail if the Kubernetes version used in the cluster is no longer supported.

If your application uses outdated resource versions or depends on a particular version of Kubernetes for some other reason, check whether it’s supported and set it explicitly.

Cluster architecture selection

Deckhouse Virtualization Platform supports several cluster architecture options — from single-node installations to distributed configurations. The choice of a specific option depends on the requirements: the need for quick test deployment, high availability, or isolation of system services from user workloads.

Single-node cluster (Single Node / Edge): All management components, auxiliary services, and virtual machines are placed on a single server. Suitable for test environments and edge scenarios. Requires minimal resources but does not provide fault tolerance.

Cluster with one master node and worker nodes: One node performs management functions, virtual machines are placed on dedicated worker nodes. Suitable for small clusters where separation of system services and user workloads is required. Fault tolerance is absent.

Three-node cluster (High Availability): Management components are distributed across three master nodes, which ensures fault tolerance of the control plane and continued operation if one of the nodes fails. User workloads can run on these same servers or on dedicated worker nodes. Recommended for production environments.

Highly available distributed cluster: Management components are deployed on three dedicated master nodes; if necessary, system services, monitoring, and ingress are moved to separate system or frontend nodes. User virtual machines run exclusively on worker hypervisors. Provides high availability, scalability, and isolation of user workloads from system services. Used in large clusters.

See also the Architecture options section.

Resource requirements

Before deploying a cluster, you need to plan the resources that may be required for its operation. To do this, answer a few questions:

  • What workload is planned for the cluster?
  • How many virtual machines are planned to be launched?
  • Does the cluster require high availability mode?
  • What type of storage will be used (SDS or external)?

Answers to these questions will help determine the required cluster architecture and node resources.

When using software-defined storage (SDS) on nodes that participate in storage organization, you need to provide additional disks beyond the specified minimum requirements. These disks will be used by SDS to store virtual machine data.

Depending on the architecture, the following minimum resources are required for the platform to operate correctly:

Architecture Workload placement Master node Worker node System node Frontend node
Single-node platform
(Single Node / Edge)		 On a single node 3 vCPU
10 GB RAM		 — — —
Multi-node platform
(1 master node + worker nodes)		 On all nodes 6 vCPU
6 GB RAM		 2 vCPU
4 GB RAM		 — —
Three-master platform
(3 master nodes, High Availability)		 On all nodes 6 vCPU
14 GB RAM		 — — —
Platform with dedicated worker nodes
(3 master nodes + worker nodes)		 On worker nodes only 5 vCPU
11 GB RAM		 2 vCPU
5 GB RAM		 — —
Distributed architecture On worker nodes only 4 vCPU
9 GB RAM		 1 vCPU
2 GB RAM		 4 vCPU
10 GB RAM		 1 vCPU
2 GB RAM

The number of virtual machines that can be run on nodes is limited by the maxPods parameter in the NodeGroup resource properties. When planning the number of VMs, keep in mind that the maxPods limit includes all pods on the node: system pods, containerized workloads, and virtual machines. Each virtual machine occupies one pod in the cluster.

Minimum requirements are specified for the basic platform configuration. When increasing the load, the number of virtual machines, or enabling additional modules, it may be necessary to increase node resources.

Configuration features

Cluster subnet planning

All cluster subnets must not overlap with each other.

Define IP address subnets for the cluster:

  • Node subnet: Used by nodes to communicate with each other. This is the only subnet that physically exists in the organization’s network and is routed in your infrastructure. Must be a real network in your datacenter.
  • Pod subnet (podSubnetCIDR): Virtual network inside the cluster, used to assign IP addresses to Kubernetes pods (including system pods and containerized workloads).
  • Service subnet (serviceSubnetCIDR): Virtual network inside the cluster, used to assign IP addresses to Kubernetes ClusterIP services for intra-cluster communication.
  • Virtual machine address subnets (virtualMachineCIDRs): Virtual networks inside the cluster, used to assign IP addresses to virtual machines. Multiple subnets can be specified.

See also the VM Network section.

Master nodes

A cluster must include three master nodes with fast 400+ IOPS disks.

Always use three master nodes — this number ensures fault tolerance and allows safe updates of master nodes. There is no need for more master nodes, and two nodes will not provide quorum.

If you need to run workloads (virtual machines) on control plane nodes, which is typical for Single-node cluster (Single Node / Edge) and Three-node cluster (High Availability) configurations, you need to configure tolerations in the virtual machine configuration or in the virtual machine class to allow VMs to be placed on master nodes.

See also the Working with static nodes section.

Frontend nodes

Use two or more frontend nodes.

Use the HostPort inlet with an external load balancer.

Frontend nodes balance incoming traffic. Ingress controllers run on them. The NodeGroup of frontend nodes has the node-role.deckhouse.io/frontend label. For details on allocating nodes for specific load types, refer to Advanced scheduling.

Use more than one frontend node. Frontend nodes must be able to continue handling traffic even if one of them fails.

For example, if the cluster has two frontend nodes, each frontend node must handle the entire cluster load in case the second node fails. If the cluster has three frontend nodes, each frontend node must handle at least one and a half times the load increase.

The platform supports three ways of receiving traffic from the external world:

  • HostPort: Installs an Ingress controller accessible on node ports via hostPort.
  • HostPortWithProxyProtocol: Installs an Ingress controller accessible on node ports via hostPort and uses proxy-protocol to obtain the client’s real IP address.
  • HostWithFailover: Installs two Ingress controllers (primary and standby).

The HostWithFailover inlet is suitable for clusters with a single frontend node. It reduces downtime for the Ingress controller during updates. This inlet type is suitable for important development environments, however, it is not recommended for production.

Monitoring nodes

For high-load clusters, use two monitoring nodes equipped with fast disks.

Monitoring nodes are used to run Grafana, Prometheus, and other monitoring components. The NodeGroup for monitoring nodes has the node-role.deckhouse.io/monitoring label.

In high-load clusters with many alerts and large volumes of metrics, we recommend allocating dedicated nodes for monitoring. If this is not done, monitoring components will be placed on system nodes.

When allocating monitoring nodes, it’s important to equip them with fast disks. You can do so by providing a dedicated storageClass on fast disks for all platform components (the storageClass global parameter) or allocating a dedicated storageClass to monitoring components only (the storageClass and longtermStorageClass parameters of the prometheus module).

If the cluster is initially created with nodes allocated for a specific type of workload (system nodes, monitoring nodes, etc.), we recommend that you explicitly specify the corresponding nodeSelector in the configuration of modules using persistent storage volumes. For example, for the prometheus module, this parameter is nodeSelector.

System nodes

Use two system nodes.

System nodes are used to run platform modules. The NodeGroup for system nodes has the node-role.deckhouse.io/system label.

Allocating two system nodes ensures that platform modules run without interfering with user applications in the cluster. For details on allocating nodes for specific load types, refer to Advanced scheduling.

Fast disks are recommended for platform components (the storageClass global parameter).

Storage configuration

Configure one or more storage systems for virtual machine disks:

  • Software-defined storage (SDS):
    • [sds-local-volume](../../../modules/sds-local-volume/stable/): Local storage based on LVM. High performance but without replication. Suitable for temporary data or when external backup is available.
    • [sds-replicated-volume](../../../modules/sds-replicated-volume/stable/): Replicated storage based on DRBD. Provides fault tolerance through replication between nodes. Recommended for production.
  • External storage: Ceph, NFS, TATLIN.UNIFIED (Yadro), Huawei Dorado, HPE 3par, iSCSI. Connected via corresponding CSI drivers.

When using SDS on nodes participating in storage organization, you need to provide additional disks beyond the minimum requirements for node resources. The size of additional disks depends on the planned volume of virtual machine data.

See also the Storage configuration section.

Virtual machine class configuration

Create your own virtual machine class (one or more).

The default generic class is not recommended for production.

Configure sizing policies to control VM resources:

  • Use type: Host for identical nodes (same processor architecture).
  • Use type: Discovery for different processor types.

Sizing policies limit the allowed VM resource configurations (number of cores, memory, core usage fraction) and prevent creating VMs with suboptimal configurations. The coreFractions parameter controls CPU overcommit: by setting the minimum core usage fraction, you guarantee each VM a corresponding portion of CPU and thereby limit the maximum allowed resource oversubscription level.

See also the VirtualMachineClass settings section.

Access control

Configure user authentication and access control. For production, it is recommended to use projects with role model configuration.

The platform supports managing internal users and groups, as well as integration with external authentication providers:

  • Internal users and groups: Created through User and Group resources of the user-authn module.
  • External authentication providers: LDAP, OIDC, GitHub, GitLab, Crowd, Bitbucket Cloud. Multiple providers can be connected simultaneously.

See also the Integration with external authentication systems section.

Configure projects and access rights in accordance with the planned use of the cluster. Projects (the Project resource) provide isolated environments for creating user resources. Project settings allow you to set resource quotas, limit network interaction, and configure a security profile.

Access control is configured through a role model using the standard Kubernetes RBAC approach. You can use existing roles or create your own:

  • Manage roles: For platform administrators. Allow configuring the cluster, managing virtual machines at the platform level, and creating projects (tenants) for users.
  • Use roles: For project users. Allow managing resources (including virtual machines) within assigned projects.

See also the Projects and Role model sections.

Monitoring event notifications

Configure alert delivery via the internal Alertmanager or connect an external one.

Monitoring will work immediately after platform installation, but this is not enough for production. To receive notifications about incidents, configure the built-in Alertmanager in the platform or connect your own Alertmanager.

Using the CustomAlertmanager custom resource, you can configure sending notifications to e-mail, Slack, Telegram, via webhooks, or by other means.

The list of all available alerts in the monitoring system is provided on a separate documentation page.

Backups

Be sure to set up etcd data backups — this is your last chance to restore the cluster in case of unforeseen events. Keep backups as far away from the cluster as possible.

The backups won’t help if they don’t work or if you don’t know how to use them for recovery. We recommend compiling a Disaster Recovery Plan with specific steps and commands to restore the cluster from a backup.

This plan should be periodically updated and tested in drills.

Community

Follow the project news on Telegram.

Join the community to stay up to date with important changes and news. You will be able to communicate with people engaged in the same work and avoid many typical problems.

The platform team knows how much effort it takes to organize a production cluster in Kubernetes. We will be glad if the platform allows you to realize your plans. Share your experience and inspire others to switch to Kubernetes.