Deckhouse Kubernetes Platform on VMware vSphere
Before starting the installation, ensure you have the cloud provider quotas required to deploy your cluster.
The cloud-init package must be installed on the VMs.
After the virtual machine is started, the following services associated with this package must be started:
cloud-config.service;cloud-final.service;cloud-init.service.
List of required vSphere resources
Deckhouse uses the ens192 interface as the default interface for virtual machines in vSphere. Therefore, when using static IP addresses in mainNetwork, you must create an interface named ens192 in the OS image as the default interface.
- User with required set of permissions.
- Network with DHCP server and access to the Internet
- Datacenter with a tag in
k8s-regioncategory. - Cluster with a tag in
k8s-zonecategory. - Datastore with required tags.
- Template — the prepared VM image.
vSphere configuration
Installing govc
You’ll need the vSphere CLI — govc — to proceed with the rest of the guide.
After the installation is complete, set the environment variables required to work with vCenter:
export GOVC_URL=example.com
export GOVC_USERNAME=<username>@vsphere.local
export GOVC_PASSWORD=<password>
export GOVC_INSECURE=1
Creating tags and tag categories
Instead of “regions” and “zones”, VMware vSphere provides Datacenter and Cluster objects. We will use tags to match them with “regions”/”zones”. These tags fall into two categories: one for “regions” tags and the other for “zones” tags.
Create a tag category using the following commands:
govc tags.category.create -d "Kubernetes Region" k8s-region
govc tags.category.create -d "Kubernetes Zone" k8s-zone
Create tags in each category. If you intend to use multiple “zones” (Cluster), create a tag for each one of them:
govc tags.create -d "Kubernetes Region" -c k8s-region test-region
govc tags.create -d "Kubernetes Zone Test 1" -c k8s-zone test-zone-1
govc tags.create -d "Kubernetes Zone Test 2" -c k8s-zone test-zone-2
Attach the “region” tag to Datacenter:
govc tags.attach -c k8s-region test-region /<DatacenterName>
Attach “zone” tags to Cluster objects:
govc tags.attach -c k8s-zone test-zone-1 /<DatacenterName>/host/<ClusterName1>
govc tags.attach -c k8s-zone test-zone-2 /<DatacenterName>/host/<ClusterName2>
Datastore configuration
For dynamic PersistentVolume provisioning, a Datastore must be available on each ESXi host (shared datastore).
Assign the “region” and “zone” tags to the Datastore objects to automatically create a StorageClass in the Kubernetes cluster:
govc tags.attach -c k8s-region test-region /<DatacenterName>/datastore/<DatastoreName1>
govc tags.attach -c k8s-zone test-zone-1 /<DatacenterName>/datastore/<DatastoreName1>
govc tags.attach -c k8s-region test-region /<DatacenterName>/datastore/<DatastoreName2>
govc tags.attach -c k8s-zone test-zone-2 /<DatacenterName>/datastore/<DatastoreName2>
Creating and assigning a role
We’ve intentionally skipped User creation since there are many ways to authenticate a user in the vSphere.
This all-encompassing Role should be enough for all Deckhouse components. For a detailed list of privileges, refer to the documentation. If you need a more granular Role, please contact your Deckhouse support.
Create a role with the corresponding permissions:
govc role.create deckhouse \
Cns.Searchable Datastore.AllocateSpace Datastore.Browse Datastore.FileManagement \
Global.GlobalTag Global.SystemTag Network.Assign StorageProfile.View \
$(govc role.ls Admin | grep -F -e 'Folder.' -e 'InventoryService.' -e 'Resource.' -e 'VirtualMachine.')
Assign the role to a user on the vCenter object:
govc permissions.set -principal <username>@vsphere.local -role deckhouse /
Preparing a virtual machine image
It is recommended to use a pre-built cloud image/OVA file provided by the OS vendor to create a Template:
- Ubuntu
- Debian
- CentOS
- Rocky Linux (Generic Cloud / OpenStack section)
If you need to use your own image, please refer to the documentation.