Deckhouse Kubernetes Platform on VMware vSphere

Before starting the installation, ensure you have the cloud provider quotas required to deploy your cluster.

List of required vSphere resources

  • User with required set of permissions.
  • Network with DHCP server and access to the Internet
  • Datacenter with a tag in k8s-region category.
  • Cluster with a tag in k8s-zone category.
  • Datastore with required tags.
  • Template — the prepared VM image.

vSphere configuration

Installing govc

You’ll need the vSphere CLI — govc — to proceed with the rest of the guide.

After the installation is complete, set the environment variables required to work with vCenter:

export GOVC_URL=example.com
export GOVC_USERNAME=<username>@vsphere.local
export GOVC_PASSWORD=<password>
export GOVC_INSECURE=1
export GOVC_URL=example.com export GOVC_USERNAME=<username>@vsphere.local export GOVC_PASSWORD=<password> export GOVC_INSECURE=1

Creating tags and tag categories

Instead of “regions” and “zones”, VMware vSphere provides Datacenter and Cluster objects. We will use tags to match them with “regions”/”zones”. These tags fall into two categories: one for “regions” tags and the other for “zones” tags.

Create a tag category using the following commands:

govc tags.category.create -d "Kubernetes Region" k8s-region
govc tags.category.create -d "Kubernetes Zone" k8s-zone
govc tags.category.create -d "Kubernetes Region" k8s-region govc tags.category.create -d "Kubernetes Zone" k8s-zone

Create tags in each category. If you intend to use multiple “zones” (Cluster), create a tag for each one of them:

govc tags.create -d "Kubernetes Region" -c k8s-region test-region
govc tags.create -d "Kubernetes Zone Test 1" -c k8s-zone test-zone-1
govc tags.create -d "Kubernetes Zone Test 2" -c k8s-zone test-zone-2
govc tags.create -d "Kubernetes Region" -c k8s-region test-region govc tags.create -d "Kubernetes Zone Test 1" -c k8s-zone test-zone-1 govc tags.create -d "Kubernetes Zone Test 2" -c k8s-zone test-zone-2

Attach the “region” tag to Datacenter:

govc tags.attach -c k8s-region test-region /<DatacenterName>
govc tags.attach -c k8s-region test-region /<DatacenterName>

Attach “zone” tags to Cluster objects:

govc tags.attach -c k8s-zone test-zone-1 /<DatacenterName>/host/<ClusterName1>
govc tags.attach -c k8s-zone test-zone-2 /<DatacenterName>/host/<ClusterName2>
govc tags.attach -c k8s-zone test-zone-1 /<DatacenterName>/host/<ClusterName1> govc tags.attach -c k8s-zone test-zone-2 /<DatacenterName>/host/<ClusterName2>

Datastore configuration

For dynamic PersistentVolume provisioning, a Datastore must be available on each ESXi host (shared datastore).

Assign the “region” and “zone” tags to the Datastore objects to automatically create a StorageClass in the Kubernetes cluster:

govc tags.attach -c k8s-region test-region /<DatacenterName>/datastore/<DatastoreName1>
govc tags.attach -c k8s-zone test-zone-1 /<DatacenterName>/datastore/<DatastoreName1>

govc tags.attach -c k8s-region test-region /<DatacenterName>/datastore/<DatastoreName1>
govc tags.attach -c k8s-zone test-zone-2 /<DatacenterName>/datastore/<DatastoreName2>
govc tags.attach -c k8s-region test-region /<DatacenterName>/datastore/<DatastoreName1> govc tags.attach -c k8s-zone test-zone-1 /<DatacenterName>/datastore/<DatastoreName1> govc tags.attach -c k8s-region test-region /<DatacenterName>/datastore/<DatastoreName1> govc tags.attach -c k8s-zone test-zone-2 /<DatacenterName>/datastore/<DatastoreName2>

Creating and assigning a role

We’ve intentionally skipped User creation since there are many ways to authenticate a user in the vSphere.

This all-encompassing Role should be enough for all Deckhouse components. For a detailed list of privileges, refer to the documentation. If you need a more granular Role, please contact your Deckhouse support.

Create a role with the corresponding permissions:

govc role.create deckhouse \
   Cns.Searchable Datastore.AllocateSpace Datastore.Browse Datastore.FileManagement \
   Global.GlobalTag Global.SystemTag Network.Assign StorageProfile.View \
   $(govc role.ls Admin | grep -F -e 'Folder.' -e 'InventoryService.' -e 'Resource.' -e 'VirtualMachine.')
govc role.create deckhouse \ Cns.Searchable Datastore.AllocateSpace Datastore.Browse Datastore.FileManagement \ Global.GlobalTag Global.SystemTag Network.Assign StorageProfile.View \ $(govc role.ls Admin | grep -F -e 'Folder.' -e 'InventoryService.' -e 'Resource.' -e 'VirtualMachine.')

Assign the role to a user on the vCenter object:

govc permissions.set -principal <username>@vsphere.local -role deckhouse /
govc permissions.set -principal <username>@vsphere.local -role deckhouse /

Preparing a virtual machine image

It is recommended to use a pre-built cloud image/OVA file provided by the OS vendor to create a Template:

If you need to use your own image, please refer to the documentation.