Deckhouse Platform on VMware vSphere

1
Selecting infrastructure
2
Installation information
3
Preparing environment
4
Installation
5
Getting access to the cluster
6
What can I do next?

Before starting the installation, ensure you have the cloud provider quotas required to deploy your cluster..

List of required vSphere resources

  • User with required set of permissions.
  • Network with DHCP server and access to the Internet
  • Datacenter with a tag in k8s-region category.
  • ComputeCluster with a tag in k8s-zone.
  • Datastore with required tags.
  • Template — prepared VM image.

vSphere configuration

You’ll need the vSphere CLI — govc to proceed with the rest of the guide.

govc configuration

Copy content
export GOVC_URL=example.com
export GOVC_USERNAME=<USER_NAME>
export GOVC_PASSWORD=<USER_PASSWORD>
export GOVC_INSECURE=1
export GOVC_URL=example.com export GOVC_USERNAME=<USER_NAME> export GOVC_PASSWORD=<USER_PASSWORD> export GOVC_INSECURE=1

Creating tags and tag categories

VMware vSphere doesn’t have “regions” and “zones”. It has Datacenters and ComputeClusters.

To establish relation between these and “regions”/”zones” we’ll use tags that fall into two tag categories. One for “region” tags and another for “zones tags”.

For example, if you’ve got two Datacenters with a similarly named zone in each one:

Copy content
govc tags.category.create -d "Kubernetes region" k8s-region
govc tags.category.create -d "Kubernetes zone" k8s-zone
govc tags.create -d "Kubernetes Region #1" -c k8s-region test_region_1
govc tags.create -d "Kubernetes Region #2" -c k8s-region test_region_2
govc tags.create -d "Kubernetes Zone Test" -c k8s-zone test_zone
govc tags.category.create -d "Kubernetes region" k8s-region govc tags.category.create -d "Kubernetes zone" k8s-zone govc tags.create -d "Kubernetes Region #1" -c k8s-region test_region_1 govc tags.create -d "Kubernetes Region #2" -c k8s-region test_region_2 govc tags.create -d "Kubernetes Zone Test" -c k8s-zone test_zone

“Region” tags are attached to Datacenters:

Copy content
govc tags.attach -c k8s-region test_region_1 /DC1
govc tags.attach -c k8s-region test_region_2 /DC2
govc tags.attach -c k8s-region test_region_1 /DC1 govc tags.attach -c k8s-region test_region_2 /DC2

“Zone” tags are attached to ComputeClusters and Datastores:

Copy content
govc tags.attach -c k8s-zone test_zone /DC/host/test_cluster
govc tags.attach -c k8s-zone test_zone /DC/datastore/test_lun
govc tags.attach -c k8s-zone test_zone /DC/host/test_cluster govc tags.attach -c k8s-zone test_zone /DC/datastore/test_lun

Datastore tags

You can dynamically provision PVs (via PVCs) if all Datastores are present on all ESXis in a selected zone (ComputeCluster). StorageClasses will be created automatically for each Datastore that is tagged with region and zone tags.

Permissions

We’ve intentionally skipped User creation since there are many ways to authenticate a user in the vSphere.

This all-encompassing Role should be enough for all Deckhouse components. If you need a more granular Role, please contact your Deckhouse support.

You have to create a role with a following list of permissions and attach it to vCenter.

Copy content
govc role.create kubernetes \
  Datastore.AllocateSpace Datastore.Browse Datastore.FileManagement Folder.Create Global.GlobalTag Global.SystemTag \
  InventoryService.Tagging.AttachTag InventoryService.Tagging.CreateCategory InventoryService.Tagging.CreateTag \
  InventoryService.Tagging.DeleteCategory InventoryService.Tagging.DeleteTag InventoryService.Tagging.EditCategory \
  InventoryService.Tagging.EditTag InventoryService.Tagging.ModifyUsedByForCategory InventoryService.Tagging.ModifyUsedByForTag \
  InventoryService.Tagging.ObjectAttachable \
  Network.Assign Resource.AssignVMToPool Resource.ColdMigrate Resource.HotMigrate Resource.CreatePool \
  Resource.DeletePool Resource.RenamePool Resource.EditPool Resource.MovePool StorageProfile.View System.Anonymous System.Read System.View \
  VirtualMachine.Config.AddExistingDisk VirtualMachine.Config.AddNewDisk VirtualMachine.Config.AddRemoveDevice \
  VirtualMachine.Config.AdvancedConfig VirtualMachine.Config.Annotation VirtualMachine.Config.CPUCount \
  VirtualMachine.Config.ChangeTracking VirtualMachine.Config.DiskExtend VirtualMachine.Config.DiskLease \
  VirtualMachine.Config.EditDevice VirtualMachine.Config.HostUSBDevice VirtualMachine.Config.ManagedBy \
  VirtualMachine.Config.Memory VirtualMachine.Config.MksControl VirtualMachine.Config.QueryFTCompatibility \
  VirtualMachine.Config.QueryUnownedFiles VirtualMachine.Config.RawDevice VirtualMachine.Config.ReloadFromPath \
  VirtualMachine.Config.RemoveDisk VirtualMachine.Config.Rename VirtualMachine.Config.ResetGuestInfo \
  VirtualMachine.Config.Resource VirtualMachine.Config.Settings VirtualMachine.Config.SwapPlacement \
  VirtualMachine.Config.ToggleForkParent VirtualMachine.Config.UpgradeVirtualHardware VirtualMachine.GuestOperations.Execute \
  VirtualMachine.GuestOperations.Modify VirtualMachine.GuestOperations.ModifyAliases VirtualMachine.GuestOperations.Query \
  VirtualMachine.GuestOperations.QueryAliases VirtualMachine.Hbr.ConfigureReplication VirtualMachine.Hbr.MonitorReplication \
  VirtualMachine.Hbr.ReplicaManagement VirtualMachine.Interact.AnswerQuestion VirtualMachine.Interact.Backup \
  VirtualMachine.Interact.ConsoleInteract VirtualMachine.Interact.CreateScreenshot VirtualMachine.Interact.CreateSecondary \
  VirtualMachine.Interact.DefragmentAllDisks VirtualMachine.Interact.DeviceConnection VirtualMachine.Interact.DisableSecondary \
  VirtualMachine.Interact.DnD VirtualMachine.Interact.EnableSecondary VirtualMachine.Interact.GuestControl \
  VirtualMachine.Interact.MakePrimary VirtualMachine.Interact.Pause VirtualMachine.Interact.PowerOff \
  VirtualMachine.Interact.PowerOn VirtualMachine.Interact.PutUsbScanCodes VirtualMachine.Interact.Record \
  VirtualMachine.Interact.Replay VirtualMachine.Interact.Reset VirtualMachine.Interact.SESparseMaintenance \
  VirtualMachine.Interact.SetCDMedia VirtualMachine.Interact.SetFloppyMedia VirtualMachine.Interact.Suspend \
  VirtualMachine.Interact.TerminateFaultTolerantVM VirtualMachine.Interact.ToolsInstall \
  VirtualMachine.Interact.TurnOffFaultTolerance VirtualMachine.Inventory.Create \
  VirtualMachine.Inventory.CreateFromExisting VirtualMachine.Inventory.Delete \
  VirtualMachine.Inventory.Move VirtualMachine.Inventory.Register VirtualMachine.Inventory.Unregister \
  VirtualMachine.Namespace.Event VirtualMachine.Namespace.EventNotify VirtualMachine.Namespace.Management \
  VirtualMachine.Namespace.ModifyContent VirtualMachine.Namespace.Query VirtualMachine.Namespace.ReadContent \
  VirtualMachine.Provisioning.Clone VirtualMachine.Provisioning.CloneTemplate VirtualMachine.Provisioning.CreateTemplateFromVM \
  VirtualMachine.Provisioning.Customize VirtualMachine.Provisioning.DeployTemplate VirtualMachine.Provisioning.DiskRandomAccess \
  VirtualMachine.Provisioning.DiskRandomRead VirtualMachine.Provisioning.FileRandomAccess VirtualMachine.Provisioning.GetVmFiles \
  VirtualMachine.Provisioning.MarkAsTemplate VirtualMachine.Provisioning.MarkAsVM VirtualMachine.Provisioning.ModifyCustSpecs \
  VirtualMachine.Provisioning.PromoteDisks VirtualMachine.Provisioning.PutVmFiles VirtualMachine.Provisioning.ReadCustSpecs \
  VirtualMachine.State.CreateSnapshot VirtualMachine.State.RemoveSnapshot VirtualMachine.State.RenameSnapshot VirtualMachine.State.RevertToSnapshot \
  Cns.Searchable StorageProfile.View

govc permissions.set  -principal username -role kubernetes /DC
govc role.create kubernetes \ Datastore.AllocateSpace Datastore.Browse Datastore.FileManagement Folder.Create Global.GlobalTag Global.SystemTag \ InventoryService.Tagging.AttachTag InventoryService.Tagging.CreateCategory InventoryService.Tagging.CreateTag \ InventoryService.Tagging.DeleteCategory InventoryService.Tagging.DeleteTag InventoryService.Tagging.EditCategory \ InventoryService.Tagging.EditTag InventoryService.Tagging.ModifyUsedByForCategory InventoryService.Tagging.ModifyUsedByForTag \ InventoryService.Tagging.ObjectAttachable \ Network.Assign Resource.AssignVMToPool Resource.ColdMigrate Resource.HotMigrate Resource.CreatePool \ Resource.DeletePool Resource.RenamePool Resource.EditPool Resource.MovePool StorageProfile.View System.Anonymous System.Read System.View \ VirtualMachine.Config.AddExistingDisk VirtualMachine.Config.AddNewDisk VirtualMachine.Config.AddRemoveDevice \ VirtualMachine.Config.AdvancedConfig VirtualMachine.Config.Annotation VirtualMachine.Config.CPUCount \ VirtualMachine.Config.ChangeTracking VirtualMachine.Config.DiskExtend VirtualMachine.Config.DiskLease \ VirtualMachine.Config.EditDevice VirtualMachine.Config.HostUSBDevice VirtualMachine.Config.ManagedBy \ VirtualMachine.Config.Memory VirtualMachine.Config.MksControl VirtualMachine.Config.QueryFTCompatibility \ VirtualMachine.Config.QueryUnownedFiles VirtualMachine.Config.RawDevice VirtualMachine.Config.ReloadFromPath \ VirtualMachine.Config.RemoveDisk VirtualMachine.Config.Rename VirtualMachine.Config.ResetGuestInfo \ VirtualMachine.Config.Resource VirtualMachine.Config.Settings VirtualMachine.Config.SwapPlacement \ VirtualMachine.Config.ToggleForkParent VirtualMachine.Config.UpgradeVirtualHardware VirtualMachine.GuestOperations.Execute \ VirtualMachine.GuestOperations.Modify VirtualMachine.GuestOperations.ModifyAliases VirtualMachine.GuestOperations.Query \ VirtualMachine.GuestOperations.QueryAliases VirtualMachine.Hbr.ConfigureReplication VirtualMachine.Hbr.MonitorReplication \ VirtualMachine.Hbr.ReplicaManagement VirtualMachine.Interact.AnswerQuestion VirtualMachine.Interact.Backup \ VirtualMachine.Interact.ConsoleInteract VirtualMachine.Interact.CreateScreenshot VirtualMachine.Interact.CreateSecondary \ VirtualMachine.Interact.DefragmentAllDisks VirtualMachine.Interact.DeviceConnection VirtualMachine.Interact.DisableSecondary \ VirtualMachine.Interact.DnD VirtualMachine.Interact.EnableSecondary VirtualMachine.Interact.GuestControl \ VirtualMachine.Interact.MakePrimary VirtualMachine.Interact.Pause VirtualMachine.Interact.PowerOff \ VirtualMachine.Interact.PowerOn VirtualMachine.Interact.PutUsbScanCodes VirtualMachine.Interact.Record \ VirtualMachine.Interact.Replay VirtualMachine.Interact.Reset VirtualMachine.Interact.SESparseMaintenance \ VirtualMachine.Interact.SetCDMedia VirtualMachine.Interact.SetFloppyMedia VirtualMachine.Interact.Suspend \ VirtualMachine.Interact.TerminateFaultTolerantVM VirtualMachine.Interact.ToolsInstall \ VirtualMachine.Interact.TurnOffFaultTolerance VirtualMachine.Inventory.Create \ VirtualMachine.Inventory.CreateFromExisting VirtualMachine.Inventory.Delete \ VirtualMachine.Inventory.Move VirtualMachine.Inventory.Register VirtualMachine.Inventory.Unregister \ VirtualMachine.Namespace.Event VirtualMachine.Namespace.EventNotify VirtualMachine.Namespace.Management \ VirtualMachine.Namespace.ModifyContent VirtualMachine.Namespace.Query VirtualMachine.Namespace.ReadContent \ VirtualMachine.Provisioning.Clone VirtualMachine.Provisioning.CloneTemplate VirtualMachine.Provisioning.CreateTemplateFromVM \ VirtualMachine.Provisioning.Customize VirtualMachine.Provisioning.DeployTemplate VirtualMachine.Provisioning.DiskRandomAccess \ VirtualMachine.Provisioning.DiskRandomRead VirtualMachine.Provisioning.FileRandomAccess VirtualMachine.Provisioning.GetVmFiles \ VirtualMachine.Provisioning.MarkAsTemplate VirtualMachine.Provisioning.MarkAsVM VirtualMachine.Provisioning.ModifyCustSpecs \ VirtualMachine.Provisioning.PromoteDisks VirtualMachine.Provisioning.PutVmFiles VirtualMachine.Provisioning.ReadCustSpecs \ VirtualMachine.State.CreateSnapshot VirtualMachine.State.RemoveSnapshot VirtualMachine.State.RenameSnapshot VirtualMachine.State.RevertToSnapshot \ Cns.Searchable StorageProfile.View govc permissions.set -principal username -role kubernetes /DC

Building a VM image

  1. Install Packer.
  2. Clone the Deckhouse repository:
    Copy content
    git clone https://github.com/deckhouse/deckhouse/
    git clone https://github.com/deckhouse/deckhouse/
  3. cd into the ee/modules/030-cloud-provider-vsphere/packer/ folder of the repository:
    Copy content
    cd deckhouse/ee/modules/030-cloud-provider-vsphere/packer/
    cd deckhouse/ee/modules/030-cloud-provider-vsphere/packer/
  4. Create a file name vsphere.auto.pkrvars.hcl with the following contents:
    Copy content
    vcenter_server = "<hostname or IP of a vCenter>"
vcenter_username = "<username>"
vcenter_password = "<password>"
vcenter_cluster = "<ComputeCluster name, in which template will be created>"
vcenter_datacenter = "<Datacenter name>"
vcenter_resource_pool = <"ResourcePool name">
vcenter_datastore = "<Datastore name>"
vcenter_folder = "<Folder name>"
vm_network = "<VM network in which you will build an image>"
    vcenter_server = "<hostname or IP of a vCenter>" vcenter_username = "<username>" vcenter_password = "<password>" vcenter_cluster = "<ComputeCluster name, in which template will be created>" vcenter_datacenter = "<Datacenter name>" vcenter_resource_pool = <"ResourcePool name"> vcenter_datastore = "<Datastore name>" vcenter_folder = "<Folder name>" vm_network = "<VM network in which you will build an image>"

  5. If your PC (the one you are running Packer from) is not located in the same network as vm_network (if you are connected through a tunnel), change {{ .HTTPIP }} in the <UbuntuVersion>.pkrvars.hcl to your PCs VPN IP:

     " url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/preseed.cfg",

  6. Build a version of Ubuntu:

    Copy content
    # Ubuntu 20.04
packer build --var-file=20.04.pkrvars.hcl .
# Ubuntu 18.04
packer build --var-file=18.04.pkrvars.hcl .
    # Ubuntu 20.04 packer build --var-file=20.04.pkrvars.hcl . # Ubuntu 18.04 packer build --var-file=18.04.pkrvars.hcl .
Go back Next: Installation