Available in: EE, SE+
The module is automatically enabled for all cloud clusters deployed in vSphere.
If the cluster control plane is hosted on a virtual machines or bare-metal servers, the cloud provider uses the settings from the cloud-provider-vsphere
module in the Deckhouse configuration (see below). Otherwise, if the cluster control plane is hosted in a cloud, the cloud provider uses the VsphereClusterConfiguration structure for configuration.
You can configure the number and parameters of ordering machines in the cloud via the NodeGroup
custom resource of the node-manager
module. Also, in this custom resource, you can specify the instance class’s name for the above group of nodes (the cloudInstances.ClassReference
parameter of NodeGroup). In the case of the vSphere cloud provider, the instance class is the VsphereInstanceClass
custom resource that stores specific parameters of the machines.
Storage
The module automatically creates a StorageClass for each Datastore and DatastoreCluster in the zone (or zones).
Also, it can set the name of StorageClass that will be used in the cluster by default (the default parameter), and filter out the unnecessary StorageClasses (the exclude parameter).
CSI
By default, the storage subsystem uses CNS volumes with the ability of online-resize. FCD volumes are also supported, but only in the legacy or migration modes. You can set this via the compatibilityFlag parameter.
Important information concerning the increase of the PVC size
Due to the nature f volume-resizer, CSI, and vSphere API, you have to do the following after increasing the PVC size:
- On the node where the Pod is located, run the
kubectl cordon <node_name>
command. - Delete the Pod.
- Make sure that the resize was successful. The PVC object must not have the
Resizing
condition.The
FileSystemResizePending
state is OK. - On the node where the Pod is located, run the
kubectl uncordon <node_name>
command.
Environment requirements
- vSphere version required:
v7.0U2
(required for theOnline volume expansion
work). - vCenter to which master nodes can connect to from within the cluster.
- Datacenter with the following components:
- VirtualMachine template.
- VM image should use
Virtual machines with hardware version 15 or later
(required for online resize to work). - The following packages must be installed in the VM image:
open-vm-tools
,cloud-init
andcloud-init-vmware-guestinfo
(if thecloud-init
version lower than 21.3 is used).
- VM image should use
- The network must be available on all ESXi where VirtualMachines will be created.
- One or more Datastores connected to all ESXi where VirtualMachines will be created.
- A tag from the tag category in
zoneTagCategory
(k8s-zone
by default) must be added to Datastores. This tag will indicate the zone. All Clusters of a specific zone must have access to all Datastores within the same zone.
- A tag from the tag category in
- The cluster with the required ESXis.
- A tag from the tag category in
zoneTagCategory
(k8s-zone
by default) must be added to the Cluster. This tag will indicate the zone.
- A tag from the tag category in
- Folder for VirtualMachines to be created.
- An optional parameter. By default, the root vm folder is used.
- Create a role with the appropriate set of privileges.
- Create a user and assign the above role to it.
- VirtualMachine template.
- A tag from the tag category in
regionTagCategory
(k8s-region
by default) must be added to the Datacenter. This tag will indicate the region.
List of required privileges
Read the documentation on how to create and assign a role to a user.
A detailed list of privileges required for Deckhouse Kubernetes Platform to work in vSphere:
List of privileges | Purpose |
---|---|
Cns.Searchable StorageProfile.View Datastore.AllocateSpace Datastore.Browse Datastore.FileManagement |
To provision disks when creating virtual machines and ordering PersistentVolumes in a cluster. |
Global.GlobalTag Global.SystemTag InventoryService.Tagging.AttachTag InventoryService.Tagging.CreateCategory InventoryService.Tagging.CreateTag InventoryService.Tagging.DeleteCategory InventoryService.Tagging.DeleteTag InventoryService.Tagging.EditCategory InventoryService.Tagging.EditTag InventoryService.Tagging.ModifyUsedByForCategory InventoryService.Tagging.ModifyUsedByForTag InventoryService.Tagging.ObjectAttachable |
Deckhouse Kubernetes Platform uses tags to identify the Datacenter , Cluster and Datastore objects available to it, as well as, to identify the virtual machines under its control. |
Folder.Create Folder.Delete Folder.Move Folder.Rename |
To group a Deckhouse Kubernetes Platform cluster in a single Folder in vSphere Inventory. |
Network.Assign Resource.ApplyRecommendation Resource.AssignVAppToPool Resource.AssignVMToPool Resource.ColdMigrate Resource.CreatePool Resource.DeletePool Resource.EditPool Resource.HotMigrate Resource.MovePool Resource.QueryVMotion Resource.RenamePool VirtualMachine.Config.AddExistingDisk VirtualMachine.Config.AddNewDisk VirtualMachine.Config.AddRemoveDevice VirtualMachine.Config.AdvancedConfig VirtualMachine.Config.Annotation VirtualMachine.Config.ChangeTracking VirtualMachine.Config.CPUCount VirtualMachine.Config.DiskExtend VirtualMachine.Config.DiskLease VirtualMachine.Config.EditDevice VirtualMachine.Config.HostUSBDevice VirtualMachine.Config.ManagedBy VirtualMachine.Config.Memory VirtualMachine.Config.MksControl VirtualMachine.Config.QueryFTCompatibility VirtualMachine.Config.QueryUnownedFiles VirtualMachine.Config.RawDevice VirtualMachine.Config.ReloadFromPath VirtualMachine.Config.RemoveDisk VirtualMachine.Config.Rename VirtualMachine.Config.ResetGuestInfo VirtualMachine.Config.Resource VirtualMachine.Config.Settings VirtualMachine.Config.SwapPlacement VirtualMachine.Config.ToggleForkParent VirtualMachine.Config.UpgradeVirtualHardware VirtualMachine.GuestOperations.Execute VirtualMachine.GuestOperations.Modify VirtualMachine.GuestOperations.ModifyAliases VirtualMachine.GuestOperations.Query VirtualMachine.GuestOperations.QueryAliases VirtualMachine.Hbr.ConfigureReplication VirtualMachine.Hbr.MonitorReplication VirtualMachine.Hbr.ReplicaManagement VirtualMachine.Interact.AnswerQuestion VirtualMachine.Interact.Backup VirtualMachine.Interact.ConsoleInteract VirtualMachine.Interact.CreateScreenshot VirtualMachine.Interact.CreateSecondary VirtualMachine.Interact.DefragmentAllDisks VirtualMachine.Interact.DeviceConnection VirtualMachine.Interact.DisableSecondary VirtualMachine.Interact.DnD VirtualMachine.Interact.EnableSecondary VirtualMachine.Interact.GuestControl VirtualMachine.Interact.MakePrimary VirtualMachine.Interact.Pause VirtualMachine.Interact.PowerOff VirtualMachine.Interact.PowerOn VirtualMachine.Interact.PutUsbScanCodes VirtualMachine.Interact.Record VirtualMachine.Interact.Replay VirtualMachine.Interact.Reset VirtualMachine.Interact.SESparseMaintenance VirtualMachine.Interact.SetCDMedia VirtualMachine.Interact.SetFloppyMedia VirtualMachine.Interact.Suspend VirtualMachine.Interact.SuspendToMemory VirtualMachine.Interact.TerminateFaultTolerantVM VirtualMachine.Interact.ToolsInstall VirtualMachine.Interact.TurnOffFaultTolerance VirtualMachine.Inventory.Create VirtualMachine.Inventory.CreateFromExisting VirtualMachine.Inventory.Delete VirtualMachine.Inventory.Move VirtualMachine.Inventory.Register VirtualMachine.Inventory.Unregister VirtualMachine.Namespace.Event VirtualMachine.Namespace.EventNotify VirtualMachine.Namespace.Management VirtualMachine.Namespace.ModifyContent VirtualMachine.Namespace.Query VirtualMachine.Namespace.ReadContent VirtualMachine.Provisioning.Clone VirtualMachine.Provisioning.CloneTemplate VirtualMachine.Provisioning.CreateTemplateFromVM VirtualMachine.Provisioning.Customize VirtualMachine.Provisioning.DeployTemplate VirtualMachine.Provisioning.DiskRandomAccess VirtualMachine.Provisioning.DiskRandomRead VirtualMachine.Provisioning.FileRandomAccess VirtualMachine.Provisioning.GetVmFiles VirtualMachine.Provisioning.MarkAsTemplate VirtualMachine.Provisioning.MarkAsVM VirtualMachine.Provisioning.ModifyCustSpecs VirtualMachine.Provisioning.PromoteDisks VirtualMachine.Provisioning.PutVmFiles VirtualMachine.Provisioning.ReadCustSpecs VirtualMachine.State.CreateSnapshot VirtualMachine.State.RemoveSnapshot VirtualMachine.State.RenameSnapshot VirtualMachine.State.RevertToSnapshot |
To manage the virtual machines lifecycle in a Deckhouse Kubernetes Platform cluster. |
The module is configured using the ModuleConfig custom resource named cloud-provider-vsphere
(learn more about setting up Deckhouse…).
Example of the ModuleConfig/cloud-provider-vsphere
resource for configuring the module:
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
name: cloud-provider-vsphere
spec:
version: 1
enabled: true
settings: # <-- Module parameters from the "Parameters" section below.
Parameters
Schema version: 1
- object
Kubernetes load balancer support using NSX-T for the vSphere cloud controller manager.
- array
Additional section to define Load Balancer Classes (to use class, set annotation
loadbalancer.vmware.io/class: <class name>
to SVC).Examples:
loadBalancerClass: []
loadBalancerClass: name: LBC1 ipPoolName: pool2
loadBalancerClass: name: LBC1 ipPoolName: pool2 tcpAppProfileName: profile2 udpAppProfileName: profile3
- string
A flag allowing the use of the old CSI version:
legacy
— use the old version of the driver. FCD discs only, no online-resizing;migration
— in this case, both drivers will be available in the cluster at the same time. This mode is used to migrate from an old driver.
Allowed values:
legacy
,migration
- string
The name of StorageClass that will be used in the cluster by default.
If the parameter is omitted, the default StorageClass is either:
- an arbitrary StorageClass present in the cluster that has the default annotation;
- the first (in lexicographic order) StorageClass created by the module.
Parameter is deprecated. Instead, use the global parameter global.defaultClusterStorageClass.
Example:
default: fast-lun102-7d0bf578