Cloud provider — VMware vSphere: provider configuration

If the cluster control plane is hosted on a virtual machines or bare-metal servers, the cloud provider uses the settings from the cloud-provider-vsphere module in the Deckhouse configuration. Otherwise, if the cluster control plane is hosted in a cloud, the cloud provider uses the VsphereClusterConfiguration structure for configuration.

Additional info about Vsphere Cloud Load Balancers.

VsphereClusterConfiguration

Version: deckhouse.io/v1

Describes the configuration of a cloud cluster in vSphere.

Used by the cloud provider if a cluster’s control plane is hosted in the cloud.

Run the following command to change the configuration in a running cluster:

kubectl -n d8-system exec -ti svc/deckhouse-leader -c deckhouse -- deckhouse-controller edit provider-cluster-configuration

After updating the node parameters, you need to run the dhctl converge command to apply the changes.

Example:

apiVersion: deckhouse.io/v1
kind: VsphereClusterConfiguration
sshPublicKey: "<SSH_PUBLIC_KEY>"
layout: Standard
vmFolderPath: folder/prefix
regionTagCategory: k8s-region
zoneTagCategory: k8s-zone
region: region2
zones:
- region2-a
externalNetworkNames:
- net3-k8s
internalNetworkNames:
- K8S_3
internalNetworkCIDR: 172.16.2.0/24
baseResourcePool: kubernetes/cloud
masterNodeGroup:
  replicas: 1
  instanceClass:
    numCPUs: 4
    memory: 8192
    template: Templates/ubuntu-focal-20.04
    mainNetwork: net3-k8s
    additionalNetworks:
    - K8S_3
    datastore: lun10
    rootDiskSize: 50
    runtimeOptions:
      nestedHardwareVirtualization: false
nodeGroups:
- name: worker
  replicas: 1
  zones:
  - ru-central1-a
  instanceClass:
    numCPUs: 4
    memory: 8192
    template: Templates/ubuntu-focal-20.04
    datastore: lun10
    mainNetwork: net3-k8s
provider:
  server: "<SERVER>"
  username: "<USERNAME>"
  password: "<PASSWORD>"
  insecure: true

apiVersion
string
Required value
Allowed values: deckhouse.io/v1, deckhouse.io/v1alpha1
baseResourcePool
string
A path (relative to vSphere Cluster) to the existing parent resourcePool for all resourcePool created in each zone.
disableTimesync
boolean
Disable time synchronization on the vSphere side.

Caution! This parameter will not disable the NTP daemons in the guest OS, but only disable the time correction on the part of ESXi.

Default: true
externalNetworkNames
array of strings
Names of networks (just the name and not the full path) connected to VirtualMachines and used by vsphere-cloud-controller-manager to insert ExternalIP into the .status.addresses field in the Node API object.

Example:
```
externalNetworkNames:
- MAIN-1
- public
```
internalNetworkCIDR
string
Subnet for master nodes in the internal network.

Addresses are allocated starting with the tenth address. E.g., if you have the 192.168.199.0/24 subnet, addresses will be allocated starting with 192.168.199.10.

The internalNetworkCIDR is used if additionalNetworks are defined in masterInstanceClass.

Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(3[0-2]|[1-2][0-9]|[0-9]))$
internalNetworkNames
array of strings
Names of networks (just the name and not the full path) connected to VirtualMachines and used by vsphere-cloud-controller-manager to insert InternalIP into the .status.addresses field in the Node API object.

Example:
```
internalNetworkNames:
- KUBE-3
- devops-internal
```
kind
string
Required value
Allowed values: VsphereClusterConfiguration
layout
string
Required value
The way resources are located in the cloud.

Read more about possible provider layouts.
masterNodeGroup
object
Required value
The definition of the master’s NodeGroup.

For the changes to take effect, run dhctl converge after modifying the parameters of the masterNodeGroup section.
- masterNodeGroup.instanceClass
  object
  Required value
  Partial contents of the fields of the VsphereInstanceClass.
  - masterNodeGroup.instanceClass.additionalNetworks
    array of strings
    Paths to networks that VirtualMachines’ secondary NICs will connect to. Relative to the datacenter.
    
    Example:
    additionalNetworks: - DEVOPS_32 - DEVOPS_50
  - masterNodeGroup.instanceClass.datastore
    string
    Required value
    Path to a Datastore in which VirtualMachines will be cloned. Relative to the datacenter.
    
    Example:
    datastore: lun-1201
  - masterNodeGroup.instanceClass.mainNetwork
    string
    Required value
    Path to the network that VirtualMachines’ primary NICs will connect to (default gateway). Relative to the datacenter.
    
    Example:
    mainNetwork: k8s-msk-178
  - masterNodeGroup.instanceClass.mainNetworkIPAddresses
    array of objects
    A list of static IP addresses (with a CIDR prefix) sequentially allocated to nodes in the mainNetwork.
    
    By default, the DHCP client is enabled.
    
    Example:
    mainNetworkIPAddresses: - address: 10.1.14.20/24 gateway: 10.1.14.254 nameservers: addresses: - 8.8.8.8 - 8.8.4.4
    - masterNodeGroup.instanceClass.mainNetworkIPAddresses.address
      string
      Required value
      An IP address with a CIDR prefix.
      
      Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(3[0-2]|[1-2][0-9]|[0-9]))$
    - masterNodeGroup.instanceClass.mainNetworkIPAddresses.gateway
      string
      Required value
      The IP address of the default gateway.
      
      It must be located in the subnet specified in the address parameter
      
      Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$
    - masterNodeGroup.instanceClass.mainNetworkIPAddresses.nameservers
      object
      Required value
      - masterNodeGroup.instanceClass.mainNetworkIPAddresses.nameservers.addresses
        array of strings
        A list of DNS servers.
        
        Example:
        addresses: - 8.8.8.8 - 8.8.4.4
        
        Element of the array
        string
        Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$
      - masterNodeGroup.instanceClass.mainNetworkIPAddresses.nameservers.search
        array of strings
        A list of DNS search domains.
        
        Example:
        search: - tech.lan
  - masterNodeGroup.instanceClass.memory
    integer
    Required value
    Memory in MiB to allocate to vSphere VirtualMachines.
    
    Example:
    memory: 8192
  - masterNodeGroup.instanceClass.numCPUs
    integer
    Required value
    Count of vCPUs to allocate to vSphere VirtualMachines.
    
    Example:
    numCPUs: 2
  - masterNodeGroup.instanceClass.resourcePool
    string
    Path to a Resource Pool in which VirtualMachines will be cloned. Relative to the zone (vSphere Cluster).
    
    Example:
    resourcePool: rp-2012
  - masterNodeGroup.instanceClass.rootDiskSize
    integer
    Root disk size in GiB to use in vSphere VirtualMachines.
    
    The disk will be automatically enlarged if its size in the template is less than specified.
    
    Example:
    rootDiskSize: 50
  - masterNodeGroup.instanceClass.runtimeOptions
    object
    Additional VM’s parameters.
    - masterNodeGroup.instanceClass.runtimeOptions.cpuLimit
      integer
      CPU limit in MHz.
    - masterNodeGroup.instanceClass.runtimeOptions.cpuReservation
      integer
      CPU reservation in MHz.
    - masterNodeGroup.instanceClass.runtimeOptions.cpuShares
      integer
      The relative amount of CPU Shares for VMs to be created.
    - masterNodeGroup.instanceClass.runtimeOptions.memoryLimit
      integer
      Memory limit in MB.
    - masterNodeGroup.instanceClass.runtimeOptions.memoryReservation
      integer
      VM memory reservation in percent (relative to .spec.memory).
      
      Allowed values: 0 <= X <= 100
    - masterNodeGroup.instanceClass.runtimeOptions.memoryShares
      integer
      The relative amount of Memory Shares for VMs to be created.
      
      Allowed values: 0 <= X <= 100
    - masterNodeGroup.instanceClass.runtimeOptions.nestedHardwareVirtualization
      boolean
      Whether to enable or disable nested hardware virtualization.
  - masterNodeGroup.instanceClass.template
    string
    Required value
    Path to the template to be cloned. Relative to the datacenter.
    
    Example:
    template: dev/golden_image
- masterNodeGroup.replicas
  integer
  Required value
  The number of master nodes to create. It is important to have an odd number of masters to ensure a quorum.
  
  Allowed values: 1 <= X
- masterNodeGroup.zones
  array of strings
  A limited set of zones in which master nodes can be created.
  
  The order in which zones are specified does not influence the sequence of node creation. Nodes are assigned to zones in alphabetical order: the first node is placed in the alphabetically first zone, the second node in the second zone, and so forth. If there are more nodes than zones, the assignment process restarts from the beginning of the zone list after placing a node in the last alphabetical zone.
nodeGroups
array of objects
An array of additional NodeGroups for creating static nodes (e.g., for dedicated front nodes or gateways).
- nodeGroups.instanceClass
  object
  Required value
  Partial contents of the fields of the VsphereInstanceClass.
  - nodeGroups.instanceClass.additionalNetworks
    array of strings
    Paths to networks that VirtualMachines’ secondary NICs will connect to. Relative to the datacenter.
    
    Example:
    additionalNetworks: - DEVOPS_32 - DEVOPS_50
  - nodeGroups.instanceClass.datastore
    string
    Required value
    Path to a Datastore in which VirtualMachines will be cloned. Relative to the datacenter.
    
    Example:
    datastore: lun-1201
  - nodeGroups.instanceClass.mainNetwork
    string
    Required value
    Path to the network that VirtualMachines’ primary NICs will connect to (default gateway). Relative to the datacenter.
    
    Example:
    mainNetwork: k8s-msk-178
  - nodeGroups.instanceClass.mainNetworkIPAddresses
    array of objects
    A list of static IP addresses (with a CIDR prefix) sequentially allocated to nodes in the mainNetwork.
    
    By default, the DHCP client is enabled.
    
    Example:
    mainNetworkIPAddresses: - address: 10.1.14.20/24 gateway: 10.1.14.254 nameservers: addresses: - 8.8.8.8 - 8.8.4.4
    - nodeGroups.instanceClass.mainNetworkIPAddresses.address
      string
      Required value
      An IP address with a CIDR prefix.
      
      Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(3[0-2]|[1-2][0-9]|[0-9]))$
    - nodeGroups.instanceClass.mainNetworkIPAddresses.gateway
      string
      Required value
      The IP address of the default gateway.
      
      It must be located in the subnet specified in the address parameter
      
      Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$
    - nodeGroups.instanceClass.mainNetworkIPAddresses.nameservers
      object
      Required value
      - nodeGroups.instanceClass.mainNetworkIPAddresses.nameservers.addresses
        array of strings
        A list of DNS servers.
        
        Example:
        addresses: - 8.8.8.8 - 8.8.4.4
        
        Element of the array
        string
        Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$
      - nodeGroups.instanceClass.mainNetworkIPAddresses.nameservers.search
        array of strings
        A list of DNS search domains.
        
        Example:
        search: - tech.lan
  - nodeGroups.instanceClass.memory
    integer
    Required value
    Memory in MiB to allocate to vSphere VirtualMachines.
    
    Example:
    memory: 8192
  - nodeGroups.instanceClass.numCPUs
    integer
    Required value
    Count of vCPUs to allocate to vSphere VirtualMachines.
    
    Example:
    numCPUs: 2
  - nodeGroups.instanceClass.resourcePool
    string
    Path to a Resource Pool in which VirtualMachines will be cloned. Relative to the zone (vSphere Cluster).
    
    Example:
    resourcePool: rp-2012
  - nodeGroups.instanceClass.rootDiskSize
    integer
    Root disk size in GiB to use in vSphere VirtualMachines.
    
    The disk will be automatically enlarged if its size in the template is less than specified.
    
    Example:
    rootDiskSize: 50
  - nodeGroups.instanceClass.runtimeOptions
    object
    Additional VM’s parameters.
    - nodeGroups.instanceClass.runtimeOptions.cpuLimit
      integer
      CPU limit in MHz.
    - nodeGroups.instanceClass.runtimeOptions.cpuReservation
      integer
      CPU reservation in MHz.
    - nodeGroups.instanceClass.runtimeOptions.cpuShares
      integer
      The relative amount of CPU Shares for VMs to be created.
    - nodeGroups.instanceClass.runtimeOptions.memoryLimit
      integer
      Memory limit in MB.
    - nodeGroups.instanceClass.runtimeOptions.memoryReservation
      integer
      VM memory reservation in percent (relative to .spec.memory).
      
      Allowed values: 0 <= X <= 100
    - nodeGroups.instanceClass.runtimeOptions.memoryShares
      integer
      The relative amount of Memory Shares for VMs to be created.
      
      Allowed values: 0 <= X <= 100
    - nodeGroups.instanceClass.runtimeOptions.nestedHardwareVirtualization
      boolean
      Whether to enable or disable nested hardware virtualization.
  - nodeGroups.instanceClass.template
    string
    Required value
    Path to the template to be cloned. Relative to the datacenter.
    
    Example:
    template: dev/golden_image
- nodeGroups.name
  string
  Required value
  The name of the NodeGroup to use for generating node names.
- nodeGroups.nodeTemplate
  Parameters of Node objects in Kubernetes to add after registering the node.
  - nodeGroups.nodeTemplate.annotations
    object
    The same as the metadata.annotations standard field.
    
    Example:
    annotations: ai.fleet.com/discombobulate: 'true'
  - nodeGroups.nodeTemplate.labels
    object
    A list of labels to attach to cluster resources.
    
    The same as the metadata.labels standard field.
    
    Example:
    labels: environment: production app: warp-drive-ai
  - nodeGroups.nodeTemplate.taints
    array of objects
    The same as the .spec.taints field of the Node object.
    
    Available fields: effect, key, and values.
    
    Example:
    taints: - effect: NoExecute key: ship-class value: frigate
    - nodeGroups.nodeTemplate.taints.effect
      string
      Allowed values: NoSchedule, PreferNoSchedule, NoExecute
    - nodeGroups.nodeTemplate.taints.key
      string
    - nodeGroups.nodeTemplate.taints.value
      string
- nodeGroups.replicas
  integer
  Required value
  The number of nodes to create.
- nodeGroups.zones
  array of strings
  A limited set of zones in which nodes can be created.
nsxt
object
Kubernetes load balancer support using NSX-T for the vSphere cloud controller manager.
- nsxt.defaultIpPoolName
  string
  Required value
  Name of the default IP pool used for the SVC’s without loadbalancer.vmware.io/class annotation set.
  
  Example:
  defaultIpPoolName: pool1
- nsxt.defaultTcpAppProfileName
  string
  Name of default NSX-T application profile used for TCP connections.
  
  Default: "default-tcp-lb-app-profile"
  Examples:
  defaultTcpAppProfileName: default-tcp-lb-app-profile
  defaultTcpAppProfileName: tcp-profile1
- nsxt.defaultUdpAppProfileName
  string
  Name of default NSX-T application profile used for UDP connections.
  
  Default: "default-udp-lb-app-profile"
  Examples:
  defaultUdpAppProfileName: default-udp-lb-app-profile
  defaultUdpAppProfileName: udp-profile1
- nsxt.host
  string
  Required value
  NSX-T host.
  
  Example:
  host: 1.2.3.4
- nsxt.insecureFlag
  boolean
  To be set to true if NSX-T uses self-signed certificate.
  
  Examples:
  insecureFlag: true
  insecureFlag: false
- nsxt.loadBalancerClass
  array
  Additional section to define Load Balancer Classes (set annotation loadbalancer.vmware.io/class: <CLASS NAME> to SVC to use the class).
  
  Examples:
  loadBalancerClass: []
  loadBalancerClass: name: LBC1 ipPoolName: pool2
  loadBalancerClass: name: LBC1 ipPoolName: pool2 tcpAppProfileName: profile2 udpAppProfileName: profile3
  - nsxt.loadBalancerClass.ipPoolName
    string
    Required value
    Name of the IP pool.
  - nsxt.loadBalancerClass.name
    string
    Required value
    Load Balancer Class name to use in SVC annotation loadbalancer.vmware.io/class: <CLASS NAME>.
  - nsxt.loadBalancerClass.tcpAppProfileName
    string
    Name of application profile used for TCP connections.
    
    Default: "defaultTcpAppProfileName"
  - nsxt.loadBalancerClass.udpAppProfileName
    string
    Name of application profile used for UDP connections.
    
    Default: "defaultUdpAppProfileName"
- nsxt.password
  string
  Required value
  NSX-T password.
  
  Example:
  password: password
- nsxt.size
  string
  Size of load balancer service.
  
  Default: "MEDIUM"
  Allowed values: SMALL, MEDIUM, LARGE, XLARGE
  Example:
  size: SMALL
- nsxt.tier1GatewayPath
  string
  Required value
  Policy path for the NSX-T tier1 gateway.
  
  Example:
  tier1GatewayPath: "/path/tier1"
- nsxt.user
  string
  Required value
  NSX-T user name.
  
  Example:
  user: user
provider
object
Required value
Parameters for connecting to the vCenter.
- provider.insecure
  boolean
  Set to true if vCenter has a self-signed certificate.
  
  Default: false
- provider.password
  string
  Required value
  The user’s password.
- provider.server
  string
  Required value
  The host or the IP address of the vCenter server.
- provider.username
  string
  Required value
  The login ID.
region
string
Required value
Is a tag added to the vSphere Datacenter where all actions will occur: provisioning VirtualMachines, storing virtual disks on datastores, connecting to the network.
regionTagCategory
string
Required value
The name of the tag category used to identify the region (vSphere Datacenter).

Default: "k8s-region"
sshPublicKey
string
Required value
A public key for accessing nodes.
useNestedResourcePool
boolean
Create nested resource pool (true) or use main resource pool (false).

Default: true
vmFolderExists
boolean
Set the value to true if the path specified in vmFolderPath exists. Installing more than one cluster in a folder is not possible.

Default: false
vmFolderPath
string
Required value
The path to the VirtualMachine Folder where the cloned VMs will be created.

Example:
```
vmFolderPath: dev/test
```
zoneTagCategory
string
Required value
The name of the tag category used to identify the zone (vSphere Cluster).

Default: "k8s-zone"
zones
array of strings
Required value
The globally restricted set of zones that this Cloud Provider works with.

Cloud provider — VMware vSphere: provider configuration

VsphereClusterConfiguration

Request trial access

Thank you

Error

Request callback

Thank you

Something went wrong

Book your sessions

Thank you

Error

Request demo

Thank you

Error

Get the PCI SSC Compliance Report

Thank you

Error