This feature is available in Enterprise Edition only.

If the cluster control plane is hosted on a virtual machines or bare-metal servers, the cloud provider uses the settings from the cloud-provider-vsphere module in the Deckhouse configuration. Otherwise, if the cluster control plane is hosted in a cloud, the cloud provider uses the VsphereClusterConfiguration structure for configuration.

Additional info about Vsphere Cloud Load Balancers.

VsphereClusterConfiguration

Version: deckhouse.io/v1

A structure describing the configuration of a cloud cluster in vSphere.

The cloud provider uses this structure for its configuration if a cluster’s control plane is hosted in the cloud.

It is located in the d8-provider-cluster-configuration Secret in the kube-system namespace.

  • apiVersionstring

    Required value

    Allowed values: deckhouse.io/v1, deckhouse.io/v1alpha1

  • baseResourcePoolstring

    A path (relative to vSphere Cluster) to the existing parent resourcePool for all resourcePool created in each zone.

  • disableTimesyncboolean

    Disable time synchronization on the vSphere side.

    Caution! This parameter will not disable the NTP daemons in the guest OS, but only disable the time correction on the part of ESXi.

  • externalNetworkNamesarray of strings

    Names of networks (just the name and not the full path) connected to VirtualMachines and used by vsphere-cloud-controller-manager to insert ExternalIP into the .status.addresses field in the Node API object.

    Example:

    externalNetworkNames:
    - MAIN-1
    - public
    
  • internalNetworkCIDRstring

    Subnet for master nodes in the internal network.

    Addresses are allocated starting with the tenth address. E.g., if you have the 192.168.199.0/24 subnet, addresses will be allocated starting with 192.168.199.10.

    The internalNetworkCIDR is used if additionalNetworks are defined in masterInstanceClass.

    Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(3[0-2]|[1-2][0-9]|[0-9]))$

  • internalNetworkNamesarray of strings

    Names of networks (just the name and not the full path) connected to VirtualMachines and used by vsphere-cloud-controller-manager to insert InternalIP into the .status.addresses field in the Node API object.

    Example:

    internalNetworkNames:
    - KUBE-3
    - devops-internal
    
  • kindstring

    Required value

    Allowed values: VsphereClusterConfiguration

  • layoutstring

    Required value

    The way resources are located in the cloud.

    Read more about possible provider layouts.

  • masterNodeGroupobject

    Required value

    The definition of the master’s NodeGroup.

    • instanceClassobject

      Required value

      Partial contents of the fields of the VsphereInstanceClass.

      • additionalNetworksarray of strings

        Paths to networks that VirtualMachines’ secondary NICs will connect to. Relative to the datacenter.

        Example:

        additionalNetworks:
        - DEVOPS_32
        - DEVOPS_50
        
      • datastorestring

        Required value

        Path to a Datastore in which VirtualMachines will be cloned. Relative to the datacenter.

        Example:

        datastore: lun-1201
        
      • mainNetworkstring

        Required value

        Path to the network that VirtualMachines’ primary NICs will connect to (default gateway). Relative to the datacenter.

        Example:

        mainNetwork: k8s-msk-178
        
      • mainNetworkIPAddressesarray of objects

        A list of static IP addresses (with a CIDR prefix) sequentially allocated to master nodes in the mainNetwork.

        By default, the DHCP client is enabled.

        Example:

        mainNetworkIPAddresses:
        - address: 10.1.14.20/24
          gateway: 10.1.14.254
          nameservers:
            addresses:
            - 8.8.8.8
            - 8.8.4.4
        
        • addressstring

          Required value

          An IP address with a CIDR prefix.

          Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(3[0-2]|[1-2][0-9]|[0-9]))$

        • gatewaystring

          Required value

          The IP address of the default gateway.

          It must be located in the subnet specified in the address parameter

          Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$

        • nameserversobject

          Required value

          • addressesarray of strings

            A list of DNS servers.

            Example:

            addresses:
            - 8.8.8.8
            - 8.8.4.4
            
            • Element of the arraystring

              Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$

          • searcharray of strings

            A list of DNS search domains.

            Example:

            search:
            - tech.lan
            
      • memoryinteger

        Required value

        Memory in MiB to allocate to vSphere VirtualMachines.

        Example:

        memory: 8192
        
      • numCPUsinteger

        Required value

        Count of vCPUs to allocate to vSphere VirtualMachines.

        Example:

        numCPUs: 2
        
      • resourcePoolstring

        Path to a Resource Pool in which VirtualMachines will be cloned. Relative to the zone (vSphere Cluster).

        Example:

        resourcePool: rp-2012
        
      • rootDiskSizeinteger

        Root disk size in GiB to use in vSphere VirtualMachines.

        The disk will be automatically enlarged if its size in the template is less than specified.

        Example:

        rootDiskSize: 20
        
      • runtimeOptionsobject

        Additional VM’s parameters.

        • cpuLimitinteger

          CPU limit in MHz.

        • cpuReservationinteger

          CPU reservation in MHz.

        • cpuSharesinteger

          The relative amount of CPU Shares for VMs to be created.

        • memoryLimitinteger

          Memory limit in MB.

        • memoryReservationinteger

          VM memory reservation in percent (relative to .spec.memory).

          Allowed values: 0 <= X <= 100

        • memorySharesinteger

          The relative amount of Memory Shares for VMs to be created.

          Allowed values: 0 <= X <= 100

        • nestedHardwareVirtualizationboolean

          Whether to enable or disable nested hardware virtualization.

      • templatestring

        Required value

        Path to the template to be cloned. Relative to the datacenter.

        Example:

        template: dev/golden_image
        
    • replicasinteger

      Required value

      The number of master nodes to create.

      Allowed values: 1 <= X

    • zonesarray of strings

      A limited set of zones in which nodes can be created.

  • nodeGroupsarray of objects

    An array of additional NodeGroups for creating static nodes (e.g., for dedicated front nodes or gateways).

    • instanceClassobject

      Required value

      Partial contents of the fields of the VsphereInstanceClass.

      • additionalNetworksarray of strings

        Paths to networks that VirtualMachines’ secondary NICs will connect to. Relative to the datacenter.

        Example:

        additionalNetworks:
        - DEVOPS_32
        - DEVOPS_50
        
      • datastorestring

        Required value

        Path to a Datastore in which VirtualMachines will be cloned. Relative to the datacenter.

        Example:

        datastore: lun-1201
        
      • mainNetworkstring

        Required value

        Path to the network that VirtualMachines’ primary NICs will connect to (default gateway). Relative to the datacenter.

        Example:

        mainNetwork: k8s-msk-178
        
      • mainNetworkIPAddressesarray of objects

        A list of static IP addresses (with a CIDR prefix) sequentially allocated to master nodes in the mainNetwork.

        By default, the DHCP client is enabled.

        Example:

        mainNetworkIPAddresses:
        - address: 10.1.14.20/24
          gateway: 10.1.14.254
          nameservers:
            addresses:
            - 8.8.8.8
            - 8.8.4.4
        
        • addressstring

          Required value

          An IP address with a CIDR prefix.

          Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(3[0-2]|[1-2][0-9]|[0-9]))$

        • gatewaystring

          Required value

          The IP address of the default gateway.

          It must be located in the subnet specified in the address parameter

          Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$

        • nameserversobject

          Required value

          • addressesarray of strings

            A list of DNS servers.

            Example:

            addresses:
            - 8.8.8.8
            - 8.8.4.4
            
            • Element of the arraystring

              Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$

          • searcharray of strings

            A list of DNS search domains.

            Example:

            search:
            - tech.lan
            
      • memoryinteger

        Required value

        Memory in MiB to allocate to vSphere VirtualMachines.

        Example:

        memory: 8192
        
      • numCPUsinteger

        Required value

        Count of vCPUs to allocate to vSphere VirtualMachines.

        Example:

        numCPUs: 2
        
      • resourcePoolstring

        Path to a Resource Pool in which VirtualMachines will be cloned. Relative to the zone (vSphere Cluster).

        Example:

        resourcePool: rp-2012
        
      • rootDiskSizeinteger

        Root disk size in GiB to use in vSphere VirtualMachines.

        The disk will be automatically enlarged if its size in the template is less than specified.

        Example:

        rootDiskSize: 20
        
      • runtimeOptionsobject

        Additional VM’s parameters.

        • cpuLimitinteger

          CPU limit in MHz.

        • cpuReservationinteger

          CPU reservation in MHz.

        • cpuSharesinteger

          The relative amount of CPU Shares for VMs to be created.

        • memoryLimitinteger

          Memory limit in MB.

        • memoryReservationinteger

          VM memory reservation in percent (relative to .spec.memory).

          Allowed values: 0 <= X <= 100

        • memorySharesinteger

          The relative amount of Memory Shares for VMs to be created.

          Allowed values: 0 <= X <= 100

        • nestedHardwareVirtualizationboolean

          Whether to enable or disable nested hardware virtualization.

      • templatestring

        Required value

        Path to the template to be cloned. Relative to the datacenter.

        Example:

        template: dev/golden_image
        
    • namestring

      Required value

      The name of the NodeGroup to use for generating node names.

    • nodeTemplate

      Parameters of Node objects in Kubernetes to add after registering the node.

      • annotationsobject

        The same as the metadata.annotations standard field.

        Example:

        annotations:
          ai.fleet.com/discombobulate: 'true'
        
      • labelsobject

        A list of labels to attach to cluster resources.

        The same as the metadata.labels standard field.

        Example:

        labels:
          environment: production
          app: warp-drive-ai
        
      • taintsarray of objects

        The same as the .spec.taints field of the Node object.

        Caution! Only the effect, key, values fields are available.

        Example:

        taints:
        - effect: NoExecute
          key: ship-class
          value: frigate
        
        • effectstring

          Allowed values: NoSchedule, PreferNoSchedule, NoExecute

        • keystring
        • valuestring
    • replicasinteger

      Required value

      The number of nodes to create.

    • zonesarray of strings

      A limited set of zones in which nodes can be created.

  • nsxtobject

    Kubernetes load balancer support using NSX-T for the vSphere cloud controller manager.

    • defaultIpPoolNamestring

      Required value

      Name of the default IP pool used for the SVC’s without loadbalancer.vmware.io/class annotation set.

      Example:

      defaultIpPoolName: pool1
      
    • defaultTcpAppProfileNamestring

      Name of default NSX-T application profile used for TCP connections.

      Default: "default-tcp-lb-app-profile"

      Examples:

      defaultTcpAppProfileName: default-tcp-lb-app-profile
      
      defaultTcpAppProfileName: tcp-profile1
      
    • defaultUdpAppProfileNamestring

      Name of default NSX-T application profile used for UDP connections.

      Default: "default-udp-lb-app-profile"

      Examples:

      defaultUdpAppProfileName: default-udp-lb-app-profile
      
      defaultUdpAppProfileName: udp-profile1
      
    • hoststring

      Required value

      NSX-T host.

      Example:

      host: 1.2.3.4
      
    • insecureFlagboolean

      To be set to true if NSX-T uses self-signed certificate.

      Examples:

      insecureFlag: true
      
      insecureFlag: false
      
    • loadBalancerClassarray

      Additional section to define Load Balancer Classes (to use class, set annotation loadbalancer.vmware.io/class: <class name> to SVC).

      Examples:

      loadBalancerClass: []
      
      loadBalancerClass:
        name: LBC1
        ipPoolName: pool2
      
      loadBalancerClass:
        name: LBC1
        ipPoolName: pool2
        tcpAppProfileName: profile2
        udpAppProfileName: profile3
      
      • ipPoolNamestring

        Required value

        Name of the IP pool.

      • namestring

        Required value

        Load Balancer Class name to use in SVC annotation loadbalancer.vmware.io/class: <class name>.

      • tcpAppProfileNamestring

        Name of application profile used for TCP connections.

        Default: "defaultTcpAppProfileName"

      • udpAppProfileNamestring

        Name of application profile used for UDP connections.

        Default: "defaultUdpAppProfileName"

    • passwordstring

      Required value

      NSX-T password.

      Example:

      password: password
      
    • sizestring

      Size of load balancer service.

      Default: "MEDIUM"

      Allowed values: SMALL, MEDIUM, LARGE, XLARGE

      Example:

      size: SMALL
      
    • tier1GatewayPathstring

      Required value

      Policy path for the NSX-T tier1 gateway.

      Example:

      tier1GatewayPath: "/path/tier1"
      
    • userstring

      Required value

      NSX-T user name.

      Example:

      user: user
      
  • providerobject

    Required value

    Parameters for connecting to the vCenter.

    • insecureboolean

      Set to true if vCenter has a self-signed certificate.

      Default: false

    • passwordstring

      Required value

      The user’s password.

    • serverstring

      Required value

      The host or the IP address of the vCenter server.

    • usernamestring

      Required value

      The login ID.

  • regionstring

    Required value

    Is a tag added to the vSphere Datacenter where all actions will occur: provisioning VirtualMachines, storing virtual disks on datastores, connecting to the network.

  • regionTagCategorystring

    Required value

    The name of the tag category used to identify the region (vSphere Datacenter).

    Default: "k8s-region"

  • sshPublicKeystring

    Required value

    A public key for accessing nodes.

  • useNestedResourcePoolboolean

    Create nested resource pool (true) or use main resource pool (false).

    Default: true

  • vmFolderPathstring

    Required value

    The path to the VirtualMachine Folder where the cloned VMs will be created.

    Example:

    vmFolderPath: dev/test
    
  • zoneTagCategorystring

    Required value

    The name of the tag category used to identify the zone (vSphere Cluster).

    Default: "k8s-zone"

  • zonesarray of strings

    Required value

    The globally restricted set of zones that this Cloud Provider works with.