OpenStackClusterConfiguration

Version: deckhouse.io/v1

Describes the configuration of a cloud cluster in OpenStack.

Used by the cloud provider if a cluster’s control plane is hosted in the cloud.

Run the following command to change the configuration in a running cluster:

kubectl -n d8-system exec -ti svc/deckhouse-leader -c deckhouse -- deckhouse-controller edit provider-cluster-configuration

After updating the node parameters, you need to run the dhctl converge command to apply the changes.

Example:

apiVersion: deckhouse.io/v1
kind: OpenStackClusterConfiguration
layout: Standard
sshPublicKey: "<SSH_PUBLIC_KEY>"
zones:
- eu-3a
standard:
  internalNetworkDNSServers:
  - 8.8.8.8
  internalNetworkCIDR: 192.168.195.0/24
  internalNetworkSecurity: false
  externalNetworkName: external-network
provider:
  authURL: "<AUTH_URL>"
  domainName: "<DOMAIN_NAME>"
  tenantID: "<TENANT_ID>"
  username: "<USERNAME>"
  password: "<PASSWORD>"
  region: eu-3
masterNodeGroup:
  replicas: 1
  instanceClass:
    rootDiskSize: 50
    flavorName: m1.large
    imageName: debian-11-genericcloud-amd64-20220911-1135
  volumeTypeMap:
    eu-3a: fast.eu-3a
nodeGroups:
- name: front
  replicas: 2
  instanceClass:
    flavorName: m1.large
    imageName: debian-11-genericcloud-amd64-20220911-1135
    rootDiskSize: 50
    configDrive: false
    floatingIPPools:
    - public
    - shared
    additionalSecurityGroups:
    - sec_group_1
    - sec_group_2
  zones:
  - eu-1a
  - eu-1b
  • apiVersion
    string

    Required value

    Allowed values: deckhouse.io/v1, deckhouse.io/v1alpha1

  • kind
    string

    Required value

    Allowed values: OpenStackClusterConfiguration

  • layout
    string

    Required value

    The way resources are located in the cloud.

    Read more about possible provider layouts.

  • masterNodeGroup

    Required value

    The definition of the master’s NodeGroup.

    For the changes to take effect, run dhctl converge after modifying the parameters of the masterNodeGroup section.

    • masterNodeGroup.instanceClass
      object

      Required value

      Partial contents of the fields of the OpenStackInstanceClass.

      • masterNodeGroup.instanceClass.additionalSecurityGroups
        array of strings

        Security groups that will be applied to VM’s network ports. Default group should also be added to this list.

        They allow you to set firewall rules for provisioned instances.

        The SecurityGroups may not be supported by the cloud provider.

      • masterNodeGroup.instanceClass.additionalTags
        object

        The additional tags to attach to the instances created (in addition to those specified in the cloud provider configuration).

        Example:

        additionalTags:
  project: cms-production
  severity: critical
      • masterNodeGroup.instanceClass.etcdDiskSizeGb
        integer

        Etcd disk size in gigabytes.

        Default: 10

        Example:

        etcdDiskSizeGb: 10
      • masterNodeGroup.instanceClass.flavorName
        string

        Required value

        Flavor of OpenStack servers.

        To get a list of all available flavors, run the command: openstack flavor list.

        For all non-master nodes it is advisable to use flavor’s with a local disk. If cloud provider supports local disks they are usually faster and cheaper. The disadvantage of using such flavors is the inability to migrate nodes between hypervisors.

        Flavor create example: openstack flavor create c4m8d50 --ram 8192 --disk 50 --vcpus 4

      • masterNodeGroup.instanceClass.imageName
        string

        Required value

        Image to use while provisioning OpenStack servers.

        Use this command to get a list of available images: openstack image list.

        The list of OS and their versions supported by Deckhouse can be found in the documentation (take into account the Deckhouse version used).

      • masterNodeGroup.instanceClass.rootDiskSize
        integer

        The size of a root disk (in gigabytes).

        This parameter also has influence on type of volume that will be used for root disk; the “How to use rootDiskSize and when it is preferred” section describes how to use it.

    • masterNodeGroup.replicas
      integer

      Required value

      The number of master nodes to create. It is important to have an odd number of masters to ensure a quorum.

      Allowed values: 1 <= X

    • masterNodeGroup.serverGroup
      object

      Object groups instances together. The instances in the group are placed on the same hypervisor (affinity) or different hypervisors (anti-affinity). This allows you to increase the fault tolerance of the cluster.

      • masterNodeGroup.serverGroup.manuallyManaged
        object
        • masterNodeGroup.serverGroup.manuallyManaged.id
          string

          Required value

          The id of the existing ServerGroup object managed outside of Deckhouse.

      • masterNodeGroup.serverGroup.policy
        string

        Required value

        The policy that determines how instances are distributed among hypervisors.

        • AntiAffinity — instances are placed on different hypervisors. This policy is managed by Deckhouse.
        • ManuallyManaged — instances are placed on different hypervisors. This policy is managed outside of Deckhouse.

        Allowed values: AntiAffinity, ManuallyManaged

    • masterNodeGroup.volumeTypeMap
      object

      Required value

      A dictionary of disk types for storing etcd data and Kubernetes configuration files.

      Format of dictionary elements: <AVAILABILITY ZONE>: <VOLUME TYPE> (see the example).

      If the rootDiskSize parameter is specified, the same disk type will be used for the VM’s boot drive.

      We recommend using the fastest disks provided by the provider in all cases.

      If the value specified in replicas exceeds the number of elements in the dictionary, the master nodes whose number exceeds the length of the dictionary get the values starting from the beginning of the dictionary. For example, if replicas: 5, then master-0, master-2, master-4 will have the fast-eu-1a disk type, while master-1, master-3 will have the fast-eu-1b disk type.

      Master nodes will be created in zones specified at this param following by alphabetical order. Based on the example below, master-0 will be created in zone eu-1a, master-1 will be created in zone eu-1b, and master-2 will be created in zone eu-1a.

      Useful commands:

      • openstack availability zone list — get list of availability zones.
      • openstack volume type list — get list of volume types.

      Example:

      volumeTypeMap:
  eu-1a: fast-eu-1a
  eu-1b: fast-eu-1b
  • nodeGroups
    array of objects

    An array of additional NodeGroups for creating static nodes (e.g., for dedicated front nodes or gateways).

    • nodeGroups.instanceClass
      object

      Partial contents of the fields of the OpenStackInstanceClass.

      • nodeGroups.instanceClass.additionalNetworks
        array of strings

        Paths to networks that VirtualMachines secondary NICs will connect to. To get a list of all available networks, run the command: openstack network list.

        Example:

        additionalNetworks:
- BGP-network-VLAN-3894
- External-VLAN-3699
      • nodeGroups.instanceClass.additionalSecurityGroups
        array of strings

        Security groups that will be applied to VM’s network ports. Default group should also be added to this list.

        They allow you to set firewall rules for provisioned instances.

        The SecurityGroups may not be supported by the cloud provider.

      • nodeGroups.instanceClass.additionalTags
        object

        The additional tags to attach to the instances created (in addition to those specified in the cloud provider configuration).

        Example:

        additionalTags:
  project: cms-production
  severity: critical
      • nodeGroups.instanceClass.configDrive
        boolean

        Not required value.

        Specifies whether an additional disk containing the bootstrapping configuration will be mounted to the node.

        You must set it if DHCP is disabled in the mainNetwork.

        Default: false

      • nodeGroups.instanceClass.flavorName
        string

        Required value

        Flavor of OpenStack servers.

        To get a list of all available flavors, run the command: openstack flavor list.

        For all non-master nodes it is advisable to use flavor’s with a local disk. If cloud provider supports local disks they are usually faster and cheaper. The disadvantage of using such flavors is the inability to migrate nodes between hypervisors.

        Flavor create example: openstack flavor create c4m8d50 --ram 8192 --disk 50 --vcpus 4

      • nodeGroups.instanceClass.floatingIPPools
        array of strings

        A list of networks to assign Floating IPs to nodes.

      • nodeGroups.instanceClass.imageName
        string

        Required value

        Image to use while provisioning OpenStack servers.

        Use this command to get a list of available images: openstack image list.

        The list of OS and their versions supported by Deckhouse can be found in the documentation (take into account the Deckhouse version used).

      • nodeGroups.instanceClass.mainNetwork
        string

        Required value

        Path to the network that VirtualMachines primary NICs will connect to (default gateway). To get a list of all available networks, run the command: openstack network list.

      • nodeGroups.instanceClass.networksWithSecurityDisabled
        array of strings

        A list of mainNetwork and additionalNetworks in which SecurityGroups and AllowedAddressPairs on ports cannot be configured.

      • nodeGroups.instanceClass.rootDiskSize
        integer

        The size of a root disk (in gigabytes).

        This parameter also has influence on type of volume that will be used for root disk; the “How to use rootDiskSize and when it is preferred” section describes how to use it.

    • nodeGroups.name
      string

      The name of the NodeGroup to use for generating node names.

    • nodeGroups.nodeTemplate

      Parameters of Node objects in Kubernetes to add after registering the node.

      • nodeGroups.nodeTemplate.annotations
        object

        The same as the metadata.annotations standard field.

        Example:

        annotations:
  ai.fleet.com/discombobulate: 'true'
      • nodeGroups.nodeTemplate.labels
        object

        A list of labels to attach to cluster resources.

        The same as the metadata.labels standard field.

        Note that you have to re-create all the machines to add new tags if tags were modified in the running cluster.

        Example:

        labels:
  environment: production
  app: warp-drive-ai
      • nodeGroups.nodeTemplate.taints
        array of objects

        The same as the .spec.taints field of the Node object.

        Available fields: effect, key, and values.

        Example:

        taints:
- effect: NoExecute
  key: ship-class
  value: frigate
        • nodeGroups.nodeTemplate.taints.effect
          string

          Allowed values: NoSchedule, PreferNoSchedule, NoExecute

        • nodeGroups.nodeTemplate.taints.key
          string
        • nodeGroups.nodeTemplate.taints.value
          string
    • nodeGroups.replicas
      integer

      The number of nodes to create.

    • nodeGroups.volumeTypeMap
      object

      A dictionary of disk types for root drive.

      Format of dictionary elements: <AVAILABILITY ZONE>: <VOLUME TYPE> (see the example).

      If the value specified in replicas exceeds the number of elements in the dictionary, the nodes whose number exceeds the length of the dictionary get the values starting from the beginning of the dictionary. For example, if replicas: 5, then worker-0, worker-2, worker-4 will have the fast-eu-1a disk type, while worker-1, worker-3 will have the fast-eu-1b disk type.

      Caution. Limits by availability zones works only when using the zones parameter (nodeGroups.zones). If zones parameter is not specified, all availability zones will be used.

      Useful commands:

      • openstack availability zone list — get list of availability zones.
      • openstack volume type list — get list of volume types.

      Example:

      volumeTypeMap:
  eu-1a: fast-eu-1a
  eu-1b: fast-eu-1b
    • nodeGroups.zones
      array of strings

      Not required value.

      A limited set of zones in which nodes can be created.

  • provider
    object

    Required value

    Contains settings to connect to the OpenStack API.

    These settings are the same as those in the connection field of the cloud-provider-openstack module.

    • provider.authURL
      string

      An OpenStack Identity API URL.

    • provider.caCert
      string

      Specify the CA x509 certificate used for signing if the OpenStack API has a self-signed certificate. Certificate should be passed in PEM format as multiline string.

      Example:

      caCert: |
  -----BEGIN CERTIFICATE-----
  MIIFyDCCBLCgAwIBAgIQBwDIWH1asdaKNaALUa4NUzANBgkqhkiG9w0BAQsFADBc
  ...
  -----END CERTIFICATE-----
    • provider.domainName
      string

      The domain name.

      OS_USER_DOMAIN_NAME variable from the openrc file.

    • provider.password
      string

      The user’s password.

    • provider.region
      string

      The OpenStack region where the cluster will be deployed.

    • provider.tenantID
      string

      The project id.

      Cannot be used together with tenantName.

    • provider.tenantName
      string

      The project name.

      Cannot be used together with tenantID.

    • provider.username
      string

      The name of the user that has full project privileges.

  • simple
    object

    Settings for the Simple layout.

    • simple.externalNetworkDHCP
      boolean

      This parameter defines if DHCP is enabled in the external network.

      Default: true

    • simple.externalNetworkName
      string

      Required value

      The name of the network for external connections. To get a list of all available networks, run the command: openstack network list.

    • simple.podNetworkMode
      string

      Sets the traffic mode for the network that the pods use to communicate with each other (usually, it is an internal network; however, there can be exceptions):

      • DirectRouting — nodes are directly routed (SecurityGroups are disabled in this mode).
      • VXLAN — direct routing does NOT work between nodes, VXLAN must be used (SecurityGroups are disabled in this mode).

      Caution. After changing this parameter, you need to run dhctl converge command for the changes to take effect.

      Caution. All cluster nodes must be rebooted after switching work mode from/to VXLAN.

      Default: "VXLAN"

      Allowed values: VXLAN, DirectRouting

  • simpleWithInternalNetwork
    object

    Settings for the SimpleWithInternalNetwork layout.

    • simpleWithInternalNetwork.externalNetworkDHCP
      boolean

      This parameter defines if DHCP is enabled in the external network.

      Default: true

    • simpleWithInternalNetwork.externalNetworkName
      string

      The name of the network for external connections. To get a list of all available networks, run the command: openstack network list.

    • simpleWithInternalNetwork.internalSubnetName
      string

      Required value

      The name of the subnet in which the cluster nodes will run.

    • simpleWithInternalNetwork.masterWithExternalFloatingIP
      boolean

      Defines if Floating IP must be assigned to master nodes.

      Default: true

    • simpleWithInternalNetwork.podNetworkMode
      string

      Sets the traffic mode for the network that the pods use to communicate with each other (usually, it is an internal network; however, there can be exceptions):

      • DirectRouting — nodes are directly routed (SecurityGroups are disabled in this mode).
      • DirectRoutingWithPortSecurityEnabled — direct routing is enabled between the nodes, but only if the range of addresses of the internal network is explicitly allowed in OpenStack for Ports:
        • Caution. Make sure that the username can edit AllowedAddressPairs on Ports connected to the internalNetworkName network. Usually, an OpenStack user doesn’t have such a privilege if the network has the shared flag set.
      • VXLAN — direct routing does NOT work between nodes, VXLAN must be used (SecurityGroups are disabled in this mode).

      Caution. After changing this parameter, you need to run dhctl converge command for the changes to take effect.

      Caution. All cluster nodes must be rebooted after switching work mode from/to VXLAN.

      Default: "DirectRoutingWithPortSecurityEnabled"

      Allowed values: VXLAN, DirectRouting, DirectRoutingWithPortSecurityEnabled

  • sshAllowList
    array of strings

    A list of CIDR’s allowed to connect to nodes via SSH.

    By default, 0.0.0.0/0.

  • sshPublicKey
    string

    Required value

    A public key for accessing nodes.

  • standard
    object

    Settings for the Standard layout.

    • standard.bastion
      object

      The definition of the bastion instance.

      • standard.bastion.instanceClass
        object

        Partial contents of the fields of the OpenStackInstanceClass.

        • standard.bastion.instanceClass.additionalTags
          object

          The additional tags to attach to the instance created (in addition to those specified in the cloud provider configuration).

          Example:

          additionalTags:
  project: cms-production
  severity: critical
        • standard.bastion.instanceClass.flavorName
          string

          Required value

          Flavor of OpenStack servers.

          To get a list of all available flavors, run the command: openstack flavor list.

        • standard.bastion.instanceClass.imageName
          string

          Required value

          Image to use while provisioning OpenStack servers.

          Use this command to get a list of available images: openstack image list.

          The list of OS and their versions supported by Deckhouse can be found in the documentation (take into account the Deckhouse version used).

        • standard.bastion.instanceClass.rootDiskSize
          integer

          The size of a root disk (in gigabytes).

          This parameter also has influence on type of volume that will be used for root disk; the “How to use rootDiskSize and when it is preferred” section describes how to use it.

          Default: 50

      • standard.bastion.volumeType
        string

        Root disk type.

      • standard.bastion.zone
        string

        The zone to create an instance for the bastion node.

    • standard.externalNetworkName
      string

      Required value

      The name of the network for external connections. To get a list of all available networks, run the command: openstack network list.

    • standard.internalNetworkCIDR
      string

      Required value

      Routing for the internal cluster network.

      Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(3[0-2]|[1-2][0-9]|[0-9]))$

    • standard.internalNetworkDNSServers
      array of strings

      Required value

      A list of addresses of the recursive DNSs of the internal cluster network.

      • Element of the array
        string

        Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$

    • standard.internalNetworkSecurity
      boolean

      Defines whether SecurityGroups and AllowedAddressPairs must be configured for ports of the internal network.

      Default: true

  • standardWithNoRouter
    object

    Settings for the StandardWithNoRouter layout.

    • standardWithNoRouter.externalNetworkDHCP
      boolean

      This parameter defines if DHCP is enabled in the external network.

      Default: true

    • standardWithNoRouter.externalNetworkName
      string

      Required value

      The name of the network for external connections. To get a list of all available networks, run the command: openstack network list.

    • standardWithNoRouter.internalNetworkCIDR
      string

      Required value

      Routing for the internal cluster network.

      Pattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(3[0-2]|[1-2][0-9]|[0-9]))$

    • standardWithNoRouter.internalNetworkSecurity
      boolean

      Defines whether SecurityGroups and AllowedAddressPairs must be configured for ports of the internal network.

      Default: true

  • tags
    object

    Not required value.

    A dictionary of tags to create on all resources that support this feature.

    You have to re-create all the machines to add new tags if tags were modified in the running cluster.

  • zones
    array of strings

    Not required value.

    The globally restricted set of zones that this Cloud Provider works with.