Some cluster parameters that affect control plane management are derived from the ClusterConfiguration resource.

The module has 7 alerts.

The module is enabled by default in the Default bundle. The module is disabled by default in the following bundles: Managed, Minimal.

How to explicitly enable the module…

Set the spec.enabled module parameter to true or false in the ModuleConfig/control-plane-manager resource (create it, if necessary) to explicitly enable or disable the module, or use the deckhouse-controller module command in the d8-system/deckhouse pod.

Example of enabling the module:

  • by using the ModuleConfig resource:

    apiVersion: deckhouse.io/v1alpha1
    kind: ModuleConfig
    metadata:
      name: control-plane-manager
    spec:
      enabled: true
    
  • by using the deckhouse-controller command (you need a kubectl, configured to work with the cluster):

    kubectl -ti -n d8-system exec svc/deckhouse-leader -c deckhouse -- deckhouse-controller module enable control-plane-manager
    

Example of disabling the module:

  • by using the ModuleConfig resource:

    apiVersion: deckhouse.io/v1alpha1
    kind: ModuleConfig
    metadata:
      name: control-plane-manager
    spec:
      enabled: false
    
  • by using the deckhouse-controller command (you need a kubectl, configured to work with the cluster):

    kubectl -ti -n d8-system exec svc/deckhouse-leader -c deckhouse -- deckhouse-controller module disable control-plane-manager
    

The module is configured using the ModuleConfig custom resource named control-plane-manager (learn more about setting up Deckhouse…).

Example of the ModuleConfig/control-plane-manager resource for configuring the module:

apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: control-plane-manager
spec:
  version: 1
  enabled: true
  settings: # <-- Module parameters from the "Parameters" section below.

Parameters

Schema version: 1

Example:

apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: control-plane-manager
spec:
  version: 1
  enabled: true
  settings:
    apiserver:
      bindToWildcard: true
      certSANs:
      - bakery.infra
      - devs.infra
      loadBalancer: {}
  • apiserver
    object

    kube-apiserver parameters.

    • apiserver.admissionPlugins
      array of strings

      List of enabled additional admission plugins.

      Note, that in addition to the admission plugins enabled by default in Kubernetes, the following admission plugins are also always enabled:

      • ExtendedResourceToleration
      • PodNodeSelector
      • PodTolerationRestriction
      • EventRateLimit with the following config:

        apiVersion: eventratelimit.admission.k8s.io/v1alpha1
        kind: Configuration
        limits:
        - type: Namespace
          qps: 50
          burst: 100
          cacheSize: 2000
        

      Note that th PodNodeSelector admission plugin does not require specifiying a global configuration, it relies on annotated Namespaces.

      Example:

      admissionPlugins:
      - AlwaysPullImages
      - NamespaceAutoProvision
      
      • Element of the array
        string

        Allowed values: AlwaysPullImages, NamespaceAutoProvision, OwnerReferencesPermissionEnforcement, PodNodeSelector, PodTolerationRestriction

    • apiserver.auditLog
      object

      Audit policy settings

      Default: {"output":"File"}

      • apiserver.auditLog.output
        string

        Required value

        Audit logs target stream.

        Default: "File"

        Allowed values: File, Stdout

        Example:

        output: Stdout
        
      • apiserver.auditLog.path
        string

        Directory path for logs if the output is “File”, otherwise ignored.

        Default: "/var/log/kube-audit"

        Pattern: ^[a-zA-Z0-9_/.-]+[a-zA-Z0-9_.-]$

    • apiserver.auditPolicyEnabled
      boolean

      Set the audit policies using the configuration from the kube-system/audit-policy Secret.

      Default: false

    • apiserver.authn
      object

      Optional authentication parameters for Kubernetes API clients.

      By default, they are taken from user-authn module ConfigMap.

      • apiserver.authn.oidcCA
        string

        OIDC provider CA.

      • apiserver.authn.oidcIssuerAddress
        string

        OIDC provider network address alias.

        Examples:

        oidcIssuerAddress: 1.2.3.4
        
        oidcIssuerAddress: ''
        
      • apiserver.authn.oidcIssuerURL
        string

        OIDC provider URL.

        Example:

        oidcIssuerURL: https://my-super-site.tech/
        
      • apiserver.authn.webhookCA
        string

        Authorization webhook CA.

      • apiserver.authn.webhookCacheTTL
        string

        The duration to cache responses from the webhook token authenticator.

        It is specified as a string containing the time unit in hours and minutes: 30m, 1h, 2h30m, 24h.

        Pattern: ^([0-9]+h)?([0-9]+m)?([0-9]+s)?$

        Example:

        webhookCacheTTL: 5m
        
      • apiserver.authn.webhookURL
        string

        Authentication webhook URL.

        Example:

        webhookURL: https://127.0.0.1:40443/
        
    • apiserver.authz
      object

      Optional authorization parameters for Kubernetes API clients.

      By default, they are taken from user-authz module ConfigMap.

      • apiserver.authz.webhookCA
        string

        Authorization webhook CA.

      • apiserver.authz.webhookURL
        string

        Authorization webhook URL.

        Example:

        webhookURL: https://127.0.0.1:40443/
        
    • apiserver.basicAuditPolicyEnabled
      boolean

      Enforce basic Deckhouse audit policies.

      Default: true

    • apiserver.bindToWildcard
      boolean

      Specifies whether to listen on 0.0.0.0.

      By default, the API server listens on the hostIP. The latter usually corresponds to the Internal node address; however, the actual IP depends on the cluster type (Static or Cloud) and the layout selected.

      Default: false

    • apiserver.certSANs
      array of strings

      Array of SANs, with which the API server certificate will be generated.

      In addition to the passed list, the following list is always used:

      • kubernetes;
      • kubernetes.default;
      • kubernetes.default.svc;
      • kubernetes.default.svc.cluster.local;
      • Kubernetes API server address;
      • 127.0.0.1;
      • host name;
      • host IP.

      Example:

      certSANs:
      - my-site.com
      - 192.168.67.76
      
      • Element of the array
        string

        Pattern: ^[0-9a-zA-Z\.-]+$

    • apiserver.encryptionEnabled
      boolean

      Enables encrypting secret data at rest.

      Generates kube-system/d8-secret-encryption-key Secret with encryption key.

      Note! This mode cannot be disabled!

      Default: false

    • apiserver.loadBalancer
      object

      If set, a service kube-system/d8-control-plane-apiserver of the LoadBalancer type will be created.

      • apiserver.loadBalancer.annotations
        object

        Annotations to attach to a service to fine-tune the load balancer.

        Caution! The module does not take into account the specifics of setting annotations in various cloud environments. If the annotations for load balancer provisioning are only applied when creating a service, you will need to delete and add the apiserver.loadBalancer parameter to update such parameters.

      • apiserver.loadBalancer.port
        integer

        External LoadBalancer TCP port.

        Default: 443

        Allowed values: 1 <= X <= 65534

      • apiserver.loadBalancer.sourceRanges
        array of strings

        A list of CIDRs that are allowed to connect to the API.

        The cloud provider may not support this option or ignore it.

        • Element of the array
          string

          Pattern: ^[0-9]{1,}\.[0-9]{1,}\.[0-9]{1,}\.[0-9]{1,}\/[0-9]+$

    • apiserver.serviceAccount
      object

      ServiceAccount issuing settings.

      Examples:

      serviceAccount: {}
      
      serviceAccount:
        additionalAPIAudiences:
        - istio-ca
      
      • apiserver.serviceAccount.additionalAPIAudiences
        array of strings

        A list of API audiences to add when provisioning ServiceAccount tokens.

      • apiserver.serviceAccount.issuer
        string

        ServiceAccount issuer. This is the URL of the API server. The values of this field are used as the iss claim of the token and to verify Service Account JWT tokens.

        Note, all pods in the cluster using ServiceAccount tokens must be restarted upon changing this option.

        Example:

        issuer: https://api.example.com
        
  • etcd
    object

    etcd parameters.

    • etcd.externalMembersNames
      array of strings

      etcd external members array (they will not be deleted).

      Example:

      externalMembersNames:
      - main-master-1
      - my-external-member
      
      • Element of the array
        string

        Pattern: ^[0-9a-zA-Z\.-:\-\/]+$

    • etcd.maxDbSize
      number

      quota-backend-bytes parameter. Deckhouse automatically manages the quota-backend-bytes parameter. If the maxDbSize parameter is set, deckhouse will use this value for the quota-backend-bytes etcd parameter.

      Minimum: 512MB.

      Maximum: 8GB.

      Experimental. It can be removed in the future.

      Allowed values: 536870912 <= X <= 8589934592

  • failedNodePodEvictionTimeoutSeconds
    integer

    The number of seconds after which pods will be deleted from the node with the Unreachable status.

    Note! If you change the parameter, the pods must be restarted.

    Default: 300

  • nodeMonitorGracePeriodSeconds
    integer

    The number of seconds after which the node will enter the Unreachable status in case of lost connection.

    Default: 40