Managing control plane: configuration

• Updates from version 1 to 2:

  Delete etcd.externalMembersNames

Schema version: 2

apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: control-plane-manager
spec:
  version: 2
  enabled: true
  settings:
    apiserver:
      bindToWildcard: true
      certSANs:
      - bakery.infra
      - devs.infra
      loadBalancer: {}

• settings

  object
  • settings.apiserver

    object

    kube-apiserver parameters.

    • settings.apiserver.admissionPlugins

      array of strings

      List of enabled additional admission plugins.

      Note, that in addition to the admission plugins enabled by default in Kubernetes, the following admission plugins are also always enabled:

      • ExtendedResourceToleration
      • PodNodeSelector
      • PodTolerationRestriction

      • EventRateLimit with the following config:

        apiVersion: eventratelimit.admission.k8s.io/v1alpha1
        kind: Configuration
        limits:
        - type: Namespace
          qps: 50
          burst: 100
          cacheSize: 2000
        

      Note that th PodNodeSelector admission plugin does not require specifiying a global configuration, it relies on annotated Namespaces.

      Example:

      admissionPlugins:
      - AlwaysPullImages
      - NamespaceAutoProvision
      

      • Element of the array

        string

        Allowed values: AlwaysPullImages, NamespaceAutoProvision, OwnerReferencesPermissionEnforcement, PodNodeSelector, PodTolerationRestriction

    • settings.apiserver.auditLog

      object

      Audit policy settings

      Default: {"output":"File"}

      • settings.apiserver.auditLog.output

        string

        Required value

        Audit logs target stream.

        Default: File

        Allowed values: File, Stdout

        Example:

        output: Stdout
        

      • settings.apiserver.auditLog.path

        string

        Directory path for logs if the output is “File”, otherwise ignored.

        Default: /var/log/kube-audit

        Pattern: ^[a-zA-Z0-9_/.-]+[a-zA-Z0-9_.-]$

    • settings.apiserver.auditPolicyEnabled

      boolean

      Set the audit policies using the configuration from the kube-system/audit-policy Secret.

      Default: false

    • settings.apiserver.authn

      object

      Optional authentication parameters for Kubernetes API clients.

      By default, they are taken from user-authn module ConfigMap.

      • settings.apiserver.authn.oidcCA

        string

        OIDC provider CA.

      • settings.apiserver.authn.oidcIssuerAddress

        string

        OIDC provider network address alias.

        Examples:

        oidcIssuerAddress: 1.2.3.4
        

        oidcIssuerAddress: ''
        

      • settings.apiserver.authn.oidcIssuerURL

        string

        OIDC provider URL.

        Example:

        oidcIssuerURL: https://my-super-site.tech/
        

      • settings.apiserver.authn.webhookCA

        string

        Authorization webhook CA.

      • settings.apiserver.authn.webhookCacheTTL

        string

        The duration to cache responses from the webhook token authenticator.

        It is specified as a string containing the time unit in hours and minutes: 30m, 1h, 2h30m, 24h.

        Pattern: ^([0-9]+h)?([0-9]+m)?([0-9]+s)?$

        Example:

        webhookCacheTTL: 5m
        

      • settings.apiserver.authn.webhookURL

        string

        Authentication webhook URL.

        Example:

        webhookURL: https://127.0.0.1:40443/
        

    • settings.apiserver.authz

      object

      Optional authorization parameters for Kubernetes API clients.

      By default, they are taken from user-authz module ConfigMap.

      • settings.apiserver.authz.webhookCA

        string

        Authorization webhook CA.

      • settings.apiserver.authz.webhookURL

        string

        Authorization webhook URL.

        Example:

        webhookURL: https://127.0.0.1:40443/
        

    • settings.apiserver.basicAuditPolicyEnabled

      boolean

      Enforce basic Deckhouse audit policies.

      Default: true

    • settings.apiserver.bindToWildcard

      boolean

      Specifies whether to listen on 0.0.0.0.

      By default, the API server listens on the hostIP. The latter usually corresponds to the Internal node address; however, the actual IP depends on the cluster type (Static or Cloud) and the layout selected.

      Default: false

    • settings.apiserver.certSANs

      array of strings

      Array of SANs, with which the API server certificate will be generated.

      In addition to the passed list, the following list is always used:

      • kubernetes;
      • kubernetes.default;
      • kubernetes.default.svc;
      • kubernetes.default.svc.cluster.local;
      • Kubernetes API server address;
      • 127.0.0.1;
      • host name;
      • host IP.

      Example:

      certSANs:
      - my-site.com
      - 192.168.67.76
      

      • Element of the array

        string

        Pattern: ^[0-9a-zA-Z\.-]+$

    • settings.apiserver.encryptionEnabled

      boolean

      Enables encrypting secret data at rest.

      Generates kube-system/d8-secret-encryption-key Secret with encryption key.

      Note! This mode cannot be disabled!

      Default: false

    • settings.apiserver.loadBalancer

      object

      If set, a service kube-system/d8-control-plane-apiserver of the LoadBalancer type will be created.

      • settings.apiserver.loadBalancer.annotations

        object

        Annotations to attach to a service to fine-tune the load balancer.

        Caution! The module does not take into account the specifics of setting annotations in various cloud environments. If the annotations for load balancer provisioning are only applied when creating a service, you will need to delete and add the apiserver.loadBalancer parameter to update such parameters.

      • settings.apiserver.loadBalancer.port

        integer

        External LoadBalancer TCP port.

        Default: 443

        Allowed values: 1 <= X <= 65534

      • settings.apiserver.loadBalancer.sourceRanges

        array of strings

        A list of CIDRs that are allowed to connect to the API.

        The cloud provider may not support this option or ignore it.

        • Element of the array

          string

          Pattern: ^[0-9]{1,}\.[0-9]{1,}\.[0-9]{1,}\.[0-9]{1,}\/[0-9]+$

    • settings.apiserver.serviceAccount

      object

      ServiceAccount issuing settings.

      Examples:

      serviceAccount: {}
      

      serviceAccount:
        additionalAPIAudiences:
        - istio-ca
      

      • settings.apiserver.serviceAccount.additionalAPIAudiences

        array of strings

        A list of API audiences to add when provisioning ServiceAccount tokens.

        The default audiences list includes:

        • serviceAccount.issuer, if specified.
        • serviceAccount.additionalAPIIssuers, if specified.
        • https://kubernetes.default.svc.${clusterDomain}, for example, https://kubernetes.default.svc.cluster.local – this audience is always added for backward compatibility.

        The service account token authenticator will validate that tokens used against the API are bound to at least one of these audiences.

        This option is beneficial when migrating from one API issuer to another.

      • settings.apiserver.serviceAccount.additionalAPIIssuers

        array of strings

        A list of additional issuers to include when provisioning ServiceAccount tokens.

        Issuers (iss) are used to verify the source of the tokens, ensuring they originate from trusted entities.

        The first issuer is automatically generated based on the template https://kubernetes.default.svc.${clusterDomain}, for example, https://kubernetes.default.svc.cluster.local. When multiple issuers are specified, the first issuer is used to generate tokens, and all provided issuers are accepted for token verification.

        This option is beneficial when migrating from one API issuer to another.

      • settings.apiserver.serviceAccount.issuer

        string

        ServiceAccount issuer. This is the URL of the API server. The values of this field are used as the iss claim of the token and to verify Service Account JWT tokens.

        Note, all pods in the cluster using ServiceAccount tokens must be restarted upon changing this option.

        Example:

        issuer: https://api.example.com
        

  • settings.etcd

    object

    etcd parameters.

    • settings.etcd.backup

      object

      etcd backup parameters.

      • settings.etcd.backup.cronSchedule

        string

        Backup schedule etcd in cron format. The local time zone of the kube-controller-manager is used.

        Default: 0 0 * * *

        Example:

        cronSchedule: 0 1 * * *
        

      • settings.etcd.backup.enabled

        boolean

        Enable etcd backup.

        Default: true

      • settings.etcd.backup.hostPath

        string

        Host path on masters nodes for etcd backup.

        Default: /var/lib/etcd

    • settings.etcd.maxDbSize

      number

      quota-backend-bytes parameter. Deckhouse automatically manages the quota-backend-bytes parameter. If the maxDbSize parameter is set, deckhouse will use this value for the quota-backend-bytes etcd parameter.

      Minimum: 512MB.

      Maximum: 8GB.

      Experimental. It can be removed in the future.

      Allowed values: 536870912 <= X <= 8589934592

  • settings.failedNodePodEvictionTimeoutSeconds

    integer

    The number of seconds after which pods will be deleted from the node with the Unreachable status.

    Note! If you change the parameter, the pods must be restarted.

    Default: 300

  • settings.nodeMonitorGracePeriodSeconds

    integer

    The number of seconds after which the node will enter the Unreachable status in case of lost connection.

    Default: 40