Modules settings

Schema version: 1

• settings

  object
  • settings.denyVulnerableImages

    object

    Trivy provider will deny creation of the Pod/Deployment/StatefulSet/DaemonSet with vulnerable images in namespaces with security.deckhouse.io/trivy-provider: "" label.

    This feature is available in enterprise edition only.

    • settings.denyVulnerableImages.enabled

      boolean

      Enable trivy provider.

      Default: false

    • settings.denyVulnerableImages.registrySecrets

      array of objects

      List of additional registry secrets to use for downloading images from private registries.

      By default, the deckhouse-registry secret is used to download images for scanning.

      Default: []

      • settings.denyVulnerableImages.registrySecrets.name

        string

        Required value

      • settings.denyVulnerableImages.registrySecrets.namespace

        string

        Required value

    • settings.denyVulnerableImages.storageClass

      string

      The name of the StorageClass to use for trivy-provider.

      If the value is not specified, the StorageClass will be used according to the global storageClass parameter setting.

      The global storageClass parameter is only considered when the module is enabled. Changing the global storageClass parameter while the module is enabled will not trigger disk re-provisioning.

      Warning. Specifying a value different from the one currently used (in the existing PVC) will result in disk re-provisioning and all data will be deleted.

      If false is specified, emptyDir will be forced to be used.

      Examples:

      storageClass: ceph-ssd
      

      storageClass: 'false'
      

  • settings.podSecurityStandards

    object

    Pod Security Standards policy settings (PSS).

    • settings.podSecurityStandards.defaultPolicy

      string

      Sets the default Pod Security Standards policy for all non-system namespaces:

      • Privileged — an unrestricted policy. Privilege escalation is possible with this policy;
      • Baseline — a policy with minimum restrictions that prevents privilege escalation;
      • Restricted — a policy with maximum restrictions that conforms to current best practices for securely running applications in a cluster.

      By default:

      • Baseline — if a Deckhouse version starting with v1.55 is being installed;
      • Privileged — if a Deckhouse version lower than v1.55 is being installed (upgrading Deckhouse in a cluster to v1.55+ does not automatically result in a default policy change).

      Allowed values: Privileged, Baseline, Restricted

    • settings.podSecurityStandards.enforcementAction

      string

      The enforcement action to control what to do with the result of the constraint.

      • Deny — Deny action.
      • Dryrun — No action. It is used when debugging. Information about the event can be viewed in Grafana or in the console via kubectl.
      • Warn — Same as Dryrun. In addition to the event information, it provides some info on why that constraint would have been denied if you had set Deny instead of Warn.

      Default: "Deny"

      Allowed values: Warn, Deny, Dryrun

    • settings.podSecurityStandards.policies

      object

      Sets additional policy parameters.

      • settings.podSecurityStandards.policies.hostPorts

        object

        HostPort constraint settings.

        • settings.podSecurityStandards.policies.hostPorts.knownRanges

          array of objects

          Set the range of known ports which will be allowed in a hostPort binding.

          • settings.podSecurityStandards.policies.hostPorts.knownRanges.max

            integer
          • settings.podSecurityStandards.policies.hostPorts.knownRanges.min

            integer

Schema version: 1

• settings

  object
  • settings.cleanupOrphanSecrets

    boolean

    Delete a secret with a certificate automatically if the corresponding Certificate resource was deleted from the cluster.

    Default: false

    Examples:

    cleanupOrphanSecrets: true
    

    cleanupOrphanSecrets: false
    

  • settings.cloudDNSServiceAccount

    string

    The Service Account for Google Cloud for the same project that has the DNS Administrator role.

    Example:

    cloudDNSServiceAccount: eyJzYSI6ICJhYmNkZWZnaEBzZXJ2aWNlYWNjb3VudC5jb20iLCAicHJvamVjdF9pZCI6ImFhYWFhIn0=
    

  • settings.cloudflareAPIToken

    string

    API Tokens allow application-scoped keys bound to specific DNS zones.

    API Tokens are recommended for higher security, since they have more restrictive permissions and are more easily revocable.

    It allows you to verify that domains specified in the Certificate resource are managed by cert-manager and kept by the Cloudflare DNS provider. Verification is performed by adding special TXT records for the ACME DNS01 Challenge Provider domain.

    Example:

    cloudflareAPIToken: token
    

  • settings.cloudflareEmail

    string

    The email used for accessing the Cloudflare platform.

    Example:

    cloudflareEmail: example@example.com
    

  • settings.cloudflareGlobalAPIKey

    string

    The Cloudflare Global API key for managing DNS records

    It allows you to verify that domains specified in the Certificate resource are managed by cert-manager and kept by the Cloudflare DNS provider.

    Verification is performed by adding special TXT records for the ACME DNS01 Challenge Provider domain.

    Example:

    cloudflareGlobalAPIKey: key
    

  • settings.digitalOceanCredentials

    string

    The Access Token for the Digital Ocean API (you can create it in the API section).

    Example:

    digitalOceanCredentials: creds
    

  • settings.disableLetsencrypt

    boolean

    Disable letsencrypt and letsencrypt-staging ClusterIssuer objects (if set to true).

    Examples:

    disableLetsencrypt: true
    

    disableLetsencrypt: false
    

  • settings.email

    string

    The email used for sending notifications by LetsEncrypt.

    Example:

    email: example@example.com
    

  • settings.enableCAInjector

    boolean

    Enable CAInjector. It only needs to inject CA certs into ValidatingWebhookConfiguration, MutatingWebhookConfiguration, CustomResourceDefinition and APIService. Deckhouse does not use CAInjector, so you have to enable it only if you use custom CA injections in your services.

    Default: false

    Examples:

    enableCAInjector: true
    

    enableCAInjector: false
    

  • settings.maxConcurrentChallenges

    integer

    The maximum number of challenges that can be scheduled as ‘processing’ at once. (default 60)

    Allowed values: 0 <= X

    Example:

    maxConcurrentChallenges: 25
    

  • settings.nodeSelector

    object

    The same as in the pods’ spec.nodeSelector parameter in Kubernetes.

    If the parameter is omitted or false, it will be determined automatically.

    Example:

    nodeSelector:
      has-gpu: 'true'
    

  • settings.route53AccessKeyID

    string

    The Access Key ID of the user with the attached Amazon Route53 IAM Policy for managing domain records.

    Example:

    route53AccessKeyID: key_id
    

  • settings.route53SecretAccessKey

    string

    The Secret Access Key of the user with privileges to manage domain records.

    Example:

    route53SecretAccessKey: secret
    

  • settings.tolerations

    array of objects

    The same as in the pods’ spec.tolerations parameter in Kubernetes.

    If the parameter is omitted or false, it will be determined automatically.

    Example:

    tolerations:
    - key: dedicated.deckhouse.io
      operator: Equal
      value: cert-manager
    

    • settings.tolerations.effect

      string
    • settings.tolerations.key

      string
    • settings.tolerations.operator

      string
    • settings.tolerations.tolerationSeconds

      integer
    • settings.tolerations.value

      string

Schema version: 1

• settings

  object
  • settings.ntpServers

    array of strings

    NTP servers to sync.

    Default: ["pool.ntp.org"]

    Examples:

    ntpServers:
    - pool.ntp.org
    

    ntpServers:
    - pool.ntp.org
    - ntp.ubuntu.com
    - time.google.com
    

Schema version: 2

• settings

  object
  • settings.auth

    object

    Options related to authentication or authorization in the Hubble web UI.

    • settings.auth.allowedUserGroups

      array of strings

      An array of user groups that can access Hubble web UI.

      This parameter is used if the user-authn module is enabled or the externalAuthentication parameter is set.

      Caution! Note that you must add those groups to the appropriate field in the DexProvider config if this module is used together with the user-authn one.

    • settings.auth.externalAuthentication

      object

      Parameters to enable external authentication based on the NGINX Ingress external-auth mechanism that uses the Nginx auth_request module.

      External authentication is enabled automatically if the user-authn module is enabled.

      • settings.auth.externalAuthentication.authSignInURL

        string

        The URL to redirect the user for authentication (if the authentication service returned a non-200 HTTP response code).

        Example:

        authSignInURL: https://example.com/dex/sign_in
        

      • settings.auth.externalAuthentication.authURL

        string

        The URL of the authentication service. If the user is authenticated, the service should return an HTTP 200 response code.

        Example:

        authURL: https://example.com/dex/auth
        

    • settings.auth.whitelistSourceRanges

      array of strings

      An array if CIDRs that are allowed to authenticate in Hubble web UI.

      Example:

      whitelistSourceRanges:
      - 1.1.1.1/32
      

  • settings.debugLogging

    boolean

    Enabled debug logging for Cilium Hubble component.

    Default: false

  • settings.https

    object

    What certificate type to use.

    This parameter completely overrides the global.modules.https settings.

    Examples:

    https:
      mode: Disabled
    

    https:
      mode: OnlyInURI
    

    https:
      mode: CustomCertificate
      customCertificate:
        secretName: foobar
    

    https:
      mode: CertManager
      certManager:
        clusterIssuerName: letsencrypt
    

    • settings.https.certManager

      object

      Parameters for certmanager.

      • settings.https.certManager.clusterIssuerName

        string

        What ClusterIssuer to use for getting an SSL certificate (currently, letsencrypt, letsencrypt-staging, selfsigned are available; also, you can define your own).

        Default: "letsencrypt"

        Examples:

        clusterIssuerName: letsencrypt
        

        clusterIssuerName: letsencrypt-staging
        

        clusterIssuerName: selfsigned
        

    • settings.https.customCertificate

      object

      Parameters for custom certificate usage.

      • settings.https.customCertificate.secretName

        string

        The name of the secret in the d8-system namespace to use with the Hubble web UI.

        This secret must have the kubernetes.io/tls format.

    • settings.https.mode

      string

      The HTTPS usage mode:

      • CertManager — the web UI is accessed over HTTPS using a certificate obtained from a clusterIssuer specified in the certManager.clusterIssuerName parameter;
      • CustomCertificate — the web UI is accessed over HTTPS using a certificate from the d8-system namespace;
      • Disabled — in this mode, the documentation web UI can only be accessed over HTTP;
      • OnlyInURI — the documentation web UI will work over HTTP (thinking that there is an external HTTPS load balancer in front of it that terminates HTTPS traffic). All the links in the user-authn will be generated using the HTTPS scheme. Load balancer should provide a redirect from HTTP to HTTPS.

      Default: "CertManager"

      Allowed values: Disabled, CertManager, CustomCertificate, OnlyInURI

  • settings.ingressClass

    string

    The class of the Ingress controller used for Hubble.

    Optional. By default, the modules.ingressClass global value is used.

    Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$

  • settings.nodeSelector

    object

    The same as the spec.nodeSelector pod parameter in Kubernetes.

    If the parameter is omitted or false, it will be determined automatically.

  • settings.tolerations

    array of objects

    The same as spec.tolerations for the Kubernetes Pod.

    If the parameter is omitted or false, it will be determined automatically.

    • settings.tolerations.effect

      string
    • settings.tolerations.key

      string
    • settings.tolerations.operator

      string
    • settings.tolerations.tolerationSeconds

      integer
    • settings.tolerations.value

      string

Schema version: 1

• settings

  object
  • settings.bpfLBMode

    string

    eBPF LoadBalancer mode:

    • SNAT — traffic from the client to the pod (and back) passes through NAT, and accordingly the sender’s address is lost.
    • DSR — traffic from the client to the pod passes with the sender’s address preserved, and back - according to the routing rules (bypassing the balancer). This mode saves network traffic and reduces delays, but only works for TCP traffic.
    • Hybrid — TCP traffic is processed in DSR mode, and UDP traffic is processed in SNAT mode.

    Default: "Hybrid"

    Allowed values: SNAT, Hybrid, DSR

  • settings.createNodeRoutes

    boolean

    Create routes to Pods on other Nodes.

    All Nodes must be located in the same L2 domain.

  • settings.debugLogging

    boolean

    Enabled debug logging for Cilium components.

    Default: false

  • settings.labelsRegex

    array of strings

    Cilium creates security identifiers based on the k8s entity labels. The more labels are involved in this process, the higher the access granularity that can be achieved. However, in large clusters, excessive granularity can create a heavy load. You can use the labelsRegex option to explicitly specify which labels are to be used for security policies, and which labels are to be neglected. Refer to the documentation to learn more about reducing the identity cardinality

    Each label must be specified by a RegExp expression in YAML quoted string format. Note that special characters must be escaped.

    Example:

    labelsRegex:
    - k8s:!app\.kubernetes\.io
    - k8s:io\.cilium\.k8s\.policy
    

  • settings.masqueradeMode

    string

    Cilium masquerade work mode for pods traffic leaving the cluster.

    • BPF - use cilium BPF. Basic operation mode.

      In this mode, masquerade will not be used if the destination IP address is within the podSubnetCIDR or InternalIP/ExternalIP of any of the cluster nodes.

    • Netfilter - use kernel Netfilter(iptables/nf_tables).

      In this mode, masquerade will not be used if the destination IP address is within the podSubnetCIDR or InternalIP/ExternalIP of any of the cluster nodes.

    In BPF mode, if ExternalIP other than InternalIP is specified for a cluster node, and that IP address is not currently assigned to any interface on the node, it will not be reachable from neighboring nodes from pods network. The Netfilter mode allows you to bypass this limitation, but some other features stop working, such as EgressGateway`.

    Default: "BPF"

    Allowed values: Netfilter, BPF

  • settings.policyAuditMode

    boolean

    Do not enforce any Network Policy. Just log allowed/disallowed connections.

    Default: false

  • settings.resourcesManagement

    Settings for CPU and memory requests and limits by cilium agent pods.

    Examples:

    resourcesManagement:
      mode: VPA
      vpa:
        mode: Auto
        cpu:
          min: 50m
          max: 2
          limitRatio: 1.5
        memory:
          min: 256Mi
          max: 2Gi
          limitRatio: 1.5
    

    resourcesManagement:
      mode: Static
      static:
        requests:
          cpu: 55m
          memory: 256Mi
        limits:
          cpu: 2
          memory: 2Gi
    

    • settings.resourcesManagement.mode

      string

      Resource management mode:

      • Static is a classic one. In it, you specify requests/limits. The parameters of this mode are defined in the static parameter section;
      • VPA mode uses VPA. You can configure this mode by modifying parameters in the vpa parameter section.

      Default: "VPA"

      Allowed values: VPA, Static

    • settings.resourcesManagement.static

      object

      Resource management options for the Static mode.

      • settings.resourcesManagement.static.limits

        object

        Configuring CPU and memory limits.

        • settings.resourcesManagement.static.limits.cpu

          Configuring CPU limits.

        • settings.resourcesManagement.static.limits.memory

          Configuring memory limits.

      • settings.resourcesManagement.static.requests

        object

        Resource requests settings for pods.

        • settings.resourcesManagement.static.requests.cpu

          Configuring CPU requests.

        • settings.resourcesManagement.static.requests.memory

          Configuring memory requests.

    • settings.resourcesManagement.vpa

      object

      Resource management options for the VPA mode.

      • settings.resourcesManagement.vpa.cpu

        object

        CPU-related VPA settings.

        • settings.resourcesManagement.vpa.cpu.limitRatio

          number

          The CPU limits/requests ratio.

          This ratio is used for calculating the initial CPU limits for a pod.

          If this parameter is set, the VPA will recalculate the CPU limits while maintaining the specified limits/requests ratio.

        • settings.resourcesManagement.vpa.cpu.max

          The maximum value that the VPA can set for the CPU requests.

          Default: 4

        • settings.resourcesManagement.vpa.cpu.min

          The minimum value that the VPA can set for the CPU requests.

          Default: 100m

      • settings.resourcesManagement.vpa.memory

        object

        Memory-related VPA settings.

        • settings.resourcesManagement.vpa.memory.limitRatio

          number

          The memory limits/requests ratio.

          This ratio is used for calculating the initial memory limits for a pod.

          If this parameter is set, the VPA will recalculate the memory limits while maintaining the specified limits/requests ratio.

        • settings.resourcesManagement.vpa.memory.max

          The maximum memory requests the VPA can set.

          Default: 4Gi

        • settings.resourcesManagement.vpa.memory.min

          The minimum memory requests the VPA can set.

          Default: 128Mi

      • settings.resourcesManagement.vpa.mode

        string

        VPA operating mode.

        Default: "Initial"

        Allowed values: Initial, Auto

  • settings.svcSourceRangeCheck

    boolean

    For services of the loadBalancer type, enables checking the source IP for compliance with loadBalancer.sourceRanges. The check is not performed if access is made from within the cluster or the sourceRanges parameter is empty/not specified. Some cloud providers support filtering of incoming traffic based on the sourceRanges parameter in their loadBalancer implementations. In such cases, the cilium documentation recommends disabling the svcSourceRangeCheck option.

    Do not enable if another balancer or any type of SNAT is running in front of the services.

    Default: false

  • settings.tunnelMode

    string

    Tunnel mode.

    Note! After changing the parameter, it is necessary to restart all nodes, otherwise, there may be problems with the availability of Pods!

    Note! VXLAN mode is not compatible with following bpfLBMode modes: Hybrid, DSR. bpfLBMode will be overriden to SNAT if the tunnelMode is VXLAN.

    Default: "Disabled"

    Allowed values: Disabled, VXLAN

Schema version: 1

apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: control-plane-manager
spec:
  version: 1
  enabled: true
  settings:
    apiserver:
      bindToWildcard: true
      certSANs:
      - bakery.infra
      - devs.infra
      loadBalancer: {}

• settings

  object
  • settings.apiserver

    object

    kube-apiserver parameters.

    • settings.apiserver.admissionPlugins

      array of strings

      List of enabled additional admission plugins.

      Note, that in addition to the admission plugins enabled by default in Kubernetes, the following admission plugins are also always enabled:

      • ExtendedResourceToleration
      • PodNodeSelector
      • PodTolerationRestriction

      • EventRateLimit with the following config:

        apiVersion: eventratelimit.admission.k8s.io/v1alpha1
        kind: Configuration
        limits:
        - type: Namespace
          qps: 50
          burst: 100
          cacheSize: 2000
        

      Note that th PodNodeSelector admission plugin does not require specifiying a global configuration, it relies on annotated Namespaces.

      Example:

      admissionPlugins:
      - AlwaysPullImages
      - NamespaceAutoProvision
      

      • Element of the array

        string

        Allowed values: AlwaysPullImages, NamespaceAutoProvision, OwnerReferencesPermissionEnforcement, PodNodeSelector, PodTolerationRestriction

    • settings.apiserver.auditLog

      object

      Audit policy settings

      Default: {"output":"File"}

      • settings.apiserver.auditLog.output

        string

        Required value

        Audit logs target stream.

        Default: "File"

        Allowed values: File, Stdout

        Example:

        output: Stdout
        

      • settings.apiserver.auditLog.path

        string

        Directory path for logs if the output is “File”, otherwise ignored.

        Default: "/var/log/kube-audit"

        Pattern: ^[a-zA-Z0-9_/.-]+[a-zA-Z0-9_.-]$

    • settings.apiserver.auditPolicyEnabled

      boolean

      Set the audit policies using the configuration from the kube-system/audit-policy Secret.

      Default: false

    • settings.apiserver.authn

      object

      Optional authentication parameters for Kubernetes API clients.

      By default, they are taken from user-authn module ConfigMap.

      • settings.apiserver.authn.oidcCA

        string

        OIDC provider CA.

      • settings.apiserver.authn.oidcIssuerAddress

        string

        OIDC provider network address alias.

        Examples:

        oidcIssuerAddress: 1.2.3.4
        

        oidcIssuerAddress: ''
        

      • settings.apiserver.authn.oidcIssuerURL

        string

        OIDC provider URL.

        Example:

        oidcIssuerURL: https://my-super-site.tech/
        

      • settings.apiserver.authn.webhookCA

        string

        Authorization webhook CA.

      • settings.apiserver.authn.webhookCacheTTL

        string

        The duration to cache responses from the webhook token authenticator.

        It is specified as a string containing the time unit in hours and minutes: 30m, 1h, 2h30m, 24h.

        Pattern: ^([0-9]+h)?([0-9]+m)?([0-9]+s)?$

        Example:

        webhookCacheTTL: 5m
        

      • settings.apiserver.authn.webhookURL

        string

        Authentication webhook URL.

        Example:

        webhookURL: https://127.0.0.1:40443/
        

    • settings.apiserver.authz

      object

      Optional authorization parameters for Kubernetes API clients.

      By default, they are taken from user-authz module ConfigMap.

      • settings.apiserver.authz.webhookCA

        string

        Authorization webhook CA.

      • settings.apiserver.authz.webhookURL

        string

        Authorization webhook URL.

        Example:

        webhookURL: https://127.0.0.1:40443/
        

    • settings.apiserver.basicAuditPolicyEnabled

      boolean

      Enforce basic Deckhouse audit policies.

      Default: true

    • settings.apiserver.bindToWildcard

      boolean

      Specifies whether to listen on 0.0.0.0.

      By default, the API server listens on the hostIP. The latter usually corresponds to the Internal node address; however, the actual IP depends on the cluster type (Static or Cloud) and the layout selected.

      Default: false

    • settings.apiserver.certSANs

      array of strings

      Array of SANs, with which the API server certificate will be generated.

      In addition to the passed list, the following list is always used:

      • kubernetes;
      • kubernetes.default;
      • kubernetes.default.svc;
      • kubernetes.default.svc.cluster.local;
      • Kubernetes API server address;
      • 127.0.0.1;
      • host name;
      • host IP.

      Example:

      certSANs:
      - my-site.com
      - 192.168.67.76
      

      • Element of the array

        string

        Pattern: ^[0-9a-zA-Z\.-]+$

    • settings.apiserver.encryptionEnabled

      boolean

      Enables encrypting secret data at rest.

      Generates kube-system/d8-secret-encryption-key Secret with encryption key.

      Note! This mode cannot be disabled!

      Default: false

    • settings.apiserver.loadBalancer

      object

      If set, a service kube-system/d8-control-plane-apiserver of the LoadBalancer type will be created.

      • settings.apiserver.loadBalancer.annotations

        object

        Annotations to attach to a service to fine-tune the load balancer.

        Caution! The module does not take into account the specifics of setting annotations in various cloud environments. If the annotations for load balancer provisioning are only applied when creating a service, you will need to delete and add the apiserver.loadBalancer parameter to update such parameters.

      • settings.apiserver.loadBalancer.port

        integer

        External LoadBalancer TCP port.

        Default: 443

        Allowed values: 1 <= X <= 65534

      • settings.apiserver.loadBalancer.sourceRanges

        array of strings

        A list of CIDRs that are allowed to connect to the API.

        The cloud provider may not support this option or ignore it.

        • Element of the array

          string

          Pattern: ^[0-9]{1,}\.[0-9]{1,}\.[0-9]{1,}\.[0-9]{1,}\/[0-9]+$

    • settings.apiserver.serviceAccount

      object

      ServiceAccount issuing settings.

      Examples:

      serviceAccount: {}
      

      serviceAccount:
        additionalAPIAudiences:
        - istio-ca
      

      • settings.apiserver.serviceAccount.additionalAPIAudiences

        array of strings

        A list of API audiences to add when provisioning ServiceAccount tokens.

        The defautl audiences is automatically generated based on the template https://kubernetes.default.svc.${clusterDomain}, for example, https://kubernetes.default.svc.cluster.local. The service account token authenticator will validate that tokens used against the API are bound to at least one of these audiences.

        This option is beneficial when migrating from one API issuer to another.

      • settings.apiserver.serviceAccount.additionalAPIIssuers

        array of strings

        A list of additional issuers to include when provisioning ServiceAccount tokens.

        Issuers (iss) are used to verify the source of the tokens, ensuring they originate from trusted entities.

        The first issuer is automatically generated based on the template https://kubernetes.default.svc.${clusterDomain}, for example, https://kubernetes.default.svc.cluster.local. When multiple issuers are specified, the first issuer is used to generate tokens, and all provided issuers are accepted for token verification.

        This option is beneficial when migrating from one API issuer to another.

      • settings.apiserver.serviceAccount.issuer

        string

        ServiceAccount issuer. This is the URL of the API server. The values of this field are used as the iss claim of the token and to verify Service Account JWT tokens.

        Note, all pods in the cluster using ServiceAccount tokens must be restarted upon changing this option.

        Example:

        issuer: https://api.example.com
        

  • settings.etcd

    object

    etcd parameters.

    • settings.etcd.externalMembersNames

      array of strings

      etcd external members array (they will not be deleted).

      Example:

      externalMembersNames:
      - main-master-1
      - my-external-member
      

      • Element of the array

        string

        Pattern: ^[0-9a-zA-Z\.-:\-\/]+$

    • settings.etcd.maxDbSize

      number

      quota-backend-bytes parameter. Deckhouse automatically manages the quota-backend-bytes parameter. If the maxDbSize parameter is set, deckhouse will use this value for the quota-backend-bytes etcd parameter.

      Minimum: 512MB.

      Maximum: 8GB.

      Experimental. It can be removed in the future.

      Allowed values: 536870912 <= X <= 8589934592

  • settings.failedNodePodEvictionTimeoutSeconds

    integer

    The number of seconds after which pods will be deleted from the node with the Unreachable status.

    Note! If you change the parameter, the pods must be restarted.

    Default: 300

  • settings.nodeMonitorGracePeriodSeconds

    integer

    The number of seconds after which the node will enter the Unreachable status in case of lost connection.

    Default: 40

Schema version: 2

• settings

  object
  • settings.certificates

    object

    Settings for monitoring the certificates in the Kubernetes cluster.

    • settings.certificates.exporterEnabled

      boolean

      Enables certExporter.

      Default: false

  • settings.events

    object

    Settings for monitoring the events in the Kubernetes cluster.

    • settings.events.exporterEnabled

      boolean

      Enables eventsExporter.

      Default: false

    • settings.events.severityLevel

      string

      Whether to expose only crucial events.

      Default: "OnlyWarnings"

      Allowed values: All, OnlyWarnings

  • settings.imageAvailability

    object

    Settings for monitoring the availability of images in the cluster.

    • settings.imageAvailability.exporterEnabled

      boolean

      Enables imageAvailabilityExporter.

      Default: true

    • settings.imageAvailability.forceCheckDisabledControllers

      array of strings

      A list of controller kinds for which image is forcibly checked, even when workloads are disabled or suspended.

      Specify All to check all controller kinds.

      Example:

      forceCheckDisabledControllers:
      - Deployment
      - StatefulSet
      

      • Element of the array

        string

        Allowed values: Deployment, StatefulSet, DaemonSet, CronJob, All

    • settings.imageAvailability.ignoredImages

      array of strings

      A list of images to ignore when checking the presence in the registry, e.g., alpine:3.12 or quay.io/test/test:v1.1.

      Example:

      ignoredImages:
      - alpine:3.10
      - alpine:3.2
      

    • settings.imageAvailability.mirrors

      array of objects

      List of mirrors for container registries.

      Example:

      mirrors:
      - original: docker.io
        mirror: mirror.gcr.io
      - original: internal-registry.com
        mirror: mirror.internal-registry.com
      

      • settings.imageAvailability.mirrors.mirror

        string

        Required value

      • settings.imageAvailability.mirrors.original

        string

        Required value

    • settings.imageAvailability.registry

      object

      Connection settings for container registry.

      • settings.imageAvailability.registry.scheme

        string

        Container registry access scheme.

        Default: "HTTPS"

        Allowed values: HTTP, HTTPS

      • settings.imageAvailability.registry.tlsConfig

        object

        Connection settings for container registry.

        • settings.imageAvailability.registry.tlsConfig.ca

          string

          Root CA certificate to validate the container registry’s HTTPS certificate (if self-signed certificates are used).

        • settings.imageAvailability.registry.tlsConfig.insecureSkipVerify

          boolean

          Whether to skip the verification of the container registry certificate.

          Default: false

  • settings.nodeSelector

    object

    The same as in the pods’ spec.nodeSelector parameter in Kubernetes.

    If the parameter is omitted or false, it will be determined automatically.

  • settings.tolerations

    array of objects

    The same as in the pods’ spec.tolerations parameter in Kubernetes;

    If the parameter is omitted or false, it will be determined automatically.

    • settings.tolerations.effect

      string
    • settings.tolerations.key

      string
    • settings.tolerations.operator

      string
    • settings.tolerations.tolerationSeconds

      integer
    • settings.tolerations.value

      string

Schema version: 2

• settings

  object
  • settings.defaultClusterStorageClass

    string

    The name of the StorageClass that DKP will define as the default StorageClass used in the cluster.

    DKP prohibits manually adding or modifying the default annotation (storageclass.kubernetes.io/is-default-class='true') for StorageClass. Use the parameter to define or change the StorageClass that is used as the default in the cluster.

    If the parameter is omitted, the default StorageClass is either:

    • A StorageClass in the cluster that has the default annotation (storageclass.kubernetes.io/is-default-class='true');
    • The first (alphabetically) StorageClass created by the cloud provider module (if any cloud provider module is used).

    Use this parameter to change the default StorageClass in the cluster, as manually adding the default annotation to a StorageClass is not allowed.

    Default: ""

  • settings.highAvailability

    boolean

    A global switch to enable the high availability mode for modules that support it.

    If not defined, the value is determined automatically as true for clusters with more than one master node. Otherwise, it is determined asfalse.

    Examples:

    highAvailability: true
    

    highAvailability: false
    

  • settings.modules

    object

    Common parameters of Deckhouse modules.

    • settings.modules.https

      object

      The HTTPS implementation used by the Deckhouse modules.

      Examples:

      https:
        certManager:
          clusterIssuerName: letsencrypt
        mode: CertManager
      

      https:
        mode: Disabled
      

      https:
        mode: OnlyInURI
      

      https:
        mode: CustomCertificate
        customCertificate:
          secretName: plainstring
      

      • settings.modules.https.certManager

        object
        • settings.modules.https.certManager.clusterIssuerName

          string

          Name of a ClusterIssuer to use for Deckhouse modules.

          The cert-manager module offers the following ClusterIssuer: letsencrypt, letsencrypt-staging, selfsigned, clouddns, cloudflare, digitalocean, route53. Also, you can use your own ClusterIssuer.

          Default: "letsencrypt"

      • settings.modules.https.customCertificate

        object
        • settings.modules.https.customCertificate.secretName

          string

          The name of the secret in the d8-system namespace to use with Deckhouse modules.

          This secret must have the kubernetes.io/tls format.

          Default: "false"

      • settings.modules.https.mode

        string

        The HTTPS usage mode:

        • CertManager — Deckhouse modules use HTTPS and get a certificate from the ClusterIssuer defined in the certManager.clusterIssuerName parameter;
        • CustomCertificate — Deckhouse modules use HTTPS using the certificate from the d8-system namespace;
        • Disabled — Deckhouse modules use HTTP only (some modules may not work, e.g., user-authn);
        • OnlyInURI — Deckhouse modules use HTTP in the expectation that an HTTPS load balancer runs in front of them and terminates HTTPS. Load balancer should provide a redirect from HTTP to HTTPS.

        Default: "CertManager"

        Allowed values: Disabled, CertManager, CustomCertificate, OnlyInURI

    • settings.modules.ingressClass

      string

      The class of the Ingress controller (Ingress class) used for Deckhouse modules.

      Default: "nginx"

      Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$

      Example:

      ingressClass: nginx
      

    • settings.modules.placement

      object

      Parameters regulating the layout of Deckhouse module components.

      • settings.modules.placement.customTolerationKeys

        array of strings

        A list of custom toleration keys; use them to allow the deployment of some critical add-ons (such as cni and csi) on dedicated nodes.

        Example:

        customTolerationKeys:
        - dedicated.example.com
        - node-dedicated.example.com/master
        

    • settings.modules.publicDomainTemplate

      string

      The template with the %s key as the dynamic string parameter.

      Deckhouse modules use this template for creating Ingress resources.

      E.g., if the template is %s.kube.company.my, the prometheus module will create an Ingress resource for the grafana.kube.company.my hosts to access Grafana.

      Do not use DNS names (nor do create Ingress resources) that match this template to avoid conflicts with the Ingress resources created by Deckhouse.

      Pay attention to the following:

      • Domain must be different from clusterDomain!**
      • Domain used in the template must not match the domain specified in the clusterDomain parameter. For example, if clusterDomain is set to cluster.local (the default value), publicDomainTemplate cannot be set to %s.cluster.local.
      • Domain used in the template should not match the domain specified in the clusterDomain parameter and the internal service network zone. For example, if clusterDomain is set to cluster.local (default value) and the service network zone is ru-central1.internal, then publicDomainTemplate cannot be %s.cluster.local or %s.ru-central1.internal.
      • If this parameter is omitted, no Ingress resources will be created.

      Pattern: ^(%s([-a-z0-9]*[a-z0-9])?|[a-z0-9]([-a-z0-9]*)?%s([-a-z0-9]*)?[a-z0-9]|[a-z0-9]([-a-z0-9]*)?%s)(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$

      Examples:

      publicDomainTemplate: "%s.kube.company.my"
      

      publicDomainTemplate: kube-%s.company.my
      

    • settings.modules.resourcesRequests

      object

      The amount of resources (CPU and memory) allocated to Deckhouse components running on each node of the cluster (usually these are DaemonSets, for example, cni-flannel, monitoring-ping).

      More about resource units in Kubernetes.

      • settings.modules.resourcesRequests.controlPlane

        object

        The amount of resources (CPU and memory) allocated to control-plane components on each master node. Do not work in clouds with not-managed control-plane (GKE for example).

        Example:

        controlPlane:
          cpu: 1000m
          memory: 500M
        

        • settings.modules.resourcesRequests.controlPlane.cpu

          The combined CPU requests for control-plane components on each master node.

        • settings.modules.resourcesRequests.controlPlane.memory

          string

          The combined memory requests for control-plane components on each master node.

          Pattern: ^[0-9]+(\.[0-9]+)?(E|P|T|G|M|K|Ei|Pi|Ti|Gi|Mi|Ki)?$

    • settings.modules.storageClass

      string

      The StorageClass to use with all Deckhouse Kubernetes Platform components (Prometheus, Grafana, OpenVPN, etc.).

      • If the value is not specified, the default StorageClass set in the cluster is used.
      • Use this parameter only in exceptional circumstances.
      • This parameter is applied during module activation.

      Default: ""

  • settings.storageClass

    Deprecated

    string

    DEPRECATED. Moved to global.modules.storageClass.

    The StorageClass to use with all Deckhouse Kubernetes Platform components (Prometheus, Grafana, OpenVPN, etc.).

    • If the value is not specified, the default StorageClass set in the cluster is used.
    • Use this parameter only in exceptional circumstances.
    • This parameter is applied during module activation.

    Default: ""

Schema version: 1

• settings

  object
  • settings.defaultControllerVersion

    The version of the ingress-nginx controller that is used for all controllers by default if the controllerVersion parameter is omitted in the IngressNginxController CR.

    Default: 1.9

  • settings.highAvailability

    boolean

    Manually enable the high availability mode.

    By default, Deckhouse automatically decides whether to enable the HA mode. Click here to learn more about the HA mode for modules.

    Example:

    highAvailability: true
    

Schema version: 1

• settings

  object
  • settings.debug

    boolean

    Enable or disable agent debug logging. Default: false.

    Default: false

  • settings.nodeSelector

    object

    The same as in the Pods’ spec.nodeSelector parameter in Kubernetes.

    If the parameter is omitted or false, nodeSelector will be determined automatically.

  • settings.resourcesRequests

    object

    Max amounts of CPU and memory resources that the pod can request when selecting a node.

    If the vertical-pod-autoscaler module is disabled, then these values become the default ones.

    Examples:

    resourcesRequests:
      mode: VPA
      vpa:
        mode: Auto
        cpu:
          min: 50m
          max: 2
        memory:
          min: 256Mi
          max: 2Gi
    

    resourcesRequests:
      mode: VPA
      vpa:
        mode: Initial
        cpu:
          min: 1
          max: 3000m
        memory:
          min: 1024
          max: 4096
    

    resourcesRequests:
      mode: Static
      static:
        cpu: 55m
        memory: 256Ki
    

    • settings.resourcesRequests.mode

      string

      Required value

      Resource request management mode:

      • Static is a classic one. In it, you explicitly specify requests. The parameters of this mode are defined in the static parameter section;
      • VPA mode uses VPA. You can configure this mode by modifying parameters in the vpa parameter section.

      Default: "VPA"

      Allowed values: VPA, Static

    • settings.resourcesRequests.static

      object

      Resource request management options for the Static mode.

      • settings.resourcesRequests.static.cpu

        Required value

        Configuring CPU requests.

        Default: 50m

      • settings.resourcesRequests.static.memory

        Required value

        Configuring memory requests.

        Default: 64Mi

    • settings.resourcesRequests.vpa

      object

      Resource request management options for the VPA mode.

      • settings.resourcesRequests.vpa.cpu

        object

        Required value

        CPU-related parameters.

        • settings.resourcesRequests.vpa.cpu.max

          The maximum value that the VPA can set for the CPU requests.

          Default: 500m

        • settings.resourcesRequests.vpa.cpu.min

          The minimum value that the VPA can set for the CPU requests.

          Default: 50m

      • settings.resourcesRequests.vpa.memory

        object

        Required value

        The amount of memory requested.

        • settings.resourcesRequests.vpa.memory.max

          The maximum memory requests the VPA can set.

          Default: 2048Mi

        • settings.resourcesRequests.vpa.memory.min

          The minimum memory requests the VPA can set.

          Default: 64Mi

      • settings.resourcesRequests.vpa.mode

        string

        Required value

        The VPA usage mode.

        Default: "Initial"

        Allowed values: Initial, Auto

  • settings.tolerations

    array of objects

    The same as in the Pods’ spec.tolerations parameter in Kubernetes;

    If the parameter is omitted or false, tolerations will be determined automatically.

    • settings.tolerations.effect

      string
    • settings.tolerations.key

      string
    • settings.tolerations.operator

      string
    • settings.tolerations.tolerationSeconds

      integer
    • settings.tolerations.value

      string

Schema version: 1

• settings

  object
  • settings.diskSizeGigabytes

    integer

    Disk size for log storage or PVC size when specifying the storageClass parameters. You should manually specify the required disk size for storing logs for the period defined by the retentionPeriodHours parameter. Loki cannot automatically delete old data when the disk is full. The extended-monitoring module automatically monitors the percentage of used disk space. CAUTION! If the Loki data disk becomes full, the application will stop working. You must find the right balance between retentionPeriodHours and diskSizeGigabytes values.

    Default: 2

  • settings.lokiConfig

    object

    Loki configuration options available for tuning.

    See Loki documentation for more information.

    Caution! Incorrect settings may lead to the malfunction of Loki.

    • settings.lokiConfig.grpcServerMaxConcurrentStreams

      integer

      Limit on the number of concurrent streams for gRPC calls.

      Default: 100

    • settings.lokiConfig.grpcServerMaxRecvMsgSize

      integer

      Limit on the size of a gRPC message this server can receive in bytes.

      Default: 104857600

    • settings.lokiConfig.grpcServerMaxSendMsgSize

      integer

      Limit on the size of a gRPC message this server can send in bytes.

      Default: 4194304

    • settings.lokiConfig.ingestionBurstSizeMB

      number

      Overall ingestion burst size in megabytes. E.g., maximum push request size.

      Default: 6

    • settings.lokiConfig.ingestionRateMB

      number

      Overall ingestion rate limit in megabytes per second.

      Default: 4

    • settings.lokiConfig.maxChunksPerQuery

      integer

      The maximum number of chunks that can be fetched per query.

      Default: 2000000

    • settings.lokiConfig.maxEntriesLimitPerQuery

      integer

      The maximum number of entries returned for the query.

      Default: 5000

    • settings.lokiConfig.maxStreamsPerUser

      integer

      The maximum total number of streams for the Loki instance.

      Each stream has unique labels. The number of streams is the number of unique label sets. E. g. {job="my-app", instance="kube-node-1", namespace="production", pod_name="backend-79dbf6fcb7-v5gs7", app="backend"}.

      0 means unlimited.

      Default: 0

    • settings.lokiConfig.perStreamRateLimit

      Maximum byte rate per second per stream. Can be either a number or a string with a unit (KB, MB, GB).

      Default: 3MB

    • settings.lokiConfig.perStreamRateLimitBurst

      Maximum burst size per stream. Can be either a number or a string with a unit (KB, MB, GB).

      Default: 15MB

  • settings.nodeSelector

    object

    The same as the Pods’ spec.nodeSelector parameter in Kubernetes. If the parameter is omitted or false, nodeSelector will be determined automatically.

  • settings.resourcesManagement

    Settings for CPU and memory requests and limits by Loki pods.

    Examples:

    resourcesManagement:
      mode: VPA
      vpa:
        mode: Auto
        cpu:
          min: 50m
          max: 2
          limitRatio: 1.5
        memory:
          min: 256Mi
          max: 2Gi
          limitRatio: 1.5
    

    resourcesManagement:
      mode: Static
      static:
        requests:
          cpu: 55m
          memory: 256Mi
        limits:
          cpu: 2
          memory: 2Gi
    

    • settings.resourcesManagement.mode

      string

      Resource management mode:

      • Static is a classic one. In it, you specify requests/limits. The parameters of this mode are defined in the static parameter section;
      • VPA mode uses VPA. You can configure this mode by modifying parameters in the vpa parameter section.

      Default: "VPA"

      Allowed values: VPA, Static

    • settings.resourcesManagement.static

      object

      Resource management options for the Static mode.

      • settings.resourcesManagement.static.limits

        object

        Configuring CPU and memory limits.

        • settings.resourcesManagement.static.limits.cpu

          Configuring CPU limits.

        • settings.resourcesManagement.static.limits.memory

          Configuring memory limits.

      • settings.resourcesManagement.static.requests

        object

        Resource requests settings for pods.

        • settings.resourcesManagement.static.requests.cpu

          Configuring CPU requests.

        • settings.resourcesManagement.static.requests.memory

          Configuring memory requests.

    • settings.resourcesManagement.vpa

      object

      Resource management options for the VPA mode.

      • settings.resourcesManagement.vpa.cpu

        object

        CPU-related VPA settings.

        • settings.resourcesManagement.vpa.cpu.limitRatio

          number

          The CPU limits/requests ratio.

          This ratio is used for calculating the initial CPU limits for a pod.

          If this parameter is set, the VPA will recalculate the CPU limits while maintaining the specified limits/requests ratio.

        • settings.resourcesManagement.vpa.cpu.max

          The maximum value that the VPA can set for the CPU requests.

          Default: 2

        • settings.resourcesManagement.vpa.cpu.min

          The minimum value that the VPA can set for the CPU requests.

          Default: 50m

      • settings.resourcesManagement.vpa.memory

        object

        Memory-related VPA settings.

        • settings.resourcesManagement.vpa.memory.limitRatio

          number

          The memory limits/requests ratio.

          This ratio is used for calculating the initial memory limits for a pod.

          If this parameter is set, the VPA will recalculate the memory limits while maintaining the specified limits/requests ratio.

        • settings.resourcesManagement.vpa.memory.max

          The maximum memory requests the VPA can set.

          Default: 2Gi

        • settings.resourcesManagement.vpa.memory.min

          The minimum memory requests the VPA can set.

          Default: 256Mi

      • settings.resourcesManagement.vpa.mode

        string

        VPA operating mode.

        Default: "Auto"

        Allowed values: Initial, Auto

  • settings.retentionPeriodHours

    integer

    How many hours to keep logs before deleting.

    Default: 168

  • settings.storageClass

    The name of StorageClass that will be used in the cluster by default.

    If omitted, the StorageClass of the existing Loki PVC is used. If there is no PVC yet, the StorageClass will be used according to the global storageClass parameter setting.

    The global storageClass parameter is only considered when the module is enabled. Changing the global storageClass parameter while the module is enabled will not trigger disk re-provisioning.

    Warning. Specifying a value different from the one currently used (in the existing PVC) will result in disk re-provisioning and all data will be deleted.

    Examples:

    storageClass: false
    

    storageClass: default
    

  • settings.storeSystemLogs

    boolean

    Save logs from the d8-* namespaces to loki.

    The log-shipper module must be enabled.

    Default: true

  • settings.tolerations

    array of objects

    The same as the Pods’ spec.tolerations parameter in Kubernetes. If the parameter is omitted or false, tolerations will be determined automatically.

    • settings.tolerations.effect

      string
    • settings.tolerations.key

      string
    • settings.tolerations.operator

      string
    • settings.tolerations.tolerationSeconds

      integer
    • settings.tolerations.value

      string

Schema version: 2

bgpPeers:
- peer-address: 192.168.1.1
  peer-asn: 1111
  my-asn: 2222
  source-address: 192.168.1.2
  hold-time: 10s
  node-selector:
    matchLabels:
      node: test
addressPools:
- name: my-pool-bgp
  protocol: bgp
  addresses:
  - 192.168.100.1-192.168.100.10
  - 192.168.101.0/24
  bgp-advertisements:
  - aggregation-length: 32
    localpref: 100
    communities:
    - no-advertise
bgpCommunities:
  no-advertise: 65535:65282
speaker:
  nodeSelector:
    mylabel: speaker

addressPools:
- name: my-pool-l2
  protocol: layer2
  addresses:
  - 192.168.100.1-192.168.100.10
  - 192.168.101.0/24
speaker:
  nodeSelector:
    mylabel: speaker
  tolerations:
  - key: dedicated
    operator: Equal
    value: speaker

• settings

  object
  • settings.addressPools

    array of objects

    A list of IP ranges to assign to services.

    Format — a data array similar to that of MetalLB’s.

    Default: []

    • settings.addressPools.addresses

      array of strings

      A list of ranges, where each range can look like a subnet/mask or a numeric address range (with “-“ as a delimiter).

    • settings.addressPools.auto-assign

      boolean

      Auto-assign flag used to prevent MetalLB from automatic allocation for a pool.

      Default: true

    • settings.addressPools.avoid-buggy-ips

      boolean

      Prevents addresses ending with .0 and .255 to be used by a pool.

      Default: false

    • settings.addressPools.bgp-advertisements

      array of objects

      Defines BGP advertisements.

      • settings.addressPools.bgp-advertisements.aggregation-length

        integer

        The aggregation-length advertisement option lets you “roll up” prefix into a larger one.

        Works for IPv4 addresses.

        Default: 32

        Allowed values: 1 <= X

      • settings.addressPools.bgp-advertisements.communities

        array of strings

        Keys from the bgpCommunities parameter to be associated with the announcement.

        Example:

        communities:
        - no-advertise
        

      • settings.addressPools.bgp-advertisements.localpref

        integer

        The BGP LOCAL_PREF attribute which is used by BGP best path algorithm.

        Path with higher localpref is preferred over one with lower localpref.

    • settings.addressPools.name

      string

      The name of the pool (you can specify it using the service annotation as follows: metallb.universe.tf/address-pool: <name>). It should conform to RFC 1123: dot-separated parts in lowercase, consists of alphanumeric characters, ‘-‘. Each part must start and end with an alphanumeric character.

      Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$

    • settings.addressPools.protocol

      string

      The protocol used by the speaker to announce services.

      Allowed values: bgp, layer2

  • settings.bgpCommunities

    object

    The BGP communities list.

    Example:

    bgpCommunities:
      no-advertise: 65535:65282
    

  • settings.bgpPeers

    array of objects

    A list of external BGP routers to use with the module.

    Format — a data array similar to that of MetalLB’s.

    The parameter is optional if only the layer2 mode is used to announce services.

    Default: []

    • settings.bgpPeers.hold-time

      integer or string

      The timeout after which the neighboring BGP peer is considered dead. This value is divided by three to get the keep-alive interval.

      The recommended value is 3s (i.e., keep-alive packets are sent once per second). Note that the BGP protocol does not support values lower than this. By default, the parameter is set to 90s (i.e., keep-alive packets are sent every 30 seconds).

      Pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$

    • settings.bgpPeers.my-asn

      integer

      Required value

      The AS number in the cluster.

      Allowed values: 0 <= X <= 4294967295

    • settings.bgpPeers.node-selector

      object

      The additional pseudo-selector implemented by the speaker application. It selects nodes that are allowed to connect to external BGP routers. Do not confuse it with speaker.nodeSelector and nodeSelector.

      An optional parameter.

      The format is matchLabels or matchExpressions.

      • settings.bgpPeers.node-selector.matchExpressions

        array of objects
        • settings.bgpPeers.node-selector.matchExpressions.key

          string
        • settings.bgpPeers.node-selector.matchExpressions.operator

          string
        • settings.bgpPeers.node-selector.matchExpressions.values

          array of strings
      • settings.bgpPeers.node-selector.matchLabels

        object
    • settings.bgpPeers.password

      string

      Authentication password for routers enforcing TCP MD5 authenticated sessions.

    • settings.bgpPeers.peer-address

      string

      Required value

      The IP address of the external BGP router.

      Pattern: ^([0-9]{1,3}\.){3}[0-9]{1,3}$

    • settings.bgpPeers.peer-asn

      integer

      Required value

      The AS number on the external BGP router.

      Allowed values: 0 <= X <= 4294967295

    • settings.bgpPeers.peer-port

      integer

      Port to dial when establishing the session.

      Default: 179

      Allowed values: 0 <= X <= 16384

    • settings.bgpPeers.router-id

      string

      BGP router ID to advertise to the peer.

    • settings.bgpPeers.source-address

      string

      The source IP address for outbound connections.

      Pattern: ^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$

  • settings.loadBalancerClass

    string

    An optional field describing the LoadBalancer class. The LoadBalancerClass field should be used in L2 LoadBalancer mode to specify the MetalLoadBalancerClass that defines the balancer parameters for the Service.

  • settings.nodeSelector

    object

    A selector for the main controller. It is the same as the Pod’s spec.nodeSelector parameter in Kubernetes.

    If the parameter is omitted or false, it will be determined automatically.

  • settings.speaker

    object

    Parameters of the speaker component, which announces services (using bgp or layer2 (LVS) routing protocol) and routes application traffic to its node.

    • settings.speaker.nodeSelector

      object

      Required value

      A selector for the speaker DaemonSet. It is the same as the Pod’s spec.nodeSelector parameter in Kubernetes.

      If the parameter is omitted or false, it will be determined automatically.

    • settings.speaker.tolerations

      array of objects

      Tolerations for the speaker DaemonSet. They are the same as the Pod’s spec.tolerations parameter in Kubernetes.

      If the parameter is omitted or false, it will be determined automatically.

      • settings.speaker.tolerations.effect

        string
      • settings.speaker.tolerations.key

        string
      • settings.speaker.tolerations.operator

        string
      • settings.speaker.tolerations.tolerationSeconds

        integer
      • settings.speaker.tolerations.value

        string
  • settings.tolerations

    array of objects

    Tolerations for the main controller. They are the same as the Pod’s spec.tolerations parameter in Kubernetes.

    If the parameter is omitted or false, it will be determined automatically.

    • settings.tolerations.effect

      string
    • settings.tolerations.key

      string
    • settings.tolerations.operator

      string
    • settings.tolerations.tolerationSeconds

      integer
    • settings.tolerations.value

      string

Schema version: 1

• settings

  object
  • settings.ebpfExporterEnabled

    boolean

    Set to ‘false’ to disable ebpf_exporter in case it behaves incorrectly.

    Default: true

    Example:

    ebpfExporterEnabled: false
    

  • settings.highAvailability

    boolean

    Manually enable the high availability mode.

    By default, Deckhouse automatically decides whether to enable the HA mode. Click here to learn more about the HA mode for modules.

    Examples:

    highAvailability: true
    

    highAvailability: false
    

  • settings.nodeSelector

    object

    The same as in the pods’ spec.nodeSelector parameter in Kubernetes.

    If the parameter is omitted or false, it will be determined automatically.

    Example:

    nodeSelector:
      disktype: ssd
    

  • settings.tolerations

    array of objects

    The same as in the pods’ spec.tolerations parameter in Kubernetes;

    If the parameter is omitted or false, it will be determined automatically.

    Example:

    tolerations:
    - key: key1
      operator: Equal
      value: value1
      effect: NoSchedule
    

    • settings.tolerations.effect

      string
    • settings.tolerations.key

      string
    • settings.tolerations.operator

      string
    • settings.tolerations.tolerationSeconds

      integer
    • settings.tolerations.value

      string
  • settings.vpa

    object

    VPA settings for kube-state-metrics pods.

    Example:

    vpa:
      kubeStateMetricsMaxCPU: 130m
      kubeStateMetricsMaxMemory: 115Mi
    

    • settings.vpa.kubeStateMetricsMaxCPU

      The maximum value that the VPA can set for the CPU requests.

      Examples:

      kubeStateMetricsMaxCPU: 3
      

      kubeStateMetricsMaxCPU: 1000m
      

    • settings.vpa.kubeStateMetricsMaxMemory

      The maximum memory requests the VPA can set.

      Example:

      kubeStateMetricsMaxMemory: 3Mi
      

Schema version: 1

• settings

  object
  • settings.externalTargets

    array of objects

    A list of additional hosts to monitor.

    Example:

    - name: google-primary
      host: 8.8.8.8
    - name: yaru
      host: ya.ru
    - host: youtube.com
    

    • settings.externalTargets.host

      string

      Required value

      IP address or host name to monitor.

    • settings.externalTargets.name

      string

      Monitoring record name.

Schema version: 1

• settings

  object
  • settings.allowNamespacesWithoutProjects

    boolean

    If this option is disabled - prohibit to create a namespace which is not a part of a Project.

    Default: true

  • settings.highAvailability

    boolean

    Manually enable the high availability mode.

    By default, Deckhouse automatically decides whether to enable the HA mode. Click here to learn more about the HA mode for modules.

    Example:

    highAvailability: true
    

Schema version: 1

• settings

  object
  • settings.nodeSelector

    object

    The same as in the pods’ spec.nodeSelector parameter in Kubernetes.

    If the parameter is omitted or false, it will be determined automatically.

  • settings.tolerations

    array of objects

    The same as in the pods’ spec.tolerations parameter in Kubernetes;

    If the parameter is omitted or false, it will be determined automatically.

    • settings.tolerations.effect

      string
    • settings.tolerations.key

      string
    • settings.tolerations.operator

      string
    • settings.tolerations.tolerationSeconds

      integer
    • settings.tolerations.value

      string

Schema version: 2

• settings

  object
  • settings.auth

    object

    Options related to authentication or authorization in the application.

    • settings.auth.allowedUserGroups

      array of strings

      An array of user groups that can access Grafana & Prometheus.

      This parameter is used if the user-authn module is enabled or the externalAuthentication parameter is set.

      Caution! Note that you must add those groups to the appropriate field in the DexProvider config if this module is used together with the user-authn one.

    • settings.auth.externalAuthentication

      object

      Parameters to enable external authentication based on the NGINX Ingress external-auth mechanism that uses the Nginx auth_request module.

      External authentication is enabled automatically if the user-authn module is enabled.

      • settings.auth.externalAuthentication.authSignInURL

        string

        The URL to redirect the user for authentication (if the authentication service returned a non-200 HTTP response code).

        Example:

        authSignInURL: https://example.com/dex/sign_in
        

      • settings.auth.externalAuthentication.authURL

        string

        The URL of the authentication service. If the user is authenticated, the service should return an HTTP 200 response code.

        Example:

        authURL: https://example.com/dex/auth
        

    • settings.auth.satisfyAny

      boolean

      Enables single authentication.

      If used together with the whitelistSourceRanges parameter, it authorizes all the users from above networks (no need to enter a username and password).

      Default: false

      Example:

      satisfyAny: true
      

    • settings.auth.whitelistSourceRanges

      array of strings

      An array if CIDRs that are allowed to authenticate in Grafana & Prometheus.

      Example:

      whitelistSourceRanges:
      - 1.1.1.1/32
      

  • settings.externalLabels

    object

    The set of external labels to add to the metrics.

    It’s possible to expand the environment variables of the config-reloader container in external labels such as:

    • HOSTNAME/POD_NAME - contains the name of the pod (for example prometheus-main-0, prometheus-main-1, etc.).
    • SHARD - contains the shard number.

    Example:

    externalLabels:
      prometheus_replica: "$(POD_NAME)"
      shard: "$(SHARD)"
      hostname: "$(HOSTNAME)"
    

  • settings.grafana

    object

    Grafana installation-related settings.

    • settings.grafana.customPlugins

      array of strings

      A list of custom Grafana plugins. Contains plugin names from the official repository.

      Here is how you can add custom plugins (in this case, clickhouse-datasource and flowcharting-panel plugins are used):

      grafana:
        customPlugins:
        - agenty-flowcharting-panel
        - vertamedia-clickhouse-datasource
      

      You can also install plugins from other sources by passing a link to the plugin zip archive in the format <url to plugin zip>;<plugin name>:

      grafana:
        customPlugins:
        - http://10.241.32.16:3000/netsage-bumpchart-panel-1.1.1.zip;netsage-bumpchart-panel
      

      Example:

      customPlugins:
      - agenty-flowcharting-panel
      - vertamedia-clickhouse-datasource
      

    • settings.grafana.useDarkTheme

      boolean

      The dark theme is enabled by default.

      Default: false

      Example:

      useDarkTheme: true
      

  • settings.highAvailability

    boolean

    Manually enable the high availability mode.

    By default, Deckhouse automatically decides whether to enable the HA mode. Click here to learn more about the HA mode for modules.

    Example:

    highAvailability: true
    

  • settings.https

    object

    What certificate type to use with Grafana/Prometheus.

    This parameter completely overrides the global.modules.https settings.

    Examples:

    https:
      mode: CustomCertificate
      customCertificate:
        secretName: foobar
    

    https:
      mode: CertManager
      certManager:
        clusterIssuerName: letsencrypt
    

    • settings.https.certManager

      object
      • settings.https.certManager.clusterIssuerName

        string

        What ClusterIssuer to use for Grafana/Prometheus.

        Currently, letsencrypt, letsencrypt-staging, selfsigned are available. Also, you can define your own.

        Default: "letsencrypt"

    • settings.https.customCertificate

      object
      • settings.https.customCertificate.secretName

        string

        The name of the secret in the d8-system namespace to use with Grafana/Prometheus.

        This secret must have the kubernetes.io/tls format.

        Default: "false"

    • settings.https.mode

      string

      The HTTPS usage mode:

      • Disabled — Grafana/Prometheus will work over HTTP only;
      • CertManager — Grafana/Prometheus will use HTTPS and get a certificate from the clusterissuer defined in the certManager.clusterIssuerName parameter.
      • CustomCertificate — Grafana/Prometheus will use HTTPS using the certificate from the d8-system namespace.
      • OnlyInURI — Grafana/Prometheus will work over HTTP (thinking that there is an external HTTPS load balancer in front that terminates HTTPS traffic). All the links in the user-authn will be generated using the HTTPS scheme. Load balancer should provide a redirect from HTTP to HTTPS.

      Default: "Disabled"

      Allowed values: Disabled, CertManager, CustomCertificate, OnlyInURI

  • settings.ingressClass

    string

    The class of the Ingress controller used for Grafana/Prometheus.

    An optional parameter. By default, the modules.ingressClass global value is used.

    Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$

    Example:

    ingressClass: public
    

  • settings.longtermMaxDiskSizeGigabytes

    Deprecated

    integer

    Deprecated and will be removed. Doesn’t affect anything. The maximum size (in GiB) to which the Longterm Prometheus’ disk can be automatically resized.

  • settings.longtermNodeSelector

    object

    The same as in the Pods’ spec.nodeSelector parameter in Kubernetes.

    If the parameter is omitted or false, it will be determined automatically.

    Example:

    longtermNodeSelector:
      disktype: ssd
    

  • settings.longtermPodAntiAffinity

    string

    Defines the podAntiAffinity configuration for the Prometheus longterm instance relative to the Prometheus main instance.

    • Preferred — allows scheduling Prometheus longterm instance alongside the Prometheus main instance if it is not possible to place them on different nodes.
    • Required — does not allow scheduling Prometheus longterm instance on the same node as the Prometheus main instance.

    Default: "Preferred"

    Allowed values: Preferred, Required

  • settings.longtermRetentionDays

    integer

    How long to keep the data in longterm Prometheus.

    Setting this parameter to 0 will result in Longterm Prometheus not running in the cluster.

    Default: 1095

  • settings.longtermScrapeInterval

    string

    Sets the interval for making “data snapshots” of the main Prometheus by the longterm Prometheus.

    Default: "5m"

  • settings.longtermStorageClass

    string

    The name of the StorageClass to use for Longterm Prometheus.

    If omitted, the StorageClass of the existing Longterm Prometheus PVC is used. If there is no PVC yet, the StorageClass will be used according to the global storageClass parameter setting.

    The global storageClass parameter is only considered when the module is enabled. Changing the global storageClass parameter while the module is enabled will not trigger disk re-provisioning.

    Warning. Specifying a value different from the one currently used (in the existing PVC) will result in disk re-provisioning and all data will be deleted.

    Warning. When migrating Prometheus with local storage to other nodes, the pod will hang in the Pending state. In this case, it will be necessary to save the Prometheus database, delete the old PVC and restart the pod manually. Local storage refers to a StorageClass that is associated not with network storage, but with a local volume on a node (for example, StorageClass created by the local-path-provider module).

    If false is specified, emptyDir will be forced to be used.

    Example:

    longtermStorageClass: ceph-ssd
    

  • settings.longtermTolerations

    array of objects

    The same as in the Pods’ spec.tolerations parameter in Kubernetes;

    If the parameter is omitted or false, it will be determined automatically.

    Example:

    longtermTolerations:
    - key: key1
      operator: Equal
      value: value1
      effect: NoSchedule
    

    • settings.longtermTolerations.effect

      string
    • settings.longtermTolerations.key

      string
    • settings.longtermTolerations.operator

      string
    • settings.longtermTolerations.tolerationSeconds

      integer
    • settings.longtermTolerations.value

      string
  • settings.mainMaxDiskSizeGigabytes

    Deprecated

    integer

    Deprecated and will be removed. Doesn’t affect anything. The maximum size (in GiB) that the main Prometheus’ volume can automatically resize to.

  • settings.nodeSelector

    object

    The same as in the Pods’ spec.nodeSelector parameter in Kubernetes.

    If the parameter is omitted or false, it will be determined automatically.

    Example:

    nodeSelector:
      disktype: ssd
    

  • settings.retentionDays

    integer

    How long to keep the data.

    Default: 15

  • settings.scrapeInterval

    string

    Sets the interval for scraping metrics from targets.

    Evaluation Interval is always equal to scrapeInterval.

    Default: "30s"

    Pattern: ^([\d]*y)?([\d]*w)?([\d]*d)?([\d]*h)?([\d]*m)?([\d]*s)?$

  • settings.storageClass

    string

    The name of the StorageClass to use for Longterm Prometheus.

    If omitted, the StorageClass of the existing Prometheus PVC is used. If there is no PVC yet, the StorageClass will be used according to the global storageClass parameter setting.

    The global storageClass parameter is only considered when the module is enabled. Changing the global storageClass parameter while the module is enabled will not trigger disk re-provisioning.

    Warning. Specifying a value different from the one currently used (in the existing PVC) will result in disk re-provisioning and all data will be deleted.

    If false is specified, emptyDir will be forced to be used.

    Examples:

    storageClass: ceph-ssd
    

    storageClass: 'false'
    

  • settings.tolerations

    array of objects

    The same as in the Pods’ spec.tolerations parameter in Kubernetes;

    If the parameter is omitted or false, it will be determined automatically.

    Example:

    tolerations:
    - key: key1
      operator: Equal
      value: value1
      effect: NoSchedule
    

    • settings.tolerations.effect

      string
    • settings.tolerations.key

      string
    • settings.tolerations.operator

      string
    • settings.tolerations.tolerationSeconds

      integer
    • settings.tolerations.value

      string
  • settings.vpa

    object

    VPA settings for pods.

    Default: {"updateMode":"Initial"}

    Examples:

    vpa:
      updateMode: Initial
      longtermMaxCPU: '1'
      longtermMaxMemory: 1500Mi
      maxCPU: 1000m
      maxMemory: 1500Mi
    

    vpa:
      updateMode: 'Off'
    

    • settings.vpa.longtermMaxCPU

      The maximum value that the VPA can set for the Longterm Prometheus Pods.

      The default value is chosen automatically based on the maximum number of Pods that can be created in the cluster considering the current number of nodes and their settings. For more information, see the detect_vpa_max hook of the module.

      Example:

      longtermMaxCPU: 0.1
      

    • settings.vpa.longtermMaxMemory

      The maximum memory requests the VPA can set for the longterm Prometheus Pods.

      The default value is chosen automatically based on the maximum number of Pods that can be created in the cluster considering the current number of nodes and their settings. For more information, see the detect_vpa_max hook of the module.

      Example:

      longtermMaxMemory: 4Mi
      

    • settings.vpa.maxCPU

      The maximum value that the VPA can set for the CPU requests for the main Prometheus Pods.

      The default value is chosen automatically based on the maximum number of Pods that can be created in the cluster considering the current number of nodes and their settings. For more information, see the detect_vpa_max hook of the module.

      Example:

      maxCPU: '3'
      

    • settings.vpa.maxMemory

      The maximum memory requests the VPA can set for the main Prometheus Pods.

      The default value is chosen automatically based on the maximum number of Pods that can be created in the cluster considering the current number of nodes and their settings. For more information, see the detect_vpa_max hook of the module.

      Example:

      maxMemory: 3Mi
      

    • settings.vpa.updateMode

      string

      The VPA usage mode.

      Default: "Initial"

      Allowed values: Initial, Auto, Off