Deckhouse Platform in existing cluster

Select the Deckhouse Platform revision

The recommended settings for a Deckhouse Platform Community Edition installation are generated below:

  • config.yml — a file with the configuration needed to bootstrap the cluster. Contains the installer parameters, access parameters, and the initial cluster parameters.

Please pay attention to:

  • highlighted parameters you must define.
  • parameters you might want to change.

We recommend that you read the section If something went wrong first; perhaps, your ISP case is already addressed there. Refer to it if you have any problems during the installation process.

To learn more about the Deckhouse Platform release channels, please see the relevant documentation.

# section for bootstrapping the Deckhouse cluster (InitConfiguration)
# version of the Deckhouse API
apiVersion: deckhouse.io/v1
# type of the configuration section
kind: InitConfiguration
# Deckhouse parameters
deckhouse:
  # the release channel in use
  releaseChannel: Stable
  # the Minimal bundle is used when installing Deckhouse in an existing cluster
  bundle: Minimal
  configOverrides:
    global:
      modules:
        # template that will be used for system apps domains within the cluster
        # e.g., Grafana for %s.example.com will be available as grafana.example.com
        publicDomainTemplate: "%s.example.com"
        # you might consider changing this
        placement:
          customTolerationKeys:
          - SystemLoad
    certManagerEnabled: true
    deckhouseWebEnabled: true
# section for bootstrapping the Deckhouse cluster (InitConfiguration) # version of the Deckhouse API apiVersion: deckhouse.io/v1 # type of the configuration section kind: InitConfiguration # Deckhouse parameters deckhouse: # the release channel in use releaseChannel: Stable # the Minimal bundle is used when installing Deckhouse in an existing cluster bundle: Minimal configOverrides: global: modules: # template that will be used for system apps domains within the cluster # e.g., Grafana for %s.example.com will be available as grafana.example.com publicDomainTemplate: "%s.example.com" # you might consider changing this placement: customTolerationKeys: - SystemLoad certManagerEnabled: true deckhouseWebEnabled: true

Про nodeSelector, taints и tolerations...

You can control on which nodes the Deckhouse core components will run by setting nodeSelector/tolerations in the configOverrides.deckhouse parameter of the installation configuration. You can also specify cluster node taints in the configOverrides.global.modules.placement.customTolerationKeys array so that Deckhouse can automatically add the appropriate toleration to its components.

Below is an example of setting nodeSelector/tolerations for the deckhouse module and specifying tolerations for other Deckhouse components in the customTolerationKeys array (do not copy this example without adapting it to your configuration as the values will be different):

deckhouse:
  configOverrides:
    # ...
    deckhouse:
      tolerations:
        - key: dedicated.deckhouse.io
          operator: Exists
    # ...
    global:
      modules:
        # ...
        placement:
          customTolerationKeys:
          - SystemLoad
          - app.kubernetes.io/instance

Enter license key

Enter

The recommended settings for a Deckhouse Platform Enterprise Edition installation are generated below:

  • config.yml — a file with the configuration needed to bootstrap the cluster. Contains the installer parameters, access parameters, and the initial cluster parameters.

Please pay attention to:

  • highlighted parameters you must define.
  • parameters you might want to change.

We recommend that you read the section If something went wrong first; perhaps, your ISP case is already addressed there. Refer to it if you have any problems during the installation process.

To learn more about the Deckhouse Platform release channels, please see the relevant documentation.

# section for bootstrapping the Deckhouse cluster (InitConfiguration)
# version of the Deckhouse API
apiVersion: deckhouse.io/v1
# type of the configuration section
kind: InitConfiguration
# Deckhouse parameters
deckhouse:
  # address of the Docker registry where the Deckhouse images are located
  imagesRepo: registry.deckhouse.io/deckhouse/ee
  # a special string with your token to access Docker registry (generated automatically for your license token)
  registryDockerCfg: <YOUR_ACCESS_STRING_IS_HERE>
  # the release channel in use
  releaseChannel: Stable
  # the Minimal bundle is used when installing Deckhouse in an existing cluster
  bundle: Minimal
  configOverrides:
    global:
      modules:
        # template that will be used for system apps domains within the cluster
        # e.g., Grafana for %s.example.com will be available as grafana.example.com
        publicDomainTemplate: "%s.example.com"
        # you might consider changing this
        placement:
          customTolerationKeys:
          - SystemLoad
    certManagerEnabled: true
    deckhouseWebEnabled: true
# section for bootstrapping the Deckhouse cluster (InitConfiguration) # version of the Deckhouse API apiVersion: deckhouse.io/v1 # type of the configuration section kind: InitConfiguration # Deckhouse parameters deckhouse: # address of the Docker registry where the Deckhouse images are located imagesRepo: registry.deckhouse.io/deckhouse/ee # a special string with your token to access Docker registry (generated automatically for your license token) registryDockerCfg: <YOUR_ACCESS_STRING_IS_HERE> # the release channel in use releaseChannel: Stable # the Minimal bundle is used when installing Deckhouse in an existing cluster bundle: Minimal configOverrides: global: modules: # template that will be used for system apps domains within the cluster # e.g., Grafana for %s.example.com will be available as grafana.example.com publicDomainTemplate: "%s.example.com" # you might consider changing this placement: customTolerationKeys: - SystemLoad certManagerEnabled: true deckhouseWebEnabled: true

Про nodeSelector, taints и tolerations...

You can control on which nodes the Deckhouse core components will run by setting nodeSelector/tolerations in the configOverrides.deckhouse parameter of the installation configuration. You can also specify cluster node taints in the configOverrides.global.modules.placement.customTolerationKeys array so that Deckhouse can automatically add the appropriate toleration to its components.

Below is an example of setting nodeSelector/tolerations for the deckhouse module and specifying tolerations for other Deckhouse components in the customTolerationKeys array (do not copy this example without adapting it to your configuration as the values will be different):

deckhouse:
  configOverrides:
    # ...
    deckhouse:
      tolerations:
        - key: dedicated.deckhouse.io
          operator: Exists
    # ...
    global:
      modules:
        # ...
        placement:
          customTolerationKeys:
          - SystemLoad
          - app.kubernetes.io/instance

Use a Docker image to install the Deckhouse Platform. It is necessary to transfer configuration files to the container.

Run the installer on the personal computer.

Linux / macOS Windows

docker run --pull=always -it -v "$PWD/config.yml:/config.yml" \
  -v "$HOME/.kube/config:/kubeconfig" registry.deckhouse.io/deckhouse/ce/install:stable bash
docker run --pull=always -it -v "$PWD/config.yml:/config.yml" \ -v "$HOME/.kube/config:/kubeconfig" registry.deckhouse.io/deckhouse/ce/install:stable bash
docker run --pull=always -it -v "%cd%\config.yml:/config.yml" -v "%userprofile%\.kube\config:/kubeconfig"  registry.deckhouse.io/deckhouse/ce/install:stable bash -c "chmod 400 /tmp/.ssh/id_rsa; bash"
docker run --pull=always -it -v "%cd%\config.yml:/config.yml" -v "%userprofile%\.kube\config:/kubeconfig" registry.deckhouse.io/deckhouse/ce/install:stable bash -c "chmod 400 /tmp/.ssh/id_rsa; bash"

Notes:

  • Kubectl configuration file with access to Kubernetes API must be mount as the /kubeconfig file in the container.

Now, to initiate the process of installation, you need to execute inside the container:

dhctl bootstrap-phase install-deckhouse --kubeconfig=/kubeconfig --config=/config.yml
dhctl bootstrap-phase install-deckhouse --kubeconfig=/kubeconfig --config=/config.yml

After the installation is complete, you will be returned to the command line.

Almost everything is ready for a fully-fledged Deckhouse Platform to work!

If something went wrong

Some providers’ clusters may require extra steps before or after installing Deckhouse.

Here are some common problems and ways to solve them. Should you run into difficulties installing Deckhouse in an existing cluster, please, share them by creating an issue.

Installation errors at the 'Waiting for Deckhouse to become Ready' step

  • Error of the following kind:
    │ │ ┌ Waiting for Deckhouse to become Ready
    │ │ │ Deckhouse pod found: deckhouse-64649df6f9-mf6dt (Pending)
    │ │ │ Deckhouse pod found: deckhouse-64649df6f9-mf6dt (Pending)
    

    Probably, there is no node in the cluster with the node-role.kubernetes.io/control-plane: "" label which is originally used in the nodeSelector of the deckhouse deployment manifest.

    Ways to fix the error:

    • Insert the proper nodeSelector into the deckhouse deployment:
      kubectl -n d8-system edit deployment/deckhouse
      
    • Delete nodeSelector in the deckhouse deployment:
      kubectl patch -n d8-system deployment deckhouse --type json -p '[{"op": "remove", "path": "/spec/template/spec/nodeSelector"}]'
      

There is no Ingress controller in the cluster...

The deckhouse-web module, which provides a web interface to the cluster documentation, requires the Ingress controller to be active in the cluster. If there is no Ingress controller in your cluster, you can use the built-in ingress-nginx module:

  • Enable the module using one the following options:
    • At the Deckhouse installation step, add the following to the deckhouse.configOverrides section of the config.yml file:
      ingressNginxEnabled: true
      
      ingressNginxEnabled: true

      Example:

      deckhouse:
        configOverrides:
          ingressNginxEnabled: true
      
    • Once Deckhouse is installed, edit the deckhouse ConfigMap:

      kubectl -n d8-system edit cm deckhouse
      
      kubectl -n d8-system edit cm deckhouse

      ... and add the following to it:

      ingressNginxEnabled: "true"
      
      ingressNginxEnabled: "true"

      Example:

      deckhouse:
        ingressNginxEnabled: "true"
      
  • Create an ingress-nginx-controller.yml file with the following contents:

    ingress-nginx-controller.ymlCopy filenameCopy content
    # section containing the parameters of nginx ingress controller
    # version of the Deckhouse API
    apiVersion: deckhouse.io/v1
    kind: IngressNginxController
    metadata:
      name: nginx
    spec:
      # the name of the Ingress class to use with the Ingress nginx controller
      ingressClass: nginx
      # Ingress version to use (use version 1.1 with Kubernetes 1.23+)
      controllerVersion: "1.1"
      # the way traffic goes to cluster from the outer network
      inlet: HostPort
      hostPort:
        httpPort: 80
        httpsPort: 443
      # describes on which nodes the component will be located
      # you might consider changing this
      nodeSelector:
        node-role.kubernetes.io/control-plane: ""
      tolerations:
      - operator: Exists
    
    # section containing the parameters of nginx ingress controller # version of the Deckhouse API apiVersion: deckhouse.io/v1 kind: IngressNginxController metadata: name: nginx spec: # the name of the Ingress class to use with the Ingress nginx controller ingressClass: nginx # Ingress version to use (use version 1.1 with Kubernetes 1.23+) controllerVersion: "1.1" # the way traffic goes to cluster from the outer network inlet: HostPort hostPort: httpPort: 80 httpsPort: 443 # describes on which nodes the component will be located # you might consider changing this nodeSelector: node-role.kubernetes.io/control-plane: "" tolerations: - operator: Exists
  • Once Deckhouse is installed, apply the file using the command below:

    kubectl create -f ingress-nginx-controller.yml
    
    kubectl create -f ingress-nginx-controller.yml

Cluster in EKS AWS (Amazon Elastic Kubernetes Service)

If you are installing Deckhouse to an EKS AWS (Amazon Elastic Kubernetes Service) cluster, install aws-cli in the running installer container using the following command:

apk add python3 py3-pip && pip3 install --upgrade pip && pip3 install awscli
apk add python3 py3-pip && pip3 install --upgrade pip && pip3 install awscli

Cluster in VK Cloud Solutions (MailRu Cloud Solutions)

  • Add a CriticalAddonsOnly taint to customTolerationKeys in the Deckhouse installation configuration.

    Example:

    deckhouse:
      releaseChannel: Stable
      bundle: Minimal
      configOverrides:
        global:
          modules:
            placement:
              customTolerationKeys:
              - CriticalAddonsOnly
            publicDomainTemplate: "%s.example.com"
    
  • VK Cloud Solutions version 1.21+ clusters have a Gatekeeper (OPA) that requires setting Pod requests and limits. However, the deckhouse Pod has no requests/limits, while for all other Deckhouse components and modules, requests/limits are calculated by Deckhouse.

    As a result, the following error may pop up in deckhouse deckhouse Deployment events:

    admission webhook "validation.gatekeeper.sh" denied the request: [container-must-have-limits] container <...> has no resource limits...

    For Deckhouse to work, add a GateKeeper exception (OPA) for the Deckhouse component namespaces (d8*-) before installing Deckhouse in clusters of this type.

    For this, run the following command in the cluster:

    kubectl patch constraints container-must-have-limits --type json -p '[{"op": "replace", "path": "/spec/match/excludedNamespaces", "value": ["d8-*"]}]'
    
    kubectl patch constraints container-must-have-limits --type json -p '[{"op": "replace", "path": "/spec/match/excludedNamespaces", "value": ["d8-*"]}]'

Use a Docker image to install the Deckhouse Platform. It is necessary to transfer configuration files to the container.

Run the installer on the personal computer.

Linux / macOS Windows

 echo <LICENSE_TOKEN> | docker login -u license-token --password-stdin registry.deckhouse.io
docker run --pull=always -it -v "$PWD/config.yml:/config.yml" \
  -v "$HOME/.kube/config:/kubeconfig" registry.deckhouse.io/deckhouse/ee/install:stable bash
echo <LICENSE_TOKEN> | docker login -u license-token --password-stdin registry.deckhouse.io docker run --pull=always -it -v "$PWD/config.yml:/config.yml" \ -v "$HOME/.kube/config:/kubeconfig" registry.deckhouse.io/deckhouse/ee/install:stable bash

Log in on the personal computer to the container image registry by providing the license key as a password:

docker login -u license-token registry.deckhouse.io
docker login -u license-token registry.deckhouse.io

Run a container with the installer:

docker run --pull=always -it -v "%cd%\config.yml:/config.yml" -v "%userprofile%\.kube\config:/kubeconfig" registry.deckhouse.io/deckhouse/ee/install:stable bash -c "chmod 400 /tmp/.ssh/id_rsa; bash"
docker run --pull=always -it -v "%cd%\config.yml:/config.yml" -v "%userprofile%\.kube\config:/kubeconfig" registry.deckhouse.io/deckhouse/ee/install:stable bash -c "chmod 400 /tmp/.ssh/id_rsa; bash"

Notes:

  • Kubectl configuration file with access to Kubernetes API must be mount as the /kubeconfig file in the container.

Now, to initiate the process of installation, you need to execute inside the container:

dhctl bootstrap-phase install-deckhouse --kubeconfig=/kubeconfig --config=/config.yml
dhctl bootstrap-phase install-deckhouse --kubeconfig=/kubeconfig --config=/config.yml

After the installation is complete, you will be returned to the command line.

Almost everything is ready for a fully-fledged Deckhouse Platform to work!

If something went wrong

Some providers’ clusters may require extra steps before or after installing Deckhouse.

Here are some common problems and ways to solve them. Should you run into difficulties installing Deckhouse in an existing cluster, please, share them by creating an issue.

Installation errors at the 'Waiting for Deckhouse to become Ready' step

  • Error of the following kind:
    │ │ ┌ Waiting for Deckhouse to become Ready
    │ │ │ Deckhouse pod found: deckhouse-64649df6f9-mf6dt (Pending)
    │ │ │ Deckhouse pod found: deckhouse-64649df6f9-mf6dt (Pending)
    

    Probably, there is no node in the cluster with the node-role.kubernetes.io/control-plane: "" label which is originally used in the nodeSelector of the deckhouse deployment manifest.

    Ways to fix the error:

    • Insert the proper nodeSelector into the deckhouse deployment:
      kubectl -n d8-system edit deployment/deckhouse
      
    • Delete nodeSelector in the deckhouse deployment:
      kubectl patch -n d8-system deployment deckhouse --type json -p '[{"op": "remove", "path": "/spec/template/spec/nodeSelector"}]'
      

There is no Ingress controller in the cluster...

The deckhouse-web module, which provides a web interface to the cluster documentation, requires the Ingress controller to be active in the cluster. If there is no Ingress controller in your cluster, you can use the built-in ingress-nginx module:

  • Enable the module using one the following options:
    • At the Deckhouse installation step, add the following to the deckhouse.configOverrides section of the config.yml file:
      ingressNginxEnabled: true
      
      ingressNginxEnabled: true

      Example:

      deckhouse:
        configOverrides:
          ingressNginxEnabled: true
      
    • Once Deckhouse is installed, edit the deckhouse ConfigMap:

      kubectl -n d8-system edit cm deckhouse
      
      kubectl -n d8-system edit cm deckhouse

      ... and add the following to it:

      ingressNginxEnabled: "true"
      
      ingressNginxEnabled: "true"

      Example:

      deckhouse:
        ingressNginxEnabled: "true"
      
  • Create an ingress-nginx-controller.yml file with the following contents:

    ingress-nginx-controller.ymlCopy filenameCopy content
    # section containing the parameters of nginx ingress controller
    # version of the Deckhouse API
    apiVersion: deckhouse.io/v1
    kind: IngressNginxController
    metadata:
      name: nginx
    spec:
      # the name of the Ingress class to use with the Ingress nginx controller
      ingressClass: nginx
      # Ingress version to use (use version 1.1 with Kubernetes 1.23+)
      controllerVersion: "1.1"
      # the way traffic goes to cluster from the outer network
      inlet: HostPort
      hostPort:
        httpPort: 80
        httpsPort: 443
      # describes on which nodes the component will be located
      # you might consider changing this
      nodeSelector:
        node-role.kubernetes.io/control-plane: ""
      tolerations:
      - operator: Exists
    
    # section containing the parameters of nginx ingress controller # version of the Deckhouse API apiVersion: deckhouse.io/v1 kind: IngressNginxController metadata: name: nginx spec: # the name of the Ingress class to use with the Ingress nginx controller ingressClass: nginx # Ingress version to use (use version 1.1 with Kubernetes 1.23+) controllerVersion: "1.1" # the way traffic goes to cluster from the outer network inlet: HostPort hostPort: httpPort: 80 httpsPort: 443 # describes on which nodes the component will be located # you might consider changing this nodeSelector: node-role.kubernetes.io/control-plane: "" tolerations: - operator: Exists
  • Once Deckhouse is installed, apply the file using the command below:

    kubectl create -f ingress-nginx-controller.yml
    
    kubectl create -f ingress-nginx-controller.yml

Cluster in EKS AWS (Amazon Elastic Kubernetes Service)

If you are installing Deckhouse to an EKS AWS (Amazon Elastic Kubernetes Service) cluster, install aws-cli in the running installer container using the following command:

apk add python3 py3-pip && pip3 install --upgrade pip && pip3 install awscli
apk add python3 py3-pip && pip3 install --upgrade pip && pip3 install awscli

Cluster in VK Cloud Solutions (MailRu Cloud Solutions)

  • Add a CriticalAddonsOnly taint to customTolerationKeys in the Deckhouse installation configuration.

    Example:

    deckhouse:
      releaseChannel: Stable
      bundle: Minimal
      configOverrides:
        global:
          modules:
            placement:
              customTolerationKeys:
              - CriticalAddonsOnly
            publicDomainTemplate: "%s.example.com"
    
  • VK Cloud Solutions version 1.21+ clusters have a Gatekeeper (OPA) that requires setting Pod requests and limits. However, the deckhouse Pod has no requests/limits, while for all other Deckhouse components and modules, requests/limits are calculated by Deckhouse.

    As a result, the following error may pop up in deckhouse deckhouse Deployment events:

    admission webhook "validation.gatekeeper.sh" denied the request: [container-must-have-limits] container <...> has no resource limits...

    For Deckhouse to work, add a GateKeeper exception (OPA) for the Deckhouse component namespaces (d8*-) before installing Deckhouse in clusters of this type.

    For this, run the following command in the cluster:

    kubectl patch constraints container-must-have-limits --type json -p '[{"op": "replace", "path": "/spec/match/excludedNamespaces", "value": ["d8-*"]}]'
    
    kubectl patch constraints container-must-have-limits --type json -p '[{"op": "replace", "path": "/spec/match/excludedNamespaces", "value": ["d8-*"]}]'