Deckhouse Kubernetes Platform in existing cluster

Select the Deckhouse Kubernetes Platform revision

The recommended settings for a Deckhouse Kubernetes Platform Community Edition installation are generated below:

  • config.yml — a file with the configuration needed to bootstrap the cluster. Contains the installer parameters, access parameters, and the initial cluster parameters.

Please pay attention to:

  • highlighted parameters you must define.
  • parameters you might want to change.
  • We recommend that you read the section If something went wrong first; perhaps, your ISP case is already addressed there. Refer to it if you have any problems during the installation process.

Create the config.yml file.

# Deckhouse module settings.
# https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/002-deckhouse/configuration.html
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: deckhouse
spec:
  version: 1
  enabled: true
  settings:
    bundle: Minimal
    # Deckhouse release channel. The Early Access channel is stable enough to be used in productive environments.
    # If you plan to use several clusters, it is recommended to use different release channels on them.
    # More info: https://deckhouse.io/products/kubernetes-platform/documentation/v1/deckhouse-release-channels.html
    releaseChannel: EarlyAccess
    logLevel: Info
---
# Global Deckhouse settings.
# https://deckhouse.ru/products/kubernetes-platform/documentation/v1/deckhouse-configure-global.html#parameters
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: global
spec:
  version: 1
  settings:
    modules:
      # Template that will be used for system apps domains within the cluster.
      # E.g., Grafana for %s.example.com will be available as 'grafana.example.com'.
      # The domain MUST NOT match the one specified in the clusterDomain parameter of the ClusterConfiguration resource.
      # You can change it to your own or follow the steps in the guide and change it after installation.
      publicDomainTemplate: "%s.example.com"
      # You might consider changing this.
      placement:
        customTolerationKeys:
        - SystemLoad
---
# cert-manager module settings.
# https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/101-cert-manager/configuration.html
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: cert-manager
spec:
  version: 1
  enabled: true
---
# documentation module settings.
# https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/810-documentation/configuration.html
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: documentation
spec:
  version: 1
  enabled: true
# Deckhouse module settings. # https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/002-deckhouse/configuration.html apiVersion: deckhouse.io/v1alpha1 kind: ModuleConfig metadata: name: deckhouse spec: version: 1 enabled: true settings: bundle: Minimal # Deckhouse release channel. The Early Access channel is stable enough to be used in productive environments. # If you plan to use several clusters, it is recommended to use different release channels on them. # More info: https://deckhouse.io/products/kubernetes-platform/documentation/v1/deckhouse-release-channels.html releaseChannel: EarlyAccess logLevel: Info --- # Global Deckhouse settings. # https://deckhouse.ru/products/kubernetes-platform/documentation/v1/deckhouse-configure-global.html#parameters apiVersion: deckhouse.io/v1alpha1 kind: ModuleConfig metadata: name: global spec: version: 1 settings: modules: # Template that will be used for system apps domains within the cluster. # E.g., Grafana for %s.example.com will be available as 'grafana.example.com'. # The domain MUST NOT match the one specified in the clusterDomain parameter of the ClusterConfiguration resource. # You can change it to your own or follow the steps in the guide and change it after installation. publicDomainTemplate: "%s.example.com" # You might consider changing this. placement: customTolerationKeys: - SystemLoad --- # cert-manager module settings. # https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/101-cert-manager/configuration.html apiVersion: deckhouse.io/v1alpha1 kind: ModuleConfig metadata: name: cert-manager spec: version: 1 enabled: true --- # documentation module settings. # https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/810-documentation/configuration.html apiVersion: deckhouse.io/v1alpha1 kind: ModuleConfig metadata: name: documentation spec: version: 1 enabled: true

About nodeSelector, taints and tolerations...

You can control on which nodes the Deckhouse core components will run by setting nodeSelector/tolerations in the configOverrides.deckhouse parameter of the installation configuration. You can also specify cluster node taints in the configOverrides.global.modules.placement.customTolerationKeys array so that Deckhouse can automatically add the appropriate toleration to its components.

Below is an example of setting nodeSelector/tolerations for the deckhouse module and specifying tolerations for other Deckhouse components in the customTolerationKeys array (do not copy this example without adapting it to your configuration as the values will be different):

deckhouse:
  configOverrides:
    # ...
    deckhouse:
      tolerations:
        - key: dedicated.deckhouse.io
          operator: Exists
    # ...
    global:
      modules:
        # ...
        placement:
          customTolerationKeys:
          - SystemLoad
          - app.kubernetes.io/instance

Enter license key

Enter

Have no key?

The recommended settings for a Deckhouse Kubernetes Platform Enterprise Edition installation are generated below:

  • config.yml — a file with the configuration needed to bootstrap the cluster. Contains the installer parameters, access parameters, and the initial cluster parameters.

Please pay attention to:

  • highlighted parameters you must define.
  • parameters you might want to change.
  • We recommend that you read the section If something went wrong first; perhaps, your ISP case is already addressed there. Refer to it if you have any problems during the installation process.

Create the config.yml file.

# Settings for the bootstrapping the Deckhouse cluster
# https://deckhouse.io/products/kubernetes-platform/documentation/v1/installing/configuration.html#initconfiguration
apiVersion: deckhouse.io/v1
kind: InitConfiguration
deckhouse:
  # Address of the Docker registry where the Deckhouse images are located.
  imagesRepo: registry.deckhouse.io/deckhouse/ee
  # A special string with your token to access Docker registry (generated automatically for your license token).
  registryDockerCfg: <YOUR_ACCESS_STRING_IS_HERE>
---
# Deckhouse module settings.
# https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/002-deckhouse/configuration.html
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: deckhouse
spec:
  version: 1
  enabled: true
  settings:
    bundle: Minimal
    releaseChannel: Stable
    logLevel: Info
---
# Global Deckhouse settings.
# https://deckhouse.ru/products/kubernetes-platform/documentation/v1/deckhouse-configure-global.html#parameters
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: global
spec:
  version: 1
  settings:
    modules:
      # Template that will be used for system apps domains within the cluster.
      # E.g., Grafana for %s.example.com will be available as 'grafana.example.com'.
      # The domain MUST NOT match the one specified in the clusterDomain parameter of the ClusterConfiguration resource.
      # You can change it to your own or follow the steps in the guide and change it after installation.
      publicDomainTemplate: "%s.example.com"
      # You might consider changing this.
      placement:
        customTolerationKeys:
        - SystemLoad
---
# cert-manager module settings.
# https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/101-cert-manager/configuration.html
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: cert-manager
spec:
  version: 1
  enabled: true
---
# documentation module settings.
# https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/810-documentation/configuration.html
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: documentation
spec:
  version: 1
  enabled: true
# Settings for the bootstrapping the Deckhouse cluster # https://deckhouse.io/products/kubernetes-platform/documentation/v1/installing/configuration.html#initconfiguration apiVersion: deckhouse.io/v1 kind: InitConfiguration deckhouse: # Address of the Docker registry where the Deckhouse images are located. imagesRepo: registry.deckhouse.io/deckhouse/ee # A special string with your token to access Docker registry (generated automatically for your license token). registryDockerCfg: <YOUR_ACCESS_STRING_IS_HERE> --- # Deckhouse module settings. # https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/002-deckhouse/configuration.html apiVersion: deckhouse.io/v1alpha1 kind: ModuleConfig metadata: name: deckhouse spec: version: 1 enabled: true settings: bundle: Minimal releaseChannel: Stable logLevel: Info --- # Global Deckhouse settings. # https://deckhouse.ru/products/kubernetes-platform/documentation/v1/deckhouse-configure-global.html#parameters apiVersion: deckhouse.io/v1alpha1 kind: ModuleConfig metadata: name: global spec: version: 1 settings: modules: # Template that will be used for system apps domains within the cluster. # E.g., Grafana for %s.example.com will be available as 'grafana.example.com'. # The domain MUST NOT match the one specified in the clusterDomain parameter of the ClusterConfiguration resource. # You can change it to your own or follow the steps in the guide and change it after installation. publicDomainTemplate: "%s.example.com" # You might consider changing this. placement: customTolerationKeys: - SystemLoad --- # cert-manager module settings. # https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/101-cert-manager/configuration.html apiVersion: deckhouse.io/v1alpha1 kind: ModuleConfig metadata: name: cert-manager spec: version: 1 enabled: true --- # documentation module settings. # https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/810-documentation/configuration.html apiVersion: deckhouse.io/v1alpha1 kind: ModuleConfig metadata: name: documentation spec: version: 1 enabled: true

About nodeSelector, taints and tolerations...

You can control on which nodes the Deckhouse core components will run by setting nodeSelector/tolerations in the configOverrides.deckhouse parameter of the installation configuration. You can also specify cluster node taints in the configOverrides.global.modules.placement.customTolerationKeys array so that Deckhouse can automatically add the appropriate toleration to its components.

Below is an example of setting nodeSelector/tolerations for the deckhouse module and specifying tolerations for other Deckhouse components in the customTolerationKeys array (do not copy this example without adapting it to your configuration as the values will be different):

deckhouse:
  configOverrides:
    # ...
    deckhouse:
      tolerations:
        - key: dedicated.deckhouse.io
          operator: Exists
    # ...
    global:
      modules:
        # ...
        placement:
          customTolerationKeys:
          - SystemLoad
          - app.kubernetes.io/instance

Use a Docker image to install the Deckhouse Kubernetes Platform. It is necessary to transfer configuration files to the container.

Run the installer on the personal computer.

Linux / macOS Windows

docker run --pull=always -it -v "$PWD/config.yml:/config.yml" \
  -v "$HOME/.kube/config:/kubeconfig" registry.deckhouse.io/deckhouse/ce/install:stable bash
docker run --pull=always -it -v "$PWD/config.yml:/config.yml" \ -v "$HOME/.kube/config:/kubeconfig" registry.deckhouse.io/deckhouse/ce/install:stable bash
docker run --pull=always -it -v "%cd%\config.yml:/config.yml" -v "%userprofile%\.kube\config:/kubeconfig"  registry.deckhouse.io/deckhouse/ce/install:stable bash -c "chmod 400 /tmp/.ssh/id_rsa; bash"
docker run --pull=always -it -v "%cd%\config.yml:/config.yml" -v "%userprofile%\.kube\config:/kubeconfig" registry.deckhouse.io/deckhouse/ce/install:stable bash -c "chmod 400 /tmp/.ssh/id_rsa; bash"

Notes:

  • Kubectl configuration file with access to Kubernetes API must be mount as the /kubeconfig file in the container.

Now, to initiate the process of installation, you need to execute inside the container:

dhctl bootstrap-phase install-deckhouse --kubeconfig=/kubeconfig --config=/config.yml
dhctl bootstrap-phase install-deckhouse --kubeconfig=/kubeconfig --config=/config.yml

The installation process may take about from 5 to 30 minutes, depending on the connection.

After the installation is complete, the installer will output the IP of the master node (you will need it further). Example output:

...
┌ 🎈 ~ Common: Kubernetes Master Node addresses for SSH
│ cloud-demo-master-0 | ssh ubuntu@1.2.3.4
└ 🎈 ~ Common: Kubernetes Master Node addresses for SSH (0.00 seconds)

Almost everything is ready for a fully-fledged Deckhouse Kubernetes Platform to work!

If something went wrong

Some providers’ clusters may require extra steps before or after installing Deckhouse.

Here are some common problems and ways to solve them. Should you run into difficulties installing Deckhouse in an existing cluster, please, share them by creating an issue.

Installation errors at the 'Waiting for Deckhouse to become Ready' step

  • Error of the following kind:
    │ │ ┌ Waiting for Deckhouse to become Ready
    │ │ │ Deckhouse pod found: deckhouse-64649df6f9-mf6dt (Pending)
    │ │ │ Deckhouse pod found: deckhouse-64649df6f9-mf6dt (Pending)
    

    Probably, there is no node in the cluster with the node-role.kubernetes.io/control-plane: "" label which is originally used in the nodeSelector of the deckhouse deployment manifest.

    Ways to fix the error:

    • Insert the proper nodeSelector into the deckhouse deployment:
      kubectl -n d8-system edit deployment/deckhouse
      
    • Delete nodeSelector in the deckhouse deployment:
      kubectl patch -n d8-system deployment deckhouse --type json -p '[{"op": "remove", "path": "/spec/template/spec/nodeSelector"}]'
      

There is no Ingress controller in the cluster...

The documentation module (deckhouse-web in Deckhouse 1.45 or older), which provides a web interface to the cluster documentation, requires the Ingress controller to be active in the cluster. If there is no Ingress controller in your cluster, you can use the built-in ingress-nginx module:

  • Enable the ingress-nginx module, by executing the following command:

    kubectl create -f - <<EOF
    apiVersion: deckhouse.io/v1alpha1
    kind: ModuleConfig
    metadata:
      name: ingress-nginx
    spec:
      enabled: true
    EOF
    
    kubectl create -f - <<EOF apiVersion: deckhouse.io/v1alpha1 kind: ModuleConfig metadata: name: ingress-nginx spec: enabled: true EOF
  • Create an ingress-nginx-controller.yml file with the following contents:

    ingress-nginx-controller.ymlCopy filenameCopy content
    # Section containing the parameters of NGINX Ingress controller.
    # https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/402-ingress-nginx/cr.html
    apiVersion: deckhouse.io/v1
    kind: IngressNginxController
    metadata:
      name: nginx
    spec:
      ingressClass: nginx
      # The way traffic goes to cluster from the outer network.
      inlet: HostPort
      hostPort:
        httpPort: 80
        httpsPort: 443
      # Describes on which nodes the component will be located.
      # You might consider changing this.
      nodeSelector:
        node-role.kubernetes.io/control-plane: ""
      tolerations:
      - operator: Exists
    
    # Section containing the parameters of NGINX Ingress controller. # https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/402-ingress-nginx/cr.html apiVersion: deckhouse.io/v1 kind: IngressNginxController metadata: name: nginx spec: ingressClass: nginx # The way traffic goes to cluster from the outer network. inlet: HostPort hostPort: httpPort: 80 httpsPort: 443 # Describes on which nodes the component will be located. # You might consider changing this. nodeSelector: node-role.kubernetes.io/control-plane: "" tolerations: - operator: Exists
  • Once Deckhouse is installed, apply the file using the command below:

    kubectl create -f ingress-nginx-controller.yml
    
    kubectl create -f ingress-nginx-controller.yml

Use a Docker image to install the Deckhouse Kubernetes Platform. It is necessary to transfer configuration files to the container.

Run the installer on the personal computer.

Linux / macOS Windows

 echo <LICENSE_TOKEN> | docker login -u license-token --password-stdin registry.deckhouse.io
docker run --pull=always -it -v "$PWD/config.yml:/config.yml" \
  -v "$HOME/.kube/config:/kubeconfig" registry.deckhouse.io/deckhouse/ee/install:stable bash
echo <LICENSE_TOKEN> | docker login -u license-token --password-stdin registry.deckhouse.io docker run --pull=always -it -v "$PWD/config.yml:/config.yml" \ -v "$HOME/.kube/config:/kubeconfig" registry.deckhouse.io/deckhouse/ee/install:stable bash

Log in on the personal computer to the container image registry by providing the license key as a password:

docker login -u license-token registry.deckhouse.io
docker login -u license-token registry.deckhouse.io

Run a container with the installer:

docker run --pull=always -it -v "%cd%\config.yml:/config.yml" -v "%userprofile%\.kube\config:/kubeconfig" registry.deckhouse.io/deckhouse/ee/install:stable bash -c "chmod 400 /tmp/.ssh/id_rsa; bash"
docker run --pull=always -it -v "%cd%\config.yml:/config.yml" -v "%userprofile%\.kube\config:/kubeconfig" registry.deckhouse.io/deckhouse/ee/install:stable bash -c "chmod 400 /tmp/.ssh/id_rsa; bash"

Notes:

  • Kubectl configuration file with access to Kubernetes API must be mount as the /kubeconfig file in the container.

Now, to initiate the process of installation, you need to execute inside the container:

dhctl bootstrap-phase install-deckhouse --kubeconfig=/kubeconfig --config=/config.yml
dhctl bootstrap-phase install-deckhouse --kubeconfig=/kubeconfig --config=/config.yml

The installation process may take about from 5 to 30 minutes, depending on the connection.

After the installation is complete, the installer will output the IP of the master node (you will need it further). Example output:

...
┌ 🎈 ~ Common: Kubernetes Master Node addresses for SSH
│ cloud-demo-master-0 | ssh ubuntu@1.2.3.4
└ 🎈 ~ Common: Kubernetes Master Node addresses for SSH (0.00 seconds)

Almost everything is ready for a fully-fledged Deckhouse Kubernetes Platform to work!

If something went wrong

Some providers’ clusters may require extra steps before or after installing Deckhouse.

Here are some common problems and ways to solve them. Should you run into difficulties installing Deckhouse in an existing cluster, please, share them by creating an issue.

Installation errors at the 'Waiting for Deckhouse to become Ready' step

  • Error of the following kind:
    │ │ ┌ Waiting for Deckhouse to become Ready
    │ │ │ Deckhouse pod found: deckhouse-64649df6f9-mf6dt (Pending)
    │ │ │ Deckhouse pod found: deckhouse-64649df6f9-mf6dt (Pending)
    

    Probably, there is no node in the cluster with the node-role.kubernetes.io/control-plane: "" label which is originally used in the nodeSelector of the deckhouse deployment manifest.

    Ways to fix the error:

    • Insert the proper nodeSelector into the deckhouse deployment:
      kubectl -n d8-system edit deployment/deckhouse
      
    • Delete nodeSelector in the deckhouse deployment:
      kubectl patch -n d8-system deployment deckhouse --type json -p '[{"op": "remove", "path": "/spec/template/spec/nodeSelector"}]'
      

There is no Ingress controller in the cluster...

The documentation module (deckhouse-web in Deckhouse 1.45 or older), which provides a web interface to the cluster documentation, requires the Ingress controller to be active in the cluster. If there is no Ingress controller in your cluster, you can use the built-in ingress-nginx module:

  • Enable the ingress-nginx module, by executing the following command:

    kubectl create -f - <<EOF
    apiVersion: deckhouse.io/v1alpha1
    kind: ModuleConfig
    metadata:
      name: ingress-nginx
    spec:
      enabled: true
    EOF
    
    kubectl create -f - <<EOF apiVersion: deckhouse.io/v1alpha1 kind: ModuleConfig metadata: name: ingress-nginx spec: enabled: true EOF
  • Create an ingress-nginx-controller.yml file with the following contents:

    ingress-nginx-controller.ymlCopy filenameCopy content
    # Section containing the parameters of NGINX Ingress controller.
    # https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/402-ingress-nginx/cr.html
    apiVersion: deckhouse.io/v1
    kind: IngressNginxController
    metadata:
      name: nginx
    spec:
      ingressClass: nginx
      # The way traffic goes to cluster from the outer network.
      inlet: HostPort
      hostPort:
        httpPort: 80
        httpsPort: 443
      # Describes on which nodes the component will be located.
      # You might consider changing this.
      nodeSelector:
        node-role.kubernetes.io/control-plane: ""
      tolerations:
      - operator: Exists
    
    # Section containing the parameters of NGINX Ingress controller. # https://deckhouse.io/products/kubernetes-platform/documentation/v1/modules/402-ingress-nginx/cr.html apiVersion: deckhouse.io/v1 kind: IngressNginxController metadata: name: nginx spec: ingressClass: nginx # The way traffic goes to cluster from the outer network. inlet: HostPort hostPort: httpPort: 80 httpsPort: 443 # Describes on which nodes the component will be located. # You might consider changing this. nodeSelector: node-role.kubernetes.io/control-plane: "" tolerations: - operator: Exists
  • Once Deckhouse is installed, apply the file using the command below:

    kubectl create -f ingress-nginx-controller.yml
    
    kubectl create -f ingress-nginx-controller.yml