The module is available only in Deckhouse Enterprise Edition.

Preliminary steps

Several preparatory steps are required prior to installing the Deckhouse Observability Platform.

Configuring Node Groups (nodegroups) and Labels

The components of the Deckhouse Observability Platform are automatically assigned to nodes based on the names of the node groups (nodegroups) and the labels attached to them. The selection algorithm proceeds as follows: first, node groups with matching names are selected, then node groups with matching labels are selected. If no suitable nodes are found, the component startup is delayed until the necessary node groups are created or the necessary labels are attached to the existing groups.

Naming Node Groups

The minimum configuration requires defining a node group with the name observability. All platform components will be run on the nodes belonging to this group. You can create additional node groups for more granular component distribution. Possible options are:

  • observability – for user interface (UI) components and others if a separate group is not created for them.
  • observability-ceph – for deploying Ceph components.
  • observability-metrics – for deploying components responsible for metrics collection and processing.
  • observability-logs – for deploying components responsible for logs collection and processing.

Configuring Node Groups Using Labels

You can also distribute components by using labels. Attach the dedicated/observability: "" label to the target node group. Possible label options are:

  • dedicated/observability: "" – for user interface (UI) components and others if a separate group is not created for them.
  • dedicated/observability-ceph: "" – for deploying Ceph components.
  • dedicated/observability-metrics: "" – for deploying components responsible for metrics collection and processing.
  • dedicated/observability-logs: "" – for deploying components responsible for logs collection and processing.

Setting Labels on an Existing Node Group

To set a label on an existing NodeGroup, you can use the following command as an example:

kubectl patch ng worker -p '{"spec":{"nodeTemplate": {"labels": {"dedicated/observability": ""}}}}' --type=merge

Enabling the operator-ceph Module

The Deckhouse Observability Platform depends on the operator-ceph module.

Command to enable the module:

kubectl -n d8-system exec svc/deckhouse-leader -- deckhouse-controller module enable operator-ceph

Enabling the operator-postgres Module (for in-cluster DB)

The Deckhouse Observability Platform uses PostgreSQL. To deploy the database in the cluster, the operator-postgres module must be activated.

Command to enable the module:

kubectl -n d8-system exec svc/deckhouse-leader -- deckhouse-controller module enable operator-postgres

Configuring External Authentication (Optional)

The Deckhouse Observability Platform integrates with various third-party authentication systems. Keycloak and Okta are among the systems supported out of the box. To connect to other systems such as LDAP or GitLab, use an intermediary solution called Dex, a part of the Deckhouse Kubernetes Platform. The complete list of supported systems can be found in the documentation: List of Supported Systems.

Connecting Dex for Authentication

To connect, for example, LDAP, you first need to create a DexProvider. This will allow the system to correctly interact with the third-party authentication service. Let’s consider an example of setting up a connection to LDAP:

  1. Create a configuration file dop-dex-provider.yaml with the following content:

    apiVersion: deckhouse.io/v1
    kind: DexProvider
    metadata:
      name: dop-active-directory
    spec:
      displayName: Active Directory
      ldap:
        bindDN: cn=admin,dc=example,dc=org
        bindPW: admin
        groupSearch:
          baseDN: ou=dop,dc=example,dc=org
          filter: (objectClass=groupOfNames)
          nameAttr: cn
          userMatchers:
          - groupAttr: member
            userAttr: DN
        host: ad.example.com::389
        insecureNoSSL: true
        insecureSkipVerify: true
        startTLS: false
        userSearch:
          baseDN: ou=dop,dc=example,dc=org
          emailAttr: mail
          filter: (objectClass=inetOrgPerson)
          idAttr: uidNumber
          nameAttr: cn
          username: mail
        usernamePrompt: Email Address
      type: LDAP
    

    Note that in the example, the provided data should be replaced with real values specific to your system.

  2. Apply the file to the cluster by executing the command:

    kubectl apply -f dop-dex-provider.yaml
    

Additional examples of provider settings can be found in the Deckhouse Observability Platform documentation.

Important: Currently, it is not possible to configure an explicit mapping between DexProvider and the specific applications that use it. If DexProviders are already configured in your Deckhouse Kubernetes Platform, users of Deckhouse Observability Platform will see them in the list of available authentication methods.

Installation in Cloud Environments

In this example, a PostgreSQL database is used, deployed within the cluster. The operator-postgres module must be activated.

Установка в Yandex Cloud

  1. Prepare the configuration file

    Create the dop-mc.yaml file with the settings. Ensure that all the changeable parameters, such as the domain, resource quantities, and identifiers (random-string), are replaced with the actual values for your environment:

    apiVersion: deckhouse.io/v1alpha1
    kind: ModuleConfig
    metadata:
      name: observability-platform
    spec:
      enabled: true
      settings:
        general:
          baseDomain: dop.example.com
          clusterName: dc1
          tls:
            issuer: letsencrypt
        storage:
          ceph:
            configOverride: |
              [osd]
              osd_memory_cache_min = 1Gi
              bluestore_cache_autotune = true
              bluestore_min_alloc_size = 4096
              osd_pool_default_pg_autoscale_mode = off          
            mon:
              storageClass: network-ssd
              storageSize: 10Gi
            osd:
              count: 3
              storageClass: network-ssd-nonreplicated
              storageSize: 80Gi
          metrics:
            defaultStorageClass: network-ssd
            etcd:
              storageSize: 5Gi
            ingester:
              resources:
                limits:
                  memory: 4Gi
                requests:
                  cpu: 1
                  memory: 4Gi
              storageSize: 10Gi
            storeGateway:
              storageSize: 10Gi
        ui:
          auth:
            mode: default
          clusterBootstrapToken: random-string
          postgres:
            backup:
              enabled: false
            internal:
              resources:
                limits:
                  cpu: "1"
                  memory: 1Gi
                requests:
                  cpu: 500m
                  memory: 1Gi
              storage:
                class: network-ssd
                size: 10Gi
            mode: Internal
          secretKeyBase: random-string
          tenantHashSalt: random-string
      version: 1
    
  2. Apply the configuration

Apply the settings to the Deckhouse Kubernetes Platform cluster:

kubectl apply -f dop-mc.yaml
  1. Installation outcome

The Deckhouse Observability Platform will be deployed in the Deckhouse Kubernetes Platform with storage enabled for metrics. All the necessary resources will be provisioned in the cloud, and the database will run in the cluster.

Installation in Bare-Metal Cluster

Installation in a bare-metal cluster requires additional steps for storage setup and node preparation.

Enabling sds-node-configurator and sds-local-volume Modules

These modules are required for creating persistent volumes (PV) based on physical disks and organizing long-term storage using Ceph.

Enable the modules by running the commands:

kubectl -n d8-system exec svc/deckhouse-leader -- deckhouse-controller module enable sds-node-configurator
kubectl -n d8-system exec svc/deckhouse-leader -- deckhouse-controller module enable sds-local-volume

Setting Up LocalPathProvisioner for Stateful Components

Deckhouse Observability Platform components that operate in a stateful mode require persistent volumes (PV). In cases where Deckhouse Observability Platform is deployed on bare metal clusters or virtual machines without the possibility of using cloud PV, LocalPathProvisioner is used.

Preparing Cluster Nodes

Ensure that a block device of the required size is mounted in the /opt/local-path-provisioner directory and this mountpoint is listed in the /etc/fstab on all nodes of the node group where the DOP components will be deployed. This device will be used for data storage, so it is crucial to ensure its volume meets your requirements.

Creating a LocalPathProvisioner

  1. Create the local-path-provisioner.yaml file with the following content:

    ---
    apiVersion: deckhouse.io/v1alpha1
    kind: LocalPathProvisioner
    metadata:
      name: dop-localpath
    spec:
      nodeGroups:
      - observability
      path: /opt/local-path-provisioner
      reclaimPolicy: Delete
    
  2. Apply the file to the cluster by executing the command:

    kubectl apply -f local-path-provisioner.yaml
    

    LocalPathProvisioner resources must be created for each NodeGroup that matches by name or label. For example, they might be observability-ceph, observability-metrics, observability-logs, or any NodeGroups with the label dedicated/observability*.

Creating StorageClass for Long-Term Storage

Long-term storage components in the Deckhouse Observability Platform require dedicated block devices. In a situation where the Deckhouse Observability Platform is installed in a Deckhouse Kubernetes Platform running on bare metal or virtual machines without access to cloud PV, this task is solved by the модуль sds-local-volume.

  1. Check whether the block devices are available:

    Execute the command:

    kubectl get bd
    

    Ensure that there are devices with the consumable=true flag in the output. For example:

    NAME                                           NODE                CONSUMABLE   SIZE          PATH        AGE
    dev-44587ffa2c48e7e403db6abc699cc3b809489c1d   dop-bare-metal01    true         331093016Ki   /dev/sda   3h
    dev-5f61dadefe049b620c6fc5433046cf02a80247a0   dop-bare-metal02    true         331093016Ki   /dev/sda   3h2m
    dev-665628749db2e1c93a74f3c224bb98502111fdd6   dop-bare-metal03    true         331093016Ki   /dev/sda   175m
    
  2. Create LvmVolumeGroups from available block devices on nodes labeled with node.deckhouse.io/group=observability. The node selection can be changed by specifying the necessary label according to your cluster configuration. It may be node.deckhouse.io/group=observability-ceph or dedicated=observability-ceph

    Run the following command:

    kubectl get no -l node.deckhouse.io/group=ceph -o name | cut -f 2 -d / | while read node; do cat <<EOF | kubectl apply -f -
    apiVersion: storage.deckhouse.io/v1alpha1
    kind: LVMVolumeGroup
    metadata:
      name: "dop-ceph-$node"
    spec:
      type: Local
      local:
        nodeName: "$node"
      blockDeviceSelector:
        matchLabels:
          kubernetes.io/hostname: "$node"
      actualVGNameOnTheNode: "dop-ceph"
    EOF
    done
    

    Check whether the creation has been successful with:

    kubectl get lvg
    

    Sample output:

    NAME                         HEALTH        NODE                SIZE       ALLOCATED SIZE   VG         AGE
    dop-ceph-dop-bare-metal01    Operational   dop-bare-metal01    329054Mi   0                dop-ceph   37s
    dop-ceph-dop-bare-metal02    Operational   dop-bare-metal02    329054Mi   0                dop-ceph   37s
    dop-ceph-dop-bare-metal03    Operational   dop-bare-metal03    329054Mi   0                dop-ceph   37s
    
  3. Create LocalStorageClass:

    Execute the command:

    kubectl get lvg -o json | jq 'reduce .items[].metadata.name as $name ({apiVersion: "storage.deckhouse.io/v1alpha1", kind: "LocalStorageClass", metadata: {name: "dop-ceph"}, spec: {reclaimPolicy: "Delete", volumeBindingMode: "WaitForFirstConsumer", lvm: {type: "Thick", thick: {contiguous: true}}}}; .spec.lvm.lvmVolumeGroups += [{name: $name}])' | kubectl apply -f -
    
  4. Confirm that StorageClass has been successfully created:

    Ensure that there is a StorageClass named dop-ceph in the output of the command:

    kubectl get sc
    

Installing Deckhouse Observability Platform

  1. Create and configure the dop-mc.yaml file. Remember to insert values from your environment:

    apiVersion: deckhouse.io/v1alpha1
    kind: ModuleConfig
    metadata:
      name: observability-platform
    spec:
      enabled: true
      settings:
        general:
          baseDomain: dop.example.com
          clusterName: dc1
          tls:
            issuer: letsencrypt
        storage:
          ceph:
            configOverride: |
              [osd]
              osd_memory_cache_min = 1Gi
              bluestore_cache_autotune = true
              bluestore_min_alloc_size = 4096
              osd_pool_default_pg_autoscale_mode = off          
            mon:
              storageClass: dop-localpath
              storageSize: 10Gi
            osd:
              storageClass: dop-ceph
          metrics:
            defaultStorageClass: dop-localpath
            etcd:
              storageSize: 5Gi
            ingester:
              resources:
                limits:
                  memory: 4Gi
                requests:
                  cpu: 1
                  memory: 4Gi
              storageSize: 10Gi
            storeGateway:
              storageSize: 10Gi
        ui:
          auth:
            mode: default
          postgres:
            backup:
              enabled: false
            internal:
              resources:
                limits:
                  cpu: "1"
                  memory: 1Gi
                requests:
                  cpu: 500m
                  memory: 1Gi
              storage:
                class: dop-localpath
                size: 10Gi
          clusterBootstrapToken: random-string
          secretKeyBase: random-string
          tenantHashSalt: random-string
      version: 1
    
  2. Apply the settings using the command:

    kubectl apply -f dop-mc.yaml
    

The platform will be deployed with the storage for metrics enabled and the associated resources used.

Checking Functionality

  1. Ensure that all pods in d8-observability-platform are in Running state:

    kubectl -n d8-observability-platform get po
    
  2. Check if the user interface (UI) is available at .spec.settings.general.baseDomain (in this example, https://dop.example.com).

Use theese credentials to log in:

  • Username: admin@deckhouse.ru;

  • Password: password.

    Note that you will need to change the password upon the first login.