This module installs and configures the Prometheus monitoring system. Also, it configures metrics scraping for many typical applications and provides the basic set of Prometheus alerts and Grafana dashboards.
If a storage class supports automatic volume expansion (allowVolumeExpansion: true), it can automatically expand the volume if there is not enough disk space for Prometheus data. Otherwise, you will receive an alert that the volume space in Prometheus is running out.
The Vertical Pod Autoscaler module makes it possible to automatically request CPU and memory resources based on the utilization history when the Pod is recreated. Also, the Prometheus memory consumption is minimized by caching requests to it via Trickster.
Both pulling and pushing of metrics are supported.
Monitoring hardware resources
Prometheus tracks hardware resources and displays the corresponding utilization graphs for:
- disk drive,
The graphs can be grouped by:
Deckhouse enables monitoring of a large set of “health” parameters of Kubernetes and its components out-of-the-box, including:
- overall cluster utilization;;
- connectivity of Kubernetes nodes (by measuring RTT between nodes);
- availability and operability of control plane components:
- time synchronization on nodes, etc.
The detailed description is available here.
Advanced monitoring mode
Deckhouse also provides the advanced monitoring mode that implements custom metrics-based alerts. The following exporters are supported:
extended-monitoring-exporter. Implements extended scraping of metrics for namespaces (that have the
extended-monitoring.flant.com/enabled=””annotation attached), including information about available inodes/space on disks, monitoring the node usage, the availability of Deployment,
image-availability-exporter. Adds metrics (and send alerts) for tracking the availability of the container image specified in the
imagefield in the Pod’s spec in
Alerting in advanced monitoring mode
Deckhouse allows you to flexibly configure alerting for each namespace and assign criticality depending on the threshold value. You can set thresholds in various namespaces for parameters such as:
- empty space and inodes on a disk;
- CPU usage for a node and a container;
- percent of 5xx errors on
- number of unavailable Pods in a
The Deckhouse monitoring also implements event notifications. The standard edition includes a large set of necessary alerts for monitoring the state of the cluster and its components. At the same time, you can add custom alerts.
Sending alerts to external systems
Deckhouse supports sending alerts using
- via the SMTP protocol;
- to PagerDuty;
- to Slack;
- to Telegram;
- via the Webhook mechanism;
- by any other means supported in Alertmanager.
Components installed by Deckhouse
|prometheus-main||The primary Prometheus instance that scrapes metrics every 30 seconds (you can change this value using the
|prometheus-longterm||The secondary Prometheus instance that scrapes the data of the primary Prometheus instance (
|trickster||The caching proxy that reduces the load on Prometheus.|
|grafana||The managed observability platform with ready-to-use dashboards for all Deckhouse modules and popular applications. Grafana instances are highly available, stateless, and configured by CRDs.|
|metrics-adapter||The component connecting Prometheus and Kubernetes metrics API. It enables HPA support in a Kubernetes cluster.|
|vertical-pod-autoscaler||An autoscaling tool to help size Pods for the optimal CPU and memory resources required by the Pods.|
|Various Exporters||Precooked exporters connected to Prometheus. The list includes exporters for all necessary metrics:
Deckhouse has interfaces to integrate with various popular solutions in the following ways:
|Alertmanagers||Alertmanagers could be connected to Prometheus and Grafana and deployed to the Deckhouse cluster or out of it.|
|Long-term metrics storages||Utilizing remote write protocol, it is possible to send metrics from Deckhouse to plenty of storages, including Cortex, Thanos, VictoriaMetrics.|