The module lifecycle stageGeneral Availability
The module has requirements for installation

Reference for all metrics collected and shipped by the Deckhouse Observability Platform agent (opAgent). Metrics are grouped by topic: base system, disks, network, hardware, processes and containers, virtualization, databases, queues, web servers, JVM, mail, security, custom, and internal metrics.

How to read this reference

Metric names

Inside the agent, metric names use dots (cpu.usage.user). When shipped to the storage, names are converted to a Prometheus-compatible form:

  • a dot . is replaced with __ (double underscore): cpu.usage.usercpu__usage__user;
  • a dash - is replaced with ____ (four underscores);
  • a name starting with a digit gets the ___ prefix: 1min___1min.

All names in this document are given in the form they are stored and queried (that is, with __).

Common labels

Every agent metric automatically gets the following labels:

Label Value
job, okagent_plugin name of the plugin that collected the metric (cpu, postgresql, nginx, …)
conf, okagent_plugin_instance plugin instance: config path, service address, or container
instance hostname of the server where the agent runs

Metrics from Kubernetes containers additionally get k8s_namespace, k8s_pod, k8s_container, k8s_namespace_pod. Custom labels from the labels: section of a plugin config are added to all its metrics.

The tables below list only metric-specific labels — the common ones are not repeated.

Metric types

Type Description
gauge instantaneous value (memory size, connection count)
counter monotonically increasing counter (bytes since start); use rate() to get a per-second rate
rate per-second rate already computed by the agent over the scrape interval (~60 seconds); no rate() needed
text text value (version, MAC address). Stored as a series with the value=<text> label and metric_type="text"
status status metric: value 1 and value="empty" — everything is fine; value 0 and value=<error text> — a problem

Integration status

All auto-discovered integrations (PostgreSQL, MySQL, Redis, …) publish a status or plugin__status metric: empty text — collection works, error text — what prevents collection (missing permissions, unreachable port, etc.).

Base system metrics

CPU (plugin cpu)

Collected automatically on every server.

Metric Labels Type Description
cpu__logical_cores gauge Number of logical cores
cpu__physical_sockets gauge Number of physical CPU sockets
cpu__physical_cores gauge Total number of physical cores
cpu__usage__user rate Share of CPU time in user mode, %
cpu__usage__system rate Share of CPU time in system mode, %
cpu__usage__nice rate Share of CPU time at modified priority (nice), %
cpu__usage__iowait rate Share of time waiting for disk I/O, %
cpu__usage__irq rate Share of time handling hardware interrupts, %
cpu__usage__softirq rate Share of time handling software interrupts, %
cpu__usage__steal rate Share of time “stolen” by the hypervisor, %
user, system, nice, idle, iowait, irq, softirq, steal counter Accumulated CPU time per mode, seconds (legacy names)
cpu__frequency__mhz core gauge Current core frequency, MHz
cpu__governor core, governor gauge Flag (=1) for the active scaling governor of a core (Linux only)
cpu_flags model_name, flag gauge Flag (=1) for a CPU feature flag (avx, sse4, etc.)
cpu__thermal_throttle__core_throttle_count cpu counter Core thermal throttling events (Linux only)
cpu__thermal_throttle__package_throttle_count cpu counter CPU package thermal throttling events (Linux only)

Memory and swap (plugin memory)

Metric Labels Type Description
memory__total gauge RAM size, bytes
memory__available gauge Available memory (including reclaimable caches), bytes
memory__usage__used gauge Used memory (total − free − buffers − cached), bytes
memory__usage__free gauge Free memory, bytes
memory__usage__buffers gauge Memory in buffers, bytes
memory__usage__cached gauge Memory in page cache, bytes
total, available, free, buffers, cached gauge Same values (legacy names)
memory__errors type = correctable | uncorrectable counter ECC memory errors from EDAC (Linux only, when /sys/devices/system/edac exists)
memory__numa__allocated node, allocation = local_node | other_node | interleaved counter Page allocations per NUMA node (only with ≥2 nodes)
swap__total gauge Swap size, bytes
swap__space__free gauge Free swap, bytes
swap__space__used gauge Used swap, bytes
swap__io__in rate Swap-in rate, pages/s
swap__io__out rate Swap-out rate, pages/s
swap_total, swap_free gauge Swap size and free swap (legacy names)
swap_in, swap_out counter Accumulated swap page counters (legacy names)

Load average (plugin load_average)

Metric Labels Type Description
load_average__1min gauge 1-minute load average
load_average__5min gauge 5-minute load average
load_average__15min gauge 15-minute load average
___1min, ___5min, ___15min gauge Same values (legacy names)

OS information (plugin uname)

Metric Labels Type Description
uname os, machine, distrib, version, kernel, gather_tool gauge Flag (=1) with OS metadata: distribution, version, kernel, architecture
host__uptime gauge Server uptime, seconds
os, machine, distrib, version, kernel text Separate text metrics with the same values
uname_gather_error gauge =1 if the distribution could not be determined

Time and NTP (plugin time)

Metric Labels Type Description
time__ntp__status status NTP poll status: empty — OK, text — error (servers not found, timeout)
time__current_offset, time__offset__ntp gauge Local clock offset relative to NTP, seconds (can be negative)

NTP servers are taken from /etc/ntp.conf; pool.ntp.org is used as fallback.

Host identity (plugins metadata, machine_identity)

Metric Labels Type Description
machine_identity machine_id, source = dmi_uuid | dmi_serial | machine_id gauge Flag (=1) with a stable machine identifier (DMI UUID, serial number, or /etc/machine-id)
local_metadata__host__hostname_status status Hostname change tracking: text when it changed
local_metadata__host__group text Host group from the local agent config (if set)
local_metadata__host__description text Host description from the local config (if set)

Disks and storage

File systems (plugin disk)

Per mounted partition; label mount_point.

Metric Labels Type Description
bytes_total mount_point gauge Total file system size, bytes
bytes_used, disk__partition__bytes_used mount_point gauge Used, bytes
disk__partition__bytes_available, disk__partition__space__available mount_point gauge Available to the user, bytes
disk__partition__bytes_reserved, disk__partition__space__reserved mount_point gauge Reserved for root, bytes
disk__partition__space__used mount_point gauge Used, bytes
inodes_total mount_point gauge Total inodes
inodes_used, disk__partition__inodes__used mount_point gauge Used inodes
disk__partition__inodes__free mount_point gauge Free inodes
disk__partition__read_write mount_point gauge 1 — mounted rw, 0 — ro
fs, disk__partition__fs mount_point text File system type (ext4, xfs, …)
device, disk__partition__device mount_point text Partition block device
mount_options, disk__partition__mount_options mount_point text Mount options
disk__fs__errors device, fs="ext4" counter ext4 error counter from /sys/fs/ext4/*/errors_count (Linux only)

Disk I/O (plugin disk)

Rates are computed by the agent over the scrape interval.

Metric Labels Type Description
disk__device__ops__read device, pci_bdf rate Read operations per second on a block device
disk__device__ops__write device, pci_bdf rate Write operations per second
disk__device__bytes__read device, pci_bdf rate Bytes read per second
disk__device__bytes__write device, pci_bdf rate Bytes written per second
disk__device__io_time__read device, pci_bdf rate Read time, ms per second
disk__device__io_time__write device, pci_bdf rate Write time, ms per second
disk__device__io_time__total device, pci_bdf rate Total I/O time, ms per second (utilization)
disk__partition__ops__read / __write mount_point rate Read/write operations per second per partition (incl. ZFS datasets)
disk__partition__bytes__read / __write mount_point rate Read/write bytes per second per partition

SMART (plugin disk)

Disk health diagnostics. ATA attributes are read directly (SG_IO), NVMe via ioctl, SCSI/SAS via smartctl. All metrics carry the device (e.g. /dev/sda) and pci_bdf labels.

Metric Type Description
disk_smart_has_smartctl gauge 1 — smartctl is installed on the host
disk_smart_device_model text Disk model (ATA IDENTIFY)

ATA attributes — family disk_smart_<attr>_value (normalized value 0–255), disk_smart_<attr>_raw (raw value where a parser is defined), disk_smart_<attr>_threshold (failure threshold). Supported <attr>:

<attr> (SMART ID) Description
raw_read_error_rate (1) Surface read errors
reallocated_sectors_ct (5) Reallocated sectors (>0 — degradation)
power_on_hours (9) Power-on hours
spin_retry_count (10) Spindle spin-up retries
power_cycle_count (12) Power cycles
wear_leveling_count (177) Samsung SSD wear (value = remaining life, %)
program_fail_cnt_total (181) / erase_fail_cnt_total (182) NAND program/erase failures
end_to_end_error (184) End-to-end integrity errors
reported_uncorrect (187) Uncorrectable errors
command_timeout (188) Command timeouts
temperature_celsius (194) Temperature, °C (raw)
hardware_ecc_recovered (195) Corrected ECC errors
reallocated_event_count (196) Reallocation events
offline_uncorrectable (198) Uncorrectable sectors from offline scan
udma_crc_error_count (199) Interface CRC errors (cabling)
percent_lifetime_remain (202) / ssd_life_left (231) / media_wearout_indicator (233) SSD remaining life, %
total_lbas_written (241) / total_lbas_read (242) Total sectors written/read

NVMe — all gauge:

Metric Description
disk_smart_nvme_available_spare Remaining spare blocks, %
disk_smart_nvme_available_spare_threshold Spare warning threshold, %
disk_smart_nvme_percentage_used Endurance used, % (0–255, >100 — beyond rated)
disk_smart_nvme_temperature_celsius Controller temperature, °C
disk_smart_nvme_media_errors Uncorrectable media errors
disk_smart_nvme_power_on_hours Power-on hours
disk_smart_nvme_unsafe_shutdowns Unsafe power-off events
disk_smart_nvme_power_cycles Power cycles
disk_smart_nvme_data_units_read / _written Read/written (1 unit = 500 KiB)
disk_smart_nvme_host_read_commands / _write_commands Host read/write commands
disk_smart_nvme_critical_warning Raw critical warning byte (0 — OK)
disk_smart_nvme_critical_warning_spare_below 1 — spare exhausted
disk_smart_nvme_critical_warning_temp_exceeded 1 — overheating
disk_smart_nvme_critical_warning_reliability_degraded 1 — reliability degraded
disk_smart_nvme_critical_warning_read_only 1 — disk went read-only
disk_smart_nvme_critical_warning_backup_failed 1 — backup power failure

SCSI/SAS (via smartctl) — all gauge:

Metric Description
disk_smart_scsi_temperature_celsius Temperature, °C
disk_smart_scsi_grown_defects Grown defect list
disk_smart_scsi_non_medium_errors Non-medium errors
disk_smart_scsi_read_uncorrected / _write_uncorrected / _verify_uncorrected Uncorrected read/write/verify errors
disk_smart_scsi_info_exceptions 1 — SMART status “failed”

RAID (plugin raid)

Hardware controllers (MegaRAID/storcli, Adaptec/arcconf, HPE/ssacli) and Linux software md. Linux only. controller_id: megaraid:0, hpe:0, adaptec:1, soft-raid. The state_class label is a normalized state: healthy, rebuild, predictive, failed, offline, unknown (for arrays: optimal, degraded, offline, rebuilding, unknown).

Metric Labels Type Description
raid_controller_info controller_id, model, serial, firmware, driver gauge =1; controller inventory record
raid_controller_state controller_id, state_class, raw_state gauge =1; controller state
raid_controller_temperature_celsius controller_id gauge Controller temperature, °C
raid_controller_bbu controller_id, state_class, raw_state gauge =1; battery (BBU) state
raid_controller_bbu_health_percent controller_id gauge Battery health, 0–100 %
raid_controller_cachevault controller_id, state_class, raw_state gauge =1; CacheVault state (MegaRAID)
raid_array_info controller_id, array, raid_level, array_type gauge =1; array (VD/LD) inventory record
raid_array_state controller_id, array, state_class, raw_state gauge =1; array state (incl. md arrays)
raid_pd_state controller_id, slot_id, model, serial, wwn, media_type, state_class, raw_state gauge =1; state of a physical drive behind the controller
raid_md_degraded array, controller_id="soft-raid" gauge 1 — md array degraded
raid_md_rebuild_pct array, controller_id="soft-raid" gauge md rebuild progress, %
plugin__status controller status Empty text — collection OK; text — controller detected via PCI driver but the CLI tool is missing or parsing failed

Network

Network interfaces (plugin net)

Per interface except veth*; label interface.

Metric Labels Type Description
net__interface__bytes__in / __out interface rate Bytes/s received/sent
net__interface__packets__in / __out interface rate Packets/s
net__interface__errors__in / __out interface rate Errors/s
net__interface__drops__in / __out interface rate Dropped packets/s
in_bytes, in_packets, in_errors, in_dropped, out_bytes, out_packets, out_errors, out_dropped interface counter Cumulative counters from /proc/net/dev (legacy names)
net__interface__status, status interface gauge 1 — interface is UP, 0 — DOWN
mtu interface gauge Interface MTU
hw_address interface text MAC address
net__ip__status interface, ip, version = v4 | v6 gauge =1; flag for an IP address present on the interface
ipv4_addresses, ipv6_addresses interface text JSON lists of interface addresses
net__conntrack__count gauge Current number of conntrack entries (Linux only)
net__conntrack__max gauge conntrack table limit

Sockets and TCP (plugin netstat)

Linux only; collected in the host network namespace.

Metric Labels Type Description
netstat__protocol__packets__in / __out protocol = tcp | udp | icmp | ip counter Received/sent packets (segments, datagrams) per protocol
netstat__protocol__packets__errors protocol, error counter Protocol errors: UDP — NoPorts, RcvbufErrors, SndbufErrors, InErrors; TCP — OutRsts, InErrs; IP — InHdrErrors, InAddrErrors, InUnknownProtos, InDiscards, OutNoRoutes, ReasmFails, FragFails
netstat__protocol__packets__retransmits protocol="tcp" counter TCP retransmits
netstat__listen__status listen_ip, listen_port gauge =1 per listening port (with >200 ports they are grouped into ranges)
netstat__listen__current_ack_backlog listen_ip, listen_port gauge Current accept queue backlog
netstat__listen__max_ack_backlog listen_ip, listen_port gauge Maximum backlog
netstat__connections__inbound__count listen_ip, listen_port, remote_ip, state gauge Inbound TCP connections per state (ESTABLISHED, TIME_WAIT, SYN_RECV, …)
netstat__connections__outbound__count remote_ip, state gauge Outbound connections per state
netstat__connections__inbound__with_read_queue / outbound… same as count gauge Connections with a non-empty read queue
netstat__connections__inbound__with_write_queue / outbound… same as count gauge Connections with a non-empty write queue
netstat__connections__inbound__rtt / outbound… same as count + percentile="95" gauge 95th percentile of connection RTT, seconds
okagent__netstat__listen__status status Text too many listen ports when the port list was truncated

Non-local remote addresses are aggregated into remote_ip="~nonlocal".

SNMP devices (plugin netdev, configured)

Polls network equipment via SNMP v2c (IF-MIB). Configured by a YAML file with host and community.

Metric Labels Type Description
in_bytes / out_bytes interface, description counter Bytes received/sent on a device port
in_packets / out_packets interface, description counter Packets
speed interface, description gauge Port speed, bytes/s

Hardware and inventory

Hardware inventory (plugin cmdb)

Full hardware inventory of bare-metal servers: SMBIOS, sysfs, RAID CLI tools. Disabled on virtual machines. All *_info metrics are gauges with value 1; identity is carried by labels (component_key, serial, model, manufacturer, etc.). Rescan every 5 minutes.

Metric Type Description
cmdb_system_info gauge System (SMBIOS Type 1): manufacturer, model, serial number
cmdb_baseboard_info gauge Motherboard (Type 2)
cmdb_bios_info gauge BIOS (Type 0)
cmdb_chassis_info gauge Chassis (Type 3)
cmdb_cpu_socket_info gauge One series per CPU socket
cmdb_cpu_sockets_total / _populated gauge Total/populated CPU sockets
cmdb_memory_slot_info gauge One series per DIMM slot
cmdb_memory_slots_total / _populated gauge Total/populated memory slots
cmdb_memory_total_bytes gauge Installed memory, bytes
cmdb_memory_max_capacity_bytes gauge Maximum capacity per SMBIOS
cmdb_storage_block_device_info gauge One series per block device (/sys/class/block: disks, partitions, LVM, md); label type = physical_disk | raid_volume | partition | lvm | mdraid | unknown
cmdb_storage_block_device_capacity_bytes gauge Block device capacity
cmdb_storage_block_device_slave_info gauge Storage stack graph edge: deviceslave (LV → md → partition → disk)
cmdb_storage_physical_disk_info gauge Physical disk (incl. hidden behind RAID); labels attachment = direct | raid | hba, interface, state_class
cmdb_storage_physical_disk_capacity_bytes gauge Physical disk capacity
cmdb_storage_controller_info gauge RAID/HBA controller
cmdb_storage_raid_volume_info gauge RAID volume (controller virtual drive); volume_wwn, os_device_name link it to the OS
cmdb_storage_raid_volume_capacity_bytes gauge RAID volume capacity
cmdb_disks_total / cmdb_disks_total_bytes gauge Number of disks and their total capacity
cmdb_nic_info / cmdb_nics_total gauge Physical network cards
cmdb_gpu_info / cmdb_gpus_total gauge Discrete GPUs
cmdb_system_slot_info, cmdb_system_slots_total / _populated gauge PCIe expansion slots (SMBIOS Type 9)
cmdb_sata_port_info, cmdb_sata_ports_total / _occupied gauge SATA ports
cmdb_psu_info gauge Power supply unit (SMBIOS Type 39); labels status = ok | non_critical | critical | non_recoverable | unknown, present, unplugged
cmdb_psu_total / _present / _healthy gauge Total/present/healthy PSUs
cmdb_inventory_scan_success gauge 1 — scan collected data
cmdb_inventory_partial gauge 1 — some collectors failed
cmdb_inventory_scan_timestamp gauge Unix time of the last rescan
cmdb_inventory_generation gauge Monotonic counter of hardware composition changes
cmdb_inventory_fingerprint_info gauge =1; hash of the current inventory (label fingerprint)
cmdb_inventory_source_available gauge Per source (source = smbios | sysfs): 1 — source available

Sensors (plugin sensors)

lm-sensors, Linux only. Labels: device (chip), sensor.

Metric Type Description
sensor__temperature__current gauge Current sensor temperature, °C
sensor__temperature__high / __critical gauge High/critical thresholds, °C (when defined)
sensor__fan__rpm__current gauge Fan speed, RPM
sensor__fan__rpm__minimum gauge Minimum speed (when defined)

Cloud provider and packages (plugins provider_info, package_info)

Metric Labels Type Description
provider provider_name, type, virtualization_type gauge =1; detected cloud provider (amazon, azure, google, yandex, hetzner, selectel, …)
package_version package, version_full, version_major, version_minor, version_patch, version_additional gauge =1; installed package version (currently tracks sudo); sources — dpkg/dnf/yum

Processes, containers, cgroups

Processes (plugin process_info)

Metrics are aggregated by groups (process, username, container); container = ~host for processes outside containers.

Metric Extra labels Type Description
process__cpu__user / __system rate CPU consumption of a process group, cores (CPU-seconds per second)
process__max_cpu_percent_per_thread gauge Maximum load of a single thread in the group, %
process__mem__rss gauge Total RSS, KB
process__mem__swap gauge Total swap, KB
process__proc_count state gauge Process count per state (running, sleeping, zombie, …)
process__thread_count gauge Total thread count
process__disk__ops__read / __write rate Disk operations of the group, ops/s
process__disk__bytes__read / __write rate Disk traffic of the group, bytes/s
process__open_files__count gauge Open file descriptors
process__open_files__max_usage_percent gauge Maximum fd limit usage in the group, %
process__uptime gauge Maximum process uptime in the group, seconds

Cgroups (plugin cgroups)

Linux only. Common labels: cgroup_type = systemd | docker | lxc | k8s, cgroup, container; for k8s also k8s_qos and pod labels.

Metric Extra labels Type Description
cgroup__cpu__usage space = user | system counter CPU time consumed by the cgroup, seconds
cgroup__cpu__shares gauge CPU shares
cgroup__cpu__quota__period gauge CFS quota period, seconds
cgroup__cpu__quota__normalized gauge Quota in core fractions (quota/period)
cgroup__cpu__throttling__nr_periods counter Total CFS periods
cgroup__cpu__throttling__nr_throttled counter Throttled periods
cgroup__cpu__throttling__throttled_time counter Throttled time, seconds
cgroup__memory__usage__rss / __cache gauge Cgroup RSS and page cache, bytes
cgroup__memory__limit gauge Memory limit, bytes (when set)
cgroup__memory__dirty / __writeback gauge Dirty/writeback pages, bytes
cgroup__memory__pages__in / __out counter Pages paged in/out
cgroup__memory__pages__major_faults counter Major page faults
cgroup__blkio__bytes operation = read | write, device counter Cgroup disk traffic, bytes
cgroup__blkio__ops operation, device counter Disk operations
cgroup__blkio__limit__bps / __ops operation, device gauge Bandwidth/IOPS limits (when set)

Docker volumes (plugin docker)

Metric Labels Type Description
docker__container__disk__space__used container, mount_point, device gauge Used space on a container rw volume, bytes
docker__container__disk__space__available container, mount_point, device gauge Free space on the volume, bytes

Virtualization

Virtual machines (plugins qemu, vmware)

QEMU/KVM is auto-discovered from processes; VMware is configured with a vCenter address. Common VM metric labels: vm_id, vm_name, hypervisor_type, hypervisor_host; for KubeVirt — namespace, pod.

Metric Extra labels Type Description
vm_info state, guest_os gauge =1; VM identity
vm_nic_info mac, model, netdev gauge =1; one series per VM network interface
vm_disk_size_bytes drive, file, format, bus, device, device_name gauge Virtual disk size, bytes
vm_allocated_vcpu gauge Allocated vCPUs
vm_allocated_mem_bytes gauge Allocated memory, bytes
vm_cpu_seconds_total mode = user | system counter QEMU process CPU time, seconds
vm_rss_bytes gauge QEMU process RSS, bytes
vm_net_rx_bytes_total / vm_net_tx_bytes_total counter VM network traffic, bytes
vm_blk_read_bytes_total / vm_blk_write_bytes_total device counter VM disk traffic, bytes
qemu_host_kvm_module_loaded gauge 1 — KVM module loaded (host metric)
qemu_host_libvirt_socket_present gauge 1 — libvirt socket exists
qemu_host_vm_count gauge Number of QEMU processes on the host

VMware (vCenter realtime counters):

Metric Type Description
vm_vmware_cpu_usage_percent gauge VM CPU usage, 0–100 %
vm_vmware_memory_active_bytes gauge Active guest memory, bytes
vm_vmware_memory_consumed_bytes gauge Host memory consumed by the VM
vm_vmware_net_rx_rate_bps / _tx_rate_bps gauge Network traffic, bytes/s
vm_vmware_disk_read_rate_bps / _write_rate_bps gauge Disk traffic, bytes/s
vm_vmware_esxi_info gauge =1; ESXi host (labels host_name, vendor, model, cpu_model, connection_state, maintenance)
vm_vmware_esxi_cpu_cores gauge Physical host cores
vm_vmware_esxi_cpu_usage_percent gauge Host CPU usage, %
vm_vmware_esxi_memory_total_bytes / _usage_bytes gauge Host memory: total/used

Databases and caches

All plugins in this section are auto-discovered: the agent finds the database process and connects on its own. On access problems it publishes plugin__status with instructions.

PostgreSQL (plugin postgresql)

Metric Labels Type Description
postgresql__db__size database gauge Database size, bytes
postgresql__db__commits / __rollbacks database counter Transaction commits and rollbacks
postgresql__db__deadlocks database counter Deadlocks
postgresql__db__tmp_bytes database counter Written to temp files, bytes
postgresql__db__max_transaction_age database gauge Age of the oldest transaction (XID), primary only
postgresql__db__transactions_left_before_shutdown database gauge XIDs remaining before wraparound shutdown
postgresql__locks__count database, mode gauge Locks per mode
postgresql__connections__count user, database, state, client_addr gauge Sessions per state (active, idle, idle in transaction, waiting, …)
postgresql__connections__max_transaction_age user, database gauge Maximum open transaction age, seconds
postgresql__autovacuum_workers__common / __wraparound gauge Autovacuum workers (regular and anti-wraparound)
postgresql__bgwriter__buffers_clean / __buffers_backend / __buffers_alloc / __maxwritten_clean / __buffers_backend_fsync counter bgwriter statistics (PostgreSQL 17 — from pg_stat_io)
postgresql__bgwriter__checkpoints_timed / __checkpoints_req counter Scheduled/requested checkpoints
postgresql__bgwriter__checkpoints_write_time / __checkpoints_sync_time counter Checkpoint write/sync time, seconds
postgresql__bgwriter__buffers_checkpoint counter Buffers written by checkpoints
postgresql__checkpointer__timed / __req / __write_time / __sync_time / __buffers_written counter Same values under the new names (PostgreSQL 17 pg_stat_checkpointer)
postgresql__replication__role text master or slave
postgresql__replication__wal__current_xlog_location, postgresql__replication__wal__location__current_lsn gauge Current WAL position (primary)
postgresql__replication__wal__last_xlog_receive_location, …__location__receive_lsn gauge Received WAL position (standby)
postgresql__replication__wal__last_xlog_replay_location, …__location__replay_lsn gauge Replayed WAL position (standby)
postgresql__replication__lag__seconds gauge Replica time lag, seconds
postgresql__replication__wal__replay__paused gauge 1 — WAL replay paused
postgresql__replication__lag__bytes slave_ip, application_name gauge Replica lag in WAL bytes (from the primary side)
postgresql__replication__session_age standby_ip, application_name gauge Replication session age, seconds
postgresql__replication__phases__lag__bytes standby_ip, application_name, phase gauge Lag per phase: sending_to_standby, writing_standby_wal, flushing_standby_wal, replaying_standby_wal
postgresql__replication__phases__lag__seconds same gauge Lag per phase in time (PostgreSQL ≥ 10)
postgresql__replication__slots__retained_wal__bytes slot_name, slot_type, database gauge WAL retained by a replication slot
postgresql__wal_archiver__success__count / __fail__count counter Successful/failed WAL archivals
postgresql__wal_archiver__last_success__age gauge Seconds since the last successful archival
postgresql__wal_archiver__unarchived_files__count / __bytes gauge Unarchived WAL segments: count and size
postgresql__wal__size__bytes / postgresql__wal__files__count gauge WAL size and file count on disk

Query statistics (pg_stat_statements, top-50 queries + the ~other aggregate; labels database, user, query) — all rate:

postgresql__query__calls, postgresql__query__rows, postgresql__query__time__total, postgresql__query__time__cpu, postgresql__query__time__disk_read, postgresql__query__time__disk_write, postgresql__query__blocks__hit, postgresql__query__blocks__read, postgresql__query__blocks__written, postgresql__query__blocks__dirtied, postgresql__query__temp_blocks__read, postgresql__query__temp_blocks__written.

Table statistics (top per database; labels database, schema, table) — counters are published with the __rate suffix (rate): postgresql__table__seq_scan__rate, …__idx_scan__rate, …__seq_tuple_read__rate, …__idx_tuple_fetch__rate, …__inserted__rate, …__updated__rate, …__deleted__rate, …__autovacuum_count__rate, …__heap_blks_read__rate, …__heap_blks_hit__rate, …__idx_blks_read__rate, …__idx_blks_hit__rate, …__toast_blks_read__rate, …__toast_blks_hit__rate, …__tidx_blks_read__rate, …__tidx_blks_hit__rate; sizes — gauge: postgresql__table__table_size, postgresql__table__indexes_size.

Settings — the postgresql__settings__<parameter name> family: numeric parameters (e.g. postgresql__settings__max_connections, …__shared_buffers, …__autovacuum_max_workers) — gauge, string ones (e.g. …__wal_level, …__archive_mode, …__data_directory) — text. All recognized pg_settings parameters (~300) are published.

Status metrics: plugin__status (connection), plugin__status__pg_database, …__pg_settings, …__pg_stat_activity, …__pg_stat_replication, …__pg_stat_statements — text describes missing permissions or a missing extension.

PgBouncer (plugin pgbouncer)

Metric Labels Type Description
pgbouncer__total_requests, pgbouncer__total_query_count database counter Processed queries
pgbouncer__total_received / __total_sent database counter Traffic, bytes
pgbouncer__total_query_time / __total_xact_time / __total_wait_time database counter Total query/transaction/wait time, seconds
pgbouncer__total_xact_count database counter Transactions
pgbouncer__databases__pool_size / __reserve_pool / __max_connections database, upstream_addr gauge Pool settings
pgbouncer__databases__paused / __disabled database, upstream_addr gauge 0/1 — database paused/disabled
pgbouncer__databases__upstream_addr database text Upstream server address
pgbouncer__pools__limits__pool_size / __reserve_pool database, user, upstream_addr gauge Active pool limits
pgbouncer__server_connections__max / __avg database, user, state, upstream_addr gauge Server connections per state (active, idle, login, tested, used)
pgbouncer__server_connection_maxwait__max database, user, upstream_addr gauge Maximum wait for a server connection, seconds
pgbouncer__clients__count database, user, state, client_address gauge Client connections per state
pgbouncer__clients__wait__max / __sum database, user, client_address gauge Client wait, seconds
pgbouncer__config__<parameter> gauge Numeric parameters from SHOW CONFIG (e.g. pgbouncer__config__max_client_conn)
plugin__status status Connection status

MySQL / MariaDB (plugin mysql)

Values of SHOW GLOBAL STATUS / SHOW VARIABLES (counter when the source variable is cumulative):

Family Metrics
Connections mysql__connection__max_connections, mysql__connections__aborted_clients, …__aborted_connects, …__connections, mysql__connection__connections_by_state (gauge, label state)
Traffic mysql__traffic__bytes_received, …__bytes_sent
Commands mysql__commands__{select,insert,update,delete,replace,insert_select,replace_select,multi_update,multi_delete,call_procedure}
Prepared statements mysql__prepared_commands__{prepare,execute,dealloc}
Transactions mysql__transaction_commands__{begin,commit,rollback}
Handler API mysql__handler__{commit,delete,read_first,read_key,read_next,read_prev,read_rnd,read_rnd_next,rollback,savepoint,savepoint_rollback,update,write,prepare}
InnoDB buffer pool mysql__innodb__buffer_pool__usage__{data,dirty,free,misc}, mysql__innodb__buffer_pool__operations__{flushed,read_ahead,read_ahead_evicted,read_ahead_rnd,read_ahead_seq,read_requests,reads_from_disk,wait_free,write_requests}, mysql__innodb__buffer_pool__activity__{pages_created,pages_read,pages_written}
InnoDB I/O mysql__innodb__io__{read_bytes,written_bytes,reads_count,writes_count}, mysql__innodb__{fsyncs,current_pending_fsyncs,current_pending_reads,current_pending_writes}, mysql__innodb__doublewrite__{pages_written,writes_count}
InnoDB log mysql__innodb__log__{waits,write_requests,physical_writes,fsyncs,pending_fsyncs,pending_writes,written_bytes}
InnoDB rows/locks mysql__innodb__rows__{inserted,updated,deletes,read}, mysql__innodb__row_lock__{current_waits,total_time,waits}, mysql__innodb__page_size
MyISAM mysql__myisam__key_cache__{blocks_not_flushed,blocks_unused,blocks_used,reads_requests,reads_from_disk,write_requests,writes_to_disk}
Query cache mysql__query_cache__{free_blocks,total_blocks,hits,inserts,lowmem_prunes,not_cached,queries_in_cache}
Temp/sort/select mysql__tmp__created_{disk_tables,files,tables}, mysql__sort__{merge_passes,range,sorted_rows,scan}, mysql__select__{full_join,full_range_join,range,range_check,scan}
Delayed mysql__delayed__{errors,insert_threads,rows_written,not_flushed_rows}
Other mysql__queries_count, mysql__questions_count, mysql__slow_queries, mysql__current_open_files, mysql__current_open_tables, mysql__opened_tables, mysql__locks__{immediate,waited}, mysql__threads__{cached,connected,created,running}, mysql__uptime, mysql__version (text)
Galera/wsrep mysql__wsrep__local_recv_queue__avg, …__local_send_queue__avg, …__flow_control__{received,sent}, …__open_connections, …__open_transactions, …__writesets__received__{count,bytes}, …__writesets__sent__{count,bytes}

Other metrics:

Metric Labels Type Description
mysql__connections__max_transaction_age user, database gauge Maximum open InnoDB transaction age, seconds
mysql_gather_error source, reason, mysql_error_code gauge =1 when collecting a part of the data failed
mysql__replication__slave_io_running / __slave_sql_running master_host, channel_name gauge 0/1 — replication thread state
mysql__replication__seconds_behind_master same gauge Replica lag, seconds
plugin__status status Connection status

Query statistics (performance schema, top-50; labels database, query) — all rate: mysql__query__calls__rate, mysql__query__total_time__rate, mysql__query__time__lock_wait__rate, mysql__query__rows__sent__rate, mysql__query__rows__examined__rate, mysql__query__rows__sorted__rate, mysql__query__tmp_tables_created__rate (+ label table_type = disk | in-memory), mysql__query__full_join__rate, mysql__query__range_check__rate, mysql__query__full_scan__rate, mysql__query__sort__merge__rate.

MongoDB (plugin mongo)

Metrics are collected by passive traffic analysis (sniffer); top-50 queries. Common labels: query, query_type, db, collection.

Metric Type Description
mongo__query__calls__rate rate Queries per second
mongo__query__total_time__rate rate Total query time, seconds per second
mongo__query__traffic__in / __out rate Request/response traffic, bytes/s
mongo__query__times__mean / __min / __max gauge Mean/min/max latency, seconds
mongo__query__times__percentile gauge Latency percentiles (label percentile = 50 | 95 | 99)

Redis (plugin redis)

Metric Labels Type Description
redis__version text Redis version
redis__uptime gauge Uptime, seconds
redis__memory__used / __rss / __lua gauge Memory: allocated/RSS/Lua, bytes
redis__maxmemory gauge Memory limit, bytes
redis__cpu__user / __system process = server | children counter CPU time, seconds
redis__clients__connected / __blocked gauge Connected/blocked clients
redis__clients__connections_received counter Accepted connections
redis__connections__rejected counter Rejected connections
redis__commands__total_calls counter Total commands executed
redis__commands__calls / __seconds command counter Calls and total time per command
redis__cache__evicted_keys counter Evicted keys
redis__keys__hits / __misses counter Key hits/misses
redis__db__keys / __expires db gauge Keys in the database / with TTL
redis__pubsub__channels / __patterns gauge Pub/sub channels and patterns
redis__persistence__rdb__changes_since_last_save gauge Changes since the last RDB save
redis__persistence__rdb__time_since_last_save gauge Seconds since the last save
redis__persistence__rdb__loading_in_progress / __bgsave_in_progress gauge 0/1 — load/save in progress
redis__persistence__aof__enabled / __rewrite_in_progress gauge 0/1 — AOF enabled / rewrite in progress
redis__persistence__aof__current_size / __buffer_length gauge AOF and buffer size, bytes
redis__persistence__aof__pending_bio_fsync / __delayed_fsync gauge Pending fsyncs
redis__replication__connected_slaves gauge Connected replicas (on the master)
redis__replication__last_io_seconds_ago gauge Seconds since the last exchange with the master (on a replica)
redis__replication__sync_in_progress / __sync_left_bytes / __link_down_since_seconds gauge Sync state with the master

Memcached (plugin memcached)

Metric Labels Type Description
memcached__version text Version
memcached__uptime gauge Uptime, seconds
memcached__cpu__user / __system counter CPU time (rusage), seconds
memcached__items__current / __total gauge/counter Items cached now / stored ever
memcached__bytes__used / __max / __malloced gauge Used/limit/allocated, bytes
memcached__connections__current / __total gauge/counter Connections
memcached__evictions / __reclaimed counter Evictions/reclaims of entries
memcached__expired_unfetched / __evicted_unfetched / __crawler_reclaimed / __lrutail_reflocked counter LRU eviction details
memcached__traffic__in / __out counter Traffic, bytes
memcached__threads gauge Threads
memcached__command__requests / __hits / __misses / __badval command (get, set, cas, …) counter Per-command statistics
memcached__slab__chunk_size chunk_size gauge Slab class chunk size
memcached__slab__chunks__free / __used chunk_size gauge Free/used chunks
memcached__slab__bytes__free / __used chunk_size gauge Same in bytes

Elasticsearch (plugin elasticsearch)

Common label for all metrics — cluster. Cluster metrics are published only by the master node.

Metric Extra labels Type Description
elasticsearch__version text Version
elasticsearch__cluster__status status Cluster health: green — value 1, otherwise 0 + status text
elasticsearch__index__settings index, setting = number_of_replicas | number_of_shards gauge Index settings
elasticsearch__shards__count index, shard_type = primary | replica, shard_state = active | initializing | relocating | unassigned gauge Shards by type and state
elasticsearch__index__size index gauge Index size, bytes (top-50 indexes)
elasticsearch__index__docs__count / __deleted index gauge Documents: total / marked deleted
elasticsearch__index__segments__count / __size index gauge Lucene segments: count and memory
elasticsearch__index__operations__count__rate / __time__rate index, operation = get | query | fetch | scroll | merge | index | delete | flush | refresh | warmer | … rate Index operations: frequency and time spent
elasticsearch__index__cache__size / __objects index, cache_type = query | fielddata | request gauge Index caches
elasticsearch__index__cache__hits__rate / __misses__rate / __evictions__rate / __caches__rate index, cache_type rate Cache activity
elasticsearch__index__translog__size / __operations__rate index gauge/rate Translog
elasticsearch__http__current_open__rate rate HTTP connection openings
elasticsearch__transport__rx_count__rate / __rx_bytes__rate / __tx_count__rate / __tx_bytes__rate rate Inter-node traffic
elasticsearch__script__compilations__rate / __cache_evictions__rate rate Script compilations
elasticsearch__thread_pool__threads / __queue / __active thread_pool gauge Thread pools
elasticsearch__thread_pool__rejected__rate / __completed__rate thread_pool rate Rejected/completed tasks
elasticsearch__breaker__limit / __estimated breaker gauge Circuit breakers: limit and estimate, bytes
elasticsearch__breaker__overhead breaker gauge Overhead coefficient
elasticsearch__breaker__tripped__rate breaker rate Breaker trips

RabbitMQ (plugin rabbitmq)

Via the Management API; top-50 queues. Labels: vhost, queue.

Metric Extra labels Type Description
rabbitmq__queue__messages gauge Messages in the queue
rabbitmq__queue__memory gauge Queue memory, bytes
rabbitmq__queue__published__rate rate Publish rate, messages/s
rabbitmq__queue__delivered__rate ack = manual_ack | auto_ack rate Delivery rate to consumers
plugin__status status Errors: management plugin disabled, missing permissions, etc.

Kafka (plugin kafka)

Instance-level labels: broker_id, cluster_id. Partition metrics are published by the partition leader broker.

Metric Labels Type Description
kafka__version text Kafka version
kafka__brokers__count gauge Brokers in the cluster
kafka__topic__partition__count topic gauge Partitions per topic
kafka__topic__partition__size topic, partition gauge Partition size on disk, bytes
kafka__topic__partition__offset__newest / __oldest topic, partition gauge Partition boundary offsets
kafka__topic__partition__replication__isr_count topic, partition gauge In-sync replicas
kafka__topic__partition__replication__replicas_count topic, partition gauge Replica count
kafka__consumer_group__status group status Group state: empty — stable, otherwise text (preparingrebalance, …)
kafka__consumer_group__members__count group gauge Group members
kafka__consumer_group__protocol / __protocol_type group text Group protocol
kafka__consumer_group__topic__partition__offset__committed topic, partition, group, member_host, member_id gauge Committed group offset (lag = newestcommitted)

ZooKeeper (plugin zookeeper)

Metric Labels Type Description
zookeeper__status status Empty — OK; zookeeper cluster is down / zookeeper node in standalone mode
plugin__status status Text please add commands in whitelist when the stat/mntr four-letter commands are disabled
zookeeper__version text Version
zookeeper__role text Node role (leader, follower)
zookeeper__nodes__current / __expected gauge Ensemble nodes: actual / per config
zookeeper__znodes__regular / __ephemerals gauge znode count: regular / ephemeral
zookeeper__watchers gauge Watcher count
zookeeper__leader_elections gauge Epoch number (leader election count)
zookeeper__current_transaction_number__<role> gauge Transaction counter from zxid (<role> — node mode)
zookeeper__client_latency__min / __avg / __max gauge Client request latency, seconds
zookeeper__packets__received / __sent counter Packets
zookeeper__outstanding_requests gauge Requests in flight
zookeeper__outstanding_requests__limit gauge Limit (globalOutstandingLimit)
zookeeper__connection__packets__queued / __received / __sent client_ip rate Per-client-connection activity

Sphinx (plugin sphinx)

Metric Labels Type Description
sphinx__uptime gauge searchd uptime
sphinx__connections__count / __overflows counter Connections / maxed_out rejections
sphinx__agent__connections / __retries counter Distributed search agent connections
sphinx__query__count query_type = local | distributed counter Queries
sphinx__query__total_time query_type counter Total query time
sphinx__query__time query_type, stage = cpu | local | wait | disk_read counter Time per stage
sphinx__query__disk__ops__read / sphinx__query__disk__bytes__read query_type="local" counter Disk reads
sphinx__command__calls command counter Calls per command type (search, update, …)
sphinx__index__documents__count / __bytes index, index_type gauge Documents in an index: count and size
sphinx__index__memory__used / __limit index, index_type gauge Index memory
sphinx__index__disk__used index, index_type gauge Disk used by the index

Resque (plugin resque, configured)

Requires a YAML config with redis_address and prefix.

Metric Labels Type Description
resque__tasks__pending queue gauge Tasks in the queue
resque__tasks__processed worker, queue rate Tasks processed/s
resque__tasks__failed worker, queue rate Failures/s

Web servers and proxies

Nginx (plugin nginx)

Metrics are built by parsing access logs (the format is detected automatically from the nginx config) and by checking TLS certificates from the config. Common log-metric labels: file (log path), log_format.

Metric Extra labels Type Description
nginx__requests__rate method, status, cache_status, url rate Requests/s; url — top-N URLs with normalization, the rest — ~other
nginx__traffic__rate url rate Bytes served per second
nginx__response_time__percentiles percentile = 50 | 75 | 95 | 99 gauge Response time percentiles, seconds
nginx__response_time__histogram method, cache_status, url, levels, level rate Response time histogram (default buckets 0.5 and 1 s)
nginx__upstream_response_time__percentiles percentile gauge Upstream response time percentiles
nginx__upstream_response_time__histogram same as histogram rate Upstream response time histogram
nginx__ssl_certificate__seconds_to_expire certificate_file gauge Seconds until certificate expiry
nginx__ssl_certificate__revoke_status certificate_file gauge OCSP: 1 — good, 0 — revoked
nginx__ssl_certificate__check_status certificate_file status Certificate read/parse error
status access_log or log_format status Configuration problems: unreadable log, unparsed log_format

PHP-FPM (plugin php_fpm)

Via the FCGI status page (pm.status_path). Common label — pool.

Metric Extra labels Type Description
php_fpm__pool__uptime gauge Pool uptime, seconds
php_fpm__pool__accepted_connections counter Accepted connections
php_fpm__pool__listen_queue__current / __limit gauge Request queue: current/limit
php_fpm__pool__processes__count state = active | idle gauge Workers per state
php_fpm__pool__processes__limit_reached counter Times max_children was reached
plugin__status / status pool status Status page disabled / connection error

Envoy (plugin envoy)

Transparent passthrough of the Envoy admin interface metrics (/stats?format=prometheus): names and labels are forwarded as is (e.g. envoy_cluster_upstream_rq_total, envoy_server_live). Only counters and gauges are forwarded; histograms are skipped.

JVM and Cassandra

JVM (plugin jvm)

For every Java process with JMX enabled (without authentication). Instance-level labels: jvm, user.

Metric Labels Type Description
jvm__memory__heap__used / __max gauge Heap: used/max, bytes
jvm__memory__nonheap__used / __max gauge Non-heap memory
jvm__memory__pool__used / __max pool gauge Memory pools (Eden, Old Gen, Metaspace, …)
jvm__gc__counter collector counter Garbage collections per collector
jvm__gc__total_time collector counter GC time, seconds
jvm__threads__current / __daemon gauge Live/daemon threads
jvm__threads__created counter Total threads created
jvm__uptime gauge JVM uptime, milliseconds
status status JMX disabled, please enable it / JMX authentication enabled, please disable it, etc.

Cassandra (branch of the jvm plugin, published as plugin=cassandra)

Extra labels: cluster, datacenter, rack.

Family Metrics Labels
Version cassandra__version (text)
Caches cassandra__row_cache__{max_bytes,used_bytes,hits,requests,entries}, cassandra__key_cache__{…}, cassandra__counter_cache__{…}
Client requests cassandra__client_requests__{count,unavailables,timeouts,failures,total_seconds} operation (Read/Write/…), opt. consistency_level
Commit log cassandra__commitlog__{waiting_on_segment_allocation,pending_tasks,waiting_on_commit,completed_tasks,total_size}
Compactions cassandra__compaction__{completed_tasks,bytes_compacted,completed_compactions,pending_tasks}
Storage cassandra__storage__{bytes_used,exceptions_count,hints_count,hints_in_progress}
CQL cassandra__cql__prepared_statements__{executed,count,evicted}, cassandra__cql__regular_statements__executed
Read repair cassandra__read_repair__{attempted,blocking,background}
Inter-node messaging cassandra__nodes__{messages_received,messages_sent,pending_responses,pending_requests,messages_dropped,timeouts} node, for dropped opt. type = large | small
Dropped messages cassandra__dropped_messages__count message_type
Tables cassandra__table__{waiting_on_memtable_free,disk_space_used,sstables_count,pending_flushes,snapshots_size,compression_ratio,pending_compactions,speculative_retries,mean_row_size,min_row_size,max_row_size}, cassandra__table__row_cache__{hit,miss,hit_out_of_range}, cassandra__table__ops__{count,total_time} (+operation), cassandra__memtable__{size__heap,size__off_heap,switch_count,columns_count}, cassandra__bloom_filter__{false_positives,disk_space_used} keyspace, table

Mail

Mail servers (plugin mail)

Auto-discovery of Exim and Postfix. Queue metrics come from direct polling; mail flow metrics from log parsing. Common label — mta = exim | postfix; log metrics also carry file.

Metric Extra labels Type Description
mail__queue__size queue (postfix: maildrop, hold, incoming, active, deferred; exim: main) gauge Messages in the queue
mail__messages__received rate Messages received/s
mail__messages__delivered postfix: recipient_domain rate Messages delivered/s
mail__messages__deferred rate Deferred/s
mail__messages__failed rate Failed (bounced)/s
mail__delivery_errors response_code (SMTP 4xx/5xx) rate Delivery errors per code
mail__delivery_time__histogram recipient_domain, levels, level rate Postfix delivery time histogram (buckets 5 s/1 min/5 min/30 min)
status status Exim log parser initialization error

The syslog plugin publishes no metrics of its own — it locates log files for Postfix.

Security

File permissions (plugin security_file_permissions)

Metric Labels Type Description
security_file_permissions file_path, mode gauge Existence (0/1) and access mode of critical files: /etc/passwd, /etc/shadow, /etc/group, /etc/gshadow, /etc/hosts, /etc/fstab, etc.
security_file_permissions_errors_total gauge Number of files whose check failed

Custom metrics

Plugins in this section publish metrics whose names are defined by the user or derived from data. Configured via YAML files in config.d/.

StatsD (plugin statsd)

Built-in StatsD server (UDP). The metric name comes from the StatsD packet; name parts of the form key=value or key_is_value are extracted into labels. Label values starting with / are normalized as URL paths; password is replaced with ~hidden.

StatsD type Published metrics
gauge (g) <name> — gauge
counter (c, m) <name>__rate — rate (events/s)
timer (ms, h) <name>__lower, <name>__upper (min/max), <name>__rate (events/s), <name>__sum (sum/s), <name>__mean (average), <name>__percentile with label percentile = 50 | 75 | 90 | 95 | 97 | 99; values in seconds
set (s) not supported

Logparser (plugin logparser)

Turns log lines into metrics per config: regex + a metrics list with types rate, threshold (histogram), percentiles, max, min. The metric name is the name field from the config (as is, dots → __). Labels come from the config; expressions, top-N variables (rare values → ~other), and User-Agent parsing are supported. The file label is added to all metrics.

Metric Type Description
<name from config> rate/gauge User metric; threshold adds levels, level labels; percentiles adds percentile
logparser__lines__succeeded rate Successfully parsed lines/s
logparser__lines__unmatched rate Lines that did not match the regex
logparser__lines__failed rate Lines with value parse errors

SQL queries (plugins mysql_query, postgresql_query, mssql_query, clickhouse_query)

Runs SQL from the config. The metric name is the metric_name field. The value result column is the value; the remaining columns become labels.

Metric Type Description
<metric_name> gauge Values of the result rows
status status Connection/query error or OK

Redis queries (plugin redis_query)

Runs Redis commands from the config. Metric name: <key>__<command> (e.g. GET mykeymykey__get); command arguments go into the param label.

Command execution (plugin execute)

Runs an executable and parses stdout (regex with named groups or JSON). The metric name comes from the config (name) or the JSON name field.

Metric Type Description
<name> gauge/text Values from the command output
status status Exit code / parse error
duplicate_metric_count gauge Number of duplicated metrics in the output

HTTP checks (plugin http)

Performs an HTTP request; metrics are extracted from the response body with regex groups (metrics[].metric in the config).

Metric Type Description
<name from config> gauge Value from the response body
http__request_time gauge Request duration, seconds
status status Request error / response mismatch

Go expvar (plugin expvar)

Reads /debug/vars of a Go application; a fixed set from memstats is published:

go__memory__bytes__alloc, go__memory__bytes__sys, go__memory__total_alloc, go__memory__ops (label operation = lookup | malloc | free), go__memory__heap__alloc, go__memory__heap__sys, go__memory__heap__bytes (label state = used | idle), go__gc__count, go__gc__total_pause, go__gc__max_pause, plus status.

Prometheus exporters (plugin prometheus)

Scrapes any OpenMetrics/Prometheus endpoint and forwards metrics with their original names and labels (no prefixes added). Every metric additionally gets a metric_type label (the exposition type). Histograms are expanded into <name>_count, <name>_sum, <name>_bucket (+le); summaries into <name>_count, <name>_sum, <name> (+quantile). Filtering — metrics_white_list / metrics_black_list by name prefix; the per-endpoint series limit is 5000 (when exceeded, only plugin__status with an error is published).

Service metrics: status (scrape success), plugin__status (label source — config file).

Internal agent metrics

Self-monitoring (plugins heartbeat, okagent_stat)

Metric Labels Type Description
okagent_heartbeat gauge =1; agent liveness flag
okagent_up gauge Agent process uptime, seconds
okagent_uuid text Agent UUID
okagent__version text Agent version
okagent__memstats__alloc / __sys / __heap__* / __stack__* / … gauge Agent Go runtime memory (full runtime.MemStats set)
okagent__gc__time / __count counter Agent GC: total pause time and cycle count
lock_status agent_random_uuid gauge 0 — lock file acquired normally; 1 — a second agent instance is running; 2 — lock check error

Internal counters (via the okagent_stat plugin)

Internal counters are published with the okagent__ prefix. Main families:

Metric Labels Description
okagent__agent__stale_nans, …__plugin_metrics_swapped, …__frequent_packages, …__multipackage_labelset, …__duplicate_labelset, …__plugin_instances, …__duplicate_ls, …__dns_fallback per counter Metric pipeline diagnostics: staleness markers, duplicates, DNS fallbacks
okagent__metric_group__metrics_dropped reason = timestamp | cardinality | invalid, pname Dropped metrics: stale timestamp, cardinality overflow, invalid name
okagent__metric_writer__metricpacks_dropped, …__metrics_dropped, …__buffer_input_metrics, …__switch_proxy, okagent__writer__empty_labels writer, reason Transport: buffer overflow, proxy switchover
okagent__scheduler__ticks_skipped obj_name, func_name Skipped scheduler ticks (a plugin did not finish within the interval)
okagent__process_multiplexer__pushes_skipped subscriber Skipped process list deliveries to subscribers
okagent__statsd__packets__udp, …__errors, …__measurements, …__metrics type, mtype StatsD server diagnostics
okagent__sniffer__errors, …__skipped_packets, …__non_data_packets, …__duplicate_packets error, proto Sniffer diagnostics (MongoDB)
okagent__mongo__errors error MongoDB protocol parse errors
okagent__pg__settings__unknown_variables / __missed_variables instance Unrecognized/missing pg_settings parameters

Internal delivery metrics (plugin prometheus_internal_scraper)

Publishes everything from the agent’s internal Prometheus registry — primarily the remote write delivery subsystem metrics (odarix_core_delivery_*): queue state, sending, refill, transport. Names are forwarded as is.

Status metric conventions

Metric Description
status Status of a plugin instance (configured plugins and some auto-discovered ones)
plugin__status Integration status: empty text (value 1) — collection works; error text (value 0) — what needs fixing
plugin__status__invalid_names Appears when a plugin produced metrics with invalid names (they were dropped; the list is in the text)