The module lifecycle stage: General Availability

The module has requirements for installation

Release notes for the Prom++ module. Each entry follows a unified structure: a release date, an optional short summary and highlights, and sections ordered as Highlights → New features → Improvements → Fixes → Security updates → Breaking changes → Upgrade notes → Known issues → Docs → Dependencies. Empty sections are omitted.

v3.8.17

Release date: 2026-09-08

Summary: fixed a chunk querier bug that could crash the process, a remote-write config reload issue that could miss a rotated secret, and a shutdown race that could also crash the process; plus a grpc/crypto security update.

Fixes

  • Fixed a chunk querier bug that could crash the process when running out of chunks to recode instead of correctly stopping the scan.
  • Fixed remote-write config reload not detecting a changed secret (for example a rotated authorization token), which left the destination sending with the old, invalid credential.
  • Fixed a race between shutdown and a head-rotation trigger that could panic the process.
  • The /debug/jemalloc profiling endpoint now serializes concurrent requests and returns 404 when jemalloc profiling isn’t actually enabled, instead of failing unpredictably.

Security updates

  • Updated google.golang.org/grpc and golang.org/x/crypto with upstream security fixes.

Breaking changes

  • The experimental pp_protocol remote-write WebSocket and refill endpoints are temporarily disabled (return 404) while the protocol is reworked.

v3.8.16

Release date: 2026-09-01

Summary: disabled the shrink_shard_copier feature flag by default after it caused metrics received via Remote Write to occasionally disappear.

Fixes

  • Disabled the shrink_shard_copier feature by default: it could cause metrics received via Remote Write to occasionally disappear.

v3.8.15

Release date: 2026-08-27

Summary: MADV_RANDOM for block file mappings is now opt-in to avoid extra disk IOPS on some setups, and the scraper now validates UTF-8 across the whole scrape payload by default.

Fixes

  • Disabled MADV_RANDOM for block index/chunk file memory mappings by default; on some storage backends it could increase disk IOPS. Re-enable it with the enable_madvise_random feature flag.

Improvements

  • The scraper now validates UTF-8 across the entire scraped payload by default, catching invalid characters anywhere in it instead of only in specific fields as before. The old behavior can be restored with the disable_scraper_full_utf8 feature flag.

v3.8.14

Release date: 2026-08-21

Summary: fixed crashes when converting a head into a historical block, removed a possible state error when processing out-of-order samples that could lead to a SIGSEGV, plus dependency security updates.

Fixes

  • Fixed an issue when converting a head into a historical block: under high churn, an out-of-bounds array access could occur, causing a crash.
  • Fixed a possible state error when processing out-of-order samples.

Security updates

  • Updated the Go module golang.org/x/mod and the web UI package nanoid with upstream security fixes.

v3.8.13

Release date: 2026-08-18

Summary: a module-only release that restores the weight field in the module definition, required to install the module on older DKP versions.

Fixes

  • Restored the weight field in module.yaml, dropped in v3.7.10: without it the module cannot be installed on older DKP versions.

v3.8.12

Release date: 2026-08-14

Summary: blocks can now be grouped by segment before compacting in the block-manager storage scheme, remote-write retries are now traceable, and a new flag lets you force remote write onto HTTP/1.1.

Improvements

  • Reworked the block-manager storage scheme’s compactor: blocks can now be grouped by time range and segment before compaction planning, so blocks from different segments are never combined.
  • Remote-write retries now carry a stable idempotency key and the real retry attempt number; failed-send logs report the destination, attempt, duration and message size.
  • Lower page-cache footprint for block files: their memory mappings no longer trigger the kernel’s sequential readahead.

Fixes

  • Fixed a potential race condition in the series index’s internal reference counting.

v3.8.11

Release date: 2026-08-07

Summary: scrapes over 4 GiB are no longer dropped, lower query memory, faster scrape parsing, and better behaviour with a large number of rule groups.

Highlights

  • Scrapes larger than 4 GiB are no longer dropped in full: neither metrics nor metadata were stored before.
  • Better performance with a large number of rule groups (1000+), where the memory allocator itself used to slow down.

Improvements

  • Lower memory use for queries with many label matchers.
  • Faster scrape parsing.
  • Short-lived storages no longer create memory arenas that brought them no benefit and only slowed the allocator down.
  • C++ memory is now reclaimed earlier under growing load.

Fixes

  • Fixed scraping of very large responses: markup offsets are now 64-bit, so a /federate response over 4 GiB is parsed and appended correctly instead of being lost entirely.

Security updates

  • Updated the web UI packages postcss and sanitize-html with upstream security fixes.

Docs

  • Fixed broken links in the module documentation and added the configuration reference pages.

v3.8.10

Release date: 2026-07-31

Summary: lower query-path memory usage, crash fixes for the active query log and DataStorage metrics path, and faster WAL segment reads.

Highlights

  • Query results no longer keep large off-heap buffers alive until GC, reducing memory growth under query load.
  • Fixed SIGBUS crashes related to the active query log on some filesystems.

Improvements

  • Lower memory usage when executing queries by releasing query result buffers as soon as the response is built.
  • Faster WAL segment reads through pooled buffer reuse.

Fixes

  • Fixed SIGBUS in ActiveQueryTracker when the active query log was created as a sparse file.
  • Fixed a race condition during static object initialization that could cause startup errors on older processors.

Security updates

  • Updated Go dependencies (github.com/klauspost/compress, go.opentelemetry.io/otel) and the web UI postcss package with upstream security fixes.

v3.8.9

Release date: 2026-07-24

Summary: correct CPU feature detection for better out-of-the-box performance, faster scrape parsing, lower ingestion memory, and stability and security fixes.

Highlights

  • Fixed CPU feature detection so the engine selects the right instruction set on modern CPUs instead of always falling back to the baseline, improving performance out of the box.
  • Faster scrape parsing.

Improvements

  • Faster scrape parsing through a reworked scraper write-ahead-log encoding.
  • Lower peak memory during ingestion: the transaction snapshot is now released as soon as the transaction is committed.
  • The module image now uses a statically-linked curl, reducing runtime dependencies.
  • Module release notes are now shipped with the release and shown in the Deckhouse UI (unified changelog format).

Fixes

  • Fixed CPU feature detection that always selected baseline instruction bindings even on modern CPUs, degrading performance.

Security updates

  • Updated Go dependencies (golang.org/x/text, golang.org/x/net, google.golang.org/grpc) and web UI packages (body-parser, webpack-dev-server, immutable) with upstream security fixes.

v3.8.8

Release date: 2026-07-17

Summary: incident-recovery tooling and observability improvements, lower memory usage when writing blocks, and stability and security fixes.

Highlights

  • Added a persist-head command to prompptool for recovering an individual head to TSDB blocks during incident handling.
  • Reduced memory consumption when writing blocks over wide or sparse time ranges.
  • Added on-demand jemalloc heap profiling over HTTP for troubleshooting memory usage.

Improvements

  • The block writer now skips blocks that already fall entirely outside the retention period, avoiding wasted disk writes.
  • Added a /debug/jemalloc HTTP endpoint to capture a jemalloc heap profile on demand (requires profiling enabled at startup).
  • Added a metric for the total size of unknown objects in local storage, making disk leftovers visible.
  • Reduced memory consumption when writing blocks over wide or sparse time ranges.

Fixes

  • Fixed a rare crash (heap-buffer-overflow) in the outdated chunk merger.
  • Leftover temporary block directories from an interrupted compaction or persist are now cleaned up on startup.

Security updates

  • Updated the Go toolchain, addressing known security vulnerabilities.

v3.8.7

Release date: 2026-07-16

Fixes

  • Removed internal C++ metrics registration because it led to a memory leak.

v3.8.6

Release date: 2026-07-15

Fixes

  • Disabled the internal C++ metrics collector by default to prevent a rare crash (use-after-free) that could occur during metrics collection.

v3.8.5

Release date: 2026-07-09

Improvements

  • Reworked historical block storage with a new block-manager engine, improving compaction planning and adding block layout diagnostics metrics.
  • Core dumps are no longer written on crash, preventing unexpected disk usage in constrained environments such as Kubernetes.
  • Added metrics for observing the internal data storage.
  • Improved performance of label processing and index writing.

Security updates

  • Updated the build toolchain and dependencies, addressing known security vulnerabilities.

v3.8.4

Release date: 2026-06-11

Improvements

  • Improved instant query performance.
  • Reduced memory consumption during head rotation and when processing high-cardinality metrics through the federation API.

Security updates

  • Updated base images and security-related components.

v3.8.3

Release date: 2026-06-02

Fixes

  • Fixed a chunk encoder issue that could, in rare cases, lead to historical data corruption.
  • Updated PromQL range query behavior to match the current Prometheus implementation. As a result, the output of the rate, sum_over_time, and count_over_time functions may differ slightly from previous versions.

Security updates

  • Fixed a vulnerability that could lead to memory exhaustion when processing remote read queries.
  • Fixed multiple XSS vulnerabilities in the web interface.

v3.8.1

Release date: 2026-05-15

Improvements

  • Improved data processing performance and reduced memory consumption when executing queries in sharded environments.

Fixes

  • Fixed WAL processing during shard rotation.

Security updates

  • Updated dependencies and addressed known security vulnerabilities.

v3.8.0

Release date: 2026-04-22

Improvements

  • Added metrics for analyzing mutex and semaphore contention in the Go runtime.
  • Added a metric for jemalloc resident memory usage.
  • Improved resource cleanup during WAL rotation.

v3.7.11

Release date: 2026-05-15

Improvements

  • Added metrics for monitoring jemalloc arena reuse efficiency.

v3.7.10

Release date: 2026-04-22

Improvements

  • Added support for WAL v2 format in remote write.

Fixes

  • Fixed false positives for the PrometheusRemoteWriteDesiredShards alert.
  • Resolved potential issues affecting remote write under high load.

v3.7.9

Release date: 2026-03-30

New features

  • Added new PromQL functions: op_defined, op_replace_nan, op_smoothie, op_zero_if_none, and op_top.

Improvements

  • Improved remote write performance.
  • Added a data age limit for remote write to prevent WAL file accumulation when external systems are unavailable.

Fixes

  • Fixed ingestion metric calculations.
  • Resolved increasing memory consumption when using custom jemalloc arenas.

Security updates

  • Updated dependencies and addressed known security vulnerabilities.

v3.7.6

Release date: 2026-02-20

Improvements

  • Improved rule evaluation and remote write performance.

Security updates

  • Updated security-related dependencies.

v3.7.5

Release date: 2026-01-23

Improvements

  • Optimized data ingestion and processing workflows.

v3.7.4

Release date: 2026-01-12

Improvements

  • Added support for splitting high-cardinality metrics when using the federation API, reducing peak memory consumption.

v3.7.2

Release date: 2025-12-08

Improvements

  • Improved rule evaluation and query execution performance.

Fixes

  • Fixed stability issues that could cause intermittent failures.
  • Fixed handling of empty labels in OTLP Protobuf.

v3.7.0

Release date: 2025-11-05

Improvements

  • Data copying during rotation is now performed in the background, reducing the impact of rotation on system responsiveness.
  • Improved query execution and rule evaluation performance.

Fixes

  • Fixed handling of StaleNaN values during data rotation.

v3.6.5

Release date: 2025-10-22

Improvements

  • Added a mechanism for offloading infrequently used time series to disk, significantly reducing memory consumption in large-scale deployments.
  • Reduced memory consumption during metrics collection and processing.

Fixes

  • Fixed regular expression handling in matchers.
  • Improved resilience to corrupted WAL files and invalid data.
  • Fixed handling of StaleNaN values for backfilled data.

v3.5.0

Release date: 2025-08-25

Improvements

  • Upgraded the underlying Prometheus version to 2.55.1.
  • Added support for the Prometheus 3.x storage format.

Security updates

  • Updated dependencies and addressed known security vulnerabilities.