The module lifecycle stage: Generally available version

The module has requirements for installation

v1.1.3

Features

  • Added the SetTimestamp transformation to ClusterLogDestination: sets the event time from a log field.
  • Added the keepDeletedFilesOpenedFor parameter to the file source of ClusterLoggingConfig: how long a deleted (rotated) file is kept open for reading.

Changes

  • Optimized the ClusterLogDestination transformations to reduce the agent CPU load.

Bug fixes

  • Fixed handling of special characters ({{ }}, $, non-ASCII) in label values.
  • Fixed addLabels references to label keys with /, such as {{ .pod_labels."app.kubernetes.io/name" }}.
  • Fixed In/NotIn filters for numeric field values: string filter values now match integer fields.
  • Fixed the syslog format of the Socket destination: an unknown or numeric syslog.severity, a missing timestamp or a non-string message no longer make the agent send the message without the syslog header.

Security

  • Fixed various vulnerabilities.

v1.1.2

Changes

  • File patterns in ClusterLoggingConfig are validated. The include list of the file source is now required and must not be empty. The include and exclude patterns are checked by the controller: *, ?, [...], and ** as a whole path component are supported. An invalid pattern is reported with the name of the resource, and the agent configuration is not updated until the resource is fixed.
  • The tenantID of the Loki destination accepts only the characters allowed by Loki, {{ field }} templates and strftime specifiers.
  • The agent no longer requests all labels of the node for every log entry.

Bug fixes

  • Fixed the agent failing to start after a crash (node reboot, OOM kill) when the disk buffer of a destination had been left in an inconsistent state.
  • Fixed the disk buffer maxSize of a destination of 4Gi and more being applied as a smaller value.

Security

  • Fixed various vulnerabilities.

v1.1.1

Features

  • Dropping log entries per destination. The new DropEvent transformation in ClusterLogDestination drops a log entry when all conditions in when are true (the same condition syntax as in AddLabels). It applies only to the destination where it is set, unlike logFilter and labelFilter in ClusterLoggingConfig, which filter for all destinations.
  • Read order and read portion size for log files. The kubernetesPods and file sources in ClusterLoggingConfig and PodLoggingConfig accept two new parameters: maxReadBytes limits how much is read from one file before moving on to the next one, so reads are spread evenly across many active files; oldestFirst makes the agent finish older files before newer ones. Without them the agent reads as before.

Changes

  • The log collection agent is updated to Vector 0.58. Existing ClusterLoggingConfig, PodLoggingConfig and ClusterLogDestination resources keep working without changes. The buffer metrics vector_buffer_events and vector_buffer_byte_size are removed by Vector; custom dashboards and alerts should use vector_buffer_size_events, vector_buffer_size_bytes and vector_buffer_max_size_events instead.

Bug fixes

  • Fixed the event time for the Loki, Splunk, Vector and Socket (GELF codec) destinations: they stored the time a log entry was sent instead of the time it was written. The timestamp field for these destinations is now in UTC instead of the node local timezone; for Splunk the local-time string is kept in the datetime field. Other destinations are not affected.

v1.1.0

Changes

  • Log collection is now managed by a dedicated controller. The configuration of the log collection agent used to be generated by module hooks; it is now built by the log-shipper-controller Deployment that runs in the module namespace on system nodes. The controller watches ClusterLogDestination, ClusterLoggingConfig and PodLoggingConfig resources, updates the agent configuration on every change, and starts or stops the agent DaemonSet depending on whether there is anything to ship. Existing resources and module configuration keep working without changes. The controller exposes its own metrics, which are collected by the platform Prometheus.

Enhancements

  • Logs collected from Kubernetes pods now carry the host_ip label with the IP address of the node the pod runs on. Previously the label was set only for logs read from files.

Security

  • Fixed various vulnerabilities.

v1.0.7

Features

  • Read order and read portion size for log files. The kubernetesPods and file sources in ClusterLoggingConfig and PodLoggingConfig accept two new parameters. maxReadBytes limits how many bytes are read from one file in a pass before the agent moves on to the next file, so reads are spread more evenly across many actively growing files. oldestFirst makes the agent finish older files before reading newer ones. When not set, the agent behaves as before: pod logs are read oldest first, files are read evenly, and a read portion is 2048 bytes. The agent also no longer attaches the container ID, image ID, pod IP list, pod annotations, pod UID and namespace labels to each collected pod log entry. These fields were removed before sending, so the exported logs do not change.

Bug fixes

  • Fixed the event time for the Loki, Splunk, Vector and Socket (GELF codec) destinations. The agent converted the timestamp of every log entry to a string in the node local timezone, and these destinations, which take the event time only from a typed timestamp, stored the time the entry was sent instead of the time it was written. Now the timestamp reaches them unchanged, so the timestamp field in the entry is in UTC instead of the node local timezone; for Splunk the local-time string is kept in the datetime field. Elasticsearch, Logstash, Kafka and Socket with other codecs still receive the timestamp as a string in the node local timezone. The conversion now runs right before the destination transformations, and a timestamp already replaced by a transformation is left as is.

v1.0.6

Features

  • Dropping log entries per destination. The new DropEvent transformation in ClusterLogDestination drops a log entry when all conditions in when are true. The conditions use the same syntax as in AddLabels, and the transformation runs at its position in the list, so it can rely on labels added by earlier transformations. Unlike logFilter and labelFilter in ClusterLoggingConfig, which filter at the source for all destinations, DropEvent applies only to the destination where it is set.

Changes

  • Logs collected from Kubernetes pods now carry the host_ip label with the IP address of the node the pod runs on. Logs collected from files already had it.

v1.0.5

Enhancements

  • The examples documentation now explains how multiple ParseMessage transformations are processed: they are applied to each log entry from top to bottom, and once one of them parses the entry successfully, the rest are skipped. Transformations for specific formats, such as JSON, should therefore come first, and the universal string transformation last.

v1.0.4

Bug fixes

  • Fixed the ReplaceValue transformation with a literal replacement string (without named groups in target). The log collection agent rejected the generated configuration, so the transformation did not work.

v1.0.3

Changes

  • System metrics of the log collection agent now carry the d8_source="dkp" label. It allows platform metrics to be told apart from metrics exposed by user applications.
  • The module namespace is now checked against the restricted Pod Security Standards profile. Violations are reported as warnings only and do not block workloads. The deviations the agent requires — running as root and mounting node paths to read container and node logs and to store its read position — are declared and explained in the SecurityPolicyException resource shipped with the module, so log collection keeps working without any action.
  • The privileges of the agent are reduced as far as log collection permits: the RuntimeDefault seccomp profile is applied, all Linux capabilities are dropped, privilege escalation is denied, and the container root filesystem is read-only.
  • The registry.dockercfg module configuration parameter is removed. Module images are always pulled with the platform registry credentials, so if the parameter is set in the module configuration, remove it before updating.

Security

  • Fixed various vulnerabilities.

v1.0.2

Bug fixes

  • Fixed links between documentation pages and in custom resource descriptions that had been broken after the module was moved to a separate repository.

v1.0.1

Changes

  • Module images are now built using the container-base images and the pm package manager.

Security

  • Fixed various vulnerabilities. The module now ships a VEX document listing the vulnerabilities of its dependencies that cannot affect it, along with the reason for each.

v1.0.0

Changes

  • Resource management in the Static mode is reworked: CPU and memory requests are now set in the resourcesRequests.static.requests section, and CPU and memory limits for the agent can be configured in resourcesRequests.static.limits. Existing module configuration is converted automatically, so no action is required.

Bug fixes

  • Fixed the CPU and memory requests configured for the Static resource management mode not reaching the log collection agent container.