The security-events-manager module: FAQ

Do I need this module?

Use it if you want to:

• Collect security-relevant events (audit, runtime, application) in a single canonical format.
• Route events to Loki, Elasticsearch, Splunk, or other SIEM/storage based on source and severity.
• Let other teams define event extraction (Shipper) while you own the registry (SecurityEventDefinition) and destinations.

If you only need generic log collection without security-event validation or routing, the log-shipper module alone may be enough.

How do I enable delivery to Loki?

1. Create a ClusterSecurityEventDestination with spec.type: Loki and your Loki endpoint/auth/tls. More details about the configuration and examples can be found in CR and Examples.
2. Create a ClusterSecurityEventConfig that references this destination in spec.destinations and defines which sources are allowed via spec.enabledSources or spec.enabledSourcesMasks.
3. Ensure you have at least one SecurityEventDefinition so the controller creates the ClusterLogDestination for the gateway; and at least one Shipper that produces events matching your config.

Events are not reaching the destination. What should I check?

• Source allowlist — in ClusterSecurityEventConfig you must list the source in enabledSources or enabledSourcesMasks. Source format —
  • clusterSecurityEventShipper/<name of ClusterSecurityEventShipper>/<source> (e.g. clusterSecurityEventShipper/kube-audit/kube-apiserver)
  • podSecurityEventShipper/<namespace>/<name of PodSecurityEventShipper>/<source> (e.g. podSecurityEventShipper/my-ns/audit/my-component)
• Masks — if using enabledSourcesMasks, * matches any substring including /. You cannot set both enabledSources and enabledSourcesMasks.
• Severity — defaultSeverityThreshold filters by .event.severity; events below the threshold for that config are not sent to the corresponding destinations.
• Registry — if SecurityEventDefinition is used, events whose (event.code, source.component) pair is not in the registry are dropped at the gateway. Ensure the Shipper’s produces[].eventCode and source match a SecurityEventDefinition.

More details about checking gateway config and logs can be found in Advanced usage.

What is the difference between PodSecurityEventShipper and ClusterSecurityEventShipper?

• PodSecurityEventShipper is namespaced. It reads logs from pods in that namespace (KubernetesPods source). Use it for application or namespace-scoped audit/runtime events.
• ClusterSecurityEventShipper is cluster-scoped. It can read from pods in selected namespaces (KubernetesPods) or from files on nodes (File). Use it for cluster-wide audit logs, node-level logs, or shared parsers.

Both produce events that are identified by eventCode and source; routing in ClusterSecurityEventConfig uses the same source format for both (with the appropriate prefix podSecurityEventShipper/... or clusterSecurityEventShipper/...).

How do I add a new event type?

1. Create or reuse a SecurityEventDefinition with the desired code, severity, category, source, and fields (required optional).
2. Create or update a PodSecurityEventShipper or ClusterSecurityEventShipper that has a spec item with produces[].eventCode matching that definition and source matching the definition’s source. Configure input, parser or parserRef, and optionally produces[].transform / enrich.
3. Ensure a ClusterSecurityEventConfig allows this source (in enabledSources or enabledSourcesMasks) and references the destination where you want events stored.

severity and category are defined only in SecurityEventDefinition, not in Shipper.

How do I send events to an external system (OpenSearch, Elasticsearch, Splunk, etc.)?

The security-events-manager module supports sending events to several types of external storage and analytics systems: Loki, Elasticsearch, Kafka, Splunk (HEC), Vector, File, Socket, and Console. Let’s look at the setup using OpenSearch as an example.

OpenSearch is backward-compatible with the Elasticsearch API, so the Elasticsearch destination type and the _bulk/_search endpoints are used for sending events.

To configure event delivery:

1. configure an event destination — the ClusterSecurityEventDestination resource;
2. configure delivery rules — the ClusterSecurityEventConfig resource;
3. ensure the cluster has at least one ClusterSecurityEventShipper or PodSecurityEventShipper producing events from the allowed sources.

Configure the destination

The destination is configured through the ClusterSecurityEventDestination resource.

For OpenSearch, the endpoint field specifies the HTTPS address of the API (via Ingress — https://opensearch-api.example.com, or internal — https://opensearch-cluster-master:9200). Authentication strategies are None, Bearer, or Basic. Server certificate verification is configured in the tls block. The index field specifies the index name or index template, for example security-events-%Y.%m.%d for daily index rotation.

Example:

apiVersion: security.deckhouse.io/v1alpha1
kind: ClusterSecurityEventDestination
metadata:
  name: opensearch-external
spec:
  type: Elasticsearch
  elasticsearch:
    endpoint: https://opensearch-api.example.com
    # Daily index rotation. Remove the suffix for a single index.
    index: security-events-%Y.%m.%d
    auth:
      # None | Bearer | Basic.
      strategy: Basic
      username: admin
      password: P@ssw0rd-CHANGE-ME
    tls:
      # For a public Let's Encrypt certificate via Ingress.
      # If the gateway has no system CA, verification can be disabled.
      # For production — set ca (Base64 PEM) and enable verification.
      verifyCertificate: false
      verifyHostname: false
      # ca: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0t...

The auth.password and auth.token fields are stored in the resource in plain text. For production, it is recommended to create a dedicated user in OpenSearch with write-only permissions to the target index.

Instead of storing credentials in the CR spec, you can reference them from a Secret using passwordSecretRef/tokenSecretRef. The Secret must be in the d8-security-events-manager namespace with key value in data and labeled security-events-manager.deckhouse.io/credential-secret: "true". Inline credential fields and their *SecretRef counterparts are mutually exclusive.

Configure delivery rules

Delivery rules are configured through the ClusterSecurityEventConfig resource.

The destinations field specifies the destination name set in metadata.name of the ClusterSecurityEventDestination resource. The enabledSources (or enabledSourcesMasks) field lists the sources whose events are sent to the destination. The defaultSeverityThreshold field sets the minimum severity level — events below the specified level are discarded.

Example:

apiVersion: security.deckhouse.io/v1alpha1
kind: ClusterSecurityEventConfig
metadata:
  name: opensearch-routing
spec:
  # Low | Medium | High — discards events below the threshold.
  defaultSeverityThreshold: Low
  enabledSourcesMasks:
    - clusterSecurityEventShipper/* # All cluster-scoped sources.
    - podSecurityEventShipper/* # All namespace-scoped sources.
  destinations:
    - opensearch-external # Name of the ClusterSecurityEventDestination resource.

Multiple ClusterSecurityEventDestination resources can be created and listed in the destinations of a single ClusterSecurityEventConfig — selected events will be sent to all destinations simultaneously. To split the stream by severity, use multiple ClusterSecurityEventConfig resources with different defaultSeverityThreshold values and non-overlapping destinations or enabledSources.

How do I collect sshd events (successful and failed SSH connections)?

Let’s look at collection using Ubuntu and the /var/log/auth.log file as an example. The File source type reads files on nodes via the log-shipper DaemonSet, so the path must be readable on every node.

Example log lines:

2026-07-28T05:41:36.390129+00:00 master-0 sshd[1127780]: Accepted publickey for a.dyakonov from 10.12.0.1 port 60280 ssh2: RSA SHA256:...
2026-07-28T05:53:10.932848+00:00 master-0 sshd[1194201]: Invalid user test from 127.0.0.1 port 35176

Two resources are required:

1. SecurityEventDefinition — event definitions (successful and failed connection).
2. ClusterSecurityEventShipper — pipeline: file source, regex parser for the syslog line, produces rules for detection.

After creating the resources, do not forget to allow the source clusterSecurityEventShipper/sshd/sshd in ClusterSecurityEventConfig (enabledSources or enabledSourcesMasks), otherwise events will be dropped at the gateway.

Step 1. Event definitions

severity and category are defined only in SecurityEventDefinition. The source field must match the source value in the Shipper.

The description and descriptionRu fields hold the short event name in English and Russian respectively. The metadata.docs.desc block (with ru and en sub-keys) provides a longer description shown in the Deckhouse Console UI — see the Russian documentation for the full bilingual example.

apiVersion: security.deckhouse.io/v1alpha1
kind: SecurityEventDefinition
metadata:
  name: ssh-login-success
spec:
  code: SSH_LOGIN_SUCCESS
  category: Auth
  severity: Low
  description: "Successful SSH login"
  descriptionRu: "Successful SSH login (RU)"
  # metadata.docs.desc — description for displaying extra information in the Deckhouse Console UI.
  metadata:
    docs:
      desc:
        en: |
          Detected a successful SSH login to the node. The event records user authentication via public key or password.
  source: sshd
  fields:
    - name: actor.id
      required: true
    - name: metadata.extra.src_ip
    - name: metadata.extra.src_port
    - name: metadata.extra.auth_method
---
apiVersion: security.deckhouse.io/v1alpha1
kind: SecurityEventDefinition
metadata:
  name: ssh-login-failed
spec:
  code: SSH_LOGIN_FAILED
  category: Auth
  severity: Medium
  description: "Failed SSH login attempt"
  descriptionRu: "Failed SSH login attempt (RU)"
  # metadata.docs.desc — description for displaying extra information in the Deckhouse Console UI.
  metadata:
    docs:
      desc:
        en: |
          Detected a failed SSH login attempt to the node. The event records a login with an invalid user, wrong key, or rejected authentication.
  source: sshd
  fields:
    - name: actor.id
      required: true
    - name: metadata.extra.src_ip
    - name: metadata.extra.src_port

Step 2. ClusterSecurityEventShipper

The parser uses type Regex with multiple patterns: the first successfully matched pattern wins, and named capture groups are written into .parsed_data. The extract rules (on field message) filter lines on the log-shipper side — only lines matching the sshd events reach the gateway. The transform rules in producesDefaults map parsed fields into the canonical SecurityEvent, and enrich sets source.component.

apiVersion: security.deckhouse.io/v1alpha1
kind: ClusterSecurityEventShipper
metadata:
  name: sshd
spec:
  - source: sshd
    input:
      type: File
      files:
        - /var/log/auth.log
    parser:
      - name: file
        parser:
          type: Regex
          regex:
            patterns:
              # Accepted <method> for <user> from <ip> port <port> ssh2
              - '^(?P<syslog_timestamp>\S+)\s+(?P<hostname>\S+)\s+sshd\[(?P<pid>\d+)\]:\s+Accepted\s+(?P<auth_method>\S+)\s+for\s+(?P<user>\S+)\s+from\s+(?P<src_ip>\S+)\s+port\s+(?P<src_port>\d+).*$'
              # Invalid user <user> from <ip> port <port>
              - '^(?P<syslog_timestamp>\S+)\s+(?P<hostname>\S+)\s+sshd\[(?P<pid>\d+)\]:\s+Invalid\s+user\s+(?P<user>\S+)\s+from\s+(?P<src_ip>\S+)\s+port\s+(?P<src_port>\d+).*$'
              # Failed <method> for [invalid user ]<user> from <ip> port <port>
              - '^(?P<syslog_timestamp>\S+)\s+(?P<hostname>\S+)\s+sshd\[(?P<pid>\d+)\]:\s+Failed\s+(?P<auth_method>\S+)\s+for\s+(?:invalid\s+user\s+)?(?P<user>\S+)\s+from\s+(?P<src_ip>\S+)\s+port\s+(?P<src_port>\d+).*$'
        fields:
          - name: src_port
            type: Int
          - name: pid
            type: Int
    producesDefaults:
      transform:
        - key: timestamp
          value: syslog_timestamp
        - key: actor.id
          value: user
        - key: metadata.extra.src_ip
          value: src_ip
        - key: metadata.extra.src_port
          value: src_port
        - key: metadata.extra.hostname
          value: hostname
        - key: metadata.extra.auth_method
          value: auth_method
        - key: metadata.extra.pid
          value: pid
      enrich:
        - target: source.component
          source: Static
          value: sshd
    produces:
      - eventCode: SSH_LOGIN_SUCCESS
        extract:
          field: message
          operator: Regex
          values:
            - '.*sshd\[.*Accepted\s+\S+\s+for\s+\S+\s+from\s+\S+\s+port\s+\d+.*'
      - eventCode: SSH_LOGIN_FAILED
        extract:
          field: message
          operator: Regex
          values:
            - '.*sshd\[.*Invalid\s+user\s+\S+\s+from\s+\S+\s+port\s+\d+.*'
            - '.*sshd\[.*Failed\s+\S+\s+for\s+(?:invalid\s+user\s+)?\S+\s+from\s+\S+\s+port\s+\d+.*'

Buffer configuration

What happens when a destination is down?

When a destination (e.g., Loki, Elasticsearch) is temporarily unreachable, the gateway buffers events. The behavior depends on the buffer settings:

• With whenFull: Block (default): the gateway applies backpressure — the pipeline slows down but NO events are lost. The log-shipper agents also buffer on their side, cascading backpressure to sources.
• With whenFull: DropNewest: new events are discarded when the buffer is full. Data loss occurs.

The D8SecurityEventsManagerGatewayBufferHighUsage alert fires when buffer usage exceeds 80% for 10 minutes, and D8SecurityEventsManagerGatewayBufferEventsDropped fires when events are being dropped.

Why is my gateway slow?

If the gateway appears slow, it may be applying backpressure due to a full buffer (whenFull: Block). Check:

1. Whether any destinations are unreachable (check kubectl get clustersecurityeventdestination -o wide and destination endpoint health).
2. Buffer usage metrics on the Grafana dashboard.
3. Gateway logs for sink errors: kubectl -n d8-security-events-manager logs deploy/gateway -c vector --tail=500 | grep -E "buffer|backpressure|sink|error"

The slowness is by design — it prevents data loss. Once the destination recovers, the pipeline resumes normal speed.

How much disk does the buffer use?

With the default Disk buffer type:

• Gateway sinks: up to gateway.buffer.maxSize (default 512Mi) per replica.
• Log-shipper agents: up to gateway.logShipperBuffer.maxSize (default 257Mi) per node.
• The gateway vector-data emptyDir has sizeLimit set to match maxSize when Disk buffer is used.

Total gateway disk usage = gateway.replicas × gateway.buffer.maxSize.

Block vs DropNewest — which should I use?

• Block (default): Use in production. The pipeline slows down under backpressure but no events are lost. This is the correct choice for security events.
• DropNewest: Use only in test/dev environments where data loss is acceptable and you don’t want backpressure to affect the log-shipper agents.

How to calculate buffer size?

Buffer duration (hours) = maxSize_bytes / (average_event_size_bytes × events_per_second × 3600)

With defaults (512Mi = 536870912 bytes, 500 bytes/event, 10 ev/s): ~30 hours of buffering. With high rate (10,000 ev/s, 1 KB events): ~9 minutes — increase maxSize accordingly.

See Advanced usage — Buffer tuning for more details.