ClusterLogDestination

Scope: Cluster
Version: v1alpha1

CustomResource for storage in log-pipeline.

Each CR ClusterLogDestination descride one log storage, which you can use in many log sources.

metadata.name — is an upstream name, whick you should use in CR ClusterLoggingConfig.

  • spec (object)

    Required value.

    • elasticsearch (object)
      • auth (object)
        • awsAccessKey (string)

          Base64 encoded AWS ACCESS_KEY.

        • awsAssumeRole (string)

          The ARN of an IAM role to assume at startup.

        • awsRegion (string)

          AWS region for aws authentication.

        • awsSecretKey (string)

          Base64 encoded AWS SECRET_KEY.

        • password (string)

          Base64 encoded basic authentication password.

        • strategy (string)

          The authentication strategy to use.

          Default: "Basic"

          Allowed values: Basic, AWS

        • user (string)

          The basic authentication user name.

      • endpoint (string)

        The base URL of the Elasticsearch instance.

        Required value.

      • index (string)

        Index name to write events to.

      • pipeline (string)

        Name of the pipeline to apply.

      • tls (object)

        Configures the TLS options for outgoing connections.

        • caFile (string)

          Base64 encoded CA certificate in PEM format.

        • clientCrt (object)

          Configures client certificate for outgoing connections.

          • crtFile (string)

            Base64 encoded certificate in PEM format. keyFile Must also be set.

            Required value.

          • keyFile (string)

            Base64 encoded private key in PEM format (PKCS#8). If this is set, crtFile must also be set.

            Required value.

          • keyPass (string)

            Base64 encoded pass phrase used to unlock the encrypted key file.

        • verifyHostname (boolean)

          Vector will validate the TLS certificate of the remote host.

          Default: true

    • extraLabels (object)

      A set of labels that will be attached to each batch of events. You can use simple templating here: {{ app }}. There are some reserved keys:

      • parsed_data
      • pod
      • pod_labels
      • pod_ip
      • namespace
      • image
      • container
      • node
      • pod_owner More about field path notation

      Example:

      extraLabels:
  forwarder: vector
  key: value
  app_info: "{{ app }}"
  array_member: "{{ array[0] }}"
  symbol_escating_value: "{{ pay\.day }}"
    • logstash (object)
      • endpoint (string)

        The base URL of the Loki instance.

        Required value.

      • tls (object)

        Configures the TLS options for outgoing connections.

        • caFile (string)

          Base64 encoded CA certificate in PEM format.

        • clientCrt (object)

          Configures client certificate for outgoing connections.

          • crtFile (string)

            Base64 encoded certificate in PEM format. keyFile Must also be set.

            Required value.

          • keyFile (string)

            Base64 encoded private key in PEM format (PKCS#8). If this is set, crtFile must also be set.

            Required value.

          • keyPass (string)

            Base64 encoded pass phrase used to unlock the encrypted key file.

        • verifyCertificate (boolean)

          Vector will validate the configured remote host name against the remote host’s TLS certificate.

          Default: true

        • verifyHostname (boolean)

          Vector will validate the TLS certificate of the remote host.

          Default: true

    • loki (object)
      • auth (object)
        • password (string)

          Base64 encoded basic authentication password.

        • strategy (string)

          The authentication strategy to use.

          Default: "Basic"

          Allowed values: Basic, Bearer

        • token (string)

          The token to use for bearer authentication.

        • user (string)

          The basic authentication user name.

      • endpoint (string)

        The base URL of the Loki instance.

        Required value.

      • tls (object)

        Configures the TLS options for outgoing connections.

        • caFile (string)

          Base64 encoded CA certificate in PEM format.

        • clientCrt (object)

          Configures client certificate for outgoing connections.

          • crtFile (string)

            Base64 encoded certificate in PEM format. keyFile Must also be set.

            Required value.

          • keyFile (string)

            Base64 encoded private key in PEM format (PKCS#8). If this is set, crtFile must also be set.

            Required value.

          • keyPass (string)

            Base64 encoded pass phrase used to unlock the encrypted key file.

        • verifyHostname (boolean)

          Vector will validate the TLS certificate of the remote host.

          Default: true

    • type (string)

      Set on of possible output destinations.

      Allowed values: Loki, Elasticsearch, Logstash

      Required value.

ClusterLoggingConfig

Scope: Cluster
Version: v1alpha1

CustomResource for source in log-pipeline.

Each CustomResource ClusterLoggingConfig describes rules for log fetching from cluster.

  • spec (object)

    Required value.

    • destinationRefs (array of strings)

      Array of ClusterLogDestination CustomResource names which this source will output with. Fields with float or boolean values will be converted to string.

      Required value.

    • file (object)
      • exclude (array of strings)

        Array of file patterns to exclude.

        Example: "/var/log/nginx/error.log", "/var/log/audit.log"

      • include (array of strings)

        Array of file patterns to include.

        Example: "/var/log/*.log", "/var/log/nginx/*.log"

      • lineDelimiter (string)

        String sequence used to separate one file line from another.

        Example: "\r\n"

    • kubernetesPods (object)

      Default: {}

      • labelSelector (object)

        Specifies the label selector to filter Pods with.

        You can get more into here.

        • matchExpressions (array of objects)

          List of label expressions for Pods.

          Example:

          matchExpressions:
- key: tier
  operator": In
  values:
  - production
  - staging
          • key (string)

            Required value.

          • operator (string)

            Allowed values: In, NotIn, Exists, DoesNotExist

            Required value.

          • values (array of strings)
        • matchLabels (object)

          List of labels which Pod should have.

          Example: ["foo","bar"], ["baz","whooo"]

      • namespaceSelector (object)

        Specifies the Namespace selector to filter Pods with.

        • matchNames (array of strings)
    • logFilter (array of objects)

      List of filter for logs. Only matched lines would be stored to log destination.

      Example:

      logFilter:
- field: tier
  operator: Exists
- field: tier
  operator: In
  values:
  - production
  - staging
- field: foo
  operator: NotIn
  values: 
  - dev
  - 42
  - "true"
  - "3.14"
- field: bar
  operator: Regex
  values:
  - ^abc
  - ^\d.+$
      • field (string)

        Field name for filtering.

        Required value.

      • operator (string)

        Operator for log field comparations:

        • In operator find substring in string
        • NotIn operator is negative version of In operator
        • Regex operator is trying to match regexp over field, only log events with matching fields will pass
        • NotRegex operator is negative version of Regex operator, log events without fields or with not matched fields will pass
        • Exists operator drop log event if it contains some fields
        • DoesNotExist operator drop log event if it does not contain some fields

        Allowed values: In, NotIn, Regex, NotRegex, Exists, DoesNotExist

        Required value.

      • values (array)

        Array of ClusterLogDestination CustomResource names which this source will output with. Fields with float or boolean values will be converted to strings during comparation.

    • type (string)

      Set on of possible input sources.

      KubernetesPods source reads logs from Kubernetes Pods.

      File source reads local file from node filesystem.

      Allowed values: KubernetesPods, File

      Required value.

PodLoggingConfig

Scope: Namespaced
Version: v1alpha1

CustomResource for namespaced Kubernetes source in log-pipeline.

Each CustomResource PodLoggingConfig describes rules for log fetching from specified Namespace.

  • spec (object)

    Required value.

    • clusterDestinationRefs (array of strings)

      Array of ClusterLogDestination CustomResource names which this source will output with.

      Required value.

    • labelSelector (object)

      Specifies the label selector to filter Pods with.

      You can get more into here.

      • matchExpressions (array of objects)

        List of label expressions for Pods.

        Example:

        matchExpressions:
- key: tier
  operator": In
  values:
  - production
  - staging
        • key (string)

          Required value.

        • operator (string)

          Allowed values: In, NotIn, Exists, DoesNotExist

          Required value.

        • values (array of strings)
      • matchLabels (object)

        List of labels which Pod should have.

        Example: ["foo","bar"], ["baz","whooo"]

    • logFilter (array of objects)

      List of filter for logs. Only matched lines would be stored to log destination.

      Example:

      logFilter:
- field: tier
  operator: Exists
- field: tier
  operator: In
  values:
  - production
  - staging
- field: foo
  operator: NotIn
  values: 
  - dev
  - 42
  - "true"
  - "3.14"
- field: bar
  operator: Regex
  values:
  - ^abc
  - ^\d.+$
      • field (string)

        Field name for filtering.

        Required value.

      • operator (string)

        Operator for log field comparations:

        • In operator find substring in string
        • NotIn operator is negative version of In operator
        • Regex operator is trying to match regexp over field, only log events with matching fields will pass
        • NotRegex operator is negative version of Regex operator, log events without fields or with not matched fields will pass
        • Exists operator drop log event if it contains some fields
        • DoesNotExist operator drop log event if it does not contain some fields

        Allowed values: In, NotIn, Regex, NotRegex, Exists, DoesNotExist

        Required value.

      • values (array)

        Array of ClusterLogDestination CustomResource names which this source will output with. Fields with float or boolean values will be converted to strings during comparation.