ClusterLogDestination
Scope: Cluster
Version: v1alpha1
Describes setting for a log storage, which you can use in many log sources.
metadata.name
— is an upstream name, which you should use in CR ClusterLoggingConfig.
- specobject
Required value
- elasticsearchobject
- authobject
- awsAccessKeystring
Base64 encoded AWS
ACCESS_KEY
. - awsAssumeRolestring
The ARN of an IAM role to assume at startup.
- awsRegionstring
AWS region for authentication.
- awsSecretKeystring
Base64 encoded AWS
SECRET_KEY
. - passwordstring
Base64 encoded Basic authentication password.
- strategystring
The authentication strategy to use.
Default:
"Basic"
Allowed values:
Basic
,AWS
- userstring
The Basic authentication user name.
- awsAccessKeystring
- dataStreamEnabledboolean
Use for storage indexes or datastreams (https://www.elastic.co/guide/en/elasticsearch/reference/master/data-streams.html).
Datastream usage is better for logs and metrics storage but they works only for Elasticsearch >= 7.16.X.
Default:
false
- docTypestring
The
doc_type
for your index data. This is only relevant for Elasticsearch <= 6.X.- For Elasticsearch >= 7.X you do not need this option since this version has removed
doc_type
mapping; - For Elasticsearch >= 6.X the recommended value is
_doc
, because using it will make it easy to upgrade to 7.X; - For Elasticsearch < 6.X you can’t use a value starting with
_
or empty string. Use, for example, values likelogs
.
- For Elasticsearch >= 7.X you do not need this option since this version has removed
- endpointstring
Required value
The base URL of the Elasticsearch instance.
- indexstring
Index name to write events to.
- pipelinestring
Name of the pipeline to apply.
- tlsobject
Configures the TLS options for outgoing connections.
- caFilestring
Base64 encoded CA certificate in PEM format.
- clientCrtobject
Configures client certificate for outgoing connections.
- crtFilestring
Required value
Base64 encoded certificate in PEM format.
You must also set the
keyFile
parameter. - keyFilestring
Required value
Base64 encoded private key in PEM format (PKCS#8).
You must also set the
crtFile
parameter. - keyPassstring
Base64 encoded pass phrase used to unlock the encrypted key file.
- crtFilestring
- verifyHostnameboolean
Validate the configured remote host name against the remote host’s TLS certificate.
Default:
true
- caFilestring
- authobject
- extraLabelsobject
A set of labels that will be attached to each batch of events.
You can use simple templating here:
{{ app }}
.There are some reserved keys:
- parsed_data
- pod
- pod_labels
- pod_ip
- namespace
- image
- container
- node
- pod_owner
Example:
extraLabels: forwarder: vector key: value app_info: "{{ app }}" array_member: "{{ array[0] }}" symbol_escating_value: "{{ pay\.day }}"
- logstashobject
- endpointstring
Required value
The base URL of the Loki instance.
- tlsobject
Configures the TLS options for outgoing connections.
- caFilestring
Base64 encoded CA certificate in PEM format.
- clientCrtobject
Configures client certificate for outgoing connections.
- crtFilestring
Required value
Base64 encoded certificate in PEM format.
You must also set the
keyFile
parameter. - keyFilestring
Required value
Base64 encoded private key in PEM format (PKCS#8).
You must also set the
crtFile
parameter. - keyPassstring
Base64 encoded pass phrase used to unlock the encrypted key file.
- crtFilestring
- verifyCertificateboolean
Validate the TLS certificate of the remote host.
Default:
true
- verifyHostnameboolean
Validate the configured remote host name against the remote host’s TLS certificate.
Default:
true
- caFilestring
- endpointstring
- lokiobject
- authobject
- passwordstring
Base64 encoded Basic authentication password.
- strategystring
The authentication strategy to use.
Default:
"Basic"
Allowed values:
Basic
,Bearer
- tokenstring
The token to use for Bearer authentication.
- userstring
The Basic authentication user name.
- passwordstring
- endpointstring
Required value
The base URL of the Loki instance.
Agent automatically adds
/loki/api/v1/push
into URL during data transmission. - tlsobject
Configures the TLS options for outgoing connections.
- caFilestring
Base64 encoded CA certificate in PEM format.
- clientCrtobject
Configures client certificate for outgoing connections.
- crtFilestring
Required value
Base64 encoded certificate in PEM format.
You must also set the
keyFile
parameter. - keyFilestring
Required value
Base64 encoded private key in PEM format (PKCS#8).
You must also set the
crtFile
parameter. - keyPassstring
Base64 encoded pass phrase used to unlock the encrypted key file.
- crtFilestring
- verifyHostnameboolean
Validate the configured remote host name against the remote host’s TLS certificate.
Default:
true
- caFilestring
- authobject
- typestring
Type of a log storage backend.
Allowed values:
Loki
,Elasticsearch
,Logstash
- elasticsearchobject
ClusterLoggingConfig
Scope: Cluster
Version: v1alpha1
Describes a log source in log-pipeline.
Each CustomResource ClusterLoggingConfig
describes rules for log fetching from cluster.
- specobject
Required value
- destinationRefsarray of strings
Required value
Array of
ClusterLogDestination
CustomResource names which this source will output with. Fields with float or boolean values will be converted to string. - fileobject
- excludearray of strings
Array of file patterns to exclude.
Example:
exclude: - "/var/log/nginx/error.log" - "/var/log/audit.log"
- includearray of strings
Array of file patterns to include.
Example:
include: - "/var/log/*.log" - "/var/log/nginx/*.log"
- lineDelimiterstring
String sequence used to separate one file line from another.
Example:
lineDelimiter: "\\r\\n"
- excludearray of strings
- kubernetesPodsobject
- labelSelectorobject
Specifies the label selector to filter Pods with.
You can get more into here.
- matchExpressionsarray of objects
List of label expressions for Pods.
Example:
matchExpressions: - key: tier operator: In values: - production - staging
- keystring
Required value
- operatorstring
Required value
Allowed values:
In
,NotIn
,Exists
,DoesNotExist
- valuesarray of strings
- keystring
- matchLabelsobject
List of labels which Pod should have.
Example:
matchLabels: foo: bar baz: who
- matchExpressionsarray of objects
- namespaceSelectorobject
Specifies the Namespace selector to filter Pods with.
- excludeNamesarray of strings
Include all namespaces except a particular set.
- matchNamesarray of strings
Include only a particular set of namespaces.
- excludeNamesarray of strings
- labelSelectorobject
- logFilterarray of objects
List of filter for logs.
Only matched lines would be stored to log destination.
Example:
logFilter: - field: tier operator: Exists - field: tier operator: In values: - production - staging - field: foo operator: NotIn values: - dev - 42 - "true" - "3.14" - field: bar operator: Regex values: - ^abc - ^\d.+$
- fieldstring
Required value
Field name for filtering.
- operatorstring
Required value
Operator for log field comparations:
In
— finds a substring in stringNotIn
— is a negative version of theIn
operatorRegex
— is trying to match regexp over field, only log events with matching fields will passNotRegex
— is a negative version of theRegex
operator, log events without fields or with not matched fields will passExists
— drops log event if it contains some fieldsDoesNotExist
— drops log event if it does not contain some fields
Allowed values:
In
,NotIn
,Regex
,NotRegex
,Exists
,DoesNotExist
- valuesarray
Array of
ClusterLogDestination
CustomResource names which this source will output with.Fields a with float or boolean values will be converted to strings during comparison.
- fieldstring
- multilineParserobject
Multiline parser for different patterns
- typestring
Required value
Parser types:
None
- do not parse logsGeneral
- tries to match general multiline logs with space or tabulation on extra linesBackslash
- tries to match bash style logs with backslash on all lines except the last event lineLogWithTime
- tries to detect events by timestampMultilineJSON
- tries to match JSON logs, assuming the event starts with the{
symbol
Default:
"None"
Allowed values:
None
,General
,Backslash
,LogWithTime
,MultilineJSON
- typestring
- typestring
Required value
Set on of possible input sources.
KubernetesPods
source reads logs from Kubernetes Pods.File
source reads local file from node filesystem.Allowed values:
KubernetesPods
,File
- destinationRefsarray of strings
PodLoggingConfig
Scope: Namespaced
Version: v1alpha1
CustomResource for namespaced Kubernetes source in log-pipeline.
Each CustomResource PodLoggingConfig
describes rules for log fetching from specified Namespace.
- specobject
Required value
- clusterDestinationRefsarray of strings
Required value
Array of
ClusterLogDestination
CustomResource names which this source will output with. - labelSelectorobject
Specifies the label selector to filter Pods with.
You can get more into here.
- matchExpressionsarray of objects
List of label expressions for Pods.
Example:
matchExpressions: - key: tier operator: In values: - production - staging
- keystring
Required value
- operatorstring
Required value
Allowed values:
In
,NotIn
,Exists
,DoesNotExist
- valuesarray of strings
- keystring
- matchLabelsobject
List of labels which Pod should have.
Example:
matchLabels: foo: bar baz: whooo
- matchExpressionsarray of objects
- logFilterarray of objects
List of filter for logs.
Only matched lines would be stored to log destination.
Example:
logFilter: - field: tier operator: Exists - field: tier operator: In values: - production - staging - field: foo operator: NotIn values: - dev - 42 - "true" - "3.14" - field: bar operator: Regex values: - ^abc - ^\d.+$
- fieldstring
Required value
Field name for filtering.
- operatorstring
Required value
Operator for log field comparations:
In
— finds a substring in stringNotIn
— is a negative version of theIn
operatorRegex
— is trying to match regexp over field, only log events with matching fields will passNotRegex
— is a negative version of theRegex
operator, log events without fields or with not matched fields will passExists
— drops log event if it contains some fieldsDoesNotExist
— drops log event if it does not contain some fields
Allowed values:
In
,NotIn
,Regex
,NotRegex
,Exists
,DoesNotExist
- valuesarray
Array of
ClusterLogDestination
CustomResource names which this source will output with.Fields a with float or boolean values will be converted to strings during comparison.
- fieldstring
- multilineParserobject
Multiline parser for different patterns.
- typestring
Required value
Parser types:
None
- do not parse logsGeneral
- tries to match general multiline logs with space or tabulation on extra linesBackslash
- tries to match bash style logs with backslash on all lines except the last event lineLogWithTime
- tries to detect events by timestampMultilineJSON
- tries to match JSON logs, assuming the event starts with the{
symbol
Default:
"None"
Allowed values:
None
,General
,Backslash
,LogWithTime
,MultilineJSON
- typestring
- clusterDestinationRefsarray of strings