ClusterLoggingConfig
Scope: Cluster
Version: v1alpha1
Describes a log source in log-pipeline.
Each custom resource ClusterLoggingConfig describes rules for log fetching from cluster.
- objectspec
Required value
- array of stringsspec.destinationRefs
Required value
Array of
ClusterLogDestinationcustom resource names which this source will output with.Fields with float or boolean values will be converted to strings.
- objectspec.file
Describes a rule for collecting logs from files on a node.
- array of stringsspec.file.exclude
Array of file patterns to exclude when collecting logs.
Wildcards are supported.
Examples:
exclude: "/var/log/nginx/error.log"exclude: "/var/log/audit.log" - array of stringsspec.file.include
Array of file patterns to include.
Wildcards are supported
Examples:
include: "/var/log/*.log"include: "/var/log/nginx/*.log" - stringspec.file.lineDelimiter
String sequence used to separate one file line from another.
Example:
lineDelimiter: "\\r\\n"
- objectspec.kubernetesPods
Describes a rule for collecting logs from the cluster’s pods.
- stringspec.kubernetesPods.keepDeletedFilesOpenedFor
Specifies the time to keep deleted files opened for reading. Vector will keep pods metadata for this time as well to read logs from deleted pods. This option is useful in cases of a log storage unavailability or a network partition. Vector will keep log files opened until finally sending them to the destination.
Enabling this option may affect the resource consumption of the Vector and also flood a disk with deleted logs. Use it with caution.
The format is a string containing the time unit in hours and minutes: 30m, 1h, 2h30m, 24h.
Pattern:
^([0-9]+h([0-9]+m)?|[0-9]+m)$ - objectspec.kubernetesPods.labelSelector
Specifies the label selector to filter Pods with.
You can get more into here.
- array of objectsspec.kubernetesPods.labelSelector.matchExpressions
List of label expressions for Pods.
Example:
matchExpressions: - key: tier operator: In values: - production - staging - key: tier operator: NotIn values: - production- stringspec.kubernetesPods.labelSelector.matchExpressions.key
A label name.
- stringspec.kubernetesPods.labelSelector.matchExpressions.operator
A comparison operator.
Allowed values:
In,NotIn,Exists,DoesNotExist - array of stringsspec.kubernetesPods.labelSelector.matchExpressions.values
A label value.
- stringElement of the array
Pattern:
[a-z0-9]([-a-z0-9]*[a-z0-9])?Length:
1..63
- objectspec.kubernetesPods.labelSelector.matchLabels
List of labels which Pod should have.
Example:
matchLabels: foo: bar baz: who
- objectspec.kubernetesPods.namespaceSelector
Specifies the namespace selector to filter Pods with.
The filter can use one of the three available ways to set the condition (parameters
matchNames,excludeNames,labelSelector)- array of stringsspec.kubernetesPods.namespaceSelector.excludeNames
A list of namespaces, from the pods of which you need to exclude the collection of logs, but collect from the rest.
- objectspec.kubernetesPods.namespaceSelector.labelSelector
Specifies the label selector to filter namespaces from which logs should be collected.
You can get more into here.
- array of objectsspec.kubernetesPods.namespaceSelector.labelSelector.matchExpressions
List of label expressions that a namespace should have to qualify for the filter condition.
Example:
matchExpressions: - key: tier operator: In values: - production - staging- stringspec.kubernetesPods.namespaceSelector.labelSelector.matchExpressions.key
Required value
A label name.
- stringspec.kubernetesPods.namespaceSelector.labelSelector.matchExpressions.operator
Required value
A comparison operator.
Allowed values:
In,NotIn,Exists,DoesNotExist - array of stringsspec.kubernetesPods.namespaceSelector.labelSelector.matchExpressions.values
A label value.
- objectspec.kubernetesPods.namespaceSelector.labelSelector.matchLabels
List of labels that a namespace should have to qualify for the filter condition.
Example:
matchLabels: foo: bar baz: who
- array of stringsspec.kubernetesPods.namespaceSelector.matchNames
A list of namespaces from whose pods logs should be collected.
- array of objectsspec.labelFilter
Rules to filter log lines by their metadata labels.
Example:
labelFilter: - field: container operator: In values: - nginx - field: pod_labels.tier operator: Regex values: - prod-.+ - stage-.+- stringspec.labelFilter.field
Required value
Label name for filtering.
Must not be empty.
Pattern:
.+ - stringspec.labelFilter.operator
Required value
Operator for log field comparations:
In— finds a substring in a string.NotIn— is a negative version of theInoperator.Regex— is trying to match regexp over the field; only log events with matching fields will pass.NotRegex— is a negative version of theRegexoperator; log events without fields or with not matched fields will pass.Exists— drops log event if it contains some fields.DoesNotExist— drops log event if it does not contain some fields.
Allowed values:
In,NotIn,Regex,NotRegex,Exists,DoesNotExist - arrayspec.labelFilter.values
Array of values or regexes for corresponding operations. Does not work for
ExistsandDoesNotExistoperations.Fields a with float or boolean values will be converted to strings during comparison.
- array of objectsspec.logFilter
A list of filters for logs that are applied to messages in JSON format.
Only matched lines would be stored to log destination.
Example:
logFilter: - field: tier operator: Exists - field: foo operator: NotIn values: - dev - 42 - 'true' - '3.14' - field: bar operator: Regex values: - "^abc" - "^\\d.+$"- stringspec.logFilter.field
Required value
Field name for filtering. It should be empty for non-JSON messages.
- stringspec.logFilter.operator
Required value
Operator for log field comparations:
In— finds a substring in a string.NotIn— is a negative version of theInoperator.Regex— is trying to match regexp over the field; only log events with matching fields will pass.NotRegex— is a negative version of theRegexoperator; log events without fields or with not matched fields will pass.Exists— drops log event if it contains some fields.DoesNotExist— drops log event if it does not contain some fields.
Allowed values:
In,NotIn,Regex,NotRegex,Exists,DoesNotExist - arrayspec.logFilter.values
Array of values or regexes for corresponding operations. Does not work for
ExistsandDoesNotExistoperations.Fields a with float or boolean values will be converted to strings during comparison.
- objectspec.multilineParser
Multiline parser for different patterns.
- objectspec.multilineParser.custom
Multiline parser custom regex rules.
- objectspec.multilineParser.custom.endsWhen
It’s a condition to distinguish the last log line of multiline log.
- stringspec.multilineParser.custom.endsWhen.notRegex
Regex string, which treats as match only strings that DOESN’T match regex.
- stringspec.multilineParser.custom.endsWhen.regex
Regex string, which treats as match only strings that match regex.
- objectspec.multilineParser.custom.startsWhen
It’s a condition to distinguish the first log line of multiline log.
- stringspec.multilineParser.custom.startsWhen.notRegex
Regex string, which treats as match only strings that DOESN’T match regex.
- stringspec.multilineParser.custom.startsWhen.regex
Regex string, which treats as match only strings that match regex.
- stringspec.multilineParser.type
Required value
Parser types:
None— do not parse logs.General— tries to match general multiline logs with space or tabulation on extra lines.Backslash— tries to match bash style logs with backslash on all lines except the last event line.LogWithTime— tries to detect events by timestamp.MultilineJSON— tries to match JSON logs, assuming the event starts with the{symbol.Custom- tries to match logs with the user provided regex inspec.multilineParser.customfield.
Default:
"None"Allowed values:
None,General,Backslash,LogWithTime,MultilineJSON,Custom
- stringspec.type
Required value
Set on of possible input sources.
KubernetesPodssource reads logs from Kubernetes Pods.Filesource reads local file from node filesystem.Allowed values:
KubernetesPods,File