If Kubernetes cluster nodes are analyzed by security scanners (antivirus tools), you may need to configure them to avoid false positives.
Deckhouse uses the following directories when working (download their list in csv…):
-
/mnt/kubernetes-data
(master node) — only exists in clusters deployed in the cloud when a separate disk is used for the etcd database. -
/var/lib/etcd
(master node) — etcd database. -
/var/lib/deckhouse/
(master node) — files of Deckhouse modules, which dynamically loads from a registry. -
/var/lib/upmeter
(master node, theupmeter
module) — the upmeter module database. -
/etc/kubernetes
(any node) — manifests of static pods, PKI certificate files. -
/var/lib/bashible
(any node) — node configuration files. -
/var/lib/containerd
(any node) — used to store data related to CRI operation (e.g., containerd); contains container image layers, container file system snapshots, meta information, logs, and other container information. -
/mnt/vector-data
(any node, thelog-shipper
module) — checkpoints of sent logs. -
/var/log/containers
(any node) — logs of containers (when usingcontainerd
). -
/var/lib/kubelet/
(any node) —kubelet
configuration files. -
/opt/cni/bin/
(any node) — CNI plugin executables. -
/opt/deckhouse/bin/
(any node) — executable files required for Deckhouse to work. -
/var/log/pods/
(any node) — logs of all pod containers that are running on this cluster node. -
/etc/cni/
(any node) — CNI plugin configuration files.
Recommendations for configuring KESL (Kaspersky Endpoint Security for Linux) to work with Deckhouse
To ensure that KESL does not affect Deckhouse’s performance, follow these configuration recommendations by running them in the command line directly on the host or through the centralized remote management system of Kaspersky Security Center.
Configuration of KESL is carried out using tasks that have specific numbers. Below is an overview of the general setup and the configuration of the tasks used when setting up KESL with Deckhouse.
General Setup
Modify KESL settings related to network packet marking bits as there are overlaps with Deckhouse’s own network packet markings. To do so:
-
Stop KESL if it is running, and modofy the following settings:
- Change the
BypassFwMark
parameter value from0x400
to0x700
; - Change the
NtpFwMark
parameter value from0x200
to0x600
.
- Change the
-
Restart KESL.
Below are some example commands you can run to restart KESL:
systemctl stop kesl sed -i "s/NtpFwMark=0x200/NtpFwMark=0x600/" /var/opt/kaspersky/kesl/common/kesl.ini sed -i "s/BypassFwMark=0x400/BypassFwMark=0x700/" /var/opt/kaspersky/kesl/common/kesl.ini systemctl start kesl
Task 1. File Threat Protection
Exclude Deckhouse directories from analysis by running the following commands:
kesl-control --set-settings 1 --add-exclusion /etc/cni
kesl-control --set-settings 1 --add-exclusion /etc/Kubernetes
kesl-control --set-settings 1 --add-exclusion /mnt/kubernetes-data
kesl-control --set-settings 1 --add-exclusion /mnt/vector-data
kesl-control --set-settings 1 --add-exclusion /opt/cni/bin
kesl-control --set-settings 1 --add-exclusion /opt/deckhouse/bin
kesl-control --set-settings 1 --add-exclusion /var/lib/bashable
kesl-control --set-settings 1 --add-exclusion /var/lib/containerd
kesl-control --set-settings 1 --add-exclusion /var/lib/deckhouse
kesl-control --set-settings 1 --add-exclusion /var/lib/etcd
kesl-control --set-settings 1 --add-exclusion /var/lib/kubelet
kesl-control --set-settings 1 --add-exclusion /var/lib/upmeter
kesl-control --set-settings 1 --add-exclusion /var/log/containers
kesl-control --set-settings 1 --add-exclusion /var/log/pods
When adding, a notification may be shown that some directories do not exist. The rule will still be added (this is expected behavior).
Task 2. Scan My Computer
Exclude Deckhouse directories from analysis by running the following commands:
kesl-control --set-settings 2 --add-exclusion /etc/cni
kesl-control --set-settings 2 --add-exclusion /etc/Kubernetes
kesl-control --set-settings 2 --add-exclusion /mnt/kubernetes-data
kesl-control --set-settings 2 --add-exclusion /mnt/vector-data
kesl-control --set-settings 2 --add-exclusion /opt/cni/bin
kesl-control --set-settings 2 --add-exclusion /opt/deckhouse/bin
kesl-control --set-settings 2 --add-exclusion /var/lib/bashable
kesl-control --set-settings 2 --add-exclusion /var/lib/containerd
kesl-control --set-settings 2 --add-exclusion /var/lib/deckhouse
kesl-control --set-settings 2 --add-exclusion /var/lib/etcd
kesl-control --set-settings 2 --add-exclusion /var/lib/kubelet
kesl-control --set-settings 2 --add-exclusion /var/lib/upmeter
kesl-control --set-settings 2 --add-exclusion /var/log/containers
kesl-control --set-settings 2 --add-exclusion /var/log/pods
When adding, a notification may be shown that some directories do not exist. The rule will still be added (this is expected behavior).
Task 3. Selective Scan
Exclude Deckhouse directories from analysis by running the following commands:
kesl-control --set-settings 3 --add-exclusion /etc/cni
kesl-control --set-settings 3 --add-exclusion /etc/Kubernetes
kesl-control --set-settings 3 --add-exclusion /mnt/kubernetes-data
kesl-control --set-settings 3 --add-exclusion /mnt/vector-data
kesl-control --set-settings 3 --add-exclusion /opt/cni/bin
kesl-control --set-settings 3 --add-exclusion /opt/deckhouse/bin
kesl-control --set-settings 3 --add-exclusion /var/lib/bashable
kesl-control --set-settings 3 --add-exclusion /var/lib/containerd
kesl-control --set-settings 3 --add-exclusion /var/lib/deckhouse
kesl-control --set-settings 3 --add-exclusion /var/lib/etcd
kesl-control --set-settings 3 --add-exclusion /var/lib/kubelet
kesl-control --set-settings 3 --add-exclusion /var/lib/upmeter
kesl-control --set-settings 3 --add-exclusion /var/log/containers
kesl-control --set-settings 3 --add-exclusion /var/log/pods
Note: when adding, a notification may be shown that some directories do not exist. The rule will still be added (this is expected behavior).
Task 4. Critical Areas Scan
Exclude Deckhouse directories from analysis by running the following commands:
kesl-control --set-settings 4 --add-exclusion /etc/cni
kesl-control --set-settings 4 --add-exclusion /etc/Kubernetes
kesl-control --set-settings 4 --add-exclusion /mnt/kubernetes-data
kesl-control --set-settings 4 --add-exclusion /mnt/vector-data
kesl-control --set-settings 4 --add-exclusion /opt/cni/bin
kesl-control --set-settings 4 --add-exclusion /opt/deckhouse/bin
kesl-control --set-settings 4 --add-exclusion /var/lib/bashable
kesl-control --set-settings 4 --add-exclusion /var/lib/containerd
kesl-control --set-settings 4 --add-exclusion /var/lib/deckhouse
kesl-control --set-settings 4 --add-exclusion /var/lib/etcd
kesl-control --set-settings 4 --add-exclusion /var/lib/kubelet
kesl-control --set-settings 4 --add-exclusion /var/lib/upmeter
kesl-control --set-settings 4 --add-exclusion /var/log/containers
kesl-control --set-settings 4 --add-exclusion /var/log/pods
When adding, a notification may be shown that some directories do not exist. The rule will still be added (this is expected behavior).
Task 11. System Integrity Monitoring
Exclude Deckhouse directories from analysis by running the following commands:
kesl-control --set-settings 11 --add-exclusion /etc/cni
kesl-control --set-settings 11 --add-exclusion /etc/Kubernetes
kesl-control --set-settings 11 --add-exclusion /mnt/kubernetes-data
kesl-control --set-settings 11 --add-exclusion /mnt/vector-data
kesl-control --set-settings 11 --add-exclusion /opt/cni/bin
kesl-control --set-settings 11 --add-exclusion /opt/deckhouse/bin
kesl-control --set-settings 11 --add-exclusion /var/lib/bashable
kesl-control --set-settings 11 --add-exclusion /var/lib/containerd
kesl-control --set-settings 11 --add-exclusion /var/lib/deckhouse
kesl-control --set-settings 11 --add-exclusion /var/lib/etcd
kesl-control --set-settings 11 --add-exclusion /var/lib/kubelet
kesl-control --set-settings 11 --add-exclusion /var/lib/upmeter
kesl-control --set-settings 11 --add-exclusion /var/log/containers
kesl-control --set-settings 11 --add-exclusion /var/log/pods
When adding, a notification may be shown that some directories do not exist. The rule will still be added (this is expected behavior).
Task 12. Firewall Management
The task must be disabled. Do not enable it once disabled. It will render Deckhouse inoperable.
This task removes all iptables rules not related to KESL (link to the KESL documentation).
If the task is enabled, disable it by running the following command:
kesl-control --stop-task 12
Task 13. Anti-Cryptor
Exclude Deckhouse directories from analysis by running the following commands:
kesl-control --set-settings 13 --add-exclusion /etc/cni
kesl-control --set-settings 13 --add-exclusion /etc/Kubernetes
kesl-control --set-settings 13 --add-exclusion /mnt/kubernetes-data
kesl-control --set-settings 13 --add-exclusion /mnt/vector-data
kesl-control --set-settings 13 --add-exclusion /opt/cni/bin
kesl-control --set-settings 13 --add-exclusion /opt/deckhouse/bin
kesl-control --set-settings 13 --add-exclusion /var/lib/bashable
kesl-control --set-settings 13 --add-exclusion /var/lib/containerd
kesl-control --set-settings 13 --add-exclusion /var/lib/deckhouse
kesl-control --set-settings 13 --add-exclusion /var/lib/etcd
kesl-control --set-settings 13 --add-exclusion /var/lib/kubelet
kesl-control --set-settings 13 --add-exclusion /var/lib/upmeter
kesl-control --set-settings 13 --add-exclusion /var/log/containers
kesl-control --set-settings 13 --add-exclusion /var/log/pods
When adding, a notification may be shown that some directories do not exist. The rule will still be added (this is expected behavior).
Task 14. Web Threat Protection
We recommended disabling the task. If you need to enable the task for some reason, configure it independently to avoid affecting Deckhouse performance.
If the task is enabled and its negative impact on Deckhouse is detected, disable the task by executing the command below:
kesl-control --stop-task 14
Task 17. Network Threat Protection
We recommended disabling the task. If you need to enable the task for some reason, configure it independently to avoid affecting Deckhouse performance.
If the task is enabled and its negative impact on Deckhouse is detected, disable the task by executing the command below:
kesl-control --stop-task 17
Task 20. Behavior Detection
With default settings, this task has no negative impact on Deckhouse performance. If you need to enable the task for some reason, configure it independently to avoid affecting Deckhouse performance.
If the task is enabled and its negative impact on Deckhouse is detected, disable the task by executing the command below:
kesl-control --stop-task 20
Task 21. Application Control
With default settings, this task has no negative impact on Deckhouse performance. If you need to enable the task for some reason, configure it independently to avoid affecting Deckhouse performance.
If the task is enabled and its negative impact on Deckhouse is detected, disable the task by executing the command below:
kesl-control --stop-task 21
Task 22. Web Control
We recommended disabling the task. If you need to enable the task for some reason, configure it independently to avoid affecting Deckhouse performance.
If the task is enabled and its negative impact on Deckhouse is detected, disable the task by executing the command below:
kesl-control --stop-task 22