This module is responsible for providing a network between multiple nodes in a cluster using the cilium module.

Limitations

  1. Service types NodePort and LoadBalancer do not work with hostNetwork endpoints in the DSR LB mode. Switch to SNAT if it is required.
  2. HostPort Pods will bind only to one interface IP. If there are multiple interfaces/IPs present, Cilium will select only one of them, preferring private IP space.
  3. Kernel requirements.
    • The cni-cilium module requires a Linux kernel version >= 5.7.
    • For the cni-cilium module to work together with the istio, openvpn or node-local-dns module, a Linux kernel version >= 5.7 is required.
  4. OS compatibility issues:
    • Ubuntu:
      • not working on 18.04
      • to work on 20.04 you need to install HWE kernel
    • CentOS:

A note about CiliumClusterwideNetworkPolicies

  1. Make sure that you deploy initial set of CiliumClusterwideNetworkPolicies with policyAuditMode configuration options set to true. Otherwise you are degrading cluster operation or even completely losing SSH connectivity to all Kubernetes Nodes. You can remove the option once all CiliumClusterwideNetworkPolicy objects are applied and you’ve verified their effect in the Hubble UI.

  2. Make sure to deploy the following rule, otherwise control-plane will fail for up to 1 minute on cilium-agent restart. This happens due to conntrack table reset. Referencing kube-apiserver entity helps us to “circumvent” the bug.

    apiVersion: "cilium.io/v2"
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: "allow-control-plane-connectivity"
spec:
  ingress:
  - fromEntities:
    - kube-apiserver
  nodeSelector:
    matchLabels:
      node-role.kubernetes.io/control-plane: ""

A note about Cilium work mode change

If you change the Cilium operating mode (the tunnelMode parameter) from Disabled to VXLAN or vice versa, you must restart all nodes, otherwise there may be problems with the availability of Pods.

A note about disabling the kube-proxy module

Cilium has the same functionality as the kube-proxy module, so the latter is automatically disabled when the cni-cilium module is enabled.

A note about fault-tolerant Egress Gateway

Feature is only available in Enterprise Edition

Basic mode

Using pre-configured public IPs of egress-gateway nodes.

Virtual IP mode

Allows you to dynamically assign additional IP addresses to nodes.