Egress Gateway configuration

Egress Gateway enables centralized management and control of outgoing traffic and provides features such as encryption, routing, and monitoring.

Egress Gateway usage modes

Basic mode

Pre-configured IP addresses are used on egress nodes.

Virtual IP mode

Egress Gateway lets you dynamically assign additional IP addresses to nodes.

Configuration principle

Configuring an egress gateway requires two custom resources:

1. EgressGateway: Describes the group of nodes, one of which will be selected as the active egress gateway, while the rest will remain in standby mode:

   • Among the group of nodes matching the spec.nodeSelector the eligible nodes will be selected. One of them will be assigned as the active gateway. The active node is selected in alphabetical order.

     Attributes of an eligible node:

     • The node is in Ready state.
       • The node is not in the maintenance state (not cordoned).
       • The cilium-agent on the node is in the Ready state.
     • When using EgressGateway in VirtualIP mode, an agent is launched on the active node which emulates a “virtual” IP address using the ARP protocol. The status of this agent’s pod is also taken into account when determining the eligibility of a node.
     • Different EgressGateways can use the same nodes for operation. The active node is selected independently for each EgressGateway, which allows for load balancing between them.
2. EgressGatewayPolicy: Describes the policy for routing network requests from pods in the cluster to a specific egress gateway defined using EgressGateway.

Node maintenance

To perform maintenance on a node that is currently the active egress gateway, follow these steps:

1. Remove the node label to exclude it from the egress gateway candidate pool. <egress-label> is the label specified in spec.nodeSelector of your EgressGateway.

   d8 k label node <node-name> <egress-label>-
   

2. Cordon the node to prevent new pods from starting:

   d8 k cordon <node-name>
   

   After this, Cilium will automatically select a new active node from the remaining candidates. Traffic will continue routing through the new gateway without interruption.

3. After maintenance is complete, return the node to service:

    d8 k uncordon <node-name>
    d8 k label node <node-name> <egress-label>=<value>
   

Comparison with CiliumEgressGatewayPolicy

The CiliumEgressGatewayPolicy implies configuring only one node as an egress gateway. If it fails, there are no failover mechanisms and the network connection will be broken.

Configuration examples

EgressGateway in PrimaryIPFromEgressGatewayNodeInterface mode (basic mode)

apiVersion: network.deckhouse.io/v1alpha1
kind: EgressGateway
metadata:
  name: my-egressgw
spec:
  nodeSelector:
    node-role.deckhouse.io/egress: ""
  sourceIP:
    mode: PrimaryIPFromEgressGatewayNodeInterface
    primaryIPFromEgressGatewayNodeInterface:
      # The "public" interface must have the same name on all nodes that matching the nodeSelector.
      # If the active node fails, traffic will be redirected through the backup node and
      # the source IP address of the network packets will change.
      interfaceName: eth1 

EgressGateway in VirtualIPAddress mode

apiVersion: network.deckhouse.io/v1alpha1
kind: EgressGateway
metadata:
  name: my-egressgw
spec:
  nodeSelector:
    node-role.deckhouse.io/egress: ""
  sourceIP:
    mode: VirtualIPAddress
    virtualIPAddress:
      # Each node must have all the necessary routes configured to access all external public services,
      # the "public" interface must be prepared for automatic configuration of the "virtual" IP as a secondary IP address.
      # In case of failure of the active node, traffic will be redirected through the backup node and
      # the source IP address of the network packets will not change.
      ip: 172.18.18.242
      # List of network interfaces for Virtual IP.
      interfaces:
      - eth1

EgressGatewayPolicy

apiVersion: network.deckhouse.io/v1alpha1
kind: EgressGatewayPolicy
metadata:
  name: my-egressgw-policy
spec:
  destinationCIDRs:
  - 0.0.0.0/0
  egressGatewayName: my-egressgw
  selectors:
  - podSelector:
      matchLabels:
        app: backend
        io.kubernetes.pod.namespace: my-ns