The alb module

The alb module implements an Application Load Balancer (ALB) and allows you to publish applications through Kubernetes Gateway API. It deploys and configures the infrastructure for receiving and routing external requests, and also verifies the user configuration of the Gateway API.

The current version of Envoy Proxy in use is v1.38.3.

The module supports:

• traffic balancing across multiple protocols;
• automatically creating and flexibly configuring infrastructure for Gateway objects (a service resource for configuring the traffic entry point into the cluster);
• handling incoming requests with ListenerSet objects (a custom resource);
• routing incoming requests with HTTPRoute, GRPCRoute, TCPRoute, UDPRoute, and TLSRoute objects (custom resources);;
• validating Gateway API configuration.

Advantages of the Gateway API approach

The module is built on the Kubernetes Gateway API, the modern standard for managing inbound traffic that succeeds the Ingress API. Building on this standard, the module provides:

• a single declarative API for several protocols at once (HTTP/HTTPS, gRPC, TCP, UDP, and TLS passthrough), including scenarios that the Ingress API does not cover;
• a separation of responsibilities between the cluster administrator (infrastructure through ClusterALBInstance and ALBInstance), the namespace administrator (traffic handling through ListenerSet), and the application team (routing through HTTPRoute and other route objects), which simplifies multitenancy;
• extended request-handling capabilities out of the box: per-route WAF (ModSecurity/Coraza), external authentication, allowlists of source IP addresses, request rate limiting, session affinity, GeoIP, BackendTLSPolicy, Proxy Protocol, and HTTP/3.

The module can run in a cluster alongside the ingress-nginx module, so the Gateway API can be adopted gradually: new applications are published through the module while existing ones keep their current scheme.

Difference between the Kubernetes Gateway API and an API gateway

Despite the similar names, these are different concepts:

• The Kubernetes Gateway API is a set of Kubernetes resources (a specification) that describe how inbound traffic is routed to services. It is a configuration interface implemented by controllers, and the successor to the Ingress API.
• An API gateway is an architectural component (or product) that aggregates several application APIs behind a single entry point and centralizes cross-cutting functions such as authentication, authorization, and request rate limiting for API consumers.

In other words, the Kubernetes Gateway API describes how to configure traffic routing, while an API gateway is a type of infrastructure that handles that traffic. Some API gateways can be configured through the Kubernetes Gateway API. The alb module is an implementation of the Kubernetes Gateway API.

Comparison of the ingress-nginx and alb modules

Both modules solve the same task — receiving and routing external traffic to applications — but rely on different standards: ingress-nginx uses the Ingress API with annotations, while alb uses the Kubernetes Gateway API. The modules can be used in a cluster simultaneously (for more information, see the Admin guide). The table below compares their capabilities in the current versions.

Traffic balancing across multiple protocols

The module allows using the following protocols for receiving and further routing traffic:

• HTTP/HTTPS: The main traffic routing protocols. HTTP/1.1 and HTTP/2 are supported by default. HTTP/3 can be enabled via the ALBInstance and ClusterALBInstance parameters.
• WebSocket: Supported natively by Envoy Proxy, without additional configuration by the user.
• gRPC: Routing of gRPC routes is supported via GRPCRoute.
• Proxy Protocol: Receiving Proxy Protocol traffic is supported by utilizing the corresponding parameters in ALBInstance and ClusterALBInstance.
• SSL Passthrough: End-to-end routing of SSL traffic is supported via TLSRoute.
• TCP: Routing of TCP traffic is supported via TCPRoute.
• UDP: Routing of UDP traffic is supported via UDPRoute.

SSL/TLS Support

By default, Envoy Proxy supports TLSv1.2 and TLSv1.3. When using TLSv1.2, the following cryptographic parameters are applied:

• Key exchange: ECDHE;
• Authentication: ECDSA, RSA;
• Encryption (AEAD): AES-GCM (128/256), ChaCha20-Poly1305;
• Integrity check: SHA256, SHA384.

When using TLSv1.3, the cryptographic protocol suite is determined by the BoringSSL library.

Monitoring and statistics

The alb module collects Envoy Proxy metrics and exposes them to Prometheus in Prometheus format. When the operator-prometheus module is enabled, the d8-alb-proxy PodMonitor scrapes the /stats/prometheus endpoint from every ready ALB proxy pod. The collected metrics are exported with the d8_alb_gateway_ prefix.

Depending on the metric, the data can be grouped by:

• namespace;
• gateway;
• clusteralbinstance;
• hostname;
• route;
• response_code;
• method for the HTTP request counter by method;
• country for GeoIP metrics;
• city for GeoIP metrics by city;
• cluster_name for upstream clusters;
• node and tier for proxy target metadata.

The graphs are collected in Grafana dashboards stored in monitoring/grafana-dashboards/alb:

• requests.json (Requests) provides an overview of HTTP requests, HTTP methods, HTTP, TCP, and UDP traffic, a country map and Top cities for GeoIP requests, errors, upstreams, TLS, and listeners;
• route-details.json (Route Details) provides drill-down statistics for a selected route, including request count, RPS, HTTP traffic, P50/P95/P99 latency, response codes, and error signals;
• upstream-details.json (Upstream Details) provides drill-down statistics for a selected upstream cluster, including request count, latency, retries, timeouts, connection failures, pending requests, and healthy endpoints.

Basic principles of collecting statistics

1. Envoy Proxy generates standard metrics and custom HTTP statistics. Custom request statistics include the route, requested hostname, and response code as metric labels.
2. Envoy Proxy exposes metrics at /stats/prometheus. The ALB proxy’s kube-rbac-proxy makes this endpoint available on the https-metrics port and authorizes access to the corresponding proxy workload. Gateway controller metrics are also published through its kube-rbac-proxy.
3. When operator-prometheus is enabled, the d8-alb-proxy and d8-alb-gateway-controller PodMonitors scrape ALB proxy and Gateway controller metrics; metric names are prefixed with d8_alb_gateway_.
4. Grafana dashboards query the resulting Prometheus metrics and provide filters for namespace, gateway, ALB instance, hostname, route, response code, or upstream cluster.

What information does Prometheus collect, and in what form

• HTTP: d8_alb_gateway_envoy_http_custom_downstream_rq_by_path counts requests with route, hostname, and response_code labels; d8_alb_gateway_envoy_http_custom_downstream_rq_by_method counts requests with method, route, and hostname labels; d8_alb_gateway_envoy_http_custom_downstream_rq_{rx,tx}_bytes_total counts received and sent HTTP traffic; d8_alb_gateway_envoy_http_custom_downstream_rq_duration_by_path_{sum,count,bucket} records request duration; no-route, no-cluster, timeout, and downstream reset metrics expose error signals;
• GeoIP: d8_alb_gateway_envoy_http_custom_geoip_requests counts HTTP requests by the country label; d8_alb_gateway_envoy_http_custom_geoip_requests_by_city counts them by country and city. The first metric is created when GeoIP City DB and geoIP.headers.country are configured; the second additionally requires geoIP.headers.city.
• GeoIP status: d8_alb_gateway_geoip_configured is 1 when an ALBInstance has a configured GeoIP database source (GeoProxy or a direct mirror), and 0 otherwise. It is shown in the GeoIP status panel of the Requests dashboard; it reports configuration, not database readiness or freshness.
• Upstream: d8_alb_gateway_envoy_cluster_upstream_rq_time_{sum,count,bucket} records upstream latency; request totals, retries, retry overflows, timeouts, connection failures, active and pending requests, resets, and healthy endpoint counts are also collected;
• Listener and TLS: d8_alb_gateway_envoy_listener_downstream_cx_active and d8_alb_gateway_envoy_listener_ssl_certificate_* expose active downstream connections and TLS certificate statistics;
• TCP: d8_alb_gateway_envoy_tcp_custom_downstream_cx_total and d8_alb_gateway_envoy_tcp_downstream_cx_{rx,tx}_bytes_total expose connection counts and traffic volume;
• UDP: d8_alb_gateway_envoy_cluster_udp_sess_{rx,tx}_datagrams expose received and transmitted datagrams for UDP sessions.

Automatically creating and flexibly configuring infrastructure for Gateway objects

The module provides two custom resources (CRDs) for declarative description and management of infrastructure for Gateway objects: ClusterALBInstance and ALBInstance. The characteristics of these resources and the differences between them are described in the table below.

Creating a ClusterALBInstance object or an ALBInstance object results in creation of a managed Gateway object in the cluster. At the same time:

• Each Gateway object is served by at least one Envoy Proxy instance.
• Traffic reaches it through a Service object of type LoadBalancer or directly by using HostPort parameters.
• Each Gateway object creates two default listeners: d8-http (port 80) and d8-https (port 443). They are intended for service tasks such as gateway availability checks or cert-manager HTTP-01 challenges. They are not recommended for publishing applications; use ListenerSet for that purpose instead.

Several ClusterALBInstance objects or ALBInstance objects may point to the same Gateway object through the gatewayName field. In that case, they describe one shared gateway, but the request handling infrastructure may differ depending on settings. You can think of gatewayName as an analogue of ingressClass for IngressNginxController objects.

Handling incoming requests with ListenerSet objects

The ListenerSet object describes system and user traffic handlers that define hostname, TLS mode, port, and protocol. Each ListenerSet object is linked to a specific parent Gateway object through the spec.parentRef field, and routes are then attached to it.

Placement of ListenerSet objects depends on the type of Gateway object in use:

• for ClusterALBInstance, ListenerSet objects may be placed in any namespace;
• for ALBInstance, ListenerSet objects are recommended to be placed in the same namespace.

In both cases, it is recommended to place the ListenerSet object in the same namespace as the HTTPRoute, GRPCRoute, and TLSRoute objects attached to it. This improves configuration readability and helps avoid additional setup such as ReferenceGrant objects.

TCPRoute and UDPRoute objects for plain TCP/UDP ports defined in additionalPorts are attached directly to the corresponding managed Gateway listener, rather than to a ListenerSet. This gives network administrators finer control over which additional ports are exposed in the cluster. For details, see Opening an additional TCP/UDP port in the Administrator guide.

Routing incoming requests with HTTPRoute, GRPCRoute, TCPRoute, UDPRoute, and TLSRoute objects

At the moment, the module supports the following route types:

• HTTPRoute: For routing HTTP/HTTPS/TLS requests.
• GRPCRoute: For routing gRPC traffic.
• TLSRoute: For TLS passthrough routing.
• TCPRoute: For routing TCP traffic (plain TCP via a Gateway listener from additionalPorts, or after TLS termination on a ListenerSet).
• UDPRoute: For routing UDP traffic via a Gateway listener from additionalPorts.

HTTPRoute objects support extended settings through annotations that complement the current Gateway API specification.

Validating Gateway API configuration

In addition to Gateway API infrastructure configuration, the module validates user settings to prevent conflicting configurations from being applied. For example, the module checks for conflicts between identical traffic handlers in different ListenerSet objects when they point to the same Gateway object.