Migrating from Ingress NGINX to ALB

This guide describes how to migrate application publishing and Deckhouse Kubernetes Platform (DKP) system interfaces from the ingress-nginx module to the alb module, including replacing application Ingress resources with Gateway API resources and switching traffic to ALB.

Motivation

Migrating from the Ingress API to the Gateway API aligns application traffic management with the current Kubernetes networking model. Active maintenance of the upstream Ingress NGINX project used by DKP has ended, so further upstream development of features, fixes, and integrations is no longer expected. New and actively developed application publishing scenarios should use the Gateway API.

The Gateway API is the current standard for Kubernetes application networking. Compared with the Ingress API, it provides a more flexible and expressive model: protocol-specific route resources, explicit listener configuration, controlled route attachment, and dedicated resources for cross-namespace access. This model supports complex traffic configurations without relying primarily on controller-specific annotations.

The Gateway API also establishes clearer responsibility boundaries. Cluster and network administrators manage traffic infrastructure and Gateway objects through ClusterALBInstance or ALBInstance, while application teams define listeners and routing rules through ListenerSet and route resources. This separation simplifies delegation, validation, and gradual migration.

Scope

This guide covers migration from the DKP application publishing model based on ingress-nginx to the Gateway API model provided by the alb module. It describes the architectural differences, the infrastructure changes required to introduce ALBInstance or ClusterALBInstance and the managed Gateway, and the application changes required to replace Ingress resources with Gateway API resources.

The guide is not a general Gateway API tutorial. It does not cover migration from other ingress controllers, application architecture changes, service mesh migration, or every ingress-nginx annotation. Annotation compatibility is considered only where it affects migration to the alb module.

Model comparison

This section compares how administrators define traffic infrastructure, how DKP provisions it, and how application routing configuration is translated into data-plane configuration.

Ingress API and ingress-nginx

The diagram below shows the resource model and traffic flow through the ingress-nginx module.

HTTP and HTTPS traffic processing through the ingress-nginx module is configured as follows:

1. A cluster administrator creates a cluster-scoped IngressNginxController object.
2. The IngressNginxController specifies the name of the IngressClass it uses. If the name is omitted, nginx is used.
3. DKP reconciles the object and provisions the required infrastructure, including the IngressClass. Multiple IngressNginxController objects can use the same IngressClass.
4. By default, DKP resources are published through the IngressClass named nginx. A different class can be selected in the DKP global configuration.
5. Network administrators or application teams create Ingress objects that select the required IngressClass explicitly or implicitly.
6. The resulting nginx configuration combines infrastructure settings from IngressNginxController with the Ingress objects selected by the IngressClass.

Gateway API and alb

The diagram below shows the resource model and traffic flow through the alb module.

HTTP, HTTPS, gRPC, TLS, TCP, and UDP traffic processing through the alb module is configured as follows:

1. A cluster administrator creates a cluster-scoped ClusterALBInstance object.
2. The ClusterALBInstance specifies infrastructure parameters and the mandatory gatewayName, which identifies the managed Gateway.
3. The ALB controller reconciles the object and provisions the managed Gateway and the required traffic-processing infrastructure. Multiple ClusterALBInstance objects can use the same gatewayName and therefore the same Gateway.
4. If the default DKP Gateway is configured, DKP modules create their ListenerSet, HTTPRoute, and other Gateway API resources for that Gateway.
5. Network administrators or application teams create Gateway API resources and attach them to the managed Gateway.
6. The resulting Envoy Proxy configuration combines infrastructure settings from ClusterALBInstance with the configuration represented by the Gateway API resources attached to the Gateway.

Key architectural differences

The diagrams highlight the following architectural differences:

Migrating infrastructure

Choosing ALBInstance or ClusterALBInstance

Use ClusterALBInstance for a shared or platform-level Gateway, for publishing DKP system interfaces, or when the HostPort inlet is required. To publish DKP system interfaces, follow the procedure in “Accessing DKP system components using Gateway API”. Use ALBInstance for a Gateway dedicated to an application or team and managed within its namespace. ALBInstance supports the LoadBalancer inlet only.

For a detailed comparison, refer to “Automatically creating and flexibly configuring infrastructure for Gateway objects”.

Inlet configuration

In IngressNginxController, the inlet type combines the method of accepting traffic with optional behaviors such as Proxy Protocol and SSL passthrough. The alb module configures these concerns separately:

• ClusterALBInstance supports the LoadBalancer and HostPort inlet types.
• ALBInstance supports the LoadBalancer inlet type.
• Proxy Protocol is enabled by the spec.useProxyProtocol parameter. It can be enabled or disabled on an existing instance without restarting the Envoy Proxy Pod objects or recreating the instance.
• TLS passthrough is configured with a TLS listener and a TLSRoute, rather than with a dedicated inlet type.

Use the following mapping when selecting an inlet for the alb module:

Map related inlet parameters as follows:

Consider the following when migrating these parameters:

• Copy only service annotations supported by the target load balancer implementation.
• spec.inlet.loadBalancer.loadBalancerClass is immutable after creation.
• The default HTTP and HTTPS ports remain 80 and 443. In the alb module, set a port to 0 to disable the corresponding default listener.
• HostPort parameters are available for ClusterALBInstance only. At least one port must be specified.
• Preserve the required source CIDR restrictions in spec.acceptRequestsFrom.
• Configure trusted proxy CIDRs explicitly. Do not trust client-IP headers from arbitrary sources.
• loadBalancerSourceRanges is passed to the LoadBalancer Service. Cloud providers may not support or may ignore this parameter. Verify its behavior with the target load balancer implementation.

TLS and certificates

For instructions on configuring an HTTP-01 issuer with the Gateway API solver, see “Adding a custom HTTP-01 ClusterIssuer or Issuer for ALB”.

Migrating DKP interfaces

To publish DKP system interfaces through the Gateway API, follow “Accessing DKP system components using Gateway API” in the administrator guide.

Migrating application exposure

Supported Gateway API resources

The alb module creates and manages Gateway objects referenced by ALBInstance and ClusterALBInstance resources. One or more instances may refer to the same managed Gateway. Sharing a Gateway allows the instances to use the same Gateway API configuration graph, including ListenerSet, route, and policy resources, while providing separate traffic entry points with different inlet and per-instance proxy deployment settings.

Some instance parameters, such as additionalPorts, frontendTLS, and backendTLS, affect the shared Gateway and therefore cannot have different effective values. For these parameters, the oldest instance by creation timestamp is authoritative. Newer, non-authoritative instances continue to provide their own inlets and proxy workloads, but their conflicting Gateway-level settings are ignored and the conflict is reported in their status.

Do not edit the managed Gateway directly. Define how the Gateway accepts HTTP, HTTPS, gRPC, and TLS traffic with ListenerSet resources, and attach route resources to the corresponding listeners. For plain TCP and UDP traffic, the instance’s additionalPorts parameter defines listeners directly on the managed Gateway.

The alb module supports the following Gateway API resources:

• Gateway — represents a traffic entry point controlled by DKP. DKP creates and continuously reconciles the resource from the parameters of the corresponding ALBInstance or ClusterALBInstance. Do not edit the Gateway directly.
• ListenerSet — defines listeners for a managed Gateway, including their ports, protocols, hostnames, and TLS settings. In the alb module, use ListenerSet as the primary way to define HTTP, HTTPS, gRPC, and TLS listeners instead of editing Gateway.spec.listeners.
• HTTPRoute — routes HTTP and HTTPS requests from a ListenerSet listener to backend Service objects. It supports matching and processing requests by hostname, path, headers, query parameters, and other HTTP attributes.
• GRPCRoute — routes gRPC requests from a ListenerSet listener to backend Service objects, with optional matching by gRPC service, method, or headers.
• TCPRoute — routes plain TCP connections from an additional TCP listener directly to a backend Service. The listener and port are created on the managed Gateway from spec.inlet.additionalPorts.
• TLSRoute — routes TLS connections by SNI without terminating TLS on the Gateway. Attach it to a TLS or HTTPS listener in Passthrough mode defined in a ListenerSet when the application terminates TLS.
• UDPRoute — routes UDP datagrams from an additional UDP listener directly to a backend Service. The listener and port are created on the managed Gateway from spec.inlet.additionalPorts.
• ReferenceGrant — explicitly permits supported references across namespace boundaries, for example when a route and its parent ListenerSet are in different namespaces.
• BackendTLSPolicy — configures TLS and server-certificate validation for connections from the Gateway to a backend Service.

Converting Ingress to Gateway API

Create the target ALBInstance or ClusterALBInstance before converting application resources, wait for it to become ready, and obtain the managed Gateway namespace and name from its status. Convert each Ingress and its related configuration into the appropriate ListenerSet, route, and policy resources for that Gateway. The built-in conversion tool can generate a starting point, but review the resulting manifests before applying them because not every ingress-nginx feature has a direct Gateway API equivalent.

Using the built-in ingress2gateway tool

The gateway controller provides an HTTP endpoint that accepts a single object, a Kubernetes List object, or a YAML payload containing multiple documents. The converter reads the following input resources:

• Ingress — the resources selected for conversion.
• Service — used to resolve named backend ports and service metadata.
• DexAuthenticator — used to resolve spec.applicationDomain when an ingress-nginx external-auth configuration contains nginx variables such as $host.

Pass all related resources as input so the converter can preserve the supported parts of their configuration. The endpoint is disabled by default and listens only inside gateway controller Pod objects.

1. Temporarily enable migrations.ingress2Gateway.enabled in the alb module configuration and wait for the gateway controller to restart:

   d8 k patch moduleconfig alb --type merge \
     --patch '{"spec":{"settings":{"migrations":{"ingress2Gateway":{"enabled":true}}}}}'
   d8 k -n d8-alb rollout status deployment/gateway-controller

2. Forward the endpoint from a gateway controller Pod:

   d8 k -n d8-alb port-forward deployment/gateway-controller 8082:8082

3. In another terminal, export all recognized resource types and send the resulting Kubernetes List directly to the converter:

   d8 k get ingress,service,dexauthenticator --all-namespaces --output yaml | curl --fail-with-body --silent --show-error --request POST --header 'Content-Type: application/yaml' --data-binary @- --output gateway-api.yaml 'http://127.0.0.1:8082/ingress2gateway?gateway=<GATEWAY_NAMESPACE>/<GATEWAY_NAME>&scope=<SCOPE>&ingress-class=<INGRESS_CLASS>'

   where:

   • <GATEWAY_NAMESPACE> — namespace of the managed Gateway;
   • <GATEWAY_NAME> — name of the managed Gateway;
   • <SCOPE> — ALB infrastructure scope: cluster or namespaced. The default is cluster;
   • <INGRESS_CLASS> — IngressClass selected for conversion. The default is nginx.

The query parameters have the following meanings:

• gateway — the target managed Gateway in <GATEWAY_NAMESPACE>/<GATEWAY_NAME> format. The default is d8-alb/public-gw;
• scope — the target ALB infrastructure scope: cluster or namespaced. The default is cluster;
• ingress-class — the IngressClass selected for conversion. The default is nginx.

Review gateway-api.yaml and the conversion diagnostics included as YAML comments before applying the generated resources. After completing the conversion, disable the endpoint:

d8 k patch moduleconfig alb --type merge \
  --patch '{"spec":{"settings":{"migrations":{"ingress2Gateway":{"enabled":false}}}}}'

Extending Gateway API with annotations

The Gateway API specification does not cover every implementation-specific traffic-management feature required by DKP. The alb module therefore uses HTTPRoute annotations to provide configuration options that are not yet represented by standard Gateway API fields. As the corresponding features become available in the Gateway API, the alb module will gradually replace annotation-based configuration with native Gateway API resources and fields. During migration, use standard Gateway API fields where possible and replace ingress-nginx annotations only with supported alb annotations. See the current list in “Supported HTTPRoute annotations”.

Switching traffic to ALB

Run ingress-nginx and alb simultaneously until the ALB path has been validated and the rollback window has closed. Migrate individual domains or namespaces when they can use separate DNS records. Otherwise, switch the shared external entry point by changing DNS records or the backend pool of an external load balancer.

Testing ALB through Ingress NGINX

The spec.migrationGateway parameter of IngressNginxController enables source-IP-based testing without changing DNS. Requests from sourceCIDRs continue to enter through the existing Ingress NGINX entry point but are proxied to the internal Service of the target ALB instance. Other requests continue to use the original Ingress backends.

Despite its use in this guide for ALB migration, migrationGateway is not specific to ALB. Its serviceRef can point to the entry-point Service of any Gateway API implementation that accepts ordinary HTTP and HTTPS traffic on the configured ports.

Locate the ALB configuration Service:

d8 k get service --all-namespaces --selector alb.deckhouse.io/configuration-service

Then configure the source IngressNginxController, using narrow tester CIDRs initially:

apiVersion: deckhouse.io/v1
kind: IngressNginxController
metadata:
  name: <CONTROLLER_NAME>
spec:
  # Existing settings are omitted.
  migrationGateway:
    sourceCIDRs:
      - <SOURCE_CIDR>
    serviceRef:
      namespace: <SERVICE_NAMESPACE>
      name: <SERVICE_NAME>
      ports:
        http: <HTTP_PORT>
        https: <HTTPS_PORT>

where:

• <CONTROLLER_NAME> — name of the source IngressNginxController;
• <SOURCE_CIDR> — CIDR of clients whose requests are proxied to ALB during testing;
• <SERVICE_NAMESPACE> — namespace of the ALB configuration Service;
• <SERVICE_NAME> — name of the ALB configuration Service;
• <HTTP_PORT> — HTTP port on the target Service;
• <HTTPS_PORT> — HTTPS port on the target Service.

HTTP requests are forwarded to the configured HTTP port. For HTTPS, nginx terminates the incoming TLS connection and establishes a new TLS connection to the configured HTTPS port, using the original hostname in the Host header and for SNI. The setting applies to all Ingress resources served by the controller, so verify every hostname accessible from the selected CIDRs.

Remove spec.migrationGateway to immediately return selected clients to the original Ingress backends:

d8 k patch ingressnginxcontroller <CONTROLLER_NAME> --type json \
  --patch '[{"op":"remove","path":"/spec/migrationGateway"}]'

where <CONTROLLER_NAME> is the name of the source IngressNginxController.

Preserving HTTP-01 validation

migrationGateway handles selected application requests, while migrations.http01CertificateSolverBridging keeps cert-manager Gateway API HTTP-01 challenges reachable through Ingress NGINX while public DNS still points to it. When Issuer or ClusterIssuer resources with HTTP-01 solvers are used, enable bridging before requesting certificates for the ALB path:

d8 k patch moduleconfig alb --type merge \
  --patch '{"spec":{"settings":{"migrations":{"http01CertificateSolverBridging":{"enabled":true,"ingressClassName":"<INGRESS_CLASS>"}}}}}'
d8 k -n d8-alb rollout status deployment/gateway-controller

where <INGRESS_CLASS> is the IngressClass that currently receives public traffic on port 80.

The ALB controller creates temporary Ingress resources for cert-manager solver HTTPRoute resources, allowing challenges to pass through Ingress NGINX to the solver Service. This feature requires the ingress-nginx module to remain enabled.

Bridging allows certificates for the Gateway API path to be issued before the traffic cutover. It is then possible to test the prepared TLS configuration through migrationGateway or by connecting directly to the ALB address with a hostname override, for example using the curl --resolve option.

After DNS or the external load balancer sends public traffic to ALB, verify HTTP-01 issuance through the Gateway API path and disable bridging:

d8 k patch moduleconfig alb --type merge \
  --patch '{"spec":{"settings":{"migrations":{"http01CertificateSolverBridging":{"enabled":false}}}}}'

Choosing the switching method

Automatically provisioned load balancer

When the Ingress NGINX Controller uses a LoadBalancer Service provisioned by the Kubernetes cloud provider, the source and target controllers normally have separate load balancer IP addresses or hostnames. Wait for the ALB load balancer and its health checks to become ready, lower the DNS TTL in advance, and change each application DNS record from the Ingress NGINX address to the ALB address.

Use weighted DNS records for a gradual switch only if the DNS provider supports them. A provider-managed load balancer does not normally allow node-by-node migration between two independently managed Service objects. If MetalLB or another implementation uses a fixed address, the same address cannot be assigned to both Service objects simultaneously. Transfer it only during a coordinated cutover.

Manually managed load balancer

When a cloud or private load balancer is managed outside Kubernetes, keep the public DNS record unchanged and migrate its backend pool. Add ALB nodes and their configured ports as healthy backends, shift traffic from Ingress NGINX backends gradually, and remove the old backends only after validation. This supports node-by-node migration and weighted traffic distribution when the external load balancer provides those features.

Configure health checks against /healthz on the ALB HTTP port. Preserve the original protocol and client-address handling: align Proxy Protocol settings for traffic and health checks, or configure trusted forwarded headers for an L7 load balancer. If both controllers run on the same nodes, use different host ports. Alternatively, select disjoint node sets.

Direct HostNetwork or HostPort access

When DNS records point directly to node addresses, deploy ALB on a separate node set or use non-conflicting host ports. For a node-by-node switch, add ALB node addresses to the DNS pool and remove Ingress NGINX node addresses after each node passes validation. DNS pools do not provide deterministic weighting or connection draining, so account for TTLs and client-side caching.

DNS cannot distinguish controllers that use different ports on the same node address. If clients must continue using ports 80 and 443, use separate nodes or introduce a load balancer or NAT rule. HostWithFailover has no direct ALB equivalent. Use the recommended LoadBalancer inlet with MetalLB when equivalent failover behavior is required.

Validating the cutover

Before shifting production traffic:

• Verify that the target ALBInstance or ClusterALBInstance reports ready and synced, and inspect its conflict fields.
• Verify the Gateway, ListenerSet, and route status conditions, including accepted references and programmed listeners.
• Test every hostname and protocol through the ALB address or with migrationGateway, including TLS certificates, redirects, authentication, long-lived connections, and application-specific policies.
• Verify preservation of the client address, Proxy Protocol or forwarded-header processing, source restrictions, and load balancer health checks.
• Confirm that metrics, logs, alerts, and dashboards identify traffic and errors on both controllers.
• Keep Ingress resources, the Ingress NGINX Controller, its external entry point, and valid certificates available throughout the rollback window.

Rolling back

Define rollback criteria before the cutover. If validation fails:

1. For migrationGateway testing, remove spec.migrationGateway.
2. For a DNS switch, restore the Ingress NGINX address and wait for the previous DNS TTL to expire.
3. For a manually managed load balancer, restore Ingress NGINX nodes and ports in its backend pool.
4. If public traffic again enters through Ingress NGINX and the Issuer or ClusterIssuer resources for Gateway API certificates use HTTP-01 solvers, re-enable http01CertificateSolverBridging.

Do not delete the old Ingress resources, Ingress NGINX Controller, load balancer, or DNS values until rollback has been tested and the agreed stabilization period has elapsed.

Cleanup

After the rollback window closes, remove migrationGateway, disable http01CertificateSolverBridging and ingress2Gateway, delete obsolete Ingress resources and Ingress NGINX Controllers, and remove unused load balancers, certificates, Secret objects, and DNS records. Restore normal DNS TTLs after confirming that no clients use the old entry point.