Available in editions:  Open/CE, BE, SE, SE+, Ultimate/EE, Core

Included in extensions: Advanced Infrastructure Security

The module lifecycle stage: General Availability

The registry-packages-proxy module provides an in-cluster HTTP proxy service for accessing packages from container registries. It acts as an intermediary between cluster components and external or internal registries, offering caching capabilities to optimize bandwidth usage and improve package retrieval performance.

This module is a critical infrastructure component that runs on master nodes and is used during cluster bootstrap and runtime operations to fetch packages from container registries.

The module deploys a highly-available proxy service that:

  • Runs as a deployment on master nodes with hostNetwork enabled to ensure availability during bootstrap when CNI is not yet available.
  • Listens on port 4219 (HTTPS) on each master node’s IP address.
  • Provides a /package endpoint for retrieving registry packages by digest.
  • Implements local caching of retrieved packages (up to 1 GB) to reduce network traffic and improve performance.
  • Watches ModuleSource custom resources to obtain registry credentials for different repositories.
  • Uses kube-rbac-proxy to secure access to the proxy and metrics endpoints.
  • Exposes a public HTTPS API (via Ingress) for Deckhouse CLI binaries.
  • Exposes an in-cluster HTTPS API for package icons (no public Ingress).

Architecture

The proxy service consists of two containers:

  1. registry-packages-proxy: The main proxy application that:
    • Fetches packages from remote registries using digests.
    • Caches packages locally in an ephemeral volume (1 GB max).
    • Supports authentication to registries via credentials from ModuleSource resources.
    • Provides health checks and Prometheus metrics.
    • Listens on 127.0.0.1:5080 (HTTP, internal).
  2. kube-rbac-proxy: Provides RBAC-based access control:
    • Exposes the service on port 4219 (HTTPS).
    • Secures /metrics endpoint with Kubernetes RBAC authorization.
    • Secures /package endpoint requiring appropriate permissions.
    • Secures /v1/images/* (Deckhouse CLI downloads) with Kubernetes RBAC authorization.
    • Allows unauthenticated access to /healthz.

HTTP API

After the cluster is bootstrapped and the DNS name template is configured in the publicDomainTemplate parameter, the module creates an Ingress for registry-packages-proxy using that template (for example, registry-packages-proxy.company.my for publicDomainTemplate: "%s.company.my").

The endpoints listed below are reachable in two ways:

  • Public (via the Ingress on the registry-packages-proxy public host): only /v1/images/*.
  • In-cluster only (via the registry-packages-proxy.d8-cloud-instance-manager.svc Service on port 443, or :4219 on each master node for bootstrap): all routes, including /v1/packages/*.

Package icons (/v1/packages/) — in-cluster only

Package icons are served without authentication (kube-rbac-proxy excludes these paths from RBAC) but are only reachable from inside the cluster. They are intentionally not routed through the public Ingress, so the public domain registry-packages-proxy.<publicDomain> will not serve them.

Method Path Description
GET, HEAD /v1/packages/<PACKAGE-REPOSITORY>/<PACKAGE-NAME>/metadata/icon/ Icon of the latest semver tag
GET, HEAD /v1/packages/<PACKAGE-REPOSITORY>/<PACKAGE-NAME>/metadata/icon Same as above
GET, HEAD /v1/packages/<PACKAGE-REPOSITORY>/<PACKAGE-NAME>/metadata/icon/<VERSION> Icon of a specific version (<VERSION> is a semantic version, e.g. v1.0.1)

<PACKAGE-REPOSITORY> is the metadata.name of a PackageRepository custom resource; its spec.registry.repo field tells the proxy which registry path to pull the icon from. The proxy reads the icon from the OCI image <spec.registry.repo>/<PACKAGE-NAME>:<TAG>.

The proxy looks for the following files inside the package image, in this priority order, and returns the first one it finds:

Path Content-Type
docs/icon.svg image/svg+xml
docs/icon.png image/png
docs/icon.jpg image/jpeg
docs/icon.jpeg image/jpeg

If none of these are present (or the file is larger than 4 MiB), the proxy returns 404 Not Found; callers should fall back to a default icon. SVG is preferred because it is resolution-independent.

Example request from a pod in the cluster:

curl -fsSk "https://registry-packages-proxy.d8-cloud-instance-manager.svc/v1/packages/my-repo/my-module/metadata/icon/"

Example response headers (when the image ships docs/icon.svg):

Content-Type: image/svg+xml
Content-Disposition: attachment; filename="<PACKAGE-NAME>.svg"

Deckhouse CLI downloads (/v1/images/) — public

These endpoints are reachable through the public Ingress (registry-packages-proxy.<PUBLIC_DOMAIN>) and require a valid Kubernetes token or a client certificate accepted by kube-rbac-proxy. The account must have a RBAC permission to get the deployments/cli-binary subresource registry-packages-proxy in the namespace d8-cloud-instance-manager.

To grant the required permission, use the ClusterRole d8:registry-packages-proxy:cli-download. For details, refer to “Granting access to Deckhouse CLI downloads”.

The same permission is used to download the Deckhouse CLI plugins. The plugins are available at deckhouse-cli/plugins/<PLUGIN> through the same /v1/images/ endpoints.

Method Path Description
GET /v1/images/<IMAGE>/tags Returns a list of tags in JSON format
GET, HEAD /v1/images/<IMAGE>/images/<VERSION> Returns an OCI image in the application/x-gzip format (with flattened layers)
GET /v1/images/<IMAGE>/manifests/<REFERENCE> Returns an image manifest

Allowed <IMAGE> values:

  • deckhouse-cli
  • deckhouse-cli/plugins/<PLUGIN>, where <PLUGIN> is a single path segment

Deckhouse CLI artifacts can live at one of two places. The official registry publishes them once for all editions, at the registry root one level above the cluster’s edition repository. A registry filled by d8 mirror push keeps them right under the cluster repository, exactly as pushed.

The proxy takes the repository from the d8-system/deckhouse-registry secret and probes the candidate roots in order: first the repository as is, then the repository without the trailing edition segment (ce, be, se, se-plus, ee, fe). The root that answers is remembered and tried first afterwards; a 404 on the remembered root or a repository change in the secret triggers a re-probe. For example, a cluster with the registry.deckhouse.io/deckhouse/ee repository reads the images from registry.deckhouse.io/deckhouse/deckhouse-cli and registry.deckhouse.io/deckhouse/deckhouse-cli/plugins/<PLUGIN>, while a cluster mirrored to registry.local/dest/ee reads them from registry.local/dest/ee/deckhouse-cli/plugins/<PLUGIN>. A repository without an edition segment is used as is. The credentials come from the same secret in all cases.

The /v1/images/<IMAGE>/images/<VERSION> endpoint accepts an optional platform=<OS>-<ARCH> query parameter and picks the matching child manifest from a multi-platform index. If the parameter isn’t set, the linux/amd64 platform is used.

The following is an example of a query for a list of Deckhouse CLI tags:

curl -fsS \
  -H "Authorization: Bearer ${TOKEN}" \
  "https://registry-packages-proxy.example.com/v1/images/deckhouse-cli/tags"

Internal /package endpoint

The legacy /package?digest=... endpoint (used during bootstrap and by internal components) remains protected by RBAC (deployments/http subresource). It is not exposed through the public Ingress.

Package retrieval flow

When a component requests a package:

  1. The request includes a digest parameter (required) and optional repository and path parameters.
  2. The proxy checks its local cache for the requested digest.
  3. If cached, the package is served directly from cache.
  4. If not cached:
    • The proxy fetches credentials for the specified registry from watched ModuleSource resources.
    • The package is retrieved from the remote registry.
    • The package is streamed to the client while simultaneously being cached for future requests.
  5. Responses include appropriate HTTP headers for caching (Cache-Control, ETag, Content-Length).

High availability

The module ensures high availability through:

  • Running multiple replicas on master nodes (in HA configurations).
  • Pod anti-affinity rules to distribute pods across different masters.
  • PodDisruptionBudget to prevent simultaneous disruption of all replicas.
  • Vertical Pod Autoscaler support for automatic resource adjustments.

RBAC roles provided by the module

ClusterRole Purpose
d8:registry-packages-proxy:cli-download Access to /v1/images/*: the deckhouse-cli binary and its plugins
d8:registry-packages-proxy:packages-download Reserved for future authenticated /v1/packages/* routes (icons are served anonymously, in-cluster only)

Neither role is bound to anyone out of the box.

Granting access to Deckhouse CLI downloads

For Deckhouse CLI to work, a user requires permissions to download binaries and obtain the proxy public address.

To be able to create ClusterRoleBinding or RoleBinding, the account must have permissions to assign a respective role. Use the account with the corresponding permissions if necessary, such as /etc/kubernetes/super-admin.conf.

  1. Grant access to download binaries. To do that, use the ClusterRole d8:registry-packages-proxy:cli-download. This role grants permissions required to self-update Deckhouse CLI (d8 cli) and download plugins (d8 plugins):

    Create a ClusterRoleBinding:

    d8 k create clusterrolebinding d8-cli-download \
      --clusterrole=d8:registry-packages-proxy:cli-download \
      --group=<GROUP>
    

    Instead of --group=<GROUP>, you can grant permissions to a specific user (--user=<EMAIL>) or service account (--serviceaccount=<NAMESPACE>:<NAME>).

  2. Grant access to obtain public address of the proxy. Deckhouse CLI obtains public address from the registry-packages-proxy Ingress.

    • Create the d8-cli-ingress Role, which grants the get permission for that Ingress:

      d8 k -n d8-cloud-instance-manager create role d8-cli-ingress \
        --verb=get \
        --resource=ingresses \
        --resource-name=registry-packages-proxy
      
    • Create a RoleBinding for assigning the created role to a target group:

      d8 k -n d8-cloud-instance-manager create rolebinding d8-cli-ingress \
        --role=d8-cli-ingress \
        --group=<GROUP>
      

    Without granted access, the proxy address needs to be specified manually:

    d8 cli check --rpp-endpoint https://registry-packages-proxy.<PUBLIC_DOMAIN>
    
  3. Check the granted permissions without logging in as the user. The permissions are limited by resourceNames, therefore make sure to specify the resource name.

    • Ensure the access to download binaries has been granted:

      d8 k auth can-i get deployments/registry-packages-proxy \
        -n d8-cloud-instance-manager \
        --subresource=cli-binary \
        --as=<EMAIL_OR_SERVICE_ACCOUNT>
      
    • Ensure the access to obtain Ingress registry-packages-proxy has been granted:

      d8 k auth can-i get ingresses/registry-packages-proxy \
        -n d8-cloud-instance-manager \
        --as=<EMAIL_OR_SERVICE_ACCOUNT>
      

    Kube-rbac-proxy caches authorization decisions: a denial for 30 seconds, an approval for 5 minutes. Consider that while checking the granted permissions: once the permission has been granted, a 403 response clears in half a minute, and after a permission has been revoked, the access is still granted for up to 5 minutes.

Limitations

  • The module runs exclusively on master nodes.
  • It requires hostNetwork: true to function during bootstrap phase.
  • Cache size is limited to 1 GB per pod.
  • Most HTTP endpoints require Kubernetes RBAC; only health checks (healthz) and package icons are anonymous. Package icons are additionally limited to in-cluster access (no public Ingress route).
  • Package icons are read from fixed paths inside the package image (docs/icon.svg, docs/icon.png, docs/icon.jpg, docs/icon.jpeg) with SVG preferred. Maximum icon size is 4 MiB.