Release Notes

This page is the canon for release notes of the ai-models module. Every release block lists only what matters to a cluster administrator or a namespace user; the changelog shipped with the release image is a short projection of the same content.

v0.1.0

Module: ai-models · Version: v0.1.0 · Date: 2026-08-06

Highlights

• Model catalog can now be filled from an external provider: a ModelCatalogSource indexes a Hugging Face catalog into the cluster, and the indexed entries are searchable through the catalog API without registering every model by hand.
• Catalog entries carry provider metadata and inference sizing facts, so a model can be evaluated before anything is downloaded into the cluster.
• The ai-inference module resolves a model reference through an internal lookup API instead of reading catalog custom resources.
• Catalog, distribution and upload endpoints are routable through Gateway API, next to the existing Ingress routing.
• The module survives a slow or contended cluster start instead of crashlooping, and its NodeCache runtime is admissible under the restricted Pod Security Standards profile.
• The module says up front whether it can deliver models in this cluster at all: the storage prerequisites of the configured delivery mode are evaluated before any model or workload exists, and a volume the cluster cannot provide ends with a stated cause instead of an unbounded wait.
• Model storage is a fact the module reports rather than a number a consumer computes: occupancy and budget are served per scope, and every catalog entry carries a pre-download verdict on whether a download fits.

New features

• ModelCatalogSource for the Hugging Face provider: catalog indexing runs on source events and on a timer, large catalogs are indexed incrementally with a resume cursor, and provider sync starts only after an explicit annotation on the source.
• Catalog browse results expose descriptive provider metadata and inference sizing facts derived from the model configuration.
• GET /api/internal/v1/models/lookup returns model facts for in-cluster consumers by model reference.
• HTTPRoute resources for the catalog and distribution endpoints, and a separate route for the upload endpoint that streams without a request body size cap.
• A module-level storage-prerequisites check states whether the module can deliver models in this cluster, independently of whether any Model, ClusterModel or workload exists: the verdict and its reason are exported as the d8_ai_models_storage_prerequisites_satisfied metric with the D8AIModelsStoragePrerequisitesNotSatisfied alert over it, and logged by the controller at startup for clusters without monitoring. It is re-evaluated on every scrape, so a fixed cluster clears the signal without a restart, and the module stays enabled while degraded — catalog browse, sources and the read APIs keep working.
• GET /api/catalog-import/v1/storage-summary reports the occupancy of the module-owned model store, split into namespaced and cluster copies, together with the storage budget where the module has one. What the module cannot determine is absent from the response instead of being reported as a zero: capacityKnown and usageKnown say which half is a fact.
• Catalog entries carry a pre-download verdict: downloadVerdict is Feasible, NotFeasible or EstimateUnavailable, with requiredBytes when known, computed with the same reservation mechanic a real download applies. An entry whose size is not known is never blocked on an estimate the module does not have.
• Storage accounting reports its own state: gauges for the inventory pass health and its last success, and for the accounting ledger’s presence and confirmation, with alerts over them — a download refused because occupancy is unknown is no longer visible only as one log line per minute.
• The whole public read surface of the module is granted at the User access level: Model, ClusterModel, ModelCatalogSource and the catalog import projection used to browse a provider catalog. Previously a user could read models but got Forbidden on the catalog sources they have to name in spec.source.catalog.sourceName, and browsing required ClusterAdmin. Editor, ClusterEditor and ClusterAdmin are write-only deltas on top, because Deckhouse access levels are cumulative. ClusterModel and ModelCatalogSource are cluster-scoped, so a level granted with limitNamespaces still cannot read them — grant it cluster-wide when a user needs the shared catalog.

Improvements

• The catalog API is generated from an OpenAPI 3 contract, published as an OpenAPI 3 document: closed vocabularies became typed enums, timestamps became typed date-time fields, and the gateway failure responses 502 and 503 are part of the contract.
• Model catalog indexing keeps provider entries per source, so several sources no longer overwrite each other’s index entries.
• The storage summary is served from a five-second snapshot with a single-flight refresh, so listing the catalog no longer carries a live read of the whole accounting ledger on every browse request.
• Reading the provisioner’s explanation for an unbound volume is scoped to the claim by a server-side field selector and bounded to one page, instead of listing a whole namespace uncached on every requeue.
• Every documented condition reason is one the module actually emits: the reasons that were declared but never reported are removed, and the docs state which surface each vocabulary lives on — the Model conditions or the workload’s delivery report.

Fixes

• The catalog read API keeps serving the stored catalog while a source is not ready, instead of failing browse requests with an error.
• Shared volume delivery no longer overloads a cluster with parallel materialization jobs: maxConcurrentMaterializations caps how many run at once and defaults to 2.
• The internal model API serves a certificate issued by the platform CA for its in-cluster DNS name, so callers no longer have to skip TLS verification.
• Module tags are valid again, so the module installs on a freshly built release.
• The controller records Kubernetes events for model processing and republishes a corrupt artifact instead of leaving the model in a stuck state.
• Module components no longer crashloop while starting under load: startup is gated on /healthz through a startup probe, the catalog schema migration is off the readiness path, and GOMEMLIMIT stays below the container memory limit.
• A stopped delivery no longer destroys a materialized volume. A delivery that could not proceed — a referenced model that briefly lost Ready, a workload blocked by its own contract — released every shared volume of its owner, after which the unreferenced-set sweep collected a claim that was already bound and carried the downloaded model. Stopping a delivery now keeps the claims that are already bound, together with the references that hold them.
• An unprovisionable shared volume ends with a stated cause instead of an indefinite SharedPVCClaimPending: the provisioner’s own failure is surfaced verbatim as SharedPVCProvisioningFailed, a claim still unbound past a bounded wait becomes SharedPVCProvisioningTimedOut, a class that defers binding is named at once instead of after the deadline, and a volume that lost its PersistentVolume has its own reason. No report is latched, so a cluster fix returns delivery to ready.
• Storage accounting never accounts against lost state: an absent or emptied ledger reads as “occupancy unknown” rather than “nothing is stored”, new downloads are refused until an inventory pass confirms the ledger against the artifacts actually stored, and a publish commit that re-creates the ledger to record one artifact no longer passes it off as confirmed.
• An inventory pass attempts every artifact and joins the failures, so one artifact whose accounting cannot be written no longer hides the rest of the inventory while blocking confirmation for all of it.
• The storage verdict is reported by every replica as soon as it starts, instead of waiting for the leader lease — the report only reads and logs, and its entire value is being early.

Security updates

• Findings from the CVE scan are cleared across all Go modules of the module, including the update of golang.org/x/net to 0.56.0.
• The NodeCache CSI runtime is admissible under the restricted Pod Security Standards profile: containers drop all capabilities, the node /dev tree is no longer mounted, and the single privilege the CSI node plugin still needs is declared through a SecurityPolicyException.
• The report of why a shared volume is unbound only accepts an Event whose involvedObject UID matches the claim exactly. Creating Events is part of ordinary edit access in a workload namespace and the claim name is derived deterministically, so without the UID match a namespace user could put text of their choosing into the module’s own report. The Events access this report needs is read-only and granted only for SharedPVC delivery.

Breaking changes

• The DownloadTask and ModelDelivery custom resources are removed, together with the Model and ClusterModel fields that were declared and never filled: spec.availability, spec.metadata, status.state, status.discovered and status.downloadTasks. No controller ever created, read or wrote any of them, so nothing a cluster does today changes — but an object or a manifest that mentions them is no longer part of the module’s API. status.phase and status.resolved are the lifecycle and enrichment fields that work, and they are no longer marked deprecated in favour of the removed ones. The internal model lookup contract drops its always-empty state field for the same reason.

Upgrade notes

• The downloadtasks.ai.deckhouse.io and modeldeliveries.ai.deckhouse.io definitions stay registered in a cluster that installed an earlier version: the platform applies the definitions a module ships but does not delete the ones it stops shipping. They keep appearing in kubectl api-resources with whatever objects were created under them, and the controller no longer has any access to them. Nothing breaks if they are left in place; remove them when convenient with d8 k delete crd downloadtasks.ai.deckhouse.io modeldeliveries.ai.deckhouse.io, which also deletes any objects of those types.
• A StorageClass with volumeBindingMode: WaitForFirstConsumer cannot serve SharedPVC delivery: delivery waits for the volume to bind before it creates the volume’s first consumer, so such a volume never binds. This used to surface as an eternal SharedPVCClaimPending and is now reported up front as an unsatisfied prerequisite. Select a ReadWriteMany-capable class with volumeBindingMode: Immediate through aiModels.delivery.sharedPVCStorageClassName, the Deckhouse global storage class settings, or a single Kubernetes default StorageClass.

Documentation

• The Hugging Face provider catalog is described in the user and administration guides, including request and response examples for the catalog browse API.
• The administration guide states the storage prerequisites up front: what each delivery mode requires, what each verdict reason means, and how to read the storage summary and the per-entry download verdict.
• The administration guide lists what each access level reads and writes, states that the levels are cumulative, and says that the two cluster-scoped kinds need a cluster-wide binding — including for the browse API, which answers 403 to a namespace-limited subject at the right access level.
• Operational facts that used to be answered in chat are on the pages now: a Deployment gets a second ReplicaSet on delivery and the superseded one is cleaned up by the controller, the storage preflight covers volumeBindingMode while ReadWriteMany is confirmed by the first volume ordered, one bucket serves one cluster, and spec.source.catalog.name is the ClusterModel name in the publishing cluster.

Dependencies

• deckhouse_lib_helm: 1.72.0 → 1.72.10.
• The module requires Deckhouse 1.75.0 or newer and Kubernetes 1.34 or newer.