FAQ

Most questions about this module are really one question — why is my order not Ready — so the second half of this page is organised by refusal reason. The reasons are a closed set: whatever an order says, it says one of these.

Why can I not choose the accelerator for my order?

Because the platform can choose better than a manifest can. An order names a class and a model; the accelerator, its share, the placement and the cpu, memory and model-volume estimates are computed against the hardware the cluster actually has and reported back in status.resolved.

A field for it in the order would be a field the platform has to either honour — losing the co-placement that makes two half-card models share one card — or override, which is worse than not offering it.

Why does my order have no field for the runtime?

The runtime name and every launch parameter come from the recipe of the launch plan, compiled against the chosen hardware. A different value means a different recipe or a different class, which is an administrator’s decision.

An order that carries a runtime block anyway is rejected by a client asking for strict field validation and silently pruned by one that does not.

How do I find out which API contract my order got?

Read status.model.endpointType. It answers Chat, Embeddings or Rerank.

This is your only window on that choice, and deliberately so: the contract is picked from modelPolicy.allowedEndpointTypes of the class, and a namespace user is not granted read access to classes.

Why was my embedding model served as chat?

Because the class allowed several contracts and the model source carried no facts about the model.

The platform takes the first element of the effective set, where the effective set is the class’s list narrowed by whatever the source says the model can serve. From the ai-models catalog, which carries those facts, an embedding model gets the embedding contract. From a bare Hugging Face reference, which carries none, the whole list stands and the first element wins — and in the bundled class default-llm the first element is Chat.

The fix is a class whose list names one contract. Ask your administrator, or see Examples for a class for embeddings only.

Can I read the class my order uses?

No. kubectl get inferenceserviceclass is not granted to a namespace user, on purpose. What the class decided about your order is echoed back into the order’s own status: status.model.endpointType for the contract and status.constraints for the replica bounds and the priority class.

My order says ClassNotFound or ClassNotReady. What now?

The class your order names does not exist, or it exists and is not ready. Read the name back from your own order — spec.inferenceServiceClassName — and compare it with the classes your administrator ships. This is the reason to expect right after an upgrade that renamed the bundled class: an order naming the old name keeps naming it, and nothing rewrites it for you.

ClassNotReady is not yours to fix: the class exists, and something in it does not hold. Ask your administrator.

My order says NamespaceNotAllowed. What now?

The class does not admit orders from your namespace. The allow-list lives on the class, in admissionPolicy.allowedNamespaces, and only an administrator can widen it. A different class may admit you.

My order says ModelNotFound or ModelNotReady. What now?

The model reference does not resolve, or the model exists but is not ready. This one is yours: check the repository id for a Hugging Face source, or the object name and kind for an ai-models source.

kind selects between a namespaced Model and a cluster-scoped ClusterModel, and a namespace inside ref is allowed for Model only — the API server refuses the other combination at write time.

My order says InvalidModelSource. What now?

The model reference does not have the shape the source requires. This one is yours, and the fastest read is spec.model: a Hugging Face source wants a repository id, an ai-models source wants an object name and a kind.

My order says AuthSecretNotFound. What now?

Your order pulls a model that needs credentials, and the Secret it names is not in your namespace. Create it, or correct the name in spec.model.authSecretRef.

My order says EndpointTypePolicyConflict. What now?

What the class allows and what the model can serve do not intersect at all. Nothing in the order can fix that: the list belongs to the class, and you need a different class.

My order says FormatNotAllowed or ParameterCountNotAllowed. What now?

The model is denied by the policy of the class — its format is not in allowedFormats, or its parameter count exceeds maxParameterCount. A different model, or a different class.

My order says CatalogDisabled. What now?

Catalog integration is off in the module settings (catalog.mode: None) and your order names a catalog model. Either name a direct Hugging Face source, or ask for the setting to be changed. See Configuration.

My order says CatalogLookupFailed or CatalogKindNotServed. What now?

The catalog was asked about your model and did not answer usefully. CatalogLookupFailed means the request itself failed — usually transient, and the next pass asks again. CatalogKindNotServed means the catalog does not serve the kind your reference names, which is not transient: correct the kind, or name a direct Hugging Face source.

My order says NoCapacity, NoAcceleratorWithEnoughMemory or NoCompatiblePlacement. What now?

Usually the cluster has no room for this order right now: either nothing is free, or no accelerator has enough memory for this model at any share the class permits. Waiting is a valid answer for those two — capacity is a moving quantity, and the plan is recomputed. status.resolved counts the recomputations in replanCount.

NoCompatiblePlacement has a second cause that waiting does not cure: the memory profile of the hardware the order would land on is incomplete, so no share can be computed at all. The message tells the two apart. That one is a platform-side subject and needs an administrator.

My order says PlannerRequestFailed or PlanningPreemptionFailed. What now?

The platform could not finish planning your order. Neither is yours to fix and neither is permanent: the first says the planner did not answer, the second that freeing capacity for your order did not work out. Both are retried. If either stays for long, it is an administrator’s subject.

My order says LaunchPlanNotAPlacement. What now?

The launch configuration stored for your order describes something that is not a placement: one of its fields would turn the device claim or the accounting of capacity into something nobody asked for. The message names the field and the value — a share outside 1..100, an accelerator asked for with no device class, an amount of memory the class does not carry.

An order arrives here two ways. Either the configuration was written by hand in the namespace of the order and its content does not hold up: delete that object and the module computes a new one. Or the answer of the module itself did not hold up — that is a defect of this module, and the message is what to report.

My order says PriorityClassNotFound. What now?

The priority class the class of your order names does not exist in the cluster. Your order names no priority of its own, so this is an administrator’s fix.

My order says QuantizationMismatch or NoCompatibleRuntimeAvailable. What now?

No compiled recipe fits this model on this hardware — the available recipes use another quantization, or no runtime supports this combination at all. This is not resolvable from the order; it needs a recipe, which is a platform-side subject.

What is the difference between ResourceClaimExhausted and PlacementUndecidable?

This is the most useful distinction on the page, because the two look alike and ask for opposite things.

• ResourceClaimExhausted — the search happened. Every device class the order was allowed to try has been tried, and none of them holds it. The subject is capacity: go looking for room, or wait for some;
• PlacementUndecidable — the search never started. There was nothing to try, because the class cannot be judged at all. The subject is the class: its selector expression needs an administrator’s eye, and waiting is not an answer.

If you see the second one, do not wait for it to clear. It will not.

My order says ResourceClaimPending. Is that an error?

No. The workload is waiting for its resource claim to be allocated. It is a stage, not a refusal.

My order says WorkloadNotReady or WorkloadError. What now?

These two are the workload half of an order, and they are not the same answer. WorkloadNotReady means the objects are there and the replicas have not come up yet — pulling an image, downloading the model, waiting for a node. It passes on its own.

WorkloadError means the objects could not be applied or the workload is refused by the cluster, and it does NOT pass on its own. The message names what failed. One case worth knowing: a workload whose volume the order has outgrown is refused for ever, because a volume cannot be resized — the order carries a SettledRegionTooSmall event saying so, and Release notes give the procedure.

My order says WorkloadStartupFailed. What now?

Every pod of your order fails to start its container, and waiting will not help: either the image cannot be pulled, or the container falls over as soon as it starts. The message names the pod, the container, the reason the node gave and the restart count.

The difference from WorkloadNotReady is what you are waiting for. There the replicas are still coming up and it passes on its own. Here the node has already said the start failed, and the next attempt ends the same way: something has to change — the image, the settings of the order, or its environment.

The message also carries what the container itself printed before it died: a few lines of its output where the failure is described. The name CrashLoopBackOff says the container kept falling over; the lines say what over — memory that was not there, a weight file that was not found, a setting that did not add up. When no lines are there, the output could not be had: a container that never started, because its image could not be pulled, has none.

While the order stands in the refusal, the container, the reason of the node and the restart count in the message are kept current, while the lines of output stay the ones read when the failure was first seen: fetching them costs a request to the server, and that is not done on every pass. When a different container of the same pod starts failing, the output is read again.

The refusal lifts itself as soon as a pod comes up: no edit to the order is needed to leave it, if none was needed in the first place. The workload stays in place while the order is refused, so the module sees the pod come up.

My order is Degraded, but the endpoint answers. What is wrong?

Exactly what it says: the inference API is healthy, and scaling up is blocked. The reason is usually ScaleUpBlocked — preemption exhausted the capacity the autoscaler needed to grow the order.

Your endpoint keeps serving. What you do not have is headroom.

The Planned condition says LaunchPlanInForce. Is the order broken?

No: the order is ready and serving. The reason LaunchPlanInForce means the order runs on the launch plan it already holds, while recomputing that plan did not succeed on this pass. What exactly did not succeed is in the message of the same condition — the refusal reason of the planner and its own words.

Why a recomputation can fail for an order that works: it is about capacity beyond the place the order already holds. That place itself the planner does see — the driver withdraws an allocated device from publication, but the order sends what it holds separately, and the recomputation counts it as available to itself.

What to do: usually nothing. If you need the order to grow, go looking for free capacity on its device class — the reasons NoCapacity and NoAcceleratorWithEnoughMemory in the message name what was missing. The memory refusal names both the required amount and the largest one found, so «the model is too large for the fleet» and «the fleet shows nothing large right now» are told apart from one message.

The order is ready, but a second replica never arrives. It carries a GrowthClaimUnplaced event

The order is short of capacity to grow, and that is all. The replica the autoscaler asks for got no device; the replicas without which the order is not served hold theirs, so the order stays ready and stays on its device class.

Why it is not moved to another class: the placement of an order is one per set — a single claim template whose spec the server will not change. Moving the order to another class is possible only by taking the set down, which trades a working replica for one that does not exist yet. The module does not make that trade.

What to do: go looking for free capacity on the device class of the order. As soon as a partition frees up the cluster scheduler places the waiting replica itself — nothing needs replanning. The text of the event names what the claim said.

My order says ServiceUnhealthy. What is the difference from HealthCheckFailed?

Both are the health probe, at two different points in the life of an order:

• HealthCheckFailed with phase Pending — the order has never been ready. It is still bootstrapping, and the probe has not succeeded yet;
• ServiceUnhealthy with phase Failed — the order was ready and then the API stopped answering, for long enough that the failure is not a blip.

Which phase means what?

Read the conditions rather than the phase when something is wrong: the phase says that, the conditions say what.

Where do I look for diagnostics?

kubectl get inferenceservice <name> -n <namespace> \
  -o jsonpath='{range .status.conditions[*]}{.type}{"="}{.status}{" "}{.reason}{"\n"}{end}'

Five conditions answer in order — ClassResolved, ModelResolved, Planned, WorkloadReady, Ready — so the first one that is not True is where to look. A sixth, ScalingHealthy, is about growth rather than readiness, and True on it means healthy.

Why does my order have only one replica?

Because its class names no scalingPolicy. A class without that block gives its orders one replica and no autoscaler at all — that is a capacity decision the module does not make on an administrator’s behalf.

status.constraints shows the bounds actually in force for your order.

What happens to the objects when I delete an order?

Everything the order owns goes with it: the workload, the service, the published endpoint objects and the auth Secret. They all carry ownerReferences pointing at the order.

Your own Secret holding a Hugging Face token is not owned by the order and is left alone.

Does the module need ai-models installed?

No. Orders naming a direct Hugging Face source work without it. Set catalog.mode: None in the module settings for a cluster where the catalog is not installed, and the catalog clients and their permissions are not created.

What must I clean up before disabling the module?

Delete every InferenceService in every namespace first, then the InferenceServiceClass objects those namespaces reference. Disabling stops the controller and the resource planner, and with them every order workload; disabling without the cleanup leaves order objects in the cluster with nothing reconciling them.