Usage

In DRA mode a workload requests a GPU at the Pod level, not at the node level. There are two ways to do it:

• Via an extended resource — the administrator publishes a GPUClass, and the user requests it by name in one line. Recommended.
• By creating a ResourceClaimTemplate yourself — the user authors a DeviceClass with a CEL selector and a ResourceClaimTemplate. Use this when the published classes are not enough.

Requesting a GPU by name

Ask your administrator which classes are available, or look them up yourself with the following commands:

List all GPUClass objects in the cluster:

d8 k get gpuclasses

List the device classes for dynamic GPU allocation (DRA) created by the controller:

d8 k get deviceclasses -l gpu.deckhouse.io/managed-by=gpu-controller

Each published class exposes an extended resource named gpu.deckhouse.io/<device-class-name>. Request it like any other resource:

apiVersion: v1
kind: Pod
metadata:
  name: cuda-probe
spec:
  restartPolicy: Never
  containers:
    - name: app
      image: nvidia/cuda:12.8.0-devel-ubuntu24.04
      command: ["nvidia-smi", "-L"]
      resources:
        requests:
          gpu.deckhouse.io/ml-ai-h100: 1
        limits:
          gpu.deckhouse.io/ml-ai-h100: 1

On admission the module replaces the extended resource with a generated ResourceClaim and wires it into the container for you — the user does not write spec.resourceClaims or resources.claims.

What value to request

It depends on the class. Use the class name to determine the value:

For example, a quarter of a shared 1g12gb partition:

resources:
  requests:
    gpu.deckhouse.io/ml-ai-h100-1g12gb-mps-percent: 25
  limits:
    gpu.deckhouse.io/ml-ai-h100-1g12gb-mps-percent: 25

To see which units a class uses, check its gpu.deckhouse.io/capacity-units label — 1 for exclusive classes, 100 for -percent classes.

What to watch for when requesting a GPU by name

The class name must exist. A typo is rejected outright rather than silently ignored:

Error from server (Forbidden): admission webhook "podgpuresourceclaim.gpu.deckhouse.io" denied
the request: get DeviceClass "no-such-class": deviceclasses.resource.k8s.io "no-such-class" not found

Requesting a GPU by name does not work in system namespaces. The webhook deliberately skips kube-system and every d8-* namespace. There, the extended resource is left untouched in the Pod spec — and since no node advertises it, the Pod stays Pending forever with no alert or warning. Run GPU workloads in user namespaces.

Sharing across namespaces

By default a shared GPU is only reused by workloads from the same namespace. The check runs on the node: the pre-bind conflict predicate rejects the allocation before bind, and the vendor adapter re-checks it at prepare. A rejection appears as the condition BindingFailed=True with reason NamespaceConflict in ResourceClaim.status.devices. The restriction applies per placement, so different MIG partitions of one card stay available to different namespaces. Cluster administrators can lift it with the dra.allowCrossNamespaceSharing setting.

Requesting a device directly

Use this path when the published classes do not meet the user’s needs — a specific card by PCI address, an unusual attribute combination, or an opaque device configuration such as VFIO passthrough.

The attributes a selector can match on are listed in Device attribute reference.

When requesting a device directly, the user:

1. Creates a DeviceClass whose CEL selector picks suitable devices.
2. Creates a ResourceClaimTemplate describing what to allocate from that class.
3. Describes in a Job resource a Pod that references the template and passes the claim from the ResourceClaimTemplate into the container.

Requesting resources in a Pod

spec:
  resourceClaims:                       # Declare the ResourceClaimTemplate for the Pod.
    - name: gpu
      resourceClaimTemplateName: my-claim
  containers:
    - name: app
      image: nvidia/cuda:12.8.0-devel-ubuntu24.04
      resources:
        claims:
          - name: gpu                   # Pass the claim from the ResourceClaimTemplate to the container.

Requesting a whole GPU exclusively

In the example below, the !has(...sharingStrategy) clause makes the allocation exclusive — it excludes the shared variants of the same card (a request for the entire card):

apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: nvidia-whole-gpu
spec:
  selectors:
    - cel:
        expression: |
          device.attributes["gpu.deckhouse.io"].vendor == "nvidia" &&
          device.attributes["gpu.deckhouse.io"].computeAPI == "CUDA" &&
          has(device.attributes["gpu.deckhouse.io"].deviceType) &&
          device.attributes["gpu.deckhouse.io"].deviceType == "physical" &&
          !has(device.attributes["gpu.deckhouse.io"].sharingStrategy)
---
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: whole-gpu
spec:
  spec:
    devices:
      requests:
        - name: gpu
          exactly:
            deviceClassName: nvidia-whole-gpu
            count: 1

Requesting a MIG partition

To request a MIG partition, add the partition attributes to the selector. Profile names here are the hardware names, with dots — 1g.5gb, not 1g5gb:

apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: nvidia-mig-1g5gb
spec:
  selectors:
    - cel:
        expression: |
          device.attributes["gpu.deckhouse.io"].vendor == "nvidia" &&
          device.attributes["gpu.deckhouse.io"].computeAPI == "CUDA" &&
          has(device.attributes["gpu.deckhouse.io"].deviceType) &&
          device.attributes["gpu.deckhouse.io"].deviceType == "partition" &&
          has(device.attributes["gpu.deckhouse.io"].partitionTechnology) &&
          device.attributes["gpu.deckhouse.io"].partitionTechnology == "MIG" &&
          has(device.attributes["gpu.deckhouse.io"].partitionProfile) &&
          device.attributes["gpu.deckhouse.io"].partitionProfile == "1g.5gb"

The claim has the same shape as for a whole card.

The user does not need to run nvidia-smi -mig. The module owns MIG state:

• If the requested partition already exists and is free, it is allocated as-is.
• If it does not exist, the module enables MIG on the card and creates the partition during preparation.
• If the card currently has a different, conflicting layout, the module rebuilds the layout.

A rebuild can only happen if the conflicting layout is not currently claimed. MIG partitions exist only while something holds them — they are created during preparation and destroyed when the claim is released — so if another workload is holding an incompatible partition, your Pod stays Pending until that workload finishes. This is not a failure: the scheduler refuses to allocate the card’s slices twice.

Going back to a whole card

Requesting a physical device on a card that currently has MIG enabled makes the module disable partition mode — again with a drain. Watch the transition with:

d8 k get physicalgpu <name> -o jsonpath='{.status.currentState.partition.mode}'

Requesting a GPU share via MPS

To request a GPU share via MPS, add capacity.requests and an opaque config selecting MPS. For a whole card use GpuConfig:

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: gpu-mps-quarter
spec:
  spec:
    devices:
      requests:
        - name: gpu
          exactly:
            deviceClassName: nvidia-gpu-mps
            count: 1
            capacity:
              requests:
                sharePercent: "25"
                gpu.deckhouse.io/memory: 1Gi
      config:
        - requests: ["gpu"]
          opaque:
            driver: gpu.deckhouse.io
            parameters:
              apiVersion: resource.gpu.deckhouse.io/v1alpha1
              kind: GpuConfig
              sharing:
                strategy: MPS
                mpsConfig:
                  defaultActiveThreadPercentage: 25

The matching DeviceClass must select shared devices, so require the sharing strategy instead of excluding it:

… && has(device.attributes["gpu.deckhouse.io"].sharingStrategy) &&
device.attributes["gpu.deckhouse.io"].sharingStrategy == "mps"

To share a MIG partition rather than a whole card, use kind: MigDeviceConfig and a partition selector. On a partition, sharePercent alone is enough — you do not need to request memory as well.

On a partition only MPS is allowed. Asking for time-slicing there is rejected by the webhook with MIG device sharing supports only MPS strategy.

Workloads sharing the same GPU share one MPS control daemon; it is started on demand and reused. Deleting one of them does not disturb the others.

Requesting a GPU share without specifying a strategy

If you do not care whether you get MPS or time-slicing, request capacity and leave the strategy out:

      requests:
        - name: gpu
          exactly:
            deviceClassName: nvidia-gpu-any
            count: 1
            capacity:
              requests:
                sharePercent: "25"
                gpu.deckhouse.io/memory: 1Gi

This works because a whole card publishes no sharePercent capacity at all — only its shared variants do. Requesting sharePercent therefore forces the scheduler onto a shared variant. If MPS is unavailable on the chosen device, the request lands on time-slicing instead.

For this to work the DeviceClass selector must be permissive: do not add !has(...sharingStrategy), or you exclude the very devices that can satisfy the request and nothing binds.

VFIO passthrough

Passthrough hands the raw PCI device to the container instead of a CUDA device. Inside the container there is no CUDA and no nvidia-smi — you get /dev/vfio/<iommu_group> and /dev/vfio/vfio, for a consumer such as a VM, QEMU/KVM, or DPDK.

apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: h100-vfio
spec:
  selectors:
    - cel:
        expression: |
          device.attributes["gpu.deckhouse.io"].vendor == "nvidia" &&
          has(device.attributes["gpu.deckhouse.io"].productName) &&
          device.attributes["gpu.deckhouse.io"].productName.lowerAscii().matches("^.*h100.*$") &&
          has(device.attributes["gpu.deckhouse.io"].deviceType) &&
          device.attributes["gpu.deckhouse.io"].deviceType == "physical" &&
          has(device.attributes["gpu.deckhouse.io"].bareMetal) &&
          device.attributes["gpu.deckhouse.io"].bareMetal == true
---
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: h100-vfio
spec:
  spec:
    devices:
      requests:
        - name: gpu
          exactly:
            deviceClassName: h100-vfio
            allocationMode: ExactCount
            count: 1
      config:
        - requests: ["gpu"]
          opaque:
            driver: gpu.deckhouse.io
            parameters:
              apiVersion: resource.gpu.deckhouse.io/v1alpha1
              kind: VfioDeviceConfig

Note the absence of capacity and sharePercent: passthrough requires the entire card. A request for the full sharePercent: "100" also counts as exclusive, but anything less is rejected.

The Pod must land on a node prepared for VFIO. Readiness is not published as a device attribute, so it cannot go in the CEL selector — use a node label instead:

spec:
  nodeSelector:
    node.deckhouse.io/gpu-vfio-ready: "true"
  resourceClaims:
    - name: gpu
      resourceClaimTemplateName: h100-vfio
  containers:
    - name: vfio-consumer
      image: busybox:1.36
      command: ["/bin/sh", "-c", "ls -l /dev/vfio && sleep infinity"]
      resources:
        claims:
          - name: gpu

A card in passthrough reports .status.phase: Passthrough and disappears from the CUDA device pool.

Requirements checked during preparation; if any fails, preparation fails with an explicit message:

Preparation also requires SR-IOV to be disabled on the device and the card to be free of running processes. If MIG is enabled but the card is idle, the module tears the MIG configuration down and binds the card to vfio-pci for you.

Module capabilities by GPU vendor

The table describes module capabilities by GPU vendor:

Hardware partitioning gives isolation and predictable performance. Software partitioning gives flexible slicing without isolation, so workloads on the same card affect each other.

Vendor specifics:

• sGPU shares a MetaX card by quota — a percentage of compute and an amount of memory — rather than by fixed slices.
• AMD cards are discovered and appear as PhysicalGPU objects, but cannot yet be allocated.
• Passthrough hands over the whole physical function — it is neither vGPU nor SR-IOV. On NVIDIA it requires a bare-metal node, and an enabled SR-IOV blocks it.

Card capabilities and support are covered in the Glossary.

How the module decides

The two rows are detected differently, and the difference shows up in practice.

Hardware partitioning is detected per card and lands in .status.capabilities.partitionSupport. Software sharing is advertised per node: every card on a node carries the same gpu.deckhouse.io/sharing.mps and sharing.ts labels, regardless of model.

So on a card that cannot do MPS the label still appears and the claim is still accepted. The mismatch surfaces during preparation: the module falls back to time-slicing and emits the event MPSFallbackToTimeSlicing. The workload gets sharing, just not the kind it asked for.

To list all physical GPUs in the cluster and their labels:

d8 k get physicalgpus --show-labels

To get detailed sharing support information (MIG and so on) for a specific GPU:

d8 k get physicalgpu <name> -o jsonpath='{.status.capabilities.partitionSupport}' | jq

Choosing a suitable partitioning type

Hardware partitioning is suitable when you need isolation and predictable performance and the card is on the vendor’s list.

Software partitioning is suitable when hardware partitioning is unavailable on the model, or when its fixed profiles are too coarse for the workload.

Getting information about the allocated GPU

To see which device was allocated:

d8 k -n <namespace> get resourceclaim -o json \
  | jq '.items[].status.allocation.devices.results[] | {device, pool}'

Information about the allocated device is encoded in its name:

To list the objects for which the ResourceClaim is reserved:

d8 k -n <namespace> get resourceclaim <name> -o jsonpath='{.status.reservedFor}'

Inside the container, the usual check:

nvidia-smi -L

If the output is empty, see Troubleshooting — the most common cause is a missing resources.claims entry in the Pod spec.

Device attribute reference

DeviceClass selectors match on attributes in the gpu.deckhouse.io domain, addressed in CEL as device.attributes["gpu.deckhouse.io"].<name>. Always guard optional attributes with has(...) — a missing attribute makes the whole expression fail rather than evaluate to false.

To see what your cluster actually publishes:

To get information about all devices registered in the cluster via DRA:

d8 k get resourceslices -o json \
  | jq '.items[].spec.devices[]? | {name, attributes, capacity}'

Capacity axes

A ResourceClaim can request measurable amounts on two axes: