What the scheduling stack does, and where it stops
The GPU scheduling stack improved sharply in the last year. Dynamic Resource Allocation went GA in Kubernetes 1.34, replacing the decade-old device-plugin model where a GPU was an opaque integer count.¹² A workload can now request a device by memory, MIG profile, or NVLink topology and have the scheduler satisfy the constraint at bind time.² KAI Scheduler adds gang scheduling, hierarchical queues, and priority preemption, running alongside kube-scheduler rather than replacing it, opt-in per pod through schedulerName.³ MIG slices an H100 into isolated tenants, time-slicing and MPS handle softer sharing, and DCGM reports a hundred-plus metrics per device.⁴⁵ This is real progress, and none of it is what the execution fabric asks anyone to remove.
All of it takes one thing as given: the pod. DRA describes the hardware better, KAI places the pod more intelligently, MIG carves the card, but the unit being scheduled is still the container handed to the cluster, with its CPU stages and GPU stages bundled inside it. The scheduler decides which node that pod lands on and when. It does not reach inside the pod to send the CPU preprocessing to CPU capacity while the GPU stage goes to an accelerator. That is the boundary of what a placement engine does, not a gap in configuration.
The cost of that boundary shows up in the dashboards. A bundled pod holds a GPU node for its whole life even when the GPU stage is a fraction of the wall-clock time, so mixed clusters show DCGM reporting activity on a device the scheduler counts as busy while average utilization across production fleets sits near 5 percent.⁶⁷ The everyday version: a job requests four GPUs, runs briefly, and leaves them allocated while another team's work queues behind capacity that exists but is locked up, because standard scheduling treats allocated as unavailable.⁷
What changes
The execution fabric changes one thing: the path a unit of work takes from the pod to the hardware. It sits beneath the orchestrator and above the silicon. The scheduler still issues intent, deciding what runs, at what priority, against which quota. Beneath that decision, the fabric breaks the unit into smaller pieces, sends each to the resource best suited to run it, and runs it there, across whatever hardware is in the fleet.
In practice, the CPU stages of a job stop riding on a GPU node. The GPU stages consolidate onto fewer accelerators that stay fed. Work that today forces a whole-accelerator reservation, held against a burst that rarely arrives, can be placed against real capacity across the fleet instead of pinned to one node. The scheduler's decision is unchanged. The decision now executes against the right resource instead of the resource the pod happened to be packaged for.
This is the move the field already validated at the serving layer, generalized below the pod. NVIDIA shipped Dynamo to general availability in March 2026 to split prefill and decode across separate hardware pools, and the Kubernetes-native way to run it, llm-d, Grove PodCliqueSets, the Gateway API Inference Extension, is a stack assembled by hand today, pool by pool, with separate tooling for routing and lifecycle.⁸⁹ The fabric does that disaggregation as a property of where it sits, not as a deployment maintained per workload.
What does not change
No rip-and-replace. The fabric does not replace kube-scheduler, KAI, Volcano, or Kueue. It runs beneath them. The scheduler keeps scheduling.
No migration. Adoption is per-workload, the same opt-in pattern already used to route training to KAI and inference to kube-scheduler by schedulerName.³ One workload points at the fabric, gets measured in existing DCGM numbers, and the result decides whether a second one follows. Nothing forces a cluster-wide cutover, and nothing breaks the workloads left alone.
No new observability island. The fabric reports against the metrics already scraped. DCGM stays the source of truth for device utilization, and the question that matters, did real utilization rise on hardware already paid for, is answered in the existing dashboard.⁵ If the number does not move, the workload backs out.
No loss of control. Quotas, priorities, queues, namespaces, RBAC, and gang-scheduling constraints stay in the scheduler. The fabric does not arbitrate fairness between teams or decide who preempts whom. Those policies keep living in the control plane that owns them.
Where it fits against the alternatives
There are three other ways to chase the same utilization. Each does real work and each stops at the same wall.
Better placement, through DRA and KAI, packs pods onto devices more tightly and gang-schedules distributed jobs so a partial allocation does not sit idle waiting for the rest.³ It cannot reach inside a pod that is mostly CPU work to free the GPU it holds.
GPU sharing, through MIG, MPS, time-slicing, or middleware like HAMi, puts more than one workload on a card. It helps when many small jobs each need a slice, but the sharing is coarse and the isolation is a real hazard: time-slicing gives no memory isolation, so one workload can out-of-memory its neighbors, and KAI's fractional sharing does not enforce a memory limit between processes.¹⁰¹¹ That trades utilization for blast radius.
Utilization-aware preemption, published by HPE as a scheduler plugin, queries DCGM and evicts a low-priority pod holding a GPU it is not using.⁷ It attacks the idle-allocation problem directly, but it reclaims by killing, depends on hand-tuned usage thresholds, and still operates on whole pods.
The fabric is the move none of these make. It does not place the bundled unit better, share the card harder, or evict the idle holder. It changes the unit, so CPU work and GPU work go to different resources in the first place and the GPU stops being held by work that never needed it. The other three manage the symptom of a bundled unit. This addresses the bundling.
The operational case
The cost of trying is bounded. One workload, opted in by the existing mechanism, measured in trusted metrics, backed out cleanly if the number does not move. No cluster bet, no second control plane to learn, no fairness policy handed to a vendor. The test is whether a unit of work that today holds the wrong resource for most of its life can be made to hold the right one, with the answer read off the existing dashboard.
If it works, the recovered utilization is on hardware already racked, powered, and paid for, which is the only capacity available without a purchase order and a multi-year wait for power. If it does not, one workload reverts. That asymmetry is the case.
The point
The execution fabric is not a new scheduler and not a migration. It sits beneath the one already in place and changes how a unit of work reaches the metal, so CPU stages and GPU stages stop sharing a node and the accelerator stops being held by work that does not need it. The scheduler, queues, quotas, metrics, and policies all stay. Adoption is one workload at a time, measured in existing numbers, reversible by default. The scheduler keeps deciding what runs. The fabric changes what the scheduler is scheduling.
A note on the numbers
The figures here are reported with their source and what they measure. The 5 percent figure is fleet-level average GPU utilization across tens of thousands of production clusters, a different quantity from the per-device numbers DCGM shows on an active node; the two are cited separately and not merged. The DRA, KAI, MIG, and Dynamo capabilities are described from primary and practitioner sources current to Q1 and Q2 2026, and version-specific details, Kubernetes 1.34 for DRA general availability, will move as the stack moves. The sharing and isolation limits, no memory isolation under time-slicing, no enforced memory limit under KAI fractional sharing, are documented behaviors, not projections. As across the series, the argument does not rest on any single number. It rests on the shape of all of them together: a scheduling stack that improved a great deal and still takes the bundled pod as the unit it places.
References
- Dynamic Resource Allocation graduated to general availability in Kubernetes 1.34, enabled by default, replacing the count-based device-plugin model with attribute-rich resource claims. KubeNatives; CloudOptimo on the 2026 production stack. KubeNatives CloudOptimo
- DRA lets a workload request a GPU by memory capacity, MIG profile, or NVLink topology, satisfied at bind time, versus the legacy device plugin's opaque integer count shipped in 2017. Spheron Kubernetes GPU orchestration guide (Apr 2026). Spheron
- KAI Scheduler adds gang scheduling, hierarchical queues, and priority preemption, runs alongside kube-scheduler rather than replacing it, and is opted into per pod via schedulerName; donated to the CNCF and accepted as a Sandbox project at KubeCon EU 2026. TechPlained (Apr 2026); Rafay on the KubeCon EU 2026 contributions. TechPlained Rafay
- The NVIDIA GPU Operator installs and manages the node stack (driver, container toolkit, device plugin, feature discovery, MIG manager, DCGM exporter); MIG partitions an H100 into isolated tenants, with time-slicing and MPS for softer sharing. KubeNatives; TechPlained. KubeNatives TechPlained
- DCGM exporter provides a hundred-plus GPU metrics per device via Prometheus and is the standard source of truth for device utilization in production clusters. Introl Kubernetes GPU orchestration; TechPlained. Introl TechPlained
- Average GPU utilization of 5 percent across tens of thousands of production clusters, with mixed clusters showing high per-device memory use while the scheduler counts the node as busy. Cast AI 2026 State of Kubernetes Optimization Report; dasroot analysis of mixed-workload underutilization. Cast AI dasroot
- Idle-but-allocated GPUs (a job requests four, runs briefly, leaves them held) and the standard scheduler treating allocated as unavailable; the ReclaimIdleResource plugin querying DCGM to preempt low-priority pods not using their GPUs. CNCF / HPE engineers (Jan 2026). CNCF
- NVIDIA Dynamo reached general availability at GTC on March 16, 2026, splitting prefill and decode across separate hardware pools. NVIDIA Dynamo product page and design docs. NVIDIA NVIDIA docs
- Kubernetes-native disaggregated inference assembled from separate components (llm-d, Grove PodCliqueSets, the Gateway API Inference Extension for KV-cache-aware routing), wired together per deployment. Spheron Kubernetes GPU orchestration guide. Spheron
- Time-slicing provides no memory isolation, so one workload can out-of-memory others sharing the GPU; MIG is recommended for production isolation. K8s Recipes on KAI GPU sharing and bin packing (Jan 2026). K8s Recipes
- KAI Scheduler fractional GPU sharing allocates by requested memory or fraction but does not enforce a memory allocation limit or isolate memory between processes. KAI Scheduler GPU-sharing documentation. KAI Scheduler