Kubernetes GPU cost is the hourly price of the GPU nodes, and it accrues while the node runs, not while the GPU computes: a pod that requests a whole GPU and uses a fifth of it pays for all of it. Because a GPU instance costs ten to forty times an ordinary node, the unused GPU is the largest single waste in a cluster with accelerators. This page describes how the providers bill GPU nodes, how time-slicing and multi-instance GPU (MIG) share one GPU between pods and what that means for allocation, why idle dominates, what dynamic resource allocation changes, and what a cost agent inside the cluster can and cannot see about GPUs. GPU capacity as a purchase decision, reservations and quotas are GPU cost in the AI chapter.

How it works

A GPU reaches a pod through an extended resource: the vendor's device plugin reports a count of nvidia.com/gpu per node to the kubelet, and a container requests a whole number of them, no fractions and no limit above the request. The scheduler places the pod on a node with a free GPU, and from then on the GPU belongs to the container whether it computes or waits. The price sits with the node: a GPU instance bundles GPU, vCPU, memory and often local storage into one hourly price, and for allocation the GPU is the most expensive of the resources the node price is split into (what a pod costs). The OpenCost specification values a GPU like CPU, at the greater of request and usage; because the request is an integer, it is almost always the greater.

Two methods share a physical GPU between pods. Time-slicing lets the driver switch the GPU between containers in time: the device plugin reports a configured number of replicas instead of one GPU, each container requests one of them and gets the whole GPU for its slice, with no memory limit and no isolation between containers. MIG partitions a GPU of the A100, H100 and newer classes into up to seven instances with their own memory and compute units, isolated in hardware; each instance appears as its own resource such as nvidia.com/mig-1g.10gb. For allocation: with time-slicing a pod's share is its share of the replicas, not of the compute time; with MIG it is the instance's share of the GPU, by memory or compute units.

not at alltime-slicingMIGA physical GPU on a node,its price inside the node'sShared how?A whole GPU percontainer request,computing or waitingReplicas reported insteadof one GPU: each containergets the whole GPU forits slice, no memorylimit, no isolationUp to seven instances withtheir own memory andcompute units, isolated inhardware, each a resourceof its own
A GPU belongs whole to the container that requests it, time-slicing shares it in time without isolation, and MIG partitions it into instances of its own

Dynamic resource allocation (DRA), stable since Kubernetes 1.35, replaces the integer extended resource with ResourceClaim objects through which a pod requests a device with properties, and several containers or pods share the same device through one claim. Allocation thereby gains an object that says which pod holds which device or part of it.

AWS

GPU instances of the G and P families carry the GPU in the instance price and appear in the Cost and Usage Report as EC2 instance hours like any other; split cost allocation data spreads instances with accelerators always by requests, never by usage. EKS Auto Mode ships the GPU drivers and the device plugin, with its management fee per instance.

Azure

AKS GPU node pools use the N series with NVIDIA GPUs and are billed as virtual machines, Spot VMs included; AKS can install the device plugin, and MIG profiles are set per pool.

Google Cloud

GKE Standard bills GPU nodes as Compute Engine instances with an attached GPU, Autopilot bills accelerator pods with a premium per GPU plus requests; time-sharing and MIG are built into GKE and configured per node pool.

What it means for your cost

A GPU's utilisation is in no billing file. What a cost agent in the cluster sees is the pod's request, the node and its price, that is allocation by request; what it sees only with driver metrics, say from the DCGM exporter, is the utilisation of compute units and memory, that is the idle inside the request. Without those metrics every report shows a full GPU per pod and the dominant item stays invisible. With them, GPU idle becomes a number per workload: requested GPU hours times the share in which the GPU did not compute.

without the metricsThe billing file's node and price,and the pod's request as thecost agent sees itAllocation by request: a fullGPU per pod that requests oneDriver metrics, say from theDCGM exporter: compute unitsand memory in useGPU idle per workload: requestedGPU hours times the share inwhich the GPU did not computeThe dominant itemstays invisible
Billing data and the request show a full GPU per pod; only driver metrics turn requested GPU hours into the idle inside them

The levers follow. Inference services with low utilisation share a GPU by time-slicing where latency allows, or by MIG where isolation matters; batch jobs run on spot GPU nodes whose interruption a checkpoint absorbs (spot and preemptible nodes); a GPU node pool with a taint and a minimum of zero keeps a node from running on without a GPU pod (node pools and instance selection). The rule for shared GPUs, by replicas, by MIG instance or by measured utilisation, is an allocation rule with a date, and distributing by usage needs the metrics that allocation by request does not.

Where it goes wrong

  • Full GPU, little load. Symptom: an inference service holds a GPU around the clock at single-digit utilisation. Cause: the request is an integer; without sharing every pod pays for a whole GPU.
  • GPU node without a GPU pod. Symptom: a GPU node runs for days with system pods only. Cause: the pool's minimum is one, or a pod without a GPU request landed there because the taint is missing.
  • Time-slicing mistaken for isolation. Symptom: one container pushes the others out of GPU memory. Cause: time-slicing enforces no memory limit; only MIG isolates in hardware.
  • Allocation without utilisation. Symptom: every report shows GPU cost equal to the node price, idle is missing. Cause: the agent sees requests, not driver metrics; utilisation is not exported.
  • Spot GPU without checkpoints. Symptom: a training run starts over after every interruption. Cause: the job writes no checkpoints; the cheaper hour costs the lost compute time.
  • Wrong instance for the task. Symptom: an inference service runs on a training GPU with 80 GB of memory for a 6 GB need. Cause: the family was chosen by availability, not by the model's memory need.

Example

List prices read on 2026-10-21: in Frankfurt (eu-central-1) a g5.xlarge (1 NVIDIA A10G GPU with 24 GB, 4 vCPU, 16 GiB) costs USD 1.258 per hour on demand under Linux and a g6.xlarge (1 NVIDIA L4 GPU) USD 1.0064; in Germany West Central a Standard_NC24ads_A100_v4 (1 NVIDIA A100 GPU with 80 GB, 24 vCPU) costs USD 4.775 per hour regular and USD 0.88242 as a Spot VM, read through the Azure Retail Prices API.

A pod holding the GPU of a g5.xlarge around the clock costs 1.258 × 730 = USD 918.34 a month; if the GPU computes 20 percent of the time, USD 734.67 of that is idle only driver metrics make visible. If four services share the GPU by time-slicing with four replicas, each carries a quarter by request, USD 229.59. An A100 at 4.775 × 730 = USD 3,485.75 a month, partitioned by MIG into seven instances, costs USD 497.96 per instance by share; as a Spot VM the same machine costs 0.88242 × 730 = USD 644.17 a month as long as it is not reclaimed.

Sources

Read on 2026-10-21. Kubernetes scheduling GPUs and dynamic resource allocation, NVIDIA time-slicing GPUs in Kubernetes and MIG support in Kubernetes, OpenCost specification, AWS split cost allocation data, AWS Price List bulk API, offer AmazonEC2, eu-central-1, Microsoft GPUs on AKS and Azure Retail Prices API, Google Cloud GPU time-sharing in GKE and multi-instance GPUs in GKE.