AI training and inference are billed in different units: training is a run paid by accelerator hours or training tokens that ends, fine-tuning is such a run plus the cost of serving the adapted model, and inference is paid per token or request on an API, or per hour on an endpoint or reserved capacity for as long as the model serves. For cost planning, what a training run costs matters less than which charges keep running afterwards, even without a single request.
How it works
Each phase has its own unit. A training job runs on instances with GPUs or other accelerators and is billed by run time, like the instances themselves (GPU cost). Fine-tuning through a model API costs per training token or per training hour; AWS and Google define training tokens as the tokens in the dataset times the number of epochs. Inference costs per token on an API (tokens, requests and context), per instance or node hour on your own endpoints, and per unit per hour with provisioned throughput (AI and machine learning platform services).
At the model providers themselves the offer is changing: OpenAI is winding down self-serve fine-tuning. Since 2026-05-07 organisations that never fine-tuned cannot create jobs, since 2026-07-02 neither can those that have not used a fine-tuned model in the past 60 days, and from 2027-01-06 active existing customers cannot either; fine-tuned models stay available until their base model is retired. The price list gives training per million training tokens or per hour and only token prices for inference; tokens a grader model uses in reinforcement fine-tuning are billed at that model's rate.
AWS
Amazon SageMaker AI training jobs bill like the service's other instances, by type and run time, plus ML storage per GB-month. A managed spot training job never switches to on-demand instances: if no spot capacity turns up before the maximum wait time it was given, it ends. Checkpoints that SageMaker keeps in Amazon S3 let an interrupted job pick up where it stopped. SageMaker HyperPod pricing excludes connected services such as Amazon EKS, FSx for Lustre and S3.
Amazon Bedrock prices model customisation per 1,000 training tokens (supervised fine-tuning) or per training hour (reinforcement fine-tuning), plus storage per model per month. For Amazon Nova, models adapted with parameter-efficient techniques run on demand or with Provisioned Throughput at base model prices, fully fine-tuned ones only with Provisioned Throughput. Imported models (Custom Model Import) scale to zero after five minutes without an invocation; the example shows how they are billed.
Azure
Training in Azure Machine Learning costs only the resources it uses. For fine-tuning in Microsoft Foundry, Microsoft names three charges: training per token or per hour depending on the model, hosting per hour per deployed model even when unused, and inference per token. Training types are Standard (in the resource's region), Data zone (US only), Global (cheaper, with data and weights copied to the training region) and Developer, on spare capacity, where a job can be interrupted and continued later. The Developer deployment type has no hourly fee, is for evaluation only and ends after 24 hours; Provisioned Throughput is billed in PTUs.
Google Cloud
Custom training on the Gemini Enterprise Agent Platform can run on Spot VMs and Compute Engine reservations; these are billed at Compute Engine prices, for reservations with committed use discounts, plus a platform management fee. Tuning Gemini models costs per 1,000 training tokens. From Gemini 3 on, prediction on a tuned model's endpoint costs 1.5 times the base model's price; older Gemini models keep the base price.
What it means for your cost
Training is project work with an end: run time, data volume, epochs and the number of attempts set its cost. Inference grows with use and runs as long as the feature lives. None of the providers states a fixed ratio between the two. For running cost, what matters is which charges accrue without requests:
| Phase and route | Billing unit | Cost without requests |
|---|---|---|
| Training job (SageMaker AI, custom training) | instance or node time | only while the job runs |
| Fine-tuning through Bedrock or Gemini | training tokens or training hours | Bedrock: storage per model per month |
| Tuned model in Foundry, Standard | hour per deployment plus tokens | yes, the hourly fee |
| Customised model in Bedrock | tokens (on demand) or model unit per hour | yes with Provisioned Throughput |
| Tuned Gemini model from Gemini 3 | tokens at 1.5 times the base model | not according to the price list |
| Endpoint on your own instances | instance or node hour | yes, until removed |
| Provisioned throughput | unit per hour | yes |
A base and a tuned model therefore compare only with every charge included: the tuned model's token price plus its hourly hosting fee divided by that hour's requests, against the base model's token price with its usually longer prompt. What drives token prices per model is covered in AI model pricing and rate cards.
Around the model come further charges: the dataset in object storage, checkpoints, which SageMaker keeps in S3, and evaluation runs. Microsoft's fine-tuning overview counts further training rounds, with their grading and evaluation, among the costs to plan for next to hosting and inference. These charges land on other services than the training and can be tied to the project only through tags or projects.
Where it goes wrong
- A tuned deployment keeps running. An hourly Foundry fee, Bedrock model units or an endpoint's node hours continue with no token lines, often after an experiment ended. Finding such charges is covered in idle and orphaned resources; they are idle cost.
- Base and tuned model compared on token price alone. The tuned model looks cheaper because the shorter prompt counts, but the hosting fee or Gemini's factor of 1.5 does not.
- Training restarts from zero after an interruption. Without checkpoints an interrupted spot job repeats its work, and managed spot training without capacity waits instead of switching to on-demand. When spot capacity pays is covered in spot instances, spot VMs and preemptible capacity.
- Experiments without tags or projects. Training runs, storage and endpoints of different attempts land indistinguishably on one account, and nobody can say what each attempt cost.
- The base model is retired. A tuned model depends on its base model's lifecycle; OpenAI retires fine-tuned models with their base model, for GPT-4o and GPT-4.1 on 2027-01-14. The training cost then falls due again.
Example
List prices in USD, read on 2026-10-16. Supervised fine-tuning of Gemini 3.1 Flash Lite on the Gemini Enterprise Agent Platform costs USD 0.003 per 1,000 training tokens, or USD 3 per million. A dataset of D million tokens trained for E epochs therefore costs 3 × D × E USD; doubling the epochs doubles the amount. A model brought to Bedrock with Custom Model Import costs USD 0.07144 per Custom Model Unit per minute in Europe (Frankfurt); one active five-minute window costs 5 × 0.07144 = USD 0.3572 per Custom Model Unit, plus USD 1.95 storage per Custom Model Unit per month. Bedrock sets the number of units a model needs at import.
Sources
Read on 2026-10-16. AWS Amazon SageMaker AI pricing, managed spot training, checkpoints in Amazon SageMaker AI and Amazon Bedrock pricing; Microsoft Plan and manage costs for Microsoft Foundry, fine-tuning overview and Azure Machine Learning pricing; Google Gemini Enterprise Agent Platform pricing and generative AI pricing on the Agent Platform; OpenAI pricing and deprecations.