Cloud AI services are priced in two ways: the platforms Amazon SageMaker AI, Azure Machine Learning with Microsoft Foundry and the Gemini Enterprise Agent Platform bill the running time of instances or nodes for notebooks, training and endpoints, while prebuilt AI services and models bill per token, character or audio hour. The difference decides the bill: a deployed endpoint costs every hour whether requests arrive or not, a price per unit only when used.
How it works
Four billing patterns occur: instances for notebooks, training jobs and real-time endpoints cost per unit of time while they exist; serverless and asynchronous inference bills by duration and data and can drop to zero without load; prebuilt services such as translation or speech recognition cost per unit processed; language models cost per token (Tokens, requests and context) or, with provisioned capacity, per reserved unit per hour.
AWS
Amazon SageMaker AI has its own pricing page; the page of the unified Amazon SageMaker with Unified Studio mostly points to the prices of the services used. SageMaker AI bills the chosen instance type by duration for Studio notebooks, Processing, Training, Batch Transform and Real-Time Inference, so a real-time endpoint costs as long as its instances run. Asynchronous Inference can autoscale to zero instances, and Serverless Inference bills compute time by configured memory plus data processed, optionally with Provisioned Concurrency. Amazon Bedrock bills models per token in the Standard, Flex, Priority and Reserved tiers, batch inference for selected models at a lower price, and Provisioned Throughput per model unit per hour with no commitment or with one or six months. Customised models cost training plus monthly storage, imported models per Custom Model Unit per minute in five-minute windows. Prebuilt services such as Amazon Translate bill per character.
Azure
The Microsoft Foundry platform itself is free; each service in it has its own pricing. Models sold by Azure, including Azure OpenAI, are billed by Microsoft per token; models from partners and the community go through Azure Marketplace, their meters appear under the resource group, and Azure Prepayment cannot pay for them. A fine-tuned model adds an hourly hosting charge per deployment, even without calls. Foundry Tools, formerly Azure AI services, bill per character or audio hour; Azure Speech in Foundry Tools, for instance, bills real-time transcription per audio hour, and custom speech models add training per compute hour and endpoint hosting per model per hour, neither available in Germany West Central (the pricing page shows N/A there). Azure Machine Learning adds no charge of its own: the virtual machines cost the VM price, plus the Blob Storage, Key Vault, Container Registry and Application Insights the workspace uses.
Google Cloud
Google now calls Vertex AI the Gemini Enterprise Agent Platform, and its pricing page says Gemini platform for short. Training and prediction have no minimum duration and are charged in 30-second increments. Node hours count while a node stands ready, and every model deployed to an endpoint costs money without predictions until it is undeployed; unlike its predecessor, the platform does not scale to zero. Batch prediction jobs are charged after they complete. Workbench instances cost the Compute Engine price plus a management fee while they run, disks also while stopped. Generative models have their own pricing page with token prices, and Cloud Translation bills per character, with 500,000 characters free per month.
| Provider | Service | Billed unit | Cost without requests |
|---|---|---|---|
| AWS | SageMaker AI Real-Time Inference | instance hour | yes, while the endpoint runs |
| AWS | SageMaker AI Serverless Inference | compute time by memory, data | no, unless Provisioned Concurrency |
| AWS | Bedrock | token, model unit per hour | with Provisioned Throughput |
| Azure | Foundry Models | token, hosting per hour | for fine-tuned models |
| Google Cloud | online prediction | node hour in 30-second increments | yes, per deployed model |
| Google Cloud | batch prediction | node hour | no |
What it means for your cost
SageMaker AI bills instance and GPU hours under the ML service at its own prices, not under Amazon EC2. On Google this holds for training and prediction on on-demand nodes, while Workbench costs the Compute Engine price plus a management fee, and Azure Machine Learning bills the virtual machines at the VM price. SageMaker AI names instances with an ml. prefix before the familiar type name, such as ml.m5.xlarge; the logic of the families is in Virtual machines and instance families. On Google, billing splits across two SKUs once reservations or Spot VMs are used: a Compute Engine SKU labelled vertex-ai-online-prediction and the management fee SKU. Interruptible capacity is covered in Spot and preemptible capacity; commitments for ML instances such as SageMaker Savings Plans are covered in AWS Savings Plans.
Token costs appear as Bedrock or Foundry lines on the cloud bill, while models bought directly from OpenAI or Anthropic arrive on a separate invoice outside the cloud (AI provider billing) with their own usage export (Usage and cost exports of the AI providers). Models that run on the GPU nodes of your own cluster show up there instead (Containers and Kubernetes).
An endpoint on fixed instances costs per hour regardless of load, so the fewer requests it serves, the higher the cost per prediction. Unit economics therefore has to count the idle hours. Microsoft Foundry tags the usage of every project automatically with a project tag (Preview) that Cost Management can filter on, so far only for models sold by Azure.
Where it goes wrong
- An endpoint left running overnight. Symptom: hosting or node hours run around the clock with few requests. Cause: real-time endpoints cost per instance hour, and on Google every deployed model costs money with no scaling to zero.
- A forgotten fine-tuned model. Symptom: a hosting meter in Foundry runs with no token lines. Cause: deployed fine-tuned models cost per hour even when nobody calls them.
- A notebook over the weekend. Symptom: notebook or Workbench hours on days without work. Cause: the instance is billed for running time, not for use.
- A budget alert after the batch job. Symptom: a large prediction job breaks the budget before any alert arrives. Cause: Google charges batch prediction after the job completes, and budget alerts do not fire while it runs.
- Partner models missing from the project view. Symptom: costs by Foundry project tag fall short of the invoice. Cause: attribution by project covers only models sold by Azure; Marketplace models sit under the resource group.
Example
List prices in USD, read on 2026-10-16: SageMaker AI Real-Time Inference in Europe (Frankfurt) USD 0.276 per hour for ml.m5.xlarge and USD 1.761 per hour for ml.g5.xlarge; Gemini Enterprise Agent Platform prediction node e2-standard-4 in Frankfurt (europe-west3) USD 0.198568 per node hour; Azure Speech in Foundry Tools in Germany West Central USD 1 per hour of real-time transcription.
Sources
Read on 2026-10-16. Amazon SageMaker AI pricing, Amazon SageMaker pricing, Amazon Bedrock pricing, Amazon Translate pricing, Microsoft Foundry pricing, Plan and manage costs for Microsoft Foundry, Azure Machine Learning pricing, Foundry Tools pricing, Azure Speech in Foundry Tools pricing, Gemini Enterprise Agent Platform pricing, Cloud Translation pricing.