SageMaker endpoint idle cost is the instance hours of an inference endpoint that stands ready and gets no request; the same pattern is node hours of a Vertex AI endpoint with a deployed model, provisioned throughput units on Azure OpenAI and Microsoft Foundry that run hourly whether tokens flow or not, and model units with an hourly price on Amazon Bedrock. The saving comes from an endpoint without invocations being torn down, scaled to zero or replaced by a serverless or token-based variant, and from reserved throughput without load going back to a pay-per-token deployment. The fix belongs to the team that runs the model, because only there is it known which latency a user expects.

Symptom

The bill shows endpoint and capacity hours without invocations. On AWS these are lineItem/UsageType in the pattern EUC1-Host:ml.g5.xlarge under AmazonSageMaker with product/component Hosting, at 720 hours per endpoint, and under AmazonBedrock the hours per model unit of a provisioned throughput. On Azure they are MeterCategory Azure OpenAI or Foundry with PTU hour meters per deployment that start at creation and end at deletion. On Google Cloud Vertex AI carries the SKUs for online prediction per machine type and node hour, from deployment to undeployment. What AI services charge in principle is in AI and machine learning services and in the glossary under provisioned throughput unit; this page covers endpoints and throughput without use.

Created, deployedor boughtHours count from then on:Host hours per endpoint,hours per model unit, PTUhours per deployment,node hoursUntil deletion orundeploymentInvocations besidethem: noneS
Endpoint and throughput hours run from creation to deletion whether invoked or not, so hours without invocations are the symptom

How to detect it

AWS

The endpoints with status and their instances:

aws sagemaker list-endpoints --status-equals InService --query 'Endpoints[].[EndpointName,CreationTime,LastModifiedTime]' --output table
aws sagemaker describe-endpoint --endpoint-name <endpoint> --query 'ProductionVariants[].[VariantName,CurrentInstanceCount,CurrentWeight]' --output table

One endpoint's invocations over two weeks, as a sum per day:

aws cloudwatch get-metric-statistics --namespace AWS/SageMaker --metric-name Invocations \
  --dimensions Name=EndpointName,Value=<endpoint> Name=VariantName,Value=AllTraffic --statistics Sum \
  --start-time 2026-10-07T00:00:00Z --end-time 2026-10-21T00:00:00Z --period 86400

Compute Optimizer classifies a SageMaker endpoint as idle that had zero invocations over 14 days, and recommends scale-to-zero for inference components or the deletion (read on 2026-10-21): aws compute-optimizer get-idle-recommendations --filters name=ResourceType,values=SageMakerEndpoint. Reserved throughput on Bedrock is billed per model unit and hour, deletable at any time without a commitment, only afterwards with a one or six month commitment, and runs until it is deleted (Provisioned Throughput in Amazon Bedrock):

aws bedrock list-provisioned-model-throughputs --query 'provisionedModelSummaries[].[provisionedModelName,modelArn,modelUnits,commitmentDuration,status]' --output table

In billing data the candidate is a Host line with full hours and no Invocations, and a Bedrock line per model unit with no token lines beside it (AWS Cost and Usage Report).

Azure

The deployments per resource with SKU and capacity:

az cognitiveservices account deployment list --name <account> --resource-group <rg> \
  --query "[].{name:name, model:properties.model.name, sku:sku.name, capacity:sku.capacity}" -o table

Deployments with the SKUs ProvisionedManaged, DataZoneProvisionedManaged and GlobalProvisionedManaged are reserved throughput: they are billed per PTU and hour regardless of the tokens consumed, from creation to deletion, and a reservation over one month or one year lowers the rate without securing capacity (provisioned throughput in Microsoft Foundry). The metric Provisioned-managed Utilization V2 per deployment shows the utilisation; a deployment that stays below 10 percent for days is the candidate. Azure Advisor recommends reservations for provisioned throughput, not a shutdown. In the cost export the candidate is a deployment with a PTU hour meter and no token meter (Azure cost exports).

Google Cloud

The endpoints per region and the deployed models with machine type and replicas:

gcloud ai endpoints list --region=<region> --format="table(name.basename(),displayName,createTime,deployedModels[].id.list())"
gcloud ai endpoints describe <endpoint-id> --region=<region> --format="yaml(deployedModels[].dedicatedResources)"

A deployed model bills node hours by machine type and minReplicaCount from the moment of deployment until undeployment, requests or not; the metric aiplatform.googleapis.com/prediction/online/prediction_count per endpoint over two weeks is the evidence. There is no recommender for endpoints. In the billing export the candidate is an online prediction SKU with full node hours per resource.name (Google Cloud billing export).

How to fix it

  1. Record per endpoint and deployment the invocations per day and the latency requirement; an endpoint with zero invocations over 14 days is torn down, one with a few invocations a day changes its mode of operation, one with steady load stays and gets the right size.
  2. Tear down: aws sagemaker delete-endpoint --endpoint-name <endpoint>, with delete-endpoint-config where needed; aws bedrock delete-provisioned-model-throughput --provisioned-model-id <id> for throughput without a commitment; az cognitiveservices account deployment delete --name <account> --resource-group <rg> --deployment-name <deployment>; gcloud ai endpoints undeploy-model <endpoint-id> --region=<region> --deployed-model-id=<deployed-model-id>. None of these calls has a dry run; the list from the detection stands in for it, and the model artifacts stay in S3, Blob Storage or the model registry.
  3. Change the mode of operation where the load is irregular: SageMaker Serverless Inference or an endpoint with inference components and a minimum instance count of zero, which scales to zero when idle; on Azure a standard deployment per token instead of PTUs, with spillover as a buffer for peaks; on Bedrock on-demand per token instead of model units; on Vertex AI a smaller machine type, fewer replicas or a batch prediction job instead of a standing endpoint.
  4. Correct the template: endpoints from experiments get an expiry date and a tag with an owner; reserved throughput is bought only once a token deployment has shown the load over weeks, by the rule of provisioned throughput against pay-as-you-go.

The hours end with the teardown; the daily export shows it the next day.

nonesomeyesno: a few a dayPer endpoint and deployment:invocations per day andthe latency requirementInvocationsover 14 days?Tear down: no dry run, thelist stands in; the modelartifacts staySteady load?Stays and getsthe right sizeChange the mode: serverlessor scale to zero, per tokeninstead of PTUs or modelunits, batch instead ofa standing endpointCorrect the template: anexpiry date and owner tagfor experiments; reservedthroughput only after weeksof shown load
No invocations means teardown, irregular load changes the mode, steady load keeps the endpoint, and the template guards new ones

Risks and side effects

  • Zero on the endpoint means a cold start at the first invocation: minutes for a large model until the instance is up and the weights are loaded. A service with a latency commitment needs the instance or a serverless variant with provisioned concurrency.
  • Reserved throughput with a commitment runs to the end of its term, load or not; the question is then whether other workloads use it, not whether it is deleted.
  • Capacity is not assured: a deleted PTU deployment or a released GPU instance may not be available later in the same region. Whoever only wants to save time scales down instead of deleting.
  • Idle GPU instances are the same pattern at another level; how training and serving clusters use their GPUs is in GPU cost.
  • Model artifacts and logs of a deleted endpoint stay in storage; their retention is a case of its own (object storage without lifecycle rules).

How to verify the saving

Before: per endpoint and deployment, the instance, node or unit hours and their amount in the last full month, beside the invocations and the utilisation. After: the same resources in the first full month after the teardown or change; the hour line has to fall to zero, and a token or serverless line that grows with the invocations takes its place. The hour line without invocations is the estimate, the team's release makes it accepted, the teardown implemented, and the following month's invoice realises it in the pattern of estimated, accepted, implemented, verified. The utilisation of reserved throughput and the number of endpoints without invocations are the figures for the cost review.

Sources

Read on 2026-10-21. Compute Optimizer idle resource recommendations, Provisioned Throughput in Amazon Bedrock, aws sagemaker, Provisioned throughput in Microsoft Foundry, Azure Advisor cost recommendations, az cognitiveservices account deployment, Vertex AI, compute resources for inference, gcloud ai endpoints.