Dataflow pricing is vCPU-hours and GiB-hours of the workers a job runs on, by the second, at a batch rate, a cheaper FlexRS rate for batch jobs that may wait, and a higher streaming rate, plus the data the service shuffles per GiB in batch or the Streaming Engine compute units in streaming, the workers' disks per GiB-hour, and GPUs per hour; Dataflow Prime replaces the resources with data compute units. Managed Service for Apache Spark, formerly Dataproc, charges serverless jobs per data compute unit-hour and clusters a management fee per vCPU-hour on top of the Compute Engine price of every node; Managed Service for Apache Airflow, Cloud Composer, charges an environment per compute unit-hour and database storage, or in its second version the environment fee by size plus the workers' CPU, memory and storage. On the bill, Dataflow streaming jobs that never stop and Dataproc clusters that nobody deletes dominate, and a Composer environment costs its fee every hour whether a DAG runs or not.
Pricing dimensions
Google prices the three services by the resources the job occupies and the time it occupies them, and every service and rate has its own SKU per region; the Compute Engine nodes behind Dataproc clusters keep their own SKUs.
| Dimension | Unit | What drives it | Optional |
|---|---|---|---|
| Dataflow workers | vCPU-hour and GiB-hour per worker, by the second, at the batch, FlexRS or streaming rate; streaming with one- or three-year commitments | The job's worker count, machine type and runtime; autoscaling bounds | No |
| Dataflow Shuffle and Streaming Engine | GiB of data shuffled in batch and FlexRS; Streaming Engine compute units per hour in streaming, or legacy streaming data processed per GiB | Group-by and join stages, state size, message volume | No, the service-based variants are the default |
| Dataflow Prime | Data compute units per hour in batch and streaming instead of vCPU, memory and shuffle | Jobs run on Prime | Yes |
| Dataflow disks and GPUs | GiB-hour of Persistent Disk or Hyperdisk per worker; GPU per hour by type with a Dataflow premium | Worker disk size, accelerators | Yes |
| Dataproc serverless | Data compute unit-hour at the standard or premium tier, by the second with a one-minute minimum, plus shuffle storage per GiB-hour; one- or three-year commitments | Spark batches and sessions without a cluster | Yes |
| Dataproc clusters | Management fee per vCPU-hour across master, worker and secondary workers, by the second with a one-minute minimum, plus every node at the Compute Engine price with its disks; Spot secondary workers at the Spot price | Clusters from creation to deletion, idle or not | Yes |
| Composer 3 | Compute units per hour for the environment's scheduler, workers, web server and triggerer, scaled with the load, plus database storage per GiB-month | Environment size and DAG load | No |
| Composer 2 | Environment fee per hour by size (small, medium, large) and resilience, plus workers' vCPU-hours, GiB-hours and storage, and database storage | Environment size, worker autoscaling bounds | No while the environment exists |
| Around the jobs | Cloud Storage for inputs, staging and checkpoints, BigQuery storage and queries, Pub/Sub throughput, egress | Every pipeline's sources and sinks | Yes (BigQuery pricing, Pub/Sub and Eventarc pricing) |
| Purchase option | Dataflow streaming and Dataproc serverless commitments over one or three years; the compute flexible CUD does not cover them | Steady streaming and Spark load | Yes (Google Cloud committed use discounts) |
Time is the price: a Dataflow batch job pays its workers from the moment they start to the moment they stop, so the runtime and the worker count are the bill, and a streaming job pays them around the clock, at a rate about a quarter above batch, with the Streaming Engine's compute units on top for the state and shuffle it keeps in the service. FlexRS cuts the batch rate by about a third by scheduling the job within six hours on preemptible and regular workers together, which suits nightly jobs and not deadlines. Dataproc clusters are Compute Engine with a management fee: every node pays its instance price whether a job runs or not, the fee adds a cent per vCPU-hour, and a cluster left running over the weekend pays the weekend; serverless Spark replaces that with data compute units billed per second while the batch runs, at a rate that includes the machine, so it costs more per unit and nothing between jobs. Composer is the opposite: an environment is a scheduler, a database and workers that run all the time, so its fee or its compute units accrue every hour whether a DAG runs or not, and the load adds workers on top; small environments pay the base, large ones the workers. The mechanics across all three providers are in data warehouses and analytics.
How it appears in your billing data
In the billing export the Dataflow rows carry service.description Cloud Dataflow and a sku.description per resource and mode: vCPU Time Batch Frankfurt, RAM Time Batch Frankfurt, vCPU Time Streaming Frankfurt, RAM Time Streaming Frankfurt, vCPU Time FlexRS Frankfurt, Shuffle Data Processed Frankfurt, Streaming Engine Compute Unit Frankfurt, Local Disk Time PD Standard Frankfurt, Data Compute Unit Batch Frankfurt for Prime; usage.unit is hour, gibibyte-hour, gibibyte or count. resource.name in the export with resource-level detail names the job, and the job's labels in labels are the attribution per pipeline and team; without resource-level detail, a project's jobs sum into one row per SKU and day. Dataproc rows carry service.description Cloud Dataproc with Licensing Fee for Cloud Dataproc per vCPU-hour for clusters, while the nodes sit under Compute Engine with the label goog-dataproc-cluster-name that Dataproc sets; serverless batches appear as Dataproc Serverless Standard DCU Frankfurt and Dataproc Serverless Shuffle Storage. Composer rows carry service.description Cloud Composer with Cloud Composer 3 Standard DCU Frankfurt and Cloud Composer 3 Database Storage, or for the second version Cloud Composer Compute CPUs, Cloud Composer Compute Memory, Cloud Composer Compute Storage and Small Cloud Composer Environment Fee; resource.name names the environment.
Commitments appear as credits.type COMMITTED_USAGE_DISCOUNT beside the streaming or serverless rows, and the Compute Engine nodes of a Dataproc cluster carry sustained use and Spot credits as any instance. The export's structure is described in the Google Cloud billing export; in the FOCUS export, ServiceName carries Cloud Dataflow, Cloud Dataproc or Cloud Composer, ServiceCategory Analytics, SkuId the SKU and ChargeCategory Usage (FOCUS columns). What a job cost is also in the job's own metrics, the Dataflow job page shows the billed resources per job and Dataproc the batch's DCU-hours.
Levers
Configuration: on Dataflow the first lever is the worker machine type and the autoscaling maximum, because a job that scales to 100 workers for a skewed stage pays 100 workers; the second is FlexRS for batch jobs that can wait six hours; the third is the Streaming Engine and Dataflow Prime, which move state and shuffle off the workers and let the workers be smaller. On Dataproc the lever is the cluster's life: ephemeral clusters per job, deleted by the workflow, or serverless batches, and Spot secondary workers for the part of the cluster that may be preempted; an idle cluster pays its nodes and its fee for nothing, as idle and orphaned resources describes. On Composer the lever is the environment size and the number of environments: one environment for many DAGs, sized by the schedulers and workers the load needs, not one per team.
Purchase option: commitments on Dataflow streaming and Dataproc serverless for the baseline that runs around the clock; batch jobs and clusters take the Compute Engine commitments and Spot of their nodes (commitment strategy).
Architecture: a streaming job only where latency needs it, because the same data processed hourly in batch pays a fraction; serverless Spark for jobs that run minutes, clusters for jobs that run hours on packed nodes; and the pipeline's storage in the region of the job, because every shuffle to a bucket elsewhere pays egress (network pricing). What the nodes cost is in Compute Engine pricing.
Where it goes wrong
- The streaming job runs for a feature nobody ships. A streaming pipeline pays its workers and compute units every hour, for months. The job list by start date names the old ones; drain and cancel.
- The cluster survived the job. A Dataproc cluster created by hand pays its nodes and fee until deleted. Workflow templates with deletion, or an idle-deletion timer, end it.
- Autoscaling to the skew. One hot key makes the job scale to its maximum for one stage and pay all the workers for the whole stage. A lower maximum and a repartition are the fix.
- Batch at the streaming rate. A batch job written as an unbounded pipeline pays streaming rates and the Streaming Engine. Bounded sources and batch mode cost a quarter less.
- The shuffle is the bill. A job that groups terabytes pays the shuffle per GiB on top of the workers. Combiners before the group-by and the Streaming Engine lower it.
- A Composer environment per team. Every environment pays its fee or compute units all month. One environment with folders per team carries the DAGs.
- Workers sized for the peak. Composer workers at the maximum all month pay for the nightly peak around the clock. Autoscaling bounds set to the measured load.
Example
List prices in USD for Frankfurt (europe-west3), read on 2026-10-21 from the Dataflow, Managed Service for Apache Spark and Managed Service for Apache Airflow pricing pages: Dataflow batch workers cost USD 0.0672 per vCPU-hour and USD 0.0042684 per GiB-hour, FlexRS USD 0.0433 and USD 0.0027499, streaming USD 0.0828 and USD 0.0042684, USD 0.06624 and USD 0.04968 per vCPU-hour with a one- and three-year commitment; batch shuffle USD 0.01417 per GiB, streaming data processed USD 0.02319 per GiB, a Streaming Engine compute unit USD 0.105 per hour, a Prime batch data compute unit USD 0.071 per hour; worker Standard Persistent Disk USD 0.0000648 per GiB-hour. Dataproc serverless costs USD 0.077307 per standard data compute unit-hour and USD 0.114672 per premium one, shuffle storage USD 0.000065753 per GiB-hour, the cluster management fee USD 0.01 per vCPU-hour. Composer 3 costs USD 0.0708 per compute unit-hour and USD 0.000274795 per GiB-hour of database storage; a Composer 2 small environment fee USD 0.45 per hour, medium USD 0.71, large USD 1.10, workers USD 0.058 per vCPU-hour and USD 0.006 per GiB-hour.
A nightly Dataflow batch job on 20 workers n2-standard-4 that runs two hours and shuffles 500 GiB costs 30 × (2 × 20 × (4 × 0.0672 + 16 × 0.0042684) + 500 × 0.01417) = 30 × (13.48 + 7.09) = USD 617.08 a month; on FlexRS USD 408.40; the same pipeline as a streaming job on five such workers around the clock 730 × 5 × (4 × 0.0828 + 16 × 0.0042684) = USD 1,458.03 plus the Streaming Engine's units. A Dataproc cluster with one master and four workers n2-standard-4 that runs around the clock costs 730 × 5 × 0.250248 = USD 913.41 of nodes plus 730 × 20 × 0.01 = USD 146.00 of fee; the same cluster for the job's two hours a day USD 60.06 plus USD 9.60; the job as a serverless batch with 16 DCUs for two hours a day 30 × 2 × 16 × 0.077307 = USD 74.21. A Composer 2 small environment with two workers of 2 vCPUs and 8 GiB costs 730 × 0.45 = USD 328.50 of fee plus 730 × 2 × (2 × 0.058 + 8 × 0.006) = USD 239.44 of workers, USD 567.94 a month before a DAG runs. The figures show that the clock, not the data, carries these bills, that a cluster deleted after the job costs a tenth of one left running, and that an orchestrator pays its base every hour.
Sources
Read on 2026-10-21. Prices: Dataflow pricing, Managed Service for Apache Spark pricing, Managed Service for Apache Airflow pricing. Service: Dataflow FlexRS, Dataflow Streaming Engine, Dataproc cluster scheduled deletion. Billing data: structure of the standard Cloud Billing data export.