Cloud Run GPU Inference: Limits, Cold Starts, Costs

Cloud Run GPU attaches an NVIDIA accelerator to a fully managed container service: no drivers to install, no reservations to book, and a service that can scale down to zero between requests. For bursty inference traffic, the two facts that shape the architecture are that GPU instances with drivers pre-installed start in approximately 5 seconds, so a request-driven inference endpoint does not need a permanently warm fleet — and that everything around the accelerator (memory floors, billing, quotas) pushes back against large idle fleets. This guide walks through the limits Google documents, the costs that follow from them, and a checklist for deciding between this model and a fully managed model service such as Amazon Bedrock.

What Cloud Run GPU provides

Cloud Run exposes two accelerator classes for services. Google provides NVIDIA L4 GPUs with 24 GB of GPU memory and RTX PRO 6000 Blackwell GPUs with 96 GB, and the platform enforces one GPU per Cloud Run instance: multi-GPU instances are not an option, and when you run sidecar containers the accelerator attaches to a single container only. The layer underneath is fully managed — Google installs the NVIDIA driver (version 580.x.x, CUDA 13.0), so your container image only carries the application and the model runtime. Minimum instance sizes scale with the card: an L4 service needs at least 4 vCPU and 16 GiB of memory, while the RTX PRO 6000 Blackwell card requires at least 20 vCPU and 80 GiB.

Cold starts and scale to zero

Google’s documented number covers GPU readiness, not your application: instances with an attached L4 or RTX PRO 6000 Blackwell GPU and drivers pre-installed become usable by container processes in approximately 5 seconds, and model weight loading happens after that point. As a sizing example, not a measured case study: if your container needs 20 seconds to load a 7-billion-parameter model into VRAM, first-token latency on a cold instance is roughly 25 seconds, and the standard mitigation — a minimum-instance floor — is a fixed cost that the billing section shows is not free. Scale to zero is the feature that makes spiky, internal, or batch-triggered inference cheap, because idle periods cost nothing beyond storage. Sustained traffic flips the math toward always-on capacity, whether that is provisioned throughput on a model service or dedicated GPU fleets elsewhere in your infrastructure. If the deployment also has to prove that the code touching model weights is exactly the workload you built, attestation is a separate layer on top of the managed fleet — see confidential computing attestation for AI inference.

Billing rules that change sizing

Cloud Run GPU pricing is instance-based, not request-based. Google’s pricing notes state that there are no per request fees, but you must use instance-based billing, minimum instances are charged at the full rate even when idle, and the GPU is billed for the entire duration of the instance lifecycle — not just while a request executes. Two consequences follow. First, any min-instances floor is a fixed cost line on the invoice, so treat it as a reserved-capacity decision rather than a cold-start patch. Second, zonal redundancy is on by default: Cloud Run reserves GPU capacity for your service across multiple zones in the region, which costs more per GPU second but significantly raises the odds of surviving a zone failure; disabling it lowers unit cost and leaves failover best-effort. For latency-sensitive inference serving, that trade-off belongs in the availability budget, not in a cost spreadsheet alone.

Quotas, regions, and instance limits

GPU capacity on Cloud Run is bounded twice: the maximum-instances setting caps instances per service, and per-project GPU quota caps instances across every service in the region. Your first deployment in a region comes with an automatic first-use grant of 3 L4 GPUs, while the RTX PRO 6000 Blackwell card grants 3,000 milliGPUs — the equivalent of 3 GPUs; anything larger requires a quota increase request through your account team. Region choice matters for EU teams: L4 is available in europe-west1 (Belgium) and europe-west4 (Netherlands), plus Asia and US regions, whereas the Blackwell region list is shorter — within the EU, europe-west4 only. Google is also explicit about capacity risk: during a zonal outage, GPU failover depends on unused capacity in the remaining zones, and that capacity might not always be available because GPUs are constrained resources.

Decision checklist

Work through these six questions before committing a serving architecture:

1. Do you need a custom container at all? If you only call hosted foundation models, a fully managed model service is the lighter path: Amazon Bedrock is a fully managed service that provides secure, enterprise-grade access to high-performing foundation models, and AWS owns the entire serving fleet on your behalf.
2. Does traffic actually burst? Scale to zero plus roughly 5-second GPU readiness suits spiky workloads; steady traffic belongs on cheaper always-on capacity.
3. What is the min-instances floor for? Every idle minimum instance bills at the full GPU rate, so size it against the cost of a cold start, not against peak load.
4. Will quota hold at peak? Three L4 GPUs per region is the starting allowance; request an increase before launch, not after the pager fires.
5. Is your region supported? L4 runs in Belgium and the Netherlands; the Blackwell card only in the Netherlands within the EU.
6. Where does the team’s operational experience live? Teams rooted in the AWS ecosystem weigh these trade-offs differently, and that depth is uneven in practice — the field report on AWS training in Brazil and the real practitioner experience is a useful reality check.

Sources

• Google Cloud — GPU support for services (consulted 2026-11-06)
• Amazon Web Services — Amazon Bedrock Overview (consulted 2026-11-06)