Cloud AI costs split into two distinct billing models: per-token inference on managed model APIs and per-hour compute on GPUs you schedule yourself. Engineers in Portugal evaluating a production deployment usually compare these without realizing they behave like completely different cost curves. Token pricing scales linearly with usage and hides nothing; GPU hours scale with time, whether or not the accelerator is doing useful work. Choosing the wrong model for your workload pattern is the most expensive architectural mistake in this space, more than any vendor or region choice.
This analysis breaks down how the two models actually bill, what levers exist on each platform, and which EU compliance dates change the total cost of ownership for teams shipping cloud AI costs under the AI Act. For the wider delivery model, see how cloud AI delivers AI capabilities as a service.
Two Bills for the Same Model
The same open-weight model can appear on a managed API, on a serverless endpoint, or on a dedicated GPU instance, and each path produces a different invoice. A managed API bills per million input and output tokens, so your unit economics are driven by prompt length and output verbosity. A dedicated endpoint bills per hour of accelerator time, so your unit economics are driven by utilisation. At low and bursty volume, token pricing wins almost every time. At sustained high volume with predictable traffic, dedicated capacity becomes cheaper per request — but only after you profile real throughput.
Amazon Bedrock lists Flex tier and Batch pricing at 50% discount to Standard tier pricing, which is the single largest lever for non-latency-sensitive workloads. Batch document processing, nightly summarization, and dataset evaluation do not need interactive latency, and routing them to the discounted tier halves the dominant line item. Conversely, Priority tier pricing carries a 75% premium for guaranteed throughput, which only makes sense when a latency budget is contractual.
Safety layers also bill separately. Content filters on Bedrock Guardrails cost $0.15 per 1,000 text units, and the charge applies only to the filters your guardrail actually invokes. A pipeline that runs both content filters and denied-topic filters on 3,000 text units per hour accumulates roughly $0.90 per hour in guardrail charges alone — small in isolation, but material when multiplied across every request in a high-volume system.
Token Pricing in Practice
When you model token costs, four variables dominate: input tokens, output tokens, caching behaviour, and tier selection. Output tokens are consistently priced higher than input tokens across providers, often by a factor of three to five, so response-length discipline in prompt engineering has direct financial impact. Prompt caching, where available, discounts repeated input prefixes and is one of the most underused cost controls in RAG architectures that resend the same system prompt on every call.
The table below summarizes how the two billing models compare on the dimensions engineers most often get wrong:
| Dimension | Per-token managed API | Per-hour GPU instance |
|---|---|---|
| Cost driver | Tokens in and out | Elapsed accelerator time |
| Idle cost | Zero when unused | Full rate while running |
| Best fit | Bursty, variable traffic | Saturated, steady throughput |
| Discount levers | Batch/Flex tiers, caching | Commitments, spot capacity |
| Scaling latency | Provider-managed | Your autoscaling logic |
Notice that the discount levers column is where most savings live. A team that moves batch workloads to a discounted tier and trims output tokens routinely cuts spend by half without touching architecture.
GPU Hours and Billing Increments
On the compute side, granularity matters more than headline hourly rates. Google Cloud Agent Platform charges no minimum usage duration for training and prediction: usage is billed in 30 second increments instead. That granularity changes deployment strategy — an endpoint that can scale down between bursts pays only for the fractional windows it actually serves, while platforms with one-hour minimums charge full hours for the same spiky pattern.
Training costs are equally concrete. AutoML image model training on Agent Platform is listed at $3.465 per hour, while deployment and online prediction for classification runs at $1.375 per hour. For custom training, you sum the machine-type hourly rate plus separate accelerator charges, and spot VMs follow Compute Engine spot pricing for interruptible training runs. The practical consequence: a training pipeline that checkpoints and retries on spot capacity can absorb interruptions and pay a fraction of on-demand rates, while a pipeline without checkpointing pays full price for every restart. Billing increments and commitment tiers matter more than the vendor logo on the invoice.
Compliance Dates That Matter
Cost analysis for EU teams is incomplete without the regulatory calendar, because compliance work is a real cost line. Strict obligations for high-risk AI systems under the EU AI Act start applying on 2 December 2027, which is the date most relevant to production cloud AI deployments in Portugal. Systems that fall into high-risk categories — CV-sorting tools, credit scoring, and safety components in critical infrastructure — will need risk assessment, logging, documentation, and human oversight before they can be placed on the market. Deployers using managed APIs inherit part of the burden from the model provider, but the deployer obligations around logging and oversight remain with the team operating the system.
Transparency obligations arrive earlier, in August 2026, covering disclosure when humans interact with chatbots and labelling of AI-generated content. Rules for general-purpose AI models took effect in August 2025, which is why managed providers now publish training-content summaries. If your architecture decision depends on where data is processed and who is the provider versus the deployer, read our analysis of EU cloud AI compliance after the 2026 deadline.
A Practical Cost Checklist
Before committing a budget, run through this ordered procedure:
- Classify each workload as latency-sensitive or batch, and route batch traffic to a discounted tier such as Bedrock Flex or Batch at half the standard rate.
- Measure real input and output token distributions over a week; multiply the 95th percentile by your projected volume to model the managed API bill.
- Profile GPU memory and throughput utilisation of any dedicated endpoint; if sustained utilisation sits below roughly 50 percent, per-token serving is usually cheaper.
- Verify the billing increment of your platform before committing to short-lived endpoints, since fractional increments and per-hour minimums change burst economics dramatically.
- Price in guardrail and safety filters as separate line items, not as noise on the model bill.
- Map each system against the AI Act risk categories and budget the documentation, logging, and oversight work ahead of the December 2027 deadline.
Teams that follow this sequence typically discover that their real cost structure is tier selection and utilisation, not the choice between two flagship models. Both managed APIs and dedicated GPUs can be cheap or ruinous depending on how traffic matches the billing model.