Attention-FFN disaggregation (AFD) is the next architectural split beyond prefill-decode disaggregation for large mixture-of-experts (MoE) models: it separates the attention modules from the feed-forward expert modules onto distinct GPU pools inside the decode phase, fusing the extra activation transfer with the all-to-all dispatch that MoE expert parallelism already requires. The …
Tuning vLLM KV Cache and Preemption in Production Serving
vLLM’s PagedAttention and iteration-level continuous batching are the two mechanisms that take an autoregressive transformer from the 20–40% GPU utilization typical under static batching toward the 2–4× throughput the SOSP 2023 paper reports. Production value does not come from enabling them — both are on by default — but from …
Claude Code + Playwright: O Stack Definitivo para Web Scraping com IA em 2026
Claude Code + Playwright: O Stack Definitivo para Web Scraping com IA em 2026 Web scraping tradicional está morrendo.
FinOps for AI: Engineering Cost Out of Inference
Cloud FinOps has spent years building cost discipline around virtual machines, storage tiers, and reserved instances. Then AI inference arrived and rewrote the spending model. GPU-hour billing, token-based pricing, and model serving infrastructure follow cost patterns that traditional FinOps tooling was never built to track. The result is a widening …
AI Agent Reliability: SLOs That Survive Production
A production AI agent ran at 99.4% uptime for a quarter, returned HTTP 200s on every request, and was functionally broken for three weeks — the model had silently regressed when a cheaper variant was swapped in, and the pipeline kept emitting success codes for outputs no human could use. …
Gemma 4 vs DeepSeek V4: O Grande Confronto dos Modelos Open Source em 2026
Gemma 4 vs DeepSeek V4: O Grande Confronto dos Modelos Open Source em 2026 O ecossistema de modelos open source nunca esteve tão competitivo.
Gemma 4 vs DeepSeek V4: Comparativo Completo dos Melhores Modelos Open Source de IA em 2026
Gemma 4 vs DeepSeek V4: Comparativo Completo dos Melhores Modelos Open Source de IA em 2026 O ecossistema de IA open source nunca esteve tão competitivo. E
Model Distillation: 32B Beats o1-Mini at Half the Cost
A 32-billion-parameter student model fine-tuned on DeepSeek-R1’s reasoning traces scored 72.6% Pass@1 on AIME 2024, beating OpenAI’s o1-mini (63.6%) while costing roughly an order of magnitude less per token to serve. Released with the DeepSeek-R1 checkpoints, it is the strongest production evidence yet that knowledge distillation—not larger GPUs—is the dominant …
LLM Routing Cuts 85% of API Spend. Here’s the Engineering.
LLM routing — the practice of classifying each incoming request and directing it to the cheapest model that can handle it — reduces inference API costs by 40–85% while retaining 95–98% of frontier-model quality, according to RouteLLM benchmarks from LMSYS and 2026 production data from gateway providers like Requesty and …
Trainium2 Halves Frontier Compute. Neuron Is the Catch
Anthropic now runs roughly 65% of its $4.0–4.5 billion annualized compute bill on AWS Trainium2 silicon, paying an effective committed rate near $0.50 per chip-hour versus $2–$5 for reserved H100 capacity — about half the cost per FLOP of OpenAI’s Azure-NVIDIA stack. That is a structural margin advantage for Claude …
Cloud GPU Capacity Up 20% at AWS: Engineers Adapt Now
On July 1, 2026, AWS raised EC2 Capacity Blocks for ML prices by roughly 20% per accelerator-hour, its second hike in six months after a ~15% increase in January (Converge Digest). A reserved P6-B300 (Blackwell Ultra) now costs $14.04/hr. Cloud GPU capacity — the scarce fabric every foundation-model training and …
Spot GPUs Save 70% on Paper. Production Says Otherwise.
Amazon SageMaker’s managed spot training can cut ML training cost up to 90% over on-demand — but only if your job checkpoints. AWS explicitly caps non-checkpointing spot jobs at MaxWaitTimeInSeconds of 3600 seconds, so any LLM fine-tune longer than one hour that cannot resume from a checkpoint is structurally impossible …