The headline number everyone quotes for LLM-as-Judge is 80%: GPT-4 agrees with human evaluators roughly 80% of the time, the same rate at which human annotators agree with each other. That figure comes from Lianmin Zheng and colleagues’ 2023 MT-Bench study, built on about 3,000 expert votes, and it made …
Editorial team 
Schema-Valid LLM Output Still Gets 20% of Values Wrong
97% JSON Pass, 20% Wrong Values The Structured Output Benchmark (SOB), published in April 2026, evaluated 21 frontier LLMs on schema-constrained extraction tasks. The result that should stop you from shipping: every model clears 97%+ on JSON Pass Rate, but Value Accuracy — whether the extracted values are actually correct …
Rate Limits Hit 60% of LLM Errors. Retries Amplify Damage
The Scale of the Problem In February 2026, nearly 60% of all LLM production errors tracked by Datadog were caused by rate limits — not model failures, not hallucinations, not context window overflows. Rate limits. HTTP 429s. By March that number dropped to 30%, but organizations still logged approximately 8.4 …