LLM Output Truncation: A Safe Recovery Decision Guide

LLM output truncation should be handled as an incomplete application result, not as a short but successful answer. Inspect the provider’s completion signal, validate the returned content against the task, and either retry within an approved budget or return an explicit incomplete-result outcome. In Microsoft’s chat-completion documentation, a length finish reason means the model output is incomplete because of a completion-token setting or token limit. A response can still contain fluent text, so checking whether the text merely looks finished is not a reliable release gate.
This is a proposed engineering policy, not the result of a measured provider comparison. The provider documentation cited below was consulted on 2026-11-22. No model invocation, latency test or cost benchmark was performed for this article. The application owner should identify the author, reviewer, publication date and revision date in the eventual editorial and change records; none can be inferred from this research. The method applies to applications that can inspect a provider response before presenting or acting on it.
Separate transport from completion
An accepted request and a usable answer are different outcomes. Treat transport success as evidence that a response arrived, then classify the response itself. The classification belongs at the application boundary, before the answer enters a user interface, a document store or a downstream workflow. A sentence ending in a period is not proof of completeness: the requested final item, citation or constraint might still be missing.
Amazon Bedrock’s Converse interface accepts a maxTokens setting in its inference configuration and returns a stopReason in its response. Those fields give a Bedrock integration places to express an output limit and inspect how generation stopped. The Converse API reference documents the request and response shapes. Do not translate one provider’s status values into another provider’s values by string matching alone; write an adapter that maps documented signals into application-level outcomes.
For Microsoft chat completions, a streaming response can have a null finish reason until its final chunk completes the response. The Microsoft chat-completion guide also distinguishes a normal stop from a length outcome and other endings. A consumer that renders streaming text before receiving its terminal signal should label that text provisional. If the connection closes without a final signal, classify the result as unknown rather than guessing that generation ended normally.
Use a response decision table
The following table is an original, hypothetical application policy. It is not a report of provider behavior observed in a test. Configure each row for the API and model your application actually uses, and keep any answer with an unresolved outcome away from actions that require a complete result.
| Observed application outcome | Proposed decision | Evidence to retain |
|---|---|---|
| Documented normal completion; task checks pass | Release the answer | Terminal signal and validation result |
| Documented output-limit completion | Withhold the answer; consider one controlled recovery attempt | Limit setting, terminal signal and recovery decision |
| Terminal signal missing after an interrupted stream | Return an incomplete or unknown outcome | Last received event and connection outcome |
| Content omitted for a filtering reason | Apply the application’s safety policy, not a larger token allowance | Provider classification and policy decision |
| Normal completion but required content missing | Fail task validation and investigate separately | Validation failure without storing sensitive answer text |
The distinction in the final row matters. A provider can report normal completion while the application still receives an unusable answer. Conversely, a length outcome must not be promoted to success merely because an early sentence answers part of the question. Keep provider completion and task validation as separate fields so reviewers can identify which boundary failed.
Design a bounded recovery
Start with the consequence of a partial answer. A chat interface may show a clearly marked incomplete result; an automated approval or document-generation workflow may need to reject it entirely. Record that choice before adding retries. For a recovery attempt, check whether the selected model and API permit a larger output allowance, whether the application’s own deadline leaves room, and whether repeating the request could duplicate a downstream action. Raising a limit is a configuration decision, not proof that the next answer will be complete.
- Inspect the terminal provider signal through the adapter and keep an explicit unknown state for missing signals.
- Run task-specific checks on the complete response: required fields, requested scope and any evidence checks appropriate to the workload.
- If the output limit was reached, decide whether to request a fresh answer with an approved allowance or ask for a narrower result. Never append a continuation and assume the combined text is coherent without validating it.
- Keep a recovery budget and deadline at the application layer. On exhaustion, return a labelled incomplete outcome rather than silently releasing the partial answer.
- Review recovered answers against the same task checks as first attempts. Record the disposition, not a fabricated success rate.
A hypothetical support-summary workload illustrates the boundary. Suppose a fixture asks for a concise summary of an approved, non-sensitive document and requires a final statement identifying uncertainty. If the response ends before that statement and reports an output-limit completion, the application withholds it. A fresh request with an approved allowance is eligible only if the deadline and policy permit it. If that attempt still fails validation, the test expects an incomplete-result message. This fixture defines expected behavior; no provider execution or performance result is claimed.
For retrieval-backed summaries, a complete generation signal does not prove that the cited material supports the answer. Use the existing RAG citation verification test plan as a separate evidence gate. If a recovery attempt depends on retrieving or rebuilding context, the idempotent chunk indexing guide addresses a different failure boundary. Neither check can be replaced by increasing the model’s output allowance.
Test the release boundary
Exercise the policy with synthetic inputs in a controlled environment before relying on it. Supply an adapter fixture with a documented normal-completion signal, an output-limit signal and a stream that lacks a terminal signal. For each case, assert both the application classification and what the user or downstream service receives. Then add a normally completed response that omits a required task element. That case catches an implementation that equates provider completion with task success.
Review what the test retains. A correlation identifier, selected model reference, configured allowance, completion classification and validation disposition may be sufficient for triage. Do not copy prompts, retrieved documents, credentials or personal information into a general-purpose log simply to diagnose truncation. Set retention and access controls under the workload’s own policy.
The release decision is narrow: the adapter recognizes the tested terminal outcomes, the application withholds incomplete or unknown results where required, and the recovery path respects its approved budget and deadline. Repeat these tests when the model, API, streaming client or validation rules change. This procedure does not establish a universal token allowance, guarantee a complete answer or measure recovery cost.
Sources
- AWS: Converse API reference. Consulted 2026-11-22.
- Microsoft: Work with chat completion models. Consulted 2026-11-22.
Keep exploring
Artificial Intelligence
Test Bedrock Geo Inference Failover Without Leaving EU
Artificial Intelligence
RAG Ingestion Retries: Idempotent Chunk Indexing Guide
Artificial Intelligence
RAG Citation Verification: A Test Plan for AI Teams
Artificial Intelligence