Context Compaction and Session Pruning in Production AI Agents: Architecture, Hierarchical Summarization, and Constraint Preservation

Long-running autonomous agents executing multi-step workflows (codebase refactoring, recursive research, multi-turn debugging, and system operations) face an inevitable physical ceiling: working context exhaustion. As an agent executes shell commands, inspects repository trees, parses LSP diagnostics, and consumes tool outputs, the active context window fills rapidly.

When context approaches capacity, systems must decide what to retain, what to summarize, and what to discard. Naive truncation drops early task constraints, while lossy summarization frequently introduces hallucinated completions and drops negative constraints. Furthermore, in systems utilizing prompt caching, ad-hoc client-side context trimming invalidates cache prefixes on every step, increasing operational costs and latency.

Modern production agent runtimes employ layered context management architectures that combine server-side tool result clearing, deterministic deletion-based compaction, hierarchical state summarization, and external artifact offloading.

1. The Anatomy of Agent Working Context

An autonomous agent trajectory is fundamentally different from a standard multi-turn chat transcript. The token distribution within an agent session divides into four distinct structural categories:

• Immutable Invariants and System Directives: The system prompt, tool definitions, operational boundaries, security restrictions, and behavioral guidelines. These tokens establish deterministic execution parameters and must never be altered or pruned.
• Root User Intent and Dynamic Steering: The initial task specification and any out-of-band user corrections delivered mid-trajectory. These define success criteria and explicit constraints.
• High-Entropy Ephemeral Observations: Verbose tool execution results, such as full-file read outputs, raw compiler logs, HTTP response bodies, and directory walks. These data payloads are critical at the moment of execution but rapidly lose informational utility once the model has reasoned over them.
• Intermediate Reasoning Traces and Working State: Model scratchpads, chain-of-thought tokens, extended thinking blocks, and intermediate sub-goal tracking.

In a typical 50-turn software engineering trajectory, high-entropy tool observations account for over 70% of total consumed tokens, while actual decision-making rationale occupies less than 15%. Managing context effectively requires targeting high-entropy ephemeral payload bloat without degrading invariant task constraints.

2. Compaction Taxonomies: Pruning vs. Summarization vs. Deletion

Context management strategies operate across three distinct technical paradigms, each presenting different trade-offs regarding computational overhead, syntactic fidelity, and semantic loss:

+-----------------------------------------------------------------------------------+
|                            AGENT CONTEXT MANAGEMENT                               |
+-----------------------------------------------------------------------------------+
|  1. Mechanical Pruning (Tool/Thinking Clearing)                                   |
|     - Replaces past tool payloads/thinking blocks with minimal placeholders       |
|     - 100% syntactic preservation of remaining turns                              |
|                                                                                   |
|  2. Deletion-Based Compaction (Morph-style)                                       |
|     - Identifies and removes low-signal lines/blocks verbatim                     |
|     - Zero hallucination risk on surviving tokens                                 |
|                                                                                   |
|  3. Hierarchical Semantic Summarization (Deep Agents / ReSum)                     |
|     - LLM synthesizes historical turns into structured state markdown             |
|     - Highest compression ratio; vulnerable to semantic drift & constraint loss   |
+-----------------------------------------------------------------------------------+

Mechanical Pruning (Server-Side Context Editing)

Mechanical pruning identifies specific message blocks that have fulfilled their utility and replaces their payload with a lightweight static marker.

Anthropic introduced server-side context editing primitives through dedicated strategies:

• clear_tool_uses_20250919: Drops older tool invocation results once context exceeds a configured token threshold, preserving only the $N$ most recent tool payloads while retaining tool call metadata.
• clear_thinking_20251015: Strips historical extended thinking tokens from earlier conversational turns while keeping the latest turns intact for continuity.

Because mechanical pruning leaves message envelopes and structural indices intact, the model retains visibility into which tools were invoked without paying the token cost of stale outputs.

Deletion-Based Compaction

Deletion-based compaction identifies redundant, repetitive, or low-signal tokens across code or log streams and deletes them without semantic rewriting. As documented by Morph, compaction operates on the premise that surviving output must remain character-for-character identical to the input.

This approach prevents the subtle hallucination of function signatures, variable names, or line numbers that frequently occurs when an LLM summarizes technical artifacts.

Hierarchical Semantic Summarization

Semantic summarization deploys a secondary LLM call to synthesize an extended execution history into a condensed state representation. In architectures such as LangChain Deep Agents, the runtime splits the context into a historical head and an active tail.

The historical turns are collapsed into a structured progress document containing session intent, completed sub-tasks, modified artifacts, and pending operations.

3. The Prompt Caching Dilemma: Client-Side vs. Server-Side Compaction

In modern LLM inference architectures, prompt caching provides 50% to 90% reductions in time-to-first-token (TTFT) and input token billing by reusing pre-computed key-value (KV) states for shared prefix sequences.

However, naive client-side context compaction acts as a cache-invalidation engine. When a client-side agent framework modifies historical messages in the conversation array (e.g., retroactively truncating turn 4 or inserting a summary at turn 2), it mutates the exact byte prefix. Every subsequent token in the conversation must undergo a full prefill calculation, eliminating cache reuse.

Client-Side Mutation:
Turn 1 [Cache Hit] -> Turn 2 [Edited: Truncated Tool] -> Turn 3 [Cache Miss] -> Turn 4 [Cache Miss]

Server-Side Context Editing:
Turn 1 [Prefix Match] -> Turn 2 [API Clears Payload] -> Turn 3 [Cache Preserved] -> Turn 4 [Cache Preserved]

To preserve cache efficiency, production architectures enforce two operational rules:

1. Append-Only Compaction Milestones: In client-side orchestrations, compressions are appended as state checkpoint blocks at designated intervals rather than continually mutating historical turn arrays.
2. Server-Side API Integration: Leveraging provider-level context management APIs where KV cache lookup occurs prior to server-side tool result stripping, maintaining cache validity across long sessions.

4. Critical Failure Modes in Agent Compaction

Compacting context in autonomous workflows introduces failure modes distinct from standard chat summarization:

• Constraint Evaporation: Research from Penn State and empirical studies on long-horizon agent trajectories show that lossy context compression frequently discards negative constraints (e.g., "do not modify files in /config", "do not use sudo", or "format output according to schema V2"). When an LLM generates a free-form summary of prior turns, it prioritizes progress reporting over operational boundaries. Subsequent agent turns then violate original system directives.
• Pointer and Artifact Invalidation: When an agent reads a file, notes line numbers (e.g., lines 140-165 in parser.py), and executes edits, compacting the file read output can leave the agent with invalid line references. If the summary states "inspected parser.py and identified the regex bug" without preserving exact line offsets or function signatures, subsequent patch attempts fail due to context mismatches.
• Hallucinated Progress and Premature Closure: In multi-turn error recovery loops, an agent may make three failed attempts to resolve an issue before finding a solution. If a summarization prompt naively compresses the sequence into "investigated and addressed the database connection issue", the model on the next turn assumes the fix was verified, bypassing test execution and declaring the task complete prematurely.

5. Production Architecture for Resilient Session Compaction

To mitigate constraint loss and reference drift, production systems employ a structured, multi-tier context lifecycle:

+-----------------------------------------------------------------------------------+
|                        PRODUCTION CONTEXT WINDOW LAYOUT                           |
+-----------------------------------------------------------------------------------+
| [TIER 1: IMMUTABLE ROOT]                                                          |
|  - System Prompts, Security Directives, Tool Schemas                              |
|  - Initial User Prompt & Explicit Negative Constraints                            |
+-----------------------------------------------------------------------------------+
| [TIER 2: STRUCTURED PROGRESS CHECKPOINT]                                          |
|  - Active Objective & Acceptance Criteria                                         |
|  - Verified Completed Milestones (with test proof hashes)                         |
|  - Active File Handles & Persistent Disk URIs                                     |
|  - Unresolved Blockers & Explicit Invariants                                      |
+-----------------------------------------------------------------------------------+
| [TIER 3: ROLLING SLIDING WINDOW (TAIL)]                                           |
|  - Recent N Tool Invocations & Verbatim Responses (raw fidelity)                  |
|  - Active Working Scratchpad                                                      |
+-----------------------------------------------------------------------------------+

Head-and-Tail Sliding Buffers

Rather than summarizing the entire trajectory, systems partition the message array into three zones:

• The Root Head: The initial system instruction and the first user turn containing the master task specification are pinned permanently.
• The Condensed Middle: Older operational turns (turns 2 through K-N) are compacted into a single structured checkpoint.
• The Active Tail: The most recent N turns (typically 10% to 20% of the total context budget) remain in verbatim, uncompressed form to maintain immediate conversational and execution continuity.

Schema-Enforced Compaction Checkpoints

Free-form summaries must be replaced with strict, schema-validated state representations. Compaction prompts enforce structured key-value fields:

## Active Session State
- Master Objective: [Unchanged root goal]
- Immutable Constraints: [Explicit list of forbidden actions, style guidelines, and schemas]
- Completed & Verified Steps:
  * Step 1: Created database schema (verified via test_db.py)
  * Step 2: Implemented auth middleware (verified via curl test)
- Active File References:
  * /repo/src/auth.ts (Modified, exports validateToken)
  * /repo/src/config.ts (Read-only, JWT secret defined at line 42)
- Current Blocker: Token expiry edge case returning 500 instead of 401
- Immediate Next Action: Patch handleTokenExpiry in auth.ts

External Artifact Offloading

Large tool outputs (such as repository searches, code diffs, or database dumps exceeding 2,000 tokens) are written directly to disk storage (e.g., /workspace/.artifacts/<hash>.txt). The agent's context receives only a summary header and a canonical file path handle. If detailed inspection is required later, the agent reads specific byte slices rather than retaining massive payloads in the active context window.

Compaction Trigger Thresholds and Completion Buffers

Triggering compaction at 95% of the context window creates severe operational instability, forcing emergency summarization in the middle of atomic tool-call sequences. Production systems trigger proactive compaction at 65% to 75% utilization. This leaves a 25% to 35% completion buffer, ensuring sufficient headroom for complex reasoning and long generation outputs without mid-task disruption.

6. Context Strategy Comparison

• Server-Side Tool Clearing: Token reduction: 40% to 60%. Syntactic fidelity: 100% on active turns. Cache friendliness: High (preserves prefix KV cache). Risk of constraint loss: Very Low. Ideal for tool-heavy API workflows and coding agents.
• Deletion-Based Compaction: Token reduction: 50% to 70%. Syntactic fidelity: High (verbatim lines preserved). Cache friendliness: Low (requires client-side array mutation). Risk of constraint loss: Low. Ideal for code search, log analysis, and diff inspection.
• Hierarchical Summarization: Token reduction: 70% to 90%. Syntactic fidelity: Low (rewritten semantic state). Cache friendliness: Low to Moderate. Risk of constraint loss: High (requires schema-enforced verification). Ideal for multi-day research and open-ended exploration.
• Head-Tail Partitioning: Token reduction: 60% to 80%. Syntactic fidelity: High (recent tail remains verbatim). Cache friendliness: Moderate. Risk of constraint loss: Low. Ideal for general-purpose autonomous workflows.

Sources

• Anthropic Platform: Context Editing Documentation
• Anthropic Cookbook: Context Engineering, Memory, and Tool Clearing
• LangChain: Context Management for Deep Agents
• LangChain: Context Engineering for Agents
• Acon: Optimizing Context Compression for Long-Horizon LLM Agents (arXiv:2510.00615)
• Context Compression for LLM Agents: A Survey of Methods, Failure Modes, and Benchmarks
• Morph: Context Compaction Architecture and Evaluation