As large language models take on longer multi-turn workflows and agentic tasks, systems increasingly rely on context compaction to summarize past conversation histories and keep prompt lengths within memory budgets. However, this compression comes at a severe cost to user control.
A new study by Pennsylvania State University researchers Zhiqi Wang, Yichi Zhang, Dongwon Lee, and Yuchen Yang reveals that standard context compaction drops an average of 83 percent of user-defined operational constraints. In their benchmark evaluations, only 17 percent of session constraints survived typical summarization routines.
The Fragility of Session Constraints
The researchers distinguish between core task instructions (such as writing code or summarizing documents) and session constraints. Session constraints represent dynamic behavioral rules provided during an interaction, such as requiring confirmation before modifying calendars, refusing to delete emails, or masking sensitive identifiers.
Because context summarizers are designed to preserve forward task momentum, current system state, and immediate objectives, they consistently discard negative rules and side constraints as non-essential historical context. Once a session undergoes compaction, the underlying model often operates as if those rules were never established.
This behavioral loss creates critical reliability and security hazards in autonomous agent deployments. When side constraints disappear, agents can perform unauthorized tool calls, skip required human-in-the-loop validation steps, or expose withheld internal variables.
Benchmarking Compaction Integrity with COMPINT
To quantify the scope of the problem, the Penn State team introduced COMPINT, an evaluation suite assessing compactors across three primary long-context environments: agent execution trajectories, extended research sessions, and multi-turn conversational chat.
Key empirical findings from the study include:
- Baseline Compliance: In uncompressed contexts where full conversational history is retained, models complied with session constraints 59 to 71 percent of the time.
- Post-Compaction Degradation: Following standard compaction, constraint retention fell to an average of 17 percent. Downstream compliance dropped close to control baselines where no constraint had been provided.
- Prompting Limitations: Applying targeted prompt engineering to instruct summarizers to preserve user constraints yielded marginal improvements, with constraint retention remaining below 40 percent.
Plug-and-Play Mitigation via Small Language Models
Rather than requiring architectural modifications to base models or retraining context compactors, the researchers designed an auxiliary constraint extractor.
Operating as an independent sidecar module powered by Qwen3.5-9B, the system scans incoming user turns to detect and register session constraints into an explicit list. When the context window reaches compaction thresholds, this isolated constraint list is appended directly to the generated context summary.
In benchmark tests, this dual-stream approach achieved over 90 percent constraint retention across all evaluated workloads:
- Agent Trajectories: 95.6 percent retention
- Long-Horizon Research Tasks: 95.1 percent retention
- Multi-Turn Chat Sessions: 90.3 percent retention
The authors have released the COMPINT evaluation suite and reference implementation on GitHub.
Sources
- arXiv: Lost in Compaction: Evaluating Side-Constraint Loss under Context Compaction (arXiv:2608.11242)
- Code repository: GitHub ZhiqiEliWang/compaction-integrity
- Secondary reporting: The Decoder
