In a technical report published on August 18, 2026, researchers at IBM Research detailed empirical evaluations on sizing and calibrating agentic memory across eight large language models. The study, conducted using the open-source ALTK-Evolve framework across the AppWorld benchmark, demonstrates that agentic memory performance is governed by capability-dependent dosage rather than uniform prompt accumulation.
Agentic memory architectures typically extract procedural guidelines from prior execution trajectories and re-inject them into context during subsequent inference tasks. By evaluating models ranging from 117-billion-parameter open-weight architectures to proprietary frontier systems on AppWorld's 585 multi-step interactive tasks, the study identified three distinct operational regimes.
Three Regimes of Memory Capacity
The research evaluated two primary retrieval patterns against a baseline lacking historical memory: full guideline injection across every ReAct step versus curated retrieval, which pairs a compact static core with cosine-similarity-retrieved task rules.
- Strong Models with Headroom: High-capacity frontier models absorbed comprehensive guideline sets effectively. DeepSeek-V3.2 (671B MoE) improved Task Goal Completion (TGC) by 9.5 percentage points (79.8% to 89.3%) and Scenario Goal Completion (SGC) by 16.1 percentage points (64.3% to 80.4%) when supplied with its complete mined guideline set. Claude Opus 4.6 recorded a 4.1 percentage point TGC increase (90.5% to 94.6%) and a 7.1 percentage point SGC improvement (87.5% to 94.6%).
- Selective / Constrained Models: Smaller or intermediate models degraded when presented with extensive guideline corpuses. For gpt-oss-120b (117B MoE), injecting full memory sets introduced interference and elevated prompt token consumption by 50%. In contrast, curated retrieval increased TGC by 16.1 percentage points (39.9% to 56.0%) and SGC by 16.1 percentage points (21.4% to 37.5%) with an incremental token overhead of only 5%.
- Saturated Regimes: Highly capable models operating near task ceilings exhibited negligible deltas. GLM-5 (745B MoE) remained invariant across configurations at 87.5% TGC and 80.4% SGC, indicating that general self-distilled guidelines failed to resolve remaining edge failure modes.
Mechanism and Implementation
ALTK-Evolve operates entirely around model weights, extracting behavioral heuristics from both successful and unsuccessful run trajectories on training splits. Because no parameter updates occur, the system relies on prompt management techniques to balance cost and latency.
For strong models requiring broad context, the authors highlight prompt caching as essential to maintaining production viability. For smaller serving footprints, selective retrieval achieves equivalent or superior task completion while keeping context inflation minimal.
The researchers note that cosine similarity remains an imperfect proxy for guideline utility and cited outcome-supervised guideline selection as a priority for future iterations.
