Agentic Memory and Context Management Systems in Production: Comparing Letta, Zep, Mem0, and LangMem

Stateless large language model APIs present a fundamental bottleneck for autonomous agents operating across extended multi-turn sessions: context window exhaustion, quadratic attention overhead, and memory drift. While standard Retrieval-Augmented Generation (RAG) retrieves static document chunks based on semantic similarity, autonomous agents require dynamic, stateful memory capable of updating beliefs, invalidating superseded facts, tracking chronological state transitions, and preserving operational constraints.

Four major open-source memory architectures have emerged to address agent state management: Letta (the evolution of MemGPT), Zep (powered by Graphiti), Mem0, and LangMem. Each framework adopts a fundamentally distinct mental model, varying from operating system virtual memory paging to temporal knowledge graphs, two-phase candidate extractors, and graph-native storage primitives.

1. Letta (MemGPT): OS-Style Virtual Context Management

Introduced by Packer et al. (2023), MemGPT established the concept of virtual context management, treating the LLM context window as physical RAM and external storage as a persistent disk hierarchy. Letta extends this architecture into a standalone stateful agent operating system.

+-------------------------------------------------------------+
|                      LETTA RUNTIME                          |
|                                                             |
|  +-------------------------------------------------------+  |
|  | Context Window (RAM)                                  |  |
|  |  - System Instructions                                |  |
|  |  - Core Memory Blocks (Human Persona / Agent Persona)  |  |
|  |  - FIFO Working Message Buffer (FIFO Eviction)        |  |
|  +-------------------------------------------------------+  |
|                             ^                               |
|       Tool Calls (core_memory_replace, archival_search)     |
|                             v                               |
|  +---------------------------+  +------------------------+  |
|  | Recall Memory (Disk Log)  |  | Archival Memory (HNSW) |  |
|  | Full message history      |  | Vector embeddings for  |  |
|  | SQL / Postgres storage    |  | unbounded knowledge    |  |
|  +---------------------------+  +------------------------+  |
+-------------------------------------------------------------+

Memory Hierarchy

Letta organizes agent memory into three distinct tiers:

• Core Memory (In-Context Working Memory): Read and write memory blocks embedded directly into the system prompt. Typically partitioned into human (user profile and preferences) and persona (agent identity and operational rules). The LLM autonomously modifies these blocks via function calls such as core_memory_append and core_memory_replace.
• Recall Memory (Conversation History Log): An append-only relational store capturing the chronological transcript of all previous user, agent, and tool interaction events. Allows the agent to paginate backward in conversation history using structured search filters.
• Archival Memory (Vector Store): An unbounded, vector-indexed database for long-term document fragments, facts, and external knowledge. The agent queries this tier via archival_memory_search and writes via archival_memory_insert.

Execution Dynamics and Self-Editing

In Letta, context management is agent-driven rather than middleware-driven. When the FIFO message queue approaches context limits, the runtime triggers a recursive context compaction routine. The agent receives system warnings indicating memory pressure and executes tool calls to write critical facts from the working window into core or archival memory before eviction occurs.

Key Strengths:

• High autonomy: The agent controls what it remembers and forgets without external heuristic filters.
• Explicit user modeling: Core memory blocks provide deterministic, highly visible context for personalization.

Limitations:

• In-band latency overhead: Memory operations require explicit tool-calling turns, increasing response latency and token consumption on user-facing requests.
• Vulnerable to tool hallucination: If an agent enters a degenerative loop, it can accidentally overwrite core memory blocks with corrupted data.

2. Zep / Graphiti: Bi-Temporal Knowledge Graphs

Zep approaches agent memory through Graphiti, an open-source temporal knowledge graph engine documented by Zep AI (2025). Rather than storing flat text chunks or key-value blobs, Zep models memory as an evolving, multi-relational graph with first-class temporal awareness.

+-------------------------------------------------------------+
|                      ZEP / GRAPHITI                         |
|                                                             |
|  Incoming Message Stream ---> Asynchronous Graph Extractor  |
|                                     |                       |
|                                     v                       |
|  +-------------------------------------------------------+  |
|  | Bi-Temporal Knowledge Graph                           |  |
|  |                                                       |  |
|  |  (User: Alice) ---[WorksAt {t_valid: 2024-2025}]--->   |  |
|  |        |                                              |  |
|  |        +---------[WorksAt {t_valid: 2026-inf}]----->  |  |
|  |                                       (Acme Corp)     |  |
|  +-------------------------------------------------------+  |
|                             ^                               |
|         Hybrid Context Search (Graph Traversal + HNSW)      |
|                             |                               |
|  Agent Request <------------+                               |
+-------------------------------------------------------------+

Bi-Temporal Modeling

Graphiti implements bi-temporal indexing, tracking two distinct time dimensions for every assertion:

• Valid Time ($t_{valid}$): The time interval during which a fact was true in the real world.
• Transaction Time ($t_{trans}$): The timestamp when the fact was recorded in the database.

When an agent learns that a user has relocated from London to San Francisco, Graphiti does not delete the London relationship. Instead, it invalidates the edge by setting $t_{end}$ on the previous relation and instantiates a new active edge with $t_{start}$ corresponding to the transition timestamp.

Graph Synthesis and Low-Latency Retrieval

• Dynamic Entity Extraction: Incoming messages are asynchronously parsed by background LLM workers into subject-predicate-object triples, resolving aliases to existing entity nodes.
• Hybrid Context Lake: Zep pairs graph traversal with vector similarity search. When an agent queries the memory layer, Zep computes semantic similarity over entity and edge embeddings, performs localized breadth-first graph expansions, and filters by temporal validity.
• Sub-200ms Serving: By utilizing a custom C++ graph engine and pre-computed graph neighborhoods, Zep delivers p95 retrieval latencies below 200ms, making it suitable for inline context injection before the primary LLM generation pass.

Key Strengths:

• Native contradiction resolution: Resolves evolving facts without data loss or ambiguous state collisions.
• Temporal reasoning: Capable of answering complex chronological queries (for example, "Where did the user work prior to March 2025?").

Limitations:

• Computational overhead: Asynchronous graph construction requires heavy background LLM token throughput to parse entities and relationships.
• Complex local deployment: Full self-hosting requires managing graph storage and vector index dependencies.

3. Mem0: Two-Phase Semantic Extraction and Scoped Memory

Mem0 (detailed in Prateek et al., 2025) provides a modular, multi-tiered memory architecture focused on structured fact extraction, semantic deduplication, and hierarchical scoping.

+-------------------------------------------------------------+
|                        MEM0 PIPELINE                        |
|                                                             |
|  Message Pair (User + Assistant)                            |
|       |                                                     |
|       v                                                     |
|  +-------------------------------------------------------+  |
|  | Stage 1: Candidate Extraction                         |  |
|  | LLM extracts atomic candidate facts from messages     |  |
|  +-------------------------------------------------------+  |
|       |                                                     |
|       v                                                     |
|  +-------------------------------------------------------+  |
|  | Stage 2: Semantic Matching & Conflict Resolution      |  |
|  | Retrieve top-k similar memories from Vector Store     |  |
|  | LLM evaluates candidate vs existing:                  |  |
|  |   --> ADD     (New independent fact)                  |  |
|  |   --> UPDATE  (Modify / refine existing fact)         |  |
|  |   --> DELETE  (Invalidate contradicted fact)          |  |
|  |   --> NOOP    (Redundant duplicate)                   |  |
|  +-------------------------------------------------------+  |
|       |                                                     |
|       v                                                     |
|  Hierarchical Persistence: [User_ID] [Agent_ID] [Session_ID]|
+-------------------------------------------------------------+

Two-Phase Extraction and Mutation Loop

Mem0 decouples memory writes into two explicit LLM pipeline stages:

1. Candidate Extraction Phase: An extraction model processes each new message pair along with a running conversation summary to extract atomic candidate factual statements.
2. Conflict Resolution and Update Phase: The system queries the underlying vector store for the top-$k$ most similar existing memories. An LLM receives the candidate fact and the retrieved matches, outputting a structured decision:

• ADD: Insert as a new memory record.
• UPDATE: Merge or update the text and metadata of an existing record.
• DELETE: Remove a memory record that has been explicitly contradicted.
• NOOP: Discard candidate as redundant information already present.

Multi-Level Scoping

Mem0 partitions memory records across four isolation levels:

• User Level (user_id): Persistent preferences, profile details, and long-term facts shared across all applications for a given user.
• Agent Level (agent_id): Operational knowledge, domain guidelines, and execution learnings specific to a particular agent configuration.
• Session Level (session_id): Transient contextual state isolated to a single active conversation thread.
• Organization Level (org_id): Multi-tenant enterprise boundaries preventing cross-account memory leakage.

Key Strengths:

• High retrieval precision: Atomic fact extraction avoids polluting prompt context with irrelevant conversational filler.
• Drop-in framework compatibility: Clean REST and Python SDKs enable seamless integration with any orchestration library (CrewAI, AutoGen, LangGraph).

Limitations:

• High write token consumption: Two LLM inference calls per message turn for extraction and conflict resolution create noticeable token costs on high-volume pipelines.
• Lack of temporal graph relations: Cannot natively reconstruct chronological timelines without external timestamp filtering.

4. LangMem: Graph-Native State Primitives and Prompt Optimization

Developed by LangChain, LangMem provides memory primitives built directly on top of LangGraph's BaseStore. Rather than running as an independent service, LangMem provides low-level abstractions that execute inside LangGraph nodes or asynchronous worker tasks.

+-------------------------------------------------------------+
|                       LANGMEM STACK                         |
|                                                             |
|  LangGraph Agent Execution                                  |
|       |                                                     |
|       +---> In-Loop Store Tool (create_manage_memory_tool)  |
|       |                                                     |
|       +---> Background Thread Consolidation Manager         |
|                 |                                           |
|                 v                                           |
|  +-------------------------------------------------------+  |
|  | LangGraph BaseStore (AsyncPostgresStore / InMemory)   |  |
|  | Namespace: ("memories", user_id, "semantic")          |  |
|  | Hybrid Search (Embedding Vector + BM25 Filter)       |  |
|  +-------------------------------------------------------+  |
|                 |                                           |
|                 v                                           |
|  +-------------------------------------------------------+  |
|  | Reflective Self-Improvement (RSI)                     |  |
|  | Prompt Optimization Node updates Agent System Prompt  |  |
|  +-------------------------------------------------------+  |
+-------------------------------------------------------------+

Storage Architecture and Namespaces

LangMem delegates storage, indexing, and vector similarity search to langgraph.store.BaseStore implementations (such as InMemoryStore for local development or AsyncPostgresStore with pgvector for production).

State is organized into hierarchical tuple namespaces:

namespace = ("memories", user_id, "profile")
store.put(namespace, key="user_prefs", value={"preferred_language": "Python", "framework": "PyTorch"})

Dual Extraction Modes

1. Tool-Driven Active Memory: Exposes create_manage_memory_tool and create_search_memory_tool to the agent, allowing the LLM to search, insert, and update structured schemas during graph execution.
2. Background Store Manager: Executes asynchronous extraction via create_memory_store_manager. When a conversation thread concludes, a background worker runs semantic extraction and consolidates findings into the long-term store without blocking real-time user interactions.

Reflective Self-Improvement (RSI)

Beyond conversational fact retention, LangMem includes prompt optimization primitives. By analyzing user feedback, failure traces, and agent trajectories, LangMem's optimization routines automatically refine and patch system prompt instructions over time, allowing agents to adapt their operational behavior to recurring user preferences.

Key Strengths:

• Zero external infrastructure overhead: Integrates directly into existing PostgreSQL / LangGraph production deployments.
• Meta-prompt optimization: Unique capability to refine system prompts based on task trajectories.

Limitations:

• Framework lock-in: Deeply coupled with LangGraph runtime conventions and data models.
• Basic retrieval logic: Relies primarily on standard top-k vector and keyword lookups without native graph traversal or entity resolution.

Architectural Comparison and Trade-off Profiles

+---------------------------------------------------------------------------------------------------------+
| Feature / Metric       | Letta (MemGPT)      | Zep (Graphiti)      | Mem0               | LangMem       |
+------------------------+---------------------+---------------------+--------------------+---------------+
| Primary Mental Model   | OS Virtual Memory   | Temporal Graph      | Two-Phase Pipeline | Graph Store   |
| State Mutation         | Tool Calls          | Edge Invalidation   | LLM Classification | Key Overwrite |
| Temporal Modeling      | Append-only log     | Bi-temporal windows | Timestamp metadata | Store Timers  |
| Write Latency Impact   | High (In-loop tools)| Low (Async worker)  | Med (Two LLM calls)| Low (Async)   |
| Read Query Latency     | 50ms - 150ms        | Sub-200ms (p95)     | 40ms - 120ms       | 20ms - 80ms   |
| Primary Storage        | Postgres / HNSW     | C++ Graph Engine    | Vector Store + DB  | PostgreSQL    |
| Prompt Self-Refinement | Manual persona edits| No                  | No                 | Native RSI    |
| Open-Source License    | Apache-2.0          | Apache-2.0 / Comm.  | Apache-2.0         | MIT           |
+---------------------------------------------------------------------------------------------------------+

1. Token Economics vs Context Drift

Agent memory introduces a classic systems trade-off between write-time token overhead and read-time prompt efficiency:

• Write-Heavy Pipelines (Mem0, Zep): Processing every conversational turn through extraction and graph-building models generates ongoing token costs (often 500 to 1,500 tokens per message turn). However, they keep the primary agent prompt lean, retrieving only 200 to 400 tokens of highly relevant factual context.
• Paging Architectures (Letta): Deferring extraction until context window thresholds are reached minimizes background processing costs during short interactions. However, active context windows remain larger, increasing prefill compute costs across consecutive turns.

2. KV Cache Reuse Dynamics

In modern inference serving engines (vLLM, SGLang), system prompt prefix caching significantly reduces time-to-first-token (TTFT).

• Constantly mutating working memory blocks directly inside the system prompt (as in Letta core memory) invalidates the KV cache prefix on every update.
• In contrast, injecting retrieved memories into a designated user-turn message boundary (as in Mem0 and Zep) preserves the static system prompt prefix, maximizing KV cache hit rates.

3. Failure Modes and Memory Poisoning

• Hallucinatory Drift: When agents autonomously extract facts, ambiguous colloquial statements can be converted into false persistent beliefs. Without confidence scoring or verification gates, bad memories persist indefinitely.
• Context Collisions: Flat vector retrieval often returns conflicting statements from different historical dates. Applications requiring strict state accuracy must use temporal invalidation (Zep) or deterministic update classifiers (Mem0) to prune obsolete records.

Architectural Decision Framework

                              [Production Memory Need]
                                         |
            +----------------------------+----------------------------+
            |                                                         |
  [Need Autonomous OS Paging]                              [Need Managed State Extraction]
  [Agent Manages Own Working Memory]                                  |
            |                                                         |
            v                                                         v
       Choose Letta                                    [Require Time-Series & Evolution?]
                                                              |
                                            +-----------------+-----------------+
                                            |                                   |
                                          (Yes)                                (No)
                                            |                                   |
                                            v                                   v
                                    Choose Zep / Graphiti             [LangGraph Native Stack?]
                                    (Bi-temporal graph engine)                  |
                                                              +-----------------+-----------------+
                                                              |                                   |
                                                            (Yes)                                (No)
                                                              |                                   |
                                                              v                                   v
                                                        Choose LangMem                       Choose Mem0
                                                    (PostgreSQL BaseStore)             (Universal Memory API)

When to Deploy Each Framework

• Deploy Letta when: You are building long-lived autonomous agents that require full self-directed context management, explicit editable persona blocks, and operating-system-level agent state persistence.
• Deploy Zep (Graphiti) when: Your application depends heavily on time-series facts, evolving customer states, complex relationship networks, and chronological accuracy across months of interactions.
• Deploy Mem0 when: You require a lightweight, framework-agnostic memory API that extracts structured atomic facts across user, session, and organization tiers with minimal integration complexity.
• Deploy LangMem when: Your production stack is already built on LangGraph and PostgreSQL, and you need tight state persistence coupled with prompt self-improvement capabilities.

Sources

• Packer, C., Fang, V., Patil, S. G., Lin, K., Wooders, S., & Gonzalez, J. E. (2023). MemGPT: Towards LLMs as Operating Systems. arXiv:2310.08560.
• Zep AI Research. (2025). Zep: A Temporal Knowledge Graph Architecture for Agent Memory. arXiv:2501.13956.
• Prateek, D., et al. (2025). Mem0: Building Scalable Long-Term Memory for AI Agents. arXiv:2504.19413 (ECAI 2025).
• Letta GitHub Repository and Architecture Documentation.
• Graphiti Temporal Knowledge Graph Engine.
• Mem0 Open-Source Agent Memory Platform.
• LangMem Documentation and LangGraph Store Integration.