Agent Memory Engines in Production: Comparing Mem0, Zep, and Letta Architecture, Graph Stores, and Retrieval Latency

Large language models are inherently stateless functions. Every invocation begins with an empty memory register, relying entirely on the tokens packed into its context window. In production multi-turn applications, naive context management strategies quickly collapse: appending raw conversational logs inflates inference costs and eventually hits hard context limits, while standard semantic vector search (retrieval-augmented generation) lacks temporal awareness, treats contradictions blindly, and fails to track how user preferences evolve over time.

To solve persistent statefulness across sessions, engineering teams have converged on dedicated agent memory engines. However, the three leading open-source systems (Mem0, Zep, and Letta, formerly MemGPT) approach the problem from fundamentally divergent architectural paradigms.

Comparison of AI Agent Memory Architectures

Choosing between them is not a matter of feature checklists, but a core systems design decision regarding where memory mutation occurs, how time is represented, and whether the agent or the infrastructure owns state reconciliation.

The Three Architectural Philosophies

The current ecosystem divides agent memory into three distinct architectural models:

Background Extract-and-Resolve (Mem0): Treats memory as an asynchronous extraction pipeline that sits outside the agent's execution loop. Conversational turns are analyzed by background LLMs to extract atomic factual candidates, which are compared against existing records via vector similarity and reconciled through deterministic CRUD operations.
Temporal Knowledge Graphs (Zep / Graphiti): Treats memory as a dynamic graph where facts possess explicit temporal validity windows (valid_at and invalid_at). When new information contradicts old facts, edges are invalidated rather than deleted, preserving historical provenance and enabling time-aware multi-hop reasoning.
OS-Style Self-Editing Virtual Memory (Letta): Treats memory as a hierarchical virtual memory system modeled after operating systems. The agent itself is stateful, maintaining fixed in-context memory blocks (analogous to RAM registers) and out-of-context vector/relational databases (analogous to disk storage) that it modifies directly through tool calls.

Mem0: Scoped Fact Extraction and the AUDN Mutation Cycle

Mem0 decouples memory maintenance from application runtime. Instead of requiring the primary agent to make explicit tool calls to update its memory, Mem0 intercepts conversational turns (messages between user and assistant) and processes them through a two-phase extraction pipeline.

In the first phase, a background language model parses the raw text to extract atomic candidate facts, user preferences, and entity attributes. In the second phase, each candidate fact is evaluated against existing records in the vector database using what the developers designate as the AUDN cycle (Add, Update, Delete, No-Op):

ADD: If the candidate represents completely new information with low similarity to existing records, a new vector and metadata entry are created.
UPDATE: If the candidate complements, refines, or updates an existing memory, the prior record is modified in place to reflect the new state.
DELETE: If the candidate directly contradicts an older memory, the obsolete record is purged from the index.
NOOP: If the candidate is redundant or semantic duplicates already exist, the write is dropped without mutating the database.

Mem0 organizes these operations across three hierarchical scopes: user_id (persistent across all sessions for a specific user), agent_id (shared state across agents), and session_id (scoped to an ephemeral conversation thread). When hydrating context, Mem0 performs a semantic vector search filtered by the relevant user and session IDs, returning a compact list of top-k factual strings injected into the system prompt.

Zep: Temporal Graph Traversal and Dynamic Edge Invalidation

Standard vector search struggles with temporal evolution. If a user states in March that they live in London, and in August states they moved to New York, standard embedding cosine similarity retrieves both chunks with near-equal relevance, forcing the downstream LLM to arbitrate conflicting statements without clear temporal context.

Zep addresses this failure mode via Graphiti, its dynamic, temporally-aware knowledge graph engine. Zep processes conversational streams and business data into an interconnected graph composed of episodic, semantic, and community subgraphs.

Instead of hard-deleting contradictory records, Graphiti implements a bitemporal data model:

Transaction Time ( $T_{tx}$ ): When the system ingested the record into the database.
Valid Time ( $T_{valid}$ ): When the real-world fact or event occurred.

When an incoming edge contradicts an existing relationship, an LLM evaluator identifies the semantic conflict. Rather than executing a destructive DELETE operation, Graphiti sets the invalid_at timestamp on the historical edge to match the valid_at timestamp of the incoming edge. The obsolete fact remains preserved in the graph for historical auditing and point-in-time querying, while runtime context retrieval automatically filters for currently valid edges unless explicitly queried across historical timelines.

Retrieval in Zep utilizes a hybrid search pipeline combining dense vector embeddings, BM25 lexical search, and graph traversal algorithms, followed by a cross-encoder reranker to assemble context subgraphs within sub-100 millisecond response budgets.

Letta: Stateful Agents and In-Context Self-Editing Blocks

Evolving from the academic research behind MemGPT, Letta rejects the external pipeline model in favor of the LLM-as-an-Operating-System paradigm. In Letta, agents are persistent, stateful entities that own and govern their own memory hierarchy.

Letta structures agent state into three distinct tiers:

Core Memory (In-Context Working Memory): Structured text blocks pinned directly inside the LLM's system prompt. By default, Letta provides a human block (facts about the user) and a persona block (agent identity and instructions), though custom blocks can be defined with rigid character budgets.
Recall Memory (Conversation History Buffer): A paginated, indexed database containing the complete raw conversation log and tool call history.
Archival Memory (Out-of-Context Vector Store): A long-term semantic storage layer where arbitrary documents, code snippets, or historical facts are indexed for on-demand similarity retrieval.

The defining characteristic of Letta is self-editing memory. The primary agent is equipped with native function-calling tools, such as core_memory_append, core_memory_replace, archival_memory_insert, and archival_memory_search. When a user discloses a key constraint or instruction, the agent issues a tool call to update its core memory block before generating its response. For large multi-agent deployments, Letta also supports secondary background memory-manager agents that asynchronously read conversation transcripts and compress salient points into the primary agent's core memory.

Storage Backends and Systems Architecture

The architectural choices of each framework dictate their underlying database infrastructure:

Write Path Overhead and Ingestion Economics

A critical operational metric in production is the write tax: the computational and financial overhead required to mutate memory state.

Mem0 incurs a double LLM inference cost on every ingestion cycle: one extraction call to parse candidate facts from the raw message turn, and a second verification call to decide the AUDN operation against retrieved vector neighbors. While this work can execute asynchronously in the background via message queues (e.g., Celery, Redis Streams), it multiplies token consumption per conversation.

Zep offloads extraction to continuous background batch workers in the Graphiti service. Node and edge construction, entity resolution, and temporal boundary updates run against the graph asynchronously. Because graph extraction requires structured schema alignment, token overhead is higher than simple vector ingestion, but it completely shields the end-user request path from write latency.

Letta shifts the write burden into the primary agent's synchronous execution loop (unless configured with background memory workers). When the agent decides to modify core memory, it must execute a multi-step tool call sequence: generating the tool arguments, pausing generation, updating the local database, and resuming context. This adds inference round-trips and increases time-to-first-token (TTFT) for responses requiring memory writes, but guarantees that memory mutations are strictly aligned with the agent's active reasoning state.

Retrieval Latency and Context Hydration

On the read path, system performance depends on how memories are fetched and injected into the prompt before the model begins generation:

Mem0: Delivers the lowest retrieval latency (typically 10 to 30 milliseconds). Because it queries a flat vector index with user-scoped metadata filters, retrieved facts are formatted as concise bullet points. The resulting token overhead is minimal (usually 100 to 300 tokens), preserving context window capacity for active tasks.
Zep: Executes hybrid vector search and graph traversal in 50 to 100 milliseconds. It retrieves structured relationship subgraphs and temporal metadata, allowing the LLM to understand entity dependencies and historical state changes at the cost of moderately higher prompt token usage (300 to 800 tokens).
Letta: Core memory has zero retrieval latency (0 ms) because the core memory blocks are statically pinned within the system prompt. However, if the agent needs information from archival memory, it must dynamically issue an archival_memory_search tool call, introducing a full inference round-trip penalty (300 to 1,000+ ms depending on the inference provider).

Decision Matrix: Choosing the Right Engine

Engineering teams should evaluate their memory architecture based on workflow complexity and latency constraints:

Choose Mem0 when:

Building user-facing customer support chatbots, recommendation systems, or personal assistants that require lightweight, user-scoped fact recall.
Minimizing token overhead and inference latency is a primary architectural priority.
You prefer keeping memory logic decoupled from prompt engineering and agent tool definitions.

Choose Zep when:

Building complex enterprise workflows, CRM assistants, or business intelligence agents where data relationships and timelines constantly change.
Handling contradictory facts over time without losing auditability or data provenance is mandatory.
Multi-hop entity reasoning ("How has client X's project scope shifted since Q1?") is a core product requirement.

Choose Letta when:

Building fully autonomous, long-running stateful agents that require explicit self-awareness and active control over their internal state.
Developing interactive character personas or virtual coworkers with strict memory budgets.
Implementing the Model Context Protocol (MCP) or multi-agent architectures where agents must read, share, and edit collaborative memory blocks directly.