Agent Memory Architectures in Production: Comparing Mem0, Zep, Letta, and Cognee Architecture, Graph vs Vector Topologies, Extraction Pipelines, and Serving Economics

Autonomous AI agents deployed in production environments face a fundamental architectural limitation: large language models are stateless functions. While expanding context windows to hundreds of thousands or millions of tokens allows developers to append raw message logs into prompts, naive context stuffing degrades rapidly in practice. It introduces quadratic computational overhead, inflates time-to-first-token (TTFT) latency, degrades retrieval precision via "lost in the middle" phenomena, and fails to handle state updates or contradictions over long horizons.

To maintain coherence across multi-turn and multi-session interactions, production systems decouple working state from raw chat logs through dedicated agent memory architectures. Modern implementations range from extraction-based vector sidecars to bi-temporal knowledge graphs and operating system-inspired virtual memory hierarchies.

The Agent Memory Taxonomy

Production agent memory is categorized into four distinct functional tiers:

Working Memory (In-Context Scratchpad): The active context window holding current task state, system instructions, dynamic variables, and immediate tool execution results.
Episodic Memory (Interaction History): A chronological record of past events, interactions, and raw conversation traces, indexed for temporal and lexical retrieval.
Semantic Memory (Structured Knowledge and Facts): Consolidated, deduplicated knowledge about users, domains, entities, and relationships extracted across multiple interactions.
Procedural Memory (Executable Workflows): Fixed policies, tool definitions, dynamic skills, and execution playbooks that govern agent behavior.

The engineering challenge lies in the read and write paths: extracting semantic facts from unstructured conversations without unbounded LLM inference costs, resolving conflicting updates, and retrieving relevant context within strict latency budgets.

Architectural Deep Dive: Four Production Paradigms

Four frameworks represent the leading architectural patterns for production agent memory: Mem0, Zep, Letta, and Cognee.

1. Mem0: Two-Phase Extraction and Logit-Gated Vector Operations

Mem0 (arXiv:2504.19413) operates as an external, drop-in memory middleware layer sitting between the application and the LLM runtime. It processes conversational state through an incremental two-phase pipeline:

Extraction Phase: When new messages arrive, Mem0 evaluates the conversation window (typically $m=10$ messages) alongside previous summaries using an extraction model to distill atomic, candidate facts (such as user preferences, technical constraints, or specific entity attributes).
Update Phase: Each candidate fact is queried against a dense vector database to retrieve the top- $s$ semantically similar existing memories (typically $s=10$ ). An LLM evaluates the candidate against retrieved entries using a tool-calling interface to execute one of four atomic operations:
ADD: Inserts genuinely novel information into the vector store.
UPDATE: Augments or refines an existing memory record with higher-fidelity details.
DELETE: Removes obsolete records that are directly contradicted by new information.
NOOP: Discards the candidate if identical facts already exist.

For relationship-heavy tasks, the Mem0g graph extension introduces soft-deletions and entity-relationship edges, marking superseded facts as invalid rather than deleting them, which preserves historical state.

2. Zep and Graphiti: Bi-Temporal Knowledge Graphs

Zep (arXiv:2501.13956) approaches agent memory through Graphiti, an open-source temporal knowledge graph engine designed specifically for dynamic conversational state.

Unlike static knowledge graphs or standard RAG indices, Graphiti implements a bi-temporal data model that maintains two distinct temporal axes for every entity and relation:

Valid Time ( $t_{valid}$ ): The time period during which a fact is true in the real world.
Transaction Time ( $t_{tx}$ ): The time period during which the fact was recorded in the database.

Graphiti organizes data across a multi-tier subgraph hierarchy:

Episodic Subgraph: Captures raw interaction nodes, temporal sequence, and message provenance.
Semantic Subgraph: Contains deduplicated entities and dynamic relationship edges.
Community Subgraph: Clusters related entity subgraphs to support higher-level thematic queries.

When new facts contradict older records, Graphiti invalidates the existing edge by setting an expiration timestamp ( $t_{valid\_end}$ ) rather than executing a hard delete. This architecture enables agents to resolve complex temporal queries (such as "What was the user working on before switching projects last Tuesday?") while maintaining sub-200ms p95 retrieval latency over millions of nodes. Benchmark evaluations on LongMemEval demonstrate an 18.5% accuracy gain and up to 90% latency reduction compared to standard vector retrieval approaches.

Agent Memory Topologies and Storage Layers

3. Letta (MemGPT): Operating System Virtual Memory Hierarchy

Letta (formerly MemGPT, arXiv:2310.08560) models agent memory after the memory management units of traditional operating systems. Rather than relying on implicit background middleware, Letta makes memory management an explicit, agent-driven capability through functional tool calling.

The architecture partitions state into distinct memory tiers:

Tier 1 (Main Context / Core Memory): Fixed, in-context blocks (such as persona for system behavior and human for user profile details) that are always visible to the model in the primary context window.
Tier 2 (External Storage / Out-of-Context Memory): Divided into Recall Storage (a searchable conversational event log stored in a relational database) and Archival Storage (an unbounded vector-indexed store for long-form documents and domain knowledge).

Letta provides the LLM with deterministic memory management tools:

core_memory_append and core_memory_replace: Dynamically modify in-context persona and user state blocks during execution.
archival_memory_insert and archival_memory_search: Explicitly write to and retrieve from external vector storage when in-context capacity is reached.

This self-editing paradigm grants the agent explicit agency over its own context, though it relies heavily on model instruction-following and tool-calling discipline.

4. Cognee: Extract-Cognify-Load (ECL) Tri-Store Architecture

Cognee frames agent memory as a structured data engineering problem, implementing an Extract, Cognify, Load (ECL) pipeline connected to a hybrid tri-store persistence backend:

Extract: Ingests raw data streams, conversational turns, structured payloads, and external documents.
Cognify: Processes data through entity extraction, deterministic entity linking, RDF-based ontology mapping, and dense embedding generation.
Load: Simultaneously writes structured components to three specialized databases:
Graph Store (Kùzu, Neo4j, FalkorDB): Stores entities, relationships, and topological graph paths for multi-hop graph traversal.
Vector Store (LanceDB, Qdrant, Redis): Indexes semantic chunk embeddings for similarity search.
Relational Store (SQLite, PostgreSQL): Tracks document metadata, chunk offsets, and execution audit trails.

By combining deterministic ontology constraints with hybrid graph-vector retrieval, Cognee eliminates hallucinated memory drift and provides verifiable data lineage.

Structural and Operational Comparison

Evaluating agent memory frameworks requires balancing write-path overhead, read-path latency, and reasoning depth:

Mem0
Primary Storage: Vector Database (Qdrant, pgvector, Chroma) + optional graph extension.
Write-Path Mechanism: Two-phase extraction and logit-gated tool-calling (ADD, UPDATE, DELETE, NOOP).
Read-Path Latency: Low (single dense vector query per turn).
Temporal Reasoning: Limited (basic timestamp tagging; soft invalidation in Mem0g).
Best Suited For: Stateless agent wrappers, fast user profile persistence, and multi-user personalization.
Zep (Graphiti)
Primary Storage: Custom Context Graph Engine / Neo4j + Vector Index.
Write-Path Mechanism: Incremental entity resolution, edge generation, and bi-temporal interval tagging.
Read-Path Latency: Medium-Low (sub-200ms hybrid graph-vector search and reranking).
Temporal Reasoning: High (native bi-temporal valid time and transaction time tracking).
Best Suited For: Enterprise assistants, customer support agents with evolving state, and complex temporal synthesis.
Letta
Primary Storage: In-context RAM blocks + PostgreSQL (Recall) + Vector Store (Archival).
Write-Path Mechanism: Explicit model tool calls (core_memory_append, archival_memory_insert).
Read-Path Latency: Instant for Core Memory; tool-call round-trip latency for Archival Search.
Temporal Reasoning: Medium (chronological recall pagination).
Best Suited For: Autonomous long-running agents, persistent digital assistants, and complex stateful workflows.
Cognee
Primary Storage: Tri-store (Kùzu graph + LanceDB vector + SQLite relational).
Write-Path Mechanism: Modular ECL pipeline with deterministic entity linking and RDF ontology enforcement.
Read-Path Latency: Medium (combined graph traversal and vector scoring).
Temporal Reasoning: Moderate (graph lineage and provenance tracking).
Best Suited For: Enterprise RAG systems, compliance-critical agent pipelines, and structured domain ontologies.

Serving Economics and Production Latency Trade-offs

Deploying persistent memory layers introduces critical cost and operational considerations:

The Write-Path LLM Tax

In traditional RAG systems, ingestion costs are paid upfront during document indexing, while query costs scale linearly with user traffic. In agent memory systems, the economic model is inverted.

Every conversational turn triggers write-path evaluation: extracting candidate facts, searching existing records, and executing conflict resolution. For high-volume agent applications, running secondary LLM calls (even using lightweight models like GPT-4o-mini or Claude 3.5 Haiku) on every turn increases operational token costs significantly. Engineering teams must implement debounce mechanisms, background async queues, and semantic change filters to prevent redundant extraction passes on trivial conversational turns.

Read-Path Latency Impact

Injecting retrieved memory into the active prompt introduces pre-generation latency. While pure vector similarity lookups add 15ms to 50ms, multi-hop graph traversals and hybrid reranking pipelines can add 150ms to 300ms to time-to-first-token (TTFT). For real-time conversational agents, memory retrieval must be parallelized with prompt compilation or executed asynchronously during the user's turn entry.

Context Pollution and Memory Drift

Without rigorous conflict resolution, agent memory stores degrade into inconsistent, noisy state repositories. In-context memory architectures (such as Letta's Core Memory) risk prompt bloat if the model fails to prune obsolete facts. External vector stores (such as Mem0) risk semantic collision when conflicting statements receive high cosine similarity scores. Graph-based architectures (such as Zep and Cognee) mitigate these failures through bi-temporal invalidation and deterministic ontology boundaries, ensuring that superseded facts remain accessible for historical queries without contaminating active agent decision loops.