Agentic Memory Systems in Production: Comparing Mem0, Letta, Zep Graphiti, and Cognee Architecture, State Consolidation, Temporal Graphs, and Retrieval Latencies

Large language models are inherently stateless across API calls. While context windows have expanded to hundreds of thousands or millions of tokens, stuffing entire interaction histories into prompt context degrades retrieval accuracy, inflates time-to-first-token (TTFT) latency, and creates linear or quadratic cost scaling per interaction turn.

For production AI agents operating over days, weeks, or months, persistent memory is a necessary architectural layer. Production memory systems differ fundamentally from basic document retrieval-augmented generation (RAG). Document RAG searches static text corpora. Agent memory systems must dynamically capture unstructured conversational turns, extract salient facts and user preferences, update or invalidate stale state, track temporal validity, and inject relevant context into the model context window with minimal latency overhead.

Four primary frameworks have emerged to address long-term agent memory: Mem0, Letta (formerly MemGPT), Zep (powered by Graphiti), and Cognee. Each makes distinct architectural tradeoffs across state extraction, memory hierarchy, temporal tracking, and execution runtime.

Core Architectural Paradigms

Agent memory frameworks diverge primarily on whether memory is managed externally through asynchronous pipelines, actively controlled by the agent via tool execution, or structured as a temporal knowledge graph.

1. Passive Extraction and Dynamic Update Layer: Mem0

Mem0 operates as a pluggable, framework-agnostic memory middleware. Rather than requiring the agent to manage its own memory explicitly, Mem0 intercepts conversation turn pairs (user-assistant or user-user) and executes background extraction pipelines.

The base Mem0 pipeline uses a secondary LLM call to identify entities, facts, preferences, and operational constraints from conversational turns. Extracted facts are compared against existing entries in a vector store using semantic similarity. When new information contradicts or refines existing records, Mem0 executes automated CRUD (Create, Read, Update, Delete) operations to overwrite or merge state rather than appending duplicate chunks.

In its extended graph variant, Mem0g, the system pairs vector storage with a directed labeled property graph, mapping relationships across entities to enable multi-hop associative retrieval across user sessions, as detailed in the Mem0 architecture paper (arXiv:2504.19413).

2. The Operating System Memory Hierarchy: Letta (MemGPT)

Letta, based on the MemGPT research architecture (arXiv:2310.08560), approaches memory through an operating system analogy:

In-Context Core Memory (RAM): A dedicated section of the model system prompt containing fixed blocks such as persona definitions and user profile facts. This context is always visible to the model.
Recall Memory (Cache): A sequential, searchable event log containing previous interaction turns and raw message logs.
Archival Memory (Disk): An external semantic vector store with unbounded capacity for storing long-term knowledge and documents.

Unlike passive middleware, Letta treats the agent as an active manager of its own memory. The agent is provisioned with explicit memory management tools (core_memory_append, core_memory_replace, archival_memory_insert, archival_memory_search). The model executes these tools during its inference loop to modify its system prompt or page out detailed logs to archival storage. Letta also provides sleep-time consolidation routines, running background jobs to defragment, summarize, and reorganize stored memories when the agent is idle.

3. Bi-Temporal Knowledge Graphs: Zep and Graphiti

Zep uses Graphiti, an open-source temporal knowledge graph engine designed specifically for dynamic agent state.

Traditional vector databases struggle with state transitions over time. If a user states in January that they live in Chicago and in June that they moved to London, cosine similarity on "where does the user live?" returns both chunks with high relevance scores. Zep addresses this by building a bi-temporal knowledge graph, as described in the Zep architecture paper (arXiv:2501.13956).

Graphiti tracks two distinct timelines for every extracted edge:

Valid Time: The real-world window during which the relationship or fact was true.
Transaction Time: The exact timestamp when the knowledge graph recorded or ingested the fact.

When a user provides updated information, Graphiti identifies conflicting edges, closes their valid time window, and establishes new edges with updated timestamps. Superseded facts remain in the graph with expired validity rather than being deleted, preserving provenance. Retrieval combines vector similarity, BM25 keyword matching, and graph traversal algorithms over its dedicated Context Graph Engine.

4. Cognitive Graph-Vector Pipelines: Cognee

Cognee structures memory around cognitive science principles and formal ontology generation. The system processes raw data through an ECL (Extract, Cognify, Load) pipeline:

Ingestion and Chunking: Text, documents, and transcripts are split into semantically coherent segments.
Cognify Pipeline: Asynchronous LLM tasks identify domain concepts, classify hierarchical relationships, and construct typed graph ontologies.
Hybrid Storage: Nodes and semantic embeddings are synchronized across vector stores (such as Qdrant or LanceDB) and graph engines (such as FalkorDB or Neo4j).

Cognee emphasizes deterministic, structured memory topologies, making it suitable for multi-agent workflows where several autonomous workers must query a shared, consistent domain knowledge graph.

Memory Operations and Retrieval Mechanics

Each framework structures its retrieval and persistence operations differently across the agent lifecycle.

Ingestion and Memory Extraction

Mem0: Executes an extraction prompt on raw message pairs, resolving user identifiers (user_id, agent_id, run_id) to maintain scoped memory buckets. It requires zero cognitive overhead from the primary acting LLM.
Letta: Ingestion occurs either via explicit agent tool calls during execution or via background event streams. The agent decides what is worth remembering and writes it directly to its core or archival memory.
Zep (Graphiti): Extracts entity-relation-entity triples along with temporal constraints. Edge extraction runs asynchronously, updating the temporal graph without blocking the immediate chat response.
Cognee: Runs graph-extraction algorithms to map text against configurable Pydantic schema ontologies, generating interconnected node networks.

Retrieval Latency and Token Overhead

Retrieval latency directly affects time-to-first-token in conversational applications:

Pre-prompt Middleware (Mem0, Zep): Incurs a single retrieval round-trip (vector lookup, keyword search, or graph traversal) before assembling the system prompt. Typical retrieval latencies range between 50ms and 200ms depending on database configuration.
Self-Editing Tool Execution (Letta): Because memory operations require the model to emit tool calls, inspect tool responses, and continue reasoning, memory retrieval can require multiple sequential LLM generation steps. This increases token consumption and end-to-end latency during complex memory traversals, but provides the agent with explicit control over search refinement.
Hybrid Graph Traversals (Cognee, Mem0g): Graph hops across multi-entity neighborhoods require additional index Lookups. Graph-only queries can range from 100ms to several hundred milliseconds depending on graph depth and database engine selection.

Evaluation and Temporal Benchmark Performance

Standard RAG benchmarks like MMLU or basic HotpotQA evaluate static document retrieval. They fail to test how memory frameworks handle evolving state, contradictory facts, and temporal decay.

Recent evaluations using the LongMemEval benchmark focus specifically on multi-turn conversations where facts change over time:

Temporal State Shifts: On tasks requiring an agent to resolve which fact is currently true versus historically true, temporal graph systems (Zep/Graphiti) demonstrate substantial accuracy advantages over flat vector stores. Tracking explicit validity windows prevents obsolete facts from polluting the active context.
Deep Associative Retrieval: On LOCOMO and multi-session associative retrieval tasks, self-editing architectures (Letta) and graph-augmented stores (Mem0g, Cognee) outperform naive conversation buffer memory by maintaining structured entity linkages across hundreds of turns.
Memory Bloat and Drift: Passive vector stores that append memories without deduplication experience retrieval degradation as noise accumulates. Both Mem0 (via automated update/merge prompts) and Letta (via sleep-time consolidation) mitigate memory drift by pruning redundant entries.

Architectural Tradeoffs and Selection Criteria

Choosing an agent memory architecture depends on the autonomy level of the agent, latency tolerances, and data dynamics:

Choose Mem0 when: You need a lightweight, framework-agnostic memory layer for user personalization, support bots, or existing agent codebases (LangChain, CrewAI, AutoGen). It integrates via simple SDK calls and manages CRUD deduplication automatically.
Choose Letta when: You are building long-running, autonomous agents that require explicit self-editing capabilities, OS-like memory management tools, and persistent server-side agent runtimes.
Choose Zep (Graphiti) when: Your application deals with rapidly changing facts, temporal dependencies (such as order statuses, changing preferences, or time-sensitive user updates), and requires enterprise sub-200ms p95 graph retrieval with full provenance.
Choose Cognee when: You require an open-source, self-hosted framework to construct structured domain ontologies and interconnected graph-vector networks for multi-agent collaboration.