Subscribe
Topic

LLM Architecture

Architecture patterns for LLM applications and RAG systems. Posts compare GraphRAG frameworks, explain RAPTOR tree-organized retrieval, listwise reranking and query transformation, and cover context compaction for long-running AI agents.

Articles

1-4
  1. Hierarchical Tree-Organized Retrieval (RAPTOR) in Production RAG: Recursive Summarization, Gaussian Mixture Clustering, and Cross-Scale Querying

    Standard retrieval-augmented generation (RAG) architectures operate on flat document chunks. Corpora are split into fixed token windows (typically 256 to 1024 tokens), mapped into vector space via dense embedding models, and queried through approximate nearest neighbor (ANN) search. While this setup efficiently resolves localized factual lookups ("What is the termination clause in contract X?"), it systematically fails on thematic synthesis, cross-document comparison, and high-level aggregation

    RAG
    7 min read
  2. Listwise Reranking in Production: Comparing Pointwise, Pairwise, and Listwise LLM Architectures, Sliding-Window Permutations, and Serving Economics

    Information retrieval systems in production Retrieval-Augmented Generation (RAG) and enterprise search have transitioned through multiple reranking paradigms. While first-stage retrieval (dense vector embeddings and sparse lexical BM25/SPLADE) retrieves candidate sets of 50 to 200 documents in under 20 milliseconds, the precision of downstream generation depends heavily on the reranking stage. Traditional neural rerankers evaluate candidates through pointwise scoring or pairwise classification.

    Information Retrieval
    5 min read
  3. Query Transformation in Production RAG: Architecture, Latency Economics, and Retrieval Trade-Offs for HyDE, Multi-Query Expansion, and Step-Back Prompting

    Standard retrieval-augmented generation (RAG) architectures operate on a naive assumption: that the raw user query is suitable for direct retrieval against a vector database or lexical search index. In production, this assumption fails across significant query distributions. Raw user queries are frequently short (averaging 4 to 8 words), structurally underspecified, conversational, or laden with unresolved pronoun bindings. Conversely, indexed document chunks typically contain 256 to 1024 tokens

    RAG
    7 min read
  4. Context Compaction and Session Pruning in Production AI Agents: Architecture, Hierarchical Summarization, and Constraint Preservation

    Context Compaction and Session Pruning in Production AI Agents: Architecture, Hierarchical Summarization, and Constraint Preservation Long-running autonomous agents executing multi-step workflows (codebase refactoring, recursive research, multi-turn debugging, and system operations) face an inevitable physical ceiling: working context exhaustion. As an agent executes shell commands, inspects repository trees, parses LSP diagnostics, and consumes tool outputs, the active context window fills rap

    Agents
    7 min read