Subscribe
Topic / page 29

LLM

Articles, page 29

561-580
  1. Attention with Linear Biases (ALiBi): How Static Positional Slopes Enable Zero-Shot Context Extrapolation

    Large language models process sequences by transforming discrete tokens into continuous vector representations. Standard dot-product self-attention is permutation-invariant: without explicit positional information, the attention operation treats a sequence as an unordered bag of tokens. Early transformer architectures addressed this limitation using Absolute Positional Embeddings (APE), either through fixed sinusoidal functions or learned lookup tables added directly to token embeddings. While

    AI
    6 min read
  2. Vals AI Raises $40M Series A at $400M Valuation Led by a16z to Build Real-World AI Benchmarks

    San Francisco evaluation startup Vals AI announced a $40 million Series A funding round at a $400 million post-money valuation, led by Andreessen Horowitz. The round included participation from existing seed backers 8VC, Pear VC, and Bloomberg Beta, alongside new institutional investors HRT Ventures and Next Ladder Ventures. The financing brings total capital raised by the company to $45 million, following a $5 million seed round. Founded by Stanford computer science graduates Rayan Krishnan a

    AI
    2 min read
  3. Small Language Models in Production: Task Specialization, Serving Economics, and the Frontier Offloading Pattern

    The default architecture for first-generation enterprise AI agents routed every prompt, tool selection, and intermediate evaluation step to a single frontier large language model. While this monolithic approach simplified initial orchestration, it introduced severe latency bottlenecks and unsustainable inference unit economics in high-throughput production environments. In production agentic loops, between 40% and 70% of model invocations are narrow, highly structured operations: classifying in

    AI
    6 min read
  4. Contrastive Decoding in Large Language Models: How Comparing Expert and Amateur Logits Suppresses Hallucination and Reasoning Errors

    Contrastive Decoding in Large Language Models: How Comparing Expert and Amateur Logits Suppresses Hallucination and Reasoning Errors Autoregressive large language models operate by predicting the conditional probability distribution of the next token given a sequence of preceding tokens. However, translating these continuous probability vectors into coherent, factual, and logically sound sequences remains one of the fundamental challenges of modern natural language processing. Traditional deco

    AI
    6 min read
  5. Reinforcement Learning with Verifiable Rewards: How Programmatic Oracles Eliminate Reward Hacking in LLM Reasoning

    Post-training paradigms for large language models have undergone a fundamental architectural shift. While the initial wave of alignment relied on Reinforcement Learning from Human Feedback (RLHF) and direct preference optimization (DPO), frontier reasoning systems increasingly depend on Reinforcement Learning with Verifiable Rewards (RLVR). Traditional RLHF relies on neural reward models trained on human pairwise comparisons. These neural proxies suffer from reward overoptimization, vulnerabili

    AI
    6 min read
  6. Human-in-the-Loop Architectures for Production AI Agents: Interrupt Patterns, State Resumption, and Permission Escalation

    Deploying autonomous AI agents into production environments exposes a fundamental tension between system velocity and operational safety. While read-only tasks such as data extraction and document summarization carry minimal operational blast radius, agents equipped with write-access tools (database mutations, API transactions, cloud infrastructure provisioning, and outbound communications) introduce severe operational risks. Hallucinations, prompt injections, and logical drift can trigger irrev

    AI
    6 min read
  7. Machine Unlearning in Large Language Models: How Gradient Ascent, Representation Misdirection, and Negative Alignment Purge Neural Memory

    Large language models memorize massive amounts of training data, including copyrighted literature, personally identifiable information (PII), proprietary codebases, and dual-use knowledge spanning cyber exploits and biological hazards. When copyright holders, regulators, or safety researchers demand the removal of specific data, complete retraining from scratch is economically infeasible, often costing tens of millions of dollars in compute. Machine unlearning aims to remove the influence of ta

    AI
    7 min read
  8. Semantic Routers in Production: Architecture, Embedding Classification vs. LLM Triage, and Cascade Topologies

    Routing user requests to specialized tools, APIs, or tiered foundation models is central to production AI architectures. However, relying on large language models (LLMs) or native JSON tool-calling mechanisms to perform initial intent classification introduces significant latency and cost overheads. A single LLM-based classification pass typically adds between 200 milliseconds and 1.5 seconds of time-to-first-token (TTFT) latency, alongside linear token billing for routine triage queries. To el

    AI
    5 min read
  9. Model Collapse in Large Language Models: How Recursive Training on Synthetic Data Degrades Neural Distributions

    As large language models scale and generate a growing share of digital text, code, and media, the web datasets used to train next-generation models increasingly consist of machine-generated outputs. When generative models are trained recursively on data produced by earlier model generations without sufficient ground-truth anchoring, they undergo a systematic degradation process known as model collapse. First formalized in foundational statistical literature and demonstrated across modern deep l

    AI
    7 min read
  10. Self-Correction and Reflection Loops in Production AI Agents: Architecture, Verification Oracles, and the Over-Correction Trap

    Autonomous AI agents frequently fail on initial generation when solving multi-step reasoning, code generation, and complex API orchestration tasks. To address initial execution failures, system architects widely deploy self-correction and reflection loops. However, the mechanism through which reflection operates determines whether a system converges on a valid solution or degrades into hallucinations and infinite loops. Recent research demonstrates a sharp division in reflection paradigms: whil

    AI
    6 min read
  11. Anthropic Expands Claude Cowork to Web and Mobile, Adds Direct Actions to Gmail and Google Drive

    Anthropic has updated its Claude ecosystem, expanding the reach of its agentic environment Claude Cowork and introducing write-capable actions to its Google Workspace integrations. The updates address two persistent friction points in AI agent deployment: interface accessibility and execution boundaries within external productivity tools. Claude Cowork Expands Across Web and Mobile Claude Cowork, Anthropic's multi-agent workspace for managing multi-step workflows and local project state, was

    AI
    2 min read
  12. The Logit Lens: How Direct Unembedding Decodes Intermediate Representations in Large Language Models

    Autoregressive large language models process input tokens through deep stacks of transformer blocks, generating a final probability distribution over the vocabulary at the final layer. For years, the intermediate computations occurring within these hidden layers were treated as opaque black boxes. Mechanistic interpretability research has demonstrated that modern transformer architectures operate through a structured, iterative refinement process across their residual stream. The logit lens and

    AI
    6 min read
  13. LLM Streaming in Production: Server-Sent Events vs. WebSockets, Incomplete JSON Parsing, and Backpressure Architecture

    Token streaming is the primary user experience mechanism in generative AI applications. By piping generated tokens to the client as they leave the transformer layers rather than awaiting full sequence completion, systems dramatically reduce perceived latency from tens of seconds to sub-second time to first token (TTFT). However, moving from batch request-response models to long-lived streaming pipelines introduces brittle failure modes across the networking stack. In production, engineering tea

    AI
    7 min read
  14. Text-to-SQL in Production: Schema Linking, Value Retrieval, and Execution-Guided Self-Correction

    Translating natural language into executable database queries is one of the most widely deployed applications of large language models in enterprise software. It is also one of the most brittle. On synthetic academic benchmarks such as Spider 1.0, frontier models regularly exceed 90% execution accuracy. However, evaluating those same models on realistic enterprise estates yields a steep drop. On the Spider 2.0 benchmark, which evaluates real-world data warehouses spanning BigQuery and Snowflake

    AI
    7 min read
  15. Data Filtering and Deduplication in Large Language Models: How Heuristics, MinHash LSH, SemDeDup, and Quality Classifiers Curate Pre-Training Corpora

    Data Filtering and Deduplication in Large Language Models: How Heuristics, MinHash LSH, SemDeDup, and Quality Classifiers Curate Pre-Training Corpora Pre-training data quality governs the downstream reasoning, factual accuracy, and sample efficiency of foundation large language models (LLMs). While early pre-training efforts relied on uncurated or lightly processed dumps from web repositories like Common Crawl, modern state-of-the-art models depend on multi-stage data curation pipelines that fi

    AI
    10 min read
  16. Agentic Search and Multi-Hop Retrieval in Production: Query Decomposition, Evidence-Gap Loops, and Synthesis Architecture

    Standard retrieval-augmented generation (RAG) relies on a single-pass paradigm: an incoming prompt is embedded, matched against a vector or hybrid index, and the top-k passages are injected into the generator's context window. This linear workflow functions reliably for direct fact retrieval, but it breaks down on complex research tasks. When a query requires chained dependencies, comparative analysis across isolated documents, or multi-step reasoning, single-shot retrieval fails because the req

    AI
    5 min read
  17. Supervised Fine-Tuning in Large Language Models: Loss Masking, Sequence Packing, and Alignment Dynamics

    Pre-training endows a large language model with broad linguistic patterns, world knowledge, and reasoning primitives by predicting the next token across trillions of uncurated web tokens. However, a raw base model remains a document completer rather than an interactive assistant. Given a prompt such as "Explain how a compiler works," a base model is as likely to generate additional exam questions or web navigation headers as it is to answer the query. Supervised Fine-Tuning (SFT), frequently te

    AI
    7 min read
  18. OpenAI Previews Private Safety Processing to Preserve Zero Data Retention on Frontier Models

    OpenAI has previewed Private Safety Processing, a safety architecture designed to detect multi-turn misuse patterns across frontier model interactions while maintaining Zero Data Retention (ZDR) privacy guarantees for enterprise API customers. Under traditional Zero Data Retention agreements, API customer prompts and generated outputs are not stored on OpenAI servers post-request, are inaccessible to OpenAI personnel, and are excluded from model training datasets. However, evaluating requests s

    OpenAI
    2 min read
  19. Benchmarking LLM Inference in Production: Architecture, Metrics, and Tooling Across AIPerf, GuideLLM, and LLMPerf

    Traditional HTTP load-testing tools such as Apache Bench, wrk, and Locust evaluate systems using uniform request-response cycles. These tools send a payload, wait for the full response, and compute metrics such as requests per second (RPS) and round-trip latency percentiles. For stateless REST APIs, this model aligns directly with user experience. Large language model (LLM) serving fundamentally breaks this abstraction. An inference request is not an atomic computation. It consists of two disti

    AI
    7 min read