Subscribe
Subscribe
Archive / page 25 of 49

Latest articles, page 25

  1. Hallucination Detection and Faithfulness Verification in Production RAG: Architecture, NLI Claim Decomposition, and Runtime Guardrail Economics

    Retrieval-Augmented Generation (RAG) is commonly deployed under the assumption that grounding generation in retrieved passages eliminates factual inaccuracies. In practice, grounding provides an evidence boundary but does not guarantee factual fidelity. For why models hallucinate in the first place, see our AI hallucinations explainer. Production language models regularly synthesize claims absent from the retrieved context (extrinsic hallucinations) or directly assert statements conflicting with

    AI
    6 min read

  2. Sparse Attention and BigBird: How Window, Global, and Random Graphs Preserve Turing Completeness in Linear Time

    Standard self-attention in transformer architectures scales quadratically with sequence length. Computing full pairwise interactions between n tokens requires evaluating an n x n attention matrix, yielding O(n^2) computational complexity and memory consumption. While hardware accelerators and IO-aware tiling algorithms like FlashAttention optimize memory traffic, the quadratic compute and KV footprint remains a barrier for processing long contexts, document-level summarization, and genomic seque

    AI
    7 min read

  3. Oxford Study Details Chinese Gray-Market Proxies Reselling Claude Tokens at 90% Discounts

    An investigation by the Oxford China Policy Lab reveals that Chinese developers routinely access Anthropic's frontier Claude models at discounts between 70% and 90% below list price, bypassing geographical blocks, payment filters, and biometric identity verification through a decentralized network of API proxies known locally as "transfer stations" (中转站). The analysis, authored by Oxford researcher Zilan Qian and published via ChinaTalk, outlines the modular supply chain and economic mechanics

    AI
    3 min read

  4. Low-Precision Quantization Kernels in Production: Comparing Marlin, ExLlamaV2, FlashInfer, and BitBLAS

    Low-Precision Quantization Kernels in Production: Comparing Marlin, ExLlamaV2, FlashInfer, and BitBLAS Architecture, Memory Bandwidth, and Decoding Throughput Autoregressive large language model (LLM) serving operates under two distinct compute regimes: a compute-bound prefill phase and a memory-bandwidth-bound decode phase. While processing the initial prompt involves matrix-matrix multiplications (GEMM) with high arithmetic intensity, generating tokens one by one requires matrix-vector multip

    AI
    9 min read

  5. xLSTM: How Exponential Gating and Matrix Memory Scale Recurrent Neural Networks

    xLSTM: How Exponential Gating and Matrix Memory Scale Recurrent Neural Networks For over two decades following its introduction by Hochreiter and Schmidhuber (1997), the Long Short-Term Memory (LSTM) network served as the dominant architecture for sequence modeling. By introducing the constant error carousel and multiplicative gating, LSTMs mitigated the vanishing gradient problem that plagued vanilla recurrent neural networks. However, the emergence of the Transformer architecture (Vaswani et

    AI
    10 min read

  6. Anthropic Nears $7B Acquisition of AI Infrastructure Startup Decart Ahead of IPO

    Anthropic is finalizing negotiations to acquire Israeli artificial intelligence infrastructure startup Decart in a transaction valued at approximately $7 billion, according to reporting from Calcalist and Reuters. The acquisition, expected to be settled primarily in Anthropic equity, would mark the Claude developer's largest purchase to date as it prepares for a planned initial public offering. Founded in September 2023 by Dr. Dean Leitersdorf and Moshe Shalev, Decart specializes in hardware-ag

    AI
    2 min read

  7. LLM Inference on AMD ROCm in Production: MI300X Architecture, Triton Kernel Parity, and vLLM Serving Benchmarks

    LLM Inference on AMD ROCm in Production: MI300X Architecture, Triton Kernel Parity, and vLLM Serving Benchmarks Serving frontier large language models in enterprise production has historically been synonymous with NVIDIA CUDA infrastructure. However, the deployment of AMD Instinct MI300X accelerators across tier-one hyperscalers and neoclouds has established a viable alternative for high-throughput inference fleets. With 192 GB of high-bandwidth memory (HBM3) and 5.3 TB/s of peak theoretical m

    Inference
    5 min read

  8. Identity Preference Optimization (IPO): How Exact Loss Inversion Prevents Overfitting in Direct Alignment

    The post-training alignment of large language models underwent a structural shift with the introduction of Direct Preference Optimization (Rafailov et al., 2023). By reparameterizing the closed-form optimal policy under a Kullback-Leibler (KL) constrained Reinforcement Learning from Human Feedback (RLHF) objective, DPO eliminated the need to fit an explicit reward model or maintain complex actor-critic rollout loops. However, standard DPO introduces a distinct mathematical pathology: under dete

    AI
    7 min read

  9. AI Workflow Startup Relay Shuts Down as Team Joins Google Chrome to Build Browser Agents

    AI-driven workflow automation startup Relay is shutting down its independent product operations, with founder and chief executive officer Jacob Bank and key engineering staff joining Google's Chrome division to develop browser-native AI agent capabilities. Relay, founded in July 2021 to compete with legacy workflow platforms like Zapier through generative AI integrations, raised $8.1 million across two venture funding rounds. The company phased out free tier access on August 15, 2026, and will

    AI
    2 min read

  10. Groq Secures $350M at $3.5B Valuation to Expand Nvidia-Powered AI Neocloud

    AI infrastructure provider Groq has raised $350 million in a Series A funding round at a $3.5 billion valuation, led by investment firm Disruptive with expected participation from Nvidia subject to customary closing conditions. The financing accelerates the company's structural pivot from developing custom inference silicon toward operating an enterprise-grade inference cloud powered by Nvidia accelerated computing systems. The round follows a $650 million capital raise completed in June 2026 a

    AI
    2 min read