Subscribe
Research / page 7

Research

470 articles

Articles, page 7

121-140 of 470

  1. Transformers as Support Vector Machines: Mathematical Foundations of Implicit Bias, Margin Maximization, and Hard-Attention Convergence

    Understanding why overparameterized deep neural networks generalize well despite possessing sufficient capacity to memorize training data remains a fundamental question in theoretical machine learning. In classical linear models trained on separable data under exponential-tailed losses (such as logistic or cross-entropy loss), gradient descent exhibits an implicit bias: parameter iterates diverge in norm while their directions asymptotically converge to the maximum-margin hyperplane, known as th

    AI
    9 min read

  2. Matryoshka Representation Learning: Mathematical Foundations, Nested Subspace Optimization, and Adaptive Dimension Truncation in Modern Embedding Systems

    In modern deep learning systems, embedding vectors serve as the foundational intermediate representation for search, recommendation, retrieval-augmented generation (RAG), and zero-shot classification. Historically, these representations have been rigid: a model trained to produce 1536-dimensional or 2048-dimensional vectors outputs a fixed-width array for every input. Downstream systems must ingest, store, index, and compute distance metrics across the entire dimensional width, regardless of whe

    AI
    8 min read

  3. Activation-Aware Weight Quantization (AWQ): Mathematical Foundations, Salient Weight Protection, and Hardware-Efficient Low-Bit Inference

    Large language model inference during autoregressive generation is overwhelmingly memory bandwidth bound. While the prefill phase processes multiple prompt tokens in parallel with high arithmetic intensity, the token generation phase computes matrix-vector multiplications ($M=1$) for each sequential token. In this regime, the GPU spends the vast majority of its cycle budget streaming model parameters from High Bandwidth Memory (HBM) or GDDR into SRAM rather than performing floating-point arithme

    AI
    7 min read

  4. GPTQ: Mathematical Foundations, Optimal Brain Surgeon Inversion, and Second-Order Error Minimization in LLM Quantization

    Large language model inference during autoregressive generation is overwhelmingly memory-bandwidth bound. For batch size 1 decoding, each generated token requires streaming every parameter of a model from High Bandwidth Memory (HBM) into GPU SRAM and Tensor Cores. A 70-billion parameter model in 16-bit precision (FP16 or BF16) requires roughly 140 GB of VRAM, exceeding the capacity of a single 80 GB NVIDIA A100 or H100 GPU and demanding multi-GPU tensor parallelism solely to hold the model weigh

    AI
    14 min read

  5. LLM Evaluation Frameworks in Production: Comparing Promptfoo, DeepEval, Ragas, and Inspect Architecture, Metric Calibration, and Quality Gate Economics

    Testing large language model applications in production requires shifting from deterministic software unit tests to probabilistic evaluation harnesses. Traditional software engineering relies on binary assertions (assert output == expected), but generative models exhibit non-deterministic outputs, variable token distributions, and nuanced semantic drift across prompt revisions, model updates, and temperature configurations. To prevent regressions and quantify system capabilities before deployme

    Evaluation
    7 min read

  6. SmoothQuant: Mathematical Foundations, Per-Channel Outlier Migration, and Hardware-Efficient W8A8 Inference in Large Language Models

    SmoothQuant: Mathematical Foundations, Per-Channel Outlier Migration, and Hardware-Efficient W8A8 Inference in Large Language Models Serving large language models (LLMs) in production environments presents two distinct hardware bottlenecks. During the autoregressive generation (decode) phase with small batch sizes, inference is memory-bandwidth bound, as billions of parameters must be streamed from High Bandwidth Memory (HBM) to on-chip SRAM for every generated token. Conversely, during the pro

    Quantization
    12 min read

  7. Digs Raises $25.3M Series A Led by Builders FirstSource for Residential Construction AI

    Digs, a startup developing AI software for residential construction management, has raised a $25.3 million Series A funding round led by building materials supplier Builders FirstSource. Alongside the equity investment, the two companies entered into a five-year commercial partnership to deploy Digs' document intelligence and digital twin platform across Builders FirstSource's distribution network. The Series A brings Digs' total funding to more than $47 million, following seed and pre-Series A

    Enterprise AI
    2 min read

  8. FP8 Mixed-Precision Serving in Production: Comparing E4M3 vs. E5M2 Formats, Per-Tensor vs. Block-Wise Dynamic Scaling, FP8 KV Cache, and Tensor Core GEMM Economics

    FP8 Mixed-Precision Serving in Production: Comparing E4M3 vs. E5M2 Formats, Per-Tensor vs. Block-Wise Dynamic Scaling, FP8 KV Cache, and Tensor Core GEMM Economics Serving large language models at enterprise scale requires balancing computational throughput against high-bandwidth memory (HBM) capacity. While 16-bit floating-point formats (FP16 and BF16) remain the standard for model pre-training and fine-tuning, their memory footprint and arithmetic bandwidth create severe bottlenecks during hi

    Inference
    6 min read

  9. Odds Ratio Preference Optimization (ORPO): Mathematical Foundations, Monolithic SFT-Alignment, and Reference-Free Odds Penalties

    Post-training large language models typically relies on a decoupled, two-stage pipeline: Supervised Fine-Tuning (SFT) on curated instruction-response pairs, followed by Preference Alignment using algorithms such as Reinforcement Learning from Human Feedback (PPO) or Direct Preference Optimization (DPO). While effective, this multi-stage paradigm introduces substantial operational and computational friction. First, DPO and PPO require maintaining auxiliary reference models in GPU memory to preve

    AI
    9 min read

  10. QLoRA: Mathematical Foundations of 4-Bit NormalFloat Quantization, Double Quantization, and Paged Optimizers

    Fine-tuning large language models in full 16-bit precision requires substantial hardware infrastructure. For a 65-billion parameter model, storing weights in 16-bit brain floating point (BF16) requires 130 GB of VRAM. When optimizing those parameters with 32-bit AdamW, storing gradients (130 GB), first moments (260 GB), second moments (260 GB), and master weights (260 GB) drives the minimum memory footprint beyond 780 GB of VRAM, even before accounting for sequence activations. While Low-Rank A

    AI
    7 min read

  11. Projecting Conflicting Gradients (PCGrad): Mathematical Foundations, Orthogonal Projections, and Multi-Task Optimization in Deep Learning

    Projecting Conflicting Gradients (PCGrad): Mathematical Foundations, Orthogonal Projections, and Multi-Task Optimization in Deep Learning In modern machine learning systems, models rarely optimize for a single objective. Foundation models are trained simultaneously on diverse data distributions spanning natural language, source code, mathematical reasoning, and multimodal inputs. Similarly, post-training alignment pipelines must simultaneously optimize for helpfulness, factual accuracy, harmles

    AI
    11 min read

  12. Mixture-of-Depths: Mathematical Foundations, Dynamic Compute Routing, Capacity-Constrained Tensors, and IsoFLOP Scaling

    Mixture-of-Depths (MoD): Mathematical Foundations, Dynamic Compute Routing, Capacity-Constrained Tensors, and IsoFLOP Scaling In standard autoregressive Transformer architectures, computational effort is distributed uniformly across all tokens in a sequence. Every token position $i \in \{1, \dots, S\}$ passes through every layer $l \in \{1, \dots, L\}$, executing identical matrix multiplications across multi-head self-attention and feed-forward networks (FFN). This architectural constraint igno

    AI
    11 min read

  13. LLM Gateways in Production: Comparing LiteLLM, Portkey, Kong AI Gateway, and Cloudflare AI Gateway Architecture, Fallback Cascades, and Serving Economics

    As enterprise production architectures scale from single prototype endpoints to multi-model agentic pipelines, coupling application code directly to foundation model provider APIs creates severe operational bottlenecks. Direct client integration leads to fractured telemetry, credential sprawl across microservices, unhandled upstream rate limits, and vendor lock-in. To isolate application logic from downstream provider volatility, engineering teams deploy dedicated LLM gateways. These specialize

    AI
    8 min read

  14. Activation Addition and Representation Engineering: Mathematical Foundations, Linear Subspace Projections, and Inference-Time Steering in Large Language Models

    Large language models encode vast linguistic, factual, and behavioral properties within their internal hidden representations. While traditional alignment and behavioral modification rely on Supervised Fine-Tuning (SFT) or Reinforcement Learning from Human Feedback (RLHF), these gradient-based techniques modify billions of parameter weights, require substantial compute, and frequently suffer from catastrophic forgetting or alignment tax. An alternative paradigm grounded in mechanistic interpret

    AI
    6 min read

  15. Synthetic Data Pipelines in Production LLM Post-Training: Architecture, Prompt Evolution, Quality Filtering, and Contamination Control

    Scaling supervised fine-tuning (SFT) and preference alignment (DPO, PPO, GRPO) through human annotation faces severe economic and operational constraints. Human annotation costs between $5.00 and $50.00 per complex instruction-response trajectory, exhibits significant variance across labeler cohorts, and scales linearly with dataset volume. The LIMA study by Zhou et al. (2023) showed that post-training requires strict data quality rather than raw volume, yet modern frontier models demand million

    AI
    6 min read

  16. Induction Heads: Mathematical Foundations, Two-Layer Circuit Composition, and the Emergence of In-Context Learning in Transformers

    Induction Heads: Mathematical Foundations, Two-Layer Circuit Composition, and the Emergence of In-Context Learning in Transformers One of the defining capabilities of modern autoregressive large language models is in-context learning: the ability to infer rules, adapt to task formats, and execute complex few-shot instructions purely from prompt context without updating neural network weights. For years, in-context learning was treated as an enigmatic, emergent property of large-scale autoregres

    AI
    13 min read

  17. Figure AI Unveils Index Platform with 16 Million Crowdsourced Videos for Robot Foundation Models

    Humanoid robotics startup Figure AI has launched Index, a global crowdsourced data collection platform engineered to capture real-world human task demonstrations at scale. Operating in stealth for four months prior to its public unveiling, the platform has compiled 16 million video demonstrations from contributors across 108 countries, generating embodied training data for Figure's physical AI foundation models. The initiative directly targets the primary bottleneck in scaling embodied AI: the

    AI
    2 min read

  18. Anthropic Claude Autonomously Designs Validated Protein Binders Across 14 Targets

    Anthropic has released results from an autonomous de novo protein binder design campaign run by its Claude models, with wet-lab validation from two outside evaluators. Claude designed binders against 15 targets and succeeded against 14 of them, according to Anthropic. Across all campaigns, 27% of designs bound, and 49% of the designs Claude ranked first for each target bound, according to Anthropic's technical report. The campaign used Claude Opus 4.8 and Claude Mythos Preview inside Claude Sci

    AI
    3 min read

  19. Pipeline Parallelism in Production LLM Training: Comparing 1F1B, Interleaved 1F1B, Zero-Bubble, and DualPipe Schedules

    Pipeline Parallelism in Production LLM Training: Comparing 1F1B, Interleaved 1F1B, Zero-Bubble, and DualPipe Schedules Training modern large language models spanning hundreds of billions of parameters requires distributing model layers across multiple compute nodes. While Tensor Parallelism (TP) partitions individual matrix multiplications across GPUs within a single node, its reliance on high-frequency, all-reduce communications limits its practical scaling to the high-bandwidth domain of NVLi

    AI
    8 min read

  20. Anthropic Launches $5M Grant Program to Fund AI Wellbeing Evaluations and Benchmarks

    Anthropic has launched a $5 million grant initiative to support independent development of open-source benchmarks and evaluation harnesses measuring the impact of artificial intelligence systems on user wellbeing. The program will supply research teams with direct financial grants, subsidized API access to Claude models, and technical support from Anthropic's Safeguards team. All evaluation frameworks, datasets, and grading methodology developed under the grant program will be released publicly

    Anthropic
    2 min read