Subscribe
Topic / page 2

Machine Learning

Articles, page 2

21-40
  1. Matryoshka Representation Learning: Mathematical Foundations, Nested Subspace Optimization, and Adaptive Dimension Truncation in Modern Embedding Systems

    In modern deep learning systems, embedding vectors serve as the foundational intermediate representation for search, recommendation, retrieval-augmented generation (RAG), and zero-shot classification. Historically, these representations have been rigid: a model trained to produce 1536-dimensional or 2048-dimensional vectors outputs a fixed-width array for every input. Downstream systems must ingest, store, index, and compute distance metrics across the entire dimensional width, regardless of whe

    AI
    8 min read
  2. GPTQ: Mathematical Foundations, Optimal Brain Surgeon Inversion, and Second-Order Error Minimization in LLM Quantization

    Large language model inference during autoregressive generation is overwhelmingly memory-bandwidth bound. For batch size 1 decoding, each generated token requires streaming every parameter of a model from High Bandwidth Memory (HBM) into GPU SRAM and Tensor Cores. A 70-billion parameter model in 16-bit precision (FP16 or BF16) requires roughly 140 GB of VRAM, exceeding the capacity of a single 80 GB NVIDIA A100 or H100 GPU and demanding multi-GPU tensor parallelism solely to hold the model weigh

    AI
    14 min read
  3. Odds Ratio Preference Optimization (ORPO): Mathematical Foundations, Monolithic SFT-Alignment, and Reference-Free Odds Penalties

    Post-training large language models typically relies on a decoupled, two-stage pipeline: Supervised Fine-Tuning (SFT) on curated instruction-response pairs, followed by Preference Alignment using algorithms such as Reinforcement Learning from Human Feedback (PPO) or Direct Preference Optimization (DPO). While effective, this multi-stage paradigm introduces substantial operational and computational friction. First, DPO and PPO require maintaining auxiliary reference models in GPU memory to preve

    AI
    9 min read
  4. QLoRA: Mathematical Foundations of 4-Bit NormalFloat Quantization, Double Quantization, and Paged Optimizers

    Fine-tuning large language models in full 16-bit precision requires substantial hardware infrastructure. For a 65-billion parameter model, storing weights in 16-bit brain floating point (BF16) requires 130 GB of VRAM. When optimizing those parameters with 32-bit AdamW, storing gradients (130 GB), first moments (260 GB), second moments (260 GB), and master weights (260 GB) drives the minimum memory footprint beyond 780 GB of VRAM, even before accounting for sequence activations. While Low-Rank A

    AI
    7 min read
  5. Projecting Conflicting Gradients (PCGrad): Mathematical Foundations, Orthogonal Projections, and Multi-Task Optimization in Deep Learning

    Projecting Conflicting Gradients (PCGrad): Mathematical Foundations, Orthogonal Projections, and Multi-Task Optimization in Deep Learning In modern machine learning systems, models rarely optimize for a single objective. Foundation models are trained simultaneously on diverse data distributions spanning natural language, source code, mathematical reasoning, and multimodal inputs. Similarly, post-training alignment pipelines must simultaneously optimize for helpfulness, factual accuracy, harmles

    AI
    11 min read
  6. Mixture-of-Depths: Mathematical Foundations, Dynamic Compute Routing, Capacity-Constrained Tensors, and IsoFLOP Scaling

    Mixture-of-Depths (MoD): Mathematical Foundations, Dynamic Compute Routing, Capacity-Constrained Tensors, and IsoFLOP Scaling In standard autoregressive Transformer architectures, computational effort is distributed uniformly across all tokens in a sequence. Every token position $i \in \{1, \dots, S\}$ passes through every layer $l \in \{1, \dots, L\}$, executing identical matrix multiplications across multi-head self-attention and feed-forward networks (FFN). This architectural constraint igno

    AI
    11 min read
  7. Synthetic Data Pipelines in Production LLM Post-Training: Architecture, Prompt Evolution, Quality Filtering, and Contamination Control

    Scaling supervised fine-tuning (SFT) and preference alignment (DPO, PPO, GRPO) through human annotation faces severe economic and operational constraints. Human annotation costs between $5.00 and $50.00 per complex instruction-response trajectory, exhibits significant variance across labeler cohorts, and scales linearly with dataset volume. The LIMA study by Zhou et al. (2023) showed that post-training requires strict data quality rather than raw volume, yet modern frontier models demand million

    AI
    6 min read
  8. Induction Heads: Mathematical Foundations, Two-Layer Circuit Composition, and the Emergence of In-Context Learning in Transformers

    Induction Heads: Mathematical Foundations, Two-Layer Circuit Composition, and the Emergence of In-Context Learning in Transformers One of the defining capabilities of modern autoregressive large language models is in-context learning: the ability to infer rules, adapt to task formats, and execute complex few-shot instructions purely from prompt context without updating neural network weights. For years, in-context learning was treated as an enigmatic, emergent property of large-scale autoregres

    AI
    13 min read
  9. Context Parallelism in Production: Comparing DeepSpeed Ulysses, Megatron Sequence Parallelism, Ring Attention, and Unified Sequence Parallelism (USP) Architecture, Communication Overhead, and Scale-Out Economics

    Scaling context windows in Large Language Models (LLMs) from 8,192 tokens to 128k, 256k, and 1M tokens introduces severe memory and compute constraints. Standard Data Parallelism (DP) duplicates entire model activations per sequence, failing when a single sequence exceeds GPU VRAM. Tensor Parallelism (TP) splits weight matrices across GPUs, but scaling TP beyond a single 8-GPU node degrades throughput due to high-frequency, low-latency cross-GPU collective communications. Pipeline Parallelism (P

    AI
    10 min read
  10. Process Reward Models (PRMs): Mathematical Foundations, Step-Level Credit Assignment, and Inference-Time Search in Large Language Models

    In complex multi-step reasoning tasks such as mathematics, formal logic, and software verification, standard autoregressive language models frequently exhibit compounding logical drift. While a model may produce fluent chain-of-thought derivations, an error introduced early in the chain typically invalidates all downstream deductions. Evaluating these reasoning trajectories has traditionally relied on Outcome-Supervised Reward Models (ORMs), which assign a scalar reward solely based on the final

    AI
    8 min read
  11. Kahneman-Tversky Optimization (KTO): Mathematical Foundations, Prospect Theory, and Binary Signal Alignment in Large Language Models

    Post-training alignment of large language models has long relied on pairwise comparison datasets. Methods such as Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO) assume access to curated pairs $(x, y_w, y_l)$, where a human or automated judge explicitly marks completion $y_w$ as superior to $y_l$ for a given prompt $x$. In production environments, however, collecting paired preferences is logistically complex, expensive, and artificial. Real-world telem

    Explainers
    10 min read
  12. Low-Rank Adaptation (LoRA) and DoRA: Mathematical Foundations, Intrinsic Dimensionality, and Directional Weight Decomposition

    Low-Rank Adaptation (LoRA) and DoRA: Mathematical Foundations, Intrinsic Dimensionality, and Directional Weight Decomposition Full-parameter fine-tuning of large language models presents prohibitive memory requirements during training. Standard full fine-tuning requires updating every weight matrix in a model, demanding storage not only for the model parameters themselves, but also for activation tensors, backward gradients, and first- and second-moment optimizer states. For a 70-billion-parame

    AI
    15 min read
  13. Model Merging in Production: Comparing Task Arithmetic, TIES-Merging, DARE, and SLERP Architecture, Weight Interference Mitigation, and Serving Economics

    Model Merging in Production: Comparing Task Arithmetic, TIES-Merging, DARE, and SLERP Architecture, Weight Interference Mitigation, and Serving Economics In production language model deployment, teams frequently encounter a fundamental operational tension: specialized task performance versus serving cost. Fine-tuning distinct base model instances on specific internal domains (such as code generation, legal document analysis, mathematical reasoning, and safety alignment) produces high task accur

    AI
    9 min read
  14. Hugging Face Introduces gr.Workflow to Turn AI Pipelines into Visual Graphs and REST APIs

    Hugging Face has released gr.Workflow, a native extension to the Gradio framework designed to convert multi-stage artificial intelligence pipelines into interactive node graphs, visual user interfaces, and deployable REST APIs. Modern machine learning applications increasingly rely on compound pipelines that chain heterogeneous models: generating text via large language models, feeding prompts into diffusion systems, processing outputs through background removal or audio synthesis models, and a

    AI
    2 min read
  15. Batch Normalization: Mathematical Foundations, Gradient Smoothing Dynamics, and Why Sequence Models Adopted Layer Normalization

    Batch Normalization remains one of the most widely implemented algorithmic developments in the history of deep learning. Introduced by Sergey Ioffe and Christian Szegedy in their 2015 paper, Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, the technique enabled stable training of deep feedforward networks and convolutional architectures at significantly higher learning rates. While initially designed for computer vision architectures such as ResNet a

    AI
    7 min read
  16. Wasserstein GANs: How Earth Mover's Distance, Kantorovich-Rubinstein Duality, and Gradient Penalties Fixed Generative Adversarial Training

    Wasserstein GANs: How Earth Mover's Distance, Kantorovich-Rubinstein Duality, and Gradient Penalties Fixed Generative Adversarial Training Generative Adversarial Networks (Goodfellow et al., 2014) established an influential paradigm for generative modeling: formulating data synthesis as a minimax game between a generator mapping latent noise to data space and a discriminator distinguishing synthetic samples from real empirical data. Despite early empirical success, the original GAN formulation

    AI
    11 min read
  17. Latent Reasoning in Large Language Models: How Continuous Thoughts and Recurrent Hidden States Bypass Discrete Tokenization

    Standard autoregressive language models solve multi-step reasoning tasks by generating explicit verbal scratchpads. Under the Chain-of-Thought (CoT) paradigm formalized by Wei et al. (2022), a Transformer expands its effective computational depth by emitting intermediate natural language tokens into the prompt context. Each emitted token provides an additional forward pass through the network's layers, transforming reasoning into a sequence of left-to-right text predictions. While language-base

    AI
    8 min read
  18. Unlikelihood Training in Large Language Models: How Negative Candidate Loss and Sequence-Level Penalties Suppress Repetition Loops and Hallucinations

    Unlikelihood Training in Large Language Models: How Negative Candidate Loss and Sequence-Level Penalties Suppress Repetition Loops and Hallucinations Autoregressive language models generate text by iteratively predicting the probability distribution of the next token conditioned on preceding tokens. While standard maximum likelihood estimation (MLE) via cross-entropy loss serves as the universal training objective across modern foundation models, it possesses a fundamental structural asymmetry:

    AI
    13 min read
  19. The Fisher Information Matrix and Natural Gradient Descent: How Information Geometry, Riemannian Manifolds, and Curvature Invariance Reshape Neural Optimization

    The Fisher Information Matrix and Natural Gradient Descent: How Information Geometry, Riemannian Manifolds, and Curvature Invariance Reshape Neural Optimization Standard gradient descent operates under an implicit, often unexamined assumption: that parameter space is Euclidean. When an optimizer updates network weights via $\theta_{t+1} = \theta_t - \eta \nabla_\theta \mathcal{L}(\theta)$, it calculates the direction of steepest descent within an arbitrary coordinate system. If the network is r

    AI
    10 min read
  20. Agent Self-Evolution and Experiential Learning in Production: Trajectory Reflection, Skill Library Curation, and Verifiable Policy Distillation

    Deploying autonomous AI agents into complex environments reveals a persistent operational bottleneck: frozen model weights. When an agent fails at a non-trivial workflow, traditional engineering setups rely on humans to diagnose the failure, rewrite prompt templates, adjust few-shot exemplars, or add custom heuristic wrappers. This manual iteration loop fails to scale across diverse, long-horizon production environments where agents encounter thousands of unique edge cases daily. To break this

    AI
    6 min read