Subscribe
Topic

Machine Learning

Machine learning techniques behind modern AI, explained with the math and the production tooling. Recent posts cover flow matching, machine unlearning in LLMs, preference optimization methods like DPO and KTO, and fine-tuning frameworks such as Unsloth and Axolotl.

Articles

1-20
  1. Chinese Mathematicians Prove 3D Ghys Conjecture Using Shanghai AI Lab's Shusheng-S2 and Lean Verification

    A research team led by mathematicians from Great Bay University and the Shanghai Artificial Intelligence Laboratory has published a complete proof of the three-dimensional Ghys conjecture, resolving a 31-year-old open problem in dynamical systems. The proof relied on Shanghai AI Lab's Shusheng-S2 multimodal scientific foundation model to explore deduction pathways, followed by formal computer verification in the Lean proof assistant conducted by researchers at Peking University. The achievement

    AI
    3 min read
  2. Open-Source Safety and Moderation Classifiers in Production: Comparing Llama Guard 3, ShieldGemma, WildGuard, and Prompt-Guard

    Production large language model (LLM) applications require structured defenses against adversarial jailbreaks, prompt injections, and policy-violating content. While guardrail orchestration frameworks handle workflow logic, the actual enforcement decisions rely on specialized safety classifier models. Open-source safety classifiers have diverged into distinct architectural paradigms, ranging from lightweight bidirectional encoders to full autoregressive decoders and multi-task refusal evaluator

    AI
    5 min read
  3. Machine Unlearning in Large Language Models: Mathematical Foundations of Gradient Ascent, Negative Preference Optimization, and Representation Misdirection

    Machine Unlearning in Large Language Models: Mathematical Foundations of Gradient Ascent, Negative Preference Optimization, and Representation Misdirection Machine unlearning in large language models (LLMs) addresses the challenge of removing specific training data, copyrighted material, private identifiers, or hazardous capabilities from a pre-trained model without retraining the network from scratch. Because training frontier language models requires millions of GPU-hours and petabyte-scale c

    AI
    8 min read
  4. Mixture-of-Depths (MoD): Mathematical Foundations, Dynamic Token-Level Compute Routing, Top-k Capacity Budgeting, and FLOP-Optimal Transformer Architectures

    Standard autoregressive Transformers allocate an identical compute budget to every token in a sequence. Regardless of whether a token represents a trivial punctuation mark, a common grammatical function word, or a complex semantic reasoning step, the model applies the exact same sequence of multi-head self-attention and multilayer perceptron (MLP) operations across all $L$ layers. In Mixture-of-Depths: Dynamically allocating compute in transformer-based language models, researchers at Google De

    AI
    6 min read
  5. Simple Preference Optimization (SimPO): Mathematical Foundations, Reference-Free Implicit Reward, Length Normalization, and Target Margin Dynamics

    Simple Preference Optimization (SimPO): Mathematical Foundations, Reference-Free Implicit Reward, Length Normalization, and Target Margin Dynamics Post-training preference alignment has become the definitive step in transforming raw pretrained large language models into instruction-following assistants. While Reinforcement Learning from Human Feedback (RLHF) via Proximal Policy Optimization (PPO) established the initial standard, its requirement to maintain actor, critic, reference, and reward

    AI
    11 min read
  6. Contrastive Preference Optimization (CPO): Mathematical Foundations, Direct Likelihood Regularization, Out-of-Distribution Error Bounds, and Non-Degenerate Policy Alignment

    Supervised fine-tuning (SFT) and preference alignment represent the standard two-stage training paradigm for post-training large language models. While SFT establishes foundational task adherence by minimizing cross-entropy loss against target references, it suffers from two structural limitations: it cannot surpass the quality ceiling of its training demonstrations, and it lacks an explicit mechanism to penalize near-correct outputs that contain subtle hallucinations or omissions. Direct Prefe

    AI
    11 min read
  7. Post-Training Quantization Toolkits in Production: Comparing AutoAWQ, AutoGPTQ, bitsandbytes, and llm-compressor Architecture, Calibration Profiling, Kernel Execution, and Serving Economics

    Large language model deployment in high-concurrency production environments is governed primarily by memory bandwidth and hardware economics. In standard autoregressive decoding, memory traffic dominates computation: for every token generated, every parameter of the model must be loaded from GPU High Bandwidth Memory (HBM) into SRAM and tensor cores. Serving a 70-billion parameter model in 16-bit precision requires at least 140 GB of VRAM solely for model weights, necessitating multi-GPU tensor

    LLM
    9 min read
  8. Activation Checkpointing and Selective Rematerialization: Mathematical Foundations, Sub-Linear Memory Scaling, Recomputation Trade-Offs, and Megatron-LM Attention Partitioning

    Activation Checkpointing and Selective Rematerialization: Mathematical Foundations, Sub-Linear Memory Scaling, Recomputation Trade-Offs, and Megatron-LM Attention Partitioning Training modern large language models requires orchestrating hundreds of billions of parameters across distributed GPU clusters. While parameter counts and optimizer states are fixed for a given model architecture, the activation memory generated during forward passes scales directly with sequence length, batch size, and

    AI
    10 min read
  9. Chinchilla Scaling Laws: Mathematical Foundations of Compute-Optimal Pre-Training, IsoFLOP Loss Profiles, Parametric Power Laws, and Data-Compute Allocation

    Chinchilla Scaling Laws: Mathematical Foundations of Compute-Optimal Pre-Training, IsoFLOP Loss Profiles, Parametric Power Laws, and Data-Compute Allocation When allocating a fixed computational budget to train an autoregressive Transformer, engineers face a fundamental trade-off: should FLOPs be spent increasing the model parameter count ($N$), or should they be spent streaming a larger volume of training tokens ($D$)? For several years, frontier AI development followed the empirical scaling

    AI
    14 min read
  10. Synthetic Data Generation and Data Curation Pipelines in Production: Comparing Distilabel, NVIDIA NeMo Curator, Data-Juicer, and InstructLab

    As frontier language models exhaust human-authored web text, pretraining and post-training performance increasingly depends on automated data curation and synthetic data synthesis. Unfiltered public corpora introduce duplicate content, synthetic artifacts, low-reasoning text, and licensing liabilities. Modern foundation models, including Meta's Llama 3.1, NVIDIA's Nemotron-4 340B, Alibaba's Qwen 2.5, and IBM's Granite, are trained on datasets where synthetic generation and rigorous multi-stage f

    Data Curation
    9 min read
  11. Knowledge Distillation in Large Language Models: Mathematical Foundations of Forward vs. Reverse KL Divergence, Sequence-Level Distillation (SeqKD), and On-Policy Mode-Seeking Objectives

    Deploying frontier large language models in latency-critical and compute-constrained environments remains an active challenge in artificial intelligence systems. While parameter scale correlates directly with downstream generalization and emergent reasoning capabilities, the memory footprint and FLOP intensity of dense multi-hundred-billion parameter architectures impose prohibitive operational costs. Post-training compression strategies such as quantization and structural pruning reduce precisi

    AI
    11 min read
  12. Model Merging in Weight Space: Mathematical Foundations of Task Vectors, TIES-Merging, DARE, and Spherical Linear Interpolation

    Training large language models across multiple downstream domains typically requires expensive multi-task supervised fine-tuning (SFT) or sequential continual learning. Both paths present steep practical challenges: multi-task training demands simultaneous access to massive, curated datasets and massive compute budgets, while sequential fine-tuning suffers from catastrophic forgetting. Model merging has emerged as a data-free, compute-efficient alternative. By combining the parameters of multip

    AI
    8 min read
  13. LLM Fine-Tuning and Post-Training Frameworks in Production: Comparing Unsloth, Axolotl, Torchtune, and LLaMA-Factory

    Post-training has transitioned from a specialized research task into a standard production engineering discipline. As open-weight base models such as Llama 3.1, Qwen 2.5, and DeepSeek-V3 establish competitive baselines, the primary engineering challenge has shifted toward domain adaptation, instruction alignment, and reasoning distillation. However, selecting a post-training framework requires balancing competing architectural trade-offs: low-level kernel fusion, distributed multi-node scaling,

    AI
    6 min read
  14. LLM Evaluation Frameworks in Production: Comparing DeepEval, Ragas, Promptfoo, and TruLens Architecture, Synthetic Data Generation, Judge Calibration, and CI/CD Automation

    Automated evaluation is the primary engineering bottleneck in deploying reliable LLM applications. While traditional software engineering relies on deterministic unit and integration tests with binary pass/fail conditions, generative AI systems produce non-deterministic, open-ended natural language outputs. Relying on manual human review or ad-hoc prompting fails to catch regressions across prompt modifications, model version updates, and retrieval pipeline adjustments. To establish rigorous qu

    AI
    8 min read
  15. Direct Preference Optimization (DPO): Mathematical Foundations, Implicit Reward Derivation, Closed-Form Bradley-Terry Equivalence, and Reference Policy Regularization

    Direct Preference Optimization (DPO): Mathematical Foundations, Implicit Reward Derivation, Closed-Form Bradley-Terry Equivalence, and Reference Policy Regularization Post-training preference alignment is the central mechanism for transforming pretrained base large language models into instruction-following, steerable, and safe conversational agents. Historically, the dominant paradigm for preference alignment has been Reinforcement Learning from Human Feedback (Christiano et al., 2017; Ouyang

    AI
    15 min read
  16. Label Smoothing Regularization: Mathematical Foundations, Maximum Entropy Dynamics, and the Regular Simplex Geometry in LLMs

    In supervised classification and autoregressive language modeling, standard cross-entropy loss trains neural networks against discrete, one-hot target distributions. While intuitive, one-hot supervision creates an inherent optimization pathology: minimizing the cross-entropy objective to zero requires pushing the logit of the target class toward positive infinity while driving all competing non-target logits toward negative infinity. In deep networks, this dynamic inflates parameter norms, cause

    AI
    11 min read
  17. Transformers as Support Vector Machines: Mathematical Foundations of Implicit Bias, Margin Maximization, and Hard-Attention Convergence

    Understanding why overparameterized deep neural networks generalize well despite possessing sufficient capacity to memorize training data remains a fundamental question in theoretical machine learning. In classical linear models trained on separable data under exponential-tailed losses (such as logistic or cross-entropy loss), gradient descent exhibits an implicit bias: parameter iterates diverge in norm while their directions asymptotically converge to the maximum-margin hyperplane, known as th

    AI
    9 min read