llms.blog
Diffusion Models3 articles

Diffusion Models

Articles

  • Discrete Diffusion in Large Language Models: How Continuous-Time Markov Chains, Absorbing States, and Score Entropy Challenge Autoregressive Generation

    The dominance of autoregressive architectures in large language models rests on a fundamental mathematical formulation: the chain rule of probability. By factoring the joint distribution of a sequence into a product of conditional probabilities, $p(x) = \prod_{i=1}^N p(x_i \mid x_{<i})$, autoregressive models reduce text generation to sequential next-token prediction. While this left-to-right causal factorization has scaled effectively across compute regimes, it imposes rigid operational constr

    1 min

  • Flow Matching for Generative Modeling: How Continuous Normalizing Flows and Optimal Transport Paths Replace Diffusion SDEs

    Flow Matching for Generative Modeling: How Continuous Normalizing Flows and Optimal Transport Paths Replace Diffusion SDEs Generative modeling underwent a structural shift with the introduction of Flow Matching (FM), formulated independently by Lipman et al. (2022), Albergo and Vanden-Eijnden (2022), and Liu et al. (2022). While Denoising Diffusion Probabilistic Models (DDPM) and score-based Stochastic Differential Equations (SDEs) established state-of-the-art sample quality across vision and a

    1 min

  • Classifier-Free Guidance: How Score Extrapolation and Implicit Classification Steer Generative Models

    Conditional generative models face an inherent tension between mode coverage and prompt adherence. When a model is trained to maximize data log-likelihood, its learned distribution matches the broad, messy variety of the underlying dataset. In unconditional generation, this diversity is desirable. In conditional generation, however, unconditional priors dilute the prompt: models generate generic, average samples that only weakly align with nuanced text descriptions, spatial layouts, or class lab

    1 min