Subscribe
Topic

Diffusion

Diffusion models for images, video and audio, explained and compared. Posts explain DDPM and DDIM, score-based SDEs, Diffusion Transformers and Transfusion, and compare open video models like HunyuanVideo and audio models like MusicGen.

Articles

1-6
  1. Open-Source Music and Audio Generation Models in Production: Comparing MusicGen, Stable Audio Open, AudioLDM 2, and YuE

    Neural music and sound generation architectures in production have diverged into two competing paradigms: discrete autoregressive sequence modeling and continuous latent diffusion. While early research relied on cascaded pipelines with separate acoustic and semantic models, modern open-source systems unify generation into single-stage or dual-stage foundations capable of synthesizing high-fidelity instrumentals, sound effects, and full polyphonic songs with vocals. Deploying generative audio in

    AI Music
    7 min read
  2. Transfusion: Mathematical Foundations of Unified Discrete Next-Token Prediction and Continuous Latent Diffusion in Multimodal Transformers

    Generative foundation models have historically operated under two distinct mathematical paradigms: autoregressive sequence modeling over discrete vocabularies (used in text and code generation) and iterative denoising diffusion over continuous vector spaces (used in image and video synthesis). Early attempts to unify both modalities in a single end-to-end transformer forced continuous perceptual data into discrete codebooks using vector quantization (such as VQ-VAE or VQGAN). While discretizatio

    Education
    6 min read
  3. Open-Source Video Diffusion Models in Production: Comparing HunyuanVideo, CogVideoX, Mochi 1, and LTX-Video

    Generating temporally coherent, high-resolution video from natural language prompts has transitioned from UNet-based frame interpolation pipelines to scalable Diffusion Transformers (DiT) operating over compressed spatiotemporal latent spaces. The release of open-weight video foundation models has established production-viable alternatives to proprietary commercial APIs. However, engineering teams deploying video generation pipelines face stark trade-offs across transformer parameter scaling, 3D

    AI
    7 min read
  4. Denoising Diffusion Probabilistic Models and DDIM: Mathematical Foundations, Variance Schedules, and Fast Deterministic Sampling

    Generative modeling in continuous state spaces underwent a fundamental transformation with the formulation of Denoising Diffusion Probabilistic Models (DDPM) by Ho et al. (2020), building on the non-equilibrium thermodynamics foundations established by Sohl-Dickstein et al. (2015). Prior to diffusion models, deep generative synthesis was dominated by Generative Adversarial Networks (GANs), which suffered from training instability and mode collapse, and Variational Autoencoders (VAEs), which ofte

    AI
    11 min read
  5. Score-Based Generative Modeling via Stochastic Differential Equations: How Continuous SDEs and Score Matching Unify Diffusion Models

    Diffusion and score-based models represent one of the foundational paradigms of modern generative artificial intelligence, underpinning systems across image synthesis, video generation, audio modeling, and continuous multimodal representations. For years, generative diffusion was approached from two distinct perspectives: discrete-step denoising diffusion probabilistic models (DDPM) pioneered by Sohl-Dickstein et al. and Ho et al., and score matching with Langevin dynamics (SMLD / NCSN) introduc

    AI
    8 min read
  6. Diffusion Transformers (DiT): How Patchification and adaLN-Zero Replaced U-Nets in Generative AI

    Generative visual models relied for years on convolutional U-Net architectures to execute iterative denoising. From Denoising Diffusion Probabilistic Models (DDPM) and Ablated Diffusion Models (ADM) to Latent Diffusion Models (LDMs) behind Stable Diffusion, convolutional backbones served as the default engine for image synthesis. While convolutional inductive biases provided translation equivariance and local spatial hierarchies, they imposed architectural rigidities that resisted compute scalin

    AI
    8 min read