llms.blog
edu3 articles

edu

Articles

  • Non-Contrastive Representation Learning: How Barlow Twins and VICReg Prevent Feature Collapse via Redundancy Reduction

    Self-supervised representation learning provides the foundation for modern foundation models across computer vision, audio, and multimodal systems. By training deep neural networks to produce compact vector embeddings without human annotations, self-supervised pre-training enables models to capture rich semantic structures directly from raw data. Historically, the dominant approach to self-supervised learning was contrastive learning, popularized by architectures such as SimCLR (Chen et al., 20

    1 min

  • Neural Collapse: How Simplex Equiangular Tight Frames Emerge at the Terminal Phase of Training

    In classification tasks, deep neural networks exhibit an unexpected geometric simplicity during late-stage optimization. While the internal activations of early training appear high-dimensional and complex, the penultimate layer representations and linear classifiers converge toward an exact, symmetrical geometric structure known as Neural Collapse (NC). First identified empirically by Papyan, Han, and Donoho (2020), Neural Collapse emerges during the Terminal Phase of Training (TPT). This regi

    1 min

  • Neural Tangent Kernel: How Infinite-Width Networks Linearize Gradient Descent

    title: "Neural Tangent Kernel: How Infinite-Width Networks Linearize Gradient Descent" slug: "neural-tangent-kernel-how-infinite-width-networks-linearize-gradient-descent" feature_image: "https://cms.llms.blog/content/images/2026/08/neural-tangent-kernel-cover.png" tags: ["edu", "theory", "foundations"] status: published The Neural Tangent Kernel (NTK) describes the behavior of infinitely wide neural networks during gradient descent. In the infinite-width limit, network training reduces to kern

    1 min