edu3 articles

edu

Articles

Non-Contrastive Representation Learning: How Barlow Twins and VICReg Prevent Feature Collapse via Redundancy Reduction
Self-supervised representation learning provides the foundation for modern foundation models across computer vision, audio, and multimodal systems. By training deep neural networks to produce compact vector embeddings without human annotations, self-supervised pre-training enables models to capture rich semantic structures directly from raw data. Historically, the dominant approach to self-supervised learning was contrastive learning, popularized by architectures such as SimCLR (Chen et al., 20
August 23, 20261 min
Neural Collapse: How Simplex Equiangular Tight Frames Emerge at the Terminal Phase of Training
In classification tasks, deep neural networks exhibit an unexpected geometric simplicity during late-stage optimization. While the internal activations of early training appear high-dimensional and complex, the penultimate layer representations and linear classifiers converge toward an exact, symmetrical geometric structure known as Neural Collapse (NC). First identified empirically by Papyan, Han, and Donoho (2020), Neural Collapse emerges during the Terminal Phase of Training (TPT). This regi
August 22, 20261 min
Neural Tangent Kernel: How Infinite-Width Networks Linearize Gradient Descent
title: "Neural Tangent Kernel: How Infinite-Width Networks Linearize Gradient Descent" slug: "neural-tangent-kernel-how-infinite-width-networks-linearize-gradient-descent" feature_image: "https://cms.llms.blog/content/images/2026/08/neural-tangent-kernel-cover.png" tags: ["edu", "theory", "foundations"] status: published The Neural Tangent Kernel (NTK) describes the behavior of infinitely wide neural networks during gradient descent. In the infinite-width limit, network training reduces to kern
August 22, 20261 min

edu

Articles

Non-Contrastive Representation Learning: How Barlow Twins and VICReg Prevent Feature Collapse via Redundancy Reduction

Neural Collapse: How Simplex Equiangular Tight Frames Emerge at the Terminal Phase of Training

Neural Tangent Kernel: How Infinite-Width Networks Linearize Gradient Descent