Representation Learning4 articles

Representation Learning

Articles

Alignment and Uniformity on the Hypersphere: How Geometric Losses Govern Contrastive Representation Learning
Alignment and Uniformity on the Hypersphere: The Geometric Foundations of Contrastive Representation Learning Contrastive representation learning serves as the foundational objective behind modern neural embeddings, powering dense retrieval systems, visual-language models such as CLIP, and metric learning pipelines. While early literature justified contrastive learning through the InfoMax principle (maximizing mutual information between augmented views), theoretical and empirical analyses have
August 24, 20261 min
SimCLR and MoCo: How Momentum Encoders, Dynamic Queues, and InfoNCE Scaled Self-Supervised Representation Learning
Self-supervised visual representation learning underwent a foundational transformation in 2020 with the introduction of contrastive learning frameworks. Prior to this period, pre-training computer vision models without human annotations relied on heuristic pretext tasks such as jigsaw puzzle solving, rotation prediction, or autoencoding color channels. These methods forced networks to learn low-level geometric or statistical artifacts rather than generalizable semantic abstractions. Contrastive
August 24, 20261 min
Non-Contrastive Representation Learning: How Barlow Twins and VICReg Prevent Feature Collapse via Redundancy Reduction
Self-supervised representation learning provides the foundation for modern foundation models across computer vision, audio, and multimodal systems. By training deep neural networks to produce compact vector embeddings without human annotations, self-supervised pre-training enables models to capture rich semantic structures directly from raw data. Historically, the dominant approach to self-supervised learning was contrastive learning, popularized by architectures such as SimCLR (Chen et al., 20
August 23, 20261 min
Noise-Contrastive Estimation and InfoNCE: How Partition Function Estimation and Mutual Information Lower Bounds Power Modern AI
Noise-Contrastive Estimation and InfoNCE: How Partition Function Elimination and Mutual Information Lower Bounds Power Modern AI In statistical machine learning and generative modeling, evaluating the exact probability of an observed event frequently requires calculating a normalizing constant known as the partition function. For continuous spaces or discrete spaces with high cardinality, such as a natural language vocabulary spanning over one hundred thousand tokens or high-dimensional pixel d
August 23, 20261 min

Representation Learning

Articles

Alignment and Uniformity on the Hypersphere: How Geometric Losses Govern Contrastive Representation Learning

SimCLR and MoCo: How Momentum Encoders, Dynamic Queues, and InfoNCE Scaled Self-Supervised Representation Learning

Non-Contrastive Representation Learning: How Barlow Twins and VICReg Prevent Feature Collapse via Redundancy Reduction

Noise-Contrastive Estimation and InfoNCE: How Partition Function Estimation and Mutual Information Lower Bounds Power Modern AI