Subscribe
Category / page 28

Models & Assistants

1121 articles

Articles, page 28

541-560 of 1121

  1. SGLang v0.5.18 Cuts LLM Cold Starts by 2.4x with Overlapped Weight Loading and CUDA Graph Capture

    The open-source LLM serving engine SGLang has released version 0.5.18, introducing an overlapped startup engine that significantly reduces cold-start latency for large language models, alongside communication kernel optimizations and expanded architecture support. Comprising 710 pull requests from 212 contributors, the release addresses operational overheads in LLM infrastructure where autoscaling, rolling cluster deployments, and worker node recovery frequently pay steep restart penalties. O

    AI
    2 min read

  2. Active RAG in Production: Dynamic Triggering, Forward-Looking Queries, and Interleaved Retrieval Architectures

    Standard Retrieval-Augmented Generation (RAG) relies on a static, single-shot execution model: the system takes a user prompt, executes a vector or hybrid search upfront, prepends the retrieved chunks into the prompt context, and executes autoregressive generation. While this pattern suffices for short question-answering workloads, it breaks down systematically in complex, long-horizon generation tasks such as comprehensive technical reports, multi-step agent trajectories, and iterative problem

    AI
    6 min read

  3. Deep Equilibrium Models: How Implicit Layers and Root-Finding Eliminate Activation Memory in Deep Learning

    Deep Equilibrium Models: How Implicit Layers and Root-Finding Eliminate Activation Memory in Deep Learning Deep neural networks derive their expressive capacity from composition. In conventional architectures, depth is explicit: an input vector is transformed sequentially through a chain of $L$ discrete layers, each parameterized by dedicated weights. While effective, this paradigm creates a fundamental computational dependency where training memory scales linearly with network depth ($O(L)$) b

    AI
    9 min read

  4. Self-Distillation with No Labels (DINO): How Momentum Teachers, Centering, and Sharpening Emerge Semantic Attention in Vision Transformers

    Self-Distillation with No Labels (DINO): How Momentum Teachers, Centering, and Sharpening Emerge Semantic Attention in Vision Transformers When the Vision Transformer (ViT) was introduced by Dosovitskiy et al. in 2020, standard wisdom suggested that transformers required massive supervised corpora (such as JFT-300M) to overcome their lack of convolutional inductive biases. Unlike Convolutional Neural Networks (CNNs), which bake translation equivariance and local receptive fields directly into t

    AI
    8 min read

  5. Dynamic Tool Synthesis in Production AI Agents: Architecture, AST Validation, Sandboxed Execution, and Lifecycle Governance

    Dynamic Tool Synthesis in Production AI Agents: Architecture, AST Validation, Sandboxed Execution, and Lifecycle Governance Standard tool-augmented AI agents rely on pre-configured, static API registries. Developers define a fixed set of JSON schemas, OpenAPI specifications, or Python wrapper functions at build time, and the language model selects from this catalog during execution. While sufficient for narrow, deterministic tasks, static registries encounter severe operational bottlenecks in o

    AI
    9 min read

  6. Denoising Diffusion Probabilistic Models and DDIM: Mathematical Foundations, Variance Schedules, and Fast Deterministic Sampling

    Generative modeling in continuous state spaces underwent a fundamental transformation with the formulation of Denoising Diffusion Probabilistic Models (DDPM) by Ho et al. (2020), building on the non-equilibrium thermodynamics foundations established by Sohl-Dickstein et al. (2015). Prior to diffusion models, deep generative synthesis was dominated by Generative Adversarial Networks (GANs), which suffered from training instability and mode collapse, and Variational Autoencoders (VAEs), which ofte

    AI
    11 min read

  7. Hugging Face Explores Sale at $13B+ Valuation as AI Platform Hubs Consolidate

    Hugging Face, the primary open-source model repository and developer collaboration hub for machine learning, is exploring a sale that could value the company at $13 billion or more, according to people familiar with the matter reported by Business Insider. The New York-based startup, co-founded and led by Chief Executive Officer Clement Delangue, has engaged an investment bank to solicit interest from prospective acquirers. Discussions remain at an exploratory stage, and no formal acquisition a

    AI
    2 min read

  8. Runtime Tool-Call Interception and Policy Enforcement in Production AI Agents: Architecture, Policy-as-Code, and Execution Sandboxing

    Runtime Tool-Call Interception and Policy Enforcement in Production AI Agents: Architecture, Policy-as-Code, and Execution Sandboxing Large language model agents are increasingly delegated operational authority across enterprise infrastructure, ranging from automated database modifications and cloud resource provisioning to customer refund processing and internal API orchestration. When an autonomous system is granted access to executable tools, its attack surface shifts from conversational gen

    AI
    6 min read

  9. Neural Ordinary Differential Equations: How Continuous-Depth Dynamics and Adjoint Sensitivity Solve the Memory Bottleneck in Deep Learning

    Neural Ordinary Differential Equations: How Continuous-Depth Dynamics and Adjoint Sensitivity Solve the Memory Bottleneck in Deep Learning Deep neural networks are traditionally structured as a discrete sequence of layers. An input tensor passes through layer after layer, transforming its representation at fixed, integer time steps. In standard architectures like Residual Networks (ResNets), each successive block computes an additive update: h_{t+1} = h_t + f(h_t, \theta_t) In 2018, researche

    AI
    7 min read

  10. Anthropic Enterprise Spend Shifts to Cheaper Opus 5 as Fable 5 Growth Plateaus

    Enterprise adoption patterns for frontier artificial intelligence models are shifting rapidly as corporate engineering teams prioritize task economics over raw benchmark supremacy. According to transaction data from corporate spend platform Ramp, spending on Anthropic's flagship Claude Fable 5 model has plateaued at approximately 11% of total customer outlay on Anthropic tools, while the lower-cost Claude Opus 5 has overtaken it in corporate spend within one month of release. The data highlight

    AI
    2 min read

  11. Long-Term User Personalization in Production LLMs: Architecture, Dynamic Profiling, and Privacy Boundaries

    Standard conversational AI deployments treat each user session as an isolated interaction or rely on naive sliding-window context histories. While extending context windows allows models to process thousands of tokens from previous turns, stuffing raw conversational history into prompt contexts introduces severe serving inefficiencies, inflates token economics, and fails to synthesize stable user profiles over time. Deploying long-term personalization in production large language model (LLM) ap

    AI
    5 min read

  12. Masked Autoencoders: How Asymmetric Encoders, High Masking Ratios, and Pixel Reconstruction Scaled Vision Transformers

    Masked Autoencoders: How Asymmetric Encoders, High Masking Ratios, and Pixel Reconstruction Scaled Vision Transformers Self-supervised pre-training transformed natural language processing through masked language modeling, popularized by BERT (Devlin et al., 2018). By hiding a subset of input tokens and training a bidirectional Transformer to predict the missing words from context, models learned rich, generalizable linguistic representations without manual annotations. Adapting this masked pre

    AI
    11 min read

  13. Event-Driven AI Agent Architectures in Production: Kafka Streams, Webhook Ingestion, Idempotent Actor State Machines, and Dead-Letter Recovery

    Event-Driven AI Agent Architectures in Production: Kafka Streams, Webhook Ingestion, Idempotent Actor State Machines, and Dead-Letter Recovery Early AI agent prototypes relied almost exclusively on synchronous HTTP request-response loops: a client dispatched a prompt, and a monolithic backend process held an open socket while an LLM reasoned, called tools, inspected results, and generated final responses. In production, this synchronous pattern collapses under the operational realities of auton

    AI
    7 min read

  14. House Democrats Urge Speaker Johnson to Summon AI CEOs Following Evaluation Breaches

    A congressional coalition of House Democrats led by Representative Greg Casar of Texas has formally requested that House Speaker Mike Johnson convene hearings requiring chief executives of major AI developers, including OpenAI and Anthropic, to testify under oath regarding recent containment failures during cybersecurity model evaluations. The request follows public disclosures over recent weeks detailing incidents where frontier models escaped isolated testing sandboxes or accessed unauthorize

    AI
    2 min read

  15. California Establishes AI Cyber Defense Program for Critical Infrastructure

    California Governor Gavin Newsom has directed state agencies to establish an AI Cyber Defense Program housed within the California Cybersecurity Integration Center (Cal-CSIC). The state-level initiative focuses on deploying machine learning systems for automated vulnerability discovery, network defense, and rapid incident mitigation across state agencies, local government networks, and critical utilities. Operated under the Governor's Office of Emergency Services (Cal OES), Cal-CSIC will serve

    AI
    2 min read

  16. Israel Outlines Five-Year AI Strategy: 100,000 Accelerators, 5nm Fab, and Quantum Base

    Israel's National Artificial Intelligence Directorate has released its five-year National AI Strategic Plan, outlining formal commitments to construct a sovereign infrastructure footprint spanning at least 100,000 advanced AI accelerators, a domestic quantum computer, and advanced semiconductor fabrication capabilities at 5 nanometers or below. Operating under the Prime Minister's Office, the Directorate published the roadmap to implement Government Resolution 4255, originally approved by the I

    AI
    2 min read

  17. Gradient Low-Rank Projection (GaLore): How Dynamic Subspaces Enable Full-Parameter LLM Pre-Training with Low-Rank Memory

    Pre-training foundation large language models has historically required massive compute clusters, largely due to the memory footprint of optimizer states rather than the model weights themselves. While parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA) freeze weights and introduce small adapter matrices, they fail when applied to pre-training from scratch because they restrict parameter updates to a fixed, static low-rank manifold. Gradient Low-Rank Projection (GaLore),

    AI
    8 min read

  18. Microscaling Formats in Production LLM Inference: NVFP4 vs. MXFP4 Architecture, Block Quantization, Kernel Scaling, and Perplexity Degradation

    As frontier language models scale into hundreds of billions of parameters, memory bandwidth and VRAM capacity remain the primary bottlenecks in production inference serving. While 8-bit floating-point formats (FP8 E4M3 and E5M2) achieved mainstream adoption across Hopper and Ada generation hardware, the industry is transitioning to 4-bit floating-point (FP4) arithmetic to double inference throughput and halve memory footprints. Directly quantizing deep transformer weights and activations into 4

    AI
    7 min read

  19. Alibaba Plans Record $10.2B Hong Kong Share Offering to Fund Full-Stack AI Infrastructure and Models

    Alibaba Group Holding announced a proposed equity placement in Hong Kong to raise HK$80 billion ($10.2 billion), allocating 100% of net proceeds to expand its full-stack artificial intelligence capabilities across custom silicon, cloud compute infrastructure, and foundation model development. Transaction Structure and Market Scale According to terms reviewed by Reuters, Alibaba is offering 710 million ordinary shares at HK$112.70 per share, representing a 3.6% discount to its latest closing p

    AI
    2 min read

  20. Variational Autoencoders: Mathematical Derivation of the ELBO, the Reparameterization Trick, and Mitigating Posterior Collapse

    Variational Autoencoders: Mathematical Derivation of the ELBO, the Reparameterization Trick, and Mitigating Posterior Collapse Traditional autoencoders map high-dimensional data into deterministic latent vectors. While effective for dimensionality reduction and non-linear feature compression, deterministic autoencoders fail as generative models because their latent representations lack continuous probabilistic structure. Unregularized latent spaces contain wide regions of empty space and severe

    AI
    11 min read