Subscribe
Latest / page 21

The latest stories

1230 articles

Articles, page 21

401-420 of 1230

  1. Matryoshka Representation Learning: Mathematical Foundations, Nested Subspace Optimization, and Adaptive Dimension Truncation in Modern Embedding Systems

    In modern deep learning systems, embedding vectors serve as the foundational intermediate representation for search, recommendation, retrieval-augmented generation (RAG), and zero-shot classification. Historically, these representations have been rigid: a model trained to produce 1536-dimensional or 2048-dimensional vectors outputs a fixed-width array for every input. Downstream systems must ingest, store, index, and compute distance metrics across the entire dimensional width, regardless of whe

    AI
    8 min read

  2. DeepSeek Generates $70.7M in Revenue with $106M Net Loss in First Seven Months of 2026

    Hangzhou-based artificial intelligence laboratory DeepSeek generated approximately 475 million yuan ($70.7 million) in revenue and recorded a net loss of $106 million during the first seven months of 2026, according to financial figures reported by The Information. The performance marks a roughly tenfold revenue surge compared to the lab's full-year 2025 revenue, alongside a modest contraction in net burn from the $139 million net loss reported for all of 2025. The disclosures provide a rare ac

    AI
    3 min read

  3. Activation-Aware Weight Quantization (AWQ): Mathematical Foundations, Salient Weight Protection, and Hardware-Efficient Low-Bit Inference

    Large language model inference during autoregressive generation is overwhelmingly memory bandwidth bound. While the prefill phase processes multiple prompt tokens in parallel with high arithmetic intensity, the token generation phase computes matrix-vector multiplications ($M=1$) for each sequential token. In this regime, the GPU spends the vast majority of its cycle budget streaming model parameters from High Bandwidth Memory (HBM) or GDDR into SRAM rather than performing floating-point arithme

    AI
    7 min read

  4. Intel Details 256-Core Xeon 7 and 480GB Crescent Island Inference GPU at Hot Chips 2026

    At the Hot Chips 2026 conference, Intel outlined architectural disclosures for three upcoming computing platforms tailored for AI workflows: the Xeon 7 enterprise processor (codename Diamond Rapids), the Crescent Island data center inference GPU, and the Wildcat Lake client processor (Intel Core Series 3). The announcements detail Intel's shift toward modular multi-die packaging, open chiplet interconnect standards, and expanded on-chip memory to meet the computational demands of multi-agent AI

    AI
    3 min read

  5. GPTQ: Mathematical Foundations, Optimal Brain Surgeon Inversion, and Second-Order Error Minimization in LLM Quantization

    Large language model inference during autoregressive generation is overwhelmingly memory-bandwidth bound. For batch size 1 decoding, each generated token requires streaming every parameter of a model from High Bandwidth Memory (HBM) into GPU SRAM and Tensor Cores. A 70-billion parameter model in 16-bit precision (FP16 or BF16) requires roughly 140 GB of VRAM, exceeding the capacity of a single 80 GB NVIDIA A100 or H100 GPU and demanding multi-GPU tensor parallelism solely to hold the model weigh

    AI
    14 min read

  6. AM Intelligence Orders 9,000 Nvidia Vera Rubin Systems for $8B AI Infrastructure Project

    Indian AI infrastructure platform AM Intelligence (AMI) has placed a binding purchase order for 9,000 Nvidia Vera Rubin computing systems. The procurement represents one of the earliest hyperscale commitments for Nvidia's next-generation Rubin architecture across Asia and anchors an $8 billion capital expenditure initiative to build 1 gigawatt (GW) of dedicated AI computing capacity. The first phase of the deployment will take place at AMI's upcoming data center facility in Hyderabad, India. Th

    AI
    2 min read

  7. LLM Evaluation Frameworks in Production: Comparing Promptfoo, DeepEval, Ragas, and Inspect Architecture, Metric Calibration, and Quality Gate Economics

    Testing large language model applications in production requires shifting from deterministic software unit tests to probabilistic evaluation harnesses. Traditional software engineering relies on binary assertions (assert output == expected), but generative models exhibit non-deterministic outputs, variable token distributions, and nuanced semantic drift across prompt revisions, model updates, and temperature configurations. To prevent regressions and quantify system capabilities before deployme

    Evaluation
    7 min read

  8. SmoothQuant: Mathematical Foundations, Per-Channel Outlier Migration, and Hardware-Efficient W8A8 Inference in Large Language Models

    SmoothQuant: Mathematical Foundations, Per-Channel Outlier Migration, and Hardware-Efficient W8A8 Inference in Large Language Models Serving large language models (LLMs) in production environments presents two distinct hardware bottlenecks. During the autoregressive generation (decode) phase with small batch sizes, inference is memory-bandwidth bound, as billions of parameters must be streamed from High Bandwidth Memory (HBM) to on-chip SRAM for every generated token. Conversely, during the pro

    Quantization
    12 min read

  9. Digs Raises $25.3M Series A Led by Builders FirstSource for Residential Construction AI

    Digs, a startup developing AI software for residential construction management, has raised a $25.3 million Series A funding round led by building materials supplier Builders FirstSource. Alongside the equity investment, the two companies entered into a five-year commercial partnership to deploy Digs' document intelligence and digital twin platform across Builders FirstSource's distribution network. The Series A brings Digs' total funding to more than $47 million, following seed and pre-Series A

    Enterprise AI
    2 min read

  10. XPeng Robotics Raises $900M at $6.3B Valuation to Scale IRON Humanoid Production

    XPeng announced that its robotics subsidiary has secured over $900 million in private capital at a post-money valuation exceeding $6.3 billion. The transaction marks the largest single-round private financing in China's physical AI and humanoid robotics sector to date. The funding round was led by IDG Capital, with participation from Gaorong Ventures as well as strategic backing from internet conglomerates Tencent and Alibaba. Concurrently, XPeng Chairman and Chief Executive Officer He Xiaopeng

    Physical AI
    2 min read

  11. FP8 Mixed-Precision Serving in Production: Comparing E4M3 vs. E5M2 Formats, Per-Tensor vs. Block-Wise Dynamic Scaling, FP8 KV Cache, and Tensor Core GEMM Economics

    FP8 Mixed-Precision Serving in Production: Comparing E4M3 vs. E5M2 Formats, Per-Tensor vs. Block-Wise Dynamic Scaling, FP8 KV Cache, and Tensor Core GEMM Economics Serving large language models at enterprise scale requires balancing computational throughput against high-bandwidth memory (HBM) capacity. While 16-bit floating-point formats (FP16 and BF16) remain the standard for model pre-training and fine-tuning, their memory footprint and arithmetic bandwidth create severe bottlenecks during hi

    Inference
    6 min read

  12. Odds Ratio Preference Optimization (ORPO): Mathematical Foundations, Monolithic SFT-Alignment, and Reference-Free Odds Penalties

    Post-training large language models typically relies on a decoupled, two-stage pipeline: Supervised Fine-Tuning (SFT) on curated instruction-response pairs, followed by Preference Alignment using algorithms such as Reinforcement Learning from Human Feedback (PPO) or Direct Preference Optimization (DPO). While effective, this multi-stage paradigm introduces substantial operational and computational friction. First, DPO and PPO require maintaining auxiliary reference models in GPU memory to preve

    AI
    9 min read

  13. Ringg Raises $10M from Peak XV to Scale Enterprise Voice AI Agents

    Indian voice AI startup Ringg has secured $10 million in fresh capital from Peak XV Partners as an extension to its Series A round. The funding brings the startup's total Series A capital to $15.5 million, following an initial $5.5 million tranche closed earlier in 2026. The capital injection comes as voice remains the dominant channel for commercial communication in India. According to survey data from Truecaller, over 76% of Indian consumers prefer phone interactions when contacting businesse

    AI Voice
    2 min read

  14. QLoRA: Mathematical Foundations of 4-Bit NormalFloat Quantization, Double Quantization, and Paged Optimizers

    Fine-tuning large language models in full 16-bit precision requires substantial hardware infrastructure. For a 65-billion parameter model, storing weights in 16-bit brain floating point (BF16) requires 130 GB of VRAM. When optimizing those parameters with 32-bit AdamW, storing gradients (130 GB), first moments (260 GB), second moments (260 GB), and master weights (260 GB) drives the minimum memory footprint beyond 780 GB of VRAM, even before accounting for sequence activations. While Low-Rank A

    AI
    7 min read

  15. Projecting Conflicting Gradients (PCGrad): Mathematical Foundations, Orthogonal Projections, and Multi-Task Optimization in Deep Learning

    Projecting Conflicting Gradients (PCGrad): Mathematical Foundations, Orthogonal Projections, and Multi-Task Optimization in Deep Learning In modern machine learning systems, models rarely optimize for a single objective. Foundation models are trained simultaneously on diverse data distributions spanning natural language, source code, mathematical reasoning, and multimodal inputs. Similarly, post-training alignment pipelines must simultaneously optimize for helpfulness, factual accuracy, harmles

    AI
    11 min read

  16. Amazon to Shut Down Mechanical Turk on September 30 After 21 Years

    Amazon Web Services has announced that it will permanently shut down Amazon Mechanical Turk on September 30, 2026. The retirement brings an end to a 21-year-old crowdsourcing platform that served as an early foundation for modern machine learning dataset annotation and human-in-the-loop computing. In a formal notice posted to the Mechanical Turk portal, AWS stated that the decision followed an internal assessment of its service portfolio. The platform had previously halted new customer registra

    Amazon
    3 min read

  17. Hierarchical KV Cache Offloading in Production LLM Serving: Host RAM, Local NVMe, Remote Storage, and LMCache Architecture

    High-concurrency large language model (LLM) serving faces an acute memory capacity bottleneck. While modern GPUs deliver high floating-point compute throughput, High-Bandwidth Memory (HBM) capacity remains severely constrained. In workloads involving multi-turn conversations, agentic coding loops, and long-document retrieval-augmented generation (RAG), Key-Value (KV) cache tensors rapidly consume available GPU memory, forcing engines to either reject requests, preempt running sequences, or recom

    AI
    9 min read

  18. Mixture-of-Depths: Mathematical Foundations, Dynamic Compute Routing, Capacity-Constrained Tensors, and IsoFLOP Scaling

    Mixture-of-Depths (MoD): Mathematical Foundations, Dynamic Compute Routing, Capacity-Constrained Tensors, and IsoFLOP Scaling In standard autoregressive Transformer architectures, computational effort is distributed uniformly across all tokens in a sequence. Every token position $i \in \{1, \dots, S\}$ passes through every layer $l \in \{1, \dots, L\}$, executing identical matrix multiplications across multi-head self-attention and feed-forward networks (FFN). This architectural constraint igno

    AI
    11 min read

  19. LLM Gateways in Production: Comparing LiteLLM, Portkey, Kong AI Gateway, and Cloudflare AI Gateway Architecture, Fallback Cascades, and Serving Economics

    As enterprise production architectures scale from single prototype endpoints to multi-model agentic pipelines, coupling application code directly to foundation model provider APIs creates severe operational bottlenecks. Direct client integration leads to fractured telemetry, credential sprawl across microservices, unhandled upstream rate limits, and vendor lock-in. To isolate application logic from downstream provider volatility, engineering teams deploy dedicated LLM gateways. These specialize

    AI
    8 min read

  20. OpenAI Data Center Lead Chris Malone Departs Amid Infrastructure Realignment

    OpenAI head of data centers Chris Malone has departed the company, marking another high-level transition within the artificial intelligence lab as it restructures its infrastructure divisions ahead of a planned initial public offering. Malone, who joined OpenAI in March 2025 after five years leading data center strategy at Meta and over a decade as an engineer at Google, oversaw technical deployment across OpenAI's compute expansion efforts. Infrastructure Realignment Malone's exit coincides

    AI
    1 min read