Subscribe
Topic / page 14

LLM

Articles, page 14

261-280
  1. Activation-Aware Weight Quantization (AWQ): Mathematical Foundations, Salient Weight Protection, and INT4 Tensor Core Execution

    Activation-Aware Weight Quantization (AWQ): Mathematical Foundations, Salient Weight Protection, and INT4 Tensor Core Execution Large language models have transformed AI applications, but their deployment remains constrained by memory and compute barriers. A 70B parameter model in FP16 occupies ~140 GB of VRAM — exceeding even the 192 GB of NVIDIA's flagship B200 GPU, let alone edge devices. Quantization addresses this by reducing weight precision from 16-bit floats to 4-bit integers, shrinking

    AI
    6 min read
  2. KV Cache Optimization and Prefix Caching in LLM Serving: Comparing RadixAttention, Automatic Prefix Caching, and Static Context Sharing

    Serving large language models in multi-turn conversational agents, complex retrieval-augmented generation (RAG) pipelines, and few-shot reasoning workflows presents a fundamental memory and compute asymmetry. During autoregressive decoding, every newly generated token must attend to all previous tokens in the sequence. To avoid recomputing Key and Value projection matrices at each decoding step, inference engines store intermediate activations in high-bandwidth GPU memory (HBM) as the KV cache.

    AI
    7 min read
  3. Mixture of Experts: How Sparse Activations Scale Models to Trillions of Parameters Without Trillion-Dollar Bills

    Mixture of Experts: How Sparse Activations Scale Models to Trillions of Parameters Without Trillion-Dollar Bills In a dense transformer, every parameter participates in every forward pass. The feed-forward layer — a two-layer perceptron with a hidden expansion of four to eight times the model dimension — alone accounts for roughly two thirds of the FLOPs per token. Scale the model, and cost scales linearly: 10 times the parameters means roughly 10 times the compute at inference, 10 times the me

    AI
    7 min read
  4. OpenAI Expands ChatGPT for Teachers to Over 100,000 Additional Educators

    OpenAI announced on August 26, 2026 that it is bringing ChatGPT for Teachers to more than 100,000 additional educators and staff through new partnerships with 55 school systems across 20 states. The expansion builds on the initial 2025 launch that reached nearly 150,000 teachers and staff, now totaling over 300,000 educators across 30 states. The new cohort includes one in five of the nation’s 20 largest public school districts and some of the most diverse school systems. Alongside the expansi

    AI
    2 min read
  5. Gemini Live Adds Agentic Spark Tasks, Daily Brief and Voice Inbox Control

    Google said on August 26, 2026 that Gemini Live adds Spark agentic background tasks across Docs, Sheets, Drive and the web, a spoken Daily Brief from Gmail and Calendar, hands-free Gmail triage, and Personal Intelligence across chats and apps. Spark needs AI Pro; Daily Brief needs AI Plus.

    AI
    3 min read
  6. Prompt Compression and Context Pruning Engines in Production: Comparing LLMLingua-2, LongLLMLingua, Selective-Context, and RECOMP

    Prompt Compression and Context Pruning Engines in Production: Comparing LLMLingua-2, LongLLMLingua, Selective-Context, and RECOMP Every non-obvious claim below links to a source. Benchmarks are from the papers as cited; the comparative numbers are taken directly from the LLMLingua-2 paper and the RECOMP paper, not synthesized from prose. The context window paradox is real: modern LLMs accept 128k to 1M tokens, but API cost scales linearly with input length, attention compute scales quadratical

    AI
    9 min read
  7. Google Releases Gemini 3.5 Transcribe with Disfluency Filtering and Task Delegation

    Google has launched Gemini 3.5 Transcribe, a dedicated speech-to-text model designed for real-time streaming, automated disfluency cleanup, and agentic task delegation. The release introduces two API interfaces alongside integration across Google developer tooling and consumer operating system surfaces. Dual API Architecture for Live and Batch Audio Gemini 3.5 Transcribe is split into two operational endpoints tailored for distinct latency profiles: * Real-time streaming (gemini-3.5-transcr

    AI
    2 min read
  8. Model Routing and LLM Gateway Architectures in Production: Comparing LiteLLM, Portkey, RouteLLM, and Not Diamond Architecture, Classifier Algorithms, Cascading Fallbacks, and Serving Economics

    Production AI systems rarely rely on a single monolithic language model. As production traffic scales, routing every incoming prompt to a frontier model creates prohibitive compute costs and risks severe provider rate limits. Conversely, standardizing entirely on smaller, lightweight models compromises reasoning depth, domain compliance, and structured extraction quality. To resolve this trade-off, modern AI infrastructure employs a decoupled architectural layer: model gateways and intelligent

    AI
    7 min read
  9. Direct Preference Optimization (DPO): Mathematical Foundations, Implicit Reward Derivation, Closed-Form Bradley-Terry Equivalence, and Reference Policy Regularization

    Direct Preference Optimization (DPO): Mathematical Foundations, Implicit Reward Derivation, Closed-Form Bradley-Terry Equivalence, and Reference Policy Regularization Post-training preference alignment is the central mechanism for transforming pretrained base large language models into instruction-following, steerable, and safe conversational agents. Historically, the dominant paradigm for preference alignment has been Reinforcement Learning from Human Feedback (Christiano et al., 2017; Ouyang

    AI
    15 min read
  10. SandboxAQ Open-Sources Switch, a Coordination Layer for AI Agents in Slack and Teams

    SandboxAQ released Switch on August 26, 2026, and made its agent coordination software free to use (PR Newswire). Switch connects AI agents built with Claude Code, Google's Agent Development Kit, LangChain, OpenAI tooling and other frameworks into shared Slack, Microsoft Teams or Discord channels where employees already work. Architecture Switch turns a chat channel into a "room": a shared workspace where people and agents operate with the same context, resources and history. Agents connect t

    AI
    2 min read
  11. Z.ai releases GLM-5.3-Flash, a 320B parameter hybrid sparse-linear attention model with 18B active parameters

    Z.ai releases GLM-5.3-Flash, a 320B parameter hybrid sparse-linear attention model with 18B active parameters Chinese AI startup Z.ai (formerly Zhipu AI) has released GLM-5.3-Flash, the first natively multimodal model in the GLM-5 series. The model was previously known in stealth as "Ox Alpha" and topped OpenRouter's leaderboard before its official release. GLM-5.3-Flash features a hybrid architecture combining sparse and linear attention with Manifold-Constrained Hyper-Connections (mHC), redu

    AI
    2 min read
  12. MiniMax Reports H1 2026 Revenue Surging 283% YoY to $116.6M Amid China AI Race

    Shanghai-based artificial intelligence foundation model developer MiniMax Group Inc. reported that its revenue increased 283% year-over-year to $116.6 million for the first half of 2026. The financial disclosure, reported by Bloomberg following the company's interim earnings filing on the Hong Kong Stock Exchange, highlights accelerated commercial monetization even as domestic foundation model competition intensifies across China. The 283% top-line expansion in the six months ending June 30, 20

    AI
    2 min read
  13. Label Smoothing Regularization: Mathematical Foundations, Maximum Entropy Dynamics, and the Regular Simplex Geometry in LLMs

    In supervised classification and autoregressive language modeling, standard cross-entropy loss trains neural networks against discrete, one-hot target distributions. While intuitive, one-hot supervision creates an inherent optimization pathology: minimizing the cross-entropy objective to zero requires pushing the logit of the target class toward positive infinity while driving all competing non-target logits toward negative infinity. In deep networks, this dynamic inflates parameter norms, cause

    AI
    11 min read
  14. Tool Retrieval and Dynamic Function Selection in Production AI Agents: Architecture, Semantic Routing, Schema Compression, and Context Bloat Mitigation

    Tool Retrieval and Dynamic Function Selection in Production AI Agents: Architecture, Semantic Routing, Schema Compression, and Context Bloat Mitigation As enterprise AI agents evolve from static single-purpose chatbots into orchestrators interacting with hundreds or thousands of external tools (REST APIs, SQL databases, Model Context Protocol servers, and internal microservices), system architects encounter a fundamental scalability barrier: context bloat and tool interference. In standard fun

    AI
    6 min read
  15. FlashAttention: Mathematical Foundations, Online Softmax Tiling, IO-Awareness, and Exact Attention Scaling

    Standard multi-head self-attention in the Transformer architecture exhibits quadratic time and memory complexity with respect to sequence length $N$. While the $O(N^2)$ computational complexity is widely cited, the primary performance bottleneck in production hardware is not arithmetic throughput, but memory access overhead. On modern GPU architectures such as NVIDIA A100 and H100, tensor processing cores execute matrix multiplications at teraflop and petaflop scales, but memory bandwidth betwee

    AI
    13 min read
  16. Moonshot AI in Revenue-Sharing Talks with Microsoft, Amazon, and Google to Host Kimi K3

    Beijing-based artificial intelligence developer Moonshot AI is negotiating revenue-sharing partnerships with Microsoft, Amazon, and Alphabet's Google to host its flagship open-weight model, Kimi K3, across major cloud platforms. According to reporting from Reuters, the startup is seeking up to a 30 percent share of revenue generated from Kimi K3 inference services hosted on Microsoft Azure, Amazon Web Services (AWS), and Google Cloud. Moonshot released Kimi K3 in July 2026 as a 2.8-trillion par

    AI
    2 min read
  17. Zhipu Confirms Ox Alpha Stealth Model Is GLM Derivative Ahead of Open-Weight Release

    Beijing-based AI lab Z.AI, widely known as Zhipu, has confirmed that the mystery stealth model "Ox Alpha" is an upcoming release in its GLM model series. The company confirmed the model's identity in response to inquiries from Bloomberg on Wednesday and stated it will release the open model weights tonight. Ox Alpha first appeared on August 20, 2026, as an uncredited stealth model on the OpenRouter model routing platform. Operating under a free evaluation tier, the model quickly surged to the t

    AI
    2 min read
  18. LLM Inference Engines in Production: Comparing vLLM, SGLang, TensorRT-LLM, and TGI Architecture, KV Cache Topologies, Kernel Optimizations, and Serving Economics

    LLM Inference Engines in Production: Comparing vLLM, SGLang, TensorRT-LLM, and TGI Architecture, KV Cache Topologies, Kernel Optimizations, and Serving Economics Serving large language models in enterprise production has evolved beyond naive execution runtimes. As context windows expand to 128k+ tokens and agentic workloads generate complex multi-turn execution graphs, the efficiency of the underlying inference engine dictates both latency Service Level Objectives (SLOs) and hardware infrastruc

    LLM
    7 min read
  19. Transformers as Support Vector Machines: Mathematical Foundations of Implicit Bias, Margin Maximization, and Hard-Attention Convergence

    Understanding why overparameterized deep neural networks generalize well despite possessing sufficient capacity to memorize training data remains a fundamental question in theoretical machine learning. In classical linear models trained on separable data under exponential-tailed losses (such as logistic or cross-entropy loss), gradient descent exhibits an implicit bias: parameter iterates diverge in norm while their directions asymptotically converge to the maximum-margin hyperplane, known as th

    AI
    9 min read