Subscribe
Topic / page 29

AI

Articles, page 29

561-580
  1. Arithmetic Intensity and the Roofline Model: Why LLM Generation Is Memory-Bound and Prefill Is Compute-Bound

    Arithmetic Intensity and the Roofline Model in Large Language Models: Why Generation Is Memory-Bound and Prefill Is Compute-Bound Every optimization in modern large language model (LLM) serving exists to solve a single hardware reality: running a Transformer model is not one computational problem, but two completely distinct workloads operating on opposite sides of a physical performance barrier. When an LLM processes a prompt during the prefill phase, it saturates modern GPU Tensor Cores, ope

    AI
    9 min read
  2. DeepSeek Unveils Experimental Vision Model Challenging Anthropic's Opus 4.8

    DeepSeek Unveils Experimental Vision Model Challenging Anthropic's Opus 4.8 DeepSeek announced an experimental multimodal version of its V4 Flash model that can analyze visual prompts, claiming near-parity with Anthropic's Opus 4.8 on multimodal agentic benchmarks. The new release, deepseek-v4-flash-vision-exp, extends DeepSeek's flagship text-only V4 Flash model with vision capabilities. The experimental model processes images alongside text, enabling use cases like describing pictures, rea

    AI
    2 min read
  3. Anthropic Deploys Claude Mythos 5 in Security Tools, Commits $35M to Open-Source Defense Fund

    Anthropic Deploys Claude Mythos 5 in Security Tools, Commits $35M to Open-Source Defense Fund Claude Mythos 5 is now running vulnerability scans in Claude Security for Enterprise customers, marking the third stage of Anthropic's deliberate rollout of its cyber-capable model. Anthropic announced on August 21 that Claude Mythos 5, limited to vetted security defenders since April 2026, is now available for Enterprise customers to scan codebases for vulnerabilities. The company is also launching

    AI
    2 min read
  4. Nscale Targets $3 Billion U.S. IPO as Soon as September

    Nscale Targets $3 Billion U.S. IPO as Soon as September Nscale signage during the Nvidia GTC conference in San Jose, California. London-based AI infrastructure startup Nscale is preparing for a U.S. initial public offering that could raise up to $3 billion, according to people familiar with the matter. The offering may launch as early as September, joining a wave of AI data center companies tapping public markets. Goldman Sachs and JPMorgan Chase are leading the IPO process, Bloomberg reporte

    AI
    2 min read
  5. Agent Memory Engines in Production: Comparing Mem0, Zep, and Letta Architecture, Graph Stores, and Retrieval Latency

    Large language models are inherently stateless functions. Every invocation begins with an empty memory register, relying entirely on the tokens packed into its context window. In production multi-turn applications, naive context management strategies quickly collapse: appending raw conversational logs inflates inference costs and eventually hits hard context limits, while standard semantic vector search (retrieval-augmented generation) lacks temporal awareness, treats contradictions blindly, and

    AI
    7 min read
  6. NVIDIA Researchers Map Cross-Model KV Caches via Linear Ridge Regression to Accelerate Model Cascading Up to 25x

    In multi-agent systems and tiered model cascades, routing requests across different model sizes is a standard architecture for balancing inference cost and generation quality. However, switching models mid-session has historically imposed a severe compute tax: because each neural architecture maintains its own distinct internal representations, the receiving model cannot read the key-value (KV) states generated by the previous model. The target model must recompute the entire conversation histor

    AI
    4 min read
  7. Post-Training RL Frameworks in Production: Comparing verl, OpenRLHF, TRL, and DeepSpeed-Chat Architecture, Distributed Scheduling, and Serving Trade-Offs

    Post-training reinforcement learning (RL) has replaced standard supervised fine-tuning (SFT) as the primary mechanism for frontier model alignment and reasoning expansion. Whether running classic Proximal Policy Optimization (PPO), Direct Preference Optimization (DPO), Group Relative Policy Optimization (GRPO), or Reinforcement Learning with Verifiable Rewards (RLVR), modern post-training pipelines introduce an execution pattern distinct from standard pre-training: they alternate between high-th

    AI
    8 min read
  8. Chunked and Fused Cross-Entropy: How Online Logit Tiling Slashes Large-Vocabulary VRAM Bottlenecks in LLM Training

    Chunked and Fused Cross-Entropy: How Online Logit Tiling Slashes Large-Vocabulary VRAM Bottlenecks in LLM Training As frontier large language models have scaled, tokenizer vocabularies have expanded substantially. Where early architectures such as LLaMA and Mistral relied on 32,000 subword tokens, contemporary models routinely employ vocabularies of 128,256 tokens (Llama 3), 152,064 tokens (Qwen 2.5), and 256,000 tokens (Gemma 2). Larger vocabularies compress text more densely, improve multilin

    AI
    7 min read
  9. Kakao Splits Into KakaoAI and KakaoX to Accelerate AI and Messenger Integration

    South Korean platform giant Kakao Corp. announced a corporate split that will separate its core operations into two independent publicly traded entities: KakaoAI and KakaoX. The restructuring, approved by Kakao's board of directors, aims to isolate and accelerate the company's artificial intelligence engineering and messaging ecosystem from its broader investment portfolio. Under the spin-off terms, existing shareholders will receive shares based on a net asset book value split ratio of 36% for

    AI
    1 min read
  10. US Warns 35 Partner Countries to Choose Between Pax Silica and China's WAICO AI Coalition

    The U.S. Department of State is preparing formal diplomatic notices instructing 35 partner nations to select between Washington's AI alliance and Beijing's competing framework. According to a draft cable reviewed by Reuters and reported by The Decoder and CNBC, the U.S. warns that countries joining China's newly established AI initiative will be excluded from the U.S.-led Pax Silica coalition. The diplomatic draft states: "To be part of everything is to be part of nothing. Signature of the Pax

    AI
    2 min read
  11. Emergent Outlier Features in Large Language Models: Why Hidden Dimension Spikes Arise at Scale and How They Reshape Quantization

    Emergent Outlier Features in Large Language Models: Why Hidden Dimension Spikes Arise at Scale and How They Reshape Quantization When language models scale past approximately 6.7 billion parameters, their internal representations undergo a sharp qualitative phase transition. In smaller models (125M to 2.7B parameters), hidden state activations remain relatively compact, bounded within predictable normal distributions across all embedding dimensions. However, as demonstrated by Dettmers et al. (

    AI
    8 min read
  12. NVIDIA Takes Minority Stake in Cloverleaf Infrastructure to Accelerate AI Factory Site Development

    NVIDIA has taken a minority stake in Cloverleaf Infrastructure, a Houston-based developer of powered data-center sites, as part of a strategic partnership to accelerate the build-out of so-called AI factories. The companies did not disclose the size of the investment or its financial terms. Cloverleaf occupies the powered-land layer of the data-center supply chain. It identifies and prepares sites with grid-connected power, then delivers shovel-ready locations to infrastructure builders and ope

    AI
    2 min read
  13. Starcloud Raises $250M Series A Extension at $2.3B Valuation for Orbital AI Data Centers

    Orbital compute startup Starcloud has closed a $250 million Series A extension at a $2.3 billion post-money valuation, more than doubling its valuation from March 2026. The round brings the company's total funding to $450 million since its founding in 2024. The extension was led by Manhattan West, with partner Lauren Selig joining as a board observer. Participating investors include existing backers Benchmark, EQT, Soma, NFX, and 776, alongside new strategic investments from NVIDIA, Cisco Inves

    AI
    2 min read
  14. NVIDIA AVO Achieves 100% on ARC-AGI-3 with Claude Opus 5

    NVIDIA Research has published details on Agentic Variation Operators (AVO), a general-purpose autonomous agent architecture designed to sustain long-horizon engineering and reasoning workflows. In benchmark evaluations on ARC-AGI-3, an interactive reasoning suite that tests fluid intelligence without instructions or stated goals, AVO scored 100.00 Relative Human Action Efficiency (RHAE) across all 25 public-set environments and 183 levels using Anthropic's Claude Opus 5. The result highlights a

    AI
    3 min read
  15. Vision-Language Model Serving in Production: Visual Token Pruning, Encoder Caching, Dynamic Resolution, and Inference Economics

    Deploying Vision-Language Models (VLMs) into high-concurrency production environments introduces a distinct set of systems bottlenecks that text-only large language models do not exhibit. While text models ingest prompts with compact token densities, visual inputs require processing high-dimensional pixel arrays through vision encoders, expanding a single image into hundreds or thousands of visual tokens before autoregressive generation begins. In production architectures running models such as

    AI
    8 min read
  16. Duke Study Finds Creative Output Across Frontier LLMs Is Converging Toward Semantic Monoculture

    A longitudinal study by researchers at Duke University has documented a measurable decline in output diversity across major large language model families. Tracking 69 foundation model checkpoints released between March 2023 and July 2026, the researchers found that generative responses to open-ended creative and ideation prompts are increasingly converging toward semantic uniformity across competing providers. The paper, titled Are LLMs becoming similarly creative? Evidence from three years of

    AI
    2 min read
  17. Study Exposes Benchmark Overfitting and Acoustic Leakage in Speech Recognition Models

    A joint empirical study by researchers from Hugging Face and Hume AI has uncovered evidence that top-ranking automatic speech recognition (ASR) models frequently exhibit benchmark optimization, commonly referred to as "benchmaxxing." The findings demonstrate that models often achieve low word error rates on public leaderboards by memorizing test-set reference anomalies and responding to subtle acoustic channel signatures rather than generalizing audio transcription. Evaluating 11 open-source sp

    AI
    2 min read
  18. DeepSeek Releases DeepSeek-V4-Flash-Vision-Exp with Multimodal Tool Calling and Files API

    DeepSeek has expanded its flagship lightweight model with the release of DeepSeek-V4-Flash-Vision-Exp, an experimental multimodal variant that adds visual comprehension and document parsing to its API platform. The release bridges the gap between DeepSeek's high-throughput text architecture and vision-centric agent workflows. According to DeepSeek, the experimental model retains the text reasoning, agentic tool-use capabilities, and world knowledge of the base DeepSeek-V4-Flash checkpoint while

    AI
    2 min read
  19. Auxiliary-Loss-Free Load Balancing in Mixture-of-Experts: How Dynamic Bias Adjustments Eliminate Gradient Conflict and Routing Collapse

    Sparse Mixture-of-Experts (MoE) architectures decouple parameter count from per-token compute cost by activating only a small subset of feed-forward network (FFN) parameters for any given token. While dense transformers evaluate every parameter across all sequence positions, MoE models route tokens dynamically to specialized sub-networks, enabling parameter scaling to hundreds of billions or trillions of parameters at the inference and training cost of much smaller dense models. However, condit

    AI
    9 min read
  20. Confidential LLM Inference in Production: Hardware TEEs, GPU Enclaves, Attestation, and Serving Performance Trade-Offs

    Deploying large language models in multi-tenant cloud environments introduces a fundamental security boundary problem. Standard transport encryption (TLS) secures prompts in transit, and encryption-at-rest protects checkpoints on disk, but model weights, prompt tokens, and key-value (KV) caches exist in plaintext within system memory during active inference. For organizations processing protected health information (PHI), proprietary intellectual property, or regulated financial records, host op

    AI
    8 min read