Subscribe
Category / page 40

Models & Assistants

1121 articles

Articles, page 40

781-800 of 1121

  1. Google DeepMind Deploys Backstory to Fact-Checkers for Multi-Agent AI Image Verification

    Google DeepMind has expanded live testing of Backstory, an experimental verification platform designed to investigate the origin, manipulation, and dissemination history of digital images. The system, built on the Gemini model family, is currently deployed across newsrooms, open-source intelligence (OSINT) groups, academic researchers, and fact-checking teams participating in Google's Trusted Testers program. Beyond Binary Synthetic Detection Traditional automated image forensic tools typical

    AI
    2 min read

  2. Sliding Window Attention in Large Language Models: How Bounded Receptive Fields, Interleaved Layers, and Rolling KV Buffers Scale Contexts

    Standard causal multi-head attention imposes two severe computational constraints as sequence lengths expand into tens or hundreds of thousands of tokens. First, calculating pairwise query-key dot products scales quadratically with sequence length, requiring $O(N^2)$ floating-point operations. Second, autoregressive generation requires caching key and value projections for all preceding tokens, causing the key-value (KV) cache to grow linearly with sequence length $O(N)$ across all layers and at

    AI
    8 min read

  3. Custom LLM Kernel Optimization in Production: Triton vs. CUDA C++ vs. torch.compile vs. CUTLASS

    Serving large language models at scale requires extracting maximum performance from modern GPU architectures like NVIDIA Ampere, Hopper, and Blackwell. While early production deployments relied on standard PyTorch eager execution and standard cuBLAS calls, high-throughput serving systems such as vLLM, SGLang, and TensorRT-LLM depend on specialized fused GPU kernels to eliminate memory bandwidth bottlenecks and saturate Tensor Cores. Engineering teams face four primary paradigms for kernel devel

    AI
    7 min read

  4. Self-Play Fine-Tuning (SPIN): How Iterative Games Against Historical Checkpoints Eliminate Preference Labels

    Self-Play Fine-Tuning (SPIN): How Iterative Games Against Historical Checkpoints Eliminate Preference Labels Supervised fine-tuning (SFT) remains the standard entry point for adapting foundation language models to downstream instructions. By training on curated prompt-response pairs via token-level cross-entropy loss, SFT forces a model to mirror reference demonstrations. However, standard maximum likelihood estimation (MLE) suffers from exposure bias and covariate shift. During inference, the

    AI
    8 min read

  5. DeepSeek Open-Sources DeepSeek Harness: Modular Agent Runtime Built on Cordis Plugin Framework

    DeepSeek AI has released DeepSeek Harness (dsh), an open-source agent harness built on a modular "everything is a plugin" design. Published under the MIT license on GitHub, the project provides a runtime framework where all core agent functions—including model adapters, session storage, tool registries, sandboxes, and the execution loop itself—are implemented as interchangeable, hot-swappable plugins. Rather than developing a monolithic agent runner, DeepSeek structured the system around Cordis

    AI
    2 min read

  6. DeepReinforce Releases Ornith-1.5: MIT-Licensed 9B, 35B, and 397B Open Weights Trained on Autonomous Self-Improvement Loops

    DeepReinforce and the Ornith team have released Ornith-1.5, a family of open-weight foundation models trained using autonomous self-improvement loops. Released under the permissive MIT license, the model family spans three architectures: a 9B dense model, a 35B mixture-of-experts model activating approximately 3B parameters per token (35B-A3B), and a flagship 397B mixture-of-experts model. The release expands on the team's earlier Ornith-1.0 release by automating the entire post-training optimi

    AI
    2 min read

  7. GPU Cluster Networking in Production: InfiniBand vs. RoCEv2 vs. Ultra Ethernet Architecture, Congestion Control, and NCCL Collective Latency

    Distributed training and high-throughput inference workloads are fundamentally bound by the network fabric. While traditional cloud applications rely on asynchronous request-response cycles that absorb latency jitter, distributed deep learning relies on synchronous collective communication. Operations such as All-Reduce, All-Gather, and All-to-All require hundreds or thousands of GPUs to exchange tensors and synchronize at strict barrier points before execution can proceed. In this execution mo

    AI
    8 min read

  8. Rejection Sampling Fine-Tuning: How Filtering Model Outputs by Reward Replaced RLHF Complexity

    Rejection Sampling Fine-Tuning: How Filtering Model Outputs by Reward Replaced RLHF Complexity Rejection sampling fine-tuning (RAFT) has emerged as the practical workhorse of LLM alignment. While PPO-based RLHF dominated early literature, production systems from Llama 2 to DeepSeek-R1 rely on a simpler loop: generate multiple completions per prompt, score them with a reward model, keep the best, and fine-tune on the filtered data. The technique converts the reinforcement learning problem into s

    Education
    4 min read

  9. OKX and Goldman Sachs Cut Off Hong Kong Staff From Anthropic's Claude AI

    Crypto exchange OKX has restricted access to Anthropic's Claude model for employees based in Hong Kong or traveling through China after its corporate account was briefly suspended, according to Bloomberg. The restriction comes despite OKX making AI proficiency part of employee performance evaluations and encouraging workers to integrate AI tools into daily workflows. OKX stopped providing Claude access to staff in the region in early August following the suspension, Bloomberg reported. Internal

    Policy
    2 min read

  10. Vector Indexing in Production: HNSW vs. DiskANN vs. IVF-PQ Architecture, Memory Footprint, and Search Economics

    Vector Indexing in Production: HNSW vs. DiskANN vs. IVF-PQ Architecture, Memory Footprint, and Search Economics Scaling vector search beyond prototype deployments exposes a fundamental tension across three competing constraints: retrieval recall, query latency, and memory footprint. In high-dimensional representation spaces, exact k-nearest neighbor search via brute-force flat scans requires $O(N \cdot d)$ floating-point operations per query. For a corpus of 100 million 1536-dimensional FP32 em

    AI
    7 min read

  11. Rotational Quantization: How Orthogonal Transforms and Hadamard Incoherence Enable Outlier-Free Low-Bit Inference

    Large language model serving is governed by two physical bottlenecks: memory bandwidth during autoregressive decoding and compute throughput during prompt prefill. While weight-only post-training quantization (such as GPTQ or AWQ) reduces parameter footprint to 4 bits, it leaves activations in 16-bit floating-point formats. As a result, inference engines cannot utilize high-throughput INT4 tensor cores, and the key-value (KV) cache continues to consume massive memory pools. Attempting to quanti

    AI
    8 min read

  12. Anthropic Eyes $10B+ Credit Line Ahead of Planned Public Listing

    Anthropic is working to expand its revolving credit facility beyond an initial $10 billion target as it prepares for a planned initial public offering, according to reporting from Bloomberg. Wall Street investment banks are actively competing for lending allocations in the facility to improve their positioning for underwriting mandates on the eventual share sale. Under the framework currently under discussion, Anthropic has asked lead banks to commit approximately $1.25 billion each. Secondary

    Anthropic
    1 min read

  13. Multi-LoRA Serving in Production: Architecture, Dynamic Adapter Swapping, and GPU Memory Management

    Deploying hundreds or thousands of fine-tuned language models across enterprise workflows presents a fundamental infrastructure dilemma. While parameter-efficient fine-tuning (PEFT) methods like Low-Rank Adaptation (LoRA) reduce training compute by freezing base model weights and training compact low-rank matrices, naive deployment strategies fail at scale. Merging adapter weights directly into the base model creates a distinct 7B to 70B parameter checkpoint for every specialized task, requiring

    AI
    8 min read

  14. Weight-Decomposed Low-Rank Adaptation (DoRA): How Decoupling Magnitude and Direction Closes the LoRA Gap

    Weight-Decomposed Low-Rank Adaptation (DoRA): How Decoupling Magnitude and Direction Closes the LoRA Gap Parameter-efficient fine-tuning (PEFT) has become the standard operational paradigm for adapting large language models to domain-specific downstream tasks. Among existing PEFT methodologies, Low-Rank Adaptation (LoRA) remains the default implementation across industry and academia due to its minimal parameter footprint and zero inference overhead. However, empirical studies consistently reve

    AI
    8 min read

  15. Attention with Linear Biases (ALiBi): How Static Positional Slopes Enable Zero-Shot Context Extrapolation

    Large language models process sequences by transforming discrete tokens into continuous vector representations. Standard dot-product self-attention is permutation-invariant: without explicit positional information, the attention operation treats a sequence as an unordered bag of tokens. Early transformer architectures addressed this limitation using Absolute Positional Embeddings (APE), either through fixed sinusoidal functions or learned lookup tables added directly to token embeddings. While

    AI
    6 min read

  16. Vals AI Raises $40M Series A at $400M Valuation Led by a16z to Build Real-World AI Benchmarks

    San Francisco evaluation startup Vals AI announced a $40 million Series A funding round at a $400 million post-money valuation, led by Andreessen Horowitz. The round included participation from existing seed backers 8VC, Pear VC, and Bloomberg Beta, alongside new institutional investors HRT Ventures and Next Ladder Ventures. The financing brings total capital raised by the company to $45 million, following a $5 million seed round. Founded by Stanford computer science graduates Rayan Krishnan a

    AI
    2 min read

  17. Small Language Models in Production: Task Specialization, Serving Economics, and the Frontier Offloading Pattern

    The default architecture for first-generation enterprise AI agents routed every prompt, tool selection, and intermediate evaluation step to a single frontier large language model. While this monolithic approach simplified initial orchestration, it introduced severe latency bottlenecks and unsustainable inference unit economics in high-throughput production environments. In production agentic loops, between 40% and 70% of model invocations are narrow, highly structured operations: classifying in

    AI
    6 min read

  18. Contrastive Decoding in Large Language Models: How Comparing Expert and Amateur Logits Suppresses Hallucination and Reasoning Errors

    Contrastive Decoding in Large Language Models: How Comparing Expert and Amateur Logits Suppresses Hallucination and Reasoning Errors Autoregressive large language models operate by predicting the conditional probability distribution of the next token given a sequence of preceding tokens. However, translating these continuous probability vectors into coherent, factual, and logically sound sequences remains one of the fundamental challenges of modern natural language processing. Traditional deco

    AI
    6 min read

  19. TrueFoundry Open-Sources TrueForge Agent Harness to Cut Managed Inference Spend

    Enterprise ML infrastructure platform TrueFoundry has open-sourced its custom agent runtime, TrueForge, under the permissive MIT license. The framework is designed to orchestrate autonomous reasoning loops, tool interactions, and long-running execution across vendor-neutral LLM backends while systematically reducing token overhead and operational compute costs. According to benchmark data published by TrueFoundry, pairing TrueForge with the open-weight GLM-5.2 model completed 11 of 14 enterpris

    AI
    2 min read

  20. Reinforcement Learning with Verifiable Rewards: How Programmatic Oracles Eliminate Reward Hacking in LLM Reasoning

    Post-training paradigms for large language models have undergone a fundamental architectural shift. While the initial wave of alignment relied on Reinforcement Learning from Human Feedback (RLHF) and direct preference optimization (DPO), frontier reasoning systems increasingly depend on Reinforcement Learning with Verifiable Rewards (RLVR). Traditional RLHF relies on neural reward models trained on human pairwise comparisons. These neural proxies suffer from reward overoptimization, vulnerabili

    AI
    6 min read