Subscribe
Category / page 12

Chips & Infrastructure

314 articles

Articles, page 12

221-240 of 314

  1. Self-Consistency Decoding: How Sampling Diverse Reasoning Paths Eliminates Single-Trajectory Fragility in LLMs

    Self-Consistency Decoding: How Sampling Diverse Reasoning Paths Eliminates Single-Trajectory Fragility in LLMs Standard inference pipelines in large language models rely on greedy decoding or low-temperature sampling to generate answers. For tasks requiring direct factual recall or basic classification, greedy token selection generates fluent and accurate outputs. However, when large models are tasked with complex multi-step reasoning, greedy generation exhibits severe structural fragility. In

    AI
    6 min read

  2. Data Ingestion and Incremental Sync for Production RAG: CDC Streams, Content Hashing, Backpressure, and Zero-Downtime Indexing

    Maintaining retrieval-augmented generation (RAG) systems in production introduces a fundamental distributed systems challenge that rarely surfaces in proof-of-concept architectures: state synchronization. While initial ingestion across a static document corpus is straightforward, production data sources (PostgreSQL databases, transactional stores, object storage, and enterprise knowledge hubs) undergo continuous mutation. Records are inserted, updated, soft-deleted, and reassigned new access per

    RAG
    8 min read

  3. Unsloth Releases Dynamic V3.0 GGUFs for Qwen 3.8 27B with 1-Bit Mode and MTP

    Unsloth AI has published its Dynamic V3.0 quantization suite for Alibaba's Qwen 3.8 27B model family, releasing optimized GGUF and NVFP4 checkpoints alongside public calibration matrices. The release claims a greater than 10 percent increase in top-1 percent accuracy at identical file sizes compared to standard baseline quantizations, while introducing an ultra-low-bit dynamic tier that operates within 8GB of memory. Qwen 3.8 27B is a dense vision-language model utilizing hybrid attention layer

    Open Weights
    2 min read

  4. State Data Farms Account for 20% of China's Humanoid Robot Shipments

    Government-backed data collection hubs and municipal training centers accounted for roughly 20 percent of China's more than 20,000 humanoid robot shipments last year, according to an analysis from Bernstein. The purchases reflect a national strategy to overcome the primary bottleneck in physical artificial intelligence: the scarcity of high-fidelity physical interaction data needed to train embodied foundation models. Unlike large language models that train on vast public text corpora scraped f

    Robotics
    2 min read

  5. Skanska Signs $1.2B Contract to Build Four AI Data Centers in Southeast USA

    Swedish construction and development group Skanska has signed a $1.2 billion (SEK 11.2 billion) contract with an existing client to construct four new data center facilities in the southeastern United States. The full contract value will be included in Skanska's US order bookings for the third quarter of 2026, representing the largest single data center award in the contractor's history. Scope of Work and Campus Specifications The multi-facility project spans four standalone structures total

    AI
    2 min read

  6. UK Heterogeneous Compute Startup Callosum Raises $100M Seed Backed by Atomico and Sovereign AI Fund

    London-based AI infrastructure startup Callosum has raised $100 million in seed financing to develop systems software that orchestrates machine learning workloads across heterogeneous processor environments. The funding round was led by European venture capital firm Atomico, with participation from Plural and DCVC, alongside a major investment from the UK government's £500 million ($677 million) Sovereign AI Fund. Callosum did not disclose its post-money valuation. Breaking Homogeneous Comput

    AI
    2 min read

  7. Sliding Window Attention in Large Language Models: How Bounded Receptive Fields, Interleaved Layers, and Rolling KV Buffers Scale Contexts

    Standard causal multi-head attention imposes two severe computational constraints as sequence lengths expand into tens or hundreds of thousands of tokens. First, calculating pairwise query-key dot products scales quadratically with sequence length, requiring $O(N^2)$ floating-point operations. Second, autoregressive generation requires caching key and value projections for all preceding tokens, causing the key-value (KV) cache to grow linearly with sequence length $O(N)$ across all layers and at

    AI
    8 min read

  8. Custom LLM Kernel Optimization in Production: Triton vs. CUDA C++ vs. torch.compile vs. CUTLASS

    Serving large language models at scale requires extracting maximum performance from modern GPU architectures like NVIDIA Ampere, Hopper, and Blackwell. While early production deployments relied on standard PyTorch eager execution and standard cuBLAS calls, high-throughput serving systems such as vLLM, SGLang, and TensorRT-LLM depend on specialized fused GPU kernels to eliminate memory bandwidth bottlenecks and saturate Tensor Cores. Engineering teams face four primary paradigms for kernel devel

    AI
    7 min read

  9. GPU Cluster Networking in Production: InfiniBand vs. RoCEv2 vs. Ultra Ethernet Architecture, Congestion Control, and NCCL Collective Latency

    Distributed training and high-throughput inference workloads are fundamentally bound by the network fabric. While traditional cloud applications rely on asynchronous request-response cycles that absorb latency jitter, distributed deep learning relies on synchronous collective communication. Operations such as All-Reduce, All-Gather, and All-to-All require hundreds or thousands of GPUs to exchange tensors and synchronize at strict barrier points before execution can proceed. In this execution mo

    AI
    8 min read

  10. Rotational Quantization: How Orthogonal Transforms and Hadamard Incoherence Enable Outlier-Free Low-Bit Inference

    Large language model serving is governed by two physical bottlenecks: memory bandwidth during autoregressive decoding and compute throughput during prompt prefill. While weight-only post-training quantization (such as GPTQ or AWQ) reduces parameter footprint to 4 bits, it leaves activations in 16-bit floating-point formats. As a result, inference engines cannot utilize high-throughput INT4 tensor cores, and the key-value (KV) cache continues to consume massive memory pools. Attempting to quanti

    AI
    8 min read

  11. Multi-LoRA Serving in Production: Architecture, Dynamic Adapter Swapping, and GPU Memory Management

    Deploying hundreds or thousands of fine-tuned language models across enterprise workflows presents a fundamental infrastructure dilemma. While parameter-efficient fine-tuning (PEFT) methods like Low-Rank Adaptation (LoRA) reduce training compute by freezing base model weights and training compact low-rank matrices, naive deployment strategies fail at scale. Merging adapter weights directly into the base model creates a distinct 7B to 70B parameter checkpoint for every specialized task, requiring

    AI
    8 min read

  12. Contrastive Decoding in Large Language Models: How Comparing Expert and Amateur Logits Suppresses Hallucination and Reasoning Errors

    Contrastive Decoding in Large Language Models: How Comparing Expert and Amateur Logits Suppresses Hallucination and Reasoning Errors Autoregressive large language models operate by predicting the conditional probability distribution of the next token given a sequence of preceding tokens. However, translating these continuous probability vectors into coherent, factual, and logically sound sequences remains one of the fundamental challenges of modern natural language processing. Traditional deco

    AI
    6 min read

  13. WhiteFiber Proposes $250M Convertible Debt Offering to Expand AI Data Center Capacity

    AI infrastructure provider WhiteFiber announced a proposed private placement of $250 million in convertible senior notes due 2032, with an option for initial purchasers to acquire up to an additional $37.5 million in notes. The proceeds are designated to fund data center campus acquisitions, facility buildouts, utility interconnection agreements, and hardware procurement for the company's AI cloud business. The financing coincides with WhiteFiber's agreement to acquire two industrial sites in Y

    AI
    2 min read

  14. Hyve Solutions Selects Nevada for Dual AI Server Manufacturing Campuses

    Hyve Solutions, the rack-scale server design and manufacturing subsidiary of TD SYNNEX, announced plans to construct two advanced manufacturing facilities in Nevada to expand domestic production of compute, storage, and networking systems for AI data centers. The development encompasses a 624,000-square-foot flagship campus in Reno alongside a secondary facility in North Las Vegas, with the combined projects projected to create approximately 3,000 jobs. The project follows formal approval of ta

    AI
    2 min read

  15. Nvidia Discusses Investment in AI Data Supplier Mercor at $20B Valuation

    Nvidia is in discussions to participate in a funding round for AI training data marketplace Mercor that would value the three-year-old startup at $20 billion, according to reporting from The Information and Bloomberg. The transaction would double Mercor's valuation from its $10 billion Series C round closed in October 2025. It also signals an expanding capital allocation strategy from Nvidia, moving beyond compute infrastructure and cloud hardware into the upstream data curation layers powering

    Nvidia
    2 min read

  16. Stripe Tells Investors Singularity Began Jan. 1 as H1 Revenue Surges 41% and Firm Rules Out IPO

    In a mid-year letter to shareholders, payments infrastructure company Stripe declared that January 1, 2026 marked the "beginning of the singularity," framing rapid advancements in artificial intelligence and corporate formation as justification to remain private. The letter, obtained by Axios, links Stripe's long-term business strategy directly to AI compute economics and autonomous agent adoption, while reporting accelerated financial growth across its core payment platforms. Financial Metri

    AI
    2 min read

  17. AI Chip Startup Fractile Seeks $6.5B Valuation on $600M Round After Anthropic Supply Deal

    London-based semiconductor startup Fractile is in advanced negotiations to raise approximately $600 million at a $6.5 billion pre-money valuation, according to reports from Bloomberg. The funding round represents a steep escalation in valuation from the startup's previous round in May 2026, which valued the company at roughly $1 billion. The capital raise follows an initial supply agreement with Anthropic valued at approximately $250 million. Under the commercial arrangement, Fractile will supp

    AI
    2 min read

  18. Ramp Launches Router.com Model Gateway with Free Routing Through 2026

    Corporate spend management company Ramp has launched Router.com, a unified AI gateway designed to dynamically direct model inference requests to the lowest-cost model that satisfies a developer's performance requirements. The service, unveiled on August 19, 2026, enters general availability with zero routing fees through 2026, charging customers only standard list prices for consumed tokens and offering $26 in starting credits. Ramp, which oversees more than $200 billion in annual transaction v

    AI
    2 min read

  19. Benchmarking LLM Inference in Production: Architecture, Metrics, and Tooling Across AIPerf, GuideLLM, and LLMPerf

    Traditional HTTP load-testing tools such as Apache Bench, wrk, and Locust evaluate systems using uniform request-response cycles. These tools send a payload, wait for the full response, and compute metrics such as requests per second (RPS) and round-trip latency percentiles. For stateless REST APIs, this model aligns directly with user experience. Large language model (LLM) serving fundamentally breaks this abstraction. An inference request is not an atomic computation. It consists of two disti

    AI
    7 min read

  20. Grok 4.6 Launches on Amazon Bedrock with 500K Context and Cross-Region Routing

    xAI's flagship reasoning model, Grok 4.6, is now generally available across Amazon Web Services through Amazon Bedrock. Released on August 19, 2026 under the model ID xai.grok-4.6, the deployment gives enterprise AWS customers managed API access to xAI's frontier model alongside existing foundational offerings from Anthropic, Meta, and Mistral. The integration comes one week after xAI initially launched Grok 4.6 on August 12, marking a significantly faster enterprise cloud deployment than its p

    AI
    2 min read