llms.blog
Infrastructure13 articles

Infrastructure

Articles

  • Self-Hosted Embedding and Reranking Serving in Production: TEI vs. Infinity vs. vLLM Architecture, Dynamic Batching, and Serving Economics

    While generative large language models dominate inference infrastructure discussions, vector embeddings and cross-encoder rerankers handle order-of-magnitude higher request volumes in production retrieval-augmented generation (RAG) and search pipelines. Serving embedding and reranking models presents fundamentally different computational characteristics than auto-regressive text generation. Without auto-regressive token generation loops or key-value (KV) cache state management, the primary engin

    1 min

  • Modular Open-Sources Mojo Language Compiler and Toolchain Under Apache 2.0

    Modular Open-Sources Mojo Language Compiler and Toolchain Under Apache 2.0 Modular has released the complete source code for the Mojo programming language compiler, standard tooling, and runtime infrastructure under the Apache 2.0 license with LLVM exceptions. The announcement, delivered on August 18, 2026 during the company's ModCon developer conference, fulfills a multi-year roadmap commitment to transition the systems programming language to a fully open development model. The compiler sour

    1 min

  • LLM Fine-Tuning Frameworks in Production: Unsloth vs. Axolotl vs. LLaMA-Factory vs. Torchtune Architecture, Throughput, and Distributed Scaling

    Modern post-training pipelines have moved beyond basic training scripts. As model parameter counts, context windows, and alignment techniques expand, the choice of fine-tuning framework directly dictates GPU memory overhead, token throughput, and developer iteration speed. Four open-source frameworks dominate the enterprise fine-tuning landscape: Unsloth, Axolotl, LLaMA-Factory, and Meta's Torchtune. While all four orchestrate parameter-efficient fine-tuning (PEFT) and full parameter adaptation

    1 min

  • Mistral Expands Platform to Host Third-Party Open Weights Starting with GLM-5.2

    Mistral AI has broadened its API platform to host external open-weight foundation models, beginning with Zhipu AI's GLM-5.2. The move marks a strategic shift for the Paris-based AI company from serving only in-house architectures (such as Mistral Small, Mistral Medium, Mistral Large, and Voxtral) toward operating as a sovereign managed inference hub for third-party open weights. The integration introduces GLM-5.2 under the model identifier zai-glm-5-2 in public preview. The model is hosted with

    1 min

  • OpenAI Signs 20-Year Lease for Multi-Gigawatt Ohio AI Campus with Nvidia Backstop

    OpenAI has signed a 20-year lease agreement to anchor a multi-gigawatt AI data center campus in Pike County, Ohio, developed by SoftBank subsidiary SB Energy. Under the tripartite agreement, Nvidia will serve as the exclusive compute infrastructure provider and supply financial guarantees covering the site's land, power, and shell development. The facility, designated the PORTS-Pike Technology Campus, is designed to support an initial 4.25 gigawatts of information technology load (IT-GW) dedica

    1 min

  • Stripe Finalizes B+ Acquisition of AI Gateway OpenRouter

    Stripe has finalized an agreement to acquire AI model routing startup OpenRouter for more than $7 billion, according to reports from Bloomberg and TechCrunch. The deal marks one of the largest acquisitions in the artificial intelligence infrastructure sector to date and signals Stripe's intention to control the payment, metering, and routing layers of foundational AI services. OpenRouter, founded in 2023 by Alex Atallah, operates an API gateway that aggregates access to hundreds of large langua

    1 min

  • Nvidia and Amazon channel billions into AI power infrastructure

    Nvidia and Amazon are putting serious money into the power generation needed to keep AI data centers running, with two separate deals announced this week that underscore how energy has become a bottleneck for the industry. Nvidia is investing up to $3 billion in Lancium, the power infrastructure developer behind the OpenAI and Oracle data center in Texas, according to The Information. A roughly $2 billion stake would give Nvidia about 20 percent of the company, which is valued at around $10 bil

    1 min

  • Amazon Data Center Could Be Powered by One of the Nation's Most Polluting Power Plants

    Amazon is investing in a new natural-gas power plant in Pecos County, Texas, to supply a West Texas data center, and the project holds a permit that would allow it to emit more carbon dioxide than any coal plant in the country, according to The Verge and the New York Times. The plant, tracked as GW Ranch by Cleanview, a service that monitors data center power projects, would deploy 35 natural-gas turbines generating about 7.65 gigawatts. At least initially, the plant would not connect to

    1 min

  • Google Offloads $35 Billion in Anthropic Chip Risk via SPV Structure

    Google has assembled one of the largest infrastructure financing programs in history to supply Anthropic with its proprietary AI chips while keeping the hardware off everyone's balance sheets, according to Financial Times reporting. The arrangement involves Google, Broadcom, Morgan Stanley, Apollo, Blackstone, and several cryptocurrency mining companies in a chain of deals designed to sidestep the balance-sheet constraints that would otherwise make the transaction impossible. How the st

    1 min

  • NSF Commits $100 Million to Regional AI Infrastructure Hubs

    The U.S. National Science Foundation launched a $100 million program on August 4 to build state and regional AI infrastructure hubs, with NVIDIA, AMD, Intel, and Dell Technologies among the private-sector partners. The NSF State and Regional Artificial Intelligence Infrastructure Hubs program will fund up to 10 regional consortia that pool compute, data, and expertise for researchers, students, and educators who currently lack access to AI-scale computing resources. Public-private cost

    1 min

  • Google Brings AI Model Routing to API Gateway

    Google Cloud has added managed AI model routing to API Gateway, giving developers one OpenAI-compatible endpoint for models hosted through Vertex AI Model Garden. The feature is now in public preview and can direct requests to Gemini, Anthropic Claude, and OpenAI GPT-family models without requiring teams to operate a separate proxy service, according to Google's announcement. Routing moves into the gateway Developers define virtual model names and backend targets inside an OpenAPI 3.x s

    1 min

  • Anthropic Signs $10B Compute Deal With Cloud Startup That Didn't Exist Six Months Ago

    Anthropic has locked in $10 billion worth of computing capacity from Volta Infra Holdings, a cloud infrastructure startup founded in early 2026, in a six-year deal that underscores the AI industry's increasingly aggressive scramble for compute. The deal, reported by Bloomberg, will draw on a data center in Tydal, Norway, where Bitcoin miner Bitdeer Technologies operates hydropower-fed infrastructure packed with Nvidia's latest Vera Rubin chips. Volta describes the capacity at 133 megawatt

    1 min

  • Claude suffers global outage as network failures hit all models

    Anthropic's Claude platform experienced a widespread service disruption on July 29, 2026, with elevated error rates and latency affecting every model across its entire product suite. The incident knocked out claude.ai, the Claude API, Claude Code, and Claude Cowork for nearly three hours. The outage began at 19:49 UTC when Anthropic's status page first flagged elevated errors across multiple models. Users immediately reported 529 "Overloaded" errors, a non-standard HTTP response indicating the

    1 min