At Hot Chips 2026, NVIDIA announced that the Groq 3 LPX dedicated inference accelerator has entered full commercial production. Built as a workload-optimized extension to the Vera Rubin data center architecture, the accelerator addresses the decode bottlenecks inherent in multi-step agentic AI workloads by offloading token generation from primary GPU clusters onto specialized Language Processing Units (LPUs).
In independent benchmarking conducted by Artificial Analysis, the Groq 3 LPX delivered 3,400 output tokens per second on the open-source Gemma 4 31B model with a 100,000-token input context. According to NVIDIA, this marks the fastest token generation speed recorded for the model, providing roughly four times the interactive responsiveness of alternative hardware platforms on latency-sensitive agentic tasks.
Disaggregating Prefill and Decode in Agentic Architectures
Modern agentic loops require models to repeatedly inspect file trees, evaluate tool outputs, verify reasoning traces, and execute code in iterative sequences. While rack-scale GPU systems such as the Vera Rubin NVL72 efficiently ingest large context windows during the prefill phase, sequential token generation remains memory-bandwidth bound.
Groq 3 LPX decouples this execution path. In a full rack deployment, systems can integrate up to 256 LP30 accelerators linked across high-bandwidth interconnects. While Vera Rubin GPUs handle large-scale context processing and multimodal ingestion, the LPX accelerators process the sequential autoregressive decode steps, maintaining interactive token output rates even across long context windows.
Neocloud and Hardware Deployments
Nebius Group N.V. has signed on as the launch cloud partner, integrating Groq 3 LPX into its Nebius Token Factory production inference platform. Nebius CTO Danila Shtan noted that the integration exposes the hardware through standard inference APIs without requiring customer stack migrations.
Purpose-built inference cloud provider Groq Inc., whose technology NVIDIA licensed in December for $20 billion alongside bringing on founder Jonathan Ross, is also among the early deployment partners. In parallel announcements at Hot Chips, NVIDIA disclosed that SpaceX will adopt Vera CPUs for terrestrial and orbital orchestration workloads, while previewing Spectrum-X Multiplane networking fabrics designed to support clusters of up to 512,000 GPUs.
