Google has made Pathways, the distributed training system it built for its own large models including Gemini, generally available on Google Cloud. The company announced general availability on Nov 9, 2026, in a Google Developers Blog post titled "What is Pathways? Single-controller training for large models," the first entry in a series called "Pathways on Google Cloud." Most of the post is an explanation of how the system works and how it differs from the setup that most large-model training uses today.
What Google announced
Pathways is now generally available on Google Cloud. It runs JAX programs on TPUs, Google's AI accelerators (for background, see What is a TPU?). Google describes it as the system behind its own large-model training, including Gemini.
The underlying design was described in a 2022 research paper, "Pathways: Asynchronous Distributed Dataflow for ML", and Google now publishes product documentation for the Cloud version. The announcement also points to two open-source repositories: pathways-utils and MaxText.
Why lockstep training breaks down
Most large-model training today runs as SPMD, or Single Program, Multiple Data. Every accelerator runs the same program in lockstep and coordinates through collective operations such as AllReduce. PyTorch's DDP and FSDP follow this model. It is a good fit when the work is uniform: the same operation, repeated across every chip, at every step.
Google argues SPMD struggles when the job is not uniform. Two examples in the post: training pipelines whose stages wait on each other, and reinforcement learning post-training loops where a sampler, a reward model and a trainer are three different programs that should overlap rather than take turns.
Failures are the other problem. Google's post says hardware failures interrupted Gemini 2.5 training multiple times per hour, citing the Gemini 2.5 technical report. In an SPMD job, one lost machine stops every process and sends the whole run back to its last checkpoint.
The post also notes that bandwidth inside a training job is lopsided. On TPU v5p, each chip gets 1,200 GB/s of bidirectional chip-to-chip bandwidth inside its group but only 50 Gb/s (about 6 GB/s) to the datacenter network, roughly two orders of magnitude less.
How Pathways works
Instead of many copies of one program, Pathways uses one controller. A single Python client, running on an ordinary CPU machine, describes the whole computation as a graph of blocks and the tensors that flow between them. The code targets virtual devices. A resource manager maps those virtual devices onto real chips, and a worker on each accelerator host runs the compiled programs.
If a host dies, the resource manager remaps the work onto healthy chips and the run keeps going, instead of relaunching the whole job. Google says this is the basis for elastic training. Because the program targets virtual devices rather than physical ones, compilation can also start before physical chips are provisioned.
On the Cloud side, a Shared Pathways Service keeps a cluster up as a long-running service on GKE. Researchers connect from a laptop or a VM, get their own slices, and hand them back when they disconnect. When every chip is busy, the next workload waits.
The runtime dispatches work asynchronously using futures, so it can prepare the next block's buffers and transfers before the previous block finishes. A central scheduler puts operations into one consistent order, which Google says avoids deadlocks in pipelined training. Debugging follows the same single-controller shape: developers can drop into a shell, set a breakpoint and inspect a run from the one client, whether the job spans one host or thousands.
Pathways and Ray
The post addresses Ray directly, noting that Pathways can sound like an orchestrator developers already know. Google says Ray handles orchestration: scheduling Python tasks and actors, plus data, training, serving and tuning. The job running inside each Ray worker still executes as SPMD. Pathways changes the execution model underneath that layer.
Google calls the two complementary rather than competing, writing: "So concretely, Ray's job is about orchestration while Pathways' job is about execution."
Why it matters
The practical change is access. The system Google built to train its own large models, including Gemini, is now generally available to Google Cloud customers who run JAX on TPUs. The announcement came the same day Google detailed its new TPU 8tx and 8i Max systems.
How much that matters depends on the workload. Google's own post says SPMD works well when a job is the same program running on every device. The benefits described in the post concentrate on jobs that are non-uniform, jobs that mix several different programs, and long runs that meet hardware failures often enough for restart costs to add up.
What Google did not say
The post gives no pricing for Pathways on Google Cloud, and it offers no benchmark comparing Pathways to SPMD training speed. It also does not quantify how much time remapping onto healthy chips saves compared with restarting from a checkpoint.
Sources
- What is Pathways? Single-controller training for large models (Google Developers Blog, Nov 9, 2026)
- Pathways on Cloud introduction (Google Cloud documentation)
- Pathways: Asynchronous Distributed Dataflow for ML (arXiv, 2022)
- Gemini 2.5 technical report (arXiv)
- pathways-utils and MaxText (GitHub, AI-Hypercomputer)
