Every large language model you have talked to is a decoder. So was the last one you paid an API bill for, and the last open-weights model somebody on your team tried to run on a single GPU. That was not inevitable. For about two years, the best evidence in the field pointed the other way.
This post is about the three designs a transformer can take, why the field converged on one of them, and what actually happened to the others.
The original machine had both halves
The transformer described in Attention Is All You Need (Vaswani et al., 2017) was not a chatbot architecture. It was a machine translation system with two stacks of layers. An encoder read the source sentence and built a representation of it, with every token able to see every other token in the sentence. A decoder generated the translation one token at a time, consulting the encoder's representation through cross-attention and consulting its own output so far through masked, one-directional attention.
Three usable architectures come out of that design: use both stacks, use only the encoder, or use only the decoder. The NLP field spent 2018 arguing about which one mattered.
The 2018 fork: BERT and GPT
Google's BERT (Devlin et al., 2019) kept only the encoder stack. Its training objective was masked language modeling: hide roughly 15% of the words in a sentence and make the model restore them from context. Because each token sees both its left and its right neighbors, BERT reads like a system that genuinely understands a whole sentence at once. When it was published it posted a new state of the art on eleven language understanding tasks, and for two years almost every serious NLP pipeline had a BERT variant in it.
OpenAI's GPT (Radford et al., 2018) kept only the decoder stack. Its objective was simpler and older: predict the next token, given the tokens before it. A causal mask means no token ever sees the future. That restriction makes the model a generator by construction and a sentence-understanding model by accident.
In 2019, the encoder was winning. GPT-1 was a respectable research result; BERT was the event.

What each design buys you, and what it costs
Bidirectional attention is the better reading instrument. Every token's representation is informed by the full text, so encoder models produce strong embeddings and strong classifications in a single forward pass. What they cannot do is generate: as Sebastian Raschka puts it, encoder-only models do not decode in an autoregressive fashion, so they do not produce coherent output sequences. They need a task-specific head bolted on top, and a fine-tuning run per task.
Causal attention is the worse reading instrument and the only one that writes. Because a decoder is trained to continue any text, any task you can describe in words becomes a text-continuation problem. That single property is where the modern LLM comes from.
Why the decoder won
Three reasons, in rough order of how much they actually decided things.
1. The task became the prompt. GPT-3 (Brown et al., 2020) showed that a large enough decoder could adapt to new tasks from examples placed in the context window, with no weight updates. BERT-style models needed a supervised fine-tuning run and a task head for every task. Yi Tay, a researcher who worked on both camps at Google, argues the deprecation of BERT was mostly a story of unification: teams wanted one general-purpose model for everything, and the autoregressive format made that possible while the classification-head format did not.
2. Better compute economics. A causal language model computes a loss on nearly every token in every training document. A masked model supervises only the masked fraction. Tay calls this the loss-exposure problem: denoising objectives are simply less efficient per FLOP. The encoder-decoder design has a related cost. As noted in the T5 paper (Raffel et al., 2020) and repeated by Tay, an encoder-decoder with roughly 2N parameters costs about the same compute as a decoder-only model with N parameters, so it cannot spend its budget at the same per-layer depth.
3. One loop to serve. A decoder-only model is a single autoregressive stack, and the entire serving ecosystem, from KV caches to continuous batching, standardized on that one shape. Cameron Wolfe's survey of the field makes the blunt observation that the architecture of most modern LLMs is nearly identical to the original GPT, just scaled and post-trained.
The winner's record is not as clean as the lore
The shift to decoder-only happened fast and, as a 2025 comparative study noted, "without rigorous comparative analysis." The authors pretrained encoder-decoder and decoder-only models from 150M to 8B parameters under identical recipes and found the encoder-decoder variant, trained with prefix language modeling, scales comparably, and after instruction tuning matches or beats the decoder-only baseline on a range of downstream tasks while being more efficient at inference. Decoder-only pretraining was still more compute-optimal overall, which is the honest version of the story: the decoder had real advantages, and the field's convergence was also path dependence. Once labs started building everything around one loop, alternatives got less investment.

Where the encoder went
Nowhere important, it turned out. Three lineages survive.
- Embedding and retrieval models. Production search indexes still run on BERT-family encoders, because the embedding task is bidirectional by nature. A 2025 study on text embeddings argues the one-directional attention of autoregressive models is a fundamental mismatch for embedding, even as LLM-based embedding models catch up by brute-force scale.
- Classification at production scale. For sentiment, routing, moderation and similar tasks, a small encoder still usually beats a decoder of the same size, which is why they remain the workhorses of cheap text pipelines. Tay's summary: bidirectional attention helps a lot at small scale and is mostly optional at large scale.
- The hybrids. Prefix language modeling, where the prompt tokens see each other bidirectionally and only the generated text stays causal, sits between the two camps. Tay describes prefix LMs as "non-causal decoders" that keep BERT's bidirectional advantage without giving up generation, and the UL2 line of work folded denoising objectives into causal training as supporting data transformations rather than treating them as a rival philosophy. Fill-in-the-middle training in code models is the same idea expressed as a data format.
Takeaways
- A transformer can be an encoder, a decoder, or both. GPT chose the decoder stack; BERT chose the encoder stack.
- Encoders read better; only decoders write. Generation turned out to subsume understanding as an interface.
- Decoder-only won on loss exposure per FLOP, task unification through prompting, and a unified serving stack.
- The convergence was partly path dependence: controlled 2025 experiments show encoder-decoder models scale comparably when trained with the same modern recipes.
- Encoders never died. They are the embedding model inside your RAG pipeline and the classifier routing your tickets.
Sources
- Vaswani et al., 2017, Attention Is All You Need
- Devlin et al., 2019, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- Radford et al., 2018, Improving Language Understanding by Generative Pre-Training
- Brown et al., 2020, Language Models are Few-Shot Learners
- Raffel et al., 2020, Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Encoder-Decoder or Decoder-Only? Revisiting Encoder-Decoder Large Language Model, 2025
- Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective, 2025
- Yi Tay, What happened to BERT & T5?
- Sebastian Raschka, Encoder and Decoder Transformers
- Cameron Wolfe, Decoder-Only Transformers: The Workhorse of Generative LLMs
