Every modern language model you use is a reused model. The base capabilities of GPT, Claude, Gemini and Llama come from one expensive pre-training run on enormous amounts of text. The specific behaviors that make them useful chat assistants, such as following instructions, refusing harmful requests or formatting answers, come from cheaper training passes on top of that shared foundation. That reuse of a trained model for a new job is called transfer learning, and it is the economic engine of the current AI industry.
What transfer learning means
The standard definition comes from a 2010 survey by Sinno Jialin Pan and Qiang Yang: given a source domain and task, transfer learning aims to improve learning of a target predictive function using knowledge from the source, where the source and target differ in data or task. In plainer terms: instead of training a new model from scratch on scarce, expensive data, you start from a model that already learned something related.
The idea predates deep learning. Bozinovski and Fulgosi published a mathematical and geometrical model of transfer in neural network training in 1976. Lorien Pratt formalized a discriminability-based transfer algorithm in the early 1990s, and the 1998 book Learning to Learn, edited by Sebastian Thrun and Pratt, collected early work on the topic. What changed in the 2010s was scale: reuse stopped being a curiosity and became the default way any usable model gets built.
Why reuse works
A 2014 paper by Jason Yosinski, Jeff Clune, Yoshua Bengio and Hod Lipson, "How transferable are features in deep neural networks?", produced the evidence still cited most often. They showed that early layers of trained vision networks learn generic features (edge detectors similar to Gabor filters and color blobs) that transfer well across datasets, while late layers learn features specific to the original task. Transferring even from a distant, loosely related task still beat initializing from random weights, and networks that were fine-tuned after transfer outperformed networks trained from scratch on the same target data.
The practical lesson: a model trained on one pile of data encodes structure that is partially useful for other piles of data. Pre-training is expensive once; adaptation is cheap many times.

How it is done in practice
There are three common mechanics, in rough order of invasiveness:
- Feature extraction. Freeze the pre-trained weights and only train a small new output head on top. Cheap, but the model keeps its old representations.
- Fine-tuning. Unfreeze some or all layers and continue gradient training on the target task with a low learning rate. This is what a LoRA adapter or a QLoRA run modifies, usually just a small slice of the weights.
- Prompting and in-context adaptation. Change nothing in the weights at all. Give the model instructions and a few examples in the context window and let its pre-trained ability to pick up patterns from text do the work.
The middle option, fine-tuning of a pre-trained language model, became the standard recipe in natural language processing around 2018. ULMFiT, by Jeremy Howard and Sebastian Ruder, was the first widely noted universal version: it reduced error by 18 to 24 percent across six text classification tasks versus the previous state of the art, and with only 100 labeled examples it matched what training from scratch needed 100 times more data to reach.
BERT made the paradigm dominant. Google's 2018 paper described a model pre-trained on unlabeled text by masking and reconstructing words, then fine-tuned with one added output layer. No task-specific architecture edits. It reported new state-of-the-art results on eleven NLP tasks, including a GLUE score of 80.5 percent and a SQuAD v1.1 F1 of 93.2. After BERT, publishing a new NLP result without a pre-trained starting point became nearly unthinkable.
The LLM twist: transfer without touching weights
GPT-3 pushed the reuse idea in an unexpected direction. The 2020 paper "Language Models are Few-Shot Learners" argued that the pre-train-then-fine-tune recipe still required task-specific datasets of thousands of examples, while humans can do a new language task from a few examples or plain instructions. The authors trained a 175 billion parameter model, ten times larger than any previous non-sparse language model at the time, and tested it with no gradient updates and no fine-tuning at all: tasks were specified purely in text, with a handful of demonstrations in the prompt. At some tasks the zero-shot and few-shot results competed with prior fine-tuned state of the art.
That is transfer learning taken to its logical extreme. The knowledge transfer happens inside the context window instead of inside a training run. The instruct models and chat assistants that followed are the product of that idea, and the whole prompt engineering industry is downstream of it.
The Stanford Center for Research on Foundation Models gave this regime its name in 2021. Bommasani and colleagues defined "foundation models" as models like BERT, DALL-E and GPT-3 that are trained on broad data at scale and adaptable to a wide range of downstream tasks, and warned that while they rest on standard deep learning and transfer learning, their scale produces emergent capabilities. Their cautionary point still reads well: because downstream adaptation inherits the foundation model, homogenization means the defects of the base model propagate to everything built on it.
What practitioners actually reuse
In day-to-day LLM work, nearly every step in the pipeline is transfer learning in Pan and Yang's sense:
- An open-weights base model is the transferred source. Llama, Qwen and Mistral releases exist specifically to be adapted.
- Domain adaptation means continuing training on medical, legal or code text.
- Instruction tuning and RLHF are transfers of general text skill into assistant behavior.
- Embedding models fine-tuned for retrieval, and reward models trained from the same base, are more transfers.
- Model merging, where two fine-tuned variants of one base are blended in weight space, is transfer learning compressed into arithmetic.
Andrew Ng argued in a 2016 NIPS tutorial that transfer learning would be the next driver of commercial machine learning after supervised learning. In LLMs it became the whole business model: whoever pays for pre-training once sells reuse thousands of times.
When transfer fails
Reuse is not free of risk. The literature has two well-documented failure modes. Negative transfer: a source model can hurt the target if the domains are too different or the adaptation is done wrong, a phenomenon demonstrated experimentally as early as 1981 and a standing concern in fine-tuning recipes that need layer freezing, warmup and small learning rates to avoid destroying pre-trained structure. Catastrophic forgetting is its close relative: aggressive fine-tuning on a narrow task can erase general capabilities the base model had.
The modern answer to both is to transfer less destructively. LoRA and its variants change a low-rank slice of weights instead of everything. Distillation copies behavior into a smaller model without copying weights. And in-context learning sidesteps the problem entirely by leaving the weights frozen.

The short version
Transfer learning means reusing what a model already learned instead of paying to learn it again. Vision experiments in 2014 showed early network layers carry generic features worth stealing. NLP industrialized the idea in 2018 with pre-train-then-fine-tune. GPT-3 in 2020 showed the reuse could happen at inference time with a prompt, and the 2021 foundation-model framing made it the organizing concept of the field. Every fine-tuned specialist, every instruction-tuned assistant and every RAG pipeline with a frozen embedding model is the same idea at a different scale.
Sources
- Transfer Learning, Wikipedia (history section)
- Pan & Yang, A Survey on Transfer Learning, IEEE TKDE 2010
- Yosinski, Clune, Bengio, Lipson, How transferable are features in deep neural networks?, NeurIPS 2014
- Howard & Ruder, Universal Language Model Fine-tuning for Text Classification, 2018
- Devlin et al., BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2018
- Brown et al., Language Models are Few-Shot Learners (GPT-3), 2020
- Bommasani et al., On the Opportunities and Risks of Foundation Models, 2021
- Ruder, Recent Advances in Language Model Fine-tuning
