Fine-tuning is what happens after pre-training. A language model first reads a very large pile of internet text and learns to predict the next token. Then, in fine-tuning, that same model keeps training, but on a smaller, deliberately chosen dataset, and its weights update to match. The result is a model that behaves differently: it answers instead of rambling, sticks to a format, or talks like a specialist.
That is the whole idea. The details that matter are which data you tune on, how much of the model you let change, and what quietly breaks along the way.
Base models do not answer questions
A raw pretrained model is a completion engine, not an assistant. Give the GPT-3 base model the text "Write a poem about the ocean" and it may respond by listing more requests, because on the internet, instructions are usually followed by other instructions. OpenAI's InstructGPT paper made this the headline result of 2022: a 1.3-billion-parameter model trained to follow instructions beat the 175-billion GPT-3 on human ratings, despite being 100 times smaller (Ouyang et al., 2022).
The pipeline they used became the template for nearly every assistant model since. Stage one is supervised fine-tuning (SFT): train on human-written demonstrations of ideal responses. Stage two is reinforcement learning from human feedback (RLHF): train on rankings of which response annotators preferred. The industry later compressed stage two into simpler preference-optimization methods like DPO, but stage one, SFT, remains standard. Every model card that says "instruct" or "chat" is advertising that an SFT pass happened.
The idea that tuning on many tasks makes a model better at tasks it has never seen goes back one step further. Google's FLAN paper (2021) fine-tuned a model on 62 NLP datasets described as instructions and found the tuned model became a zero-shot learner on held-out tasks (Wei et al., 2021). Around the same time, the T0 work reached a similar conclusion from a different dataset (Sanh et al., 2021). Instruction tuning, not scale, was what turned next-token predictors into general assistants.
What the tuning data actually buys you

The most useful result about fine-tuning data is a negative one. Meta's LIMA team fine-tuned a 65B LLaMA model on only 1,000 hand-curated question-answer pairs, with no RLHF, and human raters found its responses equivalent or preferred to GPT-4 in 43% of comparisons (Zhou et al., 2023). Their conclusion: almost all knowledge in a language model is learned during pre-training, and fine-tuning mostly teaches style, format, and how to present what the model already knows.
Practitioners summarize this as: fine-tuning changes behavior, not memory. It is good at teaching a model to answer in your JSON schema, adopt a consistent tone, follow a legal-document format, or stay inside a narrow task distribution. It is a weak and unreliable way to pour new facts into a model, because the few hundred or few thousand examples in a typical fine-tuning set are drowned out by trillions of pre-training tokens. For fresh facts, retrieval (RAG) still beats tuning, and the model will hallucinate less when the facts are in the window rather than half-remembered from a training run.
Full fine-tuning versus LoRA, in one paragraph

The naive approach, full fine-tuning, updates every weight in the model. It works, and it is expensive: you need memory for the weights, the gradients, and the optimizer state, which is several times the model size.
LoRA (Low-Rank Adaptation) changed the economics. Instead of touching the big weight matrices, you freeze them and train two small skinny matrices whose product approximates the update. On GPT-3 175B, LoRA cut trainable parameters by 10,000 times and required GPU memory by 3x versus full fine-tuning, with no added inference latency, because the two small matrices can be multiplied into the original weights after training (Hu et al., 2021). QLoRA pushed further by combining LoRA with 4-bit quantization of the frozen base model, making it possible to fine-tune a 65-billion-parameter model on a single 48GB GPU (Dettmers et al., 2023). Adapter layers, the method LoRA displaced, insert small neural modules between frozen layers and had already shown that under 8% of parameters could match full fine-tuning on NLP benchmarks (Houlsby et al., 2019).
The practical consequence: a team with one workstation GPU can specialize an open-weights model, and a company can ship one base model plus a drawer of small adapters instead of many full copies.
The failure modes people discover the hard way
Catastrophic forgetting. Tuning on new data degrades old capabilities. Work that probes the mechanism with "instruction vectors" found that the capability most damaged by fine-tuning is instruction following, not stored knowledge: the tuned model mostly adds narrow reasoning patterns that overwrite how it responds to prompts, which then looks like forgetting (Jiang et al., 2024). The standard mitigations are unglamorous: low learning rates, mixing general instruction data into the domain data, and evals that include the old tasks, not just the new ones.
The alignment tax. InstructGPT's authors named it themselves: making a model follow instructions cost measurable performance on some public NLP benchmarks, because optimizing for one behavior trades away others (Ouyang et al., 2022). Every fine-tuning pass spends some of the base model's generality.
Overfitting to the demo set. A model tuned on a few hundred examples learns their phrasing, their length, their quirks. It will ace your sample prompts and fall apart on the long tail. LIMA's authors were candid about the flip side of their result: without preference training the model is less robust, and one unlucky decoding sample can produce a bad answer (Zhou et al., 2023).
The decision, stated plainly
The 2026 default order of operations for adapting a model is roughly: prompt it well first; give it retrieval when it needs facts it does not have; fine-tune when you need consistent behavior, format, or voice at scale, and are willing to run evals that watch for forgetting. Fine-tuning is not how you make a model smarter. It is how you make a smart model dependable in one direction.
Sources
- Ouyang et al., 2022, "Training language models to follow instructions with human feedback" - https://arxiv.org/abs/2203.02155
- Wei et al., 2021, "Finetuned Language Models Are Zero-Shot Learners" - https://arxiv.org/abs/2109.01652
- Sanh et al., 2021, "MultiTask Fine-tuning after Self-Supervised Pretraining" (T0) - https://arxiv.org/abs/2110.08207
- Zhou et al., 2023, "LIMA: Less Is More for Alignment" - https://arxiv.org/abs/2305.11206
- Hu et al., 2021, "LoRA: Low-Rank Adaptation of Large Language Models" - https://arxiv.org/abs/2106.09685
- Dettmers et al., 2023, "QLoRA: Efficient Finetuning of Quantized LLMs" - https://arxiv.org/abs/2305.14314
- Houlsby et al., 2019, "Parameter-Efficient Transfer Learning for NLP" - https://arxiv.org/abs/1902.00751
- Jiang et al., 2024, "Refine Large Language Model Fine-tuning via Instruction Vector" - https://arxiv.org/abs/2406.12227
