This is an opinion piece. The research facts are linked; the argument is mine.
"Model collapse" is one of the most quotable ideas in AI. The story goes like this: the web fills up with AI-written text, new models train on that text, and each generation gets a little worse until the models are useless. It sounds like a law of nature. It is not. Read the two best-known papers side by side and a narrower, more useful lesson comes out. Collapse is a risk created by one specific habit, throwing away real human data and replacing it with synthetic data. Keep the real data, and the risk shrinks a lot.
That matters for anyone who uses AI, not only for the labs. If you believe collapse is inevitable, you will expect models to decay and you will discount every new release. If you understand what actually causes it, you can ask better questions about how a model was trained.
What the Nature paper found
The paper that made the term famous is "AI models collapse when trained on recursively generated data", published in Nature in July 2024 by Ilia Shumailov and colleagues. Its central claim is direct. The authors write that "indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear."
The "tails" are the low-probability parts of the data. The paper notes that such low-probability events "are often relevant to marginalized groups" and are "vital to understand complex systems." As the training loop repeats, those tails are what the models lose first. The paper splits this into two stages. In "early model collapse, the model begins losing information about the tails of the distribution," and in "late model collapse, the model converges to a distribution that carries little resemblance to the original one."
The authors showed the effect in language models (they fine-tuned Meta's OPT-125m model on the wikitext2 dataset), and also in variational autoencoders and Gaussian mixture models. That breadth is why the paper landed so hard. It was not one odd result in one setup.
For the math of how the tails shrink from one generation to the next, see our technical explainer on model collapse in large language models.
The detail most summaries skip
The same Nature paper ran a second setting that gets far less attention. In it, each new generation was trained with "a random 10% of the original data points" kept in the mix. The authors found that "preservation of the original data allows for better model fine-tuning and leads to only minor degradation of performance." They add that both training regimes still led to degraded performance.
So even the paper that made the term famous showed that keeping some real data softens the outcome. The scary version of the story depends on a setup where the real data is gone.
What the follow-up paper found
A second paper, "Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data" by Matthias Gerstgrasser and colleagues, tested that assumption head on. Its abstract notes that earlier studies "largely assumed that new data replace old data over time," and argues that "an arguably more realistic assumption is that data accumulate over time."
The results split cleanly along that line. The authors "confirm that replacing the original real data by each generation's synthetic data does indeed tend towards model collapse," then show that "accumulating the successive generations of synthetic data alongside the original real data avoids model collapse." They report that these results "hold across a range of model sizes, architectures, and hyperparameters," and they found similar results for diffusion models and variational autoencoders on other kinds of data.
They also back it with math. In a simple linear model setup, they prove that if data accumulate, "the test error has a finite upper bound independent of the number of iterations, meaning model collapse no longer occurs."
My take: collapse is a data management problem
Put the two papers together and the honest summary is not "AI will eat itself." It is closer to this: training only on recycled output is harmful, and the fix is to keep the original human data in the mix. That is a choice about how training data is handled, not a fate.
I think the doom framing is overhyped for three reasons.
- The worst case needs a careless setup. Collapse shows up most clearly when real data is replaced. Both papers show that keeping real data blunts the effect.
- The word "indiscriminate" is doing real work. The Nature paper warns against "indiscriminate use" of model output. That is a warning about careless training, not a ban on all synthetic data.
- The fix is already named. The Nature authors say "we need to make sure that access to the original data source is preserved and that further data not generated by LLMs remain available over time," and the follow-up paper shows that keeping real data alongside synthetic data avoids collapse in its tests.
But I also think the opposite take, that model collapse is a myth, is wrong. The Nature authors raise a real problem that the follow-up paper does not make go away. They write that "it is unclear how content generated by LLMs can be tracked at scale," and they name "community-wide coordination" as one option, so that the different parties involved in building and deploying LLMs share "the information needed to resolve questions of provenance." They also write that their evaluation "suggests a 'first mover advantage' when it comes to training models such as LLMs." If you cannot tell human text from AI text, you cannot reliably keep the human part, and the safeguard in the follow-up paper depends on doing exactly that.
So the underrated part of this story is not collapse itself. It is provenance: knowing where training data came from. That is the piece that decides whether the safe setup is possible at all.
Why this matters to everyday users
Most people will never train a model, but I think the lesson carries over. The Nature paper is about models losing the rare, low-probability parts of their data. My own rule follows from that: when you ask a chatbot about a niche topic, check the answer harder than you would for a common one, the same way you would guard against a chatbot that may make things up.
It is also, in my view, a reason to be careful with simple scorecards. A single score says little about how a model handles rare cases, which is one more reason a leaderboard rank tells you less than you think. And when you feed a model your own material, the quality of what you give it shapes what you get back, a point that applies to the text you paste into its context window too.
The bottom line
Model collapse is a real effect with a clear cause. Treat it as a warning about one bad habit, replacing human data with machine output, not as a prophecy that AI must decline. The research says the habit can be avoided. Whether it is avoided depends on something less exciting than the headline: keeping good records of where data comes from.
