Subscribe
AI
6 min read

2,217 Decisions Later: What the Docket Data Says About AI-Hallucinated Citations

In June 2023, a federal judge in Manhattan sanctioned two lawyers for filing a brief with six invented case citations (Mata v. Avianca) and set the template for what everyone insisted would be an isolated embarrassment. Three years on, the isolated-embarrassment theory is dead. Damien Charlotin, a research fellow at HEC Paris, has catalogued 2,217 judicial decisions worldwide in which a court addressed generative-AI hallucinated content in filings. The database, last updated on 25 October 2026,

2,217 Decisions Later: What the Docket Data Says About AI-Hallucinated Citations

In June 2023, a federal judge in Manhattan sanctioned two lawyers for filing a brief with six invented case citations (Mata v. Avianca) and set the template for what everyone insisted would be an isolated embarrassment. Three years on, the isolated-embarrassment theory is dead. Damien Charlotin, a research fellow at HEC Paris, has catalogued 2,217 judicial decisions worldwide in which a court addressed generative-AI hallucinated content in filings. The database, last updated on 25 October 2026, counts decisions, not incidents, and it explicitly does not cover the wider universe of fake citations that never reached a judge's attention. The true number of filings with fabricated authority is, by construction, larger than the number of rulings about them.

The growth curve is the story. By one count of the database's CSV export, US decisions involving a lawyer went from 14 in 2024 to 207 in 2025 and 361 in the first ten months of 2026. The 2026 US total also includes 885 decisions involving self-represented litigants, more than the lawyer entries. Courts are no longer catching the occasional associate who skipped Shepardizing. They are processing a stream.

Stack of legal briefs with papers lifting off beside a gavel

The sanctions are receipts, not deterrents

Most documented outcomes are warnings. When money changes hands it is usually four or five figures: the Sixth Circuit's March 2026 order in Whiting v. City of Athens imposed $15,000 punitive sanctions on each of two Tennessee attorneys, plus double costs and a disciplinary referral, after briefs in three consolidated appeals contained what the court called over two dozen fake citations and misrepresentations of fact. In June 2026, a federal district court in Mississippi dealt with a contract dispute where filings from both sides cited hallucinated cases; the trial was canceled and the two lead attorneys were suspended from the district for two years. A New York case in June 2026 drew an $80,056 adverse costs order and a bar referral after a filing contained fabricated cases and quotations, with remnants of AI drafting prompts still visible in the document. The largest single-matter penalty one tracker can source is roughly $109,700.

Aggregate to $145,000 in US court-ordered AI-filing penalties for all of Q1 2026, the figure ComplexDiscovery compiled, and the mismatch with the workload becomes stark. Judges in the cases Princeton researchers collected describe the burden as an enormous waste of judicial resources, and increasingly say lesser sanctions have been insufficient to deter the conduct. The fines are small because most flagged filings come from litigants with little money and from lawyers whose misconduct courts treat as negligent rather than cynical. The cost that never appears on the sanctions docket is clerk hours.

Model progress did not fix this

The assumption behind all the certification rules was that the problem would shrink as models improved. A Princeton study, Who Checks the Citations?, tested that directly. Querying eight generations of ChatGPT on 92 legal drafting prompts and verifying over 8,000 generated citations against CourtListener and Westlaw, the authors found hallucination rates fell from roughly 25% in the GPT-3.5 era to a low of 1.23% in mid-2024 GPT-4o releases, then climbed again: GPT-5.1 fabricated citations at 6.57%, significantly worse than the best 2024 model.

Worse, the verification burden grows even when error rates fall. Newer models emit more citations per document, drawn from a broader and less canonical set of cases that are individually harder to check. The authors borrow an economist's term for it: as AI cuts the per-unit cost of drafting, filing volume rises, and total verification work rises with it.

The error signature has also shifted. Early cases involved cases that simply never existed, trivially checkable. The Charlotin rows from late 2026 are dominated by subtler categories: real cases cited for propositions they do not support, quotations that do not appear in the cited opinion, wrong reporter numbers. A Federal Labor Relations Authority decision from October 2026 found 22 of an agency's 29 footnotes and 35 of its 42 legal or record citations were fictitious, erroneous or irrelevant. Litigation counsel tracking the trend, including Norton Rose Fulbright, describe fabricated quotations and mischaracterized holdings as the harder-to-detect successor to the outright phantom case.

The other comforting assumption, that premium tools grounded in real databases would solve it, died in a Stanford and Yale study published in the Journal of Empirical Legal Studies. Testing the tools as they stood in May 2024, the researchers found Lexis+ AI hallucinated on 17% of queries, Westlaw AI-Assisted Research on 33%, and GPT-4 on 43%. The retrieval-grounded products were meaningfully better than a raw chatbot and still wrong on roughly one in six to one in three answers, with errors including invented statements of procedural rules and citations to cases that held the opposite.

The rulebook that is not one

Courts have responded in three incompatible ways, and the choice is a matter of venue. Some judges and circuits require affirmative disclosure plus certification. The Tenth Circuit added an AI certification requirement, and coordinated Fifth and Eleventh Circuit revisions in May 2026 layered disclosure of tools used on top of a verification attestation. Others went the other way. New York's Part 161 policy, effective 1 June 2026, states that filers should not be required to disclose AI use at all, on the theory that the duty to verify every submission already covers it. The Fifth Circuit's own subcommittee recommended against a court-wide certification rule before its Fletcher decision this year, concluding that Rule 11 and existing duties suffice. ABA Formal Opinion 512 sits above the fray: AI is permitted, verification is mandatory, check the local judge's standing orders.

The practical consequence is that a firm filing in multiple jurisdictions maintains a compliance patchwork per courtroom, not per model.

Hand holding a magnifying glass over an open law book with a stack of volumes beside it

Can AI police what AI produces?

Princeton's benchmark answer is: not reliably yet. Their best agentic checker, GPT-5 with search tools, reached 84.4% recall on injected citation errors with a 55% F1 score while averaging 15.3 retrieval steps per excerpt. Recall on wrong pincites was the weak spot, and false positive rates varied enormously by configuration, from about 11% in one Claude-based agent to 66% in a Gemini 2.5 Flash baseline. An error detector with a 16% miss rate and a coin-flip precision profile on mixed errors does not discharge a Rule 11 obligation. It also inherits an access problem: much case law is free to query and page-level detail is not, which penalizes exactly the self-represented filers who make up the largest share of the docket data.

What the numbers recommend

The docket record supports a narrow, boring conclusion. AI drafting is now normal in litigation and will keep getting cheaper; the scarce input is human verification of citations, quotations and pincites against primary sources. Firms that treat that step as a billable line item, with a logged citation ledger, are aligned with how courts actually sanction. Tools that auto-check citations are worth deploying as a first filter, not as the signature on the brief. And anyone relying on a subscription legal AI to avoid the problem is buying the 17% tail, not the marketing page.

The sanctions data says the mistake is not being caught. It is being caught: the Charlotin US counts for 2026 work out to roughly 30 decisions per week, and the volume is still rising.

Sources

AILLM