Optical character recognition, OCR, is the electronic conversion of images of typed, handwritten or printed text into machine-encoded text. The definition comes from Wikipedia, and the problem it solves is one anyone has met who ever tried to copy a line out of a scanned PDF and found the cursor had turned into a crosshair. A scanned page is a picture. The letters are there, but they carry no text underneath them. OCR is what puts the text layer back.
That sounds like a solved and dusty problem. It is neither. Every AI assistant that answers questions from a company's PDFs depends on an OCR or document-parsing step between the raw file and the model's context window. Mistral, when it launched its OCR product, cited the estimate that roughly 90% of the world's organizational data is stored in documents. OCR is the tollgate between that archive and anything a language model can read.
The reading machine was built for the blind
The technology is older than the computer it now feeds. Early OCR traces to telegraphy and to devices meant to let blind people read. In 1912 and 1914, Edmund Fournier d'Albe demonstrated the Optophone, a handheld scanner moved across a printed page that produced tones corresponding to specific letters, according to the timeline of OCR maintained on Wikipedia. In 1929 the Austrian engineer Gustav Tauschek built the first device recognized as an OCR machine, a "Reading Machine" that matched words against stored templates. Physicist Emanuel Goldberg patented his "Statistical Machine" for reading characters and converting them to telegraph code in 1931, a patent IBM later acquired.
Industry came next. Reader's Digest installed an OCR reader to convert typewritten sales reports into punched cards in 1954, as Incode's history of the field records. Canada Post began using OCR in 1971 to read envelope addresses and print routing bars in ultraviolet ink. The first big commercial leap was font independence. In 1974 Ray Kurzweil founded Kurzweil Computer Products and developed omni-font OCR, software that could recognize text printed in virtually any typeface. He turned it into a reading machine for the blind, unveiled in 1976 with the National Federation of the Blind; a commercial version followed in 1978, and LexisNexis was one of the first customers, according to Wikipedia's OCR article. Kurzweil sold the company to Xerox in 1980. Robert Noyce, co-founder of Intel, had in parallel founded Caere Corporation in 1977, which built the first passport scanner for the US State Department in 1984. Retailers Sears, Kmart and J.C. Penney started scanning price tags in 1987. The ABBYY founding followed in 1989. OCR had become plumbing.
For two decades the workhorse was Tesseract, an engine developed at Hewlett Packard Labs between 1985 and 1994 and open-sourced in 2005 by HP with the University of Nevada, Las Vegas. Its own tutorial history describes the "dark ages" of 1995 to 2005 when the code gathered dust before revival, and notes that conventional engines of that era never fully handled proportionally spaced type, which is to say virtually all real documents.
How classic OCR works

A traditional OCR engine is a pipeline, and the stages are still visible in modern products. Wikipedia's techniques section lays it out.
Preprocessing fixes the image before any reading happens: de-skewing a scan that came out of the feeder crooked, despeckling noise, and binarizing the page into black and white so the recognition algorithms have a clean separation between ink and background. Layout analysis, historically called zoning, identifies columns, captions and blocks. Line and word detection establishes baselines, and character segmentation pulls individual glyphs apart, or glues back together the ones a bad photocopy broke in half.
Recognition then comes in two flavors. Matrix matching compares an input glyph pixel by pixel against a stored glyph, which works on typewritten text in a known font and falls apart on anything else. Feature extraction decomposes a glyph into parts, lines, closed loops, intersections, and compares that feature signature against character prototypes. Feature methods scale to handwriting and to arbitrary fonts, which is why they won. Tesseract uses a two-pass trick: letters recognized with high confidence on the first pass help disambiguate the letters the engine struggled with on the second.
Post-processing is where language enters. Engines constrain output to a lexicon of allowed words, and co-occurrence statistics correct the rest: a page that appears to say "Washington DOC" almost certainly says "Washington, D.C." Some systems never trusted the algorithm at all and instead constrained the input. The fonts OCR-A and OCR-B, and the magnetic ink character recognition scheme still printed at the bottom of paper checks, are typefaces engineered so machines can read them, an admission that reading arbitrary human type was harder than designing new type for the machine.
Transformers ate the pipeline

The machine-learning turn started inside the old structure. Tesseract 5 replaced the character classifiers with an LSTM-based recognition engine that reads whole lines rather than isolated glyphs.
The break came in 2021 with Microsoft's TrOCR. The paper's argument was that the classic stack, CNN for image understanding plus RNN for character-level generation plus a separate language model for post-processing, was exactly what pretrained transformers made unnecessary. TrOCR splits a text-line image into patches, feeds them to an image transformer encoder, and generates wordpiece text autoregressively with a text transformer decoder. Pretrained on large synthetic data and fine-tuned on human-labeled sets, it beat the state of the art on both printed and handwritten recognition. The pipeline's first and last stages, segmentation and dictionary cleanup, began evaporating.
The current era: the page is the prompt
Vision-language models finished the job. Hugging Face's guide to open OCR models describes the field as it now stands: most serious open OCR systems are fine-tuned VLMs, and capabilities extend past transcription into answering questions about documents and retrieving them by query. A modern model takes a whole page as an image and emits Markdown, HTML or DocTags in one pass, tables and equations included. Reading order, which older engines reconstructed with brittle manual post-processing, is now emitted by the model itself, often alongside bounding boxes. The HF post notes this "grounding" is not decoration: anchoring each token to a region on the page measurably reduces hallucinated text.
The commercial markers of the shift are cheap to list. Mistral launched Mistral OCR in March 2025 as an API priced at 1,000 pages per dollar, claiming up to 2,000 pages per minute on a single node and pitching the product squarely as the ingestion layer for RAG over messy multimodal PDFs. Its vendor-run internal benchmark scored it 94.89 overall against GPT-4o's 89.77; vendor benchmarks deserve the salt they get, but the direction was hard to miss: OCR had stopped being a scanner feature and become a model-serving decision. The successor generation, Mistral OCR 4, adds block-level bounding boxes, block classification, inline per-word confidence scores and 170 languages, and ships as a single self-hostable container.
The open side raced ahead on method. AllenAI's olmOCR line, the flagship being olmOCR 2 from October 2025, is a 7B VLM trained with reinforcement learning on verifiable unit tests: synthetic documents with known ground-truth HTML source generate binary checks, and the model is optimized to pass them. The largest gains landed exactly where classic engines bled: math conversion, table parsing, multi-column layouts. It reads a historical letter that earlier versions consistently garbled, in this case Lincoln's handwriting. Ai2 ships an FP8 quantized version running at 3,400 output tokens per second on one H100, which the release post rounds to processing 10,000 pages for under $2. Kurzweil sold a reading machine to institutions in 1978. The equivalent capability is now a utility rate.
Still not solved
The honest framing is in the benchmarks themselves. OlmOCR-Bench tests 900 public-domain books against 1,500 hand-graded checks, and the category scores show where models still fail: dense tables, multi-column academic text, ancient degraded scans, anything handwritten. LlamaIndex's review of the benchmark concludes document parsing "is not getting 100% solved anytime soon, even by the frontier models," and recommends teams build small test suites against their own documents rather than trusting any leaderboard. Which matters to language models specifically because parsing errors do not stay in the parser. A misread invoice number becomes a bad chunk in the index, which becomes a confident wrong answer two steps later, and nobody downstream ever sees the scan that caused it. The New York Times found the same economics with its own Document Helper tool, built so reporters could triage roughly 5,400 pages an hour. Document volume has outrun every reading technology since; the current models just hold the record.
Terms you will run into
- OCR: printed or typed text converted to machine text, glyph by glyph or line by line.
- ICR and IWR: intelligent character and word recognition, the same task on handwriting, which is harder and needs training on messy human samples.
- Scene text: reading signs, license plates and billboards photographed at angles, the domain of automatic number-plate recognition.
- Text layer: the invisible machine text stored under a scanned PDF image. Whether it exists, and whether it is right, is the whole product.
- Document parsing or extraction: the VLM-era word for OCR plus layout plus tables plus images, now priced by the page.
Sources:
- Optical character recognition, Wikipedia
- Timeline of optical character recognition, Wikipedia
- The History of Optical Character Recognition, Incode
- Tesseract OCR history, DAS 2016 tutorial
- TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models (arXiv 2109.10282)
- Supercharge your OCR Pipelines with Open Models, Hugging Face
- Mistral OCR announcement, March 2025
- Mistral OCR 4 announcement
- olmOCR 2: Unit Test Rewards for Document OCR, Ai2
- olmOCR 2 paper, arXiv 2510.19817
- OlmOCR-Bench Review, LlamaIndex
