OCR Benchmarks 2026: What Each Measures and Who Leads

Three public benchmarks carry most of the OCR scores quoted in 2026. OmniDocBench scores document parsing on 1,651 PDF pages and reports an Overall score from 0 to 100, higher is better; TeleOCR leads its v1.6 leaderboard at 96.91. olmOCR-Bench runs 7,010 pass/fail unit tests on 1,403 English PDFs. OCRBench v2 asks 10,000 question-answer pairs of multimodal models, and SEEU DOC leads the maintainers' 2026.09 English board on its private test set at 76.2.

None of the three contains invoices. Business documents appear in a narrower set of places: the key-information-extraction datasets such as SROIE, CORD and DocILE; 67 SROIE receipt questions among the 1,000 in the original OCRBench; and the key-information track of CC-OCR, which fails a field if a single character is missed.

Each score below is given with its source and, wherever the source states them, the benchmark version it was measured on, who ran it (the benchmark's maintainers, Ai2 on its own benchmark, the model's own makers, or an open submission) and the date the source gives. The leaderboards were read on 2026-10-21. A model scored on two versions of the same benchmark has two different measurements, so scores from different versions never share a column here.

---

The OCR benchmarks side by side: contents, metric and who ran the scores

A benchmark dataset and its leaderboard are different things. The dataset is fixed for each version; the leaderboard is whoever has been scored on that version since, by whoever ran the evaluation. Most of what circulates as an OCR leaderboard is the second without the first: a list of numbers with no version and no runner attached.

The metrics run in both directions, which is the first thing to check before comparing two figures. An edit distance of 0.15 is good; a TEDS of 15 is not.

• Normalized edit distance (0 to 1, lower is better): how many character edits turn the model's output into the ground truth, scaled by length. OmniDocBench uses it for text and reading order.
• TEDS (higher is better): tree-edit-distance similarity between the predicted and true table, covering structure and cell text, introduced with IBM's PubTabNet. OmniDocBench reports it on a 0-100 scale, alongside a structure-only variant, TEDS-S.
• CDM (higher is better): the formula-recognition score OmniDocBench uses.
• Pass rate (0-100, higher is better): the share of binary unit tests passed, as in olmOCR-Bench.
• F1 and Hmean (higher is better): the balance of precision and recall over extracted fields. SROIE calls it Hmean; CC-OCR's version counts a field as failed if one character is wrong.
• Average precision (higher is better): used by DocILE for locating and extracting fields.
• ANLS (higher is better): DocVQA's answer-similarity score.
• A count out of 1,000: OCRBench v1's score, the number of questions answered correctly.

OmniDocBench: what it measures, the v1.6 leaderboard and why v1.5 scores differ

OmniDocBench is published by OpenDataLab at the Shanghai AI Laboratory. Its paper (arXiv 2412.07626) was accepted at CVPR 2025. The evaluation code is under the Apache 2.0 licence; the dataset is released "for research purposes only and not for commercial use".

What OmniDocBench contains

The first version had 981 PDF pages across nine types, according to the OmniDocBench paper: book 104, slides 133, research and financial reports 81, colourful textbooks 96, exam papers 114, magazines 97, academic literature 129, notes 116 and newspapers 111. By language, 290 pages were English, 612 Simplified Chinese and 79 mixed.

The set has grown twice since. The current README describes 1,651 PDF pages covering 10 document types, 5 layout types and 5 language types: the original 981, plus 374 added at v1.5 and 296 at v1.6. There is no invoice, receipt or business-form type at any version.

How it scores

Text and reading order are scored by normalized edit distance (lower is better). Tables are scored by TEDS and formulas by CDM (both higher is better). From v1.5 onward the headline number combines them:

Overall = ((1 − text edit distance) × 100 + table TEDS + formula CDM) / 3

Formula recognition is therefore a third of the Overall score. For a team whose documents contain no equations, a third of the headline number measures something they will never use.

The v1.6 leaderboard

Source: the maintainers' OmniDocBench v1.6 leaderboard, table labelled v1.6_full, read 2026-10-21. The maintainers' news entries say they "updated the model evaluation" for the models they name, so the rows are labelled here as the maintainers' leaderboard.

The top five span 1.69 points, from 96.91 to 95.22, and all five are specialised models of 1.2 billion parameters or fewer. GPT-5.2, at 86.59, sits below several open models of one to three billion parameters. The board has no Claude row and no Chandra row.

OmniDocBench v1.6 vs v1.5: why versions don't compare

Each major version changed both the pages and the way predictions are matched to the ground truth.

• v1.5 (2025-09-25) changed the matching algorithm, raised newspaper and note images from 72 DPI to 200 DPI, added 374 pages, dropped the separate Chinese and English scores and introduced the Overall formula.
• v1.6 (2026-04-10) added 296 harder pages and introduced Multi-Granularity Adaptive Matching.
• v1.7 (2026-04-30) added a Qianfan-OCR leaderboard and skills-based evaluation. The headline table is still labelled v1.6_full, so v1.7 reads as a tooling release.

A model's v1.5 score and its v1.6 score are not two attempts at the same test, and a list that ranks models by "their OmniDocBench score" while mixing versions is comparing different exams.

The v1.0 paper reported end-to-end results as overall edit distance, before the Overall score existed:

Source: OmniDocBench paper, Table 2, v1.0 (paper v2, 2025-03-25).

Where the ground truth comes from

The ground truth is not entirely hand-made: Hugging Face's guide to open OCR models (2025-10-21) notes that OmniDocBench's annotations "are not solely human-generated but are acquired through SoTA VLMs or conventional OCR methods".

olmOCR-Bench: 7,010 unit tests, its categories and the headers-and-footers rule

olmOCR-Bench is published by the Allen Institute for AI (Ai2), a non-profit. The original olmOCR paper (2025-02-25) introduced it as "a curated set of 1,400 PDFs". The olmOCR-Bench dataset and leaderboard page now gives the exact count, 1,403 PDF files with 7,010 unit tests, released under ODC-BY-1.0 for research and educational use. It is English only: the olmOCR 2 paper calls it "our English-language OCR benchmark".

olmOCR-Bench categories

Source: olmOCR-Bench dataset card.

The two maths categories hold 3,385 of the 7,010 tests, or 48.3%. The documents come from arXiv papers, public-domain maths textbooks, historical letters and typewritten documents from the Library of Congress, and multi-column and dense small-print pages. No category covers invoices, receipts or business forms.

Each test is pass or fail, in one of five classes: text presence, text absence, natural reading order, table accuracy (whether each cell sits beside the right neighbours) and maths formula accuracy (LaTeX checked by rendering it with KaTeX). The score is the pass rate from 0 to 100, higher is better, with a ± confidence interval. The olmOCR-Bench README does not say whether the Overall figure averages by category or by test.

Scores in Ai2's own table

Table 1 of the olmOCR 2 paper (2025-10-22) marks which figures Ai2 reproduced and which it took from the model's authors.

Source: olmOCR 2 paper, Table 1.

Seven of the 19 rows are the model authors' own figures, including both rows above olmOCR 2. The highest independently reproduced score in Ai2's table is therefore Ai2's own model on Ai2's own benchmark. olmOCR's own climb is recorded in the bench README: 68.2 at first release, 72.8 at v0.1.60, 75.8 at v0.1.68, 78.5 at v0.2.0 and v0.3.0, and 82.4 at v0.4.0.

The per-category view shows where models differ, which the Overall score hides:

Sources: the olmOCR-Bench README for every row except Chandra 2, which comes from Datalab's Chandra 2 model card; both read 2026-10-21.

For anyone extracting invoice line items, the Tables column is the one to read. It runs from 60.6 to 92.1 across these rows, a far wider spread than the Overall column.

The headers-and-footers rule

One of the five test classes, text absence, checks that a given piece of text does not appear in the output. The bench README gives the reason: "We generally want our OCR systems to filter out content like headers/footers/page numbers."

That suits Ai2's purpose, which is clean text from papers and books. On an invoice, the header and footer are often where the invoice number, the supplier's tax ID, the bank details and the page totals sit. A model tuned to this benchmark is rewarded for dropping exactly those regions. The class is visible in the scores: Nanonets-OCR2-3B passes 93.0 on Long Tiny Text and 32.1 on Headers & Footers, in Ai2's own run. A model that transcribes everything on the page loses points here.

The Hugging Face leaderboard

The dataset page also hosts an open leaderboard. Read on 2026-10-21, its top four were SoMarkAI/OXR-1.0 at 87.84, Infinity-Parser2-Pro at 87.6, LightOnOCR-3-4B at 86.3 and chandra-ocr-2 at 85.8.

Treat every row as self-reported. Hugging Face's documentation on how Hugging Face leaderboard scores are submitted describes evaluation results as "a work in progress feature": scores are stored as YAML files in model repositories, "can be submitted via PRs and marked as community-provided", and carry a "verified" badge only when the evaluation ran in Hugging Face Jobs. None of these four rows carries that badge.

OCRBench and OCRBench v2: question answering for multimodal models

OCRBench and its successor test large multimodal models rather than dedicated OCR engines: they ask a question about an image and check the answer. Both come from Yuliang Liu and colleagues, whose OCRBench repository records the first version's acceptance by Science China Information Sciences (2024-12-11) and OCRBench v2's at the NeurIPS 2025 Datasets & Benchmarks Track (2025-09-18).

OCRBench v1

The OCRBench paper (first posted 2023-05-13) describes 1,000 manually verified question-answer pairs drawn from 29 datasets:

A response scores if the ground-truth answer is present in the model's output, and the result is a count out of 1,000. In the paper, Gemini scored 659, GPT-4V 645 and Monkey 514; GPT-4V refused to answer 84 of the images.

Receipts make up 67 of the 1,000 questions, and there are no invoices. The scoring rule is also looser than field matching: a model that prints the right total somewhere in a long answer gets the point.

OCRBench v2

The OCRBench v2 paper (v2, 2025-06-05) reports "4x more tasks" than the first version across 31 scenarios, with "10,000 human-verified question-answering pairs" in English and Chinese, plus "a private test set with 1,500 manually annotated images". Its own finding was that "most LMMs score below 50 (100 in total)".

The public pairs and the private images are two separate test sets. A model's OCRBench v2 score means little until you know which one it came from.

The OCRBench v2 leaderboard, 2026.09

The maintainers' OCRBench v2 leaderboard is titled "The 2026.09 leaderboard" and scores the private data. The page says "We aim to update this benchmark every quarter", and does not say which rows the maintainers ran and which were submitted. Read on 2026-10-21:

The Chinese average has a different order: U2-OCR leads at 66.7, then TeleMM-2.0 at 66.2 and Qwen3.5-9B at 64.1, with Claude Opus 4.6 at 59.8. Claude Opus 4.6 has no English row on the 2026.09 board; its English 48.4 is on the earlier 2026.03 board.

The sub-scores split sharply. SEEU DOC scores 93.8 on Extraction and 42.4 on Parsing; Gemini 3 Pro Preview scores 93.9 and 27.1. On this board, pulling a named field out of a document is close to its ceiling for the leading models, and converting a whole page into structured output is not.

CC-OCR: the benchmark that fails a field for one wrong character

CC-OCR comes from Alibaba, the maker of the Qwen models. The CC-OCR paper (December 2024) splits it into four tracks: multi-scene text, multilingual text, document parsing and key information extraction. Together they hold "39 subsets with 7,058 full annotated images, of which 41% are sourced from real applications".

The key-information track is the one with receipts. Its constrained category has 1,008 images: 347 from SROIE, 100 from CORD, 311 from EPHOIE and 250 from POIE. An open category adds another 1,000. Scoring is field-level F1 under a strict rule: "a key-value pair is treated as a field, and even if a single character is missed, the score assumes the field extraction is failed."

That is the closest public analogue to how an accounts payable team judges extraction. A total read as 1,284.50 instead of 1,284.30 is not 99% correct. It is a wrong total, and the invoice either gets held or gets paid at the wrong amount. A benchmark scored by edit distance would rate the same output as a near-perfect match, because one wrong character is a tiny share of a page.

Results from the CC-OCR repository, tested around 2024-11-20 (all higher is better):

The paper breaks the key-information track down by source set (field-level F1):

The best SROIE score and the best CORD score belong to different models, and GPT-4o leads on CORD while finishing last of these four on the track total.

Alibaba ran all of these evaluations, and Alibaba's Qwen model tops the key-information track. The maintainers keep no leaderboard of 2025 or 2026 models. Later Qwen scores on CC-OCR do circulate, taken from Alibaba's own model cards: those are the benchmark maker's self-reported figures for its own model. The tables above describe late-2024 models, and the field has moved since.

---

Which OCR benchmarks contain invoices and receipts

OmniDocBench and olmOCR-Bench have no invoice, receipt or business-form category. Business documents enter the scored benchmarks in two places already covered, OCRBench v1's 67 SROIE questions and CC-OCR's key-information track, and in the key-information-extraction sets below, which are where receipt and invoice extraction has actually been measured.

DocILE's authors are from Rossum.ai, an invoice-processing company.

The Fraunhofer IAIS study scores recent multimodal LLMs on invoices directly. It also found that giving the model the page image directly generally beat converting the page to markdown first and extracting from that. Document-parsing benchmarks such as OmniDocBench score that markdown step, so a high parsing score does not settle whether parsing first is the better route for invoices at all.

Line items are where the business-document sets agree on difficulty. DocILE gives them a task of their own, VRDU's authors single them out, and ReceiptBench scores nested line items as a separate sub-task. A header field is one value in one place; a line-item table is many values whose rows and columns must stay aligned, which is the problem covered in more depth in extracting invoice line items. Receipts add their own trouble (thermal print, crumpled paper, abbreviations), set out in how receipt OCR works.

The table metric most parsing benchmarks use also comes from this neighbourhood. TEDS was introduced with IBM's PubTabNet and the TEDS metric, built on 568k table images, and IBM's FinTabNet financial tables hold complex tables from the annual reports of S&P 500 companies, more than 70K pages with structure annotations. TEDS compares cell text by edit distance, so a wrong digit in an amount costs a fraction of one cell rather than failing it.

Most of these sets can be downloaded and used to build a test of your own, a route covered in the guide to invoice datasets for building your own evaluation set.

The 2026 benchmarks that test OmniDocBench's 96s against real-world pages

TeleOCR's 96.91 on OmniDocBench v1.6 is a score on pages in their original digital or scanned form. A group of benchmarks published in 2026 asks what happens to parsing on pages that have been printed, photographed, warped or written in less-resourced languages. Their best scores sit below the 96s, PureDocBench's far below.

PureDocBench covers "10 domains, 66 subcategories, and 1,475 pages", and every page exists in three versions. Against the Clean version, scores drop by 3.34 points on Digital and by 11.61 points on Real, on average. A different model leads each track: TeleOCR on Clean, GLM-5.3-Flash on Digital and Gemini 3.6 Flash on Real.

Real5-OmniDocBench takes OmniDocBench's own v1.5 pages and rebuilds them physically under five conditions: scanning, warping, screen photography, uneven illumination and skew. Its authors conclude that "the reality gap in document parsing is far from closed". The paper comes from the PaddlePaddle team.

MDPBench, from the group behind OCRBench, was accepted at the NeurIPS 2026 Datasets & Benchmarks Track. The MDPBench paper (2026-03-30) reports that open-source models drop "an average of 17.8% on photographed documents and 14.0% on non-Latin scripts", and that document parsing has advanced "almost exclusively on clean, digital, well-formatted pages in a handful of dominant languages". Its leaderboard lists 34 models, from Gemini-3-pro-preview at 86.4 down to MinerU-2.5-pipeline at 33.5; behind the leader come Kimi K3 at 83.6, MonkeyOCRv2-B-Parsing at 83.3, MonkeyOCRv2-S-Parsing at 82.5 and dots.mocr at 80.5.

Three more benchmarks widen the language and layout coverage:

• MORE multilingual parsing benchmark (ICML 2026, 2026-07-03) scores document parsing across 149 languages, with code blocks, tables and catalogs among the structures it checks.
• KITAB-Bench Arabic OCR benchmark (ACL 2025) has "8,809 samples across 9 major domains and 36 sub-domains". Vision-language models beat traditional OCR "by an average of 60% in Character Error Rate", yet the best PDF-to-Markdown model reaches only 65% accuracy.
• socOCRbench (Noah Dasanaike, Harvard, 2026-02-12) scores 280 full-page images from five world regions, covering handwritten text, printed text and handwritten tables. Its author argues that "Most benchmarks, like OmniDocBench v1.5, over-index on modern printed text, clean scans, and well-resourced languages."

Supplier invoices arrive as phone photos of paper copies, scans of faxes, and layouts in whatever language the supplier works in. Those conditions correspond to PureDocBench's Real track and MDPBench's photographed split, not to the clean pages behind the headline numbers.

---

Same model, different numbers: five cases with their sources

The same model often appears with two scores on the same benchmark name. In each case below both figures are real and both are sourced; they differ because the version, the model release or whoever ran the evaluation differs.

These figures are stated side by side and not averaged or reconciled, because none of the sources says which is right. A roundup that prints a single OmniDocBench number for Mistral OCR, with no version and no runner, has chosen between 78.83, 85.66 and 93.07 without telling the reader.

Who ran the score: maintainers, model makers, open submissions and vendors

The labels on this page come down to six kinds, and each says how much independent checking stands behind the number.

• Maintainer leaderboard: the benchmark's own authors ran the evaluation or accepted the row onto their board.
• Ai2-run: Ai2 reproduced the result itself on olmOCR-Bench, its own benchmark.
• Reported by the model's authors: a figure the model's makers produced and the benchmark's table carries as theirs.
• Self-reported (model card): the model's makers publish the figure on their own page.
• Organiser-reported: a competition's organisers published the results of submitted entries.
• Community submission: anyone can add the row to an open board.

The distinction matters because a self-reported score was produced with the makers' own prompts, settings and preprocessing, which nobody else has necessarily repeated. On Hugging Face, scores arrive as pull requests to model repositories and can be marked community-provided; only a "verified" badge means the evaluation ran on Hugging Face's own infrastructure.

Aggregator sites that compile benchmark tables add a further layer. Some list OCRBench v2 and CC-OCR scores without saying which split was used or who ran the evaluation, and the model they show leading OCRBench v2 does not appear on the maintainers' private-split board at all.

Two OCR leaderboards are run by companies that sell document processing. OCR Arena is run by Extend, a document-processing vendor. It ranks models by Elo rating from user votes between two anonymised outputs on documents the users upload. Elo built from online votes depends on the order in which votes arrive, voters judge which output looks better rather than whether each field is correct, and the mix of documents behind the ranking is unknown. The IDP Leaderboard is run by Nanonets, also a document-processing vendor.

The name OCR Arena also turns up in research. The dots.mocr paper reports an "OCR Arena Elo leaderboard" of its own: a pairwise Elo ranking judged by a model, which is unrelated to the ocrarena.ai site.

Some of the benchmarks are published by groups whose own models are scored on them:

• OmniDocBench and MinerU come from the same OpenDataLab group, and MinerU2.5-Pro is fourth on the v1.6 board.
• Real5-OmniDocBench is by PaddlePaddle authors, and PaddleOCR-VL-1.6 leads it.
• CC-OCR is Alibaba's, and Alibaba's Qwen model tops its key-information track.
• olmOCR 2 was trained with rewards from unit tests of the kind olmOCR-Bench uses, and it is the highest independently reproduced row in Ai2's table.

What the public benchmarks leave untested for invoices

The benchmarks' own authors are among the plainest about their limits. Ai2's olmOCR 2 paper notes that floating elements such as tables "lack a definitive ground truth representation", and that edit distance "often rewards/penalizes these cases differently". Mistral AI, a model maker writing in its Mistral OCR 4 announcement (2026-06-23), says ground-truth errors and artifacts "concentrate in mathematical, scientific, and multi-column documents", and that it treats "the aggregate score as directional rather than definitive". Hugging Face's guide to open OCR models advises collecting "a dataset of representative examples of your task domain" and testing a few different models on it.

For an invoice pipeline, the gap between the public boards and the work is specific. No headline benchmark checks whether:

• the invoice total, tax amount, invoice number, supplier tax ID and bank details were read exactly, character for character;
• line items that run across two or three pages were joined into one table without duplicated or dropped rows;
• the line amounts add up to the stated net, tax and gross totals;
• the header and footer content that olmOCR-Bench rewards models for dropping was kept;
• any of this holds on your own suppliers' layouts, scan quality and languages.

Only the key-information tests score fields at all (CC-OCR's track, OCRBench's extraction questions and the receipt and invoice sets), and even they measure someone else's documents. How to build that test on your own invoices is covered in testing invoice automation on your own documents. For the separate question of which engine to run, see choosing a Python OCR library for invoices, and for commercial services, invoice OCR API benchmarks on accuracy, speed and cost. Readers who need invoice capture as a working tool rather than a research model can compare invoice OCR software built for that job.