Correction (28 September 2026): this note argued that where the Archive's reading and ours agree on nineteen words in twenty, the free text is at least as reliable as ours. A first measurement of both readings against 122 independently proofread pages found that the Archive's text silently misreads about one printed number in a hundred (a preliminary figure, resting on eight paired cases), a 5180 written as 6180, and that a word-agreement score cannot see a one-digit change. If you quote a date or a page reference from Archive text, check the image. The measurement is in Checking the Archive's OCR against proofread pages.
Correction (13 September 2026): the leaf-offset repair described under “The rejects that were our fault” was itself a mistake, and a more interesting one than the bug it claimed to fix. The Archive's text file and its page images skip exactly the same leaves, so the correct offset is always zero. The books our search appeared to rescue were books whose reference pages had been transcribed from our own scans, which were themselves one leaf out. The test was faithfully compensating for a defect in its own instrument. About 940 pages were written carrying the neighbouring leaf's text and have since been repaired and checked by hand; about 50,900 more carry a wrong leaf reference while still reading correctly on screen, and wait on a separate repair to the images. The importer now refuses these books instead of compensating. The rest of this note stands as written.
The library holds 21.2 million page images. Machines have read 6.5 million of them into text you can search, quote and translate. The other 12.6 million pages are pictures. A picture of a page cannot be searched, cannot be cited by sentence, and cannot be translated, so until a page is read it is not really in the library at all.
Reading pages costs money. Our current reader, a Gemini vision model, transcribes a page for about a seventh of a cent when we batch the work, and rather more when we do not. Across 12.6 million pages that is real money, and it is why so many books in the catalogue still show the first twenty-five pages as text and the rest as scans.
But 5.5 million of those unread pages, in 26,348 books, came to us from the Internet Archive. And the Archive reads everything it scans. Every item carries a file of machine text, one block per leaf, produced by a conventional optical character reader, ABBYY FineReader or Tesseract depending on the year. That reading is free, it is instant, and for a hundred years of clean roman type it is often very good.
It is also, on other books, very bad. This note is about how we tell the difference without looking at every page, what we found when we ran the test across our English books, and whether a cheap language model can fix the readings that fail.
Where the unread pages are
Before deciding anything, we counted. The unread pages are not spread evenly.
| Language | Pages not yet read | Of which Internet Archive scans |
|---|---|---|
| Latin | 6,446,000 | 2,511,000 |
| Chinese | 1,660,000 | 1,591,000 |
| Greek | 1,633,000 | 234,000 |
| German | 756,000 | 147,000 |
| English | 458,000 | 447,000 |
| Sanskrit | 237,000 | 212,000 |
| French | 104,000 | 82,000 |
English is a small slice of the whole, but almost all of it is Archive material, and English roman type is the case the Archive's reader was built for. So English is where we started. Latin is the prize: two and a half million Archive pages, mostly nineteenth-century editions of classical and patristic texts in clean type. Chinese is the trap: the Archive's reader produces almost nothing usable on it, which we already knew from a July comparison of its output with ours across scripts and centuries.
Two readers, one leaf
The test is simple. Almost every book in the catalogue, however unfinished, already has its first twenty-five pages read by our own model. Those pages were paid for. The Archive read the same leaves. So for each book we can lay the two readings side by side, page for page, and ask how often they agree.
Agreement is measured as the share of words that appear in the same order in both readings. A page where both say “Arcas then, being the grandson of the antediluvian Lycaon” scores near one. A page where the Archive says “Areas then, being the grandfon of the ante-diluvian Lycaon, and the fon of CaUifto” scores lower, and the words it lost tell you why: the book is from 1803, it is set with the long s, and a conventional reader sees an f.
We take a book only when the median agreement across its sample pages is 0.85 or better. That threshold is deliberately strict. Agreement is not accuracy: two readers can make the same mistake, and where they differ we do not know from the score alone which one is right. What the score gives us is a bound. Where the two independent readings match on nineteen words in twenty, the free text is at least as reliable as the text we would have paid for, because the readers share no mechanism. One is a generative model that can, on a bad day, recite what it remembers instead of what it sees. The other is a pattern matcher that cannot remember anything. When they agree, neither failure mode is in play.
The rule is per book, never per date. A 1848 Shaker memoir clears the bar at 0.93. A 1812 Shaker treatise, three printers earlier and still using the long s, fails at 0.75. Same shelf, same decade, opposite verdicts, and the score sees it without anyone reading a page.
What happened on the English shelf
We ran the test across every English book in the catalogue with Archive scans and unread pages: 2,076 books. The Archive had no text file at all for 258 of them. For 452 more we had fewer than five pages of our own reading to compare against, so no verdict was possible. That left 1,366 books with a score.
| Verdict | Books | Pages we can fill |
|---|---|---|
| Accepted, agreement 0.85 or better | 624 | 66,322 |
| Rejected | 742 | 136,545 |
| No reference pages to compare | 452 | 197,914 |
| No Archive text file | 258 | — |
The accepted pages, 66,322 of them, were written into the library on 12 September. They carry a label in the page record saying where the text came from and what the agreement score was, so they can be found, measured separately, and replaced if a better reading ever arrives. Showing that label in the reader is the next step.
The books that gained the most are the ones that were nearly done. Max Müller's 1881 translation of Kant's Critique of Pure Reason had 214 pages read and 380 unread; it gained 369 pages at agreement 0.93 and is now complete but for eleven leaves with too little text to take. The general index to the Sacred Books of the East gained 118 pages and is now searchable end to end. A Report upon the Herculaneum Manuscripts (1811), Facts and Documents Illustrative of the History, Doctrine and Rites of the Ancient Albigenses (1832), the first volume of Blavatsky's Secret Doctrine, and Evans-Wentz's Fairy-Faith in Celtic Countries each gained between forty and seventy pages.
The rejects that were our fault
The first pass rejected far more than it should have. 292 books, a fifth of everything scored, sat in a narrow band of agreement between 0.10 and 0.20. That is too low to be bad reading: a reader that garbles every fifth word still scores around 0.8. A cluster that large and that uniform is nearly always the instrument, not the books.
It was. On those items the Archive's text file starts one leaf later than our page numbering. Compare each of our pages with the Archive's block for the leaf before, and Jung's Studies in Word-Association goes from 0.15 to 0.94 across 592 pages; Collectanea Chemica goes from 0.16 to 0.91. The reader had been fine all along. The test now tries each book at several offsets, takes the one most pages agree on, and refuses any book whose alignment drifts from page to page. Re-scored that way, 170 of the 292 came back as accepts, another 18,614 pages, and one book was refused because its alignment drifted, which is the refusal working.
That last paragraph is wrong, and it took two days and a hand check to find out. The Archive's text file and its page images skip the same leaves, so the two sequences never disagree and the correct offset is always zero. What our search had actually found was that the pages we were comparing against (our own earlier transcriptions) had been made from scans that were one leaf out, a fault in our image importer dating from last year. The offset search was not correcting the Archive. It was quietly matching the Archive's correct text to our incorrect scans, and every book it “rescued” was a book it should have refused. A calibration fitted against a corrupted reference does not look like a failure. It looks like a rescue. That is the part worth remembering.
The two long-s books in the same cluster went from 0.15 to 0.77 and 0.69, and stayed rejected. That is the test working: the alignment was wrong and the reading was also not good enough.
Can a cheap model fix the rest?
The obvious next idea: the rejected text is mostly right and wrong in systematic ways, so hand it to a small language model, tell it to fix the long s and the broken words, and skip the image entirely. Text in, text out, no picture to upload. Surely that is cheaper than reading the page again.
We tried it on sixty pages from ten books across the whole range, from a 1917 journal that already agreed at 0.92 down to the 1662 English Van Helmont on this page's header, with the cheapest current Gemini model and its reasoning switched off. Agreement with our image reading, before and after cleanup:
| Books | Before | After cleanup |
|---|---|---|
| Journal proceedings, 1912 and 1917 | 0.92 | 0.94 |
| Illustrated magazine, 1880s, text pages | 0.90 | 0.94 |
| Same magazine, title and index pages | 0.61 | 0.62 |
| Van Helmont, 1662, long s throughout | 0.67 | 0.76 |
| Aristotle and Faber, 1801 and 1803, long s | 0.70 | 0.78 |
It helps, a little, everywhere. It does not get any rejected book over the bar. And two other things came out of those sixty pages that matter more than the averages.
First, the price. The cleanup cost 0.13 cents a page. Reading the image with the same model in batch costs 0.15. The saving is not a tenth of the cost; it is a seventh, because what you pay for is the words coming out, and a cleaned page has as many words as a read one. The picture going in was never the expensive part.
Second, what the model does when it cannot read something. On the Van Helmont title page there is an ink blot over the word “Toparch” in “Toparch or Governor, in Merode, Royenborch, Oorschot, Pellines”. Our image reader transcribed the line and flagged the blot. The Archive's reader produced a string of symbols. The cleanup model, given only those symbols, wrote “Lord of Merode, Royenborch, Oorschot, Pellines”. Fluent, plausible, and not on the page.
On the index page of an 1888 magazine volume, the Archive had mangled every author's name into fragments. The cleanup model restored them: John G. Nicolay for the article on Lincoln and secession, M. G. van Rensselaer for the one on Mary Magdalene, William L. Greene for the postal service. The page says R. D. Mussey, H. W. Compton and Thomas L. Greene. Of eight names it supplied, one was right. The text it produced was twice as long as the text it was given, and every added word was a guess dressed as a reading.
That is the failure we built this whole test to avoid. A conventional reader cannot invent; a language model with no image in front of it can do nothing else where the input is broken. So the answer is no. Where the Archive's reading fails, we read the page again, from the picture, and we pay the seventh of a cent.
What it adds up to
For English, the free share is worth a few hundred dollars of reading and a great deal of waiting: the accepted pages arrive today rather than when the queue reaches them. The 452 books with no reference pages are the larger opportunity. Reading eight pages of each, enough to run the test, costs about five dollars in total, and would let the same free reading fill on the order of a hundred thousand pages more.
For Latin, the same test will now run shelf by shelf. That July comparison of the Archive's reading against ours, by script and century, says what to expect: nineteenth-century French agreed at 82 percent, nineteenth-century Latin at 68, German in Fraktur at 59, and everything printed before 1700 at half or less. Most early books will be refused, and that is the point. The test costs nothing to run and the refusals are what makes the acceptances worth having.
The first shelves are in as this goes out, and they say the same thing. French: a quarter of the scorable books clear the bar. German: one in eight. Latin, from the first six hundred of 7,684 books: one in twelve. Where Latin fails it fails honestly. On a page of an 1856 double-column edition of Suárez, the Archive reads “qua non rcpelemnsea, qiire ibi dictasunt” and splices the two columns together; our reader has “qua non repetemus ea, quæ ibi dicta sunt”. A conventional reader trained on modern English does not know Latin, does not know the long s, and does not know what a column is. The books it can read for us are the ones that look like the books it was built for.
The principle underneath is one we keep returning to. A reading is not trustworthy because of who made it; it is trustworthy because something independent agrees with it. The Archive gave us a second witness for five and a half million pages. We are using it as a witness, not as an oracle.
Books to check
These are the books that gained the most pages from the Archive's reading in the first pass, with the agreement score the test measured for each. If you read one and find a page the reading got wrong, the feedback button on the page reaches us.
Method and code: the test is scripts/import/ia-ocr-ingest.mjs in the public repository; the cleanup experiment and its sixty page-triplets are recorded there too. All counts are from 12 September 2026 and will drift.
