Almost every page in this library was read and translated by one commercial model, Google’s Gemini. That gives us about 6.9 million transcribed pages and 5.2 million translated ones. Can we use them to train smaller, open models of our own, which we could run on our own computer, publish, and hand to other libraries? Our answer, after measuring: yes for some of the work, not yet for the rest. Where our evaluations show the text is good, it is usable training data. Where Gemini reads badly, as in early Greek print, copying Gemini cannot do better than Gemini, and we need text that people made. Our first open translation model, trained on our Latin pages, failed its test: it would cost very little to run, but it is clearly less faithful than Gemini.
Two ideas in one sentence each
Distillation means training a small “student” model to copy the output of a large “teacher”; the student is cheaper to run, and at best it is as good as the teacher. LoRA (low-rank adaptation) is a cheap way to do that training: instead of changing the whole model, it trains a small add-on layer on top of an existing open model, in hours rather than weeks.
One distinction decides most of what follows. Training a student on the teacher’s text is ordinary practice; the open OCR model olmOCR and Google’s TranslateGemma were both built partly that way. Grading the student only against the teacher’s text is circular: it measures how well the student imitates, not whether either is right. For grading we need an independent reference, a transcription or translation that a person made.
How good is the text we have?
We do not have to guess. Over the past month we scored Gemini’s readings against published transcriptions of the same pages, one page per book, with the test pages sealed before any engine read them (the method is in How We Measure OCR Quality). The measure is character error rate: the share of letters that differ from the reference.
Gemini’s character error rate on printed pages, by language and period
Hover, tap or tab to a row for the exact figure and whether the text is usable for training.
Greek (114 pages · usable: no): 14.5% of characters differ from a published transcription. Not usable for training: about one character in seven is wrong.
Numbers: Gemini’s readings scored against published transcriptions, one page per book, in the open-engine comparison of October 2026 (#5660).
Source: the open-engine comparison in #5660. Most cells hold fewer than fifty books, which we treat as directional rather than decisive.
Translations were measured two ways. In an audit of 311 served pages, one per book across 15 languages, a separate judge model (Claude, a different family from the translator) checked each English page against its source, and we read 20 of them ourselves. 87 to 89 percent were rated faithful (4 or 5 on a 5-point scale), and 11 to 14 percent carried at least one major defect (#5274). Latin-script languages did best: 93 percent faithful. Greek was at 75 percent, Sanskrit and Tibetan lower still. The second measure has a human reference: for the Suda, the Byzantine lexicon on the page above, scholars at the Suda On Line have translated some 31,000 entries. On 150 entries drawn at random, 77 percent of ours were faithful to the Greek, 21 percent had minor issues and 2 percent major errors (#3884; our scan, p. 88 of Bekker’s edition).
So the text is good training data on the slices where it measures well, and that is most of the Latin, German and English. It cannot teach a model to read what Gemini misreads, and it cannot grade itself. Human corrections in our own pages are still very few, a few dozen. The human-made texts we lean on are published ones: proofread Wikisource pages, EEBO-TCP keyings, the Kanripo Chinese canon, the 84000 Tibetan translations, the Suda On Line.
What we have already tried
| When | What | Result |
|---|---|---|
| Oct 2026 | An open translation model trained on our Latin pages (#5793) | Failed the preset bar: against published human translations of 71 pages it scored 3.32 of 5, Gemini 4.16. Training helped (the untrained model scored 2.75), but it often reverses the sense of a clause |
| Oct 2026 | Kraken, an open line-by-line reader, fine-tuned on Latin print of 1500–1699 (#5730) | Running |
| Oct 2026 | Open page-reading models against Gemini on the same 632 pages (#5660) | olmOCR wins only on English of 1600–1699 (3.6% vs 5.3%), where it reads the long s correctly; on Greek every open model failed badly (48–51% error) |
| Aug 2026 | A Gemini model tuned on 15,332 of our Greek translations, for $42 (#4320) | Preferred on 158 of 200 test pages, but graded against our own translations, so the test was circular. It also learned to copy our editorial notes into its output |
Latin to English on 71 pages: faithfulness to published human translations
Hover, tap or tab to a row for the exact score. Bars are the mean score; whiskers the 95% interval.
Our trained model (Qwen3-8B with LoRA, 18,587 Latin pages): 3.32 of 5 mean fidelity, judged blind, 95% interval 3.17 to 3.48 of 5. It lost to Gemini on 54 of the 71 pages.
Numbers: the test of 4 October 2026 (#5793), 71 Latin pages from books outside the training data, each scored by two blind judges; intervals from the experiment write-up in the repository.
Where training pays
- Reading the scripts Gemini gets wrong. We hold 2.2 million pages of Greek in 5,014 books, and about 340,000 of them have not been read yet. Gemini misreads about one Greek character in seven on our early printed pages. There is published ground truth for Greek print: line transcriptions of the Patrologia Graeca, released on Zenodo, and a reported fine-tune of the open Qwen3-VL-8B model at about 1 percent error. We have not yet tested it ourselves (#4744). The same pattern holds for Syriac, Tibetan, Persian and the Chinese imperial library, each with a published text to align against.
- Page checkers. Small models that sort pages: is this a plate or text, which language and script, is the reading garbled or looping, is the image the right leaf? Here Gemini’s own labels are acceptable training data, because the checker only sorts and a set of pages we read by eye tests it. Run on our own GPU, such a checker costs almost nothing per page, where asking Gemini about every one of 10 million pages would cost tens of thousands of dollars.
- Translation, as a test. Because 93 percent of our Latin-script translations measure as faithful, a student trained on the good ones might match Gemini at a fraction of the price. We tested that on 4 October (#5793): Qwen3-8B, an open model of 8 billion parameters, trained with LoRA on 18,587 filtered Latin pages, then judged blind against published human translations beside Gemini’s, on 71 pages from books it never saw. The bar, set before training, was “no worse than Gemini”. It scored 3.32 of 5 to Gemini’s 4.16 and lost on 54 of the 71 pages. Its typical error is fluent English that reverses the Latin: “not counted among” for numerantur, “the learned” for indocti. Cost is not the obstacle: on our own GPU it would translate for about €0.15 per thousand pages. Quality is. A larger model, cleaner data, or training against human translations are the next levers; we will not retrain a model on its own output, because errors compound that way.
- Search across languages. Our semantic search only finds translated books. Open multilingual models can index the Latin, Greek and Chinese directly; we are testing ready-made ones before training anything (#5729).
What it costs, and what it saves
Gemini Flash-Lite reads a page for about $0.45 per thousand in batch mode and translates one for $0.60 to $1.70 per thousand. An open model of olmOCR’s size, on the GPU server we are setting up, reads for about $0.26 per thousand. Each training run costs $10 to $60 of rented GPU time.
The sums are modest. The 10.5 million pages still waiting to be read would cost Gemini about $4,700 in batch mode; open models would save part of that only where they read at least as well, which so far is one cell. Translating the backlog on our own machine could have saved $6,000 to $17,000 once, but our first open translation model is not good enough yet. The larger gain is reach. Some 24,500 Latin books we hold are not yet public because nobody has read them; with our own models, reading them becomes a matter of machine time rather than a budget decision.
Who else is doing this
Many projects train models on historical texts. Each covers one tradition or one task. We have not found another that holds the page image, the transcription and an English translation, aligned page by page, across Latin, Greek, Hebrew, Arabic, Syriac, Tibetan, Chinese and the European vernaculars.
| Project | What they train or publish |
|---|---|
| Dharmamitra (MITRA) | Open translation models from Sanskrit, Tibetan and Classical Chinese into English, trained on 1.74 million mined parallel sentences |
| Buddhist Digital Resource Center | An open Tibetan OCR app, and open Buddhist datasets for AI drawn from over 30 million scanned pages |
| Monlam AI | Tibetan OCR, translation and speech models |
| Pleias, Common Corpus | An open corpus of public-domain text, a model that corrects bad OCR (OCRonos) and one that flags OCR errors word by word (OCRerrcr) |
| Allen Institute, olmOCR | An open page-reading model, with its training code |
| CATMuS | Open Kraken models for medieval and early-modern hands |
| Harvard Institutional Books | 983,004 public-domain volumes released as an open dataset |
| Transkribus | A paid platform with over 500 handwriting and print models, many trained by its users |
| Dicta | Language models for rabbinic Hebrew, trained on the Sefaria and Dicta libraries |
What we will publish
Whatever we train that works will be released openly with a DOI: line readers to the shared Zenodo and HTR-United collections, larger models and the page-aligned image, transcription and translation set to Hugging Face, each with a note saying where it fails. Some published references carry share-alike licences, and we will check each before release. If you hold a transcription or translation of a text we have scanned, or would like to test a model against our pages, we would like to hear from you.
What this cannot say
The OCR cells are small, and the translation audit used a model as judge, checked by our own reading of 20 pages but not by a second human. Whether an open model can match Gemini on translation is exactly what the running test is for; this note does not assume the answer. The savings above are estimates from measured unit prices, not money saved. We will report the result on #5793 whichever way it goes.
