OCR Pipeline for Historical Print Periodicals using Mistral AI

A Jupyter-based batch OCR pipeline for processing scanned documents (PDF) and images (PNG, JPEG) using the Mistral OCR API, pinned to OCR 4 (mistral-ocr-4-0) for reproducible corpus processing. Designed for historical print periodicals; handles printed text, forms, tables, and (with limitations) handwritten content (modern scripts; historical German scripts like Kurrent remain challenging - seperate pipeline is planned). Features Structured text extraction (headings, paragraphs, footnotes) in Markdown format API confidence scores per page (default) or per word (optional) as primary quality signal, with per-page details in a JSON sidecar Optional layout-block extraction: typed blocks (title, text, caption, table, ...) with bounding boxes Pinned model version — a corpus is never processed with silently changing model versions Automatic PDF splitting for large documents (>50 MB or >1000 pages) SQLite checkpoint system for resumable processing Zero Data Retention by default for Mistral OCR Multilingual OCR support (Mistral OCR 4 covers 170 languages) Quick Setup Open the repository in a Jupyter-compatible environment (e.g., VSCode with Jupyter extension) Get a Mistral API key from console.mistral.ai Rename .env.template to .env and add your API key Place your documents (PDF) or images (PNG, JPEG) in data/input/ Run notebooks/ocr_pipeline.ipynb Output Markdown files with preserved document structure Plain text files for downstream processing JSON metadata (model version, confidence scores, page count) plus optional sidecars for per-page confidence details and layout blocks Cost Experiment accounts available without cost via console.mistral.ai $4 USD per 1000 pages (Mistral OCR 4 API, as of June 2026) Requires mistralai>=2.7.0 (Python SDK v2)