Archaeological Analysis of Legacy Data in Public Databases: From Unstructured Files to Curated Corpora for NLP and Text Recognition

Many historical public databases and archives exist in formats (microfilm-derived PDFs, unstructured HTML, flat lists) that no longer provide information directly usable as datasets for NLP or for modern text recognition and extraction techniques. This paper presents a methodology of archaeological analysis of such data bodies: extraction, normalisation, textual quality canon, and systematic characterisation, with the aim of turning them into structured, curated, and traceable corpora. It is illustrated with a case study (618 records from the Condon Report and Project Blue Book, 1947–1969) integrating HTML transcrip- tions, lists, an external API, and NARA archive PDFs. The ETL pipeline, relational schema, quality canon, and characterisation results are detailed. It is discussed how this approach revives legacy datasets as corpora suitable for NLP, text recognition, and quantitative analysis.