Convert Once, Consume Many: SDF for Cacheable, Typed Semantic Extraction from Web Pages

Read full text
Authors: Pranab Sarkar
Cited by: 0

Abstract

AI agents and large language models (LLMs) typically consume web pages by fetching HTML and repeatedly performing boilerplate removal, chunking, and semantic extraction at query time. We present Structured Data Format (SDF), an open, schema-validated JSON protocol for publishing pre-extracted, agent-oriented semantic representations of web content. A production crawler processed 6,206 URLs and generated 2,335+ schema-valid SDF documents across 10 parent types and 74 observed type combinations. A fine-tuned 1.5B + 3B pipeline achieves 4.1x latency reduction versus a 14B baseline with 90% exact extraction accuracy. A downstream consumption experiment showed the SDF path achieved 0.739 mean accuracy versus 0.352 raw at 7B (t(29) = 11.890, p < 0.05), with 58.5% latency reduction and 99.2% token reduction from raw HTML.

Fields of Study & Subjects

Web Data Mining and Analysis Topic Modeling Information Retrieval and Search Behavior

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
9 February 2026
Language
EN
Document type
Preprint