Who Fails Where? LLM and Human Error Patterns in Endometriosis Ultrasound Report Extraction

In this study, we evaluate locally deployed large language models (LLMs) for converting unstructured endometriosis transvaginal ultrasound (eTVUS) reports into structured data. Across 49 de-identified reports, we compared three on-premise LLMs (7B/8B and 20B parameters) against expert human extraction using a 185-field schema. The 20B model achieved the highest mean accuracy (86.02%), substantially outperforming the smaller models. Crucially, LLMs and humans exhibited complementary error patterns: the LLM excelled on structured fields (date formatting, measurement decomposition) where humans made protocol errors, while humans demonstrated superior performance on interpretive fields involving negation and clinical terminology. Targeted prompt engineering yielded only marginal gains, indicating that these errors reflect model limitations rather than instruction gaps. These findings support a human-in-the-loop workflow in which the LLM generates structured drafts, automated validation flags rule-verifiable errors, and human review focuses on fields requiring clinical interpretation.