Data Annotation Errors in AI-Enabled Perception System Development: A Multi-Organisation Case Study in the Automotive Domain

Data annotation is a critical yet error-prone activity in developing AI-enabled perception systems (AIePS) for automated driving. Annotation quality directly affects AI model development performance, safety and reliability. Yet, there is little empirically grounded understanding of how annotation errors arise and propagate across the multi-organisational automotive supply chain. This study investigates the types, causes, and effects of data annotation errors through a multi-organisation case study involving six companies (e.g., technology and AI companies that develop software for advanced driver assistance systems and self-driving cars) and four research institutes in Europe and the UK. We conducted 19 semi-structured interviews with 20 experts (≈ 50 hours of transcripts) and applied a six-phase thematic analysis. The resulting data annotation errors taxonomy identifies 18 recurring error types across three major data-quality dimensions: completeness (attribute omission, missing feedback loop, privacy/compliance omission, edge-case omission, selection bias, sensor synchronisation issues), accuracy (wrong class label, bounding-box errors, granularity mismatch, insufficient guidance, bias-driven errors), and consistency (inter-annotator disagreement, ambiguous instructions, lack of purpose knowledge, misaligned hand-offs, limited review and logging, lack of frameworks and standards, cross-modality misalignment). For research rigour and triangulation, we further validated the proposed taxonomy of data annotation errors with industry. Practitioners confirmed the usefulness of the taxonomy for root-cause analysis of data annotation errors, supplier quality reviews, new project onboarding, and optimising data annotation guidelines. Practitioners described it as a “failure-mode catalogue” comparable to FMEA (Failure Mode and Effects Analysis). By framing annotation quality as an AI-enabled system development lifecycle and supply-chain concern, this work advances SE4AI by providing a shared vocabulary, diagnostic checklist, and actionable guidance for trustworthy AIePS development.