Engineering Truthiness: A Validation Framework for Silver Label Evaluation in Machine Learning

Machine learning increasingly relies on proxy-derived silver labels rather than human-verified gold labels. These proxies are widely used across clinical informatics, information retrieval, NLP, and computer vision, yet no general framework exists for establishing when silver-label evaluation is defensible.We introduce three contributions: SILVER, a six-dimension standard for documenting proxy labels; Silver Evaluation Constraints (SEC), which prevent evaluation circularity; and the Silver Risk Index (SRI), which quantifies ranking stability under proxy disagreement. We define engineering truthiness as the condition arising when silver labels are used without such governance.Five empirical studies demonstrate the framework: synthetic noise simulation shows ranking instability is regime-dependent; IMDB/Snorkel weak supervision shows 0% ranking flip despite massive calibration gap; clinical trial matching (5.27M pairs) reveals sharp threshold sensitivity and subgroup fairness disparities; CIFAR-10 analysis demonstrates error-structure auditing; and external validation against TREC 2021 expert gold judgments yields SRI = 0.00, confirming that silver-based model selection agrees with independent gold evaluation.