Does Peer Review Need to Change? A Panel on Reporting Standards and Checklists in the Age of AI

Many scientific fields of study use formally established reporting standards to foster research and experimental design, transparency, replicability, peer review, and student training. Examples include CONSORT in medicine, the What Works Clearinghouse in education, JARS in psychology. Such standards yield agreement on study reporting and evaluation, even if using different methodologies. CHI has not adopted reporting standards. Like other fields, CHI has seen an increased number of low-quality submissions and reviews fueled by AI. This panel’s objective is to discuss advantages and barriers of adopting reporting standards for SIGCHI. Panelists include representatives with significant experience creating, adopting and operationalizing reporting standards in adjacent fields: software engineering, CS education, and Programming Languages. The panel will include an overview of the history of reporting standards, a live demo of a standards-based peer review system, discussions of opportunities, challenges, limitations for SIGCHI reporting standards, and an interactive discussion between attendees and panelists.