Reasoning About Human Values in GitHub Issues: What Can a Large Language Model Reveal?

The recent U.S. Senate hearing on child safety failures in major social media platforms, gender bias in Amazon’s recruitment algorithms, and privacy lawsuits against Google over vague location tracking illustrate how software can neglect human values with societal and economic consequences. Addressing such oversights requires understanding how design choices impact values. GitHub Issues provide a key venue where these choices are documented: through feature requests, improvements, and bug reports, they capture both technical complexities and the tradeoffs reflecting stakeholders’ beliefs. They thus offer insights into how values such as privacy, fairness, and inclusivity influence and are shaped by software design. Yet inferring value alignments, often implied rather than explicit, demands complex reasoning beyond keyword searches and cannot be done manually at scale. In this paper, we use a Large Language Model (LLM) to infer (detect and explain) the alignment of GitHub Issues with human values across three open-source projects, and evaluate the accuracy of the findings through human evaluation.