Predicting Workers’ Compensation Dispute Outcomes with Large Language Models

Workers' compensation insurance is one of the oldest social insurance programs in the United States, predating both Social Security and unemployment insurance. When disputes arise between employees and employers (or their insurance carriers) over benefit entitlements, most states require resolution through administrative boards. In this study, we evaluate whether modern large language models (LLMs) can predict the outcomes of workers' compensation cases more accurately than traditional, domain-specific natural language processing (NLP) techniques. We compare performance under two input scenarios-using only the initial "Issues" filed and using the full "Finding of Facts" narrative of each case, and measure predictive accuracy against actual board decisions. Our results show that, with access to a sufficiently large context window, LLMs match or surpass the performance of specialized NLP pipelines, despite having no task-specific training on workers' compensation data. This finding underscores the practical utility of LLMs for case outcomes for the plaintiff, the employer, and the insurance carrier.