Large Language Models Should Be the New Gold Standard for Orthopaedic Surgical Registry Curation, but Likely Will Not Be without Clearer Direction

In the past decade, registry-based research in arthroplasty has increased at an exceptional rate, around 28% annually compared with 8% in the general arthroplasty literature1. In parallel, the largest hip and knee registry to date, the American Joint Replacement Registry (AJRR), now boasts 4.6 million procedures, with a 19% increase in reporting from 20242. These trends reflect the field’s increasing reliance on large-scale data to evaluate clinical outcomes or guide policy decisions. This growth carries promises but also exposes a crucial vulnerability. Even as registries expand in size and ambition, the processes used to curate them have largely remained unchanged. Anyone involved with registry development recognizes the bottleneck immediately: hours spent slogging away through charts, keyword searching, abstracting information from complex operative notes, or decrypting niche acronyms to fit into rigid database fields. Until recently, this has been an accepted reality for clinical research. Budding research over the past several years in natural language processing (NLP) has shown its promise for automating data extraction from free text3. Briefly, NLP algorithms are trained with rule-based methods that identify patterns or phrases in bodies of text. However, the application of NLP algorithms in data extraction from postoperative notes, although sufficient for simple categorization tasks, encounters difficulty with linguistically variable, context-dependent information. Operative notes in particular are more like clinical narratives dominated by negation, conditional reasoning, or provider-specific shorthand. Consider the following hypothetical note: “The initial plan was cementless fixation. Intraoperative assessment revealed osteoporotic bone with inadequate press-fit stability. A cemented femoral component was implanted after conversion.” In a rule-based NLP algorithm, the inclusion of both keywords “cementless” and “cemented” may introduce ambiguity to the phrase search, even though there is a clear sequential logic that makes the category of “cemented” readily apparent to humans. In cases of negation (i.e., “a tourniquet was applied but not inflated”), NLP algorithms may have difficulty in interpreting the meaning. However, it is precisely in these cases involving reasoning that artificial intelligence large language models (LLMs) excel. Yang et al. compared LLMs with standard NLP techniques and found that LLMs were superior overall when assessing total hip arthroplasty postoperative notes for the surgical approach, bearing surface, and fixation technique. They credit this superiority to the flexibility in phrasing and contextual knowledge of LLMs that can infer missing information within an operative note, such as the bearing surface. To individuals who are familiar with the rapidly advancing field of generative artificial intelligence, this increase in accuracy across metrics and outcomes is hardly surprising. However, more notable is the framework for future discussions on LLM safety, speed, cost, and implementation that Yang et al. introduce. Prior studies have compared performance between local open-source LLM models and those that are hosted by a third party, but the implications of speed and compute have yet to be aptly discussed4. Yang et al. found that their local LLMs took approximately 1 minute for each note, whereas the NLPs took less than 1 second. At a glance, this does not seem like a difficult tradeoff. Waiting a few seconds for processes that can run in the background is almost certainly worth the extra time if it reliably gives us more clinically accurate, relevant data. Furthermore, the promise of LLM agents orchestrating even more complex workflows using implant databases, as suggested by Yang et al., provides a compelling vision for future registry automation. However, the practical reality of hospital privacy regulations and information technology infrastructures complicates these decisions. Broadly speaking, institutions currently face 2 paths for deploying LLMs: local deployment of open-source models or models that are hosted externally by third parties using a Health Insurance Portability and Accountability Act (HIPAA)-compliant Business Associate Agreement. Each of these approaches comes with distinct advantages. Local models provide the most control over the data, but demand substantial upfront investment in the computational infrastructure, ongoing maintenance, and specialized expertise to integrate with the legacy systems. In contrast, hosted models often provide superior performance and lower maintenance labor, but create more reliance on third parties and, thus, more unpredictability. Although Yang et al. opted to work within the constraints of their institution using local models, this option may not be tenable for institutions lacking technical staff or $10,000 graphics processing units (GPUs). If other institutions have different models or architectures operating their data curation pipelines, it is likely that there will be variation in the quality of extraction. In response to these challenges, open-source frameworks have been developed specifically to address and streamline many of these technical hurdles5. It may also be beneficial for organizing registry bodies such as the AJRR to provide centralized recommendations. Whether these approaches can meaningfully lower the barrier to adoption has yet to be seen. Clearly, LLMs possess a unique potential for data curation. They excel at human-like reasoning and can provide the nuance and granularity necessary for arthroplasty studies that go beyond traditional methods, as shown by Yang et al. The promise is there, but until stronger standards, infrastructure, and coordination exist, it remains too premature to crown LLMs the new de facto standard for orthopaedic registry curation.