A comparative study on DeepSeek and ChatGPT for bone and soft tissue tumor clinical practice

Background: Artificial intelligence (AI) models are increasingly applied in clinical oncology, yet their comparative utility in specialized domains like bone and soft tissue tumors remains understudied. This study evaluates the diagnostic accuracy and clinical reasoning capabilities of DeepSeek and ChatGPT. Methods: A two-phase evaluation framework was implemented. First, 249 validated clinical questions (191 single-choice, 58 multiple-choice) spanning five domains (diagnosis, imaging, pathology, staging, treatment) were administered, with expert-derived answers serving as ground truth. Second, nine blinded clinicians scored model-generated analyses of a complex sarcoma case across seven clinical dimensions. Statistical analysis employed chi-square tests for accuracy comparisons, Cohen's kappa for inter-rater reliability, and independent t-tests for expert ratings (α = 0.05). Results: DeepSeek outperformed ChatGPT in overall accuracy (74.7% vs 55.4%, p < 0.001), excelling in single-choice questions (86.9% vs 64.9%, p < 0.001) and two key domains: Pathology & Genetics (72.5% vs 40.0%, p = 0.006) and Treatment (71.3% vs 51.2%, p = 0.015). Experts rated DeepSeek higher in imaging interpretation (7.11 vs. 6.00, p = 0.002) and overall case analysis (54.11 vs. 51.56, p = 0.022). Cross-model analysis revealed DeepSeek uniquely answered 60 questions correctly where ChatGPT erred, while both models shared 51 errors. Conclusions: DeepSeek outperforms ChatGPT in diagnostic accuracy and specialized clinical reasoning for bone/soft tissue tumors, particularly in pathology and treatment domains. The significant performance gap (p < 0.001) and 24.1% unique correct responses position DeepSeek as a more reliable diagnostic aid, though shared errors (51 questions) necessitate hybrid AI-clinician workflows.