Improving Data Quality with Active Monitoring: Design and Evaluate an Interactive Assessment System in Healthcare

Artificial Intelligence (AI) has significantly impacted healthcare, yet shifting applications from laboratory development to real-world deployment remains challenges: the inevitable scenario change leads to incompatible data inputs and potentially unreliable AI outputs. Human-centered monitoring and intervention during data collection should be reconsidered as a solution. By choosing Parkinson’s wearable assessment as a scenario (existing application PD Assist, which failed in deployment), we conducted a user study and distilled the primary usability issues into three categories: understanding, timing, and disturbance. Accordingly, we redesigned an interactive version of PD Assist+, which contained active monitoring and intervention to ensure users’ behavior meets expectations. Through evaluation involving 38 Parkinson’s participants, our design increased accuracy by 16.9% and participants’ use willingness by 34.8% compared to the original version. Our work derived awareness of data work in AI-based healthcare applications and proposed a pipeline to bridge the gap from development to deployment.