Knowing When to Help: A Framework for Error Detection in Meal Preparation Tasks for Older Adults

Older adults with mild cognitive impairment (MCI) often experience difficulty completing multi-step tasks such as meal preparation. Existing assistive technologies typically provide step-by-step guidance without determining when assistance is actually needed, which risks undermining the autonomy of the user if intervention is not necessary. Thus, we present a framework for detecting moments when a user requires assistance during a meal preparation task. Using a location-based state representation, we classify three error types observed in a real-world study with older adults: visiting irrelevant locations, retrieving incorrect items, and overlooking necessary items. Our method leverages LLMs to interpret each state, identifies when assistance is required, and provides specific suggestions to resume progression. We evaluate the approach on a synthetic dataset with systematically injected errors and a real-world meal preparation sequence of making a banana split. Our results demonstrate that our method achieves F1 scores of 0.80 and 1.00 in real-world data for the two most common error types. These findings highlight the potential for this method to support timely interventions in assistive systems that promote independence in daily living activities.