Human-in-the-loop learning: Making smarter and safer AI decisions

Introduction, This thesis explores the benefit of utilizing Human-in-the-loop (HITL) in the Budgeted UCB algorithm. Addressing this limitation is critical, as the budgeted UCB algorithm lacks mechanisms to incorporate human knowledge. This deficiency often leads to unnecessarily costly explorations and terminal constraint violations, which pose a safety risk and reduce the life cycles of IoT devices, a critical factor in resource sensitive systems. Research Question, The study answers the research question “To what extent does human-in-the-loop impact the total performance of a budgeted UCB algorithm, measured in cumulative regret, in a simulated IoT environment?” Method, A controlled and isolated Python-simulation experiment was conducted for evaluating the impact of Human-in-the-loop. The method involved comparing the budgeted UCB baseline against three HITL variants: oracle with perfect knowledge, a noisy expert that does probabilistic errors and an estimate based variant. These were evaluated within a single simulation run, with dynamically changing constraints to replicate a non-stationary environment. Results, The results demonstrate that the HITL-extension reduced cumulative regret by approximately 34% compared to the standard budgeted UCB algorithm. Discussion, These findings suggest that HITL-extended Budgeted UCB algorithms can be of crucial value for IoT-devices and scenarios where constraint violations are harmful. Further research could explore theoretical results and the impact in real world scenarios to determine if the simulated results could transfer to actual IoT environments.