DQS: a low-budget query strategy for enhancing unsupervised data-driven anomaly detection approaches

Abstract What constitutes an unsupervised approach varies across the time series anomaly detection literature, as methods described as unsupervised often rely on labelled data for threshold selection or hyperparameter tuning. These require a labelled data subset, which is often not available in the real world. Though manually and indiscriminately labelling all recorded data would be possible, it would also be associated with a significant cost, and, especially in big data settings, is unlikely to be feasible. To address this, active learning can be used to select a representative subset of samples. Specifically, this work integrates active learning with unsupervised anomaly detection by querying labels for multivariate time series to more accurately refine the threshold selection process. To achieve this, we introduce a novel query strategy called the dissimilarity-based query strategy (DQS). DQS aims to maximise the diversity of queried samples by evaluating the similarity between anomaly scores using dynamic time warping. We assess the detection performance of DQS and explore the impact of mislabelling, a topic that is underexplored in the literature. Our findings indicate that, for the dataset used, DQS performs best in small-budget scenarios, though the other methods appear to be more robust when faced with mislabelling. While no longer truly unsupervised, all query strategies outperform the unsupervised threshold even in the presence of mislabelling. Thus, whenever it is feasible to query an oracle, employing an active learning-based threshold is recommended.