Enhancing Anomaly Alert Prioritization Through Calibrated Standard Deviation Uncertainty Estimation With an Ensemble of Auto-Encoders

Deep auto-encoders (AEs) are widely employed deep learning methods in the field of anomaly detection across diverse domains (e.g., cybersecurity analysts managing large volumes of alerts, or medical practitioners monitoring irregular patient signals). In such contexts, practitioners often face challenges of scale and limited processing resources. To cope, strategies such as false positive reduction, human-in-the-loop review, and alert prioritization are commonly adopted. This paper explores the integration of uncertainty quantification (UQ) methods into alert prioritization for anomaly detection using ensembles of AEs. UQ models highlight doubtful classification decisions, enabling analysts to address the most certain alerts first, since higher certainty typically correlates with greater accuracy. Our study reveals a nuanced issue where applying UQ to ensembles of AEs can produce skewed distributions of large reconstruction errors (errors exceeding a pre-defined threshold), which may falsely suggest high uncertainty when standard deviation is used as the metric. Conventionally, a high standard deviation indicates high uncertainty. However, contrary to intuition, large reconstruction errors often reflect AE is strongly confident that an input is anomalous—not uncertainty about it. Moreover, ensembles of AEs generate reconstruction errors with varying ranges, complicating interpretation. To address this, we propose an extension that calibrates the standard deviation distribution of uncertainties, mitigating erroneous prioritization. Evaluation on 10 benchmark datasets demonstrates that our calibration approach improves the effectiveness of UQ methods in prioritizing alerts, while maintaining favorable trade-offs across other key performance metrics.