Performance of Sequential And Quickest Detection Based on Machine Learning

**Abstract:** With the rapid proliferation of Machine-Learning (ML) and Deep Learning (DL) based decision systems, properly characterizing their often unpredictable performance is a key challenge. In this work we introduce data-driven analogues to the Sequential Probability Ratio Test (SPRT), and Page Test, denoted the Sequential Data-Driven Decision Function (S-D3F) and Quickest Data-Driven Decision Function (Q-D3F), respectively. It is shown that the S-D3F and Q-D3F inherit many key properties from the SPRT and Page test which allow for a reliable extension of their performance prediction strategies to these data-driven approaches via the process of statistical characterization. As a result, one is able to perform sequential analysis and quickest detection using a ML driven detector while attaining near-optimal performance and reliable performance modeling; that is, the performance can be predicted from the training data itself. The reliability of these predictions is demonstrated via extensive numerical experimentation, including a challenging low-SNR passive acoustic detection problem with an ambient acoustic dataset injected with a realistic synthetic signal. **Expected Duration of Reproducible Run:** ~2 hours