Manifold learning for source separation in confusion-limited gravitational-wave data

Abstract The Laser Interferometer Space Antenna (LISA) will observe gravitational-waves in a regime that differs from what ground-based detectors handle. Instead of searching for rare signals buried in loud instrumental noise, LISA’s main challenge is that its data stream contains millions of unresolved galactic binaries. These blend together into a confusion background, and the problem becomes distinguishing sources that genuinely stand out from that sea of signals. In this work we explore whether manifold-learning tools can help with that separation task using a controlled synthetic LISA-like dataset containing instrumental noise and simulated confusion backgrounds. We built a convolutional neural network autoencoder trained solely on the confusion background. The model operates on scalograms produced using the continuous wavelet transform and provides a reconstruction error, denoted ϵ ( x ) , which measures how well the input scalogram lies on the learned background manifold. To incorporate geometric information from the latent space, we introduce an additional anomaly term, δ ⊥ ( ϕ ( x ) , defined as the off-manifold distance obtained from a local tangent-space estimate. The combined anomaly score therefore takes the form s ( x ) = α ⋅ ϵ ( x ) + β ⋅ δ ⊥ ( ϕ ( x ) ) . Performance is evaluated using the area under the receiver operating characteristic curve(ROC-AUC) and average precision (AP). A grid search over α and β in the combined score revealed the best performance near α = 0.5 and β = 2.0 . This indicates that the latent-space geometry provides additional discriminative information beyond the reconstruction error alone. With this combination, the method reaches ROC-AUC = 0.752 and AP = 0.810 ; at the threshold that optimizes the F 1 score, precision = 0.81 and recall = 0.61 . This corresponds to roughly a