Scale Invariant Neural Networks Enhanced by a Covariance-Based Dynamic Filter

Neural networks frequently waste capacity on learning transformations (like scale changes in time series) that are irrelevant to the main classification or identification task. Current invariance approaches, whether architecture- or data-centric, remain computationally expensive, need careful tuning, and generalize poorly; moreover, none achieves online scale invariance without prior knowledge of the amplitude. To address this gap, it is here proposed a general time-series framework that imposes online scale invariance via an explicit, non-learned functional transformation: a covariance-based filter. This filter is first benchmarked against standard offline normalizers using real seismic signals, demonstrating preservation of waveform shape and frequency content (Fréchet distance 1.9 × 10−4;spectral correlation > 0.99). The invariant filter is then used as the initial layer of a Long Short-Term Memory (LSTM) network for frequency estimation. A comparative evaluation with amplitude-varying sinusoidal signals, using the Sum of Squared Errors (SSEs) as the metric, shows notable gains: a conventional 10-unit LSTM with data augmentation needed 50, 000 training epochs to reach an SSE of 686 rad2/s2, while the invariant LSTM achieved the same error with only 4 units in 30, 766 epochs. The contributions of this work are threefold: theoretical, by proposing a deterministic, non-learned invariant function; methodological, by designing a generalizable preprocessing filter; and applied, by showing faster convergence and lower model complexity for LSTM-based time-series analysis.