New Metrics for Disambiguating Feature Overlap and Catastrophic Forgetting in Incremental Learning Contexts (Student Abstract)

Catastrophic forgetting remains a central challenge in lifelong learning, where newly acquired knowledge interferes with previously learned tasks, degrading performance over time. Mitigation strategies such as rehearsal and regularization have been proposed, but both introduce limitations, either by retaining old data or by constraining model updates in ways that may impair learning. Complicating matters, recent findings show that feature-space overlap between tasks can produce similar performance drops even in models that memorize data, making it difficult to distinguish true forgetting from representational interference. Current accuracy-based metrics fail to disentangle these effects, undermining diagnostic clarity. In this paper, we introduce the Overlap Index, an incremental cluster validity index adapted from the inter-cluster component of the iCONN index, which quantifies overlap between feature representations in input or latent space. We then introduce the Overshadowing and Forgetting Index, an online meta-metric that leverages the Overlap Index to attribute performance degradation to catastrophic forgetting, class overshadowing, or both. Our experimental results demonstrate that these tools enable more precise online and batch-mode evaluation of continual learning systems, paving the way for more targeted mitigation strategies.