Designing explainable cognitive systems and explaining neural networks with plastic dynamical systems

Modern powerful Artificial Intelligence (AI) based on neural networks (NNs) lacks explainability of its decision-making. We propose to aim for an extended explainability, namely, interpretability of the cognitive process underlying intelligence, which includes learning. Designing explainable cognitive machines with prescribed features requires precise technical definitions for cognition and its constituents, such as memories, behavioural code, learning and thinking, and for explainability itself. To describe cognition, we introduce a cohesive mathematical framework comprising a general model and technical definitions. Our definition of cognition is an interpretation in terms of dynamical systems theory of the existing definition from computational neuroscience, which builds upon an earlier idea of cognition as self-organisation of a vector field. This vector field, serving as both the behavioural code and the substrate for memory imprinting, evolves obeying some learning rules. We apply our framework to achieve two main goals. Firstly, we design a “template” explainable cognitive system that overcomes common NN issues, such as catastrophic forgetting, spurious memories, lack of lifelong learning, and limited explainability, while incorporating additional functions seen in humans and animals. Secondly, we reveal learning rules for the vector field of a paradigmatic Hopfield NN with Hebbian learning, and benchmark them against those of the “template” system. We use these rules to partly explain how the NN learns, to interpret connection weights, and to highlight the inherent lack of explainability. Our framework fosters deeper comprehension of the existing AI and paves the way for developing AI with explainable and controllable cognitive functions.