Semantic-Spatial Subgraph-Based Relational Reasoning for Visual Question Answering

Visual question answering requires a comprehensive understanding of object relationships and attributes within an image. Utilizing Graph Neural Networks (GNNs) or attention mechanisms to explore these relationships provides an intuitive method that integrates all node features, presenting a global view for predicting the answer to the question. These graph-based methods treat each object as an independent region and build connections between nodes. However, full connections in the graph are often redundant, as question-conditioned connections are more important for accurate answer prediction, introducing irrelevant regions as noise in the reasoning process. To this end, this paper explores subgraph-based relational reasoning for visual question answering, which captures object interactions based on semantic-spatial relationships between regions and learns discriminative subgraph-level representations. Specifically, we first propose a novel subgraph construction principle that considers bothspatialandsemanticrelationships of each candidate region. These subgraphs provide a basis to distinguish similar objects while maintaining discriminative power. Subsequently, to further eliminate the noise from irrelevant nodes and propagate vital messages for accurate answer prediction, an inter-subgraph relation learning (IRL) module is proposed to adaptively prune inter-subgraph connections according to the constructed subgraphs. Finally, to explicitly employ significant representations for reasoning, an intra-subgraph message aggregation (IMA) module is proposed to merge the region representations to infer the answer. Extensive experiments conducted on three benchmarks validate the effectiveness of our proposed method, and visualization results further demonstrate its interpretability.