BOPA-Fast: Bipolar Outer Product Attention for Efficient and Interpretable Multimodal Fusion

Multimodal learning benefits from combining complementary signals, yet prevailing fusion strategies have inherent limits. Token-level attention emphasizes sample-tosample alignment and overlooks fine-grained feature interactions, while bilinear pooling typically relies on dimensionality reduction or coarse approximations that discard structure. Even recent attention variants compress features and suppress negative evidence through linear projections and ReLU, weakening polarity-sensitive reasoning. We introduce Bipolar Outer Product Attention (BOPA), a lightweight fusion module that models feature-level dependencies explicitly. BOPA constructs an uncompressed Feature-Level Interaction Matrix (FLIM) via per-sample outer products and converts signed correlations into nonnegative energies with a polarity-aware squared activation, preventing symmetry-induced cancellation and enhancing selective amplification. We further derive BOPA-Fast, a mathematically equivalent reformulation that avoids materializing outer products, reducing complexity from O(nd 2) to O(nd) without loss of fidelity. Across MMHS150K, Hateful Memes, and MS-COCO, BOPA-Fast achieves state-of-the-art accuracy and higher throughput, and yields sparse, aligned with human attribution maps that support transparent diagnosis. By combining exact feature-to-feature modeling with polarity preservation and linear-time computation, BOPA-Fast offers an interpretable and scalable alternative to existing attention and bilinear fusion methods for high-stakes multimodal applications.