TLFormer: Scalable Taylor Linear Attention in Transformer for Collaborative Filtering

Graph Neural Networks (GNNs) have become foundational models in recommender systems due to their ability to propagate information over user–item bipartite graphs via neighborhood aggregation. Despite their empirical success, GNNs are inherently constrained by their reliance on local connectivity, which limits their ability to capture global interaction patterns, particularly in large-scale recommendation scenarios characterized by severe data sparsity. To address these challenges, we propose the Taylor Linear attention in Transformer (TLFormer), which enhances recommendation performance by enabling global attention across all user–item pairs while preserving graph structural information. Unlike existing Transformer-based recommendation approaches that focus on local attention patterns, TLFormer introduces a novel linear attention mechanism derived from the first-order Taylor approximation, allowing efficient computation of all-pair interactions. TLFormer integrates spatial topology as positional encoding while maintaining linear complexity, effectively balancing computational efficiency with model expressiveness for large-scale recommendation scenarios. Extensive experiments across multiple datasets demonstrate that TLFormer significantly outperforms state-of-the-art methods, particularly in scenarios with sparse interactions and long-tail distributions.