Referring Multi-Object Tracking With Conflict-Free Decoder Learning for Intelligent Visual Retrieval in Consumer Electronics

Recently, referring multi-object tracking (RMOT) has shown great potential for intelligent visual information retrieval in consumer electronics applications, such as smart home systems and autonomous driving.ecently, referring multi-object tracking (RMOT) has shown great potential for intelligent visual information retrieval in consumer electronics applications, such as smart home systems and autonomous driving. However, existing transformer-based RMOT methods often suffer from suboptimal performance due to two key conflicts during decoder training, namely jointly learning non-consistent optimization objectives for cross-modal fusion, object detection, and data association tasks, and the opposing effects between the self-attention and cross-attention layers on the queries of the decoder, which significantly hinders the precision and quality of consumer electronics applications. To deal with these obstacles, we propose Referring Multi-Object Tracking with Conflict-Free Decoder Learning for intelligent visual retrieval across various consumer applications, denoted as CoFr-Track. Unlike existing models, we leverage an external object detector to generate high-quality proposals as anchors to reduce the burden of the decoder in object detection. Then, we introduce a novel Tucker bilinear cross-modal encoder to extract enhanced visual and text embeddings, improving cross-modal fusion efficiency. Additionally, our grounding fusion decoder employs separate cross-attention layers for accurate data association and reference matching, along with an attention-divided training strategy to resolve learning conflicts. Comparisons with state-of-the-art methods demonstrate the superiority of the proposed method, while extensive ablation studies and analyses validate the effectiveness of its key modules. It effectively promotes the performance of intelligent information retrieval in consumer electronic applications, and significantly enhances the user experience and product intelligence.