EvR-DETR:Event-RGB Fusion for Lightweight End-to-End Object Detection

Event cameras based on neuromorphic spike signals can provide information about illumination changes,compensating for the performance degradation of traditional RGB cameras in object detection under adverse environments.However,existing methods fusing event cameras with conventional cameras suffer from large model parameters and non-end-to-end training approaches,which restrict the effectiveness of modality fusion.To address this,this paper proposes a lightweight end-to-end object detection framework that integrates event and RGB information through multi-granularity fusion of multi-scale features across different network levels.By implementing lightweight fusion modules with reparameterized convolutions and enabling end-to-end training,the proposed framework enhances the model’s capability to extract complementary information from both modalities,overcoming challenging conditions in autonomous driving.Evaluated on the large-scale PKU-SOD dataset containing vehicular visual data under low-light,high-speed motion blur,and normal illumination scenarios,the proposed method significantly reduces model parameters compared to state-of-the-art multimodal approaches while improving detection accuracy and inference speed,demonstrating superior performance over existing methods.