Event2Cor: Correlation Image Simulator for Dynamic Vision

Event cameras asynchronously detect changes in light intensity with microsecond-level temporal resolution, offering rich motion information under dynamic scenes. However, the sparse and discrete nature of event streams hinder effective feature extraction and reconstruction for vision tasks. To address this, we propose a correlation image simulator that generates compact and structured representation of intensity variation. Our method simulates the output of a three-phase correlation image sensor (CIS), capturing temporal correlation between the incident light intensity and an external reference signal. Depending on whether the event camera supports intensity frame output (i.e., conventional image), we design two distinct simulation methods. (1) Simulator1 based on the pure event stream: by iteratively accumulating events in the temporal dimension, we reconstruct a relative intensity signal and compute its correlation with the reference. (2) Simulator2 based on the event stream and intensity frame: by establishing the mathematical relationship between the intensity frame, event stream, and actual intensity based on event-triggered mechanism, we reconstruct an absolute intensity signal and compute its correlation with the reference. Experimental results on synthetic and real dynamic scenes show that our two simulators provide efficient and informative event-to-image transformation, facilitating compact motion characterization. Furthermore, we apply our simulation results to the image deblurring and temporal super-resolution tasks, achieving significantly better performance than existing widely used event representations.