Falcon: Event-Driven Hybrid Object Tracking SoC With Bit-Column-Similarity-Based MRAM PIM and Heterogeneous Read-Only-Once Attention Flow

Hybrid frame-event cameras offer high spatial fidelity and & micro;s-level temporal response, making them well suited for low-power, low-latency object tracking under motion blur and sparse activity. This work presents an event-driven vision SoC that integrates heterogeneous MRAM-based processing-in-memory (PIM), near-memory-computing (NMC), lookup table (LUT), and single-instruction multiple-data (SIMD) compute units for end-to-end real-time hybrid tracking. First, the CNN feature encoder is mapped onto a bit-column-similarity-optimized MRAM PIM with activation skipping and weight compression, achieving 2.44 & times; energy efficiency and 61.9% latency reduction. Second, attention-based tracking is executed using a read-once Q/K/V dataflow combining column-priority NMC generation, mixed-product SIMD execution, and LUT-accelerated SoftMax, reducing buffer reads by 46% and attention latency by 30%. Third, a two-stage event-driven activation pipeline selectively enables computation only upon motion, cutting system power by 63.5% and overall latency by 68.8%. Fabricated in 28-nm FDSOI (12.96 mm2), the SoC delivers macro-level 750.18 TOPS/W for 1b-1b-3b MAC operations at 0.7 V, 50 MHz, and system-level 16.34 TOPS/W for INT8 CNN + FP8 attention at 1.15 V, 200 MHz, enabling real-time 50 frames/p tracking at 196.5-mW power consumption.