GS-Net: Heterogeneous Vehicle Data Reuse via Generalizable Plug-and-Play 3DGS Module

Read full text
Authors: Yichen Zhang, Zihan Wang, Jiali Han, Peilin Li, Jiaxun Zhang, Jian‐Qiang Wang, Lei He, Keqiang Li
Cited by: 0
References: 29

Abstract

End-to-end autonomous driving is increasingly data-driven, yet data reuse across vehicles remains limited. Each new vehicle often requires additional data collection and retraining because camera translation, orientation, and field of view differ across sensor layouts. Cross-sensor view synthesis offers a promising route for cross-platform data reuse by synthesizing images under novel sensor configurations from existing sensor data. To realize this goal, we propose GS-Net, a lightweight plug-and-play module that aggregates local geometric context from sparse Structure-from-Motion (SfM) point clouds and expands each point into multiple dense Gaussian primitives in a single forward pass, learning a cross-scene generalizable initialization for standard 3DGS that improves rendering quality for both interpolated views along the original sensor trajectories and extrapolated views at new sensor positions. In such settings, target camera viewpoints often lie beyond the convex hull of training cameras, whereas existing methods and benchmarks predominantly evaluate in-hull interpolation, leaving the extrapolation regime underexplored. To enable quantitative evaluation under this setting, we introduce CARLA-NVS, the first benchmark explicitly designed for cross-sensor view synthesis. Unlike existing autonomous driving datasets that typically employ no more than eight cameras in fixed configurations, CARLA-NVS features 12 cameras uniformly distributed at 30-degree azimuth intervals, enabling controlled evaluation of both interpolated and extrapolated viewpoints. Experiments on CARLA-NVS show that GS-Net improves rendering quality by 2.08 dB PSNR on interpolated views and 1.86 dB on extrapolated views over standard 3DGS, while achieving a 50x faster initialization compared to MVS-based densification. These results offer a practical step toward scalable cross-vehicle data reuse in autonomous driving.

Fields of Study & Subjects

Industrial Vision Systems and Defect Detection Image Processing Techniques and Applications Advanced Data Compression Techniques

Publication details

Journal / source
IEEE Robotics and Automation Letters
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Volume
11
Issue
11
Pages
12624–12631
Publication date
7 September 2026
Language
EN