Criticality-Aware Gen-AI Inference via Dynamic Step Control in Resource-Efficient Vehicular Computing

Integrating Generative AI (Gen-AI) into vehicular computing can significantly enhance road safety and driving experience. The performance of Gen-AI inference in vehicular computing is highly sensitive to the number of inference steps, where even minor adjustments can disrupt the balance between delay and inference quality. This raises a critical question: how can inference steps be optimally determined for diverse tasks, given the inherent delay-quality trade-offs? Existing approaches fail to offer an optimal solution due to the lack of flexible inference services and efficient memory bandwidth allocation strategies. To address these challenges, we develop aCriticality-awareResource-efficientInference (CARIN) framework, where inference steps are dynamically adjusted to balance delay and quality for multi-criticality tasks. Leveraging an accurate inference model, CARIN fully exploits in-vehicle resources to accelerate both parameter loading and task computing during inference. The joint optimization problem of step control, memory bandwidth allocation, and compute resource scheduling is formulated as a mixed-integer nonlinear programming (MINLP) problem, and solved by a novel learning-to-optimize (L2O) algorithm efficiently. Experimental results demonstrate that, for high-criticality tasks, the proposed approach achieves$28.4\%$latency reduction and$36\%$quality improvement over baseline methods.