SORT-AI: Accelerator Runtime Coherence in Heterogeneous AI Inference Infrastructure A Structural Analysis of Cross-Layer Instability in Large-Scale Systems

Read full text Summarise with VIRAT
Authors: Gregor Herbert Wegener
Cited by: 0
References: 52

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Large scale AI inference is shifting from relatively homogeneous accelerator fleets toward structurally heterogeneous infrastructure composed of mixed accelerator generations, disaggregated serving paths, virtualized execution layers, and cross environment placement. This transition introduces a class of instability phenomena that does not arise from component failure in isolation, but from cross layer incoherence between accelerator runtimes, network topology, virtualization boundaries, and migration induced reconfiguration. Classical performance metrics such as utilization, average latency, and tokens per second remain necessary, but they do not capture these structural effects because they observe individual layers rather than the coupled system formed across them. This paper develops a vendor agnostic structural analysis of heterogeneous inference infrastructure and argues that runtime coherence functions as a hidden performance variable that conditions effective capacity, tail latency, and cost efficiency before conventional optimization can succeed. The paper introduces a taxonomy of five instability modes, latency asymmetry drift, memory path incoherence, interconnect induced capacity inaccessibility, virtualization induced control distortion, and migration induced runtime reconfiguration risk. It then maps these modes to four diagnostic domains, accelerator runtime control, structural network scalability, virtualization overhead stability, and structural cloud migration risk. The contribution is analytical rather than prescriptive. It does not propose implementation mechanisms or vendor specific remedies, but provides a research framework for identifying where and why heterogeneous inference systems produce emergent instability under scale.

Fields of Study & Subjects

Parallel Computing and Optimization Techniques Cloud Computing and Resource Management Software System Performance and Reliability

Publication details

Journal / source
Preprints.org
Publication date
20 March 2026
Language
EN
Document type
Preprint