Construction of highly reliable distributed computing model in multi-source heterogeneous data environment
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
In order to meet the requirements of high reliability and low latency in multi-source heterogeneous data scenarios, this paper proposes a highly reliable distributed computing model based on "data-computing-fault tolerance". Firstly, the heterogeneous data adaptation layer is designed, and the lossless fusion of cross-domain data is realized by dynamic format conversion and ontology alignment mechanism, which reduces the preprocessing delay to 90 ms and improves the throughput to 5.8 GB/s; Secondly, an adaptive fault-tolerant framework based on reinforcement learning (RL) is constructed. Through LSTM fault prediction and Q-learning task migration, the task success rate is still 96% under the 15% node failure rate, and the average recovery time is only 2.1s; Finally, the game theory hybrid resource scheduler is introduced to balance the heterogeneous CPU/GPU/FPGA load with the node utility function, and the Pareto optimization of throughput and delay is realized. The 32-node cluster experiment shows that the end-to-end delay of the model is 4.5× lower than that of Spark, and the delay fluctuation under fault impact is less than 8%, which can support millisecond-level key tasks such as remote surgery and industrial control.
Fields of Study & Subjects
Cloud Computing and Resource Management
Distributed and Parallel Computing Systems
Distributed systems and fault tolerance
Publication details
- Journal / source
- IET conference proceedings.
- Volume
- 2025
- Issue
- 47
- Pages
- 695–700
- Publication date
- 1 March 2026
- Language
- EN