RNTuple: Towards First-Class Support for HPC Data Centers

Abstract Future HENP experiments, e.g., at the HL-LHC, will increase the volume of generated data by an order of magnitude compared to LHC Run 1 and Run 2. In order to sustain the expected analysis throughput, ROOT’s RNTuple I/O subsystem has been engineered to overcome the bottlenecks of the TTree I/O subsystem, focusing also on a compact data format, asynchronous and parallel requests, and a layered architecture that allows supporting distributed filesystem-less storage systems, e.g., HPC-oriented object stores. In a previous publication, we introduced and evaluated the RNTuple’s native backend for Intel DAOS. Since its first prototype, we carried out several improvements, both on RNTuple and its DAOS backend, aiming to saturate the physical link, such as support for vector writes and an improved RNTuple-to-DAOS mapping. In parallel, the latest developments allow for better integration between RNTuple and ROOT’s storage-agnostic, declarative interface to write HENP analyses, RDataFrame. In this work, we contribute with the following: ( i ) a redesign of the RNTuple DAOS backend, including a mechanism for efficient population of the object store based on existing data; and ( ii ) an experimental evaluation on a single-node platform, showing a significant increase in the analysis throughput for typical HENP workflows.