Characterizing Multi-Host CXL 2.0: Latency, Bandwidth, and Synchronization Costs

Compute Express Link (CXL) is an interconnect that extends the traditional memory hierarchy beyond a single machine. It enables disaggregated memory architecture, where memory can be pooled and shared across multiple hosts effectively. Previous systems and approaches were limited to CXL 1.0 which is restricted to intra-host scaling of memory resources with no out of the box support for sharing across multiple hosts. We evaluate a commercially available CXL 2.0 memory switching platform for multi-host shared memory. While prior work has largely focused on emulation, FPGA-based or non-available prototypes, we investigate the real-world performance and system implications of deploying CXL as shared memory across multiple hosts on a commercially available setup. Our evaluation reveals fundamental trade-offs: Access to shared CXL memory exhibits up to 4x higher latency and significantly lower bandwidth compared to local DRAM, but enables flexible memory pooling with fair sharing across hosts. More importantly, we show that the lack of hardware cache coherence in CXL 2.0 introduces non-trivial challenges, requiring carefully chosen instruction patterns.