HarmonyCache: Scalable In-Network Cache With Read-Write Separation

In a key-value storage system, a small amount of hot item will account for most of the traffic. Skewed workloads can lead to load imbalance between servers, and some servers that keep hotspots will become system bottlenecks and impact the performance of the entire system. The recent studies have shown that the load imbalance can be eliminated by placing a small, fast cache node in front of the back-end servers, i.e., in-network cache. And the appearance of programmable switches made it possible to place this node on the switch, the place through which the traffic must pass. While existing in-network cache effectively balances loads between servers in large-scale storage systems, they struggle in write-intensive workloads and lose scalability with increasing client numbers due to imbalanced cache nodes. This paper introduces HarmonyCache, a scalable and high-performance in-network cache system that supports write-back methods. HarmonyCache use cache replication and read-write separation mode, only one node in all the cache is responsible for handling write requests, the other node can only handle read requests. To achieve system scalability and minimize cache coherence overhead, HarmonyCache proposes an adaptive cache replication scheme to decide where the cache should be replicated and how many replications should be made. Additionally, according to the requirements of different cache nodes, we design different types of in-network caches using different switch resources and propose a hybrid cache scheme. Our HarmonyCache prototype and extensive experiments demonstrate substantial improvements in key-value store throughput across various access patterns (read/write intensive), achieving a throughput gain of up to 7.6× over state-of-the-art solutions under skewed write-intensive workloads.