Feasible Solutions for the Implementation and Analysis of Commercial High-Performance Interconnection Network Architectures

Interconnection networks play a fundamental role in High-Performance Computing (HPC) and Data Center (DC) systems. These systems consist of thousands of compute and storage nodes that work cooperatively and jointly, acting as a single unit. Therefore, they require an interconnection network that offers high bandwidth and low latency to avoid becoming the system’s bottleneck. Within this context, this thesis focuses specifically on networks that utilize Remote Direct Memory Access (RDMA) technology. Initially implemented in HPC systems, the adoption of RDMA has accelerated significantly in DCs. The increase in services that handle large volumes of data, such as the development of Artificial Intelligence (AI) applications, Big Data, and distributed storage, demands the same high-performance characteristics (high bandwidth and low latency) that RDMA offers. Given all the above, network performance depends on several key factors. One of these is the topology, which defines how these nodes are interconnected. New topologies are continuously being proposed to efficiently interconnect a large number of nodes at a low cost. These topologies require efficient routing algorithms to balance the network load. The challenge is that while many of these topologies are often proposed theoretically, they are rarely implemented in a real system due to the technical difficulties involved. However, topology is only one factor among many. The massive amount of data traffic handled by these networks is so high that congestion is a frequent occurrence, degrading the overall system performance. To address this, various Congestion Control (CC) methods exist that seek to eliminate congestion or, failing that, reduce its adverse effects. The problem is that these mechanisms are often very complex to configure effectively. Furthermore, the mechanisms most commonly used in commercial networks were designed for needs and technologies different from those of today. Since the behavior of congestion has changed, it is crucial to update existing mechanisms and propose new solutions capable of addressing these new situations and technologies. Ultimately, simulators play a crucial role in the development of new solutions. They are highly useful in the early stages of development, as they enable design errors to be quickly identified and corrected. They also make it possible to study whether solutions are viable before implementing them on a large scale in real systems. Additionally, they enable the evaluation of new network architectures that are not yet available in real devices, as well as the proposal of new solutions for them. In this thesis, viable solutions are proposed to improve the performance of these interconnection networks. The work spans from the implementation of novel topologies and their routing algorithms in real systems, the enhancement of CC mechanisms through network state monitoring, to the modeling and evaluation of new architectures using an interconnection network simulator.