MXM-RVV: Multicore and Multithreaded RISC-V Vectors

Vector processors exploit data-level parallelism to accelerate compute-intensive workloads, but their execution units can be underutilized due to short vector lengths, memory latency, and pipeline stalls caused by read-after-write (RAW) hazards. This work improves utilization by adding programmer-visible parallel instances of the vector engine to the scalar engine. Two types of parallel instances are created: the first is done by multithreading, where additional vector programmer state is added to a vector core, and the second is done by multicore, where additional copies of the multithreaded vector core are added to the scalar core. To the programmer, these all appear to be hardware microthreads which execute concurrently in much the same way as out-of-order instructions execute in parallel. However, the programmer explicitly selects a hardware microthread before issuing it any vector instructions; these instructions are absorbed into a queue and run asynchronously even when the programmer switches to another microthread. Multithreading hides RAW hazards, allowing the vector instruction scheduler to fill a memory stall or pipeline bubble by executing instructions from the queue of a different microthread. Multicore provides a way to improve the performance of microthreads by offering additional vector cores to execute them. Experiments with select microkernels demonstrate up to 1.36 × speedup with multithreading on one vector core, and up to 2.66 × speedup with multithreading on two vector cores.1