"Beyond the GPU: Efficiency, Limitations, and Future Trends in FPGA LLM Inference"

"Large-language model (LLM) inference is a rapidly growing class of computer workload, with over 100~GW of compute capacity expected to come online in the next 5 years. The most popular chips used for LLM inference are graphics processing units (GPUs), which are expensive and power-intensive. We present a review of the latest literature on LLM inference using field-programmable gate arrays (FPGAs), which are chips that can be programmatically optimized for inference tasks. We find that current FPGA implementations can achieve similar inference speeds to GPUs while using up to 80 percent less energy. We also find that FPGA performance is primarily constrained by memory bandwidth, limiting them to smaller models. We discuss these limitations, provide analysis of upcoming FPGA products that may overcome these limitations, and analyze the implications of our findings on the future of LLM inference hardware."