MoEA: A Mixture of Experts Accelerator with Direct Token Access and Dynamic Expert Scheduling

Transformer-based large language models (LLMs) have achieved widespread adoption, but their growing model sizes impose substantial computational costs. Mixture-of-Experts (MoE) mechanism alleviates this by sparsely activating a subset of experts for each token. However, it still faces two critical challenges: (1) token rearrangement incurs non-trivial overhead of off-chip data movement, and (2) static execution flow fails to exploit inter-expert token reuse. In this paper, we present MoEA, a specialized accelerator designed to address the inefficiencies in MoE inference. MoEA introduces two key innovations: a Direct Token Access mechanism that leverages a hardware-managed metadata queue to eliminate off-chip token rearrangement, and a Dynamic Expert Scheduler that captures inter-expert token reuse patterns and optimizes expert execution order to maximize token reuse. Evaluations on representative MoE benchmarks show that MoEA reduces off-chip memory access by 12.06% and achieves $259.24 \times, 9.63 \times$ and $1.16 \times$ speedups over CPU, GPU and EdgeMoE accelerator, respectively.