T-REX: A 68-to-567μs/Token 0.41-to-3.95μJ/Token Transformer Accelerator with Reduced External Memory Access and Enhanced Hardware Utilization in 16nm FinFET

作者：Seunghyun Moon, Mao Li, Gregory K. Chen, Phil Knag, Ram Krishnamurthy, Mingoo Seok · 年份：2025 · DOI：10.1109/isscc49661.2025.10904793 · 被引用次数：19 · 研究领域：Semiconductor materials and devices、Advancements in Semiconductor Devices and Circuit Design、VLSI and Analog Circuit Testing

Transformer, a recent mainstream model in deep learning, has revolutionized a wide range of AI applications, which motivates a surge in research to develop energy-efficient hardware accelerators. Most prior efforts have concentrated on enhancing on-chip computational energy efficiency through several strategies such as encoder-only models [1]–[7], quantization/sparsity [8]–[18], and layer pruning [19]. However, recent works [20], [21] show that external memory access (EMA) dominates total energy consumption. Our analysis based on [22], [23] also indicates that EMA accounts for up to 81% of the total energy usage (Fig. 23.1.1). Additionally, we recognize that the prior works exhibit low hardware utilization, as low as 9% in [4], which negatively impacts latency performance.