Toward Efficient Eye Tracking in AR/VR Devices: A Near-Eye DVS-Based Processor for Real-Time Gaze Estimation

作者：Shihang Tan, Jinqiao Yang, Jiayu Huang, Ziyi Yang, Qinyu Chen, Li‐Rong Zheng, Zhuo Zou · 发表于：IEEE Transactions on Circuits and Systems I Regular Papers · 年份：2025 · DOI：10.1109/tcsi.2025.3553497 · 被引用次数：11 · 研究领域：Gaze Tracking and Assistive Technology、Advanced Computing and Algorithms、EEG and Brain-Computer Interfaces

This paper presents an efficient near-eye dynamic vision sensor (DVS)-based processor for real-time eye tracking in augmented reality/virtual reality (AR/VR) devices. The processor takes advantage of the sparse event data with fine time resolution from the DVS, addressing the need for high frame-rate, low-power, and accurate eye tracking on wearable devices with extended battery life. Exploiting the inherent sparsity of event data, we propose an event-density-based region of interest (ROI) determination method that operates directly on event stream, which requires$47\times $fewer operations than the traditional methods, effectively overcoming the latency problem caused by the heavy computational loads. To eliminate the issue of decreasing accuracy at the edges of the field of view (FoV), we customized and fine-tuned a neural network for gaze estimation, ensuring uniformly distributed sub-degree accuracy. An estimator with a streamlined output mapping strategy and an adaptive window-sliding convolution scheme is implemented for gaze estimation acceleration. The processor is designed and fabricated in UMC 40-nm LP technology with a core area of 2.52 mm2 and performs end-to-end eye tracking exclusively with the raw event stream from DVS, achieving an average accuracy of 0.91° within a$96^{\circ } \times 64^{\circ }$FoV. Operating at 200 MHz, it achieves a dynamic frame rate of up to 1.2 kHz and requires only$12.7~\mu $J of energy per gaze estimation. By integrating the DVS, the ...