What Do You See in Vehicle? Comprehensive Vision Solution for In-Vehicle Gaze Estimation

作者：Yihua Cheng, Yaning Zhu, Zongji Wang, Hongquan Hao, Yongwei Liu, Shiqing Cheng, Xi Wang, Hyung Jin Chang · 年份：2024 · DOI：10.1109/cvpr52733.2024.00154 · 被引用次数：31 · 研究领域：Gaze Tracking and Assistive Technology、Visual Attention and Saliency Detection、Retinal Imaging and Analysis

Driver's eye gaze holds a wealth of cognitive and intentional cues crucial for intelligent vehicles. Despite its sig-nificance, research on in-vehicle gaze estimation remains limited due to the scarcity of comprehensive and well-annotated datasets in real driving scenarios. In this pa-per, we present three novel elements to advance in-vehicle gaze research. Firstly, we introduce IVGaze, a pioneering dataset capturing in-vehicle gaze, collected from 125 sub-jects and covering a large range of gaze and head poses within vehicles. In this dataset, we propose a new vision-based solution for in-vehicle gaze collection, introducing a refined gaze target calibration method to tackle annotation challenges. Second, our research focuses on in-vehicle gaze estimation leveraging the IvGaze. In-vehicle face images often suffer from low resolution, prompting our in-troduction of a gaze pyramid transformer that leverages transformer-based multilevel features integration. Expanding upon this, we introduce the dual-stream gaze pyramid transformer (GazeDPTR). Employing perspective transfor-mation, we rotate virtual cameras to normalize images, uti-lizing camera pose to merge normalized and original images for accurate gaze estimation. GazeDPTR shows state-of-the-art performance on the IVGaze dataset. Thirdly, we explore a novel strategy for gaze zone classification by extending the GazeDPTR. A foundational tri-plane and project gaze onto these planes are newly defined. Leveraging both position...