Roads Digital Twin: Predictive Situational Awareness Using 360$^{\circ }$ Video Streaming and Graph Neural Networks

作者：Sotirios Messinis, Oussama El Marai, Nicholas Ε. Protonotarios, Tarik Taleb, Nikolaos Doulamis · 发表于：IEEE Transactions on Vehicular Technology · 年份：2024 · DOI：10.1109/tvt.2024.3512457 · 被引用次数：4 · 研究领域：Traffic Prediction and Management Techniques、Advanced Neural Network Applications、Autonomous Vehicle Technology and Safety

As vehicle technologies rapidly advance, video streaming capabilities emerge as a significant feature of modern on-board vehicular systems. The integration of 360$^{\circ }$video streaming in vehicles is expected to enhance road situational awareness, by providing panoramic live streaming and recording capabilities. This paper introduces the concept ofroads digital twin, combining, for the first time, 360$^{\circ }$video streaming with graph neural networks, in order to enhance predictive situational awareness in road environments. To this end, we have developed eGAT, a selective edge-enhanced graph attention network architecture, that uses graph neural networks with attention mechanisms. eGAT is capable of effectively predicting road coverage, considering uplink bandwidth limitations that may affect video streaming quality and user quality of experience (QoE). For the evaluation of our novel method, we utilized four different datasets, considering several vehicular scenarios. For performance comparison purposes, we employed three metrics, namely the overall percentage of the covered region, the normalized mutual information (NMI), and the precision-recall scores. In terms of overall coverage percentage, eGAT provided superior coverage performance against similar studies in all nine scenarios and for all numbers of streaming vehicles investigated, reaching an increase of up to 32.3%. In terms of NMI score, for low values of prediction horizon, eGAT outperformed similar attent...