SIFDriveNet: Speed and Image Fusion for Driving Behavior Classification Network

作者：Yan Gong, Jianli Lu, Wenzhuo Liu, Zhiwei Li, Xinmin Jiang, Xin Gao, Xingang Wu · 发表于：IEEE Transactions on Computational Social Systems · 年份：2023 · DOI：10.1109/tcss.2023.3303334 · 被引用次数：38 · 研究领域：Advanced Neural Network Applications、Autonomous Vehicle Technology and Safety、Video Surveillance and Tracking Methods

Driving behavior classification is an important direction in the field of social transportation systems and advanced driving assistance system (ADAS), which has attracted more and more attention in recent years. An accurate driving behavior classification algorithm plays a great role in traffic safety, energy saving, and other fields. In this article, we propose a novel vehicle speed and image fusion for driving behavior classification network (SIFDriveNet), which classifies driver behaviors into normal driving, aggressive driving, and drowsy driving. Our method has the following key advantages. First, in the research of driving behavior classification, we are the first to introduce a 2-D image with rich roadside information and convert speeds into a 2-D spectrogram expressing the time–frequency characteristics of speeds through short-time Fourier transform (STFT) while unifying the data space of image information and speed information. Second, we propose a tensor fusion method based on weight decomposition to fully fuse the vectors of the two modalities. This method maps the tensor outer product results to the low-dimensional space through weight decomposition and has a low computational cost while maintaining the fusion effect of the tensor outer product. In addition, we evaluated our model on the public UAH-DriveSet and compared it with the most advanced model. Experimental results show that our model has a better performance, and F1-score is 97.9% on all roads. Especially...