A Real-Time and Efficient Optical Flow Tracking Accelerator on FPGA Platform

作者：Yifan Gong, Jinshuo Zhang, Xin Liu, Jialin Li, Ying Lei, Zhe Zhang, Chen Yang, Li Geng · 发表于：IEEE Transactions on Circuits and Systems I Regular Papers · 年份：2023 · DOI：10.1109/tcsi.2023.3298969 · 被引用次数：13 · 研究领域：Advanced Vision and Imaging、Advanced Image and Video Retrieval Techniques、Image Processing Techniques and Applications

Optical flow is a highly efficient visual tracking algorithm, which is commonly used to estimate pixel movement between two consecutive images in a video sequence. However, its high computational complexity and large number of computations become a bottleneck that hinders the performance of embedded vision systems. When applied to simultaneous localization and mapping (SLAM), it is necessary to consider not only time consumption, but also the overall accuracy of the system, causing even greater difficulties. In this paper, a real-time multi-scale Lucas Kanade (LK) optical flow hardware accelerator with parallel pipeline architecture is proposed. The designed circuit meets the high precision and real-time performance required by SLAM while fully considering the limitations of hardware resources. It is deployed on Xilinx Zynq SoC and achieves a frame rate of 93 fps for feature tracking of continuous frame images at$752\times 480$resolution. Compared with the implementation on ARM CPU, the average speed is increased by$4.5\times $. Finally, the feasibility and applicability of the hardware accelerator system designed in this paper are verified on the SLAM system. Experimental results on a public dataset show that the average Root Mean Square Error (RMSE) of this work is 0.189 m, indicating that the hardware accelerator has comparable precision with existing state-of-the-art software algorithms, achieving a great balance of performance and precision.