Cola-HRL: Continuous-Lattice Hierarchical Reinforcement Learning for Autonomous Driving

作者：Lingping Gao, Ziqing Gu, Cong Qiu, Lanxin Lei, Shengbo Eben Li, Sifa Zheng, Wei Jing, Junbo Chen · 发表于：2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · 年份：2022 · DOI：10.1109/iros47612.2022.9982041 · 被引用次数：20 · 研究领域：Autonomous Vehicle Technology and Safety、Reinforcement Learning in Robotics、Traffic control and management

Reinforcement learning (RL) has shown promising performance in autonomous driving applications in recent years. The early end-to-end RL method is usually unexplainable and fails to generate stable actions, while the hierarchical RL (HRL) method can tackle the above issues by dividing complex problems into multiple sub-tasks. Prior HRL works either select discrete driving behaviors with continuous control commands, or generate expected goals for the low-level controller. However, they typically have strong scenario dependence or fail to generate goals with good quality. To address the above challenges, we propose a Continuous-Lattice Hierarchical RL (Cola-HRL) method for autonomous driving tasks to make high-quality decisions in various scenarios. We utilize the continuous-lattice module to generate reasonable goals, ensuring temporal and spatial reachability. Then, we train and evaluate our method under different traffic scenarios based on real-world High Definition maps. Experimental results show our method can handle multiple scenarios. In addition, our method also demonstrates better performance and driving behaviors compared to existing RL methods.