MARVEL: Bringing Multi-Agent Reinforcement- Learning Based Variable Speed Limit Controllers Closer to Deployment

作者：Yuhang Zhang, Marcos Quiñones-Grueiro, Zhiyao Zhang, Yanbing Wang, William Barbour, Gautam Biswas, Daniel B. Work · 发表于：IEEE Access · 年份：2024 · DOI：10.1109/access.2024.3489474 · 被引用次数：12 · 研究领域：Traffic control and management、Human-Automation Interaction and Safety、Autonomous Vehicle Technology and Safety

Variable Speed Limit (VSL) is a promising highway traffic management strategy deployed worldwide. Most algorithms deployed operate based on a set of predefined rules which hinders their ability to perform optimally across diverse traffic conditions. Recent studies applying multi-agent reinforcement learning (MARL) techniques to VSL control problems have shown promising results. However, none of the existing MARL frameworks comply with real-world requirements imposed by most traffic agencies across the United States. In this work, we propose MARVEL (Multi-Agent Reinforcement-learning for large-scale Variable spEed Limits), a novel MARL framework for large-scale VSL highway control with real-world deployment capabilities. MARVEL only relies on sensing information available in most real-world settings and learns through a reward structure that incorporates three abstract goals to ensure reasonable behaviors under different traffic conditions. Through parameter sharing among all VSL agents, the proposed framework scales to cover corridors equipped with multiple VSL gantries. The policies are trained in a microscopic traffic simulation environment, focusing on a short freeway stretch with 8 VSL agents spanning 7 miles. For testing, these policies are applied to a more extensive network with 34 VSL agents spanning 17 miles of Interstate 24 (I-24) near Nashville, Tennessee, the United States. MARVEL improves traffic safety by 63.4% compared to the no-control scenario and enhances tr...