A Study on the Impact of Obstacle Size on Training Models Based on DQN and DDQN

作者：Siyu Lu, Ye Tao, Junwei Zeng, Qi Zuo · 发表于：ITM Web of Conferences · 年份：2025 · DOI：10.1051/itmconf/20257301004 · 被引用次数：4

This paper presents a comparative analysis of Deep Q-Network (DQN) and Double Deep Q-Network (DDQN) algorithms in a simulated car racing environment, focusing on how variations in obstacle density affect each algorithm’s learning and trajectory planning, with the goal of enhancing adaptability and safety in autonomous driving systems. In the Gymnasium car racing environment, each episode generates a unique track where obstacles of varying sizes, colours, and quantities are introduced to test the agent's adaptability. Collisions result in immediate penalties and termination of the episode, while avoiding obstacles grants rewards. The model applies penalties to encourage fast completion and minimize the number of frames used. Various parameters such as obstacle size and complexity influence the agent's performance, promoting efficient learning and policy optimization using both DQN and DDQN algorithms under different configurations. After our experiment, comparing with DQN, we found that DDQN could be a better algorithm in car racing scenarios as in some extreme environments, DQN would not very accurately estimate future reward, therefore, agent could not make better decisions because of the misguided evaluation. While DDQN introduces another network for evaluation, which could better decrease overestimation and explore more possible actions to improve training performance. In this experiment, we evaluated both DQN and DDQN algorithms for obstacle avoidance in reinforcement lea...