Exploring Noise Deep Q Network Based on Cross-Connected

作者：Xiaming Wu, Enzhi Chen, Mingqiu Li, Chunyang Wang · 年份：2019 · DOI：10.1109/mlbdbi48998.2019.00039 · 研究领域：Reinforcement Learning in Robotics、Autonomous Vehicle Technology and Safety、Anomaly Detection Techniques and Applications

Deep Q Network, which is a combination of CNN and Q-Learning, is an important turning point in deep reinforcement learning. DQN algorithm has achieved great success on the Atari 2600 game platform. However, in some complex game environments, on the one hand, CNN is not accurate enough to extract the original environment feature information points, which leads to unreasonable action strategy selection in the later stage of DQN; on the other hand, the local ε-perturbation strategy adopted by DQN can not be effectively explored on a large scale, resulting in the optimal strategy of action space perturbation. To solve these problems, a new exploratory noise deep Q network based on cross-connected is proposed. Firstly, the model is a six-layer network structure, which includes the input layer, three convolution layer, one full-connection layer and the output layer. The second convolution layer is connected to the full-connection layer, making full use of the characteristics of low-level. Secondly, parameter noise is introduced into the whole connection layer and output layer, which adds parameter noise to its weight and causes network output changes. According to the changes, a large number of new states are explored, more abundant samples are provided, and effective decision information is provided. Finally, the experimental results show that the effectiveness of the model is validated in multiple game environments of Atari 2600.