Active perception and recognition learning system based on Actor‐Q architecture

作者：Katsunari Shibata, T. Nishino, Yoichi Okabe · 发表于：Systems and Computers in Japan · 年份：2002 · DOI：10.1002/scj.10207 · 被引用次数：8 · 研究领域：Neural Networks and Applications、Neural dynamics and brain function、Cognitive Science and Education Research

Abstract This paper proposes the Actor‐Q architecture, which is a combination of Q‐Learning and Actor‐Critic architecture, as well as the active perception and recognition learning system based on that architecture. In Actor‐Q architecture, the system output is divided into the “action,” which is a discrete value as intention, and the “motion,” which is a continuous‐valued vector. As the first step, the “action” is determined from Q‐values. If the “action” is accompanied with a “motion,” the “motion” is executed according to the corresponding Actor output. Q‐value is learned by Q‐learning, and Actor is trained with the Q‐value corresponding to that “action” on behalf of the Critic output. In this study, the action is defined as the decision of the sensor motion or the recognition of the respective pattern. Q‐value is assigned to each of those. When the sensor motion is selected, the sensor is moved according to the Actor output. When recognition is selected, the recognition result that the presented pattern is the one corresponding to the selected Q‐value is output. The Q‐value is learned, using the reinforcement signal representing the true/false of the result. Both Q‐value computing module and Actor are composed of neural networks, with the visual sensor signals as input. By this architecture, the following three problems of the conventional active perception and recognition learning system are dissolved. (1) The sensor can be trapped in a local maximum of the recognition e...