Reinforcement Learning Controller Design for Discrete-Time-Constrained Nonlinear Systems With Weight Initialization Method

作者：Jiahui Xu, Jingcheng Wang, Yanjiu Zhong, Junjin Rao, Shunyu Wu · 发表于：IEEE Transactions on Systems, Man & Cybernetics. Systems · 年份：2024 · DOI：10.1109/tsmc.2023.3344883 · 被引用次数：7 · 研究领域：Computer Science

Extensive research has been dedicated to reinforcement learning (RL) for acquiring proficient optimal controllers through interactions with the environment. However, real-world demands, including enhanced safety performance, introduce considerable challenges to the present design of optimal controllers rooted in RL algorithms. A novel approach is introduced in this article for designing RL-based optimal controllers, employing a control barrier function (CBF) alongside a nonquadratic loss function related to the control signal. The aim is to enable the agent to learn the optimal controller in a secure and efficient manner. To tackle the instability issue in neural network training inherent to traditional RL-based controller design processes, the nonlinear model predictive control (NMPC) technique is employed for initializing the controller network’s weights. A formal demonstration of the method’s optimality is presented. Numerical simulations validate the proposed approach, illustrating its capacity to effectively learn the optimal controller while adhering to the input and state constraints of the system.