Multi-Zone HVAC Control With Model-Based Deep Reinforcement Learning

作者：Xianzhong Ding, Alberto Cerpa, Wan Du · 发表于：IEEE Transactions on Automation Science and Engineering · 年份：2024 · DOI：10.1109/tase.2024.3410951 · 被引用次数：27 · 研究领域：Building Energy and Comfort Optimization

The application of reinforcement learning in controlling Heating, Ventilation, and Air Conditioning (HVAC) systems has been extensively researched. Existing studies primarily focus on Model-Free Reinforcement Learning (MFRL), which involves trial-and-error interactions with real buildings to train the agent. However, MFRL encounters a significant challenge: it requires a large amount of training data to achieve satisfactory performance. While simulation models have been used to generate training data and expedite the training process, they necessitate high-fidelity building models that are difficult to calibrate. As a result, Model-Based Reinforcement Learning (MBRL) has been employed for HVAC control. Although MBRL demonstrates remarkable sample efficiency, it often falls short in terms of asymptotic control performance, particularly in achieving substantial energy savings while ensuring occupants’ thermal comfort. In this study, we conduct experiments to analyze the limitations of current MBRL-based HVAC control methods, focusing on model uncertainty and controller effectiveness. Leveraging the insights gained from these experiments, we develop MB2C, an innovative MBRL-based HVAC control system that combines high control performance with exceptional sample efficiency. MB2C learns the dynamics of the building by employing an ensemble of environment-conditioned neural networks and utilizes a novel control method called Model Predictive Path Integral (MPPI) for HVAC control. M...