Real-world challenges for multi-agent reinforcement learning in grid-interactive buildings

作者：Kingsley Nweye, Bo Liu, Peter Stone, Zoltán Nagy · 发表于：Energy and AI · 年份：2022 · DOI：10.1016/j.egyai.2022.100202 · 被引用次数：52 · 研究领域：Smart Grid Energy Management、Microgrid Control and Optimization、Building Energy and Comfort Optimization

Building upon prior research that highlighted the need for standardizing environments for building control research, and inspired by recently introduced challenges for real life reinforcement learning (RL) control, here we propose a non-exhaustive set of nine real world challenges for RL control in grid-interactive buildings (GIBs). We argue that research in this area should be expressed in this framework in addition to providing a standardized environment for repeatability. Advanced controllers such as model predictive control (MPC) and RL control have both advantages and disadvantages that prevent them from being implemented in real world problems. Comparisons between the two are rare, and often biased. By focusing on the challenges, we can investigate the performance of the controllers under a variety of situations and generate a fair comparison. As a demonstration, we implement the offline learning challenge in CityLearn, an OpenAI Gym environment for the easy implementation of RL agents in a demand response setting to reshape the aggregated curve of electricity demand by controlling the energy storage of a diverse set of buildings in a district. We use CityLearn to study the impact of different levels of domain knowledge and complexity of RL algorithms and show that the sequence of operations (SOOs) utilized in a rule based controller (RBC) that provides fixed logs to RL agents during offline training affect the performance of the agents when evaluated on a set of four e...