An Augmented Lagrangian-Based Safe Reinforcement Learning Algorithm for Carbon-Oriented Optimal Scheduling of EV Aggregators

作者：Xiaoying Shi, Yinliang Xu, Guibin Chen, Ye Guo · 发表于：IEEE Transactions on Smart Grid · 年份：2023 · DOI：10.1109/tsg.2023.3289211 · 被引用次数：66 · 研究领域：Electric Vehicles and Infrastructure、Smart Grid Energy Management、Microgrid Control and Optimization

This paper proposes an augmented Lagrangian-based safe off-policy deep reinforcement learning (DRL) algorithm for the carbon-oriented optimal scheduling of electric vehicle (EV) aggregators in a distribution network. First, practical charging data are employed to formulate an EV aggregation model, and its flexibility in both emission mitigation and energy/power dispatching is demonstrated. Second, a bilevel optimization model is formulated for EV aggregators to participate in day-ahead optimal scheduling, which aims to minimize the total cost without exceeding the given carbon cap. Third, to tackle the nonlinear coupling between the carbon flow and power flow, a bilevel model with a carbon cap constraint is formed as a constrained Markov decision process (CMDP). Finally, the CMDP is efficiently solved by the proposed augmented Lagrangian-based DRL algorithm featuring the soft actor-critic (SAC) method. Comprehensive numerical studies with IEEE distribution test feeders demonstrate that the proposed approach can achieve a fine tradeoff between cost and emission mitigation with a higher computation efficiency compared with the existing DRL methods.