Two-layer distributionally robust planning for hydro-wind-solar-storage systems based on reinforcement learning

Abstract Under the large-scale integration of wind turbine and photovoltaic into the grid, the power system faces the challenge of insufficient flexibility for regulation. Coordinated planning of hydro-wind-solar-storage systems can effectively mitigate the output volatility of renewable energy sources. This paper proposes a distributionally robust planning method for hydro-wind-solar-storage systems based on the Wasserstein distance. First, taking into account the spatiotemporal correlations of factors such as wind speed and solar irradiance, an auxiliary classifier generative adversarial network (AC-GAN) is employed to generate a set of wind turbine and photovoltaic output scenarios. Then, a bilevel capacity planning model is constructed for the integrated system. The upper level aims to minimize investment costs by determining the optimal energy storage capacity, while the lower level focuses on minimizing operational costs through optimizing storage operation states and the output of various devices. Subsequently, an improved proximal policy optimization (PPO) algorithm, grounded in the Markov decision process framework, is used to solve the model. Finally, an actual case study based on a hydro-wind-solar system in Qinghai China is conducted to validate the effectiveness of the proposed method.