Optimal planning of hybrid energy storage systems using curtailed renewable energy through deep reinforcement learning

Abstract

Energy management systems are becoming increasingly important to utilize the continuously growing curtailed renewable energy. Promising energy storage systems, such as batteries and green hydrogen, should be employed to maximize the efficiency of energy stakeholders. However, optimal decision-making, i.e., planning the leveraging between different strategies, is confronted with the complexity and uncertainties of large-scale problems. A sophisticated deep reinforcement learning methodology with a policy-based algorithm is proposed to achieve real-time optimal energy storage systems planning under the curtailed renewable energy uncertainty. A quantitative performance comparison proved that the deep reinforcement learning agent outperforms the scenario-based stochastic optimization algorithm, even with a wide action and observation space. A robust performance, with maximizing net profit and a stable system, confirmed the uncertainty rejection capability of the deep reinforcement learning under a large uncertainty of the curtailed renewable energy. Action mapping was performed to visually assess the action the deep reinforcement learning agent took according to the state. The corresponding results confirmed that the deep reinforcement learning agent learns how the deterministic solution performs and demonstrates more than 90% profit accuracy compared to the solution. © 2023 Elsevier Ltd