Smart microgrids can restore power more efficiently and reliably in an outage

They describe their new AI model and show that it outperforms traditional power restoration techniques in a new paper published in the journal IEEE Transactions on Control of Network Systems, a top journal in the field of control systems and network science. Shourya Bose, a Ph.D. student in Zhang's lab, is the paper's first author.

"Nowadays, microgrids are really the thing that both people in industry and in academia are focusing on for the future power distribution systems," Zhang said.

In many communities, infrastructure and its users are totally reliant on a local power generating utility company for electricity. This means that in the case of a disaster or extreme weather event, or even just a tree falling on a line, power goes out until repairs can be made.

Today, many electricity systems are smart in that they are interconnected with computers and sensors. They often incorporate local renewable energy sources such as rooftop solar panels or small wind turbines, and some households and buildings rely on backup generators and/or energy batteries for their electricity demand.

This mix of power sources presents an opportunity to address outages locally by using alternative energy sources to provide electricity before upstream power is restored. One way to do this is with a microgrid, which distributes electricity to small areas such as a few buildings or a town -- although the size of the microgrid can vary.

The microgrid can be connected to the main power utility source, but also can function while disconnected in "islanding mode," self-supported by alternate energy sources and unaffected by the issues impacting the main utility. Zhang's research team focuses on optimizing how microgrids pull from these various alternate sources such as renewables, generators, and batteries to restore power quickly and correctly.

"Essentially, we want to bring the power generation closer to the demand side in order to get rid of the long transmission lines," Zhang said. "This can improve the power quality and reduce the power losses over the lines. In this way, we will make the grid smaller, but stronger and more resilient."

To optimally operate microgrids, Zhang's lab developed an AI-based technique called deep reinforcement learning, the same concept that underpins large language models, to create an efficient framework that includes models of many components of the power system. Reinforcement learning depends on rewarding the algorithm for successfully responding to the changing environment -- so an agent is rewarded when it is able to successfully restore the demanded power of all components of the network. They explicitly model the practical constraints of the real-world system, such as the branch flows that power lines can handle.

"We're modeling a whole bunch of things -- solar, wind, small generators, batteries, and we're also modeling when people's electricity demand changes," Bose said. "The novelty is that this specific flavor of reinforcement learning, which we call constrained policy optimization (CPO), is being used for the first time."

Their CPO approach takes into account real-time conditions and uses machine learning to find long-term patterns that affect the output of renewables, such as the varying demand on the grid at a given time and intermittent weather factors that affect renewable sources. This is unlike traditional systems which often use a technique called model predictive control (MPC) that bases decisions simply on the available conditions at the time of optimization.

For example, if the CPO method predicts that the sun will shine brightly in an hour, it would use up its supply of solar energy with the knowledge that it will later be replenished -- a different strategy than it might take if the day was cloudy. It can also learn about the system based on long-term patterns of how the grid uses solar.

The researchers found that their CPO technique significantly outperforms traditional MPC methods when the forecasts of renewable sources are lower than the reality because of its better understanding of all the possible solar profiles throughout any given day.

They also found that the reinforcement learning controller is able respond much faster than traditional optimization methods in the moment of a power outage.

The research team recently proved the success of their method when they placed first in a global competition that invited participants to use reinforcement learning or similar techniques to operate a power grid. The competition, called L2RPN Delft 2023, was co-sponsored by France's electricity transmission system operator (Réseau de Transport d'Électricité), which the UC Santa Cruz researchers see as an indicator that now large-scale grid operators may start moving toward AI and renewable energy techniques.

Now that they've developed a successful algorithm in simulations, the research team is working to test their model on microgrids in their lab. In the long-term, the researchers hope to implement their solution on the UC Santa Cruz campus's energy system to address outage issues that the residential campus community faces. They also hope to see further interest and collaboration from industry.