Distributed multi-agent algorithms for micro grid control

Principal Investigator

Prof Shalabh Bhatnagar (Professor, Robert Bosch Centre for Cyber-Physical Systems, Department of Computer Science and Automation)

Abstract

A microgrid is a networked group of distributed energy sources with the goal of generating, converting and storing energy. This scenario is being envisaged as an important alternative to the conventional scheme with large power stations transmitting energy over long distances. The microgrid technology is useful particularly in the Indian context where extending power supply from the main grids to remote villages is a challenge. In order to take full advantage of the modularity and flexibility of micro-grid technologies, smart control mechanisms are required to manage and coordinate these distributed energy systems so as to minimize the costs of energy production, conversion and storage, without jeopardizing grid stability.

The implementation of such smart controls is by no means easy for the following reasons:

1. Small scale energy production and storage is intrinsically related to intermittency of wind/solar energy and to variability in the load profile. So an important challenge is to increase resilience and reliability under stochastic supply and demand.
2. Micro-grids can operate in two different modes: (1) when they are connected to the main power grid, and (2) in the isolated or island mode. Moreover, they can share energy with other microgrids that require energy. Thus, one needs to make dynamic decisions on (a) when to operate in the connected (to the power grid) or isolated modes, (b) when to share energy with other microgrids and when to store energy for future use, and (c) which form to store energy given that storage management itself involves heterogeneous storage technologies with different operating characteristics.
3. Each microgrid has access to only its local (and not global) state information. While microgrids can exchange information with one another, there are problems with frequent communication between microgrids due to (1) privacy issues and (2) risk of cyber attacks. Important challenges here include (a) optimizing global grid performance with limited communications, (b) voltage and frequency control for grid stabilization.
4. Decision making in microgrids needs to take place on different timescales. On the slower timescale of say hours, one needs to make decisions on energy generation, conversion and storage, while on the faster timescale of minutes to seconds, one needs to make decisions related to dynamic demand response as well as ensuing grid stability, i.e., of frequency and voltage regulation.

In order to tackle these challenges we shall propose a multi-agent reinforcement learning framework for problems of optimal grid energy and storage management.  Because of their model-free nature, reinforcement learning approaches that are primarily data-driven control techniques will play a significant role in these problems. Note that models of energy generation and demand are highly imprecise given the intermittent nature of the energy sources as well as the stochastic nature of the demand. Unlike many other optimization approaches, reinforcement learning does not require a system model nor does it attempt to construct one. Because of the hybrid nature of the microgrid and availability of limited information and communication capabilities, reinforcement learning algorithms for such systems will have to be developed as well. We shall also design suitable hierarchical control algorithms to take into account the multiscale nature of microgrid control.