A Multi-Agent Reinforcement Learning Approach for Blockchain-based Electricity Trading System

Abstract

In microgrid, peer-to-peer (P2P) electricity trading has quickly ascended to the spotlight and gained enormous popularity. However, there are inevitable credit problems and system security problems. Besides, the current model in the electricity trading system cannot balance the utilities of multiple trading entities. In this paper, we propose a blockchain-based distributed P2P electricity trading system. We define elecoins as currency in circulation within our trading system. In order to jointly optimize the utilities of both parties in the elecoins trading, we formulate the elecoins purchasing problem as a hierarchical Stackelberg game. Then, we design a distributed multi-agent utility-balanced reinforcement learning (DMA-UBRL) algorithm to search the Nash equilibrium. Finally, we factually build a blockchain system with a blockchain explorer and deploy an electricity trading smart contract (ETSC) on Ethereum, with a website interface for operating. The numerical results and the implemented realistic system show the advantages of our work.