A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem

Abstract

Financial portfolio management is the process of constant redistribution of a\nfund into different financial products. This paper presents a\nfinancial-model-free Reinforcement Learning framework to provide a deep machine\nlearning solution to the portfolio management problem. The framework consists\nof the Ensemble of Identical Independent Evaluators (EIIE) topology, a\nPortfolio-Vector Memory (PVM), an Online Stochastic Batch Learning (OSBL)\nscheme, and a fully exploiting and explicit reward function. This framework is\nrealized in three instants in this work with a Convolutional Neural Network\n(CNN), a basic Recurrent Neural Network (RNN), and a Long Short-Term Memory\n(LSTM). They are, along with a number of recently reviewed or published\nportfolio-selection strategies, examined in three back-test experiments with a\ntrading period of 30 minutes in a cryptocurrency market. Cryptocurrencies are\nelectronic and decentralized alternatives to government-issued money, with\nBitcoin as the best-known example of a cryptocurrency. All three instances of\nthe framework monopolize the top three positions in all experiments,\noutdistancing other compared trading algorithms. Although with a high\ncommission rate of 0.25% in the backtests, the framework is able to achieve at\nleast 4-fold returns in 50 days.\n