Deep Reinforcement Learning for Cryptocurrency Portfolio Management: A Free-Energy PPO Framework with Geodesic Transaction Costs and Thermodynamic Efficiency Bounds

Abstract

This paper develops a deep reinforcement learning (DRL) framework for cryptocurrency portfolio management in which transaction costs are derived from the Riemannian geometry of the underlying volatility model rather than assumed constant. A Proximal Policy Optimisation (PPO) agent is trained on a reward function derived from non-equilibrium thermodynamics: the free-energy Bellman equation, in which (i) transaction costs are the geodesic slippage S∗ on the Fisher information manifold of a maximum-entropy Markov-switching GARCH model, and (ii) regime-transition costs are the Wasserstein-2 distance Wt between the calm and turbulent return distributions. The agent is embedded in the WOW-E-W quadrilogy, a four-paper research programme that integrates statistical mechanics, fluid dynamics, Riemannian information geometry, and thermodynamic control into a unified cryptocurrency risk architecture. The PPO agent observes an 11-dimensional state vector ot that combines turbulent-regime probabilities \( \hat{\xi}_t(2) \) and parameter estimates \( \hat{\theta}_t \) from a maximum-entropy Markov-switching GARCH model, a viscosity-filtered velocity signal ht and gate states zt, rt from a GRU viscosity filter, and the Fisher curvature Gt, Ricci scalar κt, Betti numbers β0,t, β1,t,Wasserstein dissipation Wt, and topological alarm dI(t) from the Riemannian execution geometry layer. The framework establishes a thermodynamic Carnot bound on portfolio efficiency: η ≤ 1 − Hturb/Hcalm, where Hturb and Hcalm are the maximum-entropy values of the turbulent and calm regime distributions. Five hypotheses are tested across Bitcoin, Ethereum, Ripple, Litecoin, and Bitcoin Cash over January 2017 to March 2026: the geometric-cost PPO agent achieves higher Sharpe ratio than Buy-and-Hold, Greedy signal-following, and flat-fee PPO baselines (bootstrap p < 0.05 for four of five assets); portfolio turnover is reduced by 56 to 83 percent relative to signal-following; the thermodynamic friction point at which the agent prefers no-trade is asset-specific and ranges from 0.6 percent (Bitcoin) to 1.8 percent (Ethereum), ordered by turbulent half-life (Spearman ρ = 0.94, p = 0.017); a joint topological and geometric circuit breaker reduces Maximum Drawdown by 28 to 38 percent; and ablation confirms that every component of ot contributes a statistically significant performance gain (Diebold-Mariano p < 0.05 for at least four of five assets per component). The framework requires liquid cryptocurrency markets with validated parametric volatility models; transferability to other asset classes requires upstream recalibration and is an explicitly bounded limitation.