Dual-Timescale Trust and Bottom-Up Reputation for Improved Cooperation in Decentralized MARL

Abstract

Sustaining cooperation in decentralized multi-agent reinforcement learning (MARL) remains challenging in social-dilemma environments, where agents operate under partial observability, unreliable communication, and incentives to free ride. This paper proposes a decentralized social mechanism for repeated local Public Goods Games that integrates structured commitment-based communication, bottom-up multi-dimensional reputation, and dual-timescale dyadic trust.Agents exchange messages about intended behavior, which are evaluated against realized actions to estimate commitment honesty. Reputation aggregates local evidence of cooperation, honesty, and behavioral consistency, while trust is updated at short and long timescales to balance responsiveness with robustness to noisy interactions. Without explicit social reward shaping, these signals influence learning through two coupled pathways: credibility-weighted message aggregation and trust-based participation filtering, which reshapes the effective local interaction pool used for payoff formation.We evaluate the framework across multiple public-good multipliers using a common topology-local payoff setting. Comparisons include a baseline, a communication-only configuration, an Only-Reputation configuration with hard reputation-based exclusion, participation-filtering variants, and single-timescale trust ablations. The Full Model improves the cooperation--payoff trade-off and maintains cooperation more consistently than reduced-mechanism configurations. Analyses of learning dynamics, across-seed variability, participation mechanisms, trust timescales, and threshold sensitivity indicate that improved cooperation is associated with combining credibility-aware communication, trust and reputation, and payoff-relevant participation filtering. These results suggest that socially grounded information and interaction mechanisms can support more stable cooperation in decentralized MARL without explicit reward shaping.