DyG-LA: Dynamic Graph Learning via Linear Attention and Recurrent Matrix States

Abstract

Continuous-Time Dynamic Graphs (CTDGs) are essential for modeling event-driven dynamics in complex, evolving systems, ranging from streaming temporal knowledge graphs (tKGs) and real-time recommendation systems to decentralized finance (DeFi) networks. State-of-the-art temporal graph learning methods predominantly compress historical interactions into flat, one-dimensional state vectors. However, we demonstrate that this architectural choice suffers from severe structural interference, akin to catastrophic forgetting, in heterogeneous networks where entities maintain multiple concurrent relational identities (e.g., decentralized finance wallets acting simultaneously as lenders, swappers and borrowers). In this work, we propose DYG-LA (Dynamic Graph Learning via Linear Attention and Recurrent Matrix States), a novel architecture that resolves structural interference by expanding node memory into Matrix-Valued Hidden States (MVHS). Each node maintains a multi-head H × (D/H) × (D/H) state matrix, geometrically updated via asymmetric outer products and regulated by a selective, data-dependent Ebbinghaus decay. To overcome the O(L²) bottleneck of Transformer-based methods without succumbing to the random sampling paradox of pure sequence models (where nodes lose identity due to sparse or noisy temporal sampling), DYG-LA integrates a dual-memory approach. It pairs an RWKV-6 linear attention short-term temporal scanner with the long-term MVHS global memory. The architecture further incorporates a Dynamic Gated Fusion mechanism, effectively acting as an adaptive mixture-of-experts to route signals from the temporal scanner, spatial structure and memory. We evaluate DYG-LA across twelve benchmark datasets under transductive settings. Comprehensive ablation studies demonstrate that the dual-memory design is critical for complex, heterogeneous networks, with the full model achieving state-of-the-art performance.