Performance Improvement of RoBERTa Training with Gradient Accumulation for Ethereum Transaction Anomaly Detection

Abstract

The growth of Decentralized Finance (DeFi) demands advanced fraud detection, yet current methods face a trade-off: tabular models lack semantic understanding, while language models like RoBERTa struggle with structured data and high computational costs. This paper introduces a novel pipeline that transforms structured Ethereum transaction data into natural language sentences, enabling a standard RoBERTa model to analyze financial behavior efficiently using Gradient Accumulation. A stratified 5-fold cross-validation on a public dataset revealed a key performance trade-off: while Random Forest achieved the highest F1-Score (0.796), our RoBERTa GA model proved superior in the critical metric of Recall (0.769). This finding validates our semantic approach not merely as a competitive alternative, but as a strategically advantageous method when the primary goal is minimizing missed fraudulent transactions. Our work confirms the viability of applying NLP to blockchain security and provides a foundation for future language-model-driven monitoring systems.