Enhancing Vulnerability Detection in Smart Contracts Using Transformer-Based Embeddings and Graph Neural Networks

Abstract

This paper proposes a cutting-edge vulnerability detection method for smart contracts, combining Transformer-based embeddings and Graph Neural Networks (GNNs). Critical opcodes are identified and dynamically weighted using attention mechanisms, enhancing feature representation. The GNN captures both relational patterns and critical opcode characteristics, enabling robust detection of vulnerabilities. Experimental results show significant improvements in F1-scores for both binary and multi-class detection tasks, outperforming traditional models like LightGBM. This approach leverages modern AI advancements to address challenges in accuracy and generalization, providing a scalable and effective solution for smart contract analysis.