Fusion Learning for Blockchain Security: Detecting Vulnerabilities in Smart Contracts with GNN and CodeBERT

Abstract

Blockchain smart contracts have been a groundbreaking technology yet are still prone to numerous security vulnerabilities that can lead to large financial and operational losses. In our prior work, we presented an in-depth methodology for data preprocessing and dataset preparation towards facilitating effective vulnerability detection in smart contracts. In this paper, building on that work, we proceed with our work by presenting a new hybrid deep learning architecture that integrates Graph Neural Networks (GNN) and CodeBERT in an efficient way to capture both structural and semantic code features. The hybrid model processes parallel representations of smart contracts: the GNN extracts graph-based control and data flow dependency relations, and CodeBERT makes use of pretrained contextual source code token embeddings. The two embeddings are concatenated and then fed into a shared classifier to predict the existence and types of vulnerabilities. We test our approach on a diverse collection of smart contracts and compare it with single-model baselines. Our hybrid model outperforms individual GNN and CodeBERT approaches with significant performance gains in precision, recall, and F1-score for different types of vulnerabilities. These findings confirm the effectiveness of our fusion approach and introduce the possibility of employing hybrid deep learning models in real-world smart contract security auditing.