Enhancing Smart Contract Vulnerability Detection with Hybrid Pre-Trained Models on Source Code and Byte Code

Abstract

Smart contracts have caused significant economic losses due to various vulnerabilities, highlighting the importance of their security. Traditional vulnerability detection methods often focus on a single aspect, either neglecting the semantic and structural information in the source code or overlooking critical execution changes in the bytecode. This can result in incomplete security assessments. To address this, we propose HCBERT (HybridCodeBERT), a pre-trained hybrid model for smart contract vulnerability detection. HCBERT efficiently ex- tracts features from both source code and bytecode by fusing their information and integrating attention mechanisms with graph-based structures such as data flow and control flow graphs. Additionally, leveraging advanced pre-training techniques, we enhance the model's ability to capture complex patterns. Experimental results show that HCBERT achieves detection accuracies of 89.57%, 82.91 %, 94.73%, and 85.79% for four key vulnerabilities: reentrancy, delegate call, timestamp dependency, and overflow/underflow, outperforming existing state-of-the-art methods. Ablation experiments further confirm the benefits of feature fusion and the integration of graph structures and pre-training techniques in improving detection performance.