The smart contract vulnerability detection based on pre-trained model feature fusion

Abstract

With the widespread application of blockchain technology across various fields, the security of smart contracts has become increasingly important. In the field of anomaly detection, particularly in smart contract vulnerability detection, pre-trained models have demonstrated tremendous potential, improving the accuracy and efficiency of vulnerability detection. However, these models typically focus on a single modality, which limits their applicability. While integrating multiple models can mitigate this issue, effectively fusing features from different pretrained models remains a challenge that needs to be addressed.To tackle this problem, this paper proposes a feature fusion method for smart contract vulnerability detection based on pre-trained models (PFSCV). The method uses contrastive learning (CL) to capture fine-grained relational information between smart contracts, generating sample pairs based on the relationships between contracts to guide the fine-tuning of the pre-trained model CodeBERT, enabling it to learn contextual information from the source code. At the same time, the UnixCoder model is fine-tuned to extract data flow information from the contracts. Subsequently, we introduce an att-BiLSTM (Attention-based Bidirectional Long Short-Term Memory) model to fuse and integrate features extracted from different pre-trained models. Finally, extensive experiments on real-world smart contract datasets validate the effectiveness and reliability of the proposed method.