Multi-Teacher Knowledge Distillation for Smart Contract Vulnerability Detection

Abstract

Smart contracts facilitate transactions on the blockchain, while their vulnerabilities can lead to financial losses for the parties involved. Recent significant security incidents e.g., DAO attacks, have prompted more researchers to explore vulnerability detection in smart contracts. Most existing studies convert smart contract source code into sequences or graphs for feature extraction, while they typically focus solely on capturing the sequential features within sequences or the spatial features of graphs, failing to explore both sequential and spatial correlations implicit in the source code. Although converting source code into a graph helps capture complex spatial relationships, this inevitably loses some information, e.g., data flow is ignored in the control flow graph. Moreover, these methods often design complicated data processing or complex network structures to extract such features. To address the issues, we propose a Multi-Teacher Knowledge Distillation method (i.e., teacher-student framework) for more accurate and efficient Smart Contract Vulnerability Detection, namely MTKD-SCVD. Specifically, we designed a simple dual-channel student model that can extract both sequential and spatial characteristics from sequence and graph data, thus facilitating a comprehensive understanding of vulnerability patterns. Since the superior performance of existing methods in feature extraction from a certain aspect (e.g., spatial), we consider using these state-of-the-art (SOTA) models as teacher models, thus distilling effective features to guide the student model learning. Therefore, MTKD-SCVD can enable the student model to achieve more accurate vulnerability detection with a simple network structure. We conduct extensive experiments on smartbugs public datasets, demonstrating the effectiveness of MTKD-SCVD over SOTA models.