RAG-SmartVuln: Enhancing Smart Contract Vulnerability Detection via Retrieval-Augmented LLMs

Abstract

The burgeoning adoption of economically incentivized smart contracts faces persistent security vulnerabilities, resulting in significant financial losses due to their immutability post-deployment. This paper presents a novel framework integrating fine-tuned large language models (LLMs) with Retrieval-Augmented Generation (RAG) to enhance the precision and explainability of smart contract vulnerability detection. By fine-tuning an open-source LLM and employing RAG, our model dynamically incorporates domain-specific external knowledge during inference, significantly improving threat identification. On two public benchmarks, SolidiFI-Benchmark and Smart Bugs Curated, our fine-tuned Qwen2.5-Coder-14B model (QC-14B-FT) outperforms zero-shot LLMs (GPT-3.5 with and without RAG) in terms of F1-score. Specifically, QC-14B-FT achieves an F1-score of 0.64 on SolidiFI, surpassing GPT-3.5-RAG by 9% and GPT-3.5 by 10%. On Smart Bugs Curated, QC-14B-FT achieves an F1-score of 0.73, outperforming GPT-3.5-RAG by 14% and GPT-3.5 by 19%. These results demonstrate the effectiveness of combining RAG with fine-tuning to provide accurate and clear smart contract security assessments.