VulnDetective: Using LLM Agents to Analyze Common Weaknesses and Identify Smart Contract Vulnerabilities

Abstract

Blockchain technology is a promising innovation to store information online while being secure and transparent at the same time. Despite the benefits, the immutable nature of blockchain makes it prone to vulnerabilities as any smart contract uploaded onto the blockchain cannot be modified later. Thus, it is imperative to design secure smart contracts during development stages and incorporate effective vulnerability detection mechanisms. Large-Language-Models (LLMs) are a propitious approach to automate vulnerability detection. However, recent attempts at using LLMs to detect vulnerabilities have resulted in high false positive rates and seem prone to obsolescence due to evolving Solidity code. Moreover, existing research often conflate weaknesses with vulnerabilities, despite their apparent differences, and all prior works only focus on vulnerability detection based on broad categories rather than their exact vulnerability identifiers (VIDs). To address these gaps, we propose a novel framework, VulnDetective, of fine-tuned LLM agents and a CWE-VID database to map weaknesses to their corresponding vulnerability, thereby increasing detection accuracy. Our findings show that pre-trained base models struggle significantly with detecting exact vulnerabilities, while VulnDetective shows significant improvement over base models.