Smart contract vulnerability detection using large language models

Abstract

This research presents a novel approach to detecting role-based access control vulnerabilities in smart contracts using large language models (LLMs). Smart contracts deployed on blockchain platforms often contain critical security vulnerabilities that can lead to significant financial losses, with improper access control being a common issue. Our methodology combines Abstract Syntax Tree (AST) analysis with strategic context slicing to enable effective vulnerability assessment by LLMs. The system first identifies potentially vulnerable functions through structural analysis, extracts relevant security context, and then leverages LLMs to make the final vulnerability and exploitability determinations. We evaluate our approach using a dataset of 28 smart contracts across five different language models, including GPT-4 and its variants and GPT-o3-mini as well. Results demonstrate strong performance, with the best model achieving 93% accuracy and 100% precision in vulnerability detection. The system not only identifies vulnerable functions but also assesses their practical exploitability, achieving up to 100% accuracy in exploitability determination with GPT-4. The comparative analysis across different models reveals interesting trade-offs between precision and recall, with some models excelling at avoiding false positives while others prioritize catching all vulnerabilities. Our findings suggest that LLMs, when properly guided through context slicing and structured prompting, can effectively reason about complex security properties in smart contracts. This approach offers a promising direction for augmenting traditional static analysis tools with the contextual understanding capabilities of large language models, potentially improving the security of blockchain applications.