The Role of Prompt Patterns in LLM-Based Smart Contract Vulnerability Detection

Abstract

This paper investigates the impact of prompt patterns on the effectiveness of LLMs in detecting smart contract vulnerabilities. Building on the zero-shot prompt used by Chen et al., which combines persona and chain-of-thought (CoT) patterns, we conducted an ablation study to measure the impact of removing specific patterns on detection accuracy. We further extended the original prompt with a few-shot examples to evaluate potential performance gains over the baseline. Finally, we investigated the impact of prompt patterns on a reasoning model.The results show that LLMs without reasoning mechanisms benefit modestly from persona and CoT patterns, while few-shot prompting achieves the best precision–recall balance. In reasoning-enabled LLMs, simple task-definition prompts outperform persona and CoT patterns, whereas few-shot examples further improve precision and generalization.