SCRUBD: Smart Contracts Reentrancy and Unhandled Exceptions\n Vulnerability Dataset

Abstract

Smart Contracts (SCs) handle transactions in the Ethereum blockchain worth\nmillions of United States dollars, making them a lucrative target for attackers\nseeking to exploit vulnerabilities and steal funds. The Ethereum community has\ndeveloped a rich set of tools to detect vulnerabilities in SCs, including\nreentrancy (RE) and unhandled exceptions (UX). A dataset of SCs labelled with\nvulnerabilities is needed to evaluate the tools' efficacy. Existing SC datasets\nwith labelled vulnerabilities have limitations, such as covering only a limited\nrange of vulnerability scenarios and containing incorrect labels. As a result,\nthere is a lack of a standardized dataset to compare the performances of these\ntools. SCRUBD aims to fill this gap. We present a dataset of real-world SCs and\nsynthesized SCs labelled with RE and UX. The real-world SC dataset is labelled\nthrough crowdsourcing, followed by manual inspection by an expert, and covers\nboth RE and UX vulnerabilities. On the other hand, the synthesized dataset is\ncarefully crafted to cover various RE scenarios only. Using SCRUBD we compared\nthe performance of six popular vulnerability detection tools. Based on our\nstudy, we found that Slither outperforms other tools on a crowdsourced dataset\nin detecting RE vulnerabilities, while Sailfish outperforms other tools on a\nmanually synthesized dataset for detecting RE. For UX vulnerabilities, Slither\noutperforms all other tools.\n