Smart contracts are commonly audited through static analysis to explore vulnerabilities. However, static approaches typically produce heterogeneous findings rather than reproducible, executable proof-of-concept (PoC) test cases, leading to costly and ad hoc manual validation. Large language models (LLMs) offer a promising way to translate audit reports into PoC test cases, but face three major challenges: noisy inputs, lack of execution grounding, and missing runtime oracles. We present SmartPoC, an end-to-end approach for validating reported vulnerabilities in audit reports by generating and executing PoC test cases with automated exploitability verification. SmartPoC first extracts a focused function-level slice from each report to reduce noise, centering on the key functions referenced in a finding and augmenting them with execution-relevant neighbors. To improve executability, we wrap LLM-based PoC synthesis in a generate-repair-execute loop, combining deterministic pre-execution sanitization with feedback-driven post-execution debugging. We further use differential verification as an oracle to confirm the exploitability of generated test cases. On the SmartBugs-Vul and FORGE-Vul benchmarks, SmartPoC achieves confirmation precision of 98.32% and 98.65%, with recall of 84.17% and 85.28%, respectively. On a recent Etherscan verified-source corpus, SmartPoC confirms 64 bugs from 545 audit findings at an average cost of $0.03.
Christoph Hochrainer, Valentin Wüstholz, Maria Christakis
Zero-knowledge virtual machines (zkVMs) are increasingly deployed in decentralized applications and blockchain rollups since they enable verifiable off-chain computation. These VMs execute general-purpose programs, frequently written in Rust, and produce succinct cryptographic proofs. However, zkVMs are complex, and bugs in their constraint systems or execution logic can cause critical soundness (accepting invalid executions) or completeness (rejecting valid ones) issues. We present Arguzz, the first automated tool for testing zkVMs for soundness and completeness bugs. To detect such bugs, Arguzz combines a novel variant of metamorphic testing with fault injection. In particular, it generates semantically equivalent program pairs, merges them into a single Rust program with a known output, and runs it inside a zkVM. By injecting faults into the VM, Arguzz mimics malicious or buggy provers to uncover overly weak constraints. We used Arguzz to test six real-world zkVMs (RISC Zero, Nexus, Jolt, SP1, OpenVM, and Pico) and found eleven bugs in three of them. One RISC Zero bug resulted in a $50,000 bounty, despite prior audits, demonstrating the critical need for systematic testing of zkVMs.
Blockchain technologies had a significant impact on many sectors of contemporary society, with virtual currencies being the most prominent example. The introduction of the Ethereum blockchain and its native support for smart contracts, self-enforcing programs that enable trustworthy digital interactions, has broadened the possible adoption contexts. These programs possess unique characteristics, such as code immutability and autonomous execution, which necessitate innovative testing methodologies. Despite their growing adoption, current testing practices and tools for smart contracts lag behind those available for traditional software systems, raising concerns about the reliability of decentralized applications. This dissertation addresses these challenges through three core research objectives. First, it advances mutation testing for Ethereum smart contracts by introducing a practical framework and tool that support Solidity-specific test adequacy assessment and incremental mutation analysis during development. This enables developers to systematically evaluate and improve the fault-detection capabilities of their test suites based on metrics that go beyond simple code coverage. Second, it explores how smart contract auditing practices can benefit from mutation testing by integrating live mutant inspection into code reviews and automating the generation of missing test cases. This contribution enhances the auditors’ ability to identify weaknesses in the test suite and provide actionable feedback to clients. Third, it supports smart contract maintenance activities by proposing a novel capture-replay testing framework and tool for upgradeable contracts. The approach harnesses historical blockchain transactions as tests, allowing developers to detect behavioral inconsistencies introduced by upgrades without the need to manually reconstruct testing scenarios. Through these contributions, the thesis aims to advance the state of smart contract quality assurance, offering both practical frameworks and theoretical insights that enhance the reliability of blockchain-based systems.
Smart contract (SC) fuzzing is a critical technique for detecting vulnerabilities in blockchain applications. However, its adoption remains challenging for practitioners due to fundamental differences between SCs and traditional software systems. In this study, we investigate the challenges practitioners face when adopting SC fuzzing tools by conducting an inductive content analysis of 381 GitHub issues from two widely used SC fuzzers: Echidna and Foundry. Furthermore, we conducted a user study to examine how these challenges affect different practitioner groups, SC developers, and traditional software security professionals, and identify strategies practitioners use to overcome them. We systematically categorize these challenges into a taxonomy based on their nature and occurrence within the SC fuzzing workflow. Our findings reveal domain-specific ease-of-use and usefulness challenges, including technical issues with blockchain emulation, and human issues with a lack of accessible documentation and process automation. Our results provide actionable insights for tool developers and researchers, guiding future improvements in SC fuzzer tool design.
Abstract- The evolution of blockchain and Web3 technologies has paved the way for decentralized application platforms that enable transparent, tamper-proof transactions without relying on centralized servers. However, existing solutions such as Gitcoin and Giveth are either too complex or unsuitable for modular deployment in educational and lightweight environments. This paper proposes MetaSuite, a role-based, blockchain-driven Software-as-a-Service (SaaS) platform that enables users to create, transfer, and donate tokens while allowing an administrator to securely withdraw funds. Built entirely on the Ethereum blockchain using Solidity smart contracts, MetaSuite operates without a backend and integrates wallet-based authentication via MetaMask and Ethers.js. The platform ensures transparent fund management through on-chain event logging and role-based access controls. Performance evaluations on the Ethereum HoleskyTestnet demonstrate the system’s reliability, gas-efficiency, and real-time responsiveness. By eliminating backend dependencies and emphasizing traceability, MetaSuite serves as a minimalistic yet scalable Web3 solution suitable for academic, experimental, and small-scale real-world deployments. Keywords—Blockchain, Web3, Smart Contracts, Ethereum, MetaMask, Tokenization, SaaS, Ethers.js, Decentralized Applications.
The popularity of smart contracts has cemented their place in the Blockchain Ecosystem.This is because of the immense number of use cases smart contracts provide.They have become the go-to solution for improving transparency and security for all parties involved in the transaction.Furthermore, a smart contract is immutable after it is deployed.Thus optimization of the smart contract is very important before deployment.Sol-Repairer is a tool that provides the implementation for identifying dead code segments from solidity-written smart contracts and then repairing them.Extensive experiments show that Sol-Repairer optimizes dead code better than the solidity compiler.The study also demonstrates that optimizing dead code reduces gas consumption significantly for smart contracts. CCS Concepts• Software and its engineering → Software testing and debugging.
In smart contract fuzz testing, it is crucial to consider the inter-dependencies between the contract functions. To effectively test the business logic of a contract, its functions must be invoked in a meaningful order. In this paper, we propose techniques that utilize static analysis on Ethereum bytecode to tackle this challenge. When compared with the current state-of-the-art, our approach takes Solidity compiler’s variable packing optimization into account and allows more precise analysis of the data-flows between functions. In addition, we devise a novel test case initialization algorithm for fuzz testing, which minimizes the redundancy in the generated seed set. Our algorithm reduces test cases that share similar function call patterns and leads to more effective testing of the contract code during the fuzz testing. Experimental results show that the proposed techniques improve the effectiveness of smart contract fuzz testing for vulnerability detection. Specifically, our techniques enabled the fuzz testing tool to trigger the target bugs in the benchmark 3.0 times faster on average.
Irving Jared Villanueva, Madhusudan Srinivasan, Faqeer Ur Rehman
Blockchain smart contracts play a crucial role in automating and securing agreements in diverse domains such as finance, healthcare, and supply chains. Despite their critical applications, testing these contracts often receives less attention than their development, leaving significant risks due to the immutability of smart contracts post-deployment. A key challenge in the testing of smart contracts is the oracle problem, where the exact expected outcomes are not well defined, complicating systematic testing efforts.Metamorphic Testing (MT) addresses the oracle problem by using Metamorphic Relations (MRs) to validate smart contracts. MRs define how output should change relative to specific input modifications, determining whether the tests pass or fail. In this work, we apply MT to test an Ethereum-based crowdfunding smart contract, focusing on core functionalities such as state transitions and donation tracking.We identify a set of MRs tailored for smart contract testing and generate test cases for these MRs. To assess the effectiveness of this approach, we use the Vertigo mutation testing tool to create faulty versions of the smart contract. The experimental results show that our Metamorphic Relations (MRs) detected 25.65% of the total mutants generated, with the most effective MRs achieving a mutant-killing rate of 89%. These results highlight the utility of MT to ensure the reliability and quality of blockchain-based smart contracts.
Igor Konnov, Jure Kukovec, Thomas Pani, Roberto Saltini · 5 authors
We investigate automated model-checking of the Ethereum specification, focusing on the Accountable Safety property of the 3SF consensus protocol. We select 3SF due to its relevance and the unique challenges it poses for formal verification. Our primary tools are TLA+ for specification and the Apalache model checker for verification. Our formalization builds on the executable Python specification of 3SF. To begin, we manually translate this specification into TLA+, revealing significant combinatorial complexity in the definition of Accountable Safety. To address these challenges, we introduce several layers of manual abstraction: (1) replacing recursion with folds, (2) substituting abstract graphs with integers, and (3) decomposing chain configurations. To cross-validate our results, we develop alternative encodings in SMT (CVC5) and Alloy. Despite the inherent complexity, our results demonstrate that exhaustive verification of Accountable Safety is feasible for small instances - supporting up to 7 checkpoints and 24 validator votes. Moreover, no violations of Accountable Safety are observed, even in slightly larger configurations. Beyond these findings, our study highlights the importance of manual abstraction and domain expertise in enhancing model-checking efficiency and showcases the flexibility of TLA+ for managing intricate specifications.
Testing in the software development process is one of the crucial disciplines. The immutability of deployed smart contracts makes tests particularly important in their design and implementation. The objective is to minimize the costs of smart contract testing by reducing the number of test cases. The article presents the test suite reduction method for smart contracts, which exploits the knowledge of their design manner. In particular, the design of a mechanism for evaluating the list of verification rules controlling the execution of transactions was used. The proposed method defines a minimum set of test cases for smart contracts designed according to a specific design pattern. The test case reduction level was shown using analytical formulas enclosed in the paper. Compared to full coverage, the method reduces over 96% of test cases for a smart contract with at least eight verification rules. The performance tests were conducted and the analysis of the results was enclosed as well. Both the smart contract evaluation time and the test suite execution time are less than 0.01 of a millisecond. Besides, the method reduces the number of test data for smart contracts by over 79%. Therefore, the method can be especially useful in continuous software integration and delivery.
Smart contracts are self-executing programs on the blockchain, representing the core of Decentralized Finance (DeFi). They can carry billions of dollars, making their correctness and security essential. However, like any other software application, smart contracts may contain vulnerabilities that can be exploited by malicious adversaries — a concern that is further exacerbated by their immutability and transparent nature. Fuzzing is a widely adopted automated testing approach for assessing and ensuring smart contract quality. Although powerful, its effectiveness heavily depends on the quality of the initial inputs, also referred to as seeds. This thesis identifies a key limitation in traditional smart contract fuzzers, such as Echidna, which often rely on randomly generated seeds. It investigates how these fuzzers can be improved in exploring deep contract states, reaching hard-to-trigger conditional paths, and increasing the number of executed instructions by providing higher-quality initial seeds. This thesis proposes AutumnEchidna, a smart contract pre-processing tool that leverages static analysis to generate optimized initial seeds for fuzzing. The methodology involves generating transaction sequences based on state dependencies and producing arguments through constraint solving, aiming to guide execution toward critical contract states. Experiments are conducted on two datasets: a Motivation Dataset and a Maze Dataset, designed to simulate complex input constraints and deep state transitions. Performance is evaluated based on instruction coverage and execution time under consistent configurations for both baseline Echidna (with random seeds) and AutumnEchidna (with optimized seeds). The experiment results show that AutumnEchidna improves instruction coverage by 1.26% on the aggregated Motivation Dataset and by 4.45% on the aggregated Maze Dataset. Additionally, it also reduces the execution time to achieve comparable or higher coverage. These findings demonstrate that optimized seed generation can enhance both the effectiveness and efficiency of smart contract fuzzing. This thesis concludes that incorporating static analysis to generate high-quality initial seeds is a promising approach for enhancing the performance of fuzzing strategies.
Zero-knowledge (ZK) proofs have been increasingly popular in privacy-preserving applications and blockchain systems.To facilitate handy and efficient ZK proof generation for normal users, the industry has designed domain-specific languages (DSLs) and ZK compilers.Given a program in ZK DSL, a ZK compiler compiles it into a circuit, which is then passed to the prover and verifier for ZK checking.However, the correctness of ZK compilers is not well studied, and recent works have shown that de facto ZK compilers are buggy, which can allow malicious users to generate invalid proofs that are accepted by the verifier, causing security breaches and financial losses in cryptocurrency.In this paper, we propose MTZK, a metamorphic testing framework to test ZK compilers and uncover incorrect compilations.Our approach leverages deliberately designed metamorphic relations (MRs) to mutate ZK compiler inputs.This way, ZK compilers can be automatically tested for compilation correctness using inputs and mutated variants without requiring manual intervention.We propose a set of design considerations and optimizations to deliver an efficient and effective testing framework.In the evaluation of four industrial ZK compilers, we successfully uncovered 21 bugs, out of which the developers have promptly patched 15.We also show possible exploitations of the uncovered bugs to demonstrate their severe security implications.
Web3 describes an idea for the next evolution of the Internet, where blockchain technology enables the Internet of Value. As Web3 software, decentralized applications (DApps) have emerged in recent years. There exists a natural link between DApps and cryptocurrencies, where faults in DApps could directly lead to monetary losses associated with cryptocurrencies. Hence, efficient fault localization technology is of paramount importance for urgent DApp rescue operations and the mitigation of financial losses. However, fault localization methods applied in traditional applications are not well-suited for this specific field, due to their inability to identify DApp-specific fault features, e.g., a substantial amount of cryptocurrency is transferred from DApps to hackers. In order to explore the root cause of DApp faults, some researchers try to identify suspicious code snippets through mutation testing. Nonetheless, applying mutation testing for DApp fault localization is time-consuming and thus limited in practice. This paper conducts the first comprehensive study of DApp fault localization. We introduce DAppFL, a learning-based DApp fault localization tool that performs reverse engineering to gather executed source code and then trace cryptocurrency flow to assist in locating faulty functions. We also present the inaugural dataset for DApp fault localization, providing a new benchmark for this domain.Our experimental results demonstrate that DAppFL locates 63% of faults within the Top-5, 23% more than the state-of-the-art method. To facilitate further research, our code and dataset are freely available online: https://github.com/xplanet-sysu/awesome-works#dappfl.
Solidity compiler plays a key role in enabling the development of smart contract applications on Ethereum by governing the syntax of a domain-specific language called Solidity and performing compilation and optimization of Solidity code. The correctness of Solidity compiler is critical in fostering transparency, efficiency, and trust in industries reliant on smart contracts. However, like other software systems, Solidity compiler is prone to bugs, which may produce incorrect bytecodes on blockchain platforms, resulting in severe security concerns. As a domain-specific compiler for smart contracts, Solidity compiler differs from other compilers in many perspectives, posing unique challenges to detect its bugs. To understand the bugs in Solidity compiler and benefit future research, in this paper, we present the first systematic study on 533 Solidity compiler bugs. We carefully examined their characteristics (including symptoms, root causes, and distribution), and their triggering test cases. Our study leads to seven bug-revealing takeaways for Solidity compiler. Moreover, to study the limitations of Solidity compiler fuzzers and bring our findings into practical scenarios, we evaluate three Solidity compiler fuzzers on our constructed benchmark. The results show that these fuzzers are inefficient in detecting Solidity compiler bugs. The inefficiency arises from their failure to consider the interesting bug-inducing features, bug-related compilation flags, and test oracles.
Enzo Nicourt, Benjamin Kushigian, Chandrakana Nandi, Ylìès Falcone
This paper presents a successful industrial case study on the application of mutation testing to evaluate and improve test suites for smart contracts. ERCx is a comprehensive, hand-written test suite and framework for smart contract testing, created by Runtime Verification. Despite its thoroughness, hand-written tests can miss edge cases. To address this, we employed mutation testing, which introduces small, syntactic changes, known as mutants, to the program. Mutants that go undetected by the test suite highlight its potential weaknesses, and by presenting them as testing goals, mutation testing helps developers iteratively improve their test suites. In this study, we used mutation testing to expand the ERCx test suite with five new test cases, including one potential vulnerability identified as critical by the ERCx developers. We also developed a test redundancy metric by analyzing pairwise correlation of test data on mutants; we used this redundancy metric to minimize the test suite by removing redundant tests. Finally, we ran both the full and minimized test suites on 106 real-world, faulty ERC-20 contracts to compare the suites' effectiveness and efficiency. Our findings reveal that although the minimized test suite has systematically lower running times compared to the full suite, it still detected faults in 105 of the 106 real-world tokens, retaining nearly all of the full suite's fault-detection capability.
Sebastian Bănescu, Morena Barboni, Andrea Morichetta, Andrea Polini · 5 authors
Smart contracts hold the potential to revolutionize various industries, but their implementation requires thorough testing due to the associated financial risks. Mutation testing is a powerful technique that can boost the fault-detection capabilities of a test suite, but it can also foster a deeper understanding of smart contract behavior. This work investigates the productivity of mutants with respect to their capabilities in disclosing Solidity issues. Based on these findings, it proposes an enhanced mutation strategy to better assist smart contract auditors during code inspection activities. 9 novel mutation operators are introduced in this paper and 13 existing operators are improved. The results show a $30 \%$ reduction in the number of generated mutants and time savings of $62 \%$, while increasing the set of productive mutants related to issues by $43 \%$ overall. We note that the most valuable type of mutants that could help disclose an issue as a result of manual mutant inspection was increased by $125 \%$.
Delays are inevitable in complex distributed environments. Timeout mechanisms are commonly used to handle unexpected failures in distributed systems. However, incorrect timeout handling or implementation errors in timeout mechanisms can lead to system hang-ups or crashes. Such timeout bugs may be crucial and pose a significant threat to the availability and security of distributed systems.In this work, we introduce Chronos, a general testing framework for automatically detecting timeout bugs in distributed systems with deep-priority transient delays. First, we propose general runtime delayed libraries that dynamically inject fine-grained delays in a Distributed System Under Test (DSUT). To effectively trigger delays and constantly explore timeout bugs in deep paths, Chronos harnesses a deep-priority guided fuzzing that dynamically generates high-quality delay sequences in the runtime. Then, Chronos utilizes transient delays to eliminate the time overhead caused by actual delays and accelerate the test process. We implemented and evaluated Chronos on four widely used distributed systems, including ZooKeeper, MySQL-Cluster, HDFS, and Go-Ethereum. Compared with the state-of-the-art techniques, Random, Brute-Force, and Coverage-Guided fault injection, Chronos covers 26.40%, 21.69%, and 15.14% more timeout mechanism logic, respectively. Furthermore, Chronos has detected 27 timeout bugs in these real-world applications, which have been repaired by the corresponding maintainers.
Decentralized finance (DeFi) protocols are crypto projects developed on the blockchain to manage digital assets. Attacks on DeFi have been frequent and have resulted in losses exceeding \$77 billion. However, detection methods for malicious DeFi events are still lacking. In this paper, we propose DeFiTail, the first framework that utilizes deep learning to detect access control and flash loan exploits that may occur on DeFi. Since the DeFi protocol events involve invocations with multi-account transactions, which requires execution path unification with different contracts. Moreover, to mitigate the impact of mistakes in Control Flow Graph (CFG) connections, we validate the data path by employing the symbolic execution stack. Furthermore, we feed the data paths through our model to achieve the inspection of DeFi protocols. Experimental results indicate that DeFiTail achieves the highest accuracy, with 98.39% in access control and 97.43% in flash loan exploits. DeFiTail also demonstrates an enhanced capability to detect malicious contracts, identifying 86.67% accuracy from the CVE dataset.
Pedro Delgado‐Pérez, Ignacio Meléndez‐Lapi, Juan Boubeta-Puig
Abstract Smart contracts (SC) are programs embodying certain business logic stored on a blockchain network like Ethereum. The execution of transactions on SC has a cost, measured in gas units, that depends on the low‐level operations performed. Therefore, a poor choice of high‐level language constructs could lead to overcharging users for their transactions. Thus, a testing process focused on possible deviations of the gas used in diverse scenarios could provide substantial global savings. This paper presents a gas‐centered mutation testing approach for taking care of the gas consumed by Solidity SCs. This approach can be useful to improve the test quality to detect gas‐related problems, reason about performance issues that only manifest in certain situations, and identify alternative more optimal implementations. We define and implement several mutation operators specifically designed to perturb gas consumption while preserving contract semantics in general. Our experiments using several real‐world SCs show the feasibility of the technique, with some mutants reproducing meaningful differences in the consumption and exposing some gas limits not tight enough in historic transactions. Therefore, our approach is shown to be a good ally to prevent the appearance of gas‐related issues and lays the groundwork for researchers seeking to improve performance testing practices.
Decentralized Finance (DeFi) has emerged as a contemporary competitive as well as complementary to traditional centralized finance systems. As of 23rd January 2024, per Defillama approximately USD 55 billion is the total value locked on the DeFi applications on all blockchains put together. A Byzantine Fault Tolerant (BFT) State Machine Replication (SMR) protocol, popularly known as the consensus protocol, is the central component of a blockchain. If forks are possible in a consensus protocol, they can be misused to carry out double spending attacks and can be catastrophic given high volumes of finance that are transacted on blockchains. Formal verification of the safety of consensus protocols is the golden standard for guaranteeing that forks are not possible. However, it is considered complex and challenging to do. This is reflected by the fact that not many complex consensus protocols are formally verified except for Tendermint and QBFT. We focus on Supra's Pipelined Moonshot consensus protocol. Similar to Tendermint's formal verification, we too model Pipelined Moonshot using IVy and formally prove that for all network sizes, as long as the number of Byzantine validators is less than one thirds, the protocol does not allow forks, thus proving that Pipelined Moonshot is safe and double spending cannot be done using forks. The IVy model and proof of safety is available on Github.
Krzysztof Gogol, Benjamin Kraner, Malte Schlosser, Tao Yan · 6 authors
Liquid staking has become the largest category of decentralized finance protocols in terms of total value locked. However, few studies exist on its implementation designs or underlying risks. The liquid staking protocols allow for earning staking rewards without the disadvantage of locking the capital at the validators. Yet, they are seen by some as a threat to the Proof-of-Stake blockchain security. This paper is the first work that classifies liquid staking implementations. It analyzes the historical performance of major liquid staking tokens in comparison to the traditional staking for the largest Proof-of-Stake blockchains. Furthermore, the research investigates the impact of centralization, maximum extractable value and the migration of Ethereum from Proof-of-Work to Proof-of-Stake on the tokens' performance. Examining the tracking error of the liquid stacking providers to the staking rewards shows that they are persistent and cannot be explained by macro-variables of the currency, such as the variance or return.
As blockchains shift from energy-hungry Proof-of-Work to capital-intensive Proof-of-Stake, they trade electricity costs for a new vulnerability: the risk of a capital run that can destabilize consensus and security. We model investors who choose between staking their coin to earn rewards or exiting to cash out, potentially triggering mass withdrawals. These "staking runs" are more likely when protocols are weak, when failure would hit coin prices hard, or when staking rewards are low. Leverage worsens things: margin calls accelerate exits and amplify run dynamics. Longer lock-up periods slow the run but may not prevent it. Previous research shows that low rewards are good for protocol security. We show they also raise the risk of a run. A run on a major Proof-of-Stake chain---like Ethereum---could destabilize the entire crypto ecosystem, threatening DeFi platforms that depend on it.