Replicated databases and permissioned blockchain systems rely on Byzantine Fault-Tolerant (BFT) consensus to maintain a globally consistent order of transactions across distributed replicas. These protocols preserve safety even under asynchrony, as they commit a transaction only after agreement among a strong quorum of replicas. During network partitions, however, when no strong quorum is reachable, they lose liveness and cannot make useful progress. In this paper, we present Cassandra, a consensus protocol that enables partial progress without sacrificing safety. Cassandra achieves this through a two-tier certification framework that decouples availability from commitment, allowing each partition to extend its own chain and reconcile these chains once the network is restored. To support this, Cassandra introduces a pacemaker that advances views without requiring a strong quorum and calibrates each replica's timeout off the critical path. Our evaluation results show that Cassandra remains competitive with state-of-the-art BFT protocols under stable conditions, sustaining 900K TPS at 16 replicas and 480K TPS at 104 replicas, with latency ranging from 0.31s at 16 replicas to 0.75s at 104 replicas. Under severe partitions, Cassandra maintains non-zero speculative throughput through PoA-backed progress, preserving work that can be reconciled once connectivity is restored.
The literature on blockchain-based databases is divided into permissioned blockchains and permissionless account-based blockchains. However, the former is not fully decentralized, and the latter suffers from challenges in scalability and practicality. We propose SpendableStore, a hybrid on/off-chain database that operates on top of permissionless UTXO-based blockchains as a novel approach to the problem of data decentralization. Our design integrates atomic data units into individual UTXOs to create a new blockchain concept called Spendable Data Objects that perform traditional CRUD operations. The integrity, immutability, and ownership of these Spendable Data Objects are safeguarded directly by the blockchain peer nodes, thus constraining the power of database administrators to achieve true data decentralization. We further support database transactions and propose an isolation mechanism called Future Now Snapshot Isolation to reason about transactional correctness in SpendableStore. We performed experiments on a major blockchain's Mainnet and observed up to 16x better throughput compared to a state-of-the-art blockchain-based database.
Zhengqing Liu, Alberto Sonnino, Igor Zablotchi, Eleftherios Kokoris-Kogias · 5 authors
Modern blockchains rely on a modular architecture that decouples consensus from execution. Recent advances in consensus algorithms have shifted the bottleneck to the execution layer, which must deterministically follow the consensus order and handle increasingly complex, compute-intensive smart contracts. We identify that single-node validators cannot keep up, motivating the need for a scale-out design. We design Remora, a scale-out smart contract execution engine. Remora adopts an efficient asymmetric architecture with centralized transaction dispatching and distributed execution, and depends on an object versioning scheme with a strict ownership model to guarantee deterministic scale-out execution. Remora achieves up to 3x throughput improvement compared to state-of-the-art deterministic execution schemes, scales up to 250k TPS, matching modern consensus performance, and reduces latency by up to 5ms. We also show that Remora elastically adapts to bursty workloads and dynamic access patterns using real-world traces. Remora's main performance benefits come from a novel stateless-stateful separation during smart contract execution, which overlaps the execution of state-independent tasks with consensus, and a new locality-aware and load-balanced scheduling scheme.
Knob tuning plays a critical role in improving the performance of permissioned blockchains. However, efficient tuning remains challenging due to the architectural complexity of blockchains and the semantic gap between knob-specific logic and the numerical optimization requirements of tuning tools. In addition, configuration changes are often coupled across different stages of the transaction pipeline, making their performance impact difficult to isolate and predict. Since each trial requires deployment and distributed benchmarking, ineffective exploration incurs substantial cost. These challenges motivate BCTuner, a Large Language Model (LLM)-guided framework that combines knowledge-guided reasoning with structured search. BCTuner organizes multi-source tuning knowledge to support LLM-based reasoning over knob semantics, constraints, and deployment context. It formulates tuning as a Monte Carlo Tree Search (MCTS) process over structured action trajectories, where configurations are incrementally constructed, validated, evaluated, and refined rather than generated in one step. BCTuner further applies adaptive pruning to discard infeasible or low-potential branches before system evaluation. We evaluate BCTuner on Hyperledger Fabric and ChainMaker under diverse workloads and network settings. Experimental results show that BCTuner achieves up to 211.38% throughput improvement over default configurations and outperforms the state-of-the-art blockchain tuning method by up to 20% in performance, while requiring up to 8x fewer interactions with the blockchain system.
Data is critical for the operation of any organization and needs to be protected, especially against attacks that compromise the state of the database. In this paper, we explore an approach based on Byzantine-fault tolerant replicated state machines, built on top of a deterministic extension of PostgreSQL. Each replica deterministically executes transactions recorded in a shared log/blockchain. Our focus is on creating a practical system that is designed for efficient and quick detection of corruption, as well as quick repair concurrent with execution of transactions. We also present a performance study showing the efficiency and practicality of our approach. We believe our work lays the foundations for the practical use of the BFT replicated state machine approach in the context of databases.
This study presents a structured dataset of blockchain-registered artificial intelligence agents under the ERC-8004 standard on Ethereum. The dataset integrates on-chain identity records, minting transactions, transfer events, reputation summaries, and individual feedback records, together with resolved off-chain metadata where available. Data were collected from Ethereum mainnet using Web3 RPC queries and processed into tabular form to enable reproducible analysis. The dataset covers 10,000 agents within a defined block range and includes both event-level records and aggregated summaries. It enables empirical research on agent identity formation, reputation systems, service exposure, and early-stage decentralized AI ecosystems. This resource supports studies in blockchain analytics, decentralized trust infrastructure, and the emerging agentic economy.
Luigi Crisci, Lorenz Schuler, Herbert Jordan, Bernhard Scholz
The Ethereum state database uses Merkle Patricia Trie (MPT), which suffers from large witness proof sizes and high storage overhead. Verkle Tries have been proposed as a replacement, offering witness proofs below 150 bytes through vector commitments and Inner Product Argument aggregation. However, deploying a Verkle Trie in a high-throughput, short block-time blockchain such as Sonic, which produces a block every 300 milliseconds, introduces substantial engineering challenges related to storage efficiency, commitment computation costs, and the need to serve both live and historical state queries in real time. We present SonicDB S6, a production-grade Rust Verkle Trie database for the Sonic blockchain, which leverages its non-forking property to enable aggressive storage optimizations. Occupancy-aware node specializations, selected via an $\mathcal{O}(k n^2)$ dynamic program, reduce live storage by 97.8\%. Delta nodes that record only changed slots reduce archive storage by 95\%. Batched updates, multi-threaded commitment computation, and homomorphic Pedersen caching yield $3.2\times$ higher throughput than a persistent Geth Verkle baseline while sustaining production block-rate performance.
Optimizing asset exchanges on blockchain-driven platforms poses a novel and challenging graph query optimization problem. In this model, assets represent vertices and exchanges form edges, recasting the graph query task as a routing problem over a large-scale, dynamic graph. However, the existing solutions fail to solve the problem efficiently due to the non-linear nature of the edge weights defined by a concave swap function. To address the challenge, we propose PRIME, a two-stage iterative graph algorithm designed for the Token Graph Routing Problem (TGRP). The first stage employs a pruned graph search to efficiently identify a set of high-potential routing paths. The second stage formulates the allocation task as a strongly convex optimization problem, which we solve using our novel Adaptive Sign Gradient Method (ASGM) with a linear convergence rate. Extensive experiments on real-world Ethereum data confirm PRIME's advantages over industry baselines. PRIME consistently outperforms the widely-used Uniswap routing algorithm, achieving up to 8.42 basis points (bps) better execution prices on large trades while reducing computation up to 96.7%. The practicality of PRIME is further validated by its deployment in hedge fund production environments, demonstrating its viability as a scalable graph query processing solution for high-frequency decentralized markets.
Dense retrieval services increasingly underpin semantic search, recommendation, and retrieval-augmented generation, yet clients typically receive only a top-$k$ list with no auditable evidence of how it was produced. We present V3DB, a verifiable, versioned vector-search service that enables audit-on-demand correctness checks for approximate nearest-neighbour (ANN) retrieval executed by a potentially untrusted service provider. V3DB commits to each corpus snapshot and standardises an IVF-PQ search pipeline into a fixed-shape, five-step query semantics. Given a public snapshot commitment and a query embedding, the service returns the top-$k$ payloads and, when challenged, produces a succinct zero-knowledge proof that the output is exactly the result of executing the published semantics on the committed snapshot -- without revealing the embedding corpus or private index contents. To make proving practical, V3DB avoids costly in-circuit sorting and random access by combining multiset equality/inclusion checks with lightweight boundary conditions. Our prototype implementation based on Plonky2 achieves up to $22\times$ faster proving and up to $40\%$ lower peak memory consumption than the circuit-only baseline, with millisecond-level verification time. Github Repo at https://github.com/TabibitoQZP/zk-IVF-PQ.
Blockchain provides a decentralized and tamper-resistant ledger for securely recording transactions across a network of untrusted nodes. While its transparency and integrity are beneficial, the substantial storage requirements for maintaining a complete transaction history present significant challenges. For example, Ethereum nodes require around 23TB of storage, with an annual growth rate of 4TB. Prior studies have employed various strategies to mitigate the storage challenges. Notably, COLE significantly reduces storage size and improves throughput by adopting a column-based design that incorporates a learned index, effectively eliminating data duplication in the storage layer. However, this approach has limitations in supporting chain reorganization during blockchain forks and state pruning to minimize storage overhead. In this paper, we propose COLE$^+$, an enhanced storage solution designed to address these limitations. COLE$^+$ incorporates a novel rewind-supported in-memory tree structure for handling chain reorganization, leveraging content-defined chunking (CDC) to maintain a consistent hash digest for each block. For on-disk storage, a new two-level Merkle Hash Tree (MHT) structure, called prunable version tree, is developed to facilitate efficient state pruning. Both theoretical and empirical analyses show the effectiveness of COLE$^+$ and its potential for practical application in real-world blockchain systems.
Transaction flow networks are crucial in detecting illicit activities such as wash trading, credit card fraud, cashback arbitrage fraud, and money laundering. \revise{Our collaborator, Grab, a leader in digital payments in Southeast Asia, faces increasingly sophisticated fraud patterns in its transaction flow networks. In industry settings such as Grab's fraud detection pipeline, identifying fraudulent activities heavily relies on detecting dense flows within transaction networks. Motivated by this practical foundation,} we propose the \emph{\(S\)-\(T\) densest flow} (\SDMF{}) query. Given a transaction flow network \( G \), a source set \( \Src \), a sink set \( \Dst \), and a size threshold \( k \), the query outputs subsets \( \Src' \subseteq \Src \) and \( \Dst' \subseteq \Dst \) such that the maximum flow from \( \Src' \) to \( \Dst' \) is densest, with \(|\Src' \cup \Dst'| \geq k\). Recognizing the NP-hardness of the \SDMF{} query, we develop an efficient divide-and-conquer algorithm, CONAN. \revise{Driven by industry needs for scalable and efficient solutions}, we introduce an approximate flow-peeling algorithm to optimize the performance of CONAN, enhancing its efficiency in processing large transaction networks. \revise{Our approach has been integrated into Grab's fraud detection scenario, resulting in significant improvements in identifying fraudulent activities.} Experiments show that CONAN outperforms baseline methods by up to three orders of magnitude in runtime and more effectively identifies the densest flows. We showcase CONAN's applications in fraud detection on transaction flow networks from our industry partner, Grab, and on non-fungible tokens (NFTs).
Andrey Chursin, Lefteris Kokoris-Kogias, Alex Orlov, Alberto Sonnino · 5 authors
Log-Structured Merge-Trees (LSM-trees) dominate persistent key-value storage but suffer from high write amplification from 10x to 30x under random workloads due to repeated compaction. This overhead becomes prohibitive for large values with uniformly distributed keys, a workload common in content-addressable storage, deduplication systems, and blockchain validators. We present Tidehunter, a storage engine that eliminates value compaction by treating the Write-Ahead Log (WAL) as permanent storage rather than a temporary recovery buffer. Values are never overwritten; and small, lazily-flushed index tables map keys to WAL positions. Tidehunter introduces (a) lock-free writes that saturate NVMe drives through atomic allocation and parallel copying, (b) an optimistic index structure that exploits uniform key distributions for single-roundtrip lookups, and (c) epoch-based pruning that reclaims space without blocking writes. On a 1 TB dataset with 1 KB values, Tidehunter achieves 830K writes per second, that is 8.4x higher than RocksDB and 2.9x higher than BlobDB, while improving point queries by 1.7x and existence checks by 15.6x. We validate real-world impact by integrating Tidehunter into Sui, a high-throughput blockchain, where it maintains stable throughput and latency under loads that cause RocksDB-backed validators to collapse. Tidehunter is production-ready and is being deployed in production within Sui.
In primary-backup replication, consensus latency is bounded by the time for backup nodes to replay (re-execute) transactions proposed by the primary. In this work, we present Ira, a framework to accelerate backup replay by transmitting compact \emph{hints} alongside transaction batches. Our key insight is that the primary, having already executed transactions, possesses knowledge of future access patterns which is exactly the information needed for optimal replay. We use Ethereum for our case study and present a concrete protocol, Ira-L, within our framework to improve cache management of Ethereum block execution. The primaries implementing Ira-L provide hints that consist of the working set of keys used in an Ethereum block and one byte of metadata per key indicating the table to read from, and backups use these hints for efficient block replay. We evaluated Ira-L against the state-of-the-art Ethereum client reth over two weeks of Ethereum mainnet activity ($100,800$ blocks containing over $24$ million transactions). Our hints are compact, adding a median of $47$ KB compressed per block ($\sim5\%$ of block payload). We observe that the sequential hint generation and block execution imposes a $28.6\%$ wall-time overhead on the primary, though the direct cost from hints is $10.9\%$ of execution time; all of which can be pipelined and parallelized in production deployments. On the backup side, we observe that Ira-L achieves a median per-block speedup of $25\times$ over baseline reth. With $16$ prefetch threads, aggregate replay time drops from $6.5$ hours to $16$ minutes ($23.6\times$ wall-time speedup).
Modern blockchain applications benefit from the ability to specify sequencing constraints on the transactions that interact with them. This paper proposes a principled and axiomatically justified way of adding sequencing constraints on smart contract function calls that balances expressivity with the tractability of block production. Specifically, we propose a system in which contract developers are allowed to set an integer global priority for each of their calls, so long as that the call's chosen priority is no higher than the priority of any of its referenced calls. Block builders must then simply sequence transactions in priority order (from high to low priority), breaking ties however they would like. We show that this system is the unique system that satisfies five independent axioms.
Decentralized lending protocols, exemplified by Aave V3, have transformed financial intermediation by enabling permissionless, multi-chain borrowing and lending without intermediaries. Despite managing over $10 billion in total value locked, empirical research remains severely constrained by the lack of standardized, cross-chain event-level datasets. This paper introduces the first comprehensive, event-driven data infrastructure for Aave V3 spanning six major EVM-compatible chains (Ethereum, Arbitrum, Optimism, Polygon, Avalanche, and Base) from respective deployment blocks through October 2025. We collect and fully decode eight core event types -- Supply, Borrow, Withdraw, Repay, LiquidationCall, FlashLoan, ReserveDataUpdated, and MintedToTreasury -- producing over 50 million structured records enriched with block metadata and USD valuations. Using an open-source Python pipeline with dynamic batch sizing and automatic sharding (each file less than or equal to 1 million rows), we ensure strict chronological ordering and full reproducibility. The resulting publicly available dataset enables granular analysis of capital flows, interest rate dynamics, liquidation cascades, and cross-chain user behavior, providing a foundational resource for future studies on decentralized lending markets and systemic risk.
Jordan, Herbert, Jezek, Kamil, Subotic, Pavle, Scholz, Bernhard
The State Database of a blockchain stores account data and enables authentication. Modern blockchains use fast consensus protocols to avoid forking, improving throughput and finality. However, Ethereum's StateDB was designed for a forking chain that maintains multiple state versions. While newer blockchains adopt Ethereum's standard for DApp compatibility, they do not require multiple state versions, making legacy Ethereum databases inefficient for fast, non-forking blockchains. Moreover, existing StateDB implementations have been built on key-value stores (e.g., LevelDB), which make them less efficient. This paper introduces a novel state database that is a native database implementation and maintains Ethereum compatibility while being specialized for non-forking blockchains. Our database delivers ten times speedups and 99% space reductions for validators, and a threefold decrease in storage requirements for archive nodes.
Digital forensics faces unprecedented challenges with the emergence of digital twins and metaverse technologies. This paper presents the first comparative analysis between blockchain-based and traditional database systems for managing digital twin evidence in forensic investigations. We conducted controlled experiments comparing the Ethereum blockchain with IPFS storage against traditional SQL databases for digital twin evidence management. Our findings reveal that while blockchain provides superior data integrity and immutability, crucial for forensic applications, traditional databases offer better performance consistency. The blockchain implementation showed faster average storage times but higher variability in retrieval operations. Both systems maintained forensic integrity through hash verification, though blockchain's immutable nature provides additional security guarantees essential for legal proceedings. This research contributes to the development of robust digital forensic methodologies for emerging technologies in the metaverse era.
The potential of agricultural data (AgData) to drive efficiency and sustainability is stifled by the "AgData Paradox": a pervasive lack of trust and interoperability that locks data in silos, despite its recognized value. This paper introduces AgriTrust, a federated semantic governance framework designed to resolve this paradox. AgriTrust integrates a multi-stakeholder governance model, built on pillars of Data Sovereignty, Transparent Data Contracts, Equitable Value Sharing, and Regulatory Compliance, with a semantic digital layer. This layer is realized through the AgriTrust Core Ontology, a formal OWL ontology that provides a shared vocabulary for tokenization, traceability, and certification, enabling true semantic interoperability across independent platforms. A key innovation is a blockchain-agnostic, multi-provider architecture that prevents vendor lock-in. The framework's viability is demonstrated through case studies across three critical Brazilian supply chains: coffee (for EUDR compliance), soy (for mass balance), and beef (for animal tracking). The results show that AgriTrust successfully enables verifiable provenance, automates compliance, and creates new revenue streams for data producers, thereby transforming data sharing from a trust-based dilemma into a governed, automated operation. This work provides a foundational blueprint for a more transparent, efficient, and equitable agricultural data economy.
Setchain has been proposed to increase blockchain scalability by relaxing the strict total order requirement among transactions. Setchain organizes elements into a sequence of sets, referred to as epochs, so that elements within each epoch are unordered. In this paper, we propose and evaluate three distinct Setchain algorithms, that leverage an underlying block-based ledger. Vanilla is a basic implementation that serves as a reference point. Compresschain aggregates elements into batches, and compresses these batches before appending them as epochs in the ledger. Hashchain converts batches into fixed-length hashes which are appended as epochs in the ledger. This requires Hashchain to use a distributed service to obtain the batch contents from its hash. To allow light clients to safely interact with only one server, the proposed algorithms maintain, as part of the Setchain, proofs for the epochs. An epoch-proof is the hash of the epoch, cryptographically signed by a server. A client can verify the correctness of an epoch with $f+1$ epoch-proofs (where $f$ is the maximum number of Byzantine servers assumed). All three Setchain algorithms are implemented on top of the CometBFT blockchain application platform. We conducted performance evaluations across various configurations, using clusters of four, seven, and ten servers. Our results show that the Setchain algorithms reach orders of magnitude higher throughput than the underlying blockchain, and achieve finality with latency below 4 seconds.
Hybrid storage solutions have emerged as potent strategies to alleviate the data storage bottlenecks prevalent in blockchain systems. These solutions harness off-chain Storage Services Providers (SPs) in conjunction with Authenticated Data Structures (ADS) to ensure data integrity and accuracy. Despite these advancements, the reliance on centralized SPs raises concerns about query correctness. Although ADS can verify the existence of individual query results, they fall short of preventing SPs from omitting valid results. In this paper, we delineate the fundamental distinctions between data search in blockchains and traditional database systems. Drawing upon these insights, we introduce BPI, a lightweight framework that enables efficient keyword queries and maintenance with low overhead. We propose "Articulated Search", a query pattern specifically designed for blockchain environments that enhances search efficiency while significantly reducing costs during data user updates. Furthermore, BPI employs a suite of validation models to ensure the inclusion of all valid content in search results while maintaining low overhead. Extensive experimental evaluations demonstrate that the BPI framework achieves outstanding scalability and performance in keyword searches within blockchain, surpassing EthMB+ and state of the art search databases commonly used in mainstream hybrid storage blockchains (HSB).
Herbert Jordan, Kamil Jezek, Pavle Subotic, Bernhard Scholz
Operating nodes in an L1 blockchain remains costly despite recent advances in blockchain technology. One of the most resource-intensive components of a node is the blockchain database, also known as StateDB, that manages balances, nonce, code, and the persistent storage of accounts/smart contracts. Although the blockchain industry has transitioned from forking to forkless chains due to improved consensus protocols, forkless blockchains still rely on legacy forking databases that are suboptimal for their purposes. In this paper, we propose a forkless blockchain database, showing a 100x improvement in storage and a 10x improvement in throughput compared to the geth-based Fantom Blockchain client.
Due to HIPAA and other privacy regulations, it is imperative to maintain patient privacy while conducting research on patient health records. In this paper, we propose AegisBlock, a patient-centric access controlled framework to share medical records with researchers such that the anonymity of the patient is maintained while ensuring the trustworthiness of the data provided to researchers. AegisBlock allows for patients to provide access to their medical data, verified by miners. A researcher submits a time-based range query to request access to records from a certain patient, and upon patient approval, access will be granted. Our experimental evaluation results show that AegisBlock is scalable with respect to the number of patients and hospitals in the system, and efficient with up to 50% of malicious miners.
Synthetic time series are essential tools for data augmentation, stress testing, and algorithmic prototyping in quantitative finance. However, in cryptocurrency markets, characterized by 24/7 trading, extreme volatility, and rapid regime shifts, existing Time Series Generation (TSG) methods and benchmarks often fall short, jeopardizing practical utility. Most prior work (1) targets non-financial or traditional financial domains, (2) focuses narrowly on classification and forecasting while neglecting crypto-specific complexities, and (3) lacks critical financial evaluations, particularly for trading applications. To address these gaps, we introduce \textsf{CTBench}, the first comprehensive TSG benchmark tailored for the cryptocurrency domain. \textsf{CTBench} curates an open-source dataset from 452 tokens and evaluates TSG models across 13 metrics spanning 5 key dimensions: forecasting accuracy, rank fidelity, trading performance, risk assessment, and computational efficiency. A key innovation is a dual-task evaluation framework: (1) the \emph{Predictive Utility} task measures how well synthetic data preserves temporal and cross-sectional patterns for forecasting, while (2) the \emph{Statistical Arbitrage} task assesses whether reconstructed series support mean-reverting signals for trading. We benchmark eight representative models from five methodological families over four distinct market regimes, uncovering trade-offs between statistical fidelity and real-world profitability. Notably, \textsf{CTBench} offers model ranking analysis and actionable guidance for selecting and deploying TSG models in crypto analytics and strategy development.
Ethereum, a leading blockchain platform, has revolutionized the digital economy by enabling decentralized transactions and the execution of smart contracts. Ethereum transactions form the backbone of its network, facilitating peer-to-peer exchanges and interactions with complex decentralized applications. Smart contracts extend Ethereum's capabilities by automating processes and enabling trustless execution of agreements. Hence, understanding how these smart contracts interact is important in order to facilitate various performance optimizations, such as warming objects before they are being accessed and enabling concurrent execution. Of particular interest to us are the development of the calling graph, as well as the read sets and write sets of invocations within the same block, and the properties of the associated conflict graph that is derived from them. The latter is important for understanding the parallelization potential of smart contracts on Ethereum. We traced upwards of 2 million recent Ethereum blocks using call tracer and prestate tracer, out of a total of 21.4 million blocks at the time of writing. We report on the transactions per block distribution, the structure of call trees in smart contract invocations, the ratio of value-transfer transactions to smart contract invocations, as well as provide a comprehensive study of the structure of blocks' conflict graphs. We find that conflict graphs predominantly show a star like configuration, as well as other noteworthy structural properties.