Herbert Jordan, Kamil Jezek, Pavle Subotic, Bernhard Scholz
Operating nodes in an L1 blockchain remains costly despite recent advances in blockchain technology. One of the most resource-intensive components of a node is the blockchain database, also known as StateDB, that manages balances, nonce, code, and the persistent storage of accounts/smart contracts. Although the blockchain industry has transitioned from forking to forkless chains due to improved consensus protocols, forkless blockchains still rely on legacy forking databases that are suboptimal for their purposes. In this paper, we propose a forkless blockchain database, showing a 100x improvement in storage and a 10x improvement in throughput compared to the geth-based Fantom Blockchain client.
Due to HIPAA and other privacy regulations, it is imperative to maintain patient privacy while conducting research on patient health records. In this paper, we propose AegisBlock, a patient-centric access controlled framework to share medical records with researchers such that the anonymity of the patient is maintained while ensuring the trustworthiness of the data provided to researchers. AegisBlock allows for patients to provide access to their medical data, verified by miners. A researcher submits a time-based range query to request access to records from a certain patient, and upon patient approval, access will be granted. Our experimental evaluation results show that AegisBlock is scalable with respect to the number of patients and hospitals in the system, and efficient with up to 50% of malicious miners.
Synthetic time series are essential tools for data augmentation, stress testing, and algorithmic prototyping in quantitative finance. However, in cryptocurrency markets, characterized by 24/7 trading, extreme volatility, and rapid regime shifts, existing Time Series Generation (TSG) methods and benchmarks often fall short, jeopardizing practical utility. Most prior work (1) targets non-financial or traditional financial domains, (2) focuses narrowly on classification and forecasting while neglecting crypto-specific complexities, and (3) lacks critical financial evaluations, particularly for trading applications. To address these gaps, we introduce \textsf{CTBench}, the first comprehensive TSG benchmark tailored for the cryptocurrency domain. \textsf{CTBench} curates an open-source dataset from 452 tokens and evaluates TSG models across 13 metrics spanning 5 key dimensions: forecasting accuracy, rank fidelity, trading performance, risk assessment, and computational efficiency. A key innovation is a dual-task evaluation framework: (1) the \emph{Predictive Utility} task measures how well synthetic data preserves temporal and cross-sectional patterns for forecasting, while (2) the \emph{Statistical Arbitrage} task assesses whether reconstructed series support mean-reverting signals for trading. We benchmark eight representative models from five methodological families over four distinct market regimes, uncovering trade-offs between statistical fidelity and real-world profitability. Notably, \textsf{CTBench} offers model ranking analysis and actionable guidance for selecting and deploying TSG models in crypto analytics and strategy development.
Ethereum, a leading blockchain platform, has revolutionized the digital economy by enabling decentralized transactions and the execution of smart contracts. Ethereum transactions form the backbone of its network, facilitating peer-to-peer exchanges and interactions with complex decentralized applications. Smart contracts extend Ethereum's capabilities by automating processes and enabling trustless execution of agreements. Hence, understanding how these smart contracts interact is important in order to facilitate various performance optimizations, such as warming objects before they are being accessed and enabling concurrent execution. Of particular interest to us are the development of the calling graph, as well as the read sets and write sets of invocations within the same block, and the properties of the associated conflict graph that is derived from them. The latter is important for understanding the parallelization potential of smart contracts on Ethereum. We traced upwards of 2 million recent Ethereum blocks using call tracer and prestate tracer, out of a total of 21.4 million blocks at the time of writing. We report on the transactions per block distribution, the structure of call trees in smart contract invocations, the ratio of value-transfer transactions to smart contract invocations, as well as provide a comprehensive study of the structure of blocks' conflict graphs. We find that conflict graphs predominantly show a star like configuration, as well as other noteworthy structural properties.
Lunodzo J. Mwinuka, Massimo Cafaro, Lucas Pereira, Hugo Morais
Energy systems generate vast amounts of data in extremely short time intervals, creating challenges for efficient data management. Traditional data management methods often struggle with scalability and accessibility, limiting their usefulness. More advanced solutions, such as NoSQL databases and cloud-based platforms, have been adopted to address these issues. Still, even these advanced solutions can encounter bottlenecks, which can impact the efficiency of data storage, retrieval, and analysis. This review paper explores the research trends in big data management for energy systems, highlighting the practices, opportunities and challenges. Also, the data regulatory demands are highlighted using chosen reference architectures. The review, in particular, explores the limitations of current storage and data integration solutions and examines how new technologies are applied to the energy sector. Novel insights into emerging technologies, including data spaces, various data management architectures, peer-to-peer data management, and blockchains, are provided, along with practical recommendations for achieving enhanced data sharing and regulatory compliance.
Although differential privacy (DP) is widely regarded as the de facto standard for data privacy, its implementation remains vulnerable to unfaithful execution by servers, particularly in distributed settings. In such cases, servers may sample noise from incorrect distributions or generate correlated noise while appearing to follow established protocols. This work addresses these malicious behaviours in a distributed client-server-verifier setup, under Verifiable Distributed Differential Privacy (VDDP), a novel framework for the verifiable execution of distributed DP mechanisms. We systematically capture end-to-end security and privacy guarantees against potentially colluding adversarial behaviours of clients, servers, and verifiers by characterizing the connections and distinctions between VDDP and zero-knowledge proofs (ZKPs). We develop three novel and efficient instantiations of VDDP: (1) the Verifiable Distributed Discrete Laplace Mechanism (VDDLM), which achieves up to a 400,000x improvement in proof generation efficiency with only 0.1--0.2x error compared with the previous state-of-the-art verifiable differentially private mechanism and includes a tight privacy analysis that accounts for all additional privacy losses due to numerical imprecisions, applicable to other secure computation protocols for DP mechanisms based on cryptography; (2) the Verifiable Distributed Discrete Gaussian Mechanism (VDDGM), an extension of VDDLM that incurs limited overhead in real-world applications; and (3) an improved solution to Verifiable Randomized Response (VRR) under local DP, as a special case of VDDP, achieving up to a 5,000x reduction in communication costs and verifier overhead.
The convergence of blockchain and artificial intelligence (AI) has led to the emergence of AI-based tokens, which are cryptographic assets designed to power decentralized AI platforms and services. This paper provides a comprehensive review of leading AI-token projects, examining their technical architectures, token utilities, consensus mechanisms, and underlying business models. We explore how these tokens operate across various blockchain ecosystems and assess the extent to which they offer value beyond traditional centralized AI services. Based on this assessment, our analysis identifies several core limitations. From a technical perspective, many platforms depend extensively on off-chain computation, exhibit limited capabilities for on-chain intelligence, and encounter significant scalability challenges. From a business perspective, many models appear to replicate centralized AI service structures, simply adding token-based payment and governance layers without delivering truly novel value. In light of these challenges, we also examine emerging developments that may shape the next phase of decentralized AI systems. These include approaches for on-chain verification of AI outputs, blockchain-enabled federated learning, and more robust incentive frameworks. Collectively, while emerging innovations offer pathways to strengthen decentralized AI ecosystems, significant gaps remain between the promises and the realities of current AI-token implementations. Our findings contribute to a growing body of research at the intersection of AI and blockchain, highlighting the need for critical evaluation and more grounded approaches as the field continues to evolve.
Recent account allocation studies in sharded blockchains are typically miner-driven, requiring miners to perform global optimizations for all accounts to enhance system-wide performance. This forces each miner to maintain a complete copy of the entire ledger, resulting in significant storage, communication, and computation overhead. In this work, we explore an alternative research direction by proposing Mosaic, the first client-driven framework for distributed, lightweight local optimization. Rather than relying on miners to allocate all accounts, Mosaic enables clients to independently execute a local algorithm to determine their residing shards. Clients can submit migration requests to a beacon chain when relocation is necessary. Mosaic naturally addresses key limitations of miner-driven approaches, including the lack of miner incentives and the significant overhead. While clients are flexible to adopt any algorithm for shard allocation, we design and implement a reference algorithm, Pilot, to guide them. Clients execute Pilot to maximize their own benefits, such as reduced transaction fees and confirmation latency. On a real-world Ethereum dataset, we implement and evaluate Pilot against state-of-the-art miner-driven global optimization solutions. The results demonstrate that Mosaic significantly enhances computational efficiency, achieving a four-order-of-magnitude reduction in computation time, with the reduced input data size from 1.44 GB to an average of 228.66 bytes per account. Despite these efficiency gains, Pilot introduces only about a 5% increase in the cross-shard ratio and maintains approximately 98% of the system throughput, demonstrating a minimal trade-off in overall effectiveness.
This paper introduces Web3DB, a decentralized relational database management system (RDBMS) designed to align with the principles of Web 3.0, addressing critical shortcomings of traditional centralized DBMS, such as data privacy, security vulnerabilities, and single points of failure. Several similar systems have been proposed, but they are not compatible with the legacy systems based on RDBMS. Motivated by the necessity for enhanced data sovereignty and the decentralization of data control, Web3DB leverages blockchain technology for fine-grained access control and utilizes decentralized data storage. This system leverages a novel, modular architecture that contributes to enhanced flexibility, scalability, and user-centric functionality. Central to the Web3DB innovation is its decentralized query execution, which uses cryptographic sortition and blockchain verification to ensure secure and fair query processing across network nodes. The motivation for integrating relational databases within decentralized DBMS primarily stems from the need to combine the robustness and ease of use of relational database structures with the benefits of decentralization. This paper outlines the architecture of Web3DB, its practical implementation, and the system's ability to support SQL-like operations on relational data, manage multi-tenancy, and facilitate open data sharing, setting new standards for decentralized databases in the Web 3.0 era.
In the rapidly evolving landscape of digital assets and blockchain technologies, the necessity for robust, scalable, and secure data management platforms has never been more critical. This paper introduces a novel software architecture designed to meet these demands by leveraging the inherent strengths of cloud-native technologies and modular micro-service based architectures, to facilitate efficient data management, storage and access, across different stakeholders. We detail the architectural design, including its components and interactions, and discuss how it addresses common challenges in managing blockchain data and digital assets, such as scalability, data siloing, and security vulnerabilities. We demonstrate the capabilities of the platform by employing it into multiple real-life scenarios, namely providing data in near real-time to scientists in help with their research. Our results indicate that the proposed architecture not only enhances the efficiency and scalability of distributed data management but also opens new avenues for innovation in the research reproducibility area. This work lays the groundwork for future research and development in machine learning operations systems, offering a scalable and secure framework for the burgeoning digital economy.
The integration of blockchain technology with data analytics is essential for extracting insights in the cryptocurrency space. Although academic literature on blockchain data analytics is limited, various industry solutions have emerged to address these needs. This paper provides a comprehensive literature review, drawing from both academic research and industry applications. We classify blockchain analytics tools into categories such as block explorers, on-chain data providers, research platforms, and crypto market data providers. Additionally, we discuss the challenges associated with blockchain data analytics, including data accessibility, scalability, accuracy, and interoperability. Our findings emphasize the importance of bridging academic research and industry innovations to advance blockchain data analytics.
The ability of tracing states of logistic transportations requires an efficient storage and retrieval of the state of logistic transportations and locations of logistic objects. However, the restriction of sharing states and locations of logistic objects across organizations from different countries makes it hard to deploy a centralized database for implementing the traceability in a cross-border logistic system. This paper proposes a semantic data model on Blockchain to represent a logistic process based on the Semantic Link Network model where each semantic link represents a logistic transportation of a logistic object between two parties. A state representation model is designed to represent the states of a logistic transportation with semantic links. It enables the locations of logistic objects to be derived from the link states. A mapping from the semantic links to the blockchain transactions is designed to enable schema of semantic links and states of semantic links to be published in blockchain transactions. To improve the efficiency of tracing a path of semantic links on blockchain platform, an algorithm is designed to build shortcuts along the path of semantic links to enable a query on the path of a logistic object to reach the target in logarithmic steps on the blockchain platform. A reward-penalty policy is designed to allow participants to confirm the state of links on blockchain. Analysis and simulation demonstrate the flexibility, effectiveness and the efficiency of Semantic Link Network on immutable blockchain for implementing logistic traceability.
With its decentralization and immutability, blockchain has emerged as a trusted foundation for data management and querying. Because blockchain storage space is limited, large multimodal data files, such as videos, are often stored offline, leaving only lightweight metadata on the chain. While this hybrid storage approach enhances storage efficiency, it introduces significant challenges for executing advanced queries on multimodal data. The metadata stored on-chain is often minimal and may not include all the attributes necessary for queries like time range or fuzzy queries. In addition, existing blockchains do not provide native support for multimodal data querying. Achieving this capability would necessitate extensive modifications to the underlying blockchain framework, even reconstructing its core architecture. Consequently, enabling blockchains with multimodal query capabilities remains a significant problem, which necessitates overcoming the following three key challenges: (1) Designing efficient indexing methods to adapt to varying workloads that involve frequent insertions and query operations; (2) Achieving seamless integration with existing blockchains without altering the underlying infrastructure; (3) Ensuring high query performance while minimizing gas consumption. To address these challenges, we propose MulChain, a novel middleware architecture to enable smooth integration with existing blockchains. At the core of MulChain is the BHashTree, a flexible data structure that dynamically switches between tree and hash nodes based on workload characteristics, ensuring efficient insertion and query operations. Furthermore, the middleware provides standardized interfaces for blockchain systems, unifying query methods across different platforms.
As a valuable digital resource, graph data is an important data asset, which has been widely utilized across various fields to optimize decision-making and enable smarter solutions. To manage data assets, blockchain is widely used to enable data sharing and trading, but it cannot supply complex analytical queries. vChain was proposed to achieve verifiable boolean queries over blockchain by designing an embedded authenticated data structure (ADS). However, for generating (non-)existence proofs, vChain suffers from expensive storage and computation costs in ADS construction, along with high communication and verification costs. In this paper, we propose a novel NetChain framework that enables efficient top-k queries over on-chain graph data with verifiability. Specifically, we design a novel authenticated two-layer index that supports (non-)existence proof generation in block-level and built-in verifiability for matched objects. To further alleviate the computation and verification overhead, an optimized variant NetChain+ is derived. The authenticity of our frameworks is validated through security analysis. Evaluations show that NetChain and NetChain+ outperform vChain, respectively achieving up to 85X and 31X improvements on ADS construction. Moreover, compared with vChain, NetChain+ reduces the communication and verification costs by 87% and 96% respectively.
Isaac Zhang, Ryan Zarick, Daniel Wong, Thomas Kim · 7 authors
Quick Merkle Database (QMDB) addresses longstanding bottlenecks in blockchain state management by integrating key-value (KV) and Merkle tree storage into a single unified architecture. QMDB delivers a significant throughput improvement over existing architectures, achieving up to 6X over the widely used RocksDB and 8X over NOMT, a leading verifiable database. Its novel append-only twig-based design enables one SSD read per state access, O(1) IOs for updates, and in-memory Merkleization on a memory footprint as small as 2.3 bytes per entry, enabling it to run on even modest consumer-grade PCs. QMDB scales seamlessly across both commodity and enterprise hardware, achieving up to 2.28 million state updates per second. This performance enables support for 1 million token transfers per second (TPS), marking QMDB as the first solution achieving such a milestone. QMDB has been benchmarked with workloads exceeding 15 billion entries (10X Ethereum's 2024 state) and has proven the capacity to scale to 280 billion entries on a single server. Furthermore, QMDB introduces historical proofs, unlocking the ability to query its blockchain's historical state at the latest block. QMDB not only meets the demands of current blockchains but also provides a robust foundation for building scalable, efficient, and verifiable decentralized applications across diverse use cases.
The rise of blockchain and Digital Ledger Technology (DLT) has gained wide traction. Instead of relying on a traditional centralized data authority, a blockchain system consists of digitally entangled block data shared across a distributed network. The specially designed chain data structure and its consensus mechanism protect blockchain data from being tampered by unauthorized adversaries. However, implementing a full-fledged blockchain system to protect a database can be technically cumbersome. In this work, we introduce an in-database design, named chain table, to protect data integrity without the need for a blockchain system. It features a succinct design without significant technology barriers or storage overhead. To realize rigorous data security, we also propose a set of data writing principles for the chain table. We prove that the chain table, together with the data writing principles, will guarantee flexible data integrity, named table-level data integrity (TDI).
Dakai Kang, Junchao Chen, Tien Tuan Anh Dinh, Mohammad Sadoghi
The rise of cryptocurrencies like Bitcoin and Ethereum has driven interest in blockchain database technology, with smart contracts enabling the growth of decentralized finance (DeFi). However, research has shown that adversaries exploit transaction ordering to extract profits through attacks like front-running, sandwich attacks, and liquidation manipulation. This issue affects blockchains where block proposers have full control over transaction ordering. To address this, a more fair transaction ordering mechanism is essential. Existing fairness protocols, such as Pompe and Themis, operate on leader-based consensus protocols, which not only suffer from low throughput caused by the single-leader bottleneck, but also allow adversarial block proposers to manipulate transaction ordering. To address these limitations, we propose a new framework, FairDAG, that runs fairness protocols on top of DAG-based consensus protocols. FairDAG improves protocol performance in both throughput and fairness quality by leveraging the multi-proposer design and validity property of DAG-based consensus protocols. We conducted a comprehensive analytical and experimental evaluation of two FairDAG variants - FairDAG-AB and FairDAG-RL. Our results demonstrate that FairDAG outperforms prior fairness protocols in both throughput and fairness quality.
This paper presents an experimental framework for implementing Decentralized Autonomous Database Systems (DADBS) using the Rust programming language. As traditional centralized databases face challenges in scalability, security, and autonomy, DADBS emerge as a promising solution, using blockchain principles to create distributed, self-governing database systems. Our framework explores the practical aspects of building a DADBS, focusing on Rust's unique features that improves system reliability and performance. We evaluated our DADBS implementation across several key performance metrics: throughput, latency(read), latency(write), scalability, CPU utilization, Memory Usage and Network I/O, The average results obtained over a 24-hour period of continuous operation were 3,000 transactions/second, 75 ms, 250 ms, 55%, 2.5 GB, 100MB/s. The security analysis depicts that even with an increase in the percentage of malicious nodes, DADBS still maintains high throughput and consistency. The paper discusses key design decisions, highlighting how Rust's ownership model and concurrency features address common challenges in distributed systems. We also examine the current limitations of our approach and potential areas for future research. By providing this comprehensive overview of a Rust-based DADBS implementation, we aim to contribute to the growing body of knowledge on decentralized database architectures and their practical realization.
Modern database systems are expected to handle dynamic data whose characteristics may evolve over time. Many popular database benchmarks are limited in their ability to evaluate this dynamic aspect of the database systems. Those that use synthetic data generators often fail to capture the complexity and unpredictable nature of real data, while most real-world datasets are static and difficult to create high-volume, realistic updates for. This paper introduces CrypQ, a database benchmark leveraging dynamic, public Ethereum blockchain data. CrypQ offers a high-volume, ever-evolving dataset reflecting the unpredictable nature of a real and active cryptocurrency market. We detail CrypQ's schema, procedures for creating data snapshots and update sequences, and a suite of relevant SQL queries. As an example, we demonstrate CrypQ's utility in evaluating cost-based query optimizers on complex, evolving data distributions with real-world skewness and dependencies.
In database applications involving sensitive data, the dual imperatives of data confidentiality and provable (verifiable) query processing are important. This paper introduces PoneglyphDB, a database system that leverages non-interactive zero-knowledge proofs (ZKP) to support both confidentiality and provability. Unlike traditional databases, PoneglyphDB enhances confidentiality by ensuring that raw data remains exclusively with the host, while also enabling verifying the correctness of query responses by providing proofs to clients. The main innovation in this paper is proposing efficient ZKP designs (called circuits) for basic operations in SQL query processing. These basic operation circuits are then combined to form ZKP circuits for larger, more complex queries. PoneglyphDB's circuits are carefully designed to be efficient by utilizing advances in cryptography such as PLONKish-based circuits, recursive proof composition techniques, and designing with low-order polynomial constraints. We demonstrate the performance of PoneglyphDB with the standard TPC-H benchmark. Our experimental results show that PoneglyphDB can efficiently achieve both confidentiality and provability, outperforming existing state-of-the-art ZKP methods.
Blockchains are being positioned as the "technology of trust" that can be used to mediate transactions between non-trusting parties without the need for a central authority. They support transaction types that are native to the blockchain platform or user-defined via user programs called smart contracts. Despite the significant flexibility in transaction programmability that smart contracts offer, they pose several usability, robustness, and performance challenges. This paper proposes an alternative transaction framework that incorporates more primitives into the native set of transaction types (reducing the likelihood of requiring user-defined transaction programs often). The framework is based on the concept of declarative blockchain transactions whose strength lies in the fact that it addresses several of the limitations of smart contracts simultaneously. A formal and implementation framework is presented, and a subset of commonly occurring transaction behaviors are modeled and implemented as use cases, using an open-source blockchain database, BigchchainDB, as the implementation context. A performance study comparing the declarative transaction approach to equivalent smart contract transaction models reveals several advantages of the proposed approach.
Yunjie Pan, Omkar Bhalerao, C. Seshadhri, Nishil Talati
Counting the number of small subgraphs, called motifs, is a fundamental problem in social network analysis and graph mining. Many real-world networks are directed and temporal, where edges have timestamps. Motif counting in directed, temporal graphs is especially challenging because there are a plethora of different kinds of patterns. Temporal motif counts reveal much richer information and there is a need for scalable algorithms for motif counting. A major challenge in counting is that there can be trillions of temporal motif matches even with a graph with only millions of vertices. Both the motifs and the input graphs can have multiple edges between two vertices, leading to a combinatorial explosion problem. Counting temporal motifs involving just four vertices is not feasible with current state-of-the-art algorithms. We design an algorithm, TEACUPS, that addresses this problem using a novel technique of temporal path sampling. We combine a path sampling method with carefully designed temporal data structures, to propose an efficient approximate algorithm for temporal motif counting. TEACUPS is an unbiased estimator with provable concentration behavior, which can be used to bound the estimation error. For a Bitcoin graph with hundreds of millions of edges, TEACUPS runs in less than 1 minute, while the exact counting algorithm takes more than a day. We empirically demonstrate the accuracy of TEACUPS on large datasets, showing an average of 30$\times$ speedup (up to 2000$\times$ speedup) compared to existing GPU-based exact counting methods while preserving high count estimation accuracy.
Blockchain is widely applied in logistics, finance, and agriculture. As single blockchain users grow, scalability becomes crucial. However, existing works lack a comprehensive summary of blockchain scalability. They focus on single chains or cross-chain technologies. This survey summarizes scalability across the physical and logical layers, as well as inner-chain, inter-chain, and technology dimensions. The physical layer covers data and protocols, while the logical layer represents blockchain architecture. Each component is analyzed from inner-chain and inter-chain perspectives, considering technological factors. The aim is to enhance researchers' understanding of blockchain's architecture, data, and protocols to advance scalability research.
This paper presents an approach to using decentralized distributed digital (DDD) ledgers like blockchain with multi-level verification. In regular DDD ledgers like Blockchain, only a single level of verification is available, which makes it not useful for those systems where there is a hierarchy and verification is required on each level. In systems where hierarchy emerges naturally, the inclusion of hierarchy in the solution for the problem of the system enables us to come up with a better solution. Introduction to hierarchy means there could be several verification within a level in the hierarchy and more than one level of verification, which implies other challenges induced by an interaction between the various levels of hierarchies that also need to be addressed, like verification of the work of the previous level of hierarchy by given level in the hierarchy. The paper will address all these issues, and provide a road map to trace the state of the system at any given time and probability of failure of the system.