Blockchain Papers

Follow blockchain research across journals, conferences, and preprint repositories.

1,962 papersLast indexed Aug 31, 2026
Search papers

Paper index

1,962 results · page 21 of 82

Clear filters
Sep 18, 2024·arXiv
0 cites
Blockchain-Enabled IoV: Secure Communication and Trustworthy Decision-Making

Jingyi Sun, Qi Shi, Guodong Jin, Hao Xu · 5 authors

The Internet of Vehicles (IoV), which enables interactions between vehicles, infrastructure, and the environment, faces challenges in maintaining communication security and reliable automated decisions. This paper introduces a decentralized framework comprising a primary layer for managing inter-vehicle communication and a sub-layer for securing intra-vehicle interactions. By implementing blockchain-based protocols like Blockchain-integrated Secure Authentication (BiSA) and Decentralized Blockchain Name Resolution (DBNR), the framework ensures secure, decentralized identity management and reliable data exchanges, thereby supporting safe and efficient autonomous vehicle operations.

Open access
cs.CR
cs.DC
cs.NI
Original source
Sep 17, 2024·arXiv
0 cites
Delay Analysis of EIP-4844

Pourya Soltani, Farid Ashtiani

Proto-Danksharding, proposed in Ethereum Improvement Proposal 4844 (EIP-4844), aims to incrementally improve the scalability of the Ethereum blockchain by introducing a new type of transaction known as blob-carrying transactions. These transactions incorporate binary large objects (blobs) of data that are stored off-chain but referenced and verified on-chain to ensure data availability. By decoupling data availability from transaction execution, Proto-Danksharding alleviates network congestion and reduces gas fees, laying the groundwork for future, more advanced sharding solutions. This letter provides an analytical model to derive the delay for these new transactions. We model the system as an $\mathrm{M/D}^B/1$ queue which we then find its steady state distribution through embedding a Markov chain and use of supplementary variable method. We show that transactions with more blobs but less frequent impose higher delays on the system compared to lower blobs but more frequent.

Open access
cs.DC
Original source
Sep 16, 2024·arXiv (Cornell University)
0 cites
Deterministic Bounds in Committee Selection: Enhancing Decentralization and Scalability in Distributed Ledgers

Grigorii Melnikov, Sebastian Müller, Nikita Polyanskii, Yury Yanovich

Consensus plays a crucial role in distributed ledger systems, impacting both scalability and decentralization. Many blockchain systems use a weighted lottery based on a scarce resource such as a stake, storage, memory, or computing power to select a committee whose members drive the consensus and are responsible for adding new information to the ledger. Therefore, ensuring a robust and fair committee selection process is essential for maintaining security, efficiency, and decentralization. There are two main approaches to randomized committee selection. In one approach, each validator candidate locally checks whether they are elected to the committee and reveals their proof during the consensus phase. In contrast, in the second approach, a sortition algorithm decides a fixed-sized committee that is globally verified. This paper focuses on the latter approach, with cryptographic sortition as a method for fair committee selection that guarantees a constant committee size. Our goal is to develop deterministic guarantees that strengthen decentralization. We introduce novel methods that provide deterministic bounds on the influence of adversaries within the committee, as evidenced by numerical experiments. This approach overcomes the limitations of existing protocols that only offer probabilistic guarantees, often providing large committees that are impractical for many quorum-based applications like atomic broadcast and randomness beacon protocols.

Open access
2 source records
cs.DC
cs.CR
Game Theory and Voting Systems
Original source
Sep 12, 2024·arXiv
0 cites
Validated Strong Consensus Protocol for Asynchronous Vote-based Blockchains

Yibin Xu, Jianhua Shao, Tijs Slaats, Boris Düdder · 5 authors

Vote-based blockchains construct a state machine replication (SMR) system among participating nodes, using Byzantine Fault Tolerance (BFT) consensus protocols to transition from one state to another. Currently, they rely on either synchronous or partially synchronous networks with leader-based coordination or costly Asynchronous Common Subset (ACS) protocols in asynchronous settings, making them impractical for large-scale asynchronous applications. To make Asynchronous SMR scalable, this paper proposes a \emph{validated strong} BFT consensus model that allows leader-based coordination in asynchronous settings. Our BFT consensus model offers the same level of tolerance as binary byzantine agreement but does not demand consistency among honest nodes before they vote. An SMR using our model allows nodes to operate in different, tentative, but mutually exclusive states until they eventually converge on the same state. We propose an asynchronous BFT protocol for vote-based blockchains employing our consensus model to address several critical challenges: how to ensure that nodes eventually converge on the same state across voting rounds, how to assure that a blockchain will steadily progress through epochs while reaching consensus for previous epochs, and how to maintain robust byzantine fault tolerance. Our protocol greatly reduces message complexity and is the first one to achieve linear view changes without relying on threshold signatures. We prove that an asynchronous blockchain built on our protocol can operate with the \emph{same} simplicity and efficiency as partially synchronous blockchains built on, e.g. HotStuff-2. This facilitates deploying asynchronous blockchains across large-scale networks.

Open access
cs.DC
cs.CR
Original source
Sep 12, 2024·ICT Express, Volume 10, Issue 6, December 2024, Pages 1253-1258
12 cites
Prediction of Permissioned Blockchain Performance for Resource Scaling Configurations

Seungwoo Jung, Yeonho Yoo, Gyeongsik Yang, Chuck Yoo

Blockchain is increasingly offered as blockchain-as-a-service (BaaS) by cloud service providers. However, configuring BaaS appropriately for optimal performance and reliability resorts to try-and-error. A key challenge is that BaaS is often perceived as a ``black-box,'' leading to uncertainties in performance and resource provisioning. Previous studies attempted to address this challenge; however, the impacts of both vertical and horizontal scaling remain elusive. To this end, we present machine learning-based models to predict network reliability and throughput based on scaling configurations. In our evaluation, the models exhibit prediction errors of ~1.9%, which is highly accurate and can be applied in the real-world.

Open access
2 source records
cs.DC
cs.LG
eess.SY
Original source
Sep 10, 2024·Foundations and Trends in Information Systems: Vol. 7: No. 4, pp 310-356 (2024)
0 cites
Responsible Blockchain: STEADI Principles and the Actor-Network Theory-based Development Methodology (ANT-RDM)

Yibai Li, Ahmed Gomaa, Xiaobing Li

This paper provides a comprehensive analysis of the challenges and controversies associated with blockchain technology. It identifies technical challenges such as scalability, security, privacy, and interoperability, as well as business and adoption challenges, and the social, economic, ethical, and environmental controversies present in current blockchain systems. We argue that responsible blockchain development is key to overcoming these challenges and achieving mass adoption. This paper defines Responsible Blockchain and introduces the STEADI principles (sustainable, transparent, ethical, adaptive, decentralized, and inclusive) for responsible blockchain development. Additionally, it presents the Actor-Network Theory-based Responsible Development Methodology (ANT-RDM) for blockchains, which includes the steps of problematization, interessement, enrollment, and mobilization.

Open access
cs.MA
cs.CR
cs.DC
Original source
Sep 3, 2024·arXiv
0 cites
Quantifying Liveness and Safety of Avalanche's Snowball

Quentin Kniep, Maxime Laval, Jakub Sliwinski, Roger Wattenhofer

This work examines the resilience properties of the Snowball and Avalanche protocols that underlie the popular Avalanche blockchain. We experimentally quantify the resilience of Snowball using a simulation implemented in Rust, where the adversary strategically rebalances the network to delay termination. We show that in a network of $n$ nodes of equal stake, the adversary is able to break liveness when controlling $Ω(\sqrt{n})$ nodes. Specifically, for $n = 2000$, a simple adversary controlling $5.2\%$ of stake can successfully attack liveness. When the adversary is given additional information about the state of the network (without any communication or other advantages), the stake needed for a successful attack is as little as $2.8\%$. We show that the adversary can break safety in time exponentially dependent on their stake, and inversely linearly related to the size of the network, e.g. in 265 rounds in expectation when the adversary controls $25\%$ of a network of 3000. We conclude that Snowball and Avalanche are akin to Byzantine reliable broadcast protocols as opposed to consensus.

Open access
cs.DC
Original source
Aug 28, 2024·arXiv
0 cites
Monadring: A lightweight consensus protocol to offer Validation-as-a-Service to AVS nodes

Yu Zhang, Xiao Yan, Gang Tang, Helena Wang

Existing blockchain networks are often large-scale, requiring transactions to be synchronized across the entire network to reach consensus. On-chain computations can be prohibitively expensive, making many CPU-intensive computations infeasible. Inspired by the structure of IBM's token ring networks, we propose a lightweight consensus protocol called Monadring to address these issues. Monadring allows nodes within a large blockchain network to form smaller subnetworks, enabling faster and more cost-effective computations while maintaining the security guarantees of the main blockchain network. To further enhance Monadring's security, we introduce a node rotation mechanism based on Verifiable Random Function (VRF) and blind voting using Fully Homomorphic Encryption (FHE) within the smaller subnetwork. Unlike the common voting-based election of validator nodes, Monadring leverages FHE to conceal voting information, eliminating the advantage of the last mover in the voting process. This paper details the design and implementation of the Monadring protocol and evaluates its performance and feasibility through simulation experiments. Our research contributes to enhancing the practical utility of blockchain technology in large-scale application scenarios.

Open access
cs.DC
Original source
Aug 28, 2024·arXiv
5 cites
EdgeLinker: Practical Blockchain-based Framework for Healthcare Fog Applications to Enhance Security in Edge-IoT Data Communications

Mahdi Akbari Zarkesh, Ehsan Dastani, Bardia Safaei, Ali Movaghar

The pervasive adoption of Internet of Things (IoT) has significantly advanced healthcare digitization and modernization. Nevertheless, the sensitive nature of medical data presents security and privacy challenges. On the other hand, resource constraints of IoT devices often necessitates cloud services for data handling, introducing single points of failure, processing delays, and security vulnerabilities. Meanwhile, the blockchain technology offers potential solutions for enhancing security, decentralization, and data ownership. An ideal solution should ensure confidentiality, access control, and data integrity while being scalable, cost-effective, and integrable with the existing systems. However, current blockchain-based studies only address some of these requirements. Accordingly, this paper proposes EdgeLinker; a comprehensive solution incorporating Proof-of-Authority consensus, integrating smart contracts on the Ethereum blockchain for access control, and advanced cryptographic algorithms for secure data communication between IoT edge devices and the fog layer in healthcare fog applications. This novel framework has been implemented in a real-world fog testbed, using COTS fog devices. Based on a comprehensive set of evaluations, EdgeLinker demonstrates significant improvements in security and privacy with reasonable costs, making it an affordable and practical system for healthcare fog applications. Compared with the state-of-the-art, without significant changes in the write-time to the blockchain, EdgeLinker achieves a 35% improvement in data read time. Additionally, it is able to provide better throughput in both reading and writing transactions compared to the existing studies. EdgeLinker has been also examined in terms of energy, resource consumption and channel latency in both secure and non-secure modes, which has shown remarkable improvements.

Open access
2 source records
cs.DC
cs.NI
IoT and Edge/Fog Computing
Original source
Aug 22, 2024·arXiv
0 cites
Fair Combinatorial Auctions: Endogenous Best Execution in Blockchain Trade-Intent Markets

Andrea Canidio, Felix Henneke

Trade-intent auctions intermediate around USD~9~billion in monthly trading volume. In these auctions, specialized intermediaries called solvers compete for the right to execute orders across fragmented blockchain-based financial markets. These auctions are combinatorial because executing multiple trade intents jointly generates additional efficiencies. However, there is no best-execution benchmark to determine how to share those efficiencies: the best possible execution of a trade is solvers' private information and must be elicited. We study theoretically the two main mechanisms: batch auctions, in which a group of trades is auctioned off jointly, and independent trade-by-trade auctions. Batch auctions return more total value to traders, but their outcome may be unfair, in the sense of leaving one trader worse off than under independent auctions. We propose a fair combinatorial auction: solvers bid on individual trades and on batches of trades, but a batched bid is filtered out if any trader earns less than an execution benchmark constructed from the bids on individual trades and a counterfactual mechanism. Whether fairness guarantees arise in equilibrium depends on the counterfactual mechanism: independent first-price auctions generate such guarantees; independent second-price auctions do not. These fairness guarantees come at a cost: a lower total value returned to traders.

Open access
econ.TH
cs.DC
cs.GT
Original source
Aug 19, 2024·arXiv (Cornell University)
1 cites
CountChain: A Decentralized Oracle Network for Counting Systems

Behkish Nassirzadeh, Albert Heinle, Stefanos Leonardos, Anwar Hasan · 5 authors

Blockchain integration in industries like online advertising is hindered by its connectivity limitations to off-chain data. These industries heavily rely on precise counting systems for collecting and analyzing off-chain data. This requires mechanisms, often called oracles, to feed off-chain data into smart contracts. However, current oracle solutions are ill-suited for counting systems since the oracles do not know when to expect the data, posing a significant challenge. To address this, we present CountChain, a decentralized oracle network for counting systems. In CountChain, data is received by all oracle nodes, and any node can submit a proposition request. Each proposition contains enough data to evaluate the occurrence of an event. Only randomly selected nodes participate in a game to evaluate the truthfulness of each proposition by providing proof and some stake. Finally, the propositions with the outcome of True increment the counter in a smart contract. Thus, instead of a contract calling oracles for data, in CountChain, the oracles call a smart contract when the data is available. Furthermore, we present a formal analysis and experimental evaluation of the system's parameters on over half a million data points to obtain optimal system parameters. In such conditions, our game-theoretical analysis demonstrates that a Nash equilibrium exists wherein all rational parties participate with honesty.

Open access
3 source records
Data Stream Mining Techniques
Data Management and Algorithms
Traffic Prediction and Management Techniques
Original source
Aug 19, 2024·2024 IEEE International Conference on Blockchain (Blockchain), Copenhagen, Denmark, 2024, pp. 605-610
5 cites
A Systematisation of Knowledge: Connecting European Digital Identities with Web3

Ben Biedermann, Matthew Scerri, Victoria Kozlova, Joshua Ellul

The terms self-sovereign identity (SSI) and decen-tralised identity are often used interchangeably, which results in increasing ambiguity when solutions are being investigated and compared. This article aims to provide a clear distinction between the two concepts in relation to the revised Regulation as Regards establishing the European Digital Identity Framework (eIDAS 2.0) by providing a systematisation of knowledge of technological developments that led up to implementation of eIDAS 2.0. Applying an inductive exploratory approach, relevant literature was selected iteratively in waves over a nine months time frame and covers literature between 2005 and 2024. The review found that the decentralised identity sector emerged adjacent to the OpenID Connect (OIDC) paradigm of Open Authentication, whereas SSI denotes the sector's shift towards blockchain-based solutions. In this study, it is shown that the interchangeable use of SSI and decentralised identity coincides with novel protocols over OIDC. While the first part of this paper distinguishes OIDC from decentralised identity, the second part addresses the incompatibility between OIDC under eIDAS 2.0 and Web3. The paper closes by suggesting further research for establishing a digital identity bridge for connecting applications on public-permissionless ledgers with data originating from eIDAS 2.0 and being presented using OIDC.

Open access
2 source records
Digital Rights Management and Security
Digital Humanities and Scholarship
Digital and Traditional Archives Management
Original source
Aug 12, 2024·arXiv (Cornell University)
1 cites
Decentralized Health Intelligence Network (DHIN)

Abraham Nash

Decentralized Health Intelligence Network (DHIN) extends the Decentralized Intelligence Network (DIN) framework to address challenges in healthcare data sovereignty and AI utilization. Building upon DIN's core principles, DHIN introduces healthcare-specific components to tackle data fragmentation across providers and institutions, establishing a sovereign architecture for healthcare provision. It facilitates effective AI utilization by overcoming barriers to accessing diverse health data sources. This comprehensive framework leverages: 1) self-sovereign identity architecture coupled with a personal health record (PHR), extending DIN's personal data stores concept to ensure health data sovereignty; 2) a scalable federated learning (FL) protocol implemented on a public blockchain for decentralized AI training in healthcare, tailored for medical data; and 3) a scalable, trustless rewards mechanism adapted from DIN to incentivize participation in healthcare AI development. DHIN operates on a public blockchain with an immutable record, ensuring that no entity can control access to health data or determine financial benefits. It supports effective AI training while allowing patients to maintain control over their health data, benefit financially, and contribute to a decentralized ecosystem. Unique to DHIN, patients receive rewards in digital wallets as an incentive to opt into the FL protocol, with a long-term roadmap to fund decentralized insurance solutions. This approach introduces a novel, self-financed healthcare model that adapts to individual needs, complements existing systems, and redefines universal coverage, showcasing how DIN principles can transform healthcare data management and AI utilization while empowering patients.

Open access
2 source records
Genetics, Bioinformatics, and Biomedical Research
cs.CR
cs.AI
Original source
Aug 9, 2024·arXiv
0 cites
Verifiable Decentralized IPFS Cluster: Unlocking Trustworthy Data Permanency for Off-Chain Storage

Sid Lamichhane, Patrick Herbke

In Decentralized Applications, off-chain storage solutions such as the InterPlanetary File System (IPFS) are crucial in overcoming Blockchain storage limitations. However, the assurance of data permanency in IPFS relies on the pinning of data, which comes with trust issues and potential single points of failure. This paper introduces Verifiable Decentralized IPFS Clusters (VDICs) to enhance off-chain storage reliability with verifiable data permanency guarantees. VDICs leverage Decentralized Identifier, Verifiable Credentials, and IPFS Clusters to create a trustworthy ecosystem where the storage of pinned data is transparent and verifiable. Performance evaluations demonstrate that VDICs are competitive with traditional pinning services. Real-life use cases validate their feasibility and practicality for providers of Decentralized Applications focused on ensuring data permanency.

Open access
cs.DC
cs.CR
Original source
Aug 4, 2024·arXiv
0 cites
Blockchain-Enabled Dynamic Spectrum Sharing for Satellite and Terrestrial Communication Networks

Zixin Wang, Mingrui Cao, Hao Jiang, Bin Cao · 7 authors

Dynamic spectrum sharing (DSS) between satellite and terrestrial networks has increasingly engaged the academic and industrial sectors. Nevertheless, facilitating secure, efficient and scalable sharing continues to pose a pivotal challenge. Emerging as a promising technology to bridge the trust gap among multiple participants, blockchain has been envisioned to enable DSS in a decentralized manner. However, satellites with limited resources may struggle to support the frequent interactions required by blockchain networks. Additionally,given the extensive coverage of satellites, spectrum sharing needs vary by regions, challenging traditional blockchain approaches to accommodate differences. In this work, a partitioned, self-governed, and customized dynamic spectrum sharing approach (PSC-DSS) is proposed for spectrum sharing between satellite access networks and terrestrial access networks. This approach establishes a sharded and tiered architecture which allows various regions to manage spectrum autonomously while jointly maintaining a single blockchain ledger. Moreover, a spectrum-consensus integrated mechanism, which decouples DSS process and couples it with blockchain consensus protocol, is designed to enable regions to conduct DSS transactions in parallel and dynamically innovate spectrum sharing schemes without affecting others. Furthermore, a theoretical framework is derived to justify the stability performance of PSC-DSS. Finally, simulations and experiments are conducted to validate the advantageous performance of PSC-DSS in terms of low-overhead, high efficiency, and robust stability.

Open access
cs.DC
Original source
Aug 1, 2024·Proceedings of the VLDB Endowment
2 cites
OFL-W3: A One-shot Federated Learning System on Web 3.0

Linshan Jiang, Moming Duan, Bingsheng He, Yulin Sun · 7 authors

Federated Learning (FL) addresses the challenges posed by data silos, which arise from privacy, security regulations, and ownership concerns. Despite these barriers, FL enables these isolated data repositories to participate in collaborative learning without compromising privacy or security. Concurrently, the advancement of blockchain technology and decentralized applications (DApps) within Web 3.0 heralds a new era of transformative possibilities in web development. As such, incorporating FL into Web 3.0 paves the path for overcoming the limitations of data silos through collaborative learning. However, given the transaction speed constraints of core blockchains such as Ethereum (ETH) and the latency in smart contracts, employing one-shot FL, which minimizes client-server interactions in traditional FL to a single exchange, is considered more apt for Web 3.0 environments. This paper presents a practical one-shot FL system for Web 3.0, termed OFL-W3. OFL-W3 capitalizes on blockchain technology by utilizing smart contracts for managing transactions. Meanwhile, OFL-W3 utilizes the Inter-Planetary File System (IPFS) coupled with Flask communication, to facilitate backend server operations to use existing one-shot FL algorithms. With the integration of the incentive mechanism, OFL-W3 showcases an effective implementation of one-shot FL on Web 3.0, offering valuable insights and future directions for AI combined with Web 3.0 studies.

Open access
2 source records
cs.DC
Privacy-Preserving Technologies in Data
Cryptography and Data Security
Original source
Jul 30, 2024·arXiv
0 cites
Securing Proof of Stake Blockchains: Leveraging Multi-Agent Reinforcement Learning for Detecting and Mitigating alicious Nodes

Faisal Haque Bappy, Tariqul Islam, Kamrul Hasan, Md Sajidul Islam Sajid · 5 authors

Proof of Stake (PoS) blockchains offer promising alternatives to traditional Proof of Work (PoW) systems, providing scalability and energy efficiency. However, blockchains operate in a decentralized manner and the network is composed of diverse users. This openness creates the potential for malicious nodes to disrupt the network in various ways. Therefore, it is crucial to embed a mechanism within the blockchain network to constantly monitor, identify, and eliminate these malicious nodes without involving any central authority. In this paper, we propose MRL-PoS+, a novel consensus algorithm to enhance the security of PoS blockchains by leveraging Multi-agent Reinforcement Learning (MRL) techniques. Our proposed consensus algorithm introduces a penalty-reward scheme for detecting and eliminating malicious nodes. This approach involves the detection of behaviors that can lead to potential attacks in a blockchain network and hence penalizes the malicious nodes, restricting them from performing certain actions. Our developed Proof of Concept demonstrates effectiveness in eliminating malicious nodes for six types of major attacks. Experimental results demonstrate that MRL-PoS+ significantly improves the attack resilience of PoS blockchains compared to the traditional schemes without incurring additional computation overhead.

Open access
2 source records
Blockchain Technology Applications and Security
cs.DC
cs.CR
Original source
Jul 30, 2024·arXiv
0 cites
Impact of Conflicting Transactions in Blockchain: Detecting and Mitigating Potential Attacks

Faisal Haque Bappy, Tariqul Islam, Kamrul Hasan, Joon S. Park · 5 authors

Conflicting transactions within blockchain networks not only pose performance challenges but also introduce security vulnerabilities, potentially facilitating malicious attacks. In this paper, we explore the impact of conflicting transactions on blockchain attack vectors. Through modeling and simulation, we delve into the dynamics of four pivotal attacks - block withholding, double spending, balance, and distributed denial of service (DDoS), all orchestrated using conflicting transactions. Our analysis not only focuses on the mechanisms through which these attacks exploit transaction conflicts but also underscores their potential impact on the integrity and reliability of blockchain networks. Additionally, we propose a set of countermeasures for mitigating these attacks. Through implementation and evaluation, we show their effectiveness in lowering attack rates and enhancing overall network performance seamlessly, without introducing additional overhead. Our findings emphasize the critical importance of actively managing conflicting transactions to reinforce blockchain security and performance.

Open access
cs.DC
cs.CR
Original source
Jul 29, 2024·IEEE Latam Transactions 2024
0 cites
Prichain II: CloudGuardian Cloud Security Proposal with Blockchain

Rodrigo Craveiro Rodrigues, Pedro Miguel Calhau Mateus, Valderi Reis Quietinho Leithardt

With the advancement of cloud computing, data storage, and security have become crucial. The growing adoption of cloud services by companies, accompanied by increased threats from cybersecurity, highlights the importance of privacy and ownership of user data. Between 2022 and 2023, there has been an increase of around 48% in cloud security threats, emphasizing the urgent need for strong security solutions. To face these challenges, in this project, we propose integrating the Ethereum network's blockchain technology with a database located in the PostgreSQL cloud. The proposed solution aims to provide bidirectional data synchronization and strict control of access mechanisms. Blockchain technology ensures immutability and transparency of transactions, while PostgreSQL provides efficient and scalable storage. Through rigorous testing in an adaptive traffic control scenario, the results obtained indicate that this solution offers a significantly high level of security due to the decentralization of data, confirming that this solution is effective, and making it a powerful new option to improve security in cloud environments. In conclusion, the solution proposed in this project not only increases information security but also demonstrates the practical feasibility of integrating blockchain with cloud relational databases. This two-way alignment improves protection against cyberattacks and ensures that user data is protected from unauthorized access and malicious changes.

Open access
cs.CR
cs.DC
Original source
Jul 29, 2024·arXiv
0 cites
Half a Century of Distributed Byzantine Fault-Tolerant Consensus: Design Principles and Evolutionary Pathways

Huanyu Wu, Chentao Yue, Yixuan Fan, Yonghui Li · 5 authors

The concept of distributed consensus originated in the 1970s and gained widespread attention following Leslie Lamport's influential publication on the Byzantine Generals Problem in the 1980s. Over the past five decades, distributed consensus has become an extensively researched field. Practical Byzantine Fault Tolerance (PBFT) has emerged as a prominent and widely adopted solution due to its conceptual clarity, effectiveness, and resilience to arbitrary failures. However, PBFT does not universally address all scenarios, highlighting the necessity of developing a comprehensive understanding of the history, evolution, and foundational principles of distributed consensus. This article systematically reviews the historical evolution and foundational principles of distributed consensus, examining pivotal advancements including fault-tolerant state machine replication (SMR), consensus protocols in partially synchronous and asynchronous networks, and recent innovations in Directed Acyclic Graph (DAG)-based consensus mechanisms. We further analyse the core design rationales, essential components, and underlying primitives across various distributed fault-tolerant protocols. The relationship between BFT consensus mechanisms and their applications in environments requiring robust resilience against adversarial faults is also explored. Finally, we discuss emerging research areas and challenges, such as consensus for wireless and blockchain scenarios, highlighting potential future developments. This comprehensive overview offers valuable insights to inform the design, optimisation, and implementation of distributed consensus systems across multiple application scenarios.

Open access
cs.DC
Original source
Jul 29, 2024·arXiv
0 cites
Model Agnostic Hybrid Sharding For Heterogeneous Distributed Inference

Claudio Angione, Yue Zhao, Harry Yang, Ahmad Farhan · 7 authors

The rapid growth of large-scale AI models, particularly large language models has brought significant challenges in data privacy, computational resources, and accessibility. Traditional centralized architectures often struggle to meet required data security and scalability needs which hinders the democratization of AI systems. Nesa introduces a model-agnostic sharding framework designed for decentralized AI inference. Our framework uses blockchain-based sequential deep neural network sharding to distribute computational tasks across a diverse network of nodes based on a personalised heuristic and routing mechanism. This enables efficient distributed training and inference for recent large-scale models even on consumer-grade hardware. We use compression techniques like dynamic blockwise quantization and mixed matrix decomposition to reduce data transfer and memory needs. We also integrate robust security measures, including hardware-based trusted execution environments to ensure data integrity and confidentiality. Evaluating our system across various natural language processing and vision tasks shows that these compression strategies do not compromise model accuracy. Our results highlight the potential to democratize access to cutting-edge AI technologies by enabling secure and efficient inference on a decentralized network.

Open access
cs.AI
cs.CL
cs.CR
Original source
Jul 29, 2024·arXiv
0 cites
Performance Optimization of High-Conflict Transactions within the Hyperledger Fabric Blockchain

Alexandros Stoltidis, Kostas Choumas, Thanasis Korakis

Hyperledger Fabric (HLF) is a secure and robust blockchain (BC) platform that supports high-throughput and low-latency transactions. However, it encounters challenges in managing conflicting transactions that negatively affect throughput and latency. This paper proposes a novel solution to address these challenges and improve performance, especially in applications incorporating extensive volumes of highly conflicting transactions. Our solution involves reallocating the Multi-Version Concurrency Control (MVCC) of the validation phase to a preceding stage in the transaction flow to enable early detection of conflicting transactions. Specifically, we propose and evaluate two innovative modifications, called Orderer Early MVCC (OEMVCC) and OEMVCC with Execution Avoidance (OEMVCC-EA). Our experimental evaluation results demonstrate significant throughput and latency improvements, providing a practical solution for high-conflict applications that demand high performance and scalability.

Open access
cs.DC
Original source
Jul 26, 2024·ACM SIGKDD Explorations Newsletter
13 cites
Blockchain for Large Language Model Security and Safety: A Holistic Survey

Caleb Geren, Amanda Board, Gaby G. Dagher, Tim Andersen · 5 authors

With the growing development and deployment of large language models (LLMs) in both industrial and academic fields, their security and safety concerns have become increasingly critical. However, recent studies indicate that LLMs face numerous vulnerabilities, including data poisoning, prompt injections, and unauthorized data exposure, which conventional methods have struggled to address fully. In parallel, blockchain technology, known for its data immutability and decentralized structure, offers a promising foundation for safeguarding LLMs. In this survey, we aim to comprehensively assess how to leverage blockchain technology to enhance LLMs' security and safety. Besides, we propose a new taxonomy of blockchain for large language models (BC4LLMs) to systematically categorize related works in this emerging field. Our analysis includes novel frameworks and definitions to delineate security and safety in the context of BC4LLMs, highlighting potential research directions and challenges at this intersection. Through this study, we aim to stimulate targeted advancements in blockchain-integrated LLM security.

Open access
2 source records
cs.CR
cs.AI
cs.DC
Original source
Jul 25, 2024·arXiv
0 cites
Generative AI like ChatGPT in Blockchain Federated Learning: use cases, opportunities and future

Sai Puppala, Ismail Hossain, Md Jahangir Alam, Sajedul Talukder · 8 authors

Federated learning has become a significant approach for training machine learning models using decentralized data without necessitating the sharing of this data. Recently, the incorporation of generative artificial intelligence (AI) methods has provided new possibilities for improving privacy, augmenting data, and customizing models. This research explores potential integrations of generative AI in federated learning, revealing various opportunities to enhance privacy, data efficiency, and model performance. It particularly emphasizes the importance of generative models like generative adversarial networks (GANs) and variational autoencoders (VAEs) in creating synthetic data that replicates the distribution of real data. Generating synthetic data helps federated learning address challenges related to limited data availability and supports robust model development. Additionally, we examine various applications of generative AI in federated learning that enable more personalized solutions.

Open access
cs.LG
cs.AI
cs.CR
Original source