Chenhao Ying, Fuyuan Xia, David S. L. Wei, Xinchun Yu · 11 authors
Harnessing the benefits of blockchain, such as decentralization, immutability, and transparency, to bolster the credibility and security attributes of federated learning (FL) has garnered increasing attention. However, blockchain-enabled FL (BFL) still faces several challenges. The primary and most significant issue arises from its essential but slow validation procedure, which selects high-quality local models by recruiting distributed validators. The second issue stems from its incentive mechanism under the transparent nature of blockchain, increasing the risk of privacy breaches regarding workers’ cost information. The final challenge involves data eavesdropping from shared local models. To address these significant obstacles, this paper proposes a Blockchain-enabled Incentivized and Secure Federated Learning (BIT-FL) framework. BIT-FL leverages a novel loop-based sharded consensus algorithm to accelerate the validation procedure, ensuring the same security as non-sharded consensus protocols. It consistently outputs the correct local model selection when the fraction of adversaries among validators is less than$1/2$with synchronous communication. Furthermore, BIT-FL integrates a randomized incentive procedure, attracting more participants while guaranteeing the privacy of their cost information through meticulous worker selection probability design. Finally, by adding artificial Gaussian noise to local models, it ensures the privacy of trainers’ local models. With the careful design of Gaussian noise, the excess empirical risk of BIT-FL is upper-bounded by$\mathcal {O}(\frac{\ln n_{\min}}{ n_{\min}^{3/2}}+\frac{\ln n}{n})$, where$n$represents the size of the union dataset, and$n_{{\min}}$represents the size of the smallest dataset. Our extensive experiments demonstrate that BIT-FL exhibits efficiency, robustness, and high accuracy for both classification and regression tasks.
In recent years, the confluence of federated learning and model parallelism has revolutionized the landscape of deep learning on distributed systems, significantly enhancing efficiency and scalability. Federated learning, a decentralized approach, enables multiple edge devices to collaboratively train a model without sharing their data, thereby preserving privacy and reducing latency. Model parallelism, on the other hand, divides a large model across several devices, allowing for simultaneous computation and faster processing. By synergizing these two paradigms, researchers have developed innovative frameworks that leverage the strengths of both approaches, achieving superior performance and resource utilization. This hybrid strategy addresses the limitations of traditional centralized training, offering a robust solution for large-scale, privacy-sensitive applications.The integration of federated learning and model parallelism not only optimizes computational resources but also mitigates communication bottlenecks inherent in distributed systems. This amalgamation is particularly advantageous for deep learning tasks involving vast datasets and complex models, as it distributes the computational load and enhances fault tolerance. Moreover, this approach supports continuous learning from distributed data sources, facilitating real-time updates and adaptability. As a result, turbocharged AI systems leveraging these technologies can efficiently handle the growing demands of contemporary deep learning applications, paving the way for advancements in fields such as healthcare, finance, and autonomous systems.
Federated Learning (FL) has become a transformative approach in machine learning, allowing decentralized training of models across multiple devices while preserving data privacy. This paradigm addresses critical concerns related to data privacy, security, and communication overhead, making it particularly relevant for applications in domains such as healthcare, finance, and the Internet of Things (IoT). Resource-constrained FL extends this concept to environments where computational, communication, and energy resources are limited, such as edge networks and IoT devices. This extension focuses on optimizing various aspects of the learning process to enable effective model training even in resource-limited settings. The primary aim of this survey is to provide a comprehensive and structured overview of the current state of research in FL and resource-constrained FL. By examining 62 key publications, this survey synthesizes insights and developments across these domains, highlighting advancements, challenges, and gaps that exist. This survey aims to provide a holistic view of the advancements and ongoing challenges in FL and resource-constrained FL. It identifies research gaps and proposes future directions, such as improving communication efficiency, developing adaptive learning algorithms, and enhancing resource management strategies. This survey serves as a valuable resource for researchers, practitioners, and stakeholders in the field, offering practical insights and guiding future exploration and innovation in FL and its applications in resource-constrained environments.
Hangchao Ding, Huayun Tang, Jia Chen, Yanzhao Wang
As a privacy-preserving technique, Secure Multi-Party Computation (SMPC) has been extensively applied in Neural Networks (NN) privacy-preserving schemes, including secret sharing, homomorphic encryption, and zero-knowledge proof. Federated Learning (FL) is applied to train models from scattered data, which can protect the client's data. Therefore, the leakage of gradient in Federated Learning has been a risk of privacy and security. We construct a post-quantum privacy-preserving cryptography algorithm, in which gradient can be protected by SMPC techniques. Non-interactive zero-knowledge proof, Shamir secret sharing, and homomorphic encryption techniques are applied to guarantee the security of gradient transmission. L WE-based Kyber key exchange protocol is also applied to be resistant to quantum attack.
Rongping Lin, Fan Wang, Shan Luo, Xiong Wang · 5 authors
Federated Learning (FL) is a distributed machine learning method that ensures the privacy and security of participants’ data by avoiding direct data upload to a central node for training. However, the traditional FL typically applies a star structure with cloud servers as the central aggregator for the model parameters from different terminals, leading to problems such as central failure, malicious tampering and malicious participants, resulting in training errors or system crashes. To address these issues, a permissioned blockchain is used to build a secure and reliable data-sharing platform among participating terminals, replacing the central aggregator in the traditional FL called blockchain-based federated learning. However, the block generation method of the blockchain system may introduce significant latency in the federated learning where distributed model parameters upload randomly, resulting in low efficiency of the federated learning. To overcome this, we propose a block generation strategy that groups terminals and generates a block for each group, which minimizes the latency of a single round of federated learning, and an optimal block generation algorithm that considers data distribution, terminal resources, and network resources is provided. The analysis shows that the proposed algorithm can effectively obtain the optimal solution of block generation to minimize the authentication time, and we conduct extensive experiments that demonstrate the time efficiency of the proposed algorithm.
Rafael Marin Machado de Souza, A. I. S. Holm, Márcio Biczyk, Leandro Nunes de Castro
Federated learning (FL) and bioinspired computing (BIC), two distinct, yet complementary fields, have gained significant attention in the machine learning community due to their unique characteristics. FL enables decentralized machine learning by allowing models to be trained on data residing across multiple devices or servers without exchanging raw data, thus enhancing privacy and reducing communication overhead. Conversely, BIC draws inspiration from nature to develop robust and adaptive computational solutions for complex problems. This paper explores the state of the art in the integration of FL and BIC, introducing BIC techniques and discussing the motivations for their integration with FL. The convergence of these fields can lead to improved model accuracy, enhanced privacy, energy efficiency, and reduced communication overhead. This synergy addresses inherent challenges in FL, such as data heterogeneity and limited computational resources, and opens up new avenues for developing more efficient and autonomous learning systems. The integration of FL and BIC holds promise for various application domains, including healthcare, finance, and smart cities, where privacy-preserving and efficient computation is paramount. This survey provides a systematic review of the current research landscape, identifies key challenges and opportunities, and suggests future directions for the successful integration of FL and BIC.
Chenfei Nie, Yannan Li, Yuxin Yang, Yuede Ji · 5 authors
Federated learning (FL) is an emerging distributed learning paradigm without sharing participating clients' private data. However, existing works show that FL is vulnerable to both Byzantine (security) attacks and data reconstruction (privacy) attacks. Almost all the existing FL defenses only address one of the two attacks. A few defenses address the two attacks, but they are not efficient and effective enough. We propose BPFL, an efficient Byzantine-robust and provably privacy-preserving FL method that addresses all the issues. Specifically, we draw on state-of-the-art Byzantine-robust FL methods and use similarity metrics to measure the robustness of each participating client in FL. The validity of clients are formulated as circuit constraints on similarity metrics and verified via a zero-knowledge proof. Moreover, the client models are masked by a shared random vector, which is generated based on homomorphic encryption. In doing so, the server receives the masked client models rather than the true ones, which are proven to be private. BPFL is also efficient due to the usage of non-interactive zero-knowledge proof. Experimental results on various datasets show that our BPFL is efficient, Byzantine-robust, and privacy-preserving.
Linh Tran, Sanjay Chari, Md. Saikat Islam Khan, Aaron Zachariah · 6 authors
We present the Differentially Private Blockchain-Based Vertical Federal Learning (DP-BBVFL) algorithm that provides verifiability and privacy guarantees for decentralized applications. DP-BBVFL uses a smart contract to aggregate the feature representations, i.e., the embeddings, from clients transparently. We apply local differential privacy to provide privacy for embeddings stored on a blockchain, hence protecting the original data. We provide the first prototype application of differential privacy with blockchain for vertical federated learning. Our experiments with medical data show that DP-BBVFL achieves high accuracy with a tradeoff in training time due to on-chain aggregation. This innovative fusion of differential privacy and blockchain technology in DP-BBVFL could herald a new era of collaborative and trustworthy machine learning applications across several decentralized application domains.
The sixth-generation (6 G) wireless networks are envisioned to build a data-driven digital world with widespread Artificial Intelligence (AI). Federated learning (FL) is a distributed AI paradigm that coordinates different data owners to train shared AI models cooperatively. However, traditional FL faces challenges in practically deploying in 6 G networks: (i) the central server becomes the bottleneck and fails to identify clients' malicious behaviors, and (ii) the lack of incentive mechanisms makes heterogeneous nodes hard to collaborate when considering unilateral returns. To address the above challenges, we first propose a blockchain-enabled FL (BFL) framework where clients' malicious behaviors could be identified without a central server. Then we propose a Bayesian game-driven incentive mechanism to encourage honest nodes to provide valid models while hindering the training interference from malicious clients. Moreover, we propose a dynamic data contribution scheme to schedule data resources equitably while ensuring model performance. Finally, a Proof-of-Incentive consensus mechanism is designed as benign impetuses to guide the system toward the direction of more secure model aggregation and higher incentives. Experimental results show that our proposed schemes can obtain high-precision models even with malicious clients and effectively motivate honest nodes to join FL in 6 G networks.
Cheng Zhang, Yang Xu, Xiaowei Wu, En Wang · 6 authors
Decentralized federated learning (DFL) overcomes the single point of failure issue of centralized federated learning. Building upon DFL, blockchain-based federated learning (BFL) takes further strides in establishing trust, enhancing security, and fault tolerance. However, BFL based on the classical linear blockchain exhibits diminished training efficiency in heterogeneous environments and is limited by the performance bottleneck of blockchain. Recent solutions introduce the directed acyclic graph (DAG) blockchain to address these issues, yet they compromise the verifiability of BFL, struggle with handling outdated models, and have a slow convergence speed. In this paper, we propose TGFL, a decentralized federated learning framework based on the Tree-Graph blockchain. The underlying blockchain structure of TGFL is designed as a block-centered DAG to support verifiable and semi-asynchronous training. To facilitate fast convergence, we design a pivot chain generation algorithm that topologically sorts the semi-asynchronous training process, guiding participants in sampling appropriate models. The consensus mechanism, which is closely integrated with federated learning, ensures that the TGFL can effectively resist attacks on the model and the blockchain system. Extensive experiments in various settings demonstrate that TGFL can achieve better training efficiency and model accuracy compared to three baselines.
This project focuses on Zero-Knowledge Proofs (ZKPs), a groundbreaking cryptographic technique reshaping data authentication while preserving maximum confidentiality.ZKPs enable the verification of truthfulness in statements without disclosing associated data, ensuring the utmost protection of sensitive information.With applications spanning various domains, including secure authentication protocols, privacy-preserving transactions in decentralized systems like blockchain, and confidential data verification across digital interactions, ZKPs offer versatile solutions for secure communications.The project aims to safeguard sensitive business information during outsourcing service processes.The implementation of ZKPs intends to establish a secure communication framework that fosters trust among stakeholders without compromising sensitive details, ensuring enhanced confidentiality in outsourced operations.At its core, ZKPs empower a prover to convince a verifier of a statement's validity without revealing underlying data, establishing an unmatched level of security and privacy.This concept shields against unauthorized access and data breaches, fostering trust between entities without the exchange of sensitive details.The versatility of ZKPs extends beyond authentication, influencing secure voting systems, safeguarding digital identities, and facilitating confidential transactions while upholding user privacy.
Chaehyeon Lee, Jonathan Heiss, Stefan Tai, James Won‐Ki Hong
Verifiable decentralized federated learning (FL) systems combining blockchains and zero-knowledge proofs (ZKP) make the computational integrity of local learning and global aggregation verifiable across workers. However, they are not end-to-end: data can still be corrupted prior to the learning. In this paper, we propose a verifiable decentralized FL system for end-to-end integrity and authenticity of data and computation extending verifiability to the data source. Addressing an inherent conflict of confidentiality and transparency, we introduce a two-step proving and verification (2PV) method that we apply to central system procedures: a registration workflow that enables non-disclosing verification of device certificates and a learning workflow that extends existing blockchain and ZKP-based FL systems through non-disclosing data authenticity proofs. Our evaluation on a prototypical implementation demonstrates the technical feasibility with only marginal overheads to state-of-the-art solutions.
Federated learning allows multiple clients to collaboratively train a global model without revealing their private data. Despite its success in many applications, it remains a challenge to prevent malicious clients to corrupt the global model through uploading incorrect model updates. Hence, one critical issue arises in how to validate the training is truly conducted on legitimate neural networks. To address the issue, we proposeVPNNT, a zero-knowledge proof scheme for neural network backpropagation.VPNNTenables each client to prove to others that the model updates (gradients) are indeed calculated on the global model of the previous round, without leaking any information about the client's private training data. Our proof scheme is generally applicable to any type of neural network. Different from conventional verification schemes constructing neural network operations by gate-level circuits, we improve verification efficiency by formulating the training process using custom gates — matrix operations, and apply an optimized linear time zero knowledge protocol for verification. Thanks to the recursive structure of neural network backward propagation, common custom gates are combined in verification thereby reducing prover and verifier costs over conventional zero knowledge proofs. Experimental results show thatVPNNTis a lightweighted verification scheme for neural network backpropagation with an improved prove time, verification time and proof size.
Secure aggregation enables federated learning (FL) to perform collaborative training of clients from local gradient updates without exposing raw data. However, existing secure aggregation schemes inevitably perform an expensive fresh setup per round because each client needs to establish fresh input-independent secrets over different rounds. The latest research, Flamingo (S&P 2023), designed a share-transfer-based reusable secret key to support the server continuously performing multiple rounds of aggregation. Nevertheless, the share transfer mechanism it proposed can only be achieved with P probability, which has limited reliability. To tackle the aforementioned problems, we propose a more reliable and anonymously authenticated scheme called Chu-ko-nu for multi-round secure aggregation. Specifically, in terms of share transfer, Chu-ko-nu breaks the probability P barrier by supplementing a redistribution process of secret key components (the sum of all components is the secret key), thus ensuring the reusability of the secret key. Based on this reusable secret key, Chu-ko-nu can efficiently perform consecutive aggregation in the following rounds. Furthermore, considering the client identity authentication and privacy protection issue most approaches ignore, Chu-ko-nu introduces a zero-knowledge proof-based authentication mechanism. It can support clients anonymously participating in FL training and enables the server to authenticate clients effectively in the presence of various attacks. Rigorous security proofs and extensive experiments demonstrated that Chu-ko-nu can provide reliable and anonymously authenticated aggregation for FL with low aggregation costs, at least a 21.02% reduction compared to the state-of-the-art schemes.
In cross-silo federated learning (FL), organizations cooperatively train a global model with their local datasets. However, some organizations may act as free riders such that they only contribute a small amount of resources but can obtain a high-accuracy global model. Meanwhile, some organizations can be business competitors, and they do not trust each other or any third-party entity. In this work, our goal is to design a framework that motivates efficient cooperation among organizations without the coordination of a central entity. To this end, we propose a blockchain-empowered incentive mechanism framework for cross-silo FL. Under this incentive mechanism framework, we develop a distributed algorithm that enables organizations to achieve social efficiency, individual rationality, and budget balance without private information of the organizations. Our proposed algorithm has a proven convergence guarantee and empirically achieves a higher convergence rate than a benchmark method. Moreover, we propose a transaction minimization algorithm to reduce the number of transactions made among organizations in the blockchain. This algorithm is proven to achieve a performance no worse than twice the minimum value. The experimental results in a testbed show that our proposed framework enables organizations to achieve social efficiency within a relatively short iterative process.
Distributed machine learning, and Decentralized Federated Learning in particular, is emerging as an effective solution to cope with the ever-increasing amount of data and the need to process it faster and more reliably. It enables machine learning models to be trained without centralizing user data, which improves data confidentiality and optimizes performance compared with centralized approaches. However, scaling up such systems can have limitations in terms of data and model traceability and security. To address this limitation, the integration of Blockchain has been proposed, forming a global system leveraging Blockchain, called Blockchain Based Decentralized Federated Learning (BDFL), and taking advantage of the benefits of this technology, namely transparency, immutability and decentralization. For the time being, few studies have sought to characterize these BDFL systems, although it seems that they can be broken down into a set of layers (blockchain, interconnection of DFL nodes, client selection, data transmission, consensus management) that could have a major impact on the operation of the BDFL as a whole. The aim of this article is therefore to respond to this limitation by highlighting the different layers existing in the architecture of a BDFL system and the solutions proposed in the literature that can be integrated to optimise both the performance and the security of the system. This could ultimately lead to the design of more secure and efficient architectures with greater resilience to attacks and architectural changes.
Federated Learning (FL) has emerged as a revolutionary machine learning approach, enabling the training of algorithms across decentralized devices or servers while maintaining data privacy. Unlike traditional centralized methods that pool data into a single repository, FL keeps data localized, enhancing the protection of sensitive information and ensuring compliance with privacy standards like GDPR and CCPA. This paradigm shift is particularly relevant in today's data-driven world, where concerns over data breaches and regulatory compliance are paramount. FL allows organizations and individuals to collaboratively train powerful machine learning models without sharing sensitive data. By adopting FL approaches, leveraging distributed data and computing power across different sources while respecting user privacy becomes possible. The architecture of FL involves a central system coordinating updates from multiple sources to improve a global model. Edge devices, such as smartphones or IoT devices, perform local training using their unique datasets. Each edge device trains the model locally, sending only updates (like gradients) to the central server, ensuring sensitive data is never exposed1. Furthermore, privacy-preserving technologies like differential privacy and homomorphic encryption strengthen data confidentiality and compliance with regulations. Differential privacy introduces noise to data or model updates to prevent the reconstruction of individual information, while homomorphic encryption allows computations on encrypted data without decryption1. The rise of 5G networks will significantly enhance FL by reducing latency and improving communication between edge devices and central servers, enabling faster model training and real-time applications. Blockchain technology offers a decentralized and immutable ledger for tracking data usage and model updates, creating a transparent and tamper-proof mechanism, addressing trust issues in federated systems and further strengthening security
Interest in supporting Federated Learning (FL) using blockchains has grown significantly in recent years. However, restricting access to the trained models only to actively participating nodes remains a challenge even today. To address this concern, we propose a methodology that incentivizes model parameter sharing in an FL setup under Local Differential Privacy (LDP). The nodes that share less obfuscated data under LDP are awarded higher quantum of tokens, which they can later use to obtain session keys for accessing encrypted model parameters updated by the server. If one or more of the nodes do not contribute to the learning process by sharing their data, or share only highly perturbed data, they earn less number of tokens. As a result, such nodes may not be able to read the new global model parameters if required. Local parameter sharing and updating of global parameters are done using the distributed ledger of a permissioned blockchain, namely HyperLedger Fabric (HLF). Being a blockchain-based approach, the risk of a single point of failure is also mitigated. Appropriate chaincodes, which are smart contracts in the HLF framework, have been developed for implementing the proposed methodology. Results of an extensive set of experiments firmly establish the feasibility of our approach.
Ganga Rama Koteswara Rao, Hayder M. A. Ghanimi, V. S. Ramachandran, Dokhyl Al-Qahtani · 6 authors
A significant novel approach in distributed ML, Federated Learning (FL), enables multiple parties to work simultaneously on developing models while securing the confidentiality of their unique datasets. There are issues regarding privacy with FL, particularly for models that are being trained, because private information can be accessed from shared gradients or updates to the model. This investigation proposes SecureHE-Fed, a novel system that improves FL’s defense against attacks on privacy through the use of Homomorphic Encryption (HE) and Zero-Knowledge Proofs (ZKP). Before data from clients becomes involved in the learning procedure, SecureHE-Fed encrypts it. The following lets us determine encrypted messages without revealing the data as it is. As an additional security test, ZKP is employed to verify if modifications to models are valid without sharing the true nature of the information. By evaluating SecureHE-Fed with different FL techniques, researchers demonstrate that it enhances confidentiality while maintaining the precision of the model. The results of this work obtained validate SecureHE-Fed as a secure and scalable FL approach, and we recommend its use in applications where user confidentiality is essential.
Blockchain technology enables the Internet of Things (IoT) to build trust among various stakeholders. Offloading the computation-intensive IoT data processing and blockchain mining computations as tasks to mobile edge computing (MEC) and nearby devices in a device-to-device (D2D) network is a promising solution for resource constraint IoT devices. In this article, to decide on the offloading plan, we consider enhancing the users’ Quality of Experience (QoE) parameters reflecting the token-based cost of delay and energy consumption. For this purpose, we formulate the problem as a mixed integer nonlinear bilevel programming (MINLBP), which is NP-hard. Therefore, we propose a suboptimal solution with the help of two-sided matching theory games in two phases: 1) we decide to offload IoT data processing tasks using a many-to-one and 2) make a decision for mining tasks elaborating a many-to-many matching game. We also present algorithms based on the deferred acceptance (DA) mechanism to guarantee the stability of the matching results. The simulation results show that the proposed method can perform better in terms of delay and energy cost compared to the existing offloading approaches. Furthermore, it improves the utility of the users deciding to offload their tasks.
Privacy and trust are highly demanding in practical recommendation engines. Although Federated Learning (FL) has significantly addressed privacy concerns, commercial operators are still worried about several technical challenges while bringing FL into production. In addition, classical FL has several intrinsic operational limitations such as single-point failure, data and model tampering, and heterogenic clients participating in the FL process. To address these challenges in practical recommenders, we propose a responsible recommendation generation framework based on blockchain-empowered asynchronous FL that can be adopted for any model-based recommender system. In standard FL settings, we build an additional aggregation layer in which multiple trusted nodes guided by a mediator component perform gradient aggregation to achieve an optimal model locally in a parallel fashion. The mediator partitions users into K clusters, and each cluster is represented by a cluster head. Once a cluster gets semi-global convergence, the cluster head transmits model gradients to the FL server for global aggregation. In addition the trusted cluster heads are responsible to submit the converged semi-global model to a blockchain to ensure tamper resilience. In our settings, an additional mediator component works like an independent observer that monitors the performance of each cluster head, updates a reward score, and records it into a digital ledger. Finally, evaluation results on three diversified benchmarks illustrate that the recommendation performance on selected measures is considerably comparable with the standard and federated version of a well-known neural collaborative filtering recommender.