The fast proliferation of digital twin (DT) establishes a direct connection between the physical entity and its deployed digital representation. As markets shift toward mass customization and new service delivery models, the digital representation has become more adaptive and agile by forming digital twin networks (DTNs). The DTN institutes a real-time single source of truth everywhere. However, there are several issues preventing DTNs from further application, including centralized processing, data falsification, privacy leakage, lack of incentive mechanism, and so on. To make DTN better meet the ever changing demands, we propose a novel block-chain-enabled adaptive asynchronous federated learning (FedTwin) paradigm for privacy-preserving and decentralized DTNs. We design Proof-of-Federalism (PoF), which is a tailor-made consensus algorithm for autonomous DTNs. In each DT's local training phase, generative adversarial network enhanced differential privacy is used to protect the privacy of local model parameters, while a modified Isolation Forest is deployed to filter out the falsified DTs. In the global aggregation phase, an improved Markov decision process is leveraged to select optimal DTs to achieve adaptive asynchronous aggregation while providing a rollback mechanism to redact the falsified global models. With this article, we aim to provide insights to forthcoming researchers and readers in this under-explored domain.
Jun 1, 2022·2022 IEEE 9th International Conference on Cyber Security and Cloud Computing (CSCloud)/2022 IEEE 8th International Conference on Edge Computing and Scalable Cloud (EdgeCom)
Muhammad Firdaus, Harashta Tatimma Larasati, Kyung-Hyune Rhee
Federated learning (FL) has considerably emerged as a promising solution to enhance user privacy and data security by enabling collaboratively multi-party model learning without exchanging confidential data. Nevertheless, most existing FL approaches still rely on a central server to obtain a global model by collecting all uploaded models from participants, which may lead to several threats from malicious participants and even expose participant privacy. Therefore, to tackle these problems, we proposed a secure FL framework by empowering blockchain to replace the centralized aggregator sever and utilize Differential Privacy (DP) to address various attacks, e.g., membership inference attacks, during the collaborative FL model training process. The proposed framework has been implemented through two scenarios, i.e., blockchain-based FL to form a decentralized system and DP-based FL to construct the randomized privacy protection using the IBM DP Library.
Truc Nguyen, Phuc Thai, Tre’ R. Jeter, Thang N. Dinh · 5 authors
Despite the great potential of Federated Learning (FL) in large-scale distributed learning, the current system is still subject to several privacy issues due to the fact that local models trained by clients are exposed to the central server. Consequently, secure aggregation protocols for FL have been developed to conceal the local models from the server. However, we show that, by manipulating the client selection process, the server can circumvent the secure aggregation to learn the local models of a victim client, indicating that secure aggregation alone is inadequate for privacy protection. To tackle this issue, we leverage blockchain technology to propose a verifiable client selection protocol. Owing to the immutability and transparency of blockchain, our proposed protocol enforces a random selection of clients, making the server unable to control the selection process at its discretion. We present security proofs showing that our protocol is secure against this attack. Additionally, we conduct several experiments on an Ethereum-like blockchain to demonstrate the feasibility and practicality of our solution.
Τα τελευταία χρόνια έχει γίνει αντιληπτό ότι ο αποδοτικότερος τρόπος να εκπαιδευτεί κάποιο σύνθετο μοντέλο τεχνητής νοημοσύνης είναι η αξιοποίηση εξαιρετικά μεγάλων όγκων δεδομένων και μεγάλης υπολογιστικής ισχύος. Το γεγονός αυτό δίνει ένα ανυπέρβλητο προβάδισμα στις λίγες εταιρίες που κατέχουν αυτά τα στοιχεία, με αποτέλεσμα αυτές να τείνουν να κυριαρχήσουν στις εξελίξεις. Συνεπώς θα ήταν επιθυμητό να δημιουργήσουμε πρότυπα τα οποία: 1) Εγκαταλείπουν τη λογική των μεγάλων ιδιωτικών υπολογιστικών κέντρων. 2) Βασίζονται, όχι απλώς σε αποκεντρωμένο, αλλά σε πλήρως κατανεμημένο μοντέλο, υπό την έννοια ότι λειτουργούν δίχως την ανάγκη ενός κεντρικού συντονιστή. Ακολουθώντας αυτή την οδό μπορούν να δημιουργηθούν αρχιτεκτονικές οι οποίες: 1) Αξιοποιώντας πολυάριθμους κόμβους προσεγγίζουν (η ακόμα και ξεπερνούν) την υπολογιστική ισχύ ενός μεγάλου data center. 2) Λόγω ισοτιμίας των συμμετεχόντων δεν έχουν "single point of failure", το οποίο πρακτικά σημαίνει ότι η λειτουργία τους συνεχίζεται απρόσκοπτα ακόμα και μετά την αποσύνδεση ενός μεγάλου ποσοστού των κόμβων τους. 3) Είναι "εκδημοκρατισμένες", δηλαδή όλοι οι συμμετέχοντες έχουν ίση πρόσβαση στα παραγόμενα αποτελέσματα. Οι μέχρι τώρα υλοποιήσεις των αλγορίθμων ανάπτυξης νευρωνικών δικτύων επικεντρώνονται πρωτίστως στον τομέα της ταχύτητας εκπαίδευσης του μοντέλου. Παρόλο που έχουν αναπτυχθεί εκδοχές για κατανεμημένη επεξεργασία, αυτές έχουν ανάγκη από έναν κεντρικό κόμβο-διαχειριστή ο οποίος θα αναλαμβάνει τον συντονισμό των υπολοίπων. Δεν έχει όμως γίνει επαρκής μελέτη για μία υλοποίηση που δεν απαιτεί έναν τέτοιο διακομιστή. Η πρότασή μας αξιοποιεί νέας τεχνολογίας ομότιμες τοπολογίες που έχουν ως θεμέλιο λίθο την κρυπτογραφική επαλήθευση (Distributed Ledger Technology - DLT), με στόχο τη "διάχυση" του διαχειριστικού ρόλου σε ολόκληρο το δίκτυο. Κατά την εκπόνηση της έρευνας δοκιμάστηκαν διαφορετικές διαμορφώσεις της κατανεμημένης λειτουργίας και αποτιμήθηκαν οι επιπτώσεις που είχε αυτή η παραμετροποίηση, τόσο στην ταχύτητα εκπαίδευσης όσο και στην ποιότητα του παραγόμενου μοντέλου. Το αποτέλεσμα είναι ένα καινοτόμο οικοσύστημα νεοφυών τεχνολογιών, για εκδημοκρατισμένη εκπαίδευση βαθέων νευρωνικών δικτύων.
Federated learning is known to be vulnerable to both security and privacy issues. Existing research has focused either on preventing poisoning attacks from users or on concealing the local model updates from the server, but not both. However, integrating these two lines of research remains a crucial challenge since they often conflict with one another with respect to the threat model. In this work, we develop a principle framework that offers both privacy guarantees for users and detection against poisoning attacks from them. With a new threat model that includes both an honest-but-curious server and malicious users, we first propose a secure aggregation protocol using homomorphic encryption for the server to combine local model updates in a private manner. Then, a zero-knowledge proof protocol is leveraged to shift the task of detecting attacks in the local models from the server to the users. The key observation here is that the server no longer needs access to the local models for attack detection. Therefore, our framework enables the central server to identify poisoned model updates without violating the privacy guarantees of secure aggregation.
Sikha Pentyala, Davis Railsback, Ricardo Maia, Rafael Dowsley · 7 authors
In the classical setting of differential privacy, a privacy-preserving query is performed on a private database, after which the query result is released to the analyst; a differentially private query ensures that the presence of a single database entry is protected from the analyst’s view. In this work, we contribute the first definitional framework for differential privacy in the trusted curator setting (Fig. 1); clients submit private inputs to the trusted curator, which then computes individual outputs privately returned to each client. The adversary is more powerful than the standard setting; it can corrupt up to n-1 clients and subsequently decide inputs and learn outputs of corrupted parties. In this setting, the adversary also obtains leakage from the honest output that is correlated with a corrupted output. Standard differentially private mechanisms protect client inputs but do not mitigate output correlation leaking arbitrary client information, which can forfeit client privacy completely. We initiate the investigation of a novel notion of correlated-output differential privacy to bound the leakage from output correlation in the trusted curator setting. We define the satisfaction of both standard and correlated-output differential privacy as round differential privacy and highlight the relevance of this novel privacy notion to all application domains in the trusted curator model. \nWe explore round differential privacy in traditional "dark pool" market venues, which promise privacy-preserving trade execution to mitigate front-running; privately submitted trade orders and trade execution are kept private by the trusted venue operator. We observe that dark pools satisfy neither classic nor correlated-output differential privacy; in markets with low trade activity, the adversary may trivially observe recurring, honest trading patterns, and anticipate and front-run future trades. In response, we present the first round differentially private market mechanisms that formally mitigate information leakage from all trading activity of a user. This is achieved with fuzzy order matching, inspired by the standard randomized response mechanism; however, this also introduces a liquidity mismatch as buy and sell orders are not guaranteed to execute pairwise, thereby weakening output correlation; this mismatch is compensated for by a round differentially private liquidity provider mechanism, which freezes a noisy amount of assets from the liquidity provider for the duration of a privacy epoch, but leaves trader balances unaffected. We propose oblivious algorithms for realizing our proposed market mechanisms with secure multi-party computation (MPC) and implement these in the Scale-Mamba Framework using Shamir Secret Sharing based MPC. We demonstrate practical, round differentially private trading with comparable throughput as prior work implementing (traditional) dark pool algorithms in MPC; our experiments demonstrate practicality for both traditional finance and decentralized finance settings.
Jiasi Weng, Jian Weng, Gui Hong Tang, Anjia Yang · 6 authors
We propose a new approach for privacy-preserving and verifiable convolutional neural network (CNN) testing in a distrustful multi-stakeholder environment. The approach is aimed to enable that a CNN modeldeveloperconvinces auserof the truthful CNN performance over non-public data frommultiple testers, while respecting model and data privacy. To balance the security and efficiency issues, we appropriately integrate three tools with the CNN testing, including collaborative inference, homomorphic encryption (HE) and zero-knowledge succinct non-interactive argument of knowledge (zk-SNARK). We start with strategically partitioning a CNN model into a private part kept locally by the model developer, and a public part outsourced to an outside server. Then, the private part runs over the HE-protected test data sent by a tester, and transmits its outputs to the public part for accomplishing subsequent computations of the CNN testing. Second, the correctness of the above CNN testing is enforced by generating zk-SNARK based proofs, with an emphasis on optimizing proving overhead for two-dimensional (2-D) convolution operations, since the operations dominate the performance bottleneck during generating proofs. We specifically present a new quadratic matrix program (QMP)-based arithmetic circuit witha single multiplication gatefor expressing 2-D convolution operations between multiple filters and inputs in a batch manner. Third, we aggregate multiple proofs with respect to a same CNN model but different testers’ test data (i.e., different statements) into one proof, and ensure that the validity of the aggregated proof implies the validity of the original multiple proofs. Lastly, our experimental results demonstrate that our QMP-based zk-SNARK performs nearly 13.9× faster than the existing quadratic arithmetic program (QAP)-based zk-SNARK in proving time, and 17.6× faster in Setup time, for high-dimension matrix multiplication. Besides, the limitation on handling a bounded number of multiplications of QAP-based zk-SNARK is relieved.
Federated learning enables clients to train a machine learning model jointly without sharing their local data. However, due to the centrality of federated learning framework and the untrustworthiness of clients, traditional federated learning solutions are vulnerable to poisoning attacks from malicious clients and servers. In this paper, we aim to mitigate the impact of the central server and malicious clients by designing a Privacy-preserving Byzantine-robust Federated Learning (PBFL) scheme based on blockchain. Specifically, we use cosine similarity to judge the malicious gradients uploaded by malicious clients. Then, we adopt fully homomorphic encryption to provide secure aggregation. Finally, we use blockchain system to facilitate transparent processes and implementation of regulations. Our formal analysis proves that our scheme achieves convergence and provides privacy protection. Our extensive experiments on different datasets demonstrate that our scheme is robust and efficient. Even if the root dataset is small, our scheme can achieve the same efficiency as FedSGD.
The rapid expansion of data worldwide invites the need for more distributed solutions in order to apply machine learning on a much wider scale. The resultant distributed learning systems can have various degrees of centralization. In this work, we demonstrate our solution FLoBC for building a generic decentralized federated learning system using the blockchain technology, accommodating any machine learning model that is compatible with gradient descent optimization. We present our system design comprising the two decentralized actors: trainer and validator, alongside our methodology for ensuring reliable and efficient operation of said system. Finally, we utilize FLoBC as an experimental sandbox to compare and contrast the effects of trainer-to-validator ratio, reward-penalty policy, and model synchronization schemes on the overall system performance, ultimately showing by example that a decentralized federated learning system is indeed a feasible alternative to more centralized architectures.
Deep learning techniques with neural networks are developing prominently in recent years and have been deployed in numerous applications. Despite their great success, in many scenarios it is important for the users to validate that the inferences are truly computed by legitimate neural networks with high accuracy, which is referred to as the integrity of machine learning predictions. To address this issue, in this paper, we propose zkCNN, a zero knowledge proof scheme for convolutional neural networks (CNN). The scheme allows the owner of the CNN model to prove to others that the prediction of a data sample is indeed calculated by the model, without leaking any information about the model itself. Our scheme can also be generalized to prove the accuracy of a secret CNN model on a public dataset.
Many researchers have proposed replacing the aggregation server in federated learning with a blockchain system to improve privacy, robustness, and scalability. In this approach, clients would upload their updated models to the blockchain ledger and use a smart contract to perform model averaging. However, the significant delay and limited computational capabilities of blockchain systems make it inefficient to support machine learning applications on the blockchain. In this paper, we propose a new public blockchain architecture called DFL, which is specially optimized for distributed federated machine learning. Our architecture inherits the merits of traditional blockchain systems while achieving low latency and low resource consumption by waiving global consensus. To evaluate the performance and robustness of our architecture, we implemented a prototype and tested it on a physical four-node network, and also developed a simulator to simulate larger networks and more complex situations. Our experiments show that the DFL architecture can reach over 90\% accuracy for non-I.I.D. datasets, even in the presence of model poisoning attacks, while ensuring that the blockchain part consumes less than 5\% of hardware resources.
Juan Ma, Yuling Chen, Ziping Wang, Guoxu Liu · 5 authors
Abstract The delegating computation has become an irreversible trend, together comes the pressing need for fairness and efficiency issues. To solve this problem, we leverage game theory to propose a smart contract-based solution. First, according to the behavioral preferences of the participants, we design an incentive contract to describe the motivation of the participants. Next, to satisfy the fairness of the rational delegating computation, we propose a rational delegating computation protocol based on reputation and smart contract. More specifically, rational participants are to gain the maximum utility and reach the Nash equilibrium in the protocol. Besides, we design a reputation mechanism with a reputation certificate, which measures the reputation from multiple dimensions. The reputation is used to assure the client’s trust in the computing party to improve the efficiency of the protocol. Then, we conduct a comprehensive experiment to evaluate the proposed protocol. The simulation and analysis results show that the proposed protocol solves the complex traditional verification problem. We also conduct a feasibility study that involves implementing the contracts in Solidity and running them on the official Ethereum network. Meanwhile, we prove the fairness and correctness of the protocol.
With the widespread of 5G networks, the application of Federated Learning (FL) in Internet of Things (IoT) has become a trend. However, the trust problem caused by the centralized aggregation server, and the inefficiency problem caused by the low-performance devices, are still key challenges. Several studies involving asynchronous FL have been conducted to accelerate the training process, but they usually have a decreased model performance. In this paper, a blockchain-based asynchronous federated learning framework with a dynamic scaling factor is proposed. By adopting the blockchain, the trust problem among devices can be addressed. Meanwhile, the novel dynamic scaling factor is proposed to help improve the FL efficiency and accuracy. Extensive experiments are conducted on heterogeneous devices and the results show that the proposed framework mitigates the impact of low-performance devices while being as efficient as traditional FL with the extra benefit of alleviating the trust problem among IoT devices.
Deep learning has achieved the high-accuracy of state-of-the-art algorithms in long-standing AI tasks. Due to the obvious privacy issues of deep learning, Google proposes Federal Deep Learning (FDL), in which distributed participants only upload local gradients and and a centralized server updates parameters based on the collected gradients. But few users are willing to participate in federated learning due to the lack of contribution evaluation and reward mechanisms. So a decentralized federated deep learning, called DFDL, has been proposed by introducing blockchain to form an effective incentive mechanism for participants. However, DFDL still faces serious privacy issues as blockchain does not guarantee the privacy of training data and model. In this paper, in order to address the aforementioned issues, we propose a new Privacy-preserving DFDL scheme, called PDFDL. With PDFDL, parties can securely learn a global model with their local gradients in the assistance of blockchain, and the parties’ sensitive data and the global model are well protected. Specifically, with a secure multi-party aggregation computing, all local gradients are encrypted by their owners before being sent to the smart contract, and can be directly aggregated without decryption. Detailed security analysis shows that PDFDL can resist various known security threats. Moreover, we give an implementation prototype by integrating deep learning module with a Blockchain development platform (Ethereum V1.6.4). We demonstrate the encryption performance and the training accuracy of our PDFDL on benchmark datasets.
A wider coverage and a better solution to latency reduction in 5G necessitates its combination with mobile edge computing (MEC) technology. Decentralized deep learning (DDL) as a promising solution to privacy-preserving data processing for millions of edge smart devices, it leverages federated learning within the networking of local models, without disclosing a client's raw data. Especially, in industries such as finance and healthcare where sensitive data of transactions and personal medical records is cautiously maintained, DDL facilitates the collaboration among these institutes to improve the performance of local models, while protecting data privacy of participating clients. In this survey paper, we demonstrate technical fundamentals of DDL for benefiting many walks of society through decentralized learning. Furthermore, we offer a comprehensive overview of recent challenges of DDL and the most relevant solutions from novel perspectives of communication efficiency and trustworthiness.
Wider coverage and a better solution to a latency reduction in 5G necessitate its combination with multi-access edge computing (MEC) technology. Decentralized deep learning (DDL) such as federated learning and swarm learning as a promising solution to privacy-preserving data processing for millions of smart edge devices, leverages distributed computing of multi-layer neural networks within the networking of local clients, whereas, without disclosing the original local training data. Notably, in industries such as finance and healthcare where sensitive data of transactions and personal medical records is cautiously maintained, DDL can facilitate the collaboration among these institutes to improve the performance of trained models while protecting the data privacy of participating clients. In this survey paper, we demonstrate the technical fundamentals of DDL that benefit many walks of society through decentralized learning. Furthermore, we offer a comprehensive overview of the current state-of-the-art in the field by outlining the challenges of DDL and the most relevant solutions from novel perspectives of communication efficiency and trustworthiness.
Federated learning enables the joint training of machine learning models by utilizing distributed data and computing resources while protecting local data privacy.The existing asynchronous federated learning can effectively solve the problems such as waste of computing resources and low training efficiency caused by synchronous learning.However, it aggregates local models from different nodes and updates the global model through the central server,which makes it endogenously subject to the centralized trust mode and suffers from some issues such as single point of failure andprivacy leakage. In this paper, we propose a blockchain-based privacy-preserving asynchronous federated learning,which ensures the trustability by storing local models into the blockchain and generating the global model through the consensus algorithm.In order to guarantee the privacy of federated learning and improve the model utility,the exponential mechanism of differential privacy is used to select model gradients with high contribution at high probability,and a lower privacy budget is allocated to ensure the model privacy.In addition, in order to solve the problem of clock desynchronization in asynchronous federated learning,we propose a two-factor adjustment mechanism to further improve the global model utility. Finally,theoretical analysis and experimental results demonstrate that our proposed scheme can effectively guarantee the trustability and privacy of the asynchronous federated learning while improving the model utility.
Xiumei Deng, Jun Li, Chuan Ma, Kang Wei · 8 authors
The blockchain technology has been extensively studied to enable distributed and tamper-proof data processing in federated learning (FL). Most existing blockchain assisted FL (BFL) frameworks have employed a third-party blockchain network to decentralize the model aggregation process. However, decentralized model aggregation is vulnerable to pooling and collusion attacks from the third-party blockchain network. Driven by this issue, we propose a novel BFL framework that features the integration of training and mining at the client side. To optimize the learning performance of FL, we propose to maximize the long-term time average (LTA) training data size under a constraint of LTA energy consumption. To this end, we formulate a joint optimization problem of training client selection and resource allocation (i.e., the transmit power and computation frequency at the client side), and solve the long-term mixed integer non-linear programming based on a Lyapunov technique. In particular, the proposed dynamic resource allocation and client scheduling (DRACS) algorithm can achieve a trade-off of [$\mathcal{O}(1/V)$, $\mathcal{O}(\sqrt{V})$] to balance the maximization of the LTA training data size and the minimization of the LTA energy consumption with a control parameter $V$. Our experimental results show that the proposed DRACS algorithm achieves better learning accuracy than benchmark client scheduling strategies with limited time or energy consumption.
Shuo Yuan, Bin Cao, Yao Sun, Zhiguo Wan · 5 authors
Introducing blockchain into Federated Learning (FL) to build a trusted edge computing environment for transmission and learning has attracted widespread attention as a new decentralized learning pattern. However, traditional consensus mechanisms and architectures of blockchain systems face significant challenges in handling large-scale FL tasks, especially on Internet of Things (IoT) devices, due to their substantial resource consumption, limited transaction throughput, and complex communication requirements. To address these challenges, this paper proposes ChainFL, a novel two-layer blockchain-driven FL system. It splits the IoT network into multiple shards within the subchain layer, effectively reducing the scale of information exchange, and employs a Direct Acyclic Graph (DAG)-based mainchain as the mainchain layer, enabling parallel and asynchronous cross-shard validation. Furthermore, the FL procedure is customized to integrate deeply with blockchain technology, and a modified DAG consensus mechanism is designed to mitigate distortion caused by abnormal models. To provide a proof-of-concept implementation and evaluation, multiple subchains based on Hyperledger Fabric and a self-developed DAG-based mainchain are deployed. Extensive experiments demonstrate that ChainFL significantly surpasses conventional FL systems, showing up to a 14% improvement in training efficiency and a threefold increase in robustness.
Federated learning is a branch of machine learning where a shared model is created in a decentralized and privacy-preserving fashion, but existing approaches using blockchain are limited by tailored models. We consider the possibility to extend a set of supported models by introducing the oracle service and exploring the usability of blockchain-based architecture. The investigated architecture combines an oracle service with a Hyperledger Fabric chaincode. We compared two logistic regression implementations in Go language—a pure chaincode and an oracle service—at various data (2–32 k instances) and network (3–13 peers) sizes. Experiments were run to assess the performance of blockchain-based model inference using 2D synthetic and EEG eye state datasets for a supervised machine learning detection task. The benchmarking results showed that the impact on performance is acceptable with the median overhead of oracle service reaching 2–4%, depending on the dimensionality of the dataset. The overhead tends to diminish at large dataset sizes with the runtime depending on the network size linearly, where additional peers increased the runtime by 6.3 and 6.6 s for 2D and EEG datasets, respectively. Demonstrated negligible difference between implementations justifies the flexible choice of model in the blockchain-based federated learning and other machine learning applications.