Securing Blockchain-Based Distributed Learning for Heterogeneous Clients Through Knowledge Distillation and Zero-Knowledge Proofs

Abstract

Distributed learning (DL) is gaining popularity as it enables clients (e.g., AI Agents) to enhance their machine learning (ML) models’ performance by exchanging knowledge without revealing private datasets. State-of-the-art DL approaches primarily focus on transferring knowledge between heterogeneous clients with diverse model architectures, connecting clients with those that can improve their models, and protecting data privacy. However, they overlook the threat of malicious clients that potentially downgrade the models’ performance by sharing inaccurate knowledge or excluding high-performing clients from the training procedure.Therefore, we introduce the Zero-Knowledge Blockchain-Based Knowledge Distillation Learning Framework (zkBKD). In zkBKD, heterogeneous clients communicate with a blockchain network to discover high-performing clients, verify zero-knowledge proofs (ZKPs) to ensure the correctness of the knowledge shared from other clients, and vote to eliminate malicious clients. We analyze security and privacy risks and show that zkBKD prevents membership, poisoning, and collusion attacks. We conduct extensive experiments on two standard datasets across heterogeneous clients with four model architectures. The experimental results demonstrate that zkBKD relatively improves the average model accuracy of all clients by 25.71%. Even lightweight models such as ResNet-2 achieve up to a 103.35% accuracy gain compared to independent training.