Blockchain-Based Crowdsourcing Framework for Machine Learning Ground Truth

Abstract

Machine learning has evolved from a lab curiosity to a widely used technology that is fundamentally reliant on ground truth data for model training and evaluation. This research addresses the challenges in obtaining accurate ground truth data due to limited domain expertise, sparse and unrepresentative datasets, and the high costs associated with data acquisition. The quality of this data significantly influences the reliability of machine learning models, prompting research into methods to improve ground-truth reliability. This research proposes a framework that utilize blockchain-based crowdsourcing for ground-truth data annotation. Blockchain technology, with its decentralized immutable ledger system, offers a secure method for data verification and collection from decentralized entities. The proposed framework was implemented in an Ethereum network environment using blockchain technology and smart contracts. Next, we evaluated the collected ground truth by measuring the inter-rater agreement among the participants. The experimental results indicate that blockchain can enhance annotation consistency, showing a higher reliability of crowd-sourced data compared to expert opinions. Most annotator pairs demonstrated moderate to strong agreement, confirming the potential of blockchain technology in improving ground truth data annotation.