BHODM: A Blockchain-Based Heuristic Deduplication Model for Secure, Transparent, and Redundancy-Aware Cloud Storage

Abstract

Cloud storage systems have become an integral part of modern data management by enabling users to store and access data remotely.However, traditional cloud storage architectures rely on centralized servers, which introduce several critical challenges such as single-point failure, redundant data storage, increased storage costs, and security vulnerabilities.In earlier systems, data was stored in centralized data centers where duplicate files were repeatedly maintained, leading to inefficient utilization of storage resources.Although basic deduplication methods were implemented, they often compromised data confidentiality and lacked transparency in metadata management.Furthermore, failure of the central server could result in permanent data loss.To overcome these issues, this research proposes the Blockchain-enabled Heuristic Optimized Deduplication Model (BHODM), which integrates blockchain technology, InterPlanetary File System (IPFS), Convergent Encryption (CE), and heuristic-based chunking techniques.In this model, files are divided into optimized chunks based on file size using a heuristic approach.Each chunk is encrypted using CE, where the encryption key is derived from the hash of the data itself, allowing secure deduplication without exposing plaintext data.Duplicate chunks are identified through hash comparison, ensuring that only unique data is stored.The encrypted chunks are stored in IPFS, a decentralized peer-to-peer storage network, while metadata such as file names, block numbers, and hash values are securely maintained in an Ethereum blockchain smart contract, ensuring immutability and transparency.The system is implemented using Django, Web3, IPFS API, and AES-CTR encryption.Experimental results based on storage utilization and computation time demonstrate improved efficiency over traditional approaches.