TRep: temperature-aware data replication for web3 data infrastructure

Abstract

The proliferation of web3 has underscored the need for robust and efficient decentralized data infrastructure. However, prominent decentralized storage systems like IPFS suffer from poor data availability and the risk of cold data loss, while incentive-based solutions like Filecoin and Storj employ uniform replication strategies that are often inefficient and costly. These systems fail to distinguish between frequently accessed ("hot") and infrequently accessed ("cold") data, leading to a suboptimal allocation of storage resources. This paper introduces TRep, a novel Temperature-aware Data Replication strategy designed to address these challenges. TRep implements a dynamic replication policy by classifying data based on its access temperature. The system is built on three key innovations: (1) a highly accurate data classification algorithm using double exponential smoothing to analyze access frequency and trends; (2) an intelligent data distribution strategy that selects optimal backup nodes based on storage availability, online stability, and network proximity; and (3) a dynamic backup regulation mechanism, managed by a regulator node, to globally control replica counts for hot and cold data. Our performance evaluation demonstrates that TRep's classification algorithm significantly reduces prediction errors compared to traditional methods. Furthermore, TRep achieves speeds up to four times faster than IPFS for large files while effectively managing storage overhead. TRep thus provides a flexible and performant framework for data replication, enhancing the durability and efficiency of web3 data infrastructure.