Ensuring the Big Data Traceability in Heterogeneous Data Systems

Abstract

The work corresponds to the security of Big Data, the key element of the modern digital world. The distributed ledger technology is widely applied to achieve data integrity and confidentiality. However, all relevant solutions are grounded in specific application environments and platforms, what determines the goal to create a universal approach in which Big Data processing components can vary and interchange. The paper proposes a method for ensuring the traceability of Big Data processing in heterogeneous data systems. Traceability is the ability to ensure that data is completely traceable across the entire system. This allows data to be easily followed all the way back to the original sources, audited and controlled. Big Data processing models have been presented. HashGraph and Blockchain distributed ledger mechanisms have been adapted to monitor Big Data processing chains. For tracing Big Data, the appropriate components of the distributed ledger have been modified and implemented. The experiments have been arranged on the test data system processing the big network traffic input, and it has been confirmed that the proposed solution is universal for the ledger technology, does not depend on the Big Data processing platform, and provides Big Data security preserving the computing resources of the protected system.