In the Ethereum network, miners are incentivized to include transactions in a block depending on the gas price specified by the sender. The sender of a transaction therefore faces a trade-off between timely inclusion and cost of his transaction. Existing recommendation mechanisms aggregate recent gas price data on a per-block basis to suggest a gas price. We perform an empirical analysis of historic block data to motivate the use of a predictive model for gas price recommendation. Subsequently, we propose a novel mechanism that combines a deep-learning based price forecasting model as well as an algorithm parameterized by a user-specific urgency value to recommend gas prices. In a comprehensive evaluation on real-world data, we show that our approach results on average in costs savings of more than 50% while only incurring an inclusion delay of 1.3 blocks, when compared to the gas price recommendation mechanism of the most widely used Ethereum client.
Alex Kaplunovich, Karuna Pande Joshi, Yelena Yesha
While adopting Blockchain technologies to automate their enterprise functionality, organizations are recognizing the challenges of scalability and manual configuration that the state of art present. Scalability of Hyperledger Fabric is an open challenge recognized by the research community. We have automated many of the configuration steps of installing Hyperledger Fabric Blockchain on AWS infrastructure and have benchmarked the scalability of that system. We have used the UCR (University of California Riverside) Time Series Archive with 128 timeseries datasets containing over 191,177 rows of data totaling 76,453,742 numbers. Using an automated Serverless approach, we have loaded this dataset, by chunks, into different AWS instances, triggering the load by SQS messaging. In this paper, we present the results of this benchmarking study and describe the approach we took to automate the Hyperledger Fabric processes using serverless Lambda functions and SQS triggering. We will also discuss what is needed to make the Blockchain technology more robust and scalable.
Cryptocurrencies, such as Bitcoin, are becoming increasingly popular, having been widely used as an exchange medium in areas such as financial transaction and asset transfer verification. However, there has been a lack of solutions that can support real-time price prediction to cope with high currency volatility, handle massive heterogeneous data volumes, including social media sentiments, while supporting fault tolerance and persistence in real time, and provide real-time adaptation of learning algorithms to cope with new price and sentiment data. In this paper we introduce KryptoOracle, a novel real-time and adaptive cryptocurrency price prediction platform based on Twitter sentiments. The integrative and modular platform is based on (i) a Spark-based architecture which handles the large volume of incoming data in a persistent and fault tolerant way; (ii) an approach that supports sentiment analysis which can respond to large amounts of natural language processing queries in real time; and (iii) a predictive method grounded on online learning in which a model adapts its weights to cope with new prices and sentiments. Besides providing an architectural design, the paper also describes the KryptoOracle platform implementation and experimental evaluation. Overall, the proposed platform can help accelerate decision-making, uncover new opportunities and provide more timely insights based on the available and ever-larger financial data volume and variety.
This study applies reinforcement learning (RL) from the AI machine learning field to derive an optimal Bitcoin-like blockchain mining strategy. A salient feature of the RL learning framework is that an optimal (or near-optimal) strategy can be obtained without knowing the details of the blockchain network model. Previously, the most profitable mining strategy was believed to be honest mining encoded in the default blockchain protocol. It was shown later that it is possible to gain more mining rewards by deviating from honest mining. In particular, the mining problem can be formulated as a Markov Decision Process (MDP) which can be solved to give the optimal mining strategy. However, solving the mining MDP requires knowing the values of various parameters that characterize the blockchain network model. In real blockchain networks, these parameter values are not easy to obtain and may change over time. This hinders the use of the MDP model-based solution. In this study, we employ RL to dynamically learn a mining strategy with performance approaching that of the optimal mining strategy. Since the mining MDP problem has a nonlinear objective function (rather than linear functions of standard MDP problems), we design a new multidimensional RL algorithm to solve the problem. Experimental results indicate that, without knowing the parameter values of the mining MDP model, our multidimensional RL mining algorithm can still achieve optimal performance over time-varying blockchain networks.
Abstract A stable reference of Internet resources is crucial not only to identify a resource in a trustworthy and certified way but also to guarantee continuous access to it over time. The current practice in scientific publication as the use of a Persistent Identifier (PID) like a DOI or Handle, is becoming attractive also for the datasets. In fact, in the era of Big Data, the aspects of replicability and verification of the scientific result are paramount. In this paper we verify the functional feasibility of permissioned blockchain technology as a tool to implement a Trustworthy Persistent Identifier (T-PID) system for datasets in the scientific domain.
Francesco Zola, Jan L. Bruse, Maria Eguimendia, Mikel Galar · 5 authors
The Bitcoin network not only is vulnerable to cyber-attacks but currently represents the most frequently used cryptocurrency for concealing illicit activities. Typically, Bitcoin activity is monitored by decreasing anonymity of its entities using machine learning-based techniques, which consider the whole blockchain. This entails two issues: first, it increases the complexity of the analysis requiring higher efforts and, second, it may hide network micro-dynamics important for detecting short-term changes in entity behavioral patterns. The aim of this paper is to address both issues by performing a “temporal dissection” of the Bitcoin blockchain, i.e., dividing it into smaller temporal batches to achieve entity classification. The idea is that a machine learning model trained on a certain time-interval (batch) should achieve good classification performance when tested on another batch if entity behavioral patterns are similar. We apply cascading machine learning principles—a type of ensemble learning applying stacking techniques—introducing a “k-fold cross-testing” concept across batches of varying size. Results show that blockchain batch size used for entity classification could be reduced for certain classes (Exchange, Gambling, and eWallet) as classification rates did not vary significantly with batch size; suggesting that behavioral patterns did not change significantly over time. Mixer and Market class detection, however, can be negatively affected. A deeper analysis of Mining Pool behavior showed that models trained on recent data perform better than models trained on older data, suggesting that “typical” Mining Pool behavior may be represented better by recent data. This work provides a first step towards uncovering entity behavioral changes via temporal dissection of blockchain data.
Bitcoin-NG, a scalable blockchain protocol, divides each block into a key block and many micro blocks to effectively improve the transaction processing capacity. Bitcoin-NG has a special incentive mechanism (i.e. splitting transaction fees to the current and the next leader) to maintain its security. However, this design of the incentive mechanism ignores the joint effect of transaction fees, mint coins and mining duration lengths on the expected mining reward. In this paper, we identify the advanced mining attack that deliberately ignores micro blocks to enlarge the mining duration length to increase the likelihood of winning the mining race. We first show that an advanced mining attacker can maximize its expected reward by optimizing its mining duration length. We then formulate a game-theoretical model in which multiple mining players perform advanced mining to compete with each other. We analyze the Nash equilibrium for the mining game. Our analytical and simulation results indicate that all mining players in the mining game converge to having advanced mining at the equilibrium and have no incentives for deviating from the equilibrium; the transaction processing capability of the Bitcoin-NG network at the equilibrium is decreased by advanced mining. Therefore, we conclude that the Bitcoin-NG blockchain protocol is vulnerable to advanced mining attack. We discuss how to reduce the negative impact of advanced mining for Bitcoin-NG.
In Bitcoin, if a miner is able to solve a computationally hard problem called proof of work, it will receive an amount of bitcoin as a reward which is the sum of the fees for the transactions included in a block plus an amount inversely proportional to the number of blocks discovered so far. At the moment of writing, the block reward is several orders of magnitude greater than the sum of transaction fees. Usually, miners try to collect the largest reward by including transactions associated with high fees. The main purpose of transaction fees is to prevent network spamming. However, they are also used to prioritize transactions. In order to use the minimum amount of fees, users usually have to find a compromise between fees and urgency of a transaction. In this paper, we develop a probabilistic logic model to experimentally analyze how fees affect confirmation time and miner’s revenue and to predict if an increase of average fees will generate a situation when the miner gets more reward by not following the protocol.
Evan Brinckman, Andrey Kuehlkamp, Jarek Nabrzyski, Ian Taylor
As the public Ethereum network surpasses half a billion transactions and enterprise Blockchain systems becoming highly capable of meeting the demands of global deployments, production Blockchain applications are fast becoming commonplace across a diverse range of business and scientific verticals. In this paper, we reflect on work we have been conducting recently surrounding the ingestion, retrieval and analysis of Blockchain data. We describe the scaling and semantic challenges when extracting Blockchain data in a way that preserves the original metadata of each transaction by cross referencing the Smart Contract interface with the on-chain data. We then discuss a scientific use case in the area of Scientific workflows by describing how we can harvest data from tasks and dependencies in a generic way. We then discuss how crawled public blockchain data can be analyzed using two unsupervised machine learning algorithms, which are designed to identify outlier accounts or smart contracts in the system. We compare and contrast the two machine learning methods and cross correlate with public Websites to illustrate the effectiveness such approaches.
Machine learning and blockchain are two of the most notable technologies of recent years. The first is the foundation of artificial intelligence and big data analysis, and the second has significantly disrupted the financial industry. Both technologies are data‐driven, and thus there are rapidly growing interests in integrating both for more secure and efficient data sharing and analysis. In this article, we review existing research on combining machine learning and blockchain technologies and demonstrate that they can collaborate efficiently and effectively. In the end, we point out some future directions and expect more research on deeper integration of these two promising technologies.
Naman Goel, Cyril van Schreven, Aris Filos-Ratsikas, Boi Faltings
Blockchain based systems allow various kinds of financial transactions to be executed in a decentralized manner. However, these systems often rely on a trusted third party (oracle) to get correct information about the real-world events, which trigger the financial transactions. In this paper, we identify two biggest challenges in building decentralized, trustless and transparent oracles. The first challenge is acquiring correct information about the real-world events without relying on a trusted information provider. We show how a peer-consistency incentive mechanism can be used to acquire truthful information from an untrusted and self-interested crowd, even when the crowd has outside incentives to provide wrong informations. The second is a system design and implementation challenge. For the first time, we show how to implement a trustless and transparent oracle in Ethereum. We discuss various non-trivial issues that arise in implementing peer-consistency mechanisms in Ethereum, suggest several optimizations to reduce gas cost and provide empirical analysis.
Ethereum is the largest blockchain platform supporting smart contracts with the second biggest market capitalization. Ethereum data can yield many useful insights because of the large volume of transactions, accounts and blocks as well as the popular applications developed as smart contracts. Studying Ethereum data can also reveal many new attacks to the platform and its smart contracts. Unfortunately, it is non-trivial to systematically explore Ethereum because it involves massive heterogeneous data, which are produced and stored in different ways. Although a few recent studies report some interesting observations about Ethereum, they are limited by their data acquisition methods which cannot provide comprehensive and precise data. In this paper, to fill the gap, we propose DataEther, a systematic and high-fidelity data exploration framework for Ethereum by exploiting its internal mechanisms. Besides supporting the analyses in existing studies, DataEther further empowers users to explore unknown phenomena and obtain in-depth understandings. We first describe how we tackle the challenging issues in developing DataEther, and then use four data-centric applications to demonstrate its usage and report many new observations.
Predicting the trends in Bitcoin market prices is a very challenging task due to the many uncertainties and variables influencing the market value. The market is susceptible to quick changes, causing seemingly random fluctuations in the Bitcoin price. Due to the chaotic and highly volatile nature of Bitcoin behavior, investments come with high risk. To minimize the risk involved, knowledge of the Bitcoin price movement in the future is desirable. Different studies have shown that Machine Learning algorithms can predict, to varying degrees, the price fluctuations of Bitcoin. However, most researches do not explore the relationship between the price and other features outside the transaction network, such as market capitalization, Bitcoin mining speed, or entity behavior. Also, most of the features are extracted from the network level, which means obtaining the number of transactions, users, Bitcoins mined, etc. In this research, we focus on additional features, such as features outside the transaction network and node-based features inside the transaction network, which could improve the price prediction of Bitcoin. The investigated features are the “fairness and goodness” measure and the “1-ARW-betweenness cen- trality” measure. Fairness and goodness are entity behavior measures. The goodness of a Bitcoin address captures how much this address is liked/trusted by other addresses, while the fairness of a Bitcoin address captures how fair the address is in rating other addresses’ likeability or trust level. The 1-ARW-betweenness centrality is a feature based on absorbing random walks. The feature captures the extent to which a Bitcoin address has control over the money flow between different addresses. A benchmark, based on the machine learning algorithm Random Forest with commonly used features, is used to test the impact of the additional features. The Random Forest tries to predict the sign (up-down movement) of the price per day, using data from the two previous days. Comparing this benchmark with a similar model, but then including the additional features, will gain more information about how these additional features influence the Bitcoin price.
Francesco Zola, Cristina Pérez‐Solà, Jon Egana, Maria Eguimendia · 5 authors
Bitcoin is a peer-to-peer distributed cryptocurrency system, that keeps all transaction history in a public ledger known as blockchain. The Bitcoin network is implicitly pseudoanonymous and its nodes are controlled by independent entities making network analysis difficult. This calls for the development of a fully controlled testing environment. This paper presents Kriptosare.gen, a dockerized automatized Bitcoin testbed, for deploying full-scale custom Bitcoin networks. The testbed is deployed in a single machine executing four different experiments, each one with different network configuration. We perform a cost analysis to investigate how the resources are related with network parameters and provide experimental data quantifying the amount of computational resources needed to run the different types of simulations. Obtained results demonstrate that it is possible to run the testbed with a configuration similar to a real Bitcoin system.
Blockchain technology, and more specifically Bitcoin (one of its foremost applications), have been receiving increasing attention in the scientific community. The first publications with Bitcoin as a topic, can be traced back to 2012. In spite of this short time span, the production magnitude (1162 papers) makes it necessary to make a bibliometric study in order to observe research clusters, emerging topics, and leading scholars. Our paper is aimed at studying the scientific production only around bitcoin, excluding other blockchain applications. Thus, we restricted our search to papers indexed in the Web of Science Core Collection, whose topic is "bitcoin". This database is suitable for such diverse disciplines such as economics, engineering, mathematics, and computer science. This bibliometric study draws the landscape of the current state and trends of Bitcoin-related research in different scientific disciplines.
The Internet of Things (IoT) is a remarkable data producer and these data may be used to prevent or detect security vulnerabilities and increase productivity by the adoption of statistical and Artificial Intelligence (AI) techniques. However, these desirable benefits are gained if data from IoT networks are dependable—this is where blockchain comes into play. In fact, through blockchain, critical IoT data may be trusted, i.e., considered valid for any subsequent processing. A simple formal model named “the Mirror Model” is proposed to connect IoT data organized in traditional models to assets of trust in a blockchain. The Mirror Model sets some formal conditions to produce trusted data that remain trusted over time. A possible practical implementation of an application programming interface (API) is proposed, which keeps the data and the trust model in synch. Finally, it is noted that the Mirror Model enforces a top-down approach from reality to implementation instead of going the opposite way as it is now the practice when referring to blockchain and the IoT.
We study the applicability of blockchain technology for distributed event detection under resource constraints. Therefore we provide a test-suite with several promising consensus methods (Proof-of-Work, Proof-of-Stake, Distributed Proof-of-Work, and Practical Proof-of-Kernel-Work). This is the first work analyzing the communication costs of blockchain consensus methods for knowledge discovery tasks in resource constraint devices. The experiments reveal that our proposed implementations of Distributed Proof-of-Work and Practical Proof-of-Kernel-Work provide a benefit over Proof-of-Work in CPU usage and communication costs. The tests show further that in cases of low data rates, where latencies by mining do not cause harm proposed blockchain implementations could be integrated. However, usage of blockchain requires data broadcasts, which leads to communication overhead as well as memory requirements based on the address list.
Patrick Ocheja, Brendan Flanagan, Hiroshi Ueda, Hiroaki Ogata
It is a common practice to issue a summary of a learner’s learning achievements in form of a transcript or certificate. However, detailed information on the depth of learning and how learning or teachings were conducted is not present in the transcript of scores. This work presents the first practical implementation of a new platform for keeping track of learning achievements beyond transcripts and certificates. This is achieved by maintaining digital hashes of learning activities and managing access rights through the use of smart contracts on the blockchain. The blockchain of learning logs (BOLL) is a platform that enable learners to move their learning records from one institution to another in a secure and verifiable format. This primarily solves the cold-start problem faced by learning data analytic platforms when trying to offer personalized experience to new learners. BOLL enables existing learning data analytic platforms to access the learning logs from other institutions with the permission of the learners and/or institution who originally have ownership of the logs. The main contribution of this paper is to investigate how learning records could be connected across institutions using BOLL. We present an overview of how the implementation has been carried out, discuss resource requirements, and compare the advantages BOLL has over other similar tools.
Mining pools, the main components of the Bitcoin network, dominate the computing resources and play essential roles in network security and performance aspects. Although many existing measurements of the Bitcoin network are available, little is known about the details of mining pool behaviors (e.g., empty blocks, mining revenue and transaction collection strategies) and their effects on the Bitcoin end users (e.g., transaction fees, transaction delay and transaction acceptance rate). This paper aims to fill this gap with a systematic study of mining pools. We traced over 1.56 hundred thousand blocks (including about 257 million historical transactions) from February 2016 to January 2019 and collected over 120.25 million unconfirmed transactions from March 2018 to January 2019. Then we conducted a board range of measurements on the pool evolutions, labeled transactions (blocks) as well as real-time network traffics, and discovered new interesting observations and features. Specifically, our measurements show the following. 1) A few mining pools entities continuously control most of the computing resources of the Bitcoin network. 2) Mining pools are caught in a prisoner's dilemma where mining pools compete to increase their computing resources even though the unit profit of the computing resource decreases. 3) Mining pools are stuck in a Malthusian trap where there is a stage at which the Bitcoin incentives are inadequate for feeding the exponential growth of the computing resources. 4) The market price and transaction fees are not sensitive to the event of halving block rewards. 5) The block interval of empty blocks is significantly lower than the block interval of non-empty blocks. 6) Feerate plays a dominating role in transaction collection strategy for the top mining pools. Our measurements and analysis help to understand and improve the Bitcoin network.
Open access
3 source records
Blockchain Technology Applications and Security
Data Stream Mining Techniques
Advanced Steganography and Watermarking Techniques
Evangelos Pournaras, Srivatsan Yadhunathan, Ada Diaconescu
Structure plays a key role in learning performance. In centralized computational systems, hyperparameter optimization and regularization techniques such as dropout are computational means to enhance learning performance by adjusting the deep hierarchical structure. However, in decentralized deep learning by the Internet of Things, the structure is an actual network of autonomous interconnected devices such as smart phones that interact via complex network protocols. Self-adaptation of the learning structure is a challenge. Uncertainties such as network latency, node and link failures or even bottlenecks by limited processing capacity and energy availability can significantly downgrade learning performance. Network self-organization and self-management is complex, while it requires additional computational and network resources that hinder the feasibility of decentralized deep learning. In contrast, this paper introduces a self-adaptive learning approach based on holarchic learning structures for exploring, mitigating and boosting learning performance in distributed environments with uncertainties. A large-scale performance analysis with 864,000 experiments fed with synthetic and real-world data from smart grid and smart city pilot projects confirm the cost-effectiveness of holarchic structures for decentralized deep learning.
The cryptocurrency market is a very huge market without effective supervision. It is of great importance for investors and regulators to recognize whether there are market manipulation and its manipulation patterns. This paper proposes an approach to mine the transaction networks of exchanges for answering this question. By taking the leaked transaction history of Mt. Gox Bitcoin exchange as a sample, we first divide the accounts into three categories according to its characteristic and then construct the transaction history into three graphs. Many observations and findings are obtained via analyzing the constructed graphs. To evaluate the influence of the accounts' transaction behavior on the Bitcoin exchange price, the graphs are reconstructed into series and reshaped as matrices. By using singular value decomposition (SVD) on the matrices, we identify many base networks which have a great correlation with the price fluctuation. When further analyzing the most important accounts in the base networks, plenty of market manipulation patterns are found. According to these findings, we conclude that there was serious market manipulation in Mt. Gox exchange and the cryptocurrency market must strengthen the supervision.
With the rapid development of the Internet and the Internet of Things, product recall, especially pharmaceutical recall, have attracted various stakeholders as a phase of product service. However, there are some the long-term problems on pharmaceutical recall, such as low effectiveness, seldom transparent, data can be tampered easily and so on. We have proposed a service system of pharmaceutical recall based on blockchain to solve these problems. The solution runs through the entire recall process that includes decision making, the OOS/OOT investigation, the action of working group and the Corrective and Preventative Action (CAPA). Through the management of recall, we are capable of shortening time, enhancing transparency and protecting the integrity of data in pharmaceutical recall.
Applications of blockchain technologies got a lot of attention in recent years. They exceed beyond exchanging value and being a substitute for fiat money and traditional banking system. Nevertheless, being able to exchange value on a blockchain is at the core of the entire system and has to be reliable. Blockchains have built-in mechanisms that guarantee whole system's consistency and reliability. However, malicious actors can still try to steal money by applying well known techniques like malware software or fake emails. In this paper we apply supervised learning techniques to detect fraudulent accounts on Ethereum blockchain. We compare capabilities of Random Forests, Support Vector Machines and XGBoost classifiers to identify such accounts basing on a dataset of more than 300 thousands accounts. Results show that we are able to achieve recall and precision values allowing for the designed system to be applicable as an anti-fraud rule for digital wallets or currency exchanges. We also present sensitivity analysis to show how presented models depend on particular feature and how lack of some of them will affect the overall system performance.