This study applies reinforcement learning (RL) from the AI machine learning field to derive an optimal Bitcoin-like blockchain mining strategy. A salient feature of the RL learning framework is that an optimal (or near-optimal) strategy can be obtained without knowing the details of the blockchain network model. Previously, the most profitable mining strategy was believed to be honest mining encoded in the default blockchain protocol. It was shown later that it is possible to gain more mining rewards by deviating from honest mining. In particular, the mining problem can be formulated as a Markov Decision Process (MDP) which can be solved to give the optimal mining strategy. However, solving the mining MDP requires knowing the values of various parameters that characterize the blockchain network model. In real blockchain networks, these parameter values are not easy to obtain and may change over time. This hinders the use of the MDP model-based solution. In this study, we employ RL to dynamically learn a mining strategy with performance approaching that of the optimal mining strategy. Since the mining MDP problem has a nonlinear objective function (rather than linear functions of standard MDP problems), we design a new multidimensional RL algorithm to solve the problem. Experimental results indicate that, without knowing the parameter values of the mining MDP model, our multidimensional RL mining algorithm can still achieve optimal performance over time-varying blockchain networks.
The scale of criminal networks (e.g. drug syndicates and terrorist networks) extends globally and poses national security threat to many nations as they also tend to be technologically advance (e.g. Dark Web and Silk Road cryptocurrency). Therefore, it is critical for law enforcement agencies to be equipped with the latest tools in criminal network analysis (CNA) to obtain key hidden links (relationships) within criminal networks to preempt and disrupt criminal network structures and activities. Current hidden or missing link predictive models that are based on Social Network Analysis models rely on ML techniques to improve the performance of the models in terms of predictive accuracy and computing power. Given the improvement in the recent performance of Deep Reinforcement Learning (DRL) techniques which could train ML models through self-generated dataset, DRL can be usefully applied to domains with relatively smaller dataset such as criminal networks. The objective of this study is to assess the comparative performance of a CNA hidden link prediction model developed using DRL techniques against classical ML models such as gradient boosting machine (GBM), random forest (RF) and support vector machine (SVM). The experiment results exhibit an improvement in the performance of the DRL model of about 7.4% over the next best performing classical RF model trained within 1500 iterations. The performance of these link prediction models can be scaled up with the parallel processing capabilities of graphical processing units (GPUs), to significantly improve the speed of training the model and the prediction of hidden links.
Blockchain is a disruptive technology that is normally used within financial applications, however it can be very beneficial also in certain robotic contexts, such as when an immutable register of events is required. Among the several properties of Blockchain that can be useful within robotic environments, we find not just immutability but also decentralization of the data, irreversibility, accessibility and non-repudiation. In this paper, we propose an architecture that uses blockchain as a ledger and smart-contract technology for robotic control by using external parties, Oracles, to process data. We show how to register events in a secure way, how it is possible to use smart-contracts to control robots and how to interface with external Artificial Intelligence algorithms for image analysis. The proposed architecture is modular and can be used in multiple contexts such as in manufacturing, network control, robot control, and others, since it is easy to integrate, adapt, maintain and extend to new domains.
DI-fusion, le Dépôt institutionnel numérique de l'ULB, est l'outil de référencementde la production scientifique de l'ULB.L'interface de recherche DI-fusion permet de consulter les publications des chercheurs de l'ULB et les thèses qui y ont été défendues.
Swarms of robots will revolutionize many industrial applications, from targeted material delivery to precision farming. However, several of the heterogeneous characteristics that make them ideal for certain future applications --- robot autonomy, decentralized control, collective emergent behavior, etc. --- hinder the evolution of the technology from academic institutions to real-world problems. Blockchain, an emerging technology originated in the Bitcoin field, demonstrates that by combining peer-to-peer networks with cryptographic algorithms a group of agents can reach an agreement on a particular state of affairs and record that agreement without the need for a controlling authority. The combination of blockchain with other distributed systems, such as robotic swarm systems, can provide the necessary capabilities to make robotic swarm operations more secure, autonomous, flexible and even profitable. This work explains how blockchain technology can provide innovative solutions to four emergent issues in the swarm robotics research field. New security, decision making, behavior differentiation and business models for swarm robotic systems are described by providing case scenarios and examples. Finally, limitations and possible future problems that arise from the combination of these two technologies are described.
ubiant, Lyon,
France, Sébastien Mazac, Frédéric Armetta, Salima Hassas
The theory of cognitive development from Jean Piaget (1923) is a constructivist perspective of learning that has substantially influenced cognitive science domain.Indeed it seems that constructivism is a possible trail in order to overcome the limitations of classical techniques stemming from cognitivism or connectionism and create autonomous agents, fitted with strong adaptation ability within their environment, modelled on biological organisms.Potential applications concern intelligent agents in interaction with a complex environment, with objectives that cannot be predefined.There are numerous interesting works in developmental robotics going in this direction.In this work we investigate the application of these principles to a close domain: Ambient intelligence, which is extremely challenging but which also presents interesting aspects to exploit, like the participation of human users.From the perspective of a constructivist theory, the learning agent has to build a representation of the world that relies on the learning of sensori-motor patterns starting from its own experience only.This step is difficult to set up for systems evolving in continuous environments, using raw data from sensors without a priori modelling, primarily because they face a bootstrap problem.In this paper we address this particular issue and propose a decentralized approach based on a multi-agent framework, where the system's representations are constructed through a self-organization process that handles the dynamics between experience discretization and learning.