As a new investment method, quantitative investment is expanding its market scale and share due to its stable investment performance. In this paper we propose a prediction model based on LSTM. It is helpful for the traders to predict the future price to formulate the best trading strategy. Using this model, we can precisely forecast each price separately to determine when the asset should be traded based on future price fluctuations. Simulation results show that our model can successfully predict the future price trend of the two assets within the acceptable range of error, which helps us to better optimize our portfolio. In addition, the RMSE (root mean square error) is selected as the loss function to describe the accuracy of our prediction model.
The objective of this paper is to assess the performances of dimensionality reduction techniques to establish a link between cryptocurrencies. We have focused our analysis on the two most traded cryptocurrencies: Bitcoin and Ethereum. To perform our analysis, we took log returns and added some covariates to build our data set. We first introduced the pearson correlation coefficient in order to have a preliminary assessment of the link between Bitcoin and Ethereum. We then reduced the dimension of our data set using canonical correlation analysis and principal component analysis. After performing an analysis of the links between Bitcoin and Ethereum with both statistical techniques, we measured their performance on forecasting Ethereum returns with Bitcoin s features.
Single- or multi-layer perceptrons, commonly known as neural networks, are universal approximators that can approximate any continuous functions arbitrarily well when the number of perceptrons is allowed to grow indefinitely. When prior knowledge about the target function is available, constraints can be imposed on the neural network to improve approximation accuracy. As an example, it is well known that convolutional neural networks (CNN) \cite{lecun1990handwritten} yield good performance on image classification. And its parameter-sharing scheme can also reduce the risk of overfitting. One of the key properties of image classification is that it is invariant to translation of input image. The translation invariance in CNN is achieved with two steps. On the one hand, the convolutional layer gives a translation equivariance as it is a linear time invariant system. The translation of the input image will be kept as a transformation of the output. On the other hand, the pooling layer will introduce local invariance. The stack of convolutional layers and pooling layers will then reach a receptive field the same size as input and also enforces global invariance. Inspired by the success of CNN on image related tasks, plenty of CNN generalizations \cite{gens2014deep, henaff2015deep, simonovsky2017dynamic, cohen2018spherical,chidester2018rotation} have been studied in other tasks. \emph{Domain invariance}, which refers to the property that the output is invariant to certain transformation of the input features is a crucial property which can help to generalize CNN to other machine learning applications. The works which introduce domain invariance fall into two categories. One of the categories introduces the invariance by injecting invariant constrains. The other category builds the equivariant network layer then achieves invariance by applying the pooling or normalization layer. In the equivariant network layer, the input is transformed by some group transformation the output is transformed correspondingly. As in the CNN example, the invariance can be achieved by first using equivariant layers then apply the pooling layer. In this dissertation, we will focus on the invariance and equivariance of neural networks and analyze the neural network architecture which can achieve invariance or equivariance. The main works can be summarized as following. \noindent\textbf{The Connections Between Convolutional Architecture and Equivariant Property} Convolutional neural networks have achieved great success in speech, image, and video signal processing tasks in recent years. There have been several attempts to justify the convolutional architecture and to generalize the convolution operation for treatment of other data types such as graphs and manifolds. Based on group representation theory and noncommutative harmonic analysis, it has recently been shown that the so-called group equivariance requirement of a feed-forward neural network necessitates the convolutional architectures. In our work, based on the familiar concepts of linear time-invariant systems, we develop an elementary proof of the same result. The nonlinear activation function, being a necessary components of practical deep neural networks, has been glossed over in previous analyses of the connection between equivariance and convolution. We identify sufficient conditions for the non-linear activation functions to preserve equivariance, and hence the necessity of the group convolution structure. Our analysis method is simple and intuitive, and holds the potential to be applied to more challenging scenarios such as non-transitive domains and multiple simultaneous equivariances. \noindent\textbf{Characteristics of Generalized Convolutional Neural Networks} Based on the analysis method above, a theoretical support for details of network architecture design is then provided in our work. We first analyze the necessary and sufficient condition for a neural network to be group equivariant when the group acts on the sub-domain of input/output. This part gives a theoretical support for how should we combine channels in a neural network layer. Meanwhile, it also gives guidelines for designing neural networks with partial equivariance/invariance. To further investigate the characteristics of generalized convolutional neural networks, we then analyze the multiple equivariance case. The result shows the connection between model parameter sharing and equivariant property. In particular, we find that a model will have more parameter sharing if more equivariances are required. Meanwhile, we establish a universality result for shallow and deep generalized group convolutional neural networks as approximators of continuous group-equivariant functions. After choosing the network architecture, one common question we need to answer is that how many neurons do we want in our hidden layer? We show that the generalized convolution mapping to a quotient space is a projection of the image of a generalized convolution which maps to the maximum quotient space. This can be used to obtain guidelines for choosing the feature size of hidden layer. \noindent\textbf{Neural Networks with Invariant Properties} In the traditional convolutional neural networks, the spatial invariant properties are introduced by adding a global pooling layer after the convolutional layers. For some specific applications, there are still some potential for us to further reduce the model complexity by introduce invariant property in earlier layers. An partial permutation invariant model for learning graph node embedding is introduced based on the classical invariant theory. In the image processing application, an affine equivariant preprocessing method is proposed to build neural networks invariant to affine transformation. \noindent\textbf{Partially Permutation Invariant Graph Node Embedding Model} Graph node embedding aims at learning a vector representation for all nodes given a graph. It is a central problem in many machine learning tasks (e.g., node classification, recommendation, community detection). The key problem in graph node embedding lies in how to define the dependence to neighbors. Existing approaches specify (either explicitly or implicitly) certain dependencies on neighbors, which may lead to loss of subtle but important structural information within the graph and other dependencies among neighbors. This intrigues us to ask the question: can we design a model to give the maximal flexibility of dependencies to each node's neighborhood. In our recent work, we propose a novel graph node embedding method (named \textbf{\OM}) via a novel notion of \textit{partial permutation invariant set function}, to capture any possible dependence. The partial permutation invariant set function is designed based on the set of \textit{invariant basis} with respect to permutation group. Our method 1) can learn an \textit{arbitrary} form of the representation function from the neighborhood, without losing any potential dependence structures, and 2) is applicable to both homogeneous and heterogeneous graph embedding, the latter of which is challenged by the diversity of node types. Furthermore, we provide theoretical guarantee for the representation capability of our method for general homogeneous and heterogeneous graphs. Empirical evaluation results on benchmark data sets show that our proposed {\OM} method outperforms the state-of-the-art approaches on producing node vectors for various learning tasks of both homogeneous and heterogeneous graphs. \noindent\textbf{Learning Models Invariant to Affine Transformations for Image Processing} The way designing invariant/partial invariant neural networks with invariant basis is difficult to extend to groups other than permutation group. Because the invariant basis is expensive to calculate. Even if we get the set of invariant basis, it will be very large and is not helpful to reduce model complexity. An alternative which can introduce invariant property to neural network is adding an equivariant transformer as a pre-processing layer. This equivariant transformer will revert all the affine transformations happens in the input. Thus the model will be invariant to affine transformations. Invariance to affine transformations is desirable property for many computer vision tasks like image classification and motion tracking. Enforcing and taking advantage of such affine invariance property is an essential part for building efficient models for machine learning. However, it has been challenging to build models that are provably affine invariant. Previous works have achieved partial invariance, such as invariance to shifting and to rotation. We propose to build an affine-invariant model in two steps. First, we apply image normalization, which renders the density function represented by the image to have zero mean and identity covariance matrix. The resulting normalized image is then processed by a second stage neural network that is either an equivariant transformer network (ETN) or group convolution networks. We provide proof of invariance and demonstrate the performance improvement of such neural networks in processing images that have been affine transformed.
This study will investigate the predictability of a Chaotic time-series data using Reservoir computing (Echo State Network), Deep-Learning(LSTM) and Machine- Learning(Linear, Bayesian, ElasticNetCV , Random Forest, XGBoost Regression and a machine learning Neural Network) on the short (1-day out prediction), medium (5-day out prediction) and long-term (30-day out prediction) pricing of Bitcoin and Ethereum Using a range of machine learning tools, to perform feature selection by permutation importance to select technical indicators on the individual cryptocurrencies, to ensure the datasets are the best for predictions per cryptocurrency while reducing noise within the models. The predictability of these two chaotic time-series is then compared to evaluate the models to find the best fit model. The models are fine-tuned, with hyperparameters, design of the network within the LSTM and the reservoir size within the Echo State Network being adjusted to improve accuracy and speed. This research highlights the effect of the trends within the cryptocurrency and its effect on predictive models, these models will then be optimized with hyperparameter tuning, and be evaluated to compare the models across the two currencies. It is found that the datasets for each cryptocurrency are different, due to the different permutation importance, which does not affect the overall predictability of the models with the short and medium-term predictions having the same models being the top performers. This research confirms that the chaotic data although can have positive results for shortand medium-term prediction, for long-term prediction, technical analysis basedprediction is not sufficient.
This paper is discusses the problems of the short-term forecasting of financial time series using supervised machine learning (ML) approach. For this goal, we applied several the most powerful methods including Support Vector Machine (SVM), Multilayer Perceptron (MLP), Random Forests (RF) and Stochastic Gradient Boosting Machine (SGBM). As dataset were selected the daily close prices of two stock index: SP 500 and NASDAQ, two the most capitalized cryptocurrencies: Bitcoin (BTC), Ethereum (ETH), and exchange rate of EUR-USD. As features we used only the past price information. To check the efficiency of these models we made out-of-sample forecast for selected time series by using one step ahead technique. The accuracy rates of the forecasted prices by using ML models were calculated. The results verify the applicability of the ML approach for the forecasting of financial time series. The best out of sample accuracy of short-term prediction daily close prices for selected time series obtained by SGBM and MLP in terms of Mean Absolute Percentage Error (MAPE) was within 0.46-3.71 %. Our results are comparable with accuracy obtained by Deep learning approaches.
Abstract The paper deals with the architecture and design of the complex distributed classifier for the intelligent video surveillance systems considering the contemporary tendency to detect the abnormal or suspicious behavior of the individuals by means of behavioral features set analysis. This paper focuses on the implementation of multiagent systems concept and the distributed ledger technology to the distributed message board architecture. Two selected approaches to the distributed ledger implementation are analyzed and estimated in terms of classifiers cooperation. Some simulation results are provided and discussed in terms of time consumption.
Clément Bénard, Gérard Biau, Sébastien da Veiga, Erwan Scornet
We introduce SIRUS (Stable and Interpretable RUle Set) for regression, a stable rule learning algorithm which takes the form of a short and simple list of rules. State-of-the-art learning algorithms are often referred to as "black boxes" because of the high number of operations involved in their prediction process. Despite their powerful predictivity, this lack of interpretability may be highly restrictive for applications with critical decisions at stake. On the other hand, algorithms with a simple structure-typically decision trees, rule algorithms, or sparse linear models-are well known for their instability. This undesirable feature makes the conclusions of the data analysis unreliable and turns out to be a strong operational limitation. This motivates the design of SIRUS, which combines a simple structure with a remarkable stable behavior when data is perturbed. The algorithm is based on random forests, the predictive accuracy of which is preserved. We demonstrate the efficiency of the method both empirically (through experiments) and theoretically (with the proof of its asymptotic stability). Our R/C++ software implementation sirus is available from CRAN.
zkSNARKS can be described as zero-knowledge: No secret information is revealed by the proof; Succinct: The size of the proof that is generated is small; Non-interactive: no challenge-response protocol; and ARgument of Knowledge: It is computationally intractable for the prover to produce a fake proof.
Artificial neural networks are modern methods suitable for solving the problem of nonlinear dependency approximation, which is successfully applied in many fields. This paper compares the predictive capabilities of Back Propagation, Radial Basis Function, Extreme Learning Machine, and Long-Short Term Memory neural networks to determine which artificial intelligence algorithm is best for modeling the price of Bitcoin opening. The criterion for comparing network performance was the standard deviation, the mean absolute deviation, and the accuracy of predicting the direction of change of course. At the same time, in the study of time series, it is recommended to perform a comprehensive data analysis using appropriate networks, depending on the length of the series and the specificity of the database.
Nor Azizah Hitam, Amelia Ritahani Ismail, Faisal Saeed
Forecasting accurate future price is very important in financial sector. An optimized Support Vector Machine (SVM) based on Particle Swarm Optimization (PSO) is introduced in forecasting the cryptocurrency future price. It is part of Artificial Intelligence (AI) that uses previous experience to forecast future price. Analysts and investors generally combine fundamental and technical analysis prior to decide the best price to execute their trades. Some may use Machine Learning Algorithms to execute their trades. However, forecasting result using basic SVM algorithms does not really promising. On the other hands, Particle Swarm Optimization (PSO) is known as a better algorithm for a static and simple optimization problem. Therefore, PSO is introduced to optimize the algorithms of SVM in cryptocurrency forecasting. The experiment of selected cryptocurrencies is conducted for this classifier. The experimental result demonstrates that an optimized SVM-PSO algorithm can effectively forecast the future price of cryptocurrency thus outperforms the single SVM algorithms.
From a visual standpoint it is often easy to point out whether a system is considered to be self-organizing or not, though a quantitative approach would be more helpful. Information theory, as introduced by Shannon, provides the right tools not only quantify self-organization, but also to investigate it in relation to the information processing performed by individual agents within a collective. This thesis sets out to introduce methods to quantify spatial self-organization in collective systems in the continuous domain as a means to investigate morphogenetic processes. In biology, morphogenesis denotes the development of shapes and form, for example embryos, organs or limbs. Here, I will introduce methods to quantitatively investigate shape formation in stochastic particle systems. In living organisms, self-organization, like the development of an embryo, is a guided process, predetermined by the genetic code, but executed in an autonomous decentralized fashion. Information is processed by the individual agents (e.g. cells) engaged in this process. Hence, information theory can be deployed to study such processes and connect self-organization and information processing. The existing concepts of observer based self-organization and relevant information will be used to devise a framework for the investigation of guided spatial self-organization. Furthermore, local information transfer plays an important role for processes of self-organization. In this context, the concept of synergy has been getting a lot attention lately. Synergy is a formalization of the idea that for some systems the whole is more than the sum of its parts and it is assumed that it plays an important role in self-organization, learning and decision making processes. In this thesis, a novel measure of synergy will be introduced, that addresses some of the theoretical problems that earlier approaches posed.
Information theory is a powerful tool to express principles to drive autonomous systems because it is domain invariant and allows for an intuitive interpretation. This paper studies the use of the predictive information (PI), also called excess entropy or effective measure complexity, of the sensorimotor process as a driving force to generate behavior. We study nonlinear and nonstationary systems and introduce the time-local predicting information (TiPI) which allows us to derive exact results together with explicit update rules for the parameters of the controller in the dynamical systems framework. In this way the information principle, formulated at the level of behavior, is translated to the dynamics of the synapses. We underpin our results with a number of case studies with high-dimensional robotic systems. We show the spontaneous cooperativity in a complex physical system with decentralized control. Moreover, a jointly controlled humanoid robot develops a high behavioral variety depending on its physics and the environment it is dynamically embedded into. The behavior can be decomposed into a succession of low-dimensional modes that increasingly explore the behavior space. This is a promising way to avoid the curse of dimensionality which hinders learning systems to scale well.