In a world of increasing closed-source commercial machine learning models, model evaluations from developers must be taken at face value. These benchmark results-whether over task accuracy, bias evaluations, or safety checks-are traditionally impossible to verify by a model end-user without the costly or impossible process of re-performing the benchmark on black-box model outputs. This work presents a method of verifiable model evaluation using model inference through zkSNARKs. The resulting zero-knowledge computational proofs of model outputs over datasets can be packaged into verifiable evaluation attestations showing that models with fixed private weights achieve stated performance or fairness metrics over public inputs. We present a flexible proving system that enables verifiable attestations to be performed on any standard neural network model with varying compute requirements. For the first time, we demonstrate this across a sample of real-world models and highlight key challenges and design solutions. This presents a new transparency paradigm in the verifiable evaluation of private models.
Over recent decades, machine learning has significantly advanced network communication, enabling improved decision-making, user behavior analysis, and fault detection. Simultaneously, the growth of communication networks has facilitated the efficient collection of large-scale training data. Traditional centralized machine learning, however, requires collecting data from users, raising significant concerns about privacy and security. Decentralized approaches, where participants exchange computation results instead of raw private data, mitigate these risks but introduce challenges related to trust and verifiability. A critical issue arises: How can one ensure the integrity and validity of computation results shared by other participants? Existing survey articles predominantly address security and privacy concerns in decentralized machine learning, whereas this survey uniquely highlights the emerging issue of verifiability. Recognizing the critical role of zero-knowledge proofs in ensuring verifiability, we present a comprehensive review of Zero-Knowledge Proof-based Verifiable Machine Learning (ZKP-VML). To clarify the research problem, we present a definition of ZKP-VML consisting of four algorithms and several key security properties. In addition, we provide an overview of the current research landscape by systematically organizing the research timeline and categorizing existing schemes based on their security properties. Furthermore, through an in-depth analysis of each existing scheme, we summarize their technical contributions and optimization strategies, aiming to uncover common design principles underlying ZKP-VML schemes. Building on the reviews and analysis presented, we identify current research challenges and suggest future research directions. To the best of our knowledge, this is the most comprehensive survey to date on verifiable decentralized machine learning and ZKP-VML.
Haochen Sun, Tonghe Bai, J. Li, Change Institutions to: University of Waterloo
The recent advancements in deep learning have brought about significant changes in various aspects of people’s lives. Meanwhile, these rapid developments have raised concerns about the legitimacy of the training process of deep neural networks. To protect the intellectual properties of AI developers, directly examining the training process by accessing the model parameters and training data is often prohibited for verifiers. In response to this challenge, we present zero-knowledge deep learning (zkDL), an efficient zero-knowledge proof for deep learning training. To address the long-standing challenge of verifiable computations of non-linearities in deep learning training, we introduce zkReLU, a specialized proof for the ReLU activation and its backpropagation. zkReLU turns the disadvantage of non-arithmetic relations into an advantage, leading to the creation of FAC4DNN, our specialized arithmetic circuit design for modelling neural networks. This design aggregates the proofs over different layers and training steps, without being constrained by their sequential order in the training process. With our new CUDA implementation that achieves full compatibility with the tensor structures and the aggregated proof design, zkDL enables the generation of complete and sound proofs in less than a second per batch update for an 8-layer neural network with 10M parameters and a batch size of 64, while provably ensuring the privacy of data and model parameters. To our best knowledge, we are not aware of any existing work on zero-knowledge proof of deep learning training that is scalable to million-size networks.
Bitcoin is a kind of Cryptocurrency and now is one of type of investment on the stock market.Stock markets are influenced by many risks of factor.And bitcoin is one kind of cryptocurrency that keep rising in recent few years, and sometimes sudden fall without knowing influence behind it on the stock market.Because it's fluctuations, there's a need and automation tool to predict bitcoin on the stock market.This research study learns how to create model prediction bitcoin stock market prediction using LSTM, LSTM (Long Short Term Memory) is another type of module provided for RNN later developed and popularized by many researchers, like RNN, the LSTM also consists of modules with recurrent consistency.The contribution of this study is providing a new forecasting framework for bitcoin price prediction can overcome and improve the problem of input variables selection in LSTM without strict assumptions of data assumption.The results revealed its possible applicability in various cryptocurrencies prediction, industry instances such as medical data or financial timeseries data.The Method that we apply on this research, also technique and tools to predict Bitcoin on stock market yahoo finance can predict the result above $ 12600 USD for next days after prediction, in the last section we make conclusions and discuss future works.The proposed methodology is then applied to train a simple Long Short Term Memory (LSTM) model to predict the bitcoin price for the upcoming 5 days.When the LSTM model is trained with a suitable data chunk, thus identified, sustainable results are found for the prediction.In the end of this paper, the work culminates with future improvements.
The long short-term memory (LSTM) network and a cutting-edge method that combines wavelet decomposition and LSTM (W-LSTM) were applied to deep learning in this study's analysis of Bitcoin's price and movement. To be specific, it predicted next day’s both price and price movement (trend) with historical data. The input of the model is close price itself, basic trading information, and technical indicators calculated solely on basic trading information. Large number of numerical experiments come to the same conclusion that: for price prediction, only close price as input obtains the best performance for regression, and minor improvement achieved after 1-order wavelet decomposition; for price movement, no improvement after changing the number of input features or with the model W-LSTM has been spotted for the same network structure and hyper-parameters, and enlarging time step and batch size will improve accuracy and Matthews correlation coefficient despite of number of input and model used in this paper.
Anurag Dutta, Liton Chandra Voumik, A. Ramamoorthy, Samrat Ray · 5 authors
Cryptocurrencies are in high demand now due to their volatile and untraceable nature. Bitcoin, Ethereum, and Dogecoin are just a few examples. This research seeks to identify deception and probable fraud in Ethereum transactional processes. We have developed this capability via ChaosNet, an Artificial Neural Network constructed using Generalized Luröth Series maps. Chaos has been objectively discovered in the brain at many spatiotemporal scales. Several synthetic neuronal simulations, including the Hindmarsh–Rose model, possess chaos, and individual brain neurons are known to display chaotic bursting phenomena. Although chaos is included in several Artificial Neural Networks (ANNs), for instance, in Recursively Generating Neural Networks, no ANNs exist for classical tasks entirely made up of chaoticity. ChaosNet uses the chaotic GLS neurons’ property of topological transitivity to perform classification problems on pools of data with cutting-edge performance, lowering the necessary training sample count. This synthetic neural network can perform categorization tasks by gathering a definite amount of training data. ChaosNet utilizes some of the best traits of networks composed of biological neurons, which derive from the strong chaotic activity of individual neurons, to solve complex classification tasks on par with or better than standard Artificial Neural Networks. It has been shown to require much fewer training samples. This ability of ChaosNet has been well exploited for the objective of our research. Further, in this article, ChaosNet has been integrated with several well-known ML algorithms to cater to the purposes of this study. The results obtained are better than the generic results.
Bitcoin, the largest cryptocurrency, is extremely volatile and hence needs a better model for its pricing. In the literature, many researchers have studied the effect of data normalization on regression analysis for stock price prediction. How has data normalization affected Bitcoin price prediction? To answer this question, this study analyzed the prediction accuracy of a Legendre polynomial-based neural network optimized by the mutated climb monkey algorithm using nine existing data normalization techniques. A new dual normalization technique was proposed to improve the efficiency of this model. The 10 normalization techniques were evaluated using 15 error metrics using a multi-criteria decision-making (MCDM) approach called technique for order performance by similarity to ideal solution (TOPSIS). The effect of the top three normalization techniques along with the min–max normalization was further studied for Chebyshev, Laguerre, and trigonometric polynomial-based neural networks in three different datasets. The prediction accuracy of the 16 models (each of the four polynomial-based neural networks with four different normalization techniques) was calculated using 15 error metrics. A 16 × 15 TOPSIS analysis was conducted to rank the models. The convergence plot and the ranking of the models indicated that data normalization plays a significant role in the prediction capability of a Bitcoin price predictor. This paper can significantly contribute to the research with a new normalization technique for utilization in varied fields of research. It can also contribute to international finance as a decision-making tool for different investors as well as stakeholders for Bitcoin pricing.
Our work presents two fundamental contributions. On the application side, we tackle the challenging problem of predicting day-ahead crypto-currency prices. On the methodological side, a new dynamical modeling approach is proposed. Our approach keeps the probabilistic formulation of the state-space model, which provides uncertainty quantification on the estimates, and the function approximation ability of deep neural networks. We call the proposed approach the deep state-space model. The experiments are carried out on established cryptocurrencies (obtained from Yahoo Finance). The goal of the work has been to predict the price for the next day. Benchmarking has been done with both state-of-the-art and classical dynamical modeling techniques. Results show that the proposed approach yields the best overall results in terms of accuracy.
This paper is based on current market prediction value of cryptocurrency Bitcoin.The Bitcoins is like shareable as piece of cakes but whenever, the price of Bitcoins will be fluctuated more.It will be difficult to predict.This problem discussed is based on computer science and delivered through Bitcoin market frequence, project using machine learning high level computer concept and predict highly closed accurate Bitcoin price using various real world dataset.
Experimental data was obtained from Bitcoin public library made available at https://www.investing.com/crypto/bitcoin/historical-dat which has attributes: open price, high, low and closes as shown in Table 1.
RENATO MOTA RUIZ, INACIO HENRIQUE YANO, Alexandre de Castro, Julio Cezar Souza Vasconcelos
This work aims to evaluate the storing and retrieving data response time using an Ethereum Smart Contract application to verify the feasibility of its utilization instead of using relational databases in web application development. To achieve the objectives of this work. There was a comparison between the Ethereum Smart Contract and the SQLite, considering response time as the user experience for database choice decisions in future application development. This study consisted of the development of two similar applications. The first one was the Ethereum Smart Contract Application, and the other was the SQLite Application. Using these applications to build graphs of response time behavior as the number of records processed grows. For storing data Blockchain application was much faster than the SQLite application. When retrieving data, the Blockchain application usually starts slower but finishes faster than the SQLite application. Blockchain is a recent technology for secure data storage in a distributed architecture. The hypothesis to be checked was if it also has a good response time compared with other databases. The contribution of this work is to provide information about the efficiency and possible user satisfaction of Blockchain applications.
Open access
Neural Networks and Applications
Advanced Computational Techniques and Applications
When trying to fit a deep neural network (DNN) to a $G$-invariant target function with $G$ a group, it only makes sense to constrain the DNN to be $G$-invariant as well. However, there can be many different ways to do this, thus raising the problem of ``$G$-invariant neural architecture design'': What is the optimal $G$-invariant architecture for a given problem? Before we can consider the optimization problem itself, we must understand the search space, the architectures in it, and how they relate to one another. In this paper, we take a first step towards this goal; we prove a theorem that gives a classification of all $G$-invariant single-hidden-layer or ``shallow'' neural network ($G$-SNN) architectures with ReLU activation for any finite orthogonal group $G$, and we prove a second theorem that characterizes the inclusion maps or ``network morphisms'' between the architectures that can be leveraged during neural architecture search (NAS). The proof is based on a correspondence of every $G$-SNN to a signed permutation representation of $G$ acting on the hidden neurons; the classification is equivalently given in terms of the first cohomology classes of $G$, thus admitting a topological interpretation. The $G$-SNN architectures corresponding to nontrivial cohomology classes have, to our knowledge, never been explicitly identified in the literature previously. Using a code implementation, we enumerate the $G$-SNN architectures for some example groups $G$ and visualize their structure. Finally, we prove that architectures corresponding to inequivalent cohomology classes coincide in function space only when their weight matrices are zero, and we discuss the implications of this for NAS.
We demonstrate a novel application of online transfer learning for a digital assets trading agent. This agent uses a powerful feature space representation in the form of an echo state network, the output of which is made available to a direct, recurrent reinforcement learning agent. The agent learns to trade the XBTUSD (Bitcoin versus US Dollars) perpetual swap derivatives contract on BitMEX on an intraday basis. By learning from the multiple sources of impact on the quadratic risk-adjusted utility that it seeks to maximise, the agent avoids excessive over-trading, captures a funding profit, and can predict the market's direction. Overall, our crypto agent realises a total return of 350\%, net of transaction costs, over roughly five years, 71\% of which is down to funding profit. The annualised information ratio that it achieves is 1.46.
As a new investment method, quantitative investment is expanding its market scale and share due to its stable investment performance. In this paper we propose a prediction model based on LSTM. It is helpful for the traders to predict the future price to formulate the best trading strategy. Using this model, we can precisely forecast each price separately to determine when the asset should be traded based on future price fluctuations. Simulation results show that our model can successfully predict the future price trend of the two assets within the acceptable range of error, which helps us to better optimize our portfolio. In addition, the RMSE (root mean square error) is selected as the loss function to describe the accuracy of our prediction model.
The objective of this paper is to assess the performances of dimensionality reduction techniques to establish a link between cryptocurrencies. We have focused our analysis on the two most traded cryptocurrencies: Bitcoin and Ethereum. To perform our analysis, we took log returns and added some covariates to build our data set. We first introduced the pearson correlation coefficient in order to have a preliminary assessment of the link between Bitcoin and Ethereum. We then reduced the dimension of our data set using canonical correlation analysis and principal component analysis. After performing an analysis of the links between Bitcoin and Ethereum with both statistical techniques, we measured their performance on forecasting Ethereum returns with Bitcoin s features.
Abstract The paper proposes a new approach to constructing a decentralized hierarchical network of modular type for solving complex problems in the paradigm of training artificial neural networks (ANN) with a teacher. The main idea of the proposed approach to solving the problem is the formation of a system model in the form of a multilevel hierarchical network, consisting of typical autonomous modules built on an ANN of two configurations: а coordinator and a terminal model. The coordinator is a specially designed ANN, which forms, based on the target set of its level, the target set of the next subordinate level of the network hierarchy. The terminal model is a traditional ANN of any architecture and organization of the learning process. The proposed hierarchical structure of the network allows solving a complex initial problem asynchronously in time and in parallel in space, using distributed computing resources and geographically dispersed teams of researchers. The paper provides an example of numerical modeling that demonstrates the proposed approach for the synthesis of a multilevel hierarchical network and confirms all the declared and expected results of its use.
Advanced Data Processing Techniques
Technology and Human Factors in Education and Health
Single- or multi-layer perceptrons, commonly known as neural networks, are universal approximators that can approximate any continuous functions arbitrarily well when the number of perceptrons is allowed to grow indefinitely. When prior knowledge about the target function is available, constraints can be imposed on the neural network to improve approximation accuracy. As an example, it is well known that convolutional neural networks (CNN) \cite{lecun1990handwritten} yield good performance on image classification. And its parameter-sharing scheme can also reduce the risk of overfitting. One of the key properties of image classification is that it is invariant to translation of input image. The translation invariance in CNN is achieved with two steps. On the one hand, the convolutional layer gives a translation equivariance as it is a linear time invariant system. The translation of the input image will be kept as a transformation of the output. On the other hand, the pooling layer will introduce local invariance. The stack of convolutional layers and pooling layers will then reach a receptive field the same size as input and also enforces global invariance. Inspired by the success of CNN on image related tasks, plenty of CNN generalizations \cite{gens2014deep, henaff2015deep, simonovsky2017dynamic, cohen2018spherical,chidester2018rotation} have been studied in other tasks. \emph{Domain invariance}, which refers to the property that the output is invariant to certain transformation of the input features is a crucial property which can help to generalize CNN to other machine learning applications. The works which introduce domain invariance fall into two categories. One of the categories introduces the invariance by injecting invariant constrains. The other category builds the equivariant network layer then achieves invariance by applying the pooling or normalization layer. In the equivariant network layer, the input is transformed by some group transformation the output is transformed correspondingly. As in the CNN example, the invariance can be achieved by first using equivariant layers then apply the pooling layer. In this dissertation, we will focus on the invariance and equivariance of neural networks and analyze the neural network architecture which can achieve invariance or equivariance. The main works can be summarized as following. \noindent\textbf{The Connections Between Convolutional Architecture and Equivariant Property} Convolutional neural networks have achieved great success in speech, image, and video signal processing tasks in recent years. There have been several attempts to justify the convolutional architecture and to generalize the convolution operation for treatment of other data types such as graphs and manifolds. Based on group representation theory and noncommutative harmonic analysis, it has recently been shown that the so-called group equivariance requirement of a feed-forward neural network necessitates the convolutional architectures. In our work, based on the familiar concepts of linear time-invariant systems, we develop an elementary proof of the same result. The nonlinear activation function, being a necessary components of practical deep neural networks, has been glossed over in previous analyses of the connection between equivariance and convolution. We identify sufficient conditions for the non-linear activation functions to preserve equivariance, and hence the necessity of the group convolution structure. Our analysis method is simple and intuitive, and holds the potential to be applied to more challenging scenarios such as non-transitive domains and multiple simultaneous equivariances. \noindent\textbf{Characteristics of Generalized Convolutional Neural Networks} Based on the analysis method above, a theoretical support for details of network architecture design is then provided in our work. We first analyze the necessary and sufficient condition for a neural network to be group equivariant when the group acts on the sub-domain of input/output. This part gives a theoretical support for how should we combine channels in a neural network layer. Meanwhile, it also gives guidelines for designing neural networks with partial equivariance/invariance. To further investigate the characteristics of generalized convolutional neural networks, we then analyze the multiple equivariance case. The result shows the connection between model parameter sharing and equivariant property. In particular, we find that a model will have more parameter sharing if more equivariances are required. Meanwhile, we establish a universality result for shallow and deep generalized group convolutional neural networks as approximators of continuous group-equivariant functions. After choosing the network architecture, one common question we need to answer is that how many neurons do we want in our hidden layer? We show that the generalized convolution mapping to a quotient space is a projection of the image of a generalized convolution which maps to the maximum quotient space. This can be used to obtain guidelines for choosing the feature size of hidden layer. \noindent\textbf{Neural Networks with Invariant Properties} In the traditional convolutional neural networks, the spatial invariant properties are introduced by adding a global pooling layer after the convolutional layers. For some specific applications, there are still some potential for us to further reduce the model complexity by introduce invariant property in earlier layers. An partial permutation invariant model for learning graph node embedding is introduced based on the classical invariant theory. In the image processing application, an affine equivariant preprocessing method is proposed to build neural networks invariant to affine transformation. \noindent\textbf{Partially Permutation Invariant Graph Node Embedding Model} Graph node embedding aims at learning a vector representation for all nodes given a graph. It is a central problem in many machine learning tasks (e.g., node classification, recommendation, community detection). The key problem in graph node embedding lies in how to define the dependence to neighbors. Existing approaches specify (either explicitly or implicitly) certain dependencies on neighbors, which may lead to loss of subtle but important structural information within the graph and other dependencies among neighbors. This intrigues us to ask the question: can we design a model to give the maximal flexibility of dependencies to each node's neighborhood. In our recent work, we propose a novel graph node embedding method (named \textbf{\OM}) via a novel notion of \textit{partial permutation invariant set function}, to capture any possible dependence. The partial permutation invariant set function is designed based on the set of \textit{invariant basis} with respect to permutation group. Our method 1) can learn an \textit{arbitrary} form of the representation function from the neighborhood, without losing any potential dependence structures, and 2) is applicable to both homogeneous and heterogeneous graph embedding, the latter of which is challenged by the diversity of node types. Furthermore, we provide theoretical guarantee for the representation capability of our method for general homogeneous and heterogeneous graphs. Empirical evaluation results on benchmark data sets show that our proposed {\OM} method outperforms the state-of-the-art approaches on producing node vectors for various learning tasks of both homogeneous and heterogeneous graphs. \noindent\textbf{Learning Models Invariant to Affine Transformations for Image Processing} The way designing invariant/partial invariant neural networks with invariant basis is difficult to extend to groups other than permutation group. Because the invariant basis is expensive to calculate. Even if we get the set of invariant basis, it will be very large and is not helpful to reduce model complexity. An alternative which can introduce invariant property to neural network is adding an equivariant transformer as a pre-processing layer. This equivariant transformer will revert all the affine transformations happens in the input. Thus the model will be invariant to affine transformations. Invariance to affine transformations is desirable property for many computer vision tasks like image classification and motion tracking. Enforcing and taking advantage of such affine invariance property is an essential part for building efficient models for machine learning. However, it has been challenging to build models that are provably affine invariant. Previous works have achieved partial invariance, such as invariance to shifting and to rotation. We propose to build an affine-invariant model in two steps. First, we apply image normalization, which renders the density function represented by the image to have zero mean and identity covariance matrix. The resulting normalized image is then processed by a second stage neural network that is either an equivariant transformer network (ETN) or group convolution networks. We provide proof of invariance and demonstrate the performance improvement of such neural networks in processing images that have been affine transformed.
This study will investigate the predictability of a Chaotic time-series data using Reservoir computing (Echo State Network), Deep-Learning(LSTM) and Machine- Learning(Linear, Bayesian, ElasticNetCV , Random Forest, XGBoost Regression and a machine learning Neural Network) on the short (1-day out prediction), medium (5-day out prediction) and long-term (30-day out prediction) pricing of Bitcoin and Ethereum Using a range of machine learning tools, to perform feature selection by permutation importance to select technical indicators on the individual cryptocurrencies, to ensure the datasets are the best for predictions per cryptocurrency while reducing noise within the models. The predictability of these two chaotic time-series is then compared to evaluate the models to find the best fit model. The models are fine-tuned, with hyperparameters, design of the network within the LSTM and the reservoir size within the Echo State Network being adjusted to improve accuracy and speed. This research highlights the effect of the trends within the cryptocurrency and its effect on predictive models, these models will then be optimized with hyperparameter tuning, and be evaluated to compare the models across the two currencies. It is found that the datasets for each cryptocurrency are different, due to the different permutation importance, which does not affect the overall predictability of the models with the short and medium-term predictions having the same models being the top performers. This research confirms that the chaotic data although can have positive results for shortand medium-term prediction, for long-term prediction, technical analysis basedprediction is not sufficient.