Transformer Models for Bitcoin Price Prediction

Abstract

Time Series forecasting has been approached by a multiplicity of techniques including deep learning methods of various degrees of sophistication, showcasing notable advancements and improved performance over the past few years. More recently, there has been a sustained interest in the study of Transformers, a class of models renowned for their remarkable capacity to capture intricate long-range dependencies and interactions. This ability is perceived as particularly relevant and impactful in the context of time series modeling, reflecting a growing recognition of their potential in enhancing forecasting accuracy and understanding of complex temporal patterns. However, taking advantage of this principle to deploy successful forecasting methods is not yet clearly understood, and requires significant experimentation or engineering. Therefore, in this paper, we compare multiple variations of the Transformer model (standard Transformer, Autoformer, Informer), coupled with diverse combinations of embedding data. In particular, as the emphasis of our work is on forecasting, we investigate the relationship between Transformers’ input segment length and prediction performance in a multi-step time intervals framework. Our results suggest that the Autoformer outperforms both standard Transformer and Informer across various prediction steps. We also observe that shorter input lengths and shorter prediction lengths generally produce better model performance.