S&P500 Index Direction Prediction Using Textual Tweets and Their Corresponding Sentiment

Mohammadalizadeh, Parman; Jafari, Mohammadjavad

doi:10.22034/jaisis.2021.290661.1029

S&P500 Index Direction Prediction Using Textual Tweets and Their Corresponding Sentiment

Document Type : Original Article

Authors

Parman Mohammadalizadeh ¹

Mohammadjavad Jafari ²

¹ Artificial Intelligence and Robotics

² Science and Research Branch, Islamic Azad University, Tehran, Iran

10.22034/jaisis.2021.290661.1029

Abstract

In this paper, a novel method is proposed to predict the direction of Standard & Poor 500 (S&P500) index using the tweets in this regard as well as the index amount from the day before. At the beginning, using a dataset of all tweets and their corresponding posting times about S&P500 index, companies and securities are considered as features of the study. Next, these feature vectors are assigned three different labels based on the direction of the index change from the day before and whether the change is significant enough, creating a classification problem. Building a sentiment analysis tool based on T5 transformer which attempts to combine all the downstream tasks into a text-to-text format, sentiment feature is added to each tweet in the dataset. Lastly, after balancing the data and preprocessing the textual information through an NLP pipeline, a deep neural network is proposed to classify the processed data. It is shown that using the tweets and their corresponding sentiments, the proposed method for movement direction prediction of the S&P500 index outperformed other existing models.

Keywords

Natural Language Processing

Predictive Modeling

Sentiment Analysis

Deep Neural Networks

Transformer Learning