Leveraging Large Language Models and Data Augmentation in Cognitive Computing to Enhance Stock Price Predictions ^†

Abstract

Precise stock price forecasting is essential for informed decision-making in financial markets. This study examines the combination of large language models (LLMs) with data augmentation approaches, utilizing improvements in cognitive computing to enhance stock price prediction. Traditional methods rely on structured data and basic time-series analysis. However, new research shows that deep learning and transformer-based architectures can effectively process unstructured financial data, such as news articles and social media sentiment. This study employs models, such as RNN, mBERT, RoBERTa, and GPT-4 based architectures, to illustrate the efficacy of our suggested method in forecasting stock movements. The research employs data augmentation techniques, including synthetic data creation using Generative Pre-trained Transformers, to rectify imbalances in training datasets. We assess metrics like accuracy, F1-score, recall, and precision to verify the models’ performance. We also investigate the influence of preprocessing methods like text normalization and feature engineering. Extensive tests show that transformer models are much better at predicting how stock prices will move than traditional methods. For example, the GPT-4 based model got an F1 score of 0.92 and an accuracy of 0.919, which shows that LLMs have a lot of potential in financial applications.

1. Introduction

Due to its importance to investors, policymakers, and economists, stock price forecasting has become a major financial study topic. Predicting market behavior is difficult and time-consuming, despite its rising importance. Financial markets’ inherent volatility and stochasticity make forecasting models unreliable [1]. Machine learning has been used to address these issues, but stock market volatility still makes predictions inaccurate [2]. Market predictability has been shaped by traditional financial theories like the EMH and Random Walk Theory. According to the EMH, stock prices instantly reflect all publicly available information, making past data useless for prediction [3]. The Random Walk Theory states that price variations are unforeseeable based on historical patterns because they follow a random route. These theories stress that exogenous variables like financial news and macroeconomic developments typically impact stock price fluctuations more than historical trends, especially during market turmoil. Under some situations, past stock data patterns can produce useful projections, according to empirical study [4]. When applied to well-structured information, time-series models like the Autoregressive Integrated Moving Average (ARIMA) can improve short-term forecasting. ARIMA-based models have shown significant prediction accuracy in case studies of the Indian stock market [5]. When data is tainted with noise or unstructured material, such as financial news or social media sentiments, their performance drops considerably [6]. Big data and artificial intelligence have revived interest in text mining and deep learning for stock market prediction. Deep neural networks excel in extracting complicated characteristics from various data sources, improving forecasting accuracy. Although deep learning architectures have improved [7], many cannot adapt to the complex and turbulent financial markets. Research continues to adapt model designs to rapid market swings and exogenous shocks. Behavioral finance highlights the significant impact of investors’ emotions and psychological patterns on financial decision-making [8]. Deng et al. [9] conducted a recent investigation into the potential of utilizing sentiment from retail, institutional, and foreign investors to predict changes in the Shanghai Stock Exchange Index. The growing influence of social media on investor sentiment has rendered such data a valuable component for predictive stock market models [10,11]. The proliferation of online platforms has established textual content as a significant medium for expressing investor emotions, which often influence market dynamics [12]. Extracting meaningful insights from vast and unstructured textual data continues to pose significant challenges. In response, analytical techniques, including sentiment analysis (SA) [4], natural language processing (NLP) [13], and opinion mining, have become increasingly important for interpreting investor sentiment and improving forecasting accuracy [14]. Even though a lot of research has been done in this area, many existing models still have big prediction mistakes and are becoming more complicated, which has led researchers to ask for better, clearer, and more useful solutions for stakeholders. This study uses cognitive computing methods like large language models (LLMs) [15,16] and data augmentation [17,18] to improve stock price prediction model accuracy and interpretability. Traditional modeling methods struggle to capture subtle sentiment and contextual clues in financial text data from news, analyst reports, and social media due to its volume and complexity. GPT-4 and BERT-based LLMs can derive predictive insights from unstructured financial narratives because of their enhanced language understanding. Data augmentation strategies synthetically extend training datasets, reduce class imbalance, and improve model generalization, especially in volatile or low-signal situations. This dual strategy addresses sentiment-driven forecasting restrictions in a cost-effective, scalable, and efficient manner. Paper Structure: The Section 2 examines cognitive computing, LLM, and basic financial prediction research. The Section 3 describes the data sources (stock market records, financial news, and social media), LLM architectures, and augmentation methods. Empirical observations comparing our model to traditional baselines are presented in the Section 4. In the Section 5, the study’s main findings and future research areas, such as multimodal data and real-time learning frameworks, are discussed.

2. Related Works

Zhao et al. [19] established SA-DLSTM, a hybrid framework, in 2023 to improve stock market forecasting and simulation trading. This design utilizes LSTM, DAE, and emotion-enhanced convolutional neural networks. Initial stock market data were supplemented with online user-generated content and sentiment characteristics retrieved using ECNN. DAE was used to gather market data’s key properties. The methodology improved sentiment indices by connecting temporal sentiment variations with market dynamics. These and key market indicators were fed into the LSTM for prediction. SA-DLSTM outperformed traditional models in predicting accuracy, return on investment, and risk reduction, which improved investor decision-making. Mu et al. [10] developed the MS-SSA-LSTM model, which combines multi-source basic data, deep learning, sentiment analysis, and swarm intelligence, the same year. They calculated sentiment indices using a modified sentiment lexicon from East Money forum postings. The Sparrow Search Algorithm (SSA) was then used to optimize LSTM hyperparameters [1]. It then used the improved LSTM to predict stock prices using sentiment and trading fundamentals. MS-SSA-LSTM has shown higher performance and generalizability, with a median R² increase of 10.74% above typical LSTM models. Swathi et al. used Twitter-based sentiment analysis with a TLBO-enhanced LSTM model to estimate stock prices in 2022. Tweets are short and unstructured; therefore, data preparation was necessary to reduce noise and prepare LSTM inputs. After classifying sentences as positive or negative, TLBO optimized the LSTM output layer. Twitter data validations showed that the TLBO-LSTM model outperformed traditional models in several performance parameters. Zhao et al. [20] created a machine learning approach in 2023 that combined traditional financial indicators with social media sentiment, commodity prices (oil and gold), and financial news to increase stock-forecasting accuracy. Gradient Boosting Classifier (GBM), using sentiment and oil price data, achieved an 87.2% prediction accuracy. Wang et al. [10] built a hybrid market prediction model using technical indicators and social sentiment data. The study used several algorithms, such as MLP, NB, DT, LR, RF, XGBoost, LSTM, and CNN, reaching an accuracy of 73.41% and an F1-score of 84.19%. These studies indicated that public emotion improves prediction. Lastly, Huang et al. [4] developed a hybrid genetic algorithm (HGA) with gray relational analysis (GRA) to predict stock markets. Social media sentiment analysis was utilized to discover chip-based trade signals, which were combined with LSTM outputs to improve forecasting.

3. Materials and Methods

The objective of this section is to present a comprehensive overview of the research methodology employed to integrate large language models (LLMs) [21] and data augmentation techniques [22] within a cognitive computing framework, with the specific aim of improving the accuracy and reliability of stock price prediction. This approach demonstrates strong adaptability across diverse economic contexts and financial environments beyond a single cultural or regional focus. The method is broken down into clear stages, with each stage containing a set of connected steps that help combine LLMs with improved datasets. These phases are visually depicted in Figure 1, which provides a schematic representation of the entire research workflow. The following sections will explore each phase in more detail, explaining how data acquisition, preprocessing, sentiment analysis, and performance evaluation fit into this combined predictive system.

3.1. Word Embedding

BERT [23] is a prevalent deep learning model engineered for natural language processing jobs, utilizing its capacity to comprehend linguistic context proficiently. BERT improves robots’ understanding of intricate language patterns by gathering contextual clues from surrounding words. The architecture facilitates pre-training on large textual corpora, subsequently allowing fine-tuning for downstream applications like sentiment analysis. BERT, constructed on the transformer structure, employs an attention mechanism to allocate variable weights to input-output relationships. The last concealed layer of the transformer encodes the semantic attributes of an input sentence S into a word vector matrix SD, offering a comprehensive representation for further processing.

$$SD=[A_0,A_1,A_2,A_3,\dots ,A_n,A_{n+1}]$$

A submatrix contains the specific terms of interest, notably those about $D_r$.

$$D_r=[A_1,A_2,A_3,A_4,\dots ,A_{1+m-1}]$$

$D_r\in {\mathbb{R}}^{m\times l}$, where m represents the required length. Max-pooling uses target vectors to select the most important qualities from all target words for each dimension.

$$V=max\{D_r,L=0\},V\in {\mathbb{R}}^{l\times L}$$

3.2. Cognitive Computing Models

3.2.1. RNN

RNNs [3] are artificial neural networks that store and use information from earlier inputs due to their cyclical nature. RNNs keep memory through feedback connections, unlike feedforward networks. RNNs are ideal for sequential data types, including time series, text, and audio signals, because of their architecture. NLP uses RNNs for language modeling, machine translation, and sentiment categorization. Beyond NLP, they are used in speech recognition and time-dependent predictive modeling. RNNs remain useful for sequence-based applications despite the rise of contemporary deep learning architectures like Transformers, as shown in Figure 2.

3.2.2. Multilingual BERT (mBERT)

The mBERT system is a multilingual modification of the original BERT transformer architecture. It was trained on a heterogeneous corpus that included 104 different languages. In accordance with the BERT basic configuration, mBERT is made up of twelve transformer layers, each of which has seven hundred and sixty-eight hidden units and twelve self-attention heads. Additionally, it facilitates input sequences that are as long as 512 tokens [23].

3.2.3. RoBERTa

Several hybrid deep learning architectures use the Robustly Optimized BERT Pretraining Approach (RoBERTa) [25], an upgraded BERT model based on Transformer architecture. Deep learning advances like the Transformer model use self-attention techniques to determine the relative importance of input sequence pieces. Transformers provide concurrent processing and dynamic contextual knowledge across all sequence points, unlike RNNs. Self-attention layers and feed-forward neural networks create context-aware input representations in the encoder. The decoder integrates self-attention, cross-attention between encoder and decoder outputs, and feed-forward layers. Addressing predictive modeling’s difficulties requires certain architectural aspects. The encoder component of RoBERTa, which generates contextual embeddings from input data, is the subject of this study.

3.2.4. GPT-4

The state-of-the-art big language model GPT-4 (Generative Pre-trained Transformer 4) [7] by OpenAI predicts the next word in a sequence based on contextual awareness to create human-like prose. It uses the Transformer architecture, a neural network framework adept at managing sequential input using attention processes, to improve on its predecessors. GPT-4 captures sophisticated language, semantics, and syntax patterns by training on a vast corpus of text data from several sources. Unsupervised learning with masked language modeling and task fine-tuning improves translation, summarization, question-answering, and creative writing performance. GPT-4 has better capacity, scalability, and context-aware answers. Multiple layers of decoder blocks that use masked multi-head self-attention, layer normalization, and residual connections improve its ability to understand and represent information. GPT-4 can better simulate long-range text relationships and recognize sophisticated prompts with these architectural enhancements. GPT-4 has set new standards in natural language processing and creation, making it essential to sophisticated AI systems and applications.

4. Results and Discussion

4.1. Dataset

In this study, we used The Stock Tweets for Sentiment Analysis and Prediction dataset on Kaggle [26], which provides a detailed analysis of social media sentiment and stock market movements. As shown in Figure 3, Figure 4, Figure 5 and Figure 6, it includes almost 80,000 tweets on Yahoo Finance’s top 25 stock tickers from 30 September 2021, to 30 September 2022.

4.2. Pre-Processing Using NLP

We applied several preprocessing approaches [27] in the first stage of text data preparation to enhance its quality and relevance. The process started with tokenization, which involves dividing the text into individual units or tokens, enabling more detailed analysis. Then, to lower noise and concentrate on more useful words, stop words—usually occurring words that usually lack significant meaning—were deleted. Stemming and lemmatization were used to simplify the text data by reducing words to their basic forms, which helps combine different forms of a word into one version. Finally, word-embedding techniques changed the refined tokens into numerical vectors, making it possible to use them in machine learning models for predicting outcomes. In the end, word-embedding methods transformed the refined tokens into numerical vectors, enabling their integration into predictive analysis machine learning models. The Natural Language Processing (NLP) components of this system aimed to methodically analyze and reformat text material obtained from social media sites. This reorganization helps to highlight relevant characteristics for the sentiment analysis, which are subsequently merged with numerical stock data to improve the forecast of stock price changes. Given this background, we deemed only English-language tweets consistent and free of the complexity associated with multilingual data processing. Carefully conducted, the preparation steps—tokenization, stop-word removal, and lemmatization—made sure that the text data was well prepared for the next analysis, as illustrated in the Figure 7 and Figure 8.

4.3. Results Analysis

The outcomes of

Table 1. Model’s performance with data augmentation.

Model	Accuracy	F1-Score	Recall	Precision	Preprocessing	Data Augmentation
RNN	0.823	0.810	0.792	0.836	Yes	Yes (GPT-generated)
mBERT	0.877	0.869	0.855	0.884	Yes	Yes (GPT-generated)
RoBERTa	0.891	0.882	0.871	0.893	Yes	Yes (GPT-generated)
GPT-4	0.919	0.920	0.911	0.928	Yes	Yes (GPT-generated)

and

Table 2. Model’s performance without data augmentation.

Model	Accuracy	F1-Score	Recall	Precision	Preprocessing	Data Augmentation
RNN	0.808	0.795	0.777	0.820	Yes	No
mBERT	0.861	0.853	0.839	0.868	Yes	No
RoBERTa	0.874	0.866	0.855	0.876	Yes	No
GPT-4	0.902	0.903	0.894	0.911	Yes	No

show obvious patterns in the performance of several models employed for stock price prediction. Examining the models with data augmentation (Table 1) reveals a consistent increase in performance measures across all architectures, with the GPT-4-based model clearly beating the rest. With an accuracy of 0.919 and an F1-score of 0.92, it shows not just a satisfactory general correctness but also a solid balance between precision and recall. While mBERT and RNN exhibit reasonable but considerably lesser performance, RoBERTa likewise does well, with an F1-score of 0.882. These findings highlight how well big transformer models handle unstructured financial data.

Comparing the models without data augmentation reveals a clear decline in all measures (Table 2). Though it exhibits a little decline relative to its performance with augmentation, GPT-4 still leads with an F1-score of 0.903 and accuracy of 0.902. Following RoBERTa and mBERT, RNN once more comes in last with the lowest scores. This trend underlines the need for data augmentation in improving model generalization and durability. The continued success of transformer-based models, even without extra data, shows that they can handle complex language patterns in financial text better than traditional RNNs.

GPT-generated synthetic data appears to have made a significant contribution. With the most improvements shown in more complicated models, data augmentation has increased the performance of each model. Augmented data caused the F1-score of the GPT-4-based model to rise from 0.903 to 0.92 and its accuracy to rise from 0.911 to 0.928. GPT-generated data seems to have helped offset training data constraints by providing more varied and informative instances, which probably improved decision bounds and reduced overfitting. Likewise, mBERT and RoBERTa models showed observable increases in recall and accuracy, suggesting better sensitivity and dependability in forecasts.

All in all, these findings validate the research premise that combining LLMs with sophisticated data augmentation methods significantly improves the accuracy of stock-price-forecasting models. The outstanding performance of GPT-4, even without augmentation, indicates that modern language models are already well suited for challenging tasks involving unstructured data. Augmented data’s further enhancement, thus, highlights the synergistic advantage of enhancing training sets by means of synthetic data synthesis. These results indicate a smarter and more adaptable future for market analysis by allowing for greater use of LLMs and enhancement methods in financial forecasting processes.

The comparison between Table 1 and Table 2 emphasizes a notable drop in performance measures when data augmentation is not used, hence supporting the idea that augmented data is vital for model training and generalization. The decline in performance of the GPT-4 model from an F1-score of 0.92 to 0.903 suggests that although transformer models are naturally strong, the inclusion of synthetic data is essential in optimizing their potential. This finding is especially relevant in financial forecasting as the availability of several meaningful training examples may significantly lower the overfitting risk and enhance model resilience.

The gains seen in mBERT and RoBERTa, together with GPT-4, support even more the theory that data augmentation increases model sensitivity and dependability. The significant rises in recall and accuracy imply that enhanced data not only offers additional examples for training but also enriches the dataset with different situations, hence allowing the models to provide more informed forecasts. Particularly in specialized sectors like banking, this result corresponds with earlier research supporting the use of synthetic data to offset constraints connected with tiny or unbalanced datasets.

These results have far-reaching consequences beyond the obvious ones; they point to a changing possibility for the combination of data augmentation methods and large language models (LLMs) in financial forecasting. The performance of the GPT-4 model, even without augmentation, suggests that modern language models are well suited to handle difficult tasks, including unstructured data. The extra performance increases obtained by means of data augmentation, therefore, draw attention to the synergistic advantages of integrating sophisticated modeling approaches with creative training methodologies.

Future studies should investigate how scalable these techniques are across other financial sectors and datasets. Examining the effects of other augmentation techniques beyond GPT-generated data might provide further information on how to maximize the model performance. Building confidence and knowledge in the use of these models will also depend on their interpretability in relation to stock price projection. Integrating LLMs and data augmentation will probably be more important as financial markets change, as it will improve the predicted accuracy and help to fit dynamic market behavior.

All things considered, the results of this study support the hypothesis that using LLMs with advanced data augmentation techniques may greatly improve the accuracy of stock-price-forecasting models. The exceptional performance of the GPT-4 model, even without augmentation, emphasizes the preparedness of current language models for difficult tasks, including unstructured financial data. The shown gains in market research and financial forecasting techniques using enhanced data not only emphasize the need of synthetic data synthesis but also point a hopeful path for future developments. Developing increasingly smart and flexible systems able to negotiate the complexity of financial markets will depend on the ongoing investigation of these methods as we go ahead.

5. Conclusions

Large language models (LLMs) and synthetic data production were highlighted in this study on how data augmentation affects stock price prediction machine learning models. LLMs, notably GPT-4, outperform conventional models in unstructured financial data interpretation. With data augmentation, the GPT-4-based model performed best, with an accuracy of 0.919 and an F1-score of 0.92, demonstrating its ability to handle financial texts’ sophisticated vocabulary. Even RoBERTa and mBERT improved with GPT-generated data, whereas regular RNNs did not. The performance difference with and without data augmentation shows how synthetic data improves generalization and reduces overfitting.

LLMs are strong and adaptable stock-forecasting tools, especially when combined with modern data augmentation techniques, suggesting a paradigm change in financial modeling. The improvements in accuracy, recall, and dependability show that GPT-generated data can solve financial dataset constraints. This research verifies the usefulness of these methodologies in financial forecasting and suggests applications in risk assessment, sentiment analysis, and predictive analytics. Integrating contemporary machine learning algorithms can help create more accurate, data-driven decision-making systems as financial markets become more complicated. The study also allows for further research on integrating ensemble models, evaluating various augmentation procedures, and extending these findings to additional unstructured data domains.