Analysing Online Reviews Consumers’ Experiences of Mobile Travel Applications with Sentiment Analysis and Topic Modelling: The Example of Booking and Expedia

Abstract

This study aims to analyse consumer experiences, purchase behaviours, and emotional responses through Booking and Expedia’s mobile applications. The 2000 user reviews collected from Google Play were subjected to a comprehensive sentiment analysis, text mining, and topic modelling process to identify the key elements that shape consumers’ emotional experiences and purchase decisions. According to the results of text mining and sentiment analysis performed with Python’s WordNet library, 81.9% of Booking.com reviews are positive, 8.4% are negative, and 11.3% are neutral, whereas 55.8% of Expedia reviews are positive, 37.8% are negative, and 8.0% are neutral. In the topic modelling analysis, Booking.com emphasised ease of booking, while Expedia emphasised difficulties in cancellation and refund processes. These findings provide valuable insights into how consumers’ emotional states and purchasing behaviours are reflected in their experiences with mobile applications. The study enables the development of strategic recommendations for marketing management to better analyse consumers’ expectations and experiences.

1. Introduction

In recent years, the rapid advancement of technology has transformed how consumers interact with brands and make purchase decisions. The proliferation of mobile applications, particularly in the travel and tourism sectors, has provided users with unprecedented convenience and accessibility. As consumers increasingly rely on mobile applications for travel planning and booking, understanding their experiences, emotional responses, and purchase behaviours has become critical for businesses striving to enhance customer satisfaction and loyalty. This study focuses on two prominent online travel agencies, Booking.com and Expedia, examining user reviews from their mobile applications to uncover insights into consumer sentiment and behaviour.

With the development of digital technologies, the use of smartphones and software applications for mobile devices has increased [1,2]. Mobile applications, known as apps, are software systems that run on handheld devices such as smartphones and tablet PCs [3]. According to Chang [4], mobile applications are programmes or software that carry out duties or operations for the user and can be used on a variety of mobile devices, such as cell phones, electronic gadgets, and tablets. Mobile applications that make mobile devices useful are downloaded and used by billions of people all over the world. There are currently more than three million applications in the mobile application market, and these applications are downloaded billions of times per year from application stores, especially the Google Play Store [5] and Apple App Store.

Consumer experiences with mobile applications encompass a wide range of emotions, from satisfaction and excitement to frustration and disappointment. These emotions significantly influence purchasing behaviour, as positive experiences often lead to brand loyalty, while negative experiences can result in customer churn. Therefore, it is essential for businesses to understand the emotional state of their consumers and how these feelings are expressed in user reviews. Recent developments in data analytics have opened new avenues for analysing large volumes of unstructured data, such as customer reviews. Techniques like sentiment analysis, text mining, and topic modelling enable researchers to extract meaningful insights from these datasets, thereby shedding light on consumer sentiments and their implications for brand performance.

The tourism industry, which is based on consumer experience, is also one of the areas where mobile applications are widely used. The widespread use of mobile travel applications has brought about significant changes in the travel and tourism industry [6,7]. Mobile applications developed for the travel industry have changed the way tourists travel and transformed the way companies reach their customers [8]. Mobile apps have become an integral part of the customer experience, not only for providing information about destinations and attractions but also for various roles in travel, such as travel agents, translators, entertainment devices, checking in for airline flights, etc. [9].

User comments are accessed by mobile applications and play an important role in accessing information [10]. Mobile app user reviews contain rich information about user experiences and expectations [11]. Because customers are willing to share their comments online about the goods and services they purchase, the level of service, handling of complaints, etc. [12]. Online customer reviews influence mobile app usage and brand reputation [13,14]. The fact that businesses are losing more and more customers day by day due to negative comments and negative news on social media [15] is an important development that reveals the necessity of reputation management in online environments. With online reputation management, negative comments can be managed, brand reputation can be increased, and trust can be established with customers [16,17]. Considering that the number of online users is increasing daily [16], being able to evaluate these comments correctly is key to understanding how to manage online reputation [18]. Therefore, mobile app providers must analyse these reviews accurately to increase customer satisfaction and ensure a sustainable online reputation [19]. To help mobile app providers understand the sentiments of their app users, an analytical method that can summarise positive or negative comments is needed. This method of sentiment analysis classifies emotions [20] and examines large amounts of data to determine people’s attitudes, opinions, judgements and feelings about an issue [21].

Understanding the relationship between consumer emotions and purchasing behaviour is particularly relevant in today’s competitive landscape, where customer loyalty is paramount for sustained business success. Positive emotional experiences foster brand advocacy, prompting users to recommend services to others, while negative experiences can damage a brand’s reputation and deter potential customers. Thus, this study not only contributes to the existing literature on consumer behaviour and mobile applications but also provides actionable insights for marketing strategies in the travel sector.

By delving into the emotional experiences and purchasing behaviours of users, this research aims to equip marketers and business strategists with the tools necessary to enhance customer satisfaction and loyalty. These findings underscore the importance of integrating consumer feedback into strategic decision-making processes, allowing brands to proactively address customer needs and expectations. Moreover, the study highlights the potential of advanced analytical techniques in extracting valuable insights from user-generated content, paving the way for future research that can further explore the intricacies of consumer behaviour in the digital age.

This study provides a comprehensive analysis of consumer experiences, purchase behaviours, and emotional responses related to Booking and Expedia’s mobile applications. The results not only reveal key differences in user sentiment but also offer valuable recommendations for marketing management. As the landscape of mobile applications continues to evolve, understanding consumer emotions and behaviours will remain crucial for brands aiming to cultivate lasting relationships with their customers. This research lays the groundwork for future studies that can further investigate the dynamic interplay between technology, consumer behaviour, and brand performance in the travel and tourism sector.

This study is one of a limited number of studies in the literature that evaluates user reviews of mobile travel apps through sentiment analysis and topic modelling. In particular, the comparison of Booking.com and Expedia provides a new context for analysing consumer experiences. These findings enable mobile app providers to increase user satisfaction, strengthen brand reputation management, and optimise marketing strategies.

This study is one of the few to examine user experiences with mobile travel applications using sentiment analysis and topic modelling techniques. This study contributes to the literature by demonstrating how user comments can guide mobile application design strategies through sentiment analysis. The analysis of Booking.com and Expedia provides concrete recommendations for brands that want to improve their user experience.

In this study, user comments on mobile travel applications “Booking.com” and “Expedia” on Google Play Store, the world’s largest mobile application store, were analysed using sentiment analysis, text mining, and topic modelling. To the best of our knowledge, studies in this area are limited. The study offers suggestions for brand reputation management of mobile applications by analysing the comments in these applications that enable real-time discovery of customer feedback. The results of this study are important for the brand reputation management of mobile travel applications. The reputational value of brands is discussed along with the factors influencing it, and recommendations for enhancing brand reputation are offered. There are basically five sections in this article. An introduction is presented in Section 1, and an overview of some relevant works is presented in Section 2. Section 3 describes our study’s methodology in depth. Section 4 presents the results of the analysis. Finally, our research findings and discussion are presented in Section 5.

2. Literature Review

The utilitarian, user-friendly, and accessible nature of mobile devices has made them indispensable tools for fulfilling human needs in recent years [22]. One of the key drivers of the success of mobile devices is the extensive use of mobile applications. Unlike traditional software systems, mobile applications require new approaches to understand and maintain their functionality [3].

Initially, mobile applications were limited to basic productivity and information retrieval tasks, such as email, calendars, and weather. However, the increasing demand from users and the proliferation of developer tools have expanded applications into diverse domains [1]. Over time, mobile apps have become integral to everyday life, encompassing categories like social networking, gaming, finance, education, and travel [23].

Among these, mobile travel applications have become critical tools for tourists, enabling them to plan trips, compare prices for travel and accommodation, and complete bookings [10,24,25].

Applications such as Booking.com and Expedia are prominent examples that offer significant user bases and a unique user experience. Booking.com, based in the Netherlands, supports over 500 million users in 43 languages and provides access to more than 28 million accommodation listings [26]. Meanwhile, Expedia, a U.S.-based app, serves over 50 million users in 70 countries and offers a wide range of travel-related services [27].

The preference for mobile applications among users is driven by key factors such as efficiency, satisfaction, learnability, memorability, and error handling [28]. For instance, efficiency refers to the ability of users to achieve their goals with minimal resources, while satisfaction involves relieving discomfort and fostering positive attitudes. Learnability ensures that the application is easy to understand, even for first-time users, and memorability allows users to re-engage with the system without the need for relearning. These attributes collectively shape the user experience and serve as benchmarks for app providers aiming to refine their strategies [29].

Beyond functionality, consumer relationships with mobile applications are strongly influenced by perceptions of trust, reliability, and reputation. A brand’s online reputation, often defined by user sentiment, plays a critical role in evaluating its credibility [30]. This reputation is increasingly shaped by consumer reviews and comments on online platforms, which offer insights into the perceived quality of an application [31].

Recent advancements in sentiment analysis and topic modelling have proven invaluable in uncovering user perceptions and improving application quality [10]. These methods allow developers to identify patterns in user reviews, predict customer sentiments, and effectively address specific needs [32]. Liang et al. [14], for instance, proposed a multifaceted sentiment analysis approach to measure dimensions in consumer comments, though their work did not fully explore the impact of textual feedback on mobile applications.

While previous studies have examined user sentiments and app functionality, there is limited research focusing on how sentiment analysis and topic modelling can provide actionable insights for the mobile travel industry. This study fills this gap by analysing user reviews from two leading travel apps, Booking.com and Expedia, using advanced data analysis techniques. By integrating these methodologies, it not only contributes to the existing literature but also offers practical recommendations for app providers seeking to enhance user satisfaction and manage brand reputation.

Table 1. Comparison of Previous Studies.

Study	Methodology	Findings	Limitations	Contributions of This Study
Liang et al. (2017) [7]	Multifaceted sentiment analysis	Measured emotional dimensions of user reviews	Did not examine the textual impact of user comments	Analyses user comments using topic modelling for deeper insights
Masrury et al. (2019) [10]	Sentiment analysis and TF-IDF	Measured user perceptions of mobile app quality	Did not explore alternative analytical approaches	Provides comparative analysis using alternative methodologies

shows a comparison of previous studies.

This study provides important insights into mobile travel apps through the innovative use of sentiment analysis and topic modelling. This gap in the literature is filled by an in-depth analysis of user feedback from major apps, such as Booking.com and Expedia.

3. Materials and Methods

In this study, the user experiences of Booking.com and Expedia, leading online travel agencies in the tourism sector, were evaluated through their mobile applications. The dataset, obtained from Google Play, consists of a total of 2000 comments, with 1000 for each application. Sentiment analysis, text mining, and topic modelling techniques were applied to analyse consumer emotions, experiences, and purchasing behaviours. The Naive Bayes method, a machine learning technique, was employed to classify comments based on sentiment, while Python’s WordNet library was used for text mining.

We chose Naive Bayes for this study because it balances classification performance with computational efficiency. The fact that our dataset is relatively small and labelled is a factor that increases the accuracy of Naive Bayes. Alternatively, models such as SVM and Random Forest can be considered, but these models have longer training times and require more computational resources. While deep learning approaches such as LSTM are effective on large datasets, they are not suitable due to the size of our current dataset.

This study addresses the following research questions:

• How do user reviews reflect the differences in satisfaction between Booking.com and Expedia apps?
• How do sentiment analysis and topic modelling techniques contribute to understanding user behaviour?

3.1. Data Set

In this study, mobile applications of two online travel agencies in the tourism industry were selected to analyse user experiences and consumer behaviour. Booking.com was founded in 1996 and offers over 28 million accommodation properties, including over 6.6 million places to stay [26]. Expedia was founded in 1996 as an online travel agency owned by the Expedia Group. The application is used to book airline tickets, hotel reservations, car rentals, cruises, and vacation packages. Both online travel agencies can be accessed through their websites and mobile applications [27]. Mobile applications have been developed for Android and IOS operating systems and are available for customer use. The dataset in this study consists of customer comments on two applications on Google Play. Google Play is one of the largest mobile application platforms available on many devices worldwide. On this platform, which has many applications, users share their experiences with the applications by providing comments and ratings. Ser reviews on Google Play provide a data set that is comprehensive and detailed and includes different experiences [33].

The dataset consists of 2000 reviews of Booking.com and Expedia mobile applications. The amount of data provides sufficient data saturation to ensure data reliability and validity. Thus, sentiment analysis and text mining operations are meaningful and generalisable. There are also a variety of data that can reveal different emotional states.

Table 2. Google Play information of mobile applications (GooglePlay, 2024, [33]).

	Booking.com	Expedia
Number of downloads	500 milyon+	50 milyon+
Number of comments	4.07 milyon	812 bin
Score	4.6	4.6
Release date	4 Şub 2011	10 May 2011
Size	144 MB	63 MB

contains information about Booking.com and Expedia applications on Google Play.

According to the data in Table 2, both applications have been downloaded more than 50 million times, indicating that they have a large user base. The number of comments is approximately 4 million and 800 thousand, respectively, which shows that users share their thoughts and experiences regarding the application. This provides a rich data set for analysis. The score of both applications is 4.6. This shows that users are generally satisfied with the mobile application. The fact that their launch date was 2011 shows that both companies attach importance to technology and entered the mobile application platform early [34,35].

Table 3. Examples of mobile app reviews [34,35].

Mobile App	Comments
Booking.com	Simple, easy, convenient and useful.
	A perfect system makes our lives easier.
	Excellent, I can easily access all the information I need.
	I am extremely satisfied with the application, so far, all hotels have met our expectations, I recommend it.
	The option to show on maps has now been removed, I think it is very bad and should be brought as soon as possible.
Expedia	I always find great deals on all my hotel and flights and have never had any issues.
	The programme is slow and hasn’t been working. Trying to change a flight has been a nightmare.
	Most clean, best designed and easy to use application i have ever seen on android market.
	I love Expedia. They save me money by rebating reductions in airfare that occur after I book. They have great customer service.
	Easy booking, and rewards, customer service never gave me a hassle it’s just annoying when trying to change flights sometimes.

shows sample comments for the two mobile applications.

The dataset was pre-processed for sentiment analysis and topic modelling. During this process, the HTML tags, special characters, and URLs were cleaned. Stop words (e.g., ‘and’, ‘the’) were removed, texts were converted to lowercase, and tokenisation was applied. The lemmatisation method was preferred when finding word roots. These steps aimed to standardise the texts and improve the accuracy of the analysis.

3.2. Text Mining

Text mining is a branch of artificial intelligence that analyses large amounts of text data and extracts meaningful information from these data. Text Mining is the process of discovering new, previously unknown, potentially useful knowledge from any type of unstructured data source, including business documents, customer reviews, web pages, and XML files [36]. It targets studies such as text classification, clustering, concept/entity extraction, production of granular taxonomy, sentimental analysis, document summarisation, and entity relationship modelling. It aims to extract meaningful and useful information from text data using techniques such as text mining, natural language processing (NLP) and machine learning [37].

Information extraction, one of the important tasks in text mining, is the process of extracting structured information from text data. Text mining can automatically identify certain pieces of information from texts and transfer this information to databases or make it available in some other way. Text classification is the process of classifying text data into specific categories or classes [38]. Text mining can automatically assign textual data to different classes using machine learning techniques. Text clustering is the process of dividing text data into groups with similar characteristics. The content of texts can be related to each other and grouped based on their similarities or themes. Content recommendation is the process of recommending specific pieces of content (e.g. articles, videos, and products) to a user [39]. Text mining can be used to recommend appropriate content to users based on their past interactions, preferences, and profile information. Word frequency can be used to determine the terms or concepts that appear most frequently in a dataset. When looking at customer reviews, social media conversations, or customer feedback, it can be helpful to determine which words are used the most [40].

3.3. Sentiment Analysis

Sentiment analysis is the process of identifying and analysing emotions expressed in text. Text mining can identify emotional states in large amounts of text data, such as social media messages, product reviews, and customer feedback. Sentiment analysis is a computational study of people’s opinions, evaluations, attitudes, and feelings [41]. It is used in many areas, such as finance, medicine, the stock market, media and politics. It is a widely used technique in product reviews to measure consumer satisfaction with a product. It helps companies improve their products and services based on real and clear customer feedback [42].

Sentiment analysis is basically a text analysis that aims to determine the emotional class (positive, negative and neutral) that the given text wants to express. There are three generally accepted types of sentiment classification methods used. These are the dictionary-based approach, machine learning and hybrid approach. In dictionary-based sentiment analysis approaches sentiment dictionaries are created by classifying words or word groups (n-grams) according to the sentiment they contain (positive, negative and neutral). According to these dictionaries, the emotional states of texts can be determined. The number of positive and negative dictionaries each text contains is examined, and the emotion is revealed by assigning it to the class with the highest number. In the supervised machine learning method, a model is trained with labelled source data. The trained model then makes a prediction for output by considering the new unlabelled input data. The hybrid approach is a model formed by the combination of the dictionary-based approach and machine learning models [17,43].

3.4. Machine Learning

Machine learning is a sub-branch of artificial intelligence that uses a model trained by capturing patterns from data to predict outcomes for values that are not present in the dataset. Machine learning is the development of various algorithms and techniques to enable computers to learn in a manner similar to that of humans [44]. It makes inferences from data using mathematical and statistical methods without human intervention. There are three types of machine learning: supervised learning, unsupervised learning, and reinforcement learning. In supervised learning, a labelled dataset is used to train the algorithm. In unsupervised learning, it aims to create a pattern among unlabelled data. The output of the unsupervised learning algorithm cannot be predicted. Reinforcement learning is a type of machine learning that aims to train an agent (robot, vehicle, etc.) with the reactions it receives from the environment without training data. The reactions that occur are subject to a predetermined reward system. The agent is trained in line with the reward earned and understands how wrong or right its actions are [45].

3.5. Topic Modelling

It is a type of statistical model that investigates abstract topics within a document set using machine learning and natural language processing. It is a machine learning technique that is unsupervised learning that can automatically cluster similar expressions with phrases that best characterise a document set [46]. Topic modelling is a machine learning method that determines the semantic structure of a document containing text. Topic modelling generally gives an idea of what the topic of a document is about. Topic modelling is used to analyse topics rather than words. Co-occurring words can be thought of as a label of the collection. It has also been shown to be a research area of natural language processing [47].

Topic modelling methods can organise and summarise high-content text documents. Within a document set, groups of words called topics are maintained in a hidden and unstructured form within the documents. A characteristic of these topics is that they appear together frequently in the text and generally consist of words that share a common or similar theme. In topic modelling, words that frequently appear together in the text are clustered to produce abstract topics and related texts are positioned in one or more clusters that are closest to them according to the words they contain [48]. Topic modelling can be successfully applied in many areas, such as automatic document indexing, document classification, and topic discovery. A document can be represented by a combination of topics. Topics are calculated as a probability distribution over words, and a document is calculated as a probability distribution over topics [48].

3.6. Naive Bayes Classifier

The Naive Bayes algorithm is an effective probability-based algorithm used to solve classification problems in machine learning. Naive Bayes is a probabilistic algorithm that makes predictions about the class of a given piece of text based on Bayes’ Theorem, which is frequently used in statistics [49]. It trains the dataset by considering that the probability of an event occurring is equal to the product of the prior probability of the event. The Naive Bayes algorithm is an algorithm that provides fast and effective results in classification problems depending on the size and complexity of the data set.

Naive Bayes is widely used in natural language processing (NLP) and text-based tasks such as spam detection, sentiment analysis, and document classification [50].

The basic idea behind Naive Bayes is to calculate the probability that a piece of text belongs to a certain class, given the words in the text. The algorithm does this by calculating the probability of each word in the text, assuming that the text belongs to a certain class. It then multiplies these probabilities together and compares them with the prior probability of the last class. The class with the highest probability is selected as the classification of the text [51].

$$\mathrm{P}\left(\mathrm{X}\right)={\displaystyle \frac{\mathrm{P}\left(\mathrm{C}\right)·\mathrm{P}\left(\mathrm{C}\right)}{\mathrm{P}\left(\mathrm{X}\right)}}$$

(1)

P(A|B) is the probability that event A will occur given event B ( posterior probability). P(B|A) is the probability that event B will occur given event A. P(A) is the prior probability of event A. P(B) is the prior probability of event B occurring (also known as the marginal probability).

While the Naive Bayes algorithm is based on the Bayes theorem, the reason it is called “naive” is that the algorithm assumes independence between features. In other words, the features within a sample are independent of each other. This assumption of independence simplifies the probability calculations and allows the algorithm to detect frequent patterns in the dataset more quickly [52].

4. Results

In this study, the text preprocessing step was first performed to analyse user comments. Correct text preprocessing ensures that machine learning applications yield better results and that the models are improved. Noise removal is the removal of unwanted situations from text. In this step, the URL, HTML tags, and special characters are removed, and excess space is deleted. Stop words are words that are frequently used in texts but are usually meaningless or unnecessary in certain analyses. In text mining, stop words are usually excluded from the analysis to reduce the size of the data set and make the analysis results more meaningful. Tokenising is the process of breaking a text document into smaller pieces, or “tokens”. This process can be similar to splitting a sentence into words or text into paragraphs. Tokenising is used as a preprocessing step to process text data. When dealing with text processing, for example, when performing a simple search, we are more interested in the root form of the word than the affixed form. Generally, there are two methods to access the root of the word: stemming and lemmatisation. The purpose of both processes is the same: to clean each word from its affixes and convert it into a common base or root. Both methods are used for the analysis of words. Stemmer finds the root by cutting, while lemtiser finds the root by obtaining its real form. In this study, stop words, tokenising, and lemmatiser methods were applied [53].

4.1. Sentiment Analysis Results

In this study, the Naive Bayes method was used to classify the sentences in the data set according to their emotional states. In this probability-based method, the data set is first divided into two parts: training and testing. Training data is used to determine the parameters of the model; test data is used to evaluate the developed model. In the separation of the data set, different ratios (such as 60–40%, 70–30%, 80–20%) were tested, and the best result was obtained with a 75% training and 25% testing ratio. Accordingly, 750 comments were used to train the model and 250 comments were used during the testing phase of the model. This process was carried out meticulously to increase the overall success and accuracy of the model.

The metrics used to evaluate the performance of classification models play a critical role in measuring the predictive accuracy and effectiveness of the model. In this context, metrics such as precision, recall, accuracy, and F1 score (F-measure) are widely preferred for evaluating classification performance. Each of these metrics evaluates different aspects of the model and may have different degrees of importance in various application scenarios.

Precision is the success rate of its predictions, that is, it shows the number of predicted positive classes (classes predicted as 1) that are positive. It focuses on the success of the predictions [54].

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}}$$

(2)

TPs (True Positives) represent the number of true positive predictions, while FPs (False Positives) represent the number of false positive predictions.

Recall is the rate at which the positive class (1) is correctly predicted. It shows how many of the predicted positive classes are correctly predicted. It is an important measure because it provides information about the cost of oversights. It focuses on the success of capturing the facts [54].

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(3)

In this formula, FN (False Negatives) represents the number of false negative predictions.

Accuracy is the correct classification rate. The accuracy value is calculated by the ratio of the areas we correctly predicted in the model to the total data set. The maximum value of accuracy can be 1 [55].

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(4)

In the formula, TN (True Negatives) represents the number of correct negative predictions, while TP, FP and FN are defined as explained above.

F1 score is the harmonic mean of precision and recall values [55].

$$\mathrm{F}1={\displaystyle \frac{2\times \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\times \mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}}$$

(5)

These metrics are used to comprehensively evaluate the performance of the classification model. While precision, recall and F1 scores generally produce more meaningful results in unbalanced data sets, the accuracy metric is used to evaluate the overall success of the model. The importance of each metric varies depending on the application area and the structure of the data set [56].

Table 4. Performance measurement results.

Metrics	Booking	Expedia
Precision	0.971	0.959
Recall	0.983	0.908
Accuracy	0.958	0.918
F Score (F1)	0.977	0.933

shows the comparative results of the metrics used in the performance and success evaluation of the model developed using the Naive Bayes method, one of the machine learning models. The model created using the Naive Bayes method appears to be successful in user sentiment classification for Booking.com and Expedia. The precision metrics are 0.971 and 0.959, respectively, indicating that the model classifies positive examples correctly for both cases and that there are very few false positive classifications. The recall metric of 0.983 for Booking.com shows that it successfully detects samples that are positive and that there are few false negatives. The fact that Expedia’s score is 0.908, which is lower than that of Booking.com, shows that it cannot detect some of the positive samples. The accuracy metrics of 0.958 and 0.918, respectively, show that the model is more successful in the total correct classification of Booking.com than Expedia. The F1 score metrics of 0.977 and 0.933, respectively, show that the model both detects positives correctly and produces a small number of false positives. In general, the model developed using the Naive Bayes method is successful in classifying user reviews for Booking.com and Expedia according to their emotional state, and this model can be used.

Figure 1 presents the sentiment statistics for Booking.com and Expedia reviews on the Google Play Store. Of the 1000 reviews received for Booking.com, accessed on 12 September 2024, 819 were positive, 84 were negative, and 113 had neutral sentiments. Of the same number of reviews received for Expedia, 558 were positive, 378 were negative, and 80 had neutral sentiments. There is a significant difference between the positive and negative sentiments of the two travel apps. The content of this difference is discussed throughout this article.

4.2. Text Mining Results

The b-gram and TF-IDF methods were used for text mining operations. An n-Gram is a grouping method created by combining n consecutive words or characters in a text document. For example, 2-grams (bigrams) in a text document represent two consecutive words, while 3-grams (trigrams) represent three consecutive words [57]. The bigram grouping method was used in this study. Term Frequency indicates how often a particular term occurs in a document. The frequency of a term in a document is usually calculated as the ratio of the number of terms to the total number of words. Inverse Document Frequency determines how rare or widespread a term is. Rare terms receive a higher weight, while common terms receive a lower weight. This is usually calculated as the ratio of the total number of documents to the number of documents containing a particular term. TF-IDF (Term Frequency-Inverse Document Frequency) is a statistical measure obtained by normalising the frequency of a word in a document by its frequency in all documents. This method is used to determine the importance of a word in a document. High TF-IDF values indicate that the word in a document is unique to that document and is not associated with other documents [58,59].

Table 5. Booking.com word frequency list.

Positive	Negative	Neutral
(‘application’, 194)	(‘hotel’, 38)	(‘application’, 22)
‘hotel’, 168)	(‘reservation’, 26)	(‘hotel’, 20)
(‘made’, 108)	Booking 25	(‘find’, 14)
(‘booking’, 93)	(‘application’, 23)	(‘problems’, 13)
(‘easy’, 90)	(‘made’, 20)	(‘reservation’, 12)
(‘reservation’, 86)	(‘said’, 15)	(‘make’, 11)
(‘nice’, 81)	(‘money’, 14)	(‘Thank’, 8)
(‘Thank’, 74)	(‘find’, 13)	(‘use’, 8)
(‘great’, 63)	(‘bad’, 11)	(‘application.’, 8)
(‘useful’, 60)	(‘reach’, 11)	(‘write’, 7)
(‘use’, 57)	(‘room’, 11)	(‘reservations’, 7)
(‘find’, 50)	(‘service’, 11)	(‘looking’, 6)
(‘good’, 47)	(‘search’, 11)	(‘abroad’, 6)
(‘thank’, 46)	(‘use’, 10)	(‘prices’, 6)
(‘reliable’, 46)	(‘problem’, 10)	(‘plane’, 6)
(‘accommodation’, 41)	(‘customer’, 10)	(‘Booking’, 5)

presents the frequencies of positive, negative, and neutral words extracted from customer reviews on Booking.com. The list includes the 15 most frequent words for each sentiment category, selected from a total of 1000 reviews analysed.

Table 6. Expedia word frequency list.

Positive	Negative	Neutral
(‘Expedia’, 305)	(‘app’, 232)	(‘app’, 31)
(‘app’, 239)	(‘booking’, 200)	(‘Expedia’, 16)
(‘hotel’, 190)	(‘Expedia’, 192)	(‘booking’, 32)
(‘booking’, 172)	(‘hotel’, 146)	(‘need’, 10)
(‘use’, 139)	(‘use’, 128)	(‘find’, 10)
(‘great’, 147)	(‘service’, 77)	(‘time’, 9)
(‘room’, 76)	(‘time’, 75)	(‘issues’, 8)
(‘easy’, 66)	(‘room’, 73)	(‘charged’, 8)
(‘customer’, 64)	(‘customer’, 71)	(‘stay’, 8)
(‘flight’, 64)	(‘flight’, 66)	(‘problem’, 7)
(‘service’, 56)	(‘trying’, 52)	(‘try’, 7)
(‘travel’, 55)	(‘money’, 51)	(‘email’, 7)
(‘refund’, 54)	(‘help’, 44)	(‘people’, 7)
(‘trip’, 52)	(‘tried’, 44)	(‘review’, 7)
(‘price’, 52)	(‘refund’, 43)	(‘helpful’, 7)

shows the frequencies of positive, negative, and neutral words extracted from customer reviews of the Expedia brand. The list includes the 15 most frequent words for each sentiment category, selected from a total of 1000 reviews analysed.

4.3. Word Cloud Result

Visualisation is the process of representing text mining results visually. This can be achieved using word clouds, graphs, maps, or other visual aids. Visualisation can help us more easily understand patterns and relationships in text data. Word Clouds provide a cloud-like display of words sized based on the frequency of words in texts [60]. In this study, the Seaborn library used in the Python programming language was used for data visualisation. This library provides predefined styles and colour palettes to facilitate the data visualisation process [61].

Figure 2, Figure 3 and Figure 4 below show the positive, negative, and neutral word clouds for the Booking.com and Expedia applications, respectively. The word clouds were analysed by comparing the prominent words for each brand.

Figure 2 shows the positive word clouds for both Booking.com and Expedia, revealing that terms associated with customer satisfaction appear frequently in user comments. When comparing feedback from these international travel and planning companies, words such as “app”, “hotel”, “booking”, “use”, “great”, and “easy” prominently feature for both brands, indicating that consumers are pleased with their choice of application. Additionally, the recurring mention of “hotel” highlights that both brands successfully meet customer expectations in presenting accommodation options. Terms like “useful” and “easy” suggest that users perceive the applications as beneficial tools, while “great” reflects positive emotions related to service quality, customer relations, and overall user experience. Notably, distinctions in positive terminology provide insights into consumer behaviour; for instance, high-frequency words for Expedia, such as “room”, “reservation”, “travel”, “trip”, and “flight”, indicate the brand’s effectiveness in guiding consumers through the buying process, while “price” and “refund” suggest favourable perceptions of value and return policies. In contrast, Booking.com features terms like “reservation”, “nice”, “thank”, “find”, “good”, “reliable”, and “accommodation”, showcasing the brand’s strength in fostering customer satisfaction regarding reservations and building trust among users. The presence of emotionally positive words reflects the positive experiences consumers associate with the application, underscoring the significance of emotional connection in consumer behaviour and marketing strategies.

Figure 3 shows the negative word clouds for both Booking.com and Expedia, highlighting user concerns across the applications. Common terms such as “app”, “booking”, “hotel”, “use”, “service”, “customer”, and “money” dominate these negative word clouds, indicating a mix of satisfaction and significant dissatisfaction among users. For Expedia, the frequent use of “app” and “useful” suggests that some users struggle with navigating the application, while direct references to “Expedia” in negative comments reflect a negative impact on brand perception and reputation. Additionally, the prominence of words like “help” and “customer” points to a need for improved support services, and terms such as “Booking”, “hotel”, and “room” reveal frustration with the guidance provided. Similarly, in Booking.com’s negative word cloud, the presence of “app”, “search”, “use”, and “find” signals general discontent with usability, while “reservation”, “hotel”, and “room” indicate navigational challenges. Explicit terms like “problem” and “bad” further highlight customer experiences that fall short of expectations, necessitating enhancements in service quality. Overall, the analysis underscores critical areas for improvement in both brands’ problem-solving capabilities and service quality, with customer feedback serving as a valuable resource for identifying specific shortcomings and guiding necessary enhancements to enhance user experience and satisfaction.

Figure 4 shows the neutral word clouds for Booking.com and Expedia, which comprise terms that lack emotional weight and do not explicitly indicate customer satisfaction or dissatisfaction. In the neutral word cloud for Expedia, key terms such as “app”, “Expedia”, “booking”, “need”, and “find” are prominent, suggesting that users focus on functional aspects of the application rather than expressing strong feelings about their experiences. Similarly, the neutral word cloud for Booking.com reflects a similar trend, showcasing words that indicate straightforward engagement with the application. Although neutral words do not carry positive or negative connotations, they play a crucial role in illustrating overall brand perception among consumers. Furthermore, these neutral comments provide valuable insights for brands, highlighting potential areas for improvement that could enhance customer satisfaction. By analysing these terms, brands can better understand user needs and expectations, ultimately guiding marketing strategies and service enhancements to create a more favourable customer experience.

4.4. Topic Modelling Results

When the topic was set to 5 for Booking.com and Expedia, coherence scores of 0.71 and 0.67 were achieved, respectively, indicating a high level of semantic similarity and relevance between the topics. The coherence score is a value that measures the quality of a topic model and has a value between 0 and 1. One represents perfect coherence, and 0 represents no coherence. This shows that the five topics provide an effective balance in features that capture the required level of detail of the topics in the dataset without overly fragmenting the data. This optimal point suggests that our text collection is separated into five well-defined, distinct topics and validates the robustness of our LDA topic modelling approach [62]. In trials with a larger number of topics (10, 15, etc.), the consistency score dropped to very low levels, and the topics that emerged were seen to be vague and difficult to interpret.

Figure 5 and Figure 6 show the estimated frequency of each term within the topic compared with the overall model frequency. The LDA topic visualisation tool pyLDAvis library was used for data visualisation. PyLDAvis is designed to help users interpret topics in a topic model as appropriate for a collection of text data. The blue colour shows the importance of that word in the general dataset, and the red colour shows the importance of that word in the selected topic. The red parts of a word indicate the more specific and meaningful the word is for that subject.

Five topics of the Booking.com application and the words belonging to the topic are shown in Figure 4. When the words in Topic 1, which is dominated by the words ‘Application’, ‘thank’, ‘nice’, booking’, ‘easy’, ‘use’, ‘useful’, ‘reservation’, are examined, there are comments regarding satisfaction with the Booking.com application. There are also references to the ease of booking on the app. Additionally, there are expressions of appreciation for the application. Topic 2, which includes the words ‘hotel’, ‘great’, ‘application’, ‘reservation’, ‘want’, ‘booking’, ‘find’ and ‘thanks’, contains words that contain the functions of the application. Users have described the functions of finding the hotel they want and making a reservation as good. Functional problems predominate in the expressions in Topic 3, where the words ‘booking’, ‘hotel’, ‘problems’, ‘reservations’, ‘application’, ‘made’, ‘reservation’, ‘payment’, ‘without’ predominate. Problems with making reservations, problems originating from hotels outside the application, and payment problems are among the main problems mentioned in Topic 4, which includes the words ‘reservation’, ‘hotel’, ‘place’, ‘stay’, ‘find’, ‘accommodation’, ‘application’, and ‘one’, contains words related to the services offered in the application. Finding a hotel, accommodation, and reservation are the most prominent services of the application. The features of the application are included in Topic 5, which includes the words ‘fast’, ‘reliable’, ‘easy’, ‘useful’, ‘abroad’, ‘reservation’, ‘free’, ‘accommodation’, ‘site’, ‘options. Users have referred to the Booking.com application for its speed, reliability, ease, usability, international expansion, being free and offering options.

Five topics of the Expedia application and graphs belonging to each topic are shown in Figure 5. Topic 1, which includes the words ‘Expedia’, ‘app’, ‘get’, ‘hotel’, ‘service’, ‘use’, ‘great’, ‘refund’, ‘flight’, ‘customer’, contains words that emphasise the services provided by the Expedia application to its customers, such as finding hotels, flights, and refund services. In Topic 2, where the words ‘Expedia’, ‘hotel’, ‘app’, ‘use’, ‘room’, ‘money’, ‘booked’, ‘refund’, ‘booking’, ‘cancellation’ is dominant, users commented on issues related to payments and refunds after cancellations in the Expedia application. In Topic 3, the words ‘App’, ‘Expedia’, ‘hotel’, ‘trip’, ‘time’, ‘never’, ‘would’, ‘book’, ‘good’, ‘use’ emphasises the application’s travel menu and timing issues. In Topic 4, the words ‘App’, ‘get’, ‘booking’, ‘room’, ‘Expedia’, ‘customer’, ‘hotel’, ‘service’, ‘trying’, ‘book’ highlight that customers are trying to make a reservation with the app in their comments. ‘Topic 5, which includes the words ‘Flight’, ‘app’, ‘best’, ‘Expedia’, ‘service’, ‘customer’, ‘book’, ‘price’, ‘using’, ‘trying’, highlights the best flight finding service of the Expedia app. Considering the sentiment analysis results of the Expedia app, we can say that the word ‘Cancellation’, which has a negative meaning, is used under four headings, and the Expedia app is not able to meet the satisfaction of its customers who have experienced cancellations in their trips.

5. Conclusions

This study employed topic modelling and sentiment analysis techniques to examine consumer behaviour and emotions derived from customer comments about mobile applications used in the online travel industry, specifically focusing on Booking.com and Expedia. By analysing user feedback, the research revealed insights into the sentiments and concerns expressed by customers of these popular travel agencies, highlighting aspects that influence their overall experiences.

The analysis indicates that a significant portion of user comments reflect positive emotions, with 81.9% of reviews for Booking.com and 55.8% for Expedia being categorised as positive. This positivity suggests that users find the applications beneficial for their travel needs, particularly regarding hotel preferences and service quality. Positive sentiment words indicate that customers appreciate the usability of the applications and have had favourable experiences in areas such as customer relations and support. For Booking.com, the prevalence of words related to trust and satisfaction demonstrates effective customer engagement, while for Expedia, terms indicating customer-friendly services and the perceived value of fees point to a successful user experience.

Conversely, negative sentiments were observed in 8.4% of Booking.com reviews and 37.8% of Expedia reviews, suggesting areas of concern for both applications. For Booking.com, some negative comments stem from user frustration with the application itself. In contrast, Expedia faces a higher volume of negative feedback, with users reporting difficulties in application usage, unsatisfactory staff interactions, inadequate guidance, and issues with refunds. These negative sentiments suggest that both brands must enhance their problem-solving capabilities to effectively address customer concerns.

Additionally, neutral sentiment comments accounted for 11.3% of the reviews on Booking.com and 8.0% on Expedia. These neutral observations offer potential customers objective insights into the applications, which can help inform their purchase decisions. Overall, the findings indicate that understanding consumer emotions and feedback can significantly inform marketing strategies, guiding improvements in service delivery and enhancing overall user satisfaction.

The research findings reveal that 81.9% of Booking.com users have positive reviews, while 55.8% of Expedia users have positive reviews. This shows that there is a significant difference in terms of app performance and user satisfaction.

This study aims to gain deeper insights into the needs, expectations, and opinions of mobile application users by analysing texts that express users’ emotions, thus overcoming the limitations often associated with traditional survey methods. Recent trends in research have highlighted the use of big data analytics, particularly topic modelling and sentiment analysis, to extract meaningful insights from consumer feedback. The application of these methodologies has seen a notable increase in academic studies related to the tourism and travel services sector.

Customer reviews play a pivotal role in shaping consumer behaviour, as they directly influence perceptions of trust and reliability. Positive reviews not only foster brand advocacy but also enhance customer loyalty, while negative feedback provides brands with essential insights into areas needing improvement. Effectively analysing this feedback enables brands to address shortcomings, ultimately contributing to a more favourable consumer experience and long-term customer retention.

Furthermore, the findings from this research offer a practical and flexible approach for mobile travel application developers and researchers seeking to understand user sentiments and improve service offerings. By leveraging consumer emotions expressed in reviews, brands can refine their marketing strategies and align their services with user expectations, thereby enhancing overall consumer satisfaction and engagement.

These findings provide important insights for mobile app providers to make strategic decisions based on user reviews. While Booking.com focuses on user satisfaction, Expedia’s negative feedback indicates a need for improvement, especially in the cancellation and refund processes. These insights can be used to improve mobile app design, optimise customer support processes, and strengthen brand reputation management.

This study reflects the views of a specific group of users, as it only analysed Google Play reviews. A broader analysis of data from other platforms could strengthen the findings. Furthermore, a more in-depth understanding can be developed using different analysis methods. Future studies can offer more comprehensive industry-wide insights by making comparisons with other mobile travel apps.

Future work could include the application of sentiment analysis and topic modelling on larger and different datasets. In addition, an analysis of mobile apps in other industries and comparative studies with deep learning-based models can be conducted. This will allow testing the generalisability and applicability of the method in a wider context.

This study makes a significant contribution to the literature by analysing sentiment expressions to gain a deeper understanding of the needs, expectations, and opinions of mobile app users. Overcoming the limitations of traditional survey methods, examining texts that reflect users’ emotions allows for a more realistic assessment of consumer behaviour. Furthermore, the increasing use of big data analytics methods, such as topic modelling and sentiment analysis in tourism and travel services, has added a new dimension to academic research in this field. Future studies offer rich opportunities to apply these methodologies to larger data sets to examine the emotional responses of different user groups and to develop strategies to enhance user experience in mobile application design. Furthermore, research in this area can help brands understand consumer behaviour and strengthen customer loyalty by effectively analysing user feedback. In this context, conducting more in-depth analyses and examining applications in different sectors will make important contributions to filling the existing gaps in the literature and improving customer experience.