Sentiment Analysis Telemedicine Apps Reviews Using NVIVO ^†

Abstract

This study aims to analyze the sentiments of Halodoc users after the COVID-19 pandemic declined. This study uses a mixed method approach, preceded by quantitative research, to see user satisfaction from the rating. Qualitative research from Halodoc google play reviews from June to August 2022 of 1129 users. A total of 74.8% of users rated positive, 5% moderately positive, 2.6% neutral, 2.2% slightly negative, and 15.5% negative. Aspects that need attention from Halodoc to improve its implementation are improving service quality, adding payment methods and refunds, adding a network of health service facilities, and improving systems.

1. Introduction

The increasing growth of the world’s population has led to a decline in health facilities’ ability to serve the community’s needs. As a result, several developed countries have started to provide technology to integrate clinical care and user needs. Unlike developed countries, developing countries still have a technological gap [1]. The COVID-19 pandemic that has persisted for the last few years has succeeded in changing people’s behavior in receiving health services [2]. People use telemedicine to reduce the spread of COVID-19 due to restrictions on visits to health services, such as hospitals and clinics. Indonesian people are starting to have a significant interest in health portals and applications; this is due to increased public awareness and adoption of internet technology [3]. Halodoc is a telemedicine application that is quite popular in Indonesia and has become 1 of 150 digital startups in the health sector that have the potential to grow [4]. Halodoc is a telemedicine application that features online consultations with doctors via chat, voice, and video calls, supporting examination features, such as laboratories and drug purchases. [5]. Halodoc as a telemedicine application requires efforts to improve service quality and evaluate performance based on user service sentiment data. For example, in the early days of the COVID-19 pandemic, users of conventional health facilities switched to using telemedicine applications. In 2022, when the pandemic is under control, people can visit health facilities, such as clinics and hospitals, thus telemedicine applications must be able to provide more satisfaction if users want to continue using them.

User sentiment on the services provided can be seen through reviews on the google play store [4]. Evaluation of user satisfaction has a significant impact on the continuous development of telemedicine applications [6]. Sentiment analysis is carried out to analyze the text on opinions, feelings, and emotions after getting products or services from different communication platforms [7]. Application reviews are one of the reference information that can describe the application. Users rate apps without coercion to contribute to creating big data app user experiences [8]. Telemedicine application developers use user reviews to understand needs and make informed decisions. Telemedicine application developers focus on negative reviews by comparing these reviews with other telemedicine applications to improve performance and user satisfaction [6].

Telemedicine applications in Indonesia since the COVID-19 pandemic have seen a significant increase in the number of users. The telemedicine application seeks to improve services by facilitating users to review the services provided. The review is used as an evaluation material by telemedicine application developers to improve performance, quality, and user satisfaction. In this study, researchers will focus on one of the telemedicine applications widely used in Indonesia: Halodoc. This study aims to determine the user’s sentiment toward the Halodoc application.

2. Material and Methods

Sentiment analysis is a research methodology mainly used on digital platforms and social media to analyze the feelings given in the form of different opinions with different methods. In conducting sentiment analysis, special software, artificial intelligence, and hybrid models can be used. Another option with data mining techniques to increase the chances of success in getting results accuracy [9]. The software commonly used is NVIVO, while other data mining tools use XGBoost Classifier. This study uses a mixed methods approach preceded by quantitative research to see user satisfaction from the rating. Qualitative research from Halodoc google play review data from June to August 2022 had as many as 1129 users. This research consists of five stages: data collection, rating categorization, data preprocessing, visualization of words that often appear in both positive and negative sentiments, and analyzing aspects complained of by telemedicine application users.

Researchers collected review data from the Halodoc application using the google play scraper library on Python in the form of the Halodoc user username, the rating given, the date and time the user provided the review, and the check issued. The data that will be analyzed further are ratings and reviews from users. In the quantitative phase, the rating obtained were categorized as one being negative, two being somewhat negative, three being neutral, four being reasonably positive, and five being positive [4]. The rating data is processed descriptively so that the percentage of rating categories is negative, somewhat negative, neutral, reasonably positive, and positive.

The qualitative phase is to visualize the words that often appear and analyze the factors that telemedicine application users complain about using NVIVO (QSR International, Burlington, MA, USA). The visualization outputs of frequently occurring words are negative and positive word frequency query results and negative and positive world cloud sentiments. Researchers use the word count feature in NVIVO to identify critical themes obtained from big data. The set keywords are 4–5. The keywords obtained are to increase the researcher’s understanding of the phenomena that occur as a guide for topics that will be analyzed further [10]. In improving Halodoc services, it is necessary to know the factors many telemedicine application users complain about into four aspects, service, payment, system, and place [4].

3. Results and Discussion

3.1. Quantitative Phase

After rating data and user reviews are obtained from the google play scraper library on Python, the next step is categorizing ratings into sentiment categories for Halodoc application users. The rating data is categorized into negative, somewhat negative, neutral, reasonably positive, and positive categories. Halodoc application ratings from June to August 2022 can be seen in

Table 1. Rating Aplikasi Halodoc June–August 2022.

Category Rating	Frequency	Percentage (%)
Negative	175	15.5
Somewhat negative	25	2.2
Neutral	29	2.6
Reasonably Positive	56	5.0
Positive	844	74.8
Total	1129	100

and Figure 1 below.

Halodoc app users gave a positive rating of 74.8%, negative 15%, reasonably positive 5%, 2.6% neutral, and 2.2% somewhat negative. This shows that most of the Halodoc application users give positive ratings. According to Tarmidi et al., the better the innovation offered by telemedicine applications at affordable prices, customer satisfaction will increase [11]. Therefore, if Halodoc users increase their satisfaction after using the application, it will certainly give a positive rating. Reviews from users are valuable input compared to professional reviewers who do not have experience using the app in real-time when you need it [12].

In developed countries, the use of telemedicine applications developed before the pandemic. In developing countries, telemedicine applications were initially used for countries with a small populations to provide access to health services for people in rural areas [13]. Looking at the user sentiment analysis on one of the positive sentiments shown by Halodoc users in Indonesia shows that the Indonesian people, who have a large population spread across urban and rural areas, need these services. Halodoc makes it easy for users to consult more than 20,000 licensed doctors via smartphone or desktop and has collaborated with more than 1200 pharmacies and online transportation services to deliver medicines [14].

3.2. Qualitative Phase

This stage begins with preprocessing Halodoc user review data, ensuring that the reviews obtained are structured, and no noise is found. In this study, the review data is in Indonesian because users of the Halodoc application are people who live in Indonesia. After preprocessing the data, the researcher visualizes the words that most often appear in positive and negative sentiments. This is to make it easier for researchers to explore the factors complained of and the factors that provide customer satisfaction with the Halodoc application [4]. Figure 2, Figure 3, Figure 4 and Figure 5 shows the words that most often appears in both positive and negative sentiments from Halodoc users.

From the data visualization above, the most frequent words with positive sentiment are “membantu”, “cepat”, “ramah”, “aplikasi”, respon”. The most frequent words with negative sentiment are “dokter”, “aplikasi”, “konsul”, “resep”, “konsultasi”. The significant growth of the mobile healthcare industry in the last few years is marked by the emergence of similar applications. [15], Halodoc has a positive sentiment from users. According to the research of Afifah et al., the sentiment of Halodoc users has been quite good since the COVID-19 pandemic [4]. In the June–August 2022 period, where the pandemic condition has decreased, and similar applications have begun to appear, Halodoc’s sentiment is still positive.

As shown in

Table 2. Categorization Summary From Halodoc’s Data Sheet Review.

Category	Total	Percentage (%)
Service	593	52.48
System	333	29.50
Payment	174	15.37
Place	30	2.64
Total	1129	100

, Halodoc application users gave more reviews related to service (52.48%), than to system (29.50%), payment (15.37%), and place (2.64%). Thus, Halodoc users can provide positive reviews, the main focus is to provide complete service so that users feel satisfied. The second focus is on improving the system so that when it is accessed, it does not make frequent errors, and determines an accurate position when drug delivery. The third focus is adding payment methods and making it easier for users to refund. The fourth focus is to expand the network of healthcare facilities that work together, such as hospitals, clinics, pharmacies, and laboratories. Similar to Halodoc, Alodoc has the main aspects that affect user sentiment, namely attitudes and behavior by 44.83%, expected usefulness at 27.12%, trustworthiness at 18.5%, and ease of use at 9.56% [3].

Telemedicine application developers can also consider the target market and user preferences. For example, the target market for telemedicine application users in Indonesia is single women aged 35 years and under, with an education below a bachelor’s degree, working as employees or students with an average income of under 9 million per month, living in urban areas, and using applications for personal health. As for the user’s preference, choosing the telemedicine application is an application that can help their health care, an application that is easy to use provides information confidence, and can be a part of efforts to improve health [16].

4. Conclusions

The sentiment of users of telemedicine applications in Indonesia, especially Halodoc, during the COVID-19 pandemic has begun to decline, but are all positive. Aspects that need attention from Halodoc to improve its application are services, payment systems, adding a network of health service facilities, and system improvements.