- To develop a novel and systematic method for converting social media comments to SERVQUAL dimensions and analyzing online sentiments in Malaysia via supervised learning.
- To classify topics based on an established methodology for service quality; SERVQUAL that is extensively used to assess the quality of health care services, overcoming obstacles, and providing policymakers with precise action implications.
- By identify the determinants of positive sentiment as well as its relationship with hospital accreditation in Malaysia using advanced statistical analysis.
- Via real-time monitoring of hospital quality and patient perceptions of health care services through the translation of social media data.
- Through the machine learning technology that can be utilized as an early-warning system for immediate quality improvement in healthcare.
2. Literature Reviews
2.1. Social Media Data
2.2. SERVQUAL Dimensions
2.3. Automation of SERVQUAL and Sentiment Classification
2.4. Topics and Sentiments in Patient Online Reviews
2.5. Proposed Work
3. Materials and Methods
3.1. Hospital Facebook Data
3.2. SERVQUAL Dimensions Classification
- Two hospital quality managers or SERVQUAL domain experts were appointed to do an initial “open” coding on batches of 100–300 Facebook reviews based on the MOH SERVQUAL patient satisfaction survey in order to create the source coding standard (Appendix A.1). Additionally, we supplemented descriptions in relevant dimensions using survey questions from previous SERVQUAL research.
- Next, a randomly selected subsample of 300 Facebook reviews was used to assess intercoder reliability. The reliability subsample was coded independently by the raters. Cohen’s Kappa values were used to determine inter-rater agreement for each SERVQUAL dimension. The agreement between the coding of Tangible (Cohen’s = 0.885, p < 0.001), Empathy (Cohen’s = 0.875, p < 0.001), Reliability (Cohen’s = 0.736, p < 0.001), and Responsiveness (Cohen’s = 0.72, p < 0.001) characteristics from Facebook reviews was high, but agreement for Assurance (Cohen’s = 0.626, p < 0.001) was modest. Cohen’s coefficient averaged 0.769 across all dimensions.
- Then, we utilized a sample of 900 manually labeled Facebook reviews to train our machine learning quality control tool.
3.3. Outcome: Sentiment in Facebook Reviews
3.4. Comparison with Hospital Accreditation
3.5. Statistical Analysis
4.1. Hospital and Facebook Characteristics
4.2. Facebook Review Characteristics and Sentiment
4.3. SERVQUAL Dimensions
4.4. Determinants of Positive Sentiment
4.5. Association of Hospital Accreditation and Sentiment in Facebook Reviews
5.1. Service Quality and Sentiment Analysis
5.2. Accreditation and Sentiment Analysis
5.4. Limitation and Future Scope
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A.1. SERVQUAL Guideline
|Domain||Description||Facebook Reviews Example|
|Tangible||General: The appearance of employees, equipment, and physical facilities of the hospital.|
The hospitals have up to date equipment. The physical facilities are visually new or outdated.
The staff are well dressed, appear neat and good looking.
The appearance of the physical facilities of the hospital are well maintained with the type of services provided.
|“Cleanliness of the Hospital is good”|
“Car parking is difficult and limited”
“Satisfied with the facilities. Large room, feels like a hotel.”
“The hospital is well maintained, and their food is delicious.”
|Reliability||General: Accurate, dependable, and consistent performance of the service.|
When the hospital promised to do something by a certain time, it does so.
Hospital service is efficient and dependable.
The hospital provides services at the time as promise to do so.
The hospital keeps the records accurately or at online.
|“My appointment scheduled at 9 a.m. but then it was postponed to 12.00 p.m. Unbelievable.”|
“System needs to be improved especially discharge process. It took hours to settle it.”
“Efficient and top-quality hospital services”
“Staff mistakenly collected medical record of other patient with similar name of mine”
|Responsiveness||General: Willingness to provide prompt service to the patients.|
The hospital let patients know exactly when the services will be performed.
The staff give prompt services to patients upon request.
The staff are always willing to help their patients.
The staff give medical attention promptly.
|“My specialist took his time to explain me about my disease and how he will treat it”|
“They answered all my questions during the admission.”
“Arrived at emergency department due to road traffic accident and the medical team immediately respond to it.”
“I don’t feel any pain throughout the minor surgery on my arm, and it was done in a flash”
|Assurance||General: the staff knowledge and courtesy, ability to inspire trust, confidence, and security; also reflects on confidentiality and privacy of patients.|
The staff are trustworthy.
Patients feel safe in their transactions with the hospitals.
The staff are polite, friendly.
The staff have adequate support from the hospitals to do their jobs well.
|“The surgery was successful. Mr A is a competent and trusted surgeon.”|
“I feel comfortable and safe in this hospital. Just like at home”
“The staff at the front desk was rude.”
“The doctors and staff nurses in this hospital are skillful and well-trained”
|Empathy||General: Providing convenient services and giving attention or patience of the staff to the patients’ needs.|
The staff give patient personal attention and helpful.
The staff are knowledgeable to understand patient’s specific needs.
The hospital has patient best interests at heart.
The hospital has operating hours convenient to all the patients.
Cost of treatment is affordable for patients
|“Nurses are very helpful.”|
“A staff came and offered to help my father climb stairs without we ask him. We appreciated his kindness.”
“They are very concerned about patient’s condition and served it with their heart”
“The price is affordable compared to private hospital.”
Appendix A.2. Sentiment Analysis Guideline
|Category||Description||Facebook Reviews Example|
|Positive||Expression of liking, approval, gratefulness|
(Like, love, support, thankful, etc.)
|“I like this hospital. Doctors and nurses are pleasant and helpful.”|
“Thank you for your service, Doctor and nurses.”
|Positive qualities of hospital services and facilities|
(Clean room, efficient, fast appointment, affordable, etc.)
|“The wait time was brief. The pharmacy counter did an excellent job.”|
“The room is neat and tidy, and the food is delicious. I really like it.”
|Positive qualities of staff|
(Polite, friendly, helpful, responsive, etc.)
|“Staff are polite and kind.”|
“Dr. B took her time explaining my health condition until I understood it. It was greatly appreciated.”
|Encourage or recommend others to use||“I recommend having your baby delivered at this hospital.”|
“I like their antenatal counselling and will recommend it to other couples. It is extremely beneficial to us.”
|Positive/desirable effects of service|
(Successful treatment/procedures, good health outcome, etc.)
|“I’d like to thank Mr A for performing bowel surgery on my father. He is now doing well.”|
“I found the physiotherapy session to be beneficial. I’m able to walk with less pain now.”
|Negative||Expression of disliking or disapproval|
(Do not like, hate, etc.)
|“I hate the security guard.” He was impolite to me!”|
“I’m not a fan of the food service here. The food has no taste.”
|Negative characteristic of hospital services or facilities|
(Poor maintenance, slow service, expensive, long waiting time, etc.)
|“The discharge procedure was extremely slow.”|
“There are a limited number of parking spaces available, and getting one is difficult.”
“We waited for 5 h at the out-patient clinic before seeing the doctor. This is intolerable.”
|Negative qualities of staff|
(Rude, not-friendly, not-helpful, slow responsive, incompetency, etc.)
|“Staff nurses were rude and stubborn. I requested assistance but received no response.”|
“The doctor criticised us for arriving at the emergency department at 3 a.m. for treatment. We were annoyed by his attitude.”
(Surgical or procedural complications, medicolegal, poor health outcome, etc.)
|“My father fell in the toilet and was left alone for a few minutes. The hospital director must explain the incident to our family.”|
“After being admitted to this hospital two days ago, my husband’s condition has deteriorated. No one, however, can explain the situation to us”.
|Neutral||Review that reports factual|
|“Serdang Hospital is one of the Klang Valley’s cardiac centres”.|
“A Muslim-friendly hospital”
|Review as questions||“Do you have any spine surgeon in your hospital?”|
“How to get an appointment with your ear. Nose and throat (ENT) clinic?”
|Too ambiguous/unclear/greetings only||“Good morning.”|
“Let’s wait and see first”
- Jo Delaney, L. Patient-centred care as an approach to improving health care in Australia. Collegian 2018, 25, 119–123. [Google Scholar] [CrossRef]
- Hincapie, A.L.; Slack, M.; Malone, D.C.; MacKinnon, N.J.; Warholak, T.L. Relationship Between Patients’ Perceptions of Care Quality and Health Care Errors in 11 Countries: A Secondary Data Analysis. Q. Manag. Healthc. 2016, 25, 13–21. [Google Scholar] [CrossRef][Green Version]
- Hong, Y.A.; Liang, C.; Radcliff, T.A.; Wigfall, L.T.; Street, R.L. What Do Patients Say About Doctors Online? A Systematic Review of Studies on Patient Online Reviews. J. Med. Internet Res. 2019, 21, e12521. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Rozenblum, R.; Miller, P.; Pearson, D.; Marelli, A. Patient-centered healthcare, patient engagement and health information technology: The perfect storm: An integrated perspective from patients, families, clinicians and researchers. In Information Technology for Patient Empowerment in Healthcare; Walter De Gruyter: Berlin, Germany, 2015; pp. 3–22. [Google Scholar]
- Millman, E.; Pronovost, P.; Makary, M.; Wu, A. Patient-Assisted Incident Reporting: Including the Patient in Patient Safety. J. Patient Saf. 2011, 7, 106–108. [Google Scholar] [CrossRef] [PubMed]
- Chute, C.; French, T. Introducing Care 4.0: An Integrated Care Paradigm Built on Industry 4.0 Capabilities. Int. J. Environ. Res. Public Health 2019, 16, 2247. [Google Scholar] [CrossRef][Green Version]
- Gardner, J.W.; Linderman, K.W.; McFadden, K.L. Managing Quality Crossroads in Healthcare: An Integrative Supply Chain Perspective. Q. Manag. J. 2018, 25, 2–17. [Google Scholar] [CrossRef]
- Kruk, M.E.; Gage, A.D.; Joseph, N.T.; Danaei, G.; García-Saisó, S.; Salomon, J.A. Mortality due to low-quality health systems in the universal health coverage era: A systematic analysis of amenable deaths in 137 countries. Lancet 2018, 392, 2203–2212. [Google Scholar] [CrossRef][Green Version]
- Fung, C.H.; Lim, Y.-W.; Mattke, S.; Damberg, C.; Shekelle, P.G. Systematic Review: The Evidence That Publishing Patient Care Performance Data Improves Quality of Care. Ann. Intern. Med. 2008, 148, 111–123. [Google Scholar] [CrossRef]
- Lagu, T.; Goff, S.L.; Hannon, N.S.; Shatz, A.; Lindenauer, P.K. A Mixed-Methods Analysis of Patient Reviews of Hospital Care in England: Implications for Public Reporting of Health Care Quality Data in the United States. Jt. Comm. J. Q. Patient Saf. 2013, 39, 7-AP4. [Google Scholar] [CrossRef]
- Hawkins, J.B.; Brownstein, J.S.; Tuli, G.; Runels, T.; Broecker, K.; Nsoesie, E.O.; McIver, D.J.; Rozenblum, R.; Wright, A.; Bourgeois, F.T.; et al. Measuring patient-perceived quality of care in US hospitals using Twitter. BMJ Q. Saf. 2016, 25, 404. [Google Scholar] [CrossRef][Green Version]
- Ranard, B.L.; Werner, R.M.; Antanavicius, T.; Schwartz, H.A.; Smith, R.J.; Meisel, Z.F.; Asch, D.A.; Ungar, L.H.; Merchant, R.M. Yelp Reviews of Hospital Care Can Supplement and Inform Traditional Surveys of The Patient Experience Of Care. Health Aff. 2016, 35, 697–705. [Google Scholar] [CrossRef] [PubMed]
- Zun, A.B.; Ibrahim, M.I.; Hamid, A.A. Level of satisfaction on service quality dimensions based on SERVQUAL model among patients attending 1 Malaysia clinic in Kota Bharu, Malaysia. Oman Med. J. 2018, 33, 416. [Google Scholar] [CrossRef]
- Shafiq, M.; Naeem, M.A.; Munawar, Z.; Fatima, I. Service Quality Assessment of Hospitals in Asian Context: An Empirical Evidence from Pakistan. INQUIRY J. Health Care Organ. Provis. Financ. 2017, 54, 46958017714664. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Alanazi, M.R.; Alamry, A.; Al-Surimi, K. Validation and adaptation of the hospital consumer assessment of healthcare providers and systems in Arabic context: Evidence from Saudi Arabia. J. Infect. Public Health 2017, 10, 861–865. [Google Scholar] [CrossRef] [PubMed]
- Westbrook, K.; Babakus, E.; Grant, C. Measuring Patient-Perceived Hospital Service Quality: Validity and Managerial Usefulness of HCAHPS Scales. Health Mark. Q. 2014, 31, 97–114. [Google Scholar] [CrossRef] [PubMed]
- Greaves, F.; Laverty, A.A.; Cano, D.R.; Moilanen, K.; Pulman, S.; Darzi, A.; Millett, C. Tweets about hospital quality: A mixed methods study. BMJ Q. Saf. 2014, 23, 838. [Google Scholar] [CrossRef][Green Version]
- Bjertnaes, O.; Iversen, H.H.; Skyrud, K.D.; Danielsen, K. The value of Facebook in nation-wide hospital quality assessment: A national mixed-methods study in Norway. BMJ Q. Saf 2019, bmjqs-2019-009456. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Verhoef, L.M.; Van de Belt, T.H.; Engelen, L.J.L.P.G.; Schoonhoven, L.; Kool, R.B. Social media and rating sites as tools to understanding quality of care: A scoping review. J. Med. Internet Res. 2014, 16, e56. [Google Scholar] [CrossRef]
- MCMC. Internet Users Survey 2020, “IUS 2020”; Malaysian Communications and Multimedia Commission: Cyberjaya, Malaysia, 2020; p. 160. [Google Scholar]
- Lagu, T.; Greaves, F. From Public to Social Reporting of Hospital Quality. J. Gen. Intern. Med. 2015, 30, 1397–1399. [Google Scholar] [CrossRef][Green Version]
- Campbell, L.; Li, Y. Are Facebook user ratings associated with hospital cost, quality and patient satisfaction? A cross-sectional analysis of hospitals in New York State. BMJ Q. Saf. 2018, 27, 119. [Google Scholar] [CrossRef] [PubMed]
- Gaudet Hefele, J.; Li, Y.; Campbell, L.; Barooah, A.; Wang, J. Nursing home Facebook reviews: Who has them, and how do they relate to other measures of quality and experience? BMJ Q. Saf. 2018, 27, 130. [Google Scholar] [CrossRef] [PubMed]
- Li, Y.; Cai, X.; Wang, M. Social media ratings of nursing homes associated with experience of care and “Nursing Home Compare” quality measures. BMC Health Serv. Res. 2019, 19, 260. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Boylan, A.-M.; Turk, A.; van Velthoven, M.H.; Powell, J. Online patient feedback as a measure of quality in primary care: A multimethod study using correlation and qualitative analysis. BMJ Open 2020, 10, e031820. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Lee, H.J.; Lee, M.; Lee, H. Tracking Social Perception on Healthcare Service Quality Using Social Media. In Proceedings of the Management Knowledge and Learning International Conference 2018, Naples, Italy, 20–23 June 2018; p. 18. [Google Scholar]
- Lin, Y.; Hong, Y.A.; Henson, B.S.; Stevenson, R.D.; Hong, S.; Lyu, T.; Liang, C. Assessing Patient Experience and Healthcare Quality of Dental Care Using Patient Online Reviews in the United States: Mixed Methods Study. J. Med. Internet Res. 2020, 22, e18652. [Google Scholar] [CrossRef]
- Khanbhai, M.; Anyadi, P.; Symons, J.; Flott, K.; Darzi, A.; Mayer, E. Applying natural language processing and machine learning techniques to patient experience feedback: A systematic review. BMJ Health Care Inform. 2021, 28, e100262. [Google Scholar] [CrossRef]
- Gohil, S.; Vuik, S.; Darzi, A. Sentiment Analysis of Health Care Tweets: Review of the Methods Used. JMIR Public Health Surveill. 2018, 4, e43. [Google Scholar] [CrossRef]
- Nawab, K.; Ramsey, G.; Schreiber, R. Natural Language Processing to Extract Meaningful Information from Patient Experience Feedback. Appl. Clin. Inform. 2020, 11, 242–252. [Google Scholar] [CrossRef]
- Zunic, A.; Corcoran, P.; Spasic, I. Sentiment Analysis in Health and Well-Being: Systematic Review. JMIR Med. Inform. 2020, 8, e16023. [Google Scholar] [CrossRef] [PubMed]
- Farsi, D. Social Media and Health Care, Part I: Literature Review of Social Media Use by Health Care Providers. J. Med. Internet Res. 2021, 23, e23205. [Google Scholar] [CrossRef] [PubMed]
- Giustini, D.; Ali, S.M.; Fraser, M.; Kamel Boulos, M.N. Effective uses of social media in public health and medicine: A systematic review of systematic reviews. Online J. Public Health Inform. 2018, 10, e215. [Google Scholar] [CrossRef][Green Version]
- Placona, A.M.; Rathert, C. Are Online Patient Reviews Associated With Health Care Outcomes? A Systematic Review of the Literature. Med. Care Res. Rev. 2021, 10775587211014534. [Google Scholar] [CrossRef]
- Chakraborty, S.; Church, E.M. Social media hospital ratings and HCAHPS survey scores. J. Health Organ. Manag. 2020, 34, 162–172. [Google Scholar] [CrossRef] [PubMed]
- Richter, J.P.; Kazley, A.S. Social media: How hospital facebook activity may influence patient satisfaction. Health Mark. Q. 2020, 37, 447–460. [Google Scholar] [CrossRef]
- Synan, L.; Eid, M.; Lamb, C.; Wong, S. Crowd-sourced hospital ratings are correlated with patient satisfaction but not surgical safety. Surgery 2021, 170, 764–768. [Google Scholar] [CrossRef]
- Rahim, A.I.; Ibrahim, M.I.; Musa, K.I.; Chua, S.-L. Facebook Reviews as a Supplemental Tool for Hospital Patient Satisfaction and Its Relationship with Hospital Accreditation in Malaysia. Int. J. Environ. Res. Public Health 2021, 18, 7454. [Google Scholar] [CrossRef]
- Lee, J.Y.; Gowen, C.R.; McFadden, K.L. An empirical study of U.S. hospital quality: Readmission rates, organizational culture, patient satisfaction, and Facebook ratings. Q. Manag. J. 2018, 25, 158–170. [Google Scholar] [CrossRef]
- Ladhari, R. A review of twenty years of SERVQUAL research. Int. J. Q. Serv. Sci. 2009, 1, 172–198. [Google Scholar] [CrossRef]
- Parasuraman, A.; Zeithaml, V.A.; Berry, L.L. A Conceptual Model of Service Quality and Its Implications for Future Research. J. Mark. 1985, 49, 41–50. [Google Scholar] [CrossRef]
- Parasuraman, A.P.; Zeithaml, V.; Berry, L. SERVQUAL: A multiple- Item Scale for measuring consumer perceptions of service quality. J. Retail. 1988, 64, 12–40. [Google Scholar]
- Tan, C.N.-L.; Ojo, A.O.; Cheah, J.-H.; Ramayah, T. Measuring the Influence of Service Quality on Patient Satisfaction in Malaysia. Q. Manag. J. 2019, 26, 129–143. [Google Scholar] [CrossRef]
- Muhammad Butt, M. Private healthcare quality: Applying a SERVQUAL model. Int. J. Health Care Q. Assur. 2010, 23, 658–673. [Google Scholar] [CrossRef] [PubMed]
- John, J.; Yatim, F.M.; Mani, S.A. Measuring service quality of public dental health care facilities in Kelantan, Malaysia. Asia-Pacific J. Public Health 2011, 23, 742–753. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Geletta, S. Measuring patient satisfaction with medical services using social media generated data. Int. J. Health Care Q. Assur. 2018, 31, 96–105. [Google Scholar] [CrossRef] [PubMed]
- Hu, G.; Han, X.; Zhou, H.; Liu, Y. Public Perception on Healthcare Services: Evidence from Social Media Platforms in China. Int. J. Environ. Res. Public Health 2019, 16, 1273. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Greaves, F.; Ramirez-Cano, D.; Millett, C.; Darzi, A.; Donaldson, L. Use of Sentiment Analysis for Capturing Patient Experience from Free-Text Comments Posted Online. J. Med. Internet Res. 2013, 15, e239. [Google Scholar] [CrossRef] [PubMed]
- Greaves, F.; Ramirez-Cano, D.; Millett, C.; Darzi, A.; Donaldson, L. Harnessing the cloud of patient experience: Using social media to detect poor quality healthcare. BMJ Q. Saf. 2013, 22, 251–255. [Google Scholar] [CrossRef]
- Friedrich, S.; Groß, S.; König, I.R.; Engelhardt, S.; Bahls, M.; Heinz, J.; Huber, C.; Kaderali, L.; Kelm, M.; Leha, A.; et al. Applications of artificial intelligence/machine learning approaches in cardiovascular medicine: A systematic review with recommendations. Eur. Heart J.-Digit. Health 2021. [Google Scholar] [CrossRef]
- Alemi, F.; Torii, M.; Clementz, L.; Aron, D.C. Feasibility of real-time satisfaction surveys through automated analysis of patients’ unstructured comments and sentiments. Q. Manag. Health Care 2012, 21, 9–19. [Google Scholar] [CrossRef][Green Version]
- Doing-Harris, K.; Mowery, D.L.; Daniels, C.; Chapman, W.W.; Conway, M. Understanding patient satisfaction with received healthcare services: A natural language processing approach. AMIA Annu. Symp. Proc. 2017, 2016, 524–533. [Google Scholar]
- Jiménez-Zafra, S.M.; Martín-Valdivia, M.T.; Maks, I.; Izquierdo, R. Analysis of patient satisfaction in Dutch and Spanish online reviews. J. Proces. Leng. Nat. 2017, 58, 101–108. [Google Scholar]
- Zaman, N.; Goldberg, D.M.; Abrahams, A.S.; Essig, R.A. Facebook Hospital Reviews: Automated Service Quality Detection and Relationships with Patient Satisfaction. Decis. Sci. 2021. [Google Scholar] [CrossRef]
- Abirami, A.M.; Askarunisa, A. Sentiment analysis model to emphasize the impact of online reviews in Healthcare industry. Online Inf. Rev. 2017, 41, 471–486. [Google Scholar] [CrossRef]
- Daniulaityte, R.; Chen, L.; Lamy, F.R.; Carlson, R.G.; Thirunarayan, K.; Sheth, A. “When ‘Bad’ is ‘Good’”: Identifying Personal Communication and Sentiment in Drug-Related Tweets. JMIR Public Health Surveill. 2016, 2, e162. [Google Scholar] [CrossRef][Green Version]
- Cole-Lewis, H.; Varghese, A.; Sanders, A.; Schwarz, M.; Pugatch, J.; Augustson, E. Assessing Electronic Cigarette-Related Tweets for Sentiment and Content Using Supervised Machine Learning. J. Med. Internet Res. 2015, 17, e208. [Google Scholar] [CrossRef] [PubMed]
- Jung, Y.; Hur, C.; Jung, D.; Kim, M. Identifying Key Hospital Service Quality Factors in Online Health Communities. J. Med. Internet Res. 2015, 17, e90. [Google Scholar] [CrossRef] [PubMed]
- James, T.L.; Villacis Calderon, E.D.; Cook, D.F. Exploring patient perceptions of healthcare service quality through analysis of unstructured feedback. Expert Syst. Appl. 2017, 71, 479–492. [Google Scholar] [CrossRef]
- Ko, D.-G.; Mai, F.; Shan, J.; Zhang, D. Operational efficiency and patient-centered health care: A view from online physician reviews. J. Oper. Manag. 2019, 65, 353–379. [Google Scholar] [CrossRef]
- Hao, H.; Zhang, K.; Wang, W.; Gao, G. A tale of two countries: International comparison of online doctor reviews between China and the United States. Int. J. Med. Inform. 2017, 99, 37–44. [Google Scholar] [CrossRef]
- Bahja, M.; Lycett, M. Identifying patient experience from online resources via sentiment analysis and topic modelling. In Proceedings of the 3rd IEEE/ACM International Conference on Big Data Computing, Applications and Technologies, Shanghai, China, 6–9 December 2016; pp. 94–99. [Google Scholar]
- Hao, H.; Zhang, K. The Voice of Chinese Health Consumers: A Text Mining Approach to Web-Based Physician Reviews. J. Med. Internet Res. 2016, 18, e108. [Google Scholar] [CrossRef]
- Lee, H.J.; Lee, M.; Lee, H.; Cruz, R. Mining service quality feedback from social media: A computational analytics method. Gov. Inf. Q. 2021, 38, 101571. [Google Scholar] [CrossRef]
- Rastegar-Mojarad, M.; Ye, Z.; Wall, D.; Murali, N.; Lin, S. Collecting and Analyzing Patient Experiences of Health Care from Social Media. JMIR Res. Protoc. 2015, 4, e78. [Google Scholar] [CrossRef][Green Version]
- Yang, M.; Kiang, M.; Shang, W. Filtering big data from social media—Building an early warning system for adverse drug reactions. J. Biomed. Inform. 2015, 54, 230–240. [Google Scholar] [CrossRef][Green Version]
- Wallace, B.C.; Paul, M.J.; Sarkar, U.; Trikalinos, T.A.; Dredze, M. A large-scale quantitative analysis of latent factors and sentiment in online doctor reviews. J. Am. Med. Inform. Assoc. 2014, 21, 1098–1103. [Google Scholar] [CrossRef]
- NHS. The Power of Information: Putting All of Us in Control of the Health and Care Information We Need. Available online: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/213689/dh_134205.pdf (accessed on 10 November 2020).
- Huppertz, J.W.; Otto, P. Predicting HCAHPS scores from hospitals’ social media pages: A sentiment analysis. Health Care Manag. Rev. 2018, 43, 359–367. [Google Scholar] [CrossRef]
- Alkazemi, M.F.; Bayramzadeh, S.; Alkhubaizi, N.B.; Alayoub, A. The physical environment and patient satisfaction ratings on social media: An exploratory study. Facilities 2020, 38, 86–97. [Google Scholar] [CrossRef]
- Boylan, A.M.; Williams, V.; Powell, J. Online patient feedback: A scoping review and stakeholder consultation to guide health policy. J. Health Serv. Res. Policy 2020, 25, 122–129. [Google Scholar] [CrossRef] [PubMed]
- Oh, M.; Kim, S. Role of Emotions in Fine Dining Restaurant Online Reviews: The Applications of Semantic Network Analysis and a Machine Learning Algorithm. Int. J. Hosp. Tour. Adm. 2021, 1–29. [Google Scholar] [CrossRef]
- Kaspa, L.P.; Akella, V.N.S.S.; Chen, Z.; Shi, Y. Towards Extended Data Mining: An Examination of Technical Aspects. Procedia Comput. Sci. 2018, 139, 49–55. [Google Scholar] [CrossRef]
- Moore, K.; Cottrell, E.; Chambers, R. Facebook in general practice: A service evaluation in one health economy. BJGP Open 2017, 1, bjgpopen17X101181. [Google Scholar] [CrossRef] [PubMed]
- Bari, V.; Hirsch, J.S.; Narvaez, J.; Sardinia, R.; Bock, K.R.; Oppenheim, M.I.; Meytlis, M. An approach to predicting patient experience through machine learning and social network analysis. J. Am. Med. Inform. Assoc. 2020, 27, 1834–1843. [Google Scholar] [CrossRef]
- Szymański, P.; Kajdanowicz, T. A Network Perspective on Stratification of Multi-Label Data. In Proceedings of the First International Workshop on Learning with Imbalanced Domains: Theory and Applications, Skopje, Macedonia, 22 September 2017; pp. 22–35. [Google Scholar]
- Wagland, R.; Recio-Saucedo, A.; Simon, M.; Bracher, M.; Hunt, K.; Foster, C.; Downing, A.; Glaser, A.; Corner, J. Development and testing of a text-mining approach to analyse patients’ comments on their experiences of colorectal cancer care. BMJ Q. Saf. 2016, 25, 604–614. [Google Scholar] [CrossRef]
- Yang, P.-C.; Lee, W.-C.; Liu, H.-Y.; Shih, M.-J.; Chen, T.-J.; Chou, L.-F.; Hwang, S.-J. Use of Facebook by Hospitals in Taiwan: A Nationwide Survey. Int. J. Environ. Res. Public Health 2018, 15, 1188. [Google Scholar] [CrossRef][Green Version]
- Sewalk, K.C.; Tuli, G.; Hswen, Y.; Brownstein, J.S.; Hawkins, J.B. Using Twitter to Examine Web-Based Patient Experience Sentiments in the United States: Longitudinal Study. J. Med. Internet Res. 2018, 20, e10043. [Google Scholar] [CrossRef][Green Version]
- Ham, C.; Berwick, D.; Dixon, J. Improving Quality in the English NHS; The King’s Fund: London, UK, 2016. [Google Scholar]
- Huesch, M.D.; Currid-Halkett, E.; Doctor, J.N. Public hospital quality report awareness: Evidence from National and Californian Internet searches and social media mentions, 2012. BMJ Open 2014, 4, e004417. [Google Scholar] [CrossRef][Green Version]
- Moorhead, S.A.; Hazlett, D.E.; Harrison, L.; Carroll, J.K.; Irwin, A.; Hoving, C. A new dimension of health care: Systematic review of the uses, benefits, and limitations of social media for health communication. J. Med. Internet Res. 2013, 15, e85. [Google Scholar] [CrossRef][Green Version]
- Timian, A.; Rupcic, S.; Kachnowski, S.; Luisi, P. Do Patients “Like” Good Care? Measuring Hospital Quality via Facebook. Am. J. Med. Q. 2013, 28, 374–382. [Google Scholar] [CrossRef]
- MSQH. Performance Indicators: Hospital Accreditation Standards, 5th ed.; Malaysian Society for Quality in Health: Kuala Lumpur, Malaysia, 2017. [Google Scholar]
- Kherwa, P.; Bansal, P. Topic Modeling: A Comprehensive Review. ICST Trans. Scalable Inf. Syst. 2018, 7, 159623. [Google Scholar] [CrossRef][Green Version]
- Kalaivaani, P.C.D.; Thangarajan, R. Enhancing the Classification Accuracy in Sentiment Analysis with Computational Intelligence using Joint Sentiment Topic Detection with MEDLDA. Intell. Autom. Soft Comput. 2019, 26, 71–79. [Google Scholar] [CrossRef]
- Singh, S.K.; Azzaoui, A.E.; Kim, T.W.; Pan, Y.; Park, J.H. DeepBlockScheme: A deep learning-based blockchain driven scheme for secure smart city. Hum.-Centric Comput. Inf. Sci. 2021, 11, 12. [Google Scholar]
- Wu, S.; Liu, Y.; Wang, J.; Li, Q. Sentiment Analysis Method Based on Kmeans and Online Transfer Learning. Comput. Mater. Contin. 2019, 60, 1207–1222. [Google Scholar] [CrossRef][Green Version]
- Feng, A.; Zhengjie, G.; Song, X.; Ke, K.; Xu, T.; Zhang, X. Modeling Multi-Targets Sentiment Classification via Graph Convolutional Networks and Auxiliary Relation. Comput. Mater. Contin. 2020, 64, 909–923. [Google Scholar] [CrossRef]
- Singh, S.K.; Sharma, P.K.; Pan, Y.; Park, J.H. BIIoVT: Blockchain-based Secure Storage Architecture for Intelligent Internet of Vehicular Things. IEEE Consum. Electron. Mag. 2021. [Google Scholar] [CrossRef]
|Topic Classification||Sentiment Analysis|
|Study||Data Source||Population of Study||Number of|
|Supervised||Non-Supervised||Topics/Themes||Supervised||Non-Supervised||Other Tool||Associations. *|
|Lee et al, (2021) ||UK||50,716||X||5||X||X|
|Zaman et al, (2021) ||USA||6581||X||7||X||X|
|Boylan et al (2020) ||NHS Choices||UK||1396||3||NVivo||X|
|Lin et al (2020) ||Health Grades||USA||204,751||X||17||X|
|Nawab et al, (2020) ||Press Ganey||USA||2830||X||13||Keras|
|Hu et al (2019) ||WeChat,|
|Ko et al (2019) ||Vitals||USA||1,560,639||X||5|
|Huppertz & Otto (2018) ||USA||57,985||X||X|
|Abirami & Askarunisa, (2017) ||Multiple sources including Facebook, Twitter etc.||India||1941||X||5||X||X|
|Doing- Harris et al (2017) ||Press Ganey||USA||51,235||X||X||7/30||X|
|Jimenez- Zafra et al (2017) ||Zorgkaart Nederland,|
|156,975 of COPOD & 743 of COPOS||X|
|James et al (2017) ||RateMDs||USA||3712||X||6||Diction|
|Hao et al (2017) ||RateMDs,|
|156,558 of RateMD,|
57,342 of Haodf
|Ranard el al (2016) ||Yelp||USA||16,862||X||50||X||X|
|Bahja & Lycett (2016) ||NHS Choice||UK||76,151||X||30||X|
|Daniulaityte et al (2016) ||USA||4000||X||3||X|
|Hao & Zhang (2016) ||Haodf||China||731,264||X||10|
|Hawkin et al (2016) ||USA||11,602||X||10||TextBlob||X|
|Cole-Lewis et al (2015) ||USA||17,098||X||10||X|
|Jung et al (2015) ||Naver & Daum Web||South|
|Rastegar-Mojarad et al (2015) ||Yelp||USA||6914||X*||X*||20||SentiWordNet|
|Yang et al (2015) ||MedHelp||USA||3000||X*||X*||10||X|
|Greaves et al (2014) ||UK||1000||X||6||TheySay||X|
|Wallace et al (2014) ||RateMDs||USA||58,110||X*||X*||3||X||X|
|Greaves et al (2013) ||NHS Choice||UK||6412||X||3||X||X|
|Alemi et al (2012) ||RateMDs||USA||955||X||9||X|
|Proposed Work||Justification||Comparison Studies|
|Facebook as Data Source||Limited studies utilized|
Facebook data. Yet, Facebook is popular among patients and healthcare providers in
|Studies that used Facebook data including Zaman et al. (2021) , Huppertz & Otto (2018) , and Abirami & Askarunisa, (2017) |
|Asian as Study |
|Limited studies among Asian population||Chinese study by Hu et al. (2019) , Hao et al. (2017)  and Hao & Zhang (2016) , Indian study by Abirami & Askarunisa, (2017) , and Korean study by Jung et al. (2015) .|
|Supervised learning via |
remains the ‘gold standard’ method for analyzing free text comments for patient online reviews.
|Zaman et al. (2021), Abirami & Askarunisa, (2017), Daniulaityte et al. (2016) , Cole-Lewis et al. (2015) , Jung et al. (2015), Greaves et al. (2013) , and Alemi et al. (2012)  employed supervised learning for both topic and sentiment classifications.|
|SERVQUAL||Domains of a traditional|
survey of patient experiences (SERVQUAL) serve as a foundation for our ML topic classifier.
|SERVQUAL by Lee et al. (2021) ,|
CAHPS Dental Plan Survey by Lin et al. (2020) , and HCAHPS by Ranard et al. (2016) .
|Most patient online review studies were descriptive. Hence, we aim to test the|
associations using advanced statistical analysis.
|ANOVA by Lin et al. (2020), regression analysis by Zaman et al. (2021), Ko et al. (2019) , Huppertz & Otto (2018), James et al. (2017) , Wallace et al. (2014)  and Hawkin et al. (2016) , Pearson Correlation by Abirami & Askarunisa, (2017) and Ranald et al. (2017), Spearman’s rank correlation by Boylan et al. (2020) , Abirami & Askarunisa, (2017) and Greaves et al. (2014) .|
|Comparison with health care quality measures||Only a few studies compared standard health care quality measures such as HCAHPS, SERVQUAL, hospital|
accreditation or national quality indicators, etc.
|GPPS and the FFT by Boylan et al. (2020), CAHPS Dental Plan Survey by Lin et al. (2020), HCAHPS survey by Zaman et al. (2021), Ranard et al. (2016), Huppertz & Otto (2018), and Hawkin et al. (2016), hospital ranking by Abirami & Askarunisa, (2017) and NHS inpatient survey by Greaves et al. (2014) and Greaves et al. (2013).|
|Multilabel Classifier||Model||Accuracy||Recall||Precision||F1-Score||Hamming Loss|
|Beds (Median, IQR)||730||(604)||704||(563)|
|Adequate Hospital Information||No||35||(6.9)||76||(5.8)|
|Variables||Crude||95% CI||p-Value *|
|East Coast||1.80||1.14, 2.86||0.012|
|Location of Hospital||Rural||Ref|
|Type of Hospital||Primary||Ref|
|Numbers of Bed||1.00||1.00, 1.00||0.017|
|Admin Response to Review||No||Ref|
|Adequate Hosp Info||No||Ref|
|Previous Facebook Star Ratings||1.09||1.01, 1.17||0.033|
|Variable||Adjusted OR||95% CI|
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).