Research on the Characteristics and Usefulness of User Reviews of Online Mental Health Consultation Services: A Content Analysis
Abstract
:1. Introduction
1.1. Background
1.2. Literature Review
1.2.1. Online Psychological Consultation
1.2.2. Usefulness of User Reviews
2. Materials and Methods
2.1. Data Collection
2.2. Topic Analysis
2.3. Sentiment Analysis
3. Results
3.1. Topic Analysis Results
3.2. Sentiment Analysis Results
3.3. Variable Descriptive Statistical Analysis
3.4. Regression Analysis Results
4. Discussion
4.1. Principal Findings
4.2. Theoretical Contributions and Practical Guidance
4.3. Limitations and Future Directions
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Conflicts of Interest
References
- Barak, A.; Klein, B.; Proudfoot, J.G. Defining internet-supported therapeutic interventions. Ann. Behav. Med. 2009, 38, 4–17. [Google Scholar] [CrossRef] [PubMed]
- Ramaswamy, A.; Yu, M.; Drangsholt, S.; Ng, E.; Culligan, P.J.; Schlegel, P.N.; Hu, J.C. Patient Satisfaction With Telemedicine During the COVID-19 Pandemic: Retrospective Cohort Study. J. Med. Internet Res. 2020, 22, e20786. [Google Scholar] [CrossRef] [PubMed]
- King, R.; Bambling, M.; Lloyd, C.; Gomurra, R.; Smith, S.; Reid, W.; Wegner, K. Online counselling: The motives and experiences of young people who choose the Internet instead of face to face or telephone counselling. Couns. Psychother. Res. 2006, 6, 169–174. [Google Scholar] [CrossRef]
- Chan, J.K.; Farrer, L.M.; Gulliver, A.; Bennett, K.; Griffiths, K.M. University Students’ Views on the Perceived Benefits and Drawbacks of Seeking Help for Mental Health Problems on the Internet: A Qualitative Study. JMIR Hum. Factors 2016, 3, e3. [Google Scholar] [CrossRef]
- Paul, C.L.; Boyes, A.W.; O’Brien, L.; Baker, A.L.; Henskens, F.A.; Roos, I.; Clinton-McHarg, T.; Bellamy, D.; Colburn, G.; Rose, S.; et al. Protocol for a Randomized Controlled Trial of Proactive Web-Based Versus Telephone-Based Information and Support: Can Electronic Platforms Deliver Effective Care for Lung Cancer Patients? JMIR Res. Protoc. 2016, 5, e202. [Google Scholar] [CrossRef]
- Weiss, M.; Hildebrand, A.; Braun-Scharm, H.; Weckwerth, K.; Held, D.; Stemmler, M. Are suicidal young people reached by online-counselling? Evaluation of the target group outreach of [U25] online suicide prevention. Z. Kinder Jugendpsychiatrie Psychother. 2020, 48, 204–214. [Google Scholar] [CrossRef]
- Huang, H.; Yan, X.; Yu, L.; Yu, Z. Relation of attitude towards online counseling to personality traits andinternet self-efficacy in college students. Chin. Ment. Health J. 2013, 27, 299–304. [Google Scholar]
- McMillan, S.J.; Hwang, J.S. Measures of perceived interactivity: An exploration of the role of direction of communication, user control, and time in shaping perceptions of interactivity. J. Advert. 2002, 31, 29–42. [Google Scholar] [CrossRef]
- Mayer, G.; Gronewold, N.; Alvarez, S.; Bruns, B.; Hilbel, T.; Schultz, J.H. Acceptance and Expectations of Medical Experts, Students, and Patients Toward Electronic Mental Health Apps: Cross-Sectional Quantitative and Qualitative Survey Study. JMIR Ment. Health 2019, 6, e14018. [Google Scholar] [CrossRef]
- March, S.; Day, J.; Ritchie, G.; Rowe, A.; Gough, J.; Hall, T.; Yuen, C.Y.J.; Donovan, C.L.; Ireland, M. Attitudes Toward e-Mental Health Services in a Community Sample of Adults: Online Survey. J. Med. Internet Res. 2018, 20, e59. [Google Scholar] [CrossRef]
- Bird, M.D.; Chow, G.M.; Yang, Y. College students’ attitudes, stigma, and intentions toward seeking online and face-to-face counseling. J. Clin. Psychol. 2020, 76, 1775–1790. [Google Scholar] [CrossRef] [Green Version]
- Baumel, A.; Schueller, S.M. Adjusting an Available Online Peer Support Platform in a Program to Supplement the Treatment of Perinatal Depression and Anxiety. JMIR Ment. Health 2016, 3, e11. [Google Scholar] [CrossRef]
- Navarro, P.; Bambling, M.; Sheffield, J.; Edirippulige, S. Exploring Young People’s Perceptions of the Effectiveness of Text-Based Online Counseling: Mixed Methods Pilot Study. JMIR Ment. Health 2019, 6, e13152. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Pretorius, C.; Chambers, D.; Cowan, B.; Coyle, D. Young People Seeking Help Online for Mental Health: Cross-Sectional Survey Study. JMIR Ment. Health 2019, 6, e13524. [Google Scholar] [CrossRef] [PubMed]
- Hao, H.; Zhang, K. The Voice of Chinese Health Consumers: A Text Mining Approach to Web-Based Physician Reviews. J. Med. Internet Res. 2016, 18, e108. [Google Scholar] [CrossRef] [PubMed]
- Hong, Y.A.; Liang, C.; Radcliff, T.A.; Wigfall, L.T.; Street, R.L. What Do Patients Say About Doctors Online? A Systematic Review of Studies on Patient Online Reviews. J. Med. Internet Res. 2019, 21, e12521. [Google Scholar] [CrossRef] [Green Version]
- Ghose, A.; Ipeirotis, P.G. Designing novel review ranking systems: Predicting the usefulness and impact of reviews. In Proceedings of the 9th International Conference on Electronic Commerce, Minneapolis, MN, USA, 19–22 August 2007; pp. 303–310. [Google Scholar]
- Zhao, K.; Yang, X.; Tao, X.; Xu, X.; Zhao, J. Exploring the Differential Effects of Online Reviews on Film’s Box-Office Success: Source Identity and Brand Equity From an Integrated Perspective. Front. Psychol. 2020, 11, 217. [Google Scholar] [CrossRef]
- Fang, B.; Ye, Q.; Kucukusta, D.; Law, R. Analysis of the perceived value of online tourism reviews: Influence of readability and reviewer characteristics. Tour. Manag. 2016, 52, 498–506. [Google Scholar] [CrossRef]
- Chen, S.; Lv, X.; Gou, J. Personalized Recommendation Model: An Online Comment Sentiment Based Analysis. Int. J. Comput. Commun. 2020, 15. [Google Scholar] [CrossRef] [Green Version]
- Paulsell, D.; Thomas, J.; Monahan, S.; Seftor, N.S. A Trusted Source of Information: How Systematic Reviews Can Support User Decisions About Adopting Evidence-Based Programs. Eval. Rev. 2017, 41, 50–77. [Google Scholar] [CrossRef]
- Salehan, M.; Kim, D.J. Predicting the performance of online consumer reviews: A sentiment mining approach to big data analytics. Decis. Support Syst. 2016, 81, 30–40. [Google Scholar] [CrossRef]
- Hong, H.; Xu, D.; Wang, G.A.; Fan, W.G. Understanding the determinants of online review helpfulness: A meta-analytic investigation. Decis. Support Syst. 2017, 102, 1–11. [Google Scholar] [CrossRef]
- Mallen, M.J.; Vogel, D.L.; Rochlen, A.B.; Day, S.X. Online counseling: Reviewing the literature from a counseling psychology framework. Couns. Psychol. 2005, 33, 819–871. [Google Scholar] [CrossRef]
- Grohol, J.M. The Insider’s Guide to Mental Health Resources Online. Cyberpsychol. Behav. 1999, 2, 173. [Google Scholar] [CrossRef]
- Wetterlin, F.M.; Mar, M.Y.; Neilson, E.K.; Werker, G.R.; Krausz, M. eMental health experiences and expectations: A survey of youths’ Web-based resource preferences in Canada. J. Med. Internet Res. 2014, 16, e293. [Google Scholar] [CrossRef] [Green Version]
- Kauer, S.D.; Mangan, C.; Sanci, L. Do online mental health services improve help-seeking for young people? A systematic review. J. Med. Internet Res. 2014, 16, e66. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Hewitt, H.; Gafaranga, J.; McKinstry, B. Comparison of face-to-face and telephone consultations in primary care: Qualitative analysis. Br. J. Gen. Pract. 2010, 60, e201–e212. [Google Scholar] [CrossRef] [Green Version]
- Lal, S.; Adair, C.E. E-mental health: A rapid review of the literature. Psychiatr. Serv. 2014, 65, 24–32. [Google Scholar] [CrossRef] [PubMed]
- Im, S.; Lee, J.; Han, S. Video-counseling: Needs Assessment and Perception of Service Utilization. Korean J. Stress Res. 2017, 25, 57–67. [Google Scholar] [CrossRef] [Green Version]
- Serhal, E.; Kirvan, A.; Sanches, M.; Crawford, A. Client Satisfaction and Experience With Telepsychiatry: Development and Validation of a Survey Using Clinical Quality Domains. J. Med. Internet Res. 2020, 22. [Google Scholar] [CrossRef]
- Chan, M.; Li, X.Q. Smartphones and psychological well-being in China: Examining direct and indirect relationships through social support and relationship satisfaction. Telemat. Inform. 2020, 54. [Google Scholar] [CrossRef]
- Schindler, R.M.; Bickart, B. Perceived helpfulness of online consumer reviews: The role of message content and style. J. Consum. Behav. 2012, 11, 234–243. [Google Scholar] [CrossRef] [Green Version]
- Mudambi, S.M.; Schuff, D. What Makes a Helpful Online Review? A Study of Customer Reviews on Amazon.com. MIS Q. 2010, 34, 185–200. [Google Scholar] [CrossRef] [Green Version]
- Hao, H. The development of online doctor reviews in China: An analysis of the largest online doctor review website in China. J. Med. Internet Res. 2015, 17, e134. [Google Scholar] [CrossRef] [PubMed]
- Ghose, A.; Ipeirotis, P.G. Estimating the Helpfulness and Economic Impact of Product Reviews: Mining Text and Reviewer Characteristics. IEEE Trans. Knowl. Data Eng. 2011, 23, 1498–1512. [Google Scholar] [CrossRef] [Green Version]
- Laczniak, R.N.; DeCarlo, T.E.; Ramaswami, S.N. Consumers’ Responses to Negative Word-of-Mouth Communication: An Attribution Theory Perspective. J. Consum. Psychol. 2001, 11, 57–73. [Google Scholar] [CrossRef]
- Forman, C.; Ghose, A.; Wiesenfeld, B. Examining the relationship between reviews and sales: The role of reviewer identity disclosure in electronic markets. Inform. Syst. Res. 2008, 19, 291–313. [Google Scholar] [CrossRef]
- Korfiatis, N.; Garcia-Bariocanal, E.; Sanchez-Alonso, S. Evaluating content quality and helpfulness of online product reviews: The interplay of review helpfulness vs. review content. Electron. Commer. Res. Appl. 2012, 11, 205–217. [Google Scholar] [CrossRef]
- Lee, S.W.; Jiang, G.B.; Kong, H.Y.; Liu, C. A difference of multimedia consumer’s rating and review through sentiment analysis. Multimed. Tools Appl. 2020. [Google Scholar] [CrossRef]
- Baek, H.; Ahn, J.; Choi, Y. Helpfulness of Online Consumer Reviews: Readers’ Objectives and Review Cues. Int. J. Electron. Commer. 2014, 17, 99–126. [Google Scholar] [CrossRef]
- Yin, D.; Bond, S.D.; Zhang, H. Anxious or Angry? Effects of Discrete Emotions on the Perceived Helpfulness of Online Reviews. MIS Q. 2014, 38, 539–560. [Google Scholar] [CrossRef]
- Hu, Y.-H.; Chen, K. Predicting hotel review helpfulness: The impact of review visibility, and interaction between hotel stars and review ratings. Int. J. Inf. Manag. 2016, 36, 929–944. [Google Scholar] [CrossRef]
- Zhou, Y.; Yang, S.; Li, Y.; Chen, Y.; Yao, J.; Qazi, A. Does the review deserve more helpfulness when its title resembles the content? Locating helpful reviews by text mining. Inform. Process. Manag. 2020, 57. [Google Scholar] [CrossRef]
- Chen, M.-J.; Farn, C.-K. Examining the Influence of Emotional Expressions in Online Consumer Reviews on Perceived Helpfulness. Inform. Process. Manag. 2020, 57. [Google Scholar] [CrossRef]
- Evans, A.M.; Stavrova, O.; Rosenbusch, H. Expressions of doubt and trust in online user reviews. Comput. Hum. Behav. 2021, 114. [Google Scholar] [CrossRef]
- Alodadi, N.; Zhou, L.N. Predicting the Helpfulness of Online Physician Reviews. In Proceedings of the 2016 IEEE International Conference on Healthcare Informatics (ICHI), Chicago, IL, USA, 4–7 October 2016; pp. 1–6. [Google Scholar] [CrossRef]
- Berkovic, D.; Ackerman, I.N.; Briggs, A.M.; Ayton, D. Tweets by People With Arthritis During the COVID-19 Pandemic: Content and Sentiment Analysis. J. Med. Internet Res. 2020, 22, e24550. [Google Scholar] [CrossRef] [PubMed]
- Hu, N.; Koh, N.S.; Reddy, S.K. Ratings lead you to the product, reviews help you clinch it? The mediating role of online review sentiments on product sales. Decis. Support. Syst. 2014, 57, 42–53. [Google Scholar] [CrossRef]
- Duan, W.; Cao, Q.; Gan, Q. Investigating determinants of voting for the “helpfulness” of online consumer reviews: A text mining approach. AMCIS 2010, 2, 1304–1313. [Google Scholar]
- Zhou, J.; Zuo, M.; Ye, C. Understanding the factors influencing health professionals’ online voluntary behaviors: Evidence from YiXinLi, a Chinese online health community for mental health. Int. J. Med. Inform. 2019, 130, 103939. [Google Scholar] [CrossRef] [PubMed]
- Wang, Y.; Wang, J.; Yao, T.; Li, M. What makes peer review helpfulness evaluation in online review communities? An empirical research based on persuasion effect. Online Inf. Rev. 2020, 44, 1267–1286. [Google Scholar] [CrossRef]
- Blei, D.M.; Ng, A.Y.; Jordan, M.I. Latent Dirichlet Allocation. J. Mach. Learn. Res. 2003, 3, 993–1022. [Google Scholar]
- Xie, R.; Chu, S.K.W.; Chiu, D.K.W.; Wang, Y. Exploring Public Response to COVID-19 on Weibo with LDA Topic Modeling and Sentiment Analysis. Data Inf. Manag. 2021, 5, 86–99. [Google Scholar] [CrossRef]
- Pengnate, S.; Riggins, F.J. The role of emotion in P2P microfinance funding: A sentiment analysis approach. Int. J. Inf. Manag. 2020, 54. [Google Scholar] [CrossRef]
- Geetha, M.; Singha, P.; Sinha, S. Relationship between customer sentiment and online customer ratings for hotels—An empirical analysis. Tour. Manag. 2017, 61, 43–54. [Google Scholar] [CrossRef]
- Mohammad, S.M.; Turney, P.D. Crowdsourcing a Word-Emotion Association Lexicon. Comput. Intell. 2013, 29, 436–465. [Google Scholar] [CrossRef] [Green Version]
- Malik, M.S.I.; Hussain, A. Helpfulness of product reviews as a function of discrete positive and negative emotions. Comput. Hum. Behav. 2017, 73, 290–302. [Google Scholar] [CrossRef] [Green Version]
- Lee, J. What makes people read an online review? The relative effects of posting time and helpfulness on review readership. Cyberpsychol. Behav. Soc. Netw. 2013, 16, 529–535. [Google Scholar] [CrossRef]
- Henseler, J.; Ringle, C.M.; Sinkovics, R.R. The use of partial least squares path modeling in international marketing. Adv. Int. Mark. 2009, 20, 277–319. [Google Scholar] [CrossRef] [Green Version]
Author | Theoretical Background | IV | DV | Data Source |
---|---|---|---|---|
Mudambi and Schuff (2010) [34] | Information economics theory | Review extremity Review depth | Review helpfulness | Amazon.com |
Baek et al. (2014) [41] | Dual process theories | Rating inconsistency Reviewer credibility Word count Negative word | Review helpfulness | Amazon.com |
Yin et al. (2014) [42] | Theories of emotion | Emotions (anxiety; anger) | Review helpfulness | --- |
Hu and Chen (2016) [43] | Review content analysis Review polarity | Review content Review sentiment Review author Review visibility | Review helpfulness | TripAdvisor.com |
Zhou et al. (2020) [44] | Mere exposure theory | Title-content similarity | Review helpfulness | Amazon.com |
Chen and Farn (2020) [45] | Attribution theory Cognitive appraisal theory | Negative emotions Positive emotions | Review helpfulness | --- |
Evans et al. (2021) [46] | Expressions of doubt and trust | Expressions of doubt | Review helpfulness | Yelp |
Pro | Quantity | Percent |
---|---|---|
0 | 28,769 | 87.12% |
1 | 2834 | 8.58% |
2 | 463 | 1.40% |
3 | 207 | 0.63% |
4 | 105 | 0.32% |
5 | 70 | 0.21% |
6–10 | 206 | 0.62% |
11–15 | 175 | 0.53% |
16–20 | 152 | 0.46% |
21–27 | 43 | 0.13% |
Sum | 33,024 | 100% |
Topic 1 | Topic 2 | Topic 3 | Topic 4 | Topic 5 | Topic 6 |
---|---|---|---|---|---|
gain | feel | thinking | patient | expect | satisfied |
method | emotions | ponder | comfortable | feel | thanks |
kids | unblock | clear | professional | first time | feeling |
grow up | self | relationship | analysis | hope | very good |
change | understand | guide | listen | next time | meticulous |
suggest | inner heart | cognition | help | help | objective |
face | accept | idea | talk | resolve | great |
life | walk out | explore | communicate | heart | warm |
confidence | fear | define | voice | truly | gentle |
love | subconscious | value | warm | demand | happy |
gain change 13.83% | ease emotions 13.44% | clear thinking 14.60% | patient listening 28.54% | consulting expectations 17.66% | consulting feelings 11.93% |
Variables | Description | Mean | SD | Min | Max |
---|---|---|---|---|---|
Dependent variable | |||||
Pro | Number of likes from other users | 2.983 | 4.486 | 1 | 27 |
Topic features | |||||
Topic 1 | Consultation effect and related topics | 0.138 | 0.178 | 0.0149 | 0.857 |
Topic 2 | Emotional acceptance and related topics | 0.134 | 0.172 | 0.0149 | 0.902 |
Topic 3 | Clear thinking and related topics | 0.146 | 0.192 | 0.0174 | 0.889 |
Topic 4 | Listening patiently and related topics | 0.285 | 0.268 | 0.0150 | 0.889 |
Topic 5 | Consultation expectations and related topics | 0.177 | 0.213 | 0.0159 | 0.893 |
Sentiment features | |||||
Scores | Sentiment score calculated by sentiment dictionary | 18.52 | 2.269 | 0 | 38 |
Polarity | Ratio of positive emotions to negative emotions | 1.339 | 1.591 | 0 | 17 |
Contextual features | |||||
Rating | Review rating (1–5) | 4.934 | 0.259 | 3 | 5 |
Age | Difference between review time and data collection time | 411.5 | 313.0 | 0 | 1730 |
Control variables | |||||
Price | Consulting service price | 483.4 | 163.8 | 200 | 1700 |
Volume | Total number of consulting services | 1567 | 1577 | 9 | 8161 |
Variables | Pro | |||||
---|---|---|---|---|---|---|
Model 1 | Model 2 | Model 3 | 95% Confidence Intervals | VIF | ||
Price | 0.002 *** | 0.002 *** | 0.002 *** | 0.001 | 0.003 | 1.18 |
(0.000) t = 4.18 | (0.000) t = 4.12 | (0.000) t = 3.86 | ||||
Volume | 0.000 *** | −0.000 *** | −0.000 *** | −0.001 | −0.000 | 1.19 |
(0.000) t = −7.32 | (0.000) t = −7.77 | (0.000) t = −7.39 | ||||
Rating | 1.054 *** | 1.074 *** | 1.005 *** | 0.505 | 1.505 | 1.00 |
(0.258) t = 4.09 | (0.257) t = 4.18 | (0.255) t = 3.94 | ||||
Age | −0.003 *** | −0.003 *** | −0.003 *** | −0.003 | −0.002 | 1.01 |
(0.000) t = −13.75 | (0.000) t = −13.62 | (0.000) t = −14.28 | ||||
Topic 1 | 1.422 ** | 1.086 * | 0.070 | 2.102 | 1.96 | |
(0.521) t = 2.73 | (0.518) t = 2.10 | |||||
Topic 2 | 2.571 *** | 2.019 *** | 0.984 | 3.054 | 1.90 | |
(0.528) t = 4.87 | (0.528) t = 3.82 | |||||
Topic 3 | 1.735 *** | 1.348 ** | 0.371 | 2.325 | 2.10 | |
(0.500) t = 3.47 | (0.498) t = 2.71 | |||||
Topic 4 | 2.234 *** | 1.677 *** | 0.834 | 2.521 | 3.05 | |
(0.429) t = 5.21 | (0.430) t = 3.90 | |||||
Topic 5 | 2.384 *** | 1.943 *** | 1.012 | 2.874 | 2.35 | |
(0.476) t = 5.01 | (0.475) t = 4.09 | |||||
Scores | 0.110 *** | 0.049 | 0.172 | 1.15 | ||
(0.031) t = 3.55 | ||||||
Polarity | 0.279 *** | 0.192 | 0.366 | 1.16 | ||
(0.045) t = 6.26 | ||||||
Observations | 4.254 | 4.254 | 4.254 | |||
R2 | 0.060 | 0.068 | 0.084 | |||
Adjusted R2 | 0.0587 | 0.0656 | 0.0814 | |||
F | 67.35 | 34.17 | 35.25 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, J.; Gao, L. Research on the Characteristics and Usefulness of User Reviews of Online Mental Health Consultation Services: A Content Analysis. Healthcare 2021, 9, 1111. https://doi.org/10.3390/healthcare9091111
Liu J, Gao L. Research on the Characteristics and Usefulness of User Reviews of Online Mental Health Consultation Services: A Content Analysis. Healthcare. 2021; 9(9):1111. https://doi.org/10.3390/healthcare9091111
Chicago/Turabian StyleLiu, Jingfang, and Lu Gao. 2021. "Research on the Characteristics and Usefulness of User Reviews of Online Mental Health Consultation Services: A Content Analysis" Healthcare 9, no. 9: 1111. https://doi.org/10.3390/healthcare9091111