Human and AI Reviews Coexist: How Hybrid Review Systems Enhance Trust and Decision Confidence in E-Commerce
Abstract
1. Introduction
2. Theoretical Framework and Hypotheses Development
2.1. Human–AI Complementarity Theory
2.2. Information Diagnosticity Perspective
2.3. Dual-Process and Cognitive Load Theory
3. Study 1: Complementarity Between Human-Generated Reviews and AI-Generated Summaries
3.1. Purpose and Overview
3.2. Participants and Design
3.3. Stimuli and Procedure
3.4. Measures
3.5. Results
3.6. Discussion
4. Study 2: The Moderating Role of Presentation Sequence
4.1. Purpose and Overview
4.2. Participants and Design
4.3. Stimuli and Procedure
4.4. Measures
4.5. Results
4.6. Discussion
5. Study 3: AI Literacy and Information Overload
5.1. Purpose and Overview
5.2. Participants and Design
5.3. Stimuli and Procedure
5.4. Measures
5.5. Results
5.6. Discussion
6. General Discussion
6.1. Theoretical Implications
6.2. Managerial Implications
6.3. Limitations and Future Research Directions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A. Experimental Stimuli (Study 3)
Appendix A.1. AI Literacy Manipulation Text
Appendix A.2. Review Stimuli (Low Information Load Condition)
Appendix A.3. Review Stimuli (High Information Load Condition)
References
- Chevalier, J.A.; Mayzlin, D. The Effect of Word of Mouth on Sales: Online Book Reviews. J. Mark. Res. 2006, 43, 345–354. [Google Scholar] [CrossRef]
- Mudambi, S.M.; Schuff, D. What Makes a Helpful Online Review? A Study of Customer Reviews on Amazon.com. MIS Q. 2010, 34, 185–200. [Google Scholar] [CrossRef]
- Filieri, R.; Raguseo, E.; Vitari, C. What Moderates the Influence of Extremely Negative Ratings? The Role of Review and Reviewer Characteristics. Int. J. Hosp. Manag. 2019, 77, 333–341. [Google Scholar] [CrossRef]
- Park, D.-H.; Lee, J. eWOM Overload and Its Effect on Consumer Behavioral Intention Depending on Consumer Involvement. Electron. Commer. Res. Appl. 2008, 7, 386–398. [Google Scholar] [CrossRef]
- Dogru, T.; Line, N.; Zhang, T.; Altin, M.; Olya, H.; Zhang, Y.; Ye, B.H.; Wang, C.; Law, R.; Guillet, B.D.; et al. Generative Artificial Intelligence in the Hospitality and Tourism Industry: Developing a Framework for Future Research. J. Hosp. Tour. Res. 2025, 49, 235–253. [Google Scholar] [CrossRef]
- Longoni, C.; Bonezzi, A.; Morewedge, C.K. Resistance to Medical Artificial Intelligence. J. Consum. Res. 2019, 46, 629–650. [Google Scholar] [CrossRef]
- Flavián, C.; Pérez-Rueda, A.; Belanche, D.; Casaló, L.V. Intention to Use Analytical Artificial Intelligence (AI) in Services—The Effect of Technology Readiness and Awareness. J. Serv. Manag. 2022, 33, 293–320. [Google Scholar] [CrossRef]
- Brynjolfsson, E.; McAfee, A. Machine, Platform, Crowd: Harnessing Our Digital Future; W.W. Norton & Company: New York, NY, USA, 2017. [Google Scholar]
- Petty, R.E.; Cacioppo, J.T. Communication and Persuasion: Central and Peripheral Routes to Attitude Change; Springer: New York, NY, USA, 2012. [Google Scholar]
- Carichon, F.; Ngouma, C.; Liu, B.; Caporossi, G. Objective and Neutral Summarization of Customer Reviews. Expert Syst. Appl. 2024, 255, 124449. [Google Scholar] [CrossRef]
- Kempf, D.S.; Smith, R.E. Consumer Processing of Product Trial and the Influence of Prior Advertising: A Structural Modeling Approach. J. Mark. Res. 1998, 35, 325–338. [Google Scholar] [CrossRef]
- Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
- Gursoy, D.; Cai, R. Artificial Intelligence: An Overview of Research Trends and Future Directions. Int. J. Contemp. Hosp. Manag. 2025, 37, 1–17. [Google Scholar] [CrossRef]
- Sweller, J. Cognitive Load During Problem Solving: Effects on Learning. Cogn. Sci. 1988, 12, 257–285. [Google Scholar] [CrossRef]
- Hayes, A.F. Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach, 2nd ed.; Guilford Press: New York, NY, USA, 2017. [Google Scholar]
- Schmidt, P.; Biessmann, F.; Teubner, T. Transparency and Trust in Artificial Intelligence Systems. J. Decis. Syst. 2020, 29, 260–278. [Google Scholar] [CrossRef]
- McGrath, M.J.; Duensen, A.; Lacey, J.; Paris, C. Collaborative Human–AI Trust (CHAI-T): A Process Framework for Active Management of Trust in Human–AI Collaboration. Comput. Hum. Behav. Artif. Hum. 2025, 6, 100200. [Google Scholar] [CrossRef]
- Faul, F.; Erdfelder, E.; Buchner, A.; Lang, A.-G. Statistical Power Analyses Using G*Power 3.1: Tests for Correlation and Regression Analyses. Behav. Res. Methods 2009, 41, 1149–1160. [Google Scholar] [CrossRef]
- Sousa, V.D.; Rojjanasrirat, W. Translation, Adaptation and Validation of Instruments or Scales for Use in Cross-Cultural Health Care Research: A Clear and User-Friendly Guideline. J. Eval. Clin. Pract. 2011, 17, 268–274. [Google Scholar] [CrossRef]
- Lee, A.Y.; Labroo, A.A. The Effect of Conceptual and Perceptual Fluency on Brand Evaluation. J. Mark. Res. 2004, 41, 151–165. [Google Scholar] [CrossRef]
- Kong, X.; Fang, H.; Chen, W.; Xiao, J.; Zhang, M. Examining Human–AI Collaboration in Hybrid Intelligence Learning Environments: Insight from the Synergy Degree Model. Humanit. Soc. Sci. Commun. 2025, 12, 821. [Google Scholar] [CrossRef]
- Kulal, A. Cognitive Risks of AI: Literacy, Trust, and Critical Thinking. J. Comput. Inf. Syst. 2025, online ahead of print. [Google Scholar] [CrossRef]
- Fu, S.; Li, H.; Liu, Y.; Pirkkalainen, H.; Salo, M. Social Media Overload, Exhaustion, and Use Discontinuance: Examining the Effects of Information Overload, System Feature Overload, and Social Overload. Inf. Process. Manag. 2020, 57, 102307. [Google Scholar] [CrossRef]
- Wang, L.; Che, G.; Hu, J.; Chen, L. Online Review Helpfulness and Information Overload: The Roles of Text, Image, and Video Elements. J. Theor. Appl. Electron. Commer. Res. 2024, 19, 1243–1266. [Google Scholar] [CrossRef]
- Almatrafi, O.; Johri, A.; Lee, H. A Systematic Review of AI Literacy Conceptualization, Constructs, and Implementation and Assessment Efforts (2019–2023). Comput. Educ. Open 2024, 6, 100173. [Google Scholar] [CrossRef]
- Hemmer, P.; Schemmer, M.; Kühl, N.; Vössing, M.; Satzger, G. Complementarity in Human–AI Collaboration: Concept, Sources, and Evidence. Eur. J. Inf. Syst. 2025, 34, 979–1002. [Google Scholar] [CrossRef]
- Rahwan, I.; Cebrian, M.; Obradovich, N.; Bongard, J.; Bonnefon, J.-F.; Breazeal, C.; Crandall, J.W.; Christakis, N.A.; Couzin, I.D.; Jackson, M.O.; et al. Machine behaviour. Nature 2019, 568, 477–486. [Google Scholar] [CrossRef]
- Dellermann, D.; Ebel, P.; Söllner, M.; Leimeister, J.M. Hybrid intelligence. Bus. Inf. Syst. Eng. 2019, 61, 637–643. [Google Scholar] [CrossRef]
- Cheung, M.Y.; Luo, C.; Sia, C.L.; Chen, H. Credibility of Electronic Word-of-Mouth: Informational and Normative Determinants of On-line Consumer Recommendations. Int. J. Electron. Commer. 2009, 13, 9–38. [Google Scholar] [CrossRef]
- Zhang, Y.; Norman, D.A. Representations in distributed cognitive tasks. Cogn. Sci. 1994, 18, 87–122. [Google Scholar] [CrossRef]
- Marusich, L.R.; Files, B.T.; Bancilhon, M.; Rawal, J.C.; Raglin, A. Trust Calibration for Joint Human/AI Decision-Making in Dynamic and Uncertain Contexts. In Artificial Intelligence in HCI; Degen, H., Ntoa, S., Eds.; Lecture Notes in Computer Science, Volume 15819; Springer: Cham, Switzerland, 2025. [Google Scholar]
- Ulfert, A.-S.; Georganta, E.; Centeio Jorge, C.; Mehrotra, S.; Tielman, M. Shaping a Multidisciplinary Understanding of Team Trust in Human–AI Teams: A Theoretical Framework. Eur. J. Work Organ. Psychol. 2024, 33, 158–171. [Google Scholar] [CrossRef]
- Lukyanenko, R.; Maass, W.; Storey, V.C. Trust in Artificial Intelligence: From a Foundational Trust Framework to Emerging Research Opportunities. Electron. Mark. 2022, 32, 1993–2020. [Google Scholar] [CrossRef]
- Eppler, M.J.; Mengis, J. The Concept of Information Overload: A Review of Literature from Organization Science, Accounting, Marketing, MIS, and Related Disciplines. Inf. Soc. 2004, 20, 325–344. [Google Scholar] [CrossRef]
- Hofstede, G. Culture’s Consequences: Comparing Values, Behaviors, Institutions, and Organizations Across Nations; Sage Publication: Thousand Oaks, CA, USA, 2001. [Google Scholar]



| Hypothesis | Path/Relationship Tested | Statistical Result | Supported |
|---|---|---|---|
| H1 | Review type (Human-only vs. AI-only vs. Hybrid) → Review trust/Decision confidence | F(2, 201) = 7.92, p < 0.001; Hybrid > Human-only, AI-only | Yes |
| H2 | Review type → Perceived authenticity/objectivity → Review trust (Mediation) | Indirect effects significant via authenticity (β = 0.21, 95% CI [0.09, 0.37]) and objectivity (β = 0.18, 95% CI [0.06, 0.32]) | Yes |
| H3 | Perceived authenticity and objectivity → Review trust → Decision confidence (Sequential Mediation) | Indirect effect = 0.27, 95% CI [0.12, 0.48]; full mediation observed | Yes |
| Moderator (Z) | ΔR2 (Interaction) | F-Change | Interaction β (SE) | Simple Slope (Low Z) | Simple Slope (High Z) | Support |
|---|---|---|---|---|---|---|
| AI literacy | 0.020 | 5.81 * | 0.19 (0.08) | 0.20 (t = 1.97, p > 0.05) | 0.46 (t = 5.12, p < 0.001) | Yes |
| Information overload | 0.015 | 3.57 † | −0.13 (0.07) | 0.41 (t = 4.86, p < 0.001) | 0.18 (t = 1.82, p > 0.05) | partial |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Share and Cite
Li, Y.; Ha, H.-Y. Human and AI Reviews Coexist: How Hybrid Review Systems Enhance Trust and Decision Confidence in E-Commerce. J. Theor. Appl. Electron. Commer. Res. 2026, 21, 14. https://doi.org/10.3390/jtaer21010014
Li Y, Ha H-Y. Human and AI Reviews Coexist: How Hybrid Review Systems Enhance Trust and Decision Confidence in E-Commerce. Journal of Theoretical and Applied Electronic Commerce Research. 2026; 21(1):14. https://doi.org/10.3390/jtaer21010014
Chicago/Turabian StyleLi, Yunzhe, and Hong-Youl Ha. 2026. "Human and AI Reviews Coexist: How Hybrid Review Systems Enhance Trust and Decision Confidence in E-Commerce" Journal of Theoretical and Applied Electronic Commerce Research 21, no. 1: 14. https://doi.org/10.3390/jtaer21010014
APA StyleLi, Y., & Ha, H.-Y. (2026). Human and AI Reviews Coexist: How Hybrid Review Systems Enhance Trust and Decision Confidence in E-Commerce. Journal of Theoretical and Applied Electronic Commerce Research, 21(1), 14. https://doi.org/10.3390/jtaer21010014

