Evaluation of an Artificial Intelligence-Generated Health Communication Material on Bird Flu Precautions
Simple Summary
Abstract
1. Introduction
2. Methods
2.1. Study Design and Procedure
2.2. Message Development
2.2.1. Text Development
2.2.2. Visual Development
2.2.3. Formatting and Comic Design
2.3. Participant Recruitment and Data Collection
2.4. Measures
2.4.1. Dependent Variable
2.4.2. Independent Variables
2.4.3. Sociodemographic Covariates
2.5. Data Analysis
2.6. Data Quality
3. Results
3.1. Participant Characteristics
3.2. Message Characteristics
3.3. Bivariate and Multivariate Associations with Perceived Message Effectiveness
3.4. Mediation Analysis of Visual Informativeness
4. Discussion
4.1. Implications for Practice and Research
4.2. Strengths and Limitations
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
AI | Artificial Intelligence |
CDC | Centers for Disease Control and Prevention |
HBM | Health Belief Model |
PME | Perceived Message Effectiveness |
USDA | United States Department of Agriculture |
References
- Bhattacharya, S.; Sharma, N.; Hoedebecke, K.; Hossain, M.M.; Gökdemir, Ö.; Singh, A. Harnessing the potential of uploading health educational materials on medical institutions’ social media for controlling emerging and re-emerging disease outbreaks. J. Educ. Health Promot. 2020, 9, 213. [Google Scholar] [CrossRef] [PubMed]
- Garg, S.; Reinhart, K.; Couture, A.; Kniss, K.; Davis, C.T.; Kirby, M.K.; Murray, E.L.; Zhu, S.; Kraushaar, V.; Wadford, D.A.; et al. Highly Pathogenic Avian Influenza A(H5N1) Virus Infections in Humans. N. Engl. J. Med. 2025, 392, 843–854. [Google Scholar] [CrossRef] [PubMed]
- CDC. CDC A(H5N1) Bird Flu Response Update March 19, 2025, Avian Influenza (Bird Flu). 2025. Available online: https://www.cdc.gov/bird-flu/spotlights/h5n1-response-03192025.html (accessed on 22 March 2025).
- Google Trends. Google Trends: Bird Flu. 2025. Available online: https://trends.google.com/trends/explore?date=today%205-y&geo=US&q=bird%20flu&hl=en (accessed on 22 March 2025).
- Cooper, L.N.; Diaz, M.I.; Hanna, J.J.; Most, Z.M.; Lehmann, C.U.; Medford, R.J. Birds of a feather? Mis- and dis-information on the social media platform X related to avian influenza. Antimicrob. Steward. Healthc. Epidemiol. 2025, 5, e4. [Google Scholar] [CrossRef] [PubMed]
- Arambul, N.; Sraboni, S.; Chukwunweike, J.; Olagoke, A. Exploring the Association between Trust in Healthcare Entities and Exposure to Emerging Health Misinformation in Nebraska: A Pilot Study. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 2023, 67, 1731–1734. [Google Scholar] [CrossRef]
- Aldoory, L. The Status of Health Literacy Research in Health Communication and Opportunities for Future Scholarship. Health Commun. 2017, 32, 211–218. [Google Scholar] [CrossRef] [PubMed]
- Gentili, A.; Villani, L.; Osti, T.; Corona, V.F.; Gris, A.V.; Zaino, A.; Bonacquisti, M.; De Maio, L.; Solimene, V.; Gualano, M.R.; et al. Strategies and bottlenecks to tackle infodemic in public health: A scoping review. Front. Public Health 2024, 12, 1438981. [Google Scholar] [CrossRef] [PubMed]
- COvelman; Kugley, S.; Gartlehner, G.; Viswanathan, M. The use of a large language model to create plain language summaries of evidence reviews in healthcare: A feasibility study. Cochrane Evid. Synth. Methods 2024, 2, e12041. [Google Scholar] [CrossRef] [PubMed]
- Razack, H.I.A.; Mathew, S.T.; Saad, F.F.A.; Alqahtani, S.A. Artificial intelligence-assisted tools for redefining the communication landscape of the scholarly world. Sci. Ed. 2021, 8, 134–144. [Google Scholar] [CrossRef]
- Rodriguez, D.V.; Lawrence, K.; Gonzalez, J.; Brandfield-Harvey, B.; Xu, L.; Tasneem, S.; Levine, D.L.; Mann, D. Leveraging Generative AI Tools to Support the Development of Digital Solutions in Health Care Research: Case Study. JMIR Hum. Factors 2024, 11, e52885. [Google Scholar] [CrossRef] [PubMed]
- Bharel, M.; Auerbach, J.; Nguyen, V.; DeSalvo, K.B. Transforming Public Health Practice With Generative Artificial Intelligence. Health Aff. 2024, 43, 776–782. [Google Scholar] [CrossRef] [PubMed]
- King, A.J.; Jensen, J.D.; Davis, L.A.; Carcioppolo, N. Perceived visual informativeness (PVI): Construct and scale development to assess visual information in printed materials. J. Health Commun. 2014, 19, 1099–1115. [Google Scholar] [CrossRef] [PubMed]
- Jensen, J.D.; King, A.J.; Carcioppolo, N.; Davis, L. Why are Tailored Messages More Effective? A Multiple Mediation Analysis of a Breast Cancer Screening Intervention. J. Commun. 2012, 62, 851–868. [Google Scholar] [CrossRef] [PubMed]
- Ijaz, S.; Khan, R.; Saleem, M.; Jamil, S.; Bhatti, M.F.E.; Rizvi, S.B.H.; Liaqat, C.; Fatima, N.; Rehman, A. Emerging Threats to Regional Public Health Posed by Zoonoses. In Zoonosis; Unique Scientific Publishers: Faisalabad, Pakistan, 2023; pp. 104–114. [Google Scholar] [CrossRef]
- Rakower, J.; Hallyburton, A. Disease Information Through Comics: A Graphic Option for Health Education. J. Med. Humanit. 2022, 43, 475–492. [Google Scholar] [CrossRef] [PubMed]
- Berman, M.; Bozsik, F.; Shook, R.P.; Meissen-Sebelius, E.; Markenson, D.; Summar, S.; DeWit, E.; Carlson, J.A. Evaluation of the Healthy Lifestyles Initiative for Improving Community Capacity for Childhood Obesity Prevention. Prev. Chronic. Dis. 2018, 15, E24. [Google Scholar] [CrossRef] [PubMed]
- Sonke, J.; Sams, K.; Morgan-Daniel, J.; Schaefer, N.; Pesata, V.; Golden, T.; Stuckey, H. Health Communication and the Arts in the United States: A Scoping Review. Am. J. Health Promot. 2021, 35, 106–115. [Google Scholar] [CrossRef] [PubMed]
- Palan, S.; Schitter, C. Prolific. ac—A subject pool for online experiments. J. Behav. Exp. Financ. 2018, 17, 22–27. [Google Scholar] [CrossRef]
- Peer, E.; Brandimarte, L.; Samat, S.; Acquisti, A. Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. J. Exp. Soc. Psychol. 2017, 70, 153–163. [Google Scholar] [CrossRef]
- Fishbein, M.; Hall-Jamieson, K.; Zimmer, E.; Von Haeften, I.; Nabi, R. Avoiding the boomerang: Testing the relative effectiveness of antidrug public service announcements before a national campaign. Am. J. Public Health 2002, 92, 238–245. [Google Scholar] [CrossRef] [PubMed]
- Cho, H.; Boster, F.J. First and Third Person Perceptions on Anti-Drug Ads Among Adolescents. Commun. Res. 2008, 35, 169–189. [Google Scholar] [CrossRef]
- Zhao, X.; Peterson, E. Effects of Temporal Framing on Response to Antismoking Messages: The Mediating Role of Perceived Relevance. J. Health Commun. 2017, 22, 37–44. [Google Scholar] [CrossRef] [PubMed]
- Olagoke, A.; Hebert-Beirne, J.; Floyd, B.; Caskey, R.; Boyd, A.; Molina, Y. The effectiveness of a religiously framed HPV vaccination message among Christian parents of unvaccinated adolescents in the United States. J. Commun. Healthc. 2023, 16, 215–224. [Google Scholar] [CrossRef] [PubMed]
- Sobel, M.E. Asymptotic confidence intervals for indirect effects in structural equation models. Sociol. Methodol. 1982, 13, 290–312. [Google Scholar] [CrossRef]
- Cumming, G. The new statistics: Why and how. Psychol. Sci. 2014, 25, 7–29. [Google Scholar] [CrossRef] [PubMed]
- Peer, E. Prolific: Crowdsourcing Academic Online Research. In The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences: Volume 2: Performing Research; Nichols, A.L., Edlund, J.E., Eds.; Cambridge University Press: Cambridge, UK, 2024; pp. 72–92. [Google Scholar] [CrossRef]
- Noar, S.M.; Bell, T.; Kelley, D.; Barker, J.; Yzer, M. Perceived message effectiveness measures in tobacco education campaigns: A systematic review. Commun. Methods Meas. 2018, 12, 295–313. [Google Scholar] [CrossRef] [PubMed]
- Abraham, T. Risk and outbreak communication: Lessons from alternative paradigms. Bull World Health Organ 2009, 87, 604–607. [Google Scholar] [CrossRef] [PubMed]
- Vicari, R.; Komendatova, N. Systematic meta-analysis of research on AI tools to deal with misinformation on social media during natural and anthropogenic hazards and disasters. Humanit. Soc. Sci. Commun. 2023, 10, 332. [Google Scholar] [CrossRef]
- Rohde, J.A.; Noar, S.M.; Prentice-Dunn, H.; Kresovich, A.; Hall, M.G. Comparison of Message and Effects Perceptions for The Real Cost E-Cigarette Prevention Ads. Health Commun. 2021, 36, 1222–1230. [Google Scholar] [CrossRef] [PubMed]
- Reigeluth, C.M.; An, Y. Formative Evaluation. In Merging the Instructional Design Process with Learner-Centered Theory; Routledge: Abingdon, UK, 2020. [Google Scholar]
- Elwy, A.R.; Wasan, A.D.; Gillman, A.G.; Johnston, K.L.; Dodds, N.; McFarland, C.; Greco, C.M. Using formative evaluation methods to improve clinical implementation efforts: Description and an example. Psychiatry Res. 2020, 283, 112532. [Google Scholar] [CrossRef] [PubMed]
- Telenta, J.; Jones, S.C.; Francis, K.L.; Polonsky, M.J.; Beard, J.; Renzaho, A.M.N. Australian lessons for developing and testing a culturally inclusive health promotion campaign. Health Promot. Int. 2020, 35, 217–231. [Google Scholar] [CrossRef] [PubMed]
- Cresswell, K.; Sheikh, A.; Franklin, B.D.; Krasuska, M.; Nguyen, H.T.; Hinder, S.; Lane, W.; Mozaffar, H.; Mason, K.; Eason, S.; et al. Theoretical and methodological considerations in evaluating large-scale health information technology change programmes. BMC Health Serv. Res. 2020, 20, 477. [Google Scholar] [CrossRef] [PubMed]
- Karinshak, E.; Liu, S.X.; Park, J.S.; Hancock, J.T. Working With AI to Persuade: Examining a Large Language Model’s Ability to Generate Pro-Vaccination Messages. Proc. ACM Hum.-Comput. Interact. 2023, 7, 1–29. [Google Scholar] [CrossRef]
- Danmaisoro, H.B. Development of the cognitive-behavioral communication (CBC) framework: A novel approach for mass communication in contact tracing during infectious disease outbreaks. World J. Adv. Res. Rev. 2024, 24, 1136–1147. [Google Scholar] [CrossRef]
- Lim, S.; Schmälzle, R. Artificial intelligence for health message generation: An empirical study using a large language model (LLM) and prompt engineering. Front. Commun. 2023, 8, 1129082. [Google Scholar] [CrossRef]
- Petty, R.E.; Cacioppo, J.T. The elaboration likelihood model of persuasion. In Communication and Persuasion; Springer: Berlin/Heidelberg, Germany, 1986; pp. 1–24. [Google Scholar]
- Caudell, M.A.; Charoonsophonsak, P.V.; Miller, A.; Lyimo, B.; Subbiah, M.; Buza, J.; Call, D.R. Narrative risk messages increase uptake and sharing of health interventions in a hard-to-reach population: A pilot study to promote milk safety among Maasai pastoralists in Tanzania. Pastoralism 2019, 9, 7. [Google Scholar] [CrossRef]
- Saraiva, I.; Ferreira, C. The Impact of Visual Communication in COVID-19′s Prevention and Risk Mitigation. In Advances in Design and Digital Communication; Martins, N., Brandão, D., Eds.; Springer International Publishing: Cham, Germany, 2021; pp. 433–442. [Google Scholar] [CrossRef]
- Davenport. Hyper-Personalization for Customer Engagement with Artificial Intelligence. 2023. Available online: https://journals.sagepub.com/doi/abs/10.1177/2694105820230301006 (accessed on 23 March 2025).
- Izah, S.C.; Sylva, L.; Hait, M. Cronbach’s Alpha: A Cornerstone in Ensuring Reliability and Validity in Environmental Health Assessment. ES Energy Environ. 2023, 23, 1057. [Google Scholar] [CrossRef]
- Prabhakaran, V.; Qadri, R.; Hutchinson, B. Cultural Incongruencies in Artificial Intelligence. arXiv 2022, arXiv:2211.13069. [Google Scholar] [CrossRef]
- Dillard, J.P.; Shen, L.; Vail, R.G. Does Perceived Message Effectiveness Cause Persuasion or Vice Versa? 17 Consistent Answers. Hum. Commun. Res. 2007, 33, 467–488. [Google Scholar] [CrossRef]
- Bults, M.; Beaujean, D.J.M.A.; Richardus, J.H.; Voeten, H.A.C.M. Perceptions and Behavioral Responses of the General Public During the 2009 Influenza A (H1N1) Pandemic: A Systematic Review. Disaster Med. Public Health Prep. 2015, 9, 207–219. [Google Scholar] [CrossRef] [PubMed]
- Popova, L.; Li, Y. Perceived message effectiveness: Do people need to think about message effectiveness to report the message as effective? Health Educ. Behav. 2023, 50, 441–449. [Google Scholar] [CrossRef] [PubMed]
Variable | n (%) |
---|---|
Age | |
Less than 45 | 68 (68%) |
45 or older | 32 (32%) |
Sex | |
Female | 51 (51%) |
Male | 46 (46%) |
Race | |
White | 69 (69%) |
Non-White | 31 (31%) |
Income | |
Less than USD 75,000 | 49 (49%) |
USD 75,000 or more | 51 (51%) |
Education | |
Less than a bachelor’s degree | 27 (27%) |
Bachelor’s degree or more | 73 (73%) |
Most liked message feature | |
Message clarity | 36 (36%) |
Simplicity | 39 (39%) |
Relatability/visuals/overall design | 13 (13%) |
None of the above/others | 13 (13%) |
Mean (SD) | |
Perceived message quality | 3.83 ± 0.98 |
Perceived realism | 3.84 ± 0.97 |
Perceived message relevance | 3.64 ± 1.02 |
Material attractiveness | 3.35 ± 1.15 |
Perceived visual informativeness | 3.48 ± 1.15 |
Perceived message effectiveness | 3.83 ± 0.99 |
Exposure Variables | Model 1 | Model 2 | ||
---|---|---|---|---|
Estimates β (95% CI) | p-Value | Estimates β (95% CI) | p-Value | |
Perceived message quality | 0.81 (0.69–0.92) | <0.0001 | 0.83 (0.71–0.95) | <0.0001 |
Perceived realism | 0.69 (0.54–0.84) | <0.0001 | 0.71 (0.56–0.87) | <0.0001 |
Perceived message relevance | 0.64 (0.49–0.78) | <0.0001 | 0.66 (0.51–0.79) | <0.0001 |
Material attractiveness | 0.61 (0.47–0.74) | <0.0001 | 0.61 (0.47–0.74) | <0.0001 |
Perceived visual informativeness | 0.51 (0.38–0.65) | <0.0001 | 0.51 (0.37–0.66) | <0.0001 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Olagoke, A.A.; Adebayo, C.T.; Aderonmu, J.A.; Adeaga, E.A.; Johnson, K.J. Evaluation of an Artificial Intelligence-Generated Health Communication Material on Bird Flu Precautions. Zoonotic Dis. 2025, 5, 22. https://doi.org/10.3390/zoonoticdis5030022
Olagoke AA, Adebayo CT, Aderonmu JA, Adeaga EA, Johnson KJ. Evaluation of an Artificial Intelligence-Generated Health Communication Material on Bird Flu Precautions. Zoonotic Diseases. 2025; 5(3):22. https://doi.org/10.3390/zoonoticdis5030022
Chicago/Turabian StyleOlagoke, Ayokunle A., Comfort Tosin Adebayo, Joseph Ayotunde Aderonmu, Emmanuel A. Adeaga, and Kimberly J. Johnson. 2025. "Evaluation of an Artificial Intelligence-Generated Health Communication Material on Bird Flu Precautions" Zoonotic Diseases 5, no. 3: 22. https://doi.org/10.3390/zoonoticdis5030022
APA StyleOlagoke, A. A., Adebayo, C. T., Aderonmu, J. A., Adeaga, E. A., & Johnson, K. J. (2025). Evaluation of an Artificial Intelligence-Generated Health Communication Material on Bird Flu Precautions. Zoonotic Diseases, 5(3), 22. https://doi.org/10.3390/zoonoticdis5030022