Evaluation of the Psychometric Properties of the Digital Health Literacy Instrument (DHLI-BrA) for Use in Brazilian Adolescents
Abstract
:1. Introduction
2. Materials and Methods
2.1. Study Design and Population
2.2. Eligibility Criteria
2.3. Instrument
2.4. Data Collection
2.5. Statistical Analysis
2.6. Ethical Aspects
3. Results
4. Discussion
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Lobo, M.L.C.G. O uso das TICs como ferramenta de ensino-aprendizagem no Ensino Superior ASM. CAD Geogr. 2015, 25, 16–26. [Google Scholar] [CrossRef]
- Ferreira, E.Z.; Oliveira, A.M.N.; Medeiros, S.P.; Gomes, G.C.; Cezar-Vaz, M.R.; Ávila, J.A. Internet influence on the biopsychosocial health of adolescents: An integratitive review. Rev. Bras. Enferm. 2020, 73, e20180766. [Google Scholar] [CrossRef] [PubMed]
- Kostkova, P. Grand challenges in digital health. Front. Public Health 2015, 3, 134. [Google Scholar] [CrossRef]
- Park, E.; Kwon, M. Testing the Digital Health Literacy Instrument for Adolescents: Cognitive Interviews. J. Med. Internet Res. 2021, 23, e17856. [Google Scholar] [CrossRef]
- IBGE. Pesquisa Nacional por Amostra de Domicílios Contínua: Acesso à Internet e à Televisão e Posse de Telefone Móvel Celular Para uso Pessoal. 2019. Available online: https://www.ibge.gov.br/estatisticas/sociais/populacao/17270-pnadcontinua.html?edicao=30362 (accessed on 12 March 2022).
- Dunahee, M.; Lebo, H.; Digitalcenter. The World Internet Project International Report (Sixth Ed). 2016. Available online: https://www.digitalcenter.org/wp-content/uploads/2013/06/2015-World-Internet-Report.pdf (accessed on 12 March 2022).
- Park, E.; Kwon, M. Health-Related Internet Use by Children and Adolescents: Systematic Review. J. Med. Internet Res. 2018, 20, e120. [Google Scholar] [CrossRef] [PubMed]
- Crone, E.A.; Konijn, E.A. Media use and brain development during adolescence. Nat. Commun. 2018, 9, 588. [Google Scholar] [CrossRef]
- Wong, C.A.; Merchant, R.M.; Moreno, M.A. Using social media to engage adolescents and young adults with their health. Healthcare 2014, 2, 220–224. [Google Scholar] [CrossRef]
- Swire-Thompson, B.; Lazer, D. Public health and online misinformation: Challenges and recommendations. Annu. Rev. Public Health 2020, 41, 433–451. [Google Scholar] [CrossRef]
- Lotto, M.; Jorge, O.S.; Machado, M.A.A.M.; Cruvinel, T. Exploring online oral health misinformation: A content analysis. Braz. Oral Res. 2023, 37, e049. [Google Scholar] [CrossRef]
- Vosoughi, S.; Roy, D.; Aral, S. The spread of true and false news online. Science 2018, 359, 1146–1151. [Google Scholar] [CrossRef]
- Fleary, S.A.; Joseph, P.; Pappagianopoulos, J.E. Adolescent health literacy and health behaviors: A systematic review. J. Adolesc. 2018, 62, 116–127. [Google Scholar] [CrossRef] [PubMed]
- Norman, C.D.; Skinner, H.A. eHEALS: The eHealth Literacy Scale. J. Med. Internet Res. 2006, 8, e27. [Google Scholar] [CrossRef] [PubMed]
- Van Der Vaart, R.; Drossaert, C. Development of the Digital Health Literacy Instrument: Measuring a Broad Spectrum of Health 1.0 and Health 2.0 Skills. J. Med. Internet Res. 2017, 19, e27. [Google Scholar] [CrossRef] [PubMed]
- Van De Belt, T.H.; Engelen, L.J.; Berben, S.A.; Schoonhoven, L. Definition of Health 2.0 and Medicine 2.0: A systematic review. J. Med. Internet Res. 2010, 12, e18. [Google Scholar] [CrossRef]
- Barbosa, M.C.F.; Baldiotti, A.L.P.; Lopes, C.T.; Paiva, S.M.; Granville-Garcia, A.F.; Ferreira, F.M. Cross-cultural adaptation of the Digital Health Literacy Instrument (DHLI) for use on Brazilian Adolescents. Braz. Dent. J. 2023, 34, 104–114. [Google Scholar]
- Agormedah, E.K.; Quansah, F.; Ankomah, F.; Hagan, J.E., Jr.; Srem-Sai, M.; Abieraba, R.S.K.; Frimpong, J.B.; Schack, T. Assessing the validity of digital health literacy instrument for secondary school students in Ghana: The polychoric factor analytic approach. Front. Digit. Health 2022, 4, 968806. [Google Scholar] [CrossRef]
- Martins, S.; Augusto, C.; Martins, M.R.O.; José Silva, M.; Okan, O.; Dadaczynski, K.; Duarte, A.; Fronteira, I.; Ramos, N.; Rosário, R. Adaptation and validation of the Digital Health Literacy Instrument for Portuguese university students. Health Promot. J. Austr. 2022, 33, 390–398. [Google Scholar] [CrossRef]
- Rivadeneira, M.F.; Miranda-Velasco, M.J.; Arroyo, H.V.; Caicedo-Gallardo, J.D.; Salvador-Pinos, C. Digital Health Literacy Related to COVID-19: Validation and Implementation of a Questionnaire in Hispanic University Students. Int. J. Environ. Res. Public Health 2022, 19, 4092. [Google Scholar] [CrossRef]
- Lorini, C.; Velasco, V.; Bonaccorsi, G.; Dadaczynski, K.; Okan, O.; Zanobini, P.; Vecchio, L.P. Validation of the COVID-19 Digital Health Literacy Instrument in the Italian Language: A Cross-Sectional Study of Italian University Students. Int. J. Environ. Res. Public Health 2022, 19, 6247. [Google Scholar] [CrossRef]
- Chun, H.; Park, E.J.; Choi, S.K.; Yoon, H.; Okan, O.; Dadaczynski, K. Validating the Digital Health Literacy Instrument in Relation to COVID-19 Information (COVID-DHL-K) among South Korean Undergraduates. Int. J. Environ. Res. Public Health 2022, 19, 3437. [Google Scholar] [CrossRef]
- Anthoine, E.; Moret, L.; Regnault, A.; Sébille, V.; Hardouin, J.B. Sample size used to validate a scale: A review of publicationson newly-developed patient reported outcomes measures. Health Qual. Life Outcomes 2014, 12, 176. [Google Scholar] [CrossRef] [PubMed]
- Barbosa, M.C.F.; Baldiotti, A.L.P.; Dias, M.L.L.S.; Granville-Garcia, A.F.; Paiva, S.M.; Ferreira, F.M. Cross-cultural adaptation of the eHealth Literacy Instrument (eHEALS) for application in Brazilian Adolescents. Braz. Oral Res. 2024, 38, e094. [Google Scholar] [CrossRef] [PubMed]
- Fleck, M.P.A.; Louzada, S.; Xavier, M.; Chamovich, E.; Vieira, G.; Santos, L.; Pinzon, V. Aplicação da versão em português do instrumento abreviado de avaliação da qualidade de vida “WHOQOL-bref”. Rev. Saude Publica 2000, 34, 178–188. [Google Scholar] [CrossRef] [PubMed]
- The Whoqol Group. The word Health Organization Quality of Life Assessment (WHOQOL): Position paper from the Health Organization. Soc. Sci. Med. 1995, 41, 1403–1409. [Google Scholar] [CrossRef]
- Cruvinel, A.F.P.; Cusicanqui Méndez, D.A.; Campos Chaves, G.; Gutierres, E.; Lotto, M.; Marchini Oliveira, T.; Cruvinel, T. The Brazilian validation of a health literacy instrument: The newest vital sign. Acta Odontol. Scand. 2018, 76, 587–594. [Google Scholar] [CrossRef]
- Cruvinel, A.F.P.; Méndez, D.A.C.; Oliveira, J.G.; Gutierres, E.; Lotto, M.; Machado, M.A.A.M.; Oliveira, T.M.; Cruvinel, T. The Brazilian version of the 20-item rapid estimate of adult literacy in medicine and dentistry. Peer J. 2017, 5, e3744. [Google Scholar] [CrossRef]
- Landis, J.R.; Koch, G.G. The meadurement of observer agrément for categorical data. Biometrics 1977, 33, 159. [Google Scholar] [CrossRef]
- Dunn, T.J.; Baguley, T.; Brunsden, V. From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. Br. J. Psychol. 2014, 105, 399–412. [Google Scholar] [CrossRef]
- Kuder, G.F.; Richardson, M.W. The theory of the estimation of test reliability. Psychometrika 1937, 2, 151–160. [Google Scholar] [CrossRef]
- Revelle, W. Psych: Procedures for Personality and Psychological Research; Northwestern University: Evanston, IL, USA, 2023; Available online: https://CRAN.R-project.org/package=psych (accessed on 30 September 2023).
- Kline, R.B. Principles and Practice of Structural Equation Modeling, 4th ed.; Gu3ilford Press: New York, NY, USA, 2015. [Google Scholar]
- Norman, S. Biostatistics: The Bare Essentials, 3rd ed.; PMPH: Raleigh, NC, USA, 2008. [Google Scholar]
- Brown, T.A. Confirmatory Factor Analysis for Applied Research; Guilford Press: New York, NY, USA, 2006. [Google Scholar]
- Chalmers, P. Mirt: A Multidimensional Item Response Theory Package for the R Environment. J. Stat. Softw. 2012, 48, 1–29. [Google Scholar] [CrossRef]
- Bond, T.; Yan, Z.; Heene, M. Applying the Rasch Model: Fundamental Measurement in the Human Sciences, 4th ed.; Routledge: London, UK, 2020. [Google Scholar]
- Boone, W.J. Rasch Analysis for Instrument Development: Why, When, and How? CBE—Life Sci. Educ. 2016, 15, rm4. [Google Scholar] [CrossRef] [PubMed]
- Xie, L.; Mo, P.K.H. Comparison of eHealth Literacy Scale (eHEALS) and Digital Health Literacy Instrument (DHLI) in Assessing Electronic Health Literacy in Chinese Older Adults: A Mixed-Methods Approach. Int. J. Environ. Res. Public Health 2023, 20, 3293. [Google Scholar] [CrossRef] [PubMed]
- Miyawaki, R.; Kato, M.; Kawamura, Y.; Ishikawa, H.; Oka, K. [Developing a Japanese version of the Digital Health Literacy Instrument]. Nihon Koshu Eisei Zasshi 2024, 71, 3–14. (In Japanese) [Google Scholar] [PubMed]
- Xie, L.; Hu, H.; Lin, J.; Mo, P.K.H. Psychometric validation of the Chinese digital health literacy instrument among Chinese older adults who have internet use experience. Int. J. Older People Nurs. 2024, 19, e12568. [Google Scholar] [CrossRef]
- Ventura-León, J.; Peña-Calero, B.N. The world should not revolve around Cronbach’s alpha ≥ 0.70. Adicciones 2021, 33, 369–372, (In English, Spanish). [Google Scholar] [CrossRef]
- Field, A. Descobrindo a Estatística Usando o SPSS, 2nd ed.; Artmed: Porto Alegre, Brazil, 2009. [Google Scholar]
- Lima, L.C.M.; Neves, É.T.B.; Dutra, L.D.C.; Firmino, R.T.; Araújo, L.J.S.; Paiva, S.M.; Ferreira, F.M.; Granville-Garcia, A.F. Psychometric properties of BREALD-30 for assessing adolescents’ oral health literacy. Rev. Saude Publica 2019, 53, 53. [Google Scholar] [CrossRef]
- Lopes, R.T.; Neves, É.T.B.; Dutra, L.D.C.; Gomes, M.C.; Paiva, S.M.; Abreu, M.H.N.G.; Ferreira, F.M.; Granville-Garcia, A.F. Socioeconomic status and family functioning influence oral health literacy among adolescents. Rev. Saude Publica 2020, 54, 30. [Google Scholar] [CrossRef]
- Zarcadoolas, C.; Pleasant, A.; Greer, D.S. Understanding health literacy: An expanded model. Health Promot. Int. 2005, 20, 195–203. [Google Scholar] [CrossRef]
- Irwin, L.; Siddiqi, A.; Hertzman, C. Early Child Development: A Powerful Equalizer: Final Report for the World Health Organization’s Commission on Social Determinants of Health; University of British Columbia: Vancouver, Columbia, 2007. [Google Scholar]
- Bröder, J.; Okan, O.; Bauer, U.; Bruland, D.; Schlupp, S.; Bollweg, T.M.; Saboga-Nunes, L.; Bond, E.; Sørensen, K.; Bitzer, E.-M.; et al. Health literacy in childhood and youth: A systematic review of definitions and models. BMC Public Health 2017, 17, 361. [Google Scholar]
- Zhu, J.; Garin, C.M.; Qi, X.L.; Machado, A.; Wang, Z.; Hamed, S.B.; Stanford, T.R.; Salinas, E.; Whitlow, C.T.; Anderson, A.W.; et al. Brain structure and activity predicting cognitive maturation in adolescence. bioRxiv 2024, e608315, preprint. [Google Scholar]
Scale Mean If Item Deleted | Scale Variance If Item Deleted | Item–Total Correlation | Cronbach’s Alpha If Item Deleted | |
---|---|---|---|---|
Q1 | 59.71 | 58.2 | 0.38 | 0.83 |
Q2 | 59.75 | 58.6 | 0.30 | 0.83 |
Q3 | 59.88 | 57.8 | 0.36 | 0.83 |
Q4 | 60.90 | 55.3 | 0.53 | 0.82 |
Q5 | 60.59 | 53.5 | 0.61 | 0.81 |
Q6 | 60.89 | 54.3 | 0.55 | 0.82 |
Q7 | 61.24 | 53.8 | 0.56 | 0.82 |
Q8 | 60.73 | 56.3 | 0.30 | 0.83 |
Q9 | 60.35 | 54.8 | 0.50 | 0.82 |
Q10 | 60.53 | 55.6 | 0.42 | 0.82 |
Q11 | 60.82 | 55.9 | 0.39 | 0.83 |
Q12 | 60.87 | 55.3 | 0.40 | 0.83 |
Q13 | 60.47 | 57.7 | 0.37 | 0.83 |
Q14 | 59.87 | 57.2 | 0.34 | 0.83 |
Q15 | 60.96 | 56.4 | 0.40 | 0.83 |
Q16 | 60.31 | 55.0 | 0.51 | 0.83 |
Q17 | 60.65 | 54.2 | 0.47 | 0.83 |
Q18 | 60.38 | 56.2 | 0.38 | 0.83 |
Q19 | 60.35 | 57.5 | 0.31 | 0.83 |
Q20 | 59.97 | 60.1 | 0.11 | 0.84 |
Q21 | 59.79 | 59.8 | 0.15 | 0.83 |
Factors44 | Cronbach’s Alpha | Correlations | ||||||
---|---|---|---|---|---|---|---|---|
1 | 2 | 3 | 4 | 5 | 6 | |||
1 | Operational skills | 0.534 | 1 | |||||
2 | Navigation skills | 0.528 | 0.556 * | 1 | ||||
3 | Skills to assess the reliability | 0.598 | 0.620 * | 0.841 * | 1 | |||
4 | Skills to Search and determine the relevance for information | 0.757 | 0.587 * | 0.577 * | 0.663 * | 1 | ||
5 | Skills to include self-generated | 0.626 | 0.644 * | 0.645 * | 0.588 * | 0.689 * | 1 | |
6 | Skills to protect privacy | 0.624 | 0.175 | 0.049 | 0.006 | 0.453* | 0.093 | 1 |
Scale Mean If Item Deleted | Scale Variance If Item Deleted | Item–Total Correlation | Cronbach’s Alpha If Item Deleted | |
---|---|---|---|---|
Q22 | 3.452 | 2.197 | 0.407 | 0.492 |
Q23 | 3.394 | 2.436 | 0.255 | 0.548 |
Q24 | 3.544 | 2.213 | 0.350 | 0.512 |
Q25 | 3.533 | 2.372 | 0.236 | 0.557 |
Q26 | 3.748 | 2.366 | 0.235 | 0.558 |
Q27 | 3.617 | 2.275 | 0.290 | 0.537 |
Q28 | 3.623 | 2.460 | 0.300 | 0.535 |
Discrimination | Difficulty | Outfit | Infit | |
---|---|---|---|---|
Q22 | 1.827 | −0.718 | 0.513 | 0.694 |
Q23 | 0.856 | −1.530 | 0.846 | 0.938 |
Q24 | 1.233 | −0.459 | 0.737 | 0.803 |
Q25 | 0.762 | −0.719 | 0.885 | 0.911 |
Q26 | 0.710 | 0.611 | 0.894 | 0.914 |
Q27 | 0.886 | −0.182 | 0.849 | 0.872 |
Q28 | 0.858 | 1.020 | 0.848 | 0.893 |
Variables | DHLI-BrA Self-Reported | DHLI-BrA Performance-Based | ||
---|---|---|---|---|
r | p | r | p | |
Adolescent’s age | 0.267 | <0.001 | 0.324 | <0.001 |
Average family income | 0.167 | 0.015 | 0.242 | <0.001 |
Digital Health Literacy–eHEALS-BrA 1 | 0.452 | <0.001 | 0.065 | 0.294 |
Health Literacy–NVS 2 | 0.078 | 0.215 | 0.472 | <0.001 |
Health Literacy–REALMD-20 3 | 0.192 | 0.002 | 0.445 | <0.001 |
Quality of life–physical | 0.199 | 0.001 | −0.059 | 0.346 |
Quality of life–psychological aspects | 0.213 | 0.001 | −0.101 | 0.106 |
Quality of life–social relationships | 0.055 | 0.383 | −0.071 | 0.257 |
Quality of life–environment | 0.183 | 0.033 | 0.054 | 0.392 |
Variables | DHLI-BrA Self-Reported | p | DHLI-BrA Performance-Based | p | |
---|---|---|---|---|---|
General health self-assessment | Very Good | 3.14 (0.39) | 0.051 * | 3.90 (1.73) | 0.319 * |
Good | 3.01 (0.37) | 4.32 (1.73) | |||
Regular | 2.97 (0.34) | 4.04 (1.63) | |||
Bad/Very Bad | 2.72 (0.51) | 3.70 (1.48) | |||
Self-assessment of Internet usage skills | Very Good | 3.11 (0.34) | 0.002 * | 4.24 (1.78) | 0.012 * |
Good | 2.97 (0.35) | 4.27 (1.69) | |||
Bad | 2.87 (0.48) | 3.33 (1.41) | |||
Frequency of Internet use | Every day | 3.02 (0.37) | 0.433 * | 4.22 (1.71) | 0.022 * |
Almost all | 3.04 (0.42) | 3.50 (1.55) | |||
Almost never | 2.78 (0.23) | 2.17 (1.15) | |||
Do you have mobile Internet? | Yes | 3.09 (0.36) | 0.002 * | 4.24 (1.79) | 0.026 * |
Sometimes | 2.92 (0.36) | 4.19 (1.66) | |||
No | 2.95 (0.40) | 3.32 (1.21) | |||
Searched for a doctor/dentist | Yes | 3.07 (0.35) | >0.001 | 4.16 (1.75) | 0.705 # |
No | 2.83 (0.39) | 4.11 (1.59) | |||
Search for health information on the Internet | Yes | 3.03 (0.37) | 0.243 | 4.26 (1.68) | 0.001 # |
No | 2.91 (0.42) | 2.98 (1.69) | |||
Schedule appointment | Yes | 3.06 (0.38) | 0.014 | 4.29 (1.72) | 0.090 # |
No | 2.96 (0.36) | 3.97 (1.70) | |||
Read health information | Yes | 3.04 (0.37) | 0.246 | 4.36 (1.66) | 0.002 # |
No | 2.97 (0.37) | 3.59 (1.76) | |||
Use health apps | Yes | 3.07 (0.39) | 0.086 | 4.44 (1.76) | 0.040 # |
No | 2.99 (0.36) | 3.99 (1.66) | |||
Search for symptoms online | Yes | 3.01 (0.37) | 0.728 | 4.24 (1.71) | 0.046 # |
No | 3.03 (0.38) | 3.74 (1.70) | |||
Post an evaluation/review about a medical treatment | Yes | 3.08 (0.37) | 0.402 | 2.77 (1.96) | <0.001 # |
No | 3.01 (0.37) | 4.290 (1.63) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Barbosa, M.C.F.; Baldiotti, A.L.P.; Firmino, R.T.; Paiva, S.M.; Granville-Garcia, A.F.; Ferreira, F.d.M. Evaluation of the Psychometric Properties of the Digital Health Literacy Instrument (DHLI-BrA) for Use in Brazilian Adolescents. Int. J. Environ. Res. Public Health 2024, 21, 1458. https://doi.org/10.3390/ijerph21111458
Barbosa MCF, Baldiotti ALP, Firmino RT, Paiva SM, Granville-Garcia AF, Ferreira FdM. Evaluation of the Psychometric Properties of the Digital Health Literacy Instrument (DHLI-BrA) for Use in Brazilian Adolescents. International Journal of Environmental Research and Public Health. 2024; 21(11):1458. https://doi.org/10.3390/ijerph21111458
Chicago/Turabian StyleBarbosa, Mariane Carolina Faria, Ana Luiza Peres Baldiotti, Ramon Targino Firmino, Saul Martins Paiva, Ana Flávia Granville-Garcia, and Fernanda de Morais Ferreira. 2024. "Evaluation of the Psychometric Properties of the Digital Health Literacy Instrument (DHLI-BrA) for Use in Brazilian Adolescents" International Journal of Environmental Research and Public Health 21, no. 11: 1458. https://doi.org/10.3390/ijerph21111458
APA StyleBarbosa, M. C. F., Baldiotti, A. L. P., Firmino, R. T., Paiva, S. M., Granville-Garcia, A. F., & Ferreira, F. d. M. (2024). Evaluation of the Psychometric Properties of the Digital Health Literacy Instrument (DHLI-BrA) for Use in Brazilian Adolescents. International Journal of Environmental Research and Public Health, 21(11), 1458. https://doi.org/10.3390/ijerph21111458