2. Materials and Methods
2.1. MCR Description
2.2. MCR Applications
2.2.3. Qualitative Analysis
3.1. Report-Back Using MCR
3.2. Benefits and Challenges
3.2.1. Benefits and Challenges of Manual Report-Back
3.2.2. Benefits and Challenges of Report-Back with MCR
“[Team member] has been able to, like within a week, turn around a variety of requests to create alternate graphics and visuals… she can do it so quickly and present us a variety of options to go back over and see what one makes the most sense for what we’re trying to communicate. We definitely couldn’t have played around that much with it in that first report-back.”
“We’ve been able to apply some tools from the health literacy field again to reduce the level of numeracy and graphicacy that’s required to engage in the material. And it’s been helpful to do that iterative process of ‘here’s the graph that we have, how can we reduce the [graphicacy] demand [for the reader]?’”
“Now we can look at all the data and say ‘oh this isn’t going to look right because these people are too low or too high so we have to make some changes to adopt and better understand how they will look.’ We can see everything ahead of time, so we can pre-plan how we’re going to try to help get people to understand their results.”
“Once you get into bigger studies, like maybe something that’s a cohort of a thousand, the ability to create those individual reports by hand just disappears, it’s not feasible time-wise, and I think there’s a growth and interest in the environmental health field and community engagement in particular to provide this data back to people. I think it’s going to be useful as larger and larger studies are providing that data back to people. I think once you start to get into a couple hundred or a thousand, you don’t have a choice but to automate it.”
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
- National Academies of Sciences, Engineering and Medicine. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm; National Academies Press: Cambridge, MA, USA, 2018. [Google Scholar]
- Brody, J.G.; Dunagan, S.C.; Morello-Frosch, R.; Brown, P.; Patton, S.; Rudel, R.A. Reporting individual results for biomonitoring and environmental exposures: Lessons learned from environmental communication case studies. Environ. Health 2014, 13, 1–8. [Google Scholar] [CrossRef]
- Ramirez-Andreotta, M.D.; Brody, J.G.; Lothrop, N.; Loh, M.; Beamer, P.I.; Brown, P. Reporting back environmental exposure data and free choice learning. Environ. Health 2016, 15, 1–6. [Google Scholar] [CrossRef] [PubMed]
- Brody, J.G.; Morello-Frosch, R.; Brown, P.; Rudel, R.A.; Altman, R.G.; Frye, M.; Osimo, C.A.; Perez, C.; Seryak, L.M. Improving Disclosure and Consent: “Is It Safe?”: New Ethics for Reporting Personal Exposures to Environmental Chemicals. Am. J. Public Health 2007, 97, 1547–1554. [Google Scholar] [CrossRef] [PubMed]
- Nelson, J.W.; Scammell, M.K.; Altman, R.G.; Webster, T.F.; Ozonoff, D.M. A new spin on research translation: The Boston Consensus Conference on Human Biomonitoring. Environ. Health Perspect. 2009, 117, 495–499. [Google Scholar] [CrossRef]
- Wolf, S.M.; Evans, B.J. Return of results and data to study participants. Science 2018, 362, 159–160. [Google Scholar] [CrossRef]
- Claudio, L.; Gilmore, J.; Roy, M.; Brenner, B. Communicating environmental exposure results and health information in a community-based participatory research study. BMC Public Health 2018, 18, 784. [Google Scholar] [CrossRef]
- Ohayon, J.L.; Cousins, E.; Brown, P.; Morello-Frosch, R.; Brody, J.G. Researcher and institutional review board perspectives on the benefits and challenges of reporting back biomonitoring and environmental exposure results. Environ. Res. 2017, 153, 140–149. [Google Scholar] [CrossRef]
- Brug, J.; Glanz, K.; Van Assema, P.; Kok, G.; Van Breukelen, G.J. The impact of computer-tailored feedback and iterative feedback on fat, fruit, and vegetable intake. Health Educ. Behav. 1998, 25, 517–531. [Google Scholar] [CrossRef] [PubMed]
- Kannan, S.; Schulz, A.; Israel, B.; Ayra, I.; Weir, S.; Dvonch, T.J.; Rowe, Z.; Miller, P.; Benjamin, A. A community-based participatory approach to personalized, computer-generated nutrition feedback reports: The Healthy Environments partnership. Prog. Community Health Partnersh. Res. Educ. Action 2008, 2, 41. [Google Scholar] [CrossRef] [PubMed]
- Unertl, K.M.; Schaefbauer, C.L.; Campbell, T.R.; Senteio, C.; Siek, K.A.; Bakken, S.; Veinot, T.C. Integrating community-based participatory research and informatics approaches to improve the engagement and health of underserved populations. J. Am. Med. Inform. Assoc. 2016, 23, 60–73. [Google Scholar] [CrossRef] [PubMed]
- Dunagan, S.; Brody, J.; Morello-Frosch, R.; Brown, P.; Goho, S.; Tovar, J.; Patton, S.; Danford, R. When pollution is personal: Handbook for reporting results to participants in biomonitoring and personal exposure studies. Newton MA Silent Spring Inst. 2013. Available online: https://silentspring.org/sites/default/files/news/2018-11/personal_exposure_report_handbook_0.pdf (accessed on 1 June 2020).
- Han, P.K.; Klein, W.M.; Arora, N.K. Varieties of uncertainty in health care: A conceptual taxonomy. Med. Decis. Mak. 2011, 31, 828–838. [Google Scholar] [CrossRef]
- Arcia, A.; Bales, M.E.; Brown III, W.; Co Jr, M.C.; Gilmore, M.; Lee, Y.J.; Park, C.S.; Prey, J.; Velez, M.; Woollen, J.; et al. Method for the development of data visualizations for community members with varying levels of health literacy. AMIA Annual. Symp. Proc. 2013, 16, 51–60. [Google Scholar]
- Data Viz Project. Available online: https://datavizproject.com/ (accessed on 1 September 2020).
- Graphic Cheat Sheet. Available online: https://billiondollargraphics.com/graphic-cheat-sheet/ (accessed on 1 September 2020).
- Strecher, V.; Zikmund-Fisher, B.J.; Fagerlin, A.; An, L.; Resnicow, K. Visualizing Health; University of Michigan and the Robert Wood Johnson Foundation: Ann Arbor, MI, USA, 2021; Available online: http://www.vizhealth.org (accessed on 1 September 2020).
- Wongsuphasawat, K.; Moritz, D.; Anand, A.; Mackinlay, J.; Howe, B.; Heer, J. Voyager: Exploratory analysis via faceted browsing of visualization recommendations. IEEE Trans. Vis. Comput. Graph. 2015, 22, 649–658. [Google Scholar] [CrossRef]
- Boronow, K.E.; Susmann, H.P.; Gajos, K.Z.; Rudel, R.A.; Arnold, K.C.; Brown, P.; Morello-Frosch, R.; Havas, L.; Brody, J.G. DERBI: A digital method to help researchers offer “right-to-know” personal exposure results. Environ. Health Perspect. 2017, 125, A27–A33. [Google Scholar] [CrossRef]
- Arcia, A.; Suero-Tejeda, N.; Bales, M.E.; Merrill, J.A.; Yoon, S.; Woollen, J.; Bakken, S. Sometimes more is more: Iterative participatory design of infographics for engagement of community members with varying levels of health literacy. J. Am. Med. Inform. Assoc. 2016, 23, 174–183. [Google Scholar] [CrossRef]
- Forster, H.; Walsh, M.C.; O’Donovan, C.B.; Woolhead, C.; McGirr, C.; Daly, E.; O’Riordan, R.; Celis-Morales, C.; Fallaize, R.; Macready, A.L. A dietary feedback system for the delivery of consistent personalized dietary advice in the web-based multicenter Food4Me study. J. Med. Internet Res. 2016, 18, e150. [Google Scholar] [CrossRef]
- Perovich, L.J.; Ohayon, J.L.; Cousins, E.M.; Morello-Frosch, R.; Brown, P.; Adamkiewicz, G.; Brody, J.G. Reporting to parents on children’s exposures to asthma triggers in low-income and public housing, an interview-based case study of ethics, environmental literacy, individual action, and public health benefits. Environ. Health 2018, 17, 1–12. [Google Scholar] [CrossRef] [PubMed]
- R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2017; Available online: https://www.R-project.org/ (accessed on 1 June 2020).
- Chu, M.T.; Gillooly, S.E.; Levy, J.I.; Vallarino, J.; Reyna, L.N.; Laurent, J.G.C.; Coull, B.A.; Adamkiewicz, G. Real-time indoor PM2. 5 monitoring in an urban cohort: Implications for exposure disparities and source control. Environ. Res. 2020, 193, 110561. [Google Scholar] [CrossRef]
- Gillooly, S.E.; Zhou, Y.; Vallarino, J.; Chu, M.T.; Michanowicz, D.R.; Levy, J.I.; Adamkiewicz, G. Development of an in-home, real-time air pollutant sensor platform and implications for community use. Environ. Pollut. 2019, 244, 440–450. [Google Scholar] [CrossRef]
- Tomsho, K.S.; Schollaert, C.; Aguilar, T.; Bongiovanni, R.; Alvarez, M.; Scammell, M.K.; Adamkiewicz, G. A Mixed Methods Evaluation of Sharing Air Pollution Results with Study Participants via Report-Back Communication. Int. J. Environ. Res. Public Health 2019, 16, 4183. [Google Scholar] [CrossRef] [PubMed]
- Schollaert, C.; Alvarez, M.; Gillooly, S.; Tomsho, K.; Bongiovanni, R.; Chacker, S.; Aguilar, T.; Vallarino, J.; Adamkiewicz, G.; Scammell, M.K. Reporting Results of a Community-Based in-Home Exposure Monitoring Study: Developing Methods and Materials. Prog. Community Health Partnersh. Res. Educ. Action 2021, 15, 117–125. [Google Scholar] [CrossRef] [PubMed]
- Boselli, E.; Wu, Z.; Friedman, A.; Claus Henn, B.; Papautsky, I. Validation of Electrochemical Sensor for Determination of Manganese in Drinking Water. Environ. Sci. Technol. 2021. [Google Scholar] [CrossRef]
- Hsieh, H.-F.; Shannon, S.E. Three approaches to qualitative content analysis. Qual. Health Res. 2005, 15, 1277–1288. [Google Scholar] [CrossRef]
- Lebow-Skelley, E.; Yelton, S.; Janssen, B.; Erdei, E.; Pearson, M.A. Identifying Issues and Priorities in Reporting Back Environmental Health Data. Int. J. Environ. Res. Public Health 2020, 17, 6742. [Google Scholar] [CrossRef] [PubMed]
- Houts, P.S.; Doak, C.C.; Doak, L.G.; Loscalzo, M.J. The role of pictures in improving health communication: A review of research on attention, comprehension, recall, and adherence. Patient Educ. Couns. 2006, 61, 173–190. [Google Scholar] [CrossRef]
- Giannini, C.M.; Herrick, R.L.; Buckholz, J.M.; Daniels, A.R.; Biro, F.M.; Pinney, S.M. Comprehension and perceptions of study participants upon receiving perfluoroalkyl substance exposure biomarker results. Int. J. Hyg. Environ. Health 2018, 221, 1040–1046. [Google Scholar] [CrossRef]
- Haynes, E.N.; Elam, S.; Burns, R.; Spencer, A.; Yancey, E.; Kuhnell, P.; Alden, J.; Walton, M.; Reynolds, V.; Newman, N. Community engagement and data disclosure in environmental health research. Environ. Health Perspect. 2016, 124, A24–A27. [Google Scholar] [CrossRef] [PubMed]
- Centers for Disease Control and Prevention (CDC), Clear Communication Index. 2019. Available online: https://www.cdc.gov/ccindex/pdf/ClearCommUserGuide.pdf (accessed on 1 October 2020).
- Ramirez-Andreotta, M.D.; Brody, J.G.; Lothrop, N.; Loh, M.; Beamer, P.I.; Brown, P. Improving environmental health literacy and justice through environmental exposure results communication. Int. J. Environ. Res. Public Health 2016, 13, 690. [Google Scholar] [CrossRef]
|1. Tell me a bit about the previous process for preparing environmental or health data in individual reports for research participants (i.e., “report-back”).|
| a. What were the considerations for the methods of data presentation (e.g., in text, tables, or figures)|
b. How much time do you think each report took?
c. What was the process for review and editing by the research team?
|2. What were some of the benefits of the way these reports were developed before?|
|3. What were some of the limitations of this report development process?|
|4. With MCR, what has changed about your work?|
| a. How much time does each report take?|
b. How has this changed the considerations for the methods of data presentation (e.g., in text, tables, or figures)?
c. How has this changed the scope or goals for future environmental health projects?
|5. What are the benefits of MCR?|
|6. What are the limitations of MCR?|
|a. How much technical expertise is needed to adapt or modify MCR to develop new reports?|
|7. What suggestions do you have to adapt or modify MCR?|
|Study||Report Component||Manual Report-Back||Report-Back with MCR|
|CRESSH HOME||Number of pages||7||19|
|Number of individualized numerical or text results||4||23|
|Number of individualized tables||2||6|
|Number of individualized graphs||4||8|
|six individualized |
|ACHIEVE 1||Number of pages||2||2|
|Number of individualized numerical or text results||0||5|
|Number of individualized tables||0||1|
|Number of individualized graphs||0||1|
|Report-Back Process Qualities||Manual|
|Report-Back with MCR|
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).