Advice and Frequently Asked Questions (FAQs) for Citizen-Science Environmental Health Assessments

Citizen science provides quantitative results to support environmental health assessments (EHAs), but standardized approaches do not currently exist to translate findings into actionable solutions. The emergence of low-cost portable sensor technologies and proliferation of publicly available datasets provides unparalleled access to supporting evidence; yet data collection, analysis, interpretation, visualization, and communication are subjective approaches that must be tailored to a decision-making audience capable of improving environmental health. A decade of collaborative efforts and two citizen science projects contributed to three lessons learned and a set of frequently asked questions (FAQs) that address the complexities of environmental health and interpersonal relations often encountered in citizen science EHAs. Each project followed a structured step-by-step process in order to compare and contrast methods and approaches. These lessons and FAQs provide advice to translate citizen science research into actionable solutions in the context of a diverse range of environmental health issues and local stakeholders.


Introduction
Cumulative risk assessments (CRAs), health impact assessments (HIAs), cumulative impacts, environmental justice (EJ) scores, and related aggregators of environmental health information capture a range of factors that positively and negatively impact humans, animals, ecosystems, and the general state of the natural environment [1][2][3][4].Each approach has benefits and limitations and typically requires both quantitative measurements and qualitative weights of evidence [5][6][7].For example, the Centers for Disease Control and Prevention (CDC) lists six different types of health assessments on their website (https://www.cdc.gov/healthyplaces/types_health_assessments.htm).
For the scope of this paper, an Environmental Health Assessment (EHA) is a methodical evaluation of air, water, and soil pollution impacts on human health and the environment in the context of all possible health determinants, and the subsequent development of risk reduction actions.However, the term health is widely encompassing, ranging from cancer clusters, asthma attacks, cardiovascular emergency room visits, and species degradation, to psycho-social stress, quality of life, and community vulnerability [8].The surrounding environment includes anything the receptor can see, feel, touch, breathe, hear, taste, or otherwise sense and react to.The reason for this level of openness is to accommodate the variety of stressors (negative contributors) and salutary factors (positive contributors) in local areas and communities [9][10][11], as well as the range of local stakeholders associated with environmental health [1,12].
For groups with limited human, technical, and financial resources, EHA efficiency and effectiveness are important considerations.Publicly available EHA approaches do not currently exist that would mandate enforcement and compliance actions.Stakeholders (people directly or indirectly involved or affected) could include community residents and organizations, academics, commercial interests, and local/federal governments [1,13].Yet no standardized approach exists to transfer results from one entity to another, so that each entity is allowed a subjective decision to accept or reject the findings of another, not only between like entities, such as two separate community groups or state agencies, but especially between different entity types.For example, government agencies are under no legal obligation to change policies based on citizen science measurements [1,14,15].
The network of partners who develop EHAs can be more influential in promoting change than the quantitative results themselves [6,16].This is an important consideration for inexperienced groups who hope that quantitative results alone will promote improvements in human and environmental health.Sometimes the process is more impactful than the utility of the data collected [17].This paper presents two citizen science EHAs and reflects on a decade of collaborations.This work culminated in three lessons learned and a set of Frequently Asked Questions (FAQs), focused on public communication for citizen science projects.This work is not intended as a framework or conceptual approach, a step-by-step method, or a roadmap.It provides advice to translate research into action, manage expectations, facilitate understanding, and target resources.

Methods
Two locally-based projects and over a decade of collaborative input contributed to this work (Table 1).The two projects included partners from government, community, academia, and non-profit organizations.State and local agencies and commercial interests were not included; however, they have worked with each respective partner outside of these efforts.Partners represented different types of organizations and geographic scales of interest, from residents focused on their neighborhood, to regional agencies covering multiple states, to federal researchers hoping to transfer successful methods across the country.
Projects were led by partnerships that encountered professional and inter-personal challenges.This is the nature of real-world, non-regulatory assessments and one of the pivotal reasons for this research: to help mediate the scientific, personal, and political aspects of environmental health decision-making.Both projects, and several from Table 1, followed an identical, structured, iterative process [1] with rigorous documentation, allowing compare-and-contrast among the different projects.These findings led to development of the lessons learned and FAQs.Steps included: 1.
Form partnership and identify stakeholders 2.
Identify environmental health stressors and salutary factors 4.
Collect data, topic-expert knowledge, and local input 5.
Rank environmental health stressors and salutary factors 6.
Identify risk mitigation strategies 7.
Collect information on technical, financial, and human resources for mitigation 8.
Plan post-project long-term goals 10.Measure success using agreed-upon metrics Newark, Newport News, and several projects from Table 1 maintained regular monthly meetings with the entire partnership throughout the course of each project, with interim meetings between smaller groups to maintain specific elements.Rigorous meeting notes and summaries were compiled for each project, with challenges and limitations being a primary discussion topic.More than 40 meetings were held between the Newark and Newport News projects alone, and over 100 meetings total for all projects, with participants having equal time to voice topics and concerns.Documentation for each assessment step, including meeting notes, summary tables, data outputs, and proposed approaches, were then compared and contrasted across projects in order to summarize the primary challenges (i.e., lessons learned) and research questions (i.e., FAQs).
While FAQs were in development with several of the earlier projects listed in Table 1, they came to fruition during the Newark project primarily.That project had significant challenges with interpretation and communication, with multiple meetings to address them.For example, the community group expressed concerns that local health issues were not being addressed and that sensor measurements would not influence policy, whereas Environmental Protection Agency (EPA) sensor experts stated that the research design and sensor capabilities were unable to directly address these concerns and that a local scale characterization of air quality concentrations was the extent of the research, in addition to the local training on sensor capabilities.Multiple facilitated discussions led to a communications plan to address this disconnect.After the project, remaining partners synthesized the FAQs presented here to help increase transparency for future projects.These FAQs were then applied to the Kansas City project (Table 1) during its research design, including with the Community Action Group (CAG) liaison, and were also vetted through EPA citizen science communications experts and multiple EPA Regional personnel who work directly with communities on HIAs.descendants.The area's name refers to the proliferation of railway and metalworking industries.It is bounded by highways, waterways, and lies adjacent to the Newark International Airport and the Port of Newark/Elizabeth, making it a hub of multi-modal freight transport.The population of more than 50,000 represents one of the most densely populated and diverse communities of the city.Two-thirds of the population are foreign-born, and 75% of those over the age of five speak a foreign language at home, typically Portuguese or Spanish.Educational attainment is relatively low with 55% of those over 18 not having a high school diploma.Census tracts range from 25% to 55% of households below the poverty level.Hundreds of tenants from three public housing complexes (comprised of more than 700 units) live near industrial land uses, hazardous and Superfund sites, and major highways.More information on the Ironbound community and this project can be found in Kaufman et al. [18].

Partners, Goals, Objectives
Local, state, and federal partners were represented in this project.The Ironbound Community Corporation (ICC) was founded in 1969 with the mission to, "engage and empower individuals, families, and groups in realizing their aspirations and, together, work to create a just, vibrant, and sustainable community."EPA Region 2 is responsible for the execution of EPA programs within the states of New Jersey, New York, the Commonwealth of Puerto Rico, and the U.S. Virgin Islands.EPA ORD develops knowledge, assessments, and scientific tools that form the underpinnings of the majority of EPA protective standards and guidance.The overall goal of this project was to characterize urban pollution using portable sensors, particularly near roadways.While maintaining a common goal, each partner also had their own objectives (Tables 2 and 3).Data collection in The Ironbound community involved the deployment of portable sensor assemblies called Citizen Science Air Monitors (CSAM).CSAM units measured nitrogen dioxide (NO 2 ) and particulate matter (PM) simultaneously.NO 2 is a highly reactive gas that can irritate the lungs and contribute to bronchitis, pneumonia, and other respiratory impacts [19,20].NO 2 occurs naturally in the atmosphere, as well as through man-made processes such as power plants, industrial processes, and fuel combustion, forming quickly from automobile emissions; elevated concentrations are often found near roadways [21].PM consists of particles such as soot, smoke, dirt, and dust occurring in various sizes.PM can adversely affect breathing and aggravate respiratory and cardiovascular problems and a host of other issues [22], with the greatest impacts from the smallest particles, including those less than 2.5 microns in diameter (PM 2.5 ) [23].In addition to NO 2 and PM 2.5 , CSAM units also recorded temperature and relative humidity (RH) with measurements recorded in 5-min intervals.
The self-contained CSAM units included a CairPol CairClip NO 2 sensor, a Thermo Scientific personal DataRAM PM 2.5 sensor, and a Honeywell temperature and RH sensor (Figure 1a,b).NO 2 was recorded in parts per billion (ppb); PM 2.5 in micrograms per cubic meter (µg/m 3 ); temperature in degrees Celsius ( • C); and relative humidity in percent at a given temperature (% at • C).Sensors were housed in a weatherproof container with a lockable access door.Data could be retrieved via removable memory cards or accessed directly through a universal serial bus (USB) port on the installed datalogger.Four CSAM units were moved every 2-4 weeks to different sampling locations between February and July of 2015.CSAM units were either mounted on a tripod outdoors and connected to a battery supply, or placed indoors and plugged into a socket while using a snorkel tube to sample outdoor air through a window.They were not designed to sample indoor air.All four units were collocated with FRM monitors for a one-week period near the middle of the study in order to help correct measurement offsets.

Data Collection
Data collection in The Ironbound community involved the deployment of portable sensor assemblies called Citizen Science Air Monitors (CSAM).CSAM units measured nitrogen dioxide (NO2) and particulate matter (PM) simultaneously.NO2 is a highly reactive gas that can irritate the lungs and contribute to bronchitis, pneumonia, and other respiratory impacts [19,20].NO2 occurs naturally in the atmosphere, as well as through man-made processes such as power plants, industrial processes, and fuel combustion, forming quickly from automobile emissions; elevated concentrations are often found near roadways [21].PM consists of particles such as soot, smoke, dirt, and dust occurring in various sizes.PM can adversely affect breathing and aggravate respiratory and cardiovascular problems and a host of other issues [22], with the greatest impacts from the smallest particles, including those less than 2.5 microns in diameter (PM2.5)[23].In addition to NO2 and PM2.5, CSAM units also recorded temperature and relative humidity (RH) with measurements recorded in 5-min intervals.
The self-contained CSAM units included a CairPol CairClip NO2 sensor, a Thermo Scientific personal DataRAM PM2.5 sensor, and a Honeywell temperature and RH sensor (Figure 1a,b).NO2 was recorded in parts per billion (ppb); PM2.5 in micrograms per cubic meter (µg/m 3 ); temperature in degrees Celsius (°C); and relative humidity in percent at a given temperature (% at °C).Sensors were housed in a weatherproof container with a lockable access door.Data could be retrieved via removable memory cards or accessed directly through a universal serial bus (USB) port on the installed datalogger.Four CSAM units were moved every 2-4 weeks to different sampling locations between February and July of 2015.CSAM units were either mounted on a tripod outdoors and connected to a battery supply, or placed indoors and plugged into a socket while using a snorkel tube to sample outdoor air through a window.They were not designed to sample indoor air.All four units were collocated with FRM monitors for a one-week period near the middle of the study in order to help correct measurement offsets.

Location Description
Founded in 1896, Newport News, Virginia is located along the James River and has a long history of port activities, shipbuilding, maritime commerce, military activity, and technological

Location Description
Founded in 1896, Newport News, Virginia is located along the James River and has a long history of port activities, shipbuilding, maritime commerce, military activity, and technological development.In addition to port and maritime operations, Newport News includes major rail lines and yards, interstate highways, and coal-handling facilities.There are four major ports in or around Newport News: The Newport News Marine Terminal, Norfolk International Terminals, Portsmouth Marine Terminal, and the Virginia International Gateway.In 2013, Newport News had a reported population of 182,015 people, 41% of whom were African American, 24% of had obtained a bachelor's degree or higher, and 15% lived below the poverty level.

Partners, Goals, Objectives
The primary goal of the partnership was to characterize environmental health stressors and identify potential risk reduction actions (Table 4).Partners for the Newport News project included the Southeast CARE Coalition (SCC), a local community organization; EPA Region 3, headquartered in Philadelphia, Pennsylvania, and EPA ORD; The Sierra Club of Virginia, a non-governmental organization (NGO); The University of North Carolina Capstone Course, an undergraduate self-guided research course; a doctoral graduate from Old Dominion University, who had worked previously with the community; and The Greater Southeast Development Corporation, who had worked with SCC previously and continued working with them outside of this particular project.Data collection for the Newport News project included a combination of online databases, research-grade (i.e., non-regulatory) air quality models, and local and expert knowledge.Portable sensor measurements were not a part of this study.Prior to the formation of the partnership, SCC hosted town hall environmental health symposiums throughout the Southeast Community, where participants were asked to list and rank their environmental concerns.The partnership utilized this prior information to target research activities, iteratively refining a list of environmental health issues on which to focus data analysis and risk reduction options.

Newark
Sensor measurements helped to understand overall conditions for the area, as well as possible high-concentration areas, such as near roads.However, definitive results were difficult to ascertain.Newark measurement collection occurred over several months, from February to July of 2015, with four sensor units being moved every 2-4 weeks between sites, with beginning and end times not always synchronized.Occasional technical malfunctions in certain units limited data collection and contributed to uncertainty about their resulting measurements.Snorkel tubes sampled outdoor air for the units kept indoors, which also contributed to uncertainty.Presented here are a summary of measurements and snorkel considerations, and collocation results with data correction approaches.Results are intended to capture the challenges of translating citizen science results into quantifiable impacts and follow-up actions.

Data Summary
A summary of recorded measurements is presented in Table 5, including locations and CSAM unit designations, time periods of placement and number of records, snorkel use, data quality flags, and mean values for PM 2.5 and NO 2 concentrations, temperature, and relative humidity.Kaufman et al. [18] includes a geographic distribution of measurements.For all time periods and CSAM units (excluding the single day of measurements on 30 July), mean PM 2.5 ranged between 10.4 and 20.1 µg/m 3 ; and 90th percentile values, between 10.9 and 31.8 µg/m 3 .Overall mean PM 2.5 was 14.5 µg/m 3 ; overall 90th percentile mean was 20.3 µg/m 3 .For reference, it must be acknowledged that these CSAM results are not comparable to federal standards for several reasons (technology, quality control, calibration, calculation methods, etc.), ambient standards for PM 2.5 are 35 µg/m 3 for a 24-h average, and 12 µg/m 3 for an annual average.Mean NO 2 ranged between 4.0 and 47.2 ppb; and 90th percentile values, between 8.1 and 87.4 ppb.Overall mean NO 2 was 18.9 ppb; overall 90th percentile mean was 40.4 ppb.Like PM 2.5 , federal standards are not directly comparable; however, for reference, NO 2 ambient standards are 100 ppb for a 1-h average, and 53 ppb for an annual average.As expected, because of its dependence on local sources, NO 2 demonstrated greater variability between sites compared to PM 2.5 .Data were marked with a green, yellow, or red flag based on good, questionable, or bad data quality, respectively.Green meant that no known issues were found; yellow referred to technical issues or measurement abnormalities; and red typically meant that no measurements were collected because of technical malfunctions.Red flags represented technical failures that precluded measurement collection.Most yellow flags relate to the snorkel assemblies, suggesting leakage of indoor air into the snorkel.Figure 2 shows a comparison between two units at different locations for the same time period, one with a snorkel tube and one without.Snorkel air temperatures recorded slight variations throughout the day compared to the outdoor unit, the snorkel unit hovering closer to room temperature than outdoor temperatures; however, minor variations based on outdoor patterns do appear.Snorkel RH also did not reflect outdoor diurnal patterns and showed minimal variability due to outdoor conditions.Snorkel PM 2.5 did reflect outdoor variability, but recorded consistently lower values, possibly suggesting attenuation through infiltration and sampling of indoor air.NO 2 showed no variation and recorded near-zero concentrations for the entire period, suggesting that outdoor air was not sampled.Most snorkeled values reflected these same patterns, and were flagged for potential data quality issues, which resulted in over half the samples having questionable quality.Data were marked with a green, yellow, or red flag based on good, questionable, or bad data quality, respectively.Green meant that no known issues were found; yellow referred to technical issues or measurement abnormalities; and red typically meant that no measurements were collected because of technical malfunctions.Red flags represented technical failures that precluded measurement collection.Most yellow flags relate to the snorkel assemblies, suggesting leakage of indoor air into the snorkel.Figure 2 shows a comparison between two units at different locations for the same time period, one with a snorkel tube and one without.Snorkel air temperatures recorded slight variations throughout the day compared to the outdoor unit, the snorkel unit hovering closer to room temperature than outdoor temperatures; however, minor variations based on outdoor patterns do appear.Snorkel RH also did not reflect outdoor diurnal patterns and showed minimal variability due to outdoor conditions.Snorkel PM2.5 did reflect outdoor variability, but recorded consistently lower values, possibly suggesting attenuation through infiltration and sampling of indoor air.NO2 showed no variation and recorded near-zero concentrations for the entire period, suggesting that outdoor air was not sampled.Most snorkeled values reflected these same patterns, and were flagged for potential data quality issues, which resulted in over half the samples having questionable quality.

Collocation and Sensor Corrections
The four CSAM units were collocated for seven days (7-15 April) at a New Jersey Department of Environmental Protection (NJDEP) National Core (NCore) station with FRM monitors.Two units experienced pump failures that resulted in sparse data collection.Measurements were used to develop a regression equation for each CSAM unit to adjust its measurements.Even though regressions were based on one week of collocation, regressions were applied to the entire dataset.Figures 3 and 4 show an example of a CSAM unit before and after the regression adjustments, respectively.Relative humidity incurred the least adjustment, agreeing well between CSAM and the FRM.Temperatures agreed well except for maximum excursions, which decreased with adjustment.CSAM represented diurnal NO2 trends well, but with consistently lower concentrations, which increased to FRM values after adjustment.CSAM PM2.5 were generally greater than FRM, especially for maximum values, and diurnal patterns were not as clear or pronounced in the CSAM

Collocation and Sensor Corrections
The four CSAM units were collocated for seven days (7-15 April) at a New Jersey Department of Environmental Protection (NJDEP) National Core (NCore) station with FRM monitors.Two units experienced pump failures that resulted in sparse data collection.Measurements were used to develop a regression equation for each CSAM unit to adjust its measurements.Even though regressions were based on one week of collocation, regressions were applied to the entire dataset.Figures 3 and 4 show an example of a CSAM unit before and after the regression adjustments, respectively.Relative humidity incurred the least adjustment, agreeing well between CSAM and the FRM.Temperatures agreed well except for maximum excursions, which decreased with adjustment.CSAM represented diurnal NO 2 trends well, but with consistently lower concentrations, which increased to FRM values after adjustment.CSAM PM 2.5 were generally greater than FRM, especially for maximum values, and diurnal patterns were not as clear or pronounced in the CSAM measurements.These corrections demonstrate the value of collocation for citizen science projects, especially for emerging sensor technologies.
measurements.These corrections demonstrate the value of collocation for citizen science projects, especially for emerging sensor technologies.

Next Steps
Methods and results were presented at a public forum to characterize measurements and discuss potential next steps.This meeting represented the culmination of the project, so next steps would involve local community partners and how they wanted to proceed.Two primary concepts were conveyed during the report-out: technical abilities of the sensors and context for the measurements.Technical concepts included sensor capabilities and examples of how CSAM and FRM measurements differed.Context included explaining the limitations of the measurements in terms of deducing health impacts and the inability to compare CSAM results directly to regulatory standards.Instead, average measurements were compared to non-regulatory annual averages for 10 different cities, which also was not a direct comparison, nor was it a comparison to regulations.
Once sensors and context were discussed, participants contributed to possible next steps.The audience provided several suggestions, including a water quality assessment; incorporating a broader network of sensors; evaluating the impact of the nearby international airport; assessing cumulative risks; strengthening collaborative research opportunities with local, academic, government, and NGO partners; targeting sensor measurements based on high-concentration areas; and a health-based (e.g., cancer cluster) epidemiological approach.The depth and breadth of considerations exemplifies the power of local knowledge to contribute to targeted solutions, and the capacity of non-scientists to understand complex information if presented transparently and in an appropriate context.

Data Summary
The Newport News partnership collected and analyzed local environmental health information, with an emphasis on potential impacts from port activities, goods movement, and transportation, focusing specifically on port operations, coal piles, shipbuilding, toluene, interstates, asthma, brownfields, food deserts, wastewater treatment facilities, stormwater and sewer line breaks, polychlorinated biphenyl (PCB) contamination, and susceptible and vulnerable populations.Interstate traffic led to concerns about ozone, particulate matter, acetaldehyde, acrolein, benzene, 1,3 butadiene, formaldehyde, diesel particles, carbon monoxide, nitrogen oxide (NO X ), sulfur oxide (SO X ), and volatile organic compounds (VOCs).Coal holding facilities led to additional concerns about air quality (see EPA [24] for more information).A list of industrial and traffic-related pollutants and their potential impacts was compiled for stakeholders (Table 6).Exposure scientists clarified that this summary was not conclusive of exposure-health associations.To facilitate risk mitigation efforts, a further table was compiled about potential actions, including information for each environmental health issue, quantification metrics, data sources, and potential risk-reduction actions (Table 7).

Next Steps
The Newport News project culminated in an EPA summary report, including environmental health issues, data summaries, and potential next steps [24].SCC and EPA Region 3 continued working together after the project to provide training and continue education/outreach and mitigation options.The City of Newport News considered the community work and concerns that came out of SCC's actions, and identified five key priorities: 1.
Increase Trust-Engage local groups to reduce mistrust between them and the city, and assist city employees to improve their comfort level of working with community-based organizations.

2.
Encourage Business Development-Secure Housing and Urban Development funds and create an environment for small businesses to open and thrive and make the city more livable.

3.
Transportation-Evaluate quick, reliable transportation options to shuttle workers to jobs to facilitate local employment and create more livable circumstances in the community.

4.
Promote Home Ownership-Develop a rental rehabilitation and rental inspection program; explore an education program to promote relationships between banks and borrowers; explore a demolition program to remove old infrastructure and rehab vacant lots.

5.
Infrastructure-Promote building a new grocery store; improve natural landscapes by planting trees.

Discussion
One limitation of this study is that no formal questionnaires or interviews were performed to specifically ask participants for lessons and FAQs.Other studies that included online questionnaires [25], semi-structured interviews [26], or a combination of both [27] have an advantage of quantifying results to provide weights and relative importance of responses, as well as to have a comprehensive and consistent response strategy.While the projects presented here and reported in Table 1 did not include these survey mechanisms, they did follow a consistent and structured step-by-step approach (outlined in Methods) similar to the rubric to evaluate citizen science programs presented by Tredick et al. [28].As part of federal research program standards, rigorous notes, and regular deliverables (summaries, graphs, tables, etc.) were necessary for each project, thus providing thorough, comparable materials from which to draw conclusions.In addition, local community stakeholders provided regular feedback during project formulation stages that contributed to these materials, and EPA Regional and Federal personnel who work closely with local and community organizations and agencies reviewed the findings and provided feedback.

Decision-Making: People Versus Data
Empowerment has two connotations: the first is to grant decision-making authority, another is to build strength and confidence to take control of a situation.Between these two endpoints is a gray area into which citizen science EHAs could fall, providing participants with an understanding of detrimental situations without the legal authority to enforce mitigation.Unlike regulatory standards, citizen science measurements and data analyses are subjective forms of legal tender-only those who accept them will place value in the results [29].The more substantive the research plan, quality assurance requirements, citizen training, expert input, and peer review a citizen science plan has, the more likely the data will be accepted as objective reality by outsiders [30,31].Groups may not be able to afford projects like the ones outlined above; for example, CSAM units cost approximately $6000-$7000 for parts and professional assembly.However, outside of the sensors, time was largely voluntary or part of EPA researchers' official duties, and data mining drew from free, publicly available sources.Limitations in terms of technical abilities to access and analyze data may prove prohibitive, yet limited financial resources can go a long way given the appropriate expertise and a thoughtful research design that is targeted to the decision-making audience.
Multi-partner collaborations could contribute to a perception of empowerment as well as a perception of adversity, based on whether stakeholders consider themselves insiders or outsiders to accomplishing the goal.If a community is partnering with a government agency, as occurred in the above examples, there could be a de facto expectation that policy changes and mitigation actions will ensue; that partnering with a federal agency empowers the community with legal authority to take action.In some projects listed in Table 1, the communities expressed excitement and willingness to partner with the EPA, yet this enthusiasm waned when community expectations of health improvement did not directly follow the relatively limited data collection effort.This perception of empowerment was not realized.Similarly, local health and environmental agencies, and private/commercial interests, could retain a perception of adversity, being cautious to pursue a collaboration should there be expectations that local agencies implement new policies or relinquish control, or commercial interests undergo enforcement and compliance actions or change business practices, based on the results, especially if the federal government's main partner is the local community.Conversely, these perceptions could reverse depending with whom the federal government partners.Several projects in Table 1 had to address this dynamic, and in most cases, potential opponents to the goal were not engaged, possibly to the detriment of long-term success.Perceptions of both empowerment and adversity can be a barrier to building partnerships with precisely the parties needed to implement change.
Through this lens of data and decisions, three lessons and multiple FAQs came into focus.While locally-based, citizen science EHAs may share certain commonalities, each is unique in its people, problems, and approaches [32,33].Sometimes the greatest environmental risk is not always the highest priority because of local values or resources.Thus, lessons and FAQs presented here are based on commonalities from multiple projects, but tailorable to each unique situation.

Partners May Share Goals, but Not Objectives
In a multi-partner EHA, it is quite possible that each partner will agree on the same goal, but have a different set of objectives, such as for the Newark project (Table 2).For the purpose of this research, a goal is defined as a long-term aspiration or aim, and could be nebulous and difficult to measure; examples include, "Improve community health," or, "Protect vulnerable populations from environmental stressors."Alternatively, objectives are concrete, measureable achievements that can be attained after certain steps, typically with short-or medium-term timeframes; examples include, "Host town hall meetings to educate community on emission sources," or, "Measure air quality to characterize near-road hotspots."All partners should fully understand the exact goals and objectives of data collection, answering, "Why, exactly, are we doing this?" Or, as a community leader aptly described, to "work with intention." Objectives should be as specific and unambiguous as possible.The Newark and Newport News approaches were quite different-one being sensor-driven and the other focused on existing data-and yet the community stakeholders had similar objectives: to improve community health through various actions, including education/outreach, policy change, and emissions mitigation.However, the Newark sensor project experienced technical challenges and translation of results to action.Also, in Newark, EPA experts provided the community with sensors and sensor expertise, but the community wanted measurements to assess health impacts and support mitigation actions.There was a disconnect between results and solutions that never fully resolved.In the end, the community was given a thorough summary of all the technical aspects, and retained ownership of the measurements, but follow-through for actions was primarily their responsibility.In Newport News, the initial approach was quite different.First, the project never planned to use sensors because of the project funding and research logistics.Also, the primary goal was to test and apply an assessment process and new GIS and data mining methods, so education/outreach was a primary goal.There were also several complementary efforts going on in the EPA Region to continue working with the Newport News community, so follow-up actions were also supported by the professional network rather than project results.For successful EHAs, goals and objectives should be delineated for each group in the partnership, and roles and responsibilities for each group and individual [34].These should be discussed early and revisited at planned intervals throughout the project.Stakeholder engagement involves consideration of each group or individual and determination of whether their objectives are supported by the data collection and assessment efforts.

Graphs Do Not Make Decisions, People Do
Data and measurements alone do not enact change; people do [35,36].There is no established method to translate citizen science sensor measurements into regulatory action, for example [18,30].Arguably, the best use of data in these cases is to develop a hypothesis-driven narrative that uses data as evidence of potential emissions, exposures, and impacts.The exact same data is capable of either substantiating bias or changing minds, depending on how they are presented and who is looking at them.For example, average PM 2.5 concentrations in Newark did not exceed national standards for PM, but did place the community within the top 10 cities with high annual average PM 2.5 .However, measurements and calculations for the national standards and the citywide approach were not equivalent to the project design, so were not representative comparisons, but at least provided some context.Newark data were suggestive but not conclusive.Would community members take results as proof of their concerns?Would regulators cast doubt on the measurement quality?Would both groups agree to a more detailed study or precautionary actions, or conversely, conclude it was all a waste of time and effort?
The answer to combatting subjectivity is simple: engage all decision-making authorities early in the process; plan a scientifically-defensible, peer-reviewed research plan; and develop early and ongoing consensus on how the group will respond to results (e.g., "If we find this, then we'll do that").The answer is simple; implementation is not.In Newark, local government authorities were not engaged early in the process, yet it became clear that partners were interested in presenting results to influence local policies.The partners for Newport News agreed on a general characterization approach and not political action, so decision-making authorities were not involved.Neither project considered the specific question of how data were to be used after collection to influence decisions or next steps.
Two things that any project can do to help ensure relevance are: (1) develop a scientifically sound research design, and (2) know your audience.Sound strategies include sensor collocation and calibration; expert training; representative spatial and temporal scales; and regular quality control checks [30,36].The EPA provides sampling strategies on their Citizen Science Toolkit webpage (https:// www.epa.gov/air-sensor-toolbox).Knowing your audience means researching who would be involved in decision-making based on the data and knowing how to communicate to them.For community members, how will data be presented in a meaningful way?For local agencies or policy makers, what are their data quality requirements?A sound strategy and a practical approach will help ensure that time, effort, and passion are not wasted.

Measure Success and Manage Expectations Regularly and Often
Groups are composed of individuals who have opinions on how things should run, who is in charge, and what things mean.In a collaborative EHA, question and clarify expectations constantly to determine whether the data will support project goals and objectives.For example, researchers have a proclivity to assume that more research is necessary is a reasonable conclusion to a project.After more than two years of developing and implementing a citizen science project in Newark, this conclusion would have been unacceptable to all partners as the sole result.EHAs evolve through time, so adaptive management is necessary to stay focused on the original goal or develop a new one when applicable.Set a few reasonable, attainable objectives, and when they are met, celebrate them.Advertise success; market it as justification for the work; use it to incentivize participation.Success is the fuel that keeps collaborations running smoothly.
These projects offered benefits for all participants, yet neither project led directly to improvements in health, reductions in contaminant emissions, or changes in policy.However, both projects led to long term collaborations with city and state agencies to promote environmental health, and while unexpected, were welcome outcomes.Since Newport News was an MVD community, several other projects were similarly occurring in the area, and ultimately the community and city worked together to develop a common understanding and potential future directions.In Newark, the sensor results were not used to assess health impacts or identify local hotspots.However, the empowerment of a community conducting such an intensive sampling campaign led to discussions with the city of Newark to engage and promote environmental health activities and bring greater awareness to local community concerns.While results from neither project directly supported change, the process and empowerment eventually promoted environmental health efforts through enhanced collaboration and awareness.

FAQs
Over 30 FAQs for citizen science EHAs were developed based on a compare and contrast of each step of the assessment process for the Newark and Newport News projects, as well as for several from Table 1.Lead organizers for the citizen science projects in Newark and Newport News, as well as for Kansas City, and HIA and EJ experts on locally-based projects provided input and feedback to the FAQs.Each step included interactions with various partners, including communities, local/state agencies, research design, training, and data analysis; and T.M.B. organized and managed stakeholder engagement, analyzed data, interpreted, and communicated results.T.M.B. and J.E. provided research support to the design, analysis, interpretation, communication, education/outreach for the Newport News case study.

Figure 1 .
Figure 1.(a) Interior of Citizen Science Air Monitors (CSAM) air quality sensor assembly; (b) Tripod mounting system with battery attachment for outdoor use.

Figure 1 .
Figure 1.(a) Interior of Citizen Science Air Monitors (CSAM) air quality sensor assembly; (b) Tripod mounting system with battery attachment for outdoor use.

Figure 2 .
Figure 2. Comparison of two CSAM units with contemporaneous measurements, one with a snorkel tube and one without (not collocated in space).

Figure 2 .
Figure 2. Comparison of two CSAM units with contemporaneous measurements, one with a snorkel tube and one without (not collocated in space).

Figure 3 .
Figure 3.Comparison of one CSAM unit with federal reference monitors during collocation at NCORE station.

Figure 4 .
Figure 4. Results of applying regression equation to one CSAM unit based on corrections from federal monitors during collocation at NCORE station.

Figure 3 .
Figure 3.Comparison of one CSAM unit with federal reference monitors during collocation at NCORE station.

Figure 3 .
Figure 3.Comparison of one CSAM unit with federal reference monitors during collocation at NCORE station.

Figure 4 .
Figure 4. Results of applying regression equation to one CSAM unit based on corrections from federal monitors during collocation at NCORE station.Figure 4. Results of applying regression equation to one CSAM unit based on corrections from federal monitors during collocation at NCORE station.

Figure 4 .
Figure 4. Results of applying regression equation to one CSAM unit based on corrections from federal monitors during collocation at NCORE station.Figure 4. Results of applying regression equation to one CSAM unit based on corrections from federal monitors during collocation at NCORE station.

Table 1 .
Description of locally-focused projects related to environmental health assessments facilitated by the Environmental Protection Agency (EPA) Office of Research and Development (ORD) (does not represent totality of ORD projects; Chicago includes two separate projects).Projects with (*) are covered in this paper.

Table 2 .
List of partner groups and respective objectives for the Newark, Ironbound community project.

Table 3 .
List of individual roles and responsibilities for the Newark, Ironbound community project.

Table 4 .
List of partners, roles, and responsibilities/objectives for Newport News project.

Table 5 .
Data summary for The Ironbound community citizen science measurements.

Table 6 .
List of chemicals and potential exposure-response symptoms associated with emissions sources in the Newport News project study area.

Table 7 .
List of environmental health issues and their related quantitative metrics, data sources, and potential risk reduction approaches for Newport News project.