Does Training Improve Sanitary Inspection Answer Agreement between Inspectors? Quantitative Evidence from the Mukono District, Uganda
Abstract
:1. Introduction
2. Materials and Methods
2.1. Study Area and Supply Type Selection
2.2. SI Form Development
2.3. Fieldwork and Inspector Recruitment and Training
2.4. Study Design
2.5. Data Analysis
2.5.1. Pearson’s Correlation Coefficient and Lin’s Concordance Correlation Coefficient (Lin’s CCC)
2.5.2. Coefficient of Variation
2.5.3. Index of Dispersion
- (a)
- The mean index of dispersion per individual question (1–12)
- (b)
- The mean index of dispersion per individual supply (1–36)
2.6. Missing Data Treatment
2.7. Research Ethics
3. Results
3.1. SI Score Correlation and Agreement between the Instructor and Each Inspector, Pre- and Post-Training
3.2. SI Score Variability between Inspectors, Pre- and Post-Training
3.3. Consistency of Answer Agreement between Inspectors for Each Individual SI Question, Pre- and Post-Training
- (a)
- Training improved mean answer (Yes/No and risk level) agreement of SI questions (1–12) (Table 6), though comparable standard deviations pre- and post-training for both answer types indicate certain questions still exhibited nonagreement. Answer agreement for Yes/No questions improved post-training for questions 2–5 and 8–11. Slight decreases in answer agreement were observed for questions 1, 6, 7 and 12 (Figure 4). When estimating risk level, training improved answer agreement for questions 1–5 and 8–11. Slight decreases in answer agreement were observed for questions 6, 7 and 12 (Figure 5).
- (b)
- Training improved mean answer (Yes/No and risk level) agreement per supply (Table 7), however, increases in standard deviation values post-training for both answer types indicate sustained/increased nonagreement of answers at some supplies, for example, supply 31. Greater answer nonagreement was observed in estimation of risk level as opposed to Yes/No questions. Of the seven supplies (6,9,24,27,28,31,36) that exhibited lower mean Yes/No answer agreement post-training, four of these supplies (6,9,24,31) also exhibited lower mean risk level answer agreement post-training (Figure 6 and Figure 7).
3.4. Missing Data Treatment
4. Discussion
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Bain, R.; Bartram, J.; Elliott, M.; Matthews, R.; McMahan, L.; Tung, R.; Chuang, P.; Gundry, S. A summary catalogue of microbial drinking water tests for low and medium resource settings. Int. J. Environ. Res. Public Health 2012, 9, 1609–1625. [Google Scholar] [CrossRef] [PubMed]
- WHO. Guidelines for Drinking-Water Quality, 4th ed.; World Health Organization: Geneva, Switzerland, 2011; Available online: https://www.who.int/water_sanitation_health/publications/2011/dwq_guidelines/en/ (accessed on 5 July 2020).
- WHO. Guidelines for Drinking-Water Quality, 3rd ed.; World Health Organization: Geneva, Switzerland, 2004; Volume 1. Available online: https://www.who.int/water_sanitation_health/publications/gdwq3/en/ (accessed on 1 July 2020).
- Bain, R.; Cronk, R.; Wright, J.; Yang, H.; Slaymaker, T.; Bartram, J. Fecal contamination of drinking-water in low-and middle-income countries: A systematic review and meta-analysis. PLoS Med. 2014, 5, e1001644. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- WHO. Sanitary Survey Fact Sheet 2.1: Sanitary Inspections; World Health Organization: Geneva, Switzerland, 1996; Available online: http://helid.digicollection.org/en/d/Js13461e/2.2.html (accessed on 1 July 2020).
- Pond, K.; King, R.; Herschan, J.; Malcolm, R.; McKeown, R.M.; Schmoll, O. Improving Risk Assessments by Sanitary Inspection for Small Drinking-Water Supplies—Qualitative Evidence. Resources 2020, 9, 71. [Google Scholar] [CrossRef]
- Kelly, E.R.; Cronk, R.; Kumpel, E.; Howard, G.; Bartram, B. How we assess water safety: A critical review of sanitary inspection and water quality analysis. Sci. Total Environ. 2020, 718, 137237. [Google Scholar] [CrossRef] [PubMed]
- Lloyd, B.J.; Bartram, J.K. Surveillance solutions to microbiological problems in water quality control in developing countries. Water Sci. Technol. 1991, 24, 61–75. [Google Scholar] [CrossRef]
- Haruna, R.; Ejobi, F.; Kabagambe, E.K. The quality of water from protected springs in Katwe and Kisenyi parishes, Kampala city, Uganda. Afr. Health Sci. 2005, 5, 14–20. [Google Scholar] [PubMed]
- Luby, S.P.; Gupta, S.K.; Sheikh, M.A.; Johnston, R.B.; Ram, P.K.; Islam, M.S. Tubewell water quality and predictors of contamination in three flood-prone areas in Bangladesh. J. Appl. Microbiol. 2008, 105, 1002–1008. [Google Scholar] [CrossRef]
- Parker, A.H.; Youlten, R.; Dillon, M.; Nussbaumer, T.; Carter, R.C.; Tyrrel, S.F.; Webster, J. An assessment of microbiological water quality of six water source categories in north-east Uganda. J. Water Health 2010, 8, 550–560. [Google Scholar] [CrossRef] [Green Version]
- Mushi, D.; Byamukama, D.; Kirschner, A.K.; Mach, R.L.; Brunner, K.; Farnleitner, A.H. Sanitary inspection of wells using risk-of-contamination scoring indicates a high predictive ability for bacterial faecal pollution in the peri-urban tropical lowlands of Dar es Salaam, Tanzania. J. Water Health 2012, 10, 236–243. [Google Scholar] [CrossRef] [Green Version]
- Oliver, J.T. Predictors of E. Coli Contamination at Rural Water Points in Kenya, Malawi, Mozambique, Uganda, and Zambia. Ph.D. Thesis, The University of North Carolina, Chapel Hill, NC, USA, 2015. [Google Scholar]
- Ercumen, A.; Naser, A.M.; Arnold, B.F.; Unicomb, L.; Colford, J.M., Jr.; Luby, S.P. Can sanitary inspection surveys predict risk of microbiological contamination of groundwater sources? Evidence from shallow tubewells in rural Bangladesh. Am. J. Trop. Med. Hyg. 2017, 96, 561–568. [Google Scholar] [CrossRef] [Green Version]
- Misati, A.G.; Ogendi, G.; Peletz, R.; Khush, R.; Kumpel, E. Can sanitary surveys replace water quality testing? Evidence from Kisii, Kenya. Int. J. Environ. Res. Public Health 2017, 14, 152. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Snoad, C.; Nagel, C.; Bhattacharya, A.; Thomas, E. The effectiveness of sanitary inspections as a risk assessment tool for thermotolerant coliform bacteria contamination of rural drinking water: A review of data from west Bengal, India. Am. J. Trop. Med. Hyg. 2017, 96, 976–983. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Tosi, R.D.; Schertenleib, A.; Kunwar, B.M.; Shrestha, R.; Bhatta, M.; Marks, S.J. Assessing the impact of a risk-based intervention on piped water quality in rural communities: The case of Mid-Western Nepal. Int. J. Environ. Res. Public Health 2018, 15, 1616. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Lloyd, B.; Suyati, S. A pilot rural water surveillance project in Indonesia. Waterlines 1989, 7, 10–13. [Google Scholar] [CrossRef]
- Darker, C.; Whittaker, A.C. Risk Perception. In Encyclopedia of Behavioral Medicine; Gellman, M., Ed.; Springer: New York, NY, USA, 2018. [Google Scholar]
- Ropeik, D. Understanding factors of risk perception. Nieman Rep. 2002, 56, 52. [Google Scholar]
- Weber, E.U.; Blais, A.; Betz, N.E. A domain-specific risk-attitude scale: Measuring risk perceptions and risk behaviors. J. Behav. Decis. Mak. 2002, 15, 263–290. [Google Scholar] [CrossRef]
- Paek, H.J.; Hove, T. Risk perceptions and risk characteristics. In Oxford Research Encyclopedia of Communication; Oxford University Press: New York, NY, USA, 2017. [Google Scholar]
- Eliasson, K.; Palm, P.; Nyman, T.; Forsman, M. Inter-and intra-observer reliability of risk assessment of repetitive work without an explicit method. Appl. Ergon. 2017, 62, 1–8. [Google Scholar] [CrossRef]
- Thomson, M.E.; Önkal, D.; Avcioğlu, A.; Goodwin, P. Aviation risk perception: A comparison between experts and novices. Risk Anal. Int. J. 2004, 24, 1585–1595. [Google Scholar] [CrossRef]
- Hunter, D.R. Risk perception among general aviation pilots. Int. J. Aviat. Psychol. 2006, 16, 135–144. [Google Scholar] [CrossRef]
- McMurtrie, K.J.; Molesworth, B.R. The variability in risk assessment between flight crew. Int. J. Aerosp. Psychol. 2017, 27, 65–78. [Google Scholar] [CrossRef]
- Crocker, J.; Shields, K.F.; Venkataramanan, V.; Saywell, D.; Bartram, J. Building capacity for water, sanitation, and hygiene programming: Training evaluation theory applied to CLTS management training in Kenya. Soc. Sci. Med. 2016, 166, 66–76. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- WHO. Rapid Assessment of Drinking-Water Quality: A Handbook for Implementation; World Health Organization: Geneva, Switzerland, 2012; Available online: https://apps.who.int/iris/bitstream/handle/10665/331485/9789241504683-eng.pdf (accessed on 8 June 2020).
- Okotto-Okotto, J.; Wanza, P.; Kwoba, E.; Yu, W.; Dzodzomenyo, M.; Thumbi, S.M.; da Silva, D.G.; Wright, J.A. An Assessment of Inter-Observer Agreement in Water Source Classification and Sanitary Risk Observations. Exp. Health 2019, 11, 1–14. [Google Scholar] [CrossRef] [Green Version]
- Yentumi, W.; Dzodzomenyo, M.; Sashie-Doe, K.; Wright, J. An assessment of the replicability of a standard and modified sanitary risk protocol for groundwater sources in Greater Accra. Environ. Monit. Assess. 2019, 191, 59. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- European Commission. Proposal for a Directive of the European Parliament and of the Council on the Quality of Water Intended for Human Consumption. 2020. Available online: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM%3A2017%3A753%3AFIN (accessed on 5 May 2020).
- Uganda Water Supply Atlas. Available online: http://wsdb.mwe.go.ug/index.php/reports/district/70 (accessed on 2 May 2020).
- Krejcie, R.V.; Morgan, D.W. Determining sample size for research activities. Educ. Psychol. Meas. 1970, 30, 607–610. [Google Scholar] [CrossRef]
- WHO. Guidelines for drinking-water quality. In Surveillance and Control of Community Supplies, 2nd ed.; World Health Organization: Geneva, Switzerland, 1997; Available online: http://apps.who.int/iris/handle/10665/42002 (accessed on 5 June 2020).
- McCambridge, J.; Witton, J.; Elbourne, D.R. Systematic review of the Hawthorne effect: New concepts are needed to study research participation effects. J. Clin. Epidemiol. 2014, 67, 267–277. [Google Scholar] [CrossRef] [Green Version]
- Santo, S.; Ayres-de-Campos, D. Human factors affecting the interpretation of fetal heart rate tracings: An update. Curr. Opin. Obstet. Gynecol. 2012, 24, 84–88. [Google Scholar] [CrossRef]
- Timmers, J.M.H.; van Doorne-Nagtegaal, H.J.; Verbeek, A.L.M.; Den Heeten, G.J.; Broeders, M.J.M. A dedicated BI-RADS training programme: Effect on the inter-observer variation among screening radiologists. Eur. J. Radiol. 2012, 81, 2184–2188. [Google Scholar] [CrossRef]
- Brealey, S.; Piper, K.; King, D.; Bland, M.; Caddick, J.; Campbell, P.; Gibbon, A.; Highland, A.; Jenkins, N.; Petty, D.; et al. Observer agreement in the reporting of knee and lumbar spine magnetic resonance (MR) imaging examinations: Selectively trained MR radiographers and consultant radiologists compared with an index radiologist. Eur. J. Radiol. 2013, 82, 597–605. [Google Scholar] [CrossRef]
- Jin, E.H.; Chung, S.J.; Lim, J.H.; Chung, G.E.; Lee, C.; Yang, J.I.; Kim, J.S. Training effect on the inter-observer agreement in endoscopic diagnosis and grading of atrophic gastritis according to level of endoscopic experience. J. Korean Med Sci. 2018, 33, 15. [Google Scholar] [CrossRef] [Green Version]
- Ministry of Foreign Affairs of the Netherlands. Climate Change Profile: Uganda; Ministry of Foreign Affairs of the Netherlands: The Hague, The Netherlands, 2018. Available online: https://www.government.nl/binaries/government/documents/publications/2019/02/05/climate-change-profiles/Uganda.pdf (accessed on 15 June 2020).
- Pande, G.; Kwesiga, B.; Bwire, G.; Kalyebi, P.; Riolexus, A.; Matovu, J.K.; Makumbi, F.; Mugerwa, S.; Musinguzi, J.; Wanyenze, R.K.; et al. Cholera outbreak caused by drinking contaminated water from a lakeshore water-collection site, Kasese District, south-western Uganda, June–July 2015. PLoS ONE 2018, 13, e0198431. [Google Scholar] [CrossRef]
- Cronin, A.A.; Breslin, N.; Gibson, J.; Pedley, S. Monitoring source and domestic water quality in parallel with sanitary risk identification in Northern Mozambique to prioritise protection interventions. J. Water Health 2006, 4, 333–345. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Sanitary Inspection Questions | NO | YES (Risk) | Risk Level (circle Risk Only if YES Is Ticked) | What Action is Needed? | |||||
---|---|---|---|---|---|---|---|---|---|
1 | Is the masonry, concrete wall or spring box absent or inadequate to prevent contamination? It is important that spring water is not exposed to contamination during the period of time between leaving the ground and being collected by the user. Masonry, a concrete wall or a spring box will protect the water from contamination during this time period. a | Low | Medium | High | |||||
2 | If there is a spring box, is the inspection cover or overflow pipe absent or inadequate to prevent contamination? A missing or inadequate (e.g., damaged, corroded, cracked, leaking) inspection cover or overflow pipe may increase the likelihood of contamination entering the spring box. If present, they should not allow the entry of vermin and other pollution into the spring box. | Low | Medium | High | |||||
3 | If there is a spring box, and there is an air vent, is it inadequately covered to prevent contamination? An air vent that is open to the environment may increase the likelihood of contamination entering the spring box. If present, the air vent should not allow the entry of vermin and other pollution into the spring box. | Low | Medium | High | |||||
4 | If there is a spring box, does it contain any visible sign of contamination (e.g., animal waste, sediment accumulation)? Contamination in the spring box may constitute a risk to water quality. Small deposits of silt at the bottom of the spring box are less likely to threaten water quality compared to animal waste, floating solids or biological growth. | Low | Medium | High | |||||
5 | Is the backfill area eroded or prone to erosion due to absence of vegetation? If the backfill area (directly behind the spring box or concrete wall) becomes eroded (e.g., due to absence of vegetation), it may act as a direct pathway for contamination to enter the spring water before it is collected by the user. | Low | Medium | High | |||||
6 | Is the fencing or barrier around the spring absent or inadequate to prevent contamination? If there is no fence or barrier around the spring (or if the fence is damaged or not fit for purpose), animals can access the spring site and may damage the structure as well as pollute the area with excreta. | Low | Medium | High | |||||
7 | Is the fencing or barrier upstream of the spring inadequate to stop local pollution? b If there is no fence/barrier upstream of the spring (or if the fence/barrier is damaged or not fit for purpose) then the shallower groundwater may become contaminated as it approaches the spring structure. | Low | Medium | High | |||||
8 | Is a storm water diversion ditch above the spring absent or inadequate to prevent contamination? If the diversion ditch is absent or inadequate (e.g., blocked, not wide or deep enough), contaminated surface water may enter the spring facility from above during periods of rain, or other events that may cause excess water to flow down towards the spring site. | Low | Medium | High | |||||
9 | Is there a latrine, septic tank or sewer line within 10 meters of the spring? Latrines close to groundwater supplies may affect water quality (e.g., by infiltration). You may need to visually check structures to see if they are latrines, in addition to asking residents about the presence of septic tanks and sewer lines. | Low | Medium | High | |||||
10 | Is there a latrine, septic tank or sewer line on higher ground within 30 meters of the spring? Pollution on higher ground poses a risk, especially in the wet season, as faecal material may flow into the spring. Groundwater may also flow towards the spring from the direction of the latrine, septic tank or sewer line. | Low | Medium | High | |||||
11 | Can signs of other sources of pollution be seen within 10 meters of the spring (e.g., animals, rubbish, human settlement, open defecation)? Animals or human faeces on the ground close to the spring constitute a serious risk to water quality. Presence of other waste (household, laundry, agricultural etc.) also constitutes a risk to water quality. | Low | Medium | High | |||||
12 | Is there an open/uncapped well or borehole within 100 meters of the spring? Any point of entry to the aquifer that is unprotected is a direct pathway for contaminants to enter the spring. | Low | Medium | High | |||||
| Risk level | Number of risks | Multiply by: | Score | |||||
Low | 1 | ||||||||
Medium | 3 | ||||||||
High | 5 | ||||||||
Sanitary risk score (max. 60) | Total: |
Supply 1 | Q1 | Q2 | Q3 | Q4 | Q5 | Q6 | Q7 | Q8 | Q9 | Q10 | Q11 | Q12 |
---|---|---|---|---|---|---|---|---|---|---|---|---|
High | 1 | 3 | 2 | 1 | 1 | 3 | 2 | 3 | 0 | 1 | 0 | 0 |
Medium | 3 | 1 | 1 | 2 | 1 | 0 | 1 | 0 | 0 | 0 | 3 | 0 |
Low | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
No | 0 | 0 | 1 | 1 | 2 | 1 | 1 | 1 | 4 | 2 | 0 | 4 |
Sum squares | 10 | 10 | 6 | 6 | 6 | 10 | 6 | 10 | 16 | 6 | 10 | 16 |
Index of dispersion | 0.500 | 0.500 | 0.833 | 0.833 | 0.833 | 0.500 | 0.833 | 0.500 | 0.000 | 0.833 | 0.500 | 0.000 |
Yes/No SI Scores | Inspector 1 | Inspector 2 | Inspector 3 | Inspector 4 | |
---|---|---|---|---|---|
Pre-training | Pearson’s r | (34) = 0.313, p = 0.063 | (34) = 0.216, p = 0.206 | (34) = 0.465, p = 0.004 | (34) = 0.374, p = 0.025 |
Post-training | Pearson’s r | (34) = 0.686, p < 0.001 | (34) = 0.399, p = 0.016 | (34) = 0.531, p = 0.001 | (34) = 0.533, p = 0.001 |
Risk level SI scores | |||||
Pre-training | Pearson’s r | (34) = 0.351, p = 0.036 | (34) = 0.308, p = 0.068 | (34) = 0.292, p = 0.084 | (34) = 0.396, p = 0.017 |
Post-training | Pearson’s r | (34) = 0.748, p < 0.001 | (34) = 0.409, p = 0.013 | (34) = 0.606, p < 0.001 | (34) = 0.572, p < 0.001 |
Yes/No SI Scores | Inspector 1 | Inspector 2 | Inspector 3 | Inspector 4 | |
---|---|---|---|---|---|
Pre-training | Lin’s CCC (95% confidence intervals) | 0.231 (−0.02–0.45) | 0.122 (−0.07–0.30) | 0.332 (0.10–0.53) | 0.361 (0.05–0.61) |
Post-training | Lin’s CCC (95% confidence intervals) | 0.680 (0.46–0.82) | 0.353 (0.07–0.58) | 0.408 (0.17–0.60) | 0.517 (0.24–0.72) |
Risk level SI scores | |||||
Pre-training | Lin’s CCC (95% confidence intervals) | 0.318 (0.03–0.56) | 0.277 (−0.02–0.53) | 0.287 (−0.04–0.56) | 0.340 (0.06–0.57) |
Post-training | Lin’s CCC (95% confidence intervals) | 0.607 (0.41–0.75) | 0.201 (0.03–0.36) | 0.405 (0.20–0.58) | 0.308 (0.13–0.47) |
Yes/No SI Scores | Mean SI Score Coefficient of Variation across 36 Supplies | Standard Deviation |
---|---|---|
Pre-training | 21.25 | 8.13 |
Post-training | 16.16 | 6.87 |
Risk level SI scores | ||
Pre-training | 24.12 | 10.29 |
Post-training | 19.62 | 10.74 |
Yes/No SI Scores | Mean Answer Index of Dispersion across 12 Questions | Standard Deviation |
---|---|---|
Pre-training | 0.41 | 0.28 |
Post-training | 0.27 | 0.21 |
Risk level SI scores | ||
Pre-training | 0.55 | 0.12 |
Post-training | 0.41 | 0.21 |
Yes/No SI Scores | Mean Answer Index of Dispersion across 36 Supplies | Standard Deviation |
---|---|---|
Pre-training | 0.41 | 0.10 |
Post-training | 0.27 | 0.14 |
Risk level SI scores | ||
Pre-training | 0.55 | 0.08 |
Post-training | 0.41 | 0.12 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
King, R.; Okurut, K.; Herschan, J.; Lapworth, D.J.; Malcolm, R.; McKeown, R.M.; Pond, K. Does Training Improve Sanitary Inspection Answer Agreement between Inspectors? Quantitative Evidence from the Mukono District, Uganda. Resources 2020, 9, 120. https://doi.org/10.3390/resources9100120
King R, Okurut K, Herschan J, Lapworth DJ, Malcolm R, McKeown RM, Pond K. Does Training Improve Sanitary Inspection Answer Agreement between Inspectors? Quantitative Evidence from the Mukono District, Uganda. Resources. 2020; 9(10):120. https://doi.org/10.3390/resources9100120
Chicago/Turabian StyleKing, Richard, Kenan Okurut, Jo Herschan, Dan J. Lapworth, Rosalind Malcolm, Rory Moses McKeown, and Katherine Pond. 2020. "Does Training Improve Sanitary Inspection Answer Agreement between Inspectors? Quantitative Evidence from the Mukono District, Uganda" Resources 9, no. 10: 120. https://doi.org/10.3390/resources9100120