- How effective are “123” persuasive space graphics (PSG) at motivating hand hygiene?
- How efficient are “123” persuasive space graphics (PSG) in communicating hand hygiene?
2. Evaluation Frameworks
- “Reach” is a measure of the representativeness of individuals who have agreed to participate in an intervention.
- “Effectiveness” (referred to as “Efficiency” in some versions of the framework) is a measure of targeted outcomes including quality of life, economic costs and unintended negative consequences.
- “Adoption” is a measure of the proportion and representativeness of settings that agree to the intervention.
- “Implementation” is a measure of the degree to which settings deliver interventions as intended.
3. Overview of Evaluation Design
3.1. Evaluation Methods
4. Evaluation 1: Participant Demographic
4.1.1. School and Museum Recruitment
4.1.2. Data Collection
4.4. Evaluation Summary and Conclusion
- the evaluation in this study is potentially transferable for children aged 4–11 from the north of England irrespective of socio-economic status and ethnic background;
- further piloting is required to determine transferability of the PSG to other regions and poor performing schools.
5. Evaluation 2: Handwashing Frequency
5.1.1. Participants and Data Collection
5.1.2. Data Analysis
5.4. Evaluation Summary and Conclusion
- PSG have been very effective in increasing handwashing frequency in the school settings. Soap consumption increased between 41% and 60% when the PSG were present.
- handwashing frequency was not maintained when the PSG were absent. Whether longer term exposure to the PSG would help maintain behaviour change when the designs were present and/or absent needs to be tested in future work.
- the PSG were moderately effective in increasing handwashing frequency in the museum.
6. Evaluation 3: Handwashing Quality
6.1.1. Participants and Recruitment
6.1.2. Procedure and Data Collection
6.1.3. Plate Preparation and Incubation
6.1.4. Data Analysis
6.4. Evaluation Summary and Conclusion
- the “123” PSG were effective in increasing handwashing quality in the school settings;
- the “123” PSG were not effective in increasing handwashing quality in the museum setting.
7. Evaluation 4: Design Persuasiveness
7.1.1. Participant Recruitment and Data Collection Procedure
7.1.2. Interview Design and Analysis
7.2.1. Step 1: Attention (Reach and Effectiveness)
7.2.2. Step 2: Comprehension (Effectiveness)
7.2.3. Step 3: Acceptance (Effectiveness)
7.2.4. Step 4: Attitude Change (Effectiveness)
7.4. Evaluation Summary and Conclusion
- situating communication designs in toilets (i.e., PSG) is a powerful way to reach children, and is effective in gaining children’s attention (step 1);
- through the PSG, hand hygiene messages were comprehended (step 2) and thus successfully communicated. The core “123” message was more effective in schools where children had a longer exposure to the message. In single-visit settings, such as museums, a simpler approach may be required;
- the “123” PSG were effective in gaining children’s trust (step 3) mostly because children found the information useful and because the information triangulated with what they already knew;
- with over half of the children reporting that the designs led to a change in behaviour and nearly half either now washing their hands or using the soap and hand-dryer when they had not before, the “123” PSG can be considered effective in behaviour change (step 4), particularly as some children would already be washing their hands.
8. Evaluation 5: Stakeholder Views
8.1.1. Participants and Recruitment
8.1.2. Interview Design and Data Collection
8.1.3. Data Analysis
“It’s certainly created a buzz and the kids were talking about it.”(Head teacher, School 1)
“Children are more alert, like they’re asking questions about them. Then, you explain to them what it is. I think it has helped them to wash their hands more.”(Teacher, School 3)
“We’ve noticed a difference in the cleanliness of the toilets at the end of the school day. Girls, no boys in particular which surprised us. There seems to be, toilet seats seem to be down more at the end of the day and there seems to be less toilet paper on the floor, no idea why that is.”(Head teacher, School 1)
“Initially the children were going to the toilet more just to read the stickers and were probably spending longer in there reading the stickers than actually doing the toilet but actually since they have been there for a while its, they’re just going in there and doing what they need to do.”(Teacher, School 1)
“I think they would work for a while and then eventually they would just become wallpaper.”(Teacher, School 3)
“It’s the fact that they are a bit bright and colourful and it makes the environment look nice because it does, because toilets are very bland environments aren’t they? So just making it a more colourful and attractive place I think has probably had a bit of an impact that we weren’t really thinking about.”(Head teacher, School 1)
“They look bright and colourful and inviting. We’ve even had Jeremy Corbyn [leader of a UK political party] use one of them, so there you go.”(Head teacher, School 3)
“Because they are so visual and they’re there all the time that they do have a really good impact and I think the fact that they are professionally done as well, I think it ups, it ups the status of it.”(Head teacher, School 1)
“I think possibly a disadvantage is it’s possibly a little too in your face maybe but that might be what the kids need whereas I’m going in as an adult.”(Museum)
“They are in the right place. That it’s all very well to teach children about hand hygiene in an assembly or in the classroom but actually this is reminding them at the very point at which it is something that they need to do.”(Head teacher, School 3)
“There should be a clear link to learning; personal social health education.”(Head teacher, School 2)
“Some were coming off. You just stick your hand up and put it back up. And on the floor when you are mopping, they were coming up, but you just stick back down afterwards.”(Cleaner, School 2)
“It’s got to be something that doesn’t come off and that can’t be picked off because you’re always going to get inquisitive little fingers that, ooh look, there’s a sticker lets peel it off cos that’s what kids, that’s what kids do.”(Head teacher, School 1)
“If we had different sets so that you know after, you know if we had three different sets one for each term then you know the message is, because otherwise it does become a bit wallpaper. So if you had three different focuses you know for autumn, spring and summer, that would, you know, just keep it fresh.”(Teacher, School 2)
“Possibly looking at the design just to fit in a little bit better with the museum itself, not strictly to but moving towards the way we, you know we have our branding guidelines, it just means it fits with the rest of the things we have in the museum.”(Museum)
8.2.6. Economic Cost
“We know for a fact that at the moment our budget is in absolutely dire straits there is no way that we would be able to squeeze anything out it would go by the by, couldn’t be done.”(Teacher, School 2)
“Putting them in the right place as well. Little things like the toilets, the ones on the toilets, it needs to be in the right place on the toilet seat and if you get it you get it in the wrong place it’s going to, yeah so I would say putting them in the right place would be a difficult thing and getting them flat so that that don’t bubble with big stickers.”(Teacher, School 3)
“I wouldn’t need any proof. I think they’re a great idea. And if they hadn’t been funded and they were just something we could buy I would certainly consider buying them. I wouldn’t need any proof that they worked, I think they are self evidently a really great idea.”(Head teacher, School 3)
“I’d be interested to see the results from the other schools with the use of soap, see if that’s increased. Like I have said, we have noticed a difference here but I think we would get more proof of it being effective.”(Head teacher, School 1)
8.4. Evaluation Summary and Conclusion
- PSG are effective in encouraging hand hygiene with most stakeholders strongly agreeing and none disagreeing that the PSG are required in their setting;
- PSG are effective in engaging children with the topic of handwashing and also encouraging more generally hygienic toilet practices;
- study settings adopted the designs because they are visually appealing, engaging to children and remind children what to do at the point of use;
- to increase adoption PSG should be integrated into the school curriculum and/or incorporated into classroom activities. Some flexibility in PSG selection including the ability to exchange and adapt designs would further aid adoption;
- the durability of the design material (the adhesiveness of the glue and the ease with which children could remove designs) would need to be improved for schools to adopt and maintain designs;
- evidence of the PSG working in more school settings and evidence of their long-term effectiveness would be required for schools to maintain the PSG;
- financial constraints could inhibit settings from implementation.
9.1. RQ1: How Effective Are “123” Persuasive Space Graphics (PSG) at Motivating Hand Hygiene?
9.2. RQ2: How Efficient Are “123” Persuasive Space Graphics (PSG) in Communicating Hand Hygiene?
11. Future Work
- improve upon the weaknesses of “123” PSG identified in the evaluation. In particular, the PSG will be modified for comprehension and new solutions for durability issues will be sought;
- work with new settings (both within the UK and beyond) to consider how “123” can be best adapted for different settings;
- conduct a larger-scale implementation and evaluation, including a longitudinal evaluation.
Availability of data and materials
Conflicts of Interest
- HM Government. Tackling Antimicrobial Resistance 2019–2024; The UK’s Five-Year National Action Plan; HM Government: London, UK, 2019.
- HM Government. Contained and Controlled; The UK’s 20-Year Visition for Antimicrobial Resistance; HM Government: London, UK, 2019.
- Curtis, V.; Schmidt, W.; Luby, S.; Florez, R.; Toure, O.; Biran, A. Hygiene: New hopes, new horizons. Lancet Infect. Dis. 2011, 11, 312–321. [Google Scholar] [CrossRef]
- Freeman, M.C.; Stocks, M.E.; Cumming, O.; Jeandron, A.; Higgins, J.P.; Wolf, J.; Prüss-Ustün, A.; Bonjour, S.; Hunter, P.; Fewtrell, L.; et al. Systematic review: Hygiene and health: Systematic review of handwashing practices worldwide and update of health effects. Trop. Med. Int. Health 2014, 19, 906–916. [Google Scholar] [CrossRef] [PubMed]
- Johansen, A.; Denbæk, A.M.; Bonnesen, C.T.; Due, P. The Hi Five study: Design of a school-based randomized trial to reduce infections and improve hygiene and well-being among 6–15 year olds in Denmark. BMC Public Health 2015, 15, 207. [Google Scholar] [CrossRef]
- Curtis, V.; Danquah, L.O.; Aunger, R.V. Planned, motivated and habitual hygiene behaviour: An eleven country review. Health Educ. Res. 2009, 24, 655–673. [Google Scholar] [CrossRef]
- 1 2 3 to Get Germ-Free: Transforming Primary School Toilets to Improve Hand Hygiene. Available online: www.123germfree.co.uk (accessed on 30 March 2020).
- Chittleborough, C.R.; Nicholson, A.L.; Basker, E.; Bell, S.; Campbell, R. Factors influencing hand washing behaviour in primary schools: Process evaluation within a randomized controlled trial. Health Educ. Res. 2012, 27, 1055–1068. [Google Scholar] [CrossRef] [PubMed]
- Chittleborough, C.R.; Nicholson, A.L.; Young, E.; Bell, S.L.; Campbell, R. Implementation of an educational intervention to improve hand washing in primary schools: Process evaluation within a randomised controlled trial. BMC Public Health 2013, 13, 757. [Google Scholar] [CrossRef] [PubMed]
- e-Bug. Available online: www.e-bug.eu (accessed on 16 December 2019).
- McNulty, C.; Lecky, D.; Farrell, D.; Kostkova, P.; Adriaenssens, N.; Herotova, T.K.; Holt, J.; Touboul, P.; Merakou, K.; Koncan, R.; et al. Overview of e-Bug: An antibiotic and hygiene educational resource for schools. J. Antimicrob. Chemother. 2011, 66, v3–v12. [Google Scholar] [CrossRef]
- Eley, C.; Young, V.; Hoekstra, B.; McNulty, C.A.M. An evaluation of educators’ views on the e-Bug resources in England. J. Boil. Educ. 2017, 52, 166–173. [Google Scholar] [CrossRef]
- Lecky, D.; McNulty, C.A.M.; Touboul, P.; Herotova, T.K.; Beneš, J.; Dellamonica, P.; Verlander, N.Q.; Weinberg, J.; Goossens, H.; Adriaenssens, N.; et al. Evaluation of e-Bug, an educational pack, teaching about prudent antibiotic use and hygiene, in the Czech Republic, France and England. J. Antimicrob. Chemother. 2010, 65, 2674–2684. [Google Scholar] [CrossRef]
- Randle, J.; Metcalfe, J.; Webb, H.; Luckett, J.; Nerlich, B.; Vaughan, N.; Segal, J.; Hardie, K.R. Impact of an educational intervention upon the hand hygiene compliance of children. J. Hosp. Infect. 2013, 85, 220–225. [Google Scholar] [CrossRef]
- Sussman, R.; Gifford, R. Please turn off the lights: The effectiveness of visual prompts. Appl. Ergon. 2012, 43, 596–603. [Google Scholar] [CrossRef] [PubMed]
- Green, J.; Tones, K.; Cross, R.; Woodall, J. Health Promotion: Planning & Strategies, 3rd ed.; SAGE: Los Angeles, CA, USA, 2015. [Google Scholar]
- Judah, G.; Aunger, R.; Schmidt, W.-P.; Michie, S.; Granger, S.; Curtis, V. Experimental pretesting of hand-washing interventions in a natural setting. Am. J. Public Health 2009, 99, S405–S411. [Google Scholar] [CrossRef] [PubMed]
- Tabak, R.G.; Khoong, E.C.; Chambers, D.; Ross, C. Bridging research and practice. Am. J. Prev. Med. 2013, 43, 337–350. [Google Scholar] [CrossRef] [PubMed]
- RE-AIM. Available online: http://www.re-aim.org/ (accessed on 16 December 2019).
- PRECIS-2. Available online: www.precis-2.org (accessed on 16 December 2019).
- Aqil, A.; Lippeveld, T.; Hozumi, D. PRISM framework: A paradigm shift for designing, strengthening and evaluating routine health information systems. Health Policy Plan. 2009, 24, 217–228. [Google Scholar] [CrossRef] [PubMed]
- CFIR. Available online: www.cfirguide.org (accessed on 16 December 2019).
- Caetano, R. Standards for reporting non-randomized evaluations of behavioral and public health interventions: The TREND statement. Addiction 2004, 99, 1075–1080. [Google Scholar] [PubMed]
- Craig, P.; Dieppe, P.; MacIntyre, S.; Michie, S.; Nazareth, I.; Petticrew, M. Developing and evaluating complex interventions: The new Medical Research Council guidance. BMJ 2008, 337, a1655. [Google Scholar] [CrossRef]
- Glasgow, R.E.; Vogt, T.M.; Boles, S.M. Evaluating the public health impact of health promotion interventions: The RE-AIM framework. Am. J. Public Health 1999, 89, 1322–1327. [Google Scholar] [CrossRef]
- RE-AIM. About RE-AIM RE-AIM.org. 2019. Available online: http://www.re-aim.org/ (accessed on 22 February 2019).
- Al-Achi, A.; Greenwood, R.; Junker, J. Evaluating students’ course evaluations. J. Pharm. Teach. 1994, 4, 53–62. [Google Scholar] [CrossRef]
- Find an Inspection Report. Available online: Reports.ofsted.gov.uk (accessed on 30 March 2020).
- Taylor, C. The reliability of free school meal eligibility as a measure of socio-economic disadvantage: Evidence from the millennium cohort study in Wales. Br. J. Educ. Stud. 2017, 66, 29–51. [Google Scholar] [CrossRef]
- Rutter, S.; Macduff, C.; Stones, C.; Gomez-Escalada, M. Evaluating children’s handwashing in schools: An integrative review of indicative measures and measurement tools. Int. J. Environ. Health Res. 2019, 29, 1–19. [Google Scholar] [CrossRef]
- Image J. Image Processing in Java. Available online: https://imagej.nih.gov/ij/ (accessed on 30 March 2020).
- Lumley, T.; Diehr, P.; Emerson, S.; Chen, L. The importance of the normality assumption in large public health data sets. Annu. Rev. Public Health 2002, 23, 151–169. [Google Scholar] [CrossRef] [PubMed]
- Minitab. Choosing Between a Nonparametric Test and a Parametric Test The Minitab Blog. 2015. Available online: https://blog.minitab.com/blog/adventures-in-statistics-2/choosing-between-a-nonparametric-test-and-a-parametric-test (accessed on 28 September 2019).
- Statistics Solutions. Should you Use a One-tailed Test or a Two-tailed Test for Your Data Analysis? Statistics Solutions. 2019. Available online: https://www.statisticssolutions.com/should-you-use-a-one-tailed-test-or-a-two-tailed-test-for-your-data-analysis/ (accessed on 28 September 2019).
- McGuire William, J. Attitudes and attitude change. In The Handbook of Social Psychology, 3rd ed.; Lindzey, G., Aronson, E., Eds.; Random House: New York, NY, USA, 1985; Volume II, pp. 233–346. [Google Scholar]
- Houts, P.S.; Doak, C.C.; Doak, L.G.; Loscalzo, M.J. The role of pictures in improving health communication: A review of research on attention, comprehension, recall, and adherence. Patient Educ. Couns. 2006, 61, 173–190. [Google Scholar] [CrossRef] [PubMed]
- Johnson, F.; Sbaffi, L.; Rowley, J. Students’ approaches to the evaluation of digital information: Insights from their trust judgments. Br. J. Educ. Technol. 2016, 47, 1243–1258. [Google Scholar]
- Rutter, S.; Stones, C.; Macduff, C. Communicating handwashing to children, as told by children. Health Commun. 2019, 34, 1–10. [Google Scholar] [CrossRef] [PubMed]
- Brownson, R.C.; Jacobs, J.A.; Tabak, R.G.; Hoehner, C.M.; Stamatakis, K.A. Designing for dissemination among public health researchers: Findings from a national survey in the United States. Am. J. Public Health 2013, 103, 1693–1699. [Google Scholar] [CrossRef] [PubMed]
- Huebschmann, A.G.; Leavitt, I.; Glasgow, R.E. Making health research matter: A call to increase attention to external validity. Annu. Rev. Public Health 2019, 40, 45–63. [Google Scholar] [CrossRef] [PubMed]
- Zhang, Y.; Wildemuth, B.M. Qualitative analysis of content. Appl. Soc. Res. Methods Quest. Inf. Libr. Sci. 2009, 308, 319. [Google Scholar]
- Lendrum, A.; Humphrey, N. The importance of studying the implementation of interventions in school settings. Oxf. Rev. Educ. 2012, 38, 635–652. [Google Scholar] [CrossRef]
- Bonnesen, C.T.; Plauborg, R.; Denbaek, A.M.; Due, P.; Johansen, A.; Denbæk, A.M. Process evaluation of a multi-component intervention to reduce infectious diseases and improve hygiene and well-being among school children: The Hi Five study. Health Educ. Res. 2015, 30, 497–512. [Google Scholar] [CrossRef]
- National Education Union. Workload National Education Union. 2018. Available online: https://neu.org.uk/press-releases/workload (accessed on 16 December 2019).
- National Association of Head Teachers (NAHT). New Poll Reveals Full Impact of School Funding Crisis NAHT. 2018. Available online: https://www.naht.org.uk/news-and-opinion/press-room/new-poll-reveals-full-impact-of-school-funding-crisis/ (accessed on 16 December 2019).
- Alibali, M.W.; Nathan, M.J. Conducting research in schools: A practical guide. J. Cogn. Dev. 2010, 11, 397–407. [Google Scholar] [CrossRef]
|School 1||School 2||School 3||Museum||Total|
|Evaluation 1: Participant demographic||n/a—use of publicly available documents|
|Evaluation 2: Handwashing frequency||38 days pre and 39 days post, 30.11.2018 to 24.05.2019||n/a—data collection error||20 days pre and 38 days post, 06.03.2019 to 14.06.2019||129 days pre and 39 days post, 21.10.2018 to 19.04.2019||187 days pre and 116 days post|
|Evaluation 3: Handwashing quality||139 children pre and 138 children post, 14.02.2019 and 12.03.2019||-||-||75 children pre and 69 children post, 04.01.2019 and 09.03.2019||214 children pre and 207 children post|
|Evaluation 4: Design persuasiveness||43 children, 22.03.2019, 26.03.2019 and 27.03.2019||20 children, 21.05.2019||24 children, 09.05.2019||47 children, 15.04.2019 and 16.04,2019||134 children|
|Evaluation 5: Stakeholder views||8 staff, 01.05.2019||6 staff, 06.06.2019||8 staff, 09.05.2019||1 staff, 12.06.2019||23 staff|
|Setting||Location||Age of Children||Size of Setting/No. of Visitors||Ofsted Rating||Catchment|
|Partner museum||City setting north of England||Target audience is 0–11||302,460 visitors (in 2014)||/||At weekends and during school holidays parents bring children. During weekdays school visits predominate.|
|School 1||Semi-rural setting East Midlands||4–11||Larger than average1 (approx. 420 places)||Good||Majority of children are Caucasian and British. A proportionately below average number1 of children are eligible for free school meals. Below average proportion of special educational needs.|
|School 2||City setting north of England||4–11||Very large (approx. 700 places)||Good||Mostly minority ethnic backgrounds. A proportionately above average number1 of children are eligible for free school meals. Above average proportion of special educational needs.|
|School 3||City setting north of England||4–11||Smaller than average1 (approx. 210 places)||Good||High proportion from minority ethnic groups. A proportionately above average number1 of children are eligible for free school meals. Slightly below average proportion of special educational needs.|
|Schools||Time Period||Soap Consumption||Dryer Consumption|
|Total||Per Day||% Increase on Baseline||Total||Per Day||% Increase on Baseline|
(20 school days, 06.03.2019 to 29.03.2019
|1707 mL||85 mL||-||-||-||-|
|Designs installed, 2 Apr|
|Post-installation 1–4 weeks|
(18 school days, 01.04.2019 to 10.05.2019)
|2162 mL||120 mL||41%||-||-||-|
|Post-installation 5–8 weeks|
(20 school days, 13.05.2019 to 14.06.2019)
|2600 mL||130 mL||53%||-||-||-|
(38 school days, 30.11.2019 to 06.02.2019)
|2750 mL||72.4 mL||-||15 rolls||0.39 rolls||-|
|Designs installed, 13 Mar|
|Post-installation I, 1–4 weeks|
(19 school days, 14.03.2019 to 26.04.2019)
|2200 mL||115.7 mL||60%||7 rolls||0.37 rolls||−5%|
|Post-installation II, 5–8 weeks|
(20 school days, 26.04.2019 to 24.05.2019)
|1500 mL||75 mL||4%||9 rolls||0.45||15%|
|Designs re-installed, 11 Jun|
|Post-installation III, 9–13 weeks|
(26 school days, 03.06.2019 to 09.07.2019)
|2800 mL||107.7 mL||49%||12 rolls||0.46||18%|
|Time Period||Toilets without PSG||Toilets with PSG|
(129 days, 21.10.2019 to 28.02.2109)
|No. soap changes||9 changes||4 changes||3 changes||2 changes|
|Mean no. of days between changes||15 days||38 days||53 days||50 days|
(39 days, 11.03.2019 to 19.04.2019)
|No. soap changes||5 changes||2 changes||3 changes||2 changes|
|Mean no. of days between changes||7 days||35 days||19 days||39 days|
|% increase in soap usage on baseline||53%||8%||64%||22%|
|Design Persuasiveness||Mapped to (McGuire, 1985)||Mapped to RE-AIM||Interview Question||Analysis|
|Step 1: Attention||Attention (step 2)||Reach/effectiveness||Q1: When you were in the toilet, did you look at the posters/stickers?||Responses counted.|
|Step 2: Comprehension||Comprehension of arguments (step 4)||Effectiveness||Q2.1: What do you remember seeing or reading? What did you find out?||Responses verified for comprehension and counted.|
|Q2.2: What does 123 mean?||Responses verified for comprehension and counted. Chi-squared statistical tests for age, gender and setting.|
|Step 3: Acceptance||Acceptance of the arguments (step 7)||Effectiveness||Q3.1: Do you trust the posters/sticker?||Responses counted.|
|Q3.2: Why do you/don’t you trust the posters/sticker?||Responses mapped onto a validated scheme of children’s trust criteria .|
|Step 4: Attitude change||Attitude change (step 11)||Effectiveness||Q4.1: Have the posters/stickers changed what you do when you are in the toilets?||Responses counted.|
|Q4.2: If so, in what way?||Responses categorised inductively.|
|Q4.3: Did you wash your hands just now?||Responses counted and compared with Q4.1.|
|Trust Reasons||Definition from ||No. of Reasons Given|
|Usefulness||“The extent to which the user is informed by and can make use of the information”||63||0|
|Triangulation||“The extent to which the information is consistent with other information on the same topic”||27||0|
|Authority||“The expertise and standing of the author or organisation responsible for providing the information”||15||0|
|Credibility||“The believability and impartiality of the information”||6||1|
|Style||“The way in which the information is presented and written”||2||1|
|Content||“The core characteristics of the information, such as reliability, accuracy and currency”||0||0|
|Brand||“Brand indicators and reputation”||0||0|
|Ease of Use||“The ease of locating, accessing and using the information”||0||0|
|Recommendation||“Recommendations regarding the information from known person(s)”||0||0|
|Do not know||The reason why is not known or cannot be explained.||16||3|
|Effectiveness||The posters/stickers have encouraged children to wash their hands. Strongly disagree/disagree/neither agree nor disagree/agree/strongly agree/do not know||Head teacher, teachers and museum staff only|
|Maintenance||The school needs posters/stickers like these in the toilets. Strongly disagree/disagree/neither agree nor disagree/agree/strongly agree/do not know||All staff|
|Effectiveness||Have the PSG had an impact on the way toilets are used (including unintended consequences)?||All staff|
|Adoption||What do staff think will be barriers/incentives to adoption?||Head teacher, teachers and museum staff only|
|Implementation||Have staff/children adapted or wanted to adapt the installation? What do staff think the incentives/barriers to implementation would be in other settings?||All staff|
|Maintenance||What do staff think are the barriers/incentives to maintaining the PSG long term?||All staff|
|Engagement||How children have/have not engaged with the PSG and the impact this has had|
|Appeal||How the PSG did/did not appeal to children and other audiences|
|Positioning||The location of the PSG and the connection with other activities in the settings|
|Durability||Issues affecting the durability of the PSG|
|Flexibility||The need for PSG to be exchanged and adapted|
|Economic costs||The financial cost and implications if settings had to self-fund|
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).