Enhancing Urban Accessibility: Reliability and Validity Assessment of the Stakeholders’ Walkability/Wheelability Audit in Neighbourhoods Tool
Abstract
:1. Introduction
2. Materials and Methods
2.1. Research Design
2.1.1. Recruitment
- Do you have a mobility/hearing/cognitive disability?
- ○
- If so, are you 19 years of age or older?
- ○
- If not, are you 60 years or older?
- Can you independently walk or wheel for four city blocks over a period of up to three to four hours?
- Do you reside in the city of Vancouver, Burnaby, North Vancouver, Richmond, or Surrey?
- Are you capable of providing informed consent?
2.1.2. Locations for Data Collection
2.2. Data Collection
2.2.1. Demographic Form
2.2.2. Stakeholders’ Walkability/Wheelability Audit in Neighbourhoods Tool
2.2.3. Home and Community Environment Instrument
2.2.4. Sidewalk Index
2.2.5. Walk Score
2.3. Data Analysis
- The study assessed construct validity through hypotheses testing, using Pearson correlations to examine relationships between SWAN tool domains and related constructs in the Home and Community Environment (HACE) Instrument, Sidewalk Index (SI), and Walk Score. The following hypotheses were tested and we expected moderate positive correlations between SWAN domains and corresponding questions in complementary tools:
- Sidewalk functionality in SWAN tool versus question on sidewalk evenness in SI;
- Appearance and maintenance in SWAN tool versus question on surface condition in SI;
- Street crossing functionality in SWAN tool versus question on intersection design in SI;
- Pedestrian safety in SWAN tool versus question on sidewalk evenness in HACE;
- Traffic safety in SWAN tool versus question on safety in HACE;
- Social domain in SWAN tool versus question on places to rest in HACE;
- Land use and supportive features in SWAN tool versus Walk Score.
3. Results
3.1. Inter-Rater Reliability
3.2. Construct Validity, Hypotheses, and Testing
3.3. Internal Consistency
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Tremblay, M.-C.; Martin, D.H.; McComber, A.M.; McGregor, A.; Macaulay, A.C. Understanding Community-Based Participatory Research through a Social Movement Framework: A Case Study of the Kahnawake Schools Diabetes Prevention Project. BMC Public Health 2018, 18, 487. [Google Scholar] [CrossRef] [PubMed]
- Government of Canada, Statistics Canada. New Data on Disability in Canada, 2022. Available online: https://www150.statcan.gc.ca/n1/pub/11-627-m/11-627-m2023063-eng.htm (accessed on 2 March 2024).
- Bigonnesse, C.; Mahmood, A.; Chaudhury, H.; Mortenson, W.B.; Miller, W.C.; Martin Ginis, K.A. The Role of Neighborhood Physical Environment on Mobility and Social Participation among People Using Mobility Assistive Technology. Disabil. Soc. 2018, 33, 866–893. [Google Scholar] [CrossRef]
- Kapsalis, E.; Jaeger, N.; Hale, J. Disabled-by-Design: Effects of Inaccessible Urban Public Spaces on Users of Mobility Assistive Devices—A Systematic Review. Disabil. Rehabil. Assist. Technol. 2022, 19, 604–622. [Google Scholar] [CrossRef]
- Prescott, M.; Labbé, D.; Miller, W.C.; Borisoff, J.; Feick, R.; Mortenson, W.B. Factors That Affect the Ability of People with Disabilities to Walk or Wheel to Destinations in Their Community: A Scoping Review. Transp. Rev. 2020, 40, 646–669. [Google Scholar] [CrossRef]
- Rosso, A.L.; Auchincloss, A.H.; Michael, Y.L. The Urban Built Environment and Mobility in Older Adults: A Comprehensive Review. J. Aging Res. 2011, 2011, 816106. [Google Scholar] [CrossRef]
- Seetharaman, K.; Mahmood, A.; Rikhtehgaran, F.; Akbarnejad, G.; Chishtie, F.; Prescott, M.; Chung, A.; Mortenson, W.B. Influence of the Built Environment on Community Mobility of People Living with Visual Disabilities: A Scoping Review. Urban Plan. Transp. Res. 2024, 12, 2296891. [Google Scholar] [CrossRef]
- Ham, J.; Towle, A.; Shyng, G. Deaf and Hard of Hearing Awareness Training: A Mentor-Led Workshop. Clin. Teach. 2021, 18, 180–185. [Google Scholar] [CrossRef] [PubMed]
- Marquardt, G. Wayfinding for People With Dementia: A Review of the Role of Architectural Design. HERD 2011, 4, 75–90. [Google Scholar] [CrossRef] [PubMed]
- International Classification of Functioning, Disability and Health (ICF). Available online: https://www.who.int/standards/classifications/international-classification-of-functioning-disability-and-health (accessed on 3 August 2024).
- Brownson, R.C.; Hoehner, C.M.; Day, K.; Forsyth, A.; Sallis, J.F. Measuring the Built Environment for Physical Activity. Am. J. Prev. Med. 2009, 36, S99–S123.e12. [Google Scholar] [CrossRef]
- Sullivan, G.M. A Primer on the Validity of Assessment Instruments. J. Grad Med. Educ. 2011, 3, 119–120. [Google Scholar] [CrossRef]
- Sallis, J.F.; Slymen, D.J.; Conway, T.L.; Frank, L.D.; Saelens, B.E.; Cain, K.; Chapman, J.E. Income Disparities in Perceived Neighborhood Built and Social Environment Attributes. Health Place 2011, 17, 1274–1283. [Google Scholar] [CrossRef]
- Cain, K.L.; Millstein, R.A.; Sallis, J.F.; Conway, T.L.; Gavand, K.A.; Frank, L.D.; Saelens, B.E.; Geremia, C.M.; Chapman, J.; Adams, M.A.; et al. Contribution of Streetscape Audits to Explanation of Physical Activity in Four Age Groups Based on the Microscale Audit of Pedestrian Streetscapes (MAPS). Soc. Sci. Med. 2014, 116, 82–92. [Google Scholar] [CrossRef] [PubMed]
- Gibson, B.E. Parallels and Problems of Normalization in Rehabilitation and Universal Design: Enabling Connectivities. Disabil. Rehabil. 2014, 36, 1328–1333. [Google Scholar] [CrossRef]
- Moudon, A.V.; Lee, C. Walking and Bicycling: An Evaluation of Environmental Audit Instruments. Am. J. Health Promot. 2003, 18, 21–37. [Google Scholar] [CrossRef] [PubMed]
- Saelens, B.E.; Sallis, J.F.; Black, J.B.; Chen, D. Neighborhood-Based Differences in Physical Activity: An Environment Scale Evaluation. Am. J. Public Health 2003, 93, 1552–1558. [Google Scholar] [CrossRef] [PubMed]
- Millstein, R.A.; Cain, K.L.; Sallis, J.F.; Conway, T.L.; Geremia, C.; Frank, L.D.; Chapman, J.; Van Dyck, D.; Dipzinski, L.R.; Kerr, J.; et al. Development, Scoring, and Reliability of the Microscale Audit of Pedestrian Streetscapes (MAPS). BMC Public Health 2013, 13, 403. [Google Scholar] [CrossRef]
- Ewing, R.; Handy, S.; Brownson, R.C.; Clemente, O.; Winston, E. Identifying and Measuring Urban Design Qualities Related to Walkability. J. Phys. Act. Health 2006, 3, S223–S240. [Google Scholar] [CrossRef]
- Lovasi, G.S.; Neckerman, K.M.; Quinn, J.W.; Weiss, C.C.; Rundle, A. Effect of Individual or Neighborhood Disadvantage on the Association Between Neighborhood Walkability and Body Mass Index. Am. J. Public Health 2009, 99, 279–284. [Google Scholar] [CrossRef]
- Karros, D.J. Statistical Methodology: II. Reliability and Validity Assessment in Study Design, Part B. Acad. Emerg. Med. 1997, 4, 144–147. [Google Scholar] [CrossRef]
- Messick, S. Test Validity: A Matter of Consequence. Soc. Indic. Res. 1998, 45, 35–44. [Google Scholar] [CrossRef]
- De Barros Ahrens, R.; Da Silva Lirani, L.; De Francisco, A.C. Construct Validity and Reliability of the Work Environment Assessment Instrument WE-10. Int. J. Environ. Res. Public Health 2020, 17, 7364. [Google Scholar] [CrossRef] [PubMed]
- Ben-Joseph, E.; Lee, J.S.; Cromley, E.K.; Laden, F.; Troped, P.J. Virtual and Actual: Relative Accuracy of On-Site and Web-Based Instruments in Auditing the Environment for Physical Activity. Health Place 2013, 19, 138–150. [Google Scholar] [CrossRef]
- Mahmood, A.; O’Dea, E.; Bigonnesse, C.; Labbe, D.; Mahal, T.; Qureshi, M.; Mortenson, W.B. Stakeholders Walkability/Wheelability Audit in Neighbourhoods (SWAN): User-Led Audit and Photographic Documentation in Canada. Disabil. Soc. 2020, 35, 902–925. [Google Scholar] [CrossRef]
- Mahmood, A.; Rikhtehgaran, F.; Nasiri, R.; Hedayati, N.; Pandsheno, S.; Sharrock, A.; Mora, D.; Hosseini, S.; Routhier, F.; Mortenson, W. The Stakeholders’ Walkability/Wheelability Audit in Neighbourhoods (SWAN): Protocol for a Mixed Methods Study. JMIR Res. Protoc. 2025, 14, e60553. [Google Scholar] [CrossRef]
- Tremblay, M.; Kingsley, B.; Gokiert, R. Engaging Vulnerable Youth in Community/Based Participatory Research: Opportunities and Challenges. J. Community Engagem. High. Educ. 2018, 10, 5. [Google Scholar]
- Mahmood, A.; Labbé, D. ‘An Accessible Route Is Always the Longest’: Older Adults’ Experience of Their Urban Environment Captured by User-Led Audits and Photovoice. In Aging People, Aging Places; Policy Press: Bristol, UK, 2021; pp. 27–44. [Google Scholar]
- Cain, K.L.; Millstein, R.A.; Geremia, C.M. Microscale Audit of Pedestrian Streetscapes (MAPS): Data Collection & Scoring Manual; University California San Diego: San Diego, CA, USA, 2012; Available online: http://sallis.ucsd.edu/Documents/Measures_documents/MAPS%20Manual_v1_010713.pdf (accessed on 8 August 2013).
- Sallis, J.F.; Cain, K.L.; Conway, T.L.; Gavand, K.A.; Millstein, R.A.; Geremia, C.M.; Frank, L.D.; Saelens, B.E.; Glanz, K.; King, A.C. Is Your Neighborhood Designed to Support Physical Activity? A Brief Streetscape Audit Tool. Prev. Chronic Dis. 2015, 12, e141. [Google Scholar] [CrossRef]
- Glanz, K.; Handy, S.L.; Henderson, K.E.; Slater, S.J.; Davis, E.L.; Powell, L.M. Built Environment Assessment: Multidisciplinary Perspectives. SSM Popul. Health 2016, 2, 24–31. [Google Scholar] [CrossRef] [PubMed]
- Hess, P.M.; Farrow, J.; Walk, J. Walkability in Toronto’s High-Rise Neighbourhoods: 2011 Final Report; Canadian Electronic Library: Ottawa, ON, Canada, 2011. [Google Scholar]
- Keysor, J.; Jette, A.; Haley, S. Development of the Home and Community Environment (HACE) Instrument. J. Rehabil. Med. 2005, 37, 37–44. [Google Scholar] [CrossRef]
- Ferreira, M.; Sanches, S. Proposal of a Sidewalk Accessibility Index. J. Urban Environ. Eng. 2007, 1, 1–9. [Google Scholar] [CrossRef]
- Mokkink, L.B.; Terwee, C.B.; Patrick, D.L.; Alonso, J.; Stratford, P.W.; Knol, D.L.; Bouter, L.M.; de Vet, H.C.W. The COSMIN Study Reached International Consensus on Taxonomy, Terminology, and Definitions of Measurement Properties for Health-Related Patient-Reported Outcomes. J. Clin. Epidemiol. 2010, 63, 737–745. [Google Scholar] [CrossRef]
- McHugh, M.L. Interrater Reliability: The Kappa Statistic. Biochem. Med. 2012, 22, 276–282. [Google Scholar] [CrossRef]
- Cohen, J. A Coefficient of Agreement for Nominal Scales. Educ. Psychol. Meas. 1960, 20, 37–46. [Google Scholar] [CrossRef]
- Flo, J.; Landmark, B.; Hatlevik, O.E.; Fagerström, L. Using a new interrater reliability method to test the modified Oulu Patient Classification instrument in home health care. Nurs. Open 2018, 5, 167–175. [Google Scholar] [CrossRef] [PubMed] [PubMed Central]
Stream | No. of Participants | Age (Mean) | Gender Distribution (%) |
---|---|---|---|
People using mobility assistive devices | 24 | 51 | Female (42%) |
People who are deaf or hard of hearing | 10 | 41 | Female (40%) |
People living with cognitive impairment | 10 | 70 | Female (30%) |
Older adults aged 60+ | 10 | 65 | Female (80%) |
Domain/Subdomain | % Agreement | K |
---|---|---|
1. Functionality | 84.58 | 0.69 |
1-1. Crossing Functionality | 84.58 | 0.69 |
1-1-1. Crossing C Functionality | 84.58 | 0.69 |
1-1-2. Crossing D Functionality | 83.48 | 0.67 |
1-2. Sidewalk Functionality | 70.04 | 0.47 |
2. Safety | 90.97 | 0.73 |
2-1. Traffic Safety | 90.97 | 0.73 |
2-2. Pedestrian Safety | 79.57 | 0.59 |
3. Land Use and Supportive Features | 81.38 | 0.61 |
4. Appearance and Maintenance | 79.79 | 0.66 |
5. Social Aspects | 80.64 | 0.64 |
Total | 85.22 | 0.67 |
SWAN Domain | Comparator Measure | Correlation (r) (95% CI) | Interpretation |
---|---|---|---|
Sidewalk Functionality | SI (Sidewalk Evenness) | 0.44 (0.27, 0.58) | Moderate |
Street Crossing Functionality | SI (Intersection Design) | 0.79 (0.71, 0.85) | Strong |
Appearance and Maintenance | SI (Surface Condition) | 0.12 (−0.07, 0.30) | Weak |
Pedestrian Safety | HACE (Unevenness) | 0.31 (0.04, 0.53) | Moderate |
Traffic Safety | HACE (Safety) | 0.29 (0.02, 0.52) | Moderate |
Social Aspects | HACE (Places to Rest) | 0.30 (0.03, 0.53) | Moderate |
Land Use and Supportive Features | Walk Score | 0.28 (0.01, 0.51) | Moderate |
Domain | Raw Alpha |
---|---|
1. Functionality | 0.87 |
1.1. Street Crossing Functionality | 0.87 |
1.2. Sidewalk Functionality | 0.80 |
2. Safety | 0.74 |
2.1. Traffic Safety | 0.81 |
2.2. Pedestrian Safety | 0.54 |
3. Land Use and Supportive Features | 0.69 |
4. Appearance and Maintenance | 0.55 |
5. Social Aspects | 0.45 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Nasiri, R.; Mahmood, A.; Mortenson, W.B. Enhancing Urban Accessibility: Reliability and Validity Assessment of the Stakeholders’ Walkability/Wheelability Audit in Neighbourhoods Tool. Disabilities 2025, 5, 42. https://doi.org/10.3390/disabilities5020042
Nasiri R, Mahmood A, Mortenson WB. Enhancing Urban Accessibility: Reliability and Validity Assessment of the Stakeholders’ Walkability/Wheelability Audit in Neighbourhoods Tool. Disabilities. 2025; 5(2):42. https://doi.org/10.3390/disabilities5020042
Chicago/Turabian StyleNasiri, Rojan, Atiya Mahmood, and W. Ben Mortenson. 2025. "Enhancing Urban Accessibility: Reliability and Validity Assessment of the Stakeholders’ Walkability/Wheelability Audit in Neighbourhoods Tool" Disabilities 5, no. 2: 42. https://doi.org/10.3390/disabilities5020042
APA StyleNasiri, R., Mahmood, A., & Mortenson, W. B. (2025). Enhancing Urban Accessibility: Reliability and Validity Assessment of the Stakeholders’ Walkability/Wheelability Audit in Neighbourhoods Tool. Disabilities, 5(2), 42. https://doi.org/10.3390/disabilities5020042