Development, Validation, and Application of School Audit Tool (SAT): An Effective Instrument for Assessing Traffic Safety and Operation Around Schools

Abstract

There is a need for a reliable school audit tool with well-defined scales to convert qualitative evaluation of existing school sites into a quantitative assessment in order to help public agencies to improve schools’ safety and efficiency. In this study, a new, simple, and versatile School Audit Tool (SAT) was developed and tested. SAT was formed using a 30-item checklist categorized into four domains: school site assessment, road network assessment, parking/loading assessment, and active transport assessment. The tool was applied on a sample of 22 schools. Then, categorical and item-by-item Intraclass Correlation Coefficients (ICC) were calculated to validate the tool. The results showed acceptable overall test-retest (ICC = 0.919) and inter-rater reliability (ICC = 0.843) across all items and domains. SAT’s adaptable framework to assess and compare the safety and efficiency of schools is reliable, easy-to-use, and comprehensive. The tool is also effective in ranking schools and identifying items that may require upgrades or modifications.

1. Introduction

Environmentally safe and operationally efficient schools are neighborhood-level goals of any public agency when it comes to children’s health and safety. Schools are indispensable infrastructures within communities, and they are generally connected to roadways or situated adjacent to road networks. While connected road networks are essential to providing direct routes to schools [1], this arrangement opens the possibilities of road crashes and consequent injuries sustained by all those associated with schools, school children being the most vulnerable. There are many road design guidelines that consider both traffic operation and safety around schools. Despite undertaking such design measures, in many cases, road conditions before, during, and after construction might lead to crashes and consequent casualties. For this reason, there is always a need for reliable tools that can help deliver safer and more efficient roads around schools.

Road safety audits are one of the widely accepted tools for the evaluation of transportation projects. By definition, depending on the scope of the project, road safety audits are formal reports submitted by one or more than one qualified independent examiner [2]. The examiner studies the probable interaction between the road environment and all road users to identify any potential crashes or road safety issues. The report is prepared based on some widely accepted safety principles and safety checklists. Based on the findings and recommendations of the auditor, the project designer and the client then take preventative actions such as changing the design or the practice before any crashes occur. Also, according to the New Zealand Transport Agency’s guideline for road safety audit procedures for projects, a road safety audit qualitatively reports the road safety issues on roads and, consequently, provides suggestions for improvement [3]. In most cases, the cost of a road safety audit is negligible in comparison to the losses that are incurred following crashes that occur on roads on which road safety audits were not conducted. Moreover, applying road design and construction standards cannot entirely ensure traffic safety on roads. According to Huvarinen et al., one of the main tools for reducing the risk of traffic accidents for the past decades has been road safety audits [4]. In other words, the prime objective of a road safety audit tool is to help deliver safer roads and not to evaluate compliance with the design standards [3].

Likewise, a school audit tool with well-defined scales could be used to convert qualitative evaluation of existing school sites into a quantitative assessment to help improve school safety and efficiency. Hence, the aim of this study is to develop a school audit tool with a simple and comprehensive framework that could be easily modified and used by public agencies and urban planners worldwide to rate existing schools. The audit tool would further aid in identifying what aspects of a school must be changed or improved to get an acceptable score.

2. Literature Review

School assessment studies and instruments that are available in the literature are mostly evaluating school site walkability from the perspective of street connectivity, traffic exposure, and residential density [1,5] or the perception of students and parents based on community design, organization, and safety [6,7]. That is, the first approach requires micro-level (assessing concepts of pedestrian-friendly design) and macro-level (density, design, and diversity of land use) data surrounding a school site. Whereas, the second approach is dependent on the assessment of survey data or travel diary completed by students and parents constituting walking information and their perceptions of walkability to school. Besides, a third approach found in the literature is related to studies based on school children’s Body Mass Index (BMI) in relation to walking to school. In short, most studies have focused on school site walkability and active transport assessments. However, road network and parking/loading assessments which are also integral to the overall safety and efficiency assessment of schools have not received as much attention.

Proper school site selection, healthy community design, and ensuring safe routes in a built environment are some crucial strategies that could influence children living at walking and biking distances to establish life-long healthy habits as well as improve their performance at school [8]. Accordingly, Lee et al. proposed a school audit tool due to an evident research gap in this key area of health, safety, and operation of the school environment within communities [9]. Although they developed an audit tool for evaluating urban design qualities, streets, urban paths, parks, and trails, they also identified the necessity for a comprehensive audit tool to assess the school environment and transport features such as school area, frontage, and adjacent streets. Nonetheless, their proposed three-tier school audit, namely street, site, and map-based audit, is by far the most comprehensive school audit tool.

Besides, Zhu and Lee studied walkability and perceptions of safety in terms of economic or ethnic disparities in mostly Hispanic neighborhoods in the United States [10]. They used a 5-point Likert-type scoring scale and binary scale for assessing subjective and objective variables respectively. Binary scale included a yes or no response based on the presence of buffers between walkways and roads, obstructions on sidewalks, perceivable slope, on-street parking, and power lines along streets. Subjective variables were mainly categorized into five sub-groups: maintenance of walkways and roads, visual quality of surrounding buildings, physical amenities such as tree shades along sidewalks, safety with regards to the perceived degree of surveillance from windows along walkways, and other conveniences like air quality and convenience of walking.

Most recently, Tarun et al. investigated 16 schools in Delhi, India, based on a field audit tool combined with structured outdoor observation [11]. They utilized the SPEEDY (Sport, Physical activity and Eating behavior: Environmental Determinants in Young people) audit tool developed by Van Sluijs et al. [12] and validated in the U.K. [13]. They evaluated 39 items divided into six categories, namely access to school, surrounding area, school grounds, aesthetics, usage, and overall environment. The majority of the observed schools had car or bus parking, drop-off, and pick-up areas (above 62.5%), while a quarter of the schools had traffic calming devices. Besides, only one or two schools had separate cycle lanes, pavements, and marked pedestrian crossings. However, none of the schools had proper traffic signage such as school zone signs, speed limit signs, stop or yield signs, warning signs, pedestrian crossing signs, or bicycle signs.

On the other hand, other studies on schools have limited their research to studying the environmental factors, neighborhood organization, community design, and parent-child perceptions with regards to walkability. School site walkability is known as the potential for children to walk to school as part of an active school commute. Gallimore et al. identified three major barriers to making communities healthy and active such as macro-environmental elements, microenvironment factors, and perception of parents and children on the concept of walking or biking to school [7]. Features of macro levels include density, diversity of land use, and design, whereas at micro levels, assessment of pedestrian-friendly design concepts are assessed. Higher density and more diverse land-use have been linked with a higher perception of walkability among parents and children. Based on two separate macro design philosophies, two standard suburban and one pilot urban communities were studied by Gallimore et al. [7]. Additionally, micro-environmental barriers were evaluated through the assessment of 24 street blocks by two trained raters. Perception of environmental walkability was collected from 5th graders in two schools in Utah, United States. Based on their walkability audit, they found that perceived barriers were indeed associated with environmental elements such as community organization and walking conditions.

Likewise, Napier et al. also studied the effects of environmental design on parents’ and children’s perception of walkability [6]. However, they also focused their research on studying the combined and separate effects of community design elements and Child BMI (Body Mass Index) related to walking to school. Three different types of community design features were evaluated, namely walkable, less walkable, and mixed. As predicted, in most walkable communities, perception of walking to school was generally more positive. Similar to Gallimore et al., they studied the perception of 193 5th graders and 177 parents in two schools in Utah, United States based on completed surveys. To sum up, Gallimore et al. [7] and Napier et al. [6], investigated both the students’ and parents’ perceptions of walkability from an environmental and neighborhood organization and design perspective.

Moreover, as part of the macro-environmental study, Giles-Corti et al. used two indices to measure the walkability of a school site, namely the association between street connectivity and traffic exposure [1]. Studies have indicated that traffic exposure influences children’s and parent’s perceptions of walkability [14,15]. A total of 238 government primary schools in a metropolitan part of Perth, Western Australia, were studied by Giles-Corti et al. [1] to develop a walkability index (WI) for all primary government schools. The study excluded private primary schools based on the assumption that only government schools had a local school catchment area. It involved a cross-sectional study of years 5, 6 and 7 students totaling 1480 children and 1332 consenting parents.

The concept of pedshed, which is a ratio of the walkable service area (WSA) to the actual area (AA), has been utilized by many researchers to develop the WI [1,16,17]. WSA is a 2 km buffer area around a school that is confined within a polygon created by connecting the surrounding pedestrian network. Whereas AA is a direct 2 km buffer around the school area. Using school-specific WI, Giles-Corti et al. [1] found that 51% of their total school sample (n = 238) had a pedshed ratio that was <0.6 with 0.6 being the pedshed target for a walkable catchment. They concluded in their study that high walkable neighborhoods indicated by high street connectivity and low traffic exposure encouraged more children to regularly walk to school while the opposite was true in the case of high traffic exposure. Therefore, ensuring high street connectivity and low traffic volume in the neighborhood were key factors that contributed to children’s choice of active transport mode, namely, walking to school. Using their WI, they were also able to identify the least (n = 12) and most walkable schools (n = 13) in their sample. However, Giles-Corti et al. [1] also identified some limitations in their walkability index study, particularly the probable effect of residential density on traffic exposure conditions.

Consequently, Christiansen et al. included residential density as part of their evaluation of walkability [5]. They measured the walkability index using three indices and associated it with active school transport (AST). They collected questionnaires and commuting diary data from 11 to 13 years-old students in Denmark. Like the previous study by Giles-Corti et al., this study had a large sample size of 1250 Danish students. They found distance to school to be a factor that significantly weakened this association, that is, students preferred biking to school instead of walking when the distance to school was about 4 km. On the other hand, the children’s perception of the social and physical environment surrounding the school was more of an indicator of AST prediction. They identified factors such as highly walkable school site located within a 2 km neighborhood distance, multiple walking paths, and low-speed traffic in the surrounding area to be associated with AST.

Besides, based on a review paper published in 2017, around 100 studies have been conducted in the past 10–15 years on the associations of the built environment and children’s physical activity [18]. Most of these studies were conducted in Europe, the United States, Australia, and Canada, with a few conducted in Russia and Malaysia, mostly focusing on children aged 3–12 years. The author found that the negative association between distance to school and walking/biking to school was well-established in the literature. However, associations related to population/infrastructure densities, distance to the city center, land use mix, urban form/size, perceptions of neighborhood safety, walking/biking habits of parents, and differences within ethnicities/regions were not properly studied and thus warranted further investigations.

To sum up, most of the studies in the literature assess schools from design, environment, safety, or health perspective. However, there is no audit tool in the literature that could be used by public and urban planning agencies to assess the various traffic safety and operational issues that might be evident around schools. Consequently, there is a need for the development of a simple yet all-inclusive school audit tool that would aid in scoring various relevant items from traffic safety (school site and road network considerations), efficiency (active transportation considerations), and operation (parking and loading considerations) perspectives.

3. Methods

3.1. Initial School Audit Tool Development

Initially, a checklist with a yes or no response format was prepared for pilot assessment of schools based on the factors related to the safety and operation of school sites commonly mentioned in the literature [9,10,11]. From the items listed under “neighborhood-level walkability” in the study by Zhu and Lee, sidewalk completeness for the use of pedestrians, street connectivity, and land-use (residential, commercial, or mix) were considered excluding dynamic items such as estimations of potential walkers and neighborhood density [10]. In addition, the dynamic items listed under “Neighborhood-level safety”, namely average traffic volume, the proportion of high-speed roadways, and crime rate were not considered. However, adjacent road types and the presence of high-speed roads were considered as items that would give an indication of the expected traffic volume and percentage of high-speed roadways around a school.

Moreover, another good source to consider audit items was the 39-items SPEEDY audit tool used by Tarun et al. [11]. In this tool, the 39 items with mostly yes or no response options were divided into six categories, out of which the category “Aesthetics” and “Usage” yielded no relevant items for this study. From the categories “School grounds” and “Overall environment”, the presence of parking for cycles on a school site and fencing around the school site were considered respectively. From the category “Access to school”, items indicating the presence of multiple entrances to the school, speed limit signs, and roadside parking were considered. Finally, different items from the category “Surrounding area” which was defined as the area that could be seen from the school entrance were also considered. The presence of existing facilities for all major transport mode choices—car (space for parents to park, drop-off and pick-up children), bus (bus stops), bicycle (separated cycle lanes along roadways), and walking (pavement on one side or both sides of the roadway and marked crossings) were included in the item list. Additionally, items belonging to overall safety were also selected namely the presence of traffic calming measures and various types of signage (school zone and traffic signs).

A collection of these items was also found in the “Street audit”, “School-site audit”, and “Map audit” items used by Lee et al. [9] based on the spatial model commonly known as Behavioral Model of Environment (BME) [19,20] that relates physical activity (walking and biking) with various factors from the built environment. Hence, after refinement, a total of 30 items were selected and sorted into four sections: (i) site selection considerations, (ii) road network considerations, (iii) parking or loading considerations, and (iv) overall problems.

3.2. Pilot Study

Preliminary data for two randomly selected schools within Doha, Qatar, were collected by four trained auditors using an initial school audit tool. Contrary to other school audit studies where each auditor collects data independently [9,10], each response was recorded following an agreement between the auditors eliminating inter-rater issues. All possible relevant data at a school site were recorded with the aid of a four-page-long checklist form. The responses to each question were designed to be a yes or no with space for additional comments to keep the audit tool simple.

General information such as the name of the school, coordinates of the school, education level, and type of school was also recorded. Additionally, relevant pictures were captured for some items to be cross-checked later if necessary. After data collection, the handwritten data on the form were transferred to the electronic version of the form. At this stage, the captured electronic pictures were embedded with relevant items to aid off-site review and analysis. Based on further analysis of the initially collected data and auditors’ review, it was found that the initial school audit tool, although simple, was assessing the schools too generally due to its binary response format. In other words, the initial binary audit tool could not properly capture the variability among the various school assessment items.

3.3. School Audit Tool (SAT) Development

The items in the initial audit tool were recategorized into four new domains, namely school site assessment, road network assessment, parking/loading assessment, and active transport assessment for inclusion in SAT. The four domains cover the major elements of safety and operation evaluation items of schools (see

Table 1. Categorized layout of SAT.

School Audit Tool (SAT) 30-item	School Site Assessment (SSA) 7-item	S1	Proximity to high-speed roads
		S2	Presence of major roads
		S3	Land-use in the surrounding area
		S4	Fence around school
		S5	Location of entrance
		S6	Multiple access points to school
		S7	Congestion problems
	Road Network Assessment (RNA) 7-item	R8	Speed limit signs
		R9	School zone signing and pavement markings
		R10	Speed reduction methods
		R11	Road classification of the adjacent street
		R12	Adequate sight distance
		R13	Public bus accessibility
		R14	Amenities for physically challenged students
	Parking/Loading Assessment (PLA) 9-item	P15	Adequate pick-up/drop-off zones for school buses
		P16	Adequate queuing area
		P17	Parking for service and emergency vehicles
		P18	Staff parking
		P19	Visitor parking
		P20	Safety of parents
		P21	Parking for high school students
		P22	Traffic organization duty
		P23	Parking problems
	Active Transport Assessment (ATA) 7-item	A24	Walking and biking conditions
		A25	Availability of crosswalks
		A26	Availability of sidewalks
		A27	Availability of bike storage
		A28	Availability of bike lanes
		A29	Separation of travel modes
		A30	Pedestrian problems

) to ease data collection and provide a more targeted assessment of schools. Although some items were interchangeable between domains, the items were categorized in a way that addressed each domain most comprehensively. Moreover, to assess the schools in more detail, the initial binary response format for each item was changed to a 4-point Likert-type scale. The 4-point Likert scale was developed to score each item with a greater level of detail and accuracy and consisted of the following ordinal scores: 1 = undesirable, 2 = poor, 3 = acceptable, and 4 = good. Each of the four scores in an item was provided with a short description in the final school audit form to aid auditors to input scores consistently across schools.

3.4. Defining Scales for the 30 Items in SAT

3.4.1. School Site Assessment (SSA) Scales

An underlying principle behind creating the scales for each of the items in the SSA items was the United States Environment Protection Agency (EPA)’s guidelines for selecting an appropriate school site for children [21]. It recommends school sites to be located within communities such that students’ walking or biking distances are minimized, and overall public health benefits are maximized. Hence, it also becomes necessary to make sure school sites and their surrounding environments are safe and healthy for children. Accordingly, the description of the 4-point Likert scales proposed for auditing Items S1 to S7 (proximity to high-speed roads, presence of major roads, land-use in the surrounding area, fence around the school, location of entrance, multiple access points to school, and congestion problems) under the SSA category are summarized in

Table A1. Description of SSA indicators and rating scales.

Indicator	Description of Scales (1 = Undesirable; 2 = Poor; 3 = Acceptable; 4 = Good)
S1. Proximity to high-speed roads	1	The school site is adjacent to one or more high-speed roadways with no speed limit signs.
	2	The school site is adjacent to two or more high-speed roadways (more than 80 kph).
	3	The school site is adjacent to one high-speed roadway (more than 80 kph).
	4	The school site is not adjacent to any high-speed roadways.
S2. Presence of major roads	1	The school site is adjacent to two or more major roads.
	2	The school site is adjacent to one major road without proper crossings.
	3	The school site is adjacent to one major road with a proper crossing.
	4	The school site is not adjacent to any major roads.
S3. Land-use in the surrounding area	1	Surrounding land-use is industrial.
	2	Surrounding land-use is commercial.
	3	Surrounding land-use is mixed-use.
	4	Surrounding land-use is residential.
S4. Fence around school	1	Not fenced school site.
	2	Improperly fenced school site (low height, missing sections, etc.).
	3	A fenced school site with multiple entry/exit points.
	4	A fenced school site with a single entry/exit point.
S5. Location of entrance	1	School entrance is located such that a substantial number of serious conflict points may exist.
	2	School entrance is located such that some number of serious conflict points may exist.
	3	School entrance is located such that a little number of conflict points may exist.
	4	School entrance is located such that almost no serious conflict points exist.
S6. Multiple access points to school	1	More than two access points to the school site.
	2	Two access points to the school site from two different adjacent streets.
	3	Two access points to the school site from one adjacent street.
	4	Single access point to the school site.
S7. Congestion problems	1	Heavy congestion at the school site during the day.
	2	Heavy congestion at the school site during arrival and dismissal hours.
	3	Medium congestion at the school site during arrival and dismissal hours.
	4	No or low congestion at the school site during arrival and dismissal hours.

of Appendix A.

3.4.2. Road Network Assessment (RNA) Scales

According to the World Health Organization (WHO), when the vehicle speed is below 30 kph, practically all crashes involving motorists and vulnerable road users can cause non-fatal injuries [22]. Thus, the severity and frequency of traffic crashes involving motorists or vulnerable road users surrounding school sites depend on factors such as speed limits, interactions between bicyclists and motorists, visibility, road classification, and so on. The scales of the road network assessment items (R8 to R14) are designed to evaluate such risks. Accordingly, the description of the RNA indicators (speed limit signs, school zone signing and pavement markings, speed reduction methods, road classification of the adjacent street, adequate sight distance, public bus accessibility, and amenities for physically-challenged students) and the corresponding rating scales are summarized in

Table A2. Description of RNA indicators and rating scales.

Indicator	Description of Scales (1 = Undesirable; 2 = Poor; 3 = Acceptable; 4 = Good)
R8. Speed limit signs	1	Speed limit signs are not provided around the school.
	2	Speed limit signs are poorly placed around the school.
	3	Speed limit signs are provided only near entry/exit points.
	4	Speed limit signs are properly provided around the school.
R9. School zone signing and pavement markings	1	School zone signing and pavement markings are not provided.
	2	School zone signing and pavement markings are provided but are improper.
	3	School zone signing and pavement markings are properly provided but are in poor condition.
	4	School zone signing and pavement markings are properly provided and are in good condition.
R10. Speed reduction methods	1	Speed reduction methods (speed humps, speed bumps, speed cushions, speed tables, etc.) are not provided.
	2	Speed reduction methods are provided with no signing or pavement markings.
	3	Speed reduction methods are provided with improper signing and pavement markings.
	4	Speed reduction methods are provided with proper signing and pavement markings.
R11. Road classification of the adjacent street	1	An expressway road is adjacent to the school.
	2	An arterial road is adjacent to the school.
	3	A collector road is adjacent to the school.
	4	Only local roads are adjacent to the school.
R12. Adequate sight distance	1	High potential for traffic crashes due to the absence of adequate visibility or proper sight distance on the road adjacent to the school access, measured by the presence of horizontal curves, vertical curves, intersection sight distance obstructions, or other obstructions.
	2	Medium potential for traffic crashes due to the obstacles stated above.
	3	Low potential for traffic crashes due to the obstacles stated above.
	4	No or very low potential for traffic crashes due to the obstacles stated above.
R13. Public bus accessibility	1	No public bus stops exist near the school.
	2	Public bus stops exist far away from the school.
	3	Improperly designed public bus stops exist nearby school.
	4	Properly designed public bus stops exist nearby school.
R14. Amenities for physically challenged students	1	No facilities are available for physically challenged students.
	2	Improperly designed and marked facilities are available.
	3	Properly designed but improperly marked facilities are available.
	4	Properly designed and marked facilities are available.

of Appendix A.

3.4.3. Parking/Loading Assessment (PLA) Scales

While adequate parking and loading facilities are important to avoid queuing and congestion of vehicles in and around a school site, buses and cars idling at a school site for long periods also have negative implications for the health of children. Their developing lungs are more vulnerable to serious adverse effects of high concentrations of exhaust emissions from diesel engines such as cancer, bronchial irritation, asthma, and allergies [23]. Hence, cases that involved unnecessary idling of vehicles such as school buses, cars, or delivery trucks at the school site were given the lowest scores within the scales. The descriptions of the 4-point rating scales for the nine PLA indicators (adequate pick-up/drop-off zones for school buses, adequate queuing area, parking for service and emergency vehicles, staff parking, visitor parking, safety of parents, parking for high school students, traffic organization duty, and parking problems) are summarized in

Table A3. Description of PLA indicators and rating scales.

Indicator	Description of Scales (1 = Undesirable; 2 = Poor; 3 = Acceptable; 4 = Good)
P15. Adequate pick-up/drop-off zones for school buses	1	Drop-off/pick-up zones for school buses are not available.
	2	Drop-off/pick-up zones for school buses are available off-site.
	3	Inadequate drop-off/pick-up zones for school buses are available on-site.
	4	Adequate drop-off/pick-up zones for school buses are available on-site.
P16. Adequate queuing area	1	No designated area for queuing of parents’ vehicles.
	2	Limited storage area for queuing of parents’ vehicles.
	3	Adequate storage area for queuing of parents’ vehicles.
	4	Properly designed and adequate storage area for queuing of parents’ vehicles.
P17. Parking for service and emergency vehicles	1	No permanently reserved zones for service vehicles (deliveries, trash pickup, handicap, etc.) and emergency vehicles (ambulances, fire trucks, etc.).
	2	Limited and unmarked reserved zones for service and emergency vehicles.
	3	Limited and marked reserved zones for service and emergency vehicles.
	4	Adequate reserved and marked zones for service and emergency vehicles.
P18. Staff parking	1	Staff parking is not accommodated.
	2	Staff parking is accommodated off-site.
	3	Limited staff parking is accommodated on-site.
	4	Adequate staff parking is accommodated on-site.
P19. Visitor parking	1	Visitor parking is not accommodated.
	2	Visitor parking is accommodated off-site.
	3	Limited visitor parking is accommodated on-site.
	4	Adequate visitor parking is accommodated on-site.
P20. Safety of parents	1	Not safe for parents to drop-off/pick-up, park, or accompany their child in and out of school.
	2	Safe for parents to drop-off/pick up but not safe to park or accompany their child in and out of school.
	3	Safe for parents to drop-off/pick-up and park but not safe to accompany their child in and out of school.
	4	Safe for parents to drop-off/pick-up, park, and accompany their child in and out of school.
P21. Parking for high school students**	1	Student parking is not provided.
	2	Student parking is provided off-site.
	3	Limited student parking is provided on-site.
	4	Adequate student parking is accommodated on-site.
P22. Traffic organization duty	1	No one is organizing traffic in front of the school during school hours.
	2	Someone is organizing traffic in front of the school during either arrival or dismissal hours.
	3	Someone is organizing traffic in front of the school during arrival and dismissal hours.
	4	Someone is organizing traffic in front of the school during school hours.
P23. Parking problems	1	Major parking problem (single or multiple rows of illegal parking) during arrival and dismissal hours.
	2	Moderate parking problem (drivers circulate in search of parking) during arrival and dismissal hours.
	3	Minor parking problem (parking available but occasionally far from school) during arrival and dismissal hours.
	4	No parking problem (parking available) during arrival and dismissal hours.

**Not applicable in the case of elementary and middle schools.

of Appendix A.

3.4.4. Active Transport Assessment (ATA) Scales

By providing sidewalks, safe bike lanes, and safe crossings, children could be encouraged to walk or bike to school from their homes. This would, in turn, help them achieve their weekday recommended levels of 60 min of above moderate physical activity. Evidence shows that such activities bring about improvements in both health and academic achievement of children [24]. If safe and comfortable walking and biking routes are provided for school children in combination with selecting school sites that are in close proximity to the students’ residences, students’ share of active transport use will increase resulting in lower vehicular trips, emissions, and transportation costs [25]. Thus, the scales and items in this section were designed to achieve these goals. The ATA assessment indicators (walking and biking conditions, availability of crosswalks, availability of sidewalks, availability of bike storage, availability of bike lanes, separation of travel modes, and pedestrian problems) and the descriptions of the corresponding rating scales are summarized in

Table A4. Description of ATA indicators and rating scales.

Indicator	Description of Scales (1 = Undesirable; 2 = Poor; 3 = Acceptable; 4 = Good)
A24. Walking and biking conditions	1	The school site does not encourage walking or biking.
	2	The school site encourages walking and biking without proper crossings.
	3	The school site encourages walking only due to the availability of proper signs, pavement markings, and crossings.
	4	The school site encourages walking and biking due to the availability of proper signs, pavement markings, and crossings.
A25. Availability of crosswalks	1	Absence of crosswalks for safe crossing.
	2	Crosswalks are located far away from the school entrance.
	3	Crosswalks are located close to the school entrance but are improper for safe crossing.
	4	Crosswalks are close to the school entrance and are appropriate for safe crossing.
A26. Availability of sidewalks	1	Sidewalks are not available around the school.
	2	Unconnected sidewalks are available around the school.
	3	Connected sidewalks are available around the school.
	4	A well-connected sidewalk network is available in the neighborhood.
A27. Availability of bike storage	1	No space is available for bike storage.
	2	Space available off-site for bike storage.
	3	Limited space is available on-site for bike storage.
	4	Adequate space is available on-site for bike storage.
A28. Availability of bike lanes	1	Bike lanes are not available around the school.
	2	Unconnected bike lanes are available around the school.
	3	Connected bike lanes are available around the school.
	4	A well-connected bike lanes network is available in the neighborhood.
A29. Separation of travel modes	1	No separation of travel modes at the school.
	2	Separation of school buses and private vehicles at the school.
	3	Separation of school buses, private vehicles, and pedestrians at the school.
	4	Separation of all possible travel modes (school buses, private vehicles, bicycles, and pedestrians) at the school.
A30. Pedestrian problems	1	Unsafe walking and crossing conditions, and drivers do not yield to pedestrians.
	2	Safe walking conditions, unsafe crossing conditions, and drivers do not yield to pedestrians.
	3	Safe walking and crossing conditions, but drivers do not yield to pedestrians.
	4	Safe walking and crossing conditions, and drivers yield to pedestrians.

of Appendix A.

3.5. Final Data Collection

Once the scales and items were finalized, the final data collection was conducted. Using the final SAT form, two trained auditors independently collected scores for 10 randomly selected public and 12 randomly selected private schools within Doha, Qatar. The distribution of public schools (45.5%) and private schools (54.5%) was thus almost equal. The final school audit tool was applied on two major school types found in Qatar to validate and check the versatility of the model in assessing two different school types. The main difference between the public and private schools lies in their infrastructure—the former is usually properly planned and constructed to function as a school, whereas the latter is not always planned and constructed as a school. Due to a limitation of resources or other constraints, some private schools are often operated in customized commercial or residential spaces or in groups of houses converted to function as school buildings. As a result, it was expected that the assessment scores would be higher for public schools compared to the private schools in Qatar. Hence, by applying the proposed school audit tool on the selected schools as a case study, the application of the proposed tool on schools was first validated. In addition, by applying it on two different school types, the versatility of the tool was also assessed.

4. Analysis

4.1. Validation of SAT

Reliability tests are a standard requisite for validating any audit tool that generates perception-based observational data. Without such tests, the robustness and versatility of an instrument remain questionable. Since the measure of the variables (audit items) in this study was changed from dichotomous measure to a 4-point Likert-type measure, between Kappa statistics and Intraclass Correlation Coefficients (ICCs), ICCs measure was found to be the more appropriate statistical method to test reliability. On the other hand, according to Hallgren, when there is more than one rater in a study, ICCs measures provide an insight into the degree of disagreement between raters with lower ICCs indicating smaller disagreements and vice versa [26]. Moreover, with a range from 0 to 1, an ICC value of 0 and 1 indicates random and perfect agreement respectively. However, such absolute values could also be interpreted as raters having either low or no variance. Qualitatively, ICC values greater than 0.75 could be considered to hold good to excellent agreement. On the other hand, ICC values less than 0.6 could be interpreted as poor to fair, while values between 0.6 and 0.75 could be considered to have a moderate agreement [9,27].

Besides, statistical analysis for inter-rater and test-retest reliability tests is evident in the literature as a means of validating newly developed school assessment instruments [9,10]. In general, inter-rater reliability tests evaluate the agreement between two auditors independently rating the same schools, while test-retest reliability tests measure consistency between audits conducted by the same rater for the same schools with a recommended time-interval of one to two weeks in between.

Zhu and Lee tested inter-rater reliability for independent audits conducted by two researchers using ICCs [10]. Likewise, Tarun et al. conducted the same test to assess the level of agreement for each audit item rated independently by six trained auditors for 16 schools [11]. Lee et al. also calculated the ICCs for their 5-point Likert-type items. They selected the most appropriate ICC variant based on the nature of their study; that is, a two-way mixed-effects model with an absolute agreement and average measurement [9]. A two-way and mixed-effects model was selected since their auditors were specified and not randomly selected. Moreover, as auditors were expected to score similar absolute values for each item, they chose absolute agreement to characterize good inter-rater reliability. Lastly, they specified average measures of auditors as the unit of analysis based on the assumption that ICCs for average measures would be higher and more reliable than ICCs for individual measures. Consequently, the inter-reliability and the test-retest reliability test for this study also warrant similar ICC variant [26]. For this study, separate item-by-item inter-rater (see

Table 2. Results of categorical item-by-item inter-rater reliability tests- Intraclass Correlation Coefficients (ICC) values.

Item	ICC	95% Confidence Interval (CI)		p-Value
		Lower Limit	Upper Limit
SSA Category
S1. Proximity to high-speed roads	0.900	0.760	0.958	<0.0001
S2. Presence of major roads	0.884	0.718	0.952	<0.0001
S3. Land-use in the surrounding area	0.947	0.873	0.978	<0.0001
S4. Fence around school	0.900	0.756	0.958	<0.0001
S5. Location of entrance	0.798	0.523	0.915	<0.0001
S6. Multiple access points to school	0.812	0.548	0.922	<0.0001
S7. Congestion problems	0.503	−0.227	0.796	0.063
RNA Category
R8. Speed limit signs	0.983	0.957	0.993	<0.0001
R9. School zone signing and pavement markings	0.989	0.973	0.995	<0.0001
R10. Speed reduction methods	0.886	0.730	0.953	<0.0001
R11. Road classification of the adjacent street	0.929	0.832	0.970	<0.0001
R12. Adequate sight distance	0.576	0.010	0.822	0.009
R13. Public bus accessibility	1.000	-	-	-
R14. Amenities for physically challenged students	0.830	0.465	0.936	<0.0001
PLA Category
P15. Adequate pick-up/drop-off zones for school buses	0.491	−0.207	0.787	0.064
P16. Adequate queuing area	0.676	0.250	0.863	0.004
P17. Parking for service and emergency vehicles	0.846	0.629	0.936	<0.0001
P18. Staff parking	0.882	0.720	0.951	<0.0001
P19. Visitor parking	0.817	0.515	0.927	<0.0001
P20. Safety of parents	0.938	0.816	0.976	<0.0001
P21. Parking for high school students	0.916	0.800	0.965	<0.0001
P22. Traffic organization duty	0.897	0.729	0.953	<0.0001
P23. Parking problems	0.863	0.669	0.943	<0.0001
ATA Category
A24. Walking and biking conditions	0.776	0.464	0.907	0.001
A25. Availability of crosswalks	0.903	0.764	0.960	<0.0001
A26. Availability of sidewalks	0.879	0.595	0.956	<0.0001
A27. Availability of bike storage	0.792	0.506	0.913	<0.0001
A28. Availability of bike lanes	1.000	-	-	-
A29. Separation of travel modes	0.741	0.393	0.981	0.001
A30. Pedestrian problems	0.869	0.682	0.946	<0.0001

) and test-retest (see

Table 3. Results of categorical item-by-item test-retest reliability tests (ICC values).

Item	ICC	95% CI		p-Value
		Lower Limit	Upper Limit
SSA Category
S1. Proximity to high-speed roads	0.945	0.876	0.977	<0.0001
S2. Presence of major roads	0.925	0.820	0.969	<0.0001
S3. Land-use in the surrounding area	1.000	-	-	-
S4. Fence around school	0.774	0.454	0.906	0.001
S5. Location of entrance	0.823	0.568	0.927	<0.0001
S6. Multiple access points to school	0.927	0.827	0.970	<0.0001
S7. Congestion problems	0.910	0.758	0.964	<0.0001
RNA Category
R8. Speed limit signs	0.978	0.940	0.991	<0.0001
R9. School zone signing and pavement markings	0.976	0.941	0.990	<0.0001
R10. Speed reduction methods	1.000	-	-	-
R11. Road classification of the adjacent street	1.000	-	-	-
R12. Adequate sight distance	0.508	−0.900	0.788	0.041
R13. Public bus accessibility	0.879	0.712	0.949	<0.0001
R14. Amenities for physically challenged students	0.987	0.968	0.994	<0.0001
PLA Category
P15. Adequate pick-up/drop-off zones for school buses	0.892	0.740	0.955	<0.0001
P16. Adequate queuing area	0.875	0.702	0.948	<0.0001
P17. Parking for service and emergency vehicles	0.858	0.664	0.941	<0.0001
P18. Staff parking	0.938	0.849	0.974	<0.0001
P19. Visitor parking	0.964	0.903	0.985	<0.0001
P20. Safety of parents	0.946	0.870	0.977	<0.0001
P21. Parking for high school students	0.969	0.926	0.987	<0.0001
P22. Traffic organization duty	0.968	0.923	0.987	<0.0001
P23. Parking problems	0.860	0.665	0.941	<0.0001
ATA Category
A24. Walking and biking conditions	0.916	0.801	0.965	<0.0001
A25. Availability of crosswalks	0.974	0.938	0.989	<0.0001
A26. Availability of sidewalks	0.965	0.916	0.985	<0.0001
A27. Availability of bike storage	1.000	-	-	-
A28. Availability of bike lanes	1.000	-	-	-
A29. Separation of travel modes	0.919	0.805	0.966	<0.0001
A30. Pedestrian problems	0.882	0.714	0.951	<0.0001

) ICCs were measured for every sample school (n = 22), every auditor (n = 2), and every round of ratings (n = 3) under the four assessment categories, SSA, RNA, PLA, and ATA [9].

The ICCs measures provided an insight into the degree of disagreement between raters, with lower ICCs indicating smaller disagreements and vice versa [9,27]. As a result, based on analysis of the 30 item-by-item ICCs results for both statistical tests, the majority (80%) of the items in inter-rater and test-retest tests held good to excellent agreement (above 0.750). Ratings of two items (public bus accessibility and bike lane items) and five items (land-use in surrounding area, speed reduction methods, road classification of adjacent street, adequate bicycle storage, and bike lane items) in inter-rater and test-retest tests respectively were in perfect agreement (ICC = 1), that is, the ratings had no variance. In the case of the inter-rater test, only one item (adequate queuing area) with an ICC value of 0.676 was considered to be in moderate agreement.

On the other hand, three items (congestion problem, adequate sight distance, adequate parking and drop-off space) and only one item (adequate sight distance) in inter-rater and test-retest tests respectively were in poor to fair agreement due to ICC values around 0.500, indicating fair agreement between the ratings. Overall, the reliabilities achieved through the test-retest test were in higher agreement than those achieved through the inter-rater test similar to a previous study [9]. This was expected since each of the items were rated based on raters’ perceptions. The inter-rater reliabilities could be further improved through a more detailed image and discussion-based training of the school audit tool.

Besides, based on the maximum, minimum, and mean categorical and overall ICCs results summarized in

Table 4. Overall ICC values of reliability tests.

Category	SSA		RNA		PLA		ATA		Overall
Test statistics	Inter-rater	Test-retest	Inter-rater	Test-retest	Inter-rater	Test-retest	Inter-rater	Test-retest	Inter-rater	Test-retest
Max.	0.947	1.000	1.000	1.000	0.938	0.969	1.000	1.000	1.000	1.000
Min.	0.503	0.774	0.576	0.508	0.491	0.858	0.741	0.882	0.491	0.508
Mean	0.821	0.901	0.885	0.904	0.814	0.919	0.851	0.951	0.843	0.919

, the mean ICCs for SSA, RNA, PLA, and ATA categories and the overall ICCs for test-retest statistical test were always comparatively higher (above 0.9) than the mean and ICCs results for the inter-rater statistical test (above 0.8). Nevertheless, since the mean and overall ICCs were above 0.75 in both cases, the proposed model achieved good to excellent reliability. Hence, although two different types of schools were audited using the proposed audit tool, the results of the overall ICCs indicated a good to excellent level of reproducibility using SAT.

4.2. Qualitative Assessment of Schools Using SAT

Following the development and validation of SAT using the three sets of data for all 22 schools, all 4-point Likert-type ordinal scores (1 = undesirable, 2 = poor, 3 = acceptable, and 4 = good) for the schools were averaged and converted to percentage scores. Then, for each category, the percent scores were measured. Percent scores were also divided into four ranges of scales indicating the four possible qualitative assessments such as (from 0 to 24%) = undesirable, (from 25 to 49%) = poor, (from 50 to 74%) = acceptable and (from 75 to 100%) = good. Figure 1 shows the overall score distribution of assessment of all 10 public schools and 12 private schools in Doha. Overall, the majority of the schools scored acceptable (50–74%). Four schools scored below 50% (poor), including three private schools and one public school. This was an indication that schools in Qatar were mostly operating within acceptable safety and efficiency scores. However, more care must be taken to promote them to good standing.

Besides,

Table 5. Summary of mean qualitative assessment of schools using SAT.

School Type	Public (n = 10)		Private (n = 12)		All (n = 22)
Category	% Score	Quality	% Score	Quality	% Score	Quality
SSA	76	Good	75	Good	75	Good
RNA	69	Acceptable	66	Acceptable	67	Acceptable
PLA	66	Acceptable	58	Acceptable	62	Acceptable
ATA	46	Poor	49	Poor	47	Poor
Overall	64	Acceptable	62	Acceptable	63	Acceptable

tabulates the categorical and overall percent scores and qualitative assessments distributed over the two different types of schools in Doha, Qatar. There is no major difference in the qualitative assessment of public and private schools across all categories; however, public schools scored slightly higher than private schools in all categories except ATA which was expected based on previous studies in this region [28,29]. Overall, SSA categories scored best followed by RNA, PLA, and ATA. In other words, across the two different major school types in Qatar, ATA required the most improvements. That is, for schools in Qatar, active transport items needed the most attention, and parking/loading items could also largely benefit from targeted improvements.

5. Conclusions

Initially, a review of the literature had shown that although a few school audit tools were available [9,11], they were not developed from an all-inclusive (traffic safety, operation, and active transport) perspective. Hence, in this paper, a comprehensive 30-item audit tool was developed and validated with inter-rater (0.843) and test-retest (0.919) reliability tests. The overall results of the reliability tests indicate an overall good to excellent level of reproducibility using SAT. However, like previous studies, the overall reliabilities of the test-retest tests were higher than the results of the inter-rater tests since all the 30 items were largely perceptual in nature [9,30,31,32].

Besides, the proposed school audit tool is divided into four general categories, namely 7-item SSA, 7-item RNA, 9-item PLA, and 7-item ATA. A 4-point Likert scale, having four ordinal scores (undesirable, poor, acceptable, and good), is used to score each item in the SAT. The proposed 4-point Likert scale facilitated the assessment of schools with a greater level of detail and accuracy.

Moreover, the validation and application of the proposed tool are exemplified through a case study of auditing 10 public and 12 private schools in Doha, Qatar, completed by trained and independent auditors. Based on an overall assessment of the ratings of the two auditors, 14 schools (64%) scored acceptable, and an equal and small percentage (18%) of schools scored good (four schools) and poor (four schools), respectively. Furthermore, the majority of the 22 audited schools in Doha received the highest scores in the SSA category followed by RNA, PLA, and ATA categories. In summary, by successfully applying the SAT to assess two different school types, the versatility of the proposed tool in assessing schools of varying characteristics is also established in this paper.

5.1. Implications to Decision Makers

SAT is useful for public agencies and urban planners to identify shortcomings in the existing schools and to come up with improvement plans accordingly. Since the audit tool is divided into four broad categories, namely school site, road network, parking/loading, and active transport assessment, it could also provide insight towards suggesting more targeted public policy changes required to make schools safer and operationally more efficient. To sum up, SAT provides an efficient and easy-to-use framework to assess and compare schools both regionally and worldwide.

Moreover, the four categories of the proposed SAT can be used together or separately depending on the aim and need of relevant public or private agencies. This helps decision-makers to collect and analyze the data under their category of interest. The tool also helps prioritize the items that need immediate modifications or alterations under each category or based on the overall assessment. In other words, the tool’s simple and straightforward design makes it possible for users to include or exclude items that might or might not be relevant in their assessment and address them based on local or regional priorities.

In addition, the inter-rater reliability of the tool could be further improved through more in-depth graphics and discussion-based training of the auditors. During the training phase of the auditors, auditors should be made to collect data at a variety of diverse locations, followed by a controlled and detailed discussion of the issues faced during field tests. Besides following these steps to guarantee good training of the auditors, special care should be given to recruiting reliable data collectors and auditors to attain overall reliable results. The use of advanced data collection technologies such as interactive use of GIS could also help improve the reliability of the proposed tool.

5.2. Limitations

Parking for high school students (Item P21), although scored, was excluded from the analysis, since it is a feature that was applicable for only 45.5% of the sample schools that were high schools (n = 10). While underage driving exists in Qatar, students in high schools do not legally qualify for the minimum age limit (18 years old) to obtain a light-vehicle or motor-cycle driving license [33,34]. Hence, the item was deemed irrelevant for evaluating schools in Qatar using the proposed tool. However, as the item could be valid for use in the Parking/Loading Assessment category for assessing high schools in other regions where the legal age to obtain a driver’s license is 16 years old, it was not excluded from the list of audit items.

Moreover, the inter-rater and test-retest reliability results for SAT signify that one properly trained auditor would be enough to rate schools consistently using this tool. Hence, although the evaluation of the schools was completed based on average ratings, it was carried out this way due to the availability of the three sets of independent data, not due to necessity. In other words, the three sets of data were collected mainly for the purpose of validation of the tool. Besides, the items and definitions of the scales used for this study may not be applicable to schools in other regions. Nonetheless, due to its straightforward framework, the scores and items in SAT could be easily modified to accommodate features unique to other studies. The meaning of undesirable, poor, acceptable, and good could also depend on national policy and socio-economic circumstances of a country or state. Hence, the percent range of scores defining each of the qualitative assessments should be selected after careful consideration of what would be considered good or poor in the locality of interest.

Therefore, it is recommended that any organization carrying out school audits using this tool defines all the items and scales carefully, sets up the detailed manual to train the auditors, and prepares clear audit forms to collect reliable and meaningful data. In short, SAT is a validated, much-needed, and comprehensive assessment tool to evaluate the safety and operation of schools that could be applied to other studies with little caution and modification.