Occupational Risk Assessment in School Food Services: Instruments’ Construction and Internal Validation

Occupational risk assessment in school food services (SFS) should include factors related to occupational hygiene, physical–functional planning, worker health, and good meal production practices. This study aimed to develop an instrument to assess occupational risk in school food services and perform content validation and semantic evaluation. The research included three steps: (i) instrument’s development; (ii) content validation (Delphi); and (iii) semantic evaluation (focus group). After an extensive literature review, four instruments were developed. The Delphi was validated by the Content Validity Coefficient (CVC), with an agreement greater than 90%. The semantic analysis was performed through four meetings using a focus group. After the content and semantic evaluation, there were four final instruments: Identification of Socioeconomic and Demographic Conditions of Food Handlers and Geographical Mapping of SFS (30 items); Knowledge and Attitudes in Food Hygiene Assessment (33 items); Perception of Occupational Risks (16 items); and Mapping of Occupational Risks in SFS (97 items). Instrument’s content validation and semantic evaluation contributed to constructing and validating forms for analyzing occupational risks and adjusting respondents’ language and sociocultural components.


Introduction
The school strongly influences the students' behavior, and when their environment is adequate, it is considered favorable for the formation and consolidation of healthy habits [1]. The global strategy for feeding young children recommends that school menus promote healthy and adequate food to children and adolescents [2]. Thus, using varied, nutritious, and safe food is recommended to promote eating habits that contribute to students' growth and development and academic performance [3].
In Brazil, adequate food is a guaranteed right for students enrolled in public, philanthropic, and community organizations in partnership with the government through the National School Feeding Program (Programa Nacional de Aliemtação Escolar-PNAE). It is considered one of the largest, most comprehensive, and long-lasting programs in the area of school meals in the world [4] and involves the production and offering of meals at school. In this sense, school food services are essential in offering healthy and nutritious meals to Brazilian children [3, 5,6].
each year, there are 360 million non-fatal occupational accidents, representing 19% of the deaths worldwide [25]. In Brazil, despite the underreporting of accidents at work, in 2019, there was a formal record of 10,659 accidents involving foodservice workers, ranking sixth among registered work accidents [26]. In the U.S., the food services presented occupational injuries and illnesses requiring days away from work at a rate of about 7.8 per 100 full-time employees [27]. A report published in the Philippines in 2019 recorded 2959 work accidents in food service activities, representing 6.4% of work accidents recorded in the year [28]. Despite the difference in population numbers between countries and the lack of accurate information about work accidents in food services, it is notable that there are a large number of work accidents in the foodservice sector in Brazil, highlighting the importance of studies on foodservice occupational risks. Studies on work safety in food services are essential to minimize the risk of accidents and the spread of diseases [12]. Knowing the different types of risks that expose the worker's health and life allows finding strategies that help fight existing diseases [12]. In this sense, we aimed to develop an instrument to assess occupational risk in school food services. In addition, to perform its content validation and semantic evaluation, we consider the possible health problems of school food handlers and the importance of the quality of instruments in the assessment of these in public SFS and their relationship to the production of school meals following PNAE regulations.

Materials and Methods
This study is multi-purpose, methodological, and validating in nature, and it is nested within a broader, multilevel study called "Occupational Risk Analysis in School Food Service Units in the city of Salvador, Bahia". This study was carried out in stages (Figure 1), involving the development/adaptation of assessment instruments, application of content and semantic analysis techniques, and validation of the instruments.

Instruments Development
The development of the instruments proposed for investigating occupational risks considered previous studies and Brazilian legislation [29][30][31]. Instruments numbered 1 and 4 were built for application by the field research team, and instruments 2 and 3 were designed from the perspective of self-reporting by food handlers. The first instrument (Identification of socioeconomic and demographic conditions of food handlers and Characteristics of school functioning) consisted of 30 items. The characteristics of the schools were Local, Teaching modality, Operating shifts, SFS Management, Number of meals provided, Planned menu, Number and working hours of food handlers, and Food handlers' demographics (gender, age), labor (working hours), and qualification (training) [31]. After a pilot study in three municipal schools (not included in the primary study sample), other labor issues were included in the instrument (1) employment relationship with the Municipal Secretary of Education (public or contracted employee), function registered in the work, social security card, and training on occupational hygiene.
The second instrument sought to assess the "Level of knowledge and attitudes of food handlers about food hygiene" by self-reporting. It was adapted from the study by Ferreira et al. [31]. The instrument consisted of four blocks and 33 questions, which were assessed using a five-point Likert-type scale: (1) Totally disagree; (2) Partially disagree; (3) Neither agree nor disagree; (4) Partially agree; and (5) Totally agree. The characterization of the level of knowledge and attitudes considered the percentage of correct answers: ≥70% as "Satisfactory", between 50% and 69.9% "Satisfactory with Restriction", and "Unsatisfactory" when the percentage ranged from 0 to 49.9% [32].

Instruments Development
The development of the instruments proposed for investigating occupational risks considered previous studies and Brazilian legislation [29][30][31]. Instruments numbered 1 and 4 were built for application by the field research team, and instruments 2 and 3 were designed from the perspective of self-reporting by food handlers. The first instrument (Identification of socioeconomic and demographic conditions of food handlers and Characteristics of school functioning) consisted of 30 items. The characteristics of the schools were Local, Teaching modality, Operating shifts, SFS Management, Number of meals The "Perception of Occupational Risks by Food Handlers" (third instrument) was prepared with accessible language to the target population, having as a guideline the normative aspects contained in four Brazilian Regulatory Standards (NR): NR 5-Internal Accident Prevention Commission [33]; NR 6-Personal Protective Equipment (PPE) [34]; NR 09-Assessment and Control of Occupational Exposures to Physical, Chemical, and Biological Agents [35] and NR 17-Ergonomics [36]. The adaptation of the instrument also considered the characteristics of the SFS: areas for the production of food, equipment, daily production process, described in the Public Notice 01/2017 [37]. The handlers were asked to respond to the instrument using a five-point Likert-type scale: (1) Strongly disagree; (2) Partially disagree; (3) Neither agree nor disagree; (4) Partially agree; (5) Totally agree.
According to the score obtained by the food handler, the occupational risk perception scale suggested in this study was used: Very Low Perception (0 to 1.5); Low Perception (1.51 to 2.5); Regular Perception (2.51 to 3.5); High Perception (3.51 to 4.5); and Very High Perception (4.51 to 5).
The fourth instrument (Occupational Risk Mapping) also considered the recommendations of the NR mentioned above, in addition to the Technical Regulation of Good Practices for Food Services-RDC n • 216/2004 [30].  [38]. For the classification of occupational risks, the cutoff point used by Stedefeldt et al. [29] was adapted when classifying health risks in SFS. Thus, for the classification of occupational risks, according to the compliance score obtained: very high occupational risk (0 to 25%); high occupational risk (26 to 50%); regular occupational risk (51 to 75%); low occupational risk (76 to 90%); and very low occupational risk (91 to 100%).

Instruments Experts' Evaluation
After the instruments' development and/or adaptation, sixteen experts in food and nutrition were invited to participate in the experts' technique [39]. Inclusion criteria were having professional and/or academic practice of at least two years in the management of food service and/or school meals and/or work safety. There was prior contact with the experts by e-mail, sending an invitation letter, an informed consent form with the presentation of the study, methodology, and characteristics for instruments' validation. The experts received guidance to assess the instruments' content, clarity, and cohesion. It included: (a) assessment of each item's relevance to the question in the SFS; and (b) assessment of each item's relevance to the context experienced by the SFS food handlers. For each assertion, the experts could suggest the items' inclusion, change, or exclusion. The Delphi technique described the congruence between experts [39].
The Content Validity Coefficient (CVC) was used, with the item being accepted when it reached a CVC ≥ 0.80 [40]. Calculations were performed according to the equation of Tilden et al. [41]: number of experts who gave grades 4 and 5 total number of experts × 100.
The CVC calculations comprised four steps: (1) Calculation of the average grades 4 and 5 of each item. According to the average, initial CVC (CVCi) was obtained; (2) Calculation of error (Pei) to remove possible biases between the experts; (3) Final CVC of each item (CVCc) calculated from the subtraction of CVCi by Pei; (4) Total CVC (CVCt) of the questionnaire for each of the characteristics (language clarity and practical relevance), subtracting the mean CVCi from the mean Pei (MPei).
Instrument 2, "Level of Knowledge and Attitudes of Food Handlers in School FNU", and 3, "Perception of Occupational Risks by Food Handlers", were adapted to the sociocultural repertoire of food handlers, using the focus group technique [42]. The sample consisted of seven food handlers who worked in municipal public schools, which were selected by convenience. The criteria used to select the components of the focus group were proximity of the municipal schools to the place where the group meetings would take place and the release of food handlers from work activities by school managers. Focus group discussions or panels are often used to explore a specific set of issues and allow users' views on the quality of care [43], or in this case, the possibility of exploring differences in understanding the items assessed and the experiences and specialties of the workers for whom such instruments were intended. The assessment of the semantics of the instruments took place through content analysis [44].

Semantic Analysis by the Focus Group
The focus group was conducted in four weekly sessions with approximately 120 min each to adapt the instruments to the sociocultural repertoire. Each meeting had a central axis: first, PNAE: role and appreciation of food handlers; second, Foodborne illnesses (FBI): causes and symptoms; attitudes and work practices that pose a risk to the production of safe food for students and workers, limiting factors for adopting good hygiene practice (GHP)-physical structure, training, interpersonal relationships; and third, Occupational risks, the physical structure of school SFS, vector control and urban pests, factors related to the environment and time of preparation and distribution of school meals. The instruments were only presented in the fourth meeting, incorporating terms that emerged in the previous sessions. On that occasion, food handlers were motivated to read, discuss, and take a stand regarding the issues and report whether the instruments were easy to interpret. Consensus for each instrument item was obtained when 80% or more of the food handlers agreed with the assertion [40].
Focus group moderators used the dialogic research method to allow participants to interact with each other and with the moderators [42]. All dialogues between participants and moderators were recorded in all meetings with the due consent of the participants.

Data Analysis
The descriptive analysis of the Delphi Technique data took place in the Microsoft Excel for Windows 10 program. To analyze the permanence of each item, the final CVC was calculated. Items that received agreement ≥0.80 remained in the instrument.
The content analysis technique was used to analyze the semantics of the focus group. If similar concepts were found among participants with 80% agreement, the item would not be changed; otherwise, the discussion of concepts between researchers and participants would continue. Each of the assertions was analyzed considering the assumptions pointed out by Bardin [45].

Ethical Aspects
The project Occupational Risk Analysis in School Food Services in Salvador, Bahia, Brazil was approved by the Research Ethics Committee of the School of Nutrition of the Federal University of Bahia under opinion n • 2.121.882/2017. Food handlers and specialists (experts) signed the Free and Informed Consent Term. Table 1 shows the stages of construction and validation of the instruments for the analysis of occupational health in SFS, and the instruments' results after the four stages of Delphi. The analysis of the final CVC identifies that the instruments obtained values above 90%, as proposed by Alexandre and Coluci [40], who advocate CVC above 80%.

Preparation of Instruments and Content Evaluation by Experts
In the first round of Delphi, there was a consensus among the experts for some items. The first instrument, "Identification of Socioeconomic and Demographic Characteristics of Food Handlers and School Functioning Characteristics" consisted of 30 items (Tables 1 and S1). The final instrument with its 30 items will be able to identify both the characteristics of the schools and the characteristics of the food handlers for later association with instruments that will assess the presence of occupational hazards in the SFS and the level of knowledge and attitudes of food handlers. The second instrument was composed of two blocks: knowledge, with 14 items, and attitudes, with 19 items, totaling 33 items. Delphi was run in three rounds for this construct to reach a cutoff point >80% (Tables S1 and S2).
The third instrument ended with 16 items (Tables S1 and S3). This instrument also required three rounds to reach >80% consensus among experts.
The "Occupational Risk Mapping in SFS" instrument (Tables S1 and S4) was adjusted with three steps, and 51 items were included after submitting to the experts. Although long, it proved theoretically essential to analyze the congruence between knowledge and attitudes in food hygiene and risk perception, using objective criteria within Brazilian legislation [30,[33][34][35][36]. The final instrument with 97 items and five factors was distributed as follows: (a) Physical Risks-28 items; (b) Chemical Risks-12 items; (c) Biological Risks-13 items; (d) Ergonomic hazards-14 items and Mechanical hazards-30 items.

Experts Evaluation
Sixteen experts were invited to participate in the content validation of items as judges. Figure 2 shows the adhesion of judges in each cycle of the Delphi technique. The third instrument ended with 16 items (Table S1 and S3). This instrument also required three rounds to reach >80% consensus among experts.
The "Occupational Risk Mapping in SFS" instrument (Table S1 and S4) was adjusted with three steps, and 51 items were included after submitting to the experts. Although long, it proved theoretically essential to analyze the congruence between knowledge and attitudes in food hygiene and risk perception, using objective criteria within Brazilian legislation [30,[33][34][35][36]. The final instrument with 97 items and five factors was distributed as follows: (a) Physical Risks-28 items; (b) Chemical Risks-12 items; (c) Biological Risks-13 items; (d) Ergonomic hazards-14 items and Mechanical hazards-30 items.

Experts Evaluation
Sixteen experts were invited to participate in the content validation of items as judges. Figure 2 shows the adhesion of judges in each cycle of the Delphi technique. All judges responded to the first round. The percentage of experts adhesion from the first round to the second round was 50% (8) and from the second to the third round was 75% (6). Items were judged according to the criteria of relevance, clarity, and consistency.

Semantic Analysis by the Focus Group
Seven food handlers from municipal public schools were invited to form the focus group, but six attended the four meetings, with 85.71% of the guests participating. All food handlers in the focus group were female, with a mean age of 41 years and meantime in the function of >10 years and a high school degree.
For the orientation of the focus group, two researchers participated in the roles of moderator and co-moderator [42]. The moderator had experience in school meals, management of food services, and teaching. The co-moderator had experience in training and educational activities in municipal public schools with food handlers, teachers, and schoolchildren. As the objective was to identify whether the instruments' semantics met the food handlers' sociocultural repertoire, the researchers used the dialogic research All judges responded to the first round. The percentage of experts adhesion from the first round to the second round was 50% (8) and from the second to the third round was 75% (6). Items were judged according to the criteria of relevance, clarity, and consistency.

Semantic Analysis by the Focus Group
Seven food handlers from municipal public schools were invited to form the focus group, but six attended the four meetings, with 85.71% of the guests participating. All food handlers in the focus group were female, with a mean age of 41 years and meantime in the function of >10 years and a high school degree.
For the orientation of the focus group, two researchers participated in the roles of moderator and co-moderator [42]. The moderator had experience in school meals, man-agement of food services, and teaching. The co-moderator had experience in training and educational activities in municipal public schools with food handlers, teachers, and schoolchildren. As the objective was to identify whether the instruments' semantics met the food handlers' sociocultural repertoire, the researchers used the dialogic research method [42]. Thus, each item of the instruments was read and discussed with the food handlers and among them, investigating verbal expressions commonly used by the study's target audience.
In the first meeting, with the central axis, PNAE: role and valuation of food handlers, the researchers presented the study and directed the form "to whom it is intended" [45]. Subsequently, the food handlers reported their names, length of service, and why they worked in this SFS position. There was discussion about the importance of the PNAE for food and nutritional security of food handlers' activities.
In meetings 2 (Foodborne illness: causes and symptoms) and 3 (Occupational risks in SFS), the researchers identified that technical terms commonly used among nutritionists and other professionals in the food area were poorly understood by the food handlers. Among the words were "food poisoning", "safe food", names of microorganisms ("Salmonella sp.", "Shigella sp.", Clostridium botulinum), "occupational risk", and "food handler." The food handlers questioned the moderators about the concepts and searched the sociocultural repertoire for other words that would allow for the clarity and cohesion of each item [45]. Thus, the words mentioned above were replaced by others: "Food poisoning" by "gastro-intestinal infection"; "Safe food" by "Food without contamination"; "Clostridium botulinum" for "Botulism"; "Food Handler" for "You", and "noise" for "fuss".
The inclusion of the words "I think" in the sentence attributed to food handlers the importance of their opinion while carrying out activities for food safety in the SFS. The word noise was replaced in the items by the word fuss, as it was associated with gnawing, wear of a package by rodents, while the word noise was associated with excessive noise and disturbance to hear. At the fourth meeting, the food handlers read and discussed the instruments with the researchers. There was an agreement regarding the number and content of the instruments' items by all participants.
This study has some limitations: Even though the items used in the survey were discussed and approved among experts, the survey itself lacks a reliability test (Cronbach's alpha that measures the internal consistency of the items used for each block). In future research, when the survey will be administered to respondents, based on the Cronbach's alpha coefficient, the survey might require modifications.

Discussion
Food service is represented by the food and nutrition activities carried out in services [46] including canteens, buffets, catering, bakeries, industrial kitchens, institutional kitchens, delicatessens, snack bars, bakeries, patisseries, restaurants, rotisseries, and similar. In this context, the SFS is included in this group and must offer meals with nutritional quality and safety characteristics following sanitary legislation [30] and the recommendations of the PNAE [47].
Most studies in the area of school feeding assess the sanitary conditions of the SFS and the adoption of good hygiene practices by food handlers [22,48]. The approach to occupational health in SFS still lacks studies that systematically assess, through validated instruments, exposure to occupational hazards. A study identified that 35% of food handlers in SFS in the city of São Paulo had some discomfort related to the presence of pain during the service, with legs being the most affected limbs [49]. The authors justified that the daily workday takes place in a standing posture and insufficient space for food production. It is important to identify that the school SFS produces a large number of meals and for the nutritional, sensory, and hygienic quality and safety to be considered, it is necessary to properly dimension the physical space, equipment, and food handlers.
Instrument validation must occur in two stages: instrument development based on a literature review and item analysis by experts with experience in the field [39,50]. In this study, the two stages were performed, with the inclusion and maintenance of items that had clarity and relevance to the concept of occupational health.
In the first instrument's Identification of Socioeconomic and Demographic Conditions of Food Handlers and School Functioning Characteristics', after the experts' analysis, the sociodemographic and labor items remained: gender, education, and time in the position and participation in periodic food hygiene training, respectively. These items will contribute to identifying the profile of food handlers in SFS and may be related to the adoption or not of good hygiene practices and the perception of occupational risks by these workers in SFS. The data gathering on the instrument "Functioning Characteristics of Schools" will identify the geographic location and management modality of SFS. Social segregation has repercussions on the environment, such as inequality itself, and the poorest strata of the population, with fewer resources, have fewer social achievements: among these achievements, access to schools of "lower quality" [51]. Consequently, these data may identify statistical significance between the geographic location related to spatial conditions and the adoption of good hygiene practices.
The instrument "Assessment of the Level of Knowledge and Attitudes in Food Hygiene" will identify if the food handlers recognize and have attitudes toward safe food production. The PNAE recommends that the foods offered meet the sanitary criteria for promoting health and preventing illnesses in schoolchildren [13]. A study identified a positive association between hand hygiene knowledge and food handlers' self-reported practices in SFS in Hulu Langa district, Malaysia [52]. According to the researchers, these results are related to ongoing education and supervision of food handlers by those responsible for implementing sanitary standards.
The study used the Likert-type scale, with five points to assess the level of knowledge and attitudes, which differs from other authors, who opted for the three-criteria scale [31,53,54]. Regarding the classification of the level of knowledge and attitudes, the studies by Ansari-Lari et al. [53] and Ferreira et al. [31] adopted a cutoff point equal to or greater than 70% for correct items so that the food handler could obtain a satisfactory classification. The authors rated smaller percentages as unsatisfactory. Studies that assess these aspects in food handlers in SFS are scarce, so we chose to expand the strata for the classification adopted by Araújo et al. [32] to know the aspects that could be improved in the study population.
The third instrument, "Perception of Occupational Risks of Food Handlers", was composed of items that will corroborate the assessment of the adoption of the recommendations of regulatory standards [33][34][35][36] in SFS through the perception of food handlers. The exhaustive work in the food service exposes workers to the requirement of high productivity, in a limited time, in inadequate working conditions related to the absence of equipment and inadequate dimensioning of areas [55,56]. A study showed that several inadequacies in working conditions are aggravated in the production of meals, starting with menu planning [57]. Often, inappropriate postures and the execution of monotonous and repetitive activities occurred due to inadequate planning of the menus. The constant lack or insufficiency of food and the malfunction of the equipment cause problems in the execution of the planned menu.
A study in Shiraz, Iran showed a high prevalence of injuries related to working with restaurant workers (84%). The most common injuries were cuts and lacerations but also burns and falls. In this study, work experience was associated with the prevalence of occupational accidents [58]. In a study in a University restaurant in Rio de Janeiro (Brazil), with an average daily production of 2400 meals, workers reported a series of health problems that arise as a result of the demands of work [59]. Among them are high blood pressure, hearing loss, stress, nervousness, respiratory allergy, pain in the spine, legs, and hand joints; anxiety, sleep disorders; headaches, and fatigue. Chamma and Forastieri conducted a study evaluating cases of occupational accidents in restaurants in the city of São Paulo (Brazil) between 1988 and 1998 [60]. The authors found that 57.5% of accidents occurred inside kitchens, with burns being more prevalent (45.8%) followed by musculoskeletal injuries (40.6%). The main causes were heat exposure, slippery floors, improper installations, and the environment (noise, intense heat, poor lighting, and high humidity) [60].
Regarding Work-Related Musculoskeletal Disorders, a study in Ethiopia evaluated 595 restaurant workers in Gondar city [61]. The authors evaluated musculoskeletal symptoms within the past 12 months; 90.42% (538) workers had job stress. Of the 595 workers interviewed, 81.5% reported pain or ache in any part of the neck, shoulder, upper back, lower back, wrist, elbow, hip/thigh, knee, and ankle [61]. The Ethiopian study is in agreement with other studies that also showed a high percentage of Work-Related Musculoskeletal Disorders: 84% in Taiwan [62], 69.2% in Spain [63], 59% in Turkey [64], and 78% in Bangladesh [65].
All these work conditions result in lower job satisfaction, fatigue, decreased productivity, health problems, and accidents. Depression or anxiety accounts for 51% of all work-related illnesses, according to the Health and Safety Executive report on Work-related stress, anxiety, or depression statistics in Great Britain [66].
The instrument "Occupational Risk Mapping in SFS" addresses aspects related to occupational hygiene. In food and nutrition services, such as the SFS, there are five occupational hazards: physical (lighting, temperature, noise); biological (presence of pathogens); chemical (contact with cleaning products, among others), ergonomic (standing posture, repeated movements, stress, time pressure) and accidents (falls, burns, cuts) [67]. Therefore, the Regulatory Standards propose strategies to identify and minimize and/or eliminate occupational hazards in work environments [33][34][35][36]. In this context, identifying the presence of risks in SFS may sensitize the PNAE's Executing Entities to adopt preventive measures to promote occupational health in the work environment.
The literature does not indicate a consensus regarding the adequate number of experts for content validation in the instruments validating. However, it suggests between six and twenty experts [39,50]. The procedure adopted in the study allowed the opinion of experts from different areas on the relevance and cohesion of the items that corroborate to identify aspects related to occupational health in SFS.
The focus group's semantic analysis of the instruments "Assessment of the Level of Knowledge and Attitudes in Food Hygiene and Perception of Occupational Risks "allowed adjustments in the wording of items to the sociocultural repertoire of food handlers at SFS [45]. It is noteworthy that the presence of females in activities related to food production is common [49]. Since these activities are similar to domestic work and in the school environment, the presence of women in food production is associated with maternal care with schoolchildren.
In the semantic analysis, words were substituted to allow food handlers' correct interpretation of items [45]. Among the replaced words was "handling", which was interpreted as excessive touching of any object, and "safety" was related to public safety and not the concept of safe food. All items judged by the expert remained in the instruments submitted to focus group analysis. However, the semantic change contributed to the interlocution of the instruments with the sociocultural repertoire of food handlers in the SFS [45].

Conclusions
The instruments were built and validated in terms of content and semantics to identify aspects related to occupational health in SFS and will be used to identify occupational risks and assess the level of knowledge and attitudes in food hygiene and the perception of occupational risks of food handlers. These instruments were semantically validated through focus groups and are more adequate to this type of workers' reality, since the level of education affects comprehension.
It will be possible to better comprehend the occupational risk that food handlers face in food services, since studies in this area are scarce. Food handlers are exposed to many occupational risks that can, in addition to affecting the quality of the work provided, reduce the employee's useful time in the food sector.