In Search of Concrete Outcomes—A Systematic Review on the Effectiveness of Educational Interventions on Reducing Acute Occupational Injuries

Education is a common strategy used to prevent occupational injuries. However, its effectiveness is often measured using surrogate measures instead of true injury outcomes. To evaluate the effectiveness of workplace educational interventions, we selectively analyzed studies that reported injury outcomes (PROSPERO ID: CRD42019140631). We searched databases for peer-reviewed journal articles and sources of grey literature such as abstracts, registered trials, and theses published between 2000 and 2019. Studies on educational interventions that reported fatal or non-fatal occupational injury outcomes were selected. Two reviewers independently and in duplicate screened the studies, extracted data, and assessed risk of bias. Heterogeneity in the data precluded meta-analysis, and the results were reviewed narratively. In total, 35 studies were included. Of which, 17 found a significant reduction in injuries, most of which featured a multifaceted approach or non-didactic education. The remaining studies either described equivocal results or did not report statistical significance. Overall, interventions in the manufacturing industry were more effective than those in the construction sector. Risk of bias among included studies was moderate to high. In conclusion, educational interventions could be an effective part of multifaceted injury prevention programs. However, over-reliance on didactic education alone is not advised.


Introduction
Occupational injuries are a major threat to the health and well-being of workers. Each year, an estimated 380,000 workers lose their lives due to workplace injuries globally, and over 370,000,000 suffer non-fatal injuries [1]. In 2015, the International Labour Organization estimated that a total of USD $2.8 trillion is lost each year due to the direct and indirect effects of occupational hazards globally, accounting for 4% of the world's GDP [2,3].
Many prevention strategies have been implemented in recent decades in an effort to stem occupational injury levels. These initiatives can be broadly categorized into the "Three E's of Injury

Materials and Methods
The review protocol is registered in PROSPERO (ID: CRD42019140631). The review was conducted in accordance with the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guideline [12].

Types of Studies
Randomized controlled trials (RCTs), non-randomized controlled studies (NRSs), and uncontrolled before-and-after trials (uCBAs) were eligible for inclusion in this review. We expected a limited number of RCTs due to various logistical and ethical barriers inherent within the context of workplace health and safety education initiatives. For instance, educational interventions using individual-level randomization in the workplace may be prone to contamination between groups, since workers can freely communicate amongst each other. Additionally, workplace-based cluster randomization faces challenges associated with recruiting a sufficiently large sample to be robust against confounding variables. Therefore, we decided to include a number of non-randomized study designs, such as controlled before-and-after studies (CBA), interrupted time studies (ITS), historically controlled studies, prospective cohort studies, and uncontrolled before-and-after studies (uCBAs).
We defined a CBA study as a controlled trial in which the experimental and control groups were not assigned in a random fashion. A historically controlled study is similar to a CBA study, with the difference that the control was a group of participants in the past. In the absence of a control group, if a study measured three time points before and after the intervention to analyze for temporal trends, then it qualified as an ITS study. A prospective cohort study was defined as a study in which the investigators enrolled participants after exposure to a factor associated with the outcome, but before any outcome(s) of interest had developed. Finally, an uCBA study implements an intervention without including a control group [13].
Our decision to include uncontrolled before-and-after studies was due to three reasons. First, in the occupational context, controlled trials may be difficult to perform. Second, since the literature connecting educational interventions to injury outcomes is already limited to begin with, over-restriction of our search criteria may have limited our ability to draw any conclusions at all. Third, we anticipated that a sizeable proportion of the literature on this topic may be the result of quality improvement projects which were later published. By including these uncontrolled trials, we could gain additional insight into educational interventions used in a practical context.

Types of Participants
This review included studies of paid adult (≥18 years of age) employees from all industries. Temporary workers or workers employed outside of a legal contract were also included.

Types of Interventions
Educational interventions implemented with the intention of preventing occupational injuries were included. Multifaceted interventions with a prominent educational component were included as well. We included didactic education in addition to non-traditional methods, such as interactive discussions, peer feedback, hazard recognition, and social marketing/awareness campaigns.

Types of Outcome Measures
We included studies that measured the change in fatal or nonfatal injury rate following educational intervention(s). As an operational definition, we used the following modified definition from The Injury Chartbook by the World Health Organization (WHO) [14,15], "occupational injury is a body lesion at the organic level, resulting from acute exposure to energy in the work environment (mechanical, thermal, electrical, chemical or radiant) in amounts that exceed the threshold of physiological tolerance. In some cases (e.g., drowning, strangulation, freezing), the injury results from an insufficiency of a vital element." Since we anticipated that operational definitions for injury would vary by study, we considered all reasonably congruent definitions. All collection methods of quantifiable injury data were eligible for inclusion. These included, but were not limited to, self-report, company databases, and workers' compensation records.

Exclusion Criteria
We excluded studies published before 2000 to ensure the recency of our results. The year 2000 was selected due to the rapid automation of industries which caused substantial changes in the proportion of white-collar, operative/laborer/service, and agricultural workers that did not decelerate until 2000 [16,17]. This coincided with a shifting paradigm for occupational safety education, which better reflected the needs of the changing workplace [18,19]. We also excluded non-English studies due to limited translation resources. Studies that measured injury rates outside of the occupational context were excluded. For our study, "occupational context" is defined as both the workplace and commute to and from work. Further, we excluded occupational diseases, infections, intentional injuries, and musculoskeletal disorders due to chronic exposure.

Information Sources
We searched the following electronic databases up to May 2019: Embase, Ovid MEDLINE, NIOSHTIC, Web of Science Core Collection, and the Cochrane Central Register of Controlled Trials (CENTRAL). In order to avoid publication bias favoring positive results, grey literature such as theses, abstracts, and registered clinical trials were also included in our study [20]. We searched Google Scholar, Google, WorldCat, ProQuest, NDLTD (Networked Digital Library of Theses and Dissertations), and ClinicalTrials.gov for grey literature. We also manually searched the reference lists of all included studies to locate additional papers. All searches were conducted with the guidance of a biomedical librarian. The general PICO (population, intervention, context, and outcome) search structure for the databases is shown in Table 1; the detailed search strategy is listed in Appendix A.

Selection of Studies
The titles and abstracts of potentially relevant studies were screened by two review authors independently and in duplicate. Disagreements about inclusion occurred in less than 5% of all records screened, and were resolved by discussion until consensus was reached. In the case of a disagreement that could not be resolved by discussion, a third author made the final decision. Full text articles of the shortlisted studies were then reviewed independently and in duplicate by two authors against the inclusion and exclusion criteria. The same process for resolving disagreements was applied, and reasons for exclusion were documented. Reference lists of included papers and relevant systematic reviews were screened to identify additional studies. The titles of relevant grey literature were searched on Google Scholar to potentially locate their respective full papers.

Data Extraction and Management
Two review authors independently extracted data from all included articles. We designed a data extraction form in order to standardize the process between the two reviewers. The following data were collected: study design, setting (nationality and industry, as defined by the North American Industry Classification System [21]), participants (sample size, occupation, and distribution between experimental and control groups), description of the intervention(s), timeline of intervention, outcome (rate ratio, difference in means, etc.), sources of funding, and conflicts of interest.

Assessment of Risk of Bias in Included Studies
Two review authors assessed risk of bias in all included studies independently and in duplicate. The quality of RCTs was assessed using the revised Cochrane risk-of-bias tool for randomized trials (RoB 2) [22]. Quality of non-randomized controlled studies was assessed using the Cochrane risk of bias in non-randomized studies of interventions (ROBINS-I) [23]. Disagreements arose in about 10% of assessments but were all resolved by discussion without resorting to a third reviewer. The risk of bias for conference abstracts could not be assessed due to limited information, and thereby they were categorized as "uncertain risk of bias." Uncontrolled before-and-after trials were all categorized as "high risk of bias" since they are especially prone to confounding factors and regression to the mean [24].

Data Synthesis
Meta-analysis was not attempted due to heterogeneity in the target populations and study designs, as well as an insufficient number of studies from most industries. Since each industry has its unique profile of injury mechanisms and risk factors, we expected the nature of educational interventions to vary accordingly [25]. Therefore, qualitative assessment was performed at an industry-level, by considering the number and quality of studies, effect significance, and type of education involved. Included studies with a "critical" risk of bias were not factored into any narrative synthesis, as per Cochrane guidelines [23].

Results of the Search
The search yielded 4492 records, of which 3973 were from electronic databases and 519 were identified through sources of grey literature. (Later manual checking of reference lists from relevant systematic reviews and included studies yielded an additional 5 records.) After removing the duplicates, 3613 remained. After screening their titles and abstracts, 48 studies were shortlisted for full-text review. Of these 48 papers, 38 met inclusion criteria (Figure 1), and 10 were excluded. The reasons for exclusions are listed in Appendix B. The 38 included papers represented 35 unique studies. In cases where multiple papers described the same study, only the most comprehensive publication was retained for further analysis. The redundant papers are also listed in Appendix B.
All included studies were published between 2000-2018. Unless otherwise specified, all studies were published as a full paper. Characteristics of all included studies are summarized in Table 2.
All included studies were published between 2000-2018. Unless otherwise specified, all studies were published as a full paper. Characteristics of all included studies are summarized in Table 2.   Significant effect in preventing injuries-the incident rate ratio of exposed to unexposed was 0.51 (95% CI: 0.00-0.98).

Risk of Bias in Included Studies
Of the eight RCT studies, five were judged to be at moderate risk of bias using the RoB 2 tool, and three were at high risk of bias (Figure 2

Risk of Bias in Included Studies
Of the eight RCT studies, five were judged to be at moderate risk of bias using the RoB 2 tool, and three were at high risk of bias (Figure 2). Using the ROBINS-I tool, of the five CBA studies that were published as a full paper, four were at serious risk of bias and one was at critical risk of bias. Of the three ITS studies that were published as full papers, two were at moderate risk of bias, and one was at critical risk of bias. The historically controlled study and both of the prospective cohort studies were at serious risk of bias. The two studies with critical risk of bias will not be further discussed in the narrative synthesis [36,41] (Figure 3). Using the ROBINS-I tool, of the five CBA studies that were published as a full paper, four were at serious risk of bias and one was at critical risk of bias. Of the three ITS studies that were published as full papers, two were at moderate risk of bias, and one was at critical risk of bias. The historically controlled study and both of the prospective cohort studies were at serious risk of bias. The two studies with critical risk of bias will not be further discussed in the narrative synthesis [36,41] (Figure 3).
As discussed previously, all 14 uCBAs were automatically assigned a high risk of bias [24]. Risk of bias assessment was not attempted for the remaining abstract (ITS) and the grant report (CBA) due to insufficient information.
Using the ROBINS-I tool, of the five CBA studies that were published as a full paper, four were at serious risk of bias and one was at critical risk of bias. Of the three ITS studies that were published as full papers, two were at moderate risk of bias, and one was at critical risk of bias. The historically controlled study and both of the prospective cohort studies were at serious risk of bias. The two studies with critical risk of bias will not be further discussed in the narrative synthesis [36,41] (Figure 3).

Discussion
Despite widespread use of education in occupational injury prevention programs, limited reviews exist on assessing injury outcomes in recent years. Hence, we aimed to narratively summarize the effects of educational interventions on occupational injury outcomes through this study. We systematically searched multiple databases and various sources of grey literature. A strength of our study is that we included grey literature, including abstracts, theses, and grant reports to avoid publication bias. However, a necessary trade-off was that their interventions and results were not always described in detail. Further, by considering a broad range of injuries across all industries, the results of this study could be more generalizable. To the best of our understanding, this is the first systematic review of its kind that is not limited to a particular industry or type of injury.
Our results reveal modest evidence that educational interventions have a protective effect against occupational injuries. However, the overall risk of bias was moderate to high among all included studies. In addition, the majority of all studies were performed in two industries, construction and manufacturing. Multifactorial and non-didactic educational interventions were generally more effective than didactic education. This is especially evident in the manufacturing sector, which employed the former strategies more frequently.

Agriculture, Forestry, Fishing, and Hunting
The agricultural sector experiences the highest rates of non-fatal injuries among all US industries [61]. However, neither of the RCTs included were able to demonstrate a significant reduction in injuries, despite utilizing multiple forms of education, such as didactic and interactive teaching, plus behavioral-based incentives [26,29]. Both studies were conducted in developed countries with a relatively high degree of legislative and engineering support. While this gives us some insight into the effect of education in developed countries, it limits our ability to extrapolate the findings to developing countries, where most of the world's farming population resides [62]. Our findings corroborate a meta-analysis by Rautiainen et al., who concluded that there was no evidence suggesting benefit from the use of educational interventions alone in the agricultural context, and that more high-quality studies, such as RCTs or ITSs, should be conducted for behavioral interventions [11].

Arts, Entertainment, and Recreation
Only one RCT was identified, targeting sunburn reduction among swimming pool staff [33]. This limits our ability to generalize the results to the rest of this sector. Nonetheless, this study is notable for using a placebo intervention in the control group, in which participants learned about child injury prevention. This strategy could help reduce the risk of bias associated with the impracticality of blinding participants in educational interventions.

Construction
Although a large number of studies were identified in construction, not all of them were methodologically sound. Our search revealed one CBA [39], one ITS which was removed from narrative synthesis due to critical risk of bias [41], one prospective cohort [46], and five uCBAs [50][51][52][53]59]. The overall results were not encouraging. Only one uCBA study found a statistically significant effect following an educational intervention [52]. Thus, there may be challenges in influencing the construction sector through purely educational means. This echoes a previous meta-analysis by van der Molen et al., which showed no strong evidence to suggest that safety campaigns alone have a protective effect on construction workers [10]. There may be a potential synergistic effect between educational and regulatory interventions, which may serve as a potential direction for future research [63,64]. It should also be noted that construction studies besides Spangenberg et al. [52], Evanoff et al. [50], and Kidd et al. [39] featured interventions that were almost exclusively didactic in nature. Interestingly, Spangenberg et al. was also the only study to find a significant effect, and Evanoff et al. found a significant effect before adjusting for covariates. This may indicate that non-didactic education is more effective, hence suggesting a direction for future research.

Educational Services
One RCT was identified [27], which did not find a significant benefit from educational intervention. Since the study was conducted in only one region in Belgium, risk of cross-contamination may have skewed the results. Moreover, since its results were analyzed on a per-protocol basis, it limits our ability to extrapolate results to the real world, where non-adherence may be prevalent.

Health Care and Social Assistance
Only one uCBA abstract was identified [47], which did not state statistical significance. There was insufficient information for conclusions to be drawn for this sector of the economy.

Manufacturing
Manufacturing had the largest number of included studies: two RCTs [30,31], three CBAs [36][37][38] (one of which was removed from narrative synthesis due to critical risk of bias [36]), three ITS studies [40,42,43], and four uCBAs [48,[54][55][56]. Of which, all of the RCT and CBA studies, one of the ITS studies [42], and two of the uCBA studies [54,55] showed significant injury reductions. Interestingly, one commonality among the majority of effective interventions in manufacturing was that they employed either multifactorial approaches or educational methods that were not purely didactic [30,31,38,42,54,55]. Despite the possibility of cross-contamination favoring the controls, these interventions still resulted in significantly positive results, which may indicate that such efforts are especially effective. Another promising aspect is that these studies represent a range of developed and developing countries, thereby increasing the generalizability of the results.

Mining, Quarrying, and Oil and Gas Extraction
Three studies were included, one each of RCT, CBA, and uCBA. Overall the results are encouraging as the RCT and uCBA demonstrated significant protective effects [32,60], while the CBA [34] showed promising effects in the protective direction (significance not stated). The relatively large effect sizes reported in these studies give us cautious optimism that education is an effective component in future mining interventions.

Public Administration
An RCT and an historically controlled study were included [28,44]. Both of which saw significant benefits in some but not all outcomes measured [ Table 2]. Despite considerable cross-contamination between groups in the RCT, the authors persisted with an intention-to-treat (ITT) design which made the results more robust against non-adherence [28]. However, research in this sector is still lacking.

Transportation and Warehousing
One CBA and one uCBA were included [35,57]. Both studies only focused on a small subset of the overall population (i.e., stevedores and truck drivers with sleep disorders), which severely limits the generalizability of any conclusions drawn. As one of the most common causes of preventable fatal injuries, additional investigations in this field is needed [61].

Utilities
Salminen found an increase in injury numbers after implementing an anticipatory driving intervention for electricians [49]. In addition, although traffic-related incidents were reduced following another group discussion intervention, a paradoxical increase in non-traffic injury rates occurred. These could be rationalized by considering that the study was uncontrolled with a short follow-up period, and that behavioral changes may take a long time to mature. If the trial was more robust, the principles of driving safety could be extrapolated to other industries which involve driving. Another uCBA in the electrical sector was an abstract with relatively little information regarding injury outcomes, so it was not possible to draw conclusions from it [58]. Overall, educational interventions in this industry are understudied and greatly limited in scope.

Observations across Industries
Overall, the methodological quality of the included studies was poor, as shown in Figures 2 and 3. Even among the RCTs, none were at a low risk of bias. This observation can be rationalized by first considering the unique context of the occupational injury prevention scenario. Randomization in this setting may face ethical questions associated with denying potentially life-saving interventions from workers. In addition, even if that can be addressed, the study may run into logistical challenges when trying to recruit enough workplaces for randomization, as the prospect of being placed into the control group may discourage employers. On the other hand, if randomization was performed on an individual basis within individual workplaces, then serious concerns with cross-group contamination would be inevitable. Further, after randomization it would be practically and ethically unfeasible to prevent workplaces from implementing additional interventions, which would confound the results. This is especially pertinent with educational interventions, as changes in beliefs, attitudes, and work culture take time, during which confounding interventions may occur. Moreover, with an educational intervention, true blinding of the participants and instructors is practically challenging. For these reasons, sometimes the best available option is to implement a non-randomized trial or an RCT with considerable limitations. This has been echoed in previous Cochrane reviews [10,11]. As such, we have taken these constraints into account when making our recommendations.
To facilitate future RCTs, we suggest the following strategies. Conducting studies proactively (i.e., before injury rates become alarmingly high) could encourage employers to accept the risk of having their workplace being assigned to a control group. As an added benefit, it is less likely that employers will implement confounding interventions of their own during the study. Alternatively, offering crossover study designs would remove the disincentive of potentially being assigned to a control, while at the same time addressing ethical concerns of withholding beneficial interventions from workers. Finally, control groups may receive standard training or placebo intervention on another topic to enhance the blinding process. Such strategies could bring about additional high-quality studies in the field, which could in turn allow for further conclusions to be drawn.
There were four studies which attempted to indirectly prevent injuries through addressing associated conditions such as obesity, skin cancer, and sleep disorders [28,31,33,57]. Of which, Geller et al., Morgan et al., and Sullivan et al. were RCTs with significant results in at least some of their outcome measures. This suggests that education does not necessarily have to be directly focused on injuries. The fact that these three RCTs were performed in diverse settings supports the generalizability of this notion.
It should be noted that among the interventions which demonstrated a significant protective effect against occupational injuries, most featured either multifactorial strategies alongside education [33,42,44,54,57] or educational approaches that were not purely didactic in nature [30][31][32]38,52,55]. This suggests that creative and multifaceted designs should be utilized when designing future injury prevention programs, especially in the manufacturing industry, where many of the aforementioned studies took place. While it may seem obvious that a multifactorial approach would have greater success than education alone, there is a potential synergistic effect as every link along the chain of safety is strengthened [64]. On the other hand, the effects of implementing didactic education on its own are limited, as evidenced by studies in the construction sector, which may not justify its resource and opportunity costs.
In general, developing countries were underrepresented amongst the included studies. Only six studies were done on workers in developing economies [30,32,35,54,55,58]. This is concerning, as the majority of occupational injuries occur in these settings [1]. Interestingly however, all of those studies showed either a significant benefit [30,32,35,54,55] or a protective effect, without stating statistical significance [58], which suggests that education may be more effective in low-resource settings. This could be rationalized by considering that legislative, administrative, and engineering interventions presumably already exist in developed countries, thus the effects of additional education would be dwarfed in comparison. Conversely, in developing countries, those measures may not be as robust, which leaves the potential for education to impart a more pronounced effect.

Limitations
Due to limited resources, we were unable to include non-English language studies, which may have introduced a language bias. Additionally, since some educational interventions were implemented as part of a multifaceted program, it is sometimes difficult to discriminate what effects, if any, that the educational components truly imparted. However, this would not invalidate these studies as the purpose of our review is to address the pragmatic question of whether or not education is effective within the context of a real-life work environment, where there will inevitably be some degree of concurrent interventions. Due to heterogeneity in study designs and a limited number of RCTs, it was not possible for a meta-analysis to be performed and therefore quantitative conclusions cannot be drawn. Finally, since most studies were conducted in developed nations, a caveat is that our findings are not necessarily generalizable in the setting of developing economies.

Conclusions
Educational interventions are effective when implemented as part of a multifactorial approach or in a non-didactic fashion. This is especially true in the manufacturing sector. Caution should be advised when implementing didactic education on its own to prevent occupational injuries, especially in construction. Nevertheless, it could still be an effective component of a multifactorial approach.
Additional high-quality studies in underrepresented industries and developing countries are needed to better understand the effectiveness of education in their respective settings. In the future, researchers could address barriers to RCTs by implementing interventions proactively, using crossover designs, and providing controls with standard training or placebo intervention.
Funding: This research received no external funding. The intervention has very few educational components, which are also vaguely defined. Moreover, the study only measured equipment downtime and unmet production due to drowsiness, which are not sufficiently reliable outcomes according to our inclusion criteria Same study as Yu et al. 2017 [30]