Development of Lead Indicators to Reduce Injury in Australian Agriculture: Protocol for a Modified Delphi Study

Abstract

Agriculture remains one of the most hazardous industries in Australia, with consistently high rates of workplace injury and fatality. Traditional approaches to monitoring safety performance in agriculture have largely relied on reactive lag indicators, metrics that capture incidents after they occur, such as injury rates, compensation claims, or fatalities. While useful for identifying trends and measuring past performance, there is a need for proactive lead indicators, and these are preventive measures that identify risk and promote safer practices before injuries occur. With limited research available on leading indicators specific to the agricultural industry, the modified Delphi method has been identified and designed for this study. A panel of experts will be engaged with the aim to achieve consensus on a set of valid lead indicators for Australian agriculture. This study will include four rounds of Delphi questionnaires with the addition of a face-to-face consensus conference to be included in round four. The data gathered in this study to identify Work Health and Safety (WHS) leading indicators has potential to improve farmers’ safety, wellbeing, and personal health, as well as reducing injuries and fatalities across the Australian agricultural industry.

1. Introduction

Agriculture remains one of the most hazardous industries in Australia, with consistently high rates of workplace injury and fatality [1]. Despite sustained efforts through regulation, education, and engineering solutions, the sector continues to report injury rates far above the national average [2]. These injuries not only impact the health and wellbeing of farm workers and their families but also contribute to substantial economic and social burdens within rural communities [3,4].

Traditional approaches to monitoring safety performance in agriculture have largely relied on reactive lag indicators, metrics that capture incidents after they occur, such as injury rates, compensation claims, or fatalities [5]. While useful for identifying trends and measuring past performance, there is a need for proactive lead indicators, and these are preventive measures that identify risk and promote safer practices before injuries occur. Reflective of this is the stagnation in the reduction of agricultural fatalities across Australia, with data indicating that work-related deaths per million hours worked (mean 0.12), illustrated no significant changes over a 20-year period (2001–2020) [3]. As such, new approaches that address lead indicators (preventive factors) have the potential to impact on these rates in a positive manner.

Current research focuses on suggested interventions and minimum safety requirements for issue-specific topics like vehicles, however, further research is needed to measure the adoption and effectiveness of these interventions on farm [6]. In addition, there is a gap in research to further identify preventive measures to support the broader topic of farm safety and the effectiveness of lead indicators to reduce injury in the agricultural sector [7]. Lead indicators are gaining recognition as critical tools for supporting practical decision-making in modern Work, Health, and Safety (WHS) management [8,9]. While the use of lead indicators is increasing in construction [10], utility industries [11], and occupational health and safety in general [12], agriculture is yet to broadly adopt the use of lead indicators [7]. However, there is currently a lack of consensus on which lead indicators are the most relevant and effective within the unique context of Australian agriculture. Indeed, a recent international systematic review only identified four studies that had examined lead indicators within agriculture [7]. Furthermore, the quality of the studies in relation to their validity and reliability also required attention.

This study attempts to address these issues through the development of evidence-informed, context-specific lead indicators that offer an opportunity to enhance safety outcomes by supporting more proactive and tailored risk management strategies. Given the diversity of agricultural practices and the widespread nature of family-run and small-scale farms, stakeholder input is essential to ensure that the selected indicators are both practical and meaningful.

With limited research currently available on lead indicators specific to Australian agriculture, there is a need for more work in this area [7]. Furthermore, it is important that the process to define these indicators is clear and transparent for all stakeholders. This study protocol endeavours to provide a concise yet comprehensive summary of the methods to be used. This will enable the replication of studies within the industry and support further advancements.

This paper outlines a modified Delphi method aimed at identifying and reaching expert consensus on lead indicators that can be used to reduce injury risk in Australian agriculture. The Delphi method is particularly well-suited for this purpose, enabling structured communication among a diverse panel of experts through iterative rounds of surveys.

The outcome of this study will contribute to the development of a more preventive WHS framework in agriculture, supporting industry stakeholders, policymakers, and researchers in driving targeted and effective safety interventions.

2. Methods and Design

There are two phases to the development of the lead indicators: (a) a systematic review of lead indicators in agriculture, which was published elsewhere [7] and (b) a modified Delphi study to establish valid lead indicators for Australian agriculture.

2.1. Systematic Review

In summary, the systematic review assessed quantitative research studies on measurable leading safety indicators for the agricultural population. Five databases were searched: Medline, Scopus, Web of Science, Informit, and Google Scholar. The search resulted in 5472 studies identified, with only four studies meeting the criteria to be included in the review [7].

The analysis identified a total of 71 leading indicators. However, only two of the studies completed a validity assessment (face and content), with none of the studies reporting reliability assessments [7]. The systematic review recommendations focused on using the core components within the International Standards Organization Occupational Health and Safety Management Systems framework (ISO 45001) [13] as a platform to develop lead indicators. This study has adopted this recommendation, mapping each of the lead indicators to be used as a baseline in the first round of the Delphi study (Appendix A) to the corresponding elements of the ISO 45001 standard. This will enhance their alignment and applicability to health and safety practices.

2.2. The Modified Delphi Method

The Delphi method will be utilised to validate questions asked in this study to ensure they are relevant and aligned with the study objectives. The Delphi method has been used since the 1950s when it was developed by the RAND Corporation to capture the opinions of experts to gain consensus on subjects where there is limited information [14,15,16].

Typically, the Delphi method utilises a three-round process [17]. Questionnaires are sent out following a structured approach to gather feedback from a panel of experts. The responses to each round are analysed, and an anonymous summary report of the group’s opinions are provided back to the participants [18]. The participants can then evaluate the group responses against their own and decide if they want to change their opinion or maintain their original response [18]. It is expected that the group feedback will evolve over the course of questionnaire rounds and result in consensus. Modified Delphi methods are widely published and while there is no standard criteria for modifications to Delphi methods, they are used in an effort to actively achieve consensus [19]. Consensus conferencing is one of the modifications that has been utilised in published modified Delphi studies [20,21,22,23]. The use of face-to-face discussion has been utilised in multiple applications, including after each round of the study [20], or as a standalone round to reach consensus [21]. In recent studies, modified Delphi methods that include anonymous rounds and face-to-face discussions have been used effectively [22].

This study will include four rounds of Delphi questionnaires. The first three rounds will be questionnaires only, while the fourth round will include two components: (a) a face-to-face consensus conference with industry/farmer representatives and (b) a final questionnaire to be completed by all study participants.

For all rounds of this study, a 90% completion rate and an 80% acceptance rate will be required to achieve the predefined thresholds for consensus.

The ACCORD (ACcurate COnsensus Reporting Document) reporting guidelines will be utilised in conjunction with the Delphi method. The ACCORD guidelines have been developed to ensure consistency and transparency of the methods used in consensus-based studies [24,25].

2.3. Panel Selection

A research team (n = 6) with expertise in WHS and agriculture will be responsible for leading and overseeing this study. This team will facilitate the Delphi process by designing survey rounds, synthesising participant feedback, plus ensuring a systematic approach to collecting and analysing responses.

This study will engage three groups of experts.

• An existing Technical Working Group (TWG) that provides support to a national farm injury prevention project. Members (n = 6) have been selected for their WHS knowledge, plus experience in the agriculture sector. The TWG members will be invited to all four rounds of questionnaires. If a participant is unable to complete a round, they will remain eligible to participate in subsequent rounds. As the TWG members have high level expertise in WHS, these inputs have been scheduled for the first two rounds, so that the technical nuances that are embedded within the ISO framework can gain full consideration.
• A Stakeholder Reference Group (SRG) consisting of members from the Rural Safety and Health Alliance. These individuals have a cross-section of experience and are drawn from several different agricultural commodity sectors (n = 10). They will be invited for the third-round questionnaire in addition to the TWG members. These individuals will bring additional context from different commodity sectors in assessing the indicators.
• The fourth round will introduce industry/farming representatives (n = 10) and require these representatives to participate in a face-to-face (virtual) meeting. The industry/farming representatives will be identified by purposive sampling through the SRG, drawing on their links within the commodity sectors they represent. This group is fundamental to obtaining a practical and real-world perspective of the proposed indicators. In negotiation with the funding agency, farmers from four commodity sectors will be involved in this initial work—cotton, dairy, grains, and pork. This group will also be invited to complete the fourth-round questionnaire, in addition to the TWG and SRG.

Selecting participants with relevant expertise and experience will be critical to ensure the study results have practical and meaningful outcomes for Australian agriculture. There is no documented consensus in the specific number of participants required for a Delphi method study; however, it is common for panels of 20 to 30 to be used [16,24]. This study will engage 26 participants.

2.4. Study Design

The aim of this study is to identify a set of relevant and practical lead indicators for the Australian agriculture industry.

2.5. Rounds

A modified Delphi method will be utilised for this study to seek consensus from a panel of experts through iterative rounds of online questionnaires and a virtual consensus conference. The addition of the fourth round to include a face-to-face (virtual) meeting will provide industry/farmer participants an opportunity to discuss their opinions and provide practical feedback on the relevance of the lead indicators [23]. An overview is provided in Figure 1.

The initial set of questions (Appendix A) was developed in collaboration with the research team from The University of Sydney, who are overseeing the study (n = 6). The lead indicators have been directly aligned with the seven core components of the ISO 45001 and draw on data from the National Coronial Information System (NCIS) that has identified the agents of fatal injuries on Australian farms in the past 5 years (2020–2024). Key factors that may have prevented or ameliorated the impacts of these incidents (e.g., Rollover Protection Structures (ROPS), machinery guarding, seatbelt use, helmets, etc.), can be ascertained from these data to ensure that the questionnaire items target the highest fatality risk issues in Australian agriculture.

Participants will be provided with an email inviting them to complete the survey. The questionnaire will be conducted online utilising a Qualtrics survey. Each round will have a two-week completion time and a reminder email for each of the participants will be sent one week prior to the due date of each round by the facilitator. All responses collected through the Qualtrics survey will be non-identifiable. In each round, items that achieve full consensus will be accepted and identified for subsequent rounds. Any comments provided on wording will be reviewed and analysed for potential inclusion or refinement in the following round.

Participants will receive feedback after each round they take part in. For rounds one to three, this feedback will be provided at the start of the following round and will include a summary of panel responses to the first set of lead indicators, along with any comments or suggestions (e.g., re-wording questions or adding additional lead indicators). Feedback for the final (fourth) round will be provided once all data has been analysed, presenting the overall panel consensus and final set of agreed lead indicators.

2.5.1. Round One

The Delphi questionnaire will be sent to the TWG (n = 6). It will consist of 36 questions, including dichotomous questions and open text response fields. Participants will be able to express their agreement or disagreement, suggest alternative wording, and provide additional suggestions.

The responses received in round one will be analysed for consensus, trends, and themes, which will inform the development of a condensed questionnaire for round two.

2.5.2. Round Two

The same process will be followed in round two, engaging the six members of the TWG. The questionnaire for round two will be developed based on the themes and areas of consensus identified in round one. It will again include dichotomous questions and free text responses.

Responses from the second round will be analysed for consensus trends and themes to further refine and condense the questionnaire for round three.

2.5.3. Round Three

The Delphi questionnaire for round three will include revised and condensed questions informed by feedback and data analysis from round two. In this round, the SRG (n = 10) will be introduced to participate alongside the TWG. Responses will be collected using dichotomous questions.

2.5.4. Round Four

Round four will introduce selected industry/farming representatives (n = 10) in a virtual ‘face-to-face’ discussion using a structured nominal group technique. Following this discussion, a fourth-round questionnaire with dichotomous questions will be sent to the TWG (n = 6), SRG (n = 10), and the industry/farming representatives (n = 10).

2.6. Considerations

The advantage of the modified Delphi method for this study is the ability to conduct each round remotely and allow the engagement of participants regardless of location [26]. The first three rounds of the Delphi will allow the participants to submit their feedback independently and anonymously to remove influence from dominant individuals within the group. The fourth round of the study will engage the participants of the industry/farming representative group to participate in a virtual meeting space (Microsoft Teams) to discuss, express and debate their thoughts on the proposed questions in a controlled forum. This discussion is expected to enhance understanding among industry/farming representatives to facilitate consensus on the outstanding items.

The disadvantage of the modified Delphi method is that it involves multiple rounds and requires commitment from participants to commit and respond to multiple rounds. The lack of engagement with the rounds poses a threat to the study and the ability to achieve the required completion rate [27].

3. Data Analysis

The analysis of the data will follow an iterative and systematic process. Responses for each round will be collated and analysed to assess consensus amongst participants. Text responses will be examined for key themes and patterns, which will be summarised and coded. A coding framework will be developed and applied to ensure the data is interpreted systematically. The findings from each round will inform the refinement of questions for the subsequent rounds. Upon the completion of the rounds, the data will be synthesised to provide a comprehensive overview of the findings, including key themes of consensus and any unresolved lead indicators.

4. Conclusions

With limited research available on leading indicators specific to the agricultural industry, the modified Delphi method was identified as the most appropriate method for this study. The aim is to engage a panel of experts to achieve consensus on a set of valid lead indicators for Australian agriculture.

The data gathered in this study to identify WHS leading indicators has significant potential to improving farmers’ safety and reducing injuries and fatalities across the Australian agricultural industry. It is envisaged that the resulting indicators will be utilised and evaluated with a cohort of farmers in a trial program to assess the reliability of the indicators and to quantify the impact that they may have on safety systems within farming businesses.