MIAR^forest Reproducibility and Reliability for Assessing Occupational Risks in the Rainforest

Abstract

The Method for the Integrated Assessment of Risks for rainforest (MIAR^forest) is a specific methodology for assessing the risk of occupational accidents associated with working in native tropical forests. MIAR^forest was validated for the results’ reproducibility and the reliability of calculated risk levels through the Delphi approach. Two rounds of questionnaires illustrating ten scenarios of activities associated with the logging process in the Eastern Amazon’s native rainforest (Brazil) were presented to forestry and occupational health and safety (OHS) experts. In the first round, the questionnaire was answered anonymously by 55 experts, and in the second, by 46. A percentage of agreement of at least 80% in each question was considered to close the process. Questions that did not meet the criterion in the first round were reassessed in the second round. The obtained results lead to the conclusion that MIAR^forest, an occupational accident risk assessment tool, has been effectively validated, demonstrating inter-rater reproducibility and reliability in determining risk values. These results highlight the objectivity and reliability of MIAR^forest.

1. Introduction

Humanity’s growing need for resources puts pressure on and sometimes destroys critical ecosystems worldwide. Rainforests are particularly affected by recurrent logging activities [1], and their preservation is fundamental to the sustainability of life on Earth. In recent decades, several methods of native forest exploitation have been developed and fine-tuned to ensure that they not only continue to be a source of resources for people but also ensure their conservation [2,3].

In this process of sustainable exploitation of forest resources, one of the fundamental factors to consider is the safety of those who are available to work in these adverse conditions [4,5,6,7]. In this context, occupational risk assessment assumes a key role, and consequently, occupational safety and health management take a prominent place in preventive approaches [8,9]. Indeed, in the forestry industry, as in other industrial sectors, occupational risk assessment is the basis of an effective occupational health and safety management system in a company [10,11], and when efficiently implemented and conducted, it also contributes to an overall improvement in company performance [12].

To identify and prevent occupational accidents and occupational diseases, it is essential to use adequate methods, that is, methods adapted to the specificities of the activities inherent to the process. In this sense, companies committed to their employees’ occupational health and safety (OHS) have been acquiring OHS management systems or developing their own based on the experience of their technicians and their reality [12,13,14]. However, this approach has intrinsic weaknesses: its subjectivity and the impossibility of benchmarking the evaluation results with those of other companies to allow the validation of the obtained results. Thus, there is, inevitably, variability in the evaluation results, depending on each evaluator’s understanding.

Most published studies do not reflect concerns about the risk assessment methods’ validation or their reproducibility and reliability [15,16,17,18,19,20,21,22,23,24].

The validated version of the Method for the Integrated Assessment of Risks for rainforest (MIAR^forest) [25] ensures that the risk assessment uses a set of parameters suitable for dealing with the unique challenges posed by a hostile work environment, such as the native rainforest. However, it has not been proven that when applied to real-life situations, assessment results have two fundamental qualities: (1) reproducibility, ensuring consistent assessments by different evaluators for the same situation; and (2) reliability, ensuring that assessment results translate the risk perception of the majority of the evaluators.

The main objective of this study is to assess these two indicators of MIAR^forest quality, namely, the inter-rater reproducibility and reliability in determining the risk value.

2. Materials and Methods

2.1. MIAR^forest—Short Presentation

MIAR^forest was developed based on MIAR [26], a method that adopts a process approach and that has been tested and adjusted for different contexts and industrial sectors, such as metalworking [27], construction [28], industrial waste sorting [29], extractive industry [30,31], and slaughterhouses [32]. The risk assessment results in all these applications revealed high reproducibility and reliability. This means that different evaluators consistently similarly assessed the risks and that the results were found to be accurate, reflecting reality.

The hazard identification process using MIAR^forest begins by pinpointing the sequence of processes, sub-processes, activities, and tasks, delving into the necessary level of detail. This comprehensive approach encompasses identifying the equipment and materials involved in each activity, describing the work environment and the specific characteristics of the forest areas where the activities are conducted. Additionally, MIAR^forest examines the effectiveness of the existing protection measures and the company’s implemented occupational accident risk mitigation strategies, identifying any potential gaps or areas that require improvement. Then, the occupational risk can be calculated. In MIAR^forest, risk is defined as the measure of uncertainty regarding the occurrence of an event within a hazardous exposure situation. The risk level (RL) is determined using a traditional approach that involves multiplying the likelihood (Li) by the severity (S) (Equation (1)). To provide more specificity for each parameter (S and Li), the likelihood (Equation (2)) is further divided into two sub-parameters: extent of impact (Ei) and frequency of exposure (Fe). Similarly, the gravity (Equation (3)) is obtained from ten independent factors of the working environment, including two with global impact (worker protection (WP) and forest typology (FT), two controllable factors (machines and tools handling (MT) and relationship between Tasks (RBT)), and six uncontrollable factors (object fall (OF), terrain slope (TS), obstacles (Obst), wild animals (WA), precipitation intensity (PI), and wind intensity (WI)) [25].

RL = Li × S,

(1)

where

Li = Ei × Fe,

(2)

and

S = WP × FT × max (MT, RBT, OF, TS, Obst, WA, PI, WI) × median (MT, RBT, OF, TS, Obst, WA, PI, WI).

(3)

When calculating severity, the use of the maximum value and median of the parameters MT, RBT, OF, TS, Obst, WA, PI, and WI is due to the need to emphasise the most relevant ones. Using the maximum value highlights the one with the greatest impact, regardless of the context. The median is a weighting factor since it is less sensitive than the mean to extremely high or low values. Additionally, the median is not affected by small variations in sub-parameters values or errors in the evaluator’s interpretation of the situation.

Like MIAR, MIAR^forest obtains the risk level (RL) by combining severity with likelihood. The calculated value is then adjusted according to the effectiveness of the risk control (RC) systems in place in the organisation. This establishes a weighted risk level (WRL) determined according to Equation (4):

$$\mathrm{W}\mathrm{RL}=\frac{\mathrm{RL}}{\mathrm{RC}}.$$

(4)

Each parameter and sub-parameter is assessed using a five-point Likert scale.

2.2. Data Collection

The scenarios to submit to experts were selected based on information collected directly by the authors in four timber forest management areas in the Eastern Amazon granted by the Brazilian Federal Government [3].

The chosen areas were the Tapajós National Forest (Flona Tapajós—FT) created in 1974, the Saracá-Taquera National Forest (Flona Saracá-Taquera—FST) created in 1989, the Altamira National Forest (Flona Altamira—FA) created in 1998, and the Verde para Sempre Extractive Reserve (Resex Verde para Sempre—RVS) created in 2004. The largest is the RVS, with 1,289,362.78 ha in Porto de Moz; followed by the FA, with 689,012 ha in the municipalities of Altamira, Itaituba, and Trairão; the FT, with 530,620.65 ha in the municipalities of Aveiro, Belterra, Placas, and Rurópolis; and, finally, the FST, with 441,152 ha in the municipalities of Terra Santa, Oriximiná and Faro.

The authors remained for several days in each one of the areas to acquire a better understanding of the problems and gather all the relevant information.

In each of these areas, the following global procedure was followed: (1) global identification and description of the entire production process (activities and tasks); (2) characterisation of the workers’ population involved in each activity and task, as well as the tools and the personal protection equipment (PPE) they use while working, (3) environmental parameters (meteorology, site characteristics, fauna and flora); and (4) identification of the dangers and risks inherent to the logging activity.

The collected information was then processed to obtain a global synthesis of the procedures most common to all four concession areas. After this process, a set of ten risks representative of all the five risk levels defined by MIAR^forest was extracted. Each one of the ten scenarios was described following the MIAR^forest evaluation grid and illustrated with a photograph.

Each of the ten scenarios presented to the specialists (

Table 1. Scenarios to consider when evaluating the risk related to each parameter and sub-parameter of MIAR^forest.

Figures	Scenario Description
Scenario 1	Activity—Construction of infrastructure (roads and patios); Task—Vegetation removal: this task consists of tacking down the existing shrubby vegetation where the road or patio will be built using a crawler tractor; Meteorology—Precipitation probability: 70%/Humidity: 84%/Wind intensity: 14 km/h; Vegetation—Lowland dense ombrophilous forest; Wild animals—Contact with snakes and poisonous insects is possible; Terrain characteristics—Smoothly undulating surface, with very difficult obstacles to overcome; Workers involved in the task—One crawler tractor operator; Workload—18 h per week; Relationship of this task with others in this activity—No other tasks occur in the same location; Tool/Machine Used—Crawler tractor; Hazard/danger—The vegetation; Triggering factor—Moving the tractor between the vegetation to be cleared; Risk—Being hit by vegetation; Possible damages to the worker—Death or total permanent disability; Ongoing occupational risk control—The machine has a reinforced cabin, and workers wear PPE: boots, leggings, helmet, and gloves;
Scenario 2	Activity—Planned cutting and felling; Task—Cutting vegetation: this task consists of opening trails with a machete, through which workers will pass to carry out the activity; Meteorology—Precipitation probability: 9%; humidity: 79%; wind intensity: 13 km/h; Vegetation—Dense lowland ombrophilous forest; Wild animals—Contact with large mammals, snakes, and poisonous insects is possible; Terrain characteristics—Smoothly sloppy surface, with obstacles that are easy to cross; Workers involved in the task—One forestry extraction assistant (general); Workload—44 h per week; Relationship of this task with others in this activity—No other tasks occur in the same location; Tool/Machine used—Hand tool (machete); Hazard/danger—Tool; Trigger—Handling tool; Risk—Being hit by the tool blade; Possible damages to the worker—Temporary disability (less than 15 days’ leave); Ongoing occupational risk control—PPE: boots, leggings, helmet, and gloves.
Scenario 3	Activity—Planned cutting and felling; Task—Displacement between vegetation: this task involves moving the worker with the chainsaw along trails to where the cutting activity will occur; Meteorology—Precipitation probability: 10%; humidity: 77%; wind intensity: 11 km/h; Vegetation—Dense lowland ombrophilous forest; Wild animals—Possible contact with large mammals, snakes, and poisonous insects; Terrain characteristics—Strongly sloppy surface, with obstacles that are easy to cross; Workers involved in the task—One chainsaw operator; Workload—9 h per week; Relationship of this task with others in this activity—No other tasks occur in the same location; Tool/Machine used—No tool at the moment; Hazard/danger—Surface obstacles; Triggering factor—Overcoming obstacles; Risk—stumbling or falling; Possible damages to the worker—No disability (no leave); Ongoing occupational risk control—PPE: boots, leggings, helmet, and gloves.
Scenario 4	Activity—Planned cutting and felling; Task—Identification Replacement: this task consists of replacing the nameplate of the tree on its stump; Meteorology—Precipitation probability: 18%; humidity: 98%; wind intensity: 8 km/h; Vegetation—Dense lowland ombrophilous forest; Wild animals—Contact with large mammals, snakes, and poisonous insects is possible; Terrain characteristics—Flat surface with obstacles that are easy to cross; Workers involved in the task—One forestry extraction assistant (general); Workload—6 h per week; Relationship of this task with others in this activity—No other tasks occur in the same location; Tool/Machine used—Hand tool (hammer); Hazard/danger—Branch; Triggering factor—Detachment of branches that were stuck to the treetops surrounding the tree that was cut down; Risk—Being hit by a branch; Possible damages to the worker—Death or total permanent disability; Ongoing occupational risk control—PPE: boots, leggings, helmet, gloves, and ear protectors.
Scenario 5	Activity—Skidding planning; Task—Signaling: this task consists of identifying, with coloured ribbons, the vegetation that has been cut and that will be dragged; Meteorology—Precipitation probability: 10%; humidity: 98%; wind intensity: 10 km/h; Vegetation—Dense lowland ombrophilous forest; Wild animals—Contact with large mammals, snakes, and poisonous insects is possible; Terrain characteristics—Smoothly undulating surface with obstacles that are easy to cross; Workers involved in the task—One forestry extraction assistant (general); Workload—44 h per week; Relation of this task with others of this activity—In the same place, there are other workers making tree identification, in addition to the cutting team; Tool/Machine used—Hand tool (machete); Hazard/danger—Ant; Triggering factor—Contact with ants; Risk—Being bitten by ants; Possible damages to the worker—No disability (no leave); Ongoing occupational risk control—PPE: boots, leggings, helmet, and gloves.
Scenario 6	Activity—Crosscutting; Task—Crosscutting the felled logs: it consists of carrying out complementary cuts in the felled log, such as separating the crowns from the rest of the tree, separating the base of the logs with machetes or hollows, and cutting very long logs; Meteorology—Precipitation probability: 31%; humidity: 81%; wind intensity: 13 km/h; Vegetation—Dense lowland ombrophilous forest; Wild animals—Possible contact with large mammals, snakes, and poisonous insects; Terrain characteristics—Flat surfaces with obstacles that are easy to cross; Workers involved in the task—One chainsaw operator; Workload—18 h per week; Relationship of this task with others of this activity—There is another tracing team in the same place; Tool/Machine used—Chainsaw; Hazard/danger—Trunk; Triggering factor—Rolling the traced trunk over the worker; Risk—Being pinched by the trunk; Possible damages to the worker—Temporary disability (leave more than 15 days); Control of existing occupational risk—PPE: boots, leggings, helmet, gloves, and ear protectors.
Scenario 7	Activity—Skidding; Task—Preparing the log for dragging: this task involves positioning the forestry tractor and coupling the steel cable and hook around the log for lifting; Meteorology—Precipitation probability: 62%; humidity: 74%; wind intensity: 36 km/h; Vegetation—Dense lowland ombrophilous forest; Wild animals—Contact with snakes and poisonous insects is possible; Terrain characteristics—Smoothly undulating surface with obstacles that are easy to cross; Workers involved in the task—Two forest extraction assistants (general); Workload—18 h per week; Relationship of this task with others in this activity—No other tasks in the same location; Tool/Machine used—Forestry tractor with steel cable and accessories; Danger—The tree did not fall; it was propped up (monkey or widow); Triggering factor—Moving the tree with a tractor; Risk—Being crushed by a tree; Possible damages to the worker—Death or total permanent disability; Ongoing occupational risk control—PPE: boots, leggings, helmet, and gloves.
Scenario 8	Activity—Packing and stacking; Task—Identification and registration of logs; Meteorology Precipitation probability: 20%; humidity: 77%; wind intensity: 14 km/h; Vegetation—Dense lowland ombrophilous forest; Wild animals—Contact with snakes and poisonous insects is possible; Terrain characteristics—Flat surfaces with obstacles that are very easy to cross; Workers involved in the task—One production logger (note), one forestry extraction assistant (general), and one loader operator; Workload—44 h per week; Relation of this task with others of this activity—In the same place, five tasks take place that are a part of the activity: receiving the logs in the yard, quantifying their dimensions (length and diameter), recording this information (complemented with the information from the chain custody), marking the log with this information and, finally, stacking the log on the sides of the yard; Tool/Machine used—Fork loader; Danger—Fork loader; Triggering factor—Approaching the moving machine; Risk—Being run over by the fork loader; Possible damages to the worker—Permanent partial disability; Ongoing occupational risk control—PPE: boots, leggings, helmet, and gloves.
Scenario 9	Activity—Loading and transport; Task—Secure the load: this task consists of securing the load with a steel cable, using the ratchet so that it does not move during transport; Meteorology—Precipitation probability: 10%; humidity: 82%; wind intensity: 14 km/h; Vegetation—Dense lowland ombrophilous forest; Wild animals—Contact with snakes and poisonous insects is possible; Terrain characteristics—Flat surfaces with obstacles that are very easy to cross; Workers involved in the task—One truck driver; Workload—Three hours per week; Relationship of this task with others in this activity—No other tasks occur in the same location; Tool/Machine used—Timber truck, using steel cable and ratchet; Danger—Steel cable; Triggering factor—Handling the wire rope; Risk—Trap the fingers between the handle and the log; Possible damages to the worker—Temporary disability (less than 15 days’ leave); Ongoing occupational risk control—Stanchion and PPE: boots, leggings, helmet, and gloves.
Scenario 10	Activity—Loading and transport; Task—Drop the load: this task consists of releasing the steel cables from the turnstile to unload the load from the truck to the yard; Meteorology—Precipitation probability: 20%; humidity: 77%; wind intensity: 14 km/h; Vegetation—Dense lowland ombrophilous forest; Wild animals—Contact with snakes and poisonous insects is possible; Terrain characteristics—Flat surfaces with obstacles that are easy to cross; Workers involved in the task—One truck driver; Workload—Two hours per week; Relation of this task with others of this activity—There is unloading, classification and stacking of logs in the same place; Tool/Machine used—Timber truck, using steel cable and ratchet; Danger—Logs; Triggering factor—Uncontrolled sliding of logs stacked on the truck; Risk—Being crushed by logs; Possible damages to the worker—Death or total permanent disability; Ongoing occupational risk control—Stanchion and PPE: boots, leggings, helmet, and gloves.

) was framed in one of the seven activities of the logging process:

• Infrastructure construction—At this stage, vegetation is removed from the areas to be used for both the log yard and the roadways previously planned for different types of vehicles. To avoid accidents caused by tyre punctures, the roots are also uprooted.
• Planned felling and cross-cutting of logs—This phase involves the planned felling of trees with a previously defined diameter and in accordance with technical guidelines, followed by the cross-cutting of the tree tops.
• Skidding planning/Obstacle crosscutting activity—This operation defines the paths along which the tractor will drag the logs; then, all obstacles that might hinder the operation of the equipment are removed.
• Crosscutting—This process consists of cutting the treetops and sectioning the trunks of the trees. Generally, the cutting of the trunk is undertaken with a chainsaw to facilitate its dragging, as well as cutting the bases of the logs that have machetes or hollows.
• Skidding—involves moving the cut wood logs from the felling site to the storage yards.
• Cubage and stacking—This activity includes measuring the logs and confirming the species. The custody chain shown on the logging map is filled in. The logs are marked with an identification that allows their tracking according to commercial conformity. Then they are stored awaiting dispatch.
• Loading and transport—This stage consists of sorting of the species and loading the logs onto lorries to transfer them from the secondary parks to the central park. From the central park, the logs are then loaded for their final destination.

To facilitate the experts’ work, the analysis of reliability and reproducibility was performed simultaneously, and only one hazard/risk pair was identified in each scenario. In each of the ten scenarios, the occupational risk parameters were assessed using a five-level scale: level 1, corresponding to a very low risk level; level 2, low risk; level 3, moderate risk; level 4, high risk level; and level 5, corresponding to a very high or intolerable risk level. To analyse each situation, the experts had access to photographs and a short video.

2.3. Applying Delphi

The MIAR^forest reproducibility was assessed by verifying whether the experts assigned the same value to each scenario’s different parameters and sub-parameters. As for reliability, the assessment was carried out by asking the experts if they agreed with the risk level calculated by the method and, if not, which risk level they wanted to assign.

Both the reproducibility and reliability of MIAR^forest were tested using the Delphi methodology [33] in two rounds, with the experts’ opinions being collected by filling in questionnaires developed using Google Forms.

Experts were selected based on training and competence in occupational safety and health. These experts were identified through CREA, the Brazilian Regional Council of Engineering and Agronomy. The professionals who agreed to participate in the research received the link to the questionnaire, with the instructions for its completion, the description of MIAR^forest, and a set of 10 scenarios to evaluate the occupational risk.

In each scenario, a photograph and a description of the working situation were presented according to the report observation grid of the method [25]. The answers obtained were analysed quantitatively. Following the principles of the Delphi methodology, scenarios that still needed to achieve a consensus on the answers of 80% or higher underwent further evaluation by the experts. In this follow-up evaluation, the queries posed to the experts were crafted using the identical method applied in the initial round.

3. Results and Discussion

The consensus was obtained after two rounds. In the first round, 55 experts (19 technicians, 3 bachelors, 16 postgrads, 11 MSc, 5 PhD, 1 PosDoc) consented to complete the questionnaire, and the number of respondents decreased to 46 (19 technicians, 2 bachelors, 12 postgrads, 7 MSc, 5 PhD, 1 PosDoc). In each round, the evaluators were previously informed about the assumptions of MIAR^forest and the objectives inherent to each question. The participants’ anonymity was ensured.

All the experts have the necessary training and qualifications the Brazilian government requires to carry out risk analysis. They are safety engineers or have other legal qualifications that allow them to carry out this activity. Experts with a minimum of two years’ experience were selected.

3.1. Other Approaches

No studies were found to validate the results of any accident risk assessment method, neither for native rainforests nor planted forests. The current studies address various significant topics but with distinct aims, including best practices, worker perceptions of hazards, and injury prevention. These studies are inherently subjective and often suffer from limited sample sizes or insufficient data to support or disprove their claims. Further research on these matters holds significant value in verifying their relevance to impartial populations, such as other organisations or different countries [3,16,34,35,36,37,38,39,40].

Observation and technical visits produce articles more focused on risk assessment and overall OHS conditions [3,40,41,42,43].

Surveys are used to characterise populations selected non-randomly for studies. They are usually complemented with interviews or questionnaires in which questions posed to that population are answered [16,20,38,44,45].

Other papers contain reports on injuries or accidents that allow for a statistical hypothesis that supports the authors in their conclusions regarding the scope of their analysis. Many of the challenges discussed in these articles were experienced by the majority of the authors. More rigour and detail are often needed in these injury/accident reports. Forestry is sometimes lumped together with agriculture, which has completely different characteristics. The conclusions often point to the need to improve the reporting system. Most of these articles also need to characterise the study population unequivocally [35,46,47,48,49,50,51,52,53].

Quantitative studies often narrow their focus to a particular issue, primarily centred around posture assessments for equipment operators using the Ovako Working Posture Analyzing System (OWAS), the Rapid Upper Limb Assessment (RULA), and the Rapid Entire Body Assessment (REBA) [54,55,56,57,58,59,60]. The remaining articles considered risks such as exposure to heat, noise, vibrations, dust, and sometimes heart rate and physical workload [43,57,61,62,63,64,65,66].

3.2. MIAR^forest Reproducibility of the Results

Table 2. First/second round results—agreement percentages.

Scenario	Danger	Triggering Factor		Risk		Severity (G)								Exposure	Frequency	Risk Control
						MT	OF	WA	RBT	Obst	TS	PI	WI	E	F	RC
1	Vegetation	Tractor movement through the vegetation		Get hit by vegetation		82	84	73/ 94	87	87	87	84	93	96	84	86
2	Tool	Handle tools		Getting hit by the tool’s blade		91	82	75/ 91	87	86	87	84	95	91	86	96
3	surface obstacles	Overcome obstacles		Tripping or falling		72/ 94	79/ 96	76/ 96	89	84	84	86	95	95	86	89
4	Branch	Falling branches		Getting hit by a branch		85	86	76/ 96	95	86	87	93	89	93	86	98
5	Ant	Contact ants		Being stung by ants		82	84	82	80	89	87	93	91	82	87	98
6	Trunk	Roll the traced trunk over the worker		Being pressed by the torso		85	85	76/ 96	78/ 96	86	89	93	93	84	87	96
7	Tree partially fallen	Moving the tree with a tractor		Get crushed by the tree		83	91	76/ 94	87	86	87	91	89	95	87	98
8	Fork loader	Work close to the fork loader		Getting run over by the fork loader		85	85	76/ 96	80	80	84	95	93	93	86	96
9	Steel cable	Moving machine approach		Trap the fingers between the handle and the log		83	85	75/ 98	93	89	86	93	93	96	84	70
10	Logs	Handle the steel cable		Being crushed by logs		80	88	80	75/ 94	64	84	93	93	95	86	91
	Absent/Very-low		Low		Moderate		High					Very-High

MT—Machines and tools handling; OF—Object fall; WA—Wild animals; RBT—Relationship between tasks; Obst—Obstacles; TS—Terrain Slope; PI—Precipitation Intensity; WI—Wind Intensity; Ei—Extent of Impact; F—Frequency of Exposure; RC—Risk Control.

shows the approval percentages for each of the ten assessed situations. In the cells where two values occur, the first represents the result of the first round, and the second, the results of the second round. The colour of the cell represents the consensual rating for each sub-parameter. It can be seen that there was consensus in all rating levels, meaning that the method is able to respond across the entire risk scale.

Of the 120 questions presented in the first round, only 12 had to be resubmitted to the experts in a second round. These questions were distributed across nine of the ten presented scenarios, corresponding to 10% of the initial questions.

Among the questions that still need to receive a minimum 80% agreement regarding reproducibility, it is worth mentioning those related to wild animals (WA), where only scenarios 5 and 10 obtained an agreement above 80%. The analysis of the reproducibility of the answers in the first and second rounds relative to each of the sub-parameters for each scenario is presented in

Table 3. Mean and standard deviation of the agreement percentages.

	MT	OF	WA	RBT	Obst	TS	PI	WI	Ei	Fe	RC
Mean ± SD
1st Round	83 ± 4.8	85 ± 3.2	77 ± 2.6	85 ± 6.5	83 ± 7.4	86 ± 1.9	90 ± 4.2	92 ± 1.9	92 ± 5.1	86 ± 1.3	92 ± 8.8
2nd Round	85 ± 4.2	86 ± 4.1	92 ± 6.1	89 ± 5.5	87 ± 4.1	86 ± 1.9	90 ± 4.2	92 ± 1.9	92 ± 5.1	86 ± 1.3	95 ± 4.4

In italics, the values evaluated in two rounds.

.

More than an 89% consensus was obtained in the first round for all sub-parameters. Level 4 (red) assessments were the ones that gathered the highest percentages of consensus, with an average of 94% (Table 2). Levels 1 (green), 2 (yellow), 3 (orange), and 5 (purple) obtained average consensuses of 84%, 86%, 87%, and 86%, respectively.

After the second round, all sub-parameters obtained consensuses greater than 80%, and there was no variation regarding the level.

The application of the Delphi methodology to assess the reproducibility of the MIAR^forest shows average agreement values above 86% and 89% for all sub-parameters in the first and second rounds, respectively.

Table 3 presents the average results concerning the percentage of convergent answers for each sub-parameter for all scenarios, showing the average results and respective standard deviations.

3.3. Reliability of the Results

The reliability of the results ensures the evaluator that the obtained risk level accurately corresponds to the appropriate answer for the problem under analysis.

Table 4. Determination of the risk level (RL) and the weighted risk level (WRL) after the second round.

Scenario				1	2	3	4	5	6	7	8	9	10
Severity sub-parameters (S)			WP *	0.25	1	1	1	1	1	1	1	1	1
			TF **	8	8	8	8	8	8	8	8	8	8
			MT	1	4	1	1	2	4	1	8	4	1
			OF	48	3	6	48	3	12	48	3	3	48
			WA	0.25	0.25	0.25	0.25	0.5	0.5	0.25	0.25	0.25	0.25
			RBT	1	0.5	0.5	0.5	1	0.5	0.5	8	0.5	0.5
			Obst	4	1	1	1	1	1	1	0.5	0.5	1
			TS	1	1	4	0.5	1	0.5	1	0.5	0.5	0.5
			PI	1	0.5	0.5	0.5	0.5	0.5	1	0.5	0.5	0.5
			WI	1	1	1	0.5	0.5	1	2	1	1	1
Maximum				48	4	6	48	3	12	48	8	4	48
Median				1.00	1.00	1.00	0.50	1.00	0.75	1.00	0.75	0.50	0.75
Severity (Equation (3))				96	32	48	192	24	72	384	48	16	288
Extent of Impact (Ei)				4	4	4	4	4	4	4.4	4.7	4	4.0
Frequency of Exposure (Fe)				4	5	4	3.5	5	4	4	5	3.5	3.5
Likelihood (Li) (Equation (2))				16	20	16	14	20	16	17.6	23.5	14	14
Risk Level (RL) (Equation (1))				1536	640	768	2688	480	1152	6758.4	1128	224	4032
Risk Control (RC)				0.75	0.75	0.75	0.75	0.75	0.75	0.50	0.50	0.75	0.50
Weighted Risk Level (WRL) (Equation (4))				1536	640	768	2688	480	1152	6758.4			4032
Experts perception
	Absent/Very-low		Low		Moderate			High			Very-High

* Worker Protection; ** Forest Typology. Note—Cells with X (crossed lines) reflect a lack of consensus in the scenario.

presents the values of the different parameters and sub-parameters for each examined risk situation. Additionally, it displays the calculated risk levels obtained by applying the method. The final row of Table 4 depicts the experts’ perception levels for all ten situations using a colourimetric scale.

Furthermore, in Table 4, the values assigned to worker protection, where only two options exist (individual protection and collective protection), and to the forest typology, which encompasses data collected solely from a Submontane open ombrophilous forest environment, did not undergo validation. This is because an accurate classification corresponds to an evident and factual categorisation.

In the second round, the method was applied by 46 experts to analyse ten different scenarios. The results showed that in eight of the ten scenarios (1, 2, 3, 4, 5, 6, 7, 10), the risk level obtained through the method coincided with the experts’ perception in 80% of the cases. These results provide strong evidence that implementing MIAR^forest effectively reduces subjectivity and improves the accuracy of risk assessments by achieving a broad consensus among experts.

However, it is important to mention a limitation of this study related to the number of risk situations analysed. Expanding the dissemination and promotion of the method in forest extraction units in controlled logging areas is crucial for future advances. This initiative aims to strengthen occupational safety and health in these activities and continuously collect data to improve the method.

By extending the application and knowledge of MIAR^forest in these specific areas, it will be possible to obtain more comprehensive information about workers’ risks. This will allow for constant adjustments and improvements to the method, ensuring that it is aligned with the needs and challenges of forest management activities. Collaboration with the forest extraction units and obtaining direct feedback from the professionals involved will be essential in this continuous improvement process.

4. Conclusions

This study aimed to verify the reproducibility and reliability of the MIAR^forest.

Based on the results, the instrument used for assessing the risk of occupational accidents demonstrates high reproducibility. A remarkable percentage of the experts (bigger than 80%) reached an agreement in their assessments, indicating consistent outcomes. Similarly, the method exhibited a reliability rate bigger than 80%, signifying a strong alignment between the calculated values and the experts’ perceptions. These commendable characteristics enable even less experienced users to utilise the method confidently, as it offers a reduced and controlled margin of error.

Moreover, once this method reduces the influence of subjectivity, it can support future decision-making by other native forest exploitation companies and public authorities concerning safety and health in forest management activities.