Sustainability Assessment of Machinery Safety in a Manufacturing Organization Using AHP and CART Methods

Abstract

Machine safety is not only a prerequisite for successful production but also the foundation for the sustainability and growth of any manufacturing organization. The latest approaches in this rapidly developing field integrate effective risk management tools and strategies into occupational health and safety (OHS) management systems. The study, through a real example from practice, describes the use of the analytic hierarchy process (AHP) method for machine safety improvement, considering the possible types of losses. Classification and Regression Tree Analysis (CART) was applied to assess the efficiency, cost-effectiveness, and, therefore, the overall sustainability level of the relevant safety measures. These were proposed risk reduction measures that typically raised uncertainty among managers regarding the estimation of cost-effectiveness. The advantage of the application decision tree approach is the possibility to identify and establish relatively homogeneous groups of undesirable events and their impact on the organization’s objectives. A comprehensive model has been developed to support management decision making in manufacturing organizations towards implementing and improving safety measures in line with manufacturing sustainability goals.

1. Introduction

Safety-related data (SRD) are essential to maximize the effectiveness of the risk assessment process. SRD need to be as accurate and detailed as possible at all stages of the system life cycle; however, at the design stage, they are the basis for achieving the required level of occupational health and safety (OHS). According to Badri et al. [1], to improve the process of safety decision making (SDM) to achieve a safety level, SRD can be considered as the most valuable assets for all safety professionals. The combination of historical and current data strengthens real-time decision making during the manufacturing process, positively affecting industrial systems’ performance, safety, reliability, and the sustainability of industrial systems [2].

In the field of machinery safety, it is crucial to identify all hazards associated with machinery by collecting accurate and relevant safety data. Accurate machine safety-related data (MSRD) analysis is needed to use machine safety knowledge and to make more realistic risk assessments. However, the lack of appropriate communication between machine manufacturers and end users further magnifies the current problem and hinders sufficient transfer of MSRD and feedback from end-user experience [3].

Taking care of employee safety and minimizing the impact of hazards when in contact with machinery is a multi-dimensional prerequisite for any sustainable production. In addition to the technical, economic, health, social, motivational, or legislative aspects, it also has a highly ethical dimension. According to Lee et al. [4], most organizations strive for sustainability by providing workplace conditions that contribute to employee health and safety. However, choosing effective and efficient measures to proactively reduce risks is a challenging process related to experience and the application of appropriate procedures and methods supporting correct decision making. Therefore, developing a workplace safety culture is a competitive factor for sustainable companies.

2. Literature

Managers at all levels of management are responsible not only for their subordinates but also for managing safety within their scope of responsibility [5,6,7]. Any misconduct by a manager is usually perceived with great sensitivity by the injured person, as well as by the wider community. On the other hand, all relevant measures to improve OHS require incurring considerable costs. It is often economically prohibitive to carry out all remedial measures at once. From the organization’s perspective, it should be a process that ensures the cost acceptability of fulfilling the requirement to maintain and improve the safety of machinery. However, this is no cheap proposition. Particularly in larger manufacturing operations, which contain a number of machines at different levels of wear and tear (resulting from lifetime use) operating within several production lines, usually on a single shop floor, the sustainability of their safety is almost always a serious economic issue.

In manufacturing, machinery creates a dynamic environment, particularly in meeting Industry 4.0 requirements [8], where quality, accuracy, and efficiency are paramount. However, this also poses specific risks for the operators, as well as for other people in the vicinity of the machine. Ensuring safety in manufacturing is not only a legal requirement [9], but also a moral obligation to protect employees and sustain operations. Leveson [10] defines “safety as the absence of accidents, where an accident is defined as an event involving unplanned and unacceptable loss”. Safety is also associated with low and acceptable risk [11,12,13].

Nowadays, the concept of any development is associated with the concept of sustainability. Some authors point out [14,15] that OHS, injury, and illness prevention are issues that may raise concerns in terms of sustainability. Boileau [14] highlights three areas to consider in sustainable safety: integrating safety requirements at the system design stage, incorporating OHS into strategic plans, and implementing a safety culture into the organization. According to Hogevold et al. [16] and Dyllick and Muff [17], a sustainable business is a business that is economically, socially, and environmentally responsible, without negative impacts on all these areas. Currently, the direction of the business, from small to large organizations, is contributing to a more sustainable future.

However, if organizations fail to procure and maintain machine safety, provide adequate Personal Protective Equipment (PPE), ensure proper training for employees, or implement a safety culture, a variety of undesirable events that threaten the sustainability of the business are likely to occur. If an undesirable event occurs due to insufficient approaches to OHS management, the manufacturing organization may incur various costs. Many of these costs can be direct and immediate, but often more significant are the so-called indirect costs [18]. Indirect (“hidden” or “invisible”) costs of an undesirable event can include a negative impact on the employer’s reputation, lost time due to injury investigation, cleanup of the incident site, and reduced productivity and morale among employees [19].

Bearing the initial costs of implementing and complying with a comprehensive occupational health and safety program (e.g., with the support of an ISO 45001 OHS management system [20,21]), complying with regulations, and providing employees with proper training and PPE is much more likely to save more costs than the costs associated with the occurrence of an undesirable event [21].

However, safety in manufacturing is not just about compliance; it is about promoting a safety culture, minimizing risks, and optimizing productivity. It is a multifaceted effort that requires a holistic approach [22]. By implementing strict safety procedures, investing in appropriate machinery safety devices, and promoting a safety culture, machinery can significantly reduce the number of undesirable events in the workplace, protect employees, and ensure business continuity. Safety can be understood in a generic sense as a fundamental pillar of any manufacturing organization that adds value to the business’s sustainability [23].

A sustainable approach to machine operator safety in an organization involves the integration of environmental, social, and economic aspects into OHS management to not only protect the health and safety of workers but also to ensure the long-term sustainability of the business and to minimize environmental impact. This approach requires comprehensive planning, implementation, monitoring, and continuous improvement of safety procedures and systems. The perception of safety as the opposite of risk, i.e., the lower the risk, the higher the safety, and vice versa, has been challenged by several authors. Already, Möller et al. [24] concluded that it is necessary to go beyond thinking in this relationship of “safety as an antonym of risk”. A different approach to risk is taken by Aven and Renn [25], who view it as uncertainty as well as with respect to its impact on activities. The health and safety of machine operators should be seen as part of a broader corporate sustainability strategy (Corporate Social Responsibility, CSR). This approach involves considering safety measures in the context of environmental responsibility (e.g., minimizing waste and emissions, efficient use of energy, etc.) and social responsibility (e.g., ensuring fair working conditions, promoting the health and well-being of employees) [26,27].

Integrating safety objectives into the overall sustainability goals of the organization allows for better coordination and more efficient use of resources. Sustainable safety emphasizes preventive measures that focus on preventing accidents and injuries before they occur. This includes comprehensive risk identification, analysis, and assessment, as well as the development and implementation of innovative technological solutions and safety systems. A preventive approach also encourages investment in the development and training of employees to be better prepared to identify and respond correctly to potential risk situations in machinery. The safety of machinery operators in an organization is a complex area that requires a multidisciplinary approach involving engineering, ergonomics, occupational psychology, and legislation.

The aim of this research was (1) to apply risk assessment procedures based on the analysis of available data on the status of 124 devices in the manufacturing organization of machinery safety-related data; (2) to support the estimation of the cost-effectiveness of selected measures to reduce unacceptable risks using the AHP method; and (3) to use the CART technique to support the decision-making process (machinery safety decision making, MSDM) to assess the effectiveness and efficiency of the proposed measures.

3. Material and Methods

The research was conducted at an organization that is a global manufacturer of healthcare products. It focuses on the manufacture of market-leading medical devices. These devices are in the areas of advanced wound care, ostomy care, continence and critical care, and infusion devices.

A key step to ensure safety when working with machinery is to identify the potential risk scenarios associated with its operation (hazard → hazardous situation/event → harm). This process, MSRD, involves analyzing all data on the machine’s operation, including possible failures, accidents, or hazardous situations that could cause an accident. After identification of the hazards, a risk assessment must be carried out, i.e., to determine the probability of occurrence of harm related to the hazardous situation ($\mathrm{P}$) and severity of harm ($\mathrm{S}$). These estimates are usually represented by a numerical scale corresponding to a verbal expression.

Note: To avoid uncertainty and complexity in data collection, the parameter probability of a hazardous situation (P) has been defined as the overall exposure probability of persons to a hazard during a given machine operation, combined with the probability of a hazardous event occurrence, which in the description of the accident scenario is related, e.g., to unintentional human error or machine failure resulting in harm.

Due to the variety of machines in operation at the company and their life cycle (some machines were more than 20 years old), a risk assessment methodology (ISO 12100 [28], see

Table 1. Risk scoring matrix.

		Severity of Harm $(\mathit{S}$)
	$\mathit{R}=\mathit{P}\ast \mathit{S}$	Negligible 1	Minor 2	Moderate 3	Major 4	Severe 5
$\mathrm{Probability}\mathrm{of}\mathrm{occurrence}\mathrm{of}\mathrm{harm}(P$)	Very unlikely (>3 years) 1	1	2	3	4	5
	Unlikely (1–3 years) 2	2	4	6	8	10
	Possible (6 months to a year) 3	3	6	9	12	15
	Likely (1 month to six months) 4	4	8	12	16	20
	Very likely (<1 month) 5	5	10	15	20	25
Risk Assessment—Risk Category
LOW Score: 1–3		Low or acceptable risk. Proceed with due care and attention, following instructions and procedures.
MEDIUM Score: 4–7		Moderate or tolerable risk. Task must be carried out according to training and in accordance with safety procedures. Any appropriate control measures must be in place before the task/activity commences.
HIGH Score: 8–14		High or undesirable risk. Management consideration and authorization is required for its tolerance. Where possible, the task should be reassessed to take account of the associated risks and additional control measures put in place to reduce the risk (if feasible).
STOP Score: 15–20		Very high or unacceptable risk. This role must not be continued. Further action needs to be reviewed or implemented immediately. These measures should be reconsidered.
STOP—STOP Score: 25		A strongly unacceptable risk. Operation must be stopped immediately. Redesign of the machine must be considered.

) was designed by the research team and agreed upon by the OHS management of the organization. The subjective specific score value for the assessed risk parameters was estimated based on the available safety information for individual machines and consultations with safety engineers, operators, and maintenance staff.

Risk (R) is usually expressed [28] as a combination of the probability of occurrence of harm related to the hazardous situation (P) and the severity of that harm (S):

$$R=P\ast S$$

(1)

Based on the risk assessment using the risk matrix, it was possible to choose measures to reduce the risk (see Table 1).

Although the organization had previously written operational documentation in accordance with the current legislation on machinery (Directive 2006/42/EC [29]), due to the age of the machinery and the statistics of OHS accidents in the organization, the management was not satisfied with its relevance. A new “Machinery Risk Assessment” document was prepared by experts for each machine, which included an analysis and risk assessment of the machinery based on the OHS methodology approved by the management.

4. Results

An example of machinery risk analysis is selected in

Table 2. Machinery risk assessment—example.

Process	Activity	Type of Hazard	Hazardous Situation (HS)	Possible Consequences	Risk
					P	S	R	Category
Packaging manufacturing process	Material handling; shutdown and start-up	Mechanical	Kinetic energy (occurrence in the zone of product movement on the machine conveyor, movement of packaging machine parts)	Ejection, crushing	4	4	16	STOP
			Compressed air (activity in the vicinity of compressed air pipelines)	Ejection, crushing, abrasion	3	4	12	H
		Electrical	Electric arc (arc flash zone)	Burn, death	3	5	15	STOP
			Inappropriate contact (contact with live parts of electrical wiring)	Burn, death	2	5	10	H
		Radiation hazard	Optical radiation (exposure to optical radiation when checking product labeling)	Discomfort, fatigue, damage to eyes	3	4	12	H
			High-frequency el. magnetic radiation (activity in the zone of wireless transmission of production data)	Discomfort, fatigue (in the case of a pacemaker, even death)	3	4	12	H
		Material and substances	Air pollution (environmental pollution by dusty waste)	Over sensitization, infection	3	4	12	H
			Combustible materials (combustible dust in the workplace)	Over sensitization	3	5	15	STOP
				fire, explosion	3	5	15	STOP

. Based on the results of the MSRD from all 124 machines, a total of 5260 hazardous situations (HSs) and their resulting risks were identified. To assess the effectiveness and efficiency of the measures, the expert team, in cooperation with OHS management, focused on those risks whose value was higher than 9 and less than 20. Values of $\mathrm{R}$ that were less than 9 and greater than 20 were not considered. Lines with a value of less than 9 represented a lower priority risk of harm from machinery, and lines with a value of more than 20 represented the highest non-comparable priority, the so-called STOP-STOP category (Table 1), meaning immediate stoppage of machine operation (a highly unacceptable risk). The result was a database containing 489 pieces of data reaching the so-called critical risks, for which it was important to prioritize their reduction or elimination by adding additional measures. From this database, 49 machines (see

Table 3. Types of analyzed machinery.

Machinery	SUM
APS—Invisiclose pouch line	7
FUT—Bag production machine	2
Ostomy pouch assembly machine	1
Cutter	2
Folding machine	2
Automatic labeling system	10
Extruder	21
Material handling system	1
Mixer	1
Cartoon maker	2

Note: SUM—the number of machines of a given type.

) with the most frequently recurring types of hazardous situations (HS) were selected by the experts. These were type-non-homogeneous, variously worn-out machines, some of which are part of different production lines. They are arranged in the production halls in such a way that their individual hazardous zones overlap with each other. Therefore, the analysis included an assessment of the safety of the machinery in relation to both its layout and its interconnection within the production lines. Despite the heterogeneity mentioned above, these are machines that process the same material and contain similar components, and their operation is also very similar. Some types of hazardous situations were often repeated, and others were identical in content, just described differently.

This allowed experts to introduce a standard designation for the identified hazardous situations (HSs). A total of 10 main $\mathrm{H}\mathrm{S}\mathrm{i}$ types ($i=1,\dots ,10$) could be obtained from the 489 HSs after standardization, which were recurrent in the machine analysis.

Table 4. Number of occurrences of standardized HS on machinery.

$\mathbf{H}\mathbf{S}\mathbf{i}$	Hazardous Situation ($\mathbf{H}\mathbf{S}$)	Number of HS Occurrences ($\mathbf{O}$)
$\mathrm{H}\mathrm{S}1$	Air pollution	41
$\mathrm{H}\mathrm{S}2$	Combustible materials	74
$\mathrm{H}\mathrm{S}3$	Compressed air	92
$\mathrm{H}\mathrm{S}4$	Electric arc	76
$\mathrm{H}\mathrm{S}5$	Falling objects (e.g., tools and materials)	52
$\mathrm{H}\mathrm{S}6$	High-frequency electromagnetic radiation	9
$\mathrm{H}\mathrm{S}7$	Inappropriate contact	87
$\mathrm{H}\mathrm{S}8$	Instability (loose parts)	39
$\mathrm{H}\mathrm{S}9$	Kinetic energy	8
$\mathrm{H}\mathrm{S}10$	Optical radiation	15
Total sum		489

shows the $\mathrm{H}\mathrm{S}\mathrm{i}$ types with their number of occurrences O on the machines.

The possibility of avoidance of harm or impending HS is a very important parameter for risk management by appropriate measures. The expert group proposed to complement relationship (1) with the parameter $\mathrm{P}\mathrm{N}\mathrm{D}$—probability of non-detection of HS, which considers whether the undesirable situation (HS) can be prevented by preventive measures or only after its occurrence, so-called corrective measures. The experts proposed extending the risk estimation with one column labeled “Probability of non-detection of HS” (PND), inspired by the approach applied in Failure Modes, Effects, and Criticality Analysis (FMECA) [30,31]. The scores of this column are described in

Table 5. Quantified score values of PND.

Identifiable by Human Senses			Automatic Detection			Detection after $\mathbf{H}\mathbf{S}$ Occurrence (Corrective Measures)		Detection before $\mathbf{H}\mathbf{S}$ Occurrence (Preventive Measures)		Score $\mathbf{P}\mathbf{N}\mathbf{D}$
Yes	Partially	No	Directly	Indirectly	No	Yes	No	Yes	No
o										1
		o	o							1
	o			o		o				2
		o		o		o		o		2
		o		o		o			o	3
	o			o			o	o		3
		o		o			o	o		4
	o			o			o		o	4
		o		o			o		o	4
		o			o	o		o		5
		o			o	o			o	5
		o			o		o	o		5
		o			o		o		o	5

Note: HS—hazardous situations; PND—probability of non-detection HS/E, o—indication of the option.

, applying the Braglia [32] approach. In collaboration with the OHS organization experts, the relation for risk estimation was modified (see Equation (2)) and then applied to all 489 pieces of data, see

Table 6. Main $\mathrm{H}\mathrm{S}$ and assignment of parameters to determine their effects ($\mathrm{E}$).

	Label of $\mathbf{H}\mathbf{S}$	Number of $\mathbf{O}$	Process	Type of Hazard	Description of $\mathbf{H}\mathbf{S}$	Potential Harm	$\mathbf{P}$	$\mathbf{S}$	$\mathbf{P}\mathbf{N}\mathbf{D}$	${\mathbf{R}}_{\mathbf{M}}$	$\mathrm{Label}\mathrm{of}\mathbf{E}$
CAUSE	$\mathbf{H}\mathbf{S}\mathbf{1}$	41	Operation	Material/ substance	Air pollution	Over sensitization, infection	3	4	3	36	E1	EFFECT
	$\mathbf{H}\mathbf{S}\mathbf{2}$	15	Maintenance	Material/ substance	Combustible materials	Local fire	4	5	4	80	E2.1
		4				Explosion	4	5	5	100	E2.2
		12	Operation			Over sensitization	3	5	4	60	E2.3
		43				fire, explosion	2	5	5	50	E2.4
	$\mathbf{H}\mathbf{S}\mathbf{3}$	92	Operation	Mechanical	Compressed air	Ejection, crushing, abrasion	3	4	3	36	E3
	$\mathbf{H}\mathbf{S}\mathbf{4}$	31	Maintenance	Electrical	Electric arc	Burn, death	3	5	5	75	E4.1
		45	operation			Burn, death	2	5	4	40	E4.2
	$\mathbf{H}\mathbf{S}\mathbf{5}$	52	Maintenance	Mechanical	Falling objects (e.g., tools and materials)	Crushing, contusion, throwing	4	4	4	64	E5
	$\mathbf{H}\mathbf{S}\mathbf{6}$	9	Operation	Radiation	High-frequency electromagnetic radiation	Discomfort, fatigue (in the case of a pacemaker, even death)	3	4	5	60	E6
	$\mathbf{H}\mathbf{S}\mathbf{7}$	32	Operation	Electrical	Inappropriate contact	Burn, death	2	5	3	30	E7.1
		51	Maintenance	Mechanical		impact, crushing, ejection	4	3	2	24	E7.2
	$\mathbf{H}\mathbf{S}\mathbf{8}$	39	Maintenance	Mechanical	Instability (loose parts)	Impact, crushing	5	3	4	60	E8
	$\mathbf{H}\mathbf{S}\mathbf{9}$	8	Operation	Mechanical	Kinetic energy	Ejection, crushing	4	4	3	48	E9
	$\mathbf{H}\mathbf{S}\mathbf{10}$	15	Operation	Radiation	Optical radiation	discomfort, fatigue, damage to eyes	3	4	5	60	E10
	Total	489	-	-	-	-	-	-	-	823	-

Note: $\mathrm{H}\mathrm{S}$—hazardous situations; $\mathrm{O}$—number of HS occurrences; $\mathrm{P}$—probability of $\mathrm{H}\mathrm{S}$; $\mathrm{S}$—severity of the harm; $\mathrm{P}\mathrm{N}\mathrm{D}$—probability of non-detection of $\mathrm{H}\mathrm{S}$; ${\mathrm{R}}_{\mathrm{M}}$—risk of machinery; $\mathrm{E}$—effects.

.

Table 6 contains the assessment of ${\mathrm{R}}_{\mathrm{M}}$ (machine risk) based on two parameters obtained from the performed risk analysis and the third parameter, PND. These were added based on Table 5. The specific quantification ${\mathrm{R}}_{\mathrm{M}}$ is given by the relation:

$$R_M=P\ast S\ast PND$$

(2)

This is a similar relationship to the one used in the FMECA method [30,31], with the only difference being that the parameters mentioned above are estimated using a five-step scale, whereas FMECA uses standard 10-step scales. The individual rows of Table 4 contain the above-mentioned main HSs obtained from the safety analyses of the individual machines. Numerical values of the parameters P, S, and PND represent the maximum of the values of similar parameters from the risk assessment analysis of the machinery for a particular combination of HSs. Each risk presents the possibility of loss for the organization [19], the impact of which can be of different natures, e.g., loss of customer, penalties, loss of reputation, etc. For this reason, Table 6 has been extended with a column allowing for the assignment of a parameter to determine the significance value of negative impacts (E) from the point of view of the organization’s management when a given hazardous situation occurs (HS) in the performance of the activity (maintenance or operation) and severity of harm.

4.1. Estimation of Expected Cost Using the AHP Method

Estimation of the expected costs ($\mathrm{E}\mathrm{C}$) caused by the severity of harm (e.g., direct and indirect costs associated with injury caused by machine hazardous situation) generally cannot be estimated with sufficient precision [33]. On the other hand, the abovementioned costs are an important indicator of the effectiveness of measures that reduce or even eliminate the assessed risks. This problem can be solved, e.g., by using the AHP method. In this method, it is not necessary to accurately estimate the amount of expected costs for a specific criterion. It is sufficient if a pairwise comparison of the mentioned criteria can be performed [34].

AHP is a tool that can be used in decision making based on multiple criteria, where the level of information about criteria preferences is based on pairwise comparisons [35,36]. It is a powerful and flexible tool for complex problems where both qualitative and quantitative aspects need to be considered. It allows analysts to organize the critical aspects of a problem into a hierarchical structure like a family tree. By reducing complex decisions to a series of simple comparisons and rankings and then synthesizing the results, it supports coming to the best decision but also provides a clear reason for the decisions made.

Once the AHP hierarchy was defined (see Figure 1), it was possible to prioritize among the elements within each level of the hierarchy of the researched model. As mentioned above, both qualitative and quantitative criteria can be compared using informal judgments to derive priorities.

Part (a) of

Table 7. Pairwise combination comparison matrix E for estimating EC.

	E1	E2.1	E2.2	E2.3	E2.4	E3	E4.1	E4.2	E5	E6	E7.1	E7.2	E8	E9	E10	Local Priority $\mathbf{E}\mathbf{C}$
E1	-	0.2	0.2	0.2	0.2	0.25	0.33	2.00	0.25	1	1	2	0.33	0.25	5	0.027
E2.1	5	-	1	4	4	4	4	8	2	4	5	2	6	2	5	0.157
E2.2	5	1	-	4	4	4	4	8	2	4	5	2	6	2	5	0.157
E2.3	5	0.25	0.25	-	1	2	1	4	0.33	0.5	4	3	2	2	3	0.069
E2.4	5	0.25	0.25	1	-	2	1	4	0.33	0.5	4	3	2	2	3	0.069
E3	4	0.25	0.25	0.5	0.5	-	2	6	0.33	4	0.5	5	1	0.33	3	0.056
E4.1	3	0.25	0.25	1	1	0.5	-	6	0.33	4	2	0.25	1	0.33	2	0.049
E4.2	0.5	0.13	0.13	0.25	0.25	0.17	0.17	-	0.13	0.5	0.2	1	0.2	0.13	0.5	0.013
E5	4	0.5	0.5	3	3	3	3	8	-	6	3	6	3	1	4	0.115
E6	2	0.25	0.25	2	2	0.25	0.25	2	0.17	-	0.33	2	0.33	0.17	3	0.039
E7.1	2	0.2	0.2	0.25	0.25	2	0.5	5	0.33	3	-	5	1	0.33	2	0.046
E7.2	0.5	0.5	0.5	0.33	0.33	0.2	4	1	0.17	0.5	0.2	-	0.2	0.13	1	0.033
E8	3	0.17	0.17	0.5	0.5	1	1	5	0.33	3	1	5	-	0.33	4	0.049
E9	4	0.5	0.5	0.5	0.5	3	3	8	1	6	3	8	3	-	5	0.101
E10	0.2	0.2	0.2	0.33	0.33	0.33	0.5	2	0.25	0.33	0.5	1	0.25	0.2	-	0.020
(a)																(b)

Note: E—effect; EC—expected cost. (a)—pairwise comparison matrix; (b)—priority vector.

presents a pairwise comparison matrix of expected costs (EC) for all combinations of hazardous situation/effect (HS/E) of the analyzed HSs and the severity of their E, described in Table 6. For example, the second row of Table 7 indicates that E2.1 is assigned by the evaluation team as a priority that is five times higher than E1, the same priority as E2.2, four times higher than E2.3, E2.4, E3, and E4.1, eight times higher than E4.2, and so on. In terms of the AHP’s evaluation concept of pairwise comparisons, the second column, also labeled E2.1 in the table, represents the inverted values of the abovementioned priorities in the same order.

Using the AHP method, the calculated priorities of each combination are expressed as the expected cost ($EC$). These form part (b) of Table 7.

These are the values obtained using the following steps [37]:

• The calculation of the normalized comparison matrix was performed;
• The average weights (priorities) for each element were determined;
• The maximum eigenvalue was calculated (${\lambda }_{max}$);
• Specific coherence indices ($CI$) and the coherence ratio ($CR$) were calculated to verify the consistency of the comparison matrix.

The normalized matrix in Table 7 part (a) was used to determine the priorities of each HS/E combination (priority vector). Since the normalized matrix is of order 15 × 15, the Random Index $\left(RI=1.59\right)$ was used for the calculation:

• The priority vector for each HS/E combination (Table 7 part (b)) represents the priority or weight of the relevant element compared to the other elements;
• The maximum eigenvalue represents the value ${\lambda }_{max}=\mathrm{16,886}$;
• The consistent index is equal to $CI=0.1347$;
• The coherence ratio is equal to $CR=0.0847$.

Given that the coherence ratio ($CR$) is lower than 0.1, the comparison matrix can be considered sufficiently consistent by AHP standards. This means that the identified priorities can be considered reliable in the evaluation of alternatives.

In the next step of the analysis, pairwise comparisons were performed at the first level of the hierarchy to obtain value judgments.

Table 8. Variant A—pairwise comparison of criteria at the first level of the AHP hierarchy.

Criteria	$\mathbf{P}$	$\mathbf{P}\mathbf{N}\mathbf{D}$	$\mathbf{S}$	$\mathbf{E}\mathbf{C}$	$\mathbf{O}$	Priority
$\mathbf{P}$	-	3	1	1	0.50	0.2084
$\mathbf{P}\mathbf{N}\mathbf{D}$	0.33	-	0.50	0.33	0.50	0.0889
$\mathbf{S}$	1	2	-	0.50	1	0.1802
$\mathbf{E}\mathbf{C}$	1	3	2	-	2	0.3048
$\mathbf{O}$	2	2	1	0.50	-	0.2177
(a)						(b)

Note: P—probability of HS; PND—probability of non-detection of HS; S—severity of harm; EC—expected cost; O—number of HS occurrences. (a)—matrix of pairwise comparison of the criteria; (b)—prioritization vector.

part (a) shows the matrix A obtained from the pairwise comparison of criteria performed with the help of experts and OHS management. The comparison was performed in terms of the five criteria described in the previous section. The prioritization of the performance criteria was calculated similarly to the previous case. The prioritization vector (Table 8 part (b)) was determined by calculating the normalized components of the right eigenvector of the corresponding matrix.

The values in the Table 8 represent pairwise comparison weights, indicating how one criterion was evaluated against another. On the diagonal, the default values are 1, because each criterion is equally important in comparison to itself.

Similar to the first set, the procedure for calculating the priorities and consistency of this AHP matrix was as follows:

• The diagonal was filled with units, and the values were mirrored in the lower half of the matrix to the upper half to make it symmetrical;
• The matrix was normalized;
• Average weights (priorities) for each criterion were calculated;
• The consistency of the matrix was verified by calculating the consistency ratio (CR).

In the calculation described above, it was calculated:

• The maximum eigenvector is ${\lambda }_{max}=5.1835$;
• Consistent index is equal to $CI=0.0459$;
• The coherence ratio is equal to $CR=0.0409$.

The coherence ratio ($CR$) is again lower than $0.1$, so the comparison matrix is considered sufficiently consistent by AHP standards. In this case, the identified priorities can also be considered reliable in the evaluation of alternatives.

Table 8 part (b) shows the priorities for each criterion, i.e., their degree of importance, which are as follows:

• Probability of HS: $P=0.2084$;
• Severity of harm: $S=0.1802$;
• Number of HS occurrences: $O=0.2177$;
• Probability of non-detection of HS: $PND=0.0889$.
• The most important are the expected costs: $EC=0.3048$.

The value of the inconsistency ratio is equal to $0.04<0.1$, i.e., it is acceptable. This variant of pairwise comparisons was referred to as Variant A.

After evaluating the different causes of undesirable events (HSs) with respect to the criteria under consideration, all the evaluations must be aggregated in a hierarchical tree.

Table 9. Resulting prioritization for all parameters examined.

Alternative	Probability of HS $(\mathbf{P}$)		0.2084	Severity $(\mathbf{S}$)		0.1802	$\mathrm{Probability}\mathrm{of}\mathrm{Non}-\mathrm{Detection}(\mathbf{P}\mathbf{N}\mathbf{D}$)		0.0889	$\mathrm{Expected}\mathrm{Cos}\mathrm{t}(\mathbf{E}\mathbf{C}$)	0.3048	Occurrence Count (O)		0.2177	Evaluation
	P	Local Priority P	Total Priority P	S	Local Priority S	Total Priority S	PND	Local Priority PND	Total Priority PND	Local Priority EC	Total priority EC	O	Local Priority O	Total Priority O
E1	3	0.061	0.013	4	0.062	0.011	3	0.051	0.005	0.027	0.008	41	0.084	0.018	0.055
E2.1	4	0.082	0.017	5	0.077	0.014	4	0.068	0.006	0.157	0.047	15	0.031	0.007	0.091
E2.2	4	0.082	0.017	5	0.077	0.014	5	0.085	0.008	0.157	0.047	4	0.008	0.002	0.087
E2.3	3	0.061	0.013	5	0.077	0.014	4	0.068	0.006	0.069	0.021	12	0.025	0.005	0.059
E2.4	2	0.041	0.009	5	0.077	0.014	5	0.085	0.008	0.069	0.021	43	0.088	0.019	0.070
E3	3	0.061	0.013	4	0.062	0.011	3	0.051	0.005	0.056	0.017	92	0.188	0.041	0.086
E4.1	3	0.061	0.013	5	0.077	0.014	5	0.085	0.008	0.049	0.015	31	0.063	0.014	0.063
E4.2	2	0.041	0.009	5	0.077	0.014	4	0.068	0.006	0.013	0.004	45	0.092	0.020	0.052
E5	4	0.082	0.017	4	0.062	0.011	4	0.068	0.006	0.115	0.035	52	0.106	0.023	0.092
E6	3	0.061	0.013	4	0.062	0.011	5	0.085	0.008	0.039	0.012	9	0.018	0.004	0.048
E7.1	2	0.041	0.009	5	0.077	0.014	3	0.051	0.005	0.046	0.014	32	0.065	0.014	0.055
E7.2	4	0.082	0.017	3	0.046	0.008	2	0.034	0.003	0.033	0.011	51	0.104	0.023	0.062
E8	5	0.102	0.021	3	0.046	0.008	4	0.068	0.006	0.049	0.015	39	0.080	0.017	0.068
E9	4	0.082	0.017	4	0.062	0.011	3	0.051	0.005	0.101	0.031	8	0.016	0.004	0.067
E10	3	0.061	0.013	4	0.062	0.011	5	0.085	0.008	0.020	0.006	15	0.031	0.007	0.044
Total	49	1	0.2084	65	1	0.1802	59	1	0.0889	1	0.3048	489	1	0.2177	1

Note: P—probability of HS; PND—probability of non-detection of HS; S—severity of Harm; EC—expected cost; O—number of HS occurrences.

shows the priority of the probability of occurrence of HSs with respect to the five criteria from a local and total perspective (i.e., criterion priority multiplied by local cause priority (HS)). This means that in the next step, local and then total priorities were set for all five risk management parameters. The three “quantifiable” criteria can be evaluated by normalizing the quantitative assessments. These are the risk estimation parameters P, S, and PND (see Equation (2)). The normalization is performed by using the sums of all their scores (see the last row of Table 7). For example, for the parameter P, the value of $\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}=49$. The local prior for the parameter P and the value of E1 were calculated as $3/49=0.061$. A similar calculation for all rows for the three risk parameters mentioned above ${\mathrm{R}}_{\mathrm{M}}$ was followed. In the same way, the row values of the local priority were calculated for the parameter O. The local priority for the value of E1 and parameter O were calculated as $41/489=0.084$. The local priority for the parameter EC is represented by the priority vector described in Table 8 part (b). In the following step, the total priorities for each parameter were computed. The values from the priority vector described in section (b) of Table 8 were used for the calculation. For example, for the value of E1, the total prior to the parameter P was calculated as the product of the local priority times the corresponding priority from the priority vector, namely, $0.061\ast 0.2084=0.013$.

The other values of the overall priority parameter P, as well as the values of all other parameters, were calculated using the same procedure.

After calculating all five total priorities by multiplying them in the corresponding row, the values of the last column of Table 9 were obtained under the name “Evaluation”. As soon as the prioritization of the performance criteria is achieved by composing pairwise comparisons, the last step involves using the AHP framework to evaluate the different HSs. The last column of Table 9 shows the final ranking of the 15 causes/effects considered, i.e., the hazardous situations analyzed and their effects (HS/E).

As indicated in the table above, falling objects during machine maintenance (HS5) emerges as the most critical problem among the 15 alternatives in terms of expected costs (E5) resulting from the severity of harm when this hazardous event occurs, with an overall priority score of $0.092$. This is followed by ignition of combustible material during maintenance resulting in fire (E2.1), with a value of $0.091,$ and ignition of combustible material (E2.2) resulting in explosion during maintenance of the machine, with a similar value of $0.087$.

As can be seen, in the case of E5, the result of this analysis is different than the results of the previous analysis based on the risk calculation alone ${\mathrm{R}}_{\mathrm{M}}$ (Table 6). The most critical undesirable events are E2.2, with a corresponding value of $R_M=4\ast 5\ast 5=100,$ and E2.1, with the value $R_M=4\ast 5\ast 4=80$. These represent a high degree of probability of an undesirable event occurring that is both very serious and difficult to detect. This apparent contradiction is due to the way the table for Variant A has been constructed.

4.2. Sensitivity Analysis

The solution based on Table 8 for Variant A reflected a possible scenario in which the most important criteria are the expected costs (EC) and the number of undesirable event occurrences (O). In general, the model solutions may change in line with changes in the logic of the approach, given the focus of the analysis.

The sensitivity or robustness of the model solution to potential shifts in strategy priority creates the opportunity to analyze the sensitivity of the criteria weights to changes in the priority (relative importance) of the weights. In fact, each criterion is characterized by an important degree of sensitivity, i.e., the ranking of all events differs dramatically across the weight range [38]. Therefore, it is important to control to what extent changes in the evaluation ratings may lead to more or less significant changes in the final order of priorities. Sensitivity analysis is used to examine the sensitivity of alternatives to changes in the priority of criteria immediately below the target. The final solution is obviously sensitive not only to changes in the priorities of the top level of the hierarchy, but also to their interaction effects. These make the classical sensitivity analysis too complicated and difficult to implement in practice due to their high level of complexity. For simplification reasons, OHS experts and management were asked to suggest two more options that were extreme to critical but realistic. This simple, intuitive approach represents a compromise between the complexity of a precise solution and the simplicity of an easy-to-use solution in practice.

The proposed analysis sufficiently highlights the priorities of the five criteria used in the model and shows how a change in the priority of one criterion affects the priorities of the others. Clearly, as the priority of one criterion increases, the priorities of the remaining criteria must decrease in proportion to their original priorities, and the global priorities of the alternatives (see Table 9) must be recalculated.

Table 10. Variant B—pairwise comparison of criteria at the first level of the AHP hierarchy.

Criteria	$\mathbf{P}$	$\mathbf{P}\mathbf{N}\mathbf{D}$	$\mathbf{S}$	$\mathbf{E}\mathbf{C}$	$\mathbf{O}$	Priority
$\mathbf{P}$	-	3	1	0.33	0.33	0.127
$\mathbf{P}\mathbf{N}\mathbf{D}$	0.33	-	0.50	0.33	0.33	0.078
$\mathbf{S}$	2	2	-	0.33	0.33	0.145
$\mathbf{E}\mathbf{C}$	3	3	3	-	0.50	0.279
$\mathbf{O}$	3	3	3	2	-	0.369
(a)						(b)

Note: P—probability of HS; PND—probability of non-detection of HS; S—severity of harm; EC—expected cost; O—number of HS occurrences. (a)—matrix of pairwise comparison of the criteria; (b)—prioritization vector.

shows the priorities for each criterion, listed in order of importance:

• Chance of event (probability of HS): $P=0.127$;
• Severity of harm: $S=0.1802$;
• Number of HS occurrences: $O=0.369$;
• The chance of not detecting an undesirable event: $PND=0.08$;
• Expected costs: $EC=0.279$.
• This variant of the pairwise comparison will be named as Variant B.

In the calculation described above, the following were calculated:

• The largest eigenvector is ${\lambda }_{max}=5.587$;
• The consistency index is $CI=0.072$;
• The consistency ratio is $CR=0.064$.

The consistency ratio (CR) is less than $0.1$, which is the generally accepted threshold for an acceptable level of consistency in AHP analysis. This means that the comparisons between criteria in this analysis are sufficiently consistent, and the priority results can be considered reliable.

Table 11. Variant C—pairwise comparison of criteria at the first level of the AHP hierarchy.

Criteria	$\mathbf{P}$	$\mathbf{P}\mathbf{N}\mathbf{D}$	$\mathbf{S}$	$\mathbf{E}\mathbf{C}$	$\mathbf{O}$	Priority
$\mathbf{P}$	-	3	0.33	3	3	0.258
$\mathbf{P}\mathbf{N}\mathbf{D}$	0.33	-	0.33	0.5	0.33	0.078
$\mathbf{S}$	3	3	-	0.33	3	0.395
$\mathbf{E}\mathbf{C}$	0.33	2	0.33	-	0.50	0.109
$\mathbf{O}$	0.33	3	0.33	2	-	0.16
(a)						(b)

Note: P—probability of HS; PND—probability of non-detection of HS; S—severity of harm; EC—expected cost; O—number of HS occurrences. (a)—matrix of pairwise comparison of the criteria; (b)—prioritization vector.

shows the priorities of each criterion, listed in order of importance:

• Severity of harm: $S=0.395$;
• Chance of event (undesirable event): $P=0.258$;
• Number of HS occurrences: $O=0.16$;
• Expected costs: $EC=0.109$;
• The chance of not detecting an undesirable event: $PND=0.078$.

This variant of pairwise comparison will be named as Variant C.

Like the previous cases, the consistency of the matrix is verified by calculation:

• The largest eigenvector: ${\lambda }_{max}=5.331$;
• Consistency index: $CI=0.083$;
• Consistency ratio: $CR=0.074$.

A consistency ratio (CR) lower than $0.1$ indicates that the comparison matrix is sufficiently consistent for the purposes of AHP analysis. This indicates that the comparisons between criteria are consistent, and the results of the analysis are reliable (see

Table 12. Comparison of the priorities set for the different risk assessment variants.

Alternative	Evaluation—Variant A	Order of Events Variant A	Evaluation—Variant B	Order of Events Variant B	Evaluation—Variant C	Order of Events Variant C	Risk ${\mathbf{R}}_{\mathbf{M}}$	$\mathrm{Order}\mathrm{of}\mathrm{Events}\mathrm{by}{\mathbf{R}}_{\mathbf{M}}$
E1	0.055	11	0.059	7	0.057	11	36	9
E2.1	0.091	2	0.082	3	0.076	4	80	2
E2.2	0.087	3	0.075	4	0.079	2	100	1
E2.3	0.059	10	0.053	10	0.060	9	60	5
E2.4	0.070	5	0.075	4	0.072	6	75	3
E3	0.086	4	0.106	1	0.077	3	36	9
E4.1	0.063	8	0.063	6	0.071	7	75	3
E4.2	0.052	12	0.059	7	0.059	10	50	6
E5	0.092	1	0.096	2	0.082	1	64	4
E6	0.048	13	0.041	11	0.061	8	60	5
E7.1	0.055	11	0.057	9	0.052	12	40	8
E7.2	0.062	9	0.068	5	0.059	10	60	5
E8	0.068	6	0.068	5	0.075	5	75	3
E9	0.067	7	0.058	8	0.060	9	48	7
E10	0.044	14	0.040	12	0.061	8	60	5
Total	1	-	1	-	1	-	-	-

).

It should be noted that the sensitivity analysis is only relevant to the priorities of the five criteria.

If the values of only one attribute, one at a time, were to change, only the “main effects” would be considered; thus, the “interaction effects” of changes in two or more of the weights would be neglected. Therefore, an approach was adopted whereby different combinations of parameters were varied for each variant, depending on the focus of the analysis and in consultation with OHS management.

Even after different, radical changes in preferences in pairwise comparisons, some robustness can be observed for variants A, B, and C. The first four riskiest events in all variants are identical (E2.1, E2.2, E3, E5), albeit in a different order. Moreover, this (except for the E3 case) corresponds in some way to the assessment based on ${\mathrm{R}}_{\mathrm{M}}$ (see penultimate column of Table 6 or the last two columns of Table 12). In the original evaluation based on ${\mathrm{R}}_{\mathrm{M}}$, only the three parameters of P, S, and PND were considered. Thus, the other two parameters were not considered, i.e., the expected unit cost of the alternative, as well as O, i.e., the frequency of occurrence of the given alternative across all the machinery, were not considered. In the third place, when assessing the parameter ${\mathrm{R}}_{\mathrm{M}}$, three alternatives, E2.4, E4.1, and E8, were recorded immediately, but when evaluated according to the five parameters (Variants A to C), their priority was ranked at 4 and above. On the other hand, the parameter E3 was ranked as high as 9, but when evaluated on a variant-by-variant basis, it was ranked up to rank 4. This apparent mismatch in priorities is due to a more complex view, where alternative E3, although with a low-risk number $R_M=36$, has a higher occurrence than the other alternatives by an order of magnitude.

4.3. Results of the Evaluation of the Effectiveness and Efficiency of the Proposed Measures

In the previous section, the AHP method was described to determine the priority of each HS/E according to the severity of the overall impacts on the whole group of considered machinery. Thus, this examined the amount of additional costs or losses that could potentially be incurred by the organization with respect to each of the considered alternatives.

The next step of the study was focused on the assessment of the performance of the individual measures that have been proposed within the framework of the aforementioned “Machinery Risk Assessment” document. This is an important element in ensuring the sustainability of machinery safety. The MSDM process involves, in addition to the assessment of the assigned risks, the proposal of effective and efficient measures. Within the framework of the document referred to above, experts have defined the most appropriate types of measures (Ms) for each HS/E.

This means that for each of the 489 identified HS, specific measures have been proposed, and it has been estimated how these measures will reduce the resulting risk number. The reduced risk number is referred to as ${\mathrm{R}}_{\mathrm{A}\mathrm{M}}$.

After consideration of the OHS by management and evaluation of all the proposed measures, it was found that many cases are repetitive and can be standardly divided into seven basic types (see

Table 13. Types of individual measures.

$\mathbf{M}\mathbf{i}$	Measures	Hierarchy of Control [39]
M1	LOTO—safety padlock	Isolate people from the hazard
M2	Emergency stop button (ESB)	Isolate people from the hazard
M3	Guard	Elimination
M4	Education—training	Administration
M5	Automated ventilation system	Engineering
M6	PPE	PPE
M7	Periodic maintenance—inspections	Engineering

Note: Mi—measures (i = 1,…, 7); LOTO—lockout/tagout devices.

). All proposed measures represent some combination of the measures and risk management hierarchy according to NIOSH (the National Institute for Occupational Safety and Health) [39].

Two of these, i.e., $\mathrm{M}6$ and $\mathrm{M}7$, do not require any additional cost increase. These are measures that are routinely implemented in the organization. However, they have almost always been recommended in risk analyses, because many accidents or hazardous situations have been reported in which these generally legislated safety measures have not been followed. Compliance with these two measures is therefore considered to be the absolute basis for the risk management of machinery and has therefore not been considered in the following assessment.

Based on consultations with OHS management, the expected costs associated with implementing each machine and each of the five proposed measures were estimated.

It should be noted that the costs for each measure were only estimated at approximately the same time, without considering possible fluctuations in the price of materials and labor changes, etc.

The costs associated with the introduction of each measure were influenced not only by the specific machinery, but also by its location in the plant or its proximity to other machinery.

Table 14. Estimated unit costs for the identified measures.

$\mathbf{M}\mathbf{i}$	Measures	$\mathbf{C}\mathbf{M}$
		Min (Eur)	Max (Eur)
$\mathrm{M}1$	LOTO—safety padlock	25	78
$\mathrm{M}2$	E—STOP	44	88
$\mathrm{M}3$	Safety guard	500	1000
$\mathrm{M}4$	Education—training	50	500
$\mathrm{M}5$	Automated ventilation system	600	1000

Note: Mi—measures; LOTO—lockout/tagout devices; CM—cost of measures; E—STOP—emergency stop button.

shows the minimum and maximum estimated unit costs. Thus, these are the CM costs budgeted for each event, the machinery, as well as the type of measure proposed.

The effectiveness of the measures was assessed by the so-called AEI index (Action Effectiveness Index), whereby AEI is a metric used to assess the effectiveness of proposed or implemented corrective actions in reducing risk ${\mathrm{R}}_{\mathrm{M}}$. It aimed to quantify whether the proposed actions Mi reduce risk or improve detection capability of a HS before it could have a negative impact on the machinery operator.

Calculation of the AEI index is based on a comparison of the values of ${\mathrm{R}}_{\mathrm{M}}$ before and after the implementation of the measures, i.e., ${\mathrm{R}}_{\mathrm{A}\mathrm{M}}$ (AM—after measure). The index of effectiveness of the measures (AEI) can thus be calculated as a percentage change that shows how much the risk (${\mathrm{R}}_{\mathrm{M}}$) has been reduced because of the implementation of the measures. The following relation applies:

$$AEI=\frac{R_M-R_{AM}}{R_M}\ast 100\%$$

(3)

This calculation provides a quantitative indicator of measured effectiveness. If a high percentage is achieved, the measures have significantly reduced the risk. On the other hand, a low or negative percentage could indicate that the measures have not been sufficiently effective, or that the risk has not changed or has even increased.

It is important to note that the specific calculation methods and values may depend on the specific practices and policies of the organization. Therefore, when calculating AEI, it is always advisable to follow internal guidelines. Calculation of the AEI index was used to estimate the effectiveness of the implementation of the five proposed measures mentioned above (M1 to M5). For each, HS/E were almost always a combination of several of the five measures M1 to M5. Thus, the estimation of the resulting costs implied an accumulation of the mentioned values. Therefore, for Mi (M1 to M5), a CMi value was defined for each HS/E. These are the costs that potentially need to be incurred in relation to the measure $\mathrm{M}\mathrm{i}$. If any measures $\mathrm{M}\mathrm{i}$ are not for a given HSi proposed, the value CMi ($i=1,\dots ,5$) is zero. The resulting expected costs CM for each HS/E are therefore given by the relation:

$$CM=\sum _{i=1}^{5}CMi$$

(4)

The costs assigned to each measure as well as the values P and S described in relation (1) were used as predictor variables in the Classification and Regression Trees (CART) method. This is a method focusing on regression trees to model the relationship between the dependent variables.

4.4. Applying the CART Method for Assessing Effectiveness of $Mi$

This is a seven-node CART regression analysis, focusing on regression trees to model the relationship between the dependent variable (AEI) and the predictor variables (S, P, CM1 to CM5). The key parameters of the CART analysis include [40] node splitting, optimal tree, model validation, R-squared, RMSE (root mean squared error) and MSE (mean squared error), MAD (mean absolute deviation), and MAPE (mean absolute percent error), which allow alternative measures of prediction error, important for understanding model accuracy in practical terms.

Based on previous analyses related to machine risk assessment and their impacts on the organization, specified criteria for classifying subjects into the best terminal nodes were developed as predictor variables, and their thresholds define the division of data and the assignment of cases to different subgroups for prediction. These criteria and the assignment to nodes were the basis for creating a model that maps the relationships between the predictor variables and the dependent variable. Minitab software was used to analyze the regression decision tree and applied to an extended database with 489 rows. As mentioned above, the AEI response index and the predictor parameters P, S, and CM1 to CM5 characterizing the expected unit cost for one machine, one event, and the specific measure proposed were considered.

Figure 2 is a graphical representation of the dependence between the number of terminal nodes in the network and the R-squared value. R-squared is a statistical measure that expresses the degree to which the variability of the dependent variable is explained by the independent variable. In this case, the dependent variable is the number of terminal nodes, and the independent variable is R-squared. Axis $X$ shows the number of terminal nodes in the network, with the values on the axis $X$ ranging from 2 to 11. Axis $Y$ shows the value of R-squared, and the values on the axis $Y$ range from 0 to 85%. The curve in the graph shows the trend of the relationship between the number of terminal nodes and the R-squared value.

The curve has an increasing trend, indicating that the number of terminal nodes and R-squared value increases. The optimal R-squared value, which is 84.08%, is also indicated in the graph. This value has been achieved with eight terminal nodes. The graph shows that the number of terminal nodes increases; the R-squared value also increases to explain the variability in the number of terminal nodes. This indicates that as the number of terminal nodes increases, the model becomes more accurate in predicting the number of terminal nodes. The optimal value of R-squared indicates the model is most accurate in predicting the number of terminal nodes at this point. Increasing the number of terminal nodes above eight would lead to a decrease in the accuracy of the model. This shows that there is a strong dependence between the number of terminal nodes and the R-squared value.

The overall importance of the nodes in the CART tree (see Figure 3) lies in their ability to partition the dataset into predictively homogeneous groups, allowing for more accurate predictions and a better understanding of the relationships in the data. It describes the specific data in the tree nodes and the criteria for classifying subjects into the best terminal nodes within the CART model.

Figure 4 compares the relative importance of each cost measure CM1 to CM5 and parameters S and P. It shows that the costs of the measures CM4, CM1, and CM2 have a large impact on the formation of the predictive model.

Table 15. Model summary for 7 terminal nodes.

Statistics	Training	Test
R-squared	86.75%	84.08%
RMSE	0.0469	0.0514
MSE	0.0022	0.0026
MAD	0.0120	0.0137
MAPE	0.0231	0.0279

Note: RMSE—root mean squared error; MSE—mean squared error; MAD—mean absolute deviation; MAPE—mean absolute percent error.

provides quantitative measures of model accuracy, i.e., it shows the performance of the CART regression model on the training and test sets. The low values of RMSE, MSE, MAD, and MAPE indicate good prediction accuracy of the model. The potential occurrence of possible overfitting was excluded by using a statistical test to create a tree using the training group, to estimate whether a given tree needed to be changed.

The R-squared value (86.75% on the training set and 84.08% on the test set) indicates that the model explains the variability of the dependent variable relatively well. Overall, these figures show the CART model can accurately predict the dependent variable with a high degree of accuracy, and it has low prediction errors on both the training and test sets. The following

Table 16. Criteria for 7 terminal nodes.

Terminal Node	Number of HS Events	$\mathbf{A}\mathbf{E}\mathbf{I}$	Criterion	Total Cost
4	4	75%	CM1 = {78}; CM2 = {44; 88}; CM4 = {0; 50}	{122; 216}
7	70	66.43%	CM3 = {1000}; CM4 = {200; 500}	{1200; 1500}
6	147	59.85	P > 3.5; CM3 = {0; 500}; CM4 = {200; 500};	{200; 1000}
2	52	50.58%	P > 3.5; CM2 = {0; 67}; CM4 = {0; 50}	{0; 117}
3	83	50%	CM1 = {0; 25; 52}; CM2 = {44; 88}; CM4 = {0; 50}	{44; 190}
1	130	33.72%	P ≤ 3.5; CM2 = {0; 67}; CM4 = {0; 50};	{0; 117}
5	3	33.33%	P ≤ 3.5; CM3 = {0; 500}; CM4 = {200; 500}	{200; 1000}

Note: AEI—Action Effectiveness Index; P—probability of hazardous situation; CM1 to CM5—cost of measure.

describes the criteria for the seven terminal nodes.

The greatest efficiency in terms of the greatest percentage of the index value AEI is represented by terminal node 4, specifically 75%, but on the other hand, this node refers to the application of measures for HS/Es. The path to it from the root goes through node 2 and node 4, i.e., for each of the four events, it is necessary to spend CM4 (from EUR 0 to 50) + CM2 (from EUR 44 to 88) + CM1 (EUR 78). This can be interpreted as the implementation of measures M4: education—training, M2: E- STOP, and M1: LOTO—safety padlock at unit costs ranging from EUR 132 to EUR 216. This is therefore a high efficiency of the related measures at a relatively low cost. As previously mentioned, a limitation is that only four types of undesirable events (HS/E) are addressed. Another of the nodes, terminal node 7, achieves success in reducing the criticality of the event with measures M4 and M3 at corresponding costs ranging from EUR 1200 to EUR 1500. In a similar way, the unit cost was calculated for the proposed measures with respect to their effectiveness as measured by AEI for all 489 undesirable events (HS/E) considered. Thus, using the CART method, the individual HS/E events were divided into relatively homogeneous groups according to the relationship between AEI and costs CM (divided into five groups according to each measure: M1 up to M5) as well as parameters P and S. The parameters P and S were the basis for calculating the risk number R and thus naturally influence the parameter AEI. In the following, it is shown how both applied methods can be used simultaneously.

In Section 4.3, the ranking of the results of the AHP method was performed for the five investigated parameters and three different pairwise comparison matrices (Variant A, B, C, Table 14). By comparing the rankings, independently of the different variants, it was found there are four basic types of negative effects (namely, E2.1, E2.2, E3, and E5). Once they occur, this will imply the largest total costs (losses) related to HS.

In the following procedure, the 489 HS/E identified events were therefore reduced to four types of significant undesirable events.

The resulting reduced database contained 188 HS/E that corresponded to any of the types of undesirable events mentioned above. The CART method was applied to this database. The results of the Action Effectiveness Index were again used as predictors P, S, and CM1 to CM5. The results of the regression decision tree are presented in Figure 5.

The resulting parameters of the model can be described as follows. The R-squared values (91.04% on the training set and 84.08% on the test set) indicate that the model explains the variability of the dependent variable relatively well. Overall, these figures show that the CART model can accurately predict the dependent variable with a high degree of accuracy, and it has low prediction errors on both the training and test sets. The following

Table 17. Model summary for 3 terminal nodes.

Statistics	Training	Test
R-squared	91.04%	89.97%
RMSE	0.0402	0.0425
MSE	0.0016	0.0018
MAD	0.0245	0.0250
MAPE	0.0444	0.0451

Note: RMSE—root mean squared error; MSE—mean squared error; MAD—mean absolute deviation; MAPE—mean absolute percent error.

describes the criteria for the three terminal nodes.

The final evaluation of the decision tree is presented in summary in the following

Table 18. Criteria for 3 terminal nodes.

Terminal Node	Reduction in the Criticality of the Event	Events in the Node	$\mathbf{H}\mathbf{S}/\mathbf{E}$	$\mathbf{O}$	Priorities under Variant A	Priorities under Variant B	Priorities under Variant C	$\mathbf{M}\mathbf{i}$ (i = 1 to 5)	$\mathrm{Unit}\mathrm{Cos}\mathrm{ts}\mathbf{C}\mathbf{M}$ in Eur	Total Cost in EUR
3	64.54%	100	E2.2	3	3	4	2	M4, M5	1500	4500
			E3	46	4	1	3	M1, M2, M3, M4	1366	62,836
			E5	51	1	2	1	M1, M2, M4	666	33,966
2	56.66%	42	E2.1	42	2	3	4	M4, M5	1500	63,000
1	33.33%	46	E3	46	4	1	3	M1, M2, M3	619	28,474

Note: HS/E—hazardous situations analyzed and their effects; O—number of HS occurrences; Mi—measures; CM—cost of measures.

. It can be concluded which undesirable events can be expected to result in the greatest cost savings, i.e., both the cost of the event (losses from undesirable events) and the cost of the measures that will most reduce the existing risk. The presented results are obtained by the AHP method for three different variants, A, B, and C, of the pairwise comparisons matrices. The table shows the order of importance of the four events, with the highest priority for all three variants. These are the 188 events selected for E2.1, E2.2, E3, and E5. Three terminal nodes were obtained from the CART technique applied to the set of all events of these types. Each of them represents the percentage of expected criticality reduction for each event. The table also lists the measures that are expected to result in each criticality reduction. The unit and total costs that will be incurred by implementing the prescribed combination of measures are also included in the table.

From the table, is clear that the event with the highest efficiency has the designation E5 and a frequency of 51. An approximately 65% reduction in criticality for all events of this type requires the implementation of a combination of measures M1, M2, and M4, with a unit cost of EUR 666 and a total cost of EUR 33,966. In other words, up to about a 65% reduction in criticality for the 51 most serious events, as assessed by all three variants, can be achieved with the measures M1, M2, and M4 at the lowest possible unit cost of EUR 666. With even lower unit costs, it is possible to implement the measures M1, M2, and M3 to reduce 46 E3 events by one-third. In the same way, it is possible to estimate the costs of other events. If the management of the organization decides to reduce all 188 most critical events, it would need to cover costs of about EUR 192,776.

5. Discussion

The presented five-parameter machinery safety analysis appears to be an effective tool to perform a comprehensive risk assessment analysis of machinery. The proposed procedure allows for obtaining a ranking of the causes and consequences of undesirable events, which include several types of information (P—probability of hazardous situation; PND—probability of non-detection of HS; S—severity of harm; EC—expected cost; O—number of HS occurrences). The approach, based on the implementation of the AHP method for multi-attribute analysis, provides a framework in which the selection of the most critical undesirable events can be performed using different relevant characteristics. The AHP method helps safety analysts to work in a systematic and analytical manner, addressing each aspect of the events in a stepwise hierarchy. The qualitative and subjective judgments of multiple individuals can be incorporated into the prioritization process. Using a series of paired evaluations, the AHP can obtain a direct (quantitative) assessment based on intangible (qualitative) criteria. Specifically, the problem is how to assign scores based on tables listing vague and unreliable verbal ratings. However, the great advantage is that if reliable quantitative judgments are available for some criteria, they can easily be included in the AHP analysis. This possibility means that proposed procedures can replace, complement, or be integrated into a more comprehensive approach to safety studies.

Using the AHP method and the five basic parameters, a way to estimate the most critical undesirable events (HS/E) is shown. Risk management is based on EC and is applied to the whole group of machinery also based on the estimation of the parameter O. The result of such an approach is a procedure that allows for designing an effective plan of action to minimize the expected losses due to undesirable events in an organization.

Another way to increase the effectiveness of machinery safety management is to maximize the effect of the measures taken at an effective cost. The CART technique was applied for this purpose. The parameter AEI was selected as a response. This evaluates the effectiveness of the proposed or implemented corrective actions in reducing the risk of HS. As predictors, the expected unit cost for each machine (cause or effects) was applied. In the case study, we verified that it is possible to develop a proposed model that can predict which group of measures, at what expected unit cost, will reduce the criticality of a particular undesirable event to what extent.

The costs associated with the fulfilment of existing hazards in an organization realistically engaged in safety management can in principle be divided into two groups. The first is impact costs, i.e., the losses that can potentially arise in an organization from the realization of undesirable events (the fulfilment of hazards) despite measures. These costs are those that cannot, by their own nature, be defined in an exact way. For this type of cost, it is appropriate to use the AHP method, which is based on the subjective assessments of experts and OHS management. Another type of costs are those for specific actions that the organization undertakes as part of its safety management. These costs are generally known and can be defined in an exact way. For these costs, efficiency of expenditure is important. Often, measures are implemented without mature consideration, which unnecessarily increases the cost of implementation. The AEI index was used as a criterion for the effectiveness of the implemented measures. A subsequent decision tree analysis based on the empirically estimated costs over time and the AEI index helped prioritization in view of the greatest efficiency at the lowest estimated cost.

Estimating the investigated quantities exactly is a rather complex problem. Limitations may result mainly from the different level of experience and estimates of the experts consulted, as well as from the approach of the organization’s management to the issue of MSDM.

6. Conclusions

Machinery safety should be a priority for the management of every operation in an organization. As significant resources are usually required to ensure safety or reduce risks, it is appropriate to carefully consider the priority of the relevant measures as well as the expected costs of their implementation. The presented study showed, based on existing safety analyses, in a relatively simple way, those undesirable events, the fulfilment of which entails the greatest risk of an often-unexpected increase in costs (losses). Although it is almost impossible to define these costs in an exact way, by using the AHP method, it has been demonstrated that it is possible to prioritize them using pairwise comparisons.

The case study presents the results of the MSRD analysis on 124 machines in a case study organization. The analysis identified 5260 HSs and the related risks. From these, 489 critical risks were selected in collaboration with the OHS management team, from which 10 basic HS types were identified after some standardization. All the 489 critical risks were expanded by the PND parameter. As each HS can cause several different effects, 15 different HS/E types were identified from the 10 HS types. Each of them was then assigned the parameter EC using pairwise comparison and the parameter O based on the frequency of occurrences. Using AHP decision making applied to all five parameters (P, PND, S, EC, and O), the priority of each effect was determined. The study also relied on pairwise comparisons at the first level of the hierarchy, designated as matrix A, which was developed by a group of experts and OHS management. Since it was a subjective evaluation, the experts were invited to compile other variants of the pairwise comparison on the first level of the hierarchy, marked as B and C. The resulting AHP analyses showed that from the point of view of all three considered options, A, B, and C, four HS/Es were identified (albeit in different orders) that were prioritized for all considered alternatives. Priority implementation of corrective actions to reduce or even eliminate the four HS/Es for each piece of machinery represents the greatest prerequisite for minimizing the expected total losses in terms of the five criteria considered.

The study has shown that despite the extensive analysis of MSRD, with appropriate standardization, the resulting number of HS/E types may be far from large. In fact, the different hazardous situations and their effects are very similar to each other, except for negligible variations. The advantage of the presented AHP method is the fact that some parameters of the decision criteria do not have to be determined exactly, but it is sufficient to estimate their mutual priorities by pairwise comparisons.

Increasing the efficiency of machinery safety assurance can also be achieved by efficiently spending resources on individual proposed measures aimed at reducing the risk of undesirable events. The CART tool was used to determine how the cost of individual measures affects the degree of criticality reduction for each type of undesirable event. Overall, the CART model in this analysis provides an accurate and reliable technique for prediction, indicating its suitability for the data analyzed even when expanding operations to include similar machinery.

The combination of the AHP method and the CART technique can be used to extract and estimate costs for those measures that will most effectively reduce both the unit and total cost of risk within the operations of a manufacturing organization.

The study was based on the individual measures resulting from the MSRD analysis, for which the experts then estimated the actual minimum and maximum costs associated with each of these measures. Subsequently, MSDM support procedures were applied. The AEI was calculated for all 489 HSs and was used to assess the effectiveness of the corrective measures in reducing the risk of R_M. The CART decision tree was applied to the thus added data. The result was a categorization of each HS/E into groups according to the relationship between the AEI and the estimated actual CM costs. For example, terminal node 7 identified 70 HS events that, using CM3 and CM4 measures, will achieve an AEI of 66.43% at an expected actual cost of between EUR 1200 and 1500. Similarly, other HSs are categorized using the other 6 terminal nodes.

Finally, by combining both approaches, 188 HS/Es of four priority types determined using the AHP method were selected, on which the CART method was implemented. The resulting three terminal nodes showed how to achieve the best possible AEI index for each HS/E group at the expected current cost of the relevant measures, totaling EUR 192,776 (see Table 18). The resulting decision tree therefore identified the measures that require the minimum cost, on the one hand, as well as substantially reduce the risk of those HS/Es whose fulfilment would result in the greatest loss.