Lockout and Tagout in a Manufacturing Setting from a Situation Awareness Perspective

: Applying lockouts during maintenance is intended to avoid accidental energy release, whereas tagging them out keeps employees aware of what is going on with the machine. In spite of regulations, serious accidents continue to occur due to lapses during lockout and tagout (LOTO) applications. Few studies have examined LOTO e ﬀ ectiveness from a user perspective. This article studies LOTO processes at a manufacturing organization from a situation awareness (SA) perspective. Technicians and machine operators were interviewed, a focus group discussion was conducted, and operators were observed. Qualitative content analysis revealed perceptual, comprehension and projection challenges associated with di ﬀ erent phases of LOTO applications. The ﬁndings can help lockout / tagout device manufacturers and organizations that apply LOTO to achieve maximum protection.


Introduction
The past decades have seen hundreds of maintenance workers suffering fatal injuries while performing their work. Almost half of the fatalities occurred during planned preventive maintenance operations [1]. Those who are at high risk, installation, maintenance, and repair workers report an overall rate of 9.4 fatalities per 100,000 workers, compared to 2.6 for production workers and 3.6 for all workers [2]. The United States (US) Occupational Safety and Health Administration (OSHA) and the US National Institute for Occupational Safety and Health (NIOSH) find significant risk results from exposure to hazardous energy during maintenance. Studies [3] have reported the most common mechanisms of such injuries: being caught in or between parts of equipment, electrocution, and being struck by or against objects. Distinctive maintenance scenarios of these injuries include cleaning mixers, cleaning conveyors, and installing or disassembling electrical equipment. Given the risks, special procedures are necessary to protect workers.
In order to safeguard employees from the unexpected release of hazardous energy or energization from equipment during service or maintenance activities, lockout/tagout (LOTO) safety procedures [4] are used in industries. These procedures are to ensure that harmful machines under maintenance are properly tagged and shut off until the completion of maintenance work and to verify that the hazardous energy has been controlled. LOTO is considered a positive restraint [4] because a key is required to unlock, whereas, tagout is a warning device to warn employees not to reenergize the energy sources [4], thereby improving the awareness of potential hazards near equipment and machinery.

1.
To critically assess the LOTO system of a manufacturing organization through the perspective of a user and determine SA requirements for the LOTO applications.

2.
To classify the issues based on SA levels and components of the LOTO system.

3.
To prioritize the SA issues and recommend ways to rectify them.
In this study, we sought a diverse sample of participants to yield a maximally heterogeneous sample, using stratified purposeful sampling [27]. Although many different workers (e.g., team leaders, planners) are affected by LOTO, we identified the key decision makers in the LOTO tasks: machine technicians and machine operators. Technicians are not specialized in different production technologies but rather in technical disciplines such as electrical and mechanical. Machine operators are multi-skilled; they are trained to handle several production technologies. All the technicians and operators were males; the organization did not have female employees in these positions at the time of the study. Two authors were involved in data collection and analysis. Author 1 was an employee in the organization at the managerial level. Table 1 shows details of the sample participants. In this study, we sought a diverse sample of participants to yield a maximally heterogeneous sample, using stratified purposeful sampling [27]. Although many different workers (e.g., team leaders, planners) are affected by LOTO, we identified the key decision makers in the LOTO tasks: machine technicians and machine operators. Technicians are not specialized in different production technologies but rather in technical disciplines such as electrical and mechanical. Machine operators are multi-skilled; they are trained to handle several production technologies. All the technicians and operators were males; the organization did not have female employees in these positions at the time of the study. Two authors were involved in data collection and analysis. Author 1 was an employee in the organization at the managerial level. Table 1 shows details of the sample participants.

Interviews
We used unstructured and structured interviews for data collection. Participants were first educated about the research purpose of the interviews in a verbal introduction prior to the interviews. The unstructured interviews gave participants the opportunity to make open comments: for example, what goes on during LOTO applications, and what they are looking for (i.e., SA information required). The comments made during unstructured interviews led to the questions to ask during structured interviews. During the initial part of the interviews, participants were asked about their maintenance goals, the sub goals, and the decisions required in the attainment of those

Interviews
We used unstructured and structured interviews for data collection. Participants were first educated about the research purpose of the interviews in a verbal introduction prior to the interviews. The unstructured interviews gave participants the opportunity to make open comments: for example, what goes on during LOTO applications, and what they are looking for (i.e., SA information required). The comments made during unstructured interviews led to the questions to ask during structured interviews. During the initial part of the interviews, participants were asked about their maintenance goals, the sub goals, and the decisions required in the attainment of those goals. The next task was to identify specific SA information requirements to make those decisions. In structured interviews, we applied GDTA [18] to determine a situationally-based maintenance scenario describing performance in the absence of LOTO. It is pertinent to mention that the analysis was based on operators and technicians' goals in the maintenance scenario, rather than on specific support systems (i.e., LOTO). The imagined absence of an information system stimulated participants to think of what information is required for the decision-making tasks they had just listed. During the structured interviews, planned questions addressed the second research objective: the affected SA level (level 1 to 3) and the respective LOTO system component. Participants were asked to focus on an event with instructions directing them to a particular instance. Questions included: 'What is the most difficult task with LOTO?'; 'How do you attempt to handle the problem?'; 'What would help you manage the problem?'; 'If LOTO can be improved immediately, what you want to be fixed first?' Transcribing was required because the substantive content was the focus of the analysis [28].

Observations
Participants were made aware that we make observations for research purposes. LOTO application was observed during machine service and repair work. For example, the technicians were observed trying to follow the content on a tag and attempting to insert locks into heavily restricted locations. Although such observations validated the difficulties that were stated, we did not consider participative observation [29] to be a major data collection method in this study, as we observed only a few cases. On a continuum of complete observer to complete participant, involvement in this research can be considered peripheral [30]. We consider that Author 1 s regular presence in the organization minimized the Hawthorne effect [31] and supported objective observation, even in an overt role [32].

Focus Group Discussion
A focus group was arranged for participants who mentioned risk with LOTO. Focus groups can be viewed as a stage where participants tell, negotiate, and reformulate their 'self-narratives' [33]. The group was heterogeneous, representing different production departments, different technologies, and different years of experience. Preparatory work was carried out by developing topic guides and selecting stimulus materials based on the points requiring more reflection. The location of the focus group and the associations that it has for the participants are likely to have an important impact [34]; therefore, the focus group discussion was held in the organization's training room, which was a very familiar place for all the participants. All the participants knew each other, as they were employees in the same organization. As the facilitator and moderator, Author 1 began with the discussion points; then, he let the team discuss them and helped maintain the focus in the discussion without harming the dynamics. Participants were engaged not just in presenting their own narratives but in supporting and challenging others' narratives. We did not attempt to categorize individuals in terms of their views, and we attempted to contain the discussion within the frame of our study.

Qualitative Content Analysis
The qualitative data collected was analyzed to address the research questions and to understand what participants meant to say. The initial step of qualitative content analysis (QCA) [35] is to get a better interpretation of data followed by dividing the text into smaller parts, i.e., the "meaning units". These "meaning units" are further categorized as codes and subcategories [28,36].
We closely examined the qualitative data obtained from interviews and the focus group discussion to segregate what was relevant to the research objectives. Then, we delineated the meaning units related to the research objectives. One major challenge was to filter the meaning units referring to SA. The three-level SA model is conceptually similar to human information processing models [16,37,38]. We had to avoid creating a traditional information processing model [39], whilst carefully selecting the meaning units reflecting specific characteristics of SA. Criteria used for selection were dynamic aspects of the situation (not static information), situations requiring the knowledge of three levels of SA [14], situations requiring both top-down and bottom-up information processing (not merely linear information processing) [40], and the active (not passive) nature of the person who is seeking information [17]. Then, the units were condensed to shorter versions by determining the underlying meanings.
From the meaning units, we developed codes through an iterative process involving reading, reflection, and rereading. Following the coding, we determined subcategories and main categories.
Subcategories summarize what is said, and main categories are what the study wants to answer. Therefore, subcategories are data-driven (in our case, based on the interview and observation records), and main categories are concept-driven. As per our research objectives, subcategories can also be identified as causal factors for the difficulties identified in the meaning units. Whenever a pertinent theme was identified (that is, mentioned by at least two participants), we added it as a new subcategory. Our intention was to identify two dimensions: the SA level affected and the respective component in the LOTO system. These became our two main categories. Figure 2 illustrates the data collection and analysis method. Each category had a definition, description, and decision rule; decision rules ensure that categories are mutually exclusive (see Table 2).
This coding frame was examined for consistency and validity. The authors conducted independent pilot coding using the first version of the coding frame, categorizing the same set of transcripts. Coding consistency was 80% across the main categories and 70% across the subcategories during the pilot version, and 90% across the main categories and 80% across the subcategories for the final version. Since we had two main categories, we summed the content under two main categories when calculating consistency. One-third of the transcripts were again categorized by both authors during main coding. As per the definition of validity, the coding frame will not adequately describe the qualitative material if coding frequencies are high for residual categories; this requires the introduction of additional subcategories [41]. However, through this exercise, we learnt that certain subcategories became residual for reasons other than the inadequacy of the coding frame. Provided that the coding frame is well evaluated for consistency, content that is mostly spoken can still denote frequency. In fact, in this study, the amount of distinct content under each category, i.e., the coding frequency, was considered as the variable that determined the priority of concerns in terms of SA level affected and the respective LOTO component. From the meaning units, we developed codes through an iterative process involving reading, reflection, and rereading. Following the coding, we determined subcategories and main categories. Subcategories summarize what is said, and main categories are what the study wants to answer. Therefore, subcategories are data-driven (in our case, based on the interview and observation records), and main categories are concept-driven. As per our research objectives, subcategories can also be identified as causal factors for the difficulties identified in the meaning units. Whenever a pertinent theme was identified (that is, mentioned by at least two participants), we added it as a new subcategory. Our intention was to identify two dimensions: the SA level affected and the respective component in the LOTO system. These became our two main categories. Figure 2 illustrates the data collection and analysis method. Each category had a definition, description, and decision rule; decision rules ensure that categories are mutually exclusive (see Table 2).

Results and Discussion
After we removed what was not relevant to this study (following the SA criteria explained above), we were left with 109 meaning units. We further reduced data by eliminating statements with similar meanings. Therefore, the meaning units were distinct. In other words, we did not depend on the total frequency with which a certain meaning unit was mentioned; rather, we considered the number of units with distinct meanings. This prevented us from focusing on only a few major issues. However, we were not always certain whether participants were referring to the same incident multiple times and wanted to avoid repetition. Ultimately, we identified 27 (N = 27) distinct meaning units from the qualitative contents of interviews and the focus group.

SA Requirements
Following the general steps of GDTA, we identified users' major goals, sub goals, decision-making requirements for the sub goals, and SA information requirements. The major goals of both maintenance technicians and operators during a maintenance scenario are twofold: accomplishing the task correctly (maintenance/production) on time and ensuring the safety of the equipment, themselves, and affected employees. Our focus was on the SA requirements of LOTO, specifically, its major purpose of helping workers comprehend the environment and foresee risks to avoid accidents. Therefore, we excluded drilling down to the subcategories of the tasks under accomplishing the maintenance/production task; we continued analysis for the safety component only. In order to ensure employee and equipment safety, we identified four sub goals: communicating with other employees, assessing equipment status, assessing the risk of reactivation, and assessing the conditions for reactivation. Then, we determined the decision-making requirements for each of the sub goals. Finally, we identified the data required (Level 1 SA), the higher-level information on the significance of the data (Level 2 SA), and the projection of future events (Level 3 SA) for each of those sub goals (see Figure 3). and assessing the conditions for reactivation. Then, we determined the decision-making requirements for each of the sub goals. Finally, we identified the data required (Level 1 SA), the higher-level information on the significance of the data (Level 2 SA), and the projection of future events (Level 3 SA) for each of those sub goals (see Figure 3). The GDTA included results from the review of current documents as well. In addition to the existing LOTO procedures at the organization, we looked at three important documents: OSHA 3120 2002 (revised) [4]: Control of Hazardous Energy; OSHA Standard 1910.147: The control of hazardous energy; and OSHA 1910.147 Appendix A: Typical minimal lockout procedure. Since the organization studied is based in the US, we referred to applicable OSHA standards in the US. In the GDTA, we included all the dynamic information requirements mentioned in the above OSHA documents. Notably, the input from interviews provided a great deal of dynamic information on topics not mentioned in the OSHA documents; for example, these included a possible extension of work, the availability of spare parts, details on the production work-in-progress, equipment modifications, other equipment affected by blocks, the detection of accidental activation, time taken for reactivation, etc. Overall, a review of sub goals indicated that information must be communicated between technicians, operators, and other employees. SA is not only a concern of the technicians and operators who directly work with the equipment. Other employees who work around it, as well as anyone who has authority over or a connection with task accomplishment (supervisors, technicians, planners, operators of adjacent operations) must have an understanding of what is going on with the machine under maintenance. In this way, LOTO must facilitate team synchrony by informing all the stakeholders of the status of the maintenance task, thus assisting them to achieve a common goal. LOTO must also standardize the energy isolation and provide good communication with other and OSHA 1910.147 Appendix A: Typical minimal lockout procedure. Since the organization studied is based in the US, we referred to applicable OSHA standards in the US. In the GDTA, we included all the dynamic information requirements mentioned in the above OSHA documents. Notably, the input from interviews provided a great deal of dynamic information on topics not mentioned in the OSHA documents; for example, these included a possible extension of work, the availability of spare parts, details on the production work-in-progress, equipment modifications, other equipment affected by blocks, the detection of accidental activation, time taken for reactivation, etc.
Overall, a review of sub goals indicated that information must be communicated between technicians, operators, and other employees. SA is not only a concern of the technicians and operators who directly work with the equipment. Other employees who work around it, as well as anyone who has authority over or a connection with task accomplishment (supervisors, technicians, planners, operators of adjacent operations) must have an understanding of what is going on with the machine under maintenance. In this way, LOTO must facilitate team synchrony by informing all the stakeholders of the status of the maintenance task, thus assisting them to achieve a common goal. LOTO must also standardize the energy isolation and provide good communication with other areas; for example, these areas include tactical planning and the provisioning of assets, particularly during shift changeovers. Importantly, these elements keep changing, as they are subjected to dynamic situations, mainly with the progression of the maintenance work itself, changing of teams and individuals, and changes in operational priorities (e.g., tactical changes in production plan). In a longer-term perspective, lockout practices require adaptation to ever-changing dynamics of machinery and processes, tighter schedules, and regulatory compliances.
The SA requirements that we identified for LOTO are consistent with the findings in previous studies on SA requirements, generally in maintenance. With respect to maintenance fieldwork, previous studies [17,21,22] identify four important elements of SA:
Comprehending the environment and their risks to avoid accidents are SA requirements; 3.
Maintaining team synchrony by collaborating and coordinating tasks to achieve a common goal; 4.
Maintaining a good corporate environment, standardized work routine and terminology, and communicating with other supporting areas.
Although goal-directed task analysis (GDTA) implies a focus on goal-driven cognitive processes (top-down), our use of it facilitated the identification of the demands in data-driven processes (bottom-up) by asking the participants to imagine different scenarios. In fact, the critical importance of the GDTA technique was its ability to elicit user experience, beyond what was mentioned in the documentation. As a result, GDTA was able to detect the demands for both goal-driven and data-driven decisions while applying LOTO under changing environments. With this, we could establish that LOTO-related issues are subject to data-driven and goal-driven dynamic conditions. Goal-driven and data-driven iterative processes to perceive dynamic information and match mental patterns is a major character of SA [14] (this data-driven/goal-driven process should not be confused with the data-driven/concept-driven categories in QCA). Therefore, a framework comprising the three levels of SA is useful to explore how well LOTO accomplishes its key intent of making employees aware of what is going on with equipment in a dynamic maintenance environment.

SA Issues
In the QCA, the coding frame itself can be considered the main result. The categories and the interrelations between the categories serve as discussion points. In our study, we defined two main categories: the SA levels affected and the respective components of the LOTO system. By coding the frequencies with which those main categories were mentioned, we ascertained their priority. For the first main category 'affected level of SA', we determined issues of perception, comprehension, and projection. For the second main category, 'affected LOTO system components', we looked at decommissioning, locking, tagging, and recommissioning. The coding frame with the content provided the basis for the first objective: critically assessing the LOTO system for SA issues. Table 3 shows three examples, meaning units representing each level of SA (perception, comprehension, projection) with their subcategories and categories. Table A1 presents the complete coding frame with distinct meaning units (N = 27) and the developed subcategories and categories.
In summary, regardless of the main category, lack of integration of information (n = 6) has the highest coding frequency of the subcategories. This is followed by poorly standardized information (n = 4), information not being made explicit (n = 4), and a lack of understanding surrounding the context (n = 4) (see Figure 4). In summary, regardless of the main category, lack of integration of information (n = 6) has the highest coding frequency of the subcategories. This is followed by poorly standardized information (n = 4), information not being made explicit (n = 4), and a lack of understanding surrounding the context (n = 4) (see Figure 4). These issues can affect different levels in SA; for example, a lack of integration of information makes comprehension difficult. Therefore, the second research objective was to classify the issues based on the SA levels affected and the respective LOTO system components.

SA Levels Affected
The subcategories (presented in Figure 4) are data-driven, but major categories are concept-driven. Therefore, the outcome of the meaning units for our two major categories served our second objective. The first part of the second objective refers to classifications for the SA levels affected. Meaning units (N = 27) were categorized under perception (n = 4, 15%), comprehension (n = 12, 44%), and projection (n = 11, 41%) (see Figure 5a). Overall, comprehension and projection appear to be highly affected, but the perception of information is not. This finding is different in other domains; in a study of aviation, for example, 76% of the pilot errors were traced to problems in perception, and 20% were associated with comprehension [14]. However, it should be noted that the aviation study used retrospective analysis; we present how users perceive the LOTO ability to support SA. Despite the difficulties, users might be able to comprehend the situation and project future events by exerting more cognitive effort. This may not be the result in a retrospective analysis.
The three SA levels are not linear, but rather ascending [40]. SA is not a process; it represents the operator's mental model of the state of the environment [14]. Therefore, it is possible that some of the These issues can affect different levels in SA; for example, a lack of integration of information makes comprehension difficult. Therefore, the second research objective was to classify the issues based on the SA levels affected and the respective LOTO system components.

SA Levels Affected
The subcategories (presented in Figure 4) are data-driven, but major categories are concept-driven. Therefore, the outcome of the meaning units for our two major categories served our second objective. The first part of the second objective refers to classifications for the SA levels affected. Meaning units (N = 27) were categorized under perception (n = 4, 15%), comprehension (n = 12, 44%), and projection (n = 11, 41%) (see Figure 5a). Overall, comprehension and projection appear to be highly affected, but the perception of information is not. This finding is different in other domains; in a study of aviation, for example, 76% of the pilot errors were traced to problems in perception, and 20% were associated with comprehension [14]. However, it should be noted that the aviation study used retrospective analysis; we present how users perceive the LOTO ability to support SA. Despite the difficulties, users might be able to comprehend the situation and project future events by exerting more cognitive effort. This may not be the result in a retrospective analysis.

LOTO System Components Affected
The second part of the second objective was to identify and understand the affected LOTO system component. This understanding is vital for prioritizing interventions. The locking component of the LOTO system has the largest number (n = 11, 41%) of meaning units referring to a lack of SA, when all the SA levels are taken into consideration. Decommissioning (n = 9, 33%), tagging (n = 6, 22%), and recommissioning (n = 1, 4%) follow sequentially (see Figure 5b).
This result was not anticipated, as we built our justification for the study mainly on the ability of tagging to aid awareness of what is going on. Interestingly, decommissioning and locking surfaced as SA issues. The greatest number of SA requirements were identified for assessing the equipment status; here, the LOTO system components of decommissioning and locking appear more relevant. A number of SA requirements were identified with the sub goal of communication between employees (more related to tagging), while QCA reveals many SA-related issues in tagging. Notably, despite mentioning a rather high number of SA requirements for recommissioning, participants did not highlight many SA-related issues with the current LOTO system during the re-energizing phase. Only one specific incident was bought to our attention; in this case, a lithography machine was severely damaged when the machine was put back into operation, as a tool had been left inside. Figure 6 presents the coding frequency for different LOTO components with the respective levels of SA affected. The figure shows the sequence of decommissioning, locking, tagging, and recommissioning with the SA levels affected at each stage. This visualization sheds light on the issues associated with different SA levels throughout the LOTO procedure. The three SA levels are not linear, but rather ascending [40]. SA is not a process; it represents the operator's mental model of the state of the environment [14]. Therefore, it is possible that some of the SA requirements will not exactly fit into a single SA level. However, our QCA exercise enabled a disclosure of the main issues, which were categorized into different SA levels, making it possible to design specific SA interventions particular to those levels. We discuss this further under recommendations.

LOTO System Components Affected
The second part of the second objective was to identify and understand the affected LOTO system component. This understanding is vital for prioritizing interventions. The locking component of the LOTO system has the largest number (n = 11, 41%) of meaning units referring to a lack of SA, when all the SA levels are taken into consideration. Decommissioning (n = 9, 33%), tagging (n = 6, 22%), and recommissioning (n = 1, 4%) follow sequentially (see Figure 5b).
This result was not anticipated, as we built our justification for the study mainly on the ability of tagging to aid awareness of what is going on. Interestingly, decommissioning and locking surfaced as SA issues. The greatest number of SA requirements were identified for assessing the equipment status; here, the LOTO system components of decommissioning and locking appear more relevant. A number of SA requirements were identified with the sub goal of communication between employees (more related to tagging), while QCA reveals many SA-related issues in tagging. Notably, despite mentioning a rather high number of SA requirements for recommissioning, participants did not highlight many SA-related issues with the current LOTO system during the re-energizing phase. Only one specific incident was bought to our attention; in this case, a lithography machine was severely damaged when the machine was put back into operation, as a tool had been left inside.  The distribution of coding frequencies for different LOTO components with respective levels of SA affected gives insight into the completeness of the state of the employees' knowledge [14] at each LOTO stage. Although there were no perceptual challenges at the decommissioning stage, comprehension of that information seemed to be challenging. In contrast, what was challenged most at the locking stage was the projection of future status. Tagging was equally problematic for perception and comprehension. Finally, recommissioning was the least challenged component; here, only perception seemed to be challenging. In general, perceptual challenges were the least often found in this study. They became more prominent at later stages: tagging and recommissioning.

SA Level Affected and Respective LOTO System Component
In the section on SA issues (Section 4.2), we discussed factors affecting SA in general. Above, we presented the different SA levels and LOTO components affected. We now look at the causal factors that are specific to those effects. First, we establish the key difference between two types of assessments. More specifically, the assessment of the activities performed in SA is different from the assessment of the result of these activities. If the objective is only to assess SA (whether or not one is aware of a situation), then the manner in which one becomes aware of a situation is not important [17]. However, since our focus was to investigate what hinders each SA level, we also looked at difficulties during the process of gaining SA. Subcategories of the SA levels affected can reflect causal factors that hinder those levels. Figure 7 shows the causal factors on each SA level. The distribution of coding frequencies for different LOTO components with respective levels of SA affected gives insight into the completeness of the state of the employees' knowledge [14] at each LOTO stage. Although there were no perceptual challenges at the decommissioning stage, comprehension of that information seemed to be challenging. In contrast, what was challenged most at the locking stage was the projection of future status. Tagging was equally problematic for perception and comprehension. Finally, recommissioning was the least challenged component; here, only perception seemed to be challenging. In general, perceptual challenges were the least often found in this study. They became more prominent at later stages: tagging and recommissioning.
In the section on SA issues (Section 4.2), we discussed factors affecting SA in general. Above, we presented the different SA levels and LOTO components affected. We now look at the causal factors that are specific to those effects. First, we establish the key difference between two types of assessments. More specifically, the assessment of the activities performed in SA is different from the assessment of the result of these activities. If the objective is only to assess SA (whether or not one is aware of a situation), then the manner in which one becomes aware of a situation is not important [17]. However, since our focus was to investigate what hinders each SA level, we also looked at difficulties during the process of gaining SA. Subcategories of the SA levels affected can reflect causal factors that hinder those levels. Figure 7 shows the causal factors on each SA level.
There are a few notable characteristics of the SA levels and their causal factors (Figure 7). The integration of information is shown to be a major requirement for developing Level 2 SA [14]. However, four out of six 'lack of integration of information' cases affect SA up to the projection level. For example, in the absence of integrated information (number of locks installed, when they are installed, size of locks to use, etc.), it is difficult to predict the aftermath of installing locks on the disconnect switch; this ultimately hinders the closure of the electrical panel door, opening more risk opportunities (see Figure 8).
In the section on SA issues (Section 4.2), we discussed factors affecting SA in general. Above, we presented the different SA levels and LOTO components affected. We now look at the causal factors that are specific to those effects. First, we establish the key difference between two types of assessments. More specifically, the assessment of the activities performed in SA is different from the assessment of the result of these activities. If the objective is only to assess SA (whether or not one is aware of a situation), then the manner in which one becomes aware of a situation is not important [17]. However, since our focus was to investigate what hinders each SA level, we also looked at difficulties during the process of gaining SA. Subcategories of the SA levels affected can reflect causal factors that hinder those levels. Figure 7 shows the causal factors on each SA level.  There are a few notable characteristics of the SA levels and their causal factors (Figure 7). The integration of information is shown to be a major requirement for developing Level 2 SA [14]. However, four out of six 'lack of integration of information' cases affect SA up to the projection level. For example, in the absence of integrated information (number of locks installed, when they are installed, size of locks to use, etc.), it is difficult to predict the aftermath of installing locks on the disconnect switch; this ultimately hinders the closure of the electrical panel door, opening more risk opportunities (see Figure 8). Second, although lack of understanding of the context seems to be more relevant to loss of Level 2 SA (comprehension), situations such as lack of understanding about the circumstances of blocking (instead of locks) affect the projection of consequences, and this ultimately hinders some other operations. Long-term memory stores in the form of schemata and mental models can assist in making projections of the risks even with incomplete information [14]. However, novel situations caused by frequent alterations demand that projections be made with limited working memory. Designs that clearly reflect the system's alterations and associated risks can assist in the development of a correct mental model. Figure 9 shows the causal factors for each LOTO component; the analysis leads to two major discoveries. First, the lack of integration of information is a major issue at the locking stage. This suggests that care should be taken in the selection of locks and allowing for blocks. Second, a lack of understanding of the context is a major issue during decommissioning. Since there are challenges beyond what is covered in the documented procedures, there is a need for a more comprehensive assessment of scenarios that arise during decommissioning. Second, although lack of understanding of the context seems to be more relevant to loss of Level 2 SA (comprehension), situations such as lack of understanding about the circumstances of blocking (instead of locks) affect the projection of consequences, and this ultimately hinders some other operations. Long-term memory stores in the form of schemata and mental models can assist in making projections of the risks even with incomplete information [14]. However, novel situations caused by frequent alterations demand that projections be made with limited working memory. Designs that clearly reflect the system's alterations and associated risks can assist in the development of a correct mental model. Figure 9 shows the causal factors for each LOTO component; the analysis leads to two major discoveries. First, the lack of integration of information is a major issue at the locking stage. This suggests that care should be taken in the selection of locks and allowing for blocks. Second, a lack of understanding of the context is a major issue during decommissioning. Since there are challenges beyond what is covered in the documented procedures, there is a need for a more comprehensive assessment of scenarios that arise during decommissioning.
Overall, Level 2 SA (comprehension) is challenged, predominantly at decommissioning. Comprehension requires putting together the knowledge elements of Level 1 to form patterns (gestalt), which enables forming a holistic picture of the environment [40]. At decommissioning, workers have to deal with disparate data, including interconnections between energy sources, salience of their presence, multiple work instructions, illustrations that differ from alterations, documents with different standards, etc. Meaningful integration of these disparate data, filtered through their relevance to the goal of safe energy isolation, yields safe decommissioning.
(instead of locks) affect the projection of consequences, and this ultimately hinders some other operations. Long-term memory stores in the form of schemata and mental models can assist in making projections of the risks even with incomplete information [14]. However, novel situations caused by frequent alterations demand that projections be made with limited working memory. Designs that clearly reflect the system's alterations and associated risks can assist in the development of a correct mental model. Figure 9 shows the causal factors for each LOTO component; the analysis leads to two major discoveries. First, the lack of integration of information is a major issue at the locking stage. This suggests that care should be taken in the selection of locks and allowing for blocks. Second, a lack of understanding of the context is a major issue during decommissioning. Since there are challenges beyond what is covered in the documented procedures, there is a need for a more comprehensive assessment of scenarios that arise during decommissioning.  At first glance, LOTO seems to be in a static state from the time it is installed until it is removed. A major question addressed by this study was whether SA really matters when the state is static. This study makes an important point here: we did not focus on the situation assessment of a single state in LOTO, but rather on an ongoing and continuous process for acquiring SA in a dynamic and time-critical environment. Therefore, we established the dynamics of the LOTO environment. We were most concerned with the variability of the information on machine status; these include, for example, whether equipment was under maintenance or not, what level of maintenance had been completed, what modifications had been made, and the change of status followed by a shift changeover. We were also interested in the status of a group of machines locked together and situations where group lockout was being applied. Finally, an updated goal status of production and the tactical planning required to meet those plans created more dynamics. Although documented procedures and technical manuals are viewed as static information, they can incorporate attributes of dynamics in situations when machine are modified and locking devices are altered. Goals of accomplishing a production target and tight expectations of a machine bought back to life can influence how attention is directed to LOTO, how information is perceived, and how that information is interpreted. When top-down processes of goal accomplishment operate on par with bottom-up processing of perceived information, SA is required. In this context, salient cues should activate appropriate mental models of the situation, leading to correct decisions. Therefore, a focus on SA is vital to facilitate the appropriate mental model by ensuring the appropriate design of LOTO equipment and processes.

Recommendations
In the discussion, we established the dynamics involved with LOTO and thus the importance of SA. In this respect, the ideal scenario would be to rectify those dynamics as much as possible, for example, by performing the least possible number of alterations on a machine. However, the dynamic information requirements that we found in the GDTA suggest the need for a more pragmatic approach, sometimes going beyond what OSHA standards require; in such cases, SA is important. The design of a LOTO system should focus on helping employees develop a correct picture of what is going on with the machine and the environment.
The third objective of this study was to prioritize the critical SA issues of this organization's LOTO system and make recommendations to rectify the issues. We found that a lack of integration of information, poorly standardized information, information not made explicit, a lack of understanding of the context, and inadequate processes are the major issues of the LOTO system affecting SA. Endsley [18] provided a detailed and systematic methodology of the design principles of SA interventions. Below, we briefly explain how SA interventions could possibly rectify the SA issues in LOTO.

Integrating Information
The lack of integration of information subcategory includes several issues: difficulty identifying interconnections, confusing multiple work instructions, unexpected interference with existing controls, restricted access to controls, and risks with alternative blocks. Under time pressure, these conditions can lead to LOTO violations. Organizing information around the goal rather than following a technology-driven approach can help identify the goals and the information needed for each goal. For example, the goal of assessing the equipment status can be assisted by providing integrated information about the status of the interconnections between sources, not merely about the status of individual power sources. At best, this information can be made explicitly available at the point of operation (see Figure 10); otherwise, explanations of how to obtain it can be provided. Similarly, confusing multiple work instructions are often an adverse effect of a technology-based approach; integrating them based on what is required by the goal is preferable. Data-driven processing can be assisted by making information available on what is happening in the context of other employees, other machines, and tactical production plans. What is critical here is supporting trade-offs between goal-driven and data-driven goals in a such way that those complement each other. For example, such trade-offs are often needed between the goals of the timely accomplishment of work schedules and data-driven information on the context; high salience of either type of information can affect the SA. The design of a LOTO system should consider how the user can switch between the two modalities. the status of individual power sources. At best, this information can be made explicitly available at the point of operation (see Figure 10); otherwise, explanations of how to obtain it can be provided. Similarly, confusing multiple work instructions are often an adverse effect of a technology-based approach; integrating them based on what is required by the goal is preferable. Data-driven processing can be assisted by making information available on what is happening in the context of other employees, other machines, and tactical production plans. What is critical here is supporting trade-offs between goal-driven and data-driven goals in a such way that those complement each other. For example, such trade-offs are often needed between the goals of the timely accomplishment of work schedules and data-driven information on the context; high salience of either type of information can affect the SA. The design of a LOTO system should consider how the user can switch between the two modalities. Figure 10. Improved work instructions to ensure that information is integrated. Multiple energy sources are mentioned. Instructions include waiting for 15 min after disconnecting electricity for the heater to cool down.

Providing Consistency and Standardization
The poorly standardized information subcategory includes difficulty understanding illustrations and procedures, difficulty understanding tag colors, and confusion regarding the unintended use of tags. SA interventions for consistency and standardization directly address this Figure 10. Improved work instructions to ensure that information is integrated. Multiple energy sources are mentioned. Instructions include waiting for 15 min after disconnecting electricity for the heater to cool down.

Providing Consistency and Standardization
The poorly standardized information subcategory includes difficulty understanding illustrations and procedures, difficulty understanding tag colors, and confusion regarding the unintended use of tags. SA interventions for consistency and standardization directly address this issue. SA interventions using techniques to ensure logical consistency can reduce inconsistencies in the system by making consistent presentations of information and illustrations, the modes they represent, and the formats used in the presentation. For example, the closure of the valve shown in Figure 11 is anti-clockwise, which is not consistent with others, so it requires specific information. Interventions to map system functions to the goal and mental modes of the user can assist standardization. Mapping enables the operator to understand how the system works and how it is connected to achieve goals. Grouping information based on Level 2 and 3 SA requirements and goals can provide the basis for standardization and help organize the information. For example, all the information that is needed to achieve the goal of assessing conditions for safe re-energizing would ideally be grouped together; at least the sources of the information could be grouped and presented to the technician. 2019, 6, x FOR PEER REVIEW 16 of 23 Figure 11. Air cut-off value that closes in an anti-clockwise direction, which is not consistent with the other valves.

Making Information More Explicit
Information not made explicit was reflected in participants' comments on uncertainty about updated procedures, not knowing what alternations have been made, what alternative blocks are needed, and the difficulty of knowing whether LOTO should be applied at all. Attempts should be made to make information explicit. For example, in the case of a machine being stopped, there should be no uncertainty about whether it is locked out; this information needs to be very explicit. Whenever no information is available, such as in the absence of a tag, it is important to explicitly identify missing information. SA interventions using data salience can support the operator in assessing the certainty of information. For example, in assessing stored energy, estimated information should be presented as 'estimated', together with the accuracy of that information, if possible. Supporting uncertainty management activities promotes awareness of the situation and the certainty of the information.

Improving the Understanding of the Context
A lack of understanding of context includes not understanding the documented content, not knowing the risk factors, and not knowing the exact purpose of LOTO. Supporting comprehension by presenting Level 2 SA directly by integrating information can assist in the understanding of the meaning of perceived information. For example, in the case of a block applied to support a part of a machine in place, it is more meaningful if the load applied on the block, as well as the load-bearing capacity, is explicitly presented. In fact, there was an accident in the organization when workers depended on a single door damper instead of two while conducting a maintenance activity. Interventions to provide system transparency and observability can improve the understanding of the system. For example, schematics can provide system transparency by presenting how actuators Figure 11. Air cut-off value that closes in an anti-clockwise direction, which is not consistent with the other valves.

Making Information More Explicit
Information not made explicit was reflected in participants' comments on uncertainty about updated procedures, not knowing what alternations have been made, what alternative blocks are needed, and the difficulty of knowing whether LOTO should be applied at all. Attempts should be made to make information explicit. For example, in the case of a machine being stopped, there should be no uncertainty about whether it is locked out; this information needs to be very explicit. Whenever no information is available, such as in the absence of a tag, it is important to explicitly identify missing information. SA interventions using data salience can support the operator in assessing the certainty of information. For example, in assessing stored energy, estimated information should be presented as 'estimated', together with the accuracy of that information, if possible. Supporting uncertainty management activities promotes awareness of the situation and the certainty of the information.

Improving the Understanding of the Context
A lack of understanding of context includes not understanding the documented content, not knowing the risk factors, and not knowing the exact purpose of LOTO. Supporting comprehension by presenting Level 2 SA directly by integrating information can assist in the understanding of the meaning of perceived information. For example, in the case of a block applied to support a part of a machine in place, it is more meaningful if the load applied on the block, as well as the load-bearing capacity, is explicitly presented. In fact, there was an accident in the organization when workers depended on a single door damper instead of two while conducting a maintenance activity. Interventions to provide system transparency and observability can improve the understanding of the system. For example, schematics can provide system transparency by presenting how actuators are linked together. Whenever the direct presentation of comprehension is not possible, further SA interventions can help. Making critical cues for schema activation more salient can improve the understanding of a situation by referring to a prototypical situation. For example, in addition to switching off a machine using a disconnect switch, it is important to ensure that controls are in the off position to avoid unexpected activation when the machine is re-energized. As shown in Figure 12, it helps if all off positions are aligned. As mental models and schemata play an important role in achieving high SA, it is important to trigger the operator's schemata with obvious information from the system.

Improving Adequacy of Procedures and Validity of Information
Inadequate procedures and outdated information hinder comprehension. SA interventions presenting information with timelines can support temporal awareness and thus promote the awareness of outdated information. The management of change procedures can enforce the requirements to keep updated information about machine alterations. Although minimizing task complexity can lessen the demands for detailed procedures and frequent updates, the complexity of the maintenance task is largely attributed to how well the machine has been designed for maintenance [42] by the original equipment manufacturer (OEM). Within the scope of LOTO, minimizing the complexity applies during de-energizing and re-energizing, particularly by making it easy to determine interconnections between different energy sources. When dealing with multiple sources of information, SA interventions for assessing confidence of composite data can aid the operator in appraising the reliability or confidence level of the information (e.g., coming from different sensors). This level of appraisal is required to determine the fault and the need for maintenance; it is also required during de-energizing and re-energizing phases. It is indirectly associated with determining the extent of the task required, tool requirements, and estimated time for completion.

Improving Communication Structure
Poor communication structure hinders the projection of risks involved with the locking component. When the operator is pursuing a maintenance or production goal, attention is usually directed toward a subset of information. As a result, the operator may fail to acknowledge other problems in the environment. Supporting global SA means giving the operator the ability to attend to information about the overall status of the system at all times. Interventions to support global SA

Improving Adequacy of Procedures and Validity of Information
Inadequate procedures and outdated information hinder comprehension. SA interventions presenting information with timelines can support temporal awareness and thus promote the awareness of outdated information. The management of change procedures can enforce the requirements to keep updated information about machine alterations. Although minimizing task complexity can lessen the demands for detailed procedures and frequent updates, the complexity of the maintenance task is largely attributed to how well the machine has been designed for maintenance [42] by the original equipment manufacturer (OEM). Within the scope of LOTO, minimizing the complexity applies during de-energizing and re-energizing, particularly by making it easy to determine interconnections between different energy sources. When dealing with multiple sources of information, SA interventions for assessing confidence of composite data can aid the operator in appraising the reliability or confidence level of the information (e.g., coming from different sensors). This level of appraisal is required to determine the fault and the need for maintenance; it is also required during de-energizing and re-energizing phases. It is indirectly associated with determining the extent of the task required, tool requirements, and estimated time for completion.

Improving Communication Structure
Poor communication structure hinders the projection of risks involved with the locking component. When the operator is pursuing a maintenance or production goal, attention is usually directed toward a subset of information. As a result, the operator may fail to acknowledge other problems in the environment. Supporting global SA means giving the operator the ability to attend to information about the overall status of the system at all times. Interventions to support global SA can aid awareness by improving communication among team members and creating a holistic situation. Communication is particularly important when maintenance is performed by a group of technicians. Group LOTO operations typically require more coordination and communication than single-person LOTO operations. Greater coordination between employees is particularly important when more than one department is involved in the task. Design principles suggested for facilitating team SA can be useful in the collaborations of technicians, supervisors, and machine operators demanded by the LOTO process. Further interventions to support the transmission of different comprehension across teams and a shared mental model will result in more efficient communications by reducing misunderstandings.

Conclusions
We set up this study to show the importance of the SA concept for companies wishing to determine the efficacy of their LOTO systems. Our use of GDTA revealed decision requirements under a dynamic context and indicated what SA information is required. Our classification using QCA found that the comprehension and projection levels were more affected. Out of the four major components of LOTO, the locking component was found to be most affected, and in the locking component, the projection level was most affected.
As this study shows, SA interventions can be used to mitigate high-priority issues. The intent of LOTO is to make workers aware of what is going on with machines undergoing maintenance; thus, SA is well suited to attempts to improve LOTO effectiveness through special interventions. One major conclusion that we can make is that users should be involved in risk evaluation; their input will identify latent risks of a cognitive nature that might not be captured by regular physical risk assessments performed by experts.
Overall, this study confirms that SA is an applicable concept for evaluating and improving the effectiveness of LOTO systems, despite the somewhat static nature of LOTO applications. Varying production demands, shifting operations, machine modifications, and the progression of maintenance work itself make SA an important issue with LOTO. Acknowledging that these systems are and will continue to be used by humans and understanding how they can serve their primary purpose of making humans aware of what is going on will make LOTO use more effective. This study suggests the need to prioritize the SA intentions of LOTO, ultimately to make a safer workplace.