Lockout and Tagout in a Manufacturing Setting from a Situation Awareness Perspective

Abstract

Applying lockouts during maintenance is intended to avoid accidental energy release, whereas tagging them out keeps employees aware of what is going on with the machine. In spite of regulations, serious accidents continue to occur due to lapses during lockout and tagout (LOTO) applications. Few studies have examined LOTO effectiveness from a user perspective. This article studies LOTO processes at a manufacturing organization from a situation awareness (SA) perspective. Technicians and machine operators were interviewed, a focus group discussion was conducted, and operators were observed. Qualitative content analysis revealed perceptual, comprehension and projection challenges associated with different phases of LOTO applications. The findings can help lockout/tagout device manufacturers and organizations that apply LOTO to achieve maximum protection.

1. Introduction

The past decades have seen hundreds of maintenance workers suffering fatal injuries while performing their work. Almost half of the fatalities occurred during planned preventive maintenance operations [1]. Those who are at high risk, installation, maintenance, and repair workers report an overall rate of 9.4 fatalities per 100,000 workers, compared to 2.6 for production workers and 3.6 for all workers [2]. The United States (US) Occupational Safety and Health Administration (OSHA) and the US National Institute for Occupational Safety and Health (NIOSH) find significant risk results from exposure to hazardous energy during maintenance. Studies [3] have reported the most common mechanisms of such injuries: being caught in or between parts of equipment, electrocution, and being struck by or against objects. Distinctive maintenance scenarios of these injuries include cleaning mixers, cleaning conveyors, and installing or disassembling electrical equipment. Given the risks, special procedures are necessary to protect workers.

In order to safeguard employees from the unexpected release of hazardous energy or energization from equipment during service or maintenance activities, lockout/tagout (LOTO) safety procedures [4] are used in industries. These procedures are to ensure that harmful machines under maintenance are properly tagged and shut off until the completion of maintenance work and to verify that the hazardous energy has been controlled. LOTO is considered a positive restraint [4] because a key is required to unlock, whereas, tagout is a warning device to warn employees not to reenergize the energy sources [4], thereby improving the awareness of potential hazards near equipment and machinery. In many countries, LOTO activity has been regulated, and it became mandatory in the United States in 1989 when OSHA promulgated the control of hazardous energy (lockout/tagout) standards [5].

1.1. Ongoing LOTO Issues

In spite of the regulations, LOTO-related fatalities continue to occur. LOTO violations in industry ranked fifth in the top 10 violations in 2016 [6] and second in willful violations [7]. LOTO-related fatalities occur for a number of reasons [3], including individual characteristics, energy control status, and type of activity. Shortcomings in LOTO programs include incomplete programs; missing steps in general procedures; users not reading the placards; a tendency to use alternative methods without proper risk assessment; the absence of supervision and coordination of subcontractors; and a lack of audit tools and documentation of audit results [8]. To be successful, an LOTO program requires personnel training and specific plant equipment access restrictions. In addition to training, designing equipment for easy lockout helps prevent LOTO related injuries [9,10,11]. Authors have highlighted the significance of adding a safety committee for LOTO improvements [12]. A self-audit checklist was found to be useful in manufacturing firms’ efforts to assess and improve their LOTO programs [13]. However, the implementation phase of LOTO frequently lacks a human factor approach, causing a mismatch between the LOTO system and the users. A human factor perspective can also fuel a long-term pragmatic intervention; such a need has been identified in earlier studies [12]. This article identifies the circumstances of such deviations and their cognitive mechanisms, and makes suggestions for improvements.

1.2. Situation Awareness and LOTO

Situation awareness (SA) is the ability to develop and maintain awareness of ‘what is going on’ [14]. Various SA models in the literature refer to individual [14], team [15], and sociotechnical system perspectives [16]. These models provide the foundation for decision making across a range of complex and dynamic systems. A popular understanding of SA is the “perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” [14]. This view recognizes SA as the state of the knowledge about a situation, at three levels: informational cues are perceived in Level 1; multiple pieces of information are integrated in Level 2, and their relevance to goals is determined; future situations, events, and dynamics are forecast in Level 3. SA principles have been applied to the development of operator interfaces, automation concepts, and training programs in a wide variety of fields across many safety critical domains, such aircraft, air traffic control, power plants, advanced manufacturing systems and maintenance [14,16,17,18,19,20,21,22].

In this study, we hypothesized SA as an important consideration in a human factor approach to LOTO. In previous studies, the LOTO process was partially or completely validated, but few studies have attempted to look for the underlying causes. By doing so, our SA approach makes an original contribution to the literature.

1.3. Objectives

The aim of this study is to show that SA is an important concept in LOTO, and SA interventions can improve effectiveness of LOTO. The study’s objectives are the following:

• To critically assess the LOTO system of a manufacturing organization through the perspective of a user and determine SA requirements for the LOTO applications.
• To classify the issues based on SA levels and components of the LOTO system.
• To prioritize the SA issues and recommend ways to rectify them.

2. Materials and Methods

This research was an applied study, seeking to solve a practical problem. It attempted to improve the effectiveness of LOTO in a manufacturing organization by assessing user SA. Since few previous studies of LOTO use SA, we opted for a basic method.

2.1. Study Design

To fulfill the first objective of determining the SA requirements, we used goal-directed task analysis (GDTA). GDTA seeks and documents the information needed by users to perform their tasks and how this information is integrated to address a decision [18]. In this study, GDTA consists of structured interviews, observations of operators performing their tasks, and a detailed analysis of the documentation on those tasks. GDTA shows the hierarchy of goals, the decision requirements associated with the goals, and the situation awareness requirements necessary for addressing the decisions. [18]. GDTA has been successfully used in many domains, including the investigation of team SA in aviation maintenance [17]. The first step of GDTA is to identify the key decision makers. In the second step, each decision maker is asked about the main goals. In the third step, each decision maker is asked about the sub goals that are necessary to accomplish the main goal. In the final step, SA information requirements for making those decisions are identified. Importantly, GDTA focuses on cognitive demands, not on physical tasks; tasks should be distinguished from goals, because tasks are technology-dependent. In other words, the SA requirements are associated with the goal, rather than the task. For example, the goal is to restrict energy activation of the equipment under maintenance; the respective task with LOTO is to apply the lock.

To fulfill the second objective, we assessed SA levels. There are different techniques available for assessing SA; most depend on questionnaires. For example, SART (situation awareness rating technique) [23] is a post-trial subjective rating technique. SART measures 10 dimensions on a seven-point rating scale: familiarity of the situation, focusing of attention, information quantity, information quality, instability of the situation, concentration of attention, complexity of the situation, variability of the situation, arousal, and spare mental capacity. Then, the ratings are combined to form three dimensions: demands on attention resources, supply of attention resources, and understanding of the situation. These calculate a measure of participant SA. The SASHA (Situation Awareness for Solutions for Human Automation Partnerships in European ATM) questionnaire [24] assesses the effect of automation on controller situation awareness. In the questionnaire, items address three aspects of SA: information extraction, integration and anticipation. Responses to these items are given on a seven-point Likert scale, ranging from ‘never’ to ‘always’.

In this study, we investigated how SA aspects of LOTO are perceived by users. We implemented a qualitative survey design to collect and study machine technicians’ and machine operators’ responses on how well the existing LOTO system makes them aware of what is going on. Qualitative data were collected using three methods: interviews, observations, and a focus group discussion. These methods have an edge over simple questionnaires that are suitable for relatively clear and simple questions, and their choice of replies is limited to fixed categories [25]. Although the methods we used are more time consuming than a questionnaire, they gave us the advantage of collecting a large amount of data. The technique used in this study can be identified as a combination of inductive and deductive methods [26], as we condensed qualitative data under subcategories (in a deductive pathway), and then developed main categories following an inductive pathway. To make recommendations, thus fulfilling the third objective, we determined the highest priority issues, i.e., the most frequently raised ones.

2.2. Data Source and Selection Criteria

The manufacturing organization has five different production lines: offset printing, flexography printing, screen printing, thermal printing, and weaving technologies. The machines’ energy sources are electrical, mechanical, and pneumatic (Figure 1).

In this study, we sought a diverse sample of participants to yield a maximally heterogeneous sample, using stratified purposeful sampling [27]. Although many different workers (e.g., team leaders, planners) are affected by LOTO, we identified the key decision makers in the LOTO tasks: machine technicians and machine operators. Technicians are not specialized in different production technologies but rather in technical disciplines such as electrical and mechanical. Machine operators are multi-skilled; they are trained to handle several production technologies. All the technicians and operators were males; the organization did not have female employees in these positions at the time of the study. Two authors were involved in data collection and analysis. Author 1 was an employee in the organization at the managerial level.

Table 1. Heterogeneous study sample.

Interested Group	Sampling Criteria	Sample Group	No.
Machine technicians	Technical specialty	Mechanical	8 (53%)
		Electrical	7 (47%)
	Years of experience	Mean ± SD	9.6 ± 2.4
	Age (yrs)	Mean ± SD	37.3 ± 7.1
Machine operators	Production specialty	Multi skilled	20
	Years of experience	Mean ± SD	5.9 ± 1.9
	Age (yrs)	Mean ± SD	29.4 ± 6.4

shows details of the sample participants.

2.3. Interviews

We used unstructured and structured interviews for data collection. Participants were first educated about the research purpose of the interviews in a verbal introduction prior to the interviews. The unstructured interviews gave participants the opportunity to make open comments: for example, what goes on during LOTO applications, and what they are looking for (i.e., SA information required). The comments made during unstructured interviews led to the questions to ask during structured interviews. During the initial part of the interviews, participants were asked about their maintenance goals, the sub goals, and the decisions required in the attainment of those goals. The next task was to identify specific SA information requirements to make those decisions. In structured interviews, we applied GDTA [18] to determine a situationally-based maintenance scenario describing performance in the absence of LOTO. It is pertinent to mention that the analysis was based on operators and technicians’ goals in the maintenance scenario, rather than on specific support systems (i.e., LOTO). The imagined absence of an information system stimulated participants to think of what information is required for the decision-making tasks they had just listed. During the structured interviews, planned questions addressed the second research objective: the affected SA level (level 1 to 3) and the respective LOTO system component. Participants were asked to focus on an event with instructions directing them to a particular instance. Questions included: ‘What is the most difficult task with LOTO?’; ‘How do you attempt to handle the problem?’; ‘What would help you manage the problem?’; ‘If LOTO can be improved immediately, what you want to be fixed first?’ Transcribing was required because the substantive content was the focus of the analysis [28].

2.4. Observations

Participants were made aware that we make observations for research purposes. LOTO application was observed during machine service and repair work. For example, the technicians were observed trying to follow the content on a tag and attempting to insert locks into heavily restricted locations. Although such observations validated the difficulties that were stated, we did not consider participative observation [29] to be a major data collection method in this study, as we observed only a few cases. On a continuum of complete observer to complete participant, involvement in this research can be considered peripheral [30]. We consider that Author 1′s regular presence in the organization minimized the Hawthorne effect [31] and supported objective observation, even in an overt role [32].

2.5. Focus Group Discussion

A focus group was arranged for participants who mentioned risk with LOTO. Focus groups can be viewed as a stage where participants tell, negotiate, and reformulate their ‘self-narratives’ [33]. The group was heterogeneous, representing different production departments, different technologies, and different years of experience. Preparatory work was carried out by developing topic guides and selecting stimulus materials based on the points requiring more reflection. The location of the focus group and the associations that it has for the participants are likely to have an important impact [34]; therefore, the focus group discussion was held in the organization’s training room, which was a very familiar place for all the participants. All the participants knew each other, as they were employees in the same organization. As the facilitator and moderator, Author 1 began with the discussion points; then, he let the team discuss them and helped maintain the focus in the discussion without harming the dynamics. Participants were engaged not just in presenting their own narratives but in supporting and challenging others’ narratives. We did not attempt to categorize individuals in terms of their views, and we attempted to contain the discussion within the frame of our study.

3. Qualitative Content Analysis

The qualitative data collected was analyzed to address the research questions and to understand what participants meant to say. The initial step of qualitative content analysis (QCA) [35] is to get a better interpretation of data followed by dividing the text into smaller parts, i.e., the “meaning units”. These “meaning units” are further categorized as codes and subcategories [28,36].

We closely examined the qualitative data obtained from interviews and the focus group discussion to segregate what was relevant to the research objectives. Then, we delineated the meaning units related to the research objectives. One major challenge was to filter the meaning units referring to SA. The three-level SA model is conceptually similar to human information processing models [16,37,38]. We had to avoid creating a traditional information processing model [39], whilst carefully selecting the meaning units reflecting specific characteristics of SA. Criteria used for selection were dynamic aspects of the situation (not static information), situations requiring the knowledge of three levels of SA [14], situations requiring both top–down and bottom–up information processing (not merely linear information processing) [40], and the active (not passive) nature of the person who is seeking information [17]. Then, the units were condensed to shorter versions by determining the underlying meanings.

From the meaning units, we developed codes through an iterative process involving reading, reflection, and rereading. Following the coding, we determined subcategories and main categories. Subcategories summarize what is said, and main categories are what the study wants to answer. Therefore, subcategories are data-driven (in our case, based on the interview and observation records), and main categories are concept-driven. As per our research objectives, subcategories can also be identified as causal factors for the difficulties identified in the meaning units. Whenever a pertinent theme was identified (that is, mentioned by at least two participants), we added it as a new subcategory. Our intention was to identify two dimensions: the SA level affected and the respective component in the LOTO system. These became our two main categories. Figure 2 illustrates the data collection and analysis method. Each category had a definition, description, and decision rule; decision rules ensure that categories are mutually exclusive (see

Table 2. Names, descriptions, examples, and decision rules for categories. LOTO: lockout/tagout, SA: situational awareness.

Main Category	Category Name	Description	Example	Decision Rule
Affected SA level	Perception	Important information through LOTO application, such as maintenance status, progression, and changes not correctly perceived	Not able to easily perceive; energy isolation is done on the equipment due to the absence of the LOTO document in close proximity	System design failed to present correct information, or there is a failure in the communication process, so information is not correctly perceived by the user
	Comprehension	Important information through LOTO application not correctly put together to formulate a holistic view of the situation	Failure to comprehend the writing on the tag together with the observable status of the machine and energy connections	Although the information is correctly perceived by the user, the user fails to comprehend the significance or meaning of that information
	Projection	Failure to predict the future consequences of the situation using the information	Failure to predict the status of maintenance work followed by a shift changeover	Individuals may be fully aware of what is going on, but have a poor model for projecting what that means for the future
Group 2-Affected LOTO system component	Decommissioning	Factors hindering the disconnection of equipment from energy sources	Difficulty of identifying connections between energy feeds and actuators	Issues prevent or obstruct the removal and isolation of energy sources to limit the risk of accidental energy release to the lowest possible level
	Locking	Factors hindering the application of locks	Incompatible devices requiring excessive force	Issues in selecting and using locking devices leading to reduced protection
	Tagging	Factors hindering the communication expected through tags	Color codes used in LOTO tags are not standardized	Issues in the design and usage of tags leading to disturbance of the risk communication
	Recommissioning	Factors hindering the recommissioning process of LOTO	Difficulty verifying whether all the tools are removed from the machine	Issues arising during recommissioning of LOTO leading to risks

).

This coding frame was examined for consistency and validity. The authors conducted independent pilot coding using the first version of the coding frame, categorizing the same set of transcripts. Coding consistency was 80% across the main categories and 70% across the subcategories during the pilot version, and 90% across the main categories and 80% across the subcategories for the final version. Since we had two main categories, we summed the content under two main categories when calculating consistency. One-third of the transcripts were again categorized by both authors during main coding. As per the definition of validity, the coding frame will not adequately describe the qualitative material if coding frequencies are high for residual categories; this requires the introduction of additional subcategories [41]. However, through this exercise, we learnt that certain subcategories became residual for reasons other than the inadequacy of the coding frame. Provided that the coding frame is well evaluated for consistency, content that is mostly spoken can still denote frequency. In fact, in this study, the amount of distinct content under each category, i.e., the coding frequency, was considered as the variable that determined the priority of concerns in terms of SA level affected and the respective LOTO component.

4. Results and Discussion

After we removed what was not relevant to this study (following the SA criteria explained above), we were left with 109 meaning units. We further reduced data by eliminating statements with similar meanings. Therefore, the meaning units were distinct. In other words, we did not depend on the total frequency with which a certain meaning unit was mentioned; rather, we considered the number of units with distinct meanings. This prevented us from focusing on only a few major issues. However, we were not always certain whether participants were referring to the same incident multiple times and wanted to avoid repetition. Ultimately, we identified 27 (N = 27) distinct meaning units from the qualitative contents of interviews and the focus group.

4.1. SA Requirements

Following the general steps of GDTA, we identified users’ major goals, sub goals, decision-making requirements for the sub goals, and SA information requirements. The major goals of both maintenance technicians and operators during a maintenance scenario are twofold: accomplishing the task correctly (maintenance/production) on time and ensuring the safety of the equipment, themselves, and affected employees. Our focus was on the SA requirements of LOTO, specifically, its major purpose of helping workers comprehend the environment and foresee risks to avoid accidents. Therefore, we excluded drilling down to the subcategories of the tasks under accomplishing the maintenance/production task; we continued analysis for the safety component only. In order to ensure employee and equipment safety, we identified four sub goals: communicating with other employees, assessing equipment status, assessing the risk of reactivation, and assessing the conditions for reactivation. Then, we determined the decision-making requirements for each of the sub goals. Finally, we identified the data required (Level 1 SA), the higher-level information on the significance of the data (Level 2 SA), and the projection of future events (Level 3 SA) for each of those sub goals (see Figure 3).

The GDTA included results from the review of current documents as well. In addition to the existing LOTO procedures at the organization, we looked at three important documents: OSHA 3120 2002 (revised) [4]: Control of Hazardous Energy; OSHA Standard 1910.147: The control of hazardous energy; and OSHA 1910.147 Appendix A: Typical minimal lockout procedure. Since the organization studied is based in the US, we referred to applicable OSHA standards in the US. In the GDTA, we included all the dynamic information requirements mentioned in the above OSHA documents. Notably, the input from interviews provided a great deal of dynamic information on topics not mentioned in the OSHA documents; for example, these included a possible extension of work, the availability of spare parts, details on the production work-in-progress, equipment modifications, other equipment affected by blocks, the detection of accidental activation, time taken for reactivation, etc.

Overall, a review of sub goals indicated that information must be communicated between technicians, operators, and other employees. SA is not only a concern of the technicians and operators who directly work with the equipment. Other employees who work around it, as well as anyone who has authority over or a connection with task accomplishment (supervisors, technicians, planners, operators of adjacent operations) must have an understanding of what is going on with the machine under maintenance. In this way, LOTO must facilitate team synchrony by informing all the stakeholders of the status of the maintenance task, thus assisting them to achieve a common goal. LOTO must also standardize the energy isolation and provide good communication with other areas; for example, these areas include tactical planning and the provisioning of assets, particularly during shift changeovers. Importantly, these elements keep changing, as they are subjected to dynamic situations, mainly with the progression of the maintenance work itself, changing of teams and individuals, and changes in operational priorities (e.g., tactical changes in production plan). In a longer-term perspective, lockout practices require adaptation to ever-changing dynamics of machinery and processes, tighter schedules, and regulatory compliances.

The SA requirements that we identified for LOTO are consistent with the findings in previous studies on SA requirements, generally in maintenance. With respect to maintenance fieldwork, previous studies [17,21,22] identify four important elements of SA:

• Identifying abnormalities, understanding the problems, and predicting failures;
• Comprehending the environment and their risks to avoid accidents are SA requirements;
• Maintaining team synchrony by collaborating and coordinating tasks to achieve a common goal;
• Maintaining a good corporate environment, standardized work routine and terminology, and communicating with other supporting areas.

Although goal-directed task analysis (GDTA) implies a focus on goal-driven cognitive processes (top–down), our use of it facilitated the identification of the demands in data-driven processes (bottom–up) by asking the participants to imagine different scenarios. In fact, the critical importance of the GDTA technique was its ability to elicit user experience, beyond what was mentioned in the documentation. As a result, GDTA was able to detect the demands for both goal-driven and data-driven decisions while applying LOTO under changing environments. With this, we could establish that LOTO-related issues are subject to data-driven and goal-driven dynamic conditions. Goal-driven and data-driven iterative processes to perceive dynamic information and match mental patterns is a major character of SA [14] (this data-driven/goal-driven process should not be confused with the data-driven/concept-driven categories in QCA). Therefore, a framework comprising the three levels of SA is useful to explore how well LOTO accomplishes its key intent of making employees aware of what is going on with equipment in a dynamic maintenance environment.

4.2. SA Issues

In the QCA, the coding frame itself can be considered the main result. The categories and the interrelations between the categories serve as discussion points. In our study, we defined two main categories: the SA levels affected and the respective components of the LOTO system. By coding the frequencies with which those main categories were mentioned, we ascertained their priority. For the first main category ‘affected level of SA’, we determined issues of perception, comprehension, and projection. For the second main category, ‘affected LOTO system components’, we looked at decommissioning, locking, tagging, and recommissioning. The coding frame with the content provided the basis for the first objective: critically assessing the LOTO system for SA issues.

Table 3. Example meaning units with subcategories and categories.

Meaning Unit	Condensed Meaning Unit (Difficulty for Reason)	Coded Meaning Unit	Subcategory (Causal Factor)	Main Category 1—SA Level Affected	Main Category 2—LOTO System Component Affected
We use a number of tools and it is time-consuming and doubtful whether we leave any instrument inside the machine.	Difficult to verify whether all tools are removed from the machine before start-up	Doubtful tool clearance	Inadequate procedure	Perception	Recommissioning
Description in some of the procedures are not clear and locating the hazard zones is difficult	Difficult to locate hazard zones due to unclear procedures	Identification difficulty of hazard zones	Poorly presented information	Comprehension	Decommissioning
In some cases, LOTO cannot be applied at all, so we have to think so much about alternative methods considering all the risks, and we are not certain that all risks are covered	Absence of clear method to identify alternative methods when locks are not applicable	Not knowing what exactly to do for alternatives	Information not made explicit	Projection	Locking

shows three examples, meaning units representing each level of SA (perception, comprehension, projection) with their subcategories and categories.

Table A1. Coding frame with meaning units and development of categories.

Meaning Unit	Condensed Meaning Unit (Difficulty for Reason)	Coded Meaning Unit	Subcategory (Causal Factor)	Main Category 1—SA Level Affected	Main Category 2—LOTO System Component Affected
Description in some of the procedures are not clear and locating the hazard zones is difficult	Difficult to locate hazard zones due to unclear procedures	Identification difficulty of hazard zones	Poorly presented information	Comprehension	Decommissioning
Some of the visuals used in illustrations are not similar to others used in the organization, leading to difficulty in understanding them	Difficult to interpret illustrations in procedure documents due to absence of unique illustration methods	Difficult to understand illustrations	Poorly standardized information	Comprehension	Decommissioning
We see a different flow of information in procedures, so we are not very certain whether all the important points are covered in the procedures	Difficult to interpret procedures due to lack of standardized content and flow	Difficult to understand procedures	Poorly standardized information	Comprehension	Decommissioning
We are not certain what exactly to write on the tag and in many cases, writings on the warning tag contain different fields of information, so we need to contact the person who wrote it asking clarifications.	Writings on the tag cannot be comprehended due to absence of clear structure	Comprehension difficulty of tag content	Poorly organized information	Comprehension	Tagging
There are confusions with colors of LOTO tags in comparison to colors of other warning signs and safety notices at the workplace.	Difficult to interpret tags by their colors due to lack of standard	Comprehension difficulty of tag colours	Poorly standardized information	Comprehension	Tagging
Tags are sometimes used to denote maintenance work that is planned but not under execution, so we are not sure whether those are really under repair, though indicated with tags	Difficult to interpret the application of tags due to usage of them for other purposes	Confusion with unintended use of tags	Poorly standardized information	Comprehension	Tagging
LOTO procedures require verification of tools removal; we use a number of tools and it is time-consuming and doubtful whether we leave any instrument inside the machine.	Difficult to verify whether all tools are removed from the machine before start-up	Doubtful tool clearance	Inadequate procedure	Perception	Recommissioning
Tried to switch on the machine several times, could not switch on and then tried to locate the reason. Finally noted a lockout is applied. Tagout applied at the main power cut off at distribution panel.	Not able to easily perceive; energy isolation is done on the equipment, due to absence of the LOTO document in close proximity	Difficult to know whether LOTO is applied	Information not made explicit	Perception	Tagging
Tried operating the machine, could not switch on and tried to locate the issue, finally noted a lockout and tag out is applied on the main switch to restrict power to other machine in the production line.	Not able to easily perceive, energy isolation and the reason for isolation, due to absence of the LOTO document at close proximity	Difficult to know why LOTO is applied	Information not made explicit	Perception	Tagging
Sometimes we can’t put lockout, in procedure exemption allows putting tag only. However, we cannot understand from the tag alone whether there is a lock or not. We often doubt with tags whether they accompany a lock or not.	Not able to perceive whether both tag and lock are applied, due to absence of such information on the tag	Difficulty to interpret depth of LOTO	Lack of required information	Perception	Tagging
We have attended some alternations in the equipment, within the limits recommended by the OEM but we are not certain whether LOTO is adjusted accordingly.	Uncertainty of existing LOTO processes due to equipment modifications	Uncertainty of updated procedures	Information not made explicit	Projection	Locking
In some cases, LOTO cannot be applied at all, so we have to think so much about alternative methods considering all the risks, and we are not certain all risks are covered	Absence of clear method to identify alternative methods when locks not applicable	Not knowing what exactly to do for alternatives	Information not made explicit	Projection	Locking
We are not very certain what impact it would create if we do not block some of the energy sources, documents do not present the purpose of certain blocking procedures	Lack of understanding of impacts of release of energy during maintenance work	Not knowing the risk factor	Lack of understanding of context	Projection	Decommissioning
Meaning Unit	Condensed Meaning Unit (Difficulty, for, reason)	Coded Meaning Unit	Subcategory (Causal factor)	Main Category 1—SA level affected	Main Category 2—LOTO System Component affected
In the procedure manuals, there is nothing mentioned about adjustments needed on LOTO followed by local modifications; we are in doubt as to what level of local modifications will impact our own safety	Lack of understanding about the restrictions for modifications	Not knowing the risk factors	Lack of understanding of context	Projection	Decommissioning
There are transition locks, so other shifts can know maintenance is going on; however, there is no clear transition process. Individuals follow different processes to communicate the completion of repair work	Not able to predict the status of maintenance work after shift changeover, due to lack of standard communication	Difficulty of knowing the status during shift changeover	Poor communication structure	Projection	Locking
We are not very comfortable with safety of unlocking, we doubt access of multiple keys to locks	Insecure feeling due to availability of multiple keys for locks	Unsafe feeling	Inadequate procedure	Projection	Locking
When we are to apply alternative blockings, there were situations we never thought would restrict some other operations, there is no proper way to foresee such interruptions	Alternative blocks unexpectedly restricting access to other controls	Unexpected interference with existing controls	Lack of integration of information	Projection	Locking
Sometimes, we have to keep the panel doors open because bulky locks restrict closing the panel doors; we then need to think of ways to avoid non-compliance, because those should be kept closed.	Conflict of LOTO devise applications with other compliance requirements	Unexpected interference with existing controls	Lack of integration of information	Projection	Locking
Locks are installed on the main energy isolation panels; then we find it difficult to handle neighbouring controls because those are physically blocked	Locks blocking access to neighbouring controls	Restricted access to controls	Lack of integration of information	Projection	Locking
We sometimes cannot install locks and have to manage only with the tag; though we switch off and block the cut-off with alternative methods, we are not certain of accidental activation of power.	Insecure feeling with accidental energy connection when locks not used	Uncertain risk with alternative blocks	Lack of integration of information	Projection	Locking
We are not sure why we are asked to follow certain steps in locking out; there are alternative ways to perform the same task	Absence of clear understanding of the purpose of doing energy cut off	Not knowing the task purpose	Lack of understanding of context	Projection	Locking
Descriptions in some of the procedures are not clear and locating the hazard zones is difficult	Difficult to locate hazard zones due to unclear procedures	Identification difficulty of hazard zones	Poorly presented information	Comprehension	Decommissioning
Some of the visuals used in illustrations are not similar to others used in the organization, leading to difficulty in understanding them	Difficult to interpret illustrations in procedure documents due to absence of unique illustration methods	Difficult to understand illustrations	Poorly standardized information	Comprehension	Decommissioning
We see different flow of information in procedures, so we are not very certain whether all the important points are covered in the procedures	Difficult to interpret procedures due to lack of standardized content and flow	Difficult to understand procedures	Poorly standardized information	Comprehension	Decommissioning
We are not certain what exactly to write on the tag, and in many cases, writings on the warning tag contain different fields of information, so we need to contact the person who wrote it asking clarifications.	Writings on the tag cannot be comprehended due to absence of clear structure	Comprehension difficulty of tag content	Poorly organized information	Comprehension	Tagging
There are confusions with colors of LOTO tags in comparison to colors of other warning signs and safety notices at the workplace.	Difficult to interpret tags by their colors due to lack of standard	Comprehension difficulty of tag colors	Poorly standardized information	Comprehension	Tagging
Tags are sometimes used to denote maintenance work that is planned but not under execution, so we are not sure whether those are really under repair, although this is indicated with tags	Difficult to interpret the application of tags due to usage of them for other purposes	Confusion with unintended use of tags	Poorly standardized information	Comprehension	Tagging

presents the complete coding frame with distinct meaning units (N = 27) and the developed subcategories and categories.

In summary, regardless of the main category, lack of integration of information (n = 6) has the highest coding frequency of the subcategories. This is followed by poorly standardized information (n = 4), information not being made explicit (n = 4), and a lack of understanding surrounding the context (n = 4) (see Figure 4).

These issues can affect different levels in SA; for example, a lack of integration of information makes comprehension difficult. Therefore, the second research objective was to classify the issues based on the SA levels affected and the respective LOTO system components.

4.3. SA Levels Affected

The subcategories (presented in Figure 4) are data-driven, but major categories are concept-driven. Therefore, the outcome of the meaning units for our two major categories served our second objective. The first part of the second objective refers to classifications for the SA levels affected. Meaning units (N = 27) were categorized under perception (n = 4, 15%), comprehension (n = 12, 44%), and projection (n = 11, 41%) (see Figure 5a). Overall, comprehension and projection appear to be highly affected, but the perception of information is not. This finding is different in other domains; in a study of aviation, for example, 76% of the pilot errors were traced to problems in perception, and 20% were associated with comprehension [14]. However, it should be noted that the aviation study used retrospective analysis; we present how users perceive the LOTO ability to support SA. Despite the difficulties, users might be able to comprehend the situation and project future events by exerting more cognitive effort. This may not be the result in a retrospective analysis.

The three SA levels are not linear, but rather ascending [40]. SA is not a process; it represents the operator’s mental model of the state of the environment [14]. Therefore, it is possible that some of the SA requirements will not exactly fit into a single SA level. However, our QCA exercise enabled a disclosure of the main issues, which were categorized into different SA levels, making it possible to design specific SA interventions particular to those levels. We discuss this further under recommendations.

4.4. LOTO System Components Affected

The second part of the second objective was to identify and understand the affected LOTO system component. This understanding is vital for prioritizing interventions. The locking component of the LOTO system has the largest number (n = 11, 41%) of meaning units referring to a lack of SA, when all the SA levels are taken into consideration. Decommissioning (n = 9, 33%), tagging (n = 6, 22%), and recommissioning (n = 1, 4%) follow sequentially (see Figure 5b).

This result was not anticipated, as we built our justification for the study mainly on the ability of tagging to aid awareness of what is going on. Interestingly, decommissioning and locking surfaced as SA issues. The greatest number of SA requirements were identified for assessing the equipment status; here, the LOTO system components of decommissioning and locking appear more relevant. A number of SA requirements were identified with the sub goal of communication between employees (more related to tagging), while QCA reveals many SA-related issues in tagging. Notably, despite mentioning a rather high number of SA requirements for recommissioning, participants did not highlight many SA-related issues with the current LOTO system during the re-energizing phase. Only one specific incident was bought to our attention; in this case, a lithography machine was severely damaged when the machine was put back into operation, as a tool had been left inside.

4.5. SA Level Affected and Respective LOTO System Component

Figure 6 presents the coding frequency for different LOTO components with the respective levels of SA affected. The figure shows the sequence of decommissioning, locking, tagging, and recommissioning with the SA levels affected at each stage. This visualization sheds light on the issues associated with different SA levels throughout the LOTO procedure.

The distribution of coding frequencies for different LOTO components with respective levels of SA affected gives insight into the completeness of the state of the employees’ knowledge [14] at each LOTO stage. Although there were no perceptual challenges at the decommissioning stage, comprehension of that information seemed to be challenging. In contrast, what was challenged most at the locking stage was the projection of future status. Tagging was equally problematic for perception and comprehension. Finally, recommissioning was the least challenged component; here, only perception seemed to be challenging. In general, perceptual challenges were the least often found in this study. They became more prominent at later stages: tagging and recommissioning.

In the section on SA issues (Section 4.2), we discussed factors affecting SA in general. Above, we presented the different SA levels and LOTO components affected. We now look at the causal factors that are specific to those effects. First, we establish the key difference between two types of assessments. More specifically, the assessment of the activities performed in SA is different from the assessment of the result of these activities. If the objective is only to assess SA (whether or not one is aware of a situation), then the manner in which one becomes aware of a situation is not important [17]. However, since our focus was to investigate what hinders each SA level, we also looked at difficulties during the process of gaining SA. Subcategories of the SA levels affected can reflect causal factors that hinder those levels. Figure 7 shows the causal factors on each SA level.

There are a few notable characteristics of the SA levels and their causal factors (Figure 7). The integration of information is shown to be a major requirement for developing Level 2 SA [14]. However, four out of six ‘lack of integration of information’ cases affect SA up to the projection level. For example, in the absence of integrated information (number of locks installed, when they are installed, size of locks to use, etc.), it is difficult to predict the aftermath of installing locks on the disconnect switch; this ultimately hinders the closure of the electrical panel door, opening more risk opportunities (see Figure 8).

Second, although lack of understanding of the context seems to be more relevant to loss of Level 2 SA (comprehension), situations such as lack of understanding about the circumstances of blocking (instead of locks) affect the projection of consequences, and this ultimately hinders some other operations. Long-term memory stores in the form of schemata and mental models can assist in making projections of the risks even with incomplete information [14]. However, novel situations caused by frequent alterations demand that projections be made with limited working memory. Designs that clearly reflect the system’s alterations and associated risks can assist in the development of a correct mental model.

Figure 9 shows the causal factors for each LOTO component; the analysis leads to two major discoveries. First, the lack of integration of information is a major issue at the locking stage. This suggests that care should be taken in the selection of locks and allowing for blocks. Second, a lack of understanding of the context is a major issue during decommissioning. Since there are challenges beyond what is covered in the documented procedures, there is a need for a more comprehensive assessment of scenarios that arise during decommissioning.

Overall, Level 2 SA (comprehension) is challenged, predominantly at decommissioning. Comprehension requires putting together the knowledge elements of Level 1 to form patterns (gestalt), which enables forming a holistic picture of the environment [40]. At decommissioning, workers have to deal with disparate data, including interconnections between energy sources, salience of their presence, multiple work instructions, illustrations that differ from alterations, documents with different standards, etc. Meaningful integration of these disparate data, filtered through their relevance to the goal of safe energy isolation, yields safe decommissioning.

At first glance, LOTO seems to be in a static state from the time it is installed until it is removed. A major question addressed by this study was whether SA really matters when the state is static. This study makes an important point here: we did not focus on the situation assessment of a single state in LOTO, but rather on an ongoing and continuous process for acquiring SA in a dynamic and time-critical environment. Therefore, we established the dynamics of the LOTO environment. We were most concerned with the variability of the information on machine status; these include, for example, whether equipment was under maintenance or not, what level of maintenance had been completed, what modifications had been made, and the change of status followed by a shift changeover. We were also interested in the status of a group of machines locked together and situations where group lockout was being applied. Finally, an updated goal status of production and the tactical planning required to meet those plans created more dynamics. Although documented procedures and technical manuals are viewed as static information, they can incorporate attributes of dynamics in situations when machine are modified and locking devices are altered. Goals of accomplishing a production target and tight expectations of a machine bought back to life can influence how attention is directed to LOTO, how information is perceived, and how that information is interpreted. When top–down processes of goal accomplishment operate on par with bottom–up processing of perceived information, SA is required. In this context, salient cues should activate appropriate mental models of the situation, leading to correct decisions. Therefore, a focus on SA is vital to facilitate the appropriate mental model by ensuring the appropriate design of LOTO equipment and processes.

5. Recommendations

In the discussion, we established the dynamics involved with LOTO and thus the importance of SA. In this respect, the ideal scenario would be to rectify those dynamics as much as possible, for example, by performing the least possible number of alterations on a machine. However, the dynamic information requirements that we found in the GDTA suggest the need for a more pragmatic approach, sometimes going beyond what OSHA standards require; in such cases, SA is important. The design of a LOTO system should focus on helping employees develop a correct picture of what is going on with the machine and the environment.

The third objective of this study was to prioritize the critical SA issues of this organization’s LOTO system and make recommendations to rectify the issues. We found that a lack of integration of information, poorly standardized information, information not made explicit, a lack of understanding of the context, and inadequate processes are the major issues of the LOTO system affecting SA. Endsley [18] provided a detailed and systematic methodology of the design principles of SA interventions. Below, we briefly explain how SA interventions could possibly rectify the SA issues in LOTO.

5.1. Integrating Information

The lack of integration of information subcategory includes several issues: difficulty identifying interconnections, confusing multiple work instructions, unexpected interference with existing controls, restricted access to controls, and risks with alternative blocks. Under time pressure, these conditions can lead to LOTO violations. Organizing information around the goal rather than following a technology-driven approach can help identify the goals and the information needed for each goal. For example, the goal of assessing the equipment status can be assisted by providing integrated information about the status of the interconnections between sources, not merely about the status of individual power sources. At best, this information can be made explicitly available at the point of operation (see Figure 10); otherwise, explanations of how to obtain it can be provided. Similarly, confusing multiple work instructions are often an adverse effect of a technology-based approach; integrating them based on what is required by the goal is preferable. Data-driven processing can be assisted by making information available on what is happening in the context of other employees, other machines, and tactical production plans. What is critical here is supporting trade-offs between goal-driven and data-driven goals in a such way that those complement each other. For example, such trade-offs are often needed between the goals of the timely accomplishment of work schedules and data-driven information on the context; high salience of either type of information can affect the SA. The design of a LOTO system should consider how the user can switch between the two modalities.

5.2. Providing Consistency and Standardization

The poorly standardized information subcategory includes difficulty understanding illustrations and procedures, difficulty understanding tag colors, and confusion regarding the unintended use of tags. SA interventions for consistency and standardization directly address this issue. SA interventions using techniques to ensure logical consistency can reduce inconsistencies in the system by making consistent presentations of information and illustrations, the modes they represent, and the formats used in the presentation. For example, the closure of the valve shown in Figure 11 is anti-clockwise, which is not consistent with others, so it requires specific information. Interventions to map system functions to the goal and mental modes of the user can assist standardization. Mapping enables the operator to understand how the system works and how it is connected to achieve goals. Grouping information based on Level 2 and 3 SA requirements and goals can provide the basis for standardization and help organize the information. For example, all the information that is needed to achieve the goal of assessing conditions for safe re-energizing would ideally be grouped together; at least the sources of the information could be grouped and presented to the technician.

5.3. Making Information More Explicit

Information not made explicit was reflected in participants’ comments on uncertainty about updated procedures, not knowing what alternations have been made, what alternative blocks are needed, and the difficulty of knowing whether LOTO should be applied at all. Attempts should be made to make information explicit. For example, in the case of a machine being stopped, there should be no uncertainty about whether it is locked out; this information needs to be very explicit. Whenever no information is available, such as in the absence of a tag, it is important to explicitly identify missing information. SA interventions using data salience can support the operator in assessing the certainty of information. For example, in assessing stored energy, estimated information should be presented as ‘estimated’, together with the accuracy of that information, if possible. Supporting uncertainty management activities promotes awareness of the situation and the certainty of the information.

5.4. Improving the Understanding of the Context

A lack of understanding of context includes not understanding the documented content, not knowing the risk factors, and not knowing the exact purpose of LOTO. Supporting comprehension by presenting Level 2 SA directly by integrating information can assist in the understanding of the meaning of perceived information. For example, in the case of a block applied to support a part of a machine in place, it is more meaningful if the load applied on the block, as well as the load-bearing capacity, is explicitly presented. In fact, there was an accident in the organization when workers depended on a single door damper instead of two while conducting a maintenance activity. Interventions to provide system transparency and observability can improve the understanding of the system. For example, schematics can provide system transparency by presenting how actuators are linked together. Whenever the direct presentation of comprehension is not possible, further SA interventions can help. Making critical cues for schema activation more salient can improve the understanding of a situation by referring to a prototypical situation. For example, in addition to switching off a machine using a disconnect switch, it is important to ensure that controls are in the off position to avoid unexpected activation when the machine is re-energized. As shown in Figure 12, it helps if all off positions are aligned. As mental models and schemata play an important role in achieving high SA, it is important to trigger the operator’s schemata with obvious information from the system.

5.5. Improving Adequacy of Procedures and Validity of Information

Inadequate procedures and outdated information hinder comprehension. SA interventions presenting information with timelines can support temporal awareness and thus promote the awareness of outdated information. The management of change procedures can enforce the requirements to keep updated information about machine alterations. Although minimizing task complexity can lessen the demands for detailed procedures and frequent updates, the complexity of the maintenance task is largely attributed to how well the machine has been designed for maintenance [42] by the original equipment manufacturer (OEM). Within the scope of LOTO, minimizing the complexity applies during de-energizing and re-energizing, particularly by making it easy to determine interconnections between different energy sources. When dealing with multiple sources of information, SA interventions for assessing confidence of composite data can aid the operator in appraising the reliability or confidence level of the information (e.g., coming from different sensors). This level of appraisal is required to determine the fault and the need for maintenance; it is also required during de-energizing and re-energizing phases. It is indirectly associated with determining the extent of the task required, tool requirements, and estimated time for completion.

5.6. Improving Communication Structure

Poor communication structure hinders the projection of risks involved with the locking component. When the operator is pursuing a maintenance or production goal, attention is usually directed toward a subset of information. As a result, the operator may fail to acknowledge other problems in the environment. Supporting global SA means giving the operator the ability to attend to information about the overall status of the system at all times. Interventions to support global SA can aid awareness by improving communication among team members and creating a holistic situation. Communication is particularly important when maintenance is performed by a group of technicians. Group LOTO operations typically require more coordination and communication than single-person LOTO operations. Greater coordination between employees is particularly important when more than one department is involved in the task. Design principles suggested for facilitating team SA can be useful in the collaborations of technicians, supervisors, and machine operators demanded by the LOTO process. Further interventions to support the transmission of different comprehension across teams and a shared mental model will result in more efficient communications by reducing misunderstandings.

6. Conclusions

We set up this study to show the importance of the SA concept for companies wishing to determine the efficacy of their LOTO systems. Our use of GDTA revealed decision requirements under a dynamic context and indicated what SA information is required. Our classification using QCA found that the comprehension and projection levels were more affected. Out of the four major components of LOTO, the locking component was found to be most affected, and in the locking component, the projection level was most affected.

As this study shows, SA interventions can be used to mitigate high-priority issues. The intent of LOTO is to make workers aware of what is going on with machines undergoing maintenance; thus, SA is well suited to attempts to improve LOTO effectiveness through special interventions. One major conclusion that we can make is that users should be involved in risk evaluation; their input will identify latent risks of a cognitive nature that might not be captured by regular physical risk assessments performed by experts.

Overall, this study confirms that SA is an applicable concept for evaluating and improving the effectiveness of LOTO systems, despite the somewhat static nature of LOTO applications. Varying production demands, shifting operations, machine modifications, and the progression of maintenance work itself make SA an important issue with LOTO. Acknowledging that these systems are and will continue to be used by humans and understanding how they can serve their primary purpose of making humans aware of what is going on will make LOTO use more effective. This study suggests the need to prioritize the SA intentions of LOTO, ultimately to make a safer workplace.