Analysis of Results from Event Investigations in Industrial and Patient Safety Contexts

Abstract

Accident investigations are probably the most common approach to evaluate the safety of systems. The aim of this study is to analyse event investigations and especially their recommendations for safety reforms. Investigation reports were studied with a methodology based on the characterisation of organisational levels and types of recommendations. Three sets of event investigations from industrial companies and hospitals were analysed. Two sets employed an in-depth approach, while the third was based on the root-cause concept. The in-depth approach functioned in a similar way for both industrial organisations and hospitals. The number of suggested reforms varied between 56 and 143 and was clearly greater for the industry. Two sets were from health care, but with different methodologies. The number of suggestions was eight times higher with the in-depth approach, which also addressed higher levels in the organisational hierarchy and more often safety management issues. The root-cause investigations had a clear emphasis on reforms at the local level and improvement of production. The results indicate a clear need for improvements of event investigations in the health care sector, for which some suggestions are presented.

1. Introduction

Safety is an elusive notion with various principles for how it should be achieved [1]. An additional complicating factor, when it comes to evaluation of the safety performance of a system, is that it is often changing over time and could be described as dynamic. An accident investigation is a common approach to evaluate the safety of a system. However, it is not an unambiguous tool because there is a large variation of aim, scope, methodology, and execution [2,3,4].

The scope could be wider and also include the study of critical incidents and other events. The term event investigation is therefore preferred here, and it is defined as ‘the collection and examination of facts related to an occurred specific event’ [5]. Independently of how such investigations are performed, they will be documented in some way. The investigation reports could be used as a basis for analysis, which in some way will reflect the safety performance of an organisation.

The starting point of an investigation is when something has gone wrong in a system. The intention may be to give an understanding of what happened and how safety could be improved. Within various sectors, there are different traditions in how such information is obtained and used. In industry, event investigations are central to efficient safety management and a basis for learning and safety improvements [6,7]. The choice of investigation methods has an essential influence on the results. In the industrial arena, this has attracted interest for a long time, and there have been several reviews of investigation methods and their use [1,8,9,10].

In the health care sector, investigation methodologies have received increasing attention. Vincent et al. (2017) have called for a reconsideration of the benefit of incident analysis and the appropriate methodology to be used. Their conclusion is that previous methods of incident analysis were applied with too little research and evaluations. In England and Norway, national, independent safety investigation bodies have been created in the health care sector [11]. A number of issues are highlighted, and one important challenge is to develop systems-focused investigation methods. The handling and learning from sentinel events in the Netherlands have been examined [12]. The aim of the study was to use the findings to take forward a national approach to improve learning from adverse events.

Thus, there appears to be a rather common interest in developing event investigations. One suggestion [4] is to conduct fewer and deeper analyses, which could replace a larger number of quick and formal analyses. In these authors’ view, there is a need for substantial research in the development of new methods. They also suggest a number of improvements that could improve the value of event investigations.

Within the health care sector, several studies have been published where methods and approaches are scrutinised and compared. An analysis of investigation reports from a university hospital [13] concluded that there are a number of insufficiencies in the generation of reforms and how they are accomplished. For example, the investigations were mainly targeted at the micro-level of the organisation, giving a too limited perspective on system failures. Peerally et al. (2016) studied recommendations generated by event investigations. Their conclusion was that most recommendations are weak and that few of those address fundamental system issues. In another study, 227 event investigations based on root cause analysis (RCA) in health care have been analysed [14]. The focus was on recommendations in order to evaluate their quality. A finding was that few (8%) of these were judged as ‘strong’, which means that most recommendations are unlikely to be effective or sustainable.

There is also a considerable interest to advance the field of event investigations. A number of studies are reported, where specific methods are used and tested in health care [15,16,17,18]. These examples are based on systems-oriented approaches, and they include experiences from other high-risk industries.

The author’s conclusion from this short review is that there is a recognised need for improvements related to event investigations, especially in the health care sector. This need includes a clear systems perspective both on methods and how results are used. It is also essential to learn from experiences in different sectors.

The general aim of this study was to contribute to the ongoing development of event investigations and to explore how reports from event investigations can be analysed and compared. A goal was to develop a methodology for analysing recommendations from event investigations. The goal also included analysis of reports from previous investigations, which emanated from events in industrial and health care organisations. Another goal was to make systematic comparisons of the results from industry and health care. The investigation reports were based on different approaches, however, the aim was not to evaluate the specific investigation methods.

This paper presents in the introduction examples of problems and needs related to event investigations in industry and health care. Section 2 gives an account of the stages in the study, which includes a system for the classification of suggested reforms. It also includes a description of the studied material, which consists of three sets of event investigations from industrial companies and from hospitals. Section 3 gives an overview of results, which also includes an examination of safety features identified in two of the sets. Section 4 presents a detailed analysis of suggested reforms, a comparison between industrial and health care cases, and a summary of how event investigations could be improved. Conclusions are presented in Section 5.

2. Methods and Material

2.1. Analysis of Investigation Results

2.1.1. General Approach

In general, there are several difficulties in comparing investigation approaches and results [2]. The intention of this study was to find ways to handle some of these difficulties. In order to do that, a methodology has been designed for analysing reports from event investigations. Considering the vast use of such investigations, the application area has been limited to the following:

• Organisational activities, such as industry and health care;
• Investigations documented in writing.

The methodological part of the study covers a number of steps as follows:

• Selection of parameters to study and classify (see below);
• Developing a scheme for the classification of reforms (see

  Table 1. Hierarchy of levels where reforms should be made, with examples from industry and health care.

  Code	Level	Definitions	Industry	Health Care
  1	Government and Regulators	Issue binding regulations	Authorities	Authorities
  2	Association	Issue recommendations	e.g., employer organisation	e.g., association of local authorities
  3	Corporate group	Coordination and control of establishments	Company group	County, regional health care organisation
  4	Establishment	Includes coordination between sections	Factory	Hospital, or a regionally responsible unit
  5	Section	Includes coordination between local units	Production department	Health care department
  6	Local	Where work is done, includes maintenance, etc.	Worksite	Ward
  7	Individuals	Concerns what individuals are supposed to do	Industrial professionals	Health care professionals
  9	Other	Actor not defined; general suggestion, vague description

  and

  Table 2. Types of suggested reforms in event investigations.

  Code	Type	Definitions	Comments, Examples
  1	Technical equipment	Properties and handling of technical equipment, including software
  2	Human resources	Intended to change the competence and behaviour of individuals	Training, recruitment of staff
  3	Production planning	Production management, incl. design	Includes descriptions of how work shall be done and production procedures
  4	Rules and instructions	Intended to regulate the behaviour of organisations and staff
  5	Safety management	Intended to improve or maintain safety, includes work environment	Safety organisation, risk analysis, event investigations, safety rounds etc.
  6	Investigate and explore	Intended to increase knowledge as a basis for decisions on further actions
  9	Other	Unclear or vague description of what shall be done

  );

• Selection of event investigations which should be examined. These are described in Section 2.2. The investigations were grouped into three different sets. Two different types of approaches for event investigations are included, and these are described in Section 2.3;
• Each suggested reform in the investigations was coded according to the classification scheme;
• Coding of the time frame for each investigation;
• Summing up the codes for the reforms;
• Special investigation of safety features for sets 1 and 2.

2.1.2. Selection of Parameters

The number of aspects to analyse has been limited to just a few. An important area of interest is the characteristics of the recommendations generated by investigations, which have been addressed by several authors [3,13]. An important aspect is to consider a systems perspective in the analysis of reforms. Moreover, a time perspective is of interest, since a wide time frame in the investigations might be important [4]. Based on these considerations, three characteristics were selected for analysis. These were:

• Organisational levels for reforms addressed by the investigation:
  A categorisation of levels where reforms should be made was developed (see Table 1);
• Type of recommendations provided by the investigation:
  A categorisation of different types of reforms (Table 2) was constructed.
• Time frame of investigation:
  The time frame was defined by the first and the last event mentioned in the report.

2.1.3. Classification of Reforms

For the study of organisational levels, a starting point was Rasmussen’s (1997) hierarchical model [19]. Here, the two highest levels—government and association—are general, and they are used directly. At lower levels, adaptation was made to a structure that considers how large organisations also have an internal hierarchy. Table 1 shows the classification used in the coding.

It is a simplification of a complex reality, and in order to support the coding, a few rules were applied. Firstly, if a reform addresses more than one level, it will be coded at the higher level. The second rule is that if a reform concerns an outside actor, such as a consultant or a contractor, the code addresses the level of responsibility within the organisation.

The classification was also made of the types of recommendations provided by an investigation (see Table 2). The division is fairly conventional, with an orientation towards technique, personnel, and management. The management part is subdivided into four groups (codes 3 to 6).

Coding rules are needed also here. Sometimes, a reform can refer to more than one category. The coding rule was that if a suggestion concerns how a specific task shall be done, it is coded as ‘production planning’, even if it is preceded by a rule or an investigation. A second rule is that if the reform directly concerns safety in some way, it is coded as ‘safety management’, even when it is related to rules or investigation.

2.1.4. Coding of Suggested Reforms

Each report in the studied material had a separate list of recommendations, and each reform was classified according to Table 1 and Table 2. In the first stage, this was carried out by the author. In order to improve the reliability of the coding, specialists from occupational safety respectively from patient safety scrutinised the results. Based on this, a final coding of the results was made.

2.1.5. Special Investigation of Safety Features

In their review of improvements, Vincent et al. (2017) had noted a need to understand success and failure in detection and recovery in event investigations. This type of information is not always available. However, some aspects can be observed in the results from the methods deviation analysis and safety function analysis, which were applied to the investigations in sets 1 and 2 (see Section 2.2 and Section 2.3). The study approach and the findings of related to safety features are described in Section 3.3.

2.2. The Studied Material

The methodology was applied to event investigations reports, which have been available to the author. The studied material is based on three sets of event investigations, which were done a few years before this study.

Set 1

Thorough Investigations in Industrial Companies

The first set consists of four investigations of occupational accidents in industrial companies. The accidents had caused serious injuries, and the companies involved wanted a thorough investigation. They had engaged an independent investigation leader, and the aims were to understand how the accident could have happened, and how safety could be improved. All four investigations were performed using an identical methodology (see Section 2.3), and with the author as investigation leader. The results are largely confidential.

Set 2

Thorough Investigations in Health Care

The second set consists of three investigations of incidents in hospital care. The study and the results have been published earlier [20,21]. The general aim was to test system-oriented methods for the investigation of incidents in a hospital environment. All three investigations were performed with the same methodology, identical to Set 1.

In this second set, three pharmaceutical incidents in a hospital were investigated because they represent a common type of problem. One criterion for the choice was that the patient should not have been injured. The motivation for this was to avoid situations related to blame or guilt, which is often regarded as a major problem in the health care sector. Other criteria were that the incidents should be as simple as possible and that the actual ward should participate voluntarily. The three investigations concerned:

Case 1: At an emergency ward, a doctor wrote a ‘2’ instead of a ‘1’ by mistake in prescribing medication to a patient. The error was discovered and the patient did not get a double dose;

Case 2: An unconscious patient was to get nutrition through a feeding pump. It was found that the pump did not work, even though the measuring equipment indicated a satisfactory flow and the monitoring system had not given any alarm. The patient’s blood glucose value became too low, but the patient was not harmed because counter-measures were taken instantaneously;

Case 3: A patient was registered for planned surgery, but the ordinary medication was not included on the new list of prescriptions. This mistake was discovered when the patient asked the nurse about it a few days later.

Set 3

Standard Investigations in Health Care

The third set consists of 17 event investigations from a regional health care organisation. These investigations were done according to a methodology provided by the National Board of Health and Welfare [22]. This set has been included in the study because it can be considered to roughly represent the results from common event investigations in Sweden. All the studied events had serious consequences.

2.3. The Methods Used for Investigation of Events

2.3.1. The Investigation Approaches

There are a large number of methods for the analysis of events, with one summary suggesting that there are more than 100 separate methods available [10]. ‘Event’ is here used as a general term including accidents, near-accidents, incidents, and ‘adverse events’; the latter phrase is often used in health care, usually associated with an injury.

The events in Sets 1 and 2 in industry and health care were investigated with the same in-depth approach. This is based on a combination of three separate methods and is here called the combined method approach (CMA). In Set 3, the officially recommended method had been applied, here called the standard Swedish method (SSM).

The investigation approaches have a few common features. Their general aims were to create an understanding of how the events could occur, and to suggest reforms and improvements. Another common aspect is the adoption of a no-blame culture in the investigation, which is supposed to support a system perspective.

2.3.2. The Combined Method Approach

The investigation approach in Sets 1 and 2 is based on the application of three separate methods. The methods are simple event mapping, deviation analysis, and safety function analysis. These methods are earlier described in detail [23], and only a short summary is given below.

Simple event mapping is a simplified version of the method STEP [24]. The principle is to identify and arrange the individual events in time order, related to the incident sequence. The result can be a table or in a graphical form.

In deviation analysis (DA), the focus is on actual deviations in relation to the incident or on permanent deviations. A deviation is an event or a state that diverges from the correct, planned, or usual function. The method includes four principal stages. The first is the identification of deviations based on interviews and other information. This is followed by arranging the deviations in a suitable structure. At the third stage, the deviations are assessed in order to judge whether safety improvements are needed or not. The fourth stage includes the development of preventive measures [23].

Safety function analysis (SFA) was used to study the safety features that were in place and could have prevented the incident. It is based on the general concept of ‘safety function’, which is defined as a function contributing to reducing the risks in a system [21]. It is a generalisation of the more common concept of ‘barrier’, and is also in line with ‘defence’, defined by Reason (1997) [25] as ‘various means by which the goals of ensuring the safety of people and assets can be ‘achieved’. Another example is the ‘barrier function’, which can ‘arrest incident evolution so that the next event in the chain will not happen’ [26].

The SFA has five major steps [23]. The first is the identification of safety functions, based on interviews, documents, etc. At the next step, the safety functions are arranged as a list in a suitable structure. This is followed by an assessment of each function whether it worked or not and whether improvements are needed. The final steps concern the development of improvements and the report.

The combined method approach can be summarised in seven steps as follows:

(1)

Definition of investigation, aims, etc.;

(2)

Data collection through interviews, documents, etc.;

(3)

Check on data and the sequence of events;

(4)

Assessment of deviations and whether improvements are needed or not;

(5)

Assessment of safety functions;

(6)

Development of safety reforms;

(7)

Final report.

The investigation leader, in cooperation with a local investigator, was responsible for the data collection and data analysis during the first stages. At stages 3 to 6, a reference team played an important role. The team included members with different tasks in the organisation and with long experience of the operations, who were detached from the actual case. Their function was to represent the knowledge of the organisation and to make the judgements needed. The role of the investigation leader in the latter part of the investigation was to act as chairman and secretary.

2.3.3. The Standard Investigation Methodology

The investigations in Set 3 were based on a methodology provided by Sweden’s National Board of Health and Welfare [22]. Within the Swedish health care sector, this is the common method. Since it is recommended by the authority and is regularly used, it is here called the standard Swedish method.

The method is based on root cause analysis in a version originating from the Department of Veterans Affairs, USA. A manual has been developed and adapted to Swedish conditions. The investigation process included the following:

(1)

Initiate the analysis;

(2)

Collect facts;

(3)

Describe the sequence of events;

(4)

Identify contributing factors;

(5)

Develop reform proposals and the approach to follow-up;

(6)

Final report;

(7)

Decide about reforms, follow-up, and schedule for implementation.

An essential part of the investigation is to identify contributing factors. The rule of thumb is that for every error event the investigator should ask ‘why’, and for every ‘because’, ask a new ‘why’ until further asking becomes meaningless. In this way, a chain of events is created, which will lead to the underlying causes.

A central part of the Swedish version is a diagram, which is based on a chain of events. Above this, there is a hierarchy of boxes representing contributing factors. In this way, the analysis is summarised in a single graph. The manual for the method also includes support for finding causes, which are divided into the following five categories:

• Communication and information;
• Competence;
• Surroundings and organisation;
• Technique and equipment;
• Procedures, routines, and guidelines.

The method manual recommends an investigation team, which should perform the investigation together with an investigation leader.

3. Results with Comments

3.1. Overview of Studied Investigations

The studied material is based on three sets of event investigations. In Sets 1 and 2, the investigations were done on industrial and health care cases applying the same method—the combined method approach (CMA). Because one goal of the study was to compare these two fields, the findings are presented in some detail in order to show the variations between individual cases and between the fields. Set 3 is based on investigations in health care with the standard Swedish method, and the results are less suitable for such a direct comparison with industry. Therefore, the findings from this set are mainly presented as mean values in this section.

Table 3. Overview of event investigations with the combined method approach (CMA).

Set	1-Industrial Cases				2-Health Care Cases
Designation	Ic1	Ic2	Ic3	Ic4	Hc1	Hc2	Hc3
Identified actors (n) *	10	12	15	15	11	18	9
Time frame **	2 m	2 m	16 y	16 y	2 m	14 m	3 m
Identified events (n)	43	44	34	30	11	38	16
Proportion after the critical incident	49%	36%	15%	30%	64%	53%	75%

* (n) indicates the number. ** The time is given in months (m) or years (y).

shows a few general characteristics from Sets 1 and 2. The first concerns the number of identified actors, who are defined as having a function with a specific task or assignment related to the event. It could be an organisational function or an individual with a defined role. In the number of actors, the emergency services (after the incident) are not included. The number of actors mentioned in the reports varied between 9 and 18. There is no clear difference between the two sets, and the mean value happened to be 13 for both.

The time frame of an investigation is defined as the time between the first- and the last-mentioned event. Table 3 shows that around two months is most common, but two of the industrial investigations had a start time 16 years before the incident occurred. In both cases, the event was concerned with initial design and installation. Over the years, there had been changes that caused later problems.

Another characteristic is the number of events identified in the investigation reports. The variation is large, by a factor of four between 11 and 44. The mean value is 38 for industry and 22 for health care. Moreover, the proportion of events after the critical incident is presented. It was fairly high, with a mean value of 32% for industry and 64% for health care. A higher value for health care could be expected because these investigations concerned incidents without injury and also included a detection and recovery phase.

A general observation is that the reference teams in all the investigations played an important role. The participants had a long experience of work on the site. It was considered important that the selected persons were not directly involved in the investigated event.

Set 3

The third set of studied investigations consists of 17 event investigations from a regional health care organisation. All the studied events had serious consequences, and seven had a deadly outcome. The information from these investigations is less detailed, and the data in this set is mainly presented as mean values.

The investigations are based on the standard Swedish method [22]. The standardised protocols from these investigations have all summed the amount of working time the investigations took. The mean value was 33 working hours, and time varied between 20 h and 57 h. That is considerably less than for the Sets 1 and Set 2 investigations.

One aspect is the time frame of the investigations, and here the variation is large. The median value for the starting point of the investigation was roughly 16 weeks before the critical incident, and slightly longer for the time period after. The longest time interval before was 16 years, and in another case, the longest time after was over a year. The shortest time period was found in two investigations, which were one day before and one after the critical event.

In these investigations, different investigation teams were engaged in which the members represented various fields of expertise.

3.2. Analysis of Suggested Reforms

An essential part of this study was to analyse results from event investigations in order to test the methodology and to explore similarities and differences between the three sets of reports. A major aim of the studied investigations was to develop and propose reforms that could improve the safety of the system. All investigation reports presented several suggestions for change. Responsibility for the accomplishment of the reforms was usually suggested in the investigation reports.

The number of suggested reforms varied considerably. The mean value for the industrial cases (Set 1) was 108 reforms per investigation, and for the health care cases (Set 2), it was 63. For Set 3 employing the standard Swedish method, the mean value was 8.2 suggested reforms per investigation.

The suggested reforms have been analysed regarding who should be responsible for their accomplishment, and types of reform. For Sets 1 and 2,

Table 4. Distribution on organisational levels of reforms from the combined method approach.

Set		1-Industrial Cases				2-Health Care Cases
Designation		Ic1	Ic2	Ic3	Ic4	Hc1	Hc2	Hc3
Number of suggestions		88	96	143	103	60	73	56
Organisational level
1	Government and Regulators	0	0	0	0	5	12	3
2	Association	0	0	0	0	0	0	0
3	Corporate group	39	10	42	13	44	4	21
4	Establishment	24	81	38	69	6	10	6
5	Section	19	0	56	14	5	42	26
6	Local	6	5	7	7	0	0	0
7	Individuals	0	0	0	0	0	1	0
9	Other	0	0	0	0	0	4	0

shows the distribution between organisational levels, based on the hierarchical model presented in Table 1. The distribution varies considerably and is scrutinised further in Section 4.2, which also includes detailed results from Set 3.

For the same sets of investigations,

Table 5. Distribution of types of reforms from the combined method approach.

Set		1-Industrial Cases				2-Health Care Cases
Designation		Ic1	Ic2	Ic3	Ic4	He1	He2	He3
Number of suggestions		88	96	143	103	60	73	56
Type
1	Technical equipment	7	2	7	21	3	5	0
2	Human resources	7	8	4	7	6	4	3
3	Production planning	12	25	57	12	13	7	16
4	Rules and instructions	2	0	10	5	13	5	13
5	Safety management	56	57	42	46	15	31	10
6	Investigate and explore	4	4	23	12	10	21	13
9	Other	0	0	0	0	0	0	1

shows the distribution between different types of suggested reform, as defined in Table 2. There are apparent discrepancies, especially for safety management, where there is a greater emphasis in industry.

Table 4 and Table 5 have provided detailed information on the cases, and the main reason has been to give an impression of the variability. A broader analysis of the suggested reforms is given in Section 4.2, which also includes a comparison with results from Set 3.

3.3. Summary of Failures and Successes

In this study, one aspect was to explore the issue of failure and success. It was inspired by Vincent et al. (2017) who addressed a need to explore success and failure further in the detection of problems and of system recovery. It has been possible to obtain fairly concrete information about this for Sets 1 and 2, which used the combined method approach. The data come from the application of the methods deviation analysis and safety function analysis (see Section 2.3.2).

3.3.1. Deviation Analysis

The method ‘deviation analysis’ focuses on deviations which can give an estimate of problems in the system. Examples of deviations in the industrial cases include the following:

• Unclear responsibility for decisions during dangerous work tasks;
• Safety glasses with unsatisfactory functionality;
• Safety rounds not made;
• Technical design work disregarding potential weaknesses in the construction.

Some examples from health care cases are as follows:

• Local and central hospitals with different layouts for patients’ lists of drugs;
• Problems with information when prescribing a medication had been changed;
• The nutrition pump did not work;
• The measuring equipment for the pump had not shown that the pump had stopped.

For each investigation case, deviations had been identified and listed. Afterwards, the investigation team assessed the deviations whether there is a need to make an improvement or not.

Table 6. Deviations in event investigations with the combined method approach.

Set	1-Industrial Cases				2-Health Care Cases
Designation	Ic1	Ic2	Ic3	Ic4	Hc1	Hc2	Hc3
Deviations (a)	63	64	68	65	49	74	42
Assessed (b)	63	37	62	58	41	68	34
Improvement needed (c)	84%	95%	89%	95%	80%	87%	100%
Disagreement (d)	3	1	1	1	0	4	1

(a) The total number of deviations; (b) The number of deviations that were assessed regarding the need for improvement; (c) Proportions of the assessed deviations judged as in need of improvement; (d) The number of occasions when there was disagreement over the assessment.

sums up the results from the investigations. The total number of deviations ranged between 42 and 68. The proportion of deviations which were judged to call for improvements ranged between 80% and 100%.

Not all deviations were assessed, which could be due to insufficient information or lack of time. In the case of Ic2, only around half were assessed. The main explanation is that several deviations were allocated to the activities of another company. In addition, it was concluded by the reference team that the responsibility for these was beyond their scope so they did not want to make any judgment.

In the assessments, it was fully legitimate to disagree over the judgements, and these cases were simply noted in the protocol. The last row in Table 4 shows the number of disagreements, which is fairly low. When there was a disagreement, the higher value is used in this summary.

3.3.2. Safety Function Analysis

As a part of the combined method approach, the method ‘safety function analysis’ was applied to examine safety features (see Section 2.3.2). These might have contributed to the prevention of the incident or reduction of the consequences. Examples of safety functions from the industrial cases are as follows:

• Safety aspects in business contracts;
• Documented training of operators;
• Risk analysis of a planned work task;
• Safety inspection;
• Emergency shower (reducing consequences).

Examples from the health care cases include the following:

• Patient’s own knowledge of his/her disease;
• Nurse’s experience (reacts to an unusual drug);
• Local instruction for record of medication;
• Alarm for insufficient flow at intravenous medication;
• System for reporting abnormal situations.

For each investigation case, safety functions were identified and listed. In the next step, the safety functions were assessed and categorised into the following three main types:

(1)

The SF performed well in the actual case;

(2)

The SF worked partly, but not fully;

(3)

The SF did not work.

The results from the case studies are presented in

Table 7. Safety functions (SFs) in event investigations with the combined method approach.

Set	1-Industrial Cases				2-Health Care Cases
Designation	Ic1	Ic2	Ic3	Ic4	Hc1	Hc2	Hc3
Safety functions (a)	49	57	83	66	52	55	57
SF assessed (b)	42	43	66	44	35	44	46
SF performed well	13	9	9	8	12	14	8
SF performed partly well	1	4	20	4	8	12	12
SF working (c)	33%	30%	44%	27%	57%	59%	43%

(a) Total number of safety functions; (b) The number of safety functions that were assessed. (c) Proportions of the assessed safety functions that performed well or partly well.

, showing that between 49 and 83 safety functions (SF) were identified in the case studies. However, all safety functions were not assessed, usually because of lack of time at the team meetings. The table shows the number of assessed SFs and how they were assessed. The proportion of SFs that worked varied between 27% and 57%.

The proportion of working SFs was higher in health care cases. They concerned incidents with no injury, which means that the accident sequence was arrested and followed by a recovery phase. The favourable outcomes were due to individuals observing that something was wrong. The SFs that successfully prevented injuries are listed below:

Case Hc1: The patient did not comply with an incorrect new prescription, and a doctor on duty discovered the error and corrected it. The SFs can be described as the patient’s knowledge about his medication and an attentive doctor on duty;

Case Hc2: A nurse found a problem with blood glucose values, and actions were taken immediately. The SFs can be described as an attentive nurse and a recovery procedure;

Case Hc3: A missing prescription was revealed when the patient, a few days after surgery, asked the nurse about this. The SFs can be described as the patient’s knowledge about his medicine, and an attentive nurse.

4. Discussion and Analysis

4.1. The Categorisation of Safety Reforms

One objective of this study was to design and test an approach to analyse reforms suggested in event investigations. The scope of application was limited to investigations from organisational activities, such as industry and health care. The object of the analysis was the documented summaries of proposed safety measures. The focus was on two characteristics:

• Organisational levels addressed by the investigation;
• Type of recommendations provided by the investigation.

A description of the procedure for categorisation is given in Section 2.1, which includes codes for classification (Table 1 and Table 2). In order to support the coding, a few rules were formulated. The results of the coding can be seen, e.g., in Table 4 and Table 5. The coding was made in a few steps, and a preliminary version was done by the author. This was checked by two specialists from occupational safety and patient safety, in order to improve the quality of the outcome.

Sometimes, ambiguity concerned the coding of organisational levels. This could often be solved by the coding rules. The remaining issues could depend on the fact that organisations often are composed of units related to production and others with specialised functions. Other uncertainties were related to the involvement of separate organisations or subcontractors. The organisational complexity is illustrated in Table 3, which shows that between 10 and 18 actors were identified in each investigation. In Set 3, the organisational structure was found to differ from what had been anticipated, which led to several corrections, mainly from Levels 6 to 5.

Issues regarding the type of reform were somewhat less common. Reforms could refer to more than one category at the same time, or they could be indistinct. The coding rules gave some guidance, but not always.

The basic coding problem was related to the misinterpretation of levels in Set 3. Excluding that, it was found that on average about 10% of the codes were brought to discussion by the specialists. As a final result, slightly more than five percent were changed. In Table 4 and Table 5, it can be seen that only a few got code nine, indicating that they did not fit in the scheme. The common reason was that they were described too vaguely. As a whole, it can be concluded that the methodology worked well for categorisation of proposed reforms.

Improvements for potential future applications of the methodology could be more clear classification descriptions and rules. One example for the parameter ‘type’ is to add one category, which includes a combination of both production and safety. One lesson is that it could be advantageous to map the organisation involved before starting the coding process, considering that many actors often are involved.

4.2. Analysis of Suggested Reforms

One important aim of most event investigations is to propose safety improvements. All the studied investigation reports have given several suggestions for reforms. With a focus on the reforms, these can be analysed independently of the investigation approach. Section 2.1 describes the analysis and categorisations made, and how this has been applied to the investigations in Sets 1, 2, and 3. The parameters of special interest have been types of reforms and the level responsible for their accomplishment.

4.2.1. Organisational Levels of Reforms

The study has included an analysis of how the responsibility for reforms was addressed. This is based on the hierarchical categorisation presented in Table 1. A detailed account for Sets 1 and Set 2 is given in Table 4, which is based on the combined method approach. A summary has also been made for Set 3 based on the standard Swedish method.

Table 8. Organisational levels of reforms from three sets of investigations with mean values of distributions.

		Industry	Health Care
	Sets	1	2	3 *
	Number of investigations	4	3	17
	Average number of reforms per investigation	108	63	8
	Distribution mean value	%	%	%
Code	Organisational level
1	Government and Regulators	0	11	0
2	Association	0	0	0
3	Corporate group	24	36	1
4	Establishment	49	12	1
5	Section	21	39	83
6	Local	6	0	14
7	Individuals	0	0	0
9	Other	0	2	1
	Sum	100	100	100

* SSM = standard Swedish method, used in Set 3.

presents the mean values for organisational levels for the three sets.

There is an obvious difference concerning the number of proposed reforms. The combined method approach in Sets 1 and 2 generated considerably more suggestions than the standard Swedish method. When Sets 1 and 2 are compared, the following are observed:

• The health care investigations suggested several reforms (11%) that should be made at the national level (Code 1);
• The industry has the majority of reforms focused on Level 4, which here means factory level (49%);
• For health care, the largest proportion (39%) concerns Level 5—here a health care department.

In comparing Sets 2 and 3, the following are noted:

• The mean value for suggested reforms is eight times higher in Set 2;
• In Set 2, a large part of the suggestions addresses the higher Levels 1 and 3 (47%), while Set 3 has just 1%;
• Both sets have a majority of their reforms at Level 5.

For all three sets, Table 8 shows that there are no reforms addressing Levels 2 and 7. A tentative conclusion is that associations (Level 2) are not considered important stakeholders in the prevention of accidents, neither in the industry nor in health care.

4.2.2. Types of Suggested Reform

In analogy with the previous table,

Table 9. Types of suggested reforms from three sets of investigations with mean values of distributions.

		Industry	Health Care
	Sets	1	2	3 *
	Distribution mean value	%	%	%
Code	Type of reform
1	Technical equipment	9	4	2
2	Human resources	6	7	16
3	Production planning	24	19	51
4	Rules and instructions	4	16	10
5	Safety management	47	30	3
6	Investigate and explore	10	23	9
9	Other	0	1	9
		100	100	100

* SSM = standard Swedish method, used in Set 3.

presents mean values for the classification of types of reforms for all three sets. Comparing Set 1 and Set 2 by type, the following are observed:

• The largest type of reform is ‘safety management’ in both sets, and it is especially high for Industry;
• For Health care, there is a large proportion of ‘investigate and explore’ and ‘rules and instructions’;
• The type ‘production planning’ is fairly high in both sets;
• The type ‘human resources’ is small in both sets.

Sets 2 and 3 both belong to the health care sector, but they show considerable dissimilarity. The following are observed:

• The dominant type in Set 3 is Production planning¸ with more than half of the suggestions. This type of reform concerns work tasks and procedures. The suggestions were usually concrete descriptions of how changes should or could be made;
• The dominant type in Set 2 is ‘safety management’, which is almost missing in Set 3;
• In Set 3, the type ‘human resources’ is fairly common, in contrast to Set 2.

In Set 3, the types ‘human resources’ and ‘production planning’ add up to 66%, which is more than twice that in the other sets. One interpretation is that the investigations in Set 3 have a focus on professional medical issues of how they shall be performed. That can in turn imply that other aspects are disregarded or taken for granted.

The dominant presence of safety management in industrial cases is likely to be explained by the strong position of occupational safety in Sweden. There is a long tradition with obvious stakeholders and clear demands for a well-functioning system.

4.2.3. Broader Comparison

The analysis of the suggested reforms can be compared with results from the research of Wrigstad et al. (2014). They examined event investigations, which had been conducted with the same methodology as in Set 3, which here is called the standard Swedish method [22].

The study included 55 investigations with a total of 289 separate recommendations. These were analysed according to whom the recommendations were addressed. It was based on a categorisation into three organisational hierarchical levels—micro, meso, and macro. This categorisation differs somewhat from the one used above (Table 1), as it is broader. However, a translation can be made in order to make some comparisons [27]. Macro corresponds well to Levels 1, 2, and 3 in Table 1. The translation to meso and micro is rather uncertain; especially the reforms belonging to Level 5 could come in either of these. In order to make a tentative comparison, Meso is supposed to include Levels 4 and 5, and Micro Levels 6 and 7.

The distribution between the macro, meso, and micro levels is presented in

Table 10. Mean values of organisational levels of reforms in different studies.

	Industry	Health Care
Set	1	2	3 *	4 **
Distribution mean value	%	%	%	%
Organisational level
Macro	24	47	1	3
Meso	70	51	84	28
Micro	6	0	14	69
Average number of suggested reforms per investigation	108	63	8	5

3 * SSM = standard Swedish method. 4 ** Results from Wrigstad et al. (2014) with SSM.

. The last column gives results from the study by Wrigstad et al. (2014). An interesting observation concerns the investigations from the health care sector, where both SSM sets have low values at the macro level, while investigations from Set 2 ‘health care’ score high here. Due to the uncertainties in the translation, there is no basis for conclusions about differences between meso and macro in Sets 1, 2, and 3.

Another way to scrutinise recommendations is to consider their ‘strength’. An examination of 227 event investigations in health care has been performed with a focus on the strength of recommendations [14]. The investigations were based on a root cause analysis approach. The recommendations were classified based on criteria from the US Department of Veteran Affairs. A division was made in ‘strong’ (likely to be effective and sustainable), ‘medium’ (possibly effective and sustainable), and ‘weak’ (less likely to be effective and sustainable). In the study, 1137 recommendations were examined, and it was found that 8% were ‘strong’, 44% ‘medium’, and 48% were ‘weak’. On average, there were five reforms per investigation, which happened to be almost identical to the sample in Wrigstad et al. (2014).

Canham et al. (2018) have studied the investigation of a specific medication error using two separate methods. Initially, the event was examined with a root cause analysis (RCA) approach. This was followed by an analysis applying the systems theoretic accident modelling and processes (STAMP) approach [28]. The authors found that it leads to a clearly enhanced consideration of system design issues and improvements, ‘going beyond the individual–based actions proposed by the RCA’.

4.3. Comparisons of Industrial and Health Care Cases

One intent of this paper is to study event investigations from industrial and patient safety contexts in order to explore similarities and differences. Table 3, Table 4 and Table 5 have shown results from the seven investigations using the combined method approach (CMA) described in Section 2.3.

Although the case studies were few and sometimes showed large variations, it can be interesting to compare mean values for the two sets.

Table 11. Mean values for parameters in event investigations in industry and health care.

Parameters	Industry	Health Care
Identified actors	13	13
Identified events	38	22
Suggested reforms	108	63
Identified Deviations	65	55
Deviations assessed (a)	55	48
Improvement needed (b)	90%	88%
Disagreement (c)	3%	3%
Identified Safety functions	64	55
SF assessed (d)	49	42
SF performed well	20%	27%
SF performed partly well	15%	25%
SF working (e)	35%	52%

(a) The number of deviations which were assessed regarding the need for improvements; (b) Proportions of the assessed deviations judged as in need of improvement; (c) The number of occasions when there was a disagreement in the assessment; (d) The number of safety functions (SFs) that were assessed; (e) Proportions of the assessed safety functions which performed well or partly well.

gives average values for some of the parameters. The average number of identified actors became the same. For identified events and suggested reforms, the mean values were about 70% higher for the industrial cases.

The average number of deviations and safety functions (SFs) were fairly similar. The number of SFs was high for all cases, ranging between 49 and 83 (Table 7), with a somewhat higher mean value for the industrial cases. Accounts of the deviations were fairly similar. In both sets, the proportion of deviations assessed as in need of improvement was high-around 90%. The author’s interpretation is that most of the identified deviations were seen as essential to explaining the studied event. Lists of deviations are the basis for the development of reforms, according to the procedure in deviation analysis.

The large number of safety functions supports the concept that safety is composed of complex interactions between various functions in the organisation. However, functionality has been low in both sectors. The proportion of SFs performing well or partly well was on average 35% for the industry. In health care, it was higher (52%), which partly can be explained by the selection of cases. In the health care cases, near-accidents were investigated, and the accident sequence was interrupted because some important SFs really worked.

In the individual investigations, data from the SF-analysis was used in the development of reforms. The main strategy then is to find ways to strengthen weak SFs; sometimes it could be more meaningful to remove a poor SF. In short, it is not meaningful to add new safety features when the current ones do not work.

One major difference between the two branches was that the reference teams in the industry had been assigned to the task by company management, which meant that they had time available. In the health care cases, participation was voluntary. The participants were highly positive about taking part, but it was sometimes hard to find time for the meetings. This can partly explain why suggested reforms were fewer in the health care cases, not just that there were differences between the sectors.

4.4. Comparisons and Interpretations

4.4.1. Summing up the Industrial and Health Care Cases

The analysis shows that the combined method approach performed similarly in industrial and health care contexts. There are variations within the two sets and the samples are small, so general comparisons are only indicative. The mean values for identified deviations and for the proportion that calls for improvements (90%) are very similar. Moreover, the numbers of safety functions are comparable, and the proportion that worked was fairly small (35–55%).

Differences are that the industrial cases found more events on average (73% more) and also proposed more reforms (71%). This could partly be explained by a larger commitment to the investigations in the industrial cases. There are also variations in the distribution of types and hierarchical levels, which are commented upon in relation to Table 8 and Table 9.

4.4.2. Summing up the Health Care Cases

For the Sets 2 and 3 investigations in the health care sector, there are methodological differences, which have been of special interest. An analysis of suggested reforms was performed in detail in Section 4.2. Here, the differences are evident. The average number of suggested reforms per investigation varies considerably (Table 10). Set 2 has a mean value of 63, which comes eight times higher than Set 3, and 13 times higher than Set 4, which corresponds to results from Wrigstad et al. (2014).

The other large difference concerns the organisational level of reforms, where Set 2 mainly addresses the higher levels (47%), but scores zero at the local level (Table 10). By contrast, Set 3 has a majority of reforms at the local level (76%) and 1% at the highest levels. The finding that the standard Swedish method tends to generate local reforms is in line with the results of Wrigstad et al. (2014).

Other dissimilarities relate to the type of reforms (see Table 9). In Set 2, there is a focus on ‘safety management’ and ‘investigations’. By contrast, Set 3 addresses ‘production planning’ and ‘human resources’.

4.4.3. Theoretical Considerations

The differences between the results could be due to a combination of explanations, mainly related to the investigation methodology. In this study, results from two models for event investigations have been compared. The standard Swedish method [22] is based on the root cause analysis concept, which pronounces causal relationships. The method manual states that you should continue to ask ‘Why?’ for as long as possible.

Several authors [2,3,29,30] have raised principal objections to the interpretation of causes in event investigations. One argument is that ‘causes’ are not sufficiently objective in explaining the course of an event. In an environment with many activities and actors, relationships and causalities are complex to interpret, especially when probabilities for various outcomes are low and hard to predict.

In the combined method approach (CMA), the studied event is seen from different perspectives, and there is no emphasis on causality. This also means that improvements can be suggested from different standpoints, and thereby cover broader aspects. Here having a reference team was important because its members had a long experience of many earlier situations that are seldom documented.

The role and experience of the investigators are essential to the results. When investigators are all from the same organisation and of similar professional backgrounds, the perception of problems and possible solutions might be too narrow or conservative. There could also be dilemmas with the examination of actions by colleagues and of management in your own organisation.

4.4.4. Usefulness of Thorough Investigations

Results from the CMA and SSM investigations show the following two distinctive differences:

(1)

The number of reforms per investigation is around 10 times higher for CMA;

(2)

CMA addresses higher organisational levels, while SSM is aimed at mainly the local level.

How useful are more comprehensive suggestions in reality? Broader types of results give a foundation to consider the whole system, and thereby make for more holistic and efficient safety work. However, an argument against having many reforms is that it may be hard to get them implemented. This is especially the case when higher organisational levels are involved.

This raises the issue of implementation. In the studied investigations, the responsibility for accomplishing the reforms was clearly stated. A general impression is that all suggestions have been considered, but information about implementation and real effects has not been available.

The implementation of suggested reforms has been examined by Wrigstad et al. (2014) in the health care area. For the specific cases, they found that reported action was taken for 44% of the reforms. At the highest organisational levels, the implementation rate was 14%. In the same study, the authors searched in vain for a proper system for recording actions based on incident investigations. These findings of low implementation underline the importance of a proper context if event investigations shall be of use and help for a learning organisation.

4.5. Event Investigation Improvements

It should be noted that this study did not address the quality of the investigations and of the reforms. A wider scope could concern the potential effects of suggested reforms, how feasible they are, and what practical actions the proposals led to. However, the available information did not provide a basis for such studies. Such a wider perspective is a challenge for future research.

Several authors [2,3,4,13] have pointed to a need for improving the use of and learning from event investigations. A number of general guidelines for such investigations [8,23,31] have given advice on how to consider various aspects. Based on this study, the author wants to highlight the following specific issues:

(1)

Decisions about event investigations;

(2)

Accomplishment of investigations;

(3)

Decisions and accomplishment of reforms;

(4)

Integration of information and learning.

In the account earlier, there has been a focus on point 2 and the characteristics of event investigations, but here the scope is widened.

(1)

Decisions about Event Investigations:

This point deals with the role event investigations shall have in safety work. It concerns which events shall be investigated, by whom, with what methodology, and how the results shall be handled. The purpose of the investigations can be to improve safety in the whole organisation or be limited to local and temporary problems. If the first aim is essential, there is a need to consider the whole investigation framework.

The first issue is which events should be investigated. Legal and ethical demands should be met, but there are possibilities to go further. One recommendation [4] is to perform fewer but deeper investigations. One example lies in the CMA investigations presented here, which provided substantially deepened information about what could be improved.

In order to facilitate understanding of systems errors, these ‘deep’ investigations can also be made of incidents without injuries. In serious accidents, there is a stigma, and sometimes feelings of guilt or even fear, which affects both witnesses and decision makers. The results of Set 2, which were based on incidents, prove that this approach can be successful.

(2)

Accomplishment of Investigations

The performance of an investigation should match its goal and scope. As shown above, the choice of methods will affect the information obtained and the type of reform. The choice stands between using the usual method, applying a new method, or using a combination of methods. Because there may be several stakeholders when an incident has occurred, the credibility of the investigation is essential. One aspect is that the investigators should be impartial, and another is that the investigation process is systematic and transparent.

(3)

Decisions and Accomplishment of Reforms

If the investigators are supposed to be impartial, it is an advantage if they make recommendations but not take decisions on implementation. In order to maintain the integrity of investigations, there is a need for a clear process and guideline on how decisions should be made and implemented. Of course, decisions to reject suggestions must also be made because all might not be good and meaningful.

Suggested reforms at higher organisational levels are of special interest because their impact can be considerable. Event investigations addressing higher levels need special attention, and a forum to discuss them impartially could be advantageous.

(4)

Integration of Information and Learning

Event investigations can be used to collect a large body of information on problems and suggestions for improvements. If this is carried out in a systematic way, it will create a useful knowledge base and improve organisational learning with regard to safety. Within the health care sector, several authors [4,13] have stated the need for improvements in that respect. This conclusion is probably also valid for industrial organisations.

5. Conclusions

In this study, results from event investigations from industrial and patient safety contexts have been analysed. To support the analysis, a methodology for analysing proposed safety reforms from event investigations was developed. It was tested and discussed with independent specialists. The conclusion was that the methodology worked well for categorisation of proposed reforms.

The studied material has been based on three sets of event investigations. Set 1 addresses thorough investigations in industrial organisations, where three different investigation methods have been applied. In Set 2, the same methodology has been applied to incidents without injuries at health care units. The third set, also in the health care sector, includes investigations performed using the standard Swedish method, based on the root-cause concept. Because the number of cases was few and not randomly chosen, general conclusions cannot be drawn. However, tentative conclusions and observations can contribute to a fuller understanding of the use of event investigations in safety management.

The results from Sets 1 and 2 have been compared. The investigation approach functioned for both, in a similar way in several respects. All investigations resulted in numerous reform suggestions, with a somewhat higher quantity for Set 1. It could also be seen that the industrial cases had a more pronounced emphasis on safety management.

Sets 2 and 3, both from health care, were also compared. Since the methodology is different, special attention was given to the characteristics of the suggested reforms. The number of suggestions was on average eight times higher in Set 2. The analysis showed that Set 2 often addressed high levels in the organisational hierarchy and safety management issues, which indicates a pronounced systems perspective. On the other hand, Set 3 had a clear emphasis on reforms at the local level, and improvement of production.

These results confirm and strengthen the findings of Wrigstad et al. (2014), who found a majority of reforms at the lowest organisation level (micro). The findings strongly support the suggestions by Vincent et al. (2017) that a few deeper event analyses can give better and more relevant information—especially at higher systems levels. The results from Set 2 also show that investigations of near-accidents can be useful in developing reforms for safety improvements.

A general conclusion is that there is considerable space for improvements in the management and performance of event investigations in the health care sector. Probably, although not explored further here, this is also the case in the industrial sector. An important field for research and a way of advancing the field is to reconsider how event investigations are to be included in the organisation’s safety management system.