Managing the Performance of Asset Acquisition and Operation with Decision Support Tools

: Decision support tools (DSTs) are increasingly being used to assist with asset acquisition and management decisions. Whether these tools are “fit for purpose” will have both economic and non-economic implications. Despite this, the on-going governance of DST performance receives only limited attention within both the academic and industry literature. This work addresses that research gap. Within this paper a conceptual process for managing the operational performance of decision support tools is presented. The novelty of the approach is that it aligns with the ISO 5500x:2014 Asset Management Standard, therefore introducing consistency in the governance of DSTs with physical engineered assets. A case study of the UK’s National Grid Electricity Transmission (NGET) is used to inform the approach design. The evaluation demonstrates it to be both logical and useable within the context of NGET and they have expressed an intention to implement the approach within their business. A key finding of the research was that DSTs transcend functions and organisations. This is significant and can lead to a situation whereby performance and criticality are interpreted and measured differently. The introduction of a common approach for managing DST performance works towards improving consistency and creating a shared understanding.


Introduction
Improvements in economic performance and human well-being are intrinsically linked to strategic infrastructure. To meet objectives in these areas, global spending on infrastructure will significantly increase, rising from USD 4 trillion per year in 2012 to an estimated USD 9 trillion per year by 2025 [1]. One way to reduce the amount needing to be spent is to make the money invested deliver greater returns-to improve asset investment productivity. Estimates suggest that improvements in the selection, build, operation, and maintenance of infrastructure have the potential to save USD 1 trillion per annum [2]. To realise these efficiencies, infrastructure organisations are making use of manual and computerised tools to assist with asset acquisition and operational decisions. Within the sector these are commonly referred to as decision support tools (DSTs).
DSTs have been introduced to improve the efficiency, effectiveness, and consistency of asset decisions. However, benefits are not guaranteed. Across both the business process and information systems areas, there are examples of new business initiatives where expected gains have either not been realised, or initially realised but not sustained [3][4][5][6][7][8][9][10][11][12]. Despite this challenge, the literature shows only limited attention given to the governance of DST performance. This is significant. If DSTs are making sub-optimal asset decisions, this will adversely affect asset investment productivity.
The research presented within this paper was conducted in collaboration with the Engineering and Physical Sciences Research Council (EPSRC) and National Grid Electricity Transmission (NGET) under an Industrial Case Award (iCASE). Within this work, we assert that under the wording of the International Standard for Asset Management (ISO 5500x: 2014), DSTs can be considered to be organisational assets. Therefore, for consistency, organisations should manage them in the same way that they manage their physical engineered assets.
The output of this 42-month project was a conceptual approach for managing the performance of DSTs used within an asset management context. The approach consists of a DST performance management process and DST performance assessment techniques-the methods for applying the process in an industry setting [13]. The focus of this paper is the DST performance management process.
The paper is structured as follows: First, an introduction to the practice of asset management and the use of DSTs within this context (Section 2). The literature provides evidence of the creation and use of DSTs, but also identifies a gap in the on-going governance of their performance. The research approach, whereby a case study of National Grid Electricity Transmission (NGET) is used to inform the design of the approach, is described (Section 3). The DST performance management approach is detailed (Section 4). Finally, the outcome of an evaluation by NGET subject matter experts is presented (Section 5), conclusions formulated (Section 6), and future work identified (Section 7).

Background
Managing assets and asset management (AM) are not the same thing [14]. Although there is a long history of caring for assets (managing assets), the structured approach for deriving value from assets (asset management) is an emerging discipline [14][15][16]. The main contributions to the field are considered to come from government and practitioners, with academic contributions being moderate [17]. Despite increasing contributions to the field, criticism of AM practice persists. Asset-owning organisations are accused of being unsystematic in their approach to projects with many of their processes unchanged in decades [2]. ISO 5500x:2014 works towards addressing these criticisms through the introduction of an International Standard for AM practice.
Within the UK energy sector, the use of quality standards within their AM operations has been strongly encouraged by the UK regulator, Ofgem [18]. The current ISO AM Standard adopts a topdown approach in which the objectives of the organisation are translated into a coordinated system of asset plans, policies, and processes. The principle ambition of AM is to realise the value of assets and in doing so support achieving the organisational objectives.
Traditionally, the focus of AM systems was physical engineered assets. Replacement of the British Standard [19] by the International Standard [20] in 2014 redefined assets as anything which contributes potential or actual value to an organisation achieving its organisational objectives [20]. Innovative thinking in this regard has seen proposals to increase the scope of AM systems to include physical (non-engineered) assets, e.g., environmental assets [21] and people [22], as well as nonphysical assets, e.g., data [23] and software [24]. Ultimately, the scope of the assets which are managed within an AM system is defined by the organisation and will be dependent upon external and internal factors including resources, legislative or licencing requirements, and the maturity of an organisation's AM practice.

Decision Support Tools
The overarching aim of AM is to realise value from assets. In achieving this, it is not merely a case of maximising asset performance, but in achieving the optimum balance of performance, cost, opportunity, and risk throughout the lifecycle of the asset [20]. To support organisations in achieving this optimum balance, a range of DSTs are utilised.
Although providing evidence of DSTs being created within academia, this did not necessarily mean that DSTs were being used in practice. Articles within the Institute of Asset Management (IAM) members' magazine [37][38][39][40][41][42], and Subject Specific Guidelines [43] provided an insight into DST use within the field.
Subject Specific Guidelines (SSGs) are a core element of the IAM's asset management knowledge base. Created through peer review and IAM Expert Panel assessment, they represent the professional body's current thinking on a particular subject. Although published in 2015, Subject 8; Life Cycle Value Realisation [43] provides the most recent consolidation of DST "good practice". Within the SSGs, ten exemplars of DSTs used within the UK's core infrastructure organisations (e.g., Network Rail, National Grid, Severn Trent Water, and London Underground) are provided. Unlike academic DSTs which are strongly focused towards customised computer-based solutions, the ten exemplars show DSTs to include manual processes, computer-based utilising standard software (e.g., Access, Excel), and configured software solutions. Similar to the academic literature, they are seen to have a variety of attributes and are applied to a range of asset decision problems.
The evolutionary nature of DSTs underpins academic theory in this area [44][45][46][47][48]. Despite this, the focus of both the academic and industry literature is on proposing and presenting DST approaches with little consideration given to the management of their operational performance. Specifically, there were no performance governance approaches that aligned the management of DSTs with the requirements of the ISO 5500x:2014 Asset Management Standard.
Section 3 details the approach taken in creating the DST performance management process.

Research Approach
Case studies are particularly suited to understanding complex relationships in the context of social settings [49]. This research uses a case study of National Grid Electricity Transmission (NGET) to inform the design of an approach for managing DST performance. Within this section, the case study is presented (3.1) and the method used in defining the approach requirements is detailed (3.2).

Case Study -National Grid Electricity Transmissions (NGET)
National Grid Electricity Transmissions (NGET) own and operate the high-voltage electricity transmission network within England and Wales and operate (but do not own) the Scottish transmissions network. As such, they play a central role in the UK power infrastructure sector. The nature of their business means that NGET is rich in physical assets (e.g., overhead lines, underground cables, substations), with an asset portfolio valued in excess of GBP 40 billion [50]. Going forward, over the eight years from 2013, a spending programme of GBP 13.6 billion capital expenditure (Capex) and GBP 16.4 billion total expenditure (Totex) are scheduled. Within NGET, this extensive asset portfolio is managed by way of an ISO 5500x:2014 aligned AM system.
To understand the use and governance of DSTs within NGET, an in-depth study was undertaken. Figure 1 identifies the inputs used in conducting this case study. The case study showed that NGET was making extensive use of DSTs. These encompassed manual processes [51], computer-based databases and spreadsheets [52], and customised software solutions [53]. DST theory asserts that the nature of DSTs makes them cross-functional [46][47][48]54]. This was supported by the case study. Indeed, one example not only transcended functions but also organisations being built and revised by an external supplier, using data from both inside and outside of the organisation, governed by NGETs' IT department, used by NGET asset management engineers, and providing information to NGET decision-makers [53,55,56].
Gathering a full picture of the extent of DSTs' use within NGET proved challenging as no central register was held. Although no complete register was available, an inventory of "non-trivial" enduser computing (EUC) reports and databases was maintained. This provided a list of DSTs held locally, utilising standard computer software, i.e., Excel and Access [57]. Analysis of the inventory identified 195 different DSTs, with a variety of attributes and purposes, being used across seven different business functions.
The case study provided evidence that NGET was undertaking control and governance activities to ensure DST performance. However, these tended to be ad hoc, rather than coordinated through an embedded central process, with different measures for DST criticality and performance seen [51,55,57,58].

Defining the Approach Requirements
The goal of this research was to create an approach for managing DST performance which was both rigorous, but logical and useable within AM practice. In this regard, understanding the requirements of the stakeholders was key. Requirements engineering (RE) is concerned with managing the wants and needs of the stakeholders and translating these into a statement of how the design should be. The RE method used within this research was based on the cyclical approach proposed by Callele et al. [59]. Figure 2 presents the requirement engineering method and the activities undertaken at each of the four stages: elicitation, analysis, documentation, and validation. During the elicitation (Stage 1), an NGET subject matter expert was asked to identify the approach stakeholders and their requirements. To mitigate the risk that stakeholders were missed, a stakeholder map was used to guide this activity ( Figure 3). Four stakeholders were identified: owner, customers, employees, and government. Following, the expert was asked to brainstorm the requirements of each. Brainstorming is a recognised technique that encourages freethinking [61]. This resulted in the generation of fourteen requirement statements (see Table 2).
Amongst the fourteen was the requirement that the approach must conform to the ISO AM Standard. The analysis (Stage 2) looked to identify any conflicts or omissions between the remaining thirteen stakeholder requirements and the Standard.
An exploration of the Standard identified that there were no specific requirements for a DST performance management approach. Rather, there were key concepts (principles, general requirements, and guidelines) which must be met. The first step in conducting the analysis was to identify these key concepts. Identification of the key concepts was undertaken using thematic analysis [62] and the utilisation of the NVivo computer-assisted qualitative data analysis software (CAQDAS). Table 1 provides details of the nine key concepts identified and an example excerpt coded to each theme.

Key Concept
Example excerpt from the ISO 5500x:2014 Standard coded to this theme Process based "The organization should develop processes to provide for the systematic measurement, monitoring, analysis and evaluation of the organization's assets".

Process integration
"A factor of successful asset management is the ability to integrate asset management processes, activities and data with those of other organizational functions, e.g. quality, accounting, safety, risk and human resources"

Consultation and communication
"Failure to both communicate and consult in an appropriate way about asset management activities can in itself constitute a risk, because it could later prevent an organization from fulfilling its objectives".
Evolving "The organization should outline how it will establish, implement, maintain and improve the system".

Monitoring and continual Improvement
It is a "concept that is applicable to the assets, the asset management activities and the asset management system, including those activities or processes which are outsourced". The identification of "Opportunities for improvement can be determined directly through monitoring the performance of the asset management system, and through monitoring asset performance".

Life cycle approach
The stages of an asset's life are undefined but "can start with the conception of the need for the asset, through to its disposal, and includes the managing of any potential post disposal liabilities".

Defined leadership
The success in establishing, operating, and improving AM is dependent on the "leadership and commitment from all managerial levels"

Contextual
What constitutes value is contextual, it "will depend on these objectives, the nature and purpose of the organization and the needs and expectations of its stakeholders".

Risk-Based
"Asset management translates the organization's objectives into asset-related decisions, plans and activities, using a risk-based approach" Next, the NGET subject matter expert was asked to assess whether the stakeholder requirements were satisfied through the key concepts. The results are presented in Table 2. Although there were no direct mappings of stakeholder requirements, the subject matter expert confirmed that each of the stakeholder requirements was satisfied by key concepts within the Standard. No conflicts or omissions were identified.
With the nine key concepts meeting the needs and wants of the stakeholders, these were documented as the approach requirements (Stage 3).
Validation (Stage 4) was undertaken by way of a questionnaire which was completed by two NGET subject matter experts. The case study had shown DSTs' creation and operation to involve both NGET IT and engineering functions. Consequently, one participant was recruited from each area.
The questionnaire focused on validation of three areas: the stakeholders, the stakeholder requirements, and the approach requirements. The results are presented in Table 3. Are there any approach requirements you feel should not be included? If so, provide the detail and justification.

4B
Are there any approach requirements which you feel have not been identified? If so, provide the detail and justification.

No No
The responses showed complete alignment across the two participants. Both agreed that there were no stakeholders who they considered should not be included (1A). Although both identified that suppliers had been omitted as a stakeholder (1B), they did not consider that they had any additional requirements above those which had already been identified (2B). The literature review, case study, and requirement engineering activity suggest there would be a benefit (or need) to align the DST management approach with the ISO AM Standard. The responses provided further support with both respondents agreeing that the approach should conform to both the asset management and risk management standards (3A). Both respondents confirmed that the ten approach requirements were valid with none unnecessary (4A), or omitted (4B).
The DST performance management approach which was informed by the requirements defined within this section is now presented.

The DST Performance Management Approach
The DST performance management approach encompasses two parts: the DST performance management process and DST performance assessment techniques (the methods for applying the process in an industry setting). Figure 4 shows how these are positioned within the AM hierarchy. Defining the requirements for an asset management system, the International Standard for Asset Management (ISO 5500x:2014) occupies the highest level within the hierarchy.
An AM system comprises of the policies and procedures for applying AM within an organisation [20].
Forming part of the AM system, the policy for DST control and governance defines how DSTs will be managed through life. The DST performance management process manages performance during the operational stage of a DST. Wider measures will be necessary to cover the complete lifecycle (i.e., creation, in-use, and end-of-life).
The DST performance management process provides the means through which DST operational performance is managed. It defines the steps when conducting a DST performance assessment, and incorporates elements that ensure that it meets the requirements for a quality managed process as defined within ISO 9001:2014 Quality Management Systems [63].
The DST performance management techniques are the "how to" methods through which to conduct the steps contained within the process. As stated, the focus of this paper is on presenting the DST performance management process. Figure 5 presents the DST performance management process. The design of the process echoes that of the risk management process described within the ISO 31000:2009 Risk Management Standard [64]. Aligning the two processes was considered advantageous for four main reasons: (1) When defining risk and how to undertake risk management, the Asset Management Standard references the Risk Management Standard [20]. (2) The ISO organisation asserts that for consistency and control, the risk management framework and processes should be integrated within other management systems [65]. (3) A stakeholder requirement was that the approach should comply with the Risk Management Standard. (4) The inclusion of the risk management process as part of the Risk Management Standard assures that it satisfies the requirements for a process as defined within the ISO Quality Management Standard [66]. Although similar in design, within the DST performance management process, the risk assessment is replaced by the DST performance assessment. Furthermore, the risk management process is visualised as isolated from other processes, whereas the DST performance management process is integrated with other management systems and/or governance processes at the treatment stage. Table 4 presents a comparison of the elements within the two processes. The comparison shows three elements that are alike: "Communication and consultation", "Monitoring and review", and "Establishing the context".

The DST Performance Management Process
"Communication and consultation" and "Monitoring and review" are the elements through which continual improvement is embedded within the process. Consultation and communication ensure that stakeholder views are taken into consideration, are informed about how decisions are made, and advised of any action taken.
"Monitoring and review" acts to deliver assurance of the quality (e.g., efficiency and effectiveness) of the process, embedding the ongoing monitoring and periodic review of a process and its outcomes.
"Establishing the context" (renamed "Scope, Context and Criteria" in the 2018 revision of the risk management process [65]) is the means through which a generic process is tailored to meet the requirements of an individual organisation. Within the risk management process, "Establishing the

Communication and consultation
Monitoring and review context" defines the objectives, internal and external context, and sets the risk criteria for the remaining process. The DST performance management process mirrors this, defining the internal, external, and process context, and the rules for applying the rest of the process, i.e., the rules that an organisation will follow when carrying out the activities within the DST performance assessment. These include the rules for:  Establishing how critical a DST is;  Determining which DSTs shall have their performance measured;  Evaluation of DST performance assessments;


The treatment applied given a certain performance assessment outcome. At the "Assessment" element, the processes differ. Although the DST performance management process contains the same three steps as the risk management process (identification, analysis, and evaluation), the activities undertaken within each are changed. Within the DST performance management assessment, identification captures information about the DSTs used within the organisation. It delivers a complete list of all DSTs within scope (determined by the rules defined within "Establishing the context"), their business criticality, and information reporting their performance management activities and outcomes. Performance management reporting information is generated when the DST performance management process is applied.
Analysis applies the rules defined within "Establishing the context" to identify the DSTs that require their performance to be measured. Once identified, the performance of the DST is measured.
Evaluation applies the rules defined within the "Establishing the context" element to the results of the performance assessment. The outcome determines the treatment which is applied.
The 2018 revision of the risk management process saw recording and reporting made explicit through its inclusion as a separate element [65]. Non-prescriptive, the element requires the activities and outcomes of the process to be reported "through appropriate mechanisms". Decisions around the creation, retention, and handling of documented information are determined by the organisation and its context. Within the DST performance management process, "Recording and Reporting" are undertaken within the identification element with the register containing both the detail of the asset and performance management activities pertaining to it.

Evaluation
Evaluation was undertaken by way of a focus group comprised of five NGET subject matter experts. A focus group is a commonly used qualitative research approach that seeks to bring participants together to explore an idea [49,67,68].
The evaluation encompassed four elements: whether the experts felt that a research challenge existed, and strengths, weaknesses, opportunities, and threats of the DST performance management approach. Specific to the DST performance management process, it systematically explored: The outputs at each step of defining the approach requirements:


Are there any stakeholders who have been missed/should not be included?
 Are there any stakeholder requirements that have been missed/should not be included?
 Are there any approach requirements which are missing/should not be included?
Whether the proposed conceptual DST performance management process was logical and usable:  Does the process appear to satisfy the ten approach requirements? The underpinning decision system theory identifies that in the creation, build, maintenance, and operation of DSTs, a range of functions will be involved [46][47][48]54]. The NGET case study provided support for this in practice. Keen [47] considered the cross-functional nature of DSTs from the perspective of how they evolve. Within his work, he created a model of decision tool adaption which identifies three actors (user, system, and builder). Alternatively, Alter [54] focused on the decision tool use, identifying two categories of user-the intermediary (who uses the system), and the decision-maker (who applies the information to decision making).
Given the range of DST types, scope, and attributes it would not be possible to create a definitive list of participants who should be involved in the evaluation of a process for managing DST performance. In this study, the selection was guided by the literature, the case study, and the inputs of the NGET senior management to ensure a comprehensive range of roles, and appropriate level of expertise. Table 5 presents details of the participants involved in the evaluation: their department, job title, and responsibilities in relation to DSTs. Although the case study provided evidence that DSTs can involve different organisations as well as functions, at this early in the research, the involvement of external participants was not considered desirable. To mitigate researcher bias, the focus group was facilitated, and responses captured by one of the participants. During the analysis, the captured comments and audio recording of the focus group were analysed [13].
Approach requirements: The participants agreed that those identified (owner, customers, employees, and government) represented the core stakeholders and that there were no stakeholder requirements identified which should not be included. Identifying whether there were missing requirements proved more challenging. This led to discussions of what a requirement statement actually meant and consequently, whether further requirements were required, or were encompassed by an existing statement.
Logic and usability: Despite the discussion which arose around the approach requirements, the participants agreed that the process met the requirements. Additionally, it was agreed that it was logical and useable when considered in the context of each of the three NGET DSTs.
A key insight from the case study was that differences between functions have the potential to lead to inconsistency in governance. In addition to differences in how performance and criticality was measured and rated across DSTs, the focus group identified that other metrics for these two areas existed "from a data perspective we follow the business continuity categories which are operationally critical, critical, core, and efficiency and performance". Comments captured during the focus group identified "there will be different perceptions of performance".

Conclusions
The academic theory recognises that unless decision tools adapt to changes in the environment, they will become less effective. Despite this, little attention has been given to managing DST performance following implementation. The research presented within this paper addresses that gap through the creation of a DST performance management process.
A case study of NGET shows them to be making extensive use of DSTs. Unlike the academic literature, where the focus is on customised computer-based solutions, the majority of tools used by NGET are held locally and utilise standard software packages such as Excel and Access. This is significant and suggests that without proper governance, there would be a lack of transparency around how asset decisions were being made.
The academic literature asserts that the nature of DSTs means they will transcend organisational functions. This is supported by the NGET case study which sees DSTs cross both functions and organisations. The evaluation of the DST performance management process identified measures and ratings of performance and criticality as two areas where differences arose. Potentially, this could introduce inefficiencies, miscommunication, and lead to difficulties in establishing where responsibility resides.
AM standards have moved to defining an asset as anything that is considered to contribute value towards achieving organisational objectives [20]. Within this work, we propose that DSTs are assets, and for consistency organisations should manage them in the same way as their physical engineered assets. In achieving this, the DST performance management process was designed to align with the ISO 5500x:2014 AM Standard. The evaluation of the process by NGET subject matter experts found it to be both logical and useable within the context of NGET.
Indications are that this research has the potential to create impact outside of academia. NGET have indicated their intention to implement the approach within their business [69]. Interest from the practitioner community is high with requests made to present the approach at the IAM Conference (November 2017), and for a special interest group of the British Computer Society (April 2018). Additionally, an overview of the approach has been published within the members' magazines of both the IAM [70] and Chartered Quality Institute [71].

Future Work
Conducting quantitative research in a laboratory setting is relatively straightforward. There is one "reality", which can be validated using recognised, statistical methods. The challenge of creating a DST performance approach which is both rigorous and practical within an industry context is complex. This is especially true when the research is innovative and the requirements for the approach have yet to emerge or crystallise. This research created a conceptual design. It is accepted that in the course of progressing through experimental and implementation stages, perceptions of what is possible, wanted, and needed will evolve. Consequently, requirements must be continually revisited and adjusted.
Future research efforts should be focused on three key areas: 1. The aim of this research was to create a conceptual approach for measuring DST performance which is both rigorous and practical. The logic and usability of the process have been validated by NGET subject matter experts. However, it is accepted that what may be logical and useable in theory, may not be so in practice. NGET have declared an intention to implement the process within their organisation [69]. Research efforts should look to conduct field test validation studies of both the process and underpinning techniques within both NGET and other organisations; 2. The transferability of the approach to the water sector was evaluated as part of the programme of work [13]. Although providing some evidence, further studies across a wider sample of sectors and organisations are required; 3. Key to the uptake of the approach by industry is being able to demonstrate that it has value. In this regard, methods and studies to assess the whole life cost versus the benefit of DST performance management are required.

Conflicts of Interest:
The authors declare no conflict of interest.