Evaluating the Cost Efficiency of Systems Engineering in Oil and Gas Projects

Abstract

Studies of systems engineering applications have revealed that systems engineering (SE) has a high potential for transferring economically inefficient oil and gas projects into a profitable zone due to preserving the value created at the concept stage right up to the implementation stage. To implement any project, including an organizational one, the company must have an economic justification for innovation. Studies into the global experience of assessing SE efficiency based on projects of various types have revealed the lack of a universal assessment method; however, individual studies have potential to be used in developing a method for quantifying the value of SE in oil and gas projects. Considering this fact, we developed our own method and prototype to assess the economic effect from the introduction of SE into oil and gas projects. The method is based on a decision tree used to calculate the Net Present Value considering the probability of projects’ success and failure in terms of budget and deadlines. This allowed us to predict the effect from introducing SE to an oil company’s capital project. The results obtained demonstrated the model’s performance capability and its possible applications in project resource planning stages.

1. Introduction

The ability to design and build successful products that meet stakeholder requirements and bring them to market on time is a priority for most organizations. One of the most effective tools for the implementation of successful high-complexity projects is systems engineering. However, the economic effect from using SE is not fully understood. This article will describe what systems engineering is, the results from applying this engineering approach to various projects, and also our own method for calculating the economic effect of using SE in the oil and gas industry.

An increase in the complexity and scale of designed systems combined with difficulties in organizing teamwork resulted in a new methodological approach, systems engineering, which appeared in the mid-twentieth century. The term systems engineering was first used at Bell Labs in the 1940s [1]; in the USSR, the new engineering approach developed in the early 1960s; and in 1969, the USSR’s first systems engineering department was created at Moscow Energy University.

The International Council on Systems Engineering (INCOSE) defines systems engineering as a transdisciplinary and integrative approach to enable the successful realization, use, and retirement of engineered systems, using system principles and concepts and scientific, technological, and management methods [2].

Systems engineering (SE) is a modern methodology for creating complex technical systems, and for many years it has been successfully used in projects of various kinds [3,4,5]. The advantages of using SE and its impact on the success of the system can be expressed in the following: complete and consistent documentation on requirements management, a high level of stakeholder participation, a system functionality that meets the expectations of stakeholders, a high level of system reuse, and more predictable project results [6,7].

The use of SE methods and tools has become a practical necessity and, in fact, a standard that ensures high-quality results, for example, in large projects in the aerospace, nuclear, and military industries; in the design of transport systems; and in construction, which differ in a wide range of requirements for safe and successful functioning [2,5,8].

However, systems engineering is rather poorly implemented in Russian companies, despite significant advantages that stem from applying system thinking principles and the experience of building successful systems. The authors believe one of the reasons for this to be a weak justification for the efficient application of systems engineering and, as a result, its poor implementation in companies’ business processes.

The signs of complex, multicomponent systems [9,10] in the oil and gas industry, such as engineering saturation, heterogeneity, and the use of advanced technologies, indicate a high potential for applying systems engineering to oil and gas projects. However, the scope of investment in implementing and developing SE depends on the scope and specifics of the project and requires quantitative justification when making investment decisions.

The impact of systems engineering on success in project management and return on investments in SE are subjects of many years of study and discussion in the international community of systems engineers. The main areas of study are associated with determining the optimum spends on SE and measuring the success of the project, which are traditionally determined by three parameters: project schedule (time); budget (cost); and quality, which is defined as the compliance of the result with the initial requirements of stakeholders [11,12,13]. These parameters are the key indicators of project performance (KPI) and may be objective or subjective. Objective indicators are calculated using math formulas, and subjective indicators are measured based on assessing the quality, functionally, and satisfaction of stakeholders [14].

The results of quantitative research in the defense and aerospace industries indicate a relation between systems engineering and success in project management, expressed in time and budget [15,16,17,18].

The results of this Honour’s research [15], based on a study of 43 projects using SE at the amount of 1% to more than 25% of the total funding, show a correlation between the efforts of SE and the success of projects in terms of the budget and time. This fact indicates a positive profitability investment in SE. In joint research of Honour with Barry Boehm and Ricardo Valerdi, the effect of using SE in the development of software systems was calculated based on data from 161 software projects from the COCOMO II database [18]. The effect of using SE in small-sized projects was 18%, and in very large projects it was 92%.

Researchers assess the value of innovations in different domains using related assessment methods and a system of economic indicators [19,20]. In the oil and gas industry, the most famous tool for calculating the profitability of a field is Merak Peep, developed by Schlumberger. However, methods and tools for calculating the value of systems engineering in the oil and gas sector have not been implemented yet.

The recognition of systems engineering is growing in many realms of academia and manufacturing. Some universities are currently offering SE courses, which reflects the growing demand for these skills [21,22,23]. However, systems engineering is still not widespread in the oil and gas industry, despite the potential benefits. To date, work is underway to study the applicability of systems engineering and assess the benefits of adopting SE practices at different stages of an oil and gas project [24].

Studies into the global experience of assessing SE efficiency based on projects of various types revealed the lack of a universal assessment method; however, individual studies have the potential to be used in developing a method for quantifying the value of SE in oil and gas projects. International experience in the economic evaluation of high-uncertainty oil and gas projects relies on a decision tree used to calculate the expected value of the project, its EMV (Expected Monetary Value). This article presents the results of applying the method based on calculating and comparing the EMV for project options with and without using SE. Based on the general data on project performance, we built a decision tree and a mathematical model, which we used to perform calculations taking an oil and gas project as an example. This allowed us to predict the effect of introducing SE to an oil company’s capital project. The results obtained demonstrated the model’s performance capability and its possible applications in project resource planning stages.

The article includes the following points:

• Collecting and analyzing data on the success of projects without using systems engineering and with using systems engineering;
• Describing the method for constructing a decision tree and a mathematical model for calculating the economic effect;
• Carrying out a predictive calculation of the effect of using systems engineering in a large oil and gas project in an automated model.

2. Materials and Methods

A probabilistic decision tree with various outcomes of events, whose probability can be determined based on a statistical sample, is taken as the basis for the SE efficiency assessment method.

Statistics on the success of oil and gas projects were collected by studying open source data generated by the Independent Project Analysis (IPA), a leading consulting organization that analyzes projects’ results. The organization’s database contains more than 20,000 studied capital construction projects, with a geographical coverage of more than 100 countries.

Data from the IPA [25] on the success of 318 oil and gas megaprojects (including 130 in exploration and production) with budgets exceeding $1 billion show that projects are unsuccessful in 78% of cases. These projects had an average budget overrun of 33% over the planned costs, and were started with a 30% delay from the previously planned schedule.

According to the IPA study, the key failure factors are cost overruns, execution schedule slips, and problems with achieving the designed production level and Net Present Value (NPV), adopted at the Financial Investment Decision (FID) stage. In addition to these problems, the IPA study points out a negative impact of “the drive for speed” in order to facilitate the start of commercial operation, which adversely affects the planning and quality of a project’s development.

Statistics on projects, which used SE elements, are published in the report on success of major public investment projects by the Norwegian University of Science and Technology [26]. The report contains data on investment projects, half of which relate to the transport and defense industries, with an average budget of about NOK (Norwegian Krone) 3 billion ($292.5 million).

The method of evaluating success described in the report is based on the Norwegian Quality Assurance Scheme for large government projects, which includes a two-level verification system for moving a project to the next stage. This scheme is integrated into a “stage-gate” model of a project, starting from the concept stage and ending with putting it into operation.

At the first stage of QA1 (quality assurance of the choice of concept), the project’s conformity with the initial goals, objectives, needs, and requirements of key stakeholders is assessed, and opportunities for further development and alternative concepts should also be examined here with a feasibility study. The second stage of QA2 (quality assurance of the management documentation) is a more detailed assessment of the project’s budget, including the necessary reserves for contingencies and the delivery time for parts to ensure operational success.

After analyzing data on 219 projects that passed QA1 and QA2 quality control in terms of deadlines and budgets, it was discovered that about 80% of the projects were completed within the set cost limits and below, while only 37% were unsuccessful in terms of deadline.

Table 1. Global statistics on successful and unsuccessful projects.

	Success $, t	Unsuccess ($)	Unsuccess (t)
Projects without system engineering (IPA, 2012; Standish Group, 2015)	22%	56%	51%
Projects with system engineering (Norwegian University of Science and Technology (NTNU), 2016)	50,4%	20%	37%

Source: Data from IPA [25]; Norwegian University of Science and Technology (NTNU) [26]; Standish Group [27].

shows the summary of success analysis for the oil and gas industry international projects, as well as for projects using the SE methods.

Table 2. Statistics on project cost overruns and project start delays in the event of an unsuccessful outcome.

	Budget Overrun	Schedule Delays
Projects without system engineering (IPA, 2012)	33%	30%
Projects with system engineering (INCOSE, 2015)	15%	20%

Source: Data from IPA, 2012 [25]; INCOSE, 2015 [2].

shows the statistics on budget overruns and delays in the implementation schedule for unsuccessful projects.

Based on the statistical data, which are probabilistic, shown in Table 1, we have proposed a method which relies on a trade off analysis in risk conditions using decision trees [28,29,30].

The decision tree includes a comparison of two alternatives for a project’s development—with and without using SE. There are four outcomes for each option in the decision tree: the completion of the project within the planned schedule and budget (successful project); on time and beyond budget; within budget but beyond a deadline; beyond both budget and deadline.

Figure 1 shows the decision tree constructed for these alternatives and their outcomes. The figure shows data on the shares of projects resulting in the above-mentioned outcomes, expressed as percentages, which are obtained based on statistics (Table 1). Thus, the topmost branch shows that, according to statistics, 22% of the projects carried out without using SE are successful—that is, they fit into the planned time and budget of the project. The lower offsets from this branch show the share of unsuccessful projects—27% of the projects went beyond the budget, 22% went beyond the deadlines, and 29% of the projects failed both in terms of deadlines and budget overruns.

For each outcome—i.e., each branch in the decision tree—the Net Present Value (NPV) must be calculated by discounting all future cash flows (both incoming and outgoing) arising from the investment project at a given discount rate and then adding them together.

NPV is calculated using the formula [31]:

$$\mathrm{NPV}={\displaystyle \sum }_{\mathrm{t}=0}^{\mathrm{N}}\frac{{\mathrm{CF}}_{\mathrm{t}}}{{\left(1+\mathrm{r}\right)}^{\mathrm{t}}},$$

(1)

where:

• NPV is the net present value;
• CF is the cash flow;
• r is the discount rate;
• N is the number of periods, which the project must be estimated at;
• t is the length of time, which the Net Present Value must be calculated for.

The calculation of the Net Present Value includes several intermediate parameters that can be divided into two groups: income and expenses. The profitable part consists of the revenue from the realization of hydrocarbons by the netback costs (the cost of oil on the world market minus the transportation costs and export duties). The cost part includes capital (CAPEX) and operating expenses (OPEX), tax payments, and depreciation charges.

Capital investments include the costs of production wells, as well as the costs of the construction of field facilities, including:

• equipment for oil production;
• equipment for the gathering and transport of products;
• power supply facilities;
• objects of the technological preparation of hydrocarbons;
• arrangement of cluster sites;
• construction of infield roads;
• other expenses for the development of the field.

Operating expenses are formed from the costs of raising, preparing, and transporting products (Lifting Cost), the costs of servicing and repairing wells (Revex), and other expenses.

Amortization for wells and other fixed assets was calculated by the linear method, which is characterized by a uniform transfer of the cost of fixed assets to the expenses of the organization. The useful life when calculating amortization is accepted for 10 years.

Tax payments paid to the federal budget are made up of the export duty, the oil and gas production tax, property tax, income tax, and other taxes. The oil and gas production tax in the tax structure accounts for the largest share of deductions.

The costs of systems engineering in the appropriate scenario options can be calculated in two ways, the choice of which will depend on completeness of the input information and the requirements for calculation accuracy. The first method involves calculating SE costs as a percentage of the project costs, which correlates with the percentage of cost overruns and schedule slips derived from the studied projects. For example, if SE financing takes 8% of investments, a budget overrun by 15%, a deviation from the schedule by 20% in scenarios using systems engineering is expected. The optimal amount of system engineering effort to develop a successful system is 14.4% of the total program cost [14]. However, the costs of systems engineering can be calculated directly for each type of SE activity presented in Figure 2.

Then, for each alternative, the expected cash value of the field development project is calculated for a project without SE—EMV—and for a project with SE: EMV_se. It is advisable to use the EMV indicator for the economic evaluation of high-uncertainty projects, which are particularly consistent with the oil and gas experience. The EMV is calculated using the formula [29]:

$$\mathrm{EMV}={\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}\mathrm{NPVi}\mathrm{x}\mathrm{P}\left(\mathrm{NPVi}\right),$$

where:

• EMV is the expected monetary value;
• NPVi is the Net Present Value corresponding to each of the outcomes;
• P (NPVi) is the probability of obtaining the i-th NPV, which is determined by the previously quoted percentage of projects due to the statistical nature of the event probability concept.

In the node A of the decision-making, a choice is made between the best alternative in terms of the greatest EMV, and the quantitative effect (F) from the introduction of SE is calculated by the formula:

$$\mathrm{F}=\frac{\mathrm{EMV}-{\mathrm{EMV}}_{\mathrm{se}}}{{\mathrm{EMV}}_{\mathrm{se}}}\ast 100\%,$$

where:

• F is the effect from SE introduction, calculated as a percentage;
• EMV is the expected monetary value of the project without SE;
• EMVse is the expected monetary value of the project with SE.

To make multiple calculations for various projects more efficient, the developed method, which includes the constructed decision-tree based on statistical data of projects and mathematical formulas, was automated.

The automated model is implemented using VBA and includes the following modules:

• User interface for inputting initial data on the project and outputting the result.
• The base of use cases, which stores data on known projects.
• The module for search and selection in the base of use cases, which allows using queries to the base of use cases to find projects that meet the specified parameters.
• Calculation module that implements the algorithms of the developed model.
• Database of cost indicators.

3. Results

To assess the effect of SE, we used data on one of Gazpromneft’s fields located on the territory of the Yamalo-Nenets Autonomous regions [32]. Capital investment into the project is about $1 billion.

The NPV for all outcomes—EMV, EMVse, and the F-effect—were calculated taking into account all stages of the field’s life cycle, from concept development to retirement.

Carrying out a series of calculations to find the maximum value of the effect made it possible to determine the optimum costs for SE typical for oil and gas projects of this scale. Time and budget deviation values corresponding to a specific SE cost were derived from Honour’s correlation.

The results of the NPV and EMV calculations for the base project and the project using systems engineering, presented in Figure 3, showed a steady positive effect of 45%.

The set of economic efficiency indicators shown to compare two alternative options for the project implementation is presented in

Table 3. Economic efficiency indicators taking into account the impact of risks for the project without SE and with SE.

	Project without System Engineering		Project with System Engineering
	Payback Period, Years	NPV, Thousands of Dollars	Payback Period se, Years	NPVse, Thousands of Dollars
Project within the planned schedule and budget	5	76 942	5	71 569
Project within time and beyond the budget	8	16 952	5,5	54 425
Project within budget and beyond deadline	6	60 088	5,5	64 278
Project beyond budget and deadline	9,5	3 640	6,5	34 716

.

Simulating the scenario of budget overruns from the initial volume of capital investments affects the size of lost profit due to the increase in the financial burden in the initial period and, as a consequence, the later payback. Leveling the risks of budget overruns in a project using SE allows one to save 87 million dollars and get a 3.2-times higher NPV than in the base case.

The second failure scenario—namely, a slip in project start—reduces the NPV value because the project reaches the oil production plateau later and the revenue from the hydrocarbon sales decreases. The use of SE in this project reduces revenue loss by 34 million dollars, which leads to a 4 million dollar increase in NPV. The peculiarities of taxation and benefits explain the small magnitude of the effect in the time-failure scenario.

Having analyzed economic indicators for projects without systems engineering and with systems engineering, we can conclude that applying the new paradigm to oil and gas projects leads to a stable positive effect due to an increase in the probability of successful project implementation and a decrease in the number of budget overruns and schedule slips.

4. Discussion

The biggest contribution this work made is that it developed a method that would predict the most probable economic effect from applying systems engineering. Systems engineering has already established itself as one of the most effective tools in creating successful systems in aerospace, military, and other industries, which are characterized by high complexity and a large number of system components. The authors tried to find common features between the listed industries and the oil and gas industry and, as a result, established a lot of similar characteristics.

The oil and gas system comprises a large number of subsystems, from the hydrocarbon production subsystem to the external transport subsystem; it is described by a high uncertainty and a large volume of capital investments, which make the oil and gas system complex. An increase in the share of hard-to-recover reserves leads to a constant search for new technologies in field development and new approaches to organizing production. The cost of a failure at the stage of putting a field into operation can be very high, which will entail a complete non-return on investment. Therefore, it is important to develop a project implementation program and schedule at the initial stage of conceptual design to assess possible risks and opportunities, take into account stakeholders’ requirements, and translate them into requirements and architecture.

However, any innovation requires an investment justification before it is implemented. Due to this fact, the authors developed a calculation tool that would take into account the specifics of the oil and gas industry and the specifics of calculating the costs of systems engineering.

The limitation of this method is the correlation of cost of the projects used for trade-off analysis—that is, projects with and without using systems engineering. Collecting statistics on projects’ success to obtain the likelihood of outcomes should be performed depending on projects’ categories, which are small, medium, and large. Thus, the use of SE in a large project will have a higher effect value than in a small project, since the additional costs will be justified. Additionally, projects in the same domain should be compared. However, since today there are no cases of using systems engineering in the oil industry, the authors compared oil projects with investment projects in the transport and defense industry.

5. Conclusions and Future Research

Thus, the objective to evaluate the systems engineering efficiency for an oil and gas company’s specialized project portfolio has been reached.

To quantify the effects from implementing SE methods in a company’s projects, a model based on a decision tree has been developed and used to estimate the probabilistic NPV. The proposed model is based on common methods and initial data, but at the same time it differs from the well-known methods of substantiating the value of SE mentioned in Russian and foreign literature, including the sources of the International Organization of Systems Engineers, INCOSE.

The proposed economic assessment model allows us to go from listing potential benefits to a rigorous, quantitative justification of the economic effect of implementing SE in a company’s projects, and allows us to predict the effect and justify the decision to use SE in projects of various scope.

To develop this domain even further throughout the company, we plan to upscale the accumulated knowledge and experience in assessing efficiency not only for capital projects, but also for organizational, technological, and digitalization strategy projects and others.

We also plan to accumulate experience in the implementation of oil and gas projects using systems engineering practices, which will allow us to form our own database with indicators of projects’ success. Success and failure statistics will be useful for other companies in the oil industry interested in applying systems engineering.