A Forecast Model for the Level of Engineering Maturity Impact on Contractor’s Procurement and Construction Costs for Offshore EPC Megaprojects

Abstract

This paper focuses on the influence of detailed engineering maturities on offshore engineering, procurement, and construction (EPC) project procurement and construction cost performance. The authors propose a detailed engineering completion rating index system (DECRIS) to estimate the engineering maturities, from contract award to beginning of construction or steel cutting. The DECRIS is supplemented in this study with an artificial neural network methodology (ANN) to forecast procurement and construction cost performances. The study shows that R² and mean error values using ANN functions are 20.2% higher and 19.7% lower, respectively, than cost performance estimations using linear regressions. The DECRIS cutoff score at each gate and DECRIS forecasting performance of total cost impact were validated through the results of fifteen historical offshore EPC South Korean mega-projects, which contain over 300 procurement cost performance data points in total. Finally, based on the DECRIS and ANN findings and a trade-off optimization using a Monte-Carlo simulation with a genetic algorithm, the authors propose a cost mitigation plan for potential project risks based on optimizing the engineering resources. This research aids both owners and EPC contractors to mitigate cost overrun risks, which could be continuously monitored at the key engineering gates, and engineering resources could be adjusted per optimization results.

1. Introduction

Changes in an engineering, procurement, and construction (EPC) project are inevitable. Per the EPC model, contractors are responsible for procurement and construction risks arising from engineering maturities which have not been completed during the ‘define stage’ of projects. During the last two decades, EPC contractors in South Korea have carried out a large number of offshore EPC mega projects. Over 80 mega-projects including fixed platforms and floaters were ordered to the EPC contractors amounting to about 70 billion dollars, 15% of the total worldwide capital expenditures for topside oil production facilities. The Korean EPC contractors encountered numerous changes in the detailed engineering phases of the projects, often resulting in significant profit losses [1].

According to sample data from one historical project that was completed by a Korean EPC contractor from 2014 to 2017, approximately 40%~45% of the total project price was for major equipment and bulk material procurement. In addition, about 20%~25% was for the labor cost of construction activities. Therefore, over 60% of the total project cost was related to direct costs of procurement and construction. In projects that experienced significant cost increase due to growth in the construction and procurement stage, these cost increases were attributed to a combination of low engineering maturities and fast-tracking trends of EPC projects. Therefore, EPC contractors must be aware of the project cost risks before the contract award and/or during the detailed engineering execution stage.

With regard to the contractual obligations of an EPC contract, EPC contractors accept the single point of responsibility of both design and construction. Through front-end engineering and design (FEED) endorsements, EPC contractors are responsible to ensure that project execution will meet and exceed the clients’ project goals and purposes. After the FEED endorsement, clients typically do not accept any change orders even though FEED packages have often not been fully matured. Contradictorily, EPC contractors are not able to transfer these design change risks to the vendors so any further design change occurring after the equipment supply contract or the bulk material purchase order is considered as a vendor change order, which should be compensated to the contractor. Most engineering aspects must be fully defined prior to the procurement of major equipment and bulk material. We found, for example, that on the aforementioned historical project, client change orders relative to the vendor change order were less than 20% of the vendor change orders. The remaining 80% had to be covered solely by EPC contractors.

In order to reduce the project risks caused by low engineering maturities on offshore EPC mega-projects, Kim et al. [1] had developed a qualitative score for the engineering maturities using the detailed engineering completion rating index system (DECRIS). This was used to predict their impacts on project construction cost. While it is a valid addition to the general body of knowledge, construction costs are not the only project cost risks, as procurement cost risks can also have significant impacts. Therefore, the main purpose of this research is to predict the project procurement cost impact by the DECRIS framework and to develop a mitigation plan that covers the engineering, procurement, and construction risks.

1.1. Existing Literature

Existing literature covers critical factors, which cause cost overruns for both domestic and international construction projects. Shibani and Arumugam [2] studied the factors that lead to cost overruns in Indian construction projects by surveying of experts in construction fields. They reported that regular changes of contractors were indicated by 71% of the participants. An analysis by Hsieh et al. [3] showed that 23.7% of design variation orders originated from issues within the planning and design stages on 90 historical projects in Taipei. Ssegawa et al. [4] similarly reported that 45.7% of change orders resulted from either additions and/or omissions in the design, the most common cause of the project cost growth. The above studies report the importance of detailed design maturities to reduce additional cost impact on the projects. Kaming et al. identified 16 factors causing cost impact [5]. Frimpong et al. [6] and Creedy [7] highlighted 24 and 31 factors respectively. These three studies indicated that inaccurate cost estimate during execution as well as design error is one of the major causes of cost impact. The above represents the most applicable publications. The authors performed a holistic review of existing literature pertaining to critical cost impact factors on international projects. Of the total 30 literature articles reviewed, 57% have also mentioned the impact of inaccurate cost estimates.

Ibbs [8] studied the quantitative impacts of project changes that contributed at least 30% to the entire change order. After reviewing over one hundred projects, said publication determined that 0.34% of productivity decrease and 0.88% of construction labor cost increase could be expected per 1% of project change. This result means that project change globally affects the project cost performance, therefore reducing these project changes are one of the key parameters to keeping the labor productivity high and cost impact low [8]. Wang et al. [9] depicted that the relationship with clients and engineers is especially important for contractors to deliver EPC project within schedule and cost performance. Using different data analysis, they found that design management is critical to project cost performance. In addition, with successful design management, contractors can also take long-term business opportunities by demonstrating engineering competence [9].

Table 1. Researches specifying inaccurate cost estimate and design changes as main reason of cost impact.

Authors	Project Location	Inaccurate Cost Estimate	Design Changes *
Thomas and Napolitan [10]	-		O
Kaming et al. [5]	Indonesia	O	O
Kumaraswamy et al. [11]	Hong Kong		O
Ssegawa et al. [4]	South Africa		O
Frimpong et al. [6]	Ghana	O	O
Hsieh et al. [3]	Taipei		O
Long at al [12]	Vietnam	O
Elhag et al. [13]	UK		O
Acharya et al. [14]	Korea	O	O
Arain and Pheng [15]	Singapore		O
Moura et al. [16]	Portugal		O
Harisaweni [17]	Indonesia	O	O
Oladapo [18]	Nigeria		O
Azhar et al. [19]	Pakistan	O	O
Le-Hoai et al. [20]	Vietnam	O	O
El-Razek et al. [21]	Egypt		O
Enshassi et al. [22]	Gaza Strip	O	O
Ameh et al. [23]	Nigeria		O
Memon et al. [24]	Malaysia		O
Mohammad et al. [25]	Malaysia		O
Rahman et al. [26]	Malaysia	O	O
Halwatura and Ranasinghe [27]	Sri Lanka	O	O
Shibani and Arumugam [2]	India	O	O
Gunduz and Maki [28]	-	O	O
Yadeta [29]	Ethiopia	O	O
Ogunsanmi [30]	Nigeria		O
Creedy et al. [7]	Australia	O	O
Baloi and Price [31]	Developing Countries	O	O
Iyer and Jha [32]	India	O	O

summarizes research studied previously to identify which factors mostly affect the cost performance of construction projects. Most of them indicate that inaccurate cost estimates at the bidding stage and design changes during construction are the most critical factors that affect the entire project cost performance. Previous studies with the main factors highlighted were categorized by project locations in the various regions, i.e., Australia, East Asia, South-East Asia, Middle Asia, West Asia, Europe, and Africa.

Dikmen et al. [33] defined the project risks that are associated with construction, design, payment, client, and sub-contractor (procurement) risk. Publications estimated the overall project risk using a fuzzy risk assessment. The fuzzy set provides a magnified risk rating for each risk and for overall project risks. The magnified risks are limited in that the risks cannot be forecasted as a cost unit. Kim et al. [34] provided the prediction model using a DECRIS framework that estimates construction cost impacts by assessing detailed design maturities, optimizing the engineering and the construction impacts. However, the optimization model is limited to the calculation of only the construction impact. In estimating EPC project performance, the procurement cost impact must also be combined with the engineering and construction impacts. Yoo et al. [35] studied the importance level of 52 risk elements through expert survey and weighing the risks for EPC nuclear power plant projects, and concluded that the schedule of procurement and delivery is the most critical risk factor to project success. By measuring each risk level, the overall project cost and schedule could be monitored and forecasted [35]. Although the risk management in the study was well defined and structured, assessment methods of the risk rating, however, are qualitative and uncertain. A practical and objective approach to assess the risk rating is required to enhance the accuracy of cost and schedule performance forecast of the project.

Most of the literature described the importance of the design maturities on estimating contract prices and performing the EPC execution without fatal design changes, which cause cost impacts on procurement and construction. In other words, most of the previous research was limited to the acknowledgement and identification of factors affecting project cost performance without any development of practical application methods or/and tools to mitigate these interacting factors. Especially, there is little research so far focusing on engineering management processes for upstream projects in energy infrastructure to assess and measure the interaction between major gates for EPC projects, i.e., the maturity (completion progress) level of detailed engineering and procurement for equipment/material purchasing, and construction initiation.

1.2. Point of Departure and Research Contribution

Kim et al. [1] have previously described use of the DECRIS framework to define important elements of detailed engineering and correlate these elements to project construction performance. In addition, Kim et al. [34] suggested the forecasting and mitigation method using DECRIS framework at five key gates (from FEED verification to construction start). The most significant objective of this research is to expand the DECRIS framework to procurement cost performance and to determine any correlation between the DECRIS model and the project procurement performance at these key engineering gates. This validity of the proposed model’s added value is tested using artificial neural network (ANN) best-fit curves. Finally, this research proposes potential cost risk mitigation measures (including engineering, procurement, and construction consequences), using a Monte-Carlo simulation with genetic algorithm for optimization. This research contributes to industry by providing EPC contractors a tool which accurately determines procurement and construction risks at the initial stage and suggests potential mitigation measures, balancing engineering resources based on the assessment of detailed engineering maturities. With this suggested DECRIS model, project owners and EPC contractors may be able to monitor the required progress for engineering–procurement–construction at each engineering gate.

2. Research Methodology

The major purpose of this research is to expand the DECRIS prediction and mitigation model from construction performance to combine engineering, construction, and procurement performances. Therefore, the study first builds off the existing DECRIS model, which is correlated to the prediction and mitigation of the project construction cost and schedule impact.

Figure 1 describes the research procedures used in the application of the DECRIS model in procurement to a case study based on sample projects and to the model validation. In the DECRIS model application section, the DECRIS model assigns seven gates: five gates for construction risk review and another two gates for procurement risk review. The correlation between the DECRIS calculation and project performance will be reviewed using the ANN model. In the following section, a case study assessing the capability of the DECRIS model to predict procurement performance for an offshore EPC mega-project will be performed. To verify the model, a DECRIS cut-off score, which was confirmed in the previous research, will be reviewed. Finally, the prediction performance of the ANN model will be discussed. There are three main methodologies applied in this research, as well as a set of statistical methods such as independent T-test, F-test, and linear regression.

2.1. Existing DECRIS Model

The methodology needed to calculate the detailed engineering maturities with the DECRIS framework [1] was previously reported. The DECRIS model consists of 69 major factors called “elements” that affect detailed design maturities for EPC projects. Using the DECRIS level assessment of each element at the specific stage of EPC execution stage, the detailed engineering maturity could be qualitatively calculated within the range of 70 to 1000. A lower DECRIS score means higher design maturities for the EPC contractor. A correlation between the resulting DECRIS score and the project construction cost and schedule performance in the fourteen historical projects, which were completed in South Korea by an offshore EPC contractor, were statistically validated using various tools such as t-test, linear regression, fuzzy set qualitative comparative analysis, and fuzzy inference system. Additionally, the DECRIS cut-off score to ensure the project performance was established as 810, 660, 500, 380, and 300 at the five key engineering gates (i.e., (1) FEED verification, (2) major equipment procurement, (3) 60% modeling review, (4) 90% modeling review, and finally (5) steel cutting). A trade-off optimization concept using Monte Carlo simulation was proposed and validated with case studies [34]. With the DECRIS assessment and optimization, the project owner and the EPC contractor can monitor the detailed design completion rate affecting the construction risks and mitigate construction cost risks with the optimization method. The following three equations were defined in the previous DECRIS research [1,34], and required to forecast engineering, construction and procurement impacts, as further described in this paper.

Construction labor hours increase rate (CLIR)
= Construction labor hours increased by design change/Planned construction labor hours

(1)

Vendor change order rate (VCOR)
= Vendor change order cost/Original purchase order price

(2)

Engineering resource enhancement rate (ERER)
= Additional engineering labor hours/Planned engineering labor hours

(3)

2.2. Artificial Neural Network

The artificial neural network is an effective tool used to solve various issues related to curve fitting, prediction, pattern interpretation, clustering, time series forecasting, etc. For example, Kim et al. adopted the neural network model in estimating construction costs in the construction project fields [36]. In a general manner, ANN consists of a number of neurons, which include input, weight, bias, transfer function and output [37]. The input data is calculated with weight and bias using a transfer function of a neuron (which was empirically decided by a researcher). The weight of each neuron is continuously adjusted throughout the machine running, which are categorized as supervised learning, unsupervised learning, and reinforcement learning. If a designated model is not diverse, the weight and the bias of each neuron will be stable after a sufficient number of iterations of forward propagation and back-propagation.

The following Figure 2 shows a sample of a multilayer perceptron mode.

In the previous DECRIS research [1,34], linear regression was used for forecasting construction cost performance. In some cases, forecast performance of a model could be enhanced with non-linear regression. In order to increase forecast performance in this research, the ANN model was applied to find out the best-fit curve between the DECRIS score at the dedicated key gate and the project actual performance. Multilayer perceptron is a useful tool to perform curve fittings by supervised learning when researchers know a set of input data and the corresponding labels. When the dataset was constructed, data normalization processes were required to reduce the number of iterations with the equation set below. After an initialization process using a set of weights and bias (such as Widrow and Nguyen), the iteration of the machine learning could be started as the following formula.

Forward propagation
a₁⁽²⁾ = g(θ₁₁⁽¹⁾x₁ + θ₁₂⁽¹⁾x₂ + θ₁₃⁽¹⁾x₃ + b₁)
a₂⁽²⁾ = g(θ₂₁⁽¹⁾x₁ + θ₂₂⁽¹⁾x₂ + θ₂₃⁽¹⁾x₃ + b₂)
a₃⁽²⁾ = g(θ₃₁⁽¹⁾x₁ + θ₃₂⁽¹⁾x₂ + θ₃₃⁽¹⁾x₃ + b₃)
a₁⁽³⁾ = h(x) = h(θ₁₁⁽²⁾a₁⁽²⁾ + θ₁₂⁽²⁾a₂⁽²⁾ + θ₁₃⁽²⁾a₃⁽²⁾)

(4)

Back propagation
s^M = −2h’^M(n^M)(t − a) and s^m = g’^m(n^m)(W^m+1)^Ts^m+1, m = M − 1, …, 2, 1
θ ^m(k + 1) = θ ^m(k) − αS^m(a^m−1)^T and b^m(k + 1) = b^m(k) − αS^m
S⁽²⁾ = −2h’⁽²⁾(n⁽²⁾)(t − a⁽²⁾) and S⁽¹⁾ = g’⁽¹⁾(n⁽¹⁾)(W⁽²⁾)^TS²

(5)

where, ai^(j) = activation of unit i in layer j; θ^(j) = matrix of weight controlling the function from layer j to layer j+1; g(x), h(x) = the transfer functions of the output and the hidden layer, respectively; M = number of layer, k = number of learning, t = corresponding label data, and α = a learning rate

When curve fitting using ANN, a combination of log-sigmoid (at the hidden layer) and linear (at the outlet layer) was generally applied [37]. Through the iteration of forward and backward propagation, sum squared error (SSE) is subsequently decreased up to a stable condition. The performance of ANN after training is verified with the SSE histogram and with linear regression analysis between outlet data and actual data. Kim et al. found that a linear regression model could be used for the correlation function with considerable statistical significance [1,34]. However, in several cases, R² between the performance and DECRIS is lower than 0.5. In this research, ANN is adapted to find the better or the best fit to explain the correlation between DECRIS and actual performance. The target for R² value was greater than 0.6 with preferred values up to 0.7~0.8 using ANN curve fitting. The result in the next section gives the preferable range of R² value, which was found better than the R² value found using simple linear regression.

2.3. Trade-Off Using Monte-Carlo Simulation

Monte-Carlo Simulation is widely used to optimize difficult multi-dimensional models such as investment cost, resource optimization, priority of the project sequence, and warehouse optimization. Input variables are set up and deterministic algebraic functions are arranged to calculate an optimum point. After verification of inlet distribution function such as normal distribution or beta distribution, simulations are performed with several thousands of iterations. The results are shown as deterministic value of optimum point and probabilistic ranges of output variables [38].

In this research, total cost impact was calculated with deterministic algebraic function including construction and procurement cost performance, cost of engineering resource enhancement and cost deduction caused by enhanced engineering maturities. With series of simulations using commercial software (at-Risk), a deterministic optimum point of engineering resource enhancement was proposed and probabilistic ranges of total cost impacts were discussed. The optimum point was rechecked using optimization with genetic algorithm was also performed to find the exact engineering resource enhancement and total cost impact value.

3. DECRIS Model for Forecasting Procurement Performances

3.1. Definition of Key Engineering Gates for Major Equipment Procurement

In previous research, five (5) key engineering gates were defined as (1) FEED verification, (2) major equipment procurement, (3) 60% modeling review, (4) 90% modeling review, and (5) steel cutting. FEED verification (gate 1) and major equipment procurement (gate 2) were collected for the study of the influence of engineering maturity on equipment procurement. At the FEED verification session, the expected VCOR was calculated. The mitigation plan was set to satisfy the required DECRIS value at gate 2. Engineering maturity at gates 3 to 5, which occurs later than equipment procurement, does not impact on the procurement cost and therefore was not included in this study.

3.2. Data Collection from Historical Projects

Equipment purchasing data was collected from 15 historical projects. The quantity of the data is 324 items and their total cost amount is about 1.7 billion dollars. In order to review the correlation between DECRIS and VCOR, purchased cost amount and change order amount were calculated per each project.

Hagan et al. [37] described that at least 15% of the data should be used for ANN network test and the selected data shall not be used for learning of the network. Projects E, G, and O were selected through the random sampling, and DECRIS scores and VCOR results in the remaining 12 projects were calculated for the network training.

Prior to starting a neural network, linear regressions were performed for brief-checking the relationship between DECRIS score and VCOR, and R² values were found as 0.4246, 0.5451 at gates 1 and 2 respectively.

Table 2. DECRIS score and actual vendor change order rate (VCOR) on fifteen historical projects.

Project	DECRIS Score at Gate1	DECRIS Score at Gate2	VCOR	Sampling
A	817	661	4.36%	training
B	872	725	22.22%	training
C	785	630	7.14%	training
D	822	665	12.17%	training
E	790	644	1.59%	test
F	814	687	13.48%	training
G	806	661	4.46%	test
H	824	668	3.94%	training
I	790	628	2.93%	training
J	799	641	5.18%	training
K	808	638	2.51%	training
L	796	630	8.55%	training
M	813	658	7.21%	training
N	848	696	7.15%	training
O	841	701	8.11%	test

shows the details of DECRIS values and VCOR for 15 projects.

3.3. Forecasting Procurement Performance Using Artificial Neural Network

The purpose of using a neural network technique is to find an optimum curve fit that explains the relationship between engineering maturity (calculated by the DECRIS model) and procurement cost performance (defined as VCOR). With an appropriate fitting curve, contractors can predict the project’s procurement cost performance for the major equipment. A multi-layer perceptron model with supervised learning was selected, and Hebbian learning was used for backpropagation algorithm. As for linear regression, the input of the network is the DECRIS score and the output is VCOR. There are two possible methods of data normalization to perform learning through the neural network efficiently, i.e., by min–max normalization and by Z-score normalization. In this paper, both normalizations were initially applied. It was concluded that the min-max normalization procedure is better due to the narrower range of the normalized input data (−1 to 1) than from the Z-score normalization (−1.07 to 2.49). The next step of the neural network model was to determine the number of neurons. We first attempted to use the various numbers of neurons (such as 2, 3, 5, and 10) to find the best-fit model that maximizes R². We concluded that use of five neurons at the hidden layer (in Figure 3) gave the best performance. The detailed results are discussed in Section 5. The weight initialization was performed using the Widrow and Nguyen method as described in the previous section.

The log-sigmoid function was selected as the transfer function for the hidden layer and a linear function was used for the output layer to distribute the result in the range of −1 to 1, as shown in Figure 4. After 100 iterations were carried out on the 12 project data sets (in random order), the following curve fit was found after continuous set of ANN iterations and fine-tuning of learning rate to provide the greatest R².

As per the designed neural network model, the following vector formula could be established as shown in the following Equation (6). Equation (7) also show the learned formula as an algebraic expression for the relationship between DECRIS score and VCOR at FEED verification (gate 1).

a² = G(W²F(W¹p+b¹) + b²)

(6)

where, a² is the calculated VCOR value [1 × 1], W¹ [5 × 1] and W² [1 × 5] are vector of weight factors of 1^st and 2^nd neuron layer and b¹ [5 × 1] and b² [1 × 1] are bias of 1^st and 2^nd layer.

y = 1.075/(1 + e^{−(1.617x − 2.130)}) + 0.929/(1 + e^{−(1.430x − 2.041)}) + 1.088/(1 + e^{−(1.631x − 2.138)}) +
0.239/(1 + e^{−(0.513x − 1.971)}) + 1.467/(1 + e^{−(2.012x − 2.458)}) − 0.893

(7)

where, x is DECRIS score, y is VCOR.

3.4. Forecasting Construction Performance Using Artificial Neural Network

Similarly, the neural network was applied to establishing a best-fit function for the relationship between the DECRIS score and the project construction performances at each of the five key engineering gates. The DECRIS score at each gate and the CLIR (which was calculated in the previous study by Kim et al. [34]) were applied to the ANN model. The repeated learning using 100 iterations with a learning factor adjustment was carried out, and the best-fit functions found are shown in Figure 5. Network test result using two test projects and R² comparison will be discussed later.

Table 3. SSE value for final five iterations during ANN learning.

Risks	Gate	SSE
		n-4	n-3	n-2	n-2	n *
CLIR	#1	0.0861	0.0861	0.0861	0.0862	0.0862
	#2	0.0967	0.0966	0.0965	0.0964	0.0963
	#3	0.0965	0.0965	0.0964	0.0963	0.0963
	#4	0.1025	0.102	0.1015	0.1011	0.1006
	#5	0.0923	0.0921	0.0919	0.0916	0.0914
VCOR	#1	0.1626	0.1626	0.1626	0.1626	0.1626
	#2	0.1633	0.1631	0.1630	0.1628	0.1626

* n is the number of iterations. CLIR: construction labor hours increase rate.

specified the SSE value of last five iterations of ANN. For all seven sets of ANN, differences of SSE between iterations are lower than 0.001, i.e., less than 1% of SSE. As shown in the previous Figure 4 and Figure 5, the ANN curves were not over-fitted and SSE is converged.

4. Case Study: Forecasting Cost Performance of an EPC Project at Bidding Stage

4.1. DECRIS Assessments of a Sample Project

In order to review the performance of the ANN functions to define the relationship between DECRIS scores and the project cost performances, three projects were randomly selected amongst the 15 historical projects, which were DECRIS assessed in the researchers’ previous study, as described in Section 4. The brief details are shown in

Table 4. Sample project brief for cast study. EPC: engineering, procurement, and construction. EPCIC: Engineering, Procurement, Construction, Installation and Commissioning. EPCI: Engineering, Procurement, Construction and Installation.

Project	Project E	Project G	Project O
Area	South-East Asia	West Africa	South-East Asia
Project type	Fixed Platform	Floater	Floater
Project contract type	EPC	EPCIC	EPCI
Project period	35 months	32 months	39 months
DECRIS score (Gate 1)	790	806	841
Actual VCOR	1.59%	4.46%	8.11%

, below.

4.2. Sample Project Performance Prediction (ANN Network Test)

Using the ANN function, the predicted procurement/construction cost performance of major equipment and the predicted construction cost performance were calculated. Figure 6 shows the predicted performance and actual VCOR of the Project E, G, and O, as summarized in Table 4.

Table 5. Comparison of ANN and actual results on Projects G and E.

		CLIR					VCOR
Gate		Gate 1	Gate 2	Gate 3	Gate 4	Gate 5	Gate 1	Gate 2
Project G	ANN Forecasting	4.10%	5.01%	6.32%	2.76%	6.51%	5.93%	6.53%
	Actual	6.02%	6.02%	6.02%	6.02%	6.02%	4.46%	4.46%
	Difference	1.91%	1.01%	−0.30%	3.26%	−0.49%	−1.47%	−2.07%
	Deviation	0.42 σ	0.22 σ	−0.07 σ	0.7 σ	−0.11 σ	−0.26 σ	−0.37 σ
Project E	ANN Forecasting	2.87%	3.27%	3.09%	Outlier *	4.57%	5.55%	5.34%
	Actual	1.61%	1.61%	1.61%	Outlier	1.61%	1.59%	1.59%
	Difference	−1.26%	−1.66%	−1.48%	Outlier	−2.96%	−3.96%	−3.75%
	Deviation	−0.27 σ	−0.36 σ	−0.32 σ	Outlier	−0.65 σ	−0.71 σ	−0.67 σ

* On Project E, actual date of the gate 4, 90% modeling, are later than gate 5, work order.

describes that ANN forecast value of CLIR and VCOR. On Project G, CLIR values forecasted by ANN model at gates 1 to 5 were 4.10%, 5.01%, 6.32%, 2.76%, and 6.51%, respectively. On the other hand, in the same project, VCOR at gate 1 and 2 were calculated as 5.93% and 6.53% as well. The prediction result shows that the deviations for both CLIR and VCOR were ranged between 0.07 to 0.42 sigma except for CLIR on gate 4. This indicates that the ANN model can properly forecast the project procurement and construction cost risks for Project G. The data on gate 4 could be explained by the time gap of gate 4 (AFC P&ID) and gate 5 (Steel Cutting) on Project G is just 17 days, consequently the DECRIS score at the gate 4 (360) did not decrease substantially in gate (346). This relatively short time gap (between gate 4 and 5) on Project G is an offset from the normal project schedule with about a 3 months gap between the gates (i.e., 4 and 5). These results suggest that the project cost risks cannot be properly mitigated due to the low engineering maturities at the steel cutting session on Project G. The comparison between Actual versus the ANN forecast model for Project E showed a similar trend. In more detail, the deviations between Actual and ANN model for Project E showed the range of −0.27 to 0.71.

Overall, the VCOR deviations for Project E were larger than the ones for Project G, as compared in Table 5 below. The procurement tracking history for Project E revealed that one major equipment package was removed as a result of the final detailed engineering, after the purchase order was already in place. Consequently, the scope reduction should be made in equipment purchasing, which resulted in the total VCOR for Project E being relatively larger than other projects.

4.3. Mitigation Plan Using Trade-Off Optimization

At this point, all datasets to define the mitigation plan were prepared. In order to determine the optimum point for a trade-off between engineering, procurement, and construction cost impacts, A Monte-Carlo simulation was performed. Several assumptions were needed to ensure the simulation model convergence.

• Input parameters taken from historical projects are distributed as normal distribution. Its appliance was already verified as shown in the research of Kim et al. [34].
• Ibbs described that productivity in the design phase is decreased up to 80%~93% of normal productivity when project changes occur [8]. In this research, 80% efficiency was considered in case of engineering resource enhancement.
• The slope of input variables against DECRIS scores is not linear as shown in the previous section taken by neural network. However, we used the constant slope of the trend line of linear regression to perform the trade-off optimization.
• 10,000 iterations were considered per one simulation.
• For verification purposes, optimization using at-Risk commercial program was also performed with genetic algorithm.

Three (3) variables were collected. “A” is the ERER, “B” is CLIR and “C” is VCOR. The following deterministic algebraic formula was set to calculate an optimum point of the trade-off model. At the specific gate, the cost impact of the VCOR and CLIR is estimated using ANN fitting functions. The engineering maturities at the next gate are improved if additional engineering resources and labor hours are committed. In this case, additional cost impacts due to engineering resource enhancement are spent and cost deduction on VCOR and CLIR are expected with the faster decrease of DECRIS values than a normal case. If a slope of VCOR and CLIR deduction is stiffer than a slope of ERER, the total cost impacts can be reduced. When the engineering is sufficiently matured, the DECRIS scores are lower and predict a lesser cost impact in procurement and construction. The cost of further enhancement of engineering resource will be a burden of the EPC contractors. The optimum point could be expected at the point that the slope of VCOR plus that of the CLIR is the same as for ERER. The trade-off concept can be described as below.

Total Cost Impact = C_VCOR + C_CLIR + C_ERE − C_VCORD − C_CLIRD

(8)

where, C_VCOR (Cost of VCOR) = Total Equipment Cost (US$) × C (%); C_CLIR (Cost of CLIR) = Total Construction Labor hours (MH) × B (%) × Construction MH cost (US$/MH); C_ERE (Cost of ERER) = Total Engineering Labor hours (MH) × A (%) × Engineering MH cost (US$/MH); C_VCORD (VCOR Deduction) = Total Equipment Cost (US$) × VCOR Deduction due to Engineering Enhancement (%); C_CLIRD (CLIR Deduction) = Total Construction Labor hours (MH) × Construction MH cost (US$/MH) × CLIR Deduction due to Engineering Enhancement (%)

The trade-off results are shown in Figure 7. The optimal point of total change order for Project E and G was calculated with genetic algorithm (50% crossover rate, 10% mutation rate, 1000 trials and 10,000 iterations per trial). The simulation was finished when maximum changes within 100 trials were less than 0.01%. Figure 8 describes the optimization progress through the number of trials. The total cost impacts of the Project E and G were 3.4 million dollars (at 1.76%) and 3.7 million dollars (at 1.26%), respectively.

To check the trade-off optimization in mega EPC projects, the authors selected one more project sample, the largest project amongst the 15 historical projects as judgment sampling and the same trade-off model was calculated. Figure 9 shows the probabilistic results of the set of simulations between 0 to 25% of ERER. The optimum point is around 15.5%. Using a genetic algorithm with 200 trials and 10,000 iterations per trial, the optimum point was calculated. Crossover rate and mutation rates were set 0.5 and 0.1 respectively. The optimum ERER was simulated as 15.47% with 57.03 million dollars total cost impact.

From this study, the authors had two major observations. First, the effect of ERER is larger when higher DECRIS scores at gate 1 (FEED verification) were assessed. Second, three case study results commonly describe that ERER up to an optimum point could reduce the uncertainty in the total cost impact. A 95% boundary (1.96 sigma) was narrowed from 70 million dollars (no ERER) to 40 million at the optimum trade-off point. It can be explained that sufficient engineering resources were essentially required to decrease cost impacts on the major portion of project costs in procurement and construction.

5. Application and Validation

5.1. Validation of DECRIS Cut-Off Score

Kim et al. [34] explained that the DECRIS cut-off score (to ensure proper construction cost and schedule performance) could be successfully validated. In this study, DECRIS cut-off scores at each engineering key gate were rechecked to define a relationship between the project cost performance and the DECRIS cut-off score. The entire VCOR data on 15 historical projects were selected, and included of 324 items. This data was divided into two groups, variable 1 (VCOR of the equipment procured after the DECRIS cutoff score is achieved) and variable 2 (VCOR of the equipment procured before the DECRIS cutoff score is achieved). At FEED verification (gate 1), 324 items divided with 115 items in variable 1 and 209 items in variable 2. Through F-test of two samples for variances, P value was calculated as 0.0000054, much smaller than 0.05. This means that the variance of each sample set is unequal. In addition, the independent t-test result shows that the P-value of two-tail calculation is 0.016, which is also less than 0.05. Therefore, the two samples that were divided with the DECRIS cutoff score (810) statistically differ.

The same F-test and T-test were performed at gate 2, and the P-values of F-test and T-test were calculated as 0.0000063 and 0.038 respectively for the two-sample divided with DECRIS cutoff score (660). Therefore, DECRIS cutoff scores at gate 1 and gate 2 were validated.

5.2. Statistical Comparison of ANN and Regression Result

Through the ANN model, correlation functions between DECRIS score and CLIR at the five key engineering gates and another set of correlation functions between DECRIS score and VCOR at two key engineering gates was found. For validating the model performance, R² values and mean error values of seven ANN correlation functions were compared with the same values of seven linear regression results.

As shown in

Table 6. Statistical comparison of ANN and regression analysis for Project G.

	CLIR					VCOR
Gate	Gate 1	Gate 2	Gate 3	Gate 4	Gate 5	Gate 1	Gate 2
R2 (Linear Regression)	0.739	0.731	0.594	0.660	0.686	0.425	0.545
R2 (ANN)	0.771	0.764	0.760	0.792	0.796	0.611	0.681
R2 Increase	4.33%	4.51%	28.0%	20.0%	16.0%	44.0%	24.9%
Mean Error (Linear Regression)	0.242	0.239	0.286	0.228	0.238	0.366	0.314
Mean Error (ANN)	0.203	0.213	0.202	0.181	0.184	0.280	0.268
Mean Error Increase	−16.1%	−10.9%	−29.4%	−20.6%	−22.7%	−23.5%	−14.6%

, R² of linear regression ranged between 0.425 and 0.739 with a mean value, 0.626. The values of ANN function are from 0.611 to 0.796 with a mean value, 0.739. R² values of ANN functions are 20.2% higher than linear regressions on average. In addition, the mean error values at the seven gates using ANN fitting curve were 19.7% lower than linear regressions on average. The detailed mean error values are summarized in Table 6 below. With two kinds of statistical parameters, the researchers found that performance of non-linear regression using ANN is superior in comparison with the previous method.

6. Conclusions

6.1. Summaries and Contributions

The objective of this study was to expand the DECRIS prediction and mitigation model from construction performance to engineering, construction and procurement performances. Using procurement data for over 300 items in 15 historical EPC mega projects, correlations between the engineering maturities and actual project cost performances were determined. In addition, statistical differences of VCOR at the two groups divided by DECRIS cutoff score have proven using statistical tools.

The predictive function was enhanced with an ANN model, which resulted in 20.2% higher R² values and 19.7% lower mean error values compared with linear regression. Through a case study using the ANN model, we found that predicted construction and procurement performances are well matched with the actual project cost performances in three test projects.

In order to provide the suggestion of DECRIS application to EPC contractors, a trade-off optimization using Monte-Carlo simulation was performed. The trade-off optimization result showed that the optimum point of engineering enhancement could be calculated and that the procurement and construction risks could be effectively mitigated using additional engineering resources to quickly mature detail engineering and design. The total cost impact could be reduced by about 40% based on the simulation results. The actual cost impacts of three historical projects were found to be within the 95% range of statistical significance.

The research indicates that the DECRIS model can properly predict the project procurement and construction risks; and by the use of enhanced engineering resources, the total cost impact on engineering, procurement and construction activities could be optimized. EPC contractors or clients can adapt the DECRIS model at the following project activities.

• At the bidding session, proposal teams of EPC contractors can review the FEED engineering maturities using the DECRIS model and can estimate the project procurement and construction risks as cost units. Then, EPC contractors can decide whether the cost risks are to be taken or to be incorporated into their bid price as contingency or allowance.
• On the project starting session, project management teams of EPC contractors can calculate the optimum range of engineering resources to minimize the cost impact risks during project execution.
• Before major equipment procurement, purchasing teams of EPC contractors can predict the VCOR and decide when purchase orders of the major equipment could be placed. If the detailed engineering is pre-matured and the expected delivery of the equipment has some free float by comparing a “required on site” date, and then they can adjust the date of the purchase order, when feasible.
• At the construction starting stage that is the most important decision of EPC execution, the EPC contractor’s decision maker can make a macroscopic review of the detailed engineering maturities and judge “go” or “no-go” decision for steel cutting. If construction cost risks caused by poor engineering maturities is not within the acceptable range, they can adjust the steel cutting and continue the engineering progress up to the DECRIS cut-off score.
• During the early stage of EPC execution, clients can monitor the project risks using the DECRIS model. Engineering progress or material procurement status shows just the details of EPC activities; however, it is not easy to project the project cost or the schedule risks with only limited information. With the involvement of a DECRIS assessment team, they can properly monitor the engineering, procurement, and construction risks and request mitigation plans to EPC contractors for reducing DECRIS score over the threshold at each engineering key gate.

6.2. Discussions and Future Research

The authors have tried to make the DECRIS model applicable and sustainable. With an additional learning process of a neural network, a fitting function to predict the project procurement and construction performance could be continuously updated using the neural network whenever new input (project CLIR and VCOR result) comes. In addition, trade-off optimization of total cost impact is active values along with the engineering key gate and the DECRIS score measured by the assessment team.

The historical data of the current DECRIS model is limited to the EPC project results from the offshore mega-project in South Korea. Applicability of the DECRIS model to the other industrial fields or to the other nations, which have different company organizations or project characteristics, has not been established. The DECRIS model can be enhanced/developed for a variety of project types with additional study.