The Development of an Ordinary Least Squares Parametric Model to Estimate the Cost Per Flying Hour of ‘ Unknown ’ Aircraft Types and a Comparative Application †

The development of a parametric model for the variable portion of the Cost Per Flying Hour (CPFH) of an ‘unknown’ aircraft platform and its application to diverse types of fixed and rotary wing aircraft development programs (F-35A, Su-57, Dassault Rafale, T-X candidates, AW189, Airbus RACER among others) is presented. The novelty of this paper lies in the utilization of a diverse sample of aircraft types, aiming to obtain a ‘universal’ Cost Estimating Relationship (CER) applicable to a wide range of platforms. Moreover, the model does not produce absolute cost figures but rather analogy ratios versus the F-16’s CPFH, broadening the model’s applicability. The model will enable an analyst to carry out timely and reliable Operational and Support (O&S) cost estimates for a wide range of ‘unknown’ aircraft platforms at their early stages of conceptual design, despite the lack of actual data from the utilization and support life cycle stages. The statistical analysis is based on Ordinary Least Squares (OLS) regression, conducted with R software (v5.3.1, released on 2 July 2018). The model’s output is validated against officially published CPFH data of several existing ‘mature’ aircraft platforms, including one of the most prolific fighter jet types all over the world, the F-16C/D, which is also used as a reference to compare CPFH estimates of various next generation aircraft platforms. Actual CPFH data of the Hellenic Air Force (HAF) have been used to develop the parametric model, the application of which is expected to significantly inform high level decision making regarding aircraft procurement, budgeting and future force structure planning, including decisions related to large scale aircraft modifications and upgrades.


Introduction
At the present environment of decreasing defence budgets, obtaining the right balance among the components of force structure, modernization, readiness and sustainability of an Air Force is extremely challenging and requires careful analysis of the cost data.Cost Per Flying Hour (CPFH) is the main metric that many Air Forces are using to develop operating and support budgets and is defined as [1]:

CPFH =
Total O&S Costs Total Flying Hours (1) • The proportional CPFH model works: When nothing changes in the way an aircraft fleet flies (and rests) from one period to the next.It is then perfectly reasonable to use flying hours as a predictor for removal-causing failures.When fight behaviour does not change, the failure rate from each potential cause of failures remains constant.

•
The proportional CPFH model fails: When a fleet of aircraft significantly changes its flight behaviour from one time interval to the next.An example of this is the wartime surge (flying hours increase dramatically but landings remain the same) and this is a typical case in which flying hours and the other factors that affect failures begin to diverge.
In a similar work which was carried out by Lee [5], a physics-based model has been developed that considered the ground equipment, flying hours and take-off/landing cycles to predict removals.The model has been applied to the Gulf War C-5B fleet data and it has provided more accurate results than the proportional CPFH model for the Gulf War surge.
Laubacher [6] has researched forecasting techniques for various helicopter types of the USAF, with an objective to reduce the differences between forecasted budgets and actual expenses.A similar study for various helicopter types of the United States Army has been conducted by Hawkins [7].Armstrong [8] has analysed F-15 fleet data to estimate an incremental, rather than average CPFH.Hess [9] has evaluated the method used by the USAF to estimate the Flying Hour Program costs and introduced new methods to forecast future costs.His findings suggest that the assumption of a linear relationship between cost and flying hours is not appropriate.
Knowing the possible factors that cause operational and maintenance costs to fluctuate may allow for better predictions of the cost of the Flying Hour Program of an Air Force.Hawkes and White [10] investigated the predictive ability of many diverse variables such as aircraft age, average sortie duration, base location, utilization rate and engine type among others.Utilization rate, base location, block and engine type appeared to accurately predict an F-16 C/D fighter wing CPFH.Both regression models of their study are sensitive to utilization rates and that they inversely affect the CPFH, which is in good agreement with previous studies [3][4][5].The modern F-16 blocks also decrease the cost of flying, generally older technology is more expensive to maintain.Hawkes and White [11] have also investigated the relationship between the aircraft age and ownership CPFH cost growth from seventy-four different airframes in the USAF inventory.Their findings suggest that very young and very old aircraft platforms exhibit higher levels of cost growth and variability, the magnitude though of the cost growth and variability for old aircraft platforms is almost equal to that of young aircraft.By examining the empirical relationship between multiple USAF system's expenditures, flying hours and fleet sizes, Unger's [12] research has suggested a more sophisticated way to think about USAF costs than is currently used.A fixed-plus-variable cost structure has been proposed in which the expenditures do not increase nor decrease in proportion to flying hours.Unger concludes with the policy implications of his research findings, noting that current USAF budgeting methods likely overestimate funding requirements when flying hours increase and underestimate requirements when flying hours decrease.
The discussion above highlights that the accuracy of a CPFH estimate is of paramount importance, given its use as a main decision-making tool at the upper echelons of the hierarchy of the organization.Error in the cost estimates has been identified as a causal factor in cost overruns and as studies have shown, cost overruns challenge fiscal management in the United States Department of Defense (US DoD) [13,14].As a knock-on effect, they may also lead to funding instability in those programs that did not experience cost overruns but were affected by re-programming of funds.This instability in funding only further exacerbates cost overruns [15].

Materials and Methods
During the procurement process of an aircraft platform, there is an emphasis in affordability and cost management issues.Potential buyers can be always offered reliable estimates for the O&S costs of an aircraft platform which has been in operation and has reached its 'fleet maturity' stage.This is not possible though in the case of comparing and evaluating new ('unknown') aircraft platforms, therefore the decisions should be based, if possible, to cost estimates the variables of which are known at the time of the procurement process.This is one of the main objectives of the paper, to develop a CER which will enable an analyst to carry out a timely and reliable O&S cost forecast, despite the lack of actual data from the utilization and support life cycle stages of the platform, during which the largest portion of the Life Cycle Cost (LCC), nearly 60%, is incurred [16].As such, aircraft's physical and performance characteristics and parameters which are known very well in advance at the aircraft's conceptual design phase, are being researched on the present work as potential variables of an O&S parametric cost estimation model.An introductory demonstration of this concept has been presented by one of the authors at the 2016 International Training Symposium of the ICEAA [17].

The Parametric Estimation Technique
The parametric or 'top-down' technique is a relatively fast and inexpensive estimating tool.Properly applied, it may provide reliable predictions and, most important, timely estimates.According to ISPA/SCEA Parametric Handbook [18]: Parametric estimating is a technique that develops cost estimates based upon the examination and validation of the relationships which exist between a project's technical, programmatic and cost characteristics as well as the resources consumed during its development, manufacture, maintenance, and/or modification.Parametric models can be classified as simple or complex.Simple models are cost estimating relationships (CERs) consisting of one cost driver.Complex models, on the other hand, are models consisting of multiple CERs, or algorithms, to derive cost estimates.
The parametric technique is applicable during the early stages of a system's life cycle, amidst analogy and engineering estimating techniques (Figure 1) [19]: The parametric technique uses regression analysis for estimating the relationships among variables.Regression analysis helps an analyst to understand how the typical value of the dependent variable (response or criterion variable) changes when any one of the independent variables (predictors or explanatory variables) is varied, while the other independent variables are held fixed.

The Strengths and Weaknesses of the Parametric Estimation Technique
The implementation of the parametric technique is a blended process and the interpretation of the results has to be done with extreme caution.An analyst should always consider the following strengths and weaknesses of the parametric technique: Strengths:


It does not require actual and detailed cost information about a new system.Compared to the engineering or 'bottom-up' cost estimating technique, it requires less data, time and resources.


It may reveal strong CERs between cost and Reliability-Maintainability-Supportability (RMS) metrics [20], thus helping to optimize maintenance and logistic procedures.


A parametric model can be easily adjusted when the main cost drivers change.The CERs may be easily updated and sensitivity analysis may be applied.


It is a sound statistical process and can be objectively validated.


The uncertainty of the estimate can be quantified, allowing cost risk analysis.


There are many available COTS parametric tools.Additionally, general-purpose statistical packages support the parametric technique. Weaknesses:

It is a rigorous statistical technique (uses regression analysis). 
CERs are often considered 'black boxes,' especially if they derive from COTS tools with unknown data libraries, and/or if the CER mathematical expression cannot be logically explained.


Appropriate data adjustments might be required before the analysis, depending on the selected regression method (OLS, OLS-Log space, MUPE, ZMPE).Also, standard error adjustments for sample size and relevance might be required [21].


CERs must be frequently updated to ensure validity.


The validity of the PI and CI heavily depends on the residuals diagnostics.


The decision makers may feel uncomfortable to base their final decision on a parametric estimate (probably they will not be statisticians).


Wide-ranging PI or CI may render the estimate useless; why not use the 'rule of thumb' instead?

The Development of the Parametric Model
The relationship between historical CPFH and specific aircraft characteristics is investigated with an objective to identify a strong CER that will be used to estimate the hypothetical CPFH for an The parametric technique uses regression analysis for estimating the relationships among variables.Regression analysis helps an analyst to understand how the typical value of the dependent variable (response or criterion variable) changes when any one of the independent variables (predictors or explanatory variables) is varied, while the other independent variables are held fixed.

The Strengths and Weaknesses of the Parametric Estimation Technique
The implementation of the parametric technique is a blended process and the interpretation of the results has to be done with extreme caution.An analyst should always consider the following strengths and weaknesses of the parametric technique: Strengths:

•
It does not require actual and detailed cost information about a new system.Compared to the engineering or 'bottom-up' cost estimating technique, it requires less data, time and resources.

•
It may reveal strong CERs between cost and Reliability-Maintainability-Supportability (RMS) metrics [20], thus helping to optimize maintenance and logistic procedures.

•
A parametric model can be easily adjusted when the main cost drivers change.The CERs may be easily updated and sensitivity analysis may be applied.

•
It is a sound statistical process and can be objectively validated.

•
The uncertainty of the estimate can be quantified, allowing cost risk analysis.

•
There are many available COTS parametric tools.Additionally, general-purpose statistical packages support the parametric technique. Weaknesses: • It is a rigorous statistical technique (uses regression analysis).

•
CERs are often considered 'black boxes,' especially if they derive from COTS tools with unknown data libraries, and/or if the CER mathematical expression cannot be logically explained.

•
Appropriate data adjustments might be required before the analysis, depending on the selected regression method (OLS, OLS-Log space, MUPE, ZMPE).Also, standard error adjustments for sample size and relevance might be required [21].

•
CERs must be frequently updated to ensure validity.

•
The validity of the PI and CI heavily depends on the residuals diagnostics.

•
The decision makers may feel uncomfortable to base their final decision on a parametric estimate (probably they will not be statisticians).

•
Wide-ranging PI or CI may render the estimate useless; why not use the 'rule of thumb' instead?

The Development of the Parametric Model
The relationship between historical CPFH and specific aircraft characteristics is investigated with an objective to identify a strong CER that will be used to estimate the hypothetical CPFH for an 'unknown' aircraft.Typically, the CPFH includes the following six main cost categories [16] according to the O&S cost element structure: Unit-level manpower, unit operations, maintenance, sustaining support, continuing system improvements and indirect support.Since the purpose of the presented parametric model is the assessment of the relationship between cost and technical or performance characteristics, the 'indirect support' cost category is excluded from the analysis.
The Fiscal Year (FY) 2013 CPFH data from the Hellenic Air Force (HAF) aircraft fleet have been used as input for the developed parametric model (Table 1).The Ministry of National Defence of Greece restricts the publication of the CPFH data for the fighter jet fleet, CPFH data though for other than fighter jet aircraft types are publicly available without any restriction.CPFH data of Table 1 includes the contribution of the 'indirect support' cost category which, as mentioned previously, was excluded from the present study.In more detail, the 2012 CPFH Issue of the OGGG has been based on O&S data collected during the FY 2011, the 2013 CPFH Issue of the OGGG has been based on O&S data collected during the FY 2012 and the 2018 CPFH Issue of the OGGG has been based in analysis of O&S data collected during the FY 2015.At any point in time, the most recently published at that particular point in time CPFH Issue of the OGGG is used to feed the decision making of the Ministry of National Defence of Greece.
The 2013 CPFH Issue of the OGGG, which have been used for the development of the parametric model, is considered as having more reliable data than that of 2012, as the Ministry has transitioned from 2012 to a more GAO-streamlined [22] procedure for categorizing and analysing the cost input data.It was deemed necessary to exclude the contribution of the 'indirect support' cost category when developing the parametric model, as this category is influenced by extrinsic factors (institutional framework, organizational structure, integrated logistic support policies to name a few).As a matter of fact, the 'indirect support' cost category portion in HAF aircraft types ranges from 5% up to 50% of the CPFH, depending on these extrinsic factors.
The HAF fire-fighting fleet (CL-215, CL-415) case can provide some evidence to support our decision to exclude the 'indirect support' cost category.As observed from Table 1 (data including 'indirect support' cost), column 'CPFH FY 2018,' the CPFH of the CL-415 seems to be considerably higher than the CL-215 CPFH.It is worth noting that the CL-415, as opposed to CL-215, is a contemporary aircraft with enhanced reliability and maintainability provisions, equipped with very fuel efficient, latest technology turboprop engines.The paradox in the CPFH difference has to do with the fact that the CL-415 squadron experiences a higher 'indirect support' cost than the CL-215 squadron does.Almost the entire structure of 113th Wing supports the CL-415 squadron; on the contrary, the CL-215 squadron belongs in the organizational chart of 112th Wing, which supports other squadrons as well.Thus, in the case of 112th Wing, the 'indirect support' cost is allocated to multiple aircraft types (generally, each HAF Wing supports multiple aircraft squadrons and/or aircraft types [23]).Nevertheless, if the 'indirect support' cost is excluded, CL-415 experiences lower CPFH than the CL-215, which makes more sense for the comparative purpose of the development of the parametric model.
A generic view of the constraints/requirements and the parametric model performance is presented at the Table 2.The variables used for the analysis are shown at the Table 3.
Table 2. Generic view of the constraints/requirements and the parametric model performance.

Constraints and Requirements Model Performance
Use the sample of 22 aircraft types operated by the HAF.Satisfactory.
Use the appropriate cost information.Satisfactory: Official FY 2013 CPFH data was used, excluding the 'indirect support' cost category.
Use cost drivers (independent variables) that are easily accessible and quantifiable.
Satisfactory: The cost drivers are aircraft physical and performance characteristics.
The model must be as less complex as possible and include no more than two cost drivers.
Satisfactory: The selected model includes two independent variables.
The model should be statistically significant at the 5% level.Satisfactory: p − value < 0.01 The model should capture at least 75% of the CPFH variance.Satisfactory: R 2 adj = 0.82 The model's confidence and prediction intervals must be valid.Satisfactory: The residuals do pass all the appropriate tests.The model's mathematical expression should make sense.
Satisfactory: The model suggests that the aircraft empty weight and the engine SFC correlate positively with the CPFH.
The variables used for the analysis are shown at the Table 3.
Table 3.The variables used for the analysis.In most cases, the log-transformations contributed to the creation of linear simple CERs versus the LogCPFH (Figure 2).Work done by Bryant ([24], Table 2, page 10) offers an overview of independent variables used in previous CPFH research.The variables are grouped in four distinct groups, namely 'aircraft characteristics,' 'operational factors,' 'economic factors' and 'environmental factors.'The most frequently used independent variables by the researchers are 'average aircraft age' (belonging to the 'aircraft characteristics' group) and 'utilization rate' (belonging to the 'operational factors' group), which also reflect the fact that the CPFH research efforts are mainly channelled to find answers to policy hypotheses which are questioning the effects of the aircraft fleet aging and utilization.Instead, the present study aims to provide a 'universal' CER applicable to a wide range of platforms, which can be used at the very early development stages of the 'new' aircraft design and, as such, the independent variables are concentrated to aircraft design and performance characteristics.Moreover, in most cases, the values of the variables are unclassified and they are readily available online, something that enhances the usability of the developed model.
Figure 3 offers an overview of the way that the systems are being classified, based on the selected independent variables and depending on the desired systems similarity level.Because the selected independent variables serve as system identifiers, it is important to examine if the systems are perceptible in a realistic way and identified as 'different' or 'similar' through the regression process.The vertical axis of the cluster dendrogram in Figure 3 corresponds to the level of 'resolution' of the sample's 'image.'The 'resolution' is tied to the independent variables, which serve as system identifiers.
For example, the red horizontal dashed line in Figure 3 corresponds to the selection of a low difference level (high 'resolution').This line cuts 8 branches of the dendrogram, meaning that the 22 different systems are identified and classified as 8 different entities (clusters), at the selected difference level.Specifically, the systems are grouped as follows: The 1st cluster includes two training aircraft (T-2E, T-6A II); the 2nd cluster includes the rotary wing platforms (AB-205, A-109E, AS-332C1, B-212) plus two light fixed-wing, single reciprocating engine, aircraft (T-41D, PZL); the 3rd cluster includes the four-engine transporter C-130H/B; the 4th cluster includes the lighter, two-engine transporter C-27J; the 5th cluster includes the two fire-fighting aircraft (CL-215, CL-415); the 6th cluster includes the twoengine supersonic jet fighter (F/RF-4E); the 7th cluster includes the single-engine supersonic jet fighters (F-16 blocks 30/50/52+, M2000/-5); the 8th cluster includes the VIP and early warning subsonic jet aircraft (ERJ-135BJ/LR, ERJ-145H, GV) plus the A-7H single-engine subsonic jet fighter.Apart from a few pitfalls, the classification of the systems according to the aforementioned clusters seems quite realistic.
Prior to the CPFH Log-transformation, the sample's actual CPFH was multiplied by a positive real c, such that the selected model will estimate CPFH = 1 for the F-16C/D Block 52+.The F-16's CPFH was chosen as the reference point for relative comparisons against other system's CPFH.As a result, Work done by Bryant ([24], Table 2, page 10) offers an overview of independent variables used in previous CPFH research.The variables are grouped in four distinct groups, namely 'aircraft characteristics,' 'operational factors,' 'economic factors' and 'environmental factors.'The most frequently used independent variables by the researchers are 'average aircraft age' (belonging to the 'aircraft characteristics' group) and 'utilization rate' (belonging to the 'operational factors' group), which also reflect the fact that the CPFH research efforts are mainly channelled to find answers to policy hypotheses which are questioning the effects of the aircraft fleet aging and utilization.Instead, the present study aims to provide a 'universal' CER applicable to a wide range of platforms, which can be used at the very early development stages of the 'new' aircraft design and, as such, the independent variables are concentrated to aircraft design and performance characteristics.Moreover, in most cases, the values of the variables are unclassified and they are readily available online, something that enhances the usability of the developed model.
Figure 3 offers an overview of the way that the systems are being classified, based on the selected independent variables and depending on the desired systems similarity level.Because the selected independent variables serve as system identifiers, it is important to examine if the systems are perceptible in a realistic way and identified as 'different' or 'similar' through the regression process.The vertical axis of the cluster dendrogram in Figure 3 corresponds to the level of 'resolution' of the sample's 'image.'The 'resolution' is tied to the independent variables, which serve as system identifiers.
For example, the red horizontal dashed line in Figure 3   The pairwise assessment of the selected independent variables reveals multicollinearity issues.Two or more independent variables may be highly correlated, for example LogEMPTY and LogMTOW, meaning that one can be linearly estimated from the others with a substantial degree of accuracy.A parametric model should not include strongly correlated independent variables, because its predictive ability will decrease.The Pearson correlation matrix (Figure 4) offers an overview of the existing correlations among the transformed variables.Figure 5 shows examples of multicollinearity amongst various variables.The pairwise assessment of the selected independent variables reveals multicollinearity issues.Two or more independent variables may be highly correlated, for example LogEMPTY and LogMTOW, meaning that one can be linearly estimated from the others with a substantial degree of accuracy.A parametric model should not include strongly correlated independent variables, because its predictive ability will decrease.The Pearson correlation matrix (Figure 4) offers an overview of the existing correlations among the transformed variables.Figure 5 shows examples of multicollinearity amongst various variables.The pairwise assessment of the selected independent variables reveals multicollinearity issues.Two or more independent variables may be highly correlated, for example LogEMPTY and LogMTOW, meaning that one can be linearly estimated from the others with a substantial degree of accuracy.A parametric model should not include strongly correlated independent variables, because its predictive ability will decrease.The Pearson correlation matrix (Figure 4) offers an overview of the existing correlations among the transformed variables.Figure 5 shows examples of multicollinearity amongst various variables.

Selection of the Optimal CER
As seen in Figure 4, the highest correlation coefficient between LogCPFH and the independent variables is r = 0.83.Therefore, LogMTOW would be the best choice for developing a simple CER.Unluckily, this model does not comply at least with one of the requirements in Table 2, which is: R 2 adj ≥ 0.75 (indeed, r 2 = 0.83 2 = 0.69 < 0.75).
The next step is to investigate all possible complex CERs with two independent variables, including their interaction, of the form: where  0 ,  1 ,  2 ,  3 are the model's coefficients and ,  ∈ {1,2, … ,6},  ≠ .After performing stepwise regression on all possible (2 out of 6 = 15) models with two independent variables, as shown in Equation ( 2), the following model is chosen, according to the AIC as the measure of the CERs relative quality:  = −4.95006+ 0.4751  + 0.42793 Notably, the two selected independent variables do not correlate significantly (Figures 4 and 6), so there is no multicollinearity in the selected model.Also, the interaction of the two independent variables is not significant, hence the term  3     is omitted from the right hand of the equation.
The selected model explains a remarkable 82.15% of the LogCPFH variance, while the intercept, LogEMPTY and LogSFC have significant explanatory power at the 5% significance level.The assumptions of model's linearity are not rejected at the 5% significance level.Moreover, there is a statistically strong indication that no power transformation is required on LogCPFH.Table 4 shows the regression analysis details obtained from R software (v5.3.1, released on 2 July 2018) and Figure 6 shows a 2D density plot (log-scale) for the model's independent variables.

Selection of the Optimal CER
As seen in Figure 4, the highest correlation coefficient between LogCPFH and the independent variables is r = 0.83.Therefore, LogMTOW would be the best choice for developing a simple CER.Unluckily, this model does not comply at least with one of the requirements in Table 2, which is: R 2 adj ≥ 0.75 (indeed, r 2 = 0.83 2 = 0.69 < 0.75).The next step is to investigate all possible complex CERs with two independent variables, including their interaction, of the form: where β 0 , β 1 , β 2 , β 3 are the model's coefficients and i, j ∈ {1, 2, . . . ,6}, i = j.After performing stepwise regression on all possible (2 out of 6 = 15) models with two independent variables, as shown in Equation ( 2), the following model is chosen, according to the AIC as the measure of the CERs relative quality: Notably, the two selected independent variables do not correlate significantly (Figures 4 and 6), so there is no multicollinearity in the selected model.Also, the interaction of the two independent variables is not significant, hence the term β 3 X i X j is omitted from the right hand of the equation.
The selected model explains a remarkable 82.15% of the LogCPFH variance, while the intercept, LogEMPTY and LogSFC have significant explanatory power at the 5% significance level.The assumptions of model's linearity are not rejected at the 5% significance level.Moreover, there is a statistically strong indication that no power transformation is required on LogCPFH.Table 4 shows the regression analysis details obtained from R software (v5.3.1, released on 2 July 2018) and Figure 6 shows a 2D density plot (log-scale) for the model's independent variables.

Residuals Diagnostics
Getting valid prediction or confidence intervals relies on the assumptions that the residuals are normal with mean zero, have constant variance and no autocorrelations.The residuals of the selected model pass all the necessary tests (Table 5).The residual plots (Figure 7) offer some additional insight, following the tests in Table 5.The histogram indicates normality for the residuals; there are a few high-discrepancy observations (candidate outliers) shown in the boxplot, however the respective test in Table 5 supports the null hypothesis that there are no outliers.The residual plots versus the fitted values indicate absence of curvature pattern and presence of constant variance, supported by the minor slope in the spread-level plot; furthermore, no significant autocorrelation is evident in the respective plots.There are also a few high-leverage observations, however no hat value exceeds the empirical limit for small samples (3 times the hat values mean).The studentized residuals seem to follow the theoretical student-t distribution, as shown in the Q-Q plot.
Observation no.14 needs further attention as being influential for the model, since its Cook's distance exceeds the empirical cut-off limit.This data point corresponds to the C-130H/B, which is the heaviest aircraft within the sample (high-leverage observation); additionally, the model overestimates to a great degree the C-130H/B's actual CPFH (high discrepancy observation).Excluding observation no.14 from the analysis would trigger a vicious cycle of consecutive exclusions, diminishing the size of the sample.Because the sample size is small, we decided not to exclude any observation from the regression analysis.
The construction of prediction or confidence intervals requires the residuals standard error and degrees of freedom (provided in Table 4), as well as the hat matrix.For any given input X T 0 = [1, LogEMPTY 0 , LogSFC 0 ], the hat matrix P ≡ X T 0 X T X −1 X 0 can be calculated using the following information obtained from the sample:  Observation no.14 needs further attention as being influential for the model, since its Cook's distance exceeds the empirical cut-off limit.This data point corresponds to the C-130H/B, which is the heaviest aircraft within the sample (high-leverage observation); additionally, the model overestimates to a great degree the C-130H/B's actual CPFH (high discrepancy observation).Excluding observation no.14 from the analysis would trigger a vicious cycle of consecutive exclusions, diminishing the size of the sample.Because the sample size is small, we decided not to exclude any observation from the regression analysis.
The construction of prediction or confidence intervals requires the residuals standard error and degrees of freedom (provided in Table 4), as well as the hat matrix.For any given input X 0 T = Figure 7.Typical residual plots.
[1,  0 ,  0 ], the hat matrix P ≡ X 0 T (X T X) −1 X 0 can be calculated using the following information obtained from the sample: Figure 8 shows the standard deviation map for the construction of LogCPFH confidence intervals.The highest precision for mean CPFH predictions is obtained when SFC ≈ 0.75 lb/(lbf•h) or lb/(hp•h) and empty weight ≈ 15000 lb.

Predictions on the Training Sample
Results from the comparison of mean CPFH and 95% CI for the training sample are shown at Figure 9 and Table 6.

Model Predictions for the F-35A
The Lockheed Martin F-35 Lightning II is a family of fifth generation, single-seat, single engine, stealth multirole fighters undergoing final development and testing.It is intended to replace a plethora of existing aircraft types in the US Air Force, Navy and Marine Corps, while offering the most technologically advanced, effective and survivable fighter aircraft to date.The F-35 program, also known as the Joint Strike Fighter (JSF), is the most expensive weapon system in history with a projected service life up to 2070 and estimated sustainment costs of about $1 trillion [25].The F-35 is designed and built by an industrial consortium led by Lockheed Martin.Besides the USA, many NATO members and close USA allied nations participate in the funding of the F-35 development.The US DoD plans to procure nearly 2500 aircraft [25] and several other countries have ordered, or are considering ordering the F-35.
Initial O&S cost estimates for the JSF date back to 2001 [26].The focus of that study was to estimate the effect of changes in the reliability to the competition-sensitive O&S costs.As such, at the initial phases of the JSF development program it has been considered that reliability improvements have a strong effect on reducing O&S costs but not enough to recover initial investment.After the selection of the contractor has been made and the program has entered the production phase, the US DoD has moved on (2012) to establish affordability targets for the program, stating that the CPFH for the USAF, Marine Corps and US Navy could not exceed $35,200, $38,400 and $36,300 respectively [25].
The credibility of the O&S cost estimates and affordability targets made by the US DoD seem to have always been questionable though, not only for the F-35 [25] but for mature aircraft types as well [27].As a general theme, the Government Accountability Office (GAO) recommends that the USAF and the Navy should follow all best practices [22,28] to improve the credibility of the CERs, including an investigation of the potential range of costs and, perhaps most importantly, seeking independent cost estimates.Research by GAO has shown that 19 out of 20 independent estimates developed by the Office of Secretary of Defence office of the Director of Cost Assessment and Program Evaluation were higher than the service (US Air Force, Navy) estimate.Past work of GAO has also shown that an independent estimate is usually higher and more accurate than an estimate, which has been made by the Program Office [29].The present work also aims to serve the purpose of providing an independent estimate for various aircraft types and to constructively contribute to the current world-wide debate regarding the F-35 projected O&S costs.The developed parametric model predicts that the ratio of the mean CPFH of F-35A to the mean CPFH of the F-16C/D is 1.25.Figure 11 shows the cumulative distributions of the CPFH for the two systems.
an investigation of the potential range of costs and, perhaps most importantly, seeking independent cost estimates.Research by GAO has shown that 19 out of 20 independent estimates developed by the Office of Secretary of Defence office of the Director of Cost Assessment and Program Evaluation were higher than the service (US Air Force, Navy) estimate.Past work of GAO has also shown that an independent estimate is usually higher and more accurate than an estimate, which has been made by the Program Office [29].The present work also aims to serve the purpose of providing an independent estimate for various aircraft types and to constructively contribute to the current world-wide debate regarding the F-35 projected O&S costs.The developed parametric model predicts that the ratio of the mean CPFH of F-35A to the mean CPFH of the F-16C/D is 1.25.Figure 11 shows the cumulative distributions of the CPFH for the two systems.

Discussion
Developing sound O&S cost estimates for a new aircraft platform without data from its operational phase is a very demanding and complex process and possess significant, mainly non-systematic challenges for the aircraft operators, given that the CPFH estimates are affected by:

•
Various operational variables and most importantly by the fleet utilization rates [2][3][4]10,11].Many of those variables are often beyond the control of the aircraft operators, especially during situations (crisis, war conflicts) which require a sudden surge in aircraft flying hours.

•
Reliability and maintainability improvements [2,30].As above, the improvements are often beyond the control of the operators, they are mainly implemented during the production phase of the aircraft program and they are mainly based on best practices identified by analysing O&S data from historical aircraft projects.As such, there is uncertainty on the potential effect of a reliability and/or maintainability improvement to the O&S cost of a new aircraft program.The authors' professional experience indicates, especially for aircraft types which are operated by many different users across the world (F-16C/D for example), that the implementation of an improvement at the aircraft production line or even later as a modification/upgrade, might not get necessarily translated into O&S cost savings, as different users develop different operational profiles for their fleets, to suit their specific operational needs.

•
The 'maturity' of the fleet.In other words, by when the fleets are at their maximum size.This is because the ramp-up, steady-state and ramp-down phases tend to affect aircraft fleet cost [1].
With regards to the developed parametric model, it is believed that it incorporates important features which enhance its ability to accurately predict the CPFH of an 'unknown' aircraft type:

•
The aircraft sample is comprised by a well-balanced mix of fixed and rotary wing aircraft types which perform a plethora of different missions, from typical military nature (air defence and superiority, interception, aerial support to land and sea military operations) to public service nature ones (fire-fighting, search and rescue, air transportation of VIPs and equipment, medical evacuations).Despite the fact that a high degree of diversification exists in the training sample, a statistically significant CER was obtained consisting of the empty weight and the SFC.This CER seems realistic since both the SFC and aircraft weight are expected to be positively related with the energy consumed during a flying hour.

•
The aircraft sample has reached maturity in terms of maximum fleet numbers and this is considered a very crucial condition for sound cost estimates, especially in case of comparing CPFH of different aircraft types [1].Most of the aircraft sample types are legacy ones with stabilized CPFH, with the newest type (F-16C/D Block 52+Adv) approaching already ten years of operational life within the HAF.

•
There is no significant variation of the utilization rates of the sample, mainly because of the fact that the engagement of HAF aircraft to war conflicts outside its national borders, conditions which require unpredictable aircraft utilization rate 'spikes' for prolonged periods of time, is minimal.The analysis of fleet data during the Operation Desert Storm and the operations at Kosovo [4,5] suggests that the CPFH proportional model fails when a fleet of aircraft significantly changes its flight behaviour from one time interval to the next (wartime surge for example).Furthermore, Boito et al. [1] suggest that the cost of fleets should be compared using stable annual flying hours required for crew proficiency, excluding flying hours for contingency operations.Of course, as many other NATO Air Forces, HAF actively conducts and participates in large scale military exercises within the country and abroad and there might be occasions in which the utilization rates would fluctuate but overall, they are considered as stable on an annual basis.

Conclusions
The accuracy of the cost estimates is also challenged by the variety and the complexity of the missions flown by the military operators.Commercial airline 'cost per available seat mile' is a widely used, simple and effective metric of cost and effectiveness, since fleets at the commercial industry are flown for a common and simple purpose which is easily measured.At the military field, there is a variety of missions and even within a given mission area, a military aircraft often provides many different capabilities.Hence, a 'cost per capability' shall be considered by the military operators as an additional metric for the O&S cost considerations of a military air platform.In such a metric, various capabilities and respective capability levels shall be defined for the fleet, which can then get adjusted on a cost basis.An application of the proposed methodology has been developed to optimize the maintenance of an aircraft system [31].Recent evidence [25] has identified F-35 CPFH as being significantly higher than the legacy aircraft, with an F-35 Joint Program Office member of staff stating that 'this was deemed reasonable taking into the account the complexity of the next-generation aircraft and the additional capability offered by the F-35'.The application of the parametric model for many 'mature' aircraft types for which there exists officially published CPFH data yields very promising results [10,11].The calculated, by the parametric model, ratio (mean CPFH F-35A)/ (mean CPFH F-16 C/D) of 1.25 compares very well with the respective ratio of 'normalized' (CPFH F-35A)/ (CPFH F-16 C/D) of 1.27 (32,554/25,541 (2012$)) reported at the Selected Acquisition Reports (SARs) which have been submitted to the US Congress in 2013 [1].Furthermore, the most recent available SAR estimate for the CPFH F-35 [32], which excludes the 'indirect support' contribution for both F-35 and F-16 C/D, calculates the ratio (CPFH F-35A)/(CPFH F-16 C/D) as 1.17 (29,806/25,541 (2012$)).
In general, it is believed that one of the key conclusions of the present work is that, not only the F-35A but also the majority of the new generation of the advanced fighter jets will not be significantly more costly to sustain than the F-16 C/D, when they will reach their 'fleet maturity status.'It is expected that the present work will inform the decision making and policy considerations of current F-16 C/D users who are considering acquiring a new generation advanced fighter jet fleet, especially in terms of financial forecasting, budgeting and planning of the optimum future force structure.It should also

Figure 1 .
Figure 1.Typical application of estimating techniques through a system's life cycle stages [19].

Figure 1 .
Figure 1.Typical application of estimating techniques through a system's life cycle stages [19].

Figure 2 .
Figure 2. Assessment of the linear model assumptions among the selected variables, using the global test on 4 degrees of freedom.The p-values lower than 0.05 (in red) indicate the rejection of linear model assumptions at the 5% significance level.It is evident that log-transformations enhanced linear relationships among variables.

Figure 2 .
Figure 2. Assessment of the linear model assumptions among the selected variables, using the global test on 4 degrees of freedom.The p-values lower than 0.05 (in red) indicate the rejection of linear model assumptions at the 5% significance level.It is evident that log-transformations enhanced linear relationships among variables.
corresponds to the selection of a low difference level (high 'resolution').This line cuts 8 branches of the dendrogram, meaning that the 22 different systems are identified and classified as 8 different entities (clusters), at the selected difference level.Specifically, the systems are grouped as follows: The 1st cluster includes two training aircraft (T-2E, T-6A II); the 2nd cluster includes the rotary wing platforms (AB-205, A-109E, AS-332C1, B-212) plus two light fixed-wing, single reciprocating engine, aircraft (T-41D, PZL); the 3rd cluster includes the four-engine transporter C-130H/B; the 4th cluster includes the lighter, two-engine transporter C-27J; the 5th cluster includes the two fire-fighting aircraft (CL-215, CL-415); the 6th cluster includes the two-engine supersonic jet fighter (F/RF-4E); the 7th cluster includes the single-engine supersonic jet fighters (F-16 blocks 30/50/52+, M2000/-5); the 8th cluster includes the VIP and early warning subsonic jet aircraft (ERJ-135BJ/LR, ERJ-145H, GV) plus the A-7H single-engine subsonic jet fighter.Apart from a few pitfalls, the classification of the systems according to the aforementioned clusters seems quite realistic.Prior to the CPFH Log-transformation, the sample's actual CPFH was multiplied by a positive real c, such that the selected model will estimate CPFH = 1 for the F-16C/D Block 52+.The F-16's CPFH was chosen as the reference point for relative comparisons against other system's CPFH.As a result, the model estimates the CPFH of any aircraft type times the F-16 mean CPFH.Analysts may use analogy to turn the model's outputs into absolute CPFH estimates, by choosing a known system as their reference and multiplying its actual CPFH by the model's output.Aerospace 2018, 5, x 8 of 21 the model estimates the CPFH of any aircraft type times the F-16 mean CPFH.Analysts may use analogy to turn the model's outputs into absolute CPFH estimates, by choosing a known system as their reference and multiplying its actual CPFH by the model's output.

Figure 3 .
Figure 3.The cluster dendrogram of the 22 systems.

Figure 4 .
Figure 4.The variables correlation (Pearson) matrix.The symbol "×" indicates the insignificant correlations at the 5% significance level.Multicollinearity is evident among several independent variables.

Figure 3 .
Figure 3.The cluster dendrogram of the 22 systems.
the CPFH of any aircraft type times the F-16 mean CPFH.Analysts may use analogy to turn the model's outputs into absolute CPFH estimates, by choosing a known system as their reference and multiplying its actual CPFH by the model's output.

Figure 3 .
Figure 3.The cluster dendrogram of the 22 systems.

Figure 4 .
Figure 4.The variables correlation (Pearson) matrix.The symbol "×" indicates the insignificant correlations at the 5% significance level.Multicollinearity is evident among several independent variables.

Figure 4 .
Figure 4.The variables correlation (Pearson) matrix.The symbol "×" indicates the insignificant correlations at the 5% significance level.Multicollinearity is evident among several independent variables.

Figure 6 .
Figure 6.2D density plot (log-scale) for the model's independent variables.

Figure 6 .
Figure 6.2D density plot (log-scale) for the model's independent variables.

Figure 8 21 Figure 7 .
Figure 8 shows the standard deviation map for the construction of LogCPFH confidence intervals.The highest precision for mean CPFH predictions is obtained when SFC ≈ 0.75 lb/(lbf•h) or lb/(hp•h) and empty weight ≈ 15000 lb.Aerospace 2018, 5, x 12 of 21

Figure 8 .
Figure 8.The standard deviation map for the construction of LogCPFH confidence intervals.

Figure 8 .
Figure 8.The standard deviation map for the construction of LogCPFH confidence intervals.

Figure 9 .
Figure 9.Comparison of mean CPFH and 95% CI for the training sample.

Figure 11 .
Figure 11.The cumulative distribution for the F-35A CPFH compared to the F-16C/D Block52+, according to the developed parametric model.

Figure 11 .
Figure 11.The cumulative distribution for the F-35A CPFH compared to the F-16C/D Block52+, according to the developed parametric model.

Table 1 .
HAF fixed and rotary wing aircraft types used for the development of the parametric model and related CPFH data for FY 2012, 2013 and 2018.(CPFH data for fighter aircraft is classified-publication is restricted.It includes the contribution of the 'indirect support' cost category.Data retrieved by various published issues of the Official Governmental Gazette of Greece (OGGG)).

Table 4 .
Model summary and ANOVA.

Table 4 .
Model summary and ANOVA.

Table 5 .
Summary of residuals diagnostics.

Table 6 .
Comparison of mean CPFH and 95% CI for the training sample.The figures indicate 'times the mean CPFH of F-16 Block 52+.'

Table 6 .
Comparison of mean CPFH and 95% CI for the training sample.The figures indicate 'times the mean CPFH of F-16 Block 52+.'

Table 7 .
Figure 10.A comparative application of mean CPFH predictions with 95% CI for a set of 'unknown' aircraft types.

Table 7 .
A comparative application of mean CPFH predictions with 95% CI for a set of 'unknown' aircraft types.The figures indicate 'times the mean CPFH of F-16 Block 52+.'

Table 7 .
A comparative application of mean CPFH predictions with 95% CI for a set of 'unknown' aircraft types.The figures indicate 'times the mean CPFH of F-16 Block 52+.'