Utility Exchange Traded Fund Performance Evaluation. A Comparative Approach Using Grey Relational Analysis and Data Envelopment Analysis Modelling

Abstract

Selecting funds is a common problem for investors who use published available data on fund indicators while they are selecting the funds. Since this process deals with more than one indicator, the investing issue becomes multi-criteria decision-making (MCDM) problem for the investors. Therefore, the purpose of this paper is to propose an effective approach that integrates grey relational analysis (GRA) and data envelopment analysis (DEA) for selecting the best utility exchange traded funds (ETFs). The current study uses GRA for deriving the grade relational coefficients and then puts them in the output side of competing no-input DEA models to derive weighed grey relational grades. Moreover, the ETFs are also evaluated by selected DEA models. This research is implemented with real data on utility ETFs available for three consecutive years (2008–2010). The results show that the top ETFs identified by the GRA-DEA approach are also DEA efficient. The proposed GRA-DEA approach is superior to conventional DEA as regards the fund ranking and therefore, it seems to be effective as a picking fund tool.

1. Introduction

Utilities have been made an attractive sector for investors who want to diversify their portfolios. Utility exchange traded funds (ETFs) typically mirror the market performance by primarily maintaining investment portfolios of regulated utilities that usually have high and consistent profit margins. The companies mostly covered include those involved in the production or distribution of electricity, natural gas and water. The ETFs that primarily invest in utilities are regarded as one of the best investments that high-risk investors can make in volatile markets as they usually offer high yield subject to risk on individual utility stocks purchased.

In this research, we assume that the potential investors will pick up funds using methods that can handle concurrently performance measures that reflect the objectives of the fund investors (e.g., Sharpe ratio (Sharpe 1966)) and factors associated with the value investing style (e.g., Portfolio price to earnings (P/E) ratio); see also Tsolas and Charles (2015). This fund selection problem turns into a multi-criteria decision-making (MCDM) problem for the investors.

The current study proposes an effective approach for ETF benchmarking that integrates Grey Relational Analysis (GRA) and Data Envelopment Analysis (DEA) for selecting the best utility ETFs. In the DEA-based GRA, DEA is employed after the implementation of GRA and moreover, the ETFs are also evaluated using DEA as a sole method.

GRA as a method has its origin in the Grey theory and it produces the relational degree of every factor/attribute for a set of alternatives. GRA is based on distance measurement and it produces the relational degree between two measurement sequences. It is deemed suitable for problems where multiple factors and their interrelationships must be taken into account. GRA has been employed for solving multi attribute decision-making problems. The concept of the method is to derive for every alternative one attribute value by taking into account all attribute values. Thus, alternatives with multiple attributes are transformed by GRA to alternatives with a single attribute and this transformation facilitates the whole process (Kuo et al. 2008).

In this paper the ETF benchmarking process is analysed as a multi attribute decision-making problem. For the case of ETFs, we treat fund performance and fund characteristics as attributes and ETFs as alternatives.

DEA is employed for the efficiency measurement of same entities (e.g., ETFs) known as decision-making units (DMUs) using their input and output values. The efficiency of each DMU is calculated as the ratio of weighted sums of outputs to inputs; efficiency scores take values between zero (minimum performance) and unity (maximum performance). DEA for each DMU chooses the optimal output and input weights and assigns the maximum efficiency score during an optimization process (Kuo et al. 2008). DEA has the potential to be used as a screening tool to pick up ETFs (Tsolas 2011; Tsolas and Charles 2015). In this paper DEA is used in a proposed GRA-DEA approach for the derivation of weights after the implementation of GRA and as a sole method to appraise ETFs.

The present study complements existing research in two ways. It appears to be the first study that (i) integrates GRA with DEA for ETF performance evaluation and (ii) applies DEA to produce utility ETF performance ratings. This study aims to derive a consolidated measure of performance of utility ETFs by employing both a proposed GRA-DEA framework and the generalized proportional distance function (GPDF) (Kerstens and Van de Woestyne 2011) and other DEA models using data on performance metrics such as the Sharpe ratio and fund characteristics. The purpose of the study is to compare the best-in-class funds identified by employing the above methods.

The remainder of the paper is structured as follows. Section 2 provides a literature review on combined GRA and DEA approaches and DEA based ETF performance evaluation. Moreover, the research gaps in the literature are also identified. In Section 3 the proposed models are presented. Section 4 deals with the data set for the case study that carried out. Section 5 analyses results and the policy implications are presented in Section 6. The final section concludes.

2. Literature Review

The literature review is organized according to following surveys: (i) The survey on combined GRA and DEA approaches and (ii) the survey on DEA based ETF performance evaluation. In addition, the research gaps are identified.

2.1. Survey on Combined GRA and DEA Approaches

In the GRA context, there are DEA models in which only values on the output side are put (Guan and Chen 2013; Wu and Olson 2010). Guan and Chen (2013) integrate GRA with DEA to assess the urban ecosystem by first deriving the grade relational coefficients and then putting them in the output side of a no-input DEA-like model to produce weighed grey relational grades, in the GRA context. Wu and Olson (2010) propose a hybrid DEA model and fuzzy GRA to address the problem of ranking. Other combined GRA and DEA approaches appeared in the literature deal with the selection of DEA inputs and outputs (Bruce Ho 2011; Wang et al. 2010), the ranking of DEA efficient DMUs in applications that use crisp (Girginer et al. 2015) or grey data (Markabi and Sabbagh 2014). Bruce Ho (2011) measures the efficiency of US internet firms using a combined DEA and GRA approach. He selects representative indicators using GRA that they are subsequently used as the input and output indicators in the DEA analysis. Wang et al. (2010) use GRA and DEA to measure production and marketing efficiency of a sample of firms in the printing circuit board industry. Girginer et al. (2015) use first DEA to determine the efficiency of surgical services offered by a public hospital and then employ GRA to rank the efficient services and important variables in efficiency. Markabi and Sabbagh (2014) propose a hybrid method of GRA and DEA for evaluating and selecting efficient suppliers under uncertainties and moreover, they also recommend a novel ranking method for the units that their efficiencies are obtained in the form of interval grey numbers. Huang et al. (2015) integrate the super-efficiency DEA and GRA to select financial ratios in a classification (financial failure prediction) problem by proposing a hybrid financial failure prediction approach that uses a two-level DEA. In a different context, Pakkar (2016a, 2016b) proposes an integration of DEA with analytic hierarchy process (AHP) to derive weights in GRA applications to address the nuclear waste dump site selection and production scheduling problem, respectively.

2.2. Survey on DEA Based ETF Performance Evaluation

Chu et al. (2010) evaluate ETF performance by means of range directional measure (Portela et al. 2004); they put downside risk and expense ratio into the input side and monthly returns and upper deviation into the output side of the DEA model. Tsolas (2011) employed a two-step procedure combining GPDF models with a censored Tobit model to appraise natural resources ETFs. He used inputs that reflect user cost: portfolio price/cash flow (P/CF), portfolio price/book (P/B) and total expense ratio (TER) and traditional performance metrics as outputs: Sharpe ratio and Jensen’s alpha (Jensen 1968). Prasanna (2012) applied DEA for the performance evaluation of Indian ETFs. Tsolas and Charles (2015) appraised the performance of green ETFs using a two-step procedure: First DEA-based ETF ratings were produced by employing slacks-based DEA models and then regression techniques were used to model the ETF ratings. They used the same variables with Tsolas (2011) except for TER.

2.3. Research Gaps in the Literature

GRA has been already employed for addressing MCDM problems such as the facility layout and dispatching rules selection problem (Kuo et al. 2008), supplier selection (Markabi and Sabbagh 2014), nuclear waste dump site selection (Pakkar 2016a) and production scheduling problem (Pakkar 2016b). To the best of the author’s knowledge GRA has not applied for addressing the fund selection problem, thus the current research aims to propose a framework for using GRA and DEA to tackle this issue. In regard to ETFs, GRA has been applied together with artificial neural networks for high technology ETF forecasting (Chen et al. 2013). In the current study the focus is on the utility ETFs that belong to non- technology ETFs.

In addition, the current research contributes to the existing literature by employing DEA as a sole method for utility ETF appraisal.

3. Methods

3.1. Grey Relational Analysis

In GRA, the calculating procedure is as follows:

Let X₀ be the referential series with j = 1, …, k attributes (i.e., fund performance, fund characteristics) of X₁, X₂, ..., X_m (i = 1, …, m ETFs). The referential series are the target values of the attributes in the decision-making modelling that determines the reference point of the MCDM problem. Then:

$$X_0=x_0(1),x_0(2),....,x_0(j),\dots ,x_0(k)$$

(1)

where k = total number of attributes and x₀ = target value of the attributes.

The decision matrix of the problem is constructed using i = 1, …, m series:

$$X_i=x_i(1),x_i(2),....,x_i(j),\dots ,x_i(k),i=1,\dots ,m$$

(2)

The grey relational coefficient (GRC) between the series X_i and the referential series of X₀ at the jth attribute is calculated as:

$${\xi }_{0i}(j)=\frac{\Delta \min +p\Delta \max }{\Delta x_{0j}(j)+p\Delta \max }$$

(3)

where:

$$\Delta x_{0j}(j)=|x_0(j)-x_i(j)|$$

(4)

$$\Delta \max =\underset{i}{\max }\underset{j}{\max }{\Delta }_{0j}(j)$$

(5)

$$\Delta \min =\underset{i}{\min }\underset{j}{\min }{\Delta }_{0j}(j)$$

(6)

and p = 0.5.

The Integrated GRA-DEA Model

Having derived the GRC for all alternatives, DEA based weights are used to calculate the grey relational grade (GRG). In the current study, GRG for the series X_i is calculated by employing the following DEA like model:

$$Max{\displaystyle \sum _{i=1}^m{w}_i{\xi }_{ji}}$$

(7)

$$\begin{array}{l}s.t\\ {\displaystyle \sum _{i=1}^m{w}_i{\xi }_{ji}}\le 1j=1,2,\dots ,k\end{array}$$

(7a)

$${\displaystyle \sum _{i=1}^m{w}_i=1}$$

(7b)

$$0.1\le \frac{w_i}{{\displaystyle \sum _{i=1}^m{w}_i}}\le 0.5$$

(7c)

The above model is different from that proposed by Guan and Chen (2013). The difference is in the last restriction on weights (i.e., weights must be greater of 0.1 and less of 0.5 of the sum of all weights, respectively). This restriction does not allow the model to assign zero weights and weights with very high values. The selection of the weight bounds in this restriction is arbitrary, although a sensitivity analysis can be performed (Tsolas 2008).

In the conventional GRA, equal weights to attributes for each alternative are assigned but it should be noted that each of the alternatives has its own characteristics and preferences and therefore the use of equal weights may be questioned. DEA can address this issue by deriving different weights for each alternative but its use may cause some problems (Pakkar 2016a):

(i)

DEA may assign zero or very high values to the weights of some attributes and thus neglect the priorities of these attributes in the decision-making process.

(ii)

Several alternatives may receive GRGs equal to unity and thus causing a ranking problem because all of these alternatives are ranked in the first position.

To tackle the first problem the weight restrictions (7b) and (7c) can be used.

To overcome the second problem, that is, the weakness of DEA in ranking the alternatives, the super-efficiency model of Andersen and Petersen (1993) can be used. According to this model, the alternatives which had a GRG equal to unity are allowed to obtain a GRG greater than unity; the values of GRGs that were lower to unity remain the same.

From model (7) and (7a)–(7c), if we omit the restriction (7b) and replace the restriction (7c) with $w_i\ge 0$ the new model is the benefit of the doubt (BOD) model proposed by Cherchye et al. (2007) and it is equivalent to the original input oriented DEA model of Charnes et al. (1978), with all GRCs considered as outputs and a dummy input equal to one for all the alternatives. Wu and Olson (2010) have proposed the same (i.e., BOD) model with optional the same weight normalization constraint (7b) and they have recommended modifications of this model with the weight normalization constraint and other weight constraints added.

For the case of robustness, we use the super efficiency BOD (SE-BOD) model. The use of super efficiency has been also proposed by Huang et al. (2015). The use of SE-BOD model provides DEA-based GRA where the optimal GRGs obtained take positive values that may be greater than unity. Empirical results derived by the SE-BOD model are an addition to the existing literature since the aim of the BOD model as introduced was the derivation of composite indicators that take values between zero and unity.

Before the calculation of the GRC, the data in our case must be normalized according to larger the- better and smaller the-better principles (Lin et al. 2006). These normalizations are as follows:

1. Larger the-better:

$$x_i^{*}(j)=\frac{x_i(j)-\underset{j}{\min }{x}_i(j)}{\underset{j}{\max }{x}_i(j)-\underset{j}{\min }{x}_i(j)}$$

(8)

2. Smaller the-better:

$$x_i^{*}(j)=\frac{\underset{j}{\max }{x}_i(j)-x_i(j)}{\underset{j}{\max }{x}_i(j)-\underset{j}{\min }{x}_i(j)}$$

(9)

where $\underset{j}{\max }{x}_i(j)$ and $\underset{j}{\min }{x}_i(j)$ are the maximum and minimum value of performance dimension j, respectively.

3.2. The DEA Method

DEA (Charnes et al. 1978) is used to measure the performance of Decision-making Units (DMUs). With DEA an efficient frontier is constructed that is used to estimate the DMU efficiency. The derived efficiency scores take values between zero (minimum efficiency) and unity (maximum efficiency).

A DEA model can be formulated in oriented (i.e., input or output oriented) and non-oriented versions. Input orientation aims to input reduction, whereas output orientation to output expansion. Non-oriented analysis aims at both input reduction and output expansion.

The Generalized Proportional Distance Function (GPDF)

In DEA, the performance of m ETFs, i = 1, …, m, is achieved by distinguishing input-like values X ∊ ${\mathcal{R}}^{{\mathrm{s}}_{+}}$ that used to generate output-like values Y ∊ ${\mathcal{R}}^{\mathrm{p}}$; the amounts of the nth input-like and rth output-like values, used by the ith fund are denoted by x_in and y_ri, respectively. The GPDF metric $D(x,y;-\left|x_{ino}\right|,\left|y_{rio}\right|)=\beta$ for each ETF is estimated as follows (Kerstens and Van de Woestyne 2011):

$$\begin{array}{l}Max\beta \\ s.t\\ {\displaystyle \sum _{i=1}^m{\lambda }_i{y}_{ri}\ge y_{ri0}+\beta \left|y_{ri0}\right|}r=1,2,\dots ,p\\ {\displaystyle \sum _{i=1}^m{\lambda }_i{x}_{in}\le x_{in0}-\beta \left|x_{in0}\right|}n=1,2,\dots s\\ {\displaystyle \sum _{i=1}^m{\lambda }_i}=1\\ \lambda \ge 0\end{array}$$

(10)

A metric value equals to zero indicates efficiency, whereas a value greater than zero indicates inefficiency. An inefficient ETF with $\beta >0$ should simultaneously expand output-like values by $\beta y_r,r=1,2,\dots ,p$ and reducing input-like values by $\beta x_n,n=1,2,\dots ,s$.

If the restriction ${\displaystyle \sum _{i=1}^m{\lambda }_i{x}_{in}\le x_{in0}-\beta \left|x_{in0}\right|}n=1,2,\dots s$ is replaced by ${\displaystyle \sum _{i=1}^m{\lambda }_i{x}_{in}\le x_{in0}}n=1,2,\dots s$ the new model will become the output version of the GPDF (GPDF-O). The new model’s scores that are not equal to zero also indicate inefficiency. An inefficient ETF with $\beta >0$ should expand output-like values by $\beta y_r,r=1,2,\dots ,p$.

4. Data Set

The input-output variables on ten utility ETFs used in this study cover the three-year period from 2008 to 2010. The reason for considering only ten ETFs in this study is the availability of complete information on these funds. The funds evaluated are: First Trust ISE Water Idx (FIW), First Trust Utilities AlphaDEX (FXU), Guggenheim S&P Global Water Index (CGW), iShares Dow Jones US Utilities (IDU), iShares S&P Global Utilities (JXI), PowerShares Dynamic Utilities (PUI), Rydex S&P Equal Weight Utilities (RYU), Utilities Select Sector SPDR (XLU), Vanguard Utilities ETF (VPU) and WisdomTree International Utilities (DBU).

Since we want to compare the results of GRA-DEA with independent DEA models such as the GPDF we select the variables for ETF performance evaluation in line with the DEA literature. There is no consensus among researchers and investors as to which ratios as indicators of inputs and outputs should be included in a DEA model unambiguously. The ratios used in the current study as indicators of inputs are the portfolio P/E ratio and TER. The Sharpe ratio is used as indicator of output. The ratios used are explained in

Table 1. Ratio explanations.

Ratio	Ratio Explanation	Ideal Value
Portfolio P/E ratio	Portfolio price to earnings (P/E) ratio is the average ratio of price to annualized earnings of stocks in an ETF.	Minimum
TER (%)	Total expense ratio (TER) is the percentage of fund assets that fund manager may withdraw each year to pay for operating expenses.	Minimum
Sharpe ratio	Sharpe ratio is a risk-adjusted measurement of fund performance. It is a measure of excess portfolio return over the risk-free rate relative to its standard deviation.	Maximum

.

Descriptive statistics of data used in the assessments are depicted in

Table 2. Descriptive statistics of utility exchange traded funds (ETFs) data used in the assessments.

	Sharpe Ratio	Portfolio P/E Ratio	TER (%)
Mean	−0.27	13.54	0.51
Standard deviation	0.20	1.81	0.17
Median	−0.30	12.94	0.54
Min	−0.60	11.52	0.22
Max	0.04	17.46	0.70

Notes: Sharpe ratio: 3 years-Sharpe ratio.

.

5. Results

The GRA-DEA approach is used to evaluate utility ETFs on portfolio P/E ratio and TER (smaller the- better) and Sharpe ratio (larger the-better) by deriving a DEA-based GRG. These results are then compared with the results produced by GPDF models.

5.1. GRA-DEA Results

The GRG is calculated by using the proposed GRA-DEA approach (Model (7) and (7a)–(7c)) and the SE-BOD model.

The model (7) and (7a)–(7c) provides GRG values that are all lower than unity, therefore there is not a ranking problem of the alternatives. The SE-BOD model provides GRG values greater than unity.

5.2. GPDF Results

In the current study, two GPDF models are employed: The non-oriented model (10) and its output-oriented version, the only alternative oriented model due to the Sharpe ratio negative values.

The efficient (i.e., best-in-class) ETFs that can be used as benchmarks, as they stem from both the GPDF-NO and GPDF-O models, are the same top five funds identified by employing GRA-DEA. The GRG scores are ranked in ascending order, whereas according to the GPDF models results the funds are classified into efficient and inefficient (

Table 3. Results of data envelopment analysis (DEA)-based grey relational analysis (GRA) and generalized proportional distance function (GPDF) models.

Ticker	GRG (GRA-DEA)	Rank	GRG (GRA-SE-BOD)	Rank	GPDF-O	Fund Classification	GPDF-NO	Fund Classification
FIW	0.6881	4	1.3441	1	0	EFF	0	EFF
FXU	0.6837	5	1.0803	4	0	EFF	0	EFF
CGW	0.5276	10	0.7846	10	3.0991	INEFF	0.1647	INEFF
IDU	0.5817	8	0.8549	8	3.8081	INEFF	0.0451	INEFF
JXI	0.6232	6	0.8714	7	5.5425	INEFF	0.0397	INEFF
PUI	0.5672	9	0.8401	9	4.7348	INEFF	0.0521	INEFF
RYU	0.5898	7	0.8845	6	0.0247	INEFF	0.0013	INEFF
XLU	0.8528	1	1.1435	3	0	EFF	0	EFF
VPU	0.7664	2	0.9787	5	0	EFF	0	EFF
DBU	0.6933	3	1.2626	2	0	EFF	0	EFF

Notes: EFF: Efficient fund; INEFF: Inefficient fund.

).

For the case of robustness, we investigate the consistency of results of DEA as a sole method of ETF evaluation with the aid of other DEA models that can handle negative values such as the BCC input oriented model (Banker et al. 1984) and the additive model (Charnes et al. 1985). The input-oriented BCC model is translation invariant with respect to outputs and the additive DEA model is translation invariant in both inputs and outputs (Cooper et al. 2007). The results produced by these models indicate that the efficient and inefficient funds remain the same. The results are available upon request from the author.

The DEA-based GRA is superior to conventional DEA as it provides a full ranking of funds and therefore, it seems to be effective as a picking fund tool.

6. Policy Implications

The research framework (integrated GRA-DEA and independent DEA models) can be used by fund managers to monitor the performance of their funds and financial investors to appraise investments.

In general, the findings have managerial implications for those who use and develop fund rating models. Because the use of equal weights for the attributes in GRA may be questioned, GRA can be integrated with DEA to address this problem following the proposed methodological framework. Moreover, the proposed DEA-based GRA is superior to conventional DEA due to limitations of DEA to attain a full fund ranking and therefore, it seems to be effective as a picking fund tool.

The current research provides acceptable results using real available data on utility ETFs by employing GRA-DEA and DEA as a sole method of fund appraisal.

In addition, it is useful to know the performance rating of each individual fund and the results of this study can also be disseminated targeting the potential value investors. Utility ETFs may be an appropriate vehicle for those who seek for a defensive investment and moreover, in bear markets they have appealed for the yields and cushion that can provide.

7. Conclusions

This paper proposes the integration of GRA with DEA to address the fund selection problem, for the case of utility ETFs. DEA is used in the proposed GRA-DEA approach for the derivation of weights after the implementation of GRA and also as a sole method to appraise ETFs. The integration of DEA with GRA relaxes the equal weights assumption for the attributes which may be questioned. To overcome the possible ranking problem of alternatives, weight restrictions may be used or a super efficiency model may be employed. In particular, we recommend the use of both a weight restriction and SE-BOD model without weight restrictions. In a case study that carried out, the results of the GRA-DEA are compared with the results of independent DEA models such as the non-oriented and output-oriented GPDF models and the BCC input oriented and the additive model.

The top five funds identified by employing the DEA-based GRA are the efficient funds that stem from the use of DEA as a sole method of fund appraisal. Therefore, these competing methods (i.e., DEA-based GRA and DEA as a sole method) can discriminate the best-in-class funds (i.e., benchmark funds).

The results produced in this paper represent a first attempt at assessing utility exchange traded fund performance. Several limitations to this study may be recognized. First, the small number of funds used can be seen as limitation. Future researchers should probably use a bigger number of funds in order to validate our findings, probably using a class of other ETFs because the utility ETFS in the market are few. Second, the alternative models to the GPDF which used to tackle the problem of negative data may provide additional insights and future researchers must investigate further on how they can be integrated into fund selection problem.