Performance of US and European Exchange Traded Funds: A Base Point-Slack-Based Measure Approach

Abstract

This study evaluates the performance of United States (US) and European Exchange Traded Funds (ETFs) using the non-oriented version of the base point-slack-based measure (BP-SBM) Data Envelopment Analysis (DEA) model, which allows for handling negative data that can arise in some of the metrics traditionally used in this type of analysis. Our findings show that US efficient ETFs are considered benchmarks more often than European efficient ETFs. Nonetheless, it was possible to conclude that European inefficient ETFs were generally less inefficient than US ETFs. Our findings also show that ETFs’ efficiency (particularly for US ETFs) in the short run is more related to risk than to profitability factors. This implies that as the time horizon lengthens, the importance of profitability factors for the ETFs’ financial performance grows.

1. Introduction

ETFs are open-end investment funds that aim to replicate the return and risk of their reference index (Gastineau 2001; Neves et al. 2019; Tsolas 2022). Since the early 1990s, the popularity of ETFs has been continuously growing. ETFs offer diversification benefits and indirect access to overseas equities that are otherwise inaccessible or hard to acquire due to their low fees, transparency, liquidity, and tax efficiency (Dragomirescu-Gaina et al. 2021; Bowes and Ausloos 2021; Tsolas 2022; Henriques et al. 2022). Most of the research concerning ETFs has been devoted to examining their performance (e.g., Roll 1978; Golany and Roll 1989; Gouveia et al. 2018; Poterba and Shoven 2002; Kostovetsky 2003; Zopounidis et al. 2010; Hill et al. 2015; Neves et al. 2019; Navratil et al. 2021; Henriques et al. 2022). The performance assessment of these funds is generally carried out according to risk-adjusted measures (Admati and Ross 1985; Blitz and Huij 2012; Osterhoff and Kaserer 2016; Lettau and Pelger 2020; Lobato et al. 2021). The Sharpe ratio (Sharpe 1966) and Jensen’s Alpha (Jensen 1968) are the indicators mostly used to measure the performance of ETFs (Tsolas 2022; Henriques et al. 2022). However, besides the criticism inherent to their isolated use, i.e., the need to find benchmarks and the significance of market timing (Tsolas 2022; Henriques et al. 2022), there is also the challenge of combining them into a single metric (Murthi et al. 1997; Henriques et al. 2022; Tsolas 2022).

This article aims to explore the different risk-adjusted performance measures used in assessing ETFs. To this end, the BP-SBM DEA model (Tone et al. 2019) is used to address the difficulties associated with the consideration of negative values that usually arise in this type of analysis and to combine into a single score the multiplicity of metrics available in the assessment of ETFs’ performance.

In this context, the DEA approach originally proposed by Charnes et al. (1978) can be particularly useful (Basso and Funari 2001, 2007, 2014, 2016; Proença et al. 2023). DEA is a mathematical programming methodology that allows assessing the efficiency of a set of units called Decision-Making Units (DMUs), in our case the ETFs under evaluation, whose performance is characterized by multiple inputs (risk indicators) and outputs (profitability indicators). With this methodology, DMUs are classified as efficient or inefficient according to a single efficiency score. Additionally, DEA enables finding the benchmarks of inefficient DMUs, providing managers with valuable information regarding the best practices to be followed. Furthermore, the DEA methodology has been accepted and used to measure the performance of ETFs because it allows for the elimination of limitations associated with traditional performance measures (Murthi et al. 1997; Galagedera and Silvapulle 2002; Choi and Min 2017; Kiymaz 2019).

In fact, Murthi et al. (1997) highlighted the advantages offered by this tool in the assessment of portfolio performance. First, this methodology does not require any theoretical reference model (e.g., CAPM or APT). The efficiency of each fund (DMU) is measured against a set of efficient funds within the same category. Second, it allows simultaneously considering both risk (inputs) and profit measures (outputs), resulting in a single performance assessment score.

In this framework, the literature devoted to the use of DEA in the performance assessment of ETFs can be grouped into three main branches. The first considers the single use of DEA models. In this vein, Chu et al. (2010) used a range directional measure (RDM) (Portela et al. 2004) to evaluate the performance of IShare World exchange-traded funds (ETFs), including major countries in Asia Pacific, Europe, North America, and some emerging markets, from 2006 to 2009. The inputs considered in their study were downside risk, and expense ratio, and the outputs were monthly return and upside deviation. Prasanna (2012) employed the Charnes, Cooper, and Rhodes (CCR) DEA model (Charnes et al. 1978) to evaluate the performance of 82 ETFs floated and traded on the Indian stock market. They considered as inputs the standard deviation, maximum drawdown, and monthly downside deviation. The profitable month’s percentage and the compounded monthly return were used as outputs. Acharya et al. (2015) assessed five gold ETFs versus four index ETFs during 2009–2013 with the CCR and Banker, Charnes, and Cooper (BCC) DEA model (Banker et al. 1984). The standard deviation and maximum drawdown have been used as inputs, whereas market value, annual return, and the Sharpe ratio have been used as outputs.

Choi and Min (2017) analyzed the efficiency of the KOSPI 200 index, the eight ETFs that follow it, and the 200 stocks that make up the index. The RDM was used to measure the efficiency of the ETFs. The standard deviation and systematic risk were used as inputs, and the return of these funds was considered an output. Isakov (2019) assessed the performance of ETFs on the Xetra platform (Germany) through the CCR model. The expense ratio and downside risk-return were used as inputs, and the upper deviation was used as an output.

The second uses two-stage DEA assessments intending to identify performance-related factors. After obtaining the efficiency scores from the first stage, regression models are used to find the reasons behind inefficiency. In this context, Tsolas and Charles (2015) employed a two-step technique to evaluate natural resources ETFs, integrating the generalized proportional distance function (Kerstens and Van de Woestyne 2011) with a censored Tobit model. Tsolas and Charles (2015) used a two-stage process for evaluating the performance of “green ETFs” using the range-adjusted measure (RAM-BCC) DEA model, Tobit censored, ordinary least squares (OLS), and boot-strapped-truncated regression. The performance was measured by considering as inputs the price cash flow (P/CF) and price book (P/B) and as outputs the Sharpe ratio and Jensen’s Alpha.

The third combines DEA with other techniques. In this context, Tsolas (2019) assessed the efficiency of ten “utility ETFs” through grey relational analysis, the BCC model, and the additive model. The factors considered were the P/E ratio and expense ratio as inputs and the Sharpe ratio as output. Henriques et al. (2022) evaluated the performance of 60 ETFs in the energy sector from 2014 to 2018. The Weighted Russell Directional Distance DEA model was combined with a multiobjective interval portfolio model to measure the performance of these ETFs. Beta, standard deviation, and Jensen’s Alpha were used to measure risk (inputs), while the Sharpe index, mean annual return, and trailing total return were used to measure profitability (outputs). Later, Tsolas (2022) also used the superefficient RAM DEA model to assess the performance of utility ETFs.

Although the advantages associated with the use of the DEA methodology to assess ETFs’ performance have been well documented, it can be established that the available scientific literature is not very prolific (Gregoriou 2006; Gregoriou and Henry 2015). Overall, only a few studies (e.g., Tsolas and Charles 2015; Choi and Min 2017; Tsolas 2022) have dealt with negative data. Additionally, from the literature review conducted, it can be concluded that most of the studies used DEA models that are radial (unrealistically considering proportionate changes of either the inputs or the outputs) or oriented (either fixing the outputs or the inputs) models. As a result, unlike the CCR and BCC models, we employ the SBM model (Tone 2001), which offers a more comprehensive analysis of efficiency. The SBM model is non-radial (i.e., inputs and outputs are allowed to have non-proportional changes) and non-oriented in its analysis of efficiency (i.e., inputs and outputs can change simultaneously). In the SBM model, all possible improvements in inputs or outputs are fully considered in the objective function. This is a key feature of the SBM model, as it ensures that all opportunities for improving efficiency are thoroughly analyzed and considered. The main feature of the SBM model is that it leaves no input or output slack unaccounted for, i.e., all possible improvements are exhausted and properly considered in the objective function.

All in all, this work aims at contributing to the existing literature by further exploring the use of DEA in the evaluation of ETFs’ performance. Another novelty introduced by this study is the selection of ETFs in pairs, following the same reference index, with one member of the pair domiciled in the US and the other in Europe (Graham et al. 2020).

This work is structured as follows: in the next section, a description of the DEA methodology used is given. In Section 2, the main premises regarding the data collected for assessing the performance of the funds under scrutiny are given. Section 3 delivers the main results obtained and their corresponding analyses. Finally, the main conclusions of this work and future developments are established.

2. Methodology

In this work, we use the non-oriented version of the DEA model proposed by Tone (2001), called Slack-Based Measure (SBM), which is a non-radial DEA model.

This model provides a more comprehensive efficiency assessment tool because, since it is non-radial and non-oriented, it considers that the inefficiency linked to the use of a specific input by a DMU is not necessarily related to the inefficiency regarding the use of another input by the same DMU; also, it contemplates the fact that a DMU may produce distinct outputs at the same time, but with a different production capacity, and hence the production efficiency for different outputs may also be distinct.

Consider the set of $n$ DMUs $\left({DMU}_1,{DMU}_2,\dots ,{DMU}_n\right)$, where X = [x_ij, i = 1, 2, …, m, j = 1, 2, …, n] is the (m × n) matrix of inputs, Y = [y_rj, r = 1, 2, …, s, j = 1, 2, …, n] is the matrix of outputs (s × n), and the rows of these matrices corresponding to the inputs and outputs of DMU_o are given by ${\mathit{x}}_o^T$ and ${\mathit{y}}_o^T$, respectively, with ^T denoting the transpose of a vector. The SBM model is given by (Tone 2001):

$$\begin{gathered} \begin{array}{c}Min\\ \mathsf{\lambda },{\mathit{s}}^{-},{\mathit{s}}^{+}\end{array}\mathsf{\rho }=\frac{1-\frac{1}{m}\sum _{i=1}^m{s}_i^{-}/x_{io}}{1+\frac{1}{s}\sum _{r=1}^s{s}_r^{+}/y_{ro}} \\ \mathrm{s}.\mathrm{t}. \\ {\mathit{x}}_o=X\mathsf{\lambda }+{\mathit{s}}_{}^{-}, \\ {\mathit{y}}_{\mathit{o}}=Y\mathsf{\lambda }-{\mathit{s}}_{}^{+}, \\ \mathsf{\lambda }\ge \mathbf{0},{\mathit{s}}_{}^{-}\ge \mathbf{0},{\mathit{s}}_{}^{+}\ge \mathbf{0}. \end{gathered}$$

(1)

In problem (1) it is assumed that the matrices of inputs and outputs have non-negative elements, i.e., X ≥ 0 and Y ≥ 0. Furthermore, it is also possible to conclude that an increase in each element of ${\mathit{s}}_{}^{-}$ or ${\mathit{s}}_{}^{+}$, ceteris paribus, will decrease the value of the objective function of the problem (1). Therefore, it can be concluded that 0 < ρ < 1.

The value of ρ in (1) can also be written as follows:

$$\rho =\left(\frac{1}{m}{\sum }_{i=1}^m\frac{x_{io}-s_i^{-}}{x_{io}}\right){\left(\frac{1}{s}{\sum }_{r=1}^s\frac{y_{ro}+s_r^{+}}{y_{ro}}\right)}^{-1}$$

(2)

In terms of interpretation, the ratio $\frac{x_{io}-s_i^{-}}{x_{io}}$ assesses the rate of reduction of input i and therefore, $\frac{1}{m}\sum _{i=1}^m\frac{x_{io}-s_i^{-}}{x_{io}}$ assesses the average percentage reduction of inputs. Similarly, the ratio $\frac{y_{ro}+s_r^{+}}{y_{ro}}$ evaluates the rate of increase of output r and $\frac{1}{s}\sum _{r=1}^s\frac{y_{ro}+s_r^{+}}{y_{ro}}$ is the average percentage increase in outputs. Therefore, ρ can be seen as the ratio of average inefficiencies of inputs and outputs. To consider variable returns to scale, it is only necessary to add the constraint ${\mathit{e}}^T\mathsf{\lambda }=1$ to model (1). So, problem (1) can be converted into problem (3), by using a positive scalar variable t:

$$\begin{gathered} \underset{t,\mathsf{\lambda },{\mathit{s}}^{-},{\mathit{s}}^{+}}{\mathit{Min}}\tau =t-\frac{1}{m}\sum _{i=1}^m{ts}_i^{-}/x_{io} \\ \mathrm{s}.\mathrm{t}. t+\frac{1}{s}{\sum }_{r=1}^{s}t{s}_r^{+}/y_{ro}=1, \\ {\mathit{x}}_o=X\mathsf{\lambda }+{\mathit{s}}_{}^{-}, \\ {\mathit{y}}_{\mathit{o}}=Y\mathsf{\lambda }-{\mathit{s}}_{}^{+}, \\ \mathsf{\lambda }\ge \mathbf{0},{\mathit{s}}_{}^{-}\ge \mathbf{0},{\mathit{s}}_{}^{+}\ge \mathbf{0},t>0. \end{gathered}$$

(3)

Then, problem (3) becomes:

$$\begin{gathered} \begin{array}{c}Min\\ t,\mathsf{\lambda },{\mathit{s}}^{-},{\mathit{s}}^{+}\end{array}\tau =t-\frac{1}{m}{\sum }_{i=1}^m{S}_i^{-}/x_{io} \\ \mathrm{s}.\mathrm{t}. t+\frac{1}{s}{\sum }_{r=1}^s{S}_r^{+}/y_{ro}=1, \\ {t\mathit{x}}_o=X\mathit{\Lambda }+{\mathit{S}}_{}^{-}, \\ {t\mathit{y}}_{\mathit{o}}=Y\mathit{\Lambda }-{\mathit{S}}_{}^{+}, \\ \mathit{\Lambda }\ge \mathbf{0},{\mathit{S}}_{}^{-}\ge \mathbf{0},{\mathit{S}}_{}^{+}\ge \mathbf{0},t>0 \end{gathered}$$

(4)

The optimal solution corresponds to:

$${\rho }^{*}={\tau }^{*},{\mathsf{\lambda }}^{*}={\mathit{\Lambda }}^{*}/t^{*},{\mathit{s}}_{}^{-*}={\mathit{S}}_{}^{-}/t^{*},{\mathit{s}}_{}^{+*}={\mathit{S}}_{}^{+}/t^{*}$$

Definition 1.

A DMU_o is SBM-Efficient if ${\rho }^{*}=1$. This condition is equivalent to ${\mathit{s}}_{}^{-*}$ = 0 and ${\mathit{s}}_{}^{+*}$ = 0.

Definition 2.

The set of efficient reference units for the SBM-inefficient DMU_o is obtained by considering the indices of the DMUs associated with ${\lambda }_j^{*}>0$.

Consider the reference set of the SBM-inefficient DMU_o as follows:

$$E_o=\{j:{\lambda }_j^{*}>0, j=1\dots ,n\}$$

The point of the efficient frontier that can be seen as a reference DMU for the SBM-inefficient DMU_o is:

$$({\widehat{\mathit{x}}}_o,{\widehat{\mathit{y}}}_o)=({\mathit{x}}_o-{\mathit{s}}_{}^{-*},{\mathit{y}}_O+{\mathit{s}}_{}^{+*})=({\sum }_{jϵE_o}^{}{\lambda }_j^{*}{\mathit{x}}_{\mathit{j}},{\sum }_{jϵE_o}^{}{\lambda }_j^{*}{\mathit{y}}_j)$$

(5)

One of the limitations of the original model proposed by Tone (2001) is that it does not allow the treatment of negative factors. To overcome this problem, Tone et al. (2019) have recently proposed the BP-SBM DEA model, which transforms negative factors into positive factors by considering “reference points”.

In this context, Sharp et al. (2007) proposed an SBM DEA model similar to the one suggested by Tone et al. (2019). The first model uses the scheme suggested by Portela et al. (2004) to accept negative data. Nevertheless, although the Sharp et al. (2007) model, unlike the BP-SBM DEA model, has the property of translation invariance, it can only be run under the VRS assumption in order to obtain an efficiency score within [0, 1]. Furthermore, the efficiency scores obtained by this model are very sensitive to the maximum output values and the minimum input values, whereas these extreme values have only small effects on the efficiency scores computed with the BP-SBM DEA model because only inputs and outputs with negative values are translated into positive values. Finally, the first DEA model cannot prevent division by zero in the objective function.

Definition 3.

The reference points are the minimum values of inputs and outputs.

The minimum value of the input $i$, $x_i^{min}\left(i=1,2,\dots ,m\right)$ is:

$${\delta }_i=min\left\{x_{i1},x_{i2},\dots ,x_{in}\right\}\left(i=1,2,\dots ,m\right)$$

(6)

The proposed transformation is carried out as follows:

$$\begin{gathered} \mathrm{If}{\delta }_i>0,\mathrm{then}{x}_i^{min}=0 \\ \mathrm{If}{\delta }_i=0,\mathrm{then}{x}_i^{min}=-{\sigma }_i \\ \mathrm{If}{\delta }_i<0,\mathrm{then}{x}_i^{min}={\delta }_i\left(1+{\tau }_i\right) \end{gathered}$$

(7)

Therefore, if ${\delta }_i>0$, it is not necessary to change input i since it is positive for all DMUs. If ${\delta }_i=0$, i.e., if one or more DMUs do not use this input, then a very small perturbation, ${\sigma }_i>0$, should be considered instead. If ${\delta }_i<0$, i.e., if one or more DMUs have negative input $i$, this input is converted into an amount large enough to make it strictly positive for all DMUs. To ensure that the use of this input is strictly positive, another term for disturbance, ${\delta }_i{\tau }_i$, is considered. It should be noted that both $\sigma$ and ${\tau }_i$ are positive numbers.

Similarly, the minimum value of output $r$, $y_r^{min}\left(r=1,2,\dots ,s\right)$, is obtained as follows:

$${\omega }_r=min\left\{y_{r1},y_{r2},\dots ,y_{rn}\right\}\left(r=1,2,\dots ,s\right)$$

and the proposed transformation is then:

$$\begin{gathered} \mathrm{If}{\omega }_r>0,\mathrm{then}{y}_r^{min}=0 \\ \mathrm{If}{\omega }_r=0,\mathrm{then}{y}_r^{min}=-{\rho }_i \\ \mathrm{If}{\omega }_r<0,\mathrm{then}{y}_r^{min}={\omega }_r\left(1+{\gamma }_i\right) \end{gathered}$$

(8)

Finally, the transformed factors correspond to:

$${\overline{x}}_{ij}=x_{ij}-x_i^{min}>0\left({\forall }_i,j\right)\mathrm{and}{\overline{y}}_{rj}=y_{rj}-y_r^{min}>0\left({\forall }_r,j\right)$$

(9)

3. Data and Assumptions

The sample of this study consists of 38 ETFs1 ordered in pairs, each pair following the same index, domiciled in the US (19 funds), and Europe (14 funds from Ireland and five funds from Luxembourg). The indicators collected for each ETF were Jensen’s Alpha, the Sharpe ratio, the trailing total return for evaluating their performance, and Beta and standard deviation2 to account for their risk. The time horizon of the study is between 2014 and 2018 and refers to one year (2018), three years (2016–2018), and five years (2014–2018).

Table A1 in the Appendix A presents the data for the DMUs evaluated, specifically their acronyms, names, countries of origin, benchmarks, release dates, and legal structure.

The regions chosen for this study were the US and Europe (Ireland and Luxembourg), because these represent the largest ETF world markets.

According to the Irish fund3, around 968 fund managers from more than 53 countries have assets managed in Ireland. Ireland offers managers access to the European Union (EU) market passport, the Undertaking for Collective Investment in Transferable Securities (UCITS), and Alternative Investment Funds (AIFs). Ireland leads in this fast-growing sector and is the center of excellence for supporting ETFs all over Europe. The ETFs domiciled in Ireland correspond to more than half of the European ETF market.

In addition, as claimed by the Association of the Luxembourg Funds Industry (ALFI)4, Luxembourg is the second-largest center of ETFs in Europe, with funds distributed across 70 countries.

Finally, the three largest US fund providers (which are BlackRock, Vanguard, and State Street) represent more than 70% of the global ETF market.

Inputs and outputs were selected according to the factors generally considered in the literature. Thus, we have selected as inputs the Beta and standard deviation, and as outputs, we have considered Jensen’s Alpha, the Sharpe ratio, and the total trailing return.

Beta is a market risk indicator that measures the sensitivity of an ETF to a portfolio of assets representing the market (benchmark) and is calculated as the covariance between the return of the ETF and the benchmark index, divided by the variance of the return of the benchmark index. A Beta greater than one means that the ETF is high-risk, as its profitability shows higher volatility than the market.

In turn, the standard deviation represents the total risk of a fund and measures the volatility of a fund over a period. In this case, a high standard deviation means that the fund is very volatile.

The Sharpe ratio measures the excess return of an ETF compared to a risk-free asset per unit of total risk. The higher the value of this indicator, the better the risk/return ratio of the ETF, i.e., the better the historical performance adjusted to the risk of the ETF.

Regarding Jensen’s Alpha, this indicator is a risk-adjusted measure that measures the excess return of an ETF above its expected return determined by the CAPM. The higher the value of this indicator, the higher the performance of the ETF.

Finally, the trailing total return indicates the return that could be obtained from an ETF in a specific period.

Table 1 presents the four models that will be used to measure the performance of investments in ETFs after adjusting for their risk. These models differ only in the factors to be considered as outputs.

Table 1. DEA models.

Models	Input	Output
DEA 1 (M1)	Beta and standard deviation	Jensen’s Alpha and Sharpe ratio
DEA 2 (M2)	Beta and standard deviation	Jensen’s Alpha and trailing total return
DEA 3 (M3)	Beta and standard deviation	Sharpe ratio and total trailing return
DEA 4 (M4)	Beta and standard deviation	Jensen’s Alpha, Sharpe ratio, and trailing total return

Table A2 and Table A3 in the Appendix A provide information on the risk and performance indicators, respectively, for each ETF considered in the analysis.

4. Results

In this section, we present and discuss the results obtained for the four models considered in the periods of one, three, and five years, respectively.

4.1. Results for the Complete Set of ETFs

The results for DEA Model 1 (M1) show that, in general, domiciled US ETFs are used more often as benchmarks (see Table A4 in the Appendix A). For one year, five efficient ETFs were obtained, three of which were domiciled in Ireland (CPXJ, CUKS, and IEMS) and two in the US (EWA, EWM) (see Table A4 in the Appendix A). The ETFs based in the US were more often used as a benchmark (EWA, 32 times; EWM, twice)—see Table A4 in the Appendix A. Among the efficient ETFs based in Ireland, the CPXJ served as a reference only once (see Table A4 in the Appendix A). No combination of ETFs that track the same index was efficient. For the period of three years, six efficient ETFs were obtained, four of which are based in the US (IVV, URTH, EWA, and EWM) and two of which are based in Ireland (CSPX and SSAC)—see Table A4 in the Appendix A. The ETFs URTH, SSAC, and EWA are used as references 31 times, six times, and three times, respectively (see Table A4 in the Appendix A). In this case, there are two ETFs (IVV and CSPX) that track the same index but do not serve as a benchmark. Finally, the 5-year period includes three efficient ETFs, two of which are domiciled in Ireland (CSPX and SSAC) and one in the US (SCJ) (see Table A4 in the Appendix A). The ETF SCJ was used as a benchmark 34 times (having a lower sensitivity to the market movement and a higher Jensen’s Alpha), followed by the ETFs CSPX (five times) and SSAC (two times)—see Table A4 in the Appendix A. Nevertheless, it should be noted that inefficient ETFs domiciled in the US show, in general, greater variability in terms of inefficiency than the corresponding pair in Europe (the scores of efficiency of EU ETFs ranged between 0.31 and 0.91, 0.16 and 0.77, and 0.36 and 0.85, for one-, three- and five-year periods, respectively, whereas those for US ETFs ranged between 0.19 and 0.73, 0.04 and 0.92, and 0 and 0.81, for one year, three years, and five years, respectively).

Table A5 in the Appendix A presents the analysis of the descriptive statistics related to the efficient and inefficient ETFs for M1 for the three periods under analysis. It is possible to verify that the average values of the risk factors associated with efficient ETFs are maintained as the time horizon increases, always being below the unit (the opposite situation occurs for inefficient ETFs). On the other hand, in a three-year time horizon, there is a very significant reduction in the average values of the outputs associated with these same funds.

From the analysis of Figure 1a–l, it can be seen that in M1, the efficiency scores of efficient ETFs do not seem to be related to their volatility with the market, regardless of the period of analysis considered.

Figure 1. Efficiency scores vs. inputs and projections (ETFs are ordered in decreasing order of efficiency) for the different time horizons–M1.

In the case of inefficient ETFs, there seems to be an evident correlation between efficiency levels and Beta. This result is particularly corroborated for one year (see the projections associated with this factor in Figure 1d).

Regarding the standard deviation, efficient ETFs present, in general, average values that are more or less uniform, varying approximately between 7 and 13. For inefficient ETFs, these values vary between 10 and 24, showing higher volatility (see Table A5 in the Appendix A). Then again, this situation has been particularly evident for one year (see projections associated with this factor in Figure 1j).

Concerning the model’s outputs depicted in Figure 2a–l (Jensen’s Alpha and trailing total return), efficiency is much more correlated to the profitability measures over five years (see Table A6 in the Appendix A), meaning that the investor exceeds his or her profitability expectations in relation to the value he or she previously estimated.

Figure 2. Efficiency scores vs. outputs and projections (ETFs are ordered in decreasing order of efficiency) for the different time horizons–M1.

Finally, it is possible to conclude that in order to compensate for the higher levels of risk recorded over the period of one year (see Figure 1d,j), the necessary adjustments to the levels of return are always greater for this time horizon (see the projections associated with Jensen’s Alpha in Figure 2d).

The results for DEA Model 2 (M2) also illustrate that ETFs domiciled in the US were more often used as a reference for best practices (see Table A7 in the Appendix A). Then again, at 1 year, the US ETF EWA is viewed as a benchmark 32 times, and at 3 years, the US ETF URTH is considered 31 times as a benchmark (see Table A7 in the Appendix A). At 5 years, the US ETF SCJ has been used 32 times as a benchmark (see Table A7 in the Appendix A). Inefficient European ETFs, on the other hand, show once again, in general, a lower variability of inefficiency than inefficient US ETFs (the scores of efficiency of EU ETFs ranged between 0.33 and 0.83, 0.27 and 0.78, and 0.36 and 0.84, for one-, three- and five-year periods, respectively, whereas those for US ETFs ranged between 0.19 and 0.76, 0.04 and 0.92, and 0 and 0.90, for one year, three years, and five years, respectively).

Table A8 in the Appendix A presents the analysis of the descriptive statistics relating to the efficient and inefficient ETFs of M2 for the three periods under analysis. From the analysis of Table A8 and Figure 3a–l and Figure 4a–l, it is possible to conclude that the results obtained do not differ much from those of the previous model. It should be noted that M2 uses the trailing total return as an alternative variable to the Sharpe ratio, with this variable being responsible for two more efficient ETFs during the five-year period than the previous model (see Table A7 in the Appendix A), suggesting that risk-free measures of performance lead to less conservative results.

Figure 3. Efficiency scores vs. inputs and projections (ETFs are ordered in decreasing order of efficiency) for the different time horizons–M2.

Figure 4. Efficiency scores vs. outputs and projections (ETFs are ordered in decreasing order of efficiency) for the different time horizons–M2.

Table A9, in the Appendix A, provides the efficiency scores attained by ETFs according to Model 3 (M3) for the periods of one, three and five years, respectively, considering beta and standard deviation as inputs and the Sharpe ratio and the trailing total return as outputs.

For the period of one year, there are two efficient ETFs based in the US (EWA and EWM) and three in Ireland (CPXJ, SAUS, and IEMS)—see Table A9 in the Appendix A. The ETF EWA was used as benchmark 32 times, while the EWM was used twice—see Table A9 in the Appendix A. Even though efficient European ETFs outnumbered US ETFs for the first time, the former were never used as benchmarks (see Table A9 in the Appendix A).

For the three years, there were four efficient ETFs based in the US (IVV, URTH, EWA, and EWM) and two in Ireland (CSPX and SSAC). The ETFs IVV and CSPX are highlighted here because they serve as references for best practices 18 and 14 times, respectively. This pair of ETFs achieved the highest output values, demonstrating the importance of the Sharpe ratio’s contribution to efficiency (see Table A3 in the Appendix A).

Finally, for the five-year period, two efficient ETFs based in the US (EEMA and SCJ), two in Ireland (CSPX and SSAC), and one in Luxembourg (XMJD) were obtained. In this case, we will see for the first time that an ETF based in Europe—the CSPX in Ireland—serves as a reference 32 times, having presented the highest value for the Sharpe ratio. When it comes to the US, the ETF SCJ stands out, serving as a reference ten times.

When contrasting the results obtained for DEA M3 with the previous models, it was observed that the European ETFs obtained higher levels of performance with the indicators used. For the period of five years, this latter model also differed from the others, with an ETF domiciled in Ireland becoming a benchmark. In regards to inefficient ETFs, the volatility of inefficiency is, generally, lower in European ETFs (the scores of efficiency of EU ETFs ranged between 0.4 and 0.75, 0.14 and 0.86, and 0.33 and 0.90, for one-, three- and five-year periods, respectively, whereas those for US ETFs ranged between 0.33 and 0.77, 0.19 and 0.91, and 0 and 0.88, for one year, three years, and five years, respectively). According to Table A10 in the Appendix A and Figure 5a–l and Figure 6a–l, the results suggest that the risk factors of inefficient ETFs practically show no need for adjustments at this level for the three- and five-year periods, in contrast to what happens for one year. In the case of using the Sharpe ratio as a risk-adjusted performance indicator, the opposite situation is verified. This result highlights the trade-off traditionally studied in the literature between risk and return (Henriques and Neves 2019). Finally, it is important to note that the choice of indicators can influence the efficiency levels of ETFs, suggesting that the investor or manager must take a leading role in the choice of the factors to be used.

Figure 5. Efficiency scores vs. inputs and projections (ETFs are ordered in decreasing order of efficiency) for the different time horizons–M3.

Figure 6. Efficiency scores vs. outputs and projections (ETFs are ordered in decreasing order of efficiency) for the different time horizons–M3.

Table A11, in the Appendix A, shows the efficiency scores obtained from Model 4 (M4) over one, three, and five years, with all factors considered simultaneously.

According to M4, there are four efficient ETFs based in Ireland (CPXJ, SAUS, CUKS, and IEMS) and two in the US (EWA and EWM) for the one-year period—see Table A11 in the Appendix A. However, only the US-based ETF EWA has been used as a reference 31 times (see Table A11 in the Appendix A), corroborating the results of Chu et al. (2010).

During the three-year period, four efficient ETFs from the US (IVV, URTH, EWA, and EWM) and two from Ireland (CSPX and SSAC) were obtained, with the existence of a pair of efficient ETFs (IVV and CSPX) never being considered as a reference (see Table A11 in the Appendix A). The ETF URTH stood out, serving as a reference 31 times. SSAC ranked first among European ETFs, serving as a reference six times.

Over a five-year period, there were two efficient ETFs based in the US (EEMA and SCJ), two in Ireland (CSPX and SSAC), and one in Luxembourg (KXMJD)—see Table A11 in the Appendix A. The SSAC and XMJD served as references twice, the CSPX served as references three times, and the SCJ served as references 32 times, demonstrating once more the best positioning of efficient US-based ETFs.

After analyzing Table A12 in the Appendix A and Figure 7a–l and Figure 8a–r, it is possible to conclude that, as in the previous model, the risk factors of ETFs are less susceptible to undergo adjustments to make these funds efficient for the periods of three and five years, registering the opposite situation for one year. Regarding the risk-adjusted performance factors, the trade-off between risk and return is again evident.

Figure 7. Efficiency scores vs. inputs and projections (ETFs are ordered in decreasing order of efficiency) for the different time horizons–M4.

Figure 8. Efficiency scores vs. outputs and projections (ETFs are ordered in decreasing order of efficiency) for the different time horizons–M4.

From the analysis of the different models for one year, the influence of inputs on the efficiency measure obtained far exceeds that of outputs. We discovered a more balanced situation over the three-year period, with inputs having a slightly greater influence than outputs on M1 and M2, except for the Sharpe ratio on M1, and outputs having a greater influence on M3 and M4, except for Jensen’s Alpha on M4. With the exception of Jensen’s Alpha, which has a lower influence than the standard deviation over the five years, the influence of outputs on the obtained measure of efficiency is much greater than that of inputs. In summary, short-term efficiency is more related to risk factors, and with the expansion of the time horizon, the efficiency of funds becomes increasingly related to return factors.

4.2. Results by Region

In addition to the study carried out for the different models regarding the periods considered, an analogous empirical study by region was also performed, with the sample being divided into US (19) and European (18) ETFs (Table 2).

Table 2. Summary table of efficient ETFs from the global sample and by region.

Models	Period	Input	Output	Efficient ETFs Global	US Efficient ETFs	EUR Efficient ETFs
M1	1 Year	Beta Standard deviation	Jensen’s Alpha Sharpe ratio	EWA, EWM, CPXJ, CUKS, IEMS	EWA, EWM	CPXJ, CUKS, IEMS, ISFR
	3 Years			IVV, URTH, EWA, EWM, CSPX, SSAC	IVV, URTH, EWA, EWM, EWJ	CSPX, CUKS, ISFR
	5 Years			SCJ, CSPX, SSAC	SCJ, IVV	CSPX, SSAC CUKS, ISFR
M2	1 Year	Beta Standard deviation	Jensen’s Alpha trailing total return	EWA, EWM, CPXJ, CUKS, and IEMS	EWA, EWM	CPXJ, CUKS, SAUS, ISFR, IEMS
	3 Years			IVV, URTH, EWA, EWM, CSPX, SSAC	IVV, URTH, EWA, EWM, EWJ	CSPX, SSAC CUKS, ISFR
	5 Years			EEMA, SCJ, CSPX, SSAC, XMJD	EEMA, SCJ, IVV	CSPX, SSAC, CUKS, ISFR, XMJD
M3	1 Year	Beta Standard deviation	Sharpe ratio, trailing total return	EWA, EWM, CPXJ, CUKS, IEMS	EWA, EWM	CPXJ, IEMS, SAUS, ISFR
	3 Years			IVV, URTH, EWA, EWM, CSPX, SSAC	IVV, URTH, EWA, EWM, EWJ	CSPX, SSAC CUKS, ISFR
	5 Years			EEMA, SCJ, CSPX, SSAC XMJD	EEMA, SCJ, IVV	CSPX, SSAC, XMJD, ISFR
M4	1 Year	Beta Standard deviation	Jensen’s Alpha Sharpe ratio Trailing total return	EWA, EWM, CPXJ,	EWA, EWM	CPXJ, SAUS, IEMS, CUKS, ISFR
	3 Years			IVV, URTH, EWA, EWM, CSPX, SSAC	IVV, URTH, EWA, EWM, EWJ	CSPX, SSAC CUKS, ISFR
	5 Years			EEMA, SCJ, CSPX, SSAC, XMJD	EEMA, SCJ, IVV	CSPX, SSAC, XMJD, ISFR, CUKS

Note: the efficient ETFs more often selected as benchmarks are highlighted in bold.

The results obtained corroborate the conclusions previously established regarding US ETFs.

Table A13 and Table A14 in the Appendix A show the descriptive statistics of the efficient and inefficient ETFs for each region for the periods of one, three, and five years for M1.

Regardless of the period considered, European inefficient ETFs always presented a lower magnitude of average inefficiency compared to US ETFs (the scores of efficiency of EU ETFs ranged between 0.52 and 0.98, 0.19 and 0.89, and 0.40 and 0.92, for one-, three- and five-year periods, respectively, whereas those for US ETFs ranged between 0.19 and 0.73, 0.04 and 0.92, and 0 and 0.81, for one year, three years, and five years, respectively)—see Table A13 and Table A14. On the other hand, efficient US ETFs presented, on average, less risk, while efficient European ETFs showed, on average, greater performance.

For the one-year period in the US, the risk indicators (Beta and standard deviation) had a significant correlation with efficiency in the four models (see Table A15 in the Appendix A). Furthermore, these results suggest that efficiency is more related to short-term risk factors. On the European side, the results obtained indicate a situation of greater balance concerning the factors that are correlated with the efficiency of this type of fund (see Table A15 in the Appendix A).

In turn, for the three years in the US sample, the results are similar to those obtained in the previous period. Concerning Europe, risk factors are less correlated with efficiency when compared to profitability factors (see Table A15 in the Appendix A).

Finally, for the five years in the US, risk indicators had a lower correlation with efficiency when compared to profitability indicators. In Europe, the standard deviation and the Sharpe ratio showed a higher correlation with efficiency for these funds (see Table A15 in the Appendix A).

These results suggest that the US ETFs follow the trend observed in the global sample, i.e., they show a greater correlation of efficiency with risk in the short term and a greater correlation with profitability as the time horizon expands. However, this trend has not been confirmed for European ETFs.

5. Conclusions

ETFs are the future of the fund industry, and several characteristics contribute to their popularity: transparency, ease of access, tax efficiency, and low cost, among others. Traditionally, their performance is assessed by risk-adjusted profitability indicators, namely the Sharpe ratio and Jensen’s Alpha. However, these measures have limitations, as they do not incorporate some of the factors seen as essential for assessing the performance of portfolios of assets: Jensen’s Alpha is sensitive to the choice of benchmark and market timing, and the Sharpe ratio can be manipulated by individuals to present their best side (i.e., in theory, if the three-year Sharpe ratio of a portfolio is not appealing, the fund manager could calculate the ratio for a 5-year period considering that the portfolio had a good performance in the past). On the other hand, the isolated consideration of these indicators does not allow the investor to take into account the multiple factors underlying the decision-making process. To overcome this issue, in this study, a non-parametric technique that considers multiple assessment factors was used. Traditionally, the literature on ETFs has fundamentally referred to the performance of the ETFs in comparison with their benchmark index or with the performance of other types of funds, mainly mutual funds (Ippolito 1989). In this study, some originality was sought, and the DEA methodology was used to evaluate the efficiency of ETFs domiciled in the US and Europe (Ireland and Luxembourg). These ETFs were selected in pairs, following the same reference index, with one member of the pair domiciled in the US and the other in Europe. The sample included 38 ETFs (19 in the US, 14 in Ireland, and five in Luxembourg), and the time horizon considered was from 2014 to 2018, with data for one year (2018), three years (2018–2016), and five years (2018–2014), respectively. Four models were used, whose inputs are the Beta and standard deviation and whose outputs are the Sharpe ratio, Jensen’s Alpha, and the total trailing return. First, the models for the complete sample were obtained, and then the sample was separated by regions. Efficient US ETFs were viewed more often as a reference for best practices than efficient European ETFs. For inefficient ETFs, those from Europe showed, in general, less variability in terms of inefficiency than those from the US. Overall, for one year, efficiency depended more on risk factors than on profitability, and as the time horizon increased, efficiency became more related to profitability factors. This trend was also seen in the sample of ETFs from the US. However, these findings were not corroborated in the case of Europe. The results also showed that efficiency levels are related to the risk-adjusted performance factors used, thus highlighting the need to reflect the preferences of investors/managers in the choice of the factors to be used in this assessment.

Similarly to other empirical studies (e.g., Tsolas and Charles 2015; Choi 1995; Choi and Min 2017), it was found that the DEA methodology complements traditional performance measures, providing investors with a broader perspective of ETFs’ performance since it allows the contemplation of several risk-adjusted-performance assessment factors simultaneously.

This work presents relevant theoretical and practical implications, which it is important to highlight. On the one hand, academics and researchers may find this article a good foundation for further work as we identified a gap in the literature when studying ETFS on a comparative basis between the US and Europe using an innovative methodology. On the other hand, both fund managers, potential investors, and civil society can understand what contributes to the performance and risk levels of this type of diversified funds, in different periods. At a time when interest rates are rising given the high inflation rates and weak returns on bank investments, this investment alternative can translate into an increasingly attractive reality for long term investors. While in the US this type of product is already truly used to diversify portfolios, in Europe it is a reality that is still recent but that is here to stay given the increasingly adverse macroeconomic environment in terms of banking dependence in these countries and the increasing weaknesses of long-term investments in banking.

As a future study, it would be interesting to analyze the efficiency of ETFs from different regions and see if their domiciliation has any impact on efficiency, using, for example, portfolio models and considering new periods, including bear markets such as the war in Ukraine.