A Comparative Study of Robust and Improved Shrinkage Estimators Under Multicollinearity and Outliers Using Multiple Performance Criteria with Application to Health Data

Abstract

Multicollinearity reduces the reliability of ordinary least squares by increasing variances and creating unstable estimates. This issue has led to biased and penalized regression methods like ridge-, Liu- and Stein-type estimators. Here, we build existing ridge-type approaches by introducing improved ridge and Liu-type estimators, along with robust variants to handle outliers. We investigate their theoretical properties regarding bias, variance, and mean squared error. We also evaluate their performance through Monte Carlo simulations with different levels of multicollinearity and data contamination. By using several evaluation criteria, including mean squared error, akaike information criterion, mean absolute deviation, and mean absolute percentage error, along with an average-rank comparison framework applied here for the first time, we further validate our results with two health-related datasets. The findings show that the strong estimators provide more stable estimates and improved predictive performance, particularly when dealing with severe multicollinearity and outliers.

1. Introduction

Multiple linear regression is a popular tool in statistical modeling. It helps examine the relationship between a continuous response variable and multiple explanatory variables. According to the classical Gauss–Markov assumptions, the ordinary least squares (OLS) estimator is the best linear unbiased option. However, these assumptions are often not met in practice. When explanatory variables are highly intercorrelated, the model experiences multicollinearity, a concept first identified by Frisch, 1934 [1]. This condition makes the design matrix X’X ill-conditioned, often shown by one or more eigenvalues being close to zero. This situation leads to unstable coefficient estimates and unreliable statistical inference.

The main goal of multiple regression analysis is to isolate the marginal effect of each explanatory variable while keeping the others constant. However, when regressors are strongly correlated, this interpretation becomes tricky because changes in one variable are linked to changes in another. As a result, multicollinearity fluctuates the variances of the OLS estimators, leads to inconsistent coefficient signs, weakens t-tests, and results in wide confidence intervals. This ultimately harms both inferential validity and predictive accuracy.

To address these issues, researchers have studied biased and penalized regression techniques in depth. Among these, ridge regression, introduced by Hoerl and Kennard, 1970 [2], remains a key approach. By adding a positive shrinkage parameter, ridge regression intentionally accepts a small amount of bias in exchange for a potential large reduction in variance.

This often results in a lower mean squared error (MSE) than ordinary least squares (OLS) when multicollinearity is severe. Since its introduction, much effort has focused on estimating the ridge parameter and developing improved ridge-type estimators for various modeling scenarios. (McDonald and Galarneau, 1975 [3]; Lawless and Wang, 1976 [4]); Shahzad et al., 2024 [5]; Akhtar and Alharthi, 2025 [6]; Alharthi and Akhtar, 2025 [7]; Alharthi and Akhtar, 2025 [8]).

An important alternative to ridge regression is the Liu estimator. This method uses a biasing parameter limited to the unit interval and offers a linear shrinkage form that is usually easier to interpret and adjust. Liu-type estimators can perform competitively under multicollinearity and have inspired several extensions and hybrid shrinkage strategies. (Liu, 1993 [9]; Lukman et al., 2020 [10]; Hoque and Kibria, 2023 [11]). Besides the ridge and Liu estimators, other shrinkage families, like the Kibria–Lukman-type and Stein-type estimators, have been proposed to boost estimation efficiency when regressors are highly correlated. (Kibria and Lukman, 2020 [12]; Shaheen et al., 2023 [13]; Abonazel, 2025 [14]; Alharthi and Akhtar, 2025 [7]; Akhtar and Alharthi, 2025 [6]; Alharthi and Akhtar, 2025 [8]).

Despite these advancements, real-world data often show not just multicollinearity but also contamination from outliers. Least squares-based estimators and many classical biased estimators can be very sensitive to these unusual observations, especially when outliers occur in the response direction. This sensitivity can severely distort estimation accuracy and invalidate conclusions based on the model. Thus, robustification has become an essential part of modern shrinkage estimation. This is typically done by combining shrinkage techniques with robust M-estimation procedures and robust scale estimators. (Huber, 1981 [15]; Silvapulle, 1991 [16]; Arslan and Billor, 2000 [17]) Recent studies highlight that robust shrinkage estimators can perform much better than classical ones when multicollinearity and contamination happen at the same time. (Wasim et al., 2023 [18]; Yasmin and Kibria, 2025 [19]).

Despite the large number of shrinkage estimators available in the literature, no single estimator performs optimally under all combinations of multicollinearity, sample size, and contamination levels. The performance of most ridge-type estimators depends critically on the choice of the shrinkage parameter. Furthermore, many existing studies focus primarily on mean squared error (MSE) and do not simultaneously evaluate estimator performance using multiple criteria such as AIC, MAD, and MAPE. These limitations motivate the development and investigation of improved and robust shrinkage estimators that may provide more stable estimation and prediction under practical data conditions involving both multicollinearity and outliers.

Rather than proposing entirely new optimization-based estimators, this study focuses on developing data-driven extensions of existing ridge-type estimators. The proposed estimators are constructed from well-established shrinkage parameters and their combinations, motivated by the observation that estimator performance depends critically on the choice of the shrinkage parameter. The objective is to improve robustness, stability, and predictive performance under multicollinearity and contamination while preserving the computational simplicity of existing shrinkage methods. Their performance is subsequently evaluated through theoretical comparisons, Monte Carlo simulations, and real-data applications.

The proposed forms were selected because they are derived from or closely related to existing shrinkage parameters that have demonstrated good performance in previous studies. Their theoretical properties are evaluated through matrix mean squared error comparisons, while their practical performance is assessed through extensive simulation experiments and real-data applications. The results indicate that several proposed estimators outperform existing competitors under specific combinations of multicollinearity, sample size, and contamination levels.

With these considerations in mind, this study aims to improve estimation stability and predictive accuracy in multicollinear regression models that face outliers. Specifically, this paper.

(i)

selects a set of competitive ridge-type estimators based on existing evidence,

(ii)

proposes new, improved ridge- and Liu-type estimators along with their robust versions,

(iii)

conducts a thorough theoretical and Monte Carlo evaluation of all estimators using multiple performance criteria, and

(iv)

shows the practical relevance of the proposed methods using two health-related datasets, where the robust estimators exhibit better stability and predictive performance under strong multicollinearity and contamination.

In regression and forecasting studies, multiple criteria are often used to assess and compare estimator performance. Traditionally, researchers report metrics like Mean Absolute Deviation (MAD), Mean Absolute Percentage Error (MAPE), and Akaike Information Criterion (AIC). Each of these captures different aspects of estimator performance. MSE emphasizes larger deviations due to squaring errors, MAD provides a robust measure that is less affected by outliers, MAPE expresses predictive accuracy in percentage terms, and AIC evaluates model quality by finding a balance between goodness-of-fit and model complexity. In many published papers on ridge, Liu, Kibria–Lukman, Stein, and other related shrinkage estimators, authors typically use one of these criteria to compare performance. However, to our knowledge, no one has considered all of them together. In this paper, we take a comprehensive approach by evaluating all estimators using all four criteria. For each criterion, we rank the estimators, with rank 1 given to the best-performing method. We then compute an average rank across all criteria to identify the estimator that consistently performs well under various evaluation conditions, such as sample size, number of regressors, and correlation among the regressors. We believe our approach provides a more generalized and robust selection framework. It accounts for both absolute and relative error measures as well as model complexity. This ensures that the chosen estimator is not favored by a single criterion but demonstrates overall superior performance across multiple dimensions. By integrating multiple evaluation criteria, our methodology addresses the limitations of relying on a single measure and enables a better comparison of shrinkage estimators in regression analysis.

The rest of the paper is organized as follows. In Section 2, we present the statistical materials and methods for linear regression model along with various shrinkage estimators commonly used in practice. Section 3 provides a theoretical comparison results of these estimators, highlighting their key properties and differences. In Section 4, we discuss a simulation study to evaluate and compare the performance of the estimators under different scenarios and also illustrate the application of the proposed methods using two real-life datasets. Finally, Section 5 concludes the paper with a summary of the main findings and some remarks on potential directions for future research.

2. Materials and Methods

Here, the standard multiple linear regression model is considered:

$$\mathit{y}=\mathit{X}\mathit{\beta }+\mathit{ϵ},$$

(1)

where $\mathit{y}={\left(y_1,y_2,\dots ,y_n\right)}^T$ is an $n\times 1$ vector of observations on the dependent variable, $\mathit{X}$ is an $n\times p$ matrix of known predictor variables of full column rank $p$, $\mathit{\beta }={\left({\beta }_1,{\beta }_2,\dots ,{\beta }_p\right)}^T$ is a $p\times 1$ vector of unknown regression coefficients, and $\mathit{ϵ}={\left({ϵ}_1,{ϵ}_2,\dots ,{ϵ}_n\right)}^T$ is an $n\times 1$ vector of random errors. The following standard assumptions are imposed:

$$E\left(\mathit{ϵ}\right)=0,Var\left(\mathit{ϵ}\right)={\sigma }^2{\mathit{I}}_n,$$

where ${\sigma }^2$ is a positive scalar and $I_n$ denotes the $n\times n$ identity matrix.

The OLS estimator of $\mathit{\beta }$ is given by,

$${\widehat{\mathit{\beta }}}_{ols}={\left({\mathit{X}}^{\mathbf{\prime }}\mathit{X}\right)}^{-1}{\mathit{X}}^{\prime }\mathit{y}.$$

(2)

Although the OLS estimator is unbiased, it becomes inefficient and unstable when strong multicollinearity exists among the explanatory variables, as ${\mathit{X}}^{\mathbf{\prime }}\mathit{X}$ becomes nearly singular and the variance of ${\widehat{\mathit{\beta }}}_{ols}$ becomes inflated. (Hoerl and Kennard, 1970 [2]; Farebrother, 1976 [20]).

To solve this multicollinearity problem, several biased shrinkage estimators have been developed by modifying the OLS estimator through the introduction of biasing parameters that reduce variance and improve overall MSE. (Hoerl et al., 1975 [21]; Naz et al., 2025 [22]; Alharty and Akhtar, 2025 [7]).

2.1. Ridge Regression Method

In case of the multicollinearity problem, Hoerl and Kennard (1970) [2] suggested a small positive number to be added to the diagonal elements of the matrix $\mathbf{X}\mathbf{\prime }\mathbf{X}$. In fact, the ridge regression was obtained by augmenting the equation $0=\mathit{\beta }{k}^{{\displaystyle \frac{1}{2}}}+{\mathit{ϵ}}^{\prime }$ to Equation (1) and the resulting estimator is obtained as,

$${\widehat{\mathit{\beta }}}_k={\left({\mathit{X}}^{\mathbf{\prime }}\mathit{X}+k{\mathit{I}}_p\right)}^{-1}{\mathit{X}}^{\mathbf{\prime }}\mathit{y}=\mathit{W}\widehat{\mathit{\beta }}, k\ge 0,$$

(3)

where $\mathit{W}=({\mathit{I}}_p+k{\mathit{C}}^{-1})$ and $\mathit{C}={\mathit{X}}^{\mathbf{\prime }}\mathit{X}$. This is known as the ridge regression estimator, where ${\mathbf{I}}_{\mathrm{p}}$ is an identity matrix of order $p$. Since the quantity [${\mathit{X}}^{\mathbf{\prime }}\mathit{X}+k{\mathit{I}}_p$] in (3) is always invertible, there always exists a unique solution for $\widehat{{\mathit{\beta }}_{\mathit{k}}}$. The ridge regression estimator is a biased estimator and for a positive value of $k$, it provides a smaller MSE compared to the OLS estimator (Hoerl and Kennard, 1970 [2]). From (3), we observed that as k → 0, $\widehat{\mathit{\beta }}(k)$ → $\widehat{\mathit{\beta }}$ and as $k \to \infty ,$ $\widehat{\mathit{\beta }}(k) \to 0.$

The bias, variance matrix and MSE expressions of $\widehat{{\mathit{\beta }}_{\mathit{k}}}$ are respectively given as follows,

$$Bias=E\left({\widehat{\mathit{\beta }}}_k\right)-\mathit{\beta }=-k{\mathit{C}}_k^{-1}\mathit{\beta }, {\mathit{C}}_k=\left[\mathit{C}+k{\mathit{I}}_p\right]$$

$$V\left(\widehat{\mathit{\beta }}\left(k\right)\right)={\sigma }^2\left(\mathit{W}{\mathit{C}}^{-1}{\mathit{W}}^{\mathbf{\prime }}\right)$$

$$MSE\left({\widehat{\mathit{\beta }}}_k\right)={\sigma }^{2}tr\left(\mathit{W}{\mathit{C}}^{-1}{\mathit{W}}^{\prime }\right)+k^2{\mathit{\beta }}^{\prime }{\mathit{C}}_k^{-2}\mathit{\beta }.$$

The parameter $k$ is known as a “biased” parameter and it must be estimated using real data. Most of the recent efforts in multicollinearity and ridge regression estimators have concentrated on estimating the value of k (Banik, 2016 [23]). We reviewed the statistical methodology used to describe the estimation of $k$ in the section that follows.

Estimation of Ridge Parameter K

Suppose there exists an orthogonal matrix $\mathit{D}$ such that ${\mathit{D}}^{\prime }\mathit{C}\mathit{D}$ = Λ.

Here, Λ $= \mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}({\mathsf{\lambda }}_1{\mathsf{\lambda }}_2,\dots ,{\mathsf{\lambda }}_{\mathrm{P}})$ contains the eigenvalues of the matrix ${\mathbf{X}}^{\mathbf{\prime }}\mathbf{X}$. The orthogonal version of (1) is:

$$\mathit{y}=\mathit{Z}\mathit{\alpha }+\mathit{ϵ},$$

(4)

where $\mathit{Z}=\mathit{X}\mathit{D}$ and $\mathit{\alpha }={\mathit{D}}^{\mathbf{\prime }}\mathit{B}$. Then the generalized ridge regression estimator is given as

$${\widehat{\mathit{\alpha }}}_k={\left({\mathit{Z}}^{\mathbf{\prime }}\mathit{Z}+\mathit{K}\right)}^{-1}{\mathit{Z}}^{\mathbf{\prime }}\mathit{y},\mathit{K}\ge 0,$$

(5)

where $\mathbf{K}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}({\mathrm{k}}_1,{\mathrm{k}}_2,\dots ,k_p),{\mathrm{k}}_i>0$ and $\widehat{\mathsf{\alpha }}$ = ${\mathsf{\Lambda }}^{-1}{\mathbf{X}}^{\mathsf{*}\mathbf{\prime }}\mathbf{y}$ are the OLS estimators of $\mathit{\alpha }$. It follows from Hoerl and Kennard (1970) [2] that ${\mathrm{k}}_{\mathrm{i}}$ minimizes MSE ($\widehat{\mathsf{\alpha }}(\mathrm{k})$), and is defined as

$$MSE(\widehat{\mathit{\alpha }}{(}_k))={\sigma }^2{\displaystyle \sum _{i=1}^p}{\displaystyle \frac{{\mathsf{\lambda }}_i}{{({\mathsf{\lambda }}_i+k_i)}^2}}+{\displaystyle \sum _{i=1}^p}{\displaystyle \frac{k_i^2{\alpha }_i^2}{{({\mathsf{\lambda }}_i+k_i)}^2}}$$

(6)

where ${\widehat{\mathsf{\sigma }}}^2={\displaystyle {\sum }_{\mathrm{i}=1}^{\mathrm{n}}}{\displaystyle \frac{{\widehat{\mathsf{ϵ}}}_{\mathbf{i}}^2}{\mathrm{n}-\mathrm{p}}}$, ${\widehat{\mathsf{ϵ}}}_{\mathbf{i}}=\mathbf{y}-{\mathbf{X}}^{*\mathbf{\prime }}\widehat{\mathsf{\alpha }}$, ${\lambda }_i$ are eigenvalues of the matrix ${\mathbf{X}}^{\mathbf{\prime }}\mathbf{X}$, ${\widehat{\mathrm{k}}}_{\mathrm{i}}={\displaystyle \frac{{\widehat{\mathsf{\sigma }}}^2}{{\widehat{\mathsf{\alpha }}}_{\mathrm{i}}^2}}$ and ${\widehat{\mathsf{\alpha }}}_{\mathrm{i}}$ is the ith element of $\widehat{\mathsf{\alpha }}$.

There are numerous methods available in the literature to estimate the ridge parameter $k$ and to modify existing biased estimators for improved performance. However, in this study, we follow the comprehensive comparison conducted by Mermi et al. (2024) [24], who evaluated 366 ridge-type estimators under various multicollinearity settings. Based on their findings, we selected the five most promising estimators in Table 1 from their study for further consideration and provided them below.

Table 1. Top 5 Existing best ridge parameter estimators by percentage based on the Mermi et al. (2024) [24].

Estimators	Citation
$k_1=\sigma ,\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e} {\sigma }^2={\displaystyle \frac{{\mathit{y}}^{\mathit{T}}\mathit{y}-{\widehat{\mathit{\alpha }}}_{ols}{\mathit{W}}^{\mathit{T}}\mathit{y}}{n-p-1}}$	Dorugade (2016) [25]
$k_2=p\widehat{\sigma }$	Lukman and Olatunji (2018) [26]
$k_3=\sqrt{p{\sum }_{j=1}^p{t}_j}$, here $t_j={\displaystyle \frac{{\lambda }_{max}{\widehat{\sigma }}^2}{\left(n-p\right){\widehat{\sigma }}^2+{\lambda }_j{\widehat{\alpha }}_{ols},j}}$	Mermi et al. (2024) [24]
$k_4={\left({\displaystyle \prod _{j=1}^p}{t}_j\right)}^{{\displaystyle \frac{1}{2p}}}$	Mermi et al. (2024) [24]
$k_5=\mathrm{m}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{n}({\mathrm{t}}_1,\dots .t_p)$	Mermi et al. (2024) [24]

In addition to these, we proposed several new ridge estimators and robust ridge estimators,

$$k_{new1}={\displaystyle \frac{{\mathit{y}}^{\mathit{T}}\mathit{y}-{\widehat{\mathit{\alpha }}}_{ols}{\mathit{W}}^T\mathit{y}}{n-p}},$$

$${\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}2}=\widehat{\mathsf{\sigma }}\left({\displaystyle \frac{\mathrm{n}}{\mathrm{p}}}\right),$$

$${\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}3}=\widehat{\mathsf{\sigma }}\sqrt{\left({\displaystyle \frac{\mathrm{n}}{\mathrm{p}}}\right)},$$

$$k_{new4}=\widehat{\sigma }{\lambda }_{harm, } \mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e} {\lambda }_{harm}={\displaystyle \frac{p}{{\sum }_{j=1}^p{\lambda }_j^{-1}}}.$$

2.2. Robust Ridge M-Estimator

When data contains outliers, extreme observations, or does not follow a normal distribution, the alternative to the least squares method is the robust M-estimator. Silvapulle (1991) [16] proposed a robust ridge M-estimator (RM), which is defined as

$${\widehat{\mathit{\alpha }}}_{RRM}={\left(\mathsf{\Lambda }+k\mathit{I}\right)}^{-1}\mathsf{\Lambda } {\widehat{\mathit{\alpha }}}_M,$$

(7)

where ${\widehat{\mathsf{\alpha }}}_M$ is the robust M-estimator defined by the solution of M-estimating equations ${\sum }_{i=1 }^{n }\psi ({\displaystyle \frac{u_i}{s}})=0$ and ${\sum }_{i=1 }^{n }\psi ({\displaystyle \frac{u_i}{s}}){\mathit{X}}_{\mathit{i}}=0$, where ${\mathit{u}}_{\mathit{i}}={\mathit{Y}}_{\mathit{i}}-{\mathit{X}}_i^{\prime }{\widehat{\mathsf{\alpha }}}_M$, $\psi$ is the first derivative of some suitable chosen function and $s$ is an estimator of scale for the errors also.

The bias, covariance and MSE and tr(MSE) expressions of RRM estimator are, respectively:

$$Bias\left({\widehat{\mathsf{\alpha }}}_{RRM}\right)=\left( {\left(\mathsf{\Lambda }+k\mathit{I}\right)}^{-1}\mathsf{\Lambda }-1\right)\mathsf{\alpha }$$

$$Cov\left({\widehat{\mathsf{\alpha }}}_{RRM}\right)=\mathsf{\Omega }({\left(\mathsf{\Lambda }+k\mathit{I}\right)}^{-1}{\mathsf{\Lambda }}^{\mathbf{\prime }}\mathsf{\Lambda } {\left(\mathsf{\Lambda }+k\mathit{I}\right)}^{-1}, where \mathsf{\Omega }=cov({\widehat{\mathsf{\alpha }}}_M)$$

$$MSE\left({\widehat{\mathsf{\alpha }}}_{RRM}\right)=\mathsf{\Omega }({\left(\mathsf{\Lambda }+k\mathit{I}\right)}^{-1}{\mathsf{\Lambda }}^{\mathbf{\prime }}\mathsf{\Lambda } {\left(\mathsf{\Lambda }+k\mathit{I}\right)}^{-1}+k{\left(\mathsf{\Lambda }+k\right)}^{-1}{\mathit{\alpha }}^{\mathbf{\prime }}\mathit{\alpha }{\left(\mathsf{\Lambda }+k\right)}^{-1}k$$

$$tr\left(MSE\left({\widehat{\mathit{\alpha }}}_{RRM}\right)\right)=\sum _{i=1}^p{\displaystyle \frac{{\lambda }_i^2}{{\left({\lambda }_i+k_i\right)}^2}}{\Omega }_{ii}+\sum _{i=1}^p{\displaystyle \frac{k_i^2{\alpha }_i^2}{{\left({\lambda }_i+k_i\right)}^2}}$$

(8)

Here, ${\mathsf{\Omega }}_{\mathrm{i}\mathrm{i}}$ is the diagonal elements of the matrix $\mathsf{\Omega }$.

2.3. Proposed Estimators

To enhance robustness against outliers, we propose four robust ridge-type estimators, each obtained by modifying the ridge parameter $k$ through alternative definitions of the robust scale $A^2$ derived from M-estimation theory (Huber, 1981 [15]; Silvapulle, 1991 [16]). The proposed estimators are defined as follows:

$${\mathrm{k}}_{1\mathrm{r}}=k_1\left({\displaystyle \frac{{\widehat{A}}^2}{{\widehat{\sigma }}^2}}\right)\mathrm{here} A^2={\displaystyle \frac{s^{2}E\left\{{\psi }^2(ϵ/s_0)\right\}}{{\left(E\left\{{\psi }^{\prime }(ϵ/s_0)\right\}\right)}^2}} \mathrm{with} s_0=plim\left(s\right) \mathrm{and} {\psi }^{\prime }(ϵ/s_0)={\displaystyle \frac{d}{dt}}{\psi }^{\prime }(t)$$

$${\mathrm{k}}_{2\mathrm{r}}=P{\mathrm{k}}_{1r}.$$

In addition to these, we proposed several estimators,

$$\begin{gathered} {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}1}=\widehat{\mathrm{A}}\surd P \\ {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}2}=\widehat{\mathrm{A}}\overline{\lambda }; \overline{\lambda }={\sum }_{j=1}^p{\lambda }_j \\ {k}_{newr3}={\widehat{A}\lambda }_{geo}; {\lambda }_{geo}=\exp \left({\displaystyle \frac{1}{p}}\sum _{j=1}^{p}log{\lambda }_j\right) \\ k_{min}=min(k_{new1}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}2}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}3},k_{new4},{\mathrm{k}}_{1\mathrm{r}},{\mathrm{k}}_{2\mathrm{r}},{\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}1},{\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}2},k_{newr3}) \\ {k}_{max}=max(k_{new1}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}2}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}3},k_{new4},{\mathrm{k}}_{1\mathrm{r}},{\mathrm{k}}_{2\mathrm{r}},{\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}1},{\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}2},k_{newr3}) \\ {k}_{mean}=mean(k_{new1}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}2}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}3},k_{new4},{\mathrm{k}}_{1\mathrm{r}},{\mathrm{k}}_{2\mathrm{r}},{\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}1},{\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}2},k_{newr3}) \\ k_{hm}=hm(k_{new1}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}3}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}4},k_{new9},{\mathrm{k}}_{1\mathrm{r}},{\mathrm{k}}_{2\mathrm{r}},{\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}1},{\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}2},k_{newr3}) \\ k_{new5}={\displaystyle \frac{y^{T}y-{\widehat{\alpha }}_{ols}{W}^{T}y}{n-p-1}}={\displaystyle \frac{\left(n-p\right)k1}{n-p-1}} \\ k_{GM}=GM(k_{new1}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}2}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}3},k_{new4},k_{new5},{\mathrm{k}}_{1\mathrm{r}},{\mathrm{k}}_{2\mathrm{r}},{\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}1},{\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}2},k_{newr3}) \mathrm{(Considering\; geometric\; mean)} \\ k_{GM2}=GM(k_{new1}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}2}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}3},k_{new4},k_{new5}) \mathrm{(Considering\; geometric\; mean)} \\ k_{med1}=median (k_{new1}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}2}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}3},k_{new4},k_{new5},{\mathrm{k}}_{1\mathrm{r}},{\mathrm{k}}_{2\mathrm{r}},{\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}1},{\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}\mathrm{r}2},k_{newr3}) \mathrm{(Considering\; median)} \\ k_{med2}=median (k_{new1}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}2}, {\mathrm{k}}_{\mathrm{n}\mathrm{e}\mathrm{w}3},k_{new4},k_{new5}) \mathrm{(Considering\; median)} \end{gathered}$$

2.4. Liu Estimator

To address the limitations of the ridge estimator, particularly its nonlinearity with respect to the ridge parameter $k$, Liu (1993) [9] proposed a new biased estimator, defined as

$${{\widehat{\mathsf{\alpha }}}_L=\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1}(\mathsf{\Lambda }+d\mathit{I}) \widehat{\mathit{\alpha }}$$

(9)

where $0<d<1$ is a shrinkage parameter. The advantage of ${\widehat{\alpha }}_L$ over ${\widehat{\alpha }}_k$ is that ${\widehat{\alpha }}_L$ is a linear function of d. Therefore, it is easier to choose d over to choose k in the ridge parameter.

The bias, covariance, MSE and tr (MSE) expressions of Liu estimator are respectively given below,

$$\begin{gathered} Bias\left({\widehat{\mathit{\alpha }}}_L\right)=-{\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1} \left(1-d\right) \mathsf{\alpha } \\ Cov\left({\widehat{\mathsf{\alpha }}}_L\right)={\sigma }^2{\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1}\left(\mathsf{\Lambda }+d\mathit{I}\right) {\mathsf{\Lambda }}^{-1}\left(\mathsf{\Lambda }+d\mathit{I}\right){\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1} \\ MSE\left({\widehat{\mathit{\alpha }}}_L\right)=Cov\left({\widehat{\mathit{\alpha }}}_L\right)+{\left(bias\left({\widehat{\mathit{\alpha }}}_L\right)\right)}^2 \\ ={\sigma }^2{\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1}\left(\mathsf{\Lambda }+d\mathit{I}\right) {\mathsf{\Lambda }}^{-1}\left(\mathsf{\Lambda }+d\mathit{I}\right){\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1}+{\left(d-1\right)}^2{\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1} \widehat{\mathit{\alpha }} {\widehat{\mathit{\alpha }}}^{\prime } {\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1} \end{gathered}$$

$$tr(MSE({\widehat{\mathit{\alpha }}}_L))= {\sigma }^2\sum _{i=1}^p{\displaystyle \frac{{\left({\lambda }_i+d\right)}^2}{{{\lambda }_i\left({\lambda }_i+1\right)}^2}}+\sum _{i=1}^p{\displaystyle \frac{{\left(d-1\right)}^2{\alpha }_i^2}{{\left({\lambda }_i+1\right)}^2}}$$

(10)

Now, differentiating Equation (6) with respect to $d$, and equating to $0$, we have

$$d_i={\displaystyle \frac{{\alpha }_i^2-{\sigma }^2}{{\sigma }^2+\frac{{\alpha }_i^2}{{\lambda }_i}}}$$

We use an unbiased estimator of ${\widehat{\sigma }}^2$ and ${\widehat{\alpha }}_i$ instead of ${\sigma }^2$ and ${\alpha }_i$ and we obtain an estimator for $d$,

$${\widehat{d}}_i={\displaystyle \frac{{\widehat{\alpha }}_i^2-{\widehat{\sigma }}^2}{{\widehat{\sigma }}^2+\frac{{\widehat{\alpha }}_i^2}{{\lambda }_i}}}$$

Since the above $d$ can give a negative value, we will consider the following d (Hoque and Kibria, 2024) [27].

$$\widehat{d}=min\left({\displaystyle \frac{{\widehat{\alpha }}_i^2}{\frac{1}{{\lambda }_i}+{\widehat{\alpha }}_i^2}}\right)$$

2.5. Modified One-Parameter Liu Estimator

Modified one-parameter Liu estimator is proposed by Lukman et al. (2020) [10] and defined as,

$${\widehat{\mathit{\alpha }}}_{ML}={\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1}\left(\mathsf{\Lambda }-d\mathit{I}\right)\widehat{\alpha }$$

(11)

The bias, covariance, MSE and tr(MSE) expressions of the modified one-parameter Liu Estimator are, respectively,

$$\begin{gathered} Bias\left({\widehat{\mathsf{\alpha }}}_{ML}\right)=-{\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1} \left(1+d\right)\mathsf{\alpha } \\ Cov\left({\widehat{\mathsf{\alpha }}}_{ML}\right)={\sigma }^2{\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1}\left(\mathsf{\Lambda }-d\mathit{I}\right) {\mathsf{\Lambda }}^{-1}\left(\mathsf{\Lambda }-d\mathit{I}\right){\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1} \\ MSE\left({\widehat{\alpha }}_{ML}\right)=Cov\left({\widehat{\alpha }}_{ML}\right)+{\left(bias\left({\widehat{\alpha }}_{ML}\right)\right)}^2 \\ ={\sigma }^2{\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1}\left(\mathsf{\Lambda }-d\mathit{I}\right) {\mathsf{\Lambda }}^{-1}\left(\mathsf{\Lambda }-d\mathit{I}\right){\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1}+{\left(1-d\right)}^2{\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1} \widehat{\mathit{\alpha }} {\widehat{\mathit{\alpha }}}^{\mathbf{\prime }}{\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1} \end{gathered}$$

To obtain the value of d, we will consider $\mathit{D}=diag(d,d_2,\dots ,d_p)$. Then the $tr(MSE\left({\widehat{\mathsf{\alpha }}}_{ML}\right))$ becomes

$$tr(MSE({\widehat{\mathit{\alpha }}}_{ML}))= {\sigma }^2\sum _{i=1}^p{\displaystyle \frac{{\left({\lambda }_i-di\right)}^2}{{{\lambda }_i\left({\lambda }_i+1\right)}^2}}+\sum _{i=1}^p{\displaystyle \frac{(1+di){\alpha }_i^2}{{\left({\lambda }_i+1\right)}^2}}$$

(12)

The partial derivative of $tr(MSE\left({\widehat{\mathsf{\alpha }}}_{ML}\right))$ with respect to $d_i$ and setting it to zero, we obtained

$$\widehat{d_i}={\displaystyle \frac{{\lambda }_i(\sigma {\mathsf{\Lambda }}^2+{\alpha }_i^2)}{\sigma {\mathsf{\Lambda }}^2+{\lambda }_i\widehat{{\alpha }_i^2}}},$$

where we replace ${\sigma }^2$ with its unbiased estimate. In our study, we will consider the following d (Hoque and Kibria, 2024 [23]).

$$\widehat{d}=min\left({\displaystyle \frac{{\widehat{\alpha }}_i^2}{\frac{1}{{\lambda }_i}+{\widehat{\alpha }}_i^2}}\right).$$

2.6. Almost Unbiased Liu Estimator (AULE)

An almost unbiased Liu estimator (Akendiz and Erol, 2003 [28]) for the linear regression model, can be written as

$${\widehat{\mathit{\alpha }}}_{AULE}=\left(\mathit{I}-{\left(1-d\right)}^2{\left(\mathit{\Lambda }+\mathit{I}\right)}^{-2}\right)\widehat{\mathit{\alpha }}$$

(13)

The bias vector, covariance matrix, MMSE and scalar MSE are given respectively:

$$Bias \left({\widehat{\mathit{\alpha }}}_{AULE}\right)=-{\left(1-d\right)}^2{\mathit{\Lambda }}_I^{-2}\mathit{\alpha }$$

$$Cov\left({\widehat{\mathit{\alpha }}}_{AULE}\right)={\sigma }^2\left(\mathit{I}-{\left(1-d\right)}^2{\mathit{\Lambda }}_I^{-2}\right){\mathit{\Lambda }}^{-1}\left(\mathit{I}-{\left(1-d\right)}^2{\mathit{\Lambda }}_I^{-2}\right)$$

$$\begin{gathered} MMSE \left({\widehat{\mathit{\alpha }}}_{AULE}\right)={\sigma }^2\left(\mathit{I}-{\left(1-d\right)}^2{\mathit{\Lambda }}_I^{-2}\right){\mathit{\Lambda }}^{-1}\left(\mathit{I}-{\left(1-d\right)}^2{\mathit{\Lambda }}_I^{-2}\right) \\ +{\left(1-d\right)}^4{\mathit{\Lambda }}_I^{-2}\mathit{\alpha }{\mathit{\alpha }}^{\mathbf{\prime }}{\mathit{\Lambda }}_I^{-2} \end{gathered}$$

$$MSE\left({\widehat{\mathit{\alpha }}}_{AULE}\right)={\sum }_{j=1}^p{\sigma }^2{\displaystyle \frac{{\left({\lambda }_j+d\right)}^2{\left(2+{\lambda }_j-d\right)}^2}{{\lambda }_j{\left({\lambda }_j+1\right)}^4}}+{\sum }_{j=1}^p{\displaystyle \frac{{\left(1-d\right)}^4{\alpha }_j^2}{{\left({\lambda }_j+1\right)}^4}},$$

where ${\mathsf{\Lambda }}_I=\left(\mathsf{\Lambda }+\mathit{I}\right).$

Following Hoque and Kibria (2024) [27], two candidate estimators for the biasing parameter $d$ can be defined as

$${\widehat{d}}_1=\min \left({\displaystyle \frac{{\alpha }_i^2}{\frac{1}{{\lambda }_j}+{\alpha }_j^2}}\right),$$

$${\widehat{d}}_2=\min \left({\displaystyle \frac{{\lambda }_j\left(1+{\alpha }_j^2\right)}{1+{\alpha }_j^2{\lambda }_j}}\right).$$

2.7. Kibria–Lukman Estimator

Kibria and Lukman (2020) [12] introduced an estimator (KL) which is defined as:

$${\widehat{\mathit{\alpha }}}_{KL}={\mathit{W}}_K{\mathit{M}}_k{\widehat{\mathit{\alpha }}}_{ols},$$

(14)

where ${\mathit{W}}_k={\left(\mathsf{\Lambda }+\mathit{K}\right)}^{-1}$, and ${\mathit{M}}_k=\left(\mathsf{\Lambda }-\mathit{K}\right)$ and $\mathit{K}=diag(k_1,k_2,\dots ,k_p)$.

The bias, covariance, MSE and tr (MSE) expressions of KL estimator are respectively,

$$\begin{gathered} Bias\left({\widehat{\mathit{\alpha }}}_{KL}\right)=\left({\mathit{W}}_K{\mathit{M}}_K-I\right)\mathsf{\alpha } \\ Cov\left({\widehat{\mathsf{\alpha }}}_{KL}\right)={\sigma }^2{\mathit{W}}_k{\mathit{M}}_k{\mathsf{\Lambda }}^{-1}{\mathit{M}}_k^{\prime }{\mathit{W}}_k^{\prime } \\ MSE\left({\widehat{\mathsf{\alpha }}}_{KL}\right)={\sigma }^2{\mathit{W}}_k{\mathit{M}}_k{\mathsf{\Lambda }}^{-1}{\mathit{M}}_k^{\prime }{\mathit{W}}_k^{\prime }+\left({\mathit{W}}_K{\mathit{M}}_K-\mathit{I}\right)\widehat{\mathit{\alpha }}{\widehat{\mathit{\alpha }}}^{\mathbf{\prime }}{\left({\mathit{W}}_K{\mathit{M}}_K-\mathit{I}\right)}^{\prime } \end{gathered}$$

$$tr\left(MSE\left({\widehat{\mathit{\alpha }}}_{KL}\right)\right)={\sigma }^2\sum _{i=1}^p{\displaystyle \frac{{\left({\lambda }_i-ki\right)}^2}{{{\lambda }_i\left({\lambda }_i+k_i\right)}^2}}+\sum _{i=1}^p{\displaystyle \frac{4k_i^2{\alpha }_i^2}{{\left({\lambda }_i+k_i\right)}^2}}$$

(15)

Now differentiating with respect to $k_i$, and equating to $0$, we have

$$k_i={\displaystyle \frac{{\sigma }^2}{\left(2{\alpha }_i^2+\frac{{\sigma }^2}{{\lambda }_i}\right)}}$$

The optimal value of $k$ depends on the unknown parameter ${\widehat{\sigma }}^2$ and ${\widehat{\alpha }}^2$. These two estimators are replaced with their unbiased estimate. Consequently, we have

$${\widehat{k}}_{\mathrm{i}}={\displaystyle \frac{{\widehat{\sigma }}^2}{\left(2{\widehat{\alpha }}_i^2+\frac{{\widehat{\sigma }}^2}{{\lambda }_i}\right)}}.$$

We will consider the following $k$ proposed by Kibria and Lukman (2020) [12]:

$${\widehat{\mathrm{k}}}_1=\min ({\displaystyle \frac{{\widehat{\sigma }}^2}{\left(2{\widehat{\alpha }}_i^2+\frac{{\widehat{\sigma }}^2}{{\lambda }_i}\right)}}),$$

and

$${\widehat{\mathrm{k}}}_2={\displaystyle \frac{p{\widehat{\sigma }}^2}{ {\sum }_i^p\left(2{\widehat{\alpha }}_i^2+\frac{{\widehat{\sigma }}^2}{{\lambda }_i}\right)}}.$$

2.8. James–Stein Estimator

James–Stein (1956) [29] estimator (JSE) is defined as

$${\widehat{\mathsf{\alpha }}}_{JSE}=c{\widehat{\mathsf{\alpha }}}_{ols}, (0<c<1)$$

(16)

The bias, covariance, MSE and tr (MSE) expressions of the JSE estimator are given respectively,

$$\begin{gathered} Bias\left({\widehat{\mathsf{\alpha }}}_{JSE}\right)=\left(c-1\right)\mathsf{\alpha } \mathrm{and} Cov\left({\widehat{\mathsf{\alpha }}}_{JSE}\right)=c^2{\sigma }^2{\mathsf{\Lambda }}^{-1} \\ MSE\left({\widehat{\mathsf{\alpha }}}_{JSE}\right)=c^2{\sigma }^2{\mathsf{\Lambda }}^{-1}+{\left(c-1\right)}^2\mathit{\alpha }{\mathit{\alpha }}^{\mathbf{\prime }} \end{gathered}$$

$$tr\left(MSE\left({\widehat{\mathit{\alpha }}}_{JSE}\right)\right)={\sigma }^2\sum _{i=1}^p{\displaystyle \frac{c^2}{{\lambda }_i}}+\sum _{i=1}^p{\left(c-1\right)}^2{\alpha }_i^2$$

(17)

We differentiate $tr\left(MSE\left({\widehat{\mathsf{\alpha }}}_{JSE}\right)\right)$ with respect to $c$ and set the derivative equal to 0, we obtain the value of c as,

$$c={\displaystyle \frac{\left({\mathit{\alpha }}^{\mathbf{\prime }}\mathit{\alpha }\right)}{({\mathit{\alpha }}^{\mathbf{\prime }}\mathit{\alpha }+{\sigma }^{2}trace{\left(\mathsf{\Lambda }\right)}^{-1})}}$$

2.9. Robust James–Stein Estimator

Robust version of James–Stein (1956) [29] estimator (JSE) is defined as

$${\widehat{\mathsf{\alpha }}}_{RJSE}=c{\widehat{\mathsf{\alpha }}}_M,$$

(18)

where $0<c<1.$ The bias, covariance and MSE and tr (MSE) of estimator are, respectively,

$$\begin{gathered} Bias\left({\widehat{\mathsf{\alpha }}}_{RJSE}\right)=\left(c-1\right){\mathsf{\alpha }}_M \\ Cov\left({\widehat{\mathsf{\alpha }}}_{RJSE}\right)=\mathsf{\Omega }{c}^2{\mathsf{\Lambda }}^{-1} \\ MSE\left({\widehat{\mathsf{\alpha }}}_{RJSE}\right)={\mathsf{\Omega }c}^2{\mathsf{\Lambda }}^{-1}+{\left(c-1\right)}^2{\mathsf{\alpha }}_M{\mathsf{\alpha }}_M^{\prime } \end{gathered}$$

$$tr\left(MSE\left({\widehat{\mathit{\alpha }}}_{RJSE}\right)\right)=\sum _{i=1}^p{\displaystyle \frac{c^2}{{\lambda }_i}}{\Omega }_{ii}+\sum _{i=1}^p{\left(c-1\right)}^2{\alpha }_M^2$$

(19)

Now, differentiating $tr\left(MSE\left({\widehat{\mathsf{\alpha }}}_{RJSE}\right)\right)$ with respect to $c$ and equating to zero, we obtain

$$c={\displaystyle \frac{{\mathit{\alpha }}_M^{\prime }{\mathit{\alpha }}_M}{({\mathsf{\alpha }}_M^{\prime }{\mathsf{\alpha }}_M+A^{2}trace{\left(\mathsf{\Lambda }\right)}^{-1})}}.$$

2.10. Least Absolute Shrinkage and Selection Operator (Lasso)

The Least Absolute Shrinkage and Selection Operator (LASSO), proposed by Tibshirani (1996) [30], is a regularization method that simultaneously performs parameter estimation and variable selection. It improves model interpretability and predictive performance by shrinking regression coefficients toward zero through a penalty imposed on their absolute values. As a result, coefficients linked to less important predictors may drop to zero. This creates a sparse model that includes only the most relevant variables.

Because of this property, LASSO has been widely used in situations where multicollinearity exists or when there are many predictors (Knight and Fu, 2000 [31]; Lounici, 2008 [32]; Yuan and Lin, 2007 [33]). The LASSO estimator is found by minimizing the residual sum of squares and adding a constraint on the $L_1$ norm of the regression coefficients. Unlike ridge regression, which uses an $L_2$ norm penalty, LASSO substitutes the squared penalty with an absolute value penalty. This change allows some coefficients to be exactly zero, effectively selecting variables automatically while estimating the regression parameters.

$${\widehat{\mathsf{\beta }}}_{Lasso}=\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}{\left(\mathbf{y}-\mathbf{X}\mathsf{\beta }\right)}^{\mathrm{T}}(\mathit{y}-\mathit{X}\mathit{\beta })+\lambda \parallel \mathit{\beta } {\parallel }_1$$

(20)

2.11. The Elastic Net Estimator

The Elastic Net (EN), introduced by Zou and Hastie (2005) [34], expands the LASSO method by including both $L_1$ and $L_2$ penalty terms in the regression model. This combination allows the method to maintain LASSO’s variable selection ability while also benefiting from the stabilization that ridge regression provides when predictors are highly correlated. In the Elastic Net framework, we estimate the regression coefficients by minimizing a penalized least squares function that includes both absolute value and squared penalties. The $L_1$ component promotes sparsity by reducing some coefficients to zero, while the $L_2$ component helps manage multicollinearity by spreading shrinkage across correlated predictors, as noted by Park and Konishi (2016) [35]. Consequently, the Elastic Net often outperforms LASSO in cases where groups of predictors are closely related. Previous studies have shown the effectiveness of the Elastic Net in identifying key variables in complex data contexts, including high-dimensional cases, as highlighted by Liu and Li, 2017 [36]. While LASSO may choose only one variable from a group of correlated predictors, the Elastic Net tends to select the entire group, as noted by Emmert-Streib and Dehmer (2019) [37]. We obtain the estimator by minimizing a penalized loss function that accounts for both penalty terms, leading to the corresponding coefficient estimates (Nayem et al., 2024) [38].

$${\widehat{\mathsf{\beta }}}_{EN}=argmin\left\{\sum _{i=1}^n{\left({\mathit{y}}_i-\sum _i^p{\mathit{\beta }}_i{\mathit{X}}_{\mathit{l}\mathit{i}}\right)}^2+\lambda [(1-\alpha )\sum _{j=1}^p{\left|{\mathit{\beta }}_{\mathit{j}}\right|}^2+\alpha \sum _{j=1}^p\left|{\mathit{\beta }}_j\right| ]\right\}$$

(21)

where $\lambda$ is the tuning parameter and α is the weight that should be used to control the contribution to Lasso and ridge penalties such that $0 \le \alpha \le 1$; where α = 0, ${\widehat{\beta }}_{EN}$ reduces to ${\widehat{\mathit{\beta }}}_{Ridge}$ and where $\alpha = 1$, ${\widehat{\mathsf{\beta }}}_{EN}$ becomes equivalent to ${\widehat{\mathsf{\beta }}}_{Lasso}$. The MSE for Lasso and EN was computed by comparing the estimated coefficient vector of each model with the true coefficient, assumed as $\mathit{\beta }={\mathit{I}}_p$, and then averaging the results over all simulations to evaluate the model performance.

3. Results

However, many estimators can be compared; for brevity, this paper focuses on selected estimators. Similar comparisons can be extended to the remaining estimators. The following notations and lemmas are introduced to facilitate the comparison of these estimators.

Lemma 1:

Let $n\times n$ matrices $\mathit{M}>0$, $\mathit{N}>0$ (or $\mathit{N}\ge 0$), then $\mathit{M}>\mathit{N}$ if and only if ${\lambda }_i\left(\mathit{N}{\mathit{M}}^{-1}\right)<1$, where ${\lambda }_i\left(\mathit{N}{\mathit{M}}^{-1}\right)$ is the largest eigenvalue of matrix $\mathit{N}{\mathit{M}}^{-1}$.

Lemma 2 (Farebrother, 1976

[15]): Let $\mathit{M}$ be an $n\times n$ positive definite matrix, let $\mathit{\alpha }$ be a non-zero $n\times 1$ column matrix and let $c$ be a positive scalar. Then $\mathrm{c}\mathbf{M}-\mathsf{\alpha }{\mathsf{\alpha }}^{\mathbf{\prime }}>0$ iff ${\mathsf{\alpha }}^{\mathbf{\prime }}{\mathbf{M}}^{-1}\mathsf{\alpha }<\mathrm{d}$ and $\mathrm{c}\mathbf{M}-\mathsf{\alpha }{\mathsf{\alpha }}^{\mathbf{\prime }}\ge 0$ iff ${\mathsf{\alpha }}^{\mathbf{\prime }}{\mathbf{M}}^{-1}\mathsf{\alpha }\le \mathrm{d}.$

Lemma 3 (Trenkler and Toutenburg, 1990

[39]): Let ${\widehat{\mathsf{\alpha }}}_{\mathrm{i}}={\mathbf{A}}_{\mathrm{i}}\mathbf{y}, \mathrm{i}=1,2$ be two linear estimators of $\mathsf{\alpha }$. Suppose that $\mathbf{D}=\mathrm{C}\mathrm{o}\mathrm{v}({\widehat{\mathsf{\alpha }}}_1)-\mathrm{C}\mathrm{o}\mathrm{v}\left({\widehat{\mathsf{\alpha }}}_2\right)>0,$ where $\mathrm{C}\mathrm{o}\mathrm{v}\left({\widehat{\mathsf{\alpha }}}_{\mathrm{i}}\right), \mathrm{i}=1,2$ denotes the covariance matrix of ${\widehat{\mathsf{\alpha }}}_i$ and ${\mathbf{b}}_{\mathrm{i}}=\mathrm{B}\mathrm{i}\mathrm{a}\mathrm{s}\left({\widehat{\mathsf{\alpha }}}_{\mathrm{i}}\right)=({\mathbf{A}}_{\mathbf{i}}\mathbf{X}-\mathbf{I})\mathsf{\alpha }, \mathrm{i}=1,2.$ Consequently,

$$∆\left({\widehat{\mathsf{\alpha }}}_1-{\widehat{\mathsf{\alpha }}}_2\right)=\mathrm{M}\mathrm{S}\mathrm{E}\mathrm{M}\left({\widehat{\mathsf{\alpha }}}_1\right)-\mathrm{M}\mathrm{S}\mathrm{E}\mathrm{M}\left({\widehat{\mathsf{\alpha }}}_2\right)={\mathsf{\sigma }}^2\mathbf{D}+{{\mathbf{b}}_1\mathbf{b}}_1^{\mathbf{\prime }}-{{\mathbf{b}}_2\mathbf{b}}_2^{\mathbf{\prime }}>0,$$

if and only if ${{\mathit{b}}^{\mathbf{\prime }}}_2{\left[{\sigma }^2\mathit{D}+{\mathit{b}}_1{{\mathit{b}}^{\mathbf{\prime }}}_1\right]}^{-1}{\mathit{b}}_2<1$, where MSEM(${\widehat{\mathit{\alpha }}}_i$) = Cov (${\widehat{\mathit{\alpha }}}_i)$ + ${\mathit{b}}_i{\mathit{b}\mathbf{\prime }}_i$.

3.1. Comparison Between ${\widehat{\mathit{\alpha }}}_{\mathit{R}\mathit{i}\mathit{d}\mathit{g}\mathit{e}}$ and ${\widehat{\mathit{\alpha }}}_{\mathit{o}\mathit{l}\mathit{s}}$

$$\begin{gathered} MSEM\left({\widehat{\mathsf{\alpha }}}_{ols}\right)={\sigma }^2{\mathsf{\Lambda }}^{-1} \\ MSEM\left({\widehat{\mathsf{\alpha }}}_{Ridge}\right)={\sigma }^2{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}+k^2{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}{\mathit{\alpha }{\mathit{\alpha }}^{\mathbf{\prime }} \left(\mathsf{\Lambda }+k{\mathit{I}}_{\mathit{p}}\right)}^{-1} \\ ={\sigma }^2{\left(\mathsf{\Lambda }+k{\mathit{I}}_{\mathit{p}}\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k{\mathit{I}}_{\mathit{p}}\right)}^{-1}+{\mathit{b}}_1{\mathit{b}}_1^{\prime } \end{gathered}$$

here, ${\mathit{b}}_1=k{\left(\mathsf{\Lambda }+k\mathit{I}\right)}^{-1}\mathsf{\alpha }$ and ${\mathit{b}}_1^{\prime }=k{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}{\mathsf{\alpha }}^{\prime }$

So, the difference between $MSEM\left({\widehat{\mathsf{\alpha }}}_{ols}\right)$ and $MSEM\left({\widehat{\mathsf{\alpha }}}_{Ridge}\right)$ is obtained by

$$\begin{gathered} MSEM\left({\widehat{\mathsf{\alpha }}}_{ols}\right)-MSEM\left({\widehat{\mathsf{\alpha }}}_{Ridge}\right)={\sigma }^2{\mathsf{\Lambda }}^{-1}-{\sigma }^2{\left(\mathsf{\Lambda }+k\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k\right)}^{-1}-{\mathit{b}}_1{\mathit{b}}_1^{\prime } \\ ={\sigma }^2\left({\mathsf{\Lambda }}^{-1}-{\left(\mathsf{\Lambda }+k\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k\right)}^{-1}\right)-{\mathit{b}}_1{\mathit{b}}_1^{\prime } \\ ={\mathit{D}}_1-{\mathit{b}}_1{\mathit{b}}_1^{\prime }, \mathrm{here}, {\mathit{D}}_1={\sigma }^2\left({\mathsf{\Lambda }}^{-1}-{\left(\mathsf{\Lambda }+k\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k\right)}^{-1}\right) \end{gathered}$$

We have the following theorem for $k>0$.

Theorem 1:

If $k>0$, ${\mathit{b}}_1=Bias\left({\widehat{\mathsf{\alpha }}}_{RE}\right)$ is the bias of RE, the necessary and sufficient conditions for ${\widehat{\mathsf{\alpha }}}_{RE}\left(k\right)$ to be superior to ${\widehat{\mathsf{\alpha }}}_{ols}$ using the criterion of $MSEM\left({\widehat{\mathsf{\alpha }}}_{ols}\right)-MSEM\left({\widehat{\mathsf{\alpha }}}_{Ridge}\right)>0$ if and only if,

$$\begin{gathered} {\mathit{b}}_1^{\prime }\left[{\sigma }^2\left({\mathsf{\Lambda }}^{-1}-{\left(\mathsf{\Lambda }+k\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k\right)}^{-1}\right)\right]{\mathit{b}}_1\le 1 \mathrm{that} \mathrm{means} {\mathit{b}}_1^{\prime }{\mathbf{D}}_1{\mathit{b}}_1\le 1 \\ \mathrm{Here}, {{\mathit{D}}_1=\sigma }^2\left({\mathsf{\Lambda }}^{-1}-{\left(\mathsf{\Lambda }+k\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k\right)}^{-1}\right)={\sigma }^{2}diag{\left\{{\displaystyle \frac{1}{{\lambda }_i}}-{\displaystyle \frac{{\lambda }_i}{{\left({\lambda }_i+k\right)}^2}}\right\}}_{i=1 }^p \end{gathered}$$

where $\left({\mathsf{\Lambda }}^{-1}-\left({\left(\mathsf{\Lambda }+k\right)}^{-1}\right)\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k\right)}^{-1}\right)$ is positive definite if and only if ${\left({\lambda }_i+k\right)}^2-{\lambda }_i^2>0.$

For $k>0$, we observed that $2{\lambda }_i+k>0$. Therefore, $\left({\mathsf{\Lambda }}^{-1}-\left({\left(\mathsf{\Lambda }+k\right)}^{-1}\right)\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k\right)}^{-1}\right)$ is positive definite, and thereby, by Lemma 2, Theorem 1 is proved.

3.2. Comparison Between Modified Ridge (${\widehat{\mathit{\alpha }}}_{\mathit{M}\mathit{R}}$) and ${\widehat{\mathit{\alpha }}}_{\mathit{R}\mathit{i}\mathit{d}\mathit{g}\mathit{e}}$

$$\begin{gathered} MSEM({\widehat{\widehat{\mathit{\alpha }}}}_{Ridge})={\sigma }^2{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}+{\mathit{b}}_1{\mathit{b}}_1^{\prime } \\ MSEM\left({\widehat{\mathit{\alpha }}}_{MR}\right)={\sigma }^2{\mathit{R}}_k{\mathsf{\Lambda }}^{-1}{\mathit{R}}_k^{\prime }+\left({\mathit{R}}_k-\mathit{I}\right)\mathit{\alpha }{\mathit{\alpha }}^{\mathbf{\prime }}\left({\mathit{R}}_k-\mathit{I}\right)={\sigma }^2{\mathit{R}}_k{\mathsf{\Lambda }}^{-1}{\mathit{R}}_{\mathit{k}}^{\mathbf{\prime }}+{\mathit{b}}_2{\mathit{b}}_2^{\prime } \\ \mathrm{Here} {\mathit{b}}_2={(\mathit{R}}_k-\mathit{I})\mathit{\alpha } \end{gathered}$$

So, the difference between $MSEM\left({\widehat{\mathit{\alpha }}}_{MR}\right)$ and $MSEM\left({\widehat{\mathit{\alpha }}}_{Ridge}\right)$ is obtained by

$$\begin{gathered} MSEM\left({\widehat{\mathit{\alpha }}}_{Ridge}\right)-MSEM\left({\widehat{\mathit{\alpha }}}_{MR}\right){=[\sigma }^2\{{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}-{\mathit{R}}_k{\mathsf{\Lambda }}^{-1}{\mathit{R}}_k^{\prime }\}+{\mathit{b}}_1{\mathit{b}}_1^{\prime }]-{\mathit{b}}_2{\mathit{b}}_2^{\prime } \\ ={[\mathit{D}}_2+{\mathit{b}}_1{\mathit{b}}_1^{\prime }]-{\mathit{b}}_2{\mathit{b}}_2^{\prime }, \end{gathered}$$

here, ${{\mathit{D}}_2=\sigma }^2\{{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}-{\mathit{R}}_k{\mathsf{\Lambda }}^{-1}{\mathit{R}}_k^{\prime }\}$.

Theorem 2:

For $k>0$ and $0<d<1$, ${\mathit{b}}_2=Bias\left({\widehat{\mathit{\alpha }}}_{MR}\right)$ is the bias of MR, the necessary and sufficient conditions for ${\widehat{\mathit{\alpha }}}_{MR}\left(k\right)$ to be superior to ${\widehat{\mathit{\alpha }}}_{Ridge}$ using the criterion of $MSEM\left({\widehat{\mathit{\alpha }}}_{Ridge}\right)-MSEM\left({\widehat{\mathit{\alpha }}}_{MR}\right)>0$ if and only if,

$${\mathit{b}}_2^{\prime }\left[{\sigma }^2\left\{{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}-{\mathit{R}}_k{\mathsf{\Lambda }}^{-1}{\mathit{R}}_k^{\prime }\right\}+{\mathit{b}}_1{\mathit{b}}_1^{\prime }\right]{\mathit{b}}_2\le 1$$

That means

$${\mathit{b}}_2^{\prime }[{\mathit{D}}_2+{\mathit{b}}_1{\mathit{b}}_1^{\prime }]{\mathit{b}}_2\le 1$$

Here, ${{\mathit{D}}_2=\sigma }^2\left\{{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}-{\mathit{R}}_k{\mathsf{\Lambda }}^{-1}{\mathit{R}}_k^{\prime }\right\}={\sigma }^{2}diag{\left\{{\displaystyle \frac{{\lambda }_i}{{\left({\lambda }_i+k\right)}^2}}-{\displaystyle \frac{{\lambda }_i}{{\left({\lambda }_i+k\left(1+d\right)\right)}^2}}\right\}}_{i=1 }^p$.

Evidently, for $0<d<1$ and $k>0$, ${\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}-{\mathit{R}}_k{\mathsf{\Lambda }}^{-1}{\mathit{R}}_k^{\prime }$ will be positive definite if and only if ${\left({\lambda }_i+k\left(1+d\right)\right)}^2-{\left({\lambda }_i+k\right)}^2>0$. We observed that $kd>0$, so ${\left(\mathsf{\Lambda }+k{I}_p\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}-{\mathit{R}}_k{\mathsf{\Lambda }}^{-1}{\mathit{R}}_k^{\prime }$ is positive definite. By Lemma 2, Theorem 2 is proved.

3.3. Comparison Between Liu Estimator and Modified Ridge (${\widehat{\mathit{\alpha }}}_{\mathit{M}\mathit{R}}$)

$$\begin{gathered} MSEM\left({\widehat{\mathit{\alpha }}}_{MR}\right)={\sigma }^2{\mathit{R}}_k{\mathsf{\Lambda }}^{-1}{\mathbf{R}}_k^{\prime }+\left({\mathit{R}}_k-\mathit{I}\right)\mathit{\alpha }{\mathit{\alpha }}^{\mathbf{\prime }}\left({\mathit{R}}_k-\mathit{I}\right)={\sigma }^2{\mathit{R}}_k{\mathsf{\Lambda }}^{-1}{\mathit{R}}_k^{\prime }+{\mathit{b}}_3{\mathit{b}}_3^{\prime } \\ MSEM\left({\widehat{\mathit{\alpha }}}_{LIU}\right)={\sigma }^2{\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1}\left(\mathsf{\Lambda }+d\mathit{I}\right) {\mathsf{\Lambda }}^{-1}\left(\mathsf{\Lambda }+d\mathit{I}\right){\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1}+{\left(1-d\right)}^2{\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1} \widehat{\mathit{\alpha }}{\widehat{\mathit{\alpha }}}^{\mathbf{\prime }} {\left(\mathsf{\Lambda }+\mathit{I}\right)}^{-1} \end{gathered}$$

Theorem 3:

For $k>0$ and $0<d<1$, ${\mathit{b}}_3=Bias\left({\widehat{\mathit{\alpha }}}_{LIU}\right)$ is the bias of Liu, the necessary and sufficient conditions for ${\widehat{\mathit{\alpha }}}_{LIU}\left(k\right)$ to be superior to ${\widehat{\mathit{\alpha }}}_{MR}$ using the criterion of $MSEM\left({\widehat{\mathit{\alpha }}}_{MR}\right)-MSEM\left({\widehat{\mathit{\alpha }}}_{LIU}\right)>0$ if and only if,

$${\mathit{b}}_3^{\prime }\left[{\sigma }^2\left\{{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}\mathsf{\Lambda }{\left(\mathsf{\Lambda }+k{\mathit{I}}_p\right)}^{-1}-{\mathit{R}}_k{\mathsf{\Lambda }}^{-1}{\mathit{R}}_k^{\prime }\right\}+{\mathit{b}}_1{\mathit{b}}_1^{\prime }\right]{\mathit{b}}_3\le 1$$

That means

$${\mathit{b}}_3^{\prime }[{\mathit{D}}_3+{\mathit{b}}_1{\mathit{b}}_1^{\prime }]{\mathit{b}}_3\le 1$$

4. Discussion

The purpose of the simulation experiment is to evaluate how well the proposed estimators and their robust counterparts perform under various multicollinearity and contamination scenarios. Analytical comparisons are generally not feasible for this class of estimators, so we rely on a controlled Monte Carlo framework implemented in the R environment.

4.1. Simulation Design

The structure of the simulation is guided by the factors known to influence biased estimators under multicollinearity. Following the approaches used by McDonald and Galarneau (1975) [3], the predictor variables are generated using the Equicorrelation scheme:

$$x_{ij}=\sqrt{\left(1-{\lambda }^2\right)}{w}_{ij}+\lambda w_{i,p+1}; i=1,2,\dots ,p$$

where $w_{ij}$ are independent standard normal variates. The constant $\lambda$ is chosen so that the correlation between any two regressors equals ${\lambda }^2$. To explore various levels of multicollinearity, we use four values:

$$\lambda =0.8,\hspace{0.25em} 0.9,\hspace{0.25em} 0.95,\hspace{0.25em} 0.99$$

The response variable is generated from the model,

$${\mathrm{Y}}_{\mathrm{i}}={\beta }_0+{\beta }_1{x}_{i1}+{\beta }_2{x}_{i2}+\cdots +{\beta }_p{x}_{ip}+{ϵ}_i;i=1,2,\dots ,n,$$

where ${ϵ}_i$ are independently and identically distributed as $N \left(0,{\sigma }^2\right).$ For simplicity, the intercept is set to zero.

Following McDonald and Galarneau (1975) [3], the vector of regression coefficient $\beta$ is taken to be the normalized eigenvector associated with the largest eigenvalue of $x^{\prime }x$, ensuring that the signal is aligned with the dominant direction of multicollinearity.

To evaluate different dimensional settings, the number of predictors is set to $p=3 \mathrm{a}\mathrm{n}\mathrm{d} p=10,$ and the error variance is considered at $\sigma =2$ and $\sigma =5$.

The sample sizes used in the study are:

$$n=30, 100, 200.$$

Following the methodology of McDonald and Galarneau (1975) [3], we considered these parameter configurations ($n=30, 100, 200$), ($p=3, 10$) and related settings.

4.2. Outliers Generation and Replication

To assess estimator robustness, contamination is introduced into the error term. Under each scenario, a proportion τ = 0%, 10%, and 25% of the observations is randomly selected, and their errors are replaced by:

$$Q_3\left(\epsilon \right)+5\times IQR\left(\epsilon \right),$$

which produces pronounced outliers and heavy-tailed behavior.

Each simulation configuration is repeated $N=5000$ times. All computations are carried out using R-studio version 4.5.1, and robust M-estimation is performed using the rlm {} function with the Huber (1981) [10] objective function. In each Monte Carlo replication, the LASSO and Elastic Net estimators are obtained using their penalized least squares formulations described in Section 2.10 and Section 2.11.

4.3. Performance Criterion

We have considered various methods, namely, MSE, AIC, MAD and MAPE, to compare the performance of the estimators, and we explained their necessity in Section 1.

4.3.1. Mean Squared Error (MSE)

To compare the estimators, we use the mean squared error (MSE). For any estimator $\widehat{\mathit{\beta }}$ of $\mathit{\beta }$, the MSE is given by

$$MSE\left(\widehat{\mathit{\beta }}\right)=E\left(\widehat{\mathit{\beta }}-\mathit{\beta }\right){\left(\widehat{\mathit{\beta }}-\mathit{\beta }\right)}^{\prime }$$

However, for evaluation purposes, the estimated MSE(EMSE) for any estimator $\widehat{\mathit{\beta }}$ of $\mathit{\beta }$ can be computed as

$$EMSE\left(\widehat{\mathit{\beta }}\right)={\displaystyle \frac{{\sum }_{i=1}^R({\widehat{\mathit{\beta }}}_i-\mathit{\beta }{)}^{\prime }({\widehat{\mathit{\beta }}}_i-\mathit{\beta })}{R}}$$

where $R$ is the number of repetitions in a simulation and ${\widehat{\mathit{\beta }}}_i$ is the estimated value of $\widehat{\mathit{\beta }}$ in the $i^{th}$ repetition.

For the simulation study, MSE is calculated using the true parameter vector $\beta$, which is known by construction. However, in the real-data applications, the true regression coefficients are unknown. Therefore, MSE is computed based on prediction error and is given by

$$MSE=(1/n) \Sigma (y_i-{\hat{y}}_i{)}^2,$$

where $y_i$ and ${\hat{y}}_i$ denote the observed and fitted response values, respectively. This clarification has been added to the revised manuscript.

4.3.2. Akaike Information Criterion (AIC)

For a fitted model with residual sum of squares RSS and effective degrees of freedom (df), the AIC is defined as:

$$\mathrm{A}\mathrm{I}\mathrm{C}=\mathrm{n} \mathrm{l}\mathrm{n}\left({\displaystyle \frac{RSS}{n}}\right)+2df,$$

4.3.3. Mean Absolute Deviation (MAD)

To quantify the average absolute discrepancy between estimator $\widehat{\mathit{\beta }}$ and the true parameter vector $\mathit{\beta }$, we compute the MAD as:

$$MAD\left(\widehat{\mathit{\beta }}\right)={\displaystyle \frac{1}{p}}\left|{\widehat{\mathit{\beta }}}_j-{\mathit{\beta }}_j\right|$$

4.3.4. Mean Absolute Percentage Error (MAPE)

We also compute the MAPE, which measures the average absolute percentage deviation and is defined as:

$$MAPE\left(\widehat{\mathit{\beta }}\right)={\displaystyle \frac{100}{p}}\left|{\displaystyle \frac{{\widehat{\mathit{\beta }}}_j-{\mathit{\beta }}_j}{{\mathit{\beta }}_j}}\right|$$

Smaller values of MSE, MAD, MAPE, and AIC indicate better estimator performance. Since each criterion captures a different aspect of an estimator’s behavior, we compute the average MSE, average MAD, average MAPE, and average AIC across all simulation conditions to obtain a stable and comprehensive assessment.

4.4. Rank Summary Criterion

To provide an overall comparison among estimators, we employ a rank summary approach based on the four performance measures: average MSE, average MAD, average MAPE, and average AIC.

For each criterion, estimators are ranked separately:

• Rank 1 is assigned to the estimator with the smallest average value of the criterion.
• Higher ranks indicate inferior performance.

Let

• $r_j^{\left(MSE\right)}$ denote the ranks of estimator $j$ based on average MSE.
• $r_j^{\left(AIC\right)}$ denote the ranks of estimator $j$ based on average AIC.
• $r_j^{\left(MAD\right)}$ denote the ranks of estimator $j$ based on average MAD.
• $r_j^{\left(MAPE\right)}$ denote the ranks of estimator $j$ based on average MAPE.

To determine the most robust estimator among the four competing procedures, we assessed their performance using multiple evaluation criteria, namely MSE, MAD, AIC, and MAPE. Because no single criterion was given priority over the others, equal weights were assigned to all measures, and the final ranking was based on their average performance across these criteria. The Rank Score for estimator $j$ is defined as the average of its individual ranks:

$$R{S}_j={\displaystyle \frac{r_j^{\left(MSE\right)}+r_j^{\left(AIC\right)}+r_j^{\left(MAD\right)}+r_j^{\left(MAD\right)}}{4}}$$

Finally, estimators are ordered according to their average ranks, and the estimator with the smallest average rank is assigned the final rank 1, indicating the best overall performance.

4.5. Simulation Results Discussion

Estimators are ranked based on their average ranks. The one with the smallest average rank receives the rank 1, representing the best overall performance. We evaluate the performance of competing estimators using four criteria: MSE, AIC, MAD, and MAPE. To ensure a clear and thorough comparison, we present one detailed scenario, while summarizing others with rank-based comparisons. The final ranking results from averaging individual ranks across all four criteria, which allows for a fair evaluation of estimator performance amid different levels of multicollinearity and contamination.

4.5.1. Performance Under No Outliers ($\mathit{\tau }=0\%$)

Estimators are ranked based on their average ranks. The one with the smallest average rank receives the rank 1, representing the best overall performance. To ensure a clear and thorough comparison, we present one detailed scenario, while summarizing others with rank-based comparisons. The final ranking results from averaging individual ranks across all four criteria, which allows for a fair evaluation of estimator performance amid different levels of multicollinearity and contamination. Without outliers, estimators that specifically tackle multicollinearity perform significantly better than the traditional OLS estimator across all sample sizes and correlation levels. Rank summary results for both $p = 3$ and $p = 10$ show that the proposed geometric mean-based estimators, especially $k_{GM}$ and $k_{GM2}$, consistently earn top rankings. These estimators display steady performance across all evaluation criteria, proving effective in reducing multicollinearity without excessive bias.

Several improved ridge-type estimators, including $k_1$, $k_{2r}$, and ${ k}_{new2 }$, also perform competitively, especially under moderate to high correlations. Median-based estimators like $k_{mean}$, $k_{med1}$, and $k_{med2}$ excel under error-related measures (MSE, MAD, and MAPE), though their weaker AIC rankings have a slight impact on their overall rank. In contrast, classical estimators such as OLS-, Liu-, and AULE-type consistently rank the lowest, confirming their sensitivity to severe multicollinearity.

4.5.2. Performance Under Moderate Contamination ($\mathit{\tau }=10\%$)

When 10% contamination is present, the benefits of robust and median-based estimators become evident. In both $p=3$ and $p=10,$ rank summaries indicate that $K_{GM2}$, $K_{2r}$, and $K_{new2}$ often hold the top overall positions, demonstrating strong robustness while maintaining efficiency across all criteria. For lower predictor dimensions, estimators like $k_{mean}$, $k_{med1}$, and $k_{med2}$ achieve favorable rankings under MSE, MAD, and MAPE, but their AIC performance remains comparatively weaker. In higher-dimensional cases ($p=10)$, robust ridge-type estimators, particularly $k_{2r}$, lead the rankings, showing their ability to balance robustness and shrinkage effectively.

Classical estimators and non-robust shrinkage methods face a significant decline in performance with contamination. Penalized estimators such as LASSO and elastic net show moderate performance but typically fall short compared to the proposed robust estimators.

4.5.3. Performance Under Severe Contamination $(\mathit{\tau }=25\%)$

During severe contamination, the advantage of robust and median-based estimators is even more pronounced. For $p=3$ and $\sigma =2,$ $k_{GM2}$ regularly achieve the highest overall ranking, followed closely by $k_{med2}$ and $k_{mean}$, showing strong resilience to extreme outliers. As the error variance increases, median-based estimators like $k_{med2}$, $k_1$, and ${ k}_{new1}$ stand out as the most dependable options. In the higher-dimensional scenario $(p=10),$ robust ridge-type estimators dominate.

For $\sigma =2$, $k_{2r}$ secures the top overall ranking, while for $\sigma =5$, the proposed estimator ${ K}_{new5}$ performs best, closely followed by $k_1$ and ${ k}_{new1}$. These results confirm that the proposed estimators retain their effectiveness even amid combined multicollinearity and contamination challenges.

In every scenario with $\tau = 25\%,$ classical estimators, including OLS, Liu, and AULE variants, consistently earn the lowest ranks, indicating significant performance declines. Penalized methods also struggle to provide adequate robustness under heavy contamination.

4.5.4. Overall Remarks

Overall, the simulation results in Table 2 clearly show that the proposed estimators offer significant improvements compared to existing classical, biased, and penalized estimators across various settings. In the absence of outliers, $k_{GM}$ and $k_{GM2}$ emerge as the most efficient estimators. Under moderate and severe contamination, robust ridge-type and median-based estimators, particularly $k_{GM2}$, $k_{2r}$, and $k_{new5}$ consistently achieve top rankings. These findings highlight the robustness, stability, and practical utility of the proposed estimators for regression issues involving multicollinearity and data prone to outliers.

Table 2. (1) Estimated MSE values when τ = 0%, $n = 30, 100, 200$ σ = 2, and p = 3. (2) Estimated AIC values when τ = 0%, n = 30, 100, 200 σ = 2, and p = 3. (3) Estimated MAD values when τ = 0%, n = 30, 100, 200 σ = 2, and p = 3. (4) Estimated MAPE values when τ = 0%, n = 30, 100, 200 σ = 2, and p = 3.

				(1)
n	30	30	30	30	100	100	100	100
$\rho$	0.80	0.90	0.95	0.99	0.80	0.90	0.95	0.99
OLS	1.602	3.135	6.094	31.008	0.435	0.830	1.635	8.092
k1	0.558	0.567	0.467	0.199	0.302	0.422	0.500	0.332
k2	0.165	0.137	0.115	0.093	0.107	0.091	0.068	0.032
k3	1.159	1.686	2.052	1.531	0.414	0.749	1.333	3.403
k4	1.426	2.478	3.898	6.297	0.428	0.801	1.523	5.791
k5	1.426	2.476	3.893	6.273	0.428	0.801	1.523	5.789
k1r	0.625	0.681	0.608	0.288	0.305	0.429	0.518	0.365
k2r	0.266	0.223	0.168	0.087	0.181	0.188	0.162	0.068
knew1	0.558	0.567	0.467	0.199	0.302	0.422	0.500	0.332
knew2	0.151	0.122	0.100	0.078	0.075	0.062	0.055	0.046
knew3	0.372	0.326	0.241	0.104	0.180	0.185	0.157	0.064
knew4	0.175	0.260	0.450	2.096	0.072	0.074	0.119	0.528
knew5	0.543	0.545	0.445	0.189	0.301	0.420	0.496	0.327
knewr1	0.640	0.684	0.589	0.253	0.317	0.458	0.569	0.419
knewr2	0.466	0.446	0.437	0.428	0.482	0.460	0.451	0.442
knewr3	0.150	0.139	0.149	0.236	0.091	0.058	0.048	0.056
kmin	0.678	0.752	0.684	2.098	0.318	0.461	0.576	0.538
kmax	0.466	0.446	0.437	0.428	0.482	0.460	0.451	0.442
kmean	0.148	0.122	0.103	0.080	0.086	0.068	0.059	0.048
khm	0.372	0.349	0.295	0.239	0.227	0.268	0.268	0.176
kGM	0.244	0.221	0.182	0.108	0.127	0.127	0.114	0.065
kGM2	0.307	0.299	0.260	0.162	0.152	0.167	0.163	0.108
kmed1	0.301	0.300	0.302	0.176	0.180	0.185	0.159	0.126
kmed2	0.372	0.328	0.404	0.189	0.180	0.185	0.157	0.327
Liu	1.374	2.378	3.979	14.162	0.415	0.759	1.388	4.910
LiuMod	0.907	1.067	0.835	3.422	0.372	0.610	0.889	0.598
AULE_d1	1.574	2.975	5.420	21.586	0.434	0.825	1.605	6.916
AULE_d2	1.602	3.135	6.091	25.606	0.435	0.830	1.635	8.079
KLE	0.805	1.539	2.909	14.807	0.213	0.393	0.765	3.764
JSE	0.980	1.670	2.882	12.958	0.344	0.577	0.980	3.617
RJSE	1.084	1.884	3.333	15.078	0.364	0.608	1.050	3.950
Lasso	0.991	1.565	2.491	10.188	0.373	0.610	0.985	2.841
EN	0.805	1.298	2.186	10.519	0.290	0.440	0.731	2.613
n	200		200		200	200	AVG
$\rho$	0.80		0.90		0.95	0.99
k1	0.178		0.287		0.415	0.460	0.391
k2	0.089		0.092		0.078	0.033	0.092
k3	0.211		0.398		0.759	2.917	1.384
k4	0.213		0.408		0.796	3.643	2.308
k5	0.213		0.408		0.796	3.643	2.306
k1r	0.178		0.288		0.418	0.476	0.432
k2r	0.130		0.166		0.176	0.095	0.159
knew1	0.178		0.287		0.415	0.460	0.391
knew2	0.058		0.049		0.045	0.040	0.073
knew3	0.111		0.129		0.122	0.055	0.170
knew4	0.051		0.041		0.060	0.267	0.349
knew5	0.178		0.287		0.413	0.457	0.383
knewr1	0.182		0.301		0.450	0.555	0.451
knewr2	0.483		0.462		0.453	0.443	0.454
knewr3	0.080		0.045		0.030	0.029	0.093
kmin	0.183		0.301		0.451	0.558	0.633
kmax	0.483		0.462		0.453	0.443	0.454
kmean	0.071		0.057		0.050	0.042	0.078
khm	0.149		0.210		0.253	0.204	0.251
kGM	0.084		0.094		0.092	0.058	0.126
kGM2	0.097		0.117		0.126	0.095	0.171
kmed1	0.120		0.145		0.144	0.146	0.190
kmed2	0.111		0.129		0.122	0.267	0.231
Liu	0.210		0.395		0.749	2.958	2.806
LiuMod	0.199		0.354		0.602	1.001	0.905
AULE_d1	0.215		0.412		0.812	3.835	3.884
AULE_d2	0.215		0.413		0.816	4.115	4.414
KLE	0.106		0.195		0.381	1.915	2.316
JSE	0.187		0.327		0.568	2.027	2.260
RJSE	0.197		0.343		0.597	2.185	2.556
Lasso	0.208		0.362		0.602	1.579	1.900
EN	0.160		0.253		0.413	1.327	1.753
				(2)
n	30	30	30	30	100	100	100	100
$\rho$	0.80	0.90	0.95	0.99	0.80	0.90	0.95	0.99
k1	41.937	41.656	41.689	41.510	139.979	139.484	139.178	138.761
k2	42.191	42.133	42.313	42.214	139.371	138.977	138.935	138.983
k3	42.707	42.376	42.222	41.603	140.517	140.292	140.213	139.540
k4	43.040	42.863	42.799	41.861	140.584	140.420	140.446	140.128
k5	43.039	42.861	42.797	41.859	140.584	140.420	140.445	140.127
k1r	42.018	41.728	41.731	41.512	139.994	139.503	139.205	138.769
k2r	41.746	41.627	41.768	41.661	139.465	138.968	138.814	138.748
knew1	41.937	41.656	41.689	41.510	139.979	139.484	139.178	138.761
knew2	42.040	41.970	42.133	42.024	141.146	140.833	140.805	140.873
knew3	41.762	41.570	41.674	41.540	139.458	138.960	138.803	138.745
knew4	41.838	41.562	41.709	41.606	140.546	139.086	138.795	138.816
knew5	41.919	41.644	41.683	41.510	139.974	139.478	139.172	138.760
knewr1	42.052	41.749	41.746	41.521	140.052	139.576	139.270	138.788
knewr2	48.520	48.672	48.807	48.935	164.814	165.436	165.539	166.068
knewr3	42.287	41.785	41.737	41.521	142.603	140.376	139.360	138.759
kmin	42.091	41.778	41.762	41.606	140.058	139.584	139.278	138.818
kmax	48.520	48.672	48.807	48.935	164.814	165.436	165.539	166.068
kmean	42.341	42.192	42.288	42.143	142.165	141.487	141.231	141.100
khm	41.750	41.568	41.668	41.516	139.641	139.123	138.903	138.728
kGM	41.702	41.569	41.689	41.538	139.335	138.901	138.798	138.743
kGM2	41.706	41.554	41.666	41.518	139.373	138.930	138.805	138.723
kmed1	41.709	41.562	41.672	41.514	139.457	138.960	138.805	138.722
kmed2	41.762	41.570	41.686	41.510	139.458	138.960	138.803	138.760
Liu	42.871	42.635	42.558	41.844	140.494	140.259	140.173	139.591
LiuMod	43.017	43.025	43.456	44.625	140.507	140.300	140.305	140.702
AULE_d1	43.209	43.125	43.161	42.386	140.614	140.473	140.527	140.235
AULE_d2	43.257	43.265	43.484	42.781	140.618	140.489	140.582	140.684
KLE	40.943	40.947	41.167	41.031	138.319	138.171	138.256	138.356
JSE	44.532	45.802	47.024	48.050	143.113	146.695	151.563	160.232
RJSE	44.399	45.515	46.606	47.667	143.107	146.502	151.011	159.341
Lasso	42.253	41.941	41.897	41.438	140.283	139.802	139.536	138.963
EN	42.957	42.819	42.930	42.736	140.691	140.403	140.250	140.005
n	200		200		200	200	AVG
$\rho$	0.80		0.90		0.95	0.99	0.80
k1	279.421		278.703		278.389	277.118	153.152
k2	278.737		277.907		277.715	277.086	153.047
k3	279.745		279.256		279.241	278.301	153.834
k4	279.769		279.304		279.333	278.653	154.100
k5	279.769		279.304		279.333	278.652	154.099
k1r	279.425		278.710		278.400	277.127	153.177
k2r	278.991		278.138		277.828	276.982	152.895
knew1	279.421		278.703		278.389	277.118	153.152
knew2	282.989		282.216		282.106	281.576	155.059
knew3	278.850		277.993		277.730	277.002	152.841
knew4	281.836		278.548		277.764	277.034	153.262
knew5	279.419		278.701		278.386	277.116	153.147
knewr1	279.465		278.772		278.477	277.167	153.220
knewr2	331.754		332.516		332.914	332.971	182.245
knewr3	286.282		281.402		279.155	277.123	154.366
kmin	279.466		278.773		278.479	277.167	153.238
kmax	331.754		332.516		332.914	332.971	182.245
kmean	285.061		283.483		282.904	281.913	155.692
khm	279.152		278.333		277.997	277.007	152.949
kGM	278.727		277.909		277.706	276.998	152.801
kGM2	278.766		277.956		277.734	276.979	152.809
kmed1	278.911		278.053		277.765	276.988	152.843
kmed2	278.850		277.993		277.730	277.034	152.843
Liu	279.718		279.210		279.159	278.125	153.886
LiuMod	279.722		279.221		279.197	278.628	154.392
AULE_d1	279.780		279.324		279.366	278.676	154.240
AULE_d2	279.781		279.328		279.382	278.880	154.378
KLE	277.474		277.030		277.078	276.593	152.114
JSE	281.947		285.904		293.810	313.789	163.538
RJSE	282.041		285.828		293.387	312.510	163.160
Lasso	279.705		279.025		278.723	277.378	153.412
EN	279.994		279.482		279.335	278.288	154.158
				(3)
n	30	30	30	30	100	100	100	100
$\rho$	0.80	0.90	0.95	0.99	0.80	0.90	0.95	0.99
k1	0.342	0.343	0.310	0.200	0.253	0.300	0.326	0.263
k2	0.192	0.177	0.163	0.147	0.150	0.138	0.120	0.082
k3	0.495	0.598	0.660	0.566	0.297	0.400	0.532	0.847
k4	0.548	0.724	0.910	1.154	0.302	0.413	0.569	1.105
k5	0.548	0.723	0.909	1.151	0.302	0.413	0.569	1.105
k1r	0.359	0.372	0.345	0.230	0.255	0.302	0.331	0.273
k2r	0.233	0.212	0.183	0.134	0.196	0.199	0.184	0.118
knew1	0.342	0.343	0.310	0.200	0.253	0.300	0.326	0.263
knew2	0.181	0.163	0.148	0.132	0.133	0.122	0.117	0.110
knew3	0.280	0.262	0.225	0.147	0.195	0.199	0.182	0.116
knew4	0.193	0.234	0.308	0.662	0.127	0.126	0.159	0.333
knew5	0.337	0.337	0.302	0.195	0.253	0.299	0.324	0.261
knewr1	0.367	0.379	0.350	0.227	0.260	0.313	0.348	0.296
knewr2	0.388	0.381	0.377	0.374	0.399	0.390	0.386	0.383
knewr3	0.182	0.171	0.176	0.218	0.154	0.116	0.102	0.108
kmin	0.377	0.395	0.373	0.662	0.260	0.314	0.350	0.336
kmax	0.388	0.381	0.377	0.374	0.399	0.390	0.386	0.383
kmean	0.182	0.165	0.152	0.136	0.147	0.131	0.124	0.114
khm	0.279	0.269	0.247	0.221	0.220	0.239	0.238	0.191
kGM	0.226	0.214	0.194	0.149	0.164	0.164	0.155	0.117
kGM2	0.254	0.250	0.233	0.183	0.179	0.188	0.186	0.150
kmed1	0.250	0.251	0.251	0.189	0.195	0.198	0.183	0.161
kmed2	0.280	0.263	0.290	0.195	0.195	0.199	0.182	0.261
Liu	0.532	0.693	0.879	1.511	0.296	0.400	0.536	0.957
LiuMod	0.443	0.483	0.423	0.668	0.282	0.363	0.440	0.337
AULE_d1	0.574	0.788	1.056	1.985	0.304	0.419	0.583	1.182
AULE_d2	0.580	0.812	1.134	2.261	0.304	0.420	0.589	1.304
KLE	0.371	0.504	0.682	1.526	0.191	0.257	0.352	0.768
JSE	0.458	0.585	0.744	1.463	0.274	0.356	0.462	0.829
RJSE	0.478	0.616	0.795	1.578	0.281	0.365	0.474	0.863
Lasso	0.470	0.577	0.692	1.124	0.288	0.372	0.469	0.725
EN	0.393	0.471	0.568	1.030	0.240	0.286	0.352	0.574
n	200		200		200	200	AVG
$\rho$	0.80		0.90		0.95	0.99
k1	0.194		0.247		0.297	0.314	0.282
k2	0.137		0.140		0.129	0.084	0.138
k3	0.211		0.291		0.402	0.792	0.508
k4	0.213		0.295		0.412	0.885	0.627
k5	0.212		0.295		0.412	0.885	0.627
k1r	0.194		0.248		0.299	0.318	0.294
k2r	0.166		0.188		0.193	0.142	0.179
knew1	0.194		0.247		0.297	0.314	0.282
knew2	0.121		0.114		0.110	0.107	0.130
knew3	0.154		0.166		0.161	0.109	0.183
knew4	0.111		0.094		0.113	0.239	0.225
knew5	0.194		0.247		0.297	0.312	0.280
knewr1	0.197		0.253		0.310	0.345	0.304
knewr2	0.401		0.392		0.388	0.384	0.387
knewr3	0.151		0.106		0.082	0.078	0.137
kmin	0.197		0.253		0.310	0.346	0.348
kmax	0.401		0.392		0.388	0.384	0.387
kmean	0.140		0.126		0.118	0.110	0.137
khm	0.178		0.211		0.232	0.209	0.228
kGM	0.134		0.141		0.140	0.112	0.159
kGM2	0.143		0.158		0.164	0.143	0.186
kmed1	0.159		0.176		0.175	0.176	0.197
kmed2	0.154		0.166		0.161	0.239	0.215
Liu	0.210		0.289		0.397	0.774	0.623
LiuMod	0.206		0.275		0.360	0.473	0.396
AULE_d1	0.213		0.296		0.416	0.901	0.726
AULE_d2	0.213		0.296		0.417	0.941	0.773
KLE	0.135		0.180		0.250	0.559	0.481
JSE	0.200		0.267		0.352	0.645	0.553
RJSE	0.206		0.273		0.360	0.665	0.580
Lasso	0.211		0.283		0.369	0.588	0.514
EN	0.180		0.217		0.266	0.438	0.418
				(4)
n	30	30	30	30	100	100	100	100
$\rho$	0.80	0.90	0.95	0.99	0.80	0.90	0.95	0.99
k1	59.993	59.763	53.766	34.610	44.012	52.009	56.460	45.532
k2	33.468	30.649	28.267	25.532	26.026	24.016	20.797	14.241
k3	86.737	104.073	114.732	98.028	51.550	69.322	92.266	146.722
k4	96.069	125.973	158.078	199.929	52.395	71.680	98.594	191.407
k5	96.051	125.921	157.961	199.506	52.394	71.679	98.590	191.370
k1r	62.954	64.742	59.938	39.890	44.232	52.398	57.309	47.242
k2r	40.821	36.893	31.831	23.155	33.987	34.535	31.900	20.497
knew1	59.993	59.763	53.766	34.610	44.012	52.009	56.460	45.532
knew2	31.511	28.255	25.652	22.799	23.027	21.070	20.236	19.019
knew3	49.159	45.612	39.044	25.403	33.930	34.441	31.598	20.045
knew4	33.719	40.670	53.420	114.680	21.985	21.881	27.574	57.689
knew5	59.144	58.597	52.482	33.730	43.937	51.858	56.205	45.172
knewr1	64.375	65.998	60.771	39.252	45.120	54.218	60.308	51.234
knewr2	67.463	66.050	65.423	64.756	69.221	67.618	66.955	66.299
knewr3	31.748	29.798	30.544	37.727	26.721	20.100	17.622	18.768
kmin	66.069	68.831	64.839	114.717	45.204	54.398	60.639	58.174
kmax	67.463	66.050	65.423	64.756	69.221	67.618	66.955	66.299
kmean	31.700	28.630	26.397	23.554	25.471	22.701	21.416	19.672
khm	48.850	46.863	42.830	38.334	38.147	41.423	41.258	33.168
kGM	39.612	37.283	33.671	25.751	28.446	28.473	26.864	20.200
kGM2	44.491	43.527	40.435	31.656	31.173	32.676	32.220	26.000
kmed1	43.820	43.682	43.577	32.695	33.911	34.392	31.733	27.958
kmed2	49.159	45.753	50.335	33.730	33.930	34.441	31.598	45.171
Liu	93.277	120.633	152.610	261.738	51.499	69.344	92.930	165.768
LiuMod	77.600	84.166	73.440	115.746	49.044	62.923	76.255	58.437
AULE_d1	100.683	137.145	183.417	343.988	52.782	72.686	101.001	204.725
AULE_d2	101.672	141.371	196.938	391.811	52.829	72.924	102.109	225.915
KLE	64.966	87.800	118.530	264.339	33.271	44.524	61.018	133.009
JSE	80.052	101.773	129.233	253.559	47.522	61.726	79.993	143.662
RJSE	83.645	107.124	138.070	273.489	48.756	63.211	82.180	149.493
Lasso	82.412	100.475	120.209	194.708	49.963	64.469	81.354	125.520
EN	68.854	81.916	98.714	178.420	41.716	49.565	61.030	99.430
n	200		200		200	200	AVG
$\rho$	0.80		0.90		0.95	0.99
k1	33.662		42.846		51.520	54.312	49.040
k2	23.801		24.193		22.276	14.479	23.979
k3	36.656		50.484		69.694	137.220	88.124
k4	36.863		51.084		71.384	153.347	108.900
k5	36.863		51.083		71.383	153.342	108.845
k1r	33.701		42.927		51.730	55.072	51.011
k2r	28.784		32.572		33.463	24.570	31.084
knew1	33.662		42.846		51.520	54.312	49.040
knew2	20.991		19.701		19.138	18.536	22.495
knew3	26.632		28.705		27.912	18.856	31.778
knew4	19.242		16.335		19.660	41.426	39.023
knew5	33.647		42.810		51.446	54.131	48.597
knewr1	34.093		43.860		53.672	59.749	52.721
knewr2	69.437		67.902		67.187	66.512	67.069
knewr3	26.197		18.321		14.280	13.562	23.782
kmin	34.101		43.878		53.713	59.883	60.370
kmax	69.437		67.902		67.187	66.512	67.069
kmean	24.256		21.745		20.478	19.070	23.758
khm	30.810		36.636		40.197	36.157	39.556
kGM	23.196		24.512		24.261	19.338	27.634
kGM2	24.838		27.372		28.343	24.735	32.289
kmed1	27.651		30.482		30.370	30.567	34.236
kmed2	26.632		28.705		27.912	41.426	37.399
Liu	36.505		50.118		68.788	134.040	108.104
LiuMod	35.649		47.745		62.427	81.895	68.777
AULE_d1	36.959		51.346		72.038	156.152	126.077
AULE_d2	36.967		51.389		72.261	162.942	134.094
KLE	23.342		31.271		43.322	96.751	83.512
JSE	34.764		46.227		61.043	111.763	95.943
RJSE	35.689		47.233		62.447	115.169	100.542
Lasso	36.574		48.995		63.926	101.870	89.206
EN	31.180		37.541		46.024	75.867	72.521

For the remaining simulation scenarios, to avoid excessive tabulation and to focus on the relative performance of the competing estimators, we report only the overall rank summaries rather than the full numerical results. The rankings are obtained by averaging the individual ranks based on MSE, AIC, MAD, and MAPE across the corresponding parameter settings in Table 3.

Table 3. (1) Rank summary of estimators based on MSE, AIC, MAD, and MAPE when τ = 0%, n = 30, 100, 200 σ = 2, and p = 3. (2) Rank summary of estimators based on MSE, AIC, MAD, and MAPE when τ = 0%, n = 30, 100, 200 σ = 2, and p = 10. (3) Rank summary of estimators based on MSE, AIC, MAD, and MAPE when τ = 0%, n = 30, 100, 200 σ = 5, and p = 3. (4) Rank summary of estimators based on MSE, AIC, MAD, and MAPE when τ = 0%, n = 30, 100, 200 σ = 5, and p = 10. (5) Rank summary of estimators based on MSE, AIC, MAD, and MAPE when τ = 10%, n = 30, 100, 200 σ = 2, and p = 3. (6) Rank summary of estimators based on MSE, AIC, MAD, and MAPE when τ = 10%, n = 30, 100, 200, σ = 2, and p = 10. (7) Rank summary of estimators based on MSE, AIC, MAD, and MAPE when τ = 10%, n = 30, 100, 200, σ = 5, and p = 3. (8) Rank summary of estimators based on MSE, AIC, MAD, and MAPE when τ = 10%, n = 30, 100, 200, σ = 5, and p = 10. (9) Rank summary of estimators based on MSE, AIC, MAD, and MAPE when τ = 25%, n = 30, 100, 200, σ = 2, and p = 3. (10) Rank summary of estimators based on MSE, AIC, MAD, and MAPE when τ = 25%, n = 30, 100, 200, σ = 2, and p = 10. (11) Rank summary of estimators based on MSE, AIC, MAD, and MAPE when τ = 25%, n = 30, 100, 200, σ = 5, and p = 3. (12) Rank summary of estimators based on MSE, AIC, MAD, and MAPE when τ = 25%, n = 30, 100, 200, σ = 5, and p = 10.

			(1)
Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
kGM	5.000	5.000	5.000	2.000	4.250	1
k2	3.000	4.000	4.000	9.000	5.000	2
k2r	6.000	6.000	6.000	7.000	6.250	3
knew3	7.000	7.000	7.000	4.000	6.250	3
kGM2	8.000	8.000	8.000	3.000	6.750	4
knew2	1.000	1.000	1.000	28.000	7.750	5
kmed1	9.000	9.000	9.000	5.500	8.125	6
knewr3	4.000	2.500	3.000	24.000	8.375	7
kmean	2.000	2.500	2.000	29.000	8.875	8
kmed2	10.000	10.000	10.000	5.500	8.875	8
khm	11.000	12.000	12.000	8.000	10.750	9
knew5	13.000	13.000	13.000	10.000	12.250	10
knew4	12.000	11.000	11.000	16.000	12.500	11
k1	14.500	14.500	14.500	11.500	13.750	12
knew1	14.500	14.500	14.500	11.500	13.750	12
k1r	16.000	16.000	16.000	13.000	15.250	13
knewr1	17.000	17.000	17.000	14.000	16.250	14
kmin	20.000	18.000	18.000	15.000	17.750	15
KLE	28.000	23.000	23.000	1.000	18.750	16
k3	22.000	24.000	24.000	18.000	22.000	17
LiuMod	21.000	21.000	21.000	26.000	22.250	18
EN	23.000	22.000	22.000	22.000	22.250	18
knewr2	18.500	19.500	19.500	32.500	22.500	19
kmax	18.500	19.500	19.500	32.500	22.500	19
Lasso	24.000	25.000	25.000	17.000	22.750	20
k5	26.000	29.500	29.000	20.000	26.125	21
Liu	30.000	28.000	28.000	19.000	26.250	22
k4	27.000	29.500	30.000	21.000	26.875	23
JSE	25.000	26.000	26.000	31.000	27.000	24
RJSE	29.000	27.000	27.000	30.000	28.250	25
AULE_d1	31.000	31.000	31.000	23.000	29.000	26
AULE_d2	32.000	32.000	32.000	25.000	30.250	27
OLS	33.000	33.000	33.000	27.000	31.500	28
			(2)
Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
k2r	3.000	3.000	3.000	4.000	3.250	1
k2	2.000	2.000	2.000	10.000	4.000	2
knew2	6.000	4.000	4.000	3.000	4.250	3
kGM	7.000	5.000	5.000	6.000	5.750	4
kmean	1.000	1.000	1.000	27.000	7.500	5
kmed1	8.000	8.000	8.000	7.000	7.750	6
knewr3	14.000	9.000	9.000	2.000	8.500	7
kGM2	9.000	10.000	10.000	8.000	9.250	8
knewr1	10.000	11.000	11.000	13.000	11.250	9
knew3	11.000	12.000	12.000	12.000	11.750	10
Khm	12.000	13.000	13.000	9.000	11.750	10
knewr2	4.500	6.500	6.500	32.500	12.500	11
Kmax	4.500	6.500	6.500	32.500	12.500	11
kmed2	13.000	14.000	14.000	11.000	13.000	12
knew5	15.000	16.000	16.000	15.000	15.500	13
knew4	20.000	20.000	20.000	5.000	16.250	14
k1	16.500	18.500	18.500	16.500	17.500	15
knew1	16.500	18.500	18.500	16.500	17.500	15
k3	18.000	17.000	17.000	20.000	18.000	16
k1r	19.000	21.000	21.000	18.000	19.750	17
EN	21.000	15.000	15.000	29.000	20.000	18
Lasso	24.000	22.000	22.000	14.000	20.500	19
Kmin	22.000	23.000	23.000	19.000	21.750	20
KLE	30.000	29.000	29.000	1.000	22.250	21
LiuMod	23.000	24.000	24.000	22.000	23.250	22
Liu	25.000	26.000	26.000	21.000	24.500	23
k5	26.000	27.500	27.000	23.000	25.875	24
k4	27.000	27.500	28.000	24.000	26.625	25
JSE	28.000	25.000	25.000	31.000	27.250	26
AULE_d1	29.000	31.000	31.000	25.000	29.000	27
RJSE	31.000	30.000	30.000	30.000	30.250	28
AULE_d2	32.000	32.000	32.000	26.000	30.500	29
OLS	33.000	33.000	33.000	28.000	31.750	30
			(3)
Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
kGM	1.000	1.000	1.000	7.000	2.500	1
kGM2	6.000	4.000	4.000	3.000	4.250	2
k2r	2.000	2.000	2.000	13.000	4.750	3
kmed1	7.000	5.000	5.000	2.000	4.750	3
knewr3	4.000	3.000	3.000	17.000	6.750	4
kmed2	8.000	8.000	8.000	4.000	7.000	5
khm	9.000	9.000	9.000	5.000	8.000	6
knew2	3.000	6.000	6.000	18.000	8.250	7
knew3	10.000	10.000	10.000	6.000	9.000	8
kmean	5.000	7.000	7.000	21.000	10.000	9
knew5	12.000	11.000	11.000	8.000	10.500	10
k1	13.500	12.500	12.500	9.500	12.000	11
knew1	13.500	12.500	12.500	9.500	12.000	11
k1r	15.000	14.000	14.000	11.000	13.500	12
knew4	16.000	15.000	15.000	14.000	15.000	13
k2	11.000	16.000	16.000	28.000	17.750	14
knewr1	19.000	19.000	19.000	15.000	18.000	15
kmin	20.000	20.000	20.000	16.000	19.000	16
Lasso	23.000	22.000	22.000	12.000	19.750	17
KLE	29.000	25.000	25.000	1.000	20.000	18
knewr2	17.500	17.500	17.500	32.500	21.250	19
kmax	17.500	17.500	17.500	32.500	21.250	19
EN	24.000	21.000	21.000	20.000	21.500	20
k3	22.000	27.000	27.000	19.000	23.750	21
LiuMod	21.000	23.000	23.000	29.000	24.000	22
JSE	25.000	24.000	24.000	31.000	26.000	23
k5	27.000	28.500	28.000	23.000	26.625	24
RJSE	26.000	26.000	26.000	30.000	27.000	25
k4	28.000	28.500	29.000	23.000	27.125	26
Liu	30.000	30.000	30.000	23.000	28.250	27
AULE_d1	31.000	31.000	31.000	25.000	29.500	28
AULE_d2	32.000	32.000	32.000	26.000	30.500	29
OLS	33.000	33.000	33.000	27.000	31.500	30
			(4)
Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
kGM	4.000	3.000	3.000	2.000	3.000	1
k2r	1.000	1.000	1.000	16.000	4.750	2
knew2	7.000	5.000	5.000	3.000	5.000	3
kmean	2.000	2.000	2.000	20.000	6.500	4
kmed1	8.000	6.000	6.000	6.000	6.500	4
knew5	10.000	9.000	9.000	10.000	9.500	5
kGM2	9.000	11.000	12.000	7.000	9.750	6
k2	3.000	4.000	4.000	29.000	10.000	7
k1	11.500	11.000	10.500	11.500	11.125	8
knew1	11.500	11.000	10.500	11.500	11.125	8
khm	13.000	13.000	13.000	8.000	11.750	9
knewr3	16.000	15.000	15.000	5.000	12.750	10
knewr2	5.500	7.500	7.500	32.500	13.250	11
kmax	5.500	7.500	7.500	32.500	13.250	11
kmed2	14.000	14.000	14.000	13.000	13.750	12
k1r	15.000	16.000	16.000	14.000	15.250	13
knew4	19.000	19.000	19.000	4.000	15.250	13
knew3	17.000	17.000	17.000	15.000	16.500	14
knewr1	18.000	18.000	18.000	17.000	17.750	15
Lasso	22.000	22.000	22.000	9.000	18.750	16
kmin	20.000	21.000	21.000	18.000	20.000	17
k3	21.000	23.000	23.000	19.000	21.500	18
EN	23.000	20.000	20.000	24.000	21.750	19
KLE	29.000	30.000	30.000	1.000	22.500	20
LiuMod	24.000	24.000	24.000	26.000	24.500	21
k5	25.000	26.500	27.000	22.500	25.250	22
k4	26.000	28.000	28.000	22.500	26.125	23
Liu	28.000	29.000	29.000	21.000	26.750	24
JSE	27.000	25.000	25.000	31.000	27.000	25
RJSE	30.000	26.500	26.000	30.000	28.125	26
AULE_d1	31.000	31.000	31.000	25.000	29.500	27
AULE_d2	32.000	32.000	32.000	27.000	30.750	28
OLS	33.000	33.000	33.000	28.000	31.750	29
			(5)
Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
kGM2	3.000	2.000	2.000	2.000	2.250	1
kGM	4.000	5.000	5.000	3.000	4.250	2
kmean	1.000	1.000	1.000	15.000	4.500	3
kmed2	5.000	6.000	6.000	4.000	5.250	4
knewr3	6.000	4.000	4.000	10.000	6.000	5
knew3	7.000	7.000	7.000	5.000	6.500	6
knew2	2.000	3.000	3.000	19.000	6.750	7
kmed1	8.000	8.000	8.000	6.000	7.500	8
knew5	9.000	9.000	9.000	7.000	8.500	9
k1	10.500	10.500	10.500	8.500	10.000	10
knew1	10.500	10.500	10.500	8.500	10.000	10
knew4	15.000	12.000	12.000	14.000	13.250	11
k2r	16.000	14.000	14.000	11.000	13.750	12
khm	17.000	15.000	15.000	12.000	14.750	13
k2	12.000	13.000	13.000	28.000	16.500	14
k1r	18.000	18.000	18.000	16.000	17.500	15
knewr1	19.000	19.000	19.000	17.000	18.500	16
knewr2	13.500	16.500	16.500	32.500	19.750	17
kmax	13.500	16.500	16.500	32.500	19.750	17
kmin	20.000	21.000	21.000	18.000	20.000	18
KLE	27.000	26.000	26.000	1.000	20.000	18
Lasso	23.000	23.000	23.000	13.000	20.500	19
EN	24.000	22.000	22.000	21.000	22.250	20
RJSE	21.000	20.000	20.000	31.000	23.000	21
k3	25.000	27.000	27.000	20.000	24.750	22
LiuMod	22.000	24.000	24.000	29.000	24.750	22
JSE	26.000	25.000	25.000	30.000	26.500	23
k5	28.000	28.500	28.000	23.500	27.000	24
k4	29.000	28.500	29.000	23.500	27.500	25
Liu	30.000	30.000	30.000	22.000	28.000	26
AULE_d1	31.000	31.000	31.000	25.000	29.500	27
AULE_d2	32.000	32.000	32.000	26.000	30.500	28
OLS	33.000	33.000	33.000	27.000	31.500	29
			(6)
Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
k2r	5.000	2.000	2.000	4.500	3.375	1
knew2	6.000	4.000	4.000	2.000	4.000	2
kmean	1.000	1.000	1.000	16.000	4.750	3
kGM	7.000	7.000	7.000	4.500	6.375	4
knew5	9.000	8.000	8.000	9.000	8.500	5
kGM2	8.000	10.000	11.000	6.000	8.750	6
k2	2.000	3.000	3.000	29.000	9.250	7
k1	10.500	10.000	9.500	10.500	10.125	8
knew1	10.500	10.000	9.500	10.500	10.125	8
kmed1	12.000	12.000	12.000	8.000	11.000	9
knewr2	3.500	5.500	5.500	32.500	11.750	10
kmax	3.500	5.500	5.500	32.500	11.750	10
knew4	16.000	15.000	15.000	3.000	12.250	11
kmed2	13.000	13.000	13.000	12.000	12.750	12
knew3	14.000	14.000	14.000	14.000	14.000	13
knewr3	18.000	18.000	18.000	7.000	15.250	14
khm	15.000	16.000	16.000	15.000	15.500	15
Lasso	21.000	19.000	19.000	13.000	18.000	16
knewr1	17.000	20.000	20.000	17.000	18.500	17
k1r	19.000	21.000	21.000	18.000	19.750	18
EN	22.000	17.000	17.000	26.000	20.500	19
k3	20.000	22.000	22.000	20.000	21.000	20
kmin	23.000	23.000	23.000	19.000	22.000	21
KLE	30.000	29.000	29.000	1.000	22.250	22
LiuMod	24.000	25.000	25.000	24.000	24.500	23
k5	26.000	27.000	27.000	22.000	25.500	24
RJSE	25.000	24.000	24.000	31.000	26.000	25
k4	27.000	28.000	28.000	23.000	26.500	26
Liu	29.000	30.000	30.000	21.000	27.500	27
JSE	28.000	26.000	26.000	30.000	27.500	27
AULE_d1	31.000	31.000	31.000	25.000	29.500	28
AULE_d2	32.000	32.000	32.000	27.000	30.750	29
OLS	33.000	33.000	33.000	28.000	31.750	30
			(7)
Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
k1	3.500	4.000	3.500	4.000	3.750	1
knew1	3.500	4.000	3.500	4.000	3.750	1
knew5	2.000	4.000	5.000	4.000	3.750	1
kGM2	6.000	1.000	1.000	7.000	3.750	1
kmed2	5.000	2.000	2.000	6.000	3.750	1
kmean	1.000	6.000	6.000	12.000	6.250	2
kmed1	8.000	7.000	7.000	8.000	7.500	3
kGM	9.000	8.000	8.000	9.000	8.500	4
k2r	10.000	10.000	10.000	10.000	10.000	5
knew2	7.000	9.000	9.000	16.000	10.250	6
knewr3	13.000	11.000	11.000	13.000	12.000	7
knew3	14.000	12.000	12.000	11.000	12.250	8
knew4	16.000	13.000	13.000	14.000	14.000	9
khm	17.000	14.000	14.000	15.000	15.000	10
knewr2	11.500	15.500	15.000	21.500	15.875	11
kmax	11.500	15.500	16.000	21.500	16.125	12
Lasso	22.000	21.000	21.000	2.000	16.500	13
k1r	18.000	18.000	18.000	17.000	17.750	14
k2	15.000	17.000	17.000	23.000	18.000	15
knewr1	19.000	19.000	19.000	19.000	19.000	16
kmin	20.000	20.000	20.000	20.000	20.000	17
KLE	27.000	26.000	26.000	1.000	20.000	17
RJSE	21.000	23.000	23.000	24.000	22.750	18
JSE	26.000	25.000	25.000	18.000	23.500	19
EN	24.000	22.000	22.000	32.000	25.000	20
k3	25.000	27.000	27.000	25.000	26.000	21
LiuMod	23.000	24.000	24.000	33.000	26.000	21
k5	28.000	28.500	28.000	26.500	27.750	22
k4	29.000	28.500	29.000	26.500	28.250	23
Liu	30.000	30.000	30.000	28.000	29.500	24
AULE_d1	31.000	31.000	31.000	29.000	30.500	25
AULE_d2	32.000	32.000	32.000	30.000	31.500	26
OLS	33.000	33.000	33.000	31.000	32.500	27
			(8)
Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
k2r	2.000	2.000	2.000	6.000	3.000	1
knew5	3.000	3.000	3.000	4.000	3.250	2
k1	4.500	4.500	4.500	4.000	4.375	3
knew1	4.500	4.500	4.500	4.000	4.375	3
kmean	1.000	1.000	1.000	16.000	4.750	4
kGM	8.000	6.000	6.000	7.000	6.750	5
knew2	12.000	7.000	7.000	8.000	8.500	6
kGM2	10.000	9.000	9.000	9.000	9.250	7
kmed2	11.000	8.000	8.000	10.000	9.250	7
kmed1	13.000	10.000	10.000	11.000	11.000	8
knewr2	6.500	11.500	11.500	22.500	13.000	9
kmax	6.500	11.500	11.500	22.500	13.000	9
khm	14.000	14.000	14.000	14.000	14.000	10
knew4	16.000	15.000	15.000	12.000	14.500	11
k2	9.000	13.000	13.000	24.000	14.750	12
Lasso	21.000	19.000	19.000	2.000	15.250	13
knew3	15.000	16.000	16.000	15.000	15.500	14
knewr3	18.000	17.000	17.000	13.000	16.250	15
k1r	17.000	18.000	18.000	17.000	17.500	16
knewr1	19.000	20.000	20.000	18.000	19.250	17
kmin	20.000	22.000	22.000	19.000	20.750	18
KLE	29.000	29.000	29.000	1.000	22.000	19
k3	22.000	23.000	23.000	21.000	22.250	20
EN	23.000	21.000	21.000	30.000	23.750	21
JSE	28.000	26.000	26.000	20.000	25.000	22
RJSE	24.000	24.000	24.000	28.000	25.000	22
k5	26.000	27.500	27.000	25.500	26.500	23
k4	27.000	27.500	28.000	25.500	27.000	24
LiuMod	25.000	25.000	25.000	33.000	27.000	24
Liu	30.000	30.000	30.000	27.000	29.250	25
AULE_d1	31.000	31.000	31.000	29.000	30.500	26
AULE_d2	32.000	32.000	32.000	31.000	31.750	27
OLS	33.000	33.000	33.000	32.000	32.750	28
			(9)
Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
kGM2	2.000	1.000	1.000	2.000	1.500	1
kmed2	4.000	3.000	3.000	3.000	3.250	2
kmean	1.000	2.000	2.000	12.500	4.375	3
knew5	5.000	6.000	5.000	4.000	5.000	4
kGM	8.000	4.000	4.000	7.000	5.750	5
k1	6.500	6.000	6.500	5.500	6.125	6.
knew1	6.500	6.000	6.500	5.500	6.125	6
kmed1	9.000	8.000	8.000	8.000	8.250	7
knewr3	10.000	9.000	9.000	10.000	9.500	8
knew2	3.000	10.000	10.000	16.000	9.750	9
knew3	11.000	11.000	11.000	9.000	10.500	10
knew4	14.000	12.000	12.000	12.500	12.625	11
k2r	15.000	13.000	13.000	14.000	13.750	12
khm	17.000	14.000	14.000	15.000	15.000	13
knewr2	12.500	15.500	15.500	24.500	17.000	14
kmax	12.500	15.500	15.500	24.500	17.000	14
k1r	18.000	18.000	18.000	17.000	17.750	15
k2	16.000	17.000	17.000	22.000	18.000	16
knewr1	19.000	19.000	19.000	18.000	18.750	17
Lasso	21.000	22.000	22.000	11.000	19.000	18
kmin	20.000	20.000	20.000	19.000	19.750	19
KLE	27.000	26.000	26.000	1.000	20.000	20
RJSE	23.000	23.000	23.000	23.000	23.000	21
EN	22.000	21.000	21.000	28.000	23.000	21
JSE	26.000	25.000	25.000	20.000	24.000	22
k3	25.000	27.000	27.000	21.000	25.000	23
LiuMod	24.000	24.000	24.000	33.000	26.250	24
k5	28.000	28.000	28.000	26.500	27.625	25
k4	29.000	29.000	29.000	26.500	28.375	26
Liu	30.000	30.000	30.000	29.000	29.750	27
AULE_d1	31.000	31.000	31.000	30.000	30.750	28
AULE_d2	32.000	32.000	32.000	31.000	31.750	29
OLS	33.000	33.000	33.000	32.000	32.750	30
			(10)
Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
k2r	11.000	11.000	11.000	3.000	9.000	1
Liu	5.000	5.000	5.000	25.000	10.000	2
knew2	16.000	12.000	12.000	1.000	10.250	3
LiuMod	5.000	5.000	5.000	26.000	10.250	3
kmean	10.000	10.000	10.000	12.000	10.500	4
AULE_d1	5.000	5.000	5.000	27.000	10.500	4
AULE_d2	5.000	5.000	5.000	28.000	10.750	5
KLE	5.000	5.000	5.000	29.000	11.000	6
JSE	5.000	5.000	5.000	30.000	11.250	7
RJSE	5.000	5.000	5.000	31.000	11.500	8
Lasso	5.000	5.000	5.000	32.000	11.750	9
EN	5.000	5.000	5.000	33.000	12.000	10
knew5	17.000	13.000	13.000	6.000	12.250	11
kGM	15.000	16.000	16.000	2.000	12.250	11
k1	18.500	14.500	14.500	7.500	13.750	12
knew1	18.500	14.500	14.500	7.500	13.750	12
kGM2	20.000	17.000	17.000	4.000	14.500	13
kmed1	21.000	19.000	19.000	9.000	17.000	14
kmed2	22.000	18.000	18.000	10.000	17.000	14
knewr2	12.500	20.500	20.500	22.500	19.000	15
kmax	12.500	20.500	20.500	22.500	19.000	15
knew4	25.000	24.000	24.000	5.000	19.500	16
k2	14.000	22.000	22.000	21.000	19.750	17
khm	23.000	23.000	23.000	14.000	20.750	18
knew3	24.000	25.000	25.000	13.000	21.750	19
knewr3	26.000	26.000	26.000	11.000	22.250	20
knewr1	27.000	27.000	27.000	15.000	24.000	21
k1r	28.000	28.000	28.000	16.000	25.000	22
kmin	29.000	29.000	29.000	17.000	26.000	23
k3	30.000	30.000	30.000	18.000	27.000	24
k5	31.000	31.500	31.000	19.000	28.125	25
k4	32.000	31.500	32.000	20.000	28.875	26
OLS	33.000	33.000	33.000	24.000	30.750	27
			(11)
Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
kmed2	5.000	1.500	1.000	6.000	3.375	1
k1	2.000	4.500	4.500	4.500	3.875	2
knew1	2.000	4.500	4.500	4.500	3.875	2
knew5	2.000	6.000	6.000	3.000	4.250	3
kGM2	6.000	1.500	2.000	8.000	4.375	4
kmed1	8.000	3.000	3.000	10.000	6.000	5
kmean	4.000	7.500	7.000	7.000	6.375	6
kGM	9.000	7.500	8.000	11.000	8.875	7
knew2	7.000	10.000	10.000	9.000	9.000	8
k2r	10.000	9.000	9.000	12.000	10.000	9
knewr3	14.000	11.000	11.000	15.000	12.750	10
knewr2	11.000	14.500	14.000	14.000	13.375	11
kmax	12.000	14.500	15.000	13.000	13.625	12
knew3	15.000	12.000	12.000	17.000	14.000	13
knew4	16.000	13.000	13.000	16.000	14.500	14
k2	13.000	17.000	17.000	18.000	16.250	15
Lasso	21.000	21.000	21.000	2.000	16.250	16
khm	17.000	16.000	16.000	20.000	17.250	17
k1r	18.000	18.000	18.000	21.000	18.750	18
knewr1	19.000	19.000	19.000	22.000	19.750	19
KLE	27.000	26.000	26.000	1.000	20.000	20
kmin	20.000	20.000	20.000	23.000	20.750	21
JSE	26.000	24.000	24.000	19.000	23.250	22
RJSE	23.000	23.000	23.000	24.000	23.250	23
EN	22.000	22.000	22.000	32.000	24.500	24
k3	25.000	27.000	27.000	25.000	26.000	25
LiuMod	24.000	25.000	25.000	33.000	26.750	26
k5	28.000	28.500	28.000	26.500	27.750	27
k4	29.000	28.500	29.000	26.500	28.250	28
Liu	30.000	30.000	30.000	28.000	29.500	29
AULE_d1	31.000	31.000	31.000	29.000	30.500	30
AULE_d2	32.000	32.000	32.000	30.000	31.500	31
OLS	33.000	33.000	33.000	31.000	32.500	32
			(12)
Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
knew5	3.000	2.000	3.000	3.000	2.750	1
k1	4.500	2.000	1.500	4.500	3.125	2
knew1	4.500	2.000	1.500	4.500	3.125	2
k2r	2.000	4.500	4.000	6.000	4.125	4
kmean	1.000	4.500	5.000	11.000	5.375	5
kGM	6.000	6.000	6.000	7.000	6.250	6
kGM2	9.000	7.500	7.000	8.000	7.875	7
kmed1	10.000	7.500	8.000	10.000	8.875	8
kmed2	12.000	9.000	9.000	9.000	9.750	9
knew2	13.000	10.000	10.000	12.000	11.250	10
knewr2	7.500	11.500	11.000	18.500	12.125	11
kmax	7.500	11.500	12.000	18.500	12.375	12
khm	14.000	14.000	14.000	15.000	14.250	13
k2	11.000	13.000	13.000	21.000	14.500	14
knew4	17.000	15.000	15.000	13.000	15.000	15
Lasso	21.000	19.000	19.000	2.000	15.250	16
knew3	15.000	16.000	16.000	16.000	15.750	17
k1r	16.000	17.000	17.000	17.000	16.750	18
knewr3	18.000	18.000	18.000	14.000	17.000	19.
knewr1	19.000	21.000	21.000	20.000	20.250	20
KLE	29.000	27.000	27.000	1.000	21.000	21
kmin	20.000	22.000	22.000	22.000	21.500	22
k3	23.000	23.000	23.000	24.000	23.250	23
EN	22.000	20.000	20.000	31.000	23.250	23
JSE	24.000	24.000	24.000	23.000	23.750	24
RJSE	28.000	25.000	25.000	25.000	25.750	25
k5	26.000	28.500	28.000	26.500	27.250	26
LiuMod	25.000	26.000	26.000	33.000	27.500	27
k4	27.000	28.500	29.000	26.500	27.750	28
Liu	30.000	30.000	30.000	28.000	29.500	29
AULE_d1	31.000	31.000	31.000	29.000	30.500	30
AULE_d2	32.000	32.000	32.000	30.000	31.500	31
OLS	33.000	33.000	33.000	32.000	32.750	32

4.6. Rank-Based Comparison of Estimators

Figure 1 summarizes the best-performing estimators based on their average ranks across MSE, AIC, MAD, and MAPE for different $\sigma , p$ settings, and contamination levels. When there are no outliers $(\tau =0),$ several ridge-type estimators and some proposed estimators consistently achieve the top ranks, especially in higher-dimensional settings $(p=10).$ This highlights their effectiveness in managing multicollinearity. With moderate contamination ($\tau =10\%),$ the rankings shift toward robust and modified shrinkage estimators, showing they provide better stability than classical methods. Under heavy contamination $(\tau =25$%), robust estimators clearly lead across all scenarios, while non-robust estimators suffer significant performance loss. Overall, the rank-based results show that well-designed ridge-type estimators perform well in clean data settings, while robust versions are crucial for maintaining reliable performance when outliers are present.

Figure 1. (a) Top 10 estimators based on overall rank (average final rank) under τ = 0% across $(\sigma , p)$ scenarios. (b) Top 10 estimators based on overall rank (average final rank) under τ = 10% across $(\sigma , p)$ scenarios. (c) Top 10 estimators based on overall rank (Avg_Rank and Final_Rank) under τ = 25% across ($\sigma , p$) scenarios.

4.7. Real-Life Application

To illustrate the applications of the existing and newly proposed estimators, we consider the following data.

4.7.1. Example 1: National Health and Nutrition Examination Survey (NHANES) Data

The data used in this analysis come from the National Health and Nutrition Examination Survey (NHANES), a nationwide public health survey conducted by the Centers for Disease Control and Prevention (CDC) in the United States. NHANES assesses the health and nutritional status of adults and children through interviews, physical exams, and lab tests. The survey gathers extensive information on demographic characteristics, body measurements, blood pressure, biochemical indicators, and health-related behaviors. This makes it widely used in epidemiological and public health research. In this study, we focus on a subset of the NHANES data that examines cardiovascular and metabolic health indicators. The response variable is systolic blood pressure. The explanatory variables include age, body mass index (BMI), diastolic blood pressure, pulse rate, and related physiological measures. These variables tend to be highly correlated in population health studies, often leading to severe multicollinearity in regression modeling.

NHANES data are collected from a large, heterogeneous population and include clinical and physiological measurements that are naturally prone to extreme observations. Variables such as blood pressure, body mass index, and pulse rate may exhibit unusually high or low values due to underlying health conditions, measurement variability, or reporting errors. As a result, the response variable may contain outliers in the $\mathit{y}$ direction, even when the design matrix does not show clear leverage points in the $x$ direction.

After removing observations with missing values, the final dataset consists of n = 7936 observations and six explanatory variables. The eigenvalues are reported as $\mathrm{207,049}, 400.00, 3063.00, 837.00, 1220, 674.00, 1006, 386.00, 8677.00, \mathrm{a}\mathrm{n}\mathrm{d} 1289.09$.

The condition index number is calculated as, $\sqrt{{\displaystyle \frac{{\mathsf{\lambda }}_{\mathrm{m}\mathrm{a}\mathrm{x}}}{{\mathsf{\lambda }}_{\mathrm{m}\mathrm{i}\mathrm{n}}}}}=400.77>100$, which clearly indicates the presence of severe multicollinearity among the explanatory variables. This confirms the suitability of the NHANES data for evaluating biased, shrinkage, and robust regression estimators. We found 3.91% outliers in y-directions. The rank summary of the estimators is presented in Table 4.

Table 4. Rank summary of estimators based on MSE, MAD, MAPE, and AIC (NHANES data).

Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
knewr3	1	4	2	30	9.25	1
k2r	3	5	3	31	10.5	2
k1r	4	6	4	32	11.5	3
KLE	17	10	8	14	12.25	4
k5	22	13	14	1	12.5	5
knewr1	5	7	5	33	12.5	5
k4	23	14	13	2	13	6
JSE	18	9	6	20	13.25	7
k3	20	11	16	8	13.75	8
Liu	24	16	12	3	13.75	8
LiuMod	21	12	15	7	13.75	8
AULE_d1	25	17	11	4	14.25	9
AULE_d2	26	18	10	5.5	14.88	10
kmin	19	15	17	9	15	11
kGM2	27	19	9	5.5	15.13	12
knewr2	29	3	1	29	15.5	13
k1	15.5	20.5	18.5	10.5	16.25	14
knew1	15.5	20.5	18.5	10.5	16.25	14
RJSE	33	8	7	18	16.5	15
knew5	14	22	20	12	17	16
khm	13	23	21	13	17.5	17
kmed1	12	24	22	15	18.25	18
knew3	10.5	25.5	23.5	16.5	19	19
kmed2	10.5	25.5	23.5	16.5	19	19
OLS	9	27	25	19	20	20
k2	8	28	26	21	20.75	21
kGM	7	29	27	22	21.25	22
knew4	2	31	29	24	21.5	23
knew2	6	30	28	23	21.75	24
kmean	28	1	32	27	22	25
kmax	30	2	33	28	23.25	26
Lasso	31	33	31	25	30	27
EN	32	32	30	26	30	27

Table 3 summarizes the estimator rankings for the NHANES dataset. The robust ridge-type estimators showed the strongest overall performance, with $kNewr3$, $k2r$, and $k1r$ obtaining the top three final ranks. This suggests that robust shrinkage estimators provide better stability across MSE, MAD, MAPE, and AIC in this application. By contrast, LASSO and Elastic Net had the lowest overall ranks, indicating weaker performance under the combined ranking criteria. These results are consistent with the simulation findings and further support the advantage of robust ridge-type estimators under multicollinearity.

4.7.2. Example 2: Birthweight Dataset (Maternal and Child Health Data)

The data used in this empirical analysis is obtained from a public-health dataset available in the R environment. The dataset contains observations on key demographic and physiological variables commonly used in epidemiological and population health studies. Such datasets are routinely employed to investigate relationships between health outcomes and biological or behavioral risk factors and are well suited for regression-based modeling.

In this study, the response variable represents a continuous health outcome of interest, while the explanatory variables include multiple correlated predictors such as anthropometric measurements, physiological indicators, and related covariates. These variables are known to be biologically interrelated, which often leads to strong linear dependence among predictors in regression models. In the dataset, potential outliers are expected primarily in the response $\left(y\right)$ direction, rather than in the design matrix.

After removing observations with missing values, the final dataset consists of $n=189$ observations and nine explanatory variables, forming the design matrix $X$. The eigenvalues are reported as $\mathrm{175,986.70}$, $5114.51, 200.45, 157.39, 58.35, 34.51, 30.05, 21.12, 9.65$.

The condition index is calculated as $\sqrt{{\displaystyle \frac{{\mathsf{\lambda }}_{\mathrm{m}\mathrm{a}\mathrm{x}}}{{\mathsf{\lambda }}_{\mathrm{m}\mathrm{i}\mathrm{n}}}}}=135.02>100$, which is substantially greater than the commonly used threshold value of 100. This result provides strong evidence of severe multicollinearity among the explanatory variables. Consequently, the ordinary least squares (OLS) estimator may suffer from instability and inflated variances, motivating the use of biased and shrinkage-based regression estimators. Using robust standardized residuals with a cutoff of $2.5,$ no response-direction outliers were detected in the birthweight dataset. This suggests that the main issue in this example is multicollinearity rather than contamination by extreme response values. The rank summary of the estimators is presented in Table 4.

Table 5 summarizes the estimator rankings for the Birthweight dataset. The $k2$ estimator obtained the best overall rank, followed by $kNewr2$, $kmax$, $k2r$, and $kmean$, indicating that ridge-type shrinkage estimators performed most consistently across the four evaluation criteria. Although LASSO and Elastic Net ranked highly for MAPE and AIC, their poor MSE performance lowered their overall rankings. OLS ranked lowest, suggesting that classical least squares was less stable under severe multicollinearity and response direction outliers. These findings are consistent with the simulation results and support the use of shrinkage-based estimators in this application.

Table 5. Rank summary of estimators based on MSE, MAD, MAPE, and AIC (birthweight data).

Estimator	Rank_MSE	Rank_MAD	Rank_MAPE	Rank_AIC	Avg_Rank	Final_Rank
k2	23	17	3	3	11.5	1
knewr2	21	17	4.5	4.5	11.75	2
kmax	22	17	4.5	4.5	12	3
k2r	20	17	6	6	12.25	4
kmean	19	17	7	7	12.5	5
knew5	18	17	8	8	12.75	6
k1r	15	17	10	10	13	7
Lasso	32.5	17	1.5	1.5	13.125	8
EN	32.5	17	1.5	1.5	13.125	8
k1	16.5	17	10	10	13.375	9
knew1	16.5	17	10	10	13.375	9
kmed1	14	17	12	12	13.75	10
kGM	13	17	13	13	14	11
knewr3	12	17	14	14	14.25	12
kGM2	11	17	15	15	14.5	15
knew4	9.5	17	16.5	16.5	14.875	13
kmed2	9.5	17	16.5	16.5	14.875	13
knew2	8	17	18	18	15.25	14
khm	7	17	19	19	15.5	15
knew3	6	17	20	20	15.75	16
knewr1	4	17	21	21.5	15.875	17
kmin	5	17	22	21.5	16.375	18
JSE	3	17	24	23	16.75	19
k3	2	17	25	24	17	20
RJSE	1	17	31	29	19.5	21
k5	24	17	26	25	23	22
k4	25	17	27	26	23.75	23
Liu	27	17	29	27	25	24
LiuMod	26	17	28	32	25.75	25
AULE_d1	28	17	30	28	25.75	25
KLE	30.5	17	23	33	25.875	31
AULE_d2	29	17	32	30	27	32
OLS	30.5	17	33	31	27.875	33

5. Conclusions and Directions for Future Work

In this paper, we review some existing approaches and propose several new estimators for estimating the parameters under multicollinearity in the linear regression model. In addition to a comprehensive simulation study, the proposed estimators are examined using public-health datasets, including the NHANES and Birthweight data, both of which exhibit strong multicollinearity among covariates and natural extreme observations due to population heterogeneity. The findings from the real-data applications are consistent with the simulation results. Our proposed estimators, particularly the geometric-mean-based and robust ridge-type estimators, consistently outperform existing competitors across all evaluation criteria, including MSE, AIC, MAD and MAPE. Classical estimators such as OLS, Liu, and AULE variants exhibit noticeable performance deterioration in the presence of multicollinearity and contamination. Likewise, commonly used penalized methods such as LASSO and elastic net remain comparatively less effective under severe multicollinearity and data contamination. Overall, the real-data analysis shows that the proposed estimators are more efficient, robust, and practical for analyzing complex public health data with correlated predictors and potential outliers. The findings indicate that robust shrinkage methods are preferred when outliers may exist, while well-calibrated shrinkage rules work well when the data seem free of contamination.

From a computational perspective, the proposed estimators are attractive because they are primarily based on closed-form expressions and standard matrix operations. Consequently, their computational burden is relatively low compared with iterative optimization-based methods such as LASSO and Elastic Net. Furthermore, the shrinkage mechanism improves numerical stability under severe multicollinearity by reducing the effects of near-singularity in the design matrix.

Future research could extend these estimators to generalized linear models (GLMs), examine the effects of leverage-point contamination, investigate alternative robust loss functions to improve estimation performance, and study the asymptotic properties and limiting distributions of the proposed estimators under different multicollinearity and contamination settings. In addition, future studies may investigate formal statistical procedures for assessing whether observed performance differences among competing estimators are statistically significant. Although the rank-based summaries used in this study provide a practical framework for overall comparison across multiple criteria, they should be viewed as descriptive rather than formal inferential measures of performance.