Improved Data-Driven Shrinkage Estimators for Regression Models Under Severe Multicollinearity

Abstract

Multicollinearity is a critical issue in regression analysis, often resulting in inflated variances and unstable parameter estimates. Ridge regression is a widely adopted solution to address this challenge; however, existing ridge estimators are typically tailored to specific scenarios, limiting their universal applicability. Akhtar and Alharthi developed ridge estimators based on condition-adjusted ridge estimators (CAREs) to handle severe multicollinearity issues. However, their approach did not account for the error variances in the estimation process. In this study, we propose improvements to these CAREs by incorporating error variances, resulting in the development of multiscale ridge estimators ($\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_1$, $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2$, $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$ and $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$) that more effectively address the challenges posed by severe multicollinearity. We compare the performance of our newly proposed estimators with ordinary least square (OLS) and other existing ridge estimators using both simulation studies and real-life datasets. The evaluation, based on estimated mean squared error (MSE), demonstrates that the proposed estimators consistently outperform existing methods, particularly in scenarios with significant multicollinearity, larger sample sizes, and higher predictor dimensions. Results from three real-life datasets further validate the proposed estimators’ ability to reduce estimation error and improve predictive accuracy across diverse practical applications.

1. Introduction

Multicollinearity, a situation where predictor variables in a regression model are highly correlated, poses a significant challenge to analyzed the statistical models. It inflates the variances of parameter estimates, making them unstable and unreliable. Mathematically, the regression model is defined as

$$\mathit{y}=\mathit{X}\mathit{\phi }+\mathit{ϵ},$$

(1)

Equation (1) $\mathit{y}\in {\mathit{R}}^{\mathit{n}\times \mathbf{1}}$ is the response vector, $\mathit{X}\in {\mathit{R}}^{\mathit{n}\times \mathit{P}}$ is a design matrix, $\mathit{\phi }\in {\mathit{R}}^{\mathit{P}\times \mathbf{1}}$ is the vector of parameters of the model, and $\mathsf{ϵ}$ is the error vector of order n × 1. The OLS parameter estimates are given below:

$${\widehat{\mathit{\phi }}}_{\mathit{O}\mathit{L}\mathit{S}}={\complement }^{-\mathbf{1}}\mathit{\omega },$$

(2)

Here ${\complement =\mathit{X}}^{\prime }\mathit{X}$ and $\mathit{\omega }={\mathit{X}}^{\prime }\mathit{y}$, with the variance–covariance matrix given by

$$\mathit{C}\mathit{o}\mathit{v}\left({\widehat{\mathit{\phi }}}_{\mathit{O}\mathit{L}\mathit{S}}\right)={\mathit{\sigma }}^2{\left(\complement \right)}^{-\mathbf{1}},$$

(3)

The condition number (CN) measures the multicollinearity and is mathematically defined as Equation (4):

$$\mathsf{\kappa }={\displaystyle \frac{{\lambda }_{max}}{{\lambda }_{min}}},$$

(4)

where ${\lambda }_{max}$ and ${\lambda }_{min}$ represent the maximum and minimum eigenvalues, respectively, of matrix$\complement$. In Equation (4), the values of $\mathsf{\kappa }\ge 30$ show significant multicollinearity and the OLS estimates are unstable. Another tool to examine the multicollinearity issues in data is the variance inflation factor (VIF). Mathematically, for the $i\mathrm{t}\mathrm{h}$ predictor, the VIF is computed as

$${VIF}_i={\displaystyle \frac{1}{1-R_i^2}},$$

(5)

Here, $R_i^2$ represents the measure of how well the $i\mathrm{t}\mathrm{h}$ predictor can be explained by the remaining predictors in Equation (5). A ${VIF}_i>10$ suggests high multicollinearity and the need for remedial measures. In the presence of multicollinearity, the matrix $\complement$ becomes nearly singular, resulting in inflated variances of the OLS estimates. For reliable parameter estimates of the model in the presence of significant multicollinearity, the shrinkage parameter is used to mitigate the issue. Initially, ref. [1] introduced the shrinkage parameter ($k)$ for reliable parameter estimates of the regression model to mitigate multicollinearity issues. The ridge estimate of the model is defined as

$${\widehat{\mathit{\phi }}}_{\mathit{r}\mathit{i}\mathit{d}\mathit{g}\mathit{e}}={\left(\complement +k\mathit{I}\right)}^{-\mathbf{1}}\mathit{\omega },$$

(6)

where $k$ is the ridge parameter, and $I$ is the identity matrix. This adjustment effectively replaces the eigenvalues ${\lambda }_i$ of the $\complement$ matrix with ${\lambda }_i+k$, improving numerical stability. Most researchers have introduced basic ridge parameters for high-collinearity data. Garg [2] modified the ridge estimators to mitigate multicollinearity in regression analysis, offering a more stable solution when predictors are highly correlated with each other. Kibria [3] introduced average-based ridge estimators to optimize parameter estimation efficiency for the highly correlated predictor variables, while [4] developed generalized ridge regression to handle the multicollinearity comprehensively as compared to the other method. Ahmed et al. [5] explored kernel ridge-type estimators for partial multicollinearity.

Similarly, ref. [6] improved estimation strategies to effectively mitigate multicollinearity issues of the data. Gregory [7] modified the ridge regression method to be a helpful tool for handling the challenges of multicollinearity in regression analysis. Reference [8] introduced rank ridge estimators to tackle highly correlated genetic data, while [9,10,11,12] introduced estimators for complex multicollinearity, making it a key tool across fields like genetics, environmental science, and econometrics of correlated datasets.

In most cases, the basic ridge parameter does not perform well under severe multicollinearity. To address this limitation, ref. [13] introduced the two-parameter ridge estimator, which incorporates an additional scaling factor $d$ alongside the penalty term $k$. For the two-parameter ridge estimator, Equation (7) is expressed as

$${\widehat{\mathit{\phi }}}_{(\mathit{d},\mathit{k})}={d\left(\complement +k\mathit{I}\right)}^{-\mathbf{1}}\mathit{\omega },$$

(7)

where $d$ is provided additionally to enable more flexible solutions to ridge regression problems and mitigate the effect of severe multicollinearity. The parameter $d$ is mathematically defined in Equation (8):

$$\widehat{d}={\displaystyle \frac{{\left(\mathit{\omega }\right)}^{\prime }{\left(\complement +k\mathit{I}\right)}^{-\mathbf{1}}\mathit{\omega }}{{\left(\mathit{\omega }\right)}^{\prime }{\left(\complement +k\mathit{I}\right)}^{-1}\complement {\left(\complement +k\mathit{I}\right)}^{-\mathbf{1}}\mathit{\omega }}},$$

(8)

The estimator ${\widehat{\phi }}_{(d,k)}$ reduces to the $OLS$ estimator when $d=1$and $k=0$ and to the ridge estimator when $d=1$ and $k>0$. Toker and Kaçıranlar [14] introduced and explored ridge estimators, their applications in cross-sectional analysis and their role in improved stability under multicollinearity scenarios.

Many researchers have focused on two-parameter estimators rather than the basic ridge parameter to address severe multicollinearity among predictors in linear regression models, because two-parameter ridge estimators tend to perform better under such conditions compared to the basic ridge estimator; see references [15,16,17,18]. A recent study by Akhtar and Alharthi [19] introduced three new two-parameter ridge estimators based on the number of predictors, eigenvalues and the condition number.

However, existing CAREs remain sensitive to high noise levels and instability in variance, particularly under severe multicollinearity. This limitation motivates the present study. To address this gap, we propose four modified estimators, $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_1$, $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2,\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$ and $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$, that incorporate a variance-stabilizing term to overcome their limitations under severe multicollinearity and noise. By jointly penalizing multicollinearity through the condition number and estimation uncertainty through variance adjustment, the proposed estimators provide more stable performance across different regression settings.

This paper is organized as follows: Section 2 covers materials and methodology, including existing and newly proposed estimators. The simulation algorithm is outlined in Section 3, while Section 4 focuses on the two real-life applications. Finally, Section 5 provides concluding remarks.

2. Materials and Methodology

The canonical form of Equation (1) is expressed as

$$\mathit{y}=\mathit{W}\mathit{\theta }+\mathit{ϵ},$$

(9)

where $\mathit{W}$ = $\mathit{X}\mathit{Q}$ is a modified design matrix and $\theta$ represents the vector of the parameters in the transformed space. The matrix $\mathit{Q}$ is an orthogonal matrix obtained from the eigenvectors of $\complement$, satisfying ${\mathit{Q}}^{\prime }\mathit{Q}={\mathit{I}}_p$, where ${\mathit{I}}_p$ is the $p\times p$identity matrix. The transformation $\mathit{W}$ = $\mathit{X}\mathit{Q}$ adjusts the design matrix with the principal components of $X$, simplifying the structure for the regression problem.

$$\mathit{\Lambda }={\mathit{Q}}^{\prime }\complement \mathit{Q},$$

(10)

Here, $\mathit{\Lambda }$ is a diagonal matrix containing the eigenvalues ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_p$, arranged in increasing order. The canonical parameters are related to the original parameters of the models by $\mathit{\theta }=\mathit{Q}\prime \mathit{\phi }$, enabling the model to function in the canonical space. The OLS estimator in Equation (2) can be written in canonical form as

$$\widehat{\mathit{\theta }}={\mathit{\Lambda }}^{-\mathbf{1}}{\mathit{W}}^{\prime }\mathit{y},$$

(11)

The ridge estimate in Equation (6) can be expressed in canonical form as

$${\widehat{\mathit{\theta }}}_{\mathit{k}}={\left(\mathit{\Lambda }+k{\mathit{I}}_p\right)}^{-1}{\mathit{W}}^{\prime }\mathit{y},$$

(12)

which stabilizes the solution by adding $k$ to the diagonal elements of Λ, effectively shrinking the contributions of smaller eigenvalues and enhancing numerical stability, or $K=Diag\left(k_1,k_2,\dots ,k_p\right),k_i>0fori=1,2,\dots ,p.$ A two-parameter ridge estimator in Equation (7) can be expressed in canonical form as

$${\widehat{\mathit{\theta }}}_{(\mathit{d},\mathit{k})}={\mathit{d}\left(\mathit{\Lambda }+k{\mathit{I}}_p\right)}^{-1}{\mathit{W}}^{\prime }\mathit{y},$$

(13)

The MSEs of these estimators are given below in Equations (14)–(16):

$$MSE\left({\widehat{\theta }}_{OLS}\right)={\sum }_{i=1}^p\left({\displaystyle \frac{{\widehat{\sigma }}^2}{{\lambda }_i}}\right),$$

(14)

$$MSE\left({\widehat{\theta }}_K\right)={\widehat{\sigma }}^2{\sum }_{i=1}^p{\displaystyle \frac{{\lambda }_i}{({\lambda }_i+k{)}^2}}+{\sum }_{i=1}^p{\displaystyle \frac{k^2{\theta }_{ols}^2}{({\lambda }_i+k{)}^2}},$$

(15)

$$MSE\left({\widehat{\theta }}_{q,k}\right)=d^2{\sum }_{i=1}^p\left({\displaystyle \frac{{{\lambda }_i\widehat{\sigma }}^2}{{\left({\lambda }_i+k\right)}^2}}\right)+{\sum }_{i=1}^p{\left({\displaystyle \frac{d{\lambda }_i}{{\lambda }_i+k}}-1\right)}^2{{\widehat{\theta }}^2}_{OLS}.$$

(16)

where ${\widehat{\sigma }}^2$ represents is the error variance of linear regression model Equation (1), $\widehat{\theta }$ is the ith estimated value of parameter $\theta$ and ${\lambda }_i$ is the ith eigenvalue of the matrix $\complement$.

2.1. Existing Ridge Estimators

Given that ${\widehat{\sigma }}^2$ represents the estimated error variance of the model, ${\lambda }_{max}$denotes the maximum eigenvalue of the $\complement$ matrix and ${\widehat{\theta }}_{Max}=Max({\widehat{\theta }}_1,{\widehat{\theta }}_2,\dots ,{\widehat{\theta }}_p)$ regression estimates coefficient, several established ridge estimators, widely used for addressing multicollinearity in regression analysis, are discussed below.

Hoerl and Kennard [1] introduced the shrinkage parameter to address multicollinearity issues and estimate reliable parameters of the regression model, referred to as the HK estimator in this study. Its mathematical expression is as follows:

$${\widehat{k}}_{HK}={\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\theta }}_{Max}^2}},$$

(17)

Hoerl et al. [20] modified the HK estimator to better handle the multicollinearity issue, referring to it as the HKB estimator, mathematically defined as

$${\widehat{k}}_{HKB}={\displaystyle \frac{p{\widehat{\sigma }}^2}{{\sum }_{i=1}^p{\widehat{\theta }}_i^2}},$$

(18)

Kibria [3] developed three ridge estimators based on the average, Geometric mean, arithmetic mean and median, to enhance the significant collinearity issues of the data. These are mathematically formulated as

$$\begin{array}{c}{\widehat{k}}_{AM}={\displaystyle \frac{1}{p}}{\sum }_{i=1}^p{\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\theta }}_i^2}},\\ {\widehat{k}}_{GM}={\displaystyle \frac{{\widehat{\sigma }}^2}{{\left({\prod }_{i=1}^p{\widehat{\theta }}_i^2\right)}^{\frac{1}{p}}}},\\ {\widehat{k}}_{Med}=\mathrm{M}\mathrm{e}\mathrm{d}\left({\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\theta }}_i^2}}\right).\end{array}$$

(19)

The eigenvalue-based ridge estimator, referred to in this study as the KMS estimator, was explored by [21] and is expressed as

$${\widehat{k}}_{KMS}={\lambda }_{\mathrm{Max}}{\displaystyle \frac{{\sum }_{i=1}^p\left|{\widehat{\theta }}_i\right|}{\left\{\frac{{\widehat{\sigma }}^2}{{\widehat{\theta }}_{Max}^2}\right\}}},\mathrm{where}{\lambda }_{\mathrm{Max}}=Max\left({\lambda }_1,{\lambda }_2,\dots ,{\lambda }_{\mathrm{p}}\right).$$

(20)

Reference [13] introduced the two-parameter ridge estimator technique for highly correlated data; it is referred to as the LC estimator in this research. $k$ is a penalty parameter and $q$is a scale factor. $k$ and $d$ are calculated using Equation (17) and Equation (8), respectively.

Similarly, Toker and Kaçıranlar [14] established the dual-parameter ridge estimator to handle the high collinearity data, in a study referred to as TK estimator, with the optimal values for ${\widehat{d}}_{opt}$ and ${\widehat{k}}_{opt}$ derived as follows:

$${\widehat{d}}_{opt}={\displaystyle \frac{{\sum }_{i=1}^p{\widehat{\theta }}_i^2\frac{{\lambda }_i}{{\lambda }_i+k}}{{\sum }_{i=1}^p\frac{{\widehat{\theta }}^2{\lambda }_i+{\widehat{\gamma }}_i^2{\lambda }_i^2}{{\left({\lambda }_i+k\right)}^2}}},$$

(21)

$$\begin{array}{c}{\widehat{k}}_{opt}={\displaystyle \frac{{\widehat{q}}_{opt}{\sum }_{i=1}^p\frac{{\widehat{\sigma }}^2}{{\lambda }_i}+\left({\widehat{q}}_{opt}-1\right){\sum }_{i=1}^p{\widehat{\theta }}_i^2{\lambda }_i^2}{{\sum }_{i=1}^p{\widehat{\theta }}_i^2{\lambda }_i}}.\end{array}$$

(22)

Yasin et al. [22] modified two-parameter ridge estimators based on averages, denoted as MTP1, MTP2, and MTP3, with their respective mathematical forms expressed as

$$\begin{array}{c}{\widehat{k}}_{MTPR1}^{*}={\displaystyle \frac{{\sum }_{i=1}^p{k}_i^{*}}{p}},\\ {\widehat{k}}_{MTPR2}^{*}={\left({\prod }_{i=1}^p{k}_i^{*}\right)}^{{\displaystyle \frac{1}{p}}},\\ {\widehat{k}}_{MTPR3}^{*}={\displaystyle \frac{p}{{\sum }_{i=1}^p\frac{1}{k_i^{*}}}}.\end{array}$$

(23)

In these formulations, the modified ridge parameter for the $i\mathrm{th}$ predictor is calculated as

$$k_i^{*}={\displaystyle \frac{{\mathit{\lambda }}_{\mathit{i}}}{\left|{\theta }_i\right|}}{\widehat{k}}_{opt}$$

Equation (23) for $k$-values of the estimators MTPR1, MTPR2, and MTPR3 mentioned above is used to determine $d$ using Equation (8).

Most recently, Akhtar and Alharthi [19] developed three ridge estimators, referred to as CARE1, CARE2, and CARE3, to handle severe multicollinearity issues. They are mathematically defined as follows:

$${\widehat{k}}_{CARET}={\displaystyle \frac{\gamma }{p}}{\sum }_{i=1}^p{\left({\displaystyle \frac{{\lambda }_i\left|{\widehat{\theta }}_i\right|}{\left(1+cond\left(\mathit{C}\right)\right)}}\right)}^{\omega },T=1,2,3$$

i.

Case 1 (CARE1):$\omega =1,r=1\mathrm{a}\mathrm{n}\mathrm{d}\gamma =1$

$${\widehat{k}}_{CARE1}={\displaystyle \frac{1}{p}}{\sum }_{i=1}^p\left({\displaystyle \frac{{\lambda }_i\left|{\widehat{\theta }}_i\right|}{\left(1+cond\left(\mathit{C}\right)\right)}}\right),$$

(24)

ii.

Case 2 (CARE2): $\omega =2,r=1\mathrm{a}\mathrm{n}\mathrm{d}\gamma =2$

$${\widehat{k}}_{CARE2}={\displaystyle \frac{2}{p}}{{\sum }_{i=1}^p\left({\displaystyle \frac{{\lambda }_i\left|{\widehat{\theta }}_i\right|}{\left(1+cond\left(\mathit{C}\right)\right)}}\right)}^2,$$

(25)

iii.

Case 3 (CARE3): $\omega =3,r=2\mathrm{a}\mathrm{n}\mathrm{d}\gamma =1$

$${\widehat{k}}_{CARE3}={\displaystyle \frac{1}{p}}{{\sum }_{i=1}^p\left({\displaystyle \frac{{\lambda }_i^2\left|{\widehat{\theta }}_i\right|}{\left(1+cond\left(\mathit{C}\right)\right)}}\right)}^3$$

(26)

For the second parameter, Equation (8) is used. Among these three ridge estimators, we select Case 1, CARE1, as the competitor estimator in the study. The following existing estimators are used in this study: HK, HKB, KAM, KGM, KMed, KMS, LC, TK, MTPR1, MTPR2, MTPR3, and CARE1. These estimators are compared with our newly proposed estimators.

2.2. Proposed Ridge Estimators

In this section, we modify the condition-adjusted ridge estimators (CAREs) to address their limitations under high multicollinearity and noise conditions. While the CAREs framework effectively adjusts the ridge penalty based on the multicollinearity structure of the design matrix, it does not incorporate any direct mechanism to account for the error variance or the scale of regression coefficients.

To overcome this shortcoming, we propose an improved CARE, denoted as $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2,\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$ and $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$, which augment the original penalty formulation by introducing a variance-stabilizing term of the form $\left({\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\theta }}_{max}^2}}\right)$. This modification allows for more robust shrinkage behavior in severe mutlticollinearity settings, while maintaining adaptability to the eigenvalues structure of the design matrix.

The improved estimators retain the core idea of scaling the ridge penalty by the $\mathrm{C}\mathrm{N}(\complement )$ but now operate within a two-component structure that jointly penalizes based on both multicollinearity and estimation uncertainty. We propose the following multiscale ridge estimators.

$${\widehat{k}}_j={\displaystyle \frac{1}{p}}\left({\sum }_{i=1}^p{\displaystyle \frac{{{\lambda }_i}^a{\left|{\widehat{\theta }}_i\right|}^b}{{\left(1+cond\left(\complement \right)\right)}^f}}+\left({\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\theta }}_{max}^2}}\right)\right)\mathrm{where}j=1,2,3\mathrm{a}\mathrm{n}\mathrm{d}4.$$

(27)

a, b, and f control the degree of shrinkage applied to eigenvalues, coefficient magnitudes, and the condition number, respectively. Their selected values represent different levels of penalization to improve stability under multicollinearity. To address potential arbitrariness, a sensitivity analysis is conducted, showing that the estimators are robust to moderate changes in these parameters.

• ${\lambda }_i$ are the eigenvalues of the matrix.
• ${\widehat{\theta }}_i$ are estimated coefficients.
• ${\widehat{\theta }}_{max}^2$ is the maximum estimated regression coefficient.
• $cond\left(\complement \right)$ is the condition number of the design matrix, measuring collinearity.
• The term ${\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\theta }}_{max}^2}}$ captures estimation uncertainty.
• $a,b,\mathrm{a}\mathrm{n}\mathrm{d}f$ are non-negative exponents that control the influence of each component.

  $${\widehat{k}}_1={\displaystyle \frac{1}{p}}\left({\sum }_{i=1}^p\left({\displaystyle \frac{{\lambda }_i\left|{\widehat{\theta }}_i\right|}{1+cond(\complement )}}\right)+\left({\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\theta }}_{max}^2}}\right)\right),fora=1,b=1\mathrm{a}\mathrm{n}\mathrm{d}f=1.$$

  (28)

  $${\widehat{k}}_2={\displaystyle \frac{1}{p}}\left({\sum }_{i=1}^p{\displaystyle \frac{{{\lambda }_i}^2\left|{\widehat{\theta }}_i\right|}{\left(1+cond\left(\complement \right)\right)}}+\left({\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\theta }}_{max}^2}}\right)\right),fora=2,b=1\mathrm{a}\mathrm{n}\mathrm{d}f=1.$$

  (29)

  $${\widehat{k}}_3={\displaystyle \frac{1}{p}}\left({\sum }_{i=1}^p{\displaystyle \frac{{{\lambda }_i}^6{\left|{\widehat{\theta }}_i\right|}^3}{{\left(1+cond\left(\complement \right)\right)}^3}}+\left({\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\theta }}_{max}^2}}\right)\right),fora=6,b=3\mathrm{a}\mathrm{n}\mathrm{d}f=3.$$

  (30)

  $${\widehat{k}}_4={\displaystyle \frac{2}{p}}\left({\sum }_{i=1}^p{\displaystyle \frac{{{\lambda }_i}^4{\left|{\widehat{\theta }}_i\right|}^4}{{\left(1+cond\left(\complement \right)\right)}^4}}+\left({\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\theta }}_{max}^2}}\right)\right),fora=4,b=4\mathrm{a}\mathrm{n}\mathrm{d}f=4.$$

  (31)

This formulation generalizes the original CARE penalty and leads to the development of four specific multiscale ridge estimators, Equations (28)–(31), and the $d$–value uses Equation (8).

Main Theoretical Results.

2.3. Theoretical Comparison

We now present two key theorems that establish the theoretical properties of the proposed estimators.

Theorem 1 

(Consistency and Performance Under Severe Multicollinearity). Under standard regularity conditions, the proposed estimators ${\widehat{\theta }}_{\left(j\right)prop}$ are consistent. Moreover, when severe multicollinearity is present$(\kappa \ge 30)$ and the error variance is positive$({\sigma }^2>0)$, there exists a threshold κ₀ such that for all κ > κ₀, the proposed estimators achieve a lower MSE than the existing CARE:

$$MSE\left({{\widehat{\theta }}^{\left(j\right)}}_{prop}\right)<MSE\left({\widehat{\theta }}_{CARE}\right).$$

Proof. 

As the sample size $n$ increases, the eigenvalues ${\lambda }_i$ grow proportionally to $n$. From the construction of the proposed estimators in Equations (28)–(31), the ridge parameter $k_{prop}^j$ remains bounded (i.e., O (1)), because ${\displaystyle \frac{{\sigma }^2}{{\theta }_{max}^2}}$ converges to a finite constant, while the term involving ${\displaystyle \frac{{\lambda }_i\left|{\theta }_i\right|}{1+\kappa }}$ diverges. Consequently, ${\displaystyle \frac{k_{prop}^j}{{\lambda }_i+k_{prop}^j}}\to 0.$ Additionally, from Equation (8), the scaling parameter $d_{prop}$ converges to 1, see [23]. Therefore, ${\widehat{\theta }}_{prop}^j$ converges to the OLS estimator ${\widehat{\theta }}_{OLS}$, which is known to be consistent. □

Under severe multicollinearity, the smallest eigenvalue ${\lambda }_{min}$ approaches zero. For the CARE proposed by Akhtar and Alharthi [19], the ridge parameter is $k_{CARE}={\displaystyle \frac{1}{p}}.{\displaystyle \frac{{\Sigma }_{\left\{i=1\right\}}^p{\lambda }_i\left|{\theta }_i\right|}{1+\kappa }}$, which tends to zero as ${\lambda }_{\mathrm{m}\mathrm{i}\mathrm{n}}\to 0$. In contrast, the proposed estimators include the additional term ${\displaystyle \frac{{\sigma }^2}{{\theta }_{max}^2}}>0$, ensuring that $k_{prop}^j>k_{CARE}$ for sufficiently large condition numbers. Following the classic result of [1], the derivative ${\displaystyle \frac{\partial MSE}{\partial k}}$ is negative for small values of k when multicollinearity is present. The reduction in variance is of order $O\left({\displaystyle \frac{1}{{\lambda }_{min}}}\right)$, while the increase in squared bias is only of order $O\left(k^2\right)$. For sufficiently small ${\lambda }_{min},$ the variance reduction outweighs the bias increase, leading to a net decrease in MSE [3,24].

Theorem 2 

(Condition for Superiority Over Existing Estimators). Let $k_{prop}^{j}and{d}_{prop}^j$ denote the parameters for $MSR{E}_j$, and let $k_{exist}$ and $d_{exist}$ denote the parameters for any competing two-parameter ridge estimator. Then $MSR{E}_j$ yields a smaller MSE than the competing estimator if the following inequality holds:

The difference between the MSEs of $MSR{E}_j>0$. This means the proposed estimator has a smaller MSE when the expression involving the parameters of both estimators satisfies this inequality.

Proof. 

The mean squared error difference between the two estimators is obtained by subtracting their respective MSE expressions derived from Equation (16). When this difference is positive, the proposed estimator has a smaller MSE. The inequality above directly represents this condition. In the specific case of comparing with CARE under severe multicollinearity, Theorem 2.2 guarantees that$k_{prop}^j>k_{CARE}$ and typically $d_{prop}^j<1$, which satisfies the required inequality. □

2.3.1. Asymptotic Properties

Under the assumptions that $\stackrel{limit}{n}$→∞${\displaystyle \frac{1}{n}}X\prime X=\Sigma$, where $\Sigma$ is a positive definite matrix, and ${ϵ}_i\sim i.i.d.\left(0,{\sigma }^2\right)$, the proposed estimators ${\widehat{\theta }}_{prop\left(j\right)}$ are consistent estimators of θ as n → ∞.

Proof. 

As n → ∞, the eigenvalues ${\lambda }_i$ grow proportionally to n. Consequently, ${\lambda }_i\to \infty$ for each $i$. The proposed ridge parameter $k_{prop\left(j\right)}$ from Equations (28)–(31) satisfies $k_{prop\left(j\right)}=O\left(1\right)$ because ${\displaystyle \frac{{\sigma }^2}{{\theta }_{max}^2}}>0$ converges to a finite constant by the consistency of OLS estimators, while ${\displaystyle \frac{{\lambda }_i\left|{\widehat{\theta }}_i\right|}{\left(1+cond\left(X^{\prime }X\right)\right)}}\to \infty .$ Thus, ${\displaystyle \frac{k_{prop\left(j\right)}}{\left({\lambda }_i+k_{prop\left(j\right)}\right)}}\to 0.$ Moreover, from Equation (8), $d_{prop}\to 1$as $n\to \infty$ (see [23], Lemma 1). □

Therefore, ${\widehat{\theta }}_{prop\left(j\right)}\to {\left(\Lambda \right)}^{-1}{W}^{\prime }y={\widehat{\theta }}_{OLS}$, which is consistent [25,26].

Discussion of Theoretical Findings

The theoretical results presented above lead to several important observations about the proposed estimators:

When the variance-stabilizing term ${\displaystyle \frac{{\sigma }^2}{{\theta }_{max}^2}}$ is set to zero, the proposed estimators simplify to the CARE introduced by [19]. This demonstrates that our approach serves as a natural extension of existing methodology, rather than a completely unrelated contribution.

A key limitation of many existing ridge estimators is that they can produce shrinkage parameters arbitrarily close to zero under certain conditions, effectively reverting to the unstable OLS estimator. By incorporating ${\displaystyle \frac{{\sigma }^2}{{\theta }_{max}^2}}$, our proposed estimators ensure that $k>0$ in all practical scenarios, providing consistent stabilization.

The flexibility to choose the exponents $a,b,\mathrm{a}\mathrm{n}\mathrm{d}f$allows the estimator to adapt to varying degrees of multicollinearity. The four specific estimators proposed (MSRE₁ through MSRE₄) represent a spectrum of shrinkage intensities. MSRE₁ and MSRE₂ offer moderate penalization suitable for mild to moderate collinearity, while MSRE₃ and MSRE₄ provide more aggressive shrinkage designed for severe multicollinearity scenarios [1].

Theorem 2.1 establishes that under conditions of severe multicollinearity with non-negligible error variance, precisely the situation where standard ridge estimators struggle, the proposed estimators theoretically outperform the existing CARE. This theoretical finding will be empirically validated in the simulation studies and real-world applications presented in subsequent sections.

Remark 1. 

The specific exponent choices for the four proposed estimators are (a, b, f) = (1, 1, 1) for MSRE₁, (2, 1, 1) for MSRE₂, (6, 3, 3) for MSRE₃, and (4, 4, 4) for MSRE₄. These values were selected to represent a range of shrinkage intensities, with higher exponents providing stronger penalization suitable for more severe multicollinearity conditions.

By doing so, we not only improve the CAREs but also provide a structured classification of enhanced ridge estimators that better handle multicollinearity and error variance structures. These estimators are compared based on Monte Carlo simulation, with details provided in Section 3.

3. Monte Carlo Simulation Design

Generating predictor variables under organized multicollinearity settings are produced as follows. Predictors $x_{ij}$ are simulated using Equation (32); many researchers, see references [17,27], adopted this method for generation of the predictor variables.

$$x_{ij}={\left(1-{\rho }^2\right)}^{0.5}\hspace{0.17em}{w}_{ji}+\rho \hspace{0.17em}{w}_{jp+1},j=\mathrm{1,2},...,n\mathrm{a}\mathrm{n}\mathrm{d}i=1,2,...,p.$$

(32)

Here, $\left(\rho \right)$ denotes the correlation between predictors, $w_{ji}$ represents random samples of predictors drawn from a standard normal distribution $N(0,I),p$is the total number of predictors, and n is the sample size. To evaluate various levels of correlation values (0.80, 0.90 and 0.99), $n=20,50,100$ and $p=4,10$ are chosen to evaluate the model across different scenarios. Mathematically, the model is expressed as

$$y_j={\theta }_0+{\sum }_{i=1}^p{\theta }_i{x}_{ji}+{\mathsf{ϵ}}_j,j=1,2,...,n$$

(33)

The regression model parameters (${\theta }_i)$ are calculated by determining the optimal direction, following the methodology outlined by [28]. The error term $({\mathsf{ϵ}}_j$) is the model with mean zero and different values error variance ($\sigma )$. Three levels of error variance were considered ($\sigma =0.5,6,12)$. On this basis, the method ensures that parameters of the model are oriented to minimize errors and enhance the model’s predictive accuracy.

3.1. Estimated Mean Squared Errors

The estimators are biased; therefore, we measure their effectiveness based on estimated MSEs, calculated over $N=2500$ replications using the following formula:

$$\mathit{MSE}\left(\widehat{\mathit{\theta }}\right)={\displaystyle \frac{\mathbf{1}}{\mathit{N}}}{\sum }_{\mathit{i}=1}^{\mathit{N}}({\widehat{\mathit{\theta }}}_{\mathit{i}}-\mathit{\theta }{)}^{\prime }\left({\widehat{\mathit{\theta }}}_{\mathit{i}}-\mathit{\theta }\right)$$

(34)

Evaluating the performance of existing and proposed estimators is challenging, so we compute the estimated MSEs using specific algorithms and formulas (10) to (31). Accuracy is based on the MSE criterion.

The performance of the existing and proposed estimators is difficult to evaluate theoretically; therefore, Equations (10)–(31) were used alongside a specific algorithm to estimate MSEs. The accuracy of the estimators was assessed based on the MSE criterion. All the simulation analyses were carried out using the R programming language. These results, summarized in

Table A1. MSE of the estimators for $n=20\mathrm{a}\mathrm{n}\mathrm{d}p=4$.

$\mathit{\sigma }\to$		0.5			6			12
$\mathit{\rho }\to$ $\mathbf{Methods}\downarrow$	0.80	0.90	0.99	0.80	0.90	0.99	0.80	0.90	0.99
OLS	0.1769	0.3221	2.7819	26.1195	49.8266	399.484	104.374	182.944	1731.48
HK	0.1539	0.2103	1.1738	9.07	18.3251	139.929	36.0016	62.8926	616.844
HKB	0.1065	0.1638	0.7416	6.8351	12.1117	95.6315	31.7434	49.6878	407.248
KAM	0.172	0.3088	2.4843	22.5191	43.1769	342.98	89.9829	156.568	1494.36
KGM	0.0834	0.1119	0.5243	3.1169	4.2833	16.9778	10.6242	14.1121	53.0114
KMed	0.1139	0.1312	0.6241	3.2379	4.7485	32.2391	11.1119	16.1339	159.762
KMS	0.139	0.2203	1.472	19.6172	39.3612	353.00	87.9476	156.44	1622.99
LC	0.0167	0.0246	0.159	3.2979	2.0073	1.8012	14.4665	12.3457	6.4945
TK	1.4383	0.9848	0.1367	11.1127	14.112	22.0286	40.6555	48.5053	93.2063
MTPR1	0.0048	0.0471	0.0036	2.931	1.9616	5.7296	14.41	13.846	7.3822
MTPR2	0.0061	0.0094	0.0163	4.8687	6.52	59.6	26.435	37.722	399.51
MTPR3	0.0059	0.0916	0.9428	8.7454	18.721	200.96	43.854	81.009	1086.2
CARE1	0.0058	0.0903	0.9419	7.6701	14.7023	47.920	23.9034	26.001	44.289
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_1$	0.0578	0.0803	0.0921	6.714	9.9105	46.355	21.206	27.345	225.76
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2$	0.0058	0.0043	0.0035	2.4764	2.1117	7.2401	11.739	11.717	43.878
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	0.0052	0.0039	0.0034	2.0777	1.2458	3.2572	10.622	10.045	13.067
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$	0.1589	0.2799	1.9162	3.5673	4.2654	22.598	13.724	13.271	69.631

The bold values in the table, represent the minimum MSE.

,

Table A2. MSE of the estimators for $n=50\mathrm{and}p=4$.

$\mathit{\sigma }\to$		0.5			6			12
$\mathit{\rho }\to$ $\mathbf{Methods}\downarrow$	0.80	0.90	0.99	0.80	0.90	0.99	0.80	0.90	0.99
OLS	0.0775	0.1575	1.9201	10.6045	22.4686	275.909	43.1312	95.3144	986.359
HK	0.0725	0.1359	0.8375	3.4632	6.2104	89.5707	15.283	32.4084	296.546
HKB	0.056	0.095	0.5144	2.7285	5.3718	65.0352	12.1089	23.501	220.437
KAM	0.0763	0.1528	1.7056	9.0922	18.9141	232.746	36.9021	80.6004	813.955
KGM	0.0481	0.0516	0.2853	1.2817	1.9347	12.9058	4.0806	5.304	25.8388
KMed	0.0867	0.0631	0.3358	1.2889	1.7786	19.2237	3.8003	6.0823	60.3166
KMS	0.0678	0.1183	0.9843	7.222	15.8038	237.108	34.2391	77.449	889.259
LC	0.0059	0.0092	0.0662	0.9297	0.664	0.3448	5.6203	2.9341	4.093
TK	0.8342	0.4584	0.1528	4.7499	4.3375	3.4632	16.2169	16.2299	27.858
MTPR1	0.0023	0.0016	0.0015	0.5671	0.3518	0.2218	4.5145	2.6214	4.0617
MTPR2	0.003	0.0017	0.0039	0.7538	0.7569	7.4632	5.955	5.4653	54.012
MTPR3	0.0022	0.0016	0.2127	1.3773	2.5604	82.844	10.7	18.375	351.71
CARE1	0.02816	0.0340	0.0415	1.6920	2.4103	9.6716	8.5602	8.6012	52.012
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_1$	0.027	0.0311	0.0404	1.6906	2.4024	9.6718	8.558	8.5672	53.478
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2$	0.0025	0.0016	0.0014	0.455	0.3194	0.2873	4.4304	2.5429	12.326
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	0.0024	0.0015	0.0014	0.4155	0.2611	0.2133	3.9698	2.1895	5.3441
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$	0.0734	0.1435	1.3441	0.6405	0.6935	4.932	5.1585	3.1194	17.907

The bold values in the table, represent the minimum MSE.

,

Table A3. MSE of the estimators for$n=100\mathrm{and}p=4$.

$\mathit{\sigma }\to$		0.5			6			12
$\mathit{\rho }\to$ $\mathbf{Methods}\downarrow$	0.80	0.90	0.99	0.80	0.90	0.99	0.80	0.90	0.99
OLS	0.0383	0.0734	0.7011	5.133	10.2144	101.157	21.3262	44.1344	402.998
HK	0.0372	0.1136	0.4352	1.7544	3.5139	33.4168	7.0408	13.6317	132.064
HKB	0.0334	0.0536	0.2338	1.5558	2.7138	23.3838	5.6606	10.8235	85.2707
KAM	0.038	0.0724	0.6519	4.5163	8.8248	85.5224	18.4656	37.9827	338.645
KGM	0.0306	0.0336	0.1649	0.8421	1.4113	6.6591	2.3806	3.6634	16.1988
KMed	0.0704	0.0482	0.1771	0.8509	1.3218	7.8902	2.2717	3.7109	28.3569
KMS	0.0358	0.0637	0.3796	3.2986	6.6927	81.1235	15.8159	34.3292	351.584
LC	0.0037	0.0045	0.0389	0.4329	0.4793	0.2296	2.1459	1.6985	0.5414
TK	0.7358	0.481	0.1092	3.013	2.5288	4.2615	8.1691	8.8204	6.4584
MTPR1	0.0015	0.0062	0.0008	0.1787	0.1583	0.165	1.3855	1.2161	0.4298
MTPR2	0.0014	0.0012	0.0022	0.2316	0.3021	1.8933	1.6544	1.7275	5.1174
MTPR3	0.0009	0.0028	0.0453	0.4212	0.6961	12.75	2.2898	3.6416	72.161
CARE1	0.0140	0.1478	0.0162	0.4411	0.8011	1.6701	4.029	3.920	8.0198
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_1$	0.0130	0.0138	0.0168	0.6421	0.8106	1.5061	3.0297	3.485	7.4386
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2$	0.0011	0.0008	0.0007	0.1424	0.1207	0.0835	1.4284	1.1892	0.498
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	0.0011	0.0008	0.0007	0.1391	0.1186	0.0832	1.3427	1.031	0.408
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$	0.0366	0.069	0.5499	0.2144	0.2191	0.7068	1.6012	1.3485	0.8862

The bold values in the table, represent the minimum MSE.

,

Table A4. MSE of the estimators for $n=20\mathrm{and}p=10$.

$\mathit{\sigma }\to$		0.5			6			12
$\mathit{\rho }\to$ $\mathbf{Methods}\downarrow$	0.80	0.90	0.99	0.80	0.90	0.99	0.80	0.90	0.99
OLS	1.1538	2.6227	31.972	159.889	378.909	4775.58	598.242	1457.87	17856.2
HK	0.7038	1.208	11.3061	61.4178	141.157	1626.53	220.418	543.017	6031.92
HKB	0.3421	0.5507	5.5382	25.9338	65.1459	797.847	110.41	235.708	2703.19
KAM	1.1177	2.5037	29.7906	150.55	354.204	4436.91	562.452	1361.58	16549.9
KGM	0.0934	0.1472	1.251	6.1264	10.8812	106.707	20.4898	37.5026	356.483
KMed	0.0993	0.1682	1.5875	7.0863	12.7194	153.868	26.7882	46.7362	611.17
KMS	0.6324	1.3246	20.1563	132.834	317.523	4416.04	532.639	1314.13	17061.8
LC	0.0063	0.0078	0.0462	1.8906	1.0329	0.425	14.1373	7.5404	2.8092
TK	0.728	0.1907	0.8059	30.6342	27.9542	71.8887	106.564	8.8204	6.4584
MTPR1	0.0031	0.0019	0.0015	2.6387	1.096	4.306	20.888	1.2161	0.4298
MTPR2	0.0041	0.0033	0.0332	10.513	18.675	387.17	67.36	1.7275	5.1174
MTPR3	0.1573	0.3778	22.182	50.188	134.99	2928.4	209.95	3.6416	72.161
CARE1	0.3590	0.5503	1.5021	33.998	66.129	300.01	111.034	3.8203	7.980
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_1$	0.3461	0.5256	1.4733	34.79	63.946	321.57	106.27	3.485	7.4386
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2$	0.007	0.0037	0.0015	2.7747	2.6795	5.8748	19.767	1.1892	0.498
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	0.003	0.0018	0.0011	1.1383	0.5808	0.1819	11.325	1.031	0.408
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$	0.9143	1.9703	22.625	13.087	24.154	276.61	34.177	1.3485	0.8862

The bold values in the table, represent the minimum MSE.

,

Table A5. MSE of the estimators for $n=50\mathrm{and}p=10$.

$\mathit{\sigma }\to$		0.5			6			12
$\mathit{\rho }\to$ $\mathbf{Methods}\downarrow$	0.80	0.90	0.99	0.80	0.90	0.99	0.80	0.90	0.99
OLS	0.2444	0.5102	5.4106	36.1705	76.0053	752.029	149.405	301.708	3026.61
HK	0.2182	0.2277	2.4521	15.1092	30.0347	289.142	64.8937	127.151	1230.78
HKB	0.1235	0.1991	1.1515	7.0268	14.1168	131.001	30.7631	51.4977	553.235
KAM	0.2423	0.5023	5.2196	34.8483	72.8123	718.292	143.84	289.299	2892.11
KGM	0.036	0.0489	0.3659	2.2569	3.5229	29.4633	7.9342	13.144	92.9034
KMed	0.0405	0.0553	0.5041	2.8977	4.5335	41.7828	10.4263	18.76	164.809
KMS	0.1805	0.3002	2.8598	27.2766	58.4745	662.566	126.427	258.968	2822.74
LC	0.0018	0.0022	0.0156	0.3396	0.2698	0.2742	3.0765	2.5362	0.5005
TK	0.2946	0.8608	0.0537	7.2532	5.9805	3.3952	46.633	51.0523	54.1969
MTPR1	0.00091	0.0008	0.0051	0.2009	0.1576	0.0812	4.0694	2.4974	0.4447
MTPR2	0.0013	0.001	0.0015	0.7545	1.1393	12.806	12.185	18.87	267.09
MTPR3	0.0009	0.0252	0.6391	2.1132	4.936	105.33	31.894	66.136	1165.5
CARE1	0.0940	0.1280	0.3401	4.5021	14.029	40.037	30.098	44.890	217.93
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_1$	0.0931	0.1261	0.2387	8.7615	12.29	39.707	28.909	42.908	214.89
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2$	0.0011	0.0008	0.0006	0.1904	0.1117	0.1012	2.9647	3.2888	6.1599
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	0.0009	0.00081	0.0005	0.1508	0.1015	0.0772	2.1447	1.9709	0.3637
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$	0.2291	0.4497	4.0838	1.318	1.9586	16.761	4.721	5.914	30.741

The bold values in the table, represent the minimum MSE.

and

Table A6. MSE of the estimators for $n=100\mathrm{and}p=10$.

$\mathit{\sigma }\to$		0.5			6			12
$\mathit{\rho }\to$ $\mathbf{Methods}\downarrow$	0.80	0.90	0.99	0.80	0.90	0.99	0.80	0.90	0.99
OLS	0.1222	0.2511	2.8693	16.7981	38.6759	386.167	72.0525	154.871	1552.27
HK	0.1148	0.2182	1.4974	7.2472	16.6853	146.273	29.8029	60.2581	596.186
HKB	0.0747	0.1182	0.6225	3.2581	7.4458	63.6506	13.5338	28.476	270.669
KAM	0.1216	0.2484	2.7723	16.1659	37.0887	367.232	69.3161	148.182	1479.04
KGM	0.0243	0.0256	0.2018	1.2234	2.1992	14.4614	4.1972	7.4097	47.7355
KMed	0.0259	0.0284	0.2679	1.6961	2.8829	20.9089	5.3284	10.2013	79.2249
KMS	0.0993	0.1645	1.4074	11.3654	27.8233	325.093	57.7646	126.62	1411.43
LC	0.001	0.0009	0.0051	0.1434	0.1766	0.1722	0.7771	0.5424	2.1638
TK	0.7343	0.0161	0.0007	2.4258	1.8673	0.9043	20.3499	22.4509	16.1109
MTPR1	0.0006	0.0005	0.0003	0.103	0.0546	0.0359	0.7163	0.3076	2.0594
MTPR2	0.0008	0.0007	0.0004	0.1506	0.1314	1.0195	2.2781	3.2564	38.929
MTPR3	0.0015	0.0008	0.0011	0.2596	0.9088	16.323	7.0097	12.456	250.31
CARE1	0.0570	0.0731	0.1302	4.303	5.930	13.602	12.780	18.401	66.013
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_1$	0.0474	0.0606	0.1129	3.6937	5.7722	13.048	11.155	17.659	67.958
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2$	0.0005	0.0004	0.0002	0.0762	0.0534	0.0359	0.585	0.3852	2.4822
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	0.0004	0.0003	0.0002	0.0712	0.0514	0.0356	0.4803	0.2612	2.1363
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$	0.1176	0.2316	2.1816	0.3941	0.7791	6.436	1.0062	1.4895	6.5394

The bold values in the table, represent the minimum MSE.

in the Appendix A, highlight the lowest estimated MSEs for each ridge estimator, with the lowest values bolded for easy reference in tables. A lower MSE reflects an estimator’s ability to closely approximate the true parameters, indicating higher accuracy and reliability. The simulation results are further discussed in Section 3.2.

We created a summary in

Table 1. Performance Summary of the Estimators.

$\mathit{n}$	$\mathit{\rho }\to$ $\mathit{\sigma }\downarrow$	$\mathit{p}=4$				$\mathit{p}=10$
		0.80	0.90	0.99	0.80	0.90	0.99
20	0.5	MTPR1	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$
	6	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$
	12	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	MSRE1	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$
50	0.5	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$
	6	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$
	12	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	MTRE1	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$
100	0.5	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$
	6	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$
	12	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$

, based on the results in simulation tables, highlighting the performance of various ridge estimators across different scenarios.

In the summary in Table 1, it is clear that our newly modified estimators perform better in all scenarios as compared to other existing methods.

3.2. Simulation Results Analysis

• Effect of Sample Size (n): As the sample size increases, all estimators show a decrease in the estimated MSEs. The newly proposed MSRE estimators, in particular, perform well across all sample sizes, often surpassing traditional OLS and other existing methods, especially for larger n values.
• Effect of Number of Predictors $\left(p\right)$: When the number of predictors increases, MSEs generally rise. However, MSRE estimators remain more stable than OLS and other methods as $p$ grows, showing better resilience in high-dimensional settings.
• Effect of Multicollinearity $\left(\rho \right)$: High multicollinearity $(\rho =0.99$) significantly impacts OLS, leading to a higher MSE. MSRE estimators, particularly $MSR{E}_3$, handle multicollinearity more effectively, providing lower MSE compared to OLS and other methods.
• Effect of Error Variance $\left(\sigma \right)$: As error variance increases, estimated MSEs rise for all estimators. the newly proposed MSRE estimators are less sensitive to larger error variances, maintaining superior performance compared to other methods, especially in higher error variance scenarios $(\sigma =12)$.

4. Practical Applications

In this section, we utilized three datasets of a similar nature to the simulation to examine performance of our proposed estimators.

(a)

The first dataset, the Economic Report of the President Dataset [29], was accessed via the U.S. Government Printing Office and provided key economic indicators.

(b)

The second dataset, the Body Fat Dataset [30], includes detailed anthropometric measurements and is publicly available online.

(c)

The third dataset, the (Automobile Demand dataset) Car Passenger Data, was sourced from the U.S. Dept. of Commerce 1986 [31].

4.1. Economic Dataset

The Economic Indicators Dataset includes the dependent variable $Y$ (outstanding mortgage debt in trillions), and three other independent variables, $X_1$ (personal consumption in trillions), $X_2$ (personal income in trillions), and $X_3$ (consumer credit in trillions), covering the 1990–2006 period. The high correlations make it ideal for studying multicollinearity and econometric performance. Therefore, the regression line model for this dataset is

$$Y={\theta }_0+{\theta }_1{X}_1+{\theta }_2{X}_2+{\theta }_3{X}_3+\mathsf{ϵ}.$$

(35)

The eigenvalues of the dataset are ${\lambda }_1=2.94$, ${\lambda }_2=0.058$, and ${\lambda }_3=0.0017$, confirming significant multicollinearity issues in data examine through the CN and VIF. The CN is based on the ratio of maximum and minimum eigenvalues of the data, so CN is approximately 41.59, indicating a significant multicollinearity issue in the data. The calculated VIF values are $79.90,521.82$ and $919.65,$ corresponding to $R^2$ values of $0.9875,0.99808\mathrm{a}\mathrm{n}\mathrm{d}0.99891$, all VIF values greater than $10.$ Furthermore, Figure 1 graphically illustrates the highly correlated nature of the dataset.

The results in

Table 2. MSEs and estimated parameters of the models for economic indicator data.

Methods	$\mathbf{M}\mathbf{S}\mathbf{E}$	${\widehat{\mathit{\theta }}}_0$	${\widehat{\mathit{\theta }}}_1$	${\widehat{\mathit{\theta }}}_2$	${\widehat{\mathit{\theta }}}_3$
$\mathrm{O}\mathrm{L}\mathrm{S}$	2.59541	−0.49885	−0.16383	−0.49901	−0.10491
$\mathrm{H}\mathrm{K}$	2.37807	−0.47585	0.028385	−0.01013	0.014182
$\mathrm{H}\mathrm{K}\mathrm{B}$	2.48442	−0.30751	−0.49832	−0.00057	−0.50005
$\mathrm{K}\mathrm{A}\mathrm{M}$	2.57722	0.166985	−0.44789	5.52 × 10−5	−0.38898
$\mathrm{K}\mathrm{G}\mathrm{M}$	2.46806	−0.49633	−0.17819	−0.4995	−0.08394
$\mathrm{K}\mathrm{M}\mathrm{e}\mathrm{d}$	2.47841	−0.36642	0.033126	−0.04427	0.01053
$\mathrm{K}\mathrm{M}\mathrm{S}$	2.45759	−0.06876	−0.49808	−0.00265	−0.49957
LC	2.39194	0.008212	−0.43629	0.00026	−0.04941
TK	2.25374	−0.49838	−0.14969	−0.50059	−0.00299
MTPR1	$2.24719$	−0.45073	0.024307	−0.16365	0.000293
MTPR2	2.25046	−0.1866	−0.50023	−0.01305	−0.49898
MTPR3	2.27872	0.036241	−0.36929	0.001317	−0.0081
CARE1	2.428506	−0.44247	−0.49901	0.014182	0.014188
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_1$	2.42847	−0.47419	0.008277	−0.46752	4.39 × 10−5
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2$	2.25114	0.136262	−0.06638	0.076072	−0.47557
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	2.2466	−0.49821	−0.00416	−0.49983	−0.30539
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$	2.591845	−0.44247	0.00041	−0.40961	0.160769

The bold values in the table, represent the estimator with the minimum MSE.

clearly show the superior performance of the proposed MSRE estimators compared to OLS and other methods in terms of the MSE criterion, aligning with the simulation results.

4.2. Model Selection Criteria

To evaluate the performance of the various estimation methods, the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC) are employed. These criteria provide a balanced assessment of model fit and complexity, with lower values indicating a more parsimonious and well-fitting model.

Table 3. AIC and BIC Values for Different Estimators in the Economic Indicators Model.

Methods	AIC	BIC	Methods	AIC	BIC
OLS	24.2129	26.9392	TK	21.8176	24.5439
HK	22.7237	25.4500	MTPR1	21.7632	24.4895
HKB	23.4751	26.2014	MTPR2	21.7904	24.5167
KAM	24.0990	26.8253	MTPR3	21.9978	24.7241
KGM	23.3612	26.0875	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_1$	23.0790	25.8053
KMed	23.4343	26.1606	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2$	21.7955	24.5218
KMS	23.2864	26.0127	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	21.7564	24.4827
LC	22.8257	25.5520	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$	24.1908	26.9171
CARE1	23.16	26.4098

The bold values in table represent the lowest AIC and BIC of the estimators.

presents the AIC and BIC values for each estimator applied to the economic indicators’ dataset.

Table 3 presents the AIC and BIC values for various estimation methods applied to the economic indicator dataset. The methods compared include existing along with newly proposed estimators (MSRE₁ to MSRE₄).

From Table 3, we observe that the newly proposed MSRE estimators (MSRE₁ to MSRE₄) consistently demonstrate lower AIC and BIC values compared to OLS and other methods, suggesting superior performance in terms of model selection criteria. Among these, MSRE₃ has the lowest AIC and BIC values, indicating it may be the most optimal model for this dataset.

4.3. Medical Dataset

This Body Fat dataset (medical data) includes body composition and physical measurement data for 252 individuals. It includes variables such as body fat percentage (Y), density (${\mathrm{X}}_1$), age (${\mathrm{X}}_2$), weight (${\mathrm{X}}_3$), height (${\mathrm{X}}_4$), adiposity (${\mathrm{X}}_5)$, and various circumferential measurements (e.g., neck (${\mathrm{X}}_6$), chest (${\mathrm{X}}_7$), abdomen (${\mathrm{X}}_8)$, hip (${\mathrm{X}}_9)$, thigh (${\mathrm{X}}_{10})$, knee (${\mathrm{X}}_{11})$, ankle (${\mathrm{X}}_{12})$, biceps (${\mathrm{X}}_{13})$, forearm (${\mathrm{X}}_{14})$, and wrist (${\mathrm{X}}_{15})$). The given dataset is suitable for analyzing the relationships between body fat and physical attributes, as well as for developed predictive regression models in health and fitness research. We utilized the following regression model:

$$Y={\theta }_0+{\sum }_{i=1}^{15}{\theta }_i{X}_i+\mathsf{ϵ}$$

(36)

In this analysis, multicollinearity is assessed using CN, eigenvalues, and VIF. Additionally, a heatmap of the correlation matrix in Figure 2 is produced to visually depict strong correlations and provide insights into the multicollinearity structure of the predictors. The eigenvalues for this body fat data ranged from $0.65\mathrm{to}9.77$, with a $\mathrm{C}\mathrm{N}$ value of $1234.89$, which is greater than 30, which shows significant multicollinearity issues in the data. The VIF values varied in the dataset between 2.31 and 62.63; the threshold of VIF, which is 10, also confirmed the presence of high collinearity among the predictors of the dataset. The results of CN and VIFs suggest that there could be numerical instability in regression models due to inflated coefficient variances. To address the severe multicollinearity issue, we used newly proposed and existing ridge estimators, as they help mitigate multicollinearity and enhance the stability and interpretability of the model.

As shown in

Table 4. Estimated MSE for medical Dataset.

Methods	OLS	HK	HKB	KAM	KGM	KMed	KMS	LC
$\mathrm{M}\mathrm{S}\mathrm{E}$	1.46984	1.18140	1.11318	1.45609	1.05897	1.04723	1.10757	0.943481
Methods	TK	MTPR1	MTPR2	MTPR3	CARE1	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_1$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$
MSE	3.074038	0.898925	0.946894	1.000825	1.39279	1.36888	1.026669	0.89886	1.195515

The bold values in table represent the lowest MSE of the estimator.

, the real data validate the simulation results.

4.4. Automobile Demand Dataset

The Automobile demand dataset includes 16 observations, with the dependent variable (Y) representing new car sales (thousands), and independent variables: $X_1$ (CPI for new cars), $X_2$ (overall CPI), $X_3$ (disposable income), $X_4$ (interest rate), and $X_5$ (labor force). A linear regression model that can be used is

$$Y={\theta }_0+{\theta }_1{X}_1+{\theta }_2{X}_2+{\theta }_3{X}_3+{\theta }_4{X}_4+{\theta }_5{X}_5+\mathsf{ϵ},$$

(37)

The eigenvalues of the dataset are 4.2867, 1.3603, 0.3252, 0.0214, 0.0054, and 0.0010, with the very small eigenvalues, particularly 0.0010 and 0.0054, strongly indicating the presence of multicollinearity. The CN (the ratio of the largest eigenvalue to the smallest eigenvalue) is 4125.53, which is greater than 30, confirming significant collinearity in the data. Additionally, the VIF values further support this conclusion, with $Y$ having a VIF of 4.07, while $X_1$, $X_2$, $X_3$, and $X_4$ show extremely high VIF values of 255.27, 602.26, 290.57, and 42.74, respectively, all significantly exceeding the critical threshold of 10.

The analysis in

Table 5. Estimated MSE and parameters of the models of Automobile demand dataset.

Methods	MSE	${\widehat{\mathit{\theta }}}_0$	${\widehat{\mathit{\theta }}}_1$	${\widehat{\mathit{\theta }}}_2$	${\widehat{\mathit{\theta }}}_3$	${\widehat{\mathit{\theta }}}_4$	${\widehat{\mathit{\theta }}}_5$
$\mathrm{O}\mathrm{L}\mathrm{S}$	4.283758	−0.47378	−0.47378	−0.47377	−0.47655	−0.47378	−0.47624
$\mathrm{H}\mathrm{K}$	2.397191	0.130851	0.13085	0.130842	0.042616	0.130852	0.111277
$\mathrm{H}\mathrm{K}\mathrm{B}$	3.566581	0.150425	0.150422	0.150382	0.011799	0.150425	0.07447
$\mathrm{K}\mathrm{A}\mathrm{M}$	4.255817	−0.43255	−0.4324	−0.43067	−0.00224	−0.43246	−0.02437
$\mathrm{K}\mathrm{G}\mathrm{M}$	2.787107	−0.13251	−0.13232	−0.13023	−0.00017	−0.13239	−0.00194
$\mathrm{K}\mathrm{M}\mathrm{e}\mathrm{d}$	2.508995	−1.3665	−1.35631	−1.25367	−0.00034	−1.35992	−0.00393
$\mathrm{K}\mathrm{M}\mathrm{S}$	3.985858	−0.47325	−0.47365	−0.47434	−0.47687	−0.47434	−0.47653
LC	2.364552	0.130387	0.130734	0.130586	0.052334	0.130569	0.042028
TK	2.187984	0.148221	0.149865	0.147919	0.015794	0.147801	0.011578
MTPR1	2.182174	−0.35289	−0.40933	−0.33404	−0.00308	−0.33058	−0.00219
MTPR2	2.184134	−0.0696	−0.10799	−0.06055	−0.00024	−0.05907	−0.00017
MTPR3	2.204982	−0.24165	−0.62985	−0.19158	−0.00047	−0.18431	−0.00034
CARE1	2.353281	0.14401	0.13639	0.093041	0.13403	0.13993	0.11940
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_1$	2.359325	0.130824	0.13059	0.072519	0.121189	0.13058	0.130851
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2$	2.197826	−0.42695	−0.38397	−0.00569	−0.04961	−0.33278	−0.43255
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$	2.182052	−0.12590	−0.08796	−0.00044	−0.00414	−0.0600	−0.13250
$\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$	4.282798	−1.07555	−0.37865	−0.00088	−0.00849	−0.18888	−1.36615

The bold values in table represent the minimum MSE of the estimator.

, also shows the superior performance of the proposed MSRE estimators over OLS and other methods, consistent with the simulation results.

5. Conclusions

This research study introduced four new ridge estimators, named $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_1$, $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_2,\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$ and $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_4$, to tackle the severe challenges of multicollinearity in regression models. The study comprehensively evaluated the performance of various estimators under different scenarios, using Monte Carlo simulations and three real-world datasets to highlight their effectiveness in addressing severe multicollinearity. The newly proposed estimators showed better performance as compared to other existing estimators, particularly $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$, which consistently achieved the minimum MSE across different datasets and scenarios.

In simulation results, $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$ exceled under different levels of multicollinearity, sample sizes, and predictor counts, striking an effective balance between bias and variance. Real-world datasets further validated these findings, where in the Economic Indicators dataset, $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$ outperformed others by minimizing estimation errors even under severe multicollinearity; in the Body Fat dataset, it showed resilience against high condition numbers and variance inflation factors, delivering the best MSE; and in the Automobile Demand dataset, it achieved the lowest MSE, proving its robustness and reliability in practical applications.

Overall, the adaptive regularization techniques employed in the proposed estimators provide a clear advantage over traditional ridge regression methods. $\mathrm{M}\mathrm{S}\mathrm{R}{\mathrm{E}}_3$, in particular, stood out as a highly effective and versatile estimator for managing multicollinearity, reducing estimation errors, and improving predictive accuracy in both simulated and real-world settings.

Future research could focus on extending the applicability of the newly proposed estimators to dynamic systems, non-linear models, and high-dimensional datasets, while also exploring improvements through advanced computational and machine learning techniques.