Newly Improved Two-Parameter Ridge Estimators: A Better Approach for Mitigating Multicollinearity in Regression Analysis

Abstract

This study tackles the common issues of multicollinearity arising in regression models due to high correlations among predictor variables, leading to unreliable coefficient estimates and inflated variances, ultimately affecting the model’s accuracy. To address this issue, we introduce four improved two-parameter ridge estimators, named as MIRE1, MIRE2, MIRE3, and MIRE4, which incorporate innovative adjustments such as logarithmic transformations and customized penalization strategies to enhance estimation efficiency. These biased estimators are evaluated through a comprehensive Monte Carlo simulation using the minimum estimated mean square error (MSE) criterion. Although no single ridge estimator performs optimally under all conditions, our proposed estimators consistently outperform existing estimators in most scenarios. Notably, MIRE2 and MIRE3 emerge as the best-performing estimators across a variety of conditions. Their practical utility is further demonstrated through applications to two real-world datasets. The results of the analysis confirm that the proposed ridge estimators offer a reliable and effective approach for improving estimation precision in regression models, as they consistently yield the lowest MSE compared to other estimators.

1. Introduction

Linear regression is a widely used statistical technique for modeling the relationship between a dependent variable and one or more independent variables. The standard linear regression model is expressed as:

$$y=X\varnothing +ϵ,$$

(1)

where $y=\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_n\end{array}\right]$ represents an $n\times 1$ vector of response observations, $\mathrm{\varnothing }=\left[\begin{array}{c}{\mathrm{\varnothing }}_1\\ {\mathrm{\varnothing }}_2\\ ⋮\\ {\mathrm{\varnothing }}_p\end{array}\right]$ is a $p\times 1$ vector of unknown regression coefficients, $X=\left[\begin{array}{cccc}{x}_{11}& x_{12}& \dots & x_{1p}\\ x_{21}& x_{22}& \dots & x_{2p}\\ ⋮& ⋮& \ddots & ⋮\\ x_{n1}& x_{n2}& \dots & x_{np}\end{array}\right]$ is an $n\times p$ matrix of predictor observations, and $\mathsf{ϵ}=\left[\begin{array}{c}{ϵ}_1\\ {ϵ}_2\\ ⋮\\ {ϵ}_n\end{array}\right]$ is an $n\times 1$ vector of error terms.

The ordinary least squares (OLS) solution for estimating the regression coefficients is given in Equation (2):

$${\widehat{\varnothing }}_{ols}=\left[\begin{array}{c}{\widehat{\varnothing }}_1\\ {\widehat{\varnothing }}_2\\ ⋮\\ {\widehat{\varnothing }}_p\end{array}\right]={\left(X^{\prime }X\right)}^{-1}{X}^{\prime }y.$$

(2)

However, OLS estimates can become unstable in the presence of multicollinearity among predictor variables, leading to inflated variances of regression coefficients [1]. Ridge regression is a robust alternative to address this problem by introducing a penalty term to the OLS method. The ridge regression estimator is defined in Equation (3) as:

$${\widehat{\varnothing }}_{Rd}={\left(X^{\prime }X+kI\right)}^{-1}{X}^{\prime }y.$$

(3)

In this equation, $I$ is the identity of the order p × p and $k$is a positive scalar usually called a ridge parameter. The ridge parameter is the level of shrinkage used on the coefficient estimates and the choice of $k$ is very integral in the determination of the tradeoff between bias and variance.

To enhance ridge regression’s flexibility, a two-parameter ridge estimator has been proposed [2]. This generalized form introduces a scaling factor $q$ and is expressed in Equation (4) as:

$${\widehat{\varnothing }}_{\left(q,k\right)}={q\left(X^{\prime }X+kI\right)}^{-1}{X}^{\prime }y.$$

(4)

Here, $q$ is a scaling factor, and the presence of both $q$ and $k$ gives a more flexible class of solutions to the ridge regression problem. The scaling factor $q$ can be computed as:

$$\widehat{q}={\displaystyle \frac{{\left(X^{\prime }y\right)}^{\prime }{\left(X^{\prime }X+kI\right)}^{-1}{X}^{\prime }y}{{\left(X^{\prime }y\right)}^{\prime }{\left(X^{\prime }X+kI\right)}^{-1}{X}^{\prime }X{\left(X^{\prime }X+kI\right)}^{-1}{X}^{\prime }y}}.$$

(5)

Equation (4) reduces to the conventional OLS and ridge regression estimating equation when $q=1$ and $k=k$ or $k=0$. Therefore, the new idea of the two-parameter ridge estimator is helpful in providing a better solution to the problem of multicollinearity in regression analysis. Ref. [3] provided further information about the cross-sectional analysis of variance and ordinary least square estimator and its development to enhance multicollinearity.

Several studies have explored specific improvements in ridge regression. Ref. [4] conducted a Monte Carlo evaluation of various ridge-type estimators, providing empirical comparisons of their performance. Further, the work of [5] offered refined methods for selecting ridge parameters, essential for optimizing model accuracy. Ref. [6] also evaluated new ridge regression estimators, contributing to the empirical understanding of these methods.

The development of ridge regression has also crossed into specialized fields. For instance, Ref. [7] examined its role in logistic regression, while [8] focused on improved ridge regression estimators for logistic models, offering more robust options in binary classification problems. Ref. [9] reviewed ridge-type estimators, highlighting raise estimators for geometric data perturbation and parameter selection. More recently, Ref. [10] developed new ridge estimators to examine the severe multicollinearity data. The authors of [11] further explored robust ridge estimators, particularly under conditions of multicollinearity and the presence of outliers. These robust methods provide a more reliable estimation framework in real-world datasets.

Modern applications of ridge regression are evident in various fields, including environmental science and econometrics. For example, Ref. [12] developed a carbon price forecasting model based on Lasso regression and optimal integration, blending traditional ridge regression with machine learning techniques for improved forecasting accuracy. Ref. [10] developed two-parameter estimators to address linear regression models with correlated predictors. Additionally, the work of [13] introduced two-parameter ridge estimators, further advancing ridge regression in dealing with correlated predictors.

Although significant progress has been made in the development of ridge estimators, no single ridge estimator has consistently demonstrated optimal performance across varying multicollinearity conditions. Existing estimators often excel in specific scenarios but lack robustness and adaptability when applied to diverse datasets. To fill this gap, this study proposes four novel ridge estimators referred to as MIRE1, MIRE2, MIRE3, and MIRE4. These estimators are rigorously evaluated through comprehensive Monte Carlo simulations and compared to other existing estimators using MSE criterion. The results of the analysis indicate that the proposed estimators offer superior performance, particularly in multicollinear datasets, providing a significant contribution to the ongoing development of robust regression techniques. This work fills a critical gap in the literature by offering a set of estimators that provide more reliable performance across diverse datasets, marking a significant advancement in the development of more flexible and effective regression techniques.

The article is structured as follows. Section 2 discusses ridge regression methodology, introducing the proposed and existing estimators. Section 3 covers the simulation study and results analysis. Section 4 applies the estimators to two real datasets, and Section 5 concludes the study.

2. Statistical Methodology for Ridge Regression

To simplify the model, we can express Equation (1) in its canonical form as:

$$y=Z\phi +ϵ,$$

(6)

where

• $Z=XD$, a transformed design matrix.
• $\phi =D^{\prime }\varnothing$, where $D$ is an orthogonal matrix containing the eigenvectors of $X^{\prime }X$.
• $\varnothing$ is the parameter vector, and ϵ represents the noise.
• $D^{\prime }D=I_p$ ensuring orthogonality, with $I_p$ being the identity matrix.
• The transformation $Z=XD$ aligns $Z$ with the principal components of $X$.

Additionally, we define:

$$\Lambda =D^{\prime }{X}^{\prime }XD.$$

Here, $\Lambda$ is a diagonal matrix of the eigenvalues ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_p$ arranged in ascending order.

Using the canonical form, ridge regression estimators can be rewritten in terms of the transformed parameters $\phi$ instead of $\varnothing$.

The standard OLS estimator in canonical form becomes:

$$\widehat{\phi }={\Lambda }^{-1}{Z}^{\prime }y.$$

(7)

Here, ${\Lambda }^{-1}$ directly scales the projections $Z^{\prime }y$ by the reciprocal of eigenvalues, leading to instability when ${\lambda }_i$ are small.

The ridge regression estimator in its canonical form is given as:

$${\widehat{\phi }}_k={\left(\Lambda +k{I}_p\right)}^{-1}{Z}^{\prime }y.$$

(8)

This formulation ensures stability by adding $k$ to each eigenvalue, effectively shrinking the coefficients corresponding to small eigenvalues.

The two-parameter ridge regression estimator in canonical form is:

$${\widehat{\phi }}_{q,k}=q{\left(\Lambda +k{I}_p\right)}^{-1}{Z}^{\prime }y.$$

(9)

2.1. Ridge Estimators

In this section, we discuss both existing ridge regression techniques and the newly proposed ridge estimators.

2.1.1. Existing Ridge Estimators

Several ridge estimators were introduced in the literature; here, we will consider some of them that are relevant to our study. In the following, ${\widehat{\sigma }}^2$ represents the estimated error variance of the regression model and ${\lambda }_{max}$ denotes the largest eigenvalue of the covariance matrix $X^{\prime }X.$

The authors of [1] introduced the first ridge estimator, referred to as HK. It is defined as:

$${\widehat{k}}_{HK}={\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\phi }}_{Max}^2}}, \mathrm{where} {\widehat{\phi }}_{Max}=Max({\widehat{\phi }}_1 , {\widehat{\phi }}_2 , \dots , {\widehat{\phi }}_p).$$

(10)

Ref. [14] modified the HK estimator by incorporating the number of predictors and the modified version is referred to as the HKB estimator. Its form is:

$${\widehat{k}}_{HKB}={\displaystyle \frac{p {\widehat{\sigma }}^2}{{\sum }_{i=1}^p{\widehat{\phi }}_i^2}}.$$

(11)

Kibria [6] introduced three ridge estimators based on averaging techniques, referred to as KAM (Arithmetic Mean), KGM (Geometric Mean), and KMed (Median). These estimators are defined as follows:

$$\begin{array}{c}\mathrm{K}\mathrm{A}\mathrm{M}={\widehat{k}}_{AM}={\displaystyle \frac{1}{p}}{\sum }_{i=1}^p{\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\phi }}_i^2}},\\ \mathrm{K}\mathrm{G}\mathrm{M}={\widehat{k}}_{GM}={\displaystyle \frac{{\widehat{\sigma }}^2}{{\left({\prod }_{i=1}^p{\widehat{\phi }}_i^2\right)}^{\frac{1}{p}}}},\\ \mathrm{K}\mathrm{M}\mathrm{e}\mathrm{d}={\widehat{k}}_{Med}=\mathrm{M}\mathrm{e}\mathrm{d}\left({\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\phi }}_i^2}}\right).\end{array}$$

(12)

Researchers in [15] focused on eigenvalue-based adjustments to introduce a new estimator referred to as KMS with the following expression:

$${\widehat{k}}_{KMS}={\displaystyle \frac{{\mathsf{\lambda }}_{\max }{\sum }_{i=1}^p\left|{\widehat{\phi }}_i\right|}{\left\{\frac{{\widehat{\sigma }}^2}{{\widehat{\phi }}_{Max}^2}\right\}}}.$$

(13)

In [16], an enhanced ridge regression method was proposed by introducing a two-parameter ridge estimator, termed LC estimator in this study. The ridge parameter $k$ is determined through Equation (10), while the second parameter $q$ is derived using Equation (5). Similarly, the authors of [3] developed another two-parameter ridge estimator, referred to as TK estimator, with optimal values for $q$ and $k$ calculated using Equation (14).

$$\begin{array}{c}{\widehat{q}}_{opt}={\displaystyle \frac{{\sum }_{i=1}^p{\widehat{\phi }}_i^2\frac{{\mathsf{\lambda }}_i}{{\mathsf{\lambda }}_i+k}}{{\sum }_{i=1}^p\frac{{\widehat{\sigma }}^2{\mathsf{\lambda }}_i+{\widehat{\phi }}_i^2{\mathsf{\lambda }}_i^2}{{\left({\mathsf{\lambda }}_i+k\right)}^2}}} ,\\ {\widehat{k}}_{opt}={\displaystyle \frac{{\widehat{q}}_{opt}{\sum }_{i=1}^p\frac{{\widehat{\sigma }}^2}{{\mathsf{\lambda }}_i}+\left({\widehat{q}}_{opt}-1\right) {\sum }_{i=1}^p{\widehat{\phi }}_i^2{\mathsf{\lambda }}_i^2}{{\sum }_{i=1}^p{\widehat{\phi }}_i^2{\mathsf{\lambda }}_i}} .\end{array}$$

(14)

The authors of [17] proposed three modified versions of the two-parameter ridge estimators, referred to as MTPR1, MTPR2, and MTPR3, which are defined as follows:

$$\begin{array}{c}\mathrm{M}\mathrm{T}\mathrm{P}\mathrm{R}1:{\widehat{k}}_{AM}^{*}={\displaystyle \frac{\sum _{i=1}^p{k}_i^{*}}{p}},\\ \mathrm{M}\mathrm{T}\mathrm{P}\mathrm{R}2:{\widehat{k}}_{GM}^{*}={\left(\prod _{i=1}^p{k}_i^{*}\right)}^{{\displaystyle \frac{1}{p}}},\\ \mathrm{M}\mathrm{T}\mathrm{P}\mathrm{R}3:{\widehat{k}}_{HM}^{*}={\displaystyle \frac{p}{\sum _{i=1}^p\frac{1}{k_i^{*}}}}.\end{array}$$

(15)

In each instance above, the adjusted $k$-value for the $i^{th}$ predictor is computed as:

$$k_i^{*}=w_i{\widehat{k}}_{opt},$$

${\widehat{k}}_{opt}$ is the optimal $k$-value determined through an optimization process in these ridge estimation models. The weighted $w_i$ is defined as:

$$w_i={\displaystyle \frac{{\mathit{\lambda }}_{\mathit{i}}}{\left|{\phi }_i\right|}},$$

where $\left|{\phi }_i\right|$is the absolute regression coefficient, and ${\mathit{\lambda }}_{\mathit{i}}$ is its corresponding eigenvalue of $X^{\prime }X.$

The $k\text{-}$values of the estimators mentioned above are used to determine $q$ from Equation (14).

The above competitive ridge estimators, HK, HKB, KAM, KGM, KMed, KMS, LC, TK, MTPR1, MTPR2, and MTPR3, were assessed and compared with our newly proposed ridge estimators, MIRE1, MIRE2, MIRE3, and MIRE4 using simulations and real-life datasets exhibiting severe multicollinearity.

2.1.2. New Proposed Ridge Estimators

We introduce four modified Lipovetsky–Conklin ridge (MLCR) estimators, which integrate the methodologies of [3,16] to determine the optimal values of $q$ and $k$. Given the maximum eigenvalue ${\mathsf{\lambda }}_{max}$, the $k$-values for our four estimators are outlined as follows:

$$\mathrm{M}\mathrm{I}\mathrm{R}\mathrm{E}1:k_1=\left.\left(log\sqrt{\left(1+{\displaystyle \frac{1}{p}}{\sum }_{i=1}^p\left(({\mathsf{\lambda }}_i\left|{\widehat{\phi }}_i\right|{)}^2\right)\right)}\right.\right)\left({\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\phi }}_{max}^2}}\right),$$

(16)

$${\mathrm{M}\mathrm{I}\mathrm{R}\mathrm{E}2:k}_2={\displaystyle \frac{{\mathsf{\lambda }}_{max}}{\sqrt{{\widehat{\phi }}_i}}} \left({\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\phi }}_{max}^2}}\right),$$

(17)

$${\mathrm{M}\mathrm{I}\mathrm{R}\mathrm{E}3:k}_3={\mathsf{\lambda }}_{max} log(1+\left|{\widehat{\phi }}_i\right|),$$

(18)

$${ \mathrm{M}\mathrm{I}\mathrm{R}\mathrm{E}4:k}_4={\displaystyle \frac{{\mathsf{\lambda }}_{max}}{{\left|{\widehat{\phi }}_i\right|}^2}} \left({\displaystyle \frac{{\widehat{\sigma }}^2}{{\widehat{\phi }}_{max}^2}}\right).$$

(19)

Equation (5) is used for calculating the estimated $q$-value. Based on these calculations, our four novel two-parameter ridge estimators, denoted as MIRE1, MIRE2, MIRE3, and MIRE4, can be obtained.

2.1.3. Performance of Estimators Under Mean Square Error

The performance of the proposed estimators was evaluated using the Mean Squared Error (MSE) criterion. This approach has been widely employed in previous studies, including [13,18] to analyze the effectiveness of various estimators. The MSE is represented by the following expression:

$$MSE\left(\widehat{\phi }\right)=E\left[(\widehat{\phi }-\phi {)}^{\prime }\left(\widehat{\phi }-\phi \right)\right]$$

(20)

$$={\displaystyle \frac{1}{p}}{\sum }_{i=1}^p({\widehat{\phi }}_i-\phi {)}^2$$

(21)

Since theoretical comparisons of ridge estimators based on MSE are complex, the ridge estimators, shown in Equation (10) to Equation (19), were evaluated through Monte Carlo simulations as described in the following section.

3. Monte Carlo Simulations

This section outlines the Monte Carlo simulation process used to assess the performance of the proposed ridge estimators in comparison to OLS and other existing ridge estimators.

3.1. Simulation Study

This study investigates ridge regression performance by generating predictor variables under controlled multicollinearity settings. Predictors $x_{ij}$ were simulated using the following equation, a process adopted by many researchers, including [19,20], to generate predictor variables as shown in Equation (22).

$$x_{ij}=\sqrt{1-{\rho }^2}. z_{ji}+\rho z_{jp}, i=\mathrm{1,2},\dots ,p and j=\mathrm{1,2},\dots ,n ,$$

(22)

where $x_{ij}$ represents the $j^{th}$ observation for the $i^{th}$ predictor, $\rho$ denotes the correlation between predictors, $z_{ji}$ are independent random samples drawn from a standard normal distribution, $p$ is the total number of predictors, and $n$ represents the sample size. To examine a range of multicollinearity conditions, $\rho$ was varied across $0.80, 0.90, 0.95, \mathrm{a}\mathrm{n}\mathrm{d} 0.99$. Sample sizes $(n=20, 50, 100)$ and predictor counts $(p=4, 10)$ were selected to assess model robustness across different dimensionalities.

The response variable $y_j$ is generated using the following model:

$$y_j={\phi }_0+{\phi }_1{x}_{j1}+{\phi }_2{x}_{j2}+\dots +{\phi }_p{x}_{jp}+{\mathsf{ϵ}}_j , j=1,2\dots ,n$$

(23)

where ${\phi }_0$ is the intercept (set to zero), ${\phi }_i$ are the regression coefficients chosen based on optimal conditions, and ${\mathsf{ϵ}}_j$ is an error term drawn from a normal distribution with variance ${\sigma }^2$. Error variance was examined at four levels: ${\sigma }^2=0.4, 1, 5, \mathrm{a}\mathrm{n}\mathrm{d} 10$. The regression coefficients ${\phi }_i$ were calculated by identifying the optimal direction, as outlined in the methodology proposed in [21]. Algorithm 1 was used to execute the estimated MSE of the estimators.

Algorithm 1: MSE Computation.

i. Generate and standardize the matrix of independent variables using Equation (22). ii. Calculate eigenvalues $\left({\lambda }_1,\dots ,{\lambda }_p\right)$ and eigenvectors $\left(e_1,\dots ,e_p\right)$ of $X^{\prime }X$, ensuring $\sum _{i=1}^p{\mathsf{\lambda }}_{\mathrm{i}}$ $=p.$ iii. Determine regression coefficients $\phi$ in canonical form using $\phi$ $={\lambda }_{\mathrm{m}\mathrm{a}\mathrm{x}} P$, where $P=\left[e_1,\dots ,e_p\right]$ and ${\lambda }_{\mathrm{m}\mathrm{a}\mathrm{x}}$corresponds to the largest eigenvalue, and generate random error terms based on distributions $N\left(\mathrm{0,1}\right)$. iv. Compute the dependent variable values using Equation (9). v. Calculate OLS and ridge regression estimates for $\phi$ using appropriate equations. vi. Repeat steps i–v for $N$ = 10,000 Monte Carlo iterations. vii. Calculate the MSE for all estimators using:

$MSE\left(\widehat{\phi }\right)={\displaystyle \frac{1}{N}}\sum _{j=1}^N\left[({\widehat{\phi }}_j-\phi {)}^{\prime }\left({\widehat{\phi }}_j-\phi \right)\right]$.  (24)

A simulation study with $N=\mathrm{10,000}$ runs was performed to evaluate the MSE for varying values of $\rho$, $n$, and $p$ using R programming language. The results are summarized in

Table A1. Estimated MSE of ridge estimators for $\mathrm{n}=20 \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{p}=4.$

${\mathsf{\sigma }}^2$	$\mathsf{\rho }$	OLS	HK	HKB	KAM	KGM	KMed	KMS	LC	TK	MTPR1	MTPR2	MTPR3	MIRE1	MIRE2	MIRE3	MIRE4
0.4	0.8	0.1572	0.1391	0.0942	0.1533	0.0667	0.0983	0.1273	0.0106	0.6058	0.0032	0.0041	0.0052	0.1427	0.08493	0.0059	0.0850
	0.90	0.3222	0.2499	0.1526	0.3079	0.0875	0.1119	0.214	0.0157	0.4671	0.0025	0.0034	0.0312	0.271	0.1046	0.004	0.105
	0.95	0.6439	0.4039	0.2297	0.5996	0.1295	0.1609	0.3562	0.0221	0.1966	0.0023	0.0035	0.1336	0.5374	0.1523	0.003	0.1534
	0.99	3.2936	1.1448	0.8946	2.8819	0.5183	0.5673	1.8061	0.1568	0.0776	0.0025	0.0174	2.4151	2.8153	0.1601	0.0022	0.156
1	0.8	0.9356	0.5324	0.3797	0.86	0.2296	0.3076	0.5465	0.0601	1.2584	0.0372	0.0469	0.2099	0.4966	0.0873	0.037	0.0881
	0.90	2.0703	0.9015	0.5898	1.8515	0.3475	0.4476	1.1502	0.0944	0.8275	0.0501	0.0424	0.3625	1.1261	0.118	0.0247	0.1186
	0.95	4.327	1.7076	1.3032	3.8014	0.5758	0.6448	2.5553	0.1582	0.7069	0.0228	0.0437	1.1201	2.0865	0.2916	0.0192	0.2948
	0.99	21.057	7.1375	4.4245	17.7172	1.4835	1.1418	14.280	0.2534	0.0888	0.0143	0.0872	9.2492	10.418	1.4964	0.0141	1.5316
5	0.8	24.988	8.734	6.5376	21.5395	2.6154	2.424	18.987	1.7665	7.6045	1.4834	2.6093	7.4336	1.1347	1.1996	3.1117	1.2293
	0.90	45.362	13.822	9.9473	37.9924	3.0894	2.9781	34.220	1.4089	6.9948	1.1041	2.8158	12.931	0.9607	0.8607	2.9719	0.8687
	0.95	103.87	33.072	26.748	88.044	6.3998	5.9378	84.482	1.0718	8.1453	0.806	6.1301	35.575	2.5594	0.6594	3.1265	0.7386
	0.99	547.51	156.54	133.47	459.806	19.7883	31.0987	485.036	0.5052	8.5894	0.8612	38.161	253.93	16.507	0.6083	4.619	0.8839
10	0.8	92.627	30.609	22.259	78.8172	7.2047	7.7303	77.268	8.4829	25.303	9.0621	14.520	33.249	0.8802	6.6461	15.771	6.7665
	0.90	203.35	71.728	48.052	172.807	11.7479	13.166	174.88	5.2299	29.5044	5.684	23.189	78.229	1.8967	4.3462	15.775	4.2586
	0.95	400.27	130.03	89.160	337.138	18.0546	23.724	352.380	4.5824	23.8706	4.226	36.609	164.97	1.3892	3.7342	15.852	3.7716
	0.99	2043.90	625.91	422.46	1709.18	46.266	99.010	1903.98	1.9432	19.0621	2.5263	177.57	1063.2	5.1256	2.0979	33.136	2.2387

,

Table A2. Estimated MSE of ridge estimators for $\mathrm{n}=50 \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{p}=4.$

${\mathsf{\sigma }}^2$	$\mathsf{\rho }$	OLS	HK	HKB	KAM	KGM	KMed	KMS	LC	TK	MTPR1	MTPR2	MTPR3	MIRE1	MIRE2	MIRE3	MIRE4
0.4	0.8	0.0613	0.0582	0.0459	0.0606	0.036	0.0742	0.0548	0.0044	0.7383	0.0013	0.0015	0.0012	0.0583	0.0298	0.0023	0.0297
	0.90	0.1051	0.128	0.0662	0.1027	0.0366	0.055	0.0835	0.0054	0.5233	0.0053	0.0016	0.023	0.0942	0.0196	0.0015	0.0196
	0.95	0.2209	0.1811	0.1179	0.2126	0.058	0.0663	0.1521	0.0098	0.0701	0.001	0.0014	0.0015	0.1937	0.049	0.0013	0.049
	0.99	1.0631	0.3847	0.3173	0.9627	0.1826	0.2173	0.5356	0.0431	0.3147	0.0011	0.0031	0.1456	0.8702	0.1388	0.0010	0.1386
1	0.8	0.3631	0.2752	0.2028	0.3456	0.1174	0.1444	0.2455	0.0245	1.216	0.0076	0.0086	0.0145	0.1814	0.0203	0.0137	0.0204
	0.90	0.6661	0.2892	0.2778	0.6156	0.1421	0.1747	0.3716	0.0319	0.8869	0.0073	0.0088	0.0385	0.2702	0.0172	0.0090	0.0177
	0.95	1.3517	0.6541	0.4161	1.2133	0.223	0.2686	0.6858	0.0542	0.2726	0.0051	0.0066	0.0354	0.6244	0.021	0.0065	0.0212
	0.99	6.5269	2.1217	1.6195	5.5927	0.7654	0.7354	3.852	0.2009	0.3528	0.0078	0.0191	0.6828	2.9425	0.0581	0.0059	0.058
5	0.8	8.6778	3.3666	2.4639	7.474	1.1708	1.0895	5.7949	0.6348	4.0054	0.3776	0.5037	1.0819	0.3777	0.2789	0.6535	0.2789
	0.90	17.2075	5.9626	3.883	14.610	1.596	1.422	11.9379	0.5488	4.1625	0.3582	0.5687	2.6312	0.4023	0.1815	0.4303	0.1816
	0.95	36.4402	11.605	8.4171	30.926	2.7448	2.1806	27.072	0.5277	4.595	0.3623	0.8003	6.7518	0.7113	0.1698	0.3332	0.1739
	0.99	157.69	42.783	32.004	130.17	6.9136	8.6012	127.97	0.2653	2.6145	0.1386	2.4877	33.586	1.7602	0.1376	0.164	0.1413
10	0.8	36.305	12.388	8.8775	31.032	3.5383	3.3746	28.207	2.9892	10.491	2.158	2.9559	5.699	0.5895	1.7446	4.8858	1.7469
	0.90	66.734	20.649	16.476	56.261	5.3862	5.4657	52.597	2.3396	13.6585	1.6146	4.0175	13.968	0.5393	1.4068	5.6226	1.4041
	0.95	135.87	43.722	28.860	113.15	6.588	7.7221	110.89	1.6307	12.882	1.2294	5.2216	28.835	0.6101	1.199	7.9673	1.1823
	0.99	670.12	219.99	162.79	561.06	21.350	33.903	598.21	0.7724	9.0354	0.7925	23.504	222.15	1.0651	0.6121	7.4994	0.6189

,

Table A3. Estimated MSE of ridge estimators for $\mathrm{n}=100 \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{p}=4.$

${\mathsf{\sigma }}^2$	$\mathsf{\rho }$	OLS	HK	HKB	KAM	KGM	KMed	KMS	LC	TK	MTPR1	MTPR2	MTPR3	MIRE1	MIRE2	MIRE3	MIRE4
0.4	0.8	0.0234	0.1464	0.0213	0.0233	0.0238	0.0756	0.0222	0.002	0.0096	0.0009	0.0009	0.0035	0.0228	0.0029	0.0012	0.0029
	0.90	0.0446	0.1077	0.0364	0.0442	0.0219	0.0364	0.0401	0.003	0.0524	0.0057	0.0009	0.0058	0.0425	0.0025	0.0008	0.0025
	0.95	0.0937	0.0856	0.0612	0.0919	0.0321	0.0348	0.0761	0.0053	0.0218	0.0006	0.0006	0.0007	0.0846	0.0144	0.0005	0.0144
	0.99	0.5002	0.3335	0.1928	0.4677	0.1029	0.1085	0.272	0.0227	0.0021	0.005	0.0005	0.0013	0.3723	0.0158	0.0004	0.0159
1	0.8	0.1463	0.1961	0.0898	0.1427	0.0711	0.1149	0.1179	0.0122	0.2332	0.0104	0.0058	0.0247	0.0831	0.0057	0.0075	0.0057
	0.90	0.2831	0.1885	0.1415	0.2707	0.0866	0.1044	0.1886	0.019	0.6082	0.005	0.0045	0.0111	0.1113	0.0044	0.0049	0.0044
	0.95	0.6037	0.3841	0.2207	0.5635	0.1392	0.1565	0.3497	0.0301	0.1568	0.0028	0.0029	0.0262	0.2622	0.006	0.0035	0.006
	0.99	3.0504	1.1849	0.8127	2.6641	0.4382	0.491	1.6223	0.1266	0.1772	0.0027	0.0042	0.1571	1.0661	0.0084	0.0026	0.0085
5	0.8	3.5238	1.2961	1.0425	3.1094	0.6772	0.7526	2.1524	0.3547	2.58	0.1175	0.1461	0.2589	0.2738	0.1002	0.2082	0.1003
	0.90	7.5212	2.4485	2.1259	6.4997	1.0068	0.9766	4.8189	0.3779	2.0142	0.1116	0.2261	0.5333	0.187	0.103	0.1387	0.1005
	0.95	15.1839	5.1756	3.6938	12.9329	1.5608	1.3638	10.244	0.3951	2.897	0.0707	0.1646	1.2594	0.2062	0.0688	0.0888	0.0689
	0.99	78.481	23.3488	18.7669	66.5133	4.6267	4.3373	62.124	0.2183	2.7013	0.0619	1.0731	11.198	0.6722	0.0618	0.0742	0.062
10	0.8	14.5675	5.1741	4.0204	12.5573	1.768	1.7265	10.3944	1.4403	5.9546	0.7701	0.9579	1.4362	0.6321	0.7298	1.949	0.7299
	0.90	29.7017	9.4508	7.4379	25.1938	2.7975	2.5946	21.787	1.1234	6.487	0.6016	0.9617	2.4428	0.4497	0.5348	1.618	0.5347
	0.95	63.6297	22.2795	15.2964	54.1393	4.6212	4.122	50.1716	0.8506	7.4214	0.544	1.9238	8.5953	0.3272	0.4189	1.0744	0.4188
	0.99	306.267	92.0396	67.4402	256.010	12.3486	19.4741	262.200	0.3972	6.2764	0.3157	4.316	54.201	0.4014	0.2954	1.4603	0.2945

,

Table A4. Estimated MSE of ridge estimators for $\mathrm{n}=20 \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{p}=10.$

${\mathsf{\sigma }}^2$	$\mathsf{\rho }$	OLS	HK	HKB	KAM	KGM	KMed	KMS	LC	TK	MTPR1	MTPR2	MTPR3	MIRE1	MIRE2	MIRE3	MIRE4
0.4	0.8	0.5188	0.3974	0.1895	0.5094	0.0592	0.063	0.3216	0.0042	0.6882	0.0025	0.0028	0.0141	0.4307	0.1427	0.0059	0.1429
	0.90	1.1397	0.705	0.3015	1.1064	0.0719	0.0777	0.5737	0.0034	0.2427	0.0013	0.0019	0.1033	0.9519	0.3046	0.0026	0.305
	0.95	2.4719	1.1873	0.4933	2.3766	0.1416	0.1683	1.1951	0.0048	0.1687	0.0073	0.0063	0.4607	2.1117	0.5925	0.0018	0.5922
	0.99	14.3129	5.7113	2.5931	13.5562	0.5878	0.724	8.3919	0.0165	0.1599	0.0012	0.0125	7.6035	12.372	4.0168	0.0010	4.0173
1	0.8	3.392	1.6948	0.8044	3.2638	0.2466	0.2927	1.9573	0.0271	1.2977	0.0182	0.023	0.2718	2.3038	0.638	0.0365	0.6398
	0.90	7.4156	3.0681	1.3627	7.0706	0.3625	0.4721	4.2623	0.0217	0.8638	0.0226	0.0212	0.8232	3.9614	0.5339	0.0177	0.5307
	0.95	14.4581	5.3748	2.5431	13.6523	0.6696	0.8717	8.4805	0.0374	0.5839	0.0400	0.035	2.2271	7.9719	0.8235	0.0104	0.8234
	0.99	85.8984	33.906	13.119	80.8446	3.0126	3.2705	62.602	0.1218	0.4148	0.0062	0.1897	33.872	46.685	7.0412	0.0061	7.0255
5	0.8	82.1771	34.506	13.697	78.2093	3.9758	4.4819	66.622	1.1106	19.7929	1.6968	5.6176	22.161	5.4694	1.2497	2.391	1.2906
	0.90	191.451	72.198	37.971	182.099	6.9819	7.522	160.060	0.7807	15.954	1.347	8.0289	59.508	16.385	1.4255	1.4916	1.4682
	0.95	383.163	140.504	73.041	362.346	12.5675	13.543	325.028	0.7357	14.259	0.9171	15.572	143.15	31.402	1.3408	0.8325	1.3746
	0.99	2042.02	782.290	354.40	1915.59	50.0066	67.519	1844.85	0.4053	27.236	1.3232	141.61	1250.6	171.50	5.821	0.3751	5.9954
10	0.8	342.561	146.341	59.225	325.671	13.8568	16.363	303.851	9.9257	69.122	15.495	38.581	124.13	6.3369	8.8586	22.912	9.1906
	0.90	744.955	294.663	137.05	706.521	22.5394	28.591	664.986	6.4457	80.670	11.259	68.164	293.66	12.957	6.5774	23.142	6.4253
	0.95	1543.34	592.034	262.43	1450.65	39.7229	51.026	1390.71	4.7824	74.992	6.4008	121.44	696.97	30.188	4.4712	15.456	5.5657
	0.99	8073.69	2801.08	1320.8	7568.42	181.469	278.566	7622.06	3.2748	128.02	22.326	911.42	5150.9	213.32	10.810	30.988	13.419

,

Table A5. Estimated MSE of ridge estimators for $\mathrm{n}=50 \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{p}=10.$

${\mathsf{\sigma }}^2$	$\mathsf{\rho }$	OLS	HK	HKB	KAM	KGM	KMed	KMS	LC	TK	MTPR1	MTPR2	MTPR3	MIRE1	MIRE2	MIRE3	MIRE4
0.4	0.8	0.1611	0.1503	0.0959	0.1602	0.0287	0.0312	0.1288	0.0012	0.191	0.0006	0.0006	0.0005	0.1494	0.0403	0.0013	0.0402
	0.90	0.3537	0.3042	0.152	0.3498	0.0404	0.0436	0.2316	0.0017	0.0039	0.0007	0.0006	0.0007	0.3025	0.0403	0.0008	0.0404
	0.95	0.6517	0.4928	0.2154	0.64	0.0591	0.0662	0.3453	0.0019	0.0012	0.0007	0.0008	0.001	0.5419	0.0575	0.0005	0.0576
	0.99	3.7016	1.894	0.8355	3.5911	0.3009	0.4016	1.8893	0.0119	0.01	0.0004	0.0006	0.3274	3.1682	0.4187	0.0003	0.4192
1	0.8	1.0702	0.7639	0.3371	1.0501	0.1132	0.1458	0.6181	0.0073	0.1767	0.0038	0.004	0.0056	0.5822	0.0221	0.0083	0.0222
	0.90	2.3335	1.3645	0.5599	2.2734	0.1876	0.2494	1.2239	0.01	0.3432	0.0088	0.0155	0.034	1.2303	0.028	0.0047	0.0281
	0.95	4.2913	2.1664	0.8881	4.1556	0.3317	0.4854	2.2131	0.0118	0.2424	0.0027	0.0048	0.0956	2.1603	0.0374	0.0034	0.0378
	0.99	22.2211	9.3209	4.4677	21.3904	1.3866	1.8931	14.4239	0.0562	0.0344	0.0021	0.0215	1.6215	11.968	0.2262	0.0020	0.2242
5	0.8	25.327	11.5173	4.8682	24.4326	1.7771	2.2477	18.1977	0.202	4.3737	0.2121	0.5231	1.3102	1.1649	0.1053	0.2355	0.107
	0.90	52.9484	23.5778	9.889	50.9119	3.0896	3.7735	39.2848	0.2153	2.8284	0.0717	0.4264	2.3203	2.1967	0.0703	0.1167	0.0772
	0.95	115.675	49.8305	23.1694	111.218	6.5548	7.7034	91.8203	0.3263	1.8101	0.0637	0.6447	4.642	4.549	0.0597	0.0754	0.0607
	0.99	523.631	226.732	95.0532	502.203	23.3557	35.6529	454.483	0.1937	2.2201	0.0519	7.0968	59.377	17.29	0.0647	0.0502	0.0681
10	0.8	104.237	43.127	20.2642	100.577	5.6847	7.6251	86.0279	1.8365	27.5209	1.9357	6.0934	14.299	0.7094	1.4182	5.3904	1.4143
	0.90	214.731	93.4446	38.3373	206.632	10.564	14.7109	181.612	1.3328	32.5583	1.4165	11.829	36.447	1.1364	0.8814	5.2433	0.8673
	0.95	434.928	189.459	78.4042	417.634	19.526	28.7565	378.218	0.6434	37.8734	0.5299	27.172	103.65	1.7978	0.3413	2.4605	0.3378
	0.99	2158.95	945.147	457.822	2074.85	82.546	150.378	2001.76	0.3442	25.8402	0.2267	177.57	757.56	11.053	0.2198	7.9883	0.2274

,

Table A6. Estimated MSE of ridge estimators for $\mathrm{n}=100 \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{p}=10.$

${\mathsf{\sigma }}^2$	$\mathsf{\rho }$	OLS	HK	HKB	KAM	KGM	KMed	KMS	LC	TK	MTPR1	MTPR2	MTPR3	MIRE1	MIRE2	MIRE3	MIRE4
0.4	0.8	0.08322	0.07963	0.05694	0.0828	0.01839	0.0208	0.07031	0.00069	0.77834	0.00035	0.00032	0.00038	0.0788	0.01913	0.0006	0.0191
	0.90	0.1874	0.16924	0.09233	0.18596	0.02143	0.0233	0.1321	0.00069	0.0035	0.00026	0.00028	0.00024	0.1665	0.01791	0.0003	0.0179
	0.95	0.3869	0.3164	0.15592	0.38141	0.03616	0.0415	0.22534	0.0013	0.00192	0.00024	0.00022	0.00029	0.3278	0.0231	0.0002	0.02312
	0.99	1.951	1.1231	0.4527	1.8891	0.1551	0.2022	0.9163	0.0043	0.00031	0.00025	0.00022	0.00043	1.6637	0.0922	0.0002	0.09234
1	0.8	0.5436	0.4286	0.228	0.5349	0.0653	0.0717	0.33	0.0039	0.0328	0.0020	0.0018	0.0017	0.2814	0.0038	0.0038	0.0039
	0.90	1.1273	0.7536	0.3536	1.1001	0.1054	0.1294	0.5832	0.0045	0.033	0.0014	0.0015	0.0019	0.6272	0.0082	0.0022	0.0082
	0.95	2.5124	1.3804	0.5669	2.4314	0.1956	0.2561	1.2375	0.0068	0.0257	0.0024	0.0057	0.0156	1.1931	0.0087	0.0015	0.0087
	0.99	12.956	5.478	2.4577	12.4041	0.8515	1.2475	7.6283	0.0234	0.7811	0.0013	0.0089	0.7602	6.1691	0.0285	0.0011	0.0284
5	0.8	14.344	6.3146	2.9493	13.8207	0.9877	1.4118	9.6045	0.0951	1.9611	0.0469	0.0663	0.1611	0.5999	0.0488	0.1002	0.0491
	0.90	28.380	11.5972	5.3825	27.1553	1.7907	2.2452	19.211	0.1168	1.5417	0.0408	0.0911	0.4681	1.7073	0.0333	0.0504	0.0336
	0.95	60.136	25.0217	10.178	57.5029	3.2949	4.1037	44.086	0.167	1.4223	0.0345	0.1586	1.2242	1.8804	0.0339	0.0410	0.0341
	0.99	313.459	122.492	55.550	298.862	12.946	19.024	261.999	0.1712	0.4428	0.0286	0.4559	8.4157	16.182	0.0327	0.0300	0.0331
10	0.8	55.798	23.4185	10.171	53.6356	3.1436	3.9559	43.335	0.4513	11.4542	0.3115	0.7444	3.0202	0.4237	0.1773	0.8335	0.1776
	0.90	112.121	45.6048	20.177	107.133	5.8409	7.5225	88.221	0.3561	7.9432	0.1783	1.1472	4.8744	0.3227	0.1319	0.5234	0.1322
	0.95	244.091	97.5826	39.892	233.215	11.764	15.372	202.59	0.3595	13.4349	0.1174	3.7388	20.4618	0.9118	0.1096	0.3972	0.1100
	0.99	1309.14	560.092	243.801	1251.66	49.395	81.049	1188.6	0.2131	9.0287	0.1051	22.754	219.699	4.7756	0.1048	0.1494	0.1054

and

Table A7. Detailed summary of the performance of ridge estimators.

$\mathit{n}$	${\mathit{\sigma }}^2$	$\mathit{p} = 4$				$\mathit{p} = 10$
		0.80	0.90	0.95	0.99	0.80	0.90	0.95	0.99
20	0.4	MTPR1	MTPR1	MTPR1	MIRE3	MTPR1	MTPR1	MIRE3	MIRE3
	1	MIRE3	MIRE3	MIRE3	MIRE3	MITPR2	MIRE3	MIRE3	MIRE3
	5	MIRE3	MIRE2	MIRE2	MIRE2	LC	LC	LC	MIRE3
	10	MIRE1	MIRE1	MIRE1	MIRE2	MIRE1	MIRE4	MIRE2	LC
50	0.4	MTPR3	MIRE3	MIRE3	MIRE3	MTPR3	MTPR2	MIRE3	MIRE3
	1	MTPR2	MTPR2	MIRE3	MIRE3	MTPR1	MIRE3	MIRE3	MIRE3
	5	MIRE4	MIRE2	MIRE2	MIRE2	MIRE2	MIRE2	MIRE2	MIRE3
	10	MIRE1	MIRE1	MIRE1	MIRE2	MIRE1	MIRE4	MIRE4	MIRE2
100	0.4	MTPR1	MTPR1	MIRE3	MIRE3	MTPR2	MPTR3	MIRE3	MIRE3
	1	MIRE2	MIRE2	MTPR1	MIRE3	MTPR3	MTPR1	MIRE3	MIRE3
	5	MIRE2	MIRE2	MIRE2	MIRE2	MTPR1	MIRE2	MIRE2	MIRE3
	10	MIRE1	MIRE1	MIRE1	MIRE4	MIRE4	MIRE2	MIRE2	MIRE2

in Appendix A, where bolded values indicate the lowest MSE for each estimator, highlighting higher accuracy and improved predictive performance. A detailed analysis and discussion of these results are provided in following section.

3.2. Discussion of the Results

The performance of the newly proposed estimators (MIRE1, MIRE2, MIRE3, and MIRE4), OLS, and other existing estimators was tested using the estimated MSE as the evaluation criterion. The study comprehensively considered varying sample sizes $(n=20, 50, 100)$, numbers of predictors $(p=4, 10)$, levels of correlation $\left(\rho \right)$, and error variances $\left({\sigma }^2\right)$. The simulation findings can be highlighted in the following points.

Proposed Estimators (MIRE1, MIRE2, MIRE3, and MIRE4): Across different scenarios of $n, p, \rho , \mathrm{a}\mathrm{n}\mathrm{d} {\sigma }^2$, the proposed estimators consistently achieved lower MSE values compared to most existing estimators. Among the proposed estimators, MIRE3 often outperformed MIRE1 and MIRE2, particularly in scenarios with higher correlations $(\rho \ge 0.95)$ and larger numbers of predictors$\left(p=10\right)$. This suggests that MIRE3 is better suited to handling multicollinearity in these scenarios.

Existing Estimators: OLS and estimator KAM estimator showed higher MSE values in nearly all scenarios, particularly when $\rho$ and $p$ are large. This highlights their sensitivity to multicollinearity. Other existing estimators such as KMed, KMS, and HK tended to perform better than other existing estimators but were generally outperformed by our proposed MIRE estimators. The performance of the estimators LC, TK, MTPR1, MTPR2, and MTPR3 was heavily dependent on the combination of $n, p$, and ${\sigma }^2$. For instance, lc and tk achieved lower mse in smaller error variances (${\sigma }^2=0.40$); however, they fell short in scenarios with high${\sigma }^2$.

Impact of correlation ($\mathsf{\rho }$): As the correlation increased, the challenge of multicollinearity becomes more pronounced. The proposed estimators demonstrated robustness under high correlations $(\mathsf{\rho }=0.95,0.99)$, maintaining lower mse values compared to other estimators. The ols and hk estimators suffered significantly under high $\mathsf{\rho }$, with their mse values increasing dramatically, particularly when $\mathrm{p}=10$.

Impact of Sample Size ($\mathbf{n}$): Larger sample sizes $(\mathrm{n}=50,100)$ generally led to reduced MSE values across all estimators, reflecting the benefits of increased data availability. For small sample sizes $(\mathrm{n}=20)$, our MIRE estimators remained competitive, with MTPR3 emerging as particularly reliable.

The performance gap between the proposed and existing estimators narrowed with increasing $\mathrm{n}$; however, the proposed estimators maintained their advantage, especially in high-dimensional settings $(\mathrm{p}=10)$.

Impact of Predictors ($\mathbf{p}$): Increasing the number of predictors ($\mathrm{f}\mathrm{o}\mathrm{r} \mathrm{e}\mathrm{x}\mathrm{a}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e} \mathrm{p}=10)$ exacerbated multicollinearity, leading to a noticeable rise in MSE for existing estimators. The proposed estimators showed their strength in handling high-dimensional data, with MIRE3 consistently delivering the lowest MSE among all methods.

Impact of Error Variance (${\mathit{\sigma }}^2$): A smaller error variance (${\sigma }^2=0.40)$ favored all estimators, resulting in a reduced MSE. As ${\sigma }^2$ increased, the gap between the proposed and existing estimators widened, highlighting the robustness of MIRE1, MIRE2, MIRE3, and MIRE4 under high error variances.

The summary of Table A7 (provided in Appendix A) demonstrates the performance of ridge estimators across varying sample sizes (n), error variances ($\sigma$), and multicollinearity levels $\left(\rho \right)$, showcasing that the proposed estimators (MIRE1, MIRE2, MIRE3, and MIRE4) performed best in 30 out of 32 scenarios. They exceled under severe multicollinearity $(\rho >0.95$) and high noise $({\sigma }^2=5, 10)$. For a small sample $(n=20)$, MTPR1 performed well in five cases under low noise, while MIRE3 and MIRE2 dominated in eight scenarios as noise increases. For larger samples $(n=50, 100)$, MIRE3 and MIRE2 achieved the lowest MSE in $15$ scenarios, with MTPR’s estimators excelling in three low-noise cases. These results underline the adaptability and consistency of the proposed estimators, particularly in challenging multicollinear settings.

4. Real-Life Applications

The first dataset we considered is the Blood Pressure dataset, sourced from Applied Regression Analysis [22], which serves as a benchmark in regression analysis. The second dataset, Acetylene Conversion data, was taken from [23] and foundational research by [24]. We utilized these two real datasets to evaluate the performance of the proposed and existing ridge estimators.

4.1. Blood Pressure Patients Dataset

The Blood Pressure (BP) Patients dataset includes 20 patients and seven variables, with BP as the response variable and six predictors, age ($X_1)$, weight ($X_2)$, BSA ($X_3)$, duration ($X_4)$, pulse ($X_5)$, and stress ($X_6$). We fit a linear regression model using BP as the dependent variable and the six predictors as independent variables. The linear regression model can be expressed as:

$$y={\phi }_0+{\phi }_1{X}_1+{\phi }_2{X}_2+{\phi }_3{X}_3+{\phi }_4{X}_4+{\phi }_5{X}_5+{\phi }_6{X}_6+\mathsf{ϵ},$$

where ${\phi }_0$ is the intercept and ${\phi }_1$, ${\phi }_2$, ${\phi }_3$, ${\phi }_4, {\phi }_5, \mathrm{a}\mathrm{n}\mathrm{d} {\phi }_6$ are the coefficients for each predictor, and ϵ represents the error term.

To check for multicollinearity in the Blood Pressure Patients dataset, we use the Condition Number $\left(CN\right)$, Eigenvalues, Variance Inflation Factors (VIFs), and Heatmap pairs wise correlation display. A $\mathrm{C}\mathrm{N}$ above 30, small eigenvalues, or $\mathrm{V}\mathrm{I}\mathrm{F}$ values exceeding 5 indicate multicollinearity, which can distort regression results. Addressing multicollinearity may require methods such as ridge regression.

The dataset demonstrates severe multicollinearity, evidenced by a high value of $CN$(1746.33) and a smallest eigenvalue of 0.0022, indicating near-linear dependency among predictors. VIF values are also excessively high for key variables such as BP (259.76), age (29.02), and weight (161.35). Furthermore, Figure 1 reveals strong positive correlations between BP and predictors such as weight and BSA, while stress shows weaker correlations with other variables. These findings confirm the significant multicollinearity present, which can lead to unstable and unreliable regression coefficient estimates. To mitigate this issue, the newly proposed and existing ridge estimators were applied to stabilize the model and improve estimation precision.

The analysis of this real dataset validates the simulation results, confirming that our proposed ridge parameter estimator (MIRE2) performs better than other existing ridge estimators as shown in

Table 1. Estimated MSE of ridge estimators for Blood Pressure dataset.

Estimators	MSE	${\widehat{\mathit{\phi }}}_0$	${\widehat{\mathit{\phi }}}_1$	${\widehat{\mathit{\phi }}}_2$	${\widehat{\mathit{\phi }}}_3$	${\widehat{\mathit{\phi }}}_4$	${\widehat{\mathit{\phi }}}_5$	${\widehat{\mathit{\phi }}}_6$
OLS	2.188755	−0.0454	0.163987	0.120501	0.031811	0.015822	−0.25681	−0.12711
HK	0.17853	0.222696	0.473453	0.013541	−0.005	−0.08594	−0.08294	0.040346
HKB	0.217631	0.037362	−0.25313	−0.08097	−0.07582	0.03347	0.229902	0.073195
KAM	1.433083	−0.2745	−0.04539	0.035688	0.240916	0.031939	0.022055	−0.02748
KGM	0.174262	0.180185	0.222602	0.039666	0.02516	−0.00502	−0.12498	−0.07193
KMed	0.175311	0.651653	0.03733	−0.00662	−0.14654	−0.05415	0.05123	0.240152
KMS	0.534015	−3.69391	−0.27418	−0.04536	0.062261	0.25003	0.051809	0.051425
LC	0.179084	−0.04377	0.179822	0.222172	0.065845	0.038097	−0.00832	−0.16124
TK	0.17798	0.202627	0.646686	0.03718	−0.01076	−0.26317	−0.04791	0.096102
MTPR1	0.179128	0.030995	−2.89173	−0.27269	−0.10028	0.14928	0.234429	0.042678
MTPR2	0.17908	−0.2146	−0.03557	0.178178	0.185786	0.279471	0.03916	−0.00226
MTPR3	0.192746	0.122337	0.128407	0.624832	0.015046	−0.06611	−0.28684	−0.00647
MIRE1	0.400804	−0.04505	0.039991	0.191766	0.029953	0.039203	−1.05962	−0.19052
MIRE2	0.173548	0.218182	0.045384	0.015773	−0.0047	−0.28349	−0.0614	0.031881
MIRE3	0.17806	0.035825	−0.00764	−0.08565	−0.09856	0.178682	0.253537	0.395008
MIRE4	0.751334	−0.25938	−0.03442	0.033347	0.192158	0.507683	0.008764	−1.9928

.

4.2. Acetylene Conversion Dataset

This dataset includes 16 observations with a response variable $y$, and three predictors$X_1$, $X_2$, and $X_3$. The corresponding linear regression model is expressed as:

$$y={\phi }_0+{\phi }_1{X}_1+{\phi }_2{X}_2+{\phi }_3{X}_3+\mathsf{ϵ},$$

where ${\phi }_0$ is the intercept and ${\phi }_1$, ${\phi }_2$, and ${\phi }_3$ are the coefficients for each predictor, and ϵ represents the error term.

Figure 2 reveals severe multicollinearity, particularly a strong correlation between the independent variables $X_1$ and $X_3$ (−0.96). These dependencies can undermine the stability of regression models. To address this, we compared the performance of our proposed and existing ridge estimators using the estimated MSE as the evaluation criterion.

It is clear from

Table 2. Estimated MSE of ridge estimators for Acetylene Conversion dataset.

Estimators	MSE	${\widehat{\mathit{\phi }}}_0$	${\widehat{\mathit{\phi }}}_1$	${\widehat{\mathit{\phi }}}_2$	${\widehat{\mathit{\phi }}}_3$
OLS	1.224021	0.488659	−0.13759	0.527294	−0.11288
HK	1.141057	−0.53522	−0.74052	−0.33586	−0.48008
HKB	1.156782	−0.14273	0.48835	−0.01194	0.488664
KAM	1.219503	−0.80977	−0.5341	−0.02796	−0.53522
KGM	1.113903	0.488323	−0.13925	0.514907	−0.1427
KMed	1.147368	−0.53399	−0.76219	−0.43638	−0.80942
KMS	1.176266	−0.13895	0.488473	−0.02578	0.488996
LC	0.760537	−0.75815	−0.53454	−0.0642	−0.53521
TK	0.659068	0.488391	−0.14062	0.498588	−0.13954
MTPR1	0.33718	−0.53424	−0.78045	−0.5124	−0.79075
MTPR2	0.430533	−0.1397	0.493592	−0.06977	0.527509
MTPR3	0.56701	−0.76818	−0.53073	−0.21949	−0.33276
MIRE1	1.223442	−0.14253	0.495315	−0.14236	0.489342
MIRE2	1.192328	−0.80704	−0.52534	−0.80425	−0.53506
MIRE3	0.335577	0.488197	−0.09392	0.493304	−0.09916
MIRE4	1.172576	−0.53354	−0.34577	−0.53142	−0.78466

that our proposed estimator (MIRE3) achieved the minimum estimated MSE compared to OLS and other existing ridge estimators. This demonstrates the superior performance and effectiveness of MIRE3 in handling multicollinearity and providing more accurate regression estimates.

Additionally, we assessed the significance of the regression coefficients for each estimator for the Acetylene Conversion dataset by calculating the corresponding t-statistics. These t-statistics were obtained by dividing the estimated coefficients by their respective standard errors, which were derived from the MSE values. The t-statistics for the coefficients across all estimators are summarized in

Table 3. Significance coefficient of ridge estimators.

Estimator	MSE	t-Test for  ${\widehat{\mathit{\phi }}}_0$	t-Test for  ${\widehat{\mathit{\phi }}}_1$	t-Test for  ${\widehat{\mathit{\phi }}}_2$	$\mathit{t}\text{-}\mathbf{Test} \mathbf{for} {\widehat{\mathit{\phi }}}_3$	$\mathit{p}\text{-}\mathbf{Value} \mathbf{for} {\widehat{\mathit{\phi }}}_0$	$\mathit{p}\text{-}\mathbf{Value} \mathbf{for} {\widehat{\mathit{\phi }}}_1$	$\mathit{p}\text{-}\mathbf{Value} \mathbf{for} {\widehat{\mathit{\phi }}}_2$	$\mathit{p}\text{-}\mathbf{Value} \mathbf{for} {\widehat{\mathit{\phi }}}_3$
OLS	1.224021	1.766735	−0.49745	1.906419	−0.40812	0.10497	0.628663	0.083043	0.691021
HK	1.141057	−2.00419	−2.77296	−1.25766	−1.79771	0.070298	0.018131	0.234557	0.099696
HKB	1.156782	−0.53082	1.816207	−0.04441	1.817375	0.606093	0.096663	0.965377	0.096474
KAM	1.219503	−2.93312	−1.9346	−0.10128	−1.93866	0.013614	0.079165	0.921154	0.078621
KGM	1.113903	1.850732	−0.52775	1.951484	−0.54083	0.091225	0.608151	0.076923	0.599406
KMed	1.147368	−1.99408	−2.84624	−1.62957	−3.02261	0.071526	0.015903	0.131467	0.011601
KMS	1.176266	−0.51247	1.801556	−0.09508	1.803485	0.618457	0.099058	0.925961	0.09874
LC	0.760537	−3.4774	−2.45177	−0.29447	−2.45484	0.005172	0.032145	0.773885	0.03197
TK	0.659068	2.406371	−0.69285	2.456613	−0.68753	0.034838	0.502767	0.03187	0.505985
MTPR1	0.33718	−3.68015	−5.37618	−3.5297	−5.44714	0.003625	0.000225	0.004717	0.000202
MTPR2	0.430533	−0.85163	3.009019	−0.42533	3.215782	0.412574	0.011887	0.678798	0.082211
MTPR3	0.56701	−4.08063	−2.81928	−1.16595	−1.76765	0.001818	0.016689	0.268294	0.104811
MIRE1	1.223442	−0.51544	1.791224	−0.51482	1.769623	0.616449	0.10078	0.616865	0.104467
MIRE2	1.192328	−2.95636	−1.92443	−2.94614	−1.96004	0.01306	0.080545	0.013301	0.075809
MIRE3	0.335577	3.371002	−0.64852	3.406266	−0.6847	0.006241	0.529956	0.005864	0.507703
MIRE4	1.172576	−1.97086	−1.27725	−1.96303	−2.89848	0.074422	0.227814	0.075423	0.014484

.

The MIRE2, MIRE3, and MTPR1 estimators showed significance for most of their coefficients in this dataset, suggesting that these models are particularly effective in capturing the relationships between the predictors and the response variable. In contrast, OLS and other existing estimators, including HK, HKB, KAM, KGM, KMed, KMS, LC, and TK, exhibited several non-significant coefficients, which indicated that while these models capture certain relationships within the data, they may not fully account for the variability in the dependent variable. Overall, the existing estimator MTPR1 and the newly proposed estimator MIRE3 outperform others in both predictive accuracy and the statistical significance of their coefficients.

5. Conclusions

In this study, we introduced four novel improved two-parameter ridge estimators (MIRE1, MIRE2, MIRE3, and MIRE4) developed especially to combat the issue of multicollinearity in regression analysis. Based on the results of simulations and the analysis of the two real-life datasets, our newly proposed estimators consistently demonstrated superior performance compared to existing estimators, particularly in scenarios characterized by severe multicollinearity and high error variance.

MIRE2 and MIRE3, in particular, had the minimum estimated MSE and offered superior performance in both simulated and real datasets. When using t-tests to evaluate the significance of the regression coefficients for both OLS and ridge regression estimators in the Acetylene Conversion dataset, our newly proposed estimators, MIRE2 and MIRE3, demonstrated statistical significance for most of their coefficients, further confirming their effectiveness. Overall, these new estimators provided more accurate and stable estimates, making them a reliable solution for handling multicollinearity in regression models.

Future research could focus on improving the proposed ridge estimators for more complex data, incorporating variable selection techniques, and evaluating their performance across diverse real-world applications to further enhance their robustness and applicability.