Newly Improved Two-Parameter Ridge Estimators: A Better Approach for Mitigating Multicollinearity in Regression Analysis
Abstract
:1. Introduction
2. Statistical Methodology for Ridge Regression
- , a transformed design matrix.
- , where is an orthogonal matrix containing the eigenvectors of .
- is the parameter vector, and ϵ represents the noise.
- ensuring orthogonality, with being the identity matrix.
- The transformation aligns with the principal components of .
2.1. Ridge Estimators
2.1.1. Existing Ridge Estimators
2.1.2. New Proposed Ridge Estimators
2.1.3. Performance of Estimators Under Mean Square Error
3. Monte Carlo Simulations
3.1. Simulation Study
Algorithm 1: MSE Computation. |
|
. (24) |
3.2. Discussion of the Results
4. Real-Life Applications
4.1. Blood Pressure Patients Dataset
4.2. Acetylene Conversion Dataset
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A
OLS | HK | HKB | KAM | KGM | KMed | KMS | LC | TK | MTPR1 | MTPR2 | MTPR3 | MIRE1 | MIRE2 | MIRE3 | MIRE4 | ||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0.4 | 0.8 | 0.1572 | 0.1391 | 0.0942 | 0.1533 | 0.0667 | 0.0983 | 0.1273 | 0.0106 | 0.6058 | 0.0032 | 0.0041 | 0.0052 | 0.1427 | 0.08493 | 0.0059 | 0.0850 |
0.90 | 0.3222 | 0.2499 | 0.1526 | 0.3079 | 0.0875 | 0.1119 | 0.214 | 0.0157 | 0.4671 | 0.0025 | 0.0034 | 0.0312 | 0.271 | 0.1046 | 0.004 | 0.105 | |
0.95 | 0.6439 | 0.4039 | 0.2297 | 0.5996 | 0.1295 | 0.1609 | 0.3562 | 0.0221 | 0.1966 | 0.0023 | 0.0035 | 0.1336 | 0.5374 | 0.1523 | 0.003 | 0.1534 | |
0.99 | 3.2936 | 1.1448 | 0.8946 | 2.8819 | 0.5183 | 0.5673 | 1.8061 | 0.1568 | 0.0776 | 0.0025 | 0.0174 | 2.4151 | 2.8153 | 0.1601 | 0.0022 | 0.156 | |
1 | 0.8 | 0.9356 | 0.5324 | 0.3797 | 0.86 | 0.2296 | 0.3076 | 0.5465 | 0.0601 | 1.2584 | 0.0372 | 0.0469 | 0.2099 | 0.4966 | 0.0873 | 0.037 | 0.0881 |
0.90 | 2.0703 | 0.9015 | 0.5898 | 1.8515 | 0.3475 | 0.4476 | 1.1502 | 0.0944 | 0.8275 | 0.0501 | 0.0424 | 0.3625 | 1.1261 | 0.118 | 0.0247 | 0.1186 | |
0.95 | 4.327 | 1.7076 | 1.3032 | 3.8014 | 0.5758 | 0.6448 | 2.5553 | 0.1582 | 0.7069 | 0.0228 | 0.0437 | 1.1201 | 2.0865 | 0.2916 | 0.0192 | 0.2948 | |
0.99 | 21.057 | 7.1375 | 4.4245 | 17.7172 | 1.4835 | 1.1418 | 14.280 | 0.2534 | 0.0888 | 0.0143 | 0.0872 | 9.2492 | 10.418 | 1.4964 | 0.0141 | 1.5316 | |
5 | 0.8 | 24.988 | 8.734 | 6.5376 | 21.5395 | 2.6154 | 2.424 | 18.987 | 1.7665 | 7.6045 | 1.4834 | 2.6093 | 7.4336 | 1.1347 | 1.1996 | 3.1117 | 1.2293 |
0.90 | 45.362 | 13.822 | 9.9473 | 37.9924 | 3.0894 | 2.9781 | 34.220 | 1.4089 | 6.9948 | 1.1041 | 2.8158 | 12.931 | 0.9607 | 0.8607 | 2.9719 | 0.8687 | |
0.95 | 103.87 | 33.072 | 26.748 | 88.044 | 6.3998 | 5.9378 | 84.482 | 1.0718 | 8.1453 | 0.806 | 6.1301 | 35.575 | 2.5594 | 0.6594 | 3.1265 | 0.7386 | |
0.99 | 547.51 | 156.54 | 133.47 | 459.806 | 19.7883 | 31.0987 | 485.036 | 0.5052 | 8.5894 | 0.8612 | 38.161 | 253.93 | 16.507 | 0.6083 | 4.619 | 0.8839 | |
10 | 0.8 | 92.627 | 30.609 | 22.259 | 78.8172 | 7.2047 | 7.7303 | 77.268 | 8.4829 | 25.303 | 9.0621 | 14.520 | 33.249 | 0.8802 | 6.6461 | 15.771 | 6.7665 |
0.90 | 203.35 | 71.728 | 48.052 | 172.807 | 11.7479 | 13.166 | 174.88 | 5.2299 | 29.5044 | 5.684 | 23.189 | 78.229 | 1.8967 | 4.3462 | 15.775 | 4.2586 | |
0.95 | 400.27 | 130.03 | 89.160 | 337.138 | 18.0546 | 23.724 | 352.380 | 4.5824 | 23.8706 | 4.226 | 36.609 | 164.97 | 1.3892 | 3.7342 | 15.852 | 3.7716 | |
0.99 | 2043.90 | 625.91 | 422.46 | 1709.18 | 46.266 | 99.010 | 1903.98 | 1.9432 | 19.0621 | 2.5263 | 177.57 | 1063.2 | 5.1256 | 2.0979 | 33.136 | 2.2387 |
OLS | HK | HKB | KAM | KGM | KMed | KMS | LC | TK | MTPR1 | MTPR2 | MTPR3 | MIRE1 | MIRE2 | MIRE3 | MIRE4 | ||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0.4 | 0.8 | 0.0613 | 0.0582 | 0.0459 | 0.0606 | 0.036 | 0.0742 | 0.0548 | 0.0044 | 0.7383 | 0.0013 | 0.0015 | 0.0012 | 0.0583 | 0.0298 | 0.0023 | 0.0297 |
0.90 | 0.1051 | 0.128 | 0.0662 | 0.1027 | 0.0366 | 0.055 | 0.0835 | 0.0054 | 0.5233 | 0.0053 | 0.0016 | 0.023 | 0.0942 | 0.0196 | 0.0015 | 0.0196 | |
0.95 | 0.2209 | 0.1811 | 0.1179 | 0.2126 | 0.058 | 0.0663 | 0.1521 | 0.0098 | 0.0701 | 0.001 | 0.0014 | 0.0015 | 0.1937 | 0.049 | 0.0013 | 0.049 | |
0.99 | 1.0631 | 0.3847 | 0.3173 | 0.9627 | 0.1826 | 0.2173 | 0.5356 | 0.0431 | 0.3147 | 0.0011 | 0.0031 | 0.1456 | 0.8702 | 0.1388 | 0.0010 | 0.1386 | |
1 | 0.8 | 0.3631 | 0.2752 | 0.2028 | 0.3456 | 0.1174 | 0.1444 | 0.2455 | 0.0245 | 1.216 | 0.0076 | 0.0086 | 0.0145 | 0.1814 | 0.0203 | 0.0137 | 0.0204 |
0.90 | 0.6661 | 0.2892 | 0.2778 | 0.6156 | 0.1421 | 0.1747 | 0.3716 | 0.0319 | 0.8869 | 0.0073 | 0.0088 | 0.0385 | 0.2702 | 0.0172 | 0.0090 | 0.0177 | |
0.95 | 1.3517 | 0.6541 | 0.4161 | 1.2133 | 0.223 | 0.2686 | 0.6858 | 0.0542 | 0.2726 | 0.0051 | 0.0066 | 0.0354 | 0.6244 | 0.021 | 0.0065 | 0.0212 | |
0.99 | 6.5269 | 2.1217 | 1.6195 | 5.5927 | 0.7654 | 0.7354 | 3.852 | 0.2009 | 0.3528 | 0.0078 | 0.0191 | 0.6828 | 2.9425 | 0.0581 | 0.0059 | 0.058 | |
5 | 0.8 | 8.6778 | 3.3666 | 2.4639 | 7.474 | 1.1708 | 1.0895 | 5.7949 | 0.6348 | 4.0054 | 0.3776 | 0.5037 | 1.0819 | 0.3777 | 0.2789 | 0.6535 | 0.2789 |
0.90 | 17.2075 | 5.9626 | 3.883 | 14.610 | 1.596 | 1.422 | 11.9379 | 0.5488 | 4.1625 | 0.3582 | 0.5687 | 2.6312 | 0.4023 | 0.1815 | 0.4303 | 0.1816 | |
0.95 | 36.4402 | 11.605 | 8.4171 | 30.926 | 2.7448 | 2.1806 | 27.072 | 0.5277 | 4.595 | 0.3623 | 0.8003 | 6.7518 | 0.7113 | 0.1698 | 0.3332 | 0.1739 | |
0.99 | 157.69 | 42.783 | 32.004 | 130.17 | 6.9136 | 8.6012 | 127.97 | 0.2653 | 2.6145 | 0.1386 | 2.4877 | 33.586 | 1.7602 | 0.1376 | 0.164 | 0.1413 | |
10 | 0.8 | 36.305 | 12.388 | 8.8775 | 31.032 | 3.5383 | 3.3746 | 28.207 | 2.9892 | 10.491 | 2.158 | 2.9559 | 5.699 | 0.5895 | 1.7446 | 4.8858 | 1.7469 |
0.90 | 66.734 | 20.649 | 16.476 | 56.261 | 5.3862 | 5.4657 | 52.597 | 2.3396 | 13.6585 | 1.6146 | 4.0175 | 13.968 | 0.5393 | 1.4068 | 5.6226 | 1.4041 | |
0.95 | 135.87 | 43.722 | 28.860 | 113.15 | 6.588 | 7.7221 | 110.89 | 1.6307 | 12.882 | 1.2294 | 5.2216 | 28.835 | 0.6101 | 1.199 | 7.9673 | 1.1823 | |
0.99 | 670.12 | 219.99 | 162.79 | 561.06 | 21.350 | 33.903 | 598.21 | 0.7724 | 9.0354 | 0.7925 | 23.504 | 222.15 | 1.0651 | 0.6121 | 7.4994 | 0.6189 |
OLS | HK | HKB | KAM | KGM | KMed | KMS | LC | TK | MTPR1 | MTPR2 | MTPR3 | MIRE1 | MIRE2 | MIRE3 | MIRE4 | ||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0.4 | 0.8 | 0.0234 | 0.1464 | 0.0213 | 0.0233 | 0.0238 | 0.0756 | 0.0222 | 0.002 | 0.0096 | 0.0009 | 0.0009 | 0.0035 | 0.0228 | 0.0029 | 0.0012 | 0.0029 |
0.90 | 0.0446 | 0.1077 | 0.0364 | 0.0442 | 0.0219 | 0.0364 | 0.0401 | 0.003 | 0.0524 | 0.0057 | 0.0009 | 0.0058 | 0.0425 | 0.0025 | 0.0008 | 0.0025 | |
0.95 | 0.0937 | 0.0856 | 0.0612 | 0.0919 | 0.0321 | 0.0348 | 0.0761 | 0.0053 | 0.0218 | 0.0006 | 0.0006 | 0.0007 | 0.0846 | 0.0144 | 0.0005 | 0.0144 | |
0.99 | 0.5002 | 0.3335 | 0.1928 | 0.4677 | 0.1029 | 0.1085 | 0.272 | 0.0227 | 0.0021 | 0.005 | 0.0005 | 0.0013 | 0.3723 | 0.0158 | 0.0004 | 0.0159 | |
1 | 0.8 | 0.1463 | 0.1961 | 0.0898 | 0.1427 | 0.0711 | 0.1149 | 0.1179 | 0.0122 | 0.2332 | 0.0104 | 0.0058 | 0.0247 | 0.0831 | 0.0057 | 0.0075 | 0.0057 |
0.90 | 0.2831 | 0.1885 | 0.1415 | 0.2707 | 0.0866 | 0.1044 | 0.1886 | 0.019 | 0.6082 | 0.005 | 0.0045 | 0.0111 | 0.1113 | 0.0044 | 0.0049 | 0.0044 | |
0.95 | 0.6037 | 0.3841 | 0.2207 | 0.5635 | 0.1392 | 0.1565 | 0.3497 | 0.0301 | 0.1568 | 0.0028 | 0.0029 | 0.0262 | 0.2622 | 0.006 | 0.0035 | 0.006 | |
0.99 | 3.0504 | 1.1849 | 0.8127 | 2.6641 | 0.4382 | 0.491 | 1.6223 | 0.1266 | 0.1772 | 0.0027 | 0.0042 | 0.1571 | 1.0661 | 0.0084 | 0.0026 | 0.0085 | |
5 | 0.8 | 3.5238 | 1.2961 | 1.0425 | 3.1094 | 0.6772 | 0.7526 | 2.1524 | 0.3547 | 2.58 | 0.1175 | 0.1461 | 0.2589 | 0.2738 | 0.1002 | 0.2082 | 0.1003 |
0.90 | 7.5212 | 2.4485 | 2.1259 | 6.4997 | 1.0068 | 0.9766 | 4.8189 | 0.3779 | 2.0142 | 0.1116 | 0.2261 | 0.5333 | 0.187 | 0.103 | 0.1387 | 0.1005 | |
0.95 | 15.1839 | 5.1756 | 3.6938 | 12.9329 | 1.5608 | 1.3638 | 10.244 | 0.3951 | 2.897 | 0.0707 | 0.1646 | 1.2594 | 0.2062 | 0.0688 | 0.0888 | 0.0689 | |
0.99 | 78.481 | 23.3488 | 18.7669 | 66.5133 | 4.6267 | 4.3373 | 62.124 | 0.2183 | 2.7013 | 0.0619 | 1.0731 | 11.198 | 0.6722 | 0.0618 | 0.0742 | 0.062 | |
10 | 0.8 | 14.5675 | 5.1741 | 4.0204 | 12.5573 | 1.768 | 1.7265 | 10.3944 | 1.4403 | 5.9546 | 0.7701 | 0.9579 | 1.4362 | 0.6321 | 0.7298 | 1.949 | 0.7299 |
0.90 | 29.7017 | 9.4508 | 7.4379 | 25.1938 | 2.7975 | 2.5946 | 21.787 | 1.1234 | 6.487 | 0.6016 | 0.9617 | 2.4428 | 0.4497 | 0.5348 | 1.618 | 0.5347 | |
0.95 | 63.6297 | 22.2795 | 15.2964 | 54.1393 | 4.6212 | 4.122 | 50.1716 | 0.8506 | 7.4214 | 0.544 | 1.9238 | 8.5953 | 0.3272 | 0.4189 | 1.0744 | 0.4188 | |
0.99 | 306.267 | 92.0396 | 67.4402 | 256.010 | 12.3486 | 19.4741 | 262.200 | 0.3972 | 6.2764 | 0.3157 | 4.316 | 54.201 | 0.4014 | 0.2954 | 1.4603 | 0.2945 |
OLS | HK | HKB | KAM | KGM | KMed | KMS | LC | TK | MTPR1 | MTPR2 | MTPR3 | MIRE1 | MIRE2 | MIRE3 | MIRE4 | ||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0.4 | 0.8 | 0.5188 | 0.3974 | 0.1895 | 0.5094 | 0.0592 | 0.063 | 0.3216 | 0.0042 | 0.6882 | 0.0025 | 0.0028 | 0.0141 | 0.4307 | 0.1427 | 0.0059 | 0.1429 |
0.90 | 1.1397 | 0.705 | 0.3015 | 1.1064 | 0.0719 | 0.0777 | 0.5737 | 0.0034 | 0.2427 | 0.0013 | 0.0019 | 0.1033 | 0.9519 | 0.3046 | 0.0026 | 0.305 | |
0.95 | 2.4719 | 1.1873 | 0.4933 | 2.3766 | 0.1416 | 0.1683 | 1.1951 | 0.0048 | 0.1687 | 0.0073 | 0.0063 | 0.4607 | 2.1117 | 0.5925 | 0.0018 | 0.5922 | |
0.99 | 14.3129 | 5.7113 | 2.5931 | 13.5562 | 0.5878 | 0.724 | 8.3919 | 0.0165 | 0.1599 | 0.0012 | 0.0125 | 7.6035 | 12.372 | 4.0168 | 0.0010 | 4.0173 | |
1 | 0.8 | 3.392 | 1.6948 | 0.8044 | 3.2638 | 0.2466 | 0.2927 | 1.9573 | 0.0271 | 1.2977 | 0.0182 | 0.023 | 0.2718 | 2.3038 | 0.638 | 0.0365 | 0.6398 |
0.90 | 7.4156 | 3.0681 | 1.3627 | 7.0706 | 0.3625 | 0.4721 | 4.2623 | 0.0217 | 0.8638 | 0.0226 | 0.0212 | 0.8232 | 3.9614 | 0.5339 | 0.0177 | 0.5307 | |
0.95 | 14.4581 | 5.3748 | 2.5431 | 13.6523 | 0.6696 | 0.8717 | 8.4805 | 0.0374 | 0.5839 | 0.0400 | 0.035 | 2.2271 | 7.9719 | 0.8235 | 0.0104 | 0.8234 | |
0.99 | 85.8984 | 33.906 | 13.119 | 80.8446 | 3.0126 | 3.2705 | 62.602 | 0.1218 | 0.4148 | 0.0062 | 0.1897 | 33.872 | 46.685 | 7.0412 | 0.0061 | 7.0255 | |
5 | 0.8 | 82.1771 | 34.506 | 13.697 | 78.2093 | 3.9758 | 4.4819 | 66.622 | 1.1106 | 19.7929 | 1.6968 | 5.6176 | 22.161 | 5.4694 | 1.2497 | 2.391 | 1.2906 |
0.90 | 191.451 | 72.198 | 37.971 | 182.099 | 6.9819 | 7.522 | 160.060 | 0.7807 | 15.954 | 1.347 | 8.0289 | 59.508 | 16.385 | 1.4255 | 1.4916 | 1.4682 | |
0.95 | 383.163 | 140.504 | 73.041 | 362.346 | 12.5675 | 13.543 | 325.028 | 0.7357 | 14.259 | 0.9171 | 15.572 | 143.15 | 31.402 | 1.3408 | 0.8325 | 1.3746 | |
0.99 | 2042.02 | 782.290 | 354.40 | 1915.59 | 50.0066 | 67.519 | 1844.85 | 0.4053 | 27.236 | 1.3232 | 141.61 | 1250.6 | 171.50 | 5.821 | 0.3751 | 5.9954 | |
10 | 0.8 | 342.561 | 146.341 | 59.225 | 325.671 | 13.8568 | 16.363 | 303.851 | 9.9257 | 69.122 | 15.495 | 38.581 | 124.13 | 6.3369 | 8.8586 | 22.912 | 9.1906 |
0.90 | 744.955 | 294.663 | 137.05 | 706.521 | 22.5394 | 28.591 | 664.986 | 6.4457 | 80.670 | 11.259 | 68.164 | 293.66 | 12.957 | 6.5774 | 23.142 | 6.4253 | |
0.95 | 1543.34 | 592.034 | 262.43 | 1450.65 | 39.7229 | 51.026 | 1390.71 | 4.7824 | 74.992 | 6.4008 | 121.44 | 696.97 | 30.188 | 4.4712 | 15.456 | 5.5657 | |
0.99 | 8073.69 | 2801.08 | 1320.8 | 7568.42 | 181.469 | 278.566 | 7622.06 | 3.2748 | 128.02 | 22.326 | 911.42 | 5150.9 | 213.32 | 10.810 | 30.988 | 13.419 |
OLS | HK | HKB | KAM | KGM | KMed | KMS | LC | TK | MTPR1 | MTPR2 | MTPR3 | MIRE1 | MIRE2 | MIRE3 | MIRE4 | ||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0.4 | 0.8 | 0.1611 | 0.1503 | 0.0959 | 0.1602 | 0.0287 | 0.0312 | 0.1288 | 0.0012 | 0.191 | 0.0006 | 0.0006 | 0.0005 | 0.1494 | 0.0403 | 0.0013 | 0.0402 |
0.90 | 0.3537 | 0.3042 | 0.152 | 0.3498 | 0.0404 | 0.0436 | 0.2316 | 0.0017 | 0.0039 | 0.0007 | 0.0006 | 0.0007 | 0.3025 | 0.0403 | 0.0008 | 0.0404 | |
0.95 | 0.6517 | 0.4928 | 0.2154 | 0.64 | 0.0591 | 0.0662 | 0.3453 | 0.0019 | 0.0012 | 0.0007 | 0.0008 | 0.001 | 0.5419 | 0.0575 | 0.0005 | 0.0576 | |
0.99 | 3.7016 | 1.894 | 0.8355 | 3.5911 | 0.3009 | 0.4016 | 1.8893 | 0.0119 | 0.01 | 0.0004 | 0.0006 | 0.3274 | 3.1682 | 0.4187 | 0.0003 | 0.4192 | |
1 | 0.8 | 1.0702 | 0.7639 | 0.3371 | 1.0501 | 0.1132 | 0.1458 | 0.6181 | 0.0073 | 0.1767 | 0.0038 | 0.004 | 0.0056 | 0.5822 | 0.0221 | 0.0083 | 0.0222 |
0.90 | 2.3335 | 1.3645 | 0.5599 | 2.2734 | 0.1876 | 0.2494 | 1.2239 | 0.01 | 0.3432 | 0.0088 | 0.0155 | 0.034 | 1.2303 | 0.028 | 0.0047 | 0.0281 | |
0.95 | 4.2913 | 2.1664 | 0.8881 | 4.1556 | 0.3317 | 0.4854 | 2.2131 | 0.0118 | 0.2424 | 0.0027 | 0.0048 | 0.0956 | 2.1603 | 0.0374 | 0.0034 | 0.0378 | |
0.99 | 22.2211 | 9.3209 | 4.4677 | 21.3904 | 1.3866 | 1.8931 | 14.4239 | 0.0562 | 0.0344 | 0.0021 | 0.0215 | 1.6215 | 11.968 | 0.2262 | 0.0020 | 0.2242 | |
5 | 0.8 | 25.327 | 11.5173 | 4.8682 | 24.4326 | 1.7771 | 2.2477 | 18.1977 | 0.202 | 4.3737 | 0.2121 | 0.5231 | 1.3102 | 1.1649 | 0.1053 | 0.2355 | 0.107 |
0.90 | 52.9484 | 23.5778 | 9.889 | 50.9119 | 3.0896 | 3.7735 | 39.2848 | 0.2153 | 2.8284 | 0.0717 | 0.4264 | 2.3203 | 2.1967 | 0.0703 | 0.1167 | 0.0772 | |
0.95 | 115.675 | 49.8305 | 23.1694 | 111.218 | 6.5548 | 7.7034 | 91.8203 | 0.3263 | 1.8101 | 0.0637 | 0.6447 | 4.642 | 4.549 | 0.0597 | 0.0754 | 0.0607 | |
0.99 | 523.631 | 226.732 | 95.0532 | 502.203 | 23.3557 | 35.6529 | 454.483 | 0.1937 | 2.2201 | 0.0519 | 7.0968 | 59.377 | 17.29 | 0.0647 | 0.0502 | 0.0681 | |
10 | 0.8 | 104.237 | 43.127 | 20.2642 | 100.577 | 5.6847 | 7.6251 | 86.0279 | 1.8365 | 27.5209 | 1.9357 | 6.0934 | 14.299 | 0.7094 | 1.4182 | 5.3904 | 1.4143 |
0.90 | 214.731 | 93.4446 | 38.3373 | 206.632 | 10.564 | 14.7109 | 181.612 | 1.3328 | 32.5583 | 1.4165 | 11.829 | 36.447 | 1.1364 | 0.8814 | 5.2433 | 0.8673 | |
0.95 | 434.928 | 189.459 | 78.4042 | 417.634 | 19.526 | 28.7565 | 378.218 | 0.6434 | 37.8734 | 0.5299 | 27.172 | 103.65 | 1.7978 | 0.3413 | 2.4605 | 0.3378 | |
0.99 | 2158.95 | 945.147 | 457.822 | 2074.85 | 82.546 | 150.378 | 2001.76 | 0.3442 | 25.8402 | 0.2267 | 177.57 | 757.56 | 11.053 | 0.2198 | 7.9883 | 0.2274 |
OLS | HK | HKB | KAM | KGM | KMed | KMS | LC | TK | MTPR1 | MTPR2 | MTPR3 | MIRE1 | MIRE2 | MIRE3 | MIRE4 | ||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0.4 | 0.8 | 0.08322 | 0.07963 | 0.05694 | 0.0828 | 0.01839 | 0.0208 | 0.07031 | 0.00069 | 0.77834 | 0.00035 | 0.00032 | 0.00038 | 0.0788 | 0.01913 | 0.0006 | 0.0191 |
0.90 | 0.1874 | 0.16924 | 0.09233 | 0.18596 | 0.02143 | 0.0233 | 0.1321 | 0.00069 | 0.0035 | 0.00026 | 0.00028 | 0.00024 | 0.1665 | 0.01791 | 0.0003 | 0.0179 | |
0.95 | 0.3869 | 0.3164 | 0.15592 | 0.38141 | 0.03616 | 0.0415 | 0.22534 | 0.0013 | 0.00192 | 0.00024 | 0.00022 | 0.00029 | 0.3278 | 0.0231 | 0.0002 | 0.02312 | |
0.99 | 1.951 | 1.1231 | 0.4527 | 1.8891 | 0.1551 | 0.2022 | 0.9163 | 0.0043 | 0.00031 | 0.00025 | 0.00022 | 0.00043 | 1.6637 | 0.0922 | 0.0002 | 0.09234 | |
1 | 0.8 | 0.5436 | 0.4286 | 0.228 | 0.5349 | 0.0653 | 0.0717 | 0.33 | 0.0039 | 0.0328 | 0.0020 | 0.0018 | 0.0017 | 0.2814 | 0.0038 | 0.0038 | 0.0039 |
0.90 | 1.1273 | 0.7536 | 0.3536 | 1.1001 | 0.1054 | 0.1294 | 0.5832 | 0.0045 | 0.033 | 0.0014 | 0.0015 | 0.0019 | 0.6272 | 0.0082 | 0.0022 | 0.0082 | |
0.95 | 2.5124 | 1.3804 | 0.5669 | 2.4314 | 0.1956 | 0.2561 | 1.2375 | 0.0068 | 0.0257 | 0.0024 | 0.0057 | 0.0156 | 1.1931 | 0.0087 | 0.0015 | 0.0087 | |
0.99 | 12.956 | 5.478 | 2.4577 | 12.4041 | 0.8515 | 1.2475 | 7.6283 | 0.0234 | 0.7811 | 0.0013 | 0.0089 | 0.7602 | 6.1691 | 0.0285 | 0.0011 | 0.0284 | |
5 | 0.8 | 14.344 | 6.3146 | 2.9493 | 13.8207 | 0.9877 | 1.4118 | 9.6045 | 0.0951 | 1.9611 | 0.0469 | 0.0663 | 0.1611 | 0.5999 | 0.0488 | 0.1002 | 0.0491 |
0.90 | 28.380 | 11.5972 | 5.3825 | 27.1553 | 1.7907 | 2.2452 | 19.211 | 0.1168 | 1.5417 | 0.0408 | 0.0911 | 0.4681 | 1.7073 | 0.0333 | 0.0504 | 0.0336 | |
0.95 | 60.136 | 25.0217 | 10.178 | 57.5029 | 3.2949 | 4.1037 | 44.086 | 0.167 | 1.4223 | 0.0345 | 0.1586 | 1.2242 | 1.8804 | 0.0339 | 0.0410 | 0.0341 | |
0.99 | 313.459 | 122.492 | 55.550 | 298.862 | 12.946 | 19.024 | 261.999 | 0.1712 | 0.4428 | 0.0286 | 0.4559 | 8.4157 | 16.182 | 0.0327 | 0.0300 | 0.0331 | |
10 | 0.8 | 55.798 | 23.4185 | 10.171 | 53.6356 | 3.1436 | 3.9559 | 43.335 | 0.4513 | 11.4542 | 0.3115 | 0.7444 | 3.0202 | 0.4237 | 0.1773 | 0.8335 | 0.1776 |
0.90 | 112.121 | 45.6048 | 20.177 | 107.133 | 5.8409 | 7.5225 | 88.221 | 0.3561 | 7.9432 | 0.1783 | 1.1472 | 4.8744 | 0.3227 | 0.1319 | 0.5234 | 0.1322 | |
0.95 | 244.091 | 97.5826 | 39.892 | 233.215 | 11.764 | 15.372 | 202.59 | 0.3595 | 13.4349 | 0.1174 | 3.7388 | 20.4618 | 0.9118 | 0.1096 | 0.3972 | 0.1100 | |
0.99 | 1309.14 | 560.092 | 243.801 | 1251.66 | 49.395 | 81.049 | 1188.6 | 0.2131 | 9.0287 | 0.1051 | 22.754 | 219.699 | 4.7756 | 0.1048 | 0.1494 | 0.1054 |
0.80 | 0.90 | 0.95 | 0.99 | 0.80 | 0.90 | 0.95 | 0.99 | ||
---|---|---|---|---|---|---|---|---|---|
20 | 0.4 | MTPR1 | MTPR1 | MTPR1 | MIRE3 | MTPR1 | MTPR1 | MIRE3 | MIRE3 |
1 | MIRE3 | MIRE3 | MIRE3 | MIRE3 | MITPR2 | MIRE3 | MIRE3 | MIRE3 | |
5 | MIRE3 | MIRE2 | MIRE2 | MIRE2 | LC | LC | LC | MIRE3 | |
10 | MIRE1 | MIRE1 | MIRE1 | MIRE2 | MIRE1 | MIRE4 | MIRE2 | LC | |
50 | 0.4 | MTPR3 | MIRE3 | MIRE3 | MIRE3 | MTPR3 | MTPR2 | MIRE3 | MIRE3 |
1 | MTPR2 | MTPR2 | MIRE3 | MIRE3 | MTPR1 | MIRE3 | MIRE3 | MIRE3 | |
5 | MIRE4 | MIRE2 | MIRE2 | MIRE2 | MIRE2 | MIRE2 | MIRE2 | MIRE3 | |
10 | MIRE1 | MIRE1 | MIRE1 | MIRE2 | MIRE1 | MIRE4 | MIRE4 | MIRE2 | |
100 | 0.4 | MTPR1 | MTPR1 | MIRE3 | MIRE3 | MTPR2 | MPTR3 | MIRE3 | MIRE3 |
1 | MIRE2 | MIRE2 | MTPR1 | MIRE3 | MTPR3 | MTPR1 | MIRE3 | MIRE3 | |
5 | MIRE2 | MIRE2 | MIRE2 | MIRE2 | MTPR1 | MIRE2 | MIRE2 | MIRE3 | |
10 | MIRE1 | MIRE1 | MIRE1 | MIRE4 | MIRE4 | MIRE2 | MIRE2 | MIRE2 |
References
- Hoerl, A.E.; Kennard, R.W. Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics 1970, 12, 55–67. [Google Scholar] [CrossRef]
- Muniz, G.; Kibria, B.M.G. On Some Ridge Regression Estimators: An Empirical Comparisons. Commun. Stat.-Simul. Comput. 2009, 38, 621–630. [Google Scholar] [CrossRef]
- Toker, S.; Kaçıranlar, S. On the performance of two parameter ridge estimator under the mean square error criterion. Appl. Math. Comput. 2013, 219, 4718–4728. [Google Scholar] [CrossRef]
- McDonald, G.C. A Monte Carlo Evaluation of Some Ridge-Type Estimators. J. Am. Stat. Assoc. 1975, 70, 407–416. [Google Scholar] [CrossRef]
- Khalaf, G.; Shukur, G. Choosing Ridge Parameter for Regression Problems. Commun. Stat.-Theory Methods 2005, 34, 1177–1182. [Google Scholar] [CrossRef]
- Kibria, B.M.G. Performance of some New Ridge regression estimators. Commun. Stat. Part B Simul. Comput. 2003, 32, 419–435. [Google Scholar] [CrossRef]
- Månsson, K.; Shukur, G. On Ridge Parameters in Logistic Regression. Commun. Stat.-Theory Methods 2011, 40, 3366–3381. [Google Scholar] [CrossRef]
- Saleh AM, E.; Kibria, B.G. Improved ridge regression estimators for the logistic regression model. Comput. Stat. 2013, 28, 2519–2558. [Google Scholar] [CrossRef]
- O’Driscoll, D.; Ramirez, D.E. Mitigating collinearity in linear regression models using ridge, surrogate and raised estimators. Cogent Math. 2016, 3, 1144697. [Google Scholar] [CrossRef]
- Khan, M.S.; Ali, A.; Suhail, M.; Kibria, B.M.G. On some two parameter estimators for the linear regression models with correlated predictors: Simulation and application. Commun. Stat.-Simul. Comput. 2024, 1, 1–15. [Google Scholar] [CrossRef]
- Suhail, M.; Chand, S.; Kibria, B.M.G. Quantile-based robust ridge m-estimator for linear regression model in presence of multicollinearity and outliers. Commun. Stat.-Simul. Comput. 2021, 50, 3194–3206. [Google Scholar] [CrossRef]
- Li, Y.; Yang, R.; Wang, X.; Zhu, J.; Song, N. Carbon Price Combination Forecasting Model Based on Lasso Regression and Optimal Integration. Sustainability 2023, 15, 9354. [Google Scholar] [CrossRef]
- Akhtar, N.; Alharthi, M.F.; Khan, M.S. Mitigating Multicollinearity in Regression: A Study on Improved Ridge Estimators. Mathematics 2024, 12, 3027. [Google Scholar] [CrossRef]
- Hoerl, A.E.; Kannard, R.W.; Baldwin, K.F. Ridge regression: Some simulations. Commun. Stat.-Theory Methods 1975, 4, 105–123. [Google Scholar] [CrossRef]
- Khalaf, G.; Månsson, K.; Shukur, G. Modified Ridge Regression Estimators. Commun. Stat.-Theory Methods 2013, 42, 1476–1487. [Google Scholar] [CrossRef]
- Lipovetsky, S.; Conklin, W.M. Ridge regression in two-parameter solution. Appl. Stoch. Model. Bus. Ind. 2005, 21, 525–540. [Google Scholar] [CrossRef]
- Yasin, S.; Salem, S.; Ayed, H.; Kamal, S.; Suhail, M.; Khan, Y.A. Modified Robust Ridge M-Estimators in Two-Parameter Ridge Regression Model. Math. Probl. Eng. 2021, 2021, 1845914. [Google Scholar] [CrossRef]
- Jensen, D.R.; Ramirez, D.E. On mitigating collinearity through mixtures. J. Stat. Comput. Simul. 2018, 88, 1437–1453. [Google Scholar] [CrossRef]
- Akhtar, N.; Alharthi, M.F. A comparative study of the performance of new ridge estimators for multicollinearity: Insights from simulation and real data application. AIP Adv. 2024, 14, 14. [Google Scholar] [CrossRef]
- Özbay, N. Two-Parameter Ridge Estimation for the Coefficients of Almon Distributed Lag Model. Iran. J. Sci. Technol. Trans. A Sci. 2019, 43, 1819–1828. [Google Scholar] [CrossRef]
- Halawa, A.M.; El Bassiouni, M.Y. Tests of regression coefficients under ridge regression models. J. Stat. Comput. Simul. 2000, 65, 341–356. [Google Scholar] [CrossRef]
- Norman, H.S.; Draper, R. Applied Regression Analysis, 3rd ed.; WILLEY: Hoboken, NJ, USA, 1998. [Google Scholar]
- Irandoukht, A. Optimum Ridge Regression Parameter Using R-Squared of Prediction as a Criterion for Regression Analysis. J. Stat. Theory Appl. 2021, 20, 242–250. [Google Scholar] [CrossRef]
- Kunugi, T.N.T. New Acetylene Process Uses Hydrogen Dilution. Chem. Eng. Prog. 1961, 57, 43–49. [Google Scholar]
Estimators | MSE | |||||||
---|---|---|---|---|---|---|---|---|
OLS | 2.188755 | −0.0454 | 0.163987 | 0.120501 | 0.031811 | 0.015822 | −0.25681 | −0.12711 |
HK | 0.17853 | 0.222696 | 0.473453 | 0.013541 | −0.005 | −0.08594 | −0.08294 | 0.040346 |
HKB | 0.217631 | 0.037362 | −0.25313 | −0.08097 | −0.07582 | 0.03347 | 0.229902 | 0.073195 |
KAM | 1.433083 | −0.2745 | −0.04539 | 0.035688 | 0.240916 | 0.031939 | 0.022055 | −0.02748 |
KGM | 0.174262 | 0.180185 | 0.222602 | 0.039666 | 0.02516 | −0.00502 | −0.12498 | −0.07193 |
KMed | 0.175311 | 0.651653 | 0.03733 | −0.00662 | −0.14654 | −0.05415 | 0.05123 | 0.240152 |
KMS | 0.534015 | −3.69391 | −0.27418 | −0.04536 | 0.062261 | 0.25003 | 0.051809 | 0.051425 |
LC | 0.179084 | −0.04377 | 0.179822 | 0.222172 | 0.065845 | 0.038097 | −0.00832 | −0.16124 |
TK | 0.17798 | 0.202627 | 0.646686 | 0.03718 | −0.01076 | −0.26317 | −0.04791 | 0.096102 |
MTPR1 | 0.179128 | 0.030995 | −2.89173 | −0.27269 | −0.10028 | 0.14928 | 0.234429 | 0.042678 |
MTPR2 | 0.17908 | −0.2146 | −0.03557 | 0.178178 | 0.185786 | 0.279471 | 0.03916 | −0.00226 |
MTPR3 | 0.192746 | 0.122337 | 0.128407 | 0.624832 | 0.015046 | −0.06611 | −0.28684 | −0.00647 |
MIRE1 | 0.400804 | −0.04505 | 0.039991 | 0.191766 | 0.029953 | 0.039203 | −1.05962 | −0.19052 |
MIRE2 | 0.173548 | 0.218182 | 0.045384 | 0.015773 | −0.0047 | −0.28349 | −0.0614 | 0.031881 |
MIRE3 | 0.17806 | 0.035825 | −0.00764 | −0.08565 | −0.09856 | 0.178682 | 0.253537 | 0.395008 |
MIRE4 | 0.751334 | −0.25938 | −0.03442 | 0.033347 | 0.192158 | 0.507683 | 0.008764 | −1.9928 |
Estimators | MSE | ||||
---|---|---|---|---|---|
OLS | 1.224021 | 0.488659 | −0.13759 | 0.527294 | −0.11288 |
HK | 1.141057 | −0.53522 | −0.74052 | −0.33586 | −0.48008 |
HKB | 1.156782 | −0.14273 | 0.48835 | −0.01194 | 0.488664 |
KAM | 1.219503 | −0.80977 | −0.5341 | −0.02796 | −0.53522 |
KGM | 1.113903 | 0.488323 | −0.13925 | 0.514907 | −0.1427 |
KMed | 1.147368 | −0.53399 | −0.76219 | −0.43638 | −0.80942 |
KMS | 1.176266 | −0.13895 | 0.488473 | −0.02578 | 0.488996 |
LC | 0.760537 | −0.75815 | −0.53454 | −0.0642 | −0.53521 |
TK | 0.659068 | 0.488391 | −0.14062 | 0.498588 | −0.13954 |
MTPR1 | 0.33718 | −0.53424 | −0.78045 | −0.5124 | −0.79075 |
MTPR2 | 0.430533 | −0.1397 | 0.493592 | −0.06977 | 0.527509 |
MTPR3 | 0.56701 | −0.76818 | −0.53073 | −0.21949 | −0.33276 |
MIRE1 | 1.223442 | −0.14253 | 0.495315 | −0.14236 | 0.489342 |
MIRE2 | 1.192328 | −0.80704 | −0.52534 | −0.80425 | −0.53506 |
MIRE3 | 0.335577 | 0.488197 | −0.09392 | 0.493304 | −0.09916 |
MIRE4 | 1.172576 | −0.53354 | −0.34577 | −0.53142 | −0.78466 |
Estimator | MSE | t-Test for | t-Test for | t-Test for | |||||
---|---|---|---|---|---|---|---|---|---|
OLS | 1.224021 | 1.766735 | −0.49745 | 1.906419 | −0.40812 | 0.10497 | 0.628663 | 0.083043 | 0.691021 |
HK | 1.141057 | −2.00419 | −2.77296 | −1.25766 | −1.79771 | 0.070298 | 0.018131 | 0.234557 | 0.099696 |
HKB | 1.156782 | −0.53082 | 1.816207 | −0.04441 | 1.817375 | 0.606093 | 0.096663 | 0.965377 | 0.096474 |
KAM | 1.219503 | −2.93312 | −1.9346 | −0.10128 | −1.93866 | 0.013614 | 0.079165 | 0.921154 | 0.078621 |
KGM | 1.113903 | 1.850732 | −0.52775 | 1.951484 | −0.54083 | 0.091225 | 0.608151 | 0.076923 | 0.599406 |
KMed | 1.147368 | −1.99408 | −2.84624 | −1.62957 | −3.02261 | 0.071526 | 0.015903 | 0.131467 | 0.011601 |
KMS | 1.176266 | −0.51247 | 1.801556 | −0.09508 | 1.803485 | 0.618457 | 0.099058 | 0.925961 | 0.09874 |
LC | 0.760537 | −3.4774 | −2.45177 | −0.29447 | −2.45484 | 0.005172 | 0.032145 | 0.773885 | 0.03197 |
TK | 0.659068 | 2.406371 | −0.69285 | 2.456613 | −0.68753 | 0.034838 | 0.502767 | 0.03187 | 0.505985 |
MTPR1 | 0.33718 | −3.68015 | −5.37618 | −3.5297 | −5.44714 | 0.003625 | 0.000225 | 0.004717 | 0.000202 |
MTPR2 | 0.430533 | −0.85163 | 3.009019 | −0.42533 | 3.215782 | 0.412574 | 0.011887 | 0.678798 | 0.082211 |
MTPR3 | 0.56701 | −4.08063 | −2.81928 | −1.16595 | −1.76765 | 0.001818 | 0.016689 | 0.268294 | 0.104811 |
MIRE1 | 1.223442 | −0.51544 | 1.791224 | −0.51482 | 1.769623 | 0.616449 | 0.10078 | 0.616865 | 0.104467 |
MIRE2 | 1.192328 | −2.95636 | −1.92443 | −2.94614 | −1.96004 | 0.01306 | 0.080545 | 0.013301 | 0.075809 |
MIRE3 | 0.335577 | 3.371002 | −0.64852 | 3.406266 | −0.6847 | 0.006241 | 0.529956 | 0.005864 | 0.507703 |
MIRE4 | 1.172576 | −1.97086 | −1.27725 | −1.96303 | −2.89848 | 0.074422 | 0.227814 | 0.075423 | 0.014484 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Alharthi, M.F.; Akhtar, N. Newly Improved Two-Parameter Ridge Estimators: A Better Approach for Mitigating Multicollinearity in Regression Analysis. Axioms 2025, 14, 186. https://doi.org/10.3390/axioms14030186
Alharthi MF, Akhtar N. Newly Improved Two-Parameter Ridge Estimators: A Better Approach for Mitigating Multicollinearity in Regression Analysis. Axioms. 2025; 14(3):186. https://doi.org/10.3390/axioms14030186
Chicago/Turabian StyleAlharthi, Muteb Faraj, and Nadeem Akhtar. 2025. "Newly Improved Two-Parameter Ridge Estimators: A Better Approach for Mitigating Multicollinearity in Regression Analysis" Axioms 14, no. 3: 186. https://doi.org/10.3390/axioms14030186
APA StyleAlharthi, M. F., & Akhtar, N. (2025). Newly Improved Two-Parameter Ridge Estimators: A Better Approach for Mitigating Multicollinearity in Regression Analysis. Axioms, 14(3), 186. https://doi.org/10.3390/axioms14030186