Ridge-Type Pretest and Shrinkage Estimation Strategies in Spatial Error Models with an Application to a Real Data Example

Abstract

Spatial regression models are widely available across several disciplines, such as functional magnetic resonance imaging analysis, econometrics, and house price analysis. In nature, sparsity occurs when a limited number of factors strongly impact overall variation. Sparse covariance structures are common in spatial regression models. The spatial error model is a significant spatial regression model that focuses on the geographical dependence present in the error terms rather than the response variable. This study proposes an effective approach using the pretest and shrinkage ridge estimators for estimating the vector of regression coefficients in the spatial error mode, considering insignificant coefficients and multicollinearity among regressors. The study compares the performance of the proposed estimators with the maximum likelihood estimator and assesses their efficacy using real-world data and bootstrapping techniques for comparison purposes.

1. Introduction

Data collected over a geographic region may be generally more comparable than data from far away. This phenomenon can be modeled using a covariance structure in conventional statistical models. Spatial regression models, incorporating various spatial dependencies, are increasingly being used in geology, epidemiology, disease monitoring, urban planning, and econometrics.

Time series autoregressive models represent data at time t as linear combinations of the most recent observations. Similarly, in the spatial framework, these models display data from a specific location based on neighboring locations. Data are typically collected from a geographical location known as a site, and proximity is defined by a distance metric.

One of the most used autoregressive models is the spatial error(SE) model, in which a linear regression with a spatially lagged autoregressive error component is used to model the spatial response variable’s mean. Ref. [1] investigated the quantile regression estimation for the SE model with potentially variable coefficients. The authors established the proposed estimators’ asymptotic properties. Ref. [2] applied the SE model to examine the existence of spatial clustering and correlation between neighboring counties for the data from Egypt’s 2006 census. Ref. [3] used the SE model to evaluate the social disorganization theory. Ref. [4] used the combined application of the SE model and spatial lag model based on cross-sectional data from 20 districts in Chengdu. The authors found that the haze had a negative impact on both the selling and rental prices of houses. Ref. [5] proposed a robust estimation method based on SE models, demonstrating reduced bias, a more stable empirical influence function, and robustness to outliers through simulation. More information about the spatial autoregressive models can be found in [6,7,8], among others.

In frequentist statistics, sample information is used to establish inferences about unknown parameters, while Bayesian statistics combines sample information with uncertain prior information (UPI) to draw conclusions. Subjective UPI may not always be available. However, model selection procedures like Akaike’s information criterion (AIC), Bayesian information criterion (BIC), or model selection techniques can still provide UPI.

An initial trial to estimate the regression parameters using sample information and UPI is called a pretest estimation. The pretest estimator selects significant regression coefficients and chooses between the full model estimator or the revised model estimator, which contains fewer coefficients based on a binary weight. A new modification of the pretest estimator, known as the shrinkage estimator, uses smooth weights between full and sub-model estimators to adjust regression coefficient estimates toward a target value impacted by UPI. Nevertheless, the modified shrinkage estimator suffers sometimes from an over-shrinkage phenomenon. Later, an improved version of this estimator, known as positive shrinkage, controls the over-shrinkage issue.

The concept of using pretest and shrinkage estimating methodologies has received considerable attention from many researchers; for example, ref. [9] introduced an efficient estimation using pretest and shrinkage methods to estimate the regression coefficient vector of the marginal model in the case of multinomial responses. Ref. [10] developed different shrinkage and penalty estimation strategies for the negative binomial regression model when over-fitting and uncertain information exists about the subspace. Ref. [11] proposed shrinkage estimation for the parameter vector of the linear regression model with heteroscedastic errors, and extended their study to the high-dimensional heteroscedastic regression model.

Multicollinearity is a major issue when fitting a multiple linear regression model using the ordinary least squares (OLS) method, which arises when some regressor variables are correlated, especially when the correlation between any two is high. There are several techniques discussed in the literature to reduce the risk of this issue. Ref. [12] introduced the concept of ridge regression as a solution to nonorthogonal problems. They showed that the estimator improves the mean square error of estimation. Ref. [13] introduced a new biased estimator and demonstrated, both theoretically and numerically, the improvements of the new one. Ref. [14] proposed a new version of the Liu estimator for the vector of parameters in a linear regression model based on some prior information.

Using the idea of shrinkage, ref. [15] introduced an improved form of the Liu-type estimator. Analytical and numerical results were used to demonstrate the proposed method’s superiority. Ref. [16] suggested the use of the ridge estimator as a suitable approach for handling high-dimensional multicollinearity data. Recently, ref. [17] proposed a novel pretest and shrinkage estimate technique, known as the Liu-type approach, developed for the conditional autoregressive model.

In this article, we propose the ridge-type pretest and shrinkage estimation strategy for the $p\times 1$ regression coefficients vector in the SE model when some prior information is available about the irrelevant coefficients. We will partition the vector $\mathit{\beta }$ as ${({\mathit{\beta }}_1,{\mathit{\beta }}_2)}^T$, where ${\mathit{\beta }}_1$ is a $p_1\times 1$ vector that contains the coefficients of the main effect, and ${\mathit{\beta }}_2$ is a $p_2\times 1$ vector of irrelevant coefficients, with $p_1+p_2=p$. Mainly, we focus on estimating the vector ${\mathit{\beta }}_1$ when the UPI indicates that ${\mathit{\beta }}_2$ is ineffective, which can be achieved by testing a statistical hypothesis of the form $H_0:{\mathit{\beta }}_2=\mathbf{0}$. In some instances, the estimator of the full model may exhibit considerable variability and provide challenges in terms of interpretation. Conversely, the estimator of the sub-model may yield a significantly biased and under-fitted estimate. To tackle this matter, we took into account the pretest, shrinkage, and positive shrinkage ridge estimators for the vector ${\mathit{\beta }}_1$.

In accordance with our goal, this paper is organized as follows. Section 2 offers an overview of the SE model. A discussion about the maximum likelihood estimators for the parameters of the SE model is presented in Section 3. In Section 4, we propose the pretest and shrinkage ridge estimators. Asymptotic analysis of the proposed estimators and some theoretical results are presented in Section 5. The estimators are compared numerically using simulated and real data examples in Section 6. Some concluding remarks are presented in Section 7. An appendix containing some proofs is presented at the end of this manuscript.

2. Spatial Error Model

Let $\mathit{s}=\{s_1,s_2,\dots ,s_n\}$ represent a set of $\left(n\right)$ spatial sites (frequently known as locations, regions, etc.). Set $\mathit{s}$ forms what is commonly referred to as a lattice, and the set of nearby sites for $s_j$, denoted by $K\left(s_j\right)$, is defined as follows: $K\left(s_j\right)=\{s_i:s_i \mathrm{is} \mathrm{a} \mathrm{neighbor} \mathrm{of} s_j\}$, $i=1,2,\dots ,n$. A neighborhood structure can be determined using a predefined adjacency metric. In regular lattices, if two sites just share edges, they are rook-based neighbors; if they also share borders and/or corners, they are queen-based neighbors.

Let ${\mathit{Y}}_n\left(\mathit{s}\right)=\{Y\left(s_1\right),Y\left(s_2\right),\dots ,Y\left(s_n\right)\}$ be a vector of observations collected at sites $\{s_1,s_2,\dots ,s_n\}$, $\mathit{X}\left(\mathit{s}\right)=\left(\mathit{X}\left(s_1\right),\mathit{X}\left(s_2\right),\dots ,\mathit{X}\left(s_n\right)\right)$ be the $(n\times p)$ matrix of covariates, and $\mathit{\beta }={\left({\beta }_1,{\beta }_2,\dots ,{\beta }_p\right)}^T$ be the $(p\times 1)$ vector of unknown regression parameters, known as the large-scale effect. Following Cressie and Wikle [7], the SE model models the response $\mathit{Y}$ at the $j^{th}$ site $s_j$ as follows:

$$\begin{array}{ccc}\hfill \mathit{Y}\left({\mathit{s}}_j\right)& =& {\mathit{X}}^T\left(s_j\right)\mathit{\beta }+ϵ\left(s_j\right),j=1,2,\dots ,n,\hfill \end{array}$$

(1)

$$\begin{array}{ccc}\hfill \mathrm{with}ϵ\left(s_j\right)& =& {\displaystyle \sum _{i\ne j}^n{\lambda }_{ji}ϵ\left(s_i\right)+e\left(s_j\right),j=1,2,\dots ,n,}\hfill \end{array}$$

(2)

where $\mathit{e}\left(\mathit{s}\right)={\left(e\left(s_1\right),e\left(s_2\right),\dots ,e\left(s_n\right)\right)}^T$ is the noise vector that has a Gaussian distribution with a mean of $\mathbf{0}$ and the covariance matrix $\mathsf{\Omega }=diag\{{\sigma }_j^2,j=1,2,\dots ,n\}$. Parameters ${\lambda }_{ji}$ are used to model the spatial dependencies among the errors ${ϵ}_j,j=1,2,\dots ,n$, with ${\lambda }_{jj}=0$. Let $\mathsf{\Lambda }={\left\{{\lambda }_{j,i}\right\}}_{j,i=1}^n$, and assume that $\left(\mathit{I}-\mathsf{\Lambda }\right)$ is invertible, where $\mathit{I}$ is the $(n\times n)$ identity matrix, then by ignoring the spatial indices, the SE in (1) can be rewritten in matrix form, as follows:

$$\begin{array}{c}\hfill \mathit{Y}=\mathit{X}\mathit{\beta }+\mathit{ϵ}\mathrm{with}\mathit{ϵ}\sim N\left(\mathbf{0},{\left(\mathit{I}-\mathsf{\Lambda }\right)}^{-1}\mathsf{\Omega }{\left(\mathit{I}-{\mathsf{\Lambda }}^T\right)}^{-1}\right)\end{array}$$

(3)

Nature exhibits sparsity in many situations, which means that a small number of factors can account for the majority of the observed variability. In the context of regression analysis, sparsity means a few numbers of the coefficients are significantly different from zero, while the bulk of the coefficients are insignificant and remain zero. Sparsity is frequently used in spatial regression models to imply covariance structures that are easier to compute. Consequently, by setting $\mathsf{\Omega }={\sigma }^2\mathit{I}$, and $\mathsf{\Lambda }=\rho \mathit{W}$, where ${\sigma }^2$ is the variance component, $\rho$ is the spatial dependence parameter, and $\mathit{W}$ is the weight or proximity-known matrix with a main diagonal of zeros, and off-diagonal entices $w_{ji}=1$ if the location j is a neighbor to location i; otherwise, $w_{ji}=0 \mathrm{for} j\ne i$, the preceding model yields a straightforward and frequently used version. Usually, the weight matrix is normalized as ${\mathit{W}}^{*}={\left\{\frac{w_{ji}}{w_{j+}}\right\}}_{j,i=1}^n$. So, the SE regression model can be rewritten as follows:

$$\begin{array}{ccc}\hfill \mathit{Y}& =& \mathit{X}\mathit{\beta }+\mathit{ϵ},\mathrm{where}\mathit{ϵ}\sim N(\mathbf{0},{\sigma }^2{\mathit{V}}_n)\hfill \end{array}$$

(4)

$$\begin{array}{ccc}\hfill \mathrm{and}{\mathit{V}}_n& =& {\left(1-\rho {\mathit{W}}^{*}\right)}^{-1}{\left(1-\rho {\mathit{W}}^{*T}\right)}^{-1}\hfill \end{array}$$

(5)

3. Maximum Likelihood Estimation

Let $\mathit{\theta }=\left(\mathit{\beta },{\sigma }^2,\rho \right)$; the maximum likelihood estimator (MLE) of $\mathit{\theta }$ may be acquired by the use of a two-step profile-likelihood method; see [6]. At first, we fix $\rho$ and find the MLEs of $\mathit{\beta },{\sigma }^2$ as a function of $\rho$, which are given as follows:

$$\begin{array}{ccc}\hfill \hat{\mathit{\beta }}\left(\rho \right)& =& {\left({\mathit{X}}^T{\mathit{V}}_n^{-1}\mathit{X}\right)}^{-1}{\mathit{X}}^T{\mathit{V}}_n^{-1}\mathit{Y}\hfill \end{array}$$

(6)

$$\begin{array}{ccc}\hfill {\hat{\sigma }}^2\left(\rho \right)& =& \frac{{(\mathit{Y}-\mathit{X}\hat{\mathit{\beta }}\left(\rho \right))}^T{\mathit{V}}_n^{-1}(\mathit{Y}-\mathit{X}\hat{\mathit{\beta }}\left(\rho \right))}{n}.\hfill \end{array}$$

(7)

Then, we plug $\hat{\mathit{\beta }}$ and ${\hat{\sigma }}^2$ into the log-likelihood and obtain the MLE of $\rho$ by maximizing the profile’s log-likelihood function. Finally, the MLEs of $\mathit{\beta }$ and ${\sigma }^2$ are computed by replacing $\rho$ with $\hat{\rho }$ in Equations (6) and (7), respectively. Ref. [18] proved that $\hat{\mathit{\theta }}$ is a consistent estimator of $\mathit{\theta }$, and asymptotically has normal distribution. This finding makes it simple to demonstrate that $\hat{\mathit{\beta }}$ is asymptotically normal and consistent. The significance of regression coefficients can often be determined subjectively or through certain model selection techniques in various situations. As a result of this information, the $(p\times 1)$ regression coefficient vector $\mathit{\beta }$ is divided into two sub-vectors, as $\mathit{\beta }=({\mathit{\beta }}_1,{\mathit{\beta }}_2)$, where ${\mathit{\beta }}_1$ is a $p_1\times 1$ vector of important coefficients and ${\mathit{\beta }}_2$ is a $p_2\times 1$ vector of unimportant coefficients with $p_1+p_2=p$. Similarly, the matrix of covariates $\mathit{X}$ is also partitioned as $\mathit{X}=\left({\mathit{X}}_1,{\mathit{X}}_2\right)$, where ${\mathit{X}}_1$ and ${\mathit{X}}_2$ consist of the first $p_1$ and the last $p_2$ columns of the design matrix $\mathit{X}$ of dimensions $n\times p_1$ and $n\times p_2$, respectively. Consequently, the SE full model in (3) can be rewritten as follows:

$$\begin{array}{ccc}\hfill \mathit{Y}& =& {\mathit{X}}_1{\mathit{\beta }}_1+{\mathit{X}}_2{\mathit{\beta }}_2+\mathit{ϵ}\hfill \end{array}$$

(8)

For the full model in (8), we can obtain the MLEs of $({\mathit{\beta }}_1,{\mathit{\beta }}_2)$ using the same technique employed in model (4); see [19]. The  MLEs are as follows:

$$\begin{array}{ccc}\hfill {\hat{\mathit{\beta }}}_1& =& {\left({\mathit{X}}_1^T{\mathit{A}}_{{\mathit{X}}_2}{\mathit{X}}_1\right)}^{-1}{\mathit{X}}_1^T{\mathit{A}}_{{\mathit{X}}_2}\mathit{Y}, \mathrm{where}\hfill \end{array}$$

(9)

$$\begin{array}{ccc}\hfill {\mathit{A}}_{{\mathit{X}}_2}& =& {\hat{\mathit{V}}}_n^{-1}-{\hat{\mathit{V}}}_n^{-1}{\mathit{X}}_2{\left({\mathit{X}}_2^T{\hat{\mathit{V}}}_n^{-1}{\mathit{X}}_2\right)}^{-1}{\mathit{X}}_2^T{\hat{\mathit{V}}}_n^{-1}\hfill \end{array}$$

and ${\hat{\mathit{\beta }}}_2$ has an identical formula as ${\hat{\mathit{\beta }}}_1$ by exchanging indices 1 and 2 in the above two equations. The full model estimation may be prone to significant variability and may be difficult to interpret. Our primary goal is to estimate the value of ${\mathit{\beta }}_1$ when the set of regressors included in the partitioned matrix ${\mathit{X}}_2$ does not sufficiently explain the variability in the response variable, which can be achieved by formulating a linear hypothesis as follows:

$$\begin{array}{ccc}\hfill H_0:{\mathit{\beta }}_2& =& \mathbf{0}\hfill \end{array}$$

(10)

Assuming the null hypothesis in (10) is true, the updated model based on this assumption of the model, given (8), becomes

$$\begin{array}{ccc}\hfill \mathit{Y}& =& {\mathit{X}}_1{\mathit{\beta }}_1+\mathit{ϵ}.\hfill \end{array}$$

(11)

We will refer to the model in (11) as the restricted SE model. Let ${\hat{\mathit{\beta }}}_1^S$ be the MLE of ${\mathit{\beta }}_1$ of the model in (11), then

$$\begin{array}{ccc}\hfill {\hat{\mathit{\beta }}}_1^S& =& {\left({\mathit{X}}_1^T{\hat{\mathit{V}}}_n^{-1}{\mathit{X}}_1\right)}^{-1}{\mathit{X}}_1^T{\hat{\mathit{V}}}_n^{-1}\mathit{Y}.\hfill \end{array}$$

(12)

Obviously, ${\hat{\mathit{\beta }}}_1^S$ will have better performance than ${\hat{\mathit{\beta }}}_1$ if the null hypothesis in (10) is true, while the opposite occurs when ${\mathit{\beta }}_2$ begins to move away from the null space. Yet, the restricted strategy method can provide the under-fitted and highly biased model. One goal of this research is to overcome the problem of significant bias in spatial error models when multicollinearity is present among the regressors. To address this issue, we suggest using a ridge-type estimate approach for both the full and sub-models to enhance these estimators by incorporating pretest and shrinkage estimating approaches, to reduce biases and improve the overall accuracy of the estimators. To dominate the large bias, we propose the ridge-type estimation strategy of the full and reduced models and then improve the two estimators using the pretest and shrinkage estimation idea.

4. Materials and Methods: Developing Pretest and Shrinkage Ridge Estimation Strategies

In this section, we propose a set of estimators for the SE model parameter vector ${\mathit{\beta }}_1$ in (11). Following [12], the ridge estimator of $\mathit{\beta }$ for the model given in (4) is defined as

$$\begin{array}{ccc}\hfill {\hat{\mathit{\beta }}}^{RF}& =& {\left({\mathit{X}}^T{\hat{\mathit{V}}}_n^{-1}\mathit{X}+k{\mathit{I}}_p\right)}^{-1}{\mathit{X}}^T{\hat{\mathit{V}}}_n^{-1}\mathit{Y},\hfill \end{array}$$

(13)

where $k>0$ is known as the ridge parameter. Clearly, when $k=0$, the ridge estimator reduces to the MLE of $\mathit{\beta }$, but if $k⟶\infty$, the ridge estimator ${\hat{\mathit{\beta }}}^{RF}=\mathbf{0}$.

4.1. Full and Reduced Model Ridge Estimators

The unrestricted full model ridge estimator of ${\mathit{\beta }}_1$, denoted by ${\hat{\mathit{\beta }}}_1^{\mathrm{UR}}$, is defined as follows:

$$\begin{array}{ccc}\hfill {\hat{\mathit{\beta }}}_1^{\mathrm{UR}}& =& {\left({\mathit{X}}_1^T{\mathit{A}}_{{\mathit{X}}_2}{\mathit{X}}_1+k_f{\mathit{I}}_{p_1}\right)}^{-1}{\mathit{X}}_1^T{\mathit{A}}_{{\mathit{X}}_2}\mathit{Y},\hfill \end{array}$$

(14)

where $k_f$ represents the ridge parameter for the unretracted full model estimator, denoted as ${\hat{\mathit{\beta }}}_1^{\mathrm{UR}}$. Assuming the null hypothesis in (10) is true, the restricted ridge estimator of ${\mathit{\beta }}_1$ for the model in (11), denoted by ${\hat{\mathit{\beta }}}_1^{\mathrm{RR}}$, is given by

$$\begin{array}{ccc}\hfill {\hat{\mathit{\beta }}}_1^{\mathrm{RR}}& =& {\left({\mathit{X}}_1^T{\hat{\mathit{V}}}_n^{-1}{\mathit{X}}_1+k_r{\mathit{I}}_{p_1}\right)}^{-1}{\mathit{X}}_1^T{\hat{\mathit{V}}}_n^{-1}\mathit{Y},\hfill \end{array}$$

(15)

where $k_r$ is the ridge parameter for restricted model estimator ${\hat{\mathit{\beta }}}_1^{\mathrm{RR}}$. When the null hypothesis in (10) is accurate or almost accurate (i.e., when ${\mathit{\beta }}_2$ is close to zero), ${\hat{\mathit{\beta }}}_1^{\mathrm{RR}}$ is generally a more effective estimator than ${\hat{\mathit{\beta }}}_1^{\mathrm{UR}}$. Nevertheless, as ${\mathit{\beta }}_2$ deviates from the zero space, ${\hat{\mathit{\beta }}}_1^{\mathrm{RR}}$ becomes inefficient compared with the unrestricted estimator ${\hat{\mathit{\beta }}}_1^{\mathrm{UR}}$. In addition to the gain obtained by employing the ridge estimation idea to the MLE of ${\mathit{\beta }}_1$, we also aim to find estimators that are functions of ${\hat{\mathit{\beta }}}_1^{\mathrm{UR}}$ and ${\hat{\mathit{\beta }}}_1^{\mathrm{RR}}$, and intend to lessen the dangers connected with any of these two estimators over the majority of the parameter space. The pretest and shrinkage estimators, which will be built in the following subsection, can help with this.

4.2. Pretest, Shrinkage, and Positive Shrinkage Ridge Estimators

In line with testing the null hypothesis in (10), the pretest estimator selects either the full model estimator ${\hat{\mathit{\beta }}}_1^{\mathrm{UR}}$ if $H_0$ is rejected or the restricted ridge estimator ${\hat{\mathit{\beta }}}_1^{\mathrm{RR}}$ if not. An appropriate test statistic to test the hypothesis in (10) is

$$\begin{array}{ccc}\hfill T_n& =& \frac{{\left({\hat{\mathit{\beta }}}_2^{\mathrm{UR}}\right)}^T\left({\mathit{X}}_2^T{\mathit{A}}_{{\mathit{X}}_1}{\mathit{X}}_2\right)\left({\hat{\mathit{\beta }}}_2^{\mathrm{UR}}\right)}{s^2},\hfill \end{array}$$

where ${\mathit{A}}_{{\mathit{X}}_1}$ is defined in a similar manner, as ${\mathit{A}}_{{\mathit{X}}_2}$, ${\hat{\mathit{\beta }}}_2^{\mathrm{UR}}={\left({\mathit{X}}_2^T{\mathit{A}}_{{\mathit{X}}_1}{\mathit{X}}_2\right)}^{-1}{\mathit{X}}_2^T{\mathit{A}}_{{\mathit{X}}_1}\mathit{Y}$, and $s^2={(\mathit{Y}-\mathit{X}{\hat{\mathit{\beta }}}^{RF})}^T(\mathit{Y}-\mathit{X}{\hat{\mathit{\beta }}}^{RF})/(n-p)$, which is a consistent estimator of ${\sigma }^2$, and statistic $T_n$ asymptotically follows a chi-square distribution with $p_2$ degrees of freedom under the null hypothesis. Hence, the pretest estimator, denoted by ${\hat{\mathit{\beta }}}_1^{\mathrm{PTR}}$, is given by

$$\begin{array}{ccc}\hfill {\hat{\mathit{\beta }}}_1^{\mathrm{PTR}}& =& {\hat{\mathit{\beta }}}_1^{\mathrm{UR}}-\left({\hat{\mathit{\beta }}}_1^{\mathrm{UR}}-{\hat{\mathit{\beta }}}_1^{\mathrm{RR}}\right)I\left(T_n\le {\chi }_{\alpha ,p_2}^2\right),\hfill \end{array}$$

(16)

where $I(.)$ is an indicator function, and ${\chi }_{\alpha ,p_2}^2$ is the upper $\alpha$th quantile of the chi-square distribution with $p_2$ degrees of freedom. The pretest estimator depends on the level of the significance $\left(\alpha \right)$, and selects ${\hat{\mathit{\beta }}}_1^{\mathrm{UR}}$ if the null hypothesis is rejected, and ${\hat{\mathit{\beta }}}_1^{\mathrm{RR}}$ otherwise, based on binary weights. These drawbacks can be improved using smoother weights of the two estimators ${\hat{\mathit{\beta }}}_1^{\mathrm{UR}}$ and ${\hat{\mathit{\beta }}}_1^{\mathrm{RR}}$ instead, which is known as the shrinkage estimator. It is denoted by, ${\hat{\mathit{\beta }}}_1^{\mathrm{SR}}$, and given by

$$\begin{array}{ccc}\hfill {\hat{\mathit{\beta }}}_1^{\mathrm{SR}}& =& {\hat{\mathit{\beta }}}_1^{\mathrm{RR}}+\left({\hat{\mathit{\beta }}}_1^{\mathrm{UR}}-{\hat{\mathit{\beta }}}_1^{\mathrm{RR}}\right)\{1-(p_2-2)T_n^{-1}\},p_2\ge 3.\hfill \end{array}$$

(17)

The shrinkage estimator may experience an over-shrinkage in which negative coordinates may be produced whenever $(T_n<p_2-2)$. The positive shrinkage estimator, a modified version of ${\hat{\mathit{\beta }}}_1^{\mathrm{SR}}$, resolves this issue. It is denoted by ${\hat{\mathit{\beta }}}_1^{\mathrm{PSR}}$, and given by

$$\begin{array}{ccc}\hfill {\hat{\mathit{\beta }}}_1^{\mathrm{PSR}}& =& {\hat{\mathit{\beta }}}_1^{\mathrm{RR}}+\left({\hat{\mathit{\beta }}}_1^{\mathrm{UR}}-{\hat{\mathit{\beta }}}_1^{\mathrm{RR}}\right){\{1-(p_2-2)T_n^{-1}\}}^{+},\hfill \end{array}$$

(18)

where $x^{+}=max(x,0).$ It is easy to see that all the pronounced shrinkage estimators satisfy the following general form

$$\begin{array}{c}\hfill {\hat{\mathit{\beta }}}_1^{\mathrm{Shrinkage}}={\hat{\mathit{\beta }}}_1^{\mathrm{UR}}-\left({\hat{\mathit{\beta }}}_1^{\mathrm{UR}}-{\hat{\mathit{\beta }}}_1^{\mathrm{RR}}\right)g\left(T_n\right).\end{array}$$

(19)

Simply, for ${\hat{\mathit{\beta }}}_1^{\mathrm{PTR}}$, ${\hat{\mathit{\beta }}}_1^{\mathrm{SR}}$, and ${\hat{\mathit{\beta }}}_1^{\mathrm{PSR}}$, the corresponding $g(·)$ functions are given by $I\left(T_n\le {\chi }_{\alpha ,p_2}^2\right)$, $(p_2-2)T_n^{-1}$, and $(1-(p_2-2)T_n^{-1})I(T_n\le {\chi }_{\alpha ,p_2}^2)$, respectively.

5. Asymptotic Analysis

In this section, we will study the asymptotic performances of all estimators based on their asymptotic quadratic risks. Our goal is to investigate the behavior of the set of estimators near the null space, so we consider a sequence of local alternatives given by

$$\begin{array}{ccc}\hfill H_{\left(n\right)}:{\mathit{\beta }}_{2\left(n\right)}& =& \frac{\mathit{\xi }}{\sqrt{n}},\mathit{\xi }\in {\Re }^{p_2}, \mathrm{with} \mathit{\xi }\ne \mathbf{0}\hfill \end{array}$$

(20)

Obviously, when $\mathit{\xi }=\mathbf{0}$, the local alternatives in (20) may be simplified to the null hypothesis given in (10). Assuming that $K\left(\mathit{x}\right)$ represents the cumulative distribution function of any estimator of ${\mathit{\beta }}_1$, say ${\hat{\mathit{\beta }}}_1^{*}$, then: ${\displaystyle K\left(\mathit{x}\right)=\underset{n⟶\infty }{lim}{P}_{H_{\left(n\right)}}\left(\sqrt{n}\left({\hat{\mathit{\beta }}}_1^{*}-{\mathit{\beta }}_1\right)\right)}$. Thus, for any $(p_1\times p_1)$ positive definite matrix $\mathit{M}$, the weighted quadratic loss function is defined as

$$\begin{array}{ccc}\hfill W\left({\hat{\mathit{\beta }}}_1^{*},{\mathit{\beta }}_1\right)& =& n{\left({\hat{\mathit{\beta }}}_1^{*}-{\mathit{\beta }}_1\right)}^T\mathit{M}\left({\hat{\mathit{\beta }}}_1^{*}-{\mathit{\beta }}_1\right)\hfill \\ & =& tr\left[\mathit{M}\left[n\left({\hat{\mathit{\beta }}}_1^{*}-{\mathit{\beta }}_1\right){\left({\hat{\mathit{\beta }}}_1^{*}-{\mathit{\beta }}_1\right)}^T\right]\right],\hfill \end{array}$$

where $tr\left(\mathit{A}\right)$ is the trace of the matrix $\mathit{A}$. Define ${\mathit{\vartheta }}_n^{*}=\sqrt{n}\left({\hat{\mathit{\beta }}}_1^{*}-{\mathit{\beta }}_1\right)$, then if ${\vartheta }_n^{*}\stackrel{D}{⟶}{\vartheta }^{*}$, where $\stackrel{D}{⟶}$ denotes the convergence in distribution, then the asymptotic (distributional) quadratic risk (ADQR) of ${\hat{\mathit{\beta }}}_1^{*}$, denoted by $\Gamma \left({\mathit{\beta }}_1^{*}\right)$, is given by

$$\begin{array}{ccc}\hfill \Gamma ({\mathit{\beta }}_1^{*},\mathit{M})& =& E\left({\mathit{\vartheta }}_n^{*T}\mathit{M}{\mathit{\vartheta }}_n^{*}\right)=\int \left({\mathit{x}}_1^T\mathit{M}{\mathit{x}}_1\right)dK\left({\mathit{x}}_1\right)\hfill \end{array}$$

(21)

The asymptotic (distributional) bias (ADB) of ${\hat{\mathit{\beta }}}_1^{*}$ can be obtained via

$$\begin{array}{c}\hfill \mathrm{ADB}\left({\hat{\mathit{\beta }}}_1^{*}\right)=E\left(\underset{n\to \infty }{lim}\sqrt{n}\left({\hat{\mathit{\beta }}}_1^{*}-{\mathit{\beta }}_1\right)\right).\end{array}$$

(22)

To derive asymptotic distributional properties, in addition to the first four assumptions of [18], we set the following regularity conditions:

(A1)

${max}_{1\le i\le n}\frac{1}{n}{\mathit{x}}_i^T{\left({\mathit{X}}^T{\hat{\mathit{V}}}_n^{-1}\mathit{X}\right)}^{-1}{\mathit{x}}_i\to 0$, as $n\to \infty$, where ${\mathit{x}}_i$ is the ith row of $\mathit{X}$.

(A2)

Let ${\mathit{C}}_n={\mathit{X}}^T{\hat{\mathit{V}}}_n^{-1}\mathit{X}$. Then, $\underset{n\to \infty }{lim}\frac{1}{n}{\mathit{C}}_n=\mathit{C}$, where $\mathit{C}$ is the $(p\times p)$ positive definite matrix.

(A3)

Let

$$\begin{array}{ccc}\hfill {\mathit{C}}_n^{-1}& =& {\left(\begin{array}{cc}{\mathit{X}}_1^T{\hat{\mathit{V}}}_n^{-1}{\mathit{X}}_1& {\mathit{X}}_1^T{\hat{\mathit{V}}}_n^{-1}{\mathit{X}}_2\\ {\mathit{X}}_2^T{\hat{\mathit{V}}}_n^{-1}{\mathit{X}}_1& {\mathit{X}}_2^T{\hat{\mathit{V}}}_n^{-1}{\mathit{X}}_2\end{array}\right)}^{-1}\mathrm{and}\hfill \\ \hfill {\mathit{C}}^{-1}& =& {\left(\begin{array}{cc}{\mathit{C}}_{11}& {\mathit{C}}_{12}\\ {\mathit{C}}_{21}& {\mathit{C}}_{22}\end{array}\right)}^{-1}=\left(\begin{array}{cc}{\mathit{C}}_{11.2}^{-1}& -{\mathit{C}}_{11}^{-1}{\mathit{C}}_{12}{\mathit{C}}_{22.1}^{-1}\\ -{\mathit{C}}_{22}^{-1}{\mathit{C}}_{21}{\mathit{C}}_{11.2}^{-1}& {\mathit{C}}_{22.1}^{-1}\end{array}\right),\hfill \end{array}$$

Then, $\underset{n\to \infty }{lim}{\left(\frac{1}{n}{\mathit{C}}_n\right)}^{-1}={\mathit{C}}^{-1}$, where ${\mathit{C}}_{ii.j}={\mathit{C}}_{ii}-{\mathit{C}}_{ij}{\mathit{C}}_{jj}^{-1}{\mathit{C}}_{ji}$ for $i,j=1,2$.

In the following, we refer to the above assumptions as the “named regularity condition (NRC)”.

The primary tool to derive expressions of the asymptotic quadratic risks for the proposed estimators is to find the asymptotic distribution of the unrestricted full model ridge estimator ${\hat{\mathit{\beta }}}_1^{\mathrm{UR}}$ and the restricted ridge estimator ${\hat{\mathit{\beta }}}_1^{\mathrm{RR}}$. To this end, we make use of the following lemma. The proof is provided in Appendix A.

Lemma 1. 

Assume the NRC. If $k/\sqrt{n}\to k_o\ge 0$, then

$$\begin{array}{c}\hfill \sqrt{n}({\hat{\mathit{\beta }}}^{\mathrm{RF}}-\mathit{\beta })\stackrel{D}{\to }{N}_p(-k_o{\mathit{C}}^{-1}\mathit{\beta },{\sigma }^2{\mathit{C}}^{-1}),\end{array}$$

where $\stackrel{D}{\to }$ denotes convergence in distribution. Indeed, Lemma 1 enables us to provide some asymptotic distributional results about the estimators ${\hat{\mathit{\beta }}}_1^{\mathrm{UR}}$ and ${\hat{\mathit{\beta }}}_1^{\mathrm{RR}}$, as presented in the following theorem, which are easy to prove.

Theorem 1. 

Let ${\mathit{\vartheta }}_n^{\left(1\right)}=\sqrt{n}\left({\hat{\mathit{\beta }}}_1^{\mathrm{UR}}-{\mathit{\beta }}_1\right)$, ${\mathit{\vartheta }}_n^{\left(2\right)}=\sqrt{n}\left({\hat{\mathit{\beta }}}_1^{\mathrm{RR}}-{\mathit{\beta }}_1\right)$, and ${\mathit{\vartheta }}_n^{\left(3\right)}=\sqrt{n}\left({\hat{\mathit{\beta }}}_1^{\mathrm{UR}}-{\hat{\mathit{\beta }}}_1^{\mathrm{RR}}\right)$. Assume the local alternatives in (20) and NRC. Then, as $n\to \infty$, we have

1. 

${\mathit{\vartheta }}_n^{\left(1\right)}\sim N_{p_1}(-{\mathit{\eta }}_{11.2},{\sigma }^2{\mathit{C}}_{11.2}^{-1})$

2. 

${\mathit{\vartheta }}_n^{\left(2\right)}\sim N_{p_1}(\mathit{\delta }-{\mathit{\eta }}_{11.2},{\sigma }^2{\mathit{C}}_{11}^{-1})$

3. 

${\mathit{\vartheta }}_n^{\left(3\right)}\sim N_{p_1}(\mathit{\delta },{\sigma }^2({\mathit{C}}_{11.2}^{-1}-{\mathit{C}}_{11}^{-1}))$

4. 

$\left(\begin{array}{c}{\mathit{\vartheta }}_n^{\left(1\right)}\\ {\mathit{\vartheta }}_n^{\left(3\right)}\end{array}\right)\sim N_{2p_1}\left(\left(\begin{array}{c}-{\mathit{\eta }}_{11.2}\\ \mathit{\delta }\end{array}\right), {\sigma }^2\left(\begin{array}{cc}{\mathit{C}}_{11.2}^{-1}& {\mathit{C}}_{11.2}^{-1}-{\mathit{C}}_{11}^{-1}\\ {\mathit{C}}_{11.2}^{-1}-{\mathit{C}}_{11}^{-1}& {\mathit{C}}_{11.2}^{-1}-{\mathit{C}}_{11}^{-1}\end{array}\right)\right)$

5. 

$\left(\begin{array}{c}{\mathit{\vartheta }}_n^{\left(2\right)}\\ {\mathit{\vartheta }}_n^{\left(3\right)}\end{array}\right)\sim N_{2p_1}\left(\left(\begin{array}{c}\mathit{\delta }-{\mathit{\eta }}_{11.2}\\ \mathit{\delta }\end{array}\right), {\sigma }^2\left(\begin{array}{cc}{\mathit{C}}_{11}^{-1}& \mathbf{0}\\ \mathbf{0}& {\mathit{C}}_{11.2}^{-1}-{\mathit{C}}_{11}^{-1}\end{array}\right)\right)$

6. 

E$\left[{\mathit{\vartheta }}_n^{\left(1\right)}|{\mathit{\vartheta }}_n^{\left(3\right)}\right]=-{\mathit{\eta }}_{11.2}+{\vartheta }_n^{\left(3\right)}-\mathit{\delta }$

7. 

$Pr(T_n\le x)=H_q(x;\Delta )$, where $H_q(x;\Delta )$ is the cumulative distribution function of a non-central chi-square distribution with q degrees of freedom and the Δ non-centrality parameter.

Where $\mathit{\eta }={({\mathit{\eta }}_1^T,{\mathit{\eta }}_2^T)}^T=-k_o{\mathit{C}}^{-1}\mathit{\beta }$, ${\mathit{\eta }}_{11.2}={\mathit{\eta }}_1-{\mathit{C}}_{12}{\mathit{C}}_{22}^{-1}\left(({\mathit{\beta }}_2-\mathit{\xi })-{\mathit{\eta }}_2\right)$, $\mathit{\delta }={\mathit{C}}_{11}^{-1}{\mathit{C}}_{12}\mathit{\xi }$.

With Lemma 1 in hand, it is pretty straightforward to reach the asymptotic distributional properties of the shrinkage estimators. Through the subsequent theorems, we will provide the asymptotic bias and weighted quadratic risk functions.

Theorem 2. 

Under the assumptions of Lemma 1, the asymptotic distributional bias of the shrinkage estimators are given by

1. 

$\mathrm{ADB}\left({\hat{\mathit{\beta }}}_1^{\mathrm{PTR}}\right)=-{\mathit{\eta }}_{11.2}-\mathit{\delta }{H}_{p_2+2}({\chi }_{\alpha ,p_2}^2;\Delta )$

2. 

$\mathrm{ADB}\left({\hat{\mathit{\beta }}}_1^{\mathrm{SR}}\right)=-{\mathit{\eta }}_{11.2}-(p_2-2)\mathit{\delta }E\left({\chi }_{p_2+2}^{-2}(\Delta )\right)$

3. 

$\mathrm{ADB}\left({\hat{\mathit{\beta }}}_1^{\mathrm{PSR}}\right)=-{\mathit{\eta }}_{11.2}-\mathit{\delta }{H}_{p_2+2}({\chi }_{\alpha ,p_2}^2;\Delta )+(p_2-2)\mathit{\delta }E\left[{\chi }_{p_2+2}^{-2}(\Delta )I({\chi }_{p_2+2}^2(\Delta )\le p_2-2)\right],$

where

$$E\left({\chi }_q^{-2i}(\Delta )\right)={\int }_{x=0}^{x=\infty }{x}^{-2i}d{H}_q(x;\Delta ),i=1,2.$$

For the proof, refer to Appendix A.

The following results reveal the expressions for the ADQR of the proposed shrinkage estimators.

Theorem 3. 

Under the assumptions of Lemma 1, the asymptotic distributional quadratic risks of the shrinkage estimators are given by

1. 

$$\begin{array}{ccc}\hfill \Gamma ({\hat{\mathit{\beta }}}_1^{\mathrm{PTR}},\mathit{M})& =& \Gamma ({\hat{\mathit{\beta }}}_1^{\mathrm{UR}},\mathit{M})-2{\mathit{\eta }}_{11.2}^T\mathit{M}\mathit{\delta }{H}_{p_2+2}({\chi }_{\alpha ,p_2}^2;\Delta )\hfill \\ & & -{\sigma }^{2}tr\left[\mathit{M}({\mathit{C}}_{11.2}^{-1}-{\mathit{C}}_{11}^{-1})\right]H_{p_2+2}({\chi }_{\alpha ,p_2}^2;\Delta )\hfill \\ & & {\mathit{\delta }}^T\mathit{M}\mathit{\delta }\left[2H_{p_2+2}({\chi }_{\alpha ,p_2}^2;\Delta )-H_{p_2+4}({\chi }_{\alpha ,p_2}^2;\Delta )\right]\hfill \end{array}$$

2. 

$$\begin{array}{ccc}\hfill \Gamma ({\hat{\mathit{\beta }}}_1^{\mathrm{SR}},\mathit{M})& =& \Gamma ({\hat{\mathit{\beta }}}_1^{\mathrm{UR}},\mathit{M})+2(p_2-2){\mathit{\eta }}_{11.2}^T\mathit{M}\mathit{\delta }E\left({\chi }_{p_2+2}^{-2}(\Delta )\right)\hfill \\ & & -(p_2-2){\sigma }^{2}tr\left(\mathit{M}{\mathit{C}}_{11}^{-1}{\mathit{C}}_{12}{\mathit{C}}_{22.1}^{-1}{\mathit{C}}_{21}{\mathit{C}}_{11}^{-1}\right)\hfill \\ & & \left\{2E\left({\chi }_{p_2+2}^{-2}(\Delta )\right)-(p_2-2)E\left({\chi }_{p_2+2}^{-4}(\Delta )\right)\right\}\hfill \\ & +& (p_2-2){\mathit{\delta }}^T\mathit{M}\mathit{\delta }\hfill \\ & & \times \left\{2E\left({\chi }_{p_2+2}^{-2}(\Delta )\right)-2E\left({\chi }_{p_2+4}^{-2}(\Delta )\right)-(p_2-2)E\left({\chi }_{p_2+4}^{-4}(\Delta )\right)\right\}.\hfill \end{array}$$

3. 

$$\begin{array}{ccc}\hfill \Gamma ({\hat{\mathit{\beta }}}_1^{\mathrm{PSR}},\mathit{M})& =& \Gamma ({\hat{\mathit{\beta }}}_1^{\mathrm{SR}},\mathit{M})-2{\mathit{\eta }}_{11.2}^T\mathit{M}\mathit{\delta }{A}_1\hfill \\ & & +(p_2-2){\sigma }^{2}tr\left(\mathit{M}{\mathit{C}}_{11}^{-1}{\mathit{C}}_{12}{\mathit{C}}_{22.1}^{-1}{\mathit{C}}_{21}{\mathit{C}}_{11}^{-1}\right)A_2\hfill \\ & & -{\sigma }^{2}tr\left(\mathit{M}{\mathit{C}}_{11}^{-1}{\mathit{C}}_{12}{\mathit{C}}_{22.1}^{-1}{\mathit{C}}_{21}{\mathit{C}}_{11}^{-1}\right)H_{p_2+2}({\chi }_{\alpha ,p_2}^2;\Delta )\hfill \\ & & +{\mathit{\delta }}^T\mathit{M}\mathit{\delta }\left[2H_{p_2+2}({\chi }_{\alpha ,p_2}^2;\Delta )-H_{p_2+4}({\chi }_{\alpha ,p_2}^2;\Delta )\right]\hfill \\ & & -(p_2-2){\mathit{\delta }}^T\mathit{M}\mathit{\delta }{A}_3,\hfill \end{array}$$

where $\mathit{M}$ is a positive definite weight matrix,

$$\begin{array}{c}\hfill \Gamma ({\hat{\mathit{\beta }}}_1^{\mathrm{UR}},\mathit{M})={\mathit{\eta }}_{11.2}^T\mathit{M}{\mathit{\eta }}_{11.2}+{\sigma }^{2}tr\left(\mathit{M}{\mathit{C}}_{11.2}^{-1}\right), and\end{array}$$

$$\begin{array}{ccc}\hfill A_1& =& E\left\{1-(p_2-2){\chi }_{p_2+2}^{-2}(\Delta )\right\}I\left({\chi }_{p_2+2}^2(\Delta )\le p_2-2\right),\hfill \\ \hfill A_2& =& 2E\left\{{\chi }_{p_2+2}^{-2}(\Delta )I\left({\chi }_{p_2+2}^2(\Delta )\le p_2-2\right)\right\}\hfill \\ & & -(p_2-2)E\left\{{\chi }_{p_2+2}^{-4}(\Delta )I\left({\chi }_{p_2+2}^2(\Delta )\le p_2-2\right)\right\},\hfill \\ \hfill A_3& =& 2E\left\{{\chi }_{p_2+2}^{-2}(\Delta )I\left({\chi }_{p_2+2}^2(\Delta )\le p_2-2\right)\right\}\hfill \\ & & -2E\left\{{\chi }_{p_2+4}^{-2}(\Delta )I\left({\chi }_{p_2+4}^2(\Delta )\le p_2-2\right)\right\}\hfill \\ & & +(p_2-2)E\left\{{\chi }_{p_2+2}^{-4}(\Delta )I\left({\chi }_{p_2+2}^2(\Delta )\le p_2-2\right)\right\}.\hfill \end{array}$$

For the proof, refer to Appendix A.

6. Numerical Analysis

To demonstrate our theoretical findings, we first use Monte Carlo simulation experiments and then apply the set of proposed estimators to a real dataset. The Monte Carlo simulation is used to investigate the performance of the ridge-type set of estimators in comparison to the MLE $\left({\hat{\mathit{\beta }}}_1\right)$ given in (9) via the simulated mean square error of each estimator.

6.1. Simulation Experiments

In this section, we compare the set of ridge-type estimators with respect to the MLE using Monte Carlo simulation experiments based on their simulated mean squared errors. In each of these experiments, we consider $(N\times N)$ square lattices using $N=7,10$ with the corresponding sample sizes of $n=N^2=49,100$, respectively. To show the performance of the proposed estimators when multicollinearity exists, we generate the design matrix $\mathit{X}$ from the multivariate normal distribution with a mean of $\mathbf{0}$, and a variance-covariance matrix with a first-order autoregressive structure, in which $cov({\mathit{X}}_i,{\mathit{X}}_j)=\{\begin{array}{cc}{\rho }_x^{|i-j|}& i\ne j\\ 1& i=j\end{array}$ and apply it to ${\rho }_x\in \{0.3,0.6,0.9\}$, while the error term $\mathit{ϵ}$ is generated from another multivariate normal with a mean of $\mathbf{0}$ and an SE variance matrix with ${\mathit{V}}_n={\sigma }^2{\left(\mathit{I}-\rho {\mathit{W}}^{*}\right)}^{-1}{\left(\mathit{I}-\rho {\mathit{W}}^{*\prime }\right)}^{-1}$. We set ${\sigma }^2=1$. For the weight matrix ${\mathit{W}}^{*}$, a queen-based contiguity neighborhood was used. The set of values for $\rho$ is $\{0.3,0.6,0.9\}$. We partitioned the vector of coefficients $\mathit{\beta }$ as $\mathit{\beta }=\left({\mathit{\beta }}_1,{\mathit{\beta }}_2\right)$, where ${\mathit{\beta }}_1={\mathbf{1}}_{p1}$ is a $p_1\times 1$ vector of ones, and  ${\mathit{\beta }}_2=(\Delta ,{\mathbf{0}}_{p_2-1})$, ${\mathbf{0}}_{p_2-1}$ is a zero vector of dimension $(p_2-1\times 1)$, and $\Delta =\parallel \mathit{\beta }-{\mathit{\beta }}_0\parallel$, where $\parallel \mathit{A}\parallel$ is the Euclidean norm of $\mathit{A}$, and $\Delta$ represents the non-centrality parameter. The range of values for $\Delta$ is set to be from 0 to 2. Then we fit the model in (8) using the spautolm function within the R-package spdep [20], obtain the values of all estimators considered in our study, and compute the simulated mean square error (SMSE) of each estimator as ${\displaystyle SMSE\left(\widehat{{\mathit{\beta }}_1^{*}}\right)=\sum _{i=1}^{p_1}{\left(\widehat{{\mathit{\beta }}_{1i}^{*}}-{\mathit{\beta }}_{1i}\right)}^2}$. The simulated relative efficiency (SRE) of any estimator, say ${\widehat{{\mathit{\beta }}_1}}^{\circ }$, with respect to the MLE $\left(\widehat{{\mathit{\beta }}_1}\right)$, is calculated as follows:

$$\begin{array}{ccc}\hfill SRE\left({\widehat{{\mathit{\beta }}_1}}^{\circ }\right)& =& \frac{SMSE\left(\widehat{{\mathit{\beta }}_1}\right)}{SMSE\left({\widehat{{\mathit{\beta }}_1}}^{\circ }\right)},\hfill \end{array}$$

(23)

where ${\widehat{{\mathit{\beta }}_1}}^{\circ }$ is any of the estimators $\{{\hat{\mathit{\beta }}}_1^{\mathrm{UR}},{\hat{\mathit{\beta }}}_1^{\mathrm{RR}},{\hat{\mathit{\beta }}}_1^{\mathrm{PTR}},{\hat{\mathit{\beta }}}_1^{\mathrm{SR}},{\hat{\mathit{\beta }}}_1^{\mathrm{PSR}}\}$. It is evident that when the $SRE\left({\widehat{{\mathit{\beta }}_1}}^{\circ }\right)$ is greater than one, it signifies that this estimate outperforms the MLE of the full model, and vice versa. We run the simulation for $(p_1,p_2)\in \{(5,10),(5,20),(5,30)\}$, and use $\alpha =0.05$ to test the hypothesis in (20). No statistically significant change is seen while altering the spatial dependency parameter. Therefore, we simply exhibit the graphs for $\rho =0.90$. Figure A1, Figure A2 and Figure A3 in Appendix A show the results of the SRE against various values of $\Delta$. The findings support the following conclusions:

(i)

Across all values, the ridge-type full model estimator $\left({\hat{\mathit{\beta }}}_1^{\mathrm{UR}}\right)$ consistently outperforms the traditional MLE estimator. Furthermore, as $p_2$ increases, so does its efficiency for fixed values of $\rho$ and ${\rho }_x$. Additionally, when the multicollinearity among the explanatory variables in the design matrix becomes stronger, ${\hat{\mathit{\beta }}}_1^{\mathrm{UR}}$ efficiency increases as expected.

(ii)

The ridge-type sub-model estimator $\left({\hat{\mathit{\beta }}}_1^{\mathrm{RR}}\right)$ outperforms all other estimators when $\Delta =0$. Since the null hypothesis is correct, it is expected. However, once $\Delta$ begins to depart from the null space, the estimator’s SRE drops precipitously and approaches zero, making it less effective than the other estimators.

(iii)

The SRE values grow while holding other parameters constant, as the correlation coefficient ${\rho }_x$ increases among the explanatory factors.

(iv)

As the number of zero coefficients $\left(p_2\right)$ increase =, all SRE estimators also increase.

(v)

The ridge-type positive shrinkage estimator $\left({\hat{\mathit{\beta }}}_1^{\mathrm{PSR}}\right)$ uniformly prevails over the competing estimators.

6.2. Data Example

In 1970, ref. [21] examined the use of housing market data for census tracts in the Boston Statistical Metropolitan Area. The authors’ major objective was to establish a relationship between a set of (15) variables and the median cost of owner-occupied residences in Boston. Ref. [22] offered a corrected version of the dataset along with new spatial data. The dataset is accessible through the R-Package spdep version 1.3-1. There are 506 observations in the data, each of which relates to a single census tract. The variables in the data include the tract identification number (TRACT), median owner-occupied housing prices in US dollars (MEDV), corrected median owner-occupied housing prices in US dollars (CMEDV), percentages of residential land zoned for lots larger than 2500 square feet per town (constant for all Boston tracts) (ZN), percentages of non-retail business areas per town (INDUS), average room sizes per home (RM), the percentage of owner-occupied homes built before 1940 (AGE), a dummy variable with two levels, which is 1 if the tract borders the Charles River and 0 otherwise (CHAS), the crime rate per capita (CRIM), the weighted distance to main employment centers (DIS), nitrogen oxide concentration (parts per 10 million) per town (NOX), an accessibility index to radial highway per town (constant for all Boston tracts) (RAD), property tax rate per town ($10,000) (constant for all Boston tracts) (TAX), percentage of the lower-class population (LSTAT), pupil–teacher ratios per town (constant for all Boston tracts) (PTRATIO), and the variable $1000{(b-0.63)}^2$, where b is the proportion of blacks (B). Ref. [23] added the location of each tract in latitude (LAT), and longitude (LON) variables. Assuming an SE model, we can predict the response variable log(CMEDV) using all available variables, which will be referred to as the full SE model. For these data, a variety of selection techniques were used to determine the sub-model. One sub-model that was used by [19] is the model obtained by the adaptive LASSO algorithm, which will be referred to as our SE sub-model. The two models are summarized in

Table 1. Full and sub-model.

Selection Criterion	Model
Full	log(CMEDV) = log(LSTAT) + I(RM^2) + TAX
	+ B + log(RAD) + CHAS + CRIM + PTRATIO
	+ AGE + LAT + LON + log(RAD) + I(NOX^2)
	+ log(DIS) + ZN + INDUS
Sub-model	log(CMEDV) = log(LSTAT) + I(RM^2) + TAX + B + CRIM + PTRATIO

.

Figure 1 displays a colored plot of the correlation coefficients for each variable. The notation (***) indicates high significance with a p-value of less than 0.001. The notation (**) indicates significance at a 1% level, while (*) indicates significance at a 5% level. If none of these symbols are present, it signifies that the correlation coefficient between the two variables is not statistically significant. The CMEDV and a few other factors have a strong linear relationship, as seen in the plot. This plot is useful for examining the strength of linearity between the original response CMEDV and any other variable if it exists. The selected variables by the adaptive LASSO algorithm appear to have strong, medium, and weak relationships with the response variable. Moreover, some variables exhibit collinearity; this issue will show how ridge-type estimators show high performance compared to MLE estimators.

Table 2. Estimated values.

Coefficient	${\hat{\mathit{\beta }}}_1^{\mathbf{UR}}$	${\hat{\mathit{\beta }}}_1^{\mathbf{RR}}$	${\hat{\mathit{\beta }}}_1^{\mathbf{PTR}}$	${\hat{\mathit{\beta }}}_1^{\mathbf{SR}}$	${\hat{\mathit{\beta }}}_1^{\mathbf{PSR}}$
Intercept	3.8310	3.6490	3.8310	3.7882	3.7882
log(LSTAT)	−0.2635	−0.2872	−0.2635	−0.2691	−0.2691
I(RM^2)	0.0081	0.0077	0.0081	0.0080	0.0080
TAX	−0.0005	−0.0003	−0.0005	−0.0005	−0.0005
B	0.0006	0.0005	0.0006	0.0006	0.0006
CRIM	−0.0053	−0.0047	−0.0053	−0.0051	−0.0051
PTRATIO	−0.0168	−0.0157	−0.0168	−0.0165	−0.0165

presents the estimated ridge-type values of the proposed estimators.

To assess the effectiveness of the suggested estimators, we will use two different methods, aiming to provide a reliable and valid evaluation of our results. The first method employs the bootstrapping methodology, whereas the second one involves validation using out-of-sample data.

The bootstrapping technique suggested by [24], computes the mean squared prediction error (MSPE) for any estimator as follows:

• Fit the SE full and sub-models as they appear in Table 1 using the spautolm function and obtain the MLEs of ${\mathit{\beta }}_1$, ${\sigma }^2$, the spatial dependence parameter $\rho$, and the covariance matrix ${\mathit{V}}_n$.
• As the columns of matrices ${\left({\mathit{X}}_1^T{\mathit{A}}_{{\mathit{X}}_2}{\mathit{X}}_1\right)}^{-1}$ and ${({\mathit{X}}_1^T{\hat{\mathit{V}}}_n^{-1}{\mathit{X}}_1)}^{-1}$ are not orthogonal, and the sample size is large, we follow [25] to estimate the tuning ridge parameters for the two estimators, ${\hat{\mathit{\beta }}}_1^{\mathrm{UR}}$ and ${\hat{\mathit{\beta }}}_1^{\mathrm{RR}}$, which are, respectively, given by
  $k_f=\frac{{\hat{\sigma }}^{2}tr{\left(\mathit{X}{1}^T{\mathit{A}}_{{\mathit{X}}_2}\mathit{X}1\right)}^{-1}}{{\left(\widehat{{\mathit{\beta }}_1}\right)}^T{\left(\mathit{X}{1}^T{\mathit{A}}_{{\mathit{X}}_2}\mathit{X}1\right)}^{-1}\widehat{{\mathit{\beta }}_1}}$, and $k_r=\frac{{\hat{\sigma }}^{2}tr{\left(\mathit{X}{1}^T{\hat{\mathit{V}}}_n^{-1}\mathit{X}1\right)}^{-1}}{{\left({\hat{\mathit{\beta }}}_1^S\right)}^T{\left(\mathit{X}{1}^T{\hat{\mathit{V}}}_n^{-1}\mathit{X}1\right)}^{-1}{\hat{\mathit{\beta }}}_1^S}$.
• Use the Cholesky decomposition method to express the matrix $\widehat{{\mathit{V}}_n}$ in a decomposed form as $\widehat{{\mathit{V}}_n}=\mathit{U}{\mathit{U}}^T$, where $\mathit{U}$ is an $(n\times n)$ lower triangular matrix.
• Let $\widehat{\mathit{ϵ}}={\mathit{U}}^{-1}\left(\mathit{Y}-\mathit{X}\hat{\mathit{\beta }}\right)$, where $\widehat{\mathit{ϵ}}=\left({\widehat{ϵ}}_1,{\widehat{ϵ}}_2,\dots ,{\widehat{ϵ}}_n\right)$; we define the centered residual as ${ϵ}_i^c={\widehat{ϵ}}_i-\frac{1}{n}{\sum }_{j=1}^n{\widehat{ϵ}}_j$, and then select with the replacement a sample of size$\left(n\right)$$({ϵ}_1^c,{ϵ}_2^c,\dots ,{ϵ}_n^c)$ to obtain ${\mathit{ϵ}}^{★}=\left({ϵ}_1^{★},{ϵ}_2^{★},\dots ,{ϵ}_n^{★}\right)$.
• Calculate the bootstrapping response value as ${\mathit{Y}}^{★}=\mathit{X}\hat{\mathit{\beta }}+{\mathit{U}}^{-1}{\mathit{ϵ}}^{★}$, and then use it to fit the full and sub-models and obtain bootstrapping estimated values of all estimators.
• Calculate the predicted value of the response variable using each estimator as follows: ${\displaystyle \widehat{y_{ki}^{★}}={\mathit{X}}_1\widehat{{\mathit{\beta }}_1^{*}}+\widehat{{\rho }^{★}}\sum _{j=1}^n{W}_{ij}^{*}\left(\widehat{y_{kj}^{★}}-{\mathit{X}}_j\widehat{{\mathit{\beta }}_1^{*}}\right)}$, where $\widehat{{\mathit{\beta }}_1^{*}}$ represents any of the estimators in the set $\{\widehat{{\mathit{\beta }}_1},{\hat{\mathit{\beta }}}_1^{\mathrm{UR}},{\hat{\mathit{\beta }}}_1^{\mathrm{RR}},{\hat{\mathit{\beta }}}_1^{\mathrm{PTR}},{\hat{\mathit{\beta }}}_1^{\mathrm{SR}},{\hat{\mathit{\beta }}}_1^{\mathrm{PSR}}\}$.
• For the $k^{th}$ bootstrapping sample, we calculate the square root of the mean square prediction error (MSPE) as

  $$\begin{array}{ccc}\hfill MSP{E}_k\left(\widehat{{\mathit{\beta }}_1^{*}}\right)& =& \sqrt{\frac{{\displaystyle \sum _{i=1}^n{(\widehat{y_i^{*}}-y_i)}^2}}{n}},k=1,2,\dots ,B,\hfill \end{array}$$

  (24)

  where B is the number of bootstrapping samples.
• Calculate the relative efficiency (RE) of any estimator with respect to the MLE $\widehat{{\mathit{\beta }}_1}$ as follows:

  $$\begin{array}{ccc}\hfill RE\left(\widehat{{\mathit{\beta }}_1^{•}}\right)& =& \frac{MSPE\left(\widehat{{\mathit{\beta }}_1}\right)}{MSPE\left(\widehat{{\mathit{\beta }}_1^{•}}\right)},\hfill \end{array}$$

  (25)

  where $\widehat{{\mathit{\beta }}_1^{•}}$ is any of the ridge-type proposed estimators. We apply the bootstrapping technique $B=2000$ times.

Table 3. RE of the proposed estimators.

Estimator	${\hat{\mathit{\beta }}}_1^{\mathbf{UR}}$	${\hat{\mathit{\beta }}}_1^{\mathbf{RR}}$	${\hat{\mathit{\beta }}}_1^{\mathbf{PTR}}$	${\hat{\mathit{\beta }}}_1^{\mathbf{SR}}$	${\hat{\mathit{\beta }}}_1^{\mathbf{PSR}}$
$RE$	1.0198	2.9468	2.8624	2.3070	2.3287

summarizes the results of the relative efficiencies, where a relative efficiency value exceeding one indicates the superior performance of the estimator in the denominator.

The second approach is based on out-of-sample data. In general, when using out-of-sample data for non-spatial regression models, it is assumed that the errors are independent. However, the errors in the SE regression model are not independent. Nonetheless, by employing a transformation, we may overcome this challenge. We suggest modifying the SE model to ensure the errors are independent while keeping ${\sigma }^2$ and $\rho$ constant. A related transformation technique in spatial models can be found in [6]. Note that the $(n\times n)$ covariance matrix ${\mathit{V}}_n$ is positive definite, so ${\mathit{V}}_n$ can be rewritten as ${\mathit{V}}_n={\mathit{A}}^T\mathit{A}$, where $\mathit{A}$ is an upper triangular matrix with positive entries on the diagonal; see ([26], p. 338). By multiplying the model in (4) by ${\left({\mathit{A}}^{-1}\right)}^T$, we obtain

$$\begin{array}{ccc}\hfill {\mathit{Y}}^{★}& =& {\mathit{X}}^{★}\mathit{\beta }+{\mathit{ϵ}}^{★},\hfill \end{array}$$

(26)

where ${\mathit{Y}}^{★}={\left({\mathit{A}}^{-1}\right)}^T\mathit{Y}$, ${\mathit{X}}^{★}={\left({\mathit{A}}^{-1}\right)}^T\mathit{X}$, and ${\mathit{ϵ}}^{★}={\left({\mathit{A}}^{-1}\right)}^T\mathit{ϵ}$, with ${\mathit{ϵ}}^{★}\sim N(\mathbf{0},{\sigma }^2{\mathit{I}}_n)$. Practically, we obtain the MLEs of ${\sigma }^2$, and $\rho$, and use these estimates to find the estimated matrix $\hat{\mathit{A}}$. The steps of using out-of-sample data are as follows:

• Create a data frame containing the columns of ${\mathit{X}}^{★}$ and ${\mathit{Y}}^{★}$.
• Divide the data frame into training and testing subsets. The testing data subset is known as the out-of-sample dataset.
• Using the training dataset, we follow the same procedure discussed in Section 3. That is, we divide the training data into two subsets, as ${\mathit{X}}_{train}^{★}=({\mathit{X}}_{1,train}^{★},{\mathit{X}}_{2,train}^{★})$, fit the full and sub-models, and obtain the array of estimators, which are denoted by ${\hat{\mathit{\beta }}}_{1,train}$, ${\hat{\mathit{\beta }}}_{1,train}^{UR}$, ${\hat{\mathit{\beta }}}_{1,train}^{RR}$, ${\hat{\mathit{\beta }}}_{1,train}^{PTR}$, ${\hat{\mathit{\beta }}}_{1,train}^{SR}$, ${\hat{\mathit{\beta }}}_{1,train}^{PSR}$.
• Divide the testing data into two subsets, as ${\mathit{X}}_{test}^{★}=({\mathit{X}}_{1,test}^{★},{\mathit{X}}_{2,test}^{★})$, and calculate the predicted response values as follows:

  $$\begin{array}{ccc}\hfill {\hat{\mathit{Y}}}^{•}& =& {\mathit{X}}_{1,test}^{★}{\hat{\mathit{\beta }}}_{1,train}^{•},\hfill \end{array}$$

  where ${\hat{\mathit{\beta }}}_{1,train}^{•}$ is any of the estimators obtained in step (2).
• Compute the average of the MSPE using Equation (24), replacing ${\widehat{y_i}}^{*}$ with ${\widehat{y_i}}^{•}$, and $y_i$ by $y_{i,test}^{★}$.

We divide the dataset into $80\%$ for the taring set and $20\%$ for the testing set, respectively; we repeat steps (2–5) 2000 times, and then obtain the relative efficiency as in Equation (25).

Table 4. The RE of the proposed estimators using out-of-sample data.

Estimator	${\hat{\mathit{\beta }}}_1^{\mathbf{UR}}$	${\hat{\mathit{\beta }}}_1^{\mathbf{RR}}$	${\hat{\mathit{\beta }}}_1^{\mathbf{PTR}}$	${\hat{\mathit{\beta }}}_1^{\mathbf{SR}}$	${\hat{\mathit{\beta }}}_1^{\mathbf{PSR}}$
$RE$	1.0447	5.1399	1.2816	1.5649	1.5649

shows the relative efficiency results based on out-of-sample data.

Table 3 and Table 4 illustrate the better performance of the sub-model ridge-type estimator $\left({\hat{\mathit{\beta }}}_1^{\mathrm{RR}}\right)$ compared to all other estimators. It is then followed by the pretest estimator $\left({\hat{\mathit{\beta }}}_1^{\mathrm{PTR}}\right)$, demonstrating the correctness of the sub-model that was selected. Also, the ridge-positive shrinkage estimator performs better than the shrinkage one. Furthermore, all ridge-type estimators outperformed the MLE of ${\mathit{\beta }}_1$.

7. Conclusions

This paper discusses the pretest, shrinkage, and positive shrinkage ridge-type estimators of the parameter vector $\left(\mathit{\beta }\right)$ for the SE model when there is a previous suspicion that certain coefficients are insignificant, and multicollinearity exists between two or more regressor variables. To obtain the proposed set of estimators for the main effect vector of coefficients $\left({\mathit{\beta }}_1\right)$, we test the hypothesis $H_0:{\mathit{\beta }}_2=\mathbf{0}$. The proposed estimators were compared analytically via their asymptotic distributional quadratic risks, and numerically through simulation experiments and a real data example.

Our results showed that there is no significant effect of the spatial dependence parameters $\left(\rho \right)$, while the performance of the ridge estimators increases as the correlation among the regressor variables increases. Moreover, the performance of the ridge estimators is always better than the MLE. In addition, the estimator $\left({\hat{\mathit{\beta }}}_1^{\mathrm{RR}}\right)$ dominates all estimators under the null hypothesis $H_0:{\mathit{\beta }}_2=\mathbf{0}$, or when near the null space, and delivers higher efficiency than the other estimators. However, the proposed positive shrinkage ridge estimators $\left({\hat{\mathit{\beta }}}_1^{\mathrm{PSR}}\right)$ perform better than the MLE in all seniors. Further, we apply the set of estimators to a real data example, and use bootstrapping and validation based on out-of-sample data techniques to evaluate their performance based on the relative efficacy of the square root of the mean squared prediction error.

The ridge-type pretest and shrinkage estimators significantly reduced the MSPE. These estimators handled spatial error models well by collecting and minimizing prediction variation. The lower MSPE shows that the proposed estimation strategy makes more accurate and trustworthy forecasts than MLE. This suggests that adding these estimators to the model improves predicted accuracy and model performance. This part of the residual analysis shows that ridge-type estimators are effective at addressing prediction errors in spatial error modeling.

The idea of a ridge-type pretest and shrinkage estimation strategy applies to a wide range of spatial regression models. Regarding continuous data, the method can be used for different models, such as the conditional autoregressive model, simultaneous autoregressive model, and spatial autoregressive moving average model. For discrete spatial data types, it can be applied to generalized linear models with a conditional autoregressive covariance structure, auto-logistic models for binary data, and auto-Poisson and negative binomial models for count data, among others. Lee, L. F [27] considered the estimation of the spatial model, which includes a spatial lag of the dependent variable and spatially autoregressive disturbances and provides the most effective spatial two-stage least squares (2SLS) estimator, which are instrumental variable estimators and optimal in the asymptotic sense. Such a method may be used and benefit from the pretest and shrinkage estimation strategy to improve the estimators of several spatial regression models. Liu and Yang [28] discussed the impact of spatial dependence on the convergence rate of quasi-maximum likelihood (QML) estimators and guided how to rectify the finite sample bias in the spatial error dependence model. Based on our findings, it is expected that employing the ridge-type pretest and shrinkage estimation for the spatial dependence model will be a beneficial addition, and provide better results in terms of the estimators’ biases.