Bayesian Analysis of Partially Linear Additive Spatial Autoregressive Models with Free-Knot Splines

Abstract

This article deals with symmetrical data that can be modelled based on Gaussian distribution. We consider a class of partially linear additive spatial autoregressive (PLASAR) models for spatial data. We develop a Bayesian free-knot splines approach to approximate the nonparametric functions. It can be performed to facilitate efficient Markov chain Monte Carlo (MCMC) tools to design a Gibbs sampler to explore the full conditional posterior distributions and analyze the PLASAR models. In order to acquire a rapidly-convergent algorithm, a modified Bayesian free-knot splines approach incorporated with powerful MCMC techniques is employed. The Bayesian estimator (BE) method is more computationally efficient than the generalized method of moments estimator (GMME) and thus capable of handling large scales of spatial data. The performance of the PLASAR model and methodology is illustrated by a simulation, and the model is used to analyze a Sydney real estate dataset.

1. Introduction

Spatial econometrics models are frequently proposed to analyze spatial data that arise in many disciplines such as urban, real estate, public, agricultural, environmental economics and industrial organizations. These models address relationships across geographic observations caused by spatial autocorrelation in cross-sectional or longitudinal data. Spatial econometrics models have a long history in both econometrics and statistics. Early developments and relevant surveys can be found in Cliff and Ord [1], Anselin [2], Case [3], Cressie [4], LeSage [5,6], Anselin and Bera [7].

Among spatial econometrics models, spatial autoregressive (SAR) models [1] have gained much attention by theoretical econometricians and applied researchers. Many approaches have been used to estimate the SAR models, which include the maximum likelihood estimator (MLE) [8], the generalized method of moment estimator (GMME) [9], and the quasi-maximum likelihood estimator (QMLE) [10]. However, these methods mainly focused on parametric SAR models, which are frequently assumed to be linear, few researchers have explicitly examined non-/semi-parametric SAR models. Indeed, it has been confirmed that a lot of economic variables exhibit highly nonlinear relationships on the dependent variables [11,12,13]. Neglecting the latent nonlinear functional forms often results in an inconsistent estimation of the parameters and misleading conclusions [14].

Although many empirical studies and econometric analyses applying the parametric SAR models ignore latent nonlinear relationships, several nonlinear forms [15,16,17,18] have been considered. Nevertheless, the nonlinear parametric SAR models can at most supply certain protection against some specific nonlinear functional forms. Since the nonlinear function is unknown, it is unavoidable to assume the risk of misspecifying the nonlinear function. As nonparametric techniques advance, the advantage of nonparametric SAR models are often used to model nonlinear economic relationships. However, nonparametric components are only suitable for low dimensional covariates, otherwise the “curse of dimensionality” [19] problem is often encountered. Some nonparametric dimension-reduction tools have been considered to address this problem, for example, single-index model [20], partially linear model [21], the additive model [22], varying-coefficient model [23], among others. In recent years, many researchers have started using the advantage of semiparametric modeling in spatial econometrics. For example, Su and Jin [14] proposed the QMLE for semiparametric partially linear SAR models; Su [24] discussed GMME of semiparametric SAR models; Chen et al. [25] studied a Bayesian method for the semiparametric SAR models; Wei and Sun [26] considered GMME for the space-varying coefficients of a spatial model; Krisztin [27] investigated a novel Bayesian semiparametric estimation for the penalized spline SAR models; Krisztin [28] presented a genetic-algorithms for a nonlinear SAR model; Du et al. [29] established GMME of PLASAR models; Chen and Cheng [30] developed a GMME of a partially linear additive spatial error model.

Semiparametric models have received much attention from both econometrics and statistics owing to the explanatory of the parameters and the flexibility of nonparameters. The partially linear additive (PLA) model is probably one of the most popular among the various semiparametric models. As they can not only avoid the “curse of dimensionality” phenomenon encountered in nonparametric regression but also provide a more flexible structure than the generalized linear models. As a result, the PLA models provide good equilibrium between flexibility of the additive model and interpretation of the partially linear model. Many researchers have considered many approaches to analyze such models: local linear method [31], spline estimation [32,33,34], quantile regression [35,36,37,38], variable selection [39,40,41,42], etc. Characterizing the flexibility of nonparametric forms and attempting to explain the potential nonlinearity of PLASAR models are unique challenges faced by analysts of spatial data.

Combining PLA models with SAR models, we consider a class of PLASAR models for spatial data to capture the linear and nonlinear effects between the related variables in addition to spatial dependence between the neighbors in this article. We specify the prior of all unknown parameters, which led to a proper posterior distribution. The posterior summaries are obtained via MCMC tools. We develop an improved Bayesian method with free-knot splines [43,44,45,46,47,48,49,50,51], along with MCMC techniques to estimate unknown parameters, use a spline approach to approximate the nonparametric functions, and design a Gibbs sampler to explore the joint posterior distributions. Treating the number and the positions of knots as random variables can make the model spatially adaptive is an attractive feature of Bayesian free-knot splines [45,46]. In order to improve the rapidly-convergent performance of our algorithm, we further modify the movement step of Bayesian free-knots splines such that all knots can be repositioned in each iteration instead of only one knot moving. Finally, the performance of the PLASAR model and methodology is illustrated by a simulation, and they are used to analyze real data.

The rest of this paper is organized as follows. In Section 2, we propose the PLASAR model to analyze spatial data and discuss the proposed model’s identifiability condition, then acquire the likelihood function by fitting the nonparametric functions with a Bayesian free-knots splines approach. To provide a Bayesian framework, we specify the priors for the unknown parameters, derive the full conditional posterior distributions of the unknown parameters, modify the movement step of the Bayesian free-knots splines approach to accelerate the convergent performance of our algorithm, and describe the detailed sampling algorithm in Section 3. The applicability and practicality of the PLASA model and methodology for spatial data are evaluated by a simulation study, and the model is used to analyze a real dataset in Section 4. Section 5 concludes the paper with a summary.

2. Methodology

2.1. Model

We begin with the PLASAR model that is defined as

$$y_i=\rho \sum _{l=1}^n{w}_{il}{y}_l+x_i^T\alpha +\sum _{j=1}^p{g}_j\left(z_{ij}\right)+{\epsilon }_i,i=1,\dots ,n,$$

(1)

where $x_i={(x_{i1},\dots ,x_{iq})}^T$ and $z_i={(z_{i1},\dots ,z_{ip})}^T$ are covariate vectors, $y_i$ is a response variable, $w_{il}$ is a specified constant spatial weight, $g_j(·)$ is unknown univariate nonparametric function for $j=1,\dots ,p$, $\alpha ={({\alpha }_1,\dots ,{\alpha }_q)}^T$ is $q\times 1$ vector of the unknown parameters, the unknown spatial parameter $\rho$ reflects the spatial autocorrelation between the neighbors with stability condition $\left|\rho \right|<1$, and ${\epsilon }_i$’s are mutually independent and identically distributed normal with zero mean and variance ${\sigma }^2$. In order to ensure model identifiability of the nonparametric function, it is often assumed that the condition $E\left[g_j\left(z_j\right)\right]=0$ for $j=1,\dots ,p$.

2.2. Likelihood

We plan on approximating unknown functions $g_j(·)$ in (1) by free-knot splines for $j=1,\dots ,p$. Assuming that $g_j(·)$ has a polynomial spline of degree $m_j$ with $k_j$ order interior knots ${\xi }_j={({\xi }_{j1},\dots ,{\xi }_{jk})}^T$ with $a_j<{\xi }_{j1}<\cdots <{\xi }_{jk}<b_j$, i.e.,

$$g_j\left(u_j\right)=\sum _{l=1}^{K_j}{B}_{jl}\left(u_j\right){\beta }_{jl}=B_j^T\left(u_j\right){\beta }_j{u}_j\in [a_j,b_j],$$

(2)

where $K_j=1+m_j+k_j$, the vector of spline basis $B_j\left(u_j\right)={(B_{j1}\left(u_j\right),\dots ,B_{j{K}_j}\left(u_j\right))}^T$ is determined by the knot vector ${\xi }_j$, the spline coefficients ${\beta }_j={({\beta }_{j1},\dots ,{\beta }_{j{K}_j})}^T$ is a $K_j\times 1$ vector, and

$$a_j=\underset{1\le i\le n}{min}\left\{z_{ij}\right\}\mathrm{and}{b}_j=\underset{1\le i\le n}{max}\left\{z_{ij}\right\}$$

(3)

are boundary knots for $j=1,\dots ,p$. Let ${\mathbf{B}}_j={(B_1\left(z_{1j}\right),\dots ,B_n\left(z_{nj}\right))}^T$ and $\mathbf{1}={(1,\dots ,1)}^T$. To achieve identification, we set ${\sum }_{i=1}^n{\sum }_{l=1}^{K_j}{B}_{jl}\left(z_{ij}\right){\beta }_{jl}=0$, which is written as ${\mathbf{1}}^T{\mathbf{B}}_j{\beta }_j=0$. Denote $Q_j={\mathbf{1}}^T{\mathbf{B}}_j$, then the constraint becomes $Q_j{\beta }_j=0$.

It follows that the model (1) can be equivalent to

$$y_i=\rho \sum _{l=1}^n{w}_{il}{y}_j+x_i^T\alpha +\sum _{j=1}^p{B}_j^T\left(u_j\right){\beta }_j+{\epsilon }_i=\rho \sum _{l=1}^n{w}_{il}{y}_j+x_i^T\alpha +B_i^T\left(u\right)\beta +{\epsilon }_i,i=1,\dots ,n,$$

where $\beta ={({\beta }_1^T,\dots ,{\beta }_p^T)}^T$ and $B_i^T\left(u\right)={(B_1^T\left(u_1\right),\dots ,B_p^T\left(u_p\right))}^T$. Then the matrix form of the model (1) can be represented as

$$y=\rho Wy+x^T\alpha +B^T\left(u\right)\beta +\epsilon ,$$

where $x={(x_1,\dots ,x_n)}^T$, $y={(y_1,\dots ,y_n)}^T$, $\epsilon ={({\epsilon }_1,\dots ,{\epsilon }_n)}^T$, $W=\left(w_{il}\right)$ is an $n\times n$ specified constant spatial wight matrix, $K={\sum }_{j=1}^p{K}_j$, and $B^T\left(u\right)$ is an $n\times K$ matrix with $B_i^T\left(u\right)$ as its ith row.

The likelihood function for the PLASAR model is proportional to

$$\begin{array}{ccc}\hfill & & L(\alpha ,\beta ,k,\xi ,{\sigma }^2,\rho |y,x,z)\hfill \\ & \propto & {\sigma }^{-n}\left|I_n-\rho W\right|exp\left\{-\frac{1}{2{\sigma }^2}{[y-\rho Wy-x^T\alpha -B^T\left(u\right)\beta ]}^T[y-\rho Wy-x\alpha -B^T\left(u\right)\beta ]\right\}\hfill \\ & =& {\sigma }^{-n}\left|A\left(\rho \right)\right|exp\left\{-\frac{1}{2{\sigma }^2}{[A\left(\rho \right)y-x\alpha -B^T\left(u\right)\beta ]}^T[A\left(\rho \right)y-x^T\alpha -B^T\left(u\right)\beta ]\right\}\hfill \\ & \doteq & {\sigma }^{-n}\left|A\left(\rho \right)\right|exp\left\{-\frac{1}{2{\sigma }^2}{[A\left(\rho \right)y-\mathfrak{B}(x,u)\theta ]}^T[A\left(\rho \right)y-\mathfrak{B}(x,u)\theta ]\right\},\hfill \end{array}$$

(4)

where $x={(x_1,\dots ,x_n)}^T$, $z={(z_1,\dots ,z_n)}^T$, $\theta ={({\alpha }^T,{\beta }^T)}^T$ is $(q+K)\times 1$ vector of regression coefficient, $\mathfrak{B}(x,u)=(x,B^T\left(u\right))$ is an $n\times (q+K)$ matrix, $A\left(\rho \right)=I_n-\rho W$, and $I_n$ is an identity matrix of order n.

3. Bayesian Estimation

In this section, we consider a Bayesian free-knots splines approach with MCMC techniques to analyze the PLASAR model. We begin with the specification of the prior distributions of all unknown parameters, then the derivations of posterior distributions and the narration of the detailed sampling scheme for all of the unknown parameters. Meanwhile, we modify the movement step of Bayesian free-knots splines approach so that all the knots can be repositioned in each iteration.

3.1. Priors

As we will consider a Bayesian approach with free-knots splines to analyze the PLASAR models, all unknown parameters are assigned prior distributions. Note that besides regression coefficient vectors $\theta$, the spatial autocorrelation coefficient $\rho$ and the quantities ${\sigma }^2$, the number of knots $k={(k_1^T,\dots ,k_p^T)}^T$ and location of knots $\xi ={({\xi }_1^T,\dots ,{\xi }_p^T)}^T$ also need prior distributions in the sense that they are random variables in the Bayesian approach with free-knot splines. We avoid the use of improper prior distributions to prevent improper joint posterior distributions.

For $j=1,\dots ,p$, we followed Poon and Wang [49] by puting a Poisson prior with mean ${\lambda }_j$ for number $k_j$ of the knots

$$\pi \left(k_j\right)=\frac{{\lambda }^{k_j}}{k_j!}{e}^{-{\lambda }_j}$$

and a conditional flat prior for knot location ${\xi }_j$

$$\pi ({\xi }_j\mid k_j)=\frac{k_j!}{{(b_j-a_j)}^{k_j}}{\Delta }_j,$$

where ${\Delta }_j=I\left\{a_j={\xi }_{j0}<{\xi }_{j1}<\cdots <{\xi }_{j{k}_j}<{\xi }_{j,k_{j+1}}=b_j\right\}$, $a_j$ and $b_j$ are defined in (3).

We set a conjugate normal inverse-gamma prior for the unknown parameters $(\theta ,{\sigma }^2)$, which is a composite of inverse-Gamma prior distributions for ${\sigma }^2$

$$\pi \left({\sigma }^2\right)\propto {\left({\sigma }^2\right)}^{-\frac{r_0}{2}-1}exp\left\{-\frac{s_0^2}{2{\sigma }^2}\right\}$$

where $r_0$ and $s_0^2$ are hyperparameters; a conditional normal prior distribution with mean vector $\mathbf{0}$ and covariance matrix ${\tau }_0{\sigma }^2{I}_q$ for $\alpha$

$$\pi \left(\alpha \right|{\sigma }^2,{\tau }_0)\propto {\left(2\pi {\tau }_0{\sigma }^2\right)}^{-\frac{q}{2}}exp\left\{-\frac{{\alpha }^T\alpha }{2{\tau }_0{\sigma }^2}\right\},$$

and a conditional normal prior distribution with mean vector $\mathbf{0}$ and covariance matrix ${\tau }_j{\sigma }^2{I}_{K_j}$ for ${\beta }_j$ with the constraint $Q_j{\beta }_j=0$ as follows:

$$\pi \left({\beta }_j\right|k_j,{\xi }_j,{\tau }_j,{\sigma }^2)\propto {\left(2\pi {\tau }_j{\sigma }^2\right)}^{-\frac{K_j}{2}}exp\left\{-\frac{{\beta }_j^T{\beta }_j}{2{\tau }_j{\sigma }^2}\right\}I\{Q_j{\beta }_j=0\}$$

for $j=1,\dots ,p$. In order to improve the robustness of our method, we choose an inverse-gamma prior

$$\pi \left({\tau }_0\right)\propto {\tau }_0^{-\frac{r_{{\tau }_{\alpha 0}}}{2}-1}exp\left\{-\frac{s_{{\tau }_{\alpha 0}}^2}{2{\tau }_0}\right\}\mathrm{and}\pi \left({\tau }_j\right)\propto {\tau }_j^{-\frac{r_{{\tau }_{{\beta }_{j}0}}}{2}-1}exp\left\{-\frac{s_{{\tau }_{{\beta }_{j}0}}^2}{2{\tau }_j}\right\}$$

for $j=0,1,\dots ,p$, where $r_{{\tau }_0}$ and $s_{{\tau }_0}^2$ are pre-specified hyper-parameters. Throughout this article we set $r_0=s_0^2=1$ to obtain a Cauchy distribution of ${\sigma }^2$ and assign $r_{{\tau }_{\alpha 0}}=r_{{\tau }_{{\beta }_{j}0}}=1$ and $s_{{\tau }_{\alpha 0}}^2=s_{{\tau }_{{\beta }_{j}0}}^2=0.005$ to acquire a highly dispersed inverse gamma prior on ${\tau }_j$ for $j=0,1,\dots ,p$.

In addition, we follow LeSage and Pace [52] by eliciting a uniform prior $\mathrm{U}({\lambda }_{min}^{-1},{\lambda }_{max}^{-1})$ for the spatial autocorrelation coefficient $\rho$

$$\pi \left(\rho \right)\propto 1,$$

where ${\lambda }_{max}$ and ${\lambda }_{min}$ are the maximum and minimum eigenvalues of the standardized spatial weight matrix W, respectively.

Therefore, the joint priors of all of the quantities are defined as

$$\pi (\rho ,\alpha ,\beta ,k,\xi ,{\sigma }^2,\tau )=\pi \left(\rho \right)\pi \left({\sigma }^2\right)\pi \left({\tau }_0\right)\pi \left(\alpha \right|{\sigma }^2,{\tau }_0)\prod _{i=1}^p\pi \left({\beta }_j\right|k_j,{\xi }_j,{\tau }_j,{\sigma }^2)\pi \left(k_j\right)\pi \left({\xi }_j\right|k_j)\pi \left({\tau }_j\right),$$

(5)

where $\tau =({\tau }_0,{\tau }_1,\dots ,{\tau }_p)$ is a hyperparameters vector. For computational convenience, we have treated the hyperparameter vector $\tau$ as a unknown parameter vector.

3.2. The Full Conditional Posterior Distributions of Unknown Quantities

Since the joint posterior distribution of the quantities is very complicated, it is difficult to generate samples directly. To solve this problem, we derive the full conditional posterior distributions of unknown quantities, modify the movement step of Bayesian free-knots splines to speed up the convergence, and describe the detailed sampling method in our algorithm.

It follows from the likelihood function (4) and the joint priors (5) that the conditional posterior distribution of $\rho$ given the remaining unknown parameters is proportional to

$$\begin{array}{ccc}\hfill & & p\left(\rho \right|y,x,z,\alpha ,\beta ,k,\xi ,{\sigma }^2,\tau )\hfill \\ & \propto & \left|A\left(\rho \right)\right|exp\left\{-\frac{1}{2{\sigma }^2}{[A\left(\rho \right)y-x^T\alpha -B^T\left(u\right)\beta ]}^T[A\left(\rho \right)y-x^T\alpha -B^T\left(u\right)\beta ]\right\}.\hfill \end{array}$$

(6)

It is not easy to directly simulate from (6), which does not have the form of any standard density function. Therefore, we prefer the Metropolis–Hastings algorithm [53,54] to solve this difficulty: draw ${\rho }^{*}$ from a truncated Cauchy distribution with location $\rho$ and scale ${\sigma }_{\rho }$ on $(-1,1)$, where ${\sigma }_{\rho }$ is treated as a tuning parameter; and accept the candidate value ${\rho }^{*}$ with probability

$$min\left\{1,\frac{p\left({\rho }^{*}\right|x,y,z,\alpha ,\beta ,k,\xi ,{\sigma }^2,\tau )}{p\left(\rho \right|x,y,z,\alpha ,\beta ,k,\xi ,{\sigma }^2,\tau )}\times C_{\rho }\right\},$$

where

$$C_{\rho }=\frac{arctan[(1-\rho )/{\sigma }_{\rho }]-arctan[(-1-\rho )/{\sigma }_{\rho }]}{arctan[(1-{\rho }^{*})/{\sigma }_{\rho }]-arctan[(-1-{\rho }^{*})/{\sigma }_{\rho }]}.$$

From likelihood function (4) and priors (5), we can see that given $(\rho ,\tau )$, the joint posterior of $(\alpha ,\beta ,k,\xi ,{\sigma }^2)$ is given by

$$\begin{array}{ccc}\hfill & & p(\alpha ,\beta ,k,\xi ,{\sigma }^2|x,y,z,\rho ,\tau )\hfill \\ & \propto & {\sigma }^{-n}exp\left\{-\frac{1}{2{\sigma }^2}{[A\left(\rho \right)y-x^T\alpha -B^T\left(u\right)\beta ]}^T[A\left(\rho \right)y-x^T\alpha -B^T\left(u\right)\beta ]\right\}\hfill \\ & & \times {\sigma }^{-r_0-q-2}exp\left\{-\frac{s_0^2}{2{\sigma }^2}-\frac{{\alpha }^T\alpha }{2{\tau }_0{\sigma }^2}\right\}\times \prod _{j=1}^p{\left(\frac{{\lambda }_j}{b_j-a_j}\right)}^{k_j}{\Delta }_j\hfill \\ & & \times \prod _{j=1}^p{\left(2\pi {\tau }_j{\sigma }^2\right)}^{-\frac{K_j}{2}}exp\left\{-\frac{{\beta }_j^T{\beta }_j}{2{\tau }_j{\sigma }^2}\right\}I\{Q_j{\beta }_j=0\}\hfill \\ & \propto & {\sigma }^{-n-r_0-K-q-2}exp\left\{-\frac{1}{2{\sigma }^2}{[A\left(\rho \right)y-\mathfrak{B}(x,u)\theta ]}^T[A\left(\rho \right)y-\mathfrak{B}(x,u)\theta ]\right\}\hfill \\ & & \times exp\left\{-\frac{s_0^2}{2{\sigma }^2}-\frac{{\theta }^T\mathrm{diag}\left\{{\tau }^{-1}\right\}\theta }{2{\sigma }^2}\right\}I\{Q\beta =\mathbf{0}\}\times \prod _{j=1}^p{\left(\frac{{\lambda }_j{\tau }_j}{b_j-a_j}\right)}^{k_j}{\Delta }_j\hfill \\ & \propto & {|\Xi |}^{-\frac{1}{2}}{\left(S^2+s_0^2\right)}^{-\frac{n+r_0}{2}}\prod _{j=1}^p{\left(\frac{{\lambda }_j{\tau }_j^{-\frac{1}{2}}}{b_j-a_j}\right)}^{k_j}{\Delta }_j\hfill \\ & & \times {\left(S^2+s_0^2\right)}^{\frac{n+r_0}{2}}{\left({\sigma }^2\right)}^{-\frac{n+r_0}{2}-1}exp\left\{-\frac{S^2+s_0^2}{2{\sigma }^2}\right\}\hfill \\ & & \times {\left(2\pi {\sigma }^2\right)}^{-\frac{K+q}{2}}{|\Xi |}^{\frac{1}{2}}exp\left\{-\frac{1}{2{\sigma }^2}{(\theta -\widehat{\theta })}^T\Xi (\theta -\widehat{\theta })\right\}I\{Q\beta =\mathbf{0}\},\hfill \end{array}$$

(7)

where $\theta ={({\alpha }^T,{\beta }^T)}^T$, $\mathrm{diag}\left\{{\tau }^{-1}\right\}=\mathrm{diag}\{{\tau }_0^{-1}{I}_q,{\tau }_1^{-1}{I}_{K_1},\dots ,{\tau }_p^{-1}{I}_{K_p}\}$, $Q=(Q_1,\dots ,Q_p)$, $\Xi =\mathrm{diag}\left\{{\tau }^{-1}\right\}+{\mathfrak{B}}^T(x,u)\mathfrak{B}(x,u)$, $\widehat{\theta }={\Xi }^{-1}{\mathfrak{B}}^T(x,u)A\left(\rho \right)y$, and $S^2=y^{T}A{\left(\rho \right)}^{T}A\left(\rho \right)y-{\widehat{\theta }}^T\Xi \widehat{\theta }$, which gives rise to a marginal posterior distribution

$$p(k,\xi |y,x,z,\rho ,\tau )\propto {|\Xi |}^{-\frac{1}{2}}{\left(S^2+s_0^2\right)}^{-\frac{n+r_0}{2}}\prod _{j=1}^p{\left(\frac{{\lambda }_j{\tau }_j^{-\frac{1}{2}}}{b_j-a_j}\right)}^{k_j}{\Delta }_j.$$

(8)

It is easy to see from (8) that

$$p(k_j,{\xi }_j|y,x,z,\rho ,\alpha ,k_{-j},{\xi }_{-j},{\beta }_{-j},\tau )\propto {\left|{\Xi }_j\right|}^{-\frac{1}{2}}{\left(S_j^2+s_0^2\right)}^{-\frac{n+r_0}{2}}{\left(\frac{{\lambda }_j{\tau }_j^{-\frac{1}{2}}}{b_j-a_j}\right)}^{k_j}{\Delta }_j,j=1,\dots ,p.$$

(9)

where ${\Xi }_j={\tau }_j^{-1}{I}_{K_j}+B_j\left(u_j\right)B_j^T\left(u_j\right)$, $y_j^{\star }=A\left(\rho \right)y-x^T\alpha -B_{-j}^T\left(u_{-j}\right){\beta }_{-j}$, ${\widehat{\beta }}_j={\Xi }_j^{-1}{B}_j\left(u_j\right)y_j^{\star }$, $S_j^2=y_j^{\star T}{y}_j^{\star }-{\widehat{\beta }}_j^T{\Xi }_j{\widehat{\beta }}_j$ and $k_{-j},{\xi }_{-j},{\beta }_{-j}$ are $k,\xi ,\beta$ with $k_j,{\xi }_j,{\beta }_j$ excluded, respectively.

It follows from (7) that the approach of composition [55] can be used to generate ${\sigma }^2$ from a conditional inverse-gamma posterior

$$p\left({\sigma }^2\right|y,x,z,\rho ,\alpha ,k,\xi ,\beta ,\tau )\propto {\left(S^2+s_0^2\right)}^{\frac{n+r_0}{2}}{\left({\sigma }^2\right)}^{-\frac{n+r_0}{2}-1}exp\left\{-\frac{S^2+s_0^2}{2{\sigma }^2}\right\}$$

(10)

and to sample $\theta$ from a conditional normal posterior

$$\begin{array}{c}\hfill p\left(\theta \right|y,x,z,\rho ,k,\xi ,{\sigma }^2,\tau )\propto {\left(2\pi {\sigma }^2\right)}^{-\frac{K+q}{2}}{|\Xi |}^{\frac{1}{2}}exp\left\{-\frac{1}{2{\sigma }^2}{(\theta -\widehat{\theta })}^T\Xi (\theta -\widehat{\theta })\right\}I\{Q\beta =\mathbf{0}\}.\end{array}$$

(11)

It follows from (11) that

$$p\left(\alpha \right|y,x,z,\rho ,k,\xi ,\beta ,{\sigma }^2,\tau )\propto {\left(2\pi {\sigma }^2\right)}^{-\frac{q}{2}}{\left|{\Xi }_0\right|}^{\frac{1}{2}}exp\left\{-\frac{1}{2{\sigma }^2}{(\alpha -\widehat{\alpha })}^T{\Xi }_0(\alpha -\widehat{\alpha })\right\},$$

(12)

where ${\Xi }_0={\tau }_0^{-1}{I}_q+x^{T}x$, $y_0^{\star }=A\left(\rho \right)y-B^T\left(u\right)\beta$, $\widehat{\alpha }={\Xi }_0^{-1}{x}^T{y}_0^{\star }$, and

$$p\left({\beta }_j\right|y,x,z,\rho ,\alpha ,k,\xi ,{\beta }_{-j},{\sigma }^2,\tau )\propto {\left(2\pi {\sigma }^2\right)}^{-\frac{K_j}{2}}{\left|{\Xi }_j\right|}^{\frac{1}{2}}exp\left\{-\frac{1}{2{\sigma }^2}{({\beta }_j-{\widehat{\beta }}_j)}^T{\Xi }_j({\beta }_j-{\widehat{\beta }}_j)\right\}I\{Q_j{\beta }_j=0\},$$

(13)

for $j=1,\dots ,p.$ To achieve identification, we focus on the constraint $Q_j{\beta }_j=0$, which should be imposed on ${\beta }_j$. According to Panagiotelis and Smith [56], drawing ${\beta }_j$ from (13) is equivalent to drawing ${\beta }_j^{\star }$ from a normal distribution with mean vector ${\widehat{\beta }}_j$ and covariance matrix ${\Xi }_j$, then ${\beta }_j^{\star }$ is transformed to ${\beta }_j$ by

$${\beta }_j={\beta }_j^{\star }-{\Xi }_j^{*}{Q}_j^T{\left(Q_j{\Xi }_j^{*}{Q}_j^T\right)}^{-1}{Q}_j{\beta }_j^{\star }.$$

(14)

As it is convenient to sample $({\sigma }^2,\alpha ,\beta )$ from the conditional posterior (10), (12) and (13), we concentrate on sampling from (9). A sampling method is applied, in which the original Bayesian free-knots spline [43,44,45,46,47,48,49,50,51] is used as a reversible-jump sampler [57]. It includes three types of movement: the deletion, the addition and the movement of only one knot [48]. We keep the first two move-types unchanged but improve the movement step through the hit-and-run algorithm [58] so that all the knots can be repositioned in each iteration instead of only one knot: for $j=1,\dots ,p$ select a $k_j$ dimension direction vector $c_j={(c_{j1},\dots ,c_{jk})}^T$ randomly, and define

$${\Omega }_j=\left\{{\omega }_j:{\xi }_j^{*}={\xi }_j+c_j\mathrm{with}{a}_j<{\xi }_{ji}^{*}<b_j,i=1,\dots ,k_j\right\}=\left({\omega }_{j1},{\omega }_{j2}\right);$$

generate ${\omega }_j$ from a Cauchy distribution with location 0 and scale ${\sigma }_{{\xi }_j}$ truncated on $\left({\omega }_{j1},{\omega }_{j2}\right)$, where ${\sigma }_{{\xi }_j}$ acts as a tuning parameter; assign ${\xi }_j^{*}={\xi }_j+c_j$ and reorder all of the knots. The proposed number and positions of knots are finally accepted with probability

$$min\left\{1,A_j{\left(\frac{|{\Xi }_j|}{|{\Xi }_j^{*}|}\right)}^{\frac{1}{2}}\times {\left(\frac{S_j^2+s_0^2}{S_j^{*2}+s_0^2}\right)}^{\frac{n+r_0}{2}}\right\},$$

where ${\Xi }_j^{*}$ and $S_j^{*2}$ correspond to ${\Xi }_j$ and $S_j^2$ in the candidate posterior, respectively, and the factor

$$A_j=\frac{arctan({\omega }_{j2}/{\sigma }_{{\xi }_j})-arctan({\omega }_{j1}/{\sigma }_{{\xi }_j})}{arctan[({\omega }_{j2}-{\omega }_j)/{\sigma }_{{\xi }_j}]-arctan[({\omega }_{j1}-{\omega }_j)/{\sigma }_{{\xi }_j}]}.$$

It is evident that the posterior of hyperparameter ${\tau }_j$ is a conditional inverse-gamma distribution

$$p\left({\tau }_0\right|{\sigma }^2,\alpha )\propto {\tau }^{-\frac{q+r_{{\tau }_{\alpha 0}}}{2}-1}exp\left\{-\frac{s_{{\tau }_{\alpha 0}}^2+{\alpha }^T\alpha /{\sigma }^2}{2{\tau }_0}\right\}$$

(15)

and

$$p\left({\tau }_j\right|{\beta }_j,k_j,{\xi }_j,{\sigma }^2)\propto {\tau }_j^{-\frac{K_j+r_{{\tau }_{{\beta }_{j}0}}}{2}-1}exp\left\{-\frac{s_{{\tau }_{{\beta }_{j}0}}^2+{\beta }_j^T{\beta }_j/{\sigma }^2}{2{\tau }_j}\right\},j=1,\dots ,p,$$

(16)

which can be simulated directly from (15) and (16).

3.3. Sampling Scheme

The Bayesian estimate of $\Theta =\{\rho ,\alpha ,k,\xi ,\beta ,{\sigma }^2,\tau \}$ is obtained by observations generated from the posterior of all unknown quantities by running the Gibbs sampler. Moreover, simulating ${\beta }_j$ from (13) is challenging and nonstandard, and the parameter space on the constraint $Q_j{\beta }_j=0$ for $j=1,\dots ,p$. According to Panagiotelis and Smith [56], it is equivalent that ${\beta }_j^{\star }$ is transformed to ${\beta }_j$ by (14). The MCMC sampling algorithm (Algorithm 1) is described in the following manner.

Algorithm 1 The MCMC sampling algorithm.

Input: Samples ${\left\{(x_i,y_i,z_i)\right\}}_{i=1,\dots ,n}$. Initialization: Initialize ${\Theta }^{\left(0\right)}=\{{\rho }^{\left(0\right)},{\alpha }^{\left(0\right)},k^{\left(0\right)},{\xi }^{\left(0\right)},{\beta }^{\left(0\right)},{\sigma }^{2\left(0\right)},{\tau }^{\left(0\right)}\}$ in the MCMC algorithm, where the unknown parameters are generated from the priors, respectively. MCMC iterations: Given the current state of ${\Theta }^{(t-1)}$ successively, draw ${\Theta }^{\left(t\right)}$ from $p(\Theta |x,y,z)$, for $t=1,2,3,\dots$ The detailed MCMC sampling cycles are outlined in the following manner. (a) Generate ${\rho }^{\left(t\right)}$ from $p\left(\rho \right|x,y,z,{\theta }^{(t-1)},{\alpha }^{(t-1)},k^{(t-1)},{\xi }^{(t-1)},{\sigma }^{2(t-1)},{\tau }^{(t-1)})$; (b) Generate ${\sigma }^{2\left(t\right)}$ from $p\left({\sigma }^2\right|x,y,z,{\alpha }^{\left(t\right)},k^{\left(t\right)},{\xi }^{\left(t\right)},{\rho }^{\left(t\right)},{\tau }^{(t-1)})$; (c) Generate ${\alpha }^{\left(t\right)}$ from $p\left(\alpha \right|x,y,z,{\rho }^{\left(t\right)},k^{(t-1)},{\xi }^{(t-1)},{\beta }^{(t-1)},{\tau }^{(t-1)})$; (d) Generate $(k_j^{\left(t\right)},{\xi }_j^{\left(t\right)})$ from $p(k_j^{\left(t\right)},{\xi }_j^{\left(t\right)}|x,y,z,{\rho }^{\left(t\right)},k_{-j}^{(t-1)},{\xi }_{-j}^{(t-1)},{\beta }_{-j}^{(t-1)}{\tau }^{(t-1)})$ for $j=1,\dots ,p$; (e) Generate ${\beta }_j^{\left(t\right)}$ from $p\left({\beta }_j^{\left(t\right)}\right|x,y,z,{\rho }^{\left(t\right)},{\alpha }^{\left(t\right)},k^{\left(t\right)},{\xi }^{\left(t\right)},{\beta }_{-j}^{(t-1)},{\sigma }^{2\left(t\right)},{\tau }^{(t-1)})$,      and adjust ${\beta }_j^{\left(t\right)}$ according to (14) for $j=1,\dots ,p$; (f) Generate ${\tau }_j^{\left(t\right)}$ from $p\left({\tau }_j\right|k^{\left(t\right)},{\xi }^{\left(t\right)},{\beta }_j^{\left(t\right)},{\sigma }^{2\left(t\right)})$ for $j=1,\dots ,p$; (g) Generate ${\tau }_0^{\left(t\right)}$ from $p\left({\tau }_0\right|{\alpha }^{\left(t\right)},{\sigma }^{2\left(t\right)})$. Output: The MCMC sampling from the conditional posteriors of ${\left\{{\Theta }^{\left(t\right)}\right\}}_{t=1,2,3,\dots }$.

4. Empirical Illustrations

We demonstrate the performance of the PLSISAR model and methodology by a simulation and use them to analyze a real data. We set the Rook weight matrix [2] and the Case weight matrix [3] to examine the influence of the spatial weight matrix W. The Rook weight matrix is sampled from Rook contiguity in [59] by randomly allocating the n spatial units on a lattice of $m\times m$ ($\ge n$) squares, finding the neighbors for the unit, and then row normalizing. Meanwhile, we generated the Case weight matrix from the spatial scenario $W=I_r⨂T_m$ in [3] with m members in a district and r districts, and each neighbor of a member in each district given equal weight [10], where ⨂ is the Kronecker product, $T_m=(1/(m-1))({\mathbf{1}}_m{\mathbf{1}}_m^T-I_m)$ and ${\mathbf{1}}_m={(1,\dots ,1)}^T$ is an m-dimensional vector.

4.1. Simulation

Consider the following PLSISAR models:

$$y_i=\rho \sum _{l=1}^n{w}_{il}{y}_l+x_i^T\alpha +g_1\left(z_{i1}\right)+g_2\left(z_{i2}\right)+{\epsilon }_i,i=1,\dots ,n,$$

where $x_i={(x_{i1},x_{i2})}^T$ follows a bivariate standard normal distribution, $z_i={(z_{i1},z_{i2})}^T$ is a bivariate vector, where $z_{i1}$ and $z_{i2}$ are mutually independent and follow uniform distributions on $(-1,1)$ and $(0,1)$, respectively. The nonparametric functions $g_1\left(z_1\right)=sin\left(\pi z_1\right)$ and $g_2\left(z_2\right)=4z_2(1-z_2^2)-1$, ${\epsilon }_i\sim N(0,{\sigma }^2)$, the parameters are assumed as $\alpha ={(1,-1)}^T$ and two cases of variance ${\sigma }^2=\{0.25,0.75\}$, respectively. We consider three different cases of spatial parameters $\rho =\{0.2,0.5,0.7\}$, which represent the spatial dependence of the response from weak to strong. The sample size of the Case weight matrix and the Rook weight matrix is $(r,m)=\left\{\right(20,5),(80,5\left)\right\}$ and $n=\{100,400\}$, respectively.

In our computation, we run each simulation with 1000 replications, adopt a quadratic B-spline and set hyper-parameters $(r_0,s_0^2,r_{{\tau }_{\alpha 0}},s_{{\tau }_{\alpha 0}}^2)=(1,1,1,0.005)$ and $(r_{{\tau }_{{\beta }_{j}0}},s_{{\tau }_{{\beta }_{j}0}}^2)=(1,0.005)$ for $j=1,\dots ,p$. The initial state of the Markov chain of all unknown parameters is selected as follows. All unknown parameters are sampled from the respective priors by gradually decreasing or increasing the use of tuning parameters ${\sigma }_{\rho }$ and ${\sigma }_{{\xi }_j}$ for $j=1,\dots ,p$ so that the acceptable rates are about 25%. For each replication, we generate 6000 sampled values and then delete the first 2000 sampled values as a burn-in period by running our MCMC sampling. Based on the last 4000 sampled values, we compute the corresponding average of 1000 replications as the posterior mean (mean), the 95% posterior credible intervals (95% CI), and standard error (SE). In addition, the standard derivations (SD) of the estimated posterior mean are calculate to compare them with the mean of the estimated posterior SE.

We evaluate the performance of nonparametric estimators by the integrated squared bias (Bias), the root integrated mean squared errors (SSE), the mean absolute deviation errors (MADE)

$$\mathrm{Bias}\left({\widehat{g}}_j\right)=\int {\left[E{\widehat{g}}_j\left(z\right)-g_j\left(z\right)\right]}^{2}dz,\mathrm{SSE}\left({\widehat{g}}_j\right)={\left\{\int E{\left[{\widehat{g}}_j\left(z\right)-g_j\left(z\right)\right]}^{2}dz\right\}}^{\frac{1}{2}},$$

$${\mathrm{MADE}}_j=\frac{1}{200}\sum _{i=1}^{200}\left|{\widehat{g}}_j\left(z_{ji}\right)-g_j\left(z_{ji}\right)\right|\mathrm{and}\mathrm{MADE}=\frac{1}{p}\sum _{j=1}^p{\mathrm{MADE}}_j$$

for $j=1,\dots ,p$, where mathematical expectations are estimated by their corresponding empirical version, and the integrations are performed applying a Riemannian sum approximation at 200 fixed grid points ${\left\{z_{ji}\right\}}_{i=1}^{200}$ that are equally-spaced chosen from $[a_j,b_j]$. From the model (1), the marginal effects are given by $\frac{\partial y}{\partial x_j}={(I_n-\rho W)}^{-1}{I}_n{\alpha }_j$ for $j=1,\dots ,q$. According to LeSage and Pace [52] suggestions, the mean of either the row sums or the column sums of the non-diagonal elements is used as the indirect effects, the mean of the diagonal elements is used as the direct effects, and the sum of the indirect and direct effects is taken as the total effects.

To check the convergence of our algorithm, we run five Markov chains corresponding to different starting values through the MCMC sampling algorithm to perform each replication. The sampled traces of some parameters and nonparametric functions on grid points are displayed in Figure 1. It is obvious that the five parallel sequences mix quite well. We compute the “potential scale reduction factor” $\sqrt{\widehat{R}}$ [60] for all unknown parameters and nonparametric functions at 20 selected grid points. Figure 2 shows all the values of $\sqrt{\widehat{R}}$ against the iteration numbers. According to the suggestion of Gelman and Rubin [60], it is easy to see that 2000 burn-in iterations are enough to make the MCMC algorithm converge as all the values of $\sqrt{\widehat{R}}$ were less than 1.2.

The boxplots of the Bias values are displayed in Figure 3. Under the Rook weight matrix, the medians of which are ${\mathrm{Bias}}_1=0.0147$ and ${\mathrm{Bias}}_2=0.0104$ for $n=100$, and ${\mathrm{Bias}}_1=0.0039$ and ${\mathrm{Bias}}_2=0.0030$ for $n=400$, respectively. Under the Case weight matrix, the medians of which are ${\mathrm{Bias}}_1=0.0148$ and ${\mathrm{Bias}}_2=0.0104$ for $(r,m)=(20,5)$, and ${\mathrm{Bias}}_1=0.0038$ and ${\mathrm{Bias}}_2=0.0030$ for $(r,m)=(80,5)$, respectively. Figure 3 show the boxplots of the SEE values. Under the Rook weight matrix, the medians are ${\mathrm{SSE}}_1=0.2490$ and ${\mathrm{SSE}}_2=0.2271$ for $n=100$, and ${\mathrm{SSE}}_1=0.1253$ and ${\mathrm{SSE}}_2=0.1151$ for $n=400$, respectively. Under the Case weight matrix, the medians are ${\mathrm{SSE}}_1=0.2495$ and ${\mathrm{SSE}}_2=0.2267$ for $(r,m)=(20,5)$, and ${\mathrm{SSE}}_1=0.1257$ and ${\mathrm{SSE}}_2=0.1154$ for $(r,m)=(80,5)$, respectively. The results show that the Bias values and the SEE values of the nonparametric functions decrease with the increase in the sample size, indicating that the nonparametric estimation is convergent. It is evident that the weight matrix of Case and Rook can obtain a reasonable estimation effect.

The estimation results are reported in

Table 1. Simulation results of parametric estimation.

Parameter	n	Rook Weight Matrix				$(\mathit{r},\mathit{m})$	Case Weight Matrix
		Mean	SE	SD	$95\%$ CI		Mean	SE	SD	$95\%$ CI
$\rho =0.2000$	100	$0.2018$	$0.0552$	$0.0492$	$(0.0929,0.3099)$	(20, 5)	$0.1987$	$0.0512$	$0.0473$	$(0.0973,0.2990)$
${\alpha }_1=1.0000$		$0.9930$	$0.0649$	$0.0623$	$(0.8655,1.1203)$		$0.9930$	$0.0648$	$0.0620$	$(0.8656,1.1202)$
${\alpha }_2=-1.0000$		$-0.9956$	$0.0650$	$0.0633$	$(-1.1231,-0.8680)$		$-0.9955$	$0.0650$	$0.0632$	$(-1.1202,-0.8679)$
${\sigma }^2=0.2500$		$0.2716$	$0.0447$	$0.0381$	$(0.1980,0.3723)$		$0.2714$	$0.0447$	$0.0382$	$(0.1979,0.3720)$
Total effect
$x_1=1.2500$		$1.2544$	$0.1188$	$0.1064$	$(1.0378,1.5042)$		$1.2487$	$0.1134$	$0.1083$	$(1.0340,1.4853)$
$x_2=-1.2500$		$-1.2581$	$0.1189$	$0.1115$	$(-1.5083,-1.0414)$		$-1.2516$	$0.1136$	$0.1057$	$(-1.4886,-1.0424)$
$\rho =0.5000$		$0.5000$	$0.0462$	$0.0414$	$(0.4081,0.5896)$		$0.4986$	$0.0344$	$0.0318$	$(0.4304,0.5655)$
${\alpha }_1=1.0000$		$0.9930$	$0.0652$	$0.0627$	$(0.8650,1.1210)$		$0.9931$	$0.0652$	$0.0620$	$(0.8651,1.1211)$
${\alpha }_2=-1.0000$		$-0.9957$	$0.0653$	$0.0633$	$(-1.1239,-0.8674)$		$-0.9957$	$0.0653$	$0.0635$	$(-1.1239,-0.8676)$
${\sigma }^2=0.2500$		$0.2719$	$0.0450$	$0.0383$	$(0.1980,0.3731)$		$0.2717$	$0.0453$	$0.0384$	$(0.1978,0.3728)$
Total effect
$x_1=2.0000$		$2.0153$	$0.2283$	$0.1971$	$(1.6229,2.4969)$		$1.9972$	$0.1818$	$0.1734$	$(1.6633,2.3758)$
$x_2=-2.0000$		$-2.0215$	$0.2311$	$0.2053$	$(-2.5037,-1.6281)$		$-2.0018$	$0.1821$	$0.1687$	$(-2.3817,-1.6673)$
$\rho =0.7000$		$0.6994$	$0.0348$	$0.0319$	$(0.6300,0.7667)$		$0.6988$	$0.0217$	$0.0200$	$(0.6557,0.7409)$
${\alpha }_1=1.0000$		$0.9932$	$0.0655$	$0.0629$	$(0.8648,1.1219)$		$0.9934$	$0.0656$	$0.0621$	$(0.8648,1.1222)$
${\alpha }_2=-1.0000$		$-0.9959$	$0.0655$	$0.0635$	$(-1.1245,-0.8673)$		$-0.9934$	$0.0656$	$0.0639$	$(-1.1249,-0.8671)$
${\sigma }^2=0.2500$		$0.2721$	$0.0451$	$0.0385$	$(0.1981,0.3738)$		$0.2718$	$0.0451$	$0.0386$	$(0.1977,0.3736)$
Total effect
$x_1=3.3333$		$3.3823$	$0.4428$	$0.3911$	$(2.6360,4.3697)$		$3.3273$	$0.3034$	$0.2895$	$(2.7702,3.9615)$
$x_2=-3.3333$		$-3.3924$	$0.4434$	$0.4034$	$(-4.3811,-2.6451)$		$-3.3346$	$0.3037$	$0.2810$	$(-3.9697,-2.7772)$
$\rho =0.2000$	100	$0.1982$	$0.0766$	$0.0750$	$(0.0469,0.3471)$	(20, 5)	$0.1914$	$0.0713$	$0.0738$	$(0.0506,0.3293)$
${\alpha }_1=1.0000$		$0.9839$	$0.1108$	$0.1075$	$(0.7663,1.2011)$		$0.9840$	$0.1107$	$0.1071$	$(0.7665,1.2013)$
${\alpha }_2=-1.0000$		$-0.9882$	$0.1108$	$0.1095$	$(-1.2059,-0.7708)$		$-0.9878$	$0.1108$	$0.1093$	$(-1.2050,-0.7701)$
${\sigma }^2=0.7500$		$0.8234$	$0.1385$	$0.1386$	$(0.5957,1.1359)$		$0.8230$	$0.1393$	$0.1391$	$(0.5955,1.1353)$
Total effect
$x_1=1.2500$		$1.2487$	$0.1858$	$0.1743$	$(0.9164,1.6442)$		$1.2369$	$0.1767$	$0.1771$	$(0.9169,1.6083)$
$x_2=-1.2500$		$-1.2552$	$0.1865$	$0.1841$	$(-1.6525,-0.9221)$		$-1.2408$	$0.1771$	$0.1734$	$(-1.6130,-0.9189)$
$\rho =0.5000$		$0.4947$	$0.0637$	$0.0630$	$(0.3676,0.6170)$		$0.4934$	$0.0480$	$0.0498$	$(0.3975,0.5853)$
${\alpha }_1=1.0000$		$0.9847$	$0.1112$	$0.1082$	$(0.7667,1.2031)$		$0.9849$	$0.1112$	$0.1069$	$(0.7667,1.2033)$
${\alpha }_2=-1.0000$		$-0.9890$	$0.1112$	$0.1095$	$(-1.2073,-0.7707)$		$-0.9887$	$0.1113$	$0.1097$	$(-1.2070,-0.7703)$
${\sigma }^2=0.7500$		$0.8250$	$0.1398$	$0.1402$	$(0.5955,1.1404)$		$0.8248$	$0.1404$	$0.1407$	$(0.5951,1.1405)$
Total effect
$x_1=2.0000$		$2.0079$	$0.3449$	$0.3154$	$(1.4275,2.7545)$		$1.9782$	$0.2826$	$0.2831$	$(1.4639,2.5733)$
$x_2=-2.0000$		$-2.0187$	$0.3449$	$0.3321$	$(-2.7676,-1.4365)$		$-1.9844$	$0.2830$	$0.2778$	$(-2.5805,-1.4696)$
$\rho =0.7000$		$0.6942$	$0.0479$	$0.0477$	$(0.5984,0.7853)$		$0.6955$	$0.0301$	$0.0314$	$(0.6351,0.7528)$
${\alpha }_1=1.0000$		$0.9854$	$0.1115$	$0.1086$	$(0.7670,1.2044)$		$0.9857$	$0.1116$	$0.1074$	$(0.7673,1.2052)$
${\alpha }_2=-1.0000$		$-0.9896$	$0.1116$	$0.1099$	$(-1.2090,-0.7708)$		$-0.9857$	$0.1117$	$0.1104$	$(-1.2087,-0.7705)$
${\sigma }^2=0.7500$		$0.8268$	$0.1417$	$0.1417$	$(0.5959,1.1454)$		$0.8263$	$0.1420$	$0.1418$	$(0.5949,1.1455)$
Total effect
$x_1=3.3333$		$3.3768$	$0.6705$	$0.6079$	$(2.3087,4.8873)$		$3.2961$	$0.4704$	$0.4722$	$(2.4414,4.2873)$
$x_2=-3.3333$		$-3.3950$	$0.6743$	$0.6351$	$(-4.9107,-2.3229)$		$-3.3061$	$0.4716$	$0.4637$	$(-4.2986,-2.4486)$
$\rho =0.2000$	400	$0.1996$	$0.0245$	$0.0235$	$(0.1514,0.2475)$	$(80,5)$	$0.2006$	$0.0218$	$0.0208$	$(0.1577,0.2431)$
${\alpha }_1=1.0000$		$1.0005$	$0.0297$	$0.0286$	$(0.9422,1.0587)$		$1.0005$	$0.0297$	$0.0287$	$(0.9423,1.0587)$
${\alpha }_2=-1.0000$		$-0.9972$	$0.0297$	$0.0290$	$(-1.0554,-0.9389)$		$-0.9972$	$0.0297$	$0.0291$	$(-1.0554,-0.9390)$
${\sigma }^2=0.2500$		$0.2549$	$0.0186$	$0.0185$	$(0.2210,0.2939)$		$0.2549$	$0.0186$	$0.0185$	$(0.2210,0.2939)$
Total effect
$x_1=1.2500$		$1.2521$	$0.0524$	$0.0503$	$(1.1525,1.3578)$		$1.2532$	$0.0496$	$0.0477$	$(1.1584,1.3527)$
$x_2=-1.2500$		$-1.2480$	$0.0524$	$0.0514$	$(-1.3539,-1.1484)$		$-1.2490$	$0.0494$	$0.0480$	$(-1.3484,-1.1546)$
$\rho =0.5000$		$0.4995$	$0.0207$	$0.0198$	$(0.4587,0.5399)$		$0.5003$	$0.0148$	$0.0140$	$(0.4712,0.5290)$
${\alpha }_1=1.0000$		$1.0006$	$0.0299$	$0.0287$	$(0.9420,1.0590)$		$1.0005$	$0.0299$	$0.0288$	$(0.9419,1.0590)$
${\alpha }_2=-1.0000$		$-0.9972$	$0.0299$	$0.0287$	$(-1.0558,-0.9387)$		$-0.9972$	$0.0299$	$0.0292$	$(-1.0557,-0.9387)$
${\sigma }^2=0.2500$		$0.2550$	$0.0187$	$0.0186$	$(0.2210,0.2941)$		$0.2549$	$0.0187$	$0.0186$	$(0.2209,0.2941)$
Total effect
$x_1=2.0000$		$2.0051$	$0.0974$	$0.0932$	$(1.8227,2.2045)$		$2.0049$	$0.0797$	$0.0765$	$(1.8526,2.1651)$
$x_2=-2.0000$		$-1.9984$	$0.0973$	$0.0949$	$(-2.1977,-1.8162)$		$-1.9983$	$0.0795$	$0.0772$	$(-2.1580,-1.8465)$
$\rho =0.7000$		$0.6996$	$0.0158$	$0.0151$	$(0.6684,0.7304)$		$0.7001$	$0.0093$	$0.0087$	$(0.6817,0.7183)$
${\alpha }_1=1.0000$		$1.0006$	$0.0300$	$0.0288$	$(0.9419,1.0594)$		$1.0004$	$0.0300$	$0.0289$	$(0.9416,1.0593)$
${\alpha }_2=-1.0000$		$-0.9972$	$0.0300$	$0.0291$	$(-1.0560,-0.9385)$		$-0.9971$	$0.0300$	$0.0293$	$(-1.0560,-0.9384)$
${\sigma }^2=0.2500$		$0.2550$	$0.0187$	$0.0183$	$(0.2210,0.2942)$		$0.2549$	$0.0188$	$0.0187$	$(0.2208,0.2942)$
Total effect
$x_1=3.3333$		$3.3475$	$0.1920$	$0.1833$	$(2.9942,3.7466)$		$3.3415$	$0.1335$	$0.1272$	$(3.0867,3.6098)$
$x_2=-3.3333$		$-3.3364$	$0.1917$	$0.1862$	$(-3.7347,-2.9837)$		$-3.3304$	$0.1332$	$0.1283$	$(-3.5980,-3.0764)$
$\rho =0.2000$	400	$0.1986$	$0.0366$	$0.0365$	$(0.1266,0.2701)$	$(80,5)$	$0.1994$	$0.0326$	$0.0323$	$(0.1354,0.2630)$
${\alpha }_1=1.0000$		$0.9999$	$0.0512$	$0.0493$	$(0.8996,1.1001)$		$0.9999$	$0.0512$	$0.0495$	$(0.8997,1.1001)$
${\alpha }_2=-1.0000$		$-0.9942$	$0.0512$	$0.0501$	$(-1.0945,-0.8940)$		$-0.9943$	$0.0512$	$0.0501$	$(-1.0945,-0.8942)$
${\sigma }^2=0.7500$		$0.7597$	$0.0555$	$0.0558$	$(0.6587,0.8760)$		$0.7596$	$0.0555$	$0.0560$	$(0.6586,0.8757)$
Total effect
$x_1=1.2500$		$1.2527$	$0.0847$	$0.0827$	$(1.0936,1.4257)$		$1.2530$	$0.0807$	$0.0786$	$(1.1000,1.4165)$
$x_2=-1.2500$		$-1.2457$	$0.0847$	$0.0843$	$(-1.4187,-1.0866)$		$-1.2459$	$0.0805$	$0.0800$	$(-1.4090,-1.0936)$
$\rho =0.5000$		$0.4982$	$0.0306$	$0.0306$	$(0.4376,0.5578)$		$0.4995$	$0.0219$	$0.0216$	$(0.4563,0.5420)$
${\alpha }_1=1.0000$		$1.0001$	$0.0514$	$0.0494$	$(0.8994,1.1007)$		$1.0000$	$0.0514$	$0.0497$	$(0.8994,1.1007)$
${\alpha }_2=-1.0000$		$-0.9944$	$0.0513$	$0.0501$	$(-1.0950,-0.8939)$		$-0.9943$	$0.0514$	$0.0502$	$(-1.0951,-0.8938)$
${\sigma }^2=0.7500$		$0.7600$	$0.0559$	$0.0564$	$(0.6585,0.8770)$		$0.7598$	$0.0559$	$0.0567$	$(0.6582,0.8769)$
Total effect
$x_1=2.0000$		$2.0068$	$0.1541$	$0.1509$	$(1.7231,2.3272)$		$2.0048$	$0.1293$	$0.1256$	$(1.7601,2.2668)$
$x_2=-2.0000$		$-1.9956$	$0.1538$	$0.1538$	$(-2.3151,-1.7121)$		$-1.9935$	$0.1289$	$0.1280$	$(-2.2544,-1.7496)$
$\rho =0.7000$		$0.6984$	$0.0232$	$0.0234$	$(0.6524,0.7435)$		$0.6996$	$0.0137$	$0.0136$	$(0.6725,0.7262)$
${\alpha }_1=1.0000$		$1.0002$	$0.0515$	$0.0496$	$(0.8993,1.1012)$		$1.0001$	$0.0516$	$0.0499$	$(0.8991,1.1013)$
${\alpha }_2=-1.0000$		$-0.9945$	$0.0515$	$0.0502$	$(-1.0956,-0.8937)$		$-0.9944$	$0.0516$	$0.0504$	$(-1.0955,-0.8936)$
${\sigma }^2=0.7500$		$0.7603$	$0.0563$	$0.0569$	$(0.6581,0.8782)$		$0.7600$	$0.0564$	$0.0573$	$(0.6575,0.8781)$
Total effect
$x_1=3.3333$		$3.3537$	$0.2978$	$0.2935$	$(2.8195,3.9858)$		$3.3415$	$0.2155$	$0.2091$	$(2.9334,3.7779)$
$x_2=-3.3333$		$-3.3351$	$0.2970$	$0.2967$	$(-3.9661,-2.8023)$		$-3.3226$	$0.2150$	$0.2134$	$(-3.7580,-2.9153)$

. We observe that the mean values of all estimators are very close to the corresponding true values, and the mean value of SE is close to the respective SD. The results show that the parameter estimation and SE are precise. Meanwhile, the larger the sample sizes under the same weight matrix, the more precise the estimates are. The above experiences corresponding to different starting values have been repeated, and the results are similar. It implies that the MCMC sampling works well. Moreover, we find that the estimation effect of $\rho$ with the Case weight matrix is slightly better than that with Rook weight matrix under the same sample sizes. The possible main reason is that the performance of the Case weight matrix is superior to the Rook weight matrix under different variances ${\sigma }^2$. In addition, the general pattern of estimates reported in Table 1 is that all estimators impose a relatively bigger bias on the total effect when the same sample sizes have a strong positive spatial dependence. Figure 4 depicts the fitted functions, together with its 95% CI from a typical sample with $\rho =0.5$ and ${\sigma }^2=0.25$. The typical sample is selected in such a way that the SSE values are equal to the median in the 1000 replications. It is obvious that the fitted nonparametric functions are improving with increasing the sample size.

For comparison purposes, we use the Bayesian P-splines approach to approximate the nonparametric functions [61], where we assign a second-order random walk prior to the spline coefficients. The boxplots of MADE values with the Case weight matrix in Figure 5. In our method, the medians of MADE are ${\mathrm{MADE}}_1=0.0997$, ${\mathrm{MADE}}_2=0.0804$ and $\mathrm{MADE}=0.0916$ for $(r,m)=(20,5)$, and the medians of MADE are ${\mathrm{MADE}}_1=0.0504$, ${\mathrm{MADE}}_2=0.0433$ and $\mathrm{MADE}=0.0476$ for $(r,m)=(80,5)$, respectively, which are slightly smaller than the Bayesian P-splines approach. The results show that the Bayesian free knots splines approach is superior to the Bayesian P-splines approach in terms of fitting unknown nonparametric functions and computing time. Furthermore, we also compare the performance between the generalized method of moment estimator (GMME) in Du et al. [29] and the Bayesian MCMC estimator (BE) in our method. In order to evaluate the estimation effect of the nonparametric functions, we calculate the integrated squared bias (Bias) and the root integrated mean squared errors (SSE).

Table 2. Simulation results of the nonparametric estimation.

Functions	$(\mathit{r},\mathit{m})$	With GMME		With BE
		Bias	SSE	Bias	SSE
$g_1(·)$	(60, 5)	$0.014$	$7.947$	0.006	0.146
$g_2(·)$		$0.009$	$7.246$	0.007	0.141
$g_1(·)$	(80, 5)	$0.011$	$6.860$	$0.004$	$0.131$
$g_2(·)$		$0.004$	$6.235$	$0.001$	$0.117$

reports the results of the nonparameter estimation for GMME and BE (only a replication with $(\rho ,{\sigma }^2)=(0.5,0.25)$ is displayed). It is evident that the estimates are improving with increasing the sample size, the Bias of the BE estimates are slightly smaller than the Bias of the GMME, and the SSE of the BE estimates are very smaller than Bias of the GMME under the same sample size, showing that BE is better than GMME, although the latter can also obtain a reasonable estimation.

4.2. Application

We use the proposed model and estimation methods to analyze the well-known Sydney real estate data. A detailed description of the data set can be found in Harezlak et al. [62]. The data set contains a total of 37,676 properties sold in the Sydney Statistical Division in the calendar year of 2001, which is available from the HRW package in R. We only focus on the last week of February to avoid the temporal issue, and there are 538 properties.

In this application, the house price (Price) is explained by four variables that include average weekly income (Income), levels of particulate matter with a diameter of less than a 10 micrometers level recorded at the air pollution monitoring station closest to the house (PM${}_{10}$), lot size (LS), and distance from house to the nearest coastline location in kilometers (DC). On the one hand, Income and PM${}_{10}$ have a linear effect on the response Price. On the other hand, LS and DC have a nonlinear effect on the response Price. Meanwhile, logarithmic transformation is performed on all variables to alleviate the trouble caused by large gaps in the domain. In addition, all variables are transformed such that the marginal distribution is approximately a standard normal distribution. This motivates us to consider the PLASAR model:

$$y_i=\rho \sum _{l=1}^n{w}_{il}{y}_l+x_i^T\alpha +g_1\left(z_{i1}\right)+g_2\left(z_{i2}\right)+{\epsilon }_i,i=1,\dots ,n,$$

(17)

where the response variable $y_i=log\left({\mathrm{Price}}_i\right)$, $x_{i1}=log\left({\mathrm{Income}}_i\right)$, $x_{i2}=log\left({\mathrm{PM}}_{10i}\right)$, $z_{i1}=log\left({\mathrm{LS}}_i\right)$, $z_{i2}=log\left({\mathrm{DC}}_i\right)$. Regarding the choice of the weight matrix, we use the Euclidean distance between any two houses to calculate the spatial weight matrix [63]. The spatial weight $w_{il}$ is

$$w_{il}=exp\{-\parallel s_i-s_l\parallel \}/\sum _{k\ne i}exp\{-\parallel s_i-s_k\parallel \},$$

where $s_i=(Lo{n}_i,La{t}_i)$ is represented as the longitude and latitude of location. We apply a quadratic B-splines and assign hyperparameters $(\lambda ,r_0,s_0^2,r_{{\tau }_{\alpha 0}},s_{{\tau }_{\alpha 0}}^2,r_{{\tau }_{{\beta }_{j}0}},s_{{\tau }_{{\beta }_{j}0}}^2)=(2,1,1,1,0.005,1,0.005)$ for $j=1,\dots ,p$ in our computation. We gradually decrease or increase the use of tuning parameters ${\sigma }_{\rho }$ and ${\sigma }_{{\xi }_j}$ such that the acceptable rates for updating $\rho$ and $(k_j,{\xi }_j)$ are around 25% for $j=1,\dots ,p$.

We generate 10,000 sampled values following a burn-in of 10,000 iterations and run the proposed Gibbs sampler five times with different initial states in each replication. Figure 6 plot the traces of some unknown parameters and nonparametric functions on grid points. It is obvious that the five parallel Markov chains mix well. We further calculate the “potential scale reduction factor” $\sqrt{\widehat{R}}$ for each of the unknown parameters and nonparametric functions on 20 selected grid points, which are plotted in Figure 7. The result indicates that the Markov chains have converged within the first 10,000 burn-in iterations.

Table 3. Parametric estimation in the model (17) for Sydney real estate data.

Parameter	Mean	SE	95% CI
$\rho$	$0.5548$	$0.0307$	$(0.4932,0.6160)$
${\alpha }_1$	$0.3269$	$0.0326$	$(0.2630,0.3908)$
${\alpha }_2$	$-0.0810$	$0.0318$	$(-0.1433,-0.0186)$
${\sigma }^2$	$0.3269$	$0.0326$	$(0.2630,0.3908)$
Total effect
$x_1$	$0.7343$	$0.0875$	$(0.5573,0.9073)$
$x_2$	$-0.1819$	$0.0866$	$(-0.3531,-0.0087)$

lists the estimated parameters together with their SE and 95% CI, which show that the estimation of $\widehat{\rho }$ is 0.5548 with $\mathrm{SE}=0.0307$. It implies that there exists a significant and positive spatial relationship on the housing price. We find that two covaraites have significant effects on the housing price, and the effects of Income are positive, but PM${}_{10}$ is negative. The regression coefficient of Income is ${\widehat{\alpha }}_1=0.3269>0$, which indicates that the Income has a positive effect on the housing price. Moreover, the regression coefficient of PM${}_{10}$ is ${\widehat{\alpha }}_2=-0.0810<0$, which reveals that the housing price would decrease as the PM${}_{10}$ increases.

Figure 8 depicts the fitted functions, together with its 95% CI, which look like two nonlinear functions. The curves show that $g_1\left(z_1\right)$ has a local maximum 0.6184 at around $z_1=3.9198$ and a local minimum 0.1557 at around $z_1=-0.8237$, and $g_2\left(z_2\right)$ has a local minimum −0.8224 at around $z_2=2.1605$. The results provide evidence that the significant effects of LS and DC on the housing price have a nonlinear S-shape and U-shape, respectively.

5. Summary

Spatial data are frequently encountered in practical applications and can be analyzed through the SAR model. To avoid some serious shortcomings of fully nonparametric models and reduce the high risk of misspecification of the traditional SAR models, we have considered PLASAR models for spatial data, which combine the PLA model and SAR model. In addition to spatial dependence between neighbors, it captures the linear and nonlinear effects between the related variables. We specify the prior of all unknown parameters, which led to a proper posterior distribution. The posterior summaries are obtained via the MCMC technique, and we have considered a fully Bayesian approach with free-knot splines to analyze the PLASAR model and design a Gibbs sampler to explore the full conditional posterior distributions. To obtain a rapidly-convergent algorithm, a modified Bayesian free-knot splines approach incorporated with powerful MCMC techniques is employed. We have illustrated that the finite sample of the proposed model and estimation method perform satisfactorily through a simulation study. The results show that the Bayesian estimator is efficient relative to the GMME, although the latter can also obtain reasonable estimations. Finally, the proposed model and methodology are applied to analyze real data.

This article focuses only on symmetrical data and the homoscedasticity of independent errors. Since spatial data cannot easily meet the conditions, it is fairly straightforward to analyze the proposed model and methodology to deal with spatial errors and heteroscedasticity. While we use PLASAR models to assess the linear and nonlinear effects of the covariates on the spatial response, the other models, such as partially linear single-index SAR models and partially linear varying-coefficient SAR models, can also be considered. Moreover, it would be interesting to develop a model selection method in which covariates are linear or nonlinear. We leave these topics for future research.