Consistency and Asymptotic Normality of Estimator for Parameters in Multiresponse Multipredictor Semiparametric Regression Model

Abstract

A multiresponse multipredictor semiparametric regression (MMSR) model is a combination of parametric and nonparametric regressions models with more than one predictor and response variables where there is correlation between responses. Due to this correlation we need to construct a symmetric weight matrix. This is one of the things that distinguishes it from the classical method, which uses a parametric regression approach. In this study, we theoretically developed a method of determining a confidence interval for parameters in a MMSR model based on a truncated spline, and investigating asymptotic properties of estimator for parameters in a MMSR model, especially consistency and asymptotic normality. The weighted least squares method was used to estimate the MMSR model. Next, we applied a pivotal quantity method, a Cramer–Wold theorem, and a Slutsky theorem to determine the confidence interval, investigate consistency, and asymptotic normality properties of estimator for parameters in a MMSR model. The obtained results were that the estimated regression function is linear to observation. We also obtained a $100\left(1-\alpha \right)\%$ confidence interval for parameters in the MMSR model, and the estimator for parameters in MMSR model was consistent and asymptotically normally distributed. In the future, these obtained results can be used as a theoretical basis in designing a standard toddlers growth chart to assess nutritional status.

1. Introduction

A regression model which is used to analyze the functional relationship between response variable and predictor variable in various fields is widely used for both prediction and interpretation purposes. This functional relationship is represented by a regression function. If we consider the form of the regression function, we recognize two types of regression models, namely a parametric regression (PR) model and a nonparametric regression (NR) model. The combination of the two types of regression models produces a semiparametric regression (SR) model. Furthermore, if this model has more than one predictor and response variables where there is a correlation between responses, the model is called a multiresponse multipredictor semiparametric regression (MMSR) model.

In general, the main problem in the regression modeling is estimation of the regression function. To estimate the regression function, there are several estimators which are frequently used in NR modeling; for example local linear, local polynomial, kernel, and spline. For prediction purposes, the use of local linear, local polynomial, and kernel estimators is highly recommended as discussed by [1,2,3,4,5,6,7,8,9,10,11]. Meanwhile, for prediction and interpretation purposes, the use of spline estimators is better and more flexible as discussed by [12,13,14]. Because of the flexible nature of this spline estimator, many researchers have been interested in using and developing it in several cases. For examples, an M-type spline estimator was discussed by [15]; truncated spline and B-spline estimators were discussed by [16,17], respectively; penalized spline has been proposed by [18] to analyze current status data. Additionally, spline smoothing has been proposed by [19] to estimate the regression function drawing the association between cortisol and ACTH hormones, and spline regression was proposed by [20] to estimate regression function applied to censored data. Next, [21,22] used both smoothing kernel and spline estimators to estimate the regression function and select the optimal smoothing parameter of uni-response nonparametric regression (UNR) models, and multiresponse nonparametric regression (MNR) models, respectively. A Kernel estimator was proposed by [23] to estimate UNR model through a simulation study. Additionally, [24] discussed kernel and spline smoothing techniques to estimate coefficient in a rates model. However, local linear, local polynomial and kernel estimators are highly dependent on the neighborhood of the target point, called the bandwidth, so that if used for model estimation of fluctuating data, a small bandwidth is required and this will result in an estimation curve that is too rough. So the estimators only consider the goodness of fit and do not consider smoothness. Thus, these estimators are not good to use for estimating models of fluctuating data in the sub intervals, because the estimation results will provide a large mean square error (MSE) value. This is different from the spline estimator which considers goodness of fit and smoothness factors as has been discussed by several researchers. Furthermore, [25] compared smoothing and truncated splines in a model for estimating blood pressures model, and the results showed that the smoothing spline is better at estimating the model than truncated spline, where it is shown by the MSE value if we use a smoothing spline estimator that is smaller than the truncated spline estimator. This means that for prediction purposes, the smoothing spline is better than the truncated spline. Additionally, [26] have discussed estimating a regression function in MNR model using smoothing spline and investigated asymptotic properties of the regression function. Application of smoothing spline and Fourier series was discussed by [27]. Although those researchers mentioned above have discussed several estimators to estimate the regression functions of the regression models, but those researchers discussed these estimators for UNR models and MNR models only. This means that those researchers mentioned above discussed estimators in NR models only.

Next, there are researchers who have discussed estimators in SR models; for examples [28,29] discussed smoothing techniques for estimating SR models; Ref. [30] used a spline estimator for determining the number of knots and their locations based on statistical criteria; Ref. [31] used an iterative weighted partial spline least squares estimator to estimate longitudinal SR model; Ref. [32] used a local linear estimator for designing the standard growth chart of children; Ref. [33] predicted GDP in Turkey using a SR model approach; Ref. [34] discussed a SR model applied to censored data; Refs. [35,36,37] discussed smoothing spline in SR models; Ref. [38] used bias-correction technique to construct the empirical likelihood ratios for estimating semiparametric model. However, those researchers discussed estimators in uni-response semiparametric regression (USR) models only. However, there are several researchers who have discussed estimators in a multiresponse semiparametric regression (MSR) model; for examples [39] studied estimating MSR using a smoothing spline estimator; Ref. [40] discussed determining confidence interval for the parameter of a parametric component of binary response SR model using truncated spline estimator; Ref. [41] discussed estimating the regression function and confidence interval of a parameter MSR model using a smoothing spline estimator. Furthermore, Ref. [42] discussed estimating the confidence interval for parameters of a MMSR model using a truncated spline estimator. However, all of the previous researchers mentioned above have not yet discussed determining consistency and asymptotic normality properties of parameters in a MMSR model by using truncated spline estimator.

In practice, we are often faced with the problem of analyzing the functional relationship between more than one response variable and more than one predictor variable where some of the response variables have a linear functional relationship with the response variable and some of the other predictor variables do not form a functional relationship that points to a certain pattern, and there is a correlation between responses. To deal with this problem, the MMSR model approach is used. Basically, the main goal of MMSR modeling is to get a better model than USR modeling, considering that this model not only considers the effect of predictors on responses, but also the relationship between responses. The representation of the relationship between responses is usually expressed in the form of a covariance matrix, which is used as a weighting in estimating the parameters of model. Hence, the problem of estimating the regression function is more complicated for a MMSR model, because the regression function in this model consists of a parametric component and a nonparametric component. Additionally, in this model there is a correlation between responses, so that in the estimation process the regression function requires a weight matrix in the form of a symmetric matrix, especially a diagonal matrix.

Therefore, in this article we discuss a new method for determining the confidence interval of parameters in the MMSR model and investigating asymptotic normality and consistency properties of parameters in the MMSR model based on a truncated spline estimator, which has a very good ability to handle data whose behavior changes (fluctuates) at certain sub-intervals.

2. Materials and Methods

In this section, we describe materials and methods which are used to determine asymptotic normality and consistency of parameters in the MMSR model based on truncated spline.

2.1. Multiresponse Multipredictor Semiparametric Regression (MMSR) Model

A paired observations set $\left(y_{ki},x_{k1i},x_{k2i},\dots ,x_{kqi},t_{k1i},t_{k2i},\dots ,t_{kri}\right)$ where $k=1,2,\dots ,p$; $i=1,2,\dots ,n$; $q+r=n$; $y$ represents response variable; and $x$, and $t$ represent predictor variables follows a MMSR model if the relationship between observations of the response variable, namely $y_{ki}$, and observations of the predictor variables, namely $\left(x_{k1i},x_{k2i},\dots ,x_{kqi},t_{k1i},t_{k2i},\dots ,t_{kri}\right)$, satisfies a model as follows:

$$y_{ki}=f_k\left(x_{k1i},x_{k2i},\dots ,x_{kqi}\right) + g_k\left(t_{k1i},t_{k2i}\dots ,t_{kri}\right) + {\epsilon }_{ki}$$

(1)

where $y_{ki}$ is the ith observation value in the kth response, $f_k\left(x_{k1i},x_{k2i},\dots ,x_{kqi}\right)$ is a parametric component of the kth response, $g_k\left(t_{k1i},t_{k2i}\dots ,t_{kri}\right)$ is a nonparametric component of the kth response in which $g_k$ is assumed to be smooth in the sense that it fits in the Sobolev space $W_2^m\left[a_k,b_k\right]$, and ${\epsilon }_{ki}$ is zero-mean random error with variance ${\sigma }_{ki}^2$.

The multiresponse multipredictor semiparametric regression (MMSR) model presented by Equation (1) can be written as follows:

$$y_{ki}=h_k\left(x_{k1i},x_{k2i},\dots ,x_{kqi},t_{k1i},t_{k2i},\dots ,t_{kri}\right) + {\epsilon }_{ki}$$

(2)

where $h_k\left(x_{k1i},x_{k2i},\dots ,x_{kqi},t_{k1i},t_{k2i},\dots ,t_{kri}\right) = f_k\left(x_{k1i},x_{k2i},\dots ,x_{kqi}\right) + g_k\left(t_{k1i},t_{k2i}\dots ,t_{kri}\right)$ is an unknown regression function of the MMSR model presented by Equation (1). This regression function consists of a parametric component and nonparametric component. Next, to estimate the regression function of the MMSR model presented by Equations (1) and (2) based on truncated spline estimator, we need to develop the truncated spline proposed by [12].

2.2. Truncated Spline

According to [12], the truncated section of a polynomial spline called as a piecewise polynomial, is a continuous segmented polynomial. A truncated spline regression model can adapt to the data characteristics, and it has the ability to overcome the data pattern showing a sharp rise or fall with the help of both knot points, and the number of knot points is such that its resulting curve is relatively smooth. Next, suppose we have a multiresponse multipredictor nonparametric regression (MMNR) model which can be expressed as follows:

$$y_{ki}=\sum _{l=1}^r{g}_{kl}\left(t_{kli}\right) + {\epsilon }_{ki}; i=1,2,\dots ,n; k=1,2,\dots ,p$$

(3)

Then, the truncated function with B knots (i.e., $b_1,b_2,\dots ,b_B$ are knot points) and degree d is defined as follows [12]:

$${\left(t_{kli}-b_{klj}\right)}_{+}^d=\left\{\begin{array}{c}\begin{array}{ccc}{\left(t_{kli}-b_{klj}\right)}^d\hfill & \mathrm{for}\hfill & t_{kli}\ge b_{klj}\hfill \end{array}\hfill \\ \begin{array}{ccc}0\hfill & \mathrm{for}\hfill & t_{kli}<b_{klj}\hfill \end{array}\hfill \end{array}\right.$$

(4)

where d represents degree of polynomial in which generally for $d=1, 2$ and $3$, the Equation (4) gives the functions of linear polynomial, quadratic polynomial and cubic polynomial, respectively. Hence, based on Equations (3) and (4), the general form of truncated spline with degree of polynomial d and the number of knot points B for the regression function of MMNR model presented by Equation (3) can be expressed as follows:

$$g_{kl}\left(t_{kli}\right)={\alpha }_{k0}+{\displaystyle \sum }_{s=1}^d{\alpha }_{kls}{t}_{kli}^s+{\displaystyle \sum }_{j=1}^B{\beta }_{klj}{\left(t_{kli}-b_{klj}\right)}_{+}^d$$

(5)

Furthermore, we can develop this technique to the MMSR model for estimating the regression function of MMSR model presented by Equation (1) or Equation (2) based on truncated spline. Next, to determine the confidence interval for parameters in the MMSR model, we need the pivotal quantity method as proposed by [43].

2.3. Pivotal Quantity

Suppose we have a random sample of size n, $X_1$,$X_2$,…,$X_n$, of a population $X$ with probability density function $f\left(x,\delta \right)$ where $\delta$ is an unknown parameter. If $\mathit{T}$ is a function of $X_1$,$X_2$,…,$X_n$ and $\delta$ where its probability distribution is independent of the parameter $\delta$, then $\mathit{T}$ is called a pivotal quantity [43].

Finally, based on the truncated spline estimator proposed by [12] and by applying weighted least square (WLS) method, we can estimate the MMSR model presented by Equation (1). Next, we apply development of pivotal quantity method proposed by [43] to determine confidence interval for parameters in MMSR model. Additionally, we investigate consistent property of estimator for parameters in MMS model, and then we apply Cramer–Wold theorem [44] and Slutsky theorem [45] to determine the asymptotic normality of estimator for parameters in the MMSR model presented by Equation (1).

2.4. Simulation

In this simulation we generated three samples sized $n=30,50 \mathrm{and} 100$. The response vector $y={\left(y_1,y_2,y_3\right)}^{\mathrm{T}}$, consisting of three response variables and the design vector $X=\left(x_1,x_2, x_3\right)$ were produced from a uniform distribution. In addition, the MMSR model to be generated includes two different smooth functions, ${\mathrm{g}}_1\left({\mathrm{t}}_1\right)$ and ${\mathrm{g}}_2\left({\mathrm{t}}_2\right)$ with nonparametric covariates ${ \mathrm{t}}_1$ and ${\mathrm{t}}_2$, respectively. The number of replications for each sample used in simulation experiments were considered as 1000. Finally, the random error terms ${\mathsf{\epsilon }}_{\mathrm{i}}$-s were independent and identically distributed from the multivariate normal distribution ${\mathsf{\epsilon }}_{\mathrm{i}}~\mathrm{MN}\left(\mathbf{0},{\Sigma }_{\mathsf{\epsilon }}^2\right)$ for three models.

3. Results and Discussions

The results and discussions presented in this section include estimating a MMSR model, estimating confidence interval of the parameters in a MMSR model, investigating the consistency of estimator for parameters in a MMSR model, determining asymptotic normality of estimator for parameters in a MMSR model, and simulation study.

3.1. Estimating MMSR Model

By considering Equations (4) and (5), the estimation of a MMSR model presented by Equation (1) or Equation (2) based on truncated spline estimator is approximated by a linear function that is in the form of truncated spline with degree of polynomial $d=1$ (i.e., a linear polynomial) and knot point b and the number of knots B such that the MMSR model presented by Equation (1) can be expressed as follows:

$${\mathrm{y}}_{\mathrm{ki}}={\mathsf{\alpha }}_{\mathrm{k}0}+{\displaystyle \sum }_{\mathrm{l}=1}^{\mathrm{q}}{\mathsf{\alpha }}_{\mathrm{kl}}{\mathrm{x}}_{\mathrm{kli}}+{\displaystyle \sum }_{\mathrm{j}=1}^{\mathrm{r}}\left[{\mathsf{\beta }}_{\mathrm{kj}}{\mathrm{t}}_{\mathrm{kji}}+{\displaystyle \sum }_{\mathrm{m}=1}^{\mathrm{B}}{\mathsf{\beta }}_{\mathrm{k}\left(\mathrm{j}+\mathrm{m}\right)}{\left({\mathrm{t}}_{\mathrm{kji}}-{\mathrm{b}}_{\mathrm{kjm}}\right)}_{+}^1\right]+{\mathsf{\epsilon }}_{\mathrm{ki}}$$

(6)

where $k=1,2,\dots ,p$ and $i=1,2,\dots ,n$.

Therefore, the model presented by Equation (6) can be written in matrix notation as follows:

$$y = \left(\begin{array}{cc}X& t\end{array}\right)\left(\begin{array}{c}\alpha \\ \beta \end{array}\right) + \epsilon$$

(7)

Next, let $H = \left(\begin{array}{cc}X& t\end{array}\right)$ and $\delta =\left(\begin{array}{c}\alpha \\ \beta \end{array}\right)$ then MMSR model presented by Equation (7) can be written as follows:

$$y=H\delta +\epsilon$$

(8)

where $=\left(\begin{array}{c}{y}_1\\ y_2\\ \begin{array}{c}⋮\\ y_p\end{array}\end{array}\right)$; $H=\left(\begin{array}{ccc}{H}_1& \mathbf{0}& \begin{array}{cc}\cdots & \mathbf{0}\end{array}\\ \mathbf{0}& H_2& \begin{array}{cc}\cdots & \mathbf{0}\end{array}\\ \begin{array}{c}⋮\\ \mathbf{0}\end{array}& \begin{array}{c}⋮\\ \mathbf{0}\end{array}& \begin{array}{cc}\begin{array}{c}\ddots \\ \cdots \end{array}& \begin{array}{c}⋮\\ H_p\end{array}\end{array}\end{array}\right)$; $\delta =\left(\begin{array}{c}{\alpha }_{k\mathbf{0}}\\ {\alpha }_{k1}\\ \begin{array}{c}⋮\\ {\alpha }_{kq}\\ \begin{array}{c}{\beta }_{k1}\\ {\beta }_{k2}\\ \begin{array}{c}⋮\\ {\beta }_{kr}\end{array}\end{array}\end{array}\end{array}\right)$; $\epsilon =\left(\begin{array}{c}{\epsilon }_1\\ {\epsilon }_2\\ \begin{array}{c}⋮\\ {\epsilon }_p\end{array}\end{array}\right)$.

Further, suppose $\epsilon$ is independently and identically distributed with zero mean and covariance $W$ (namely). Note that there is correlation between responses. This implies that there is correlation between random errors of each response variable, too. Therefore, for estimating parameters of the MMSR model we use the weighted least squares (WLS) method, which needs a symmetrical weight matrix $W^{-1}$ that is the inverse of the covariance matrix $W$. We can obtain construction of the covariance matrix $W$ as follows:

$$\mathrm{Cov}\left(\epsilon \right) = \mathrm{E}\left(\epsilon {\epsilon }^{\mathrm{T}}\right) = \left[\begin{array}{ccc}\mathrm{E}\left({\epsilon }_1{\epsilon }_1^{\mathbf{T}}\right)& \mathrm{E}\left({\epsilon }_1{\epsilon }_2^{\mathbf{T}}\right)& \begin{array}{cc}\cdots & E\left({\epsilon }_1{\epsilon }_p^{\mathbf{T}}\right)\end{array}\\ \mathrm{E}\left({\epsilon }_2{\epsilon }_1^{\mathbf{T}}\right)& \mathrm{E}\left({\epsilon }_2{\epsilon }_2^{\mathbf{T}}\right)& \begin{array}{cc}\cdots & \mathrm{E}\left({\epsilon }_2{\epsilon }_p^{\mathbf{T}}\right)\end{array}\\ \begin{array}{c}⋮\\ \mathrm{E}\left({\epsilon }_p{\epsilon }_1^{\mathbf{T}}\right)\end{array}& \begin{array}{c}⋮\\ \mathrm{E}\left({\epsilon }_p{\epsilon }_2^{\mathbf{T}}\right)\end{array}& \begin{array}{cc}\begin{array}{c}\ddots \\ \cdots \end{array}& \begin{array}{c}⋮\\ \mathrm{E}\left({\epsilon }_p{\epsilon }_p^{\mathbf{T}}\right)\end{array}\end{array}\end{array}\right]$$

$$=\left[\begin{array}{ccc}{W}_{11}& W_{12}& \begin{array}{cc}\cdots & W_{1\mathrm{p}}\end{array}\\ W_{21}& W_{22}& \begin{array}{cc}\cdots & W_{2\mathrm{p}}\end{array}\\ \begin{array}{c}⋮\\ W_{\mathrm{p}1}\end{array}& \begin{array}{c}⋮\\ W_{\mathrm{p}2}\end{array}& \begin{array}{cc}\begin{array}{c}\ddots \\ \cdots \end{array}& \begin{array}{c}⋮\\ W_{\mathrm{pp}}\end{array}\end{array}\end{array}\right]=W (\mathrm{namely}).$$

(9)

where $W_{kl}=\left\{\begin{array}{c}\begin{array}{ccc}diag\left({\sigma }_{k1}^2,{\sigma }_{k2}^2,\dots ,{\sigma }_{kn}^2\right)& \mathrm{for}& k=l\end{array}\\ \begin{array}{ccc}diag\left({\sigma }_{\left(kl\right)1},{\sigma }_{\left(kl\right)2},\dots ,{\sigma }_{\left(kl\right)n}\right)& \mathrm{for}& k\ne l\end{array}\end{array}\right.$; $k=1,2,\dots ,p$.

Note that matrix $W$, given in (9), is a symmetrical matrix. Next, based on assumptions of the MMSR model given in (1) that ${\epsilon }_{ki}$ is zero-mean random error with variance ${\sigma }_{ki}^2$, then it implies that ${\sigma }_{kl}={\rho }_{kl}{\sigma }_k{\sigma }_l$ and ${\rho }_{kl}=\left\{\begin{array}{c}\begin{array}{ccc}{\rho }_k& \mathrm{for}& k=l\end{array}\\ \begin{array}{ccc}0& \mathrm{for}& k\ne l\end{array}\end{array}\right.$; $k=1,2,\dots ,p$.

Hence, for $k=1,2,\dots ,p$ we have:

$$W_{\mathit{k}l}=\left\{\begin{array}{c}\begin{array}{ccc}diag\left({\sigma }_{k1}^2,{\sigma }_{k2}^2,\dots ,{\sigma }_{kn}^2\right)& \mathrm{for}& k=l\end{array}\\ \begin{array}{ccc}\mathbf{0}& \mathrm{for}& k\ne l\end{array}\hfill \end{array}\right.$$

Therefore, the matrix given in (9) can be written as follows:

$$W\left({\mathit{\sigma }}^{\mathbf{2}}\right)=\left(\begin{array}{ccc}{W}_{\mathbf{1}}\left({\sigma }_{\mathbf{1}}^{\mathbf{2}}\right)& \mathbf{0}& \begin{array}{cc}\mathbf{\cdots }& \mathbf{0}\end{array}\\ \mathbf{0}& W_{\mathbf{2}}\left({\sigma }_{\mathbf{2}}^{\mathbf{2}}\right)& \begin{array}{cc}\mathbf{\cdots }& \mathbf{0}\end{array}\\ \begin{array}{c}\mathbf{⋮}\\ \mathbf{0}\end{array}& \begin{array}{c}\mathbf{⋮}\\ \mathbf{0}\end{array}& \begin{array}{cc}\begin{array}{c}\mathbf{\ddots }\\ \mathbf{\cdots }\end{array}& \begin{array}{c}\mathbf{⋮}\\ W_{\mathit{p}}\left({\sigma }_{\mathit{p}}^{\mathbf{2}}\right)\end{array}\end{array}\end{array}\right)=diag\left(W_{\mathbf{1}}\left({\sigma }_{\mathbf{1}}^{\mathbf{2}}\right),W_{\mathbf{2}}\left({\sigma }_{\mathbf{2}}^{\mathbf{2}}\right),\mathbf{\dots },W_{\mathit{p}}\left({\sigma }_{\mathit{p}}^{\mathbf{2}}\right)\right)$$

where 0 is the null matrix, that is, a matrix in which all its element are null, and matrix $W_{\mathit{k}}\left({\sigma }_{\mathit{k}}^{\mathbf{2}}\right)$ is given as follows:

$$W_{\mathit{k}}\left({\sigma }_{\mathit{k}}^2\right)=\left(\begin{array}{ccc}{\sigma }_{k1}^2& {\sigma }_{k\left(1,2\right)}& \begin{array}{cc}\cdots & {\sigma }_{k\left(1,n\right)}\end{array}\\ {\sigma }_{k\left(2,1\right)}& {\sigma }_{k2}^2& \begin{array}{cc}\cdots & {\sigma }_{k\left(2,n\right)}\end{array}\\ \begin{array}{c}⋮\\ {\sigma }_{k\left(n,1\right)}\end{array}& \begin{array}{c}⋮\\ {\sigma }_{k\left(n,2\right)}\end{array}& \begin{array}{cc}\begin{array}{c}\ddots \\ \cdots \end{array}& \begin{array}{c}⋮\\ {\sigma }_{kn}^2\end{array}\end{array}\end{array}\right).$$

Estimation of parameters in the MMSR model presented by Equation (1) can be obtained by taking solution of the WLS optimization problem as follows:

$$\underset{\delta }{\min }Q\left(\delta \right)=\underset{\delta }{\min }{\left(y-H\delta \right)}^{\mathit{T}}{W}^{-\mathbf{1}}\left(y-H\delta \right)$$

(10)

Hence, by taking the partial derivative of $Q\left(\delta \right)$ with respect to $\delta$ as follows:

$$\frac{\partial Q\left(\delta \right)}{\partial \delta }=\frac{\partial }{\partial \delta }\left\{{\left(y-H\delta \right)}^{\mathbf{T}}{W}^{-\mathbf{1}}\left(y-H\delta \right)\right\}=\mathbf{0}$$

we get:

$$\widehat{\delta }={\left(H^{\mathrm{T}}{W}^{-1}H\right)}^{-1}{H}^{\mathrm{T}}{W}^{-1}y$$

(11)

Next, based on the MMSR model presented in Equation (2) and the estimated parameters given by Equation (11), we have the estimated regression function of a MMSR model, namely $\widehat{h}$, as follows:

$$\widehat{h}=H\widehat{\delta }$$

(12)

Thus, by considering Equations (8), (11) and (12), we get an estimation of the MMSR model given by Equation (1) based on a truncated spline estimator as follows:

$$\widehat{y}=H\widehat{\delta }=H{\left(H^{\mathrm{T}}{W}^{-1}H\right)}^{-1}{H}^{\mathrm{T}}{W}^{-1}y=My$$

(13)

where

$$M=H{\left(H^{\mathrm{T}}{W}^{-1}H\right)}^{-1}{H}^{\mathrm{T}}{W}^{-1}.$$

(14)

3.2. Estimating Confidence Interval of Parameters in MMSR Model

We assume that ${\epsilon }_{ki}$ in the MMSR model presented by Equation (1) is normally distributed independently and identically with zero mean and variance ${\sigma }_{ki}^2$. It is commonly written as ${\epsilon }_{ki}~{}_{i.i.d}N\left(0, {\sigma }_{ki}^2\right)$ where ${\sigma }_{ki}^2$ is unknown. Next, suppose we design the $100\left(1-\alpha \right)\%$ confidence interval for ${\delta }_v$ where $v=1,2,\dots ,p^{\ast }$; $p^{\ast }=p+pq+rp+pB$ and B is the number of knots. Therefore, we have a pivotal quantity as follows:

$${\mathit{T}}_v\left(y_k, x_{k1},\dots ,x_{kq},t_{k1},\dots ,t_{kr}\right)=\frac{{\widehat{\delta }}_v-{\delta }_v}{\sqrt{MSE {\left(H^{\mathit{T}}{W}^{-1}H\right)}_{vv}^{-1}}}$$

(15)

where $MSE=\frac{y^{\mathit{T}}\left[I-H{\left(H^{\mathit{T}}{W}^{-\mathbf{1}}H\right)}^{-\mathbf{1}}{H}^{\mathit{T}}{W}^{-\mathbf{1}}\right] y}{np-p^{\ast }}=\frac{y^{\mathit{T}}\left[I-M\right] y}{np-p^{\ast }}=\frac{y^{\mathit{T}} \Omega y}{np-p^{\ast }}$; $\Omega =I-M$; and $M=H{\left(H^{\mathit{T}}{W}^{-1}H\right)}^{-\mathbf{1}}{H}^{\mathit{T}}{W}^{-1}$. The Equation (15) is pivotal quantity for parameter ${\delta }_v$ where ${\delta }_v$ is the $vth$ element of the parameter vector $\delta$, and ${\left(H^{\mathit{T}}{W}^{-1}H\right)}_{vv}^{-1}$ is the diagonal element of ${\left(H^{\mathrm{T}}{W}^{-\mathbf{1}}H\right)}^{-\mathbf{1}}$. The pivotal quantity given by Equation (15) has a distribution of $t$-student with a degree of freedom of $\left(np-p^{\ast }\right)$.

Hereinafter, to determine the $100\left(1-\alpha \right)\%$ confidence interval for ${\delta }_v$, we must take a solution of the following probability Equation:

$$P\left[L_v\le {\mathit{T}}_v\left(y_k, x_{k1},\dots ,x_{kq},t_{k1},\dots ,t_{kr}\right)\le U_v\right]=1-\alpha$$

(16)

where $L_v$ is the lower limit value of the confidence interval and $U_v$ is the upper limit value of the confidence interval, and $\left(1-\alpha \right)$ is the confidence level.

Next, by substituting Equation (15) into Equation (16) we get:

$$P\left[L_v\le \frac{{\widehat{\delta }}_v-{\delta }_v}{\sqrt{MSE {\left(H^{\mathit{T}}{W}^{-1}H\right)}_{vv}^{-1}}}\le U_v\right]=1-\alpha$$

(17)

Here, the Equation (17) can be written as follows:

$$P\left({\widehat{\delta }}_v-U\le {\delta }_v\le {\widehat{\delta }}_v-L\right)=1-\alpha$$

(18)

where $L=L_v\sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D$; $U=U_v\sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D$; $\Omega =I-M$; $M=H{\left(H^{\mathit{T}}{W}^{-1}H\right)}^{-1}{H}^{\mathit{T}}{W}^{-1}$; $D={\left(H^{\mathrm{T}}{W}^{-\mathbf{1}}H\right)}_{vv}^{-1}$; and subscript $“vv”$ of ${\left(H^{\mathrm{T}}{W}^{-\mathbf{1}}H\right)}_{vv}^{-1}$ in Equation (17) represents diagonal element of ${\left(H^{\mathrm{T}}{W}^{-\mathbf{1}}H\right)}^{-1}$.

A confidence interval is called good if it has the shortest interval length. Because of this, we should determine values of $L_v\in ℝ$ and $U_v\in ℝ$ such that the length of the confidence interval in Equation (18) is the shortest.

Therefore, if $length\left(L_v,U_v\right)$ represents the length of the confidence interval in Equation (18), then we have:

$$\begin{gathered} length\left(L_v,U_v\right)=\left({\widehat{\delta }}_v-L_v\sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D\right)-\left({\widehat{\delta }}_v-U_v\sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D\right) \\ =\left(U_v-L_v\right)\sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D. \end{gathered}$$

Hence, we must determine the solution of the following optimization problem to take the shortest length of confidence interval for ${\delta }_v$:

$$\underset{L_v,U_v\in ℝ}{\mathrm{Min}}\left\{length\left(L_v,U_v\right)\right\}=\underset{L_v,U_v\in ℝ}{\mathrm{Min}}\left\{\left(U_v-L_v\right)\sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D\right\}$$

(19)

and the following condition must be satisfied by Equation (19):

$${{\displaystyle \mathrm{\int }}}_{L_v}^{U_v}\psi \left(\omega \right)d\omega =1-\alpha \mathrm{or} \phi \left(U_v\right)-\phi \left(L_v\right)-\left(1-\alpha \right)=0$$

(20)

where function $\psi \left(·\right)$ is a probability distribution of $t_{np-p^{\ast }}$ and function $\phi \left(·\right)$ is a cumulative probability distribution of $t_{np-p^{\ast }}$.

Next, by using a Lagrange multiplier method we have:

$$R\left(L_v,U_v,\gamma \right)=\left(U_v-L_v\right)\sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D + \gamma \left(\phi \left(U_v\right)-\phi \left(L_v\right)-\left(1-\alpha \right)\right)$$

(21)

where $\gamma$ is a Lagrange constant.

Hence, we get:

$$\frac{\partial R\left(L_v,U_v,\gamma \right)}{\partial L_v}=-\sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D-\gamma {\phi }^{\prime }\left(L_v\right)=0$$

(22)

$$\frac{\partial R\left(L_v,U_v,\gamma \right)}{\partial U_v}=\sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D+\gamma {\phi }^{\prime }\left(U_v\right)=0$$

(23)

$$\frac{\partial R\left(L_v,U_v,\gamma \right)}{\partial \gamma }=\phi \left(U_v\right)-\phi \left(L_v\right)-\left(1-\alpha \right)=0$$

(24)

Based on Equations (22) and (23), we obtain:

$${\phi }^{\prime }\left(L_v\right)={\phi }^{\prime }\left(U_v\right)$$

(25)

The Equation (25) implies $L_v=U_v$ or $L_v=-U_v$. Since, in this case $L_v=U_v$ is not satisfied, then the shortest confidence interval for parameters vector ${\delta }_v$ must be taken from the values of $L_v$ and $U_v$ that satisfy the following Equation:

$${{\displaystyle \mathrm{\int }}}_{-\infty }^{L_v}\psi \left(\omega \right)d\omega ={{\displaystyle \mathrm{\int }}}_{U_v}^{\infty }\psi \left(\omega \right)d\omega =\frac{\alpha }{2}$$

(26)

By using the confidence level $\left(1-\alpha \right)$, the values of $L_v$ and $U_v$ that satisfy Equation (26) can be found in table of $t_{np-p^{\ast }}$ distribution.

Hence, the shortest confidence interval for parameters of a MMSR model based on the truncated spline estimator satisfies the following probability:

$$P\left[{\widehat{\delta }}_v-U_v\sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D\le {\delta }_v\le {\widehat{\delta }}_v+U_v\sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D\right]=1-\alpha$$

where the $U_v$ value can be obtained from Equation (26) that is ${{\displaystyle \int }}_{U_v}^{\infty }\psi \left(\omega \right)d\omega =\frac{\alpha }{2}$.

Therefore, we have:

$$P\left[{\widehat{\delta }}_v-t_{\left(\frac{\alpha }{2};np-p^{\ast }\right)} \sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D\le {\delta }_v\le {\widehat{\delta }}_v+t_{\left(\frac{\alpha }{2};np-p^{\ast }\right)}\sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D\right]=1-\alpha$$

Thus, by using $t$-student distribution, the $100\left(1-\alpha \right)\%$ confidence interval for parameters vector ${\delta }_v$ of the MMSR model presented by Equation (1) based on the truncated spline estimator is:

$${\widehat{\delta }}_v-t_{\left(\frac{\alpha }{2};np-p^{\ast }\right)} \sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D\le {\delta }_v\le {\widehat{\delta }}_v+t_{\left(\frac{\alpha }{2};np-p^{\ast }\right)}\sqrt{\frac{y^{\mathrm{T}}\Omega y}{np-p^{\ast }}}D$$

(27)

where $v=1,2,\dots ,p^{\ast }$; $p^{\ast }=p+pq+rp+pB$ and B is the number of knots; $\Omega =I-M$; $M=H{\left(H^{\mathit{T}}{W}^{-1}H\right)}^{-\mathbf{1}}{H}^T{W}^{-1}$; $D={\left(H^{\mathrm{T}}{W}^{-\mathbf{1}}H\right)}_{vv}^{-1}$; subscript $“vv”$ of ${\left(H^{\mathrm{T}}{W}^{-\mathbf{1}}H\right)}_{vv}^{-1}$ represents the diagonal elements of matrix ${\left(H^{\mathrm{T}}{W}^{-\mathbf{1}}H\right)}^{-1}$; $H$ is a matrix of the predictor as given in Equation (8); and $W$ is a symmetrical covariance matrix as given in Equation (9).

3.3. Investigating Consistency of Estimator for Parameters in MMSR Model

Before investigating consistency of the estimator for parameters in the MMSR model namely $\widehat{\delta }$, we need the following assumptions.

Assumption 1.

$x_{ki}=\frac{2i-1}{2n}$, $i=1,2,\dots ,n$and $t_{ki}=\frac{2i-1}{2n}$, $i=1,2,\dots ,n$; $k=1,2,\dots ,p$.

Assumption 2.

${\theta }_{\mathrm{k}}=\left(x_{\mathrm{k}1},x_{\mathrm{k}2},\dots ,x_{\mathrm{kq}},t_{\mathrm{k}1},t_{\mathrm{k}2},\dots ,t_{\mathrm{kr}}\right)$, $k=1,2,\dots ,p$are independently and identically distributed with zero mean, the covariance matrix$\mathit{Z}$, and the third absolute moment is finite.

Assumption 3.

$\underset{n\to \infty }{\mathrm{Lim}}{\displaystyle \sum }_{i=1}^n{w}_{kj}=\mathbf{\vartheta }<\infty$, $k=1,2,\dots ,p$.

For investigating consistent property of the estimator for the parameters of the MMSR model namely $\widehat{\delta }$, we need the following Lemma 1 and Theorem 1.

Lemma 1.

If$\widehat{\delta }$as presented in Equation (11) is a truncated spline estimator for the parameters of the multiresponse multipredictor semiparametric regression (MMSR) model, then

$$\widehat{\delta }-\delta ={\left(\frac{H^{\mathit{T}}{W}^{-\mathbf{1}}H}{n}\right)}^{-\mathbf{1}}\left(\frac{H^{\mathit{T}}{W}^{-\mathbf{1}}\epsilon }{n}\right)$$

(28)

Proof of Lemma 1.

Based on Equations (8), and (11)–(14), we have:

$$\begin{array}{ll}\widehat{\delta }-\delta & =\left({\left(H^{\mathrm{T}}{W}^{-1}H\right)}^{-1}{H}^{\mathrm{T}}{W}^{-1}y\right)-\delta \\ & ={\left(H^T{W}^{-1}H\right)}^{-1}{H}^T{W}^{-1}\left(H\delta +\epsilon \right)-\delta \\ & ={\left(H^T{W}^{-1}H\right)}^{-1}{H}^T{W}^{-1}H\delta +{\left(H^T{W}^{-1}H\right)}^{-1}{H}^T{W}^{-1}\epsilon -\delta \\ & ={\left(H^T{W}^{-1}H\right)}^{-1}{H}^T{W}^{-1}H\delta +{\left(H^T{W}^{-1}H\right)}^{-1}{H}^T{W}^{-1}\epsilon -{\left(H^T{W}^{-1}H\right)}^{-1}\left(H^T{W}^{-1}H\right)\delta \\ & ={\left(H^T{W}^{-1}H\right)}^{-1}{H}^T{W}^{-1}\epsilon \\ & ={\left(\frac{H^T{W}^{-1}H}{n}\right)}^{-1}\left(\frac{H^T{W}^{-1}\epsilon }{n}\right).\end{array}$$

Thus, we obtain:

$$\widehat{\delta }-\delta ={\left(\frac{H^{\mathit{T}}{W}^{-\mathbf{1}}H}{n}\right)}^{-\mathbf{1}}\left(\frac{H^{\mathit{T}}{W}^{-\mathbf{1}}\epsilon }{n}\right)$$

□

Theorem 1.

If Assumptions 1–3 hold, then

$$(\mathrm{a}). \frac{H^{\mathit{T}}{W}^{-\mathbf{1}}H}{\mathit{n}} \stackrel{P}{\to }\mathit{Z}\mathit{\vartheta } \mathrm{as} n\to \infty .$$

(29)

$$(\mathrm{b}). \frac{H^{\mathit{T}}{W}^{-\mathbf{1}}\epsilon }{\mathit{n}} \stackrel{P}{\to } \mathit{0} \mathrm{as} n\to \infty .$$

(30)

Proof of Theorem 1.

Given matrices $M=\left\{m_{kij}\right\}$, $M^{T}WM{W}^{-1}=\left\{c_{kij}\right\}$, and $W=diag\left\{w_{k1},w_{k2},\dots ,w_{kn}\right\}$; $k=1,2,\dots ,p$; $i,j=1,2,\dots ,n.$ Because Assumptions 1–3 hold, then:

(a)

Based on the Strong Law of Large Numbers [45], we have:

$$\frac{H^{T}H}{n} \stackrel{P}{\to } Z$$

it implies $\frac{H^T{W}^{-1}H}{n} \stackrel{\mathrm{P}}{\to }Z\vartheta$ as $n\to \infty$.

(b)

Note that $\frac{H^{\mathit{T}}{W}^{-\mathbf{1}}\epsilon }{\mathit{n}} \stackrel{P}{\to } \mathbf{0}$ as $n\to \infty$ hold if $Var{\left(\frac{H^{\mathit{T}}{W}^{-\mathbf{1}}\epsilon }{\mathit{n}}\right)}_i \to \mathit{0}$ as $n\to \infty$.

On the other hand, we have:

$$E{\left(H^{\mathit{T}}{W}^{-\mathbf{1}}\epsilon {\epsilon }^{\mathit{T}}{W}^{-\mathbf{1}}H\right)}_i={\sigma }_{ki}^2 tr\left(z_{ii}{W}^{-\mathbf{1}}\right)=o\left(n\right) \mathrm{as} n\to \infty .$$

It means that:

$$Var{\left(\frac{H^{\mathit{T}}{W}^{-\mathbf{1}}\epsilon }{\mathit{n}}\right)}_i \to \mathbf{0} \mathrm{as} n\to \infty .$$

Therefore, we obtain:

$$\frac{H^{\mathit{T}}{W}^{-\mathbf{1}}\epsilon }{\mathit{n}} \stackrel{P}{\to } \mathbf{0} \mathrm{as} n\to \infty .$$

□

Furthermore, the following Theorem provides a consistency property of $\widehat{\delta }$ that is a truncated spline estimator for parameters of MMSR model presented in Equation (1).

Theorem 2.

If $\widehat{\delta }$is a truncated spline estimator for parameters of MMSR model presented in Equation (1), then

$$\underset{\mathit{n}\to \infty }{\mathrm{Lim}}P\left(\left|\widehat{\delta }-\delta \right|<\xi \right)=\mathbf{1}, \xi >\mathbf{0}.$$

(31)

In other word, an estimator$\widehat{\delta }$that satisfies the condition given in Equation (31) is said to be a consistent estimator.

Proof of Theorem 2.

Suppose $\widehat{\delta }$ is estimator as given in Equation (11). Based on Lemma 1 and Theorem 1, we get:

$$\left|\widehat{\delta }-\delta \right|=\left|{\left(\frac{H^{\mathit{T}}{W}^{-\mathbf{1}}H}{\mathit{n}}\right)}^{-\mathbf{1}}\left(\frac{H^{\mathit{T}}{W}^{-\mathbf{1}}\epsilon }{\mathit{n}}\right)\right| \stackrel{P}{\to } \mathbf{0} \mathrm{as} n\to \infty .$$

(32)

According to [45], for every $\xi >\mathbf{0}$ we can express Equation (32) as follows:

$$\underset{\mathit{n}\to \infty }{\mathrm{Lim}}P\left(\left|\widehat{\delta }-\delta \right|\ge \xi \right)=\mathbf{0}.$$

Next, by using probability property, we get:

$$\underset{\mathit{n}\to \infty }{\mathrm{Lim}}P\left(\left|\widehat{\delta }-\delta \right|<\xi \right)=\underset{n\to \infty }{\mathrm{Lim}}\left\{1-P\left(\left|\widehat{\delta }-\delta \right|\ge \xi \right)\right\}=\mathbf{1}.$$

Thus, $\underset{\mathit{n}\to \infty }{\mathrm{Lim}}P\left(\left|\widehat{\delta }-\delta \right|<\xi \right)=\mathbf{1}$, $\xi >\mathbf{0}$. It means that $\widehat{\delta }$ is a consistent estimator. □

3.4. Determining Asymptotic Normality of Estimator for Parameters in MMSR Model

In this section, we determine asymptotic normality of $\delta$ that is a truncated spline estimator for parameters of the MMSR model presented in Equation (1). Before investigating asymptotic normality of $\delta$, firstly we need to consider the following Lemma 2 and Theorem 3.

Lemma 2.

If$M$is the matrix as given in Equation (14) and $h$is the vector of the regression function of MMSR model, then

$$\underset{ n\to \infty }{\mathrm{Lim}}{n}^{-3/2}{\displaystyle \sum }_i{\left|{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right|}^3=\mathbf{0}$$

Proof of Lemma 2.

We have:

$$\begin{gathered} n^{-3/2}{\displaystyle \sum }_i{\left|{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right|}^3=n^{-3/2}{\displaystyle \sum }_i\left|{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right|{\left|{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right|}^2 \\ \le n^{-3/2}\underset{j}{\max }\left|{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right|{\displaystyle \sum }_j{\left|{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right|}^2 \end{gathered}$$

On the other hand, we have:

$$\begin{gathered} n^{-3/2}{\displaystyle \sum }_i{\left|{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right|}^3\le n^{-3/2}\sqrt{{\displaystyle \sum }_i{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i^2} {\displaystyle \sum }_i{\left|{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right|}^2 \\ =n^{-3/2}{\left(h^{\mathrm{T}}{W}^{-\mathbf{1}}\left(I-M^{\mathit{T}}\right)\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right)}^{3/2} \end{gathered}$$

Hence, we obtain:

$$n^{-3/2}\sum _i{\left|{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right|}^3=\mathit{o}\left(1\right) \mathrm{or} \underset{N\to \infty }{\mathrm{Lim}}{n}^{-3/2}\sum _i{\left|{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right|}^3=\mathbf{0}.$$

□

Theorem 3.

If$M$is matrix as given in Equation (14) and$h$is vector of regression function of MMSR model, then for n$\to \infty$

$$\frac{H^{\mathrm{T}}\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h+W^{-\mathbf{1}}\epsilon \right]}{\sqrt{n}}\stackrel{d}{\to }{D}^{\ast }~N\left(\mathbf{0},{\sigma }_{ki}^{2}Z\mathit{\vartheta }\right).$$

Proof of Theorem 3.

To prove this theorem, we apply the Cramer–Wold theorem [44,45]. Given a vector $a$ such that:

$$\frac{a^{\mathrm{T}}{H}^{\mathrm{T}}\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h+W^{-\mathbf{1}}\epsilon \right]}{\sqrt{n}}={\displaystyle \sum }_i{K}_i$$

where ${\mathbf{K}}_i=\frac{{\left(Ha\right)}_i\left[{\left(W^{-\mathbf{1}}\epsilon \right)}_i+i\right]}{\sqrt{n}}$ is the zero mean independent random variable. Therefore, ${\mathbf{K}}_i$ has a mean of 0 and variance as follows:

$${\displaystyle \sum }_i\mathrm{Var}(K_i)=a^{\mathrm{T}}{\sigma }_{ki}^{2}Z(\frac{1}{n}{\displaystyle \sum }_{\mathit{i}=1}^n{w}_{ki}) a+\left({\mathit{a}}^{\mathrm{T}}Z\mathit{a}\right)\frac{1}{\mathit{n}}{\displaystyle \sum }_{\mathit{i}}{\left({\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right)}^2$$

Next, by taking into account the Assumptions 1–3, then ${\displaystyle \sum }_i\mathrm{Var}(K_i)$ converges to $a^{\mathit{T}}{\sigma }_{ki}^{2}Z\mathit{\vartheta }a$. Next, we have:

$$\begin{gathered} {\displaystyle \sum }_{i}E{\left|K_i\right|}^3=n^{-3/2}{\displaystyle \sum }_{i}E\left({\left|{\left(Ha\right)}_i\right|}^3{\left|{\left(W^{-\mathbf{1}}\epsilon \right)}_i+{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right|}^3\right) \\ =n^{-3/2}E{\left|{\left(Ha\right)}_1\right|}^3{\displaystyle \sum }_{i}E\left({\left|{\left(W^{-\mathbf{1}}\epsilon \right)}_i+{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right|}^3\right) \end{gathered}$$

On the other hand, we obtain:

$${\displaystyle \sum }_{i}E{\left|K_{\mathrm{i}}\right|}^3\le E{\left|{\left(Ha\right)}_1\right|}^3\left(n^{-1/2}\underset{i}{\max }\left(E{\left|{\left(W^{-\mathbf{1}}\epsilon \right)}_i\right|}^3\right)+n^{-3/2}{\displaystyle \sum }_i{\left|{\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h\right]}_i\right|}^3\right)$$

Due to Lemma 2 and finite third absolute moment of ${\left(W^{-\mathbf{1}}\epsilon \right)}_i$ then ${\displaystyle \sum }_{i}E{\left|K_i\right|}^3$ converges to zero.

Thus, ${\displaystyle \sum }_i{K}_i$ converges to a Normal distribution that is $N\left(\mathbf{0},a^{\mathrm{T}}{\mathsf{\sigma }}_{ki}^{2}Z\mathbf{\vartheta }a \right)$. □

Furthermore, the following Theorem provides asymptotic normality of $\widehat{\delta }$ that is a truncated spline estimator for parameters of the MMSR model presented in Equation (1). □

Theorem 4.

If $\widehat{\delta }$is a truncated spline estimator of the parameters in the MMSR model presented in Equation (1), then for $n\to \infty$,

$$\sqrt{n}\left(\widehat{\delta }-\delta \right)\stackrel{d}{\to }D ~ N\left(\mathbf{0},{\mathsf{\sigma }}_{\mathit{k}\mathit{i}}^{\mathbf{2}}{Z}^{-1}{\mathit{\vartheta }}^{\mathbf{-}1}\right)$$

Proof of Theorem 4.

By considering Equation (28), we can express $\sqrt{n}\left(\widehat{\delta }-\delta \right)$ as follows:

$$\begin{array}{c}\sqrt{\mathbf{n}}\left(\widehat{\delta }-\delta \right)={\left(\frac{H^{\mathit{T}}{W}^{-\mathbf{1}}H}{\mathit{n}}\right)}^{-\mathbf{1}}\left(\frac{H^{\mathit{T}}{W}^{-1}\epsilon }{\sqrt{\mathit{n}}}\right)\hfill \\ ={\left(\frac{H^{\mathrm{T}}\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}H}{n}\right)}^{-1}\left\{\frac{H^{\mathrm{T}}\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h+W^{-\mathbf{1}}\epsilon \right]}{\sqrt{n}}-\frac{H^{\mathrm{T}}{M}^{\mathit{T}}{W}^{-\mathbf{1}}\epsilon }{\sqrt{n}}\right\}\end{array}$$

Hence, based on this we obtain:

$${\left(\frac{H^{\mathrm{T}}\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}H}{n}\right)}^{-\mathbf{1}} \stackrel{P}{\to } Z^{-\mathbf{1}}{\mathit{\vartheta }}^{-\mathbf{1}}$$

as $n\to \infty$ and $\frac{H^{\mathrm{T}}{M}^{\mathit{T}}{W}^{-\mathbf{1}}\epsilon }{\sqrt{n}}\stackrel{P}{\to }\mathbf{0}$ as $n\to \infty$.

Furthermore, based on Theorem 3, we have:

$$\frac{ H^{\mathrm{T}}\left[\left(I-M^{\mathit{T}}\right)W^{-\mathbf{1}}h+W^{-\mathbf{1}}\epsilon \right]}{\sqrt{n}}\stackrel{d}{\to }{D}^{\ast }~N\left(\mathbf{0} ,{ \mathsf{\sigma }}_{ki}^{2}Z\mathit{\vartheta }\right) \mathrm{as} n\to \infty .$$

Next, by applying Slutsky theorem [45], we obtain:

$$\sqrt{n}\left(\widehat{\delta }-\delta \right)\stackrel{d}{\to }D ~ N\left(\mathbf{0} ,{\mathsf{\sigma }}_{\mathit{k}\mathit{i}}^{\mathbf{2}}{Z}^{-1}{\mathit{\vartheta }}^{-1}\right) \mathrm{as} n\to \infty .$$

□

3.5. Simulation Study

In this section we give a simulation study for a MMSR model constructed by three response variables and three predictor variables.

(i)

Simulation Design: The simulation study scenarios are decided as follows:

• We generate three samples sized $n=30, 50 \mathrm{and} 100$.
• The response vector $y={\left(y_1,y_2,y_3\right)}^T$, consisting of three response variables, is created from the MMSR model given in (ii).
• The design vector $X=\left(x_1,x_2, x_3\right)$ is produced from a uniform distribution.
• For each model, a total of nine regression coefficients specified as ${\alpha }_1={\left(-1, 2, 4\right)}^{\mathit{T}}$, ${\alpha }_2={\left(-2, 3, 5\right)}^{\mathit{T}},$ ${\alpha }_3={\left(-0.5, 1, 3\right)}^{\mathit{T}}$ are considered here.
• In addition, the MMSR model to be generated includes two different smooth functions, $g_1\left(t_1\right)$ and $g_2\left(t_2\right)$ with nonparametric covariates $t_1$ and $t_2$, respectively.
• The number of replications for each sample used in simulation experiments is considered as 1000.

(ii)

Data Generation: The MMSR model can be written as follows, according to given information in the simulation design;

$$\left(\begin{array}{c}{y}_1\\ y_2\\ y_3\end{array}\right)=\left(x_1 x_2 x_3\right)\left(\begin{array}{c}{\alpha }_1\\ {\alpha }_2\\ {\alpha }_3\end{array}\right)+{\displaystyle \sum _{\mathrm{j}=1}^{q=2}}{g}_j\left(t_{ji}\right)+{\epsilon }_i, 1\le i\le n$$

where $X=\left(x_1,x_2, x_3\right)$ is a $\left(n\times 9\right)$-dimensional design matrix and each $x_i$ is generated from a uniform distribution, that is, $x_i~U\left[{\mathbf{0}}_{n\times 3},{\mathbf{1}}_{n\times 3}\right]$. The vector of regression coefficients $\alpha =\left({\alpha }_1, {\alpha }_2, {\alpha }_3\right)$ is defined in (i) above.

▪

$g_1\left(t_1\right)$ and $g_2\left(t_2\right)$ are computed by using $t_1={\left\{\left(i-0.5\right)/n\right\}}_{i=1}^n$ and $t_2~U\left[-2, 2\right]$ as follows:

$$\begin{array}{c}{g}_1\left(t_1\right)=1-48t_1+218t_1^2-315t_1^3+145t_1^4\hfill \\ g_2\left(t_2\right)=\sin \left(2t_2\right)+2e^{-16t_2^2}\hfill \end{array}$$

▪

Finally, the random error terms ${\epsilon }_i$’s are independent and identically distributed from the multivariate normal distribution ${\epsilon }_i~MN\left(0,{\Sigma }_{\epsilon }^2\right)$ for three models.

The results and comments regarding the simulation study are given in the following tables and figures below.

Table 1. Estimated regression coefficients from parametric component of the MMSR model.

	95% Confidence Interval
	$\mathit{n}=30$			$\mathit{n}=50$			$\mathit{n}=100$
Coefficients Vector	Lower	${\widehat{\mathit{\alpha }}}_{\mathit{j}}$	Upper	Lower	${\widehat{\mathit{\alpha }}}_{\mathit{j}}$	Upper	Lower	${\widehat{\mathit{\alpha }}}_{\mathit{j}}$	Upper
${\alpha }_1={\left(-1, 2, 4\right)}^{\mathit{T}}$	−1.308	−0.917	−0.527	−1.164	−0.929	−0.693	−1.033	−0.923	−0.813
	1.714	2.114	2.513	1.895	2.120	2.346	1.977	2.089	2.20
	3.644	4.044	4.444	3.811	4.041	4.271	3.979	4.087	4.196
${\alpha }_2={\left(-2, 3, 5\right)}^{\mathit{T}}$	−2.308	−1.917	−1.527	−2.164	−1.929	−1.693	−2.033	−1.923	−1.813
	2.714	3.114	3.513	2.895	3.120	3.346	2.977	3.089	3.200
	4.644	5.044	5.444	4.811	5.041	5.271	4.979	5.087	5.196
${\alpha }_3={\left(-0.5, 1, 3\right)}^{\mathit{T}}$	−0.808	−0.417	−0.027	−0.664	−0.429	−0.193	−0.533	−0.423	−0.313
	0.714	1.114	1.513	0.895	1.120	1.346	0.977	1.089	1.200
	2.644	3.044	3.444	2.811	3.041	3.271	2.979	3.087	3.196

contains the estimated regression coefficients for the parametric component of the MMSR model and their 95% confidence intervals. From this, it can be said that as the sample size increases, the confidence intervals narrow. Note that this inference is supported by the boxplots in Figure 1. In addition, it can be said that the proposed model has succeeded in obtaining satisfactory estimates for the parametric component.

Figure 1 is drawn to see the convergence of the estimated regression coefficients to the true coefficients. It can be seen from the boxplots that, the range of the boxplot gets smaller as the sample sizes get larger. We would like to point out that this is an expected situation. Regarding the non-parametric component of the MMSR model,

Table 2. Results of Mann–Whitney U test for the nonparametric components.

		$\mathit{n}=30$		$\mathit{n}=50$		$\mathit{n}=100$
		$\mathbf{Med}\left(\mathbf{diff}\right)$	p-Value	$\mathbf{Med}\left(\mathbf{diff}\right)$	p-Value	$\mathbf{Med}\left(\mathbf{diff}\right)$	p-Value
${\widehat{y}}_1$	$g_1-{\widehat{g}}_1$	0.567	0.911 *	0.566	0.806 *	0.603	0.690 *
	$g_2-{\widehat{g}}_2$	0.842	0.830 *	0.675	0.817 *	0.675	0.789 *
${\widehat{y}}_2$	$g_1-{\widehat{g}}_1$	1.005	0.030	0.644	0.086 *	0.694	0.033
	$g_2-{\widehat{g}}_2$	0.804	0.030	0.701	0.046	0.671	0.020
${\widehat{y}}_3$	$g_1-{\widehat{g}}_1$	0.568	0.912 *	0.566	0.807 *	0.603	0.689 *
	$g_2-{\widehat{g}}_2$	0.638	0.830 *	0.675	0.817 *	0.675	0.790 *

$*$: $H_0:\left(g_1-{\widehat{g}}_1\right)=0, H_1:\left(g_1-{\widehat{g}}_1\right)\ne 0; H_0:\left(g_2-{\widehat{g}}_2\right)=0, H_1:\left(g_2-{\widehat{g}}_2\right)\ne 0$.

and Figure 2 are obtained. Additionally, a Mann–Whitney U test is performed to determine the statistical significance of the difference between fitted curve and the real functions for both ${\mathrm{g}}_1\left(t_1\right)$ and ${\mathrm{g}}_2\left(t_2\right)$, and the results are given in Table 2 below.

When the results given in Table 2 are examined carefully, it is seen that most of the p-Values are greater than 0.05 (these are shown in bold). As can be seen from this, it is clear that in most cases, the null hypothesis claiming that there is no difference between the medians of the fitted curves and the real functions cannot be rejected. In addition, these results are confirmed by the graphs given in Figure 2. Moreover, the influence of sample size is clearly visible from the panels in Figure 2.

After the parametric and nonparametric components, Figure 3 is obtained to examine the overall residuals from the total MMSR model. It should also be noted that Figure 3 includes the scatter plots for residuals versus fitted values of the MMSR model for all sample sizes. Note that although there is some improvement in the performance of the model when n = 100, it is clear that the residuals are not randomly distributed around zero. The existence of a heteroscedasticity problem is debatable. However, it has also been observed that when the sample size is large, the residuals do not exceed the standard deviation limits.

4. Conclusions

The estimated MMSR model we obtained is a combination of estimations between the parametric and nonparametric components, and it is linear to observation. Additionally, we found that the $100\left(1-\alpha \right)\%$ confidence interval for parameters in a MMSR model depend on $t$-student distribution namely $t_{\left(\frac{\alpha }{2};np-p^{\ast }\right)}$ and the estimator of parameters in a MMSR model is consistent and asymptotically normally distributed. Based on simulation results, the influence of sample size is clearly visible and it also has been observed that when the sample size is large, the residuals do not exceed the standard deviation limits.