The Curve Estimation of Combined Truncated Spline and Fourier Series Estimators for Multiresponse Nonparametric Regression

Abstract

Nonparametric regression becomes a potential solution if the parametric regression assumption is too restrictive while the regression curve is assumed to be known. In multivariable nonparametric regression, the pattern of each predictor variable’s relationship with the response variable is not always the same; thus, a combined estimator is recommended. In addition, regression modeling sometimes involves more than one response, i.e., multiresponse situations. Therefore, we propose a new estimation method of performing multiresponse nonparametric regression with a combined estimator. The objective is to estimate the regression curve using combined truncated spline and Fourier series estimators for multiresponse nonparametric regression. The regression curve estimation of the proposed model is obtained via two-stage estimation: (1) penalized weighted least square and (2) weighted least square. Simulation data with sample size variation and different error variance were applied, where the best model satisfied the result through a large sample with small variance. Additionally, the application of the regression curve estimation to a real dataset of human development index indicators in East Java Province, Indonesia, showed that the proposed model had better performance than uncombined estimators. Moreover, an adequate coefficient of determination of the best model indicated that the proposed model successfully explained the data variation.

1. Introduction

As one of the renowned methods of regression analysis, parametric regression has been used for many years in various scientific fields. However, some parametric regression assumptions are too restrictive, such as identifying a regression curve’s shape with some prespecified functional form (e.g., linear, quadratic, or cubic) [1]. In real datasets, not all regression curves have a visible pattern due to the absence of relationship information between response and predictor variables. Therefore, in such scenarios, nonparametric regression analysis is more recommended [2]. Nonparametric regression is able to degrade the misspecification risk because data inherently represent the real shape of a regression curve without interference due to the researcher’s subjectivity [2].

To date, researchers have investigated various functions for estimating the regression curve in nonparametric regression, such as spline [3,4,5,6], Fourier series [7,8,9], kernel [10,11,12], polynomial [13,14,15], and wavelet [16,17,18] functions. The present study explores the nonparametric regression approach to analyzing spatial data, i.e., the use of the nonparametric truncated spline function in geographically weighted regression [19,20]. In nonparametric regression with more than one predictor variable (multivariable), the pattern of the relationship between each predictor variable and the response variable is not always the same; therefore, in this study, we employ a nonparametric regression model with a combined estimator. Considering the concept of semiparametric regression, some studies have proposed combined estimator models to estimate the regression curve, such as spline and kernel functions [21,22,23]. Similarly, in [24], the researchers developed a combined estimator of kernel and Fourier series, whereas in [25,26,27], the researchers presented nonparametric regression with a mixed estimator of truncated spline and Fourier series. Recently, researchers have also been giving attention to a mixed model of longitudinal data, as carried out by [28].

So far, studies on nonparametric regression with combined estimators have not dealt with more than two responses, i.e., multiresponse nonparametric regression. Numerous studies on nonparametric regression have been applied to many scientific fields, such as sociodemographic analysis [23,24,25,26], finance [5,29], climatology [4], and economics [30], wherein some cases have involved real data with two or more correlated response variables. Therefore, this study makes a major contribution to the research on multiresponse nonparametric regression by using a combined estimator. Of the several types of regression functions for nonparametric regression, the truncated spline function has the major advantages of high flexibility, good visual interpretation, and the ability to handle smooth function characters and data that have changed behavior at certain subintervals [2]. On the other hand, the Fourier series function is used to estimate the regression curve if the data are smooth and follow a repeated pattern at specific intervals [9]. The Fourier series function is also a trigonometric polynomial that can adjust the data’s local nature effectively. In this study, we adopt a cosine Fourier series function [9], which follows a trend line as the regression curve estimator. A Fourier series with a cosine function was employed since it is an even function, such that the second derivative usually mathematically obtains a scalar or a nonzero value. Furthermore, the penalties in the penalized least square method can be well defined. Considering some of the advantages of these two functions, as outlined above, this study highlights the combined use of truncated spline and Fourier series estimators for multiresponse nonparametric regression.

One of the most well-known tools for the estimation of regression is the ordinary least square (OLS) method. However, the OLS method cannot be directly used in nonparametric regression because of the unknown curve shape. By utilizing smooth, continuous, and differentiable properties, as well as other components, the OLS method was modified with conditional optimization to create the penalized least-square (PLS) regression method [31]. The combined use of estimators for multiresponse nonparametric regression leads to the use of an error variance–covariance matrix as a weighting matrix. Therefore, PLS optimization is still added with a weight; accordingly, the proposed model was obtained through penalized weighted least square (PWLS) optimization. To evaluate the proposed model performance, we employed a simulation study with three different sample sizes and three error variances and applied the proposed model to a real dataset.

Given the view above, the objective of this study is to obtain a curve estimation using combined truncated spline and Fourier series estimators for multiresponse nonparametric regression. This aim is followed by estimating an error variance–covariance matrix as a weighting matrix. To estimate the proposed model, a two-stage estimation method was used. Stage 1 was to estimate the Fourier series component using PWLS optimization. The following stage was to estimate the truncated spline component by utilizing weighted least square (WLS) optimization. In addition to curve estimation, we attempted to simulate and apply the proposed model to a dataset of the HDI in East Java Province, Indonesia. The paper is organized as follows: Section 1 provides an overview of the topic while identifying the knowledge gap and stating the aims of the research. Section 2 describes the truncated spline function and the Fourier series function, along with the PWLS method. Section 3 presents the findings of the regression curve estimation via the combined use of truncated spline and Fourier series estimators for multiresponse nonparametric regression, along with an estimation of the variance–covariance matrix, smoothing parameter selection, and a simulation study following the application of a real dataset. Section 4 gives a brief conclusion and makes future recommendations.

2. Materials and Methods

2.1. Multiresponse Nonparametric Regression, Truncated Spline Function, and Fourier Series Function

Given paired data $\left(y_{1i},y_{2i},\dots ,y_{ri}{x}_{1i},x_{2i},\dots ,x_{pi},z_{1i},z_{2i},\dots ,z_{qi}\right),$ the relationship between response $\left(y_{1i},y_{2i},\dots ,y_{ri}\right)$ and predictor $\left(x_{1i},x_{2i},\dots ,x_{pi},z_{1i},z_{2i},\dots ,z_{qi}\right)$ variables is assumed to follow multiresponse nonparametric regression, as follows:

$$y_{hi}={\mu }_{hi}\left(x_{1i},x_{2i},\dots ,x_{pi},z_{1i},z_{2i},\dots ,z_{qi}\right)+{\epsilon }_{hi},{\epsilon }_{hi}~N\left(0,{\sigma }_h^2\right)$$

(1)

where $h=1,2,\dots ,r$ and $i=1,2,\dots ,n$. In this study, r is defined as the number of response variables, and n refers to the number of observations for each response and predictor variable. For convenience, Equation (1) can be rewritten in the following matrix form:

$$\mathbf{y}=\mathsf{\mu }\left(\mathbf{x},\mathbf{z}\right)+\mathsf{\epsilon }$$

(2)

The regression curve in Equation (1) can be assumed to be an additive model.

$${\mu }_{hi}\left(x_{1i},x_{2i},\dots ,x_{pi},z_{1i},z_{2i},\dots ,z_{qi}\right)={\displaystyle \sum }_{j=1}^p{f}_{hj}\left(x_{ji}\right)+{\displaystyle \sum }_{k=1}^q{g}_{hk}\left(z_{ki}\right).$$

(3)

In Equation (3), the regression curve $f_{hj}\left(x_{ji}\right);j=1, 2, \dots , p$ is assumed to be smooth and is approached by a truncated spline function. Meanwhile, the regression curve $g_{hk}\left(z_{ki}\right);k=1,2,\dots ,q$ is approached by a Fourier series function, which assumes that the regression curve is unknown and is contained in continuous space $C\left[0,\pi \right]$. Hence, the multiresponse nonparametric regression in Equation (3) can be written as

$$y_{hi}={\displaystyle \sum }_{j=1}^p{f}_{hj}\left(x_{ji}\right)+{\displaystyle \sum }_{k=1}^q{g}_{hk}\left(z_{ki}\right)+{\epsilon }_{hi}, {\epsilon }_{hi}~N\left(0,{\sigma }_h^2\right).$$

(4)

Under the assumption ${\epsilon }_{hi}~N\left(0,{\sigma }_h^2\right)$, as described in [32], $corr\left({\epsilon }_{hi}{\epsilon }_{\ell i}\right)=\rho$ for $h\ne \ell ;$ $h,\ell =1,2,\dots ,r$. This term refers to situations in which the response variables $y_{hi}$ and $y_{\ell i}$ are a pair, such that a correlation between the $h$-th response and the $\ell$-th response yields $corr\left({\epsilon }_{hi}{\epsilon }_{\ell i}\right)=\rho$, and zero otherwise. The error correlation between responses is the same for every response, defined as $\rho =\frac{\mathrm{cov}({\epsilon }_{hi}{\epsilon }_{\ell i})}{\sqrt{{\sigma }_{hh}{\sigma }_{\ell \ell }}}$; $i=1,2,\dots ,n$; $h\ne \ell ;$ $h,\ell =1,2,\dots ,r$.

Thus, the regression curve $f_{hj}\left(x_{ji}\right)$ in Equation (4) is approached by a linear truncated spline function with knots $K_{hj1}, K_{hj2},\dots ,K_{hju}$, as follows:

$$f_{hj}\left(x_{ji}\right)={\alpha }_{hj}{x}_{ji}+{\displaystyle \sum }_{s=1}^u{\beta }_{hjs}{\left(x_{ji}-K_{hjs}\right)}_{+},$$

(5)

with the truncated function

$${\left(x_{ji}-K_{hjs}\right)}_{+}=\{\begin{array}{cc}{\left(x_{ji}-K_{hjs}\right)}_{+}\hfill & ,xji\ge K_{hjs}\hfill \\ 0\hfill & ,x_{ji}<K_{hjs}\hfill \end{array}$$

Adopting the Fourier series function in [9], $g_{hk}\left(z_{ki}\right)$ in Equation (4) is approached by a Fourier series cosine function following the trend line ($b_{hk}{z}_{ki}$), as given in Equation (6):

$$g_{hk}\left(z_{ki}\right)=b_{hk}{z}_{ki}+\frac{1}{2}{a}_{ohk}+{\displaystyle \sum }_{t=1}^T{a}_{thk}\cos t{z}_{ki}.$$

(6)

2.2. Penalized Weighted Least Square

Suppose that the dataset $\left(y_{1i},y_{2i},\dots ,y_{ri},x_{1i},x_{2i},\dots ,x_{pi},z_{1i},z_{2i},\dots ,z_{qi}\right)$ follows the multiresponse nonparametric regression in Equation (1) and is rewritten in matrix form as Equation (2). The random error $\mathsf{\epsilon }$ in Equation (2) is normally distributed, with mean 0 and error variance–covariance matrix W, such that it can be written as $\mathit{\epsilon }~N\left(\mathbf{0},\mathbf{W}\right)$. In particular, $\mathbf{W}$ as a weighting matrix plays an important role in accommodating the correlation between responses. If there is a correlation between responses, the correlation is defined as $\rho =\frac{{\sigma }_{h\ell }}{\sqrt{{\sigma }_{hh}{\sigma }_{\ell \ell }}}$, such that ${\sigma }_{h\ell }=\rho \sqrt{{\sigma }_{hh}{\sigma }_{\ell \ell }}$. Note that $\mathbf{I}$ is the identity matrix. When all observations come in pairs, $\mathbf{W}$ has ${\sigma }_{h,\ell }$ elements, as described below:

$$\mathbf{W}=\left(\begin{array}{cccc}{\sigma }_{11}& {\sigma }_{12}& \cdots & {\sigma }_{1r}\\ {\sigma }_{12}& {\sigma }_{22}& \cdots & {\sigma }_{2r}\\ ⋮& ⋮& \ddots & ⋮\\ {\sigma }_{1r}& {\sigma }_{2r}& \cdots & {\sigma }_{rr}\end{array}\right)\otimes \mathbf{I}$$

$$=\mathsf{\Sigma }\otimes \mathbf{I}$$

The regression curve of the combined truncated spline and Fourier series estimators for multiresponse nonparametric regression was estimated by carrying out the following PWLS optimization:

$$\underset{g_k\in C\left[0,\pi \right]}{\mathrm{Min}}\left\{N^{-1}{\displaystyle \sum }_{h=1}^r{\displaystyle \sum }_{i=1}^n{w}_{hi}{\left(y_{hi}-{\displaystyle \sum }_{j=1}^p{f}_{hj}\left(x_{ji}\right)-{\displaystyle \sum }_{k=1}^q{g}_{hk}\left(z_{ki}\right)\right)}^2+{\displaystyle \sum }_{k=1}^q{\lambda }_k{{\displaystyle \int }}_0^{\pi }\frac{2}{\pi }{\left(g_k^{″}\left(z_k\right)\right)}^{2}d{z}_k\right\},$$

(7)

where $N$ refers to the number of observations for all response variables, alternatively written as $N={\sum }_{h=1}^r{n}_h$. The first component in Equation (7) is a function that measures the goodness-of-fit (GoF), while the second component is the penalty. In addition, $w_{hi}$ is a weighted component, and ${\lambda }_k$ serves as a positive smoothing parameter that controls the balance between the GoF and the penalty. In this study, during the process of regression curve estimation, the parameters $w_{hi}$ and ${\lambda }_k$ are given. Note that the superscript ^T refers to transpose and the superscript “ refers to the second derivative of the function.

As stated in the Introduction section, we estimated the regression curve of the proposed model in two stages. The first stage involves completing the estimation of the regression curve $g_{hk}\left(z_{ki}\right)$ through PWLS optimization, which results in Theorem 1 (see the Section 3.1). Subsequently, the second stage involves completing the estimated regression curve $f_{hj}\left(x_{ji}\right)$ using WLS optimization, which results in Theorem 2 (see the Section 3.1).

3. Results

3.1. Curve Estimation of Combined Truncated Spline and Fourier Series Estimators for Multiresponse Nonparametric Regression

As previously stated, PWLS and WLS optimization were used to obtain combined truncated spline and Fourier series estimators for multiresponse nonparametric regression. Therefore, some lemmas and theorems were required. Whenever Lemma 1 presents the GoF, Lemma 2 presents the penalty of the PWLS optimization. By using the results of Lemma 1 and Lemma 2, the first stage estimation is obtained, which is formulated in Theorem 1. The second stage involves estimating the regression curve using WLS optimization, as presented in Lemma 3, with the main result presented in Theorem 2. All the proofs of the lemmas and theorems are presented in Appendix A, Appendix B, Appendix C, Appendix D and Appendix E.

Lemma 1.

If the multiresponse nonparametric regression model is written as Equation (4) and the regression curve$g_{hk}\left(z_{ki}\right)$is as given in Equation (6), the GoF can be formulated as follows:

$$R\left({\mathbf{g}}_1,\dots ,{\mathbf{g}}_r\right)=N^{-1}{\left(\mathbf{v}-\mathbf{Z}\mathbf{a}\right)}^T\mathbf{W}\left(\mathbf{v}-\mathbf{Z}\mathbf{a}\right),$$

where$\mathbf{W}$is the$nh\times nh$weighting matrix,

$$\mathbf{v}={\left[\begin{array}{cccc}{\mathbf{v}}_1^T& {\mathbf{v}}_2^T& \cdots & {\mathbf{v}}_r^T\end{array}\right]}^T;{\mathbf{v}}_h={\left[\begin{array}{cccc}{v}_{h1}& v_{h2}& \cdots & v_{hn}\end{array}\right]}^T;v_{hi}=y_{hi}-{\sum }_{j=1}^p{f}_{hj}(x_{ji})$$

$$\mathbf{Z}=\left[\begin{array}{cccc}{\mathbf{Z}}_1& \mathbf{0}& \cdots & \mathbf{0}\\ \mathbf{0}& {\mathbf{Z}}_2& \cdots & \mathbf{0}\\ ⋮& ⋮& \ddots & ⋮\\ \mathbf{0}& \mathbf{0}& \cdots & {\mathbf{Z}}_r\end{array}\right], \mathbf{a}={\left[\begin{array}{ccc}{a}_1^T& a_2^T& \begin{array}{cc}\dots & a_r^T\end{array}\end{array}\right]}^T,$$

$${\mathbf{Z}}_h=\left[\begin{array}{ccccccccccccc}{z}_{11}& 1/2& \cos 1z_{11}& \cos 2z_{11}& \cdots & \cos T{z}_{11}& \cdots & z_{q1}& 1/2& \cos 1z_{q1}& \cos 2z_{q1}& \cdots & \cos T{z}_{q1}\\ z_{12}& 1/2& \cos 1z_{12}& \cos 2z_{12}& \cdots & \cos T{z}_{12}& \cdots & z_{q2}& 1/2& \cos 1z_{q2}& \cos 2z_{q2}& \cdots & \cos T{z}_{q2}\\ ⋮& ⋮& ⋮& ⋮& \ddots & ⋮& \cdots & ⋮& ⋮& ⋮& ⋮& \ddots & ⋮\\ z_{1n}& 1/2& \cos 1z_{1n}& \cos 2z_{1n}& \cdots & \cos T{z}_{1n}& \cdots & z_{qn}& 1/2& \cos 1z_{qn}& \cos 2z_{qn}& \cdots & \cos T{z}_{qn}\end{array}\right],$$

$${\mathbf{a}}_h={\left[\begin{array}{cccccccccccc}{b}_{1h}& a_{01h}& a_{11h}& a_{21h}& \cdots & a_{T1h}& \cdots & b_{qh}& a_{0qh}& a_{1qh}& a_{2qh}& \begin{array}{cc}\cdots & a_{Tqh}\end{array}\end{array}\right]}^T.$$

The proof of Lemma 1 is given in Appendix A.

Lemma 2.

If the multiresponse nonparametric regression model is written as Equation (4) and the regression curve$g_{hk}\left(z_{ki}\right)$is as presented in Equation (6), the penalty component is as follows:

$$P\left({\lambda }_1,{\lambda }_2,\dots ,{\lambda }_q\right)={\mathbf{a}}^T\mathbf{D}\left(\lambda \right)\mathbf{a}$$

where

$$\mathbf{D}(\lambda )=\left[\begin{array}{ccccccccccccc}0& 0& 0& 0& \cdots & 0& \cdots & 0& 0& 0& 0& \cdots & 0\\ 0& 0& 0& 0& \cdots & 0& \cdots & 0& 0& 0& 0& \cdots & 0\\ 0& 0& {\lambda }_1{1}^4& 0& \cdots & 0& \cdots & 0& 0& 0& 0& \cdots & 0\\ 0& 0& 0& {\lambda }_1{2}^4& \cdots & 0& \cdots & 0& 0& 0& 0& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮& \ddots & ⋮& \cdots & ⋮& ⋮& ⋮& ⋮& \ddots & ⋮\\ 0& 0& 0& 0& \cdots & {\lambda }_1{T}^4& \cdots & 0& 0& 0& 0& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮& \cdots & ⋮& \ddots & ⋮& ⋮& ⋮& ⋮& \cdots & ⋮\\ 0& 0& 0& 0& \cdots & 0& \cdots & 0& 0& 0& 0& \cdots & 0\\ 0& 0& 0& 0& \cdots & 0& \cdots & 0& 0& 0& 0& \cdots & 0\\ 0& 0& 0& 0& \cdots & 0& \cdots & 0& 0& {\lambda }_q{1}^4& 0& \cdots & 0\\ 0& 0& 0& 0& \cdots & 0& \cdots & 0& 0& 0& {\lambda }_q{2}^4& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮& \ddots & ⋮& \cdots & ⋮& ⋮& ⋮& ⋮& \ddots & ⋮\\ 0& 0& 0& 0& \cdots & 0& ⋮& 0& 0& 0& 0& \cdots & {\lambda }_q{T}^4\end{array}\right].$$

A complete description of the proof of Lemma 2 can be found in Appendix B.

Having discussed how to construct the GoF and the penalty, Theorem 1 addresses potential solutions to the first-stage estimation using PWLS optimization in Equation (7).

Theorem 1.

If the GoF and the penalty of the model are given in Lemma 1 and Lemma 2, the curve estimation for multiresponse nonparametric regression can be attained from PWLS optimization, as follows:

$${\widehat{\mathbf{g}}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}(\mathbf{x},\mathbf{z})=\left(\mathbf{Z}{\left[{\mathbf{Z}}^T\mathbf{W}\mathbf{Z}+N\mathbf{D}(\lambda )\right]}^{-1}{\mathbf{Z}}^T\mathbf{W}\right)\mathbf{v}.$$

The evidence of Theorem 1 is given in Appendix C. The second stage involves estimating the regression curve using WLS optimization, as described in Lemma 3, with the result in Theorem 2.

Lemma 3.

If the regression curve$f_{hj}\left(x_{ji}\right)$is as described in Equation (5), WLS optimization can be obtained as

$${\left[\left(\mathbf{I}-\mathbf{H}\right)\left(\mathbf{y}-\mathbf{J}\mathsf{\gamma }\right)\right]}^T\mathbf{W}\left[\left(\mathbf{I}-\mathbf{H}\right)\left(\mathbf{y}-\mathbf{J}\mathsf{\gamma }\right)\right],$$

where$\mathbf{H}=\left(\mathbf{Z}{\left[{\mathbf{Z}}^T\mathbf{W}\mathbf{Z}+N\mathbf{D}(\lambda )\right]}^{-1}{\mathbf{Z}}^T\mathbf{W}\right),$

$${\mathbf{S}}_i=\left[\begin{array}{ccccccccc}{(x_{11}-K_{h11})}_{+}& {(x_{11}-K_{h12})}_{+}& \cdots & {(x_{11}-K_{h1u})}_{+}& \cdots & {(x_{p1}-K_{hp1})}_{+}& {(x_{11}-K_{hp2})}_{+}& \cdots & {(x_{11}-K_{hpu})}_{+}\\ {(x_{12}-K_{h11})}_{+}& {(x_{12}-K_{h12})}_{+}& \cdots & {(x_{12}-K_{h1u})}_{+}& \cdots & {(x_{p2}-K_{hp1})}_{+}& {(x_{12}-K_{hp2})}_{+}& \cdots & {(x_{12}-K_{hpu})}_{+}\\ ⋮& ⋮& \ddots & ⋮& \cdots & ⋮& ⋮& \ddots & ⋮\\ {(x_{1n}-K_{h11})}_{+}& {(x_{1n}-K_{h12})}_{+}& \cdots & {(x_{1n}-K_{h1u})}_{+}& \cdots & {(x_{pn}-K_{hp1})}_{+}& {(x_{1n}-K_{hp2})}_{+}& \cdots & {(x_{1n}-K_{hpu})}_{+}\end{array}\right],\mathsf{\gamma }=\left[\frac{\begin{array}{c}{\mathsf{\alpha }}_1\\ {\mathsf{\alpha }}_2\\ ⋮\\ {\mathsf{\alpha }}_r\end{array}}{\begin{array}{c}{\mathsf{\beta }}_1\\ {\mathsf{\beta }}_2\\ ⋮\\ {\mathsf{\beta }}_r\end{array}}\right]$$

$${\mathsf{\alpha }}_h={\left[\begin{array}{cccc}{\alpha }_{1h}& {\alpha }_{2h}& \cdots & {\alpha }_{ph}\end{array}\right]}^T,{\mathsf{\beta }}_h={\left[\begin{array}{ccccccccc}{\beta }_{h11}& {\beta }_{h12}& \cdots & {\beta }_{h1u}& \cdots & {\beta }_{hp1}& {\beta }_{hp2}& \cdots & {\beta }_{hpu}\end{array}\right]}^T.$$

As a note, Lemma 3′s proof is provided in Appendix D.

Theorem 2.

If WLS optimization is given by Lemma 3, the regression curve estimation for multiresponse nonparametric regression can be attained from WLS optimization such that

$${\widehat{\mathbf{f}}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)=\mathbf{J}{\left[{\mathbf{J}}^T\left(\mathbf{I}-2{\mathbf{H}}^T\right)\mathbf{W}\mathbf{J}+{\mathbf{J}}^T{\mathbf{H}}^T\mathbf{W}\mathbf{H}\mathbf{J}\right]}^{-1}\left[{\mathbf{J}}^T\left({\mathbf{H}}^T-\mathbf{I}\right)\mathbf{W}\left(\mathbf{H}-\mathbf{I}\right)\right]\mathbf{y}.$$

(8)

$$=\mathbf{J}{\mathbf{K}}^{-1}\mathbf{L}\mathbf{y},$$

(9)

where$\mathbf{K}=\left[{\mathbf{J}}^T\left(\mathbf{I}-2{\mathbf{H}}^T\right)\mathbf{W}\mathbf{J}+{\mathbf{J}}^T{\mathbf{H}}^T\mathbf{W}\mathbf{H}\mathbf{J}\right]$and$\mathbf{L}=\left[{\mathbf{J}}^T\left({\mathbf{H}}^T-\mathbf{I}\right)\mathbf{W}\left(\mathbf{H}-\mathbf{I}\right)\right]$.

The proof of Theorem 2 is given in Appendix E.

After obtaining ${\widehat{\mathbf{f}}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)$ in Theorem 2, the result of ${\widehat{\mathbf{g}}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}(\mathbf{x},\mathbf{z})$ in Theorem 1 can be rewritten as Equation (12). First, substituting Equation (9) into Equation (A9) gives Equation (10):

$$\widehat{\mathbf{a}}={\left[{\mathbf{Z}}^T\mathbf{W}\mathbf{Z}+N\mathbf{D}(\lambda )\right]}^{-1}{\mathbf{Z}}^T\mathbf{W}\mathbf{v}$$

$$={\left[{\mathbf{Z}}^T\mathbf{W}\mathbf{Z}+N\mathbf{D}(\lambda )\right]}^{-1}{\mathbf{Z}}^T\mathbf{W}\left(\mathbf{y}-\mathbf{f}\right)$$

$$={\left[{\mathbf{Z}}^T\mathbf{W}\mathbf{Z}+N\mathbf{D}(\lambda )\right]}^{-1}{\mathbf{Z}}^T\mathbf{W}\left(\mathbf{y}-\mathbf{J}{\mathbf{K}}^{-1}\mathbf{L}\mathbf{y}\right)$$

$$={\left[{\mathbf{Z}}^T\mathbf{W}\mathbf{Z}+N\mathbf{D}(\lambda )\right]}^{-1}{\mathbf{Z}}^T\mathbf{W}\left(\mathbf{I}-\mathbf{J}{\mathbf{K}}^{-1}\mathbf{L}\right)\mathbf{y}.$$

(10)

If the result of Equation (10) is substituted into Equation (A10), the regression curve estimation of the Fourier series can be rewritten as Equation (12). As a note, Equation (A9) and Equation (A10) can be found in Appendix C.

$${\widehat{\mathbf{g}}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)=\mathbf{Z}\widehat{\mathbf{a}}$$

$$=\mathbf{Z}\left({\left[{\mathbf{Z}}^T\mathbf{W}\mathbf{Z}+N\mathbf{D}(\lambda )\right]}^{-1}{\mathbf{Z}}^T\mathbf{W}\left(\mathbf{I}-\mathbf{J}{\mathbf{K}}^{-1}\mathbf{L}\right)\mathbf{y}\right)$$

(11)

$$=\mathbf{H}\left(\mathbf{I}-\mathbf{J}{\mathbf{K}}^{-1}\mathbf{L}\right)\mathbf{y}.$$

(12)

Another main finding of this study is the regression curve of the combined truncated spline and Fourier series estimators for multiresponse nonparametric regression, as presented in Corollary 1.

Corollary 1.

According to the main result in Equation (8) and Equation (11), the regression curve estimation of the combined truncated spline and Fourier series estimators for the additive model is presented below:

$${\widehat{\mathsf{\mu }}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)=\left[\mathbf{J}{\mathbf{K}}^{-1}\mathbf{L}+\mathbf{H}\left(\mathbf{I}-\mathbf{J}{\mathbf{K}}^{-1}\mathbf{L}\right)\right]\mathbf{y}.$$

Proof. 

By using an additive model in Equation (3), the regression curve estimation of the combined estimator for multiresponse nonparametric regression can be written in the following matrix form:

$${\widehat{\mathsf{\mu }}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)={\widehat{\mathbf{f}}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)+{\widehat{\mathbf{g}}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right).$$

Based on the result of ${\widehat{\mathbf{f}}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)$ in Equation (8) and ${\widehat{\mathbf{g}}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)$ in Equation (11), thus ${\widehat{\mathsf{\mu }}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)$ can be drawn as

$$\begin{array}{c}{\widehat{\mathsf{\mu }}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)=\left\{\mathbf{J}{\left[{\mathbf{J}}^T\left(\mathbf{I}-2{\mathbf{H}}^T\right)\mathbf{W}\mathbf{J}+{\mathbf{J}}^T{\mathbf{H}}^T\mathbf{W}\mathbf{H}\mathbf{J}\right]}^{-1}\left[{\mathbf{J}}^T\left({\mathbf{H}}^T-\mathbf{I}\right)\mathbf{W}\left(\mathbf{H}-\mathbf{I}\right)\right]\mathbf{y}\right\}\\ +\left\{\mathbf{Z}\left({\left[{\mathbf{Z}}^T\mathbf{W}\mathbf{Z}+N\mathbf{D}(\lambda )\right]}^{-1}{\mathbf{Z}}^T\mathbf{W}\left(\mathbf{I}-\mathbf{J}{\mathbf{K}}^{-1}\mathbf{L}\right)\mathbf{y}\right)\right\}.\end{array}$$

(13)

$$=\mathbf{C}\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)\mathbf{y}$$

For simplification, by substituting Equation (9) and Equation (12) into Equation (13), ${\widehat{\mathsf{\mu }}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)$ can also be expressed as

$${\widehat{\mathsf{\mu }}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)=\left[\mathbf{J}{\mathbf{K}}^{-1}\mathbf{L}+\mathbf{H}\left(\mathbf{I}-\mathbf{J}{\mathbf{K}}^{-1}\mathbf{L}\right)\right]\mathbf{y}.$$

 □

3.2. Estimation of Error Variance–Covariance Matrix

The result in Equation (13) shows that curve estimation of the proposed model leads to the use of an error variance–covariance matrix as a weighting matrix. Hence, estimation of the error variance–covariance matrix is presented in Theorem 3.

Theorem 3.

If the regression curve estimation is given by Corollary 1 and random error ε is normally distributed with mean 0 and error variance–covariance matrix W such that it can be written as$\mathit{\epsilon }~N\left(\mathbf{0},\mathit{W}\right)$, the error variance–covariance matrix estimation is as follows:

$$\widehat{\mathbf{W}}=\left(\begin{array}{cccc}{\widehat{\sigma }}_{11}& {\widehat{\sigma }}_{12}& \cdots & {\widehat{\sigma }}_{1r}\\ {\widehat{\sigma }}_{12}& {\widehat{\sigma }}_{22}& \cdots & {\widehat{\sigma }}_{2r}\\ ⋮& ⋮& \ddots & ⋮\\ {\widehat{\sigma }}_{1r}& {\widehat{\sigma }}_{2r}& \cdots & {\widehat{\sigma }}_{rr}\end{array}\right)\otimes \mathbf{I}$$

where

$${\widehat{\sigma }}_{11}=\frac{{\left[{\mathit{y}}_1-\left(\mathit{J}\mathit{F}+\mathit{Z}\mathit{G}\right){\mathit{y}}_1\right]}^T\left[{\mathit{y}}_1-\left(\mathit{J}\mathit{F}+\mathit{Z}\mathit{G}\right){\mathit{y}}_1\right]}{n}, {\widehat{\sigma }}_{22}=\frac{{\left[{\mathit{y}}_2-\left(\mathit{J}\mathit{F}+\mathit{Z}\mathit{G}\right){\mathit{y}}_2\right]}^T\left[{\mathit{y}}_2-\left(\mathit{J}\mathit{F}+\mathit{Z}\mathit{G}\right){\mathit{y}}_2\right]}{n}, {\widehat{\sigma }}_{rr}=\frac{{\left[{\mathit{y}}_r-\left(\mathit{J}\mathit{F}+\mathit{Z}\mathit{G}\right){\mathit{y}}_r\right]}^T\left[{\mathit{y}}_r-\left(\mathit{J}\mathit{F}+\mathit{Z}\mathit{G}\right){\mathit{y}}_r\right]}{n},$$

$${\widehat{\sigma }}_{12}=\frac{{\left[{\mathit{y}}_1-\left(\mathit{J}\mathit{F}+\mathit{Z}\mathit{G}\right){\mathit{y}}_1\right]}^T\left[{\mathit{y}}_2-\left(\mathit{J}\mathit{F}+\mathit{Z}\mathit{G}\right){\mathit{y}}_2\right]}{n}, {\widehat{\sigma }}_{1r}=\frac{{\left[{\mathit{y}}_1-\left(\mathit{J}\mathit{F}+\mathit{Z}\mathit{G}\right){\mathit{y}}_1\right]}^T\left[{\mathit{y}}_r-\left(\mathit{J}\mathit{F}+\mathit{Z}\mathit{G}\right){\mathit{y}}_r\right]}{n}, {\widehat{\sigma }}_{2r}=\frac{{\left[{\mathit{y}}_2-\left(\mathit{J}\mathit{F}+\mathit{Z}\mathit{G}\right){\mathit{y}}_2\right]}^T\left[{\mathit{y}}_r-\left(\mathit{J}\mathit{F}+\mathit{Z}\mathit{G}\right){\mathit{y}}_r\right]}{n},$$

$$\mathbf{F}=\left(\left({\left[\mathbf{I}-{\left({\mathbf{J}}^T\mathbf{J}\right)}^{-1}{\mathbf{J}}^T\mathbf{Z}{\left({\mathbf{Z}}^T\mathbf{Z}\right)}^{-1}{\mathbf{Z}}^T\mathbf{J}\right]}^{-1}{\left({\mathbf{J}}^T\mathbf{J}\right)}^{-1}{\mathbf{J}}^T\right)-\left({\left({\mathbf{J}}^T\mathbf{J}\right)}^{-1}{\mathbf{J}}^T\mathbf{Z}{\left({\mathbf{Z}}^T\mathbf{Z}\right)}^{-1}{\mathbf{Z}}^T\right)\right),$$

$$\mathbf{G}=\left(\left({\left[\mathbf{I}-{\left({\mathbf{Z}}^T\mathbf{Z}\right)}^{-1}{\mathbf{Z}}^T\mathbf{J}{\left({\mathbf{J}}^T\mathbf{J}\right)}^{-1}{\mathbf{J}}^T\mathbf{Z}\right]}^{-1}{\left({\mathbf{Z}}^T\mathbf{Z}\right)}^{-1}{\mathbf{Z}}^T\right)-\left({\left({\mathbf{Z}}^T\mathbf{Z}\right)}^{-1}{\mathbf{Z}}^T\mathbf{J}{\left({\mathbf{J}}^T\mathbf{J}\right)}^{-1}{\mathbf{J}}^T\right)\right).$$

The proof of Theorem 3 is given in Appendix F.

3.3. Smoothing Parameter Selection

Another critical step in nonparametric regression modeling is selecting the optimal knot, oscillation parameter, and smoothing parameter. A large smoothing parameter produces a smoother estimator function but constrains the capability mapping of data. In contrast, if the smoothing parameter is too small, the function estimator is rougher [33]. We concluded that a suitable method was required to determine the optimal smoothing parameters. The smoothing parameter selection in this study was based on the GCV method, as conducted by [33]; the detailed procedure and optimal properties of this method have been carried out by [34]. The modified GCV method for combined truncated spline and Fourier series estimators in the multiresponse nonparametric regression model is as follows:

$$GCV\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)=\frac{MSE\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}{{\left[N^{-1}trace\left(\mathbf{I}-\mathbf{C}\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)\right)\right]}^2}$$

(14)

where

$$MSE\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)=N^{-1}{\left(\mathbf{y}-\widehat{\mathsf{\mu }}\right)}^T\left(\mathbf{y}-\widehat{\mathsf{\mu }}\right)=N^{-1}{\left(\mathbf{y}-\mathbf{C}\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)\mathbf{y}\right)}^T\left(\mathbf{y}-\mathbf{C}\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)\mathbf{y}\right)$$

$$=N^{-1}{\parallel \mathbf{I}-\mathbf{C}\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)\mathbf{y}\parallel }^2$$

(15)

By substituting Equation (15) into Equation (14), the optimum smoothing parameter is obtained by taking the minimum of modified GCV for the proposed model, as presented below

$$GCV\left({\mathit{K}}_{opt},{\mathit{\lambda }}_{opt},{\mathit{T}}_{opt}\right)=\underset{\mathit{K},\mathit{\lambda },\mathit{T}}{\mathrm{Min}}\left\{\frac{N^{-1}{\parallel \mathbf{I}-\mathbf{C}\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)\mathbf{y}\parallel }^2}{{\left[N^{-1}trace\left(\mathbf{I}-\mathbf{C}\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)\right)\right]}^2}\right\}$$

where $\mathbf{C}\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)$ is the regression curve estimation of the proposed model, as in Equation (13).

3.4. Simulation Study

Simulations with 100 replications were carried out to verify and validate the theoretical result of the proposed model. Using three data sample sizes ($n$ = 20, 50, and 100) and three different error variances (${\sigma }^2$ = 0.1, 0.5, and 1), random data were generated for multiresponse nonparametric regression. As shown in Figure 1, each predictor variable describes different functions, i.e., a polynomial function represents the truncated spline ($x_i$), while a trigonometry function with a trend represents the Fourier series function ($z_i$). As a note, Figure 1 only presents a partial scatterplot of the numerical example function for $n$ = 100 and ${\sigma }^2$ = 0.1. The simulation study followed the model in Equation (4), so the polynomial and trigonometric functions used in this numerical example were obtained from the following function:

$$y_{1i}=15\left(x_i-1\right){\left(1-x_i\right)}^2-10.5z_i-4\cos \left(2\pi z_i\right)+{\epsilon }_{1i}$$

$$y_{2i}=14\left(x_i-1\right){\left(1-x_i\right)}^2-7.5z_i-4\cos \left(2\pi z_i\right)+{\epsilon }_{2i}$$

$$y_{3i}=13\left(x_i-1\right){\left(1-x_i\right)}^2-4.5z_i-4\cos \left(2\pi z_i\right)+{\epsilon }_{3i}$$

In particular, the predictor variables $x_i$ and $z_i$ are generated from a Uniform (0,1) distribution, while random errors ${\epsilon }_{hi}$ are generated from a multivariate normal distribution. In addition, ${\epsilon }_{1i}$, ${\epsilon }_{2i}$, and ${\epsilon }_{3i}$ are correlated with $corr\left({\epsilon }_{1i},{\epsilon }_{2i}\right)=corr\left({\epsilon }_{1i},{\epsilon }_{3i}\right)=corr\left({\epsilon }_{2i},{\epsilon }_{3i}\right)=\rho =0.9$. In order for ease of the computational process and based on the results of the partial scatterplot identification from Figure 1, the simulation study was carried out with the combination of three knots (K = 1, 2, 3) and three oscillation parameters (T = 1, 2, 3). To determine the optimal smoothing parameter, the minimum GCV criteria were used.

Table 1. Summary of statistical results for n = 100 with error variances, the number of knots, and oscillation parameters.

Variance	Number of Oscillations	Number of Knots	Generalized Cross-Validation (GCV)	R2	Mean Square Error (MSE)
0.1	1	1	7.050	91.866	6.817
		2	6.996	92.100	6.719
		3	6.525	92.991	5.853
	2	1	6.413	92.964	5.875
		2	6.391	93.072	5.784
		3	6.386	93.167	5.704
	3	1	7.017	91.876	6.785
		2	6.435	93.066	5.789
		3	6.098 *	93.202	5.675
0.5	1	1	6.915	92.454	6.686
		2	6.759	92.716	6.492
		3	6.634	93.023	6.328
	2	1	8.183	91.121	7.913
		2	6.720	92.757	6.454
		3	6.383	93.671	5.640
	3	1	6.145	93.645	5.663
		2	6.260	93.660	5.650
		3	6.381	93.673	5.638
1.0	1	1	6.408	94.856	5.905
		2	6.479	94.906	5.847
		3	6.582	94.933	5.816
	2	1	6.358	94.895	5.860
		2	6.417	94.955	5.791
		3	6.490	95.004	5.734
	3	1	6.358	94.895	5.860
		2	6.417	94.955	5.791
		3	6.490	95.004	5.734

* The smallest value of GCV.

provides a comparison of the statistical results for $n=$ 100 with the error variances, the number of knots, and oscillation parameters. A complete summary of the statistical results for $n=$ 20 and 50 is provided in Appendix G. Table 1 shows that the smallest GCV (6.098) occurs for variance ${\sigma }^2=$ 0.1 with the combination of the three knots–three oscillation model. This model yields adequate results, with coefficient of determination (R²) = 93,202 and mean-square-error (MSE) = 5.675. Surprisingly, the same result is seen for the other sample sizes ($n=$ 20 and 50), where the smallest GCV is obtained for variance ${\sigma }^2=$ 0.1. In addition, a comparison between sample sizes gives the result in which the smallest GCV is gained from the simulation with the largest sample number ($n=$ 100). Thus, a smaller variance with a large sample size will produce a minimum GCV, which implies that the regression curve will be better estimated as well. These findings are consistent with [23], who developed mixed estimators in nonparametric regression for cross-sectional data.

3.5. Data Application

This section attempts to provide the application of the proposed model to a real dataset. The proposed model was applied to four indicators of HDI data in 38 regencies/cities across East Java Province, Indonesia, in 2018. To measure human development in some regions, the UNDP developed the HDI in 1990. In the 1990 Human Development Report, three basic dimensions of human development were defined in the HDI: income (economy), health, and education [35]. From these dimensions, four indicators of HDI data were derived: life expectancy rate, expected years of schooling, mean years of schooling, and adjusted per capita expenditure [35]. In this study, these four indicators are the response variables, $y_1, y_2,y_3,$ and $y_4$, respectively.

The issue of the HDI has received considerable critical attention in every province/regency in Indonesia as it is one of the determining components of the general allocation fund (DAU) [36]. As one of the largest provinces in Indonesia, East Java Province contributes significantly to Indonesia’s HDI. East Java Province’s HDI in 2018 was 70.77 [37], slightly lower than the national achievement. In addition, it was the lowest among the five other provinces of Java island in the last three years. Therefore, further studies on East Java Province’s HDI are to be potentially discussed.

To determine whether the correlation matrix has an identity matrix (homogeneity of variances), we used Bartlett’s test of sphericity with statistical significance $\alpha =$ 0.05. The results showed a statistical value of ${\chi }^2$= 121.993 with p-value = 0.00, which means that the decision was to reject the null hypothesis as p-value < $\alpha$. According to this result, we can infer that there is a significant correlation among the response variables; hence, multiresponse nonparametric regression analysis can be used.

Several studies have highlighted the potential factors associated with Indonesia’s HDI, such as the percentage of people living in poverty [25,38], the unemployment rate [25,38,39], population density [25,38], and the per capita gross regional domestic product (GRDP) [25,39,40]. Based on these, the predictor variables used in this study were population density and the percentage of people living in poverty. In addition, the relationship between the response variable and each predictor variable was identified by a partial scatterplot, as shown in Figure 2. The partial scatter plot between the four indicators of the HDI and population density ($x$) showed changes at a particular subinterval that is fit for the truncated spline function. Meanwhile, the partial scatterplot between the response variable and the percentage of people living in poverty ($z$) followed a pattern that repeated at a certain interval, with a particular trend; thus, this variable was approached by the Fourier series function.

As shown in

Table 2. Summary of statistical results for the case study.

Model 1	Combined Truncated Spline and Fourier Series Estimator for Multiresponse Nonparametric Regression
	Number of Oscillations	Number of Knots	GCV	MSE
	1	1	1.43056	1.16177
		2	1.45686	1.14882
		3	1.41989	1.07056
	2	1	1.43464	1.11886
		2	1.46877	1.07960
		3	1.43883	1.07192
	3	1	2.10749	1.91785
		2	1.44040	1.13633
		3	1.39189 *	1.08377
Model 2	Truncated Spline Estimator for Multiresponse Nonparametric Regression
	Number of Knots		GCV	MSE
	1		1.478036	1.114668
	2		1.522833	1.013464
	3		1.586664	1.055945
Model 3	Fourier Series Estimator for Multiresponse Nonparametric Regression
	Number of Oscillations		GCV	MSE
	1		1.524463	1.149681
	2		1.536081	1.089298
	3		1.578724	1.05066

* The smallest value of GCV.

, some scenarios using a combined estimator (Model 1) and uncombined estimators (Model 2 and Model 3) were performed to compare the effectiveness of the proposed model. By using the modified GCV formula in Section 3.3, we selected the best model according to the minimum GCV as the criterion. In this study, a comparison of these three models revealed that the proposed model (Model 1) had the smallest GCV compared to the other uncombined estimator models for multiresponse nonparametric regression. In summary, these results indicate that the proposed model is better recommended for modeling the 2018 HDI data of East Java Province.

Another important result from Table 2 is the comparison of the number of knots and oscillation combinations from the proposed model. Similar to the simulation studies, we used a combination of three knot and oscillation parameters each. According to the minimum GCV as the criterion, the leading model was a three knots–three oscillations model with a GCV equal to 1.39189 and MSE = 1.08377. In addition, this model yields a coefficient of determination (R²) of 99.84%. Due to the satisfactory result of R², it can be said that the proposed model is able to describe the variance of the response variable through the predictor variables exceptionally well. Another interesting result was the number of knot and oscillation combinations of the best model in the simulation study, in line with those of data application.

Appendix H presents the results of parameter estimation from the best model obtained from the minimum GCV in Table 2. As such, we obtained a three knots–three oscillations model. Based on the parameter estimation results for each response variable for the 2018 HDI data of East Java Province, Indonesia, the multiresponse nonparametric regression model with a combined truncated spline and Fourier series estimator can be written as

$$\begin{array}{c}{\widehat{y}}_1=-0.932+0.869x_{1i}-3057.409{\left(x_{1i}-3.718\right)}_{+}+6107.433{\left(x_{1i}-3.933\right)}_{+}-3050.878 {\left(x_{1i}-4.148\right)}_{+}-0.677z_{1i}+\\ 1/2\left(1355.724\right)-935.001\cos z_{1i}+423.443\cos 2z_{1i}-92.348 \cos 3z_{1i}.\\ =676.929+0.869x_{1i}-3057.409{\left(x_{1i}-3.718\right)}_{+}+6107.433{\left(x_{1i}-3.933\right)}_{+}-3050.878 {\left(x_{1i}-4.148\right)}_{+}-0.677z_{1i}-\\ 935.001\cos z_{1i}+423.443\cos 2z_{1i}-92.348 \cos 3z_{1i}.\end{array}$$

$$\begin{array}{c}{\widehat{y}}_2=-0.747+0.393x_{1i}-619.153{\left(x_{1i}-3.718\right)}_{+}+1235.608{\left(x_{1i}-3.933\right)}_{+}-616.769 {\left(x_{1i}-4.148\right)}_{+}-0.180z_{1i}+\\ 1/2\left(274.146\right)-189.489\cos z_{1i}+85.224\cos 2z_{1i}-18.760 \cos 3z_{1i}.\\ =136.326+0.393x_{1i}-619.153{\left(x_{1i}-3.718\right)}_{+}+1235.608{\left(x_{1i}-3.933\right)}_{+}-616.769 {\left(x_{1i}-4.148\right)}_{+}-0.180z_{1i}-\\ 189.489\cos z_{1i}+85.224\cos 2z_{1i}-18.760\cos 3z_{1i}.\end{array}$$

$$\begin{array}{c}{\widehat{y}}_3=-0.102+0.865x_{1i}-510.292{\left(x_{1i}-3.718\right)}_{+}+1015.208{\left(x_{1i}-3.933\right)}_{+}-505.678 {\left(x_{1i}-4.148\right)}_{+}-0.411z_{1i}+\\ 1/2\left(224.955\right)-156.440\cos z_{1i}+69.027\cos 2z_{1i}-15.588 \cos 3z_{1i}.\\ =112.375+0.865x_{1i}-510.292{\left(x_{1i}-3.718\right)}_{+}+1015.208{\left(x_{1i}-3.933\right)}_{+}-505.678 {\left(x_{1i}-4.148\right)}_{+}-0.411z_{1i}-\\ 156.440\cos z_{1i}+69.027\cos 2z_{1i}-15.588 \cos 3z_{1i}.\end{array}$$

$$\begin{array}{c}{\widehat{y}}_4=-0.482+0.969x_{1i}-684.712{\left(x_{1i}-3.718\right)}_{+}+1363.397{\left(x_{1i}-3.933\right)}_{+}-679.163 {\left(x_{1i}-4.148\right)}_{+}-0.456z_{1i}+\\ 1/2\left(302.146\right)-209.894\cos z_{1i}+92.927\cos 2z_{1i}-20.891 \cos 3z_{1i}.\\ =150.591+0.969x_{1i}-684.712{\left(x_{1i}-3.718\right)}_{+}+1363.397{\left(x_{1i}-3.933\right)}_{+}-679.163 {\left(x_{1i}-4.148\right)}_{+}-0.456z_{1i}-\\ 209.894\cos z_{1i}+92.927\cos 2z_{1i}-20.891 \cos 3z_{1i}.\end{array}$$

Figure 3 presents a comparison between the response variable and the fitted values using the proposed model. From the graph, we can see that all fitted values resemble the pattern of the real data; although some do not, there are only a few deviations. Thus, the proposed model can produce a prediction model.

4. Discussion and Conclusions

This study presents the major findings from regression curve estimation with a combined truncated spline and Fourier series estimator for an additive model in multiresponse nonparametric regression using PWLS and WLS optimization, as shown below:

$${\widehat{\mathbf{f}}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)=\mathbf{J}{\left[{\mathbf{J}}^T\left(\mathbf{I}-2{\mathbf{H}}^T\right)WJ+{\mathbf{J}}^T{\mathbf{H}}^{T}WHJ\right]}^{-1}\left[{\mathbf{J}}^T\left({\mathbf{H}}^T-\mathbf{I}\right)\mathbf{W}\left(\mathbf{H}-\mathbf{I}\right)\right]\mathbf{y},$$

$${\widehat{\mathbf{g}}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)=\mathbf{Z}\left({\left[{\mathbf{Z}}^{T}WZ+N\mathbf{D}(\lambda )\right]}^{-1}{\mathbf{Z}}^T\mathbf{W}\left(\mathbf{I}-\mathbf{J}{\mathbf{K}}^{-1}\mathbf{L}\right)\mathbf{y}\right),$$

$${\widehat{\mathsf{\mu }}}_{\left(\mathbf{K},\mathsf{\lambda },\mathbf{T}\right)}\left(\mathbf{x},\mathbf{z}\right)=\left[\mathbf{J}{\mathbf{K}}^{-1}\mathbf{L}+\mathbf{H}\left(\mathbf{I}-\mathbf{J}{\mathbf{K}}^{-1}\mathbf{L}\right)\right]\mathbf{y}$$

Furthermore, the result of error variance–covariance matrix estimation is as follows:

$$\widehat{\mathbf{W}}=\left(\begin{array}{cccc}{\widehat{\sigma }}_{11}& {\widehat{\sigma }}_{12}& \cdots & {\widehat{\sigma }}_{1r}\\ {\widehat{\sigma }}_{12}& {\widehat{\sigma }}_{22}& \cdots & {\widehat{\sigma }}_{2r}\\ ⋮& ⋮& \ddots & ⋮\\ {\widehat{\sigma }}_{1r}& {\widehat{\sigma }}_{2r}& \cdots & {\widehat{\sigma }}_{rr}\end{array}\right)\otimes \mathbf{I}$$

From the simulation study, the minimum GCV obtained a satisfactory outcome through a large sample with small variance. In addition, data application to a real dataset of the 2018 HDI in East Java revealed that the proposed model had a better result than the uncombined model. An adequate coefficient of determination (R²) from the best model indicates that the proposed model can explain the data variation remarkably well. Interestingly, the number of knots and oscillations of the best model in this simulation study and data application is consistent; that is a combination of three knots–three oscillations. In summary, these findings have significant implications for the understanding of regression curve estimation when using combined estimators for multiresponse nonparametric regression. Although the focus of the research was on combined truncated spline and Fourier series estimators, the procedure in this study can be applied to other combinations of estimators for multiresponse nonparametric regression.

The scope of this study was limited in terms of curve estimation and estimation of the variance–covariance matrix; thus, the major limitation was the absence of hypothesis testing and confidence intervals testing. Considering its necessity for good statistical practice, there is a commitment to perform such a goodness-of-fit test (model checking). Therefore, further research will be conducted to achieve hypothesis testing and confidence intervals testing. Another possible area is that a further simulation study could be performed using another function, such as an exponential or logarithmic function, to validate the capability of the proposed model. An additional variation of $\rho$ in a simulation study could be used to gain more insights into the performance evaluation of the proposed model.