Ridge Fuzzy Regression Modelling for Solving Multicollinearity

Abstract

This paper proposes an $\alpha$-level estimation algorithm for ridge fuzzy regression modeling, addressing the multicollinearity phenomenon in the fuzzy linear regression setting. By incorporating $\alpha$-levels in the estimation procedure, we are able to construct a fuzzy ridge estimator which does not depend on the distance between fuzzy numbers. An optimized $\alpha$-level estimation algorithm is selected which minimizes the root mean squares for fuzzy data. Simulation experiments and an empirical study comparing the proposed ridge fuzzy regression with fuzzy linear regression is presented. Results show that the proposed model can control the effect of multicollinearity from moderate to extreme levels of correlation between covariates, across a wide spectrum of spreads for the fuzzy response.

1. Introduction

Often times in practical applications, the available data may not always be precise. The researcher may be only accessible to minimum and maximum values of data. Sometimes the data may not even be given in numbers. For instance, consider linguistics data such as “young”, “tall”, or “high”, and medicine data such as “healthy” and “not healthy”. In such cases where the given data are imprecise and vague, classical representation of numbers may be insufficient. The fuzzy set theory introduced by Zadeh [1,2] can handle such uncertainty in data. In the view of fuzzy set theory, uncertain data are what is called fuzzy. Fuzzy data are prevalent in various fields such as linguistics, survey, medicine and so forth [3,4,5,6,7]. The development of fuzzy set theory has led to statistical methods for analyzing fuzzy data. When the measure of indeterminacy is needed, the neutrosophic set introduced by Smarandache [8] considered the measure of indeterminacy in addition to the fuzzy set. The neutrosophic statistics based on the the neutrosophic set can be applied for the analysis of the data when data are selected from the population with uncertain, fuzzy, and imprecise observations [9].

In 1982, Tanaka et al. [10] proposed the fuzzy linear regression model which generalizes the usual linear regression model to fuzzy data. Fuzzy regression models have been since then widely used to analyze fuzzy data [11,12,13,14,15,16].

In classical linear regression models, the multicollinearity phenomenon is frequently observed in which two or more explanatory variables are highly linearly related. Common examples of collinear covariates are: a person’s height and weight, a person’s level of education, gender, race, and starting salary. When multicollinearity occurs, the least squares estimator may not be obtainable or be subject to very high variance. Once the researcher identifies the collinear variables, there are several techniques the researcher can use to handle multicollinearity. Among these techniques, the two most widely used approaches are lasso regression and ridge regression. Lasso regression developed by Tibshirani [17] and ridge regression developed by Hoerl and Kennard [18] improve model performance by adding a penalty term to the classical linear regression model. Both methods aim to shrink the model parameters towards zero. This induces a sparse model which increases the model bias, but decreases the model variance even more, thus improving overall performance. Ridge regression decreases the parameters of low contributing variables towards zero, but not exactly to zero, and stabilizing the parameter variance of the least squares estimator in the presence of multicollinearity. Lasso regression sets the model parameters exactly to zero, removing low contributing variables as well as improving model fitting. However, sometimes the researcher may want to include all the available covariates in the model without having to reduce the dimension of the data. In such cases, ridge regression is preferred to lasso regression.

Similar to classical linear regression models, multicollinearity occurs frequently in fuzzy linear regression models as well, causing problems in the estimation procedure. Often times the number of covariates is not particularly large for fuzzy data. Consequently, dropping any explanatory variables may not be an option. As in the classical statistical setting, we prefer to use ridge regression to lasso regression to handle multicollinearity in such datasets. In this paper, we incorporate fuzzy set theory with ridge regression developed by Hoerl and Kennard [18] to handle multicollinearity observed in fuzzy data. Only some works have suggested ridge estimation methods for fuzzy linear regression, and are limited to obtaining fuzzy ridge estimators which are dependent on the distance between fuzzy numbers [19,20,21]. We instead propose an $\alpha$-level estimation algorithm for ridge fuzzy regression modelling. The proposed algorithm is an extension of the ridge regression model introduced in Choi et al. [22]. By applying $\alpha$-levels to the estimation procedure, we are able to construct a fuzzy ridge estimator which does not depend on the distance between fuzzy numbers. Simulation experiments show the proposed ridge fuzzy regression model can solve moderate to severe degrees of multicollinearity across a wide range of spreads for the fuzzy response. An empirical study using Tanaka’s house prices data [10] with multicollinearity, the most widely applied data set in the fuzzy linear regression literature, is conducted to demonstrate the practical implementations.

The rest of this paper is organized as follows. Section 2 introduces key definitions and results from fuzzy set theory. Section 3 describes the classical ridge regression, followed by a step-by-step procedure for the proposed $\alpha$-level estimation algorithm of ridge fuzzy regression modeling. Section 4 and Section 5 illustrates the performance of the model with simulation studies and a numerical example, respectively. Section 6 concludes the paper.

2. Fuzzy Numbers

A fuzzy set is a set of ordered pairs $A=\{(x,{\mu }_A(x)):x\in X\}$ where ${\mu }_A(x):X\to [0,1]$ is a membership function which represents the degree of membership of x in a set A. Please note that when A is a crisp (classical) set, its membership function can take only the values one or zero depending on whether or not x does or does not belong to A. In this case, ${\mu }_A(x)$ reduces to the indicator function $I_A(x)$ of a set A. For any $\alpha$ in $[0,1]$, the $\alpha$-level set of a fuzzy set A is a crisp set $A(\alpha )=\{x\in X:{\mu }_A(x)\ge \alpha \}$ which contains all the elements in X with membership value in A greater than or equal to $\alpha$. The $\alpha$-level set of a fuzzy set A can also be represented by $A(\alpha )=[l_A(\alpha ),r_A(\alpha )]$. Here $l_A(\alpha )$ and $r_A(\alpha )$ are the left and right end-points of the $\alpha$-level set, respectively. Zadeh’s [23] resolution identity theorem states that a fuzzy set can represented by its membership function or by its $\alpha$-level set. Let A be a fuzzy number with membership function ${\mu }_A(x)$ and $\alpha$-cut $A(\alpha )$. Then we have ${\mu }_A(x)=Sup\{\alpha ·I_{A(\alpha )}(x):\alpha \in [0,1]\}$.

A fuzzy number is a normal and convex subset of the real line R with bounded support. The support of a fuzzy set A is defined by $supp(A)=\{x\in R|{\mu }_A(x)>0\}$. The following parametric class of fuzzy numbers, the so-called $LR$-fuzzy numbers denoted by $A={(a_m,s_l,s_r)}_{LR}$, is often used as a special case:

$$\begin{array}{c}\hfill {\mu }_A(x)=\left\{\begin{array}{c}L\left({\displaystyle \frac{a_m-x}{s_l}}\right)\mathrm{if}x\le a_m\hfill \\ R\left({\displaystyle \frac{x-a_m}{s_r}}\right)\mathrm{if}x>a_m\hfill \end{array}\right.\mathrm{for}x\in R\end{array}$$

(1)

where $L,R:R\to [0,1]$ are fixed, left-continuous, and non-increasing functions with $R(0)=L(0)=1$, and $R(1)=L(1)=0$. L and R are called left and right shape functions of A, respectively. $a_m$ is the mean value of A, and $s_l,s_r>0$ are each the left and right spreads of A. The spreads $s_l$ and $s_r$ represent the fuzziness of the fuzzy number and can be symmetric or asymmetric. If $s_l=s_r=0$, the $LR$-fuzzy number becomes a precise real number with no fuzziness. Thus, a precise real number can be considered to be a special case of a fuzzy number. For a precise observation $a\in R$, its corresponding membership function is ${\mu }_a(x)=1$.

In the fuzzy set theory, triangular and trapezoidal fuzzy numbers are special cases of $LR$-fuzzy numbers and are used extensively [24]. The membership function of a triangular fuzzy number $A={(a_l,a_m,a_r)}_T$ is given by

$$\begin{array}{c}\hfill {\mu }_A(x)=\left\{\begin{array}{c}{\displaystyle \frac{x-a_l}{a_m-a_l}}\mathrm{if}x\le a_m\hfill \\ {\displaystyle \frac{a_r-x}{a_r-a_m}}\mathrm{if}x>a_m\hfill \end{array}\right.\mathrm{for}x\in R\end{array}$$

(2)

where $a_l,a_m$, and $a_r$ are the left end-point, mid-point, and right end-point, respectively.

3. Ridge Fuzzy Regression

In this section, we propose the $\alpha$-level estimation algorithm for the proposed ridge fuzzy regression model. This algorithm modifies the method based on Choi et al. [22] to estimate the fuzzy parameters. The term $\alpha$-level estimation indicates that our algorithm uses $\alpha$-levels to describe fuzzy data. By using $\alpha$-level, we are able to develop a ridge fuzzy estimator which is not restricted to the distance between fuzzy numbers. We first briefly examine the original formulation of ridge regression model for crisp data.

3.1. Ridge Regression

Given a data set ${\{y_i,x_{i1},x_{i2},\cdots ,x_{ip}\}}_{i=1}^N$, a multiple linear regression model assumes that the relationship between a dependent variable $y_i$, $i=1,\cdots ,N$ and a set of explanatory variables $x_{i1},x_{i2},\cdots ,x_{ip}$, $i=1,\cdots ,N$ is linear. The model takes the form

$$\begin{array}{c}\hfill y_i={\beta }_0+{\beta }_1{x}_{i1}+\cdots {\beta }_p{x}_{ip}+{ϵ}_i={\mathit{X}}_i^t\mathit{\beta }i=1,\cdots ,N\end{array}$$

(3)

or written alternatively in matrix notation as $\mathit{Y}=\mathit{X}\mathit{\beta }+\mathit{ϵ}$. A vector $\mathit{Y}={(y_1,\cdots ,y_N)}^t$ is a vector of observations on the dependent variable, $\mathit{X}={({\mathit{X}}_1^t,\cdots ,{\mathit{X}}_N^t)}^t$ is a matrix of explanatory variables, $\mathit{\beta }={({\beta }_0,{\beta }_1,\cdots ,{\beta }_p)}^t$ is a vector of regression coefficients to be estimated, and $\mathit{ϵ}={({ϵ}_1,\cdots ,{ϵ}_N)}^t$ is a vector of error terms. The standard estimator for $\beta$ is the least squares estimator defined by

$$\begin{array}{c}\hfill \widehat{\mathit{\beta }}={({\mathit{X}}^t\mathit{X})}^{-1}{\mathit{X}}^t\mathit{Y}.\end{array}$$

(4)

In the presence of multicollinearity, i.e., in state of extreme correlations among the explanatory variables $\widehat{\mathit{\beta }}$ is poorly determined and susceptible to high variance. Thus, we may deliberately bias the regression coefficient estimates so as to control their variance. In this manner, the ridge regression estimator was introduced by Hoerl and Kennard [18] as a penalized least squares estimator. It is achieved by minimizing the residual sum of squares (RSS) subject to a constraint on the size of the estimated coefficient vector [25]:

$$\begin{array}{c}\hfill RSS(\lambda )={(\mathit{Y}-\mathit{X}\mathit{\beta })}^t(\mathit{Y}-\mathit{X}\mathit{\beta })+\lambda {\mathit{\beta }}^{\mathit{t}}\mathit{\beta }.\end{array}$$

(5)

Here $\lambda \ge 0$ is a shrinkage parameter which controls the size of the coefficients. The larger the value of $\lambda$, the greater the amount of shrinkage, and we have coefficients close to zero. The smaller the value of $\lambda$ is close to 0, we obtain the least squares solutions. Please note that by convention the input matrix $\mathit{X}$ is assumed to be standardized and $\mathit{Y}$ is assumed to be centered before solving RSS($\lambda$). The ridge regression solution is

$$\begin{array}{c}\hfill {\widehat{\mathit{\beta }}}^{\mathit{ridge}}={({\mathit{X}}^t\mathit{X}+\lambda \mathit{I})}^{-1}{\mathit{X}}^t\mathit{Y}\end{array}$$

where $\mathit{I}$ is the $p\times p$ identity matrix. The shrinkage parameter $\lambda$ is usually selected via K-fold cross validation. Cross validation is a simple and powerful tool often used to calculate the shrinkage parameter and the prediction error in ridge regression. The entire dataset is divided into K parts, and trains the model on all but the kth part. The model is validated on the $k^{th}$ part, iterating for all $k=1,\cdots ,K$. The choice of K is $K=5$ or $K=10$ in general.

3.2. Ridge Fuzzy Regression Algorithm

Let us consider a set of observations

$$\begin{array}{c}\hfill {\{y_i,x_{i1},x_{i2},\cdots ,x_{ip}\}}_{i=1}^N={\{{(y_{il},y_{im},y_{ir})}_T,{(x_{1l},x_{1m},x_{1r})}_T,\cdots ,{(x_{pl},x_{pm},x_{pr})}_T\}}_{i=1}^N\end{array}$$

(6)

where the dependent variable $y_i$, $i=1,\cdots ,N$ and the explanatory variables $x_{i1},x_{i2},\cdots ,x_{ip}$, $i=1,\cdots ,N$ are triangular fuzzy numbers. We assume a linear relationship between the dependent and explanatory variables:

$$\begin{array}{c}\hfill y_i=A_0\oplus A_1\odot x_{i1}\oplus \cdots \oplus A_p\odot x_{ip}\oplus {ϵ}_i,i=1,\cdots ,N\end{array}$$

(7)

where ${\{A_j\}}_{j=0}^p={\{{(A_{jl},A_{jm},A_{jr})}_T\}}_{j=0}^p$ are the fuzzy regression parameters and ${\{{ϵ}_i\}}_{i=1}^N={\{{({ϵ}_{il},{ϵ}_{im},{ϵ}_{ir})}_T\}}_{i=1}^N$ are the fuzzy error terms. ⊕ and ⊙ represent addition and multiplication between two fuzzy numbers, respectively. Often the N equations are stacked together and written in matrix notation as

$$\begin{array}{c}\hfill \mathit{Y}=\mathit{X}\odot \mathit{A}\oplus \mathit{ϵ}.\end{array}$$

(8)

For more details on arithmetic operations between fuzzy numbers, see [10,26]. Please note that the above fuzzy variables can be symmetric or asymmetric, and be extended to various forms such as normal, parabolic, or square root fuzzy data. Since crisp sets are a special case of fuzzy sets, fuzzy inputs and fuzzy outputs, or fuzzy inputs and crisp outputs combinations are also possible. For illustration purposes, in this section, we present our ridge fuzzy regression model using triangular membership functions.

We divide the given data into training and test sets. The model is computed from the training set ${\{y_i,x_{i1},x_{i2},\cdots ,x_{ip}\}}_{i=1}^n$, and later its performance is evaluated on the test set ${\{y_i,x_{i1},x_{i2},\cdots ,x_{ip}\}}_{i=1}^m$. Note again that N is the total number of observations, n is the number of observations for the training set, and m is the number of observations for the test set, such that $n+m=N$. We fit our ridge fuzzy regression model on the training set by the following estimation algorithm:

Step 1: Create $\alpha$-level sets of the triangular fuzzy input and output as illustrated in Figure 1. For any $\alpha$-level in $[0,1]$,

$$\begin{array}{cc}\hfill y_i(\alpha )& =[l_{y_i}(\alpha ),r_{y_i}(\alpha )]\hfill \\ \hfill & =[(\alpha -1)s_{il}^y+y_{im},-(\alpha -1)s_{ir}^y+y_{im}]\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill x_{ij}(\alpha )& =[l_{x_{ij}}(\alpha ),r_{x_{ij}}(\alpha )]\hfill \\ \hfill & =[(\alpha -1)s_{ijl}^x+x_{ijm},-(\alpha -1)s_{ijr}^x+x_{ijm}]i=1,\cdots ,n,j=1,\cdots ,p\hfill \end{array}$$

(10)

where $s_{il}^y,s_{ir}^y,s_{ijl}^x,s_{ijr}^x\ge 0$ are the left and right spreads of the dependent and explanatory variables, respectively. The $\alpha$-levels are denoted by the sequence ${({\alpha }_k)}_{k=0}^K$ for some K with ${\alpha }_k\in [0,1]$.

Step 2: Perform ridge regression of $\mathit{Y}({\alpha }_k)$ on $\mathit{X}({\alpha }_k)$ for each $k=0,\cdots ,K$. Find the intermediate estimators $\overline{l_{\mathit{A}}}({\alpha }_k)$ and $\overline{r_{\mathit{A}}}({\alpha }_k)$ of $l_{\mathit{A}}({\alpha }_k)$ and $r_{\mathit{A}}({\alpha }_k)$ by minimizing the following respective ridge loss functions (see Figure 2).

$$\begin{array}{cc}\hfill & {(l_{\mathit{Y}}({\alpha }_k)-l_{\mathit{X}}({\alpha }_k)l_{\mathit{A}}({\alpha }_k))}^t(l_{\mathit{Y}}({\alpha }_k)-l_{\mathit{X}}({\alpha }_k)l_{\mathit{A}}({\alpha }_k))+\lambda l_{\mathit{A}}{({\alpha }_k)}^t{l}_{\mathit{A}}({\alpha }_k)\hfill \\ \hfill & {(r_{\mathit{Y}}({\alpha }_k)-r_{\mathit{X}}({\alpha }_k)r_{\mathit{A}}({\alpha }_k))}^t(r_{\mathit{Y}}({\alpha }_k)-r_{\mathit{X}}({\alpha }_k)r_{\mathit{A}}({\alpha }_k))+\lambda r_{\mathit{A}}{({\alpha }_k)}^t{r}_{\mathit{A}}({\alpha }_k)\hfill \end{array}$$

(11)

We assume the endpoints of the $\alpha$-level set of $\mathit{Y}$ has been centered and the endpoints of $\alpha$-level set of $\mathit{X}$ has been standardized as is by convention in classical ridge regression [25].

Step 3: Obtain the estimators $\widetilde{l_{\mathit{A}}}({\alpha }_k)$ and $\widetilde{r_{\mathit{A}}}({\alpha }_k)$ of $l_{\mathit{A}}({\alpha }_k)$ and $r_{\mathit{A}}({\alpha }_k)$ by modifying the intermediate estimators $\overline{l_{\mathit{A}}}({\alpha }_k)$ and $\overline{r_{\mathit{A}}}({\alpha }_k)$ so that the estimated coefficients form the membership function of a triangular fuzzy number. For this the following operations are performed (see Figure 3).

$$\begin{array}{cc}\hfill & \widetilde{l_{\mathit{A}}}({\alpha }_K)=\widetilde{r_{\mathit{A}}}({\alpha }_K)=\overline{l_{\mathit{A}}}({\alpha }_K)=\overline{r_{\mathit{A}}}({\alpha }_K)\hfill \\ \hfill & \widetilde{l_{\mathit{A}}}({\alpha }_k)=\mathit{min}\{\overline{l_{\mathit{A}}}({\alpha }_k),\widetilde{l_{\mathit{A}}}({\alpha }_{k+1})\}k=K-1,K-2,\cdots ,0\hfill \\ \hfill & \widetilde{r_{\mathit{A}}}({\alpha }_k)=\mathit{max}\{\overline{r_{\mathit{A}}}({\alpha }_k),\widetilde{r_{\mathit{A}}}({\alpha }_{k+1})\}k=K-1,K-2,\cdots ,0\hfill \end{array}$$

(12)

Step 4: Estimate the triangular fuzzy coefficient $\widehat{\mathit{A}}={(\widehat{{\mathit{A}}_l},\widehat{{\mathit{A}}_m},\widehat{{\mathit{A}}_r})}_T$ and its membership function ${\mu }_{\widehat{\mathit{A}}}$ by fitting a linear regression line through $\widetilde{l_{\mathit{A}}}({\alpha }_k)$ and $\widetilde{r_{\mathit{A}}}({\alpha }_k)$ for $k=0,\cdots ,K$, respectively. A constraint is given so that ${\mu }_{\widehat{\mathit{A}}}$ satisfy the condition of ${\mu }_{\widehat{\mathit{A}}}(\widehat{l_{\mathit{A}}}(1))={\mu }_{\widehat{\mathit{A}}}(\widehat{r_{\mathit{A}}}(1))=1$ (see Figure 4).

Step 5: Symmetric fuzzy inputs or outputs do not always guarantee that the estimated membership function ${\mu }_{\widehat{\mathit{A}}}$ will also be symmetric. To reduce the difference between the true values with the fitted values we consider the following candidates:

$$\begin{array}{c}\hfill {\mu }_{\widehat{\mathit{A}}}(x)=\left\{\begin{array}{c}{\displaystyle \frac{x-\widehat{l_{\mathit{A}}}(0)}{\widehat{l_{\mathit{A}}}(1)-\widehat{l_{\mathit{A}}}(0)}}\mathrm{if}x\le \widehat{l_{\mathit{A}}}(1)=\widehat{r_{\mathit{A}}}(1)\hfill \\ {\displaystyle \frac{\widehat{r_{\mathit{A}}}(0)-x}{\widehat{r_{\mathit{A}}}(0)-\widehat{r_{\mathit{A}}}(1)}}\mathrm{if}x>\widehat{l_{\mathit{A}}}(1)=\widehat{r_{\mathit{A}}}(1)\hfill \end{array}\right.\mathrm{for}x\in R\end{array}$$

(13)

or

$$\begin{array}{c}\hfill {\mu }_{\widehat{\mathit{A}}}(x)=\left\{\begin{array}{c}{\displaystyle \frac{x-\widehat{l_{\mathit{A}}}(0)}{\widehat{l_{\mathit{A}}}(1)-\widehat{l_{\mathit{A}}}(0)}}\mathrm{if}x\le \widehat{l_{\mathit{A}}}(1)\hfill \\ {\displaystyle \frac{\widehat{l_{\mathit{A}}}(0)+x}{\widehat{l_{\mathit{A}}}(1)-\widehat{l_{\mathit{A}}}(0)}}\mathrm{if}x>\widehat{l_{\mathit{A}}}(1)\hfill \end{array}\right.\mathrm{for}x\in R\end{array}$$

(14)

where $\widehat{r_{\mathit{A}}}(0)$ is chosen as $\widehat{r_{\mathit{A}}}(0)=2\times \widehat{l_{\mathit{A}}}(1)-\widehat{l_{\mathit{A}}}(0)$, or

$$\begin{array}{c}\hfill {\mu }_{\widehat{\mathit{A}}}(x)=\left\{\begin{array}{c}{\displaystyle \frac{\widehat{r_{\mathit{A}}}(0)+x}{\widehat{r_{\mathit{A}}}(0)-\widehat{r_{\mathit{A}}}(1)}}\mathrm{if}x\le \widehat{r_{\mathit{A}}}(1)\hfill \\ {\displaystyle \frac{\widehat{r_{\mathit{A}}}(0)-x}{\widehat{r_{\mathit{A}}}(0)-\widehat{r_{\mathit{A}}}(1)}}\mathrm{if}x>\widehat{r_{\mathit{A}}}(1)\hfill \end{array}\right.\mathrm{for}x\in R\end{array}$$

(15)

where $\widehat{l_{\mathit{A}}}(0)$ is chosen as $\widehat{l_{\mathit{A}}}(0)=2\times \widehat{r_{\mathit{A}}}(1)-\widehat{r_{\mathit{A}}}(0)$.

We present two performance criteria based on Diamond’s fuzzy distance measure [27] to evaluate the proposed fuzzy estimators. Denote the dependent variable as $y_i={(y_{il},y_{im},y_{ir})}_T$, $i=1,\cdots ,n$, and its predicted value as $\widehat{y_i}={(\widehat{y_{il}},\widehat{y_{im}},\widehat{y_{ir}})}_T={({\mathit{X}}_{il}^t\widehat{{\mathit{A}}_l},{\mathit{X}}_{im}^t\widehat{{\mathit{A}}_m},{\mathit{X}}_{ir}^t\widehat{{\mathit{A}}_r})}_T$, $i=1,\cdots ,n$. Here n is the number of observations for the training set. We defined ${\mathrm{RMSE}}_F$ (root mean square error for fuzzy numbers) and ${\mathrm{MAPE}}_F$ (mean absolute percentage error for fuzzy numbers) as below.

$$\begin{array}{cc}\hfill & {\mathrm{RMSE}}_F=\sqrt{\frac{1}{n}\sum _{i=1}^n\{{(y_{il}-\widehat{y_{il}})}^2+{(y_{im}-\widehat{y_{im}})}^2+{(y_{ir}-\widehat{y_{ir}})}^2\}}\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill & {\mathrm{MAPE}}_F=\frac{100\%}{n}\sum _{i=1}^n(|y_{il}-\widehat{y_{il}}|+|y_{im}-\widehat{y_{im}}|+|y_{ir}-\widehat{y_{ir}}|)\hfill \end{array}$$

(17)

Compute the ${\mathrm{RMSE}}_F$ for each of the membership functions, then select the one which minimizes the criterion.

Step 6: Repeat Steps 1–5 for selected $\alpha$-level sequences ${({\alpha }_k)}_{k=0}^K$ with ${\alpha }_k$ equally spaced between 0 and 1. Choose the optimal set of $\alpha$-levels which minimizes ${\mathrm{RMSE}}_F$. Finally, compute the fuzzy ridge coefficient estimate $\widehat{\mathit{A}}$ based on that selected sequence.

4. Simulation Study

A simulation study was conducted to illustrate the performance of the proposed ridge fuzzy regression model in the presence of multicollinearity. Simulation results are compared with the fuzzy linear regression model with varying degrees of correlation. The fuzzy least squares estimator is obtained by setting the tuning parameter $\lambda$ as zero in Step 2 of Section 3.2.

We generated $N=100$ observations for each of the $p=4$ crisp explanatory variables. The number of data dimensions is in line with commonly found fuzzy data. Following Gibbons [28], the explanatory variables $x_{ij}$ are generated by

$$\begin{array}{c}\hfill x_{ij}={(1-{\rho }^2)}^{1/2}{z}_{ij}+\rho z_{ip},i=1,\cdots ,nj=1,\cdots ,p\end{array}$$

(18)

where $\rho$ is a given constant and $z_{ij}$ are generated from independent normal distributions with mean 50 and variance 1. Here $x_{ij}$ are assumed to be non-negative so as to reflect the non-negative characteristics of real world fuzzy data. The degree of linear association between explanatory variables is controlled via $\rho$, where in this case is the correlation between any two explanatory variables is ${\rho }^2$. Three different sets of correlation are considered corresponding to $\rho =0.8,0.9,\mathrm{and}0.99$. Each value of $\rho$ stands for moderate, high, and very high correlation between the variables. Observations on the fuzzy dependent variable are generated by

$$\begin{array}{cc}\hfill & y_{il}=A_{0l}+A_{1l}{x}_{i1}+\cdots +A_{pl}{x}_{ip}+{ϵ}_i^1\hfill \\ \hfill & y_{im}=A_{0m}+A_{1m}{x}_{i1}+\cdots +A_{pm}{x}_{ip}+{ϵ}_i^2\hfill \\ \hfill & y_{ir}=A_{0r}+A_{1r}{x}_{i1}+\cdots +A_{pr}{x}_{ip}+{ϵ}_i^3,i=1,\cdots ,N\hfill \end{array}$$

(19)

where ${ϵ}_i^1,{ϵ}_i^2,{ϵ}_i^3$ are generated from independent normal distributions with mean 0 and variance ${\sigma }^2$. Four values of $\sigma$ are investigated in this study: 0.5, 1, 1.5, and 2. Large values of $\sigma$ correspond to bigger variation in the spreads of the fuzzy dependent variable. ${\mathit{S}}_l^y={(s_{1l}^y,\cdots ,s_{Nl}^y)}^t$ the vector of left spreads and ${\mathit{S}}_r^y={(s_{1r}^y,\cdots ,s_{Nr}^y)}^t$ the vector of right spreads are determined by

$$\begin{array}{cc}\hfill & s_{il}^y=y_{im}-y_{il}\hfill \\ \hfill & s_{ir}^y=y_{ir}-y_{im}i=1,\cdots ,N.\hfill \end{array}$$

(20)

Cases of asymmetric spreads, ${\mathit{S}}_l^y\ne {\mathit{S}}_r^y$, and symmetric spreads, ${\mathit{S}}_l^y={\mathit{S}}_r^y$ are also compared. The supposed parameters of the model are: ${\mathit{A}}_l=(0,0.1,0.15,0.2,0.25),{\mathit{A}}_m=(0,0.4,0.45,0.5,0.55),\mathrm{and}{\mathit{A}}_r=(0,0.7,0.75,0.8,0.85)$. In order to analyze the effects of factors $\rho$ and $\sigma$, we controlled for the effects of varying $\alpha$-level sequences in Step 6 of Section 3.2. For both models we fixed the $\alpha$-level sequence as (0, 0.25, 0.5, 0.75, 1).

200 replicates for each scenario are generated. The explanatory variables and the fuzzy coefficients remain fixed, while the error terms and hence the fuzzy dependent variable changes. We separated the simulated data into training and test sets. Once the ridge fuzzy regression model and the fuzzy linear regression model are fit to the training data, ${\mathrm{RMSE}}_F$ and ${\mathrm{MAPE}}_F$ are computed from the test set for $t=1,\cdots ,200$ replicates. Let ${\mathrm{RMSE}}_F^t$ and ${\mathrm{MAPE}}_F^t$ be the performance measures when the fuzzy model is applied to the replicate t. The following quantities are then computed for each fuzzy estimator:

$$\begin{array}{cc}\hfill & \mathrm{Ave}.{\mathrm{RMSE}}_F=\frac{1}{200}\sum _{t=1}^{200}{\mathrm{RMSE}}_F^t\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill & \mathrm{Ave}.{\mathrm{MAPE}}_F=\frac{1}{200}\sum _{t=1}^{200}{\mathrm{MAPE}}_F^t.\hfill \end{array}$$

(22)

In addition, we fit the ridge regression model and the linear regression model on the mid-point of our training data ${\{y_{im},x_{i1m},x_{i2m},\cdots ,x_{ipm}\}}_{i=1}^n$ for comparison with fuzzy methods. The test Ave. RMSE and Ave. MAPE values of 200 replicates are recorded for both models. The output from numerical experiments is suggested below in

Table 1. The performance measures when $\rho =0.8$ and the dependent variable is an asymmetric triangular fuzzy number.

$\mathit{\sigma }$:		0.5	1.0	1.5	2.0
Ave. $RMS{E}_F$	Ridge Fuzzy Reg.	1.161	4.459	10.955	19.419
	Fuzzy Reg.	1.706	8.979	23.130	40.243
Ave. $RMSE$	Ridge Reg.	0.478	0.957	1.437	1.917
	Linear Reg.	0.481	0.962	1.444	1.925
Ave. $MAP{E}_F$	Ridge Fuzzy Reg.	1.84%	7.64%	19.26%	34.43%
	Fuzzy Reg.	2.83%	15.81%	41.14%	71.71%
Ave. $MAPE$	Ridge Reg.	0.31%	0.61%	0.91%	1.21%
	Linear Reg.	0.31%	0.61%	0.92%	1.23%

,

Table 2. The performance measures when $\rho =0.8$ and the dependent variable is a symmetric triangular fuzzy number.

$\mathit{\sigma }$:		0.5	1.0	1.5	2.0
Ave. $RMS{E}_F$	Ridge Fuzzy Reg.	3.604	8.150	19.944	22.572
	Fuzzy Reg.	6.273	19.228	39.020	61.870
Ave. $RMSE$	Ridge Reg.	0.478	0.957	1.437	1.917
	Linear Reg.	0.481	0.962	1.444	1.925
Ave. $MAP{E}_F$	Ridge Fuzzy Reg.	5.89%	13.64%	25.52%	38.88%
	Fuzzy Reg.	10.72%	33.72%	69.03%	109.8%
Ave. $MAPE$	Ridge Reg.	0.31%	0.61%	0.91%	1.21%
	Linear Reg.	0.31%	0.61%	0.92%	1.23%

,

Table 3. The performance measures when $\rho =0.9$ and the dependent variable is an asymmetric triangular fuzzy number.

$\mathit{\sigma }$:		0.5	1.0	1.5	2.0
Ave. $RMS{E}_F$	Ridge Fuzzy Reg.	1.210	4.612	11.625	20.995
	Fuzzy Reg.	2.879	17.321	38.918	63.449
Ave. $RMSE$	Ridge Reg.	0.478	0.959	1.440	1.921
	Linear Reg.	0.481	0.962	1.444	1.925
Ave. $MAP{E}_F$	Ridge Fuzzy Reg.	2.02%	8.29%	21.47%	39.05%
	Fuzzy Reg.	5.19%	32.26%	72.62%	118.4%
Ave. $MAPE$	Ridge Reg.	0.32%	0.64%	0.95%	1.27%
	Linear Reg.	0.32%	0.64%	0.96%	1.28%

,

Table 4. The performance measures when $\rho =0.9$ and the dependent variable is a symmetric triangular fuzzy number.

$\mathit{\sigma }$:		0.5	1.0	1.5	2.0
Ave. $RMS{E}_F$	Ridge Fuzzy Reg.	3.306	8.356	16.036	25.076
	Fuzzy Reg.	8.378	30.691	60.041	91.664
Ave. $RMSE$	Ridge Reg.	0.478	0.959	1.440	1.921
	Linear Reg.	0.481	0.962	1.444	1.925
Ave. $MAP{E}_F$	Ridge Fuzzy Reg.	5.60%	14.71%	28.85%	45.47%
	Fuzzy Reg.	15.23%	56.89%	111.7%	170.6%
Ave. $MAPE$	Ridge Reg.	0.32%	0.64%	0.95%	1.27%
	Linear Reg.	0.32%	0.64%	0.96%	1.28%

,

Table 5. The performance measures when $\rho =0.99$ and the dependent variable is an asymmetric triangular fuzzy number.

$\mathit{\sigma }$:		0.5	1.0	1.5	2.0
Ave. $RMS{E}_F$	Ridge Fuzzy Reg.	0.952	2.378	6.822	14.514
	Fuzzy Reg.	34.201	101.27	171.70	243.33
Ave. $RMSE$	Ridge Reg.	0.440	0.874	1.310	1.746
	Linear Reg.	0.443	0.885	1.328	1.771
Ave. $MAP{E}_F$	Ridge Fuzzy Reg.	1.80%	4.70%	14.53%	31.64%
	Fuzzy Reg.	75.07%	222.3%	377.1%	535.0%
Ave.$MAPE$	Ridge Reg.	0.37%	0.74%	1.11%	1.49%
	Linear Reg.	0.38%	0.76%	1.14%	1.52%

and

Table 6. The performance measures when $\rho =0.99$ and the dependent variable is a symmetric triangular fuzzy number.

$\mathit{\sigma }$:		0.5	1.0	1.5	2.0
Ave. $RMS{E}_F$	Ridge Fuzzy Reg.	2.471	6.266	13.195	23.077
	Fuzzy Reg.	55.455	142.97	233.84	325.40
Ave. $RMSE$	Ridge Reg.	0.440	0.874	1.310	1.746
	Linear Reg.	0.443	0.885	1.328	1.771
Ave. $MAP{E}_F$	Ridge Fuzzy Reg.	4.70%	12.53%	27.65%	49.26%
	Fuzzy Reg.	121.5%	313.5%	513.2%	714.9%
Ave. $MAPE$	Ridge Reg.	0.37%	0.74%	1.11%	1.49%
	Linear Reg.	0.38%	0.76%	1.14%	1.52%

. Measures of performance are summarized for all combinations of factors $\rho ,\sigma$ and whether the fuzzy output is symmetric or not. The following remarks can be made on the basis of Table 1, Table 2, Table 3, Table 4, Table 5 and Table 6:

• Ave. RMSE and Ave. MAPE do not depend on whether the spreads are symmetric or not as they are computed from the mid-points of the generated data. Ridge regression achieves smaller Ave. RMSE than linear regression in all cases. Ridge regression achieves smaller or nearly equal Ave. MAPE with linear regression in all cases. If the Ave. RMSE of ridge regression is smaller than linear regression a similar pattern is observed for $\mathrm{Ave}.{\mathrm{RMSE}}_F$ values. This relationship is observed for Ave. MAPE and $\mathrm{Ave}.{\mathrm{MAPE}}_F$ as well.
• $\mathrm{Ave}.{\mathrm{RMSE}}_F$ increases as $\sigma$ increases for both models. As $\sigma$ and $\rho$ increases, the $\mathrm{Ave}.{\mathrm{RMSE}}_F$ difference between ridge fuzzy regression and fuzzy linear regression increases as well. $\mathrm{Ave}.{\mathrm{RMSE}}_F$ values are larger when the spreads are symmetric. In all scenarios, ridge fuzzy regression $\mathrm{Ave}.{\mathrm{RMSE}}_F$ values almost always outperform those of fuzzy linear regression.
• $\mathrm{Ave}.{\mathrm{MAPE}}_F$ exhibit near identical patterns with $\mathrm{Ave}.{\mathrm{RMSE}}_F$. For both ridge fuzzy regression and fuzzy linear regression, $\mathrm{Ave}.{\mathrm{MAPE}}_F$ is larger for bigger $\sigma$ values. The difference between the two models increases as $\sigma$ and $\rho$ increases. When the fuzzy dependent variable is symmetric the $\mathrm{Ave}.{\mathrm{MAPE}}_F$ values are larger than when it is asymmetric. $\mathrm{Ave}.{\mathrm{MAPE}}_F$ is in general lower for ridge fuzzy regression than fuzzy linear regression for all $\sigma$ and $\rho$ combinations and asymmetric, symmetric outputs.

5. Empirical Study

In this section, we demonstrate the performance of the proposed ridge fuzzy regression model on an illustrative example taken from Tanaka [10]. The performance of the ridge fuzzy regression estimator is compared with the fuzzy least squares estimator for crisp explanatory variables and a fuzzy dependent variable. The linear regression fuzzy model from Tanaka [10] is further compared to illustrate the performance of the ridge fuzzy regression model. For both the ridge fuzzy regression and the linear fuzzy model, the $\alpha$-level sequences ${\alpha }_k=r\times k$, $k=0,\cdots ,K$ for some r and K are chosen as candidates for Step 6 of the estimation algorithm in Section 3.2. The list of $\alpha$-level sequences is presented in

Table 7. The list of $\alpha$-level sequences for Step 6 of the estimation algorithm.

${\mathit{\alpha }}_{\mathit{k}}=\mathit{r}\times \mathit{k}$, $\mathit{k}=0,\cdots ,\mathit{K}$
No.	r	K
1	0.01	100
2	0.02	50
3	0.025	40
4	0.04	25
5	0.05	20
6	0.1	10
7	0.15	6
8	0.2	5
9	0.25	4
10	0.3	3
11	0.5	2
12	1	1

.

Example: House Prices Data

Tanaka et al. [10] presents a data set concerning the price mechanism of prefabricated houses. The relationship between five crisp inputs (rank of material, first floor space (m²), second first floor space (m²), number of rooms and number of Japanese-style rooms) and a fuzzy output (house price) is investigated. The complete data is shown in

Table 8. Houses prices data.

No.	$\mathit{Y}=({\mathit{y}}_{\mathit{m}},{\mathit{S}}_{\mathit{l}}^{\mathit{y}}={\mathit{S}}_{\mathit{r}}^{\mathit{y}})$	${\mathit{x}}_1$	${\mathit{x}}_2$	${\mathit{x}}_3$	${\mathit{x}}_4$	${\mathit{x}}_5$
1	(6060, 550)	1	38.09	36.43	5	1
2	(7100, 50)	1	62.10	26.50	6	1
3	(8080, 400)	1	63.76	44.71	7	1
4	(8260, 150)	1	74.52	38.09	8	1
5	(8650, 750)	1	75.38	41.40	7	2
6	(8520, 450)	2	52.99	26.49	4	2
7	(9170, 700)	2	62.93	26.49	5	2
8	(10,310, 200)	2	72.04	33.12	6	3
9	(10,920, 600)	2	76.12	43.06	7	2
10	(12,030, 100)	2	90.26	42.64	7	2
11	(13,940, 350)	3	85.70	31.33	6	3
12	(14,200, 250)	3	95.27	27.64	6	3
13	(16,010, 300)	3	105.98	27.64	6	3
14	(16,320, 500)	3	79.25	66.81	6	3
15	(16,990, 650)	3	120.50	32.25	6	3

. The fitted values for the ridge fuzzy model and the linear fuzzy model is shown in

Table 9. Fitted values of house prices data.

No.	$\mathit{Y}=({\mathit{y}}_{\mathit{m}},{\mathit{S}}_{\mathit{l}}^{\mathit{y}}={\mathit{S}}_{\mathit{r}}^{\mathit{y}})$	Ridge Fuzzy Reg.	Fuzzy Reg.
1	(6060, 550)	(5812.32, 1024.08)	(5735.21, 1621.95)
2	(7100, 50)	(6819.04, 996.70)	(6785.09, 1668.23)
3	(8080, 400)	(7988.07, 1118.22)	(8059.27, 1868.14)
4	(8260, 150)	(8214.21, 1091.78)	(8143.87, 1866.29)
5	(8650, 750)	(9233.67, 1158.99)	(8684.11, 2002.89)
6	(8520, 450)	(8894.65, 1026.66)	(8884.70, 1708.12)
7	(9170, 700)	(9493.11, 1042.17)	(9436.42, 1,770.07)
8	(10,310, 200)	(11,051.53, 1143.94)	(10,266.90, 1992.65)
9	(10,920, 600)	(11,272.79, 1170.98)	(11,276.09, 2024.78)
10	(12,030, 100)	(12,309.87, 1190.31)	(12,561.81, 2108.54)
11	(13,940, 350)	(13,837.21, 1153.57)	(13,781.85, 2059.16)
12	(14,200, 250)	(14,315.80, 1144.40)	(14,372.32, 2080.39)
13	(16,010, 300)	(15,122.16, 1161.12)	(15,372.29, 2147.15)
14	(16,320, 500)	(15,677.92, 1375.23)	(16,093.51, 2388.31)
15	(16,990, 650)	(16,517.66, 1213.89)	(17,106.59, 2285.64)

. Results show the predicted values from the ridge fuzzy regression more accurately describes the original data than fuzzy linear regression. This is again clarified in Figure 5. In the triangular fuzzy plot of the observed and fitted values, a comparison of the two models is shown. The black triangles correspond to the observed values, the red triangles in Figure 5a to the ridge fuzzy fitted values, and the blue triangles in Figure 5b to the fuzzy linear fitted values. Both methods estimated the mid-points of the fuzzy dependent variable well. The spreads however, are shorter for the proposed ridge fuzzy regression than the other. The fitted equation for the ridge fuzzy regression is given by

$$\begin{array}{cc}\hfill \widehat{\mathit{Y}}=& (-1839.23,-1156.78,-474.32)+(1874.56,1874.56,1874.56)x_1+\hfill \\ \hfill & (73.73,75.29,76.85)x_2+(59.04,65.57,72.10)x_3+\hfill \\ \hfill & (-149.93,-149.93,-149.93)x_4+(543.50,587.74,631.98)x_5\hfill \end{array}$$

(23)

and for the fuzzy linear regression, the fitted equation is

$$\begin{array}{cc}\hfill \widehat{\mathit{Y}}=& (-2038.12,-1129.61,-221.09)+(2386.56,2386.56,2386.56)x_1+\hfill \\ \hfill & (87.13,93.37,99.60)x_2+(71.72,82.13,92.54)x_3+\hfill \\ \hfill & (-376.35,-376.35,-376.35)x_4+(-285.03,-188.25,-91.48)x_5.\hfill \end{array}$$

(24)

Please note that the fitted equation for the linear regression fuzzy model shown in Tanaka et al. [10] is

$$\begin{array}{cc}\hfill \widehat{\mathit{Y}}=& (10,220,11,040,11,860)+(1810,1810,1810)x_1+\hfill \\ \hfill & (1770,2140,2510)x_2+(870,870,870)x_3+\hfill \\ \hfill & (-540,-540,-540)x_4+(-180,-180,-180)x_5.\hfill \end{array}$$

(25)

An analysis of the $\alpha$-level sequences used in Step 6 of the estimation algorithm is presented in Figure 6. The $\alpha$-level sequence which minimizes ${\mathrm{RMSE}}_F$ was chosen as the optimal $\alpha$-level sequence for each of the models. The red dots in Figure 6a,b each indicate the chosen $\alpha$-level sequence based on ${\mathrm{RMSE}}_F$. For the ridge fuzzy regression, ${\alpha }_k=r\times k$, $k=0,\cdots ,K$ with $r=0.01,K=100$ was chosen. In the case of fuzzy linear regression, $r=0.5,K=2$ was selected.

In

Table 10. ${\mathrm{RMSE}}_F$ and ${\mathrm{MAPE}}_F$ fuzzy performance measures for ridge fuzzy regression, fuzzy linear regression, and the linear regression fuzzy model from Tanaka et al. [10].

	RMSEF	MAPEF
Ridge Fuzzy Reg.	1327.78	18%
Fuzzy Reg.	2321.24	33%
Tanaka et al.	349,851.8	55%

the performance measures ${\mathrm{RMSE}}_F$ and ${\mathrm{MAPE}}_F$ ridge fuzzy regression are compared with the fuzzy linear regression model and the linear regression fuzzy model from Tanaka et al. [10]. Clearly both measures are greatly reduced for the ridge fuzzy regression compared to the other models, suggesting that the proposed ridge fuzzy regression model provides a better fit of the data in comparison to the two methods.

6. Conclusions

This paper proposes an $\alpha$-level estimation algorithm for ridge fuzzy regression modeling, extending the ridge regression model introduced in Choi et al. [22]. As shown in simulation studies and an empirical study, the proposed ridge fuzzy regression model can handle fuzzy data sets with crisp inputs and triangular fuzzy outputs. The same procedure is available with fuzzy inputs and fuzzy outputs, or fuzzy inputs and crisp outputs. In previous works, estimation methods for ridge fuzzy regression depend on the distance between fuzzy numbers. By incorporating $\alpha$-levels to ridge fuzzy regression, we are able to construct the ridge fuzzy estimator without having to define the distance between fuzzy numbers. Simulation results show the ridge fuzzy regression model reduces the effect of multicollinearity over a wide range of spreads for the fuzzy response, for various levels of correlation between inputs. In the illustrative example taken from Tanaka et al. [10], we have shown the practical implementations of our method. Comparison is made with fuzzy linear regression with respect to RMSE and MAPE for fuzzy numbers. Overall these results demonstrate the effectiveness of ridge regression in fuzzy data.

An importance point to note is that typically ridge regression is preferred over lasso regression when the objective of research is to handle multicollinearity while not wanting to remove low contributing variables. However, when the dimension of the data is large and dropping collinear variables is necessary, one may use lasso regression rather than ridge regression. To manage such cases, in future studies we plan to extend the proposed $\alpha$-level estimation algorithm for ridge fuzzy models to lasso fuzzy regression models. Lasso fuzzy regression will be especially useful for modeling correlated genetic data sets.

In addition, the present study can be extended for neutrosophic statistics [9,29,30,31,32,33,34,35] as future research.