The Use of Fuzzy Linear Regression and ANFIS Methods to Predict the Compressive Strength of Cement

Abstract

In this paper, the prediction of compressive cement strength using the fuzzy linear regression (FLR) and adaptive neuro-fuzzy inference system (ANFIS) methods was studied. Specifically, an accurate prediction method is needed as the modeling of cement strength is a difficult task, which is based on its composite nature. However, many approaches are widely implemented in strength-predicting problems, such as the artificial neural network (ANN), Mamdani fuzzy rules in MATLAB, FLR and ANFIS models. Applying these methods and comparing the results with the corresponding observed ones, we concluded that the ANFIS method successfully decreased the level of uncertainty in predicting cement strength, as the average percentage error level was extremely low. Although the FLR method had the highest average percentage error level compared with the other methods, it provides a standard equation to estimate the output values by using symmetric triangular fuzzy numbers and determines the most important factor in increasing compressive strength, in contrast to ANFIS and ANN, which are black box models, and to the fuzzy method, which uses rules without providing the specific way by which the results come out. Thus, ANFIS and FLR are appropriate methods for dealing with engineering mathematical models by using fuzzy logic.

1. Introduction

Concrete is recognized as the most broadly used material in the construction of buildings, subway projects and other civil structures, which consists of cement, water and aggregates. Sometimes, powdered or liquid additives are also included in the mixture in order to improve its specific properties, such as the durability of the concrete, or to control setting or hardening. Concrete is widely used, as its raw materials are available worldwide. Concrete’s properties depend on the quality of its materials. One of the most important properties that it has is the high compressive strength at 28 days after casting. Therefore, it is necessary to test the concrete’s compressive strength, so as to define the quality and also the strength of the construction.

Cement is a material consisting of a binder within, widely used in the production of construction essentials as it acquires strong stability and durability after hardening. In this research, CEM I 42.5R European standard EN197-1 [1] was used as the type of cement, containing 95–100% clinker composition and 0–5% other additional constituents. Cement hydration is a complex process resulting from water’s reaction with cement. There are many physical and chemical parameters that affect cement hydration and, therefore, its compressive strength. One of them is the Portland cement clinker composition [2], as it mainly consists of tricalcium silicate (C₃S), which is responsible for the early strength and early characteristics of cement, dicalcium silicate (C₂S), which contributes to cement’s long-term compressive strength, tricalcium aluminate (C₃A), which in a low amount provides resistance to sulfates, tetracalcium aluminoferrite (C₄AF) that works as a flux material and sometimes trioxide sulfate (SO₃) [3], which is a gypsum that is added to the clinker after its cooling for minimizing the amount of grinding energy needed. Depending on its application, cement contains over 50–70% C₃S, 15–30% C₂S, 5–10% C₃A, 5–15% C₄AF [4] and less than 4% SO₃ [3]. Cement hydration also depends on the Blaine fineness of the cement particles, which ranges in value between 2000 and 5000 cm²/g, and by the alkali content in the cement composition.

Portland cement production is not an environmentally friendly process as it releases a massive amount of carbon dioxide (CO₂) into the atmosphere, causing climate change and greenhouse gas emissions. On the other hand, non-conventional cementitious materials [5,6,7,8,9] are being developed in order to reduce environmental pollution and replace ordinary Portland cement with sustainable materials like self-healing concretes or alternative geopolymer-based concretes, which seem to be promising in concrete construction. Non-conventional cementitious materials also improve the characteristics of the cement and increase its compressive strength. Τhe ordinary cement compressive strength can also be improved by incorporating nanoparticles [10] in its composition to increase the density of the concrete. The prediction of the compressive strength of these types of concrete is a very important process, as it provides an option to modify the mix proportion in circumstances where the mandatory design strength is not attained, in order to avoid construction failures and substitute successfully the stability offered by Portland cement. As indicated, analytical models that include the effects of each of these factors on the compressive strength may be very complicated. Consequently, the use of fuzzy regression models, adaptive neuro-fuzzy inference systems (ANFISes) [11,12,13], as well as artificial neural networks (ANNs) [11,14,15] and fuzzy logic, seems to be a promising approach to the strength prediction problem, providing useful tools in the concrete industry.

In a previous study [16], the 28-day compressive strength was predicted with the use of ANNs and Mamdani fuzzy rules in MATLAB. The ANN method [17] was based on the structure of an artificial neural network with three levels. The first layer (input), as well as the second layer (hidden), had four neurons, and there was one output parameter, the compressive strength, in the last layer. The inputs, which were C₃S, SO₃, Blaine fineness and alkali, were first normalized to ranges from 0.1 to 0.9, using the equation

$$X=\alpha +\left(b-\alpha \right)·\frac{(X_i-X{\min }_i)}{\left(X{\max }_i-X{\min }_i\right)}$$

(1)

where α and b are the lowest and highest values of the range of normalization, respectively, Xmin_i and Xmax_i are the lowest and highest values of each input, respectively, and X_i is the value of each input of the ith node.

Then, those values were multiplied by the weight factors in order to train the model, with the use of the equation

$$ne{t}_j={{\displaystyle \sum }}^{}{x}_i{v}_{ij}$$

(2)

where x_i is the input parameter from the ith neuron and v_ij is the weight from the previous level i in the next level j. Afterwards, the aforementioned values were transferred to the hidden layer. For determining the activation level, the sigmoid transfer function was used with the following form:

$$f\left(ne{t}_j\right)=\frac{1}{1+e^{-ne{t}_j}}.$$

(3)

Furthermore, a backpropagation algorithm [18] was used to minimize the error level between the output value data and the calculated values, optimizing the weight vector with the following form:

$$E={\displaystyle \sum }_P{\displaystyle \sum }_p{\left(y_i-t_i\right)}^2$$

(4)

in which y_i is the vector of the calculated output value, t_i is the target output, P is the number of training patterns and p is the number of output neurons. The modification of the network weights was accomplished by using the following equation:

$$v_{ij}^{new}=v_{ij}^{old}-\mathsf{\delta }\frac{dE}{d{v}_{ij}}$$

(5)

where δ, the learning rate, was equal to 0.01. The model was trained for 20,000 epochs. More details on the ANN method can be retrieved from [19].

The fuzzy logic algorithm was based on Mamdani rules and consisted of four components. Fuzzification was the first one, where each variable was represented by degrees of membership between the values 0 and 1. The variables were characterized as very low, low, medium and high. Then, rules were defined in order to comprise all possible fuzzy combinations between the processing parameters. Those rules were expressed in an If-Then conditional statement. In the fuzzy inference engine, all the fuzzy rules were taken into account and the inputs were converted to the corresponding outputs. The product method (prod) was used as an inference operator. Finally, in the last component, defuzzification [20], outputs from the previous step were converted into a number using the centroid method

$$u=\frac{{{\displaystyle \sum }}_i\mu \left(x_i\right)x_i}{{{\displaystyle \sum }}_i\mu \left(x_i\right)}$$

(6)

where x_i is the value of the output and μ(x_i) is its membership value in the membership function.

In this study, the same inputs were applied in order to develop more effective fuzzy models to predict the compressive strength of cement at 28 days. Fuzzy linear regression and ANFIS methods were used to predict 50 sets of cement strength and the results of them were compared with those of the ANN and fuzzy logic algorithm methods.

2. Fuzzy Linear Regression (FLR)

Linear regression [21] provides a crisp approach between a dependent output Y and independent inputs X, using the following form:

$$Y={\alpha }_0+{\alpha }_1{{\rm X}}_1+\dots +{\alpha }_n{X}_n+u$$

(7)

where u includes the deviations between the observed and the predicted parameters, known as the disturbance term.

Fuzzy linear regression is an alternative process of probabilistic specification, expressed as

$$Y=A_0+A_1{{\rm X}}_1+\dots +A_n{X}_n$$

(8)

where A_i are symmetric fuzzy numbers, which includes the inability to determine an exact association between the dependent and independent parameters. They are expressed as A = (r_i,c_i)_L, where r_i is the center in which the membership function is equal to 1 and c is the range of values. Thus, the membership function [22] is defined as

$${\mu }_A\left(x\right)=L\left(\frac{x-r}{c}\right)$$

(9)

where L(x) is the symmetric reference function of a fuzzy number that satisfies the constraints below:

L(x) = L(−x),

(10)

L(0) = 1,

(11)

L(x) is decreasing in [0, ꚙ).

(12)

Therefore, the membership function for the linear possibility Equation (8) is formed as

$${\mu }_{Y_j}\left(y_j\right)=L\left[\frac{y_j-(r_0+{{\displaystyle \sum }}_{i=1}^n{r}_i{x}_{ij})}{c_0+{{\displaystyle \sum }}_{i=1}^n{c}_{i }|x_{ij}|}\right]$$

(13)

where i is the number of inputs and j is the number of sets. Then, the degree h is determined in order to include the data in the estimated output Y, which is

$${\mu }_{Yj} \left(y_j\right) \ge h.$$

(14)

In this study, the reference function L(x) had the following form:

$$L\left(x\right)=max\left(0, 1-|x|\right)$$

(15)

where L(x) is decreasing in (0,1). As a result, the coefficients of the system (8) were symmetric triangular fuzzy numbers and the following linear programming problem was turned out:

$$J=min\{m{c}_0+{\displaystyle \sum }_{j=1}^m{\displaystyle \sum }_{i=1}^n{c}_i|x_{ij}|\}$$

(16)

$$y_j\ge r_0+{\displaystyle \sum }_{i=1}^n{r}_i{x}_{ij}-\left(1-h\right)\left(c_{0 }+{\displaystyle \sum }_{i=1}^n{c}_i|x_{ij}|\right)$$

(17)

$$y_j\le r_0+{\displaystyle \sum }_{i=1}^n{r}_i{x}_{ij}+\left(1-h\right)\left(c_{0 }+{\displaystyle \sum }_{i=1}^n{c}_i|x_{ij}|\right)$$

(18)

$$c_i\ge 0, i=1,2,\dots ,n$$

(19)

where the relations (17)–(19) were used to minimize the objective function in (16).

Furthermore, the simplex method was used to solve the linear programming problem by combining the relations (16)–(19). The results of the fuzzy triangular numbers are represented in

Table 1. The results of the fuzzy triangular numbers.

Variable	Estimate Ri	Estimate Ci	SE	T	P-Value
A0	13.624	0.000	12.268	1.111	0.273
A1	0.411	0.056	0.135	3.047	0.004
A2	6.633	0.000	1.482	4.475	0.000
A3	−0.002	0.000	0.002	−0.802	0.427
A4	4.192	0.000	3.813	1.099	0.277

.

The equation of fuzzy linear regression was formed as follows:

$$Y=A_0+A_1{{\rm X}}_1+A_2{X}_2+A_3{X}_3+A_4{X}_4.$$

(20)

With regard to the membership function, it depends on the distance of the observed output values from the center of regression. In particular, the closer the 28-day compressive strength is to the center of the regression, the higher the increase of the membership function is. The values of the root mean square error (RMSE) and the mean absolute percentage error (MAPE) were calculated as

$$RMSE=\sqrt{\frac{{{\displaystyle \sum }}_{t=1}^n{\left(e_t\right)}^2}{n}}=2.04$$

(21)

$$MAPE=100\% \frac{1}{n}{\displaystyle \sum }_{t=1}^n\left|\frac{e_t}{A_t}\right|=100\%\frac{1}{n}{\displaystyle \sum }_{t=1}^n\left|\frac{A_t-F_t}{A_t}\right|=3.28\%$$

(22)

where A_t and F_t are the observed and computed values, respectively, and n is the number of sets.

Theil’s inequality coefficient is a measure of the accuracy of regression and was equal to

$$Theil’s U=\frac{\sqrt{\frac{1}{n}{{\displaystyle \sum }}_{t=1}^n{\left(F_t-A_t\right)}^2}}{ \sqrt{\frac{1}{n}{{\displaystyle \sum }}_{t=1}^n{\left(F_t\right)}^2}+\sqrt{\frac{1}{n}{{\displaystyle \sum }}_{t=1}^n{\left(A_t\right)}^2}}=0.019$$

(23)

which indicated that the fuzzy linear regression had a successful predictive capacity. The degree of the fuzziness [21] of this model was formed as

$$F\left(M^h\right)=\sqrt{{\left(c_0^h\right)}^2+{\left(c_1^h\right)}^2+\cdots +{\left(c_n^h\right)}^2}=0.056$$

(24)

where c_i contributes to the fuzziness of the system. In order to find the effect of independent inputs on the calculation of the dependent variable Y, the degree of fuzziness was calculated separately for four different models. Each model was comprised of three input variables instead of four and each time a different variable was subtracted. The results are summarized in

Table 2. The results of the degree of fuzziness.

Model with Included Inputs	F(Mh)
SO3, Blaine, alkali	1.355
C3S, Blaine, alkali	0.639
C3S, SO3, alkali	0.577
C3S, SO3, Blaine	1.283

.

Taking those results into account, it was concluded that C₃S was the most important factor in increasing the compressive strength at 28 days, as it had the highest effect on the system’s degree of fuzziness. It also constitutes 50–70% of cement’s composition, which defines it as the main component in the cement. However, all variables were important for developing a successful fuzzy model to predict the 28-day compressive strength of cement. More details on the fuzzy linear regression method can be retrieved from [23,24].

3. Adaptive Neuro-Fuzzy Modeling (ANFIS)

ANFIS is a fuzzy model used as a method of constructing rule systems, by entering the data of the prediction model to derive more efficient membership functions. The membership function parameters are tuned with the use of methods based on ANNs, as ANFIS is a type of adaptive network that functions as a fuzzy system. The structure of this model is shown in Figure 1 and it is composed of four inputs, hidden layers and one output datum. The nodes of each level were connected to each other through links that indicated the direction of the information of the neural network.

In the first level, membership grades were generated for each input with the following form:

$$O_{i,1}={\mu }_{Ai}\left(x_1\right)$$

(25)

$$O_{i,1}={\mu }_{Bi}\left(x_2\right)$$

(26)

$$O_{i,1}={\mu }_{Ci}\left(x_3\right)$$

(27)

$$O_{i,1}={\mu }_{Di}\left(x_4\right)$$

(28)

where μ_Ai(x₁), μ_Bi(x₂), μ_Ci(x₃) and μ_Di(x₄) were the trapezoidal membership functions in this study and i = 1,2,3. At each node, the incoming signals were multiplied and this information was sent to the output, using the min or prod operator. The prod method was used in this study with the following form:

$$O_{i,2} ={\mu }_{Ai}(x_1)· {\mu }_{Bi}(x_2)·{\mu }_{Ci}\left(x_3\right)·{\mu }_{Di}\left(x_4\right)=w_i$$

(29)

where w_i is the weight parameter from the ith neuron. In the third layer, the degree of membership function of a rule from each node resulted, using the equation

$$I_{i,3}=\overline{w_i}=\frac{w_i}{w_1+w_{2 }+w_3+w_4}$$

(30)

where $(w_1+w_2+w_3+w_4$) is the total weights extracted from the previous level. Then, in the fourth layer, each node was defined as

$$I_{i,4}=\overline{w_i}{f}_i=\overline{w_i}\left(p_{i}x+q_{i}y+r_i\right)$$

(31)

where $\overline{w_i}$ is the parameter from the previous layer and {$p_i,q_i,r_i$} are the consequent parameters for each rule. Thus, every node layer was specified as an adaptive node. In the final layer, all of the inputs were used to calculate the output by using

$$O_{i,5}={\displaystyle \sum }_i\overline{w_i} f_i=\frac{{{\displaystyle \sum }}_i{w}_i{f}_i}{{{\displaystyle \sum }}_i{w}_i}.$$

(32)

The ANFIS algorithm used Takagi–Sugeno–Kang rules and the output membership function was a linear function derived from the input values, in contrast to the Mamdani system, where the output of every rule was a fuzzy set.

The adjustment of the weights of each parameter provided minimization of the error level between the observed and predicted values of the output. A hybrid learning algorithm that associates the least squares method and the backpropagation gradient descent method was used in order to train Sugeno-type parameters and calculate input and output membership function parameters.

The model was trained for 20,000 epochs. The predicted and the observed outputs for testing data are presented in Figure 2. The checkpoints coincided with the data, as the error level was extremely low (MAPE = 0.0270%). Thus, the model responded satisfactorily to the observed data.

4. Model Application

The estimation of 50 sets of the 28-day compressive strength of cement was analyzed by using two fuzzy methods: fuzzy linear regression and ANFIS. The results of these methods are reported in

Table 3. Comprehensive strength of cement at 28 days as determined by the fuzzy linear regression (FLR) and ANFIS methods across 50 sets.

C3S	SO3	Blaine	Alkali	Strength	FLR (Left)	FLR (Center)	FLR (Right)	μA(yi)	ANFIS
54	3	3530	1.1	53.9	50.7	53.7	56.8	0.9	53.9
54.8	2.9	3680	0.9	51.9	49.2	52.3	55.3	0.9	51.9
57.3	2.8	3560	1	53.9	50.1	53.3	56.5	0.8	53.9
64.6	2.6	3850	1	50.8	50.8	54.4	58	0	50.8
56.9	2.7	3580	0.8	54.5	48.4	51.6	54.8	0.1	54.5
61.3	2.3	3780	0.9	50.4	47.3	50.8	54.2	0.9	50.5
62.3	2.8	3640	0.9	55.4	51.3	54.8	58.3	0.8	55.4
62.4	2.8	3590	0.9	58.4	51.4	54.9	58.4	0	58.4
64.6	2.5	4090	0.8	54.8	48.8	52.5	56.1	0.4	54.7
59.3	2.8	3500	1.1	51.8	51.3	54.6	58	0.1	51.8
61.8	2.7	3630	1.1	51.3	51.3	54.8	58.2	0	51.3
61.3	3	3580	1	54.7	52.8	56.2	59.7	0.6	54.7
60.4	2.6	3680	1	54.1	49.6	53	56.4	0.7	54.1
55.6	3.1	3510	1	54.5	51.6	54.7	57.8	0.9	54.6
62.4	2.5	3590	1.1	51.5	50.3	53.8	57.2	0.4	51.5
63.1	2.6	3540	0.9	52.1	50.4	54	57.5	0.5	52.1
61.2	2.7	3610	0.9	51.7	50.3	53.7	57.1	0.4	51.7
55.6	2.7	3620	0.9	54.2	48.3	51.4	54.5	0.1	54.2
67.3	2.6	4020	0.8	53.8	50.6	54.4	58.1	0.8	53.8
58.7	3	3550	0.9	51.5	51.5	54.8	58.1	0	51.5
65.4	2.3	3730	0.9	48.9	48.9	52.6	56.2	0	48.9
58	2.7	3420	1	53.2	49.9	53.2	56.4	0.99	53.3
65	2.5	4070	0.8	54.7	49	52.7	56.3	0.4	54.7
62	2.9	3720	1	54.3	52.1	55.6	59.1	0.6	54.3
61.4	2.7	3840	0.9	52.5	49.9	53.4	56.8	0.7	52.5
63.5	2.5	3540	1	51.3	50.3	53.9	57.4	0.3	51.3
62.8	2.3	3580	0.9	51.1	48.3	51.8	55.3	0.8	51.1
56.4	3	3370	1.1	52.5	51.9	55	58.2	0.2	52.5
62.8	3	3750	1.1	54.1	53.4	56.9	60.5	0.2	54.1
58.9	3	3540	1	53.5	52	55.3	58.6	0.5	53.5
62.3	2.5	3910	0.9	53.6	48.8	52.3	55.8	0.6	53.6
57.7	2.7	3480	1	55.4	49.7	52.9	56.2	0.2	55.5
55.8	3.1	3420	0.9	53.7	51.4	54.5	57.6	0.7	53.7
55.9	2.8	3620	1	55.6	49.5	52.6	55.7	0.04	55.6
60.7	2.8	3740	1.1	55.2	51.4	54.8	58.2	0.9	55.2
50.3	2.5	3750	1.1	55.5	48.9	52.2	55.5	0	55.5
60.8	2.2	3520	1.1	49.8	47.8	51.2	54.6	0.6	49.8
60.7	3	3840	0.9	55.6	51.7	55.1	58.5	0.8	55.6
63.2	2.5	4010	0.9	52.1	48.9	52.5	56	0.9	52.1
59.3	2.6	3450	1	51.6	49.7	53	56.3	0.6	51.6
65.8	2.6	4050	0.9	53	50.4	54.1	57.8	0.7	53
57.4	2.5	3390	1.1	50.5	48.9	52.1	55.3	0.5	50.7
62	2.4	3490	1	54	49.2	52.7	56.2	0.6	54
59.7	2.2	3890	1	52.1	46.3	49.7	53	0.3	52.1
56.8	2.7	3620	1	53.8	49.1	52.3	55.5	0.5	53.8
61.7	2.4	3630	0.9	53.6	48.4	51.9	55.3	0.5	53.6
63.9	2.8	3680	0.9	53	51.7	55.2	58.8	0.4	53
61.6	2.8	3630	1.1	53.5	51.9	55.3	58.8	0.5	53.5
64.9	2.4	3900	1	49.9	49.5	53.1	56.8	0.1	49.9
61	2.8	3700	0.9	54.2	50.7	54.1	57.5	0.98	54.2

in order to compare the two fuzzy models. According to the results, the fuzzy linear regression and ANFIS methods successfully predicted the compressive strength of cement with small deviations. The data, which were obtained from a previous study [16], had a limited range of values and led to reliable predictions of the observed cement strength just within the aforementioned range. However, the purpose of the model is to apply and control input parameters in more favorable experimental conditions in order to provide the cement compressive strength with higher accuracy.

The values of C₃S, SO₃, Blaine fineness and alkali are the inputs, strength is the output, FLR (left), FLR (center) and FLR (right) are the values of the fuzzy linear regression in the boundaries and in the center of regression, μ_A(y_i) is the membership grade of the FLR method and ANFIS presents the results obtained from the ANFIS algorithm.

For a better classification, the root mean square error (RMSE) and the mean absolute percentage error (MAPE) were determined for every method and are presented in

Table 4. The root mean square error (RMSE) and mean absolute percentage error (MAPE) results of all of the methods.

Metric	ANN	Fuzzy	Fuzzy Linear Regression	ANFIS
RMSE	1.70	1.84	2.04	0.04
MAPE	2.41%	2.69%	3.28%	0.0270%

, where the values of the ANN and fuzzy model methods were calculated in a previous study [16].

In Figure 3, the fuzzy linear regression is shown, where the boundaries and the center of the regression, as well as the observed values, are represented.

As indicated, all methods had satisfactory results, with small deviations from the observed values of strength. Although the fuzzy linear regression method had a higher RMSE factor than the ANN, ANFIS and fuzzy models, it provided a reliable way to estimate the compressive strength of cement. This method uses a specific equation to compute the predicted values in order to define the coefficients on which the input variables depend for obtaining the result. In addition, the FLR method is an effective way to define the degree of fuzziness in order to calculate the effect of independent inputs on the dependent variable and determine the major factor in increasing the compressive strength of cement at 28 days.

However, the ANFIS algorithm provided the most accurate approach to calculating the compressive strength. The value of the mean square error (0.04) was lower compared with that of all of the other methods, which made the results of the ANFIS method quite satisfactory. The MAPE was computed as 0.0270%, which proved that the values of the observed and calculated outputs were very close. Although ANFIS is a black box model, as there is no access to the rules that have been created, the valid combination of verbal rules, as well as the successful choice of membership function, make it a successful fuzzy model.

5. Conclusions

The estimation of the best compressive strength of cement data is not an easy process, as it contains highly complex factors. In this study, fuzzy linear regression and ANFIS methods were developed in order to create more effective fuzzy models to estimate the 28-day compressive strength of cement. In particular, four inputs were applied in order to determine the output, which was the compressive strength of the cement. After comparing these methods with ANN and fuzzy logic models, which were both studied in a previous publication [16], and evaluating the results, we concluded that the ANFIS algorithm provided the most accurate values of the cement’s compressive strength, as the value of the mean square error was extremely low (0.04).

It was also demonstrated that fuzzy linear regression was a more valid method as it provided a standard equation to estimate the output values, although it yielded higher error values than the other fuzzy models. This is in contrast to the black box methods of the ANFIS algorithm and the ANN model, as well as fuzzy logic, which used rules without providing the specific way by which the results come out. Another advantage of the FLR method was the determination of the degree of fuzziness, which specified the consequence of the independent variables for the compressive strength of cement.

In conclusion, all methods led to reliable results with relatively small deviations. Even though the ANFIS algorithm provided the smallest deviations from the observed values, the most valid method for predicting the compressive strength of cement was the fuzzy linear regression method, which gave the most reliable estimation of the cement’s strength by providing a standard equation. This proves its capability of dealing with the perceptual uncertainties that are involved in strength prediction problems and leads to successful predictions of the predicted cement strength values, improving the design and providing a useful modeling tool in the field of engineering.