Feed-Forward Neural Network Prediction of the Mechanical Properties of Sandcrete Materials

This work presents a soft-sensor approach for estimating critical mechanical properties of sandcrete materials. Feed-forward (FF) artificial neural network (ANN) models are employed for building soft-sensors able to predict the 28-day compressive strength and the modulus of elasticity of sandcrete materials. To this end, a new normalization technique for the pre-processing of data is proposed. The comparison of the derived results with the available experimental data demonstrates the capability of FF ANNs to predict with pinpoint accuracy the mechanical properties of sandcrete materials. Furthermore, the proposed normalization technique has been proven effective and robust compared to other normalization techniques available in the literature.


Introduction
Low-cost buildings have been constructed at numerous different locations across the globe during history. Eco-friendly, low-cost, sustainable construction materials for utilization in civil engineering projects have attracted much attention during the last decades. At present, low-cost construction materials are mainly used in Africa and Asia and involve a wide variety, like gypsum, lime, clay, sand and stabilised soil, rice husk ash, ground granulated blast furnace slag etc., as partial or total replacement of cement, aggregates, and concrete. An average cost reduction of up to 30% as compared to conventional construction methods can be achieved. As reviewed by Bahar et al. [1], different types of earth construction such as soil-based construction blocks have been used in North Africa for centuries, especially in rural regions and in the desert. Soil can be stabilised either by manual compaction, mechanical compaction, or, the addition of natural fibers. However, with the development of masonry and reinforced concrete, soil-based constructions are designed and addressed to poor population and, hence, are of lower quality. This is mainly due to their durability problems such as the lack of water resistance and erosion [1]. However, due to its higher cost and lower thermal performance, much interest is going back to earth construction, which is known for its cheap labour and low cost and comparable thermal insulation characteristics [1]. This was primarily owed to the inefficiencies associated with such materials, mainly in terms of strength and durability (i.e., lack of water resistance and erosion [1]), as well as the high cost associated mainly with masonry construction. Furthermore, materials such as concrete and steel were easier to use in construction and required less skilled labour.
The nature of soils available for such applications is very diverse due to their origin from very different types of geological deposits including several types of clays, sand resources, crushed rock formations, etc. Apart from the inorganic constituents, a very diverse group of organic components can be also identified.
To this end, a promising alternative is soilcrete, a type of non-conventional construction material produced by mixing natural soil, such as natural clay or limestone sand, with a hydraulic binder [2][3][4][5][6]. Soilcrete belongs to a family of concretes that can be used under conditions whereby environmental or economic constraints limit the use of coarse aggregates, as it exploits amounts of proper soils, rocks or even recycled concrete materials that are present in abundance. Its main components are fine aggregate (clay or/and sand), ordinary Portland cement, water at an appropriate ratio and rarely mineral admixtures, thus rendering it a composite material.
Artificial neural networks (ANNs) have emerged over the last decades as an attractive meta-modelling technique applicable to a vast number of scientific fields including material science among others. An important characteristic of ANNs is that they can be used to build soft-sensors, i.e., models with the ability to estimate critical quantities without having to measure them [7]. In particular, such surrogate models can be constructed after a training process with only a few available data, which can be used to predict pre-selected model parameters, reducing the need for timeand money-consuming experiments. Thus far, the literature includes studies in which ANNs were used for predicting the mechanical properties of concrete materials [8][9][10][11][12][13][14][15][16]. In their study Asteris et al. [16] used ANNs to estimate the compressive strength of self-compacting concrete through a training process involving as input parameters the eleven parameters of synthesis with output parameter the value of the compressive strength. Moreover, similar methods such as fuzzy logic and genetic algorithms have also been used for modelling the compressive strength of concrete materials [17][18][19][20][21][22][23][24]. A detailed state-of-the-art report can be found in [25][26][27][28].
The mechanical properties of sandcrete materials exhibit a strong nonlinear nature derived from the parameters involved in their composition; it is this nonlinear behaviour that makes the development of an analytical formula for the prediction of the mechanical properties using deterministic methods a rather difficult task. In this work, the modelling of the mechanical characteristics of soilcrete materials has been investigated in-depth using soft-computing techniques such as artificial intelligence (AI) techniques. In particular, this study investigates the application of Artificial Neural Networks (ANNs) models for the prediction of the 28-day compressive strength and the modulus of elasticity of sandcrete materials, using the four parameters of synthesis and the value of the ultrasonic velocity as input parameters.

Significance of the Subject
Much effort in recent years involves the development of reliable methods for the assessment of the vulnerability of existing structures by means of the accurate estimation of the in-situ mechanical characteristics of materials used. The estimation of the mechanical characteristics of concrete has been traditionally carried out through either destructive or non-destructive methods. The greatest disadvantages of the conventional destructive method of extracting cylindrical specimens (cores) to evaluate the mechanical properties of concrete include cost of intervention, damage caused during the execution of the test, and limited number of extracted specimens, which in turn affects negatively the reliability of results.
On the other hand, non-destructive methods, such as the ultrasonic pulse velocity method, provide a simple, inexpensive method to approximate the concrete properties. Based on the measured ultrasonic velocity, a number of analytical expressions for estimating the compressive strength and dynamic modulus of elasticity of concrete have been proposed in the literature [29][30][31][32][33][34]. Nevertheless, the relatively low correlation between the estimated values with the experimental results, highlights the need to develop new methods for the reliable assessment of these important properties.
To this end, soft-computing techniques, such as the Artificial Neural Networks (ANN), can contribute as a feasible tool for the estimation of the mechanical properties of concrete [10,[35][36][37][38]. In this study, Artificial Neural Networks have been developed for the prediction of the compressive strength and modulus of elasticity of sandcrete materials, using as input parameters the mix compositions and the value of ultrasonic velocity, while as output parameter the value of compressive strength or the value of modulus of elasticity.

Artificial Neural Networks
This section summarizes the mathematical and computational aspects of artificial neural networks.
In general, ANNs are information-processing models configured for a specific application through a training process. A trained ANN maps a given input onto a specific output, and thereby can be considered to be similar to a response surface method. The main advantage of a trained ANN over conventional numerical analysis procedures (e.g., regression analysis) is that the results can be produced with much less computational effort [16,25,[39][40][41][42][43][44][45].

General
The concept of an artificial neural network is based on the concept of the biological neural network of the human brain. The basic building block of the ANN is the artificial neuron, which is a mathematical model trying to mimic the behaviour of the biological neuron. Information is passed into the artificial neuron as input and processed with a mathematical function leading to an output that determines the behaviour of the neuron (similar to fire-or-not situation for the biological neuron). Before the information enters the neuron, it is weighted in order to approximate the random nature of the biological neuron. A group of such neurons consists of an ANN in a manner similar to biological neural networks. In order to set up an ANN, one needs to define: (i) the architecture of the ANN; (ii) the training algorithm, which will be used for the ANN learning phase; and (iii) the mathematical functions describing the mathematical model. The architecture or topology of the ANN describes the way the artificial neurons are organized in the group and how information flows within the network. For example, if the neurons are organized in more than one layers, then the network is called a multilayer ANN. Regarding the training phase of the ANN, it can be considered as a function minimization problem, in which the optimum value of weights need to be determined by minimizing an error function. Depending on the optimization algorithms used for this purpose, different types of ANNs exist. Finally, the two mathematical functions that define the behaviour of each neuron are the summation function and the activation function. In the present study, we use a back-propagation neural network (BPNN), which is described in the next section.

Architecture of BPNN
A BPNN is a feed-forward, multilayer network [39], i.e., information flows only from the input towards the output with no feedback loops, and the neurons of the same layer are not connected to each other, but they are connected with all the neurons of the previous and subsequent layer. A BPNN has a standard structure that can be written as: where N is the number of input neurons (input parameters); H i is the number of neurons in the i-th hidden layer for i = 1, . . . , NHL; NHL is the number of hidden layers; and M is the number of output neurons (output parameters). Figure 1 depicts an example of a BPNN composed of an input layer with five neurons, two hidden layers with seven and four neurons, respectively, and an output layer with 1 neuron, i.e., a 5-7-4-1 BPNN.  For each neuron i, the individual element inputs , … , are multiplied by the corresponding weights , , … , , and the weighted values are fed to the junction of the summation function, in which the dot product ( • ) of the weight vector = , , … , , and the input vector = , … , is generated. The threshold b (bias) is added to the dot-product forming the net input , which is the argument of the transfer function ƒ: The choice of the transfer (or activation) function ƒ may strongly influence the complexity and performance of the ANN. Although sigmoidal transfer functions are the most commonly used, one may use different type of functions. Previous studies [46,47] have proposed a large number of alternative transfer functions. In the present study, the Logistic Sigmoid and the Hyperbolic Tangent transfer functions were found to be appropriate for the problem investigated. During the training phase, the training data are fed into the network which tries to create a mapping between the input and the output values. This mapping is achieved by adjusting the weights in order to minimise the following error function: where and are the measured value and the prediction of the network, respectively, within an optimization framework. The training algorithm used for the optimization plays a crucial role in building a quality mapping, thus an exhaustive investigation was performed in order to find the most suitable for this problem. The most common method used in the literature is the back-propagation technique, in which, as stated by its name, the information propagates to the network in a backward manner in order to adjust the weights and minimize the error function. To adjust the weights   For each neuron i, the individual element inputs , … , are multiplied by the corresponding weights , , … , , and the weighted values are fed to the junction of the summation function, in which the dot product ( • ) of the weight vector = , , … , , and the input vector = , … , is generated. The threshold b (bias) is added to the dot-product forming the net input , which is the argument of the transfer function ƒ: The choice of the transfer (or activation) function ƒ may strongly influence the complexity and performance of the ANN. Although sigmoidal transfer functions are the most commonly used, one may use different type of functions. Previous studies [46,47] have proposed a large number of alternative transfer functions. In the present study, the Logistic Sigmoid and the Hyperbolic Tangent transfer functions were found to be appropriate for the problem investigated. During the training phase, the training data are fed into the network which tries to create a mapping between the input and the output values. This mapping is achieved by adjusting the weights in order to minimise the following error function: where and are the measured value and the prediction of the network, respectively, within an optimization framework. The training algorithm used for the optimization plays a crucial role in building a quality mapping, thus an exhaustive investigation was performed in order to find the most suitable for this problem. The most common method used in the literature is the back-propagation technique, in which, as stated by its name, the information propagates to the network in a backward manner in order to adjust the weights and minimize the error function. To adjust the weights For each neuron i, the individual element inputs p 1 , . . . , p R are multiplied by the corresponding weights w i,1 , . . . , w i,R and the weighted values are fed to the junction of the summation function, in which the dot product (W·p) of the weight vector W = [w i,1 , . . . , w i,R ] and the input vector p = [p 1 , . . . , p R ] T is generated. The threshold b (bias) is added to the dot-product forming the net input n, which is the argument of the transfer function f : The choice of the transfer (or activation) function f may strongly influence the complexity and performance of the ANN. Although sigmoidal transfer functions are the most commonly used, one may use different type of functions. Previous studies [46,47] have proposed a large number of alternative transfer functions. In the present study, the Logistic Sigmoid and the Hyperbolic Tangent transfer functions were found to be appropriate for the problem investigated. During the training phase, the training data are fed into the network which tries to create a mapping between the input and the output values. This mapping is achieved by adjusting the weights in order to minimise the following error function: where x i and y i are the measured value and the prediction of the network, respectively, within an optimization framework. The training algorithm used for the optimization plays a crucial role in building a quality mapping, thus an exhaustive investigation was performed in order to find the most suitable for this problem. The most common method used in the literature is the back-propagation technique, in which, as stated by its name, the information propagates to the network in a backward manner in order to adjust the weights and minimize the error function. To adjust the weights properly, a general method called gradient descent is applied, in which the gradients of the error function with respect to the network weights is calculated. Further discussion on the training algorithms is given in the numerical example section.

Results and Discussion
This section presents the process for tuning optimum ANNs used for the prediction of the 28-day compressive strength and the modulus of elasticity of sandcrete materials, based on experimental data available in the literature.

Experimental
A detailed description of the experimental set-up is given by Kolovos [2][3][4]. Data and results within the provided research database (Table 1) present the measured compressive strength ( f c ) at 28 days, as well as the modulus of elasticity ( Figure 3) of a large set of cylindrical specimens (134) with a height-to-diameter (h/d) ratio equal to 2 (h/d = 2) which have been tested under uniaxial compression. For each sample, four to eight soilcrete specimens were tested and the mean value of these measurements is presented in Table 1 [48]. Before the uniaxial compression test, the value of ultrasonic velocity of all specimens has been measured using the widely-adopted TICO (testing instrument of concrete) ultrasonic instrument. properly, a general method called gradient descent is applied, in which the gradients of the error function with respect to the network weights is calculated. Further discussion on the training algorithms is given in the numerical example section.

Results and Discussion
This section presents the process for tuning optimum ANNs used for the prediction of the 28day compressive strength and the modulus of elasticity of sandcrete materials, based on experimental data available in the literature.

Experimental
A detailed description of the experimental set-up is given by Kolovos [2][3][4]. Data and results within the provided research database (Table 1) present the measured compressive strength  at 28 days, as well as the modulus of elasticity ( Figure 3) of a large set of cylindrical specimens (134) with a height-to-diameter (h/d) ratio equal to 2 (h/d = 2) which have been tested under uniaxial compression. For each sample, four to eight soilcrete specimens were tested and the mean value of these measurements is presented in Table 1 [48]. Before the uniaxial compression test, the value of ultrasonic velocity of all specimens has been measured using the widely-adopted TICO (testing instrument of concrete) ultrasonic instrument.   Table 1.  Three categories of binders (B), variable in synthesis, were investigated; the first that consisted of 100% w/w CEM I 42.5 N Portland cement (PC) was used as reference, the second produced by mixing 90% w/w PC and 10% w/w metakaolin (MK) in the dry mix, and the third by mixing 80% w/w PC and 20% w/w MK (in the dry mix) as partial replacement. Homogeneity of all three categories of blends was reached after mixing MK and PC without further grinding in a laboratory swing mill for 1 h. Mixing of the binder-CQLS mixtures with tap water at 20 • C was conducted in an 80 L capacity laboratory mixer appropriate for concrete production. After casting the molds were covered to minimize water evaporation, stripped after 24 h and the specimens were immersed in lime-saturated water at 20 • C, until testing. A detailed and in-depth description of the experimental set-up can be found in the above-mentioned references.
Each input training vector p is of dimension 1 × 5 and consists of the values of the four parameters of synthesis and the value of the ultrasonic velocity (R = 5), namely the water-to-binder ratio (W/B), the metakaolin addition (MK), the binder (B), the superplasticizer (SP), and the ultrasonic velocity (UV). The corresponding output training vectors are of dimension 1 × 1 and consist, in the first case, of the value of the 28 days compressive strength and, in the second case, of the value of the modulus of elasticity of the sandcrete specimens. Their mean values together with the minimum and maximum values are listed in Table 2.

Normalization of Data
As mentioned previously, the normalization of the input and output parameters has a significant impact on the ANN training. In the present study, during the pre-processing stage, the Min-Max [49] and the ZScore normalization methods have been used. In particular, the five input parameters (Table 1), as well as the output parameters of the 28-day compressive strength and the modulus of elasticity have been normalized using the Min-Max normalization method. As stated in Iruansi et al. [50], in order to avoid problems associated with low learning rates of the ANN, the normalization of the data should be made within a range defined by appropriate upper and lower limit values of the corresponding parameter. In this work, the input and output parameters have been normalized in the range [0,1] and [−1,1], respectively. Moreover, in this work a recently proposed transformation technique called Central has been applied [51], in which the origin of the training data is shifted to the centre of the data with the following formula: where x (x 1 , x 2 , . . . , x n ) are the original data and z i is the i-th transformed data.

BPNN Model Development
In this work, a large number of different BPNN models have been developed and implemented in four different computers in order to investigate the sensitivity of the ANN results to the very nature of the floating-point arithmetic of each computer. Each one of these ANN models was trained over 90 data-points out of the total of 134 data-points, (66.86% of the total number) and the validation and testing of the trained ANN were performed with the remaining 44 data-points. More specifically, 22 data-points (16.57%) were used for the validation of the trained ANN and 22 (16.57%) data-points were used for the testing (estimating the Pearson's correlation coefficient R). The parameters used for the ANN training are summarized as follows.
After a detailed and in-depth investigation among a plethora of training algorithms, the Levenberg-Marquardt algorithm [52] has been used as the optimum training algorithm for the ANN models. The development and training of the ANNs occurs with a number of hidden layers ranging from one to two and with a number of neurons ranging from one to 30 for each hidden layer. Each one of the ANNs is developed and trained for a number of different activation functions, such as the Logistic Sigmoid and the Hyperbolic Tangent functions.
In order to achieve a fair comparison of the various ANNs, the datasets used for their training are manually divided by the user into training, validation, and testing sets using appropriate indices to state whether the data belongs to the training, validation or testing set. In the general case, the division of the datasets into the three groups is made randomly.
Based on the above, a total of 372,000 different BPNN models have been developed and investigated in order to find the optimum NN model for the prediction of the compressive strength (186,000) and the modulus of elasticity (186,000) of sandcrete materials.
The developed ANN models were sorted in a decreasing order based on the Pearson's correlation coefficient value. The architectures of the top twenty models are presented in Table 3 for the case of compressive strength and in Table 4 for the case of modulus of elasticity. Based on these results, the optimum BPNN model for the prediction of the compressive strength is that of 5-7-7-1 ( Figure 4) with Pearson's correlation coefficient R equal to 0.99001, while for the prediction of the modulus of elasticity is that of 5-3-25-1 ( Figure 5) with Pearson's correlation coefficient R equal to 0.988105. Figures 4 and 5 present in detail the architectures of the two optimum NNs for the compressive strength and the modulus of elasticity. Furthermore, these figures depict the methodology followed to achieve the two NN models. Based on the information provided in these figures and the final values of weights presented in the next section, one could readily develop a simple code to be used as a tool both for the practicing engineers, as well as researchers in the field, for the prediction of the mechanical properties of sandcrete materials.             Figure 6. The Pearson's correlation coefficient R of the experimental and predicted compressive strength for the best with two hidden layers BPNN (5-7-7-1).

Final Values of Weights of the FF-NN Model
It is common practice in the majority of published papers on NN models, for authors to present the architecture of the optimum NN model without providing any information regarding the final values of NN weights. Admittedly, this practice has very little value for other researchers and practicing engineers. In order to be useful, a proposed NN architecture should be accompanied by the (quantitative) values of weights. In such a case, the NN model can be readily implemented in an

Final Values of Weights of the FF-NN Model
It is common practice in the majority of published papers on NN models, for authors to present the architecture of the optimum NN model without providing any information regarding the final values of NN weights. Admittedly, this practice has very little value for other researchers and practicing engineers. In order to be useful, a proposed NN architecture should be accompanied by the (quantitative) values of weights. In such a case, the NN model can be readily implemented in an MS-Excel spreadsheet, thus available to anyone interested in the problem of modelling.
In this study, the final values of weights and biases for both cases of compressive strength and modulus of elasticity are explicitly reported in Figures 10 and 11 and Tables A1 and A2 of the Appendix A. By employing the properties defined in Table 1 and using the reported values of weights and biases, one can easily build an ANN-based estimator for the compressive strength and the modulus of elasticity of sandcrete materials.

Final Values of Weights of the FF-NN Model
It is common practice in the majority of published papers on NN models, for authors to present the architecture of the optimum NN model without providing any information regarding the final values of NN weights. Admittedly, this practice has very little value for other researchers and practicing engineers. In order to be useful, a proposed NN architecture should be accompanied by the (quantitative) values of weights. In such a case, the NN model can be readily implemented in an MS-Excel spreadsheet, thus available to anyone interested in the problem of modelling.
In this study, the final values of weights and biases for both cases of compressive strength and modulus of elasticity are explicitly reported in Figures 10 and 11 and Tables A1 and A2 of the appendix A. By employing the properties defined in Table 1 and using the reported values of weights and biases, one can easily build an ANN-based estimator for the compressive strength and the modulus of elasticity of sandcrete materials.  Table A1 of Appendix A).  Table A1).  Table A2 of Appendix A).

Discussion
Based on the above optimum NN models for the prediction of the compressive strength and modulus elasticity of sandcrete materials, the following can be stated: 1. Among the training algorithms available in the literature, the best by far ANN prediction of the sandcrete compressive strength was achieved by using the Levenberg-Marquardt algorithm. 2. Different optimum ANN architectures were found in different computers, which means that the computational environment affects the procedure of ANN training, and subsequently its performance. This can be attributed to the fact that the tested algorithms ultimately rely on basic  Table A2).

Discussion
Based on the above optimum NN models for the prediction of the compressive strength and modulus elasticity of sandcrete materials, the following can be stated:

1.
Among the training algorithms available in the literature, the best by far ANN prediction of the sandcrete compressive strength was achieved by using the Levenberg-Marquardt algorithm.

2.
Different optimum ANN architectures were found in different computers, which means that the computational environment affects the procedure of ANN training, and subsequently its performance. This can be attributed to the fact that the tested algorithms ultimately rely on basic arithmetic operations that can yield different results, when performed in different environments, due to the very nature of floating-point arithmetic.

3.
The recently-proposed formula for the normalization of data proved effective and robust compared to available ones. 4.
For the top twenty models the optimum number of hidden layers was found to be two. This is an indication that the complexity of the problem cannot be dealt with effectively with a single hidden layer.

5.
The best activation functions corresponding to all of the top-twenty best NN models, both for the case of compressive strength and modulus of elasticity, are the same, namely Hyperbolic tangent sigmoid transfer function for the first hidden layer, Log-sigmoid transfer function for the second hidden layer, and also Hyperbolic tangent sigmoid transfer function for the output layer.

Conclusions
This study investigated the application of Artificial Neural Networks (ANNs) models for the prediction of the mechanical properties of sandcrete materials. The comparison of the derived results with the experimental findings demonstrates the effectiveness of ANNs to build soft sensors with the ability to predict in a reliable manner the mechanical properties of sandcrete materials. Moreover, the results obtained using the proposed technique for pre-processing the data were favourably compared to the results obtained by other known normalization techniques available in the literature. This fact demonstrates the need for further research on data pre-processing, prior to the use of the data toward the training and the development of ANNs, taking into account the present-day limitations and constraints in this promising research area.
Author Contributions: Panagiotis G. Asteris and Panayiotis C. Roussis conceived and designed the research; Maria G. Douvika performed the computational implementation of the ANN models and contributed with novel analyses; and all the authors reviewed and revised the manuscript.

Conflicts of Interest:
The authors declare no conflicts of interest.