Predicting Compressive Strength of Cement-Stabilized Rammed Earth Based on SEM Images Using Computer Vision and Deep Learning

: Predicting the compressive strength of cement-stabilized rammed earth (CSRE) using current testing machines is time-consuming and costly and may harm the environment due to the samples’ waste. This paper presents an automatic method using computer vision and deep learning to solve the problem. For this purpose, a deep convolutional neural network (DCNN) model is proposed, which was evaluated on a new in-house scanning electron microscope (SEM) image database containing 4284 images of materials with di ﬀ erent compressive strengths. The experimental results show reasonable prediction results compared to other traditional methods, achieving 84% prediction accuracy and a small (1.5) oot Mean Square Error (RMSE). This indicates that the proposed method (with some enhancements) can be used in practice for predicting the compressive strength of CSRE samples.


Aim and Scope of the Research
This article aims to present that machine learning can predict the compressive strength of building materials with a high degree of accuracy based on scanning electron microscope (SEM) images. The methodology of predicting the compressive strength of cement-stabilized rammed earth (CSRE) is described and the results achieved CSRE is a material used to build construction walls from locally available, inorganic soil found under the layer of humus. Most importantly, CSRE is a highly sustainable construction material. In the study [1], resulting from an analysis using the Building Research Establishment Environmental Assessment Method (BREEAM), external building partitions containing a load-bearing rammed earth layer obtained the highest A+ rating. According to Hall and Swaney [2], such a rating would also be obtained by a partition with a load-bearing layer of rammed earth stabilized with cement. However, designing CSRE of sufficiently high compressive strength is difficult because this feature depends on several mixture properties and technological issues, such as a compaction method [3], density of elements [4][5][6], porosity of elements [6], particle size distribution of the soil [4,5,7], mineral composition of the soil [8], content and type of cement used as a stabilizer [6, 9,10], moisture content of the soil-cement mixture during construction [4,9,[11][12][13], moisture content of element during service [11,12], exposure conditions of the construction elements [5,14], and age of the construction elements [6, 15,16].
A multitude of factors hampers the practical application of rammed earth technology. Even if a method of predicting the compressive strength of CSRE based on these factors is developed, assembling a structure using locally sourced soil would require detailed laboratory tests on a case-by-case basis, to determine the basic parameters of that soil-e.g the mineral composition and particle size distribution of the soil. Other properties of the soil-cement mixture, such as the moisture content, may change unexpectedly due to changes in atmospheric conditions prevailing at the construction site. It is proposed to model the compressive strength of CSRE partitions based on SEM images using machine learning and deep learning techniques instead of the whole process involving laboratory testing and the creation of models of relationships between mixture properties and parameters. The proposed method is presented in the article.

Method of Erecting Monolithic CSRE Walls
The method of erecting monolithic CSRE walls (see Figure 1) includes the following stages [17,18]: Laying concrete foundations and assembling on them a formwork for the designed CSRE walls. -Preparing the soil-cement mixture. I If necessary, the particle size composition of the locally available soil is adjusted. Next, Portland cement is added. The components are mixed in an air-dry state until they reach a uniform consistency. Just before the planned ramming, water is added in a quantity that gives the mixture its predetermined optimal moisture content. -Ramming the moist soil mixture in the formwork in layers. Obtaining the required compaction depends on the method of ramming, energy used for the compaction process, and the thickness of the CSRE layer. The effectiveness of the compaction process should be verified experimentally under construction conditions. -Curing of the finished wall in the formwork for a minimum of one day, followed by demolding.
Further curing of the wall after demolding should be continued until obtaining full serviceable properties by CSRE structure., The minimum curing time is 28 days in the air temperatures above 0 • C (Celsius scale). During this period, it is recommended to cover the partition against precipitation and excessive sun exposure.

Related Work
It was found that various machine learning techniques were used to predict the state of various materials' features based on SEM images of these materials. Lahoti et al. used machine learning-based classifiers and statistical analyses to predict the compressive strength of metakaolin-based geopolymers. They applied random forests; Naive Bayes, and the k-nearest neighbors classifiers, while the statistical analysis was based on an analysis of variance (ANOVA) [19].
The prediction of the compressive strength of concrete modified with glass cullet was conducted by Mirzahosseini and co-workers [20], who used genetic programming to build the prediction models of the compressive strength. Their sensitivity and parametric analyses confirmed that the compressive strength is very sensitive to curing time and temperature, as well as to the surface area of the particles.
Qin et. al. employed a deep learning technique to establish an end-to-end mapping of the nonlinear relationship between SEM images of the cemented paste backfill and its mechanical strengths. However, the average accuracy of the prediction of the mechanical strength was very low [21]. There more examples of this approach e.g. [22,23].
Perhaps the most related study to this paper is the work of Gallagher and co-workers [24], which explores the application of computer vision and machine learning methods to predict the compressive strength of consolidated molecular solid (triaminotrinitrobenzene (TATB)) samples based on their SEM images. However, the TATB samples are not as complex as the CSRE ones, and therefore the reported results may not be able to be generalized to the CSRE, the core of this paper.
The deep learning techniques allow for a wide range of their applications and better solutions for many problems in the computer vision domain, as shown by [25][26][27][28][29][30][31][32][33][34][35], and others. All of these works recommend the use of deep learning techniques to find better solutions in the area of computer optics, in the computer vision domain, which is the main motivation for using the deep learning technique in this paper.
To the best of the authors' knowledge, this work is the first attempt to the problem of predicting the compressive strength of CSRE based on SEM images using computer vision and deep learning. The main contribution of this paper is twofold. It proposes a new deep learning approach to solve the research problem; secondly, it describes a new SEM image database of the CSRE samples, which was created for this study.

Related Work
It was found that various machine learning techniques were used to predict the state of various materials' features based on SEM images of these materials. Lahoti et al. used machine learning-based classifiers and statistical analyses to predict the compressive strength of metakaolin-based geopolymers. They applied random forests; Naive Bayes, and the k-nearest neighbors classifiers, while the statistical analysis was based on an analysis of variance (ANOVA) [19].
The prediction of the compressive strength of concrete modified with glass cullet was conducted by Mirzahosseini and co-workers [20], who used genetic programming to build the prediction models of the compressive strength. Their sensitivity and parametric analyses confirmed that the compressive strength is very sensitive to curing time and temperature, as well as to the surface area of the particles.
Qin et. al. employed a deep learning technique to establish an end-to-end mapping of the nonlinear relationship between SEM images of the cemented paste backfill and its mechanical strengths. However, the average accuracy of the prediction of the mechanical strength was very low [21]. There more examples of this approach e.g. [22,23].
Perhaps the most related study to this paper is the work of Gallagher and co-workers [24], which explores the application of computer vision and machine learning methods to predict the compressive strength of consolidated molecular solid (triaminotrinitrobenzene (TATB)) samples based on their SEM images. However, the TATB samples are not as complex as the CSRE ones, and therefore the reported results may not be able to be generalized to the CSRE, the core of this paper.
The deep learning techniques allow for a wide range of their applications and better solutions for many problems in the computer vision domain, as shown by [25][26][27][28][29][30][31][32][33][34][35], and others. All of these works recommend the use of deep learning techniques to find better solutions in the area of computer optics, in the computer vision domain, which is the main motivation for using the deep learning technique in this paper.
To the best of the authors' knowledge, this work is the first attempt to the problem of predicting the compressive strength of CSRE based on SEM images using computer vision and deep learning. The main contribution of this paper is twofold. It proposes a new deep learning approach to solve the research problem; secondly, it describes a new SEM image database of the CSRE samples, which was created for this study.

Materials
Through adding the appropriate amount of dry ingredients (seven types of loams with different mineral compositions, as well as sand of pure quartz and gravel of 75% quartz and carbonate crumbs), seven artificial soils with two different particle size distributions labeled LC and MC (LC with 4% and MC with 16% of clay fraction, see Figure 2) were obtained. These artificial soil mixtures simulated the inorganic soils with different mineral compositions that could be obtained locally in Europe and were used as the main component of CSRE. Then, cement CEM I 42.R was added in an amount of 6% or 9% by weight of the artificial soil. Finally, water was added to dry mixtures to obtain optimum moisture content (OMC). The compositions of all 14 mixtures are shown in Table 1.

Materials
Through adding the appropriate amount of dry ingredients (seven types of loams with different mineral compositions, as well as sand of pure quartz and gravel of 75% quartz and carbonate crumbs), seven artificial soils with two different particle size distributions labeled LC and MC (LC with 4% and MC with 16% of clay fraction, see Figure 2) were obtained. These artificial soil mixtures simulated the inorganic soils with different mineral compositions that could be obtained locally in Europe and were used as the main component of CSRE. Then, cement CEM I 42.R was added in an amount of 6% or 9% by weight of the artificial soil. Finally, water was added to dry mixtures to obtain optimum moisture content (OMC). The compositions of all 14 mixtures are shown in Table 1.

Preparation of Samples
All CSRE samples were prepared at the construction materials laboratory (Warsaw University of Technology, Warsaw, Poland). In addition to the soil mixture properties listed in Section 3.1, the shape of the samples and the compaction technique also affect the compressive strength of the CSRE. Therefore, cubic 100 × 100 × 100 mm samples for compressive strength tests were formed as described in the work of Hall and Djerbib [36]. The shaping of the samples was carried out by ramming the moist soil-cement mixture in three equal layers. Each layer was compacted by freely lowering a 6.5 kg rammer from a height of 30 cm to the surface of the mixture. Samples were removed from steel molds after 24 h. From the moment of compaction until the compressive strength test, the samples were cured in conditions of a temperature of 20 • C (±1 • C) and relative humidity of 95% (± 2%). Ten samples were prepared for each of the 14 soil-cement mixtures.
Initially, samples were formed with a height of 105 mm. The day before the planned compressive strength test, they were cut to a height of 100 mm (Figure 3). Thin sections to be examined using an SEM were made from the cut fragments of the samples.

Preparation of Samples
All CSRE samples were prepared at the construction materials laboratory (Warsaw University of Technology, Warsaw, Poland). In addition to the soil mixture properties listed in Section 3.1, the shape of the samples and the compaction technique also affect the compressive strength of the CSRE. Therefore, cubic 100 × 100 × 100 mm samples for compressive strength tests were formed as described in the work of Hall and Djerbib [36]. The shaping of the samples was carried out by ramming the moist soil-cement mixture in three equal layers. Each layer was compacted by freely lowering a 6.5 kg rammer from a height of 30 cm to the surface of the mixture. Samples were removed from steel molds after 24 h. From the moment of compaction until the compressive strength test, the samples were cured in conditions of a temperature of 20 °C (±1 °C) and relative humidity of 95% (± 2%). Ten samples were prepared for each of the 14 soil-cement mixtures.
Initially, samples were formed with a height of 105 mm. The day before the planned compressive strength test, they were cut to a height of 100 mm (Figure 3). Thin sections to be examined using an SEM were made from the cut fragments of the samples.

CSRE Compressive Strength Test
CSRE retains the layer structure both in the monolithic wall and in molded samples. For this reason, the method of testing the compressive strength by loading the sample in the direction of its formation was considered representative. Tests on the compressive strength of CSRE samples were carried out in a testing machine with a measuring range of 0-3000 kN, characterized by a measurement error of less than 1%.
The results of the median of compressive strength of each of 14 tested series are shown in Figure  4. Comparing the results of the LC (4% clay fraction) and MC (16% clay fraction) series, it can be seen that the particle size of the soil has a great impact on the CSRE compressive strength. As expected, differences were observed between the compressive strengths of samples with the same particle size and cement content and with different mineral composition.

CSRE Compressive Strength Test
CSRE retains the layer structure both in the monolithic wall and in molded samples. For this reason, the method of testing the compressive strength by loading the sample in the direction of its formation was considered representative. Tests on the compressive strength of CSRE samples were carried out in a testing machine with a measuring range of 0-3000 kN, characterized by a measurement error of less than 1%.
The results of the median of compressive strength of each of 14 tested series are shown in Figure 4. Comparing the results of the LC (4% clay fraction) and MC (16% clay fraction) series, it can be seen that the particle size of the soil has a great impact on the CSRE compressive strength. As expected, differences were observed between the compressive strengths of samples with the same particle size and cement content and with different mineral composition.

SEM Methodology
To present the exact texture of the samples at the micro area, images were taken using an SEM. The magnifications used allowed the observation of small structures within the material matrix as well as small clusters of clay minerals.
Thin sections were sputtered with a thin layer of carbon coating (≈20 nm) to remove charging effects from the surface of the samples (

SEM Methodology
To present the exact texture of the samples at the micro area, images were taken using an SEM. The magnifications used allowed the observation of small structures within the material matrix as well as small clusters of clay minerals.
Thin sections were sputtered with a thin layer of carbon coating (≈20 nm) to remove charging effects from the surface of the samples (

SEM Methodology
To present the exact texture of the samples at the micro area, images were taken using an SEM. The magnifications used allowed the observation of small structures within the material matrix as well as small clusters of clay minerals.
Thin sections were sputtered with a thin layer of carbon coating (≈20 nm) to remove charging effects from the surface of the samples (

Deep Learning Methodology
To predict the compressive strength of the CSRE samples from their SEM images, a deep convolutional neural network (DCNN) regression model was built. The architecture of the proposed model consisted of 24 layers, and the input images were resized and mapped to be 250 × 250. The other DCNN architecture details are described in Figure 7. The proposed neural network was trained using stochastic gradient descent with momentum (SGDM) optimizer built-in Matlab 2019a software (Mathworks, Natick, MA, USA), the mini-batch size assigned to be 128.

Results and Discussion
To validate and test the proposed DCNN model, a 10-fold cross-validation approach was used. The training and validation process of the proposed model is presented in Figure 8.
As shown in Figure 8, the model-which is designed to be validated every 428 iterations to ensure that the network was not overfitting-starts to fit the data early in the process. The final root mean square error (RMSE) in the training process is approximately 1.39.
To measure the prediction accuracy of the proposed DCNN, variable α was set to represent the gap between the predicted and the actual values. α is assumed to be an acceptable margin of error; for example, if α is set at 0.5 and the actual value of one sample is 1.7, this prediction would be accepted as correct if the predicted value was in the range 1.7 ± 0.5. In this work, the prediction accuracy is measured using various α values. The prediction accuracy and its corresponding α value are presented in Table 2. The final prediction accuracy was calculated by averaging the accuracy of all folds.

Results and Discussion
To validate and test the proposed DCNN model, a 10-fold cross-validation approach was used. The training and validation process of the proposed model is presented in Figure 8.
As shown in Figure 8, the model-which is designed to be validated every 428 iterations to ensure that the network was not overfitting-starts to fit the data early in the process. The final root mean square error (RMSE) in the training process is approximately 1.39.
To measure the prediction accuracy of the proposed DCNN, variable α was set to represent the gap between the predicted and the actual values. α is assumed to be an acceptable margin of error; for example, if α is set at 0.5 and the actual value of one sample is 1.7, this prediction would be accepted as correct if the predicted value was in the range 1.7 ± 0.5. In this work, the prediction accuracy is measured using various α values. The prediction accuracy and its corresponding α value are presented in Table 2. The final prediction accuracy was calculated by averaging the accuracy of all folds.   As can be seen in Figure 9 and Table 2, the prediction accuracy starts to be acceptable (accuracy >50%) when α is assigned to be greater than 0.8 of the tested α values. Moreover, if it is recalled that the minimum compressive strength of the CSRE samples was 3.41 MPa and the maximum was 13.97 MPa, then α = 2 might be considered as an acceptable error threshold. Then a reasonable prediction accuracy of 84% is achieved. The accuracy of the prediction is proportional to the margin of error; however, the margin of error cannot be significantly increased to claim a higher prediction accuracy. Therefore, other measures such as the RMSE can be used to acquire a better understanding of the prediction system's performance [37].  As can be seen in Figure 9 and Table 2, the prediction accuracy starts to be acceptable (accuracy >50%) when α is assigned to be greater than 0.8 of the tested α values. Moreover, if it is recalled that the minimum compressive strength of the CSRE samples was 3.41 MPa and the maximum was 13.97 MPa, then α = 2 might be considered as an acceptable error threshold. Then a reasonable prediction accuracy of 84% is achieved. The accuracy of the prediction is proportional to the margin of error; however, the margin of error cannot be significantly increased to claim a higher prediction accuracy. Therefore, other measures such as the RMSE can be used to acquire a better understanding of the prediction system's performance [37]. Furthermore, to provide more information about the predictions of the proposed deep learning model, the covariance and correlation coefficient are calculated as results of the experiments. Table 3 demonstrates the correlation coefficient and the covariance for each fold separately.
As it can be observed in Table 3, the correlation is close to 0.9, which statistically means that the predictions and the actual values have a very strong relationship between them. Moreover, the covariance shows that these sets of data have a positive relationship. Furthermore, to provide more information about the predictions of the proposed deep learning model, the covariance and correlation coefficient are calculated as results of the experiments. Table 3 demonstrates the correlation coefficient and the covariance for each fold separately.
As it can be observed in Table 3, the correlation is close to 0.9, which statistically means that the predictions and the actual values have a very strong relationship between them. Moreover, the covariance shows that these sets of data have a positive relationship. In addition to the prediction, which is based on regression, the proposed DCNN can offer deep features that can be used by another regression method to obtain the prediction results. Two more regression methods are used: the random forest and linear regression features. In the case of calculating the prediction accuracy of the DCNN, the RMSE is one of the important measures that is usually used to measure the performance of a prediction system [37]. The number of deep features obtained by the proposed DCNN was quite large (4096 features); therefore, a principal component analysis (PCA) was used to reduce the number of features [25,38,39], keeping 95% of the data variance to attain only 572 features. The average RMSE values (over 10 folds) of predicting the CSRE samples' compressive strength achieved by each method are presented in Table 4. It can be noticed (based on the results presented in Table 4) that the proposed DCNN with a regression model based on an artificial neural network, applied on all the obtained features (4096), outperforms the other regression models when compared. This is due to the use of all the deep features obtained from the DCNN, unlike the PCA, which reduces the quantity of the features. However, using a large number of features it is a time-consuming process.
To compare the proposed DCNN prediction model with other state-of-the-art traditional computer vision methods, the selection of the features from the SEM images was made with the use of the following algorithms: histogram of oriented gradients (HOG) [40], local binary pattern (LBP) [41], and scale-invariant feature transform (SIFT) [42]. The values of RMSE calculated for the compressive strength (of CSRE samples) predictions made with the use of these traditional features extraction methods are presented in Table 5. It can be noticed (see Table 5) that none of the traditional methods could handle the prediction better than the proposed DCNN. This may be attributed to the representative power of the deep features obtained by the proposed model compared to that of the handcrafted low-level features (HOG, LBP, and SIFT). A similar effect is achieved by many researchers, e.g. in [43][44][45].

Conclusions
In addition to creating a new in-house SEM image database to be used for evaluating the model proposed in this paper, we provide a new deep learning model for predicting the compressive strength of CSRE based on SEM images using computer vision and deep learning. The comparative results of the experiments conducted on the SEM image database to predict the compressive strength of CSRE samples show that the proposed DCNN model outperforms other traditional computer vision methods, obtaining reasonable prediction results and comparatively low RMSE. Therefore, the method can be recommended for practical use as time, and cost-saving tool, as well as the environment protecting one, if applied instead of the current strength-testing machines.
Despite the reasonable performance of the proposed DCNN, the overall prediction accuracy process is still imperfect because it is the error threshold dependent. Such imperfect performance may be attributed to the number of SEM images used to train the deep features. This deficit can be alleviated by obtaining more SEM images of a higher number of CSRE samples to feed the DCNN. Higher prediction accuracy is expected then. Moreover, using the image enhancement techniques at a preprocessing stage may enhance the prediction results. It is considered to verify these statements in future analysis. Prospective future works will also include the use of DCNN to solve other problems described in [46][47][48]. It is expected to overcome the problem of prediction speed. It should increase with the use of more efficient indexing techniques, such as presented in [49][50][51].