A Thermal Sensor-Based Decision Support System for the Identification of Roof Leaks and Cracks ^†

Abstract

The leaks in roofs and cracks in walls of buildings are common and need immediate attention. The roof leaks or cracks lead to water seepage which results in structural damage to the ceiling wall. In this work, the roof leaks or cracks are identified using the proposed thermal sensor-based decision support system. Further, the thermal camera is interfaced with a handy single on-board computer. The supervised machine learning algorithm is coded inside the single on-board computer and the thermal images captured using the thermal camera is utilized for the fault identification. Further, the trained network is tested using a new set of thermal images for the identification of faults. Results demonstrate that the proposed system is efficient in locating and the identification of faults. Since the single on-board has an inbuilt Wi-Fi, the decision support can be stored in the cloud server with a specific unique Uniform Resource Locator (URL) address. Also, by accessing the appropriate URL, the decision support system can be accessed from remote locations.

1. Introduction

Recent years, the majority of the domestic and commercial buildings in developing countries like India, Pakistan, Bangladesh, etc. are constructed with cement material. Due to the climatic changes, the cracks and leaks in the roofs/walls are very common since the walls are subjected to several seasons. So, it is mandatory to identify the leaks and cracks of walls at the initial stage to minimize severe effects. The early detection of leaks and cracks avoids various drawbacks such as structural damage, repair cost etc.

The advancement in technologies leads to the design of various paint materials which covers the outer layer of walls. In many cases, the paint coating hides the cracks of walls which occurred internally and leads to leaks at later stages. These internal cracks cannot be visualized by normal sights lead for the development of some assistance systems.

Machine Learning is a subfield of Artificial Intelligence and it is concerned with the development of methods which enables the computer to learn. Nowadays, the machine learning algorithms are widely used in many fields such as automatic flight control, vehicle maneuvering, surveillance and security etc. [1]. An automated decision support system has increased significantly over the past few years and algorithms such as Naive Bayes classifier, support vector machines, artificial neural networks, etc., are widely used techniques in approaching the decision support problem [2].

Maniruzzaman et al., (2017) [3] have adapted Gaussian Process based classification technique using three kernels such as linear, polynomial, and radial basis kernel to classify diabetes. Further, the authors have investigated the performance of a GP-based classification technique in comparison to existing techniques such as Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), and Naïve Bayes (NB). Nikam (2015) [4] have discussed the various classification algorithms such as K-Nearest Neighbor, Naïve Bayes, SVM, and ANN along with their features and limitations.

The objective of this work is to develop an automated decision support system to identify the leaks and cracks in walls of buildings.

2. Materials and Methods

2.1. Proposed Decision Support System

In this work, the AMG8833 type thermal camera module is utilized for thermal image acquisition which is connected to single on-board computer to identify leaks and cracks in walls. Further, the Raspberry PI 3 Model B+ is used as a single on-board computer. Figure 1 shows the block diagram of a proposed decision support system to identify leaks and cracks in walls. The images acquired using thermal camera module is given to the Raspberry PI type single on-board computer. Further, the informative features are extracted using two different feature extraction techniques, namely Image Entropy and Gray level Co-occurrence Matrix (GLCM) Energy. Also, the extracted features are given as an input to supervised machine learning algorithm, namely Support Vector Machine (SVM) with three different kernels such as Linear, Gaussian, and Polynomial is utilized as a classifier, which is coded inside the single on-board computer. The output of the SVM classifier with three different kernels are analyzed and the classifier with accuracy and sensitivity is utilized for decision support to identify leaks and cracks in walls. Since the raspberry PI has inbuilt WiFi module, the decision support is uploaded to cloud server. Therefore, the inspector can access the decision support remotely by using appropriate website URL. Also, the python software is utilized to code the SVM algorithms with three different kernels inside single on-board computer.

2.2. Feature Extraction Techniques

Image entropy is defined as a scalar value which represents the entropy of grayscale image I and is a measure of the disorder or randomness that can be used to characterize the texture of the input image [5,6]. Images with lesser entropy have lot of black regions, less contrast, and a large number of pixels. Image entropy is expressed by an equation [7]:

$$\mathit{IM}=-\sum _i{P}_i{\mathit{log}}_2{P}_i$$

(1)

where P_i is the probability that the difference between two adjacent pixels is equal to i, and log₂ is the base 2 logarithm [7]. The GLCM Energy is a sum of squared elements in the GLCM matrix [8].

$$E=\sum _{i,j}p{(i,j)}^2$$

(2)

2.3. Support Vector Machine Classifier

Boser et al. [9] (1992) have proposed a classification algorithm called a support vector machine based on statistical learning theory. Since the proposed research work is a two-class binary classification problem, the adopted SVM classifier finds the most effective hyperplane, which is a classification line and divides all input samples into two classes. The input samples nearest to the classification line are called support vectors. The training set of the SVM classifier is expressed as [10]:

$$X=\left\{\right(y^1,z^1),(y^2,z^2),\dots ..,(x^n,y^n)\},y\in S^n,z\in \{-1,1\}$$

(3)

The classification line is derived as:

$$X(n)=(w.n)+a$$

(4)

where n is a sample vector represented to a high dimensional space, z is the class label of X. Further, the SVM classifier estimates and obtains the equation of classification line with the help of w and a parameters. Also, the SVM classifier has different kernel functions such as Linear, Gaussian, Polynomial, etc. which can be assigned for the decision function. The SVM model can be developed by representing the input vectors onto a new feature space. The kernel functions are used to construct optimal hyperplane in the new high dimensional feature space. The Linear SVM (LSVM) maps the non-linear input space into the new linearly separable space and it can be expressed as:

$$K(x_i,x_j)=\mathsf{\Phi }\left(x_i\right).\mathsf{\Phi }\left(x_j\right)$$

(5)

where x_i and x_j are the input feature vectors. Further, the optimal separating hyperplane is constructed and all the vectors lying on one side of the hyperplane are labeled as −1, and all vectors lying on another side are labeled as +1 [11,12].

In this work, the SVM based supervised machine learning classifier with three different kernels are trained using 100 train images with leaks and cracks of walls. After the training process, the 50 test images with leaks and cracks of walls are utilized to find the performance of the adopted classifier. The two different performance parameters, namely accuracy and sensitivity, are calculated for SVM classifier with three different kernels such as Linear, Gaussian, and Polynomial.

$$\mathit{Accuracy}=(\mathit{TP}+\mathit{TN})/(\mathit{TP}+\mathit{FP}+\mathit{TN}+\mathit{FN})$$

(6)

$$\mathit{Sensitivity}=\mathit{TP}/(\mathit{TP}+\mathit{FN})$$

(7)

3. Results and Discussion

Figure 2a,b show typical image without leaks acquired from both the PI camera and the thermal camera module, respectively. It is seen that the two different cameras were utilized to acquire images in a same location. Further, the acquired thermal image using the thermal camera module was taken for further processes. Also, the Figure 3a,b show typical image with leaks acquired from both the PI camera and the thermal camera module, respectively. The 100 train images with and without leaks were acquired with the thermal camera module at different locations were utilized to train SVM classifiers. Also, 50 test images were acquired using the thermal camera module, in which 25 images were with leaks and 25 images were without leaks.

Figure 4 shows the performance measures of SVM classifier with three different kernels such as Linear, Gaussian, and Polynomial (order = 8). It is seen that the accuracy and sensitivity of SVM classifier with polynomial kernel is higher when compared to the other SVM kernels. Also, it is observed that the SVM classifier with Gaussian kernel has very good sensitivity (91.6%), almost similar to the SVM classifier with Polynomial kernel. Therefore, the SVM classifier with Polynomial kernel is utilized to design an automated decision support system since it has good performance measures when compare to other SVM classifiers. The performance measures, namely accuracy and sensitivity of SVM classifier with three different kernels are presented in

Table 1. Performance Measures of SVM Classifier with three different kernels.

Performance Measures	Accuracy (%)	Sensitivity (%)
Linear SVM Classifier	78	77
Gaussian SVM Classifier	90	91.6
Polynomial SVM Classifier (order = 8)	94	92

. It is seen that the accuracy and sensitivity of the adopted SVM classifier with polynomial kernel (order = 8) is 94% and 92%, respectively. It is also observed that the performance of an SVM classifier can be improved by increasing various train images.

Figure 5 shows the front-end display for automated decision support system to identify cracks in walls. Further, the webpage is designed using Hypertext Markup language (HTML) code and the remote open source cloud server namely localtunnel.me is utilized to store decision output universally.

4. Conclusions

In this work, a decision support system was developed to the identify leaks and cracks in walls of buildings. The images of walls with and without leaks/cracks were acquired using the thermal camera module. Further, the informative features namely Image Entropy and GLCM Energy was extracted from the acquired thermal images and was utilized to develop efficient classification systems. The supervised learning-based Support Vector Machine (SVM) with three different kernels such as Linear, Gaussian, and Polynomial (order = 8) were used for the classification of images with and without leaks/cracks. Results demonstrate that the SVM classifier with Polynomial kernel is good when compared to SVM with Linear and Gaussian kernels. It is observed that the accuracy and sensitivity of SVM classifier with Polynomial kernel is 94% and 92%, respectively. Therefore, the SVM classifier with Polynomial kernel was adopted as decision support system for the classification of leaks/cracks in walls of buildings.