An Efficient Fault Detection Method for Induction Motors Using Thermal Imaging and Machine Vision

Abstract

Induction motors (IMs) are the backbone of industry, and play a vital role in daily life as well. However, induction motors face various faults during their operation, which may cause overheating, energy losses, and failure in the motors. Keeping in mind the severity of the issues associated with fault occurrence, this paper proposes a novel method of fault detection in induction motors by using “Machine Vision (MV)” along with “Infrared Thermography (IRT)”. It is worth mentioning that the timely prevention of faults in the IM ensures the motor’s safety from failures, and provides longer service life. In this work, a dataset of thermal images of an induction motor under different conditions (i.e., normal operation, overloaded, and fault) was developed using an infrared camera without disturbing the working condition of the motor. Then, the extracted thermal images were effectively used for the feature extraction and training by local octa pattern (LOP) and support-vector machine (SVM) classifiers, respectively. In order to enhance the quality of feature extraction from images, the LOP was implemented along with a genetic algorithm (GA). Finally, the proposed methodology was implemented and validated by detecting the faults introduced in an induction motor in real time. In addition to that, a comparative study of the suggested methodology with existing methods also verified the supremacy and effectiveness of the proposed method in comparison to the previous techniques.

1. Introduction

Induction motors are known as industrial workhorses in the present era, as they are the most popular rotating machines being used in industry [1]. Induction motors play a pivotal role in many industrial applications, such as the oil and gas industry, manufacturing plants, refining and milling plants, and many more. The usage of induction motors in various industrial applications also demands the smooth and fault-free operation of the motors under the harsh conditions of the industrial environment. Fault occurrence in the motors may lead to reduction in output, loss of revenue, and energy losses, ultimately posing a great challenge in terms of smooth industrial operations. Moreover, it should be noted that the continuous usage of faulty equipment (such as induction motors) within the system can severely affect the performance of other electrically connected equipment. Therefore, timely fault diagnosis in electrical motors and other electronic equipment is of utmost importance for safety, long life, and high efficiency. According to a previous study [2], the effective implementation of fault prediction in industry for electronic equipment can reduce the cost of operations by 33–50%, which was neglected by most production and generation centers for a long time. It was found that predictive maintenance has several advantages, such as reduction in unscheduled system shutdowns, diminution of the required manpower for maintenance, and improvement of the equipment’s service life. Overall, it can be deduced from the above that timely fault diagnosis and condition monitoring play an important role in the smooth and steady operation of induction motors and industrial operations.

Theoretically, IMs can experience different types of faults, including both electrical and mechanical faults [3]. The electrical faults can be further divided into stator and rotor faults. The stator faults include short-circuiting in the stator winding, damage to the core, core losses (e.g., hysteresis and eddy current losses), or defects in the frame of the stator [4]. Meanwhile, the rotor faults include short-circuiting in the rotor winding, eccentricity in the rotor, and copper losses [3,5,6]. On the other hand, the mechanical faults include damage to the bearings and to the shaft of the motor [7]. However, it is worth noting that in the induction motors, approximately 36% of the faults are due to stator winding faults [4], because the stator parts undergo extreme stress (whether electrical or mechanical stress). Usually, the electrical faults in the stator winding are phase-to-phase faults, phase-to-ground faults, and inter-turn faults [8,9,10]. Furthermore, as time passes, the IM performance also deteriorates due to corrosion, faults in wiring, overloading, unbalanced loading, and usage of poor materials in construction [11], which also enhance the chances of fault occurrence.

In addition to the aforementioned problems, the faults also cause a rise in the temperature of the IM beyond the allowable temperature limit of the motors, ultimately affecting the service life of the IM drastically and reducing its service life by half for every 10 °C rise in temperature. Fortunately, this temperature rise due to faults plays a crucial role in the fault detection, because the increase in the thermal energy due to faults can be effectively detected in real time by the thermal imaging of the machine. Normally, all electrical equipment with a temperature above 0 °C emits infrared (IR) radiation, because the current passing through the equipment causes an increase in the heat signature of the equipment. However, the naked human eye cannot detect the heat signature of the object, due to the fact that the heat waves from the heated objects are in the infrared range. Therefore, IRT can be used to capture the heat signature of the heated objects in order to visualize the heat energy distribution of the objects. In recent years, IRT has become an important tool for detecting hot spots and defects in the electrical equipment due to its simplicity and effectiveness. Several previous studies have shown that IRT has a variety of applications and can be a useful technique for fault detection by capturing the heat signature of the electrical equipment under operation [12,13,14,15,16,17,18,19,20,21]. Thus, thermography can identify fault occurrence in real time, and can protect the equipment and system from various problems occurring due to faults. Currently, run-to-failure (RTF) maintenance is the most widely used method for the life extension and protection against faults of IMs in industry. Meanwhile, in infrared thermography (IRT), the detection of thermal defects of the object is achieved by the inspection of its temperature change (∆T), and this procedure is called qualitative-based estimation of temperature [22,23,24]. Thus, IRT enables the timely detection of the faults in the equipment, and is more efficient compared to RTF.

Previously, various artificial intelligence (AI) techniques—such as fuzzy logic [25] and artificial neural networks (ANNs)—have been implemented along with IRT for fault detection in motors, due to their ability to execute complex tasks without significant cost outlays. Artificial intelligence has numerous applications in industry [26,27,28], power systems, and medicine. For example, a few works [24,29] have reported the inclusion of artificial neural network (ANN) techniques along with IRT in order to classify the defects in different materials, because the ANN is a known strong and powerful mathematical model for pattern recognition models. In these works, the different thermal images from the machine were taken as inputs for ANNs to classify the faults in the electrical hardware. In another work [30], the neuro-fuzzy approach was explored to detect the faults in the machines, while the inputs to this system were thermal images of the machine. However, in the abovementioned works [24,29,30], system performance of more than 10% could not be obtained, making them unsuitable for effective fault detection in the objects. Despite the availability of these AI-based fault-diagnosis techniques, there is still a chance of failure in the induction motor due to their low effectiveness, cost factor, and increased time requirements for data collection and analysis.

Therefore, in view of the above constraints, machine learning can be a suitable choice for the fault detection of industrial loads, if used along IRT. It has been reported in some recent works that machine learning can handle the challenge of fault detection efficiently [31,32,33]. Machine vision for fault detection mainly consists of feature extraction and training classifiers. There are multiple options for feature extractors and classifiers for the implementation of machine learning. For instance, in [34], the author obtained 93.38% of the accurate data for a three-layer ANN classifier, and the RBG color scheme was explored for scaling and to identify the internal temperature of the machine. Furthermore, additional reports on the support-vector machine (SVM) and the genetic structure of ANN multilayer perception (MLP) in recent works [35,36] on the fault diagnosis of electrical equipment showed accuracy levels of 83% and 79.78%, respectively. The total dataset for SVM was 20 pictures with a feature sample of Zernike moments, while the MLP was able to show further improvement in results with fewer figures. For the purpose of feature extraction in machine vision techniques, the local octa patterns (LOPs) can also be used, due to their special property of “four derivative directions”. The LOP technique can be a suitable candidate for the fault detection in industrial loads if used along with the IRT, due to its excellent feature-extraction ability. It is essentially a feature-extraction algorithm, and is an extension of the local tetra pattern (LTP) [37] algorithm. It processes an image by considering the diagonal, vertical, and horizontal derivative directions of the image pixels to represent a region (four derivative directions), whereas the previously reported methods—such as “LTP”—take into account only the horizontal and vertical directions (two directions) [38]. Thus, it may be concluded from the above that the inclusion of the diagonal direction in LOP makes the feature more descriptive, and the more descriptive the feature, the better the retrieval result. As the retrieval result gets better, the detection accuracy of the algorithm in the fault detection cases also increases. Finally, it can be concluded from the whole discussion that IRT can be effectively used along with machine vision techniques for the predictive maintenance of equipment without complete shutdown of industry, and using this technique can reduce manpower and several other wastes of resources.

In this paper, the IRT method was used along with machine vision for the timely diagnosis of IM faults in order to ensure safe and smooth operation of the IM. A dataset of thermal images of IM was generated under normal, overload, and fault conditions, and then this dataset was trained using machine vision for the diagnosis of faults. There are three stages in the proposed diagnostic method—(i) data acquisition, (ii) feature extraction and training, and (iii) fault identification—but among these stages the feature extraction and training is the most important stage. In this work, the LOP method was used along with GA for the feature extraction, because this provides better image description, as mentioned above. For the data training, the SVM classifier trains the data based on samples as received from the LOP. Finally, the methodology was validated in real time by introducing and detecting faults in an IM. The system effectively recognizes different fault patterns of the induction motor, and informs the user about fault occurrence after identification. Overall, this work presents a novel technique for fault diagnosis in IMs by using IRT and machine vision simultaneously, and provides a decent precision rate for the diagnosis of faults.

2. Experimental Setup

The experimental setup of the suggested method can be divided into three stages of data acquisition, feature extraction, and training. The detailed explained for each section is provided below.

2.1. Data Acquisition

The very first step in this method is to capture the thermal images (data acquisition stage) of the IM under various (normal, overload, and faulty) conditions to generate the database of thermal images. For this purpose, a digital infrared camera was used to capture the thermograms of the IM by keeping the camera in line with the IM at 1–2 feet distance. The complete specifications of the three-phase IM (equipped with a variable-frequency drive) used in this experiment are provided in

Table 1. The specifications of the three-phase IM.

No.	Parameter	Specification (Value)
1	Rated Power	2 kW
2	Rated Frequency	50 Hz
3	Stator Resistance	1.47 Ω
4	Rotor Resistance	1.39 Ω
5	Magnetizing Reactance	54.1 Ω
6	Rated Speed	1410 RPM
7	Efficiency	87%

. The corresponding experimental setup (with different view angles) for capturing the thermal images of the IM is provided in Figure 1. The bottom-right image provides a closer view of the IR camera interfaced with a mobile phone for capture purposes. The IR camera was kept at a certain distance from the IM (connected with three-phase supply), and the images were captured of the IM under various conditions. In order to develop the database of thermograms, the thermal images of the IM were captured for each condition, i.e., normal, overload, and fault. The sample thermal images defining the different heat signatures of the IM under the three mentioned conditions can be seen in Figure 2. In this work, the state of the motor is divided into three different levels based on the temperature variation (ΔT), as shown in

Table 2. The state of the IM based on ΔT.

No.	State of Motor	ΔT	Recommended Action
1	Normal Operating Condition	30 < T < 45	No Need for Action
2	Overloaded Condition	45 < T < 63	Wait for Some Time
3	Fault Condition	T > 65	Shut Down

. The variable ΔT is the temperature difference between the ambient temperature and the hotspot temperature of the IM. Moreover, the recommended outputs in this proposed work are also dependent on the parameter ΔT. It is evident from the table that no action is required under normal operating conditions, while under overload conditions it is necessary to wait for some time, and the system should be shut down completely under fault conditions. It should be noted that the overloaded condition can severely damage the motor if it persists for longer periods; therefore, the wait time for the overload condition should be chosen accordingly.

The complete flowchart of the fault detection is shown in Figure 3. Figure 3 shows that after the initial data acquisition, the next stage is “feature extraction” of thermal images, and this stage plays a key role in the development of the fault detection and diagnosis method [37,39,40]. The hundreds of thermal images (dataset) captured in the first stage are used as the input for the feature-extraction stage. After the initial processing of these sample images, features are extracted from the images using the LOP technique. In the next stage, the data are forwarded to the SVM classifier for the training of the data. After the completion of the training cycle, the proposed method can be used effectively and accurately for fault detection of the IM by taking the IR thermal images of the IM as inputs and correlating them with the trained data.

2.2. Feature Extraction through LOP

The data obtained through IRT are forwarded to the feature extractor in order to perform feature extraction on the dataset of thermal images. The mechanism follows Equations (1)–(18) in order to perform feature extraction on an image using local octa patterns. For the feature extraction of thermal images, the LOPs are used due to their effectiveness and simplicity. The thermal images are extracted in the rectangular shape for better feature extraction; however, the image retrieval performance of content-based image retrieval (CBIR) is still very low because of the huge number of semantic concepts available and the smaller training sample in the class of interest. It is worth noting that the feature extraction by the local octa pattern has horizontal, vertical, and diagonal derivative directions, along with a uniform pattern of different diagonals to describe the relevant regions of images. The nth-order LOP descriptor computes the (n − 1)thorder derivative at $0^{°}, {45}^{°}, {90}^{°},$and ${135}^{°}$ angles to represent eight distinct values at the central gray pixel (g_c), as mentioned in Equation (1). This equation is used to compute the direction of the central pixel (i.e., referenced pixel) in an image K. There could be 8 possible directions and, based on some conditions, the direction of the central gray pixel (g_c) of image K is computed. For instance, if the (n − 1)th order derivative of g_c in the 0° angle direction (i.e., horizontal direction) is greater than or equal to zero, the(n − 1)th-order derivative of g_c in the 900°angle direction (i.e., vertical direction) is also greater than or equal to zero, and the (n − 1)th-order derivative of g_c in the 45° and 135° angle directions (i.e., diagonal directions) is also greater than or equal to zero, then the direction of the central gray pixel (g_c) of image K is 1.

$${\mathrm{K}}_{\mathrm{D}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)=\{\begin{array}{ll}1,& {\mathrm{K}}_{0^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\ge 0\wedge {\mathrm{K}}_{{90}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\ge 0\wedge \left({\mathrm{K}}_{{45}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\right)\vee {\mathrm{K}}_{{135}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\ge 0\\ 2,& {\mathrm{K}}_{0^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)<0\wedge {\mathrm{K}}_{{90}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\ge 0\wedge \left({\mathrm{K}}_{{45}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\right)\vee {\mathrm{K}}_{{135}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\ge 0\\ 3,& {\mathrm{K}}_{0^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)<0\wedge {\mathrm{K}}_{{90}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)<0\wedge \left({\mathrm{K}}_{{45}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\right)\vee {\mathrm{K}}_{{135}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\ge 0\\ 4,& {\mathrm{K}}_{0^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\ge 0\wedge {\mathrm{K}}_{{90}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)<0\wedge \left({\mathrm{K}}_{{45}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\right)\vee {\mathrm{K}}_{{135}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\ge 0\\ 5,& {\mathrm{K}}_{0^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\ge 0\wedge {\mathrm{K}}_{{90}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\ge 0\wedge \left({\mathrm{K}}_{{45}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\right)\vee {\mathrm{K}}_{{135}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)<0\\ 6,& {\mathrm{K}}_{0^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)<0\wedge {\mathrm{K}}_{{90}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\ge 0\wedge \left({\mathrm{K}}_{{45}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\right)\vee {\mathrm{K}}_{{135}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\ge 0\\ 7,& {\mathrm{K}}_{0^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)<0\wedge {\mathrm{K}}_{{90}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)<0\wedge \left({\mathrm{K}}_{{45}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\right)\vee {\mathrm{K}}_{{135}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)<0\\ 8,& {\mathrm{K}}_{0^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\ge 0\wedge {\mathrm{K}}_{{90}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)<0\wedge \left({\mathrm{K}}_{{45}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\right)\vee {\mathrm{K}}_{{135}^{°}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)<0\\ & \end{array}\}$$

(1)

Equation (2) is then used to calculate the local octa pattern (8-digit pattern) of the central gray pixel (g_c) in the direction computed from Equation (1). Equation (3) elaborates the function f₇ used in Equation (2). Using this function f₇, the 8-digit local octa pattern is computed.

$${\mathrm{TLOP}}_{\mathrm{P}}^{\mathrm{n}}\left({\mathrm{g}}_{\mathrm{c}}\right)=\left\{{\mathrm{f}}_7\left({{\mathrm{K}}^{\prime }}_{\mathrm{D}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{P}}\right),{{\mathrm{K}}^{\prime }}_{\mathrm{D}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\right)\right\}|\mathrm{p}=1,2,\dots ,\mathrm{P}$$

(2)

where

$${\mathrm{f}}_7\left({{\mathrm{K}}^{\prime }}_{\mathrm{D}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{p}}\right),{{\mathrm{K}}^{\prime }}_{\mathrm{D}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\right)=\{{{\mathrm{K}}^{\prime }}_{\mathrm{D}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{p}}\right),{\mathrm{if}\hspace{0.17em}{\mathrm{K}}^{\prime }}_{\mathrm{D}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{p}}\right)\ne {{\mathrm{K}}^{\prime }}_{\mathrm{Dir}.}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right);0,\mathrm{else}$$

(3)

It should be noted that an 8-digit local octa pattern can be further divided into 7 binary patterns; Equations (4)–(6) elaborate this division into 7 binary patterns. Keeping in mind the direction of the central pixel, the T-LOP code from Equation (2) is considered, and is further divided into 7 binary patterns, as shown in Equations (4)–(6).

$$\begin{array}{l}{\mathrm{TLOP}}_{\mathrm{P}}^{\mathrm{n}}{|}_{\{\overline{\mathrm{D}}\hspace{0.17em}|\forall \mathrm{D},\exists \hspace{0.17em}\neg \hspace{0.17em}{\mathrm{K}}^{\prime }{}_{\mathrm{D}.}^{\mathrm{n}-1}({\mathrm{g}}_{\mathrm{c}})\}}\hspace{0.17em}=\hspace{0.17em}\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\mathrm{f}}_8({\mathrm{TLOP}}_{\mathrm{P}}^{\mathrm{n}}({\mathrm{g}}_{\mathrm{c}})){|}_{\{\hspace{0.17em}\overline{\mathrm{D}}\hspace{0.17em}|\forall \mathrm{D},\hspace{0.17em}\exists \hspace{0.17em}\neg \hspace{0.17em}{\mathrm{K}}^{\prime }{}_{\mathrm{D}.}^{\mathrm{n}-1}({\mathrm{g}}_{\mathrm{c}})\}}\end{array}$$

(4)

$${\mathrm{f}}_8{(\mathrm{TLOP}}_{\mathrm{P}}^{\mathrm{n}}{(\mathrm{g}}_{\mathrm{c}}{\left)\right)|}_{\overrightarrow{\mathrm{D}}\in \hspace{0.17em}\hspace{0.17em}\overline{\mathrm{D}}\hspace{0.17em}}\hspace{0.17em}=\hspace{0.17em}{\{\begin{array}{l}1,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\mathrm{if}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\mathrm{TLOP}}_{\mathrm{P}}^{\mathrm{n}}{(\mathrm{g}}_{\mathrm{c}})=\overrightarrow{\mathrm{D}}\\ 0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\mathrm{else}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\end{array}}_{ }$$

(5)

where $\overline{\mathrm{D}}$ is the set of all quadrants except the quadrant of the referenced pixel, and $\overline{\mathrm{D}}$ is one of the quadrants of $\overline{\mathrm{D}}$. Afterwards, the T-LOP code can be generated as follows:

$${\mathrm{TLOP}}_{\mathrm{P}}^{\mathrm{n}}{|}_{\{\overline{\mathrm{D}}|\forall \mathrm{D},\exists \neg {\mathrm{K}}_{\mathrm{D}.}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\}}={\displaystyle \sum }_{\mathrm{p}=1}^{\mathrm{P}}{2}^{\left(\mathrm{p}-1\right)}{ \ast \mathrm{f}}_8\left({\mathrm{TLOP}}_{\mathrm{P}}^{\mathrm{n}}\left({\mathrm{g}}_{\mathrm{c}}\right)\right){|}_{\left\{\overline{\mathrm{D}}\right|\forall \mathrm{D},\exists \neg {\mathrm{K}}_{\mathrm{D}}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{c}}\right)\}}$$

(6)

Then, an eighth binary pattern known as the magnitude pattern (MP) can be computed using Equations (7)–(9).

$$\mathrm{MP}={\displaystyle \sum _{\mathrm{p}=1}^{\mathrm{P}}{2}^{\mathrm{n}-1}}{ \ast \mathrm{f}}_9\left({\mathrm{M}}_{{\mathrm{K}(\mathrm{g}}_{\mathrm{p}})}-{\mathrm{M}}_{{\mathrm{K}(\mathrm{g}}_{\mathrm{c}})}\right)$$

(7)

where

$${\mathrm{M}}_{\mathrm{K}\left({\mathrm{g}}_{\mathrm{p}}\right)}=\sqrt{{{\displaystyle \sum }}^{ }\left(\begin{array}{c}{({{\mathrm{K}}^{\prime }}_{0^0}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{p}}\right))}^2+{({{\mathrm{K}}^{\prime }}_{{45}^0}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{p}}\right))}^2+\\ {({{\mathrm{K}}^{\prime }}_{{90}^0}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{p}}\right))}^2+{({{\mathrm{K}}^{\prime }}_{{135}^0}^{\mathrm{n}-1}\left({\mathrm{g}}_{\mathrm{p}}\right))}^2\end{array}\right)}$$

(8)

$${\mathrm{f}}_9(\mathrm{x})=\left\{\begin{array}{l}1,{{\displaystyle }}^{}\mathrm{x}\ge 0\\ 0,{{\displaystyle }}^{}\mathrm{else}\end{array}\right\}$$

(9)

The local octa pattern of a central pixel (represented in blue) for the specific segment of an image with neighbors (shown in orange) can be understood from Figure 4. A detailed pictorial description is shown in Figure 4, where the direction of a central gray pixel (g_c) in an image window is first computed (using Equation (1)). Then, based on this direction, the local octa pattern of the central pixel is calculated (using Equations (2)–(3)). Afterwards, using Equations (4)–(6), seven binary patterns are calculated, and the magnitude pattern is computed using Equations (7)–(9). This shows that D(c) is the direction of the center, whereas D(1) and D(2) represent the directions of the 1st and 2nd neighboring pixels, respectively. Similarly, M(c) represents the magnitude at the center, while M(1) and M(2) represent the magnitude at the 1st and 2nd neighbors, respectively. The T-LOP bit is considered to be zero if the direction of the center pixel is the same as that of a neighboring pixel. The T-LOP obtained in this case was 31205116. The seven binary patterns can be obtained if this pattern is further subdivided, by replacing 1 with 1 and all other values with zero for the first pattern, by replacing 2 with 1 and all other values to zero for the second pattern, and so on. The magnitude pattern value obtained by comparing the magnitude of the central pixel with that of the neighboring pixels is 11010101. These patterns were used to extract the image texture, and better texture representation and information extraction can be achieved using higher-order T-LOP. However, the second-order pattern also gives the best results for the thermal images used in this study on faults. In the pictorial explanation, nearest neighbors are considered only for pattern calculations; however, neighbors at diagonals 5 and 7 can also be used in T-LOP.

The neighbors at different diagonal levels are shown in Figure 5, where the center, third diagonal, fifth diagonal, and seventh diagonal are represented by brown, blue, pink, and yellow, respectively. Meanwhile, the vertical and horizontal pixels of a yellow central pixel are represented in green. The feature vector for the experiments was designed using a reduced concatenating histogram of each pattern, where each pattern histogram carries 20 bin-making feature vectors of 160 in length. The class misbalancing is a common phenomenon that occurs due to the smaller number of required samples in the concerned class in comparison to neighboring classes. The misbalancing can cause image retrieval issues as well. The arrays shown in Figure 3 and Figure 4 are actually windows of pixels taken from an RGB image in order to explain the local octa pattern algorithm.

In this study, the LOP’s main concern was to extract the elaborated image vectors. For this purpose, class misbalancing can be reduced by using a higher number of enhanced samples for training, resulting in enhanced precision with reduced misbalancing. If the $\left|B\right|$ images are taken from the “n” number of classes, then the whole image dataset can be divided into $\mathrm{S}=\left\{{\mathrm{S}}_1 ,{ \mathrm{S}}_2,{\mathrm{S}}_3\dots \dots \dots {\mathrm{S}}_{\mathrm{n}}\right\}$ subsets, where each subset has ${\left|\mathrm{B}\right|}_{\mathrm{s}}=\frac{\left|\mathrm{B}\right|}{\mathrm{S}\times 0.3}$ images for each category (where 0.3 is a normalized ratio of images taken for training equivalent to 30%). All of the abovementioned subsets can be further categorized into two categories: positive ${\left|\mathrm{b}\right|}^{+}$ and negative ${\left|\mathrm{b}\right|}^{-}$ samples, as provided in Equations (10) and (11) [41]:

$${\left|\mathrm{b}\right|}^{+}=\left\{\mathrm{j}=1,2,3\dots \dots ,\hspace{1em}\hspace{1em}\hspace{1em}\left|{\mathrm{B}}_{{\mathrm{S}}_{\mathrm{j}}}\right|\left|{\mathrm{K}}_{\mathrm{j}}\in \right|{\mathrm{B}}_{{\mathrm{S}}_{\mathrm{j}}}|\right\}$$

(10)

$${\left|\mathrm{b}\right|}^{-}=\left\{\mathrm{j}=1,2,3\dots \dots \dots ,\hspace{1em}\hspace{1em}\left|{\mathrm{B}}_{\mathrm{S}{-}_{{\mathrm{S}}_{\mathrm{j}}}}\right|\left|{\mathrm{K}}_{\mathrm{j}}\in \right|{\mathrm{B}}_{\mathrm{S}{-}_{{\mathrm{S}}_{\mathrm{j}}}}|\right\}$$

(11)

Equations (10) and (11) correspond to the division of positive and negative training samples, respectively, in order to enhance the precision. A classifier is normally trained on a set of positive samples and a set of negative samples generated by LOP. These equations explain that a subset is further divided into positive and negative samples before training through the classifier to reduce misbalancing.

It is important to have high-quality feature extraction for the fault detection applications in order to ensure better training of the data by the classifier. The misbalancing issue can pose serious challenges, and should be addressed through inclusion of a genetic algorithm. To avoid the classifier being misled due to the misbalancing, different sets—as mentioned in the above equations—can be used for training purposes through the AI-based genetic algorithm. Equations (12)–(18) are relevant to the genetic algorithm (GA). The genetic algorithm is actually used in order to increase the number of positive samples. It must be noted that if the number of training samples is enhanced, the classifier would be better trained, and better retrieval results can be obtained in return. In a genetic algorithm, genes with individual values (alleles) are combined to make chromosomes—also called decision variables—to find problems. The GA, as the name suggests, takes parent samples from a pool of positive samples and then creates offspring. Out of these offspring, only those that satisfy an evaluation test are selected and added to the parent sample pool. Equation (12) defines a chromosome that is actually a parent sample—or in this case is a feature vector of an image in database:

$$\mathsf{\Theta }=[{\mathsf{\theta }}_{\mathrm{c},1}{,\mathsf{\theta }}_{\mathrm{c},2},\dots \dots {\mathsf{\theta }}_{\mathrm{c},\mathrm{G}}],\hspace{1em}\mathrm{c}=\left\{1,2,\dots \dots ,\mathrm{P}\right\}$$

(12)

where G represents the number of genes in a chromosome, and P represents the population size. In this work, the feature vectors of the original images were used as the initial population for the process, and the original images processed using LOP were used as the parent population. For the generation of the offspring population, various genetic operators are usually applied to the chromosomes, such as mutation [42], crossover [43], etc. The crossover technique can be subdivided into 2-point crossover and mid-point crossover. The 2-point crossover was used for the offspring population generation in this study. The new offspring were generated by exchanging the segments of 2 random parents and cutoff points. If Θ₁ and Θ₂ are the chromosomes of the selected parents, Θ_1,off and Θ_2,off are the chromosomes of the newly generated offspring, and p₁ = {p_1,1,p_1,2,…p_1,G}, p₂ = {p_2,1,p_2,2,…p_2,G} represent the two parents respectively, then new offspring can be formed using Equations (13)–(16) (where the crossovers or cutoff points are represented by C).

$$\mathrm{parents}\left(\mathrm{p}\right)\left[1,2\right]=\mathrm{rand}\left(2\right)$$

(13)

$$\mathrm{crossover}\left(\mathrm{C}\right)\left[1,2\right]{=\mathrm{rand}\left(2\right)}_{ }$$

(14)

$${\mathsf{\Theta }}_{1,\mathrm{off}}=[{\mathsf{\theta }}_{\mathrm{p}1,1}\to {\mathsf{\theta }}_{\mathrm{p}1,\mathrm{C}1}{;\mathsf{\theta }}_{\mathrm{p}2,\mathrm{C}1}\to {\mathsf{\theta }}_{\mathrm{p}2,\mathrm{C}2}{;\mathsf{\theta }}_{\mathrm{p}1,\mathrm{C}2}\to {\mathsf{\theta }}_{\mathrm{p}1,\mathrm{G}}{]}_{ }$$

(15)

$${\mathsf{\Theta }}_{2,\mathrm{off}}=[{\mathsf{\theta }}_{\mathrm{p}2,1}\to {\mathsf{\theta }}_{\mathrm{p}2,\mathrm{C}1}{;\mathsf{\theta }}_{\mathrm{p}1,\mathrm{C}1}\to {\mathsf{\theta }}_{\mathrm{p}1,\mathrm{C}2}{;\mathsf{\theta }}_{\mathrm{p}2,\mathrm{C}2}\to {\mathsf{\theta }}_{\mathrm{p}2,\mathrm{G}}{]}_{ }$$

(16)

Equation (13) shows that 2 random parents are selected from the database, and Equation (14) shows that 2 random cutoff points are selected. After selecting 2 parents and 2 cut-off points, these 2 parents then cross over to make 2 new offspring. Equations (15) and (16) show the 2 new offspring as a result of the crossover. If the two parents have the same value—such as p₁ = p₂—then the generation loop of random offspring will keep running until 2 random parents with different values are processed. The whole procedure is then repeated for C₁ = C₂.

Equation (17) essentially shows an evaluation test for the offspring. After generating multiple offspring, only those offspring that are very close to the parents or—in mathematical terminology—have less distance than any of their parents, are selected. After passing the evaluation test, all of the newly generated offspring are ready to be part of the elite parent (e) population. In the next cycle, for the generation of new offspring as well as new iterations, this new elite population is selected.

$${\mathsf{\Theta }}_{\mathrm{e}}=\Vert {\mathsf{\Theta }}_{\mathrm{off}}<\mathsf{\Theta }\Vert \hspace{1em}\mathrm{e}=\left\{\mathrm{p}+1\to \mathrm{P}\right\}$$

(17)

Now, after generating multiple pools of selected offspring, only a specific pool is selected. Equation (18) shows that only the pool of offspring with the minimum average distance compared to the average distance of the original positive training sample pool is selected. In the current scenario, a population size of 150 was used. Furthermore, the following fitness function was used for the selection of the population taking part in the new offspring generation, which relies on the minimum threshold regarding the average distance from the original training sample:

$$\mathrm{J}({\mathsf{\Theta }}_{\mathrm{j}})={\Vert \arg \min ({\mathrm{F}}_{\mathrm{avg}{\left|\mathrm{b}\right|}^{+}}^{\mathrm{j}}-{\mathrm{F}}_{\mathrm{avg} {\mathsf{\Theta }}_{\mathrm{off}}}^{\mathrm{j}}\Vert }^{\mathsf{\beta }}$$

(18)

In Equation (18), the total population generated is represented by β, and the value β = 10 was used during the experiments. Initially, only the parent population was used to generate the chromosome population, whereas in the advanced step, the population under consideration was composed of the entire set of the elite population, including both the offspring and the elite parents. The complete summary of the genetic algorithm used in the proposed study is as follows (Algorithm 1):

Algorithm 1: Genetic algorithm (GA)

Input: positive image sample set ${\left|\mathrm{b}\right|}^{+}$,images in database ${\left|\mathrm{D}\right|}_{\mathrm{S}}$, β (number of generated populations), population size “PS”, genetic algorithm method “Gm”

1. for j←1, β do

2. ${\mathsf{\Theta }}^{+}$$\leftarrow \mathrm{genetic} \mathrm{algorithm} ({\left|\mathrm{b}\right|}^{+})$$\mathrm{with} \mathrm{elite} \mathrm{parents} {\left|\mathrm{b}\right|}^{+}$

3. for k←1, PS/2 do

4. [p1, p2] ←rand (2)

5. [C1, C2] ←rand (2)

6. ${\mathsf{\Theta }}_{1,\mathrm{off}}=\left[{\mathsf{\theta }}_{\mathrm{PS}1,1}\to {\mathsf{\theta }}_{\mathrm{PS}1,\mathrm{C}1}{;\mathsf{\theta }}_{\mathrm{PS}2,\mathrm{C}1}\to {\mathsf{\theta }}_{\mathrm{PS}2,\mathrm{C}2}{;\mathsf{\theta }}_{\mathrm{PS}1,\mathrm{C}2}\to {\mathsf{\theta }}_{\mathrm{PS}1,\mathrm{G}}\right]$

7. ${\mathsf{\Theta }}_{2,\mathrm{off}}=\left[{\mathsf{\theta }}_{\mathrm{PS}2,1}\to {\mathsf{\theta }}_{\mathrm{PS}2,\mathrm{C}1}{;\mathsf{\theta }}_{\mathrm{PS}1,\mathrm{C}1}\to {\mathsf{\theta }}_{\mathrm{PS}1,\mathrm{C}2}{;\mathsf{\theta }}_{\mathrm{PS}2,\mathrm{C}2}\to {\mathsf{\theta }}_{\mathrm{PS}2,\mathrm{G}}\right]$

8. if Gm = elitism asymmetric, then

9. compute ${\mathrm{J}(\mathsf{\Theta }}_{\mathrm{j}})={\Vert \arg \min ({\mathrm{F}}_{{\left|\mathrm{b}\right|}^{+}}^{\mathrm{j}}-{\mathrm{F}}_{{\mathsf{\Theta }}_{\mathrm{off}}}^{\mathrm{j}})\Vert }^{\mathsf{\beta }}$

10. else

11. compute ${\mathsf{\Theta }}_{\mathrm{e}}=\Vert \mathsf{\Theta }{ }_{\mathrm{off}}<\mathsf{\Theta }\Vert ,\mathrm{e}=\{1\to \mathrm{P}\}$

12. compute ${\mathrm{J}(\mathsf{\Theta }}_{\mathrm{j}})={\Vert \arg \min ({\mathrm{F}}_{{\left|\mathrm{b}\right|}^{+}}^{\mathrm{j}}-{\mathrm{F}}_{{\mathsf{\Theta }}_{\mathrm{off}}}^{\mathrm{j}})\Vert }^{\mathsf{\beta }}$

13. end if

14. end for

15. end for

Output: ${\left|\mathrm{b}\right|}^{+}={\left|\mathrm{b}\right|}^{+}{+\mathsf{\Theta }}_{\mathrm{off}}$

2.3. Data Training Using the SVM Classifier

In the next stage, the newly generated features from LOP and GA are trained using an SVM classifier [44]. There are many other classifier options for the training of data, but the choice of the SVM classifier for training purposes in this work was due to the following reasons:

• SVM works well when there is a clear margin of separation between classes;
• SVM is memory-efficient as compared to other classifiers;
• SVM has better computational complexity;
• The execution time of SVM is very short.

In SVM, the relative positive feature vectors are initialized at the value 1, whereas other vectors are labeled as 0 to fully train the classifier for all categories of images. This is how the class association problem is addressed, and only those images with minimum distance w.r.t. the query image are returned to the user. After SVM classifier training, this method is fully set to effectively predict the motor’s operation and work state.

3. Results and Discussions

In order to validate the effectiveness of the proposed methodology (based on IRT and MV) of fault detection in IMs, it was applied on the experimental test bench, as presented in Figure 1. The experimental setup was composed of a three-phase 2 kW induction motor (powered with a three-phase supply) and an IR camera, both interfaced with a central workstation. The workstation was connected with the IM through a variable-frequency drive, and could generate control signals for the frequency drive to control the IM speed. The workstation contained the trained data generated through IRT and MV for different working conditions of the IM, as already explained in detail in the previous sections. The complete flow diagram of the test bench is provided in Figure 6. According to Figure 6, the thermal image of the IM was captured and transmitted to the workstation to determine the working state of the IM. The workstation looks up the image in the trained data and issues control signals to the drive accordingly for the control of the IM.

In this work, the IM was diagnosed under different conditions, and the corresponding results were generated and validated after detailed analysis in order to calculate the accuracy in a precise manner. The different operating conditions introduced for the IM in the experimental setup were normal operation, overload, and short-circuit. For instance, a short-circuit fault was introduced in the IM by short-circuiting the winding, which caused a rise in the heat profile of the IM. A thermal image was captured and sent to workstation for assessment through MV. The workstation declared the “fault condition” for the IM after the required processing of the thermal image. Meanwhile, the control signal was also issued to frequency drive in order to protect the IM from faults. The proposed method was validated for all three of the abovementioned conditions in the experimental test bench. The experimental setup was able to provide an accuracy level greater than 90% for all three scenarios, which is obviously a reasonable precision level. For this purpose, multiple iterations of the experimentation were carried out for the validation of the proposed algorithm under different conditions. A suitable database was selected and stored initially for the design of the retrieval system by capturing hundreds of IR images of the IM under the three considered scenarios, i.e., normal condition, overloaded condition, and fault condition. After the induction of the trained system in the scenario for fault detection, the retrieval system performance was calculated using the evaluation parameters “precision rate” and “recall rate” provided in Equations (19) and (20), respectively. The evaluation rate of precision can be defined as follows:

$$\mathrm{P}=\frac{{\mathrm{R}}_{\mathrm{ret}}}{{\mathrm{T}}_{\mathrm{ret}}}$$

(19)

where the precision rate is represented by P, the total number of relevant retrieved images is represented by ${\mathrm{R}}_{\mathrm{ret}}$, and the total number of retrieved images is represented by ${\mathrm{T}}_{\mathrm{ret}}$. Likewise, the recall rate R can be defined as follows:

$$\mathrm{R}=\frac{{\mathrm{R}}_{\mathrm{ret}}}{{\mathrm{T}}_{\mathrm{rel}}}$$

(20)

where the recall rate is represented by R, the total number of relevant images retrieved is represented by ${\mathrm{R}}_{\mathrm{ret}}$, and ${\mathrm{T}}_{\mathrm{rel}}$ represents the total number of relevant images.

It is worth mentioning that, in this work, 12 random images from the dataset of hundreds of thermal images were selected for each of the three scenarios (for a total of 36 images for feature extraction and training). The top 30 images were retrieved to calculate the precision and recall rate based on the results of seven iterations. For LOP, experiments were also performed to predict the performance of the feature-extraction technique by changing the diagonal neighbors and order of the LOP. First of all, the experiments on the local octa pattern (2nd-order) with neighbors at diagonals (D) 3, 5, and 7 were performed for the three different operating conditions of the IM, as shown in Figure 7. Figure 7a shows that the precision rates for a neighbor at diagonal 3 are highly precise and accurate for all three operating conditions of the IM, as compared to diagonals 5 and 7. Later on, the second set of experiments were performed by keeping in mind diagonal (D) = 3 only, and changing the order (2nd, 3rd, and 4th) of the LOP. The variations in precision rate between different orders of LOP performance for the various conditions can be seen in Figure 7b, and the results obtained in Figure 7b show that the neighbor at diagonal 3 was highly accurate with 2nd-order LOP for all three conditions.

As mentioned before, a GA was used with LOP in this work to achieve high-quality feature extraction. Therefore, to verify the benefit of the GA’s inclusion with LOP for feature extraction, a comparative study of the LOC with and without the use of the genetic algorithm was also carried out for the normal, overload, and fault conditions of the IM, as shown in Figure 8. It is evident from Figure 8 that higher precision is attainable by the inclusion of the genetic algorithm with LOP for the normal, overload, and fault conditions of the IM. In the end, to further affirm the superiority and effectiveness of the proposed method compared to existing techniques, a comparative study was performed, as summarized in Figure 9 and

Table 3. The comparison of the accuracy of the proposed method with the existing techniques reported in the literature.

No.	Method	Accuracy
1	Proposed Method	0.97
2	Multilayer Perceptron (MLP)	0.8
3	MLP with Graph-Cut	0.84
4	ANN Classifier [34]	0.93
5	SVM [36]	0.83

. Figure 9 affirms the outstanding performance and accuracy of the purposed method for fault detection as compared to MLP, ANN, MLP with graph-cut, and SVM. This method is able to provide an accuracy rate of 0.97, which is obviously high, and this level of accuracy also endorses its reliability for implementation in industry for fault-detection purposes.

4. Conclusions

This work presents the idea of fault diagnosis for an induction motor by incorporating the technique of infrared thermography with machine vision. The obtained results show that the proposed methodology was able to detect the operating mode of the IM under three different conditions of motor operation. In this work, initially, the dataset of thermal images was generated, and was later processed through LOP and SVM in order to perform the automated fault detection of industrial loads. The results further verified that the machine-learning-based feature extraction for the fault detection in an IM is quite useful in comparison to previous techniques, because it detects the faults without overall system shutdown, and avoids energy and economic losses. It was also found that the feature-extraction quality of LOP can be further enhanced if used along GA. In this work, the proposed methodology was also validated by generating an experimental test bench and introducing three scenarios (normal operation, overload, and fault) for the IM. The experimental validation affirmed the effectiveness of the method by diagnosing the working condition of the IM accurately. Overall, it may be concluded from the whole discussion that the inclusion of machine vision with IRT for fault detection provided high accuracy and precision rates as compared to other techniques. Furthermore, this algorithm can be easily implemented with minor changes to other fault-detection applications existing in industrial systems, such as inverters, switching faults, etc.