Developing Inspection Methodology of Solar Energy Plants by Thermal Infrared Sensor on Board Unmanned Aerial Vehicles

Abstract

Photovoltaic (PV) power generation facilities have been built on various scales due to rapid growth in response to demand for renewable energy. Facilities built on diverse terrain and on such a scale are required to employ fast and accurate monitoring technology for stable electrical production and maintenance. The purpose of this study was to develop a technology to analyze the normal operation and failure of solar modules by acquiring images by attaching optical and thermal infrared sensors to unmanned aerial vehicles (UAVs) and producing orthographic images of temperature information. The results obtained in this study are as follows: (1) a method of using optical and thermal infrared sensors with different resolutions at the same time is able to produce accurate spatial information, (2) it is possible to produce orthographic images of thermal infrared images, (3) the analysis of the temperature fluctuation characteristics of the solar panel and cell showed that the abnormal module and cell displayed a larger temperature change than the normal module and cell, and (4) the abnormal heat generation of the panel and cell can be accurately discerned by the abnormal state panel and cell through the spatial distribution of the temperature. It is concluded that the inspection method of the solar module using the obtained UAV-based thermal infrared sensor can be useful for safety inspection and monitoring of the rapidly growing solar power generation facility.

1. Introduction

Concerns about renewable energy have increased more than ever in recent years, including climate change, depletion of fossil fuels, and safety issues regarding nuclear-power generation. In particular, solar-power generation is receiving the most global attention in terms of profitability and efficiency of renewable energy. The installed capacity of photovoltaic (PV) power generation worldwide has grown rapidly from 18.2 GW in 2010 to 88 GW in 2018 [1]. In 2018, the impact of the Chinese government’s reduction of solar-power support has led to a decline in solar demand and a 30% drop in all equipment prices in the supply market, creating stagnation in the solar PV market. However, it is anticipated that China will increase its expectations by presenting policy measures to recover the domestic market through its announcement of future policies, and increase the demand to 1 GW or more for 16 countries. The Republic of Korea became the first 1 GW producer in 2018. Accordingly, demand for PV power-generation capacity, which was 88 GW in 2018, is expected to reach 112 GW in 2019 due to increased demand from China, the US, and India [1]. The European Photovoltaic Industry Association (EPIA) is expected to achieve 550 GW by 2020, and the solar industry is experiencing rapid growth [2,3].

Since the mid-1970s, the Republic of Korea has started to apply solar power to homes. In the early days, solar panels were mostly used for hot water. In 2000, solar PV applications diversified, extending to household, industrial, agricultural, lake, and reservoir applications. Republic of Korea has been mainly used to utilize mountainous areas, which occupy 63% of the entire country. Many solar power plants are generally operated by using large flat terrain, but many facilities have been applied to sloped terrain conditions. In recent years, the number of PV power generation facilities installed on the water, i.e., reservoirs and lakes, has increased. Facilities located on such terrain and water incur heavy cost and require a lot of manpower to check periodic monitoring and problematic solar panels, and there is a high risk of safety problems. In order to operate the solar power plant efficiently, it is necessary to have not only a solar panel but also equipment for monitoring and maintaining the solar power plant [4].

PV power plants are affected by various factors that can damage or reduce their efficiency. Selecting an optimal location for a power plant, one that maximizes power generation efficiency, is important in reducing power-generation and maintenance costs [5]. Once installed, the first time damage occurs to any part, the cell will overload and cause abnormal heat generation. After that, the entire module is short-circuited, and the overall output is lowered. Defects and losses that can occur in solar panels are caused by various factors. Decrease of power generation performance may be caused by initial failure of panel equipment, accident by animal, excretion of bird, effect of atmosphere containing materials such as yellow dust, deterioration of facilities and function, and poor contact due to impact. PV module defects are being categorized by their effects on their origin, nature, install environment, climate, disaster conditions and panel performance. Among them, the main deficiency of PV panels is the occurrence of “hot spots”. Most studies on “hot spots” have progressed from materials to applications. This “hot spot” has been the subject of much research on the recombination of electrons that can make holes in solar cells [6,7,8,9]. The breakage of the panel and the cell occurs in the form of pinholes, V-shaped breaks, edge breaks, and edge breaks. The microcracks in the panel are difficult to distinguish from the boundaries of the cells and are not easily detected, which can cause serious problems, which can significantly shorten the life of the panel. Panel scratches break the thin antireflective coatings and potentially result in output degradation, which reduces the overall performance of the panel because the output of the scraped area is reduced to the lowest cell [10]. As such, defects and losses occurring on the surface and inside of the panel are difficult to grasp because of eyes, and it is not easy to know the cause of reducing or blocking the charge carrier flow, and this part reduces the output of the entire panel. These defects and loss points emit in the form of heat as they release a lot of energy. This phenomenon and process cause great damage to both the cell as well as the entire panel. Therefore, it is important to use a monitoring system together with periodic inspections of the plants to prevent these problems.

Because the solar module of the solar power plant can be located at a high altitude, depending on the situation, the solar module and the array should be subjected to tests by climbing on the solar panel or by using a thermal camera to evaluate the abnormal heat generation [11,12]. However, it is a slow way to use expensive electrical sensing circuitry. It takes a lot of time to inspect it due to the nature of the solar power plant, which has a large area and high installation site, and there are various safety problems. Another ideal solution for this issue is to use thermal cameras. Many defects, such as hot spots, have a noticeably contrasting color compared to other solar cells and their surroundings, so they can be detected through thermal images [8,10]. However, it is not effective or safe to carry out many defect checks with handheld thermal cameras in many cases. In the Republic of Korea, most PV panels are often installed on pitched roofs and sloped hills with poor accessibility, and solar power plants require a very large surface area. Republic of Korea has many hilly areas where accessibility is very poor. Natural obstacles such as sloped terrain and wild weeds are often difficult to move and access panels. To reduce this problem, recently, some research has used drones fitted with thermal imaging cameras for rapid monitoring of solar power plants [13,14,15]. However, the current state of technology is such that the images recorded by the drone are stored in the form of a single sheet, and confirmed by the laboratory or by video shooting [16,17].

The increasing demand for fast inspection speed (while maintaining reasonable accuracy) calls for new tech-savvy approaches, such as unmanned aerial vehicles (UAVs). Research using UAVs has been actively conducted mainly in surveying and agriculture. UAVs are able to acquire agricultural information quickly and precisely at a desired time from anywhere, so that orthographic images of RGB can be acquired and utilized [18,19,20]. UAVs with thermal infrared sensor systems offer an elegant and economically efficient solution [21,22,23]. There is the problem that the position of the accurate failure PV module must be manually displayed on the workstation in the drawing by the operator on the position corresponding to the photographed thermal image file. As the shape of installed solar modules is very similar, there is a restriction that it is not possible to identify the exact position of the solar module unless the photographed image data are accurately grasped [24,25]. The resolution difference between thermal camera image and RGB image has many constraints for utilizing both images together. Many studies have used rubber covers in some solar power plants to test whether temperature sensing for solar modules is possible using fixed wing and rotor UAVs [10,26]. For this reason, most of the techniques have not been used to make 3D orthographic images based on accurate terrain and location information.

The PV system will only be able to demonstrate its value if it is equipped with regular inspection and anomaly detection and protection technologies. To realize this technology, it is important to develop a systematic monitoring method and a surveillance technology of a safe PV system. Therefore, we tried to develop a technology that could be used for monitoring and diagnosis of solar panels by combining the research of solar irradiation modeling with changing weather conditions and the technology of UAV utilization in agriculture [27,28,29].

In this study, we developed a method for monitoring and observing the defects of a solar power generation facility quickly and accurately by making an orthographic image with a thermal infrared sensor attached to a UAV.

2. Materials and Methods

2.1. Study Site

For our field experiment, three PV installations were inspected in a small solar power plant located in Sooyoungro, Cheongju, Chungbuk in the Korea. Figure 1 shows the location of the solar power plant. It is broadly divided into three arrays (A1, A2, A3). The three arrays are installed at an angle of 8.5 degrees to the ground. Each array consists of 4 rows and 5 columns. The installed PV modules consist of 365 watt crystalline silicon, each array is 4.2 m × 20 m and 84 square meters.

Four ground control points (GCPs) are selected for accurate power station location, and a ground reference point survey is conducted using Real Time Kinematic (RTK)-Virtual Reference Service (VRS) surveying method.

2.2. Materials

The UAV for building the optical sensor (RGB) and thermal infrared orthographic image and 3D terrain modeling of the study site utilized the Inspire 2 model produced by DJI, China (

Table 1. Inspire 2 specifications.

Item	Specifications
Weight	3290 g
Diagonal distance (propeller excluded)	605 mm
Max. speed	94 km/h
Max. flight time	25 min
Battery type	LiPo 6S/4280 mAh

). The Inspire 2 model has the stability and payload to mount an optical sensor and thermal infrared sensor as the rotor UAV. This UAV has the advantages of being able to shoot with more stability and a longer flight time than other small UAVs. Figure 2 shows the Inspire 2 model with an optical sensor (Zenmuse X5s, DJI, Shenzhen, China) and a thermal infrared sensor (Vue Pro R, FLIR, Wilsonville, OR, USA). Figure 2 shows the conditions in which images were taken by mounting an optical sensor (Zenmuse X5s) and a thermal infrared sensor (Vue Pro R). Specifications for both sensors are listed in

Table 2. Specifications of optical and thermal infrared sensors.

Item	Specifications
	Vue Pro R	Zenmuse X5s
Resolution	640 × 512	5280 × 3956
Size (mm)	63 × 45 × 45	140 × 45 × 132
Temperature range (°C)	−25 to 135	-
Spectral range	7.5 to 13.5 µm	Red, Green, Blue
Field of view (FOV)	32° × 26°	72°
Output Ttype	Radiometric JPEG, TIFF	DNG, JPEG, DNG+JPEG

. The imaging area is 0.007 km² (0.7154 ha) for optical sensor and 0.005 km² (0.5287 ha) for thermal infrared sensor.

2.3. Research Process

The research progress of the orthographic process in terms of the temperature distribution of the solar panel tested in this study is as follows (Figure 3).

(A)

Selection of target area: The Republic of Korea has a very diverse PV system. The purpose of this study is to obtain accurate temperature and location information of the problem point by utilizing UAV and thermal infrared sensor for the PV system. Therefore, the target area selected was small with a small installation size that could accurately grasp the solar panel information.

(B)

RS/GIS/GPS data collection: UAV-based RGB and thermal infrared imaging and preprocessing were performed around the solar power plant. In particular, to find the sensitivity problem of the thermal infrared sensor, we tried to find the optimum condition by changing the UAV’s height, flight speed, and image overlap. Geographic information system (GIS) information collected property information such as spatial information (location, altitude, area, etc.), social information (land number, landmark etc.), and statistical information. Global positioning system (GPS) information was measured by RTK-VRS method using GCPs, and error of UAV image and field condition was minimized.

(C)

Field survey: In order to review and supplement the collected data, UAV imaging was carried out under various conditions such as time change to identify differences according to time, flight altitude, and climatic conditions. Image acquisition was performed at 10:00 am, 12:00 pm, 2:00 pm, and 7:00 pm according to time. The flight altitude was changed by 10 m to search for optimal image acquisition conditions. The effects of weather conditions were observed on sunny days and overcast weather.

(D)

Orthographic image production: The orthographic images were produced using RGB (optical sensor) and thermal infrared imaging of UAV and field infrared imaging.

(E)

Production of temperature distribution map: The temperature distributions of the PV panels in the study area were prepared by orthographic image.

(F)

Review and output of results: The temperature information and field survey results measured by UAV were compared using statistical techniques, and the results were output.

2.4. Methods

As shown in Figure 3, we used Pix4D Capture (Prilly, Switzerland) to search for optimal flight conditions by changing the shooting resolution, flight altitude, and longitudinal/transverse redundancy. The flight path was determined by considering the degree of redundancy between the flight paths to fully cover the target area. The UAV image was determined by setting the image height, taking into consideration the focal length and pixel size of the sensor, and then determining the spatial resolution. The images were taken with the Pix4D Capture software and combined with the UAV location information and posture expression factors, and then subjected to image fusion (Mosaic) using a preprocessing program (Pix4D Mapper, Switzerland). Total flight time varied depending on the planning conditions, but it took an average about 10 min. A total of 373 photographs per flight were taken at a flight altitude of 30 m and 85% of the longitudinal and lateral redundancy.

In order to secure the position of the solar panel accurately, an RTK-VRS survey was performed based on the GRS80 TM coordinate system by selecting four GCPs [29,30]. Geometric correction was performed using GPS values in the UAV, and the data were converted into orthographic images using tie points automatically generated in the point cloud densification process. For image fusion, Pix4D Capture was used, and four GCP points obtained by VRS survey were used to improve image joint position accuracy. The Pix4D Capture and Mapper allow orthogenesis, point cloud, and Digital Surface Model (DSM) generation, and the 3D terrain modeling data are constructed using orthographic image and DSM data.

Accurate panel temperature difference analysis and error evaluation were performed using R statistical software.

3. Results and Discussion

3.1. Optimum UAV Imaging Conditions for Monitoring Solar Panels

UAVs can determine the spatial resolution of the image in the range of 1~20 cm/pixel depending on the altitude. When the flight altitude is low, the spatial resolution is high, but the shooting area is narrowed. On the contrary, when the flight altitude is high, the shooting area is relatively increased. Therefore, it is necessary to analyze the optimum flight condition for the monitoring of the solar panel and the problem point search. In particular, thermal infrared sensors have a much lower spectral resolution than optical sensors. The altitude for accurate matching of the two images was investigated by incrementally changing the spatial resolution of the two sensors by 10 m. The spatial resolution of a sensor is defined by a measure called ground sample distance (GSD) or pixel size. GSD in an orthographic image of the ground from space is the distance between pixel centers measured on the ground [30].

GSD was obtained from the following equation:

$$\mathrm{GSD}=\frac{\mathrm{Flight} \mathrm{Altitude}\times \mathrm{Sensor} \mathrm{Width} \mathrm{or} \mathrm{Height}}{\mathrm{Forcal} \mathrm{Length}\times \mathrm{Image} \mathrm{Width} \mathrm{or} \mathrm{Height}}$$

(1)

The results are shown in Figure 4, which shows the spatial resolution of each altitude of the thermal infrared image.

The higher the spatial resolution, the more accurate the status of the solar panel and fault detection. However, it is important to consider limited schedules, costs, and manpower, and to determine the spatial resolution with minimum panel readability, taking into account UAV performance, workstation performance, and the altitude of the shooting area. Therefore, it is important to consider the topography and size of the target area for optimal flight and shooting conditions. Also, it is costly to know how much inspection can be done in the day and accuracy will be ensured by using UAV. The most accurate thermographic images were obtained from 12 noon to 2 p.m. In the case of 7 p.m., which is a very short time of solar radiation, the power of the solar panel was automatically cut off and it was impossible to confirm the abnormality of the panel. The weather conditions were found to be inadequate to detect the exact anomalies due to insufficient solar radiation in cloudy weather.

Table 3. Condition acquired by mounting optical sensor and thermal infrared sensor.

Item	TIR	RGB
Photographing altitude (m)		30
Photographing interval (s)	1	-
Covered area (km2; ha)	0.005; 0.5287	0.007; 0.7154
Number of photos	373	114
Resolution (cm)	2.59	0.65
Coordinate system	WGS 84/UTM zone 52N (EGM 96 Geoid); same coordinate
Image size (MB)	86.1	318

shows the optimum shooting conditions that were searched using the optical sensor and the thermal infrared sensor at the same time. In this study, the conditions in Table 3 were found to be the optimal spatial resolution and effective shooting area for information on the target area.

3.2. Terrain Measurement and Accuracy Verification Using UAV and RTK-VRS Surveying Technology

The positioning accuracy depends on flight altitude. When the reference point of the ground is corrected, it can be used as topographic surveying with a flight altitude of 30 m and within 1 cm in the horizontal (x, y) and vertical (z) direction, as shown in

Table 4. Horizontal and vertical errors in connection with GCP points by applying RTK-VRS.

	X	Y	Z	Error X (m)	Error Y (m)	Error Z (m)
GCP.1	357,342.278513	4,052,358.77720	52.4394019224	−0.003	0.004	0.001
GCP.2	357,376.697018	4,052,332.18593	53.7239821130	0.002	−0.003	−0.010
GCP.3	357,377.335893	4,052,308.40536	53.7136950012	−0.006	−0.008	0.015
GCP.4	357,364.698915	4,052,292.90675	53.9219348016	0.007	0.007	−0.005
Mean	-	-	-	0.000067	−0.000154	0.000378
RMSE	-	-	-	0.004808	0.006013	0.009287

.

In addition, a UAV is autonomous, and high-resolution aerial images can be acquired from an altitude of about 30 m by the onboard sensor. Inspire 2 automatically sets the flight plan and returns to the specified location after photographing the specified area. Therefore, it can be used even if it is not visible. In addition, in the case of an 85% overlapped photograph, it is possible to obtain 0.007 km² of data on a flight of ten minute flight. After shooting, it can be output as digital data after it is merged with GCP data or the like, after image processing (creation of original image) and stereoization. Figure 5 shows the orthographic images of RGB and thermal infrared completed by linking with the GCP point using the RTK-VRS surveying technique for UAV images [30]. When shooting with UAV, the most sensitive element is to ensure the location accuracy of the problematic point. If the VRS-RTK survey method is not used, there is a problem that the exact position of the abnormal panel cannot be found. Therefore, the VRS-RTK method applied in this study enabled us to accurately and precisely identify the size and cause of the problematic solar cell. The VRS-RTK method applied in this study is characterized by minimizing the error that occurs at the position of the solar panel and ensuring accurate positional accuracy.

3.3. Precise Orthographic Image of Temperature

Using the UAV-based thermal infrared camera, the accurate temperature distribution of the PV module arrays and the cells was fabricated by orthographic imaging. The solar arrays and modules were designed and assigned unique ID codes so that they can be analyzed automatically and abnormally. The array is given as A , and the module as C . Figure 5 shows the optical image and thermal infrared image, which shows temperature distribution of the solar panel.

Figure 6 shows the temperature distribution and statistics for each array of the solar panel. There is not much difference between the temperature distribution of the solar cell array and the panel. For this reason, the solar power generation facilities used in this study were installed by small-scale operators, but the failure and abnormal rates are not very large, as the installation years do not amount to much. In this case, the test results will be able to identify elements that are sensitive to the poor performance or to the non-uniformity of the performance over the entire panel. As can be seen from the statistics, A1 is the most stable and homogeneous temperature distribution among A1, A2, and A3. In Figure 7, the temperature statistics for the A1 and A2 arrays are very similar, but the A3 array is slightly lower than the two arrays. Array 3′s standard deviation is the highest, so it is considered that the A3 includes the unstable panel in the stability of the panel. Therefore, the A3 array is considered to be a mixture of normal modules and modules that are likely to fail or be abnormal.

3.4. Evaluation of Fluctuation Component Characteristics of Observation Temperature

The temperature stability of each cell was examined in the following order. First, the temperature (T) of the solar module was measured using a UAV-based thermal infrared sensor. Next, the average temperature ($\overline{T}$) was calculated for each solar module. Next, the temperature fluctuation (T’) component was calculated by subtracting the average temperature ($\overline{T}$) from the observe temperature (T).

$$T^{\prime }=T-\overline{T}$$

(2)

$$\overline{T}=\frac{1}{N}{{\displaystyle \int }}_0^{N}T\left(n\right)dn$$

where, N is the number of cells in the panel.

As shown in Figure 8, the fluctuation component (T’) of the temperature obtained by subtracting the array average temperature ($\overline{T}$) from the temperature (T) of each cell was evaluated. Array A1 exhibits a constant temperature distribution over almost all panels and is believed to produce the most stable power. A1 shows that the stability of the panel is the highest because there is almost no temperature difference between the cells in comparison with other two arrays. Array A2 shows a stable temperature distribution at the boundary of the array and a relatively high value at the center. Array A3 showed a stable constant temperature distribution in the eastern part of the panel, but the remaining panels were slightly higher than the average temperature ($\overline{T}$). The lower values of the temperature variation component (T’) in all arrays were found to occur at the boundaries and edges of each panel. Array A3, which was identified as having the lowest stability of the panel, was examined in detail.

Boxplot analysis was performed on the A3 array where the performance of the entire array was not constant in order to identify the normal module and the failure and abnormality module for each cell. Each cell is identified by a code number, as shown in Figure 9a. Figure 9b,c show the optical and thermal infrared orthographic image, respectively. Box 1 (C7) represents a cell with a high temperature, box 2 (C55) represents a cell with a stable temperature distribution, and box 3 (C60) represents a cell with a low temperature or faulty module.

The most sensitive element of thermal infrared sensor image is solar radiation. In a day when the solar radiation amount is high, a stable thermal image can be acquired by normal operation of the module. However, in the evening when insufficient solar radiation, the power was automatically shut off, and the temperature distribution of the module appeared to be different from the normal condition. Figure 10 shows images taken at 7 p.m., a time of insufficient solar radiation, as a failure case that did not occur under other conditions.

The temperature difference evaluation was performed by boxplot analyzing statistical values for each cell and analyzing the differences between other cells. Figure 11 shows the A3 distribution of the temperature variation of each cell in the boxplot. As shown in Figure 9, C7 (corresponding to box 1) has a very high value compared with other cells, and it is necessary to perform a close inspection because it corresponds to a cell having an abnormal value. Most of the cells and the cells located near box 2 (C55) are located in a place where the change of the temperature fluctuation close to the normal value is not large and correspond to the cells that operate normally. Box 3 (C60) and lower cells (C58, C59) will need to be precisely inspected to identify the cause and take action against cells that are in poor condition compared to other cells. The Boxplot method will be a useful tool for statistical analysis of the entire panel as well as to find out the precise inspection objects that appear as abnormal values compared to other panels.

As a result of analyzing the fluctuation characteristics of solar cell-specific temperature, it was found that the abnormal module and cell showed a larger change than the normal module and cell. The inspection method of the solar module using the UAV-based thermal infrared sensor was found to be useful because it can accurately detect abnormal phenomena such as abnormal heat of the solar panel by the cell.

4. Conclusions

In this study, we attempted to evaluate the fault diagnosis of solar modules using optical and thermal infrared sensors for UAVs. The images were taken on a UAV by using an optical sensor and a thermal infrared sensor, taking 114 RGB images and 373 infrared images. The obtained images were able to generate precision orthographic images and 3D terrain modeling data by inputting four GCP points observed through the RTK-VRS survey method.

The precise temperature distribution of the solar module layer and cell was prepared from the orthographic image. The temperature information obtained by the UAV thermal infrared camera design created an identification code for the solar array and module to automatically analyze the failure of the solar module. A UAV-based thermal infrared camera was used to measure the temperature of the solar module, and then the average temperature of each array was calculated. The fault diagnosis of the PV module was performed by evaluating the fluctuation component of the temperature obtained by subtracting the average temperature from the temperature of each cell so that the normal module and the faulty or abnormal module could be discerned. As a result of analyzing the temperature fluctuation data for each PV module, it was found that the abnormal module had a bigger change than the normal module according to the panel condition compared to the normal module. This feature can be applied to GIS spatial analysis to automatically extract abnormal cells and modules, and to accurately analyze the location of modules and cells, using unique identification codes for each module.

The results obtained in this study are as follows: (1) A method of precisely using both optical and thermal infrared sensors with different resolutions was found, (2) orthographic images of thermal infrared were produced, and (3) abnormal phenomena such as abnormal heat generation of the solar panel can be accurately recognized by each cell.

It was found that the method of checking the solar module using the UAV-based thermal infrared sensor can be useful for monitoring, because it can accurately detect abnormal phenomena such as abnormal heat of the solar panel by the cell.