An Inspection Robot for Belt Conveyor Maintenance in Underground Mine—Infrared Thermography for Overheated Idlers Detection

Abstract

It is well known that mechanical systems require supervision and maintenance procedures. There are a lot of condition monitoring techniques that are commonly used, and in the era of IoT and predictive maintenance one may find plenty of solutions for various applications. Unfortunately in the case of belt conveyors used in underground mining a list of possible solutions shrinks quickly. The reason is that they are specific mechanical systems—the typical conveyor is located in the mining tunnel and its length may vary between 100 and 1000 m. According to mining regulations, visual inspection of the conveyor route should be done before it will start the operation. On the other hand, since environmental conditions in mining tunnels are extremely harsh and the risk of accidents is high, there is a tendency to minimize human presence in the tunnels. In this paper, we propose a prototype of an inspection robot based on a UGV platform that could support maintenance staff during the inspection. At present, the robot is controlled by an operator using radio however, we plan to make it autonomous. Moreover, its support could be significant—the robot can “see” elements of the conveyor route (RGB camera) and can identify hot spots using infrared thermography. Moreover, the detected hot spots could be localized and its position can be stored together with both types of images. In parallel, it is possible to preview images in a real-time and stored data allow analysing state of conveyor system after the inspection mission. It is also important that due to radio control systems, an operator can stay in a safe place. Such a robot can be classified as a mobile monitoring system for spatially distributed underground infrastructure.

1. Introduction

The demand for raw materials has increased significantly in recent years. Easily accessible deposits have been exhausted, so we need to reach deeper and deeper to get the treasures of the Earth. Mining is usually considered to be risky and dangerous and miners arouse respect and fear. However, current effective mining is a matter of advanced technologies rather than magic. Increased needs, harsh conditions in deep mines, global competition related to production cost- all these factors lead to the conclusion that advanced technologies that must be used for modern mining. The Mine Magazine recently defined 10 emerging technologies that may change the mining industry. Among others, robotics, predictive maintenance, remote operating and monitoring centers and advanced analytics has been selected as future of mining [1]. Deep underground mining is a general term; however, every single mine is very different and depends on geology, raw material type, the geometry of deposit, used technology, size of the mine, used machines, natural hazard conditions (poisonous gasses, rockburst tendencies), etc. Therefore, each advanced solution should be tuned to the considered use case. In a deep underground copper ore mine considered here, the majority of mentioned technologies have been, or currently are implemented. One of the serious challenges is to maintain a spatially distributed belt conveyor-based ore transport network over a very large area.

A belt conveyor system is used for bulk material transportation for large distances. A system considered here consists of dozen belt conveyors driven by 1–4 drive units, see Figure 1. A transportation system structure looks like a branch of a tree, and the main transportation line consists of conveyors connected in series, see Figure 2. A failure of any of them might stop material transport in the mine and massive production losses. Reliable operation of the belt conveyor system is critically important to keep the production volume on the assumed level. Therefore, each drive unit (see Figure 1) used in conveyor is equipped with electric current and temperature monitoring systems to supervise their operation. Acquired SCADA data is monitored and processed in Data Monitoring and Analysis Center that has been created in the mine (called One Room Control), see Figure 3. In case of any suspicious situation, the maintenance team is asked to do a detailed inspection and advanced diagnostics including vibration measurement, see Figure 4.

Figure 1. A belt conveyor drive: a temperature sensor location (a), an electric motor, a coupling and a gearbox (b).

Figure 2. The conveyor system in KGHM OZG Polkowice-Sieroszowice mine. Each elementary segment is a separate conveyor, scale of the system makes frequent inspection difficult. Each arrow means conveyor, each circle bunker for ore storage.

Figure 3. One Control Room in KGHM OZG Polkowice-Sieroszowice mine (source: http://polskamiedz.wp.pl/artykul/kopalnie-kghm-jak-w-filmowym-avatarze-gornik-pod-ziemia-najmniej-potrzebuj-dzis-kilofu).

Figure 4. Intervention of the inspection team.

While monitoring of drive units is almost solved thanks to SCADA or occasional detailed inspections, unfortunately supervision of the elements of the conveyor route is still a challenge. It should be done regularly, as various components may be damaged or simply may require some corrections. Thanks to the Internet of Things (IoT) concept, some authors proposed solutions based on distributed smart sensors network, but taking into account the number of idlers to be monitored, slow development of wireless transmission system in underground mine, reliability of such solutions, etc., there is a need to search for more realistic/practical scenarios [2].

One of the most critical issues is friction between stationary elements and moving belt, as well as increased rolling resistance of idlers supporting the belts. Research on idlers is an important topic in belt conveyor exploitation and maintenance [3,4]. Damaged idlers may be a source of increased energy consumption or even initiate belt damage or fire. Therefore, to avoid catastrophic failures, a visual inspection is performed by miners along the conveyor route Figure 5. Note, (see Figure 2 that there are tens of kilometers of conveyors and they should be checked from both sides. It requires human resources, takes time and it is pretty risky for people. The idea of using mobile robots for inspection is not a new one. There are some promising papers on that subject [5,6,7,8,9]. Several inspection scenarios have been defined under the umbrella of THING project [10]. Interesting examples of using inspection robots in underground mine is mapping or exploration [11,12,13,14,15,16], supporting of rescue action [17,18,19] or other “mine disasters” [20].

Figure 5. Conveyor belt route inspection currently.

In this paper, we propose a UGV (Unmanned Ground Vehicle) platform as a virtual miner that will do the same work as inspection staff, namely it will ride along the belt conveyor and collect information about conveyor operation using sensors. In Section 2, we will provide a brief description of the robotic platform. Section 3 contains the use case description—an experiment in our laboratory where the belt conveyor test rig is located. We also provide a plan and results of our experiments. Section 4 contains the methods and results of inspection data analysis. Finally, sections Discussions and Conclusions are provided.

2. A UGV Platform

A developed mobile robot belongs to the UGV (Unmanned Ground Vehicle) class of systems. The frame has been assembled using light aluminum construction rails. Drive block has been installed in the front part, and the back contains the cargo space connected with the front using the two-axial joint (Figure 6). It allows for all of the wheels to make contact with the ground all the time, amid the stiff suspension. Additionally, such a solution has good stability (Figure 7).

Figure 6. A scheme of the chassis of the robot for inspection in underground mine.

Figure 7. A robot for belt conveyor inspection.

The wheels are driven by 24V 250W DC motors integrated with a cylindrical gearbox with 19:1 ratio. Additional reduction and power transmission are achieved by using chain and cogwheels as an intermediate step. The driving system is a classic 2.0-class structure. Maneuvering is performed by applying the difference in speeds of the wheels. The robot is powered by two 12 V batteries with 12 Ah capacity each. Voltage for powering the electronics (5 V) is obtained by using a step-down transformer.

2.1. Control System of the Mobile Platform

The robot control is presented in Figure 8. Driving the robot is performed by controlling the speed of the wheels using PWM (Pulse Width Modulation) signals as well as direction signal from the ATMega 328p micro-controller in the standard Arduino Nano module. PWM signal travels to the motors via H-bridges powered by additional relay enabling power cutoff (remotely or in case of an emergency stop). Motor current measurement system has been implemented in the driver to monitor the potential overload [21]. The implementation of the control system is visible in Figure 9.

Figure 8. A concept of control system.

Figure 9. Hardware of control system.

An additional camera was used to watch the area in front of the robot for manual steering. The video feed is wirelessly transmitted using a selected channel at the frequency of 5.8 GHz to remote control panel [22]. The operator steers the robot based on the video feed from the camera displayed on the external monitor, sending PPM (Pulse Position Modulation) wireless control signals from the remote manipulator via an independent 2.4 GHz band. The remote control panel structure is presented in Figure 10.

Figure 10. Mobile platform remote control panel.

2.2. Robot Localization System

During the inspection it is important to precisely localize the place where a fault occurred. One of the systems that are precise enough for event localization is GPS. It can be successfully used for positioning in an open space with high reliability and easy to use interface. However, in the case of underground mines there is no possibility for GPS communication in the underground conditions, so for such a use case the Ultra-Wideband (UWB) transceivers were proposed for the localization system (see Figure 11).

Figure 11. Sensory and location module.

The localization system is based on DWM1001 modules equipped with firmware, which ensures bi-directional distance estimation (TWR) and real-time localization system (RTLS). Inside the module, besides the system directly responsible for distance estimation (UWB Transceiver) there is also a Bluetooth® Microprocessor Nordic nRF52832 and high-performance three-axis linear accelerometer with digital I2C/SPI serial interface standard output LIS2DH12 [23]. Configuration of the module is realized via API interface which can be accessed using various interfaces such as UART, SPI, i2C, and BT. Specific modules configured as Anchor (A) play a role of fixed beacons, and a module installed on the robot operates as a tag (T).

The bi-directional communication and data exchange between the modules as well as the known position in space of each anchor allows us to precisely calculate distances between them and using trilateration algorithms to estimate the position of a tag in space (see Figure 12).

Figure 12. Localization of Tag.

The inspection itself is carried out by driving the robot along the conveyor keeping close distance to it. Deviation of the measurement system in a vertical direction with respect to the conveyor is expected to be minimal, probably negligible. Hence, the number of localization modules can be very limited.

Modules configured as anchors have been installed along the conveyor in such spacing that a tag is always in the range of at least one of them, see Figure 13. For the purpose of the experiment the beacon has been placed on a stand (Figure 14). During the inspection the position relative to the unique antenna in parallel to the sensor data. Hence, it is possible to localize the place of any event. Using the proposed system it is possible to achieve an accuracy of 10 cm.

Figure 13. A plan of the experiment in the lab.

Figure 14. UWB anchor.

2.3. Onboard Sensors

While at the moment the authors focus on hotspot detection based on thermal imaging, it is planned to equip the robot with an extensive sensor suite. Considering the fact that the final result of the diagnostic mission is the ability to provide all the necessary information to the maintenance crew and actually send people to the location, it is important to ensure that the environment is safe for them to arrive. Hence, it is crucial to be able to monitor additional parameters, such as air temperature and humidity, the concentration of dangerous gases such as CO, H${}_2$S or NO${}_x$, or the velocity of air current (some corridors are used for ventilation and air currents can be really strong). Besides the safety parameters, an inspection of other features is planned (i.e., idler set dislocation or internal faults such as defects of bearings possible to be detected via vibroacoustic data analysis).

To address those needs, besides the already present thermal and visual cameras, we plan to equip the robot with sensors such as LIDAR, gas concentration sensors, temperature and humidity sensors, air velocity sensor, and a microphone for noise measurements.

The algorithm for reading and recording sensory data is shown in Figure 15. The application works in two threads. One thread corresponds to the reading of location data, the other thread for reading data from cameras.

Figure 15. Reading and recording of sensory data algorithm.

2.4. Inspection Data Analysis Module

At the current stage of development the inspection is focused on hotspot detection using thermal imaging. Data analysis is carried out offline (ultimately to be carried out in real time during the inspection). First, the image is acquired, then the threshold based on color is performed. When the detected area has a sufficiently large surface, a message is generated. Besides, the event view along with the location is saved and forwarded for verification by the service team. The algorithm was implemented in Python using the OpenCV library [24], and the general flowchart is presented in Figure 16.

Figure 16. An algorithm of hot area detection.

2.5. Data Transmission and Recording

Data acquisition and transmission take place in parallel using several channels. Thermal and RGB images are registered locally within the board computer. Additionally, together with the vision, the robot position relative to the localization system and time information is recorded, which allows us to synchronize two independent camera feeds, as well as identify the event location such as a fault. To confirm the correctness of the view, it is possible to see the camera feed via WiFi (for example using remote desktop). Downloading of the data takes place after the mission ends.

The structure of the system is prepared in a way that it is possible to conduct a mission autonomically; however, in the early stage of the development the robot works in teleoperation mode. Further publications will describe more results related to autonomous applications.

3. The Use Case Description

3.1. A Belt Conveyor Test Rig

The biggest belt conveyor research center in Europe is located in the GEO3EM Research Center at the Wroclaw University of Technology, Faculty of GeoEngineering Mining and Geology. One of the available test rigs is a 7m belt conveyor test rig with artificially introduced belt damages, idlers in different conditions and ability to work under various belt speeds (Figure 17). This test rig has been developed for research related to belt conveyor diagnostics and maintenance. In this paper, we used this test rig to investigate the application of mobile inspection robot (UGV).

Figure 17. Belt conveyor test rig in GEO3EM Research Center.

3.2. Plan of the Experiment

The experiment consisted of two components: traveling along a conveyor belt during his operation (see Figure 13 and Figure 18) and inspection data acquisition.

Figure 18. Robot in action—A view on a laptop screen.

3.3. Results

In this section we present results of experiments performed in lab conditions. In Figure 19 we present an infrared thermography image with one of idler with significantly higher temperature. At the same time RGB images have been acquired by the second camera. Hot areas detection was performed according to the algorithm presented in Figure 16.

Figure 19. An infrared thermography image with one of idler with significantly higher temperature.

4. The Inspection Data Processing—Methods and Results

In this section, we will present the results of our experiment in the lab as well as results of image processing and analysis. Our procedure consists of 3 steps: infrared image analysis, RGB image analysis, and fusion of these results to validate proper hot spot detection.

4.1. IR Data Processing for Hot Spot Detection

From image presented in Figure 19 we extracted a part of image with possible hot spot, see Figure 20. Using tools from Matlab (Image Segmentation Using the Color Thresholder App) we segmented hot spot as presented in Figure 21.

Figure 20. Localized neighbourhood of a hotspot.

Figure 21. Segmented hotspot.

4.2. RGB Image Data Processing for Idler Detection

By analogy to the IR image, we selected an RGB image with a belt conveyor on it. It was acquired at the same time as the IR image, with the same localization of robot. Using the same Matlab tool as before (Image Segmentation Using the Color Thresholder App) we segmented the image presented in Figure 22 as presented in Figure 23.

Figure 22. A Visual reference for hotspot localization.

Figure 23. Segmented idlers.

4.3. The Fusion of RGB Image and IR Data Processing for Hot Idler Detection

Along the belt conveyor route, one may detect many hot spots. To validate that detected hot spot as detected hot idler it is proposed to fuse RGB image and IR data. If segmented areas will be similar, one can conclude that detected hot spot is in fact the idler. The idea of comparing two types of images is presented in Figure 24. In Figure 25 we present two segmented images. Based on shape and location we can confirm that the detected hot spot is a damaged idler.

Figure 24. An infrared thermography image and RGB image fusion for one of idler with significantly higher temperature.

Figure 25. Result of infrared thermography image and RGB image fusion for one of idler with significantly higher temperature.

5. Discussion

The experiment in the lab is usually an approximation of real environment use cases. Indeed, we are aware and we expect that the same experiment in the main will bring new challenges. However, we checked that usage of an infrared camera is possible and the quality of the IR picture is good enough to perform hot idler detection, see Figure 26. In this figure, one can identify three idlers with higher temperatures (above the belt).

Figure 26. Belt Conveyor as IR picture.

The next step will be testing the prototype in the underground mine and collecting inspection data for various belt conveyors.

6. Conclusions

The UGV mobile robot was proposed as a tool for inspection data collecting for belt conveyor maintenance. The robot, driven by a radio-based remote controller operated by humans, was tested in lab conditions in the GEO3EM Research Center at Wroclaw University of Science and Technology. Artificially damaged idler (bearing fault) was introduced, so as a result we were able to use Infrared Thermography to identify hot spots along the conveyor route. The hot spot was detected using standard techniques for image segmentation available in Matlab software. The same approach was used for idler detection using an RGB image. To validate that detected hot spot is the damaged idler, we fused an RGB image with an IR image. There is still a lot of work related to operator-human interaction or robot autonomy, and locomotion of a robot in the mining environment remains difficult. Automation of image analysis is also a challenge. However, we believe that, at this stage, we can send an inspection robot to do inspection in teleoperation mode along the conveyor route without human assistance. The mission could be limited to data acquisition, up to several hundred meters we can transmit data from robot to local computer at the beginning of the conveyor. Considering that the maximum length of the conveyor is 1000 m—the inspection could be done in real time if two access points (beginning and end of the conveyor route) are available. Knowing that each drive unit has sensors connected to SCADA—it is practically possible.