Off-Site Geological Surveying of Longwall Face Based on the Fusion of Multi-Source Monitoring Data

Abstract

A high-precision coal seam model is crucial to improving the adaptability of unmanned mining technology to geological conditions. However, the accuracy of a coal seam model constructed with boreholes and geophysical data is far from the required accuracy of unmanned mining (sub-decimeter level). Therefore, it is necessary to collect geological data revealed by mining and to update the coal seam model dynamically. As a solution to this problem, this paper proposes a new method for conducting off-site geological surveying of longwall faces by integrating multi-source monitoring data. The spatial attitudes of hydraulic supports are monitored to estimate the local dip angles of longwall face. A roof line calculation model was established, which integrates the local inclination angle of the longwall face, the number of hydraulic supports, and the roof elevation of the two roadways. Meanwhile, the local coal–rock columns at the camera observation point are extracted automatically using image segmentation and a proportional relationship between the picture and the actual scene. Coal and rock walls and a support guarding plate in the longwall face image are identified accurately using the coal-rock support segmentation model trained with U-net. Then, the height of the coal (or rock) wall above the coal–rock interface is estimated automatically according to the image segmentation and the similar proportion equation of actual longwall face and longwall face image. Combined with mining height information, the local coal–rock column can be extracted. Finally, the geological surveying profile of longwall face can be obtained by integrating the estimated roof line and local coal–rock columns. The field test demonstrated the efficacy of the method. This study helps to address a long-standing limitation of insufficient geological adaptability of intelligent mining technology.

1. Introduction

Geological surveying of longwall face is essential in underground coal mine geology. A geological technician can analyze the formation occurrence of coal seams (thickness, roof or floor level, and geological structure) in the area to be mined on the basis of longwall face profiles [1,2,3]. With accurate geological surveying, geological forecasting has enhanced the safety level and production efficiency of coal mines [4,5]. For instance, advanced geological exploration in a coal mine in Inner Mongolia, China, issues warnings for a reverse fault 30 days in advance, which buys crucial time for adjusting the cutting parameters of the shearer and avoids equipment damage and production halts [6]. The significance of the geological profile extends beyond safety management. As the most reliable input for coal seam modeling, it fundamentally guarantees the reliable operation of an intelligent mining system. High-precision 3D models of coal seams can optimize three key parameters in real time: the drum height and advancement sequence of the coal shearer, and the inclination angle of the scraper conveyor. Field tests show that model-oriented dynamic adjustments can increase the coal recovery rate by 3–5% and reduce rock cutting by 18–22% simultaneously [7]. The longwall face profile is the most reliable data type for constructing an accurate coal seam model. Meanwhile, a high-precision coal seam model can guide the dynamic adjustment of the mining height and advancing direction of the shearer, thereby enhancing the adaptability of intelligent mining technology to geological conditions. Currently, efficient and accurate geological measurement of longwall working faces is a research hotspot in the coal mining industry [8].

Traditionally, the surveying of longwall face mainly relies on manual mapping with an optical level. This method requires technicians to measure the elevation point-by-point along the longwall face and simultaneously record various geological elements, including lithological changes, structural features, and stratigraphic contact relationships [9]. Due to the complex and harsh working environment of longwall face, manual surveying is often of low frequency and the data reliability is not strong. Therefore, lots of researchers have carried out a series of intelligent perception studies on the geological information of longwall face. The following paper discusses the current research status in terms of intelligent perception of spatial information and image recognition of coal and rock information on the longwall face.

Currently, there are mainly three intelligent perception methods for longwall face space information. The first one is to utilize high-precision pose sensors installed on fully mechanized coal mine mining equipment to sense the slope and height of the longwall face. Liu et al. [10] used multiple sensors to monitor the pose of the shearer in real time, calculated the three-dimensional coordinates and cutting drum trajectory, and obtained the top and bottom plate curves of the longwall face. The JOY Company collected relevant data on the longwall face by integrating various pose sensors on the fully mechanized mining equipment. And the bottom line of the longwall face was calculated by fusing the elevations of the two roadways and monitoring data. The LASC research team in Australia integrated multi-source sensor data, including inertial navigation and coal and rock detectors, to construct a dynamic cutting model to guide shearer operations [11,12,13,14]. The second method is to use three-dimensional LiDAR to scan the spatial contour of the longwall face. Jiang et al. [15] proposed a point cloud slicing technique and a feature point extraction algorithm based on 3D LiDAR point cloud data, which automatically extracts the coal cutting roof line. Wang et al. [16] utilized a mobile 3D laser scanning system to quickly reconstruct the 3D scene of the roadway and longwall face. CSIRO of Australia developed an explosion-proof laser radar, which can efficiently scan the contour of the longwall face and the fully mechanized mining equipment [17,18]. The third method is to perceive the undulating shape of the coal seam from the geological marker layer. Jonathon and Andrew [19] used a thermal infrared camera installed on the shearer to automatically track geological marker zones closely related to the horizontal orientation of coal seams. The geological marker layers in infrared images were extracted by using the line segment detection algorithm and the Kalman filtering algorithm, and the coal seam dip angle was determined by the geological marker layers. Wang et al. [20] found that the lithological indicators of coal seams exhibit different optical characteristics due to differences in composition and structure, and explored their potential application in the intelligent mining of coal and rock identification. Zhu et al. [21] proposed an iterative modeling method based on drilling constraints along coal seams. This method effectively solves the problem of insufficient accuracy caused by sparse drilling in traditional modeling.

The intelligent perception of coal and rock information is generally achieved through the recognition of the longwall face. Wu et al. [22] proposed various coal and rock image recognition methods, including local constraint self-learning, Curvelet transform and compressive sensing, and dictionary learning. Sun and Chen [23] used an asymmetric generalized Gaussian model in the wavelet domain to characterize the texture features of coal and rock images. Si et al. [24] proposed a coal and rock image recognition method based on an improved U-net network model, which improves recognition accuracy and efficiency. Zhang et al. [25] improved the accuracy of coal and rock infrared image recognition by actively stimulating the coal and rock walls in the longwall face and using a temperature gradient model to divide the isotherms of the infrared image. Wu et al. [26] proposed a coal and rock interface recognition method based on a boundary tracking algorithm and artificial neural network. Gao et al. [27] constructed a coal and rock segmentation tower network using tower pooling architecture and convolutional neural networks to enhance the global information representation of feature maps. Yang et al. [28] conducted a study on hyperspectral identification of coal and rock interfaces by utilizing the differences in the mineral content of Al-OH vibration structures in coal and rock layers. Wei et al. [29] also created hyperspectral identification methods for coal and rock walls in the Huainan mining area of China.

In summary, the current research on geological information perception in mining sites generally suffers from the lack of synchronous perception of spatial morphology and coal rock attributes. Traditional coal rock identification techniques struggle to support dynamic geological model updates. Therefore, an off-site geological survey method for long-wall working faces that integrates a longwall face image and the pose perception data of the hydraulic support is proposed. Firstly, multiple cameras installed on the hydraulic support collect longwall face images, and the tilt angle detection sensors collect the tilt angle of the hydraulic support. Then, the convolutional neural network (CNN) is adopted to extract the distribution information of coal and rock, and the local coal and rock columns are obtained by fusing the mining height. Then, a top line calculation model is constructed by integrating data such as the measurements of the roadways on both sides, the inclination angle, and the position of the hydraulic supports. Finally, profiles of the longwall face are generated based on the roof line and the local coal–rock columns. This method greatly improves the surveying efficiency of longwall faces.

2. Demand for High-Precision Coal Seam Model

Since the end of the 12th Five-Year Plan, China has developed a memory cutting intelligent mining technology in a longwall panel with simple geological conditions [30]. The technical bottleneck in the real-time identification of coal–rock interface has restricted the application of the memory intelligent cutting system in longwall panels with complex geological condition. Therefore, domestic scholars have successively put forward academic concepts such as precise coal mining [31], transparent mining [32], and autonomous navigation cutting [33]. The core ideas involve exploring longwall panel geological conditions, establishing high-precision coal seam geological models, then digitally simulating mining operations to plan shearer cutting trajectories, and ultimately achieving autonomous coal mining through the coordinated control of fully mechanized mining machinery. The static coal seam geological model constructed using pre-mining geologic survey data such as well logging [34] and in-seam survey [35] has an overall accuracy at the meter-level due to the limitations of geological exploration resolution and exploration grid. However, the accuracy is still of centimeter-level accuracy when using collaborative control with comprehensive mining machinery. This precision gap has been identified as the critical bottleneck in autonomous mining systems [36].

Chinese scholars have proposed a dynamic geological modeling framework to deal with the challenge of high-precision longwall panel modeling. That is, the geological information of the longwall face is dynamically surveyed, and the static longwall panel geological model is modified accordingly during the mining process, ensuring the accuracy of the geological model of the area to be mined meets the requirements of autonomous planning and mining (Figure 1). The practices of cascade construction for a geological model of a longwall panel for autonomous planning coal mining based on manual surveying have been implemented in mines such as Yangquan Xinyuan Coal Mine and Huang ling Coal Mine. By dynamically updating the geological model of the longwall face using the geological information revealed during mining, the modeling error of roof, floor, and thickness of ab un-mined coal seam within 8–15 m in front of the longwall face is less than 30 cm, which basically meets the accuracy requirements of autonomous planning coal mining. The prediction error of the top and bottom plates and thickness changes in the coal seam within the range of 8 to 15 m in front of the working face is less than 30 cm [37]. In addition, the LASC (Longwall Automation Steering Committee) research team of the Australian CSIRO uses multiple sensors to perceive geological information revealed during mining and updates the cutting model for unmanned coal mining [38]. Therefore, intelligent geological information surveying of longwall panels is a key breakthrough in high-precision geological modeling, which is of great significance for the development of autonomous planning coal mining.

3. Geological Surveying Method of Longwall Face Profile

3.1. Geological Surveying Principle

3.1.1. Layout of Hydraulic Supports on the Longwall Face

As is shown in Figure 2, hydraulic supports are arranged in the longwall face to support the roof and maintain the safety of the working space. The hydraulic support is one of the most important pieces of equipment in the longwall face. The other main equipment includes the shearer, scraper conveyor, etc. To achieve unmanned operation in the longwall face, one high-definition camera is installed for every 4 to 6 hydraulic supports to monitor the working conditions of the longwall face in real time, replacing personnel inspection. In addition, the hydraulic support is also integrated with a variety of other sensors, including a height sensor for monitoring the mining height and an inclinometer for monitoring the spatial attitude of the hydraulic support. The key point of this paper lies in how to utilize the multi-source monitoring data obtained from these sensors to acquire the geological information of the longwall face.

3.1.2. Geological Surveying Principle of Longwall Face

Figure 3 shows the geological surveying principle of the longwall face. The geological surveying of the longwall face consists of three parts: the calculation of the top and bottom plate lines of the longwall face, the extraction coal–rock columnar, and geological surveying. (1) Calculation of the top and bottom plate lines of the longwall face. The local slopes α_i (1≤ $i$ ≤ n) of the longwall face can be estimated according to the spatial posture of the hydraulic supports. The slope lengths l_i (1≤ $i$ ≤ n) are calculated by the distance from each support to the roadway. The bottom plate line of the longwall face is calculated by taking the elevation of the bottom plates on both sides of the roadways and high-precision geological data such as drilling and logging as hard constraints. Based on the bottom plate line, the top plate line of the longwall face can be obtained by integrating the mining height data recorded by the hydraulic support and the top plate elevations of the two side roadways. (2) Range of coal and rock walls. The image intelligent segmentation algorithm is adopted to automatically identify the distribution of coal and rock wall in the longwall face image. The coal–rock columnar at the video observation point can be constructed by combination with the geological exploration information of the longwall face, markers (such as hydraulic support face guard), mining height, and other information. (3) Fusion of the longwall face geological profile. The longwall face geological profile is generated by integrating the top and bottom plate lines and the coal–rock columns at each observation point. The updated geological profile can serve as the key dynamic data for the update of the longwall face geological model.

3.2. Roof Line Calculation Model

It is necessary to set some assumptions and simplify the actual problem for the purpose of estimating the roof line of longwall face. Therefore, the following three assumptions are made:

• The undulating shape of the longwall face changes continuously and gently, without abrupt change;
• All of the longwall face images are taken under the condition that the cameras are level and the optical axes is vertical to the longwall face;
• The longwall face is photographed with a certain horizontal spacer.

A simplified roof line calculating model, as shown in Figure 3, was constructed on the basis of the above assumptions. The relevant variables are defined as follows:

The roof heights at (n + 1) observation points are as follows:

$$z_0,\hspace{0.33em}\hspace{0.33em}{z}_1,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}{z}_i,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}{z}_{n-1},\hspace{0.33em}{z}_n\hspace{0.33em}(i=0,\hspace{0.33em}1,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}n),$$

(1)

The local dip angles at (n + 1) observation points are as follows:

$${\alpha }_0,\hspace{0.33em}{\alpha }_1,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}{\alpha }_i,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}{\alpha }_{n-1},\hspace{0.33em}{\alpha }_n\hspace{0.33em}(i=0,\hspace{0.33em}1,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}n),$$

(2)

When longwall face extends upward, set the dip angle to positive; otherwise, set the dip angle to negative.

The horizontal distances from (n + 1) observation points to the left roadway wall are as follows:

$$l_0,\hspace{0.33em}{l}_1,\hspace{0.33em}{l}_2,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}{l}_i,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}{l}_{n-1},\hspace{0.33em}{l}_n\hspace{0.33em}(i=0,\hspace{0.33em}1,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}n),$$

(3)

$i$ = 0 and $i$ = n in Equations (1)–(3) denote the left and right roadway observation points, respectively. $i$ = 1, 2, …, n − 1 denote the ordinal camera observation points from left to right in the longwall face. The dip angles α_i ($i$ = 1, 2, …, n − 1) are determined according to the spatial posture of the hydraulic supports. The horizontal distances l_i ($i$ = 1, 2, …, n − 1) are determined through the serial number of the corresponding supports.

Thus, the following variables are known:

$$\left\{\begin{array}{c}{z}_0,\hspace{0.33em}{z}_n\hfill \\ {\alpha }_0,\hspace{0.33em}{\alpha }_1,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}{\alpha }_i,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}{\alpha }_{n-1},\hspace{0.33em}{\alpha }_n\hspace{0.33em}(i=0,\hspace{0.33em}1,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}n)\hfill \\ l_0,\hspace{0.33em}{l}_1,\hspace{0.33em}{l}_2,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}{l}_i,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}{l}_{n-1},\hspace{0.33em}{l}_n\hspace{0.33em}(i=0,\hspace{0.33em}1,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}n)\hfill \end{array}\right.,$$

(4)

The following variables are unknown:

$$z_1,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}{z}_i,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}{z}_{n-1}\hspace{0.33em}(i=1,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}n-1),$$

(5)

The dip angle of the longwall face is generally small and changes gently. Meanwhile, one camera is mounted for every 2~6 supports in the intelligent longwall face. The horizontal distance of adjacent observation points is less than 9 m (l_i − l_i−1 < 9 m) given that the supporting space of a support is general about 1.5 m. Therefore, the red broken line in Figure 3 can be used to approximate the roof line of the longwall face (green line in Figure 3).

The quantitative relationship of the roof heights at adjacent observation points can be given by the following:

$$z_i=z_{i-1}+(l_i-l_{i-1})\tan {\alpha }_{i-1}\hspace{0.33em}\left(i=1,\hspace{0.33em}2,\hspace{0.33em}...\hspace{0.33em},\hspace{0.33em}n\right),$$

(6)

The definition of variable, $\mathsf{\Delta }{z}_{i-1,i}$, is shown as Equation (7),

$$\mathsf{\Delta }{z}_{i-1,i}=z_i-z_{i-1}\hspace{0.33em}\hspace{0.33em}\left(i=1,...,n\right),$$

(7)

Equation (4) applied to Equation (7) leads to Equation (8):

$${\sum }_{i=1}^n\mathsf{\Delta }{z}_{i-1,i}=z_n-z_0,$$

(8)

Due to the simplification of the problem and the residuals of l_i and α_i, Equation (8) is not valid. Therefore, Equation (8) can be modified to contain the residual ε.

$${\sum }_{i=1}^n\mathsf{\Delta }{z}_{i-1,i}+\epsilon =z_n-z_0,$$

(9)

ε can be averaged into $\mathsf{\Delta }{z}_{i-1,i}$ ($i$ = 1, 2, …, n) based on the surveying adjustment theory:

$$\left\{\begin{array}{c}{\displaystyle \frac{{\epsilon }_1}{\mathsf{\Delta }{z}_{0,\hspace{0.33em}1}}}={\displaystyle \frac{{\epsilon }_2}{\mathsf{\Delta }{z}_{1,\hspace{0.33em}2}}}=...={\displaystyle \frac{{\epsilon }_n}{\mathsf{\Delta }{z}_{n-1,\hspace{0.33em}n}}}\hfill \\ {\sum }_{i=1}^n\left(\mathsf{\Delta }{z}_{i-1,i}+{\epsilon }_i\right)=z_n-z_0\hfill \\ {\sum }_{i=1}^n{\epsilon }_i=\epsilon \hfill \end{array}\right.,$$

(10)

where ${\epsilon }_i$ ($i$ = 1, …, n) is the error component adjusted to $\mathsf{\Delta }{z}_{i-1,i}$. The mathematical solution of ${\epsilon }_i$ is derived as follows

$${\epsilon }_i={\displaystyle \frac{\epsilon }{{\sum }_{i=1}^n\mathsf{\Delta }{z}_{i-1,i}}}\mathsf{\Delta }{z}_{i-1,i}\hspace{0.33em}\hspace{0.33em}\left(i=1,...,n\right),$$

(11)

Equations (8) and (9) applied to Equation (6) leads to the following equation of z_i ($i$ = 1, …, n − 1):

$$z_i=z_{i-1}+(l_i-l_{i-1})\tan {\alpha }_{i-1}+{\displaystyle \frac{\epsilon }{{\sum }_{i=1}^n\mathsf{\Delta }{z}_{i-1,i}}}\mathsf{\Delta }{z}_{i-1,i},$$

$$=z_{i-1}+\left(l_i-l_{i-1}\right)\tan {\alpha }_{i-1}+\left({\displaystyle \frac{\left(z_n-z_0\right)-{\sum }_{i=1}^n\left(l_i-l_{i-1}\right)\tan {\alpha }_{i-1}}{{\sum }_{i=1}^n\left(l_i-l_{i-1}\right)\tan {\alpha }_{i-1}}}\right)(l_i-l_{i-1})\tan {\alpha }_{i-1},\left(i=1,...,n-1\right).$$

(12)

The variable z_i, l_i ($i$ = 1, …, n − 1) determines the approximate roof line of the longwall face.

3.3. Local Coal–Rock Columnar Surveying

3.3.1. Coal–Rock Column Calculation Model

Figure 4 shows the imaging principle of the camera. When the optical axis of the camera is perpendicular to the longwall face, the image and the real scene of the longwall face satisfy Equation (13).

$${\displaystyle \frac{H_r}{H_{r^{\prime }}}}={\displaystyle \frac{a}{b}}=m,$$

(13)

where $H_r$, $H_{r^{\prime }}$ represent, respectively, the actual and image heights of rock or coal wall located above the coal–rock interface; a, b are the actual and image breadths of the support guarding plate; m is the magnification of the camera.

For the specific longwall face, the breadth of the support guarding plate is constant. In addition, the image breadth b of the support guarding plate can be determined automatically according to image segmentation. That is, magnification m is a known quantity. The bottom boundary of the longwall face is generally unclear due to accumulated loose coal. However, the distance h₀ from the bottom of the support guarding plate to the longwall face roof is known for a specific support, as shown in Figure 5. Therefore, H_r can be represented as Equation (14).

$$H_r=h_0+\mathsf{\Delta }h,$$

(14)

where ∆h is the height difference between the coal–rock interface and the bottom of support guarding plate. ∆h can be calculated using Equation (15).

$$\mathsf{\Delta }h=m\left({\bar{y}}_{r,c}-{\bar{y}}_b\right),$$

(15)

where ${\overline{y}}_{r,c}$ and ${\overline{y}}_b$ are the average Y-coordinates of the coal–rock interface and the bottom line of the support guarding plate in the longwall face image, which can be obtained automatically with image segmentation. When the coal–rock interface locates above the bottom of the support guarding plate, ∆h < 0; otherwise, ∆h > 0.

The height H_f of the rock or coal wall located below the coal–rock interface can be calculated using Equation (16).

$$H_f=H-h_r$$

(16)

where H is the mining height monitored by hydraulic pillars of the roof support or the clinometer mounted in shearer cutting drums.

Finally, the local coal–rock columns at the camera observation points can be determined according to H_r and H_f.

3.3.2. Coal–Rock–Support Image Segmentation Model

• Training data preparation

A total of 892 original coal wall images and 181 original rock wall images were collected as a basic experimental dataset. Typical coal and rock wall images are shown in Figure 6a and b, respectively. Coal walls are characterized by sporadic reflective surfaces and inky black color. Meanwhile, rock walls are off-white.

Data augmentation methods are used to expand the image training set. The selected data augmentation method should be in line with the actual situation. Otherwise, new data not only fail to improve the robustness of the training model but may also lead to the model not converging. Due to the lack of features such as up and down flipping, and complex noise in the longwall face image, data augmentation methods such as up and down flipping, adding noise, and filling are not applicable. Therefore, a data augmentation scheme suitable for the semantic segmentation of longwall face images has been proposed by integrating geometric transformations and multi-sample data augmentation methods (Figure 7).

Multi-sample rock wall data enhancement. Considering the scarcity of basic rock wall data, a multi-sample data enhancement method was adopted to artificially synthesize a large number of rock wall images. There are three steps in the specific process, as follows. Step 1: Using image editing software to extract the rock wall areas from the basic rock wall image. A typical rock wall area sub-graph is shown in Figure 8a. Step 2: For high-quality coal wall images (such as Figure 8b), selecting a suitable area (such as the coal wall area between adjacent guard plates) and removing the image (as shown in Figure 8c). Step 3: Selecting the sub image of the rock wall area and moving it to the bottom of the coal wall image where a portion of the area has been removed, generating a new rock wall image (as shown in Figure 8d). It should be noted that the transition at the coal–rock interface is natural and the stratum direction is nearly horizontal in the artificial rock wall image.

Geometry and color enhancement. The original coal wall images, rock wall images, and multi-sample enhanced rock wall images constitute a new basic training image set. Horizontal mirroring, central rotation, top-left cropping, and bottom-right cropping operations on each image in the new image set were, respectively, performed to increase the size of the training set by four times. A total of 3370 coal wall images and 3385 rock wall images were obtained through the data augmentation scheme shown in Figure 9. The masks of the longwall face images were labeled by technical personnel with rich on-site working experience using the open-source software package LabelMe (v4.5.13) [39,40]. White (RGB color value: (255, 255, 255)), gray (RGB color value: (130, 130, 130)), and black (RGB color value: (0, 0, 0)) areas in the mask images denote support guarding plate, rock wall, and coal wall, respectively. A case of longwall face image labeling is shown as Figure 9.

2.

Training, validation, and testing sets

To train and evaluate the segmentation model of longwall face images, all images are randomly divided into training, validation, and testing sets. The functions of the training and testing sets are model fitting, model evaluation, and hyperparameter adjustment, respectively. The testing set is used to evaluate the generalization ability of the trained model. First of all, 50 coal wall images and 50 rock wall images are selected randomly, which comprised the testing set. The remaining coal and rock wall images are divided into training and testing sub-groups at a ratio of 9:1, as shown in

Table 1. Division of training, validation, and testing sets.

Dataset	Coal Wall Image	Rock Wall Image	Total
Training set	2988	3001	5989
Validation set	332	1334	666
Testing set	50	50	100
Total	3370	3385	6755

.

3.

Model training strategies

The U-Net architecture has the advantages of high segmentation accuracy with only small training samples required. Therefore, a U-Net network was built to train coal–rock–support image segmentation model by using Kera’s deep learning frameworks. To obtain a better performance of coal–rock–support image segmentation, two different training strategies are designed and compared, as shown in

Table 2. Two kinds of model training strategies.

Parameter	Strategy I	Strategy II
Epoch	100	100
Loss function	Categorical cross entropy	Categorical cross entropy
Classifier	SoftMax	SoftMax
Monitoring index	Loss, IOU	Loss, IOU
Optimization function	Adam	RMSprop
Lambda layer	Normalization	No-operation
Initial learning rate	0.001	0.001
Batch size	16	8
Learning rate optimizer	Adam (adaptive LR)	LR = 0.1 × LR when Val loss does not decrease for two times
Early stopping	Val loss does not decrease for 10 times	Val loss does not decrease for 10 times

(Note: IOU: Intersection over union; LR: Learning rate; Val loss: Validation loss.).

.

The learning curves of Strategy Ⅰ (Figure 10a,b) and Ⅱ (Figure 10c,d) are similar generally, such as in curvilinear trend, final IOU, and Loss. However, Strategy Ⅱ works better in terms of details, as is mainly reflected in the following two aspects:

Firstly, the loss and Val loss values of Strategy Ⅱ are stable at 0.0328 and 0.0460, respectively, after the 15^th epoch. Meanwhile, the Val loss value fluctuates slightly around 0.05, and the loss value continue to decrease, indicating that the model is over-fitting.

Secondly, the IOU value of Strategy Ⅱ tends to be stable at 0.75 after the 25^th epoch, with a significant decrease in the growth rate. Meanwhile, the IOU value of Strategy Ⅰ still maintains a high growth rate, showing a trend of overfitting.

Therefore, the coal–rock–support image segmentation model trained with Strategy Ⅱ was selected.

4.

Model testing

Typical coal–rock–support image segmentation of the testing set is shown in Figure 11. In each sub-image, the left shows the original longwall face image, and the right shows the image segmentation. The grayscale segmentation images were converted into RGB color images for highlighting, where the blue, red, and green areas correspond to support guarding plate, coal wall, and rock wall, respectively. Both of the coal and rock wall image segmentation cases demonstrate the applicability of the trained coal–rock–support image segmentation model.

4. Field Test

4.1. Overview of the Surveyed Longwall Face

The No. 31004 Longwall Panel of Xinyuan Coal Mine was the surveying case with a geological structure. The coal seam thickness varies from 2.52 to 2.82 m, with an average of 2.72 m. The dip angle of the coal seam generally varies from 2 to 8°, with an average of 6°. There are 160 supports in the longwall face, with a width of 1.5 m per a support. One camera is mounted for every 10 supports from the No. 1 support to monitor the operation of the shearer and the roof supports. That is, the No. 1, No. 10, No. 20 …, No. 160 supports are mounted with cameras.

The surveyed longwall face was cut on 12 December 2020. The roof level, coal seam thickness, and dip angle of longwall face on the side of auxiliary intake airway were 504.82 m, 3.13 m, and 7.5°, respectively. The roof level, coal seam thickness, and dip angle of the longwall face on the side of the intake airway were 517.63 m, 2.55 m, and 0.4°, respectively. Typical longwall face images are shown in Figure 12. The whole images show that the area where the No. 65#~115# supports are located is the fault-affected area. Detailed information about the rock and coal walls in the fault-affected area is listed in

Table 3. Information on rocks and coal walls in the fault affected area.

Support Serial Number	Foot or Upper Wall	Coal or Rock Wall
165#~75#	Foot wall	Upper coal and lower rock
75#	Fault plane	-
75#~90#	Foot wall	Whole rock wall
90#~115#	Foot wall	Upper rock and lower coal

.

4.2. Roof Line

Table 4. Monitoring and measured dip angles.

Support Serial Number	Dip Angle (°)			Roof Coordinate Calculation (m)			Remark
	Position Monitoring	Manual Measuring	Absolute Error	Position Monitoring	Manual Measuring	Absolute ∆z
1	7.5	7.5	0	(0, 504.82)	(0, 504.82)	0	Measuring
10	8.3	8.4	0.1	(15, 506.79)	(15, 506.85)	0.05	Coal seam
20	7.4	7.8	0.4	(30, 509.01)	(30, 509.10)	0.09	Coal seam
30	7.8	7.7	0.1	(45, 511.06)	(45, 511.10)	0.04	Coal seam
40	5.3	5.1	0.2	(60, 513.09)	(60, 513.21)	0.12	Coal seam
50	3.4	3.4	0	(75, 514.43)	(75, 514.64)	0.21	Coal seam
60	2.8	2.9	0.1	(90, 515.32)	(90, 515.56)	0.24	Coal seam
70	−7.6	−7.2	0.4	(105, 516.08)	(105, 516.31)	0.23	Upper coal and lower rock
75	132.1	132	0.1	—	—	—	Fault plane
80	−4.8	−5	0.2	(120, 514.51)	(120, 514.26)	0.25	Rock wall
90	−0.9	−1	0.1	(135, 513.19)	(135, 512.96)	0.23	Rock wall
100	1.1	1	0.1	(150, 512.93)	(150, 512.72)	0.21	Upper rock and lower coal
110	3.5	3.4	0.1	(165, 513.19)	(165, 513.01)	0.18	Upper rock and lower coal
120	5.3	5.1	0.2	(180, 514.08)	(180, 513.96)	0.13	Coal seam
130	4.9	4.9	0	(195, 515.42)	(195, 515.39)	0.04	Coal seam
140	3	3.1	0.1	(210, 516.71)	(210, 516.71)	0	Coal seam
150	0.4	0.4	0	(225, 517.52)	(225, 517.52)	0	Coal seam
160	0.4	0.4	0	(240, 517.63)	(240, 517.63)	0	Measuring

shows the local dip angles estimated from images and measured manually. The roof line coordinates calculated according to position monitoring and manual measuring were also illustrated. Two roof lines are calculated on the basis of the estimated and measured dip angles of longwall face, as shown in Figure 13. Compared with the measured roof line, the maximum and average absolute errors of the image-based roof line are 0.25 m and 0.12 m. Due to error accumulation, the error of the roof level gradually accumulates from the two side roadways to the middle of the longwall face.

4.3. Coal–Rock Columns in Fault-Affected Area

Figure 14a shows the longwall face image at the No. 70# support. Here exists the footwall of a normal fault, with upper coal and lower rock. Figure 14b shows the image segmentation of Figure 14a using the trained CNN model. The blue, red, and green areas denote the support, coal wall and rock Wall, respectively. The support model of No. 31004 Longwall Panel is ZY8000/18/37D. The support dimension shows that the height difference between the guarding plate bottom and the longwall face roof is 1.25 m and the width of the guarding plate is 1.20 m. The magnification m is calculated to be 0.0045 on the basis of the actual width and image width (Figure 14b). The recorded mining height is 3.10 m. Figure 14c,d shows the coal–rock columns, which are estimated with a longwall face image and measured manually, respectively. The absolute error in the coal (rock) wall height estimated with the image is 0.08 m, and the relative errors of coal and rock wall height are 3.7% and 8.6%, respectively.

Table 5. Detailed information of coal and rock pillars at the No. 100#, 110# supports.

Support Serial Number	Image Identification (Pixel)		k	Rock Wall Height (m)	Mining Height (m)	Coal Wall Height (m)
	Guarding Plate Width	Rock Wall Height
100	89	110	74.03	1.49	3.03	1.54
110	121	75	100.90	0.74	2.95	2.21

shows the detailed information of the coal–rock columns at the No. 100#, 110# supports extracted from the longwall face images. Figure 15a,b shows the extracted coal–rock columns at the No. 100# (a), 110# (b) supports, respectively.

4.4. Longwall Face Geological Profile

A longwall face geological profile can be drawn by integrating the roof line and coal–rock columns. Figure 16a,b shows the geological profiles are estimated with images and measured manually, respectively. Both the manual and image-based surveys utilize the mining height data to draw coal–rock columns. Therefore, the difference between the manual and image-based surveying profiles are mainly the roof line level and the coal/rock heights. The error analysis of the roof line and coal/rock wall heights are discussed in Section 3.2 and Section 3.3, respectively. The precision of the image-based surveying profiles mainly depends on the posture perception information of the hydraulic supports, which indicate the undulation form of the longwall face.

4.5. Factors Influencing Geological Information Extraction

There is a lot of geological coal seam information in the longwall face images. However, there are also many negative factors that affect the identification and extraction of the geological information, mainly including the following three factors:

• Coal mining dust and spray. Dust and spray attached to the camera lens have a severe impact on imaging quality, that is, the longwall face images become blurred. On the other hand, coal walls, rock walls, and supports will become less recognizable when dust is attached to these objects, especially for the coal and rock walls. In this case, the extraction of local coal–rock columnar information may result in significant errors.
• High-precision measurement of the local roof/floor inclination angle. The field test shows that the accuracy of longwall face geological surveying mainly depends on the roof line estimated with the high-precision measurement of the local roof/floor inclination angle.
• Overlap of highlights, shearers, and workers. The geological information in the image data naturally decreases when these objects block out the longwall face.

The above discussions are useful for improving off-site geological surveying methods and indicate the applicable conditions of the proposed method. Further investigation needs to be undertaken to guarantee the stable operation of off-site geological surveying. The above discussions show that the image quality determines the accuracy of geological surveying of the longwall face. Therefore, obtaining high-quality longwall face images and high-precision local roof/floor inclination angles are the key to improving the accuracy of geological surveying. In the future, several measures can be taken to improve the image quality and efficiency of automatic surveying, including a standardized workflow, dark image enhancement, self-cleaning camera, and automatic measurement of support space attitude. Meanwhile, the development of this technology can effectively deliver accurate geological information, providing new ideas for the future development of intelligent mining technology.

5. Conclusions

• Longwall face images capture dynamic geological information, such as coal–rock distribution, seam fluctuations, and geological structures. Roof line estimation based on hydraulic support pose measurements, together with image-based coal–rock height extraction, enables the reconstruction of longwall geological profiles with reasonable accuracy. However, factors like dust, poor visibility, and interference from equipment and workers hinder the extraction of these data.
• A method to estimate the roof line based on the inclination angle of hydraulic supports, which reflect the undulating shape of the longwall face, has been developed. The derived mathematical model shows that the maximum and average errors in roof line estimation are 0.25 m and 0.12 m, respectively, compared to manual measurements.
• Automatic extraction of coal–rock columns is achieved through image segmentation and proportional relationships between the image and the actual scene. A U-net-based model accurately identifies coal, rock walls, and support guarding plates. The height of the coal or rock wall above the interface is estimated using proportional equations, while the height below the interface is calculated based on the mining height. The field test shows an absolute error of 0.08 m in height estimation.
• A geological survey map is generated by integrating the estimated roof line and local coal–rock columns, with the accuracy depending mainly on the roof line accuracy.

This study combines longwall face images and hydraulic support data to create a roof line model and a U-net coal–rock segmentation algorithm. The method enables automatic height estimation and dynamic geological profile generation, improving the efficiency of geological information measurement and facilitating real-time updates of geological models. Achieving dynamic updates of geological models is of great significance.

Although the proposed method shows encouraging performance, further improvements are required to enhance its robustness and extend its applicability. In future research, active lighting and optimized camera configurations can be introduced to reduce the adverse effects of dust, occlusion, and low-light environments, thereby improving the stability of image-based segmentation. The generated geological profiles will also be integrated with borehole and geophysical data to enable continuous updates of high-resolution coal seam models, which can provide stronger support for intelligent and adaptive mining operations. Moreover, because the shearer does not always strictly follow the natural undulation of the coal seam, future efforts will focus on recognizing geological marker layers within the seam (such as bedding planes and partings) to extract local undulation and refine seam geometry representation. Collectively, these efforts will advance the proposed workflow toward a more comprehensive and high-fidelity geological modeling framework for intelligent coal mining.