Research on Positioning Method of Coal Mine Mining Equipment Based on Monocular Vision

Abstract

In view of the insufficient characteristics and depth acquisition difficulties encountered in the process of uniocular vision measurement, the posture measurement scheme of tunneling equipment based on uniocular vision was proposed in this study. The positioning process of coal mine tunneling equipment based on monocular vision was proposed to extract the environmental features and match the features, and the RANSAC algorithm was used to eliminate the pair of mismatching points. This was done to solve the optimized matching pair and realize the pose estimation of the camera. The pose solution model based on the triangulation depth calculation method was proposed, and the PNP solution method was adopted based on the three-dimensional spatial point coordinates so as to improve the visual measurement accuracy and stability and lay the foundation for the 3D reconstruction of the roadway. This was done to simulate the downhole environment to build an experimental verification platform for monocular visual positioning. The experimental results showed that the position measurement accuracy of the uniocular visual roadheader positioning method within 60 mm and 1.3° could realize the accurate registration of the point cloud in the global coordinate system. The time required for the whole monocular visual positioning was only 179 ms, so it had good real-time performance.

1. Introduction

The global positioning of underground mine tunneling equipment is the key to realizing directional navigation, automatic cutting and collaborative control between equipment [1]. With the characteristics of limited underground operation space, numerous equipment and insufficient light [2], it is difficult to realize roadway visualization and equipment positioning, mainly due to the accurate detection and equipment positioning of the underground environment [3].

In view of the positioning problem of tunneling equipment, many research institutions and scholars have carried out fruitful research. Wu Miao China university’s mining team [4], with the help of boring machine fuselage posture measurement, by placing a three-target prism on the tunneling robot, using the whole station of three prism point observation constituting three spatial features and inputting the observed feature points into a computer for further processing, realized the boring machine in the roadway space pose solution. Zhang Xuhui of Xi’an University of Science and Technology used the whole station and the inertial guide for joint positioning and fused the measurement data of the whole station and the Kalman filter to realize the accurate positioning of the roadheader [5]. However, due to the simultaneous observation of the three-target prism, it is difficult to achieve measurement in real time using the measurement method based on the whole station, and due to the existence of large-scale underground station moving problems, it has not been widely used. Fu Shishen et al. put forward an initial autonomous positioning guidance method based on UWB positioning [6], and Liu Chao et al. put forward a hybrid algorithm of UWB, ranging and the TSOA principle to calculate the 3D spatial pose of the roadheader [7]. The above pose measurement method has deficiencies such as time accumulation error and unstable measurement accuracy. Reid P.B. [8] of Queensland Mining Technology Center in Australia applied inertial navigation to the offset detection of hydraulic supports in a fully mechanized mining face, providing a basis for the straightening of the scraper conveyor. The Australian research organization used inertial navigation to obtain the dynamic pose data of fully mechanized mining equipment and designed the LASC system [9] to measure the pose of fully mechanized mining equipment. KHONZI [10] applied inertial navigation to the autonomous positioning of underground equipment, but there is a large cumulative error in the real-time positioning of the equipment using inertial navigation.

The sensors adopted in visual measurement systems mainly include monocular vision, binocular vision and multiocular vision. By collecting the measured object or environmental information, the real-time solution of the equipment position is realized by using post-stage image processing technologies. With the help of a cross laser, Du Yuxin and others [11] established the features and the fuselage position so as to realize the measurement of a roadheader position in a tunneling roadway. Yang Wenjuan [12] set up two parallel laser lines behind the roadway of Xi’an University of Science and Technology and proposed a roadheader position solution model based on two points and three lines to realize the accurate measurement of the roadheader position. A tunneling-robot-positioning method based on binocular vision obtains roadway environmental features through a binocular vision sensor, establishes a relative relationship model between the tunneling robot and the roadway through collected roadway environmental image recognition information and calculates the relative roadway position of the tunneling robot [13]. Aiming at the problem of point cloud registration, Professor Besl of Stanford University in the United States proposed an iterative closest-point algorithm, namely the ICP algorithm [14]. Since then, there have been many improved algorithms based on ICP to achieve accurate point-cloud matching. For example, Zhang [15] used a k–d tree to accelerate the corresponding point search, which had good real-time performance. Senin Bouaziz et al. [16] used sparse representation to improve the real-time performance and accuracy of ICP.

With the continuous progress of computer technology, intelligent visual perception technology has been preliminarily applied to personnel monitoring and video recognition in underground coal mines. The visual SLAM method [17,18] has shown great potential in the field of robot localization. Therefore, this paper proposed a method of tunneling equipment positioning and environmental reconstruction based on monocular vision. The main contributions of this paper are as follows: (1) The positioning process of underground mine tunneling equipment based on uniocular vision was proposed. (2) The improved BRIEF (Binary Robust Independent Elementary Features) descriptor method was used to realize the feature point matching on two frames of images in a complex environment, and the location of the tunneling equipment was solved according to the method based on depth recovery. (3) The ORB (Oriented Fast and Rotated Brief) feature point [19] extraction method was adopted to realize the environmental 3D sparse-point cloud map construction.

The rest of this paper is arranged as follows: Section 2 describes the positioning process of tunnel equipment. The improved FAST (accelerated segmented test feature) corner extraction method combined with the improved BRIEF descriptor method is used to match the environmental feature points. RANSAC algorithm is used to remove the wrong matching points. The pole geometry method and triangulation depth recovery method are used to solve the pose of the tunneling robot. In Section 3, monocular visual feature extraction and three-dimensional reconstruction of roadway are verified through experiments, and then pose measurement is realized. In Section 4, this paper and the experiment are summarized.

2. Methods

2.1. Positioning Process of Tunneling Equipment Based on Monocular Vision

The underground tunneling face of a coal mine has the characteristics of low illumination, long and narrow channels and numerous pieces of equipment, which poses challenges to the precise positioning of tunneling equipment. Therefore, with the coal mine roadway environment as the feature, the tunneling equipment positioning method with uniocular vision as the core was constructed. The monocular camera and the tunneling equipment were firmly connected. By collecting the environmental images of the underground coal mine roadway, introducing the in-camera parameters with calibration in advance, the image preprocessing method was used to realize image enhancement, filtering and distortion correction, and then the ORB feature extraction method was used to extract the feature points, and the RANSAC algorithm was used to eliminate the mismatched points [20]. Finally, the pose estimation between adjacent frames was realized according to the principle of polar geometry constraints. The three-dimensional spatial coordinates of feature points were calculated by the triangulation depth calculation method. Then, the position information of the tunneling equipment was calculated in real time. By solving the pose of the tunneling robot, the 3D point-cloud registration between image sequences was realized, and sparse 3D space points were recovered from the collected images. Positioning process of coal mine heading equipment based on uniocular vision is shown in Figure 1.

2.2. Image Feature Extraction and Matching Method in Complex Background

2.2.1. Image Preprocessing

Due to the unstructured and complex environment of an underground coal mine and the hardware differences between the monocular cameras themselves, it was easy to produce image noise and image distortion in the image acquisition process. Further image processing was needed to realize the image feature extraction and calculation, so the image acquisition and pre-processing link was particularly critical. Image preprocessing mainly includes camera de-distortion and an image filtering process. In the process of image acquisition, a pinhole camera model will produce camera distortion, specifically divided into radial distortion, eccentric distortion and thin prism distortion. By establishing a mathematical model, camera lens distortion implements a camera lens distortion correction, while the interference of random signals around the image will cause noise on the image, so the filtering effect has a direct impact on the effectiveness and reliability of the visual measurement system to ensure the extraction and calculation of spot features.

Distortion Correction

The flaw on the camera lens was the main cause of the radial distortion. Due to its particularity, the closer to the outside, the more diffuse or concentrated inside. Let the tangential distortion be ${\delta }_{xr, } {\delta }_{yr}$, thus the mathematical model is shown in Equation (1):

$$\{\begin{array}{l}{\delta }_{xr}=x(k_1{(x+y)}^2+k_2{(x+y)}^4+\cdot \cdot \cdot \\ {\delta }_{yr}=y(k_1{(x+y)}^2+k_2{(x+y)}^4+\cdot \cdot \cdot \end{array}$$

(1)

where the coefficient k₁, k₂… represents the radial distortion coefficient.

Both radial and tangential distortion were included, mainly because the camera’s optical center and geometric center did not coincide. Let the eccentric distortion be A, thus the mathematical model is shown in Equation (2):

$$\{\begin{array}{l}{\delta }_{xd}=p_1(3x^2+y^2)+2p_{2}xy+\cdot \cdot \cdot \\ {\delta }_{yd}=2p_{1}xy+p_2(3x^2+y^2)+\cdot \cdot \cdot \end{array}$$

(2)

An effective camera model with all distortion types could not be established, so we considered integrating nonlinear models in the camera calibration process. Too many nonlinear parameters will cause instability in the solution, so at most, two distortion factors were considered, and the improvement method of fusion radial distortion and eccentric distortion was adopted.

For the above lens distortion analysis, the correction model combining the two models is:

$$\{\begin{array}{l}{\delta }_x(x,y)={\delta }_{xr}+{\delta }_{xd}\\ {\delta }_y(x,y)={\delta }_{yr}+{\delta }_{yd}\end{array}$$

(3)

Figure 2 shows the image before and after the distortion correction. It can be seen that the monocular camera had a more obvious pillow-shaped distortion, which could better recover the real scene after the correction.

Gaussian Filtering

Gaussian filtering builds mathematical models to transform the energy of the image data and filter out the low-energy part of the noise. The 3 × 3 Gaussian filter template is shown in Equation (4):

$$\frac{1}{16}\times \left[\begin{array}{l}1\hspace{1em}2\hspace{1em}1\\ 2\hspace{1em}4\hspace{1em}2\\ 1\hspace{1em}2\hspace{1em}1\end{array}\right]$$

(4)

Based on the selected kernel size k and the size of the weight coefficient, the weighted average is calculated by the method of each pixel multiplied by the associated weight with the weighted average instead of the current pixel value. The weight coefficients are obtained from a one-dimensional Gaussian function.

$$G\left(x\right)=\frac{1}{\sqrt{2\pi {\sigma }^2}}\exp \left(\frac{-x^2}{2{\sigma }^2}\right)$$

(5)

In Equation (5), σ represents the standard deviation and x represents the pixels in the image. For 2D images, the 2D Gaussian function is established as:

$$G\left(\mu ,v\right)=\frac{1}{\sqrt{2\pi {\sigma }^2}}\exp \left(\frac{-\left({\mu }^2+v^2\right)}{2{\sigma }^2}\right)$$

(6)

In Equation (6), A represents the standard deviation, B represents the pixel abscissa in the image and C represents the pixel ordinate in the image.

By making σ = 5 and the Gaussian filter core k = 3, the acquired image can be filtered to compare the filtered image with the pre-filtered image. The comparison results are shown in Figure 3.

2.2.2. Feature Extraction

Due to the difficulty of estimating the camera motion directly from the matrix changes, selecting the appropriate feature extraction algorithm was crucial for image matching and the pose solution calculation. The feature points included corners, blocks and edges, etc. The ORB feature extraction [21] belonged to diagonal feature extraction, and the feature point description was performed by improving the FAST feature extraction and the BRIEF descriptor algorithm. The experiments showed that ORB performed better than the BRIEF and SURF methods. By analyzing the feature extraction method, the ORB feature extraction principle could realize fast feature extraction and description, so the ORB feature extraction method was adopted.

Improved FAST Angle-Point Extraction

A pixel point p was first selected in the image and its pixel brightness was H_p. The judgment threshold for corner points was further set at 20% of ${\mathrm{H}}_{\mathrm{p}}$. Then, the pixel p was selected as the center point according to the setting threshold, and 16 pixels were selected on a neighborhood circle with a radius of 3 pixels.

According to the above selection principle, the feature point was judged. If the brightness T of N points on the neighborhood circle met $\mathrm{T}>{\mathrm{H}}_{\mathrm{p}}+20{\%\mathrm{H}}_{\mathrm{p}} \mathrm{or} \mathrm{T}<{\mathrm{H}}_{\mathrm{p}}+20{\%\mathrm{H}}_{\mathrm{p}}$, the p point could be regarded as the corner point, while it was not the characteristic corner point. The selection was cyclic according to the above selection principle, and the image was traversed until the FAST angles that met the requirements were extracted.

Further, the improved FAST method in ORB achieved scale invariance by constructing image pyramids and detecting corners at each layer of the pyramid and realized its rotation invariance with the help of the grayscale centroid-of-mass method.

The specific implementation process of the grayscale centroid method is as follows:

The moment of defining an image block in an image block B is:

$$m_{pq}={\displaystyle \sum _{x,y\in B}{x}^p{y}^q}I(x,y),p,q\in \left\{0,1\right\}$$

(7)

In the formula, x, y represents the pixel coordinates, and I (x, y) represents the gray value of the pixel coordinates.

The image block center of mass is further obtained by moment calculation:

$$C=\left(\frac{m_{10}}{m_{00}},\frac{m_{01}}{m_{00}}\right)$$

(8)

The direction vector $\overrightarrow{\mathrm{OC}}$ is obtained from the geometric center and center of mass C of the image block B and can be used to represent the feature point direction:

$$\theta =\arctan \left(\frac{m_{01}}{m_{10}}\right)$$

(9)

The above improved FAST algorithm can describe the FAST, which improves the robustness of the algorithm.

Improving on the BRIEF Descriptor

After obtaining the feature points, N pairs of points are selected around the key points in the form of BRIEF. The key point P is selected as the center, and the radius is d, and n points are randomly selected in a certain neighborhood range for comparison. The selection conditions are as follows:

Suppose A, B, H_a, H_b are the gray scale of A, B,

$$F(P(A,B))=\{\begin{array}{l}0\hspace{1em}{H}_a>H_b\\ 1\hspace{1em}{H}_a\le H_b\end{array}$$

(10)

The n-point pair (x_i, y_i) is defined as the 2 × n matrix S according to the above selection conditions:

$$S=\left[\begin{array}{ccc}{x}_1& x_2& \cdot \cdot \cdot x_n\\ y_1& y_2& \cdot \cdot \cdot y_n\end{array}\right]$$

(11)

The obtained θ pair S matrix rotation is used to obtain:

$$S_{\theta }=R_{\theta }S$$

(12)

In the formula, R_θ is the rotation matrix of the angle θ.

Thus, ORB feature point extraction and descriptor establishment are realized. Through the established descriptor of feature extraction, the robot position acquisition was then realized by the position solution algorithm. However, because the perspective changed greatly or had similar regions, the mismatching situation was prone in the process of feature matching, so the mismatching situation needed to be eliminated.

2.2.3. Mismatch Elimination of the RANSAC Algorithm

The feature matching process is a particularly critical step in the visual SLAM algorithm, but there is a mismatching situation, so the RANSAC algorithm was used to eliminate the mismatching. The specific steps of the RANSAC algorithm are as follows:

• Data from s samples are randomly drawn from the dataset, multiple models are fit and the most suitable unistress matrix H_3×3 is calculated according to the principle of the most matrix data points.

$$s\left[\begin{array}{l}{x}^{’}\\ y^{’}\\ z^{;}\end{array}\right]=\left[\begin{array}{l}{h}_{11}\hspace{1em}{h}_{12}\hspace{1em}{h}_{13}\\ h_{21}\hspace{1em}{h}_{22}\hspace{1em}{h}_{32}\\ h_{31}\hspace{1em}{h}_{32}\hspace{1em}{h}_{33}\end{array}\right]\left[\begin{array}{l}x\\ y\\ z\end{array}\right]$$

(13)

As in Equation (13), make h₃₃ = 1 and normalize the matrix H_3×3. Since there are eight unknowns in the matrix, it should be solved by at least four sets of matching points in the extracted image.

2.

Calculate the projection errors of all the data models H in the dataset and set the threshold value. If the error is less than the threshold, it meets the requirements, and it is added to the inner point set I, and the data greater than the threshold value are removed. The projection error function is:

$${\displaystyle \sum _{i=1}^n{\left(x_i^{’}-\frac{h_{11}{x}_i+h_{12}{y}_i+h_{13}}{h_{31}{x}_i+h_{32}{y}_i+h_{33}}\right)}^2+{\left(y_i^{’}-\frac{h_{21}{x}_i+h_{22}{y}_i+h_{23}}{h_{31}{x}_i+h_{32}{y}_i+h_{33}}\right)}^2}$$

(14)

3.

If the number of inner points is insufficient, it is necessary to reselect the 4 pairs of matching points in the image and calculate the model H until the number of inner point sets exceeds the selected optimal inner point set threshold;

4.

If the number of elements of the inner point set I is greater than the optimal inner point set condition, the number of inner points is recorded, and the iteration number k is updated;

5.

If the number of iterations is greater than the maximum number K, the iteration ends; otherwise resume steps 1−5. The maximum number of iterations, K, is equal to:

$$K=\frac{\log (1-P)}{\log 1-w^m}$$

(15)

In Equation (15), the confidence degree of P is made to be 0.0995, w the “inner point” ratio and m the minimum number of samples required to calculate the model.

2.3. Pose Position Solution Based on Depth Recovery

2.3.1. Position and Pose Measurement of Roadheader Robot

According to the extraction matching point for the camera pose estimation, as shown in Figure 4 below, good feature points were paired from two pictures according to the feature points in the two-dimensional image correspondence, the camera image between the frame pose was restored, and the camera posture solution was preliminarily completed in order to realize the uniocular visual pose measurement based on the feature point depth.

Assuming the two frames of images are C₁ and C₂ during the camera motion, the motion from images C₁ to C₂ is R t, and the two cameras are centered at O₁ and O₂, respectively. p₁ and p₁ are the corresponding feature points obtained by matching the P points between the two images, respectively, and O₁p₁ and O₂p₂ are connected to obtain the rays $\overrightarrow{O_1{p}_1}$ and $\overrightarrow{O_2{p}_2}$ intersecting at the point P.

The plane composed of the three points of O₁, O₂ and P is called the polar plane, and the intersection between O₁, O₂ and the image plane C₁, C₂ is e₁ and e₂, respectively. e₁ and e₂ are the poles, and O₁O₂ is the baseline.

The intersection lines l₁ and l₂ between the pole plane and the two image planes C₁ and C₂ are the pole line.

The coordinates of this spatial point P can be expressed as:

$$P={\left[x,y,z\right]}^T$$

(16)

According to the camera imaging model, the two-pixel p₁·p₂ locations are as follows:

$$s_1{p}_1=k_{2}p,s_2{p}_2=k_2(RP+t)$$

(17)

In Equation (17), k₂ is the in-camera parameter, R, and t is the rotation matrix, and the translation matrix of the camera motion multiplied by the non-zero constant can be reduced to:

$$p_1=k_{2}p,p_2=k(RP+t)$$

(18)

Let the pixel point normalized plane coordinate $x_1=k_2^{-1}{p}_1,x_2=k_2^{-1}{p}_2$, bring x₁, x₂ into Formula (18), and multiply by $x_2^T{t}^{\wedge }$ on both sides:

$$x_2^T{t}^{\wedge }{x}_2=x_2^T{t}^{\wedge }R{x}_1$$

(19)

In Equation (19), the $t^{\wedge }{x}_2$ vector is perpendicular to t and x₂, so it is 0, then:

$$x_2^T{t}^{\wedge }R{x}_1=0$$

(20)

Bring p₁ and p₂ in again with:

$$p_2^T{k}_2^{-T}{t}^{\wedge }R{k}_2^{-1}{p}_1=0$$

(21)

In the formula, p₁, p₂ are the corresponding feature points obtained by matching P points between two images, and k₂ is the in-camera parameter.

The above Equation (20) and Equation (21) are collectively referred to as the polar constraint, indicating the geometric meaning of the O₁, P and O₂ three common planes, and by including the rotation matrix R and translation matrix t, the formula can be obtained:

$$x_2^{T}E{x}_1=p_2^{T}F{p}_1=0$$

(22)

E = t^R is the essential matrix and A is the basic matrix, so the camera pose solution calculation can be simplified as follows: solve the essential matrix E or the basic matrix F according to the pixel position of the pairing point, and then solve the camera pose R and t combined with the solved E or F substitution Formula (22).

According to the principle of monocular visual measurement analysis, one cannot obtain pixel depth information through only one single picture, so this needs to be conducted by triangulation. According to the frame between postures, to extract the roadway characteristic point depth estimation, triangulation between two perspective angles and the realization of the point distance calculation is required, thus the measurement principle is as shown in Figure 5:

According to the two frames in Figure 5, the two images are C₁ and C₂, with the left being C₁ for the previous moment and the right being moment C₂, p₁ and p₂, respectively, show the P point in the pixel coordinates in the previous and later point, and O₁ and O₂ are the center of the camera, and the two-image camera posture transformation matrix R and t, and the calibration in the camera parameters for k₂ are obtained according to the visual odometer. According to the definition of polar geometry, the two points p₁ and p₂ of the normalized plane can satisfy the following relationship:

$$d_1{p}_1=d_{2}R{p}_2+t$$

(23)

where d₁ and d₂ represent the camera depth value of the point P at the previous moment and the second moment, respectively. Since R and t are solved according to the visual odometer, Equation (23) is written in a matrix form:

$$\left[\begin{array}{cc}-R·p_1& p_2\end{array}\right]\left[\begin{array}{c}{d}_1\\ d_2\end{array}\right]=t$$

(24)

Further, Formula (24) is written as a system of linear equations such as:

$$Ax=b$$

(25)

Among A = [−R · p₁ p₂], x = [d₁ d₂]^T, b = t.

The optimization function can be built to solve the depth:

$$f(x)=\Vert Ax-b\Vert$$

(26)

According to the above formula, the x result can be obtained as follows:

$$x={\left(A^{T}A\right)}^{-1}{A}^{T}b$$

(27)

The matrix $A^{T}A$ is a non-singular matrix, and the feature point depth solution of R and t implemented for the extracted roadway images is obtained by the combination of the visual odometer calculation. In addition, when A is a singular matrix, the LM algorithm is used to transform A into a non-singular matrix before solving it. After the depth value solution is realized, the tunneling robot pose solution based on the depth recovery can be realized through the 3D–2D method.

2.3.2. Position and Pose Measurement of Roadheader Robot

The monocular visual odometer solves the camera pose by matching the correspondence between the features, namely, the rotation matrix R and the translation matrix t. The general solution methods are 2D–2D as well as 2D–3D. Since the spatial coordinates of the feature points are found according to the method of triangulation depth, the 3D–2D method, namely, the PNP algorithm, was used to use the matching relationship between the recovered feature points’ 3D-point coordinates and the image feature points to optimize and minimize the camera attitude according to the reprojection error.

The spatial coordinates of the same feature point $X_i^k$ and $X_i^{k-1}$ in the two frame pictures at the ck and ck-1 moment camera frame where the rotation matrix is R and the flat matrix is t. According to the motion relationship of different frame images, the following equation is established:

$$X_i^k=R\ast X_i^{k-1}+t$$

(28)

For the feature points recovered by triangulation, $X_i^k$ is normalized, where ${\overline{X}}_i^k$ is the normalized coordinate of $X_i^k$ and $z_i^k$ is the modular length of $X_i^k$ in the z-axis direction, and thus this is obtained according to the above formula:

$$X_i^k=R\ast X_i^{k-1}+t$$

(29)

The transformation matrices $R$ and $t$ are decomposed into three row vectors, R_h and t_h, where $h\in \left\{1,2,3\right\}$, and this is brought into the upper formula, eliminating $z_i^k$ by the elimination method and obtaining the following system of equations:

$$\{\begin{array}{l}\left(R_1-{\overline{X}}_i^k{R}_1\right)X_i^{k-1}+t_1-{\overline{X}}_i^k{t}_3=0\\ \left(R_2-{\overline{X}}_i^k{R}_3\right)X_i^{k-1}+t_1-{\overline{X}}_i^k{t}_3=0\end{array}$$

(30)

According to the above Equation (30), through at least six pairs of matching points, the linear solution of the pose transformation matrix T can be realized, that is, the direct linear transformation method can be realized.

3. Results and Discussion

For the roadway environment characteristics texture, the tunneling robot tunnel environment perception is difficult. Through the study of visual feature extraction, using the ORB feature point extraction method of 3D sparse point cloud map construction, the 3D reconstruction method based on single eye visual SLAM was verified. For the corridor environment simulation roadway, the experimental verification platform was set to evaluate the effectiveness and real-time performance of the method.

Based on the uniocular vision tunneling robot 3D reconstruction experiment, in the corridor control robot for continuous motion, using the monocular visual image feature extraction matching method and 3D reconstruction method of the tunneling robot in the running image processing and solution, the function points, by calculating the global coordinate system of the 3D point cloud and the results in the visual interface, the uniocular vision roadway 3D reconstruction effect was completed and the sparse 3D point map was evaluated. In order to verify the reliability of the monocular visual feature extraction and 3D reconstruction, the 3D reconstruction experiment of the tunneling robot was carried out in a corridor, and the picture information of the roadway was collected immediately during the movement process, and the feature point matching and point cloud registration were completed. As shown in Figure 6, some pictures of feature points in the roadway were extracted during the robot’s movement.

Figure 6 shows the matching process of four adjacent frames. It can be seen that in the dark corridor environment, the roadway feature extraction effect obtained by the ORB feature extraction method is represented in color in the figure. After mismatching and elimination, fewer feature points were extracted to meet the requirements of the ORB feature extraction. Based on the feature point extraction, the feature point matching between each picture frame was realized, and the average matching time of each of the two frames was 179 ms, which met the real-time requirements of feature extraction and matching.

Figure 7 shows the process of the 3D sparse-point cloud-map-mapping process for the monocular visual SLAM realized with the help of the ORB feature extraction.

The green part of the feature extraction interface is the extracted and matched feature points of the corridor and corridor, the green part of the map state interface is the track of the robot and the red points are the three-dimensional sparse roadway points under construction. It can be seen that the visual feature points extracted on each frame picture were relatively rich, and the robot’s movement track was relatively stable with good stability, which is not easy to obtain in feature loss.

Based on the above feature extraction and matching process, the optimized corridor 3D sparse-point cloud map was obtained. Figure 8a shows the top view of the corridor, and Figure 8b shows the construction of the corridor. It can be seen that the trend was roughly consistent with the direction of the corridor, and the roadway was constructed using the uniocular visual SLAM method based on ORB. Due to the small amount of feature extraction and mismatching elimination, the sparse 3D reconstruction of the tunnel was thus realized.

Figure 8a–c shows the constructed three views of the sparse-point cloud for the three-dimensional roadway realized by monocular vision, respectively. The black point cloud shows the roadway feature points extracted and registered, the green line is the robot track, the green box is the real-time pose of the robot and the red points are the feature points under construction. The global map of the roadway composed of sparse point cloud could be found by overlooking, from a side view and face view, which was roughly in line with the roadway trend, and the three-dimensional roadway reconstruction based on the sparse-point cloud was initially realized.

Using the monocular visual SLAM method based on ORB feature extraction, the position measurement accuracy was within ±60 mm, and the angle measurement error was within ±1.3°. The robot position and the boundary perception of the point cloud in the global coordinate system in the system was obtained from the real-time acquisition of the ORB-based feature image reconstruction time, which was 179 ms.

4. Conclusions

According to the process of monocular visual SLAM, this paper introduced in detail many processes, such as image preprocessing, feature extraction, feature matching and mismatching elimination, pose solution calculation and sparse 3D point-cloud reconstruction. This paper also simulated the experimental verification platform of the downhole environment and set up a platform for the verification of the monocular visual positioning in the laboratory.

The experimental results showed that the position measurement accuracy of the uniocular visual roadheader positioning method was within ±60 mm and the angle measurement error was within ±1.3°, and that it could realize the accurate registration of the point cloud under the global coordinate system. The required time of the whole monocular visual positioning was only 179 ms, so it had good real-time performance. However, there was still a certain gap between the environmental simulation and the real excavation face in the experimental verification stage. Coal walls on both sides of an underground coal mine roadway are not very smooth, and many large pieces of equipment would also have a certain influence on the field of view of the monocular camera, so this will be continuously improved in further research.