# Automatic Coal and Gangue Segmentation Using U-Net Based Fully Convolutional Networks

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Input Data

#### 2.1. Raw Data Collection

#### 2.2. Data Preparation

#### 2.3. Data Augmentation

## 3. The Proposed Approach

#### 3.1. Network Architecture

#### 3.2. Model Training

#### 3.3. Evaluation Metrics

## 4. Results and Discussion

#### 4.1. Visual Results

#### 4.2. Network Performance

#### 4.3. Effects of Data Augmentation

#### 4.4. Impacts of Input Image Size

#### 4.5. Comparisons with Other Methods

## 5. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Wu, B.; Yang, M. Analysis on technical and economic policy of comprehensive utilization of coal ash and gangue in China. Energy China
**2012**, 11, 8–11. [Google Scholar] - Wang, H.; Wang, Y.; Zhao, X.; Wang, G.; Huang, H.; Zhang, J. Lane Detection of Curving Road for Structural High-way with Straight-curve Model on Vision. IEEE Trans. Veh. Technol.
**2019**, 68, 5321–5330. [Google Scholar] [CrossRef] - Tripathy, D.P.; Reddy, K.G.R. Novel Methods for Separation of Gangue from Limestone and Coal using Multispectral and Joint Color-Texture Features. J. Inst. Eng. Ser. D
**2017**, 98, 109–117. [Google Scholar] [CrossRef] - Hong, H.; Zheng, L.; Zhu, J.; Pan, S.; Zhou, K. Automatic Recognition of Coal and Gangue based on Convolution Neural Network. arXiv
**2017**, arXiv:1712.00720. [Google Scholar] - Su, L.; Cao, X.; Ma, H.; Li, Y. Research on Coal Gangue Identification by Using Convolutional Neural Network. In Proceedings of the 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control, Conference (IMCEC), Xi’an, China, 25–27 May 2018; IEEE: Piscataway, NJ, USA, 2018. [Google Scholar]
- Gao, K.; Du, C.; Wang, H.; Zhang, S. An efficient of coal and gangue recognition algorithm. Int. J. Signal Process. Image Process. Pattern Recognit.
**2013**, 6, 345–354. [Google Scholar] - Hou, W. Identification of coal and gangue by feed-forward neural network based on data analysis. Int. J. Coal Prep. Util.
**2019**, 39, 33–43. [Google Scholar] [CrossRef] - Liu, K.; Zhang, X.; Chen, Y. Extraction of Coal and Gangue Geometric Features with Multifractal Detrending Fluctuation Analysis. Appl. Sci.
**2018**, 8, 463. [Google Scholar] [CrossRef] [Green Version] - Sun, Z.; Lu, W.; Xuan, P.; Li, H.; Zhang, S.; Niu, S.; Jia, R. Separation of gangue from coal based on supplementary texture by morphology. Int. J. Coal Prep. Util.
**2019**, 1–17. [Google Scholar] [CrossRef] - Li, D.; Zhang, Z.; Xu, Z.; Xu, L.; Meng, G.; Li, Z.; Chen, S. An Image-Based Hierarchical Deep Learning Framework for Coal and Gangue Detection. IEEE Access
**2019**, 7, 184686–184699. [Google Scholar] [CrossRef] - Minar, M.; Naher, J. Recent advances in deep learning: An overview. arXiv
**2018**, arXiv:1807.08169. [Google Scholar] - Shahri, A.A.; Spross, J.; Johansson, F.; Larsson, S. Landslide susceptibility hazard map in southwest Sweden using artificial neural network. Catena
**2019**, 183, 104225. [Google Scholar] [CrossRef] - Ghaderi, A.; Shahri, A.A.; Larsson, S. An artificial neural network based model to predict spatial soil type distribution using piezocone penetration test data (CPTu). Bull. Eng. Geol. Environ.
**2019**, 78, 4579–4588. [Google Scholar] [CrossRef] [Green Version] - LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature
**2015**, 521, 436–444. [Google Scholar] [CrossRef] [PubMed] - Buslaev, A.; Seferbekov, S.S.; Iglovikov, V.; Shvets, A. Fully Convolutional Network for Automatic Road Extraction from Satellite Imagery. In Proceedings of the CVPR Workshops, Salt Lake City, UT, USA, 18–22 June 2018. [Google Scholar]
- Long, J.; Shelhamer, E.; Darrell, T. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5–9 October 2015; Springer: Berlin/Heidelberg, Germany, 2015. [Google Scholar]
- Wang, J.; Perez, L. The effectiveness of data augmentation in image classification using deep learning. arXiv
**2017**, arXiv:1712.04621. [Google Scholar] - Fabijańska, A. Segmentation of corneal endothelium images using a U-Net-based convolutional neural network. Artif. Intell. Med.
**2018**, 88, 1–13. [Google Scholar] [CrossRef] [PubMed] - Zhang, W.; Itoh, K.; Tanida, J.; Ichioka, Y. Parallel distributed processing model with local space-invariant interconnections and its optical architecture. Appl. Opt.
**1990**, 29, 4790–4797. [Google Scholar] [CrossRef] [PubMed] - Nair, V.; Hinton, G.E. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel, 21–24 June 2010. [Google Scholar]
- Zhang, Z.; Liu, Q.; Wang, Y. Road extraction by deep residual u-net. IEEE Geosci. Remote Sens. Lett.
**2018**, 15, 749–753. [Google Scholar] [CrossRef] [Green Version]

**Figure 1.**An overview of our deep learning-based coal and gangue segmentation technique. (

**a**) Training stage; (

**b**) Testing stage.

**Figure 2.**Data collection and gangue separating systems: (

**a**) Data collection and; (

**b**) gangue grabbing manipulator.

**Figure 6.**The results of gangue segmentation using the U-Net based approach: first column, testing images; second column, the manually labeled ground truth; third column, the probability maps generated by the trained model; and the last column; results overlaid on the original images.

**Figure 8.**The P-R curves and the ROC curves with their corresponding AUC values. (

**a**) The P-R curve; (

**b**) the ROC curve.

**Figure 10.**Performance of the trained model without data augmentation. (

**a**) The P-R curve; (

**b**) the ROC curve.

**Figure 11.**Performance of the trained model with an input image size $256\times 256$. (

**a**) The P-R curve; (

**b**) the ROC curve.

Ground Truth | |||
---|---|---|---|

Gangue (P) | Background (N) | ||

Predicted Result | Gangue (${\mathrm{P}}^{\prime}$) | True Positive (TP) | False Positive (FP) |

Background (${\mathrm{N}}^{\prime}$) | False Negative (FN) | True Negative (TN) | |

$\mathrm{ACCURACY}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{P}+\mathrm{N}}$$\mathrm{PRECISION}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$\mathrm{RECALL}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$\mathrm{TPR}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$\mathrm{FPR}=\frac{\mathrm{FP}}{\mathrm{FP}+\mathrm{TN}}$ |

Images | AUROC | AUPRC | Accuracy | Precision | Recall | IoU |
---|---|---|---|---|---|---|

Test01.png | 0.94 | 0.90 | 0.88 | 0.83 | 0.94 | 0.80 |

Test02.png | 0.98 | 0.99 | 0.93 | 0.97 | 0.81 | 0.79 |

Test03.png | 0.99 | 0.99 | 0.98 | 0.98 | 0.98 | 0.96 |

Test04.png | 0.94 | 0.90 | 0.90 | 0.83 | 0.99 | 0.82 |

Test05.png | 0.99 | 0.99 | 0.95 | 0.95 | 0.96 | 0.92 |

Test06.png | 0.96 | 0.97 | 0.93 | 0.90 | 0.91 | 0.82 |

Mean ($\sigma $) | 0.96 (2.13%) | 0.96 (4.07%) | 0.93 (3.24%) | 0.90 (6.19%) | 0.94 (6.04%) | 0.86 (6.44%) |

Without Augmentation | With Augmentation | |||
---|---|---|---|---|

Confusion matrix | 654,689 | 77,528 | 661,849 | 70,368 |

73,321 | 767,326 | 40,913 | 799,734 | |

Accuracy | 0.90 | 0.93 | ||

Precision | 0.89 | 0.90 | ||

Recall | 0.90 | 0.94 | ||

IoU | 0.81 | 0.86 |

Task | Predict Time | Train the Model | Physical Size Per Pixel |
---|---|---|---|

$512\times 512$ Input size | 48.2 ms per image | 5.8 h | 1.56 mm |

$256\times 256$ Input size | 18.5 ms per image | 1.8 h | 3.13 mm |

Methods | Dataset Size | Image Attributes | Coal and Gangue Position | Algorithm Category | Detection Accuracy |
---|---|---|---|---|---|

LeNet [3] | 20,000 | $224\times 224$ 8-bit | Single | Classification | 95.9% |

AlexNet [4] | 2012 | $224\times 224$ RGB | Single | Classification | 96.0% |

CG-RPN [10] | 2316 | $1024\times 1024$ RGB | Sparse | Classification with box | 98.3% |

Our method | 60 | $512\times 512$ 8-bit | Multiple heaped | Pixel-wise segmentation | 93.0% |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Gao, R.; Sun, Z.; Li, W.; Pei, L.; Hu, Y.; Xiao, L.
Automatic Coal and Gangue Segmentation Using U-Net Based Fully Convolutional Networks. *Energies* **2020**, *13*, 829.
https://doi.org/10.3390/en13040829

**AMA Style**

Gao R, Sun Z, Li W, Pei L, Hu Y, Xiao L.
Automatic Coal and Gangue Segmentation Using U-Net Based Fully Convolutional Networks. *Energies*. 2020; 13(4):829.
https://doi.org/10.3390/en13040829

**Chicago/Turabian Style**

Gao, Rong, Zhaoyun Sun, Wei Li, Lili Pei, Yuanjiao Hu, and Liyang Xiao.
2020. "Automatic Coal and Gangue Segmentation Using U-Net Based Fully Convolutional Networks" *Energies* 13, no. 4: 829.
https://doi.org/10.3390/en13040829