# A Multivariate Long Short-Term Memory Neural Network for Coalbed Methane Production Forecasting

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

_{2.5}concentrations in air pollution, achieving valuable results [22,23,24,25]. In terms of public transportation, Chen et al., Tian et al., and Li and Cao [26,27,28] used the LSTM model to study traffic flow. In [29], LSTMs were used to forecast traffic speed in the Beijing area. Petersen et al. [30] used CNNs (Convolutional Neural Networks) and LSTMs to predict bus travel time. In the financial sector, Fischer and Krauss [31] and Kim and Won [32] applied the LSTM model to market forecasts and achieved superior results to those of random forests methods and deep neural networks (DNNs). Vochozka et al. [33] used the LSTM model to establish a method for predicting company bankruptcy, which provided a reference for the company’s future development. In the industrial and energy sectors, Wu et al. [34] used the LSTM network to estimate the remaining service life of an engineering system, while Peng et al. [35] used LSTMs and differential evolution to predict the price of electricity, and the prediction accuracy was superior to current models. Sagheer and Kotb [36] proposed a genetic algorithm-optimized deep LSTM method to predict oil production, which has proven to be more accurate than statistical and software calculation methods. Although the LSTM model has been widely used in research related to production and price prediction, it is rarely applied in the research of unconventional gas reservoirs. Xu et al. [37] used LSTM networks to predict the production of coalbed methane and achieved good results. However, this study did not consider the influence of multiple factors and only used coalbed methane production data.

## 2. Data and Methods

#### 2.1. Data Description

_{norm}is the normalized value, min(y

_{i}) is the smallest value of the dataset, and max(y

_{i}) is the maximum value of the dataset.

#### 2.2. M-LSTM NN Model

#### 2.3. Evaluation Index

_{i}represents the model predicted value, A

_{i}represents the true value, and n is the number of days of testing.

#### 2.4. Network Architecture

## 3. Results and Discussion

#### 3.1. Prediction Performance

^{2}value between the predicted and actual CBM production indicated that 94.4% of the variance was successfully described by the M-LSTM NN model.

#### 3.2. Comparison with Traditional LSTM NN Model (without Multivariate Inputs)

#### 3.3. Multi-Step Predictions

## 4. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## Appendix A. RNN

_{t}, O

_{t}, and S

_{t}represent the input, memory, and output, respectively, at time t. W, U, and V represent the weights between the input layer, hidden layer, and output layer, respectively. Because the weights between different layers of RNNs are shared, the network parameters can be greatly reduced. RNNs may cause gradient disappearance or gradient explosion when calculating connections between nodes with a long time interval [35]. Therefore, RNNs can only deal with short-term problems and cannot solve long-term dependence problems. CBM production is related over a long period of time, so RNNs cannot be used directly to predict it.

## Appendix B. LSTM NN

_{t}, h

_{t}, and C

_{t}respectively. The memory state C

_{t}is defined as follows:

_{t}is the forget gate, i

_{t}is the input gate, and W and b are the corresponding weights and offsets.

_{t}is as follows:

## References

- Stopa, J.; Mikolajczak, E. Empirical modeling of two-phase CBM production using analogy to nature. J. Pet. Sci. Eng.
**2018**, 171, 1487–1495. [Google Scholar] [CrossRef] - Aminian, K.; Ameri, S. Predicting production performance of CBM reservoirs. J. Nat. Gas Sci. Eng.
**2019**, 1, 25–30. [Google Scholar] [CrossRef] - Clarkson, C.R. Production data analysis of unconventional gas wells: Review of theory and best practices. Int. J. Coal Geol.
**2013**, 109, 101–146. [Google Scholar] [CrossRef] - Clarkson, C.R. Production data analysis of unconventional gas wells: Workflow. Int. J. Coal Geol.
**2013**, 109–110, 147–157. [Google Scholar] [CrossRef] - King, G.R. Material-balance techniques for coal-seam and devonian shale gas-reservoirs with limited water influx. SPE Reserv. Eng.
**1993**, 8, 67–72. [Google Scholar] [CrossRef] - Lue, Y.; Tang, D.; Xu, H.; Tao, S. Productivity matching and quantitative prediction of coalbed methane wells based on BP neural network. Sci. China Technol. Sci.
**2011**, 54, 1281–1286. [Google Scholar] [CrossRef] - Huang, X.D.; Wang, S.Q. Prediction of Bottom-Hole Flow Pressure in Coalbed Gas Wells Based on GA Optimization SVM; IEEE: New York, NY, USA, 2018. [Google Scholar]
- Fetkovich, M.J. Decline curve analysis using type curves. J. Pet. Technol.
**1980**, 32, 1065–1077. [Google Scholar] [CrossRef] - Jang, H.; Kim, Y.; Park, J.; Lee, J. Prediction of production performance by comprehensive methodology for hydraulically fractured well in coalbed methane reservoirs. Int. J. Oil Gas Coal Technol.
**2019**, 20, 143–168. [Google Scholar] [CrossRef] - Li, P.; Hao, M.; Hu, J.; Ru, Z.; Li, Z. A new production decline model for horizontal wells in low-permeability reservoirs. J. Pet. Sci. Eng.
**2018**, 171, 340–352. [Google Scholar] [CrossRef] - Cipolla, C.L.; Lolon, E.P.; Erdle, J.C.; Rubin, B. Reservoir Modeling in Shale-Gas Reservoirs. SPE Reserv. Eval. Eng.
**2010**, 13, 638–653. [Google Scholar] [CrossRef] - Li, L.C.; Wei, C.T.; Qi, Y.; Cao, J.; Wang, K.Y.; Bao, Y. Coalbed methane reservoir formation history and its geological control at the Shuigonghe Syncline. Arab. J. Geosci.
**2015**, 8, 619–630. [Google Scholar] [CrossRef] - Zhou, F.D. History matching and production prediction of a horizontal coalbed methane well. J. Pet. Sci. Eng.
**2012**, 96–97, 22–36. [Google Scholar] [CrossRef] - Thararoop, P.; Karpyn, Z.T.; Ertekin, T. Development of a multi-mechanistic, dual-porosity, dual-permeability, numerical flow model for coalbed methane reservoirs. J. Nat. Gas Sci. Eng.
**2012**, 8, 121–131. [Google Scholar] [CrossRef] - Yun, M.G.; Rim, M.W.; Han, C.N. A model for pseudo-steady and non-equilibrium sorption in coalbed methane reservoir simulation and its application. J. Nat. Gas Sci. Eng.
**2018**, 54, 342–348. [Google Scholar] [CrossRef] - Shi, J.T.; Chang, Y.C.; Wu, S.G.; Xiong, X.Y.; Liu, C.; Feng, K. Development of material balance equations for coalbed methane reservoirs considering dewatering process, gas solubility, pore compressibility and matrix shrinkage. Int. J. Coal Geol.
**2018**, 195, 200–216. [Google Scholar] [CrossRef] - Sun, Z.; Shi, J.; Zhang, T.; Wu, K.; Miao, Y.; Feng, D.; Li, X. The modified gas-water two phase version flowing material balance equation for low permeability CBM reservoirs. J. Pet. Sci. Eng.
**2018**, 165, 726–735. [Google Scholar] [CrossRef] - Xia, H.M.; Qin, Y.C.; Zhang, L.J.; Cao, Y.P.; Xu, J.X. Forecasting of Coalbed Methane (CBM) Productivity Based on Rough Set and Least Squares Support Vector Machine. In Proceedings of the 2017 25th International Conference on Geoinformatics, Buffalo, NY, USA, 2–4 August 2017. [Google Scholar]
- Zhao, C.; Li, J. Equilibrium Selection under the Bayes-Based Strategy Updating Rules. Symmetry
**2020**, 12, 739. [Google Scholar] [CrossRef] - Zhang, B.; Xu, D.; Liu, Y.; Li, F.; Cai, J.; Du, L. Multi-scale evapotranspiration of summer maize and the controlling meteorological factors in north China. Agric. For. Meteorol.
**2016**, 216, 1–12. [Google Scholar] [CrossRef] - Zhang, Y.; Huang, P. Influence of mine shallow roadway on airflow temperature. Arab. J. Geosci.
**2020**, 13, 12. [Google Scholar] [CrossRef] - Bai, Y.; Zeng, B.; Li, C.; Zhang, J. An ensemble long short-term memory neural network for hourly PM2.5 concentration forecasting. Chemosphere
**2019**, 222, 286–294. [Google Scholar] [CrossRef] - Li, X.; Peng, L.; Yao, X.; Cui, S.; Hu, Y.; You, C.; Chi, T. Long short-term memory neural network for air pollutant concentration predictions: Method development and evaluation. Environ. Pollut.
**2017**, 231, 997–1004. [Google Scholar] [CrossRef] [PubMed] - Wen, C.; Liu, S.; Yao, X.; Peng, L.; Li, X.; Hu, Y.; Chi, T. A novel spatiotemporal convolutional long short-term neural network for air pollution prediction. Sci. Total Environ.
**2019**, 654, 1091–1099. [Google Scholar] [CrossRef] [PubMed] - Zhao, J.; Deng, F.; Cai, Y.; Chen, J. Long short-term memory—Fully connected (LSTM-FC) neural network for PM2.5 concentration prediction. Chemosphere
**2019**, 220, 486–492. [Google Scholar] [CrossRef] [PubMed] - Chen, W.; Zheng, Z.; Liu, J.; Chen, P.; Wu, X. LSTM network: A deep learning approach for short-term traffic forecast. IET Intell. Transp. Syst.
**2017**, 11, 68–75. [Google Scholar] - Tian, Y.; Zhang, K.; Li, J.; Lin, X.; Yang, B. LSTM-based traffic flow prediction with missing data. Neurocomputing
**2018**, 318, 297–305. [Google Scholar] [CrossRef] - Li, Y.; Cao, H. Prediction for Tourism Flow based on LSTM Neural Network. In 2017 International Conference on Identification, Information and Knowledge in the Internet of Things, Qufu, China, 19–21 October 2017; Qufu Normal University, School of Information Science and Engineering: Shandong, China, 2018; Volume 129, pp. 277–283. [Google Scholar]
- Ma, X.; Tao, Z.; Wang, Y.; Yu, H.; Wang, Y. Long short-term memory neural network for traffic speed prediction using remote microwave sensor data. Transp. Res. Part C Emerg. Technol.
**2015**, 54, 187–197. [Google Scholar] [CrossRef] - Petersen, N.C.; Rodrigues, F.; Pereira, F.C. Multi-output bus travel time prediction with convolutional LSTM neural network. Expert Syst. Appl.
**2019**, 120, 426–435. [Google Scholar] [CrossRef] [Green Version] - Fischer, T.; Krauss, C. Deep learning with long short-term memory networks for financial market predictions. Eur. J. Oper. Res.
**2018**, 270, 654–669. [Google Scholar] [CrossRef] [Green Version] - Kim, H.Y.; Won, C.H. Forecasting the volatility of stock price index: A hybrid model integrating LSTM with multiple GARCH-type models. Expert Syst. Appl.
**2018**, 103, 25–37. [Google Scholar] [CrossRef] - Vochozka, M.; Vrbka, J.; Suler, P. Bankruptcy or Success? The Effective Prediction of a Company’s Financial Development Using LSTM. Sustainability
**2020**, 12, 7529. [Google Scholar] [CrossRef] - Wu, Y.; Mei, Y.; Dong, S.; Li, L.; Liu, Y. Remaining useful life estimation of engineered systems using vanilla LSTM neural networks. Neurocomputing
**2018**, 167–179. [Google Scholar] [CrossRef] - Peng, L.; Liu, S.; Liu, R.; Wang, L. Effective long short-term memory with differential evolution algorithm for electricity price prediction. Energy
**2018**, 162, 1301–1314. [Google Scholar] [CrossRef] - Sagheer, A.; Kotb, M. Time series forecasting of petroleum production using deep LSTM recurrent networks. Neurocomputing
**2019**, 323, 203–213. [Google Scholar] [CrossRef] - Xu, X.X.; Rui, X.P.; Fan, Y.L.; Yu, T.; Ju, Y.W. Forecasting of Coalbed Methane Daily Production Based on T-LSTM Neural Networks. Symmetry
**2020**, 12, 861. [Google Scholar] [CrossRef] - Li, C.; Wang, Z.; Rao, M.; Belkin, D.; Song, W.; Jiang, H.; Xia, Q. Long short-term memory networks in memristor crossbar arrays. Nat. Mach. Intell.
**2019**, 1, 49–57. [Google Scholar] [CrossRef] - Bergmeir, C.; Benítez, J.M. On the use of cross-validation for time series predictor evaluation. Inf. Sci.
**2012**, 191, 192–213. [Google Scholar] [CrossRef]

**Figure 1.**The production data. (

**a**) is the coalbed methane (CBM) production, (

**b**) is the casing pressure, (

**c**) is the water production, (

**d**) is the lowest bottom hole temperature, and (

**e**) is the highest bottom hole temperature.

**Figure 2.**The framework of the multivariate long short-term memory (M-LSTM) model for CBM daily production forecasting. The input variables (CBM production, casing pressure, and the rest) are included in a blue solid box, where n indicates the time lag. The stacked LSTM layers are included in a green dashed box. A recursive arrow indicates that processing can be repeated.

**Figure 5.**Comparison of error distribution between the M-LSTM NN model and the traditional LSTM NN model. (

**a**) RMSE, (

**b**) MAE and (

**c**) MAPE.

Nodes | RMSE (m^{3}) | MAE (m^{3}) | MAPE (%) |
---|---|---|---|

32 | 81.84 | 50.11 | 1.50 |

64 | 79.30 | 46.85 | 1.41 |

128 | 75.77 | 41.41 | 1.24 |

256 | 80.81 | 50.02 | 1.53 |

512 | 84.46 | 52.62 | 1.58 |

Learning Rate | RMSE (m^{3}) | MAE (m^{3}) | MAPE (%) |
---|---|---|---|

0.005 | 142.42 | 121.87 | 3.69 |

0.001 | 95.37 | 72.60 | 2.22 |

0.0005 | 82.27 | 54.08 | 1.66 |

0.0001 | 76.24 | 44.68 | 1.35 |

0.00005 | 76.59 | 43.18 | 1.30 |

Model | RMSE (m^{3}) | MAE (m^{3}) | MAPE (%) |
---|---|---|---|

LSTM NN | 46.47 | 31.46 | 1.14 |

M-LSTM NN | 42.46 | 24.98 | 0.91 |

Variable | RMSE (m^{3}) | MAE (m^{3}) | MAPE (%) |
---|---|---|---|

Casing pressure | 44.15 | 31.22 | 1.10 |

Water production | 42.54 | 28.58 | 1.01 |

Lowest temperature | 46.64 | 31.45 | 1.14 |

Highest temperature | 46.63 | 31.08 | 1.15 |

No auxiliary variables | 46.47 | 31.46 | 1.14 |

All variables | 42.46 | 24.98 | 0.91 |

Time Lag (Day) | RMSE (m^{3}) | MAE (m^{3}) | MAPE (%) |
---|---|---|---|

t + 1 | 11.55 | 6.42 | 0.24 |

t + 2 | 17.39 | 12.82 | 0.48 |

t + 3 | 22.50 | 19.18 | 0.71 |

t + 4 | 23.77 | 20.66 | 0.77 |

t + 5 | 25.00 | 22.06 | 0.82 |

t + 6 | 26.16 | 23.37 | 0.87 |

t + 7 | 27.28 | 24.61 | 0.92 |

t + 8 | 38.50 | 35.39 | 1.31 |

t + 9 | 47.57 | 46.09 | 1.71 |

t + 10 | 59.51 | 56.74 | 2.09 |

Time Lag (Month) | RMSE (m^{3}) | MAE (m^{3}) | MAPE (%) |
---|---|---|---|

t + 1 | 2410 | 2230 | 2.68 |

t + 2 | 3380 | 2980 | 3.65 |

t + 3 | 5140 | 4830 | 5.95 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Xu, X.; Rui, X.; Fan, Y.; Yu, T.; Ju, Y.
A Multivariate Long Short-Term Memory Neural Network for Coalbed Methane Production Forecasting. *Symmetry* **2020**, *12*, 2045.
https://doi.org/10.3390/sym12122045

**AMA Style**

Xu X, Rui X, Fan Y, Yu T, Ju Y.
A Multivariate Long Short-Term Memory Neural Network for Coalbed Methane Production Forecasting. *Symmetry*. 2020; 12(12):2045.
https://doi.org/10.3390/sym12122045

**Chicago/Turabian Style**

Xu, Xijie, Xiaoping Rui, Yonglei Fan, Tian Yu, and Yiwen Ju.
2020. "A Multivariate Long Short-Term Memory Neural Network for Coalbed Methane Production Forecasting" *Symmetry* 12, no. 12: 2045.
https://doi.org/10.3390/sym12122045