# BBS Posts Time Series Analysis based on Sample Entropy and Deep Neural Networks

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methodology

#### 2.1. ARIMA Model

**W**

_{t}.

#### 2.2. Seasonal ARIMA Model

#### 2.3. Polynomial Regression

#### 2.4. Artificial Neural Networks

## 3. SampEn-DNN

#### 3.1. Sample Entropy

#### 3.2. Deep Neural Network

**a**,

**b**and

**W**. The energy-based model means an entropy function is applied to define the log-likelihood input data distribution over the parameters

**a**,

**b**,

**W**,

**v**and

**h**. The energy function for RBM is given by Equations (10) and (11).

#### 3.3. SampEn-DNN Approach

## 4. Experiments and Discussions

#### 4.1. Datasets

#### 4.2. Experimental Procedures

#### 4.3. Experimental Results

## 5. Concluding Remarks

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Guo, K.; Shi, L.; Ye, W.; Li, X. A survey of Internet public opinion mining. In Proceedings of the 2014 IEEE International Conference on Progress in Informatics & Computing, Shanghai, China, 16–18 May 2014; pp. 173–179. [Google Scholar]
- Tang, X.J. Exploring on-line societal risk perception for harmonious society measurement. J. Syst. Sci. Syst. Eng.
**2013**, 22, 469–486. [Google Scholar] [CrossRef] - Pond, P. Twitter time: A temporal analysis of tweet streams during televised political debate. TVNM
**2016**, 17, 142–158. [Google Scholar] [CrossRef] - Boecking, B.; Hall, M.; Schneider, J. Event prediction with learning algorithms—A study of events surrounding the Egyptian Revolution of 2011 on the basis of micro blog data. Policy Internet
**2015**, 7, 159–184. [Google Scholar] [CrossRef] - Wang, Y.J. Stock market forecasting with financial micro-blog based on sentiment and time series analysis. Shanghai Jiaotong Univ. Sci.
**2017**, 22, 173–179. [Google Scholar] [CrossRef] - Chen, X.G.; Duan, S.; Wang, L. Research on trend prediction and evaluation of network public opinion. Concurr. Comp. Pract. Exp.
**2017**, 29, 4212. [Google Scholar] [CrossRef] - Yin, F.; Zhang, B.; Su, G.; Zhang, R. Research on the public opinion pre-warning based on the logistic model. In Proceeding of the 8th IEEE International Conference on Software Engineering and Service Science (ICSESS), Beijing, China, 24–26 November 2017; pp. 381–385. [Google Scholar]
- Zhang, J.L. A study on the evaluation of the analytic hierarchy process based Enterprise network public opinion crisis response. J. Quant. Econ.
**2015**, 32, 60–63. [Google Scholar] [CrossRef] - Hu, H.; Wen, Y.; Chua, T.S.; Li, X. Cost-optimized microblog distribution over geo-distributed data centers: Insights from cross-media analysis. ACM Trans. Intel. Syst. Tech.
**2017**, 8, 1–18. [Google Scholar] [CrossRef] - Lan, Y.X.; Zeng, R.X. Research of emergency network public opinion on propagation model and warning phase. J. Intel.
**2013**, 32, 17–19. [Google Scholar] - Yang, C.; Su, G.Q.; Lan, Y.X.; He, Y.H. Research on Emergency network public opinion based on the statistical regression model. J. Chin. People’s Armed Police Force Acad.
**2014**, 30, 80–83. [Google Scholar] - Wang, H.; Li, Y.; Feng, Z.; Feng, L. ReTweeting analysis and prediction in microblogs: An epidemic inspired approach. China Commun.
**2013**, 10, 13–24. [Google Scholar] [CrossRef] - Xu, T.; Xu, X.; Hu, Y.; Li, X. An entropy-based approach for evaluating travel time predictability based on vehicle trajectory data. Entropy
**2017**, 19, 165. [Google Scholar] [CrossRef] - Men, B.; Long, R.; Zhang, J. Combined forecasting of streamflow based on cross entropy. Entropy
**2016**, 18, 336. [Google Scholar] [CrossRef] - Zhang, W.; Du, Y.H.; Yoshida, T.; Wang, Q.; Li, X. SamEn-SVR: Using sample entropy and support vector regression for bug number prediction. IET Softw.
**2018**, 12, 183–189. [Google Scholar] [CrossRef] - Zhang, Y.J.; Ma, J.L.; Liu, J.L.; Xiao, S.Z. Psr-svr network public opinion prediction model. Metall. Min. Ind.
**2015**, 9, 787–794. [Google Scholar] - Chen, J.D.; Zhou, X.J.; Tang, X.J. An empirical feasibility study of societal risk classification toward BBS posts. Syst. Sci. Syst. Eng.
**2018**, 27, 709–726. [Google Scholar] [CrossRef] - Jin, X.L.; Cheung, C.M.K.; Lee, M.K.O.; Chen, H.P. How to keep members using the information in a computer-supported social network. Comput. Hum. Behav.
**2009**, 25, 1172–1181. [Google Scholar] [CrossRef] - Chen, J.D.; Tang, X.J. Ensemble of multiple kNN classifiers for societal risk classification. Syst. Sci. Syst. Eng.
**2017**, 26, 433–447. [Google Scholar] [CrossRef] - Qu, Y.; Wu, P.F.; Wang, X. Online community response to major disaster: A study of Tianya forum in the 2008 Sichuan earthquake. In Proceedings of the 42nd Hawaii International Conference on System Sciences (HICSS’09), Waikoloa, Big Island, HI, USA, 5–8 January 2009; pp. 1–11. [Google Scholar]
- Takens, F. Detecting strange attractors in turbulence. Lect. Notes Math.
**1981**, 898, 366–381. [Google Scholar] - Sauer, T.; Yorke, J.A.; Casdagli, M. Embedology. J. Stat. Phys.
**1991**, 65, 579. [Google Scholar] [CrossRef] - Richman, J.S.; Moorman, J.R. Physiological time-series analysis using approximate entropy and sample entropy. Am. J. Physiol. Heart Circ. Physiol.
**2000**, 278. [Google Scholar] [CrossRef] - Govindan, R.B.; Wilson, J.D.; Eswaran, H.; Lowery, C.L.; Preißl, H. Revisiting sample entropy analysis. Phys. A Statist. Mech. Appl.
**2007**, 376, 158–164. [Google Scholar] [CrossRef] - Kuremoto, T.; Kimura, S.; Kobayashi, K.; Obayashi, M. Time series forecasting using a deep belief network with restricted Boltzmann machines. Neurocomputing
**2014**, 137, 47–56. [Google Scholar] [CrossRef] - Shumway, R.H.; Stoffer, D.S. Time series regression and exploratory data analysis. In Time Series Analysis and its Applications: With R Examples, 3rd ed.; Springer: New York, NY, USA, 2011. [Google Scholar]
- Box, G.E.P.; Jenkins, G. Part one-stochastic models and their forecasting. In Time Series Analysis: Forecasting and Control, 3rd ed.; Prentice-Hall: London, UK, 1994. [Google Scholar]
- Hazewinkel, M. Taylor series. In Encyclopedia of Mathematics; Springer: Dordrecht, The Netherlands, 2001. [Google Scholar]
- Zhang, R.; Duan, Y.; Zhao, Y.; He, X. Temperature compensation of Elasto-Magneto-Electric (EME) sensors in cable force monitoring using BP neural network. Sensors
**2018**, 18, 2176. [Google Scholar] [CrossRef] [PubMed] - Cantrell, C.D. ‘Vector spaces’ in Modern Mathematical Methods for Physicists and Engineers; Cambridge University Press: Cambridge, UK, 2000. [Google Scholar]
- Hinton, G.E.; Osindero, S.; Teh, Y.W. A fast learning algorithm for deep belief nets. Neural Comput.
**2006**, 18, 1527–1554. [Google Scholar] [CrossRef] [PubMed] - Memisevic, R.; Hinton, G.E. Learning to represent spatial transformations with factored higher-order Boltzmann machines. Neural Comput.
**2010**, 22, 1473–1492. [Google Scholar] [CrossRef] [PubMed] - Mann, H.B.; Whitney, R. On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat.
**1947**, 18, 50–60. [Google Scholar] [CrossRef]

**Figure 2.**DNN (deep neural networks) model consists of DBN (deep belief network) and FNN (feedforward neural network).

**Table 1.**SampEn (sample entropy) results of BBS (Bulletin Board System) posts time series with different m and δ.

δ | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|

m | |||||||||||||

2 | 1.19 | 1.22 | 1.20 | 1.24 | 1.31 | 1.32 | 1.05 | 1.32 | 1.36 | 1.26 | 1.30 | 1.38 | |

3 | 1.06 | 1.06 | 1.13 | 1.14 | 1.09 | 1.18 | 0.93 | 1.20 | 1.14 | 1.12 | 1.18 | 1.20 | |

4 | 0.96 | 0.95 | 1.01 | 1.05 | 0.96 | 1.09 | 0.85 | 1.10 | 1.06 | 1.04 | 1.06 | 1.12 | |

5 | 0.90 | 0.80 | 0.91 | 0.97 | 0.92 | 0.95 | 0.81 | 1.07 | 1.00 | 0.93 | 0.97 | 1.15 | |

6 | 0.83 | 0.75 | 0.87 | 0.96 | 0.82 | 0.93 | 0.78 | 1.00 | 0.89 | 0.99 | 0.94 | 1.09 | |

7 | 0.81 | 0.72 | 0.80 | 0.90 | 0.73 | 0.93 | 0.76 | 0.87 | 0.83 | 1.13 | 0.83 | 0.92 | |

8 | 0.68 | 0.72 | 0.90 | 0.73 | 0.70 | 0.85 | 0.71 | 0.88 | 0.97 | 1.10 | 0.76 | 0.78 | |

9 | 0.73 | 0.72 | 0.92 | 0.78 | 0.61 | 1.30 | 0.76 | 1.14 | 0.98 | 0.96 | 1.10 | 0.89 | |

10 | 0.63 | 0.74 | 0.69 | 0.98 | 0.50 | 1.50 | 0.74 | 0.98 | 1.10 | 0.92 | 1.20 | 1.95 | |

11 | 0.54 | 0.85 | 0.59 | 1.10 | 0.41 | NaN | 0.45 | NaN | 1.11 | NaN | NaN | NaN | |

12 | 0.56 | 1.50 | 0.92 | NaN | 0.47 | NaN | 0.48 | NaN | NaN | NaN | NaN | NaN | |

13 | 0.98 | NaN | NaN | NaN | 0.69 | NaN | 0.62 | NaN | NaN | NaN | NaN | NaN |

**Table 2.**MMREs (mean magnitude of relative error) of ARIMA (auto-regressive integrated moving average), seasonal ARIMA, polynomial (polynomial regression), ANN (artificial neural networks) and SampEn-DNN (sample entropy-deep neural networks) on BBS post time series.

Subset | ARIMA | Seasonal ARIMA | Polynomial | ANN | SampEn-DNN |
---|---|---|---|---|---|

1 | 0.2355 ± 0.0090 | 0.1968 ± 0.0119 | 0.2772 ± 0.0109 | 0.2003 ± 0.0129 | 0.1419 ± 0.0078 |

2 | 0.1895 ± 0.0103 | 0.1691 ± 0.0126 | 0.4735 ± 0.0179 | 0.1694 ± 0.0107 | 0.1241 ± 0.0078 |

3 | 0.1915 ± 0.0117 | 0.1704 ± 0.0107 | 0.3535 ± 0.0135 | 0.1934 ± 0.0092 | 0.1748 ± 0.0080 |

4 | 0.1325 ± 0.0049 | 0.1188 ± 0.0066 | 0.3637 ± 0.0212 | 0.1450 ± 0.0098 | 0.0878 ± 0.0036 |

5 | 0.1653 ± 0.0083 | 0.1451 ± 0.0102 | 0.2401 ± 0.0126 | 0.1494 ± 0.0101 | 0.1256 ± 0.0043 |

Model Pair | Seasonal ARIMA | Polynomial Regression | ANN | SampEn-DNN |
---|---|---|---|---|

ARIMA | < | ≫ | < | ≪ |

Seasonal ARIMA | ∼ | ≫ | ∼ | < |

Polynomial Regression | ≪ | ∼ | ≪ | ≪ |

ANN | ∼ | ≫ | ∼ | < |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Chen, J.; Du, Y.; Liu, L.; Zhang, P.; Zhang, W.
BBS Posts Time Series Analysis based on Sample Entropy and Deep Neural Networks. *Entropy* **2019**, *21*, 57.
https://doi.org/10.3390/e21010057

**AMA Style**

Chen J, Du Y, Liu L, Zhang P, Zhang W.
BBS Posts Time Series Analysis based on Sample Entropy and Deep Neural Networks. *Entropy*. 2019; 21(1):57.
https://doi.org/10.3390/e21010057

**Chicago/Turabian Style**

Chen, Jindong, Yuxuan Du, Linlin Liu, Pinyi Zhang, and Wen Zhang.
2019. "BBS Posts Time Series Analysis based on Sample Entropy and Deep Neural Networks" *Entropy* 21, no. 1: 57.
https://doi.org/10.3390/e21010057