# A Probabilistic Prediction Approach for Memory Resource of Complex System Simulation in Cloud Computing Environment

^{*}

## Abstract

**:**

## 1. Introduction

- As the memory prediction dataset is not available, the dataset is conducted by collecting static and dynamic information (entity scale, rollback events, simulation end time, etc.) of CSS applications which are deployed in the cloud environment.
- A probabilistic approach is proposed to predict memory resources required by CSS applications based on ensemble learning. A root mean squared error-based pruning (RMSEP) algorithm is proposed to select the optimal subset of the base models, which can improve the performance of the probabilistic approach.
- The experiments verified the effectiveness of the proposed approach for memory resource prediction.

## 2. Related Works

#### 2.1. Resource Prediction in Cloud Environment

#### 2.1.1. Application Resource Prediction

#### 2.1.2. Host Load Prediction

#### 2.2. Ensemble Model

## 3. Prerequisite

#### 3.1. Random Forest

#### 3.2. Back-Propagation Neural Network

#### 3.3. Gaussian Process Regression

## 4. Methodology

#### 4.1. Cloud Memory Prediction Framework

#### 4.2. Memory Probabilistic Prediction Model

- (1)
- Calculate the RMSE of base models.
- (2)
- Sort the base models by RMSE in ascending order, and the base models predict the memory resources on the dataset.
- (3)
- Calculate the mean of each base model as the output of the model set, and then calculate the RMSE of the model set.
- (4)
- Select the model set with the minimum RMSE.

Algorithm 1. RMSEP |

Input: dataset $D$, base models {${M}_{1},{M}_{2},\dots \dots ,{M}_{n}$} Output: final model set $FMS$ 1: for $i$ = 1 to $n$2: calculate$\text{}RMS{E}_{i}$ of base model ${M}_{i}$; 3: end for4: sort the base models {${M}_{1},{M}_{2},\dots \dots ,{M}_{n}$} by RMSE in ascending order, and the sorted model set can be represented by {$S{M}_{1},S{M}_{2},\dots \dots ,S{M}_{n}$}; 5: for $i$ = 1 to $n$6: make memory predictions on the dataset$\text{}D$ with the model set {$S{M}_{1},S{M}_{2},\dots \dots ,S{M}_{i}$}, the mean of the model set {$S{M}_{1},S{M}_{2},\dots \dots ,S{M}_{i}$} is used as the output of the ensemble model, and calculate the $RMS{E}_{i}$ of the model set; 7: end for8: the model set {$S{M}_{1},S{M}_{2},\dots \dots ,S{M}_{i}$} which has the min $RMS{E}_{i}$ in {$RMS{E}_{1},RMS{E}_{2},\dots \dots ,RMS{E}_{n}$} is selected as the final model set $FMS$; 9: return $FMS$; |

#### 4.3. Evaluation Metrics

## 5. Case Study

#### 5.1. Social Opinion System

#### 5.2. Dataset

## 6. Experiments and Results

#### 6.1. Experimental Environment

#### 6.2. Experimental Results

#### 6.2.1. Parameter Experiments

#### 6.2.2. Performance Experiments

## 7. Conclusions and Future Works

- (1)
- We hope to evaluate our approach on a large cloud platform and evaluate the proposed approach on other CSS applications.
- (2)
- Considering the limitation that the memory prediction method does not exploit the domain information of CSS applications, we plan to build the strong correlation features of CSS applications.
- (3)
- Commit to the investigate the correlation between the cloud resources, and provide more reliable results for the resource allocation.

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Jiang, Z.; Zhao, T.; Wang, S.; Ren, F. A Novel Risk Assessment and Analysis Method for Correlation in a Complex System Based on Multi-Dimensional Theory. Appl. Sci.
**2020**, 10, 3007. [Google Scholar] [CrossRef] - Fujimoto, R.M. Research Challenges in Parallel and Distributed Simulation. ACM Trans. Modeling Comput. Simul.
**2016**, 26, 1–29. [Google Scholar] [CrossRef] [Green Version] - Fujimoto, R.M.; Malik, A.W.; Park, A.J. Parallel and distributed simulation in the cloud. SCS M S Mag.
**2010**, 3, 1–10. [Google Scholar] - Wang, Y.; Tang, W.; Yao, Y.; Zhu, F. DA-OCBA: Distributed Asynchronous Optimal Computing Budget Allocation Algorithm of Simulation Optimization Using Cloud Computing. Symmetry
**2019**, 11, 1297. [Google Scholar] [CrossRef] [Green Version] - Shuai, W.; Feng, Z.; Yiping, Y.; Wenjie, T.; Yuhao, X.; Siqi, X. A Computing Resources Prediction Approach Based on Ensemble Learning for Complex System Simulation in Cloud Environment. Simul. Model. Pract. Theory
**2020**. [Google Scholar] [CrossRef] - Tianlin, L.; Yiping, Y.; Wenjie, T.; Feng, Z.; Zhongwei, L. An Efficient Multi-threaded Memory Allocator for PDES Applications. Simul. Model. Pract. Theory
**2020**, 100, 102067. [Google Scholar] - Zuberek, W.M. Balancing the performance of block multithreaded distributed-memory systems. Simul. Model. Pract. Theory
**2011**, 19, 1318–1329. [Google Scholar] [CrossRef] - Malik, A.; Park, A.; Fujimoto, R.M. Optimistic synchronization of parallel simulations in cloud computing environments. In Proceedings of the IEEE International Conference on Cloud Computing, Bangalore, India, 21–25 September 2009; pp. 49–56. [Google Scholar]
- Kumar, K.D.; Umamaheswari, E. Prediction methods for effective resource provisioning in cloud computing: A survey. Multiagent Grid Syst.
**2018**, 14, 283–305. [Google Scholar] [CrossRef] - Amiri, L.M.; Mohammad, K. Survey on prediction models of applications for resources provisioning in cloud. J. Netw. Comput. Appl.
**2017**, 82, 93–113. [Google Scholar] [CrossRef] - Feng, Y.; Yiping, Y.; Lining, X.; Huangke, C. An Intelligent Scheduling Algorithm for Complex Manufacturing System Simulation with Frequent Synchronizations in Clouds. Memetic Comput.
**2019**, 11, 357–370. [Google Scholar] [CrossRef] - Shaw, R.; Howley, E.; Barrett, E. An energy efficient anti-correlated virtual machine placement algorithm using resource usage predictions. Simul. Model. Pract. Theory
**2019**, 93, 322–342. [Google Scholar] [CrossRef] - Alodat, M.T.; Mohammed, K.S. Gaussian process regression with skewed errors. J. Comput. Appl. Math.
**2020**, 370, 15. [Google Scholar] [CrossRef] - Roman, K. Bayesian system for probabilistic river stage forecasting. J. Hydrol.
**2002**, 268, 16–40. [Google Scholar] - Breiman, L. Random forests. Mach. Learn.
**2001**, 45, 5–32. [Google Scholar] [CrossRef] [Green Version] - Jiao, A.; Zhang, G.; Liu, B.; Liu, W. Prediction of Manufacturing Quality of Holes Based on a BP Neural Network. Appl. Sci.
**2020**, 10, 2108. [Google Scholar] [CrossRef] [Green Version] - Al-Rawahi, M.; Edirisinghe, A.; Jeyarajan, T. Machine Learning-Based Framework for Resource Management and Modelling for Video Analytic in Cloud-Based Hadoop Environment. In Proceedings of the 2016 Intl IEEE Conferences on Ubiquitous Intelligence & Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People, and Smart World Congress, Toulouse, France, 18–21 July 2016; pp. 801–807. [Google Scholar]
- Mukhtaj, K.; Yong, J.; Maozhen, L.; Yang, X.; Changjun, J. Hadoop Performance Modeling for Job Estimation and Resource Provisioning. IEEE Trans. Parallel Distrib. Syst.
**2015**, 99, 441–454. [Google Scholar] - Gurleen, K.; Anju, B.; Inderveer, C. An intelligent regressive ensemble approach for predicting resource usage in cloud computing. J. Parallel Distrib. Comput.
**2019**, 123, 1–12. [Google Scholar] - Ganapathi, A.; Chen, Y.; Fox, A.; Katz, R.; Patterson, D. Statistics-driven workload modeling for the Cloud. In Proceedings of the 26th International Conference on Data Engineering, Long Beach, CA, USA, 1–6 March 2010; pp. 87–92. [Google Scholar]
- Gopal, K.S.; Sunilkumar, S.M. Virtual Resource Prediction in Cloud Environment: A Bayesian Approach. J. Netw. Comput. Appl.
**2016**, 65, 144–154. [Google Scholar] - Hisham, A.K. An Intelligent Swarm Based Prediction Approach For Predicting Cloud Computing User Resource Needs. Comput. Commun.
**2020**, 151, 133–144. [Google Scholar] - Mehiar, D.; Bechir, H.; Mohsen, G.; Ammar, R. Efficient Datacenter Resource Utilization Through Cloud Resource Overcommitment. In Proceedings of the 2015 IEEE INFOCOM Workshop on Mobile Cloud and Virtualization, Hong Kong, China, 26 April–1 May 2015; pp. 330–335. [Google Scholar]
- Ali, R.; Mostafa, G.A.; Sajjad, T. A learning automata-based ensemble resource usage prediction algorithm for cloud computing environment. Future Gener. Comput. Syst.
**2018**, 79, 54–71. [Google Scholar] - Mostafa, G.A.; Sam, J.; Mohammad, A.P. An autonomic resource provisioning approach for service-based cloud applications: A hybrid approach. Future Gener. Comput. Syst.
**2018**, 78, 191–210. [Google Scholar] - Jinping, L.; Jiezhou, H.; Wuxia, Z.; Tianyu, M.; Zhaohui, T.; Jean, P.N.; Weihua, G. ANID-SEoKELM: Adaptive network intrusion detection based on selective ensemble of kernel ELMs with random features. Knowl.-Based Syst.
**2019**, 177, 104–116. [Google Scholar] - Ning, A.; Huitong, D.; Jiaoyun, Y.; Rhoda, A.; Ting, F.A.A. Deep ensemble learning for Alzheimer’s disease classification. J. Biomed. Inform.
**2020**, 105, 103411. [Google Scholar] - Deepak, G.; Rinkle, R. Improving malware detection using big data and ensemble learning. Comput. Electr. Eng.
**2020**, 86, 106729. [Google Scholar] - Tiago, P.; Isabel, P.; Zita, V.; Jose, S. Ensemble learning for electricity consumption forecasting in office buildings. Neurocomputing
**2020**, in press. [Google Scholar] - Robert, H.N. Theory of the Backpropagation Neural Network. In Proceedings of the International Joint Conference on Neural Networks, Honolulu, HI, USA, 12–17 May 2002; pp. 593–605. [Google Scholar]
- Liu, Y.; Li, M.; Su, P.; Ma, B.; You, Z. Porosity Prediction of Granular Materials through Discrete Element Method and Back Propagation Neural Network Algorithm. Appl. Sci.
**2020**, 10, 1693. [Google Scholar] [CrossRef] [Green Version] - Zhenxing, L.; Xiaodan, H.; Kuangrong, H.; Lei, C.; Biao, H. Gaussian process regression with heteroscedastic noises—A machine-learning predictive variance approach. Chem. Eng. Res. Des.
**2020**, 157, 162–173. [Google Scholar] - Srikanth, B.Y.; Kalyan, S.P. Efficient Parallel Discrete Event Simulation on Cloud/Virtual Machine Platforms. ACM Trans. Modeling Comput. Simul.
**2015**, 26, 1–26. [Google Scholar] - Rafael, M.O.C.; Robert, S.; George, D.C.C.; Tsang, I.R. META-DES: A dynamic ensemble selection framework using meta-learning. Pattern Recognit.
**2015**, 48, 1925–1935. [Google Scholar] [CrossRef] [Green Version] - Feng, Z.; Yiping, Y.; Wenjie, T.; Jun, T. A hierarchical composite framework of parallel discrete event simulation for modelling complex adaptive systems. Simul. Model. Pract. Theory
**2017**, 77, 141–156. [Google Scholar] - Bonan, H.; Yiping, Y.; Bing, W. Modeling and simulation of large-scale social networks using parallel discrete event simulation. Simul. Trans. Soc. Modeling Simul. Int.
**2013**, 89, 1173–1183. [Google Scholar]

**Figure 6.**The root mean squared error (RMSE) of the first-layer base models with different ensemble sizes.

**Figure 7.**The performance of Gaussian process regression (GPR) model with different kernel functions. (

**a**) mean absolute error (MAE); (

**b**) RMSE; (

**c**) mean relative error (MRE).

**Figure 8.**Probability density function (PDF). Number of individuals (NI). Probability of leaders (PL). Probability of interpersonal network (PIN): (

**a**) NI = 50, PL = 5%, PIN = 5%; (

**b**) NI = 1000, PL = 5%, PIN = 5%; (

**c**) NI = 10000, PL = 5%, PIN = 5%.

Parameters | Value |
---|---|

Number of individuals (NI) | [50, 10,000] |

Probability of leaders (PL) | [5%, 20%] |

Probability of interpersonal network (PIN) | [5%, 20%] |

Number of media | 10 |

Number of cities | 10 |

Number of resources | 50 |

Simulation end time | (500, 1000, 1500) |

Time synchronization strategy | Time warp |

Feature | Description |
---|---|

Entities | Number of the simulation entities. |

Rollbacks | Number of the simulation event rollbacks. |

CPU | Total percentage of CPU while executing the simulation applications. |

Memory | Total percentage of memory while executing the simulation applications. |

File | File usage while executing the simulation applications. |

Network | Data received/sent by the network. |

Delay | Communication delay. |

Tend | Simulation end time. |

Pre-allocation resource | Resources allocated to the simulation applications. |

Runtime | Execution time of the simulation applications. |

Model | MAE | RMSE | MRE (%) |
---|---|---|---|

RF | 0.4197 ± 0.0085 | 0.4384 ± 0.0148 | 10.1559 ± 0.4521 |

BPNN | 0.3192 ± 0.0078 | 0.3660 ± 0.0105 | 7.8049 ± 0.3534 |

GPR | 0.3748 ± 0.0102 | 0.3967 ± 0.0124 | 9.1980 ± 0.3845 |

Bagging | 0.3051 ± 0.0065 | 0.3428 ± 0.0087 | 7.3660 ± 0.3354 |

REAP | 0.2989 ± 0.0063 | 0.3364 ± 0.0074 | 7.2807 ± 0.3856 |

Proposed probabilistic approach | 0.2665 ± 0.0042 | 0.2986 ± 0.0068 | 6.4677 ± 0.2738 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Wang, S.; Yao, Y.; Zhu, F.; Tang, W.; Xiao, Y.
A Probabilistic Prediction Approach for Memory Resource of Complex System Simulation in Cloud Computing Environment. *Symmetry* **2020**, *12*, 1826.
https://doi.org/10.3390/sym12111826

**AMA Style**

Wang S, Yao Y, Zhu F, Tang W, Xiao Y.
A Probabilistic Prediction Approach for Memory Resource of Complex System Simulation in Cloud Computing Environment. *Symmetry*. 2020; 12(11):1826.
https://doi.org/10.3390/sym12111826

**Chicago/Turabian Style**

Wang, Shuai, Yiping Yao, Feng Zhu, Wenjie Tang, and Yuhao Xiao.
2020. "A Probabilistic Prediction Approach for Memory Resource of Complex System Simulation in Cloud Computing Environment" *Symmetry* 12, no. 11: 1826.
https://doi.org/10.3390/sym12111826