Next Article in Journal
From Big Data to Deep Learning: A Leap Towards Strong AI or ‘Intelligentia Obscura’?
Next Article in Special Issue
Traffic Sign Recognition based on Synthesised Training Data
Previous Article in Journal / Special Issue
The Development of Data Science: Implications for Education, Employment, Research, and the Data Revolution for Sustainable Development
Open AccessArticle

Adaptive Provisioning of Heterogeneous Cloud Resources for Big Data Processing

1
Monitoring and Control Systems, TNO Groningen, Eemsgolaan 3, 9727 DW Groningen, The Netherlands
2
Faculty of Science and Engineering, University of Groningen, Nijenborgh 9, 9747 AG Groningen, The Netherlands
*
Author to whom correspondence should be addressed.
Big Data Cogn. Comput. 2018, 2(3), 15; https://doi.org/10.3390/bdcc2030015
Received: 31 May 2018 / Revised: 5 July 2018 / Accepted: 9 July 2018 / Published: 12 July 2018
(This article belongs to the Special Issue Big Data and Cognitive Computing: Feature Papers 2018)
Efficient utilization of resources plays an important role in the performance of large scale task processing. In cases where heterogeneous types of resources are used within the same application, it is hard to achieve good utilization of all of the different types of resources. By taking advantage of recent developments in cloud infrastructure that enable the use of dynamic clusters of resources, and by dynamically altering the size of the available resources for all the different resource types, the overall utilization of resources, however, can be improved. Starting from this premise, this paper discusses a solution that aims to provide a generic algorithm to estimate the desired ratios of instance processing tasks as well as ratios of the resources that are used by these instances, without the necessity for trial runs or a priori knowledge of the execution steps. These ratios are then used as part of an adaptive system that is able to reconfigure itself to maximize utilization. To verify the solution, a reference framework which adaptively manages clusters of functionally different VMs to host a calculation scenario is implemented. Experiments are conducted based on a compute-heavy use case in which the probability of underground pipeline failures is determined based on the settlement of soils. These experiments show that the solution is capable of eliminating large amounts of under-utilization, resulting in increased throughput and lower lead times. View Full-Text
Keywords: cloud computing; big data processing and analytics; heterogeneous cloud resources; industrial case study cloud computing; big data processing and analytics; heterogeneous cloud resources; industrial case study
Show Figures

Figure 1

MDPI and ACS Style

Kollenstart, M.; Harmsma, E.; Langius, E.; Andrikopoulos, V.; Lazovik, A. Adaptive Provisioning of Heterogeneous Cloud Resources for Big Data Processing. Big Data Cogn. Comput. 2018, 2, 15.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop