Next Article in Journal
From Big Data to Deep Learning: A Leap Towards Strong AI or ‘Intelligentia Obscura’?
Next Article in Special Issue
Traffic Sign Recognition based on Synthesised Training Data
Previous Article in Journal / Special Issue
The Development of Data Science: Implications for Education, Employment, Research, and the Data Revolution for Sustainable Development
Article Menu
Issue 3 (September) cover image

Export Article

Open AccessArticle
Big Data Cogn. Comput. 2018, 2(3), 15; https://doi.org/10.3390/bdcc2030015

Adaptive Provisioning of Heterogeneous Cloud Resources for Big Data Processing

1
Monitoring and Control Systems, TNO Groningen, Eemsgolaan 3, 9727 DW Groningen, The Netherlands
2
Faculty of Science and Engineering, University of Groningen, Nijenborgh 9, 9747 AG Groningen, The Netherlands
*
Author to whom correspondence should be addressed.
Received: 31 May 2018 / Revised: 5 July 2018 / Accepted: 9 July 2018 / Published: 12 July 2018
(This article belongs to the Special Issue Big Data and Cognitive Computing: Feature Papers 2018)
Full-Text   |   PDF [640 KB, uploaded 12 July 2018]   |  

Abstract

Efficient utilization of resources plays an important role in the performance of large scale task processing. In cases where heterogeneous types of resources are used within the same application, it is hard to achieve good utilization of all of the different types of resources. By taking advantage of recent developments in cloud infrastructure that enable the use of dynamic clusters of resources, and by dynamically altering the size of the available resources for all the different resource types, the overall utilization of resources, however, can be improved. Starting from this premise, this paper discusses a solution that aims to provide a generic algorithm to estimate the desired ratios of instance processing tasks as well as ratios of the resources that are used by these instances, without the necessity for trial runs or a priori knowledge of the execution steps. These ratios are then used as part of an adaptive system that is able to reconfigure itself to maximize utilization. To verify the solution, a reference framework which adaptively manages clusters of functionally different VMs to host a calculation scenario is implemented. Experiments are conducted based on a compute-heavy use case in which the probability of underground pipeline failures is determined based on the settlement of soils. These experiments show that the solution is capable of eliminating large amounts of under-utilization, resulting in increased throughput and lower lead times. View Full-Text
Keywords: cloud computing; big data processing and analytics; heterogeneous cloud resources; industrial case study cloud computing; big data processing and analytics; heterogeneous cloud resources; industrial case study
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Kollenstart, M.; Harmsma, E.; Langius, E.; Andrikopoulos, V.; Lazovik, A. Adaptive Provisioning of Heterogeneous Cloud Resources for Big Data Processing. Big Data Cogn. Comput. 2018, 2, 15.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Big Data Cogn. Comput. EISSN 2504-2289 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top