Next Article in Journal
Extraction and Detection of Surface Defects in Particleboards by Tracking Moving Targets
Previous Article in Journal
On Fast Converging Data-Selective Adaptive Filtering
Previous Article in Special Issue
Hadoop vs. Spark: Impact on Performance of the Hammer Query Engine for Open Data Corpora
Article Menu

Export Article

Open AccessArticle
Algorithms 2019, 12(1), 5; https://doi.org/10.3390/a12010005

MAPSkew: Metaheuristic Approaches for Partitioning Skew in MapReduce

1
Departamento de Pós Graduação em Ciências da Computação (MDCC), Universidade Federal do Ceará, Fortaleza, CE 60020-181, Brazil
2
Centro de Ciências e Tecnologia (CCT), Universidade de Fortaleza (UNIFOR), Fortaleza, CE 60020-181, Brazil
*
Author to whom correspondence should be addressed.
Received: 19 October 2018 / Revised: 23 November 2018 / Accepted: 10 December 2018 / Published: 24 December 2018
(This article belongs to the Special Issue MapReduce for Big Data)
Full-Text   |   PDF [357 KB, uploaded 24 December 2018]   |  

Abstract

MapReduce is a parallel computing model in which a large dataset is split into smaller parts and executed on multiple machines. Due to its simplicity, MapReduce has been widely used in various applications domains. MapReduce can significantly reduce the processing time of a large amount of data by dividing the dataset into smaller parts and processing them in parallel in multiple machines. However, when data are not uniformly distributed, we have the so called partitioning skew, where the allocation of tasks to machines becomes unbalanced, either by the distribution function splitting the dataset unevenly or because a part of the data is more complex and requires greater computational effort. To solve this problem, we propose an approach based on metaheuristics. For evaluating purposes, three metaheuristics were implemented: Simulated Annealing, Local Beam Search and Stochastic Beam Search. Our experimental evaluation, using a MapReduce implementation of the Bron-Kerbosch Clique Algorithm, shows that the proposed method can find good partitionings while better balancing data among machines. View Full-Text
Keywords: MapReduce; partitioning skew; metaheuristics; parallel computing; high performance computing MapReduce; partitioning skew; metaheuristics; parallel computing; high performance computing
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Pericini, M.H.M.; Leite, L.G.M.; De Carvalho-Junior, F.H.; Machado, J.C.; Rezende, C.A. MAPSkew: Metaheuristic Approaches for Partitioning Skew in MapReduce. Algorithms 2019, 12, 5.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Algorithms EISSN 1999-4893 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top