Next Article in Journal
Analyzing OpenStreetMap Road Data and Characterizing the Behavior of Contributors in Ankara, Turkey
Previous Article in Journal
Using Satellite-Borne Remote Sensing Data in Generating Local Warming Maps with Enhanced Resolution
Previous Article in Special Issue
LandQv2: A MapReduce-Based System for Processing Arable Land Quality Big Data
Article Menu

Export Article

Open AccessArticle
ISPRS Int. J. Geo-Inf. 2018, 7(10), 399; https://doi.org/10.3390/ijgi7100399

High-Performance Geospatial Big Data Processing System Based on MapReduce

IoT Research Division, Electronics and Telecommunications Research Institute (ETRI), 218 Gajeong-ro, Yuseong-gu, Daejeon 34129, Korea
*
Author to whom correspondence should be addressed.
Received: 21 August 2018 / Revised: 30 September 2018 / Accepted: 4 October 2018 / Published: 6 October 2018
(This article belongs to the Special Issue Distributed and Parallel Architectures for Spatial Data)
Full-Text   |   PDF [3294 KB, uploaded 19 October 2018]   |  

Abstract

With the rapid development of Internet of Things (IoT) technologies, the increasing volume and diversity of sources of geospatial big data have created challenges in storing, managing, and processing data. In addition to the general characteristics of big data, the unique properties of spatial data make the handling of geospatial big data even more complicated. To facilitate users implementing geospatial big data applications in a MapReduce framework, several big data processing systems have extended the original Hadoop to support spatial properties. Most of those platforms, however, have included spatial functionalities by embedding them as a form of plug-in. Although offering a convenient way to add new features to an existing system, the plug-in has several limitations. In particular, while executing spatial and nonspatial operations by alternating between the existing system and the plug-in, additional read and write overheads have to be added to the workflow, significantly reducing performance efficiency. To address this issue, we have developed Marmot, a high-performance, geospatial big data processing system based on MapReduce. Marmot extends Hadoop at a low level to support seamless integration between spatial and nonspatial operations of a solid framework, allowing improved performance of geoprocessing workflow. This paper explains the overall architecture and data model of Marmot as well as the main algorithm for automatic construction of MapReduce jobs from a given spatial analysis task. To illustrate how Marmot transforms a sequence of operators for spatial analysis to map and reduce functions in a way to achieve better performance, this paper presents an example of spatial analysis retrieving the number of subway stations per city in Korea. This paper also experimentally demonstrates that Marmot generally outperforms SpatialHadoop, one of the top plug-in based spatial big data frameworks, particularly in dealing with complex and time-intensive queries involving spatial index. View Full-Text
Keywords: big data; IoT; MapReduce; Hadoop; geospatial big data; geospatial applications big data; IoT; MapReduce; Hadoop; geospatial big data; geospatial applications
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Jo, J.; Lee, K.-W. High-Performance Geospatial Big Data Processing System Based on MapReduce. ISPRS Int. J. Geo-Inf. 2018, 7, 399.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
ISPRS Int. J. Geo-Inf. EISSN 2220-9964 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top