Skip to Content
You are currently on the new version of our website. Access the old version .
Applied SciencesApplied Sciences
  • Article
  • Open Access

11 November 2021

23 Pages

SAT-Hadoop-Processor: A Distributed Remote Sensing Big Data Processing Software for Earth Observation Applications

and
1
LIST Laboratory, FSTT, Abdelmalek Essaâdi University (UAE), The Old Way of Airport, km 10, Ziaten, Tangier 416, Morocco
2
CommSensLab-UPC, Department of Signal Theory and Communications (TSC), UPC Campus Nord, 08034 Barcelona, Spain
3
Distributed Systems Group, Department of Computer Architecture (DAC), UPC Campus Nord, 08034 Barcelona, Spain
*
Author to whom correspondence should be addressed.

Abstract

Nowadays, several environmental applications take advantage of remote sensing techniques. A considerable volume of this remote sensing data occurs in near real-time. Such data are diverse and are provided with high velocity and variety, their pre-processing requires large computing capacities, and a fast execution time is critical. This paper proposes a new distributed software for remote sensing data pre-processing and ingestion using cloud computing technology, specifically OpenStack. The developed software discarded 86% of the unneeded daily files and removed around 20% of the erroneous and inaccurate datasets. The parallel processing optimized the total execution time by 90%. Finally, the software efficiently processed and integrated data into the Hadoop storage system, notably the HDFS, HBase, and Hive.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.