Sign in to use this feature.

Years

Between: -

Subjects

remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline

Journals

remove_circle_outline

Article Types

Countries / Regions

remove_circle_outline

Search Results (1)

Search Parameters:
Keywords = Java Virtual Machine heap configuration

Order results
Result details
Results per page
Select all
Export citation of selected articles as:
19 pages, 788 KB  
Article
Optimization Techniques for a Distributed In-Memory Computing Platform by Leveraging SSD
by June Choi, Jaehyun Lee, Jik-Soo Kim and Jaehwan Lee
Appl. Sci. 2021, 11(18), 8476; https://doi.org/10.3390/app11188476 - 13 Sep 2021
Cited by 1 | Viewed by 3799
Abstract
In this paper, we present several optimization strategies that can improve the overall performance of the distributed in-memory computing system, “Apache Spark”. Despite its distributed memory management capability for iterative jobs and intermediate data, Spark has a significant performance degradation problem when the [...] Read more.
In this paper, we present several optimization strategies that can improve the overall performance of the distributed in-memory computing system, “Apache Spark”. Despite its distributed memory management capability for iterative jobs and intermediate data, Spark has a significant performance degradation problem when the available amount of main memory (DRAM, typically used for data caching) is limited. To address this problem, we leverage an SSD (solid-state drive) to supplement the lack of main memory bandwidth. Specifically, we present an effective optimization methodology for Apache Spark by collectively investigating the effects of changing the capacity fraction ratios of the shuffle and storage spaces in the “Spark JVM Heap Configuration” and applying different “RDD Caching Policies” (e.g., SSD-backed memory caching). Our extensive experimental results show that by utilizing the proposed optimization techniques, we can improve the overall performance by up to 42%. Full article
(This article belongs to the Special Issue Big Data Management and Analysis with Distributed or Cloud Computing)
Show Figures

Figure 1

Back to TopTop