Next Article in Journal
Detection by Infrared Thermography of the Effect of Local Cryotherapy Exposure on Thermal Spreadin Skin
Next Article in Special Issue
Automatic Gleason Grading of Prostate Cancer Using Shearlet Transform and Multiple Kernel Learning
Previous Article in Journal
Microwave Imaging Radiometers by Aperture Synthesis Performance Simulator (Part 2): Instrument Modeling, Calibration, and Image Reconstruction Algorithms
Previous Article in Special Issue
FPGA-Based Portable Ultrasound Scanning System with Automatic Kidney Detection
Article Menu

Export Article

Open AccessArticle
J. Imaging 2016, 2(2), 19; doi:10.3390/jimaging2020019

Optimized Distributed Hyperparameter Search and Simulation for Lung Texture Classification in CT Using Hadoop

1
Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO), Techno-Pôle 3, 3960 Sierre, Switzerland
2
University Hospitals and University of Geneva, Rue Gabrielle-Perret-Gentil 4, 1205 Geneva, Switzerland
3
Ecole Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland
These authors contributed equally to this work.
*
Author to whom correspondence should be addressed.
Academic Editors: Gonzalo Pajares Martinsanz, Philip Morrow and Kenji Suzuki
Received: 26 February 2016 / Revised: 9 May 2016 / Accepted: 27 May 2016 / Published: 7 June 2016
(This article belongs to the Special Issue Image and Video Processing in Medicine)
View Full-Text   |   Download PDF [1625 KB, uploaded 7 June 2016]   |  

Abstract

Many medical image analysis tasks require complex learning strategies to reach a quality of image-based decision support that is sufficient in clinical practice. The analysis of medical texture in tomographic images, for example of lung tissue, is no exception. Via a learning framework, very good classification accuracy can be obtained, but several parameters need to be optimized. This article describes a practical framework for efficient distributed parameter optimization. The proposed solutions are applicable for many research groups with heterogeneous computing infrastructures and for various machine learning algorithms. These infrastructures can easily be connected via distributed computation frameworks. We use the Hadoop framework to run and distribute both grid and random search strategies for hyperparameter optimization and cross-validations on a cluster of 21 nodes composed of desktop computers and servers. We show that significant speedups of up to 364× compared to a serial execution can be achieved using our in-house Hadoop cluster by distributing the computation and automatically pruning the search space while still identifying the best-performing parameter combinations. To the best of our knowledge, this is the first article presenting practical results in detail for complex data analysis tasks on such a heterogeneous infrastructure together with a linked simulation framework that allows for computing resource planning. The results are directly applicable in many scenarios and allow implementing an efficient and effective strategy for medical (image) data analysis and related learning approaches. View Full-Text
Keywords: hyperparameter optimization; grid search; random search; support vector machines; random forests; distributed computing; image analysis hyperparameter optimization; grid search; random search; support vector machines; random forests; distributed computing; image analysis
Figures

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. (CC BY 4.0).

Scifeed alert for new publications

Never miss any articles matching your research from any publisher
  • Get alerts for new papers matching your research
  • Find out the new papers from selected authors
  • Updated daily for 49'000+ journals and 6000+ publishers
  • Define your Scifeed now

SciFeed Share & Cite This Article

MDPI and ACS Style

Schaer, R.; Müller, H.; Depeursinge, A. Optimized Distributed Hyperparameter Search and Simulation for Lung Texture Classification in CT Using Hadoop. J. Imaging 2016, 2, 19.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
J. Imaging EISSN 2313-433X Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top