# Hyperparameter Optimization for Image Recognition over an AR-Sandbox Based on Convolutional Neural Networks Applying a Previous Phase of Segmentation by Color–Space

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Background

## 3. Materials and Methods

#### 3.1. Method and Approach of the Scenario

#### 3.2. Development of the Scenario

#### 3.2.1. Image Acquisition

#### 3.2.2. Image Processing

#### 3.2.3. Image Recognition through CNN

#### Base Structure of the CNN Model

#### Dictionary of Hyperparameters

^{n}; for this reason, the values between 16, 32, 64, and 128 were selected. On the other hand, it was selected as possible activation functions: ReLU and LeakyReLU; on the one hand, the first one is the activation function that is most commonly used in deep learning, and the second one is an attempt to solve the dying ReLU problem. The number of units or hidden networks was defined at a proportion of 2

^{n}, proposing 64, 128, 256, and 512 as possibilities. The elimination rate must be between 0 and 1, but having a rate that is equal to 0 or 1 would be illogical; for this reason, a range between 0.25 and 0.75 was determined with a step of 0.25. Finally, we have the hyperparameter of batch_size and the optimizer; for the first one, a sequence of 2, 4, 8, 16, and 32 was generated, given that the data set was not very large for making larger batches. The options of the established optimizer were: Adam, RMSprop, Nadam, and Adadelta.

#### Model Preselection

^{n}, up to n = 5, given the size of the training dataset.

#### Model Selection

## 4. Measurement of the Model and Performance Evaluation

#### 4.1. Function Evaluate

**DataSetOriginal.**Table 4 shows the loss value, which is 0.72, the percentage of successes, which is 70%, and the percentage of mean squared error, which is 23.26%, according to the established metric.

**DataSetFilter.**Table 5 shows the loss value, which is 0.36, the percentage of successes, which is 87%, and the percentage of mean squared error, which is 11.08%, according to that metric.

#### 4.2. Confusion Matrix

**DataSetOriginal.**Table 6 shows the number of hits that CNN had when testing with 30 images of each class in different orders; therefore, during analysis, the table of the neural network had a success rate of 61%.

**DataSetFilter.**Table 7 shows the number of hits that CNN had when testing with 30 images of each class in different orders; therefore, when analyzing the table, the neural network had an 87.8% rate of success.

#### 4.3. ROC Curve

**DataSetOriginal.**Figure 7 shows the ROC curve for the dataset with the original images, where there are five curves, two of them at a general level, and the other three at a specific level. The general curves show the averages of areas under the curve at the micro- and macro-level; on the other hand, the curves at a specific level show the area under the curve (AUC) of each of the classes, these being 0, 1, and 2 which correspond to circle, square, and triangle, respectively.

**DataSetFilter.**Figure 8 shows the ROC curve for the dataset with the processed images, where the area under the curve of the averages at the micro-level, and the macro-level of the ROC curve is shown, as well as the AUCs of each of the classes of the geometric figures that composed the test dataset.

## 5. Results, Analysis, and Discussions

## 6. Conclusions and Future Works

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Shi, W.; Caballero, J.; Husz, F.; Totz, J.; Aitken, A. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016. [Google Scholar]
- Long, J.; Shelhamer, E.; Darrell, T. Fully Convolutional Networks for Semantic Segmentation. IEEE Trans. Pattern Anal. Mach. Intell.
**2016**, 39, 640–651. [Google Scholar] - Giorgis, S.; Mahlen, N.; Anne, K. Instructor-Led Approach to Integrating an Augmented Reality Sandbox into a Large-Enrollment Introductory Geoscience Course for Nonmajors Produces No Gains. J. Geosci. Educ.
**2017**, 65, 283–291. [Google Scholar] [CrossRef] - Woods, T.; Reed, S.; His, S.; John, W.; Woods, M. Pilot Study Using the Augmented Reality Sandbox to Teach Topographic Maps and Surficial Processes in Introductory Geology Labs. J. Geosci. Educ.
**2016**, 64, 199–214. [Google Scholar] [CrossRef] - Github. Talos: Manual de Usuario. [En línea]. Available online: https://autonomio.github.io/docs_talos (accessed on 15 November 2018).
- Hijazi, S.; Kumar, R.; Rowen, C. Using Convolutional Neural Networks for Image Recognition. Cadence. 2015. Available online: https://ip.cadence.com/uploads/901/cnn_wp-pdf (accessed on 4 November 2018).
- Neha, S.; Vibhor, J.; Anju, M. An Analysis of Convolutional Neural Networks for Image Classification. Procedia Comput. Sci.
**2018**, 132, 337–384. [Google Scholar] - Niioka, E.H.; Satoshi, A.; Yoshimura, A.; Ohigashi, H.; Tagawa, S.; Miyake, J. Classification of C
_{2}C_{12}cells at differentiation by convolutional neural network of deep learning using phase contrast images. Hum. Cell**2018**, 31, 87–93. [Google Scholar] [CrossRef] [PubMed] - Congcong, Z.; Xiaoyan, X.; Xiaomei, L.; Ying-Jie, C.; Wu, Z.; Jun, C.; Chengyun, Z.Z.L. White Blood Cell Segmentation by Color-Space-Based K-Means Clustering. Sensors
**2014**, 14, 16128–16147. [Google Scholar] - Bergstra, J.; Bengio, Y. Random Search for Hyper-Parameter Optimization. J. Mach. Learn. Res.
**2012**, 13, 281–305. [Google Scholar] - Lee, K.; Lee, J.; Lee, J.; Hwang, S.; Lee, S. Brightness-Based Convolutional Neural Network for Thermal Image Enhancement. IEEE Access
**2017**, 5, 26867–26879. [Google Scholar] [CrossRef] - Yao, C.; Zhang, Y.; Zhang, Y.; Liu, H. Application of Convolutional Neural Network in Classification of High Resolution Agricultural Remote Sensing Images. Int. Arch. Photogram. Remote Sens. Spat. Inf. Sci.
**2017**, 42, 989–992. [Google Scholar] [CrossRef] - Pang, S.; Du, A.; Orgun, M.; Yu, Z. A novel fused convolutional neural networks for biomedical image classification. Med. Biol. Eng. Comput.
**2018**. [Google Scholar] [CrossRef] [PubMed] - Real, R.; Vargas, J. The Probabilistic Basis of Jaccard’s Index of Similarity. Syst. Biol.
**1996**, 45, 380–385. [Google Scholar] [CrossRef] - Shamir, R.; Duchin, Y.; Kim, J.; Sapiro, G.; Harel, N. Continuous Dice Coefficient: A Method for Evaluating Probabilistic Segmentations. Surgical Inf. Sci.
**2018**. [Google Scholar] [CrossRef] - Zou, K.; Warfield, S.; Bharatha, A.; Tempany, C.; Kaus, M.; Haker, S.; Wells, W.; Jolesz, F.; Kikinis, R. Statistical Validation of Image Segmentation Quality Based on a Spatial Overlap Index. Radiol. Alliance Health Serv. Res.
**2004**, 11, 178–189. [Google Scholar] - Hernández, F.; de Ory, E.; Aguilar, S.; Crespo, R. Residue properties for the arithmetical estimation of the image quantization table. Appl. Soft Comput.
**2018**, 67, 309–321. [Google Scholar] [CrossRef] - Meza, J.; Espitia, H.; Montenegro, C.; Crespo, R. Statistical analysis of a multi-objective optimization algorithm based on a model of particles with vorticity behavior. Soft Comput.
**2016**, 20, 3521–3536. [Google Scholar] [CrossRef] - Wang, S.-H.; Tang, C.; Sun, J.; Yang, J.; Huang, C.; Phillips, P.; Zhang, Y.-D. Multiple Sclerosis Identification by 14-Layer Convolutional Neural Network with Batch Normalization, Dropout, and Stochastic Pooling. Front. Neurosci.
**2018**, 12, 818. [Google Scholar] [CrossRef] - Kushwaha, P.; Welekar, R. Feature Selection for Image Retrieval based on Genetic Algorithm. Int. J. Interact. Multimed. Artif. Intell.
**2016**, 4, 16–21. [Google Scholar] [CrossRef] - Pacheco, A.; Barón, H.; Crespo, R.; Espada, J. Reconstruction of High Resolution 3D Objects from Incomplete Images and 3D Information. Int. J. Interact. Multimed. Artif. Intell.
**2014**, 2, 7–16. [Google Scholar] [CrossRef] - Rezaie, A.A.; Habiboghli, A. Detection of Lung Nodules on Medical Images by the Use of Fractal Segmentation. Int. J. Interact. Multimed. Artif. Intell.
**2017**, 4, 15–19. [Google Scholar] - Rosyadi, H.; Gökhan, C. Augmented reality sandbox (AR sandbox) experimental lanscape for fluvial, deltaic and volcano morphology and topography models. Turqua
**2016**. [Google Scholar] [CrossRef] - Wang, X.; Hansch, R.; Ma, L.; Hellwich, O. Comparison of Different Color Spaces for Image Segmentation using Graph-cut. In Proceedings of the International Conference on Computer Vision Theory and Applications, Lisbon, Portugal, 5–8 January 2014; pp. 301–308. [Google Scholar]
- Blessie, E.C.; Karthikeyan, E.; Selvaraj, B. Empirical study on the performance of the classifiers based on various criteria using ROC curve in medical health care. In Proceedings of the International Conference on Communication and Computational Intelligence (INCOCCI), Erode, India, 27–29 December 2010. [Google Scholar]
- Wang, X.; Shu, P.; Cao, L.; Wang, Y. A ROC Curve Method for Performance Evaluation of Support Vector Machine with Optimization Strategy. In Proceedings of the International Forum on Computer Science-Technology and Applications, Chongqing, China, 25–27 December 2009. [Google Scholar]
- Senthilnath, J.; Kulkarni, S.; Benediktsson, J.A.; Yang, X.S. A Novel Approach for Multispectral Satellite Image Classification Based on the Bat Algorithm. IEEE Geosci. Remote Sens. Lett.
**2016**, 13, 599–603. [Google Scholar] [CrossRef] - Wald, N.; Bestwick, J. Is the area under an ROC curve a valid measure of the performance of a screening or diagnostic test? J. Med. Screen.
**2014**, 21, 5160. [Google Scholar] [CrossRef] [PubMed] - Prati, R.C.; Batista, G.E.A.P.A.; Monard, M.C. Curvas ROC para avaliação de classificadores. IEEE Latin Am. Trans.
**2008**, 6, 215–222. [Google Scholar] [CrossRef]

Parameter | Value |
---|---|

Optimizer | optimizer |

Loss | categorical_crossentropy |

Metrics | Accuracy, mean squared error |

Key | List of Values | ||

Filter_1 | 16, 32 | ||

Filter_2 | 32, 64 | ||

Filter_3 | 64, 128 | ||

Activation_1 | Relu, LeakyReLU | ||

Rate_1 | 0.25, 0.5, 0.75 | ||

Rate_2 | 0.25, 0.5, 0.75 | ||

Rate_3 | 0.25, 0.5, 0.75 | ||

Rate_4 | 0.25, 0.5, 0.75 | ||

Units_1 | 64, 128, 256, 512 | ||

Batch_size | 2, 4, 8, 16, 32 | ||

Optimizer | Adam, RMSProp, Nadam, Adadelta | ||

Model’s ID | Accuracy | Mean Squared Error | Loss |

1 | 0.9871 | 0.0634 | 0.0088 |

2 | 0.9615 | 0.1021 | 0.0194 |

3 | 0.9615 | 0.0729 | 0.0138 |

4 | 0.9923 | 0.1537 | 0.0185 |

5 | 0.941 | 0.1448 | 0.0279 |

6 | 0.9333 | 0.1771 | 0.032 |

7 | 0.9641 | 0.2625 | 0.0251 |

8 | 0.9743 | 0.0958 | 0.0158 |

9 | 0.9820 | 0.1619 | 0.0129 |

10 | 0.8820 | 0.2886 | 0.0551 |

Measure | Dataset | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
---|---|---|---|---|---|---|---|---|---|---|---|

Validation_accuracy | Original | 0.7333 | 0.7111 | 0.7111 | 0.6444 | 0.5333 | 0.6111 | 0.7000 | 0.6555 | 0.7111 | 0.5888 |

Filter | 0.7777 | 0.8999 | 0.7666 | 0.8333 | 0.7555 | 0.7444 | 0.8700 | 0.7666 | 0.8666 | 0.7222 | |

Difference | 0.0444 | 0.1888 | 0.0555 | 0.1889 | 0.2222 | 0.1333 | 0.1700 | 0.1111 | 0.1555 | 0.1334 | |

Validation_Loss | Original | 1.0977 | 1.0986 | 1.0978 | 1.1144 | 1.1027 | 1,0975 | 0.7200 | 1.0978 | 1.0985 | 0.9946 |

Filter | 0.6981 | 0.6234 | 0.7078 | 0.7630 | 0.7082 | 0.7320 | 0.3600 | 0.6678 | 0.4541 | 0.8605 | |

Difference | 0.3996 | 0.4752 | 0.3900 | 0.3514 | 0.3945 | 0.3655 | 0.3600 | 0.4300 | 0.6444 | 0.1341 | |

Validation_mean_squared_error | Original | 0.2222 | 0.2222 | 0.2222 | 0.2257 | 0.2231 | 0.2219 | 0.2326 | 0.2222 | 0.2222 | 0.1766 |

Filter | 0.1061 | 0.0595 | 0.1340 | 0.0922 | 0.1052 | 0.1308 | 0.1108 | 0.1074 | 0.0651 | 0.1665 | |

Difference | 0.1161 | 0.1627 | 0.0882 | 0.1335 | 0.1179 | 0.0911 | 0.1218 | 0.1148 | 0.1571 | 0.0101 |

Parameter | Function | Value |
---|---|---|

Loss | categorical_crossentropy | 0.7200 |

Metrics | Accuracy | 0.7000 |

Mean squared error | 0.2326 |

Parameter | Function | Value |
---|---|---|

Loss | categorical_crossentropy | 0.2300 |

Metrics | Accuracy | 0.8700 |

Mean squared error | 0.1108 |

Circle | Square | Triangle | |
---|---|---|---|

Circle | 24 | 0 | 6 |

Square | 13 | 13 | 4 |

Triangle | 7 | 5 | 18 |

Circle | Square | Triangle | |
---|---|---|---|

Circle | 28 | 0 | 2 |

Square | 5 | 22 | 3 |

Triangle | 0 | 1 | 29 |

Datasets | Figure | Hits (%) | Failures (%) |
---|---|---|---|

dataSetOriginal | Circle | 80 | 20 |

Square | 43.3 | 56.3 | |

Triangle | 60 | 40 | |

dataSetFilter | Circle | 93.3 | 6.7 |

Square | 73.3 | 26.3 | |

Triangle | 96.7 | 3.3 |

Datasets | Class | Figure | AUC |
---|---|---|---|

dataSetTOrginal | 0 | Circle | 0.87 |

1 | Square | 0.93 | |

2 | Triangle | 0.92 | |

dataSetFilter | 0 | Circle | 0.98 |

1 | Square | 0.94 | |

2 | Triangle | 0.98 |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Restrepo Rodríguez, A.O.; Casas Mateus, D.E.; Gaona García, P.A.; Montenegro Marín, C.E.; González Crespo, R. Hyperparameter Optimization for Image Recognition over an AR-Sandbox Based on Convolutional Neural Networks Applying a Previous Phase of Segmentation by Color–Space. *Symmetry* **2018**, *10*, 743.
https://doi.org/10.3390/sym10120743

**AMA Style**

Restrepo Rodríguez AO, Casas Mateus DE, Gaona García PA, Montenegro Marín CE, González Crespo R. Hyperparameter Optimization for Image Recognition over an AR-Sandbox Based on Convolutional Neural Networks Applying a Previous Phase of Segmentation by Color–Space. *Symmetry*. 2018; 10(12):743.
https://doi.org/10.3390/sym10120743

**Chicago/Turabian Style**

Restrepo Rodríguez, Andrés Ovidio, Daniel Esteban Casas Mateus, Paulo Alonso Gaona García, Carlos Enrique Montenegro Marín, and Rubén González Crespo. 2018. "Hyperparameter Optimization for Image Recognition over an AR-Sandbox Based on Convolutional Neural Networks Applying a Previous Phase of Segmentation by Color–Space" *Symmetry* 10, no. 12: 743.
https://doi.org/10.3390/sym10120743