Hyperspectral Image Enhancement and Mixture Deep-Learning Classification of Corneal Epithelium Injuries
Abstract
:1. Introduction
2. Related Works
3. Materials and Methods
3.1. Experimental Set-Up
3.1.1. Hyperspectral Cornea Image Collections
3.2. Image Enhancement
3.2.1. HSI Data Normalisation
3.2.2. Brightness and Contrast Adjustment
3.2.3. Morphological Transformation
3.2.4. Laplacian of Gaussian Filter (LoG)
3.2.5. Principal Component Analysis (PCA)
3.2.6. Image Subtraction
3.3. Support Vector Machine-Gaussian Radial Basis Function (SVM-GRBF)
3.4. Convolutional Neural Networks (AlexNet)
3.4.1. Transfer Learning Using Pretrained AlexNet with a Fine-Tuned Model on the Cornea Images
3.4.2. Feature Extraction with Pretrained AlexNet on Cornea Images
3.5. Mixture AlexNet and SVM-Linear
4. Results and Discussion
5. Conclusions
Acknowledgments
Author Contributions
Conflicts of Interest
References
- World Health Organization. Visual Impairment and Blindness; World Health Organization: Geneva, Switzerland, 2014. [Google Scholar]
- Robaei, D.; Watson, S. Corneal blindness: A global problem. Clin. Exp. Ophthalmol. 2014, 42, 213–214. [Google Scholar] [CrossRef] [PubMed]
- Goetz, A.F.H.; Vane, G.; Solomon, J.E.; Rock, B.N. Imaging spectrometry for Earth remote sensing. Science 1985, 228, 1147–1153. [Google Scholar] [CrossRef] [PubMed]
- Hege, E.K.; O’Connell, D.; Johnson, W.; Basty, S.; Dereniak, E.L. Hyperspectral imaging for astronomy and space surveillance. In Proceedings of the Optical Science and Technology, SPIE’s 48th Annual Meeting, New Orleans, LA, USA, 2–6 August 2004; Volume 5159, pp. 380–391. [Google Scholar] [CrossRef]
- Huang, H.; Liu, L.; Ngadi, M.O. Recent developments in hyperspectral imaging for assessment of food quality and safety. Sensors 2014, 14, 7248–7276. [Google Scholar] [CrossRef] [PubMed]
- García-Allende, P.B.; Conde, O.M.; Mirapeix, J.M.; Cobo, A.; Lopez-Higuera, J.M. Hyperspectral imaging sustains competitiveness. SPIE Newsroom 2010, 2–5. [Google Scholar] [CrossRef]
- Marshall, S.; Kelman, T.; Qiao, T.; Murray, P.; Zabalza, J. Hyperspectral imaging for food applications. In Proceedings of the 2015 European Signal Processing Conference (EUSIPCO 2015), Nice, France, 31 August–4 September 2015; pp. 2854–2858. [Google Scholar]
- ElMasry, G.; Sun, D.W. Principles of Hyperspectral Imaging Technology; Academic Press: Oxford, UK, 2010; ISBN 9780123747532. [Google Scholar]
- Polder, G.; Van Der Heijden, G.W.; Young, I.T. Spectral Image Analysis for Measuring Ripeness of Tomatoes. Trans. ASAE 2002, 45, 1155–1161. [Google Scholar] [CrossRef]
- Liang, H. Advances in multispectral and hyperspectral imaging for archaeology and art conservation. Appl. Phys. A Mater. Sci. Process. 2012, 106, 309–323. [Google Scholar] [CrossRef]
- Fabelo, H.; Ortega, S.; Kabwama, S.; Callico, G.M.; Bulters, D.; Szolna, A.; Pineiro, J.F.; Sarmiento, R. HELICoiD project: A new use of hyperspectral imaging for brain cancer detection in real-time during neurosurgical operations. In Proceedings of the SPIE Commercial + Scientific Sensing and Imaging. International Society for Optics and Photonics, Baltimore, MD, USA, 17–21 April 2016; Volume 9860. [Google Scholar]
- Calin, M.A.; Parasca, S.V.; Savastru, R.; Manea, D. Characterization of burns using hyperspectral imaging technique—A preliminary study. Burns 2014, 41, 118–124. [Google Scholar] [CrossRef] [PubMed]
- Regeling, B.; Thies, B.; Gerstner, A.O.H.; Westermann, S.; Müller, N.A.; Bendix, J.; Laffers, W. Hyperspectral Imaging Using Flexible Endoscopy for Laryngeal Cancer Detection. Sensors 2016, 16, 1288. [Google Scholar] [CrossRef] [PubMed]
- Kumashiro, R.; Konishi, K.; Chiba, T.; Akahoshi, T.; Nakamura, S.; Murata, M.; Tomikawa, M.; Matsumoto, T.; Maehara, Y.; Hashizume, M. An integrated endoscopic system based on optical imaging and hyper spectral data analysis for colorectal cancer detection. Anticancer Res. 2016, 3932, 3925–3932. [Google Scholar]
- Zakian, C.; Pretty, I.; Ellwood, R. Near-infrared hyperspectral imaging of teeth for dental caries detection. J. Biomed. Opt. 2009, 14, 64047. [Google Scholar] [CrossRef] [PubMed]
- Timoney, P.J.; Breathnach, C.S.I.J. Allvar Gullstrand and the slit lamp 1911. Irish J. Med. Sci. 2013, 182. [Google Scholar] [CrossRef] [PubMed]
- Rio-Cristobal, A.; Martin, R. Corneal assessment technologies: Current status. Surv. Ophthalmol. 2014, 59, 599–614. [Google Scholar] [CrossRef] [PubMed]
- Sakata, L.M.; Lavanya, R.; Friedman, D.S.; Aung, H.T.; Gao, H.; Kumar, R.S.; Foster, P.J.; Aung, T. Comparison of Gonioscopy and Anterior Segment Ocular Coherence Tomography in Detecting Angle Closure in Different Quadrants of the Anterior Chamber Angle. Ophthalmology 2008, 115, 769–774. [Google Scholar] [CrossRef] [PubMed]
- Singh, R.P.; Goldberg, I.; Graham, S.L.; Sharma, A.; Mohsin, M. Central corneal thickness, tonometry, and ocular dimensions in glaucoma and ocular hypertension. J. Glaucoma 2001, 10, 206–210. [Google Scholar] [CrossRef] [PubMed]
- Chien, S. The National Institute of Biomedical Imaging and Bioengineering. Annu. Rev. Biomed. Eng. 2004, 6. [Google Scholar] [CrossRef] [PubMed]
- McCarey, B.E.; Edelhauser, H.F.; Lynn, M.J. Review of Corneal Endothelial Specular Microscopy for FDA Clinical Trials of Refractive Procedures, Surgical Devices, and New Intraocular Drugs and Solutions. Cornea 2008, 27, 1–16. [Google Scholar] [CrossRef] [PubMed]
- Grewal, D.S.; Grewal, S.P.S. Clinical applications of Scheimpflug imaging in cataract surgery. Saudi J. Ophthalmol. 2012, 26, 25–32. [Google Scholar] [CrossRef] [PubMed]
- Jain, R.; Grewal, S.P.S. Pentacam: Principle and Clinical Applications. J. Curr. Glaucoma Pract. 2009, 3, 20–32. [Google Scholar] [CrossRef]
- Bouma, B.E.; Tearney, G.J.; Vakoc, B.; Yun, S.H. Optical Coherence Tomography. Opt. Coherence Tomogr. 2015, 77, 225–254. [Google Scholar]
- Belin, M.W.; Khachikian, S.S.; McGhee, C.N.J.; Patel, D. New Technology in Corneal Imaging. Int. Ophthalmol. Clin. 2010, 50, 177–189. [Google Scholar] [CrossRef] [PubMed]
- Reynaud, J.; Beuerman, R.W.; Khoobehi, B.; Beach, J.; Lanoue, M.; Schwarz, M.; Galloway-Dawkins, R. Confocal Hyperspectral Imaging of the Cornea. Investig. Ophthalmol. Vis. Sci. 2003, 44, 360. [Google Scholar]
- Mordant, D.J.; Al-Abboud, I.; Muyo, G.; Gorman, A.; Sallam, A.; Ritchie, P.; Harvey, A.R.; McNaught, A.I. Spectral imaging of the retina. Eye 2011, 25, 309–320. [Google Scholar] [CrossRef] [PubMed]
- Mordant, D.J.; Al-Abboud, I.; Muyo, G.; Gorman, A.; Harvey, A.R.; McNaught, A.I. Oxygen saturation measurements of the retinal vasculature in treated asymmetrical primary open-angle glaucoma using hyperspectral imaging. Eye 2014, 28, 1190–1200. [Google Scholar] [CrossRef] [PubMed]
- Li, Q.; Xue, Y.; Xiao, G.; Zhang, J. New microscopic pushbroom hyperspectral imaging system for application in diabetic retinopathy research. J. Biomed. Opt. 2007, 12, 0640. [Google Scholar] [CrossRef] [PubMed]
- Fukuda, M.; Sasaki, H. Quantitative evaluation of corneal epithelial injury caused by n-heptanol using a corneal resistance measuring device in vivo. Clin. Ophthalmol. 2012, 6, 585–593. [Google Scholar] [CrossRef] [PubMed]
- Sun, D.W. Hyperspectral Imaging for Food Quality Analysis and Control; Elsevier Inc.: Oxford, UK, 2010; ISBN 978-0-12-373642-0. [Google Scholar]
- Elsheikh, A.; Alhasso, D.; Rama, P. Biomechanical properties of human and porcine corneas. Exp. Eye Res. 2008, 86, 783–790. [Google Scholar] [CrossRef] [PubMed]
- Lee, G.A.; Chiang, M.Y.M.; Shah, P. Pig eye trabeculectomy—A wet-lab teaching model. Eye 2006, 20, 32–37. [Google Scholar] [CrossRef] [PubMed]
- Noor, S.S.M.; Michael, K.; Marshall, S.; Ren, J.; Tschannerl, J.; Kao, F.J. The properties of the cornea based on hyperspectral imaging: Optical biomedical engineering perspective. In Proceedings of the 2016 International Conference on Systems, Signals and Image Processing (IWSSIP), Bratislava, Slovakia, 23–25 May 2016. [Google Scholar]
- Isaac, B. Handbook of Medical Image Processing and Analysis, 2nd ed.; Elsevier Inc.: Oxford, UK, 2009; ISBN 978-0-12-373904-9. [Google Scholar]
- Gonzalez, R.C.; Woods, R.E. Digital Image Processing, 3rd ed.; Pearson Prentice Hall: Upper Saddle River, NJ, USA, 2008. [Google Scholar]
- Dong, B.; Yang, J.; Hao, S.; Zhang, X. Research on an Improved Medical Image Enhancement Algorithm Based on P-M Model. Open Biomed. Eng. J. 2015, 9, 209–213. [Google Scholar] [CrossRef] [PubMed]
- Irgenfried, S.; Hock, J. Acquisition and storage of multispectral material signatures. In Proceedings of the 2nd International Conference on Optical Characterization of Material, Karlsruhe, Germany, 18–19 March 2015; pp. 123–135. [Google Scholar]
- Qin, J. Hyperspectral Imaging Instruments. In Hyperspectral Imaging for Food Quality Analysis and Control; Elsevier: Amsterdam, The Netherlands, 2010; pp. 129–172. [Google Scholar]
- Adams, R. Radial Decomposition of Disks and Spheres. CVGIP Graph. Models Image Process. 1993, 55, 325–332. [Google Scholar] [CrossRef]
- Sternberg, S.R. Grayscale morphology. Comput. Vis. Graph. Image Process. 1986, 35, 333–355. [Google Scholar] [CrossRef]
- Verma, R.; Mehrotra, R.; Bhateja, V. An Improved Algorithm for Noise Suppression and Baseline Correction of ECG Signals. In Advances in Intelligent Systems and Computing; Springer: Berlin/Heidelberg, Germany, 2013; Volume 327, pp. 733–739. [Google Scholar]
- Firoz, R.; Ali, M.S.; Khan, M.N.U.; Hossain, M.K.; Islam, M.K.; Shahinuzzaman, M. Medical Image Enhancement Using Morphological Transformation. J. Data Anal. Inf. Process. 2016, 4, 1–12. [Google Scholar] [CrossRef]
- Marr, B.Y.D. Theory of edge detection. Proc. R. Soc. B 1980, 217, 187–217. [Google Scholar] [CrossRef]
- De Juan, A.; Ferrer, A. Multivariate image analysis: A review with applications. Chemom. Intell. Lab. Syst. 2011, 107, 1–23. [Google Scholar] [CrossRef]
- Rodarmel, C.; Shan, J. Principal Component Analysis for Hyperspectral Image Classification. Surv. Land Inf. Syst. 2002, 62, 115–123. [Google Scholar] [CrossRef]
- Lu, G.; Fei, B. Medical hyperspectral imaging: A review. J. Biomed. Opt. 2014, 19, 10901. [Google Scholar] [CrossRef] [PubMed]
- Rasmussen, C.E.; Williams, C.K.I. Gaussian Processes for Machine Learning; Massachusetts Institute of Technology: Cambridge, MA, USA, 2006. [Google Scholar]
- Melgani, F.; Bruzzone, L. Classification of Hyperspectral Remote Sensing. IEEE Trans. Geosci. Remote Sens. 2004, 42, 1778–1790. [Google Scholar] [CrossRef]
- Krizhevsky, A.; Sutskever, I.; Hinton, G. ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the Advances in Neural Information Processing Systems 25, Lake Tahoe, NV, USA, 3–8 November 2012. [Google Scholar]
- Nogueira, R.F.; de Alencar Lotufo, R.; Machado, R.C. Fingerprint Liveness Detection using Convolutional Networks. IEEE Trans. Inf. Forensics Secur. 2016, 11, 1206–1213. [Google Scholar] [CrossRef]
- Oquab, M.; Bottou, L.; Laptev, I.; Sivic, J. Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks. IEEE Conf. Comput. Vis. Pattern Recognit. 2014, 1717–1724. [Google Scholar] [CrossRef]
- Chen, Y.; Jiang, H.; Li, C.; Jia, X.; Ghamisi, P. Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks. IEEE Trans. Geosci. Remote Sens. 2016, 54, 6232–6251. [Google Scholar] [CrossRef]
- Elleuch, M.; Maalej, R.; Kherallah, M. A New design based-SVM of the CNN classifier architecture with dropout for offline Arabic handwritten recognition. Procedia Comput. Sci. 2016, 80, 1712–1723. [Google Scholar] [CrossRef]
- Wang, B.; Wang, X.; Chen, Z. Spatial entropy based mutual information in hyperspectral band selection for supervised classification. Int. J. Numer. Anal. Model. 2012, 9, 181–192. [Google Scholar]
- Malik, F.; Baharudin, B. Analysis of distance metrics in content-based image retrieval using statistical quantized histogram texture features in the DCT domain. J. King Saud Univ. Inf. Sci. 2013, 25, 207–218. [Google Scholar] [CrossRef]
- Selvarajah, S.; Kodituwakku, S.R. Analysis and Comparison of Texture Features for Content Based Image Retrieval. Int. J. Latest Trends Comput. 2011, 2, 108–113. [Google Scholar]
- Aggarwal, N.; Agrawal, R.K. First and Second Order Statistics Features for Classification of Magnetic Resonance Brain Images. J. Signal Inf. Process. 2012, 3, 146–153. [Google Scholar] [CrossRef]
- Fawcett, T. ROC Graphs: Notes and Practical Considerations for Data Mining Researchers. HP Inven. 2003, 27. [Google Scholar]
- Bettinger, R. Cost-Sensitive Classifier Selection Using the ROC Convex Hull Method; SAS Institute: Cary, NC, USA, 2003; pp. 1–12. [Google Scholar]
Lab | Quantity | Camera Type | Image Scanned | Remarks |
---|---|---|---|---|
Lab 1 | Supplier A 5 Pigs Eyes | VIS-NIR (400 to 1000 nm) | 6 Scanned (3 injured 3 healthy) | Pilot test [34] Image Dimension after binning: 1200 to 1300 × 804 × 604 302 spectral bands. |
Lab 2 | Supplier A 30 Pigs Eyes | VIS-NIR (400 to 1000 nm) | 17 Scanned (from 8 Eyes) (5 injured + 7 Stained 1 Healthy + 1 Stained 3 No Intact Epithelium) 22 Eyes Rejected | Apply stains Image Dimension after binning: 500 to 700 × 336 × 256 256 spectral bands |
Lab 3 | Supplier B 12 Pigs Eyes | VIS-NIR (400 to 1000 nm) | 26 Scanned (8 injured + 10 stained 4 healthy + 4 stained) | Apply stains Image Dimension after binning: 250 to 400 × 336 × 256 256 spectral bands |
No | Layer | Type | Parameters |
---|---|---|---|
1 | Data | Image Input | Layer1: Convolution layer Input image size: 227 × 227 × 3 with zero centre normalisation No. of filters: 96 Filter size: 11 × 11 × 3 Stride: [4 4] Output: 224/4 × 224/4 × 96 (because of stride 4) Train Network with a CPU |
2 | Conv1 | Convolution | |
3 | Relu1 | Relu | Rectified linear units |
4 | Norm1 | Cross channel normalisation | Cross channel normalisation with 5 channels per element |
5 | Pool1 | Max pooling | Layer2: Max pooling followed by convolution Input: 55 × 55 × 96 Max pooling: 55/2 × 55/2 × 96 = 27 × 27 × 96 No. of filters: 256 Filter size: 5 × 5 × 48 Stride: [2 2] Output: 27 × 27 × 256 Train Network with a CPU |
6 | Conv2 | Convolution | |
7 | Relu2 | Relu | Rectified linear units |
8 | Norm2 | Cross channel normalisation | Cross channel normalisation with 5 channels per element |
9 | Pool2 | Max pooling | Layer3: Max pooling followed by convolution Input: 27 × 27 × 256 Max pooling: 27/2 × 27/2 × 256 = 13 × 13 × 256 No. of filters: 384 Filter size: 3 × 3 × 256 Stride: [2 2] Output: 13 × 13 × 384 Train Network with a CPU |
10 | Conv3 | Convolution | |
11 | Relu3 | Relu | Rectified linear units |
12 | Conv4 | Convolution | Layer4: Convolution layer Input: 13 × 13 × 192No. of filters: 384 Filter size: 3 × 3 × 192 Stride: [1 1] Output: 13 × 13 × 384 Train Network with a CPU |
13 | Relu4 | Relu | Rectified linear units |
14 | Conv5 | Convolution | Layer5: Convolution layer Input: 13 × 13 × 192 No. of filters: 256 Filter size: 3 × 3 × 192 Stride: [1 1] Output: 13 × 13 × 256 Train Network with a CPU |
15 | Relu5 | Relu | Rectified linear units |
16 | Pool5 | Max pooling | 3 × 3 max pooling with stride [2 2] |
17 | Fc6 | Fully connected | Layer6: Fully connected layer Input: 13 × 13 × 128 is transformed into a vector Output: 4096-dimensional feature with 2048 in each vector |
18 | Relu6 | Relu | Rectified linear units |
19 | Drop6 | Dropout | Reducing overfitting with probability 0.5 |
20 | Fc7 | Fully connected | Layer7: Fully connected layer 4096-dimensional feature with 2048 in each vector |
21 | Relu7 | Relu | Rectified linear units |
22 | Drop7 | Dropout | Reducing overfitting with probability 0.5 |
23 | Fc8 | Fully connected | Layer8: Fully connected layer 2 number of classes |
24 | Prob | SoftMax | Reducing overfitting |
25 | Output | Classification output | Classify 2 image: Healthy and Injured |
EYE Healthy | Mean Healthy | Standard Deviation Healthy | Skewness Healthy | Kurtosis Healthy | EYE Injured | Mean Injured | Standard Deviation Injured | Skewness Injured | Kurtosis Injured |
---|---|---|---|---|---|---|---|---|---|
1 | 135.31 | 28.10 | 0.69 | 4.79 | 12 | 125.85 | 26.46 | 0.51 | 5.36 |
2 | 97.23 | 28.67 | 0.92 | 5.86 | 13 | 110.69 | 26.74 | 0.83 | 6.45 |
3 | 101.50 | 27.87 | 0.84 | 6.09 | 14 | 81.23 | 22.58 | 1.43 | 8.40 |
4 | 80.91 | 22.34 | 1.22 | 8.73 | 15 | 82.07 | 23.08 | 0.83 | 5.81 |
5 | 88.11 | 25.90 | 0.95 | 7.26 | 16 | 76.56 | 19.28 | 1.16 | 8.80 |
6 | 102.41 | 26.99 | 1.04 | 6.47 | 17 | 79.44 | 28.30 | 1.02 | 5.54 |
7 | 100.73 | 19.88 | 1.10 | 7.74 | 18 | 67.84 | 20.04 | 1.11 | 8.01 |
8 | 108.48 | 21.03 | 1.25 | 9.66 | 19 | 73.76 | 27.27 | 1.11 | 6.24 |
9 | 89.85 | 24.00 | 1.26 | 8.21 | 20 | 116.46 | 27.40 | 0.61 | 4.64 |
10 | 99.75 | 27.72 | 0.89 | 6.08 | 21 | 120.36 | 21.74 | 0.76 | 6.61 |
11 | 98.96 | 22.73 | 1.18 | 8.66 | 22 | 96.72 | 27.15 | 0.94 | 6.51 |
23 | 108.97 | 28.55 | 0.67 | 5.71 | |||||
24 | 101.93 | 29.25 | 0.27 | 4.67 | |||||
25 | 105.74 | 24.04 | 0.37 | 4.67 |
Features | C = 1 | C = 500 | C = 500 | ||||||
---|---|---|---|---|---|---|---|---|---|
Sigma = 1 | Sigma = 1.658 | Sigma = 2.658 | |||||||
10-Fold Cross Validation | 10-Fold Cross Validation | 10-Fold Cross Validation | |||||||
Iterations | Accuracy | Error | Iterations | Accuracy | Error | Iterations | Accuracy | Error | |
Mean-Std. | 13 | 0.2708 | 0.4545 | 81 | 0.5625 | 0.3636 | 578 | 0.4792 | 0.4545 |
Mean-Skew | 13 | 0.8333 | 0.3636 | 148 | 0.9583 | 0.4545 | 412 | 1 | 0.4545 |
Mean-Kurt | 6 | 0.7500 | 0.3636 | 169 | 0.8125 | 0.3636 | 189 | 0.5208 | 0.2727 |
Std.-Skew | 10 | 0.6042 | 0.4545 | 161 | 0.2083 | 0.5455 | 207 | 0.1875 | 0.6364 |
Std.-Kurt | 6 | 0.3750 | 0.1818 | 419 | 0.6875 | 0.6364 | 172 | 0.7083 | 0.4545 |
Skew-Kurt | 12 | 0.6875 | 0 | 200 | 0.5833 | 0.1818 | 243 | 0.7292 | 0.0909 |
4-Features | 11 | 0.4375 | 0.3636 | 38 | 0.7292 | 0.4545 | 86 | 0.4583 | 0.4545 |
Confusion | Predict | Predict |
---|---|---|
Matrix | Healthy | Injured |
Actual | True | False |
Healthy | Negative (TN) | Positive (FP) |
Actual | False | True |
Injured | Negative (FN) | Positive (TP) |
© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Md Noor, S.S.; Ren, J.; Marshall, S.; Michael, K. Hyperspectral Image Enhancement and Mixture Deep-Learning Classification of Corneal Epithelium Injuries. Sensors 2017, 17, 2644. https://doi.org/10.3390/s17112644
Md Noor SS, Ren J, Marshall S, Michael K. Hyperspectral Image Enhancement and Mixture Deep-Learning Classification of Corneal Epithelium Injuries. Sensors. 2017; 17(11):2644. https://doi.org/10.3390/s17112644
Chicago/Turabian StyleMd Noor, Siti Salwa, Jinchang Ren, Stephen Marshall, and Kaleena Michael. 2017. "Hyperspectral Image Enhancement and Mixture Deep-Learning Classification of Corneal Epithelium Injuries" Sensors 17, no. 11: 2644. https://doi.org/10.3390/s17112644
APA StyleMd Noor, S. S., Ren, J., Marshall, S., & Michael, K. (2017). Hyperspectral Image Enhancement and Mixture Deep-Learning Classification of Corneal Epithelium Injuries. Sensors, 17(11), 2644. https://doi.org/10.3390/s17112644