# The Optimization of the Light-Source Spectrum Utilizing Neural Networks for Detecting Oral Lesions

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

#### 1.1. Light Sources and Optimization

#### 1.2. Periodontal Disease

#### 1.3. Spectral Imaging for Machine Learning

#### 1.4. The Aim and Contribution of This Study

## 2. Materials and Methods

#### 2.1. The Color Observation Model

#### 2.2. Alternating Optimization: A Light-Source Spectrum Optimization for Machine Learning

#### 2.3. The Optimization of the Light-Source Spectrum Using a Neural Network

**x**, is used for rendering, and the optimized light source is reproduced using the weighted sum of the SPD of the sub-light with $\mathit{x}$. The infrared value $\widehat{y}$ can be represented as:

#### 2.4. Problem Setting for Oral Lesions Detection

#### 2.4.1. The One-vs-Rest Classification

#### 2.4.2. The Input and Output

#### 2.4.3. Initial Lighting Weights

- A randomized vector in the range of $\left[-1,1\right]$;

- Weights which construct a cumulative SPD using weighted sub-lights that approximates the D65;
- Uniform weights with
**1**: ${1}^{{N}_{\mathrm{L}}}=\left[1,1,\dots ,1\right]$.

**N**is the number of spectral components, and the lighting weights for the sub-lights are:

_{s}#### 2.4.4. The Cost Function for Alternating Optimization

**,**respectively. Thus, the cost function corresponding to the expressions in Equations (5) and (6) is the inverse of the margin and the optimization problems, as follows:

#### 2.4.5. The Cost Function for the Neural Network-Based Optimization

#### 2.4.6. The Grid Search and Trials

#### 2.5. The Materials for Oral Lesions’ Detection Problems

#### 2.5.1. Light Sources

- Measured SPDs of 24 real LEDs in the 400–830 nm band;
- Simulated SPDs, with their mean aligned at even intervals, in the 400–1000 nm band.

#### 2.5.2. The RGB Camera

#### 2.5.3. Datasets

## 3. Results and Discussion

#### 3.1. The Performance Comparison among Methods

#### 3.2. The Effect of NIR Information and Specific Initial Lighting Weights

- The camera sensitivity in the NIR region does not differ between the three RGB channels (Figure 8) and no information appears in the RGB channel;
- Upon extending the wavelength range to 1000 nm and evenly spreading them, the variety of the cumulative SPD is lost, as represented by the sub-light in the distinct wavelength band of 400–800 nm.

#### 3.3. The Optimal SPD and Stability

#### 3.4. The Distances between Classes in the Enhanced Feature Space

## 4. Limitations and Future Work

## 5. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Commission Internationale de l’eclairage. Method of Measuring and Specifying Colour Rendering Properties of Light Sources, CIE Publication 13.3-1995, Central Bureau of the CIE, Vienna, Austria, 1995, 20 Pp. Color Res. Appl.
**1995**, 20, 212. [Google Scholar] [CrossRef] - Thornton, W.A. Color-Discrimination Index. JOSA
**1972**, 62, 191–194. [Google Scholar] [CrossRef] - Murdoch, M.J.; Member, S. Dynamic Color Control in Multiprimary Tunable LED Lighting Systems. J. Soc. Inf. Disp.
**2019**, 27, 570–580. [Google Scholar] [CrossRef] - Masuda, O.; Nascimento, S.M.C. Lighting Spectrum to Maximize Colorfulness. Opt. Lett.
**2012**, 37, 407. [Google Scholar] [CrossRef] [PubMed] - Nascimento, S.M.C.; Masuda, O. Psychophysical Optimization of Lighting Spectra for Naturalness, Preference, and Chromatic Diversity. J. Opt. Soc. Am. A
**2012**, 29, A144. [Google Scholar] [CrossRef] [PubMed] - Nascimento, S.M.C.; Masuda, O. Best Lighting for Visual Appreciation of Artistic Paintings—Experiments with Real Paintings and Real Illumination. J. Opt. Soc. Am. A
**2014**, 31, A214. [Google Scholar] [CrossRef] - Flinkman, M.; Laamanen, H.; Silfsten, P.; Hauta-Kasari, M.; Vahimaa, P. Daylight Colored Optimal Spectra for Improved Color Discrimination. In Image and Signal Processing; Elmoataz, A., Lezoray, O., Nouboud, F., Mammass, D., Eds.; Springer International Publishing: Cham, Switzerland, 2014; pp. 1–8. [Google Scholar]
- Durmus, D.; Abdalla, D.; Duis, A.; Davis, W. Spectral Optimization to Minimize Light Absorbed by Artwork. Leukos
**2018**, 16, 45–54. [Google Scholar] [CrossRef] [Green Version] - Vázquez, D.; Fernández-Balbuena, A.A.; Canabal, H.; Muro, C.; Durmus, D.; Davis, W.; Benítez, A.; Mayorga, S. Energy Optimization of a Light Projection System for Buildings that Virtually Restores Artworks. Digit. Appl. Archaeol. Cult. Herit.
**2020**, 16, e00128. [Google Scholar] [CrossRef] - Saw, Y.J.; Kalavally, V.; Tan, C.P. The Spectral Optimization of a Commercializable Multi-Channel LED Panel with Circadian Impact. IEEE Access
**2020**, 8, 136498–136511. [Google Scholar] [CrossRef] - Nakauchi, S.; Himeno, T.; Nishino, K. An Efficient Designing Method of Spectral Distribution of Illuminant for the Enhancement of Color Discrimination. Color Imaging Conf.
**2011**, 2011, 304–309. [Google Scholar] - Ito, K.; Higashi, H.; Ota, Y.; Nakauchi, S. Spectral-Difference Enhancing Illuminant for Improving Visual Detection of Blood Vessels. In Proceedings of the ICAICTA 2015—2015 International Conference on Advanced Informatics: Concepts, Theory and Applications, Chonburi, Thailand, 19–22 August 2015. [Google Scholar] [CrossRef]
- Liu, P.; Wang, H.; Zhang, Y.; Shen, J.; Wu, R.; Zheng, Z.; Li, H.; Liu, X. Investigation of Self-Adaptive LED Surgical Lighting Based on Entropy Contrast Enhancing Method. Opt. Commun.
**2014**, 319, 133–140. [Google Scholar] [CrossRef] - Kondo, T.; Nieves, J.L.; Valero, E.M.; Higashi, H.; Nakauchi, S. Functional Illumination Supporting the Visual Detection of Plaques. Color Imaging Conf.
**2016**, 2016, 219–224. [Google Scholar] [CrossRef] - Bartczak, P.; Fält, P.; Penttinen, N.; Ylitepsa, P.; Laaksonen, L.; Lensu, L.; Hauta-Kasari, M.; Uusitalo, H. Spectrally Optimal Illuminations for Diabetic Retinopathy Detection in Retinal Imaging. Opt. Rev.
**2017**, 24, 105–116. [Google Scholar] [CrossRef] - Hyttinen, J.; Fält, P.; Fauch, L.; Riepponen, A.; Kullaa, A.; Hauta-Kasari, M. Contrast Enhancement of Dental Lesions by Light Source Optimisation. In Image and Signal Processing; Mansouri, A., El Moataz, A., Nouboud, F., Mammass, D., Eds.; Springer International Publishing: Cham, Switzerland, 2018; pp. 499–507. [Google Scholar]
- Ohno, Y. Color Rendering and Luminous Efficacy of White LED Spectra. In Proceedings of the Fourth International Conference on Solid State Lighting, Denver, CO, USA, 3–6 August 2004; Volume 5530, pp. 88–98. [Google Scholar] [CrossRef] [Green Version]
- Durmus, D. Multi-Objective Optimization Trade-Offs for Color Rendition, Energy Efficiency, and Circadian Metrics. Light-Emit. Devices Mater. Appl.
**2021**, 11706, 143–151. [Google Scholar] [CrossRef] - Liu, L.; Huo, J. Apple Image Segmentation Model Based on R Component with Swarm Intelligence Optimization Algorithm. Int. J. Perform. Eng.
**2018**, 14, 1149. [Google Scholar] [CrossRef] [Green Version] - Lu, C.; Xiao, H.; Ogai, H. Low Energy Consumption Method Based on Multi-Objective PSO for Illumination Distribution Optimization. In Proceedings of the 2017 56th Annual Conference of the Society of Instrument and Control Engineers of Japan, SICE 2017, Kanazawa, Japan, 19–22 September 2017; pp. 338–342. [Google Scholar] [CrossRef]
- Higashi, H.; Bui, M.V.; Aziz, A.S.; Nakauchi, S. Computational Lighting for Extracting Optical Features from RGB Images. Measurement
**2019**, 151, 107183. [Google Scholar] [CrossRef] - He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell.
**2017**, 42, 386–397. [Google Scholar] [CrossRef] - Lawrence, H.P.; Thomson, W.M.; Broadbent, J.M.; Poulton, R. Oral Health-Related Quality of Life in a Birth Cohort of 32-Year Olds. Community Dent. Oral Epidemiol.
**2008**, 36, 305–316. [Google Scholar] [CrossRef] - Saito, A.; Hosaka, Y.; Kikuchi, M.; Akamatsu, M.; Fukaya, C.; Matsumoto, S.; Ueshima, F.; Hayakawa, H.; Fujinami, K.; Nakagawa, T. Effect of Initial Periodontal Therapy on Oral Health–Related Quality of Life in Patients With Periodontitis in Japan. J. Periodontol.
**2010**, 81, 1001–1009. [Google Scholar] [CrossRef] - World Health Organization. Oral Health Surveys: Basic Method; WHO: Geneva, Switzerland, 2013; Volume 137. [Google Scholar]
- Li, X.; Kolltveit, K.M.; Tronstad, L.; Olsen, I. Systemic Diseases Caused by Oral Infection. Clin. Microbiol. Rev
**2000**, 13, 547. [Google Scholar] [CrossRef] - Birdal, R.G.; Gumus, E.; Sertbas, A.; Birdal, I.S. Automated Lesion Detection in Panoramic Dental Radiographs. Oral Radiol.
**2016**, 32, 111–118. [Google Scholar] [CrossRef] - Lee, J.H.; Kim, D.H.; Jeong, S.N.; Choi, S.H. Detection and Diagnosis of Dental Caries Using a Deep Learning-Based Convolutional Neural Network Algorithm. J. Dent.
**2018**, 77, 106–111. [Google Scholar] [CrossRef] - Boiko, O.; Hyttinen, J.; Fält, P.; Jäsberg, H.; Mirhashemi, A.; Kullaa, A.; Hauta-Kasari, M.; Hyttinen, J.; Fält, P.; Jäsberg, H.; et al. Deep Learning for Dental Hyperspectral Image Analysis. Color Imaging Conf.
**2019**, 27, 295–299. [Google Scholar] [CrossRef] - Vimalarani, G.; Ramachandraiah, U. Automatic Diagnosis and Detection of Dental Caries in Bitewing Radiographs Using Pervasive Deep Gradient Based LeNet Classifier Model. Microprocess. Microsyst.
**2022**, 94, 104654. [Google Scholar] [CrossRef] - Mishra, P.; Sadeh, R.; Bino, E.; Polder, G.; Boer, M.P.; Rutledge, D.N.; Herrmann, I. Complementary Chemometrics and Deep Learning for Semantic Segmentation of Tall and Wide Visible and Near-Infrared Spectral Images of Plants. Comput. Electron. Agric.
**2021**, 186, 106226. [Google Scholar] [CrossRef] - Lesnoff, M.; Metz, M.; Roger, J.M. Comparison of Locally Weighted PLS Strategies for Regression and Discrimination on Agronomic NIR Data. J. Chemom.
**2020**, 34, e3209. [Google Scholar] [CrossRef] - Liland, K.H.; Næs, T.; Indahl, U.G. ROSA—A Fast Extension of Partial Least Squares Regression for Multiblock Data Analysis. J. Chemom.
**2016**, 30, 651–662. [Google Scholar] [CrossRef] - Mishra, P.; Xu, J.; Liland, K.H.; Tran, T. META-PLS Modelling: An Integrated Approach to Automatic Model Optimization for near-Infrared Spectra. Anal. Chim. Acta
**2022**, 1221, 340142. [Google Scholar] [CrossRef] [PubMed] - Zhao, W.; Du, S. Spectral-Spatial Feature Extraction for Hyperspectral Image Classification: A Dimension Reduction and Deep Learning Approach. IEEE Trans. Geosci. Remote Sens.
**2016**, 54, 4544–4554. [Google Scholar] [CrossRef] - Xu, Y.; Zhang, L.; Du, B.; Zhang, F. Spectral-Spatial Unified Networks for Hyperspectral Image Classification. IEEE Trans. Geosci. Remote Sens.
**2018**, 56, 5893–5909. [Google Scholar] [CrossRef] - Zhong, Z.; Li, J.; Luo, Z.; Chapman, M. Spectral-Spatial Residual Network for Hyperspectral Image Classification: A 3-D Deep Learning Framework. IEEE Trans. Geosci. Remote Sens.
**2018**, 56, 847–858. [Google Scholar] [CrossRef] - Shah, S.H.; Angel, Y.; Houborg, R.; Ali, S.; McCabe, M.F. Spectral-Spatial Attention Networks for Hyperspectral Image Classification. Remote Sens.
**2019**, 11, 963. [Google Scholar] [CrossRef] [Green Version] - Zhu, M.; Jiao, L.; Liu, F.; Yang, S.; Wang, J. Residual Spectral-Spatial Attention Network for Hyperspectral Image Classification. IEEE Trans. Geosci. Remote Sens.
**2021**, 59, 449–462. [Google Scholar] [CrossRef] - Hyttinen, J.; Fält, P.; Jäsberg, H.; Kullaa, A.; Hauta-Kasari, M. Oral and Dental Spectral Image Database—ODSI-DB. Appl. Sci.
**2020**, 10, 7246. [Google Scholar] [CrossRef] - Leys, C.; Klein, O.; Dominicy, Y.; Ley, C. Detecting Multivariate Outliers: Use a Robust Variant of the Mahalanobis Distance. J. Exp. Soc. Psychol.
**2018**, 74, 150–156. [Google Scholar] [CrossRef] - Pal, A.; Maiti, J. Development of a Hybrid Methodology for Dimensionality Reduction in Mahalanobis–Taguchi System Using Mahalanobis Distance and Binary Particle Swarm Optimization. Expert Syst. Appl.
**2010**, 37, 1286–1293. [Google Scholar] [CrossRef]

**Figure 1.**The observed features: (

**a**) The image was measured under the accumulated (mixed) spectral power distribution (SPD) of sub-SPDs with the intensities of $\mathit{x}=\left[{x}_{1},\dots ,{x}_{{N}_{\mathrm{L}}}\right]$. (

**b**) Images were measured for each sub-SPD, then synthesized into one image by multiplication with the linear weights, $\mathit{x}$. Equivalence was achieved between the case observed under the accumulated spectral power distribution (SPD) and a linear synthesized variant of the observed features under each sub-SPD. Considering that the observed features that were measured under the accumulated SPD ($=\left({\mathit{C}}^{\mathrm{T}}\mathit{R}\mathit{Q}\right)\mathit{x}$ ) and the synthesized features, after measuring under the sub-SPDs ($={\mathit{C}}^{\mathrm{T}}\mathit{R}\left(\mathit{Q}\mathit{x}\right)$ ), are the same, from Equation (3), the images in (

**a**,

**b**) are the same if lighting weights, $x,$ are the same as each other and measuring the images by camera has linearity.

**Figure 2.**The concept of optimization of the light-source spectra using a neural network (NN). The input to the model is a set of images of objects taken with the same camera under sub-spectral power distributions (SPDs) that have different spectral proportions from each other, where each sample is composed of ${N}_{\mathrm{L}}$ number of sub-light images of ${N}_{\mathrm{ch}}$-channels. The input image is linearly combined according to the lighting weights x to obtain a single image with ${N}_{\mathrm{ch}}$-channels (rendering process), fed into the NN model. During training, the NN model’s weights $\mathit{W}$ and $\mathit{x}$ are updated simultaneously according to the gradients, which are computed by backpropagation from the cost (error) of the NN model’s output. After sufficient training, the accumulated SPD obtained by mixing the sub-SPDs with intensities of $\mathit{x}$ becomes the optimal light source for the machine learning problem. The image measured under the optimal light source is equivalent to the image obtained by the rendering process. This can be entered into the model for inference with fewer shots of images than when the dataset was obtained.

**Figure 3.**A classification model featuring the optimization of light-source spectra: (

**a**) the model with only a fully connected layer (FCL), (

**b**) the model featuring an FCL and a convolutional neural network (CNN). In the rendering layer, input images are linearly combined with lighting weights $\mathit{x}$

**.**Rendered images are fed into the fully connected layer (FC) in (

**a**). The referenced class of the model is calculated as a one-hot coding with the softmax of FC’s output. In the case of (

**b**), the rendered image is fed into a few 2D-convolution layers (Conv2d) and then into an FC. Eventually, the class is referenced via softmax, the same as (

**a**).

**Figure 4.**A sample of the target class for $5\times 5$ patch images, adapted from the image of ODSI-DB [40] (CC BY-NC-SA 4.0).

**Figure 6.**Sets of SPDs with real light-emitting diodes (LEDs): (

**a**) measured SPD of 24 LEDs for sub-lights, and (

**b**) approximation of an SPD of D65 standard illumination with SPDs represented in (

**a**).

**Figure 7.**(

**a**) The simulated sub-light source: each spectral distribution was computed by assuming a Gaussian distribution with a full-width at half maximum (FWHM) wavelength, ${\lambda}_{\mathrm{FWHM}}=40\mathrm{nm}$, and its means are 412, 437, 462, 487, 512, 537, 562, 587, 612, 637, 662, 687, 712, 737, 762, 787, 812, 837, 862, 887, 912, 937, 962, and 987 nm. (

**b**) The approximation of the D65 standard illumination spectral distribution with band-adjusted sub-light sources is represented in (

**a**).

**Figure 8.**The quantum efficiency of complementary metal oxide semiconductor-based red–green–blue camera: AR1335.

**Figure 10.**The performance of each method in the five-fold cross-validation: alternating optimization (linear-SVM), the proposed method with only FCL (NN), and the proposed method with CNN (CNN): (

**a**) F1-score and (

**b**) accuracy.

**Figure 11.**The comparison of performance between the reference (D65) vs. the optimal light in the five-fold cross-validation: alternating optimization (linear SVM), proposed method with only FCL (NN), and proposed method with CNN (CNN): (

**a**) F1-score and (

**b**) accuracy.

**Figure 12.**The performance of each method of the band-aligned SPD utilizing the NIR in the 400–1000 nm band with the proposed method with CNN: (

**a**) F1-score for each method, and (

**b**) comparison of performance between the reference (D65) and the optimal light. In the three models utilizing the 400–1000 nm band, the lighting weights were initialized in different ways: randomized in [0,1], approximating D65, and as uniform vector 1, respectively.

**Figure 13.**The optimal SPDs of the CNN-based optimization in a five-fold cross-validation with the measured LEDs having spectral powers in the 400–830 nm band for each classification of one-vs-rest classification: (

**a**) attrition and erosion vs. others, (

**b**) calculus vs. others, (

**c**) enamel vs. others, (

**d**) initial caries vs. others, (

**e**) microfracture vs. others, and (

**f**) root vs. others.

**Figure 14.**The Mahalanobis distance of dataset samples from the distribution of target class in feature space for each one-vs-rest classification. The features have 75 dimensions consisting of three channels of RGB at $5\times 5$ patch images rendered under the D65 or under the optimal light with the CNN-based method.

Methods | Previous | Proposals | |
---|---|---|---|

Alternating Optimization (Linear Support Vector Machine) | Neural Network (NN) | Convolutional Neural Network (CNN) | |

Input | $1\times 1$ pixel | $5\times 5$ image | |

Task | One-vs-rest, 2-class Classification | ||

Target | Class of pix | Class of the central pixel | |

Light source | - 1
- 24 light-emitting diodes (optimization),
- 2
- D65 (Fixed, as reference)
| ||

Cross Validation | five-Fold | ||

Initial lighting weights | Randomized vector in $\left[-1,\text{}1\right]$ |

Parameters | NN-Based | CNN-Based |
---|---|---|

# of layers | $\left\{3,5,7\right\}$ | |

# of units | $\left\{10,15,20\right\}$ | |

Activation | {Rectified linear unit (ReLU), None} | |

Conv2d | - | Size = 3 × 3, stride = 1, $10\times 3$-channels |

Output layer | Softmax | |

Dropout | $p=0.3$ |

Sub-Light | Peek Wavelength [nm] | |
---|---|---|

Measured | Simulated | |

1 | 405 | 412 |

2 | 420 | 437 |

3 | 435 | 462 |

4 | 450 | 487 |

5 | 470 | 512 |

6 | 490 | 537 |

7 | 505 | 562 |

8 | 525 | 587 |

9 | 535 | 612 |

10 | 555 | 637 |

11 | 565 | 662 |

12 | 570 | 687 |

13 | 590 | 712 |

14 | 600 | 737 |

15 | 610 | 762 |

16 | 625 | 787 |

17 | 630 | 812 |

18 | 645 | 837 |

19 | 660 | 862 |

20 | 670 | 887 |

21 | 680 | 912 |

22 | 690 | 937 |

23 | 700 | 962 |

24 | 780 | 987 |

Class | Previous | Proposal | |
---|---|---|---|

Alternating Optimization, $1\times 1$ | NN, $1\times 1$ | $\mathbf{CNN},\phantom{\rule{0ex}{0ex}}5\times 5$ | |

Enamel | 11,836 | 11,836 | 11,836 |

Attrition and Erosion | 2500 | 2500 | 2500 |

Calculus | 1608 | 1608 | 1608 |

Initial Caries | 792 | 792 | 792 |

Microfracture | 900 | 900 | 900 |

Root | 897 | 897 | 897 |

Class | Sample Size | |
---|---|---|

One | Rest | |

Enamel | 11,836 | 6697 |

Attrition and Erosion | 2500 | 16,033 |

Calculus | 1608 | 16,925 |

Initial Caries | 792 | 17,741 |

Microfracture | 900 | 17,633 |

Root | 897 | 17,636 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Ito, K.; Higashi, H.; Hietanen, A.; Fält, P.; Hine, K.; Hauta-Kasari, M.; Nakauchi, S.
The Optimization of the Light-Source Spectrum Utilizing Neural Networks for Detecting Oral Lesions. *J. Imaging* **2023**, *9*, 7.
https://doi.org/10.3390/jimaging9010007

**AMA Style**

Ito K, Higashi H, Hietanen A, Fält P, Hine K, Hauta-Kasari M, Nakauchi S.
The Optimization of the Light-Source Spectrum Utilizing Neural Networks for Detecting Oral Lesions. *Journal of Imaging*. 2023; 9(1):7.
https://doi.org/10.3390/jimaging9010007

**Chicago/Turabian Style**

Ito, Kenichi, Hiroshi Higashi, Ari Hietanen, Pauli Fält, Kyoko Hine, Markku Hauta-Kasari, and Shigeki Nakauchi.
2023. "The Optimization of the Light-Source Spectrum Utilizing Neural Networks for Detecting Oral Lesions" *Journal of Imaging* 9, no. 1: 7.
https://doi.org/10.3390/jimaging9010007