# Spectral Reconstruction Using an Iteratively Reweighted Regulated Model from Two Illumination Camera Responses

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Multispectral Imaging Model

#### 2.1. Regularization Model

#### 2.2. Iteratively Reweighted Regularization Model

^{0}are calculated in Equation (8).

^{(t−1)}and associated weights w

_{i}

^{(t−1)}are calculated from the previous iteration.

#### 2.3. Feature Selection

_{i}is the polynomial expansion of the camera signals from the ith patch, and r

_{i}is the corresponding spectral reflectance. A randomized regression model can be built as follows:

_{j}|S), that point c

_{j}is picked from S as the reference for c is

_{w}is the distance function, and ν is the kernel function that assumes large values when d

_{w}is small. Now consider the leave-one-out application of this randomized regression model, that is, predicting the response for ${c}_{i}$ using the data in ${S}^{-i}$, and the training set S excluding the point $\left({c}_{i},{r}_{i}\right)$. The probability that point c

_{j}is picked as the reference point for ${c}_{i}$ is given by:

_{i}be the actual spectral response for c

_{i}. Let l be a loss function that measures the disagreement between r

_{i}and ${\tilde{r}}_{i}$. Then, the average value of the loss function l(r

_{i,}${\tilde{r}}_{i}$) is given by:

_{f}can be expressed as the following minimization regression error:

_{r}is weight vector for rth feature item, n is the number of observations, and p is the number of predictor variables.

## 3. Experiment and Result

#### 3.1. Camera Setup

#### 3.2. The Influence of Feature Selection

#### 3.3. The Influence of the Regression Model on the Proposed Method

#### 3.4. Methods Implementation and Comparison for Illuminant Metamerism

#### 3.5. Discussion

## 4. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Pan, Z.; Healey, G.; Prasad, M.; Tromberg, B.J. Face recognition in hyperspectral images. IEEE Trans. Pattern Anal. Mach. Intell.
**2003**, 25, 1552–1560. [Google Scholar] - Uzair, M.; Mahmood, A.; Shafait, F.; Nansen, C.; Mian, A. Is spectral reflectance of the face a reliable biometric? Opt. Express
**2015**, 23, 15160–15173. [Google Scholar] [CrossRef] [Green Version] - Berns, R.S. Digital color reconstructions of cultural heritage using color-managed imaging and small-aperture spectrophotometry. Color Res. Appl.
**2019**, 44, 531–546. [Google Scholar] [CrossRef] - Kikuchi, K.; Masuda, Y.; Hirao, T. Imaging of hemoglobin oxygen saturation ratio in the face by spectral camera and its application to evaluate dark circles. Ski. Res. Technol.
**2013**, 19, 499–507. [Google Scholar] [CrossRef] - Nishidate, I.; Maeda, T.; Niizeki, K.; Aizu, Y. Estimation of Melanin and Hemoglobin Using Spectral Reflectance Images Reconstructed from a Digital RGB Image by the Wiener Estimation Method. Sensors
**2013**, 13, 7902–7915. [Google Scholar] [CrossRef] [Green Version] - Sun, G.; Wang, X.; Sun, Y. Measurement Method Based on Multispectral Three-Dimensional Imaging for the Chlorophyll Contents of Greenhouse Tomato Plants. Sensors
**2019**, 19, 3345. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Hardeberg, J.Y. Acquisition and Reproduction of Color Images: Colorimetric and Multispectral Approach. Ph.D. Thesis, Ecole Nationale Superieure des Telecommunications, Paris, France, 1999. [Google Scholar]
- Wang, X.; Liao, N.; Li, Y. The Research on Reconstruction of Spectral Reflectance in LCTF Imaging System Based on Comparative Measurement. Spectrosc. Spect. Anal.
**2018**, 38, 290–295. [Google Scholar] - Shrestha, R.; Hardeberg, J. Evaluation and comparison of multispectral imaging systems. In Proceedings of the IS&T Color and Imaging Conference, Boston, MA, USA, 3–7 November 2014; pp. 107–115. [Google Scholar]
- Hardeberg, J.Y.; Schmitt, F.; Brettel, H. Multispectral color image capture using a liquid crystal tunable filter. Opt. Eng.
**2002**, 41, 2532–2548. [Google Scholar] - Mathews, S.A. Design and fabrication of a low-cost, multispectral imaging system. Appl. Opt.
**2008**, 47, 71–76. [Google Scholar] [CrossRef] [Green Version] - Tanksale, T.M.; Urban, P. Trichromatic Reflectance Capture Using a Tunable Light Source: Setup, Characterization and Reflectance Estimation. Electron. Imaging
**2016**, 27, 1–7. [Google Scholar] [CrossRef] [Green Version] - Jiang, J.; Gu, J. Recovering spectral reflectance under commonly available lighting conditions. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA, 16–21 June 2012. [Google Scholar]
- Shen, H.L.; Cai, P.Q.; Shao, S.J.; Xin, J.H. Reflectance reconstruction for multispectral imaging by adaptive Wiener estimation. Opt. Express
**2007**, 15, 15545–15554. [Google Scholar] [CrossRef] [Green Version] - Han, S.; Sato, I.; Okabe, T.; Sato, Y. Fast spectral reflectance recovery using DLP projector. Int. J. Comput. Vis.
**2014**, 110, 172–184. [Google Scholar] [CrossRef] - Jiang, J. Evaluation and Improvement of Workflow of Digital Imaging of Fine Art Reproduction in Museums. Ph.D. Thesis, Rochester Institute of Technology, Rochester, NY, USA, 2013. [Google Scholar]
- Tominaga, S.; Horiuchi, T. Spectral imaging by synchronizing capture and illumination. J. Opt. Soc. Am. A
**2012**, 29, 1764–1775. [Google Scholar] [CrossRef] - Liu, Z.; Liu, Q.; Gao, G.; Li, C. Optimized spectral reconstruction based on adaptive training set selection. Opt. Express
**2017**, 25, 12435–12445. [Google Scholar] [CrossRef] [PubMed] - Zhang, X.; Wang, Q.; Li, J.; Zhou, X.; Yang, I. Estimating spectral reflectance from camera responses based on CIE XYZ tristimulus values under multi-illuminants. Color Res. Appl.
**2017**, 42, 68–77. [Google Scholar] [CrossRef] - Zhao, Y.; Berns, R.S. Image-based spectral reflectance reconstruction using the matrix R method. Color Res. Appl.
**2017**, 32, 343–351. [Google Scholar] [CrossRef] - Imai, F.H.; Berns, R.S. High-Resolution Multi-Spectral Image Archives: A Hybrid Approach. In Proceedings of the Sixth Color Imaging Conference: Color Science, Systems, and Applications, Scottsdale, AZ, USA, 9–12 November 1998. [Google Scholar]
- Urban, P.; Rosen, M.R.; Berns, R.S. Spectral image reconstruction using an edge preserving spatio-spectral Wiener estimation. J. Opt. Soc. Am. A
**2009**, 26, 1865–1875. [Google Scholar] [CrossRef] - Liu, Z.; Wan, X.X.; Huang, X.G.; Liu, Q. The study on spectral reflectance reconstruction based on wideband multi-spectral acquisition system. Spectrosc. Spectr. Anal.
**2013**, 33, 1076–1081. [Google Scholar] - Yoo, J.H.; Kim, D.C.; Ha, H.G.; Ha, Y.H. Adaptive Spectral Reflectance Reconstruction Method based on Wiener Estimation Using a Similar Training Set. J. Imaging Sci. Technol.
**2016**, 60, 205031–205038. [Google Scholar] [CrossRef] - Yamaguchi, M.; Ohyama, N.; Murakami, Y. Piecewise Wiener estimation for reconstruction of spectral reflectance image by multipoint spectral measurements. Appl. Opt.
**2009**, 48, 2188–2202. [Google Scholar] - Wang, K.; Wang, H.; Wang, Z.; Yin, Y.; Zhang, Y. Medical imaging based spectral reflectance reconstruction combining PCA and regularized polynomial. Biomed. Res.
**2018**, 29, S380–S383. [Google Scholar] [CrossRef] [Green Version] - Zhang, X.; Xu, H. Reconstructing spectral reflectance by dividing spectral space and extending the principal components in principal component analysis. J. Opt. Soc. Am. A
**2008**, 25, 371–378. [Google Scholar] [CrossRef] - Harifi, T.; Amirshahi, S.H.; Agahian, F. Recovery of Reflectance Spectra from Colorimetric Data Using Principal Component Analysis Embedded Regression Technique. Opt. Rev.
**2008**, 15, 302–308. [Google Scholar] [CrossRef] - Xiao, K.; Zhu, Y.; Li, C.; Connah, D.; Yates, J.M.; Wuerger, S. Improved method for skin reflectance reconstruction from camera images. Opt. Express
**2016**, 24, 14934–14950. [Google Scholar] [CrossRef] [Green Version] - Shrestha, R.; Mansouri, A.; Hardeberg, J.Y. Multispectral imaging using a stereo camera: Concept, design and assessment. EURASIP J. Adv. Signal Process.
**2011**, 26, 57–72. [Google Scholar] [CrossRef] - Li, H.; Wu, Z.; Zhang, L.; Parkkinen, J. SR-LLA: A novel spectral reconstruction method based on locally linear approximation. In Proceedings of the IEEE International Conference on Image Processing, Victoria, Australia, 27–30 October 2014; pp. 2029–2033. [Google Scholar]
- Cao, B.; Liao, N.; Cheng, H. Spectral reflectance reconstruction from RGB images based on weighting smaller color difference group. Color Res. Appl.
**2017**, 42, 327–332. [Google Scholar] [CrossRef] - Shen, H.L.; Wan, H.J.; Zhang, Z.C. Estimating reflectance from multispectral camera responses based on partial least-squares regression. J. Electron. Imaging
**2010**, 19, 020501. [Google Scholar] [CrossRef] - Heikkinen, V.; Jetsu, T.; Parkkinen, J.; Hauta-Kasari, M.; Jaaskelainen, T. Regularized learning framework in the estimation of reflectance spectra from camera responses. J. Opt. Soc. Am. A
**2007**, 24, 2673–2683. [Google Scholar] [CrossRef] - Amiri, M.M.; Fairchild, M.D. A strategy toward spectral and colorimetric color reproduction using ordinary digital cameras. Color Res. Appl.
**2018**, 43, 675–684. [Google Scholar] [CrossRef] - Zhang, Z.C. Reconstruction of Spectral Reflectance in Imaging Systems. Master’s Thesis, Zhejiang University, Hangzhou, China, 2010. [Google Scholar]
- Liu, Z.; Xiao, K.; Pointer, M.; Li, C. Developing a multi-spectral system using a RGB camera under two illumination. In Proceeding of the IS&T Color and Imaging Conference, Chiba, Japan, 27–29 March 2020; pp. 277–281. [Google Scholar]
- Guyon, I.; Elisseeff, A. An introduction to variable and feature selection. J. Mach. Learn. Res.
**2003**, 3, 1157–1182. [Google Scholar] - Menke, W. Geophysical Data Analysis: Discrete Inverse Theory (MATLAB Edition), 3rd ed.; Academic Press: London, UK, 2012. [Google Scholar]
- Luxburg, U.V. A tutorial on spectral clustering. Stat. Comput.
**2004**, 17, 395–416. [Google Scholar] [CrossRef] - Yang, W.; Wang, K.; Zuo, W. Neighborhood component feature selection for high-dimensional data. J. Comput.
**2012**, 7, 161–168. [Google Scholar] [CrossRef] - Khan, H.A.; Thomas, J.B.; Hardeberg, J.Y.; Laligant, O. Illuminant estimation in multispectral imaging. J. Opt. Soc. Am. A
**2017**, 34, 1085–1098. [Google Scholar] [CrossRef] [PubMed] [Green Version]

**Figure 3.**The color distribution of matte charts (circle marker) and semigloss charts (square maker). (

**a**) Comparison of color distribution in the CIELAB color space. (

**b**) The chromaticity coordinates of samples in a* − b* plane.

**Figure 5.**Feature selection among 84 items. (

**a**) Feature weight of 84 extended feature items. (

**b**) Hierarchical treemap view of feature weights.

**Figure 7.**The relevant summary statistics of the proposed method and the existing methods, the outliers are plotted individually using the ‘+’ symbol. (

**a**) Boxplot distributions of the RMS. (

**b**) Boxplot distributions of the CIEDE2000 color difference.

**Figure 8.**Representative samples of reconstructed spectra of the proposed method and the traditional methods.

Order | Polynomial Regression |
---|---|

1st-order (6) | ${R}_{1},{G}_{1},{B}_{1},{R}_{2},{G}_{2},{B}_{2}$ |

2nd-order (18) | $\begin{array}{l}{R}_{1}^{2},{G}_{1}^{2},{R}_{2}^{2},{G}_{2}^{2},{B}_{2}^{2},{R}_{1}{B}_{2},{R}_{1}{G}_{2},{R}_{1}{R}_{2},{R}_{1}{B}_{1},{G}_{1}{B}_{2},\\ {G}_{1}{G}_{2},{G}_{1}{R}_{2},{B}_{1}{R}_{2},{B}_{1}{B}_{2},{B}_{1}{G}_{2},{R}_{2}{G}_{2},{B}_{2}{G}_{2},{B}_{2}{R}_{2}\end{array}$ |

3rd-order (6) | ${B}_{2}^{3},{B}_{2}^{2}{R}_{2},{B}_{2}^{2}{R}_{1},{B}_{2}{R}_{2}^{2},{B}_{2}{R}_{1}{R}_{2},{B}_{2}{R}_{1}^{2}$ |

**Table 2.**Performance of reflectance estimation using two illuminants with different polynomial expansions.

RMS | CIE DE00 | |||||||||
---|---|---|---|---|---|---|---|---|---|---|

Mean | Min | Max | SD | T-Stat | Mean | Min | Max | SD | T-Stat | |

1st-order (7) | 2.85% | 0.48% | 39.2% | 0.03 | 21.6 | 2.09 | 0.08 | 32.1 | 2.0 | 20.0 |

2nd-order (28) | 2.24% | 0.25% | 69.3% | 0.04 | 11.7 | 2.11 | 0.17 | 141.9 | 7.3 | 5.6 |

3rd-order (84) | 2.18% | 0.23% | 107.5% | 0.06 | 7.4 | 1.97 | 0.12 | 131.1 | 6.9 | 5.6 |

Selected (30) | 2.14% | 0.18% | 34.4% | 0.02 | 19.3 | 1.79 | 0.11 | 20.2 | 1.5 | 22.9 |

Model | 84 Items (3rd-Order Polynomial Expansion) | 30 Items (Feature-Selected) | ||||||
---|---|---|---|---|---|---|---|---|

Mean (%) | Max (%) | Min (%) | SD (%) | Mean (%) | Max (%) | Min (%) | SD (%) | |

RLS [34] | 2.84 | 18.37 | 0.37 | 3.39 | 2.88 | 28.06 | 0.38 | 4.88 |

Tik [36] | 2.91 | 30.70 | 0.37 | 5.26 | 2.81 | 25.66 | 0.38 | 4.49 |

PCA [7] | 3.77 | 11.45 | 0.84 | 2.48 | 3.78 | 12.09 | 0.96 | 2.53 |

Wiener [5] | 4.88 | 20.13 | 1.16 | 3.89 | 3.54 | 20.96 | 0.85 | 3.53 |

PLS [33] | 3.81 | 11.25 | 0.79 | 2.49 | 3.68 | 10.91 | 0.76 | 2.39 |

OLS [10] | 2.99 | 24.63 | 0.35 | 4.27 | 2.30 | 17.39 | 0.36 | 2.99 |

IRWR | 2.34 | 16.24 | 0.36 | 2.89 | 2.14 | 9.35 | 0.35 | 1.77 |

Model | 84 Items (3rd-Order Polynomial Expansion) | 30 Items (Feature-Selected) | ||||||
---|---|---|---|---|---|---|---|---|

Mean (%) | Max (%) | Min (%) | SD (%) | Mean (%) | Max (%) | Min (%) | SD (%) | |

RLS [34] | 2.35 | 15.32 | 0.27 | 2.70 | 2.30 | 19.06 | 0.39 | 3.30 |

Tik [36] | 2.31 | 21.73 | 0.34 | 3.69 | 2.32 | 19.99 | 0.38 | 3.42 |

PCA [7] | 2.16 | 6.16 | 0.41 | 1.23 | 2.17 | 6.05 | 0.39 | 1.22 |

Wiener [5] | 4.85 | 20.91 | 0.69 | 4.64 | 2.95 | 19.85 | 0.48 | 3.61 |

PLS [33] | 2.28 | 6.65 | 0.41 | 1.36 | 2.26 | 6.54 | 0.41 | 1.34 |

OLS [10] | 2.65 | 24.73 | 0.27 | 4.19 | 2.07 | 20.26 | 0.26 | 3.44 |

IRWR | 1.98 | 19.5 | 0.23 | 3.37 | 1.79 | 7.31 | 0.33 | 1.39 |

Illumination Temperature (Items) | 3500 K (10) | 6500 K (10) | 3500 K + 6500 K (30) | 3500 K + 6500 K (84) | |
---|---|---|---|---|---|

A | Mean | 2.65 | 2.85 | 2.39 | 2.18 |

Max | 7.09 | 7.67 | 8.12 | 5.35 | |

Min | 0.27 | 0.22 | 0.38 | 0.30 | |

SD | 1.53 | 1.70 | 1.56 | 1.10 | |

t-stat | 10.58 | 10.23 | 9.30 | 12.01 | |

F2 | Mean | 2.94 | 2.58 | 2.47 | 2.31 |

Max | 9.72 | 7.12 | 8.55 | 5.35 | |

Min | 0.32 | 0.26 | 0.29 | 0.18 | |

SD | 2.08 | 1.66 | 1.71 | 1.25 | |

t-stat | 8.60 | 9.42 | 8.80 | 11.22 | |

TL84 | Mean | 2.65 | 2.49 | 2.35 | 2.15 |

Max | 7.64 | 6.03 | 8.56 | 5.31 | |

Min | 0.27 | 0.21 | 0.31 | 0.19 | |

SD | 1.65 | 1.49 | 1.62 | 1.16 | |

t-stat | 9.77 | 10.12 | 8.79 | 11.24 | |

D50 | Mean | 2.92 | 2.81 | 2.62 | 2.40 |

Max | 7.93 | 7.04 | 9.69 | 5.72 | |

Min | 0.25 | 0.24 | 0.56 | 0.32 | |

SD | 1.85 | 1.66 | 1.80 | 1.23 | |

t-stat | 9.60 | 10.26 | 8.85 | 11.83 | |

D65 | Mean | 2.74 | 2.40 | 2.34 | 2.14 |

Max | 7.87 | 5.77 | 8.77 | 5.30 | |

Min | 0.26 | 0.21 | 0.28 | 0.19 | |

SD | 1.77 | 1.46 | 1.65 | 1.19 | |

t-stat | 9.42 | 9.97 | 8.62 | 10.99 | |

Mean (above illumination) | Mean | 2.78 | 2.62 | 2.43 | 2.23 |

Max | 8.05 | 6.73 | 8.74 | 5.41 | |

Min | 0.28 | 0.23 | 0.36 | 0.24 | |

SD | 1.77 | 1.60 | 1.67 | 1.19 | |

t-stat | 9.59 | 10.00 | 8.87 | 11.46 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Liu, Z.; Xiao, K.; Pointer, M.R.; Liu, Q.; Li, C.; He, R.; Xie, X.
Spectral Reconstruction Using an Iteratively Reweighted Regulated Model from Two Illumination Camera Responses. *Sensors* **2021**, *21*, 7911.
https://doi.org/10.3390/s21237911

**AMA Style**

Liu Z, Xiao K, Pointer MR, Liu Q, Li C, He R, Xie X.
Spectral Reconstruction Using an Iteratively Reweighted Regulated Model from Two Illumination Camera Responses. *Sensors*. 2021; 21(23):7911.
https://doi.org/10.3390/s21237911

**Chicago/Turabian Style**

Liu, Zhen, Kaida Xiao, Michael R. Pointer, Qiang Liu, Changjun Li, Ruili He, and Xuejun Xie.
2021. "Spectral Reconstruction Using an Iteratively Reweighted Regulated Model from Two Illumination Camera Responses" *Sensors* 21, no. 23: 7911.
https://doi.org/10.3390/s21237911