Physically Plausible Spectral Reconstruction†
School of Computing Sciences, University of East Anglia, Norwich NR4 7TJ, UK
*
Author to whom correspondence should be addressed.
†
This paper is an extension version of the conference paper: Lin, Y.T.; Finlayson, G.D. Physically Plausible Spectral Reconstruction from RGB Images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA, 14–19 June 2020.
Sensors 2020, 20(21), 6399; https://doi.org/10.3390/s20216399
Received: 1 October 2020 / Revised: 30 October 2020 / Accepted: 4 November 2020 / Published: 9 November 2020
(This article belongs to the Special Issue Color & Spectral Sensors)
Spectral reconstruction algorithms recover spectra from RGB sensor responses. Recent methods—with the very best algorithms using deep learning—can already solve this problem with good spectral accuracy. However, the recovered spectra are physically incorrect in that they do not induce the RGBs from which they are recovered. Moreover, if the exposure of the RGB image changes then the recovery performance often degrades significantly—i.e., most contemporary methods only work for a fixed exposure. In this paper, we develop a physically accurate recovery method: the spectra we recover provably induce the same RGBs. Key to our approach is the idea that the set of spectra that integrate to the same RGB can be expressed as the sum of a unique fundamental metamer (spanned by the camera’s spectral sensitivities and linearly related to the RGB) and a linear combination of a vector space of metameric blacks (orthogonal to the spectral sensitivities). Physically plausible spectral recovery resorts to finding a spectrum that adheres to the fundamental metamer plus metameric black decomposition. To further ensure spectral recovery that is robust to changes in exposure, we incorporate exposure changes in the training stage of the developed method. In experiments we evaluate how well the methods recover spectra and predict the actual RGBs and RGBs under different viewing conditions (changing illuminations and/or cameras). The results show that our method generally improves the state-of-the-art spectral recovery (with more stabilized performance when exposure varies) and provides zero colorimetric error. Moreover, our method significantly improves the color fidelity under different viewing conditions, with up to a 60% reduction in some cases.
View Full-Text
▼
Show Figures
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited
-
Externally hosted supplementary file 1
Link: https://github.com/EthanLinYitun/Physically_Plausible_Spectral_Reconstruction
Description: The code of the methods introduced in this paper is available at the provided link.
MDPI and ACS Style
Lin, Y.-T.; Finlayson, G.D. Physically Plausible Spectral Reconstruction. Sensors 2020, 20, 6399. https://doi.org/10.3390/s20216399
AMA Style
Lin Y-T, Finlayson GD. Physically Plausible Spectral Reconstruction. Sensors. 2020; 20(21):6399. https://doi.org/10.3390/s20216399
Chicago/Turabian StyleLin, Yi-Tun; Finlayson, Graham D. 2020. "Physically Plausible Spectral Reconstruction" Sensors 20, no. 21: 6399. https://doi.org/10.3390/s20216399
Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.
Search more from Scilit