Toward Practical Spectral Imaging beyond a Laboratory Context

Abstract

A portable, user-friendly multispectral imaging system assembled almost entirely of common photography equipment and open-source software has been developed. The system serves as an outreach and educational tool for demonstrating and promoting scientific imaging as a more routine practice in the contexts of cultural heritage digitization and photography. These efforts are aimed primarily at institutions where advanced imaging technologies are not already found, and where funding and expertise may limit access to commercial, bespoke multispectral imaging solutions that are currently available. The background and theory that were shared in tutorials given during the system’s initial testing campaign are detailed here. Testing was carried out in one-day on-site visits to six cooperating institutions of different sizes and collection types in the northeast USA. During these visits, the imaging system was presented, and the benefit of collecting spectral data using low barrier-to-entry capture and processing methods relative to conventional imaging methods was discussed. Imaging was conducted on site on selected collections objects to showcase the current capabilities of the system and to inform ongoing improvements to the setup and processing. This paper is a written companion piece to the visits, as a source of further detail and context for the two-light imaging system that was described and demonstrated.

1. Introduction

1.1. Spectral Imaging in a Laboratory Context

Since its adoption by the heritage science community several decades ago [1], spectral imaging has matured into an analytical technique regularly utilized in noninvasive scientific studies of cultural heritage objects. Spectral imaging is differentiated from conventional color imaging by the spectral resolution of the imaging system, where spectral imaging techniques are those that collect more bands—as few as five or six, to hundreds—than typical three-channel RGB imaging. Spectral imaging techniques may be further described as multispectral or hyperspectral according to their relative spectral resolution (tens versus hundreds of image bands), which is largely determined by the image band selection method (e.g., filter- or illumination-based band selection versus diffraction grating wavelength selection). The use of specific terms is fluid and often context-dependent [2]. Taken together, spectral imaging techniques have been proven to be highly effective means of performing noninvasive, spatially resolved reflectance spectroscopy in the UV-VIS-IR range, which is particularly useful for identifying and mapping the distribution of artists’ materials [3,4]. This information is often used to build a better understanding of artists’ working methods, material degradation, and prior restoration campaigns, which together provide an understanding of an object’s physical construction and history.

Despite their impressive abilities, due to the associated cost and complexity of these techniques, spectral imaging capabilities are yet largely siloed in institutions with the monetary and technical expertise necessary to support analytical imaging. Presently, mainly specialized instruments are used to carry out spectral imaging. Many systems that have been developed for lab-based research or in situ field studies are experimental [5,6,7]. They are typically applied in one-off technical studies, and are not intended or appropriate for adoption within routine imaging workflows. Additionally, such systems are not necessarily designed with user-friendly considerations, because they are built-to-purpose, and their design and operation is aligned with the lab-based context of their development. They more closely resemble analytical equipment rather than familiar cameras, and therefore sacrifice intuitive handling properties in favor of technical utility. Finally, these complex instruments are expensive to build, maintain, and operate, necessitating monetary support that smaller institutions simply cannot fund.

1.2. Toward Spectral Imaging in a Studio Workflow

Observing these barriers to its widespread adoption by more institutions, an interest developed in testing practical measures to put this technology into the hands of more users. The priority was physically demonstrating the feasibility of adopting studio-practical spectral imaging strategies during on-site visits to several institutions. The institutions were of diverse size and geographic location, and each had different experience with, approaches to, and goals for imaging collections. Dialogue with the smaller, more resource-strapped institutions was of particular interest. Their feedback is most critical to determine the best methods for getting this technology into the hands of more users moving forward.

More specifically, these visits were aimed at making both the concept and practice of multispectral imaging more approachable for non-experts. It was introduced as an advanced imaging technique, rather than a scientific analysis, and its advantages over conventional color capture for more routine photography and digitization projects were emphasized. These advantages, particularly for color-accurate reproduction [1,8,9], have been recognized for some time and have been summarized previously [10,11]. They include:

• the elimination of the need for subjective visual editing in post-production,
• the expansion of archives beyond a single set of viewing, illuminating, and observer conditions (CIE illuminant D50 and 1931 standard observer for ICC color managed archives),
• the ability to re-render an image under any desired lighting condition to inform curation, exhibition, scholarship, and conservation, and
• the prevention of undesirable metameric matches of materials used in conservation treatments.

These advantages were conveyed during the course of the visits, which loosely consisted of

• a tutorial about the theory and practice of conventional color imaging versus the proposed two-light spectral imaging method,
• a demonstration of two-light imaging, showing how it can be carried out using mainly cameras and equipment that are commonly found in photography studios and are already familiar to cultural heritage imaging professionals, and
• discussion and questions around these activities.

These efforts to bring awareness and access to spectral imaging are the culmination of recent research focused on developing more user friendly, affordable strategies for spectral capture and processing [12,13,14,15,16,17]. They are based on a history of developments for spectral imaging and color-accurate archiving of cultural heritage that have come out of the RIT Munsell Color Science Lab’s Studio for Scientific Imaging and Archiving of Cultural Heritage. The capture method of two-light imaging, to be introduced herein, is an LED-based spin on the original filter-based Dual-RGB imaging technique [18,19]. With respect to other spectral imaging techniques, dual-RGB and two-light imaging are notable for their efficiency and low complexity. Imaging with either technique is fast, because three spectral channels are collected per capture, and it is also accessible, because it can be performed with familiar, commercially available DSLR and mirrorless cameras that are already found in photography studios.

The means of processing the spectral data collected via two-light imaging had to be equally accessible and intuitive for the prospect of adopting this technique to be compelling. Toward this end, custom image processing software was developed alongside two-light imaging. The software, called Beyond RGB, is a cross-platform compatible, open source and freely available stand-alone software application that is fully operable through an intuitive graphical interface. It is designed to make processing the image sets as simple as possible. A large portion of the on-site visits was dedicated to demonstrating the capabilities of the first release of Beyond RGB and soliciting advice for improvements for future releases. Its high-level functionality and role in the overall workflow will be described below, while a more complete discussion can be found elsewhere [15].

1.3. Obstacles and Opportunities

The spectral image sets that are collected with two-light imaging are built up differently than those collected using more traditional multispectral imaging processes, which capture a single channel at a time sequentially over the wavelength range of interest. Conveying the differences between these two strategies and clearly describing the theory and practice of two-light imaging was a significant challenge. Furthermore, the most well-known application of spectral imaging is deriving reflectance properties of materials and mapping their distribution. In contrast, the emphasis in this work is on its color accuracy benefits. Color calibration based on multispectral information is both a less-common and less-intuitive use for spectral image data.

The details of two-light capture for highly color-accurate reproduction have been discussed elsewhere [16], and will be further described in the Background section below. Together with the Methods section that follows, these provide the information given in the on-site tutorials: the benefits of two-light imaging with respect to conventional color imaging, and an introduction to the setup, capture, and processing involved in the two-light imaging workflow. The Results and Discussion section summarizes the general experience of these real-world tests, impressions and feedback gathered from conversations with the host institutions, and actionable suggestions that will be implemented to improve the utility of the system moving forward.

2. Background

2.1. Dual-RGB Imaging

Given the time-consuming processes and expensive equipment necessary for analytical-level spectral imaging, an alternative, less complex approach is desirable in the context of studio photography. By the early 2000s, the Munsell Color Science Lab had been experimenting with spectral-based color reproduction for a few years. After a number of studies and design iterations to optimize filter choice and characteristics [20,21], the research culminated in the introduction of a spectral imaging method in which a set of two optimized filters are paired with a three-channel, RGB camera, enabling the capture of five spectral channels between two filtered RGB images [22]. This spectral imaging strategy came to be called dual-RGB imaging, and was later commercialized in the Sinar Color To Match (CTM) system [23].

At the time of its conception, the average color accuracy of dual-RGB of 0.9 $\Delta$E₀₀ for both calibration and verification data was superior to that of other contemporary, more complex approaches [22]. These included a system utilizing a 31-band liquid-crystal tunable filter and monochrome sensor (average $\Delta$E₀₀ 1.5) [24] and a 13-band system pairing interference filters with a monochrome sensor (average $\Delta$E₀₀ 1.5) [25]. This was an important result for the relevance of dual-RGB within studio photography, where obtaining color-accurate images objectively and efficiently, thus overcoming the need for subjective, time-consuming visual editing in post-processing, is an attractive selling point.

A high-level description of the capture and calibration strategies behind the dual-RGB technique is given below. A more detailed account of the history of its development and nuances in its implementation is available elsewhere [19], as well as image quality considerations related to color transformations [26].

Dual-RGB imaging is so named for the practice of capturing two images taken through a blue-green filter and a yellow filter placed in front of the lens of an RGB color filter array (CFA) camera. The spectral transmittance of such a pair of colored glass filters are plotted in Figure 1. To increase throughput in the long red region of the visible spectrum above ~650 nm, improving spectral estimation accuracy in this region, the internal IR cut filter of commercial RGB CFA cameras can be removed. A third, visible bandpass filter can then also be used to more desirably tune the spectral transmission and still limit it to the visible range (Figure 1, black line).

Figure 2 illustrates the result of pairing the blue-green and yellow filters with an IR-modified RGB CFA camera. The spectral sensitivity of the red, green, and blue channels are tuned differently by each colored filter, resulting in modified red, green, and blue channel sensitivities that are different between the two captures, and can be combined to create a six-channel spectral image stack.

The dual-RGB image stack can be calibrated for both colorimetric and spectral reflectance characterization via mathematical transformations that relate the six-channel spectral information ($R_{bg}{G}_{bg}{B}_{bg}{R}_y{G}_y{B}_y$) of an imaged calibration target to its measured reference values. For colorimetric calibration, these reference values are the measured CIEXYZ tristimulus values of the patches ($X_{ref}{Y}_{ref}{Z}_{ref}$), and for spectral reflectance calibration, they are the measured spectral reflectance curves of the patches ($R_{{\lambda }_1}$ to $R_{{\lambda }_n}$).

Equation (1) shows the relationship between the dual-RGB camera signals and CIEXYZ reference values. It is defined by the transformation matrix ${\mathbf{M}}_C$, which expanded is a 3-by-6 matrix, illustrated in Equation (2). The coefficients in the transformation matrix ${\mathbf{M}}_C$ are defined through iterative optimization, with the goal of minimizing the average CIEDE2000 color difference [27] between the dual-RGB camera signals averaged from a region inside each patch of the calibration target and the target’s measured reference values.

$$\left[\begin{array}{c}{X}_{ref}\\ Y_{ref}\\ Z_{ref}\end{array}\right]={\mathbf{M}}_{color}\left[\begin{array}{c}{R}_{bg}\\ G_{bg}\\ B_{bg}\\ R_y\\ G_y\\ B_y\end{array}\right]$$

(1)

where

$${\mathbf{M}}_{color}=\left[\begin{array}{cccccc}{m}_{1,1}& m_{1,2}& m_{1,3}& m_{1,4}& m_{1,5}& m_{1,6}\\ m_{2,1}& m_{2,2}& m_{2,3}& m_{2,4}& m_{2,5}& m_{2,6}\\ m_{3,1}& m_{3,2}& m_{3,3}& m_{3,4}& m_{3,5}& m_{3,6}\end{array}\right]$$

(2)

After optimizing the transformation between six-channel camera signals and reference target values, colorimetric calibration is complete. To obtain the color-calibrated image, the transformation matrix is applied to the six-channel image stack, followed by the desired linear color space matrix (e.g., ProPhotoRGB) and nonlinear encoding function. The process of rendering a highly-color-accurate image from the dual-RGB image stack is complete.

The parallel process of spectral reflectance calibration based on the six-channel image stack follows similar logic. The dual-RGB camera signals of the imaged calibration target are related to measured spectral reflectance through the spectral reflectance transformation matrix ${\mathbf{M}}_S$ as shown in Equation (3). R is an n-by-1 vector, where n is determined by the sampling of the reference spectral reflectance data (36 is typical: 380 nm to 730 nm in 10 nm increments). It follows that ${\mathbf{M}}_S$ has the dimensions n-by-6. Again, iterative optimization is used to define the transformation matrix coefficients. In this case, the minimized value is the root-mean-square error (RMSE) between the measured spectral reflectance and that estimated from the average dual-RGB camera signals sampled from the calibration target.

$$\left[\begin{array}{c}{R}_{{\lambda }_1}\\ ⋮\\ R_{{\lambda }_n}\end{array}\right]={\mathbf{M}}_S\left[\begin{array}{c}{R}_{bg}\\ G_{bg}\\ B_{bg}\\ R_y\\ G_y\\ B_y\end{array}\right]$$

(3)

where

$${\mathbf{M}}_S=\left[\begin{array}{cccccc}{m}_{1,1}& m_{1,2}& m_{1,3}& m_{1,4}& m_{1,5}& m_{1,6}\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ m_{n,1}& m_{n,2}& m_{n,3}& m_{n,4}& m_{n,5}& m_{n,6}\end{array}\right]$$

(4)

Dual-RGB was developed to provide both accurate color reproduction and spectral estimation, and importantly, it can easily be carried out using familiar cameras, studio lighting, inexpensive filters, and common color targets. As such, it served as the main influence for developing two-light imaging, which similarly builds up a six-channel spectral image cube, but does so through the use of tuned LED lighting, rather than filtered illumination.

2.2. Two-Light Imaging

Tunable, multichannel LED light sources are now widely available and more affordable, and offer a number of advantages over filter-based wavelength selection. Forgoing the need to screw on, slide, or otherwise shift filters into place reduces physical movement of the system that leads to registration error between channels. Different filters may also have varying optical properties that affect registration, and/or lead to large differences in the optimal exposure between shots, which can affect image quality. For a small investment, computer-controlled multichannel LED lights offer an elegant and flexible solution that can be integrated into an automated capture routine. Furthermore, using narrowband LEDs minimizes the object’s exposure to heat and other extraneous radiation [28,29].

Tunable light-based spectral imaging strategies that have been demonstrated and are currently in use for cultural heritage imaging have mainly paired narrowband LEDs with a monochrome sensor [28,30,31], echoing the common monochrome sensor + filter multispectral imaging approach [32]. There have been some other recent studies in which a three-channel RGB sensor is used [33,34]. This is the approach used here, in the two-light imaging technique, in which a six-channel spectral stack is created from the combination of two differently illuminated RGB captures. With the end goal of more approachable, everyday access to this kind of advanced imaging, taking advantage of the inherent three-channel nature of RGB CFA cameras provides an opportunity to explore utilizing familiar professional-level cameras for more studio-friendly spectral imaging.

Early experiments in this research verified that a dual-RGB imaging can be carried out effectively with a prosumer camera and inexpensive colored glass filters [12], and then confirmed that only a two-fold increase in the number of image channels used for colorimetric calibration, from the three channels of conventional RGB, to six spectral channels, significantly improves color rendering accuracy [13]. Others have explored trade-offs between filter-based versus light-based wavelength selection (i.e., dual-RGB versus two-light) [14,16], and effects of camera and lens choice on color accuracy with the two-light technique [17].

The two-light spectral imaging method was developed using ten-channel tunable LED light sources. The spectral power distributions of the ten channels are plotted in Figure 3a. A pair of lighting conditions, each consisting of a combination of three LEDs from the overall set of ten, were created, and their spectral power distributions are plotted in Figure 3b. The LEDs that make up each lighting condition were selected based on an exhaustive search optimization, in which all the color accuracy of all possible pairs of three-LED combinations was assessed, and the optimal combinations were computationally identified for a given camera and calibration target. The details of this process have been discussed elsewhere [16]. Note that the LED system currently utilized is a more advanced equipment option than is necessary based on the results of the optimization process. Only a subset of the ten channels is needed to create the two lighting conditions. These findings are informing future research toward purpose-built, lower cost lighting for two-light imaging.

The capture and calibration of two-light spectral image stacks follows that of dual-RGB imaging, where the objective is to modify the spectral sensitivity of the red, green, and blue channels differently under each lighting condition, resulting in sensitivities that are shifted relative to each other between the two captures, and can be combined to make a six-channel spectral image stack. Figure 4 illustrates the six channel sensitivity of the same IR modified commercial camera as above in Figure 2, but paired with the two lighting conditions from Figure 3b, rather than two filters.

Note that because the search optimization used for LED selection is defined around the color rendering accuracy of a calibration target, the optimal lighting conditions depend not only on the spectral characteristics of the camera sensitivity, but also on those of the specific calibration target chosen. In other words, the optimal lighting conditions will differ based on the camera and target used to define them. The RGB camera sensitivity shown in Figure 4 is that of an IR modified Sony $\alpha$7R III, which has been used as the model prosumer camera throughout this research. The ideal lighting conditions plotted in Figure 3 were created for it using a Digital Color Checker SG as the calibrating target.

The high-level capture and calibration strategies for carrying out two-light imaging follow those of dual-RGB imaging described above, where colorimetric and spectral calibration are carried out on the six-channel spectral image stack that is the combination of RGB image data collected under each of the optimized two lighting conditions. The colorimetric and spectral reflectance transforms are determined using CIEDE2000- and RMSE-guided matrix optimization over a reference color target. The final image can then be transformed, encoded, and rendered to a high degree of color accuracy according to these calibrations.

2.3. The Spectral Advantage

The CIEDE2000 matrix optimization method of colorimetric calibration, described above, is a versatile and adaptable method of characterizing camera color with respect to human perception [35]. In conventional imaging workflows, this process is what may be more familiarly called color profiling. It can be extended to the calibration of imaging systems with more than three channels, as was done here, with the definition of a larger transformation matrix. The larger matrix includes coefficients that characterize the contribution of the signal captured in the additional channels in the estimation of the CIEXYZ values for a given color in the image. While it may seem more intuitive to build up this transformation between quantities of the same dimensions, i.e., RGB camera signals to trichromatic human vision, even in conventional three-channel imaging, a single camera channel does not map one-to-one to a single tristimulus value—hence the need to define a 2D transformation matrix. The signals in each camera channel contribute in different amounts to the estimation of each tristimulus value. The quantity of each channel’s contribution is defined by the corresponding matrix coefficients. Increasing camera channels increases the degrees of freedom in the transformation, which improves estimation accuracy.

It follows, then, that the six-channel two-light imaging color calibration outperforms conventional RGB color calibration because of the increased amount of information used to build the transformation matrix. The additional coefficients in the matrix provide a more nuanced characterization of the visible spectrum, and operate as a means of better tuning the estimated CIEXYZ values. This is illustrated in Figure 5, in which the color accuracy of the Digital Color Checker SG target is simulated as rendered from conventional RGB versus two-light imaging data captured with the same camera. The heat maps indicate the color-coded $\Delta$E₀₀ color difference between the measured and rendered color of each patch of the target. Upon visual inspection alone, the conventional RGB rendering contains more lighter green- and yellow-coded patches, indicating that there are more patches rendered with a larger color difference relative to the two-light rendering. This is summarized well by comparing the mean and 90th percentile $\Delta$E₀₀ values across all the patches, reported beneath each map. Those of the two-light method are far smaller; the 90th percentile value of 0.4 $\Delta$E₀₀ is especially notable, as it indicates that a large majority of the color differences are below half of a $\Delta$E₀₀ unit, which is all but insignificant in the context of noticeable color differences in digital images [36].

Finally, an extreme example illustrating shortcomings of conventional color imaging is provided in Figure 6. A painting of the night sky that was imaged and rendered using both conventional (Figure 6a) and two-light (Figure 6b) techniques exhibits large differences in the appearance of a color found mainly along the horizon. To the eye, this paint looks blue, and is not visually different from rest of the dark blue sky. This is confirmed by renderings of the measured color of spots from both regions (Figure 6d). However, the measurements reveal the presence of different blue pigments in the two regions, with that at the horizon being cobalt blue, and the rest, phthalo blue. These two blue pigments, while visually similar, have strikingly different spectral reflectance shapes (Figure 6c). The six-channel sampling, particularly at longer visible wavelengths, leads to the large improvement of rendering this color more accurately, reducing the large 16.5 $\Delta$E₀₀ color difference between the measured and the rendered color exhibited by conventional RGB imaging down to 3.0 $\Delta$E₀₀ (Figure 6d).

3. Capture and Image Processing Methods

3.1. Imaging System

3.1.1. Equipment

The main camera used throughout development, testing, and demonstration of two-light imaging was the Sony $\alpha$7R III, a 42MP mirrorless digital camera equipped with pixel-shift multi-shot capabilities. This enables the direct capture of full-frame RGB images, thus bypassing the need for computational demosaicing of the CFA pattern. The particular camera used had its internal IR filter removed, evident in the plots of its spectral sensitivity in Figure 2a and Figure 4a, showing that the sensitivity of both the green and red channels extends above 700 nm. The camera was always controlled via computer tether with Sony’s Imaging Edge Remote.

The ten-channel tunable LED light sources paired with the camera were designed by LEDMotive [37], and have the spectral radiance characteristics described and plotted above (Figure 3a). The particular LEDs in the lights were previously chosen based on research into the optimal ten-channel set for a cultural heritage imaging system following a traditional sequential capture approach with a monochrome sensor [31]. Each light contains the LEDs and an internal integrating sphere for diffusion of the light within a small housing (16 × 12 × 12 cm). Externally, parabolic reflectors are attached around the port to shape the illumination and mimic studio strobe fixtures. The lights were also computer controlled using MATLAB scripts that enabled independent control of each LED channel.

Because this research focused on demonstrating the color accuracy of the two-light technique, color targets were a central part of the workflow for both calibration and verification purposes. Those used most commonly included the Digital Color Checker SG (X-Rite), the Next Generation Target V2 (Avian Rochester), and the Artist Paint Target (Image Science Associates). The Digital Color Checker SG (CCSG) is a familiar target that is already widely used in museum studio photography, and so it works well as an accessible tool in this workflow (Figure 5a). The Next Generation Target (NGT) was originally designed at the request of the Library of Congress to address concerns related to durability, sensitivity to lighting geometry, as well as more appropriate color gamut sampling for heritage materials [38] (Figure 7, left). Finally, the Artist Paint Target (APT), which was originally developed in the Munsell Color Science Lab, is particularly useful as a materially relevant target containing real artist paint mixtures [39] (Figure 7, right).

3.1.2. Setup

Imaging was carried out in a copy stand configuration, using 45°/0° illumination/ detection geometry to mimic the bi-directional detection of the reference spectrophotometer and reduce geometric error in the transfer between spectral reflectance measurement and imaging. The typical image set collected included two-light image pairs of (1) the desired color calibration target(s), (2) a flat field, (3) the dark current, with the lens cap covering the lens, and (4) the object(s). The shutter speeds used for each of the two lighting conditions were set by sampling the histograms of the calibration target’s white patches, and “exposing to the right” (ETTR) at the base ISO, while avoiding clipping, in order to maximize use of the sensor’s dynamic range [40]. All images were captured as photometric linear RAW files.

Because the only other physical components needed aside from the camera and lights are a tripod and light stands to mount them on, and a laptop for tethered computer control, the entire kit is easily packed into several Pelican cases for travel. This level of portability, as well as the simplicity of the setup, was key in enabling its demonstration in a range of different environments, including a small office, a classroom, a conservation lab, and a few imaging studios. A photo of the imaging setup is included in Figure 8.

3.2. Image Processing Software: Beyond RGB

There have been several software tools developed alongside the dual-RGB and other multispectral imaging projects in the Munsell Color Science Lab for processing these image data [41]. However, they found limited use outside the research lab environment due to concerns with stability and ease of use. This was motivation to create a software application to accompany two-light imaging that would be more easily adopted elsewhere, and more intuitive to use. A team of senior software engineering undergraduates was tasked with building such a solution, and the result was the development of Beyond RGB. It is an application that facilitates implementation of the two-light technique by providing a platform to process the resulting image data in a user-friendly way. Furthermore, it is cross-platform compatible and locally installable on both Windows and macOS systems, and it is open source and freely available at the project GitHub repository [42]. The form and function of Beyond RGB are summarized below; further details can be found elsewhere [15].

Beyond RGB carries out the colorimetric and spectral calibrations on two-light spectral image sets “under the hood” of a simple graphical user interface (Figure 9). It takes as input the RAW image set, including flatfields, darks, and the images of the targets and object. After the user provides information about the identity and spatial location of the color target, pre-processing calibration proceeds automatically. Pre-processing involves flat-fielding and dark current corrections to account for nonuniformities in the scene lighting and sensor [43], and to remove the black level of the camera, as well as spatial registration of the spectral channels to correct for chromatic aberration distortions between the two lighting conditions. These operations are all completed prior to carrying out the colorimetric and spectral calibration procedures outlined in the Background section above.

There is a simple image viewer built into Beyond RGB where the calibrated image is populated when calibration is complete. The viewing window enables preliminary visual inspection of the results, and includes a summary of the colorimetric data that characterizes the accuracy of the calibration. Additionally, there is a built-in spectral picker that allows the user to select regions of interest from which to display and export estimated reflectance spectra based on the spectral reflectance transform (Figure 10). The main focus of the application is encouraging the capture of spectral master files and enabling the calibration and export of the color managed RGB image. However, the ability to perform spectral estimation may be of more interest in future versions of the software that expand upon the more familiar applications of spectral imaging, such as pigment characterization and mapping. At present, the viewer and spectral picker are most useful for identifying regions of interest that may be of interest for further study by complementary techniques.

4. Results and Discussion

4.1. Institutions Visited

This final section summarizes the general successes and lessons learned over the course of the demonstration and testing visits. The institutions visited differed in size, kinds of collections, and geographic location, and included

• the Cary Graphic Arts Collection at the RIT Libraries (Rochester, NY, USA),
• the National Cryptologic Museum (Annapolis Junction, MD, USA),
• the US Army Heritage and Education Center (Carlisle, PA, USA),
• the Museum of Modern Art (New York, NY, USA),
• the Art Conservation Department at Buffalo State College (Buffalo, NY, USA), and
• the George Eastman Museum (Rochester, NY, USA)

A majority of the visits included both an introductory presentation on the technique and its advantages over conventional color imaging, followed by a demonstration of the system while imaging some collections objects.

4.2. Example Results

The series of images below (Figure 11, Figure 12, Figure 13 and Figure 14) are representative selections from those captured of collections objects during the visits. They were chosen to show examples of the range of materials and colors imaged. They include the marbled paper inside cover and the title page of an embroidered book from the Cary Graphic Arts Collection, a painted flag and a felt uniform patch, both from the US Army Heritage and Education Center collection, and a platinotype photograph from a study collection at RIT that was included in a previous art reproduction case study [44]. All images were calibrated using the Digital Color Checker SG. The flag, patch, and photograph were captured with both the conventional color imaging and two-light imaging to enable qualitative comparison between the two. For the book, only the two-light image rendering is available. It is included as an example of the diverse material that was imaged. Additionally, spot measurements made with a handheld spectrophotometer were collected on a few regions of interest on each object to provide ground truth against which the rendered colors are compared quantitatively. For each spot, the measured, conventional color imaging (where available), and two-light imaging CIELAB values are reported, as well as the corresponding $\Delta$E₀₀ color difference between the measured and rendered colors.

Unsurprisingly, two-light imaging outperformed conventional color imaging across the board, showing smaller color differences with respect to the true color in the sampled regions. The largest of these color differences are visually obvious when comparing the less accurate to the more accurate rendering. The images of the flag were included as an example of one of these most noticeably different renderings, in the blue background and stars. The Red 1 patch is included because the conservators commented that the olive green color is one that they have noticed in particular that does not photograph well. Two-light imaging appears to have reduced this problem. The platinotype photograph is included because it proved to be a particularly difficult object to reproduce well by museum imaging methods that were in place in a study carried out a decade ago [44]. The photograph has a very flat reflectance curve across the visible range that is not well characterized by three-channel sampling, but two-light imaging better samples this curve shape, and produces comparatively less of the pinkish cast evident in the conventional color rendering.

It is worth noting again that all of these images were calibrated using the Digital Color Checker SG. For the two-light images, the color-calibrated mean $\Delta$E₀₀ across the target did not exceed 1 $\Delta$E₀₀. A typical $\Delta$E₀₀ target heat map result for one of the calibrations is provided in Figure 15, in which the mean $\Delta$E₀₀ is 0.8. However, the mean level of color accuracy of 0.8 $\Delta$E₀₀ is not achieved in the regions of interest checked with spot measurements. This is a somewhat expected result when using a commercial target that is not the same material nor a close representation of the gamut of colors in the real objects. This shows the value that custom, materially specific and color-curated targets can add to a workflow.

As an aside, the cause of the single outlier value of 5.0 $\Delta$E₀₀ for patch C6 in Figure 15 is unknown. There is a possibility that this particular patch of the semi-glossy target caught a glare during capture, throwing off its calibration. The presence of outliers like these illustrate the value in reporting the mean and 90th percentile $\Delta$E₀₀ values as more representative statistics describing the reproduction of the vast majority of the patches in the set.

4.3. Feedback and Future Work

The demonstrated adherence to copy stand lighting geometry was a common source of concern. This setup provides consistency with 45°/0° spectrophotometric target measurement geometry. While it results in technical accuracy, this approach does not leave room for using more complicated illumination setups that better highlight the character of, for example, a painting’s surface texture, nor does it reproduce what one might expect an artwork to look like under gallery lighting conditions. A previous case study found that for conventional color imaging, as long as exposure is set correctly, straying from copy stand lighting geometry had little effect on final color accuracy [45]. This remains to be verified for two-light imaging, but opens the door to possibilities for more creative lighting setups, or even integration with multi-light 2.5D imaging techniques for simultaneously recording color and surface texture [46].

Among the most valuable discussions were those related to improvements and additional features and functions in Beyond RGB. This was a particularly important aspect to gather feedback on, as the project timeline for the development and release of the first version of the software did not allow much time for user testing of the software ahead of the visits. Positive impressions of the software included the simplicity of interacting with it through a graphical user interface, the rendered image viewer and spectral picker as a first-pass inspection tool, and the open-source, free-to-use nature of the project. There were many helpful comments made about small ways to refine the graphical user interface to improve functionality and flow, like drag-and-drop file import, auto-populating fields based on expected naming conventions, and automated target detection. Some of the most important, big picture suggestions for improvements included:

• Batch processing. At present, a single calibration run of the software handles a single object image at a time, requiring time-consuming resetting of the calibration parameters for each run. Batch processing would enable the user to calibrate an entire set of images captured under the same calibration conditions much more efficiently.
• A project website. Currently, the project is hosted entirely on its GitHub repository. This can be difficult to navigate for novice users who only need access to the installation packages, Wiki, and user guide. These distributables would be better housed on a separate, dedicated website.
• An Adobe Camera Raw (ACR) RAW to TIFF workflow. The current version of Beyond RGB was tested with and supports Canon, Fujifilm, Nikon, and Sony RAW file formats. For cases where input images are not in one of these formats, it also supports uncompressed, unprocessed linear TIFFs created from RAW files. As ACR is a popular tool for working with RAW images, it was requested that specific guidelines for setting the correct parameters to obtain unprocessed TIFFs from RAWs using ACR be provided.
• Material mapping. The ability to estimate pixel-by-pixel reflectance spectra is a feature that already exists in Beyond RGB (Figure 10). Building in the capability to group and visualize the distribution of reflectance spectra having similar spectral features would be the first steps toward exploring the applicability of two-light imaging to the more typical tasks of spectral imaging, like pigment characterization and mapping.

These suggestions and many more, both big and small, were compiled into a master wish list of features that are currently at the center of ongoing updates to Beyond RGB. They will be available in the next version of the software and documentation, which is anticipated in Spring 2023.

The feedback and impressions from the visits were overall positive. The needs and priorities for imaging at each institution differ, and conveying the benefits that two-light imaging could offer in these varied contexts was successful. The fact that two-light imaging involves a specific set of capture and processing procedures is useful in and of itself. For instance, this could aid in providing a more structured approach to documentation imaging in conservation labs in which current practice is less rigorously controlled and outdated cameras are used. Similarly, in institutions without dedicated imaging personnel or space, this is an efficient, portable means of capturing accurate color that largely removes time-consuming, subjective post-production corrections. Two-light imaging is also a useful teaching tool for introducing not just the concepts and practice of spectral imaging, but also human vision, color appearance, and quality control at academic and teaching institutions.

The system that was demonstrated is a proof-of-concept prototype. Presently, it still utilizes LED light sources that are likely out-of-budget for all but larger institutions. These visits were critical to emphasize to the community that with growing interest, there is the potential for future development of simplified lighting solutions that would be a less costly option for carrying out two-light imaging. Alternatively, tunable broadband LED lamps, such as the Broncolor F160, are now becoming a more common studio lighting option. It is not outside the realm of possibility to imagine that a dual-purpose fixture, integrating narrowband channels into such an existing LED system, to meet the needs of both conventional and two-light imaging, could be an attractive option. Regardless, the interest expressed in pursuing such ideas was encouraging, and shows that there is momentum for continuing to improve access to these advanced imaging practices across more institutions.

5. Conclusions

The two-light imaging technique has been demonstrated as an effective capture method for color-accurate rendering and spectral archiving that is practical for integration with routine studio photography workflows. The descriptions of the influential history and development of dual-RGB imaging and the method of transforming six-channel spectral data to a calibrated color-accurate rendering provide context for two-light imaging, and further supplement the information from the on-site demonstrations. Traveling with the system to several institutions with varied imaging capabilities and goals was the first true test of its flexibility beyond the research and development environment. It also provided the opportunity to describe and demonstrate the advantages of the two-light imaging technique to diverse audiences, and to gather impressions about how the system might fit into existing imaging workflows. This feedback is informing updates and improvements to the system moving forward, which will focus especially on adding features to the Beyond RGB software and developing lower-cost lighting solutions optimized specifically for two-light imaging. These will also influence continued efforts to collaborate with institutions to provide accessible education around spectral imaging, and to communicate the ways it might enhance current frameworks of cultural heritage digitization and archiving.