Next Article in Journal
Vehicle-Mounted Solar Occultation Flux Fourier Transform Infrared Spectrometer and Its Remote Sensing Application
Previous Article in Journal
Vehicle Logo Recognition Using Spatial Structure Correlation and YOLO-T
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

High-Dynamic-Range Spectral Reflectance for the Segmentation of Paint Pigment: Application to Dalí’s Oil Painting Dos Figuras (1926)

by
Antonio Alvarez Fernandez-Balbuena
1,
Angela Gómez-Manzanares
1,*,
Juan Carlos Martínez Antón
1,
Jorge García Gómez-Tejedor
2,
Santiago Mayorga-Pinilla
1,
Humberto Durán Roque
3 and
Daniel Vázquez Moliní
1
1
Department of Optics, University Complutense of Madrid, 28040 Madrid, Spain
2
Museo Nacional Centro de Arte Reina Sofía, 28012 Madrid, Spain
3
Hdurán: Imágenes para Estudios Técnicos Conservación y Restauración, 28005 Madrid, Spain
*
Author to whom correspondence should be addressed.
Sensors 2023, 23(9), 4316; https://doi.org/10.3390/s23094316
Submission received: 19 March 2023 / Revised: 18 April 2023 / Accepted: 24 April 2023 / Published: 27 April 2023
(This article belongs to the Section Optical Sensors)

Abstract

:
Restorers and curators in museums sometimes find it difficult to accurately segment areas of paintings that have been contaminated with other pigments or areas that need to be restored, and work on the painting needs to be carried out with minimum possible damage. It is therefore necessary to develop measurement systems and methods that facilitate this task in the least invasive way possible. The aim of this study was to obtain high-dynamic-range (HDR) spectral reflectance and spatial resolution for Dalí’s painting entitled Two Figures (1926) in order to segment a small area of black and white pigment that was affected by the contact transfer of reddish pigment from another painting. Using Hypermatrixcam to measure the HDR spectral reflectance developed by this research team, an HDR multispectral cube of 12 images was obtained for the band 470–690 nm in steps of 20 nm. With the values obtained for the spectral reflectance of the HDR cube, the colour of the area of paint affected by the transfer was studied by calculating the a*b* components with the CIELab system. These a*b* values were then used to define two methods of segmenting the exact areas in which there was a transfer of reddish pigment. The area studied in the painting was originally black, and the contamination with reddish pigment occupied 13.87% to 32% of the total area depending on the selected method. These different solutions can be explained because the lower limit is segmentation based on pure pigment and the upper limit considers red as an exclusion of non-black pigment. Over- and under-segmentation is a common problem described in the literature related to pigment selection. In this application case, as red pigment is not original and should be removed, curators will choose the method that selects the highest red area.

1. Introduction

Restoration and curation techniques in the area of cultural heritage require the use of technologies that allow for the characterisation of the pigments that make up artworks in a precise and non-invasive manner [1,2]. To carry out this characterisation, colorimetric analysis based on spectral reflectance measurements is used [1,3]. The spectral reflectance parameter of an artwork can be used in several ways—for example, to characterise pigments [4], to verify the presence of a pigment in different parts of the artwork through pigment segmentation [5], and to control and monitor the deterioration of the artwork over time, among many other applications.
The proper characterisation and identification of pigments in artworks depend on the spatial and spectral resolution of the system [6]. Imaging spectrophotometers have generally been used to obtain spectral reflectance for artworks via a non-contact process [7,8]. However, spectrophotometers have a limited spot measurement area and are therefore unsuitable for use with large artworks [9,10]. To eliminate this spatial limitation, multispectral imaging (MSI) and hyperspectral imaging (HSI) are used to obtain diffuse spectral reflectance for artworks [5,11,12].
MSI and HSI can obtain diffuse spectral reflectance with a resolution that depends on the pixel size of the system sensors and the optical quality of the system, as they measure the spectral information of each pixel (px) of the image [13]. Commercially available MSI and HSI capture systems provide imaging with high efficiency; however, there is a tradeoff between the two methods, as hyperspectral cameras can obtain images with a high spectral resolution but low spatial resolution, while multispectral cameras can obtain images with a high spatial resolution but low spectral resolution [14]. For example, the Specim IQ is a hyperspectral camera that is capable of imaging in 204 spectral bands, covering the spectral range 400–1000 nm, with a spatial resolution of 1260 × 960 px, whereas the Micasense ReddishEdge-P multispectral camera images in five spectral bands, with a spatial resolution of 2464 × 2056 px. The precise spatial and spectral resolution that defines the difference between the two methods is unclear. However, recent research explains that the difference lies in the continuum of the detected spectrum. Hyperspectral imaging systems are characterised by the fact that they are able to image over a continuous range of the spectrum [15,16].
Numerous studies have been carried out with the aim of capturing HSI with a high spatial resolution. In their research, Yifan et al. developed an algorithm that could obtain images with the spatial resolution of MSI and the spectral resolution of HSI, using the 3D wavelet transform [17]. In order to carry out HSI with high spatial resolution, Akhtar et al. developed an algorithm that combined an RGB image with higher spatial resolution to spectrally define the RGB channels of the image [18].
In the field of cultural heritage restoration and conservation, it is necessary to obtain HSI with high spatial resolution to perform the segmentation of large artworks. Super-resolution algorithms are widely used to perform pigment segmentation of artworks [2,19]. The aim of super-resolution segmentation is to obtain images with high-spatial-resolution information from images with a low spatial resolution, based on a prior grouping of pixels with the same spectral characteristics [5,19,20].
When documenting and analysing artworks, the gigapixel technique is widely used with RGB cameras, as commercial RGB cameras enable images with a high spatial resolution to be obtained [21].
Pigment segmentation is a technique that can be mathematically executed with the HSI technique and post-processing. Post-processing is used in segmentation because over- or under-segmentation can take place [2]. Magro et al. [2] reported that HSI reduced 43% over-segmentation in the proposed algorithm. Different pigments present different reflection characteristics, so in the visible spectrum, we can use this information to identify pigments and normalise the spectral reflectance difference to a database that can identify pigments. This method was used by Li J. et al. [5] to characterise and segment the pigments in the Mogao Caves in Dunhuang.
However, the colorimetric properties of the pigments that make up the paints alter over time due to the action of temperature, humidity, and exposure to light, among other things [22]. In addition, the pigments in the paints combine with each other, which alters their colour properties and results in variations in spectral reflectance, making characterisation difficult to carry out in the segmentation of these pigments [23]. Hence, to properly characterise and segment the pigments in an artwork, it is necessary to implement techniques and algorithms that consider these possible mixtures of pigments, in addition to using systems with a high spatial resolution.
A Spectral Angle Mapper (SAM) is a technique that is often used to identify similarities between the spectral profile of pigments in a painting and reference pigments that have been previously characterised and are available in a library [24,25]. In their study, Deborah et al. performed pigment segmentation of Edvard Munch’s painting The Scream using an algorithm based on spectral divergence [26].
In this work, we present spectral reflectance measurements of Dalí’s painting using multispectral images with high-dynamic-range and 4K spatial resolution. Based on these measurements, two methods are developed for performing pigment segmentation and identification, taking into account variations due to pigment mixing.
Section 2 of this article describes the materials and methods, including the MSI system used, and explains the measurement and calibration procedures. Section 3 presents the results, which demonstrate the high dynamic range of the spectral reflectance of the artwork, and introduces the methods used to carry out pigment segmentation. Finally, Section 4 and Section 5 present a discussion and the conclusions reached after carrying out this study.

2. Materials and Methods

Salvador Dalí’s Two Figures (1926) was affected by the transfer of a reddish pigment when another painting came into contact with this work. Figure 1b,c show the detail of the reddish pigment, obtained from a gigapixel image taken by the restoration team of the Museo Nacional Centro de Arte Reina Sofía (Figure 1a).

2.1. Multispectral Image Capture System

The segmentation and localisation of the area of the artwork affected by the reddish pigment require previous colorimetric characterisation based on the spectral reflectance. In order to identify this pigment, this research team has developed a multispectral image capture system (Hypermatrixcam) that is capable of obtaining spectral images with a high spatial resolution [27].
This image capture system consists of a matrix array of 12 RGB sensors with 4K spatial resolution (Figure 2a); patent ES2819052-B2. A spectral bandpass filter covering the visible spectrum range of 470–690 nm is placed on each of the sensors (Figure 2b).

2.2. Experimental Setup

In its recommendation 15-2004, the International Commission on Illumination (CIE) established specific geometrical conditions for spectral reflectance measurements depending on the surface to be tested and the instrument used [28]. Furthermore, Fairchild, in his book, recommends that for reflectance measurements of artworks using spectrophotometers or multispectral or hyperspectral cameras, the light source should be placed at an angle of 45° to the sample to avoid direct reflections [29].
Dalí’s painting Two Figures (1926) measures 1.98 × 1.49 m, and the distance at which the camera should be placed with respect to the painting depends on the field of view of the measurement system. Each sensor has a field of view of 30° × 27.7° (horizontal × vertical); hence, considering the geometric displacement of the 12 sensors, the field of view of Hypermatrixcam is 23.4° × 20.5° (horizontal × vertical). This gives a value of 4.8 m for the distance  d D a l i  necessary to obtain multispectral images of the entire painting while leaving a margin for colorimetric patterns.
This distance for the measurement system allows multispectral images of the painting to be obtained with a resolution of 3322 × 2510 pixels (horizontal × vertical). The expression  r h D a l i = S D a l i / H S I D a l i = 0.59   mm  was used, where  S D a l i  is the size of the painting in mm and  H S I D a l i  is the size of the image obtained in pixels. This made it possible to record areas of 0.35 mm2 of the artwork for each pixel of the HSI obtained (Figure 3).

2.2.1. Lighting System

Three Art Lux 500 Flex LED array panels with a total size of 410 × 1830 mm (horizontal × vertical) were chosen as the light source for the reflectance measurements. The choice of LED panels was made due to the absence of UV and IR radiation in their emission to avoid damaging the artwork [30]. An LED array was placed at a vertical angle of 45° to the centre of the painting in order to ensure maximum uniformity in the illumination of the painting.

2.2.2. System Calibration

Once the geometrical configuration for the spectral reflectance measurements had been established, the system was calibrated. For this purpose, an HSI image was initially captured without the painting; as a calibration standard, we used a foamed PVC sheet, a uniform material with good diffusive properties and a uniform reflectance value. The PVC sheet had to be of the same dimensions as the Dalí painting in order to calibrate the nonuniformity of the illumination system. The mean reflectance  ρ w h i t e λ  of this material was measured with a PR-655 spectrophotometer (Photo Research Inc., Topanga, CA, USA) in the geometric configuration 0–45° directional [28]. Measurements were performed at 18 different equidistant points, and a value of  ρ w h i t e λ = 85.07 % ± 2.68 %  was obtained.
An image of the PVC was then captured with Hypermatrixcam to give the spectral HDR of the reference target  I H D R w h i t e λ n , m , where (n, m) is the size in pixels of the images taken. With the resolution of Hypermatrixcam, these values were m = 3322 pixels and n = 2510 pixels.
Without changing the positions of the camera or the lighting system, images of Dalí’s painting were then captured in the same place where the foamed PVC was previously located to give the spectral HDR of the painting  I H D R s λ n , m .
To complete the measurement, multispectral images of the dark current were captured, with the sensors covered, to give the HDR of the dark current of the system  I H D R d a r k λ n , m .

2.3. HDR Image Capture

Multispectral and hyperspectral cameras typically consist of sensors with a limited dynamic range, resulting in low-dynamic-range (LDR) images [31]. These LDR images will contain saturated and/or underexposed pixels for a given exposure time. In addition, the response of these sensors is not linear and shows areas of nonlinearity for low and high values of the grey levels.
In order to obtain spectral reflectance measurements using HSI, all of the pixels must have a value for the grey level that is within the linearity zone of the sensor [32]; otherwise, the spectral characterisation will be altered.
To define the linearity limits in the response of the CMOS sensors, the curve of the sensors as a function of exposure time was obtained (Figure 4). For these measurements, white LED illumination was used, without the superimposition of spectral filters on the sensors, since the distortion between the response curves of the sensors for the different wavelengths is minimal in the visible spectrum [33].
Dalí’s painting Two Figures (1926) has very light and dark areas on its surface. As shown in Figure 5, capturing images at a certain exposure time results in pixels that are outside the linearity zone of the sensor, with both saturated (yellow) and underexposed (pink) pixels in the same image.
The system used in this work allowed HDR multispectral images to be obtained by capturing them over different exposure times. Based on LDR images, an HDR image is obtained from a combination of the grey levels in each pixel that are not saturated or underexposed.
The MSI acquisition system consisted of RGB sensors and spectral filters. The sensitivity of the RGB sensors and the transmittance of the spectral filters varied spectrally for each wavelength, and images were therefore captured with different exposure times. These exposure times were selected by adjusting them to minimise the saturated pixel values in each of the intervals of the working spectrum.
The range of exposure times chosen for wavelengths in the range 470–630 nm was 0.5–12.5 s in steps of 3 s. For wavelengths in the range 650–690 nm, the range of exposure times was 8–20 s in steps of 4 s. After obtaining the HDR multispectral images of the calibration, paint, and dark current, the high-dynamic-range spectral reflectance  ρ s λ n , m  at each pixel (n, m) was calculated using Equation (1):
ρ s λ n , m = ρ w h i t e λ n , m I H D R s λ n , m I H D R d a r k λ n , m I H D R w h i t e λ n , m I H D R d a r k λ n , m
where  ρ w h i t e λ n , m = 85.4 %  is the mean spectral reflectance of the PVC used for calibration;  I H D R w h i t e λ n , m  is the MSI of the PVC used as calibration material;  I H D R s λ n , m  is the MSI of Dali’s painting; and  I H D R d a r k λ n , m  is the HSI of the dark current.

2.4. Basis of Methods for Pigment Segmentation

A spectral reflectance cube can be used for pigment segmentation; in this case, two methods are described and applied to the painting. Both methods require processing a spectral reflectance cube with a standard illuminant to obtain CIELab data, a* and b*. These data are related to the pixel colour.
Due to the mixture between pigments and the alterations they suffer in terms of their colorimetric properties over time, the segmentation of two pigments whose reflectance spectra are close to each other is difficult [23]. When processing the reflectance spectra according to a* and b* values, this division suffers the same problem, and establishing a separation boundary becomes quite arbitrary. To help in the segmentation process, we need some extra information. We propose two methods that basically focus on image areas with a predominance of a type of pigment (black or reddish in our application case).
Method 1: Inverse segmentation by characterising a black area without the transfer of reddish pigment.
In this method, additional data consist of obtaining representative a* and b* data of the black pigment from an area that has no red pigment at all.
If you segment black according to these data, then red will be the remaining pixels because there are only two colours in the selected area.
In this method, we can define by a set U with all a*b* data in the image:
    U =   a * b *  
Additionally, we can define a subset K:
K = a b   |   b l a c k   p i g m e n t ,   with   K U
where  K  is a subset containing CIELab black pigment that was selected using an area of the image with only black pigment. In practice, this defines a closed boundary βb in the a*b* data representation that permits a simple segmentation. This process can be described by
U \ K = R M 1
where  R M 1  is the subset of data that are non-black. So, in a bi-coloured image, red and black, this is red for a different saturation level.
Method 2: Segmentation based on an area with a high incidence of red pigment.
This method takes a* and b* data from a selected area strongly related to the red pigment. Selection in other areas is made by obtaining data that have the a* and b* that are inside the boundary obtained by directly analysing the red pigment.
In this case, any image subset that only has red pigment gives:
R M 2 = { a * b * |   r e d   p i g m e n t }
In Equation (5), the  R M 2  set is obtained by selecting a point cloud area with only red pigment; then, the a*b* data are surrounded by a boundary βred that will enclose all red pigment related to the selected boundary. So, it is possible to segment, in any image with any colour, with these set data to obtain red.

3. Results

3.1. High-Dynamic-Range Spectral Reflectance

Multispectral images (MSI) were acquired with different exposure times to give HDR images of Dalí’s painting Two Figures (1926). From these images, and using Equation (1), the HDR spectral reflectance was processed, as shown in Figure 6. The spectral resolution of the measurements was 470–690 nm in steps of 20 nm, and the spatial resolution was 3322 × 2510 px.

3.2. Pigment Segmentation

The segmentation of the pigments in the painting was carried out using the multispectral images obtained as described above. Each multispectral image had a spatial resolution of  s r H D R = 3322 × 2510   px = 8.3   Mpx  and a spectral resolution of 12 channels.
To segment the pigment, it was first necessary to characterise it colorimetrically. For this purpose, a small area of the paint affected by the transfer of the reddish pigment (Reference Area 1) was selected, with a size of 6.8 × 4.9 cm. Figure 7a shows a high-spatial-resolution gigapixel RGB image corresponding to Reference Area 1, while Figure 7b shows the HDR spectral reflectance value for the same area at a wavelength of 630 nm, where the reddish pigment is detected as an area of pixels with higher reflectance (shown in yellow on the false colour map). Figure 7c shows the RGB image obtained from the HDR spectral reflectance measurements.
Using illuminant A as a reference and a standard CIE 1931 observer, the first step is to calculate the XYZ tristimulus values from the reflectance spectrum of each pixel of Reference Area 1,  n , m ρ s λ n , m .
The CIELab  a * b *  colour values of the selected area  n , m  are calculated from the tristimulus data  X Y Z n , m  [29,34].
Figure 8 shows the data for  a * n , m ,   b * n , m . The CIELab enhancement in  a *  is standard for the red colour enhancement, and the hue angle following the enhancement in the components  a * n , m ,   b * n , m  is observed, but it is not easy to find a delimiter between the reddish and black pigment. Two methods are proposed below for carrying out this segmentation.
Method 1: Inverse segmentation by characterising a black area without the transfer of reddish pigment
In this case, a black area was selected (Figure 9a), and its components  a * b * b l a c k  were calculated. The components  a * b * b l a c k  formed a point cloud, which was plotted next to the boundary of the  a * b * b l a c k  values; this boundary allowed us to segment the rest of the  a * b *  values present in the painting (Figure 9b).
Once the region with the values  a * b * b l a c k  had been defined, it was represented next to the envelope bounding the values  a * n , m ,   b * n , m  of Reference Area 1, which was selected to characterise the red pigment (Figure 10a). The geometric difference between the two regions was then found in order to define the region with the  a * b * r e d d i s h _ 1  values corresponding to the reddish pigment (Figure 10b).
To validate this method, we selected an area (called the segmentation area) that was larger than the one selected as a reference for the analysis. Figure 11a shows the segmentation area in an RGB image, and Figure 11b shows the segmentation area in a false colour image, in which the reflectance at 630 nm is represented.
Using the  a * b * r e d d i s h _ 1  values obtained above, the reddish pigment present in the segmentation area was segmented (Figure 12).
Method 2: Segmentation based on an area with a high incidence of red pigment
Rather than performing segmentation of the reddish pigment based on the previous characterisation of a black area, it is also possible to perform segmentation based on the HDR spectral reflectance of an area with only a high incidence of red pigment, called Reference Area 2 (Figure 13).
In the same way as described in the previous section, the coordinates  a * b * r e d d i s h _ 2  for Reference Area 2 were obtained using illuminant A and a standard CIE 1931 observer at 2° as the reference illuminant. The region bounding the  a * b * r e d d i s h _ 2  values was then obtained and plotted next to the region of  a * b * b l a c k  values (Figure 14).
Since these two regions do not overlap, segmentation can be performed directly on the segmentation area based on the region with the values  a * b * r e d d i s h _ 2  (Figure 15).

3.3. Comparison of Proposed Methods

The results obtained with both methods are shown in Table 1. Using Method 1, 4307 more pixels were obtained compared to those obtained with Method 2, equivalent to 18.07% of the affected segmentation area.
For each of the two proposed methods, the mean value of the spectral reflectance spectra of all pixels included in the segmentation was calculated. The spectral reflectance values for each method have been processed to obtain the corresponding RGB values for method 1 (Figure 16a) and method 2 (Figure 16b). The CIE 1931 2° Observer Standard and Illuminant A were used for the processing. The RGB values have been multiplied by a factor of two to increase their luminance level and improve visual colour appreciation.

4. Discussion

This study presents measurements of the HDR spectral reflectance for Dalí’s painting entitled Two Figures (1926) and a colorimetric analysis of the reddish pigment that affects the work due to the transfer between pigments.
To enable non-contact measurements, the segmentation and quantification of the pigments that make up a painting are typically carried out through a colorimetric analysis using spectral reflectance measurements. Since pigments are distributed in very small quantities over the artwork, in order to carry out characterisation and segmentation, it is necessary to use methods that allow spectral information to be obtained with high spectral and spatial resolution. Commercial MSI and HSI capture systems do not have sufficient spatial resolution to segment pigments in large artworks. Hence, although a combination of HSI and MSI increases the spatial resolution of HSI, it is not sufficient to perform pigment segmentation accurately [17]. Adequate spatial resolution is therefore achieved by fusing HSI with RGB images [18].
Super-resolution algorithms perform a posteriori processing of the spectral reflectance information obtained from HSI [2,19], and the information obtained in this way allows for segmentation and subsequent mapping of the different pigments used in the artwork. However, these algorithms are not able to detect small amounts of pigment in paintings or sculptures.
With the multispectral image capture system used in this research, it was possible to obtain spectral information of an area of 0.35 mm2 of Dalí’s painting in each pixel of the image obtained. The spectral and spatial resolution of these HDR spectral reflectance measurements allowed us to characterise the reddish pigment present in the painting.
Due to the difficulty of defining the thresholds between a pure pigment, its mixtures, and the rest of the pigments, SAM and divergence techniques are interesting for use in pigment segmentation [24,25,26]. When a spectral library is taken as a reference, they allow us to quantify the similarity between a pure pigment and its mixtures.
In this study, the pigment present in the paint itself was used for characterisation, and a comparison with reference spectral libraries was avoided. Furthermore, prior to segmentation, the two methods proposed here were used to characterise the pigment in its pure and mixed states based on the a* and b* coordinates, thus simplifying the process.
In Method 1, a segmentation was performed that included areas in which the pigment had been mixed and which were imperceptible to the naked eye. Method 2 involved an analysis based on the most visible areas, since Reference Area 2 was an area of the artwork containing pure red pigment. The choice between the methods described above will depend on the needs of restoration and conservation experts in the field of cultural heritage, since the line that delimits a pigment is not clear; this is especially the case if red and black pigments are compared, as was the case here, as they are colorimetrically similar to each other. As pigment is on the painting surface, the reflectance is not always strictly related to the added pigment; this is because is the deposition layer is thin, and the reflectance is a mix between the original layer and the added pigment layer. In this way, pigment segmentation has a limitation in method 2 because this selection only obtains a big pigment layer. Method 1 can select the thinnest layer of red because it is obtained by the exclusion of black; this is only useful if there is only black and red. If there are additional colours, they will be segmented like red because they are not black.
In the application case of this Dalí painting, the curator selection of interest is the highest area (Method 1); thus, all of it is intrusive pigment that must be eliminated because it is a transfer from another painting.

5. Conclusions

This study has presented HDR spectral reflectance measurements of Dalí’s painting entitled Two Figures (1926), with 12 spectral bands and high spatial resolution. The data obtained represent objective information on the colours of the painting for future interventions and analyses of its state of conservation. Our approach allows for a resolution of 3322 × 2510 px. As part of our analysis of the spectral reflectance of the painting, the reddish pigment transfer affecting a piece of the artwork was characterised and segmented with two proposed methods. In the same area, Method 1 gives 32% of reddish pigment and method 2 gives 13.87% of reddish pigment. Method 2 obtains a pure red component and method 1 obtains more colour variations around the reddish pigment. The proposed methods can provide additional tools for the objective, non-contact analysis of the pigments used in artworks. In this application case, Method 1 will be selected by curators; thus, all of the red pigment is not from the original painting and needs to be eliminated.

6. Patents

Patent ES2819052-B2. Cámara multi o hiperespectral para mantener el aumento lateral ajustando el enfoque. Fernández-Balbuena A. A., Bernárdez Vilaboa, R., Molini, D. V., Pinilla, S. M., & Gómez Manzanares, A.
Patent ES2911099-B2. Instrumento y método para calibrar la uniformidad de la iluminación con aplicación en medida de reflectancia con imágenes multiespectrales o hiperespectrales. Fernández-Balbuena A. A., Gómez Manzanares, A., Molini, D. V., Pinilla, S. M., Martínez-Antón, J. C., & Bernárdez Vilaboa, R.

Author Contributions

Conceptualisation, A.G.-M., D.V.M., A.A.F.-B. and S.M.-P.; methodology, A.G.-M., D.V.M., A.A.F.-B., S.M.-P. and J.C.M.A.; software, A.G.-M., A.A.F.-B. and S.M.-P.; validation, A.G.-M. and A.A.F.-B.; formal analysis, A.G.-M., D.V.M. and A.A.F.-B.; investigation, A.G.-M., D.V.M., A.A.F.-B., S.M.-P. and J.C.M.A.; resources, D.V.M. and A.A.F.-B.; data preparation, A.G.-M.; writing—original draft preparation, writing—review and editing, D.V.M., A.A.F.-B., S.M.-P. and J.C.M.A.; visualisation, A.A.F.-B. and S.M.-P.; project administration, D.V.M. and A.A.F.-B.; funding acquisition, D.V.M. and A.A.F.-B.; painting preparation, gigapixel image acquisition and preparation, H.D.R. and J.G.G.-T.; museum permissions and paper review, J.G.G.-T. All authors have read and agreed to the published version of the manuscript.

Funding

This work was funded by project number RTI2018-097633-A-I00 of the Ministry of Science and Innovation of Spain, entitled ‘Photonic restoration applied to cultural heritage: Application to Dalí’s painting: Two Figures’.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

Data are available on request from the corresponding author, due to restrictions on museum data related to Dalí’s painting.

Acknowledgments

The authors would like to thank Jorge García, Carmen Muro, and Humberto Durán from the Conservation and Restoration Department of the Museo Reina Sofía for their important contributions in regard to the state of conservation of Dalí’s painting.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Cucci, C.; Delaney, J.K.; Picollo, M. Reflectance hyperspectral imaging for investigation of works of art: Old master paintings and illuminated manuscripts. Acc. Chem. Res. 2016, 49, 2070–2079. [Google Scholar] [CrossRef]
  2. Magro, N.; Bonnici, A.; Cristina, S. Hyperspectral Image Segmentation For Paint Analysis. In Proceedings of the 2021 IEEE International Conference on Image Processing (ICIP), Anchorage, AK, USA, 19–22 September 2021; pp. 1374–1378. [Google Scholar]
  3. Delaney, J.K.; Picollo, M. JAIC special issue on “Reflectance hyperspectral imaging to support documentation and conservation of 2D artworks”. J. Am. Inst. Conserv. 2019, 58, 1–2. [Google Scholar] [CrossRef]
  4. Striova, J.; Dal Fovo, A.; Fontana, R. Reflectance imaging spectroscopy in heritage science. La Riv. Del Nuovo Cim. 2020, 43, 515–566. [Google Scholar] [CrossRef]
  5. Li, J.; Xie, D.; Li, M.; Liu, S.; Wei, C.A. Pigment identification of ancient wall paintings based on a visible spectral Image. J. Spectrosc. 2020, 2020, 3695801. [Google Scholar] [CrossRef]
  6. Cucci, C.; Casini, A.; Picollo, M.; Poggesi, M.; Stefani, L. Open issues in hyperspectral imaging for diagnostics on paintings: When high-spectral and spatial resolution turns into data redundancy. In O3A: Optics for Arts, Architecture, and Archaeology III; SPIE: Munich, Germany, 2011; pp. 57–66. [Google Scholar]
  7. de Luna, J.M.; Fernandez-Balbuena, A.A.; Vázquez, D.; Melgosa, M.; Durán, H.; García, J.; Muro, C. Accurate measurements of spectral reflectance in Picasso’s Guernica painting. Appl. Spectrosc. 2016, 70, 147–155. [Google Scholar] [CrossRef] [PubMed]
  8. Sanmartín, P.; Chorro, E.; Vázquez-Nion, D.; Martínez-Verdú, F.M.; Prieto, B. Conversion of a digital camera into a non-contact colorimeter for use in stone cultural heritage: The application case to Spanish granites. Measurement 2014, 56, 194–202. [Google Scholar] [CrossRef]
  9. Álvarez Fernández-Balbuena, A.; Vázquez Moliní, D.; Gómez Manzanares, Á.; Martínez Antón, J.C.; Mayorga Pinilla, S. Reflectance Measurements on Cultural Heritage. In Heritage: New Paradigm; IntechOpen: London, UK, 2021. [Google Scholar]
  10. Montagner, C.; Linhares, J.M.; Vilarigues, M.; Melo, M.J.; Nascimento, S.M. Supporting history of art with colorimetry: The paintings of Amadeo de Souza-Cardoso. Color Res. Appl. 2018, 43, 304–310. [Google Scholar] [CrossRef]
  11. Picollo, M.; Cucci, C.; Casini, A.; Stefani, L. Hyper-spectral imaging technique in the cultural heritage field: New possible scenarios. Sensors 2020, 20, 2843. [Google Scholar] [CrossRef]
  12. Cosentino, A. Panoramic, macro and micro multispectral imaging: An affordable system for mapping pigments on artworks. J. Conserv. Mus. Stud. 2015, 13, 6. [Google Scholar] [CrossRef]
  13. Abd-Elrahman, A.; Pande-Chhetri, R.; Vallad, G. Design and development of a multi-purpose low-cost hyperspectral imaging system. Remote Sens. 2011, 3, 570–586. [Google Scholar] [CrossRef]
  14. Dian, R.; Li, S.; Sun, B.; Guo, A. Recent advances and new guidelines on hyperspectral and multispectral image fusion. Inf. Fusion 2021, 69, 40–51. [Google Scholar] [CrossRef]
  15. Chang, C.-I. Hyperspectral Data Processing: Algorithm Design and Analysis; John Wiley & Sons: Hoboken, NJ, USA, 2013. [Google Scholar]
  16. Tao, L.; Mughees, A. Deep Learning for Hyperspectral Image Analysis and Classification; Springer: Berlin/Heidelberg, Germany, 2021; Volume 5. [Google Scholar]
  17. Zhang, Y.; He, M. Multi-spectral and hyperspectral image fusion using 3-D wavelet transform. J. Electron. 2007, 24, 218–224. [Google Scholar] [CrossRef]
  18. Akhtar, N.; Shafait, F.; Mian, A. Bayesian sparse representation for hyperspectral image super resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 3631–3640. [Google Scholar]
  19. Li, J.; Wan, X. Superpixel segmentation and pigment identification of colored relics based on visible spectral image. Spectrochim. Acta Part A Mol. Biomol. Spectrosc. 2018, 189, 275–281. [Google Scholar] [CrossRef] [PubMed]
  20. Zhang, Y.; Zhang, Z.; DiVerdi, S.; Wang, Z.; Echevarria, J.; Fu, Y. Texture hallucination for large-factor painting super-resolution. In Proceedings of the European Conference on Computer Vision, Glasgow, UK, 23–28 August 2020; pp. 209–225. [Google Scholar]
  21. Cabezos-Bernal, P.M.; Rodriguez-Navarro, P.; Gil-Piqueras, T. Documenting Paintings with Gigapixel Photography. J. Imaging 2021, 7, 156. [Google Scholar] [CrossRef] [PubMed]
  22. Rosado, T.; Falé, A.; Gil, M.; Mirão, J.; Candeias, A.; Caldeira, A.T. Understanding the influence of microbial contamination on colour alteration of pigments used in wall paintings—The case of red and yellow ochres and ultramarine blue. Color Res. Appl. 2019, 44, 783–789. [Google Scholar] [CrossRef]
  23. Huang, S.-Y.; Mukundan, A.; Tsao, Y.-M.; Kim, Y.; Lin, F.-C.; Wang, H.-C. Recent Advances in Counterfeit Art, Document, Photo, Hologram, and Currency Detection Using Hyperspectral Imaging. Sensors 2022, 22, 7308. [Google Scholar] [CrossRef]
  24. Grabowski, B.; Masarczyk, W.; Głomb, P.; Mendys, A. Automatic pigment identification from hyperspectral data. J. Cult. Herit. 2018, 31, 1–12. [Google Scholar] [CrossRef]
  25. Chen, A.; Jesus, R.; Vilarigues, M. Identification and Visualization of Pure and Mixed Paint Pigments in Heritage Artwork Using Machine Learning Algorithms. SN Comput. Sci. 2022, 4, 115. [Google Scholar] [CrossRef]
  26. Deborah, H.; George, S.; Hardeberg, J.Y. Spectral-divergence based pigment discrimination and mapping: A case study on The Scream (1893) by Edvard Munch. J. Am. Inst. Conserv. 2019, 58, 90–107. [Google Scholar] [CrossRef]
  27. Gómez Manzanares, Á.; Vázquez Moliní, D.; Alvarez Fernandez-Balbuena, A.; Mayorga Pinilla, S.; Martínez Antón, J.C. Measuring High Dynamic Range Spectral Reflectance of Artworks through an Image Capture Matrix Hyperspectral Camera. Sensors 2022, 22, 4664. [Google Scholar] [CrossRef]
  28. CIE. Colorimetry CIE 15: 2004; CIE: Vienna, Austria, 2004. [Google Scholar]
  29. Fairchild, M.D. Color Appearance Models; John Wiley & Sons: Hoboken, NJ, USA, 2013. [Google Scholar]
  30. Gómez Manzanares, Á.; Benítez, A.J.; Martínez Antón, J.C. Virtual Restoration and Visualization Changes through Light: A Review. Heritage 2020, 3, 1373–1384. [Google Scholar] [CrossRef]
  31. Brauers, J.; Schulte, N.; Bell, A.A.; Aach, T. Multispectral high dynamic range imaging. In Proceedings of the Color Imaging XIII: Processing, Hardcopy, and Applications, San Jose, CA, USA, 28 January 2008; pp. 26–37. [Google Scholar]
  32. Martínez, M.Á.; Valero, E.M.; Nieves, J.L.; Blanc, R.; Manzano, E.; Vílchez, J.L. Multifocus HDR VIS/NIR hyperspectral imaging and its application to works of art. Opt. Express 2019, 27, 11323–11338. [Google Scholar] [CrossRef] [PubMed]
  33. Lapray, P.-J.; Thomas, J.-B.; Gouton, P. High dynamic range spectral imaging pipeline for multispectral filter array cameras. Sensors 2017, 17, 1281. [Google Scholar] [CrossRef] [PubMed]
  34. Schanda, J. Colorimetry: Understanding the CIE System; John Wiley & Sons: Hoboken, NJ, USA, 2007. [Google Scholar]
Figure 1. (a) High-spatial-resolution RGB image of Dalí’s painting Two Figures (1926), provided by the Museo Nacional Centro de Arte Reina Sofía; (b) enlarged section of an area of the painting; (c) enlargement of one of the areas affected by the reddish pigment (image dimensions 6.8 × 4.9 cm).
Figure 1. (a) High-spatial-resolution RGB image of Dalí’s painting Two Figures (1926), provided by the Museo Nacional Centro de Arte Reina Sofía; (b) enlarged section of an area of the painting; (c) enlargement of one of the areas affected by the reddish pigment (image dimensions 6.8 × 4.9 cm).
Sensors 23 04316 g001
Figure 2. (a): Front of Hypermatrixcam; (b): transmittance curves for the spectral filters, ordered from left to right as 470, 490, 510, 530, 550, 570, 590, 610, 630, 650, 670, and 690 nm.
Figure 2. (a): Front of Hypermatrixcam; (b): transmittance curves for the spectral filters, ordered from left to right as 470, 490, 510, 530, 550, 570, 590, 610, 630, 650, 670, and 690 nm.
Sensors 23 04316 g002
Figure 3. Geometric arrangement and distance used during the spectral reflectance measurements of Dalí’s painting at the Museo Nacional Centro de Arte Reina Sofía.
Figure 3. Geometric arrangement and distance used during the spectral reflectance measurements of Dalí’s painting at the Museo Nacional Centro de Arte Reina Sofía.
Sensors 23 04316 g003
Figure 4. Experimental response curve for the IMX477 sensor.
Figure 4. Experimental response curve for the IMX477 sensor.
Sensors 23 04316 g004
Figure 5. LDR multispectral image of Two Figures (1926) at 490 nm, with underexposed (yellow) and overexposed (pink) pixels.
Figure 5. LDR multispectral image of Two Figures (1926) at 490 nm, with underexposed (yellow) and overexposed (pink) pixels.
Sensors 23 04316 g005
Figure 6. HDR spectral reflectance obtained from Dalí’s painting Two Figures (1926) in the spectral range 470–690 nm in steps of 20 nm.
Figure 6. HDR spectral reflectance obtained from Dalí’s painting Two Figures (1926) in the spectral range 470–690 nm in steps of 20 nm.
Sensors 23 04316 g006
Figure 7. Images of Reference Area 1 used for analysis: (a) high-spatial-resolution RGB image; (b) image with 4K HDR spectral reflectance information at 630 nm; and (c) RGB image obtained from HDR spectral reflectance measurements.
Figure 7. Images of Reference Area 1 used for analysis: (a) high-spatial-resolution RGB image; (b) image with 4K HDR spectral reflectance information at 630 nm; and (c) RGB image obtained from HDR spectral reflectance measurements.
Sensors 23 04316 g007
Figure 8. Graphical representation of the CIELab a*b* values for the reddish pigment and the black pigment in Reference Area 1.
Figure 8. Graphical representation of the CIELab a*b* values for the reddish pigment and the black pigment in Reference Area 1.
Sensors 23 04316 g008
Figure 9. (a) Black area of the image selected for analysis; (b a * b * b l a c k  values for this area, bounded by an envelope.
Figure 9. (a) Black area of the image selected for analysis; (b a * b * b l a c k  values for this area, bounded by an envelope.
Sensors 23 04316 g009
Figure 10. (a) Graphical representation of the bounded regions of the values  a * b * b l a c k  and the values  a * n , m ,   b * n , m ; (b) graphical representation of the bounded region of the values  a * b * r e d d i s h _ 1 .
Figure 10. (a) Graphical representation of the bounded regions of the values  a * b * b l a c k  and the values  a * n , m ,   b * n , m ; (b) graphical representation of the bounded region of the values  a * b * r e d d i s h _ 1 .
Sensors 23 04316 g010
Figure 11. (a) RGB image segmentation area; (b) reflectance image segmentation area at 630 nm.
Figure 11. (a) RGB image segmentation area; (b) reflectance image segmentation area at 630 nm.
Sensors 23 04316 g011
Figure 12. Segmentation of the reddish pigment in the segmentation area from the  a * b * r e d d i s h _ 1  values (reddish pixels represented in false red).
Figure 12. Segmentation of the reddish pigment in the segmentation area from the  a * b * r e d d i s h _ 1  values (reddish pixels represented in false red).
Sensors 23 04316 g012
Figure 13. Image of Reference Area 2 with HDR spectral reflectance information at 630 nm.
Figure 13. Image of Reference Area 2 with HDR spectral reflectance information at 630 nm.
Sensors 23 04316 g013
Figure 14. Graphical representation of bounded regions with the values  a * b * b l a c k  and with the values  a * b * r e d d i s h _ 2 .
Figure 14. Graphical representation of bounded regions with the values  a * b * b l a c k  and with the values  a * b * r e d d i s h _ 2 .
Sensors 23 04316 g014
Figure 15. Segmentation of the reddish pigment in the segmentation area from the  a * b * r e d d i s h _ 2  values (reddish pixels represented in false red).
Figure 15. Segmentation of the reddish pigment in the segmentation area from the  a * b * r e d d i s h _ 2  values (reddish pixels represented in false red).
Sensors 23 04316 g015
Figure 16. RGB patch obtained through the mean value of the spectral reflectance of the pixels included in the segmentation using (a) method 1 and (b) method 2.
Figure 16. RGB patch obtained through the mean value of the spectral reflectance of the pixels included in the segmentation using (a) method 1 and (b) method 2.
Sensors 23 04316 g016
Table 1. Results obtained with Methods 1 and 2.
Table 1. Results obtained with Methods 1 and 2.
Segmentation AreaAmount of the Segmentation Area Affected by Reddish Pigment
Method 1Method 2
PixelsArea (cm2)PixelsArea (cm2)Percentage of Segmentation AreaPixelsArea (cm2)Percentage of Segmentation Area
23,82982.98761126.4932%330411.513.87%
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Fernandez-Balbuena, A.A.; Gómez-Manzanares, A.; Martínez Antón, J.C.; Gómez-Tejedor, J.G.; Mayorga-Pinilla, S.; Durán Roque, H.; Vázquez Moliní, D. High-Dynamic-Range Spectral Reflectance for the Segmentation of Paint Pigment: Application to Dalí’s Oil Painting Dos Figuras (1926). Sensors 2023, 23, 4316. https://doi.org/10.3390/s23094316

AMA Style

Fernandez-Balbuena AA, Gómez-Manzanares A, Martínez Antón JC, Gómez-Tejedor JG, Mayorga-Pinilla S, Durán Roque H, Vázquez Moliní D. High-Dynamic-Range Spectral Reflectance for the Segmentation of Paint Pigment: Application to Dalí’s Oil Painting Dos Figuras (1926). Sensors. 2023; 23(9):4316. https://doi.org/10.3390/s23094316

Chicago/Turabian Style

Fernandez-Balbuena, Antonio Alvarez, Angela Gómez-Manzanares, Juan Carlos Martínez Antón, Jorge García Gómez-Tejedor, Santiago Mayorga-Pinilla, Humberto Durán Roque, and Daniel Vázquez Moliní. 2023. "High-Dynamic-Range Spectral Reflectance for the Segmentation of Paint Pigment: Application to Dalí’s Oil Painting Dos Figuras (1926)" Sensors 23, no. 9: 4316. https://doi.org/10.3390/s23094316

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop