Next Article in Journal
Peristaltic Propulsion of Jeffrey Nanofluid with Thermal Radiation and Chemical Reaction Effects
Previous Article in Journal
State of Charge Estimation of a Composite Lithium-Based Battery Model Based on an Improved Extended Kalman Filter Algorithm
 
 
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The CaliPhoto Method

1
CNRS, Centre de Biophysique Moléculaire, UPR4301, Avenue de la Recherche Scientifique, CS80054, 45071 Orléans CEDEX 2, France
2
CNRS, Laboratoire de l’Atmosphère et des Cyclones, UMR8105, Université de La Réunion, 15 Avenue René Cassin, CS 92003, 97744 Saint-Denis CEDEX 09, France
3
CNRS, Conditions Extrêmes et Matériaux: Haute Température et Irradiation, UPR3079, Avenue de la Recherche Scientifique, CS90055, 45071 Orléans CEDEX 2, France
4
Université d’Orléans, Château de la Source, Avenue du Parc Floral, BP 6749, 45067 Orléans Cedex 2, France
5
Institut Denis Poisson, Université d’Orléans, Collegium Sciences et Techniques, Bâtiment de mathématiques, Rue de Chartres, B.P. 6759, 45067 Orléans CEDEX 2, France
6
Société d’Accélération du Transfert de Technologies, 8, Rue Pablo Picasso, 63000 Clermont-Ferrand, France
7
Institut des Sciences de la Terre d’Orléans, 1A Rue de la Férollerie, CS 20066, 45071 Orléans CEDEX 2, France
*
Author to whom correspondence should be addressed.
Inventions 2019, 4(4), 67; https://doi.org/10.3390/inventions4040067
Received: 24 September 2019 / Revised: 3 November 2019 / Accepted: 5 November 2019 / Published: 8 November 2019

Abstract

:
We propose an innovative method based on photography and image processing of interdisciplinary relevance, permitting the uncomplicated and inexpensive evaluation of material properties. This method—CaliPhoto—consists of using a dedicated colour plate with a specific design, placed in the field of view of a photograph of the material to be characterized. A specific image processing workflow is then applied to obtain colour vectors independent of illumination conditions. The method works using commercial colour cameras (e.g., smartphone cameras), and the colour plate can be printed on any colour printer. Herein, we describe the principle of the method and demonstrate that it can be used to describe and compare samples, identify materials or make relatively precise concentration measurements. The CaliPhoto method is highly complementary to any scientific research and may find applications across a range of domains, from planetary science to oceanography. The method may also be widely used in industry.

1. Introduction

Colour is the primary parameter used in many domains to identify and characterize a material. It is used for quality control in such industries as food [1], building [2,3] or textiles [4], amongst others. For scientific purposes, colour observation is a rapid method to estimate the composition of a material, as in the geosciences (Munsell soil colour charts [5]) or in biology (e.g., dermatology [6]), to monitor changes in physical properties (e.g., rock ageing [7,8]), or to obtain specific physical properties (e.g., temperature or pH). Nevertheless, the apparent colour of material is directly dependant on light conditions and photon detectors; the human eye is limited and person-dependant (e.g., colour blindness), and camera detectors have different colour responses. Precise colour measurement is thus made using UV-Vis spectroscopy or, if based on photography, requires calibrated cameras and the use of specific light sources (CIE Standard Illuminants D50 or D65 for example) [9]. Thus, despite its ubiquitous utility for material characterization, colour is rarely used to evaluate sample properties in scientific studies.
Here, we describe a new method for material characterisation, named CaliPhoto (patent n° FR3042058), which uses colour as acquired by any commercially available camera (e.g., smartphone, tablet, webcam, compact camera, etc.) and is possible with an uncalibrated light source (e.g., the Sun, halogen lamp, LED, etc.). Cameras are probably the most common system of observation, and most scientists use photography. In addition to a camera, the method simply requires the use of a specific reference colour plate, which can be printed on any colour printer and placed in the same field of view as the material of interest. We have developed an image processing methodology, using the RGB parameters of the reference colour plate (8 bit encoding, meaning that R, G and B may vary between 0 and 255), which can be used to obtain RGB values independent of both the camera and the light source for the material of interest. This method can be used to identify or to compare materials, and also to determine specific physical properties and follow their changes, in different light conditions. Due to its very low cost, portable nature (requiring only a sheet of paper) and the facility and rapidity of its implementation, the method is particularly relevant for field investigations. It permits the comparison of materials located in different places without the need for sampling and field instrumentation and is thus highly applicable for use in harsh environmental conditions or isolated areas such as mountains, cliffs, deserts, volcanoes, deep seas, caves, archaeological sites or extraterrestrial bodies. Indeed, the method was originally developed for space exploration purposes, to aid in the identification of rocks during in situ exploration where instrumentation is generally limited for technical reasons. It is also a good companion for laboratory investigations, enabling the precharacterisation and selection of the samples best suited to be analysed in detail using potentially slow and expensive methods, or to rapidly monitor changes in material properties.
This paper explains the CaliPhoto protocol and is applicable to a wide range of scientific studies. The main part of this paper is thus voluntarily short. Nevertheless, the detailed protocol and different validation tests are given in the Appendices so that any scientist can take advantage of its use. The algorithm (encoded in MATLAB) is available online in the Supplementary Materials of this article and may be used freely for any noncommercial purpose.

2. Materials and Methods

The principle of the CaliPhoto method comprises several steps:

2.1. Printing the CaliPhoto Colour Plate

Since they cover a wide and relevant range of colours, the RGB values of the colour squares of the CaliPhoto colour plate used in this study are similar to those of the GretagMacBeth ColorChecker typically used for colour calibration (Figure 1) [9,10]. It is expensive and, moreover, quite impossible to print a colour plate with the exact theoretical colour values displayed in Figure 1. Nevertheless, contrary to colorimetry methods, the aim of the CaliPhoto method is not to obtain true colours, as for calibrating a camera or a printer, nor to obtain colours representative of human vision. The CaliPhoto colour plate (Figure A1) can thus be printed using any commercial printer and paper. The size of the colour plate, ranging from a few cm to several tens of cm, is chosen according to the material of interest and the imaging system used (see Appendix A for details). Note that the colour palette used in Figure 1 has been chosen only as an example and can be changed to be more relevant to the colour chart of the studied materials. Similarly, their number can be increased or decreased if required (and the code given in Supplementary Materials will need to be modified accordingly).

2.2. Make a Reference Image of the Printed CaliPhoto Colour Plate

The processing required to obtain the reference image of the colour plate is explained in Figure 2. Since the colour plate has been printed using a noncalibrated printer, it is necessary to make a reference image of that colour plate taking into accounts its specific colours. The CaliPhoto method requires neither high-quality colour printing nor a specific light source or camera. The reference image is obtained by simply acquiring a photograph of the printed colour plate (Figure 2a). The relatively unrestricted recommendations for the acquisition of this photograph are detailed in Appendix B. The image is then rescaled and resized in order to remove perspective distortion and to obtain an image with a width/height ratio similar to that of the numerical model (i.e., a 4:3 ratio rectangle without the logo band). The resolution of the images may also be fixed; here we used a resolution of 1200 pixels × 1020 pixels including the logo band (Figure 2b). This can easily be made using image processing software, such as GIMP (see Appendix B, Figure A3 for details). Since it is difficult to control the orientation and homogeneity of the light source, the external and internal white rectangles of the colour plate (see Figure 1) are used to evaluate and correct the inhomogeneity of the illumination. This approach is inspired by the standard “plane fitting” data processing used in Scanning Probe Microscopy in order to remove the slope induced by the nonparallelism between the sample surface and the scanning plane (e.g., [11]). Indeed, in the absence of shadow, the light orientation will form a brightness continuum over the colour plate that can be fitted using a polynomial surface (see details in Appendix B, Figure A4). The equation of this plane can then be used in order to remove the illumination inhomogeneity over the whole image. In order to avoid values higher than 255, the average R, G and B values of the white rectangles are reset to 230 (Figure 2c). The fourth step consists of setting the brightness and contrast of the image. This is done by using a polynomial regression of order 3, linking the R, G and B values measured on the previously corrected image to the theoretical values displayed in Figure 1 (Figure 2d). These two steps also compensate the chromaticity of the light source (i.e., after correction, R ≈ G ≈ B in the greyscale squares). After correction, the colours of the squares in the reference image (Figure 2d) are different from the theoretical values (Figure 1).

2.3. Making Comparable Images of the Materials

The aim of the CaliPhoto method is to compare different materials based on their colours when observed in different light conditions. The idea is thus to process the images of the materials using the colour plate in order to obtain images where all the colour squares have similar RGB values to those of the reference image (Figure 2d). The image of a material is obtained by placing the sample below the window of the same printed colour plate as used previously. It is important that the colour plate is as close as possible to, and ideally in contact with, the surface of the material of interest. This is because the light conditions must be the same on both the colour plate and the sample. Image processing is then similar to that used in order to obtain the reference image of the colour plate (Figure 2). Following this, a complementary step is applied to correct the colours of these new images, resetting the RGB values of the colour squares of the colour plate to those of the reference image using a polynomial transform similar to those explained in [6,12] (Figure 3). We have demonstrated that a polynomial regression of order 11 yields the best correction (see Appendix C).
After correction, the RGB values of the colour squares in the image of the sample must be similar to those on the reference image of the colour plate; however, the correction can never be perfect. The accuracy of the image processing has been tested under different illumination conditions, using different imaging systems and for different image resolutions (see Appendix C and Appendix D). In order to evaluate the colour correction of the images using the CaliPhoto method, the final RGB values of the colour squares on the final images of the samples are compared to those of the reference image of the colour plate using Δ, defined as:
Δ = 1 24 i = 1 i = 24 ( R i r e f R i i m ) 2 + ( G i r e f G i i m ) 2 + ( B i r e f B i i m ) 2 255 2 + 255 2 + 255 2
where Riref, Giref and Biref are the average RGB values of the square i in the reference image of the colour plate and Riim, Giim and Biim are the average RGB values of the corresponding square in the corrected image.
The detailed results of the tests are reported in Appendix C and Appendix D; however, one can conclude that:
  • The best correction is obtained when the size of the colour plate in pixels on the image is close to that of the colour plate on the reference image before resizing.
  • The correction of an image with respect to the reference image remains good, even in low-light environments. It is shown that Δ < 3% when the luminance µ = (R + G + B)/3 of the white square in the sample image is higher than approximately 100, and when the image dynamic (i.e., µ of white square minus µ of black square) is higher than approximately 100.
  • The “colour” of the light does not play a role in the final correction; for a difference of 54 between the highest and the lowest of the three RGB values of the white square, Δ = 1%.
  • It is better to use the same camera throughout the whole procedure, even if Δ < 2% when using different imaging devices.
Finally, the tests show that in “normal” conditions, the method allows the user to obtain a dataset of images in which the difference between the RGB values of colour squares in the colour plate (between two images) is always approximately Δ ≈ 1%. In other words, this means that the percentage of uncertainty in the CaliPhoto colour of a material (i.e., its colour plate-dependent colour) is only about 1%.

2.4. Comparison of Materials from the Colours Given by CaliPhoto

For samples that are relatively homogeneous in colour, materials can be compared using their average CaliPhoto RGB values in the following equation:
Δ i j = ( R i R j ) 2 + ( G i G j ) 2 + ( B i B j ) 2 255 2 + 255 2 + 255 2
where Ri, Gi, Bi and Rj, Gj, Bj are the average CaliPhoto RGB values of the sample i and j respectively. Nevertheless, depending on the materials of interest, more complex vectors may be defined (see Appendix E). In particular, we propose to use the histograms of the rg chromaticity (r,g,b) and luminance (µ) parameters and to merge them into a unique colour vector Vec, which is defined for material i as:
V e c i = H i s t ( r i ) [ 1 + H i s t ( g i ) ] [ 2 + H i s t ( b i ) ] [ 3 + H i s t ( µ i / 256 ) ] M a x { r i , g i , b i , ( µ i 256 ) }
where Hist(ri) corresponds to the histogram of ri, rounded to 0.05, over the sample area, [1 + Hist(gi)] to the histogram of gi, rounded to 0.05, displayed between 1 and 2, [2 + Hist(bi)] to the histogram of bi, rounded to 0.05, displayed between 2 and 3, and [1 + Hist(µi/256)] to the histogram of µi/256, rounded to 0.05, displayed between 3 and 4, and Max {ri, gi, bi, (µi/256)} to the maximal values of the different histograms. This simple method allows the inhomogeneity of the colour of a material to be easily taken into account while avoiding complex textural image processing. Finally, for samples that are relatively homogeneous in colour, materials 1 and 2 can be compared based on their colour vector using the following equation:
Δ V e c 12 = ( V e c 1 V e c 2 ) 2 V e c 1 + V e c 2

3. Results

The CaliPhoto method permits the comparison of materials from images taken under different light conditions. The applications of this approach are thus very wide and numerous. For instance, in the field, it can be used to measure compositional variations across a rock outcrop or to compare the different sides of an object, from a hand sample to a geological structure or a building, where the shadows and illumination necessarily vary between the different snapshots (change of illumination conditions with time and space) (Figure 4).
In addition to allowing the comparison of different materials, this method is also relevant for monitoring physical processes associated with colour change, such as alteration or chemical reactions (oxidation, weathering, irradiation, water content, etc.).
Furthermore, by creating a database of CaliPhoto colour vectors (or RGB values) the method can be used to perform preliminary characterisation of an “unknown” material based on analyses of comparable materials using laboratory-based instrumentation. The philosophy is then to link properties obtained using expensive and/or nonportable instrumentation to measurements made by low-cost, highly portable imaging. The principle consists of characterising the materials expected to be found in the field, using laboratory instruments if needed, to make their CaliPhoto image following the process previously described, then to store their RGB CaliPhoto vectors in a database linking these vectors to the different materials or to any other relevant physical parameter(s). By comparison with the database, it would then be possible to identify or determine the physical properties of an unknown material using only a photograph. The method is resilient: a database can be made and completed at any time and, thus, the characterisation of “unknown” materials is continuously improved and refined with time. Figure 5 illustrates the use of the CaliPhoto method for sample identification and for compositional measurement (detailed results can be found in Appendix E and Appendix F).
Finally, the only restriction of the method is the fact that it is necessary to associate a database to a specific colour plate. If this colour plate were to be lost or altered, the database would become obsolete. We thus propose to maintain a Master Colour plate in a secure place so that the reference image of a new colour plate can be recalibrated for use with a previously established database (see Appendix G).

4. Discussion

The rationale of the CaliPhoto method is not to provide extremely precise measurements, but to rapidly and easily give preliminary results using a very versatile and uncomplicated, low-cost, portable system. It permits the evaluation of properties that would normally need to be studied using expensive, time-consuming, high-tech methodologies or instrumentation. It can be used as a rapid monitoring and precharacterization method in the field or in the laboratory and it may, for instance, help in identifying the most appropriate samples for further in-depth analysis. The CaliPhoto method thus has applications in various domains:
  • Space exploration: As mentioned above, instrumentation used during in situ space exploration is generally limited for technical reasons (limited mass, volume, energy and computational power). On the other hand, all space probes have on board a camera. Integrating a CaliPhoto colour plate with imaging systems on future space probes would be simple since it easily complies with the imposed technical limitations and does not require the development of new instrumentation. It could be particularly relevant for geological and astrobiological investigations, permitting the comparison of images of rocks taken in different places or evaluation of lithologies using images of rocks taken in the laboratory, for example.
  • Construction: The method may find different applications in this domain, for example, to check the homogeneity of materials over large areas or on the different faces of a structure, to monitor the ageing of materials over space and time, or to control the water content of cement.
  • Geology: As for space exploration, the method allows easy comparison of samples observed in different places in the field. When associated with a database, it can also facilitate the use of soil colour charts.
  • Biology: Colour is used as a parameter to pre-identify microbial colonies or to evaluate their reaction to external stresses (sunlight, dehydration, pH, salinity, etc.). This applies more generally for any living system (the yellowing of leaves, for example). The CaliPhoto may thus be used as a noninvasive and rapid method of monitoring such types of samples.
  • Materials science: Changes in the physico-chemical characteristics of materials may be associated with a change in colour. The possibility to monitor such changes during specific experiments may be particularly useful in determining reaction kinetics. Association with a database linking specific properties to CaliPhoto colours may circumvent or limit the systematic use of sophisticated instrumentation. It may help to refine measurements based on colour, such as those obtained using pH paper.
  • Submarine investigation: Underwater imaging is generally affected by water colour and inherently limited control of the light source. Submarine instrumentation is also limited compared with laboratory instrumentation. The CaliPhoto method may thus also be particularly useful in this setting. By linking physical properties obtained by laboratory analysis to a CaliPhoto database, it could be possible to estimate these properties from images made underwater, without sampling.
  • Cosmetics and dermatology: The CaliPhoto method may assist in the selection of makeup adapted to specific types of skin. It may also provide a control for Sun exposure to prevent sunburn. In the latter case, the idea would be to take pictures regularly of the same part of the body to monitor any change in colour.
Several studies on various applications of the CaliPhoto method are in progress in field geology and space exploration. In particular, the method will be adapted for the upcoming European Space Agency (ESA)/Roscosmos ExoMars 2020 mission to Mars to assist in the identification of rocks using the Close-Up Imager (CLUPI) camera [13].
While the CaliPhoto method has many applications for scientific purposes, it could also be used by the general public, for which an app for smartphones and tablets is under development. This implies automation of the method since manual image processing and MATLAB code cannot be used. Significant work has thus been carried out to integrate the automatic detection of the colour plate and of the sample area, and for automated comparison with databases. Finally, prototypes of apps and specific colour plates are currently under development for cosmetics, paints and decoration, for example, to identify complementary colours of textiles (N.B. in aesthetic cases, the matching percentage uses the Lab colour system to take into account the higher sensitivity of the human eye to green). The method could also be particularly relevant for assisting the colour-blind.
Finally, it is important to note that the CaliPhoto method is in line with the long-term vision for future developments in scientific technologies that favour low-cost solutions prior to methods that consume large amounts of resources. The philosophy of this approach is now echoed in many domains concerned by environmental considerations and future limitations in resources.

5. Patents

A patent has been deposited for this method under the number FR3042058 (A1).

Supplementary Materials

The following are available online at https://www.mdpi.com/2411-5134/4/4/67/s1. Code S1: MATLAB code; Code S2: MATLAB code; Code S3: MATLAB code.

Author Contributions

Conceptualization, F.F.; methodology, F.F., G.G., N.B. and A.C.; software, F.F., L.L., A.C. and A.S.; formal analysis, F.F., G.G., N.B., L.L., A.C. and A.S.; investigation, F.F., G.G., N.B., K.H.-L. and E.M.; writing—original draft preparation, F.F.; writing—review and editing, all authors; supervision, F.F., G.G., N.B. and M.B.; funding acquisition, F.F. and F.W.

Funding

This research was the Centre National de la Recherche Scientifique (CNRS), the Société Accélératrice de Transfert de Technologies (SATT), and the Centre National d’Etudes Spatiales (CNES).

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Print the CaliPhoto Colour Plate

The ready-to-print version of the CaliPhoto colour plate is given in Figure A1.
Figure A1. The ready-to-print CaliPhoto colour plate. The colour plate consists of a white 4:3 ratio rectangle (without the CaliPhoto logo line), surrounded by 24 squares of different colours arranged in a rectangle, surrounding another white rectangle. The centre of the printed colour plate must be cut out and removed in order to obtain the rectangular window in which the sample will be observed.
Figure A1. The ready-to-print CaliPhoto colour plate. The colour plate consists of a white 4:3 ratio rectangle (without the CaliPhoto logo line), surrounded by 24 squares of different colours arranged in a rectangle, surrounding another white rectangle. The centre of the printed colour plate must be cut out and removed in order to obtain the rectangular window in which the sample will be observed.
Inventions 04 00067 g0a1
Choosing the colour plate dimensions is important and depends on the studied materials. For the method to work with high accuracy, the illumination conditions must be the same on the colour plate and on the material in the window. Depending on the studied materials, the size of the printed colour plate must be reduced such that the topography of the sample surface is minimal within the window. Nevertheless, even if a small colour plate decreases the shadows induced by surface roughness, the sample area appearing in the centre of the colour plate must be large enough to be representative of the material inhomogeneity (colour motif). In addition, it is important to take into account the optics of the imaging system. For very small colour plates (smaller than approximately 2 cm × 1.5 cm with a standard camera), the camera would potentially fail to take a sharp, high-resolution image of the colour plate (out of focus, limited depth of field, etc.). On the other hand, for very large colour plates (larger than approximately >32 cm × 24 cm) the risk of inhomogeneous shadows that are impossible to correct for is increased. Overall, the colour plate must be printed in order to maximise the number of pixels in the photographs taken that are relevant for the objective of the study.
The better the quality of the printer, the better the colours of the corrected images will be. Nevertheless, since CaliPhoto is not a colorimetric method, an “average” quality of printing will not hamper the efficiency of the method because the colour of the squares in the reference image will be obtained from a photograph of the printed colour plate.
Finally, note that the practical manipulation of the colour plate is greatly eased if a part of the paper sheet is left around the colour plate. Using high-grammage paper or card circumvents outward curling of the paper that may result in deformations difficult to correct in the photographs taken.

Appendix B. Making the Reference Image of the Colour Plate

Once printed, the colour plate must be photographed in order to obtain the reference image. It is best to use homogenous solar light illumination for this, and the field of view must be chosen to avoid shadows on the colour plate. Moreover, depending on the print finish, the colour plate may be relatively bright. In this case, it is important to find an orientation that avoids light reflections on the colour plate. Finally, it is strongly advised to use the same camera for this step and the rest of the study. Indeed, we have demonstrated that, after correction, the differences in RBG values are slightly higher when the cameras used are different for each step (see Figure A2).
Figure A2. Effects of using different cameras during the CaliPhoto image processing. (a) Reference image of the colour plate made using a photocopier (Ricoh MP C2504ex) and (b) image of the same colour plate made using a tablet camera (iPad) after correction (see details in Appendix B and Appendix C). The value of Δ given by Equation (1) (see main text) is 0.9%. When using the tablet camera throughout (i.e., to make the reference image of the colour plate, image in (c), and the image to correct) the value of Δ is decreased to 0.6%, as displayed in (d).
Figure A2. Effects of using different cameras during the CaliPhoto image processing. (a) Reference image of the colour plate made using a photocopier (Ricoh MP C2504ex) and (b) image of the same colour plate made using a tablet camera (iPad) after correction (see details in Appendix B and Appendix C). The value of Δ given by Equation (1) (see main text) is 0.9%. When using the tablet camera throughout (i.e., to make the reference image of the colour plate, image in (c), and the image to correct) the value of Δ is decreased to 0.6%, as displayed in (d).
Inventions 04 00067 g0a2
The photograph of the colour plate must then be processed in order to obtain the reference image. For this, the perspective must be corrected and the image resized. This can easily be done using image processing software. The example is given in Figure A3 uses GIMP.
Figure A3. Initial image processing using commercial software (GIMP 2.10). (a) Original photograph of the reference image of the colour plate in jpeg format, taken with a tablet camera (iPad), viewed in GIMP software. The area containing the colour plate is selected using the “scrap” tool, copied and pasted as a new image, as seen in (b). (c) Using the “perspective” tool, the corners of the black rectangle surrounding the colour plate are moved to fit with the corner of the image. The rescaled colour plate, now rectangular, is then copied and pasted as a new image. (d) The image is finally resized as a rectangle of dimensions 1200 px × 1020 px. (e) Image of the colour plate after rescaling and resizing.
Figure A3. Initial image processing using commercial software (GIMP 2.10). (a) Original photograph of the reference image of the colour plate in jpeg format, taken with a tablet camera (iPad), viewed in GIMP software. The area containing the colour plate is selected using the “scrap” tool, copied and pasted as a new image, as seen in (b). (c) Using the “perspective” tool, the corners of the black rectangle surrounding the colour plate are moved to fit with the corner of the image. The rescaled colour plate, now rectangular, is then copied and pasted as a new image. (d) The image is finally resized as a rectangle of dimensions 1200 px × 1020 px. (e) Image of the colour plate after rescaling and resizing.
Inventions 04 00067 g0a3
The final reference image of the colour plate is obtained after light, brightness and contrast corrections. This image processing, described in Figure A4, is performed using mathematical software (MATLAB Code S1), and is as follows:
(i).
The rescaled and resized image (Figure A4a) is imported in MATLAB.
(ii).
The RGB channels of the image are extracted and converted into double format (Figure A4b).
(iii).
Pixels corresponding to the white rectangles surrounding the colour squares and the window on each of the three images are selected (see Figure A4c).
(iv).
The light source creates a homogeneous variation in brightness over the colour plate that can be fitted, for each R, G, B channel, by a polynomial plane equation (of order 3 in the example shown) determined from the two white rectangles selected in (iii) (Figure A4c).
(v).
The principle then becomes similar to the data processing method called “plane fit”, used in Scanning Probe Microscopy to correct the slope of the sample surface [11], consisting of removing the calculated plane from the whole image. In the CaliPhoto method, the plane removal is made in order to set the average value of the white rectangles to R,G,B = 230,230,230 (Figure A4d). The R,G,B values are set to 230,230,230 instead of 243,243,242 as expected in the final image to avoid values potentially higher than 255 after brightness and contrast adjustments made in the following step. This step also corrects the chromaticity of the illumination source by resetting the white balance. If the colour print is of lower quality (e.g., the white square appearing a more yellow in the printed colour plate), this step may also lead to apparent colours that are unattractive to human expectations in the processed image. However, it is important to note that, unlike colorimetric methods, this aspect of “unattractiveness” will not alter the accuracy of the method.
(vi).
The next step involves brightness and contrast correction. This is done by fitting the average value of the greyscale rectangles of the colour plate to their theoretical values, using a polynomial regression of order 3 (Figure A4e).
(vii).
Finally, it is possible to extract, as an Excel file for example, the average values of the colour squares of the reference image of the colour plate (Table A1).
Figure A4. Image processing of the reference image of the colour plate. (a), image of the colour plate after rescaling and resizing using image processing software (see Figure A3). (b) Extraction of R, G and B channels. (c) Selection of the external and internal white rectangles (in blue when selected in the software). (d) Plane fit of the white rectangles in the R channel. (e) Corrected image after plane fit removing. (f) Theoretical values of the greyscale rectangles versus of their average values in e and polynomial regression. (g) Final reference image of the colour plate.
Figure A4. Image processing of the reference image of the colour plate. (a), image of the colour plate after rescaling and resizing using image processing software (see Figure A3). (b) Extraction of R, G and B channels. (c) Selection of the external and internal white rectangles (in blue when selected in the software). (d) Plane fit of the white rectangles in the R channel. (e) Corrected image after plane fit removing. (f) Theoretical values of the greyscale rectangles versus of their average values in e and polynomial regression. (g) Final reference image of the colour plate.
Inventions 04 00067 g0a4
Table A1. Theoretical and reference RGB values of the colour squares of the colour plate. As expected, the reference RGB values of the colour squares are relatively far from the theoretical ones, the photograph of the colour plate having been taken with noncalibrated light and cameras, and colour plate having been printed with noncalibrated printer. These differences have no influence on the CaliPhoto method.
Table A1. Theoretical and reference RGB values of the colour squares of the colour plate. As expected, the reference RGB values of the colour squares are relatively far from the theoretical ones, the photograph of the colour plate having been taken with noncalibrated light and cameras, and colour plate having been printed with noncalibrated printer. These differences have no influence on the CaliPhoto method.
Theoretical ValuesReference Image of the Colour Plate
Colour Plate SquareRtheoGtheoBtheoRrefGrefBref
Dark skin11582681269382
Light skin194150130209156146
Blue sky98122157121133154
Foliage871086711011980
Blue flower133128177148131179
Bluish green103189170133190176
Orange2141264423712370
Purple red80911668682148
Moderate red193909920998100
Purple94601081076298
Cyan813316153141158
Magenta1878614918074128
Yellow2311993125519874
Red17554601817671
Green701487310615498
Blue56611506462127
Orange yellow2241634624916694
Yellow green15718864166191112
Black525252515353
Neutral 35858585868383
Neutral 5122122121124125124
Neutral 65160160160156158157
Neutral 8200200200204201201
White243243242241243242

Appendix C. Making the CaliPhoto Image of the Material

The conditions and the parameters used to take photographs of the studied materials will be described in detail in Appendix D. Nevertheless, the recommendations relevant for the photograph used to make the reference image in Appendix B also apply for these photographs.
The processing of the image for calibration with respect to the reference image is similar to those described in Appendix A and Appendix B, but adds a final image processing for colour correction (see main text Figure 3). The associated MATLAB is available in Supplementary Materials (Code S2). This step consists of evaluating the coefficients of the polynomial transformation using the least squares method and minimizing the differences between the RGB values of the colour squares on both the image for calibration and the reference image [6]. At order 3, it requires searching for the coefficients ai, bi, ci such that:
( R 1 G 1 B 1 R 24 G 24 B 24 ) I M · ( a 1 a 2 a 3 b 1 b 2 b 3 c 1 c 2 c 3 ) = ( R 1 G 1 B 1 R 24 G 24 B 24 ) R E F
where the IM matrix refers to the image to be corrected and the REF matrix refers to the reference image. At order 6, it consists of searching for the coefficients ai, bi, ci such that:
( R 1 G 1 B 1 R 1 G 1 R 1 B 1 G 1 B 1 R 24 G 24 B 24 R 24 G 24 R 24 B 24 G 24 B 24 ) I M · ( a 1 a 6 c 1 c 6 ) = ( R 1 G 1 B 1 R 24 G 24 B 24 ) R E F
and et cetera with increasing order.
In our case, this equality is never reached, which is why the coefficients ai, bi, ci are obtained from a least squares solution to the system. The coefficients ai, bi, ci allow correction of the RGB channels of the sample image obtained after step vi) in Appendix B using, at order 3, the following Equations (A3):
Rfinal = a1R + a2G + a3B
Gfinal = b1R + b2G + b3B
Bfinal = c1R + c2G + c3B
or at order 6, the Equations (A4):
Rfinal = a1R + a2G + a3B + a4R.G + a5G.B + a6R.B
Gfinal = b1R + b2G + b3B + b4R.G + b5G.B + b6R.B
Bfinal = c1R + c2G + c3B + c4R.G + c5G.B + c6R.B
and et cetera with increasing order. In our case, we used a polynomial transformation of order 11, meaning that the RGB channels are corrected using Equations (A5):
Rfinal = a1R + a2G + a3B + a4R.G + a5G.B + a6R.B + a7R.B + a8R2 + a9G2 + a10B2 + a11R.G.B
Gfinal = b1R + b2G + b3B + b4R.G + b5G.B + b6R.B + b7R.B + b8R2 + b9G2 + b10B2 + b11R.G.B
Bfinal = c1R + c2G + c3B + c4R.G + c5G.B + c6R.B + c7R.B + c8R2 + c9G2 + c10B2 + c11R.G.B
Several tests were carried out to evaluate the effect of polynomial order used on the colour correction process, examples of which are displayed in Figure A5. Here, it is shown that order 11 is always associated with the best correction, regardless of either the size of the colour plate on the image for correction and/or the illumination, as shown in Figure A6.
Figure A5. Effect of the order of colour correction on the accuracy of CaliPhoto image processing. (a) Original photograph of the colour plate used as reference. (b) The same image after reference image processing (see Appendix B). (c) Original photograph of the image to correct. (dh) Corrected images using different orders for colour correction, from order 3 to order 11. The value of Δ, given by Equation (1), is reported on each image. The best correction is obtained at order 11 (Δ = 0.61%). Photographs taken using a tablet camera (iPad).
Figure A5. Effect of the order of colour correction on the accuracy of CaliPhoto image processing. (a) Original photograph of the colour plate used as reference. (b) The same image after reference image processing (see Appendix B). (c) Original photograph of the image to correct. (dh) Corrected images using different orders for colour correction, from order 3 to order 11. The value of Δ, given by Equation (1), is reported on each image. The best correction is obtained at order 11 (Δ = 0.61%). Photographs taken using a tablet camera (iPad).
Inventions 04 00067 g0a5
Figure A6. Effect of the order of colour correction on the accuracy of CaliPhoto image processing for different illumination conditions, and original size of the colour plate in pixels. The size of the colour plate in pixels on the original photographs (on the left) are, from top to bottom: 2312 × 1944, 1704 × 1431, 1150 × 962, 633 × 541, 350 × 297, 285 × 237, 1120 × 964, 1132 × 968, 962 × 823, 870 × 736, 1455 × 1236. The reference image is displayed on the right (size of the colour plate on the original photograph, 1242 × 1072). The correction is always best at order 11.
Figure A6. Effect of the order of colour correction on the accuracy of CaliPhoto image processing for different illumination conditions, and original size of the colour plate in pixels. The size of the colour plate in pixels on the original photographs (on the left) are, from top to bottom: 2312 × 1944, 1704 × 1431, 1150 × 962, 633 × 541, 350 × 297, 285 × 237, 1120 × 964, 1132 × 968, 962 × 823, 870 × 736, 1455 × 1236. The reference image is displayed on the right (size of the colour plate on the original photograph, 1242 × 1072). The correction is always best at order 11.
Inventions 04 00067 g0a6
Finally, it is possible to extract, as an Excel file for example, the average values of the colour squares of the colour plate on the corrected image in order to compare them with those of the reference image (see Appendix B, §vii). Table A2 compares the RGB values of the colour squares before and after correction using the CaliPhoto method for the example displayed in Figure A6.
Table A2. RGB values of the colour square of the colour plate on the reference image in Figure A5b and on the initial and final image to correct in Figure A5c,h, respectively. The average value of Δ between the image to correct and the reference image is 13.78% before correction and only 0.61% after CaliPhoto correction.
Table A2. RGB values of the colour square of the colour plate on the reference image in Figure A5b and on the initial and final image to correct in Figure A5c,h, respectively. The average value of Δ between the image to correct and the reference image is 13.78% before correction and only 0.61% after CaliPhoto correction.
Reference Image of the Colour PlateInitial Image Before CorrectionΔ12(%)Final Image After CorrectionΔ12 (%)
Colour Plate SquareRrefGrefBrefRiniGiniBiniRfinalGfinalBfinal
Dark skin112755978563310.67%11175580.39%
Light skin2031601361591158618.22%2011601360.35%
Blue sky79124159568510915.39%761241590.81%
Foliage6696585470338.52%6697590.44%
Blue flower120134180819313216.80%1221361810.66%
Bluish green1071841597514010816.87%1031811581.11%
Orange229146621891073613.90%225146630.98%
Purple red4079150415810211.91%38811500.56%
Moderate red1988993154645414.46%19689930.53%
Purple6432815435497.86%6637841.34%
Cyan111013522798913.51%21111350.39%
Magenta18370139138509115.43%182681380.46%
Yellow232202651951633114.50%231202650.30%
Red1706058125463012.38%17461570.86%
Green6814369491013512.99%69143680.27%
Blue95013823368712.18%10471360.77%
Orange yellow229185691931423315.06%233185680.93%
Yellow green155189721091473716.25%155189740.31%
Black5453544341286.92%5251530.75%
Neutral 3582838158574511.48%8381810.46%
Neutral 512112112181827316.88%1221211210.29%
Neutral 6516416316311811911218.57%1681651640.94%
Neutral 819719819715415515217.10%1991981980.38%
White24424324221021121112.83%2422442420.29%
Δ(%)13.78% Δ(%)0.61%

Appendix D. Testing the CaliPhoto Image Processing

It is obviously ideal to make a good image of the sample with the colour plate, that is, preferentially using homogenous solar light illumination, avoiding over- and underexposure and minimising high perspective effects. In the case of the photograph used to make the reference image, close attention must be paid in order to avoid shadows and reflections on the colour plate by choosing the appropriate orientation. Consequently, the accuracy of the method can be improved by optimising the photography conditions and the parameters used to take the photographs of the studied materials.

Appendix D.1. Effect of Colour Plate Size in the Original Photographs on the CaliPhoto Method

As noted previously, it is better to use the same camera to make both the photographs of the study samples and the reference image of the colour plate (see Figure A2). Nevertheless, it appears that the size of the colour plate in pixels in the photographs prior to processing plays an even more important role than the camera used; if the sizes (in pixels) of the colour plates in the two photographs are close, the image processing is seen to be more efficient, as illustrated in Figure A7.
Figure A7. Initial resolution dependency for CaliPhoto image processing. (a) Reference image of the colour plate made using a photocopier (Ricoh MP C2504ex) and (b) image of the same colour plate made using a tablet camera (iPad), in order to obtain a similar size (in pixels) for the colour plate in the original photograph and in the photocopied image. The value of Δ, given by Equation (1) (see main text), is equal to 1.1%. When using the tablet to make the reference image of the colour plate (c) and the image to correct but with a resolution approximately two times higher, Δ = 1.2%, as displayed in (d). This shows that the correction process depends more on the difference of initial resolution of the colour plate than on the imaging systems used. (e) Graph showing Δ after correction versus the difference of the size of the colour plate in pixels between the image to correct and the reference image in % for a series of images. The best correction is obtained when the size of the colour plate on the image is close to the size of the colour plate on the reference image before resizing.
Figure A7. Initial resolution dependency for CaliPhoto image processing. (a) Reference image of the colour plate made using a photocopier (Ricoh MP C2504ex) and (b) image of the same colour plate made using a tablet camera (iPad), in order to obtain a similar size (in pixels) for the colour plate in the original photograph and in the photocopied image. The value of Δ, given by Equation (1) (see main text), is equal to 1.1%. When using the tablet to make the reference image of the colour plate (c) and the image to correct but with a resolution approximately two times higher, Δ = 1.2%, as displayed in (d). This shows that the correction process depends more on the difference of initial resolution of the colour plate than on the imaging systems used. (e) Graph showing Δ after correction versus the difference of the size of the colour plate in pixels between the image to correct and the reference image in % for a series of images. The best correction is obtained when the size of the colour plate on the image is close to the size of the colour plate on the reference image before resizing.
Inventions 04 00067 g0a7

Appendix D.2. Effect of Light Intensity on the CaliPhoto Method

The method will, of course, not work when the light conditions are too extreme, for example, in near or total darkness. To test the limits of validity of the method with respect to the light conditions (extrapolated from brightness and contrast parameters of the images), images were taken using different light intensities.
Thus, we defined the luminance of a colour square as:
µ = 1 3 ( R + G + B )
where R, G and B are the average values of R, G, B over a considered square within the colour plate. Then, we used the luminance of the white square of the colour plate to measure the original brightness of the photographs, and the difference between the luminance of the white and black squares to measure the original contrast of the photographs. The value of Δ versus these parameters is displayed in Figure A8. It is shown that the CaliPhoto processing works when the brightness and the contrast, as defined above, are higher than approximately 100; this corresponds to relatively dark and low-contrast images.
Figure A8. Influence of illumination conditions on CaliPhoto image processing. (a) Δ after CaliPhoto correction versus the luminance of the white square of the colour plate and (b) versus the difference between the luminance of the white and black squares on the original photographs (displayed at the top). The first image corresponds to the reference image to which Δ refers.
Figure A8. Influence of illumination conditions on CaliPhoto image processing. (a) Δ after CaliPhoto correction versus the luminance of the white square of the colour plate and (b) versus the difference between the luminance of the white and black squares on the original photographs (displayed at the top). The first image corresponds to the reference image to which Δ refers.
Inventions 04 00067 g0a8

Appendix D.3. Effect of Light “Colour” on the CaliPhoto Method

Similarly, we studied the effect of the hue of the photograph, induced by the colour of the light source and by the camera, on the image processing. Thus, we defined parameter C corresponding to the deviation in white balance observed on the white square as:
C = MAX(R,G,B) − MIN(R,G,B)
where MAX(R,G,B) and MIN(R,G,B) are, respectively, the maximum and minimum values of the three average R, G and B values of the white square of the colour plate on the original photographs. Thus, when illuminated with a blue light, the white square will appear blue (R,G,B = 200, 200, 250 for example) and the value of C will be positive (equal to 50 in the example). The value of Δ versus C is displayed in Figure A9. The colour of the light source does not seem to play a role in the CaliPhoto correction.
Figure A9. Hue dependency on the CaliPhoto image processing. Δ after CaliPhoto correction versus C, the variation in colour of the white square defined by Equation (A7). The “colour” of the original photograph does not seem to play a role in the accuracy of the CaliPhoto image processing.
Figure A9. Hue dependency on the CaliPhoto image processing. Δ after CaliPhoto correction versus C, the variation in colour of the white square defined by Equation (A7). The “colour” of the original photograph does not seem to play a role in the accuracy of the CaliPhoto image processing.
Inventions 04 00067 g0a9

Appendix D.4. Recommendations for Photographers and Printers

Finally, to summarize, in order to improve the accuracy of the CaliPhoto image correction processing, several key procedural points should be considered. In order of importance:
  • The luminance of the white square in the photographs must be higher than 100.
  • The contrast, obtained from the difference of luminance between the white and black squares in the photographs, must be higher than 100.
  • Do not use folded colour plates or colour plates that have curled outward.
  • Avoid shadows and reflections on the colour plate by choosing the appropriate orientation.
  • It is advisable to avoid high perspective effects.
  • It is better to make photographs using homogenous light illumination.
  • Similar size of the colour plate in pixels on the reference image and on the image to be corrected is advisable.
  • It is better if the same camera is used to make both the photograph of the reference image and that of the studied materials.
  • For reasons of aesthetics, it may be preferable to use a high-quality colour printer in order to obtain corrected images appearing pleasant to human eyes.

Appendix E. Material Identification Using the CaliPhoto Method

Appendix E.1. Identification using Average CaliPhoto Colour

The CaliPhoto method can be used for identifying materials from a database. For such use, well-characterised reference samples must be photographed, taking into account the recommendations listed in Appendix D.4, and corrected using the CaliPhoto image processing described in Appendix C. The average CaliPhoto colours of the materials are then extracted and stored in a database. An example is given for coloured paper (blue, green, mauve, orange, pink, purple, yellow) in Figure A10.
Figure A10. Constitution of a CaliPhoto database of coloured paper. Reference image used for the CaliPhoto image processing correction and corrected images of the reference samples. The areas corresponding to the samples are selected, and the average colours are calculated and stored in a database, as shown in the table.
Figure A10. Constitution of a CaliPhoto database of coloured paper. Reference image used for the CaliPhoto image processing correction and corrected images of the reference samples. The areas corresponding to the samples are selected, and the average colours are calculated and stored in a database, as shown in the table.
Inventions 04 00067 g0a10
The database may also include the uncertainty on the colours, given by 256 × Δ. Table A3 shows the values of Δij and Δ for each square and each of the reference sample images after correction. The database created, including uncertainties, is displayed in Table A4.
Table A3. Values of Δ12 for each colour square of the colour plate and values of Δ for the different images of the coloured paper, displayed in Figure A10. The average global variation is only 1.15%.
Table A3. Values of Δ12 for each colour square of the colour plate and values of Δ for the different images of the coloured paper, displayed in Figure A10. The average global variation is only 1.15%.
SampleBlueGreenMauveOrangePinkPurpleYellowAverage Δ12 (%)
Dark skin0.51%0.39%0.69%0.69%0.74%0.23%1.49%0.68%
Light skin1.51%1.76%2.07%2.22%1.10%1.65%1.21%1.65%
Blue sky1.47%1.56%1.84%2.47%0.38%1.23%0.11%1.29%
Foliage1.50%1.62%1.50%1.64%0.39%1.30%1.08%1.29%
Blue flower0.37%0.28%0.21%0.72%0.55%0.18%0.60%0.42%
Bluish green0.83%1.84%1.79%3.71%1.94%1.19%2.58%1.98%
Orange2.02%1.37%1.19%1.92%1.39%1.49%1.94%1.62%
Purple red0.55%0.73%1.15%0.41%0.32%0.29%1.19%0.66%
Moderate red1.01%1.71%1.74%1.53%1.26%1.73%2.23%1.60%
Purple2.76%2.52%2.18%1.46%1.03%2.42%2.38%2.11%
Cyan0.64%0.80%0.71%2.42%1.52%0.65%1.57%1.19%
Magenta0.77%0.82%0.65%1.25%1.09%0.80%1.37%0.96%
Yellow0.63%0.61%0.55%0.14%0.42%0.79%0.25%0.48%
Red0.37%1.50%1.65%1.38%0.97%1.12%2.01%1.28%
Green0.79%0.76%0.67%0.88%1.49%0.69%1.21%0.93%
Blue1.06%1.30%1.95%0.89%1.11%0.98%0.75%1.15%
Orange yellow0.80%0.21%0.26%0.86%0.57%0.21%0.63%0.51%
Yellow green1.55%1.23%0.80%1.55%1.56%1.27%1.59%1.36%
Black0.97%0.73%0.56%0.45%0.42%0.81%0.62%0.65%
Dark grey1.54%0.81%0.88%1.54%0.33%0.87%0.41%0.91%
Grey1.99%2.31%2.47%3.45%1.19%2.00%1.84%2.18%
Light grey0.97%1.57%1.98%2.39%0.76%1.10%1.50%1.47%
Off-white0.80%1.04%1.26%1.27%0.36%0.94%0.58%0.89%
White0.18%0.28%0.38%0.31%0.11%0.36%0.16%0.25%
Δ (%)1.07%1.16%1.21%1.48%0.87%1.01%1.22%1.15%
Table A4. Database of CaliPhoto RGB values for the coloured paper displayed in Figure A10. Uncertainties are given by 256 × Δ.
Table A4. Database of CaliPhoto RGB values for the coloured paper displayed in Figure A10. Uncertainties are given by 256 × Δ.
Reference SamplesRGBUncertainties
blue77.1159.9202.2±2.7
green158.2207.0131.1±3.0
mauve172.3177.0192.4±3.1
orange255.0188.7101.5±3.8
pink254.6216.1201.6±2.2
purple157.2129.8190.4±2.6
yellow255.0212.7123.3±3.1
It is then possible to identify a material using the same image correction processing and by comparing its CaliPhoto colour to the values stored in the database (see Figure 5, main text). For that purpose, we defined a matching parameter given by:
Mij = 1 − Δij
The results are displayed in Table A5.
Table A5. Identification of a sample (orange paper) using a CaliPhoto database. The best match occurs for the orange paper, which has a matching percentage Mij = 99%. The second-best match occurs for the yellow paper which, to the human eye, is not very different from the orange paper.
Table A5. Identification of a sample (orange paper) using a CaliPhoto database. The best match occurs for the orange paper, which has a matching percentage Mij = 99%. The second-best match occurs for the yellow paper which, to the human eye, is not very different from the orange paper.
Sample to Identify255191.69101.81Orange
Reference SamplesRGBΔij (%)Mij (%)
blue77.1159.92202.1547%53%
green158.21207.02131.123%77%
mauve172.37177.01192.3828%72%
orange255188.66101.491%99%
pink254.6216.12201.6523%77%
purple157.21129.81190.4433%67%
yellow255212.7123.37%93%

Appendix E.2. Identification Using CaliPhoto Colour Vectors

The identification method can be improved by using more complex parameters to create the database. Indeed, for nonhomogeneous materials, the average colour of two materials may be similar, as illustrated in Figure A11.
Figure A11. Schematics illustrating (a) homogeneous and (b) inhomogeneous materials and their associated average colours. For materials as described in (b), the average CaliPhoto colour cannot be used in the database since these different samples are associated with identical average values.
Figure A11. Schematics illustrating (a) homogeneous and (b) inhomogeneous materials and their associated average colours. For materials as described in (b), the average CaliPhoto colour cannot be used in the database since these different samples are associated with identical average values.
Inventions 04 00067 g0a11
To circumvent this issue, we propose using the histograms of the RGB values to define a CaliPhoto colour vector given by:
V i = H i s t ( R i ) [ 256 + H i s t ( G i ) ] [ 512 + H i s t ( B i ) ] M a x { R i , G i , B i }
where Hist(Ri) corresponds to the histogram of R, that is, the number of pixels versus R value in the image of the material, rounded to 5, [256 + Hist(Gi)] to the histogram of G, rounded to 5, displayed between 256 and 511, and [512 + Hist(Bi)] to the histogram of B, rounded to 5, displayed between 512 and 767, and Max{Ri,Gi,Bi} to the maximal values of the different histograms. The histograms are rounded to 5 to take into account the uncertainty in the CaliPhoto R,G,B values.
This method takes into account the inhomogeneity of materials in the CaliPhoto database. The identification of a material is made using the parameter ΔVij, defined as:
Δ V i j = ( V i V j ) 2 V i + V j
However, inhomogeneous materials of different colours may be associated with similar CaliPhoto vectors, defined using Equation (A9), as illustrated in Figure A12.
Figure A12. Schematics illustrating (a) homogeneous and (b) inhomogeneous materials and their associated vectors defined by Equation (A9). For materials as described in (b), the average colour vector as defined by Equation (A9) cannot be used to constitute a database, since different samples could be associated with ΔVij = 0.
Figure A12. Schematics illustrating (a) homogeneous and (b) inhomogeneous materials and their associated vectors defined by Equation (A9). For materials as described in (b), the average colour vector as defined by Equation (A9) cannot be used to constitute a database, since different samples could be associated with ΔVij = 0.
Inventions 04 00067 g0a12
Thus, in order to obtain unique CaliPhoto vectors taking into account the colour inhomogeneity of materials, we defined Veci, given by Equation (3) and depending on the histograms converted in rg chromaticity (r,g,b) and luminance (µ). The MATLAB code giving this vector is provided in Supplementary Materials (Code S3). We also defined the parameter ΔVecij, given by Equation (4), for the comparison and identification of materials. The matching parameter can now be written as:
Mij = 1 − ΔVecij
Finally, it important to note that, with CaliPhoto vectors defined using Equation (3), it is not possible to differentiate materials of similar colours but exhibiting different textures, as illustrated in Figure A13. Nevertheless, this aspect is not relevant for the CaliPhoto method that was designed for textureless materials.
Figure A13. Schematics illustrating inhomogeneous materials and their associated CaliPhoto vectors, defined by Equation (3). The CaliPhoto vector as defined by Equation (3) differentiates any inhomogeneous materials of different colour (a). It cannot, however, distinguish materials of similar colours but different textures, as shown in (b). Nevertheless, this is not the objective of the method.
Figure A13. Schematics illustrating inhomogeneous materials and their associated CaliPhoto vectors, defined by Equation (3). The CaliPhoto vector as defined by Equation (3) differentiates any inhomogeneous materials of different colour (a). It cannot, however, distinguish materials of similar colours but different textures, as shown in (b). Nevertheless, this is not the objective of the method.
Inventions 04 00067 g0a13
The CaliPhoto RGB values stored in the database of coloured papers can be replaced by the newly defined CaliPhoto vectors, and material identification can be performed using Equation (A11). The results of this test are displayed in Table A6.
Table A6. Identification of a sample (orange paper) using the coloured paper CaliPhoto vector database. The best match is the orange paper, with a match percentage M = 83%. The second-best match occurs for the yellow paper, but the match percentage is only 26%.
Table A6. Identification of a sample (orange paper) using the coloured paper CaliPhoto vector database. The best match is the orange paper, with a match percentage M = 83%. The second-best match occurs for the yellow paper, but the match percentage is only 26%.
Sample to IdentifyOrange
Reference SamplesMij (%)
blue25%
green0%
mauve17%
orange83%
pink0%
purple0%
yellow26%
Obviously, the images displayed in Figure A10, Figure A11 and Figure A12 were made to show the proportion of the different colours of pixels over the area. For the textureless materials considered in this study, the distribution of the colours would be indiscriminately spread over the analysed area; the colour plate size must be chosen according to that described in §S1.
We demonstrated in Appendix C that the proposed image processing permits recalibration of the RGB values of different materials in order to obtain specific CaliPhoto colour spaces in which each material is associated with a specific vector. Irrespective of the light and geometry conditions in which the photographs are taken, following the relatively straightforward recommendations in Appendix D.4, the variability on the CaliPhoto vector of a given material is approximately 1.5%. Based on this small variability, we demonstrated in Appendix E that the CaliPhoto method provides a rapid and uncomplicated method to identify a material by comparison with a database of CaliPhoto vectors taken from known materials.
In this example, the CaliPhoto vectors stored in the reference database are associated with the parameter “colour” (blue, green, etc.). Of course, the aim of the method is to be able to link these values to less obvious physical parameters, potentially obtained using more powerful and/or expensive systems such as field spectrometers.

Appendix F. Precharacterisation of Materials Using the CaliPhoto Method

The philosophy of the CaliPhoto method is to permit rapid and low-cost precharacterisation of material in order to select the samples most suited for further investigation (on-site collection of samples in the field or sample control prior to expensive and long analyses, for example). From this perspective, the principle is to use a suite of well-characterised materials permitting to link one, or several, specific physical properties of interest to their CaliPhoto RGB values or vectors in a database. Then, using this database, these properties will be estimated for unknown materials only based on their CaliPhoto RGB values or vectors.
As an example, we estimated the concentrations in a mixture of two paints. First, a size-adapted reference colour plate was printed and photographed, prior to being processed in order to obtain the CaliPhoto reference image (Figure A14a). Then, photographs of well-characterised mixtures were taken and processed to obtain their CaliPhoto RGB values (Figure A14b). Finally, a database was made linking the paint concentrations to their associated CaliPhoto RGB values (see Table A7).
Figure A14. Constitution of a CaliPhoto database of painting mixtures. (a) Original photograph and reference image after CaliPhoto processing and (b) original photographs and processed images of reference mixtures.
Figure A14. Constitution of a CaliPhoto database of painting mixtures. (a) Original photograph and reference image after CaliPhoto processing and (b) original photographs and processed images of reference mixtures.
Inventions 04 00067 g0a14
Table A7. CaliPhoto RGB values database for yellow/blue paint mixtures. Concentration uncertainties are a function of the syringe precision and the volumes used. RC, GC and BC are the CaliPhoto RGB values of the mixtures. Uncertainties on those values are given by 256 × Δ.
Table A7. CaliPhoto RGB values database for yellow/blue paint mixtures. Concentration uncertainties are a function of the syringe precision and the volumes used. RC, GC and BC are the CaliPhoto RGB values of the mixtures. Uncertainties on those values are given by 256 × Δ.
Reference MixturesConcentration in YellowConcentration UncertaintiesRCGCBC(RGB)C Uncertainties
100% blue0%0%0.041.33180.171.8
25% yellow–75% blue25%1%3.2665,5296.982.3
33% yellow–67% blue33%1%5.2477.5985.692.0
50% yellow–50% blue50%2%18,1388.9369.671.8
67% yellow–33% blue67%1%56.88117.3461.572.3
75% yellow–25% blue75%1%84.77134.2557.241.5
100% yellow100%0%255.00239.8745.091.8
From these data, it is possible to plot the evolution of RC, GC and BC with the concentration in yellow paint. These curves can then be used to estimate this concentration in a new mixture (as displayed in Figure 5f–h). In this example, the concentration in yellow was estimated to be 33% ± 5%, whereas the true concentration was 29% ± 1%. The estimated value thus matches closely to the true value. Note that the method would be considerably improved by increasing the number of reference mixtures in the database.

Appendix G. Creation of a Master Colour Plate for the CaliPhoto Method

The most significant restriction of the method comes from the fact that any database is necessarily associated to a given colour plate. If this colour plate is lost or irreversibly altered, the database becomes obsolete.
To solve this problem, we suggest maintaining a secure Master Colour plate permitting recalibration of new colour plates for use with previously established databases. The method, described in Figure A15, is as follows:
  • Print a colour plate sufficiently large to place the colour plates used for your application in the window. This large colour plate will now become your Master Colour plate and must be kept secure (Figure A15a).
  • Apply the CaliPhoto process defined in Appendix B in order to obtain a reference image of the Master Colour plate (Figure A15b).
  • Make a good-quality image of the new colour plate, which will be used for scientific investigations, placed in the centre of the Master Colour plate (Figure A15c).
  • Calibrate the image of the new colour plate using the Master Colour plate and the CaliPhoto image processing defined in Appendix C (Figure A15d).
  • Use the image of the colour plate in the centre of the Master Colour plate as a reference image.
  • In case of a lost or irreversibly altered colour plate, print a new colour plate and redo steps 1 to 5.
Figure A15. Calibration of a colour plate using a Master Colour plate. (a) Original photograph of the Master Colour plate. (b) Reference image after CaliPhoto reference colour plate image processing (Appendix B). (c) Photograph of the colour plate to be calibrated, which is placed in the centre of the Master Colour plate. (d) Reference image of the colour plate after CaliPhoto image processing.
Figure A15. Calibration of a colour plate using a Master Colour plate. (a) Original photograph of the Master Colour plate. (b) Reference image after CaliPhoto reference colour plate image processing (Appendix B). (c) Photograph of the colour plate to be calibrated, which is placed in the centre of the Master Colour plate. (d) Reference image of the colour plate after CaliPhoto image processing.
Inventions 04 00067 g0a15

References

  1. Leon, K.; Mery, D.; Pedreschi, F.; Leon, J. Color measurement in L*a*b* units from RGB digital images. Food Res. Int. 2006, 39, 1084–1091. [Google Scholar] [CrossRef]
  2. Gökay, M.K.; Gundogdu, I.B. Color identification of some Turkish marbles. Constr. Build. Mater. 2008, 22, 1342–1349. [Google Scholar] [CrossRef]
  3. Bianconi, F.; Gonzalez, F.; Fernandez, A.; Saetta, S.A. Automatic classification of granite tiles through colour and texture features. Expert Syst. Appl. 2012, 39, 11212–11218. [Google Scholar] [CrossRef]
  4. Furferi, R. Color Classification Method for recycled Melange Fabrics. J. Appl. Sci. 2011, 11, 236–246. [Google Scholar]
  5. Kempe, D.B. Colorimetric characterisation of flatbed scanners for rock/sediment imaging. Comput. Geosci. 2014, 67, 69–74. [Google Scholar] [CrossRef]
  6. Haeghen, Y.V.; Naeyaert, J.M.; Lemahieu, I.; Philips, W. An imaging system with calibrated color image acquisition for use in dermatology. IEEE Trans. Med. Imaging 2000, 19, 722–730. [Google Scholar] [CrossRef] [PubMed]
  7. Iñigo, A.C.; García-Talegón, J.; Vicente-Tavera, S.; Martín-González, S.; Casado-Marín, S.; Vargas-Muñoz, M.; Pérez-Rodríguez, J.L. Colour and ultrasound propagation speed changes by different ageing of freezing/thawing and cooling/heating in granitic materials. Cold Reg. Sci. Technol. 2013, 85, 71–78. [Google Scholar] [CrossRef][Green Version]
  8. Oestmo, S. Digital imaging technology and experimental archeology: A methodological framework for the identification and interpretation of fire modified rock (FMR). J. Archaeol. Sci. 2013, 40, 4429–4443. [Google Scholar] [CrossRef]
  9. McCamy, C.S.; Marcus, H.; Davidson, J.G. A Color-Rendition Chart. J. Appl. Photogr. Eng. 1976, 2, 95–99. [Google Scholar]
  10. X-Rite. ColorChecker Camera Calibration. Available online: https://xritephoto.com/colorchecker-classic (accessed on 7 November 2019).
  11. Morris, V.J.; Kirby, A.R.; Gunning, A.P. Basic Principles. In Atomic Force Microscopy for Biologists; Imperial College Press: London, UK, 1999; pp. 70–71. [Google Scholar]
  12. Wannous, H.; Treuillet, S.; Lucas, Y.; Mansouri, A.; Voisin, Y. Design of a Customized Pattern for Improving Color Constancy Across Camera and Illumination Changes. In Proceedings of the VISAPP 2010—Fifth International Conference on Computer Vision Theory and Applications, Angers, France, 25 May 2010. [Google Scholar]
  13. Josset, J.-L.; Westall, F.; Hofmann, B.A.; Spray, J.; Cockell, C.; Kempe, S.; Griffiths, A.D.; De Sanctis, M.C.; Colangeli, L.; Koschny, D.; et al. The Close-Up Imager Onboard the ESA ExoMars Rover: Objectives, Description, Operations, and Science Validation Activities. Astrobiology 2017, 17, 595–611. [Google Scholar] [CrossRef] [PubMed][Green Version]
Figure 1. Model of the CaliPhoto colour plate. The colour plate consists of a white 4:3 ratio rectangle (without the logo band), surrounded by 24 squares of different colours arranged in a rectangle, surrounding another white rectangle. At the centre of the colour plate is a rectangular window in which the sample will be observed. The 8 bit RGB values of each square, chosen in this study to correspond to the GretagMacBeth ColorChecker values, and of the internal and external white rectangles are displayed.
Figure 1. Model of the CaliPhoto colour plate. The colour plate consists of a white 4:3 ratio rectangle (without the logo band), surrounded by 24 squares of different colours arranged in a rectangle, surrounding another white rectangle. At the centre of the colour plate is a rectangular window in which the sample will be observed. The 8 bit RGB values of each square, chosen in this study to correspond to the GretagMacBeth ColorChecker values, and of the internal and external white rectangles are displayed.
Inventions 04 00067 g001
Figure 2. Processing of the reference image of the colour plate. (a) Photograph of the colour plate made using an iPad camera. (b) Resized and rescaled image. (c) Image after removing the plane corresponding to the illumination continuum over the whole area. (d) Final reference image of the colour plate after brightness and contrast correction.
Figure 2. Processing of the reference image of the colour plate. (a) Photograph of the colour plate made using an iPad camera. (b) Resized and rescaled image. (c) Image after removing the plane corresponding to the illumination continuum over the whole area. (d) Final reference image of the colour plate after brightness and contrast correction.
Inventions 04 00067 g002
Figure 3. CaliPhoto correction process for sample images. (a) iPad photograph of the sample (blue paper) placed below the same Caliphoto colour plate as used to create the reference image in Figure 2. (b) Resized and rescaled image. (c) Image after removing the illumination plane. (d) Image after brightness and contrast correction. (e) Final reference image of the sample after colour calibration using the reference image of the colour plate (Figure 2d). (f) Reference image of the colour plate used for correction, shown for comparison. (g) Colour change of the sample area during the process from images (be).
Figure 3. CaliPhoto correction process for sample images. (a) iPad photograph of the sample (blue paper) placed below the same Caliphoto colour plate as used to create the reference image in Figure 2. (b) Resized and rescaled image. (c) Image after removing the illumination plane. (d) Image after brightness and contrast correction. (e) Final reference image of the sample after colour calibration using the reference image of the colour plate (Figure 2d). (f) Reference image of the colour plate used for correction, shown for comparison. (g) Colour change of the sample area during the process from images (be).
Inventions 04 00067 g003
Figure 4. Using the CaliPhoto method to compare two sides of a sample. (a) Photograph of a rock sample (trachybasalt) illuminated by the sun on the left, (bd), and by a filament bulb on the right, (eg). (b,e), images of the left and right sides of the sample with the CaliPhoto colour plate. (c,f), images of only the area in the centre of the colour plate. The average RGB values of these areas are respectively R,G,B = 43,54,72 and R,G,B = 151,137,126 before correction (Δ12 = 33%). (d,g), the same images after correction using the CaliPhoto method. The two sides of the rock are shown to be quite similar in colour, with average CaliPhoto R,G,B values equal to 93,107,84 and 90,109,81 respectively (Δ12 = 1% between the two images), suggesting that they have similar compositions. N.B. the CaliPhoto colour of the materials may differ from their colour as seen by human eyes.
Figure 4. Using the CaliPhoto method to compare two sides of a sample. (a) Photograph of a rock sample (trachybasalt) illuminated by the sun on the left, (bd), and by a filament bulb on the right, (eg). (b,e), images of the left and right sides of the sample with the CaliPhoto colour plate. (c,f), images of only the area in the centre of the colour plate. The average RGB values of these areas are respectively R,G,B = 43,54,72 and R,G,B = 151,137,126 before correction (Δ12 = 33%). (d,g), the same images after correction using the CaliPhoto method. The two sides of the rock are shown to be quite similar in colour, with average CaliPhoto R,G,B values equal to 93,107,84 and 90,109,81 respectively (Δ12 = 1% between the two images), suggesting that they have similar compositions. N.B. the CaliPhoto colour of the materials may differ from their colour as seen by human eyes.
Inventions 04 00067 g004
Figure 5. On the use of the CaliPhoto method for sample identification and for compositional measurement. (ad) Example of identification process using coloured paper. (a) Reference images of reference samples constituting the database obtained after CaliPhoto image processing. (b) Original image of the sample to be identified. (c) Processed image of the same sample. (d) Table reporting the percentage match using RGB values (M = 1 − Δ12) or the CaliPhoto colour vectors (M = 1 − ΔVec12). The sample is demonstrated to be the orange object. (eh) Example of the use of the CaliPhoto method to estimate the concentration in a mixture of two paints. (e) Reference images of reference samples of known concentration used for the database, obtained after CaliPhoto image processing. (f) Original image of the mixture to be determined. (g) Processed image of the same mixture. (h) The concentration of yellow paint can be estimated from the curves displaying changes in the CaliPhoto RGB values for the known mixtures. The estimated concentration of yellow paint is approximately 33% ± 5%, whereas the true concentration was 29% ± 1%.
Figure 5. On the use of the CaliPhoto method for sample identification and for compositional measurement. (ad) Example of identification process using coloured paper. (a) Reference images of reference samples constituting the database obtained after CaliPhoto image processing. (b) Original image of the sample to be identified. (c) Processed image of the same sample. (d) Table reporting the percentage match using RGB values (M = 1 − Δ12) or the CaliPhoto colour vectors (M = 1 − ΔVec12). The sample is demonstrated to be the orange object. (eh) Example of the use of the CaliPhoto method to estimate the concentration in a mixture of two paints. (e) Reference images of reference samples of known concentration used for the database, obtained after CaliPhoto image processing. (f) Original image of the mixture to be determined. (g) Processed image of the same mixture. (h) The concentration of yellow paint can be estimated from the curves displaying changes in the CaliPhoto RGB values for the known mixtures. The estimated concentration of yellow paint is approximately 33% ± 5%, whereas the true concentration was 29% ± 1%.
Inventions 04 00067 g005

Share and Cite

MDPI and ACS Style

Foucher, F.; Guimbretière, G.; Bost, N.; Hickman-Lewis, K.; Courtois, A.; Luengo, L.; Marceau, E.; Bergounioux, M.; Westall, F. The CaliPhoto Method. Inventions 2019, 4, 67. https://doi.org/10.3390/inventions4040067

AMA Style

Foucher F, Guimbretière G, Bost N, Hickman-Lewis K, Courtois A, Luengo L, Marceau E, Bergounioux M, Westall F. The CaliPhoto Method. Inventions. 2019; 4(4):67. https://doi.org/10.3390/inventions4040067

Chicago/Turabian Style

Foucher, Frédéric, Guillaume Guimbretière, Nicolas Bost, Keyron Hickman-Lewis, Aurélie Courtois, Lydie Luengo, Etienne Marceau, Maïtine Bergounioux, and Frances Westall. 2019. "The CaliPhoto Method" Inventions 4, no. 4: 67. https://doi.org/10.3390/inventions4040067

Article Metrics

Back to TopTop