Assessment of the Robustness of a Color Monitoring Chart Calibration Method for Crowdsourcing-Based Preventive Conservation

: Monitoring environmental factors such as pollutants, relative humidity, light, etc. is needed to ensure a proper preventive conservation of cultural heritage. Since existing systems are expensive and difﬁcult to maintain, we propose as a sustainable alternative the use crowdsourced photographs taken by visitors using their cell phones. These images of a tailor made panel, including sensitive materials and a calibration color chart, are calibrated and colorimetric information extracted from them. The aim of this paper is assessing the robustness of the color chart and the calibration method used, and evaluating how the ageing of the materials of the chart can affect the accuracy of the calibration process. A choice is made between four candidate materials and a set of industrial inks. Several tests have been performed and mathematical metrics extracted, so the most suitable material is adequately selected. It has been checked how the image calibration process, employed for homogenizing information from the crowdsourced pictures, performs with real-life materials and colors to assess the possible degradation that may happen. This work is a part of a bigger project with the aim of building a crowdsourcing-based monitoring system for preventive conservation of cultural heritage.


Introduction
All over the world, objects of historic and cultural values are exhibited and preserved in museum collections. To protect objects in collections from degradation they are held and exhibited in controlled environments, where ideally all ambient conditions are adjusted to minimize future damage to the heritage materials. This strategy, focused in preventing future deterioration acting on the potential causes of deterioration, is known as preventive conservation.
However, the perfect environment is impossible to achieve, and sometimes humidity, pollutants or lighting problems can reach a point where the material an exhibited cultural heritage item is made of starts suffering a quick degradation [1][2][3]. Sometimes harmful emissions come from inside. Building materials from furniture, showcases or panels, such as paints, woods or adhesives or cleaners, and even some objects from collections can emit gaseous pollutants. This problem is even more serious if the exhibited artifacts are stored in non-ventilated display [4,5].
Thus, even in the best conservation-suited environment, additional needs for surveillance of cultural heritage are likely to be required. In order to lighten human efforts on the task of surveillance, automatic monitoring resources are encouraged and proposed. These may take any mean for quantifying the possible decay, like sensing humidity or light or Photographic cameras represent the acquired information analogously to the human visual system [16,17]. But, considering fabrication and physical component differences between models, images representing the same scene by disjoint cameras are likely to appear visually different to the same observer [18]. This is the main concern regarding the crowdsourced information, therefore a homogenisation via color calibration is needed to perform an effective monitoring of the pieces. This requires the presence of a color chart when acquiring the content to be calibrated.
The mentioned approaches use mobile images individually adjusted using X-Rite charts as color reference. However, the X-Rite color chart has been designed provide bright colors for outdoor photography, and some of the tiles have too saturated colors [13], and its materials have not been tested to be compatible with heritage objects. Our monitoring system pretends a further step on these first approaches, designing and testing an optimized color chart, an accurate calibration method and developing an efficient color change detection system.
The previous stages of this project, the conception of the color chart, the verification of the suitability of the materials for the monitoring system, and an adaptive calibration algorithm for image processing have already been presented [11,19]. The color chart is mounted on an inert substrate together with several metal coupons used as sensitive materials.
This reference color chart has been specifically designed to cover all the range of the sRGB color space, which is shared by most phone cameras in usage. It is placed in the display case or exhibition space to be monitored in this project. Alongside the chart, metal coupons are placed. Possible inadequate environmental conditions that may affect the heritage artifacts will induce corrosion in the coupons. This corrosion process might take the form of a tarnishing, more or less uniform colored corrosion layers, or colored spots on the surface of the metal. These colorimetric cues are the relevant information to be extracted from the crowdsourced pictures, as warnings of unsuitable environmental conditions for the conservation of heritage assets.
The calibration process is one of the most critical parts of the monitoring system. Assuming most crowdsourced pictures are taken with different cameras, a quality calibration for every image is needed for homogenizing color information. According to this, the color chart needs to be consistent and stable enough to be used as a reliable reference for the color calibration algorithm.
So, the materials employed to build the display and chart and the coupons need to be carefully selected so they do not undergo possible strong chromatic deviations from the initial state; and they should not produce alterations in the metal coupons and, more important, in the heritage artifacts [11]. It is utmost important that the reference color chart remains unaffected so the quality of the picture calibration does not lower when deployed for a prolonged time. In addition, if by any reason the color references change, it is imperative to check how this change can affect the outcomes of the calibration process.
Thus, the aim of this paper is to evaluate robustness of the crowdsourced color evaluation system against environmental degradation of the reference display. The criterion adopted is mathematical robustness to discoloration. Since the main agent causing color modification in organic materials (including boards, papers and inks) is light, variations in the color chart on different materials due to exposure to UV radiation have been evaluated. For this purpose, periodic measurements of the exposed references have been carried out.
These measures are processed in different mathematical ways to evaluate the robustness of environmentally-induced color degradation in them. From this information, it is evaluated how possible light induced color variations may affect the accuracy of the calibration method.
Finally, conclusions are drawn and the selection of the most suitable material to build the crowdsourcing-based monitoring display are proposed.

Color Chart Design
The color chart has been conceived so the calibration to perform does not incur in unacceptable errors. It consists in 64 color patches of dimensions 1 cm × 1 cm arranged on 11 columns and 9 rows. There is a free space of 7 columns × 5 rows in the centre, where the metal coupons (or other references) can be placed ( Figure 2). The calibration process is performed by calculating a transfer function that projects the R, G and B information of each channel of the taken pictures into a reference space, where their corresponding coordinates take values that must be present in the calibrated image. Thus, all pictures after this operation should depict consistent color information [19], regardless of their camera of origin. With this conception in mind, the depicted colors present regularly sparse grey tones range in sRGB acting as valid as luminance values too, including maximum black and white, and the primaries red, green, blue, cyan, magenta and yellow. The remaining colors are combinations of equal shares of each RGB axis, and are furthermore designed to exist in the perceptual spaces as sRGB and inside CIELAB's boundary [19]. This avoids transformation errors when colors exist regardless of the color space to consider in further application. It is also assured that the possible range of values is covered equally, and inferring the projection of colors not present within the set of anchor colors not much accuracy is lost.
It is imperative that the manufactured color charts to be used in museum showcases are stable for a prolonged usage whilst maintaining the color characteristics mentioned above. This way they are expected to withstand the adverse ambient conditions that this project, in its global scope, is intended to monitor.

Color Chart Materials
Color charts have been printed by a graphic arts printer using selected materials for the support and UV resistant inks. Four candidate materials for the support have been evaluated: acid-free laminated paper, acid-free non-laminated paper, Forex Smart and Glasspack. The choice of these papers, within the options available on the market, has been based on the British Museum's Oddy Test database, to ensure their safety [20]. In addition, their light weight, firmness, low cost and ease of cutting and handling making it easy to build an exhibition structure with any of them, are suitable for the intended purpose.
The ink set tested for the color chart was LED-UV Xtreme Pro, by Deutsche Druckfarben. These inks are used for printing advertising posters intended for outdoor display, and they are known to resist solar radiation. Their lightfastness is rated 5 (yellow and magenta) and 8 (cyan and black) to blue wool standard according to the manufacturer [11].
The thickness of the ink has been measured using a Helios Digitrix II micrometer, with 1 µm resolution, by difference of the thickness of the blank paper and the printed patches. The thickness strongly depends on the color, ranging from 2-3 µm in the ligthest patches, up to 20-25 µm in the more saturated colors and 40 µm in the black patch.

Color Chart Evaluation
Therefore, a display structure is made from each one of the four support materials and the set of inks. Each test display has been exposed to a constant irradiation of UV light at 50 • C, using fluorescent UVA-340 lamps, following the ISO4892-3 standard in a Q-Lab QUV chamber [21].
Since the color charts are intended to be exposed in museum cases for a long time, the UV exposure time has been calculated to be equivalent to several museum years. Q-Lab QUV chamber lamps reproduce daylight conditions, which are equivalent to a 30,000 lux illuminance. As museum illuminance for non-sensitive materials is set at 300 lux, the illuminance in the chamber is (maximum) 100 times that in the museum. Thus, we can estimate that in the worst scenario (a non UV-filtered illumination), 144 h chamber exposure (six 24 h periods) can be approximated to 4 years of museum (144 h × 100 = 14,400 h of exposure /10 h per day = 1440 days = 3.9 years. Measures of the color coordinates have been taken for hours 0 (without degradation before exposure), 2,4, 6, 24, 48, 72, 96, 120 and 144 using a Konica Minolta spectrophotometer CM-700-d, with 6 mm diameter mask, D65 as illuminant and at 10 degrees to the observer, in color space CIELAB. Every measure taken consists in an automatic average of three measurements on areas of 1 cm 2 . Its accuracy specifications reveal a repeatability tolerance of a standard deviation between 0 and 0.1% of the spectral reflectance, and within 0.04∆Eab of the detected colorimetric value. As a representative set of colors of the printed charts to be evaluated, the patches corresponding to white, black, red, green, blue, cyan, magenta and yellow are taken. It is to be noted that the color patch corresponding to the white tone does not have any ink printed upon it, so it consists in the naked material. Therefore, the effect of the test on the substrate material can be tested when analysing the white patch. Consequently, the other color tones are considered indicators of the effect of the degrading ambiance on a combination of substrate and inks. Thus, the effect of the degrading ambiance on the naked substrate and on different mixtures of different amounts of inks can be observed, and its possible effects on the calibration process inferred.
The results are a list of progressive CIELAB coordinates along time for given color patches in every material combination. The CIE1976 L*a*b* color space, or CIELAB, is usually employed in heritage science matters as accurate color specification [22], while being uniform, device-independent and suitable for calculations such as calibrations [19].
Several means to describe the degradation have been considered. A graphical evolution along time of the CIELAB coordinates for the selected color patches has been observed ( Figure 3). In addition, a good descriptor of degradation is a difference metric. The ∆E 00 CIEDE2000 difference formula has been selected for this purpose, since it is designed to reflect CIELAB color differences according to human perception [23]. Equation (1) reflects its nature [24]. It defines a metric similar to a mathematical distance in the CIELAB color space, employing differences related to the physical lightness, chroma and hue (L, C and H). The S functions and parametric factors k define weights to the former quantities, and in order to improve its effectiveness for the blue range it employs an interactive term between C and H (R T ).
The difference formula has been employed to show a progressive degradation. Each CIELAB measure from each one of the time checkpoints is compared to the original zerohour measurement, its difference calculated and its progress shown.
The analysis for the white, ink-free patch is particularly interesting for what it may reveal about the nature of degradation on the bare support material, without the ink acting as a cover. Special consideration has been given to measurements of CIELAB values and differences concerning these patches. Their effect is measured by performing calibration processes using these degraded white values as reference, and the calibration error as CIEDE2000 difference is then evaluated for all materials. The next section will address the results, explaining the material to be chosen and which ones could be discarded or used as a temporary substitute. The choice ensures that the display will guarantee a stable calibration with consistent results, so the monitoring via crowdsourced images can be enabled for an extended period in a museum.
Aside, in order to demonstrate the capabilities of the conceived charts, an experiment that involves the calibration scheme outlined in [11,13] over pictures taken with commercial brands of phones has been performed. With the material chosen with the best results (explained in the following sections), a chart has been printed and photographed. Pictures have been taken with the following models: Xiaomi Mi10 lite 5G, Poco X3 NFC, Poco F3, Huawei P30, iPhone SE and Microsoft Surface Pro ( Figure 4). This way, the crowdsourcing disparity is achieved. After having collected the pictures, the calibration process has been ran over them and the robustness evaluated. The calibrated color patches of each of the pictures have been compared with the reference color values in sRGB using the CIEDE2000 difference formula.
In addition to this, the mean digital CIELAB coordinates for every patch in each calibrated picture have been compared to the original spectrophotometer measurements so the accuracy of the calibrated values with respect to the original data can be assessed.

Results and Discussion
The evolution of CIELAB coordinates of each of the considered color patches are listed in the Tables A1-A4. It should be noted that the white coordinate, corresponding to the ink-free spot of the color chart, undergoes a stronger variation in comparison to the other colors under study in all cases. This degradation, seen as a progressive shift of values, is especially remarkable in the b* coordinate. This implies a process of yellowing, extremely pronounced for the Glasspack (Table A4). The inks act as a "shield" for the substrate material, given their greater stability and less intense degradation. The most stable of the materials considering its low variation is the acid-free laminated paper (Table A1).
Based on those tables, the CIEDE2000 differences for each one of the four materials under study are shown graphically ( Figure 3). In general, the color patches with inks printed upon them show a relatively slow degrading, with a barely steep slate, staying in most cases under the ∆E 00 CIEDE2000 < 1.5 threshold, implying a minimal perceivable color difference, only noticeable by trained eyes [25][26][27]. This indicates good stability and robustness in the considered exposition period.
Considering that materials such as papers usually are not as resistant towards chromatic deviation as other, more expensive materials like ceramics, it is promising to observe how small the induced error can be for the considered application.
It is interesting to note that on half of the tested materials, the color patches with only one ink printed (black, yellow, cyan or magenta) show a subtly stronger degradation than the other colors, which need more than one ink to be printed, following the additive YCMK system. This reinforces the idea of the nature of the inks as protective covers of the substrate material.
Therefore, the white ink-free patch shows a stronger degradation compared to the rest in the described graphs. Furthermore, its degradation can exceed a CIEDE2000 difference of value 3 for the acid-free paper and the Glasspack. It can be stated that the degradation of the ink-free spots is especially necessary to consider for the whole experiment ( Figure 5), since an excessive difference with the zero-hour original value may lead to an erroneous color calibration process. This fact could compromise the whole monitoring mechanism. Therefore, a deeper observation of the white spot for all the considered materials has been done. Several "dummy" color charts have been generated. The colored spots have been conserved, but the value corresponding to white patch has been substituted in each one of the materials with the RGB color coordinates corresponding to the CIELAB of every taken measure for the given time checkpoints. This is done for all four substrates. Then, the calibration process as described in [19] has been run on each of them with respect to the corresponding zero-hour color chart as the reference. Thus, the drifting effect of the decoloring towards the yellow of the white spot on the whole calibration process can be measured.
The subjects of calibration have been seven color stripes, in order to assess its effect on a variety of surfaces. Four of them represent silver and copper extracted textures from other photographs, since the indicator material coupons for the monitoring system are will be made of both metals. The three others represent a neutral grey hue (RGB = 128, 128, 128), a uniform copper color (RGB = 199, 140, 98) and a bright yellow (RGB = 255, 230, 128). They are used to see the effects of calibration with an erroneous white reference on different zones of the color space: the center, the region of copper tones, considering silver will mainly engulf luminance-predominant hues, and the yellow stripe represents the yellowing shown by aged whites (Figure 6). After each process, the stripes are extracted and their mean CIELAB coordinates and CIEDE2000 difference calculated. Results are presented in Tables 1-4. For the time interval considered the calibrations of the modified color charts present a moderate difference, all values remaining in a close range even between the least and most deteriorated state, in every case smaller than the acceptability threshold of CIEDE2000. The most stable material is the acid-free paper (Table 1), that undergoes practically no deviation, whilst the others still fluctuate, even if slightly. Nevertheless, the acid-free nonlaminated paper presents the best balance on lower mean error (lower than the acid-free paper) with acceptable variation ( Table 2).  However, it is interesting to note that the differences shown in the Tables 1-4 are not continuous, but they fluctuate at each time step. A reason for this can be understood when the "white" values for every paper are represented in their 3D color space (Figures A1-A5) and their evolution along time observed. The white tones, represented by dots in space, do not follow a linear evolution over time, but evidently fluctuate in space when under the chromatic deviation process, following an irregular evolution from the initial white tone in different directions during each time step. This is represented using arrows, as seen in the figures. For every paper, the arrows in the a*L*, b*L* and a*b* planes ( Figures A1-A4), and in the whole L*a*b* space ( Figure A5) are represented. Depending on the position in space of any affected color during the calibration process, its distance from its projected reference value will be of different magnitude and orientation depending on its initial position, closer or farther away from the state of the white value in the chart in the moment the calibration is performed. This is extensible to any other reference color in the chart. This irregular variability of the reference color depending on degradation and time explains the fluctuating results when calibrating.
Aside of that, it is also remarkable to see how, even when performing a calibration with deteriorated color charts, the results presented here are of a higher quality than other similarly oriented cultural heritage conservation works with ideal charts [14]. Whilst state-of art accepted CIEDE2000 differences exceed the minimal perceivable difference threshold with an ideal chart, our physically printed charts on every material among the tested ones under a chromatic deviation equivalent to 4 years still depict values inferior to the threshold in all the cases, and most of them with a CIEDE2000 value between 0 and 1 after application of the calibration process.
Overall, concluding from the color difference values, it can be observed that the four materials under consideration (laminated acid-free paper, acid-free paper, Forex Smart and Glasspack) are robust enough to resist the effects of chromatic degradation over a long period without incurring in errors noticeable to the human eye in the calibration process, in spite of the material degradation that still occurs, even if slight, when observing the metrics. If the selection of the material would only depend on the mathematical chromatic stability criterion, the most suitable and balanced one would be the acid-free laminated paper. Even if the differences are subtle from a numerical perspective, Glasspack would be discarded, and acid-free paper and Forex Smart would be seen as the second and third choices, respectively.
Out of the performed tests, when it comes to deploying the structures. the selected materials should fulfil some basic requirements in order to be used in a museum environment. The most important one is that the material should be innocuous to the heritage assets in its environment. Many materials are known to release different pollutants that can cause degradation of sensitive materials [5,28]. Additionally, since the chart is to be used in combination with metallic coupons (acting as sensitive materials for the environment), the materials of the chart should also be interred to the metal coupons. In order to check these aspects, the four candidate materials have been subject to the Oddy Test. Based on these results, which can be found in detail in a related work by the authors [11], acid-free laminated paper has been chosen, considering the balance between color stability and emission of harmful pollutants.
Thus, considering the facts above, the most suitable material for building the calibration display has been the acid-free laminated paper. Its chromatic stability is the best ones among all the tests, whilst fulfilling the real-life safety requirement. The calibration process for the crowdsourced images can be therefore considered as safe to perform for a prolonged time.
Having decided the definitive materials for building the charts, one has been printed to evaluate its suitability to endure the disparity effects on crowdsourced imaging. Six pictures with different phones, as outlined in the previous section, have been taken in the lab and the CIEDE2000 differences for each color patch extracted. Also, the standard deviation for all the metrics corresponding to the same patch has been calculated (Table A5).
As seen there, not only the quality metrics are excellent, but also the standard deviation is close to 0, meaning the color calibration is adequately performed on every picture and there is and extremely low variability between the processed images, regardless of their camera of source.
In Tables A6 and A7 the accuracy of the calibration scheme thanks to the robustness of the chart is demonstrated. The average digital L*, a* and b* coordinates for the color patches of each calibrated picture have been extracted and compared with measurements from the printed chart made with the spectrophotometer. Their extreme similarity in all cases reveal the good performance of the calibration scheme, which is allowed with the robust printing on the chart.

Conclusions
The work described above has made it possible to evaluate the performance of the developed calibration system in a non-ideal context and to assess how well it works. The application of the calibration method on four different materials has allowed to evaluate the influence of the ageing of different materials in the outcomes of the calibration of the images. Mathematical criteria have been taken in order to select the most adequate support material to build the structure. Under an analysis of color mathematics over the test results, very mild differences have been found between the four of them. Robustness to externallyinduced chromatic deviation is practically analogue in every studied case, with a revealed decay that is non-noticeable by human eye.
Thus, the calibration process allows to extract valid information from any taken images, guaranteeing that no errors originating from the color change of the reference charts may hinder the proper operation of the entire system.
Finally, the results support the possibility of manufacturing customized reference color charts using low-cost commercial materials, as the developed calibration process can easily overcome the small color drift introduced by aging. These reference color charts can be used to design low-cost conservation tools, affordable for small museums with limited budget.

Future Steps
Considering that the work presented in this paper is engulfed in a the framework of a bigger project, further steps to consider from this point on are the natural continuations of the premises exposed in the previous sections.
The calibration of crowdsourced images, to obtain reliable color coordinates from RGB images obtained with different cameras is a challenging question. Although the same accuracy of a dedicated instrument (spectrophotometer) can not be expected, results shown in this and previous papers by the authors [11][12][13][14][15]19] and other colleagues [13][14][15] show promising results of this approach for our intended application (detection of degradation of heritage artifacts).
After these initial tests, deployment of the color charts in real locations is currently under way. Charts have been industrially printed, and with the corresponding metal coupons, they have been installed in museums, where they are being tested during an one-year period. Their performance and robustness will be evaluated, and possible improvements of the calibration and the crowdsourcing system will be made during that time.
Calibration of crowdsourced images, from different cameras and acquired under nonideal illumination conditions and angles, is a challenging question that will be addressed in this step. While the accuracy of a dedicated instrument (spectrophotometer or calibrated professional camera) can no be expected, preliminary results suggest that can be enough to detect color changes in the sensitive materials, thus serving as alert system of environmental conditions dangerous for the conservation of heritage artifacts.
When a good performance can be assessed, the whole system can be officially launched and installed.  Acknowledgments: Thanks are due Ignacio García, from Once34, for his generous collaboration for the printing of the charts. EC and BRB wish to acknowledge the support of CSIC Interdisciplinary Thematic Platform "Open Heritage: Research and Society" (PTI-PAIS).

Conflicts of Interest:
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results. Table A1. L*a*b* coordinate value evolution for the acid-free laminated paper.  Table A2. L*a*b* coordinate value evolution for the acid-free paper.   Table A3. L*a*b* coordinate value evolution for the Forex Smart.  Table A4. L*a*b* coordinate value evolution for the Glasspack.     Figure A2. Projections of the a*L*, b*L* and a*b* planes depicting the evolution of the CIELAB coordinates of the white patch in the Acid-Free Paper along time. Following the arrows, it can be seen that its degradation is not linear nor uniform. Figure A3. Projections of the a*L*, b*L* and a*b* planes depicting the evolution of the CIELAB coordinates of the white patch in the Forex Smart along time. Following the arrows, it can be seen that its degradation is not linear nor uniform. Figure A4. Projections of the a*L*, b*L* and a*b* planes depicting the evolution of the CIELAB coordinates of the white patch in the Glasspack along time. Following the arrows, it can be seen that its degradation is not linear nor uniform.