A Study on the Diagnosis Technology for Conservation Status of Painting Cultural Heritage Using Digital Image Analysis Program

Abstract

In order to objectively and quantitatively diagnose the conservation status of painting cultural heritages, a digital image analysis program was used for the digital image of a Korean Buddhist painting. A technical method for diagnosing the conservation status by analyzing the color space of the Buddhist painting and calculating the shape information and damage area was suggested. To verify the applicability of the program, a standard color chart on various ground materials was produced. Color difference values of color information obtained using a color meter and a digital image analysis program were compared. Work efficiency, according to image resolution, was evaluated. It was possible to work efficiently with a size similar to that of an FHD image. Through the comparative evaluation of the individual deviation by the user, the accuracy and temporal advantages of the method using the digital image analysis program could be confirmed. As a result of comparing by color region extraction conditions, it was confirmed that the error range of the measured color information decreased as the 25 pixel diameter circle (average of circled regions of interest) size region and the number of measurements increased. Color space information was adjusted according to the characteristics of each damage type, and the shape of the damage was classified. In addition, the conservation status was evaluated by quantitatively calculating the area of damage with the Intensity Mean of the divided shape image.

1. Introduction

Damage to painting cultural heritage is caused by various factors such as material and manufacturing techniques, temperature and humidity, biological damage, and artificial damage [1,2,3]. In particular, the surrounding environment where the cultural heritage is stored is closely related to damage, and the color, texture, and type of damage may vary depending on the characteristics of the ground material [4,5,6]. The conservation status of painting cultural heritage has been diagnosed through visual observation, infrared and ultraviolet imaging, ultrasonic examination, infrared thermography, and hyperspectral imaging [7,8,9,10,11]. The state of conservation is evaluated under subjective judgment such as the investigator’s experience and feelings. Therefore, if the investigators are different, the evaluation results may vary, therefore, there is a limit to presenting an objective basis for determining the conservation status. The conservation status diagnosis results need to be made on a more quantitative and objective basis, as they can significantly affect the establishment of preservation plans that directly intervene in cultural heritage [12]. Currently, in the field of cultural heritage, studies are being conducted to attempt a quantitative evaluation of cultural heritage through digital image analysis [13,14,15,16,17,18,19,20,21,22]. The method using digital images is a non-destructive method, which can synthesize RGB channels to quantitatively represent all color areas [23]. If the color information of digital images is used, cultural heritage painting damages can be objectively distinguished. Most image programs are designed for editing and printing photos, and only a few of them are used due to difficulties in image analysis methods [24]. This study analyzed the color space using the Image Analysis and Processing Program (PicMan) [25,26,27], which can analyze the color of the image quantitatively and statistically, and found a way to diagnose the conservation status by calculating the shape and area of the damage caused in painting cultural heritage.

2. Materials and Methods

2.1. Subject

In this study, a large-scale Buddhist painting (gwaebultaeng) representative of Korean Buddhist paintings was selected for the operation of a digital image analysis program. The painting was designated as a Korean national treasure in recognition of its important value among large-scale Buddhist paintings, and various colors and types of damage were identified (Figure 1,

Table 1. History of Ansimsa yeongsanhoe gwaebultaeng.

Name	Ansimsa Yeongsanhoe Gwaebultaeng (Buddhist Painting of Ansimsa Temple)
Designation	A national treasure of Korea
Age	The 1652 year
Material	Pigment on Silk
Quantity	1
Size	866.0 × 485.6 cm
Weight	67.5 kg

). The painting was painted in 1652 using natural pigments on silk. Conservation treatment work on the painting was carried out in 2000. The painting is stored in the Yeongsanjeon Hall of Ansimsa Temple. It is stored in a state of being rolled up inside the gwaebulgwe, and inside the gwaebulgwe (storage box in a gwaebultaeng) there are several sheets of cotton cloth, neutral paper, dehumidifier, and insect repellent to protect the gwaebultaeng [28].

2.2. Digital Image Analysis Program (PicMan)

The PicMan (WaferMasters, Inc., Dublin, CA, USA) digital image analysis program can process digital images to quantify shape (scale, angle, area, etc.) and brightness information (RGB, HSV, CIE-L*a*b*, Munsell Color, etc.), extracting measured data in CSV format. In addition, it can easily distinguish shapes by adjusting the brightness level for the selected color region and processing various types of image files (JPG, GIF, PNG, TIF, etc.) [25,29]. The main functions of programs that analyze painting cultural heritage’ damage include the use of Point Value (extracts the mean value of color information), Threshold Switching (extracts pixels based on a specific brightness value), and Find All Shapes with Partial (recognizes and extracts pixels with the same brightness value as a given region). Other functions include Color Mapping in HSV Space, which extracts the desired color region based on HSV color space information, and Relief, which provides relief effects by shifting the pixels on the x- and y-axes of digital images [27]. Since the digital image analysis program processes color information based on color information, the damaged area may be calculated through a quantitative value. However, differences may arise depending on the conservation scientific knowledge that recognizes the damage, therefore, quantitative numerical records of the calculated area are required.

2.3. A Program-Applicability Validation Evaluation

2.3.1. Color Difference Value Comparison Evaluation

Standard color charts were produced on fabric (silk) and paper (dakji), the main ground materials that comprise Korea’s painting cultural heritage [30]. Color information was extracted using a colorimeter (Chroma Meter CR-400, Konica Minolta, Japan, Ø8 mm) and a digital image analysis program. The shifting of the color difference (ΔE) value of the extracted color information was compared.

2.3.2. Comparative Evaluation of Work Efficiency by Image Resolution

When using a digital image analysis program, the efficiency of the program operation was compared and evaluated, such as the available information according to the resolution of the image and the time required for processing. Based on the image’s resolution (8766 × 5844 pixels), images reduced to 50% (4383 × 2922 pixels), 25% (2192 × 1461 pixels), and 10% (877 × 584 pixels) were compared.

2.3.3. Comparative Evaluation of Operating Individual Deviation by Program and User Proficiency

• Program-specific deviation comparison.

Moreover, the study distinguished between the existing method (Illustrator 2021, Adobe, Mountain View, CA, USA) and the one using the digital image analysis program, calculating and comparing the area for the same-color area.

• Comparison of deviations by program proficiency.
• When calculating the area of the same-color area using a digital image analysis program, interindividual variances according to program proficiency were compared. User proficiency was classified into four levels: beginner (users new to the program), intermediate (users who have used the program for about three months), advanced (users who have used the program for about five months), and expert (program developers).

2.4. Digital Image Analysis of Painting Cultural Heritage

2.4.1. Construction of Basic Cultural Heritage Information

Before using the digital image analysis program, information concerning the scientific analysis of the cultural heritage’s ground materials and pigments was collected, and its current conservation statuses were investigated to identify the damaged area. In addition, comprehensive humanities information and scientific data on conservation were established and used for the program’s technological operation.

2.4.2. Comparative Evaluation by Color Information Extraction Conditions

The region of interest (ROI)’s size and number of measurements were assessed differently to select extraction conditions when extracting color information using a digital image analysis program. The extraction area was extracted in sizes of 1 pix (single point), 5 pix, 10 pix, 25 pix, 50 pix, and 100 pix diameter circles (average of circle ROIs). The number of measurements observed at 1–100 points was compared following the differences in color information displayed according to the number of points.

2.4.3. Image Analysis by Damage Type

Color information (in RGB format) was extracted using the digital image analysis program’s Point Value function for three major damage types in painting cultural heritage: creases, exfoliation of the painting layer, and contaminations. The damaged and undamaged parts were compared using the extracted RGB intensity graph, and, based on color information for each damage type, shape information was extracted using the Threshold Switching, Relief, and Color Mapping in HSV Space functions according to each damage type’s characteristics. After converting the extracted shape information into a black and white (Black & White Binary) image, the area of damage was calculated using the mean value (Intensity Mean) of the entire image’s color. The conventional method (1) is used to calculate the number of pixels in the damaged area ($P_{Da}$) for the total number of pixels in the target ($P_{Fa}$), but, in this study, method (2) of calculating the Intensity Mean ($I{M}_t$) of the damaged area for the maximum intensity 255 ($I{M}_f$) was used.

$$\frac{\left(I{M}_f-I{M}_t\right)}{I{M}_f}\times 100=\mathit{Ratio} (\%)$$

(1)

$$\frac{P_{Da}}{P_{Fa}}\times 100=Ratio \left(\%\right)$$

(2)

3. Results and Discussion

3.1. A Program-Applicability Validation Evaluation

3.1.1. Color Difference Value Comparison Evaluation

This study compared variations in the color difference values of color information obtained using a colorimeter and a digital image analysis program (Figure 2). Following the comparison results, the silk’s standard color chart showed a color difference of 0.4–3.1 (

Table 2. Result of color standard chart (fabric) color difference value.

	Color Difference (ΔE)				A and B Shifting Value
	A (1 Point→2 Point)		B (2 Point→3 Point)
Method	Chroma Meter	Digital Image Analysis Program	Chroma Meter	Digital Image Analysis Program	Chroma Meter	Digital Image Analysis Program
Cinnabar	8.0	10.6	8.7	7.8	0.70	2.76
Hematite	10.7	11.6	2.9	1.8	7.80	9.83
Malachite	11.9	11.1	1.7	1.6	10.14	9.49
Azurite	22.2	24.2	6.4	7.7	15.74	16.52
Realgar	13.2	14.1	3.9	3.6	9.30	10.47
Orpiment	9.9	13.2	2.3	2.5	7.57	10.65
Carbon	4.5	5.2	7.6	7.6	3.13	2.38
Lead White	2.1	1.2	2.7	2.2	0.57	0.99

). Meanwhile, the paper’s standard color chart differed from at least 1.3 to as much as 5.6 (

Table 3. Result of color standard chart (paper) color difference value.

	Color Difference (ΔE)				A and B Shifting Value
	A (1 Point→2 Point)		B (2 Point→3 Point)
Method	Chroma Meter	Digital Image Analysis Program	Chroma Meter	Digital Image Analysis Program	Chroma Meter	Digital Image Analysis Program
Cinnabar	12.5	15.7	8.6	6.1	3.92	9.53
Hematite	11.2	9.5	3.5	3.1	7.73	6.32
Malachite	19.1	21.6	8.0	8.2	11.08	13.42
Azurite	22.6	23.2	19.0	21.5	3.61	1.75
Realgar	20.4	15.0	5.7	4.0	14.66	10.95
Orpiment	10.7	8.1	2.6	1.7	8.04	6.45
Carbon	9.7	8.2	11.2	16.9	1.57	8.73
Lead White	0.6	0.6	2.2	1.0	1.61	0.32

). When extracting color information using a colorimeter, differences in measured values occurred depending on the measurement object’s surface and the surrounding environment. If color information is extracted using a digital image analysis program, a difference occurs according to the used image’s photographing condition (a light source, number of lights, color temperature, etc.). Meanwhile, although differences in the acquired color information figures were observed, similarities in these figures’ color difference values were confirmed. The variation in the color difference value ranges from 0.4 to 5.6, visually recognizable as the same color in color recognition standards while ensuring reliability in using digital image analysis programs.

3.1.2. Comparative Evaluation of Work Efficiency by Image Resolution

In the original image’s resolution, the area of the base material with a color similar to the area to be extracted was also selected, with the 50% reduced image showing similar results. In the 10% and 25% reduced images, extracting only the areas that required high resolution was possible. For the 10% reduced image, a portion of a detailed area was calculated, but the accuracy of the detailed area was somewhat lower. It took about 90 min to process the original image, 60 min for the 50% reduced image, and 5 min for both the 10% and 25% reduced images (Figure 3). Because the performance of the computers processing the image and the digital image capacity affected the operation’s speed, there appeared to have been a difference in work speed according to the resolution. Low-resolution images had the advantage of fast working speeds but extracting clear and detailed areas was much more challenging compared to high-resolution images. Thus, this study showed efficient work on 25% reduced images similar to the full high-definition (FHD: 1920 × 1080 pixels) resolution. However, adjusting the resolution of digital images according to the purpose of the information obtained before image processing was necessary.

3.1.3. Comparative Evaluation of Operating Individual Deviation by Program and User Proficiency

• Program-specific deviation comparison.

When the damaged area of the painting was calculated using Adobe Illustrator, results showed 100% for user A, 94.6% for user B, 83.3% for user C, and 97.3% for user D, and the calculation took about 3–4 h. Meanwhile, when the area was calculated using PicMan and compared with user A, it was 97.1% for user B, 101.9% for user C, and 97.6% for user D, with the calculation taking only 10 min at most (Figure 4). By comparing the deviations between users for each method, the existing method of using Adobe Illustrator showed 2.7–16.7% interindividual variance. Meanwhile, 1.9–2.9% interindividual variance was observed for PicMan. The difference could be likely attributed to a worker omitting existing area calculation methods depending on the damage type, causing errors in selecting the damaged area and extracting the form. The digital image analysis program showed fewer deviations when processing images based on color information.

• Comparison of deviations by program proficiency

Regarding user proficiency, a difference of 0.2–3.68% was observed under advanced proficiency, where deviations of 0.5–2.3% and 0.2–2.8% in red and green areas were confirmed, respectively. Meanwhile, for the flesh area, a difference of at least 0.3% and a maximum of 2.4% was observed, while the white area had a deviation of at least 0.3% and a maximum of 3.7% (

Table 4. Results of calculating color area by proficiency.

Colors	Beginner	Intermediate	Advanced	Expert	Range
Red Color	17.47%	20.31%	19.22%	19.74%	0.52–2.84
Green Color	5.57%	3.19%	8.19%	5.37%	0.2–2.82
Skin Color	11.51%	14.22%	13.10%	11.79%	0.28–2.71
White Color	5.69%	8.78%	9.07%	9.37%	0.3–3.68

). Moreover, depending on proficiency, the minimum deviation for each color was less than 1%, and the maximum deviation was less than 3%, excluding the white area. For the beginner’s proficiency, the figure of the area calculated was generally smaller than that of other proficiency levels, which can be seen as a difference depending on the proficiency of the program. However, when calculating other proficiencies’ (intermediate, advanced, and expert) figures of the area, the size of the extracted area was not necessarily large just because the individual was highly skilled or proficient. As shown in Figure 5, when calculating the same color area, there was a difference according to proficiency. Even in an advanced image analysis program, the size of the area calculated was small if the proficiency in viewing colors was low. Conversely, even if there was a difference in the proficiency in handling the program, similar areas were calculated when the proficiency in viewing colors was similar. In this case, the worker’s proficiency determines the difference in information obtained through the results when verifying the color. However, in the digital image analysis program, the selected color region can be seen with figures, and quantitative data can be obtained regardless of proficiency if the category is recorded, and its figures are adjusted. Therefore, even if four or more users are compared, the results are considered to belong to a similar category.

3.2. Digital Image Analysis of Painting Cultural Heritage

3.2.1. Construction of Basic Cultural Heritage Information

Among the scientific analysis results of Ansimsa yeongsanhoe gwaebultaeng conducted in 2018 [28], chromaticity measurement information and pigment analysis information were collected. By collecting information on the results of the chromaticity measurement, it was found that white, black, red, green, yellow, and blue pigments were used, and

Table 5. Information on XRF and colorimeter measurement of Ansimsa yeongsanhoe gwaebultaeng [28].

No.	Color	Component	L*	a*	b*
1		Lead White [2PbCO3·Pb(OH)2]	80.8~84.4 (82.7)	−0.01~0.95 (0.50)	6.0~8.7 (6.9)
2		Ink Stick [C]	28.0~30.9 (29.4)	0.27~087 (0.57	2.2~4.7 (3.5)
3		Cinnabar [HgS], Minium [Pb3O4]	35.7~69.5 (54.3)	2.2~38.6 (21.0)	6.3~34.5 (19.3)
4		Orpiment [As2S3]	64.0~71.3 (67.9)	4.4~6.7 (5.8)	25.0~37.2 (33.2)
5		Atacamite [Cu2Cl(OH)3]	39.1~74.9 (58.2)	−18.6~2.4 (−8.1)	2.1~24.0 (14.2)
6		Azurite [Cu3(OH)2(CO3)2]	25.0~77.1 (42.4)	−4.5~1.4 (−1.8)	−10.0~6.2 (−3.2)

provides information on these pigments. In addition, types of damage to subjects identified by visual observation include creasing, the exfoliation of colored layers, contamination, separation, screen loss, and past repair (Figure 6).

3.2.2. Comparative Evaluation by Color Information Extraction Conditions

Although the mean value difference of RGB by measurement condition was not large, the categories of the minimum and maximum values of RGB decreased as the size of the measurement area and the number increased (Figure 7). After the 25 pix diameter circle (Average of circled ROIs), the minimum and maximum value categories were similar. When extracting color information from digital images, measurements should be made using the size and frequency of a certain area to reduce the error range of the measured data. In addition, the size of the measurement area should be smaller than the area to be extracted, the color information should be flexibly extracted according to the scope of color or damage, and the conditions of the digital image should be based on a worker’s judgment.

3.2.3. Image Analysis by Damage Type

Image analysis used images of the type of crease, exfoliation of painting layers, and contamination (Figure 8).

The results of the color information extraction of crease damage were R 233–237 (235), G 221–225 (223), and B 197–206 (202), and the value of color information similar to the target’s ground material appeared. In addition, the damage was mainly in horizontal directions. In the exfoliation of the painting layer, the types of damage were divided into plate type and powder type. Color information extraction results for plate-type exfoliation were measured as R 103–132 (119), G 108–132 (122), and B 83–104 (95). Meanwhile, the color information of the powder-type exfoliation was R 90–111 (97), G 110–122 (114), and B 90–98 (93), and the R-value changed depending on the damage type. In addition, the contaminated color information around red was R 137–167 (155), G 62–87 (71), and B 36–60 (47). Then, the contaminated color information around the green was R 72–99 (83), G 88–111 (96), and B 66–83 (71). The contaminated color information around the skin color was R 155–183 (164), G 119–157 (130), and B 62–110 (75). Moreover, RGB brightness levels were found to be lower than undamaged colors (

Table 6. Color information by damage type (RGB Format).

Type of Damage	Color Variable	Red	Green	Blue	Type of Damage	Color Variable	Red	Green	Blue
Crease	Min	233	221	197	Contamination (Red)	Min	137	62	36
	Max	237	225	206		Max	167	87	60
	Average	235	223	202		Average	155	71	47
Plate-Type Exfoliation	Min	103	108	83	Contamination (Green)	Min	72	88	66
	Max	132	132	104		Max	99	111	83
	Average	119	122	95		Average	83	96	71
Powder-Type Exfoliation	Min	90	110	90	Contamination (Skin Color)	Min	155	119	62
	Max	111	122	98		Max	183	157	110
	Average	97	114	93		Average	164	130	75

).

The location where the crack-type exfoliation occurred due to the crease on the screen was repainted, thus, the color information with the surrounding painting layer was similar. Therefore, to make the most characteristics occur horizontally, the x-axis was fixed at zero using the Relief function, and the shape was distinguished by moving the y-axis. The exfoliation of the painting layer increased the number of R-values according to the exfoliation progress. Using Color Mapping in the HSV space of the program, the HSV color space figures were adjusted and distinguished based on extracted color information. Contaminations were also distinguished by adjusting HSV color space numbers, such as the exfoliation of the painting layer, while contaminations in the red and skin colors could be calculated simultaneously due to the similar range of hue values. However, the contamination in the green was processed and calculated independently (Figure 9).

The extracted image was converted into a black and white (Black & White Binary) image, while the area of each damage type was calculated by substituting the mean value (Intensity Mean) of the entire color of the converted image into the equation. In addition, the crease area was 0.6%, the plate-type exfoliation was 10.5%, the powder-type exfoliation was 5.4%, and the contamination was 8.1%. If the image was not black-and-white, there would be a difference in the value calculated by a change in the Intensity Mean figure. Thus, applying the program technology to the full image resulted in an area of 2.1% for exfoliation of the painting layer, 2.1% for restoration material, and 0.1% for contamination (Figure 10).

The study confirmed that the color information of the damage types in painting cultural heritage possessed each characteristic. Differences in color according to the damage type differed depending on where it occurred, and, even if the difference in color change was significant, the degree of difference was not necessarily large. Therefore, when extracting color information from painting cultural heritage’ damage, it was necessary to identify the material’s characteristics that made up the target and the correlation between color and damage before proceeding.

4. Conclusions

A program-applicability validation evaluation confirmed the reliability of color information that could be obtained using a digital image analysis program. By applying the technology to the cultural heritage painting’s digital image, analyzing the color space of significant types of damage and identifying the characteristics of each damage type were possible. Based on the information, shape information on damage could be extracted, which allowed the damaged area of the painting cultural heritage to be quantitatively calculated. Color information analysis using the program should be based on a sufficient understanding of the characteristics constituting the painting cultural heritage materials and the color information of the digital image, rather than simply using the program’s function. In addition to the digital image analysis program used in this study, conducting complex processes, such as partial supplementation and passive image processing based on the experience and judgment of the worker according to the damage type, is also necessary. The technology used to assess the conservation status of painting cultural heritage using digital image analysis programs and color space information can be applied accurately and quickly compared to existing diagnostic methods. Objectively calculating the damaged area identifies the scope and extent of the painting cultural heritage damage, making it is possible to compare and analyze it. Quantitative numerical information on damage can be used as an essential basis for establishing conservation plans or risk evaluation criteria. In addition, the diagnosis method is expected to be used as monitoring data to determine progress and damage changes as data accumulates.