Measurement of Gem Colour Using a Computer Vision System: A Case Study with Jadeite-Jade

This paper introduces computer vision systems (CVSs), which provides a new method to measure gem colour, and compares CVS and colourimeter (CM) measurements of jadeite-jade colour in the CIELAB space. The feasibility of using CVS for jadeite-jade colour measurement was verified by an expert group test and a reasonable regression model in an experiment involving 111 samples covering almost all jadeite-jade colours. In the expert group test, more than 93.33% of CVS images are considered to have high similarities with real objects. Comparing L*, a*, b*, C*, h, and ∆E* (greater than 10) from CVS and CM tests indicate that significant visual differences exist between the measured colours. For a*, b*, and h, the R2 of the regression model for CVS and CM was 90.2% or more. CVS readings can be used to predict the colour value measured by CM, which means that CVS technology can become a practical tool to detect the colour of jadeite-jade.


Introduction
Colour is an important factor for evaluating the quality of a gem. Today, the diamond colour grading system introduced by the Gemological Institute of America (GIA) is relatively mature. There are colour grading methods for blue [1], pink [2], yellow [3], and Cape series diamonds [4]. Diamond colour grading was initially done by comparing Munsell colour cards [5] but later was mainly performed by professionally trained diamond graders to assist in visual judgment and grading. This colour judgment and grading method is highly subjective and cannot be popularised for all types of gems. It is, therefore, necessary to reduce the importance of subjective evaluation factors, and objectively as possible.
Stockton et al. first quantified the colour of gemstones in the CIE1931 space [6]. However, the poor perceptual uniformity of the CIE1931 colour space means that the geometric distance between two colour points in the colour space is often inconsistent with variations in human visual perception. To fit the colour perception of human eyes, CIELAB (same as CIE 1976 L*a*b*), a uniform colour space was introduced and applied to the study of gemstone colour. Studies using this colour space have now been done on alexandrite [7], colour-change garnet [8], blue amber [9], synthetic cubic zirconia [10], tourmaline [11], citrine [12], peridot [13] and jadeite-jade [14].
Current instruments used to measure the colour of gems include colour chips, colourimetric stones, ultraviolet-visible spectrophotometers, and colourimeters. Colour chips and colourimetric stones both assist the human eye to judge, but are still subjective assessments of colour and this methodology demands time and energy to train a professional gemologist. Although ultraviolet-visible spectrophotometers provide relatively objective and accurate colour measurements [9,15], their high costs, cumbersome operation process, and complex colour conversion prevent this method from gaining wide use for daily measurements of gem colour. Instead, colourimeters have become the mainstream tool for gem colour measurement because they are easy to use and provide reliable data [11][12][13][14]. However, colourimeters also have significant limitations; for example, a gemstone smaller than the aperture of the colourimeter leads to large errors, and colourimeters constitute a closed measurement mode and thus cannot reflect the pleochroism and discolouration of some varieties of gemstones.
Computer vision provides a new way to measure gem colour. Image processing based on computer vision has been widely used in many fields [16], such as printing, processing and manufacturing, the medical industry (e.g., automated endoscopic-image classification) [17], the food industry [18] (e.g., judging pork freshness) and agriculture [19] (automated recognition of fruit maturity).
In the field of earth science, with the continuous improvement in information and data-acquisition technology, computer vision applications need to be developed in various branches. These include online ore sorting (automatic classification of granite, limestone, etc.) [20][21][22] and mineral identification and classification using colour and texture analysis [23][24][25][26][27]. In these research areas, Oestreich et al. used digital image analysis to evaluate the mineral concentration of mixed minerals in the mines based on the colour vector angle [28], while Marshchallinger digitised mineral colours to automatically classify minerals on a macro scale [29]. For what concerns the colour analysis, Gokay et al. collected the colour codes of red, green, and blue (RGB), and the hue, saturation, and intensity (HIS) from 500 marble-surface images [30]. Chatterjee et al. collected 1200 images from 120 limestone samples and extracted five principal component features. Other work used artificial neural networks to classify ore-grade attributes, reliably predicting the ore grades of CaO, Al 2 O 3 , and SiO 2 in limestone [31]. Bianconi et al. extracted colour and texture features from twelve types of granite and used a support vector machine for their classification [32]. Aligholi et al. established a database of 45 standard thin sections based on seven optical properties of minerals (colour, pleochroism, interference colour, birefringence, opacity, isotropy, and extinction angle) and obtained an identification accuracy rate for biotite, amphibole, quartz, and calcite as high as 95% or more [33] to develop a colour-based automatic mineral identification program. They automated mineral searching by extracting the pixel point set of each of 3002 digital images of 79 standard mineral flakes to form a path composed of point sets in the CIELAB space [34].
The colour space used by computer vision may be categorised as device-dependent or device-independent [35]. Device-dependent spaces include RGB, HSV [36], YUV, YIQ, YCbCr [37], Ohta's I1I2I3 [38], and the opponent colour space RG-YeB-WhBl [39]. Deviceindependent colour spaces include CIE XYZ, CIELAB [40], and CIELUV [41]. Different colour spaces have different uses, and each has its own advantages and disadvantages. Qazi et al. compared RGB, IHLS, and CIELAB colour spaces and concluded that CIELAB provides a better description of colour texture [42]. Bianconi et al. used RGB, HSV, and CIELAB spaces to classify hardwood parquet and found little difference in performance between the three [43]. Młynarczuk et al. used 2700 microscopic images from nine different rock flakes to verify the classification of RGB, CIELAB, YIQ, and HSV; the mineral recognition rate in the CIELAB colour space was the highest, reaching 99.8% [44]. Bello-Cerezo et al. compared ten colour spaces (CIELAB, CIELUV, CIE XYZ, RGB, HSV, YIQ, YUV, YCbCr, Ohta's I1I2I3, and RG-YeB-WhBI) using a dataset of 12 different images. The performance of each space for material classification was evaluated, and the CIELAB space proved the best [45].
In China, jadeite-jade is known as the "king of jade". Chinese people love jadeite-jade as much as diamonds, so it is necessary to study the colour of it. Jadeite-jade, which generally occurs as a block, is a fibrous mineral aggregate composed of pyroxene minerals [46,47]. Pure jadeite-jade is colourless and transparent, but it is usually white due to the scattering of cracks, inclusions, and tiny grain edges. The body colour of jadeite-jade is divided into primary colours and secondary colours, including green, red, yellow, purple, blue and black. The mechanism of a primary colour is composed of metamorphic differentiation and ductile deformation, while the mechanism of a secondary colour is that impurity enters the crystal boundary voids or internal cracks. [48][49][50] Based on the uniformity and the widespread use of gem colours, the experiment reported herein used the equipment-independent CIELAB colour space to study and analyse the colour of jadeite-jade. Although the computer vision system has relatively mature applications in the field of earth science, it is a new attempt in gemstone colour. The purpose of this paper is to compare computer vision systems (CVSs) and colourimeter (CM) colour test methods, find how they are related, and analyse whether CVS can replace CM.

Samples
The experiments used 111 jadeite-jade samples coloured green (55), purple (28), and red-yellow (28). Most of the samples were polished cabochon shapes (length or diameter no more than 20 mm), with very few being special-shaped carvings.

Colour Space
The International Commission on Illumination (CIE) introduced the CIELAB (or CIE 1976 L*a*b*) uniform space to quantitatively express colours in 1976 [40]. The colour variables are L*, a*, b*, C* (chroma), and h (hue angle): L* is the luminance component, which ranges from 0 to 100 (from black to white), and the parameters a* (from negative green to positive red) and b* (from negative blue to positive yellow) are the chromaticity components of −120 to +120. C* and h are calculated from a* and b*: In the CIELAB colour space, the Euclidean distance between two points indicates the colour difference ∆Eˆ* of an object of the same size observed in the same neutral environment by a CIE standard observer using the CIE standard light source D65. ∆Eˆ* is given by ∆C * and ∆H * respectively represent the chroma difference and hue difference: where the subscripts CVS and CM indicate the colour parameters measured via the computer vision system and the colourimeter, respectively.

Colourimeter
The colour parameters of samples were measured in the CIELAB space by using an X-Rite SP62 portable spectrophotometer (4 mm aperture). The internal light source of the colourimeter was a gas-filled tungsten lamp, and an integrating sphere was used to collect the signal reflected from the sample surface. The test conditions included specular reflection, a D65 standard illumination source, and a 2 • observer field of view. Before each analysis, the device was calibrated by using the standard white tiles of the colourimeter.
The sample was placed on the N9 background of Munsell non-colour chips [51], and the experimental environment was selected from the colour assessment cabinet with a Munsell N7 neutral grey background. The final colour of each sample was the average of three measurements.

Computer Vision System
Gem colour was measured by using the CVS with the same Munsell N9 background and neutral environment ( Figure 1). tion, a D65 standard illumination source, and a 2° observer field of view. Before each analysis, the device was calibrated by using the standard white tiles of the colourimeter.
The sample was placed on the N9 background of Munsell non-colour chips [51], and the experimental environment was selected from the colour assessment cabinet with a Munsell N7 neutral grey background. The final colour of each sample was the average of three measurements.

Computer Vision System
Gem colour was measured by using the CVS with the same Munsell N9 background and neutral environment ( Figure 1). The images were captured by using a Mako G-507C industrial camera (manufactured by Allied Vision Technologies, Stadtroda, Germany). The camera has a resolution of 2464 (horizontal) by 2056 (vertical) and used a CMOS-type Sony IMX264 sensor with a size of type 2/3. The camera has a global shutter, a 50 mm focal length, and an aperture factor of 1.4.
The camera is positioned vertically so that the sample lens is 25 cm from the sample and connected to an Industrial Personal Computer (IPC) through a GigE interface. The camera's parameters were set by using Vimba Version 5.0 (a toolkit developed by Allied Vision). Use the colourimeter's standard white calibrated tiles to adjust the camera's white balance.
A shadowless dome light source with a colour temperature of 6500 K (input voltage 24 V, consumption current 1.40 A, consumption power 36 W) was positioned 10 cm above the sample. To best present the colour, the dome light source contained a special diffusereflection material so that the light evenly irradiates the surface of the cabochon jadeitejade sample from all angles. A Konica Minolta CL-200 Colour Illuminometer (Tokyo, Japan) served to measure the illumination at the sample position. The brightness of the light source was adjusted by using a controller so that the illumination at the sample position is 1500l [52].
The CVS operation process is shown in Figure 2. The image colour was extracted by using CKVisionBuilder V3.0 software (developed by Shenzhen CkVision Machine Vision The images were captured by using a Mako G-507C industrial camera (manufactured by Allied Vision Technologies, Stadtroda, Germany). The camera has a resolution of 2464 (horizontal) by 2056 (vertical) and used a CMOS-type Sony IMX264 sensor with a size of type 2/3. The camera has a global shutter, a 50 mm focal length, and an aperture factor of 1.4.
The camera is positioned vertically so that the sample lens is 25 cm from the sample and connected to an Industrial Personal Computer (IPC) through a GigE interface. The camera's parameters were set by using Vimba Version 5.0 (a toolkit developed by Allied Vision). Use the colourimeter's standard white calibrated tiles to adjust the camera's white balance.
A shadowless dome light source with a colour temperature of 6500 K (input voltage 24 V, consumption current 1.40 A, consumption power 36 W) was positioned 10 cm above the sample. To best present the colour, the dome light source contained a special diffusereflection material so that the light evenly irradiates the surface of the cabochon jadeite-jade sample from all angles. A Konica Minolta CL-200 Colour Illuminometer (Tokyo, Japan) served to measure the illumination at the sample position. The brightness of the light source was adjusted by using a controller so that the illumination at the sample position is 1500l [52].
The CVS operation process is shown in Figure 2. The image colour was extracted by using CKVisionBuilder V3.0 software (developed by Shenzhen CkVision Machine Vision Technology Co., Ltd., Shenzhen, China). The image processing algorithm was packaged in the supporting CKVisionSDK 3.0. The user drags the tool to test the colour of a specific area of the image, and the software automatically converts it into L*, a*, b* values according to the average pixel RGB value of the selected area. Avoiding the reflective position, we selected three uniformly coloured areas from the sample image and calculated the average L*, a*, b* values to serve as the colour test values of the sample's CVS.  During the test by the expert group, the image was displayed on a 13.3-inch screen of a MacBook Air (resolution 1440 × 900) with a built-in Intel HD Graphics 5000 1536 MB graphics card. We used Adobe Photoshop CC 2019 to make simulated colour blocks for During the test by the expert group, the image was displayed on a 13.3-inch screen of a MacBook Air (resolution 1440 × 900) with a built-in Intel HD Graphics 5000 1536 MB graphics card. We used Adobe Photoshop CC 2019 to make simulated colour blocks for the colour parameters in CM and CVS.

Expert Group Test
We collaborated with an expert observation team composed of ten gemologists, each with a Fellowship of the Gemological Association and Gem Testing Laboratory of Great Britain (FGA). Each expert had almost perfect gem colour sensitivity. The team members observed the images at 50 cm from the screen and independently judged whether the images were similar in colour to the jadeite-jade samples in the CVS environment. They had 30 s to answer "yes" or "no." If they answered "yes", they gave each sample a rating of 1-5 on the five-point Likert scale: 1 = "very low," 2 = "low," 3 = "moderate," 4 = "high," 5 = "very high." A Likert score of 3 or more occurring over 50% of the time was taken to indicate that the colours in the CVS photos were similar to those of the actual samples.

Statistical Analysis
Independent-sample t-tests were applied to the CM (Colourimeter) data and CVS (Computer Vision System) data, and the significance of the difference between them was judged based on the F value from Levene's homogeneity-of-variance test. We estimated the curve using IBM SPSS Statistics 25 to obtain the regression model for CM from CVS coordinates. To predict CM colour coordinates from CVS colour coordinates, the colour coordinates obtained by CM and CVS served as independent and dependent variables, respectively.
Of the 111 samples with three colours, 100 samples were used to estimate the regression equation, and the remaining 11 samples were used to validate the regression model. The 11 samples used for verification were randomly selected under restricted conditions. The limiting condition was that the L*, a*, b*, C*, and h values measured by the CM for these 11 samples were included in the 100-sample group used to estimate the regression. Residuals were used to test the correctness of the model, where a residual in mathematical statistics refers to the difference between the measured value and the estimated value (fitted value). The residual reflects the error caused between the estimated regression equation and the measured values.
All data analysis was done by using IBM SPSS Statistics 25 software.

Expert Group Validation
Ten experts evaluated 111 samples for a total of 1110 tests. The test results (Table 1) indicated a strong overall similarity between the CVS-measured images and the actual samples. For the green tones, 1.64% were rated as 2 (low), with the remaining 98.36% and all purple and red-yellow colours rated as 3 (moderate) or higher. Ratings of 4 (high) and 5 (very high) were 88.91%, 98.93%, and 96.43% for the three tones of jadeite-jade, respectively, which is over 93.33% overall. This indicated that the colour characteristics of the sample in the digital image displayed on the monitor were very similar to the actual colour characteristics of the sample. The CVS technology used herein thus provided high colour fidelity of the sample images. Therefore, this method could be used to evaluate the colour of jadeite-jade.  Figure 3 shows the distribution of sample colour in the CIELAB space measured using CM and CVS, along with photographs of some samples. Table 2 shows the specific colour parameters. The results in Table 2 show that ∆L* was negative for the three colour samples, and the lightness measured by the CM was greater overall than that measured by the CVS.  Figure 3 shows the distribution of sample colour in the CIELAB space measured using CM and CVS, along with photographs of some samples. Table 2 shows the specific colour parameters. The results in Table 2 show that ∆L* was negative for the three colour samples, and the lightness measured by the CM was greater overall than that measured by the CVS.  For the green jadeite-jade, L*, a*, and h measured by the CVS and CM differed significantly (P < 0.05), whereas b* and C* did not (P > 0.05). a* was negative, and the absolute value of a* measured by the CM was greater than that measured by the CVS, indicating that the "green" measured by the CM was stronger. The parameter b* was positive, and the absolute value of b* measured by the CVS was greater than that measured by the CM, indicating that the sample was more "yellow" according to the CVS.

Comparison between the CVS and the Colourimeter Measurements
The parameter h measured by the CM was greater than that measured by the CVS and was offset counterclockwise in the CIELAB space. According to the colour distribution in the CIELAB space, green jadeite-jade tended to be blue for the CM and yellow for the CVS. A negative value of ∆C* indicated that the colour saturation of green jadeite-jade for the CVS was below that of the CM.
For purple jadeite, significant differences appeared between b* and C* measured by the CVS and CM (P < 0.01), but no significant differences appeared in L*, a*, or h (P > 0.05). There was little difference between a* measured by the CVS and by the CM, but the difference for b* was large, and b* was negative, indicating that the sample had a stronger "blue" hue according to the CVS. The h measured by the CM was greater than that measured by the CVS and shifted counterclockwise in the CIELAB space, indicating that purple jadeite-jade tended toward red (blue) when measured by the CM (CVS). A positive ∆C* indicated that the sample had higher colour saturation when measured by the CVS.
For the red-yellow samples, only L* differed significantly (P < 0.001). The differences in C*, a*, b* when measured by the two methods were not obvious. The value of h produced by the CM was less than that produced by the CVS so the CM value was clockwise in the CIELAB space with respect to the CVS value. Level of significance: * = P < 0.05; ** = P < 0.01; *** = P < 0.001.
The overall colour difference between CM and CVS could be represented by ∆E*. As shown in Table 2, the colour differences ranged from 10.85 for purple to 18.89 for green. ∆E* = 3 is the threshold at which the human eye perceives a colour difference. When ∆E* is in the range 3-6, the human eye perceives a colour difference, but this difference is acceptable. When ∆E* > 6, the colour difference is not acceptable [53]. The average colour difference of the three tones was greater than ten, indicating that the colours of the jadeite-jade samples measured by the two systems differed significantly for the human eye. The colour difference between the two methods was shown more intuitively by using simulated colour blocks (Figure 3).

Regression Model
Considering R 2 and the maximum P-value of each regression coefficient (P cannot exceed 0.05), Figure 4 and Table 3 show the optimal regression of CM and CVS. The linear relationship was only significant for h (Figure 4d), and the best regression model for L*, a*, b*, C* was a quadratic function. The R 2 values for a*, b*, and h were 90.2% and above, indicating a high-quality fit. Therefore, 90.2% or more of the values a*, b*, and h measured by the CM could be explained by the CVS. The parameters a*, b*, and h are commonly used to determine the colour of jadeite-jade, and they all have very high R 2 values, indicating that the CVS could be used to evaluate the colour of jadeite-jade. The lower R 2 of L* and C* may have been due to the different light distribution conditions of the CM versus the CVS, as explained in detail in the discussion.
indicating a high-quality fit. Therefore, 90.2% or more of the values a*, b*, and h measured by the CM could be explained by the CVS. The parameters a*, b*, and h are commonly used to determine the colour of jadeite-jade, and they all have very high R 2 values, indicating that the CVS could be used to evaluate the colour of jadeite-jade. The lower R 2 of L* and C* may have been due to the different light distribution conditions of the CM versus the CVS, as explained in detail in the discussion.   The residual mean value of each colour parameter in the regression model was zero (Table 4), which meant that the actual value was evenly distributed on both sides of the regression line, indicative of a very reasonable regression model. The average differences between predicted and measured values of L*, a*, and b* of the 11 samples used for back-substitution verification were −2.37, −0.01, and 1.32, respectively. The calculated colour difference was 2.63, which is less than three, which also shows the reliability of the regression model.

Discussion
In the investigated samples, the colour difference ∆E* between the measurements with CM and CVS is greater than 10, which indicates that the two methods give different colours. In this paper, we believe that one of the parameters contributing to the different results for jadeite-jade colour measurements is the depth of penetration of the light source, which depends on the irradiation mode of the light source and the transparency of jadeite-jade itself. The colourimeter is close to the surface of the sample during the test, and all the light is emitted from a circular opening of 4 mm in diameter. The CVS light source is placed 10 cm from the sample, and the light is reflected evenly from the surface of the sample after diffuse reflection. Therefore, the transmission depth for the CM is greater than that for the CVS, so the overall L* of the CM test is also greater.
Jadeite-jade is a polycrystalline aggregate, and the internal particles differ in size and are unevenly distributed, which varies the transparency, so the light entering the bulk of the aggregate undergoes multiple reflections and refractions at interfaces between particles, with the number of such interactions increasing with penetration depth. This leads to the difference between the CM and CVS measurements. The specific relationship between transparency and measurement results needs to be verified by subsequent experiments. The light distribution and photographic position are the same as for observations by the eye of jadeite-jade samples, and the CVS produces results that are highly consistent with those of the expert group. Therefore, we deduce that the CM is not suitable for measuring the body colour of gems with a certain degree of transparency.
The samples in this experiment were mostly jadeite-jade of uniform colour. However, some jadeite-jades may be bicolour or tricolour. Due to the limited aperture, the CM can only measure a circular area with a minimum diameter of 4 mm, which is not sufficiently precise for gems. In contrast, the CVS can select the colour area and adjust the shape and size of the colour box (the minimum size is accurate to the pixel level). Another advantage of the CVS is that images can be stored in a database for later research and comparison. All these points further highlight the limitations of CMs.
Although the CM is still the mainstream tool for gem-colour quantification, the CM conditions differ significantly from the observation conditions of the human eye. Combined with the research results of this paper, CVSs can better replace CMs as a new tool to measure the colour of gems, as follows: 1. The CVS conforms to the observation environment and conditions of the human eye.
2. An effective, reasonable, and reliable regression model exists between the CM and the CVS.
This study focuses on only a single gemstone variety, jadeite-jade, and the experimental subjects were single jadeite-jade specimens, although the samples almost covered all the colours of coloured jadeite-jade. In the future, it is necessary to inspect gems of different varieties and transparency to determine the versatility and reliability of the CVS in gem colour testing.
With the rapid development of computer technology and artificial intelligence, gem colour measurement needs a new revolution. Using the CVS to measure the colour of gems can eliminate of a lot of subjective factors and establish a system that is in line with the gem industry, which is different from other fields, and lay the foundation for the automatic grading of gems. In the context of informatisation, it can promote the transformation and upgrading of the jewelry industry.

Conclusions
Most previous gem-colourimetry studies used colourimeters to quantify colour, despite their significant limitations and inflexibility. Compared with other instruments, the computer vision system requires less manpower and financial resources and is easy to operate and read data. It can not only simulate the observation mode of human eyes but also overcome the limitation of subjective factors of human eyes. As a new tool, the CM measurement can be accurately predicted by the CVS, so the CVS can achieve the same test results as the CM and even outperform CM in some aspects. The CVS can better restore the colour of transparent gems seen by human eyes than the CM, it is not limited by the aperture and can measure the colour of any size area. Another incomparable advantage of the CVS is that it can store the sample image at the same time of measurement, which provides the conditions for the establishment of a gem database in the future, to carry out the automatic grading of gems. Therefore, the computer vision system is a superior technique for measuring the colour of gems.