Evaluation of Color Difference Models for Wide Color Gamut and High Dynamic Range

Abstract

Color difference models (CDMs) are essential for accurate color reproduction in image processing. While CDMs aim to reflect perceived color differences (CDs) from psychophysical data, they remain largely untested in wide color gamut (WCG) and high dynamic range (HDR) contexts, which are underrepresented in current datasets. This gap highlights the need to validate CDMs across WCG and HDR. Moreover, the non-geodesic structure of perceptual color space necessitates datasets covering CDs of various magnitudes, while most existing datasets emphasize only small and threshold CDs. To address this, we collected a new dataset encompassing a broad range of CDs in WCG and HDR contexts and developed a novel CDM fitted to these data. Benchmarking various CDMs using STRESS and significant error fractions on both new and established datasets reveals that CAM16-UCS with power correction is the most versatile model, delivering strong average performance across WCG colors up to 1611 cd/m². However, even the best CDM fails to achieve the desired accuracy limits and yields significant errors. CAM16-UCS, though promising, requires further refinement, particularly in its power correction component to better capture the non-geodesic structure of perceptual color space.

1. Introduction

With rapid advancements in high dynamic range (HDR) display technology—now achieving luminance levels exceeding 1000 cd/m² in accordance with new HDR standards [1,2], which also incorporate wide color gamut (WCG) defined in the earlier BT.2020 standard [3]—tools for precise color reproduction assessment have become essential. Color difference models (CDMs) play a crucial role by estimating perceived color differences (CDs) between colors. Mathematically, a CDM can be expressed as a function $F(c_1,c_2,p)\to d$, where d is the perceived difference between colors $c_1$ and $c_2$, and p represents the observation parameters. A basic CDM approach uses the Euclidean distance in a uniform color space (UCS), described as $F(c_1,c_2,p)=\left|\right|T(c_1,p)-T(c_2,p){\left|\right|}_2$, where T is the transformation from the colorimetry color space to the particular UCS. A UCS can be used not only as a CDM but also as a space for performing color transformations. It has been shown that the performance of image processing algorithms significantly depends on the chosen CS or CDM, as demonstrated in applications such as tone mapping [4], color correction [5], and image recognition [6].

CDMs are developed and validated using datasets comprising color pairs and their perceived differences, as determined through psychophysical experiments. While various CDMs have been proposed for HDR contexts [7,8,9,10,11], HDR- and WCG-specific experimental data to evaluate these models are still sparse. Existing datasets, often collected based on textile and printed samples [12,13,14,15], lack WCG coverage, and recent monitor-based datasets are confined to luminance levels up to 300 cd/m² [16,17]. Thus, dedicated experimental setups are necessary to acquire data covering both HDR and WCG ranges.

There are various types of experiments to assess CDs with different magnitudes: threshold CD (TCD) or just noticeable difference (JND), small CD (SCD), and large CD (LCD, exceeding 5–8 JNDs). Measuring CDs across these ranges is critical due to the non-geodesic nature of perceptual color space, where the cumulative JNDs along the shortest path (geodesic) can surpass the direct CD between endpoints [18,19,20]. This discrepancy challenges traditional UCS models using Euclidean metrics for varying CD magnitudes. Consequently, UCSs specifically designed for LCD or SCD applications based on the CIECAM02 model [21] have been developed, and a generalized approach using power functions (with exponents $<1$) has been applied to current UCS models [22], including the state-of-the-art CAM16-UCS [23].

A recent review by Luo et al. [24] evaluated psychophysical datasets on CDs from over 20 studies, benchmarking CDMs across these datasets. For decades, such datasets have played a central role in the iterative development of CDMs, forming the foundation for refining gold-standard models widely used in color evaluation. This progress has, in turn, driven advancements in applied algorithms. While HDR datasets [17] (up to 1128 cd/m²) and WCG datasets [25] (covering the BT.2020 gamut) exist, the review acknowledges a significant gap: the absence of datasets encompassing both HDR and WCG. Additionally, existing HDR and WCG datasets are limited in their coverage of large CDs. Large CDs are particularly important in display manufacturing, where defective pixels significantly affect perceived color [26]. Addressing these gaps is critical for further advancing UCS development, necessitating new datasets that cover WCG, HDR, and large CD ranges. To this end, we obtained a new color difference dataset spanning threshold, small, and large CDs in HDR and WCG. Using this dataset, we developed an analytically fitted CDM and benchmarked several CDMs with STRESS and significant error fraction metrics across both new and established datasets, providing a robust foundation for future UCS development.

2. Materials and Methods

2.1. Developed Experiment Setup

To obtain psychophysical data across HDR and WCG color spaces at various color difference (CD) scales, we developed a custom experimental setup centered around a purpose-built stimulus generator. Since no commercially available device met the required HDR and WCG specifications, we collaborated with Visionica Ltd. (Moscow, Russia) (http://www.visionica.biz/, accessed 8 November 2024) to design a stimulus generator, named VCD. VCD provides four independent stimuli, each comprising several dozen LEDs per primary color, ensuring both high luminance (up to 5000 cd/m²) and uniformity of the light field.

Our goal was to cover as much of the BT.2020 gamut as possible using the VCD. While the BT.2020 [1] gamut triangle is theoretically defined, its primaries, located on the spectral locus, can only be practically achieved using lasers. With LEDs, it is feasible to achieve two out of three primaries with adequate luminance levels, but achieving a high-saturation green LED with sufficient luminance remains a challenge. This issue, known as the “green gap” [27], arises from the physical limitations of current LED technology. To address this, we incorporated two green LEDs with different spectral power distributions, resulting in a total of four LEDs in the VCD. This configuration expanded the device’s color gamut (see Figure 1). Adding more LEDs would provide only marginal gains in gamut coverage while significantly complicating the device design.

Figure 1. The VCD gamut (blue), BT.2020 (red), human visual gamut (gray), and sRGB gamut (colored) shown on the proLab chromaticity diagram.

The full experimental setup included a light booth with two rear windows (for different scenarios), an illuminator, and the custom stimulus generator discussed above. Figure 2 shows views of the setup.

Figure 2. Front and side views of the experiment setup.

The light booth, a 70 × 70 × 70 cm cube with five white paper walls, was top-lit by a Remez 7W 5700K LED lamp (color rendering index: 98.5 and correlated color temperature: 5553 K), providing 100 cd/m² at the center of the back wall and ensuring uniform lighting. Spectral measurements were conducted using an X-Rite I1Pro spectrophotometer (3.3 nm resolution) with ArgyllCMS software (version 2.1.2) (http://argyllcms.com/, accessed 8 November 2024). Figure 3 shows the spectral power distributions (SPDs) inside the booth alongside CIE D50, D55, and D65 illuminants.

Figure 3. Spectral power distributions of the luminance inside the booth compared to CIE D50, D55, and D65 illuminants.

The VCD generator produced four stimuli (10 × 10 cm quadrants), each controlled independently. A 1 mm vertical separator divided the stimuli horizontally, with a 2 cm space between the upper and lower pairs; see Figure 4. The VCD was positioned 17 cm from the booth’s back wall, and a black separator was placed between viewing windows. Observers sat 80 cm from the windows, ensuring an angular stimulus size of 3.58°.

Figure 4. Stimuli setup for SCD experiments. Quadrants: Q1, upper right; Q2, upper left; Q3, lower right; and Q4, lower left.

For TCD measurements, a 28 mm round hole in the back wall provided a 2° angular stimulus size; see Figure 5.

Figure 5. Interior of the light booth for TCD measurements.

For SCD and LCD measurements, two 5 × 5 cm rectangular windows separated by 2 cm were used. The chinrest was adjusted to block the observer’s view of gaps between the VCD quadrants.

2.1.1. VCD Stimulus Generator Characterization

The VCD stimulus generator produced four independently controlled stimuli, each in a separate quadrant (Figure 4). Each quadrant featured a board with four primary LEDs (see Section 2.1.2) controlled at 16-bit accuracy, with settings managed remotely by a computer.

Uniformity testing via photographs revealed a 15% luminance drop along the left edge of Q1 near Q2, consistent across quadrants and imperceptible to the human eye. This variation was negligible for SCD and LCD measurements (see Section 2.2.2) but became significant for TCD if colors were compared side by side. Therefore, we employed an alternate procedure for TCD, where colors were flashed alternately within the same quadrant (see Section 2.2.1).

2.1.2. Primaries Spectra

The VCD featured four primary channels—red (R), green (G), blue (B), and cyan (C)—each controlled independently via 16-bit buses, allowing luminance values from 0 to 65,535. Spectral power distributions (SPDs) were measured for various channel values in Q1 (Figure 6), with settings of 30,000 for R and B yielding luminance around 1000 and 1400 cd/m², and 5000 for G and C yielding around 1000 and 1100 cd/m². Minor SPD shifts due to LED heating, most notable in the green channel, were consistent across quadrants and accounted for in the VCD control software. Luminance linearity was assessed by measuring primary outputs at various channel values, approximated by a parabolic function to ensure accuracy (Figure 7).

Figure 6. Spectral power distributions of Q1 primaries R, G, B, and C.

Figure 7. Luminance vs. channel value for Q1 primaries (left to right, top to bottom: R, G, B, and C). Dots illustrate measurements; dashed lines represent ideal linearity; and solid lines, parabolic approximation.

To verify calibration, we compared the chromaticity coordinates of the target colors that the VCD intended to produce with the actual colors formed, which were measured using an X-Rite I1Pro spectrophotometer in its colorimeter mode. Approximately 40 different xyY values, evenly distributed across the chromaticity diagram at two luminance levels (300 and 1000 cd/m²), were tested. Chromaticity errors were calculated as the distance between the measured and target colors on the CIE 1964 xy chromaticity diagram for the 10° observer. The chromaticity error was found to depend on both the y and Y coordinates, ranging from 0.007 (for y < 0.5 at 1000 cd/m²) to 0.038 (for y ≥ 0.6 at 300 cd/m²), with a mean error of 0.017, which is considered small.

2.2. Experimental Procedures

2.2.1. TCD Measurements

TCD measurements were conducted using the “strict substitution” technique [28,29]. In this method, a color center (reference color, Stimulus A) is displayed, and a test color (Stimulus B) is shifted in a chosen direction within the color space away from the color center. Each stimulus is shown for 1.5 s in a periodic, instantaneous alternation.

Observers are given unlimited time to detect any difference; if none is observed, they increase the shift of Stimulus B by pressing a button. The just noticeable difference (JND) is the smallest shift at which the observer perceives a distinction between the two stimuli.

Stimulus B is shifted in discrete steps along specific directions in CIE xyY space. While a smooth surface would require measurement in numerous directions, which was not feasible, we used 18 directions combining chromaticity and luminance shifts and 8 directions with fixed luminance to approximate this surface (details in Figure 8).

Figure 8. Arrangement of 18 measurement directions around the color center in xyY space. Blue points indicate chromaticity-only shifts ($\Delta \mathrm{Y}=0$), yellow points show luminance shifts ($\Delta \mathrm{x}=0,\Delta \mathrm{y}=0$), and red points represent combined chromaticity and luminance shifts. (a) 3D view; (b) section showing chromaticity-only shifts; (c) section with x-coordinate fixed at 0; (d) section with y-coordinate fixed.

2.2.2. SCD and LCD Measurements

SCD and LCD were measured using the “gray-scale method”, which matches color differences to a gray reference scale [25,30,31,32]. Observers viewed two pairs of colors: a reference gray pair with a known difference (3 JND for SCD, 12 JND for LCD) and a test pair.

The viewed stimuli are shown in Figure 4. The reference gray pair (stimuli Q1 and Q2) had a fixed difference, while the test pair (Q3 and Q4) started at the same color center (Q4). Observers incrementally adjusted Q3 along one of 18 preset directions (Figure 8) until its perceived difference from Q4 matched that of the gray reference pair. Upon matching, the color center and the selected shift were recorded.

The reference pair had chromaticity coordinates corresponding to D65. One sample in the reference pair remained fixed as GS-1, with the other as GS-3 for SCD or GS-5 for LCD measurements (see Table 1). For improved observer’s brightness adaptation, gray-scale luminances were increased 20-fold, with a reference white luminance of 2000 cd/m², as detailed in Table 1, where ${\mathrm{L}}^{*}$ denotes the CIELAB lightness value.

Table 1. Gray-scale parameters from [25] and in our experiments.

Gray-Scale Number	${\mathit{L}}^{*}$	$\Delta \mathit{E}$ with GS-1	cd/m2 [25]	cd/m2, Ours
GS-1	40.0	0.0	11.2510	225.02
GS-2	41.5	1.5	12.1795	243.59
GS-3	43.0	3.0	13.1578	263.16
GS-4	46.0	6.0	15.2687	305.37
GS-5	52.0	12.0	20.1443	402.89

2.2.3. Operating Conditions

Human color perception is strongly influenced by viewing conditions. For instance, the perceived magnitude of color differences depends on the chromatic adaptation of the human visual system to dominant illumination [33,34]. Chromatic adaptation transforms (CATs) describe these dependencies [23,35] and are often integrated into UCSs [23,36]. Developing such models requires the collection of specialized datasets [37,38]. However, it is not feasible to cover all possible scenarios within a single dataset, as this would require sampling a highly multidimensional parameter space. Consequently, different studies typically focus on specific subsets of this space. In our work, we addressed a four-dimensional space comprising three color coordinates and the magnitude of color difference. Nonetheless, datasets targeting other regions of the parameter space are still required to further improve these models. We maintained consistent viewing conditions throughout the experiment. Table 2 summarizes potential biases introduced by the experimental setup and observers, as well as the measures taken to mitigate them. Despite these efforts, some residual biases may remain. These biases can be categorized into two types: biases consistent across all measurements, and biases that vary between measurements. To assess the impact of the first type, we verified that our measurements aligned with previous datasets and evaluated their consistency, as described in Section 3.1. For the second type, we confirmed that such biases were not present as the model performance was not undermined: the CDM optimized on the training subset of our measured data achieved high accuracy on the validation subset, as detailed in Section 3.2. These two independent verifications demonstrate that our dataset collection process was not significantly influenced by large biases.

Table 2. Potential biases and the measures taken to address them.

Potential Biases	Mitigation Measures
LEDs in different quadrants may vary.	LEDs were sourced from the same model and production batch. Each quadrant was calibrated individually.
LED spectral power distribution may shift due to LED heating.	The VCD was pre-warmed before use. A fan was installed to maintain a stable temperature during operation, and the LED temperature was monitored, ensuring it remained below 40 °C.
LED spectral power distribution may shift due to room temperature fluctuations.	Room temperature was stabilized at 24 °C using air conditioning to maintain consistent LED spectral output.
The VCD may exhibit day-to-day performance variability.	Random colors were verified for accuracy using a spectrophotometer during measurements.
Room lighting may vary throughout the day.	Experiments were conducted in a fully darkened room, with D50 adaptation uniform lighting within the viewing cabin.
Observers may have color vision deficiencies.	Observers were tested using the Color Assessment and Diagnosis test, and those with color vision anomalies were excluded.
Observers may have individual variations in visual systems and color perception.	Measurements were stored for each individual separately without averaging across subjects, recognizing the importance of human diversity in perception.
Observers’ visual systems may adapt to conditions other than those in the experiment.	Observers were allowed to adapt to the experimental viewing conditions for at least 1 min before testing.
Observers may have misunderstood the task.	A learning session was conducted for each observer to discuss and clarify their observations and tasks.
Observers may adjust the number of steps arbitrarily, taking approximately the same number to find the desired color difference.	Measurement directions were randomized, and each color pair was measured four times in different directions. Medians of the measurements were used for analysis.
Observers may experience fatigue, leading to less accurate observations.	Each observer completed 2–3 sessions per day with breaks between sessions. Sessions lasted approximately 40 min, requiring several days of measurements per observer.

2.3. Collected Data

We collected comprehensive human color perception threshold data, covering small and large color differences (CDs) across 136 color centers and 2702 color pairs:

• TCDs were measured in 109 color centers (2046 pairs, 17 participants);
• SCDs were measured in 18 color centers (342 pairs, 9 participants);
• LCDs were measured in 17 color centers (314 pairs, 9 participants).

Each measurement captured three-dimensional data, forming ellipsoids in WCG and HDR areas, with color center luminances up to 1000 cd/m² and a maximum luminance of 1611 cd/m². Figure 9 shows the chromaticity and luminance distribution of the collected data. The convex hull of the collected color pairs forms a polygon covering 80% of the BT.2020 or BT.2100 [1] area on the proLab chromaticity diagram [36]. ProLab is a UCS based on the 3D projective transformation of CIE XYZ, with chromaticity coordinates similar to CIE xy: $x^{+}=a^{+}/L^{+}$, $y^{+}=b^{+}/L^{+}$, where $L^{+}$, $a^{+}$, and $b^{+}$ are coordinates in proLab.

Figure 9. Collected data shown on the proLab chromaticity diagram (left) and corresponding luminance Y in cd/m² (right). The centers of the blue segment intersections indicate the centers of the measured ellipses, while the segments themselves represent the measured color differences.

The color centers were selected using two approaches. First, we measured color differences at the color centers recommended by the CIE [39] to validate our setup by comparing the results with previously published data. Second, we randomly selected color centers from a uniform distribution in proLab for luminance levels ranging from 0 to 1000 cd/m². This selection excluded the sRGB color gamut (up to 300 cd/m²). Consequently, for the standard luminance range, we focused on highly saturated colors, while, for HDR, we explored the full saturation range.

CDs were measured in 18 directions in $\mathrm{xyY}$ space, including 8 in the chromaticity plane (Figure 8). In each session, the following steps were taken:

• A total of 8 pairs were measured with $\Delta \mathrm{x}$ and $\Delta \mathrm{y}$ shifts (fixed $\Delta \mathrm{Y}$);
• If the observer was not overly fatigued, we measured 2 pairs with positive/negative $\Delta \mathrm{Y}$ (fixed $\Delta \mathrm{x}$, $\Delta \mathrm{y}$) and 8 pairs with combined $\Delta \mathrm{x},\Delta \mathrm{y}$, and $\Delta \mathrm{Y}$ shifts.

A total of 231 measurement sessions were conducted, each with an observer focusing on one color center (data in certain directions were occasionally omitted if the observer reached the device’s gamut limits):

• A total of 8 directions were measured in 150 sessions (19 had partial data due to gamut limits);
• A total of 18 directions were measured in 81 sessions (3 had partial data due to gamut limits).

The order of color centers and measurement directions was randomized per observer. Each color pair within a direction was measured four times to minimize variability (following CIE recommendations [40]), using medians for analysis.

We recruited 22 observers with normal or corrected vision, who passed the Color Assessment and Diagnosis test within age-appropriate thresholds. All participants received instructions and provided informed consent.

2.4. Available Datasets

The widely used COMBVD dataset [41] includes small CDs (under 5 CIELAB units) and consists of four sub-datasets: BFD-P [12] with 2779 pairs, Witt [13] with 418 pairs, Leeds [14] with 307 pairs, and RIT-DuPont [15] with 312 pairs. We used this dataset to validate our own data and for CDM benchmarking. Additionally, we used the Size-dependent Color Threshold (SDCTh) dataset [42], which includes 80 pairs across 10 color centers, each measured by 10–13 observers to obtain 8 median pairs per center for stimuli with a radius of 72 min. Both datasets cover sRGB, limited to a luminance of 100 cd/m² (standard dynamic range, SDR). HDR and WCG datasets exist [17,25] but are not publicly available and were thus not used here.

Table 3 summarizes the viewing conditions for each dataset (illuminant, adapting luminance $L_a$, luminance factor of the neutral background $Y_b$, and surround). Our booth lighting was set to 100 cd/m², i.e., $L_a=100$. Since stimuli were displayed against a white background, $L_b=L_w=100$, resulting in $Y_b=100(L_b/L_w)=100$.

Table 3. Viewing condition parameters for experimental CD datasets.

Dataset	Illuminant	${\mathit{L}}_{\mathit{a}}$	${\mathit{Y}}_{\mathit{b}}$	Surround
BFD-P	D65, M, C	100	20	Average
RIT-DuPont	D65	127.3	10.9	Average
Leeds	D65	100	20	Average
Witt	D65	82.8	24.9	Average
SDCTh	D65	80	50	Average
Ours	D50	100	100	Average

2.5. Quality Measures of Color Difference Models

The similarity between predicted CDs and reference measurements from human studies is commonly evaluated using the STRESS measure [43,44]. Let $\overrightarrow{x}\in {\mathbb{R}}^n$ represent the vector of experimental CDs for n color pairs from a psychophysical dataset, and $\overrightarrow{y}\in {\mathbb{R}}^n$ represent the vector of CDs for the same color pairs computed using a given CDM. The STRESS metric, $\mathrm{STRESS}(\overrightarrow{x},\overrightarrow{y})$, is defined as

$$\mathrm{STRESS}(\overrightarrow{x},\overrightarrow{y})={\displaystyle \frac{\parallel k^{*}\overrightarrow{x}-\overrightarrow{y}{\parallel }_2}{\parallel \overrightarrow{y}{\parallel }_2}}=\sqrt{{\displaystyle \frac{{\displaystyle \sum _{i=1}^n}{(k^{*}{x}_i-y_i)}^2}{{\displaystyle \sum _{i=1}^n}{y}_i^2}}},$$

(1)

where $k^{*}$ is a scaling factor used for normalization:

$$k^{*}=\underset{k}{argmin}{\parallel k\overrightarrow{x}-\overrightarrow{y}\parallel }_2={\displaystyle \frac{{\overrightarrow{x}}^T\overrightarrow{y}}{{\overrightarrow{x}}^T\overrightarrow{x}}}.$$

(2)

STRESS values range from 0 (perfect agreement) to 1 (maximum discrepancy). For context, the state-of-the-art CDM, CIEDE2000, achieves a “good” STRESS value of 0.292 on the COMBVD dataset, on which it was trained. However, this value depends on the dataset’s internal consistency and noise. The upper limit for STRESS on the new dataset is estimated in Section 3.1. In contrast, the poorly uniform CIE XYZ color space achieves a STRESS value of 0.692 on the same dataset, providing a reference for poor performance.

Various modifications have been proposed for handling multiple datasets [23,45]. Recently, a modified STRESS version, STRESS_group, was introduced to ensure that all color pairs are accounted for with equal importance and to adjust for CD scale variations across datasets [46].

While STRESS evaluates the average error, it is equally important to assess significant errors, as a CDM serves as a measurement tool for evaluating other algorithms, such as those used in image processing. To address this, we propose analyzing the error distribution and focusing on errors exceeding a certain threshold, ${\Delta }_0$, which we define as unacceptable. Errors below ${\Delta }_0=1$ JND are considered imperceptible, making such a threshold unreasonable. We selected ${\Delta }_0=2$, as errors ${\Delta }_0>2$ form the heavy tail of the error distribution for current state-of-the-art CDMs (see Figure 10).

Figure 10. Histogram of errors for the current state-of-the-art CDMs (CIEDE2000 and CAM16-UCS-PC) on the COMBVD dataset. The x-axis represents the magnitude of the CDM error, while the y-axis indicates the number of these errors.

To quantify significant errors, we propose calculating the fraction of errors exceeding 2 JND:

$$\mathrm{FEM}2\mathrm{JND}(\overrightarrow{x},\overrightarrow{y})={\displaystyle \frac{1}{n}}\sum _{i=1}^n[(k^{*}{x}_i-y_i)>2],$$

(3)

where $k^{*}$ is defined as in Equation (2).

3. Results

3.1. Comparison of the Obtained and Previously Published Data

Establishing the consistency of our experimental data with previously published data is crucial for validating our dataset. We compared our data with the COMBVD dataset [41] (see Section 2.4), which includes SCDs in the sRGB and SDR range. We focused on sub-datasets obtained under the D65 illuminant, excluding the BFD-P sub-dataset derived under the M and C illuminants.

Our consistency evaluation method leverages the fact that human color perception can be described by a continuously differentiable function without non-physiological discontinuities. This concept implies that, within a small region of color space, perceived color differences can be approximated by a linear model. By constructing a locally linear model from data in a small color space region, we can evaluate the consistency of our data with previously published datasets.

Let us consider a dataset of color pairs and their corresponding CDs. The i-th color pair in the dataset is represented as

$$p_i\stackrel{\mathrm{def}}{=}({\mathbf{c}}_{i1},{\mathbf{c}}_{i2}).$$

(4)

To define the distance between two color pairs, we first establish distances for “codirectional” and “opposite” pairs in terms of the color ordering we have chosen:

$${\rho }_{\uparrow \uparrow }(p_i,p_k)=\sqrt{{\displaystyle \frac{d({\mathbf{c}}_{i1},{\mathbf{c}}_{k1})+d({\mathbf{c}}_{i2},{\mathbf{c}}_{k2})}{2}}},$$

(5)

$${\rho }_{\uparrow \downarrow }(p_i,p_k)=\sqrt{{\displaystyle \frac{d({\mathbf{c}}_{i1},{\mathbf{c}}_{k2})+d({\mathbf{c}}_{i2},{\mathbf{c}}_{k1})}{2}}},$$

(6)

where d is a distance measure, with CIEDE2000 chosen as our metric.

The distance between color pairs is defined as

$$\rho (p_1,p_2)=min({\rho }_{\uparrow \uparrow }(p_1,p_2),{\rho }_{\uparrow \downarrow }(p_1,p_2)).$$

(7)

It should be noted that this distance measure may not satisfy all properties of a metric. To identify regions of high data density in color space, we constructed a complete weighted graph from the set of color pairs in the datasets, calculating all possible distances between them. We then retained only the edges with weights below a certain threshold value ${\rho }_{thr}$.

To identify clusters of closely positioned color pairs containing $\ge N$ pairs, we ranked the graph’s vertices in descending order based on their degree, i.e., the number of edges connecting each vertex to its neighbors.

The following algorithm was employed for the extraction of clusters from the graph $G_{thr}$:

• Initialize $i=1$.
• If the degree of the i-th vertex $k_i>N-2$, form a cluster $C_i$ of size $k_i+1$ consisting of the i-th vertex and its connected vertices.
• Remove the edges connecting each vertex in cluster $C_i$ to other vertices within the same cluster, and update $G_{thr}$.
• Repeat steps 2–4 for $i=i+1$.

Within this cluster detection framework, any color pair can belong to multiple clusters, but any two color pairs belong to only one cluster. Sub-datasets within each cluster may vary in size.

To estimate internal consistency, we set ${\rho }_{thr}=2$ and $N=100$; then, we optimized the linear model on each cluster. Since we assumed that the units of perceptual distance may differ across datasets by a proportionality coefficient, we employed $STRES{S}_{group}$ [46] as the loss function.

First, we calculated the internal consistency of the COMBVD-D65 dataset as a reference. The results of the linear model optimization are presented in Table 4 for all five clusters we detected. The dataset consistency is represented by the highest value, $STRES{S}_{group}=0.216$.

Table 4. Results of linear model optimization for COMBVD-D65 clusters.

Region	Red	White	Blue	Yellow	Green
$STRES{S}_{group}$	0.197	0.179	0.181	0.216	0.197
Cluster size	294	264	206	205	202

Next, we evaluated the consistency between our data and the COMBVD-D65 dataset. The results of the linear model optimization for each detected cluster are shown in Table 5. The highest consistency value for our data with COMBVD-D65 is $STRES{S}_{group}=0.316$. While this is slightly worse than the internal consistency of COMBVD-D65, it remains comparable. Thus, we conclude that our data are consistent with COMBVD-D65, at least regarding the CD scale factor. We will address the issue of data consistency on an absolute scale in Section 3.3.

Table 5. Consistency scores of COMBVD-D65 sub-datasets and our data. For each detected cluster, the number of points from each dataset (size) and the $STRES{S}_{group}$ metric for the correspondence of cluster points to the constructed local linear model are provided.

No.	${\mathit{STRESS}}_{\mathit{group}}$	BFDP Size	Rit-DuPont Size	Witt Size	Leeds Size	Our Size
1	0.240	151	6	85	52	72
2	0.260	155	18	78	13	65
3	0.316	107	—	83	15	83
4	0.242	121	—	85	—	64
5	0.237	108	—	84	9	43

3.2. Fitting the Color Difference Model to the Data: Optimally Fitted $\Delta E_f$

We employed a new flexible parametric model that implements a semi-metric in color space. We also provided an analytical method for finding its optimal parameters.

Our approach built on the CAM16-UCS model with power law correction:

$$\Delta E_{\mathrm{CAM}16-\mathrm{UCS}-\mathrm{PC}}=1.41{(\Delta E_{\mathrm{CAM}16-\mathrm{UCS}})}^{0.63}.$$

(8)

We enhanced this model with an additional trainable factor for improved accuracy. The model is defined in the following linear space:

$$\Delta E_f{(\mathbf{x},\mathbf{y}|\mathit{\theta })\stackrel{\mathrm{def}}{=}\parallel \mathbf{x}-\mathbf{y}\parallel }_2^p\left|{(\mathbf{b}\left(\mathbf{v}(\mathbf{x},\mathbf{y})\right)+\mathbf{b}\left(\mathbf{v}(\mathbf{y},\mathbf{x})\right))}^T\mathit{\theta }\right|,$$

(9)

where $\mathbf{x}=(J_{\mathbf{x}},a_{\mathbf{x}},b_{\mathbf{x}})$, and $\mathbf{y}=(J_{\mathbf{y}},a_{\mathbf{y}},b_{\mathbf{y}})$ are CAM16-UCS coordinates of a color pair, and $p=0.63$. Here,

$$\mathbf{v}(\mathbf{x},\mathbf{y})\stackrel{\mathrm{def}}{=}\left(J_{\mathbf{x}},{\displaystyle \frac{\sqrt{a_{\mathbf{x}}^2+b_{\mathbf{x}}^2}}{J_{\mathbf{x}}}},{\parallel \mathbf{x}-\mathbf{y}\parallel }_2\right),$$

(10)

where $\mathbf{b}:{\mathbb{R}}^3\to {\mathbb{R}}^d$ is a basis function dictionary listed in Appendix A, and $\mathit{\theta }\in {\mathbb{R}}^d$ is the vector of CDM parameters.

This model is a semi-metric, as it does not satisfy the triangle inequality, similar to the “gold standard” CIEDE2000 formula.

To train the model, we optimized the parameter vector $\mathit{\theta }$ to minimize STRESS on the dataset:

$$\begin{array}{c}{\displaystyle {\mathit{\theta }}^{*}\stackrel{\mathrm{def}}{=}arg\underset{\mathit{\theta }}{min}\mathrm{STRESS}(\mathbf{e},M\mathit{\theta })=arg\underset{\mathit{\theta }}{min}\underset{k}{min}\frac{{\parallel k\mathbf{e}-M\mathit{\theta }\parallel }_2^2}{{\parallel M\mathit{\theta }\parallel }_2^2},}\end{array}$$

(11)

where

$$\begin{array}{ccc}\mathbf{e}\hfill & \stackrel{\mathrm{def}}{=}& {\left[\begin{array}{cccc}{e}_1& e_2& \cdots & e_n\end{array}\right]}^T,\hfill \\ M\hfill & \stackrel{\mathrm{def}}{=}& {\left[\begin{array}{cccc}{\mathbf{m}}_1& {\mathbf{m}}_2& \cdots & {\mathbf{m}}_n\end{array}\right]}^T,\hfill \\ {\mathbf{m}}_i\hfill & \stackrel{\mathrm{def}}{=}& \parallel {\mathbf{x}}_i-{\mathbf{y}}_i{\parallel }_2^p(\mathbf{b}\left(\mathbf{v}({\mathbf{x}}_i,{\mathbf{y}}_i)\right)+\mathbf{b}\left(\mathbf{v}({\mathbf{y}}_i,{\mathbf{x}}_i)\right),\hfill \end{array}$$

(12)

are vectors of experimental distances and model base matrix, respectively, and n represents the number of color pairs in the dataset.

This formulation results in a quadratic fractional programming problem:

$${\displaystyle {\mathit{\theta }}^{*}\stackrel{\mathrm{def}}{=}arg\underset{\mathit{\theta }}{min}\underset{k}{min}\frac{\left[\begin{array}{cc}{\mathit{\theta }}^T& k\end{array}\right]A{\left[\begin{array}{cc}{\mathit{\theta }}^T& k\end{array}\right]}^T}{\left[\begin{array}{cc}{\mathit{\theta }}^T& k\end{array}\right]B{\left[\begin{array}{cc}{\mathit{\theta }}^T& k\end{array}\right]}^T},}$$

(13)

where

$$A\stackrel{\mathrm{def}}{=}{\left[\begin{array}{cc}M& -\mathbf{e}\end{array}\right]}^T\left[\begin{array}{cc}M& -\mathbf{e}\end{array}\right],B\stackrel{\mathrm{def}}{=}{\left[\begin{array}{cc}M& \mathbf{0}\end{array}\right]}^T\left[\begin{array}{cc}M& \mathbf{0}\end{array}\right].$$

(14)

This problem can be solved analytically, avoiding numerical optimization methods. To solve (13) analytically, we first regularize matrices A and B:

$$\widehat{A}\stackrel{\mathrm{def}}{=}A+\lambda I,\widehat{B}\stackrel{\mathrm{def}}{=}B+\lambda I,\lambda =0.01.$$

(15)

Using the Cholesky decomposition, we find an upper triangular matrix R such that $R^{T}R=\widehat{B}$. The matrix V of eigenvectors of $K\stackrel{\mathrm{def}}{=}{R}^{-T}\widehat{A}{R}^{-1}$ is

$$V\stackrel{\mathrm{def}}{=}\left[\begin{array}{cccc}{\mathbf{v}}_1& {\mathbf{v}}_2& \cdots & {\mathbf{v}}_{d+1}\end{array}\right]:K{\mathbf{v}}_i\sim {\mathbf{v}}_i.$$

(16)

The optimal solution corresponds to one of the columns of $W\stackrel{\mathrm{def}}{=}{R}^{-1}V$:

$$\left[\begin{array}{c}{\mathit{\theta }}^{*}\\ k^{*}\end{array}\right]=\underset{\left[\begin{array}{c}\mathit{\theta }\\ k\end{array}\right]\in W}{argmin}{\displaystyle \frac{{\parallel k\mathbf{e}-M\mathit{\theta }\parallel }_2^2}{{\parallel M\mathit{\theta }\parallel }_2^2}}.$$

(17)

The optimization of CDM parameters is typically performed using numerical methods, which are computationally intensive and do not guarantee the discovery of a global minimum for non-convex objectives like STRESS. In contrast, the proposed analytical optimization approach ensures that the global minimum of STRESS is achieved while maintaining low computational complexity (approximately 0.12 s on an Intel Core i7-8550U).

The model was trained on a combined dataset comprising COMBVD, SDCTh, and our data. The dataset was split into 75% for training and 25% for testing. Using the analytical optimization Equations (15)–(17), the model achieved a STRESS value of 0.390 on the training set and 0.393 on the test set. These results demonstrate that the model is not overfitted.

3.3. Benchmarking Color Difference Models

We used the optimally fitted $\Delta E_f$ to analyze our dataset. Given the similar STRESS values across both previously published datasets and ours (see Table 6 and Table 7), we conclude that the datasets are consistent in terms of absolute CD scale. To evaluate the randomness of the obtained STRESS and FEM2JND values, we constructed confidence intervals at a confidence level of $p=0.95\%$, corresponding to 1.96 standard deviations. For STRESS, bootstrapping [47] was applied, while the Wald confidence interval was used for FEM2JND [48].

Table 6. Performance of CDMs on previously published datasets (lower values indicate better consistency).

Model	STRESS
	SDR & sRGB	HDR or WCG	All
CIEDE2000	0.297 ± 0.032	0.328 ± 0.077	0.306 ± 0.031
CAM16-UCS	0.303 ± 0.036	0.320 ± 0.058	0.308 ± 0.031
CAM16-UCS-PC	0.301 ± 0.014	0.293 ± 0.030	0.311 ± 0.014
Jzazbz	0.383 ± 0.021	0.329 ± 0.050	0.371 ± 0.021
ICaCb	0.410 ± 0.022	0.355 ± 0.050	0.398 ± 0.022
Oklab	0.475 ± 0.026	0.439 ± 0.086	0.466 ± 0.030
proLab	0.475 ± 0.024	0.439 ± 0.054	0.466 ± 0.022
ICTCp	0.492 ± 0.027	0.408 ± 0.061	0.472 ± 0.026
Opt. fitted $\Delta E_f$	0.334 ± 0.021	0.303 ± 0.038	0.327 ± 0.019

Table 7. Performance of CDMs on the dataset we collected (lower values indicate better consistency).

Model	STRESS
	SDR & sRGB	HDR or WCG	All
CAM16-UCS-PC	0.326 ± 0.026	0.500 ± 0.033	0.528 ± 0.032
Jzazbz	0.470 ± 0.055	0.584 ± 0.046	0.583 ± 0.044
ICaCb	0.485 ± 0.046	0.583 ± 0.040	0.589 ± 0.037
CAM16-UCS	0.486 ± 0.033	0.625 ± 0.068	0.625 ± 0.063
Oklab	0.461 ± 0.111	0.634 ± 0.043	0.633 ± 0.043
ICTCp	0.564 ± 0.045	0.601 ± 0.038	0.635 ± 0.034
proLab	0.471 ± 0.048	0.646 ± 0.044	0.648 ± 0.042
CIEDE2000	0.525 ± 0.035	0.694 ± 0.089	0.693 ± 0.082
Opt. fitted $\Delta E_f$	0.439 ± 0.039	0.401 ± 0.099	0.408 ± 0.091

We benchmarked the following CDMs on psychophysical datasets. CAM16-UCS, introduced in 2016 [23], is the current state-of-the-art UCS [24]. CAM16-UCS with power law correction (see Equation (8)), which accounts for the non-geodesic properties of human color space. Other UCS models designed for WCG and HDR include IC_aC_b [8], IC_TC_p [9], and J_za_zb_z [10]. We also assessed Oklab [49], promising for LCD, and proLab [36], which offers good geometric properties. Additionally, we considered the CIEDE2000 formula [41], the state-of-the-art for SCD.

The results confirmed that CAM16-UCS with power correction (CAM16-UCS-PC) is the most versatile model, demonstrating the best performance on the combined dataset across both sRGB/SDR and HDR/WCG domains (see Table 8, where lower STRESS values indicate better consistency). However, CAM16-UCS-PC is not without limitations. Its accuracy could potentially improve by at least 1.3 times, as suggested by the optimally fitted $\Delta E_f$. For the combined dataset, the standard deviation for CAM16-UCS-PC is 0.013, while that for the optimally fitted $\Delta E_f$ is 0.041, meaning the difference between the mean values exceeds the standard deviation. Moreover, despite an acceptable average error, CAM16-UCS-PC shows a notable proportion of significant errors, measured as the fraction of errors above 2 JND (Table 9). A comparison between CAM16-UCS and CAM16-UCS-PC reveals that the power correction increases significant errors by 1.8 times. While CAM16-UCS-PC performs well on average, it still produces a considerable number of significant errors.

Table 8. Performance of CDMs on the combined dataset: previously published and ours (lower values indicate better consistency).

Model	STRESS
	SDR & sRGB	HDR or WCG	All
CAM16-UCS-PC	0.395 ± 0.023	0.490 ± 0.031	0.493 ± 0.026
ICaCb	0.484 ± 0.030	0.571 ± 0.039	0.558 ± 0.033
Jzazbz	0.495 ± 0.035	0.584 ± 0.045	0.589 ± 0.042
CAM16-UCS	0.438 ± 0.036	0.622 ± 0.069	0.610 ± 0.064
proLab	0.508 ± 0.027	0.635 ± 0.042	0.616 ± 0.038
ICTCp	0.528 ± 0.027	0.615 ± 0.033	0.630 ± 0.027
Oklab	0.532 ± 0.037	0.641 ± 0.042	0.656 ± 0.037
CIEDE2000	0.454 ± 0.036	0.688 ± 0.090	0.668 ± 0.090
Opt. fitted $\Delta E_f$	0.377 ± 0.024	0.395 ± 0.096	0.393 ± 0.080

Table 9. Performance of CDMs on the combined dataset: previously published and ours (lower values indicate better consistency).

Model	FEM2JND
	SDR & sRGB	HDR or WCG	All
CAM16-UCS-PC	0.026 ± 0.010	0.342 ± 0.037	0.164 ± 0.018
ICaCb	0.050 ± 0.013	0.266 ± 0.034	0.139 ± 0.017
Jzazbz	0.046 ± 0.013	0.250 ± 0.033	0.123 ± 0.016
CAM16-UCS	0.038 ± 0.012	0.194 ± 0.031	0.090 ± 0.014
proLab	0.063 ± 0.015	0.309 ± 0.036	0.172 ± 0.018
ICTCp	0.076 ± 0.016	0.356 ± 0.037	0.257 ± 0.021
Oklab	0.065 ± 0.015	0.246 ± 0.033	0.131 ± 0.016
CIEDE2000	0.043 ± 0.013	0.182 ± 0.030	0.095 ± 0.014
Opt. fitted $\Delta E_f$	0.019 ± 0.008	0.179 ± 0.030	0.081 ± 0.013

A universal model is yet to be developed, but CAM16-UCS is a promising foundation, with power correction being a key area for improvement in CAM16-UCS-PC.

4. Conclusions

We collected a new experimental dataset on perceived color differences spanning three scales (threshold, small, and large). The dataset covers a wide color gamut (80% of BT.2020) at high luminance levels (up to 1611 cd/m²) using a custom stimulus generator. Measurements were conducted with 22 observers, yielding color difference data for 2605 color pairs.

To ensure dataset consistency, we introduced a method leveraging the continuous nature of human color perception. This involved fitting linear models to small psychophysical data clusters and evaluating them using the STRESS metric. Our analysis confirmed consistency between our dataset and established datasets.

We further improved the power-corrected CAM16-UCS (CAM16-UCS-PC) model by introducing a novel color difference model, optimally fitted $\Delta E_f$. This refinement employed an analytical optimization approach capable of efficiently minimizing non-convex objectives, such as STRESS, while avoiding the risk of local minima. Our results demonstrated that existing models, including CAM16-UCS-PC, have not yet reached the theoretical accuracy limit, as evidenced by the performance of the optimally fitted $\Delta E_f$ on the combined dataset of new and established data.

Benchmarking across multiple datasets revealed that CAM16-UCS-PC is currently the most versatile model, achieving the best STRESS performance for sRGB, WCG, SDR, and HDR domains. However, the model is not without limitations. In traditional domains (SDR and sRGB), CAM16-UCS-PC exhibits fewer significant errors than CAM16-UCS. Conversely, in HDR-WCG domains, the number of significant errors for CAM16-UCS-PC is 1.8 times higher than for CAM16-UCS without power correction.

These effects can be attributed to the fact that neither model was trained on HDR or WCG data. While CAM16-UCS-PC introduces power correction to address the non-geodesic nature of color space, it remains a less appropriate model and performs worse than the base CAM16-UCS when predicting new data.

Our findings suggest that incorporating our dataset into the development of UCS could enhance future models. However, the root causes of these discrepancies remain unclear and require further investigation. Additional measurements, particularly in regions with the greatest errors, are necessary to improve model accuracy and better understand the observed effects.