1. Introduction
Imaging instruments are increasingly used to measure luminance for lighting applications. They allow the luminance of an entire scene to be captured much more quickly than with point measurement instruments, which evaluate only one position per acquisition. They also enable dense measurements, providing luminance values for each pixel of the scene with high spatial resolution. As a result, complex scenes that were previously difficult to study can now be photometrically characterized more efficiently. However, the dynamic range of matrix sensors used in imaging instruments (CCD or CMOS, typically) is generally lower than the range required for the scenes to be studied, particularly when investigating issues such as discomfort, glare, or obtrusive light, where both very bright sources and dark backgrounds must be accurately measured. To overcome this equipment limitation, high dynamic range (HDR) imaging techniques [
1,
2] are employed [
3,
4,
5]. These methods were developed many decades ago and have been a topic of high interest since the democratization of digital imaging in the 1990s. Today, HDR techniques are implemented in a large number of consumer devices, either through multi-exposure fusion or single-exposure architectures (e.g., dual-gain or multi-conversion sensors), increasingly enhanced by AI methods, notably in smartphones. Nevertheless, the performance sought in recent developments is often focused on producing visually pleasing images, which differs from the needs of the lighting applications we are interested in. Indeed, our objective is to obtain the best possible estimate of luminance values with associated uncertainties in a way that is traceable to the International System (SI) of units.
In this context, the HDR imaging methods currently used for luminance measurement rely on well-established approaches, consisting of combining several low dynamic range (LDR) images acquired at different exposure levels [
6,
7,
8]. An LDR image is a set of digital values coded by the number of bits corresponding to the dynamic range of the sensor. In a very simplified way, we can say that, for a linear camera, each value associated with a pixel of the sensor is proportional to the luminance of the portion of the scene imaged on the pixel, to the sensitivity of the camera, and to the integration time. This is true if the amount of light reaching the pixel is neither too low, in which case the signal is lost to the noise, nor too high, in which case the pixel becomes saturated. Luminance measurement is therefore possible over a limited range of values using a single LDR image, the lower and upper bounds of which can be shifted by adjusting the camera’s sensitivity (sensor gain, lens aperture, addition of neutral densities, etc.) or the integration time. Consequently, to evaluate scenes containing a very wide range of luminance values, it is necessary to take several images at several exposure levels and combine them using HDR merging algorithms. The main purpose of these algorithms is to produce images with the largest possible dynamic range while ensuring that the signal-to-noise ratio (SNR) of the luminance estimate for each pixel remains sufficiently high.
The quality of HDR measurements is typically affected by factors related to the measuring instrument, the LDR image acquisition strategy, the choice of HDR merging algorithm, and its parameter settings. The question of the best HDR imaging method, including the best merging algorithm, has been addressed before [
9,
10,
11,
12,
13]. However, to our knowledge, most studies focused on the effects of sensor noise only and did not consider other significant effects, which might have a significant impact on the calculated HDR values. In this study, we evaluate a set of relevant parameters, including sensor non-linearity, inaccuracies in integration time, algorithm threshold selection, and temporal stability of the scene. Our objective is to assess the performance of HDR imaging methods in producing accurate and traceable luminance measurements, considering the aspects relevant to lighting applications. To this end, we analyze, through a numerical approach, the impact of different HDR-method parameters on luminance measurement accuracy. This approach relies on a camera “digital twin” implemented in MATLAB software (R2023b), which models the acquisition of an image by an imperfect digital camera. The camera digital twin is used to replicate the HDR imaging process applied to capture a virtual scene, yielding a synthetic HDR luminance image that can be compared to the ground truth scene. This approach allows us to study sensitivities for each parameter and to determine whether one type of HDR merging algorithm performs better than the others or not. In this study, we chose to focus on parameters whose impact may depend on the type of HDR merging algorithm applied, as well as on parameters that can be directly adjusted by the camera user. In this context, luminance inaccuracies caused by vignetting and stray light generated by the objective lens were not considered. The latter, which is often the most limiting factor for accurate imaging measurements, will be addressed in a separate study, as experimental evaluations are more appropriate than the digital approach presented here. The HDR imaging methods are introduced in
Section 2, where the relevant parameters to be tested are also presented. The impact of noise for different HDR imaging methods is theoretically analyzed in
Section 3. The conclusion from this analysis is the basis for selecting candidate methods to be further explored in more realistic conditions, where multiple error sources beyond photon noise are considered. This exploration is detailed in
Section 4 and
Section 5.
Section 4 describes the evaluation method, in which camera digital twins are used to synthetically generate HDR images affected by various sources of error, allowing the evaluation of their impact on the preselected HDR imaging methods. The results obtained using this evaluation method are presented in
Section 5, followed by the conclusions in
Section 6.
3. Theoretical Impact of the Noise on the HDR Imaging Methods
Among the HDR merging algorithms presented above, the weighted average algorithm has been developed to reduce noise in HDR luminance measurements. However, the performance of the algorithm, which can be evaluated as its ability to reduce noise, strongly depends on the choice of weight scheme. We propose in this section a theoretical study to better understand how the weighting factor impacts the results, under the assumption that the main contribution to noise is photon noise (also called shot noise), where the variance of photoelectrons equals its average. In this case, the variance of the signal at a given pixel under a given exposition is proportional to the integration time.
For simplicity, we set the luminous sensitivity
sV to 1 in this theoretical study. Similarly to Equation (3), the general weighted-average HDR method can be expressed as:
with
where
M is the number of acquisitions,
Di is the pixel value of the
ith image, and
Dmax is the pixel saturation threshold. The reduction factor of integration time between two successive images of the LDR series can be expressed as:
By uncertainty propagation as described by the Guide to the Expression of Uncertainty in Measurement (GUM [
20]), the uncertainty of the luminance can be derived as:
where only the uncertainty of the variable
D is considered and only its random component, or noise. There is no uncertainty associated with the weighting factors, since they are part of the definition of the HDR merging method. In addition, no uncertainty on integration time is considered for this analysis.
To evaluate the performance of the method, we evaluate how much the luminance uncertainty decreases with respect to the case where the optimal integration time for each pixel is used for its calculation—corresponding to applying the best exposure algorithm described in
Section 2 (
i = 1 and
M = 1, where the integration time has the maximum value before saturation). This can be expressed as:
As the only error considered is shot noise, we have:
Hence, we obtain the following expression for the general uncertainty reduction:
This uncertainty reduction factor can be examined for different weighting factors. Since acquisitions with larger SNR should count more in the weighted average, we propose to evaluate the weight as exponential functions of the SNR: SNR2, SNR1, and SNR0 (or uniform weighting). It must be noticed that the SNR is proportional to the square root of the number of photoelectrons, and consequently to the square root of the integration time. Therefore, we obtain wi = ti, in the case of SNR2 and wi = , in the case of SNR1. Using these weight factors in Equation (14), we obtain the following expressions:
In all cases, the reduction depends on the variables
M and
ft. The representation of the functions given in Equations (15)–(17) is shown in
Figure 2 and
Figure 3. An uncertainty reduction factor below 1 means that the SNR of the HDR pixel is increased using the tested algorithm compared to using the best exposure algorithm.
Figure 2 and
Figure 3 show that only a weighted HDR algorithm with an SNR
2 as a weighting function would improve the SNR of the resulting measurement, as the value of the uncertainty reduction factor is always below 1. However, this improvement is below 15% in the reduction of the relative noise for
. Using the weighting function SNR
2, the pixel SNR can almost be doubled in the best case. In the specific case where
ft = 1, which is the singular case where there is no variation in the integration time, the pixel value is averaged over several identical captures, significantly increasing the SNR but without increasing the measurable dynamic range, which defies the purpose of HDR imaging.
The HDR imaging method based on weighted average in the logarithmic space (see Equation (4)) can be studied using a similar method to compare if it reduces uncertainty with respect to the linear approach, which yields similar results. From this analysis, we can conclude that, if the only source of error is shot noise:
A weighting factor proportional to SNR2 is more efficient for the reduction of the noise. This weighting factor corresponds with using ti, N, or N2/ti. We think that the more convenient weight to be used is ti, because using N within this factor would add unnecessary noise to the method.
The use of ti as a weighting factor would reduce the uncertainty by 30% and 15%, respectively, in the practical cases of ft = 2 and ft = 4, with respect to the case where the optimal integration time is used for each pixel.
The logarithmic approach does not improve the reduction in uncertainty in this shot-noise-limited case.
This first evaluation guided us in choosing the tested weight factor. However, the simplistic hypothesis applied in this section does not account for all the parameters that may impact the performance of an HDR imaging method. The numerical evaluation based on the modeling of a camera “digital twin”, detailed in the next section, allows us to evaluate the performance of an HDR imaging method based on a more realistic camera.
4. Evaluation Method
The evaluation framework developed to study HDR imaging methods, illustrated in
Figure 4, comprises four main elements: a high contrast ground-truth virtual luminance scene (GT), a digital twin (DT) of the camera response, HDR merging algorithms, and evaluation metrics used to compare the performance of the different HDR imaging methods. The DT generates synthetic LDR images from the GT according to the chosen camera and acquisition parameters. These LDR images are then combined using the tested HDR merging algorithms presented in
Section 2.2 [
21] to produce HDR images. The performance of the HDR imaging method is finally quantified by comparing the GT with the resulting HDR images using metrics tailored to our needs.
4.1. Digital Twin (DT) of the Camera Response
The response of arbitrary and conventional luminance cameras was simulated by considering the known error sources, that is, the effects that cause the camera response to deviate from the true luminance to be measured. This simulation, referred to hereafter as the “DT response”, enables the generation of different camera behaviors by adjusting the parameters associated with these error sources. As such, the DT response is a highly suitable tool for systematically assessing how various error sources affect the performance of each HDR imaging method under evaluation.
The DT response is based on the principle proposed in [
22] and illustrated in
Figure 5. The included error sources are photon noise, readout noise, dark signal, photo-response non-uniformity (PRNU) of the array sensor, lens-shading non-uniformity, non-linearity, internal stray light, smearing, saturation, and adjustment of the luminous responsivity. These error sources are implemented using a parametrized function and random values in the case of noise-related effects, and they are applied sequentially in a specific order. In the evaluation presented here, some effects that introduce systematic errors independently of the measurement method (HDR or LDR imaging) were removed to simplify the problem and focus on the most relevant parameters. This also reduces the computational cost of the evaluation, particularly by omitting effects modeled as image convolutions (e.g., internal stray light, which is represented by a point spread function applied to the image).
The camera model used in the evaluation method includes the following effects, applied sequentially:
For HDR capture, the target integration time of the
ith image of the simulated LDR series is defined within the range of possible integration times [
tmin:
tmax] based on the integration time reduction factor
ft as:
The next image is captured as long as there are still saturated pixels on the image, under the condition that the integration time remains above the shortest possible integration time tmin.
- 2.
Photon noise–The randomness associated with photons arriving at the detector results in a Poisson distributed noise, whose standard deviation is equal to the square root of the average number of electrons, and therefore depends on the overall “gain” of the sensor
K [count·electron
−1] (inversely proportional to the ISO) [
17]. With
randn, a random value obtained from a standard normal distribution, we have for each pixel:
- 3.
Dark signal–An electrical signal is produced by a camera sensor even in complete darkness. It is caused by the thermal generation of electrons within the semiconductor material during the integration time, photon noise of the dark, and some residual offset. With rth the rate of thermally produced counts [count·ms−1] and Doffset [count], a constant offset, the dark signal is applied as:
- 4.
Non-linearity–Although the camera is considered linear, pixel values do not increase exactly proportionally to the incident irradiance, especially for low count values and near saturation. The function
fNL proposed to model non-linearity, illustrated in
Figure 6, has been chosen empirically for its potential to mimic plausible effects while depending on only two parameters,
a and
b:
where
x is a count value within the dynamic range of the sensor described by the bit depth
Nbits. The non-linearity is applied as a multiplicative factor for each pixel:
- 5.
Readout noise–The variance in counts from repeated readings of a pixel is considered using a random noise of standard deviation σread [count]:
- 6.
Saturation and quantization–Pixel values are limited to integers within a range limited by the bit depth of the sensor Nbits, above which saturation occurs. With [.] the floor rounding operation, we have:
Using the DT response described by Equations (18) to (25), synthetic LDR images at different integration times can be obtained and merged into an HDR image applying an HDR imaging method. The comparison of these output images with the input image (GT) allows the evaluation of the performance of the studied HDR approaches.
4.2. Definition of the Virtual High Contrast Luminance Scenes (Ground Truth)
The impact of the HDR imaging method parameters can be classified into two categories: some parameters influence the value of a pixel independently of the surrounding pixels, while others impact the image spatially. In this study, we limited our evaluation to the former parameters. For this purpose, it is essential to test a wide range of luminance values; however, the inclusion of specific spatial frequency content is unnecessary. The proposed virtual ground truth (GT) scene, shown in
Figure 7, consists of a horizontal luminance gradient, where each column corresponds to a luminance value within a customized range, distributed according to a logarithmic progression. The number of columns defines the number of luminance levels tested, while the number of rows corresponds to the sampling size, enabling a statistical analysis of noise.
Temporal instability in scene luminance can affect HDR luminance measurements. For example, fluctuations from a very bright source may significantly impact its measured value because high luminance levels require short integration times. As a result, these fluctuations are less likely to be averaged out, unlike fluctuations from a low luminance source, which are measured using longer integration times and therefore benefit from temporal averaging. The proposed HDR merging algorithms may be sensitive to luminance temporal fluctuations, as they combine the images measured at different instances differently. To study the impact of source temporal fluctuations, the effect has been modeled as a multiplying factor on the GT image as part of the camera digital twin (DT):
where
σsource is the standard deviation of the source fluctuations observed with a temporal period of Δ
tsource,
tint is the integration time during which the source is acquired by the camera,
randn(
T) designates
T random samples distributed according to a normal distribution, and [.] designates floor rounding.
4.3. Evaluation Metrics
The performance of the tested HDR methods is evaluated according to criteria relevant to luminance measurement in lighting applications. Although accurate rendering of spatial information is essential, the primary focus is on the accuracy of the measured luminance values and the level of noise observed in regions of interest, typically corresponding to groups of image pixels. The good reproduction of image gradients (without discontinuities related to HDR reconstruction) is also considered important, but has not been specifically tested here, as it is redundant with evaluating the accuracy of the luminance values. Finally, the ability of the method to reproduce the widest possible dynamic range is a key criterion, including the capability to accurately measure very dark regions despite the presence of high luminance areas within the camera’s field of view. The metrics typically proposed to quantify distance between images [
23] are not sufficiently adapted to our objectives. Parameters that impact only the pixel values of an image can be evaluated using metrics that are not specific to image processing. Therefore, we derived metrics based on classical signal processing to evaluate the distance between the GT and the simulated HDR images, which we call bias and noise.
The HDR image accuracy in terms of average luminance level is characterized by the bias, which we define as the relative difference between the GT value and the average value obtained on the HDR image. The HDR image quality in terms of noise, referring to the random fluctuations around the average value, is characterized by the relative standard deviation of the values corresponding to the same luminance level.
On the gradient scene, the bias
bi for the column of index
i is computed as the relative difference between the GT value of the column
GTi and the average luminance value of the simulated HDR image (
LHDR) for the same column. The noise
εi is computed as the standard deviation over the pixel values of the column, divided by the average value over the column. With ⟨⋅⟩
i referring to the averaging operation over all pixels in column
i, we have:
4.4. Tested Parameters and Strategy
The parameters varied during the tests can be described as camera properties (parameters, noises and errors), acquisition parameters, HDR merging algorithm parameters, and scene properties. They are listed in
Table 1,
Table 2,
Table 3 and
Table 4.
A reference camera model, representing a good quality camera with relatively low noise levels, has been defined using the parameter values highlighted in bold in the tables. For each parameter, additional values corresponding to different noise levels or different performance levels have been specified.
The proposed parameters are inspired by three main hardware categories: scientific cameras (we use an ORCA Flash 4.0 v3 with a −10 °C cooled CMOS sensor from Hamamatsu), industrial cameras (we tested the acA2040-90mNIR model from Basler), and consumer cameras (such as Nikon and Canon DSLRs).
According to the literature, integration time errors are well below one millisecond for most models and can be as small as a few microseconds, as the internal clock is generally well controlled. We therefore considered small values in our study, although mechanical shutters may lead to larger deviations.
The proposed rates of thermally generated counts (or electrons, for a gain equal to 1) are in the upper range compared with data published by manufacturers. For cooled sensor sensors, this rate is very low (6 × 10−5 electrons·ms−1 for our ORCA camera), and below 1 × 10−3 electrons·ms−1 for DSLRs at room temperature. However, we surprisingly measured 0.25 electrons·ms−1 for our Basler camera (not equipped with a heat-sink) in normal ambient conditions, with even worse performance for integration times longer than 500 ms.
Finally, the mathematical model proposed to describe non-linearity was empirically determined to reproduce the general trends observed in experimental measurements. The non-linearity values measured experimentally are about 2% (following the EMVA method [
17]) for the Basler camera (corresponding to
a = 0.1 and
b = 1), and about 0.5% for the ORCA camera (approximately
a = 0.03 and
b = 1). This model is also suitable to represent residual non-linearity after correction (e.g., 0.5% and 0.2% for the Basler and ORCA cameras mentioned above, respectively).
The impact of a given parameter on the HDR luminance measurement performance is evaluated by considering a camera model in which all the parameters correspond to the reference except for the parameter being tested, which takes the range of values indicated in the tables. This approach allows us to analyze the sensitivity of the luminance measurement to each parameter individually, and to determine whether any of the four tested HDR algorithms performs better under certain conditions.
For some specific parameters, the reference model is not adequate for testing. In that case, a tailored reference is used. For example, the choice of threshold values for the HDR merging algorithm may represent a loose constraint, a moderate constraint, and a strict constraint (see
Table 3). To better observe the impact of the threshold, some non-linearity must be introduced in the camera model.
For all the tests, the luminance range of the gradient GT scene spans 6 decades, from 0.1 cd·m−2 to 100,000 cd·m−2.
6. Discussions and Conclusions
Following a theoretical study that allowed us to identify the type of HDR algorithm most likely to deliver reliable results (
Section 3), we conducted a simulation-based evaluation using a camera model. This enabled us to analyze the impact of several parameters characterizing the camera, the image-acquisition method, the HDR reconstruction method, and the properties of the scene. By evaluating the influence of each parameter, we were able to observe their effects in terms of random error (noise), systematic error (bias), and the dynamic range of the acquired image.
The parameters associated with the camera and the scene introduce varying levels of noise or bias. Inaccuracies in the integration time (εtint) and sensor non-linearity introduce only bias: limited to high luminance values for εtint, and present across the entire luminance range for non-linearity. Conversely, photon noise, dark signal (when a dark correction is applied), readout noise, and quantization all affect the noise in the HDR image. This noise is very high at low luminance levels and then typically stabilizes within a given range for higher luminance values, with photon noise dominating the overall behavior.
The HDR-method parameters tested show results that are strongly correlated with the camera characteristics. For example, the choice of a small integration time reduction factor can minimize the impact of camera noise only when the scene’s temporal stability is sufficient, as the overall measurement time increases. Short minimum and long maximum integration times also help improve the dynamic range of the HDR image, provided that noise related to integration-time accuracy, thermal photon generation, or scene instability does not degrade the captured information. Finally, the study of HDR-algorithm thresholds showed that overly strict thresholds are counterproductive: the best strategy is to select thresholds that are as loose as possible (i.e., as close as the minimum and maximum possible values) while keeping non-linearity within an acceptable range. To make an informed choice, it is therefore necessary to know the sensor’s non-linearity or, at least, to have an estimate of the residual non-linearity according to the pixel value when a non-linearity correction is applied.
The tested algorithms exhibit only slightly different performance depending on the type of noise. The best exposure (BE) and linear regression (LR) algorithms perform marginally better than the time weighted average (tint-WA) algorithms in the presence of integration-time errors and readout noise. They also show greater robustness when the scene luminance fluctuates. For other error sources (photon noise, dark signal, and non-linearity), the tint-WA algorithms provide slightly better results, especially in terms of signal-to-noise ratio (SNR) of the calculated HDR luminance values. However, these differences remain generally small and relatively insignificant compared with the magnitude of the overall HDR luminance measurement error. The order of magnitude of the error remains essentially unchanged regardless of the algorithm used, and the optimal algorithm strongly depends on the type of camera used, with a better SNR generally obtained using the tint-WA algorithm, but a possibly lower systematic error using the BE algorithm. A noteworthy exception is the case of a systematic integration-time error, for which the BE and LR algorithms can help reduce the resulting systematic bias. From these observations, criteria other than result accuracy can be considered to choose the most suitable HDR merging algorithm, such as implementation simplicity or easy calculation of uncertainty propagation. For applications where simplicity and accuracy are essential, and where SNR is not the most important criterion, the best-exposure algorithm can be recommended.
Our tests also show that a good dark correction is required for accurate measurements for low luminance levels, and that thermal rate can significantly limit the smallest measurable luminance value by impacting the noise at low luminance levels. As this rate is linked to the sensor’s temperature, using a heat sink can be an easy way to improve HDR measurement results when the sensor is prone to heating. The tests also show that errors can be easily reduced by increasing the number of captured LDR images when acquisition time is not a limiting factor.
The approach implemented in this study is similar to performing a Monte Carlo evaluation of noise propagation for HDR luminance imaging. For the uncorrelated sources of errors, the results obtained here could also be obtained by applying an analytical approach of the uncertainty propagation, following the principles of the GUM [
20]. However, the simulation-based approach also enabled us to assess the impact of certain acquisition parameters for which correlations must be considered.
Finally, although many sources of error have been considered in our evaluation, it should be noted that the camera model proposed here represents an optically ideal camera, as no stray light effects were considered. Stray light (or lens glare), caused by unwanted scattering and reflections within the objective lens, as well as diffraction at the lens aperture, is often the dominant source of error affecting dark areas when the imaged scene presents high luminance contrasts. For a fixed scene geometry and lens aperture, the optical path of light rays within the objective lens remains the same regardless of the integration time. Therefore, considering stray light would not have provided additional insight into the optimal choice of the HDR merging algorithm. However, ILMD users should be fully aware of its significant impact on luminance measurement accuracy, as observed in preliminary studies based on experimental measurements [
18]. We aim at further extending this work through dedicated experimental studies focusing on stray light characterization and its impact on luminance uncertainty. Once a realistic stray light model, applied as a convolution by a point spread function (see
Figure 5), has been incorporated into HDR image formation, the proposed approach could first be confronted with experimental measurements and then applied to consider more realistic scenes, including geometrical information. This would allow extending this work beyond purely luminance measurement issues. In particular, coupling the radiometric model with disability glare or Human Visual System (HVS) models would enable the investigation of perceptual questions.