DRM-Based Colour Photometric Stereo Using Diffuse-Specular Separation for Non-Lambertian Surfaces

This paper presents a photometric stereo (PS) method based on the dichromatic reflectance model (DRM) using colour images. The proposed method estimates surface orientations for surfaces with non-Lambertian reflectance using diffuse-specular separation and contains two steps. The first step, referred to as diffuse-specular separation, initialises surface orientations in a specular invariant colour subspace and further separates the diffuse and specular components in the RGB space. In the second step, the surface orientations are refined by first initialising specular parameters via solving a log-linear regression problem owing to the separation and then fitting the DRM using Levenburg-Marquardt algorithm. Since reliable information from diffuse reflection free from specularities is adopted in the initialisations, the proposed method is robust and feasible with less observations. At pixels where dense non-Lambertian reflectances appear, signals from specularities are exploited to refine the surface orientations and the additionally acquired specular parameters are potentially valuable for more applications, such as digital relighting. The effectiveness of the newly proposed surface normal refinement step was evaluated and the accuracy in estimating surface orientations was enhanced around 30% on average by including this step. The proposed method was also proven effective in an experiment using synthetic input images comprised of twenty-four different reflectances of dielectric materals. A comparison with nine other PS methods on five representative datasets further prove the validity of the proposed method.


Introduction
Photometric stereo (PS) estimates surface orientations using images captured from a fixed viewpoint under various illuminations and is especially powerful in acquisition of fine surface details at pixel level [1]. Surface orientation is important in a variety of fields, such as geometry segmentation for three-dimensional (3D) object recognition [2] and digital re-rendering in computer graphics. Surface geometries, which can be obtained via integrating surface orientations, have also been proven useful for applications, such as industrial quality control and reverse engineering. Due to the strength of PS and the importance of surface orientation acquisition, PS has drawn increasing interests since its debut [3]. However, making PS for a general real scene remains challenging due to the diverse reflectance properties of different materials that appear non-Lambertian [4]. This has given rise to the need for reliable estimation of surface orientations for a wide range of non-Lambertian reflectance, which essentially requires a proper imaging photometry model characterizing the forward problem and a subsequent PS method that inversely derives surface orientations.
Existing PS methods dealing with non-Lambertian reflectance can be classified into three categories. The first approximates surface reflectance using analytical bidirectional reflectance distribution function (BRDF) [5] and formulates the estimation of surface orientations as a nonlinear fitting problem. Nayar et al. [6] derived surface orientations using the Torrance-Sparrow BRDF [7] and incorporated extended sources to ensure sufficient information from specularities. Georghiades [8] inverted the same BRDF to simultaneously estimate surface orientations and resolve the generalized bas-relief ambiguity. Goldman et al. [9] assumed that general material reflectance can be represented by a convex combination of fundamental materials characterized by the Ward BRDF [10] and recovered surface orientations, fundamental material BRDFs and weight maps simultaneously for further scene editing purpose. Methods in this category exploiting information from surface reflectance are capable to derive not only surface orientations but also the other parameters in the analytical BRDFs, allowing more functionalities, such as digital relighting and material classification. However, due to the nonlinearity of analytical BRDFs and larger number of parameters to be estimated, these methods are sensitive to initialisations, numerically unstable under heavily corrupted outliers (e.g., shadows), and inapplicable when the number of observations is limited.
The second category of methods infers surface orientations through adopting the general properties of BRDF, such as isotropy, monotonicity and reciprocity. Alldrin et al. [11] developed a non-parametric PS method using bi-variate approximation of the isotropy property. Higo et al. [12] analyzed the general BRDF constraints and employed the BRDF properties of monotonicity, isotropy and visibility to vote for the most possible surface orientations for single-lobed reflectance. Shi et al. [13] proposed a bi-polynomial representation for low-frequency reflectance that was especially adaptable to the inverse problem as PS, while Ikehata et al. [14] developed another general isotropic BRDF as sum of lobes with unknown center directions. Methods in this category capitalize on the most fundamental properties of BRDF and therefore have the potential to deal with a broader range of reflectance. However, these methods are only capable to derive surface orientations with limited other functionalities and require an even larger set of observations compared with the first category. To widen the applicability, recent years have seen non-parametric BRDFs based on machine learning. Santo et al. [15,16] used a deep neural network for the first time whereas Taniai and Maehara [17] estimated surface normals and BRDFs by unsupervised learning. Ikehata [18] estimated surface normals more straightforward by deriving the so-called observation maps and using convolutional neural networks. Such data-driven approaches clearly improves the accuracy as observations directly create the BRDF or estimate surface normals. The use of a significantly large set of observations, which is key to the accuracy, and the expensive training effort using the observations make the approaches outside the scope of this paper.
Methods in the third category assume that non-Lambertian effects appear sparsely among observations and treat them as outliers. A substantial corpus of methods rely on robust statistical techniques for outlier rejection. Wu et al. [19] formulated the PS with outlier rejection as a global rank minimization problem, while Ikehata et al. [20] employed sparse Bayesian regression instead. Barsky et al. [21] initiated another line of PS researches using colour images where they first identified the significance of using specular colour in the dichromatic reflectance model (DRM) [22] for specularity rejection. Using the cue from known specular colour under the same theoretical foundation, Zickler et al. [23] derived a PS method in a novel two-dimensional (2D) specular invariant colour subspace. The major advantages of methods in the third category are their robustness and requirement for less images, while they are inefficient in the presence of dense non-Lambertian effects.
This paper presents a colour PS method to estimate surface orientations using diffusespecular separation dealing with non-Lambertian reflectance. The proposed method models the colour imaging photometry using DRM as the forward problem and inverts it following a two-step procedure, the diffuse-specular separation and the surface normal refinement. The first step, initialising surface orientations using known specular colour similarly to [23], separates the diffuse and specular components in the RGB space and identifies outliers to reject in the UV space. In the second step, the parameters characterizing the specularity are initialised via solving a log-linear regression problem and the surface orientations are finally refined by fitting the nonlinear DRM using Levenburg-Marquardt algorithm (LMA). The proposed method robustly initialises surface orientations in a specular-free colour subspace and further fits for the nonlinear DRM with a newly derived parameter initialisation strategy. The proposed method preserves the advantage of robustness as the third category of methods, while it also benefits from the separated specular component to tackle with dense non-Lambertian reflectance as the first category. Furthermore, DRM parameters besides the surface orientations can be additionally obtained where dense non-Lambertian reflectance appears, making more potential applications feasible, such as digital relighting and material classification.
The rest of the paper is organized as follows. The next section presents the problem formulation of colour PS incorporating DRM and the relevant PS methods. Section 3 first overviews the flow and original contribution of the proposed colour DRM-based PS method and then, elucidates the specific two steps in the subsequent subsections. It follows the comprehensive evaluations on surface orientation estimation using both synthetic and real images in Section 4, while the last section summarizes the conclusions and proposes the future works. Figure 1 shows the schematic diagram of the hardware components of the sensor and the necessary data for the generic colour PS. The hardware components are an RGB camera and a set of directional point sources whose positions are priorly known. Assume that ambient light is completely blocked, the colour PS requires that the RGB camera captures a colour image for the target surface with one source lit at a time. Given the N colour images illuminated under N different sources, the objective of the colour PS is to identify surface orientations pixel-wise. More specifically, given the colour image irradiance measurements at pixel (i, j), {e i,j * k } k∈ [1,N] , under the unit illumination directions, {l i,j k } k∈ [1,N] , estimated from the light positions, the colour PS aims to derive the unit surface normal,n i,j , by minimizing the sum of N corresponding square residuals:

Generic Colour PS Problem Formulation
(1) where w is a vector of weights for colour channels whereas f is a forward function of n i,j havingl i,j k as parameters, which corresponds to the weighted measurements. Note that() and () * mean the unit version and the measurement of () respectively throughout the paper.  Figure 2 shows the coordinate setups that are necessary for the colour PS to model the imaging geometry and the light configuration. The coordinate frames are the 3D camera frame, {C}, the 2D pixel frame, {P}, and the world frame, {W}, where its XY plane is known and referred to as the reference plane. These coordinate frames follow the convention given in [24]. A 3D surface point, {C} X, is projected to be a 2D image point, {P} x, in the RGB camera using the perspective projection as:

Colour Image Formation Model
where m x and m y are each the number of pixels per unit distance along x and y direction respectively. o x and o y are the principal point location in {P}, and f is the effective rasterized into a regular grid in the camera as: where ceil(·) denotes the ceiling operator. {W} and {C} are related with a 3D rigid transformation: Let the kth light position be where {C} r k , {C} θ k and {C} φ k are the radius, zenith and azimuth angle, respectively. The N lights are configured such that they have the same zenith angle and radius, and uniformly distributed azimuth angles around the optical axis, i.e., {C} θ k = θ, {C} r k = r and {C} φ k = 360 • (k−1) N + φ 0 . Having established the imaging geometry and light configuration, the imaging photometry is then modeled using the physically-based DRM [22]. The DRM properly characterizes reflectance for dielectric materials [25] and is represented by a linear combination of the diffuse and specular components. Assume that pixel (i, j) is shadow-free, the image irradiance for colour channel c after light strength normalisation and vignetting correction is written as: e i,j where e i,j d,c and e i,j s,c are the diffuse and specular components, and d i,j c and s c are diffuse and specular colours which are wavelength-dependent and represented by: In the equation, λ is the wavelength, Φ(λ) is the source spectral power density (SPD), R i,j (λ) is the spectral body reflectance, and C c (λ) is the camera spectral sensitivity for the colour channel. f i,j d and f i,j s represent the diffuse and specular geometrical scaling factors that are characterized by BRDF.
Since the scaling factors are illumination-dependent, let the image irradiance Equation (5) generalized in the RGB colour space be with the kth illumination as: whered wherev i,j is the unit viewer direction and determined by:

Colour PS Methods Using DRM
Existing colour PS methods using DRM contain two major variants, representative works of which were developed by Barsky et al. [21] and by Zickler et al. [23]. The PS method of Barsky finds the unit surface normaln i,j and the diffuse reflectance factor k i,j d such that the residual between the colour image irradiances weighted byd i,j RGB and the diffuse geometrical scaling factor is minimised: (12) where N b (≤ N) is the number of specular-free and shadow-free images. The reason for the use of N b instead of N is that the Barsky's PS method assumes that non-Lambertian reflectance appears sparsely among observations, which should be rejected as outliers. As the first process for outlier rejection, the direct principal component analysis (DPCA) [28] on A threshold on f i,j s,k is finally used to determine pixels to be rejected as outliers, and this subsequently determines N b .
The objective function (12) of this PS method is not different from that of the conventional Lambertian-based PS method [3]. Given {l i,j k } k∈[1,M] a priori,n i,j together with k i,j d , which minimizes the objective function (12), is analytically determined as: and rank( L i,j ) = 3. While the effect of specularity rejection through the use ofs RGB is exhibited, the estimation ofd i,j RGB using DPCA is erroneous due to the existence of specularities. Furthermore, all the specularities are rejected as outliers though useful information may be contained. Unlike Barsky's PS method, which performs PS in the RGB colour space, the PS method of Zickler estimates surface orientations in the SUV colour space, or the UV colour space as S is not considered: where α i,j is a scaling factor. The UV colour subspace includes more observations than the RGB colour space in surface orientation estimation. To operate in the SUV space, the colour image irradiance is transformed from RGB to SUV as: UV . It then becomes the conventional Lambertian-based PS problem and the solution is obtained in the similar form to Equation (14). Compared with the Barsky's PS, the Zickler's PS utilizes more observations and advantageously performs PS in the specularfree UV colour subspace. Shape information from specularities is however neglected again, and the method can only provide diffuse component up to the normalised RGB space. The diffuse-specular separation in the RGB space remains open. Figure 3 shows the flow and original contributions of the proposed DRM-based colour PS method using the diffuse-specular separation. The proposed method consists of two processes:

Overview of the Proposed Color PS Method
Step 1, which is diffuse-specular separation using a PS, and Step 2, which is the surface normal refinement.
Step 1 identifies good approximate surface normals, whereas Step 2 further refines and finalizes them with maximum accuracy. The steps are marked by red boxes.
Step 1 is composed of four sub-processes: diffuse colour estimation, diffuse-specular separation in RGB space, PS in UV space and the outlier estimation. After completing shadow rejection as a preprocess, and generating the shadow-free image irradiance matrix Step 1 begins with the diffuse colour estimation using the robust principal component analysis (RPCA) proposed in this paper and derivesd i,j RGB . Note that () is a quantity of () after shadow rejection. The diffuse color estimation also outputs a specularity map, which describes the distribution of specular reflection. The diffuse-specular separability is then checked for each pixel by deriving the diffuse-specular chromatic angle ψ i,j in RGB space. For separable pixels, the proposed method performs PS in the specular-free UV colour subspace similarly to Zickler's PS method and derives the initial guess of the unit surface normal n i,j 1 and the diffuse reflectance factor k i,j d 1 similarly to Barsky's PS. () 1 means the initial guess of (). In parallel, the specular geometrical scaling factors, f i,j s 1 , are derived through the diffuse-specular separation process using Equation (13)  The strength of the proposed method lies in this PS with the diffuse-specular separation. For pixels where sparse non-Lambertian reflectances appear, the first step is sufficient to reliably estimate the surface orientations. To tackle with more general problems with dense non-Lambertian reflectances, the second step is additionally required to exploit surface normal information from specularities in the DRM with the designed parameter initialisation strategy. The rest of this section elucidates the proposed method in detail.

Diffuse Color Estimation
The proposed RPCA derivesd i,j RGB and the specularity map as follows: 3. Compute residual vector: r and register (i, j) in the specularity map. Remove the corresponding row vector in E i,j * RGB and repeat from step 1.
The RPCA eliminates image irradiances that are inappropriate for diffuse colour estimation.
The validity of the RPCA can be explained as follows. The shadow-free image irradiance matrix E i,j * RGB is expanded from Equation (7) and decomposed as While the originality and superiority of the proposed RPCA algorithms have been identified, the performance of the RPCA algorithms depends on the value of T d . If T d is too large, the RPCA algorithms are tolerant to specularities and perform similarly to the DPCA algorithms. If T d is too small, the RPCA algorithms reject innocent pixels as specularities and make images sensitive to noise.

Diffuse-Specular Separability Check
Havingd i,j RGB estimated, the diffuse-specular chromatic angle, used to perform the separability check, is derived as Asymptotically, the chromatic angle gives This means that ψ i,j should be sufficiently larger than 0 in order ford i,j RGB ands RGB to be distinct and separable. The threshold of the chromatic angle to determine the separability is T c . If ψ i,j ≥ T c ,d

PS in UV Space
Since the PS is performed in the UV colour space, the unit diffuse colourd is a unit vector, and  (15) to reject noise-corrupted image irradiances as outliers using studentised residuals [29]: where N p is the number of shadow-free images, so N b ≤ N p ≤ N. The derivation of where 0 N is an N × 1 vector with all zero entries. The specular component matrix, E i,j s,RGB , is obtained by max f i,j s 1s T RGB , 0 N . With the separated colour diffuse and specular components, the proposed method allows the functionality of specular removal and intrinsic image decomposition.

Outlier Estimation
Let the residual vector r i,j be written as: where n i,j = ρ i,jni,j is the scaled normal, L i,j = l i,j 1 ,l Diagonal entries in H i,j a are leverages and the kth leverage is denoted as h i,j a,k . These leverages quantify the influence that the observed E i,j UV d i,j UV on their predicted values L i,j n i,j . Each entry in the studentized residual vectorr i,j is then approximated using: where (MSE) i,j represents the mean squared error of r i,j . If any entry in |r i,j | exceeds T o , this entry is estimated as outliers. The proposed PS rejects outliers until no more can be detected or (MSE) i,j is smaller than a tolerance, T m . With the general rule of thumb in detecting outliers using studentised residuals, T o and T m can be chosen as 2.5 and 9σ 2 n , where σ n is the standard deviation of additive white Gaussian noise that can be estimated using the method given by [30].

Specular Parameter Initialisation
If dense non-Lambertian reflectance appears at pixel (i, j), which implies that more than one entry in f i,j s 1 are in the specularity map, surface normal refinement is necessary since the sparse non-Lambertian reflectance assumption in the diffuse-specular separation is violated. With the estimated f i,j s 1 , the next objective is to initialise the specular parameters, k i,j s 1 and β i,j 1 . The specific cost functional is given by: where N q is the number of specularity maps showing pixel (i, j) and N q ≥ 2. Equation (26) suggests a nonlinear least-squares problem, while it can be manipulated to the natural logarithm domain as a linear problem given by: where S i,j = ln H i,j n i,j 1 , 1 N q (29) and 1 N q is a N q × 1 vector with all entries equal to 1.

Surface Normal Refinement in DRM
All parameters in the DRM have been initialised up to this point. The LMA is then employed to iteratively refine these parameters by solving the optimization problem: where the first three terms are from the generalised formulation, and, the fourth regularization term with constant T α is added. The regularisation term reduces the change of surface normal refinement from the initial guess of n i,j 1 and prevents overfitting. Larger value of T α strengthens the robustness but limits the capability of surface normal refinement.
The strength of the proposed PS method lies in the determination of the surface normals while separating the diffuse and specular components in RGB space, eliminating outliers and then tuning the diffuse and specular reflectance factors simultaneously. The de-termination considering diffuse and specular reflections simultaneously makes the surface normal estimation more accurately. In addition, the proposed method can be used for specular removal [31] and intrinsic image decomposition [32]. Parameters characterizing f s,k are determined at pixels, so the proposed method could also be used for digital relighting and material classification where dense non-Lambertian reflectance appears.

Evaluations Using Synthetic Images
The first experiment aims to evaluate the effectiveness of surface normal refinement for dense non-Lambertian reflectance using synthetic input images. A scene with six different-coloured spheres were rendered under 32 illuminants using the Blinn-Phong model. The sphere was adopted as the scene geometry since it samples all the possibilities of surface orientations for the visible surface. Different colours of the spheres were introduced for more comprehensive evaluations of the proposed PS method on surfaces with various spectral reflectances. The 32 illuminants were chosen to provide a sufficient number of pixels with dense non-Lambertian reflectance for evaluation.s RGB was set at   Figure 4b shows the angular error of surface orientation estimation without surface normal refinement. As is shown, the central regions of the spheres where dense non-Lambertian reflectances appear have larger error. By including the proposed surface normal refinement step, the angular errors of surface orientations around the detected dense non-Lambertian reflectance region, shown by a ring, are significantly reduced as shown by Figure 4c. This is where the reflectance parameters are most tuned through the surface normal refinement. Errors in such regions are even smaller than those with sparse non-Lambertian reflectance, which further enhances that making specularities as meaningful signals is beneficial for surface orientation estimation. Figure 4d demonstrates the improvement percentage for quantitative evaluation. The improvement percentage is defined reduction of the error divided by the error without surface normal refinement. The accuracy of surface orientations was enhanced by 34.33% in median and 32.25% on average. The first and third quantile values of the improvement were 15.76% and 54.23%, respectively. The performance improvements onto the six different-coloured spheres are shown in Table 1. The improvements were obvious for all spheres is significant, implying that the method was applicable to a wide range of surfaces with different spectral reflectances. For spheres that had larger chromatic angle, ψ i,j , thed i,j RGB estimations were more accurate. The better estimatedd i,j RGB led to the more accurate recovery of f i,j s 1 , resulting in the more reliable initialisation of n i,j 1 . Such initialisations ofn i,j ultimately affected the different improvement performances. Overall, the accuracy was enhancement by including the additional surface normal refinement step and for spectral reflectances with larger chromatic angles, the improvement was more phenomenal. In the second experiment, the performance improvement due to surface normal refinement was verified onto a wide range of reflectances. Experimental settings and parameters of the proposed method were the same as the first experiment except that different values of k i,j d , k i,j s and β i,j were applied to represent various surface reflectances when generating synthetic images. The ratio of κ i,j ds = k i,j d /k i,j s represents the relative strength between the diffuse and specular components, whereas β i,j indicates the width of the specular lobe. The mean, median, first and third quantile values of the improvement were adopted for evaluation and the angular error ofd i,j RGB estimation was used for analyses. As shown from Table 2, the improvements on all the different reflectances were significant and more obvious for larger value of β i,j and κ i,j ds . Larger value of β i,j suggesting narrower specular lobe made thed i,j RGB estimation more accurate since the sparse non-Lambertian reflectance assumption was valid. Similarly, larger value of κ i,j ds indicating stronger diffuse components made betterd i,j RGB estimation due to stronger inliers. More accurated i,j RGB estimation led to bigger improvement from the surface normal refinement. In summary, with the additional surface normal refinement step, the accuracy of surface orientation estimation is enhanced by around 30% on average and the improvement is more obvious for reflectance that has narrower specular lobe and stronger diffuse component. The third experiment aims to evaluate the overall performance of the proposed colour PS method onto a wide variety of dielectric material reflectances. Twenty-four material BRDFs divided into six categories were evaluated, where the different BRDFs were provided from the MERL database [33]. Experimental settings and the method parameters were exactly the same as the first experiment except that the BRDFs were replaced. The image irradiance for the first six materials under the first illuminant is shown in Figure 5a. The angular errors of the surface orientation estimation are given in Figure 5b. As is shown, the error is not only larger at the near central regions where specularities overlap but also at the boundaries. Boundary pixels in the spheres have shallow grazing angles and the error is due to the lack of modeling of the fresnel reflection in the Blinn-Phong model. Figure 5c shows the performance of surface normal estimation on the 24 materials using box-and-whisker plot. The red and black dot represent the mean and median value, respectively. The lower and upper bound of the box indicates the first and third quantile values. The six colours of the boxes suggest the six material categories. As is shown, the proposed method can accurately estimaten i,j for most dielectric materials within 5 degrees on average. The estimation performances on phenolic, plastic and rubber were consistent, whereas those on the wood stain, fabric and acrylic were also acceptable with two exceptions, the violet acrylic and the green fabric. The degradation for violet acrylic was due to its wide specular lobe with small κ i,j ds , while the reason for green fabric was because of its small ψ i,j .

Evaluations Using Real Images
The proposed method was evaluated on five different datasets comprised of real images in the DiLigenT database [4]. Parameters of the proposed method were set the same as the first experiment.s RGB was estimated as [0.5774, 0.5774, 0.5774] T . From the first to fifth column in Figure 6, the results on BUDDHA, BEAR, POT2, READING and GOBLET are respectively shown. Figure 6a shows the image irradiance under the first illuminant, while Figure 6b,c demonstrate the estimated normal map and the angular error of surface orientation estimation, respectively. As is shown from READING, the proposed method is only feasible for surfaces whosed i,j RGB ands RGB are distinct. Figure 6c shows that the dense non-Lambertian problem where the specularity overlaps is mostly solved for BUDDHA, BEAR and POT2, while deficits still exist on GOBLET, which is because the DRM is not proper to characterize metallic reflectance.  The results were compared with those of nine other PS methods which are listed in Table 3. The results from the nine methods were provided from the DiLigenT database and these methods were representative and well-recognized. It is to be noted that the results of the recent machine learning based techniques reviewed in Introduction are not shown for comparison since the effect of the physics based models is investigated in this paper. Due to the data availability, refs. [21,23] were not involved in the comparison, while the essence of their works in using DRM with knowns RGB was well-inherited by the proposed method.  [14] proposed Figure 6d shows the results from ten PS methods on the five datasets. The X-axis represents the method index and the Y-axis indicates the angular error ofn i,j . The five colours of the bar represent the different datasets. The angular error ofn i,j is also demonstrated using the box-and-whisker plot similar to Figure 5. As is shown, the proposed method performs well in estimating surface orientations for all the five datasets except GOBLET. The results suggest that modeling reflectance using DRM is proper for surfaces made of dielectric materials. The strength of the proposed method lies on the reliable estimation ofd i,j RGB with simultaneous specularity detection. The advantage is also accredited by exploitingn i,j not only from the diffuse components but also from the specularities owing to the diffuse-specular separation. In spite of its advantages, the proposed method is not effective for metallic surfaces compared with methods, such as [14]. The limitation originates from the fundamental assumption of DRM thats RGB are the same across the surface. This assumption is violated for metallic surfaces since the wavelength and geometry exhibit inter-dependency.

Conclusions and Future Works
A PS method using colour images dealing with non-Lambertian reflectance has been proposed. The method formulates the imaging photometry using DRM with known specular colour. It extracts surface orientations not only from the diffuse components but also specularities owing to the diffuse-specular separation. Introducing the additional surface refinement step using information from specularities, the proposed method particularly improves the accuracy for surface orientation estimation at pixels where dense non-Lambertian reflectance appear. The simultaneously acquired specular parameters can be applied for more potential functionalities, such as digital relighting and material classification.
From the experiment of validating the effectiveness of the newly proposed surface normal refinement step on surface where dense non-Lambertian reflectance appears, the results indicate that with the additional step, the accuracy is enhanced by around 30% on average and the improvement is more phenominal for surfaces with larger chromatic angles, stronger diffuse components and narrower specular lobes due to the better estimated unit diffuse colour. The result investigating the proposed method applicability on 24 different reflectances of dielectric materials suggests that the average angular error of surface orientation estimation for most materials are within 5 • and bigger errors occur at surface patches with shallower grazing angles due to the limitation of the Blinn-Phong model that does not incorporate the fresnel reflection. From the systematic comparison on five datasets with nine other representative methods, the proposed method shows its descent performance on reflectances of dielectric materials and degradation on metallic surface due to the limitation of DRM. The result has also demonstrated that the proposed method is only feasible for surfaces whose diffuse and specular colour are distinct.
This paper presented the first set of results to estimate surface orientations using colour images for non-Lambertian reflectance, while it can be extended in a variety of ways. First, other analytical BRDFs can be employed instead of the Blinn-Phong model to better account for the fresnel reflection. Second, machine learning based models can be incorporated to learn and reduce errors of analytical models. Further, an alternative method can be proposed specifically for surfaces whose diffuse and specular colours are close to extend the method's applicability. Making the method adaptive is one possible solution. Last but not least, it is also of particular interest to infer surface properties from the estimated specular parameters for the purpose of material classification.