Three-Dimensional Reconstruction Techniques and the Impact of Lighting Conditions on Reconstruction Quality: A Comprehensive Review

Abstract

Three-dimensional (3D) reconstruction has become a fundamental technology in applications ranging from cultural heritage preservation and robotics to forensics and virtual reality. As these applications grow in complexity and realism, the quality of the reconstructed models becomes increasingly critical. Among the many factors that influence reconstruction accuracy, the lighting conditions at capture time remain one of the most influential, yet widely neglected, variables. This review provides a comprehensive survey of classical and modern 3D reconstruction techniques, including Structure from Motion (SfM), Multi-View Stereo (MVS), Photometric Stereo, and recent neural rendering approaches such as Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS), while critically evaluating their performance under varying illumination conditions. We describe how lighting-induced artifacts such as shadows, reflections, and exposure imbalances compromise the reconstruction quality and how different approaches attempt to mitigate these effects. Furthermore, we uncover fundamental gaps in current research, including the lack of standardized lighting-aware benchmarks and the limited robustness of state-of-the-art algorithms in uncontrolled environments. By synthesizing knowledge across fields, this review aims to gain a deeper understanding of the interplay between lighting and reconstruction and provides research directions for the future that emphasize the need for adaptive, lighting-robust solutions in 3D vision systems.

1. Introduction

Three-dimensional (3D) reconstruction is a foundational task of computer vision, enabling the creation of precise digital replicas of real scenes from visual data, most often from 2D images or image sequences. It is a core component of a wide variety of application fields, ranging from the recording of historical artifacts for the preservation of heritage to the interpretation of the world for autonomous robotics, the creation of realistic digital environments needed for virtual and augmented reality, and the precise documentation of crime scenes for investigation and legal proceedings [1,2,3,4]. The diversity of application fields has given rise to the development of numerous reconstruction pipelines, ranging from classical photogrammetric approaches to advanced neural rendering models. Among the most widely used ones are Structure from Motion (SfM) [5,6,7], reconstructing camera poses and sparse geometry from a sequence of images, and Multi-View Stereo (MVS) [8,9], densifying these representations with pixel correspondences. In the recent years, learning-based approaches such as Neural Radiance Fields (NeRFs) [10] and 3D Gaussian Splatting (3DGS) [11,12,13] attracted attention due to their ability to generate novel views and reconstructing complex geometries with high photorealism.

Despite these advancements, a critical yet often underexplored variable that continues to impact the fidelity and reliability of 3D reconstructions is represented by the lighting conditions. Illumination directly influences the appearance of surfaces in images, affecting color consistency, feature visibility, and shading cues that many reconstruction algorithms depend upon. Variations in lighting, such as cast shadows, reflections, specular highlights, or exposure differences, can lead to significant degradation in point cloud density, surface smoothness, and overall geometric accuracy. For example, areas under poor or uneven lighting may result in missing geometry due to the lack of discernible features, while overexposed regions might introduce false depth information. Although some techniques assume stable lighting or rely on preprocessing for normalization, these assumptions rarely hold in uncontrolled or outdoor environments. As a result, the reconstruction models often struggle with robustness when deployed in the wild, and the impact of lighting on reconstruction performance remains an active area of investigation with limited systematic analysis in the existing literature.

Theoretically, this review brings together the understanding from classical and state-of-the-art 3D reconstruction pipelines by examining assumptions and constraints with respect to illumination. We compare the means by which methods represent or circumvent illumination, their photometric robustness, and the underlying geometric or learning-based theoretical frameworks that govern their sensitivity to light variation.

From a practical point of view, this work is a practitioner’s handbook for data acquisition specialists and researchers carrying out 3D reconstruction. By summarizing empirical findings from the literature, we present practical information on the effect of illumination on reconstruction outcomes under real conditions. This includes the detection of common photometric artifacts, data acquisition best practices, and guidance on the selection of appropriate reconstruction methods for varying illumination conditions.

The remainder of this paper is structured as follows. Section 2 introduces relevant 3D reconstruction approaches, comparing classical and state-of-the-art ones. Section 3 is reserved for the contribution of the light conditions at capture to the reconstruction of images, e.g., photometric challenges and modelling approaches. Section 4 shows empirical results and comparative studies measuring the reconstruction quality for different light conditions. In Section 5, we present a critical discussion of the current limitations and methodological gaps. Section 6 introduces future research directions, and Section 7 concludes the review with the most significant insights and proposals.

2. Relevant Techniques

Three-dimensional reconstruction has evolved on a spectrum of methodologies, from the classical geometric approaches to the most recent neural rendering approaches. All these classes operate on different assumptions on scene geometry, image generation, and environmental factors, particularly, lighting. In this section, we give a formal description of the most common reconstruction methods and a concise description of their principles, their strengths, and their known illumination sensitivities.

2.1. Traditional Geometry-Based Methods

2.1.1. Structure from Motion (SfM)

One of the earliest and most influential approaches for 3D reconstruction from unorganized collections of images is Structure from Motion. Structure from Motion estimates sparse 3D structure and camera poses by detecting and matching salient features across views. SURF [14,15,16] or SIFT [17,18] feature detectors would usually be employed, and bundle adjustment would be used for camera parameter and point location optimization. Despite being incredibly powerful for textured scenes with uniform illumination, SfM is sensitive to photometric inconsistencies. Feature matching can be caused by shadows, low contrast, and reflections, leading to sparse or false reconstructions.

2.1.2. Multi-View Stereo (MVS)

Once sparse reconstruction is obtained with the assistance of SfM, Multi-View Stereo can densify the point cloud by estimating depth maps from overlapping parts of the images [19]. MVS relies on photometric consistency, i.e., the fact that corresponding pixels from different views must be of the same element. Photometric consistency can be violated with the variation of light, and MVS is particularly sensitive, therefore, to illumination variation, specularities, and shadows. Despite some advanced versions incorporating surface regularization or shading models, the technique remains fundamentally light-sensitive.

2.1.3. Photometric Stereo

Photometric Stereo reconstructs high-resolution surface orientation data by viewing the same stationary scene using a number of images under differently directed lights, with the light source being known, and the surface reflectance being Lambertian [20,21]. Even though effective for the recovery of high-resolution fine geometries, the technique is heavily constrained and not suitable for unstructured environments. In addition, it depends on a known light configuration, which is not practical for most real cases outside the laboratory.

2.2. Learning-Based and Neural Rendering Methods

2.2.1. Volumetric and Depth-Fusion Approaches

Learning-based volumetric methods make use of CNNs for learning voxelized scene representations or depth maps from images [22]. Some illumination invariances can be learned by these models, but these will be heavily data-driven and may not generalize across new illumination conditions, unless trained specifically for the purpose.

2.2.2. Neural Radiance Fields (NeRFs)

NeRFs render a scene as a continuous volumetric function parameterized by a neural network describing color and density at any point in 3D, conditioned on position and view direction. NeRFs have performed extremely well on view synthesis and scene reconstruction, particularly under constrained conditions. NeRFs, however, rely on a stationary illumination condition for input images and are not invariant to dynamic light or shadows, unless specifically adapted (e.g., with relighting-aware extensions). As they integrate lighting into the learned radiance field, generalization across new light conditions or robustness against changing light remains a core challenge.

2.2.3. Three-Dimensional Gaussian Splatting (3DGS)

Three-Dimensional Gaussian Splatting is a recent rendering method that is based on the scene being represented as a group of 3D Gaussian anisotropic primitives with color and transparency information. Three-Dimensional Gaussian Splatting is capable of real-time rendering and high-visual-fidelity rendering. Despite the fact that it lacks a native dynamic or varying light mechanism, its explicit management of spatial data leaves room for the inclusion of light-aware rendering pipelines. However, the current implementations make the assumption of uniform lighting on the training images and may be suboptimal for strong illumination changes.

To facilitate a clearer comparison of the discussed 3D reconstruction methods,

Table 1. Comparison of 3D reconstruction methods and their sensitivity to lighting conditions.

Method	Type	Strengths	Weaknesses	Sensitivity to Lighting	Suitable Environments
Structure from Motion (SfM)	Classical (Geometric)	Robust for sparse geometry; well-established; scalable	Highly sensitive to lighting changes; fails under shadows, low contrast, and reflections	High	Controlled/Semi-Controlled
Multi-View Stereo (MVS)	Classical (Geometric)	Produces dense reconstructions; accurate under stable illumination	Relies on photometric consistency; breaks under specularities, shadows, or exposure shifts	High	Controlled
Photometric Stereo	Classical (Lighting-Based)	Fine surface detail recovery; accurate normals under lab settings	Requires known light sources and Lambertian surfaces; impractical in real-world scenes	High	Highly Controlled
Volumetric CNNs	Learning-Based	Can learn illumination invariance; good on learned domains	Requires large training sets; limited generalization under unseen lighting	Medium	Semi-Controlled
Neural Radiance Fields (NeRFs)	Neural Rendering	High photorealism; novel view synthesis	Assumes fixed illumination; entangles lighting with geometry; limited real-world robustness	Medium–High	Controlled
NeRF-W	Neural Rendering	Handles unstructured image sets; compensates for lighting differences	Weak metric accuracy; high data demands; computationally intensive	Medium	Uncontrolled/Semi-Controlled
3D Gaussian Splatting (3DGS)	Neural Rendering	Real-time rendering; high visual fidelity	No built-in light modeling; current versions assume static lighting	Medium–High	Controlled/Semi-Controlled

summarizes their core characteristics, including method type, strengths and weaknesses, sensitivity to lighting variation, and suitability for different environmental conditions. This comparative overview highlights the trade-offs between classical geometry-based techniques and modern learning-based or neural rendering approaches, particularly in relation to their robustness under varying illumination.

3. Lighting Conditions in 3D Reconstruction

Lighting is one of the most critical factors influencing the performance of image-based 3D reconstruction techniques. Most reconstruction pipelines, whether geometric or learning-based, rely heavily on visual cues derived from pixel intensity, texture, shading, and color consistency. Variations in lighting during image capture can significantly alter these visual features, leading to mismatches, depth estimation errors, and loss of geometric fidelity. In this section, we analyze how lighting conditions interfere with reconstruction quality and explore the typical assumptions, challenges, and emerging solutions associated with this problem.

3.1. Photometric Challenges in Reconstruction

Photometric consistency is a foundational assumption in many 3D reconstruction pipelines, particularly those based on geometry-driven approaches such as Structure from Motion (SfM) and Multi-View Stereo (MVS). This assumption implies that the appearance of a point in a 3D scene remains similar across multiple images taken from different viewpoints. In mathematical terms, it assumes Lambertian reflectance properties, according to which a surface reflects light uniformly in all directions, and image intensities vary only due to geometric transformations, not to lighting changes. However, in real-world conditions, this assumption is frequently violated.

One of the most significant disruptions to photometric consistency arises from shadowing. Shadows can obscure important features, drastically altering their local contrast or causing them to disappear entirely from certain viewpoints. This makes feature detection and matching highly unreliable in those regions. Furthermore, self-shadowing, consisting in parts of an object casting shadows onto each other, creates apparent changes in the surface structure that confuse the depth estimation algorithms.

Another challenge is introduced by specular highlights. Unlike diffuse reflections, specular reflections depend on both the viewing angle and the light source direction, causing bright spots that move across the surface depending on the camera pose. These view-dependent effects violate the assumption of photometric invariance and lead to false correspondences in stereo matching, introducing ghosting artifacts or distorted geometry [23,24,25].

Exposure variation also significantly affects photometric stability, especially in uncontrolled environments or when using automatic camera settings. Changes in exposure between frames can result in varying brightness and contrast levels, even when the scene remains static. This leads to inconsistent pixel values across views, degrading the effectiveness of both feature-based and dense reconstruction methods [26]. In learning-based approaches, inconsistent lighting between training views can confuse the neural model, causing it to encode lighting as part of the scene geometry or material, which reduces generalization and reconstruction accuracy.

Color temperature shifts further complicate the reconstruction process. Scenes illuminated by different light sources (e.g., daylight, incandescent, LED) exhibit color casts that alter the RGB values in non-linear ways. For example, an object may appear warmer under tungsten lighting and cooler under daylight, even though its geometry is unchanged. Without proper white balance or color correction, these variations interfere with feature descriptor matching and learning-based texture synthesis.

Additionally, dynamic lighting environments, where the light source moves or changes intensity over time, introduce further variability. This is particularly problematic for outdoor scenes captured over extended periods, such as drone footage, time-lapse scanning, or forensic scene reconstruction under mixed natural and artificial lighting.

In essence, any violation of the photometric consistency assumption introduces noise, ambiguity, or missing data into the reconstruction pipeline. While robust algorithms may handle minor lighting differences, significant photometric inconsistencies often require specialized preprocessing or architectural adaptations to preserve reconstruction quality.

3.2. Modeling and Mitigating the Lighting Effects

As lighting inconsistencies represent a major source of error in 3D reconstruction, various strategies have been developed to model, mitigate, or adapt to these photometric variations. These approaches can broadly be classified into preprocessing techniques, photometric-invariant descriptors, shading-aware reconstruction models, and neural methods with explicit lighting modelling. Each strategy offers different trade-offs between robustness, computational complexity, and generalizability.

3.2.1. Preprocessing and Normalization

One of the most accessible methods to deal with lighting inconsistencies is image preprocessing before feeding data into a reconstruction pipeline. This often includes techniques such as:

• Histogram equalization, which standardizes the intensity distribution across images to reduce contrast disparities.
• White balance correction, which adjusts the image to a standard neutral gray, reducing color temperature shifts due to different light sources.
• Retinex theory-based methods, which attempt to decompose an image into illumination and reflectance components, preserving structural details while suppressing the lighting effects [27].

While these methods are relatively easy to apply and computationally efficient, they may also degrade valuable photometric cues necessary for certain types of depth estimation, such as stereo matching that relies on subtle shading gradients. Furthermore, they can introduce artifacts or overcorrections in scenes with mixed lighting or high dynamic range.

3.2.2. Photometric-Invariant Feature Descriptors

To address the problem at the algorithmic level, researchers have developed illumination-invariant descriptors for feature detection and matching. Examples include

• Gradient-based descriptors (e.g., normalized gradient orientation), which are less sensitive to absolute intensity.
• Color-invariant representations, which transform RGB values into chromaticity coordinates, isolating hue from illumination.
• Moment-based features like Zernike moments and local phase information, which capture structural patterns rather than raw intensity [28,29].

These descriptors enhance robustness against brightness and color variation, particularly in SfM and MVS pipelines that rely on consistent keypoint detection. However, they may lose precision in textured regions and often trade fine details for robustness, which can result in smoother but less accurate reconstructions.

Recent learning-based approaches aim to overcome these limitations by integrating geometric priors into the reconstruction process. GeoMVSNet [30], for example, enhances Multi-View Stereo performance under non-ideal lighting by combining learned feature representations with geometric consistency constraints across multiple views.

3.2.3. Shading-Aware Reconstruction Techniques

Another line of defense involves explicitly modelling how lighting interacts with surface geometry, which is particularly relevant in shading-aware reconstruction. For instance:

• Intrinsic image decomposition techniques aim to separate an image into reflectance and shading layers, isolating an object color from illumination effects [31]
• Photometric bundle adjustment, which jointly optimizes camera parameters and surface appearance under varying lighting, has been applied in some hybrid SfM systems [32].

These methods allow reconstruction pipelines to reason about scene lighting rather than suppressing it, enabling a more accurate modeling of shape and texture. However, they often require prior knowledge about scene materials or light sources and are computationally expensive. Many of these techniques also struggle with non-Lambertian surfaces (e.g., glossy or transparent materials).

3.2.4. Learning-Based Approaches with Lighting Awareness

Recent advances in deep learning have enabled data-driven approaches to model and disentangle lighting from geometry. Several notable techniques include

• NeRF-W (Neural Radiance Fields in the Wild), which augments NeRFs with appearance embeddings that account for lighting variations between views, enabling more robust reconstructions in unconstrained environments [33].
• Neural reflectance decomposition models, such as NeRD [34], which aims to learn separate latent representations for shape, material, and illumination.
• Relighting-aware networks, trained with synthetic datasets under variable lighting, to infer canonical scene representations that can generalize across different illumination setups.

These models represent a significant leap toward real-world robustness but come with their own limitations. They often require large and diverse training datasets, multi-view supervision, or calibrated lighting information. Additionally, while many of these methods are highly effective at novel view synthesis, their utility in generating metrically accurate 3D geometry is still under active research.

While the original formulation of 3D Gaussian Splatting assumes fixed illumination, recent extensions have adapted it to challenging lighting scenarios. Luminance-GS introduces view-adaptive curve adjustments to enhance the rendering quality without modifying the core splatting mechanism [35]. Additionally, LumiGauss leverages spherical harmonics to achieve relightable Gaussian Splatting, supporting more realistic reconstructions under novel lighting conditions [36].

A qualitative comparison of several recent neural approaches under challenging lighting conditions is shown in Figure 1.

3.2.5. Hybrid and Adaptive Systems

Some recent pipelines aim to combine a number of strategies, unifying preprocessing, shading models, and data learning for the purpose of building adaptive systems that can function under a wide range of light. Real-time SLAM for robotics [37], for example, has begun incorporating learned light correction modules, and some of the uses of the technology for forensics and heritage purposes include the application of light rigs with software-level compensation. These hybrid methods show great potential but are still rare and application-specific.

To summarize the evolution of 3D reconstruction methods and their increasing attention to photometric robustness, Figure 2 provides a conceptual overview. It illustrates their development from classical geometry-based techniques to modern learning-based and hybrid pipelines, including those designed to explicitly handle illumination variation.

3.3. Controlled vs. Uncontrolled Environments

The lighting conditions during image capture significantly influence the performance of 3D reconstruction methods, especially when photometric consistency is assumed. One of the most important distinctions affecting reconstruction robustness is whether the environment is controlled or uncontrolled. This section highlights the characteristics of each setting and their impact on reconstruction fidelity.

Controlled environments refer to scenes captured under carefully regulated conditions, often found in studios, laboratories, or structured scanning setups. These typically include

• Uniform diffuse lighting to minimize harsh shadows and highlights.
• Fixed camera exposure and white balance settings to ensure photometric consistency.
• Stationary or synchronized lighting setups, such as ring lights, softboxes, or dome lighting systems that eliminate directional shadows.

Such settings exist for the digitization of heritage, factory inspection, and laboratory-based 3D scanning, for which geometric accuracy and repeatability are critical. In such cases, the illumination is incorporated into the design of the capture pipeline, and calibration is undertaken with utmost care such that photometric artifacts are eliminated or minimized [38].

Reconstruction under constrained conditions will yield high-fidelity results with dense point clouds, clean meshes, and accurate surface textures. In fact, data captured under such conditions form the test basis for most reconstruction algorithms. However, models trained or tuned on these datasets may fail to generalize when deployed in the field.

Uncontrolled environments, by contrast, feature variable and unpredictable lighting. These conditions are common in

• Outdoor environments, where illumination is affected by the time of day, weather, and shadows cast by surrounding structures.
• Indoor scenes with mixed lighting, such as a combination of natural light from windows and artificial bulbs of varying color temperatures.
• Forensic or emergency situations, where documentation must be conducted quickly without the ability to manipulate lighting.

Under these conditions, photometric inconsistencies become particularly prominent. The direction, intensity, and spectrum of light can vary from one picture to the next, even for a series of pictures. Furthermore, the presence of moving shadows, reflective and translucent surfaces, and automatic camera exposure control adds complexity.

These unstructured environments directly undermine the efficiency of methods like SfM and MVS, which are reliant on regular keypoint correspondences and brightness patterns across views. Even learning-based methods trained under stationary illumination can confuse shadow boundaries with object boundaries or confuse specular highlights with geometric features [39].

4. Empirical Evidence from the Literature

While theoretical analyses highlight the sensitivity of 3D reconstruction techniques to lighting variations, empirical validation is essential for understanding the practical impact of illumination in real-world and synthetic conditions. This section reviews notable experimental studies, benchmark datasets, and comparative evaluations that explore how lighting affects reconstruction fidelity. By synthesizing these findings, we aim to identify patterns in algorithmic robustness and outline where the current methods fall short under photometric variability.

4.1. Benchmark Datasets with Lighting Variability

Several benchmark datasets have been created to support research on 3D reconstruction. However, relatively few of them deliberately introduce lighting variation as a core variable. Among the most relevant is the DTU dataset [40,41], which captures over 120 scenes with multiple camera positions and different lighting setups. The dataset includes scans with strong shadows, side lighting, and ambient-only configurations. These variations have proven useful for testing robustness in Multi-View Stereo (MVS) pipelines, revealing substantial performance drops under directional lighting.

Similarly, the Middlebury Multi-Illumination Stereo dataset [42,43,44,45,46,47] provides stereo pairs of static objects photographed under up to 64 different lighting conditions using a dome rig. This setup enables the isolated analysis of how the lighting direction impacts correspondence matching and disparity map quality. Studies using this dataset have demonstrated that traditional stereo algorithms often fail under directional lighting due to strong cast shadows and specular reflections.

However, mainstream benchmarks like Tanks and Temples [48] or ETH3D [49,50] focus primarily on geometric diversity and scene scale rather than photometric diversity. These datasets are invaluable for testing algorithmic scalability but provide limited insights into lighting robustness. As a result, evaluations based solely on such benchmarks may overestimate the real-world generalizability of reconstruction methods.

To provide a structured overview of the available resources for evaluating lighting-related challenges in 3D reconstruction,

Table 2. Overview of benchmark datasets used in 3D reconstruction, focusing on the presence of lighting variation, capturing environment, and their typical application domains.

Dataset Name	Lighting Variation	Environment	No. of Lighting Conditions	Applications
DTU Dataset	Yes	Indoor	Multiple setups (ambient, directional)	MVS, photometric robustness testing
Middlebury Multi-Illumination	Yes	Indoor (Lab)	Up to 64 lighting configurations	Stereo matching under controlled light variation
Tanks and Temples	Minimal	Mixed (Indoor/Outdoor)	Not systematically varied	Large-scale scene reconstruction
ETH3D	Minimal	Indoor/Outdoor	Limited variation	Multi-View Stereo, SLAM evaluation

summarizes benchmark datasets with varying degrees of photometric diversity. The table outlines whether the lighting conditions vary intentionally, the type of environment captured, and the primary applications supported by each dataset.

4.2. Comparative Evaluations Under Varying Light

Controlled experimental studies have shown consistent evidence that illumination variability significantly affects reconstruction accuracy across algorithmic categories.

For example, Yoon and Kweon (2006) [51] tested local and global stereo matching algorithms under simulated lighting changes. They found that global optimization methods with adaptive cost aggregation, such as graph cuts and belief propagation, showed better resilience compared to local approaches, though none were entirely immune to strong photometric shifts.

In Goesele et al. (2007) [52], the authors explored how uncalibrated MVS performs under lighting-induced noise. They observed that reconstructions degraded visibly in shadowed regions, with increased surface roughness and geometric holes in areas lacking texture or uniform illumination. Their study emphasized the need for illumination-aware confidence maps or visibility weighting.

Recent neural approaches have attempted to mitigate the lighting effects through learned compensation. NeRF in the Wild (NeRF-W) [33] introduced per-image embeddings that allow the model to adjust to appearance changes, improving reconstructions from unconstrained photo collections. Although it offered more visually coherent results than vanilla NeRF under variable lighting, some structural accuracy was sacrificed in the process. Similarly, 3D Gaussian Splatting [11] has been used in high-fidelity rendering scenarios but has not yet been systematically tested for lighting invariance, and its robustness under photometric variation remains an open question.

4.3. Summary of the Observed Impacts

A cross-study synthesis highlighted several consistent trends:

• Traditional methods (SfM, MVS) perform best under stable, diffuse lighting and are particularly vulnerable to shadows, reflections, and low-contrast regions.
• Dense stereo algorithms degrade under directional lighting due to ambiguity in correspondence matching, especially in textureless or reflective areas.
• Neural rendering techniques like NeRFs and its variants demonstrate improved visual coherence under photometric inconsistency, but this often comes at the expense of structural precision and requires large, well-sampled datasets.
• Benchmark limitations continue to constrain our ability to comprehensively evaluate lighting robustness, as most public datasets lack intentional photometric diversity.
• Few studies provide quantitative metrics for lighting robustness, and visual inspection remains a dominant (and subjective) evaluation method in many papers.

Overall, the existing body of empirical work confirms the substantial impact of lighting on 3D reconstruction quality. However, there remains a clear need for standardized lighting-aware benchmarks, large-scale comparative studies, and robust evaluation metrics to enable fair and reproducible assessments of algorithmic performance under real-world illumination conditions.

5. Discussion

The analysis presented in the previous sections confirms that lighting variation remains one of the most disruptive yet insufficiently addressed challenges in 3D reconstruction. While both classical and neural approaches have made notable strides in geometric fidelity and realism, their robustness under lighting inconsistency is highly variable and context-dependent. As shown in Figure 1, performance can degrade dramatically without explicit mechanisms for handling photometric variation. This section provides a critical evaluation of key limitations, practical trade-offs, and the current state of mitigation strategies, with particular attention to applicability in uncontrolled real-world scenarios. In this section, we critically discuss the core challenges, trade-offs, and methodological gaps in current research, with a focus on practical implications and future research trajectories.

5.1. Sensitivity of Classical vs. Neural Methods

Classical geometry-based pipelines such as Structure from Motion (SfM) and Multi-View Stereo (MVS) are inherently sensitive to lighting inconsistencies due to their reliance on photometric consistency and feature-based matching. While these methods can produce metrically accurate reconstructions under ideal conditions, their performance deteriorates sharply in the presence of shadows, exposure variations, and specularities. Preprocessing and feature engineering can help to some extent, but their effect is limited when faced with complex lighting interactions or mixed-illumination scenes.

In contrast, learning-based approaches, particularly neural rendering techniques like NeRFs and its variants, show greater resilience to minor photometric inconsistencies. By learning scene-dependent appearance models, these methods can infer structure and texture even when lighting varies across views. However, this flexibility comes with notable drawbacks. First, the radiance fields learned by models like NeRFs inherently encode lighting into the scene representation, which makes them fragile when generalizing to new lighting conditions. Second, these methods are computationally expensive, require dense image sampling, and often depend on scene-specific training. Third, they tend to prioritize view synthesis realism over metric accuracy, which may not be acceptable in applications like forensics, engineering, or metrology.

While recent neural methods like NeRF-W and Luminance-GS partially mitigate photometric inconsistencies through learned scene appearance or view-adaptive modeling, they are not universally applicable. As seen in Figure 1, even advanced models show degradation in extreme low-light or mixed-light conditions. This indicates a need for more robust and generalizable mitigation strategies, especially for unconstrained environments.

5.2. Lack of Lighting-Aware Benchmarks and Metrics

One of the most significant bottlenecks in advancing photometric robustness in 3D reconstruction is the lack of standardized benchmarks that include diverse and controlled lighting conditions. Most existing datasets are captured in ideal or stable environments, meaning that algorithmic performance under lighting variation is rarely tested or reported. This gap leads to overfitting the used methods to photometric consistency, reducing their applicability in real-world scenarios.

Furthermore, current evaluation metrics, such as point cloud accuracy, completeness, or re-projection error, do not capture lighting-specific degradations like shadow-induced sparsity, reflectance-related misalignments, or photometric inconsistency across views. There is a pressing need to develop lighting-aware evaluation metrics that can assess not only geometric precision but also the consistency and realism of surface appearance under different lighting conditions.

5.3. Practical Implications and Application Constraints

In practical applications such as crime scene documentation, cultural heritage digitization, or drone-based 3D mapping, reconstruction often takes place in uncontrolled lighting environments. The operator may not have the luxury of adjusting lighting, setting fixed exposures, or capturing redundant views. Under such constraints, robustness with respect to photometric conditions becomes essential, not optional.

However, few existing reconstruction pipelines are optimized for such scenarios. Field users are often left to rely on manual trial-and-error adjustments or empirical best practices (e.g., photographing at dawn or dusk to avoid harsh shadows), which adds operational complexity and uncertainty. Moreover, many neural techniques currently assume computational infrastructure and data volume not readily available in mobile or real-time settings.

These limitations underscore a broader issue: the disconnect between research-oriented reconstruction techniques and real-world deployment needs, particularly in environments where lighting control is infeasible.

5.4. Toward Lighting-Resilient Reconstruction Pipelines

Bridging the gap between theoretical robustness and real-world applicability requires combining algorithmic and hardware-level strategies. Several mitigation techniques have shown potential, but each comes with distinct limitations:

• Photometric calibration tools can reduce variability at acquisition time but are impractical in emergency or mobile contexts.
• Self-supervised learning frameworks like NeRD or NeRF-W help disentangle the lighting effects, yet require dense data and often struggle with generalization across scenes.
• Confidence-based fusion techniques, such as view selection or down-weighting based on photometric inconsistency, improve robustness but reduce the reconstruction density.
• Cross-modal integration (e.g., combining visual and depth signals) is promising but still largely unexplored in outdoor or field scenarios.

Moreover, interdisciplinary collaboration between computer vision, optics, and machine learning communities will be essential for developing principled models of illumination, capable of generalizing across environments and use cases.

Ultimately, no single mitigation technique provides universal robustness. The most effective systems, such as those shown in Figure 1, integrate multiple adaptations, often combining preprocessing, architectural design, and adaptive inference. However, these systems remain rare in real-world workflows, highlighting the need for future research to prioritize deployment readiness alongside accuracy.

6. Future Directions

The problem of light variation for 3D reconstruction is a multidimensional research problem. Arguably, the most critical necessity is the formulation of reconstruction models that can learn light-invariant representations. Ideally, these representations should factorize geometry, material, and light in a structured manner, making the representations stronger and enabling applications such as scene relighting or adaptive view synthesis. Recent work on neural rendering has made some early attempts toward this goal, but most of the current models still leverage supervised data such as light direction or reflectance ground truth. A key opportunity lies in exploring unsupervised and weakly supervised learning approaches, particularly those informed by physical models of light interaction and scene geometry.

Another important direction is the creation of benchmark datasets that explicitly incorporate photometric diversity. Most current benchmarks favor geometric or textural variation but do not take into consideration the complex dynamics of light. Datasets for the future would be improved if scenes were captured under a wide range of light conditions, both indoors, under control, and outdoors, under natural, dynamic conditions. High-quality synthetic datasets with realistic light and material interactions can also be utilized for training-improved generalization models, particularly if coupled with domain adaptation techniques.

Real-time reconstruction systems that work effectively in the real world, with its unpredictable and varying light conditions, would also become increasingly desired. Embedded photometric calibration features, such as ambient light property detection from sensors or camera metadata that adjusts for exposure and white balance, would be beneficial for such systems. Beyond calibration, dynamic frame selection strategies that prefer consistent input images could render reconstruction less sensitive, with little post-processing. Real-time adaptability will be important for forensics, public safety, and mobile robotics, where the user is not controlling the surrounding conditions.

Adding other sensing modalities can further reduce the reliance on ambient light. Mixing vision with infrared, LiDAR, or even polarization sensor data is especially promising. These modalities are less sensitive to the fluctuation of light and can be utilized to complement photometric data, especially for challenging surface properties like specularity or translucency.

Finally, domain-specific toolkits and procedures should be designed. In crime scene documentation, for instance, preservation of heritage, or remote drone-based inspections, controlling the light or re-capturing data is typically not feasible. Lightweight relighting models, optimized capture procedures, or portable light gear should be tailored for such environments and incorporated into everyday workflows. In the meantime, exchange with the community of computer vision researchers and the community of field workers will be critical for ensuring that innovations in the technical domain take into consideration the practical limitations of the real world. It will involve a blend of algorithmic design, data collection, and application-driven insight. By seeing light variation, rather than as a problem, as a fundamental consideration for 3D reconstruction, future systems will be constructed stronger, truer, and better prepared for the subtleties of real-world deployment.

7. Conclusions

The lighting conditions constitute a key but yet often underappreciated variable in the field of 3D reconstruction. Even with the significant advances made with classical geometric approaches and newer neural rendering approaches, this review has shown that photometric variation is still a serious issue for reconstruction quality and consistency. Across all technique classes, be it Structure from Motion, Multi-View Stereo, or even the newer ones like NeRFs and 3D Gaussian Splatting, lighting is the most important determinant of the fidelity of the reconstruction, particularly for uncontrolled environments.

Through the analysis of theoretical principles and empirical research, we identified shared problems such as sparsity caused by shadows, specular reflection artifacts, and general degradation in correspondence matching under non-uniform lighting. Learning-based approaches are more robust than their classical counterparts but sacrifice structural correctness or require additional training data and supervision. In addition, the lack of photometrically varied benchmarks and illumination-sensitive evaluation metrics has prevented the community from systematically comparing and pushing the state of the art.

This review also indicates promising directions for future research, including learning representations invariant to light, photometric dataset construction, multimodal sensor fusion, and domain-specific reconstruction toolkits. Ultimately, the ability to produce high-fidelity 3D reconstructions under disparate light conditions is not only a technological objective but also a step toward real-world applicability across forensics, archaeology, robotics, and beyond.

By advancing the boundaries of 3D reconstruction with consideration for lighting, the field can reach toward systems that appear not only realistic but also geometrically correct and deployable in the unpredictable environments where they will be of most utility.