Enhancing Autonomous Truck Navigation in Underground Mines: A Review of 3D Object Detection Systems, Challenges, and Future Trends

Abstract

Integrating autonomous haulage systems into underground mining has revolutionized safety and operational efficiency. However, deploying 3D detection systems for autonomous truck navigation in such an environment faces persistent challenges due to dust, occlusion, complex terrains, and low visibility. This affects their reliability and real-time processing. While existing reviews have discussed object detection techniques and sensor-based systems, providing valuable insights into their applications, only a few have addressed the unique underground challenges that affect 3D detection models. This review synthesizes the current advancements in 3D object detection models for underground autonomous truck navigation. It assesses deep learning algorithms, fusion techniques, multi-modal sensor suites, and limited datasets in an underground detection system. This study uses systematic database searches with selection criteria for relevance to underground perception. The findings of this work show that the mid-level fusion method for combining different sensor suites enhances robust detection. Though YOLO (You Only Look Once)-based detection models provide superior real-time performance, challenges persist in small object detection, computational trade-offs, and data scarcity. This paper concludes by identifying research gaps and proposing future directions for a more scalable and resilient underground perception system. The main novelty is its review of underground 3D detection systems in autonomous trucks.

1. Introduction

The evolution of autonomous driving systems in the mining sector has brought significant interest in enhancing computer vision for accurate and real-time 3D object detection. Unlike urban [1,2,3] or surface mining driving [4,5,6], underground mining environments present unique constraints such as limited visibility, dust, confined spaces, and uneven terrain that pose significant challenges for 3D object detection. These object detection systems are the perceptual backbone of autonomous truck haulage for obstacle recognition, object classification, and navigation in real time.

Recent advancements in computer vision/image processing powered by deep learning (DL) and artificial intelligence (AI) have significantly improved situational awareness in autonomous driving systems [7]. AL/ML-based object detection and tracking models have seen applications in robotics [8,9,10], urban autonomous driving [4,11,12], collision avoidance systems [13,14], and security systems for monitoring and surveillance [15]. AI-ML techniques, particularly those involving deep learning, play a significant role in these systems [7,16]. They have addressed the challenges of machine–human interactions, particularly for collision prevention, injuries, and fatalities. In the underground environment, research increasingly has implemented AI/ML architectures with LiDAR (Light Detection and Ranging), thermal infrared (IR), and RGB (Red, Green, Blue) cameras to detect pedestrians, machinery, and hazards [5,6,17,18,19,20]. Three-dimensional object detection models, which give richer spatial information than 2D models, have become a critical area of research for safe autonomous truck navigation [1,21,22,23]. The DL algorithms include Convolutional Neural Networks (CNNs) and YOLO (You Only Look Once), which have become efficient in identifying and classifying objects in real time [6,24,25].

These current models have demonstrated effectiveness in improving detection capabilities under diverse and harsh conditions [15,26,27]. However, they are limited due to small and occluded object detection inaccuracies amidst noise and obstructions. Also, they lack robustness when applied to specific environments such as underground mines, necessitating more innovative solutions [15,28].

As the mining industry transitions to Industry 4.0 and 5.0, which prioritize human–machine interactions, it is crucial to continually develop and innovate new systems, methods, and solutions for object detection and anti-collision systems in underground mining environments. Deploying autonomous vehicles into the underground mining industry is a paradigm shift toward operational efficiency and safety enhancement. These autonomous haulage trucks are designed to navigate dynamic and intricate mining environments, enhancing rapid decision-making accuracy and situational awareness. To ensure this, they heavily depend on sophisticated 3D object detection systems. However, the capabilities of these models are currently limited, which hinders the exploitation of their full potential in autonomous mining operations. Current detection systems frequently encounter limitations like trade-offs between speed and accuracy, especially under adverse conditions such as variable terrains, fluctuating lighting, and diverse objects [6,12,29]. Sensor modalities such as LiDAR, IR, and RGB cameras often result in suboptimal performance, particularly in dynamic obstacles and rapidly changing settings. Also, the real-time application capabilities of these detection systems remain insufficient, resulting in latency issues that could compromise operational efficiency and safety. By analyzing how the conditions mentioned above shape model and sensor choices, this survey provides a uniquely focused perspective on underground-specific object detection design requirements, an area often under-represented in existing literature.

Many review articles such as [2,16,18,30,31,32,33,34,35,36] have explored the advancements of object detection models in autonomous vehicles and mining environments. Immam et al. [18] reviewed anti-collision systems based on computer vision in underground mines. The study examined machine learning algorithms, which included CNNs, Fast R-CNN, and YOLO series, for real-time object detection and sensors employed in autonomous trucks to reduce accidents. Tang et al. [37] studied multi-sensor fusion detection methods for 3D object detection in urban autonomous driving. They developed a taxonomy that categorized fusion approaches and assessed their efficacy in improving detection accuracy and safe driving in autonomous vehicle driving scenarios. Cui et al. [31] reviewed navigation and positioning technologies in underground coal mines. The study investigated multiple techniques, like visual image feature-based systems, inertial navigation, visible light communication (VLC), and sensor fusion methods, to improve the accuracy and robustness of the detection systems.

Patrucco et al. [32] underscored the importance of multi-sensor systems by integrating different sensors to address the limitations of individual sensors. The study also investigated anti-collision technologies like cameras, LiDAR, and radar, discussing their principles, advantages, limitations, and costs. Wang et al. [33] also reviewed detection system advancements in unmanned driving technology for coal mine transportation systems. They discussed multi-sensor fusion strategies to address the limitations of single sensors. Nevertheless, there is a limited examination of underground settings’ complex and dynamic conditions and the scarcity of standardized datasets in detection models for benchmarking.

To contextualize the focus and novelty of this study, a comparative analysis was conducted against existing review papers and the literature.

Table 1. Comparative review of prior literature on perception and navigation systems in underground environments.

Review Paper	Focus Area	Limitations	Key Contributions in This Study
Imam et al. [18]	General object detection systems in underground mining	Focuses only on anti-collision frameworks	DL-based perception, systems, and underground-specific design limitations
Cui et al. [31]	Underground mine positioning systems and algorithms	Limited discussion on fusion architecture comparison and underground constraints	Provide comparative analysis across fusion levels with underground mining focus
Patrucco et al. [32]	Anti-collision technologies and safety protocols in tunneling and underground construction	No discussion on dataset limitations or challenges in confined spaces	Highlights dataset gaps and 3D perception limitations in underground scenarios
Shahmoradi et al. [9]	Applications of drones across surface and underground mining operations	Not focused on object detection challenges or underground autonomous truck navigation	Narrows down to AHS and object perception in underground environments
Zhang et al. [38]	UAV-based navigation and mapping in underground mining inspections	Only on UAVs for mapping and inspection	Addresses underground AHS detection models and dataset challenges
Xu et al. [39]	Sensing and navigation technologies for underground vehicle navigation	Does not delve into DL-based object detection	Discusses integration of DL-based 3D object detection and fusion strategies for underground autonomous trucks

presents a summary of key survey works related to object detection, underground navigation, and collision prevention across mining environments. Each study is evaluated based on its focus, identified gaps, and how this current review builds upon or diverges from those contributions. This detailed comparison highlights the distinct emphasis of this study on DL-based 3D object detection, multi-sensor fusion, and dataset challenges tailored for underground autonomous truck deployment.

The underground mining environment requires robust sensor fusion techniques, low-latency processing, and models resilient to noise and occlusion, which conventional surveys overlook. A standardized dataset with underground conditions for practical model evaluation and performance also exists. This review paper addresses the lacuna by synthesizing the most recent developments in 3D object detection, sensor fusion strategies, and dataset challenges within underground autonomous haulage systems. It highlights the strengths, limitations, and suitability of various 3D detection systems for autonomous truck navigation in an underground environment. The objectives of this review are as follows:

• To categorize and evaluate the various sensor modalities employed in underground autonomous haulage 3D detection systems.
• To explore multi-sensor fusion approaches, their performance, and trade-offs.
• To analyze and synthesize the deep learning architectures used in object detection models, particularly YOLO variants and CNNs.
• To identify key underground dataset limitations for object detection models.
• To identify significant challenges in underground autonomous truck object detection deployment and propose future directions for developing scalable and reliable detection systems.

This paper presents a comprehensive review to evaluate the current literature on 3D object detection systems for underground autonomous trucks. The process commenced with clearly defining the research questions, followed by a comprehensive search of pertinent studies in reputable academic databases, such as SpringerLink, Multidisciplinary Digital Publishing Institute (MDPI), ResearchGate, Institute of Electrical and Electronics (IEEE) Xplore, and Google Scholar. The search was conducted using keywords such as “3D object detection”, “autonomous trucks”, and “mining”, in conjunction with relevant terms such as “pedestrian detection”, “underground”, and “object detection”. There were defined criteria for inclusion and exclusion that guided the selection process. The studies considered for article inclusion were restricted to journals published in reputable academic sources, conference papers, and peer-reviewed articles. The only research included was recent, within the last 5–10 years, and specifically addressed 3D object detection in autonomous vehicles in underground mining environments. Data extracted from the selected studies were categorized and analyzed in key areas, including sensor modalities, 3D detection systems, multi-sensor fusion, fusion strategies, underground datasets, and specific mining challenges. In real-world scenarios, categorization enabled a comparative analysis of various approaches and their performance. The review followed established frameworks to guarantee consistency, transparency, data synthesis, and reporting comprehensiveness.

The review aims to enhance the reliability and robustness of autonomous systems in the underground mining industry by comprehensively analyzing 3D object detection methodologies in autonomous vehicles. The study employs a pragmatic approach, acknowledging that underground mining environments present distinct requirements that substantially differ from those in more controlled industrial environments. It is, therefore, essential to comprehensively evaluate and compare current 3D object detection techniques in such environments to ascertain their strengths, limitations, and application-specific challenges. This investigation will help discover current literature and technology limitations for developing more efficient detection systems that deliver real-time, precise, and resilient performance in underground mining environments. The study will emphasize recent advances and establish a basis for recommending future directions, particularly optimizing algorithms for rugged, resource-limited, and unpredictable mining environments. The study contributes to the advancement of autonomous haulage truck technology consistent with the mining industry’s safety protocols for zero fatality and operational requirements, thereby fostering a more efficient and secure future for the sector.

The structure of the paper is organized as follows: Section 2 provides a comprehensive review of the key sensor-based 3D detection systems for autonomous truck navigation, multi-sensor detection systems, and their strengths and limitations. Section 3 delves into multi-sensor fusion strategies, grouping them into early, mid, and decision-level fusion, and assesses their impact on detection accuracy. Section 4 discusses current deep learning architectures, their deployments, and state-of-the-art underground 3D detection model comparisons. Section 5 synthesizes underground dataset challenges related to autonomous object detection. Section 6 identifies the key challenges detection systems face in underground settings and proposes future directions. Finally, Section 7 synthesizes the findings of this review. Figure 1 shows the structure of the entire review report, starting with the background and motivation, followed by sections on 3D sensor techniques, object detection models, sensor fusion strategies, case studies, underground dataset challenges, and future research directions. It provides a clear roadmap for understanding the paper’s flow and focus.

2. Overview of Sensor Modalities for 3D Object Detection in Underground Autonomous Haulage Trucks

Three-dimensional (3D) object detection is pivotal to autonomous haulage truck operations in underground mining environments where safety, situational awareness, and navigation are critical. These detection systems rely on different sensor modalities to provide spatial awareness, track moving and static objects such as workers, equipment, and structural features, and detect hazards. Unlike controlled urban environments, underground settings have variable lighting, dust, occlusions, and uneven terrains, which necessitate the integration of complementary and robust sensors for detection.

This section provides a detailed analysis of key sensor-based detection perception systems used in underground autonomous trucks, which include IR, RGB cameras, and LiDAR systems. Each system is evaluated for its operational principles, integration with deep learning algorithms, underground applicability, and performance trade-offs.

2.1. Infrared (IR) Thermal Systems

Infrared (IR) thermal sensors (Figure 2) detect heat signals from objects and convert them into thermal images or temperature maps. IR’s ability to perceive heat from objects or obstacles makes it significantly valuable for underground mining environments, where smoke, low light, and dust often compromise traditional optical systems. Unlike RGB cameras that depend on ambient lighting, thermal sensors can function in complete darkness and thermally unstable conditions.

These sensor modalities are effective in the following:

• Pedestrian or worker detection by identifying humans’ heat signatures.
• Collision avoidance in situations where RGB cameras and LiDAR sensors struggle.

IR imagery can be segmented and classified in real time when integrated with YOLO-based or CNN architectures. Many studies have demonstrated the successful application of different YOLO and two-stage CNN algorithms on thermal imagery for object detection in underground mining environments [18,41,42,43,44,45]. Figure 3 and Figure 4 illustrate the application of IR with DL architectures in underground environments for pedestrian detection and navigation. Keza et al. [44] presented a pedestrian detection system that enhances safety in underground mines by integrating thermal imaging with 3D sensors. Using an FLIR thermal camera and depth sensors (TOF and Kinect), the system classifies regions using four methods and segments thermal images based on temperature thresholds. However, the model demonstrated susceptibility to motion distortion and mist and lacked enough underground datasets for model development. In a related effort by Szrek et al. [42], their AMICOS project utilized infrared thermography on an unmanned ground vehicle (UGV) to support search and rescue operations in underground mines. This system demonstrated the effectiveness of thermal imaging for detecting humans and navigating low-visibility environments. Its real-time temperature-based classification shows strong potential for adaptation in autonomous safety and perception modules, especially where visual occlusion and low lighting are major challenges.

Key Features and Advantages of IR Systems

• Effective in Low- or No-Light Conditions: IR systems rely on detecting heat signatures rather than visible light, making them highly effective in poorly lit or completely dark environments in underground mines.
• Thermal Object Detection: These sensors distinguish objects based on their heat signatures to identify equipment, vehicles, and workers, even in smoke, fog, or dust.
• Long-Range Detection Capabilities: Certain IR sensors, like long-wave infrared (LWIR), can detect objects over significant distances, enabling the early identification of hazards or obstacles.
• Insensitive to Ambient Light Variations: Unlike RGB cameras, IR systems are unaffected by ambient light changes, ensuring consistent performance in dynamic lighting conditions.
• Compact and Durable Designs: IR sensors are lightweight, built to endure the harsh mining environment, and can withstand hot temperatures, vibrations, and dust.
• Resistance to Ambient Light Variations: Unlike RGB cameras, IR sensors are not affected by changes in ambient light, offering consistent performance both day and night.

Despite these strengths of IR sensor modalities, there are notable limitations:

• Limited spatial resolution often affects fine-grained object classification;
• High purchase cost of IR sensors such as long-wave IR (LWIR) are often costly;
• Inference from reflective equipment or heat surfaces with similar thermal profiles degrades their performance.

Despite these limitations, their fusion into multi-sensory systems is invaluable, which enhances system robustness and reliability. When integrated with LiDAR (for depth information) and RGB (for texture), IR sensors contribute uniquely to real-time perception and classification.

2.2. RGB-Based 3D Detection Systems

The RGB cameras in Figure 5 remain one of the foundational sensor types utilized in 3D object detection systems because of their ability to capture high-resolution visual data with texture and color details [43,45,47,48]. They are deployed in underground mining autonomous trucks for object classification, scene understanding, and equipment recognition. The RGB camera sensors illustrated in Figure 5 capture 2D images, which comprise red, green, and blue color channels combined to form a full-color image and a per-pixel depth report. In 3D object detection systems, RCB cameras are often used alone or with active stereo or time-of-flight sensing technology [18]. They are integrated with other sensors, such as LiDAR or IR sensors, for multi-sensor systems to enhance the perception model’s scene understanding and robustness. Current detection algorithms, such as YOLO and CNNs, have significantly improved the accuracy and speed of RGB camera models [19,25,48,49]. These algorithms use features such as pattern recognition and color differentiation for precise detections and classifications.

RGB sensor detection systems are vital in autonomous mining trucks because they leverage visual data from cameras to identify, classify, and track objects in the environment. The RGB camera has various configurations as follows:

• The monocular cameras, which are lightweight and cost-effective, do not have depth perception.
• Stereo cameras estimate depth by triangulating differences between two lenses.
• Depth cameras incorporate active sensors for near-field 3D imaging.

These sensor types are very compatible with DL models like YOLO and CNN. Zhang et al. [50] proposed an LDSI-YOLOv8 framework to enhance missed detection and low recognition for multiple targets in underground coal mines, leveraging an RGB camera. The work reported an accuracy of 91.4%, demonstrating a 4.3% increase compared to the original YOLOv8 algorithm. Imam et al. [51] also developed an anti-collision system for underground mines, focusing on pedestrian detection via RGB cameras with a YOLOv5 DL-based algorithm to enhance pedestrian detection accuracy in low-visibility conditions.

Philion and Fidler [52] also presented the Lift–Splat–Shoot model, which encodes multi-camera images into bird’s-eye-view (BEV) representations for autonomous vehicle driving applications. The key objective was to create an end-to-end architecture that directly transforms multi-camera data into a unified BEV frame for semantic perception, understanding, and motion planning. The research demonstrated that the model could segment vehicles, drivable areas, and lanes by combining frustum-based depth inference and pooling techniques. However, there was reduced performance in low-light conditions and reliance on simple-frame data, which affects depth estimation accuracy compared to LiDAR-based models.

Versatility and affordability make camera-based models attractive for underground applications. They are less expensive than LiDAR and can be applied in areas such as identifying potential hazards and miner helmets for safety [17,53,54]. The key features and advantages of RGB cameras include the following:

• High-Resolution Imaging. RGB cameras can capture rich and detailed visual information, providing the necessary resolution to identify and classify objects based on color and texture.
• Cost-Effectiveness. They are relatively inexpensive compared to other sensors, such as LiDAR sensors, making them a known choice for cost-effective object detection models. Their affordability enables widespread applicability in autonomous vehicles.
• Color-Based Object Recognition. An additional layer of information is provided by the ability of RGB cameras to perceive colors, to distinguish between similar-shaped objects, and to identify warning signals.
• Lightweight Design. RBG cameras are compact and easy to integrate into autonomous vehicle platforms, enabling more sensor placement, system design, and integration flexibility.

However, there are significant challenges encountered as follows:

• High reliance on lighting makes them unsuitable for low-light environments.
• Affected by visual occlusions from debris and dust, which reduce detection accuracy.
• Lack of depth perception unless integrated with a stereo or LiDAR sensor.
• High computational load for processing high-resolution image frames in real time.

Multi-sensor fusion systems integrating RGB cameras with LiDAR or thermal sensor data address these limitations [24,27,28,54,55]. Fusion provides enhanced object classification and detection by combining color from RGB cameras with depth and range information from other sensors such as LiDAR. Autonomous systems achieve a more comprehensive and accurate perception of their environments by fusing these data from complementary sensor sources. Additionally, advancements in image technologies and algorithms have been introduced to address some of these challenges. These include high-dynamic range (HDR) imaging to improve the camera’s ability to handle extreme variations in brightness and enhance visibility in poor and variable lighting conditions [56,57,58]. ML algorithms are increasingly trained on augmented datasets to handle noisy and distorted visual data inputs better, improving model robustness in adverse conditions [25,28].

2.3. LiDAR (Light Detection and Ranging) System

The capability of LiDAR technology to generate precise depth measurements of the encompassing environment has made it a pivotal sensor type for detecting objects in autonomous vehicles. As shown in Figure 6 (solid-state LiDAR), these sensor types can accurately detect the distance between objects in a scanning area of up to about 50 m [32]. The technology of LiDAR sensors has made significant strides since their inception, making it a critical component in detecting 3D objects for autonomous vehicle navigation. LiDAR technology was initially developed for military and atmospheric applications [59]. It has since improved to accommodate the needs of commercial and industrial sectors, such as the underground mine. LiDAR is the primary solution when incorporated with deep learning algorithms in mining, where the transition to autonomous operations requires real-time and precise detection. Solid-state designs, multi-beam configurations, and scanning mechanisms have significantly enhanced detection range, resolution, and robustness. This has addressed the limitations of early systems in challenging environments that depend on only single-point laser systems. LiDAR sensors emit laser pulses that travel through the atmosphere, bounce off objects, and measure their return time to the sensor, generating high-density point clouds representing 3D information about the object. This information includes object size, position, and orientation. This ability allows autonomous trucks to detect nearby equipment, obstacles, workers, and infrastructure and navigate in GPS-denied underground tunnels.

Modern LiDAR systems employ solid-state designs or rotating mirrors to perform rapid scans across 360 degrees of targeted areas. Data from these scans are processed in real time to create dense point clouds, which serve as the foundation for object detection, tracking, and classification algorithms.

Key advantages of LiDAR systems in underground mining environments include the following:

• High-Precision Depth Measurement: LiDAR sensors’ time-of-flight range ensures the precise localization of objects: This makes it particularly useful in underground mining, where exact spatial awareness is critical for safety and overall operational efficiency.
• Resilience to Environmental Interference: LiDAR is highly resistant to environmental interferences, such as vibration and glare. Advanced multi-echo and solid-state have built-in glare filters, enabling reliable performance in some of the underground mining environments.
• Wide Field of View and Real-Time Mapping: Several LiDAR sensors offer a full 360° coverage view. This ensures comprehensive detection of objects, obstacles, and workers around mining vehicles, enhancing situational awareness.
• Efficient and Reliable Performance: LiDAR sensors are unaffected by an object’s color or texture, allowing consistent detection regardless of the visual appearance of objects. Unlike RGB cameras, LiDAR systems are effective in low-light conditions as they solely rely on active laser emissions rather than ambient light.
• Compatibility with SLAM systems: LiDAR systems support simultaneous localization and mapping (SLAM), enabling autonomous haulage trucks to track their positions while dynamically building maps of the environment.

LiDAR produces precise depth measurements in the harsh and complex underground mining environment, making it significant. Advanced processing algorithms applied to LiDAR data can classify, detect, and predict objects in the environment for autonomous truck navigation. Integrating these algorithms with LiDAR can distinguish between stationary walls, moving equipment, and other objects, which is crucial for dynamic decision-making in real-time detection operations.

LiDAR sensors are essential for recent 3D object detection systems in autonomous vehicles in underground mining environments. LiDAR point cloud data comprises millions of points (Figure 7) that map the objects’ surroundings in 3D (positions, size, and shape), providing a precise view beyond 2D images or videos, which is critical for accurately identifying objects in a scene. LiDAR-based systems’ resilience in harsh environments is a significant advantage for their application. Underground mines are characterized by conditions impairing the performance of other sensor-based systems, like cameras or radar sensors. However, LiDAR systems, such as moisture, are less affected by these conditions. Combined with DL algorithms, LiDAR data can improve object detection accuracy and provide a robust obstacle avoidance mechanism for safe autonomous truck navigation. Compared to RGB data, LiDAR 3D point clouds are critical in providing structural and spatial information of precise depth. However, the 3D point clouds are unordered, sparse, and sensitive to local variations, which makes raw LiDAR data processing challenging [60].

Generally, LiDAR object detection systems can be categorized into traditional and DL systems.

2.3.1. Traditional Methods

Traditional methods often depend on geometric and clustering techniques to detect objects. Algorithms such as DBSCAN [61,62,63] assume that objects have a higher point density than their surroundings. This method groups nearby points into clusters, then applies shape fit to detect objects based on geometric assumptions. Though this approach is practical in structured environments and computationally efficient, it often struggles in noisy, complex, and unstructured environments. Usually, it fails to detect dynamic elements such as pedestrians or moving vehicles. They are more rule-based and best suited for controlled and simpler environments.

2.3.2. Deep Learning-Based Methods

Recent technological advancements have enabled more accurate detection of complex real-world environments. These models automatically learn features from large, annotated datasets and are robust to occlusions and noisy inputs, providing accurate and more flexible detection in dynamic environments. LiDAR-based DL systems are generally classified into view-based, voxel-based, point-based, and hybrid point-voxel-based methods [21,64,65,66].

• Voxel-Based Methods: Voxel-based models transform raw and irregular 3D point clouds into structured grids known as voxels for processing using 3D convolutional neural [65,66,67,68]. This generates simplified data, allowing the use of mature CNN architectures for object detection. Popular techniques include the following:

  ○

  VoxelNet [67]: This end-to-end framework partitions LiDAR point clouds into voxel grids and encodes object features using stacked voxel feature encoding (VFE) layers. This process is followed by 3D convolution to extract geometric and spatial features. The method streamlines the detection pipeline by eliminating separate feature extraction stages. Figure 8 shows the application of voxelization for autonomous truck navigation and detection, aiming to improve safety and operational efficiency

  ○

  SECOND (Sparsely Embedded Convolutions Detection) [66]: This is an optimized version of VoxelNet that employs sparse 3D convolutions, which improves computational efficiency without sacrificing accuracy [66].

These models simplify point clouds’ irregular and sparse nature and have high localization accuracy due to precise spatial encoding in the voxel grid. Additionally, their ability to process volumetric data directly makes them suitable for underground mine environments. However, the limitations of voxel-based methods include a lack of semantic richness, which limits their classification accuracy when implemented alone. Fin-grained voxelization can lead to high computational demand. Lastly, they struggle with small object detection due to limited resolution in sparse voxel grids.

2.

Point-Based Methods. Point-based methods directly process raw LiDAR point clouds without converting them into voxel grids (voxelization) to 2D projections. This method preserves fine-grained geometric details of the environment by operating on ordered 3D points. This makes them highly effective in detecting partially occluded or irregularly shaped objects. The key pioneering family of point-based methods is PointNet [69]. This uses symmetric functions to learn object global shape features from unordered point sets. PointNett++ [70] extends this by using hierarchical feature learning to capture local context in clustered regions. PointPillars [71] shown in Figure 9 are a widely adopted method in autonomous driving. The method partitions point clouds into vertical columns known as “pillars” to offer a balance between detail preservation and computational efficiency. It converts spatial features into a pseudo-image format, which enables fast detection through 2D convolution while retaining meaningful 3D structures.

Despite their advantages in handling detailed geometry, point-based methods face limitations in computational cost, especially in cluttered environments, limited scalability in real time, and the need for substantial GPU memory for training and inference.

3.

Hybrid-Based Methods: Hybrid approaches integrate the strengths of voxel-based and point-based techniques to enhance the accuracy and efficiency of 3D object detection. These methods utilize voxelization for structured data representation while leveraging fine local feature extraction of point-based models. A hybrid model typically starts by extracting local geometric object features from raw point clouds using point-based encoders. The features are then embedded into voxel grids, where the voxel-based backbone utilizes convolutions to learn global context and make predictions. This design helps the hybrid model to balance robustness, precision, and processing speed, making it suitable for cluttered and complex underground environments.

Despite the fast adaptation of LiDAR-based systems, several limitations impact their deployment in underground mining environments:

• Lack of Object Identification: LiDAR systems can detect the presence and position of objects, but do not provide detailed information about object types or characteristics, which is essential for some mining applications.
• Computational Demand: This includes the processing of dense and sparse point clouds.
• Environmental Sensitivity: Accuracy is impacted by environmental conditions, such as dust, fog, water vapor, and snow. These factors degrade signal quality and require additional hardware and software processing to mitigate detection failures.
• Sensor Surface Contamination Challenges: Dust and debris accumulating on the sensor surface impair detection functionality, necessitating regular cleaning or protective mechanisms to maintain reliability.
• High Costs and High Resolution: These LiDAR units are expensive, which often hinders their large-scale deployment in budget-sensitive underground operations
• Range and Field of View Limitations: The typical range of LiDAR sensors is limited to around 50 m [28], restricting their effectiveness in larger underground mining operations. Furthermore, planar scanning systems may miss obstacles above or below the scanning plane, posing safety risks.
• Energy Consumption and Infrastructure Requirements: LiDAR systems consume more energy than other sensors and demand robust infrastructure for effective operation, complicating deployment in confined underground mining spaces.

While models such as 3DSG [5,68] have demonstrated enhanced 3D object detection for surface mining trucks using LiDAR sensors, their direct applicability to underground environments remains limited due to differences in spatial constraints, lighting conditions, and operational challenges.

2.4. Multi-Sensor Fusion in Underground Mining: Perception Enhancement Through Integration

Multi-sensor fusion models [72,73,74] are at the forefront of recent 3D detection systems for autonomous vehicle navigation in underground mining environments. These systems integrate data from multiple sensors such as cameras, LiDAR, radar, or thermal to develop a comprehensive perception of the environment. Each sensor type has its strengths and limitations, and by combining them, multi-sensory fusion systems capitalize on their complementary capabilities to enhance detection accuracy and reliability for the safe navigation of autonomous trucks. LiDAR provides precise 3D spatial information, excelling in in-depth measurement, but often struggles in particulate-heavy environments. Cameras offer high visual data, capturing color and texture information for more detailed feature extraction, but their performance degrades in low-light or obscured conditions. Radar is reliable in adverse conditions and excels at long-range detection, but is less precise in resolution. Thermal sensors capture heat signatures from objects and surroundings, making them capable of detecting objects in completely dark and challenging conditions, such as fog, smoke, and dust. By fusing these data streams, multi-sensory models address the individual limitations of each sensor to create a robust perception framework suitable for challenging conditions in underground mines. As sensor technologies and fusion algorithms continue to evolve, they promise a future where autonomous trucks achieve even greater safety, efficiency, and adaptability in mining operations.

Szrek et al. [42] evaluated a UGV-based human detection system in underground mines using RGB and IR cameras alongside YOLOv3 and HOG algorithms. While RGB imagery provided visual context, it struggled in low-light and cluttered environments, especially for non-standing individuals. The study showed that RGB detection alone was often insufficient, but combining it with IR data improved reliability. A key limitation was using pre-trained models that were not optimized for underground settings, which affected accuracy. Xu et al. [75] proposed an autonomous vehicle localization method for underground coal mine tunnels based on fusing vision and ultrasonic sensors. They use infrared cameras to detect wall-mounted barcodes and ultrasonic sensors to measure distances, enabling geometric calculation of vehicle position without relying on complex SLAM. The method achieves sub-meter accuracy but is limited by dependence on manual barcode deployment and potential occlusion in dynamic tunnel environments.

Zhang et al. [76] also developed a real-time underground mining vehicle localization method by fusing YOLOv5-based object detection with high-precision laser distance measurement. The system identifies mining trucks visually using YOLOv5s and calculates exact positioning via laser sensors. However, the system had limitations, including its sensitivity to environmental conditions such as humidity and dust, which reduced detection robustness for small and fast-moving objects, with future improvements suggested through upgrading to YOLOv7/YOLOv8 and enhancing multi-object tracking capability. While recent advances have explored multi-sensor fusion using RGB, LiDAR, and thermal imagery with CNN or YOLO-based algorithms in environments such as search-and-rescue tunnels, urban settings, and surface mine environments, applications specifically targeting underground autonomous haulage trucks remain extremely limited. Most existing work focuses on drones, indicating a critical research gap for truck-scale 3D object detection and navigation in confined mining conditions.

Key advantages of multi-sensor fusion include the following:

• Enhanced Detection Accuracy: Combining the complementary strengths of different sensors, like LiDAR’s depth information with RGB camera’s texture data and thermal imaging’s heat signatures, will improve overall object detection performance.
• Robustness in Harsh Conditions: Multi-sensor models maintain improved detection capabilities in challenging underground conditions where individual sensors may fail.
• Redundancy for Safety: These models provide multiple sources of information, allowing the system to continue functioning safely if one sensor fails or becomes unreliable.
• Improved Localization and Mapping: Fusing vision (RGB cameras) with depth measurements (LiDAR) strengthens localization precision and supports robust mapping in GPS-denied underground environments.
• Adaptability to Dynamic Conditions: Adaptability provides dynamic sensor prioritization, where the system can rely more heavily on the most reliable sensors depending on environmental changes, like thermal sensors in low-visibility conditions and LiDAR for depth information.

With these capabilities of multi-sensor systems, notable challenges associated with them include the following:

• High Computational Demand: Processing large volumes of synchronized data from different sensors in real time requires powerful and often costly computational hardware.
• Complex Synchronization and Calibration: Multi-sensor fusion requires precise calibration and alignment across different sensors, which is technically challenging, especially in underground settings with vibrations and environmental noise.
• Scarcity of Datasets for Training: There is a lack of large, annotated, multi-sensor datasets that specifically capture underground environment conditions, limiting the ability to train robust deep learning models
• Increased System Weight: Adding multiple high-end sensors such as LiDAR, thermal, and RGB cameras raises the overall cost and maintenance complexity and may impact autonomous truck payload or energy efficiency.

3. Sensor Data Fusion Methods

Fusing data from multi-modal sensors is vital in 3D object detection for underground autonomous trucks. The harsh underground conditions require robust perception systems capable of overcoming occlusions, dust, and low-visibility environments. Fusion methods are generally classified into early-stage (raw data), mid-stage (feature-level), and late-stage (decision-level) fusions, depending on when the fusion occurs in the data preprocessing pipeline, as illustrated in Figure 10 and Figure 11. Figure 10 illustrates the key sensor fusion types for various fusion strategies aimed at enhancing detection model accuracy and improving robustness, with their different mode of fusion strategies. Figure 11 demonstrates key architectural differences between early, mid-level, and late fusion strategies. These diagrams show where sensor data, such as LiDAR, cameras, and radar, is combined within the object detection pipeline. These stages help clarify how different models balance accuracy and flexibility in perception systems. Figure 12 illustrates a comparative view of the relative accuracy and computational complexity of the three fusion strategies. It places each method within a performance-efficiency spectrum, visualizing how the choice of fusion impacts both model quality and hardware demands. Each strategy has distinct strengths and significant trade-offs, as described below. For instance, though early fusion offers high accuracy, it needs powerful processors and precise calibration. Mid-level fusion strikes a balance between accuracy and efficiency, making it well-suited for underground haulage trucks operating in real time. Late fusion trades some accuracy for simplicity and speed, which may be acceptable for non-critical tasks or as a fallback system for applications.

3.1. Multi-Fusion-Level Methods

3.1.1. Early-Stage Fusion

Early-stage fusion, or sensor-level fusion, integrates raw, unprocessed data streams from multiple sensors into a single dataset before performing feature extraction [78]. This method is valuable for creating robust detection systems in environments requiring fine-grained detection details, as it captures the full signal fidelity of individual sensor modalities.

For instance, LiDAR point clouds can be registered onto the RGB camera image pixel grid to generate depth-colored visual maps. Thermal sensor outputs can also be overlaid on visual data to identify heat-based anomalies. This strategy significantly detects small, obscured objects that a single sensor type may not fully recognize. This approach maximizes the amount of extracted information, which enables detailed feature extraction. The richness of the raw data captured ensures that critical details are not lost during the fusion process.

The primary advantage of early-stage fusion is its ability to retain rich and detailed information across multiple sensor modalities. This gives an expressive input for DL models, enhancing detection precision and accuracy. However, this imposes significant computational demands, as raw sensor data requires huge memory and processing power. This computational burden causes a bottleneck to systems requiring real-time processing, such as the underground mine.

Moreover, data temporal misalignment from different sensors, resolution disparities, and distinct sensor frame rates introduce another challenge. This can be addressed by applying advanced calibration techniques and synchronization protocols for adequate alignment [2,37,79,80]. Misaligned data can lead to inaccuracies in the final fused dataset, undermining the system’s reliability, accuracy, and effectiveness [2]. Despite the earlier limitations, early fusion is essential in environments requiring precise spatial understanding, and computational overhead is not the primary constraint.

3.1.2. Mid-Level Fusion

Mid-level fusion involves the integration of independent extraction features from individual sensor modalities to create a more unified and comprehensive representation of the environment. Rather than performing a fusion of raw sensor data, each sensor undergoes preliminary processing via a Convolutional Neural Network (CNN) or other feature encoders to extract high-level object representations such as contours, depth features, temperature gradients, and motion cues. These extracted features are consequently aligned and integrated into a shared feature space.

This fusion strategy provides an effective balance between perception richness and computational efficiency. By processing feature representations with low dimensionality, memory and bandwidth are reduced instead of the significant volume of raw data, while capturing rich information from each sensor modality. PointPainting by [81] presents a sequential (mid-level) sensor fusion approach for 3D object detection that combines LiDAR point clouds with semantic information from RGB images. The core idea is to first run a 2D semantic segmentation network on RGB images to produce class probabilities (or “paint”) for each pixel. These semantic labels are then projected onto the corresponding LiDAR points, effectively “painting” each point with class information. The enriched point cloud is then fed into a 3D detection network, such as PointRCNN or SECOND. This mid-level fusion strategy enables the model to leverage both geometric precision from LiDAR and semantic context from images. The studies demonstrated improved detection, especially for small or partially occluded objects, while maintaining modularity and flexibility in network design.

Mid-level fusion provides several advantages, including data efficiency. This reduction in computational requirements offers a more efficient system for real-time operations. Mid-level fusion is particularly relevant for real-time applications in underground autonomous trucks where edge deployment and split-second decision-making are critical.

Nevertheless, mid-level fusion introduces its challenges. There is difficulty in effectively aligning extracted features from heterogeneous sensors, which require spatial and temporal synchronization and are often trivial. Misalignments can degrade fusion quality and present detection errors. Additionally, fusion performance solely depends on the design and quality of the feature extraction modules as well as fusion architecture, such as cross-modal attention networks, which require careful design and tuning.

3.1.3. Late-Stage Fusion

Late-level (decision-level) fusion integrates the final outputs of independently processed sensor data, such as bounding boxes, to make high-level decisions. Each sensor operates autonomously in this approach, and the outputs are generated using techniques like weighted averages or voting schemes to obtain final decisions. In [82], the study employed a decision fusion method utilizing camera and LiDAR sensors for mine track object detection. This strategy is favored for its simplicity and modularity, enabling sensors to be trained and maintained separately, simplifying the model design. The decision fusion approach is computationally lightweight, making it suitable for embedded systems or as a fallback in redundant safety layers [79]. As individual sensors process their data autonomously, the fusion process occurs at the decision level, which does not require handling extensive raw data. This makes late-stage fusion suitable for resource-constrained environments and real-time processing applications.

Despite these advantages, there are limitations, such as over-reliance on high-level decisions. Additionally, it lacks access to raw or intermediate data. It cannot refine ambiguities that are introduced earlier in the training pipeline [83]. If one sensor misclassifies an object, the fusion process cannot correct it. This challenge often reduces overall detection accuracy and system performance, particularly in complex scenarios of late fusion models, especially in occluded and multiple overlapping scenes [84,85,86].

Primarily developed for autonomous surface vehicles, the hybrid fusion framework proposed by [84] demonstrates a scalable strategy that combines mid-level and late-level fusion for robust perception. It integrated LiDAR, radar, and camera inputs while addressing sensor misalignment through asynchronous fusion. SCANet [85] also introduced a spatial-channel attention network that fuses LiDAR and RGB data at multiple stages to enhance 3D object detection. Although designed for surface vehicles, its use of attention mechanisms offers an alternative to traditional late-stage fusion by selectively emphasizing informative features across modalities. This enhancement improves detection in unstructured environments, making it a promising candidate for adaptation in underground autonomous vehicles, where scene complexity and sensor noise are significant challenges. Roy et al. [86] proposed a concurrent spatial and channel “squeeze and excitation” (scSE) mechanism that enhances feature refinement by jointly recalibrating spatial and channel-wise responses. Its ability to improve semantic clarity makes it valuable for strengthening the interpretability and reliability of decision-level sensor fusion frameworks.

These innovations are especially relevant for the adaptation of underground haulage systems, where real-time performance and environmental noise pose significant challenges. While such frameworks are yet to be widely deployed in underground mining, they offer a blueprint for developing more resilient perception systems in unstructured, low-visibility environments.

Table 2. Sensor fusion strategies in 3D object detection.

Fusion Approach	Level	Advantages	Limitations
Early-Level Fusion	Raw data	- Rich joint data representation [87] - Fine-grained for small and obscured object detection - Detailed spatial modeling - High-resolution outputs	- Complex calibration - Sensitive to noise, distortions, and misalignments - High memory and computation - Demands precise spatial and temporal data alignment
Mid-Level Fusion	Feature Level	- Balances accuracy and efficiency - Lower data volume [83] - Adaptable to real-time models - Works well with diverse sensor modalities	- Requires accurate feature alignment and resolution compatibility - Loss of raw data details - Complex architecture tuning
Late-Stage Fusion	Decision Level	- No raw data alignment needed - Simple to implement - Computationally efficient - Effective in redundancy layers - Suitable for real-time detection applications	- Cannot resolve early-stage mistakes [83] - Heavy reliance on accurate individual sensor performance - Not suitable for detailed integration of raw data detection accuracy in complex scene applications

presents the three sensor fusion strategies commonly applied in multi-sensor systems. It outlines the advantages of each fusion level and discusses its key features. This summary serves as a guide for researchers in selecting a suitable fusion framework that balances accuracy and real-time performance in an underground mining environment.

This section and the comparison below demonstrate how each fusion strategy proves its unique strengths and limitations based on the deployment context. In complex underground mining environments, where safety-critical decisions are required in real time, mid-level fusion often provides the best balance between detection accuracy and real-time decision-making. However, hybrid strategies that combine early and late fusion may also provide good results for fail-safe designs. Table 2 above gives a comparative summary of the advantages and limitations of early-fusion, mid-fusion, and late sensor fusion approaches used in 3D object detection. While these strategies have gained wide applications in autonomous driving and surface mining systems [72,88,89,90], their deployment in underground autonomous truck navigation remains significantly underexplored. To the best of our knowledge, there is a lack of existing studies that have systematically implemented or benchmarked these fusion levels within the complex, sensor-degraded environments typical of underground mining. This lack of targeted evaluation highlights a critical gap. It underscores the necessity for underground-specific sensor fusion designs that can overcome dust, occlusion, denial of GNSS and GPS signals, and dynamic underground layouts.

3.2. Sensor Fusion Advantages and Limitations

3.2.1. Advantages of Sensor Fusion in the Underground Mine Environment

Sensor fusion significantly enhances perception performance in autonomous haulage systems, especially within complex and dynamic underground mine environments. By integrating different sensor modalities such as LiDAR, camera, and IR, sensor fusion improves decision-making accuracy by overcoming individual sensor limitations. The key advantages include the following:

• Resilience in Harsh Conditions: Sensor fusion compensates for individual sensor weaknesses. For example, LiDAR often degrades in dense particulate environments, cameras struggle in low light, and thermal sensors may lose range in open spaces. The fusion of these sensors provides a more consistent and robust perception across such extreme conditions [87].
• Enhanced Object Detection and Classification: Multi-modal sensor integration improves object detection precision. Combining LiDAR and thermal or camera data enhances the recognition of equipment, workers, and structural features, especially in low-visibility or occluded scenes [81].
• Improved Operational Safety: Integrating multiple sensors increases system reliability and robustness, providing better object identification and more reliable obstacle avoidance. This ensures workers’ and equipment safety in confined underground settings [72].
• Real-Time Decision-Making: Fusion systems enable faster and more informed decision-making, allowing real-time analysis of extreme environmental conditions. This enables autonomous trucks to respond promptly to dynamic changes in the environment [90].
• Model reliability and Redundancy: Fusion systems enhance fault tolerance and robustness by ensuring that the failure of one sensor modality does not compromise the entire perception system.

3.2.2. Challenges in Sensor Data Fusion

Despite the transformative benefits of fusion modalities, sensor fusion in underground autonomous haulage systems faces several technical and environmental challenges that complicate its implementation. These challenges must be addressed to ensure accurate, scalable, reliable, and real-time detection systems.

• Sensor Signal Reliability and Noise: Underground environmental conditions, such as noise, vibrations, dust, and electromagnetic interference, can significantly introduce noise into sensor outputs, degrading performance. LiDAR sensors may produce inaccurate point clouds due to reflective surfaces, while RGB cameras may struggle in low-light conditions. Filtering this noise without losing essential features requires advanced adaptive filtering and denoising techniques [91].
• Heterogeneous Sensor Integration: Each sensor type, such as LiDAR, cameras, radar, and IR, produces data in distinct resolutions, formats, and operational ranges. Integrating this heterogeneous input data requires overcoming challenges associated with data pre-processing, feature extraction, and data representation. For instance, fusing LiDAR point cloud data with pixel-based data from cameras involves significant computational effort and advanced algorithms to ensure meaningful integration and representation [2].
• Data Synchronization and Latency Management: Precise synchronization of data streams from various sensors is crucial for real-time sensor fusion performance. Differences in signal delays, sampling rates, and processing times can lead to temporal data mismatches, resulting in inaccurate fusion outputs. For example, data from a camera capturing images at 30 frames per second (fps) must be aligned with LiDAR data that operates at a different frequency (e.g., 15 Hz). Sophisticated temporal alignment algorithms or interpolation methods are required to ensure all sensor inputs contribute meaningful data to the fused output [2,37,91].
• Computational Complexity. The large volume of high-dimensional data that multiple sensors generate presents significant computational challenges. Real-time processing, critical for applications such as autonomous truck navigation, demands high-performance hardware and optimized algorithms. Resource constraints, including limited processing power and energy availability in mining vehicles, further complicate this task. Efficient techniques that balance computational load with fusion accuracy are crucial for deployment in such environments. Refs. [83,91] discuss the computational challenges in sensor fusion, emphasizing the need for efficient distributed processing strategies and the use of techniques like data compression and dimensionality reduction to manage computational and resource requirements.
• Environmental Factors: Underground conditions pose severe challenges to sensor performance. Poor lighting, dust, smoke, noise, and fog can obscure camera, radar, and LiDAR data, while extreme temperatures can impact sensor calibration and accuracy. Additionally, confined spaces and irregular terrains can cause occlusions or reflections that lead to distorted sensor readings. Designing fusion algorithms that can compensate for these environmental factors is a critical area of research. Addressing these challenges requires interdisciplinary solutions drawn from signal processing and machine learning.

4. Algorithms for 3D Object Detection in Autonomous Trucks

The rapid evolution of ML algorithms has revolutionized numerous sectors, including autonomous vehicles and industrial robotics. It enables these systems to learn from experience, process complex sensory data, adapt, and make real-time decisions. Over the past decade, ML techniques have transitioned from basic pattern recognition to sophisticated algorithms that address complex, real-world challenges. AI/ML technologies provide automation, safety, and productivity advancements in the mining industry, particularly underground operations.

Historically, traditional object detection techniques such as Histogram of Oriented Gradients (HOGs) and Support Vector Machines (SVMs) struggled to perform under harsh underground conditions because of occluded or noisy environments. Efforts to develop more robust and adaptable DL algorithms, particularly Convolutional Neural Networks (CNNs) and YOLO, have increased to address many of these challenges [92]. These algorithms allow for more precise, reliable, and adaptive perception in complex underground environments.

This section synthesizes major ML algorithms and their integration for 3D object detection in underground autonomous haulage navigation. The section emphasizes their architectural principles, recent applications in underground operations, and performance trade-offs.

4.1. Convolutional Neural Networks (CNNs)

Convolutional Neural Networks (CNNs) are the foundational technologies to modern computer vision because they can learn spatial hierarchies of image data features [93]. CNNs have proven resilient in underground environments due to poor lighting, dust, and occluded conditions.

CNN-based object detection models follow two-stage processes [94]. Girshick et al. [95] introduced the original R-CNN (region-based), which utilizes a selective search to propose regions of interest, followed by CNN-based classification. Though this method is accurate, it is computationally expensive and unsuitable for real-time applications. Subsequent models such as Fast R-CNN [96] and Faster R-CNN [97] directly used the region proposal stage in the CNN, minimizing latency and enabling near real-time applications. Mask R-CNN [98] further enhanced this by adding instance segmentation alongside detecting and classifying objects valuable for cluttered environments like underground mines.

Several studies have shown the use of CNNs for vehicle and personnel detection in underground tunnels, which validates the robustness of hierarchical feature extraction [94,97]. However, CNN algorithms remain challenged by their high computational load, local receptive fields that may miss the global context, and limitations in the 3D point cloud [23,99]. Lightweight CNN variants that employ quantization, pruning, and multispectral imaging have been explored to overcome these challenges. These techniques reduce model size and energy consumption capabilities, making CNNs suitable for edge deployment in underground autonomous haulage. However, challenges remain in meeting real-time performance and adaptability to complex underground scenarios in underground autonomous haulage.

4.2. YOLO (You Only Look Once) Series Algorithm

YOLO (You Only Look Once) algorithms have gained prominence and recognition for their single-stage framework approach. This feature enables simultaneous object localization and classification in one network pass, delivering real-time detection inference critical for dynamic underground environments [13,100,101,102]. Redmon et al. [103] pioneered the introduction of YOLOv1, redefining the detection task as a regression problem and significantly enhancing inference speed. Later versions, such as YOLOv3, YOLOv4, and YOLOv5, presented features including multi-scale detection (Darknet-53), cross-stage partial networks (CSPDarknet), and enhanced model flexibility [104]. YOLOv8, the latest version, includes lightweight designs, transformer-based enhancements, and an improved feature pyramid for better accuracy–speed balance. Figure 13 shows the traditional architecture of the YOLO framework. It processes a 448 × 448 × 3 input image through a DarkNet-based backbone to produce a 7 × 7 × 1024 feature map. This is flattened and passed through fully connected layers, resulting in a 7 × 7 × 30 output tensor. Each of the 7 × 7 grid cells predicts two bounding boxes and class probabilities, which enable real-time object detection. This architecture laid the foundation for later YOLO versions by combining localization and classification in a single pass.

Research has leveraged the YOLO algorithms to develop robust 3D object detection models [21]. Zhang et al. [19] proposed LDSI YOLOv8 to address issues of multi-target detection in the coal mine excavation environment. The work leveraged the YOLOv8 architecture and achieved an mAP improvement of 4.3% compared to the original YOLOv8 model as shown in Figure 14. The study achieved 91.4% mAP and 82.2 FPS, demonstrating strong adaptability to underground dusty, low-light, and occluded underground conditions. Despite its potential detection performance, the model’s training on a specialized excavation dataset may limit its generalization to different mining sites without retraining or domain adaptation. Ní et al. [100] also developed a YOLOv8-based pedestrian and hazard detection model for underground mining environments, which achieves real-time capability and improved accuracy. Zhang et al. [105] proposed YOLO-UCM, an enhanced YOLOv5 model, to improve pedestrian detection in underground coal mines. It integrated Vision Transformers (ViTs) and Meta-AconC for enhanced feature extraction and detection accuracy.

DDEB-YOLOv5s model [106], which incorporates a C3-Dense feature extraction module to enhance the perception of occluded and small-scale objects in a coal mining environment, builds upon the Cross-Stage Partita (CSP) architecture by introducing dense connections that enable multiple convolutional layers to share and reuse feature maps. It also introduced a weighted BiFPN (Bidirectional Feature Pyramid Network), which is used to merge image features from different layers more effectively, to improve accuracy for objects of various sizes. Finally, a decoupled detection head was leveraged to separate the tasks of classifying objects and locating them within the image, resulting in more precise results for both object type and position. It achieved multi-target tracking accuracy of 91.7% and stability in mining environments.

Li et al. [107] presented an improved YOLOv11-based miner detection model for underground coal mines, enhancing feature extraction with Efficient Channel Attention (ECA) and refining localization with a weighted CIoU loss. The model achieves 95.8% at mAP@50 and 59.6 FPS on a custom underground dataset, outperforming existing detectors. However, it mainly focuses on personnel detection, with future work needed for broader underground object recognition.

While YOLO models have achieved considerable success, they are challenged by difficulty detecting objects in highly occluded or cluttered conditions. This often requires additional computational resources as the scale increases. The trade-off between detection speed, accuracy, and model size remains a key challenge when deployed on embedded GPU systems in underground haulage trucks. Therefore, ongoing research aims to optimize YOLO’s performance in such settings, ensuring that real-time, accurate detection remains feasible even with limited computational power. A performance comparison of YOLO and CNN algorithms is presented in

Table 3. Comparison of YOLO and CNN DL models for object detection in underground mining environments.

DL-Approach	Advantages	Limitation
CNN	- Extracts hierarchical features effectively from image data - Effective in noisy and low-light underground conditions - Enhanced detection accuracy with multispectral data - Effective for segmentation and complex object shapes	- Computationally expensive, limiting real-time deployment - Need large, labeled datasets and have long training hours - Difficulty in deployment of embedded edge computers - Require large, labeled datasets for training - Prone to overfitting in complex and dynamic mine layouts
YOLO Series	- Single-stage architecture enables fast inference speed for real-time autonomous navigation - Well-structured tracing of overlapping and moving objects - Lightweight versions are suitable for deployment on GPU edge platforms - Easy to fine-tune across variable mining datasets - Latest YOLOv8 variants support transformer-based attention	- Trade-offs in accuracy for small or occluded objects. - Reduced robustness without data augmentation or custom tuning practices - Improved performance requires extensive hyperparameter tuning - Limited generalization to unseen mining data - Persistent challenge by sensor misalignment and fusion latency

. The table highlights the key advantages and limitations of CNN-based and YOLO-based object detection frameworks in terms of application constraints in underground settings, characterized by low lighting, occlusion, computational overhead, and the need for real-time processing inference.

In summary, two-stage detection frameworks such as CNNs provide high detection accuracy but introduce computational overhead, making them unsuitable for real-time underground applications. On the other hand, YOLO-based models provide a viable direction for fast and efficient object detection, with recent versions demonstrating adaptability to underground conditions. However, they often suffer in small object detection and tradeoffs between accuracy and inference speed. Continued efforts to enhance these models for edge computing and improve their robustness and real-time performance under occlusions and dynamic environments will be crucial to advancing autonomous haulage safety and efficiency in underground mining operations.

4.3. Detection Model Comparison for Underground Autonomous Haulage Systems

Current 3D object detection models designed for underground autonomous haulage systems navigation differ in structure, sensor dependency, and performance. The performance of 3D object detection systems is crucial for ensuring the safety and efficiency of autonomous trucks, particularly in complex and dynamic environments such as underground mines.

Table 4. Comparison of recent 3D detection models for underground applications, where x means no information.

Model/Framework	Detection Algorithm	Sensor Modalities	mAP (%)	FPS	Limitations
LDSI-YOLOv8 [19]	YOLOv8n	RGB Camera	91.4	82.2	It has limited scalability in other mining environments.
YOLOv8 for Hazard Detection [100]	YOLOv8-based	RGB Camera	99.5	45	Limited robustness and generalization due to reliance on a small, self-constructed dataset
YOLO-UCM [105]	YOLOv5	RGB Camera	93.5	15	Model trained on a simulated dataset
DDEB-YOLOv5s + StrongSORT [106]	YOLOv5s with StrongSORT	RGB Camera	91.7	98	High model complexity - Requires significant computational resources - Reduced speed (98 FPS compared to lighter models)
YOLOv11-based Model [107]	YOLOv11	RGB Camera	95.8	59.6	Focuses mainly on personnel detection
Pedestrian Detection Model [51]	YOLOv5 (Deep Learning)	RGB Camera	71.6	x	Challenges with occlusion and detection in crowded scenes
Slim-YOLO-PR_KD [108]	YOLOv8s	RGB Camera	92.4	67	Scope limited to pedestrian detection

below evaluates and summarizes the performance of the leading 3D object detection models used in underground autonomous truck applications. It highlights their capabilities to handle the unique challenges posed by underground environments, detection approaches, advantages, limitations, and key performance metrics.

4.4. Real World Case Scenarios of 3D Object Detection Systems in Underground Autonomous Truck Navigation

Selected case studies are presented to demonstrate the real-world application and relevance of advanced 3D object detection methods. This highlights their deployment scenarios, performance benchmarks, and architectural innovations, as well as limitations. Recent developments have sought to address some relevant practical obstacles to perception in intricate underground environments, as illustrated by these examples.

Pen et al. [6] employed 3D LiDAR to develop an efficient obstacle detection method specifically designed for real-time vehicle navigation in underground mining environments. Their methodology involved reducing the 3D LiDAR point cloud density through voxel downsampling, followed by the extraction of object boundaries using Euclidean clustering and an optimized RANSAC (Random Sample Consensus) algorithm for ground segmentation. In a 50 m range, the system demonstrated a detection accuracy of over 95% and processed frames at a rate of 0.14 s per scan, thereby confirming its suitability for dynamic environments. Distinguishing between stationary apparatus and transient hazards remains a significant limitation, despite the strong performance, suggesting the potential for integrating semantic classification models.

Hanif et al. [13] presented SOD-YOLOv5s-4L, an enhanced object detection model designed for underground mining environments. Building on the YOLOv5s architecture, the study introduced enhancements, including a decoupled detection head to better separate classification and localization tasks, a dedicated small-object detection layer for accurately identifying minor hazards, and the SIoU loss function to improve bounding box precision. The system demonstrated a strong performance, achieving a mean Average Precision (mAP) of up to 98% across multiple object categories and 99% for small targets, outperforming existing YOLO variants, such as YOLOv7 and YOLOv8, under the same conditions. The results highlight the model’s suitability for complex underground settings with poor visibility and cluttered environments. However, the study noted key limitations. The added architectural complexity resulted in longer training times and slightly slower inference speeds, which may impact real-time deployment on resource-constrained edge devices. Again, due to the model being trained on a specific dataset, its generalization to a different underground mining layout may require further fine-tuning and validation.

A YOLOv5-based object detection model was developed and deployed in multiple active underground mining sites for real-time pedestrian detection near mobile machinery [51]. The model was trained on an augmented dataset of 1944 images collected under challenging underground conditions, including poor lighting, dust, and motion blur. It achieved promising results with a mean Average Precision (mAP) of 71.6%, indicating its feasibility for underground safety applications. The system’s performance was validated through real-world testing, demonstrating its ability to detect pedestrians even in the presence of motion blur. However, limitations were observed in detecting overlapping pedestrians and achieving optimal performance under all underground conditions. The study noted that improvements in hardware, such as industrial-grade cameras, and further tuning of model parameters could enhance reliability. Additionally, the system currently stops at detection and does not integrate reactive actions such as stopping machinery or triggering alerts.

The application of LDSI-YOLOv8, a YOLOv8 variant enhanced with DenseNet-inspired modules and transformer attention mechanisms, was implemented in this case study in active excavation environments. Variable illumination, irregular tunnel geometries, and overlapping objects distinguished coal mine scenes. Focusing on the multi-class detection of miners and apparatus, the system was trained and tested on video footage from real-world excavation sites. Moreover, 91.4% mAP and 82.2 FPS were attained by LDSI-YOLOv8, demonstrating its ability to maintain high precision in challenging visual conditions [19]. Although the system’s performance in extreme occlusion conditions still required refinement, the use of attention modules substantially enhanced the accuracy of detection around object edges.

Zhang et al. [76] also presented a sensor fusion-based tracking method for underground mining vehicles, which leveraged YOLOv5 for visual object detection and combined it with laser sensors for distance measurement. The system used a high-resolution camera to detect mine cars and a laser rangefinder to determine their position in real time. These components were mounted on a motorized platform, allowing dynamic alignment with moving targets. By fusing image and range data, the system calculates the spatial position of vehicles with centimeter-level precision, overcoming traditional limitations of monocular vision in dark and confined underground environments.

The model was tested in underground tunnel scenarios with low visibility, complex geometries, and occlusions. It demonstrated reliable performance in detecting and tracking moving mine cars, proving its value for enhancing autonomous navigation and collision avoidance. However, key limitations were noted, including critical misalignment in timing, which can degrade positional accuracy, as well as system complexity and increased cost due to the integration of precision hardware. Calibration drift over time can impact accuracy, requiring routine maintenance. Additionally, the approach, while accurate, may be less scalable in fleets with numerous vehicles due to the demands of data fusion processing.

Major Original Equipment Manufacturers (OEMs) have developed and deployed Autonomous Haulage Systems (AHSs), which have primarily seen improvements in surface mines. For instance, Caterpillar’s MineStar Command for hauling has been implemented at several large-scale operations, including Fortescue Metals Group’s Solomon Hub and BHP’s Jimblebar mines in Australia and several mines in the United States [109]. These applications enable autonomous trucks to navigate without an operator, utilizing onboard GPS, radar, and LiDAR systems integrated into a central control platform. Similarly, Komatsu’s AHS has been extensively utilized in the Pilbara region of Western Australia, notably at Rio Tinto’s Yandicoogina and Nammuldi mines, where over 300 autonomous trucks operate across multiple sites [110]. These systems rely on high-precision satellite navigation and obstacle detection technologies. Hitachi’s AHS, integrated with Wenco’s Fleet Management platform, employs LiDAR, GPS, and vision sensors in surface operations across North America [111,112]. These systems exemplify the successful deployment of sensor fusion in real-world industrial settings.

While designed for surface environments, they show the scalability and reliability of multi-sensor fusion in harsh mining conditions. Their success highlights the feasibility and industrial interest in adapting such technologies to underground environments, where perception challenges are even more severe due to GPS denial, limited visibility, and dynamic tunnel layouts. These case studies underscore the transition from conventional sensor-based obstacle avoidance to sophisticated DL-driven perception systems that can function in the challenging and dynamic environment of underground mining.

5. Dataset Analysis, Challenges, and Proposed Strategies

Annotated datasets tailored for underground environments are limited. Existing datasets are heavily focused on surface mine and urban driving conditions and are limited in diverse lighting, dust interference, and confined space conditions. While synthetic datasets for underground applications have been used as a cost-effective alternative, they often lack the complexities of real-world situations. Dynamic underground conditions create challenges that synthetic data fail to replicate accurately, leading to poor model generalization and reduced performance when applied in actual mining environments. Real-world datasets are crucial for models to handle the varied and unpredictable nature of underground mining. The better the dataset reflects real underground scenarios, the more accurately and reliably the model detects objects.

For instance, Zhang et al. [19] proposed an LDSI-YOLOv8 model trained on a custom dataset of 4800 annotated images from coal mine excavation faces, covering key targets such as workers and equipment under low-light and occlusion conditions. While it achieved 91.4% mAP and 82.2 FPS, the model’s main limitation is its validation on a single-site and limited dataset, which does not fully represent the underground setting and conditions and may affect generalizability across diverse underground environments.

Trybala et al. [74] developed the MIN3D dataset, which features multi-sensor 3D mapping data collected using an unmanned ground vehicle equipped with RGB-D cameras, LiDAR, IMU (Inertial Measurement Unit), and odometry sensors. The dataset was explicitly designed to capture the spatial, geometric, and visual complexity of underground mining tunnels. It includes multiple trajectories, annotated object categories, and scenarios involving dust interference, poor illumination, and irregular structures, making it valuable for evaluating sensor fusion and 3D perception models. MIN3D supports benchmarking for SLAM, object segmentation, and fusion-based 3D object detection under underground constraints. Despite its significance, the dataset is limited in terms of diversity, particularly in object classes, tunnel layouts, and dynamic underground scenes. Additionally, the lack of high-frequency sequences and labeled occlusion scenarios presents challenges for training deep learning (DL) models in real-time tracking. Nevertheless, MIN3D remains a foundational dataset for validating perception frameworks in underground environments, highlighting the importance of domain-specific data for autonomous truck navigation and safety.

Zhang et al. [105] also introduced a specialized object detection framework, known as YOLO-UCM, based on YOLOv5, tailored for pedestrian safety in underground coal mines. Due to the limitations of publicly available underground datasets, they developed the SDUCM-dataset, which simulates underground mining scenes under conditions such as poor lighting, smoke, dust, and occlusion. The model integrates advanced techniques, including Vision Transformer modules, Merge-NMS, and Meta-AconC activation, to enhance detection accuracy and generalizability. It achieved a maximum of 93.5% mAP and a recall of 99%, outperforming traditional YOLO versions and Faster R-CNN in both accuracy and inference speed. Despite its effectiveness, limitations remain, such as the dataset’s lack of diversity across mining sites and the YOLO-UCMx model’s size (87M+ parameters), which makes real-time deployment on embedded systems challenging. The authors suggest that future work should involve knowledge distillation and GAN-based dataset expansion to enhance model compactness and robustness.

Zhao et al. [106] also introduced a real-time perception approach based on an improved YOLOv5s model for detecting key targets such as shearers and mine workers in coal mine excavation faces. The model enhances spatial feature extraction through attention mechanisms and multi-scale fusion, thereby improving accuracy under occlusion and dim lighting conditions. The dataset used consists of actual images collected from underground coal mines, representing realistic operational scenarios and object categories. However, the dataset is highly task-specific, focusing only on machinery and layouts, which may limit the model’s generalizability to different underground mining settings or broader detection applications.

Table 5. Underground specific dataset and its characteristics.

Dataset Name	Sensor Type (s)	Objects Annotated	Environment	Limitations
LDSI-YOLOv8 Excavation Scenes [19]	RGB Camera	Pedestrian	Underground coal mine	Limited scalability across diverse mining environments Scene specific
Thermal image set [54]	Thermal IR	Workers, conveyor loads	Real coal mine	Lacks scalability
YOLO-UCM [105]	RGB Camera	Pedestrians	Underground mines	Model trained on a simulated dataset; real underground variability may affect model performance
Real-time perception excavation dataset [106]	RGB Camera	Miners, Equipment	Excavation working faces in coal mines	Generalization to highly dynamic or new tunnel layouts is untested
MANAGEM Pedestrian Detection Model [51]	RGB Camera	Pedestrians	Underground coal mines	Sensitive to occlusion and crowded scenes

outlines the key underground-specific datasets and their characteristics. This section presents a comprehensive breakdown of challenges relating to datasets in underground 3D object detection and maps each challenge to implementable solutions. Addressing real-world underground constraints like dust, occlusion, sensor misalignments, and data imbalance will advance the frontier of autonomous truck navigation in an underground mine environment. This fills a significant gap in the literature and provides actionable insights for future system model designs.

Recent detection frameworks in underground mining environments have achieved impressive results using RGB imagery, LiDAR, and thermal sensors. These rely solely on single data points and pose limitations under dust, fog, smoke, or complete dark conditions. Multi-sensor fusion offers a promising direction, integrating LiDAR for depth perception, thermal infrared for heat-based detection, and a camera for rich texture. Future research should prioritize developing fusion-based models that combine complementary sensor data, enhancing situational awareness and resilience for autonomous truck operations in the highly variable and constrained underground mining environment.

5.1. Dataset Challenges in the Underground Environment

The development of scalable and effective 3D object detection systems for autonomous underground haulage trucks is hindered by significant dataset-related challenges:

• Environmental Complexity: Underground environmental conditions pose significant challenges, which hinder effective dataset collection for detection models. Such challenges include poor lighting conditions, as these settings lack natural light. This leads to images with low contrast and clarity, making it difficult for optical sensors such as cameras to capture objects accurately. Additionally, dust and smoke from drilling, blasting, and transportation activities often scatter light and obscure sensors, limiting data quality. Furthermore, uneven terrain, cluttered backgrounds, and waste materials make distinguishing between detection objects and irrelevant objects challenging for safe detection.
• Data Annotation Challenges: Annotating 3D data like LiDAR point clouds for detection models is particularly challenging as it is labor-intensive and needs expert knowledge of mining scenarios. Unlike 2D image annotation, identifying objects in 3D, especially in scenarios involving overlapping objects or partial occlusions, demands significant effort. The annotating process is prone to human errors and can significantly impact model performance [2,37].
• Suboptimal Dataset Representation: Mining datasets are often characterized by issues related to overrepresenting common objects, such as trucks, and under-representing critical but rare classes, such as pedestrians. Frequently encountered objects dominate the dataset, while less common but critical elements, such as workers, are under-represented in the data. This imbalance causes biases that prioritize common classes at the expense of rarer but safety-critical objects to be detected. This degrades model performance and limits generalizability.
• Inefficient Model Generalization: A significant challenge in underground mining is the variability in mine layout and equipment types across different mines. This makes models trained in one mine less effective in another. The dynamic nature of mining operations necessitates continuous model updates that can adapt to variable scenarios.
• High Computational Complexity and Real-Time Constraints: High-resolution sensors generate terabytes of data daily, which require massive infrastructure for storage, robust computational efficiency, and real-time processing. Managing such large datasets while maintaining efficiency is a persistent issue [74]. Latency in data processing can potentially compromise the system’s effectiveness and lead to unsafe critical decisions [82,87].
• Temporal Synchronization in Multi-sensor Models: Temporal synchronization is a significant challenge in multi-sensor 3D object detection models. Variability in sampling rates, operating modes, and data speed from different sensors causes a misaligned data stream and impairs fusion quality and detection accuracy. Delays in transmission and hardware limitations worsen synchronization challenges. The study by [113] highlights the critical issue of temporal misalignment in sensor fusion, particularly in 3D object detection scenarios. Their study demonstrates that even minor synchronization errors between sensors, such as LiDAR and cameras, can result in significant inaccuracies in object detection, underscoring the need for robust synchronization mechanisms.

5.2. Proposed Strategies for Dataset Optimization and Model Robustness in Underground

The following strategies are proposed to address these challenges:

• Enhance Sensor Capabilities: Deploy robust sensors that withstand uneven terrain, dust, extreme vibrations, and temperature fluctuations to ensure consistent detection system performance. Integrate multi-sensor configurations by combining complementary sensing modalities. This will provide a robust and reliable detection system to enhance safety in underground mining operations.
• Advanced Data Preprocessing and Augmentation Techniques: Effective preprocessing and data augmentation techniques should be employed to significantly simulate dust, noise, occlusion, and variabilities in lightning [7,9,75]. Synthetic dust clouds, altering textures, and adjusting brightness can help the system adapt to variable surface types or equipment, ensuring robust object detection models for challenging underground conditions.
• Improved Data Synchronization: Solving temporal synchronization issues in multi-modal 3D object detection demands a combination of advanced computational methods and real-time data management strategies. Use software-based interpolation, a Kalman filter, or a deep learning-driven alignment framework to offer more flexible solutions for proper data synchronization. Catching strategies can also compensate for data transmission delays.
• Effective Data Annotation: Leverage data annotation tools such as semi-automated tools, active learning, and pre-trained models [83]. This will reduce manual annotation effort and improve labeling accuracy by minimizing human error and maximizing efficiency. Additionally, domain-specific annotation guidelines can ensure consistency.
• Optimize Dataset Representation: Accurate object detection requires balanced datasets. Applying class weighting, oversampling, and Generative Adversarial Network (GAN)-based synthetic data generation will balance rare and common object instances in datasets. This approach ensures that objects like pedestrians and/or rare but critical events receive more attention during training. This improves model robustness in accurately detecting safety-critical objects in real-world scenarios.
• Improvement in Model Generalization: Employ domain adaptation techniques such as transfer learning and adversarial training to improve models’ generalizability in cross-site applications. Consistent and continuous fine-tuning based on environmental feedback and characteristics enhances model adaptation.
• Edge Processing and Optimized Data Handling: Efficient data handling is crucial in managing large data volumes. Use efficient compression techniques to reduce the size of datasets without sacrificing critical features and integrity. Employing edge computing to optimize data will enable real-time and on-site data preprocessing. This reduces latency and bandwidth usage, enhancing the system’s ability to operate in real time. It will reduce data transmission time and minimize the load on central systems, allowing for quicker decision-making.

6. Key Challenges and Future Directions in 3D Detection Systems for Underground Mines

6.1. Challenges in the Underground Environment

The application of 3D object detection systems in underground mining environments is complex due to the unique and extreme conditions. These challenges can be categorized into environmental constraints, real-time operational intricacies, and computational limitations. Overcoming these barriers, especially with sensor fusion, real-time processing, detection accuracy, and pedestrian safety, will require further advancements in AI/ML, sensor technology, and edge computing. Future 3D detection systems in autonomous trucks must be intelligent, robust, and adaptive enough to operate safely and efficiently in unpredictable, diverse underground mining environments.

• Environmental Challenges. Underground environments present near-zero natural light, which requires reliance on artificial illumination. This condition introduces variable lighting, low-light noise, shadows, and glare, adversely impacting vision-based systems. The prevalence of low-light imaging noise and incomplete data under artificial lighting conditions affects the performance of object detectors. IR modalities improve visibility but are limited in range and resolution. Dust, airborne particulates, and smoke generated by drilling, blasting, and other unit mining operational activities interfere with sensor signals [4]. These conditions degrade the quality of LiDAR point cloud data, leading to degraded object detection. Although radar sensors are less affected by these environmental factors, they lack resolution, necessitating multi-sensor fusion techniques to ensure robust detection. Creating fail-safes, redundancy in sensor data, and methods for ensuring detection capabilities and reliability are essential for reliable performance.
• Dynamic and Irregular Obstacles. Continuous machinery and other object movements of other objects create a dynamic and unpredictable detection environment. Debris, uneven terrain, and narrow layouts can lead to misclassifications due to their resemblance to natural geological features in sensor data. Advanced semantic segmentation and computationally efficient ML-based classifiers are needed to address these challenges.
• Equipment Blind Spots and Sensor Occlusion. Large mining trucks have extensive blind spots, particularly around corners and confined spaces, and occlusions from structures or materials often obscure key objects. This increases the risk of undetected obstacles, raising the likelihood of collisions causing fatalities. Multi-view and re-identification methods for occluded objects have shown improved continuity in detection. However, these solutions often introduce significant computational overheads, limiting real-time performance implementation in underground environments.
• Sensor Data Integration and Overload. High-resolution LiDAR generates millions of data points per second, requiring advanced data fusion algorithms to integrate and process information from multiple sensors efficiently. Real-time data fusion across LiDAR, IR, and camera presents a substantial computational load. Managing data overload is critical to ensuring timely and accurate object detection.
• Real-Time Latency Challenges. The maximum need for split-second decision-making in autonomous trucks operating in high-risk environments means delays are unacceptable. Edge computing systems, which process data locally to reduce latency, have demonstrated effectiveness in mitigating this issue. However, challenges remain in the trade-offs between model complexity and detection accuracy. While YOLO algorithms have shown potential enhancement in 3D object detection, they are often affected by low-light, cluttered, small objects, and dynamic underground mine settings.
• Efficiency vs. Accuracy Trade-Offs. Large models give high accuracy but are computationally intensive and unsuitable for resource-constrained environments. Lightweight models such as YOLO-Nano have high speed but may lack robustness for detecting small or partially occluded objects [114].
• Generalization Across Mining Sites. Every underground mining environment has unique variability in tunnel geometry, layout, infrastructure, machinery, and operational processes. Detection models trained in one site may not be scalable in another, which poses a significant obstacle to generalizing 3D object detection systems. Due to these variations, domain shifts cause models to fail to perform effectively on another site.

6.2. Future Research Directions

Three-dimensional object detection remains crucial for both safety and efficiency. While significant progress has been made, challenges remain in optimizing algorithms, improving sensor fusion, and enabling real-time decision-making in dynamic underground environments. The following research directions are proposed to push the frontier of autonomous haulage detection systems:

• Multi-Sensor Fusion and Edge Computing: Integrating data from sensors like LiDAR, IR, and cameras with edge computing is needed to reduce latency and improve real-time processing. Enhanced fusion techniques that combine high-resolution LiDAR data with camera visual information could provide more detailed and accurate object detection. Additionally, by processing data locally, autonomous trucks can make real-time decisions without relying on external servers, improving response times and operational efficiency.
• Collaborative Detection Systems: Future autonomous trucks may not operate in isolation but as part of a larger network of autonomous systems, including other trucks, drones, and support equipment. Collaboration between these systems to share detection data and build shared scene understanding can improve object detection. Multi-agent communication and coordination could improve detection capabilities, object tracking, awareness, and cooperative navigation.
• Design Lightweight and Real-time DL Models: Research into the development of optimized lightweight models for embedded GPUs in autonomous trucks. Advancements in pruning and quantization can minimize model size without significantly compromising detection accuracy.
• Real-Time Object Classification and Localization: Improvements should be made in detection, real-time object classification, and 3D localization. Improved situational awareness enhances safe navigation and operational efficiency in underground mining operations.
• Development of Standardized, open-access datasets: More effort is needed to develop large-scale, labeled datasets that are open-access and reflect underground mining conditions, which will improve research into more robust models for autonomous truck safe navigation. Simulation data combined with real underground mining footage will provide efficient training benchmarks.
• Advanced AI and ML Models: Leveraging cutting-edge AI techniques, such as transfer learning, reinforcement learning, or deep reinforcement learning, could significantly enhance the adaptability and robustness of 3D object detection systems. These models could enable them to learn from diverse datasets and improve their detection capabilities, even in challenging environments like the underground mine. More sophisticated and newer versions of YOLO, like YOLOv8, can be integrated to improve detection speed and accuracy for maximum system performance. Developing domain-adaptive and site-specific fine-tuning models capable of learning and generalizing across different environments needs much research focus.
• Field Validation of Object Detection Systems: Validation of detection systems in operational underground sites to evaluate the model’s real-world performance. Additionally, safety-critical metrics should be incorporated into the evaluation process to assess the developed models’ real-world capabilities and safety impact.
• Regulatory and Ethical Compliance: As autonomous trucks become more recognized in the mining industry, ensuring that object detection systems meet safety standards and regulatory requirements will be essential. Developing frameworks that align detection systems with safety regulations and ethical standards is crucial. This will ensure transparency in model design and performance benchmarking, which is key to operational and public trust.
• Long-Term Robustness and Reliability: Long-term deployment of autonomous trucks in underground mines will require systems that can withstand harsh conditions, such as exposure to dust, vibration, and moisture. Ensure long-term stability of systems through real-time health monitoring, regular sensor updates, and robustness against sensor drift and wear. This will be essential to ensure continued safety and model performance.

Addressing these challenges and leveraging emerging research directions will enable the mining industry to accelerate the safe and efficient implementation of autonomous truck haulage in underground operations. This will mark a transformative growth in automation in the industry.

7. Discussions

The application of 3D detection systems in underground autonomous trucks presents critical challenges and opportunities different from other domains like urban driving and indoor robotic machines. The underground mining environment is known for its complex terrain, occlusion, high particulate matter, limited visibility, and dynamic equipment-worker interactions. Therefore, it necessitates more specialized sensor configurations and robust perception models capable of operating in these extreme conditions. This paper reveals that multi-modal sensor fusion, especially combining LiDAR and RGB cameras, provides the most robust perception capabilities. While the camera contributes contextual texture data, LiDAR offers high-resolution depth information. When fused effectively, they compensate for each other’s limitations, especially under low visibility or high dust conditions. Mid-level fusion is demonstrated to be the most balanced fusion approach, providing efficient feature integration while maintaining sufficient data richness for real-time object detection. Deep learning algorithms, particularly the YOLO-based framework, have shown strong performance in object detection tasks. Architectures such as YOLOv5 and YOLOv8 demonstrate real-time detection capabilities with significant mean Average Precision (mAP). However, small object detection and a lack of large-scale annotated datasets often hinder their performance in the underground environment. The following key trades were:

• Accuracy vs. Speed: It was noticed that high-accuracy models like YOLOv8 often demonstrate slower inference times. This can be problematic for real-time underground navigation.
• Sensor Cost vs. Redundancy: Multi-sensor models improve robustness. However, they also increase hardware costs and integration complexity.
• Fusion Complexity vs. Benefit: The early fusion approach provides detailed insights and features but is highly computationally expensive. Late fusion is computationally efficient but is less accurate.

Benchmarking for diverse datasets shows limited consistency in model evaluation, making it difficult to compare models directly. This shows the need for standardized mining-specific datasets and benchmarking protocols for efficient performance evaluation.

8. Conclusions

This review has comprehensively surveyed 3D object detection systems tailored explicitly for underground mining autonomous truck navigation. The review uniquely bridges the research gap between urban autonomous systems and the specific challenges of underground truck navigation. This study evaluates the current state of sensor modalities, detection algorithms, and fusion techniques. Additionally, it highlights the unique constraints posed by underground complex conditions. Though technologies such as LiDAR, RGB cameras, and thermal sensors have proven individual strengths, integrating these sensors provides a promising path forward. Deep learning architecture, particularly YOLO-based object detectors, has demonstrated strong potential in real-time detection. However, challenges persist, including occlusion handling, limited underground datasets, computational overhead, and performance trade-offs. While significant strides have been made in 3D object detection for autonomous trucks in mining, ongoing innovation and research are essential to overcoming the persisting challenges. These advancements will enhance operational efficiency and play a crucial role in safeguarding the well-being of workers.