The Road to the Mine of the Future: Autonomous Collaborative Mining

Abstract

The automation of mining mobile equipment is a topic of considerable interest, as it has the potential to significantly reduce the number of accidents and implement the so-called zero-entry mining concept, which would eliminate the need for any human presence on the mine site. Nevertheless, the current state of robotics and automation technology does not yet meet the requirements for the implementation of fully autonomous operations in mines. Autonomous mining equipment continues to operate under the supervision of humans, and a considerable number of mining equipment has not yet been automated. This indicates the necessity of identifying novel strategies to increase the safety of mining operations through the utilization of robotics and automation technologies. One potential solution to address this challenge is to increase the involvement of humans in autonomous mining operations. This could entail integrating human decision-makers into the decision-making loops of autonomous mining equipment. To this end, we propose the paradigm of autonomous collaborative mining, wherein humans and autonomous machines work together in a collaborative manner to increase the safety and efficiency of mining operations. We analyze the enabling factors required to implement this paradigm and present the case of autonomous loading using LHDs based on the autonomous collaborative mining paradigm.

1. Introduction

The use of automation technologies is critical in today’s mining industry because it makes it possible to increase the safety of the operations as well as their productivity and efficiency [1,2,3]. Moreover, the use of automated systems allows minimizing waste and reducing the environmental impact of mining activities [2].

Automated equipment can be static (e.g., conveyor belts) or mobile (e.g., trucks), and it is the latter type of equipment that is the focus of the analysis in this paper. Automated mobile equipment can have different degrees of autonomy, ranging from pure remote control to full autonomy. Mobile equipment (vehicles) with full autonomy are of great interest because they have the potential to implement the so-called zero-entry mining concept [2,4], where no human is required to enter the mine site, thus drastically reducing the number of accidents.

Groundbreaking projects and initiatives related to mining automation began in Canada [5,6,7]. In [8], one of the early implementations of industrial automation is reviewed, emphasizing that the use of automation in day-to-day operations provides flexibility and convenience for operators. Shortly thereafter, the first commercially available solutions for autonomous navigation of LHD (Load–Haul–Dump) machines were developed. This groundbreaking work was followed by developments for autonomous underground equipment in Australia (e.g., [9,10,11]) and Scandinavia (e.g., [12,13]), and autonomous trucks for surface mining in Japan and the United States. The first semi-autonomous LHDs and the first autonomous trucks were used in production in Chile, at Codelco’s El Teniente and Gaby mines in 2004 and 2008, respectively.

Following this groundbreaking work, several developments have been carried out by various OEMs (Original Equipment Manufacturers), and many automation systems are in production in various mines around the world. It is interesting to review the current state of automation of mobile mining equipment in the mining industry. As already mentioned, in the case of underground mining, special attention has been given to the automation of LHD vehicles because they operate in high-risk areas [14], but also because the mine production directly depends on the operation of these vehicles, which move the material along the production areas of the mines. The current standard is that semi-autonomous LHDs, which can haul and dump autonomously but are tele-operated during loading, are already in commercial use [15,16]. The same technology used to automate the LHDs has been applied to automate the low-profile trucks that operate at the haulage level of underground mines. In the case of open pit mines, the automation of the haul truck systems has received the most attention. Today, several mines around the world are operating with autonomous haul trucks [17,18], and given the reported benefits in terms of reduced operating and maintenance costs and increased productivity [2,17,18], many other operations are implementing this technology. Similarly, autonomous drills have recently attracted the interest of mining companies, and they are increasingly being used in open pit operations, also demonstrating benefits in terms of increased safety and cost reduction [2,19].

In all of the above cases of automated mobile equipment—trucks, LHDs and drills—they operate under human supervision for safety reasons, meaning that an operator remotely monitors the operation of equipment.

It is relevant to mention that, in addition to trucks, LHDs and drills, there are dozens of other types of mobile equipment/vehicles used in mines that have not yet been automated. Mining includes unit operations related not only to production but also to mine development (i.e., construction), where high-risk activities are carried out using a wide variety of mobile equipment. For example, in [14], it is shown that out of 59 unit operations in a typical block/panel caving mine, only 6 are fully automated worldwide (10%), 8 are teleoperated (14%), and 45 are still manually operated (76%). This means that we are still a long way from having a zero-entry mine [2], i.e., a fully autonomous mine with no people in the mine. This implies that we still need to find ways to increase the safety of mining operations through the use of robotics and automation technologies. In this paper, we argue that having a way to deal with this challenge is using collaborative human–machine systems.

The recent literature in the field of mining automation analyzes the current status, trends, and challenges of mining equipment automation. However, these studies have not clearly identified the potential contributions of collaborative human–machine systems in facilitating the transition to the zero-entry mine concept. A recent review of equipment and operations automation in mining is presented in [2], where several papers describing advances in underground and surface mining operation are discussed. The review focuses on the impact of automation on the safety and efficiency of mining operations, highlighting the successful implementation of autonomous truck systems in surface mining and describing the development of automation in underground mining and its challenges, particularly limitations in communication and localization, which hinder the development and deployment of fully autonomous systems. The review further discusses the potential for machine learning and AI to play a critical role in enhancing future mining automation systems. It also underscores the significance of collaborative human–robot work environments, stating that “as operations begin integrating automation, there will inevitably be a mix of both robotic and human workforce”. However, the review does not offer specific implementation strategies or propose any operational frameworks or paradigms to address this issue. In [20], the current state and perspectives of autonomous truck systems in surface mining are presented. The paper reviews the state of autonomous truck technology and identifies several challenges, including a shortage of qualified personnel, a need for interoperability, price issues, and employee resistance to the new technology. However, the paper does not identify the need for cooperative systems. In [21], a review of automation in the mining industry is presented, addressing the following three areas: automation technology, system engineering and management processes around automation, and the role of human factor engineering in automated and semi-automated systems. Notably, the review observes that “full autonomous solutions capable of providing the complete range of ore production capabilities remain limited in industry,” and that “most projects are deficient in a holistic approach that integrates process change and the re-engineering of operational systems”. The review further examines the role of human factor engineering in automated and semi-automated systems. However, it does not place significant emphasis on the relevance of cooperative autonomous systems or on the proposal of methodologies for addressing them. In [22], a survey is presented on the use of automation and AI technology in surface mining, providing a view of on some of the engineering problems, technological innovations, robotic development, and automation efforts encountered in the mining industry. The survey identifies opportunities for automation across the spectrum in most facets of open-pit mining; however, it does not analyze the technical challenges nor the need of cooperative systems. In [23], the automation of mining equipment in surface and underground mining is examined from a robotics perspective. The study identifies the automation of various equipment using different robotics and automation technologies, as well as the challenges associated with these implementations. Among the identified challenges is the implementation of automation projects as “islands of automation”, which underscores the need for greater integration. However, the study does not specifically address the need for cooperative systems. Another review of the current status of automation in the mining industry is presented in [24], identifying benefits such as “enhanced capital and operational productivity as well as improved safety and less use of water, energy, and consumables”. The review also states that, although autonomous equipment is used in many underground and surface mining processes, about 3% of all mobile equipment used in mining is truly autonomous. The review further notes the surge of interest within the industry concerning the concept of the zero-entry mine while acknowledging the considerable time and technological advancement required before the widespread implementation of fully autonomous mining operations. It identifies the capacity of robots to effectively maintain other robots as a significant technical challenge. However, the review does not address the potential role of human–robot collaborative systems as a transitional step in this process. In [25], a recent review of robotic automation in mining automation is presented. The review describes real-world industrial implementations and presents a comprehensive collection of performance evaluation methods. Additionally, it presents timelines illustrating the evolution of automation systems for pertinent equipment, including autonomous earthmoving equipment, drilling rigs, impact hammers, and autonomous haulage systems. The review also showcases future trends, including the use of artificial intelligence, data network challenges, multi-machine cooperation, and equipment electrification. However, the role of human–robot collaborative systems is not mentioned as a future trend.

In order to address the challenge of increasing the safety of mining operations through the use of robotics and automation technologies, this paper analyzes the current state of automation in mining and presents a path to the mine of the future. The urgent and immediate goal must be to reduce the risks associated with mining operations by eliminating exposure to unsafe environments, particularly the production area. It is proposed that the future mine will be a safe environment in which humans and autonomous machines will work collaboratively. To this end, the paradigm of autonomous collaborative mining is proposed wherein humans and autonomous machines work together in a collaborative manner. The paper presents a case study of the autonomous loading of minerals using LHDs. Finally, some conclusions and outline future work in this area are drawn.

It is important to clarify that this paper presents the Autonomous Collaborative Mining paradigm at a conceptual level and that its main components still need to be further developed and implemented. Our goal is to motivate the need to develop automation solutions for the mining industry using this paradigm, which emphasizes the need for collaborative automation solutions.

2. Key Challenges in Mining Automation

2.1. Current State of Mining Automation

A common misconception is that mining automation, understood as the automation of static and mobile equipment, has been remarkably successful. This view holds that the majority of equipment utilized in underground and open mines can operate either autonomously or teleoperated, and this would allow, as a subsequent step, the drastic reduction in the number the humans inside the mine sites and the implementation of the zero-entry mining concept, where no human is required to enter the mine site. Nevertheless, we contend that this is not the case. A significant number of mobile equipment still require automation, and the manner in which current automated mobile equipment operates needs to be improved. Additionally, when different automated equipment is operated together as a fleet of heterogeneous autonomous and teleoperated equipment, coordination and optimization still needs to be addressed. To illustrate the present state of mining automation, we may consider the case of block/panel caving mines, which are massive underground mining operations.

Block/panel caving is a mass mining method employed in hard rock mining, whereby “gravity is used in combination with internal rock stresses to fracture and break the rock mass into pieces that can be handled by miners” [26]. The considerable scale of block/panel caving operations entails a high degree of mechanization across a range of operational activities. This facilitates enhanced productivity and safety within the operational framework. The high level of mechanization inherent to these operations facilitates the implementation of new technologies and improvements to existing ones in a manner that is more straightforward than in other types of mining operations. The automation of these activities plays a significant role in addressing the safety and production challenges posed by evolving rock mass conditions. By reducing personnel exposure to risk situations and enhancing control and management of operations, automation can help mitigate potential hazards and improve overall safety and efficiency. In a block/panel mine, there are over 50 distinct units and auxiliary operations related to the processes of mine operation (production), mining preparation, equipment and infrastructure maintenance, development of civil works, and auxiliary support. The authors of [14] provide a comprehensive examination of the automation status for the various operations inherent to block/panel caving mines. These findings are summarized in

Table 1. Level of automation of different unit and auxiliary operations according to the mining processes and subprocesses for a block/panel caving mine. Details of the different unit and auxiliary operation can be obtained from [14].

Process	Subprocess	Number of Operations	Level of Automation
			None	Remote Control	Autonomous
Preparation	Horizontal Development	24	21	2	1
	Installation of network and IT infrastructure	4	4	0	0
	Vertical Development	2	2	0	0
Operation (Production)	Undercut	4	4	0	0
	Production	5	1	2	2
	Ore Pass	2	0	2	0
	Ore Pass Transport	4	0	1	3
	Infrastructure	1	1	0	0
Auxiliary Activities	Support	7	6	1	0
	Maintenance	5	5	0	0
	Supply	1	1	0	0

.

As evidenced in Table 1, a total of 59 operations were identified, comprising 16 mining operations, 30 mine preparation operations, and 13 auxiliary/support activities. Of these operations, only six are fully automated worldwide, representing 10% of the total. Eight are teleoperated (14%) and 45 are still manually operated (76%). The six autonomous operations are as follows: ventilation (in horizontal development), LHD hauling (in production), LHD dumping (in production), rail hauling (in ore pass transport), conveyor belt hauling (in ore pass transport), and CAEX hauling (in ore pass transport). The eight teleoperated operations are as follows: LHD ore extraction (in horizontal developments), gallery cleaning (in horizontal developments and in ore pass), LHD loading (in production), fixed rock breaker (in production, in ore pass, and in ore pass transport), and street cleaning (in support). These numbers demonstrate that there are significant opportunities for the automation of mining equipment and that the majority of efforts have concentrated on the automation of the equipment used in production.

In order to enhance the safety of underground operations, it is essential to direct attention to the processes of development and auxiliary activities. In [14], this aspect is also analyzed. The study defines a risk index for the different operations and determines that 43 operations have a high-risk index (74%), 1 operation has a medium risk index (2%), and 14 operations have a low-risk index (24%). This indicates that there is a need for automation of these high-risk operations in order to reduce the level of worker exposure.

Other underground mining methods as well as open pit mining methods employ distinct mining processes, subprocesses, and unit operations. However, in all cases, the majority of the automation efforts have been focused on the automation of a select few unit operations, predominantly utilizing autonomous haul trucks, LHDs, and drills, along with a few additional pieces of teleoperated equipment. In light of the aforementioned, it is evident that the immediate goal should be to have mines where humans are situated at a safe distance from risky areas, and in which humans and autonomous and teleoperated machines work in an efficient and collaborative manner, instead of having a mine with no humans. In the next subsections, we argue that the current robotic technology does not allow for the full automation of the multiple unit processes and tasks required in mining.

2.2. Challenges in Robotics Technology for Autonomousvehicle Navigation

Mining is one of the few industries where autonomous mobile equipment, i.e., vehicles, are regularly used (e.g., autonomous haul trucks in open pit mines). As a result, there are few examples of other industries from which to draw technological developments. One potential source of inspiration could be the autonomous vehicle/automotive industry, where billions of dollars have been invested, but where satisfactory results have yet to be achieved. The development of general-purpose autonomous vehicles is worthy of analysis.

Although the first autonomous vehicles/cars were developed in Japan and in Germany in the early 1980s [27,28], the idea of having a massive adoption of autonomous vehicles started after the success of the second DARPA (Defense Advanced Research Projects Agency) Grand Challenge (2005) in which five vehicles were able to complete the challenge [29,30]. This success inspired companies like Google to pursue the development of this technology. Subsequently, billions of dollars have been invested in these developments. Nearly every major automotive manufacturer has initiated its own autonomous vehicle project, and numerous new companies and joint ventures have been established with this objective [28]. Nevertheless, this technology remains insufficiently mature to be used in most cities [31,32]. Consequently, following two decades of development and substantial investment, the application of this technology remains largely confined to a few select cities, including San Francisco and Phoenix, and states (e.g., Arizona and California in the United States). The technology is not yet sufficiently robust. It relies on the use of extensive LIDAR-based scans of the routes (high-density sensor maps), requiring a constant human supervision of the vehicles from central station. Furthermore, it only works under normal traffic conditions. In conclusion, the technology remains in an immature state: (1) it is not plug-and-play solution, as it requires a previous scan of the routes, (2) it does not work without human supervision, and (3) it is not operational under heavy or under unstructured traffic conditions [32]. The reasons for this lack of success can be attributed to two primary factors. First, the current state of robotics technology is not yet sufficiently robust to effectively deal with changing environmental conditions. Second, it remains challenging to accurately anticipate and respond to the myriad of potential scenarios that can arise on a populated road. Learning algorithms fail to make decisions in rare cases because there are no data to train the algorithms in these cases (out-of-distribution problem [33]).

This last point is significant because it may be assumed that the most recent AI algorithms, including convolutional neural networks, transformers, reinforcement learning, and diffusion models, provide a solution to the autonomous driving problem. While AI can undoubtedly assist in addressing the aforementioned challenges, it is imperative to acknowledge the existence of a fundamental limitation inherent to the majority of AI algorithms: the out-of-distribution (OOD) problem. An AI algorithm is trained on a specific distribution of examples or cases, and its efficacy is contingent upon its ability to generalize to similar examples or cases. If the examples or cases differ from those used for training, an out-of-distribution (OOD) problem arises, preventing the AI algorithm from inferring or reasoning correctly. This phenomenon occurs in autonomous vehicles when they encounter situations not included in their training data. Such scenarios are not uncommon when driving in real-world settings, particularly in complex urban environments.

A further DARPA Challenge, the DARPA Subterranean (SubT) Challenge, which is more closely aligned with mining, was recently launched with the objective of developing innovative technologies that can enhance underground operations. “The SubT Challenge program explored new approaches to rapidly map, navigate, search, and exploit complex underground environments such as human-made tunnel systems, urban underground, and natural cave networks” [34]. The main scientific and technical outcomes of this four-year endeavor are summarized in [35]. Advancements were observed in the domains of perception and SLAM (Simultaneous Localization and Mapping) under underground conditions, as well as in multi-robot operation and multi-sensor integration. However, with regard to decision-making, all teams still include humans in the main decision-making loops (human supervisor): “For most teams, the role of the human supervisor included commanding high-level missions, providing manual intervention when needed, deciding when to deploy communication nodes, coordinating the next system deployments with the pit crew, monitoring incoming sensor streams, reviewing artifact detections for viability, and sending artifact reports to the DARPA command post” [35]. This reinforces the continued importance of humans in ensuring the successful operation of robotic systems. In fact, one of the primary reported insights emphasizes the pivotal role of humans in system performance: “The Human Teammate Is (Often) the Critical Performance Limiter” (Insight #6 in [35]).

In conclusion, the current state of robotics technology is insufficient for full autonomous navigation (i.e., no human intervention) in complex, real-world environments. To address this challenge, it is necessary to integrate human supervision and, in many cases, incorporate human decision-makers into the decision-making loops.

2.3. Continuous Operation of an Automated Task Requires Human Intervention

Autonomous systems cannot operate without human intervention for long periods of time. As an example, let us analyze the case of the most widely used household robot in the world, the robot vacuum cleaner. A robot vacuum cleaner can vacuum autonomously, but it requires a human being to place it in the room to be cleaned, making sure that it has access to the entire place where it must perform the task for which the human must open the doors to the rooms where the cleaning must be performed and, if the house has more than one floor, take the robot to the corresponding floor. In addition, the vacuum cleaner has a waste container that must be emptied by a human being when it is full, and if water is used for cleaning, it must be filled by a human being. Moreover, after days/weeks of operation, maintenance and repair of the robot vacuum cleaner must also be performed by a human being. In summary, the robot vacuum cleaner can operate autonomously for short periods of time, but its medium- and long-term operation depends on tasks such as placement, supply, maintenance, and repair that only a human being can perform.

Similarly, if we think of any equipment that has to work inside a mine, in a shift regime, and perform tasks such as installing of infrastructure (meshes, bolts, etc.), drilling, blasting, transporting material, etc., it can operate autonomously for limited periods of time, but undoubtedly humans will be required to bring the equipment to its place of operation and provide it with the necessary supplies. Moreover, depending on the task, humans will supervise the safe operation of the equipment and evaluate the results obtained (e.g., mesh installation). In the medium and long term, humans will also perform maintenance and repair tasks, most of the time, inside the mine facilities.

In conclusion, the performance of tasks by autonomous equipment inside a mine requires human involvement.

2.4. Autonomous Equipment Confinement and Operational Interference Management

In underground mining operations, as well as in some surface mining setups, the number of interactions between mobile autonomous equipment and human personnel is minimized by isolating them from each other to achieve a safe operation. This is achieved by confining mobile autonomous equipment to designated mine areas, where human operators and manual equipment are prohibited. The isolation of autonomous equipment is facilitated by a confinement system that should encompass the following functionalities [36]:

• It must restrict unauthorized entry of equipment and people to the operating area of the autonomous systems, which is referred to as the autonomous operation zone (AOZ).
• It must be able to detect any unauthorized entry of people to the AOZ or any transgression of the system security.
• It must prevent unplanned departure of autonomous equipment in operation from the AOZ.
• It must provide controlled access for autonomous equipment to the AOZ.
• It must provide controlled and secure access for operators and manual equipment to the AOZ, when they are required to address operational interferences.

The confinement of autonomous mobile equipment may have a notable impact on system efficiency and production. As an example, the segregation of autonomous LHDs allows for the operation to be conducted in a safe manner at the production level. However, this has a detrimental impact on production, as each instance of human access to production areas disrupts the operation of autonomous equipment. It is important to consider that human operators must enter the AOZ on a regular basis to address operational interferences, such as resolving draw point hang-ups and performing secondary reduction tasks. Currently, operations implement a static confinement policy, whereby all autonomous LHDs cease functioning while human workers are present in the production area. Consequently, operations are entirely halted at the production level during the periods that human workers are required to be inside this level.

According to [36], the replacement of static confinement policies with dynamic confinement policies may prove an effective means of reconciling the competing considerations of safety and efficiency.

3. The Autonomous Collaborative Mining Paradigm

The mine of the future must be a safe environment in which humans work in areas that are free from hazardous conditions. In light of the current limitations of available robotics technology, it is not yet feasible to implement fully autonomous operation of mines. Consequently, some tasks will be conducted by autonomous mobile equipment, while many others will be performed by human teleoperation from nearby positions or from operating centers, inside or outside the mine, or directly by human operators. Furthermore, humans will continue to be essential for other activities, including equipment placement, supply provision, task evaluation, maintenance, and repair.

In this context, the key idea of the Autonomous Collaborative Mining paradigm is that humans and machines interact in a flexible, collaborative, and synergistic manner to achieve safer and more efficient mine operations. This is achieved through the use of various complementary technologies, including robotics, automation, IT, and machine learning. This paradigm is aligned and inspired in the new trends of collaborative robotics which is a “new form of robotic automation built to work safely alongside human workers in a shared, collaborative workspace” [37]. According to the Association for Advancing Automation, “the ability to work collaboratively with humans greatly expands the potential applications of robotic automation” [37].

The Autonomous Collaborative Mining paradigm includes the following components, which can be implemented individually or in combination:

-

Autonomous Operation under Human Supervision

-

Shared Autonomy

-

Dynamic Confinement and Online Monitoring

-

Augmented Reality to Enhance Human Capabilities

Furthermore, it is essential to consider the enabling factors that may facilitate or impede the implementation of the paradigm.

Before describing the main components of the paradigm, it is important to remember that the paradigm and its components are presented in this paper at a conceptual level. They still need to be further developed and implemented. Our goal is to motivate and encourage the community to adopt the ideas behind this paradigm to further develop systems based on it, which may combine some or all of its components.

3.1. Autonomous Operation Under Human Supervision

In consideration of the present state of robotics technology, automated operations, defined as operations carried out using autonomous equipment, will continue to require human supervision for safety reasons, as is the case in the autonomous vehicle industry.

Furthermore, it is also important to note that in many unit and auxiliary operations related to mine development, installation of infrastructure, cleaning, support, maintenance and supply, in which the equipment is installed and used for a fixed period of time, an increase in the level of automation of an operation does not entail the complete withdrawal of the personnel from the mine. Rather, it entails a withdrawal of the personnel from the work zone, with high associated risks, to a safer zone. This is mainly because even after an operation is automated, human operators are still required to prepare, monitor, supervise, evaluate, and complete the operation. This aspect is frequently overlooked in the design of automated mining operations. Some of the tasks that will still be carried out by humans when a manual mining operation is automated include [14].

• Inspection of the work area and the operational variables present in the environment.
• Verification of the state of the equipment to be used.
• Transportation and installation of the equipment in the area of operation.
• Administration of supplies required in the operation.
• Execution of the unitary operation.
• Uninstallation and removal of the equipment from the work area.
• Quality assurance (QA) of the result of the operation.

Figure 1 illustrates the continued involvement of humans in the operations of autonomous units, including preparation, withdrawal, quality assurance, and supervision (either locally or remotely).

Consequently, within the autonomous collaborative mining paradigm, humans remain a presence, assuming a variety of roles. One such role is that of supervision of autonomous operations.

3.2. Shared Autonomy

The concept of shared autonomy provides an alternative to full autonomy, whereby a human operator can provide situation awareness, decision-making support and other assistance to an autonomous system [38]. In the context of mining, where the operation of autonomous mobile equipment requires human supervision for safety reasons, shared autonomy may be a more suitable option than full autonomy, as it could lead to more reliable and safer operations.

Let us explain how shared autonomy works and describe its distinguishing features in comparison to alternative automation modes. The following formalization is employed: as shown in Equation (1), the orders that the mobile equipment (machine) receives in timestep t, $u\left(t\right)$, are contingent upon a function $h_c\left(t\right)$, which combine the human orders, $u_h\left(t\right)$, and the autonomous system orders, $u_a\left(t\right)$, depending on the context, c. The context in which these orders are given may be dependent upon a number of factors, including the task completeness, the environment, the human operator activity, or other human-related factors.

$$u\left(t\right)=h_c\left(u_h\left(t\right),u_a\left(t\right), t\right)$$

(1)

If we apply this definition to the levels of autonomy shown in

Table 2. Levels of autonomy of a mobile system operating in a mine.

Manual Operation	A Human Operates the Equipment
Remote Control or line-of-sight Teleoperation	The equipment is teleoperated from a distance of a few meters, with the operator observing its operation directly.
Teleoperation	The equipment is teleoperated from an operating center, inside or outside the mime.
Teleoperation with Assistance	The teleoperation process include the use of assistance whereby the operator receives some form of assistance from an artificial agent or algorithm.
Semi-Autonomy	The equipment performs certain tasks teleoperated and certain tasks autonomously.
Shared Autonomy	Adaptively, or on demand, it is decided whether the equipment operates autonomously or in a teleoperated manner.
Autonomy	The equipment operates autonomously.

, we see that in the cases of manual operation, remote control, and teleoperation without assistance, $u\left(t\right)=u_h\left(t\right)$. Conversely, in the case of autonomy, $u\left(t\right)=u_a\left(t\right)$. In the case of semi-autonomy, $u\left(t\right)=u_h\left(t\right)$ or $u\left(t\right)=u_a\left(t\right)$ depending on the task at hand. For instance, in the context of a semi-autonomous LHD, $u\left(t\right)=u_h\left(t\right)$ for loading and $u\left(t\right)=u_a\left(t\right)$ for hauling and dumping. In the case of teleoperation with assistance, $u\left(t\right)$ is a combination of $u_h\left(t\right)$ and $u_a\left(t\right)$, as defined when the system is designed. In this case, the assistance provided to the human operator may be static, or it may be contingent upon the operator’s internal state or the external environment. For instance, the assistance level may increase when the operator is fatigued or when the machine is in proximity to other vehicles or humans. Finally, in the case of shared autonomy, $u\left(t\right)$ is a combination of $u_h\left(t\right)$ and $u_a\left(t\right)$, with the autonomous system determining the specific manner and timing of this combination. Thus, $u\left(t\right)$ is controlled by the autonomous systems of transferred to the user. For instance, in the case that the autonomous system request assistance from a human operator, $u\left(t\right)=u_a\left(t\right)$ transitions to $u\left(t\right)=u_h\left(t\right)$ while the human is providing assistance and then reverts to $u\left(t\right)=u_a\left(t\right)$ when the autonomous system regains control. Naturally, we also have the case in which the human does not assume complete control of the machine. In such case, the order received by the machine is a combination of $u_h\left(t\right)$ and $u_a\left(t\right)$. What is relevant in the shared autonomy case is that the autonomous system is the one that decides how and when the human orders and its own orders are combined. However, it is important to note that in cases where the human supervisor detects unsafe or incorrect behavior, they can take full control of the machine.

Thus, we propose that within the autonomous collaborative mining paradigm, shared autonomy may be regarded as a viable alternative to full autonomy in instances where the latter is not readily attainable. To illustrate this proposition, we present a case study in Section 4, wherein the application of shared autonomy to the task of loading from draw points using LHD is proposed.

It is worth noting that some current fleet management systems for mining equipment are already incorporating the concept of shared autonomy, although not necessarily by that name.

3.3. Dynamic Confinement and Online Monitoring

As previously stated, autonomous mobile equipment is confined to specific autonomous operation zones (AOZs) for reasons of safety. The presence of human operators or manual equipment is prohibited in these zones. This so-called static confinement results in inefficiencies due to the necessity for autonomous equipment to halt operations each time human operators must enter the production area. A better solution to this suboptimal approach to confinement is the implementation of dynamic confinement policies. In contrast to the static confinement paradigm, the AOZ is updated in real time as the operation progresses under the dynamic confinement paradigm. The dimensions of the AOZ are contingent upon the prevailing operational status and are not necessarily determinable in advance. Key elements for implementing dynamic confinement policies are (1) the use of sensors embedded within the environment (e.g., laser barriers) that indicate the mobile equipment the limits of permitted areas and act as virtual gates which can be dynamically activated (see examples in [36]) and (2) the real-time provision of positional data from all mobile equipment and human operators within the AOZ to a central system. This central system is responsible for defining the AOZ at any given moment and generating alarms when any agent makes an unauthorized movement, such as an unauthorized entry or departure from the AOZ.

In order to illustrate this idea, Figure 2 presents a conceptual diagram of a central system that allows the determination of the poses of multiple agents on an AOZ. The color ellipses and arrows represent the estimated position of the different agents—humans, autonomous systems, teleoperated systems, and manually operated systems—and their direction of movement. The position of each agent is estimated based on the reported position of the agent, with variance (represented by the size of the ellipse) added to account for errors in the estimation of the own position by the agent (self-localization error) and time delays in communication. Additionally, static sensors (S) estimate the position of some agents within their field of view and provide supplementary observations.

The implementation of dynamic confinement policies not only enhances operational efficiency but also provides a greater degree of operational flexibility. This enables the coexistence of automated equipment and human personnel within the same designated areas, with the establishment of virtual boundaries delineating their respective domains.

For example, in [36], simulations are carried out to assess the performance of different confinement policies in a panel caving mine with 16 production drives. The results show a reduction in the operational losses (interference time) from a 24% in the case of using a static confinement policy to 5–7% in the case of using a dynamic confinement policy, converting most of this time into effective time and having a better utilization of the semi-autonomous LHDs. The increase in effective time results in a 35–47% increase in production.

3.4. Augmented Reality to Enhance Human Capabilities

The term augmented reality (AR) is defined as the provision of visual interfaces that superimpose computer-generated digital information onto the perceived physical world. This enables the provision of real-time insights, visualizations, and instructions in a direct and immediate manner within the user’s field of vision using devices such as smartphones, special glasses, and other wearables. There are numerous applications for this technology, including (i) the provision of information about parts and components during an inspection process of facilities or equipment, wherein the system guides the operator to the specific locations that need to be inspected and identifies components and defects, (ii) equipment maintenance or repair, wherein the operator is able to follow visual instructions superimposed on the equipment part to ensure precise execution of maintenance procedures, and (iii) remote support, whereby the onsite operator can stream their view of the equipment to a remote operator (expert) who can provide instructions or annotate data directly in the field of view of the onsite operator. The information provided to the operator may be obtained from annotations (notes, comments, and markings) made previously during a planning stage. Alternatively, the information may be provided online by an AI assistant or by an expert placed in another location.

The application of AR in a mining context has the potential to enhance the capabilities of operators, providing them with more effective tools and multi-purpose functionality. A given operator is able to perform a variety of tasks, including repairs on different types of equipment, with the assistance of AR-based systems. One can envisage a scenario in which an operator is situated inside a mine facility and is required to either initialize or repair a piece of mine equipment, despite lacking complete knowledge of the task at hand. In such a situation, a virtual assistant can furnish the operator with the necessary information through the utilization of an AR interface, thereby enabling the operator to complete the task successfully.

AR can also be used to enhance the visualization employed by a teleoperator when remotely controlling equipment. In this case, supplementary data, such as the distance to objects and the identities of object components and parts, can be superimposed upon the visualization displayed to the operator. In addition, telepresence, defined as teleoperation using special sensing and display technologies that enable the human operator to feel present at the remote location despite the operator’s physical absence [39], also enhances the quality of remote control tasks. Virtual reality (VR) also offers a variety of applications in the mining industry, including operator training and teleoperation. In the latter case, the teleoperation task can be conducted in a virtual environment generated using data from sensors monitoring the environment.

In the context of autonomous collaborative mining, the use of AR, VR and telepresence technologies facilitates an enhanced interaction between human operators and the equipment to be controlled or inspected. But more importantly, it allows for the enhancement of human operators’ capabilities, transforming them into versatile operators capable of performing a diverse range of tasks. This is relevant to achieve the goal of reducing the number of humans in potentially hazardous working areas, as multi-purpose human operators allow for a reduction in the required on-site personnel.

It should be noted that the use of AR, VR, and telepresence technologies requires an appropriate design in order to avoid mental fatigue of the operator/user. Mental workload must be considered in the design of these interfaces, and in general human factors must always be considered in the design of any computer interface. The proposed idea is not to use these interfaces permanently, but only when needed to solve specific tasks.

3.5. Integration

The various components of the paradigm can be used in an integrated fashion. Figure 3 shows a conceptual diagram that integrates the different components and technologies. In this diagram, machines with different levels of autonomy and humans using different types of interfaces work together in an AOZ using dynamic confinement. All machines are subject to human supervision for safety and efficiency reasons; at any time, a supervisor can take full control of a particular machine if unsafe or incorrect behavior is detected. Of course, supervisors can also detect incorrect human behavior and take corrective actions.

Machines can be operated either autonomously using shared autonomy, remotely using telepresence technologies, or manually. People within the AOZ can be operating the machines or performing other tasks such as sampling, inspection, or repair. In all cases, they can use AR or VR interfaces to augment their capabilities, as needed. The use of dynamic confinement allows for a safe interaction between the various machines and the people inside the AOZ.

3.6. Safety of Human–Machine Interactions

One of the most important ideas of the autonomous collaborative mining paradigm is to increase collaboration between humans and machines, which often includes closer human–machine physical interaction. This closer interaction requires the development of safety regulations, procedures, and protocols to prevent any possible collision or similar incidents that could put people in danger. In addition, proper technology, such as surveillance systems that monitor the activity of humans and machines, should be developed to avoid risky situations. The safety procedures and protocols to be implemented must consider potential system failures, algorithm errors, communication problems, and human factors. Human factors analysis is especially important because humans make errors and do not always follow the rules and safety protocols.

As mentioned earlier, the mine environments and unit operations exhibit significant variability, as does the employed mobile equipment. Therefore, safety procedures and protocols must be meticulously designed for each distinct application of collaborative automation systems that operate under the proposed paradigm. This issue is of paramount importance and must be addressed prior to any physical implementation of the paradigm.

For instance, in [36], the dynamic confinement of semi-autonomous LHDs operating in a panel/block caving mine is proposed at a conceptual level. This dynamic confinement is implemented using sensors that act as virtual gates (e.g., laser barriers), allowing a high level of segregation for both people and equipment and registration areas for control access. In the context of a practical implementation of this concept, it is imperative to ensure the redundancy of sensors in the event of failure, as well as the incorporation of safety protocols that address scenarios such as network and sensor failures, malfunction of the software controlling the LHDs, and human errors.

3.7. Enabling Factors

The advancement of the autonomous collaborative mining paradigm requires the development of interoperability standards, new regulations, the adaptation of the mining designs and procedures to facilitate the use of autonomous and teleoperated equipment, the development of new sensors to be placed in the mines, high-bandwidth and low-latency data networks, and a workforce trained in the use of the new technologies. These factors, among others, must be considered to ensure the successful implementation of the autonomous collaborative mining paradigm. In order to achieve this, it is necessary to coordinate the efforts of mining companies, original equipment manufacturers (OEMs), research and development institutions, technological companies, and regulatory bodies.

To illustrate the significance of the aforementioned enabling factors, the lack of interoperability standards for autonomous mining vehicles hinders the development of new modules and products by non-OEM entities, thereby slowing down the development of new solutions. For instance, in light of the recent advancements in artificial intelligence (AI), particularly in the domains of machine and deep learning, there arises an opportunity to leverage novel methodologies wherein AI can enhance the functionality of existing automation technologies. This may include the development of enhanced object and human detection algorithms, more advanced optimization algorithms for navigation, more sophisticated sensing methodologies, more advanced learning algorithms, etc. However, the application of AI-based solutions developed at research and development centers or at technology companies to commercial mining mobile equipment is hindered by two main factors: the lack of interoperability standards and the presence of contractual restrictions that prohibit any intervention with the equipment. In order to effectively address these challenges, it is essential to foster collaborative initiatives among OEMs, mining companies, and technology developers.

4. Case Study: Autonomous Loading with LHDs

To date, there are no commercial autonomous loading systems for LHD that can be used in in block/panel caving mines and sublevel stoping mines due to the complexity of the loading process; loading fragmented rock from draw points is a complex task due to many factors, including bucket–rock interaction forces that are difficult to model, moisture that increases cohesion forces, and the possible presence of boulders.

In order to address this challenge, the authors in [40] proposed an autonomous loading system based on the autonomous collaborative mining paradigm. The system is designed to integrate all the relevant tasks required for ore loading, including rock pile identification, LHD positioning in front of the ore pile, charging and excavating into the ore pile, pull back, and payload weighing. The system employs the concept of shared autonomy whereby, in cases where the loading process cannot be completed autonomously, the LHD’s controller is able to detect this situation and request assistance from a human operator.

4.1. On the Complexity and Requirements of the Loading Process

Loading of fragmented rock involves a sequence of steps: As the LHD’s operator approaches the draw point, a quick assessment of the condition of the rock pile is made. The LHD’s bucket is tilted down until the tip is pressed against the ground. The machine is then commanded to charge/attack the rock pile. Once the LHD has made contact, it buries the bucket into the rock pile and the bucket is tilted and lifted as it advances through the fragmented rock. When sufficient penetration is achieved, the bucket is fully retracted, and the vehicle is withdrawn with a reverse motion. The operator then shakes the bucket briefly and assesses whether or not enough ore has been loaded. If necessary, a new loading maneuver is performed.

The step in which the machine is being controlled to dig into the rock pile is herein referred to as the excavation process. During excavation, bucket–rock interaction forces affect the bucket motion through the material and ultimately define the amount of ore loaded. Inter-particle forces in granular material and bucket–rock interactions have been studied previously, but their complexity has led only to stochastic modeling [41]. As a result, an analytical approach to bucket motion control is not possible, and it is highly unlikely that these interactions can be accurately replicated to develop a system in simulation. Therefore, only a real-world environment can provide the appropriate conditions for developing and experimenting with an excavation method. Furthermore, fragmented rock is much more difficult to load than piled sand or gravel. This is exacerbated by the environmental conditions of underground mining. Depending on the mining method, the column of blasted rock can exert large compressive forces on the open face of the extraction point. Humidity can also increase cohesion forces, making the whole process more difficult [42]. In addition, large rocks may be present at the draw point. For this reason, the operator must be aware of the presence of large rocks as he approaches the draw point, and, depending on their size, may be forced to change the goal of the maneuver from filling the bucket to just loading one large rock. If the rock is too big to be loaded, the task is aborted. Furthermore, other conditions, such as narrow tunnels, uneven or unprepared terrain, and/or significant amounts of water and mud near the draw point, can make the loading process very challenging.

An autonomous system should be prepared to deal with all of these situations, in addition to performing certain processing and steps that a human operator performs with while driving the LHD, such as detecting a sufficiently accurate enough position of the rock pile before charging at it, making sure not to hit the tunnel while pulling back from the rock, and estimating whether the bucket is full enough.

4.2. Autonomous Loading Based on the Autonomous Collaborative Mining Paradigm

An autonomous loading routine can be thought of as a sequence of several steps that perform specific actions. Here, rock pile identification is the process of finding the location and status of the draw point (whether it is suitable for loading or not). Positioning refers to orienting the LHD machine to ensure that a forward thrust will end in a collision with the rock pile. Charging is the step that considers lowering the bucket and accelerating towards the pile. Excavation is the machine control that performs the digging action. Pull back comprises the backward movement of the vehicle and bucket shake. Finally, payload weighing estimates the amount of ore loaded. Most of these steps may fail due to the highly variable environment in underground mining. For example, (i) the on-board sensors may not accurately locate the rock pile, (ii) the machine positioning may not avoid colliding with a tunnel wall, (iii) the collision between the machine and the rock pile may not be detected, or (iv) the bucket of the LHD may become stuck during excavation. In any of these cases, human intervention would be required to resolve the problem. In fact, most advanced mining automation systems for mining operations require human interaction with automated mobile machines, at least to monitor/supervise a fleet of machines. Since large operations have a fleet of vehicles, it is common for one or more operators to monitor/supervise and assist multiple semi-autonomous machines, as each machine does not require constant attention. For these reasons, it is imperative that this human interaction be taken into account in the design phase of an autonomous loading system and thus included in the formulation of its behavior.

Taking these considerations into account, an autonomous loading process with these characteristics is presented in [40]. Figure 4 shows the proposed state machine that models the autonomous loading process. Two distinct assistance states are defined: positioning assistance, wherein the operator assumes control of the machine to position it in front of the rock pile, and loading assistance, wherein the operator assumes control of the LHD to perform the remaining steps of the loading maneuver up until payload weighing. Positioning assistance is called up either when the rock pile identification fails to characterize the rock or if positioning is unable to place the LHD to start the charging procedure. Once the LHD is positioned correctly, the LHD’s controller regains control of the LHD in order to commence charging the pile. Loading assistance is called up if charging, excavation, or pull back cannot be completed successfully. Following the successful completion of the pullback step, the LHD’s controller regains control of the LHD in order to complete the payload weighing step.

In Algorithm 1, a pseudocode is shown that implements the state machine of the loading process.

Algorithm 1: Autonomous loading process.

4.3. Validation in Real Mining Environments

The excavation algorithm of the autonomous loading system was first validated for six months in a sublevel stoping copper mine in northern Chile. In these experiments, an LHD model LF11H from the German supplier GHH with a bucket of 11-ton capacity was used. The main sensors used were two SICK LIDARs model LMS511-10100 (see more details of the experiments in [40]). The rock pile identification module was validated using data obtained during the same tests.

At the end of the validation period, a total of 30 complete loading trials were performed. Similar to manual operation, multiple loading attempts were required to fill the bucket during each trial (manual operation of the LHD in sublevel stopes typically requires two or three attempts to achieve a proper bucket fill factor, as the procedure is performed using line-of-sight radio control with limited visibility behind the LHD). A total of 81 loading attempts were made to complete the 30 full loads. Of these attempts, 7 failed (early collision trigger) and 74 were successful, resulting in a 91% success rate of the loading and collision detection methods. For the 30 completed loads, an average of 3.6 attempts were required to obtain a bucket full of material, with an average fill factor of 90% (see Table 3 in [40] for more details).

After this first validation in a real mine, the system was improved and adapted for use in Room and Pillar mines and subjected to a full validation (all steps) again, during six months, in a potash mine located in Germany. In these experiments, an LHD model SLP-14H from the German supplier GHH with a bucket of 14-ton capacity was used. The main sensors used were two SICK LIDARs model LMS511-10100 and one 3D Ouster LIDAR model OS2-64. To evaluate the proposed autonomous loading method, eight consecutive loads of material (trials) were performed. Then, eight consecutive loads of material were performed by an expert operator in the same sector. In each trial, the amount of material loaded and the loading time were recorded for comparison. In seven out of eight trials, the autonomous LHD was able to complete the loading process (88% of the time). In one trial, the LHD generated an alarm and aborted at the pullback stage. The average weight achieved by the autonomous loading system across all trials was 11.3 tons, 79.25% of the average weight achieved by manual loading. The details of this validation are described in [3].

It must be noted that in Room and Pillar mines, there is a wide variety of muck piles, and working under the principle of shared autonomy makes the system more flexible to deal with this diversity; thus, due to the very dynamic nature of the muck piles used in Room and Pillar mines, the here-described autonomous loading system is the first one to be successfully validated in this kind of mine.

Since November 2024, the system is being fine-tuned to achieve the production KPI defined by the customer, and there is a good chance that the system will be adapted by this mine.

The reader is referred to [40] and [3] for details of the technical implementation of the technology and how the validation process was carried out.

5. Conclusions

The future mine must be designed to provide a safe environment for humans, situated at a safe distance from potentially hazardous areas. The automation of mining operations plays a pivotal role in achieving this objective. In this context, we put forward the autonomous collaborative mining paradigm as a means of enhancing the automation of mining operations by incorporating human–robot collaborative mechanisms. This paradigm entails a flexible, collaborative, and synergistic interaction between humans and machines, with the objective of achieving safer and more efficient mining operations. The paradigm posits that autonomous operations still require human involvement, particularly in supervisory roles. Furthermore, shared autonomy is regarded as a potential alternative to full autonomy in instances where the latter is not a viable option. Dynamic confinement policies facilitate the physical coexistence of automated equipment and humans within the same working areas. Furthermore, the utilization of augmented reality (AR) technologies allows for the enhancement of human operators’ capabilities, thereby transforming them into versatile operators capable of performing a diverse range of tasks. This is a crucial aspect in achieving the objective of minimizing the number of humans in potentially hazardous working areas, as multi-purpose human operators allow for a reduction in the number of personnel required.

The autonomous collaborative mining paradigm represents not an end in itself, but rather a means of working towards the mine of the future. The implementation of the paradigm necessitates the consideration of pertinent enabling factors, including the development of interoperability standards, new regulations, the adaptation of the mining designs and procedures to facilitate the use of autonomous and teleoperated equipment, the development of new sensors to be placed in the mines, high-bandwidth and low-latency data networks, and the training of a workforce in the use of new technologies. The implementation of the paradigm cannot be addressed by a single institution; it requires the coordinated efforts of a number of stakeholders, including mining companies, original equipment manufacturers (OEMs), research and development institutions, technological companies, and regulatory bodies. This represents a significant challenge, but it is nevertheless a worthwhile endeavor to begin working towards the development and implementation of the paradigm.

It is notable that the autonomous collaborative mining paradigm has a direct application in space mining. The processes involved in mining on a planet or an asteroid can be conceptualized as analogous to those employed in the mining of distant locations, whether underground or in a remote desert. This indicates that the automation technologies currently utilized in terrestrial mining can be effectively applied in space mining. In this context, some key elements of the autonomous collaborative mining paradigm such as shared autonomy, human–machine collaboration, and general-purpose operators may be considered.