Enhanced High-Definition Video Transmission for Unmanned Driving in Mining Environments

Featured Application

This work can be applied to unmanned driving in mining environments, with a specific focus on enabling low-latency transmission of high-definition video from underground vehicles to the surface operators.

Abstract

In the development of intelligent mines, unmanned driving transportation has emerged as a key technology to reduce human involvement and enable unmanned operations. The operation of unmanned vehicles in mining environments relies on remote operation, which necessitates the low-latency transmission of high-definition video data across multiple channels for comprehensive monitoring and precise remote control. To address the challenges associated with unmanned driving in mines, we propose a comprehensive scheme that leverages the capabilities of 5G super uplink, edge collaborative computing, and advanced video transmission strategies. This approach utilizes dual-frequency bands, specifically 3.5 GHz and 2.1 GHz, within the 5G super uplink framework to establish an infrastructure designed for high-bandwidth and low-latency information transmission, crucial for real-time autonomous operations. To overcome limitations due to computational resources at terminal devices, our scheme incorporates task offloading and edge computing methodologies to effectively reduce latency and enhance decision-making speed for real-time autonomous activities. Additionally, to consolidate the benefits of low latency, we implement several video transmission strategies, such as optimized network usage, service-specific wireless channel identification, and dynamic frame allocation. An experimental evaluation demonstrates that our approach achieves an uplink peak rate of 418.5 Mbps with an average latency of 18.3 ms during the parallel transmission of seven channels of 4K video, meeting the stringent requirements for remote control of unmanned mining vehicles.

1. Introduction

The trend in the development of the mining industry is towards the establishment of intelligent mines to reduce human involvement and achieve unmanned operations [1,2]. In recent years, unmanned driving [3] has emerged as a viable solution for transportation within mines, attracting significant attention from industry experts and technology companies. The consensus among experts is that the adoption of 5G wireless communication systems for remote control of underground vehicles from the surface [4,5] represents the most effective approach to enable unmanned driving in mines. Previous research findings provide a clear and feasible technical direction for unmanned operations in mines, highlighting the practicality of remote-controlled unmanned driving [6,7]. Enabling remote-control unmanned driving in mines relies on the availability of ample uplink resources, sufficient computing power, and effective transmission strategies. This infrastructure facilitates the low-latency transmission of video and other critical data, enabling unmanned vehicles to obtain a comprehensive and accurate perception of their surroundings [8,9]. Surface operators can then observe the environment from a control platform and issue real-time control commands based on the driving conditions [10,11], thereby enabling the remote operation of unmanned vehicles in mines.

The increasing need for significant bandwidth and low latency has propelled the widespread adoption of 5G technology as the foundation of intelligent mines [12], facilitating high-throughput information transmission within mining operations. Moreover, the deployment of technologies such as network slicing [13] and edge computing [14] has enhanced the service capabilities of 5G in various underground scenarios. Overcoming the challenge of achieving low-latency transmission of high-definition videos using 5G technology is considered a primary concern for unmanned driving in mines [15]. Currently, conventional 5G primarily assigns time slots for eMBB (Enhanced Mobile Broadband) [16], with a standard downlink allocation of 70% and an uplink allocation of 30%, which proves inadequate for extensive uplink scenarios, particularly high-definition video transmission from underground to the surface [17]. Furthermore, unmanned vehicles require collaboration with tunnel infrastructures during one-dimensional movement to offload computing tasks; however, the existing coordination algorithms [18,19] lack optimization for overall low latency. Lastly, in addition to fundamental transmission capabilities, network optimization for video transmission is also essential.

In this paper, we explore the video transmission needs for remote-controlled unmanned driving in mines, present a video transmission scheme leveraging 5G super uplink, collaborative computing, and transmission strategies, and subsequently validate this scheme to guarantee the dependable and consistent operation of 5G unmanned vehicles in mining environments.

2. Video Transmission Requirements Analysis

High-definition cameras are utilized in unmanned mining operations to capture 360-degree environmental images surrounding unmanned vehicles and transmit them in real-time to the remote-control platform. In this context, the primary requirements for video transmission include the following aspects:

• Single-channel high-definition video: the system must support video resolutions exceeding 2K, equivalent to a single-channel uplink rate of over 6 Mbps [20].
• Multi-channel simultaneous transmission: the system should enable the transmission of seven video streaming channels, facilitating panoramic monitoring of the vehicle’s surroundings [21].
• Low-latency real-time transmission: Achieving an average end-to-end latency of approximately 20 ms is crucial for ensuring prompt remote monitoring [22].

To fulfill these requirements, the video transmission system for unmanned mining operations should guarantee a single vehicle’s uplink transmission rate of no less than 42 Mbps. Considering the concurrent operation of multiple vehicles (e.g., five vehicles) and other 5G service transmission demands, the uplink transmission rate should be about 400 Mbps to maintain a steady and dependable video transmission for unmanned mining vehicles.

Presently, the predominant frequency duplex mode for 5G NR in mining operations [23,24] is Time Division Duplex (TDD), wherein the uplink and downlink are transmitted within the same frequency band but are separated by a time interval. For instance, utilizing the 3.5 GHz band and typical time slot ratio, it can support a downstream transmission capacity exceeding 1 Gbps, but only 280 Mbps for the uplink. Hence, this paper advocates for the adoption of mine 5G super uplink technology [25,26,27] to enhance the uplink transmission rate and facilitate efficient and stable transmission of unmanned mining videos.

3. Design and Implementation of Video Transmission

In response to the video transmission demands for unmanned driving [28,29,30], a video transmission scheme incorporating mine 5G super uplink, collaborative computing, and transmission strategies has been conceptualized and executed.

3.1. Architecture of Mine Unmanned Driving System

The mine unmanned driving system consists of a vehicle control subsystem, an information transmission subsystem, and an unmanned vehicle subsystem, as depicted in Figure 1. This proposed architecture requires the deployment of a private mine communication network connecting to tunnel 5G base stations for mine unmanned driving system. The vehicle control subsystem is situated in the indoor machine room on the ground, and typically includes the remote-control platform, video server, network switch, and display. Its primary functions are to achieve vehicle control, provide vehicle attitude display, offer vehicle environment video display, and enable human–computer interaction. The process of the vehicle control subsystem for remote monitoring and emergency takeover is illustrated in Figure 2. This subsystem relies on high-definition video environmental information transmitted through the 5G network and is monitored by the operator. In the event that the vehicle issues an emergency takeover request, or the operator detects risk factors in the operation of the autonomous vehicle, remote operation, safety monitoring, and emergency takeover of the mining vehicle can be implemented.

The information transmission subsystem facilitates wireless network communication between the unmanned vehicle subsystem and the vehicle control subsystem. It supports remote control of unmanned vehicles through the transmission of vehicle monitoring data, video, and other services using 5G super uplink technology. The system is designed to achieve intelligent unmanned assisted transportation and enhance safety with minimal human intervention.

The unmanned vehicle subsystem serves as the primary component of the entire system and is responsible for vehicle perception, decision-making, control, and execution. This paper focuses primarily on optimizing video transmission within the unmanned vehicle subsystem.

3.2. Mine 5G Super Uplink

In order to improve the uplink transmission rate of 5G in mines, the system utilizes super uplink technology. This technology extends the usage of the 2.1 GHz band to enhance uplink transmission, while continuing to utilize the 3.5 GHz band for regular transmission. Specifically, during the downlink time slot in the 3.5 GHz band, the 2.1 GHz band is utilized for uplink transmission simultaneously, thereby boosting the uplink transmission rate. The configuration parameters of the super uplink band can be found in

Table 1. Super uplink frequency band parameters.

Uplink Link Parameters	2.1 GHz	3.5 GHz
Channel Bandwidth	40 MHz	100 MHz
Slot Configuration	FDD	7:3 (up:down)
Base Station Noise Figure/dB	2.5 dB	3.5 dB
Uplink Interference Margin	3	2
Path Loss	20 dB	23.4 dB
Edge Coverage Percentage	75%	75%
Propagation Model	3GPP UMa	3GPP UMa

, while Figure 3 illustrates the concept of mine 5G super uplink technology. In the figure, ‘D’ represents the full downlink time slot, ‘U’ represents the full uplink time slot, and ‘S’ includes the protection interval and the upstream and downstream conversion symbol.

At the user equipment (UE), the uplink signal is dynamically transmitted in the 3.5 GHz and 2.1 GHz bands using time division multiplexing (TDM), achieved through the dynamic switching of transmitters. In order to simplify the hardware circuitry and address power consumption and heat dissipation challenges arising from the use of multiple transmitters, the UE can share components such as the phase-locked loop (PLL), power amplifier (PA), and digital-to-analog converter (DAC) [31], as depicted in Figure 4.

After the transmission link is established, resource allocation becomes a crucial consideration. Once the resource block (RB) is defined as the smallest unit of resource allocation in the 5G system, the data transmission rate of a single RB can be expressed as follows:

$$C_{RB}=B_{RB}{log}_2\left(1+SNR\right)$$

(1)

where $B_{RB}$ represents the bandwidth of the RB, and $SNR$ stands for Signal-to-Noise Ratio. Assume that the packet arriving rate of the user is ${\lambda }_u$ and the length of the packet is $L_u$ bits. When the number of RB is $N$, we calculate the average delay by referring the [32] as follows:

$${\tau }_u=\frac{L_u}{C_{RB}·N-{\lambda }_u·L_u}$$

(2)

Consequently, the minimum number of RBs can be calculated as follows:

$$N^{min}=⌈\frac{L_u+{\lambda }_u·L_u·{\tau }_u^{max}}{{\tau }_u^{max}·C_{RB}}⌉$$

(3)

where ${\tau }_u^{max}$ is the maximum allowable latency, which is 20 ms for unmanned driving video transmission.

By utilizing Equation (3) as a tool, it becomes feasible to finely allocate transmission resources for the super uplink in 5G networks. This includes designing dedicated access network slices for unmanned driving services and providing varying levels of resource redundancy for cameras with different priority levels [33].

3.3. Terminal-Edge Collaborative Computing

The 5G module is connected to the front-end camera of the unmanned vehicle, and it is powered by the vehicle itself. The camera captures video information using its shooting function and the 5G module. This information is then transmitted through the 5G wireless network to both the 5G base station and the server of the remote-control subsystem for storage, analysis, and processing. Remote operators can access the remote-control subsystem to perform real-time monitoring, control management, and other relevant functions related to the front-end camera. In this setup, the deployment of mobile edge computing (MEC) units at the edge of base station becomes crucial for enhancing the Quality of Experience (QoE), especially due to the limited computing power of 5G terminals. By offloading computing tasks to edge units, faster and more efficient processing can be achieved [34]. This optimization resource utilization not only reduces latency but also improves responsiveness, thus enhancing the overall user experience. The flow of data transfer is depicted in Figure 5. The vehicle terminal offloads some computing tasks to MEC, reducing the total computation time through collaborative work.

The processing of each task involves two stages. First, the task is handled locally on the vehicle terminal, and then the second part is offloaded to an edge computing unit. Once the calculations are completed, the edge computing unit sends the results back to the vehicle. The system operates under two scenarios, which are depicted in Figure 6, where BS 1, BS 2, MEC 1, and MEC 2 represent the neighboring base stations and their corresponding edge units. T₁ and T₂ denote the respective instants when this task starts and completes.

• If the vehicle remains within the coverage range of the original base station, the results can be directly returned to the vehicle.
• If the vehicle enters the coverage range of another base station, coordination between base stations is necessary to ensure the results are sent back to the vehicle.

Assuming the data size for a single task is $B$, and the local processing ratio is $\mu$, the amount of data offload is $(1-\mu )B$. Since the local computation and offloading processes are parallel, the time required to complete this task can be expressed as follows:

$$T_{task}=max(T_{local},T_{offload}+T_{mec})$$

(4)

where $T_{task}$, $T_{local}$, $T_{offload}$, and $T_{mec}$ indicate the total computing time, local computing time, offloading time, and edge computing time, respectively. The respective formulas for $T_{local}$, $T_{offload}$, and $T_{mec}$ are as follows:

$$\begin{array}{cc}\hfill T_{local}& =\alpha \mu B/f_{local}\hfill \\ \hfill T_{offload}& =\beta (1-\mu )B/R\hfill \\ \hfill T_{mec}& =\alpha (1-\mu )B/f_{mec}\hfill \end{array}$$

(5)

where $\alpha$ is a constant determined by computational complexity, and $\beta$ is a constant for the overhead of uplink transmission, $f_{local}$ and $f_{mec}$ are the computational powers of the local and edge unit, respectively, and $R$ is the uplink speed. If there are $k$ tasks in total, then the average latency can be expressed as follows:

$$T_{avg}={\sum }_{i=1}^k{T}_{task,i}$$

(6)

It can be observed that the average latency $T_{avg}$ is influenced by factors such as the offloading rate, computation allocation, and uplink speed. It can be optimized by optimizing the offloading rate to achieve the best performance in specific scenarios.

3.4. Video Transmission Strategies

The video is produced using the visual retention principle [35] of the human eye, which essentially involves playing a series of pictures. Through video compression technology, the vehicle-mounted video transmission system eliminates duplicate information contained in consecutive frames at the transmitting end (vehicle video transmission 5G terminal) and restores it at the receiving end (remote monitoring subsystem). The H.264 video compression encoding [36,37,38] includes three types of frame data: I, P, and B, which together form a picture group. Here, the I-frame is a key frame containing all the information and can be independently decoded. P-frames, also known as inter-frame predictive encoding frames, require referencing the previous I-frames for encoding. B-frames, also known as bidirectional predictive coding frames, record the difference between this frame and the previous and subsequent frames. By utilizing H.264 video compression technology and its I-, P-, and B-frame structures, in-vehicle video transmission provides highly reliable and low-latency guarantees from three dimensions:

Generally, there is a direct relationship between latency and network utilization:

$$D=D_0/(1-U)$$

(7)

where $D_0$ is the latency when the network is idle, $U$ is the network utilization, and $D$ is the current latency. When network utilization exceeds 50%, the latency will increase rapidly. Once the utilization rate approaches 100%, the latency will approach infinity. The original data size of 4K video per 1 s (resolution: 3840 × 2160, minimum frame rate: 25 fps, 24-bit true color) is as follows: 3840 × 2160 × 24 bit × 25/(1024 × 1024 × 8) = 593.27 MB. Direct transmission would consume a large number of network resources, or even cause crashes. By using H.264 video compression technology for vehicle video transmission, a compression ratio of up to 100:1 can be achieved, significantly reducing the amount of data transmitted end-to-end, reducing network utilization, and providing low-latency protection.

To improve user perception, the 5G core network can differentiate service categories and implement differentiated resource scheduling strategies. However, due to the broad classification of services on the core network side, conventional solutions struggle to accurately locate video transmission services. By utilizing special flag bits in I-frame data from H.264 video compression coding, the 5G system can accurately identify video transmission services on the wireless side (5G base station) and provide priority scheduling and resource allocation, thereby offering targeted low-latency guarantee.

By utilizing the I-, P-, and B-frame structure of H.264, the vehicle video transmission system can ensure low-latency characteristics through dynamic frame chasing. In situations of network congestion, the system prioritizes dropping B-frames based on frame decoding priority (I > P > B). If the bandwidth remains limited, P-frames are appropriately dropped. As each frame group is transmitted from the I-frame, in case of stuttering, all current frames are directly discarded. However, when the next I-frame is received, a new frame group is sent by skipping the discarded frames, as depicted in Figure 7.

4. Experiments and Discussion

4.1. Experimental Setup

In a coal mine in northwestern China, the full deployment of 5G networking equipment and the underground coverage of 5G signals has been achieved, setting the stage for validating high-definition video transmission for remote-controlled unmanned driving. This experiment takes place in a 2 km long transport tunnel, with a width of about 5 m and a height of approximately 4 m. The deployment of the 5G base station and antenna is illustrated in Figure 8. The RF transmission power is limited to 6 watts due to safety regulations in underground coal mines, resulting in a reduced coverage radius for the base station compared to surface scenarios. Consequently, the 5G base stations are deployed at intervals ranging between 350 and 400 m.

The 5G super uplink band operates with a sub-carrier interval of 30 kHz and provides a time domain resource granularity of 0.5 ms, allowing for control of the air interface’s end-to-end latency within 7 ms. Taking into account the decoding process, which typically takes around 5 ms, the end-to-end latency can be controlled to achieve a theoretical value of 12 ms. The incorporation of 2.1 GHz resources with a lower frequency enables an expanded coverage area, resulting in improved transmission rates in edge coverage areas when compared to the 3.5 GHz band. This approach also helps in reducing the cost of network construction [39].

The peak rate test employs data encapsulation using the File Transfer Protocol (FTP). In the high-definition video transmission test [40,41], 7-channel 4K high-definition cameras are connected to the 5G terminal in the mine for data aggregation. Dual-band super uplink technology is utilized to establish a connection with the mine 5G system. The video stream is decoded at the edge of the platform and then transmitted in real-time to the remote monitoring subsystem.

4.2. Results and Analysis

By utilizing the 5G wireless channel, with a 100 MHz bandwidth at the 3.5 GHz band and 40 MHz at the 2.1 GHz band, the uplink peak rate can reach 418.5 Mbps, which is slightly lower than the theoretical value of 450 Mbps. Figure 9 illustrates the average uplink rates under different received signal powers and compares two scenarios: utilizing the 3.5 GHz band without super uplink and employing super uplink, which includes the 3.5 GHz and 2.1 GHz bands. The results clearly demonstrate that the adoption of super uplink significantly enhances the upload capability of the mine 5G system, providing a crucial foundation for high-definition video transmission in unmanned driving. Additionally,

Table 2. Speed test results for 1080P and 4K video transmission.

Video Resolution	Single Stream Rate	Average Rate
1080P	2 Mbps	13.8~14.5 Mbps
4K	6 Mbps	39.4~43.2 Mbps

presents the speed test results for 1080P and 4K video transmission, showcasing their capability to fully support unmanned mining applications.

By adjusting the local processing ratio from 0 to 100%, the average latency was tested under two scenarios: returning within the same base station and returning across base stations, as illustrated in Figure 10. It is evident that when transmitting 1080P video, the average latency is minimized at a 40% local processing ratio, achieving a controlled latency of 11.8 ms. Similarly, for transmitting 4K video, the average latency is minimized at a 30% local processing ratio, resulting in a controlled latency of 18.3 ms. Both of the results mentioned above are less than 20 ms, meeting the demands of video transmission for unmanned driving in mines.

Thanks to the improvement in information transmission, the downlink latency can now be controlled within 20 ms, which is essential for remote control.

Compared to previous schemes, our solution substantially enhances high-definition video transmission for unmanned driving in mines, effectively fulfilling the demands for low latency and multiple channels. However, the high utilization of transmission resources may pose a challenge by competing with other systems in the mine, a concern that will be promptly addressed as the mine infrastructure evolves. We conducted experiments on remote-controlled unmanned driving, as depicted in Figure 11. To avoid encroaching on other network services, only five out of seven video streams were utilized during the experiment.

5. Conclusions

Recognizing the need for real-time transmission of multi-channel video and uplink transmission of vehicle surrounding environment monitoring in mining unmanned driving, we have compiled and organized video transmission resolution, transmission rate, and latency indicators for remote monitoring in mining unmanned driving. Based on this, we propose an unmanned vehicle video transmission technology scheme that utilizes 5G super uplink, collaborative computing, and video compression.

This scheme leverages the 3.5 GHz and 2.1 GHz super uplink transmission, resulting in significant improvements in the uplink transmission rate and reduced transmission latency. It incorporates task offloading and edge computing techniques to address the limitations of terminal computing power, mitigating the delays caused by computation power, and employs video compression strategies to enhance low-latency performance.

Furthermore, we conducted an unmanned vehicle video transmission experiment utilizing 5G super uplink technology. The results of the experiment demonstrate that when transmitting seven channels of 4K high-definition video, the uplink transmission rate matched the theoretical rate, and the average latency met the requirements for unmanned remote control in mining applications.

Looking ahead, high-definition video will be seamlessly integrated with data such as LiDAR point clouds, enhancing the perception dimensions and accuracy of the vehicle’s surroundings, thus providing reliable perception data for unmanned driving in mine.