Path Loss and Auxiliary Communication Analysis of VANET in Tunnel Environments

Abstract

Vehicular ad hoc network (VANET) communications face severe fading problems due to the signal reflections and diffractions within tunnels. Unlike the open road, the space of a tunnel is very limited, so VANET communication performance in a tunnel is seriously affected. In the process of signal transmission, the reflected signal is symmetrical with the incident signal after it is reflected by the road and the wall. In this paper, we establish a mathematical model of path loss for V2V (Vehicle-to-Vehicle) communication based on the principle of signal reflection symmetry in tunnels and considering several factors, such as the tunnel surface and the color of the tunnel wall. In addition, we use cooperative communication to form a virtual multiple-input multiple-output (V-MIMO) system, to improve the communication quality in tunnels. In the proposed system, the OBU (On-Board-unit) and RSU (Road-Side-Unit) share each other’s antennas, so that wireless cooperative communication can be employed, without increasing the number of antennas in a one-way tunnel. Therefore, this multipath fading internal electromagnetic wave propagation model can be used to improve performance. A deep reinforcement learning algorithm was used to solve the pairing problem to obtain a more accurate OBU and RSU pair, to form a V-MIMO system. Here, the RSU is regarded as an agent and interacts with the OBU in the tunnel. The optimal strategy was learned in a real-time changing simulation environment, and the experiment verified the convergence of the algorithm. The simulation results showed that, compared with the Q-learning based scheme, the optimal matching algorithm based on V-MIMO and a DQN (Deep Q-network) could effectively reduce the probability of transmission outages and improve the communication efficiency in tunnels.

1. Introduction

With the development of science and technology, the latest mobile communication technology defines three major technical scenarios: enhanced mobile Internet, massive machine-type communication, and high-reliability and low-latency communication. The VANET (vehicular ad hoc network) is an important application in high-reliability and low-latency communication scenarios [1]. A vehicular ad hoc network can be divided into three parts: the intra-vehicle network, the inter-vehicle network, and the in-vehicle mobile Internet. In the intra-vehicle network, the vehicle communicates with the mobile terminal through the built-in onboard unit (OBU), realizing mutual communication between the vehicle and the driver [2,3,4]. In the inter-vehicle network, vehicles transmit messages device-to-device (D2D) within a certain range, realizing information sharing between vehicles [5,6]. In the in-vehicle mobile Internet, the base station connects the road system and the Internet, and acts as a relay for the interconnection between vehicles and the Internet, to provide services [7,8].

In recent years, with the continuous development of the world economy, the number of cars has continued to grow. On the one hand, the significant increase in the number of cars has facilitated people’s daily travel; on the other hand, the increasing number of cars on the road has brought huge challenges to the daily operation of the transportation system. The carrying capacity of ordinary roads has reached saturation in many cities, and in this context, fully developing highway tunnel traffic has become one of the effective ways to alleviate the traffic pressure for certain vehicles. Tunnels are engineering buildings buried in the ground, and unlike open spaces on land, the space of tunnels is very limited, so the traffic problems of tunnel vehicles are different from road traffic. VANET technology plays an extremely important role in addressing road traffic issues in tunnel scenarios. In this network, the vehicle is treated as an independent node equipped with specific communication devices, and it has communication, computing, storage, sensing, and control capabilities. Real-time interaction between vehicles and the road can be achieved within a tunnel. Unlike for urban highway scenarios, scholars have paid little attention to tunnel-related traffic problems. The information transmission in a tunnel environment with V2X (Vehicle-to-Everything) can be divided into line-of-sight (LOS) propagation and non-line-of-sight (NLOS) propagation [9]. Moreover, the signal transmission in a tunnel is restricted in a certain space limited by the tunnel wall, the ground, and the top wall [10]. Thus, the space is narrow and unique. On account of the special characteristics of the tunnel environment, there are many reflections and diffractions, so V2V communication has a large number of paths, in addition to the simplest LOS propagation path. Therefore, the received signal is composed of multipath signals. These propagation paths of signals are different in length and direction, which leads to a varying time, amplitude, and angle of arrival at the receiving antenna. After being superimposed at the receiving vehicle, in-phase superposition may occur and enhance the signal strength, or anti-phase superposition may occur and weaken the signal. Under these circumstances, the amplitude of the received signal changes drastically, resulting in multipath fading. This multipath fading seriously deteriorates the quality of the received signal, affects the reliability of the communication, reduces the coverage of the communication, and restricts the V2X signal transmission in a tunnel. These are the issues that affect VANET communication in a tunnel. Therefore, it is necessary to conduct in-depth research on the characteristics and rules of path loss in tunnels [11], such as the propagation mechanism in a tunnel environment.

In the process of VANET communication between vehicles in a tunnel, the topology of VANET has the characteristics of high-speed dynamic change, which is mainly manifested in the following aspects. First, the density distribution of network nodes (vehicles) is different with different times and locations [12]. Second, the network topology is not fixed, due to the uneven movement of network nodes [13]. Thus, the wireless channel is unstable, and the real-time transmission of messages by VANET is greatly affected by the signal propagation path when a vehicle is driving in a tunnel [14]. The tunnel environment is complex, and research about the signal propagation in tunnels is relatively scarce. There are still many problems that need to be deeply explored and studied.

At present, signal propagation models include the Okumura-Hata model [15], COST-231 Hata model [16], LEE model [17], CCIR model [18], Egli model [19], Longley-Rice Model [20], and Kriging model [21]. The parameters in the Okumura Data model are easy to obtain and use. However, significant factors such as the height and density of buildings, as well as the distribution of streets, were not considered, resulting in significant errors between the predicted and actual values [15]. The COST-231 Hata model can be used to estimate the path loss of cellular communication in urban environments [16]. The LEE model is suitable for measuring data, and the main parameters are easy to obtain and adjust, with high accuracy, a simple algorithm, and fast calculation [17]. The work in [20] used the Longley Rice model to predict the wireless signal propagation of rural railways, and the predicted results were compared with the measurement results using actual instruments. In addition, comparisons were made with the predictions of the Okumura Data model. The results demonstrated that the Longley Rice model is suitable for railway communication prediction with irregular terrain. The study in [19] successfully established an optimized path loss model in a mountainous environment with a frequency of 2100 MHz 3G UMTS. The results indicated that the COST-231 Hata model performed better in predicting path loss in rural and urban environments than the Hata, Egli, and ECC-33 models, and the optimized Egli model performed best in suburban areas. The study in [21] utilized a Kriging model for predicting the path loss in the very high frequency (VHF) band, using geostatistical methods. The prediction accuracy of path loss based on the widely established empirical path loss propagation model was relatively evaluated. The results revealed the applicability of geostatistical methods for predicting path loss. Each signal propagation model has a different scope and applicable environment, so these propagation models cannot be directly applied to a tunnel environment.

In most countries, tunnels generally use asphalt or concrete pavements [22]. Considering the safety issues in tunnels, high temperatures will cause serious damage to vehicles and internal facilities [23,24], so fireproof coatings are applied around tunnel walls, and different fireproof coatings and road surfaces have different influences on the signal reflections and diffractions [25]. These influences should be considered.

Multiple-input multiple-output (MIMO) communication systems utilize multiple spatially separated antennas for multiplexing and diversity gain [26]. Relatively independent signal paths can be obtained through independent channel gain matrices, which improve the spectral efficiency [27]. After the signal is transmitted in a multipath manner, it can resist multipath fading by being received and combined by multiple antennas at the receiver. Although the use of multi-antenna technology brings attractive performance gains, the cost cannot be ignored. On the one hand, multi-antenna technology requires increasing the number of radio frequency modules, thereby increasing the equipment cost. On the other hand, adding antennas will also increase the volume and power consumption of the device, as well as the cost. In order to ensure that the transmission channels are not correlated, it is necessary to ensure that the antenna spacing is greater than the maximum spacing of the channels [28].

Virtual MIMO technology (V-MIMO), which has been deeply studied for wireless sensor networks, can solve some problems of the traditional MIMO technology. V-MIMO technology utilizes multiple single antenna terminals, to cooperate with each other to form an effective antenna array [29]. After receiving the information forwarded by the cooperative partner, the terminal combines the signals to achieve spatial diversity gain. Traditional MIMO communication technology can be divided into single input single output (SISO), single input multiple output (SIMO), multiple input single output (MISO), and multiple input multiple output (MIMO) according to the configuration and number of transceiver antennas [30,31]. Similarly, in V-MIMO communication, there is also such a division according to the number of cooperative nodes participating. V-SISO means that when only the source node transmits, only the destination receives. V-MISO means that when the source node and its surrounding users transmit information at the same time, only the destination receives it. V-SIMO means when the source node alone transmits, the destination node and its cooperating nodes receive information at the same time. V-MIMO means that when the sending and receiving nodes coexist in cooperating nodes, they send and receive information at the same time.

As a technology for interconnecting thousands of vehicles, a VANET can use V-MIMO and combine artificial intelligence technology to improve the communication performance of vehicles. Reinforcement learning is one of the main topics in the field of artificial intelligence, and this includes several parts, such as the agent, environment, action, and reward [32]. In general, the model defines the main body of decision making as the agent, and the factors that can affect the agent’s decision making are referred to as the surrounding conditions. The agent can accumulate learning experience by interacting with the surrounding conditions continually [33]. In the interaction process, the agent can choose a suitable action according to a learned strategy based on the current state. The environment gives certain feedback according to the action selected by the agent in a certain state. The feedback gives a reward in the environment. The reward is achieved by continuously adjusting the strategy to maximize the long-term benefits. Intelligent agents can accumulate experience by using environmental perception and action interaction, continuously utilizing learned experience to finish the designated task. Therefore, the core of reinforcement learning is the interaction between the subject and the environment through continuous learning processes [34]. The agent learns strategies and makes beneficial choices through the information fed back by the environment. The Q-Learning algorithm, as one of the classic reinforcement learning algorithms and mainly solves reinforcement learning problems by establishing tables to store Q-values. DQN is a combination of Q-Learning and neural networks. The neural network obtains Q-values for corresponding behaviors through input state and behavior analysis of the network structure, which directly replaces the generation of Q-tables in Q-Learning and improves the efficiency of reinforcement learning [34].

The contributions of this work can be summarized as follows:

(1)

We propose a new path loss calculation scheme that can be used for information transmission between vehicles in tunnels. In the proposed scheme, not only the factors of the road and tunnel wall are considered, but also the differences in reflection coefficient between the road materials and tunnel wall fireproof coatings, which can better improve the accuracy of vehicle information transmission.

(2)

We propose a solution based on a reinforcement learning algorithm to improve the efficiency of vehicle RSU collaboration, to solve the problem of poor communication performance between vehicles in tunnels and improve the information transmission efficiency. By utilizing V-MIMO technology, vehicles can share their own data with each other, and vehicles and RSUs can also collaborate to transmit data.

The structure of the article is as follows: The first chapter introduces the relevant theories, and then the second chapter is divided into three parts. The first part is about the calculation of path length and loss, the second part is about the analysis of the V-MIMO model applied to vehicles, and finally the third part is an analysis of the simulation results.

2. System Model

2.1. Analysis of Path Loss in a Tunnel

2.1.1. Path Length

In order to improve the accuracy of vehicle information transmission in a tunnel, it is necessary to design a more suitable and accurate signal propagation model, the following assumptions were made before proposing the system model.

First, suppose the road surface in the tunnel is concrete or asphalt. Considering the particulars of the tunnel, the top of the tunnel is partly coated with fireproof paint, and the color of the fire retardant paint is black or white.

Second, suppose the speed of vehicles is constant and the moving direction is the same when driving inside the tunnel. The height of each vehicle is $H_2$ = 1.5 m. The transmitter and receiver are located in the center of the vehicle and keep in the same horizontal position. The euclidean distance between the two communication vehicles is within 10 m.

Then, suppose the top of the tunnel wall and the ground are flat, and the signal reflected from the top or the ground is regarded as specular reflection [34].

Finally, due to the particularity of the tunnel environment, there are countless indirect paths for $OB{U}_1$ to transmit signals to $OB{U}_2$, as shown in Figure 1. In this closed environment, this space is composed of countless two-dimensional planes, and the signal propagates in each two-dimensional plane, so we can intercept a plane perpendicular to the ground for research.

Shown in Figure 1 is the model proposed in this paper, where both the incident and reflected rays are on a two-dimensional plane. When the two vehicles drive at a constant speed and are relatively stationary, the signal propagation between $OB{U}_1$ and $OB{U}_2$ should also be in a two-dimensional plane. There are two paths for the signal propagation of the vehicle in the tunnel. The first is the direct path marked by the red line in Figure 1, where $OB{U}_1$ directly transmits the signal to $OB{U}_2$. The second is the indirect path, which is divided into two types. One is where the propagating signal passes through the ground and then reflects to $OB{U}_2$, as shown by the yellow line in Figure 1. The other is where the propagating signal passes through the top of the tunnel wall and then reflects to $OB{U}_2$, as shown by the blue line in Figure 1 [35]. The green line represents the ground reflection first, then the tunnel top reflection. The violet line represents the reflection from the top of the tunnel first and then from the road surface.

The following is a detailed analysis of the signal propagation path according to the difference in the number of signal reflections.

(1)

One-time reflection path

As shown in Figure 2, $H_1$ is the height of the tunnel, $H_2$ is the height of the car, and d is the length of the direct propagation path of the signal between OBUs, which directly reaches the receiver without any reflections. $D_t{}_w$(1) is the length of the indirect propagation path between vehicles, which reaches the receiver through tunnel wall reflection, and $D_t{}_g$(1) is the length of the indirect propagation path between vehicles, which reaches the receiver through ground reflection. According to the geometric relationship presented in Figure 2, $D_t{}_g$(1) and $D_t{}_w$(1) can be obtained:

$$\begin{array}{c}\hfill D_{tg}\left(1\right)=\sqrt{{\left(2H_2\right)}^2+d^2}\end{array}$$

(1)

$$\begin{array}{c}\hfill D_{tw}\left(1\right)=\sqrt{{(2H_1-2H_2)}^2+d^2}\end{array}$$

(2)

(2)

Two-time reflection path

Figure 3 shows the two-time reflections path. The blue line $D_t{}_g$(2) represents the ground reflection first, then by the tunnel top reflection. The red line $D_t{}_w$(2) represents the reflection from the top of the tunnel first and then from the road surface. According to the geometric properties shown in Figure 3, the path lengths for the two-time reflection can be expressed as

$$\begin{array}{c}\hfill D_{tg}\left(2\right)=\sqrt{{\left(2H_1\right)}^2+d^2}\end{array}$$

(3)

$$\begin{array}{c}\hfill D_{tw}\left(2\right)=\sqrt{{\left(2H_1\right)}^2+d^2}\end{array}$$

(4)

(3)

Three-time reflections path

Figure 4 shows the three-time reflection path. The blue line $D_t{}_g$(3) represents the first reflection from the road surface, followed by the reflection from the top, and then the reflection from the road surface, and finally reaching the receiver. The red line $D_t{}_w$(3) represents the reflection first from the top, next the reflection from the road surface, and then the reflection from the top, and finally reaching the receiver. According to the geometric properties of the Figure 4, the following results can be obtained for the three-time reflection case:

$$\begin{array}{c}\hfill D_{tg}\left(3\right)=\sqrt{{(2H_1+2H_2)}^2+d^2}\end{array}$$

(5)

$$\begin{array}{c}\hfill D_{tw}\left(3\right)=\sqrt{{(4H_1-2H_2)}^2+d^2}\end{array}$$

(6)

(4)

n-time refelection path

According to the above Formulas (1)–(6), we can deduce the length of the signal propagation path after n times of reflection:

$$\begin{array}{c}\hfill D_{tg}\left(n\right)=\sqrt{{\{\{n-[0.5+0.5\times {(-1)}^{n+1}]\}{H}_1+[1+{(-1)}^{n-1}]H_2\}}^2+d^2}\end{array}$$

(7)

$$\begin{array}{c}\hfill D_{tw}\left(n\right)=\sqrt{{\{\{n+[0.5+0.5\times {(-1)}^{n+1}]\}{H}_1+[{(-1)}^n-1]H_2\}}^2+d^2}\end{array}$$

(8)

where $D_t{}_g$(n) represents the path length that starts to be reflected by the road surface, and $D_t{}_w$(n) represents the path length that starts to be reflected by the top of the tunnel. Due to the different reflection coefficients, the total length of the path from the ground to the top of the tunnel is $D_t{}_g{}_d$(n):

$$\begin{array}{c}\hfill D_{tgd}\left(n\right)=\left\{\begin{array}{cc}0.5\times D_{tg}\left(n\right)& n=even\\ \frac{n - 1}{2n}{D}_{tg}\left(n\right)& n=odd\end{array}\right.\end{array}$$

(9)

Reflected from the ground, the total length of the path that has traveled to the road is $D_t{}_g{}_s$(n):

$$\begin{array}{c}\hfill D_{tg\mathrm{s}}\left(n\right)=\left\{\begin{array}{cc}0.5\times D_{tg}\left(n\right)& n=even\\ \frac{n + 1}{2n}{D}_{tg}\left(n\right)& n=odd\end{array}\right.\end{array}$$

(10)

In the same way, starting from the top of the tunnel, the total length $D_t{}_w{}_d$(n) of the path going to the road is:

$$\begin{array}{c}\hfill D_{twd}\left(n\right)=\left\{\begin{array}{cc}0.5\times D_{tw}\left(n\right)& n=even\\ \frac{n - 1}{2n}{D}_{tw}\left(n\right)& n=odd\end{array}\right.\end{array}$$

(11)

Reflected from the top of the tunnel, the total length of the path that has traveled to the top of the tunnel $D_t{}_w{}_s$(n):

$$\begin{array}{c}\hfill D_{tw\mathrm{s}}\left(n\right)=\left\{\begin{array}{cc}0.5\times D_{tw}\left(n\right)& n=even\\ \frac{n + 1}{2n}{D}_{tw}\left(n\right)& n=odd\end{array}\right.\end{array}$$

(12)

2.1.2. Path Loss Calculation

For a directly propagated signal, the signal power can be expressed as [9]

$$\begin{array}{c}\hfill P_L=\sqrt{\left(G_t{G}_r\right)}\frac{\lambda }{4\pi }{\mathsf{\Gamma }}_{}\left|\frac{e^{(-j2\pi d/\lambda )}}{L}\right|\end{array}$$

(13)

where $G_t$ and $G_r$ are the gains of the $OB{U}_1$ and $OB{U}_2$ antennas, and $\mathsf{\Gamma }$ is the reflection coefficient, $\lambda$ is the wavelength, and L is the total length of the path.

For a signal that is reflected n times from the road ground, the signal power can be calculated as

$$\begin{array}{c}\hfill P_{D_{tg}}=\sqrt{\left(G_t{G}_r\right)}\frac{\lambda }{4\pi }({\mathsf{\Gamma }}_1\left|\frac{e^{(-j2\pi D_{tgs}/\lambda )}}{D_{tgs}}\right|+{\mathsf{\Gamma }}_2\left|\frac{e^{(-j2\pi D_{tgd}/\lambda )}}{D_{tgd}}\right|)\end{array}$$

(14)

${\mathsf{\Gamma }}_1$ is the reflection coefficient of the signal passing through the road surface, and ${\mathsf{\Gamma }}_2$ is the reflection coefficient of the signal passing through the top of the tunnel.

Similarly, for the signal that starts to reflect n times through the top of the tunnel, the signal power can be calculated as:

$$\begin{array}{c}\hfill P_{D_{tw}}=\sqrt{\left(G_t{G}_r\right)}\frac{\lambda }{4\pi }({\mathsf{\Gamma }}_1\left|\frac{e^{(-j2\pi D_{twd}/\lambda )}}{D_{twd}}\right|+{\mathsf{\Gamma }}_2\left|\frac{e^{(-j2\pi D_{tws}/\lambda )}}{D_{tws}}\right|)\end{array}$$

(15)

Thus, the total power of signal transmission is equal to the signal power of direct propagation plus the signal power of indirect propagation, which is presented in the following formula:

$$\begin{array}{c}\hfill P_{total}=P_L+\sum _{i=1}^n(P_{D_{tg}}+P_{D_{tw}})\end{array}$$

(16)

The path loss when arriving at the OBU is

$$\begin{array}{c}\hfill L_{OBU}=10lg{\left|\frac{P_{total}}{P_L}\right|}^{-1}\end{array}$$

(17)

According to the ITU-RP.1238 propagation model [36], the path loss for the tunnel free space transmission can be written as

$$\begin{array}{c}\hfill L_{tf}=20lgf+20lgd-28\left(dB\right)\end{array}$$

(18)

Therefore, the total path loss of the vehicle in the tunnel is expressed as

$$\begin{array}{c}\hfill L_{\mathrm{sum}}=L_{tf}+L_{OBU}\end{array}$$

(19)

2.2. V-MIMO Model in a Tunnel

2.2.1. The Probability of Successful V2R (Vehicle-to-RSU) Transmission in a Tunnel

The V-MIMO system is composed of several vehicles and RSUs that support receiving and transmitting terminals. The adjacent relay transmitting and receiving terminals form a multi-level distributed V-MIMO system. When transmitting information, the transmitting vehicle shares the information to be sent with other wireless terminals that form multiple inputs within a certain range, ensuring that other terminals within the multiple inputs will contain copies of the sent data. Then, all wireless terminals with multiple inputs unite to send the data to the next multi-input terminal, continuously engaging in this behavior and transmitting the information until the destination terminal receives the signal sent by the source terminal.

Within Figure 5, Figure 5a shows a physical image of the vehicle driving in the tunnel. In order to more intuitively express the state of the vehicle and RSU, Figure 5b shows a simulated image of a vehicle driving in the tunnel. As shown in Figure 5b, it is assumed that a pair of RSUs are deployed on the two sides of the tunnel wall. Suppose the RSUs can transmit their own messages and receive safety messages from the automotive node. One is the agent-RSU, abbreviated as A-RSU, which is a device with the capabilities of communication, computing, storage, automatic control, and so on. The other is an ordinary RSU. The height of the two RSUs from the ground is $H_3$ meters, as shown in Figure 5b.

For the communication transmission link established by a RSU and the vehicle user, if the free space path loss model or the shadow fading model is used, it is difficult to reflect the attenuation changes in the signal with distance and obstacles at the same time. Therefore, following the literature, a mixed channel model of line-of-sight and non-line-of-sight is used to reflect signal changes. Assuming that the distance between any OBU within the RSU signal range detection and the projected position of the RSU on the ground is $d_2$, as shown in Figure 5b, the received power gain obtained by the OBU from the LOS and the NLOS respectively is as follows [37]:

$$\begin{array}{c}\hfill G_{\mathrm{r}}=\left\{\begin{array}{cc}{\sqrt{{\left(H_3\right)}^2+d_2{}^2}}^{-a_r}& LOS\\ \eta {\sqrt{{\left(H_3\right)}^2+d_2{}^2}}^{-a_r}& NLOS\end{array}\right.\end{array}$$

(20)

where $a_r$ is the path loss index of V2R transmission, and $\eta$ is the additional attenuation coefficient of NLOS transmission. $H_3$ refers to the distance from the A-RSU antenna position to the ground. Assume that the RSU transmit power is $P_t$. According to the channel transmission power formula, the received power of the OBU can be obtained:

$$\begin{array}{c}\hfill P_{re}=P_t{G}_r\end{array}$$

(21)

Due to the influence of the different positions of the OBU and the height of the RSU in the tunnel environment, the probability that the OBU can establish communication with the RSU through LOS transmission is [37]

$$\begin{array}{c}\hfill P_{LOS}=\frac{1}{1+Aexp(-B[arctan\frac{H_3}{d_2}-A])}\end{array}$$

(22)

where A and B are constants determined by the tunnel environment.

Correspondingly, the probability that the OBU can establish a transmission with the RSU through the NLOS transmission link is

$$\begin{array}{c}\hfill P_{NLOS}=1-P_{LOS}\end{array}$$

(23)

Considering the two transmission situations that actually exist in tunnel communication, the total channel power obtained by the OBU can be expressed as [38]

$$\begin{array}{c}\hfill P_{sum}=P_{LOS}.P_{re\left|LOS\right.}+(1-P_{LOS})P_{re\left|NLOS\right.}\end{array}$$

(24)

Assuming that $P_n{}_o$ is the noise power, then we obtain the signal-to-noise ratio(SNR) of the OBU:

$$\begin{array}{c}\hfill SNR=\frac{P_{sum}}{P_{no}}\end{array}$$

(25)

Let the SNR of any OBU transmitted through V2R be ${\gamma }_1$, and ${\gamma }_1$ can be obtained from Equation (25) above. According to the literature [37], the SNR threshold for successfully decoding the message received by the OBU in the LOS environment is ${\gamma }_2$, and the SNR threshold in the NLOS environment is ${\gamma }_3$. When the SNR value is greater than the SNR threshold, the OBU can pass the transmission. Therefore, for the OBU, the probability of successful transmission is expressed as

$$\begin{array}{c}\hfill \begin{array}{c}{P}_{success}=P_{LOS}Pr({\gamma }_1\ge {\gamma }_2)+P_{NLOS}Pr({\gamma }_1\ge {\gamma }_3)\\ =P_{LOS}Pr(d_2\le \sqrt{{\left(\frac{P_t}{{\gamma }_2{P}_{no}}\right)}^{\frac{2}{a_r}}-{\left(H_3\right)}^2})+P_{NLOS}Pr(d_2\le \sqrt{{\left(\frac{P_t}{{\gamma }_3{P}_{no}}\right)}^{\frac{2}{a_r}}-{\left(H_3\right)}^2})\end{array}\end{array}$$

(26)

2.2.2. Analysis of V-MIMO Transmission in Tunnels

(1)

V-MIMO Case

As shown in Figure 6, when the vehicle is driving in the tunnel, a V-MIMO system is formed by using cooperative communication to share each vehicles antennas with the two RSUs in the tunnel, improving the wireless communication quality. That is, the tunnel wireless cooperative communication can improve the channel capacity using the multipath fading of electromagnetic wave propagation in the tunnel, without increasing the number of antennas for each node, and thereby improving the wireless communication performance of vehicles in the tunnel.

In the Figure 6, the transmitter is a virtual multi-input, composed of $OB{U}_2$ and A-RSU; the receiver is a virtual multi-output, composed of $OB{U}_1$ and RSU; and a virtual MIMO system is formed. The signal transmission is mainly multipath fading, so it can be treated as a Rayleigh fading channel; and assuming that the CSI (channel state information) is unknown, then the capacity of the virtual MIMO system is obtained as [39]

$$\begin{array}{c}\hfill C_1={log}_{2}det(I_2+\frac{SNR}{2}H{H}^H)\mathrm{bps}/\mathrm{Hz}\end{array}$$

(27)

where $I_2$ is the second-order matrix, H is the channel matrix, $H^H$ is the conjugate transposed channel matrix, $H{H}^H$ is the semi-positive definite Hermitian matrix, and $det(\ast )$ represents the determinant of the correlation matrix. In the virtual MIMO environment, the outage probability of the V2V communication link is $P_o{}_u{}_t$, which is obtained using the following:

$$\begin{array}{c}\hfill P_{out}=\frac{k^m}{m!}{e}^{-k}{(1-\frac{C_j}{C_{max}}{P}_{success})}^k(j=1,2,3\cdots n)\end{array}$$

(28)

k is the maximum number of OBUs that the A-RSU can handle within a certain range at time t, and m is the actual traffic flow. $C_j$ is the actual channel capacity, and $C_m{}_a{}_x$ is the maximum channel capacity that can be achieved under the current environment.

(2)

V-SIMO Case

As shown in Figure 7, if there is only one vehicle driving in the tunnel, it needs to communicate with the external base station, and it transmits signals with the RSU. In this case, the $OB{U}_1$ is used as a virtual single input, and then the two RSUs are paired with each other as virtual multiple outputs, to form a virtual SIMO system. In this case, the capacity of the system is

$$\begin{array}{c}\hfill C_2={log}_{2}det(I_2+SNR\times H{H}^H)\mathrm{bps}/\mathrm{Hz}\end{array}$$

(29)

(3)

V-MISO Case

As shown in Figure 8, if there is only one vehicle driving in the tunnel, the external base station needs to transmit information using this vehicle. The base station and RSU first transmit the signals, and then the RSU sends messages to the $OB{U}_1$. At this time, the two RSUs are paired with each other as a virtual multi-input system, and the $OB{U}_1$ is used as a virtual single output, to form a virtual MISO system. At this time, the capacity of the system is

$$\begin{array}{c}\hfill C_3={log}_{2}det(I_1+\frac{SNR}{2}H{H}^H)\mathrm{bps}/\mathrm{Hz}\end{array}$$

(30)

where $I_1$ is the first-order matrix.

(4)

V-SISO Case

As shown in Figure 9, if there is only one vehicle driving in the tunnel and only one RSU communicates with the OBU, the RSU is used as a virtual multi-input. Meanwhile, the $OB{U}_1$ is used as a virtual single output to form a virtual SISO system. At this time, the capacity of the system is

$$\begin{array}{c}\hfill C_4={log}_{2}det(I_1+SNR\times H{H}^H)\mathrm{bps}/\mathrm{Hz}\end{array}$$

(31)

2.3. Application of Deep Reinforcement Learning Models in Tunnels

Since the path loss between vehicles in the tunnel is relatively large, in order to better use the V-MIMO to improve the communication quality, the A-RSU requires real-time interactive decision-making with the environment. To improve the V2V communication quality, a deep-Q network using deep reinforcement learning theory, that is, the DQN method, is used to analyze the V2V communication path loss by building a reinforcement learning framework. In V2V scenarios, the current state of the system is only related to the state and action of the previous moment. Thus, this process can be viewed as a Markov decision process (MDP), which can be expressed as

$$\begin{array}{c}\hfill MDP=(s_i,a_i,p_i,r_i)\end{array}$$

(32)

where $s_i$ represents the state set, $a_i$ represents the action set, $p_i$ represents the state transition probability, and $r_i$ represents the reward.

In this paper, V-MIMO technology is combined with a deep reinforcement learning algorithm DQN (named V-DQN) to obtain a better performance. The A-RSU deployed on one side of the tunnel wall is used as the learning agent in the DQN. The number of OBUs and the path loss of V2V within the detection range of the RSU signal are set as the environment. The framework of the A-RSU interacts with the continuous environment and makes decisions during the entire reinforcement learning process, as shown in the Figure 10. At time T, the A-RSU observes a current state from the environment, and then the A-RSU takes the corresponding actions based on policy $\pi$, according to the observed state. The policy $\pi$ is determined by the state-action value function; that is, the strategy function Q in reinforcement learning and the corresponding Q value is obtained, and the Q value is measured by the deep learning part. According to the behavior of the RSU, in the next slot, the environment moves to the next state $s_i{}_{+}{}_1$, and the A-RSU obtains a gain from the environment.

The three key elements of the optimal matching algorithm based on V-DQN are the state space, action space, and reward function.

The number of vehicles $n_{{}_o}$ and the path loss between vehicles $L_s{}_u{}_m$ present in (19) are used as the state space of the deep reinforcement learning algorithm, denoted as:

$$\begin{array}{c}\hfill s_i=\{L_{sum},n_o\},n_o=\{0,1,2,3\dots \dots S\}\end{array}$$

(33)

For the action space, there are four cases when the A-RSU faces tunnel vehicle communication. First, only two vehicles communicate within the monitoring range. The transmit power of auxiliary V2V communication is $P_t$. If the vehicle communication effect is not good, the A-RSU interferes with the RSU and induces the RSU to perform auxiliary communication with the following vehicles. This situation is shown in the Figure 11. At this time, the transmission power is $P_t{}_d$. Third, there is no need to communicate between the vehicles, and the vehicle only needs to communicate with the RSU. This situation is shown in Figure 11, and in this case, the A-RSU is connected with the RSU to form a virtual multi-output system, and the transmission power is set as $P_t{}_c$. The last case is where there no vehicles pass through the tunnel, so the A-RSU is in a monitoring state. In this case, the power is $P_t{}_s$. Considering the above four cases, the action space can be written as

$$\begin{array}{c}\hfill a_i=\{P_t,P_{td},P_{tc},P_{ts}\}\end{array}$$

(34)

Suppose the RSU can successfully establish transmission with the OBU, and the correct V-MIMO technology can be used in a certain period of time to maximize the channel. Then, the appropriate capacity probability under the successful transmission can be treated as the reward function, which is expressed as follows:

$$\begin{array}{c}\hfill r_i=E\left(a_i\right|s_i)=P_{success}\frac{C_j}{C_{max}}\end{array}$$

(35)

The deep reinforcement learning algorithm DQN uses the state $s_i$ and behavior parameter as the input of the neural network of ${\Theta }_{{}_n}$, and then analyzes the Q value under different behaviors using a neural network, so as to form a mapping relationship between the state−behavior and the Q value. The estimated Q function is used to replace the above method of updating the Q table, thereby effectively accelerating the learning convergence speed of the agent A-RSU and achieving an efficient learning effect. This uses two convolutional neural networks with the same structure: one is used to estimate the Q value under the current state behavior, and the other is used as the target Q network to update the Q value [40]. Here, the experience replay mechanism is adopted in the DQN method. Setting up a memory replay unit allows storing the experience data obtained by the online interaction between the agent A-RSU and the vehicle communication environment in the tunnel in the process of training the neural network. In this way, the network parameters are updated each time during training, and a small batch of sample data from the memory playback unit is randomly selected and trained [41]. Meanwhile, the stochastic gradient descent method is used to break the correlation between the sample data and make the updating of the neural network more efficient.

For a given state behavior ($s_i$, $a_i$), the Q ($s_i$$a_i$) can be calculated according to the policy $\pi$, which is a measure of the quality of the behavior made at the current moment. Once the Q value of each behavior is known, the agent A-RSU can make appropriate behavior choices in the current state:

$$\begin{array}{c}\hfill {a_i}^{\ast }=argmaxQ(s_i,a_i)\end{array}$$

(36)

The agent A-RSU performs actions that maximize the Q value. In the case that the dynamic information of the tunnel environment cannot be known in advance, the iterative update of the following equation can be used to obtain the optimal Q value under the optimal strategy

$$\begin{array}{c}\hfill Q^{\ast }(s_i,a_i)=Q(s_i,a_i)+\alpha [r_{i+1}+\gamma maxQ(s_{i+1},a_{i+1})-(s_i,a_i)]\end{array}$$

(37)

$\alpha$ represents the learning rate of the agent A-RSU. $\gamma$ represents the reward discount factor of the agent A-RSU.

The target Q network is expressed as

$$\begin{array}{c}\hfill y_i=r_i+maxQ(s_i,a_i,{\theta }_n)\end{array}$$

(38)

The loss function is the mean square error loss function, as follows:

$$\begin{array}{c}\hfill L\left({\theta }_n\right)=E\left[{(y_i-Q(s_i,a_{i,}{\theta }_n))}^2\right]\end{array}$$

(39)

The parameters of the neural network are updated using the gradient descent method:

$$\begin{array}{c}\hfill {\theta }_n={\theta }_n+\nabla Q(s_i,a_i,{\theta }_n)[r_i+\gamma maxQ(s_{i+1},a_{i+1},{\theta }_n)-Q(s_{i+1},a_{i+1},{\theta }_n)]\end{array}$$

(40)

The proposed V-DQN algorithm can be summarized as follows: the agent creates a strategy in the action space, and selects the appropriate power size as the action strategy. After the vehicle user equipment receives the action strategy, the communication module executes the action strategy. Then, the environment gives feedback and uses this feedback to reward or punish the action accordingly. This process is then executed in a loop, until the optimal strategy is found, according to the environment. M represents the maximum training round. T represents the number of loop traversals. The detail description of the proposed Algorithm 1 is shown in the following.

Algorithm 1 Optimal channel matching algorithm based on V-DQN

1: Initialize the memory playback unit D and Q network parameters 2: for  $episode=1:M$  do 3:    Initialize tunnel vehicle environment information and status 4:    for $i=1:T$ do 5:      A-RSU uses a greedy strategy to select $a_i$ according to the vehicle environment state $s_i$ 6:      A-RSU obtains instant reward $r_i$ according to the execution of action $a_i$ 7:      A-RSU transitions to the next state $s_i{}_{+}{}_1$ 8:      Store the data ($s_i$, $a_i$, $r_i$, $s_i{}_{+}{}_1$) in D 9:      Randomly extract a set of empirical data in D ($s_i{}_{+}{}_1$, $a_i{}_{+}{}_1$, $r_i{}_{+}{}_1$, $s_i{}_{+}{}_2$) 10:      $y_i=r_{i}finalstate$ 11:      $y_i=r_i+maxQ(s_i,a_i,{\theta }_n)non-finalstate$ 12:      Use the mean squared loss function to update all parameters of the Q network through gradient backpropagation 13:    end for 14: end for

3. Performance Evaluation

3.1. The Relationship between the Path Loss and the Reflection Times

As shown in Figure 12, a tunnel environment was randomly selected in this article. In the case of white walls and concrete roads, the safety distance of the vehicle remained unchanged. As the number of reflections n increased, the numerical difference was significant when n was within the range of 1 to 3. When n was greater than 3, the curve became more and more stable.

3.2. Analysis of V2V Path Loss under Different Reflections

The simulation was based on Matlab. It simulated a one-way tunnel. The related parameters in the simulation were set as follows: the height of the tunnel was $H_1$ = 8 m, and the height of the OBU was $H_2$ = 1.5 m. The frequency of the VANET radio wave was 5.8GHz, with $\lambda$ = 0.05 m. Considering that the reflection coefficient of the tunnel pavement was different from the reflection at the top of the tunnel wall, this article mainly studied the path of the first three reflections.

When two vehicles communicated in the tunnel, it was assumed that there were n times reflection paths during the communication period. Several different fireproof coatings and road surfaces were studied under different reflection coefficients. Here, the reflection coefficient of the black tunnel wall was 0.01, the reflection coefficient of the white tunnel wall was 0.78, the reflection coefficient of the asphalt road was 0.14, and the reflection coefficient of the concrete tunnel wall and road was 0.31 [42]. The simulation results are shown in the following Figure 13, Figure 14 and Figure 15:

As shown in Figure 13, Figure 14 and Figure 15, when the distance d increased, the path loss inside the tunnel increased. At the same time, as the distance increased, the difference in path loss between the black tunnel wall and the asphalt road became larger and larger compared to the path loss between the white tunnel wall and the cement road. By comparing these three different reflections, we can infer that at the same distance, when the vehicle traveled on the asphalt road and with the black tunnel wall, the signal propagation path loss was the maximum. When the vehicle traveled on the concrete road and with the white tunnel wall, the signal propagation path loss was the minimum.

The standard deviation represents the degree of dispersion of the sample data. Through vertical comparison, we could obtain the reflection times of the signal and the influence of the tunnel environment on the path loss at the same distance. The simulation results are shown in Figure 16.

In Figure 16, when n = 1, the standard deviation increased relatively steadily. Considering the reflections, the weight of the first reflection (that is n = 1) was much higher than the others, as shown in the above Figure 12. In addition, the path loss was also very sensitive to the path length distance between the transmitter and receiver. When the distance was shorter than 4 m, this meant that the direct path contributed the most compared with the other reflection path. Thus, when n = 1 and d was less than 4 m, the curve showed a small fluctuation. When n = 2 or n = 3, the standard deviation rose sharply with distance. The larger the standard deviation, the more discrete the data. On the contrary, the smaller the standard deviation, the more stable the data. Therefore, we can conclude that as the number of reflections increased, the signal became more susceptible to the influence of different environments.

3.3. Analysis of V-MIMO Channel Capacity Simulation in a Tunnel

The simulation results are shown in Figure 17.

According to Figure 17, with the increase of the SNR, the channel capacity of the system also increased. The gap between the channel capacity of V-MIMO and the channel capacity of the other three techniques gradually increased, while the gap between the other three methods was small with the increase of the SNR. Meanwhile, the MIMO channel capacity had the maximum growth rate, so it had the best communication quality. On the contrary, the SISO had the minimum growth rate, thus its communication effect was not good. The reason for this was that the channel capacity was affected by the number of receivers and senders. When both receivers and senders increased, the channel capacity continued to increase. Therefore, when the communication path loss of the two vehicles in the tunnel was relatively large, using the V-MIMO improved the reliability of OBU signal transmission.

3.4. Analysis of Deep Reinforcement Learning

The related simulation parameters are shown in the above

Table 1. Related parameters of V−DQN.

Simulation Parameters	Parameter Value
Carrier frequency	5.8 GHz
Noise power $P_n{}_o$	−125 dB
Additional attenuation coefficient of NLOS link $\eta$	20 dB
Environmental parameters A, B in tunnel scene	0.2, 12
V2R transmit power $P_t$	0.5 W
Height of A-RSU $H_3$	6 m
A-RSU interference power $P_t{}_d$	1 W
R2R transmission power	0.5 w
SNR threshold	10 dB
V2R Path Loss Index $a_r$	3
Learning rate	0.01
Reward discount Factor $\gamma$	0.8
Road type	one-way tunnel

. The body Q learning algorithm was compared and analyzed. Here, 5000 training rounds were set. The training results are shown in the following figures:

From Figure 18, we can see that the training efficiency of V-DQN was relatively high. Although there were small fluctuations during training, it was more stable than the Q-learning algorithm. With the strengthening of the deep neural network strategy, the V-DQN greatly improved the channel matching. The rate was finally trained to about 0.7 at 5000 iterations. The Q-learning algorithm had a relatively large fluctuation range during training, and finally only trained to about 0.4.

Figure 19 shows the relationship between the outage probability and the number of OBUs. In Figure 19, it appears that, for the three cases, the V-DQN algorithm performed better than the other two algorithms. Although the difference was relatively small in numerical terms, there were still some differences in a vertical comparison. From a horizontal comparison perspective, it can be seen that the V2R outage probability decreased as the number of OBUs increased. This was because, according to the vehicle movement model, the smaller the vehicle density, the smaller the average vehicle spacing. Therefore, in order to ensure the reliability of the V2R link, the RSU needed to increase the transmit power, and the total V2R channel capacity gradually increased. At the same time, the DQN with multiple hidden layers had more complex network layers, so compared with the Q-learning algorithm, the learning effect with environmental data was better. This shows that the proposed V-DQN could effectively improve the communication in a high-density traffic environment.

4. Conclusions

This paper mainly analyzed the performances of vehicle communication in a tunnel. First, by establishing a mathematical model, the influence of different road surfaces and tunnel walls on V2V communication path loss was analyzed. The simulation results showed that, at the same distance, when vehicles communicated with asphalt roads and black tunnels, the signal propagation path loss was the highest, while when vehicles communicated with concrete roads and white tunnels, the signal propagation path loss was the lowest. Then, aiming at the problem of the relatively large path loss in tunnel V2V communication, a method combining V-MIMO technology and a deep reinforcement learning algorithm(V-DQN) was proposed. The simulation results showed that the V-DQN could achieve a higher total capacity compared with the traditional Q learning method, and it had a low probability of interruption. In addition, the simulation results showed that the DQN demonstrated a significant improvement in convergence speed and stability, which has guiding significance for future research on tunnel–vehicle communication optimization.