Vehicular Delay-Tolerant Networks with Image Recognition-Based Adaptive Array Antenna for Winter Road Surveillance in Local Areas ^†

Abstract

The rapid growth of the ITS (intelligent transport system) development requires us to realize new kinds of applications, such as the winter road surveillance system. However, it is still necessary to consider the network difficulty areas for LTE (long-term evolution) or 3G transmissions when one visits sightseeing spots such as ski resorts or spas in mountain areas. Therefore, this paper proposes a winter road surveillance system in the local area based on vehicular delay-tolerant networks. The adaptive array antenna controlled by image recognition with the Kalman filter algorithm is proposed as well to the system in order to realize higher delivery rates. The implementations of the prototype system are presented in this paper as well, and the effectivity of the radio transmission in the prototype system is realized by vehicular image recognition methods and the asynchronous voltage controls for antenna elements for the rapid directional controls of the radio transmission. The experimental results showed that the radio directional controls by the prototype system for the target vehicle can proceed within one second, and that the simulation with the GIS (geographic information system) map pointed out the delivery rates of the proposed method—which are better than those of the epidemic DTN (delay-tolerant networking) routing by the nondirectional antenna. The experiments of the proposed methods indicate a higher efficiency of the data transmissions—even in the mountain area. Furthermore, future research subjects are discussed in this paper.

1. Introduction

The rapid growth of the ITS developments requires us to realize new kinds of applications, such as the winter road surveillance system. However, it is still necessary to consider the network difficulty areas for long-term evolution (LTE) or 3G transmissions when one visits sightseeing spots such as ski resorts or spas in the mountain areas. The weather in winter mountain areas changes easily, and it is necessary to consider frequent sensing of the road conditions to prevent traffic accidents. In fact, 91% of traffic accidents in snowy mountain areas are due to frozen roads, according to reports from the National Institute for Land and Infrastructure Management in Japan [1]. However, there are some difficulties in recognizing dangerous road conditions because of the sensors or cameras on traffic poles in certain locations. Therefore, it could be very helpful for the realization of an accurate road surveillance system if the sensors on each vehicle could observe the road conditions and the observed data were shared with other vehicles through vehicle-to-vehicle (V2V) communication.

However, in the actual fields, it is necessary to consider some subjects to apply to wireless communication between vehicles in mountain areas. First of all, the usage of high-frequency radio bands for rapid movements of driving vehicles might cause difficulties in terms of V2V communication. Currently, it is widely known that IEEE802.11p [2] with 5.9 GHz will be using V2V communication in the near future; however, it is expected that the data transmission range does not extend far enough to cover a couple of hundred meters between vehicles. Secondly, it is easily influenced by radio noises with high-frequency radio bands because of obstacles such as trees or houses along the roads. Therefore, it is necessary to consider the radio transmission range and the reduction of the radio noise for V2V communication with high-frequency radio usages for the road surveillance system in mountain areas in the winter.

Thus, this paper proposes a winter road surveillance system for local areas based on vehicular delay-tolerant networks (DTN) [3] and an adaptive array antenna (AAA) [4,5] controlled by image recognition with the Kalman filter algorithm—which is also proposed to the system in order to realize longer radio transmission ranges. In the proposed system, the road conditions are observed by the quasi-electrostatic field (QEF) [6,7] sensors on the vehicle’s wheels, and the observed data are transmitted to the other vehicle via DTN routing. When the data are transmitted, the proposed AAA system detects vehicles which can receive transmissions with the help of its camera. Then, the optimal antenna direction is calculated with the Kalman filter algorithm [8,9] to reduce the radio noises from the obstacles and the rapid movements of the vehicles.

Additionally, the implementations of the prototype system are presented in this paper. The effectivity of the radio transmission in the prototype system was realized using vehicular image recognition methods and the asynchronous voltage controls of antenna elements for the rapid directional controls of the radio transmission. The experiments of the proposed methods indicated higher efficiency of the data transmissions—even in the mountain area. Furthermore, future research subjects are discussed in this paper.

The proposed road surveillance system is first explained in Section 2. Secondly, the proposed adaptive array antenna controls with image recognition are explained in Section 3, and the proposed DTN routing in the system is presented in Section 4. Then, the implementations of the proposed AAA system with remote voltage controls functions for the adjustments of the antenna direction is presented in Section 5. Finally, the experiments using the prototype AAA system and the simulation using the GIS map are explained for the evaluation of the proposed methods, which are presented in Section 6, and the conclusions and the future works are discussed in Section 7.

2. Proposed Road Surveillance System

This paper proposes a winter road surveillance system consisting of one of the QEF sensors on the vehicles, an AAA system with machine learning-based image recognition, and V2V-based DTN routing as shown in Figure 1.

In the system, first of all, the road conditions are estimated by the QEF and temperature sensors on vehicles, and the estimated data are stored in each vehicle. Then, when there are vehicles which can receive transmissions or wireless devices on the utility poles along the roads, the data are communicated to others via DTN routing using the data triage methods [10,11].

Therefore, the current road surveillance systems usually consists of sensors such as cameras or thermometers at traffic lights or utility poles along roads, and the observed points and time durations are strictly limited. On the other hand, the proposed road surveillance system can provide more observed points and the recent data of the mountain roads.

However, as mentioned in the previous section, it is necessary to consider some issues to realize V2V communication in the proposed system. One of the problems is the radio range of V2V communication. Since vehicles tend to move fast in local areas, the transmittable distances are not enough to establish connections and to manipulate the data using high-frequency radio bands with a nondirectional antenna, such as IEEE802.11p. Therefore, the data duplications by IEEE802.11p must be completed within several seconds while the vehicles are running. Further, the LPWA (low power, wide area) [12] has a longer transmission range because it consists of a lower frequency, such as 920 MHz, but the throughput is not enough for the transmission of multimedia data such as movies or pictures.

Secondly, it is necessary to consider the radio noises from obstacles such as trees or houses in the proposed V2V communication. As shown in Figure 1, if there is a tree between the running vehicles, the high-frequency radio can easily cause radio noises, and it is hard to maintain a stable wireless connection in the surveillance system. Further, the optimal antenna direction is hard to set because of the driving vehicles. Even if the optimal antenna directions of the AAA system are set from the current vehicle’s position, vehicles with high speeds will move to the next position during data transmission.

Figure 2 presents the aim of wireless connectivity in this study. When the V2V system uses a nondirectional antenna, the radio transmittable distance between the vehicles is shorter. Further, because of the radio noises from the obstacles or the movements of the vehicles, the real connectivity of the RSSI (Received Signal Strength Indicator) levels becomes unstable, as shown in Figure 2. Thus, the purpose of this study is to provide a longer radio range and stable connectivity in the proposed road surveillance system.

Therefore, this paper proposed the AAA system with machine learning-based image recognition to realize a longer radio range and quick antenna controls, and the paper also introduced the antenna’s angle adjustments via the Kalman Filter algorithm for radio noise reductions and predictions of the driving vehicle’s locations.

3. Adaptive Array Antenna Controls with Image Recognitions

The AAA is a smart signal processing method to control the radio directions by the differential of the signal shifts on the multiple antenna elements [13]. Moreover, the beam-forming vectors are generally determined by the MMSE (minimum mean square error) or CMA (constant modulus algorithm) [5]. However, a previous paper [5] pointed out that the convergence of the calculation might take longer periods such as a couple of minutes with the general methods, and it is supposed that the general AAA system is not proper for V2V communication. Further, another paper [14] mentioned that there is another approach to estimate the beam-forming vector by the optimization algorithm such as the LMS (least mean square) or the RLS (recursive least square), but it explained that the complexity of these calculations might be a problem.

Therefore, first of all, this paper proposed the AAA system with machine learning-based image recognitions in the road surveillance system. In the system, the pictorial image is captured by the camera on the vehicles, and the target vehicle images are recognized with the machine learning algorism. Then, since the assumed road surveillance system needs rapid beam-forming controls, the distance and the angle of the target vehicles are calculated as shown in Figure 3.

In the left of Figure 3, the captured images are firstly used for the estimations of the distance of the vehicle by Formula (1).

$$d_1=d_2\sqrt{\frac{S_2}{S_1}.}$$

(1)

Here, the total pixel area of the field vehicle is S₁, and the total pixels in the captured image are S₂. The distance of the actual field stands for d₁, and the distance of the captured image’s pixel is d₂. For example, if the relations of total pixels S₂ and the distance d₂ are previously measured by the certain camera, the distance for the target vehicles d₁ is approximately calculated by the ratio of a square root of S₂ and S₂, as shown in Figure 1. Next, the angle $\theta$ is estimated by Formula (2), as in the right figure.

$$\theta ={\tan }^{-1}\frac{x_1}{d_1}={\tan }^{-1}\frac{x_2}{d_2}$$

(2)

Although the general estimation from the captured vehicle images uses the differences of the two cameras or the marking points such as the centerline on the road, the proposed calculation methods realize simpler calculations for the rapid beam-forming.

Then, adjustments of the beam-forming angle we made using the Kalman filter algorithm. The Kalman filter is widely known as the estimation algorithm that is used for the prediction of the weather forecast or autonomous driving systems. The calculation process mainly consists of two phases for the estimations. One is the time update phase, and it updates the current time and the predictions of the observed values. Another process is the measurement update phase, and it updates the observed values and the adjustments [8,11]. In the proposed system, as mentioned in the previous paper [11], the calculated angle and distance from the captured image continuously used for the observed values in the Kalman filter process in OpenCV’s API (Application Programming Interface) [15], and the output adjustment values were used for the angle controls of the beam-forming. Therefore, even if the target vehicles are out of sight in the system because of the obstacles, it is considered that the adjustments of the beam-forming proceed in order to keep the stable connectivity in the system.

4. Delay-Tolerant Networks for V2V Communication

DTN is the routing method by the store–carry-forward type of protocol for poor network connectivity [16]. It was originally proposed for interspace communication [3], but it is also considered to use it for the case of the ultra-large-scale disasters or the local areas where 3G/4G networks are not available. In the methods, the vehicle firstly stores the data if there is no vehicle where they can be transmitted. Then, when other vehicles come into the transmittable radio range, the data are duplicated and sent to the others. However, previous papers such as [17,18] pointed out that epidemic DTN routing has difficulties in itslower delivery rate or the high latency depending on the various factors such as hardware resources, the node’s movements, or the queuing order of the stored data. Thus, various modifications to epidemic DTN routing have been proposed, such as spray and wait [18] and MaxProp [19].

This paper introduces DTN with the data triage methods [10] for improvements of network efficiency of V2V communication in local mountain areas. The data triage methods is a queue ordered type of DTN routing such as Maxprop, and it was initially considered for the improvements of the network efficiency after a disaster in the local areas. In the methods, the queuing data are rearranged by the priority of the user policy such as life safety or rescue, as shown in Figure 4.

The right figure shows the first-in first-out (FIFO) method in which the most recent message is duplicated to others, and which most DTN routing applied in their queuing methods. On the other hand, in the data triage method, the order of the stored messages is rearranged based on priority values in accordance with the user policy after the message comes to the cache in the stored nodes [17,18]. Then, the most significant and recent message is duplicated to others when a vehicle that can receive a transmission is shown. The paper [17] also pointed out that the ordered queue in the node cache and the radio range is more important for the effectivity of DTN routing although the previous DTN approaches focus on hardware resources. Therefore, the data triage method focuses on reducing the overhead of the queued messages in the proposed methods.

Moreover, according to the paper [20], the structure of the priority ID in each message consists of the 3-digit priority values (PPP), a 14-digit date (YYYYMMDDhhmmss), and a 64-digit android ID(NNNNNNN…), and the income messages are rearranged by the order of the priority ID in the stored data. Therefore, in this paper, the priority value in the priority ID is classified into several levels by the QEF sensors on the vehicles, and the significant data are firstly duplicated to others in the proposed road surveillance system.

Further, in the data triage method, the node connections are carried by the MAC address and the Android ID to establish a quick connection [20] since the DHCP configuration takes a couple of seconds to set the local IP address. Additionally, the Global IP address for every device is difficult to set in the actual fields because of the limited number of Global IPs.

5. Implementations of the Prototype System

The prototype of the proposed AAA system is shown in Figure 5. The prototype system mainly consists of three parts. One is the image recognition part of the left side in the picture. The part includes the camera and the note PC to capture the vehicle images.

Then, the captured images are used for the image recognition of the vehicles in the note PC, and OpenCV [21] is used for the implementations. In detail, the implementations use Windows 10, OpenCV2.4 [21], and Visual C++ in MS Studio 2013 for the prototype system. The webcam is a Logicool Web camera c270 in the prototype system. Moreover, adjustments of the captured location image were made using the Kalman filter algorithm, and the prototype system uses OpenCV for the implementations.

Figure 6 shows the captured window in the prototype system. The vehicles are recognized by the captured images, and the distance and the angle are calculated from the recognized images. Then, the Kalman filter is calculated for the adjustment of the driving vehicle.

Following the adjustments of the recognized images, the remote control of the voltage meter has proceeded. The voltage meter in the center of Figure 5 is PMX18-5A by KIKUSUI Corporation, and it supports adjustments from the RS232C, USB, and LAN (Local Area Network) interfaces. In this prototype system, the LAN interface is used for the remote controls from the note PC, and the rapid voltage controls were implemented by asynchronous communication using the VISA (Virtual Instrument Software Architecture) standard [22]. In the voltage remote as shown in Figure 7, the connection between the PC and the voltage machine is established by the “OPEN” command of the VISA standard through the LAN. Then, the remote control mode is activated by the “WriteString” command of synchronous communication, and the value of the target voltage is transmitted after receiving the response from the voltage machine. Then, the current-voltage setting is continuously asking the voltage machine, and the connection is closed when the current voltage reaches the target voltage. Although synchronous communication takes a few seconds to set the target voltage because of the frequent error checks, the implemented asynchronous controls realize the rapid controls for the voltage meters in the prototype system.

Finally, Figure 8 shows the implemented antenna on the left side of Figure 5 using two antenna elements [16].

In Figure 8, the two patch antenna elements are connected to the phase shifter to control the wireless directions, and so the beam-forming can be properly controlled by the differentiations of the given voltages to each phase shifter.

6. Experiments

The experiments were confirmed for the effectivity of the prototype AAA system and the DTN routing in the proposed winter road surveillance system. First of all, the beamforming angle by the given voltages experimented in the anechoic chamber room in Fukuoka Institute of Technology, Japan. In the experiments, the antenna was fixed on the rotatable table, and the differences of the given voltages for each antenna elements were 0, 5, 10, and 15 V. The experimented radio wave was IEEE802.11b (DSSS, 2.4 GHz). Figure 9 shows the results of the RSSI levels by the given voltage.

The results show that the beam-forming angle in the prototype system was properly confirmed by the differences of the given voltage in the antenna elements. Therefore, if the proposed methods can estimate the angle to the target vehicle, it is considered that the longer radio connections can be realized.

Secondly, the quickness of the image recognition and the beam-forming controls were evaluated by the prototype system. In the experiment, the printed picture of the vehicle was previously prepared, and Figure 10 shows the experimental circumstances in Uchida Lab, Fukuoka Institute of Technology, Japan.

The A4 size printed red vehicle was fixed at a distance d = 0.5 m from the camera, and the captured images were used for the calculation of the distance and the angle to the vehicle in the note PC. Then, after calculation of the Kalman filter, the proper voltage was remotely set to the voltage meter through the LAN in the prototype system. Further, the printed vehicle in the picture was slightly moved to the points of $\theta$= 0, 5, 10, 15, 20, 25, and 30 degrees by manual controls, and the target voltage to the voltage meter was calculated in the experiments.

Table 1. Results of the beam-forming controls by the prototype system. Processing time is calculated by the logs of the captured image data and the remoted voltage meter in the experiments.

Angle (Degree)	Processing Time (s)	Observed Voltage (V)
5	0.733	4.99
10	0.843	9.99
15	0.918	15.00
20	0.937	20.00
25	0.953	24.99
30	0.965	29.99

shows the results of the experiments.

The results show the beam-forming controls wre confirmed within one second, and it is assumed that the proposed methods are effective for the rapid controls for the V2V communication. Moreover, it is considered that the parameters in the Kalman filter calculation can deal with the durations for the beam-forming and that these adjustments will be the future works in this study.

Finally, these results are introduced to the network simulator in order to evaluate the effectivity of the proposed methods in the actual fields. The ONE simulator [23,24] was used for the basic frameworks for the implementations, and the 10 km × 10 km GIS map of Takizawa City in Japan was introduced into the simulator, as shown in Figure 11.

The data delivery rate and the latency from Takazawa City Office to Iwate Prefectural University was experimented upon under the different number of the vehicles in the simulation, and the nondirectional antenna with IEEE802.11b and the proposed AAA system with IEEE802.11b were compared in terms of effectiveness in the experiments. In the experiments, the duration of the data connection was set as five seconds for the nondirectional antenna group. That is because it takes about five seconds to set the DHCP configuration in the previous experiments. Further, the duration of the processing time was set one second from the experiments as mentioned regarding beam-forming in the previous section. Then, the wireless connection by the MAC address and Android ID was confirmed by the implementations of the previous paper [25].

Table 2. The scenario of the network simulator in the experiment.

Parameters	Setting Values
Fixed Nodes	Two Stations; IPU (Iwate Prefectural University) and Takizawa City Office.
Mobile Nodes	25, 50, 75, 100, 125, 150 vehicles. The vehicles randomly move with 10–50 km/h, and the movement pattern is based on the random walk model.
DTN	Epidemic vs. Data Triage(Only one level of the priority)
Wireless Interfaces	Non-directional antenna with IEEE802.11b (2.4 GHz, 11 Mbps, 100 m) vs. The proposed AAA system with IEEE802.11b (Maximum Distance 2000 m) TTL (21,600 s)
Data	Data are carried from IPU to Takizawa City Office. 0.5–1.0 MB data (text data) are created in every 50–60 s at IPU.
Resources	2 GB storage.
Simulated Duration	6 h (21,600 s).
Process Time for the Connection	Non-directional antenna (5 s for DHCP configuration) vs. Proposed AAA (1 s for the beam-forming and the wireless establishment).

shows the major scenario in the modified simulator.

The results are shown in Figure 12; Figure 13. Firstly, Figure 11 shows the results of the data delivery rate in comparison with the epidemic DTN routing with the nondirectional antenna by DHCP configuration and the proposed DTN routing with the AAA system based on the results from the prototype system specifications. The results show the proposed AAA system is more effective than the nondirectional antenna because the delivery rate in the proposed AAA reaches nearly 95 percent even if there are only 25 vehicles. Moreover, even if the number of the vehicle in the city is increased, the percentage stays at nearly 100 percent on the proposed methods, while the nondirectional antenna group shows 80 percent in the delivery rate.

Next, Figure 12 shows the results of the latency from the Takizawa City Office to Iwate Prefectural University. As with the previous results, the latency of the proposed AAA system is rapidly decreased to about 1000 s, and the latency keeps around 500 s when the vehicles are increased. On the other hand, the nondirectional antenna groups stay around 2000 s even if the number of vehicles is increased in the simulation.

The results indicate the effectiveness of the proposed methods in the winter road surveillance in the local areas and the additional experiments, including the evaluation of the proposed DTN routing in future studies.

7. Conclusions and Future Studies

This paper described a winter road surveillance system in mountain areas, and it is necessary to establish a longer radio transmission range and to reduce radio noise from the obstacles. Therefore, this paper proposed AAA controls through image recognition to realize the longer radio distance with quick responses, and beam-forming adjustments with the Kalman filter algorithm to reduce the radio noise. Then, implementations of the prototype AAA system to realize longer wireless connections with a high-frequency radio system such as IEEE802.11b were presented in this paper. Then, the quickness of the prototype AAA system was evaluated through experiments with the prototype system, and the experiment results were considered for network simulations using the actual GIS data of the local area.

The experiment results indicate the effective beam-forming controls in the winter road surveillance system in the local areas, and it is considered that a higher data delivery rate and lower latency are expected in the actual fields. However, the additional experiments are considered to be necessary for the evaluation of the DTN with the data triage methods, and the additional implementations of the DTN devices in the vehicles are also necessary for future works. Moreover, we are planning to confirm the experiment in actual fields, and also to deal with the additional functions for V2V communication, such as the group functions for the radio connections or the FEC (forward error correction) for future studies.