A General Framework for Reconstructing Full-Sample Continuous Vehicle Trajectories Using Roadside Sensing Data

Abstract

Vehicle trajectory data play an important role in autonomous driving and intelligent traffic control. With the widespread deployment of roadside sensors, such as cameras and millimeter-wave radar, it is possible to obtain full-sample vehicle trajectories for a whole area. This paper proposes a general framework for reconstructing continuous vehicle trajectories using roadside visual sensing data. The framework includes three modules: single-region vehicle trajectory extraction, multi-camera cross-region vehicle trajectory splicing, and missing trajectory completion. Firstly, the vehicle trajectory is extracted from each video by YOLOv5 and DeepSORT multi-target tracking algorithms. The vehicle trajectories in different videos are then spliced by the vehicle re-identification algorithm fused with lane features. Finally, the bidirectional long-short-time memory model (LSTM) based on graph attention is applied to complete the missing trajectory to obtain the continuous vehicle trajectory. Measured data from Donghai Bridge in Shanghai are applied to verify the feasibility and effectiveness of the framework. The results indicate that the vehicle re-identification algorithm with the lane features outperforms the vehicle re-identification algorithm that only considers the visual feature by 1.5% in mAP (mean average precision). Additionally, the bidirectional LSTM based on graph attention performs better than the model that does not consider the interaction between vehicles. The experiment demonstrates that our framework can effectively reconstruct the continuous vehicle trajectories on the expressway.

1. Introduction

In recent years, autonomous driving has become an important topic in research, and many studies exist on positioning [1], perception [2], decision-making [3,4], motion planning [5,6], and control [7,8]. Among these, decision-making in autonomous driving is closely related to the operating conditions of other vehicles on the road, and road facilities can capture these operating conditions (including position, speed, pavement condition, etc.). Therefore, the coordination and cooperation of road facilities, autonomous driving, and traffic control have become important research topics [9,10].

An important topic in vehicle–infrastructure cooperation is the extraction of vehicle trajectories. Continuous vehicle trajectory information can play an important role in autonomous driving and intelligent traffic control [11]. It can provide the trajectory position of other vehicles for autonomous vehicles, which could help autonomous vehicles judge the current and future trajectories of surrounding vehicles and determine the next action. Meanwhile, continuous vehicle trajectory data can assist intelligent traffic control, risk assessment, and early warnings and help traffic personnel analyze road operation conditions [12].

Currently, extracting continuous vehicle trajectory information is challenging [13]. Video sensors are used in most regions to obtain motion information, such as the location and speed of road traffic participants. However, the video sensor data used today are strongly affected by occlusion and perspective [14]; some have low positioning accuracy, and the detection results are strongly affected by the environment. Additionally, common sensor data problems exist, such as different IDs marking in different sensors, which prevents direct identification of the same vehicle by different sensors. Therefore, it is difficult to directly obtain continuous vehicle trajectories from the video data.

Many studies have aimed to extract continuous vehicle trajectories from dynamic and static sensors [15,16]. This task has several directions in research. Some studies concern extracting vehicle trajectories from single roadside visual sensing data, whereas others concern finding vehicles with the same ID in different sensor data. Some studies also concern predicting and completing vehicle trajectories. Although these studies solve the corresponding problems effectively, few have proposed an entire continuous vehicle trajectory reconstruction framework, and some did not consider the particularity of the traffic scene. When special problems occur (such as multiple similar vehicles appearing on the road simultaneously), the model cannot provide high accuracy. Therefore, it is essential to study the actual traffic scene and propose the corresponding vehicle trajectory reconstruction framework.

In this research, we choose the expressway as the actual traffic scene and use the visual sensor data to construct the continuous vehicle trajectory. Constructing the vehicle trajectory on the expressway involves many problems. Firstly, the distance between the sensors on the expressway is long, resulting in gaps in the detected vehicle trajectory. Secondly, similar vehicles sometimes appear simultaneously on the expressway, making it difficult to distinguish them. To solve the above problems, we propose the corresponding reconstruction framework. The framework comprises three parts: single-region video trajectory extraction, multi-camera cross-region trajectory splicing, and vehicle trajectory completion. The framework first obtains a vehicle trajectory in a single region through video data. It then uses vehicle re-identification technology to match vehicle trajectories between different regions and splices the trajectories belonging to the same vehicle. After trajectory splicing, the framework uses a trajectory completion model based on graph attention to complete the missing parts of the vehicle trajectories.

The main contributions of this study are summarized as follows:

• We consider the spatio-temporal characteristics of the vehicle in the original vision-based vehicle re-identification model. We fuse the model with lane features to help it distinguish similar vehicles that appear simultaneously.
• We applied the bidirectional LSTM network based on graph attention. This model considers the interaction between vehicles and uses bidirectional trajectory completion. Our model can solve the problem of trajectory completion better than other baseline models.
• We propose a framework for continuous vehicle trajectory reconstruction based on video data, which uses the original video data to obtain the vehicle trajectory of a single region, employs the vehicle re-identification technology to splice the continuous vehicle trajectory, and uses the bidirectional long-short-time memory model (LSTM) based on graph attention to complete the incomplete trajectory.

The remainder of this paper is organized as follows. Section 2 reviews the previous research on continuous vehicle trajectory reconstruction. Section 3 details the continuous vehicle trajectory reconstruction framework. Section 4 describes and discusses the application effect of the framework in the actual case of the Donghai Bridge. Finally, Section 5 provides the conclusions and future prospects of the framework.

2. Related Work

This section provides an overview of the complete assessment process for the continuous vehicle trajectory reconstruction framework, including single-region video trajectory extraction, multi-camera cross-region trajectory splicing, and vehicle trajectory completion.

2.1. Single-Region Video Trajectory Extraction

The sources of vehicle trajectory data in a single region are diverse [17,18,19], and visual sensor data is the most widely used. Currently, various vehicle trajectory tracking methods are available for visual sensors [20,21,22,23,24,25]. Most of the methods are multi-target vehicle trajectory tracking using a matching algorithm for the detected objects [26,27], such as the DeepSORT algorithm [28]. Another multi-target tracking method is to use the neural network to analyze the motion information of the object to solve the association problem [29,30,31], such as the TrackFormer network based on the tracking-by-attention paradigm [32], which applies an attention mechanism to data association and tracking. It avoids dependence on information such as appearance features and realizes implicit trajectory association. There are also some multi-target tracking methods based on detection and tracking. For example, Viktor Kocur proposed a multi-target tracking algorithm based on CenterTrack object detection and neural network tracking [33]. However, fluctuations and gaps exist in the trajectories due to the accuracy of the video sensors and the error of the algorithm [14]. It is necessary to improve the continuity of the trajectories on the basis of obtaining accurate trajectories or using higher-precision sensor data.

2.2. Multi-Camera Cross-Region Trajectory Splicing

Currently, the main method of vehicle trajectory splicing is finding the trajectories of the same vehicle in different videos and then splicing the trajectories. For vehicle re-identification, the basic method is based on sensors, such as using wireless magnetic sensors to determine vehicle identity [34]. With the development of emerging sensors and technologies, such as identification tags and GPS (global positioning system), vehicles can be re-identified directly. For example, Prinsloo proposed a vehicle re-identification algorithm based on radio-frequency identification tags, which is suitable for various toll stations [35].

Considering the low penetration of emerging sensors and the large-scale use of visual sensors on the road, many studies have proposed methods to realize vehicle re-identification from videos. The early method is based on the method of artificial design features [36,37]. Researchers directly extract specific vehicle features from vehicle images to calculate the similarity to re-identify the vehicle ID (specific vehicle features: vehicle texture, vehicle color distribution, etc.), such as the BOW-SIFT method based on texture features and the BOW-CN method based on color features proposed by Liu et al. [38]. After CNN (convolution neural network) was suggested, scholars proposed using deep learning to obtain multidimensional information about vehicles to measure the similarity between vehicle images [39,40,41,42,43]. Simultaneously, considering the spatial and temporal information of vehicles, scholars have proposed methods such as screening time and geographical correlations to increase the accuracy of vehicle re-identification [44,45]. Although the current models have high accuracy, they struggle to solve some problems in practical applications, such as the appearance of multiple similar vehicles simultaneously. Therefore, in the framework of vehicle trajectory reconstruction, the vehicle re-identification model needs improvements to overcome these practical problems.

2.3. Vehicle Trajectory Completion

The completion of missing trajectories can be generalized as a trajectory prediction problem. In the early studies, trajectory prediction did not consider the interaction between individuals and only used historical data as the basis to predict the future trajectory [46,47]. In recent studies, advanced deep learning methods used networks such as RNNs and graph attention and considered the interactions between individuals in the model, combined with individual historical trajectory data, to predict future trajectories [48,49]. Zhao et al. proposed a vehicle trajectory prediction model based on graph attention [50]. The graph attention network simulates the interaction between vehicles and improves the authenticity and accuracy of the predicted trajectory. Experiments show that the model based on graph attention can effectively simulate the interaction between vehicles, and the model has a better effect on the task of vehicle trajectory prediction. In addition, geofencing has been shown to be a reliable method for trajectory prediction of vehicles (ground or aerial). Using the new kinematic model, combined with the geo-fence algorithm, the trajectory of vehicles in the area can be predicted [51,52,53]. There are also studies based on traffic flow and individual complete trajectories to complete the overall traffic flow trajectories. Chen proposed a macro–micro integration framework for completing and reconstructing vehicle trajectories on the highway in a multi-source data environment [54]. The framework completes the vehicle trajectories using fixed-position sensor data to collect the position and speed of vehicles at a certain location and a small amount of probe vehicle (PV) data that generates continuous trajectories. Considering the feasibility of practical applications, we hope to improve the accuracy of the vehicle trajectory prediction model based on graph attention in our framework.

3. Methodology

This section introduces our framework for continuous vehicle trajectory reconstruction based on visual sensors, as shown in Figure 1. The continuous vehicle trajectory reconstruction framework mainly comprises three modules: single-region video trajectory extraction, multi-camera cross-region trajectory splicing, and vehicle trajectory completion.

For video data, the framework uses a single-region video track extraction module to obtain vehicle trajectory data for each visual sensor. It then uses the multi-camera cross-region trajectory splicing module to match the vehicle trajectories of different sensors to achieve trajectory splicing within the entire region. We import the missing vehicle trajectory data into the bidirectional LSTM model based on graph attention for trajectory completion and finally reconstruct the continuous vehicle trajectory. The following section introduces each module in detail.

3.1. Single-Region Video Trajectory Extraction

Many challenges are involved in extracting vehicle trajectories from visual sensors. The videos of visual sensors used in this experiment have problems such as insufficient video pixels, video jitter, and mutual occlusion between vehicles. To obtain more accurate single-region video trajectory data, we adjusted the videos for deshaking and color contrast to stabilize their vehicle trajectory data. Aiming for the occlusion between vehicles, we used the DeepSORT multi-target tracking model in this module [28]. When the vehicle is temporarily occluded in the video, it can still be recognized and tracked by the model. Figure 2 shows the model structure.

Firstly, the model uses the YOLOv5 algorithm to extract the vehicle position information from the current frame in the video; it then uses Kalman filtering to predict the position of each vehicle in the next frame. The model then reuses YOLOv5 to extract the vehicle position information from the next frame. The model then uses the Hungarian algorithm to perform positional and visual data association of detected and predicted vehicle positions, determining the matching relationship between vehicles in the preceding and following video frames. After the matching is completed, the model updates the tracked vehicle position box. The model continuously loops to track the vehicle trajectory in the video.

The vehicle trajectory obtained from the video is in the pixel coordinate system, but the required trajectory coordinates should be in the world coordinate system. It is necessary to convert the coordinate system of the vehicle trajectory. The conversion of the pixel coordinate system to the world coordinate system usually conforms to the following Equation (1):

$$\begin{gathered} \left(\begin{array}{c}u\\ v\\ 1\end{array}\right)=\frac{1}{Z}H\left(\begin{array}{c}U\\ V\\ 1\end{array}\right) =\frac{1}{Z}\left[\begin{array}{ccc}{H}_{11}& H_{12}& H_{13}\\ H_{21}& H_{22}& H_{23}\\ H_{31}& H_{32}& H_{33}\end{array}\right]\left(\begin{array}{c}U\\ V\\ 1\end{array}\right) \\ H=A\left(\begin{array}{ccc}{R}_1& R_2& T\end{array}\right) \end{gathered}$$

(1)

In Equation (1), $H$ is the homography matrix, which is obtained by multiplying the internal and external parameters of the camera. Here, $A$ represents the intrinsic parameters of the camera, $R_1$ represents the first column of the rotation matrix, $R_2$ represents the second column of the rotation matrix, and $T$ represents the translation vector. $Z$ represents the scale factor, $u$ and $v$ represent pixel coordinates, and $U$ and $V$ represent the corresponding world coordinates.

According to the design standard, we can obtain a priori information on the relative positions of the lane lines in space (in China, the longitudinal length of lane lines along the lane direction is 6 m, the longitudinal distance between lane lines along the lane direction is 9 m, and the lane width is 3–4 m.) [55]. We can select four points located on the lane line from the video image. The distance between these four points in the world coordinate system should satisfy the prior information on the lane lines. We choose one of the points as the origin and use the prior information to obtain the coordinates of the other three points in the world coordinate system. $H$ and $Z$ can be calculated by inserting the pixel coordinate system coordinates and world coordinate system coordinates of these points into the formula. The transformation matrix can then be used to convert the pixel coordinates to world coordinates for the video. Figure 3 shows the matrix calibration process.

3.2. Multi-Camera Cross-Region Trajectory Splicing

Multi-camera cross-region trajectory splicing is an important step in continuous vehicle trajectory reconstruction. Since most of the visual sensors deployed on the road network today cannot directly capture the ID information of the vehicle, it is difficult to determine the matching relationships between the vehicle trajectories extracted from different visual sensors. An additional module is required to match the trajectories.

Considering the above issues, a multi-camera cross-region trajectory splicing module is constructed in the framework. In this module, considering that similar trucks may appear on the road simultaneously, reducing the accuracy of trajectory splicing, we built a new model. The model can extract the vehicle’s visual and lane features from the videos and calculate the feature similarity of vehicles in different videos to find the same vehicle and splice the trajectory.

As shown in Figure 4, the visual feature extraction part of the network uses the Reid network proposed by Zheng et al. [56]. The model uses the latest network pre-trained on ImageNet as the backbone module, including ResNeXt101, ResNeXt101_32x8l_wsl, and ResNet50_IBN_a. The lane feature extraction part of the network uses a fully connected layer to convert the number vector of the lane where the vehicle is located into a feature vector with the same length as the vehicle’s visual feature. We then use the fusion feature of the visual feature and lane feature to calculate the similarity between vehicles.

We fuse the vehicle’s visual features and lane features according to the following Equation (2):

$$F_{fusion}=F_{vision}+k\ast F_{lane}$$

(2)

In Equation (2), $F_{fusion}$ is the fusion feature of the vehicle, $F_{vision}$ is the visual feature of the vehicle, and $F_{lane}$ is the lane feature of the vehicle; $k$ is the scale factor, which is obtained by model training.

We use the fused features to calculate the cosine similarity between different vehicles and find the vehicle-matching result with the best similarity matching.

$$dist\left(A,B\right)=1-\cos \left(A,B\right)=\frac{{\left|\left|A\right|\right|}_2\ast {\left|\left|B\right|\right|}_2-AB}{{\left|\left|A\right|\right|}_2\ast {\left|\left|B\right|\right|}_2}$$

(3)

In Equation (3), $A$ and $B$ are the fusion features of two vehicles, and $dist\left(A,B\right)$ is the feature similarity of the two vehicles.

After obtaining the vehicle-matching results, we splice the trajectories belonging to the same vehicle.

3.3. Vehicle Trajectory Completion

Due to the insufficient coverage region of the video sensor, gaps exist in the obtained continuous vehicle trajectories, so the missing trajectories must be completed. In this module, we refer to the STGAT model proposed by Huang et al. [57]. Considering the mutual interaction of vehicles, we add the graph attention mechanism to the Seq2Seq architecture. This model combines the temporal features and the spatial interaction features between vehicles and improves the accuracy of vehicle trajectory completion. Figure 5 shows the bidirectional LSTM vehicle trajectory completion model based on graph attention.

We propose a double-stream mechanism. The two trajectories before and after the missing trajectory are input into the encoder module to obtain the temporal features. Then the combination of temporal features is input into the decoder module to obtain the final completed trajectory. In our model, two completed vehicle trajectories are generated from the two ends of the missing trajectory towards the middle, respectively. When the Euclidean distance between the two trajectories is less than $Distanc{e}_{low}$ or the maximum number of completion frames is reached (set to $Fram{e}_{max}$), the completion is stopped.

The encoding module consists of an LSTM module and a GAT module. Considering the vehicle’s temporal characteristics, we use the LSTM module in the encoder to extract the timing characteristics of the trajectory sequence, focusing on the motion mode of each vehicle. The preliminary encoding results are shown as Equation (4):

$$m_i^t=LSTM\left(m_i^{t-1},e_i^t;W_m\right)$$

(4)

In Equation (4), $m_i^t$ is the eigenvector in the network corresponding to vehicle $i$ at time $t$, $e_i^t$ is the embedded coding result of the position coordinate of vehicle $i$ at time $t$, and $W_m$ represents the weight of the model.

For vehicle interaction characteristics, this model uses the graph attention model in the encoder. The model takes all vehicles as nodes and assigns importance to aggregate neighbor vehicle information. The preliminary temporal encoding results are input into the graph attention model, and the output is

$${\widehat{m}}_i^t=\sigma ({{\displaystyle \sum }}_{j\in N_i}GAT(m_i^t,m_j^t){\mathit{W}}_j{m}_j^t)$$

(5)

$$GAT\left(m_i^t,m_j^t\right)=\frac{\exp (LeakyReLU(a^T[{\mathit{W}}_i{m}_i||{\mathit{W}}_j{m}_j]))}{{{\displaystyle \sum }}_{k\in N_i}\exp (LeakyReLU(a^T[{\mathit{W}}_i{m}_i||{\mathit{W}}_k{m}_k]))}$$

(6)

In Equations (5) and (6), $GAT\left(m_i^t,m_j^t\right)$ is the graph attention coefficient at time t of node pail $\left(\mathrm{i},\mathrm{j}\right)$, $\mathit{W}$ is the dimension mapping matrix, $N_i$ is the set of neighbor nodes of node $i$, and $\sigma$ is a nonlinear activation function.

The decoder module is proposed to aggregate temporal and spatial features and then generate the final completed trajectory. At the time point $t$ of node $i$, these characteristics are combined as the input of the LSTM module: the initial coding result ${\overline{m}}_i={\mathsf{\delta }}_1\left(m_i^t\right)$ and the output ${\overline{g}}_i={\mathsf{\delta }}_2\left(g_i^t\right)$ after the graph attention and LSTM coding. Based on this spatiotemporal information, the missing trajectory can be completed.

4. Experiments

4.1. Scene Description and Data Preprocessing

• Donghai Bridge dataset [58]: The Donghai Bridge (Shanghai, China) is 32.5 km long and equipped with more than 200 cameras to support the commercial operation of self-driving trucks in Yangshan Port, as shown in Figure 6. This dataset contains video data during peak and off-peak hours throughout the day. In this experiment, we obtained data from two visual sensors on the Donghai Bridge to realize vehicle trajectory reconstruction. The distance between the visual sensors is 1020 m. The videos were collected during the peak hours of the day and lasted for half an hour, and the traffic flow in the videos was restricted. We have adjusted the debouncing and color contrast to make the videos more suitable for multi-target tracking. We captured 101 sets of vehicle images from two videos to create a dataset for the vehicle re-identification experiment. We also obtained complete short-distance trajectory data from the videos and created a dataset to evaluate the vehicle trajectory completion model.
• CityFlow dataset [59]: To train the vehicle re-identification model, we used the CityFlow dataset. The dataset is captured by 40 cameras in the traffic surveillance environment. It contains 56,277 bounding boxes in total. The training set comprises 36,935 bounding boxes from the 333 vehicles, and the test set comprises 18,290 bounding boxes from the other 333 identities. The remaining 1052 images are the queries. The dataset contains a variety of vehicles and can be used as training data for the vehicle re-identification model.
• The Argoverse bidirectional trajectory completion dataset: To train and evaluate the trajectory completion model, we modified the Argoverse motion forecasting dataset [60] and named it the Argoverse bidirectional trajectory completion dataset. The Argoverse motion forecasting dataset can be used to support vehicle trajectory forecasting tasks, providing rich trajectories and lane centerlines collected by a fleet of AVs in Pittsburgh and Miami. The dataset contains 333 K 5 s long sequences. We split it into 200 K for training, 66 K for validation, and 67 for testing. The vehicle trajectories of the Argoverse bidirectional trajectory completion dataset are sampled at 10 Hz. We selected the (0, 2] s and the (4, 5] s as the ground truth and the (2, 4] s for prediction.

4.2. Description of Evaluation Metrics and Model Parameters

In this experiment, we used the following model evaluation metrics:

• mAP: mean average precision. The average precision is calculated as the weighted mean of precisions at each threshold; the weight is the increase in recall from the prior threshold. The mAP is calculated by finding the average precision (AP) for each class and then averaging over several classes.

$$mAP=\frac{1}{N}({\displaystyle \sum }_{i=1}^{N}A{P}_i)$$

(7)

• CMC: cumulative matching characteristics. This is a classic evaluation index in the re-identification problem. The abscissa of the curve is the rank, and the ordinate is the recognition accuracy; $ran{k}_n$ indicates the top n results in descending order of the similarity of the recognition results containing the target. The recognition accuracy is the ratio of the number of $ran{k}_n$ to the total number of query samples.
• ADE: average displacement error. This is defined as the Euclidean distance ($L_2$ distance) between the ground truth and prediction of the vehicle locations over all predicted time steps:

$$L_2=\sqrt{{\left(x1-x2\right)}^2+{\left(y1-y2\right)}^2}$$

(8)

In Equation (8), $\left(x1,y1\right)$ and $\left(x2,y2\right)$ are the locations of the ground truth and prediction of the vehicle locations, respectively.

Table 1. Some model parameters.

Module	Parameter	Value
Single-region vehicle trajectory extraction	Confidence (YOLOv5)	0.8
Multi-camera cross-region trajectory splicing	$k$ (vehicle re-identification)	0.05
Vehicle trajectory completion	$Distanc{e}_{low}$	0.05
	$Fram{e}_{max}$	200

shows some of our model parameter settings.

4.3. Single-Region Vehicle Trajectory Extraction

The video data is imported into the single-region vehicle trajectory extraction module in the framework, and the DeepSORT algorithm is used to obtain the trajectories of all vehicles in the video in the pixel coordinate system. Figure 7 shows the visualization of some trajectories.

4.4. Multi-Camera Cross-Region Trajectory Splicing

The multi-camera cross-region trajectory splicing module in the framework is used to assign the corresponding matching ID to the vehicle trajectories in each video on the road section.

First, the CityFlow dataset is used to train the vehicle REID network of this module to obtain the model of the vehicle visual feature vector. Part of the calibrated video trajectory data of the Donghai Bridge is then used to train the scale factor in the feature fusion formula.

After training, we used the module to match the vehicle trajectories in the two camera videos on the road section.

Table 2. Evaluation indicators of our vehicle re-identification model fused with lane features.

Model	mAP (Mean Average Precision)
Our model	73.46%
Original vehicle re-identification model *	71.96%

* The model is taken from reference [56].

presents the results. The resulting mAP (mean average precision) is 73.46%, 1.5 percentage points better than the original vehicle re-identification model. This confirms that considering lane features can improve the accuracy of vehicle re-identification. Figure 8 shows the CMC (cumulative matching characteristics) diagram of the model.

4.5. Missing Trajectory Completion

To illustrate the performance of the bidirectional LSTM vehicle trajectory reconstruction model based on graph attention in this module, we selected different baseline models and applied these to the Argoverse bidirectional trajectory completion dataset. We used ADE as the evaluating indicator.

Table 3. Imputation errors in the Argoverse bidirectional trajectory completion dataset.

Model	ADE (m)
Our model	0.7649
Vanilla Bi-LSTM	1.9246
CS-LSTM	1.8523
NAOMI	2.0335
STGAT	1.8139

presents the results.

In the comparison results, the model considering the interaction between vehicles (Vanilla Bi-LSTM) performs better than the model that does not consider the interaction (NAOMI). Meanwhile, the model using the graph to represent interactions (STGAT) outperforms other models (Vanilla Bi-LSTM, CS-LSTM). In this experiment, our bidirectional LSTM vehicle trajectory reconstruction model based on graph attention outperforms all mentioned models. This reveals that our model can effectively capture the spatio-temporal characteristics of vehicle trajectories and the vehicle–vehicle interactions, and the bidirectional trajectory completion performs better than the unidirectional trajectory completion.

In this experiment, we applied our trained model to the vehicle trajectory completion on the Donghai Bridge. Figure 9 represents the effect of our model and some baseline models on vehicle trajectory completion on the Donghai Bridge. The result indicates that our model outperforms other models in the trajectory completion task on Donghai Bridge.

5. Conclusions and Discussion

We propose a framework for full-sample vehicle trajectory reconstruction based on roadside sensing data. The framework includes multiple tasks, deep learning-based video data processing, multi-target tracking, vehicle re-identification, and missing trajectory completion. YOLOv5 and DeepSORT algorithms are applied to extract vehicle trajectories. A vehicle re-identification algorithm is then designed to splice the vehicle trajectories in different video data, where lane and visual features are fused to reduce the adverse effect on the accuracy caused by the simultaneous appearance of similar vehicles on the expressway. The LSTM model based on graph attention is then developed to complete the missing vehicle trajectory. Comprehensive experiments are conducted using roadside sensing data from the Donghai Bridge, Shanghai, China. The recognition accuracy mAP of our vehicle re-identification algorithm with lane features reaches 73.46%, 1.5 percentage points higher than the original visual-based vehicle re-identification algorithm. Meanwhile, the experimental results show that the bidirectional LSTM vehicle trajectory reconstruction model based on graph attention outperforms other models that do not consider interactions between vehicles. The experimental results show that our framework can effectively reconstruct continuous vehicle trajectories via roadside sensing data.

Future studies could follow several directions. Firstly, we could continue to improve the multi-camera cross-region tracking algorithm, use more advanced neural networks, or introduce traffic operation rules to improve the accuracy of vehicle re-identification. Secondly, the model of vehicle trajectory completion could be improved. We hope that the vehicle trajectory predicted by the model will follow the space–time law of vehicle operation and that there will be no wrongly predicted trajectory collisions caused by algorithm errors. Thirdly, since the quality of video data is strongly affected by natural factors, such as weather and light, more types of sensor data could be fused to improve the accuracy of the vehicle trajectories. Finally, when the framework is used in different traffic scenarios, by collecting new data sets and transfer learning, we can make the framework suitable for the traffic scenarios, such as urban areas and rural areas.