An Approach for Filter Divergence Suppression in a Sequential Data Assimilation System and Its Application in Short-Term Traffic Flow Forecasting

Abstract

Mathematically describing the physical process of a sequential data assimilation system perfectly is difficult and inevitably results in errors in the assimilation model. Filter divergence is a common phenomenon because of model inaccuracies and affects the quality of the assimilation results in sequential data assimilation systems. In this study, an approach based on an L₁-norm constraint for filter-divergence suppression in sequential data assimilation systems was proposed. The method adjusts the weights of the state-simulated values and measurements based on new measurements using an L₁-norm constraint when filter divergence is about to occur. Results for simulation data and real-world traffic flow measurements collected from a sub-area of the highway between Leeds and Sheffield, England, showed that the proposed method produced a higher assimilation accuracy than the other filter-divergence suppression methods. This indicates the effectiveness of the proposed approach based on the L₁-norm constraint for filter-divergence suppression.

1. Introduction

Short-term traffic flow forecasting is a crucial component in many intelligent transportation systems (ITSs) [1,2,3]. Short-term traffic flow forecasting refers to predicting traffic flow in units of seconds and minutes mainly for traffic control and guidance, which can better describe real traffic conditions [4,5]. To achieve dynamic traffic management or provide advanced traveler information, short-term traffic flow forecasting that reflects real-time local fluctuations or traffic congestion resulting from fast-changing traffic flow values is necessary [6,7]. Due to the stochastic nature of traffic flow, robust and accurate prediction algorithms have become increasingly important [4,8,9,10]. Taking advantage of measurements and models is commonly used to make predictions [11,12,13]. Data assimilation (DA) is an important method that can be used to estimate the state vectors by integrating physical model information and measurements [14,15,16,17]. It can take advantage of measurements and models to make predictions by fusing measurement information during the model process based on the spatial–temporal distribution of the data and errors in the measurement and background fields [18]. Related DA methods have been applied to short-term traffic state predictions [11,12,19]. A DA system has three indispensable components: the assimilation model, the measurements, and the assimilation method. The mathematical expression of DA is as follows [20]:

$$X_k={\mathit{M}}_{k,k-1}{X}_{k-1}+{\mathit{G}}_{k,k-1}{w}_{k-1}$$

(1)

$$y_k={\mathit{H}}_k{X}_k+v_k$$

(2)

Equation (1) is a dynamic state equation, which expresses changes of the state vector over the time. $\mathit{M}$ is the dynamic state model with a discrete time index k. Equation (2) is an observation equation.$\mathit{H}$ is the time-dependent observational operator that connects the state $X$ and measurements $y$. $w$ and $v$ are Gaussian random noise functions with $w_k\sim N\left(0,{\mathit{Q}}_k\right)$ and $v_k\sim N\left(0,{\mathit{R}}_k\right)$. $\mathit{G}$ is a coefficient matrix.

The assimilation model, measurements, and sequential assimilation method constitute the sequential data assimilation (S-DA) system. As one of the implementation classes of data assimilation, the sequential assimilation method posteriorly estimates state vectors when measurements are available [21,22,23,24]. The filter-divergence phenomenon always occurs in S-DA systems, especially in Kalman series assimilation algorithms such as the Kalman filter (KF) or the ensemble KF (EnKF) methods [14,25,26]. The goal of this paper was mainly to study how to suppress the filter divergence phenomenon that occurs in the KF method and how to apply it to short-term traffic flow forecasting. Although the EnKF method behaves well in nonlinear and high-dimensional systems, especially when the system is extremely complicated and its gradient is difficult to acquire, it is popular in geophysical or meteorology areas, but rarely used in short-term traffic flow forecasting. The KF method outperforms many other methods in many traffic flow predictions [27,28]. It is a well-known technique in linear system to track state values over time [29] and it is a basic algorithm in S-DA systems. It updates the state variables using real-time measurements and adapts to changes in traffic flow [29,30,31,32]. Furthermore, the KF method provides efficient computations and requires little information storage [33]. Based on these advantages, the KF algorithm was used in the S-DA systems and is presented below to study how to suppress the filter divergence phenomenon and apply it to the prediction of traffic flow. Forecasting and updating are two components of the KF method, and the equations for both are shown in

Table 1. Kalman filter method adapted from [34].

Forecast	$x_k^f={\mathit{M}}_{k,k-1}{x}_{k-1}^a$
	${\mathit{P}}_k^f={\mathit{M}}_{k,k-1}{\mathit{P}}_{k-1}^a{\mathit{M}}_{k,k-1}{}^T+{\mathit{G}}_{k,k-1}{\mathit{Q}}_{k-1}{G}_{k,k-1}{}^T$
Update	${\mathit{K}}_k={\mathit{P}}_k^f{\mathit{H}}_k{}^T{\left({\mathit{H}}_k{\mathit{P}}_k^f{\mathit{H}}_k{}^T+{\mathit{R}}_k\right)}^{-1}$
	$x_k^a=x_k^f+{\mathit{K}}_k\left(y_k-{\mathit{H}}_k{x}_k^f\right)$
	${\mathit{P}}_k^a=\left(\mathit{I}-{\mathit{K}}_k{\mathit{H}}_k\right){\mathit{P}}_k^f$

.

Matrix $\mathit{Q}$ and $\mathit{R}$ are the covariance matrix of the Gaussian noise of state and the measurement equation, respectively. In the KF method, they are assumed to be uncorrelated [30,31,34]. The $\mathit{K}$ matrix in the updated component of the KF method shown in Table 1 is the Kalman gain matrix and is the weight of the state forecast values and measurements.

In theory, the reason for the filter-divergence phenomenon in the KF method is that the $\mathit{K}$ matrix becomes increasingly smaller over the assimilation evolution. This decrease results in a weak correction of the measurements on the state model and ultimately leads to a filter-divergence phenomenon [15,35]. The filter-divergence phenomenon in the KF method mainly arises from two factors. The first factor is the limitation of the computer word length, which causes errors such as rounding errors [36]. The accumulation of these errors reduces the filtering accuracy. When the accumulation of errors is significant, it causes the calculation error variance matrix to lose its positive definiteness and symmetry. Consequently, the deviation between the calculated and theoretical $\mathit{K}$ matrices will increase. The second factor is a cognitive limitation, which results in improper and inaccurate descriptions of the model and its statistical noise information. Furthermore, this limitation causes mismatching between the simulated values from the state model and the measurements as the unknown model errors propagate in the covariance matrix during the KF method process [15]. In other words, as the algorithm progresses, the measurement covariance $\mathit{Q}$ grows, whereas the model covariance $\mathit{P}$ shrinks. Thus, the model credibility increases, and the estimated values are mainly acquired from the model’s analog outputs. This study addressed solutions for suppressing the filter divergence in the KF method of the S-DA systems resulting from these cognitive limitations. Selecting a classic case as an example, that is, target tracking with uniform motion [37], it was possible to intuitively demonstrate the phenomenon of filter divergence arising from cognitive limitations. The assimilation models for target tracking are expressed as follows:

$$\{\begin{array}{l}{x}_{k+1}=x_k+v\times k+w_k\\ y_k=x_k+{\delta }_k\end{array}$$

(3)

where $x_k$ is the position of the tracking target at time k, and $x_0=205m$.$v$ is the speed with $v$ = 1 m/s; $y_k$ denotes the measurements of the target position; $w_k$ and ${\delta }_k$ are the noise of the dynamic state and observation models with $w_k\sim N\left(0,{\mathit{\Gamma }}_w\right)$ and ${\delta }_k\sim N\left(0,{\mathit{\Gamma }}_{\delta }\right)$, respectively. ${\mathit{\Gamma }}_w$ and ${\mathit{\Gamma }}_{\delta }$ were assumed to be 1 in this test.

Due to cognition limitations, it is supposed that the model given by Equation (3) was mistakenly selected as follows:

$$\{\begin{array}{l}{x}_{k+1}=x_k+w_k\\ y_k=x_k+{\delta }_k\end{array}$$

(4)

The KF assimilation method was used to estimate the target position under the two models given by Equations (3) and (4). The errors between the predicted values and the real measurements were calculated through 1000 simulations. The results are shown in Figure 1.

As shown in Figure 1, the simulation errors acquired from the correct model in Equation (3) were far smaller than those obtained from the incorrect model in Equation (4). Filter divergence occurred in the forecasting results of the target position using the wrong forecasting model with the KF method. The main reason that filter divergence occurred in the incorrect model was that this model did not consider the speed information. As $v\times k$ items were missing in the incorrect model, the model errors were underestimated.

Various methods have been proposed to suppress the filter divergence caused by inaccurate assimilation models. Initially, gain matrix adjustment methods were commonly used [38,39] including constant gain methods, stage-constant gain methods, and finite-lower-bound gain methods. These methods force the filter gain matrix $\mathit{K}$ to decrease to a certain value to avoid filter divergence. However, these methods are not precise enough, as the determination of the constant gain matrix and its limited values always rely on experiences. The limited memory method also can effectively suppress filter divergence [40]. This method reduces the influence of historical data on the gain matrix construction; however, the length of the memory is difficult to select. In addition, the covariance weighting (C-W) method [41] and the adaptive Kalman filter (A-KF) method [42,43,44,45,46] have commonly been used and are effective for suppressing filter divergence. The C-W method [41] suppresses the filter divergence by adding a weight or a fading factor to inflate the model error covariance matrix. After this, the dependence of the forecasting results on inaccurate models can be reduced, and the suppression of the filter divergence can be achieved. This method is often used in fields that require a balance of precision and computational efficiency. The A-KF method can estimate the state model error covariance matrix $\mathit{Q}$ and the measurement error covariance matrix $\mathit{R}$ online in real time to suppress filter divergence. The A-KF method is usually used in fields in which an accurate noise model is not available, but high precision is required. These two methods suppress the filter-divergence phenomenon by adjusting the model error covariance matrices during the KF method process in real-time to reduce the influence of the model error or model discrepancy term in the propagation of the covariance matrix on the filtering results. However, selecting a weight or inflating factor in the C-W method is difficult. Additionally, in the A-KF method, it is challenging to ensure that the calculation error covariance matrix is suitable for the dynamic noise in an actual dynamic process [47].

In this paper, an approach based on an L₁-norm constraint for filter-divergence suppression in the S-DA system was proposed and applied to short-term traffic flow forecasting to verify its effectiveness. The advantages of the filter-divergence suppression method based on the L₁-norm constraint method are that it can guarantee high precision and it is easy to implement. Three critical items were investigated: (i) an inaccurate S-DA system that generates filter divergence for short-term traffic flow forecasting; (ii) an approach based on the L₁-norm constraint for filter-divergence suppression; and (iii) the application of the S-DA system based on the L₁-norm constraint method to short-term traffic flow forecasting to verify its effectiveness compared with other methods.

The remainder of the paper is organized as follows. In Section 2, an inaccurate S-DA system that generated filter divergence for short-term traffic flow forecasting is presented. In Section 3, a filter-divergence suppression method based on the L₁-norm constraint is proposed. Section 4 gives a numerical example. In Section 5, short-term traffic flow forecasting application experiments are described. Section 6 presents the results and Section 7 presents our conclusions.

2. Representation of Filter Divergence Phenomenon in Sequential Data Assimilation System for Short-Term Traffic Flow Forecasting

Filter divergence occurs in parametric models in the S-DA system. Recently, vector autoregressive (VAR) models have been widely applied to short-term traffic flow forecasting [48,49]. VAR models are based on the assumption that the traffic flow of the current path in time interval [$kT,\left(k+1\right)T$], expressed as $q_{pc}\left(k+1\right)$, is related not only to its own traffic flow values in n previous time intervals $q_{pc}\left(k\right),q_{pc}\left(k-1\right),\cdots ,q_{pc}\left(k+1-n\right)$, but also to the corresponding values of its m adjacent paths: [$q_{p{a}_1}\left(k\right)$,$q_{p{a}_2}\left(k\right)$,$\cdots$,$q_{p{a}_m}\left(k\right)$] [$q_{p{a}_1}\left(k-1\right)$,$q_{p{a}_2}\left(k\right)$,$\cdots$,$q_{p{a}_m}\left(k-1\right)$],$\cdots$, [$q_{p{a}_1}\left(k+1-n\right)$,$q_{p{a}_2}\left(k+1-n\right)$,$\cdots$,$q_{p{a}_m}\left(k+1-n\right)$]. $T$ is the forecasting time period, and $k=1,2,\dots$. Future values can be acquired from the past values and their correlations analyzed. In this study, the traffic flow forecasting of $q_{pc}\left(k+1\right)$, which can be obtained from the VAR model presented previously, is expressed as follows [50]:

$$q_{pc}\left(k+1\right)=f\left(F\left(k\right),F\left(k-1\right),\dots ,F\left(k-n\right)\right)$$

(5)

where $F\left(k\right)$ denotes the traffic flow values of the current and adjacent paths at time interval [$\left(k-1\right)T,kT$], and $f(\cdot )$ denotes a nonlinear function.

Equation (5) can be rewritten in detail as follows:

$$\begin{array}{ll}r\left(k+1\right)=& {\xi }_0\left(k\right)\times S\left(k\right)+{\xi }_1\left(k\right)\times S\left(k-1\right)+\\ & \hspace{1em}{\xi }_2\left(k\right)\times S\left(k-2\right)+\dots +{\xi }_n\left(k\right)\times S\left(k-n\right)+\nu \left(k\right)\end{array}$$

(6)

with

$$\{\begin{array}{l}r\left(k+1\right)=\frac{q_{pc}\left(k+1\right)}{\overline{q_{pc}}\left(k+1\right)}\\ S\left(k\right)={\left[\begin{array}{cc}\frac{q_{pc}\left(k\right)}{\overline{q_{pc}}\left(k\right)}& \frac{q_{p{a}_i}\left(k\right)}{\overline{q_{p{a}_i}}\left(k\right)}\end{array}\right]}^T\left(i=1,2,3,\dots ,m\right)\end{array}$$

(7)

The unknown parameters [${\xi }_0\left(k\right),{\xi }_1\left(k\right),\dots ,{\xi }_n\left(k\right)$] in Equation (6) are treated as state vectors, which are also shown in Equation (8) and these vectors need to be calculated in the S-DA system. In Equation (7), $\overline{q_{pc}}\left(k+1\right)$ denotes the average value of the traffic flow along the current path calculated by historical flow measurements at time interval [$kT,\left(k+1\right)T$] on the same day in previous consecutive weeks; $q_{pc}\left(k\right)$ represents the historical traffic flow values of the current path at time interval [$\left(k-1\right)T,kT$]; $\overline{q_{pc}}\left(k\right)$ is the corresponding average value; $q_{p{a}_i}\left(k\right)$ represents the traffic flow values of adjacent paths $p{a}_i$ at time interval [$\left(k-1\right)T,kT$], and $\overline{q_{p{a}_i}}\left(k\right)$ represents the corresponding average values. In this study, n was set to 2.

The form of Equation (6) can be related to Equations (1) and (2) as follows:

$$\{\begin{array}{l}{M}_{k,k-1}=\mathit{I},G_{k,k-1}=\mathbf{0}\\ X_k={\left[{\xi }_0\left(k\right),{\xi }_1\left(k\right),\dots ,{\xi }_n\left(k\right)\right]}^T\\ y_k=r\left(k+1\right)\\ H_k=\left[S^T\left(k\right),S^T\left(k-1\right),S^T\left(k-2\right),\dots ,S^T\left(k-n\right)\right]\end{array}$$

(8)

In the S-DA system, filter divergence will occur if the assimilation model is not correct. The essence of filter divergence is that the Kalman gain matrix $\mathit{K}$ becomes increasingly smaller and even tends to zero over the assimilation evolution process. Consequently, the relative deviation between the estimated state values and the true values becomes increasingly larger. To display this phenomenon intuitively, it was assumed in this study that our understanding of the traffic flow forecasting system was incomplete and inaccurate. The system in Equation (7) was incorrectly taken as follows:

$$S\left(k\right)={\left[\frac{q_{pc}\left(k\right)}{\overline{q_{pc}}\left(k\right)},\frac{q_{p{a}_1}\left(k\right)}{\overline{q_{p{a}_1}}\left(k\right)},\cdots q_{p{a}_j}\left(k\right),\frac{q_{p{a}_{j+1}}\left(k\right)}{\overline{q_{p{a}_{j+1}}}\left(k\right)},\cdots \right]}^T$$

(9)

As an example to display the filter divergence more intuitively, the traffic flow forecasting results of path 7768 (LM838), which is a path of England’s highway system, were obtained using the correct and incorrect models. Without loss of generality, the forecasting results on workday Monday and non-workday Saturday are given. Figure 2 shows that the filter-divergence phenomenon occurred on both Monday and Saturday using the incorrect model with the S-DA system based on the KF method.

3. Methodology

Figure 3 shows the overall technical framework of the filter-divergence suppression approach based on the L₁-norm constraint in the KF assimilation method of the S-DA system.

Filter-Divergence Suppression Approach Based on L₁-norm Constraint:

The key to suppressing filter divergence is to calculate the proper weights of the state estimation and new measurements, that is, the Kalman gain matrix $\mathit{K}$. When filter divergence occurs, it means that the Kalman gain matrix $\mathit{K}$ is too small to balance the weight between the simulated and measured values when calculating the forecasting results. This also means that the new measurements have a weak correction effect on the state model during the assimilation process. Thus, we proposed an approach to calculate the proper Kalman gain matrix $\mathit{K}$ when divergence was about to occur by improving the ability of the new measured values to correct the state. The specific way to acquire the matrix $\mathit{K}$ in the proposed method was to force the measurement value $y^{cal}$ calculated from the assimilation model to be close to the true measured value $y^{obs}$.

Thus, the objective function is the minimum value between $y^{cal}$ and $y^{obs}$ based on the L₁-norm criterion. The L₁-norm and L₂-norm are commonly used as objective functions to be minimized [51]. However, objective functions based on the L₁-norm criterion have better resistance to noise. The L₁-norm constraint function for calculating the Kalman gain matrix $\mathit{K}$ is as follows:

$$min:{\Vert y_k^{cal}\left(\mathit{K}\right)-y_k^{obs}\Vert }_1$$

(10)

where $y^{cal}$ is the estimated measurement calculated from the state information, and $y^{obs}$ is the new measurement at the current interval [$\left(k-1\right)T,kT$].

Based on Table 1 and Equations (2), (7), and (8), $y^{cal}$ can be calculated as follows:

$$y_k^{cal}\left(\mathit{K}\right)=q_{pc}\left(k\right)={\mathit{H}}_k·x_k^a\left(\mathit{K}\right)·\overline{q_{pc}}\left(k\right)$$

(11)

where $x_k^a$ is an expression involving the $\mathit{K}$ matrix.

Combining Equations (10) and (11), the objective function can be further expressed as:

$$min:{\Vert {\mathit{H}}_k·x_k^a\left(\mathit{K}\right)·\overline{q_{pc}}\left(k\right)-y_k^{obs}\Vert }_1$$

(12)

To acquire optimal solution of the gain matrix $\mathit{K}$ when Equation (12) is minimum, the gradient projection method [52,53,54] or conjugate gradient method [55,56,57] can be used.

To determine whether filter divergence occurs, a more commonly used criterion is as follows [58]:

$$v_k{v}_k^T\le r·\left({\mathit{H}}_k{\mathit{P}}_k^f{\mathit{H}}_k{}^T+{\mathit{R}}_k\right)$$

(13)

where $v_k=y_k-{\mathit{H}}_k{x}_k^f$ and $r\ge 1$. Filter divergence will occur if Equation (13) is not satisfied.

4. Numerical Study

To verify the effectiveness of the filter suppression approach based on the L₁-norm constraint, two commonly used and effective methods for suppressing filter divergence were employed, namely the C-W and A-KF methods. The original KF method [34], C-W method [41], A-KF method [44], and proposed method based on the L₁-norm constraint were applied in the numerical example of filter-divergence suppression, as shown in Equation (4) and Figure 1. As selecting a weight or inflating factor in the C-W method is difficult, in following experiments, an adaptive inflation was used to acquire the weight or fading factor to inflate the model error covariance matrix [41]. The results were then compared and analyzed.

To display the results clearly, the cumulative absolute errors average (Cum-AEA) values of each method were given, calculated as follows:

$$Cum-AEA=\frac{1}{k}{\displaystyle {\displaystyle \sum }_{i=1}^k}\left|\widehat{x}\left(i\right)-x\left(i\right)\right|$$

(14)

where $\widehat{x}\left(i\right)$ denotes the forecasted traffic flow value, and $x\left(i\right)$ denotes the corresponding true value. The smaller the values of Cum-AEA, the better the forecasted results. Figure 4 shows the Cum-AEA values acquired using the four methods mentioned previously.

To evaluate the assimilation forecasting results, two commonly used evaluation criteria, the root mean square error (RMSE) [1,6,59,60] and the mean absolute percentage error (MAPE) [1,6], were used and were calculated as follows:

$$\{\begin{array}{l}\mathrm{RMSE}=\sqrt{\frac{1}{n}{\displaystyle {\displaystyle \sum }_{k=1}^n}(\widehat{x}\left(k\right)-x\left(k\right){)}^2}\\ \mathrm{MAPE}=\frac{1}{n}{\displaystyle {\displaystyle \sum }_{k=1}^n}\frac{\left|\widehat{x}\left(k\right)-x\left(k\right)\right|}{x\left(k\right)}\end{array}$$

(15)

where $\widehat{x}\left(k\right)$ denotes the forecasted traffic flow value, and $x\left(k\right)$ denotes the corresponding true value. The smaller the values of RMSE and MAPE, the better the forecasted results.

Table 2. Root mean square error (RMSE) and the mean absolute percentage error (MAPE, %) values of the assimilation results acquired using the four methods.

	RMSE	MAPE (%)
KF method	0.5647	31.81
C-W method	0.0368	0.16
A-KF method	0.0356	0.14
Proposed method	0.0340	0.12

displays the RMSE and MAPE (%) values of the assimilation results acquired from the original KF method, the C-W method, the A-KF, and the proposed method based on the L₁-norm constraint.

As shown in Figure 4 and listed in Table 2, a filter-divergence phenomenon occurred using the KF method. The corresponding RMSE and MAPE values were 0.5647 and 31.81%, respectively. Compared with the results from the KF method, filter divergence was suppressed to various degrees under the C-W method, the A-KF method, and the proposed method based on the L₁-norm constraint. The RMSE and MAPE values acquired from the proposed method were 0.0340 and 0.12%, respectively, which were the smallest values out of all methods. Compared with the C-W and A-KF methods, the RMSE values were reduced by 7.61% and 4.49%, respectively, and the MAPE values were reduced by 25% and 14.29%, respectively. This result indicates that the proposed method based on the L₁-norm constraint could suppress the filter divergence problem efficiently with the highest assimilation accuracy. To verify the applicability of the proposed method, an empirical study is presented in the next section.

5. Empirical Study

5.1. Study Area and Material Description

The datasets used in this study were downloaded from the Highways England website (highwaysengland.co.uk). The data were from a sub-area of the highway between Leeds and Sheffield, England, as shown in Figure 5a. Data from Monday to Sunday for each path were collected. The time interval for the data was 15 min. The data for each path used in the S-DA system contained eight days from consecutive weeks. As the mean traffic flow values are needed in the assimilation models shown in Equations (6) and (7), datasets of the former seven days were used for model construction in the S-DA system, and the data from the eighth day were employed to test the effectiveness of the proposed approach. Without loss of generality, traffic flow forecasting results from Monday to Sunday were acquired and analyzed. Furthermore, as traffic flow in the early morning and late night was low and of little concern to traffic management, the forecasting results from 6:00 a.m. to 9:00 p.m. were examined.

5.2. Test Design

In this section, the S-DA system based on the proposed L₁-norm constraint-based method was applied with the original KF method [34], the C-W method with an adaptive inflation [41], and the A-KF method [44] to acquire the traffic flow forecasting results of the paths shown in Figure 5a. The effectiveness of the proposed method was verified by comparing the results. To illustrate the analysis in detail, the forecasting performances of the six paths shown in Figure 5b are listed first.

6. Results and Discussion

Traffic flow predictions on workday Monday and non-workday Sunday for path 7078(LM862), which was part of the study area shown in Figure 5b, were computed first using the KF method, the C-W method, the A-KF method, and the proposed L₁-norm constraint-based method under the incorrect model shown in Equation (9) as an example to demonstrate the performance of the proposed method in detail and the results are shown in Figure 6 and Figure 7, respectively. The results presented showed that filter divergence occurred when only using the KF method under the incorrect model; however, the filter divergence was alleviated to different degrees after using the C-W, A-KF, and L₁-norm constraint-based methods. Furthermore, the true values were added to verify the effectiveness of these three filter-divergence suppression methods. Results showed that the best performance for filter-divergence suppression was obtained using the L₁-norm constraint-based method on both Monday and Sunday.

To evaluate the assimilation forecasting results, the two commonly used evaluation criteria shown in Equation (15) were used. The RMSE and MAPE values for the six paths shown in Figure 5b were acquired from the S-DA system based on the KF, C-W, A-KF, and L₁-norm constraint-based methods under the incorrect model and the KF method under the correct model (Equation (7)). The values are presented in Figure 8 and Figure 9, respectively. The RMSE and MAPE values calculated from the KF method using the correct model were set as a reference to verify the effectiveness of each filter-divergence suppression method. Figure 9 showed that the MAPE values of each path acquired using the KF method under the incorrect model reached almost 100%, which indicated that filter divergence occurred. The traffic flow forecasting performances improved to different degrees when using the C-W, A-KF, and L₁-norm constraint-based methods. The RMSE and MAPE values of each path on Monday to Sunday acquired using the C-W, A-KF, and L₁-norm constraint-based methods were smaller than those from the KF method under the incorrect model. Furthermore, compared with the results from the C-W and A-KF methods, the results obtained using the proposed L₁-norm constraint-based method were much closer to those from the KF method under the correct model. This indicates the effectiveness of the proposed L₁-norm constraint-based method.

The average RMSE and MAPE values of the forecasting results for the six paths shown in Figure 5b acquired using the KF (under both incorrect and correct models), C-W, A-KF, and L₁-norm constraint-based methods are listed in

Table 3. Average RMSE values for the six paths from Monday to Sunday.

	RMSE
	KF	C-W	A-KF	Proposed Method	KF (Correct Model)
Monday	842.65	394.20	152.55	85.37	77.38
Tuesday	829.18	340.98	178.67	85.69	74.59
Wednesday	857.94	346.18	164.94	88.85	80.74
Thursday	856.66	360.62	227.82	84.82	75.18
Friday	901.86	412.18	211.86	78.56	69.80
Saturday	584.92	210.12	134.08	41.29	39.22
Sunday	579.88	145.76	104.04	35.77	30.16

and

Table 4. Average MAPE values for the six paths from Monday to Sunday.

	MAPE (%)
	KF	C-W	A-KF	Proposed Method	KF (Correct Model)
Monday	99.56	39.67	15.31	7.23	6.76
Tuesday	99.71	35.72	17.99	7.91	6.82
Wednesday	99.70	34.62	15.15	7.58	7.00
Thursday	99.69	37.70	22.18	7.56	6.66
Friday	99.75	40.51	20.83	7.24	6.26
Saturday	99.65	35.94	22.05	6.70	6.15
Sunday	99.48	28.73	21.58	6.60	5.74

, respectively. Compared with the KF method under the incorrect model, the average RMSE and MAPE values decreased significantly, especially those obtained using the proposed L₁-norm constraint-based method. The best performance of the L₁-norm constraint-based method reduced the average RMSE by 823.3 (from 901.86 to 78.56), and the relative accuracy improved by 91.29%. The corresponding average MAPE decreased by 92.51% (from 99.75% to 7.24%).

Table 5. Differences in average RMSE and MAPE values between three filter-divergence suppression methods and KF method under the correct model.

	RMSE			MAPE (%)
	C-W	A-KF	Proposed Method	C-W	A-KF	Proposed Method
Monday	316.82	75.17	7.99	32.91	8.55	0.47
Tuesday	266.39	104.08	11.10	28.90	11.17	1.09
Wednesday	265.44	84.20	8.11	27.62	8.15	0.58
Thursday	285.44	152.64	9.64	31.04	15.52	0.90
Friday	342.38	142.06	8.76	34.25	14.57	0.98
Saturday	170.90	94.86	2.07	29.79	15.90	0.55
Sunday	115.60	73.88	5.61	22.99	15.84	0.86

presents the differences in the average RMSE and MAPE values for the three filter-divergence suppression methods under the incorrect model and the KF method under the correct model. The values in Table 5 show that the average RMSE and MAPE values acquired from the proposed L₁-norm constraint-based method were much closer to those from the KF method under the correct model than the values from the other two filter-divergence suppression methods. The smallest average RMSE and MAPE differences were 2.07% and 0.47%, respectively.

The traffic flows for all of the paths shown in Figure 5a predicted using the S-DA system based on the KF, C-W, A-KF, and proposed L₁-norm constraint-based methods were calculated.

Table 6. Average RMSE values of all paths from Monday to Sunday.

	RMSE
	KF	C-W	A-KF	Proposed Method	KF (Correct Model)
Monday	596.29	233.39	156.14	71.47	55.70
Tuesday	591.97	204.79	145.25	70.72	53.58
Wednesday	607.66	208.28	143.06	72.54	54.74
Thursday	606.60	213.69	161.17	95.87	52.45
Friday	649.17	252.80	222.20	68.37	50.28
Saturday	419.31	127.81	111.40	38.30	26.53
Sunday	414.71	102.03	96.47	36.77	23.62

and

Table 7. Average MAPE values of all paths from Monday to Sunday.

	MAPE (%)
	KF	C-W	A-KF	Proposed Method	KF (Correct Model)
Monday	98.39	31.56	21.61	9.34	7.12
Tuesday	98.88	28.13	19.98	9.63	7.19
Wednesday	98.43	27.77	19.08	9.57	7.07
Thursday	97.99	28.91	21.59	10.06	6.87
Friday	98.15	32.42	25.47	8.90	6.55
Saturday	98.50	26.12	22.56	8.86	6.42
Sunday	97.94	25.80	24.11	9.23	6.71

show the average RMSE and MAPE values, respectively, of each path from Monday to Sunday. The average MAPE values for all paths from Monday to Sunday obtained using the KF method under the incorrect model were all above 97%. This indicates that filter divergence occurred. The average MAPE values acquired from the C-W, A-KF, and L₁-norm constraint-based methods decreased by different amounts. For the sake of analysis, the forecasting results for all paths on workday Monday and non-workday Saturday were taken as examples. The best filter-divergence suppression performances were obtained using the L₁-norm constraint-based method. The average RMSE value from the L₁-norm constraint-based method decreased by 524.82 (from 596.29 to 71.47), and the relative accuracy improved by 88.01% on Monday compared with the results from the KF method under the incorrect model. The corresponding relative accuracy improved by 90.87% on Saturday. Furthermore, the average MAPE value from the L₁-norm constraint-based method decreased by 89.05% (from 98.39% to 9.34%) on Monday and 89.64% (from 98.50% to 8.86%) on Saturday compared with those obtained using the KF method under the incorrect model.

Table 8. Differences in the average RMSE and MAPE values between three filter-divergence suppression methods and the KF method under the correct model.

	RMSE			MAPE (%)
	C-W	A-KF	Proposed Method	C-W	A-KF	Proposed Method
Monday	177.69	100.44	15.77	24.44	14.49	2.22
Tuesday	151.21	91.67	17.14	20.94	12.79	2.44
Wednesday	153.54	88.32	17.80	20.70	12.01	2.50
Thursday	161.24	108.72	43.42	22.04	14.72	3.19
Friday	202.52	171.92	18.09	25.87	18.92	2.35
Saturday	101.28	84.87	11.77	19.70	16.14	2.44
Sunday	78.41	72.85	13.15	19.09	17.40	2.52

presents the differences in the average RMSE and MAPE values of all paths for the filter-divergence suppression methods under the incorrect model and the KF method under the correct model. The average RMSE and MAPE values acquired using the L₁-norm constraint-based method were much closer to those obtained using the KF method under the correct model. On workday Monday, the difference in the average RMSE and MAPE values between the C-W and KF methods under the correct model was 177.69 and 24.44%, respectively. The corresponding differences between the A-KF and KF methods were 100.44 and 14.49%, respectively. The differences in the average RMSE and MAPE values between the proposed L₁-norm constraint-based method and the KF method were 15.77 and 2.22% on Monday, respectively. Similar results were obtained for non-workday Saturday. The smallest average RMSE and MAPE differences were 11.77 and 2.44%, respectively, which were acquired using the L₁-norm constraint-based method.

Overall, the results in Table 3, Table 4, Table 5, Table 6, Table 7 and Table 8 suggest that in suppressing the filter divergence and improving the short-term traffic flow forecasting accuracy, the L₁-norm constraint-based method outperformed the C-W and A-KF methods. This indicates that using the proposed L₁-norm constraint-based method to suppress filter divergence is effective. Unlike suppressing the filter divergence phenomenon in the C-W method by adding an inflation factor to model the covariance matrix, in the proposed method, the gain matrix $\mathit{K}$ based on the actual conditions was directly and adaptively adjusted. The difficulty of selecting a weight or inflating factor and the propagation from the model or measurement error covariance to the $\mathit{K}$ matrix estimation in the C-W method were therefore also avoided. Moreover, compared with the A-KF method, this method was simpler and required less storage space. Furthermore, the proposed method adjusted the gain matrix directly without adjusting the model error covariance matrix. This prevented the uncertainties of the indirect operations in the A-KF method from affecting the assimilation results.

7. Conclusions

In this study, an approach for filter-divergence suppression in an S-DA system for short-term traffic flow prediction was proposed. The approach allowed for the simulated values from the state model to be close to the measured values when divergence was about to occur based on an L₁-norm criterion. The proposed L₁-norm constraint-based method was compared with two other commonly used methods, the C-W and A-KF methods, to suppress filter divergence in short-term traffic flow forecasting. The empirical results confirmed the following.

• The proposed approach based on the L₁-norm constraint can suppress the filter-divergence phenomenon that occurs in the KF assimilation method of the S-DA system.
• The proposed approach based on the L₁-norm constraint had a higher assimilation accuracy for suppressing filter divergence than the other two methods.

Therefore, the proposed filter-divergence suppression method based on the L₁-norm constraint is feasible and effective for filter-divergence suppression in an S-DA system for short-term traffic flow predictions. In future work, the proposed method is planned to be used in other fields to expand its range of applications.