Dense Vehicle Counting Estimation via a Synergism Attention Network

Abstract

Along with rising traffic jams, accurate counting of vehicles in surveillance images is becoming increasingly difficult. Current counting methods based on density maps have achieved tremendous improvement due to the prosperity of convolution neural networks. However, as highly overlapping and sophisticated large-scale variation phenomena often appear within dense images, neither traditional CNN methods nor fixed-size self-attention transformer methods can implement exquisite counting. To relieve these issues, in this paper, we propose a novel vehicle counting approach, namely the synergism attention network (SAN), by unifying the benefits of transformers and convolutions to perform dense counting assignments effectively. Specifically, a pyramid framework is designed to adaptively utilize the multi-level features for better fitting in counting tasks. In addition, a synergism transformer (SyT) block is customized, where a dual-transformer structure is equipped to capture global attention and location-aware information. Finally, a Location Attention Cumulation (LAC) module is also presented to explore the more efficient and meaningful weighting regions. Extensive experiments demonstrate that our model is very competitive and reached new state-of-the-art performance on TRANCOS datasets.

1. Introduction

Nowadays, the intelligent transportation system (ITS) has become an indispensable part of our daily lives with the popularity of vehicles and the crowding of all corners of the main traffic lanes. Among them, mitigating and solving traffic congestion on the road is the primary task that the ITS needs to face today, especially in urban regions. Traffic congestion often leads to lower productivity for citizens and pollution through the emission of gases into the environment. Therefore, estimating the number of intricate condition and overlapping vehicles takes a key role in controlling more severe traffic congestion in the future. Simultaneously, it is also preemptive work for guiding traffic flow optimization, such as by planning routes dynamically to avoid congested roads [1,2], controlling traffic lights automatically, and assisting in designing new transport corridors, to name just a few.

To implement effective counting, many traditional techniques have been proposed to extract a suitable area first and then count this area by handcrafted features such as HOG [3] and SIFT. However, these handcrafted features are susceptible to external conditions. Meanwhile, traffic surveillance devices that are equipped with these technologies are mostly deployed in the outdoor environment with diverse geographical locations, suffering severely from challenges [4,5,6] such as large perspective gaps and vehicle scale differences. As shown in Figure 1, the dense traffic image has multiple occlusions, such as pedestrian bridges, street lights, and road signs. Obviously, the vehicles closer to the horizon are difficult to observe because of their micro-scale in the scenes. Vehicle counting based solely on traditional techniques is fatigued for these reasons. In order to overcome these shortcomings, some researchers have paid attention to convolution neural networks (CNNs) [7,8,9] in recent years, which are brilliant in the computer vision discipline due to its powerful ability to implement potential hierarchical representations.

Approaches modeling the vehicle counting task as a regression map estimation have been systematically proven to be state of the art [11]. This allows the derivation of estimated vehicle density maps for occluded images. Many estimation networks [12,13] utilizing the convolution structure [14] as a backbone were fabricated for mapping between the original image and the regressed density one. Then, a density map can be employed to obtain the specific quantity of vehicles. For instance, Sooksatra et al. [15] designed a multi-level estimation network with regard to traffic maps under complicated scales. Even the latest work [16] proposed an adaptive density map generator to avoid errors caused by fixed Gaussian kernels for achieving better counting performance. These CNN-based methods have achieved remarkable improvements in the counting discipline, yet they are stuck in another dilemma: being constrained by the handcrafted design of an estimation network [17].

Accordingly, transformer models sprang up in the field of natural language-processing tasks [18] on account of their global self-attention mechanism. Recently, transformers have been successfully introduced into the visual field by the vision transformer [19,20]. Although the performance of transformers in up-down stream vision tasks can keep pace with or surpass CNNs [21], the application of transformers in the counting discipline is still in its infancy. As a pioneer work, Wang et al. [22] innovatively proposed an estimation network in which a CNN and transformer are connected in series. Furthermore, by considering global information to enhance feature association, Lin et al. [23] showed great dedication to designing a robust local attention mechanism, exploring the mapping between high-density regions and accounting for addressing the critical issue of large-scale variations. Undoubtedly, the significant effects of these models, which combine a CNN and transformer, authenticate their meliority. However, it is irrational for these approaches to utilize fixed-size attention and encode single-scale features as tokens directly. Unfortunately, single-scale tokens always fail to resolve the puzzle of sophisticated scale variations.

Inspired by these insights, we propose a novel synergism attention network (SAN) that merges the strengths of both CNNs and transformers in a new mode. First, in response to the aforementioned flaws in large-scale variations, we design an SAN stage block which adopts a pyramid structure to yield corresponding multi-scale feature maps. Different from the previous models that generate fixed-size CNN outputs, the SAN stage block can produce feature maps of different receptive fields step by step. Consequently, the structure provides a reliable way to adapt to the dramatic scale variations in traffic images and achieves better counting performance.

Secondly, we design a synergism transformer (SyT) module which adopts two branches (i.e., a multi-head global branch and multi-head mask weighted branch) as the backbone. To be specific, the two divisions focus particular attention on different aspects to complement each other. While the former utilizes a multi-head attention mechanism to produce global information, the latter focuses on the relevance of the local location region. On this basis, we realize a location attention mechanism that employs the Location Attention Cumulation (LAC) module, concentrating on the most relevant information in a single-scale range. In addition, a penalty of location attentional deviation is imposed for attaining great training results. Eventually, our algorithm achieves state-of-the-art performance in both qualitative and quantitative experiments. The following are the main innovations and contributions:

• A novel network architecture, namely the synergism attention network, is proposed that adopts a pyramid structure to suit large-scale variations and effectively unify the merits of both CNNs and transformers;
• A synergism transformer (SyT) module is devised to incorporate the global information and location region attention, boosting the ability to allocate the appropriate weights to salient targets;
• These modules are delicately assembled, whose results on the TRANCOS dataset show that we reach new state-of-the-art performance both qualitatively and quantitatively.

2. Methodology

In this section, we elaborate on the proposed synergism attention network, which includes two main assemblies: the synergism transformer and the SAN stage. A detailed network diagram is shown in Figure 2. In the following, we describe in detail the SAN framework in Section 2.1, synergism transformer in Section 2.2, and the loss in Section 2.3.

2.1. Framework Overview

An overview of the framework is given in Figure 2. The dense traffic image is first put into the SAN stage block. For each input image ${\mathit{F}}_{in}\in {\mathcal{R}}^{C\times H\times W}$, the SAN stage block utilizes a series of convolution kernels to extract feature maps ${\mathit{F}}_{c1}\in {\mathcal{R}}^{C_1\times H/2\times W/2}$ and configure the convolution g_stride=2 with a particular stride equal to two for image downsampling to obtain the low-resolution features. The formula for this is given as follows:

$$\begin{array}{c}\hfill {\mathit{F}}_{c1}=\mu \left(G\left(g_{stride=2}\left({\mathit{F}}_{in}\right)\right)\right),\end{array}$$

(1)

where G(·) is the mapping function implemented by three convolution layers and μ(·) is the activation function.

Inspired by the PVT [24], we design multiple SAN stage blocks to reduce the transformer’s computational burden and generate numerous scale features (e.g., ${\mathit{F}}_{c2}\in {\mathcal{R}}^{C_2\times H/4\times W/4}$, ${\mathit{F}}_{c3}\in {\mathcal{R}}^{C_3\times H/8\times W/8}$ and ${\mathit{F}}_{c4}\in {\mathcal{R}}^{C_4\times H/16\times W/16}$ to search for the corresponding dense image objects adaptively. Subsequently, the elementswill enter the dual-branch synergismtransformer to gain hybrid attention that ismore conducive to counting tasks. Finally, a regression layer is utilized to estimate the densitymap.

2.2. Synergism Transformer

Global Self-Attention. After acquiring SAN stage block results at different hierarchies, the synergism transformer starts to work by flattening the features to calculate the self-attention. Among them, the traditional self-attention [19] branch considers the paired relationship between the input and output position and yields the global information of the current feature:

$${\mathit{S}}_{att}^G=softmax(\frac{\left({\mathit{Q}}_{\mathit{r}}{W}_1^Q\right){\left({\mathit{K}}_{\mathit{r}}{W}_1^K\right)}^T}{\sqrt{d}})\left({\mathit{V}}_{\mathit{r}}{W}_1^V\right),$$

(2)

where $softmax(·)$ denotes the softmax function, $W_1^Q$, $W_1^K$, and $W_1^V$ are the weight metrics, and ${\mathit{Q}}_{\mathit{r}}$, ${\mathit{K}}_{\mathit{r}}$, and ${\mathit{V}}_{\mathit{r}}$ represent the query, key, and value vectors from the source input, respectively.

Location Multi-Head Attention. The SAN stage mentioned above resolves the problem of inefficient learning of cross-scale spatial information, but there still exists the issue that the preset attention does not match the counting task. Therefore, we employ a modified transformer [23], namely location multi-head attention (LMHA), as one of the two branches due to its success in seeking location attention. Specifically, we apply a distinct filtering mechanism to push the features to learn the most appropriate rectangle-sized window.

First, given an input feature ${\mathit{F}}_{ci}$ ($i\in \{1,2,3,4\}$), LMHA maps them separately to query ${\mathit{Q}}_{\mathit{i}}$, key ${\mathit{K}}_{\mathit{i}}$, and value ${\mathit{V}}_{\mathit{i}}$ vectors. Among them, ${\mathit{V}}_{\mathit{i}}$ and ${\mathit{V}}_{\mathit{r}}$ share parameters. Then, the two-head attention maps can be computed by

$$\begin{array}{c}\hfill {\mathit{S}}_{att}^{L1}=softmax(\frac{\left({\mathit{Q}}_{\mathit{i}}{W}_2^Q\right){\left({\mathit{K}}_{\mathit{i}}{W}_2^K\right)}^T}{\sqrt{d}}),\\ \hfill {\mathit{S}}_{att}^{L2}=softmax(\frac{\left({\mathit{Q}}_{\mathit{i}}{W}_3^Q\right){\left({\mathit{K}}_{\mathit{i}}{W}_3^K\right)}^T}{\sqrt{d}}),\end{array}$$

(3)

where ${\mathit{S}}_{att}^{L1}$ and ${\mathit{S}}_{att}^{L2}$ denote different attention maps and $W_2^Q$, $W_2^K$, $W_3^Q$, and $W_3^K$ are trainable weight matrices.

Second, an LAC module is established for the purpose of selecting an appropriate region, focusing more on the imperceptible scale of the corresponding object and achieving more precise counting. In this regard, ${\mathit{S}}_{att}^{L1}$ and ${\mathit{S}}_{att}^{L2}$ behave as the input, which leads to the following formula:

$${\mathit{S}}_{att}^L=\mathit{LAC}({\mathit{S}}_{att}^{L1},{\mathit{S}}_{att}^{L2}).$$

(4)

The discussion of the $\mathit{LAC}(·)$ function will be given in more detail in the next subsection.

Location Attention Cumulation. For obtaining a two-dimensional attention map, ${\mathit{S}}_{att}^{L1}$ and ${\mathit{S}}_{att}^{L2}$ need to be reshaped into three-order shapes first. Then, they are divided into multiple feature maps along the first axis. Generally, determining a rectangular region requires the intersection of two areas. In other words, we choose a filter function, namely Location Attention Cumulation (LAC), to find two rectangular areas along opposite directions and then select the overlapping rectangular regions as the location attention. More specifically, we define a position $\mathit{M}$ in $(X_M,Y_M)$ where $0\le X_M\le H$ and $0\le Y_M\le W$. LAC can finally be expressed as

$${\mathit{F}}_{\mathit{LAC}}^{d1}(M,{\mathit{S}}_{att}^{L1})=\sum _{X_a=0,Y_a=0}^{H,W}\sum _{X_a\le X_M}\sum _{Y_a\le Y_M}{\mathit{S}}_{att}^{L1}(X_a,Y_a),$$

(5)

where $d1$ represents the direction from one end to the diagonal and ${\displaystyle \sum _{X_a=0,Y_a=0}^{H,W}}$ denotes the accumulation of attention scores from $(0,0)$ to $(H,W)$. Roughly, Figure 3 depicts the entire procedure of this module. After obtaining an area where attention spreads in a fixed direction, we need to concentrate on ${\mathit{F}}_{\mathit{LAC}}^{d2}$ maps spreading in the opposite direction in ${\mathit{S}}_{att}^{L2}$, and the formula is as follows:

$${\mathit{F}}_{\mathit{LAC}}^{d2}(M,{\mathit{S}}_{att}^{L2})=\sum _{X_a=H,Y_a=W}^{0,0}\sum _{X_a\ge X_M}\sum _{Y_a\ge Y_M}{\mathit{S}}_{att}^{L2}(X_a,Y_a).$$

(6)

Thus, we finally obtain the location attention by overlapping two known regions:

$${\mathit{S}}_{att}^l={\mathit{F}}_{\mathit{LAC}}^{d2}(M,{\mathit{S}}_{att}^{L2})\ast {\mathit{F}}_{\mathit{LAC}}^{d1}(M,{\mathit{S}}_{att}^{L1}),$$

(7)

where ∗ is a Hadamard product. Obviously, the ${\mathit{S}}_{att}^l$ we calculate only contains a single-scale object, which may encounter the phenomenon of assigning too many zeros to other positions and resulting in few overlapped regions. To avoid this issue, we perform opposite operations directly on the original map while calculating our cumulation attention scores and then add them to ${\mathit{S}}_{att}^l$, obtaining the required rectangular region no matter which direction the diffusion starts from. Therefore, we reset Equation (7) to be the following:

$${\mathit{S}}_{att}^L={\mathit{F}}_{\mathit{LAC}}^{d2}(M,{\mathit{S}}_{att}^{L2})\ast {\mathit{F}}_{\mathit{LAC}}^{d1}(M,{\mathit{S}}_{att}^{L1})+{\mathit{F}}_{\mathit{LAC}}^{d1}(M,{\mathit{S}}_{att}^{L2})\ast {\mathit{F}}_{\mathit{LAC}}^{d2}(M,{\mathit{S}}_{att}^{L1}).$$

(8)

Finally, after obtaining the results of two SyT branches, we combine the scores for both location attention and global self-attention, and the output can be computed as follows:

$${\mathit{F}}_{out}=((softmax(\frac{\left({\mathit{Q}}_{\mathit{i}}{W}_3^Q\right){\left({\mathit{K}}_{\mathit{i}}{W}_3^K\right)}^T}{\sqrt{d}})+softmax(\frac{\left({\mathit{Q}}_{\mathit{i}}{W}_2^Q\right){\left({\mathit{K}}_{\mathit{i}}{W}_2^K\right)}^T}{\sqrt{d}}))\ast {\mathit{S}}_{att}^L+{\mathit{S}}_{att}^G)\left(\left({\mathit{V}}_{\mathit{i}}{W}^V\right)\right).$$

(9)

2.3. Loss

In this work, we typically model the counting task as an estimation of a regression density map. Therefore, we cannot directly utilize the annotation of the TRANCOS dataset [10] as the ground truth. In other words, we need to first convert the dot-level annotation of this dataset into a density map through a Gaussian kernel function to supervise the training process. In this setting, for each traffic image ${\mathit{F}}_{in}$, the ground truth density map ${\mathit{Gt}}_d$ can be represented as follows [25]:

$$\mathit{Gt}{}_d\left(q\right)=\sum _{\eta \in \mathit{C}}Gaussian(q,\eta ,\lambda ),$$

(10)

where q denotes a position in ${\mathit{F}}_{in}$ and $Gaussian(·)$ is a normalized two-dimensional Gaussian function. The point set of all inputs is defined as $\mathit{C}$, $\eta$ is the mean value, and $\lambda$ denotes an isotropic covariance matrix. With the density map $\mathit{Gt}{}_d$, the Euclidean distance is utilized to measure the difference between the ground truth density map and the predicted one, whose formulation can be written as follows:

$$\mathit{Loss}=\frac{1}{2N}\sum _{j=1}^N{∥\mathit{SAN}\left({\mathit{F}}_{in}^j\right)-{\mathit{Gt}}_d^j∥}_2^2,$$

(11)

where N is the total number of training images, ${\mathit{F}}_{in}^j$ denotes the jth input image, and ${\mathit{Gt}}_d^j$ is the ground truth density map of ${\mathit{F}}_{in}^j$, while $\mathit{SAN}(·)$ stands for the above synergism attention network.

Aside from that, in order to ensure that location multi-head attention can frame the appropriate object and determine that the generated rectangle shape is not too large or too small, a loss $\mathit{L}\mathit{1}$ is further appended to regulate LMHA, which can be formulated as follows:

$$\mathit{L}\mathit{1}=\sum _{t=0}^{WH}(1-\frac{E_t^T\dot{E}}{∥E_t^T∥∥\dot{E}∥}),$$

(12)

where $\dot{E}$ denotes the mean allocation of the location attention values in all regions. Finally, $\mathit{L}\mathit{1}$ is considered together with $\mathit{Loss}$ to achieve better optimization.

3. Experiment

In this section, extensive experiments are performed on the TRANCOS dataset to exhibit the effectiveness and superiority of the proposed model. Across the entire model, we utilized the density maps from the dataset annotation as the ground truth (see Equation (10)). For model optimization, a stochastic gradient descent (SGD) method was employed in the training process with the learning rate set constantly.

3.1. Dataset

We used TRANCOS, a recent dataset containing vehicle images of heavy traffic, to train, validate, and test our model. More specifically, this dataset provides a set of 1244 static traffic images from different perspectives of surveillance cameras, with a total of 46,796 cars labeled. It is worth noting that traffic scenes under diverse surveillance conditions (e.g., under illumination and from multiple perspectives) are challenging for vehicle-counting tasks. In the preprocessing phase, we followed a common partitioning scheme for the training set, test set, and validation set. The train samples had 403 scenes, while the validation and test samples had 420 and 421 scenes, respectively. For better generalization performance, some data augmentation approaches were adopted. Specifically, we generated a random cropping and a random flip per image for each epoch.

3.2. Experimental Settings

All experiments were conducted using the Pytorch framework with a batch size of eight. Figure 4 shows the convergence speed of the SAN, in which the ordinate denotes the changing loss values against the epochs shown in the abscissa. Evidently, the loss dropped rapidly in the initial phase. With 150 epochs reached, though certain fluctuations still existed, the overall loss values tended to be stable. The learning rate of the optimizer was fixed at $1\times {10}^{-4}$. In order to numerically compare the estimation performance of our approach with other competing methods, the mean absolute error (MAE) and the mean squared error (MSE) were used as evaluation metrics. Specifically, their computational equations can be written as follows:

$$\begin{array}{cc}\hfill & MAE=\frac{1}{N}\sum _{i=1}^N\left|{\mathit{F}}_i^{pred}-{\mathit{F}}_i^{GT}\right|,\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill & MSE={(\frac{1}{N}\sum _{i=1}^N{\left|{\mathit{F}}_i^{pred}-{\mathit{F}}_i^{GT}\right|}^2)}^{\frac{1}{2}},\hfill \end{array}$$

(14)

where ${\mathit{F}}_i^{pred}$ denotes the predicted count of the ith traffic image, ${\mathit{F}}_i^{GT}$ denotes the corresponding ground truth count of the ith traffic image, and N is the number of test set images. The proposed approach was compared with five recently reported state-of-the-art models:

-

MCNN: Single-image crowd counting via a multi-column convolutional neural network, proposed by Zhang et al. [13];

-

P2PNet: Rethinking counting and localization in crowds through a purely point-based framework, proposed by Song et al. [26];

-

DMcount: Distribution matching for crowd counting, proposed by Wang et al. [27];

-

CCNN: An effort toward perspective-free object counting with deep learning, proposed by Daniel et al. [25];

-

Tracount: A deep convolutional neural network for highly overlapping vehicle counting, proposed by Shiv [28].

3.3. Quantitative Comparisons

At the testing phase, the images were divided and categorized into five spans based on the number of vehicles: 0–10, 11–20, 21–30, 31–40, and 41–50. According to the strategy of density division, we can intuitively observe from

Table 1. Numerical comparison with multiple methods on TRANCOS datasets (where ↑Higher is better and ↓Lower is better). The best values are highlighted in boldface, and the second-best values are highlighted by underlines.

Method		0–10	11–20	21–30	31–40	41–50
MAE (↓)	MCNN	2.51	8.74	6.15	9.23	15.28
	P2PNet	2.08	2.87	5.53	7.64	12.15
	DMcount	2.12	3.17	5.65	8.04	11.12
	Tracount	8.07	6.15	7.15	8.44	12.89
	CCNN	6.35	9.74	10.25	12.11	13.79
	Our SAN	1.98	3.23	5.08	7.42	10.97
MSE (↓)	MCNN	4.22	15.81	11.31	15.94	23.83
	P2PNet	3.16	5.68	9.72	12.91	20.73
	DMcount	4.39	5.72	10.51	11.65	20.91
	Tracount	15.18	9.16	10.33	14.43	21.17
	CCNN	9.92	16.07	19.27	22.65	24.04
	Our SAN	3.32	5.52	9.48	11.27	18.81

that with the increasing number of vehicles in the images, both the MAE and MSE evaluation metrics showed a significant upward trend. This is reasonable, since more target materials often lead to easier capturing of the required features. It is encouraging that whenever the density of vehicles was low or high, we could easily find that our SAN ranked at the top or near the top in terms of performance. From an overall perspective, specific to the scores for the MAE (MSE), the average improvement by the SAN over the MCNN, P2PNet, DMcount, TraCount, and the CCNN was 2.644 (4.542), 0.24 (0.76), 0.284 (0.956), 2.804 (4.374), and 4.712 (7.51), respectively. These promotions are exhilarating, as these baseline methods were proven to achieve state-of-the-art performance in their respective articles. Among different spans, the SAN attained the optimal results in most cases, while P2PNet obtained the second-best values at most. Moreover, as the number of vehicles rose, the performance gap between our SAN and P2PNet became increasingly clear.

Moreover, we compared the running efficiency of different models. The results are shown in

Table 2. Parameter comparison with multiple methods. The best values are highlighted in boldface.

Method	Tracount	P2PNet	DMcount	MCNN	Our SAN
Params	20.51 M	19.22 M	21.50 M	-	30.95 M
FLOPs	24,063.22 M	20,030.47 M	20,679.74 M	1350.84 M	20,274.82 M

. Due to the introduction of the latest transformer as a framework, the parameter size of our model reached 30.95 M, which was larger than the competitors. However, in terms of the computational criteria of FLOPs, the SAN reached a medium level. Considering the outstanding performance achieved by our model, we believe that a certain sacrifice of a more demanding parameter is acceptable.

In summary, we evaluated our methods in a quantitative fashion. The results demonstrate that the superiority of the SAN surpassed existing algorithms, further proving the effectiveness of the proposed modules.

3.4. Qualitative Comparisons

In this subsection, to further verify the counting performance of our model, we implement a visual operation on the density map generated by all competing models. The results are given in Figure 5, in which the main color of the density maps is based on blue and green, representing the density position of the vehicle object. Technically, it should be noted that the more vehicle targets the unit scene contained, the redder the color would be.

For the first column in Figure 5, when compared with the ground truth, it is obvious that more high-density areas existed in the whole image. Among them, the object located in the lower right area was not accurately predicted, and the object located on the left part was only partially detected. Clearly, we can claim that the MCNN failed in this experiment, presenting imprecise density representation compared with its competitors. Although P2PNet and DMcount generated more similar density maps to the ground truth, there were still phenomena that were not fine enough with a careful look and only gave approximately equal results in the qualified region. Aside from that, the density map generated by the Tracount method showed unsatisfactory results in both the dense and sparse regions. Comparatively, the results derived from our SAN method were much closer to the GT image and were more credible than all the state-of-the-art competitors; that is to say, the SAN successfully captured the target features of different scales and retained better details. Evidently, these qualitative results prove the superiority of our proposed modules.

4. Discussion

Unlike most previous studies, the SAN is more concerned with the variable object scale in the counting task. Note that any singleton network architecture, whatever the convolutions or transformers are, can only explore the global or local relationships of the input data. Accordingly, we devised a novel network structure equipped with dual branches that can adapt to the large-scale variations of the surveillance targets. In terms of the current situation in the field, the single data type, the low volume of dataset, and the poor annotation all become the shackles of algorithms achieving better performance.

In future works, we plan to explore a lightweight network structure where the defect of a large number of parameters can be relieved. A rough idea lies in the adoption of 1 × 1 convolution for a unified operation of partial dual-branch operations. In addition, our SAN would be shifted into other datasets to check if it is highly generalizable. That aside, we would also validate the efficacy and efficiency of the proposed components in other visual applications.

5. Conclusions

In this paper, we proposed a novel synergism attention network to enhance the ability of transformers in disposing of large-scale variations for dense vehicle counting. Mainly, a pyramid structure was proposed to capture multiple-scale object features. On that basis, a lightweight downscaling operation was implemented from an SAN’s multi-stage block, and the attention features that blend global information and location-aware local information from the synergism transformer block were obtained. Among them, to gain local trainable location-aware information in SyT, we devised a Location Attention Cumulation module to realize the weighted localization of the region by utilizing the diffusion of attention accumulation. Extensively quantitative and qualitative comparisons were conducted which proved the superiority of our proposed SAN in dense vehicle counting tasks.