# Pedestrian Detection under Parallel Feature Fusion Based on Choquet Integral

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Features Realignment

#### 2.1. Histogram of Oriented Gradient Feature Extraction

- Standardize gamma space and color space.In order to reduce the influence of illumination, the whole image needs to be normalized. In the texture intensity of the image, the local surface exposure contributes a large proportion, so this kind of compression can effectively reduce the local shadow and illumination changes of the image. As the color information has little effect, the original RGB image is usually converted to a gray image, and the gamma correction is used to normalize it by formula$$I(x,y)=I{(x,y)}^{\gamma}$$
- Calculate image gradient.The gradient of horizontal direction ${G}_{x}(x,y)$ and the gradient of vertical direction ${G}_{y}(x,y)$ are, respectively, calculated for the normalized image.$${G}_{x}(x,y)=I(x+1,y)-I(x-1,y)$$$${G}_{y}(x,y)=I(x,y+1)-I(x,y-1)$$The gradient value $G(x,y)$ and gradient direction $\theta (x,y)$ of each pixel are calculated from the gradient of the two directions, respectively.$$G(x,y)=\sqrt{{G}_{x}{(x,y)}^{2}+{G}_{y}{(x,y)}^{2}}$$$$\alpha (x,y){=\mathrm{tan}}^{-1}\left(\frac{{G}_{y}(x,y)}{{G}_{x}(x,y)}\right)$$
- Construct the histogram of gradient direction for each cell.The image is divided into several cells, as shown in Figure 3a, each cell is $8\times 8$ pixels. The gradient direction of 360 degrees is divided into nine ranges averagely (Figure 3b), and the histogram corresponding to these nine bins is constructed to count the gradient information of the $8\times 8$ pixels. The horizontal axis of the histogram is the nine bins in gradient directions, while the height of each bin is the superposition of the gradient value of those pixels whose gradient directions belong to the bin.
- Construct the HOG feature for an image.Each cell gets a 9-dimensioanl vector. As shown in Figure 3a, four adjacent cells constitute a block, and the vectors of four cells in a block are connected in serial to obtain a 36-dimensional vector. The block is used to scan the image with the scanning step as a cell. Finally, the vectors of all blocks are connected in serial to get the HOG feature of the image. For example, for an $128\times 64$ image, every $8\times 8$ pixel constitutes a cell and every $2\times 2$ cells constitute a block. As each cell has nine features, there are $4\times 9=36$ features in each block. Taking eight pixels as the step size, there will be seven scanning windows in the horizontal direction and 15 scanning windows in the vertical direction. In other words, $128\times 64$ images have $36\times 7\times 15=3780$ features.

#### 2.2. Histogram of LBP Descriptor

## 3. Feature Fusion in Parallel by Choquet Integral

#### 3.1. Signed Fuzzy Measure

**Definition**

**1.**

#### 3.2. Choquet Integral as Aggregation Tool

**Definition**

**2.**

#### 3.3. Feature Fusion by Choquet Integral

## 4. Pedestrian Detection Framework with Parameters Retrieved by Genetic Algorithm

#### 4.1. Parameters Retrieving under Genetic Algorithm Framework

#### 4.2. Classifier Training

#### 4.3. Classifier Training and Evaluation Criterion

- The actual value is true, and the classifier assigns it to be positive (True Positive = TP);
- The actual value is true, and the classifier assigns it to be negative (False Negative = FN);
- The actual value is false, and the classifier assigns it to be positive (False Positive = FP);
- The actual value is false, and the classifier assigns it to be negative (False Negative = TN).

## 5. Experimental Results and Analysis

#### 5.1. Data Construction

#### 5.2. Experimental Results and Analysis

- SVM classifier with HOG features, denoted as HOG-SVM;
- SVM classifier with serial fusion of HOG and LBP features, denoted as HOG-LBP-SVM;
- SVM classifier with parallel-HOG-HOLBP features whose fusion parameters are set by experience, denoted as HOG-HOLBP-SVM;
- SVM classifier with parallel-HOG-HOLBP features whose fusion parameters are optimized by GA process, denoted as HOG-HOLBP-GA-SVM.

## 6. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Zhao, Z.; Zheng, P.; Xu, S. Object detection with deep learning: A review. IEEE Trans. Neural Netw. Learn. Syst.
**2019**, 30, 3212–3232. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Dollar, P.; Wojek, C.; Schiele, B.; Perona, P. Pedestrian detection: An evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell.
**2012**, 34, 743–761. [Google Scholar] [CrossRef] [PubMed] - Dollar, P.; Appel, R.; Belongie, S.; Perona, P. Fast feature pyramids for object detection. IEEE Trans. Pattern Anal. Mach. Intell.
**2014**, 36, 1532–1545. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Liu, S.Y.; Guo, H.Y.; Hu, J.G.; Zhao, X.; Tang, M. A novel data augmentation scheme for pedestrian detection with attribute preserving GAN. Neurocomputing
**2020**, 401, 123–132. [Google Scholar] [CrossRef] - Severino, J.V.B.; Zimmer, A.; Brandmeier, T.; Freire, R.Z. Pedestrian recognition using micro Doppler effects of radar signals based on machine learning and multi-objective optimization. Expert Syst. Appl.
**2019**, 136, 304–315. [Google Scholar] [CrossRef] - Liu, W.; Liao, S.; Ren, W.; Hu, W.; Yu, Y. High-level Semantic Feature Detection: A New Perspective for Pedestrian Detection. CVPR
**2019**, 5187–5196. [Google Scholar] - Cao, Z.; Simon, T.; Wei, S.-E. Realtime multi-person 2D pose estimation using part affinity fields. CVPR
**2017**, 2017, 1302–1310. [Google Scholar] - Yang, Z.; Nevatia, R. A multi-scale cascade fully convolutional network face detector. In Proceedings of the 23rd International Conference on Pattern Recognition, Cancun, Mexico, 4–8 December 2016; IEEE Computer Society Press: New York, NY, USA, 2016; pp. 633–638. [Google Scholar]
- Sumi, A.; Santha, T. Frame level difference (FLD) features to detect partially occluded pedestrian for ADAS. J. Sci. Ind. Res.
**2019**, 78, 831–836. [Google Scholar] - Yan, Y.C.; Ni, B.B.; Liu, J.X.; Yang, X.K. Multi-level attention model for person re-identification. Pattern Recognit. Lett.
**2019**, 127, 156–164. [Google Scholar] [CrossRef] - Li, G.; Yang, Y.; Qu, X. Deep Learning Approaches on Pedestrian Detection in Hazy Weather. IEEE Trans. Ind. Electron.
**2020**, 10, 8889–8899. [Google Scholar] [CrossRef] - Dalal, N.; Triggs, W. Histograms of Oriented Gradients for Human Detection. CVPR
**2005**, 1, 886–893. [Google Scholar] - Ojala, T.; Pietikäinen, M.; Harwood, D. Performance evaluation of texture measures with classification based on Kullback discrimination of distributions. In Proceedings of the 12th IAPR International Conference on Pattern Recognition, Jerusalem, Israel, 9–13 October 1994; IEEE Computer Society Press: New York, NY, USA, 1994; Volume 1, pp. 582–585. [Google Scholar]
- Zhang, S.; Bauckhage, C.; Cremers, A.B. Informed Haar-like features improve pedestrian detection. CVPR
**2014**, 947–954. [Google Scholar] - Wang, X.; Han, T.X.; Yan, S. An HOG-LBP human detector with partial occlusion handling. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Lisbon, Portugal, 5–8 February 2009; IEEE: New York, NY, USA, 2009; pp. 32–39. [Google Scholar]
- Wei, Y.; Tian, Q.; Guo, J. Multi-vehicle detection algorithm through combining Harr and HOG features. Math. Comput. Simul.
**2019**, 155, 130–145. [Google Scholar] [CrossRef] - Yang, Z.; Kurita, T. Improvements of local descriptors in HOG/SIFT by BOF approach. IEICE Trans. Inf. Syst.
**2014**, E96-D, 1293–1303. [Google Scholar] [CrossRef] [Green Version] - Wang, L.; Guo, C.; Liu, J.; Meng, D. A novel learning-balsed frame pooling method for event detection. Signal Process.
**2017**, 140, 45–52. [Google Scholar] [CrossRef] [Green Version] - Alonso, I.P.; Fernández-Llorca, D.; Sotelo, M.A.; Bergasa, L.M.; Revenga, P.A.; Nuevo, J.; Ocaña, M.; Garrido, M.Á.G. Combination of feature extraction methods for SVM pedestrian detection. IEEE Trans. Intell. Transp. Syst.
**2007**, 8, 292–307. [Google Scholar] [CrossRef] - Bilal, M.; Hanif, M.S. High performance real-time pedestrian detection using light weight features and fast cascaded kernel SVM classification. J. Signal Process. Syst.
**2019**, 91, 117–129. [Google Scholar] [CrossRef] - Cao, J.; Pang, Y.; Li, X. Leaning multilayer channel features for pedestrian detection. IEEE Trans. Image Process.
**2017**, 26, 3210–3220. [Google Scholar] [CrossRef] [Green Version] - Li, G.; Zong, C.; Liu, G.; Zhu, T. Application of convolutional neural network (CNN)-Adaboost algorithm in pedestrian detection. Sens. Mater.
**2020**, 32, 1997–2006. [Google Scholar] [CrossRef] - He, Y.-Q.; Qin, Q.; Josef, V. A pedestrian detection method using SVM and CNN multistage classification. J. Inf. Hiding Multimed. Signal Process.
**2018**, 9, 51–60. [Google Scholar] - Kim, J.H.; Batchuluun, G.; Park, K.R. Pedestrian detection based on faster R-CNN in nighttime by fusing deep convolutional features of successive images. Expert Syst. Appl.
**2018**, 114, 15–33. [Google Scholar] [CrossRef] - Albiol, A.; Monzo, D.; Martin, A.; Sastre, J.; Albiol, A. Face recognition using HOG-EBGM. Pattern Recognit.
**2008**, 29, 1537–1543. [Google Scholar] [CrossRef] - Tudor, B. Pedestrian detection and tracking using temporal differencing and HOG features. Comput. Electr. Eng.
**2014**, 40, 1072–1079. [Google Scholar] - Hoang, V.-D.; Le, M.-H.; Jo, K.-H. Hybrid cascade boosting machine using variant scale blocks HOG features for pedestrian detection. Neurocomputing
**2014**, 135, 357–366. [Google Scholar] [CrossRef] - Zhu, Q.; Yeh, M.C.; Cheng, K.T. Fast human detection using a cascade of histograms of oriented gradients. CVPR
**2006**, 2, 1491–1498. [Google Scholar] - Maggiani, L.; Bourrsset, C.; Quinton, J.C.; Berry, F.; Sérot, J. Bio-inspired heterogeneous architecture for real-time pedestrian detection applications. J. Real Time Image Process.
**2018**, 14, 535–548. [Google Scholar] [CrossRef] [Green Version] - Hong, G.S.; Kim, B.G.; Hwang, Y.S.; Kwon, K.K. Fast multi-featured pedestrian detection algorithm based on histogram of oriented gradient using discrete wavelet transform. Multimed. Tools Appl.
**2015**, 75, 15229–15245. [Google Scholar] [CrossRef] - Wu, S.; Payeur, R.P. Improving pedestrian detection with selective gradient self-similarity feature. Pattern Recognit.
**2015**, 48, 2364–2376. [Google Scholar] [CrossRef] - Ojala, T.; Pietikäinen, M.; Harwood, D. A Comparative Study of Texture Measures with Classification Based on Feature Distributions. Pattern Recognit.
**1996**, 29, 51–59. [Google Scholar] [CrossRef] - Jiang, Y.; Tong, G.; Yin, H.; Xiong, N. A pedestrian detection method based on genetic algorithm for optimize XGBoost training parameters. IEEE Access
**2019**, 7, 118310–118321. [Google Scholar] [CrossRef] - Wang, Z.; Klir, G.K. Fuzzy Measure Theory; Plenum Press: New York, NY, USA, 1992. [Google Scholar]
- Wang, Z.; Yang, R.; Leung, K.-S. Nonlinear Integrals and Their Applications in Data Mining; World Scientific: Singapore, 2010. [Google Scholar]
- INRIA Data Set. Available online: http://pascal.inrialpes.fr/data/human/ (accessed on 20 November 2020).

**Figure 2.**The demonstration of parallel fusion of HOG (histogram of oriented gradient) and LBP (local binary pattern) features.

**Figure 3.**Illustration of HOG feature extraction: (

**a**) cells and blocks of HOG feature; (

**b**) nine ranges of gradient direction in 360 degrees.

**Figure 10.**The results of 10 trials for different combinations of signed fuzzy measure in HOG-HOLBP-SVM.

**Figure 11.**Performance comparison among four algorithms on the testing set: (

**a**) comparison from precision; (

**b**) comparison from recall; (

**c**) comparison from F1 score.

Parameter | Value | Mark |
---|---|---|

C | 0.5 | Penalty parameters for wrongly classified samples |

Tol | 1e-4s | Criteria for stopping iteration |

Multi_class | ovr | Multiclass classification strategy parameters |

Class_weight | balanced | Adjust weights based on frequency of each class |

Max_iter | 1000 | The maximum iteration number |

Loss | squared_hinge | Loss function type |

Actual Value | Prediction Value | |
---|---|---|

Positive | Negative | |

True | 980 | 146 |

False | 50 | 403 |

Actual Value | Prediction Value | |
---|---|---|

Positive | Negative | |

True | 1056 | 70 |

False | 44 | 409 |

Actual Value | Prediction Value | |
---|---|---|

Positive | Negative | |

True | 1090 | 36 |

False | 28 | 425 |

Trials | Minimum Fitness Value | Maximum Fitness Value | Mean Fitness Value |
---|---|---|---|

Trial 1 | 0.4012 | 0.9440 | 0.8063 |

Trial 2 | 0.5616 | 0.9249 | 0.7553 |

Trial 3 | 0.4928 | 0.9758 | 0.9019 |

Trial 4 | 0.6420 | 0.9535 | 0.8611 |

Trial 5 | 0.5170 | 0.9746 | 0.8740 |

Trial 6 | 0.6203 | 0.9515 | 0.8047 |

Trial 7 | 0.5417 | 0.9560 | 0.8518 |

Trial 8 | 0.7454 | 0.8752 | 0.7704 |

Trial 9 | 0.4404 | 0.9628 | 0.8958 |

Trial 10 | 0.5333 | 0.9747 | 0.7840 |

SD | 0.1003 | 0.0501 | 0.0530 |

**Table 6.**Optimization process of $\mu $ values of trial 3 in the series experiments on HOG-HOLBP-GA-SVM.

Iterations | $\mathit{\mu}(\{{\mathit{x}}_{1}\})$ | $\mathit{\mu}(\{{\mathit{x}}_{2}\})$ | F1 Score |
---|---|---|---|

1 | 0.568 | 0.781 | 0.9034 |

2 | 0.529 | 0.743 | 0.9265 |

3 | 0.526 | 0.623 | 0.8947 |

4 | 0.498 | 0.592 | 0.9321 |

5 | 0.493 | 0.588 | 0.9325 |

6 | 0.474 | 0.434 | 0.9411 |

7 | 0.429 | 0.367 | 0.9419 |

8 | 0.447 | 0.533 | 0.9416 |

9 | 0.415 | 0.219 | 0.9530 |

10 | 0.427 | 0.348 | 0.9522 |

11 | 0.421 | 0.299 | 0.9535 |

12 | 0.412 | 0.315 | 0.9568 |

13 | 0.410 | 0.310 | 0.9566 |

14 | 0.344 | 0.221 | 0.9572 |

15 | 0.374 | 0.203 | 0.9570 |

$\vdots $ | $\vdots $ | $\vdots $ | $\vdots $ |

4000 | 0.382 | 0.174 | 0.9758 |

Actual Value | Prediction Value | |
---|---|---|

Positive | Negative | |

True | 1087 | 39 |

False | 15 | 438 |

Classifier | Precision | Recall | F1 Score | Feature Extraction Time (ms/frame) |
---|---|---|---|---|

HOG-SVM | 0.9515 | 0.8703 | 0.9091 | 88.285 |

HOG-LBP-SVM | 0.9600 | 0.9378 | 0.9488 | 131.854 |

HOG-HOLBP-SVM | 0.9712 | 0.9591 | 0.9651 | 10.075 |

HOG-HOLBP-GA-SVM | 0.9864 | 0.9655 | 0.9758 | 10.126 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Yang, R.; Wang, Y.; Xu, Y.; Qiu, L.; Li, Q.
Pedestrian Detection under Parallel Feature Fusion Based on Choquet Integral. *Symmetry* **2021**, *13*, 250.
https://doi.org/10.3390/sym13020250

**AMA Style**

Yang R, Wang Y, Xu Y, Qiu L, Li Q.
Pedestrian Detection under Parallel Feature Fusion Based on Choquet Integral. *Symmetry*. 2021; 13(2):250.
https://doi.org/10.3390/sym13020250

**Chicago/Turabian Style**

Yang, Rong, Yun Wang, Ying Xu, Li Qiu, and Qiang Li.
2021. "Pedestrian Detection under Parallel Feature Fusion Based on Choquet Integral" *Symmetry* 13, no. 2: 250.
https://doi.org/10.3390/sym13020250