The Haar-like features have scalar values that represent the difference in the sum of intensities between the adjacent rectangular regions. To capture the ad hoc knowledge about the domain, these features are evaluated at different positions and with different sizes exhaustively according to the base resolution of the classifier. For example, when the classifier resolution is 24 × 18 pixels, 91,620 features are generated from the five features in Figure 1(a,b). Each feature is evaluated on all the training samples and the probability density for each of the object and non-object class is calculated as shown in Figure 2. In [2], a single threshold that separates these two distributions is selected for each feature. These features along with their respective thresholds and polarity form the weak classifiers for the learning algorithm.

A weak classifier can be mathematically described as: (1) h ( x , f , p , θ ) = { 1  if  pf ( x ) > p θ − 1   otherwise     where x is the base resolution of the classifier, f the Haar-like feature, θ the threshold for the feature and p the polarity indicating the direction of inequality. The choice of optimal threshold for the features is not stated clearly in [2] and shows to be a non-trivial task.

AdaBoost is a machine learning boosting algorithm that constructs a strong classifier by combining a set of weak classifiers. A small number of discriminative weak classifiers are selected by updating the sample distribution. The prediction of the strong classifier is produced through a weighted majority voting of the weak classifiers. Pseudo code of a variant of AdaBoost used in the implementation is given in Algorithm 1.

Given a set of features, the Bayesian decision theory for classification requires decision boundaries that minimize the error rate on the training data. Let us consider a two class problem with ω₁ and ω₂ as the state of nature. If x is the observed feature value, the decision boundary that minimizes the classification error is given in terms of the posterior probabilities as P(ω₁|x) = P(ω₂|x). The corresponding decision rule is: decide ω₁ if P(ω₁|x) > P(ω₂|x); else decide ω₂.

One of the most useful ways to represent pattern classifiers is in terms of a set of discriminant functions g_i(x); i = 1, 2,…, c, where c is the number of categories to discriminate. The classifier is said to assign a feature x to class ω_i if: (2) g i ( x ) > g j ( x )   for all   j ≠ i

The effect of the decision rule is to divide the feature space into c decision regions. The regions are separated by decision boundaries, surfaces in the feature space where ties occur among the largest discriminant functions [16]. Assuming the distribution of the univariate Haar-like features to be normal, i.e., p(x|ω_i) ∼ N(μ_i, Σ_i), the minimum error rate classification can be achieved by the use of discriminant function of the form given in Equation (3) [16]: (3) g i ( x ) = − 1 2 ( x − μ i ) t Σ i − 1 ( x − μ i ) − 1 2 ln 2 π − 1 2 ln | Σ i | + ln P ( ω i )where P(ω_i) is the priori probability of class ω_i. Taking a general univariate normal case with different variances for each category, the resulting discriminant function is given as: (4) g i ( x ) = x t W i x + W i x + w i 0where: W i = − 1 2 Σ i − 1 W i = Σ i − 1 μ i w i 0 = − 1 2 μ i t Σ i − 1 μ i − 1 2 ln | Σ i | + ln P ( ω i )

The discriminant functions of Equation (4) are inherently quadratic. The decision surfaces are hyperquadrics and in one dimensional case the decision regions needn’t be simply connected as shown in Figure 3. This observation motivates us to formulate new kind of weak classifiers without explicitly specifying the threshold for each weak classifier.

The proposed weak classifiers are based on the quadratic discriminant functions described above. Each Haar-like feature from the pool of 91,620 features is evaluated on the training samples and one-dimensional probability densities for object and non-object classes are calculated. Assuming the density of each feature to be normal, the distributions of feature on the object and non-object classes are parameterized by their maximum likelihood estimators, i.e., mean μ and variance Σ. The distribution for the object (positive) class is p(x|ω_p) ∼ N(μ_p, Σ_p), and for non-object (negative) class is p(x|ω_n) ∼ N(μ_n, Σ_n). The decision regions for the two distributions are given from Equation (2), i.e., assign the observed feature value x to class ω_p if: (5) g p ( x ) > g n ( x )

This decision rule divides the feature space into decision regions which needn’t be simply connected for the same class. The proposed weak classifiers for the Haar-like features are defined as: (6) h ( x , f , μ p , μ n , Σ p , Σ n ) = { 1       if    g p ( f ( x ) ) > g n ( f ( x ) ) − 1                          otherwisewhere x is the base resolution of the classifier and f is the Haar-like feature. Since a more general model of the distribution is considered, the proposed weak classifiers are expected to perform better than the single threshold weak classifier.

For the weak classifiers of Equation (1), each feature produces a single scalar value and the decision boundary corresponds to a scalar threshold. But the choice of this threshold is not stated clearly in [2] and determination of an optimal threshold is a nontrivial task. The proposed weak classifiers of Equation (6) are more general and do not require any explicit representation of the threshold. In fact, the weak classifiers of Equation (1) are a special case of the proposed weak classifiers when Σ_p and Σ_n are identical. The weak classifiers based on single threshold commonly employ “average of means” of the two distributions, i.e., (μ_p + μ_n)/2, as decision threshold. Under this hypothesis, it is statistically observed that most of the Haar-like features are non-discriminative and inefficient for boosting.

The error rates of these single threshold weak classifiers selected at later stages of the boosting process become large as the sample distribution consists of samples which are difficult to discriminate as shown in Figure 4. The single threshold weak classifiers are not efficient in discriminating such distributions. The proposed weak classifiers are expected to efficiently discriminate the underlying distribution of Figure 4, as disjoint decision regions are also supported as shown in Figure 3.

The experiments were carried out for detection of rear of cars. The experiments were done using 1,500 positive and 3,500 negative samples. The positive samples consisted of instances of rear of cars cropped from a video taken from a camera mounted at the front of a host car while driving in an urban environment. Each instance was resized to a base size of 24 × 18 pixels. The negative samples consisted of images cropped from random high resolution images that did not contain any instance of rear of car. Each negative sample was also resized to base size of 24 × 18 pixels. 1,000 positives and 3,000 negative samples were used for training the classifiers while the remaining 500 positive and 500 negative samples were used for validation. Figure 5 shows some of the positive and negative samples used for the experiment.

A single stage classifier was trained by AdaBoost on the training data using the proposed weak classifiers to achieve 100% hit rate on the positive samples and zero false positive on the negative samples. The final strong classifier achieved the required performance on the training data with a total of 69 proposed weak classifiers. The first weak classifier selected by AdaBoost yielded an error rate of 0.2. The subsequent selected weak classifiers yielded comparatively higher error rates. The worst error rate among the selected classifiers was 0.37 for the 66th classifier. The error rates of subsequent selected classifiers can be seen in Figure 6. Another strong classifier was trained on the same training data using the conventional single threshold based weak classifiers. These classifiers employed the average of means as the threshold. The final strong classifier required 225 weak classifiers to achieve similar performance on the training data. The error rate of the first selected weak classifier was 0.21 but it increased rapidly for the subsequent classifiers and the worst was 0.44 for the 208th classifier. A third strong classifier was trained on the same training data using single threshold based weak classifiers which employed Otsu’s method [17] for optimal threshold selection. The final classifier consisted of 125 weak classifiers. The error rate of the first weak classifier was 0.22 and the highest error rate was 0.41 for the 124th classifier. Figure 6 shows that the error rates of the proposed weak classifiers are consistently lower than the single threshold based counterparts.

Most of the features selected using the proposed weak classifiers have overlapping distributions of the object and non-object classes. Though these features have lower error rates and are boostable under the proposed hypothesis, they would have been rendered useless for boosting under the single threshold hypothesis. Some of the feature distributions are shown in Figure 7.

The selection of efficient classifiers at each round of boosting helps the learning algorithm to converge faster (in terms of the number of weak classifiers) on the training data as can be seen in Figure 8. Similar performance (in terms of hit rate and false positive rate) on the training data can be achieved with a significant reduction in the total number of weak classifiers by using the proposed weak classifiers over the conventional single threshold weak classifiers.

To investigate the generalization performance of the proposed weak classifiers, the strong classifiers were tested on a validation dataset. The validation data set consisted of 500 positive and 500 negative sample images that were not used for training. ROC curves were generated using the validation dataset. The points in the ROC curves were obtained by evaluating each of the strong classifiers against the validation dataset by sliding the stage threshold from −10 to +10 at step of 0.25. The thresholds for the stage were chosen because varying the thresholds in this range proved to be sufficient to generate the whole range in the ROC curves. The plot of the hit rate versus the false alarm rate for all the methods is given in Figure 9. From the ROC curves in Figure 9, we can see that the classifier trained using the proposed weak classifiers perform consistently better than the classifier trained using single threshold weak classifiers. The detection rate of the classifier based on the proposed weak classifiers is always higher than the classifier based on single threshold weak classifiers. And for a given detection rate, the classifier using the proposed weak classifier always has less false alarm rate than the classifiers using single threshold weak classifiers. The higher performance of the proposed method reflects the benefit of the usage of discriminant function based weak classifiers, which are more effective at discriminating car and non-car examples.

In this experiment, the classifiers were trained on relatively difficult samples than those of Section 4.2. The positive samples contained 500 positive images from the original training set. The negative samples contained 2,000 negative images. The negative samples were the false positives generated when a 10 stage cascade was evaluated on random high resolution images. The 10 stage cascade was trained on the original training set using the single threshold weak classifiers. In this sense, the negative samples are relatively difficult for the single threshold weak classifiers to discriminate. Three different classifiers were trained using the three types of weak classifiers on this data to achieve 100% hit rate and zero false positives. The classifier using the proposed weak classifiers required only 36 features whereas the classifier using single threshold weak classifier employing average of means required 236 features and the classifier employing Otsu’s thresholding method required 90 features to achieve same performance on the training data. This shows that the proposed weak classifiers are equally efficient in discriminating difficult samples than the single threshold counterparts. The generalization performance of the trained classifiers was tested on the validation set as in Section 4.2. The ROC curves in Figure 10, generated against the validation dataset show significant performance improvement of the classifier trained using the proposed weak classifiers over the classifier using single threshold weak classifiers.

The speed of a cascaded object detector is directly proportional to the number of features evaluated per scanned sub-window in the image. To compare the speed of the detector, we trained three 15 stage cascade using the single threshold classifiers and the proposed weak classifiers on the UIUC Image database for car detection. The cascades were evaluated on the UIUC car test image set at different scales. For the classifier with single threshold employing average of means, an average of 14.5 features out of the total 248 features and for the one employing Otsu’s optimal threshold an average of 13 features out of the total of 230 were evaluated per sub-window, whereas for the proposed classifiers only an average of 8 features out of the total 131 features were evaluated per sub-window. Table 1 shows the total number of sub-windows scanned and the total features evaluated for seven images randomly sampled from the UIUC car test images at different scales. The feature value calculation time for the proposed classifier is the same as that for the single threshold Haar-like features. But from Equation (3) we see that the proposed weak classifier requires additional multiplication and addition operation to make the class decision. This makes them relatively more expensive to compute than the single threshold classifiers. The experiments conducted show that the proposed weak classifier requires around 1.6 times more computation time than the single threshold classifier to make a class decision. However as seen from Table 1, the single threshold classifiers need to evaluate on average around 1.6 times more features per sub-window than the proposed weak classifiers. This makes the speed of the proposed detector comparable to that of the conventional single threshold based detector.