In this section, we propose comparing the strategies of sparse LBP histogram selection, sparse LBP bin selection, and the combination of both in the multi color space framework (see Section 5.3
). These strategies will be applied and analyzed with five image databases: NewBarktex, Outex-TC-00013, USPTex, STex and Parquet (see Section 5.1
). A discussion about processing times required by the proposed selection approaches will be presented in Section 5.4
5.1. Considered Color Texture Datasets
Outex-TC-00013 is composed of 68 color texture images acquired under controlled conditions by a 3-CCD digital color camera and whose size is
]. Each of these 68 textures is split up into 20 disjoint sub-images of size
. Among these 1360 sub-images, 680 are used for the training subset and the remaining 680 are considered as testing images for an holdout evaluation (This decomposition is available at http://www.outex.oulu.fi/index.php?page=classification
USPTex set is a more recent database [70
]. It contains 191 natural color textures acquired under an unknown but fixed light source. As for the previous set, these images are split up into
disjoint sub-images. Since the original image size is here
pixels, this makes a total of 12 sub-images by a texture. For our experiments, this initial dataset of 2292 sub-images is split up in order to build a training and a testing image subset: for each texture, 6 sub-images are considered for the training and the 6 others are used as testing images (This decomposition is available at http://www-lisic.univ-littoral.fr/~porebski/USPtex.zip
The Salzburg texture image database (STex) is a large collection of 476 color texture images, whose acquisition conditions are not defined. Each of the 476 original images is split up into 16 non-overlapping
sub-images. Half of these 7616 sub-images are used for the training subset and the remaining are considered as testing images (This decomposition is available at http://www-lisic.univ-littoral.fr/~porebski/Stex.zip
Although Outex-TC-00013, USPTex and STex sets are widely used, these image sets present a major drawback: the partitioning applied to build these three sets consists of extracting training and testing sub-images from a same original texture image. However, such a partitioning, when it is combined with a classifier such as the nearest neighbor classifier, leads to biased classification results [52
]. Indeed, testing images are spatially close to training images and are thus correlated. A simple 3D color histogram is then able to reach a high classification accuracy whereas it only characterizes the color distribution within the color space and does not take into account the spatial relationships between neighboring pixels, as a color texture feature should [31
]. That is the reason we propose considering two other challenging image sets built from the Parquet and the Barktex databases, respectively.
The Parquet database is composed of fourteen varieties of wood for flooring [71
]. Each type of wood presents several different grades ranging from 2 to 4 which are considered as independent classes, leading to a total of 38 different classes. The main challenge of this database is that, within each type of wood, the grades are very similar to each other. Moreover, the sizes of acquired images are different and the number of samples per class varies from 6 to 8. As done in [72
], six samples per class are retained and the images are centre-cropped, so that the final dimension of images ranges from
pixels. For each texture, 3 images are considered for the training and the others are used as testing images (This decomposition is available at http://www-lisic.univ-littoral.fr/~porebski/Parquet.zip
). With the experimental protocol proposed by the authors, the classification accuracy reached on this challenging database does not exceed 75.1%.
The Barktex database includes six tree bark classes, with 68 images per class [73
]. Even if the number of classes of this database is limited to 6, the textures of these different classes are close to each other and their discrimination is not easy. To build the NewBarktex set, a region of interest, centered on the bark and whose size is
pixels, is first defined. Then, four sub-images whose size is
pixels are extracted from each region. We thus obtain a set of
sub-images per class. To ensure that color texture images used for the training and the testing images are less correlated as possible, the four sub-images extracted from a same original image all belong either to the training subset or to the testing one [52
]: 816 images are thus used as training images and the remaining 816 as testing images (The NewBarktex image test suite can be downloaded at https://www-lisic.univ-littoral.fr/~porebski/NewBarkTex.zip
). Table 1
summarizes theses databases.
5.2. Performance Evaluation and Comparisons
Since the considered texture benchmark databases were built in order to apply an holdout evaluation, the classification performance is assessed by following this evaluation scheme. For this purpose, classification results reached by the proposed methods are evaluated by measuring the accuracy as the rate of well-classified testing images during the classification stage.
During this stage, the relevant histograms previously selected by one of the proposed approaches presented in Section 3
and Section 4
are computed for each testing image and compared to those of the training images in the so-selected relevant histogram or bin subspace to determine the testing image label. The purpose of this paper being to show the contribution of different LBP-based feature selection approaches, independently of the considered classifier and its parameters—such as the metric—the nearest neighbor classifier associated with the L1 distance as a similarity measure is here considered. Obviously, the classification results are expected to be improved by using more elaborated methods such as the SVM classifier for example.
As previously mentioned, the texture benchmark databases are composed of only two image subsets (training and testing images), whereas the proposed approaches need three subsets by adding validation images required by embedded selection methods. To evaluate and compare our experimental results with the same conditions of other works, which do not divide the training subset into two parts, we propose using one subset as the training subset and the second both as the validation and testing subset. During the classification stage, the supervised classifier thus uses exactly the same training subset than the one used by the compared works for determining the class labels of the testing images.
In the following, the proposed LBP-based feature selection strategies are evaluated, analyzed and compared with the results of the state-of-the-art under the same experimental protocol. Number of classes, size of images, number of images for each class, total number of images and accuracy evaluation method are the same. No changes appear about texture rotation and illumination. Therefore, comparisons exclude some other existing works that apply other protocols to the experimented databases.
5.3. Validation of the Proposed LBP-Based Feature Selection Strategies
presents the results obtained with the proposed LBP-based sparse feature selection approaches, on the NewBarktex database. The single color space and multiple color space strategies are also compared, as well as the results with and without selection.
For each color space, the accuracy estimated by the rate of well classified testing images and the dimension of the selected feature space are presented. First, we can notice that operating a selection significantly improves the accuracy while reducing the dimensionality of the feature space. The improvement reaches on average 7.8% compared to the without selection approach.
We can also notice that even if the considered color texture database is fixed, the color space that enables reaching the best accuracy is not always the same and depends on the considered feature selection strategy: (74.4%) when no selection is performed, (79.5%) when a bin selection is achieved, and with the histogram selection approach (81.3%) and the combination of both selections (83.7%). This confirms that the a priori choice of the well suited color space is not easy and so, the interest of the multi color space strategy. This table also shows that considering several color spaces significantly improves the accuracy too since an improvement of 7% is obtained compared to a single color space strategy.
We can also notice that even if the dimension of the feature space selected by the bin selection strategy is lower, the Sparse-MCSHS approach (87.3%) significantly outperforms the proposed Sparse-MCSBS approach (83.6%). This confirms the result recently obtained by Porebski et al. with the ICS-based MCSHS approach versus the MCSBS approach using Guo’s selection strategy [16
]. Finally, we notice that operating a combination of bin and histogram selections (88.4%) improves the accuracy compared to a simple Sparse-MCSBS approach or a simple Sparse-MCSHS approach.
, Table 4
, Table 5
, Table 6
and Table 7
present the classification results obtained by our proposed approaches and those obtained by the different studies which applied a color texture classification algorithm on Outex-TC-00013, USPTex, STex, Parquet, and NewBarktex, respectively. To achieve classifier-independent comparisons, only the studies using the nearest neighbor classifier are here presented. We also propose in this paper to ignore wrapper approaches, which use the classification rate as discrimination power of feature subspaces, and so involve an important learning time [74
]. We implemented on the other hand some experiments with Convolutional Neural Networks (CNN) to compare our approach with the latest popular methods. Since the considered color texture databases have few learning images, we propose using the pretrained AlexNet [75
] and GoogleNet [76
] networks to achieve this comparison.
The rows labelled as gray correspond to experiments that are carried out in this work whereas the other rows correspond to results published by other authors. The first column refers to the related papers and indicates the descriptors which were computed to discriminate the different color texture classes. The color spaces considered to classify the images are presented in the second column of each tables. Finally, the last column shows the obtained rate of well-classified testing images (Accuracy).
For the USPTex and STex databases, the best rate is 98.1%. These rates are obtained with our proposed approach of bin and histogram combination and are very encouraging. For Outex-TC-00013, Parquet and NewBarkTex, the results obtained with our approach get into a very satisfying position with the works of the state of the art, since they reach the second position with 95.7%, 83.3%, and 88.4%, respectively. For the Outex database, they are just behind the results obtained by the Multi Color Space Feature Selection proposed in [52
] where many more different color spaces are considered. For the Parquet and NewBarkTex sets, they follow the rate of well-classified testing images reached by CNN with the pretrained GoogleNet and AlexNet network, respectively.
presents the relative ranking of the considered approaches and the average accuracy reached over the five databases. This table shows that the combination of bin and histogram selections outperforms the simple MCSHS and MCSBS approaches: an average improvement of 0.8% compared to the MCSHS strategy and 3.1% compared to the MCSBS approach is observed. It also shows the relevance of the proposed approach with the experiments on pretrained CNN, with an average improvement of 8.2%. We can also notice that the results obtained with the proposed MCSHBS approach are more stable than those reached thanks to CNN, since they always get into the first or the second position, whatever the considered database.
As the aim of this paper is to reveal the real contribution of the proposed selection strategy as independent as possible from the impact of the classifier and LBP parameters, straightforward methods of texture representation (basic LBP configuration) and classification (1-NN) were preferred as well as texture databases without change of observation conditions. Obviously, the representation of color textures could be improved by using other configurations of LBP parameters (for instance by increasing the parameters P
), by adding other color spaces or by considering an ensemble of several descriptors [93
]. However, a high increase of the dimensionality of the original feature set could lead to a number of features much larger than the number of samples, and so to a risk of overfitting [94
]. In such a case, an approach based on a preliminary feature clustering should be relevant to reduce the number of features candidate for selection.
The next section studies the impact of the proposed selection approaches on the processing times.
5.4. Processing Times
Moderate consumption of computation resources is really needed for applications that require fast processing and/or low memory storage such as machine vision applications operating in real time with significant production time constraints or mobile applications embedded on smartphones and tablets where the processor and the memory are limited. Supervised selection procedure aims to define a compact representation of color textures by reducing the dimensionality of the feature space during a learning stage. This offline learning stage is carried out prior to the online classification stage and, as with deep learning methods, can last from several minutes to a few days with no impact on the final application. During the online classification stage, the so defined compact representation then increases the accuracy of the classifier and decreases the computation time.
When no selection is performed, the learning stage only consists of computing the LBP histograms from the training images but the high dimension of the feature space (
20,736 in our case) leads to a high and crippling computation time for the online classification stage with a poor accuracy as illustrated in Table 2
for the NewBarktex dataset (78.2%). When a selection procedure is performed, the learning stage is more time consuming since it consists of the computation of all the available histograms from the training and the validation images, followed by a selection phase to determine the relevant feature space. As an embedded method is used during this selection phase, the dimension determination step (see Figure 1
, Figure 2
and Figure 4
) is the most time consuming since it requires to operate several classifications to evaluate each candidate sub-space: Equation (1
) shows that
operations of classification are performed for the MCSHS approach while MCSBS and MCSHBS execute
operations (see Equation (2
for a basic LBP descriptor, the learning execution time is strongly impacted by an increasing of the LBP parameter P
. However, the low dimensional feature subspace determined by the selection procedure during the offline learning stage significantly reduces the online classification time.
Compared to a histogram selection approach (MCSHS), the online classification processing time reached by MCSHBS is close since the dimension of the selected feature sub-space is nearly the same: for instance, with the NewBarkTex dataset, the dimension of the selected feature sub-space is
for the Sparse-MCSHS approach and
11,985 for Sparse-MCSHBS (see Table 2
). The slight cost in processing time occurred with the Sparse-MCSHBS approach is counterbalanced by a slight gain of accuracy: 87.3% for the Sparse-MCSHS approach and 88.4% for Sparse-MCSHBS.
Compared to a bin selection approach (MCSBS), the online classification processing time required by MCSHBS is higher because the dimensionality of the feature sub-space selected with MCSBS is lower:
for the Sparse-MCSBS approach and
11,985 for Sparse-MCSHBS (see Table 2
). However, classification results show that MCSHBS clearly outperforms MCSBS: the accuracy reaches 83.6% for the Sparse-MCSBS approach and 88.4% for Sparse-MCSHBS.
These results show the interest of operating a selection thanks to the proposed Sparse-MCSHBS approach.