RGB Inter-Channel Measures for Morphological Color Texture Characterization

: The perception of textures is based on high-level features such as symmetry, brightness, color or direction. Texture characterization is a widely studied topic in the image processing community. The normalized volume of morphological series is used as a texture descriptor in RGB images. However, the correlation between different color channels is not exploited with this descriptor. We propose the usage of inter-channel measures in addition to the volume, to enhance the descriptors potential to discriminate textures. The experiments show that standard texture classiﬁcation techniques increase between 3%–10% in performance when using our descriptor instead of other state of the art descriptors that do not use inter-channel measures.


Introduction
Image textures are easily perceived by humans, and are regarded as a rich visual information source. Informally, a texture is a complex visual pattern composed of entities (or subpatterns) which have symmetry, brightness, color, direction, and a characteristic size [1]. However, there is no consensus on a formal definition for texture, so a lot of work is focused on extracting structural and spatial features from the textures, to classify them in qualitative taxonomies [2].
Texture analysis encompasses a wide array of topics from image processing to artificial vision [3]. Notable examples from these topics are feature extraction, texture classification, texture segmentation, texture synthesis, and shape analysis [4]. Texture analysis has applications in industrial screening, remote imaging, medical imaging, content-based image retrieval (CBIR), materials science, document processing, and others [4,5].
Feature extraction is commonly the first step for texture classification, and it is used in several topics within image processing [1]. Features are numerical calculations that quantify significant properties of a texture given by a descriptor [1,6]. A texture descriptor is a vector of extracting attributes from an image, which allow similarity comparisons between images.
Numerous techniques have been developed for texture analysis: co-occurrence matrices, binary local pattern, Laws filters, Markov models, Gabor filters, invariant moments, among others [1,7,8]. The techniques can be classified according to the method used to obtain the image texture descriptors: structural, statistical, model-based, and transformation model-based [7,8].
Our proposal uses a structural technique based on mathematical morphology, as it allows the usage of spatial relation between pixels to characterize an image [6]. Furthermore, the technique is integrative as we process color and texture information simultaneously.
The main objective of this work is to prove that the inclusion of inter-channel measures enhances morphological texture descriptors for RGB color images. We compare texture descriptors by comparing the performance of standard texture classification methods on the same dataset. The intuition is that texture descriptors that yield better classification percentages are capable of extracting more significant characteristics of the underlying image.
We choose to work on the RGB color space, as the application of morphological filters to other color spaces present several difficulties, in particular when the domain of the channels is heterogeneous [9]. Furthermore, establishing inter-channel measures for heterogeneous channels is not a trivial procedure. Section 2 lays out the formal foundations of color image processing and mathematical morphology. Section 3 presents morphological series as basic texture descriptors. Section 4 presents the proposal. Section 5 shows the experiments. Finally, Section 6 discusses the results and proposes future work.

Definitions
We present some definitions for digital image processing in order to formally define our evaluation metric: feature extraction for image classification.

Color Images and Texture
We define the digital image f as f : E → T, where the pixel arrangement, E ∈ N 2 , is a two dimensional array, and T is the range of values each pixel can get. In binary images, the values are T ∈ {0, 1}; while in grayscale, the values are traditionally T ∈ [0, 255], they can also be T ⊂ N and even T ⊂ Z.
A multispectral color image, f , has pixel values, T ∈ Z c or T ∈ R c , where c ∈ N >2 is the number of color channels [10]. Without loss of generality, f = { f ω } 1≤ω≤c , where f ω the plain of the ω-th color channel.
The color channels define a multispectral image's color space [10]. The Red-Green-Blue (RGB) color space is an additive color space of three channels, where each color is obtained by adding some amount of the primary colors. The RGB color space can be represented in a cubic 3D space where the primary colors are the axes. The range of the axes will be [0, 2 k − 1], where k is the number of bits used for each color channel [10].

Texture Characterization and Classification
In this work, we use mathematical morphology, upon which color and texture information are jointly processed. Mathematical morphology belongs to the structural approach as it describes the rules of the spatial distribution of the texture elements [6,7].
We use granulometry and morphological covariance as morphological texture descriptors. The granulometry can extract characteristics from disordered and composite textures [11], and it can describe the texture regularity [12]. On the other hand, morphological covariance is capable of describing texture directionality [12].
Finally, we can proceed with texture classification, comparing the extracted features. The classification is done by associating a set of features with a texture class [13]. A classifying algorithm can be supervised or unsupervised according to the availability of known data. Supervised classifiers use examples with known classifications as a training set, while unsupervised classifiers have no known class associations and draw the classes from the feature distribution [14].

Mathematical Morphology
Mathematical Morphology is an image analysis branch based on algebraic principles, set theory, and geometry [15,16]. The following definitions describe basic morphological filters upon which morphological series are built, which we use for texture description. This section presents the original grayscale formulations, and their extension to color as used in this work.
The basic morphological filters are erosion and dilation. The erosion, ε b ( f ), and dilation, δ b ( f ), are the minimum and maximum filters of an image, f , respectively. They are defined under a structuring element, b, as follows: where p ∈ N 2 are the pixel locations of the the image and q ∈ Z 2 are the coordinates of the structuring element pixels. The opening, γ b ( f ), and the closing, ϕ b ( f ), are defined as follows: whereb is the reflection of the structuring element, defined asb = {−q| q ∈ b}.
The pixel ordering is trivial in grayscale images, and it is given by the natural order of the pixel intensity. However, there is no consensus on ordering for color pixels. Vectors that represent a pixel's color have no natural ordering, therefore the extension of mathematical morphology to color is an open problem [17].
The main types of morphological processing for color images are [17]: • Marginal Processing: also called channel processing. Each channel is independently processed and the inter-channel correlation is ignored.

•
Vectorial Processing: the color channels are jointly processed. A pixel value, represented by vectors, is treated as the basic processing unit.

Morphological Series
The feature extraction of image texture, f , using Mathematical Morphology is generally performed in two steps: (Figure 1):

1.
Build a morphological series, which consists of successive morphological filter, ψ, application to image, f .

2.
Gather values through a measure that allows quantifying the effect of the morphological operator ψ on the image, f .
Most morphological texture descriptors are based on morphological series. The elements in the series are based on properties of the structural element, such as size, shape, and orientation [6,18]. The morphological series is later passed to a feature extraction measure for classification.

Grayscale Morphological Series
We define a morphological series, ∏ ψ , as a series of images, resulting from different applications of a morphological filter, ψ, to the input image, f . The different elements of the series result from the modifications of a property, λ, of the structuring element, b, such as size. Formally: where the first element of the series, ψ 0b ( f ) = f , is the original image. Different filters, ψ (such as dilation, erosion, opening and closing), together with the characteristics of the structuring element, allow morphological series to extract specific characteristics from the image [19,20]. The Granulometry and Morphologic Covariance will be the morphological series, we will use to extract features from the images. They were proven to be useful techniques for feature extraction in other works [6,16,21].

Granulometry
We adopt the simplified and normalized notation, Ω γ ( f ), from Aptoula et al. [20], defined as: where the structuring element, b, is incremented n times. Normalizing the function by the original volume, Vol( f ), allows for size invariance. The volume of an image, f , is defined as: Figure 2 shows examples of a structuring element which is increased two times.

Morphological Covariance
Generally speaking, textures can be characterized by the spatial distribution of their patterns. The morphological covariance provides information about periodicity, orientation and rugosity of patterns found in textures through the distribution of orientations and distance of matching pixels [20].
The morphological covariance, K, of an image, f , is defined as the volume of an image after eroding, ε, the input. The structuring element is composed by two points, b = {p 1 , p 2 }. These points are not necessarily adjacent, so their separation can be represented as the vector, − − → p 1 p 2 . Therefore, the distance and direction between them is, || − − → p 1 p 2 || and respectively. In practice, we use the normalized morphological covariance defined as: where λ represents one of the n combinations of distances and orientations for p 1 and p 2 , and Vol( f ) is the normalization factor. Figure 3 shows examples of n = 3 × 3 variations of a two-point structuring element.

Color Extension for Morphological Series
We use the Lefevre extension for vectorial processing [18]. Formally: where ω is a color channel of, f . The application of the vectorial morphological filter, ω , is taken independently in the volume calculation or any other measure that is used in the classification.
Correspondingly, the morphological filter, ψ is applied to each channel independently for marginal processing [18] as: where f ω is a color plain of f . The vectorial granulometry for a color image, f is obtained from Equation (6) in Equation (9):

Orientations Distances
The vectorial morphological covariance is obtained from Equation (8) in Equation (9): The marginal version of color granulometry and morphological covariance are obtained by applying Equations (6) and (8) in Equation (10) respectively.

Proposal
We propose the integration of inter-channel measures into color texture descriptors based on morphological series. The initial texture descriptor is the normalized volume of color channel for each element in the morphological series [9,18]. Extending the descriptor with inter-channel measures allows the characterization of the correlation between the color plains.
The correlation between the color plains means that if the intensity changes, all three components will change. This is one of the properties that characterize the RGB space in applications involving a natural color image [22]. Figure 4a,b show the normalized volume descriptors of a sample RGB texture. In both cases, the volume of a channel is normalized by the channel's volume in the original image. These values do not explicitly reflect the high correlation among the channels, but the residual intensities of the filtered image, ψ λb ( f ω ).  Analogously, Figure 5a,b show the raw volume descriptors of the sample. In theses images, the correlation between the different color channels can be noticed, as the volumes change simultaneously. However, some normalization is required.  While intra-channel measures collect and process information from each channel independently, inter-channel measures work with information of multiple channels. Previous works have shown some benefits of the usage of inter-channel measures for texture descriptors using co-occurrence matrices [23,24]. Co-occurrence matrices are two-dimensional histograms of intensity levels for pairs of color channels [23,25].
Our proposal is to use texture descriptors that join the normalized volume of the morphological series and an inter-channel measure: The volume is used as an intra-channel measure, which will complement the information retrieved by the inter-channel measure.
We compare our proposal to descriptors that use either intra-channel measures or inter-channel measures alone, and descriptors that join volume and intra-channel measures.

Characterization/Intra-Channel Measures
Previous work has shown that characterization measures that do not use spatial information obtain similar results as texture descriptors compared to ones that use spatial information [6]. Most of these measures are defined for monochromatic images-i.e., they fit the model of intra-channel measures. Therefore, we concatenate the values returned of each color channel in f to produce a color texture descriptor.
The following measures have shown good performance as texture descriptors [6].

Variance
The variance of an image, f , characterizes the divergence of pixel intensities. Formally: where |E| is the number of pixels in f , and f (p) is the mean of f (p).

Energy
The energy of an image, f , characterizes the potential of the intensities (loosely seen as frequencies) of the pixels. Formally:

Entropy
The entropy of the histogram, h, for an image, f , is defined as: where k is the amount of bits encoding the intensity level of a color channel, and h is divided in 2 k buckets according to the intensity levels.

Hypervolume
The hypervolume of an image, f , with c color channels is the average volume of ω-dimensional bounding boxes for pixel intensities [8]. Formally: where f 1 , f 2 , f 3 are the R, G, and B color channels of f .

Inter-Channel Measures
Although the measures presented in this section are not novel in themselves, we propose their usage for texture descriptors. To ease notation, we denote hereafter each color plain of a filtered color image, ψ λ (f), as Q 1 , Q 2 and Q 3 ( Figure 6). Figure 6. The morphological filter, ψ( f ), yields images in separate color channels, Q 1 , Q 2 and Q 3 , as output.
First, we present measures based on vectorial distances, which establish how far apart are two elements in vectorial space. Distance-based measures approximate zero as the compared elements increase similarity, and grow when elements decrease similarity. Then, we present proportional measures, which approximate when elements become more similar, and grow or decrease when they become less similar. Finally, we present a histogram-based measure, which retrieves a simple vector as output. The elements of the output vector provide values that express relations between channels of the input image.

Normalized Euclidean Distance
We propose that the euclidean distance between two images, Q i and Q j (which belong to the same element of the morphological series), as the euclidean distance between their volumes.
However, we require to normalize the measure to make it adimensional and independent from size.
Dividing the difference by the standard deviation of all three volumes, σ, renders the normalized euclidean distance: The inter-channel descriptor based on this measure VD S is:

Canberra Distance
The Canberra distance is a normalization of the L1 norm of a difference. We establish the Canberra distance of images, Q 1 and Q 2 , as: This distance is subject to Vol(Q i ) = 0 ∨ Vol(Q j ) = 0-i.e., at least one of the images must have a non-zero volume. Please note that the range of D C is restricted to the [0, 1] interval.
When D C (Q i , Q j ) ≈ 1, a channel dominates the expression and the distance is maximum. The inter-channel descriptor based on this measure VD C is:

Inter-Channel Proportion
Another way of establishing the relation between pairs of color channels is by their quotient. Analogous to the previous definitions, we compare images, Q i and Q 2 , through their volume as: The quotient provides a simple normalization procedure, which is also size independent. When the channels are very similar, P(Q i , Q j ) ≈ 1. However, the measure diverges from 1 when the channels are different. In particular, P(Q i , Q j ) ≈ ∞ as Q i is more intense than Q j , and P(Q i , Q j ) ≈ 0 as Q j is more intense than Q i .
The inter-channel descriptor based on this measure VP is:

RG-Volume
The RG-Histogram describes the proportional intensity of each color channel and is normalized by the total intensity, R + G + B [26]. The original formulation is given by: We transform Equation (25) to an image histogram, using the channel volume as the intensity indicator. This formulation allows size invariant expressions to compare channels.
Therefore, our formulation of the normalized volume for image Q i is: Since the relative intensities, V(Q 1 ) + V(Q 2 ) + V(Q 3 ) = 1, one can be omitted from the inter-channel descriptor as it is linearly dependent on the other two. Finally the inter-channel descriptor for this measure VRG, based on the red and green channels, is:

Experiments
Classification experiments were performed on the OutexTC00013, for general purpose color texture classification, and OutexTC00014 database, for illumination invariance testing [27]. The OutexTC00013 contains 68 textures classes obtained by acquiring a 100 dpi image of size 746 × 538 pixels illuminated by a 2856 K incandescent CIE A light source, each class was divided into 20 non-overlapping sub-images of size 128 × 128 pixels, totaling 1360 texture images (Figure 7).
The training set comprises the half of images for each class, totaling 680 images for training and test respectively.
The OutexTC00014 includes OutexTC00013 as the training set. The test set contains the same 68 textures acquired under the two illumination sources. The illumination sources used are 2300 K horizon sunlight (Hor) and a 4000 K fluorescent (TL84). For each illumination source, 1360 images are available, making a total of 2720 test images.
Featured vectors are obtained through granulometry and Morphological Covariance. For granulometry, square-shaped structuring elements and 2λ + 1 side pixels were used, varying the size parameter, λ, from 1 to 15. Small size increments lead better discrimination results [28]. The morphological covariance requires direction and length variations among structuring element pixels. We use the four basic and important directions (0 • , 45 • , 90 • , 135 • ) [12] and lengths from 1 to 20 pixels.
The classifier used is K-nn (K-nearest neighbors) with k = 1 and euclidean distance. The classification accuracy is computed as the number of successful classifications divided by the total of number of subjects.
We have implemented the morphological filters using ImageJ version 1.52, and performed the classification using Weka version 3.8.3.

Experimental Results for the OutexTC00013
We performed classification on the OutexTC00013 to compare intra-channel and inter-channel measures with morphological filters applied in vectorial and marginal version. Recall that the erosion (Equation (1)) and dilation (Equation (2)) require to find the minimum or maximum pixel within the structuring element. We use the norm of the color vector representing the pixel to order the pixels within the structuring element.
The results are shown in Table 1. Globally, one can remark the positive effect of the concatenations of the inter-channel measure with standard volume compared to only volume used. For covariance, the accuracy increased in ≈8% to ≈10% using a vectorial filter and ≈4% to ≈7% in the marginal filter.
The granulometry acruracy increased in ≈6% to ≈8% using a vectorial filter and in the marginal filter by ≈3% to ≈4%.

Experimental Results for the OutexTC00014
The OutexTC00014 was used to evaluate our propose performance in invariant light problems, and all images were preprocesed with histogram equalizations to each channel, aiming to provide illumination invariance [29]. As in the previous experiment, we use the norm of the color vector representing the pixel to order the pixels. Table 2 shows best results for standard descriptors compared with inter-channel measures. Energy, volume, and hypervolume exhibit better results. Equalization preprocessing decreases the intensity differences between volumes of channels and, consequently, the inter-channel measures are not relevant features. This can be seen in Figure 8, which shows the granulometric volumes for original sample texture, 000002, and the image with equalization.

Final Discussion and Conclusions
While several inter-channel measures were proposed, there are none that use the mathematical morphology approach. The morphological approach also shows a limited amount of texture descriptors which are dominated by the standard volume. Therefore, in this work we propose a new morphological texture descriptor based on inter-channels measures for RGB color images. The new morphological texture descriptor consists of a standard volume concatenated with inter-channels measures, extracting better features of the underlying image than state-of-the-art morphological texture descriptors.
Our proposal led us to use standard measures, distances, proportions and RG-histograms in a novel application. In particular, coupling the volume with Canberra distance, inter-channel proportion and RG-Volume lead to the best results in the experimental database OutexTC00013, which we suggest for users as the most promising variants of our proposal.
For the experiments on the OutexTC00014, the images were pre-processed with equalization for each channel, thereby exhibiting lower performance in comparison with standard measures.
We believe that future development is likely to come in four ways: adding inter-channel measures, the extending the proposal to other color spaces, employing other types of morphological series and enhancing the performance in the illumination invariance problem. While we have explored the use of some well-known formulas, we are far from an exhaustive analysis. Furthermore, other authors might prefer to develop a custom measure to enhance the proposal. These custom measures would be a likely requirement to extend the proposal to other color spaces, in particular if they are not homogeneous (such as HSL or L*a*b*).