Class Separation Improvements in Pixel Classification Using Colour Injection

This paper presents an improvement in the colour image segmentation in the Hue Saturation (HS) sub-space. The authors propose to inject (add) a colour vector in the Red Green Blue (RGB) space to increase the class separation in the HS plane. The goal of the work is the development of an algorithm to obtain the optimal colour vector for injection that maximizes the separation between the classes in the HS plane. The chromatic Chrominace-1 Chrominance-2 sub-space (of the Luminance Chrominace-1 Chrominance-2 (YC1C2) space) is used to obtain the optimal vector to add. The proposal is applied on each frame of a colour image sequence in real-time. It has been tested in applications with reduced contrast between the colours of the background and the object, and particularly when the size of the object is very small in comparison with the size of the captured scene. Numerous tests have confirmed that this proposal improves the segmentation process, considerably reducing the effects of the variation of the light intensity of the scene. Several tests have been made in skin segmentation in applications for sign language recognition via computer vision, where an accurate segmentation of hands and face is required.


Introduction
In recent years, a significant amount of work has been published in the field of colour segmentation for Human Computer Interfaces (HCI). We would like to emphasize those related to the segmentation of the natural colour of skin. In this area, Phung et al. [1] proposed a skin segmentation method using a Bayesian classifier, obtaining satisfactory results for different colour spaces such as: RGB, Hue Saturation Value (HSV), Luminance blue-Chrominance red-Chrominance (YC b C r ) and Commission Internationale de l'Éclairage's Luminosity a-channel b-channel (CIE-Lab), even under adverse illumination conditions. Hsu et al. [2] suggested the detection of face skin considering a nonlinear subspace from the YC b C r space to partially compensate the luminosity variations.
The robustness of the segmentation against luminosity changes is one of the most desirable features in colour segmentation systems. For this reason, much work on this topic has been focused on minimizing the effects of illumination changes by using colour spaces where the luminance or intensity component can be easily isolated, thus providing chromatic constancy. The actual trend in applications with important time-varying-illumination changes is to use dynamic colour models that can adapt themselves to compensate for variations of the scene illumination. In this area, an extensive overview of previous investigations in the skin colour segmentation field is presented by Sigal et al. [3].
The most frequently used colour spaces in these types of applications are HSV [3,4] and normalized Red Green (rg) [5][6][7]. The HSV space, as well as the Hue Saturation Intensity (HSI) and Hue Lightness Saturation (HLS) spaces, are widely used in image processing because it is very intuitive for the human brain to interpret the information as it is represented. In some works, only the Hue (H) and Intensity (I) components are used in the clustering process [8]. In other cases, a threshold value for the Saturation (S) of each pixel based on its intensity is defined [9]. This threshold is used before the clustering process to determine if S should be replaced by H or I.
In general, all these segmentation proposals offer good results for objects with significant size in the scene or in cases where the main goal is object tracking, but not in the case of shape recognition. If the goal is to recognize the object shape, the system requirements are higher and very accurate segmentation techniques should be applied. Further difficulties may arise if the images have low quality and spatial resolution. Sign language recognition systems based on computer vision are a good example of these types of applications. In this case, the camera should capture all the upper parts of the speaker's body, implying that the parts to segment (hands and face) constitute a small part of the captured scene. In this field, Habili et al. [10] performed a pixel-by-pixel classification of the skin colour with discriminant features of the C b C r plane, using the Mahalanobis distance, but they needed a fusion of motion cues to obtain good results. Similar skin segmentation is achieved in the work done by Chai et al. [11], where post-segmentation stages were applied, such as morphological operations, in order to surpass the limitations of the segmentation. The YC b C r space has been also used [11]. This colour space is one of the most widely used in the segmentation process.
In this field, Ribiero and Gonzana [12] presented hand segmentation in video sequences by means of the Gaussian Mixture Model (GMM) background subtraction algorithm, which is a well-known statistical model for density estimation due to its tractability and universal approximation capability. In this work, [12], an adaptive Gaussian mixture in time is used to model each pixel distribution in RGB space. In Huang and Liu's work [13], clustering of colour images using GMM technique in HSV space is performed.
Less common colour spaces are also used in other works: both linear transformation spaces, like Luminance E-channel S-channel (YES) [14], and non-linear, spaces like the Uniform Chromaticity Scale (UCS) spaces, such as Luminance u-channel v-channel (L*u*v*) and its representation in cylindrical coordinates Intensity Hue Saturation (IHS) [15], Saturation Tint Value (STV) [16] which is a representation of HSV space by the normalized RGB components. Other spaces used are the Spherical Coordinate Transform (SCT) [17] and the geodesic chromaticity space pq [18].
We can also find works related to object/background segmentation with the objective of efficiently delimiting object edges. Some of these publications present the use of graph cuts in N-dimensional images to segment medical images from computed tomography (CT) scanners [19,20], and multilevel graph cuts to accelerate the segmentation and optimize memory use [21]. From our point of view, the main disadvantage of these works is that they are not designed for real-time purposes.
The conclusion of these previous works is that important unresolved problems still exist in order to obtain efficient skin segmentation, especially if we take into account that many applications require real time processing, include complex scenes, are prone to important illumination changes, and the objects to segment (face, arms and hands) are small when compared to the captured scene.
Our contribution to the solution of this segmentation problem is to use an object/background pre-processing technique to enhance the contrast (in the HS plane) between the colours corresponding to the objects to segment and the background in each frame. This pre-processing consists of increasing the separation between the object and background classes in the HS plane to optimize the segmentation in that plane.
In our proposal, to increase the class separation, a colour vector of components  R ,  G ,  B , is added to the R, G and B images directly captured from the camera, modifying the value of each pixel (n) to (R n + R , G n + G , B n + B ). The objective of this paper is to present the process needed to obtain the values  R ,  G and  B that optimize the separation between the classes of interest once the image has been converted to the HS plane. This optimization is carried out by means of an algorithm that maximises the Fisher Ratio. We have called the colour vector addition process "colour injection". In our proposal, the colour injection process is achieved using the relationships between the RGB, YC 1 C 2 [8,22,23] and HSI [24][25][26] colour spaces, and the properties of the C 1 C 2 plane. Our system may be particularized to recognize sign language in real-time and special attention has been paid to the detection of the geometric form of the parts to segment, hands and face edges, in each frame. Our proposal has been thoroughly tested with very good results even with illumination variations, because it isolates the I component. We always attempt to work outside the instability or achromatic zone of the HS plane, due to the convenient redistribution in the HS plane of the existing classes in the colour injected image (seen in [15] for the IHS space). In order to perform a comparative qualitative study between the segmentations of the original images and the colour injected images (proposal presented in this paper), a GMM clustering technique in the HS classification domain is used. This technique has been used in a similar way for the HSV space [13]. In previous works, different formulations for the HSI space can be found [22,[24][25][26]. We use the formulation proposed in [26]. This paper has been organized as follows: Section 2 describes the basis of the proposed algorithm to increment the separation between classes. Section 3 presents the criteria considered when separating the classes. Section 4 describes the off-line initialization stage of the proposed algorithm. Section 5 details how to improve the separation between classes in the HS plane starting from their location in the C 1 C 2 plane. Section 6 presents the algorithm that performs the optimal class separation. Section 7 describes how to obtain the colour vector for injection, and its effects in the captured images. Section 8 contains the experimental results, and Section 9 provides the conclusions and future work.

Overview of the Colour Injection Algorithm
The objective of this work is to improve the segmentation process using colour injection. In order to do that, a colour vector for injection is obtained for each captured image in the RGB space. This colour vector is considered optimal, because it is calculated to maximize the separation between the classes to segment in the HS plane (subspace where the segmentation is performed). For this reason, this colour vector will be called optimal colour vector in this paper and will be denoted by i r . It is injected in the RGB space and is calculated starting from significant samples (seeds) from the object to segment and from the part of the scene considered as background. The procedure to obtain the vector i r , and the reason why it is optimal is explained in Sections 6 and 7. This optimal colour vector is given by: where  R ,  G and  B are the increments of the colour components R, G and B, respectively.
The optimal colour vector, i r , is injected in every frame of an image sequence in real-time applications to segment objects in colour images. Its efficiency has been especially tested in applications where a reduced contrast between the background colour and the colour of the object to segment exists, when there are illumination changes and the size of the object to segment is very small in comparison with the size of the captured scene.
An important property of the perceptual colour spaces (such as the HSI space) is that they produce a maximum disconnection between the chrominance and luminance components. As a result, the luminance can be almost fully isolated, making the segmentation process more invariant to the changes in shades and illumination as in [4]. For this reason, the analysis of the colour injection effects in the HSI space is made only using the H and S chromatic components (HS plane).
As the segmentation is performed using the HS components, we try to separate the representative vectors of the two classes (object and background) in angle (H component) and in magnitude (S component) using colour injection. However, special attention should be paid in this separation process to the variations of the dispersions (reliability) of both classes after the colour injection, because it has a very high incidence in the class separation process.
In short, if the original image is denoted by I, the optimal colour vector to add by i r , and the coloured image resulting of the colour injection by I i , is fulfilled: The algorithm proposed in this work is formed by two clearly different stages: an off-line and an on-line stage. The off-line stage is an initialization phase whose objective is to determine the optimal number of existing classes in the initial frame, and, from that, to obtain the object (O class) and background (B class) classes needed to carry out their separation. The off-line stage is explained in detail in Section 4. The result of this stage is the set of significant pixels (seeds) in the RGB space that represent both classes, identified by: O RGB = {r O1 , r O2 … r ON }, and B RGB = {r B1 , r B2 … r BM }, respectively, where r Or for r = 1, 2… N and r Bq for q = 1, 2… M refer to the pixel vectors of the object and background classes, respectively.
The on-line stage is the novel contribution of this paper. Its objective is to determine the optimal colour vector to inject (i r ) for each frame in order to increase, optimally, the separation between classes O and B. The on-line process is executed before the segmentation process for each frame captured in real time. Figure 1 depicts the different phases of a segmentation process that uses the colour injection proposal of this paper. The on-line process consists of the following stages: (1) For every O RGB and B RGB sample from the captured RGB image I, a transformation to the YC 1 C 2 space is done. Considering the chromatic components after the transformation, the resulting classes will be referred to as O C1C2 = {c O1 , c O2 … c ON } for the object class and as B C1C2 = {c B1 , c B2 … c BM } for the background, where the pixel vectors are denoted by "c".
(2) Using the properties of the C 1 C 2 plane and the relationship between the HSI and YC 1 C 2 colour spaces, the optimal location of the classes in the C 1 C 2 space is obtained by finding the optimal location of their respective mean vectors. The optimal location is the one that maximizes the class separation in the HS plane (maximum distance between the class means and minimum class dispersions). These optimal mean vectors will be referred to as c iOopt and c iBopt . This phase is, undoubtedly, the most important of this work, and will be described in detail in subsequent sections.
(3) From the mean vectors c iOopt and c iBopt , their corresponding ones in the RGB space, r iOopt and r iBopt , are calculated.
(4) From the vectors r iOopt and r iBopt , and the mean vectors of the original classes (O RGB , B RGB ) denoted by r O and r B , the optimal colour vector for injection is obtained. This optimal colour vector can be calculated from one of these expressions: (5) Once the optimal colour vector has been obtained, the new "injected" image I i can be calculated applying (2).
Finally, the coloured image I i is transformed from the RGB space to the HS plane, where the segmentation is done, because the colour injection has its effects in the HSI space: the increase in the separation between the classes only happens in the HSI space or HS plane (in the RGB space the colour injection only produces a translation of the classes, keeping the distance between them constant, independently of the colour injection).
The proposed method can be implemented easily and can be used in real-time applications. In the following sections, the process to obtain the optimal vector for injection is presented in detail. At the end of this paper, in order to facilitate its reading, we have included three appendices with aspects related to the relationships between the RGB, the HSI and YC 1 C 2 spaces (Appendix A), statistical analysis of vectors in the RGB space and its relationships with the components in the HSI space (Appendix B), as well as the invariants of the mean vectors in the C 1 C 2 plane (Appendix C).

Criteria for the Separation between Classes
Since the objective of our work is to obtain a higher separation between the classes to facilitate the segmentation, it is necessary to define a measure of the efficiency of our proposal. The Fisher Ratio (FR) is frequently used to measure the efficiency in the class separability in classification systems [6,27,28]. This ratio quantifies simultaneously the inter-class separation and the internal dispersion (reliability) of the classes. For a two-class system, it is interesting to achieve a large distance metric between the class means and a minimum dispersion within each class (leading to a high FR). In this work, the FR is used as a pixel classification measurement index, using as discriminant features the H and S components of each pixel.
In a multi-class system, the generalized Fisher Ratio is expressed by [29]: where M b is the inter-class (between class) covariance matrix and M w is the internal (within class) dispersion matrix of the classes. Equation (4) cannot be directly applied due to the circular form of the H component trajectory. There are two main reasons for this: (a) For two-class systems (as our case is), M b may not represent the real angular distance between the hue means of the classes (the maximum angular distance between two vectors is  radians, even if one of the vectors is in the first quadrant of the HS plane and the other one in the fourth one). These problems have already been studied, for example in [8].
(b) The second reason is the discontinuity of the hue component when it moves from 2 to 0 radians (cyclic property). This implies that M w matrix does not represent the real hue variance of a class whose mean is close to 0 (2). The reason is that some of the vectors would have small angles (close to 0), and some others would have very high ones (close to 2), resulting in a wrong and high variance. The resulting H mean would also be wrong.
For the previous reasons, and supposing that the correlation between H and S is low, a particular FR has been defined. This FR is individually calculated for each component, and, as our space is bi-dimensional, is given by [29]: where FR H and FR S represent the Fisher Ratio of the H and S components, respectively, and are given by: represents the real angular distance between the hue means, because θ h = cos −1 (CC OB ). This avoids the aforementioned problem about the angular distance between the hue means of the mean vectors of both classes in the HS plane. CC OB is the correlation coefficient between the two mean vectors of the RGB components that have generated the mean vectors in the HS plane (Equation C.11) (see Appendix C). In this work, we have performed approximations in the calculation of  HO and  HB in order to avoid the problem of the hue discontinuity. Thus, the approximation for  HO is: 2 Ho Ho σ σ σ , where  CHo is the standard deviation of every cos(H Or ) for r = 1, 2… N and  SHo is the standard deviation of every sin(H Or ).  HB is calculated with a similar method.

Initialization Stage (Off-Line Process)
The first step of the off-line process is the capture of a first frame (initial image). The seed pixels that represent the classes O and B are obtained from this image, by means of any clustering technique used to identify the existing classes of the image, such as K-Means [30], GMM [13], etc. In this paper, the GMM technique is used (in the HS domain) because it provides highly reliable classes, and, as a final result, it also provides the mean vectors, the covariance matrixes and the a priori probabilities of the classes. The clustering by means of GMM uses the EM algorithm (Expectation Maximization) to obtain the optimal location and dispersion of a predefined image class number (K), projected in the HS plane. Therefore, a Gaussian model is assumed for each existing class, considering a uniform scene illumination. The GMM algorithm is applied several times, initializing it with different K values, in order to obtain the optimal number of existing classes in the image (K opt ). K opt corresponds with the K that produces the smallest error in the log-probability function of the EM algorithm, indicating that it is the best fit between the K Gaussians and the existing classes. Figure 2b shows the K opt Gaussians projected in the HS plane, fitted to the existing classes in the initial image of the example in Figure 2a.  Once the GMM algorithm has converged, the following step is to find out the localization of the object class (O) in the HS plane. This off-line process is carried out easily, because the approximate location of the object class (O) in the HS plane is known at the beginning of the off-line process, as a result of the colour calibration adjustments of the camera. This approximate geometric locus in the HS plane is given by the mean vector h init . Taking that into account, the detection of the object class (O) is performed by simply selecting the class with the minimum Euclidean distance with h init. We preferred the Euclidean distance over the Mahalanobis distance because the detection of the object class could be incorrect if h init is close to a class with high dispersion, and this class is also close to the object class (O). The reason of this effect is the consideration of the class covariance in the Mahalanobis distance.
Once the object class is detected, the next step is to select the background class (B). The background is usually formed by several classes, identified by {B 1 , B 2 … B Kopt1 }. Our objective is to select the B k class that will be considered as representative of the background and that we will be identified simply by B. Among all the classes that form the background, we will select the B k that: where FR k is the Fisher Ratio described in Section 3, and k B P ; k = 1, 2… (K opt  1) the a priori probabilities of each B k class to be the background class (B) of the image (given by the GMM algorithm). Once the classes O and B have been identified, the seed pixels that represent both classes are obtained through an initial segmentation process of both the object class (O) and the background class (B). In this initial segmentation, the pdf (probability density function) of both classes in the HS plane are considered as unimodal bidimensional Gaussians, defined by the parameters obtained by the GMM clustering. This segmentation is carried out by selecting the pixels with higher probability to belong to the corresponding Gaussian. This stage of pixel selection is performed by truncating each class pdf with a determined threshold. This threshold corresponds to a percentage of the maximum probability of the corresponding bidimensional pdf, P o , for the class O, and P b for the class B. The values of {P o , P b }  [1, 0], have been experimentally set to P o = 0.45 and P b = 0.6, using the Receiver Operating Characteristic (ROC) curves obtained from the different tests performed with real images. In this case a ROC curve was obtained by each class (O and B), using a set of real images, without and with colour injection. The thresholds P o and P b correspond with the nearest values to the elbows of the ROC curves. As the pixel selection is carried out in the HS plane, it is necessary to truncate the pdf in the Intensity axis, in order to obtain pixel sets that reliably represent the classes O and B in the RGB space. The truncation of this unidimensional pdf is necessary because the H and I components are independent (Appendix A) (this is important when the clustering is carried out in the HS plane) and this generates correspondence problems when the pixels in RGB components are selected from its projections in the HS plane. In this second pdf truncation, the percentages selected of the maximum value of the pdf intensity of each class are P fo for O, and P fb for B. These percentages have been set to P fo = 0.4 and P fb = 0.5.
Once the previous process is completed, a random sampling is carried out, selecting N samples for the class O and M for the class B, in order to reduce the working space dimension. This is the method to obtain the sets O RGB and B RGB mentioned in Section 2.

Separation of the Classes in the HS Plane from Their Location in the C 1 C 2 Plane
This section details the most important relationships between the statistical mean and variance of the classes in the C 1 C 2 and HS planes. Also, the effect of adding the same vector (colour injection) to two vectors in RGB space on the projections of these vectors in the C 1 C 2 and HS planes is analyzed. This information is used to define an algorithm to easily calculate the optimal vector to inject in order to obtain the maximum separation between classes in the HS plane using translations in the plane C 1 C 2 .

Relationships between the HS and C 1 C 2 Planes
Given two vectors in the RGB space, r O and r B , the resulting projection vectors in the C 1 C 2 plane, c O and c B , and in the HS plane, h O and h B , fulfil (see Appendix A): It is important to note that, since the C 1 C 2 plane is linear, when adding a vector i r (injected vector) to both r O and r B in the RGB space, the distance vector d c = c O − c B in the C 1 C 2 plane remains constant. These constant magnitude and orientation values (invariants of the d c vector) are denoted by ||d c || and  (see Appendix C). Therefore, colour injections in the C 1 C 2 plane result in class translations, as in the RGB space. This effect can be achieved with a translation vector i c (corresponding to i r ) directly added in the C 1 C 2 plane.
Moreover, in the case of the C 1 C 2 plane, (10) is verified (cosine law). Therefore, given that ||d c || remains constant for the different values of i r , the values of , ||c O || and ||c B ||, will be modified as a function of the value of i r . In the case of the HS plane, it must be said that if i r is added to the vectors r O and r B (contrary to what happens in the C 1 C 2 plane) the difference vector d h also varies. The reason is that, according to (11)  In short, to calculate the value of the colour vector to be added in the RBG space to obtain a particular separation between the classes in the HS plane, the authors suggest using the relationships between the h vector components in the HS plane, and their corresponding c vector components in the C 1 C 2 plane, given by (Equation B.12) and (Equation B.13) (see Appendix B), and the relationship between pairs of vectors in these planes, given by (8, 9, 10 and 11). Therefore, the proposed algorithm is based on the analysis of the behaviour of the vectors c O and c B in the C 1 C 2 plane and the properties of its difference vector d c (||d c || and  are invariant). These invariants allow us to establish a mathematical relationship between the class mean vectors before and after the colour injection. Thus, for example, the separation angle (hue difference) between two vectors in the HS plane can be easily controlled with the separation angle of the same vectors (c O , c B ) in the C 1 C 2 plane, because both angles coincide (8).
In Figure 3, an example of the correspondence between the vectors c O and c B in the C 1 C 2 plane and the vectors h O and h B in the HS plane is shown. The relationships after performing the colour injection (vectors c iO , c iB and h iO , h iB ) are also shown, as well as the difference vector d c before and after the colour injection, where the invariance in magnitude and angle can be observed. From now on, the "i" or "i" subscript refers to "colour injection". However, the vector modules (saturation) decrease (||h iO || < ||h O ||, ||h iB || < ||h B ||), since there is an unavoidable compensation effect given by (10) (notice that for a fixed I, ||h|| = const||c||f(H)).
We could obtain a great number of class locations within the HS plane relocating d c with i c all over the C 1 C 2 plane. The determination of the optimal location is not a trivial task. In order to obtain an optimal i c , it is possible to apply learning techniques, such as fuzzy systems and neural networks, that take as parameters some functions derived from FR's (6) and the invariants of the vector d c .
In the following Section (5.2), an algorithm for the calculation of the optimal i r (corresponding to optimal i c ), conditioned to ||c iO || = ||c iB ||, is explained.

Separation between the Hue Means (Angular Separation)
The separation between the hue means is given by the angular separation between the vectors h iO and h iB , which indicate the colour separation. Once the expression of the distance between h iO and h iB is obtained, ||d ih || (see Figure 3), an optimization process can be applied to it as a function of the RGB components of i r in order to obtain the optimal i r that produces the maximum separation needed. The problem when calculating the optimal colour vector is that it is not possible to obtain its analytical expression, mainly due to the discontinuities in the function of ||d ih || (11).
In order to solve the problem posed by the discontinuities of ||d ih || in the HS plane, the authors propose to use the C 1 C 2 plane, where the distance function between the vectors c iO and c iB , (||d c ||) (10) does not present discontinuities, and, as well, remains constant in magnitude and direction for different injections of colour vectors.
The interrelationship (due to the invariants of the vector d c and the relationships between the HS and C 1 C 2 planes) between the angle ( i ) that the vectors c iO and c iB form and their modules (10) should be taken into account to obtain the separation angle (hue difference) of two vectors in the HS plane. Therefore, the maximum separation angle between the vectors may imply (due to the compensation effect) a diminution of their modules, and, consequently, the saturation of both vectors. The saturation reduction of the vectors h iO and h iB implies that they become closer to the achromatic zone (the origin of the coordinates system), which means that the colours approximate to gray scale. The consequence of this phenomenon is the loss of discriminating power in the segmentation.
Therefore, the proposed algorithm has been parameterized as a function of the mentioned separation angle  i between the vectors c iO and c iB. In our case, the optimal angle  i is obtained from an observation function that measures the effectiveness of the class separation in different locations in the HS plane. This function will be described in paragraph f of Section 6.
When the angle of separation  i reaches a maximum,  i coincides with the angle whose bisector is a straight line p, which passes through the origin of coordinates and is perpendicular to the straight line, l, whose director vector is d c (Figure 4). Therefore, the vector for injection (i r ) that causes the maximum hue difference, causes the modules of both vectors c iO and c iB to become equal (||c iO || = ||c iB ||). It also causes the distance between the intersection point of the lines p and l and the extreme of each vector to be ||d c ||/2. Figure 4 illustrates an example of the location of the vectors c O and c B after the injection of the colour vector (c iO and c iB ) with those imposed restrictions.
The authors have given more importance to the angular (H) separation, because increasing both H and S at the same time is not possible. The main reason is that H has a discrimination power higher than S. Besides, the H component is totally uncorrelated to the I component, which does not occur to the S component (see Appendix B). Parameterization only by  i implies that we can only control the separation between the hue means. The starting point to obtain the distance between the saturation means is mainly the location of the vectors c iO and c iB with respect to the saturation weighting function in the C 1 C 2 plane. As can be observed, in this process there is no control on the S component, so its contribution on the class separation will depend on the modification of the statistics of this component with the variation of  i .

Separation between the Saturation Means (Saturation Difference)
In this section, an analysis of the behaviour of the separation between the saturation components of two vectors in the HS plane is performed. Given two vectors, for example h iO

Analysis of the Class Dispersion
In order to obtain the optimal vector for injection, i r , by means of the suitable election of  i , we should take into account not only the information given by the mean vectors c O and c B in the C 1 C 2 plane, but also the dispersion of the distributions of both classes. In this section we analyze the behaviour of the class dispersions in the HS plane, that is, how the hue and saturation dispersions are affected when the classes are translated in the C 1 C 2 plane, as a result of the colour injection. A class separation measurement function will be defined to quantify the effectiveness of the colour injection. This analysis will be necessary to understand how the H and S dispersions are modified with the colour injection, in addition to the performance of the class separation measurement function.

Hue dispersion (Angular dispersion)
The hue dispersion is determined by the effects of the dispersion transformation when passing from the C 1 C 2 plane to the HS plane. If R o is the (2  N) matrix formed by the N vectors of the O class: c Or ; r = 1, 2… N, before any translation, the parameters of the O class uncertainty ellipse, i.e., the hue dispersion invariants, are obtained from the covariance matrix of R o , by: where  O is the angle formed by the semi-major axis of the class uncertainty ellipse with respect to the horizontal axis (C 1 ), and C 1Ou and C 2Ou are the eigenvector components corresponding to the highest eigenvalue ( Ou ) of the covariance matrix. The semi-major and semi-minor uncertainty ellipse axes, u O and l O respectively, which represent the maximum and minimum variance, are given by: where  Ol is the minor eigenvalue of the covariance matrix. From these dispersion invariants, it is possible to obtain the model for the hue dispersion. Therefore, our interest is to obtain a correspondence between the hue dispersions in the HS plane by means of the information offered by the angular dispersion in the C 1 C 2 plane. Knowing that the variation of the angular dispersion in the C 1 C 2 plane corresponds with the variation of the hue dispersion in the HS plane, and since the C 1 C 2 plane is a Cartesian plane, the problem is posed in the polar coordinates, taking these two considerations into account: (a) As previously indicated, in the C 1 C 2 plane, the colour injections only produce translations of the classes and, therefore, variations of their mean vector modules (||c iO ||, ||c iB ||). This causes the modification of the angular dispersions of both classes, because they depend on C i = ||c iO || = ||c iB || (distance between the dispersion centre and the origin of the C 1 C 2 plane). These effects of the hue dispersion modification have been observed when performing translations of a class by adding Gaussian noise in the RGB space [22,31]. In conclusion, the angular dispersion increases when the magnitude of its respective mean vector decreases due to the increment of the separation angle  i , according to: (b) The geometric forms of the class distributions are not predetermined, but they can vary since they depend on the samples randomly taken from the object and the background. The colour injections produce class translations in the C 1 C 2 plane, implying that from the point of view of the HS plane, the dispersion also depends on the geometric form of the classes. The reason is that, for different translations of a class, different orientations between the axis of maximum and minimum dispersion (represented by their uncertainty ellipse in a C 1 C 2 plane) with respect to the orientation of their mean vectors (c iO or c iB ) are generated. Therefore, independently of the class mean vector module, a distance d a exists that contributes to the angular deviation. This distance d a depends only on the geometric form and orientation of the dispersion after each translation. Then, d a , in this case for the O class, will depend on the values of  O , u O and l O given by (13) and (14). This d a can be approximated by means of the distance between the centre of the uncertainty ellipse and the intersection point between two right lines: one is the tangent line to the ellipse which at the same time passes through the origin of plane, and the other line is perpendicular to the previous one and it crosses the centre of the ellipse. With d a and (15) the angular deviation can be approximated by: As an example, in Figure 5 we depict the object class (O) before a translation, for the addition of a vector i c in this C 1 C 2 plane, or, for the injection of a vector i r , directly to the classes in RGB components. Over the object class, its respective uncertainty ellipse is shown. In Figure 5, we can observe that the semi-major axis of the ellipse is relatively aligned to the mean vector c O of the class, causing the perception of the minimum angular dispersion of that class. It can also be observed that the module of this mean vector, c O , before the injection is greater than the module of the vector after it has been injected, c iO , which, therefore, is also perceived as a minor angular dispersion by this effect. We may conclude then, that the initial location of this class in the HS plane represents a very favourable case, since the angular deviation before the colour injection is small.
Nevertheless, for the background class (B) before the colour injection, certain alignment between the mean vector c B and the axis of greater dispersion of this class can also be observed, implying a reduced angular deviation. However, the problem is that the module of the vector c B is reduced and, therefore, the angular deviation increases. In this case, it can be observed in Figure 5 that after the colour injection, the angular dispersion of the class B i is smaller, since the module c iB is greater. Figure 6 depicts another example, with the different class locations after four colour injections. The modifications of the angular deviations  iHO and  iHB of the object and background classes as a function of the orientation of their respective uncertainty ellipses and the modules of their respective mean vectors can be observed.
This characteristic of linearity in the C 1 C 2 plane makes the deviation of the saturation ( S ) constant, since the distance between vectors in the C 1 C 2 plane remains constant, independent of the colour injection. Nevertheless, in the HS plane  S will be different for each lobe of f(H) but will stay constant within each lobe. Evidently, if the class vectors have different intensity, the dispersion of the saturation will not be constant for each location, not even within the lobes (there is a greater variation of  S when the dispersion of the intensity component is greater). Figure 7 illustrates how the hue and saturation dispersions are modified for the four colour injections of Figure 6. However, our interest in this paragraph is to understand how the colour injections affect the saturation dispersion. This is the reason why in our algorithm the S deviations of both classes are obtained considering their original intensities.

Algorithm for the Optimal Location of the Mean Vectors of Both Classes in C 1 C 2 Plane
This section presents the strategy used to obtain, in the C 1 C 2 plane, the mean vectors that maximize the separation between the classes in the HS plane. This section constitutes the main stage in Figure 1: "Optimal location of the mean vectors of the classes in the C 1 C 2 plane". As shown in Figure 1, for each captured image, an algorithm to obtain the optimal location in the C 1 C 2 plane of the mean vectors of both classes (object and background) is executed. From these optimal vectors, c iOopt and c iBopt , and once the transformation to the RGB space is performed (r iOopt , r iBopt ), the optimal vector to inject, i r , is obtained using (3).
The proposal to obtain these optimal vectors, c iOopt and c iBopt , consists of different phases, and its general block diagram is depicted in Figure 8. As can be observed, the proposal includes an iterative algorithm to obtain a set of locations for the mean vectors of the classes (c iO and c iB ) in the C 1 C 2 plane. The location of each vector will be parameterized by the angle formed between both vectors,  i . Therefore, we try to obtain a set of  in ( i1 ,  i2 …). Each of them will have associated a measurement index of separation between classes that we will identify by  HSn ( HS1 ,  HS2 …). From the function  HSn = f ( in ), the value of  in that produces the maximum separation between classes is obtained,  in optimal:  opt .
The process begins obtaining the mean vectors of each class in C 1 C 2 plane. These mean vectors will be, From the vectors c O and c B , its difference vector, d c, is obtained. As previously indicated, the magnitude, ||d c ||, and angle, , of the vector d c are invariant against translations in the C 1 C 2 plane.
Their values are given by equation (19): This  I is the parameter to vary in order to obtain the different locations of the vectors C IO and C IB , and, therefore, of the locations of the classes in the C 1 C 2 plane. The Cartesian components of these vectors (Figure 4), particularized for the vector C IO , are given by: where h IO is the angle of the vector that can be expressed by: Similar expressions can be obtained for C IB .
The iterative algorithm is initialized with an  i equal to  ( is the angle formed by the vectors c O and c B ). In each iteration (j) of the algorithm, the value of  i is increased: We should also take into account that  i represents the hue distance between the mean vectors (H IO and H IB ) of the classes in the HS plane. This indicates that a direct relationship exists between the class translations in the C 1 C 2 plane and the hue separation distance between the class means in the HS plane. As the class separation observation function, a normalized measurement index has been defined ( HS ) from the FR described in (5). It has been normalized to obtain  HSn = 1 when the class separation is maximum. To obtain the  HSn corresponding with each  in , we consider the mean and the dispersion of H and S of the classes, according to (6). Therefore, two class separation measurement indexes as a function of  in have been defined, one for each component: The final class separation measurement index is given by: where k h is a weighting factor between  Hn and  Sn . The value of k h  [0, 1] is chosen depending on the prominence we want to give H or S in the segmentation process. Taking into account that H has a greater discriminating power than S, k h > ½ should be fixed. This iterative process is repeated until the first non valid value of  in is generated, and the pairs ( HSn ,  in ) are registered to obtain the function  HSn = f( in ) afterwards.
Once the set of pairs ( HSn ,  in ) is obtained, the  in that produces the maximum class separation measurement index is selected. A cubic interpolation is performed around that local maximum to obtain the maximum of the interpolation index,  HSmax , and its associated angle,  opt . Finally, with this  opt , the c iOopt and c iBopt vectors are obtained using (20), (21) and (22).

Figure 9.
Hue and saturation deviation of both classes as a function of  in /2. The difference between the saturation means of both classes is shown too.
As an example, Figure 9 shows the variation curves, as a function of  in /2, of the statistical data: deviations of hue ( iHO and  iHB ), deviation of saturation ( iSO and  iSB ), and difference between the saturation means ||S iO − S iB || needed to obtain the different class separation measurement indexes (25).

Calculation of the Optimal Colour Vector to Add and the Effects that it Produces on the Images
The calculation of the optimal colour vector to add, i r , is the goal of our proposal, because this vector changes the colours of the captured image in a suitable manner, so that the classes separate and, therefore, the object class can be more easily segmented. Figure 10 shows the values of  Hn ,  Sn and  HSn obtained from the values of the statistical data depicted in Figure 9. The values of ( opt ,  HSmax ) obtained by interpolation are also shown. As depicted in the block diagram of Figure 1, once the vectors c iOopt and c iBopt , that represent the optimal location of the classes in the HS plane, are obtained, the vectors r iOopt and r iBopt can be calculated. Thus, for instance, for the object class, O: if C 1Oopt and C 2Oopt are the C 1 and C 2 components of the vector c iOopt respectively, the vector r iOopt in RGB space is obtained by: where Q is the transformation matrix (Equation A.2) and Y iO is the intensity mean of the object class translated in the C 1 C 2 plane. The i r vector is obtained with this r iOopt applying (3). Considering that the colour injection can be made without modifying the mean intensity of the class after the injection of i r , Y iO = I O holds. Although it is possible to modify the saturation mean varying the intensity mean, in this case, we want the saturation mean to be only affected by the f(H) value and the Chroma component (C). Therefore, the vector to inject, i r , should have zero mean (E{i r } = 0). The fact that E{i r } = 0 implies that the intensity mean of the original image (I) and the injected one (I i ) are equal. The effect of injecting the vector i r to the original image in the new image, I i , is a greater concentration of the pixel colours around the mean colour of each one of the two classes. That is, the colour injection contributes to the histogram equalization of the captured image in the HS plane. This equalization has a concentration effect on each class, and, therefore, the injection of i r contributes to approaching the class distributions to a Gaussian shape. As an example, Figure 11 shows the 2D histograms of image I (Figure 11a) and of the coloured image resulting from the colour injection I i , (Figure11b), for a particular case (Figure 12 images).  In these figures (11a and 11b), the equalization of the histogram produced by the effect of the colour injection can be clearly observed. The segmentation of both images is shown in Figure 12c and 12d respectively. In this example, K opt = 4, the O class corresponds with the jacket and the B class with the wall.  Figure 11a and (d) injected image segmentation according to the projected classes in Figure 11b.
The effect of the class separation between O and B classes can also be directly seen, analyzing the class locations before and after the colour injection in their histograms. Figure 13   The rest of the image classes different from B, B x  B; x = 1, 2 … K opt  2, are also affected by the effects of the colour injection. In this sense, as the class selected as B is the closest to the class O that also has a high probability to be the image background (fulfils equation (7)), when the separation between the classes O and B increases, the classes B x also increase their separation with the class O. However, the colour injection decreases the separation between the class O and those classes B x ( ' B x ; x = 1, 2 …) that are closer than B to the class O but that were not selected as class B because they had a lower a priori probability. The consequence is that these classes ( ' B x ) can be considered as class O, producing false positives in the object pixel classification. Another effect of the colour injection is the automatic compensation of the illumination changes. That is, due to the equalization and the separation of the classes O and B in the injected image, there is a minimization of the problems produced by the illumination changes. The reason is that the main colour component affected by the illumination changes is S, and, as previously explained, our algorithm gives more importance to the separation of the most discriminant component, H. Then, both classes, O and B, always keep a certain separation, independently of the parameter variation of both distributions, and mainly when the mean and variance of the S component vary due to changes in the luminous intensity.
Next, in Figure 14

Experimental Results
A bank of real images from different scenes has been used in a first phase of the practical tests, in order to evaluate the effectiveness of the proposed method. Here, a Gaussian classifier has been used as a segmentation technique, supposing a unimodal Gaussian model for the respective object and background class-conditional pdfs, i.e., p(h i |O i ) and p(h i |B i ). Thus, p(h i |O i ) = g(h i ; h iO ,  iO ) is given by: where h i represents each pixel of the image I i , and  iO is the covariance matrix of the injected object class in the HS plane. The segmentation is performed by thresholding the pdf (28) The problems derived from the cyclical nature of the hue in the segmentations have been solved via software, using the convention introduced by Zhang and Wang [8].
In the evaluation, the same number of samples (seeds) for the object class (O) and the background (B) has been taken, M = N, in order to ensure that the difference between their statistical data is for intrinsic reasons, and not for differences in the sample space dimension. In the tests, the following data have been used: samples number: M = N = 50. Other tests with a higher number of seeds (M = N = 100, 200, 400, 800 and 1,600) have also been carried out, providing similar qualitative results in all of them, but with an increase in the iterative process computational cost. The increase of  used in the algorithm shown in Figure 8:  = 5º , interpolation interval  = 3 and the weighting factor k h in (26) has been experimentally selected for each experiment, always fulfilling k h  [0.75, 0.97]. In this stage, the experimental results have been quantified by means of the FR defined in (5). Table 1 shows the values of FR for 14 cases of the bank of images used in the tests. Fourteen examples of segmentation can be seen in Figure 15 (figures a, b, c, d, e, f, g, h, i, j, k, l, m and n) that correspond with the 14 cases of FR calculated in Table 1 As can be observed, our proposal to inject a colour vector allows the attainment of remarkable improvements in the segmentation process, even with a segmentation technique as studied and effective as the Gaussian classifier. As a second phase of the experimental tests, and in order to quantify the improvement in the segmentation of the injected image with respect to the original image, an analysis, pixel by pixel, has been made, comparing with the manually segmented reference images for the 14 cases. The data generated in this analysis, without noise added, are shown in Table 2. The performance of the segmentation has been measured taking into account the classification Correct Detection Rate (CDR) and False Detection Rate (FDR) and the total Classification Rate (CR). CDR is the percentage of object pixels correctly classified, FDR is the percentage of background pixels incorrectly classified and CR is the total percentage of correctly classified pixels. Table 2 also shows the number (K) of Gaussians used by the GMM algorithm, the FR obtained by the statistics given by GMM for both classes, as well as the k h used for each image. Table 2. Comparative analysis of the segmentations for the 14 example images shown in Figure 15, without noise added. Table 3 shows the results of the comparison of the same images, but contaminated by additive zero-mean Gaussian noise. As can be seen, results obtained with the colour injection technique for both tests are better than those obtained using only a Gaussian classification.
As a third phase of the tests, an example of image sequence segmentation is presented. In this case, each frame illumination has been modified before the segmentation process, in order to verify the advantage of our proposal against illumination changes. The illumination is applied to each frame in a uniform way. A zero-mean Gaussian noise with standard deviation n p = 0.15I O was also added to the pixels of the images. Moreover, a sinusoidal time variation in the luminous intensity has been set up.
With this example, we try to show the improvements in the segmentation phase when the colour injection preprocessing step proposed in this paper is used before the segmentation. In this example, the GMM technique is used as an on-line segmentation technique (the same used in the off-line process). The original image I is identified by I k for each captured frame in time kT (k = 1, 2... and T = time between consecutive images), and its corresponding image after the colour injection, i I k , are segmented using the optimal class number obtained as a result of the off-line stage. In this case K opt = 5.
For the images I k and i I k , the GMM segmentation process is applied recursively using the a priori probabilities, means and variances obtained in the images I k1 and 1 i I k , respectively. For the segmentation of the image i I k , the next steps are also added: (a) we obtain the pixels (seeds) in the RGB space of the object and background of the image 1 i I k , (b) the vector i r is subtracted from them, (c) they are transformed to the HSI space, (d) the truncation process described in Section 4 is applied, and, finally, (e) the sets O RGB k and B RGB k are obtained. These steps (a, b, c, d and e) represent the block: "Obtaining seeds: Object (O) and Background (B)", for recursive segmentation in the block-diagram of the Figure 1. Table 3. Comparative analysis of the segmentations for the 14 example images shown in Figure 15, contaminated these images by additive zero-mean Gaussian noise. In image sequence segmentation, as this example, the iterative process (seen in Section 6) has been slightly modified in order to reduce the processing time and to increase the stability of the colour injection in time. The first modification, is to use opt 1 k θ  ( opt θ of the previous frame) as a starting point to obtain opt k θ , thus reducing the search interval to: In the example of Figure 16, we have fixed experimentally  f = 12º ,  = 1º and k h = 0.91. The second modification in the iterative process is that the optimal colour vector to inject r i k is obtained recursively, using for the calculation of k  opt the following expression: opt opt opt , where k t  [1, 0] is the constant fixed to obtain a proper smoothing of the evolution of the different parameters involved in the colour injection. k t has been fixed experimentally to 0.1. The GMM technique is used in these tests, mainly to obtain a better adjustment of the K opt Gaussians in each frame, and, therefore, to obtain the maximum quality in the object segmentation. Then, a reliable comparison of the segmentation quality between the segmentation of the images I and I i in the time can be carried out, and the compensation effects in the segmentation against illumination changes, applying the colour injection or not, can be verified. However, as it is known, this technique may have a relatively high computational cost due to the convergence iterations of the EM algorithm, so, its use in video segmentation is sometimes limited. For the consecutive segmentation of an image sequence in real time, our proposal in this work is to track recursively the parameters that define each Gaussian: O and B, using the optimal estimation provided by the Kalman filter, tracking technique widely studied in the image processing field. Figure 16 shows the results of the segmentation of the images I k and i I k of the example sequence, for the frames captured in k = 21, 42, 63, 84, 105, 126, 147, 168, 189   However, if the variation of the parameters of the different classes of the scene in the image sequence is very small, that is, when the scene is relatively uniform in the time with small illumination changes, the colour injection can be carried out applying the same colour vector i r to each frame in the time kT, with no need to recalculate it. This is possible due to the illumination compensation effect previously mentioned in Section 7. In this sense, the segmentation can be carried out keeping the parameters of both Gaussians as fixed values in all the sequence. Then, the computational cost is noticeably reduced. Figure 17 depicts the results of the segmentation of the images I k and i I k corresponding to the instants: k = 50, 100, 150, 200, 250 and 300 of the previous example image sequence, but, this time, without recalculating the colour vector i r and with fixed parameters for both Gaussians. The objective of this example is to appreciate the improvement in the segmentation of the sequence of colour injected images, although the same colour vector i r is used in the injection.
In Figure 17, the upper row depicts the I k images, the central row shows the results of the segmentation without colour injection (I k segmentation), and the lower one contains the results of the segmentations after the colour injection proposed in this work ( i I k segmentation). The segmented images show the object pixels in green colour. The segmentation process used in this phase has been used in the first stage of the experimental tests. Finally, we show some results of the real-time segmentation of images captured in a scene with significative illumination changes. These results highlight again the advantages of using the colour injection proposal presented in this paper. Figure 18 depicts the comparative results in two columns: the left column (a) shows the segmented images without the colour injection, and in the right one (b) the images segmented after applying the colour injection.
The segmentation has been performed by thresholding the pdf of the skin class seen in (28), once all the classes have been obtained with the GMM algorithm. In this case, K = 10 predefined classes were used. In order to demonstrate the robustness to illumination changes, an incandescent light bulb has been used (that produces a hue change in the whole image that tends to yellow) to really change the illumination of the scenes. The different luminous Intensity levels have been quantified with the mean intensity of the image normalized between [0, 1]. The corresponding Intensity levels for the five images of each column of Figure 18, starting from the image above, are: I 1 = 0.351, I 2 = 0.390, I 3 = 0.521, I 4 = 0.565, I 5 = 0.610. Figure 18. Results of the real-time segmentations with different illumination levels: (a) segmentation of original images captured directly from the camera, (b) segmentation of the images after the colour injection.
In the performance of this last practical test, a PC with an Intel Quad Core Q6600 @ 2.4 GHz processor and 2 GB SDRAM @ 633 MHz memory has been used. Although it is a last generation PC with four processing cores (CPUs), our application has only used a single CPU. A Fire-Wire video camera with a 1/2" CCD sensor with a 640  480 spatial resolution and an image capture rate of 30 fps (RGB without compression) was used. The optic used is a C-Mount of 3/4" with a focal length of f = 12 mm. The different algorithms of our proposal (GMM, colour injection and segmentation) have been developed in C, under Linux OpenSuSE10.3 (86_64) operating system. With this configuration, the average processing time of the on-line process (T p ) is approximately 2 ms for N = 50.

Conclusions
A method to increase the separation between two classes in a pixel classification process has been proposed. The experimental results demonstrate that injecting colour in the captured image guarantees good results in maximizing the class separation, implying that class distributions adopt more Gaussian shapes, and, therefore, the segmentation of the desired object improves.
Its practical implementation results are simple and the process time is small. Even though the algorithm needs to calculate both class deviations in each iteration, these are easy to obtain, considering that classes are formed by a limited number of samples (N), the increase of  i is not very small,  = 1º , and the search interval is not very wide:  f = 12º . This implies that calculations are relatively fast.
In this work, the expressions of interest to understand the vector's behaviour in the HSI space have been demonstrated from the respective statistics in the RGB space. Moreover, the equations to convert directly from YC 1 C 2 colour space to HSI space have been obtained.
Finally, we should indicate that the images have been directly obtained from the classification process without any other auxiliary stage, such as morphological operations.
For future work, our research is currently focused on the injection of a vector i r with a non-zero mean, as a function of the intensity mean desired in the image, to increase the compensation of the effects caused by the illumination changes. We are also developing the hue and saturation dispersion model when the classes are translated all over the HS plane, using the C 1 C 2 plane, (similar to the hue and saturation deviation estimation that is made in [31] for the HSI space defined in [24]). This will diminish the processing time, because it will not be necessary to calculate the hue and saturation variances of both classes in each iteration. Finally, we are doing research into the class separation applying higher order transformations that imply scales and rotations of the classes in the C 1 C 2 plane. This could solve part of the intrinsic limitations of the colour injections for just adding the vector i r . where CC 12 is the correlation coefficient of the vectors r 1 and r 2 , whose expression is given by: Cov  (C.11) We can conclude from (Equation C.11) that the angle between two vectors in the HS plane is the arccos of the correlation coefficient between both vectors in the RGB space, and from (Equation C.10) we conclude that this angle has a range between 0 and  radians.