Fast Color Quantization by K-Means Clustering Combined with Image Sampling

: Color image quantization has become an important operation often used in tasks of color image processing. There is a need for quantization methods that are fast and at the same time generating high quality quantized images. This paper presents such color quantization method based on downsampling of original image and K-Means clustering on a downsampled image. The nearest neighbor interpolation was used in the downsampling process and Wu’s algorithm was applied for deterministic initialization of K-Means. Comparisons with other methods based on a limited sample of pixels (coreset-based algorithm) showed an advantage of the proposed method. This method signiﬁcantly accelerated the color quantization without noticeable loss of image quality. The experimental results obtained on 24 color images from the Kodak image dataset demonstrated the advantages of the proposed method. Three quality indices (MSE, DSCSI and HPSI) were used in the assessment process.


Introduction
Color image quantization plays an auxiliary, but still very important role in such tasks of color image processing as image compression, image segmentation, image watermarking, etc.This operation significantly reduces the number of colors in the image and at the same time it should maintain a quantized image similar to the original image.Among the color quantization methods, two large groups can be distinguished, i.e., splitting techniques, e.g., median-cut (MC) [1], Wu's algorithm [2] and clustering techniques, e.g., K-Means (KM) technique [3].The latter technique gives very good results, but its computational complexity makes for a long computation time.
The results obtained using the KM technique depend on the initialization, i.e., on how we determine the initial centers of the clusters.Their random selection, usually used in the KM technique, leads to unrepeatable results.The use of splitting quantization in the form of MC or Wu methods can provide a deterministic initialization of KM [4].
The high computational complexity of KM results from the necessity of carrying out a very large number of comparisons between input data.This is especially true for images that contain millions of pixels.Therefore, the use of appropriate data structures and reducing the number of unnecessarily calculated distances between data allows for reducing the time required for KM [5].Another approach, particularly useful in color quantization, can be the operation on a small sample of the image.Then, the problem of how to sample the image remains to be solved: randomly or systematically?The other challenge is to define a proper size of such sample to receive accurate results and shorter computational time [6].In this paper by Bejarano et al., the maximal size of the sample was about 0.5% of the entire data and the clustering results were accurate and achieved in a much shorter time.A reduction in the number of pixels reduced the number of unique colors and then we could carry out the color quantization, for example, on a few thousand colors, not on a few million colors.The result should be a small loss of image quality and a significantly shorter time for the whole operation.The three main contributions of the proposed method are as follows:

•
An improved color quantization method may result in high image quality and short quantization time simultaneously.

•
Our method limits its working area to the stage of determining the color palette (initialization, clustering) to the image downsampled by the Nearest Neighbor Interpolation (NNI).

•
The quality indices of our quantized images compared with results for the method based on a limited sample of pixels (coreset-based algorithm) show an advantage of our method.
This paper is organized as follows: Section 2 is a short review of the literature on the applications related to the simultaneous color quantization and image downsampling.The proposed method is elaborated in details in Section 3. Experimental results and performance analyses are presented in Section 4. Finally, conclusions are given in Section 5.

Related Work
A theoretical basis for solving our problem is the image interpolation.This operation occurs during some image processing stages such as resizing (resampling), rectification, inpainting and morphing.The problem of image interpolation has been widely described in the work [7].Commonly used non-adaptive interpolation techniques are nearest neighbor interpolation (NNI), bilinear interpolation and bicubic interpolation.NNI is a simplest interpolation method that just replicates the nearest neighboring pixels.This means that, for color images, as a result of interpolation, no new colors are created.Thus, image upsampling does not increase the number of colors in the image and image downsampling can only reduce the number of colors, but their values will remain unchanged.
Different researchers have focused on using image sampling for acceleration of color quantization processes.In 2002, Papamarkos et al. [8] presented an idea of image downsampling via a fractal scanning technique based on Hilbert's space-filling curve for their Adaptive Color Reduction (ACR) algorithm.In this paper, the advantages of such fractal scanning in relation to a conventional raster scanning were highlighted.However, the primary goal of this downsampling was to speed up the entire process, especially important in the case of large images.
In 2014, Szilagyi and coworkers [9] preceded the KM-based color quantization by a special preprocessing consisting of static color quantization on about 140,000 colors and the color histogram computation with rejection of the least frequent colors.The key parameter of preprocessing is the percentage of image pixels included in the clustering process.The resignation from 2-5% pixels of the original image reduced the number of colors to 500-5000 colors and shortened the image quantization time (2-3 speedup ratio) without losing the image details.
In the work by Valenzuela, Celebi and Schaefer [10], an accelerated KM version was proposed that utilizes downsampling (decimation) of the input image, a non-deterministic KM++ initialization algorithm, efficient coreset construction and K-Means clustering.The authors used the name coremeans (CM) for their color quantization method.The decimation is the simple transformation of the discrete image, consisting of keeping every n-th sample and rejecting the rest.Two values of the decimation factor d f were considered in the article: {2, 4} and marked as CM2 and CM4.The KM++ algorithm is known from the literature [11].The coreset was constructed based on a downsampled original image and on the initial set of centers from the KM++ algorithm.The Lucic et al. algorithm [12] was adapted to the coreset construction.Four values of the coreset fraction c f =1/2, 3/8, 1/4, 1/8 were tested.In this paper, the CM method tests were performed on only four images.CM was significantly faster than the KM method and produced the quantized images of a similar quality to KM results.A detailed comparison of CM results and the results of the proposed method are given in Section 4.
Color quantization together with image sampling have been applied in pixel art [13], which is reminiscent of the hand-crafted work of pixel artists.Pixel art images, resulting from the automatic conversion of the color input image to a pixelated (iconic) output image, use low spatial resolution and a small color palette.The described conversion procedure contained two parts: SLIC (Simple Linear Iterative Clustering) algorithm that segments the input image into superpixels and an MCDA (Mass Constrained Deterministic Annealing) fuzzy clustering algorithm.As another application of our main operations, namely color quantization and image sampling, we can see a wireless image sensor network with reduced battery consumption in its nodes [14].This network was used for monitoring natural environment phenomena e.g., bird nest monitoring.Both operations significantly reduced the amount of transferred data between cameras and server and simultaneously reduced the energy consumption.This approach worked as a lossy image compression algorithm.However, compression was limited by the need to maintain a sufficient image quality needed for labeling by domain experts.For authors, the most beneficial color quantization method was quantization with a fixed palette (8, 16, 32 colors) and the bilinear interpolation algorithm for downsampling.

The Proposed Method
The main task of this article is to assess the impact of generating a color palette from a sample of image pixels instead of the whole image on the computation time and the quality of quantization.In 2010, Palus and Frackiewicz applied Wu's splitting method as an effective method for KM initialization [4].Figure 1 presents an idea of color quantization where an original full resolution image is a starting point for all processes i.e., initialization (Wu's algorithm), clustering (KM) and final pixel mapping.The time complexity of this quantization method based on K-Means is [15], where: n is the number of pixels, k is the number of expecting colors, d is the number of color components and i is the number of iterations.In contrast to this idea, the new proposed method (Figure 2) uses the original image only for pixel mapping.Both stages-initialization and clustering-are working on downsampled (DS) images.For downsampling with a d f factor, we decided to use popular and fast nearest neighbor interpolation.Such an approach should significantly reduce computational requirements and shorten the color quantization time.The time complexity of the proposed quantization method based on K-Means with downsampling is [15], where: d f is a downsampling factor, n is the number of pixels, k is the number of expecting colors, d is the number of color components and i is the number of iterations.Because the downsampling factor is a fraction, running time is shortened.
Nearest neighbor interpolation is used for an up-or downscaling algorithm in which vertical and horizontal scaling ratios are calculated as: where: w 1 , h 1 are the width and height of an original image, and w 2 , h 2 are the width and height of the image after enlarging/shrinking.Finally, the results of multiplying x, y coordinates and ratios are rounding to the nearest integer value.In this way, we receive the coordinates of pixels used in the downscaled image.Wu's quantization algorithm as a typical splitting method uses a statistical exploration of colors appearing in the original image.It is based on orthogonal bipartitioning and splits the RGB cube into k boxes, but, at each step, the box with the largest color variance is split along the axis that minimizes the sum of the variances on both sides.The author of the algorithm efficiently solved the problem of how to compute three-dimensional color statistics (means and variances) for these boxes.The means of the colors in the final boxes are selected as a color palette.Other details of the algorithm implementation can be found in Ref. [16].
The KM algorithm consists of four stages: • Initialization of the KM algorithm with cluster centers from Wu's algorithm,

•
For each pixel x i that represents RGB components, compute its membership m(c k |x i ) in each cluster center c k : • Recompute location of each center c k from all pixels x i based on their memberships: • Repeat two former stages until convergence.
All algorithms used in the proposed method were implemented in Visual C++ and the experiments were conducted on a PC computer running under a Windows operating system, with an Intel Core i7 920 @ 2.67 GHz (Santa Clara, CA, USA) and 8 GB RAM.

Experimental Results
In this work, two tests were performed to check the quality and efficiency of the proposed fast color quantization method.For each test image, the quantization into 32, 64, 128 and 256 colors was carried out and quality indices and CPU time were calculated.Generating a color palette was performed on all pixels of the image where the downsampling factor was 1/1 and on eight subsets of pixels obtained by NNI on the original image, where the downsampling factor was: 1/2, 1/4, 1/8, 1/16, 1/32, 1/64, 1/128 and 1/256.Finally, nine quantization results were obtained for each test image, each number of colors, the quality indices were calculated and CPU time was measured.
The most popular quality index widely used in the color image quantization is the MSE and the version for color images is defined as: where: MN represents the image resolution, R ij , G ij and B ij are the color components of the pixel (i, j) in the original image and R * ij , G * ij , B * ij are the color components of this pixel in the quantized image.This index is the same as the objective function used in the K-Means technique.
In the last few years, many perceptual indices for image quality assessment (IQA) have been developed.A good example of such index is the DSCSI (Directional Statistics based Color Similarity Index) [17], which consists of three steps.The first step is the image transformation from the RGB into the S-CIELAB color space.In a second step, the local features for color similarity are calculated for three color components: hue, chroma and lightness.In this way, we obtain the following six features: the hue mean similarity, the hue dispersion similarity, the chroma mean similarity, the chroma contrast similarity, the lightness contrast similarity and the lightness structural similarity.In the third step, these six features are combined into two scores: the chromatic similarity S C and achromatic similarity S A , which are directly used in the final DSCSI formula: where: I-original image, I * -distorted image and λ is a weighting factor.The smaller the difference between the original and distorted image, the value of DSCSI metric is closer to 1.
The last considered image quality index is named HPSI (Haar wavelet-based Perceptually Similarity Index) [18].HPSI is based on the coefficients of three stages of a discrete Haar wavelet transform.These coefficients are used for assessment of local similarities between two compared images.This index uses the six simple 2D Haar wavelet filters to detect horizontal and vertical edges.It is built in both local similarity maps (horizontal and vertical) and both weight functions.Additionally, a nonlinear mapping in the form of the logistic function is introduced in the HPSI computation process.The usefulness of both new indices to assess the image quality after color quantization show the current works [19][20][21].
The first test was performed on four color images: Baboon, Lena, Pepper and Pills, which are shown in Figure 3, to compare results with a method based on the coreset sampling [10].Baboon, Lena and Peppers are the test images with 512 × 512 pixels and the Pills with 800 × 519 pixels.The article, describing coreset sampling in color quanizations by K-Means, presents results of MSE and CPU time for the original image and four downsampled images with the following factors: 1/2, 3/8, 1/4, and 1/8.The authors used a non-deterministic KM++ initialization method and therefore they present the mean and standard deviation values of the performance measures for each quantized image.The fast quantization method proposed in this article, based on NNI and Wu's splitting initialization algorithm, allows for obtaining deterministic results.Tables 1-4 show a comparison of the mean MSE values for the two versions of the coreset-based algorithm and for the proposed NNI method in the case of 32, 64,128, and 256 colors, respectively.Tables 5-8 contain CPU times for each method in milliseconds.Figures 4 and 5 show the dependencies of the obtained speedup of calculations depending on the value of the downsampling factor for k = 128 and k = 256 where the advantage of the proposed method is significant.The visualization of the results was limited to the cases of k = 128 and k = 256, for which the predominance of the proposed method is the most visible.For cases k = 32 and k = 64, the results are presented in the tables only.
The second test was performed on 24 images from the Kodak image dataset [22] where each standard resolution image is either 768 × 512 pixels or 512 × 768 pixels and they were shown in Figure 6.In addition, the same images of high resolution (3072 × 2048 or 2048 × 3072 pixels) are also available in this dataset.
Figures 7-9 show the box plots obtained for all images from the Kodak image dataset.In Figure 7, these are plots for the classic MSE quality index, while Figures 8 and 9 contain quality assessments obtained for two new indices: DSCSI and HPSI.As can be seen, the decrease in image quality associated with sampling up to 1/128 is small and the change in quantization time is significant as shown in Table 9.This situation occurs for all three indices of image quality.The maximal speedup obtained in color quantization of 24 images was equal to 357 in the case of downsampling factor equal to 1/256 and a palette of 256 colors.In this case, however, the values of the quality indices deteriorated noticeably.On the other hand, such quality dropping was not observed for the case of the downsampling factor equal to 1/128, where the maximal speedup was equal to 166.The final stage of research on the Kodak image dataset was concerned with input images of different resolutions: standard and high.In the of high resolution images, there was no noticeable decrease in image quality due to the reduction of the downsampling factor to the value 1/256 (Figure 10).

Conclusions
Color quantization is still one of the important image operations.Hence, it is of great significance to design a fast color quantization method generating high quality quantized images.Such images are the result of the KM method with deterministic initialization by means of Wu's algorithm.Unfortunately, this is not a fast method.In this paper, we introduced an improved method, which significantly accelerated the quantization (sometimes a few hundred times) while not significantly lowering the quality of the obtained image.It should be noted that not only the classic MSE index was used to assess the quantized image, but also the DSCSI and HPSI indices strongly related to the observer's visual perception.The improvement of the method's operation consisted in the application downsampling of the input image through the NNI.The article shows that, the higher the spatial resolution of the image, the smaller the dependence of the result obtained by the improved method on the number of colors in the palette.In the future, we will analyze the usefulness of our method to other even more time-consuming clustering-based quantization techniques such as Fuzzy C-Means (FCM), K-Harmonic Means (KHM), etc.

Figure 1 .
Figure 1.Idea of the previous color quantization method.

Figure 2 .
Figure 2. Idea of proposed color quantization method.

Figure 10 .
Figure 10.MSE values for Kodak images with standard and high resolutions.

Table 5 .
CPU times for 32 colors.

Table 6 .
CPU times for 64 colors.

Table 7 .
CPU times for 128 colors.

Table 8 .
CPU times for 256 colors.

Table 9 .
CPU times (in milliseconds) for the Kodak image dataset.