Parallelized Inter-Image k-Means Clustering Algorithm for Unsupervised Classification of Series of Satellite Images

Abstract

As the volume of satellite images increases rapidly, unsupervised classification can be utilized to swiftly investigate land cover distributions without prior knowledge and to generate training data for supervised (or deep learning-based) classification. In this study, an inter-image k-means clustering algorithm (IIkMC), as an improvement of the native k-means clustering algorithm (kMC), was introduced to obtain a single set of class signatures so that the classification results could be compatible among multiple images. Because IIkMC was a computationally intensive algorithm, parallelized approaches were deployed, using multi-cores of a central processing unit (CPU) and a graphics processing unit (GPU), to speed up the process. kMC and IIkMC were applied to a series of images acquired in a PlanetScope mission. In addition to the capability of the inter-image compatibility of the classification results, IIkMC could settle the problem of incomplete segmentation and class canceling revealed in kMC. Based on CPU parallelism, the speed of IIkMC improved, becoming up to 12.83 times better than sequential processing. When using a GPU, the speed improved up to 25.53 times, rising to 39.00 times with parallel reduction. From the results, it was confirmed IIkMC provided more reliable results than kMC, and its parallelism could facilitate the overall inspection of multiple images.

1. Introduction

The volume of data obtained from satellites is increasing rapidly because of the development of satellite sensors and communication technologies. The geometric, spectral, and radiometric resolutions of satellite sensors are improving in tandem, and the size of scenes is increasing geometrically. As the speed of communication between satellites and ground stations improves, the volume of data received by ground stations grows. Moreover, nanosatellites (or cubesats) can capture more and more images with larger coverages and shorter revisit cycles, based on dense constellations and low altitude. For example, the PlanetScope mission involves more than 200 satellites, less than 4 m GSD, 400 square kilometer coverage, and a 90-min revisit cycle [1]. Because of the enhancing technologies and decreasing prices of storage media, there has been a steady rise in the size of the satellite images that could be stored in data warehouses. The development of computing power has led to an improvement in the conditions in which very large satellite images could be processed. However, it is not practically easy to process large satellite images via hardware improvements alone: corresponding software technologies should be developed and optimized for satellite image processing. As such, the advent of new and complex processing algorithms does not necessarily help to reduce processing times of larger satellite images, hindering their practical utilization. Therefore, there is an evident emerging need for research on reducing processing times.

Parallel processing is currently used for high-volume data processing and computationally intensive problem solving in a variety of fields, such as engineering, mathematics, and science. Parallel processing (or parallel computing) is a computing technology that divides complex problems into smaller parts and processes their data simultaneously on several computers. In the field of spatial informatics, large volumes of data are usually dealt with, and the complexity of processing methods is increasing considerably. As such, opportunities for using parallel processing in geographic information systems [2,3], remote sensing [4,5], and laser scanning [6,7] have steadily emerged. For example, in the field of remote sensing, parallelization was implemented in an optical real-time adaptive spectral identification system of which purpose was to analyze the endmembers of hyperspectral images [8]. In order to monitor and track wildfires in (near) real-time, another study focused on parallelizing classification algorithms based on automated morphological and self-organizing maps for hyperspectral images [9]. Endmember extraction (or pixel purity index) calculations for hyperspectral images were parallelized, and performance comparisons were performed on parallel processing hardware using various platforms [10,11,12]. Typically, relevant studies share a common aspect: they process hyperspectral images and parallelize various algorithms for classification and recognition.

Efforts have been made to develop parallel processing techniques that were suitable for new and affordable hardware. Starting from early 2010, there has been a trend toward parallel processing using graphics processing units (GPUs). They were applied, based on the compute unified device architecture (CUDA) of NVIDIA, to modulation transfer function compensation, cubic convolution interpolation, and discrete and wavelet inverse transformations [13]. CUDA was also used to apply the mean shift clustering algorithm to high-resolution remote sensing images [14]. Some researchers have developed GPU-based image mosaicking [15] and Sobel filtering [16]. However, the use of GPU-based approaches is not known ideal for very large image processing because of the limited capacity of GPU memory.

More recently, interest in cloud computing has grown rapidly, and there has been an increasing use of methods that allowed large satellite images to be processed using high-performance cloud servers. Cloud computing technology is a good candidate for processing large amounts of remote sensing data if the cost of high-performance computing implementations and technical usability is considered [17]. For example, maximum likelihood classification and Mahalanobis distance clustering algorithms have been implemented in a cloud computing environment [18]. However, the basic purpose of cloud computing is to make high-performance computing resources available to large quantities of clients through the Internet. When large satellite images are uploaded to cloud servers, the process of uploading itself can lead to inefficiencies, and the cloud resources cannot be reserved exclusively for the large projects. As an alternative, distributed computing decomposes data and processes them simultaneously. This concept can correspond to parallel processing; however, it is a loosely coupled method in which each node processes the data independently. Thus, it differs from tightly coupled parallel processing, which is based on computing nodes that share memory [5].

On the other hand, unsupervised classification is a preprocessing process implemented to investigate land cover distribution from satellite images without prior knowledge. Additionally, it is utilized to generate training data for supervised classification. Recently, the trend of satellite image classification has shifted to deep learning and a wide variety of pre-trained models were available for application to various landcovers and sensors [19], with studies being conducted to transfer and augment pre-trained models [20]. However, to train a new model for new objects and sensors, a large quantity of training data is still required, but acquisition cannot completely replace the use of human resources. Therefore, it is expected, for the time being, the unsupervised classification is applied to semi-automated training data collection in deep learning approaches.

Because unsupervised classification is generally assumed to be a preprocessing task, in some cases, when repeatedly performed using various parameters on multiple images, it is strongly expected to require less processing time in a fully automatic method. However, computing-intensive methods are still likely to be relevant to the attainment of acceptable results. For example, k-means clustering, a de facto standard for unsupervised classification algorithms, is implemented in prevalent remote sensing software such as ENVI, Erdas Imagine, and SNAP. It is also implemented through extensions or plugins in GIS software such as QGIS and ArcGIS. Due to its simplicity and efficiency in clustering a large dataset, the k-means clustering algorithm is still popular and widely used in relevant studies in native and further modified forms [21,22,23,24,25,26,27,28,29,30]. K-means clustering is composed of multiple loops and may take hours depending on the volume of images and input parameters. Nevertheless, when applied to multiple images, resulting classes would not be compatible among the images because the method would lack pre-trained class signatures. Therefore, improvements are needed to increase processing speed and apply to series of images acquired from recent satellite missions.

The study aims at two objectives: (1) performing unsupervised classification to acquire consistent and compatible results from a series of satellite images and (2) improving computational efficiency by implementing parallel processing. For the objectives, the study improves the k-means clustering algorithm in two steps: firstly, it classifies a series of images attained by the PlanetScope mission and produces a single set of class signatures so the resulting images share consistent class IDs. The results generated by the proposed algorithm were compared with those obtained from the native version. Next, the algorithm is parallelized using multi-cores of a CPU and a GPU. The parallelism consists of concurrent processing of the main classification routines using numbers of computational units, followed by further optimization based on parallel reduction utilizing a GPU. The performance is compared between models with a sequential code (without parallel processing), a multi-core code, a GPU code, and a GPU code with parallel reduction.

2. Algorithm Development

2.1. Inter-Image k-Means Clustering Algorithm

The native k-means clustering algorithm (hereafter, kMC) segments points in N dimensions into k clusters so the within-cluster sum of squares of distances is minimized [31]. The k-means clustering algorithm, applied to the unsupervised classification of a satellite image, consists of the following steps in Figure 1:

Figure 1. A general workflow of kMC.

In step 1, various kinds of methods have been developed to produce more accurate and unbiased centers and less iterations. Among these methods, a simple furthest-point heuristic can reduce the clustering error [32]. The method first sets a random center and then adds new centers using further points from newly-retrieved points. Therefore, if the computational cost of retrieving points is not that large, it is better to sweep all points and set class centers using the statistics of member points. Simply, the difference between the pixels’ maximum and minimum values in each band is split into k segments, and the center of each segment is set as the initial center of the class (Algorithm 1).

Algorithm 1 Pseudocode for class initialization of kMC

for (i = 1 to nRow ∗ nColumn)//for all pixels   for (j = 1 to nBand)//for all bands     Min[j] = smaller(Min[j] vs. Pixel[i][j])//retrieve the minimum value of band j     Max[j] = larger(Min[j] vs. Pixel[i][j])//retrieve the maximum value of band j for (i = 1 to nClass)//for all classes   for (j = 1 to nBand)//for all bands     interval = (Max[j] − Min[j])/nClass//set class interval     Center[i][j] = Min[j] + interval/2 + interval ∗ (i − 1)//set initial class centers

In step 2, Euclidean distances are calculated from each point to all the class centers, and a point is assigned to the closest class. In step 3, the average value is calculated for each band of each class and is set to a new class center (Algorithm 1). Equation (1) identifies the class $\widehat{j}$ into which a pixel P will be classified in a multispectral satellite image composed of N bands:

$$\begin{gathered} \widehat{j}={\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}_j\left(\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{t}\left(P,j\right)\right) \\ \mathrm{d}\mathrm{i}\mathrm{s}\mathrm{t}\left(P,j\right)=\sqrt{{\displaystyle \sum _{b=1}^N}{\left(P_b-C_b^j\right)}^2} \end{gathered}$$

(1)

where argmin_j(dist(P, j)) stands for the class j that minimizes the argument dist(P, j), which denotes the distance from a pixel to each class center; $P_b$ denotes the pixel value of band b of the pixel P; and $C_b^j$ denotes the center of band b of class j.

The pseudocode for steps 2 to 3 consists of four for-loops (Algorithm 2). The first for-loop iterates the entire classification process. The second for-loop retrieves all pixels. The third for-loop retrieves all classes, and the fourth for-loop does so for all the bands.

Algorithm 2 Pseudocode for main classification routine of kMC

1.  for (iter = 1 to nMaxIter)//iteration 2.    for (i = 1 to nRow ∗ nColumn)//for all pixels 3.      for (j = 1 to nClass)//for all classes 4.        for (k = 1 to nBand)//for all bands 5.          Dist[i][j] += ${\left(\mathrm{Pixel}\left[\mathrm{i}\right]\left[\mathrm{k}\right]-\mathrm{Center}\left[\mathrm{j}\right]\left[\mathrm{k}\right]\right)}^2$//distance from pixel i to class center j 6.      Class[i] = argminj(Dist[i][j])//classify pixel i to the class j which minimizes the distance 7.      if (isChanged(Class[i]))//if Pixel[i] is reclassified to a different class 8.        nChanged++//increase the number of reclassified pixels 9.      nPixelClass[Class[i]]++//increase the number of pixels of the class 10. … 11.     Center[Class[i]][ ] += Pixel[i][ ]//add the pixel values to the class 12. … 13.   Center[ ][ ] = Center[ ][ ]/nPixelClass[ ][ ]    //update class centers for all classes and bands when nPixelClass is not zero 14.   if (ChangedRatio < MaxChangedRatio)//iteration ending condition 15.     break

When k-means clustering is applied to multiple images, each image attains a different constitution of class signatures. Class ID is determined by chance, and an identical class ID among images does not indicate their land cover is the same. Unless it is necessary to attain separate class signatures for each image, integrated clustering of all images is the better choice for figuring out the landcover characteristics and their pixel value distribution through the entire images. In other words, if clustering is applied to continuously acquired images, it is better for a class ID to denote the same land cover through the images. In this study, a revised version of kMC is proposed, which is defined as inter-image k-means clustering (hereafter, IIkMC), to segment entire images using a single set of class signatures (Figure 2).

Figure 2. A workflow of IIkMC.

The pseudocode for IIkMC introduces an additional for-loop compared to the native code (Algorithm 3). The number of bands (nBand) is assumed to be the same because the same sensor is used. However, the dimension of an image (nRow and nColumn) is specified separately for each image as the effect of resampling in geo-rectification. All images are processed using a single set of (initial) class centers, which is updated using all the pixels of entire images through the classification process. In Algorithm 3, red text indicates changes from the native code, and blue text indicates variables that calculate a single set of class signatures.

As the revised algorithm includes five for-loops, time spent increases exponentially as the size of the image, the number of classes, and the number of images increase, necessitating the improvement of processing speed. Especially, as the third for-loop takes the longest iteration, it becomes the main target to be divided and processed concurrently.

Algorithm 3 Pseudocode for main classification routine of IIkMC

1.  for (iter = 1 to nMaxIter)//iteration 2.    for (img = 1 to nImage)//for all images 3.      for (i = 1 to nRowimg ∗ nColumnimg)//for all pixels 4.        for (j = 1 to nClass)//for all classes 5.          for (k = 1 to nBand)//for all bands 6.            Dist[i][j] += ${\left(\mathrm{Pixel}\left[\mathrm{img}\right]\left[\mathrm{i}\right]\left[\mathrm{k}\right]-\mathsf{Center}\left[\mathrm{j}\right]\left[\mathrm{k}\right]\right)}^2$//distance from pixel i to class center j 7.        Class[img][i] = argminj(Dist[i][j])//classify pixel i to the class j which minimizes the distance 8.        if (isChanged(Class[img][i]))//if Pixel[i] is reclassified to a different class 9.          nChanged++//increase the number of reclassified pixels 10.       nPixelClass[Class[img][i]]++//increase the number of pixels of class j 11. … 12.       Center[Class[img][i]][ ] += Pixel[i][ ]//add the pixel values to the class 13. … 14.   Center[ ][ ] = Center[ ][ ]/nPixelClass[ ][ ]    //update class centers for all classes and bands when nPixelClass is not zero 15.   if (ChangedRatio < MaxChangedRatio)//iteration ending condition 16.     break

Likewise, all pixels of all images should be swept when determining the initial class centers, which is the first step in Figure 1. The corresponding pseudocode is shown in Algorithm 4.

Algorithm 4 Pseudocode for class initialization of IIkMC

for (img = 1 to nImage)//for all images   for (i = 1 to nRowimg ∗ nColumnimg)//for all pixels     for (j = 1 to nBand)//for all bands       Min[j] = smaller(Min[j] vs. Pixel[img][i][j])//retrieve the minimum value of band j       Max[j] = larger(Min[j] vs. Pixel[img][i][j])//retrieve the maximum value of band j for (i = 1 to nClass)//for all classes   for (j = 1 to nBand)//for all bands     …     Center[i][j] = Min[j] + interval/2 + interval ∗ (i − 1)//set initial class centers

2.2. Parallel Processing of Inter-Image k-Means Clustering Algorithm

Parallel processing of kMC can be divided into intra-node parallel processing and inter-node parallel processing [33]. Intra-node parallel processing uses the multiple cores of a CPU and shared memory in a PC. OpenMP [34] programming can be used to control the multiple cores of a CPU, which all share data in the random access memory (RAM) and hard disk drive (HDD). Intra-node parallel processing can be favorably applied to the parallelization of the second loop (line 2) in Algorithm 2 because the loops constitute the worst bottleneck encountered throughout the procedures. Inter-node parallel processing uses multiple CPUs, each of which has its own OS and memory. MPI [35] programming is typically used to control CPUs in the nodes (PCs or workstations), transmitting data between the nodes through an external network (such as Ethernet). The inter-node parallelism can be implemented in a hybrid fashion with a PC cluster used to implement the inter-node method and the multiple cores of its nodes deployed to implement the intra-node method [33]. The third approach is based on general-purpose GPU. A typical GPU-based parallel implementation for kMC was applied to the second loop (line 2) in Algorithm 2 [36], in which lots of GPU cores were exploited to enormously accelerate Euclidian distance calculation. The fourth approach introduces MapReduce programming based on a Hadoop cluster for large data processing [37]. However, MapReduce programs are known to take a considerable time to process problems requiring many iterations, while OpenMP and GPU allow higher processing speed and more flexible control than MapReduce [38]. Hence, the present study develops parallel IIkMC based on the intra-node approach and the GPU method (Figure 3).

Figure 3. Procedures to which intra-node and GPU-based parallel processing is applicable.

2.2.1. Intra-Node Parallel Processing

Intra-node parallel processing is conducted by simultaneously processing $N$ sub-images, each $1/N$ times the size of an entire image, using $N$ cores in a CPU. This step is simply implemented by inserting an OpenMP clause #pragma omp parallel for before the third loop (line 3) in Algorithm 3.

Unfortunately, OpenMP is known to suffer from false sharing. If this problem is not resolved, the method consumes more time than when sequential (non-parallel) processing is used. False sharing is a phenomenon that occurs when different cores modify variables that are along the same cache line. Because the memory is forced to update to maintain cache consistency, the processing speed is greatly reduced [39]. This happens frequently when the contents of variables in a for-loop are changed almost simultaneously by different cores. To prevent false sharing, the variables with changed contents in each core must be placed in a private( ) construct, and i, j, and k should be defined as private after the parallel for clause. Variables that are placed in a private( ) construct are declared independently for each core and do not interfere with each other. A reduction( ) construct is used to sum up values without false sharing. The nChanged variable is used to count the total number of pixels with changed classes, and is defined as reduction(+:nChanged). Therefore, the false sharing problem should not occur (Algorithm 5).

Algorithm 5 Implementation of intra-node parallel processing using OpenMP

for (iter = 1 to nMaxIter)//iteration   for (img = 1 to nImage)//for all images #pragma omp parallel for private(i, j, k) reduction(+:nChanged)     for (i = 1 to nRowimg ∗ nColumnimg)//for all pixels       for (j = 1 to nClass)//for all classes         for (k = 1 to nBand)//for all bands         …         nChanged++//increase the number of reclassified pixels         …

Despite these precautions, false sharing can still occur. nPixelClass and Center in Algorithm 3 are pointer arrays, which store the number of pixels classified in each class and the class center of each band. During classification, the values of nPixelClass or Center can be changed when accessed at the same time by different cores. When this happens, so does false sharing. To solve this problem, nPixelClass and Center are declared to be global variables, and nPixel_local and Center_local are declared to be local variables for each core, as illustrated in Figure 4. In each core, Center is shared during classification, but the numbers of pixels and pixel values for each class are accumulated in the local variables nPixel_local and Center_local. When classification is finished, all cores’ local variables should be summed up to the global variables Center and nPixelClass, and the centers of each class are recalculated by averaging them. This method prevents false sharing, and the pseudocode is given in Algorithm 6.

Algorithm 6 Pseudocode for recalculation of class centers

for (i = 0 to nClass)//through all classes   for (j = 0 to nThread)//through all cores     for (k = 0 to nBand)//for all bands       …     Center[i][ ] += Center_local[i][ ]//update global center using local ones for all bands     nPixelClass[i] += nPixel_local[i]//update global no. of pixels using local ones …   Center[i][ ] = Center[i][ ]/nPixelClass[i]   //recalculate class centers for all classes and bands when nPixelClass is not zero

Figure 4. Global and local declarations of variables nPixelClass and Center (revised from [33]).

As shown in Algorithm 4, when initializing the class centers as the first step in Figure 1, there is a triple for-loops, and the speed can be improved by inserting an OpenMP clause #pragma omp parallel for before the second for-loop (line 2). Since this step also causes a false sharing problem in the Min and Max variables, local variables Min_local and Max_local should be declared for each core and integrated into the global variables Min and Max. Since this is relatively simple, a more detailed explanation is omitted.

2.2.2. GPU-Based Parallel Processing

As a simple implementation method, the main classification routine (lines 2 to 7) in Algorithm 3 is divided into blocks and several blocks are concurrently processed using many arithmetic logic units (ALUs) in a GPU. In other words, a pixel of an image is assigned to an ALU, and the ALU calculates the distance to all class centers and designates the pixel to the shortest class. Thousands of ALUs concurrently conduct the same process for each pixel invoked by a kernel function. However, before the process, the class centers are initialized by a CPU and transferred to the GPU memory (hereafter, device memory), along with the image itself from main memory. The class information of each pixel, as the result of the main classification routine, is transmitted from the device memory to the main memory, and the class centers are updated by the CPU. In summary, the GPU is responsible for the distance calculation and classification of each pixel and the CPU is responsible for initializing and updating the class centers. Pseudocode for the GPU-based parallel processing is given in Algorithm 7.

Algorithm 7 Pseudocode for GPU-based parallel processsing of IIkMC

1.  for (iter = 1 to nMaxIter)//iteration 2.    cudaMemcpy(devCenter, Center, …, cudaMemcpyHostToDevice)      //upload (initial) class centers to GPU 3.    for (img = 1 to nImage)//for all images 4.      cudaMemcpy(dev_Image, Image[img], …, cudaMemcpyHostToDevice)      //upload an image to GPU 5.      cudaMemcpy(devClass, Class[img], …, cudaMemcpyHostToDevice)      //upload (initial) class information of each pixel to GPU 6.      nBlock = (nRowimg ∗ nColumnimg - 1)/nThread + 1//nThread is a multiple of 32 or 64 7.      kmeansKernel<<<nBlock, nThread>>>(devImage, devClass, devCenter, …)      //invoke parallel classification 8.      cudaMemcpy(Class[img], devClass, …, cudaMemcpyDeviceToHost)      //download new pixel class from GPU 9.      for (i = 1 to nRowimg ∗ nColumnimg)//for all pixels 10.       if (isChanged(Class[img][i]))//if Pixel[img][i] is reclassified to a different class 11.         nChanged++//increase the number of reclassified pixels 12.       nPixelClass[Class[img][i]]++//increase the number of pixels of class j 13. … 14.       Center[Class[img][i]][ ] += Pixel[i][ ]//add the pixel values to the class 15. … 16.   Center[ ][ ] = Center[ ]/nPixelClass[ ]      //recalculate class centers for all classes and bands when nPixelClass is not zero 17.   if (ChangedRatio < MaxChangedRatio)//iteration ending condition 18.     break

Initializing and updating class centers requires more complex designs be used for GPU-based parallel processing. Initialization requires searching the minimum and maximum values of all pixels, and updating requires summing all pixel values for each band of each class along with the counts of the pixels in each class. This kind of operation utilizes an array of numerous values to produce a single output and is referred to as “reduction”. In the intra-node parallel processing, data can be divided into several sub-sections, and each section can be serially reduced by each CPU core. Here, a serial reduction is defined to iterate through the entire array, performing the reduction on pairs of elements, and updating the accumulator until the result is obtained. However, this approach is not well suited for the massively parallel architecture of a GPU. Because each ALUs is not fast as a CPU core, it should be applied to finer pieces of tasks. Thus, a parallel reduction is conducted in the fashion that each ALU performs a summation or logical operation for finer entities. This procedure is iteratively performed for the results until a single result is derived. However, as the iteration goes on, the number of ALUs participating in the operation decreases exponentially. It is important to make as many ALUs still participate in the reduction. Parallel reduction has been sufficiently studied to enhance procedural efficiency based on CUDA [40]. In this study, sequential addressing, a middle-optimized parallel reduction among the relevant approaches (Figure 5), was applied to the following procedures in Figure 6.

Figure 5. Illustration of parallel reduction: (a) basic concept, (b) sequential addressing [40].

Figure 6. Procedures to which parallel reduction is applicable.

Procedure 1 is implemented in initializing class centers, procedures 2 and 3 are applied to the main classification routine, and procedure 4 is used to update the class centers. Each step is executed with a pair of a kernel invocation function and a reduction function. The kernel invocation function organizes input data into an array compatible to parallel reduction and performs the first reduction. The reduction function iterates until a result is derived using a parallel reduction routine. In this way, IIkMC is fully parallelized using a GPU without any CPU implementations. The pseudocode is shown in Algorithm 8. Unlike line 7 in Algorithm 7, one more argument is given within <<< >>> in the kernel invocation in Algorithm 8. This dramatically improves the speed of the parallel reduction using cache memory (shared memory in CUDA) allocated to each ALU. However, the details of this are omitted in the manuscript.

Algorithm 8 Pseudocode for fully parallelized GPU-based IIkMC using parallel reduction

//Class center initialization for (img = 1 to nImage)//for all images   …   minMaxKernel<<<nBlock, nThread, …>>>(devImage, devClass, outArray, …)      //invoke parallel min, max retrieval   reduce_minmax(outArray, minMaxArray)      //reduce min and max values to minMaxArray   …   //Classification routine 7. for (iter = 1 to nMaxIter)//iteration 8.   …  for (img = 1 to nImage)//for all images 9.     … 10.     kmeansKernel<<<nBlock, nThread, …>>>(devImage, devClass, devCenter, outArray, …)       //invoke parallel classification 11.     reduce(outArray, nChanged)//reduce class-changed pixel counts to nChanged 12.     … 13.     classPixelCountKernel <<<nBlock, nThread, …>>>(devClass, outArray, …)       //invoke parallel pixel count 14.     reduce(outArray, nPixelClass)//reduce pixel counts to nPixelClass 15.     … 16.     newCenterKernel<<<nBlock, nThread, …>>>(devImage, devClass, outArray, …)       //invoke parallel pixel value summation 17.     reduce(outArray, Center)//reduce pixel values to Center 18.     … 19.     Center[ ][ ] = Center[ ][ ]/nPixelClass[ ]       //recalculate class centers for all bands when nPixelClass is not zero 20.   if (ChangedRatio < MaxChangedRatio)//iteration ending condition 21.     break

3. Evaluation and Discussions

3.1. Evaluation Method

A series of adjacent PlanetScope images were processed using kMC and IIkMC, and the classification results were compared. IIkMC was implemented in a sequential code (without parallel processing), an intra-node parallel code, a GPU-based parallel code, and a GPU-based parallel code using the parallel reduction. The performance of the parallel codes was compared to the sequential variant.

A total of twelve images were achieved over two days from 30 October 2018 to 31 October 2018 (Figure 7). Each image is composed of four multi-spectral bands (red, green, blue, and infra-red) and the file type is 16-bit geotiff. The dimension of each image is approximately 8.2 k × 3.9 k and the average size in the uncompressed tiff is 244 MB. The computing system has an AMD Ryzen 9 CPU consisting of 16 cores for intra-node parallel processing and an NVIDIA RTX 3090 Ti graphics card for GPU-based parallel processing (Table 1). The number of classes (k) is eight, and the iteration termination condition is 1.0% (Table 2).

Figure 7. Placement of 12 PlanetScope images (Enlargements of (a) and (b) are in Appendix A, and location names are in local terms.)

Table 1. Specifications of the computing system.

CPU	GPU	RAM	SSD	OS	Library
AMD Ryzen 9 7950X 16-Core Processor 4.50 GHz	MSI Geforce RTX 3090 Ti 24 GB	Samsung DDR5 64 GB	Samsung SSD 980 pro 2 TB	Windows 11	CUDA Toolkit 12.2, gdal:x64-windows 3.7.2#1

Table 2. Parameters of k-means clustering.

Number of Classes (=k)	Maximum Number of Iterations	Iteration Termination Condition
8	100	Ratio of class-changed-pixels below 1.0%

3.2. Evaluation of Clustering Results

The results of the separate processing of each image using kMC are shown in Table 3 and Figure 8. A same-color map was applied to each result to match the same class ID with the same color. The number of iterations differed greatly, and image 3 was processed after three iterations, resulting in incomplete segmentation. With smaller termination condition, it would have produced different results, but it should be remarked 1.0% was not an unreasonable condition. One class was canceled in image 9, and two classes were canceled in image 11, which may have occurred accidentally due to the spectral characteristics of the initial and updated class centers. This problem might be solved by applying class-splitting step to the classification routine, which was not considered in this study. Results 1 and 7 to 12 appear to have similar class signatures compatible among themselves; images 3, 4, and 6 also might, but the rest do not.

Table 3. Results of kMC (The numbers next to each class IDs are the numbers of entities and ratios of the pixels classified to the classes.).

Image	1		2		3		4
Date	30 October 2018		30 October 2018		30 October 2018		30 October 2018
Dim.(Size)	8146 × 3914 (243 MB)		8147 × 3917 (243 MB)		8157 × 3915 (244 MB)		8166 × 3916 (244 MB)
# of iter.	52		3		42		40
Class 1	4,928,065	24.8%	13,402,654	67.4%	3,959,627	19.9%	4,160,075	20.9%
Class 2	5,810,526	29.2%	6,187,532	31.1%	5,236,923	26.3%	5,025,127	25.2%
Class 3	3,387,364	17.0%	283,050	1.4%	6,219,358	31.2%	6,108,950	30.7%
Class 4	2,655,411	13.4%	15,046	0.1%	2,281,256	11.5%	2,358,259	11.8%
Class 5	2,448,135	12.3%	1610	0.0%	1,627,263	8.2%	1,561,728	7.8%
Class 6	93,745	0.5%	385	0.0%	526,105	2.6%	588,315	3.0%
Class 7	540,304	2.7%	86	0.0%	52,455	0.3%	12,420	0.1%
Class 8	15,364	0.1%	26	0.0%	6657	0.0%	98,733	0.5%
Sum	9,878,914	100%	19,890,389	100%	19,909,644	100%	9,913,607	100%
Image	5		6		7		8
Date	30 October 2018		30 October 2018		31 October 2018		31 October 2018
Dim.(Size)	8156 × 3916 (244 MB)		8155 × 3916 (244 MB)		8147 × 3932 (244 MB)		8164 × 3934 (245 MB)
# of iter.	74		43		16		39
Class 1	3,502,094	17.6%	4,068,726	20.4%	8,492,928	42.9%	6,254,820	31.5%
Class 2	5,179,392	26.0%	5,257,784	26.4%	8,301,107	41.9%	6,985,341	35.2%
Class 3	2,772,873	13.9%	6,329,473	31.8%	2,453,941	12.4%	3,049,602	15.4%
Class 4	4,009,512	20.1%	2,397,961	12.0%	500,123	2.5%	2,583,808	13.0%
Class 5	2,542,993	12.8%	1,484,225	7.5%	45,554	0.2%	783,952	4.0%
Class 6	1,525,131	7.7%	342,262	1.7%	6556	0.0%	150,848	0.8%
Class 7	363,732	1.8%	28,155	0.1%	582	0.0%	18,135	0.1%
Class 8	17,147	0.1%	2662	0.0%	39	0.0%	254	0.0%
Sum	19,912,874	100%	9,911,248	100%	19,800,830	100%	9,826,760	100%
Image	9		10		11		12
Date	31 October 2018		31 October 2018		31 October 2018		31 October 2018
Dim.(Size)	8172 × 3935 (245 MB)		8165 × 3937 (245 MB)		8156 × 3936 (245 MB)		8148 × 3934 (244 MB)
# of iter.	15		27		32		57
Class 1	6,699,672	33.8%	6,583,322	33.2%	6,173,945	31.1%	5,084,463	25.7%
Class 2	9,142,172	46.1%	8,092,120	40.8%	7,116,448	35.9%	5,053,343	25.5%
Class 3	3,053,169	15.4%	3,194,369	16.1%	3,442,373	17.3%	3,666,710	18.5%
Class 4	830,459	4.2%	1,691,523	8.5%	2,444,176	12.3%	3,637,968	18.4%
Class 5	102,337	0.5%	262,921	1.3%	619,511	3.1%	69,454	0.4%
Class 6	14,050	0.1%	26,349	0.1%	52,898	0.3%	1,875,924	9.5%
Class 7	669	0.0%	558	0.0%	-	0.0%	429,296	2.2%
Class 8	-	0.0%	42	0.0%	-	0.0%	1441	0.0%
Sum	19,842,528	100%	9,851,204	100%	19,849,351	100%	19,818,599	100%

Figure 8. Results of kMC (Enlargements of (a) and (b) are in Appendix A, and location names are in local terms.)

The result of concurrently processing the 12 images using IIkMC is shown in Table 4 and Figure 9. The result was derived after 55 iterations, and the class signatures appeared to be compatible throughout the results. From the enlarged image of Figure 7 (a) in Figure A1 and the corresponding results in Figure A2 and Figure A3, it can be clearly seen IIkMC has better class compatibility among the results. From the enlarged image of Figure 7 (b) in Figure A4 and the corresponding results in Figure A5 and Figure A6, it could be confirmed a complete result was obtained for image 2, which was not properly segmented by kMC. From the inspection, it was also confirmed none of the classes were canceled in the results of images 9 and 11.

Table 4. Results of IIkMC.

	Class 1	Class 2	Class 3	Class 4	Class 5	Class 6	Class 7	Class 8	Sum
# of pixels	43,484,072	52,010,051	68,692,476	35,446,580	27,183,058	1,682,832	9,736,596	170,283	238,405,948
Ratio	18.2%	21.8%	28.8%	14.9%	11.4%	0.7%	4.1%	0.1%	100%

Figure 9. Results of IIkMC (Enlargements of (a) and (b) are in Appendix A, and location names are in local terms.)

However, from Figure A3 and Figure A6, it could be remarked over-segmentation might have occurred in the results of IIkMC. The possible reason for this was acceptance of the spectral features from multiple images could have broadened the spectral distribution, and, as a result, separability among classes may have decreased. Separability can be better understood by inspecting the distribution of the center and standard deviation of each class in the spectral domain. In the results of kMC, the standard deviation starts in the mid 100 s (Table A1), while in IIkMC, it starts in the late 200 s (Table A2). A larger standard deviation can imply lower separability. Therefore, it will be necessary to reduce the spectral distribution by applying histogram matching among images before clustering.

Nevertheless, the standard deviation of kMC is very different for each image and band, and the large values exceed 3000. This is compared to the standard deviation of IIkMC, which does not exceed the early 2000 s. Therefore, it can be confirmed the result of IIkMC is more stable than that of kMC, even from the view of separability. Moreover, since it is challenging to identify a set of representative class signatures from KMC (i.e., widely distributed class centers and standard deviation among images), IIkMC is considered a better solution.

3.3. Evaluation of Parallel Processing Performance

Parallel processing performance is typically evaluated using two indices, efficiency and speedup, both of which are defined in terms of the processing time versus the number of cores or computing nodes involved (Equation (2)). In the ideal case, speedup is the same as the number of cores or computing nodes, and efficiency should be 1.0. This indicates the increase in execution speed is exactly proportional to the number of cores or computing nodes involved. However, according to Amdahl’s law [41], this is not achievable in reality. Reasons for this include unequal workload among cores and nodes, as well as unparallelizable procedures. In addition, in some cases, the efficiency might exceed 1.0; this is called super-linearity. This can occur when processing an exceptional amount of data or when the complexity is beyond the ability of a single core or node [42]. Speedup $S_p$ and efficiency $E_p$ are defined as:

$$S_p=\frac{T_s}{T_p},E_p=\frac{T_s}{{p·T}_p}$$

(2)

where p is the number of cores (or computing nodes) and $T_s$ and $T_p$ denote the execution times for sequential and parallel processing, respectively.

The execution times were measured for the sequential code, the intra-node code, the GPU-based code, and the GPU-based code with parallel reduction for IIkMC. To calculate the speedup and the efficiency of the intra-node code, $T_s$ was set as the execution time of the sequential code, and $T_p$ was set as that of the intra-node code using all cores. However, since it is ambiguous to define p for the GPU-based code, the efficiency is not presented (Table 5).

Table 5. Execution time, speedup, and efficiency of parallel IIkMC.

Code	# of Cores	Execution Time	Speedup	Efficiency
Sequential	1	5765.18 s	-	-
Intra-node	16	449.27 s	12.83	0.80
GPU	-	225.83 s	25.53	-
GPU-reduction	-	174.37 s	39.00	-

The results are listed in Table 5. Processing with sequential code was very burdensome and required 5765.18 s (=1.6 h) to process 12 images (=2.86 GB). On the contrary, the intra-node code took 449.27 s (=7.5 min). This is about 13 times faster, meaning the task can be finished within an acceptable time. The GPU-based code processed the data 25.53 times faster than sequential code and 2.00 times faster than intra-node code. Furthermore, the GPU-based code with parallel reduction processed 1.53 times faster than the GPU-based code. In this study, sequential addressing, a middle-optimized parallel reduction approach, was applied. With more optimized reduction approaches, the processing speed will be further improved. Additionally, it can be predicted the speed can be maximized by using multi-GPUs.

Finally, this study verified whether the results obtained using the sequential, intra-node, and GPU-based codes with and without parallel reduction, matched. The class centers of each band and the classified number of pixels of the results were compared. No discrepancy was detected among the results, confirming they matched completely.

3.4. Discussion

Applying kMC and IIkMC to 12 planetscope images revealed quite different results. In the results of kMC, using 12 consecutively acquired images that might constitute similar land cover, the number of iterations was significantly different, some images were not clustered appropriately, and some classes were canceled during iteration. Class signatures of kMC proved challenging because choosing a representative one among images would take a lot of work. On the other hand, IIkMC conducted stable clustering while assigning consistent classes to all the images. Although the standard deviations were estimated to be slightly larger, the results were expected to provide a comprehensive result by delivering a single set of class signatures for all images.

Because of its efficiency, kMC is still widely used for unsupervised classification of satellite images. Nevertheless, as the capacity or quantity of images increases, the processing time of IIkMC increases exponentially. The intra-node parallel processing and GPU-based parallel processing succeeded in increasing the speed of IIkMC several times faster than sequential processing. In addition, the performance is further improved with the introduction of parallel reduction and is expected to be enhanced by optimizing parallel reduction and utilizing multi-GPU. Therefore, it will be possible to process data acquired in a more extensive range of areas, including satellite and drone images ranging from hundreds to thousands.

4. Conclusions

In this study, IIkMC was introduced for the unsupervised classification of satellite images acquired in series. The results can be summarized in three points as follows:

-

derived a single set of class signatures for multiple satellite images;

-

performed complete classification of all images, granting consistent class IDs;

-

implemented intra-node and GPU-base parallel processing.

A single set of class signatures was derived, and each image was consistently classified so the classification results were compatible between the images. In addition, some images were completely processed could not be correctly segmented by kMC. However, IIkMC also revealed the possibility the separability among classes could be weakened because wider spectral features from multiple images were merged into each class. Nevertheless, it should be remarked IIkMC provides more reliable results, potentially facilitating overall inspection of multiple images and training data generation for supervised classification or deep learning.

kMC is a computationally intensive algorithm, making it burdensome to apply to large images, which is also the case for IIkMC. With the parallelized IIkMC using multi-cores of a CPU and a GPU, the processing speed could be dramatically improved. Thus, large volumes of satellite images acquired from an extensive range of areas are expected to be plausibly inspected prior to intensive analysis or supervised classification.

For better performance of IIkMC, future works aim to improve the class center management and enhance class separability. The former includes class center initialization along with class separation and integration based on data characteristics, similar to ISODATA clustering. The latter will be achieved by normalizing the radiometric distributions among images. GPU-based approaches will be improved by implementing further optimized parallel reduction, manipulating high-performance operations introduced in cuBLAS [43], and modifying parallel code to fit using multi-GPUs.