Region Segmentation of Images Based on a Raster-Scan Paradigm

Abstract

This paper introduces a new method for the region segmentation of images. The approach is based on the raster-scan paradigm and builds the segments incrementally. The pixels are processed in the raster-scan order, while the construction of the segments is based on a distance metric in regard to the already segmented pixels in the neighbourhood. The segmentation procedure operates in linear time according to the total number of pixels. The proposed method, named the RSM (raster-scan segmentation method), was tested on selected images from the popular benchmark datasets MS COCO and DIV2K. The experimental results indicate that our method successfully extracts regions with similar pixel values. Furthermore, a comparison with two of the well-known segmentation methods—Watershed and DBSCAN—demonstrates that the proposed approach is superior in regard to efficiency while yielding visually similar results.

1. Introduction

Image segmentation denotes a process of dividing an image into a set of non-overlapping regions [1]. It is one of the most basic image preprocessing methods and is utilised in many applications, including object detection [2,3], medical imaging [4,5,6], remote sensing [7,8], and biometric recognition [9,10,11]. There are numerous methods available for this task, with the main objective of extracting adjacent pixel groups that share similar features and properties [12]. Different approaches can be used for performing image segmentation, which are divided into three groups [13]:

• Region segmentation.
  Region segmentation methods, also referred to as classic segmentation methods, exploit the similarity between pixels to arrange them into regions [13]. The most common region segmentation methods and their principles are considered briefly in Section 2.
• Collaborative segmentation.
  Collaborative segmentation (or co-segmentation) methods deal with the segmentation of similar objects in multiple correlated images [14]. The concept was introduced in [15], and has been used primarily for tracking the objects in videos [16,17,18].
• Semantic segmentation.
  The main goal of semantic segmentation methods is to divide an image into meaningful regions that are assigned to the most suitable predefined category labels [19,20]. Nowadays, the great majority of semantic segmentation methods are based on deep neural networks and other machine learning techniques. The most popular architectures include SegNet [21], ReSeg [22], DeepLab [23], CFNet [24], and HRViT [25]. Despite high segmentation performance [26], the downside of such methods is the demand for huge pre-annotated datasets during the training phase of the models.

The existing segmentation methods have proven their strengths in various tasks of image analysis [13]. However, depending on the technique, the available methods have several disadvantages: inefficiency, troublesome implementation, difficult manual tuning of parameters, and a prolonged training phase (machine learning approaches) [12]. The proposed approach overcomes the issues of other segmentation methods. It is efficient and simple to implement while still yielding comparable results to related region segmentation methods. It is especially suitable for deployment on edge devices [27] (e.g., surveillance cameras) due to its low computational resource demands.

The main contributions of this paper are as follows:

• A new, highly efficient method named the raster-scan segmentation method (RMS) based on a raster scan for the region segmentation of images.
• The design of different distance metrics, which define the similarity of pixels to segments, and actions, which are utilised for the incremental building of segments.
• Extensive experimental work, which demonstrates the efficiency of the RMS, and a comparison with state-of-the-art region segmentation methods (i.e., Watershed and DBSCAN).

The remainder of the paper is structured in the following way. Section 2 highlights the usage of image segmentation in sensor and actuator networks, and summarises the related methods for performing image segmentations. Section 3 describes the proposed method for the segmentation of images using the considered raster-scan approach, the RSM (raster-scan segmentation method). In Section 4, the results of the proposed method are shown and compared with other popular region segmentation methods. Section 5 concludes the paper, summarises the principles of the RSM, and describes future work.

2. Background and Related Work

Today, edge computing is one of the most popular paradigms in Computer Science [28]. The main characteristic of edge computing is that the majority of computations are performed on edge devices with low computational resources [29]. Sensor and actuator networks frequently include edge devices that gather image data. The segmentation methods should be efficient in terms of time and memory consumption in order to enable fast enough performance of various operations on images (tracking the objects in the scene). Image segmentation methods with low computational complexity have been used successfully on different edge devices in sensor and actuator networks, including remote sensing data [8,30,31], handwritten text [20], medical scans [32,33], and footage from surveillance cameras [34,35]. In robotics, image segmentation is often used for the recognition and analysis of objects within the robot’s surroundings [36,37,38].

The proposed raster-scan segmentation method belongs to the region segmentation methods, as its main goal is to split an image into regions with similar pixel values. In Table 1, the most popular related region segmentation methods and their main advantages and disadvantages are summarised. In the continuation of this section, a brief description of each listed segmentation method is provided.

Table 1. Related region segmentation methods.

Method	Advantages	Disadvantages
Differential operators [39]	Low computational complexity	Poor resistance to noise, bad results for images with small pixel gradients
Hough transform [40]	High-accuracy segmentation of regular polygons	Poor segmentation of concave and wavy structures, high computational complexity
Active contour approach [41,42]	Accurate segmentation of concave and wavy polygons	Difficult tuning of parameters
Thresholding [43,44]	Low computational complexity, robust adaptive approaches	Difficult choice of threshold value, possibility of over-segmentation
Watershed [45]	Low computational complexity, efficient marker-controlled approach	Tendency toward over-segmentation, difficult determination of markers’ positions
k-means [46,47,48]	Simplicity, low computational complexity	Difficult tuning of parameters
DBSCAN [49,50,51]	Accurate segmentation of non-linearly separable image clusters	Sensitivity to input parameters
Superpixel methods [52,53,54]	Low computational complexity	Inability to process regions with high-intensity varieties

2.1. Edge Detection

Edge detectors are the earliest and most basic methods for region segmentation. Their main goal is to detect pixels with high gradients that represent boundaries of regions [13]. There are several approaches to this task, among which the least complex are differential operators. Their main advantage is low computational complexity. However, their main disadvantage is that they are very prone to producing visually peculiar results if there is even a small amount of noise present in an image [55]. A lot of differential operators have been proposed throughout the years, among which the most popular are Roberts, Kirsch, Deriche, Prewitt, Laplacian, Sobel, and Canny operators [39]. Their time complexity is $O(nlogn)$, where n signifies the total number of pixels in an image.

Hough transform is a popular method for the detection of contours in an image. It was proposed by Duda and Hart [56], and has been adopted for many computer vision tasks, including image segmentation [40,57]. The main advantage of Hough transform is its high accuracy for the detection of regular polygons. On the other hand, it may not produce good segmentation results for concave and wavy structures in an image. Furthermore, Hough transform can be significantly demanding in terms of time complexity when dealing with such structures. In its basic form for detecting lines, the time complexity of Hough transformation is $O\left(n^4\right)$ [58].

An iterative active contour approach (also referred to as snakes) was proposed by Kass et al. [59]. It is used for extracting regions from an image by fitting simple polylines to region edges. The latter is performed by energy minimisation [59]: the image forces pull the contour to important image features (i.e., edges, vertices, and lines) while obeying the constraints (either user-given or determined computationally). This method has been proven to yield good results in some cases [41,42]. However, there are two major disadvantages of the active contours method: sensitivity to the initial positions of control points, and difficult initialisation of their input parameters [60]. The time complexity of the basic approach is $O\left(ic{m}^2\right)$, where i signifies the number of iterations, c the selected number of control points, and m the number of scales (the contour is fitted to image regions in several resolutions).

Other image segmentation methods based on edge detection have been proposed in the past, such as graph cut optimisations [61,62]. Although they can produce high-accuracy segmentation results, they are only able to divide an image into two regions (foreground and background).

2.2. Region Division

Region division segmentation methods are based either on the serial or the parallel region division [13]. Serial region division splits an image into regions sequentially, while regions are formed simultaneously when parallel region division is applied.

A typical parallel region division technique is thresholding. In the first step, a threshold value is determined either manually or heuristically. After that, each pixel is assigned a binary value [63], based on whether the pixel’s value is above or below the threshold. Consequently, the obtained binary image consists of two segments. The described approach has low computational complexity. Division into two regions, however, is not particularly useful for images with many details. Therefore, thresholding with multiple thresholds [64] and adaptive local thresholding [43,44] were proposed in order to increase the usability of this segmentation method in practice. The time complexity of the described method is $O\left(n\right)$.

Watershed techniques utilise the geology concept of drainage divides for data segmentation [65,66]. A similar approach, based upon serial region division, is used for region segmentation [67,68]. A greyscale image is interpreted as a 3D topographic relief, which is split into valleys, ridges, and slopes [69]. The main advantage of the Watershed is low time complexity, as it operates in $O\left(n\right)$. However, it tends to over-segmentise an image. In order to solve this issue, a marker-controlled Watershed was proposed [45] where significant parts of the image are marked either by a user or using a heuristic.

2.3. Clustering

Clustering is a popular technique for organising data by grouping near data samples [70]. It is also used in different fields of image analysis [71,72,73]. One of the earliest and most well-known clustering algorithms is k-means [74]. Its main advantage is low computational complexity. On the other hand, major challenges arise concerning the choice of the parameter k that would yield the best segmentation, and the initialisation of the centroid pixels’ locations. Therefore, several analyses of an image are usually performed before the segmentation procedure [46,47,48,75,76,77]. The time complexity of k-means is $O\left(knt\right)$, where t specifies the number of iterations, while k signifies the number of clusters.

Density-based spatial clustering of applications with noise (DBSCAN) was proposed by Ester et al. [78]. Compared to k-means, its major advantage is its ability to process non-linearly separable image structures. DBSCAN has been used successfully for image segmentations [49,50,51]. Its main disadvantage is having to precisely tune the input parameters in order to obtain the expected results. In the worst case, DBSCAN has a time complexity of $O\left(n^2\right)$, although, with the proper data structures, its time complexity is reduced to $O(nlogn)$ [79].

Superpixel segmentation is a clustering method that splits a weighted graph into subgraphs with low coupling and high cohesion [13]. Such an approach has been used in many image segmentation applications [52,53,54]. Despite the low time complexity using the proper implementation [54], there are several drawbacks of different superpixel segmentation methods: no explicit control over the superpixels’ numbers and compactness [52], and failing to correctly segmentise image regions that have high-intensity variety [80]. The time complexity of the most efficient superpixel segmentation methods is $O\left(n\right)$ [54].

3. Raster-Scan Segmentation Method

The proposed method is based on a raster-scan paradigm that derives from the era of analogue television [81] based on a cathode-ray tube (CRT). The CRT directs a beam of electrons to the phosphorescent coating on the screen. As a result, a spot of light is produced at a place where the electron beam hits the screen coating. In order to display the picture on the screen, the CRT directs the beam from the left to the right, and, after the whole line is drawn, the procedure is repeated in the next row. This drawing order is known as the scan-line order [82], while the covering of the entire screen in this way is referred to as a raster scan [81].

Let I be a raster image with X columns and Y rows. Consequently, the number of pixels in I is $n=X·Y$. With no loss of generality, I is considered to be greyscale with b bit-planes, yielding $2^b$ shades of grey. The goal of region image segmentation is to divide I into a set of segments $\mathcal{S}=\left\{S_i\right\},1\le i\le \left|\mathcal{S}\right|$, so that $\left|\mathcal{S}\right|\le n$. However, it is expected that $\left|\mathcal{S}\right|\ll n$.

The segmentation of I is performed in the scan-line order, which is why the method is referred to as the raster-scan segmentation method (RSM) in the continuation. The block diagram that represents the sequence of steps in the RSM is displayed in Figure 1.

Figure 1. Block diagram of RSM.

In the beginning, the segment set $\mathcal{S}$ is initialised as an empty set (block A in Figure 1). Then, the RSM iterates through I in a scan-line order (see Figure 2). After a pixel $p_{x,y},1\le x\le X,1\le y\le Y$ is reached, segments of neighbouring pixels are obtained (block B). The distances $\delta$ are calculated with the selected distance metric d (block C), based on which an action A is determined (block D). According to the local neighbourhood, $p_{x,y}$ is assigned to the segment $S_i$ (already existing or newly created). The partial result of the segmentation is stored in a set of edge-connected segments $\mathcal{S}$ (block E). After all pixels in I are processed, $\mathcal{S}$ contains the final result of the segmentation (block F). The pseudocode at the end of this section gives the details of the algorithm.

Figure 2. The processing order of the RSM in I. Already processed pixels are marked with green circles, the current pixel is marked with a yellow circle, and non-processed pixels are marked with red circles.

Edge-connected segments are created and built incrementally during the segmentation procedure with the RSM. Each pixel is assigned to a segment $S_i$. The selection of $S_i$ depends on the already processed pixels, and is based on the selected distance metric d. The following three distance metrics d were used for the calculation of the distance $\delta$:

• One-Neighbour Metric (${\mathsf{\Gamma }}_{ON})$.
  This metric is based on the differences between the current pixel value $p_{x,y}$ and the pixel values of its left and top neighbours. The differences ${\delta }_{left}$ and ${\delta }_{up}$ are calculated according to Equation (1).

  $$\begin{array}{c}\hfill {\delta }_{left}=\left|p_{x,y}-p_{x-1,y}\right|\hfill \\ \hfill {\delta }_{up}=\left|p_{x,y}-p_{x,y-1}\right|\hfill \end{array}$$

  (1)
• Two-Neighbour Metric (${\mathsf{\Gamma }}_{TN})$.
  This metric considers the differences between the current pixel value $p_{x,y}$ and the average value of two pixels to the left and at the top. The differences ${\delta }_{left}$ and ${\delta }_{up}$ are calculated according to Equation (2).

  $$\begin{array}{c}\hfill {\delta }_{left}=\left|p_{x,y}-{\displaystyle \frac{p_{x-1,y}+p_{x-2,y}}{2}}\right|\hfill \\ \hfill {\delta }_{up}=\left|p_{x,y}-{\displaystyle \frac{p_{x,y-1}+p_{x,y-2}}{2}}\right|\hfill \end{array}$$

  (2)
• Adjacent Segments’ Average Metric (${\mathsf{\Gamma }}_{ASA})$.
  Unlike the other two, this metric assesses larger areas in I instead of only separate adjacent pixels’ values. Let $p_{x-1,y}$ belong to the segment $S_{left}$ and let $p_{x,y-1}$ belong to $S_{up}$. If $\overline{S_{left}}$ and $\overline{S_{up}}$ represent the average pixel values of the segments $S_{left}$ and $S_{up}$, the differences ${\delta }_{left}$ and ${\delta }_{up}$ are calculated according to Equation (3).

  $$\begin{array}{c}\hfill {\delta }_{left}=\left|p_{x,y}-\overline{S_{left}}\right|\hfill \\ \hfill {\delta }_{up}=\left|p_{x,y}-\overline{S_{up}}\right|\hfill \end{array}$$

  (3)

In order to control the segmentation process, the user needs to provide two parameters. The first parameter is the distance threshold value $ϵ$, below which $p_{x,y}$ can be added to $S_i$. The second parameter is the maximum deviation from a segment’s average value, ${\mathsf{\Delta }}_{max}$. There are five possible actions A when assigning $p_{x,y}$ to $S_i$:

• Merge.
  If ${\delta }_{left}\le ϵ$, ${\delta }_{left}\le {\mathsf{\Delta }}_{max}$, ${\delta }_{up}\le ϵ$, ${\delta }_{up}\le {\mathsf{\Delta }}_{max}$, and $i_{left}\ne i_{up}$, segments of $p_{x-1,y}$ and $p_{x,y-1}$ are merged into a single segment $S_i$. After that, $S_i$ is expanded with the current pixel $p_{x,y}$. The latter is performed using a union: $S_i\cup \left\{p_{x,y}\right\}$ (Figure 3).
  

  Figure 3. The action Merge: before (a) and after (b) the processing of pixel $p_{x,y}$, marked by a red square. The green arrows indicate that $\delta \le ϵ$.
• Add.
  If ${\delta }_{left}\le ϵ$, ${\delta }_{left}\le {\mathsf{\Delta }}_{max}$, ${\delta }_{up}\le ϵ$, ${\delta }_{up}\le {\mathsf{\Delta }}_{max}$, $i_{left}=i_{up}$, and $p_{x,y-1}\in S_{up}$, the segment $S_{up}$ is expanded as follows: $S_{up}\cup \left\{p_{x,y}\right\}$ (Figure 4).
  

  Figure 4. The action Add: before (a) and after (b) the processing of pixel $p_{x,y}$, marked by a red square. The green arrows indicate that $\delta \le ϵ$.
• Add-Left.
  If ${\delta }_{left}\le ϵ$, ${\delta }_{left}\le {\mathsf{\Delta }}_{max}$, ${\delta }_{up}>ϵ$, and $p_{x-1,y}\in S_{left}$, the segment $S_{left}$ is expanded as follows: $S_{left}\cup \left\{p_{x,y}\right\}$ (Figure 5).
  

  Figure 5. The action Add-Left: before (a) and after (b) the processing of pixel $p_{x,y}$, marked by a red square. The green arrow indicates that $\delta \le ϵ$ while the red arrow indicates that $\delta >ϵ$.
• Add-Up.
  If ${\delta }_{left}>ϵ$, ${\delta }_{up}\le ϵ$, ${\delta }_{up}\le {\mathsf{\Delta }}_{max}$, and $p_{x,y-1}\in S_{up}$, the segment $S_{up}$ is expanded as follows: $S_{up}\cup \left\{p_{x,y}\right\}$ (Figure 6).
  

  Figure 6. The action Add-Up: before (a) and after (b) the processing of pixel $p_{x,y}$, marked by a red square. The green arrow indicates that $\delta \le ϵ$ while the red arrow indicates that $\delta >ϵ$.
• New.
  In all other cases, a new segment $S_i=\left\{p_{x,y}\right\}$ is added to $\mathcal{S}$ (Figure 7).
  

  Figure 7. The action New: before (a) and after (b) the processing of pixel $p_{x,y}$, marked by a red square. The red arrows indicate that $\delta >ϵ$.

The pseudocode of the RSM is presented in Algorithm 1. In the initialisation phase, an empty set of segments $\mathcal{S}$ is created. In order to enable direct access to the already assigned pixels’ segments, a matrix M with indices of segments is used, and is initialised to −1 in the beginning. The loops in Lines 11 and 12 iterate through I. After a pixel $p_{x,y}$ is reached, the index of the left segment $i_{left}$ and the index of the upper segment $i_{up}$ are obtained from M in Lines 13 and 14. Segments $S_{left}$ and $S_{up}$ are obtained from M in Lines 15 and 16. ${\delta }_{left}$ and ${\delta }_{up}$ are calculated in Lines 17 and 18 using the provided d (according to Equations (1), (2), or (3)). In Line 20, one of the five actions A is selected, based on ${\delta }_{left}$, ${\delta }_{up}$, $ϵ$, ${\mathsf{\Delta }}_{max}$, $i_{left}$, and $i_{up}$. If the action Merge is chosen, the segments of the left and the upper pixels are merged into a new segment, to which the current pixel $p_{x,y}$ is added in Line 24. The newly created $S_i$ is added to $\mathcal{S}$ in Line 25, while the matrix M is updated in Line 26. In the case of the action Add, $p_{x,y}$ is added to the common $S_i$ of the left and top pixels in Line 28. After that, $\mathcal{S}$ and M are updated. The actions Add-Left and Add-Top add $p_{x,y}$ to the left or the top $S_i$ in Line 32 and Line 36, respectively. In both cases, $\mathcal{S}$ and M are then updated accordingly. In the case of the action New, a new $S_i$ is created, which is added to $\mathcal{S}$ in Line 41. After that, in Line 42, M is also updated. The final result of the algorithm is a set of segments $\mathcal{S}$.

Algorithm 1 Region image segmentation with the RSM

1: function Raster-Scan-Segmentation-Method(I, X, Y, d, $ϵ$, ${\mathsf{\Delta }}_{max}$)  2:                          ▹I: an image with the resolution $X\times Y$  3:                         ▹d, $ϵ$: distance metric, distance threshold  4:                ▹${\mathsf{\Delta }}_{max}$: maximum deviation from the segment’s average value  5:                          ▹ The function returns a set of segments  6:       7:     $\mathcal{S}\leftarrow \left\{\right\}$                        ▹ Initial set of segments is empty  8:     $M\leftarrow \mathrm{InitialiseMatrix}(X,Y,-1)$        ▹ Matrix with indices of segments for pixels  9:     $segmentCounter\leftarrow 0$          ▹ Counter of segments is set to 0 in the beginning 10:      11:     for $y\leftarrow 1\mathrm{to}Y$ do 12:         for $x\leftarrow 1\mathrm{to}X$ do 13:            $i_{left}\leftarrow$$M_{x-1,y}$                   ▹ Segment index of the left pixel 14:            $i_{up}\leftarrow$$M_{x,y-1}$                  ▹ Segment index of the upper pixel 15:            $S_{left}\leftarrow \mathrm{FindSegment}(M,x-1,y)$ 16:            $S_{up}\leftarrow \mathrm{FindSegment}(M,x,y-1)$ 17:            ${\delta }_{left}\leftarrow \mathrm{CalculateDistance}(d,p_{x,y},p_{x-1,y},p_{x-2,y},S_{left})$ ▹ Equation (1), (2), or (3) 18:            ${\delta }_{up}\leftarrow \mathrm{CalculateDistance}(d,p_{x,y},p_{x,y-1},p_{x,y-2},S_{up})$▹ Equation (1), (2), or (3) 19:             20:            $A\leftarrow \mathrm{DetermineAction}({\delta }_{left},{\delta }_{up},ϵ,{\mathsf{\Delta }}_{max},i_{left},i_{up})$ 21:            switch A do 22:                case Merge: 23:                    $segmentCounter\leftarrow segmentCounter+1$ 24:                    $S_{segmentCounter}\leftarrow$ MergeSegments($i_{left}$, $i_{up}$) $\cup \left\{p_{x,y}\right\}$ 25:                    AddSegment($\mathcal{S}$, $S_{segmentCounter}$) 26:                    $M_{x,y}\leftarrow segmentCounter$ 27:                case Add: 28:                    $S_{up}\leftarrow S_{up}\cup \left\{p_{x,y}\right\}$ 29:                    UpdateSegment($\mathcal{S}$, $S_{up}$, $i_{up}$) 30:                    $M_{x,y}\leftarrow i_{up}$ 31:                case Add-Left: 32:                    $S_{left}\leftarrow S_{left}\cup \left\{p_{x,y}\right\}$ 33:                    UpdateSegment($\mathcal{S}$, $S_{left}$, $i_{left}$) 34:                    $M_{x,y}\leftarrow i_{left}$ 35:                case Add-Up: 36:                    $S_{up}\leftarrow S_{up}\cup \left\{p_{x,y}\right\}$ 37:                    UpdateSegment($\mathcal{S}$, $S_{up}$, $i_{up}$) 38:                    $M_{x,y}\leftarrow i_{up}$ 39:                case New: 40:                    $segmentCounter\leftarrow segmentCounter+1$ 41:                    AddSegment($\mathcal{S}$, $\left\{p_{x,y}\right\}$) 42:                    $M_{x,y}\leftarrow segmentCounter$ 43:            end switch 44:         end for 45:     end for 46:     return $\mathcal{S}$ 47: end function

4. Results

The results of the RSM are presented and discussed in this section. For testing purposes, the popular image datasets MS COCO [83] and DIV2K [84] were used. MS COCO is a well-known image dataset that is often used for the benchmarking of segmentation algorithms [85,86]. It consists of more than 330,000 images. Meanwhile, DIV2K is a considerably smaller dataset as it consists of 1000 images. Primarily, DIV2K is used as a benchmark dataset for super-resolution tasks [87]. In our case, we used both datasets to diversify the test image samples. The performance of the RSM is demonstrated on 10 greyscale images that were selected randomly from MS COCO and DIV2K. The high-resolution images belong to DIV2K, while the images with lower resolutions belong to MS COCO. The names of the selected images, the datasets that they belong to, their resolutions, and the total numbers of pixels n are shown in Table 2.

Table 2. Names, source datasets, resolutions, and total number of pixels of the selected test images I.

$\mathit{I}$	Dataset	Resolution	$\mathit{n}$
Airplane	MS COCO	$640\times 406$	259,840
Baseball	MS COCO	$640\times 427$	273,280
Canal	DIV2K	$2040\times 1524$	3,108,960
Fruits	DIV2K	$2040\times 1356$	2,766,240
Seashore	DIV2K	$2040\times 1356$	2,766,240
Stop Sign	MS COCO	$640\times 480$	307,200
Sunflowers	DIV2K	$2040\times 1356$	2,766,240
Taxi	MS COCO	$640\times 480$	307,200
Tennis	MS COCO	$640\times 427$	273,280
Train	DIV2K	$2040\times 1356$	2,766,240

In Figure 8, the results of image segmentation with the RSM are displayed. Different distance metrics d, described in Section 3, were used. For visualisation purposes, the segments are coloured with randomly selected colours, with the limitation that adjacent segments do not have the same colour. Such colouring enables distinguishing between adjacent segments.

Figure 8. Image segmentation with the RSM using different distance metrics d: (a) input image, (b) ${\mathsf{\Gamma }}_{ON}$, (c) ${\mathsf{\Gamma }}_{TN}$, (d) ${\mathsf{\Gamma }}_{ASA}$. Each $S_i$ is coloured with a random colour.

The numbers of segments $\left|\mathcal{S}\right|$ for each I using different d values are shown in Table 3. During the segmentation, a suitable action A (Merge, Add, Add-Left, Add-Up, or New) is selected according to the local neighbourhood of the current pixel $p_{x,y}$. The ${\mathsf{\Gamma }}_{TN}$ distance metric produces higher $\left|\mathcal{S}\right|$ than ${\mathsf{\Gamma }}_{ON}$ and ${\mathsf{\Gamma }}_{ASA}$ for all test images. On the other hand, $\left|\mathcal{S}\right|$ is consistently the lowest in all test cases if performing segmentation using ${\mathsf{\Gamma }}_{ON}$. In Figure 8b, it can be seen that the ${\mathsf{\Gamma }}_{ON}$ distance metric produces fewer segments on the ground than ${\mathsf{\Gamma }}_{TN}$ and ${\mathsf{\Gamma }}_{ASA}$. Meanwhile, ${\mathsf{\Gamma }}_{ASA}$ is able to extract segments that do not contain pixels with large gradients (e.g., segments that represent the sky). Therefore, the most appropriate distance metric d depends on the properties of I. If I contains many edges that are not clearly defined, it is sensible to use ${\mathsf{\Gamma }}_{ASA}$. On the other hand, if the pixels on the edges in I have large gradients, ${\mathsf{\Gamma }}_{ON}$ or ${\mathsf{\Gamma }}_{TN}$ can yield better results.

Table 3. Number of segments $\left|\mathcal{S}\right|$ for test images according to d.

$\mathit{I}$	${\mathsf{\Gamma }}_{\mathit{O}\mathit{N}}$	${\mathsf{\Gamma }}_{\mathit{T}\mathit{N}}$	${\mathsf{\Gamma }}_{\mathit{A}\mathit{S}\mathit{A}}$
Airplane	23,761	39,949	33,809
Baseball	10,505	16,520	12,659
Canal	215,024	349,251	282,342
Fruits	416,740	698,121	538,066
Seashore	84,695	160,417	118,450
Stop Sign	60,134	74,841	65,451
Sunflowers	349,382	588,149	457,998
Taxi	36,891	55,769	42,701
Tennis	17,703	28,048	25,093
Train	160,412	249,109	190,581

Figure 9 shows region segmentations of the image Train with different values of $ϵ$ (limited to the range that produces sensible segmentation results). The ${\mathsf{\Gamma }}_{ON}$ distance metric was used for the segmentation. Table 4 shows the number of segments $\left|\mathcal{S}\right|$ for the test images according to different values of $ϵ$. Smaller values of $ϵ$ cause the segmentation to yield higher $\left|\mathcal{S}\right|$. Furthermore, by selecting too small a value of $ϵ$, the RSM tends to over-segment the image (as seen in Figure 9b). Therefore, it is sensible not to choose $ϵ$ values that are close to 0. On the other hand, an $ϵ$ that is too large can lead to a failure in extracting segments with soft edges (e.g., in Figure 9d, where the ground and the bottom part of the train are merged into a single $S_i$).

Figure 9. Image segmentation with the RSM according to different values of $ϵ$: (a) input image, (b) $ϵ=2$, (c) $ϵ=4$, (d) $ϵ=6$. Each $S_i$ is coloured with a random colour.

Table 4. Number of segments $\left|\mathcal{S}\right|$ for test images according to the parameter $ϵ$.

$\mathit{I}$	$\mathit{ϵ}=2$	$\mathit{ϵ}=3$	$\mathit{ϵ}=4$	$\mathit{ϵ}=5$	$\mathit{ϵ}=6$
Airplane	39,469	23,761	16,017	11,789	9245
Baseball	25,948	17,356	13,076	10,505	8669
Canal	443,081	300,451	215,024	161,928	126,380
Fruits	624,147	416,740	294,988	216,854	163,314
Seashore	275,453	180,119	122,328	84,695	59,557
Stop Sign	107,356	87,775	72,513	60,134	50,191
Sunflowers	349,382	224,471	155,674	112,928	84,441
Taxi	86,543	61,972	46,599	36,891	30,081
Tennis	31,942	17,703	12,234	9405	7501
Train	355,059	258,854	200,255	160,412	131,059

Figure 10 displays a comparison between the segmentation results with different values of ${\mathsf{\Delta }}_{max}$. The numbers of segments $\left|\mathcal{S}\right|$ for the test images are shown in Table 5. Effectively, small ${\mathsf{\Delta }}_{max}$ impacts the segmentation, so that regions with small pixel gradients and large differences in pixel values are split into several segments. Consequently, it can be observed that by increasing ${\mathsf{\Delta }}_{max}$, the number of segments $\left|\mathcal{S}\right|$ decreases. The decrease in $\left|\mathcal{S}\right|$ is especially noticeable when dealing with smaller values of ${\mathsf{\Delta }}_{max}$.

Figure 10. Image segmentation with RSM according to different values of ${\mathsf{\Delta }}_{max}$: (a) input image, (b) ${\mathsf{\Delta }}_{max}=5$, (c) ${\mathsf{\Delta }}_{max}=20$, (d) ${\mathsf{\Delta }}_{max}=50$. Each $S_i$ is coloured with a random colour.

Table 5. Number of segments $\left|\mathcal{S}\right|$ for test images according to parameter ${\mathsf{\Delta }}_{max}$.

$\mathit{I}$	${\mathsf{\Delta }}_{\mathit{m}\mathit{a}\mathit{x}}=5$	${\mathsf{\Delta }}_{\mathit{m}\mathit{a}\mathit{x}}=10$	${\mathsf{\Delta }}_{\mathit{m}\mathit{a}\mathit{x}}=20$	${\mathsf{\Delta }}_{\mathit{m}\mathit{a}\mathit{x}}=50$	${\mathsf{\Delta }}_{\mathit{m}\mathit{a}\mathit{x}}=100$
Airplane	27,237	24,581	23,920	23,761	23,747
Baseball	13,126	11,032	10,505	10,360	10,344
Canal	261,231	225,578	216,978	215,024	214,898
Fruits	464,036	429,965	420,911	417,494	416,740
Seashore	124,666	93,531	86,474	84,695	84,444
Stop Sign	67,569	61,315	60,265	60,135	60,134
Sunflowers	366,326	354,651	350,808	349,382	348,971
Taxi	44,238	36,891	35,326	34,930	34,906
Tennis	20,349	18,458	17,754	17,708	17,703
Train	197,737	167,779	160,412	158,535	158,417

Examples of region image segmentations of the test images Baseball, Stop Sign, and Taxi are displayed in Figure 11, Figure 12, and Figure 13, respectively. After the segmentation is performed, it can be seen that regions with low variance in pixel values form common segments while the important image details, such as edges and vertices, are preserved. Following this, the parameters d, $ϵ$, and ${\mathsf{\Delta }}_{max}$ were tuned manually, so that the segmentation yielded the best visual results. Although the majority of the regions were detected as expected, we can observe that some parts of the image are over-segmented (especially noticeable in Figure 12 below the Stop Sign).

Figure 11. Region segmentation of Baseball: (a) input image, (b) segmented image.

Figure 12. Region segmentation of Stop Sign: (a) input image, (b) segmented image.

Figure 13. Region segmentation of Taxi: (a) input image, (b) segmented image.

In Figure 14, different region segmentation methods are compared on the test image Canal. Besides the RSM, we used the marker-controlled Watershed method from the library OpenCV [88] and DBSCAN from a popular C++ library mlpack [89]. In Table 6, the numbers of segments $\left|\mathcal{S}\right|$ are summarised for the test image Canal. Despite the fact that Watershed yielded the fewest segments among the three methods, I was over-segmented in some areas (e.g., the water surface). On the other hand, the result of DBSCAN was considerably better, as larger areas are grouped to form segments. The RSM produces visually similar results to DBSCAN.

Figure 14. Comparison of different region segmentation methods: (a) input image, (b) Watershed, (c) DBSCAN, (d) RSM.

Table 6. Number of segments $\left|\mathcal{S}\right|$ using different segmentation methods.

$\mathit{I}$	Watershed	DBSCAN	RSM
Airplane	28,409	48,542	23,761
Baseball	23,278	21,298	10,505
Canal	197,729	498,016	215,024
Fruits	243,714	583,656	416,740
Seashore	126,108	394,168	84,695
Stop Sign	23,324	113,411	60,134
Sunflowers	157,878	576,622	349,382
Taxi	27,862	96,811	36,891
Tennis	24,391	42,800	17,703
Train	153,704	538,013	160,412

In terms of theoretical time complexity, with n denoting the total number of pixels, the worst case for Watershed segmentation is $O(nlogn)$. On average, though, the algorithm performs in linear time $O\left(n\right)$ [90]. DBSCAN’s worst time complexity using proper implementation is $O(nlogn)$. The RSM processes each pixel only once, which is performed in linear time $O\left(n\right)$. Each classified pixel is assigned to a certain segment whose index is stored in a 2D matrix. Consequently, a query for a segment during the segmentation procedure is performed in $O\left(1\right)$. The operation of merging can be implemented with the union-find data structure, which has the amortised time complexity of $O\left(1\right)$ [91]. Based on this, it can be concluded that the time complexity of the RSM is $O\left(n\right)$.

The average elapsed CPU times of region segmentation on the test images with Watershed, DBSCAN, and the RSM are shown in Table 7. The experiments were conducted on a personal computer with an Intel Core i9-12900K CPU, 64 GB of DDR5 RAM, and Windows 11 OS. Watershed’s performance turned out to be almost on a par with the RSM, while DBSCAN was considerably slower for images with large numbers of pixels. Despite the good performance of Watershed, the RSM was faster in all the test cases.

Table 7. Time measurements of different region segmentation methods on test images. The measurements that are bold indicate the method that performed the fastest for a given test image.

$\mathit{I}$	Watershed [s]	DBSCAN [s]	RSM [s]
Airplane	0.011	0.743	0.006
Baseball	0.011	0.916	0.005
Canal	0.098	17.163	0.069
Fruits	0.136	10.462	0.083
Seashore	0.076	16.246	0.053
Stop Sign	0.011	0.818	0.010
Sunflowers	0.103	13.694	0.075
Taxi	0.012	0.837	0.010
Tennis	0.011	0.879	0.008
Train	0.080	13.883	0.053

5. Conclusions

This paper introduces a new method, the raster-scan segmentation method (RSM), for the region segmentation of images based on a raster scan. During the segmentation in scan-line order, image segments are built incrementally according to the similarity in the pixels’ local neighbourhoods. Different distance metrics are used to estimate the similarity between pixels. The RSM was extensively tested on images from the public image datasets MS COCO and DIV2K. Five images were selected randomly from each dataset, and were used to demonstrate the results of the proposed method. The RSM was compared with the widely used region segmentation methods Watershed and DBSCAN. Extensive experimental work revealed that the RSM brings several advantages. Besides its ability to extract segments with pixels that have small gradients, the RSM demonstrated comparable results to other state-of-the art region segmentation methods while being faster in terms of elapsed CPU time.

Region image segmentation could be used as a preprocessing step in different applications of image processing. The RSM could be applied on real-world datasets, or could be used as a real-time segmentation method in sensor and actuator networks due to its low computational complexity. Another possible use case would be remote sensing. Our method could be used as an alternative to other segmentation methods in order to extract different landscape elements from satellite images. Lastly, a possible domain where the RSM could be applied is object-based image compression, where compression could be performed on extracted segments instead of in the pixel space.

There are several ways to implement improvements to the RSM in the future. To start with, the RSM has no mechanisms to deal with noise present in an image. Additional studies on various types and intensities of noise, applied on images, should be conducted. After that, the most suitable noise-removing methods in the preprocessing step should be identified. In addition, currently, the method’s parameters are tuned manually, which can be inefficient in real-life applications. Consequently, an automatic approach for selecting the parameters, based on the extracted image’s characteristics, should be developed.