A Local Thresholding Algorithm for Image Segmentation by Using Gradient Orientation Histogram

Abstract

This paper proposes a new local thresholding method to further explore the relationship between gradients and image patterns. In most studies, the image gradient histogram is simply divided into K bins that have the same intervals in angular space. This kind of empirical approaches may not fully capture the correlation information between pixels. In this paper, a variance-based idea is applied to the gradient orientation histogram. It clusters pixels into subsets with different angular intervals. Analyzing these subsets with similar common patterns respectively will help to assist in achieving the optimal thresholds for image segmentation. For the result assessments, the proposed algorithm is compared with other 1-D and 2-D histogram-based thresholding methods, as well as hybrid local–global thresholding methods. It is shown that the proposed algorithm can effectively recognize the common features of the images that belong to the same category, and maintain the stable performances when the number of thresholds increases. Furthermore, the processing time of the present algorithm is competitive with those of other algorithms, which shows the potential application in real-time scenes.

1. Introduction

Image processing has become an essential component in various applications, with image segmentation serving as a fundamental step, and has been extensively applied in numerous fields [1,2,3,4,5]. Technically, the threshold-based image segmentation has become the most frequently used method due to its simplicity, efficiency, and stability [6]. This approach is a classic segmentation technique that separates an image into foreground and background based on pixel intensity. Although it has been studied for decades and image segmentation today is dominated by deep learning, it remains a critical preprocessing technique in many state-of-art deep learning-based image processing tasks [7,8,9,10,11,12,13]. Thresholding methods for relevant pre-segmentation are widely used in various neural network-based image processing tasks. As exemplified by their use in (a) CT image segmentation to determine the tumor regions [7], (b) dermoscopic image segmentation to determine the infected lesion regions [8], (c) ultrasound image segmentation to segment the presence of follicles [10], (d) Geostationary Satellite (GOES) imagery for wildfire segmentation to calculate the ratio of fire pixels [12], and (e) cell segmentation, they split the microscopic images into segments [13]. In these applications, the quality of segmentation results directly impacts subsequent tasks. Therefore, further research on thresholding methods is essential.

Threshold-based segmentation can generally be categorized into global and local thresholding methods. The global methods apply a single or multiple thresholds to segment the entire image, offering computational efficiency and algorithmic simplicity. In 1985, Kapur [14] proposed the maximum Shannon entropy threshold method. This method selects an optimal threshold based on the image’s one-dimensional (1-D) gray-level histogram, and then segments it into foreground and background regions. Then it was extended to the Tsallis entropy by introducing the non-extensive parameter q [6]. Another typical global thresholding is the Otsu method [15], which determines the optimal threshold by maximizing the between-class variance of the foreground and background. Both approaches are automatic thresholding algorithms that are based on the 1-D gray-level histogram of an image. However, the 1-D histogram only counts the number of pixels that have the same gray-level values without considering the association information between different pixels, such as the spatial distribution information of pixels in the image. If the spatial correlation information between different pixels is nontrivial, the abovementioned global algorithms may not perform optimally [14]. To address the limitations of 1-D histogram, Abutaleb [16] proposed a two-dimensional (2-D) histogram in 1989, which incorporates both the gray values of pixels and their local average gray values. This approach extended Kapur’s method to 2-D histogram, determining the optimal threshold vector by maximizing the sum of the 2-D Shannon entropy of the foreground and background regions, particularly performing well in images with low signal-to-noise ratios (SNRs). Since then, many other 2-D histogram-based thresholding methods have been proposed by utilizing different objective functions [17,18,19,20,21]. However, the enhanced quality comes at the cost of increased computational complexity. The computation time grows exponentially as the number of thresholds increases. Furthermore, traditional 2-D entropy methods tend to recognize the pixels located on the main diagonal of the 2-D histogram, which may result in the loss of edge information [22].

While global thresholding techniques are efficient and straightforward, local techniques offer higher segmentation quality [23,24]. The local thresholding methods first divide the image into several independent regions that satisfy a given homogeneity criterion [25,26], and these regions are segmented by using region-specific thresholds subsequently. These methods incorporate pixel-wise correlation information [27,28,29], including backscatters and texture measures [30], statistical measures like mean intensity and standard deviation [31], and color and distance [32]. The local thresholding methods employ feature-based preprocessing prior to segmentation, and this hierarchical approach effectively reduces the segmentation error, enhances noise resistance, and achieves superior image quality.

Therefore, incorporating additional image feature information can significantly enhance segmentation quality. Among these features, texture information [33], which partially reflects edge characteristics and can be quantified through gradient-based measurements, is widely utilized in image processing applications [34,35,36,37,38,39,40,41,42,43,44,45,46]. Crucially, such edge information of an image captures the outlines, structures, and transitions of objects in the image. It typically corresponds to regions where the gray values change dramatically in the image, and plays an important role in separating different areas or objects. The extraction of texture information would be helpful to keep the predominant patterns of an image. Specifically, for texture-rich images, proper utilization of such information may play a crucial role in segmentation tasks. Therefore, a unified approach that can effectively act on different texture-rich image categories is important to reveal the relationships between texture patterns and pixel gradient clustering.

While grayscale gradient histograms have long been used in image segmentation, this practice is often based on empirical experience, and the implications of their distribution are rarely discussed. This paper analyzes the relationship between grayscale gradient orientation histogram and image patterns from a fundamental point of view, and verifies the reliability of this relationship using a simple method. This approach is expected to further understand the generating of digital images and provide beneficial support to prevailing image analysis algorithms, such as deep learning.

In this paper, we propose a local thresholding method that utilizes a gradient orientation histogram to extract local texture features from images. Different from separating the gradient orientation histogram into K bins [22,40], this method clusters pixels into local subsets by the variance of their orientations. Local thresholding is then applied to each subset. The remainder to this paper is organized as follows: Section 2 briefly introduces the pixel-wise gradient orientation and reviews the 1-D Otsu method. Section 3 presents the construction of the 2-D gradient orientation histogram and the process of the local thresholding method. Section 4 reports on the experimental results from two datasets and discussions. In Section 5, the conclusions are presented.

2. Related Works

2.1. Pixel-Wise Gradient Orientation

The pixel-wise gradient orientation [47,48,49,50] serves as an effective descriptor for local structural features, particularly in texture characterization. One of the ways to generate it is as follows: for a pixel located at position $(x,y)$, its gradient along the vertical and horizontal axes can be respectively defined as

$${\displaystyle \frac{\partial f(x,y)}{\partial x}}=f(x+1,y)-f(x-1,y),$$

(1)

$${\displaystyle \frac{\partial f(x,y)}{\partial y}}=f(x,y+1)-f(x,y-1),$$

(2)

where $f(x,y)$ represents the gray-level value of pixel at $(x,y)$.

Then the orientation of this pixel can be calculated by

$$\theta (x,y)=arctan\left({\displaystyle \frac{\partial f/\partial x}{\partial f/\partial y}}\right),$$

(3)

where the domain of $\theta (x,y)$ is $[-\pi /2,\pi /2]$.

2.2. 1-D Otsu Method

Define the range of gray levels in a given image of size $M\times N$ as $i=0,1,2,\dots ,L-1$, where L represents the maximum gray-level of the image, such as 256. Then the gray-level histogram of the image can be computed by the probability distribution, as follows:

$$p_i={\displaystyle \frac{n_i}{M\times N}},p_i\ge 0,\sum _{i=0}^{L-1}{p}_i=1,$$

(4)

where $n_i$ is the number of pixels that gray-level values are equal to i.

Assuming that the gray-level histogram of the image is divided into two classes, $C_0$ and $C_1$, by a threshold at level t, then the probabilities of class occurrences and the class mean gray-level values are given by

$$w_0=\sum _{i=0}^t{p}_i,w_1=\sum _{i=t+1}^{L-1}{p}_i,$$

(5)

and

$${\mu }_0=\sum _{i=0}^t{\displaystyle \frac{i{p}_i}{w_0}},{\mu }_1=\sum _{i=t+1}^{L-1}{\displaystyle \frac{i{p}_i}{w_1}}.$$

(6)

Thus, the entire image’s mean gray-level value can be obtained as follows:

$${\mu }_T=\sum _{i=0}^{L-1}i{p}_i.$$

(7)

The between-class variance can be represented by

$$\begin{array}{cc}\hfill {\sigma }_B^2& =w_0{({\mu }_0-{\mu }_T)}^2+w_1{({\mu }_1-{\mu }_T)}^2\hfill \\ \hfill & =w_0{w}_1{({\mu }_0-{\mu }_1)}^2\hfill \end{array}.$$

(8)

The optimal threshold can be obtained by maximizing Equation (8), i.e.,

$$t^{*}=\underset{0\le t\le L-1}{argmax}\left({\sigma }_B^2\left(t\right)\right).$$

(9)

3. Proposed Method

In order to enhance the stability, the 1-D thresholding method can be extended to 2-D cases [18,51,52]. However, traditional 2-D thresholding methods may cause the loss of a considerable number of pixels, leading to the loss of nontrivial edge information. Consequently, a novel algorithm is proposed to avoid such cases so that the segmentation quality can be improved. For a texture-rich image, the edge pattern is deeply related to the gradient of pixels’ gray level. Therefore, a 2-D gradient orientation histogram is adopted to cluster the image pixels into distinct regions. Additionally the local thresholding to each region demonstrates the advantages.

3.1. Construction of 2-D Gradient Orientation Histogram

For a pixel located at position $(x,y)$ in a given image of size $M\times N$, its gray-level gradient orientation with respect to the horizontal axis is

$$\theta (x,y)=Round\left[DEG\left(arctan\left({\displaystyle \frac{\frac{\partial f}{\partial y}}{\frac{\partial f}{\partial x}}}\right)\right)\right],$$

(10)

where $DEG\left(\mu \right)$ denotes the transform of the radian angle $\mu$ to degree. ${\displaystyle \frac{\partial f}{\partial x}}$ and ${\displaystyle \frac{\partial f}{\partial y}}$ can be calculated by Equations (1) and (2), and the domain of $\theta (x,y)$ is $[-{90}^{\circ },{90}^{\circ }]$. The pixel’s gray-level value, $f(x,y)$, and the gradient orientation $\theta (x,y)$ can be adopted to construct the 2-D gradient orientation histogram $h(m,n)$ by

$$h(m,n)=\mathrm{Count}\left(f\right(x,y)=m\&\theta (x,y)=n).$$

(11)

The 2-D probability distribution is determined by the normalization of Equation (11), as follows:

$$p(m,n)=\widehat{h}(m,n)={\displaystyle \frac{h(m,n)}{M\times N}}.$$

(12)

Equation (12) represents the frequency of occurrences at gray level m and gradient orientation n, where $m=\{0,1,2,\dots ,L-1\}$ and $n=\{-{90}^{\circ },-{89}^{\circ },\dots ,{89}^{\circ },{90}^{\circ }\}$. Figure 1 shows the gradient orientation information for the image ‘board’.

It is important to note that this study primarily designed the algorithm based on simulating real human visual perception, which has default horizontal and vertical directions. This approach is used to investigate the relationship between pixel gradient clustering and image texture patterns. Therefore, the impact of geometric image transformations on the algorithm was not comprehensively considered. The gradient orientation histogram would cyclically shift with the rotation of the image. Nevertheless, there exists a global optimal clustering by using the Otsu method. It is of interest to further explore this issue for relevant complex scenes in future research.

3.2. Main Step of Segmentation

For a digital image of size $M\times N$, the normalized 2-D gradient orientation histogram $p(m,n)$ can be obtained via Equation (12). Different from traditional 2-D histogram, the distribution of $p(m,n)$ is not concentrated at the main diagonal of the 2-D histogram, as it is shown in Figure 1. The probability distribution of each gradient orientation angle in the image can be obtained by

$$p_k^{ori}=\sum _{i=0}^{L-1}p(i,k),$$

(13)

where k ranges from $[-90,90]$. Additionally, Figure 1d shows the 1-D distribution of gradient orientation angles for the image ‘board’. Using the Otsu method with a set of thresholds $\overrightarrow{t}=(t_1,t_2,\dots ,t_j)$, the distribution of Equation (13) can be segmented into $j+1$ distinct parts, denoted as $\overrightarrow{C}=(C_1,C_2,\dots ,C_{j+1})$, with each part comprising pixels that have similar gradient orientation angles. After normalization, the gray-level probability distributions of $j+1$ classes are respectively defined as

$$\begin{array}{cc}\hfill C_1:& {\displaystyle \frac{p_{-90}^{\mathrm{ori}}}{P_1^{\mathrm{ori}}}},{\displaystyle \frac{p_{-89}^{\mathrm{ori}}}{P_1^{\mathrm{ori}}}},\dots ,{\displaystyle \frac{p_{t_1}^{\mathrm{ori}}}{P_1^{\mathrm{ori}}}},\hfill \\ \hfill C_2:& {\displaystyle \frac{p_{t_1+1}^{\mathrm{ori}}}{P_2^{\mathrm{ori}}}},{\displaystyle \frac{p_{t_1+2}^{\mathrm{ori}}}{P_2^{\mathrm{ori}}}},\dots ,{\displaystyle \frac{p_{t_2}^{\mathrm{ori}}}{P_2^{\mathrm{ori}}}},\hfill \\ & ⋮\hfill \\ \hfill C_{j+1}:& {\displaystyle \frac{p_{t_j+1}^{\mathrm{ori}}}{P_{j+1}^{\mathrm{ori}}}},{\displaystyle \frac{p_{t_j+2}^{\mathrm{ori}}}{P_{j+1}^{\mathrm{ori}}}},\dots ,{\displaystyle \frac{p_{\mathrm{90}}^{\mathrm{ori}}}{P_{j+1}^{\mathrm{ori}}}}\hfill \end{array}$$

(14)

where the cumulative probabilities of $j+1$ classes are defined as

$$\begin{array}{cc}\hfill P_1^{\mathrm{ori}}& =\sum _{k=-90}^{t_1}{p}_k^{\mathrm{ori}},\hfill \\ \hfill P_2^{\mathrm{ori}}& =\sum _{k=t_1+1}^{t_2}{p}_k^{\mathrm{ori}},\hfill \\ & ⋮\hfill \\ \hfill P_{j+1}^{\mathrm{ori}}& =\sum _{k=t_j+1}^{90}{p}_k^{\mathrm{ori}}\hfill \end{array}$$

(15)

The mean gradient orientation angles of $j+1$ classes are as follows:

$$\begin{array}{cc}\hfill {\mu }_1^{\mathrm{ori}}& =\sum _{k=-90}^{t_1}{\displaystyle \frac{k{p}_k^{\mathrm{ori}}}{P_1^{\mathrm{ori}}}},\hfill \\ \hfill {\mu }_2^{\mathrm{ori}}& =\sum _{k=t_1+1}^{t_2}{\displaystyle \frac{k{p}_k^{\mathrm{ori}}}{P_2^{\mathrm{ori}}}},\hfill \\ & ⋮\hfill \\ \hfill {\mu }_{j+1}^{\mathrm{ori}}& =\sum _{k=t_j+1}^{90}{\displaystyle \frac{k{p}_k^{\mathrm{ori}}}{P_{j+1}^{\mathrm{ori}}}}\hfill \end{array}$$

(16)

The entire image’s mean gradient orientation angle can be obtained by

$${\mu }_T^{ori}=\sum _{k=-90}^{90}k{p}_k^{ori}.$$

(17)

The between-class variance of orientation can be represented by

$$\widehat{{\sigma }_B^2}(t_1,t_2,\dots ,t_j)=P_1^{ori}{({\mu }_1^{ori}-{\mu }_T^{ori})}^2+P_2^{ori}{({\mu }_2^{ori}-{\mu }_T^{ori})}^2+\dots +P_{j+1}^{ori}{({\mu }_{j+1}^{ori}-{\mu }_T^{ori})}^2.$$

(18)

Maximizing the objective function $\widehat{{\sigma }_B^2}(t_1,t_2,\dots ,t_j)$ yields the optimal set of thresholds as follows:

$${\left(\overrightarrow{t}\right)}^{*}=\underset{-90\le t_1,t_2,\dots ,t_j\le 90}{argmax}\left\{\widehat{{\sigma }_B^2}(t_1,t_2,\dots ,t_j)\right\}.$$

(19)

Using the optimal threshold ${\left(\overrightarrow{t}\right)}^{*}$, the 2-D gradient orientation histogram $p(m,n)$ is clustered into $j+1$ parts with the largest between-class variance. Each part contains pixels with similar gradient orientation information. Figure 2 shows an example; with the optimal vector ${\left(\overrightarrow{t}\right)}^{*}=[-35,34]$, the pixels of each gradient orientation part can be sorted by gray-level value and yield the corresponding histogram. It indicates that pixels within these clustered regions exhibit similar texture patterns. Processing these regions separately may enhance feature recognition and improve segmentation quality.

After clustering the image, the Otsu method mentioned in Section 2.2 is used to perform local thresholding segmentation on these $j+1$ regions. These $j+1$ segmentation results are then combined to obtain the final segmentation result. Figure 3 illustrates the overall segmentation process.

The pseudocode for the proposed algorithm is presented in Algorithm 1.

Algorithm 1 Image Segmentation based on Gradient Orientation Histogram

Input: A grayscale image f.    Output: A final segmented image $S_{final}$.   1: procedure SEGMENTATION BASED ON PIXEL GRADIENT CLUSTERING(f)   2:     ▹Step 1: Construct and normalize the 2-D gradient orientation histogram   3:     Calculate the 2-D gradient orientation histogram $h(m,n)$ of the image f, where m and n respectively represent gray-level value and gradient orientation of pixel located at position $(x,y)$.   4:     Obtain the probability distribution $p(m,n)$ by normalizing the histogram: $p(m,n)=h(m,n)/{\sum }_{m,n}h(m,n)$.   5:     ▹Step 2: Calculate the 1-D gradient orientation histogram   6:     Calculate the 1-D gradient orientation angles histogram $p_k^{\mathrm{ori}}$ by accumulating the 2-D histogram along the gradient orientation dimension: $p_k^{\mathrm{ori}}\leftarrow {\sum }_{m}p(m,k)$.   7:     ▹Step 3: Cluster the image into regions based on gradient orientation   8:     Segment the distribution $p_k^{\mathrm{ori}}$ into $j+1$ distinct parts by Equation (19).   9:     Cluster the image into $j+1$ regions $\mathcal{R}=\{R_1,\dots ,R_{j+1}\}$, where each region contains pixels with similar gradient orientation angles. 10:     ▹Step 4: Segment each region using thresholds 11:     Initialize a set of local segmentation results ${\mathcal{S}}_{local}\leftarrow \varnothing$. 12:     for all region $R_i\in \mathcal{R}$ do 13:         Obtain the gray-level histogram $h_i$ for region $R_i$. 14:         Use the Otsu method to segment $R_i$ based on $h_i$ to obtain a local segmentation result $S_i$. 15:         Add $S_i$ to ${\mathcal{S}}_{local}$. 16:     end for 17:     ▹Step 5: Splice local segmentation results to obtain the final result 18:     Splice all local segmentation results in ${\mathcal{S}}_{local}$ to obtain the final segmentation image $S_{final}$. 19:     return $S_{final}$ 20: end procedure

3.3. Comparison of Existing Similar Methods

Although the definitions of pixels’ gray-level gradient orientation in

Table 1. Summary of methods utilizing gradient orientation in recent works.

Reference	Definition of Gradient Orientation	Usage of Gradient Orientation
[22]	Defines the gradient orientation as $\theta (x,y)=arctan\left({\displaystyle \frac{\partial f}{\partial x}}/{\displaystyle \frac{\partial f}{\partial y}}\right)$.	Constructs the orientation histogram $h_{\mathrm{\Theta }}(x,y)={\sum }_{j,i}\omega (x+i,y+j)·\delta [\mathrm{\Theta },\theta (x+i,y+j)]$ for image segmentation.
[38]	Defines the gradient orientation as $\theta (x,y)=arctan(I\ast d_y/I\ast d_x)$, where I is an image window, $d_x=[-1,0,1]$ and $d_y={[-1,0,1]}^T$.	Calculates the histogram vector $v_t=\{b_t^0,b_t^1,\dots ,b_t^{N-1}\}$ to interpret geological object on seismic image.
[40]	Defines the gradient orientation as $\theta (x,y)=arctan\left({\displaystyle \frac{\partial f}{\partial y}}/{\displaystyle \frac{\partial f}{\partial x}}\right)$.	Yields the histogram of oriented gradient (HOG) to enhance the crack detection.
Proposed	Defines the gradient orientation as $\theta (x,y)=arctan\left({\displaystyle \frac{\partial f}{\partial y}}/{\displaystyle \frac{\partial f}{\partial x}}\right)$.	Constructs the gradient orientation histogram $h(m,n)=Count\left\{f\right(x,y)=m\mathrm{and}\theta (x,y)=n\}$, which is used to cluster the pixels.

are similar, the applications of such information in image processing are quite different. More specifically, HOG is always divided into several bins that have the same intervals in angular space [22,38,40], while in the present work, the pixels are clustered into distinct subsets with different angular intervals by using the Otsu criterion. The relationships between images’ visual patterns and the variance of gradient orientation can be further studied.

3.4. Image Test Sets and Quality Evaluation Parameters

Image texture not only characterizes local structural patterns but also effectively captures inter-object boundary information. Due to its important role in object recognition, texture features have been widely employed in image analysis. To further show the relationship between texture information and image segmentation, images with diverse texture characteristics are adopted in the experiments. Figure 4 displays three images with obvious texture features from the image library of MATLAB2015(8.6.0.267246), with resolutions of (a) $306\times 648$, (b) $189\times 250$, and (c) $232\times 205$.

Besides Figure 4, more real-world images from various scenes are essential to test the proposed algorithm. The Describable Textures Dataset (DTD, R1.0.1) [53] is an image dataset consisting of 47 different categories of real-world texture images, with each category named by an adjective. It is designed as a public benchmark and can be used to study the problem of extracting semantic properties of textures and patterns. The DTD dataset is available at [54]. In this paper, the segmentation experiment employs 81 ‘woven’, 119 ‘lacelike’, 107 ‘pitted’, and 119 ‘blotchy’ images. Here are a few examples from this dataset (Figure 5).

To evaluate the effectiveness of the local thresholding method proposed above, we adopt PSNR (Peak Signal-to-Noise Ratio) and FSIM (Feature Similarity Index) as quality indices. PSNR [55] represents the ratio of the peak signal to the noise. As it can precisely measure the difference between the input image and the output image, it is currently the most frequently used objective metric for evaluating image quality. A higher PSNR value indicates less distortion in the output image and better segmentation quality. It is defined as

$$PSNR=10{log}_{10}\left[{\displaystyle \frac{{(L-1)}^2}{MSE}}\right]$$

(20)

where $MSE$ is the mean squared error between the input image and the output image, and L is the maximum gray-level value in the image, usually 255. $MSE$ can be written as

$$MSE={\displaystyle \frac{1}{M\times N}}\sum _{i=1}^M\sum _{j=1}^N{\left[f\right(i,j)-g(i,j\left)\right]}^2$$

(21)

where $M\times N$ is the size of the image, and $f(i,j)$ and $g(i,j)$ represent the input image and the output image, respectively.

FSIM [56] is proposed based on the principle that the human visual system understands an image mainly according to its low-level features. It is utilized to measure the similarity between two images based on phase congruence (PC) and image gradient magnitude (GM). Here, PC serves as a dimensionless indicator of the importance of local structures and is used as the primary feature in FSIM. GM acts as the secondary feature in FSIM. Both features play complementary roles in characterizing the local quality of an image. FSIM is defined as

$$FSIM={\displaystyle \frac{{\sum }_{x\in \mathrm{\Omega }}{S}_L\left(x\right)·P{C}_m\left(x\right)}{{\sum }_{x\in \mathrm{\Omega }}P{C}_m\left(x\right)}}$$

(22)

where $\mathrm{\Omega }$ means the whole image spatial domain, $S_L\left(x\right)$ is the local similarity at pixel x, and ${PC}_m\left(x\right)$ is the maximum PC value at pixel x. Please refer to [56] for more details about $S_L\left(x\right)$ and ${PC}_m\left(x\right)$.

4. Experimental Results and Discussions

In Figure 2, the image ‘board’ is clustered into three parts by optimized vector $t^{*}=[-35,34]$. It is worth to note that the segmentation results of the proposed method are quite stable with the variation of clustering. According to Equation (19), the image can be clustered into an arbitrary number of parts, i.e., $j=2,3,4,\dots$. Each part is segmented by using the Otsu method.

Table 2. PSNR and FSIM values with different j at threshold level 2.

j	Board		Bag		Tire		Woven_0118		Lacelike_0053		Pitted_0009		Blotchy_0085
	PSNR	FSIM	PSNR	FSIM	PSNR	FSIM	PSNR	FSIM	PSNR	FSIM	PSNR	FSIM	PSNR	FSIM
2	20.5403	0.8654	21.7597	0.8037	22.1534	0.7417	24.0976	0.7335	27.1553	0.8219	22.9091	0.8141	24.6104	0.8408
3	20.5560	0.8755	21.7905	0.8198	22.1416	0.7381	24.1442	0.7331	27.1465	0.8197	22.9227	0.8236	24.6096	0.8393
4	20.5724	0.8817	21.7939	0.8200	22.1688	0.7423	24.1498	0.7366	27.1712	0.8299	22.9255	0.8271	24.6108	0.8400
5	20.5847	0.8828	21.8023	0.8210	22.1710	0.7438	24.1769	0.7420	27.1732	0.8292	22.9509	0.8372	24.6119	0.8409

lists the segmentation results of the image ‘board’, ‘bag’, ‘tire’, ‘woven_0118’, ‘lacelike_0053’, ‘pitted_0009’, and ‘blotchy_0085’ with different numbers of clustering parts.

It is shown that the variations of PSNR and FSIM for the abovementioned images are all very small when j increases. Without loss of generality, all images are clustered into three parts, i.e., $j=2$, in the following experiments.

To demonstrate the performance of the proposed method, we tested images from a MATLAB image library (Figure 4) and the DTD dataset (Figure 5). The Kapur method [57], 1-D Otsu [58], 2-D Otsu [51], QOMVO-based Tsallis entropy ($q=0.8$) [59], self-adaptive Tsallis entropy [6], ACM-Tsallis [27], and the proposed method were applied to segment these images, with the numbers of thresholds set to 2, 3, 4, and 5.

4.1. Algorithm Performance and Computational Cost

Figure 6 demonstrates the two-level thresholding performances for three images of Figure 4. It is found that the proposed method keeps more details than others, and can avoid a lot of mis-segmentations. Notably, this advantage keeps with the increasing number of thresholds and can be quantitatively illustrated.

Table 3. Comparisons of PSNR and FSIM values for multi-level thresholding on images of Figure 4 by using 7 different algorithms.

Image	Thresholds	Kapur (1-D)		Otsu (1-D)		Otsu (2-D)		QOMVO (1-D)		Adaptive (1-D)		ACM-Tsallis (2-D)		Proposed
		PSNR	FSIM	PSNR	FSIM	PSNR	FSIM	PSNR	FSIM	PSNR	FSIM	PSNR	FSIM	PSNR	FSIM
board	2	19.8486	0.8417	20.5365	0.8594	20.1805	0.8603	20.0574	0.8462	20.2337	0.8499	20.3342	0.8559	20.5403	0.8654
	3	22.0879	0.8925	23.0899	0.9254	20.9382	0.8703	22.2054	0.8947	22.4471	0.9001	22.5171	0.9196	23.0959	0.9304
	4	24.1754	0.9367	25.1001	0.9489	21.0960	0.8721	24.5454	0.9419	24.6850	0.9429	24.7531	0.9461	25.1076	0.9532
	5	25.9686	0.9539	26.7874	0.9635	21.0735	0.8718	26.1119	0.9548	26.1548	0.9552	26.3624	0.9592	26.7947	0.9665
bag	2	21.0917	0.7064	21.7318	0.7291	21.6042	0.6936	21.0939	0.7054	21.1872	0.7132	21.6345	0.7457	21.7597	0.8037
	3	23.9386	0.8154	24.1852	0.8388	22.1903	0.7096	23.9386	0.8154	23.9274	0.8146	24.0522	0.8301	24.2273	0.8677
	4	25.8303	0.8759	25.9751	0.8839	22.3528	0.7113	25.8304	0.8759	25.8208	0.8797	25.9087	0.8853	26.0317	0.9124
	5	27.2505	0.9106	27.5939	0.9154	22.3593	0.7113	27.3375	0.9114	27.2353	0.9101	27.4899	0.9119	27.6188	0.9226
tire	2	20.9682	0.6295	22.0913	0.6762	21.0869	0.6406	20.9682	0.6295	20.9206	0.6290	21.0608	0.6408	22.1534	0.7417
	3	24.7329	0.7427	24.7887	0.7439	24.2287	0.7270	24.7018	0.7433	24.6246	0.7421	24.6331	0.7433	24.8545	0.8013
	4	26.5611	0.7991	26.6432	0.8020	26.2494	0.7919	26.5435	0.7977	26.4583	0.7963	26.4832	0.7959	26.6680	0.8362
	5	27.4821	0.8282	28.2698	0.8414	26.6968	0.8075	27.4828	0.8289	27.4711	0.8282	27.6303	0.8296	28.3058	0.8686

presents the PSNR and FSIM results at different threshold levels for these three images. In each row, the maximum PSNR and FSIM values (with bold font) indicate that the corresponding method is the most suitable one for the image listed at the beginning of the row. The proposed method achieved the best PSNR and FSIM values for all three texture-rich images across all threshold levels. It performs better than 1-D histogram-based methods because it incorporates more pixel-wise correlation information for segmentation. Meanwhile, traditional 2-D methods neglect off-diagonal pixels’ contributions, leading to image information loss. In contrast, the proposed method utilizes gradient information from all pixels for segmentation, thereby achieving superior performance. This also indicates that, compared with the other two 2-D histogram-based methods, the gradient orientation histogram is more effective in analyzing the texture features of these images.

Besides QOMVO (1-D), ACM-Tsallis (2-D), and Adaptive (1-D), the other four algorithms are all performed in exhaustive calculation to ensure the accuracy.

Table 4. Computation time (seconds) of two-level thresholding of Figure 4 by using different algorithms.

	Kapur (1-D)	Otsu (1-D)	Otsu (2-D)	Proposed
board	0.32	0.27	147,103.22	2.35
bag	0.32	0.27	135,890.45	1.25
tire	0.32	0.27	142,732.34	1.22

presents the comparative computational time costs across these four methods. All of them were implemented in Python and executed on a workstation equipped with dual Intel Xeon Gold 6268CL processors, (Santa Clara, CA, USA) (2.80 GHz, 28 cores total) and 256 GB DDR4 RAM (2666 MHz). To ensure fair comparison, all methods were restricted to single-thread execution.

It is shown that the proposed method obtained better segmentation results while requiring less computational time than the 2-D Otsu method. It should be noted that the proposed method can adopt multiprocessing computation for segmenting the three clustered regions. Therefore, compared with traditional 1-D methods, it only incurs additional computational cost for constructing 2-D gradient orientation histogram and the corresponding pixels’ clustering. It is worth to mention that the computation time of the proposed method in Table 4 is based on $j=2$, and this value will increase exponentially as j increases. Nevertheless, as indicated by the results in Table 2, the increase in j contributes trivially to the quality indices. Therefore, $j=2$ achieves both stable segmentation quality improvement and low computational consumption, showing the potential application in real-time scenes.

4.2. Consistency of Algorithm Performance with Increasing Threshold

To further validate the effectiveness of the proposed method, we applied the above seven methods to perform two-level, three-level, four-level, and five-level thresholding on images from the DTD dataset mentioned in Section 3.4.

Figure 7 shows parts of the two-level thresholding results of some images. It is evident that, for these four images, both 1-D histogram-based methods and 2-D methods yielded unsatisfactory segmentation results. Specifically, the 1-D Kapur method and three Tsallis-entropy-based methods failed to capture the texture details, leading to difficulties in segmenting and preserving complex and diverse texture lines in the original images. The traditional 2-D method only considers pixels along the main diagonal of the histogram, which may result in the loss of important texture information. In comparison, the proposed method effectively preserved the texture lines and features from the original images.

In order to show the performances of these seven algorithms objectively, different categories of the DTD dataset are adopted to yield the comparisons. For example, the category named ’woven’ contains 81 images with similar texture structures. At a given number of thresholds, each image yields seven PSNR values by above seven algorithms. The algorithm corresponding to the maximum PSNR value is considered as the optimal processing method for the given image. Then the count of this optimal algorithm increases by 1 accordingly. After all the test images are processed, the distribution of optimal PSNR among seven algorithms can be obtained. It should be mentioned that two algorithms may occasionally reach the same maximum results. Therefore, the total count of seven algorithms may slightly exceed the total number of images in the category. In the same way, the distributions of FSIM can be obtained.

Table 5. The optimal PSNR/FSIM distributions among 7 algorithms by using a DTD image dataset at a given number of thresholds.

Image	Thresholds	Kapur (1-D)		Otsu (1-D)		Otsu (2-D)		QOMVO (1-D)		Adaptive (1-D)		ACM-Tsallis (2-D)		Proposed
		PSNR	FSIM	PSNR	FSIM	PSNR	FSIM	PSNR	FSIM	PSNR	FSIM	PSNR	FSIM	PSNR	FSIM
woven (81 images)	2	0	0	2	6	0	7	0	0	0	4	0	13	81	51
	3	0	0	0	1	0	0	0	2	0	2	0	22	81	54
	4	0	2	6	1	0	0	0	1	0	4	0	18	75	55
	5	0	3	1	2	0	0	0	1	0	2	0	21	80	53
lacelike (119 images)	2	0	3	1	4	0	5	0	0	0	2	0	26	119	79
	3	0	1	1	5	0	0	0	0	0	3	0	31	118	79
	4	0	3	16	6	0	0	0	0	0	3	0	26	103	81
	5	0	2	1	11	0	0	0	1	0	2	0	17	118	86
pitted (107 images)	2	0	7	10	10	0	2	0	7	0	4	0	37	104	46
	3	0	4	5	7	0	0	0	0	0	1	0	23	103	73
	4	0	2	6	14	0	0	0	2	0	1	0	27	105	64
	5	0	3	12	11	0	0	0	2	0	0	0	26	99	68
blotchy (119 images)	2	0	2	15	10	2	4	0	3	0	0	0	7	113	100
	3	0	1	13	12	0	1	0	2	0	1	0	9	112	99
	4	0	3	8	9	0	1	0	1	0	1	0	13	115	92
	5	0	1	14	8	0	0	0	0	0	0	0	17	109	95

presents the distributions of optimal PSNR and FSIM among four algorithms by using different image categories. For each image category, the values in bold represent the highest frequencies of reaching the optimal segmentation quality, which shows the superiority of the corresponding algorithm.

For the ‘woven’ category, the proposed method achieved optimal PSNR values over all 81 images at threshold levels 2 and 3, and obtained optimal FSIM values in approximately $63\%$ of cases. This demonstrates that the method maintains both high segmentation quality and effective preservation of texture details. With the increasing number of thresholds, such as four and five, the optimal rate of PSNR for the proposed algorithm is still dominant, and the corresponding optimal rate of FSIM slightly increases. This suggests that the proposed algorithm is stable in both segmentation quality and feature preservation for this image category with increasing threshold numbers.

In order to show the effectiveness of the proposed algorithm, more image categories can be involved. For the ‘lacelike’ category, the optimal PSNR rate of the present algorithm is nearly $100\%$ threshold levels 2, 3, and 5, demonstrating exceptional overall performance. For the ‘pitted’ category, the optimal PSNR rates remain above $92\%$, and those yielded by different threshold levels remain unchanged within a small interval of fluctuation. For the ‘blotchy’ category, the proposed method still outperforms other methods. From Figure 5, we can see that the texture patterns among four image categories are quite different from each other. The proposed method reached an average optimal PSNR rate of up to $96\%$, showing its powerful ability in texture pattern recognition. Regarding the optimal FSIM rate, the proposed method also remains far larger than those of the other typical algorithms. This superiority is unchanged in different image categories, with different threshold levels.

Actually, the superiority of the proposed algorithm benefits from the pre-segmentation clustering based on local texture features, which closely depends on the pixels’ gradient orientation distribution. Although the four texture categories exhibit distinct visual characteristics, the proposed method consistently obtains satisfactory PSNR values across all categories. This shows that it effectively identifies texture patterns across diverse image categories, demonstrating strong scalability. FSIM evaluation shows that the proposed method has superior capability in preserving original image features across all categories. This validates that pixel gradient clustering contributes to analyzing complex texture distributions in different images. Furthermore, the method retains relatively stable performance with overall small fluctuations across different threshold levels, indicating good robustness and stability.

4.3. Comparative Analysis of Method Performance

The two 2-D methods yield unsatisfactory performance across all texture categories, with near-zero counts of images achieving maximum PSNR values at all threshold levels. This likely stems from their threshold determination mechanism disregarding certain pixels, leading to significant loss of edge and feature information. Interestingly, while the 1-D Kapur method showed similarly poor PSNR results as two 2-D methods, it outperforms the 2-D Otsu method in the FSIM index but is lower than the other. This can be attributed to its global threshold selection based on all image pixels, which prevents substantial feature loss and consequently preserves more texture characteristics in some images. Additionally, the ACM-Tsallis method is better than these two traditional classical algorithms. However, both 1-D Tsallis-entropy-based methods yield non-ideal performance, indicating that non-extensive entropy is not suitable for texture-rich images. The 1-D Otsu method achieved relatively better results for a small subset of images in both PSNR and FSIM. This may reflect better compatibility between the between-class variance maximization criterion and these texture images.

Clearly, traditional 1-D methods show limited performance for these texture-rich images. They fail to account for spatial correlations between pixels and utilize insufficient image information during threshold selection. However, 2-D methods do not appear to perform significantly better, since the traditional 2-D histogram-based thresholding methods ignored pixels with high contrast to their surroundings. It is more suitable for low SNR scenarios. Consequently, its ability to extract texture information is weak, leading to suboptimal segmentation results.

The superior performance of the proposed method shows the importance of texture information in segmentation tasks. It also confirms that gradient orientation histogram serves as an effective descriptor for image texture patterns. This strongly validates the necessity of clustering pixels with similar gradient orientations for multi-level image segmentation.

In general, clustering pixels by their gradient orientations is beneficial to recognize an image in computer vision. For example, traditional 1-D gray level histogram cannot distinguish two pixels with the same gray-level value. However, their gradient orientations should be very different if they belong to two different image patterns. In order to verify the effectiveness of the proposed method, images other than a texture-rich one, such as medical images [60], high-resolution overhead baseball diamond images [61], BSDS0500 images [62], and TID2008 images [63], are also involved in our experiments. Based on the statistical results of more than 100 arbitrary selected samples, it is found that the proposed method still outperforms the other six methods mentioned above.

5. Conclusions

In image segmentation, 2-D thresholding methods extract additional pixel-wise features, thereby overcoming the limitations of 1-D approaches in certain scenarios. This demonstrates that proper utilization of additional image features can effectively improve segmentation quality. However, traditional 2-D entropy thresholding methods may inadvertently discard a significant proportion of pixels, thereby undermining the preservation of nontrivial edge details.

In this paper, we propose a new local thresholding algorithm to further study the importance of texture features in improving multi-level segmentation quality. To evaluate the performances, we compare our method with the Kapur method, the Otsu method, and the Tsallis entropy-based methods. Segmentation results are assessed by the PSNR index. Experimental results demonstrate that our algorithm can precisely identify common patterns in texture-rich images. Specifically, when segmenting four texture categories of images with distinct characteristics from the DTD dataset, our method achieves significantly higher segmentation quality than other algorithms. This advantage stems from clustering pixels with similar gray-level gradient orientation before segmentation, which facilitates the algorithm’s understanding and analysis of feature distribution patterns in images. Additionally, the segmentation quality remains stable as the threshold level increases. Our method also shows superior performance in the FSIM index, proving that the gradient orientation histogram effectively captures texture information across different images. It can identify diverse texture patterns and periodical structures, preserving complex texture features in segmentation results. The performance sustains stability even as threshold levels increase, highlighting strong scalability. Notably, the improved performance comes with negligible computational cost, demonstrating potential for real-time applications.

A multi-level image segmentation technique plays an important role in image preprocessing and further affects the performances of deep learning-based image processing tasks. Therefore, a threshold segmentation method with superior performance and stability can provide great help for these related tasks. In this study, the common features of images from the same category can be effectively recognized by gradient orientation clustering. Additionally, the qualities of multi-thresholding have been improved essentially. Nevertheless, this paper only explores the relationship between gradients and image patterns from a fundamental point. For pattern recognition tasks in more complex scenarios, it is necessary to combine more modern techniques such as deep learning. Therefore, in future work, it is of interest to combine the present method with the proper deep learning model and examine the effectiveness in the general scope of image categories.