Renyi’s Entropy Based Multilevel Thresholding Using a Novel Meta-Heuristics Algorithm

Abstract

Multi-level image thresholding is the most direct and effective method for image segmentation, which is a key step for image analysis and computer vision, however, as the number of threshold values increases, exhaustive search does not work efficiently and effectively and evolutionary algorithms often fall into a local optimal solution. In the paper, a meta-heuristics algorithm based on the breeding mechanism of Chinese hybrid rice is proposed to seek the optimal multi-level thresholds for image segmentation and Renyi’s entropy is utilized as the fitness function. Experiments have been run on four scanning electron microscope images of cement and four standard images, moreover, it is compared with other six classical and novel evolutionary algorithms: genetic algorithm, particle swarm optimization algorithm, differential evolution algorithm, ant lion optimization algorithm, whale optimization algorithm, and salp swarm algorithm. Meanwhile, some indicators, including the average fitness values, standard deviation, peak signal to noise ratio, and structural similarity index are used as evaluation criteria in the experiments. The experimental results show that the proposed method prevails over the other algorithms involved in the paper on most indicators and it can segment cement scanning electron microscope image effectively.

1. Introduction

With the rapid development of computers, technologies of image processing were applied widely in many areas, including quality estimation [1,2], infrared detection [3,4], disease recognition [5,6], agricultural identification [7,8], fingerprint identification [9], and many other aspects [10,11]. Especially, image analysis has become a very useful method in the study of cement microstructure based on the cement SEM (scanning electron microscope) image [12], as it is possible to analyze the mechanism of cement hydration reaction by observing the microstructure of cement in cement stone materials [13]. Image segmentation is the first step of image understanding, and any other steps (such as feature extraction and recognition) largely depend on its results, it is to partition an image into interrelated parts or regions, which are composed of image pixels with related data eigenvalues.

Threshold technology is considered as the most popular image segmentation algorithm, which has the advantages of small storage space, fast processing speed, and easy operation. Some successful thresholding approaches have been proposed for image segmentation, including inter class variance method (OTSU) [14], the maximum entropy-based thresholding method [15], Renyi’s entropy-based thresholding method [16], Tsallis’ entropy based thresholding method [17], and so on. Due to the complexity of the real world, multi-level thresholding-based partition methods have become more broadly used for practical work [18,19,20]. When the one-dimensional threshold algorithm is extended to multiple dimensions, they may show high computational cost and do not meet real-time requirements when the number of classes to be detected increases.

In essence, this problem could be viewed as a combinatorial optimization problem, and meta-heuristics algorithms are often employed to deal with this kind of problem. For instance, Huiyan J, Xiaoqi M, et al. combined improved fruit fly optimization algorithm with support vector machine and used it to classify pancreatic cancer [21], ant colony optimization were utilized for the selection of accounting models, graph anonymization and robot rescue mission [22,23,24], Reddy G T, Srivastava G et al. applied hybrid genetic algorithms to the diagnosis of heart disease [25], and many other evolutionary algorithms have been applied to solve optimization problems in various fields [26,27,28]. There is no doubt that a lot of researches have been conducted around the multi-threshold segmentation handled with evolutionary algorithms, such as hybrid whale optimization algorithm is employed in Kapur entropy for multi-threshold segmentation [29], combine particle swarm optimization algorithm with Tsallis entropy for multi-threshold segmentation [30], ant colony optimization algorithm is employed in OTSU to quickly search for multiple thresholds in images [31], some other optimization algorithms are also used for multi-threshold segmentation, such as water cycle algorithm [32], cuckoo search algorithm [33], knee evolutionary algorithm [34], differential evolution algorithm [35], and bat algorithm [36]. These efforts have achieved good results; however, it is hard for meta-heuristic optimization algorithms to obtain the same optimal solution as the exhaustive method with the increment of the number of thresholds in seconds.

Inspired by the breeding mechanism of Chinese hybrid rice, a novel meta-heuristics algorithm is proposed [37], for simplicity, it is called hybrid rice optimization algorithm (HRO). HRO has the advantages of strong stability, large search range, and strong optimization. In [37], it is shown that HRO prevails over particle swarm optimization (PSO), differential evolution algorithms (DE), genetic algorithm (GA), artificial bee colony (ABC) on the basis of six benchmark functions. At the same time, HRO has also been used for security situation prediction in [38]. Preliminary studies show that it is very prospective and competition for the problem with low or middle dimension. Therefore, HRO is suggested to seek the optimal solution for a multi-level image threshold in the paper as the number of threshold values is less than 12 in common. The presented scheme employs HRO to maximize the objective function of Renyi’s entropy, and GA [39], PSO [40], DE [41], ant lion optimization algorithm (ALO) [42], whale optimization algorithm (WOA) [43], and SALP swarm algorithm (SSA) [44] are also adopted in our presented scheme for comparison experiments.

The remainder of this work is organized as follows: In Section 2, an overview of Renyi’s entropy is given, and the basic principle of the meta-heuristics algorithm derived from the breeding mechanism of Chinese hybrid rice is explained briefly. In Section 3, the thresholding method based on the proposed meta-heuristics algorithm is introduced. In Section 4, the experimental setup and data required for the experiment are introduced, and through several experiments and quantitative analysis, the conclusion proves the effectiveness of the proposed method. Finally, we draw a conclusion in Section 5.

2. Methodological Background

2.1. Renyi’s Entropy

Renyi’s entropy is a generalized form of Shannon’s entropy, in which an adjustable parameter $\alpha$ is introduced, Therefore, the measurement of information coolness is more general and flexible, and when $\alpha$ = 1, Renyi’s entropy is equal to Shannon’s entropy. In the paper, the coefficient of $\alpha$ is 2.

The method of Renyi’s entropy with single threshold is described as follows: suppose I is an image to be segmented with L gray levels, and the probability of each gray level distribution is $\left\{p\left(1\right),p\left(2\right),p\left(3\right),\dots p\left(\mathrm{i}\right)\dots ,p\left(L\right)\right\}$ Suppose there is only one single threshold T to divide image I into two parts: the target $C_0=\left\{0,1,2,\dots ,t\right\}$ and background $C_1=\left\{t+1,t+2,\dots ,L-1\right\}$.

Target and background gray probabilities are calculated, $w_0\left(t\right)$ is the probability of $C_0$ occurrence, $w_1\left(t\right)$ is the probability of $C_1$ occurrence, and $w_0\left(t\right)+w_1\left(t\right)=1$, the definitions of $w_0\left(t\right)$ and $w_1\left(t\right)$ are as

$$w_0\left(t\right)={{\displaystyle \sum }}_{i=1}^{t}p\left(i\right)$$

(1)

$$w_1\left(t\right)={{\displaystyle \sum }}_{i=t}^{L-1}p\left(i\right)$$

(2)

According to the definition of Renyi’s entropy, the Renyi’s entropy of image background and target is defined as

$$R{t}_{\alpha }^0\left(t\right)=\frac{1}{1-\alpha }ln{{\displaystyle \sum }}_{i=1}^t{\left(\frac{p\left(i\right)}{w_0\left(t\right)}\right)}^{\alpha }$$

(3)

$$R{t}_{\alpha }^1\left(t\right)=\frac{1}{1-\alpha }ln{{\displaystyle \sum }}_{i=t+1}^{L-1}{\left(\frac{p\left(i\right)}{w_1\left(t\right)}\right)}^{\alpha }$$

(4)

The total Renyi’s entropy is calculated by

$$T\left(t\right)=R{t}_{\alpha }^0\left(t\right)+R{t}_{\alpha }^1\left(t\right)$$

(5)

The optimal threshold T should satisfy Equation (6).

$$t=argmax\left(T\left(t\right)\right)$$

(6)

According to the Renyi’s entropy method of single threshold, the Renyi’s entropy method of multi-threshold could be deduced. Suppose there are N thresholds $\left\{{\mathrm{t}}_1,{\mathrm{t}}_2,{\mathrm{t}}_3,\dots ,{\mathrm{t}}_{\mathrm{N}}\right\}$, the histogram is divided into N + 1 regions, the gray probability of the first threshold $\mathrm{t}$ could be described as,

$$w_1\left(t\right)={{\displaystyle \sum }}_{i=1}^{t_1}p\left(i\right)$$

(7)

Except for the first and N-th thresholds, the gray probability of other thresholds could be defined as

$$w_n\left(t\right)={{\displaystyle \sum }}_{i=t_{n-1}+1}^{t_n}p\left(i\right)$$

(8)

The gray probability of the last threshold t as

$$w_N\left(t\right)={{\displaystyle \sum }}_{i=t_{N-1}+1}^{L-1}p\left(i\right)$$

(9)

The Renyi’s entropy of each region is calculated by Equation (10), where n ranges from 1 to N:

$$R{t}_{\alpha }^n=\frac{1}{1-\alpha }ln{{\displaystyle \sum }}_{i=t_{n-1}+1}^{t_n}{\left(\frac{p\left(i\right)}{w_n\left(t\right)}\right)}^{\alpha }$$

(10)

The total Renyi’s entropy is calculated by Equation (11) below.

$$T\left(\left[t_1,t_2,t_3,\dots ,t_N\right]\right)={{\displaystyle \sum }}_{n=1}^{N}R{t}_{\alpha }^n\left(t_n\right)$$

(11)

The selected optimal threshold $\left[t_1,t_2,\dots ,t_N\right]$ should meet Equation (12).

$$\left[t_1,t_2,\dots ,t_N\right]=argmax\left(T\left(\left[t_1,t_2,\dots ,t_N\right]\right)\right)$$

(12)

2.2. The Meta-Heuristics Algorithm Derived from the Breeding Mechanism of Chinese Hybrid Rice

The proposed meta-heuristics algorithm is derived from the breeding mechanism of Chinese hybrid rice, which uses rice genes as value in the solution space, and the fitness function is used to measure the pros and cons of the rice gene. According to the pros and cons of their genes, Chinese hybrid rice is divided into sterile lines, maintainer lines, and restorer lines. Figure 1 shows the breeding procedure of three-line Chinese hybrid rice.

The symbols $\times$ and $\otimes$ denote hybridization and selfing, respectively. Sterile lines (A) refer to a special type of rice that looks similar to regular rice. Due to the abnormal development of male organs in sterile lines, sterile lines are often pollen-free or aborted, and male sterility is controlled by cytoplasmic genes. The development of female organs of male sterile lines is healthy and can accept normal pollen for fertilization. It is one of the important tools to obtain Chinese hybrid rice seeds. Maintainer lines (B) refer to rice varieties that can maintain cytoplasmic male sterility. Maintainer rice seeds have the same nuclear genotype as sterile rice seeds. However, the cytoplasmic genes of the maintainer line are fertile, and the pollen is fertile and capable of self-breeding. Since the maintainer line is the same nuclear genotype as the sterile line, it is not possible to alter sterility controlled by cytoplasm. Therefore, the progeny of the maintainer line and the sterile line are also sterile, that is, $A\times B\to A$. The restorer line (R) is a variety that can repair cytoplasmic male sterility. It has a nuclear gene that restores cytoplasmic male sterility. When it crosses a sterile line, the seed usually has fertility and heterosis. This offspring is the hybrid rice seed (F1) we need, that is, $A\times \mathrm{R}\to \mathrm{F}1$. Chinese hybrid rice is the progeny formed by crossing two rice cultivars, sterile line and restorer line, the yield and other important agronomic traits are better than the parent and control varieties.

A meta-heuristic algorithm based on the breeding mechanism of Chinese hybrid rice (hereinafter referred to as HRO) simulates the “three-line” breeding process. There are two main breeding behaviors, that is, crossing, a sterile line is crossed with a maintainer line to update a sterile line individual, selfing, the restorer is inbred to update the next-generation restorer. The cross-breeding of sterile lines and maintainer lines is an evolutionary process, and hybrid dominance can quickly reduce the gap between populations. Therefore, individuals with relatively significant differences are crossed. The self-crossing of restorer lines enables individuals with medium fitness value to reach the current optimal solution quickly, and has the ability to jump out of the local optimal solution.

3. Proposed Method

In this section, an effective method of multi-threshold image segmentation based on HRO is introduced. The specific implementation steps are described below.

3.1. Idea of the Algorithm

The key idea of employing HRO to deal with multi-threshold is mapping of problem solutions. The objective functions of image segmentation used in the paper are Renyi’s entropy. The real-coded HRO is used to quickly obtain the optimal solution of multiple thresholds. A seed in the HRO represents a solution of the problem. The dimension of the gene in the seed is equal to the dimension space of the problem to be solved. For example, if the number of thresholds to be solved is 2, the spatial dimension of the problem is 2, the dimension of gene is 2. The optimal threshold could be obtained by maximizing the value of the objective function. Among them, since hybrid rice is divided into three lines, it is recommended that the population is a multiple of 3, for example, 30, 45, 60.

Definition: The population of rice is N, the proportion of the maintainer line and the sterile line in the population is a%, and the number is A = N × a/100. Then the proportion of the restorer line in the population is (100–2a)%. The number of maintainer lines, sterile lines, and restorer lines is A, A, and N-2A (the number of individuals in the three lines is generally N/3). The dimension of each individual gene is D, the value of D is the number of thresholds, in the paper $D=\left\{2,4,6,8\right\}$. $X_i^t$ indicates the gene of the i-th individual in the group at the t-th breeding, $X_i^t=\left(X_i^1,X_i^2,X_i^3,\dots ,X_i^{D-1},X_i^D\right)$. When t = 0, randomly generate N solutions in the solution space: $x_i^0$ = $\left\{x_1^0,x_2^0,\dots ,x_{N-1}^0,x_N^0\right\}$. The specific generation formula is done by Equation (13).

$${\mathrm{x}}_{\mathrm{i}}^{\mathrm{j}}={\min \mathrm{x}}^{\mathrm{j}}+\mathrm{rand}\left(0,1\right)\left(\max {\mathrm{x}}^{\mathrm{j}}-{\min \mathrm{x}}^{\mathrm{j}}\right)$$

(13)

where $j\in \left\{1,2,3,\dots ,D\right\}$, $\max x^j$ and $min{x}^j$ represent the maximum and minimum values of the j-th component of the search space, respectively, the problem solved in the paper is the optimal threshold, so the maximum value is 255 and the minimum value is 0. The fitness value of each individual in the population is calculated and the current optimal value is recorded. The fitness function in HRO is the formula of Renyi’s entropy introduced in the second part of the paper.

The rice population is ranked according to the fitness value from superior to inferior. The top A individuals are taken as maintenance lines, the bottom A individuals are sterile lines, and the remaining N-2A individuals are restorer lines.

Hybridization: Select one of the following hybridization methods as the hybridization method of the entire algorithm.

Random hybridization:

$${\mathrm{x}}_{\mathrm{si}}^{\mathrm{j}}\left(\mathrm{t}+1\right)=\frac{{\mathrm{r}}_1\cdot {\mathrm{x}}_{\mathrm{ma}}^{\mathrm{j}}\left(\mathrm{t}\right)+{\mathrm{r}}_2\cdot {\mathrm{x}}_{\mathrm{sb}}^{\mathrm{j}}\left(\mathrm{t}\right)}{{\mathrm{r}}_1+{\mathrm{r}}_2}$$

(14)

$x_{si}^j\left(t+1\right)$ represents is the j-th gene in the gene sequence of the i-th rice seed in the sterile line, $r_1,r_2$ is a random number between −1 and 1, and $r_1+r_2\ne 0$, in the paper, the threshold value solved by HRO is only positive integer, so $r_1,r_2$ is a random number between −0.5 and 1.5, a, b are randomly taken from {1,2,3 … , m}, $x_{ma}^j$ is the a-th individual in the maintaining lines, and $x_{sb}^j$ is the b-th individual in the sterile lines. Each dimension of the newly generated individual gene sequence is calculated by Equation (14) from rice seed of randomly selected male sterile lines and maintainers.

Corresponding hybridization:

$${\mathrm{x}}_{\mathrm{si}}^{\mathrm{j}}\left(\mathrm{t}+1\right)=\frac{{\mathrm{r}}_1\cdot {\mathrm{x}}_{\mathrm{mc}}^{\mathrm{j}}\left(\mathrm{t}\right)+{\mathrm{r}}_2\cdot {\mathrm{x}}_{\mathrm{sc}}^{\mathrm{j}}\left(\mathrm{t}\right)}{{\mathrm{r}}_1+{\mathrm{r}}_2}$$

(15)

$x_{mc}^j\left(t\right)\mathrm{and}{x}_{sc}^j\left(t\right)$ represent the j-th gene in the c-th seed gene sequence of maintainer line and sterile line. The new individuals are randomly obtained from the c-th rice seed of the sterile line and the c-th rice seed of the maintainer line in a certain proportion.

Selfing: In the process of self-crossing breeding, rice seeds are randomly selected to update their genes along the direction of the current optimal solution. Based on the greedy strategy, after calculating the fitness value of the new individual and comparing it with the individual before self-crossing, the number of self-crossing operations of the seed is denoted as zero. Otherwise, the self-intersection operand of the individual is added by 1. When the number of self-crossing operations of restorer line individuals reaches the limit $T_{max}$, they will no longer participate in the self-crossing process of the next round of breeding but will be reset. Selfing could be described as

$${\mathrm{x}}_{\mathrm{ri}}^{\mathrm{j}}\left(\mathrm{t}+1\right)={\mathrm{x}}_{\mathrm{ri}}^{\mathrm{j}}\left(\mathrm{t}\right)+\mathrm{rand}\left(0,1\right)\left({\mathrm{x}}_{\mathrm{best}}-{\mathrm{x}}_{\mathrm{rk}}^{\mathrm{j}}\left(\mathrm{t}\right)\right)$$

(16)

where, $x_{ri}^j\left(t+1\right)$ represents the new individual generated by self-intersection of the i-th restorer line $x_{ri}^j\left(t\right)$, and $x_{rk}^j$ represents the k-th individual randomly selected from the restorer line. The $x_{best}$ represents the current optimal individual.

Reset: Resume individuals who have reached the maximum number of inbred will reset. Reset could be described as

$${\mathrm{x}}_{\mathrm{ri}}^{\mathrm{j}}={\mathrm{x}}_{\mathrm{ri}}^{\mathrm{j}}+\mathrm{rand}\left(0,1\right)\left({\mathrm{x}}_{\max }-{\mathrm{x}}_{\min }\right)$$

(17)

where, $x_{ri}^j$ represents the new rice produced by the reset operation of restorer line $x_{ri}^j$.$x_{max}$ and $x_{min}$ are the upper and lower limits of gene quantization.

If the maximum breeding number is satisfied or less than the predefined error, the current global optimal value is output as the result, otherwise skip to step “Hybridization”.

The time complexity of the multi-level image thresholding algorithm based on HRO is $O\left(Max\_iteration\ast N\ast DlogD\right)$, where $Max\_iteration$ is the maximum number of iterations, N is the size of the population, and D is the dimension of the individual, $DlogD$ represents the time it takes to sort after each position update.

3.2. The Procedure of the Algorithm

Figure 2 shows the procedure of the proposed scheme.

HRO is not easy to fall into the local optimum and has good stability. When the number of thresholds is small, the threshold value can also be calculated by the exhaustive method. However, as the number of threshold values increases, the exhaustive method has high time complexity and low efficiency. In the paper, HRO is applied to search the optimal threshold fast and effectively for Renyi’s entropy-based thresholding method.

4. Simulation Results and Discussion

We implement the proposed approach by the language of matlab2015 on a personal computer with 8.00 G RAM, a CPU of Intel (R) Core (TM) i5-3470 @3.20 GHz, under Windows10 system. In order to show the advantages of this method, half of our test images are natural images of standard images and the other half are SEM images of cement. The cement image used in the experiment was taken from po42.5 cement produced by Hubei Huaxin Cement Plant, and the water cement ratio was 0.35, which was prepared according to the standard of GB / t17671-1999. Standard images are named “baboon”, “cameraman”, “house”, “pepper”, and we have named SEM images of cement “image1”, “image2”, “image3”, “image4”. The images are shown in Figure 3.

The paper also mentions some commonly used evolutionary algorithms or swarm intelligence algorithms to seek the best multiple thresholds. In order to make a purposeful and effective comparison, the selected threshold segmentation method is Renyi’s entropy, which is optimized by evolutionary algorithm or swarm intelligence optimization algorithm. The paper uses HRO-based Renyi’s entropy (HRO-Renyi). The threshold segmentation methods for comparison experiments are: GA-based Renyi’s entropy (GA-Renyi), PSO-based Renyi’s entropy (PSO-Renyi), DE-based Renyi’s entropy (DE-Renyi), ALO-based Renyi’s entropy (ALO-Renyi), WOA-based Renyi’s entropy (WOA-Renyi), SSA-based Renyi’s entropy (SSA-Renyi). GA, PSO, DE, ALO, WOA, and SSA all use their standard algorithms, parameters of algorithm are shown in

Table 1. Parameter setting of algorithms used.

Algorithm	Parameter Setting	Value	Reference
HRO	$selfingTim{e}_{max}$	30	[37]
GA	pc	0.8	[39]
	pm	0.1
PSO	c1	2	[40]
	c2	2
	v	[−6, 6]
DE	fr	0.1	[41]
	cr	0.6
WOA	a1	[0, 2]	[43]
	a2	[−2, −1]
	R1	rand
	R2	rand
SSA	C2	rand	[44]
	C3	rand
ALO	/	/	[42]

. The number of iterations of all algorithms is 100 and the number of populations is 60. To test the stability of the algorithm, all algorithms are run 50 times. In this section, we provide some comparative experimental results, including some segmentation examples, performance evaluation tables, and function convergence graphs, which clearly demonstrate the advantages of the proposed method. Here we present 2-level, 4-level, 6-level, and 8-level thresholding for visual evaluation of natural images. Because the number of thresholds is too large, the exhaustive method is not applicable, our main interest is the optimization ability, which is mainly demonstrated through the fitness value of the objective function and stability analysis. For a further comparison, a multi-threshold segmentation method based on inter-class variance is also implemented in the paper, which is optimized using HRO (HRO-OTSU). The HRO-OTSU method is compared with the HRO-Renyi method, the accuracy of target segmentation is evaluated by peak signal to noise ratio (PSNR) and structural similarity index (SSIM).

PSNR is a widely used method of image quality evaluation index, which is based on the gray value of image pixels for statistics and average calculation. Although there may be a large deviation between the quality of some images or video quality evaluation and the quality of subjective perception, PSNR is still effective for most image quality evaluation.

Given an image $I$ with a size of $m\ast n$ and an image $K$ after threshold segmentation, MSE is defined as Equation (18):

$$\mathrm{MSE}=\frac{1}{\mathrm{mn}}{\displaystyle \sum }_{\mathrm{i}=0}^{\mathrm{m}-1}{\displaystyle \sum }_{\mathrm{j}=0}^{\mathrm{n}-1}{\left[\mathrm{I}\left(\mathrm{i},\mathrm{j}\right)-\mathrm{K}\left(\mathrm{i},\mathrm{j}\right)\right]}^2$$

(18)

PSNR is defined as Equation (19):

$$\mathrm{PSRN}=10{\log }_{10}\left(\frac{{\mathrm{MAX}}_{\mathrm{I}}^2}{\mathrm{MSE}}\right)$$

(19)

$MA{X}_I^2$ is the maximum possible pixel value of the image. If each pixel is represented by 8-bit binary, it is 255. Generally, if the pixel value is represented by B-bit binary, then $MA{X}_I=2^B-1$. Generally, for uint8 data, the maximum pixel value is 255, and for floating-point data, the maximum pixel value is 1.

SSIM is an index to measure the similarity of two images. It measures the similarity of images from three aspects of luminance, contrast and structure. SSIM is defined as:

$$SSIM\left(x,y\right)=\left[l{\left(x,y\right)}^{\alpha }·c{\left(x,y\right)}^{\beta }·s{\left(x,y\right)}^{\gamma }\right]$$

(20)

$$\mathrm{l}\left(\mathrm{x},\mathrm{y}\right)=\frac{2{\mathsf{\mu }}_{\mathrm{x}}{\mathsf{\mu }}_{\mathrm{y}}+{\mathrm{c}}_1}{{\mathsf{\mu }}_{\mathrm{x}}^2+{\mathsf{\mu }}_{\mathrm{y}}^2+{\mathrm{c}}_1}$$

(21)

$$\mathrm{c}\left(\mathrm{x},\mathrm{y}\right)=\frac{2{\mathsf{\sigma }}_{\mathrm{x}}{\mathsf{\sigma }}_{\mathrm{y}}+{\mathrm{c}}_2}{{\mathsf{\sigma }}_{\mathrm{x}}^2+{\mathsf{\sigma }}_{\mathrm{y}}^2+{\mathrm{c}}_2}$$

(22)

$$\mathrm{s}\left(\mathrm{x},\mathrm{y}\right)=\frac{{\mathsf{\sigma }}_{\mathrm{xy}}+{\mathrm{c}}_3}{{\mathsf{\sigma }}_{\mathrm{x}}{\mathsf{\sigma }}_{\mathrm{y}}+{\mathrm{c}}_3}$$

(23)

Where $\mathrm{x}$ and $\mathrm{y}$ are the two image blocks for comparison, ${\mathsf{\mu }}_{\mathrm{x}}$ is the mean of $\mathrm{x}$, ${\mathsf{\mu }}_{\mathrm{y}}$ is the mean of y, ${\mathsf{\sigma }}_{\mathrm{x}}^2$ is the variance of $\mathrm{x}$, ${\mathsf{\sigma }}_{\mathrm{y}}^2$ is the variance of $\mathrm{y}$, ${\mathsf{\sigma }}_{\mathrm{xy}}$ is the covariance of $\mathrm{x}$ and y. ${\mathrm{c}}_1={\left({\mathrm{k}}_1\mathrm{L}\right)}^2$, ${\mathrm{c}}_2={\left({\mathrm{k}}_2\mathrm{L}\right)}^2$, ${\mathrm{c}}_3={\mathrm{c}}_2/2$, ${\mathrm{c}}_1,{\mathrm{c}}_2$ and ${\mathrm{c}}_3$ are used to avoid the divisor of the equation being 0, $\mathrm{L}$ is the range of pixel values $2^{\mathrm{B}}-1$, ${\mathrm{k}}_1=0.01,{\mathrm{k}}_2=0.03$, all are default values. α, β, γ is a positive number, weight used to adjust brightness, contrast and structural correlation. When α = β = γ = 1, the SSIM equation could be described as:

$$\mathrm{SSIM}\left(\mathrm{x},\mathrm{y}\right)=\frac{\left(2{\mathsf{\mu }}_{\mathrm{x}}{\mathsf{\mu }}_{\mathrm{y}}+{\mathrm{c}}_1\right)\left(2{\mathsf{\sigma }}_{\mathrm{x}}{\mathsf{\sigma }}_{\mathrm{y}}+{\mathrm{c}}_2\right)}{\left({\mathsf{\mu }}_{\mathrm{x}}^2+{\mathsf{\mu }}_{\mathrm{y}}^2+{\mathrm{c}}_1\right)\left({\mathsf{\sigma }}_{\mathrm{x}}^2+{\mathsf{\sigma }}_{\mathrm{y}}^2+{\mathrm{c}}_2\right)}$$

(24)

Finally, SSIM (x, y) of all image blocks is averaged to obtain SSIM of the whole image:

$$\mathrm{MSSIM}\left(\mathrm{X},\mathrm{Y}\right)=\frac{1}{\mathrm{M}}{\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{M}}\mathrm{SSIM}\left({\mathrm{x}}_{\mathrm{i}},{\mathrm{y}}_{\mathrm{i}}\right)$$

(25)

Among them, M is the number of X and Y image blocks of the two compared images. If the SSIM is larger, it means that the difference between the output image and the original image is smaller, the image quality is better.

4.1. Quantitative Evaluation of Segmented Results

In order to quantitatively evaluate the performance of the threshold segmentation method based on HRO, this section uses the value of the indicator to explain the effectiveness and stability of the algorithm. The evaluation methods are: fitness value, standard deviation (STD), PSNR and SSIM, the higher the fitness value, the stronger the optimization ability, the lower the STD, the more stable the algorithm, and the higher the values of PSNR and SSIM, the better the segmentation effect.

Table 2. The average fitness values of each algorithm under Renyi’s entropy.

Test Image	k	HRO-Renyi	ALO-Renyi	DE-Renyi	SSA-Renyi	WOA-Renyi	PSO-Renyi	GA-Renyi
baboon	2	11.4686	11.4686	11.4686	11.4684	11.4686	11.4686	11.468
	4	16.5897	16.588	16.5874	16.5779	16.5791	16.5854	16.5549
	6	21.0589	21.0585	21.0371	21.0303	20.9759	21.053	20.8251
	8	24.8839	24.4965	24.7552	24.3798	24.6572	24.7754	24.3504
cameraman	2	12.4864	12.4864	12.4864	12.4864	12.4863	12.4864	12.4863
	4	18.4057	18.3471	18.4052	18.3909	18.3975	18.4009	18.3846
	6	23.7795	23.7200	23.7729	23.7570	23.7473	23.7475	23.6548
	8	28.6612	28.5974	28.6348	28.6437	28.5839	28.6417	28.3456
house	2	11.8064	11.8064	11.8064	11.8064	11.80616	11.8064	11.80631
	4	17.3329	17.3329	17.33224	17.3329	17.32379	17.33291	17.30065
	6	22.0218	22.02141	22.00228	22.01171	21.97924	22.01973	21.89545
	8	26.1522	25.69837	26.0066	25.33308	26.04371	25.76974	25.77073
pepper	2	12.5218	12.5218	12.5218	12.5218	12.5218	12.5218	12.5217
	4	18.3199	18.3199	18.3194	18.3178	18.3185	18.3198	18.3047
	6	23.4910	23.4909	23.4846	23.4854	23.463	23.4888	23.3293
	8	28.0391	27.9952	28.0121	27.9978	27.9937	28.0289	27.6193
image1	2	12.5460	12.5458	12.5460	12.5456	12.5458	12.5459	12.5456
	4	18.7229	18.7222	18.7223	18.7208	18.7188	18.7217	18.6988
	6	24.0641	24.0617	24.0522	24.0527	24.0385	24.0604	23.9386
	8	28.8219	28.8078	28.7710	28.6861	28.7250	28.7597	28.4813
image2	2	11.3539	11.3539	11.3539	11.3539	11.3521	11.3539	11.3533
	4	16.6878	16.6855	16.6869	16.6874	16.6810	16.6841	16.6592
	6	21.4421	21.2219	21.3739	21.1208	21.3276	21.4302	21.1757
	8	25.4777	25.2385	25.2277	24.4738	25.2488	24.8811	24.9524
image3	2	11.3417	11.3417	11.3417	11.3417	11.3413	11.3417	11.3416
	4	16.6759	16.6759	16.6741	16.6757	16.6687	16.6757	16.6499
	6	21.0103	20.8263	21.0759	20.7750	21.0994	20.7757	20.9413
	8	25.1005	25.0947	25.0833	25.0896	25.0238	25.0923	24.8731
image4	2	11.8064	11.8064	11.8064	11.8064	11.8063	11.8064	11.8063
	4	17.3329	17.3329	17.3327	17.3329	17.3277	17.3329	17.3053
	6	22.0218	21.9362	22.0032	22.0133	21.9971	22.0198	21.8579
	8	26.1522	25.7859	25.9856	25.4901	26.0444	25.7597	25.7633

shows the average fitness values of HRO, GA, PSO, DE, ALO, WOA, and SSA in Renyi’s entropy after 50 experiments. $\mathrm{k}$ in the table represents the threshold-levels, and the bold numbers indicate that the corresponding algorithm’s average fitness value is the largest in the test image, that is, the optimization capability is the strongest.

The results in Table 2 display that HRO is better than other algorithms in finding the best threshold of Renyi’s entropy. As is known there are 32 groups of the optimal thresholds (8 test images, 4 groups of thresholds for each image). Among the best average fitness values shown in the table, HRO-Renyi gets the optimal average solution 31 times, which takes the first place, ALO-Renyi got the optimal solution 9 times, the difference between them is 22. The number of times that other algorithms have obtained the optimal solution is less than 10 times. Figure 4 shows the proportion of the optimal average fitness values of each method, with the horizontal axis representing each method and the longitudinal axis representing the proportion. It could be seen that the number of times HRO-Renyi has found the best fitness accounts for 96.875% of the total number of experiments. In most cases, HRO-Renyi can find the optimal solution stably. In eight images, when the threshold level k = 2, all methods except GA-Renyi can find the optimal solution. When the threshold level k = 4, HRO-Renyi can get the optimal average fitness value in all images, and ALO-Renyi can get the optimal solution in some images, and the performance of other algorithms begins to decline. When the threshold-level k=6, the average fitness value obtained by HRO-Renyi in the seven images is the largest. Although WOA-Renyi performs best in “image3“, the average fitness value of HRO-Renyi is close to WOA-Renyi, the difference is only 0.0891. When the threshold-level k = 8, the average optimal fitness value of HRO-Renyi is the maximum in the eight images.

When the threshold-level k = {2,4}, all the optimization algorithms have good performance and can get better solutions, but when the number of thresholds increases, HRO is more stable and the optimization ability is better than other methods. HRO can effectively find the optimal threshold.

Figure 5 shows the convergence curve of the optimal fitness values with 8 thresholds (“house” and “image2”). These threshold methods are HRO-Renyi, DE-Renyi, PSO-Renyi, SSA-Renyi, ALO-Renyi, WOA-Renyi, GA-Renyi. It could be seen that some algorithms are prone to fall into a local optimum or not converge to the maximum entropy, resulting in a lower average fitness value. However, the HRO-based threshold segmentation algorithm has achieved the maximum fitness value in these two cases, indicating that the algorithm is more effective than other algorithms. The accuracy of HRO-Renyi to find the optimal solution is the highest among all methods, but the number of iterations required to find the optimal solution is more than that of PSO-Renyi and ALO-Renyi, it can converge at about 40 to 50 iterations. When the number of iterations is 50, the optimal solution obtained by HRO-Renyi is already better than other methods, DE-Renyi and SSA-Renyi need more iterations to converge, they probably need about 70 iterations, while the number of iterations required for convergence of WOA-Renyi and GA-Renyi is close to HRO-Renyi, but the accuracy is not as good as HRO-Renyi.

In this section, STD is used to measure the stability of each optimization algorithm in each threshold segmentation algorithm. STD is the arithmetic ordinary root of variance, and the average of the distance (deviation from average) of each data deviation. STD can reflect the dispersion degree of a data set.

Table 3. STD of each algorithm under Renyi’s entropy.

T × 10 st Imag × 10	k	HRO-R × 10 nyi	ALO-R × 10 nyi	D × 10 -R × 10 nyi	SSA-R × 10 nyi	WOA-R × 10 nyi	PSO-R × 10 nyi	GA-R × 10 nyi
baboon	2	0.00 × 10+00	0.00 × 10+00	0.00 × 10+00	4.00 × 10−04	2.00 × 10−04	0.00 × 10+00	7.00 × 10−04
	4	2.21 × 10−03	6.00 × 10−03	3.10 × 10−03	1.24 × 10−02	8.80 × 10−03	8.50 × 10−03	1.63 × 10−02
	6	1.01 × 10−02	1.01 × 10−02	1.62 × 10−02	4.95 × 10−02	8.14 × 10−02	9.60 × 10−03	1.10 × 10−01
	8	6.31 × 10−03	7.66 × 10−01	7.97 × 10−02	7.66 × 10−01	1.69 × 10−01	3.75 × 10−01	1.97 × 10−01
cam × 10 raman	2	3.65 × 10−15	3.59 × 10−15	3.59 × 10−15	3.59 × 10−15	3.65 × 10−04	3.59 × 10−15	2.13 × 10−04
	4	2.53 × 10−04	1.05 × 10−01	1.11 × 10−03	5.86 ×10 −02	1.02 × 10−02	3.48 × 10−02	1.49 × 10−02
	6	1.47 × 10−04	9.63 × 10−02	5.56 × 10−03	4.88 × 10−02	4.60 × 10−02	6.09 × 10−02	5.31 × 10−02
	8	9.16 × 10−04	9.61 × 10−02	1.48 × 10−02	4.56 × 10−02	6.76 × 10−02	4.36 × 10−02	1.29 × 10−01
hous × 10	2	3.65 × 10−15	3.65 × 10−15	3.65 × 10−15	3.65 × 10−15	5.67 × 10−04	3.65 × 10−15	2.48 × 10−04
	4	3.91 × 10−06	0.00 × 10+00	1.14 × 10−03	1.27 × 10−05	1.21 × 10−02	1.23 × 10−04	2.39 × 10−02
	6	8.07 × 10−04	8.85 × 10−04	2.74 × 10−02	2.10 × 10−02	7.33 × 10−02	1.52 × 10−03	6.43 × 10−02
	8	1.25 × 10−02	7.94 × 10−01	1.09 × 10−01	8.38 × 10−01	7.11 × 10−02	7.27 × 10−01	1.36 × 10−01
p × 10 pp × 10 r	2	0.00 × 10+00	0.00 × 10+00	0.00 × 10+00	0.00 × 10+00	0.00 × 10+00	0.00 × 10+00	5.00 × 10−04
	4	0.00 × 10+00	0.00 × 10+00	1.70 × 10−03	6.80 × 10−03	1.50 × 10−03	2.00 × 10−04	1.04 × 10−02
	6	8.44 × 10−05	3.00 × 10−04	5.10 × 10−03	2.94 × 10−02	2.41 × 10−02	1.20 × 10−03	7.44 × 10−02
	8	7.31 × 10−04	3.02 × 10−01	1.47 × 10−02	6.93 × 10−02	4.88 × 10−02	3.90 × 10−03	1.26 × 10−01
imag × 10 1	2	6.07 × 10−05	5.48 × 10−04	8.97 × 10−15	6.54 × 10−04	6.72 × 10−04	3.39 × 10−04	8.99 × 10−04
	4	0.00 × 10+00	1.67 × 10−03	1.20 × 10−03	2.63 × 10−03	3.63 × 10−03	1.92 × 10−03	1.84 × 10−02
	6	4.81 × 10−03	5.25 × 10−03	9.34 × 10−03	1.04 × 10−02	2.21 × 10−02	4.58 × 10−03	4.93 × 10−02
	8	7.13 × 10−03	4.31 × 10−02	2.72 × 10−02	9.33 × 10−02	6.17 × 10−02	7.83 × 10−02	9.49 × 10−02
imag × 10 2	2	8.97 × 10−15	8.97 × 10−15	8.97 × 10−15	8.97 × 10−15	5.75 × 10−03	8.97 × 10−15	3.95 × 10−03
	4	5.89 × 10−03	1.14 × 10−02	4.72 × 10−03	6.23 × 10−03	1.30 × 10−02	8.88 × 10−03	1.61 × 10−02
	6	1.09 × 10−14	6.39 × 10−01	5.48 × 10−02	6.77 × 10−01	8.24 × 10−02	4.22 × 10 −02	1.02 × 10−01
	8	3.52 × 10−02	5.78 × 10−01	2.25 × 10−01	9.50 × 10−01	1.42 × 10−01	8.45 × 10−01	1.71 × 10−01
imag × 10 3	2	1.79 × 10−15	1.79 × 10−15	1.79 × 10−15	1.79 × 10−15	1.37 × 10−03	1.79 × 10−15	3.79 × 10−05
	4	3.65 × 10−15	3.59 × 10−15	1.86 × 10−03	7.86 × 10−04	1.02 × 10−02	4.16 × 10−04	2.15 × 10−02
	6	1.76 × 10−01	1.84 × 10−01	6.64 × 10−02	1.49 × 10−01	1.78 × 10−02	1.48 × 10−01	9.31 × 10−02
	8	7.58 × 10−04	1.67 × 10−02	8.93 × 10−03	2.70 × 10−02	9.70 × 10−02	4.11 × 10−03	1.00 × 10−01
imag × 10 4	2	5.38 × 10−15	5.38 × 10−15	5.38 × 10−15	5.38 × 10−15	4.84 × 10−04	5.38 × 10−15	5.90 × 10−04
	4	3.91 × 10−06	0.00 × 10+00	6.52 × 10−04	1.85 × 10−04	5.93 × 10−03	5.31 × 10−05	1.82 × 10−02
	6	8.07 × 10−04	4.23 × 10−01	2.71 × 10−02	2.06 × 10−02	3.33 × 10−02	1.51 × 10−03	7.46 × 10−02
	8	1.25 × 10−02	7.24 × 10−01	1.72 × 10−01	8.28 × 10−01	9.86 × 10−02	7.17 × 10−01	1.24 × 10−01

represents the STD of each optimization algorithm after 50 tests in Renyi’s entropy (Bold is best). When the STD is smaller, the algorithm is more stable. After the minimum STD of all methods is calculated, HRO-Renyi has the lowest STD in 28 of 32 experiments, accounting for 87.5% of all experiments, the minimum STD times of other methods are: ALO-Renyi has 12 times, DE-Renyi has 8 times, SSA-Renyi has 6 times, WOA-Renyi has 2 times, PSO-Renyi has 9 times, and GA-Renyi has 0 times. When the threshold-level is 2, except for WOA-Renyi, SSA-Renyi, and GA-Renyi, other methods have a STD of 0 or close to 0 in most test images, which means that the optimal solutions obtained by these methods in 50 experiments are all equal and have strong stability. As the number of thresholds increases, the STD increases, the optimization algorithm easily falls into a local optimum, which leads to the instability of the algorithm. As could be seen from the Table 3, when the threshold-levels k = {2, 4}, STD of HRO-Renyi method is mostly 0 or close to 0. When the threshold-levels k = {6, 8}, the maximum STD is 0.176. Compared with other algorithms, HRO has the minimum STD and the best stability.

It has been manifested above that the HRO-Renyi method has achieved good results in terms of the best fitness function and algorithm stability. HRO-OTSU is also used to segment the experimental image, quantitative results are defined by PSNR, SSIM. To improve the richness of experimental results, the experimental image is divided into two parts in this discussion, one is the image in the standard data set, and the other is the SEM image of cement.

The optimal PSNR and SSIM values obtained by HRO-OTSU and HRO-Renyi are given in

Table 4. PNSR, SSIM, regional consistency, and regional contrast.

Test Image	k	PNSR		SSIM
		HRO-OTSU	HRO-Renyi	HRO-OTSU	HRO-Renyi
baboon	2	6.7981	6.7363	0.3064	0.3046
	4	8.7101	9.8992	0.4904	0.551
	6	13.9917	12.1854	0.6913	0.659
	8	15.084	14.953	0.7477	0.7657
cameraman	2	6.6695	8.5097	0.4680	0.4088
	4	11.9553	15.895	0.5338	0.5502
	6	12.0709	13.9544	0.5876	0.6126
	8	14.2463	19.6721	0.6179	0.6813
house	2	6.6118	7.0059	0.4383	0.4045
	4	12.2213	12.3194	0.6033	0.7013
	6	10.8699	12.7908	0.6534	0.6687
	8	15.1439	16.2506	0.7387	0.7747
pepper	2	6.9852	7.3017	0.4342	0.4176
	4	14.3867	11.797	0.5631	0.5484
	6	15.7046	14.2376	0.6296	0.6346
	8	20.5632	15.8249	0.6924	0.6949
image1	2	6.4293	11.6851	0.1016	0.0731
	4	9.427	14.2248	0.2454	0.2398
	6	11.5935	17.4116	0.3735	0.3996
	8	13.2339	17.299	0.4871	0.5351
image2	2	5.9105	10.412	0.1759	0.0712
	4	9.0231	11.7494	0.2733	0.2611
	6	9.8523	14.0487	0.3483	0.4269
	8	11.1423	14.2233	0.4068	0.5407
image3	2	5.6788	10.0547	0.124	0.0804
	4	8.6725	11.3931	0.2026	0.1928
	6	9.4832	13.0616	0.2672	0.31
	8	10.0703	16.5418	0.3247	0.419
image4	2	6.1726	8.7871	0.1582	0.1142
	4	9.073	11.7971	0.2418	0.2189
	6	11.1861	13.5219	0.3362	0.3522
	8	12.082	16.2838	0.4051	0.4489

. It could be found that, in terms of PNSR, performance of HRO-Renyi on the standard data set is basically the same as that of HRO-OTSU. However, on the SEM image of cement data set, no matter if threshold-levels k = {2, 4, 6, 8}. HRO-Renyi is larger than HRO-OTSU, which shows that HRO-Renyi has good robustness and good segmentation quality. In terms of SSIM, HRO-Renyi performs slightly better than HRO-OTSU on the standard data set, but when threshold-levels k = {6, 8}, the former is often larger than the latter, and on the SEM image of cement data set, regardless of whether threshold-levels k = {6, 8}, HRO-Renyi is larger than HRO-OTSU, which shows that performance of HRO-Renyi in multi-dimensional threshold segmentation is better than HRO-OTSU. Therefore, the HRO-Renyi method is more robust and more suitable for multi-dimensional threshold segmentation of SEM images of cement.

4.2. Visual Evaluation of the Segmented Image

The following Figure 6 and Figure 7 are the segmented images with threshold-level k = 8, through observation, we could obtain that our proposed method can effectively segment images. Hydration products could be separated in the microscopic image of cement.

In Figure 6, when the threshold level is 8, in the test image “Baboon”, the image segmented by ALO-Renyi, DE-Renyi, SSA-Renyi, PSO-Renyi is darker overall, the color and contrast difference between the baboon’s nose and the surrounding area is not very obvious. At the same time, the images segmented by WOA-Renyi and GA-Renyi are brighter overall. The areas of the baboon face are not distinguished clearly, but the overall image of the “Baboon” segmented by HRO-Renyi is moderate in brightness. The area of the image has obvious contrast; in the test image “cameraman”, the boundary between the person and the background in the segmented image by HRO-Renyi is clearer, and the color contrast between the regions is obvious; in the test images “house” and “pepper”, compared with other methods, the segmented images by HRO-Renyi has better visual effects and stronger layering. Therefore, HRO-Renyi has better perceptual quality and more obvious contrast on standard image datasets than other algorithms as a whole.

In the SEM image of cement “image1”, the microscopic cement material in the image divided by WOA-Renyi shows a large-area adhesion phenomenon, which makes it difficult to distinguish the components of each material. Images segmented by ALO-Renyi, DE-Renyi, SSA-Renyi, PSO-Renyi, and GA-Renyi contain relatively little cement micro-materials, while the images segmented by HRO-Renyi, not only are the cement microscopic substances easy to identify, but they are rich in variety and quantity. In the SEM image of cement “image2, image3”, the images segmented by HRO-Renyi contain the most cement microscopic matter, and it does not show blurring. Therefore, for the SEM image of cement, the image segmented by HRO-Renyi has a clear sense of hierarchy and contains rich and recognizable cement microscopic substances. In the SEM image of cement “image4”, it could be seen that HRO-Renyi is the only one that can clearly segment cement hydrate and pores. Images segmented by WOA-Renyi and GA-Renyi are relatively fuzzy, where substances cannot be classified. When ALO-Renyi, DE-Renyi, SSA-Renyi, and PSO-Renyi are used to segment cement SEM images, it is easy to segment some dark light into pores, although they also separate the pores of cement.

In Figure 7, HRO-OTSU can also segment the hydration products and pores in the cement. However, divided by HRO-OTSU, the SEM image of cement “image1, image2, image3, image4” still has some noise locally. HRO-Renyi performs better than HRO-OTSU in the PNSR and SSIM of SEM images of cement, which means proves HRO-Renyi is more effective.

5. Conclusions

In the paper, a multi-level image segmentation technique using HRO-based Renyi’s entropy algorithm for microscopic images of cement is proposed. Taking Renyi’s entropy as the objective function, HRO is used to search the optimal threshold values and compared with other optimization algorithms (GA, PSO, DE, ALO, WOA, SSA), four standard image sets and four SEM images of cement are selected, and the threshold-levels are 2-level, 4-level, 6-level, and 8-level. After 50 experiments, the experimental results are evaluated from the best fitness value and STD. It could be seen that the performance of HRO is better than other optimization algorithms in terms of average optimal fitness value and STD. Further, HRO-OTSU is employed for comparison experiments, and the results show that HRO-Renyi performs better in terms of PSNR and SSIM in standard images and microscopic images of cement, indicating that it is more suitable for threshold segmentation. In the future, chaotic strategy and other local search strategies like Levy flight could be also used to develop HRO algorithm for image segmentation and other applications. In addition, the combination of HRO and other evolutionary algorithms is also the interesting research direction.