Contextual Peano Scan and Fast Image Segmentation Using Hidden and Evidential Markov Chains ^†

Abstract

Transforming bi-dimensional sets of image pixels into mono-dimensional sequences with a Peano scan (PS) is an established technique enabling the use of hidden Markov chains (HMCs) for unsupervised image segmentation. Related Bayesian segmentation methods can compete with hidden Markov fields (HMFs)-based ones and are much faster. PS has recently been extended to the contextual PS, and some initial experiments have shown the value of the associated HMC model, denoted as HMC-CPS, in image segmentation. Moreover, HMCs have been extended to hidden evidential Markov chains (HEMCs), which are capable of improving HMC-based Bayesian segmentation. In this study, we introduce a new HEMC-CPS model by simultaneously considering contextual PS and evidential HMC. We show its effectiveness for Bayesian maximum posterior mode (MPM) segmentation using synthetic and real images. Segmentation is performed in an unsupervised manner, with parameters being estimated using the stochastic expectation–maximization (SEM) method. The new HEMC-CPS model presents potential for the modeling and segmentation of more complex images, such as three-dimensional or multi-sensor multi-resolution images. Finally, the HMC-CPS and HEMC-CPS models are not limited to image segmentation and could be used for any kind of spatially correlated data.

1. Introduction

With the recent advancements in deep learning techniques, a considerable number of deep learning models have been designed and trained to perform unsupervised image segmentation. Indeed, from classical convolutional encoder–decoders [1] to recurrent networks [2], generative adversarial networks [3], and attention encoder–decoders [4], all these methods have produced spectacular results in various domains such as semantic segmentation, cell image segmentation, and super-resolution reconstruction, outperforming more traditional approaches. However, these models still require large training databases, even if the data are unlabeled. In this article, we focus on “agnostic” statistical methods, which only require the information present in the image for segmentation, and no additional sources. In this context, methods based on hidden Markov fields (HMFs) have been widely used since the pioneering research was performed [5,6,7]. HMFs provide satisfactory results in various situations, notably in image segmentation problems [8,9,10,11,12]. However, direct fast calculations are not tractable and iterative methods such as Gibbs or Metropolis sampling are required. These calculations significantly increase the computation time and make the related methods hard to apply in some situations. Utilizing the well-known and widely used hidden Markov chains (HMCs) [13,14,15,16,17] instead of HMFs is feasible; however, transforming a bi-dimensional set of pixels into a mono-dimensional sequence is not straightforward. For example, proceeding “line by line” gives an HMC in which pixels close each to other in the image may be far apart in the sequence. Using the Peano scan partially overcomes these difficulties and allows one to bring the quality of the results obtained with the chains closer to those obtained with the fields [18,19]. The combination of the Peano scan and HMCs has been used to segment different types of images [19,20,21,22,23,24]. Peano scans have also been used in more sophisticated models than HMCs; for example, hidden fuzzy Markov chains [19,25,26] or pairwise Markov chains [26]. Even if data obtained from images via the Peano scan have a complex structure and are obviously not Markovian, the different mentioned methods can provide interesting results, demonstrating the extraordinary robustness of HMCs and their extensions.

All the studies mentioned above show the value of the Peano scan in problems where the computation time is important; indeed, thanks to direct, recursive, and exact computations, HMC-based unsupervised segmentations are incomparably faster than HMF-based ones.

We recently extended the Peano scan (PS) to the following “contextual” Peano scan (CPS) [27]. Let $s$ be a pixel in an image, and let $r,t,u,w$ be its four nearest neighbors, where $r,t$ are its neighbors in the Peano scan. In the extended model, observations on the remaining neighbors $u,w$ are also taken into account in the Peano scan: the image value observed on $s$ is completed with the two observations on $u$ and $w$. Initial experiments on synthetic images showed that using HMCs with such a “contextual” Peano scan allowed us to reduce the classification error by up to 17%. Our first contribution is to provide a simulation study comparing this recent HMC-CPS method to the classic PS-HMC, on the one hand, and to the classic HMF-based one, on the other hand. We consider several real images as well. Our second contribution is related to the use, in the context of the CPS, of extensions of HMCs called “hidden evidential Markov chains” (HEMCs) [28]. Different “evidential” Markov chains, calling on the Dempster–Shafer theory of evidence [29,30], have been proposed in [31,32,33,34]. Here, we introduce a new model associating the CPS with the HEMCs described in [28], which are particular triplet Markov chains [35,36]. Experiments indicate the existence of situations in which the new model competes with the classic HMF, while being much faster. All experiments are unsupervised, with parameters estimated using a stochastic version of the expectation–maximization (EM) method [37,38]. In the case of HMFs, we use the Gibbsian EM [39].

The organization of this article is as follows. In the next section, we present the contextual Peano scan and related HMC-CPS with Bayesian maximum posterior mode (MPM) segmentation and parameter estimation using SEM. We recall the classic HEMC model and extend it with a contextual scan in Section 3. Section 4 describes the experiments, and the last section contains the conclusions and perspectives.

2. Contextual Peano Scan and Hidden Markov Chains

2.1. Contextual Peano Scan

Let $S$ be the set of pixels of a digital image $y={{(y}_{\mathrm{s}})}_{\mathrm{s}\in \mathrm{S}}$ of size ${Card\left(S\right)=2}^k\times 2^k$. In the statistical segmentation framework we adopt in this study, ${{y=(y}_{\mathrm{s}})}_{\mathrm{s}\in \mathrm{S}}$ is a realization of a random field ${{Y=(Y}_{\mathrm{s}})}_{\mathrm{s}\in \mathrm{S}}$. Segmenting $y={{(y}_{\mathrm{s}})}_{\mathrm{s}\in \mathrm{S}}$ consists of searching $x={{(x}_{\mathrm{s}})}_{\mathrm{s}\in \mathrm{S}}$, with each $x_{\mathrm{s}}$ in a finite set of classes $\Omega =\{1,\dots ,\mathrm{K}\}$. Then, ${{x=(x}_{\mathrm{s}})}_{\mathrm{s}\in \mathrm{S}}$ is considered as a realization of a random field ${{X=(X}_{\mathrm{s}})}_{\mathrm{s}\in \mathrm{S}}$, and the segmentation problem is seen as a Bayesian problem of estimation of ${{(x}_{\mathrm{s}})}_{\mathrm{s}\in \mathrm{S}}$ from ${{(y}_{\mathrm{s}})}_{\mathrm{s}\in \mathrm{S}}$. For a given distribution $p(x,y)$ of $(X,Y)$, the problem of segmentation can be dealt with using a Bayesian classification, once the distribution $p(x,y)$ allows for related computation. The very elegant hidden Markov fields provide $p(x,y)$, for which Bayesian solutions are computable; however, the computational time needed can be prohibitive for numerous applications. Hidden Markov chains present alternative fast methods; however, their use requires converting the bi-dimensional set of pixels into a mono-dimensional sequence. Here, we consider the conversion based on the Peano scan presented in Figure 1.

Let us recall the use of the contextual scan introduced in [27]. Let $(1,\dots ,N)$ be the sequence of pixels obtained from $S$ with the Peano scan depicted in Figure 1. Let $X^N=(X_1,\dots ,X_N)$ be the related stochastic sequence of classes. The classic way of using the scan consists of setting $Y^N=(Y_1,\dots ,Y_N)$ and considering that $p(x^N,y^N)$ is a hidden Markov chain distribution. Then, both classical maximum posterior mode (MPM) segmentation and maximum a posteriori (MAP) segmentation can be computed in a fast unsupervised way (e.g., MAP can be computed with the well-known Viterbi algorithm). We consider that the contextual scan consists of the following: For each $n=2,\dots ,N-1$, let us set $s$ as the corresponding pixel in $S$. The previous point $n-1$ will be called $r$, and the next point $n+1$ will be called $t$. Let us temporarily assume that $s$ is not on the border, such that it has four neighbors in $S$. Then, we associate it with each $n=2,\dots ,N-1$ two pixels, $v_n$ and $w_n$, which are two neighbors of $n$ different from $r$ and $t$. Thus, each $n=s$ has four neighbors in $S$: two, $r=n-1$ and $t=n+1$, which belong to the Peano scan; and another two, $v_n$ and $w_n$, which do not. Then, for each $X_n$, we associate the triplet $Y_n^{\ast }={(Y_{v_n},Y}_n,Y_{w_n})$. When $s$ is on the border minus corners, there is only one neighbor, $v_n$, not lying on the Peano scan, such that $Y_n^{\ast }={(Y_{v_n},Y}_n)$. When $s$ is one of the four corners, there are two possibilities. If the scan begins or ends on the point, there is one neighbor in the scan, and one neighbor (e.g., $v_n$) outside it. Thus, $Y_n^{\ast }={(Y_{v_n},Y}_n)$. This is the case for pixels 1 and 16 in Example 1 below. If the scan neither begins nor ends on the point, there are no points that are neighbors in $S$ without being neighbors in the scan, so that $Y_n^{\ast }={(Y}_n)$. This is the case for pixels 6 and 11 in Example 1 below.

Setting $Y^{\ast N}=(Y_1^{\ast },\dots ,Y_N^{\ast })$, we consider ($X^N,Y^{\ast N})$. As we will see, $p\left(x^N\right|y^{\ast N})$ is then Markovian, which allows for the implementation of Bayesian segmentation methods.

Six possible spatial configurations of the added neighbors (green) in relation to the neighbors on the scan being not on borders (blue) are specified in Figure 2.

Example 1.

As an example, let us consider image in Figure 1b, with the Peano scan beginning in the upper left cornerPoints $n=1$, …, $16$ are specified in Figure 3 and added observations are specified in Figure 4.

Let $s$, $t$ be neighbors in $S$. If they are horizontal neighbors, let us set

$$p^h\left(y_t|x_s\right)=\sum _{x_t}{p}^h\left(x_t|x_s\right)p\left(y_t|x_t\right)$$

(1)

Similarly, if they are vertical neighbors, we set

$$p^v\left(y_t|x_s\right)=\sum _{x_t}{p}^v\left(x_t|x_s\right)p\left(y_t|x_t\right)$$

(2)

Thus, for $s$ in $S$ and $v$, $w$—which are neighbors of $s$ in the set of pixels but not neighbors in the Peano scan—we have

$$p\left(y_v,y_s,y_w|x_s\right)=p\left(y_s|x_s\right)p^{a\left(v,s\right)}\left(y_v|x_s\right)p^{a\left(w,s\right)}\left(y_w|x_s\right)$$

(3)

where $a(v,s)=h$ if $v$, $s$ are horizontal neighbors, and $a(v,s)=v$ if $v$, $s$ are vertical neighbors, and the same applies for $a(w,s)$. For example, considering ${(y_{14},y}_3,y_8)$ in Figure 4, we see that $3, 14$ are horizontal neighbors, while $3, 8$ are vertical neighbors. Then, we have

$$p\left(y_{14},y_3,y_8|x_3\right)=p\left(y_3|x_3\right)p^h\left(y_{14}|x_3\right)p^v\left(y_8|x_3\right).$$

(4)

Finally, for a given Peano scan, we associate with each pixel $s$ the two neighbors $v\left(s\right)$, $w\left(s\right)$ in the set of pixels, which are not its neighbors in the Peano scan. Numbering the Peano scan points as $(1,2,\dots ,N)$, the related contextual Peano scan is the sequence of triplets (they are couples for points on borders, and singletons on two corners where the scan begins and ends):

$$\left(\left[1,v\left(1\right),w\left(1\right)\right],\left[2,v\left(2\right),w\left(2\right)\right],\dots ,\left[N,v\left(N\right),w\left(N\right)\right]\right)$$

(5)

The classic hidden Markov chain associated with the Peano scan has the following distribution:

$$p(x^N,y^N)=p\left(x_1\right)p\left(x_2|x_1\right)\dots p\left(x_N|x_{N-1}\right)p\left(y_1|x_1\right)\dots p\left(y_N|x_N\right)$$

(6)

The new model we propose, called “hidden Markov chain for contextual Peano scan” (HMC-CPS), is defined as follows.

Let us consider

$$q(x^N,y^N)=p\left(x_1\right){\prod }_{n=1}^{N-1}p\left(x_{n+1}|x_n\right){\prod }_{n=1}^{N}p\left(y_n,y_{v\left(n\right)},y_{w\left(n\right)}|x_n\right)$$

(7)

It is to be noted that $q(x^N,y^N)$ is not the probability density of the pair $(X^N,Y^N)$. Nonetheless, considering the (unknown) normalizing constant

$$\mathsf{\kappa }={\displaystyle \frac{1}{\sum _{x^N}\int q(x^N,y^N)d{y}^N}}$$

we can consider the distribution

$$p(x^N,y^N)=\mathsf{\kappa }q(x^N,y^N)$$

(8)

Definition 1.

Let $S$ be a square set of pixels of dimensions ${N=2}^k\times 2^k$. Let $(1, 2, \dots , N)$ be a Peano scan (PS) of $S$, and let $(\left[1,v\left(1\right), w\left(1\right)\right], \left[2,v\left(2\right), w\left(2\right)\right],\dots , \left[N,v\left(N\right), w\left(N\right)\right])$ be the four nearest neighbors’ contextual PS (4NN-CPS) associated with PS. Then, the conditional probability distribution $p\left(x^N\right|y^N)$ given by the distribution $p(x^N, y^N)=\kappa q(x^N, y^N)$ defined with (3), (7), and (8) will be called the “hidden Markov chain for the contextual Peano scan” (HMC-CPS) distribution.

We note that, in HMC-CPS $(X^N,Y^N)$, the chain $X^N$ is Markovian and it is also Markovian conditional on $Y^N$, but $(X^N,Y^N)$ is not Markovian itself. As discussed in the next paragraph, $p\left(x_1\right|y^N)$ and transitions $p\left(x_{n+1}\right|{x_n,y}^N)$ are computable, which allow for Bayesian restorations. We note that $p\left(y^N\right|x^N)$, which is complicated to write, is not needed.

Remark 1.

It is possible to extend the 4NN-CPS with a richer neighborhood. Considering eight nearest neighbors in the set of pixels would lead to a contextual PS with six additional observations, except for boarding pixels, on each point in the Peano scan.

2.2. Bayesian MPM Segmentation with HMC-CPS

Let $1,\dots ,N$ be points of a Peano scan, and let $(X^N,Y^N)$ be a HMC-CPS (7) and (8). We consider the Bayesian Marginal Posterior Mode (MPM) to be defined by

$$\left[{\widehat{s}}_{MPM}\left(y^N\right)={\widehat{x}}^N\right]\iff \left[p\left({\widehat{x}}_n|y^N\right)=\underset{x_n\in \Omega }{\max }p\left(x_n|y^N\right)\right]$$

(9)

Thus, the problem lies in computing $p\left(x_n|y^N\right)$ for $n=1$, …, $N$. Let us recall the following general result.

Lemma 1.

Let $Z^N=(Z_1,\dots , Z_N)$ be a stochastic chain taking its values in a finite set $\Omega$. Then,

(i) 

$Z^N$ is Markovian if and only if there exist $N-1$ functions ${\phi }_1$, …, ${\phi }_{N-1}$ from ${\Omega }^2$ to $R^{+}$ such that

$$p\left(z^N\right)\propto \prod _{n=1}^{N-1}{\phi }_n\left(z_n,z_{n+1}\right)$$

(10)

 where $\propto$ means “proportional to”;

(ii) 

if (10) is verified, $p\left(z_1\right)$ and transitions $p\left(z_{n+1}|z_n\right)$ are given by the functions ${\phi }_1$, …, ${\phi }_{N-1}$ with

$$p\left(z_1\right)={\displaystyle \frac{{\beta }_1\left(z_1\right)}{\sum _{z_1}{\beta }_1\left(z_1\right)}};$$

$$\mathrm{f}\mathrm{o}\mathrm{r} 1<n<N, p\left(z_{n+1}|z_n\right)={\displaystyle \frac{{\phi }_n\left(z_n,z_{n+1}\right){\beta }_{n+1}\left(z_{n+1}\right)}{{\beta }_n\left(z_n\right)}},$$

(11)

where ${\beta }_1\left(z_2\right)$, …, ${\beta }_N\left(z_N\right)$ can be computed with the following backward recursion:

$${\beta }_N\left(z_N\right)=1,\mathrm{f}\mathrm{o}\mathrm{r} n=N-1,\dots ,1$$

$${\beta }_n\left(z_n\right)=\sum _{z_{n+1}}{\phi }_n\left(z_n,z_{n+1}\right){\beta }_{n+1}\left(z_{n+1}\right)$$

(12)

• Once $p\left(z_1\right)$ and $p\left(z_{n+1}|z_n\right)$ are given, each $p\left(z_n\right)$ is computed with forward recursion:

  $$for n=2,\dots ,N,p\left(z_n\right)=\sum _{z_n}p\left(z_n|z_{n-1}\right)p\left(z_{n-1}\right)$$

  (13)

Proof of Lemma 1.

• Let $Z^N$ be Markovian: $p\left(z_1, \dots , z_N\right)=p\left(z_1\right)p\left(z_2| z_1\right)p\left(z_3| z_2\right) \dots p\left(z_N| z_{N-1}\right)$. Then (10) is satisfied by ${\phi }_1\left(z_1, z_2\right)=p\left(z_1\right)p\left(z_2| z_1\right)$, ${\phi }_2\left(z_2, z_3\right)=p\left(z_3| z_2\right)$, …, ${\phi }_{N-1}\left(z_{N-1},z_N\right)=p\left(z_N| z_{N-1}\right)$.
• Conversely, let $p\left(z_1, \dots , z_N\right)$ satisfy (10). Thus, $p\left(z_1, \dots , z_N\right)=K{\phi }_1(z_1, z_2) \dots {\phi }_{N-1}(z_{N-1},z_N)$ with the $K$ constant, which implies that for each $n=1$, …, $N-1$, we have

  $$p\left(z_{n+1}|z_1, \dots , z_n\right)={\displaystyle \frac{p\left(z_1, \dots , z_n,z_{n+1}\right)}{p\left(z_1, \dots , z_n\right)}}={\displaystyle \frac{\sum _{\left(z_{n+2}, \dots , z_N,\right)}{\phi }_1\left(z_1, z_2\right){\dots \phi }_{N-1}\left(z_{N-1},z_N\right)}{\sum _{\left(z_{n+1}, \dots , z_N,\right)}{\phi }_1\left(z_1, z_2\right){\dots \phi }_{N-1}\left(z_{N-1},z_N\right)}}$$

  (14)

  which shows that $p\left(z_1, \dots , z_N\right)$ is Markovian.

Moreover, let us set

$$\mathrm{f}\mathrm{o}\mathrm{r} n=1,\dots ,N-1, {\beta }_n\left(z_n\right)={\sum }_{\left({z_{n+1},z}_{n+2}, \dots , z_N\right)}{{\phi }_n\left(z_n, z_{n+1}\right)\dots \phi }_{N-1}(z_{N-1},z_N)$$

On the one hand, we see that

$${\beta }_n\left(z_n\right)=\sum _{z_{n+1}}{\phi }_n(z_n, z_{n+1}){\beta }_{n+1}\left(z_{n+1}\right)$$

On the other hand, according to (14), we have

$$p\left(z_{n+1}|z_n\right)={\displaystyle \frac{{\phi }_n(z_n, z_{n+1}){\beta }_{n+1}\left(z_{n+1}\right) }{{\beta }_n\left(z_n\right)}}.$$

as

$$p\left(z_1\right)={\displaystyle \frac{{\beta }_1\left(z_1\right) }{\sum _{z_1}{\beta }_1\left(z_1\right)}}$$

(11) and (12) are verified, which ends the proof. □

To summarize, once the functions ${\phi }_1$, …, ${\phi }_{N-1}$ satisfying (10) are given, the distributions $p\left(z_1\right)$, $p\left(z_{n+1}|z_n\right)$, and $p\left(z_n\right)$ of the related Markov chain $Z^N=(Z_1,\dots ,Z_N)$ are computable. Thus, it is sufficient to define ${\phi }_1$, …, ${\phi }_{N-1}$ satisfying (10), which will highly simplify different model introductions in practice. Let us return to the conditional Markov chain for contextual Peano scan distribution $p{(x}^N\left|y^N\right)$, specified in Definition 1. We can say that $p{(x}^N\left|y^N\right)\propto q{(x}^N,y^N)$, and thus,

$$p{(x}^N\left|y^N\right)\propto \prod _{n=1}^{N-1}{\phi }_n\left(x_n,x_{n+1},y^N\right)$$

(15)

with

$$\begin{gathered} {\phi }_1\left(x_1,x_2,y^N\right)=p\left(x_1,x_2\right)p\left(y_1,y_{v\left(1\right)},y_{w\left(1\right)}|x_1\right)p\left(y_2,y_{v\left(2\right)},y_{w\left(2\right)}|x_2\right) \\ {\phi }_2\left(x_2,x_3,y^N\right)=p\left(x_3|x_2\right)p\left(y_3,y_{v\left(3\right)},y_{w\left(3\right)}|x_3\right) \\ \dots \\ {\phi }_{N-1}\left(x_{N-1},x_N,y^N\right)=p\left(x_N|x_{N-1}\right)p\left(y_N,y_{v\left(N\right)},y_{w\left(N\right)}|x_N\right) \end{gathered}$$

(16)

Finally, functions ${\phi }_1$, …, ${\phi }_{N-1}$ satisfying (10) are of the form

$$\begin{gathered} {\phi }_1\left(x_1,x_2,y^N\right)={\phi }_1{{(x}_1,x_2,y}_1,y_{v\left(1\right)},y_{w\left(1\right)},y_2,y_{v\left(2\right)},y_{w\left(2\right)}) \\ \begin{array}{c}{\phi }_2\left(x_2,x_3,y^N\right)={\phi }_2\left(x_2,x_3,y_3,y_{v\left(3\right)},y_{w\left(3\right)}\right) \end{array} \\ \dots \\ {\phi }_{N-1}\left(x_{N-1},x_N,y^N\right)={\phi }_{N-1}\left(x_{N-1},x_N,y_N,y_{v\left(N\right)},y_{w\left(N\right)}\right) \end{gathered}$$

(17)

and thus, they are easy to compute. Then, $p\left(x_1|y^N\right)$ and transitions $p\left(x_{n+1}|{x_n,y}^N\right)$ are computable following Lemma 1, which gives marginal distributions $p\left(x_n|y^N\right)$ of $p\left(x^N\right|y^N)$ and allows the use of MPM (9).

Remark 2.

The pair ${(X}^N, Y^N)$ has a complex and only partially known structure. In particular, neither $p\left(x^N\right)$ nor $p\left(y^N\right|x^N)$ is Markovian in general. In addition, for a Gaussian $p\left(y_n\right|x_n)$, $p\left(y^N\right|x^N)$ is not Gaussian in general. However, this is of little importance because what is important is that $p\left(x^N\right|y^N)$ is Markovian and known up to a constant, which makes $p\left(x_n|y^N\right)$ computable.

Example 2.

Let us specify ${\phi }_1$, ${\phi }_2$, …, ${\phi }_{15}$ used in Example 1. According to Figure 3, (3), and (15), we have

$$\begin{gathered} {\phi }_1\left(x_1,x_2,y^N\right)=p\left(x_1,x_2\right)p\left(y_1|x_1\right)p^v\left(y_4|x_1\right)p\left(y_2|x_2\right)p^h\left(y_{15}|x_2\right) \\ {\phi }_2\left(x_2,x_3,y^N\right)=p\left(x_3|x_2\right)p\left(y_3|x_3\right)p^v\left(y_8|x_3\right)p^h\left(y_{14}|x_3\right) \\ {\phi }_3\left(x_3,x_4,y^N\right)=p\left(x_4|x_3\right)p\left(y_4|x_4\right)p^v\left(y_1|x_4\right) \\ \dots \\ {\phi }_{13}\left(x_{13},x_{14},y^N\right)=p\left(x_{14}|x_{13}\right)p\left(y_{14}|x_{14}\right)p^v\left(y_9|x_{14}\right)p^h\left(y_3|x_{14}\right) \\ {\phi }_{14}\left(x_{14},x_{15},y^N\right)=p\left(x_{15}|x_{14}\right)p\left(y_{15}|x_{15}\right)p^h\left(y_2|x_{15}\right) \\ {\phi }_{15}\left(x_{15},x_{16},y^N\right)=p\left(x_{16}|x_{15}\right)p\left(y_{16}|x_{16}\right)p^v\left(y_{13}|x_{16}\right) \end{gathered}$$

(18)

2.3. Parameter Estimation

Let us suppose that $p(x^N,y^N)$ is a classic hidden Markov chain (CHMC) distribution, with a Gaussian $p\left(y^N\right|x^N)$ and two different transitions depending on whether it applies to horizontal neighbors or vertical neighbors in the original image. For $K$ classes $\Omega =\left\{1,\dots ,K\right\}$, the parameters are as follows: $K^2$ probabilities $p^h={\left(p_{ij}^h\right)}_{1\le i,j\le K}$, with $p_{ij}^h=p\left(x_{t+1}=j,x_t=i\right)$ for $t,t+1$ neighbors in the chain and horizontal neighbors in the image, $K^2$ probabilities ${p^v=\left(p_{ij}^v\right)}_{1\le i,j\le K}$, with $p_{ij}^v=p\left(x_{t+1}=j,x_t=i\right)$ for $t,t+1$ neighbors in the chain and vertical neighbors in the image, and $K$ means $m={\left(m_i\right)}_{1\le i\le K}$ and $K$ variances ${{\sigma }^2=\left({\sigma }_i^2\right)}_{1\le i\le K}$ of the $K$ Gaussian distributions $({p\left(y_s|x_{\mathrm{s}}=i\right))}_{1\le i\le K}$. Let us consider the HMC-CPS: by choosing $p^h\left(x_t|x_s\right)$ and $p^v\left(x_t|x_s\right)$ intervening in $p\left(y_u,y_s,y_w|x_s\right)$ to be those related to $p_{ij}^h$ and $p_{ij}^v$, respectively, the new proposed model uses exactly the same parameters as the CHMC. Thus, the problem is to estimate $\theta =(p^h,p^v,m,{\sigma }^2)$ from the observed image $Y^N=y^N$ alone. However, contrary to the CHMC model, the EM algorithm cannot be used for estimating the parameters. Indeed, according to (7) and (8), the joint likelihood $p{(x}^N,y^N,\mathsf{\theta })$ is only computable up to a constant $\kappa$, which depends on $\theta$. As such, it is not possible in general to compute $\underset{\theta }{\mathrm{argmax}}E\left[\mathrm{l}\mathrm{o}\mathrm{g}\left(p(x^N,y^N,\mathsf{\theta })\right)|y^N,{\theta }^q\right]$ for each iteration $q$.

Nonetheless, as $p\left(x^N\right|y^N,{\theta }^q)$ is computable and, being Markovian, it is possible to sample from it, we can use stochastic EM (SEM), which is performed as follows.

• Initialize the parameters ${\theta }^0=(p^{h,0},p^{v,0},m^0,{\sigma }^{2,0})$ with some simple method;
• Compute ${\theta }^{q+1}=(p^{h,q+1},p^{v,q+1},m^{q+1},{\sigma }^{2,q+1})$ from the current ${\theta }^q=(p^{h,q},p^{v,q},m^q,{\sigma }^{2,q})$ and $y^N$;

-

Sample $x^{N,q+1}=(x_1^{q+1},{\dots ,x}_N^{q+1})$ according to the Markov distribution $p\left(x^N\right|y^N,{\theta }^q)$;

-

Let $H^{q+1}$ be the set of couples $(n,n+1)$, with $n$ and $n+1$ horizontal neighbors in the set of pixels, and let $V^{q+1}$ be the set of couples $(n,n+1)$, with $n$ and $n+1$ vertical neighbors in the set of pixels. Let $S^{i,q+1}$ be the set of points $n$ such that $x_n^{q+1}=i$. Set

$$p_{ij}^{h,q+1}={\displaystyle \frac{\sum _{\left(n,n+1\right)\in H^{q+1}}{1}_{\left[x_n^{q+1}=i,x_{n+1}^{q+1}=j\right]}}{\left|H^{q+1}\right|}}, \mathrm{s}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{l}\mathrm{a}\mathrm{r} \mathrm{f}\mathrm{o}\mathrm{r} p_{ij}^{v,q+1}$$

(19)

$$m_i^{q+1}={\displaystyle \frac{{\sum }_{n\in S^{i,q+1}}{y}_n}{\left|S^{i,q+1}\right|}}, {\sigma }_{i,q+1}^2={\displaystyle \frac{{\sum }_{n\in S^{i,q+1}}{\left(y_n-m_i^{q+1}\right)}^2}{\left|S^{i,q+1}\right|}}$$

(20)

A stopping criterion is adapted to each case considered. Indeed, the Markov chain $\left({\theta }^q\right)$ obtained in this way is very complex and its stabilization depends on different factors, particularly on the noise level. To the best of our knowledge, as in the EM case, there are no general theoretical results specifying its asymptotic behavior.

Remark 3.

One can use a simpler approximated SEM by treating $p(x^N, y^N)$ as a CHMC and sampling from its posterior law $p^{CHMC}\left(x^N\right|y^N, {\theta }^q)$ instead of the real $p\left(x^N\right|y^N, {\theta }^q)$; we programmed it and it produces slightly worse results.

3. Contextual Peano Scan and Hidden Evidential Markov Chains

3.1. MPM Restoration with Hidden Triplet Markov Chains

Let us briefly examine particular triplet Markov chains, named hidden triplet Markov chains. Let $X^N=(X_1,\dots ,X_N)$ be the stochastic sequence of classes, $Y^N=(Y_1,\dots ,Y_N)$ the sequence of observations on the pixels, and $U^N=(U_1,\dots ,U_N)$ a third stochastic sequence, with each $U_n$ taking its values in $\Lambda =\left\{1,\dots ,L\right\}$. Let $T^N=(T_1,\dots ,T_N)$, with $T_n=(X_n, U_n, Y_n)$, be a triplet Markov chain. Then, $X^N$ can be estimated from $Y^N$ like in the classic HMC case. As $U^N$ is arbitrary, the family of TMCs is a very general one. As $p\left(t^N\right)$ is written $p\left(t^N\right)=p\left(t_1\right)p\left(t_2|t_1\right)\dots p\left(t_N|t_{N-1}\right)$, it has the form

$$p\left(t^N\right)={\phi }_1({x_1,x_2,u}_1,u_2,y_1,y_2)\dots {\phi }_{N-1}({x_{N-1},x_N,u}_{N-1},u_N,y_{N-1},y_N)$$

(21)

with

$$\begin{gathered} {\phi }_1({x_1,x_2,u}_1,u_2,y_1,y_2)=p(t_1,t_2) \\ {\phi }_2({x_2,x_3,u}_2,u_3,y_2,y_3)=p\left(t_3\right|t_2) \\ \dots \\ {\phi }_{N-1}({x_{N-1},x_N,u}_{N-1},u_N,y_{N-1},y_N)=p\left(t_N\right|t_{N-1}) \end{gathered}$$

(22)

Then, applying Lemma 1 to $Z^N=(X^N, U^N)$ and $p\left(z^N\right|y^N)$, we can compute $p\left(z_n|y^N\right)=p\left(x_n, u_n|y^N\right)$ for each $n=1, \dots , N$, which finally gives

$$p\left(x_n|y^N\right)={\sum }_{u_n\in \Omega }p\left(x_n, u_n|y^N\right)$$

(23)

Allowing the application of Bayesian MPM segmentation (9). Throughout this article, we will consider a particular TMC, satisfying

$$p\left(t_1\right)=p\left(x_1,u_1\right)p\left(y_1|x_1\right)$$

$$p\left(t_{n+1}|t_n\right)=p\left(x_{n+1},u_{n+1}|x_n,u_n\right)p\left(y_{n+1}|x_{n+1}\right)$$

(24)

Remark 4.

We may observe that (24) is a simplified TMC, obtained by assuming $p\left(x_{n+1},u_{n+1}|x_n,u_n, y_n\right)=p\left(x_{n+1},u_{n+1}|x_n,u_n\right)$, and $p\left(y_{n+1}|x_n,u_n, y_n, x_{n+1},u_{n+1}\right)=p\left(y_{n+1}|x_{n+1}\right)$. Setting $W_n=(X_n,U_n)$, we can see that $T^N=(W^N, Y^N)$ has the structure of a very classical HMC: $W^N$ is Markovian and $p\left(y_n|w^N\right)=p\left(y_n|w_n\right)$. In addition, it satisfies $p\left(y_n|w_n\right)=p\left(y_n|x_n,u_n\right)=p\left(y_n|x_n\right)$. This shows that such TMCs are not much different from classical HMCs, so that different computer programs related to HMCs can be adapted to these TMCs with slight modifications.

Remark 5.

Let us note the great generality of triplet Markov chains, even in their particular form (24). In particular, ${(X}^N, U^N)$ is Markovian, but $X^N$ is not necessarily Markovian. Moreover, in TMCs, $U^N$ is free, and the only condition is that each $U_n$ takes its values in a finite not-too-large set. In particular, it can be multivariate: $U_n=(U_n^1, \dots , U_n^I)$, with each $U_n^i$ modeling a particular aspect of the problem dealt with.

3.2. Models Combining Contextual Peano Scan with Triplet Markov Chains

Let $S$ be a square set of pixels of dimensions ${N=2}^k\times 2^k$. Let $(1,2,\dots ,N)$ be a Peano scan (PS) of $S$, and let $(\left[1,v\left(1\right),w\left(1\right)\right],\left[2,v\left(2\right),w\left(2\right)\right],\dots ,\left[N,v\left(N\right),w\left(N\right)\right])$ be the four nearest neighbors’ contextual PS (4NN-CPS) associated with PS. Let us consider a TMC of the form (21). We limit our presentation to particular TMCs ${(X}^N,{U^N,Y}^N)$ satisfying (24). Setting $Z_n=\left(X_n,U_n\right)$ for $n=1,\dots ,\mathrm{N}$, $T^N=(Z^N,Y^N)$ is a classic HMC, and thus, in a similar manner to Section 2.1, we can extend this model to contextual Peano scan. We define the “conditional triplet Markov chain for contextual Peano scan” (CTMC-CPS) distribution by setting

$$p^h\left(y_t|z_s\right)=\sum _{x_t,u_t}{p}^h\left(x_t,u_t|z_s\right)p\left(y_t|x_t\right)$$

(25)

if $s$, $t$ are horizontal neighbors. Furthermore,

$$p^v\left(y_t|z_s\right)=\sum _{x_t,u_t}{p}^v\left(x_t,u_t|z_s\right)p\left(y_t|x_t\right)$$

(26)

if they are vertical neighbors. Then, using Definition 1, we obtain

$$p\left(y_v,y_s,y_w|z_s\right)=p\left(y_s|z_s\right)p^{a\left(v,s\right)}\left(y_v|z_s\right)p^{a\left(w,s\right)}\left(y_w|z_s\right)$$

(27)

with $a(v,s)=h$ if $v$, $s$ are horizontal neighbors, and $a(v,s)=v$ if $v$, $s$ are vertical neighbors, and finally, we obtain the CTMC-CPS distribution:

$$p{(z}^N,y^N)=\mathsf{\kappa }p\left(z_1\right){\prod }_{n=1}^{N-1}p\left(z_{n+1}|z_n\right){\prod }_{n=1}^{N}p\left(y_n,y_{v\left(n\right)},y_{w\left(n\right)}|z_n\right)=\mathsf{\kappa }q{(z}^N,y^N)$$

(28)

with $v\left(n\right)$, $w\left(n\right)$ being the two neighbors of pixel $n$ in the set of pixels, which are not its neighbors in the Peano scan.

Similarly to Section 2.1, $p(z^N,y^N)$ is not Markovian here; thus, $\mathsf{\kappa }$ is not computable in general. However, with the distribution $p(z^N\left|y^N\right)=p(x^N,u^N\left|y^N\right)$ being Markovian, $p(x_n,u_n|y^N)$ is computable, and thus, $p(x_n\left|y^N\right)$ is also computable by applying (23). Thus, the problem is similar to that in Section 2.2; the only difference is that, here, one considers ${(x}_n,u_n)$ instead of $x_n$ and, once $p(x_n,u_n\left|y^N\right)$ is computed, one applies (23) to compute $p(x_n\left|y^N\right)$ and to apply MPM. For parameter estimation, the SEM from Section 2.3 can be used, with $p(z^N,y^N)$ instead of $p(x^N,y^N)$. For $\Omega =\left\{1,\dots ,K\right\}$ and $\Lambda =\left\{1,\dots ,L\right\}$, the parameters are as follows: $K^2{L}^2$ probabilities $p^h={\left(p_{ijkl}^h\right)}_{\begin{array}{c}1\le i, j\le K\\ 1\le k, l\le L\end{array}}$, with $p_{ijkl}^h=p\left(x_{t+1}=j, u_{t+1}=l,x_t=i, u_t=k\right)$ for $t,t+1$ neighbors in the chain and horizontal neighbors in the image, $K^2{L}^2$ probabilities ${p^v=\left(p_{ijkl}^v\right)}_{\begin{array}{c}1\le i, j\le K\\ 1\le k, l\le L\end{array}}$, with $p_{ijkl}^v=p\left(x_{t+1}=j, u_{t+1}=l,x_t=i, u_t=k\right)$ for $t,t+1$ neighbors in the chain and vertical neighbors in the image, and $K$ means $m={\left(m_i\right)}_{1\le i\le K}$ and $K$ variances ${{\sigma }^2=\left({\sigma }_i^2\right)}_{1\le i\le K}$ of the $K$ Gaussian distributions $({p\left(y_s|x_{\mathrm{s}}=i\right))}_{1\le i\le K}$.

3.3. Hidden Evidential Markov Chain for Contextual Peano Scan

Hidden evidential Markov chains (HEMCs) are particular triplet Markov chains $(X^N,{U^N,Y}^N)$ satisfying (24), in which each $U_n$ takes its values in $2^{\Omega }$, the power set of $\Omega$. To define $p\left(t^N\right)$, we first define $m\left(u^N\right)$, which is a “basic belief assignment” (bba) $m$ defined on ${\left(2^{\Omega }\right)}^N$ with

$$m\left(u_1,\dots ,u_N\right)=m\left(u_1\right)\prod _{i=1}^{N-1}m\left(u_{n+1}|u_n\right)$$

(29)

and null on $2^{{(\Omega }^{\mathrm{N}})}-{{(2}^{\Omega })}^N$. This simply means that, for each $n=1,\dots ,N-1$ and $u_nϵ2^{\Omega }$, $u_{n+1}\to m\left(u_{n+1}|u_n\right)$ is a bba on $2^{\Omega }$. To simplify, a bba $m$ on $2^{\Omega }$ in this article will be seen as a probability satisfying $m\left(\varnothing \right)=0$.

Having (29), the evidential Markov chain (EMC) distribution is the Markovian distribution $p(x^N,u^N)$ obtained with normalization, i.e., with its division by $\sum _{\left(x^N,u^N\right)}\phi \left(x^N,u^N\right)$, of the function $\phi$ defined in ${{{\Omega }^N\times (2}^{\Omega })}^N$ with

$$\phi \left(x^N,u^N\right)={\phi }_1\left(x_1,u_1,x_2,u_2\right)\prod _{n=2}^{N-1}{\phi }_n\left(u_n,x_{n+1},u_{n+1}\right)$$

(30)

where

$${\phi }_1\left(x_1,u_1,x_2,u_2\right)=1_{\left[x_1\in u_1,x_2\in u_2\right]}m\left(u_1\right)m\left(u_2|u_1\right)$$

(31)

$${\phi }_n\left(u_n,x_{n+1},u_{n+1}\right)=1_{\left[x_{n+1}\in u_{n+1}\right]}m\left(u_{n+1}|u_n\right), \mathrm{f}\mathrm{o}\mathrm{r} n=2,\dots ,N-1$$

(32)

Setting $Card\left(u_n\right)=\left|u_{\mathrm{n}}\right|$ and applying Lemma 1, we find, after calculation,

$$p\left(x_1, u_1\right)={\displaystyle \frac{\left|u_1\right|m\left(u_1\right){\sum }_{\left(x_2,u_2\right)}{\phi }_1\left(x_1,u_1,x_2,u_2\right){\beta }_2^{\ast }\left(u_2\right)}{\sum _{u_1}\left|u_1\right|m\left(u_1\right){\sum }_{\left(x_2,u_2\right)}{\phi }_1\left(x_1,u_1,x_2,u_2\right){\beta }_2^{\ast }\left(u_2\right)}}=p\left(u_1\right)p(x_1|u_1)$$

(33)

$$p\left(x_{n+1},u_{n+1}|x_n,u_n\right)=p\left(x_{n+1},u_{n+1}|u_n\right)=p\left(u_{n+1}|u_n\right)p\left(x_{n+1}|u_{n+1}\right)$$

with

$$p\left(x_n|u_n\right)={\displaystyle \frac{1_{\left[x_n\in u_n\right]}}{\left|u_n\right|}}, \mathrm{f}\mathrm{o}\mathrm{r} n=1,\dots ,N$$

(34)

$$for n=1,\dots ,N-1,p\left(u_{n+1}|u_n\right)={\displaystyle \frac{\left|u_{\mathrm{n}+1}\right|m\left(u_{n+1}|u_n\right){\beta }_{n+1}^{\ast }\left(u_{n+1}\right)}{\sum _{u_{n+1}}\left|u_{\mathrm{n}+1}\right|m\left(u_{n+1}|u_n\right){\beta }_{n+1}^{\ast }\left(u_{n+1}\right)}}$$

(35)

where ${\beta }_N^{\ast }\left(u_N\right)$, …., ${\beta }_2^{\ast }\left(u_2\right)$ are computed with backward induction:

$${\beta }_N^{\ast }\left(u_N\right)=1$$

$$for n=2,\dots ,N-1,{\beta }_n^{\ast }\left(u_n\right)={\sum }_{\left(x_{n+1},u_{n+1}\right)}{\phi }_n\left(u_n,x_{n+1},u_{n+1}\right){\beta }_{n+1}^{\ast }\left(u_{n+1}\right)$$

(36)

Having EMC $p(x^N,u^N)$, we define hidden EMC (HEMC) $p(x^N,u^N,y^N)$ with

$$p\left(x^N,u^N,y^N\right)=p\left(u_1\right)p\left(x_1|u_1\right)p\left(y_1|x_1\right){\prod }_{n=1}^{N-1}p\left(u_{n+1}|u_n\right)p\left(x_{n+1}|u_{n+1}\right)p\left(y_{n+1}|x_{n+1}\right)$$

(37)

with

$$p\left(x_n|u_n\right)={\displaystyle \frac{1_{\left[x_n\in u_n\right]}}{\left|u_n\right|}} \mathrm{f}\mathrm{o}\mathrm{r} n=1,\dots ,N$$

(38)

As HEMCs satisfy (24), to construct the hidden evidential Markov chain for the contextual Peano scan (HEMC-CPS), setting $Z_n=\left(X_n,U_n\right)$, we can use (25) and (26), which become

$$p^h\left(y_t|u_s\right)=\sum _{x_t,u_t}{p}^h\left(u_t|u_s\right)p\left(x_t|u_t\right)p\left(y_t|x_t\right)$$

(39)

if $s$, $t$ are horizontal neighbors;

$$p^v\left(y_t|u_s\right)=\sum _{x_t,u_t}{p}^v\left(u_t|u_s\right)p\left(x_t|u_t\right)p\left(y_t|x_t\right)$$

(40)

if they are vertical neighbors. Then, using Definition 1, we obtain

$$p\left(y_v,y_s,y_w|z_s\right)=p\left(y_s|x_s\right)p^{a\left(v,s\right)}\left(y_v|u_s\right)p^{a\left(w,s\right)}\left(y_w|u_s\right)$$

(41)

Finally, this leads to the HEMC-CPS distribution:

$$\begin{gathered} p{(z}^N,y^N)=\mathsf{\kappa }p\left(x_1\right)p\left(u_1|x_1\right) \\ {\prod }_{n=1}^{N-1}p\left(u_{n+1}|u_n\right)p\left(x_{n+1}|u_{n+1}\right){\prod }_{n=1}^{N}p\left(y_n,y_{v\left(n\right)},y_{w\left(n\right)}|z_n\right) \end{gathered}$$

(42)

with $v\left(n\right)$, $w\left(n\right)$, the two neighbors of pixel $n$ in the set of pixels, which are not its neighbors in the Peano scan and $p\left(x_n|u_n\right)$ satisfying (38) for $n=1,\dots ,N$.

In practice, and in the experiments below, one often uses the following simple bba:

$$m\left(u_{n+1}|u_n\right)=0 \mathrm{f}\mathrm{o}\mathrm{r} u_{n+1} \mathrm{o}\mathrm{u}\mathrm{t}\mathrm{s}\mathrm{i}\mathrm{d}\mathrm{e} \mathrm{o}\mathrm{f} \left\{\left\{1\right\}, \left\{2\right\}, \dots ,\left\{K\right\}, \left\{1, \dots , K\right\}\right\}$$

(43)

Thus, such a hidden evidential Markov chain is a light extension of the classic HMC, as we find the latter again for $m\left(u_{n+1}=\left\{1, \dots , K\right\}|u_n\right)=0$ for each $n$ and each $u_n$. For $\Omega =\left\{1,\dots ,K\right\}$ and $m\left(u_{n+1}|u_n\right)$ satisfying (43), the parameters are as follows: ${\left(K+1\right)}^2$ probabilities $p^h={\left(p_{kl}^h\right)}_{1\le k,l\le K+1}$, with $p_{kl}^h=p\left(u_{t+1}=l, u_t=k\right)$ for $t,t+1$ neighbors in the chain and horizontal neighbors in the image; ${\left(K+1\right)}^2$ probabilities $p^v={\left(p_{kl}^v\right)}_{1\le k,l\le K+1}$, with $p_{kl}^v=p\left(u_{t+1}=l, u_t=k\right)$ for $t,t+1$ neighbors in the chain and vertical neighbors in the image; and $K$ means $m={\left(m_i\right)}_{1\le i\le K}$ and $K$ variances ${{\sigma }^2=\left({\sigma }_i^2\right)}_{1\le i\le K}$ of the $K$ Gaussian distributions $({p\left(y_s|x_{\mathrm{s}}=i\right))}_{1\le i\le K}$. Let us note that $p\left(x_n|u_n\right)$ is not a parameter, as it is not free and must satisfy (38). Finally, these parameters can be estimated using the SEM from Section 2.3 by replacing (19) with

$$p_{kl}^{h,q+1}={\displaystyle \frac{\sum _{\left(n,n+1\right)\in H^{q+1}}{\sum }_{i\in \Omega }{\sum }_{j\in \Omega }{1}_{\left[u_n^{q+1}=k,x_n^{q+1}=i,x_{n+1}^{q+1}=j,u_{n+1}^{q+1}=l\right]}}{\left|H^{q+1}\right|}}, \mathrm{s}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{l}\mathrm{a}\mathrm{r} \mathrm{f}\mathrm{o}\mathrm{r} p_{kl}^{v,q+1}$$

(44)

Remark 6.

Let us briefly specify the motivation for introducing theory of evidence (TE) in an image segmentation context. For the case of two classes $\left\{1, 2\right\}$, we have noticed—see [28,40], and references therein—that, in small-sized areas, the a posteriori law of the evidential variable loads $p(u_{n+1}=\left\{\mathrm{1,2}\right\}|u_n, y^N)$ at the expense of $p(u_{n+1}=\left\{1\right\}|u_n, y^N)$ and $p(u_{n+1}=\left\{2\right\}|u_n, y^N)$, that has the effect of reducing the role of context, This reduction appears to be beneficial for segmentation. Similarly, in large-sized areas, it loads $p(u_{n+1}=\left\{1\right\}|u_n, y^N)$ and $p(u_{n+1}=\left\{2\right\}|u_n, y^N)$ to the detriment of $p(u_{n+1}=\left\{\mathrm{1,2}\right\}|u_n, y^N)$, which also improves segmentation. In cases where both types of area exist in an image, this mechanism improves (sometimes significantly) the segmentation obtained with the classic Markovian model, without the third evidential variable, which treats all areas in the same way. It is worth noting that the value of DS fusion in Markovian segmentation in a non-stationary data case has been found empirically, and we have no theoretical justification to propose at the moment. Why classical parameter estimation in evidential Markov models, which are classical triplet Markov models, provides parameters allowing for the double property mentioned above is not clear. Searching for theoretical justifications seems to be a challenging problem.

4. Experiments

4.1. Segmentation of Synthetic Images

This section shows some results of hand-drawn noisy images subjected to unsupervised segmentation based on the two models presented in this study, as well as on an HMC and on an HMF for reference. We estimate the parameters using the SEM algorithms from Section 2.3 and Section 3.3, with initializations obtained from a k-means algorithm run on the observed data. HMF parameters are estimated with the Gibbsian EM algorithm, with its initialization also obtained from a k-means algorithm. According to the results presented in Figure 5, we can propose the following.

The main conclusion is that in the frame of classic hidden Markov chains, the contextual Peano scan-based method is systematically and significantly more efficient than the classic Peano scan method. The relative average gain is about 16%. Another important conclusion is that in some, though not all, cases, the efficiency of the CPS-based method is comparable to that of the hidden Markov field-based method. This is the case in the “Zebra”, “Zebra with target”, or “Spaghetti” images. In these three cases, Markov field-based segmentation appears to be worse.

The value of extending hidden Markov to evidential Markov is less striking. However, it can significantly improve the results obtained with hidden Markov chains in the case of images containing simultaneously large areas and fine details. This means that extending HMC-CPS to HEMC-CPS should improve the model in the same fashion. This is confirmed through the experiments, particularly for “Tree” and “Zebra with letter”, for which HEMC-CPS does indeed improve HMC-CPS, while being inferior to HMF. The same occurs for “Digital” and “Nazca” where HEMC-CPS outperforms both HMC-CPS and HMF. It should also be noted that HEMC-CPS outperforms the hidden evidential Markov field model (HEMF), presented in [40], for the “Nazca” image (HEMF scores 6.7% of error, and HEMC-CPS 5.4%). Moreover, if we look at the “Spaghetti” and “Squares” images, we can see something interesting: When considering error rates, HMF is the best model, followed by HMC-CPS and then HEMC-CPS. However, some parts of the images (like the spaghetti shapes in “Spaghetti” and the three fine squares that are numbered 4,5,6 from the center of the image “Squares”) are segmented with more details using HEMC-CPS, while HMC-CPS seems to miss some of the details and HMF misses even more. Finally, in theory, HEMC-CPS is strictly more general than HMC-CPS; however, in practice, it sometimes performs worse than HMC-CPS, as we can see in “Zebra” and “Zebra with target”. Of course, parameter estimation is more difficult in HEMC-CPS than in HMC-CPS, which is a possible explanation.

4.2. Segmentation of Real Images

We chose two real grayscale images and segmented them using the same four methods: HMC-PS, HMC-CPS, HEMC-CPS, and HMF. From the results presented in Figure 6, we can make several observations:

For “Fractured Bones”, while HMF has a more satisfying segmentation of the bones, it is not able to segment the fractured zone, while this zone appears in HMC-CPS segmentation. Both HEMC-CPS and HMC-PS perform quite badly on this image.

For “Tree in the field”, if we look at the segmentation of the fine tree branches, HMC-CPS segmentation seems the least detailed. HMC-PS seems more detailed than HMC-CPS, but less than HMF and HEMC-CPS. Moreover, it is very difficult to decide between the two latter models. In particular, both segmentations of fine branches seem almost identical. This is encouraging as it demonstrates two points: first, HEMC-CPS can be useful with respect to HMC-CPS in real situations; second, in the same kind of real situations, HEMC-CPS-based segmentation can be as efficient as HMF-based segmentation.

Finally, we will discuss the computational complexity of the different methods considered. MPM restoration in HMC and HEMC has a time complexity in $O\left(N\right)$, where $N$ is the length of the sequence, if we make the assumption that the number of hidden states is negligible compared to $N$. Under the same assumption, MPM in HMC-CPS and HEMC-CPS also has a time complexity in $O\left(N\right)$ if the neighborhood considered is not too large. For the parameter estimation with SEM, the time complexity for one iteration of the algorithm is also $O\left(N\right)$ for HMC, HEMC, HMC-CPS, and HEMC-CPS. If we compare with HMF, the exact MPM time complexity is not calculable. However, one full iteration of Gibbs sampling has a time complexity of $O\left(N\right)$. Considering that for approximate MPM, we need several simulations, and each simulation is obtained by running several iterations of Gibbs sampling, the total number of Gibbs iterations becomes non-negligible in front of $N$ for a good MPM approximation. The same can be said for the Gibbsian EM algorithm in which for each iteration, we need several simulations, each of them requiring a number of Gibbs sampling iterations. In the experiments, this made HMF-based MPM about a hundred times longer than HMC-CPS- and HEMC-CPS-based ones.

To conclude, we only identified one case where the extension from Markov to evidential Markov is relevant, which is the segmentation of images combining fine details (about one pixel wide) and low noise level, but finding real-world applications presenting these particular properties is challenging. Nonetheless, we showed that both HMC-CPS and HEMC-CPS can be useful as they both improve upon HMC, while being a faster and still-relevant alternative to HMF-based MPM, even possibly in cases of real images. Furthermore, other cases where HEMC-CPS significantly outperforms HMC-CPS may remain to be discovered.

5. Conclusions and Perspectives

Using hidden Markov chains to design fast unsupervised image segmentation methods via the Peano scan (PS) is feasible. In this article, we investigated how the introduction of the “contextual” Peano scan (CPS) allows for the improvement of classical methods. According to the detailed experiments, introducing the CPS can significantly improve Bayesian unsupervised segmentations, with the average improvement measured in the error ratio being about 15%. Furthermore, in some situations, the quality of CPS-based segmentations can approach that of classic hidden Markov field-based ones, while being much faster. Moreover, hidden Markov chains are generalizable to pairwise and triplet Markov ones, and classic PS-based methods are generalizable to more sophisticated models; thus, it is the same for CPS-based models. We tested one of these models based on the theory of evidence. General conclusions are more difficult to draw; however, its value becomes apparent in particular situations, such as when both large- and small-sized areas are present in the hidden class image. The efficiency of MPM based on such evidential models can even match that of MPM based on HMFs in the case of real images, while being much faster.

In this study, we considered non-causal inferences. For $1<n<N$, we obtained $X_n$, and $Y_n^{\ast }={(Y}_n,Y_n^{\mathrm{\prime }})$, with $Y_n^{\mathrm{\prime }}=(Y_{v_n},Y_{w_n})$, where $v_n$, $w_n$ are pixel neighbors of $n$, albeit outside of the scan. Stochastic inferences in such models are illustrated by the dependency graph shown in Figure 7a. However, similar models remain valid for causal inference. The corresponding graph is the directed graph shown in Figure 7b. Thus, the general structure of the family of models considered is suitable for both causal and non-causal situations.

In terms of research perspectives, let us note that the flexibility of the Peano scan allows for its use in 3D images [41] and sequences of images [18], which are significantly more complex than the fixed images considered in this study, and in which using HMF techniques is very time-consuming. Thus, the proposed CPS-based techniques are likely to be of interest in such complex situations.

Finally, the usefulness of the models presented here are not limited to image segmentation. They can be applied to the processing of any spatially correlated multi-dimensional data, once a scan linking the variables under consideration has been established. There are numerous “multi-dimensional space-filling curves”, which are scans mapping multi-dimensional space to a one-dimensional domain [42]. For example, the models considered in this paper would be directly applicable to the multi-dimensional data classification problem, once such a curve, and a neighborhood system, chosen.