A New Hybrid Possibilistic-Probabilistic Decision-Making Scheme for Classification

Abstract

Uncertainty is at the heart of decision-making processes in most real-world applications. Uncertainty can be broadly categorized into two types: aleatory and epistemic. Aleatory uncertainty describes the variability in the physical system where sensors provide information (hard) of a probabilistic type. Epistemic uncertainty appears when the information is incomplete or vague such as judgments or human expert appreciations in linguistic form. Linguistic information (soft) typically introduces a possibilistic type of uncertainty. This paper is concerned with the problem of classification where the available information, concerning the observed features, may be of a probabilistic nature for some features, and of a possibilistic nature for some others. In this configuration, most encountered studies transform one of the two information types into the other form, and then apply either classical Bayesian-based or possibilistic-based decision-making criteria. In this paper, a new hybrid decision-making scheme is proposed for classification when hard and soft information sources are present. A new Possibilistic Maximum Likelihood (PML) criterion is introduced to improve classification rates compared to a classical approach using only information from hard sources. The proposed PML allows to jointly exploit both probabilistic and possibilistic sources within the same probabilistic decision-making framework, without imposing to convert the possibilistic sources into probabilistic ones, and vice versa.

1. Introduction

Uncertainty can be categorized into two main kinds [1]: aleatory or randomness uncertainty, aka statistical uncertainty, due to the variability or the natural randomness in a process and epistemic uncertainty, aka systematic uncertainty, which is the scientific uncertainty in the model of the process. It is due to limited data and knowledge. Epistemic uncertainty calls for alternative methods of representation, propagation, and interpretation of uncertainty than just probability. Since the beginning of the 60 s, following fruitful cross-fertilization, a convergence is emerging between physics, engineering, mathematics, and the cognitive sciences to provide new techniques and models that shows a trend of inspiration from human brain mechanism towards a unified theory to represent knowledge, belief and uncertainty [2,3,4,5,6,7,8,9].

Uncertainty is a natural and unavoidable part in real-world applications. When observing a “real-world situation”, decision making is the process of selecting among several alternatives or decisions.

The problem here is to assign a label or a class to measurements or other types of observations (data) from sensors or other sources to which the observations are assumed to belong. This is a typical classification process.

As shown in Figure 1, the general classification process can be formulated as follows. An input set of observations o ($\mathit{o}\in \mathsf{\Psi }$) is “observed” using a sensor (or a set of sensors) delivering a feature vector $\mathit{x}\in \mathsf{\Theta }$ ($\mathsf{\Theta }$ is called the features set). This feature vector x is then injected into the decision-making system or labelling system in order to recognize the most likely decision (hypothesis, alternative, class) from a given exhaustive set $\mathsf{\Omega }=\left\{C_m. m=1,\dots ,M\right\}$ of M exclusive decisions [10].

Figure 1. Structure of a multisource classification system (Source: [11]).

The development of both classification algorithms and decision-making criteria are governed by several factors mainly depending on the nature of the feature vector, the nature of the imperfection attached to the observed features as well as the available knowledge characterizing each decision. Several global constraints also drive the conception of the global classification process: the “physical” nature and quality of the measures delivered by the sensors, the categories discrimination capacity of the computed features, the nature and the quality of the available knowledge used for the development of the decision-making system.

However, in much of the literature, the decision-making system is performed by the application of two successive functionalities: the soft labeling and the hard decision (selection) functionalities. The labeling functionality [12] uses the available a priori knowledge in order to perform a mapping $\ell$ between the features set $\mathsf{\Theta }$ and the decisions set $\mathsf{\Omega } \left(\mathit{\ell }: \mathsf{\Theta }⟶\mathsf{\Omega }\right)$. For each feature vector $x\in \mathsf{\Theta }$, a soft decision label vector $\mathit{\ell }\left(x\right)=\left[{\mathit{\ell }}_{C_1}\left(x\right),\dots ,{\mathit{\ell }}_{C_m}\left(x\right),\dots ,{\mathit{\ell }}_M\left(x\right) \right]\in {\left[0,1\right]}^M$ is determined in the light of the available knowledge where ${\ell }_{C_M}\left(x\right)$ measures the degree of belief or support, that we have in the occurrence of the decision $C_m$. For instance, if the available knowledge allows probabilistic computations, the soft decision label vector is given through ${\mathit{\ell }}_{C_M}\left(x\right)=Pr\left\{C_M|x\right\}$ [13] where $Pr\left\{C_M|x\right\}$ represents the a posteriori probability of the decision $C_M$ given the observed feature vector $x\in \mathsf{\Theta }$ [14]. When the available knowledge is expressed in terms of ambiguous information, the possibility theory formalism (developed by L. Zadeh [7] and D. Dubois et al. [15,16,17]) can be used. The soft decision label vector $\mathit{\ell }\left(x\right)$ is then expressed with an a posteriori possibility distribution ${\pi }_x$ defined on the decisions set $\mathsf{\Omega }$. In this case, ${\mathit{\ell }}_{C_m}\left(x\right)={\pi }_x\left(C_m\right)$ where ${\pi }_x\left(C_m\right)$ represents the possibility degree for the decision $C_m$ to occur, given the observed feature vector $x\in \mathsf{\Theta }$.

The second functionality performed by the decision-making process is called the hard decision or the selection functionality. As the ultimate goal of most classification applications is to select one and only one class (associated with the observations “o” for which the feature vector $x\in \mathsf{\Theta }$ is extracted) out of the classes set $\mathsf{\Omega }$, then a mapping has to be applied in order to transform the soft decision label vector $\mathit{\ell }\left(x\right)$ into a hard decision label vector for which one and only one decision is selected. The goal is then to make a choice according to an optimality criterion.

In this paper, we propose a new criterion for decision-making process in classification systems called possibilistic maximum likelihood (PML). This criterion is framed within the possibility theory, but it uses corresponding notions from Bayesian decision-making. The main motivation being the development of PML is for multisource information fusion where an object or a pattern may be observed through several channels and where the available information, concerning the observed features, may be of a probabilistic nature for some features, and of an epistemic nature for some others.

In the presence of both types of information sources, most encountered studies transform one of the two information types into the other form, and then apply either the classical Bayesian or possibilistic decision-making criteria. With the PML decision-making approach, the Bayesian decision-making framework is adopted. The epistemic knowledge is integrated into the decision-making process by defining possibilistic loss values instead of the usually used zero-one loss values. A set of possibilistic loss values is proposed and evaluated in the context of pixel-based image classification where a synthetic scene, composed of several thematic classes, is randomly generated using two types of probabilistic sensors: a Gaussian and a Rayleigh sensor, complemented by an expert type of information source. Results obtained with the proposed PML criterion show that the classification recognition rates approach the optimal case, being, when all the available information is expressed in terms of probabilistic knowledge.

When the sources of information can be modelled by probability theory, the Baysesian approach has sufficient decision-making tools to fuse that information and performs classification. However, in the case where the knowledge available for the decision-making process is ill-defined in the sense that it is totally or partially expressed in terms of ambiguous information representing limitations in feature values, or, encoding linguistic expert’s knowledge about the relationship between the feature values and different potential decisions, new mathematical tools (i.e., PML) need to be developed. This type of available knowledge can be represented as a conditional possibilistic soft decision label vector $\mathit{\ell }\left(x\right)$ defined on the decisions set $\mathsf{\Omega }$ such that, ${\mathit{\ell }}_{C_m}\left(x\right)={\pi }_x\left(C_m\right)=\pi \left(C_m|x\right)$ where $\pi \left(C_m|x\right)$ represents the possibility degree for the decision C_m to occur, given the observed feature vector $x\in \mathsf{\Theta }$ and the underlying observations o.

Possibility theory constitutes the natural framework allowing to tackle this type of information imperfection (called the epistemic uncertainty type) when one and only one decision (hard decision) must be selected from the exhaustive decisions set $\mathsf{\Omega }$, with incomplete, ill-defined or ambiguous available knowledge thus encoded as a possibility distribution over $\mathsf{\Omega }$. This paper proposes a joint decision-making criterion which allows to integrate such extra possibilistic knowledge within a probabilistic decision-making framework taking into account both types of information: possibilistic and probabilistic. In spite of the fact that possibility theory deals with uncertainty, which means that a unique but unknown elementary decision is to occur, and the ultimate goal is to determine this decision, there are relatively few studies that tackle that decision-making issue [18,19,20,21,22,23,24,25]. We must however mention the considerable contributions of Dubois and Prade [26] on possibility theory as well as on clarification on the various semantics of fuzzy sets [27,28,29,30]. Denoeux et al. [31,32,33] contributed as well significantly on that topic but they consider epistemic uncertainty as a higher order uncertainty upon probabilistic models such as in imprecise probabilities of Walley [34,35] and fuzzy sets type-2 [36,37,38] which is not being the case in this current paper.

The paper is organized this way. A brief recall of the Bayesian decision-making criteria, and of possibility theory is given in Section 2 and Section 3. Three major possibilistic decision making criteria, i.e., maximum possibility, maximum necessity measure and confidence index maximization, are being detailed in Section 4. The PML criterion is presented in Section 5 followed by its evaluation in Section 6. Paper closes with conclusion in Section 7.

2. Hard Decision in the Bayesian Framework

In the Bayesian classification framework, the most widely used hard decision is based on minimizing an overall decision risk function [14]. Assuming $\mathit{o}\in \Psi$ is the pattern for which the feature vector $x\in \mathsf{\Theta }$ is observed, let ${\lambda }_{m,n}$ denotes a “predefined” conditional loss or penalty, incurred for deciding that the observed pattern o is associated with the decision C_n, whereas the true decision (class or category) for o is $C_m\left(n,m ϵ\left\{1,\cdots ,M\right\}\right)$. Therefore, the probabilistic expected loss $R(C_n|x)$, also called the Conditional risk, associated with the decision $C_n$ given the observed feature vector $x\in \mathsf{\Theta }$, is given by:

$$R\left(C_n|x\right)=E\left\{{\lambda }_{m,n}\right\}={\displaystyle \sum }_{m=1}^M{\lambda }_{m,n}Pr\left\{C_m|x\right\}$$

(1)

where $E\{·\}$ stands for the mathematical expectation. Bayes decision criterion consists in minimizing the overall risk R, also called Bayes risk, as defined in (2), by computing the conditional risk for all decisions and then, selecting the decision C_n for which $R(C_n|x)$ is minimum:

$$R\left(C_n|x\right)=E_x\left\{R(C_n|x) \right\}={\displaystyle \int }R(C_n|x)Pr\left\{x\right\}dx$$

(2)

Therefore, the minimum-risk Bayes decision criterion is based on the selection of the decision C_n which gives the smallest risk $R(C_n|x)$. This rule can thus be formulated as follows:

$$Decision\left[x\left(p\right)\right]=\arg \underset{n=1,\cdots ,M}{\min }\left({\displaystyle \sum }_{m=1}^M{\lambda }_{m,n}Pr\left\{C_m|x\right\}\right)$$

(3)

If $Pr\left\{C_m\right\}$ denotes the a priori probability of the decision $C_m$ and $Pr\left\{x|C_m\right\}$, the likelihood function of the measured feature vector x, given the decision C_m, then using Bayes’ rule, the minimum-risk Bayes decision criterion (3) can be rewritten as:

$$Decision\left[x\left(p\right)\right]=\arg \underset{n=1,\cdots ,M}{\min }{\left({\displaystyle \sum }_{m=1}^{M}Pr\left\{C_m\right\}Pr\left\{x|C_m\right\}\right)}_{ }$$

(4)

In the two-category decision case, i.e., $\mathsf{\Omega }=\left\{C_1,C_2\right\}$, it can be easily shown that the minimum-risk Bayes decision criterion, simply called Bayes criterion, can be expressed as in (5):

$$LR=\frac{Pr\left\{x|C_1\right\}}{Pr\left\{x|C_2\right\}} \begin{array}{c}\stackrel{Decision\left[x\left(p\right)\right]=C_1}{\overbrace{>}}\\ \underset{Decision\left[x\left(p\right)\right]=C_2}{\underbrace{<}}\end{array} \underset{\eta }{\underbrace{\begin{array}{ccc}\frac{{\lambda }_{2,2}-{\lambda }_{2,1 }}{{\lambda }_{1,1}-{\lambda }_{1,2}}& ·& \frac{Pr\left\{C_2\right\}}{Pr\left\{C_1\right\}}\end{array}}}$$

(5)

In other words, this decision criterion consists of comparing the likelihood ratio (LR) $Pr\left\{x|C_1\right\}/Pr\left\{x|C_2\right\}$ to a threshold η independent of the observed feature vector x. The binary cost, or zero-one loss, assignment is commonly used in classification problems. This rule, expressed in (6), gives ${\lambda }_{m,n}$ no cost for a correct decision (when the true pattern class/decision $C_{m }$ is identical to the decided class/decision $C_{n }$) and a unit cost for a wrong decision (when the true class/decision $C_{m }$ is different from the decided class/decision $C_{n }$).

$${\lambda }_{m,n}=\{\begin{array}{c}\begin{array}{ccc}0& \mathrm{if}& C_m=C_n\end{array}\\ \begin{array}{ccc}1& \mathrm{if}& C_m\ne C_n\end{array}\end{array}$$

(6)

It should be noticed that this binary cost assignment considers all errors as equally costly. It also leads to express the conditional risk as:

$$R\left(C_n|x\right)={\displaystyle \sum }_{m=1}^M{\lambda }_{m,n}Pr\left\{C_m|x\right\}=1-Pr\left\{C_n|x\right\}$$

(7)

A decision minimizing the conditional risk $R\left(C_n|x\right)$ becomes a decision maximizing the a posteriori probability $Pr\left\{C_n|x\right\}$. As shown in (8), this version of the Bayes criterion is called the maximum a posteriori criterion (MAP) since it seeks to determine the decision maximizing the a posteriori probability value. It is also obvious that this decision process corresponds to the minimum-error decision rule which leads to the best recognition rate that a decision criterion can achieve:

$$Decisio{n}_{MAP}\left[x\left(p\right)\right]=\arg \underset{n=1,\cdots M}{\max }Pr\left\{C_n|x\right\}$$

(8)

When the decisions a priori probabilities $Pr\left\{C_m\right\}$ and the likelihood functions $Pr\left\{x|C_m\right\}$ are not available, or simply difficult to obtain, the Minmax Probabilistic Criterion (MPC) can be an interesting alternative to the minimum-risk Bayes decision criterion [39]. As expressed in (9), this hard decision criterion consists in selecting the decision that minimizes the maximum decision cost:

$$Decisio{n}_{MPC}\left[x\left(p\right)\right]=\arg \underset{n=1,\cdots ,M}{\min }\left[\underset{m=1,\cdots ,M}{\max }\left\{{\lambda }_{m,n}Pr\left\{C_m|x\right\}\right\}\right]$$

(9)

3. Brief Review of Possibility Theory

Possibility theory is a relatively new theory devoted to handle uncertainty in the context where the available knowledge is only expressed in an ambiguous form. This theory was first introduced by Zadeh in 1978 as an extension of fuzzy sets and fuzzy logic theory, to express the intrinsic fuzziness of natural languages as well as uncertain information [7]. It is well established that probabilistic reasoning, based on the use of a probability measure, constitutes the optimal approach dealing with uncertainty. In the case where the available knowledge is ambiguous and encoded by a membership function, i.e., a fuzzy set, defined over the decisions set, the possibility theory transforms the membership function into a possibility distribution $\pi$. Then the realization of each event (subset of the decisions set) is bounded by a possibilistic interval defined though a possibility, $\mathsf{\Pi }$, and a necessity, N, measures [16]. The use of these two dual measures in possibility theory makes the main difference compared with the probability theory. Besides, possibility theory is not additive in terms of beliefs combination, and makes sense on ordinal structures [17]. In the following subsections, the basic concepts of a possibility distribution and the dual possibilistic measures (possibility and necessity measures) will be presented. The possibilistic decision rules will be detailed in Section 4. Full details can be found in [11].

3.1. Possibility Distribution

Let $\mathsf{\Omega }=\left\{C_1,C_2, \cdots ,C_M\right\}$ be a finite and exhaustive set of M mutually exclusive elementary decisions (e.g., decisions, thematic classes, hypothesis, etc.). Exclusiveness means that one and only one decision may occur at one time, whereas exhaustiveness states that the occurring decision certainly belongs to $\mathsf{\Omega }$. Possibility theory is based on the notion of possibility distribution denoted by $\pi$, which maps elementary decisions from $\mathsf{\Omega }$ to the interval [0, 1], thus encoding “our” state of knowledge or belief, on the possible occurrence of each class $C_m\in \mathsf{\Omega }$. The value $\pi \left(C_m\right)$ represents to what extent it is possible for $C_m$ to be the unique occurring decision. In this context, two extreme cases of knowledge are given:

■

Complete knowledge: $\exists !C_m\in \mathsf{\Omega }, \pi \left(C_m\right)=1$ and $\pi \left(C_n\right)=0, \forall C_n\in \mathsf{\Omega }, C_n\ne C_m$.

■

Complete ignorance: $\forall C_m\in \mathsf{\Omega }, \pi \left(C_m\right)=1$ (all elements from $\mathsf{\Omega }$ are considered as totally possible). $\pi (·)$ is called a normal possibility distribution if it exists at least one element $C_{m_0}$ from $\mathsf{\Omega }$ such that $\pi \left(C_{m_0}\right)=1$.

3.2. Possibility and Necessity Measures

Based on the possibility distribution concept, two dual set measures, possibility, $\mathsf{\Pi }$, and a necessity, N, measures are derived. For every subset (or event) $A\subseteq \mathsf{\Omega }$, these measures are defined by:

$$\mathsf{\Pi }\left(A\right)=\underset{C_m\in A}{\max }\left[\pi \left(C_m\right)\right]$$

(10a)

$$\mathrm{N}\left(A\right)=1-\mathsf{\Pi }\left(A^{\mathrm{c}}\right)=\underset{C_m\notin A^{\mathrm{c}}}{\min }\left[1-\pi \left(C_m\right)\right]$$

(10b)

where $A^{\mathrm{c}}$ denotes the complement of the event A (i.e., $A{\displaystyle \cup }{A}^{\mathrm{c}}=\mathsf{\Omega } \mathrm{with} A{\displaystyle \cap }{A}^{\mathrm{c}}=\varnothing$).

The possibility measure $\mathsf{\Pi }\left(A\right)$ estimates the level of consistency about event A occurrence, given the available knowledge encoded by the possibility distribution $\pi$. Thus, $\mathsf{\Pi }\left(A\right)=0$ means that A is an impossible event while $\mathsf{\Pi }\left(A\right)=1$ means that the event A is totally possible. The necessity measure N(A) evaluates the level of certainty about event A occurrence, involved by possibility distribution $\pi$. $\mathrm{N}\left(A\right)=0$ means that the certainty about the occurrence of A is null. On the contrary, $\mathrm{N}\left(A\right)=1$ means that the occurrence of A is totally certain. In a classification problem, where each decision $C_m$ refers to a given class or category, the case where all events A are composed of a single decision $(A_m=\left\{C_m\right\}, m=1, \dots , M)$, is of particular interest. In this case, the possibility $\mathsf{\Pi }(·)$, and the necessity N($·$), measures are reduced to:

$$\mathsf{\Pi }\left(A_m\right)=\mathsf{\Pi }\left(\left\{C_m\right\}\right)=\pi \left(C_m\right)$$

(11a)

$$\mathrm{N}\left(A_m\right)=\mathrm{N}\left(\left\{C_m\right\}\right)=1-\mathsf{\Pi }\left({\left\{C_m\right\}}^{\mathrm{c}}\right)=1-\underset{n\ne m}{\max }\pi \left(C_n\right)$$

(11b)

4. Decision-Making in the Possibility Theory Framework

In this section, we will investigate existing possibilistic decision-making rules. Two families of rules can be distinguished: rules based on the direct use of the information encapsulated in the possibility distribution, and rules based on the use of uncertainty measures associated with this possibility distribution. Let $\mathsf{\Omega }=\left\{C_1,C_2, \cdots ,C_M\right\}$ be a finite and exhaustive set of M mutually exclusive elementary decisions. Given an observed pattern $\mathit{o}\in \Psi$ for which the feature vector $\mathit{x}\in \Theta$ is observed, let ${\pi }_x\left(C_m\right)$ denotes the a posteriori possibility distribution $\pi (C_m|x)$ defined on $\mathsf{\Omega }$. The possibility, ${\mathsf{\Pi }}_x\left(\left\{C_m\right\}\right)$, and necessity, ${\mathrm{N}}_x\left(\left\{C_m\right\}\right)$, measures are obtained as expressed in Equation (11), using the possibility distribution ${\pi }_x\left(C_m\right)$.

4.1. Decision Rule Based on the Maximum of Possibility

The decision rule based on the maximum of possibility is certainly the most widely used in possibilistic classification—decision-making applications. Indeed, as shown in (12), this rule is based on the selection of the elementary decision $C_{m_0}\in \mathsf{\Omega }$ having the highest possibility degree of occurrence ${\Pi }_x\left(\left\{C_{m_0}\right\}\right)$:

$$Decision \left[x\left(p\right)\right]= C_{m_0} \mathrm{if} \mathrm{and} \mathrm{only} \mathrm{if} m_{0 }=\arg \underset{n=1,\cdots M}{\max }{\mathsf{\Pi }}_x\left(\left\{C_m\right\}\right)$$

(12)

A “first” mathematical justification of this “intuitive” possibilistic decision-making rule can be derived from the Minmax Probabilistic Criterion (MPC), Equation (9), using a binary cost assignment rule. Indeed, ‘converting’ the a posteriori possibility distributions ${\pi }_x(·)$ into a posteriori probability distributions $Pr\left\{·|x\right\}$ is assumed to respect the three following constraints [30]: (a) the consistency principle, (b) the preference ordering preservation, and (c) the least commitment principle. The preference ordering preservation, on which we focus the attention here, means that if decision $C_{m_1}$ is preferred to decision $C_{m_2}$, i.e., ${\pi }_x\left(C_{m_1}\right)>{\pi }_x\left(C_{m_2}\right)$, then the a posteriori probability distribution $Pr\left\{·|x\right\}$ obtained from ${\pi }_x(·)$ should satisfy $Pr\left\{C_{m_1}|x\right\}>Pr\left\{C_{m_2}|x\right\}$. Equation (13) sums up this preference ordering preservation constraint:

$${\pi }_x\left(C_{m_1}\right)>{\pi }_x\left(C_{m_2}\right)⟺Pr\left\{C_{m_1}|x\right\}>Pr\left\{C_{m_2}|x\right\}$$

(13)

Therefore, selecting the decision maximizing the a posteriori probability or selecting the decision maximizing the a posteriori possibility decision is identical: using the MPC associated with the binary cost assignment rule or using the maximum possibility decision rule led to an identical result as expressed in (14).

$$Decision \left[x\left(p\right)\right]= C_{m_0} \mathrm{iff} m_{0 }=\arg \underset{m=1,\cdots M}{\max }Pr\left\{C_m|x\right\}=\underset{m=1,\cdots M}{\max }{\pi }_x\left(C_m\right)$$

(14)

This decision-making criterion is called the Naive Bayes style possibilistic criterion Refs. [40,41,42] and most ongoing efforts are oriented into the computation of the a posteriori possibility values using numerical data [43]. An extensive study of properties and equivalence between possibilistic and probability approaches is presented in [20]. Notice that this decision rule, strongly inspired from probabilistic decision reasoning, does not provide a hard decision mechanism when several elementary decisions have the same maximum possibility measure.

4.2. Decision Rule Based on Maximizing the Necessity Measure

It is worthwhile to notice that the a posteriori measures of possibility ${\mathsf{\Pi }}_x$ and necessity ${\mathrm{N}}_x$ coming from a normal a posteriori possibility distribution ${\pi }_x(·)$, constitute a bracketing for the a posteriori probability distribution $Pr\left\{·|x\right\}$ [17]:

$${\mathrm{N}}_x\left(\left\{C_m\right\}\right)=1-\underset{\begin{array}{c}n\ne m\\ n=1.\cdots ,M\end{array}}{\max }{\pi }_x\left(C_n\right)\le Pr\left\{C_m|x\right\}\le {\mathsf{\Pi }}_x\left(\left\{C_m\right\}\right)={\pi }_x\left(C_m\right)$$

(15)

Therefore, the maximum possibility decision criterion can be considered as an optimistic decision criterion as it maximizes the upper bound of the a posteriori probability distribution. On the contrary, a pessimistic decision criterion based on maximizing the a posteriori necessity measure can be considered as a maximization of the lower bound of the a posteriori probability distribution. Equation (16) expresses this pessimistic decision criterion:

$$Decision \left[x\left(p\right)\right]= C_{m_0} \mathrm{iff} m_{0 }=\arg \underset{n=1,\cdots M}{\max }\left[{\mathrm{N}}_x\left(\left\{C_m\right\}\right)\right]$$

(16)

The question that we must raise concerns the “links” between the optimistic and the pessimistic decision criteria. Let us consider the a posteriori possibility distribution ${\pi }_x(·)$ for which $C_{m_1}\left(\mathrm{resp}. C_{m_2}\right)$ is the “winning decision” obtained using the maximum possibility (resp. necessity measure) decision criteria as given in (17):

$${\mathsf{\pi }}_x\left(C_{m_1}\right)=\underset{m}{\max }{\mathsf{\pi }}_x\left(C_m\right)and {\mathrm{N}}_x\left(\left\{C_{m_2}\right\}\right)=\underset{m}{\max }{\mathrm{N}}_x\left(\left\{C_m\right\}\right)$$

(17)

The following important question can be formulated as follows: “Is the winning decision $C_{m_1}$ (according to the maximum possibility criterion) is the same as the winning decision $C_{m_2}$ according to maximum necessity measure criterion?”

First, notice that if several elementary decisions share the same maximum possibility value $\upsilon ={\mathsf{\pi }}_x\left(C_{m_1}\right)$, then, the necessity measure becomes a useless decision criterion since:

$${\mathrm{N}}_x\left(\left\{C_m\right\}\right)=1-\underset{k\ne m}{\max }{\mathsf{\pi }}_x\left(C_k\right)=1-\upsilon \mathrm{for} \mathrm{all} \mathrm{the} \mathrm{elementary} \mathrm{decisions}.$$

Now, suppose that only one decision $C_{m_1}$ assumes the maximum possibility value $\upsilon ={\mathsf{\pi }}_x\left(C_{m_1}\right)$, it is important to raise the question whether the decision $C_{m_1}$ will (or will not) be the decision assuming the maximum necessity measure value. Let us note v’, the possibility value for the “second best” decision according to the possibility value criterion. As $C_{m_1}$ is the unique decision having the maximum possibility value $\upsilon$, we have ${\upsilon }^{\prime }<\upsilon$. Therefore, as shown in (18), the necessity measure value ${\mathrm{N}}_x\left(\left\{C_m\right\}\right)$ only gets maximum for the decision $C_{m_1}$ since $1-{\upsilon }^{\prime }>1-\upsilon$.

$${\mathrm{N}}_x\left(\left\{C_m\right\}\right)=1-\underset{k\ne m}{\max }{\mathsf{\pi }}_x\left(C_k\right)=\{\begin{array}{c}\begin{array}{cc}1-{\upsilon }^{\prime } \mathrm{if}& m=\end{array} m_1 \\ \begin{array}{cc}1-\upsilon \mathrm{if}& m\ne \end{array} m_1\end{array}$$

(18)

As a conclusion, when the maximum necessity measure criterion is useful for application (i.e., only one elementary decision assumes the maximum possibility value), then, both decision criteria (maximum possibility and maximum necessity) produce the same winning decision. In order to illustrate the difference between the maximum possibility and the maximum necessity measure criteria, Figure 2 presents an illustrative example.

Figure 2. Comparative example of the maximum possibility and maximum necessity measures decision criteria using four a posteriori possibility distributions.

In Figure 2 example, four different a posteriori possibility distributions ${\pi }_1, {\pi }_2,{\pi }_3,{\pi }_4$, all defined on a five elementary decisions set $\mathsf{\Omega }=\left\{C_1,C_2,C_3,C_4,C_5\right\}$ are considered. The necessity measures ${\mathrm{N}}_k\left(\left\{C_m\right\}\right)$ have been computed from the corresponding possibility distribution ${\pi }_k$. The underlined values indicate which decisions result from the maximum possibility decision criterion as well as the maximum necessity measure decision criterion, for the four possibility distributions ${\pi }_k$. Note that the necessity measure assumes at most two values whatever the considered possibility distribution. When the a posteriori possibility distribution has one and only one decision having the highest possibility degree, then both decision rules produce the same winning decision. This is the case of the normal possibility distribution ${\mathsf{\pi }}_1$ as well as the subnormal possibility distribution ${\mathsf{\pi }}_3$, indicated as cases (a) and (c) in Figure 2.

When several elementary decisions share the same highest possibility degree, then the maximum possibility decision criterion can randomly select one of these potential winning decisions. In this case, the maximum necessity measure decision criterion will affect a single necessity measure degree to all elementary decisions from $\mathsf{\Omega }$, and thus, it will be impossible to select any of the potential winning decisions. This behavior can be observed with a normal possibility distribution ${\mathsf{\pi }}_2$ as well as with a subnormal possibility distribution (like ${\mathsf{\pi }}_4$), cases (b) and (d) in Figure 2. This example clearly shows the weakness of the decisional capacity of the maximum necessity measure decision criterion when compared to the maximum possibility decision criterion.

4.3. Decision Rule Based on Maximizing the Confidence Index

Other possibilistic decision rules based on the use of uncertainty measures are also encountered in literature. The most frequently used criterion (proposed by Kikuchi et al. [44]) is based on the maximization of the confidence index Ind defined as a combination of the possibility and the necessity measures for each event $A\subseteq \mathsf{\Omega }$, given a possibility distribution $\pi (·)$:

$$Ind: 2^{\mathsf{\Omega }}⟶\left[-1, +1\right]$$

$$A⟶Ind\left(A\right)=\mathsf{\Pi }\left(A\right)+\mathrm{N}\left(A\right)-1, \forall A\subseteq \mathsf{\Omega }$$

(19)

where $2^{\mathsf{\Omega }}$ denotes the power set of $\mathsf{\Omega }$, i.e., the set of all subsets from $\mathsf{\Omega }$.

For an event A, this index ranges from −1 to +1:

-

Ind(A) = −1, iff $\mathsf{\Pi }\left(A\right)=\mathrm{N}\left(A\right)=0$ (the occurrence of A is totally impossible and uncertain);

-

Ind(A) = +1, iff $\mathsf{\Pi }\left(A\right)=\mathrm{N}\left(A\right)=1$ (the occurrence of A is totally possible and certain).

Restricting the application of this measure to events $A_m$ having only one decision $A_m=\left\{C_m\right\}$ shows that Ind($A_m$) measures the difference between the possibility measure of the event $A_m$ (which is identical to the possibility degree of the decision $C_m$) and the highest possibility degree of all decisions contained in ${\left(A_m\right)}^{\mathrm{c}}$ (the complement of $A_m$ in $\mathsf{\Omega }$):

$$Ind\left(A_m\right)= \mathsf{\Pi }\left(A_m\right)+\mathrm{N}\left(A_m\right)-1=\mathsf{\pi }\left(C_m\right)-\underset{m\ne n}{\max }\pi \left(C_n\right)$$

(20)

Therefore, if $A_{m_0}=\left\{C_{m_0}\right\}$ is the only event having the highest possibility measure value $\mathsf{\pi }\left(C_{m_0}\right)$, then, $A_{m_0}$ will be the unique event having a positive confidence index value, whereas all other events will have negative values, as illustrated in Figure 3 where we assume $\mathsf{\pi }\left(C_{m_0}\right)>\mathsf{\pi }\left(C_m\right), \forall m\ne m_0$, and, $C_{m_1}$ refers to the decision having the second highest possibility degree.

Figure 3. Confidence indices associated with different decisions ($A_{m_0}$: event having the highest possibility degree, $A_{m_1}$: event with the second highest possibility degree). (Source: [11]).

In a classification decision-making problem, the decision criterion associated with this index can be formulated as follows:

$$Decisio{n}_{Ind} \left[x\left(p\right)\right]= A_{m_0} \mathrm{iff} Ind\left(A_{m_0}\right)=\underset{m=1,\cdots M}{\max }\left[Ind\left(A_m\right)\right]$$

(21)

The main difference between the maximum possibility and the maximum confidence index decision criteria lies in the fact that the maximum possibility decision criterion is only based on the maximum possibility degree whereas the maximum confidence index decision criterion is based on the difference between the two highest possibility degrees associated with the elementary decisions. As already mentioned, it is important to notice that the event $A_{m_0}=\left\{C_{m_0}\right\}$ having the highest possibilistic value, will be the unique event producing a positive confidence index measuring the difference with the second highest possibility degree. All other events $A_m=\left\{C_m\right\}, \forall m\ne m_0$, will produce negative confidence indices.

When several decisions share the same highest possibility degree, their confidence index (the highest one) will be null. This shows the real capacity of this uncertainty measure for the decision-making process. However, this criterion brings the same resulting decisions as the two former ones.

5. Possibilistic Maximum Likelihood (PML) Decision Criterion

In the formulation of the Bayesian classification approach, all information sources are assumed to have probabilistic uncertainty where the available knowledge describing this uncertainty is expressed, estimated or evaluated in terms of probability distributions. In the possibilistic classification framework, the information sources are assumed to suffer from possibilistic (or epistemic) uncertainty where the available knowledge describing this uncertainty is expressed in terms of possibility distributions. In this section, the Bayesian pattern recognition framework is generalized in order to integrate both probabilistic and epistemic sources of knowledge. A joint probabilistic—possibilistic decision criterion called Possibilistic Maximum Likelihood (PML) is proposed to handle both types of uncertainties.

5.1. Sources with Probabilistic and Possibilistic Types of Uncertainties

In some situations, an object from the observation space is observed through several feature sets. This is the case, for instance, in multi-sensor environment for classification applications. In such situations, the information available for the description of the feature vectors may be of different natures: probabilistic, epistemic, etc. Yager [24,45,46] addresses the same sort of problems: multi-source uncertain information fusion in the case when the information can be both from hard sensors of a probabilistic type and from soft knowledge-expert linguistic source of a possibilistic type. He uses t-norms (‘and’ operations) to combine possibility and probability measures. As will be explained below, Yager’s product of possibilities and probabilities coincides with our ‘decision variables’ optimized through the proposed PML approach.

Let us consider the example illustrated in Figure 4, where each pattern o (from the patterns set $\mathsf{\Psi }$) is “observed” through two channels. Source 1 (resp. source 2) measures a sub-feature vector ${\mathit{x}}_1\in {\mathsf{\Theta }}_1\left(\mathrm{resp}.{\mathit{x}}_2\in {\mathsf{\Theta }}_2\right)$. Therefore, the resulting feature vector $\mathit{x}\left(o\right)$ is obtained as the concatenation of the two sub-feature vectors: $\mathit{x}\left(\mathit{o}\right)=\left[{\mathit{x}}_1 {\mathit{x}}_2\right]$. In this configuration, the available information in the sub-feature vector x₁ (resp. x₂) undergoes probabilistic (resp. epistemic) uncertainties and is encoded as an a posteriori probability soft decision label vector ${\ell }_{C_n}^1=Pr\left\{C_n|x_1\right\}, n=1,2, \dots , M$ (resp. a posteriori possibility soft decision label vector ${\ell }_{C_n}^2={\pi }_{x_2}\left(C_n\right), n=1,2, \dots , M)$.

Figure 4. Multi-source information context for pattern classification.

As an example, in a remote sensing system, Source 1 may be considered as a multispectral imaging system, where all potential a posteriori probability distributions, $Pr\left\{C_n|x_1\right\}, n=1,2, \dots , M$, are assumed to be known and well established. The second sensor, Source 2, could be a radar imaging system where the available information concerning the different thematic classes is expressed by an expert using ambiguous linguistic variables like: “the thematic class $C_{n }$ is observed as “Bright”, “Slightly Dark”, etc. in the sub-feature set ${\mathsf{\Theta }}_2$”. Each linguistic variable can be used to generate an a posteriori possibility distribution associated with each thematic class ${\pi }_{x_2}\left(C_n\right), n=1,2, \dots , M$.

5.2. Possibilistic Maximum Likelihood (PML) Decision Criterion: A New Hybrid Criterion

In the Bayesian decision framework, detailed in previous sections, the binary cost assignment approach suffers from two constraints. On one hand, all errors are considered as equally costly: the penalty (or cost) of misclassifying an observed pattern o as being associated with a decision $C_n$ whereas the true decision for “o” is $C_m$ is the same (unit loss). This situation does not reflect real applications constraints. For instance, deciding that an examined patient is healthy whereas he suffers a cancer is much more serious than the other way around. On the other hand, the loss function values ${\lambda }_{m,n}, \forall m,n\in \left\{1,2,\dots ,M\right\}$ are static (or, predefined) and do not depend on the feature vectors of the observed patterns. The possibilistic maximum likelihood (PML) criterion, proposed in this paper, is based on the use of the epistemic source of information (the a posteriori possibility distribution, defined on the sub-feature space ${\mathsf{\Theta }}_2$) in order to define possibilistic loss values and to inject, afterwards, these values into the Bayesian decision criterion.

Assume that, for each object $\mathit{o}\in \Psi$, the observed feature vector is given by $\mathit{x}\left(\mathit{o}\right)=\left[{\mathit{x}}_1 {\mathit{x}}_2\right]\in {\mathsf{\Theta }}_1\times {\mathsf{\Theta }}_2$, and denote $Pr\left\{·|x_1\right\} (\mathrm{resp}.{\pi }_{x_2}(·)$) as encoding the a posteriori probability (resp. possibility) soft decision label vectors defined over the sub-feature set ${\mathsf{\Theta }}_1$ (resp. ${\mathsf{\Theta }}_2$). The proposed PML criterion relies on the use of loss values ${\lambda }_{m,n}$ ranging from −1 (i.e., no loss) to +1 (i.e., maximum loss), and ${\lambda }_{m,n}$ refers to the risk of choosing $C_n$ whereas the real decision for the considered pattern is $C_m$. Depending on the epistemic information available through Source 2, the proposed loss values are given by:

$${\lambda }_{m,n}=\{\begin{array}{c}\begin{array}{cc}\begin{array}{cc}\underset{k\ne m}{\max }{\pi }_{x_2}\left(C_k\right) =& {\mathsf{\Pi }}_{x_2}\left({\left\{C_m\right\}}^c\right)\end{array}& \forall m\ne n\end{array}\\ \begin{array}{cc}\underset{k\ne m}{\max }{\pi }_{x_2}\left(C_k\right)-{\pi }_{x_2}\left(C_m\right) =-In{d}_{x_2}\left(\left\{C_m\right\}\right)& \mathrm{if} m=n\end{array} \end{array}$$

(22)

In the case of a wrong decision, the decision penalty values, i.e., ${\lambda }_{m,n}$ where $\ne m$, are considered as positive loss values ranging in the interval: $0\le$ ${\lambda }_{m,n}$ = $\underset{k\ne m}{\max }{\pi }_{x_2}\left(C_k\right)\le 1.$ Thus, the wrong decision unit cost in the framework of binary-cost assignment, is “softened”, in this possibilistic approach, and assumes its maximum value, i.e., unit cost, only when the wrong decision $C_n$ has a total possibility degree of occurrence.

When a correct decision $C_m$ is selected, the zero-loss value (used by the binary cost assignment approach) is substituted by ${\lambda }_{m,n}=\underset{k\ne m}{\max }{\pi }_{x_2}\left(C_k\right)-{\pi }_{x_2}\left(C_m\right)=-In{d}_{x_2}\left(\left\{C_m\right\}\right)$. If the occurrence possibility degree ${\pi }_{x_2}\left(C_m\right)$ of the true decision $C_m$, is the highest degree ${\pi }_{x_2}\left(C_m\right)>\underset{k}{\max }{\pi }_{x_2}\left(C_k\right)$, then the resulting loss value ${\lambda }_{m,n}$ becomes negative. The smallest penalty value is reached, i.e., ${\lambda }_{m,n}=-1$, when ${\pi }_{x_2}\left(C_m\right)=1$ (i.e., true decision $C_m$ has a total possibility degree of occurrence), with a null possibility degree of occurrence for all the remaining decisions (leading to $\underset{m\ne k}{\max }{\pi }_{x_2}\left(C_k\right)=0$). Two special cases are present:

(1)

If the true decision $C_m$ shares the same maximum possibility value with, at least one different wrong decision $C_m$, then, the correct decision $C_m$ loss value becomes null ${\lambda }_{m,n}=\underset{k\ne m}{\max }{\pi }_{x_2}\left(C_k\right)-{\pi }_{x_2}\left(C_m\right) =0$;

(2)

If the true decision $C_m$ does not produce the maximum occurrence possibility degree, i.e.,${\pi }_{x_2}\left(C_m\right)<\underset{k\ne m}{\max }{\pi }_{x_2}\left(C_k\right)$, then the loss value ${\lambda }_{m,n}$ is positive and will increase the conditional risk, associated with the true decision $C_m$.

Using the proposed possibilistic loss values, the conditional risk $R\left(C_k|x_1\right)$ of choosing decision C_k can thus be computed as follows:

$$R\left(C_k|x\right) =-In{d}_{x_2}\left(\left\{C_k\right\}\right)·Pr\left\{C_k|x_1\right\}+{\displaystyle \sum }_{\begin{array}{c}i=1\\ i\ne k\end{array}}^M\mathsf{\Pi }\left({\left\{C_m\right\}}^c\right)· Pr\left\{C_i|x_1\right\}$$

(23)

As already mentioned, Bayes decision criterion computes the conditional risk for all decisions, then, selects the decision $C_n$ for which $R\left(C_n|x\right)$ is minimum. Based on Equation (23), and to select the minimum conditional risk decision, the comparison of conditional risks related to two decisions $C_k$ and $C_p$, can be straightforward performed leading to:

$$R\left(C_k|x_1\right)\le R\left(C_p|x_1\right)\iff {\pi }_{x_2}\left(C_k\right)·Pr\left\{C_k|x_1\right\}\ge {\pi }_{x_2}\left(C_k\right)· Pr\left\{C_p|x_1\right\}$$

(24)

Therefore, the application of the PML criterion, for the selection of the minimum conditional risk decision (out of M potential elementary decisions) can be simply formulated by the following decision rule:

$$Decision \left[x\left(p\right)\right]=\arg \underset{n=1,\cdots M}{\max }{\pi }_{x_2}\left(C_n\right)·Pr\left\{C_n|x_1\right\}$$

(25)

This “intuitive” decision criterion allows the joint use both probabilistic and epistemic sources of information in the very same Bayesian minimum risk framework. As an example, the application of the proposed possibilistic loss values in the two-class decision case, where $\mathsf{\Omega }=\left\{C_1,C_2\right\}$, leads to the following loss matrix [$\lambda$]:

$$\left[\lambda \right]=\left[\begin{array}{cc}{\lambda }_{1,1}& {\lambda }_{1,2}\\ {\lambda }_{2,1}& {\lambda }_{2,2}\end{array}\right]=\left[\begin{array}{cc}{\pi }_{x_2}\left(C_2\right)-{\pi }_{x_2}\left(C_1\right)& {\pi }_{x_2}\left(C_2\right)\\ {\pi }_{x_2}\left(C_1\right)& {\pi }_{x_2}\left(C_1\right)-{\pi }_{x_2}\left(C_2\right)\end{array}\right]$$

(26)

The use of this loss matrix [$\lambda$] into the minimum-risk Bayes decision approach (as defined in (5)), leads to express the PML decision as follows:

$$\frac{Pr\left\{x_1|C_1\right\}}{Pr\left\{x_1|C_2\right\}} \begin{array}{c}\stackrel{Decision\left[x\left(p\right)\right]=C_1}{\overbrace{>}}\\ \underset{Decision\left[x\left(p\right)\right]=C_2}{\underbrace{<}}\end{array} \begin{array}{ccc}\frac{{\pi }_{x_2}\left(C_2\right)}{{\pi }_{x_2}\left(C_1\right)}& ·& \frac{Pr\left\{C_2\right\}}{Pr\left\{C_1\right\}}\end{array}$$

(27)

Notice that when the proposed possibilistic loss values are considered, then the PML induces a “weighting adjustment” of the a priori probabilities where the weighting factors are simply the a posteriori possibility degrees issued from the possibilistic Source 2. In the case of equal a priori probabilities, $Pr\left\{C_2\right\}=Pr\left\{C_1\right\}$, this decision criterion turns to an intuitive form using jointly probabilistic and epistemic sources of information, in the Bayesian minimum risk framework as shown by:

$${\pi }_{x_2}\left(C_1\right)·Pr\left\{x_1|C_1\right\} \begin{array}{c}\stackrel{Decision\left[x\left(p\right)\right]=C_1}{\overbrace{>}}\\ \underset{Decision\left[x\left(p\right)\right]=C_2}{\underbrace{<}}\end{array} {\pi }_{x_2}\left(C_2\right)·Pr\left\{x_1|C_2\right\}$$

(28)

It is worthwhile to notice that when the two following conditions prevail:

• when the available probabilistic information (issued from source 1) is non-informative; and,
• when the only meaningful and available information is reduced to the epistemic expert information on the sub-feature vector issued from source 2; then, the proposed PML criterion is simply reduced to the maximum possibility decision criterion:

  $${\pi }_{x_2}\left(C_1\right) \begin{array}{c}\stackrel{Decision\left[x\left(p\right)\right]=C_1}{\overbrace{>}}\\ \underset{Decision\left[x\left(p\right)\right]=C_2}{\underbrace{<}}\end{array} {\pi }_{x_2}\left(C_2\right)$$

  (29)

This raises a fundamental interpretation of the maximum possibility decision criterion as being a very special case of the possibilistic Bayesian decision making process under the total ignorance assumption of the probabilistic source of information.

5.3. PML Decision Criterion Behavior

Let $S_1$ denotes a probabilistic source of information measuring a sub-feature vector ${\mathit{x}}_1\in {\mathsf{\Theta }}_1$ and attributing to each elementary decision $C_m, m=1,2,\dots ,M$, an a posteriori probability soft decision label $Pr\left\{C_m|x_1\right\}$. Under the assumption of equal a priori probabilities and using the binary-cost assignment, the application of the maximum a posteriori criterion (MAP), Equation (8), turns to be the “optimal” criterion ensuring the minimum-error decision rate.

Assume that an additional possibilistic source of information, $S_2$, (measuring a sub-feature vector ${\mathit{x}}_2\in {\mathsf{\Theta }}_2)$ is available, see Figure 4. Based on the use of the sub-feature vector ${\mathit{x}}_2\in {\mathsf{\Theta }}_2$, $S_2$ attributes to each elementary decision, $C_m$, an a posteriori possibility soft decision label ${\pi }_{x_2}\left(C_m\right), m=1,2,\dots ,M$. To obtain a hard decision, the application of the maximum of possibility decision criterion, Equation (12), is considered.

In the previous section, we have proposed the possibilistic maximum likelihood, PML, decision criterion, Equation (25), as a hybrid decision criterion allowing the coupled use of both sources of information, $S_1$ and $S_2$, by considering the possibilistic information issued from $S_2$, i.e., ${\pi }_{x_2}\left(C_m\right), m=1,2,\dots ,M$, for the definition of the loss values in the framework of the minimum-risk Bayes decision criterion (instead of the use of the binary-cost assignment approach). In this section, we will briefly discuss, from a descriptive point of view through an illustrative example, the “decisional behavior” of the PML criterion when compared to the decisions obtained with the “individual” application of the MAP and the maximum of possibility decision criteria.

First, it is worthwhile to notice that the “decision variable” to be maximized by the PML criterion is simply the direct product $\upsilon \left(C_m\right)={\pi }_{x_2}\left(C_m\right)·Pr\left\{C_m|x_1\right\}, m=1,2,\dots ,M$, which is a T-norm fusion operator (considering both probabilistic and possibilistic information as two “similar” measures of the degree of truthfulness related to the occurrence of different elementary decisions, see also p.101 of Yager [24]). This also means that both sources of information, $S_1$ and $S_2$, are considered as having the same informative level. It is also important to notice that the PML criterion, as a decision fusion operator merging decisional information from both sources, $S_1$ and $S_2$, constitutes a coherent decision fusion criterion in the sense that:

-

when both sources $S_1$ and $S_2$ are in full agreement (i.e., leading to the same decision $C_{m_0}$), then, the decision obtained by the application of the PML criterion will be the same as $C_{m_0}$;

-

when one of the two sources $S_1$ and $S_2$ suffers from total ignorance (i.e., producing equal a posteriori probabilities, for $S_1$ and equal a posteriori possibilities, for $S_2$), then the PML criterion will “duplicate” the same elementary decision as the one proposed by the remaining reliable source of information;

-

when the two sources $S_1$ and $S_2$ lack decisional agreement, then, the decision obtained by the application of the PML criterion will be the most “plausible” elementary decision that may be different from individual decisions resulting from the MAP (resp. maximum possibility) criterion using the sub-feature vector $x_1\in {\mathsf{\Theta }}_1$ (resp. sub-feature vector $x_2\in {\mathsf{\Theta }}_2$).

This decision fusion coherence is illustrated through the examples given in Table 1. The decisions set is formed by five elementary decisions, i.e., $\mathsf{\Omega }=\left\{C_1,C_2,C_3,C_4,C_5\right\}$, and we assume that, given the observed feature ${\mathit{x}}_1\in {\mathsf{\Theta }}_1$, the probabilistic source $S_1$ produces the following a posteriori probability distribution: $Pr\left\{·|x_1\right\}$ = [0.1 0.4 0.1 0.3 0.1]. Each example fits in one sub-array which presents $S_1$ and $S_2$ specific configuration ($Pr\left\{·|x_1\right\}$, ${\pi }_{x_2}(·)$ and $\upsilon =Pr\left\{·|x_1\right\}·{\pi }_{x_2}(·))$ with the resulting decision for each decision parameter. The cases presented in Table 1 are explained as the following:

Table 1. PML decision making behavior for several cases.

	Case 1			Case 2			Case 3
	$Pr\left\{·|x_1\right\}$	${\pi }_{x_2}(·)$	$\upsilon (·)$	$Pr\left\{·|x_1\right\}$	${\pi }_{x_2}(·)$	$\upsilon (·)$	$Pr\left\{·|x_1\right\}$	${\pi }_{x_2}(·)$	$\upsilon (·)$
C1	0.1	0.2	0.02	0.1	1.0	0.1	0.2	0.8	0.16
C2	0.4	0.7	0.28	0.4	1.0	0.4	0.2	0.2	0.04
C3	0.1	0.3	0.03	0.1	1.0	0.1	0.2	0.1	0.02
C4	0.3	0.7	0.21	0.3	1.0	0.3	0.2	0.1	0.02
C5	0.1	0.1	0.10	0.1	1.0	0.1	0.2	0.2	0.04
	Case 4			Case 5			Case 6
	$Pr\left\{·|x_1\right\}$	${\pi }_{x_2}(·)$	$\upsilon (·)$	$Pr\left\{·|x_1\right\}$	${\pi }_{x_2}(·)$	$\upsilon (·)$	$Pr\left\{·|x_1\right\}$	${\pi }_{x_2}(·)$	$\upsilon (·)$
C1	0.1	1.0	0.1	0.1	0.8	0.08	0.1	0.8	0.08
C2	0.4	0.2	0.08	0.4	0.2	0.08	0.4	0.3	0.12
C3	0.1	0.0	0.0	0.1	0.1	0.01	0.1	0.4	0.04
C4	0.3	0.0	0.0	0.3	0.3	0.03	0.35	0.5	0.16
C5	0.1	0.0	0.0	0.1	0.1	0.02	0.1	0.5	0.05

• Case 1: when both sources $S_1$ and $S_2$ agree, with a winning decision $C_2$, the PML criterion maintains this agreement and obtains the same decision, $C_2$.
• Cases 2 and 3: it shows that when one of the two sources presents a total ignorance, then the PML criterion “duplicates” the same elementary decision as the one offered by the remaining reliable source of information.
• Cases 4, 5 and 6: when sources $S_1$ and $S_2$ lack agreement (i.e., dissonant sources), then, the resulting decision obtained through the application of the PML criterion is the most reasonable decision. That may not necessarily be one of the winning decisions offered by the two sources (this is specifically shown in case 6).

6. Experimental and Validation Results

In this section, the proposed PML decision-making criterion is evaluated in a pixel-based image classification context. A synthetic scene composed of five thematic classes $\mathsf{\Omega }=\left\{C_1,C_2,C_3,C_4,C_5\right\}$ is assumed to be observed through two independent imaging sensors. Sensor $S_1$ (resp. sensor $S_2$) provides an image $I_1$ (resp. $I_2$) of the simulated scene. The two considered sensors are assumed to be statistically independent. Without loss of generalization, pixels from both images $I_1$ and $I_2$ are assumed to have the same spatial resolution, thus, they represent the same observed spatial cell or object o. The value of the pixel $I_1\left(i,j\right)$ (resp. $I_2\left(i,j\right)$) provides the observed feature ${\mathit{x}}_1\left(\mathrm{resp}.{\mathit{x}}_2\right)$ delivered by the first (resp. second) sensor. According to sensors characteristics, the measured feature ${\mathit{x}}_1\left(\mathrm{resp}.{\mathit{x}}_2\right)$ follows a Gaussian $\mathcal{N}\left(m_C,{\sigma }_C^2\right)$ (resp. Rayleigh $ℛ\left({\sigma }_C^2\right)$) probability distribution with related parameters $m_C, {\sigma }_C^2$ depending on the thematic class “C” of the observed object.

Figure 5 depicts the experimental simulated images $I_1$ (resp. $I_2$) assumed to be delivered at the output of the two sensors. Figure 6 shows the possibility distributions encoding expert’s information, for the five thematic classes. Parameter values considered for each thematic class are given in the same figure. This configuration of classes’ parameters is considered as a reasonable configuration that may be encountered when real data is observed. Nevertheless, other configurations have been generated and the obtained results are in full accordance with those obtained by the considered configuration.

Figure 5. Two-sensors simulated images representing a scene of five thematic classes. Pixels from ${\mathit{I}}_1$ (resp. ${\mathit{I}}_2$) are generated using Gaussian (resp. Rayleigh) density functions.

Figure 6. Triangular-shaped possibility distributions encoding expert’s knowledge, for the five thematic classes and the two sensors.

In addition to the previously mentioned probabilistic information, we assume that each thematic class is described, by an expert, using the “simplest” linguistic variable “Close to $v_{S,C_k}$” where $v_{S,C_k}$ denotes the thematic class $C_k$ feature mean value, observed through sensor $S_s$. Therefore, the only information given by the expert is $v_{1,C_k}=m_{C_K}$ for sensor $S_1$ (underlying Gaussian distributions) and $v_{2,C_k}={\sigma }_{C_K}\sqrt{\pi /2}$ for sensor $S_2$ (underlying Rayleigh distributions). For each sensor $S_s$ and thematic class $C_k$, a standard triangular possibility distribution is considered to encode this epistemic knowledge with the summit positioned at the mean value $v_{S,C_k}$ and the support covering the whole range of the features set. It is clearly seen that the possibility distributions (considered as encoding the expert’s knowledge), represent a weak knowledge which is less informative than the initial, or even estimated, probabilistic density functions.

To evaluate the efficiency of the proposed possibilistic maximum likelihood decision making criterion, the adopted procedure consists, first, on the random generation of 1000 statistical realizations of the two synthetic Gaussian and Rayleigh images (with the five considered thematic classes) representing the analyzed scene. Second, the following average pixel-based recognition rates are evaluated:

-

$\tau \left(P{r}_{C_m}^G\right)$: Minimum-risk Bayes average pixel-based decision recognition rate, Equation (8), using zero-one loss assignment, for each thematic class $C_m, m=1,2,\dots ,5,$ based on the use of sensor $S_1$ Gaussian feature $x_1$ only.

-

$\tau \left({\pi }_{C_m}^G\right)$: Maximum possibility average pixel-based decision recognition rate, Equation (12), exploiting the epistemic expert knowledge for the description of each considered thematic class $C_m, m=1,2,\dots ,5,$ in the features set ${\mathsf{\Theta }}_1$ only.

-

$\tau \left(P{r}_{C_m}^G·{\pi }_{C_m}^R\right)$: Possibilistic maximum likelihood average pixel-based decision recognition rate, Equation (24), jointly exploiting the epistemic expert knowledge for the description of each considered thematic class $C_m, m=1,2,\dots ,5,$ in the features set ${\mathsf{\Theta }}_2$ (sensor $S_2$), and the Gaussian probabilistic knowledge for the description of the same thematic class in the features set ${\mathsf{\Theta }}_1$ (sensor $S_1$).

-

$\tau \left(P{r}_{C_m}^R\right)$: Minimum-risk Bayes average pixel-based decision recognition rate, Equation (8), using zero-one loss assignment, for each thematic class $C_m, m=1,2,\dots ,5,$ based on the use of sensor $S_2$ Rayleigh feature $x_2$ only.

-

$\tau \left({\pi }_{C_m}^G\right)$: Maximum possibility average pixel-based decision recognition rate, Equation (12), exploiting the epistemic expert knowledge for the description of each considered thematic class $C_m, m=1,2,\dots ,5,$ in the features set ${\mathsf{\Theta }}_1$ (sensor $S_1$).

-

$\tau \left(P{r}_{C_m}^R·{\pi }_{C_m}^G\right)$: Possibilistic maximum likelihood average pixel-based decision recognition rate, Equation (24), jointly exploiting the epistemic expert knowledge for the description of each thematic class $C_m, m=1,2,\dots ,5,$ in sensor $S_1$ features set ${\mathsf{\Theta }}_1$, and the Rayleigh probabilistic knowledge for the description of $C_m, m=1,2,\dots ,5,$ in sensor $S_2$ features set ${\mathsf{\Theta }}_2$.

-

$\tau \left(P{r}_{C_m}^G·P{r}_{C_m}^R\right)$: Minimum-risk Bayes average pixel-based decision recognition rate, Equation (8), using zero-one loss assignment, for each thematic class $C_m, m=1,2,\dots ,5,$ and based on the joint use of both sensors $S_1$ (associated with Gaussian probabilistic knowledge in the features set ${\mathsf{\Theta }}_1$) and S₂ (associated with Rayleigh probabilistic knowledge in the features set ${\mathsf{\Theta }}_2$. Sensors $S_1$ and $S_2$ are considered as being statistically independent. This criterion as well as all the criteria above have been calculated for the example in Table 2.

Table 2. PML decision average pixel-based recognition rates for the five thematic classes using various configurations of knowledge sources.

Knowledge Sources	Source S1: Probabilistic (G: Gaussian) Source S2: Epistemic (Expert)			Source S2: Probabilistic (R: Rayleigh) Source S1: Epistemic (Expert)			Both Sources Are Probabilistic
	${\mathit{S}}_1$	${\mathit{S}}_2$	${\mathit{S}}_1\underset{\mathbf{PML}}{\underbrace{\oplus }}{\mathit{S}}_2$	${\mathit{S}}_2$	${\mathit{S}}_1$	${\mathit{S}}_1\underset{\mathbf{PML}}{\underbrace{\oplus }}{\mathit{S}}_2$	${\mathit{S}}_1\oplus {\mathit{S}}_2$
Criterion	$\tau \left(P{r}_{C_m}^G\right)$	$\tau \left({\pi }_{C_m}^G\right)$	$\tau \left(P{r}_{C_m}^G·{\pi }_{C_m}^R\right)$	$\tau \left(P{r}_{C_m}^R\right)$	$\tau \left({\pi }_{C_m}^G\right)$	$\tau \left(P{r}_{C_m}^R·{\pi }_{C_m}^G\right)$	$\tau \left(P{r}_{C_m}^G·P{r}_{C_m}^R\right)$
C1	0.96	0.33	0.96	0.34	0.96	0.64	0.96
C2	0.90	0.32	0.91	0.49	0.98	0.68	0.92
C3	0.78	0.91	0.81	0.89	0.67	0.90	0.91
C4	0.94	0.25	0.95	0.32	0.96	0.61	0.97
C5	0.83	0.72	0.86	0.58	0.75	0.71	0.94
Average Recognition Rate	0.88	0.51	0.90	0.52	0.86	0.71	0.94

The obtained average recognition rates are summarized in Table 2 (last row). As expected, at the global scene level, the average recognition rates when a probabilistic information source is used (for modelling the observed features) are higher than those obtained by the use of epistemic knowledge (i.e., $\tau \left(P{r}^G\right)\ge \tau \left({\pi }^G\right)$, and $\tau \left(P{r}^R\right)\ge \tau \left({\pi }^R\right)$). Nevertheless, at the thematic classes’ level, this property does not hold for some classes. This is mainly due to the fact that for “sharp classes” probability density functions, i.e., small variance, (for instance, thematic classes $C_2$ and $C_4$), the possibility distributions shape used to encode the expert knowledge (i.e., a wide-based triangular possibility shape) may bias each class influence, leading to a better recognition rate to the detriment of other neighboring classes (for instance, class $C_3$). In this case, this leads to obtain $\tau \left({\pi }_{C_2}^G\right)>\tau \left(P{r}_{C_2}^G\right)$ and $\tau \left({\pi }_{C_4}^G\right)>\tau \left(P{r}_{C_4}^G\right)$.

Poorer recognition performances of the maximum possibility decision criterion clearly come from the “weak epistemic knowledge” produced by the expert (indicating just the mean values) compared to the “strong probabilistic” knowledge involved by full probability density functions (resulting from either a priori information or the densities estimation using some learning data). The most interesting and promising result can be witnessed in terms of recognition rate improvement when the epistemic knowledge is jointly used with the probabilistic one as proposed by the PML decision criterion. Indeed, Table 2 (columns 4 and 7, bold numbers) shows that for all classes $C_m, m=1,2,\dots ,5,$ we have $\tau \left(P{r}_{C_m}^G·{\pi }_{C_m}^R\right)\ge \tau \left(P{r}_{C_m}^G\right)$ and $\tau \left(P{r}_{C_m}^R·{\pi }_{C_m}^G\right)\ge \tau \left(P{r}_{C_m}^R\right)$.

It is worthwhile to notice (columns 4 and 7, bold numbers) that the level of performance improvement depends on the “informative” capacity of the “additional” knowledge source. For instance, embedding the Gaussian source of knowledge (in terms of epistemic knowledge form) into the decisional process based on the probabilistic Rayleigh source of knowledge, improves much more the performance level than the reverse (i.e., embedding epistemic Rayleigh source of knowledge into the decisional process based on the probabilistic Gaussian source of knowledge):$\tau \left(P{r}_{C_m}^G·{\pi }_{C_m}^R\right)\approx$$\tau \left(P{r}_{C_m}^G\right)$ whereas $\tau \left(P{r}_{C_m}^R·{\pi }_{C_m}^G\right)>\tau \left(P{r}_{C_m}^R\right)$.

Finally, it is important to notice that the PML decision performances are lower-upper bounds delimited as follows:

$$\begin{array}{c}\tau \left(P{r}_{C_m}^G\right)\le \tau \left(P{r}_{C_m}^G·{\pi }_{C_m}^R\right)\le \tau \left(P{r}_{C_m}^G·P{r}_{C_m}^R\right)\\ \tau \left(P{r}_{C_m}^R\right)\le \tau \left(P{r}_{C_m}^R·{\pi }_{C_m}^G\right)\le \tau \left(P{r}_{C_m}^R·P{r}_{C_m}^G\right)\end{array}$$

(30)

Given the fact that the two sources S₁ and S₂ are assumed to be statistically independent, then the joint probability distribution of the augmented feature vector $\mathit{x}=\left[{\mathit{x}}_1 {\mathit{x}}_2\right]$ is the direct product of marginal ones. This simply means that the upper bounds given in Equation (30) constitute the optimal recognition rate (obtained by considering both probabilistic sources of knowledge). Therefore, the PML criterion improves the performances of the use of a “single” probabilistic source of knowledge, and approaches for some thematic classes the optimal recognition rate upper bound (last column of Table 2).

7. Conclusions

In this paper, a new criterion for decision-making process in classification systems is proposed. After a brief recall of the Bayesian decision-making criteria, three major possibilistic decision making criteria, i.e., maximum possibility, maximum necessity measure and confidence index maximization, have been detailed. It was clearly shown that the three considered decision criteria lead to, at best, the maximum possibility decision criterion. However, the maximum possibility criterion has no physical justification. A new criterion called, possibilistic maximum likelihood (PML) framed within the possibility theory, but using notions from Bayesian decision-making, has been presented and its behavior evaluated. The main motivation being the development of such criterion is for multisource information fusion where a pattern may be observed through several channels and where the available knowledge, concerning the observed features, may be of a probabilistic nature for some features, and of an epistemic nature for some others.

In this configuration, most encountered studies transform one of the two knowledge types into the other form, and then apply either the classical Bayesian or possibilistic decision-making criteria. In this paper, we have proposed a new approach called the Possibilistic maximum likelihood (PML) decision-making approach, where the Bayesian decision-making framework is adopted and where the epistemic knowledge is integrated into the decision-making process by defining possibilistic loss values instead of the usually used zero-one loss values.

A set of possibilistic loss values is proposed and evaluated in the context of pixel-based image classification where a synthetic scene, composed of several thematic classes, was randomly generated using two types of sensors: a Gaussian and a Rayleigh sensor. The evaluation of the proposed PML criterion has clearly shown the interest of the application of PML; where the obtained recognition rates approach the optimal rates (i.e., where all the available knowledge is expressed in terms of probabilistic knowledge). Moreover, the proposed PML decision criterion offers a physical interpretation of the maximum possibility decision criterion as a special case of the possibilistic Bayesian decision-making process when all the available probabilistic information indicates equal decisions probabilities.