Inequalities for Jensen–Sharma–Mittal and Jeffreys–Sharma–Mittal Type f–Divergences

Abstract

In this paper, we introduce new divergences called Jensen–Sharma–Mittal and Jeffreys–Sharma–Mittal in relation to convex functions. Some theorems, which give the lower and upper bounds for two new introduced divergences, are provided. The obtained results imply some new inequalities corresponding to known divergences. Some examples, which show that these are the generalizations of Rényi, Tsallis, and Kullback–Leibler types of divergences, are provided in order to show a few applications of new divergences.

1. Introduction

The Sharma–Mittal entropy was introduced as a new measure of information with two parameters [1]. It has previously been studied in the context of multi-dimensional harmonic oscillator systems [2]. This entropy could also be formulated in the form of exponential families, to which many usual statistical distributions including the Gaussians and discrete multinomials (that is, normalized histograms) belong. In physical applications it plays a major role in the field of thermo-statistics [3].

The Sharma–Mittal entropy is also applied for the analysis of the results of machine learning methods [4,5]. Additionally, the divergence based on considered entropy could be a cost function in the context of so-called the Twin Gaussian Processes [6].

It was originally showed by [7] that the Sharma–Mittal entropy generalized both Tsallis and Rényi entropy in the limiting cases of these two entropies. In [8], authors suggested a physical meaning of Sharma–Mittal entropy, which is the free energy difference between the equilibrium and the off-equilibrium distribution.

Recently, was published a manuscript showing, in opposition to the work [8], that Sharma–Mittal entropy besides the convenient thermodynamic systems does not reduce only to Kullback–Leibler entropy. In [9] Verma and Merigó present the use of Sharma–Mittal entropy under intuitionistic fuzzy environment. Additionally, in [5] Koltcov et al. demonstrate that Sharma–Mittal entropy is a tool for selecting both the number of topics and the values of hyper-parameters, simultaneously controlling for semantic stability, which none of the existing metrics can do.

Another applications of considered entropy are interesting results in the cosmological setup, such as black hole thermodynamics [10]. Namely, it helps us to describe the current accelerated universe by using the vacuum energy in a suitable manner [11]. In addition [12] have established the relation between anomalous diffusion process and Sharma–Mittal entropy.

This paper is based on publications in which we introduced new types of f-divergences [13,14,15,16].

In this paper we generalize Sharma–Mittal types divergences in order to obtain new types of divergences and hence the inequalities from which it will be possible to derive new results and generalizations for known divergences in order to estimate the lower and upper bounds which determine the level of the uncertainty measure.

2. Sharma–Mittal Type Divergences

Throughout ${\mathbb{R}}_{+}$ and ${\mathbb{R}}_{++}$ denote the sets of non-negative and positive numbers, respectively, i.e., ${\mathbb{R}}_{+}=[0,\infty )$ and ${\mathbb{R}}_{++}=(0,\infty )$.

Let $\mathbf{p}=(p_1,\dots ,p_n)$ and $\mathbf{q}=(q_1,\dots ,q_n)$ with $p_i,q_i\ge 0$, $i=1,\dots ,n$. The relative entropy (also called Kullback–Leibler divergence) is defined by (see [17])

$$H_1(\mathbf{p},\mathbf{q})=\sum _{i=1}^n{p}_{i}log\frac{p_i}{q_i}.$$

(1)

In the above definition, based on continuity arguments, we use a convention that $0log(0/q)=0$ and $plog(p/0)=+\infty$. Additionally $0log(0/0)=0$.

Let f: ${\mathbb{R}}_{+}\to \mathbb{R}$ be a convex function on ${\mathbb{R}}_{+}$, and $\mathbf{p}=(p_1,\dots ,p_n)\in {\mathbb{R}}_{++}^n$, $\mathbf{q}=(q_1,\dots ,q_n)\in {\mathbb{R}}_{+}^n$.

The Csiszár f-divergence is defined by (see [15])

$$C_f\left(\mathbf{p},\mathbf{q}\right)=\sum _{i=1}^n{p}_{i}f\left(\frac{q_i}{p_i}\right).$$

(2)

with the conventions $0f\left(\frac{0}{0}\right)=0$ and $0f\left(\frac{c}{0}\right)=c\underset{t\to \infty }{lim}\frac{f\left(t\right)}{t}$, $c>0$ (see [18,19,20]).

The Tsallis divergence of order $\alpha$ is defined by (see [17])

$$T_{\alpha }(\mathbf{p},\mathbf{q})=\frac{1}{\alpha -1}\left(\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }-1\right).$$

The Rényi divergence of order $\alpha$ is defined by (see [17,21])

$$H_{\alpha }(\mathbf{p},\mathbf{q})=\frac{1}{\alpha -1}log\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }.$$

The Sharma–Mittal divergence of order $\alpha$ and degree $\beta$ is defined by (see [4])

$$S{M}_{\alpha ,\beta }(\mathbf{p},\mathbf{q})=\frac{1}{\beta -1}\left[{\left(\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }\right)}^{\frac{1-\beta }{1-\alpha }}-1\right],$$

(3)

for all $\alpha >0$, $\alpha \ne 1$ and $\beta \ne 1$.

Let $g:I\to \mathbb{R}$ be a convex function on an interval $I\subset \mathbb{R}$. Let $\mathbf{x}=(x_1,\dots ,x_n)\in I^n$ and $p_i\in [0,1)$ for $i=1,\dots n$.

The Jensen’s inequality is as follows (see [22])

$$g\left(\sum _{i=1}^n{p}_i{x}_i\right)\le \sum _{i=1}^n{p}_{i}g\left(x_i\right).$$

(4)

When the function $k:{\mathbb{R}}_{+}\to \mathbb{R}$ is convex and the function $l:{\mathbb{R}}_{+}\to \mathbb{R}$ is convex and increasing then the composition of the functions $k\circ l:{\mathbb{R}}_{+}\to \mathbb{R}$ is convex. We assume that the probabilities $p_i\ge 0$ and $q_i>0$ for $i=1,\dots ,n$.

It is known (see [4]) that if

$$\alpha =\beta \to 1\mathrm{then}S{M}_{\alpha ,\beta }(\mathbf{p},\mathbf{q})\to H_1(\mathbf{p},\mathbf{q}),$$

$$\beta \to 1\mathrm{and}\alpha \in \mathbb{R}\mathrm{then}S{M}_{\alpha ,\beta }(\mathbf{p},\mathbf{q})\to H_{\alpha }(\mathbf{p},\mathbf{q}),$$

$$\beta =\alpha \mathrm{then}S{M}_{\alpha ,\beta }(\mathbf{p},\mathbf{q})\to T_{\alpha }(\mathbf{p},\mathbf{q}).$$

Let $h:\mathbb{R}\to \mathbb{R}$ be the differentiable function. Then the Sharma–Mittal h-divergence is defined as follows:

$$S{M}_{h,\alpha ,\beta }(\mathbf{p},\mathbf{q})=\frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}{p}_i^{\alpha }{q}_i^{1-\alpha }\right]}^{\frac{1-\beta }{1-\alpha }}\right)-1}{\beta -1},$$

(5)

for all $\alpha >0$, $\alpha \ne 1$ and $\beta \ne 1$.

If we assume that $h=id$ then (5) becomes Sharma–Mittal divergence.

When for all $t>0$, $h\left(t\right)=log\left(te\right)$ then (5) becomes Rényi divergence of order $\alpha$.

We substitute for $t={\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}{p}_i^{\alpha }{q}_i^{1-\alpha }\right]}^{\frac{1-\beta }{1-\alpha }}$ and we have

$$h\left(t\right)=h\left({\left[\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }\right]}^{\frac{1-\beta }{1-\alpha }}\right)=log\left({\left[\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }\right]}^{\frac{1-\beta }{1-\alpha }}e\right)=log{\left[\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }\right]}^{\frac{1-\beta }{1-\alpha }}+1.$$

Hence, from (5)

$$S{M}_{h,\alpha ,\beta }(\mathbf{p},\mathbf{q})=\frac{log{\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}{p}_i^{\alpha }{q}_i^{1-\alpha }\right]}^{\frac{1-\beta }{1-\alpha }}}{\beta -1}=\frac{1}{\alpha -1}log\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }=H_{\alpha }(\mathbf{p},\mathbf{q}).$$

Let $\mathsf{\Psi }:{\mathbb{R}}_{+}\times {\mathbb{R}}_{+}\to \mathbb{R}$ be a differentiable function with respect to $\beta$ and

$$\mathsf{\Psi }(\alpha ,\beta )=h\left({\left[\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }\right]}^{\frac{1-\beta }{1-\alpha }}\right).$$

We assume that $h^{\prime }\left(1\right)=1$ and $\mathsf{\Psi }(\alpha ,1)=1$. Then,

$$\underset{\beta \to 1}{lim}\frac{\mathsf{\Psi }(\alpha ,\beta )-\mathsf{\Psi }(\alpha ,1)}{\beta -1}={\mathsf{\Psi }}^{\prime }(\alpha ,1)=h^{\prime }{\left({\left[\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }\right]}^{\frac{1-\beta }{1-\alpha }}\right)}_{|\beta =1}=$$

$${\frac{\partial }{\partial \beta }}_{{|}_{\beta =1}}h\left({\left[\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }\right]}^{\frac{1-\beta }{1-\alpha }}\right){\frac{\partial }{\partial \beta }}_{{|}_{\beta =1}}\left({\left[\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }\right]}^{\frac{1-\beta }{1-\alpha }}\right){\frac{\partial }{\partial \beta }}_{{|}_{\beta =1}}\left(\frac{1-\beta }{1-\alpha }\right)=$$

$$h^{\prime }\left(1\right){\left(\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }\right)}^{\frac{1-1}{1-\alpha }}log\left(\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }\right)\left(\frac{-1}{1-\alpha }\right)=$$

$$h^{\prime }\left(1\right)\frac{1}{\alpha -1}log\left(\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }\right)=\frac{1}{\alpha -1}log\left(\sum _{i=1}^n{p}_i^{\alpha }{q}_i^{1-\alpha }\right).$$

Hence, the Sharma–Mittal h–divergence tends to Rényi divergence of order $\alpha$.

Remark 1.

If, additionally, α tends to 1 then based on the proof of the Equation (11) from [16], Sharma–Mittal h-divergence tends to relative entropy (called Kullback–Leibler divergence).

Now we define a new generalized $(h,\varphi )$ Sharma–Mittal divergence as follows

$$S{M}_{h,\varphi ,\alpha ,\beta }(\mathbf{p},\mathbf{q})=\frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)-1}{\beta -1},$$

(6)

where $\varphi :(0,+\infty )\times {\mathbb{R}}_{+}\to {\mathbb{R}}_{+}$ is an increasing, non-negative and differentiable function for $\beta >1$.

We assume that $\mathcal{F}=\{f_{\alpha }:(0,\infty )\to \mathbb{R}:\alpha \in \mathbb{R}\}$ is a given family of functions such that ${\displaystyle {\displaystyle \sum }_{i=1}^n}{q}_i{f}_{{\alpha }_{{|}_{\alpha =1}}}\left(\frac{p_i}{q_i}\right)=1$ for $\alpha =1$ and which are increasing, non-negative for $\alpha >1$ and such that for every $t\in (0,+\infty )$ the function $\alpha ⟼f_{\alpha }\left(t\right)$ is differentiable.

According to [16] if we substitute the function $f_{\alpha }\left(\frac{p_i}{q_i}\right)$ from the family $\mathcal{F}$ for $\varphi (\frac{p_i}{q_i},\alpha )$ then it stands that

$$\underset{\beta \to 1}{lim}S{M}_{h,\varphi ,\alpha ,\beta }(\mathbf{p},\mathbf{q})=R_{h,f_{\alpha }}(\mathbf{p},\mathbf{q}).$$

We assume that $h\left(1\right)=1$. Then,

$$\underset{\beta \to 1}{lim}S{M}_{h,\varphi ,\alpha ,\beta }(\mathbf{p},\mathbf{q})=\underset{\beta \to 1}{lim}\frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)-1}{\beta -1}=$$

$$h^{\prime }\left({\left[\sum _{i=1}^n{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right]}^0\right)·{\left[\sum _{i=1}^n{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right]}^0\left[log\sum _{i=1}^n{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right]\frac{-1}{1-\alpha }=$$

$$h^{\prime }\left(1\right)\frac{1}{\alpha -1}log\sum _{i=1}^n{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)=\frac{1}{\alpha -1}log\sum _{i=1}^n{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)=$$

$$\frac{1}{\alpha -1}log\sum _{i=1}^n{q}_i{f}_{\alpha }\left(\frac{p_i}{q_i}\right)={\mathbb{R}}_{h,f_{\alpha }}(\mathbf{p},\mathbf{q}).$$

(7)

Remark 2.

If in (6) $\beta \to 1$ and $\varphi (\frac{p_i}{q_i},\alpha )=f_{\alpha }\left(\frac{p_i}{q_i}\right)$ then the generalized $(h,\varphi )$–Sharma–Mittal divergence tends to generalized $(h,\mathcal{F})$–Rényi divergence.

The function $\varphi$ is the generalization of the function $f_{\alpha }\left(\frac{p_i}{q_i}\right)$ which is used for example in Csiszár f-divergence. Condition $\beta \to 1$ means that the limit of generalized Sharma–Mittal divergence is equal to generalized $(h,\mathcal{F})$–Rényi divergence. Hence we have implications for generalized forms of entropies.

Remark 3.

Additionally, when in (6) $\alpha \to 1$ and $\varphi (\frac{p_i}{q_i},\alpha )={\left(\frac{p_i}{q_i}\right)}^{\alpha }$ then the generalized $(h,\varphi )$–Sharma–Mittal divergence tends to Kullback–Leibler divergence, because we have from Remark 2

$$\begin{gathered} \underset{(\alpha ,\beta )\to (1,1)}{lim}S{M}_{h,\varphi ,\alpha ,\beta }(\mathbf{p},\mathbf{q})=\underset{\alpha \to 1}{lim}\frac{log{\displaystyle {\displaystyle \sum }_{i=1}^n}{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)-log1}{\alpha -1}= \\ {\frac{\partial }{\partial \alpha }}_{{|}_{\alpha =1}}log{\displaystyle \sum _{i=1}^n}{q}_i{\left(\frac{p_i}{q_i}\right)}^{\alpha }=\frac{1}{{\displaystyle {\displaystyle \sum }_{i=1}^n}{p}_i}\sum _{i=1}^n{q}_i\frac{p_i}{q_i}log\frac{p_i}{q_i}=\sum _{i=1}^n{p}_{i}log\frac{p_i}{q_i}=H_1(\mathbf{p},\mathbf{q}). \end{gathered}$$

Remark 4.

In (6), when the parameter $\beta =\alpha$, the function $h=id$ and $\varphi (\frac{p_i}{q_i},\alpha )={\left(\frac{p_i}{q_i}\right)}^{\alpha }$ then the generalized $(h,\varphi )$–Sharma–Mittal divergence tends to the Tsallis f-divergence or order α.

This work is more theoretical than practical. Therefore, the implications are formulated in the mathematical area that is from constructing general model which gives known specific cases.

3. Jensen–Sharma–Mittal and Jeffreys–Sharma–Mittal Divergences

The Jensen–Shannon divergence (Jensen–Shannon entropy) is defined as follows (see [17]):

$$\mathrm{Jen}(\mathbf{p},\mathbf{q})=\frac{1}{2}SM\left(\mathbf{p},\frac{\mathbf{p}+\mathbf{q}}{2}\right)+\frac{1}{2}SM\left(\mathbf{q},\frac{\mathbf{p}+\mathbf{q}}{2}\right).$$

The Jeffreys divergence (Jeffreys entropy) is defined as follows (see [17]):

$$\mathrm{Jef}(\mathbf{p},\mathbf{q})=SM(\mathbf{p},\mathbf{q})+SM(\mathbf{q},\mathbf{p}).$$

We introduce a new generalized $(h,\varphi )$ Jensen–Sharma–Mittal divergence defined by

$$\mathrm{Jen}S{M}_{\alpha ,\beta }^{h,\varphi }(\mathbf{p},\mathbf{q})=\frac{1}{2}S{M}_{h,\varphi ,\alpha ,\beta }\left(\mathbf{p},\frac{\mathbf{p}+\mathbf{q}}{2}\right)+\frac{1}{2}S{M}_{h,\varphi ,\alpha ,\beta }\left(\mathbf{q},\frac{\mathbf{p}+\mathbf{q}}{2}\right)$$

(8)

with assumptions as before.

We similarly introduce a new generalized $(h,\varphi )$ Jeffreys–Sharma–Mittal divergence as follows

$$\mathrm{Jef}S{M}_{\alpha ,\beta }^{h,\varphi }(\mathbf{p},\mathbf{q})=S{M}_{h,\varphi ,\alpha ,\beta }(\mathbf{p},\mathbf{q})+S{M}_{h,\varphi ,\alpha ,\beta }(\mathbf{q},\mathbf{p}).$$

(9)

Taking into account inequality from [17]:

$$0\le \mathrm{Jen}(\mathbf{p},\mathbf{q})\le \frac{1}{2}\mathrm{Jef}(\mathbf{p},\mathbf{q}),$$

describing the relation between the Jensen–Shannon and Jeffreys divergences, we could formulate the following:

$$0\le \mathrm{Jen}S{M}_{\alpha ,\beta }^{h,\varphi }(\mathbf{p},\mathbf{q})\le \frac{1}{2}\mathrm{Jef}S{M}_{\alpha ,\beta }^{h,\varphi }(\mathbf{p},\mathbf{q}).$$

(10)

We define the Jensen–Sharma–Mittal h-divergence where, in (8), $\varphi \left(\frac{p_i}{q_i},\alpha \right)={\left(\frac{p_i}{q_i}\right)}^{\alpha }$. Then, it takes the form:

$$\mathrm{Jen}S{M}_{\alpha ,\beta }^h(\mathbf{p},\mathbf{q})=\frac{1}{2}S{M}_{h,\alpha ,\beta }\left(\mathbf{p},\frac{\mathbf{p}+\mathbf{q}}{2}\right)+\frac{1}{2}S{M}_{h,\alpha ,\beta }\left(\mathbf{q},\frac{\mathbf{p}+\mathbf{q}}{2}\right)$$

(11)

In the same way, we define the Jeffreys–Sharma–Mittal h–divergence:

$$\mathrm{Jef}S{M}_{\alpha ,\beta }^h(\mathbf{p},\mathbf{q})=S{M}_{h,\alpha ,\beta }(\mathbf{p},\mathbf{q})+S{M}_{h,\alpha ,\beta }(\mathbf{q},\mathbf{p}).$$

(12)

Additionally, if the function $h\left(t\right)=t$ then we define the Jensen–Sharma–Mittal and the Jeffreys–Sharma–Mittal divergences of order $\alpha$ and degree $\beta$, respectively.

$$\mathrm{Jen}S{M}_{\alpha ,\beta }(\mathbf{p},\mathbf{q})=\frac{1}{2}S{M}_{\alpha ,\beta }\left(\mathbf{p},\frac{\mathbf{p}+\mathbf{q}}{2}\right)+\frac{1}{2}S{M}_{\alpha ,\beta }\left(\mathbf{q},\frac{\mathbf{p}+\mathbf{q}}{2}\right),$$

(13)

$$\mathrm{Jef}S{M}_{\alpha ,\beta }(\mathbf{p},\mathbf{q})=S{M}_{\alpha ,\beta }(\mathbf{p},\mathbf{q})+S{M}_{\alpha ,\beta }(\mathbf{q},\mathbf{p}).$$

(14)

When in (8) and (9) $\beta \to 1$ and we substitute for $\varphi \left(\frac{p_i}{q_i},\alpha \right)=f_{\alpha }\left(\frac{p_i}{q_i}\right)$ then we obtain, defined in [16], the generalized $(h,\mathcal{F})$ Jensen–Rényi and Jeffreys–Rényi divergences, respectively:

$$\mathrm{Jen}S{M}_{\alpha ,1}^{h,f_{\alpha }}(\mathbf{p},\mathbf{q})=\mathrm{Jen}{\mathbb{R}}_{h,f_{\alpha }}(\mathbf{p},\mathbf{q})=\frac{1}{2}{\mathbb{R}}_{h,f_{\alpha }}\left(\mathbf{p},\frac{\mathbf{p}+\mathbf{q}}{2}\right)+\frac{1}{2}{\mathbb{R}}_{h,f_{\alpha }}\left(\mathbf{q},\frac{\mathbf{p}+\mathbf{q}}{2}\right),$$

$$\mathrm{Jef}S{M}_{\alpha ,1}^{h,f_{\alpha }}(\mathbf{p},\mathbf{q})=\mathrm{Jef}{\mathbb{R}}_{h,f_{\alpha }}(\mathbf{p},\mathbf{q})={\mathbb{R}}_{h,f_{\alpha }}(\mathbf{p},\mathbf{q})+{\mathbb{R}}_{h,f_{\alpha }}(\mathbf{q},\mathbf{p}).$$

(15)

The following theorem is the generalization and refinement of the inequalities for some known divergences and provides lower and upper bounds for the generalized $(h,\varphi )$ Jeffreys–Sharma–Mittal divergence in order to a more accurate estimation of its uncertainty measure.

Theorem 1.

Let $\mathbf{p}=(p_1,\dots ,p_n)$ and $\mathbf{q}=(q_1,\dots ,q_n)$ be two discrete probability distributions with $p_i>0$, $q_i>0$, $\frac{q_i}{p_i}\in I_0$, $\frac{p_i}{q_i}\in I_0$, $i=1,\dots ,n$, where $I_0\subset \mathbb{R}$ is an interval, such that $1\in I_0$. Let $\varphi :I_0\times {\mathbb{R}}_{+}\to {\mathbb{R}}_{+}$ be an increasing, non-negative and differentiable function for which ${\displaystyle \sum }_{i=1}^n{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\ge 1$ and ${\displaystyle \sum }_{i=1}^n{p}_i\varphi \left(\frac{q_i}{p_i},\alpha \right)\ge 1$ where $1<\beta \le \alpha$, $\alpha \in {\mathbb{R}}_{+}\setminus \left\{1\right\}$ and $h:I_0\to \mathbb{R}$ be a convex and increasing function on $I_0$.

Then, the following inequalities are valid:

$$\frac{1}{\alpha -1}log\prod _{i=1}^n{\left(\varphi \left(\frac{p_i}{q_i},\alpha \right)\right)}^{q_i}{\left(\varphi \left(\frac{q_i}{p_i},\alpha \right)\right)}^{p_i}\le \mathrm{Jef}S{M}_{\alpha ,\beta }^{h,\varphi }(\mathbf{p},\mathbf{q})\le$$

$$\frac{1}{\beta -1}\left(\mathrm{Jef}{C}_{h\circ \varphi }(\mathbf{p},\mathbf{q})-2\right).$$

(16)

Proof. 

Taking into account the assumptions, we could formulate the following inequality:

$${\left[\sum _{i=1}^n{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\le \sum _{i=1}^n{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right).$$

(17)

The function h is increasing and convex, therefore, from (4) and (17) we obtain inequalities:

$$h\left({\left[\sum _{i=1}^n{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)\le h\left(\sum _{i=1}^n{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right)\le \sum _{i=1}^n{q}_i(h\circ \varphi )\left(\frac{p_i}{q_i},\alpha \right).$$

(18)

In the same way, we obtain the following inequalities:

$$h\left({\left[\sum _{i=1}^n{p}_i\varphi \left(\frac{q_i}{p_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)\le h\left(\sum _{i=1}^n{p}_i\varphi \left(\frac{q_i}{p_i},\alpha \right)\right)\le \sum _{i=1}^n{p}_i(h\circ \varphi )\left(\frac{q_i}{p_i},\alpha \right).$$

(19)

From (9) we have

$$\mathrm{Jef}S{M}_{\alpha ,\beta }^{h,\varphi }(\mathbf{p},\mathbf{q})=S{M}_{h,\varphi ,\alpha ,\beta }(\mathbf{p},\mathbf{q})+S{M}_{h,\varphi ,\alpha ,\beta }(\mathbf{q},\mathbf{p})=$$

$$\frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)-1}{\beta -1}+\frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}{p}_i\varphi \left(\frac{q_i}{p_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)-1}{\beta -1}$$

Taking into account (2), (18), (19) and the definition of Jeffreys divergence, it stands that:

$$\mathrm{Jef}S{M}_{\alpha ,\beta }^{h,\varphi }(\mathbf{p},\mathbf{q})\le \frac{{\displaystyle {\displaystyle \sum }_{i=1}^n}{q}_i(h\circ \varphi )\left(\frac{p_i}{q_i},\alpha \right)-1}{\beta -1}+\frac{{\displaystyle {\displaystyle \sum }_{i=1}^n}{p}_i(h\circ \varphi )\left(\frac{q_i}{p_i},\alpha \right)-1}{\beta -1}=$$

$$\frac{{\displaystyle {\displaystyle \sum }_{i=1}^n}{q}_i(h\circ \varphi )\left(\frac{p_i}{q_i},\alpha \right)+{\displaystyle {\displaystyle \sum }_{i=1}^n}{p}_i(h\circ \varphi )\left(\frac{q_i}{p_i},\alpha \right)-2}{\beta -1}=\frac{\mathrm{Jef}{C}_{h\circ \varphi }(\mathbf{p},\mathbf{q})-2}{\beta -1}.$$

(20)

The above inequality is the upper bound for generalized $(h,\varphi )$ Jeffreys–Sharma–Mittal divergence.

By using the convexity of the function h with $h\left(1\right)=1$ the following inequality is valid for $\beta >1$:

$$\frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)-1}{\beta -1}\ge {\frac{\partial }{\partial \beta }}_{{|}_{\beta =1}}h\left({\left[\sum _{i=1}^n{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right).$$

(21)

From (7) the above derivative function is equal to: $\frac{1}{\alpha -1}log{\displaystyle \sum _{i=1}^n}{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)$.

The function $f\left(t\right)=logt$ is concave and increasing. Then, it stands that:

$$\frac{1}{\alpha -1}log\sum _{i=1}^n{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\ge \frac{1}{\alpha -1}\sum _{i=1}^n{q}_{i}log\left(\varphi \left(\frac{p_i}{q_i},\alpha \right)\right).$$

(22)

Hence, from (21) and (22) we have the inequality:

$$\frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)-1}{\beta -1}\ge \frac{1}{\alpha -1}\sum _{i=1}^n{q}_{i}log\left(\varphi \left(\frac{p_i}{q_i},\alpha \right)\right).$$

(23)

Similarly, we obtain the second inequality:

$$\frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}{p}_i\varphi \left(\frac{q_i}{p_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)-1}{\beta -1}\ge \frac{1}{\alpha -1}\sum _{i=1}^n{p}_{i}log\left(\varphi \left(\frac{q_i}{p_i},\alpha \right)\right).$$

(24)

We have from (6), (23) and (24) that:

$$S{M}_{h,\varphi ,\alpha ,\beta }(\mathbf{p},\mathbf{q})\ge \frac{1}{\alpha -1}\sum _{i=1}^n{q}_{i}log\left(\varphi \left(\frac{p_i}{q_i},\alpha \right)\right),$$

$$S{M}_{h,\varphi ,\alpha ,\beta }(\mathbf{q},\mathbf{p})\ge \frac{1}{\alpha -1}\sum _{i=1}^n{p}_{i}log\left(\varphi \left(\frac{q_i}{p_i},\alpha \right)\right).$$

Then, by using the definition (9) we have:

$$\mathrm{Jef}S{M}_{\alpha ,\beta }^{h,\varphi }(\mathbf{p},\mathbf{q})\ge \frac{1}{\alpha -1}\left(log\prod _{i=1}^n{\left(\varphi \left(\frac{p_i}{q_i},\alpha \right)\right)}^{q_i}+log\prod _{i=1}^n{\left(\varphi \left(\frac{q_i}{p_i},\alpha \right)\right)}^{p_i}\right)=$$

$$\frac{1}{\alpha -1}log\prod _{i=1}^n{\left(\varphi \left(\frac{p_i}{q_i},\alpha \right)\right)}^{q_i}{\left(\varphi \left(\frac{q_i}{p_i},\alpha \right)\right)}^{p_i}.$$

(25)

This result is the lower bound of the generalized $(h,\varphi )$ Jeffreys–Sharma–Mittal divergence.

Combining (20) and (25) we obtain the expected inequalities (16). □

Corollary 1.

When we substitute for $\varphi \left(\frac{p_i}{q_i},\alpha \right)={\left(\frac{p_i}{q_i}\right)}^{\alpha }$ then from (16) we obtain the inequalities for Jeffreys–Sharma–Mittal h-divergence:

$$\frac{1}{\alpha -1}log\prod _{i=1}^n{\left(\frac{p_i}{q_i}\right)}^{\alpha (q_i-p_i)}\le \mathrm{Jef}S{M}_{\alpha ,\beta }^h(\mathbf{p},\mathbf{q})\le$$

$$\frac{{\displaystyle {\displaystyle \sum }_{i=1}^n}{q}_{i}h\left({\left(\frac{p_i}{q_i}\right)}^{\alpha }\right)+{\displaystyle {\displaystyle \sum }_{i=1}^n}{p}_{i}h\left({\left(\frac{q_i}{p_i}\right)}^{\alpha }\right)-2}{\beta -1}.$$

We now formulate the theorem thanks to which the estimation of the generalized $(h,\varphi )$ Jensen–Sharma–Mittal divergence will be possible.

Theorem 2.

Let $\mathbf{p}=(p_1,\dots ,p_n)$ and $\mathbf{q}=(q_1,\dots ,q_n)$ be two discrete probability distributions with $p_i>0$, $q_i>0$, $\frac{q_i}{p_i}\in I_0$, $\frac{p_i}{q_i}\in I_0$, $i=1,\dots ,n$, where $I_0\subset \mathbb{R}$ is an interval such that $1\in I_0$. Let $\varphi :I_0\times {\mathbb{R}}_{+}\to {\mathbb{R}}_{+}$ be an increasing, non-negative and differentiable function for which ${\displaystyle {\displaystyle \sum }_{i=1}^n}\frac{p_i+q_i}{2}\varphi \left(\frac{2p_i}{p_i+q_i},\alpha \right)\ge 1$ and ${\displaystyle {\displaystyle \sum }_{i=1}^n}\frac{p_i+q_i}{2}\varphi \left(\frac{2q_i}{p_i+q_i},\alpha \right)\ge 1$ where $1<\beta \le \alpha$, $\alpha \in {\mathbb{R}}_{+}\setminus \left\{1\right\}$ and $h:I_0\to \mathbb{R}$ be a convex and increasing function on $I_0$.

Then, the following inequalities are valid:

$$\frac{1}{2(\alpha -1)}log\prod _{i=1}^n{\left(\varphi \left(\frac{2p_i}{p_i+q_i},\alpha \right)\varphi \left(\frac{2q_i}{p_i+q_i},\alpha \right)\right)}^{\frac{p_i+q_i}{2}}\le \mathrm{Jen}S{M}_{\alpha ,\beta }^{h,\varphi }(\mathbf{p},\mathbf{q})\le$$

$$\frac{{\displaystyle {\displaystyle \sum }_{i=1}^n}(p_i+q_i)\left[(h\circ \varphi )\left(\frac{2p_i}{p_i+q_i},\alpha \right)+(h\circ \varphi )\left(\frac{2q_i}{p_i+q_i},\alpha \right)\right]-4}{4(\beta -1)}$$

(26)

Proof. 

Let’s consider the function

$$\frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}\frac{p_i+q_i}{2}\varphi \left(\frac{p_i}{\frac{p_i+q_i}{2}},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)-1}{\beta -1}.$$

(27)

Using the assumptions that the function h is differentiable, convex and $h\left(1\right)=1$ we could formulate the following inequality:

$$\frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}\frac{p_i+q_i}{2}\varphi \left(\frac{p_i}{\frac{p_i+q_i}{2}},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)-1}{\beta -1}\ge {\frac{\partial }{\partial \beta }}_{{|}_{\beta =1}}h\left({\left[\sum _{i=1}^n\frac{p_i+q_i}{2}\varphi \left(\frac{p_i}{\frac{p_i+q_i}{2}},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right).$$

(28)

Then, (28) is equal to:

$$\left[log\sum _{i=1}^n\frac{p_i+q_i}{2}\varphi \left(\frac{p_i}{\frac{p_i+q_i}{2}},\alpha \right)\right]\left(\frac{-1}{1-\alpha }\right)=\frac{1}{\alpha -1}log\sum _{i=1}^n\frac{p_i+q_i}{2}\varphi \left(\frac{p_i}{\frac{p_i+q_i}{2}},\alpha \right).$$

Taking into account concavity of the function log, we have that:

$$\frac{1}{\alpha -1}log\sum _{i=1}^n\frac{p_i+q_i}{2}\varphi \left(\frac{p_i}{\frac{p_i+q_i}{2}},\alpha \right)\ge \frac{1}{\alpha -1}\sum _{i=1}^n\frac{p_i+q_i}{2}log\left(\varphi \left(\frac{p_i}{\frac{p_i+q_i}{2}},\alpha \right)\right).$$

Then, we obtain that (27) is greater than

$$\frac{1}{\alpha -1}log\prod _{i=1}^n{\left(\varphi \left(\frac{p_i}{\frac{p_i+q_i}{2}},\alpha \right)\right)}^{\frac{p_i+q_i}{2}}.$$

(29)

We do the same with the function

$$\frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}\frac{p_i+q_i}{2}\varphi \left(\frac{q_i}{\frac{p_i+q_i}{2}},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)-1}{\beta -1}.$$

(30)

Hence, we have that (30) is greater than

$$\frac{1}{\alpha -1}log\prod _{i=1}^n{\left(\varphi \left(\frac{q_i}{\frac{p_i+q_i}{2}},\alpha \right)\right)}^{\frac{p_i+q_i}{2}}.$$

(31)

Then, combining (27), (29)–(31), and using the definition (8) the following inequality occurs

$$\mathrm{Jen}S{M}_{\alpha ,\beta }^{h,\varphi }(\mathbf{p},\mathbf{q})\ge \frac{1}{2(\alpha -1)}log\prod _{i=1}^n{\left(\varphi \left(\frac{2p_i}{p_i+q_i},\alpha \right)\varphi \left(\frac{2q_i}{p_i+q_i},\alpha \right)\right)}^{\frac{p_i+q_i}{2}}$$

(32)

and it is the lower bound of the generalized $(h,\varphi )$ Jensen–Sharma–Mittal divergence.

When we consider the function

$$h\left({\left[\sum _{i=1}^n\frac{p_i+q_i}{2}\varphi \left(\frac{p_i}{\frac{p_i+q_i}{2}},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)$$

(33)

with $1<\beta \le \alpha$ then for the convex and increasing function h we have from (4) that (33) is smaller than

$$h\left(\sum _{i=1}^n\frac{p_i+q_i}{2}\varphi \left(\frac{p_i}{\frac{p_i+q_i}{2}},\alpha \right)\right)\le \sum _{i=1}^n\frac{p_i+q_i}{2}(h\circ \varphi )\left(\frac{p_i}{\frac{p_i+q_i}{2}},\alpha \right).$$

(34)

In a similar way we conclude the following inequality for the function

$$h\left({\left[\sum _{i=1}^n\frac{p_i+q_i}{2}\varphi \left(\frac{q_i}{\frac{p_i+q_i}{2}},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)$$

(35)

and we have

$$h\left(\sum _{i=1}^n\frac{p_i+q_i}{2}\varphi \left(\frac{q_i}{\frac{p_i+q_i}{2}},\alpha \right)\right)\le \sum _{i=1}^n\frac{p_i+q_i}{2}(h\circ \varphi )\left(\frac{q_i}{\frac{p_i+q_i}{2}},\alpha \right).$$

(36)

Then combining (33)–(36) and the definition (8) with the proper transformations we obtain the inequality

$$\mathrm{Jen}S{M}_{\alpha ,\beta }^{h,\varphi }(\mathbf{p},\mathbf{q})\le \frac{{\displaystyle {\displaystyle \sum }_{i=1}^n}(p_i+q_i)\left[(h\circ \varphi )\left(\frac{2p_i}{p_i+q_i},\alpha \right)+(h\circ \varphi )\left(\frac{2q_i}{p_i+q_i},\alpha \right)\right]-4}{4(\beta -1)}$$

(37)

which is the upper bound of the generalized $(h,\varphi )$ Jensen–Sharma–Mittal divergence.

When we take into account (32) and (37), then we obtain (26). □

Corollary 2.

When we substitute for $\varphi \left(\frac{2p_i}{p_i+q_i},\alpha \right)={\left(\frac{2p_i}{p_i+q_i}\right)}^{\alpha }$ and for $\varphi \left(\frac{2q_i}{p_i+q_i},\alpha \right)={\left(\frac{2q_i}{p_i+q_i}\right)}^{\alpha }$ then from (26) we obtain the inequalities for Jensen–Sharma–Mittal h–divergence:

$$\frac{1}{2(\alpha -1)}log\prod _{i=1}^n{\left[\frac{4p_i{q}_i}{{(p_i+q_i)}^2}\right]}^{\frac{\alpha (p_i+q_i)}{2}}\le \mathrm{Jen}S{M}_{\alpha ,\beta }^h(\mathbf{p},\mathbf{q})\le$$

$$\frac{{\displaystyle {\displaystyle \sum }_{i=1}^n}(p_i+q_i)\left[h\left({\left(\frac{2p_i}{p_i+q_i}\right)}^{\alpha }\right)+h\left({\left(\frac{2q_i}{p_i+q_i}\right)}^{\alpha }\right)\right]-4}{4(\beta -1)}.$$

Remark 5.

It could be seen that the lower bounds for both Jeffreys (25) and Jensen (32) Sharma–Mittal $(h,\varphi )$ divergences are independent of the function h.

Remark 6.

Taking into account the inequality (10) we obtain the alternative upper bound for the Jensen–Sharma–Mittal and the lower bound for the Jeffreys–Sharma–Mittal generalized $(h,\varphi )$ divergences, respectively.

$$\mathrm{Jen}S{M}_{\alpha ,\beta }^{h,\varphi }(\mathbf{p},\mathbf{q})\le \frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)+h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}{q}_i\varphi \left(\frac{p_i}{q_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)-2}{2(\beta -1)},$$

$$\mathrm{Jef}S{M}_{\alpha ,\beta }^{h,\varphi }(\mathbf{p},\mathbf{q})\ge \frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}\frac{p_i+q_i}{2}\varphi \left(\frac{2p_i}{p_i+q_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)-1}{\beta -1}+$$

$$\frac{h\left({\left[{\displaystyle {\displaystyle \sum }_{i=1}^n}\frac{p_i+q_i}{2}\varphi \left(\frac{2q_i}{p_i+q_i},\alpha \right)\right]}^{\frac{1-\beta }{1-\alpha }}\right)-1}{\beta -1}.$$

4. Applications

In this section we show how our theory works.

4.1. Bounds for Sharma–Mittal Divergences

For the functions $h\left(t\right)=t$, $\varphi (t,\alpha )=t^{\alpha }$ and based on Theorems 1 and 3 we obtain the lower and upper bounds for Jeffreys–Sharma–Mittal and Jensen–Sharma–Mittal divergences, respectively, as follows

$$\frac{1}{\alpha -1}log\prod _{i=1}^n{\left(\frac{p_i}{q_i}\right)}^{\alpha (q_i-p_i)}\le \mathrm{Jef}S{M}_{\alpha ,\beta }(\mathbf{p},\mathbf{q})\le \frac{{\displaystyle {\displaystyle \sum }_{i=1}^n}{\left(p_i{q}_i\right)}^{\alpha }\left(q_i^{1-2\alpha }+p_i^{1-2\alpha }\right)-2}{\beta -1}$$

(38)

$$\frac{1}{2(\alpha -1)}log\prod _{i=1}^n{\left(\frac{2\sqrt{p_i{q}_i}}{p_i+q_i}\right)}^{\alpha (p_i+q_i)}\le \mathrm{Jen}S{M}_{\alpha ,\beta }(\mathbf{p},\mathbf{q})\le \frac{{\displaystyle {\displaystyle \sum }_{i=1}^n}{\left(\frac{p_i+q_i}{2}\right)}^{1-\alpha }\left(p_i^{\alpha }+q_i^{\alpha }\right)-2}{2(\beta -1)}.$$

(39)

Remark 7.

The above lower bounds (38) and (39) are the same for Rényi types divergences because they are independent of the parameter β which in that case approaches 1.

Remark 8.

Substituting different values for the parameters α, β, such that $1<\beta \le \alpha$ and taking into account the assumptions from the Theorems 1 and 3 about the functions h and ϕ we could formulate new types of divergences and related inequalities which are based on the generalized $(h,\varphi )$ Sharma–Mittal divergence.

4.2. Bounds for Tsallis Divergences

When we make the same assumptions as for Sharma–Mittal divergences with additional that $\beta =\alpha$ we obtain the bounds for Tsallis type divergences as follows

$$\frac{1}{\alpha -1}log\prod _{i=1}^n{\left(\frac{p_i}{q_i}\right)}^{\alpha (q_i-p_i)}\le \mathrm{Jef}{T}_{\alpha }(\mathbf{p},\mathbf{q})\le \frac{{\displaystyle {\displaystyle \sum }_{i=1}^n}{\left(p_i{q}_i\right)}^{\alpha }\left(q_i^{1-2\alpha }+p_i^{1-2\alpha }\right)-2}{\alpha -1}$$

$$\frac{1}{2(\alpha -1)}log\prod _{i=1}^n{\left(\frac{2\sqrt{p_i{q}_i}}{p_i+q_i}\right)}^{\alpha (p_i+q_i)}\le \mathrm{Jen}{T}_{\alpha }(\mathbf{p},\mathbf{q})\le \frac{{\displaystyle {\displaystyle \sum }_{i=1}^n}{\left(\frac{p_i+q_i}{2}\right)}^{1-\alpha }\left(p_i^{\alpha }+q_i^{\alpha }\right)-2}{2(\alpha -1)}.$$

4.3. Bounds for Kullback–Leibler Divergences

When we have the same situation as in case of Tsallis divergence that is $h\left(t\right)=t$, $\varphi (t,\alpha )=t^{\alpha }$, $\alpha =\beta$ and additionally both $\alpha$ and $\beta$ approach 1 then we obtain new upper bounds for Jeffreys and Jensen–Shannon divergences, respectively.

$$\mathrm{Jef}S(\mathbf{p},\mathbf{q})\le \sum _{i=1}^n(p_i+q_i)log{p}_i{q}_i,$$

$$\mathrm{Jen}S(\mathbf{p},\mathbf{q})\le \sum _{i=1}^n\left[p_{i}log{p}_i+q_{i}log{q}_i-(p_i+q_i)log(p_i+q_i)\right].$$

The last inequality is equivalent to $\mathrm{Jen}S(\mathbf{p},\mathbf{q})\ge 2log2$.

5. Summary

In this paper, new types of entropy have been defined, which are generalizations of others known and used so far in information theory.

The manuscript deals more with issues in the field of pure mathematics, therefore the standard axioms of entropy used in thermodynamics could, in this case, be extended by other assumptions and properties.

These divergences have been introduced for new physical interpretations which could be generated.

Generalized Sharma–Mittal and consequently Jensen–Sharma–Mittal and Jeffrey–Sharma–Mittal divergences have been defined for obtaining better estimates for known entropies, which will allow to more accurately determination of the dispersion measure of different distributions.

The derived inequalities have both upper and lower limits for the considered f-divergences. As a consequence, we obtain specific estimates for some new order measures. Hence they provide much wider interpretation possibilities in comparing probability distributions in the sense of mutual distances in different spaces.

In the era of advancing quantum mechanics, scientists are striving to build a quantum computer with very high computing power. The obtained results, despite their mathematical and analytical complexity, will very quickly generate specific numerical intervals which are an estimation of new introduced entropies. Therefore, such results as in this paper will be very useful in developing information theory issues.

This work is from the area of pure mathematics, therefore it is more theoretical than practical and makes it possible to find the existing known entropies by means of new defined generalizations. These generalizations can be used for interpreting various physical phenomena. The aim of this manuscript was to provide some new theoretical solutions for physicists who, with their knowledge and experience, will be able to look for new applications.