Properties of a Subclass of Analytic Functions Defined by Using an Atangana–Baleanu Fractional Integral Operator

Abstract

The Atangana–Baleanu fractional integral and multiplier transformations are two functions successfully used separately in many recently published studies. They were previously combined and the resulting function was applied for obtaining interesting new results concerning the theories of differential subordination and fuzzy differential subordination. In the present investigation, a new approach is taken by using the operator previously introduced by applying the Atangana–Baleanu fractional integral to a multiplier transformation for introducing a new subclass of analytic functions. Using the methods familiar to geometric function theory, certain geometrical properties of the newly introduced class are obtained such as coefficient estimates, distortion theorems, closure theorems, neighborhoods and the radii of starlikeness, convexity, and close-to-convexity of functions belonging to the class. This class may have symmetric or assymetric properties. The results could prove interesting for future studies due to the new applications of the operator and because the univalence properties of the new subclass of functions could inspire further investigations having it as the main focus.

1. Introduction

Fractional calculus is used in many research fields due to its numerous and diverse applications. Previous papers [1,2] discuss the history of fractional calculus and provide references to its many applications in science and engineering. Applications of fractional calculus are given in [3], where a novel fractional chaotic system including quadratic and cubic nonlinearities is introduced and investigated by taking into account the Caputo derivative for the fractional model and the fractional Routh–Hurwitz criteria for studying the stability of the equilibrium points. Fractional calculus theory is used to investigate the motion of a beam on an internally bent nanowire in [4] and a new and general fractional formulation is presented in order to investigate the complex behaviours of a capacitor microphone dynamical system in [5].

Owa [6] and Owa and Srivastava [7] applied fractional integral calculus for a function that gives new possibilities in studying the function’s properties. Atangana and Baleanu [8] generalized the fractional integral, which was studied by many researchers [9,10,11,12,13]. The fractional integral was investigated in its relation to Mittag–Leffler functions by many authors (see for example [14,15,16]), connected to Bessel functions and to different operators [17].

The definition given by Atangana–Baleanu can be extended to complex values of the order of differentiation $\nu$ by using analytic continuation.

Introducing and studying new classes of univalent functions generates very interesting results and we can find only a few, very recent studies regarding this, such as new subclasses for bi-univalent functions [18,19] and classes of functions introduced using operators [20]. We have previously used fractional integrals for introducing new subclasses of functions [21], and, motivated by the interesting results obtained, we have decided to apply the operator introduced by applying the Atangana–Baleanu fractional integral to a multiplier transformation for defining a new subclass of functions.

In the next section, a new subclass of analytic functions is introduced in Definition 4 after we present the notations and definitions used during our investigation. Properties regarding coefficient inequalities for the functions contained in the newly introduced class are obtained in Section 3 of the paper. Distortion bounds for functions from the class and for their derivatives are given in Section 4, and properties regarding the closure of the class are proven in Section 5, considering partial sums of functions from the class, with extreme points of the class also being provided. In Section 6, inclusion relations are obtained for certain values of the parameters involved and neighborhood properties are discussed, while the radii of starlikeness, convexity, and close-to-convexity of the class are obtained in Section 7 of the paper.

2. Preliminaries

$\mathcal{H}\left(U\right)$ represents the class of analytic functions in $U=\{z\in \mathbb{C}:|z|<1\}$, where the open unit disc of the complex plane, $\mathcal{H}\left(a,n\right)$ represents the subclass of $\mathcal{H}\left(U\right)$ of functions having the form $f\left(z\right)=a+a_n{z}^n+a_{n+1}{z}^{n+1}+\cdots$ and ${\mathcal{A}}_n=\{f\in \mathcal{H}\left(U\right):f\left(z\right)=z+a_{n+1}{z}^{n+1}+\cdots ,z\in U\},$ where $\mathcal{A}={\mathcal{A}}_1$.

The special class of starlike functions of the order $\alpha$ is defined as

$${\mathcal{S}}^{*}\left(\alpha \right)=\left\{f\in \mathcal{A}:\mathrm{Re}\frac{z{f}^{\prime }\left(z\right)}{f\left(z\right)}>\alpha ,0\le \alpha <1\right\}$$

and the class of convex functions of the order $\alpha$ is defined as

$$\mathcal{K}\left(\alpha \right)=\left\{f\in \mathcal{A}:\mathrm{Re}\left(\frac{z{f}^{″}\left(z\right)}{f^{\prime }\left(z\right)}+1\right)>\alpha ,0\le \alpha <1\right\}.$$

For introducing the used operator in this paper, the following previously known results are necessary.

Definition 1

([22]). For $f\in \mathcal{A}$, $m\in \mathbb{N}\cup \left\{0\right\}$, $\alpha ,l\ge 0$, the multiplier transformation $I\left(m,\alpha ,l\right)f\left(z\right)$ is defined by the following infinite series

$$I\left(m,\alpha ,l\right)f\left(z\right):=z+\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{1+l}\right)}^m{a}_k{z}^k.$$

We are reminded that the Riemann–Liouville fractional integral ([23]) is defined by the following relation

$${}_c^{RL}{I}_z^{\nu }f\left(z\right)=\frac{1}{\mathsf{\Gamma }\left(\nu \right)}{\int }_c^z{\left(z-w\right)}^{\nu -1}f\left(w\right)dw,\mathrm{Re}\left(\nu \right)>0,$$

which is used in the Atangana–Baleanu fractional integral.

Definition 2

([24]). Let c be a fixed complex number and f be a complex function which is analytic on an open star-domain D centered at c. The extended Atangana–Baleanu integral, denoted by ${}_c^{AB}{I}_z^{\nu }f\left(z\right)$, is defined for any $\nu \in \mathbb{C}$ and any $z\in D\backslash \left\{c\right\}$ by:

$${}_c^{AB}{I}_z^{\nu }f\left(z\right)=\frac{1-\nu }{B\left(\nu \right)}f\left(z\right)+\frac{\nu }{B\left(\nu \right)}{}_c^{RL}{I}_z^{\nu }f\left(z\right).$$

(1)

Proposition 1

([24]). The extended Atangana–Baleanu integral proposed in Definition 2 is:

An analytic function of both $z\in D\backslash \left\{c\right\}$ and $\nu \in \mathbb{C}$, provided f and B are analytic and B is nonzero; identical to the original formula in real case when $0<\nu <1$ and $c<z$ in $\mathbb{R}$.

Therefore, it provides the analytic continuation of the original Atangana–Baleanu integral to complex values of z and ν.

Applying the Atangana–Baleanu fractional integral for $c=0$ to multiplier transformation, a new operator was defined:

Definition 3

([25]). Let $f\in \mathcal{A}$, $m\in \mathbb{N}\cup \left\{0\right\}$, $\alpha ,l\ge 0$, $\nu \in \mathbb{C}$, and any $z\in D\backslash \left\{0\right\}.$ The Atangana–Baleanu fractional integral associated with the multiplier transformation $I\left(m,\alpha ,l\right)f$ is defined by

$${}_0^{AB}{I}_z^{\nu }\left(I\left(m,\alpha ,l\right)f\left(z\right)\right)=\frac{1-\nu }{B\left(\nu \right)}I\left(m,\alpha ,l\right)f\left(z\right)+\frac{\nu }{B\left(\nu \right)}{}_0^{RL}{I}_z^{\nu }I\left(m,\alpha ,l\right)f\left(z\right).$$

After a simple calculation, the following form is obtained for this operator:

$${}_0^{AB}{I}_z^{\nu }\left(I\left(m,\alpha ,l\right)f\left(z\right)\right)=\frac{1-\nu }{B\left(\nu \right)}z+\frac{\nu }{B\left(\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}{z}^{\nu +1}$$

$$+\frac{1-\nu }{B\left(\nu \right)}\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m{a}_k{z}^k+\frac{\nu }{B\left(\nu \right)}\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\frac{\mathsf{\Gamma }\left(k+1\right)}{\mathsf{\Gamma }\left(\nu +k+1\right)}{a}_k{z}^{k+\nu },$$

for the function $f\left(z\right)=z+{\sum }_{k=2}^{\infty }{a}_k{z}^k\in \mathcal{A}$.

In this paper, we define a new class using the operator ${}_0^{AB}{I}_z^{\nu }\left(I\left(m,\alpha ,l\right)f\right)$.

Definition 4.

A function $f\in \mathcal{A}$ is said to be in the class ${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$ if it satisfies the following criterion:

$$\left|\frac{\lambda \left(1-\mu \right)\frac{{}_0^{AB}{I}_z^{\nu }I\left(m,\alpha ,l\right)f\left(z\right)}{z}{+\mu (}_0^{AB}{I}_z^{\nu }{\left(I\left(m,\alpha ,l\right)f\left(z\right)\right)}^{\prime }}{\lambda \left(1-\mu \right)\frac{{}_0^{AB}{I}_z^{\nu }I\left(m,\alpha ,l\right)f\left(z\right)}{z}{+\mu (}_0^{AB}{I}_z^{\nu }{\left(I\left(m,\alpha ,l\right)f\left(z\right)\right)}^{\prime }-\gamma }\right|<\beta ,$$

(2)

where $m\in \mathbb{N}\cup \left\{0\right\},$ $\alpha ,l,\mu \ge 0,$ $\nu \in \mathbb{C},$ $\lambda \in \mathbb{N},$ γ$\in \mathbb{C}\backslash \left\{0\right\},$ $0<\beta \le 1,$ $z\in U\backslash \left\{0\right\}$.

We will study the properties of functions belonging to the defined class regarding coefficient inequality, the distortion, growth, closure, neighborhood, radii of univalent starlikeness, convexity, and close-to-convexity of the order $\delta$, $0\le \delta <1$.

The symmetry properties of the functions used to define an equation or inequality could be investigated to obtain solutions with particular properties. Research about the properties of symmetry for some functions associated with the concept of quantum computing could also be made in a future paper.

3. Properties Regarding Coefficient Inequality

Theorem 1.

The function $f\in \mathcal{A}$ belongs to the class ${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$ if, and only if,

$$\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }\right.$$

$$\left.+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]a_k$$

$$\le \frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)},$$

(3)

where $m\in \mathbb{N}\cup \left\{0\right\},$ $\alpha ,l,\mu \ge 0,$ $\nu \in \mathbb{C},$ $\lambda \in \mathbb{N},$ γ$\in \mathbb{C}-\left\{0\right\},$ $0<\beta \le 1,$ $z\in U\backslash \left\{0\right\}$.

Proof. 

Let $f\in \mathcal{A}$. Assume that inequality (3) holds true. Taking into account that the obtained formulas are long and come out of the page, we have to make some notations. First, denote

$$L=\lambda \left(1-\mu \right)\frac{{}_0^{AB}{I}_z^{\nu }I\left(m,\alpha ,l\right)f\left(z\right)}{z}{+\mu (}_0^{AB}{I}_z^{\nu }{\left(I\left(m,\alpha ,l\right)f\left(z\right)\right)}^{\prime }$$

$$=\frac{\left(1-\nu \right)\left(\lambda +\mu -\lambda \mu \right)}{B\left(\nu \right)}+\frac{\nu \left(\lambda +\mu -\lambda \mu +\mu \nu \right)}{B\left(\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}{z}^{\nu }$$

$$+\frac{1-\nu }{B\left(\nu \right)}\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left(\lambda -\lambda \mu +k\mu \right)a_k{z}^{k-1}$$

$$+\frac{\nu }{B\left(\nu \right)}\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\frac{\mathsf{\Gamma }\left(k+1\right)}{\mathsf{\Gamma }\left(\nu +k+1\right)}\left(\lambda -\lambda \mu +k\mu +\mu \nu \right)a_k{z}^{k+\nu -1}.$$

After making an easy calculation, we find that

$$\left|\frac{\lambda \left(1-\mu \right)\frac{{}_0^{AB}{I}_z^{\nu }I\left(m,\alpha ,l\right)f\left(z\right)}{z}{+\mu (}_0^{AB}{I}_z^{\nu }{\left(I\left(m,\alpha ,l\right)f\left(z\right)\right)}^{\prime }}{\lambda \left(1-\mu \right)\frac{{}_0^{AB}{I}_z^{\nu }I\left(m,\alpha ,l\right)f\left(z\right)}{z}{+\mu (}_0^{AB}{I}_z^{\nu }{\left(I\left(m,\alpha ,l\right)f\left(z\right)\right)}^{\prime }-\gamma }\right|=\left|\frac{L}{L-\gamma }\right|.$$

We make the notation

$$\tilde{L}=\frac{\nu \left(\lambda +\mu -\lambda \mu +\mu \nu \right)}{B\left(\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}{\left|z\right|}^{\nu }$$

$$+\frac{1-\nu }{B\left(\nu \right)}\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left(\lambda -\lambda \mu +k\mu \right)a_k{\left|z\right|}^{k-1}$$

$$+\frac{\nu }{B\left(\nu \right)}\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\frac{\mathsf{\Gamma }\left(k+1\right)}{\mathsf{\Gamma }\left(\nu +k+1\right)}\left(\lambda -\lambda \mu +k\mu +\mu \nu \right)a_k{\left|z\right|}^{k+\nu -1},$$

and applying properties of a modulus function, we get the inequality

$$\left|\frac{L}{L-\gamma }\right|\le \frac{\frac{\left(1-\nu \right)\left(\lambda +\mu -\lambda \mu \right)}{B\left(\nu \right)}+\tilde{L}}{\frac{\left(1-\nu \right)\left(\lambda +\mu -\lambda \mu \right)}{B\left(\nu \right)}-\gamma -\tilde{L}}\le \beta .$$

Considering values of z on a real axis and for $z\to 1^{-}$, we find

$$\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }\right.$$

$$\left.+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]a_k$$

$$\le \frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}.$$

Conversely, assume that $f\in$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$, then we get the following inequality, using the previous notation

$$Re\left\{\frac{\lambda \left(1-\mu \right)\frac{{}_0^{AB}{I}_z^{\nu }I\left(m,\alpha ,l\right)f\left(z\right)}{z}{+\mu (}_0^{AB}{I}_z^{\nu }{\left(I\left(m,\alpha ,l\right)f\left(z\right)\right)}^{\prime }}{\lambda \left(1-\mu \right)\frac{{}_0^{AB}{I}_z^{\nu }I\left(m,\alpha ,l\right)f\left(z\right)}{z}{+\mu (}_0^{AB}{I}_z^{\nu }{\left(I\left(m,\alpha ,l\right)f\left(z\right)\right)}^{\prime }-\gamma }\right\}>-\beta ,$$

written shortly as

$$Re\left\{\frac{L}{L-\gamma }+\beta \right\}>0,$$

equivalently with

$$Re\left\{\frac{\left(\beta +1\right)L-\beta \gamma }{L-\gamma }\right\}>0.$$

(4)

Taking into account that $Re\left(e^{i\theta }\right)=r$ and $Re(-e^{i\theta })\ge -\left|e^{i\theta }\right|=-1$, using the notation

$$\mathsf{\Lambda }=\frac{\left(1-\nu \right)\left(\lambda +\mu -\lambda \mu \right)}{B\left(\nu \right)}-\frac{\nu \left(\lambda +\mu -\lambda \mu +\mu \nu \right)}{B\left(\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}{r}^{\nu }$$

$$-\frac{1-\nu }{B\left(\nu \right)}\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left(\lambda -\lambda \mu +k\mu \right)a_k{r}^{k-1}$$

$$-\frac{\nu }{B\left(\nu \right)}\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\frac{\mathsf{\Gamma }\left(k+1\right)}{\mathsf{\Gamma }\left(\nu +k+1\right)}\left(\lambda -\lambda \mu +k\mu +\mu \nu \right)a_k{r}^{k+\nu -1},$$

the inequality (4) becomes

$$\frac{\left(\beta +1\right)\mathsf{\Lambda }-\beta \gamma }{\mathsf{\Lambda }}>0.$$

Considering $r\to 1^{-}$ and applying the mean value theorem, we obtain the inequality (3), and the proof is complete. □

Corollary 1.

The function $f{\in }_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$ has the property

$$a_k\le \frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]},$$

$k\ge 2.$

4. Properties Regarding Distortion

Theorem 2.

The function $f\in$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$, with $\left|z\right|=r<1$, has the property

$$r-\frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}{r}^2\le \left|f\left(z\right)\right|$$

$$\le r+\frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}{r}^2.$$

The equality holds for the function

$$f\left(z\right)=z+\frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}{z}^2,z\in U.$$

Proof. 

Considering $f\in$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$, taking account relation (3) and

$$\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]$$

is increasing and positive for $k\ge 2$, then we obtain

$${\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]\sum _{k=2}^{\infty }{a}_k$$

$$\le \sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]a_k$$

$$\le \frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)},$$

equivalently with

$$\sum _{k=2}^{\infty }{a}_k\le \frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}.$$

(5)

Applying the properties of the modulus function for

$$f\left(z\right)=z+\sum _{k=2}^{\infty }{a}_k{z}^k,$$

we get

$$r-r^2\sum _{k=2}^{\infty }{a}_k\le r-\sum _{k=2}^{\infty }{a}_k{r}^k\le \left|z\right|-\sum _{k=2}^{\infty }{a}_k{\left|z\right|}^k\le \left|f\left(z\right)\right|$$

$$\le \left|z\right|+\sum _{k=2}^{\infty }{a}_k{\left|z\right|}^k\le r+\sum _{k=2}^{\infty }{a}_k{r}^k\le r+r^2\sum _{k=2}^{\infty }{a}_k,$$

and considering relation (5), we obtain

$$r-\frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}{r}^2\le \left|f\left(z\right)\right|$$

$$\le r+\frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}{r}^2,$$

completing the proof. □

Theorem 3.

The function $f\in$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$, with $\left|z\right|=r<1$, has the property

$$1-\frac{2\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}r\le \left|f^{\prime }\left(z\right)\right|$$

$$\le 1+\frac{2\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}r.$$

The equality holds for the function

$$f\left(z\right)=z+\frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}{z}^2,z\in U.$$

Proof. 

Applying the properties of the modulus function for

$$f^{\prime }\left(z\right)=1+\sum _{k=2}^{\infty }k{a}_k{z}^{k-1},$$

we obtain

$$1-\sum _{k=2}^{\infty }k{a}_k\left|z\right|\le 1-\sum _{k=2}^{\infty }k{a}_k{\left|z\right|}^{k-1}\le |f^{\prime }\left(z\right)|\le 1+\sum _{k=2}^{\infty }k{a}_k{\left|z\right|}^{k-1}\le 1+\sum _{k=2}^{\infty }k{a}_k\left|z\right|.$$

Using relation (5), we get

$$1-\frac{2\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}r\le \left|f^{\prime }\left(z\right)\right|$$

$$\le 1+\frac{2\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}r,$$

and the proof is complete. □

5. Properties Regarding Closure

Theorem 4.

The function h, defined by

$$h\left(z\right)=\sum _{p=1}^q{\mu }_p{f}_p\left(z\right),{\mu }_p\ge 0,z\in U,$$

where the functions $f_p\in$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$, $p=1,2,...,q,$ have the following form

$$f_p\left(z\right)=z+\sum _{k=2}^{\infty }{a}_{k,p}{z}^k,a_{k,p}\ge 0,z\in U,$$

(6)

belongs to the class ${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$, where

$$\sum _{p=1}^q{\mu }_p=1.$$

Proof. 

The function h can be written as

$$h\left(z\right)=\sum _{p=1}^q{\mu }_{p}z+\sum _{p=1}^q\sum _{k=2}^{\infty }{\mu }_p{a}_{k,p}{z}^k=z+\sum _{k=2}^{\infty }\sum _{p=1}^q{\mu }_p{a}_{k,p}{z}^k.$$

Taking into account that the functions $f_p$, $p=1,2,...,q,$ are contained in the class ${}_0^{AB}\mathcal{I}(m,\alpha ,l,$$\nu ,\lambda ,\mu ,\gamma ,\beta )$, applying Theorem 1, we get

$$\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }\right.$$

$$\left.+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]a_{k,p}$$

$$\le \frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}.$$

(7)

In this condition, we have to prove that

$$\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }\right.$$

$$\left.+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]\left(\sum _{p=1}^q{\mu }_p{a}_{k,p}\right)$$

$$\le \sum _{p=1}^q{\mu }_p\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }\right.$$

$$\left.+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]a_{k,p}$$

$$\le \sum _{p=1}^q{\mu }_p\left(\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right)$$

$$=\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}.$$

Hence, the proof is complete. □

Corollary 2.

The function h defined by

$$h\left(z\right)=(1-\xi )f_1\left(z\right)+\xi f_2\left(z\right),0\le \xi \le 1,z\in U,$$

where the functions $f_p,$ $p=1,2,$ written as in relation (6) are contained in the class ${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$, is contained in the class ${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$, too.

Theorem 5.

Considering the functions

$$f_1\left(z\right)=z,$$

and

$$f_k\left(z\right)=z+\frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{z}^k,$$

$k\ge 2,$$z\in U.$

The function f is contained in the class ${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$ if, and only if, it has the following form

$$f\left(z\right)={\mu }_1{f}_1\left(z\right)+\sum _{k=2}^{\infty }{\mu }_k{f}_k\left(z\right),z\in U,$$

with ${\mu }_1\ge 0,$ ${\mu }_k\ge 0,$ $k\ge 2$, and ${\mu }_1+{\sum }_{k=2}^{\infty }{\mu }_k=1.$

Proof. 

Letting the function

$$f\left(z\right)={\mu }_1{f}_1\left(z\right)+\sum _{k=2}^{\infty }{\mu }_k{f}_k\left(z\right)$$

$$=z+\sum _{k=2}^{\infty }\frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{\mu }_k{z}^k,$$

we get

$$\sum _{k=2}^{\infty }\frac{{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}$$

$$·\frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{\mu }_k$$

$$=\sum _{k=2}^{\infty }{\mu }_k=1-{\mu }_1\le 1.$$

Therefore, $f\in$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$.

Conversely, suppose that $f\in$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$.

Setting

$${\mu }_k=\frac{{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{a}_k,$$

and having

$${\mu }_1=1-\sum _{k=2}^{\infty }{\mu }_k,$$

we get

$$f\left(z\right)={\mu }_1{f}_1\left(z\right)+\sum _{k=2}^{\infty }{\mu }_k{f}_k\left(z\right).$$

Hence, the proof is complete. □

Corollary 3.

The extreme points of the class ${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$ are the functions

$$f_1\left(z\right)=z,$$

and

$$f_k\left(z\right)=z+\frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{z}^k,$$

$k\ge 2,$$z\in U.$

6. Properties Regarding Inclusion and Neighborhood

The $\delta$- neighborhood of a function $f\in \mathcal{A}$ is defined by

$$N_{\delta }\left(f\right)=\{g\in \mathcal{A}:g\left(z\right)=z+\sum _{k=2}^{\infty }{b}_k{z}^k\mathrm{and}\sum _{k=2}^{\infty }k|a_k-b_k|\le \delta \},$$

(8)

and for a particular function $e\left(z\right)=z$, we have

$$N_{\delta }\left(e\right)=\{g\in \mathcal{A}:g\left(z\right)=z+\sum _{k=2}^{\infty }{b}_k{z}^k\mathrm{and}\sum _{k=2}^{\infty }k|b_k|\le \delta \}.$$

(9)

A function $f\in \mathcal{A}$ is contained in the class ${}_0^{AB}{\mathcal{I}}^{\zeta }(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$ if there exists a function $h\in$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$, such that

$$\left|\frac{f\left(z\right)}{h\left(z\right)}-1\right|<1-\zeta ,z\in U,0\le \zeta <1.$$

(10)

Theorem 6.

For

$$\delta =\frac{2\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]},$$

then

$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )\subset N_{\delta }\left(e\right).$$

Proof. 

Let $f\in$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$. Using Theorem 1 and taking into account that

$$\sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]$$

$$\ge \sum _{k=2}^{\infty }{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right],$$

for $k\ge 2$, we get

$${\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]\sum _{k=2}^{\infty }{a}_k$$

$$\le \sum _{k=2}^{\infty }{\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]a_k$$

$$\le \frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)},$$

which implies

$$\sum _{k=2}^{\infty }{a}_k\le \frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}.$$

(11)

Applying Theorem 1 in conjunction with (11), we get

$$\sum _{k=2}^{\infty }k{a}_k\le \frac{2\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}=\delta ,$$

by virtue of (8), we obtain $f\in N_{\delta }\left(e\right)$, which completes the proof. □

Theorem 7.

If $h\in$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$ and

$$\zeta =1-\frac{\delta }{2\left(1-\frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}\right)}$$

(12)

then

$$N_{\delta }\left(h\right)\subset {}_0^{AB}{\mathcal{I}}^{\zeta }(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta ).$$

Proof. 

Consider $f\in N_{\delta }\left(h\right)$, relation (8)

$$\sum _{k=2}^{\infty }k|a_k-b_k|\le \delta ,$$

implies

$$\sum _{k=2}^{\infty }|a_k-b_k|\le \frac{\delta }{2}.$$

(13)

Using relation (11), considering that $h\in$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta ),$ we get

$$\sum _{k=2}^{\infty }{b}_k\le \frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}.$$

(14)

Using (13) and (14), we have

$$\left|\frac{f\left(z\right)}{h\left(z\right)}-1\right|\le \frac{{\sum }_{k=2}^{\infty }|a_k-b_k|}{1-{\sum }_{k=2}^{\infty }{b}_k}\le$$

$$\frac{\delta }{2\left(1-\frac{\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}}{{\left(\frac{1+\alpha +l}{l+1}\right)}^m\left[\frac{\lambda +\left(2-\lambda \right)\mu }{\nu }+\frac{2\left(\lambda +\left(2+\nu -\lambda \right)\mu \right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +3\right)}\right]}\right)}=1-\zeta .$$

By relation(10), we obtain $f\in$${}_0^{AB}{\mathcal{I}}^{\zeta }(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$, where $\zeta$ is given by (12). □

7. Properties Regarding Radii of Starlikeness, Convexity, and Close-to-Convexity

Theorem 8.

The function $f\in$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$ is analytic starlike of order δ, $0\le \delta <1,$ in $\left|z\right|<r_1$, with

$$r_1=\underset{k}{inf}{\left\{\frac{\left(1-\delta \right){\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{\left(k-\delta \right)\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]}\right\}}^{\frac{1}{k-1}}.$$

Proof. 

It is sufficient to prove that

$$\left|\frac{z{f}^{\prime }\left(z\right)}{f\left(z\right)}-1\right|\le 1-\delta ,\left|z\right|<r_1.$$

Since

$$\left|\frac{z{f}^{\prime }\left(z\right)}{f\left(z\right)}-1\right|=\left|\frac{{\sum }_{k=2}^{\infty }(k-1)a_k{z}^{k-1}}{1+{\sum }_{k=2}^{\infty }{a}_k{z}^{k-1}}\right|\le \frac{{\sum }_{k=2}^{\infty }(k-1)a_k{\left|z\right|}^{k-1}}{1-{\sum }_{k=2}^{\infty }{a}_k{\left|z\right|}^{k-1}},$$

we have to show that

$$\frac{{\sum }_{k=2}^{\infty }(k-1)a_k{\left|z\right|}^{k-1}}{1-{\sum }_{k=2}^{\infty }{a}_k{\left|z\right|}^{k-1}}\le 1-\delta ,$$

equivalently to

$$\sum _{k=2}^{\infty }(k-\delta )a_k{\left|z\right|}^{k-1}\le 1-\delta .$$

Applying Theorem 1, we get

$${\left|z\right|}^{k-1}\le \frac{\left(1-\delta \right){\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{\left(k-\delta \right)\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]},$$

or

$$\left|z\right|\le {\left\{\frac{\left(1-\delta \right){\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{\left(k-\delta \right)\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]}\right\}}^{\frac{1}{k-1}}.$$

Hence, the proof is complete. □

Theorem 9.

The function $f\in$${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$ is analytic convex of order $\delta ,$ $0\le \delta \le 1,$ in $\left|z\right|<r_2$, with

$$r_2=\underset{k}{inf}{\left\{\frac{\left(1-\delta \right){\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{k\left(k-\delta \right)\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]}\right\}}^{\frac{1}{k-1}}.$$

Proof. 

It is sufficient to prove that

$$\left|\frac{z{f}^{″}\left(z\right)}{f^{\prime }\left(z\right)}\right|\le 1-\delta ,\left|z\right|<r_2.$$

Since

$$\left|\frac{z{f}^{″}\left(z\right)}{f^{\prime }\left(z\right)}\right|=\left|\frac{{\sum }_{k=2}^{\infty }k(k-1)a_k{z}^{k-1}}{1+{\sum }_{k=2}^{\infty }k{a}_k{z}^{k-1}}\right|\le \frac{{\sum }_{k=2}^{\infty }k(k-1)a_k{\left|z\right|}^{k-1}}{1-{\sum }_{k=2}^{\infty }k{a}_k{\left|z\right|}^{k-1}},$$

we have to show that

$$\frac{{\sum }_{k=2}^{\infty }k(k-1)a_k{\left|z\right|}^{k-1}}{1-{\sum }_{k=2}^{\infty }k{a}_k{\left|z\right|}^{k-1}}\le 1-\delta ,$$

$$\sum _{k=2}^{\infty }k(k-\delta )a_k{\left|z\right|}^{k-1}\le 1-\delta ,$$

and applying Theorem 1, we get

$${\left|z\right|}^{k-1}\le \frac{\left(1-\delta \right){\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{k\left(k-\delta \right)\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]},$$

or

$$\left|z\right|\le {\left\{\frac{\left(1-\delta \right){\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{k\left(k-\delta \right)\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]}\right\}}^{\frac{1}{k-1}},$$

and the proof is complete. □

Theorem 10.

The function $f\in$ ${}_0^{AB}\mathcal{I}(m,\alpha ,l,\nu ,\lambda ,\mu ,\gamma ,\beta )$ is analytic close-to-convex of order $\delta ,$ $0\le \delta <1,$ in $\left|z\right|<r_3$, with

$$r_3=\underset{k}{inf}{\left\{\frac{\left(1-\delta \right){\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{k\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]}\right\}}^{\frac{1}{k-1}}.$$

Proof. 

It is sufficient to show that

$$|f^{\prime }\left(z\right)-1|\le 1-\delta ,|z|<r_3.$$

Then

$$|f^{\prime }\left(z\right)-1|=\left|\sum _{k=2}^{\infty }k{a}_k{z}^{k-1}\right|\le \sum _{k=2}^{\infty }k{a}_k{\left|z\right|}^{k-1}.$$

Thus, $|f^{\prime }\left(z\right)-1|\le 1-\delta$ if ${\sum }_{k=2}^{\infty }\frac{k{a}_k}{1-\delta }{\left|z\right|}^{k-1}\le 1$. Using Theorem 1, the inequality holds true if

$${\left|z\right|}^{k-1}\le \frac{\left(1-\delta \right){\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{k\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]}$$

or

$$\left|z\right|\le {\left\{\frac{\left(1-\delta \right){\left(\frac{1+\alpha \left(k-1\right)+l}{l+1}\right)}^m\left[\frac{\lambda +\left(k-\lambda \right)\mu }{\nu }+\frac{\left(\lambda +\left(k+\nu -\lambda \right)\mu \right)\mathsf{\Gamma }\left(k+1\right)}{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +k+1\right)}\right]}{k\left[\frac{\left(\beta -1\right)\left(\lambda +\left(1-\lambda \right)\mu \right)}{\left(\beta +1\right)\nu }-\frac{\beta \gamma B\left(\nu \right)}{\left(\beta +1\right)\left(1-\nu \right)\nu }-\frac{\lambda +\left(1+\nu -\lambda \right)\mu }{\left(1-\nu \right)\mathsf{\Gamma }\left(\nu +2\right)}\right]}\right\}}^{\frac{1}{k-1}}.$$

Hence, the proof is complete. □

8. Conclusions

A new topic is addressed in this paper concerning the operator defined in [25] by applying the Atangana–Baleanu fractional integral for multiplier transformation and presented in Definition 3. This operator was previously used for obtaining differential subordination and fuzzy differential subordination results, and it is used now for introducing and studying a new subclass of functions given in Definition 4. The interesting coefficient estimates obtained in Section 3 of this paper regarding functions from this class could inspire future investigations for studying the Fekete–Szegö problem related to this class, as seen in some very recent papers, [26,27] or a certain order Hankel determinant as done in [28,29]. In Section 4, distortion properties are obtained for the functions from this class and for the derivatives which, connected to the results regarding starlikeness, convexity, and close-to-convexity shown in Section 7, could inspire future studies concerning the geometrical properties of the new subclass of functions. Partial sums of functions from the class are considered in Section 5, proving closure properties of the class; certain inclusion relations concerning the class are proved in Section 6.