On Third Hankel Determinant for Certain Subclass of Bi-Univalent Functions

Abstract

This study presents a subclass $\mathcal{S}\left(\beta \right)$ of bi-univalent functions within the open unit disk region $D$. The objective of this class is to determine the bounds of the Hankel determinant of order 3, (${Ⱨ}_3\left(1\right)$). In this study, new constraints for the estimates of the third Hankel determinant for the class $\mathcal{S}\left(\beta \right)$ are presented, which are of considerable interest in various fields of mathematics, including complex analysis and geometric function theory. Here, we define these bi-univalent functions as $\mathcal{S}\left(\beta \right)$ and impose constraints on the coefficients ${│a}_n│$. Our investigation provides the upper bounds for the bi-univalent functions in this newly developed subclass, specifically for n = 2, 3, 4, and 5. We then derive the third Hankel determinant for this particular class, which reveals several intriguing scenarios. These findings contribute to the broader understanding of bi-univalent functions and their potential applications in diverse mathematical contexts. Notably, the results obtained may serve as a foundation for future investigations into the properties and applications of bi-univalent functions and their subclasses.

1. Introduction

Let $A$ indicate the collection of functions $f$ analytic in the open unit disk $D=\{\mathbb{ᶎ}:\mathbb{ᶎ}\in \mathbb{C} \mathrm{a}\mathrm{n}\mathrm{d}\left|\mathsf{ᶎ}\right|<1\}$. An analytic function $f\in A$ has Taylor series expansion of the form:

$$f\left(\mathsf{ᶎ}\right)=\mathsf{ᶎ}+{\sum }_{n=2}^{\infty }{a}_n{\mathsf{ᶎ}}^n, (\mathsf{ᶎ}\in D).$$

(1)

The class of all functions in $A$ which are univalent in $D$ is denoted by $S$. The Koebe-One-Quarter Theorem [1] ensures that the image of $D$ under each $f\in S$ contains a disk of radius $\frac{\mathbf{1}}{\mathbf{4}}$. Obviously, for each $f\in S$ there exists an inverse function $f^{-1}$ satisfying $f^{-1}\left(f\left(\mathsf{ᶎ}\right)\right)=\mathsf{ᶎ}$ and $f\left(f^{-1}\left(w\right)\right)=w$, $\left(\left|w\right|<r_{°}\left(f\right),r_{°}\left(f\right)\ge \frac{1}{4})\right)$, where

$$g\left(w\right)=f^{-1}\left(w\right)=w-a_2{w}^2+\left(2a_2^2-a_3\right)w^3-\left(5a_2^3-5a_2{a}_3+a_4\right)w^4+\cdots , (w\in D)$$

(2)

A function $f\in \sum$ is said to be bi-univalent in $D$ if both $f\left(\mathsf{ᶎ}\right)$ and $f^{-1}\left(\mathsf{ᶎ}\right)$ are univalent in $D$.

In 1967, Lewin [2] obtained a coefficient bound that is given by $\left|a_2\right|<1.51$ for all function $f\in \sum$ of the form (1), and he looked at the class $\sum$ of bi-univalent functions in $D$. In 1967, Clunie and Brannan [3] conjectured that $\left|a_2\right|\le \sqrt{2}$ for $f\in \sum$. After that, Netanyahu [4] proved that $\left|a_2\right|=\frac{ 4}{3}$. In 1985, Kedzierawski [5] stated that Brannan–Clunie conjectured for bi-starlike function. Brannan and Taha [6] gained evaluation estimates on the initial coefficients $\left|a_2\right|$ as well as $\left|a_3\right|$ for functions in the classes of bi-starlike functions of order $\rho$ denoted by $E_{\mathsf{\Omega }}^{*} \left(\rho \right)$ and bi-convex functions of order $\rho$ symbolled by $Y_{\mathsf{\Omega }}\left(\rho \right).$ For all of the function classes, $E_{\mathsf{\Omega }}^{*} \left(\rho \right)$ and $Y_{\mathsf{\Omega }}\left(\rho \right)$, non-sharp estimates on the first two Taylor–Maclaurin coefficients were found in these subclasses (see [7,8,9,10]). Several authors introduced initial Maclaurin coefficients bounds for subclasses of bi-univalent functions (see [11,12]). Many researchers ([11,13,14]) have studied numerous curious subclasses of the bi-univalent function class $\mathsf{\Omega }$ and observed non-sharp bounds on the first two Taylor–Maclaurin coefficients. As well as this, the coefficient problem for all of the Taylor–Maclaurin coefficients $\left|a_n\right|$, n = 3,4,... is as yet an open problem ([2]). Also, let $\mathcal{P}$ represent the class of analytic functions $\mathcal{p}$ that are normalized by the condition:

$$\mathcal{p}\left(\mathsf{ᶎ}\right)=1+{\mathcal{p}}_1\mathsf{ᶎ}+{\mathcal{p}}_2{\mathsf{ᶎ}}^2+\dots , Re\left(\mathcal{p}\left(\mathsf{ᶎ}\right)\right)>0,\mathsf{ᶎ}\in D.$$

Noonan and Thomas [15] defined the $q^{th}$ Hankel determinant of $f$, in 1976 for $n\ge 1$ and $q\ge 1$ by

$${\mathsf{Ⱨ}}_{\mathit{q}}\left(\mathit{n}\right)=\left|\begin{array}{ccccc}{\mathit{a}}_{\mathit{n}}& & {\mathit{a}}_{\mathit{n}+\mathbf{1}}& \cdots & {\mathit{a}}_{\mathit{n}+\mathit{q}-\mathbf{1}}\\ {\mathit{a}}_{\mathit{n}+\mathbf{1}}& & {\mathit{a}}_{\mathit{n}+\mathbf{2}}& \cdots & {\mathit{a}}_{\mathit{n}+\mathit{q}}\\ ⋮& & ⋮& & ⋮\\ {\mathit{a}}_{\mathit{n}+\mathit{q}-\mathbf{1}}& & {\mathit{a}}_{\mathit{n}+\mathit{q}}& \cdots & {\mathit{a}}_{\mathit{n}+\mathbf{2}\mathit{q}-\mathbf{2}}\end{array}\right|,\left({\mathit{a}}_{\mathbf{1}}=\mathbf{1}\right).$$

For $q=2$ and $n=1$, we know that the function ${Ⱨ}_2\left(1\right)=a_3-a_2^2$. The second Hankel determinant ${\mathsf{Ⱨ}}_{\mathbf{2}}\left(\mathbf{2}\right)$ is defined as $\left|{Ⱨ}_2\left(2\right)\right|=\left|{a_{2}a}_4-a_3^2\right|$ for the classes of bi-starlike and bi-convex ([16,17,18,19]). Al-Ameedee et al. [20], studied the second Hankel determinant for certain subclasses of bi-univalent functions. Also Atshan et al. [21], discussed the Hankel determinant of m-fold symmetric bi-univalent functions using a new operator. Fekete and Szegö [22] examined the Hankel determinant of $f$ as

$${\mathsf{Ⱨ}}_{\mathbf{2}}\left(\mathbf{1}\right)=\left|\begin{array}{cc}{\mathit{a}}_{\mathbf{1}}& {\mathit{a}}_{\mathbf{2}}\\ {\mathit{a}}_{\mathbf{2}}& {\mathit{a}}_{\mathbf{3}}\end{array}\right|={\mathit{a}}_{\mathbf{1}}{\mathit{a}}_{\mathbf{3}}-{\mathit{a}}_{\mathbf{2}}^{\mathbf{2}}.$$

They developed an earlier study for estimates of $\left|a_3-\mu a_2^2\right|$, where $a_1=1$ and $\mu \in \mathbb{R}$. Furthermore, for example, for those of $\left|a_3-\mu a_2^2\right|$ see [23], and third Hankel determinant, these functions are studied by [16,24,25,26] functional, given by

$${\mathsf{Ⱨ}}_{\mathbf{3}}\left(\mathbf{1}\right)=\left|\begin{array}{ccc}{\mathit{a}}_{\mathbf{1}}& {\mathit{a}}_{\mathbf{2}}& {\mathit{a}}_{\mathbf{3}}\\ {\mathit{a}}_{\mathbf{2}}& {\mathit{a}}_{\mathbf{3}}& {\mathit{a}}_{\mathbf{4}}\\ {\mathit{a}}_{\mathbf{3}}& {\mathit{a}}_{\mathbf{4}}& {\mathit{a}}_{\mathbf{5}}\end{array}\right|,\left({\mathit{a}}_{\mathbf{1}}=\mathbf{1}\right) \mathrm{and} (\mathit{n}=\mathbf{1},\mathit{q}=\mathbf{3}).$$

By applying triangle inequality for ${\mathsf{Ⱨ}}_{\mathbf{3}}\left(\mathbf{1}\right)$, we have

$$\left|{Ⱨ}_3\left(1\right)\right|\le \left|a_3\right|\left|a_2{a}_4-a_3^2\right|-\left|a_4\right|\left|a_4-a_2{a}_3\right|+\left|a_5\right|\left|a_3-a_2^2\right|.$$

(3)

Our paper provides a subclass $\mathcal{S}\left(\beta \right)$ of bi-univalent functions within the open unit disk region $D$. The objective of this class is to determine the bounds of the Hankel determinant of order 3, (${Ⱨ}_3\left(1\right)$). In this study, new constraints for the estimates of the third Hankel determinant for the class $\mathcal{S}\left(\beta \right)$ are presented.

The subsequent lemmas are important for establishing our results:

Lemma 1

 ([1]). Consider the class $\mathcal{P}$, which consists of all analytic functions $\mathcal{p}\left(\mathsf{ᶎ}\right)$ which can be represented as

$$\mathcal{p}\left(\mathit{z}\right)=\mathbf{1}+\sum _{\mathit{n}=\mathbf{1}}^{\mathit{\infty }}{\mathcal{p}}_{\mathit{n}}{\mathit{z}}^{\mathit{n}},$$

(4)

with $Re\left(\mathcal{p}\left(\mathsf{ᶎ}\right)\right)>0$ for every $\mathsf{ᶎ}\in D$. Then $\left|{\mathcal{p}}_n\right|\le 2$, for every $n=1,2,\cdots$.

Lemma 2 

([27]). If a function $\mathcal{p}\in \mathcal{P}$ is given by (4), then

$$\mathbf{2}{\mathcal{p}}_{\mathbf{2}}={\mathcal{p}}_{\mathbf{1}}^{\mathbf{2}}+\left(\mathbf{4}-{\mathcal{p}}_{\mathbf{1}}^{\mathbf{2}}\right)\mathit{x}$$

$$\mathbf{4}{\mathcal{p}}_{\mathbf{3}}={\mathcal{p}}_{\mathbf{1}}^{\mathbf{3}}+\mathbf{2}{\mathcal{p}}_{\mathbf{1}}\left(\mathbf{4}-{\mathcal{p}}_{\mathbf{1}}^{\mathbf{2}}\right)\mathit{x}-{\mathcal{p}}_{\mathbf{1}}\left(\mathbf{4}-{\mathcal{p}}_{\mathbf{1}}^{\mathbf{2}}\right){\mathit{x}}^{\mathbf{2}}+\mathbf{2}\left(\mathbf{4}-{\mathcal{p}}_{\mathbf{1}}^{\mathbf{2}}\right)\left(\mathbf{1}-{\left|\mathit{x}\right|}^{\mathbf{2}}\right)\mathsf{ᶎ},$$

for some $x,\mathsf{ᶎ}$ with $\left|x\right|\le 1$ and  $\left|\mathsf{ᶎ}\right|\le 1$.

2. Main Results

Definition 1.

A function $f$ belonging to the class $\sum$, as defined by Equation (1) is considered to be in the class $\mathcal{S}\left(\beta \right)$ if it fulfills the following requirement:

$$Re\left(\frac{\mathsf{ᶎ}{f}^{\prime }\left(\mathsf{ᶎ}\right)}{f\left(\mathsf{ᶎ}\right)}+\mathsf{ᶎ}{f}^{″}\left(\mathsf{ᶎ}\right)\right)>\beta$$

(5)

and

$$Re\left(\frac{w{g}^{\prime }\left(w\right)}{g\left(w\right)}+w{g}^{″}\left(w\right)\right)>\beta ,$$

(6)

where $\left(0<\beta \le 1\right), \mathsf{ᶎ}, w\in D$ and  $g=f^{-1}$.

Theorem 1.

Consider the function $f\left(\mathsf{ᶎ}\right)$ as defined in Equation (1), which is an element of the class $\mathcal{S}\left(\beta \right),$ where $0\le \beta <1$. Then, we have

$$\left|a_2{a}_4-a_3^2\right|\le {\left(1-\beta \right)}^2\left[\frac{208}{1215}{\left(1-\beta \right)}^2+\frac{8}{90}\right].$$

(7)

Proof. 

From (5) and (6), we have

$$\frac{\mathsf{ᶎ}{f}^{\prime }\left(\mathsf{ᶎ}\right)}{f\left(\mathsf{ᶎ}\right)}+\mathsf{ᶎ}{f}^{″}\left(\mathsf{ᶎ}\right)=\beta +\left(1-\beta \right)q\left(\mathsf{ᶎ}\right)$$

(8)

and

$$\frac{w{g}^{\prime }\left(w\right)}{g\left(w\right)}+w{g}^{″}\left(w\right)=\beta +\left(1-\beta \right)q\left(w\right),$$

(9)

where $\left(0\le \beta <1;\mathcal{p},q\in \mathcal{P}\right),\mathsf{ᶎ},w\in D$ and $g=f^{-1}$.

Assuming that there exists $u,v:D\to D$ and $u\left(0\right)=v\left(0\right)=0,\left|u\left(\mathsf{ᶎ}\right)\right|<1, \left|v\left(w\right)\right|<1$ and let the functions $\mathcal{p},q\in \mathcal{P}$, such that

$$\mathcal{p}\left(\mathsf{ᶎ}\right)=\frac{1+u\left(\mathsf{ᶎ}\right)}{1-u\left(\mathsf{ᶎ}\right)}=1+\sum _{n=1}^{\infty }{r}_n{\mathsf{ᶎ}}^n$$

and

$$q\left(\mathsf{ᶎ}\right)=\frac{1+v\left(w\right)}{1-v\left(w\right)}=1+\sum _{n=1}^{\infty }{s}_n{w}^n.$$

It follows that

$$\beta +\left(1-\beta \right)\mathcal{p}\left(\mathsf{ᶎ}\right)=1+\sum _{n=1}^{\infty }(1-\beta )r_n{\mathsf{ᶎ}}^n$$

(10)

and

$$\beta +\left(1-\beta \right)q\left(w\right)=1+\sum _{n=1}^{\infty }\left(1-\beta \right)s_n{w}^n.$$

(11)

Since $f\in \sum$ possesses the Maclurian series defined by (1), noticing the fact that simple computation shows that its inverse $g=f^{-1}$ may be expressed using the expansion given by (2), we have

$$\frac{\mathsf{ᶎ}{f}^{\prime }\left(\mathsf{ᶎ}\right)}{f\left(\mathsf{ᶎ}\right)}+\mathsf{ᶎ}{f}^{″}\left(\mathsf{ᶎ}\right)= 1+3a_2\mathsf{ᶎ}+\left(8a_3-a_2^2\right){\mathsf{ᶎ}}^2+\left(15a_4-3a_2{a}_3+a_2^3\right){\mathsf{ᶎ}}^3$$

$$+\left(24a_5-4a_2{a}_4+4a_3{a}_2^2-2a_3^2-a_2^4\right){\mathsf{ᶎ}}^4+\cdots$$

(12)

and

$$\frac{w{g}^{\prime }\left(w\right)}{g\left(w\right)}+w{g}^{″}\left(w\right)=$$

$$1-3a_{2}w+\left(15a_2^2-8a_3\right)w^2-\left(70a_2^3-72a_2{a}_3+15a_4\right)w^3+$$

$$\left(140a_2{a}_4+315a_2^4-504a_2{a}_3+70a_3^2+24a_3{a}_2^2-24a_5\right)w^4+\cdots .$$

(13)

Now comparing (10) and (12) with the coefficients of $\mathsf{ᶎ}, {\mathsf{ᶎ}}^2, {\mathsf{ᶎ}}^3$ and ${\mathsf{ᶎ}}^4$, we get

$${3a}_2=\left(1-\beta \right)r_1,$$

(14)

$$8a_3-a_2^2=\left(1-\beta \right)r_2,$$

(15)

$$15a_4-3a_2{a}_3+a_2^3=\left(1-\beta \right)r_3$$

(16)

and

$$24a_5-4a_2{a}_4+4a_3{a}_2^2-2a_3^2-a_2^4=\left(1-\beta \right)r_4.$$

(17)

Also comparing (11) and (13) with the coefficients of $w,w^2, w^3$ and $w^4$, we get

$$-3a_2=\left(1-\beta \right)s_1,$$

(18)

$$15a_2^2-8a_3=\left(1-\beta \right)s_2,$$

(19)

$$-\left(70a_2^3-72a_2{a}_3+15a_4\right)=\left(1-\beta \right)s_3$$

(20)

and

$$140a_2{a}_4+315a_2^4-504a_2{a}_3+70a_3^2+24a_3{a}_2^2-24a_5=\left(1-\beta \right)s_4.$$

(21)

From (14) and (18), we have

$$\frac{\left(1-\beta \right)r_1}{3}=a_2=-\frac{\left(1-\beta \right)s_1}{3},$$

(22)

It follows that its

$$r_1=-s_1$$

(23)

Subtracting (15) from (19) and (16) from (20), we get

$$a_3=\frac{{\left(1-\beta \right)}^2{r}_1^2}{9}+\frac{\left(1-\beta \right)\left(r_2-s_2\right)}{16}$$

(24)

and

$$a_4=\frac{{2\left(1-\mathsf{\beta }\right)}^3{\mathrm{r}}_1^3}{405}+\frac{5{\left(1-\mathsf{\beta }\right)}^2{\mathrm{r}}_1\left({\mathrm{r}}_2-{\mathrm{s}}_2\right)}{96}+\frac{\left(1-\mathsf{\beta }\right)\left({\mathrm{r}}_3-{\mathrm{s}}_3\right)}{30}.$$

(25)

Thus, using (22), (24) and (25), we get

$$a_2{a}_4-a_3^2=\frac{1}{288}{\left(1-\beta \right)}^3{r}_1^2\left(r_2-s_2\right)-{\frac{13}{1215}\left(1-\beta \right)}^4{r}_1^4$$

$$+\frac{1}{90}{\left(1-\beta \right)}^2{r}_1\left(r_3-s_3\right)-\frac{1}{256}{\left(1-\beta \right)}^2{\left(r_2-s_2\right)}^2.$$

(26)

According to Lemma 2 and $r_1=-s_1$, we receive

$$r_2-s_2=\frac{4-r_1^2}{2}\left(x-y\right)$$

(27)

and

$$r_3-s_3=\frac{r_1^3}{2}+\frac{\left(4-r_1^2\right)r_1}{2}\left(x+y\right)-\frac{\left(4-r_1^2\right)r_1}{4}\left(x^2+y^2\right)$$

$$+\frac{4-r_1^2}{2}\left[\left(1- {\left|x\right|}^2\right)\mathsf{ᶎ}-\left(1-{\left|y\right|}^2\right)w\right],$$

(28)

for some $x,y,\mathsf{ᶎ}$ and $w$ with $\left|x\right|\le 1,\left|y\right|\le 1$ and $\left|w\right|\le 1$.

Given that $\mathcal{p}\in \mathcal{P}$, we have $\left|r_1\right|\le 2$. Giving $r_1=r$, let us suppose, without loss of generality, that $r\in [0,2]$. Therefore, substituting the expressions (27) and (28) in (26), letting $\tau =\left|x\right|\le 1$ and $\eta =\left|y\right|\le 1$, we receive

$$\left|a_2{a}_4-a_3^2\right|\le E_1+E_2\left(\tau +\eta \right)+E_3\left({\tau }^2+{\eta }^2\right)+E_4{\left(\tau +\eta \right)}^2=E\left(\tau ,\eta \right),$$

where

$$E_1=E_1\left(\beta ,r\right)={(1-\beta )}^2{r}^4\left[\frac{{13\left(1-\beta \right)}^2}{1215}+\frac{1}{180}\right]\ge 0,$$

$$E_2=E_2\left(\beta ,r\right)=\frac{{\left(1-\beta \right)}^2\left(4-r^2\right)r^2}{36}\left(\frac{\left(1-\beta \right)}{16}+\frac{1}{5}\right)\ge 0,$$

$$E_3=E_3\left(\beta ,r\right)=\frac{{\left(1-\beta \right)}^2\left(4-r^2\right)r}{180}(\frac{r}{2}-1)\le 0$$

and

$$E_4=E_4\left(\beta ,r\right)=\frac{{\left(1-\beta \right)}^2{(4-r^2)}^2}{1024}\ge 0.$$

Now, we need to maximize $E\left(\tau ,\eta \right)$ within the closed square $\left[0,1\right]\times \left[\mathrm{0,1}\right]$ for $r\in [0,2]$. Since $E_3\le 0$ and $E_3+2 E_2\ge 0$, we conclude that $r\in \left(\mathrm{0,2}\right)$, $F_{\tau ,\tau }{F}_{\eta ,\eta }-{\left(F_{\tau ,\eta }\right)}^2<0$. Therefore, the function $F$ cannot have a local maximum in the interior of a closed square. Now, we investigate the maximum of $F$ on the boundary of a closed square. When $\tau =0$ and $0\le \eta \le 1$, we have

$$F\left(0,\eta \right)=\theta \left(\eta \right)= E_1+ E_2\eta +\left( E_3+ E_4\right){\eta }^2.$$

Next, we will address the following two cases:

Case 1.

The inequality $E_3+ E_4\ge 0$ holds. For the given conditions of $0\le \eta \le 1$, with any fixed $r$ and $0\le r<2$, it is evident that

$${\theta }^{\prime }\left(\eta \right)=E_2+2\left(E_3+ E_4\right)\eta >0,$$

the function $\theta \left(\eta \right)$ is an increasing function. Therefore, for a fixed value of $r\in \left[\mathrm{0,2}\right]$, the maximum of $\theta \left(\eta \right)$ is found when $\eta =1$ and

$$\mathit{max}\theta \left(\eta \right)=\theta \left(1\right)=E_1+ E_2+E_3{+E}_4.$$

Case 2.

Let $E_3+E_4<0$. Since $2\left(E_3+E_4\right)+E_2\ge 0$ for $0<\eta <1$ with $0<r<2$, it is clear that $2\left(E_3+E_4\right)+E_2<2\left(E_3+E_4\right)\eta +E_2<E_2$ and so $\theta \left(\eta \right)>0$. Hence the maximum of $\theta \left(\eta \right)$ occurs at $\eta =1$ and $0\le \eta \le 1$, we obtain

$$E\left(1,\eta \right)=\phi \left(\eta \right)=\left( E_3+ E_4\right){\eta }^2+\left(E_2+2E_4\right)\eta +E_1+E_2+E_3+E_4.$$

so, from the cases of $E_3+ E_4$, we have

$$\mathit{max}\phi \left(\eta \right)=\phi \left(1\right)=E_1+2E_2+2E_3+4E_4.$$

Since $\theta \left(1\right)\le \phi \left(1\right)$, we get $\mathit{max}\left(E\left(\tau ,\eta \right)\right)=E\left(\mathrm{1,1}\right)$ on the boundary of square $\left[\mathrm{0,1}\right]\times \left[\mathrm{0,1}\right]$. The real function $\mathcal{L}$ on the interval (0,1) is defined as follows:

$$\mathcal{L}\left(r\right)=\mathit{max}\left(E\left(\tau ,\eta \right)\right)=E\left(\mathrm{1,1}\right)=E_1+2E_2+2E_3+4E_4.$$

Now, putting $E_1,E_2,E_3$ and ${\mathrm{E}}_4$ in the function $\mathcal{L}$, we obtain

$$\mathcal{L}\left(r\right)={\left(1-\beta \right)}^2\left[K+M\right],$$

where

$$K=\left[\frac{13{\left(1-\beta \right)}^2}{1215}+\frac{1}{180}\right]r^4$$

and

$$M=\left[\frac{r^2}{60}+\frac{(1-\beta )r^2}{288}-\frac{r}{90}+\frac{(4-r^2)}{256}\right](4-r^2).$$

By elementary calculations, it is found that $\mathcal{L}\left(\mathrm{r}\right)$ is an increasing function of $r$. Hence, the maximum of $\mathcal{L}\left(r\right)$ is obtained when $r=2$ and

$$\mathit{max}\mathcal{L}\left(r\right)=\mathcal{L}\left(2\right)={\left(1-\beta \right)}^2\left[\frac{208}{1215}{\left(1-\beta \right)}^2+\frac{8}{90}\right].$$

This completes the proof. □

Theorem 2.

Let $f\left(\mathsf{ᶎ}\right)\in \mathcal{S}\left(\beta \right), 0\le \beta <1$.Then, we have

$$\left|a_2{a}_3-a_4\right|\le \left\{\begin{array}{ll}8\left(1-\beta \right)\left[\frac{{13\left(1-\beta \right)}^2}{405}+\frac{1}{60}\right],& n\le r\le 2\\ \frac{2}{15}\left(1-\beta \right) ,& 0\le r\le n,\end{array}\right.$$

(29)

where

$$n=\frac{m_3\pm \sqrt{m_3^2-12m_2\left(m_1-m_2\right)}}{3(m_1-m_2)},$$

$$m_1=\left(1-\beta \right)\left[\frac{{13\left(1-\beta \right)}^2}{405}+\frac{1}{60}\right],$$

$$m_2=\left(1-\beta \right)\left[\frac{\left(1-\beta \right)}{16}+\frac{1}{20}\right]$$

and

$$m_3=\frac{1}{30}\left(1-\beta \right).$$

Proof. 

From (22), (24) and (25), we obtain

$$\left|a_2{a}_3-a_4\right|=\left|\frac{{13\left(1-\beta \right)}^3{r}_1^3}{405}-\frac{3{\left(1-\beta \right)}^2{r}_1\left(r_2-s_2\right)}{96}-\frac{\left(1-\beta \right)\left(r_3-s_3\right)}{30}\right|.$$

Lemma 2 implies that we can assume, without any restriction, that $r\in \left[\mathrm{0,2}\right]$, where $r_1=r$, thus for $\sigma =\left|x\right|\le 1$ and $\rho =\left|y\right|\le 1$, we have

$$\left|a_2{a}_3-a_4\right|\le D_1+D_2\left(\sigma +\rho \right)+D_3\left({\sigma }^2+{\rho }^2\right)=D\left(\sigma ,\rho \right),$$

where

$$D_1(\beta ,r)={\left(1-\beta \right)r}^3\left[\frac{{13\left(1-\beta \right)}^2}{405}+\frac{1}{60}\right]\ge 0,$$

$$D_2(\beta ,r)=\left(1-\beta \right)\left(4-r^2\right)r\left[\frac{\left(1-\beta \right)}{32}+\frac{1}{60}\right]\ge 0$$

and

$$D_3\left(\beta ,r\right)=\left(1-\beta \right)\left(4-r^2\right)\left[\frac{r}{120}+\frac{1}{60}\right]\ge 0.$$

Applying the same approach as Theorem 2, we find that the maximum occur at $\sigma =1$ and $\rho =1$ within closed square $\left[\mathrm{0,2}\right]$,

$$\phi \left(r\right)=\mathit{max}\left(D\left(\sigma ,\rho \right)\right)=D_1+2\left(D_2+D_3\right).$$

Substituting the value of $D_1, D_2$ and $D_3$ in $\phi \left(r\right)$, we get

$$\phi \left(r\right)=m_1{r}^3+m_{2}r\left(4-r^2\right)+m_3\left(4-r^2\right),$$

where

$$m_1=\left(1-\beta \right)\left[\frac{{13\left(1-\beta \right)}^2}{405}+\frac{1}{60}\right],$$

$$m_2=\left(1-\beta \right)\left[\frac{\left(1-\beta \right)}{16}+\frac{1}{20}\right]$$

and

$$m_3=\frac{1}{30}\left(1-\beta \right).$$

We have

$${\phi }^{\prime }\left(r\right)=3\left(m_1-m_2\right)r^2-2m_{3}r+4m_2,$$

$${\phi }^{″}\left(r\right)=6\left(m_1-m_2\right)r-2m_3,$$

if $m_1-m_2>0$, that is ${\mathrm{m}}_1>{\mathrm{m}}_2$. Then we observe that ${\mathsf{\phi }}^{\prime }\left(\mathrm{r}\right)>0$. Therefore, $\mathsf{\phi }\left(\mathrm{r}\right)$ is an increasing function in the closed interval $\left[\mathrm{0,2}\right]$. Consequently, the function $\phi \left(r\right)$ gets the maximum value when $r=2$, meaning when

$$\left|a_2{a}_3-a_4\right|\le \phi \left(2\right)=8\left(1-\beta \right)\left[\frac{{13\left(1-\beta \right)}^2}{405}+\frac{1}{60}\right],$$

if $m_1-m_2<0$, let ${\mathsf{\phi }}^{\prime }\left(\mathrm{r}\right)=0$. Then we receive

$$r=n=\frac{m_3\pm \sqrt{m_3^2-12m_2\left(m_1-m_2\right)}}{3(m_1-m_2)},$$

when $n<r\le 2$. Subsequently, we obtain ${\phi }^{\prime }\left(r\right)>0$, which indicates that the function on the closed interval is $\left[\mathrm{0,2}\right]$. Therefore, the function $\phi \left(r\right)$ gets the maximum value at $r=2$, which means the function $\phi \left(r\right)$ is an decreasing function on the closed interval $\left[\mathrm{0,2}\right]$. Therefore, $\phi \left(r\right)$ obtains the maximum value at $r=0$. We receive

$$\left|a_2{a}_3-a_4\right|\le \phi \left(0\right)=\frac{2}{15}\left(1-\beta \right).$$

The proof is complete. □

Theorem 3.

Let $f\left(\mathsf{ᶎ}\right)\in \mathcal{S}\left(\beta \right), 0\le \beta <1$. Then we have

$$\left|a_3-a_2^2\right|\le \frac{1}{4}\left(1-\beta \right),$$

(30)

$$\left|a_3\right|\le \frac{4}{9}{\left(1-\beta \right)}^2+\frac{1}{4}\left(1-\beta \right).$$

(31)

Proof. 

By using (24) and Lemma 1, we obtain (31).

What follows the Fekete-Szegö functional is defined for $\mu \in \mathbb{C}$ and $f\in \mathcal{S}\left(\beta \right)$,

$$a_3-{\mu a}_2^2=\frac{{\left(1-\beta \right)}^2{r}_1^2}{9}\left(1-\mu \right)+\frac{\left(1-\beta \right)\left(r_2-s_2\right)}{16}.$$

By Lemma 1, we receive

$$\left|a_3-{\mu a}_2^2\right|\le \frac{4}{9}{\left(1-\beta \right)}^2\left(1-\mu \right)+\frac{1}{4}\left(1-\beta \right),$$

for $\mu =1,$ we obtain (30). □

Theorem 4.

Let$f\left(\mathsf{ᶎ}\right)\in \mathcal{S}\left(\beta \right), 0\le \beta <1$. Then, we have

$$\left|a_4\right|\le \left(1-\beta \right)\left[\frac{16}{405}{\left(1-\beta \right)}^2+\frac{5}{12}\left(1-\beta \right)+\frac{2}{5}\right],$$

(32)

$$\left|a_5\right|\le {\left(1-\beta \right)}^2\left[\frac{98}{12960}{\left(1-\beta \right)}^2+\frac{545}{432}\left(1-\beta \right)+\frac{36173}{122880}\right]+\frac{1}{6}\left(1-\beta \right).$$

(33)

Proof. 

From (25) and by Lemma 1, we receive (32).

By subtracting (17) from (21), we have

$$48a_5=144a_2{a}_4+20{a_{3}a}_2^2+{72a}_{3+}^{2}316a_2^4-504a_2{a}_3$$

$$+\left(1-\beta \right)\left(r_4-s_4\right).$$

By substituting properly (22), (24) and (25), we have

$$32a_5=\frac{712}{135}{\left(1-\beta \right)}^4{r}_1^4-\frac{56}{3}{r}_1^3-\frac{21}{2}{\left(1-\beta \right)}^2{r}_1\left(r_2-s_2\right)+\frac{131}{36}{\left(1-\beta \right)}^3{r}_1^2\left(r_2-s_2\right)+\frac{8}{5}{\left(1-\beta \right)}^2{r}_1\left(r_3-s_3\right)+\frac{9}{32}{\left(1-\beta \right)}^2{\left(r_2-s_2\right)}^2+\left(1-\beta \right)\left(r_4-s_4\right).$$

By applying Lemma 1, we obtain (33). □

Theorem 5.

Let$f\left(\mathsf{ᶎ}\right)\in \mathcal{S}\left(\beta \right), 0\le \beta <1.$Then we have

$$\left|{Ⱨ}_3\left(1\right)\right|\le \left\{\begin{array}{ll}M{\mathcal{M}}_1-{\mathcal{M}}_2\left(8\left(1-\beta \right)\left[\frac{{13\left(1-\beta \right)}^2}{405}+\frac{1}{60}\right]\right)+{\mathcal{M}}_3{\mathcal{M}}_4, & n\le r\le 2\\ M{\mathcal{M}}_1-\frac{2}{15}\left(1-\beta \right){\mathcal{M}}_2+{\mathcal{M}}_3{\mathcal{M}}_4, & 0\le r\le n,\end{array} \right.$$

(34)

where $\mathcal{M}{,\mathcal{M}}_1,{\mathcal{M}}_2,{\mathcal{M}}_3{,\mathcal{M}}_4$ and $n$ are given by (31), (7), (32), (33), and (30), respectively.

Proof. 

Since

$$\left|{Ⱨ}_3\left(1\right)\right|=a_3\left(a_2{a}_4-a_3^2\right)-a_4\left(a_4-a_2{a}_3\right)+a_5\left(a_3-a_2^2\right),$$

By utilizing the triangle inequality, we receive the result (3).

Substituting $\left|a_3\right|\le \frac{4}{9}{\left(1-\beta \right)}^2+\frac{1}{4}\left(1-\beta \right),$

$$\left|a_2{a}_4-a_3^2\right|\le {\left(1-\beta \right)}^2\left[\frac{208}{1215}{\left(1-\beta \right)}^2+\frac{8}{90}\right],$$

$$\left|a_4\right|\le \left(1-\beta \right)\left[\frac{16}{405}{\left(1-\beta \right)}^2+\frac{5}{12}\left(1-\beta \right)+\frac{2}{5}\right],$$

$$\left|a_3-a_2^2\right|\le \frac{1}{4}\left(1-\beta \right)$$

and

$$\left|a_3-a_2^2\right|\le \frac{1}{4}\left(1-\beta \right)$$

in

$$\left|{\mathrm{Ⱨ}}_3\left(1\right)\right|\le \left|a_3\right|\left|a_2{a}_4-a_3^2\right|-\left|a_4\right|\left|a_4-a_2{a}_3\right|+\left|a_5\right|\left|a_3-a_2^2\right|,$$

we obtain (34).

The proof is complete. □

3. Conclusions

This article presented a comprehensive investigation of the third Hankel determinant H3(1) for a certain subclass of bi-univalent functions, $\mathcal{S}\left(\beta \right)$. This subclass is of significant interest in various mathematical fields, including complex analysis and geometric function theory. We defined the bi-univalent functions $\mathcal{S}\left(\beta \right)$ and imposed constraints on the coefficients ${│a}_n│$. Our findings provided the upper bounds for the bi-univalent functions in this newly developed subclass, specifically for n = 2, 3, 4, and 5. Furthermore, we advanced the understanding of these functions by deriving the third Hankel determinant for this particular class, which revealed several intriguing scenarios. This achievement led to the improvement of the bound of the third Hankel determinant for the class of bi-univalent functions $\mathcal{S}\left(\beta \right)$. Our study contributes to the broader understanding of bi-univalent functions, their subclasses, and their potential applications in diverse mathematical contexts. The results obtained may serve as a foundation for future investigations into the properties and applications of bi-univalent functions and their subclasses. Future research endeavors could explore further refinements of the bounds, as well as examine other subclasses of bi-univalent functions to uncover novel insights into their characteristics and potential applications. Ultimately, this study paves the way for a deeper exploration of the fascinating world of bi-univalent functions and their role in the realm of mathematics.