Dynamical Complexity of Modified Leslie–Gower Predator–Prey Model Incorporating Double Allee Effect and Fear Effect

Abstract

This contribution concerns studying a realistic predator–prey interaction, which was achieved by virtue of formulating a modified Leslie–Gower predator–prey model under the influence of the double Allee effect and fear effect in the prey species. The initial theoretical work sheds light on the relevant properties of the solution, presence, and local stability of the equilibria. Both analytic and numerical approaches were used to address the emergence of diverse bifurcations, like saddle-node, Hopf, and Bogdanov–Takens bifurcations. It is noteworthy that while making the assumption that the characteristic equation of the Jacobian matrix J has a pair of imaginary roots $C\left(\rho \right)\pm \iota D\left(\rho \right)$, it is sufficient to consider only $C\left(\rho \right)+\iota D\left(\rho \right)$ due to symmetry. The impact of the fear effect on the proposed model is discussed. Numerical simulation results are provided to back up all the theoretical analysis. From the findings, it was established that the initial condition of the population, as well as the phenomena (fear effect) introduced, played a crucial role in determining the stability of the proposed model.

1. Introduction

The ecosystem encompasses numerous interactions between species that are dynamic and complex in nature. One such interaction is the ever-changing relationship between predators and their prey. The study of dynamical behavior of predator–prey interactions is essential for understanding various aspects like ecosystem stability, ecological health, resource management, and the balance of natural systems. Lotka and Volterra [1,2] were the first to develop a model that represented the relationship shared by prey with its predator, which was later developed by various researchers to make the model more realistic. One such proposed model is the Leslie–Gower predator–prey (LGPP) model [3], which incorporates the concept that a predator’s carrying capacity is a function of the prey population, restricting the increase in the predator’s growth rate. The coupled differential equations that govern the LGPP model are as follows:

$$\begin{array}{cc}{\displaystyle \frac{dN}{dT}}\hfill & =rN\left(1-{\displaystyle \frac{N}{K}}\right)-f\left(N\right)P,\hfill \\ {\displaystyle \frac{dP}{dT}}\hfill & =sP\left(1-{\displaystyle \frac{bP}{N}}\right),\hfill \end{array}$$

(1)

with the initial conditions $N\left(0\right)>0$ and $P\left(0\right)>0$. Here, $N\equiv N\left(T\right)$ and $P\equiv P\left(T\right)$ are the densities of the prey and predator at time T, respectively, and $f\left(N\right)$ represents the functional response, depicting the rate at which prey are consumed by the predator per unit time for a given number of prey and predators. The parameters r, s, K, and b are positive and represent the intrinsic growth rate of the prey, intrinsic growth rate of the predator, environmental carrying capacity, and maximal per capita reduction rate, respectively.

The prey population N is not abundant in the environment, so, in times of scarcity, the predator can switch to alternate food sources [4]. So, the model itself is modified in various ways to account for this. This phenomenon can be represented in the model by adding a constant (positive) to the denominator of the Leslie–Gower term ${\displaystyle \frac{bP}{N}}$ [5,6]. So, the model (1) is transformed into the following system:

$$\begin{array}{cc}{\displaystyle \frac{dN}{dT}}\hfill & =rN\left(1-{\displaystyle \frac{N}{K}}\right)-f\left(N\right)P,\hfill \\ {\displaystyle \frac{dP}{dT}}\hfill & =sP\left(1-{\displaystyle \frac{bP}{a_1+N}}\right),\hfill \end{array}$$

(2)

with the initial conditions $N\left(0\right)>0$ and $P\left(0\right)>0$. Here, $a_1$ represents the extent of protection provided by the environment to the predators.

The functional response $f\left(N\right)$ can take various forms, among which Holling type-II $\left({\displaystyle \frac{\alpha N}{a+N}}\right)$ has been studied widely, as it best describes the feeding pattern of many predators by including the handling time they require to consume their prey [6,7,8,9,10]. Introducing the Holling type-II functional response in the above model gives

$$\begin{array}{cc}{\displaystyle \frac{dN}{dT}}\hfill & =rN\left(1-{\displaystyle \frac{N}{K}}\right)-{\displaystyle \frac{\alpha NP}{a_2+N}},\hfill \\ {\displaystyle \frac{dP}{dT}}\hfill & =sP\left(1-{\displaystyle \frac{bP}{a_1+N}}\right),\hfill \end{array}$$

(3)

with the initial conditions $N\left(0\right)>0$ and $P\left(0\right)>0$. Here, $\alpha$ and $a_2$ represent the maximal predator per capita consumption rate and the extent of protection provided by the environment to prey. This model has also been studied under the condition that environmental protections given to the prey and predators are the same, that is, $a_1=a_2=a$ [6,11]. Ji et al. [6] studied how this model will behave in the long term with stochastic perturbation. Gupta and Chandra [11] undertook a bifurcation analysis of a modified LGPP model with non-linear harvesting.

The Allee effect [12], coined by ecologist W.C. Allee, is associated with the occurrence of a decrease in the fitness of the population at low densities. It is often the result of a reduction in cooperative behavior, like mating, herding, and parental care, among individuals of a species. In the literature, this phenomenon is divided into two types: strong and weak. The key that differentiates between these two is the existence of a threshold or critical value. In the weak Allee effect, the absence of a threshold value leads to the growth rate staying positive all the time, while in the strong Allee effect, the growth rate becomes negative as the population goes below the critical value [13,14]. This phenomenon has been analyzed by many researchers, as it plays a crucial role in determining the survival or extinction of a species. The possibility of the existence of more than one Allee effect has also been explored by ecologists, and the joint influence that a few such phenomena have on the species is named the multiple Allee effect [15,16,17]. González-Olivares et al. [18] studied the LGPP model when the Allee effect is introduced in prey. They emphasized how the Allee effect changed the behavior of the original model in terms of the existence of equilibrium points, along with their stability. Cai et al. [19] studied the LGPP model with the additive Allee effect and discovered how a new steady state comes into actuality when this phenomenon is introduced in the model. Feng and Kang [20] focused their interest on studying stability and bifurcation when the Allee effect is introduced both in predator and prey species in the modified LGPP model. Pal and Saha [21] studied a ratio-dependent predator–prey model with the double Allee effect. Singh et al. [22] analyzed a modified LGPP model under the influence of strong and weak Allee effects. They established conditions for all possible equilibrium points and local bifurcations. Rahmi et al. [23] analyzed the Beddington–DeAngelis functional response, memory effect, and double Allee effect in a modified LGPP model. Wang et al. [24] explored a predator–prey model with the double Allee effect and self-diffusion terms. Various researchers studied the Allee effect phenomenon in predator–prey systems, along with other phenomena, like prey refuge [25], delay [26], non-local competition [27], and hunting cooperation [28].

For a very long time, the sole focus of researchers studying the impact that predators have on prey was direct killing. But over time, it was established that the prey population can be affected by just the mere presence of predators [29,30,31]. Predators induce fear in their prey, which leads the prey to display anti-predator behavior in various forms, like foraging, herding, and abandoning high-risk areas [30,31]. So, along with direct predation, indirect interaction with the predator also alters the behavior of prey [32], which makes the study of this phenomenon, the fear effect, crucial.

Wang et al. [33] modeled the fear effect in the predator–prey model and established how the system becomes stable barring the existence of limit cycles with a high level of fear, while a low level of fear leads to multiple limit cycles, leading to bistability. Sarkar and Khajanchi [34] studied the impact that the fear effect has on prey because of the predator in a predator–prey interaction. A ratio-dependent LGPP model with fear and the Allee effect was examined by Li et al. [35]. Chen et al. [36] studied a modified LGPP model with the fear effect. Wu et al. [37] studied the fear effect, along with non-linear harvesting, in the LGPP model. They established the occurrence of two limit cycles, along with saddle-node and Bogdanov–Takens bifurcations of codimension three. A study of the LGPP model incorporating the fear effect was also undertaken, along with various phenomena like harvesting [38], prey refuge [39,40,41], disease transmission [42], and hunting cooperation [43,44].

Singh et al. [22] concluded in their study that the double Allee effect can accelerate the extinction of vulnerable species. Also, Preisser and Bolnick [32] commented on how indirect interaction with the predator, that is, the presence of fear, can alter the behavior of prey. A study where both of these phenomena are explored simultaneously has not been conducted thus far. Therefore, exploring both effects together can uncover complex dynamics of predator and prey interactions. In this article, our aim is to unveil the dynamical complexity of a modified LGPP model when both of these phenomena occur together.

This entire work contains seven sections in total. Section 2 includes the formulation of the proposed model by the incorporation of the double Allee effect and fear effect. It also presents the positivity and boundedness of the model. Following this, Section 3 comprises the computation of the equilibrium points that emerge in the proposed system, along with their stability analysis. Section 4 includes the emergence of various bifurcations, namely, saddle-node, Hopf, and Bogadanov–Takens bifurcations. Section 5 comprises the numerical simulation and the phase portraits for the proposed system. Following this, Section 6 delves into the impact of the Allee effect and fear effect parameters. This work concludes with Section 7, where we discuss the dynamic behavior of the proposed model in detail.

2. Mathematical Model

We studied model (3) with a double Allee effect and fear effect in the prey species, where we have

$$\begin{array}{cc}{\displaystyle \frac{dN}{dT}}\hfill & ={\displaystyle \frac{rN}{1+fP}}\left(1-{\displaystyle \frac{N}{K}}\right)\left({\displaystyle \frac{N-m}{N+n}}\right)-{\displaystyle \frac{\alpha NP}{a+N}},\hfill \\ {\displaystyle \frac{dP}{dT}}\hfill & =sP\left(1-{\displaystyle \frac{bP}{a+N}}\right),\hfill \end{array}$$

(4)

with the initial conditions $N\left(0\right)>0$ and $P\left(0\right)>0$. Here, $-K<m<K$ and $n>0$ represent the Allee threshold and auxiliary parameter with $m>-n$. Parameter m has a significant influence in determining whether the prey population will survive or face extinction. When the population size goes below the threshold value of m, the species will move toward extinction. The parameter f indicates the degree of fear induced in the prey by the predator.

To investigate the model given in system (4) further, we introduce the non-dimensional variables x, y, and t defined as

$$N=Kx,P={\displaystyle \frac{K}{b}}y,t=rT.$$

Using the new change of variables, system (4) becomes

$$\begin{array}{cc}{\displaystyle \frac{dx}{dt}}\hfill & ={\displaystyle \frac{x(1-x)(x-\beta )}{(1+\theta y)(x+\gamma )}}-{\displaystyle \frac{\xi xy}{x+\delta }},\hfill \\ {\displaystyle \frac{dy}{dt}}\hfill & =\rho y\left(1-{\displaystyle \frac{y}{x+\delta }}\right),\hfill \end{array}$$

(5)

with the initial conditions $x\left(0\right)>0$ and $y\left(0\right)>0$. Here, $\beta ={\displaystyle \frac{m}{K}}$, $\gamma ={\displaystyle \frac{n}{K}}$, $\theta ={\displaystyle \frac{fK}{b}}$, $\xi ={\displaystyle \frac{\alpha }{br}}$, $\rho ={\displaystyle \frac{s}{r}}$, and $\delta ={\displaystyle \frac{a}{K}}$. The parameter $\theta$ is directly proportional to the fear effect parameter f. The system is defined on the set $\Omega =\{(x,y)\in R_{+}^2:x\ge 0,y\ge 0\}$.

Positivity and Boundedness

This part of the article demonstrates that the solution $\left(x\right(t),y(t\left)\right)$ of system (5) consistently maintains positivity and boundedness. Ecologically, this is significant to prove, as the population of a species can neither be negative nor infinite.

Lemma 1.

• Under the initial conditions $x\left(0\right)>0$ and $y\left(0\right)>0$, every solution $\left(x\right(t),y(t\left)\right)$ of system (5) is positive.
• Under the initial conditions $x\left(0\right)>0$ and $y\left(0\right)>0$, every solution $\left(x\right(t),y(t\left)\right)$ of system (5) is bounded.

Proof. 

• To prove this, we start by integrating the equations in system (5) under the initial conditions $x\left(0\right)>0$ and $y\left(0\right)>0$ to obtain the solution $\left(x\right(t),y(t\left)\right)$. So, system (5)’s solution can be expressed as

  $$x\left(t\right)=x\left(0\right)exp\left[{\int }_0^t{\displaystyle \frac{(1-x(s\left)\right)\left(x\right(s)-\beta )}{(1+\theta y(s\left)\right)\left(x\right(s)+\gamma )}}-{\displaystyle \frac{\xi y\left(s\right)}{x\left(s\right)+\delta }}ds\right]>0,$$

  (6)

  $$y\left(t\right)=y\left(0\right)exp\left[\rho {\int }_0^{t}y\left(s\right)\left(1-{\displaystyle \frac{y\left(s\right)}{x(s+\delta )}}\right)ds\right]>0.$$

  (7)
  Therefore, we can conclude that every solution starting in the interior of set $\Omega$, that is, $Int(\Omega )$, remains in $Int(\Omega )$ for $t\ge 0$. Along with this, for all future time, any solution trajectory that starts from the positive x-axis (or y-axis) will remain on it. Hence, the set $\Omega$ is an invariant set.
• Under the initial conditions $x\left(0\right)>0$ and $y\left(0\right)>0$, if $\left(x\right(t),y(t\left)\right)$ represents any solution of the system, then we have two cases:
  Case 1: Let us suppose $x\left(0\right)\le 1$ and we make the claim that $x\left(t\right)\le 1$$\forall t\ge 1$. Otherwise, there will exist $t_1,t_2\in {\mathbb{R}}^{+}$ such that $t_2>t_1$, $x\left(t_1\right)=1$, and $x\left(t\right)>1$∀$t\in (t_1,t_2)$. Then, Equation (6) can be written as

  $$\begin{array}{c}x\left(t\right)=x\left(0\right)exp\left[{\int }_0^{t_1}{\displaystyle \frac{(1-x(s\left)\right)\left(x\right(s)-\beta )}{(1+\theta y(s\left)\right)\left(x\right(s)+\gamma )}}-{\displaystyle \frac{\xi y\left(s\right)}{x\left(s\right)+\delta }}ds\right]\hfill \\ \hfill exp\left[{\int }_{t_1}^{t_2}{\displaystyle \frac{(1-x(s\left)\right)\left(x\right(s)-\beta )}{(1+\theta y(s\left)\right)\left(x\right(s)+\gamma )}}-{\displaystyle \frac{\xi y\left(s\right)}{x\left(s\right)+\delta }}ds\right]\\ \hspace{1em}\hspace{1em}\hspace{1em}=x\left(t_1\right)exp\left[{\int }_{t_1}^{t_2}\left[{\displaystyle \frac{(1-x(s\left)\right)\left(x\right(s)-\beta )}{(1+\theta y(s\left)\right)\left(x\right(s)+\gamma )}}-{\displaystyle \frac{\xi y\left(s\right)}{x\left(s\right)+\delta }}\right]ds\right]<x\left(t_1\right),\end{array}$$

  as $\left[{\displaystyle \frac{(1-x(s\left)\right)\left(x\right(s)-\beta )}{(1+\theta y(s\left)\right)\left(x\right(s)+\gamma )}}-{\displaystyle \frac{\xi y\left(s\right)}{x\left(s\right)+\delta }}\right]<0$∀$t\in (t_1,t_2)$, which is a contradiction to our hypothesis. Thus, $x\left(t\right)\le 1$ for all $t\ge 0$.
  Case 2: Now, let us suppose $x\left(0\right)>1$; then, as long as $x\left(t\right)\ge 1$,

  $$x\left(t\right)=x\left(0\right)exp\left[{\int }_0^t{\displaystyle \frac{(1-x(s\left)\right)\left(x\right(s)-\beta )}{(1+\theta y(s\left)\right)\left(x\right(s)+\gamma )}}-{\displaystyle \frac{\xi y\left(s\right)}{x\left(s\right)+\delta }}ds\right]<x\left(0\right),$$

  as $\left[{\displaystyle \frac{(1-x(s\left)\right)\left(x\right(s)-\beta )}{(1+\theta y(s\left)\right)\left(x\right(s)+\gamma )}}-{\displaystyle \frac{\xi y\left(s\right)}{x\left(s\right)+\delta }}\right]<0$ for $x\left(t\right)\ge 1$.
  Hence, based on these two cases, we can deduce that every positive solution satisfies $x\left(t\right)\le max\{x\left(0\right),1\}\equiv M_1$∀$t\ge 0$.
  Next, we proceed to the predator equation of system (5):

  $${\displaystyle \frac{dy}{dt}}=\rho y\left(1-{\displaystyle \frac{y}{x+\delta }}\right)\le \rho y\left(1-{\displaystyle \frac{y}{M_1+\delta }}\right)$$
  So, we have $y\left(t\right)\le max\{y\left(0\right),M_1+\delta \}$∀$t\ge 0$, which completes our proof.

□

3. Existence of Equilibrium Points and Local Stability

This section emphasizes establishing the presence of the equilibria of system (5) and analyzes their local stability. Studying the system in the case of the strong Allee effect $(\beta >0)$, we find that system (5) exhibits axial and interior equilibria.

All possible equilibria that the system can possess are as follows:

• Axial: along with the origin, that is, $E_0=(0,0)$, system (5) exhibits three other axial equilibrium points: $E_{\delta }=(0,\delta )$, $E_1=(1,0)$, and $E_{\beta }=(\beta ,0)$.
• Interior: The abscissa and ordinate of the interior equilibria can be acquired by solving the subsequent equations:

  $$(1+\theta \xi )x^2-(1+\beta -\xi (1+\theta \delta +\theta \gamma ))x+\beta +\xi \gamma (1+\theta \delta )=0$$

  (8)

  $$y=x+\delta$$

  (9)
  The ordinate of the interior equilibrium points is given by (9). The discriminant of Equation (8) is $\Delta \left(\theta \right)={\xi }^2{(\delta -\gamma )}^2{\theta }^2+2\xi (2\gamma \delta +2\beta +(\delta +\gamma )(1+\beta )-\xi (\delta -\gamma ))\theta +{(1+\beta -\xi )}^2-4\beta -4\gamma \xi$. For the ease of calculation, consider ${\Delta }_1=(1+\beta -\xi (1+\theta \delta +\theta \gamma )$,
  ${\Delta }_2=4(1+\theta \xi )(\beta +\xi \gamma +\theta \xi \gamma \delta )$, and ${\theta }_1={\displaystyle \frac{(2\gamma \delta +2\beta +(\delta +\gamma )(1+\beta )-\xi (\delta -\gamma ))-\sqrt{{\Delta }^{*}}}{\xi {(\delta -\gamma )}^2}}$, where ${\Delta }^{*}={(2\gamma \delta +2\beta +(\delta +\gamma )(1+\beta )-\xi (\delta -\gamma ))}^2-{(\delta -\gamma )}^2({(1+\beta -\xi )}^2-4\beta -4\gamma \xi )$. For the feasibility of positive equilibrium points, we must have ${\Delta }_1>0$, $\delta <\gamma$, and ${(1+\beta -\xi )}^2>4\beta +4\gamma \xi$. Under these conditions, system (5) exhibits the following:
  • Two equilibrium points $E_2^{*}=(x_2^{*},y_2^{*})$ and $E_3^{*}=(x_3^{*},y_3^{*})$ if $0<\theta <{\theta }_1$, where $x_2^{*}={\displaystyle \frac{{\Delta }_1+\sqrt{{\Delta }_1^2-{\Delta }_2}}{2(1+\theta \xi )}}$, $x_3^{*}={\displaystyle \frac{{\Delta }_1-\sqrt{{\Delta }_1^2-{\Delta }_2}}{2(1+\theta \xi )}}$, $y_2^{*}=x_2^{*}+\delta$, and $y_3^{*}=x_3^{*}+\delta$.
  • A unique equilibrium point $E^{*}=(x^{*},y^{*})$ if $\theta ={\theta }_1$, where $x^{*}={\displaystyle \frac{{\Delta }_1}{2(1+\theta \xi )}}$ and $y^{*}=x^{*}+\delta$.
  • No equilibrium point if $\theta >{\theta }_1$.

Now, we shift our focus toward studying the local stability of each equilibrium point. For system (5), the Jacobian matrix at any given point E comes out to be

$$J_E=\left[\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right]$$

(10)

where $a_{11}={\displaystyle \frac{x(1+\beta -2x)}{(1+\theta y)(x+\gamma )}}-{\displaystyle \frac{x(1-x)(x-\beta )}{(1+\theta y){(x+\gamma )}^2}}+{\displaystyle \frac{\xi xy}{{(x+\delta )}^2}}+{\displaystyle \frac{(1-x)(x-\beta )}{(1+\theta y)(x+\gamma )}}-{\displaystyle \frac{\xi y}{x+\delta }}$, $a_{12}={\displaystyle \frac{-\theta x(1-x)(x-\beta )}{(x+\gamma ){(1+\theta y)}^2}}-{\displaystyle \frac{\xi x}{x+\delta }}$, $a_{21}={\displaystyle \frac{\rho y^2}{{(x+\delta )}^2}}$, and $a_{22}=\rho -{\displaystyle \frac{2\rho y}{(x+\delta )}}$.

Theorem 1.

The equilibrium point $E_0$ is consistently a saddle, $E_{\delta }$ is consistently an asymptotically stable point, $E_1$ is consistently a saddle, and $E_{\beta }$ is consistently unstable.

Proof. 

From the Jacobian matrix given in (10), we have the following:

• The eigenvalues at point $E_0$ are ${\displaystyle \frac{-\beta }{\gamma }}$ and $\rho$, which clearly indicate that $E_0$ is a saddle.
• The eigenvalues at point $E_{\delta }$ are −$({\displaystyle \frac{\beta }{\gamma (1+\delta \theta )}}+\xi )$ and $-\rho$, which clearly indicate that $E_{\delta }$ is an asymptotically stable point.
• The eigenvalues at point $E_1$ are $-{\displaystyle \frac{1-\beta }{1+\gamma }}$ and $\rho$, which clearly indicate that $E_1$ is a saddle, as $0<\beta <1$.
• The eigenvalues at point $E_{\beta }$ are $\beta {\displaystyle \frac{1-\beta }{\beta +\gamma }}$ and $\rho$, which clearly indicate that $E_{\beta }$ is an unstable point, as $0<\beta <1$.

□

Theorem 2.

The equilibrium points $E_2^{*}$, $E_3^{*}$, and $E^{*}$, if they exist, are stable if $x_2^{*}{\displaystyle \frac{1+\beta -2x_2^{*}-\xi (1+\theta y_2^{*})}{(1+\theta y_2^{*})(x_2^{*}+\gamma )}}+{\displaystyle \frac{\xi x_2^{*}}{x_2^{*}+\delta }}-\rho <0$, consistently a saddle point, and a degenerate singularity, respectively.

Proof. 

The equilibrium points $E_2^{*}$, $E_3^{*}$, and $E^{*}$ are obtained from the subsequent equations:

$${\displaystyle \frac{(1-x)(x-\beta )}{(1+\theta y)(x+\gamma )}}-{\displaystyle \frac{\xi y}{x+\delta }}=0,y=x+\delta .$$

Using the above two equations and the Jacobian matrix in (10), we have the determinant and trace as follows:

$$det\left(J_E\right)={\displaystyle \frac{-\rho x}{(1+\theta y)(x+\gamma )}}(1+\beta -\xi (1+\theta \delta +\theta \gamma )-2(1+\theta \xi )x)$$

and

$$Tr\left(J_E\right)=x{\displaystyle \frac{1+\beta -2x-\xi (1+\theta y)}{(1+\theta y)(x+\gamma )}}+{\displaystyle \frac{\xi x}{x+\delta }}-\rho .$$

It is clear that at $E_2^{*}$, $det\left(J_{E_2^{*}}\right)>0$. Now, the condition $x_2^{*}{\displaystyle \frac{1+\beta -2x_2^{*}-\xi (1+\theta y_2^{*})}{(1+\theta y_2^{*})(x_2^{*}+\gamma )}}+{\displaystyle \frac{\xi x_2^{*}}{x_2^{*}+\delta }}-\rho <0$ leads to $Tr\left(J_{E_2^{*}}\right)<0$. So, $E_2^{*}$ is asymptotically stable. At $E_3^{*}$, $det\left(J_{E_3^{*}}\right)<0$. Therefore, $E_3^{*}$ constitutes a saddle point. And at $E^{*}$, $det\left(J_{E^{*}}\right)=0$. Therefore, it is evident that $E^{*}$ constitutes a degenerate singularity. □

From the aforementioned theorem, we established that the point $E^{*}$ is identified as a degenerate singularity. Hence, we proceed to explore system (5)’s dynamics in the vicinity of this point.

Theorem 3.

If it exists, the positive equilibrium point $E^{*}$ is characterized as follows:

a. 

A saddle node when ${\displaystyle \frac{x^{*}(1+\beta -2x^{*}-\xi (1+\theta y^{*}))}{(1+\theta y^{*})(x^{*}+\gamma )}}+{\displaystyle \frac{\xi x^{*}}{x^{*}+\delta }}\ne \rho$ holds.

b. 

A cusp of codimension two when ${\displaystyle \frac{x^{*}(1+\beta -2x^{*}-\xi (1+\theta y^{*}))}{(1+\theta y^{*})(x^{*}+\gamma )}}+{\displaystyle \frac{\xi x^{*}}{x^{*}+\delta }}=\rho$, ${\overline{\alpha }}_{20}\ne 0$, and $(2\overline{{\alpha }_{20}}-{\overline{\alpha }}_{11})\ne 0$ holds.

Proof. 

Consider the transformation $\widehat{x}=x-x^{*}$ and $\widehat{y}=y-y^{*}$ to shift $E^{*}$ to the origin; then, using a Taylor series, system (5) reduces to

$$\begin{array}{c}\hfill {\displaystyle \frac{d\widehat{x}}{dt}}={\alpha }_{10}\widehat{x}-{\alpha }_{01}\widehat{y}+{\alpha }_{20}{\widehat{x}}^2+{\alpha }_{02}{\widehat{y}}^2+{\alpha }_{11}\widehat{x}\widehat{y}+O\left|{(x,y)}^3\right|\\ \hfill {\displaystyle \frac{d\widehat{y}}{dt}}=\rho \widehat{x}-\rho \widehat{y}-{\displaystyle \frac{\rho }{x^{*}+\delta }}{\widehat{x}}^2-{\displaystyle \frac{\rho }{x^{*}+\delta }}{\widehat{y}}^2+{\displaystyle \frac{2\rho }{x^{*}+\delta }}\widehat{x}\widehat{y}+O\left|{(x,y)}^3\right|\end{array}$$

(11)

where ${\alpha }_{10}={\alpha }_{01}={\displaystyle \frac{\xi \theta x^{*}}{1+\theta y^{*}}}+{\displaystyle \frac{\xi x^{*}}{x^{*}+\delta }}$, ${\alpha }_{20}={\displaystyle \frac{-x^{*}}{(1+\theta y^{*})(x^{*}+\gamma )}}+{\displaystyle \frac{\gamma (1+\beta -\xi (1+\theta y^{*})-2x^{*})}{(1+\theta y^{*}){(x^{*}+\gamma )}^2}}+{\displaystyle \frac{2\xi \delta }{{(x+\delta )}^2}}$, ${\alpha }_{02}={\displaystyle \frac{{\theta }^2\xi x^{*}}{{(1+\theta y^{*})}^2}}$, and ${\alpha }_{11}={\displaystyle \frac{-\theta x^{*}(1-2x^{*}+\beta )}{{(1+\theta y^{*})}^2(x^{*}+\gamma )}}-{\displaystyle \frac{\theta \gamma \xi }{(1+\theta y^{*})(x^{*}+\gamma )}}-{\displaystyle \frac{\xi \delta }{{(x^{*}+\delta )}^2}}$.

Now, the following two cases arise:

a.

If ${\displaystyle \frac{\xi \theta x^{*}}{1+\theta y^{*}}}+{\displaystyle \frac{\xi x^{*}}{x^{*}+\delta }}\ne \rho$, then $Tr\left(J_{E^{*}}\right)\ne 0$ and $det\left(J_{E^{*}}\right)=0$. This implies that one eigenvalue of the Jacobian matrix $J_{E^{*}}$ is zero, while the other is non-zero, which indicates that $E^{*}$ is a saddle node.

b.

If ${\displaystyle \frac{\xi \theta x^{*}}{1+\theta y^{*}}}+{\displaystyle \frac{\xi x^{*}}{x^{*}+\delta }}=\rho$, then system (11) reduces to

$$\begin{array}{cc}{\displaystyle \frac{d\widehat{x}}{dt}}\hfill & =\rho \widehat{x}-\rho \widehat{y}+{\alpha }_{20}{\widehat{x}}^2+{\alpha }_{02}{\widehat{y}}^2+{\alpha }_{11}\widehat{x}\widehat{y}+O\left|{(x,y)}^3\right|,\hfill \\ {\displaystyle \frac{d\widehat{y}}{dt}}\hfill & =\rho \widehat{x}-\rho \widehat{y}-{\displaystyle \frac{\rho }{x^{*}+\delta }}{\widehat{x}}^2-{\displaystyle \frac{\rho }{x^{*}+\delta }}{\widehat{y}}^2+{\displaystyle \frac{2\rho }{x^{*}+\delta }}\widehat{x}\widehat{y}+O\left|{(x,y)}^3\right|.\hfill \end{array}$$

(12)

Introducing variables $\tau =\rho t$, system (12) becomes

$$\begin{array}{cc}{\displaystyle \frac{d\widehat{x}}{d\tau }}\hfill & =\widehat{x}-\widehat{y}+\widehat{{\alpha }_{20}}{\widehat{x}}^2+\widehat{{\alpha }_{02}}{\widehat{y}}^2+\widehat{{\alpha }_{11}}\widehat{x}\widehat{y}+O\left|{(x,y)}^3\right|,\hfill \\ {\displaystyle \frac{d\widehat{y}}{d\tau }}\hfill & =\widehat{x}-\widehat{y}-{\displaystyle \frac{1}{x^{*}+\delta }}{\widehat{x}}^2-{\displaystyle \frac{1}{x^{*}+\delta }}{\widehat{y}}^2+{\displaystyle \frac{2}{x^{*}+\delta }}\widehat{x}\widehat{y}+O\left|{(x,y)}^3\right|,\hfill \end{array}$$

(13)

where $\widehat{{\alpha }_{20}}={\displaystyle \frac{1}{\rho }}{\alpha }_{20}$, $\widehat{{\alpha }_{11}}={\displaystyle \frac{1}{\rho }}{\alpha }_{11}$, and $\widehat{{\alpha }_{02}}={\displaystyle \frac{1}{\rho }}{\alpha }_{02}$.

Taking $x_1=\widehat{x}$ and $x_2=\widehat{x}-\widehat{y}$, system (13) converts to

$$\begin{array}{cc}{\displaystyle \frac{d{x}_1}{d\tau }}\hfill & =x_2+{\overline{\alpha }}_{20}{x}_1^2+\widehat{{\alpha }_{02}}{x}_2^2-{\overline{\alpha }}_{11}{x}_1{x}_2+O\left|{(x_1,x_2)}^3\right|,\hfill \\ {\displaystyle \frac{d{x}_2}{d\tau }}\hfill & ={\overline{\alpha }}_{20}{x}_1^2+(\widehat{{\alpha }_{02}}+{\displaystyle \frac{1}{x^{*}+\delta }})x_2^2-{\overline{\alpha }}_{11}{x}_1{x}_2+O\left|{(x_1,x_2)}^3\right|,\hfill \end{array}$$

(14)

where ${\overline{\alpha }}_{20}=\widehat{{\alpha }_{20}}+\widehat{{\alpha }_{11}}+\widehat{{\alpha }_{02}}$ and ${\overline{\alpha }}_{11}=\widehat{{\alpha }_{11}}+2\widehat{{\alpha }_{02}}$.

Consider the transformation $y_1=x_1$ and $y_2=x_2-{\displaystyle \frac{1}{x^{*}+\delta }}{x}_1{x}_2$; system (14) then reduces to

$$\begin{array}{cc}{\displaystyle \frac{d{y}_1}{d\tau }}\hfill & =y_2+{\overline{\alpha }}_{20}{y}_1^2+\widehat{{\alpha }_{02}}{y}_2^2+(-{\overline{\alpha }}_{11}+{\displaystyle \frac{1}{x^{*}+\delta }})y_1{y}_2+O\left|{(y_1,y_2)}^3\right|,\hfill \\ {\displaystyle \frac{d{y}_2}{d\tau }}\hfill & ={\overline{\alpha }}_{20}{y}_1^2+\widehat{{\alpha }_{02}}{y}_2^2-{\overline{\alpha }}_{11}{y}_1{y}_2+O\left|{(y_1,y_2)}^3\right|.\hfill \end{array}$$

(15)

Consider the final transformation $z_1=y_1-{\displaystyle \frac{1}{2}}(-{\overline{\alpha }}_{11}+{\displaystyle \frac{1}{x^{*}+\delta }})y_1^2+\widehat{{\alpha }_{02}}{y}_2^2$, and

$z_2=y_2+{\overline{\alpha }}_{20}{y}_1^2+O\left|{(z_1,z_2)}^3\right|$; then, system (15) reduces to

$$\begin{array}{cc}{\displaystyle \frac{d{z}_1}{d\tau }}\hfill & =z_2,\hfill \\ {\displaystyle \frac{d{z}_2}{d\tau }}\hfill & ={\overline{\alpha }}_{20}{y}_1^2+\widehat{{\alpha }_{02}}{y}_2^2+(2\overline{{\alpha }_{20}}-{\overline{\alpha }}_{11})y_1{y}_2+O\left|{(z_1,z_2)}^3\right|.\hfill \end{array}$$

(16)

Now, if ${\overline{\alpha }}_{20}\ne 0$ and $(2\overline{{\alpha }_{20}}-{\overline{\alpha }}_{11})\ne 0$, then in the $z_1{z}_2$ plane, the origin presents as a cusp of codimension two, which corresponds to $E^{*}$ being a codimension-two cusp in the $xy$-plane. □

4. Bifurcation Analysis

This section emphasizes analyzing various bifurcations, such as saddle-node, Hopf, and Bogdanov–Takens bifurcations, that system (5) may experience.

4.1. Saddle-Node Bifurcation

In Section 3, we show that if $0<\theta <{\theta }_1$, system (5) possesses two positive equilibria: $E_1^{*}$ and $E_2^{*}$. And when $\theta ={\theta }_1$, the two equilibrium points coincide and give rise to the unique equilibrium point $E^{*}$. Also, system (5) exhibits no equilibrium point when $\theta >{\theta }_1$. This kind of change, that is, the change in the number of equilibrium points in a system, occurs due to an inherent property called saddle-node bifurcation. Consider $\theta ={\theta }^{SN}={\theta }_1$. Treating $\theta$ as a bifurcation parameter, to guarantee the emergence of a saddle-node bifurcation in system (5), we use Sotomayor’s theorem.

Theorem 4.

A saddle-node bifurcation emerges in system (5) around the unique interior equilibrium point $E^{*}=(x^{*},y^{*})$ with respect to parameter θ.

Proof. 

Evidently $det\left(J_{E^{*}}\right)=0$. Thus, the Jacobian matrix $J_{E^{*}}$ has one eigenvalue that is zero. We suppose two eigenvectors, say, V and W, corresponding to matrix’s $J_{E^{*}}$ and $J_{E^{*}}^T$, zero eigenvalue, respectively, then,

$$V=\left[\begin{array}{c}1\\ 1\end{array}\right];W=\left[\begin{array}{c}1\\ -{\displaystyle \frac{1}{\rho }}({\displaystyle \frac{\xi \theta x^{*}}{1+\theta y^{*}}}+{\displaystyle \frac{\xi x^{*}}{x^{*}+\delta }})\end{array}\right].$$

Now, we have $W^T{F}_{\theta }(E^{*},{\theta }^{SN})={\displaystyle \frac{-\xi (x^{*}+\delta )}{(1+\theta y^{*})}}\ne 0$, where $F_{\theta }(E^{*},{\theta }^{SN})=\left[\begin{array}{c}{\displaystyle \frac{-\xi (x^{*}+\beta )}{(1+\theta y^{*})}}\\ 0\end{array}\right]$, and $W^T\left[D^{2}F(E^{*},{\theta }^{SN})(V,V)\right]=-{\displaystyle \frac{2+2\theta \delta +\theta (1+\beta +\xi (1+\theta \delta -\theta \gamma \left)\right)}{{(1+\theta y^{*})}^2(x^{*}+\gamma )}}\ne 0$, where $D^{2}F(E^{*},{\theta }^{SN})=\left[\begin{array}{c}-{\displaystyle \frac{2+2\theta \delta +\theta (1+\beta +\xi (1+\theta \delta -\theta \gamma \left)\right)}{{(1+\theta y^{*})}^2(x^{*}+\gamma )}}\\ 0\end{array}\right]\ne 0$, as $1+\beta -\xi \left(\right(1+\theta \delta +\theta \gamma \left)\right)>0$

Thus, the transversality condition required for a saddle-node bifurcation is satisfied. □

4.2. Hopf Bifurcation

In Section 3, we show that $E_3^{*}$, if it exists, is consistently a saddle point and $E_2^{*}$, if it exists, is stable whenever $x_2^{*}{\displaystyle \frac{1+\beta -2x_2^{*}-\xi (1+\theta y_2^{*})}{(1+\theta y_2^{*})(x_2^{*}+\gamma )}}+{\displaystyle \frac{\xi x_2^{*}}{x_2^{*}+\delta }}<\rho$. Also, $det\left(J_{E_2^{*}}\right)>0$ and $Tr\left(J_{E_2^{*}}\right)=0$ when $x_2^{*}{\displaystyle \frac{1+\beta -2x_2^{*}-\xi (1+\theta y_2^{*})}{(1+\theta y_2^{*})(x_2^{*}+\gamma )}}+{\displaystyle \frac{\xi x_2^{*}}{x_2^{*}+\delta }}=\rho$. Then, a purely imaginary nature is exhibited by the eigenvalues of $J_{E_2^{*}}$. Therefore, $E_2^{*}$ will manifest as either a center or a weak focus.

Theorem 5.

A Hopf bifurcation emerges at equilibrium point $E_2^{*}$ in system (5) if $\rho ={\rho }^{\left[\mathit{hf}\right]}=x_2^{*}{\displaystyle \frac{1+\beta -2x_2^{*}-\xi (1+\theta y_2^{*})}{(1+\theta y_2^{*})(x_2^{*}+\gamma )}}+{\displaystyle \frac{\xi x_2^{*}}{x_2^{*}+\delta }}$, and its direction is supercritical (or subcritical) if $\Lambda <0$ (or $\Lambda >0$).

Proof. 

To establish the emergence of a Hopf bifurcation in system (5), we show that the transversality conditions are fulfilled. Consider $\rho$ as a bifurcation parameter; then, a critical value $\rho ={\rho }^{\left[\mathit{hf}\right]}=x_2^{*}{\displaystyle \frac{1+\beta -2x_2^{*}-\xi (1+\theta y_2^{*})}{(1+\theta y_2^{*})(x_2^{*}+\gamma )}}+{\displaystyle \frac{\xi x_2^{*}}{x_2^{*}+\delta }}$ exists for which $det\left(J_{E_2^{*}}\right)>0$, $Tr\left(J_{E_2^{*}}\right)=0$, and ${\displaystyle \frac{\partial }{\partial \rho }}\left(Tr\left(J_{E_2^{*}}\right)\right)=-1\ne 0$.

Due to the occurrence of Hopf bifurcation, a limit cycle will arise in the system. We now explore the stability of the emergent limit cycles. We start by shifting $E_2^{*}=(x_2^{*},y_2^{*})$ to the origin by considering $\widehat{x}=x-x_2^{*}$ and $\widehat{y}=y-y_2^{*}$. Thus, system (5) is reduced to the following (retaining x and y in place of $\widehat{x}$ and $\widehat{y}$, respectively):

$$\begin{array}{cc}{\displaystyle \frac{dx}{dt}}\hfill & ={\displaystyle \frac{(x+x_2^{*})(1-x-x_2^{*})(x+x_2^{*}-\beta )}{(1+\theta (y+y_2^{*}))(x+x_2^{*}+\gamma )}}-{\displaystyle \frac{\xi (x+x_2^{*})(y+y_2^{*})}{x+x_2^{*}+\delta }}\hfill \\ {\displaystyle \frac{dy}{dt}}\hfill & =\rho (y+y_2^{*})(1-{\displaystyle \frac{y+y_2^{*}}{x+x_2^{*}+\delta }})\hfill \end{array}$$

(17)

System (17) can be represented as $\left[\begin{array}{c}{x}_t\\ y_t\end{array}\right]=J\left[\begin{array}{c}x\\ y\end{array}\right]+\left[\begin{array}{c}f(x,y)\\ g(x,y)\end{array}\right]$, where $J=J_{E_2^{*}}=\left[\begin{array}{cc}{x}_2^{*}{\displaystyle \frac{1+\beta -2x_2^{*}-\xi (1+\theta y_2^{*})}{(1+\theta y_2^{*})(x_2^{*}+\gamma )}}+{\displaystyle \frac{\xi x_2^{*}}{x_2^{*}+\delta }}& -{\displaystyle \frac{\xi \theta x_2^{*}}{1+\theta y_2^{*}}}-{\displaystyle \frac{\xi x_2^{*}}{x_2^{*}+\delta }}\\ \rho & -\rho \end{array}\right]$, $f(x,y)={\alpha }_{20}{x}^2+{\alpha }_{11}xy+{\alpha }_{02}{y}^2+{\alpha }_{30}{x}^3+{\alpha }_{21}{x}^{2}y+{\alpha }_{12}x{y}^2+{\alpha }_{03}{y}^3+\cdots$, $g(x,y)={\beta }_{20}{x}^2+{\beta }_{11}xy+{\beta }_{02}{y}^2+{\beta }_{30}{x}^3+{\beta }_{21}{x}^{2}y+{\beta }_{12}x{y}^2+\cdots$, ${\alpha }_{20}={\displaystyle \frac{-x}{(1+\theta y)(x+\gamma )}}+{\displaystyle \frac{\gamma (1+\beta -2x-\xi (1+\theta y\left)\right)}{(1+\theta y){(x+\gamma )}^2}}+{\displaystyle \frac{\xi \delta }{{(x+\delta )}^2}}$, ${\alpha }_{02}={\displaystyle \frac{{\theta }^2\xi x}{{(1+\theta y)}^2}}$, ${\alpha }_{11}={\displaystyle \frac{-\theta x(1-2x+\beta )}{{(1+\theta y)}^2(x+\gamma )}}-{\displaystyle \frac{\theta \gamma \xi }{(1+\theta y)(x+\gamma )}}-{\displaystyle \frac{\xi \delta }{{(x+\delta )}^2}}$, ${\alpha }_{30}={\displaystyle \frac{\gamma (-1-\beta +2x+\xi (1+\theta y\left)\right)}{(1+\theta y){(x+\gamma )}^3}}-{\displaystyle \frac{\gamma }{(1+\theta y){(x+\gamma )}^2}}-{\displaystyle \frac{\delta \xi }{{(x+\delta )}^3}}$, ${\alpha }_{03}={\displaystyle \frac{{\theta }^3\xi x}{{(1+\theta y)}^3}}$, ${\alpha }_{21}={\displaystyle \frac{\theta x}{{(1+\theta y)}^2(x+\gamma )}}-{\displaystyle \frac{\theta \gamma (1+\beta -2x-\xi (1+\theta y\left)\right)}{{(1+\theta y)}^2{(x+\gamma )}^2}}+{\displaystyle \frac{\xi \delta }{{(x+\delta )}^3}}$, ${\alpha }_{12}={\displaystyle \frac{{\theta }^{2}x(1-2x+\beta )}{{(1+\theta y)}^3(x+\gamma )}}+{\displaystyle \frac{{\theta }^2\gamma \xi }{{(1+\theta y)}^2(x+\gamma )}}$, ${\beta }_{10}=\rho$, ${\beta }_{01}=-\rho$, ${\beta }_{20}=b_{02}={\displaystyle \frac{-\rho }{x+\delta }}$, ${\beta }_{11}={\displaystyle \frac{2\rho }{x+\delta }}$, ${\beta }_{30}={\displaystyle \frac{\rho }{{(x+\delta )}^2}}$, ${\beta }_{03}=0$, ${\beta }_{21}={\displaystyle \frac{-2\rho }{{(x+\delta )}^2}}$, and ${\beta }_{12}={\displaystyle \frac{2\rho }{{(x+\delta )}^2}}$.

The characteristic roots of the Jacobian matrix J are $\lambda \left(\rho \right)=C\left(\rho \right)\pm \iota D\left(\rho \right)$ if $det\left(J\right)>{\left({\displaystyle \frac{1}{2}}Tr\left(J\right)\right)}^2$, where $C\left(\rho \right)={\displaystyle \frac{1}{2}}Tr\left(J\right)$ and $D\left(\rho \right)=\sqrt{det\left(J\right)-{\left({\displaystyle \frac{1}{2}}Tr\left(J\right)\right)}^2}$. $C\left({\rho }^{\left[hf\right]}\right)=0$ and $D\left({\rho }^{\left[hf\right]}\right)=\sqrt{det\left(J\right)}$ if $\rho ={\rho }^{\left[hf\right]}$, which implies that the eigenvalues ${\lambda }_1$ and ${\lambda }_2$ are purely imaginary. For further computation, due to symmetry, it is sufficient to take only one root of the characteristic equation of J: $\lambda =C\left(\rho \right)+\iota D\left(\rho \right)$. We define a matrix $P=\left[\begin{array}{cc}1& 0\\ M& N\end{array}\right]$ such that $\left[\begin{array}{c}1\\ M-\iota N\end{array}\right]$ is the eigenvector of J corresponding to $\lambda =A\left(\rho \right)+\iota B\left(\rho \right)$, where $M={\displaystyle \frac{1}{\frac{\xi \theta x}{1+\theta y}+\frac{\xi x}{x+\delta }}}({\displaystyle \frac{x(1+\beta -\xi (1+\theta y)-2x)}{(1+\theta y)(x+\gamma )}}+{\displaystyle \frac{\xi x}{x+\delta }}-A\left(\rho \right))$ and $N={\displaystyle \frac{B\left(\rho \right)}{\frac{\xi \theta x}{1+\theta y}+\frac{\xi x}{x+\delta }}}$. The inverse of matrix P is $P^{-1}=\left[\begin{array}{cc}1& 0\\ -{\displaystyle \frac{M}{N}}& {\displaystyle \frac{1}{N}}\end{array}\right]$. Consider the transformation $\left[\begin{array}{c}x\\ y\end{array}\right]$= T$\left[\begin{array}{c}u\\ v\end{array}\right]$, where we obtain $x=u$ and $y=Mu+Nv$. Thus, system (17) reduces to

$$\left[\begin{array}{c}{u}_t\\ v_t\end{array}\right]=J\left(\rho \right)\left[\begin{array}{c}u\\ v\end{array}\right]+\left[\begin{array}{c}F(u,v)\\ G(u,v)\end{array}\right],$$

(18)

where $J\left(\rho \right)=\left[\begin{array}{cc}C\left(\rho \right)& -D\left(\rho \right)\\ D\left(\rho \right)& C\left(\rho \right)\end{array}\right]$, $$\begin{gathered} F(u,v)=f(u,Mu+Nv)=({\alpha }_{20}+{\alpha }_{11}M+{\alpha }_{02}{M}^2)u^2+ \\ ({\alpha }_{11}N+2{\alpha }_{02}MN)uv+{\alpha }_{02}N{v}^2+({\alpha }_{30}+{\alpha }_{21}M+{\alpha }_{12}{M}^2+{\alpha }_{03}{M}^3)u^3+ \\ ({\alpha }_{21}N+2{\alpha }_{12}MN+3{\alpha }_{03}{M}^{2}N)u^{2}v+({\alpha }_{12}{N}^2+3{\alpha }_{03}M{N}^2)u{v}^2+{\alpha }_{03}{N}^3{v}^3+\cdots \end{gathered}$$, and $$\begin{gathered} G(u,v)=-{\displaystyle \frac{M}{N}}F(u,v)+{\displaystyle \frac{1}{N}}g(u,Mu+Nv)= \\ {\displaystyle \frac{1}{N}}(({\beta }_{20}+{\beta }_{11}M+{\beta }_{02}{M}^2-{\alpha }_{20}M-{\alpha }_{11}{M}^2-{\alpha }_{02}{M}^2)u^2+({\beta }_{11}N+2{\beta }_{02}MN-{\alpha }_{11}MN-2{\alpha }_{02}{M}^{2}N)uv+ \\ ({\beta }_{02}{N}^2-{\alpha }_{02}MN)v^2+({\beta }_{30}+{\beta }_{21}M+{\beta }_{12}{M}^2+{\beta }_{03}{M}^3-{\alpha }_{30}M-{\alpha }_{21}{M}^2-{\alpha }_{12}{M}^3-{\alpha }_{03}{M}^4)u^3+ \\ ({\beta }_{21}N+2{\beta }_{12}MN+3{\beta }_{03}{M}^{2}N-{\alpha }_{21}MN-2{\alpha }_{12}{M}^{2}N-3{\alpha }_{03}{M}^{3}N)u^{2}v+ \\ ({\beta }_{12}N+3{\beta }_{03}M{N}^2-{\alpha }_{12}M{N}^2-3{\alpha }_{03}{M}^2{N}^2)u{v}^2+({\beta }_{03}{N}^2-{\alpha }_{03}M{N}^3)v^3\cdots ) \end{gathered}$$.

Rewriting system (18) in polar coordinates, we obtain

$$\begin{array}{c}\dot{r}=C\left(\rho \right)r+c\left(\rho \right)r^3+\dots ,\hfill \\ \dot{\varphi }=D\left(\rho \right)+d\left(\rho \right)r^2+\dots \hfill \end{array}$$

(19)

Using Taylor’s expansion, at $\rho ={\rho }^{*}$, system (19) reduces to

$$\begin{array}{cc}\dot{r}\hfill & =C^{\prime }\left({\rho }^{*}\right)(\rho -{\rho }^{*})r+c\left(\rho \right)r^3+\dots \hfill \\ \dot{\varphi }\hfill & =D\left({\rho }^{*}\right)+D^{\prime }\left({\rho }^{*}\right)(\rho -{\rho }^{*})+d\left({\rho }^{*}\right)r^2+\dots \hfill \end{array}$$

(20)

The stability of the Hopf bifurcation relies on the sign of

$$\Lambda =-{\displaystyle \frac{c\left({\rho }^{*}\right)}{C^{\prime }\left({\rho }^{*}\right)}},$$

where $c\left({\rho }^{*}\right)={\displaystyle \frac{1}{16}}{(F_{xxx}+F_{xyy}+G_{xyy}+G_{yyy})}_{(0,0,{\rho }^{*})}+{\displaystyle \frac{1}{16B\left({\rho }^{*}\right)}}{(F_{xy}(F_{xx}+F_{yy})-G_{xy}(G_{xx}+G_{yy})-F_{xx}{G}_{xx}+F_{yy}{G}_{yy})}_{(0,0,{\rho }^{*})}$, $C^{\prime }\left({\rho }^{*}\right)=-1$, $F_{xx}=2({\alpha }_{20}+{\alpha }_{11}M+{\alpha }_{02}{M}^2)$,

$F_{yy}=2{\alpha }_{02}N$, $F_{xy}={\alpha }_{11}N+2{\alpha }_{02}MN$, $F_{xxx}=6({\alpha }_{30}+{\alpha }_{21}M+{\alpha }_{12}{M}^2+{\alpha }_{03}{M}^3)$,

$F_{xyy}=2({\alpha }_{12}{N}^2+3{\alpha }_{03}M{N}^2)$, $G_{xx}={\displaystyle \frac{2}{N}}({\beta }_{20}+{\beta }_{11}M+{\beta }_{02}{M}^2-{\alpha }_{20}M-{\alpha }_{11}{M}^2-{\alpha }_{02}{M}^2)$, $G_{yy}=2({\beta }_{02}N-{\alpha }_{02}M)$, $G_{xy}={\beta }_{11}+2{\beta }_{02}M-{\alpha }_{11}M-2{\alpha }_{02}{M}^2$,

$G_{xxx}={\displaystyle \frac{6}{N}}({\beta }_{30}+{\beta }_{21}M+{\beta }_{12}{M}^2+{\beta }_{03}{M}^3-{\alpha }_{30}M-{\alpha }_{21}{M}^2-{\alpha }_{12}{M}^3-{\alpha }_{03}{M}^4)$, and

$G_{yyy}=6({\beta }_{03}{N}^2-{\alpha }_{03}M{N}^2)$.

After simplification, we have

$$\begin{array}{c}\hfill \Lambda ={\displaystyle \frac{1}{8}}({\alpha }_{30}+{\beta }_{21}+{\displaystyle \frac{B\left({\rho }^{*}\right){\alpha }_{02}}{a_{12}^2}}({\alpha }_{11}-2{\alpha }_{02}{\displaystyle \frac{a_{11}}{a_{12}}}+2{\beta }_{02})+{\displaystyle \frac{B^2\left({\rho }^{*}\right)}{a_{12}^2}}({\alpha }_{21}+3{\beta }_{03})-2{\displaystyle \frac{a_{11}}{a_{12}}}({\alpha }_{21}+{\beta }_{21})\\ +{\displaystyle \frac{a_{11}^2}{a_{12}^2}}({\alpha }_{12}+3{\beta }_{03})+{\displaystyle \frac{{\beta }_{11}{\beta }_{02}}{a_{12}}}+{\displaystyle \frac{{\alpha }_{11}}{a_{12}}}({\displaystyle \frac{{\alpha }_{11}{a}_{11}}{a_{12}}}-{\alpha }_{20})+{\displaystyle \frac{{\alpha }_{02}{a}_{11}}{a_{12}^2}}(2{\alpha }_{20}-3{\displaystyle \frac{{\alpha }_{11}{a}_{11}}{a_{12}}}+\\ 2{\displaystyle \frac{{\alpha }_{02}{a}_{11}^2}{a_{12}^2}}-2{\displaystyle \frac{{\alpha }_{02}}{a_{12}}})+{\displaystyle \frac{{\beta }_{02}{a}_{11}}{a_{12}^2}}({\alpha }_{11}-2{\beta }_{02}-2{\displaystyle \frac{{\alpha }_{02}{a}_{11}}{a_{12}}}))+{\displaystyle \frac{1}{8B\left({\rho }^{*}\right)}}\left({\displaystyle \frac{{\alpha }_{02}{a}_{11}}{a_{12}}}\right(2{\displaystyle \frac{{\beta }_{02}{a}_{11}}{a_{12}}}\\ -{\displaystyle \frac{{\alpha }_{11}{a}_{11}}{a_{12}}}+2{\displaystyle \frac{{\alpha }_{02}{a}_{11}^2}{a_{12}^2}}-{\beta }_{11}\left)\right)+{\displaystyle \frac{1}{8B^2\left({\rho }^{*}\right)}}({\beta }_{11}({\beta }_{02}{a}_{12}-{\beta }_{11}{a}_{11})+{\displaystyle \frac{{\beta }_{11}{a}_{11}^2}{a_{12}}}(3{\beta }_{02}-{\alpha }_{02})\\ +{\alpha }_{20}{a}_{11}(2{\alpha }_{20}-{\beta }_{11})+(2{\alpha }_{20}{a}_{11}-2{\displaystyle \frac{{\beta }_{02}{a}_{11}^2}{a_{12}}}-{\displaystyle \frac{{\alpha }_{11}{a}_{11}^2}{a_{12}}})({\displaystyle \frac{{\beta }_{02}{a}_{11}}{a_{12}}}+{\displaystyle \frac{{\beta }_{20}{a}_{12}}{a_{11}}}-{\displaystyle \frac{{\alpha }_{02}{a}_{11}}{a_{12}}})\\ \hfill +{\displaystyle \frac{{\alpha }_{11}{a}_{11}^2}{a_{12}}}({\displaystyle \frac{{\alpha }_{11}{a}_{11}}{a_{12}}}-3{\alpha }_{20}+2{\displaystyle \frac{{\beta }_{02}{a}_{11}}{a_{12}}})-{\alpha }_{20}{a}_{11}(2{\beta }_{11}+2{\displaystyle \frac{{\beta }_{02}{a}_{11}}{a_{12}}})).\end{array}$$

(21)

□

4.3. Bogdanov–Takens Bifurcation

From Theorem (3) in Section 3, it is evident that point $E^{*}$ is a cusp of codimension two whenever ${\displaystyle \frac{x^{*}(1+\beta -2x^{*}-\xi (1+\theta y^{*}))}{(1+\theta y^{*})(x^{*}+\gamma )}}+{\displaystyle \frac{\xi x^{*}}{x^{*}+\delta }}\ne \rho$, ${\overline{\alpha }}_{20}\ne 0$, and $(2\overline{{\alpha }_{20}}-{\overline{\alpha }}_{11})\ne 0$. There might be a chance of occurrence of a Bogdanov–Takens bifurcation of codimension two. The intersection of saddle-node and Hopf bifurcation curves is called the BT-point. Now, to explore the emergence of the Bogdanov–Takens bifurcation, $\theta$ and $\rho$ are designated as the bifurcation parameters.

Theorem 6.

A Bogdanov–Takens bifurcation of codimension two emerges in system (5) around the equilibrium point $E^{*}=(x^{*},y^{*})$ in regard to parameters θ and ρ whenever ${\displaystyle \frac{x^{*}(1+\beta -2x^{*}-\xi (1+\theta y^{*}))}{(1+\theta y^{*})(x^{*}+\gamma )}}$

$+{\displaystyle \frac{\xi x^{*}}{x^{*}+\delta }}\ne \rho$, ${\overline{\alpha }}_{20}\ne 0$, and $(2\overline{{\alpha }_{20}}-{\overline{\alpha }}_{11})\ne 0$. Moreover, three bifurcation curves in the ${\lambda }_1{\lambda }_2$-plane exist through the BT-point and are given by the following:

1. 

The saddle-node bifurcation curve:

$SN=({\lambda }_1,{\lambda }_2):{\zeta }_1({\lambda }_1,{\lambda }_2)=0,{\zeta }_2({\lambda }_1,{\lambda }_2)\ne 0$;

2. 

The Hopf bifurcation curve:

$H=({\lambda }_1,{\lambda }_2):{\zeta }_2({\lambda }_1,{\lambda }_2)=\pm \sqrt{-{\zeta }_1({\lambda }_1,{\lambda }_2)},{\zeta }_1({\lambda }_1,{\lambda }_2)<0$;

3. 

The homoclinic bifurcation curve:

$HL=({\lambda }_1,{\lambda }_2):{\zeta }_2({\lambda }_1,{\lambda }_2)=\pm {\displaystyle \frac{5}{7}}\sqrt{-{\zeta }_1({\lambda }_1,{\lambda }_2)},{\zeta }_1({\lambda }_1,{\lambda }_2)<0$.

Proof. 

With the assumption that the bifurcation parameters $\theta$ and $\rho$ undergo small variations within the vicinity of the BT-point $({\theta }_0,{\rho }_0)$, we move ahead with the analysis. Consider a point $({\theta }_0+{\lambda }_1,{\rho }_0+{\lambda }_2)$ in the vicinity of the BT-point $({\theta }_0,{\rho }_0$), where ${\lambda }_1$ and ${\lambda }_2$ are small. Then, system (5) becomes

$$\begin{array}{cc}{\displaystyle \frac{dx}{dt}}\hfill & ={\displaystyle \frac{x(1-x)(x-\beta )}{(1+(\theta +{\lambda }_1)y)(x+\gamma )}}-{\displaystyle \frac{\xi xy}{x+\delta }},\hfill \\ {\displaystyle \frac{dy}{dt}}\hfill & =(\rho +{\lambda }_2)y(1-{\displaystyle \frac{y}{x+\delta }}).\hfill \end{array}$$

(22)

With respect to the variables x and y, system (22) exhibits $C^{\infty }$ smoothness in the vicinity of $({\theta }_0,{\rho }_0)$.

• Taking $u=x-x^{*}$ and $v=y-y^{*}$, system (22) turns out to be

  $$\begin{array}{cc}{\displaystyle \frac{du}{dt}}\hfill & ={\alpha }_{00}+{\alpha }_{10}u+{\alpha }_{01}v+{\alpha }_{20}{u}^2+{\alpha }_{11}uv+{\alpha }_{02}{v}^2+R_1(u,v),\hfill \\ {\displaystyle \frac{dv}{dt}}\hfill & ={\beta }_{10}u+{\beta }_{01}v+{\beta }_{20}{u}^2+{\beta }_{11}uv+{\beta }_{02}{v}^2+R_2(u,v),\hfill \end{array}$$

  (23)

  where ${\alpha }_{00}={\displaystyle \frac{x^{*}(1-x^{*})(x^{*}-\beta )}{(1+(\theta +{\lambda }_1)y^{*})(x^{*}+\gamma )}}-\xi x^{*}$, ${\alpha }_{01}=-{\displaystyle \frac{(\theta +{\lambda }_1)x^{*}(1-x^{*})(x^{*}-\beta )}{{(1+(\theta +{\lambda }_1)y^{*})}^2(x^{*}+\gamma )}}-{\displaystyle \frac{\xi x^{*}}{x^{*}+\delta }}$,

${\alpha }_{10}={\displaystyle \frac{x^{*}(1-2x^{*}+\beta )}{(1+(\theta +{\lambda }_1)y^{*})(x^{*}+\gamma )}}+{\displaystyle \frac{\gamma (1-x^{*})(x^{*}-\beta )}{(1+(\theta +{\lambda }_1)y^{*}){(x^{*}+\gamma )}^2}}-{\displaystyle \frac{\xi \delta }{{(x^{*}+\delta )}^2}}$,

${\alpha }_{20}=-{\displaystyle \frac{2x^{*}}{(1+(\theta +{\lambda }_1)y^{*})(x^{*}+\gamma )}}+{\displaystyle \frac{2\gamma (1-2x^{*}+\beta )}{(1+(\theta +{\lambda }_1)y^{*}){(x^{*}+\gamma )}^2}}-{\displaystyle \frac{2\gamma (1-x^{*})(x^{*}-\beta )}{(1+(\theta +{\lambda }_1)y^{*}){(x^{*}+\gamma )}^3}}+{\displaystyle \frac{2\xi \delta }{{(x^{*}+\delta )}^2}}$,

${\alpha }_{11}={\displaystyle \frac{-(\theta +{\lambda }_1)x^{*}(1-2x^{*}+\beta )}{{(1+(\theta +{\lambda }_1)y^{*})}^2(x^{*}+\gamma )}}-{\displaystyle \frac{(\theta +{\lambda }_1)\gamma (1-x^{*})(x^{*}-\beta )}{{(1+(\theta +{\lambda }_1)y^{*})}^2{(x^{*}+\gamma )}^2}}-{\displaystyle \frac{\xi \delta }{{(x^{*}+\delta )}^2}}$, ${\alpha }_{02}={\displaystyle \frac{2{(\theta +{\lambda }_1)}^2{x}^{*}(1-x)}{{(1+(\theta +{\lambda }_1)y^{*})}^3(x^{*}+\gamma )}}$,

${\beta }_{10}=\rho +{\lambda }_2$, ${\beta }_{01}=-(\rho +{\lambda }_2)$, ${\beta }_{20}={\displaystyle \frac{-(\rho +{\lambda }_2)}{x+\delta }}$, ${\beta }_{02}={\displaystyle \frac{-(\rho +{\lambda }_2)}{x+\delta }}$, and ${\beta }_{11}=2{\displaystyle \frac{\rho +{\lambda }_2}{x+\delta }}$; furthermore, $R_1$ and $R_2$ are power series in $(u,v)$ with powers $u^i$ and $v^j$ satisfying $i+j\ge 3$, and the coefficients smoothly depend upon ${\lambda }_1$ and ${\lambda }_2$.

Now, we use the affine transformations $y_1=u$ and $y_2={\alpha }_{10}u+{\alpha }_{01}v$ such that system (23) becomes

$$\begin{array}{cc}{\displaystyle \frac{d{y}_1}{dt}}\hfill & =y_{2+}+{\xi }_{00}+{\xi }_{20}{y}_1^2+{\xi }_{11}{y}_1{y}_2+{\xi }_{02}{y}_2^2+{\tilde{R}}_1(y_1,y_2),\hfill \\ {\displaystyle \frac{d{y}_2}{dt}}\hfill & ={\eta }_{00}+{\eta }_{10}{y}_1+{\eta }_{01}{y}_2+{\eta }_{20}{y}_1^2+{\eta }_{11}{y}_1{y}_2+{\eta }_{02}{y}_2^2+{\tilde{R}}_2(y_1,y_2),\hfill \end{array}$$

(24)

where ${\xi }_{00}={\alpha }_{00}$, ${\xi }_{20}={\alpha }_{20}-{\displaystyle \frac{{\alpha }_{11}{\alpha }_{10}}{{\alpha }_{01}}}+{\displaystyle \frac{{\alpha }_{02}{\alpha }_{10}^2}{{\alpha }_{01}^2}}$, ${\xi }_{02}={\displaystyle \frac{{\alpha }_{02}}{{\alpha }_{01}^2}}$, ${\xi }_{11}={\displaystyle \frac{{\alpha }_{11}}{{\alpha }_{01}}}-{\displaystyle \frac{2{\alpha }_{02}{\alpha }_{10}}{{\alpha }_{01}^2}}$,

${\eta }_{00}={\alpha }_{00}{\alpha }_{10}$, ${\eta }_{10}={\alpha }_{01}{\beta }_{10}-{\alpha }_{10}{\beta }_{01}$, ${\eta }_{01}={\alpha }_{10}+{\beta }_{01}$, ${\eta }_{20}={\alpha }_{20}{\alpha }_{10}+{\alpha }_{01}{\beta }_{20}-{\alpha }_{10}{\beta }_{11}-{\displaystyle \frac{{\alpha }_{11}{\alpha }_{10}^2}{{\alpha }_{01}}}+{\displaystyle \frac{{\alpha }_{10}^2{\beta }_{02}}{{\alpha }_{01}}}+{\displaystyle \frac{{\alpha }_{10}^3{\alpha }_{02}}{{\alpha }_{01}^2}}$, ${\eta }_{02}={\displaystyle \frac{{\alpha }_{10}{\alpha }_{02}}{{\alpha }_{01}^2}}+{\displaystyle \frac{{\beta }_{02}}{{\alpha }_{01}}}$, and ${\eta }_{11}={\displaystyle \frac{{\alpha }_{11}{\alpha }_{10}}{{\alpha }_{01}}}+{\beta }_{11}-{\displaystyle \frac{2{\beta }_{02}{\alpha }_{10}}{{\alpha }_{01}}}-{\displaystyle \frac{2{\alpha }_{02}{\alpha }_{10}^2}{{\alpha }_{01}^2}}$; furthermore, ${\tilde{R}}_1$ and ${\tilde{R}}_2$ are power series in $(y_1,y_2)$ with powers $y_1^i$ and $y_2^j$ satisfying $i+j\ge 3$, and the coefficients smoothly depend upon ${\lambda }_1$ and ${\lambda }_2$.

Next, we perform the $C^{\infty }$ transformations $x_1=y_1$ and $x_2=y_{2+}+{\xi }_{00}+{\xi }_{20}{y}_1^2+{\xi }_{11}{y}_1{y}_2+{\xi }_{02}{y}_2^2$ such that system (24) becomes

$$\begin{array}{cc}{\displaystyle \frac{d{x}_1}{dt}}\hfill & =x_2+{\widehat{R}}_1(x_1,x_2),\hfill \\ {\displaystyle \frac{d{x}_2}{dt}}\hfill & ={\gamma }_{00}+{\gamma }_{10}{y}_1+{\gamma }_{01}{y}_2+{\gamma }_{20}{y}_1^2+{\gamma }_{11}{y}_1{y}_2+{\gamma }_{02}{y}_2^2+{\widehat{R}}_2(x_1,x_2),\hfill \end{array}$$

(25)

where ${\gamma }_{00}={\eta }_{00}-{\xi }_{00}{\eta }_{01}+\cdots$, ${\gamma }_{10}={\eta }_{10}-{\xi }_{00}{\eta }_{11}+{\xi }_{11}{\eta }_{00}+\cdots$, ${\gamma }_{01}={\eta }_{01}-2{\xi }_{00}{\eta }_{02}+{\xi }_{02}{\eta }_{00}-{\xi }_{00}{\xi }_{11}+\cdots$, ${\gamma }_{20}={\eta }_{20}+{\xi }_{11}{\eta }_{10}-{\xi }_{20}{\eta }_{01}+\cdots$, ${\gamma }_{01}={\eta }_{02}+{\xi }_{11}-{\xi }_{02}{\eta }_{01}+\cdots$, and ${\gamma }_{11}={\eta }_{11}+2{\xi }_{20}+2{\xi }_{02}{\eta }_{10}+\cdots$; furthermore, ${\widehat{R}}_1$ and ${\widehat{R}}_2$ are power series in $(x_1,x_2)$ with powers $x_1^i$ and $x_2^j$ satisfying $i+j\ge 3$, and the coefficients smoothly depend upon ${\lambda }_1$ and ${\lambda }_2$.

Now, consider a new time variable $\tau$ where $dt=(1-{\gamma }_{02}{x}_1)d\tau$. Rewriting $\tau$ as t, system (25) can be rewritten as

$$\begin{array}{cc}{\displaystyle \frac{d{x}_1}{dt}}\hfill & =(1-{\gamma }_{02}{x}_1)x_2+{\widehat{R}}_1(x_1,x_2),\hfill \\ {\displaystyle \frac{d{x}_2}{dt}}\hfill & =(1-{\gamma }_{02}{x}_1)({\gamma }_{00}+{\gamma }_{10}{y}_1+{\gamma }_{01}{y}_2+{\gamma }_{20}{y}_1^2+{\gamma }_{11}{y}_1{y}_2+{\gamma }_{02}{y}_2^2+{\widehat{R}}_2(x_1,x_2)).\hfill \end{array}$$

(26)

Next, using the transformations $z_1=x_1$ and $z_2=(1-{\gamma }_{02}{x}_1)x_2+{\widehat{R}}_1(x_1,x_2)$, system (26) transforms to

$$\begin{array}{cc}{\displaystyle \frac{d{z}_1}{dt}}\hfill & =z_2,\hfill \\ {\displaystyle \frac{d{z}_2}{dt}}\hfill & ={\delta }_{00}+{\delta }_{10}{z}_1+{\delta }_{01}{z}_2+{\delta }_{20}{z}_1^2+{\delta }_{11}{z}_1{z}_2+{\overline{R}}_2(z_1,z_2),\hfill \end{array}$$

(27)

where ${\delta }_{00}={\gamma }_{00}$, ${\delta }_{10}={\gamma }_{10}-2{\gamma }_{02}{\gamma }_{00}$, ${\delta }_{01}={\gamma }_{01}$, ${\delta }_{20}={\gamma }_{20}+{\gamma }_{00}{\gamma }_{02}^2-2{\gamma }_{02}{\gamma }_{10}$, and ${\delta }_{11}={\gamma }_{11}-{\gamma }_{02}{\gamma }_{01}$; furthermore, ${\overline{R}}_1$ and ${\overline{R}}_2$ are power series in $(z_1,z_2)$ with powers $z_1^i$ and $z_2^j$ satisfying $i+j\ge 3$, and the coefficients have a smooth dependence upon ${\lambda }_1$ and ${\lambda }_2$.

When ${\lambda }_1$ and ${\lambda }_2$ are small, determination of the sign of ${\delta }_{20}$ is not possible. Hence, it is crucial to examine the following two cases:

Case 1: ${\delta }_{20}>0$

• Consider $U=z_1$, $V={\displaystyle \frac{z_2}{\sqrt{{\delta }_{20}}}}$, and $d\tau =\sqrt{{\delta }_{20}}dt$; then, system (27) becomes

  $$\begin{array}{cc}{\displaystyle \frac{dU}{d\tau }}\hfill & =V,\hfill \\ {\displaystyle \frac{dV}{d\tau }}\hfill & =p_{00}+p_{10}U+p_{01}V+U^2+p_{11}UV+R_3(U,V),\hfill \end{array}$$

  (28)

  where $p_{00}={\displaystyle \frac{{\delta }_{00}}{{\delta }_{20}}}$, $p_{10}={\displaystyle \frac{{\delta }_{10}}{{\delta }_{20}}}$, $p_{01}={\displaystyle \frac{{\delta }_{01}}{\sqrt{{\delta }_{20}}}}$, and $p_{11}={\displaystyle \frac{{\delta }_{00}}{\sqrt{{\delta }_{20}}}}$; furthermore, $R_3$ is a power series in $(U,V)$ with powers $U^i$ and $V^j$ satisfying $i+j\ge 3$, and the coefficients smoothly depend upon ${\lambda }_1$ and ${\lambda }_2$.

Next, we use the affine transformations $v_1=U+{\displaystyle \frac{p_{10}}{2}}$ and $v_2=V$; then, system (28) becomes

$$\begin{array}{cc}{\displaystyle \frac{d{v}_1}{d\tau }}\hfill & =v_2,\hfill \\ {\displaystyle \frac{d{v}_2}{d\tau }}\hfill & =q_{00}+q_{01}{v}_2+v_1^2+q_{11}{v}_1{v}_2+R_4(v_1,v_2),\hfill \end{array}$$

(29)

where $q_{00}=p_{00}-{\displaystyle \frac{p_{10}^2}{4}}$, $q_{01}=p_{01}-{\displaystyle \frac{p_{11}{p}_{10}}{2}}$, and $q_{11}=p_{11}$; furthermore, $R_4$ is a power series in $(v_1,v_2)$ with powers $v_1^i$ and $v_2^j$ satisfying $i+j\ge 3$, and the coefficients smoothly depend upon ${\lambda }_1$ and ${\lambda }_2$.

Making the change of variables $w_1=q_{11}^2{v}_1$, $w_2=q_{11}^3{v}_2$, and $t={\displaystyle \frac{\tau }{q_{11}}}$, system (29) becomes

$$\begin{array}{cc}{\displaystyle \frac{d{w}_1}{dt}}\hfill & =w_2,\hfill \\ {\displaystyle \frac{d{w}_2}{dt}}\hfill & ={\zeta }_1({\lambda }_1,{\lambda }_2)+{\zeta }_2\left(({\lambda }_1,{\lambda }_2)\right)w_2+w_1^2+w_1{w}_2+R_5(w_1,w_2),\hfill \end{array}$$

(30)

where

$${\zeta }_1({\lambda }_1,{\lambda }_2)=q_{00}{q}_{11}^4,{\zeta }_2({\lambda }_1,{\lambda }_2)=q_{01}{q}_{11};$$

(31)

furthermore, $R_5$ is a power series in $(w_1,w_2)$ with powers $w_1^i$ and $w_2^j$ satisfying $i+j\ge 3$, and the coefficients smoothly depend upon ${\lambda }_1$ and ${\lambda }_2$.

Case 2: ${\delta }_{20}<0$

Consider $\overline{U}=z_1$, $\overline{V}={\displaystyle \frac{z_2}{\sqrt{-{\delta }_{20}}}}$, and $d\tau =\sqrt{-{\delta }_{20}}dt$; then, system (27) becomes

$$\begin{array}{cc}{\displaystyle \frac{d\overline{U}}{d\tau }}\hfill & =\overline{V},\hfill \\ {\displaystyle \frac{d\overline{V}}{d\tau }}\hfill & ={\overline{p}}_{00}+{\overline{p}}_{10}\overline{U}+{\overline{p}}_{01}\overline{V}+{\overline{U}}^2+{\overline{p}}_{11}\overline{U}\overline{V}+{\overline{R}}_3(\overline{U},\overline{V}),\hfill \end{array}$$

(32)

where ${\overline{p}}_{00}={\displaystyle \frac{-{\delta }_{00}}{{\delta }_{20}}}$, ${\overline{p}}_{10}={\displaystyle \frac{-{\delta }_{10}}{{\delta }_{20}}}$, ${\overline{p}}_{01}={\displaystyle \frac{{\delta }_{01}}{\sqrt{-{\delta }_{20}}}}$, and ${\overline{p}}_{11}={\displaystyle \frac{{\delta }_{00}}{\sqrt{-{\delta }_{20}}}}$; furthermore, ${\overline{R}}_3$ is a power series in $(\overline{U},\overline{V})$ with powers ${\overline{U}}^i$ and ${\overline{V}}^j$ satisfying $i+j\ge 3$, and the coefficients smoothly depend upon ${\lambda }_1$ and ${\lambda }_2$.

Next, we use the affine transformations ${\overline{v}}_1=\overline{U}-{\displaystyle \frac{{\overline{p}}_{10}}{2}}$ and ${\overline{v}}_2=\overline{V}$; then, system (32) becomes

$$\begin{array}{cc}{\displaystyle \frac{d{\overline{v}}_1}{d\tau }}\hfill & ={\overline{v}}_2,\hfill \\ {\displaystyle \frac{d{\overline{v}}_2}{d\tau }}\hfill & ={\overline{q}}_{00}+{\overline{q}}_{01}{\overline{v}}_2+{\overline{v}}_1^2+{\overline{q}}_{11}{\overline{v}}_1{\overline{v}}_2+{\overline{R}}_4({\overline{v}}_1,{\overline{v}}_2),\hfill \end{array}$$

(33)

where ${\overline{q}}_{00}={\overline{p}}_{00}+{\displaystyle \frac{{\overline{p}}_{10}^2}{4}}$, ${\overline{q}}_{01}={\overline{p}}_{01}+{\displaystyle \frac{{\overline{p}}_{11}{\overline{p}}_{10}}{2}}$, and ${\overline{q}}_{11}={\overline{p}}_{11}$; furthermore, ${\overline{R}}_4$ is a power series in $({\overline{v}}_1,{\overline{v}}_2)$ with powers ${\overline{v}}_1^i$ and ${\overline{v}}_2^j$ satisfying $i+j\ge 3$, and the coefficients smoothly depend upon ${\lambda }_1$ and ${\lambda }_2$.

Consider new variables ${\overline{w}}_1=-{\overline{q}}_{11}^2{\overline{v}}_1$, ${\overline{w}}_2={\overline{q}}_{11}^3{\overline{v}}_2$, and $t=-{\displaystyle \frac{\tau }{{\overline{q}}_{11}}}$; then, system (33) becomes

$$\begin{array}{cc}{\displaystyle \frac{d{\overline{w}}_1}{dt}}\hfill & ={\overline{w}}_2,\hfill \\ {\displaystyle \frac{d{\overline{w}}_2}{dt}}\hfill & ={\overline{\zeta }}_1({\lambda }_1,{\lambda }_2)+{\overline{\zeta }}_2({\lambda }_1,{\lambda }_2){\overline{w}}_2+{\overline{w}}_1^2+{\overline{w}}_1{\overline{w}}_2+{\overline{R}}_5({\overline{w}}_1,{\overline{w}}_2),\hfill \end{array}$$

(34)

where

$${\overline{\zeta }}_1({\lambda }_1,{\lambda }_2)=-{\overline{q}}_{00}{\overline{q}}_{11}^4,{\overline{\zeta }}_2({\lambda }_1,{\lambda }_2)=-{\overline{q}}_{01}{\overline{q}}_{11};$$

(35)

furthermore, ${\overline{R}}_5$ is a power series in $({\overline{w}}_1,{\overline{w}}_2)$ with powers ${\overline{w}}_1^i$ and ${\overline{w}}_2^j$ satisfying $i+j\ge 3$, and the coefficients smoothly depend upon ${\lambda }_1$ and ${\lambda }_2$.

The number of cases can be minimized by using ${\zeta }_1({\lambda }_1,{\lambda }_2)$ and ${\zeta }_2({\lambda }_1,{\lambda }_2)$ to denote ${\overline{\zeta }}_1({\lambda }_1,{\lambda }_2)$ and ${\overline{\zeta }}_2({\lambda }_1,{\lambda }_2)$ in (35). If $|{\displaystyle \frac{\partial ({\zeta }_1,{\zeta }_2)}{\partial ({\lambda }_1,{\lambda }_2)}}{|}_{{\lambda }_1={\lambda }_2=0}\ne 0$, that is, it is non-singular, then the parameter transformations in (31) and (35) are topologically equivalent in the small neighborhood of origin. So, it can be deduced that system (22) experiences a Bogdanov–Takens bifurcation when $({\lambda }_1,{\lambda }_2)$ is in the vicinity of the origin by the result in Perko [45]. The depictions of the bifurcation curves are as follows:

• The saddle-node bifurcation curve $SN=({\lambda }_1,{\lambda }_2):{\zeta }_1({\lambda }_1,{\lambda }_2)=0,{\zeta }_2({\lambda }_1,{\lambda }_2)\ne 0$;
• The Hopf bifurcation curve $H=({\lambda }_1,{\lambda }_2):{\zeta }_2({\lambda }_1,{\lambda }_2)=\pm \sqrt{-{\zeta }_1({\lambda }_1,{\lambda }_2)},{\zeta }_1({\lambda }_1,{\lambda }_2)<0$;
• The homoclinic bifurcation curve $HL=({\lambda }_1,{\lambda }_2):{\zeta }_2({\lambda }_1,{\lambda }_2)=\pm {\displaystyle \frac{5}{7}}\sqrt{-{\zeta }_1({\lambda }_1,{\lambda }_2)},$${\zeta }_1({\lambda }_1,{\lambda }_2)<0$.

□

5. Numerical Simulation

To support and visualize all the above analysis, numerical simulation was undertaken using MATLAB R2013a and Mathematica version 12.0, and the phase portraits were drawn using MATLAB R2013a. The non-linear equations were solved using the “ode45” function present in MATLAB R2013a and the “NSolve” function present in Mathematica version 12.

• We fixed the parameters as $\beta =0.1$, $\gamma =0.3$, $\delta =0.1$, $\rho =2$, and $\xi =0.2$ in system (5). The acquired results are presented in Table 1.

  Table 1. Number and characteristics of equilibria that resulted from the emergence of a saddle-node bifurcation.

  $\mathit{\theta }$	Equilibrium Points (Stability)		Figure
  	Axial	Interior
  $0<\theta <{\theta }^{SN}=0.576118$	$E_0=(0,0)$ (saddle)	$E_2^{*}=(0.620942,0.720942)$ (stable), $E_3^{*}=(0.253568,0.353568)$ (saddle)	Figure 1a
  $\theta ={\theta }^{SN}$	$E_{\delta }=(0,\delta )$ (stable)	$E^{*}=(0.382843,0.482843)$ (degenerate singularity), emergence of saddle-node bifurcation	Figure 1b,d
  $\theta >{\theta }^{SN}$	$E_{\beta }=(\beta ,0)$ (unstable), $E_1=(1,0)$ (saddle)	None	Figure 1c

  Table 1. Number and characteristics of equilibria that resulted from the emergence of a saddle-node bifurcation.

  $\mathit{\theta }$	Equilibrium Points (Stability)		Figure
  	Axial	Interior
  $0<\theta <{\theta }^{SN}=0.576118$	$E_0=(0,0)$ (saddle)	$E_2^{*}=(0.620942,0.720942)$ (stable), $E_3^{*}=(0.253568,0.353568)$ (saddle)	Figure 1a
  $\theta ={\theta }^{SN}$	$E_{\delta }=(0,\delta )$ (stable)	$E^{*}=(0.382843,0.482843)$ (degenerate singularity), emergence of saddle-node bifurcation	Figure 1b,d
  $\theta >{\theta }^{SN}$	$E_{\beta }=(\beta ,0)$ (unstable), $E_1=(1,0)$ (saddle)	None	Figure 1c

  Figure 1. Existence of equilibrium points. (a) At $\theta =0.1$. (b) At $\theta ={\theta }^{SN}=0.576118$. (c) At $\theta =0.6$. The blue and red (separatrix) curves represent the solution trajectories of the system 5 and the arrows depict the direction of these trajectories in (a–c). (d) Saddle-node bifurcation diagram (upper and lower curves represent stable equilibria and unstable equilibria, respectively).

  Figure 1. Existence of equilibrium points. (a) At $\theta =0.1$. (b) At $\theta ={\theta }^{SN}=0.576118$. (c) At $\theta =0.6$. The blue and red (separatrix) curves represent the solution trajectories of the system 5 and the arrows depict the direction of these trajectories in (a–c). (d) Saddle-node bifurcation diagram (upper and lower curves represent stable equilibria and unstable equilibria, respectively).
• The occurrence of a limit cycle around the stable point, which led to a Hopf bifurcation, is shown in Figure 2. We fixed the parameters as $\theta =0.4$, $\beta =0.1$, $\gamma =0.3$, $\delta =0.1$, and $\xi =0.2$; the obtained results are in

  Table 2. Number and characteristics of equilibrium points that resulted from the emergence of a Hopf bifurcation.

  $\mathit{\rho }$	Equilibrium Points (Stability)		Figure
  	Axial	Interior
  $\rho =0.15$	$E_0=(0,0)$ (saddle)	$E_2^{*}$ = $(0.507277,0.607277)$ (stable), $E_3^{*}$ = $(0.296426,0.396426)$ (saddle)	Figure 2a
  $\rho =0.1031$	$E_{\delta }=(0,\delta )$ (stable)	Homoclinic loop enclosing $E_2^{*}$ = $(0.507277,0.607277)$ (stable), $E_3^{*}$ = $(0.296426,0.396426)$ (saddle)	Figure 2b
  $\rho ={\rho }^{hf}=0.0845887$	$E_{\beta }=(\beta ,0)$ (unstable)	Unstable limit cycle (as $\Lambda =0.66667>0$) around $E_2^{*}$ = $(0.507277,0.607277)$ (stable), $E_3^{*}$ = $(0.296426,0.396426)$ (saddle)	Figure 2c
  $\rho =0.01$	$E_1=(1,0)$ (saddle)	$E_2^{*}$ = $(0.507277,0.607277)$ (unstable), $E_3^{*}$ = $(0.296426,0.396426)$ (saddle)	Figure 2d

  .
• Fixing the parameters as $\beta =0.1$, $\gamma =0.3$, $\delta =0.1$, and $\xi =0.2$ in (22), we first found the threshold values, which come out to be ${\theta }_0=0.576118$ and ${\rho }_0=0.193091$. Corresponding to these parameters, system (5) exhibited a unique equilibrium point $E^{*}=(0.382843,0.482843)$. By incorporating all the specified parameters, we derived the subsequent system:

  $$\begin{array}{cc}{\displaystyle \frac{dx}{dt}}\hfill & ={\displaystyle \frac{x(1-x)(x-0.1)}{(1+(0.576118+{\lambda }_1)y)(x+0.3)}}-{\displaystyle \frac{0.2xy}{x+0.1}},\hfill \\ {\displaystyle \frac{dy}{dt}}\hfill & =(0.193091+{\lambda }_2)y(1-{\displaystyle \frac{y}{x+0.1}}).\hfill \end{array}$$

  (36)
  To transfer this equilibrium point to the origin, we made the transformations $u=x-0.382843$ and $v=y-0.482843$. Then, we made the affine transformations $y_1=u$ and $y_2={\alpha }_{10}u+{\alpha }_{01}v$. So, the system reduced to

  $$\begin{array}{cc}{\displaystyle \frac{d{y}_1}{dt}}\hfill & =y_{2+}+{\xi }_{00}+{\xi }_{20}{y}_1^2+{\xi }_{11}{y}_1{y}_2+{\xi }_{02}{y}_2^2+{\tilde{R}}_1(y_1,y_2)\hfill \\ {\displaystyle \frac{d{y}_2}{dt}}\hfill & ={\eta }_{00}+{\eta }_{10}{y}_1+{\eta }_{01}{y}_2+{\eta }_{20}{y}_1^2+{\eta }_{11}{y}_1{y}_2+{\eta }_{02}{y}_2^2+{\tilde{R}}_2(y_1,y_2)\hfill \end{array}$$

  (37)

  where ${\xi }_{00}=-0.0765685+{\displaystyle \frac{0.097868}{1.27817+0.482843{\lambda }_1}}$, ${\xi }_{20}={(-0.100088\cdots -0.0369706{\lambda }_1^2)}^2$, ${\xi }_{02}={(0.195736\cdots -0.0369706{\lambda }_1^2)}^2$, ${\xi }_{11}={(0.0666203\cdots -0.0369706{\lambda }_1^2)}^2$, ${\eta }_{00}={(3.42781\ast {10}^{-15}-\cdots +0.482843{\lambda }_1)}^2$, ${\eta }_{10}={(-2.60784\ast {10}^{-13}\cdots +0.482843{\lambda }_1)}^2$, ${\eta }_{01}=(3.9968\ast {10}^{-15}-\cdots +0.482843{\lambda }_1$, ${\eta }_{20}={(-0.024702-\cdots -0.0369706{\lambda }_1^2)}^2$, ${\eta }_{02}={(0.222134+\cdots -0.0369706{\lambda }_1^2)}^2$, and ${\eta }_{11}={(0.0164421+\cdots -0.0369706{\lambda }_1^2)}^2$; furthermore, ${\tilde{R}}_1$ and ${\tilde{R}}_2$ were power series in $(y_1,y_2)$ with powers $y_1^i$ and $y_2^j$ satisfying $i+j\ge 3$, and the coefficients had a smooth dependence upon ${\lambda }_1$ and ${\lambda }_2$.
  Next, we performed the $C^{\infty }$ transformations $x_1=y_1$ and $x_2=y_2+{\xi }_{00}+{\xi }_{20}{y}_1^2+{\xi }_{11}{y}_1{y}_2+{\xi }_{02}{y}_2^2.$ We made a change in the time variable $dt=(1-{\gamma }_{02}{x}_1)$ and rewrote T as t. Then, using the transformations $z_1=x_1$ and $z_2=(1-{\gamma }_{02}{x}_1)x_2+{\widehat{R}}_1(x_1,x_2)$, system (37) is reduced to

  $$\begin{array}{cc}{\displaystyle \frac{d{z}_1}{dt}}\hfill & =z_2\hfill \\ {\displaystyle \frac{d{z}_2}{dt}}\hfill & ={\delta }_{00}+{\delta }_{10}{z}_1+{\delta }_{01}{z}_2+{\delta }_{20}{z}_1^2+{\delta }_{11}{z}_1{z}_2+{\overline{R}}_2(z_1,z_2)\hfill \end{array}$$

  (38)

  where ${\delta }_{00}={(8.45997\ast {10}^{-16}-\cdots +0.482843{\lambda }_1)}^2$, ${\delta }_{10}={(-4.31962\ast {10}^{-14}+\cdots -0.0369706{\lambda }_1^2)}^2)$, ${\delta }_{01}={(-1.24357\ast {10}^{-15}+\cdots -0.0369706{\lambda }_1^2)}^2)$, ${\delta }_{20}={(-0.00656111-\cdots -0.0369706{\lambda }_1^2)}^4)$, and ${\delta }_{11}={(-0.0298712-\cdots -0.0369706{\lambda }_1^2)}^4)$; furthermore, ${\overline{R}}_2$ was a power series in $(z_1,z_2)$ with powers $z_1^i$ and $z_2^j$ satisfying $i+j\ge 3$, and the coefficients smoothly depended upon ${\lambda }_1$ and ${\lambda }_2$.
  When ${\lambda }_1$ and ${\lambda }_2$ were very small, then ${\delta }_{20}=-0.151939$. Now, we first considered $U=z_1$, $V={\displaystyle \frac{z_2}{\sqrt{-{\delta }_{20}}}}$, and $d\tau =\sqrt{-{\delta }_{20}}dt$; then, we used $v_1=U-{\displaystyle \frac{p_{10}}{2}}$ and $v_2=V$. Then, making a change in the variables $w_1=-q_{11}^2{v}_1$, $w_2=q_{11}^3{v}_2$, and $t=-{\displaystyle \frac{\tau }{q_{11}}}$, the system reduced to

  $$\begin{array}{cc}{\displaystyle \frac{d{w}_1}{dt}}\hfill & =w_2\hfill \\ {\displaystyle \frac{d{w}_2}{dt}}\hfill & ={\zeta }_1({\lambda }_1,{\lambda }_2)+{\zeta }_2({\lambda }_1,{\lambda }_2)w_2+w_1^2+w_1{w}_2+{\overline{R}}_5(w_1,w_2)\hfill \end{array}$$

  (39)

  where

  $${\zeta }_1({\lambda }_1,{\lambda }_2)=-q_{00}{q}_{11}^4,{\zeta }_2({\lambda }_1,{\lambda }_2)=-q_{01}{q}_{11},$$

  (40)
  $q_{00}={(1.46715\ast {10}^{-19}-\cdots -3.16471\ast {10}^{-7}{\lambda }_1^{14}{\lambda }_2^3)}^2$, $q_{01}={(-6.37002\ast {10}^{-16}-\cdots -0.0369706{\lambda }_1^2)}^4)$, and $q_{11}={(-0.0298712\cdots -0.0369706{\lambda }_1^2)}^4)$; furthermore, ${\overline{R}}_5$ was a power series in $(w_1,w_2)$ with powers $w_1^i$ and $w_2^j$ that satisfied $i+j\ge 3$, and the coefficients smoothly depended upon ${\lambda }_1$ and ${\lambda }_2$.

The determinant $E=|{\displaystyle \frac{\partial ({\zeta }_1,{\zeta }_2)}{\partial ({\lambda }_1,{\lambda }_2)}}{|}_{{\lambda }_1={\lambda }_2=0}=-65.9058\ne 0$; from this, it is evident that the rank of the matrix E was 2. So, the parametric transformation (31) was non-singular.

In the neighborhood of the BT-point $(0,0)$, the three bifurcation curves divided the ${\lambda }_1{\lambda }_2$-plane into four distinct regions, as shown in Figure 3a. The blue line represents the saddle-node bifurcation curve, the red line represents the Hopf bifurcation curve, and the green line represents the homoclinic bifurcation curve.

Behavior of the system in different regions:

When ${\lambda }_1={\lambda }_2=0$, then system (36) possessed a unique positive equilibrium point that was identified as a codimension-two cusp, as shown in Figure 3b. When we varied the values of ${\lambda }_1$ and ${\lambda }_2$ in such a way that we entered region 1, system (36) exhibited the absence of an interior equilibrium point, which implies that the system moved toward a prey-free equilibrium point, as shown in Figure 3c. As ${\lambda }_1$ and ${\lambda }_2$ entered region 2 by crossing the saddle-node bifurcation curve, the system went from exhibiting no interior equilibrium point to exhibiting two interior equilibrium points. One of the equilibrium points was consistently a saddle, while the other was characterized as unstable, as shown in Figure 3d. As the values of ${\lambda }_1$ and ${\lambda }_2$ entered region 3 by crossing the Hopf bifurcation curve, the positive equilibrium point was enclosed within an unstable limit cycle, as shown in Figure 3e. As the values of ${\lambda }_1$ and ${\lambda }_2$ entered region 4 by crossing the homoclinic bifurcation curve, the system lost the limit cycle and attained an interior equilibrium point that was stable, along with a saddle interior equilibrium point (cf. Figure 3f). The behavior of the system 5 in each region has been comprised in

Table 3. Behavior of system in various regions.

Region	Number of Interior Equilibrium Points	Dynamic Behavior Exhibited by Regions	Figure
Origin	One	Cusp	Figure 3b
Region 1	Zero		Figure 3c
Region 2	Two	One was consistently a saddle while the other was unstable	Figure 3d
Region 3	Two	One was consistently a saddle while the other was enclosed by an unstable limit cycle	Figure 3e
Region 4	Two	One was consistently a saddle while the other was stable	Figure 3f

.

6. Impact of Fear Effect

The emphasis of this section is on exploring the influence that fear and the double Allee effect have on prey species. The impact was studied in two parts: the first part dealt with a scenario where the predator’s population was constant, and the second part dealt with a scenario when the predator’s population was governed by the second differential equation of model (4).

• Consider the prey equation of model (4) while the predator was taken as a constant. In this scenario, model (4) is reduced to

  $${\displaystyle \frac{dN}{dT}}={\displaystyle \frac{rN}{1+fP}}\left(1-{\displaystyle \frac{N}{K}}\right)\left({\displaystyle \frac{N-m}{N+n}}\right)-{\displaystyle \frac{\alpha NP}{a+N}}.$$

  (41)
  In reality, this scenario where the predators remain constant in an environment is highly unlikely, but there do exist some cases in which keeping a species’ population constant is possible, and this allows us to control the dynamics of an ecological system to some extent. To fulfill the aim of this study, the predator species was considered constant so that the impact of the fear effect could be explored at great length. Since fear is a feeling that is induced in prey due to the presence of predators only, we cannot disregard the existence of predators even while doing a study on just the growth function of the prey.
  In addition to the fear effect, we also considered the influence of the double Allee effect on the growth rate of prey species. This purpose could be served by plotting the per capita growth rate of the prey, i.e., ${\displaystyle \frac{1}{N}}{\displaystyle \frac{dN}{dT}}$ against the population of the prey N (see Figure 4). From Figure 4a, we can see that the incorporation of phenomena like the double Allee effect and the fear effect led to a decline in the growth rate of the prey species. Figure 4b–d reflects that as the values of the parameters m, n, and f increased, respectively, the per capita growth rate of prey species decreased. The prey population possessed a threshold value for these parameters, which was crucial in determining the extinction or survival of the species.
  Consider $N=Kx$, $P=Ky$, and $T={\displaystyle \frac{t}{r}}$; then, system (41) reduces to the following non-dimensional form:

  $${\displaystyle \frac{dx}{dt}}={\displaystyle \frac{x(1-x)(x-\beta )}{(1+\theta y)(x+\gamma )}}-{\displaystyle \frac{\xi xy}{x+\delta }}=f\left(x\right),$$

  (42)

  where $\beta ={\displaystyle \frac{m}{K}}$, $\gamma ={\displaystyle \frac{n}{K}}$, $\delta ={\displaystyle \frac{a}{K}}$, $\theta =fK$, and $\xi ={\displaystyle \frac{\alpha }{r}}$.
  Except $x=0$, system (42) has non-trivial equilibrium points given by

  $$x^3-J{x}^2-Kx+L=0,$$

  (43)

  where $J=1+\beta -\delta$, $K=\beta \delta +\delta -\beta -\xi y-\xi \theta y^2$, and $L=\beta \delta +\gamma (\xi y+\xi \theta y^2)$. If any one of the conditions $J>0$, $K\ge 0$; $J>0$, $K\le 0$; or $J\le 0$, $K\ge 0$ holds, then by Descartes’ sign rule, it is clear that (43) has a negative root. Let this root be $-\nu$, $\nu >0$. Dividing (43) by $(x+\nu )$, we obtain the following quadratic equation:

  $$x^2-(J+\nu )x+\nu (J+\nu )-K=0,$$

  (44)

  Taking ${\Delta }_1={(J+\nu )}^2-4(\nu (J+\nu )-K)$, the ensuing outcome is described in the following theorem.
  Theorem 7.

  If any of the conditions $\left(i\right)$$J>0$, $K\ge 0$, $\left(ii\right)$$J>0$, $K\le 0$, or $\left(iii\right)$$J\le 0$, $K\ge 0$ hold, then system (42) has the following:

  (a) 

  Two equilibrium points ${\overline{E}}_1={\overline{x}}_1$ and ${\overline{E}}_2={\overline{x}}_2$ when ${\Delta }_1>0$, where ${\overline{x}}_1={\displaystyle \frac{J+\nu -\sqrt{{\Delta }_1}}{2}}$ and ${\overline{x}}_2={\displaystyle \frac{J+\nu +\sqrt{{\Delta }_1}}{2}}$;

  (b) 

  One equilibrium point ${\overline{E}}_3={\overline{x}}_3$ when ${\Delta }_1=0$, where ${\overline{x}}_3={\displaystyle \frac{J+\nu }{2}}$;

  (c) 

  No equilibrium point when ${\Delta }_1<0$.

  The stability of system (42) can be determined by the sign of $f^{\prime }\left(x\right)$ at various equilibrium points [46]. Differentiating $f\left(x\right)$ given in Equation (42), we obtain

  $$f^{\prime }\left(x\right)={\displaystyle \frac{x(1-2x+\beta )}{(1+\theta y)(x+\gamma )}}-{\displaystyle \frac{x(1-x)(x-\beta )}{(1+\theta y){(x+\gamma )}^2}}+{\displaystyle \frac{\xi xy}{{(x+\delta )}^2}}+{\displaystyle \frac{(1-x)(x-\beta )}{(1+\theta y)(x+\gamma )}}-{\displaystyle \frac{\xi y}{x+\delta }}$$

  The theorem stated below and Figure 5 show the stability of system (42).
  Theorem 8.

  (a)  The trivial equilibrium point is always stable, as $f^{\prime }\left(x\right)<0$.

  (b) 

  The equilibrium points ${\overline{E}}_1$, ${\overline{E}}_2$, and ${\overline{E}}_3$, if they exist, are unstable when ${\displaystyle \frac{{\overline{x}}_1(1-2{\overline{x}}_1+\beta )}{(1+\theta y)({\overline{x}}_1+\gamma )}}+{\displaystyle \frac{\xi {\overline{x}}_{1}y(\gamma -\delta )}{({\overline{x}}_1+\gamma ){({\overline{x}}_1+\delta )}^2}}>0$, stable when ${\displaystyle \frac{{\overline{x}}_2(1-2{\overline{x}}_2+\beta )}{(1+\theta y)({\overline{x}}_2+\gamma )}}+{\displaystyle \frac{\xi {\overline{x}}_{2}y(\gamma -\delta )}{({\overline{x}}_2+\gamma ){({\overline{x}}_2+\delta )}^2}}<0$, and semi-stable (saddle-node bifurcation), respectively.

• Now, coming back to the original system (5), we present the influence of fear and the double Allee effect. Figure 6a–c illustrate that as the Allee effect parameters ($\beta$ and $\gamma$) and fear effect parameter ($\theta$), respectively, were increased, the equilibrium point $E_2^{*}$ moved downward. This implies that an increase in the double Allee effect parameters and the fear effect led to the existence of an equilibrium point at a lower population. As soon as the critical value of these parameters was crossed, the system moved toward the prey-free equilibrium, which is stable in nature.
  Now, to observe the collective influence of the fear effect and the double Allee effect, 3D phase portraits were used. Figure 7a shows the behavior of the prey population at the stable equilibrium point $E_2^{*}$ against the parameters $\theta$ and $\beta$, and it can be seen that the prey population decreased as the values of the parameters $\theta$ and $\beta$ increased. Similarly, Figure 7b reflects the same result with respect to the parameters $\theta$ and $\gamma$. Now, we know that $y_2^{*}$ = $x_2^{*}+\delta$, which implies the predator population will also behave in the same way as the prey population regarding the parameters $\theta$ and $\beta$, as well as $\theta$ and $\gamma$. From this, we could conclude that the higher the fear and value of the Allee effect parameters, the lower the population where system (5) attained stability. Ecologically, one can infer that escalation in the fear effect leads to a decline in prey and predator populations. As this fear intensifies, the prey population indulges less in foraging, and their interaction with each other decreases, which ultimately leads to a lower reproduction rate and, consequently, a reduced overall population. The diminishing predator–prey interaction is attributed to a declining prey population, which results in a decline in the population of predators. The populations of both species drop to the point where prey go extinct and predators survive (due to the presence of additional food).

7. Discussion

The conservation of the ecosystem relies on our capability to comprehend the delicate and dynamic intricacy of ecology. A better comprehension of the dynamics enables us to recognize potential threats to ecological stability. The ecosystem comprises various interactions between species that are complex and dynamic. In this work, we studied one such complex part of the ecosystem, which is the predator–prey interaction. For this purpose, we modified the system presented by Singh et al. [22], which focuses on the study of the double Allee effect in the modified LGPP model. In their study, the authors considered direct killing as the only way in which predator populations alter the population of prey. In the current paper, we improved the realism of this system by incorporating the fear effect.

The relevant properties for the feasibility of the proposed model were studied. Analytically and numerically, the proposed system exhibits at most six equilibrium points, including both axial and interior equilibria. The existence of axial equilibria remains unaffected whatever the values of parameters, while the system admits zero, one, or two interior equilibria depending on the parametric conditions. After examining the stability, it is seen that the trivial equilibrium point is consistently a saddle; the prey-free equilibrium point is an asymptotically stable point; and one of the two predator-free equilibrium points is consistently a saddle, while the other is an unstable point. In terms of ecology, it can be said that the predator and prey population cannot go extinct simultaneously, but in some cases, the prey species will move toward extinction while predators will never go extinct. If there are two interior equilibrium points, one is consistently a saddle, while the other will switch between a stable point, unstable point, or surrounded by an unstable limit cycle depending on certain parametric restrictions. Ecologically speaking, this means that either both prey and predator populations will coexist, prey will cease to exist, or there will be oscillating behavior. If there is a unique interior equilibrium point, then it is either a saddle node or a cusp of codimension two, contingent on the parametric restrictions. Ecologically, we can say that either the species will coexist or the prey will go extinct, contingent upon the initial population that the system commenced with. The initial values of both species has a great influence in determining the future dynamics of the system.

The diverse bifurcations exhibited by the proposed system were also analyzed. The phenomenon under study, that is, the fear effect, plays a vital role in the occurrence of the bifurcation. The possibility of the existence of zero, one, or two positive equilibrium points depends on the critical value of the fear effect parameter in the proposed model. The emergence of saddle-node bifurcation was proved by using Sotomayer’s theorem. Ecologically, both populations will coexist if $\theta$ does not cross the critical value $\theta ={\theta }^{SN}$, and if the critical value is crossed, then the prey will go extinct; that is, after a certain degree of fear, the prey population will tend to extinction. The proposed model also exhibits a limit cycle because of the emergence of a Hopf bifurcation. The stability of this limit cycle and the presence of a homoclinic loop (formed by the convergence of a saddle point and limit cycle) were also studied numerically. The system also possesses a Bogdanov–Takens bifurcation of codimension two and is explored using both analytical methods and numerical simulation. Ecologically speaking, this bifurcation leads to the emergence of various regions in the vicinity of the BT-point, each possessing unique properties, like the extinction of prey species, the coexistence of both species, or the oscillating behavior of prey and predator.

The influence of fear and the double Allee effect on the species’ growth rate was also examined. The analysis of the system was undertaken using two scenarios: considering the predator as constant and then variable. Furthermore, to dig deeper into the study of the fear effect, the influence of fear with each Allee effect parameter, individually, was also explored. It was observed as the Allee effect parameters and fear effect were increased, the stable equilibrium point $E_2^{*}$ moved downward. And as these parameters crossed their critical value, the system moved toward the prey-free equilibrium point. From an ecological point of view, it can be said that the proposed system is highly dependent on the double Allee effect and fear effect parameters and the prey species can cease to exist if the degree of these phenomena is increased beyond certain values of these parameters.

In conclusion, the present study can help in better understanding the predator–prey system, as it incorporates the fear effect, which takes us one step closer to a more realistic ecological model.