Behavior-Based Herding Algorithm for Social Force Model Based Sheep Herd

Abstract

Inspired by real-world sheepdog herding behavior, in this paper, four behavior-based herding algorithms have been proposed for the social force model-based sheep herd. First, a basic behavior-based herding algorithm is designed where four types of critical sheep are rigorously defined. The decision of the sheepdog is made by constantly checking the positions of these four critical sheep. Then, on top of this basic herding algorithm, two extra mechanisms are considered to improve the performance of the basic herding algorithm, namely the dynamic far-end mechanism and the pausing mechanism, thus, forming the other three herding algorithms. The dynamic far-end mechanism helps to avoid the undesired circling behavior of the sheepdog around the destination area, while the pausing mechanism can greatly reduce the control cost of the sheepdog. To validate the effectiveness of the proposed herding algorithms, comprehensive tests have been conducted. The performance of the four algorithms is evaluated and compared from three aspects, namely, success rate, completion step, and control cost. Moreover, parameter analysis is provided to examine how different design parameters will affect the performance of the proposed algorithm. Finally, it is shown that when the size of the sheep herd increases, as expected, it takes more time and control effort to complete herding.

1. Introduction

Swarm movements for living creatures exist widely in nature, such as the V-shaped migration of migratory birds [1], the swarming phenomenon of bees [2], the coordination and self-organization of ant colonies, and the vortex swimming of sardine swarms [3], which have attracted extensive attention from researchers in various related fields. Fundamental works on swarm systems can be found in Aoki [4], Reynolds [5], and Huth and Wissel [6,7], just to name a few, leading to fruitful subsequent results. Various collective and cooperative behaviors of swarm systems have found engineering applications in many scenarios, for instance, navigation of unmanned aerial vehicles flying in formation [8,9], group patrolling of unmanned vehicles [10,11], design and application of mobile sensor network in multi-environment [12], cooperation of swarm robots to complete collective tasks [13], and cooperative exploration by a flock of underwater vehicles [14,15].

Sheep herd is a typical swarm system, and the shepherds are good at teaching sheepdogs how to work, that is, to control a swarm system. While it was not until recently that people began to study the underlying mechanism of herding. In 1996, Schultz [16] used genetic algorithms to create decision rules to replicate sheep herding behavior. Later, John [17] used cognitive modeling to enable a T-rex to drive a flock of raptors from one territory to another. Vaughan [18] constructed a potential field model of cluster behavior and used it to study generalized cluster control. The side-to-side idea for herding was initiated by [19,20], where a global dynamic road map should be planned to provide milestones and steering points. Since then, a large number of studies on herding have emerged based on the side-to-side approach. Bennett [21] proposed circling behavior, where sheepdogs move on the rear periphery of the flock. Fujioka [22] proposed a V-formation control method. Behind the flock, three points are dynamically selected to form a V-formation, and the shepherd switches positions among these three points. Zhang [23] modeled the sheep by three characteristics: dispersal preference, predator avoidance, and aggregation preference, and proposed the so-called outmost push strategy. It turns out that this strategy also leads to side-to-side motion. In practice, the herding algorithms can be used to drive passive or non-cooperative swarm robots from one location to another [24,25,26,27]. Song [24] formalized the concept of a cage in a topological way to guide a group of mobile agents to a specified goal region. The ideas of [25,26,27] were similar, that is, to let multiple dogs encircle the flock, and in this way, herding can be achieved by letting the dogs move towards the destination in formation.

In this paper, inspired by real-world sheepdog herding behavior, four behavior-based herding algorithms are proposed for the social force model-based sheep herd. The research flowchart of this paper is shown in Figure 1.

Sheep Herd Modeling: In this paper, we use the social force model [28,29,30,31,32] to describe the behavior of the sheep herd, where the interaction between individuals is treated as force, and the resultant force of all the neighbors forms the social force. In particular, in this paper, the force is fully determined by the relative position of two individuals, forming four zones from inside to outside. The first, inner-most zone is the repulsive zone; the second zone is a neutral zone where no force is activated; the third zone is the attractive zone; the fourth, outer-most zone is the zone where the two individuals are beyond the perceptive range of each other, and, thus, again no force is activated.

Behavior-based Herding Algorithm: The basic idea of the herding algorithm, like in [19,20], is to let the sheepdog move behind the sheep herd in a side-to-side manner, forming a repeated semi-circle route. Based on this idea, a basic behavior-based herding algorithm is first designed, where four types of critical sheep are rigorously defined. The decision of the sheepdog is made by constantly checking the positions of these four critical sheep. (The initial version of this part of the work was documented in our previous work [33].) While different from the existing results, we do not set external steering points or milestones outside the sheep herd to achieve side-to-side motion, but use critical sheep as temporary reference points. By doing so, the sheepdog shall stick to the contour of the sheep herd, which would greatly reduce the odds of sheep herd fragmentation due to improper route of the sheepdog as mentioned in [19,20]. Moreover, on top of this basic herding algorithm, two extra mechanisms are considered to improve the performance of the basic herding algorithm, namely the dynamic far-end mechanism and the pausing mechanism, thus, forming the other three herding algorithms.

Related Works and Contributions: In contrast to the existing works, the contributions of this work are summarized as follows. (i) In [21], the direction of the dog’s circling is switched based on the centroid of the flock. However, when the number of sheep is large, the centroid of the flock might not be available. Instead, in our work, we put a limit on the perceptive range of the sheepdog. Moreover, the sheep might be vision-blocked by other sheep from the viewpoint of the sheepdog, even if it is within the perceptive range of the sheepdog. At each step, the sheepdog can only decide based on the visible sheep to it, which makes the algorithm of this work more practical. (ii) Some existing works, say, [23], assume that the boundary of the destination area is soft. In this way, once a sheep is driven into the destination area, it cannot go outside by itself. While, in this paper, we consider the more challenging scenario where no specific requirement is imposed on the destination area, which makes herding more difficult in the final stage when the sheep herd is around the destination area. To tackle this problem, we have designed a dynamic far-end mechanism that can effectively solve the circling issue in the final stage of herding. (iii) In the real world, the sheepdog sometimes pauses when herding. This is because the “inertia” of a big sheep herd is large, and, thus, it takes time for the repulsive force imposed by the sheepdog to propagate over the sheep herd. The appropriate pause may save energy for the sheepdog, thus making herding more cost-effective. Therefore, we have further proposed a pausing mechanism, which can greatly reduce the control cost, as validated by the simulation results.

2. Notation

Let $\mathbb{R}$ and $\mathbb{N}$ denote the sets of real numbers and positive integers, respectively. Given $x\in \mathbb{N}$, $\underline{x}\triangleq \left\{y\right|y\in \mathbb{N},y\le x\}$. Given a set $\mathbb{P}$ with a finite number of elements, let $\mathbf{C}\left(\mathbb{P}\right)$ denotes the cardinality of the set $\mathbb{P}$. For a vector $x\in {\mathbb{R}}^n$, $\left|\right|x\left|\right|$ denote the Euclidean norm of x. Given a point $x\in {\mathbb{R}}^2$ and a region $\mathbb{X}\subseteq {\mathbb{R}}^2$, the distance between x and $\mathbb{X}$ is defined by

$$\mathbf{d}(x,\mathbb{X})=\underset{y\in \mathbb{X}}{inf}\left|\right|x-y\left|\right|.$$

Given angle $\theta$, the planar rotation matrix is defined by

$$\mathbf{R}\left(\theta \right)=\left[\begin{array}{cc}\cos \left(\theta \right)& -\sin \left(\theta \right)\\ \sin \left(\theta \right)& \cos \left(\theta \right)\end{array}\right].$$

For a nonzero vector $x\in {\mathbb{R}}^2$, let

$$\mathbf{o}\left(x\right)=x/\left|\right|x\left|\right|$$

denote the unit directional vector of x. Moreover, the left-hand side area of x is defined by

$$\begin{array}{cc}\hfill {\mathbb{S}}_l\left(x\right)& =\{y\in {\mathbb{R}}^2|\exists \theta \in (0,\pi ),\mathrm{s}.\mathrm{t}.,\mathbf{o}\left(y\right)=\mathbf{R}\left(\theta \right)\mathbf{o}\left(x\right)\},\hfill \end{array}$$

and the right-hand side area of x is defined by

$$\begin{array}{cc}\hfill {\mathbb{S}}_r\left(x\right)& =\{y\in {\mathbb{R}}^2|\exists \theta \in (-\pi ,0),\mathrm{s}.\mathrm{t}.,\mathbf{o}\left(y\right)=\mathbf{R}\left(\theta \right)\mathbf{o}\left(x\right)\}.\hfill \end{array}$$

For two nonzero vectors $x,y\in {\mathbb{R}}^2$, the angle between x and y is given by

$$\mathrm{\Theta }(x,y)=arccos\left(\frac{\langle x,y\rangle }{\left|\right|x\left|\right|·\left|\right|y\left|\right|}\right)$$

where $\langle x,y\rangle$ denotes the inner product of the two vectors $x,y$. For $x\in {\mathbb{R}}^n$, define the following saturation function

$$\mathrm{\Gamma }(x,\xi )=\left\{\begin{array}{cc}x,\hfill & \left|\right|x\left|\right|\le \xi \hfill \\ \xi \mathbf{o}\left(x\right),\hfill & \left|\right|x\left|\right|>\xi \hfill \end{array}\right.$$

where $\xi >0$ denotes the saturation threshold. Suppose there are three distinct points $x,y,z\in {\mathbb{R}}^2$. For point x, if

$$\mathbf{o}(x-y)=\mathbf{o}(z-y)\mathrm{and}\left|\right|x-y\left|\right|>\left|\right|z-y\left|\right|$$

then point x is called vision-blocked by point z for point y. Obviously, vision-block is mutual, i.e., if point x is vision-blocked by point z for point y, then it follows that point y is vision-blocked by point z for point x.

3. System and Task Description

In this paper, as illustrated in Figure 2, we consider the herding problem, i.e., to drive a sheep herd from an initial location to some predetermined destination area by a single sheepdog. Throughout this paper, it is assumed that the positions of all the sheep and the sheepdog are expressed in a common coordinate.

Suppose the sheep herd consists of N sheep. For $i\in \underline{N}$, the motion of the ith sheep is governed by the following equation:

$$p_i(k+1)=p_i\left(k\right)+T{v}_i\left(k\right)$$

(1)

where T denotes the sampling period, $p_i\left(k\right),v_i\left(k\right)\in {\mathbb{R}}^2$ denote the position and velocity of the ith sheep at the kth step, respectively.

The motion of the sheepdog is governed by the following equation:

$$q(k+1)=q\left(k\right)+Tu\left(k\right)$$

(2)

where $q\left(k\right),u\left(k\right)\in {\mathbb{R}}^2$ denote the position and velocity of the sheepdog at the kth step, respectively.

For $i,j\in \underline{N}$, $i\ne j$, define

$$\begin{array}{cc}\hfill p_{ij}\left(k\right)& =p_i\left(k\right)-p_j\left(k\right)\hfill \\ \hfill p_i^q\left(k\right)& =p_i\left(k\right)-q\left(k\right).\hfill \end{array}$$

At step k, for sheep i, if sheep j is not vision-blocked by any other sheep or the sheepdog, and satisfies $\left|\right|p_{ij}\left(k\right)\left|\right|\le {\rho }_{vp}$ where ${\rho }_{vp}>0$ denotes the perceptive radius of the sheep, then sheep j is called visible to sheep i. Let ${\mathbb{V}}_{pi}\left(k\right)$ denote the label set of all the sheep visible to sheep i at step k.

Similarly, at step k, for sheep i, if the sheepdog is not vision-blocked by any other sheep, and satisfies $\left|\right|p_i^q\left(k\right)\left|\right|\le {\rho }_{vp}$, then the sheepdog is called visible to sheep i. Define $a_i\left(k\right)=1$ if the sheepdog is visible to sheep i at step k, and $a_i\left(k\right)=0$ otherwise.

At step k, for the sheepdog, if sheep i is not vision-blocked by any other sheep, and satisfies $\left|\right|p_i^q\left(k\right)\left|\right|\le {\rho }_{vq}$ where ${\rho }_{vq}>0$ denotes the perceptive radius of the sheepdog, then sheep i is called visible to the sheepdog. Let ${\mathbb{V}}_q\left(k\right)$ denote the label set of all the sheep visible to the sheepdog at step k.

If $\mathbf{C}\left({\mathbb{V}}_q\left(k\right)\right)>0$, then the center of the visible sheep for the sheepdog at step k is defined by

$$p^c\left(k\right)=\frac{{\displaystyle \sum _{i\in {\mathbb{V}}_q\left(k\right)}}{p}_i\left(k\right)}{\mathbf{C}\left({\mathbb{V}}_q\left(k\right)\right)}.$$

(3)

The above definitions are illustrated by an example shown by Figure 3, where sheep 2 is vision-blocked by sheep 5 for the sheepdog. In this scenario, ${\mathbb{V}}_q\left(k\right)=\{1,3,5,6\}$, ${\mathbb{V}}_{p9}\left(k\right)=\{2,4,7,8\}$, and $a_9\left(k\right)=0$.

In this paper, we assume that the motion of the sheep herd follows the social force rules. To be more specific, for $i\in \underline{N}$, the velocity of the ith sheep is set as follows:

$$v_i\left(k\right)=\mathrm{\Gamma }(v_{qi}\left(k\right)+v_{pi}\left(k\right)+n_i\left(k\right),{\xi }_p)$$

(4)

where ${\xi }_p$ denotes the upper limit for the speed of sheep. $n_i\left(k\right)$ denotes noise, which takes the following form

$$n_i\left(k\right)=\frac{1}{4}{\xi }_p{\mathbf{1}}_{mi}\left(k\right){[\cos \left(2\pi {\mathbf{1}}_{pi}\left(k\right)\right),\sin \left(2\pi {\mathbf{1}}_{pi}\left(k\right)\right)]}^T$$

(5)

where ${\mathbf{1}}_{mi}\left(k\right),{\mathbf{1}}_{pi}\left(k\right)$ uniformly take random values from interval $[0,1]$. $v_{qi}\left(k\right)$ represents the force imposed by the sheepdog, which is given by

$$v_{qi}\left(k\right)=a_i\left(k\right)\phi \left(\right||p_i^q\left(k\right)\left|\right|)\mathbf{o}\left(p_i^q\left(k\right)\right)$$

(6)

where

$$\phi \left(x\right)=\alpha \left(\frac{{\rho }_{vp}-x}{x-{\rho }_x}\right)$$

(7)

with $\alpha >0$ denoting the intensity of sheepdog-to-sheep repulsive force, ${\rho }_x$ denoting the allowable minimal distance between sheep and sheepdog. The plot of $\phi \left(x\right)$ is shown by Figure 4a with $\alpha =1$, ${\rho }_x=1.2$, and ${\rho }_{vp}=15$. $v_{pi}\left(k\right)$ represents the force imposed by other sheep, which is given by

$$v_{pi}\left(k\right)=\sum _{j\in {\mathbb{V}}_{pi}\left(k\right)}\psi \left(\right||p_{ij}\left(k\right)\left|\right|)\mathbf{o}\left(p_{ij}\left(k\right)\right)$$

(8)

where

$$\psi \left(x\right)=\left\{\begin{array}{cc}\beta \left(\frac{{\rho }_r-x}{x-{\rho }_s}\right),\hfill & 0<{\rho }_s<x\le {\rho }_r\hfill \\ 0,\hfill & {\rho }_r<x\le {\rho }_g\hfill \\ \gamma \sqrt{x-{\rho }_g},\hfill & x>{\rho }_g\hfill \end{array}\right.$$

(9)

with $\beta >0,\gamma <0$ denoting the intensity of sheep-to-sheep repulsive and attractive force, respectively. ${\rho }_s>0$ denotes the minimal sheep-to-sheep safety distance. The plot of $\psi \left(x\right)$ is shown by Figure 4b with $\beta =3$, $\gamma =1$, ${\rho }_s=1.2$, ${\rho }_r=3$, and ${\rho }_g=7$. For (9) to make sense, it must hold that ${\rho }_{vp}>{\rho }_g$. By (8), the sheep herd tends to get together when they are scattered yet not overly sparse, and, in the meantime, keep minimal inter-sheep distance. Note that in between attractive and repulsive areas, there exists some certain area, dictated by ${\rho }_r$ and ${\rho }_g$, within which there is no force between two sheep. In this way, the sheep herd can reach and maintain a static balance.

The sheep herd polygon is defined by

$$\begin{array}{cc}\hfill {\mathbb{P}}_p\left(k\right)& =\left\{x\left(k\right)|x\left(k\right)\in {\mathbb{R}}^2,x\left(k\right)=\sum _{i=1}^N{m}_i{p}_i\left(k\right),\right.\left.0\le m_i\le 1,\sum _{i=1}^N{m}_i=1.\right\}\hfill \end{array}$$

Given $p_d\in {\mathbb{R}}^2$, the destination area for the sheep herd is defined by

$${\mathbb{P}}_d=\left\{x\right|x\in {\mathbb{R}}^2,\left|\right|x-p_d\left|\right|\le {\rho }_d\}$$

(10)

for some ${\rho }_d>0$. ${\rho }_d$ should be chosen such that the following two requirements can be simultaneously satisfied:

$$x_i\in {\mathbb{P}}_d,\forall i\in \underline{N}$$

(11)

and

$$\left|\right|x_i-x_j\left|\right|\ge {\rho }_r,\forall i,j\in \underline{N},i\ne j.$$

(12)

Mathematically, the herding problem considered in this paper is to design $u\left(k\right)$ for the sheepdog, under the initial condition

$$\mathbf{d}(q\left(0\right),{\mathbb{P}}_p\left(0\right))>0,{\mathbb{V}}_q\left(0\right)\ne \varnothing$$

(13)

and

$$\left|\right|p_i^q\left(0\right)\left|\right|>{\rho }_x,\left|\right|p_{ij}\left(0\right)\left|\right|>{\rho }_s,\forall i,j\in \underline{N},i\ne j$$

(14)

such that, for all $i,j\in \underline{N},i\ne j$, and all $k\ge 0$,

$$\left|\right|p_i^q\left(k\right)\left|\right|>{\rho }_x,\left|\right|p_{ij}\left(k\right)\left|\right|>{\rho }_s$$

(15)

and

$$\mathbf{d}(p_i\left(k^{*}\right),{\mathbb{P}}_d)=0,\forall i\in \underline{N}$$

(16)

for some $k^{*}\le K$, where K is the step limit.

4. Herding Algorithm

By the observation of the behaviors of the real-world sheepdog, in this section, four behavior-based herding algorithms will be proposed in order. First, a basic behavior-based herding algorithm is designed where four types of critical sheep are rigorously defined. The decision of the sheepdog is made by constantly checking the positions of these four critical sheep. Then, on top of this basic herding algorithm, two extra mechanisms are considered to improve the performance of the basic herding algorithm, namely the dynamic far-end mechanism and the pausing mechanism, thus, forming the other three herding algorithms.

4.1. Behavior-Based Herding Algorithm

To begin with, we introduce four important directions:

$$\begin{array}{cc}\hfill D^{cd}\left(k\right)& =\mathbf{o}(p_d-p^c\left(k\right))\hfill \\ \hfill D^{cdl}\left(k\right)& =\mathbf{R}\left(\frac{\pi }{2}\right)D^{cd}\left(k\right)\hfill \\ \hfill D^{cdr}\left(k\right)& =\mathbf{R}\left(-\frac{\pi }{2}\right)D^{cd}\left(k\right)\hfill \\ \hfill D^{qc}\left(k\right)& =\mathbf{o}(p^c\left(k\right)-q\left(k\right)).\hfill \end{array}$$

(17)

$D^{cd}\left(k\right)$ denotes the unit vector from the center of the visible sheep to the destination. $D^{cdl}\left(k\right)$ and $D^{cdr}\left(k\right)$ are the left and right perpendicular vector of $D^{cd}\left(k\right)$, respectively. $D^{qc}\left(k\right)$ denotes the unit vector from the sheepdog to the center of the visible sheep. Based on $D^{cd}\left(k\right)$, we further define

$${\mathbb{V}}_q^l\left(k\right)=\left\{i\right|p_i^q\left(k\right)\in {\mathbb{S}}_l\left(D^{qc}\left(k\right)\right)\},{\mathbb{V}}_q^r\left(k\right)=\left\{i\right|p_i^q\left(k\right)\in {\mathbb{S}}_r\left(D^{qc}\left(k\right)\right)\}.$$

(18)

Based on the direction of $D^{qc}\left(k\right)$, ${\mathbb{V}}_q^l\left(k\right)$ (${\mathbb{V}}_q^r\left(k\right)$) is the label set of the sheep which are on the left (right) hand side of the sheepdog.

Next, we define four types of critical sheep. To this end, define four functions as follows

$$\begin{array}{cc}\hfill H^l\left(i\right)& =\langle p_i^q\left(k\right),D^{cdl}\left(k\right)\rangle ,H^r\left(i\right)=\langle p_i^q\left(k\right),D^{cdr}\left(k\right)\rangle \hfill \\ \hfill F\left(i\right)& =\langle p_i^q\left(k\right),D^{qc}\left(k\right)\rangle ,G\left(p_i\left(k\right)\right)=\parallel p_i^q\left(k\right)\parallel \hfill \end{array}$$

(19)

where $H^l(·),H^r(·),F(·):\underline{N}\to \mathbb{R}$ and $G(·):{\mathbb{R}}^2\to \mathbb{R}$. For the ith sheep, $H^l\left(i\right)$, $H^r\left(i\right)$ and $F\left(i\right)$ denote the projections of $p_i^q\left(k\right)$ to the directions of $D^{cdl}\left(k\right)$, $D^{cdr}\left(k\right)$ and $D^{qc}\left(k\right)$, respectively. Then, define

$${\mathbb{V}}^{la}\left(k\right)=\underset{i\in {\mathbb{V}}_q^l\left(k\right)}{argmin}F\left(i\right),{\mathbb{V}}^{ra}\left(k\right)=\underset{i\in {\mathbb{V}}_q^r\left(k\right)}{argmin}F\left(i\right)$$

(20)

and

$${\mathcal{P}}^{la}\left(k\right)=\underset{i\in {\mathbb{V}}^{la}\left(k\right)}{argmin}G\left(p_i\left(k\right)\right),{\mathcal{P}}^{ra}\left(k\right)=\underset{i\in {\mathbb{V}}^{ra}\left(k\right)}{argmin}G\left(p_i\left(k\right)\right)$$

(21)

where ${\mathcal{P}}^{la}\left(k\right)$ and ${\mathcal{P}}^{ra}\left(k\right)$ are called left and right anchor sheep, respectively. For the sheep on the left (right) hand side of the sheepdog along the direction of $D^{qc}\left(k\right)$, the sheep with the least projection of $p_i^q\left(k\right)$ to the direction of $D^{qc}\left(k\right)$ constitute the set of ${\mathbb{V}}^{la}\left(k\right)$ (${\mathbb{V}}^{ra}\left(k\right)$), within which the nearest sheep to the sheepdog is defined to be the left (right) anchor sheep. Moreover, define

$${\mathbb{V}}^{lf}\left(k\right)=\underset{i\in \underline{N}}{argmax}{H}^l\left(i\right),{\mathbb{V}}^{rf}\left(k\right)=\underset{i\in \underline{N}}{argmax}{H}^r\left(i\right)$$

(22)

and

$${\mathcal{P}}^{lf}\left(k\right)=\underset{i\in {\mathbb{V}}^{lf}\left(k\right)}{argmax}G\left(p_i\left(k\right)\right),{\mathcal{P}}^{rf}\left(k\right)=\underset{i\in {\mathbb{V}}^{rf}\left(k\right)}{argmax}G\left(p_i\left(k\right)\right)$$

(23)

where ${\mathcal{P}}^{lf}\left(k\right)$ and ${\mathcal{P}}^{rf}\left(k\right)$ are called left and right front sheep, respectively. The sheep with the maximal projection of $p_i^q\left(k\right)$ to the direction of $D^{cdl}\left(k\right)$ ($D^{cdr}\left(k\right)$) constitute the set of ${\mathbb{V}}^{lf}\left(k\right)$ (${\mathbb{V}}^{rf}\left(k\right)$), within which the farthest sheep to the sheepdog is defined to be the left (right) front sheep.

Finally, define the left and right turning angles by

$${\theta }^{lt}\left(k\right)=\mathrm{\Theta }(D^{cd}\left(k\right),q\left(k\right)-{\mathcal{P}}^{lf}\left(k\right)),{\theta }^{rt}\left(k\right)=\mathrm{\Theta }(D^{cd}\left(k\right),q\left(k\right)-{\mathcal{P}}^{rf}\left(k\right)).$$

(24)

Examples illustrating the aforementioned definitions are shown in Figure 5 and Figure 6. Now, the behavior-based herding algorithm is given by Algorithm 1.

Below we make some explanations for Algorithm 1. To begin with, we check the algorithm termination condition, i.e., the step limit is hit, or the herding task is fulfilled. If neither of these conditions is satisfied, the algorithm continues, calculating the next move for the sheepdog. The basic idea of the herding algorithm, like in [19,20], is to let the sheepdog move behind the sheep herd in a side-to-side manner, forming a repeated semi-circle route. While different from the existing results, we do not set external steering points or milestones outside the sheep herd to achieve side-to-side motion, but use critical sheep as temporary reference points. By doing so, the sheepdog shall stick to the contour of the sheep herd, which would greatly reduce the odds of sheep herd fragmentation due to improper route of the sheepdog as mentioned in [19,20]. In general, there are two main issues to address. First, how to make the sheepdog stick to the contour of the sheep herd? We solve this issue by defining anchor sheep, which are dynamically changing as the sheepdog moves. When the sheepdog uses these anchor sheep as reference points, the detouring angle $\vartheta$ makes the route bend in the direction of outliers, thus preventing the sheepdog from breaking into the sheep herd directly. Second, how to determine the turning point for the sheepdog? The turning behavior is activated when two criteria are met simultaneously. On the one hand, the entire sheep herd is on one side of the line connecting the sheepdog and the far end of the destination, which is chosen to be the destination center in Algorithm 1. On the other hand, the turning angle should be small enough, less than the turning-angle threshold ${\overline{\theta }}^t$. In Algorithm 1, the state-flag $\lambda \left(k\right)$ is used to identify the moving direction of the sheepdog at step k.

Algorithm 1 Behavior-based Herding Algorithm

1. select turning-angle threshold ${\overline{\theta }}^t$, velocity coefficient $\upsilon$, and detouring angle $\vartheta$. 2. set state-flag $\lambda \left(0\right)=1$. 3. set up step limit K. for ($k=1$, $k\le K$, $k=k+1$) do  if $k=K$ or $\mathbf{d}(p_i\left(k\right),{\mathbb{P}}_d)=0$, $\forall i\in \underline{N}$ then   Algorithm terminates.  else   if $\forall i\in {\mathbb{V}}_q\left(k\right),(p_i\left(k\right)-p_d)\in {\mathbb{S}}_l(q\left(k\right)-p_d)$ $\&{\theta }^{lt}\left(k\right)<{\overline{\theta }}^t$ then    $u_p\left(k\right)=\upsilon \mathbf{R}(-\vartheta )({\mathcal{P}}^{ra}\left(k\right)-q\left(k\right))$, $\lambda \left(k\right)=1$.   else if $\forall i\in {\mathbb{V}}_q\left(k\right),(p_i\left(k\right)-p_d)\in {\mathbb{S}}_r(q\left(k\right)-p_d)$ $\&{\theta }^{rt}\left(k\right)<{\overline{\theta }}^t$ then    $u_p\left(k\right)=\upsilon \mathbf{R}\left(\vartheta \right)({\mathcal{P}}^{la}\left(k\right)-q\left(k\right))$, $\lambda \left(k\right)=-1$.   else if $\lambda \left(k\right)=1$ then    $u_p\left(k\right)=\upsilon \mathbf{R}(-\vartheta )({\mathcal{P}}^{ra}\left(k\right)-q\left(k\right))$.   else    $u_p\left(k\right)=\upsilon \mathbf{R}\left(\vartheta \right)({\mathcal{P}}^{la}\left(k\right)-q\left(k\right))$.   end if   $u\left(k\right)=\mathrm{\Gamma }(u_p\left(k\right),{\xi }_q)$  end if end for

4.2. Herding Algorithm with Dynamic Far-End Mechanism

Some existing works, say, [23], assume that the boundary of the destination area is soft. In this way, once a sheep is driven into the destination area, it cannot go outside by itself. While, in this paper, we consider the more challenging scenario where no specific requirement is imposed on the destination area, which makes herding more difficult in the final stage when the sheep herd is around the destination area. To tackle this problem, we improve Algorithm 1 by adding the dynamic far-end mechanism, which is detailed as follows. Define the dynamic far-end as

$$p^{df}\left(k\right)=p_d+{\rho }_d{D}^{cd}\left(k\right).$$

(25)

The definition of $p^{df}\left(k\right)$ is illustrated by Figure 7. By $p^{df}\left(k\right)$, we use the line connecting the sheepdog and the dynamic far end rather than the static destination center as in Algorithm 1 to determine whether all the sheep are on one side of the line or not, which is one of the criteria to determine the turning point for the sheepdog. In this way, we can avoid the failure that probably happens when the sheep herd is around the destination. With the dynamic far-end mechanism, the herding algorithm is given by Algorithm 2.

Algorithm 2 Herding Algorithm With Dynamic Far-end Mechanism

1. select turning-angle threshold ${\overline{\theta }}^t$, velocity coefficient $\upsilon$, and detouring angle $\vartheta$. 2. set state-flag $\lambda \left(0\right)=1$. 3. set up step limit K. for ($k=1$, $k\le K$, $k=k+1$) do  if $k=K$ or $\mathbf{d}(p_i\left(k\right),{\mathbb{P}}_d)=0$, $\forall i\in \underline{N}$ then   Algorithm terminates.  else   if $\forall i\in {\mathbb{V}}_q\left(k\right),(p_i\left(k\right)-p^{df}\left(k\right))\in {\mathbb{S}}_l(q\left(k\right)-p^{df}\left(k\right))$ $\&{\theta }^{lt}\left(k\right)<{\overline{\theta }}^t$ then    $u_p\left(k\right)=\upsilon \mathbf{R}(-\vartheta )({\mathcal{P}}^{ra}\left(k\right)-q\left(k\right))$,    $\lambda \left(k\right)=1$.   else if $\forall i\in {\mathbb{V}}_q\left(k\right),(p_i\left(k\right)-p^{df}\left(k\right))\in {\mathbb{S}}_r(q\left(k\right)-p^{df}\left(k\right))$ $\&{\theta }^{rt}\left(k\right)<{\overline{\theta }}^t$ then    $u_p\left(k\right)=\upsilon \mathbf{R}\left(\vartheta \right)({\mathcal{P}}^{la}\left(k\right)-q\left(k\right))$,    $\lambda \left(k\right)=-1$.   else if $\lambda \left(k\right)=1$ then    $u_p\left(k\right)=\upsilon \mathbf{R}(-\vartheta )({\mathcal{P}}^{ra}\left(k\right)-q\left(k\right))$.   else    $u_p\left(k\right)=\upsilon \mathbf{R}\left(\vartheta \right)({\mathcal{P}}^{la}\left(k\right)-q\left(k\right))$.   end if   $u\left(k\right)=\mathrm{\Gamma }(u_p\left(k\right),{\xi }_q)$  end if end for

4.3. Herding Algorithm with Pausing Mechanism

In the real world, the sheepdog sometimes pauses when herding. This is because the “inertia” of a big sheep herd is large, and thus it takes time for the repulsive force imposed by the sheepdog to propagate over the sheep herd. The appropriate pause may save energy for the sheepdog, thus making herding more cost-effective. Inspired by this observation, we propose a pausing mechanism as follows.

Define the back declination angle as

$${\theta }^b\left(k\right)=\mathrm{\Theta }(D^{cd}\left(k\right),D^{qc}\left(k\right)).$$

(26)

The definition of ${\theta }^b\left(k\right)$ is illustrated by Figure 8. ${\theta }^b\left(k\right)=0$ means the sheepdog is exactly behind the center of the visible sheep along the direction of $D^{cd}\left(k\right)$. Around this spot, the repulsive force on the sheep herd nearly points to the destination center.

Moreover, define the approaching ratio as

$$r\left(k\right)=\frac{\parallel q\left(k\right)-p^c\left(k\right)\parallel }{{max}_{i\in {\mathbb{V}}_q\left(k\right)}\{\parallel p_i\left(k\right)-p^c\left(k\right)\parallel \}}.$$

(27)

Let $\overline{r}$ be the approaching threshold for $r\left(k\right)$. Whenever $r\left(k\right)>\overline{r}$, the sheepdog will approach the sheep herd.

Based on ${\theta }^b\left(k\right)$ and $r\left(k\right)$, the pausing mechanism is described as follows. Let ${\overline{\theta }}^b$ be the back-declination-angle threshold. When ${\theta }^b\left(k\right)<{\overline{\theta }}^b$, the sheepdog will decide whether to pause or not. Let $\underline{r}$ be the pausing threshold for $r\left(k\right)$, satisfying $\underline{r}<\overline{r}$. Under the condition that ${\theta }^b\left(k\right)<{\overline{\theta }}^b$, if $r\left(k\right)<\underline{r}$, the sheepdog will pause to drive the sheep herd towards the destination center. With the pausing mechanism, the herding algorithm is given by Algorithm 3.

Algorithm 3 Herding Algorithm With Pausing Mechanism

1. select back-declination-angle threshold ${\overline{\theta }}^b$, turning-angle threshold ${\overline{\theta }}^t$, pausing threshold $\underline{r}$, approaching threshold $\overline{r}$, velocity coefficient $\upsilon$, and detouring angle $\vartheta$. 2. set state-flag $\lambda \left(0\right)=1$. 3. set up step limit K. for ($k=1$, $k\le K$, $k=k+1$) do  if $k=K$ or $\mathbf{d}(p_i\left(k\right),{\mathbb{P}}_d)=0$, $\forall i\in \underline{N}$ then   Algorithm terminates.  else   if $\forall i\in {\mathbb{V}}_q\left(k\right),(p_i\left(k\right)-p_d)\in {\mathbb{S}}_l(q\left(k\right)-p_d)$ $\&{\theta }^{lt}\left(k\right)<{\overline{\theta }}^t$ then    $u_p\left(k\right)=\upsilon \mathbf{R}(-\vartheta )({\mathcal{P}}^{ra}\left(k\right)-q\left(k\right))$,    $\lambda \left(k\right)=1$.   else if $\forall i\in {\mathbb{V}}_q\left(k\right),(p_i\left(k\right)-p_d)\in {\mathbb{S}}_r(q\left(k\right)-p_d)$ $\&{\theta }^{rt}\left(k\right)<{\overline{\theta }}^t$ then    $u_p\left(k\right)=\upsilon \mathbf{R}\left(\vartheta \right)({\mathcal{P}}^{la}\left(k\right)-q\left(k\right))$,    $\lambda \left(k\right)=-1$.   else if $\lambda \left(k\right)=1$ then    if ${\theta }^b\left(k\right)<{\overline{\theta }}^b\&r\left(k\right)<\underline{r}$ then     $u_p\left(k\right)=0$.    else if $r\left(k\right)>\overline{r}$ then     $u_p\left(k\right)=\upsilon (p^c\left(k\right)-q\left(k\right))$.    else     $u_p\left(k\right)=\upsilon \mathbf{R}(-\vartheta )({\mathcal{P}}^{ra}\left(k\right)-q\left(k\right))$.    end if   else    if ${\theta }^b\left(k\right)<{\overline{\theta }}^b\&r\left(k\right)<\underline{r}$ then     $u_p\left(k\right)=0$.    else if $r\left(k\right)>\overline{r}$ then     $u_p\left(k\right)=\upsilon (p^c\left(k\right)-q\left(k\right))$.    else     $u_p\left(k\right)=\upsilon \mathbf{R}\left(\vartheta \right)({\mathcal{P}}^{la}\left(k\right)-q\left(k\right))$.    end if   end if   $u\left(k\right)=\mathrm{\Gamma }(u_p\left(k\right),{\xi }_q)$  end if end for

4.4. Herding Algorithm with Dynamic Far-End and Pausing Mechanism

Finally, by combining both the dynamic far-end and pausing mechanism with the herding algorithm, we obtain Algorithm 4.

Algorithm 4 Herding Algorithm With Dynamic Far-end and Pausing Mechanism

1. select back-declination-angle threshold ${\overline{\theta }}^b$, turning-angle threshold ${\overline{\theta }}^t$, pausing threshold $\underline{r}$, approaching threshold $\overline{r}$, velocity coefficient $\upsilon$, and detouring angle $\vartheta$. 2. set state-flag $\lambda \left(0\right)=1$. 3. set up step limit K. for ($k=1$, $k\le K$, $k=k+1$) do  if $k=K$ or $\mathbf{d}(p_i\left(k\right),{\mathbb{P}}_d)=0$, $\forall i\in \underline{N}$ then   Algorithm terminates.  else   if $\forall i\in {\mathbb{V}}_q\left(k\right),(p_i\left(k\right)-p^{df}\left(k\right))\in {\mathbb{S}}_l(q\left(k\right)-p^{df}\left(k\right))$ $\&{\theta }^{lt}\left(k\right)<{\overline{\theta }}^t$ then    $u_p\left(k\right)=\upsilon \mathbf{R}(-\vartheta )({\mathcal{P}}^{ra}\left(k\right)-q\left(k\right))$,    $\lambda \left(k\right)=1$.   else if $\forall i\in {\mathbb{V}}_q\left(k\right),(p_i\left(k\right)-p^{df}\left(k\right))\in {\mathbb{S}}_r(q\left(k\right)-p^{df}\left(k\right))$ $\&{\theta }^{rt}\left(k\right)<{\overline{\theta }}^t$ then    $u_p\left(k\right)=\upsilon \mathbf{R}\left(\vartheta \right)({\mathcal{P}}^{la}\left(k\right)-q\left(k\right))$,    $\lambda \left(k\right)=-1$.   else if $\lambda \left(k\right)=1$ then    if ${\theta }^b\left(k\right)<{\overline{\theta }}^b\&r\left(k\right)<\underline{r}$ then     $u_p\left(k\right)=0$.    else if $r\left(k\right)>\overline{r}$ then     $u_p\left(k\right)=\upsilon (p^c\left(k\right)-q\left(k\right))$.    else     $u_p\left(k\right)=\upsilon \mathbf{R}(-\vartheta )({\mathcal{P}}^{ra}\left(k\right)-q\left(k\right))$.    end if   else    if ${\theta }^b\left(k\right)<{\overline{\theta }}^b\&r\left(k\right)<\underline{r}$ then     $u_p\left(k\right)=0$.    else if $r\left(k\right)>\overline{r}$ then     $u_p\left(k\right)=\upsilon (p^c\left(k\right)-q\left(k\right))$.    else     $u_p\left(k\right)=\upsilon \mathbf{R}\left(\vartheta \right)({\mathcal{P}}^{la}\left(k\right)-q\left(k\right))$.    end if   end if   $u\left(k\right)=\mathrm{\Gamma }(u_p\left(k\right),{\xi }_q)$  end if end for

5. Test and Analysis

In this section, we will test and analyze Algorithms 1–4. First, we will show the typical herding process under Algorithms 1–4. Next, we will evaluate and compare the performance of Algorithms 1–4 from three aspects, namely, success rate, completion step, and control cost. Finally, we will perform parameter analysis to see how different parameters will affect the performance of the proposed algorithm.

5.1. Typical Herding Process under Algorithms 1–4

First of all, it will be shown that with proper parameter selection, all Algorithms 1–4 can achieve successful herding. The simulation conditions are detailed as follows.

• destination area related parameters: $p_d=(10,70)$, ${\rho }_d=9$.
• sheep herd related parameters: $N=24$, $\alpha =0.7$, ${\rho }_x=1.5$, ${\rho }_{vp}=20$, $\beta =5$, $\gamma =0.02$, ${\rho }_r=2.2$, ${\rho }_s=1.2$, ${\rho }_g=5.4$, ${\xi }_p=3$.
• sheepdog related parameters: ${\rho }_{vq}=30$, ${\xi }_q=5$.
• algorithm design parameters and thresholds: $\upsilon =0.6$, $\vartheta =0.6$, ${\overline{\theta }}^t=2.5$, ${\overline{\theta }}^b=0.1$, $\underline{r}=10/7$, $\overline{r}=4$.
• The initial positions for the sheep are given by $p_1\left(0\right)=(14,14)$, $p_2\left(0\right)=(-6,2)$, $p_3\left(0\right)=(4,6)$, $p_4\left(0\right)=(0,6)$,$p_5\left(0\right)=(-2,16)$, $p_6\left(0\right)=(12,-6)$, $p_7\left(0\right)=(18,4)$, $p_8\left(0\right)=(-6,-4)$, $p_9\left(0\right)=(14,-6)$, $p_{10}\left(0\right)=(14,12)$, $p_{11}\left(0\right)=(18,-6)$, $p_{12}\left(0\right)=(18,18)$, $p_{13}\left(0\right)=(-4,10)$, $p_{14}\left(0\right)=(6,14)$, $p_{15}\left(0\right)=(20,2)$, $p_{16}\left(0\right)=(10,0)$, $p_{17}\left(0\right)=(18,0)$, $p_{18}\left(0\right)=(18,-4)$, $p_{19}\left(0\right)=(18,2)$, $p_{20}\left(0\right)=(-6,16)$, $p_{21}\left(0\right)=(4,8)$, $p_{22}\left(0\right)=(2,-6)$, $p_{23}\left(0\right)=(-4,12)$, $p_{24}\left(0\right)=(20,8)$.
• The initial position for the sheepdog is given by $q\left(0\right)=(5,-10)$.

Note that the values of algorithm design parameters and thresholds are determined by trial and error through comprehensive numerical simulation data, where more detailed analyses are given in Section 5.4. Figure 9 shows that the herding task has been successfully achieved under all Algorithms 1–4.

5.2. Evaluation and Comparison of Algorithms 1–4

Now, we will evaluate and compare the performance of Algorithms 1–4 from three aspects, namely, success rate, completion step, and control cost, which are defined as follows.

• success rate

  $$\mathrm{\Xi }=\frac{N_s}{N_t}$$

  (28)

  where, for a given algorithm, we conduct $N_t$ many tests subject to different initial positions for the sheep and sheepdog, out of which $N_s$ many tests are successful. In this subsection, $N_t$ is set to be ${10}^3$.
• Given an algorithm, consider the $N_s$ successful tests, labeled from 1 to $N_s$. For the ith successful test, suppose the completion step is ${\kappa }_i$. Then, the completion step for the associated algorithm is defined to be

  $$\kappa =\frac{{\sum }_{i=1}^{N_s}{\kappa }_i}{N_s}.$$

  (29)
• Given an algorithm, consider the $N_s$ successful tests, labeled from 1 to $N_s$. For the ith successful test, define the control cost as

  $${\mathrm{{\rm Y}}}_i=\sum _{k=0}^{{\kappa }_i}{\left|\right|u\left(k\right)\left|\right|}^2.$$

  (30)

  Then, the control cost for the associated algorithm is defined to be

  $$\mathrm{{\rm Y}}=\frac{{\sum }_{i=1}^{N_s}{\mathrm{{\rm Y}}}_i}{N_s}.$$

  (31)

In this section, we take the same simulation conditions as in Section 5.1. The initial positions for the sheep is with the range of ${[-6\sim 20,-6\sim 20]}^T$. The data of simulation results are summarized by

Table 1. The data of simulation results.

	Algorithm 1	Algorithm 2	Algorithm 3	Algorithm 4
$\mathrm{\Xi }$	$76.3\%$	$100\%$	$91.9\%$	$100\%$
Compared to Algorithm 1		$23.7\%$	$13.6\%$	$23.7\%$
$\kappa$	$\mathrm{10,443.65}$	$9498.05$	$\mathrm{11,237.85}$	$\mathrm{10,449.89}$
Compared to Algorithm 1		$-9.05\%$	$7.64\%$	$0.06\%$
$\mathrm{{\rm Y}}$	$\mathrm{124,523.73}$	$\mathrm{114,985.87}$	$\mathrm{103,448.82}$	$\mathrm{95,252.18}$
Compared to Algorithm 1		$-7.66\%$	$-16.92\%$	$-23.51\%$

, which are also illustrated by Figure 10, Figure 11 and Figure 12.

In what follows, we will compare and explain the different performance of Algorithms 1–4. We begin with Algorithms 1 and 2. From Table 1, it can be seen that the success rate of Algorithm 2 is 100%, while the success rate of Algorithm 1 is much lower, around 76%. For most of the cases, the failure of Algorithm 1 happens at the final stage of herding, i.e., when the sheep herd are around the destination area. Note that one of the criteria for the sheepdog to change moving direction is that all the sheep are on one side of the line connecting the sheepdog and the far-end of the destination area, which is chosen to be the destination center in Algorithm 1. However, this criterion cannot be easily met when the destination area is not large enough since, in such case, the resulting line would probably pass through the sheep herd in the final stage, and, thus, the sheepdog keeps circling around the destination area without turning direction, thus leading to a failure. In contrast, in Algorithm 2, the far-end of the line has been changed from the destination center to the dynamic far-end, which is on the boundary of the destination area. This modification makes it easy for the turning criterion to be met, which helps to avoid the circling behavior of the sheepdog. Regarding the other two performance indices, i.e., completion step and control cost, as seen by Figure 11 and Figure 12, Algorithm 2 exhibits some little advantage in comparison with Algorithm 1. While, since there could still be occasional successful cases of Algorithm 1 where the sheepdog circling movement does happen in the final stage, the upper limits for the complete step and control cost of Algorithm 1 are much higher than those of Algorithm 2.

Next, we compare Algorithms 1 and 3. As illustrated before, since both Algorithms 1 and 3 adopt the destination center as the far end; they both suffer from the circling problem in the final stage. As a result, the success rates of both these two algorithms are below 100%. Algorithm 3 improves Algorithm 1 by introducing the pausing mechanism, which brings about two benefits. The major benefit is the control cost of Algorithm 3 is significantly reduced, up to almost 17% as shown by Table 1. The reason is obvious that the sheepdog would pause from time to time when the timing is appropriate. The minor benefit is in the final stage; the pausing mechanism helps eliminate the circling behavior, which in turn increases the success rate in comparison with Algorithm 1. Moreover, the simulation data, on the other hand, shows that the pausing mechanism increases the completion step on the magnitude of 7%. In all, there would be a compromise when both the completion step and the control cost are taken into consideration. As aforementioned, the circling problem will occur to Algorithms 1 and 3 only for small ${\rho }_d$, which will disappear when the destination area becomes larger and larger. The specific data are shown by Figure 13 when ${\rho }_d$ increases from 8 to 11 under Algorithm 3.

Finally, if we attach both the dynamic far-end and pausing mechanism to Algorithm 1, as expected, the success rate becomes 100% with the completion step increasing a little bit and the control cost is greatly reduced, as shown in Table 1.

5.3. Comparison with Other Existing Results

In this subsection, we will compare the performance of Algorithm 4 with that of the existing algorithm proposed by Bennett [21]. Bennett’s algorithm mimics the circling behavior of sheepdogs. Driving sheep herd is achieved by alternating the direction of circling. The motion vector is calculated according to the position of the leftmost or rightmost sheep from the perspective of the sheepdog, and then the direction of the sheepdog’s circling motion is obtained by adding an angle offset. The sheepdog should describe a circle of around 1/4 to 1/3 of the way around the sheep herd. The sheepdog’s alternating between two circling directions depends on the position of the sheep with respect to the goal and the angle of $\mathrm{\Theta }(D^{cd}\left(k\right),-D^{qc}\left(k\right))$. Once a circling pattern is formed, the sheepdog will keep in this pattern until a termination condition is met. We compare the performance of Algorithm 4 with that of Bennett’s algorithm for $N_t=1000$, where the results are shown by Figure 14. We can see that the success rates of both algorithms are $100\%$. Moreover, due to the pausing mechanism, Algorithm 4 takes more time to complete herding, but the control cost is greatly reduced.

5.4. Parameter Analysis

In this subsection, we will examine how different parameters will affect the performance of the proposed algorithm. Since Algorithm 4 has integrated all the considerations, it is used as the test algorithm. By comprehensive numerical simulation, among all the design parameters, it is found that the variations of the following four parameters would make a drastic change in the algorithm performance, namely, ${\overline{\theta }}^t$, $\upsilon$, $\vartheta$, and $\underline{r}$. In this subsection, $N_t=500$. The detailed analysis is as follows.

The algorithm performance with respect to the variations of ${\overline{\theta }}^t$ is shown by Figure 15. As ${\overline{\theta }}^t$ increases from $\frac{\pi }{8}$ to $\frac{17\pi }{15}$, both completion step and control cost decrease, where the biggest decrease happens when ${\overline{\theta }}^t$ changes from $\frac{3\pi }{5}$ to $\frac{11\pi }{15}$. ${\overline{\theta }}^t$ determines when the sheepdog could change moving direction. For large ${\overline{\theta }}^t$, the sheepdog is more likely to meet the turning condition, and hence the route of the sheepdog is shortened, leading to less completion step and control cost as shown by Figure 15. On the contrary, when it is smaller, herding will fail.

The algorithm performance with respect to the variations of $\upsilon$ is shown in Figure 16. For lower $\upsilon$, the sheepdog moves too slowly that the herding task may not be successfully completed. Even if herding is achieved, the completion step is quite large, though at a rather low control cost. When $\upsilon$ varies from $0.4$ to $0.6$, the completion step decreases initially and then becomes constant, while the control cost keeps increasing. Thus, if both the completion step and control cost is taken into consideration, the values for $\upsilon$ around $0.6$ might be a good choice.

The algorithm performance with respect to the variations of $\vartheta$ is shown in Figure 17. For small detouring angle $\vartheta$, the sheepdog tends to approach the critical sheep directly. As a result, it may get too close to the sheep herd, which may cause the sheep herd to oscillate along the directions of $D^{cdl}\left(k\right)$ and $D^{cdr}\left(k\right)$, thus increasing the completion step. As $\vartheta$ increases, the oscillation is alleviated, and the control cost is also reduced. However, when $\vartheta$ further increases, the completion step almost remains the same, with a slight trend of increasing, but the control cost is increasing drastically. The reason is that for an overly large detouring angle $\vartheta$, the sheepdog keeps a rather long distance from the sheep herd. Therefore, it probably takes more control costs to achieve herding.

The algorithm performance with respect to the variations of $\underline{r}$ is shown in Figure 18. $\underline{r}$ determines when the sheepdog should pause. For large $\underline{r}$, the sheepdog pauses even if it is still quite far from the sheep herd, which greatly reduces the efficiency of the herding algorithm and also leads to a rather low success rate. If we reduce $\underline{r}$, then the sheepdog pause less, which, on the one hand, makes herding faster, and on the other hand, increases the control cost.

5.5. Size of the Sheep Herd

In this subsection, we will examine the performance of Algorithm 4 for different size of the sheep herd, namely, $N=10,24,50$. Suppose the initial positions of the sheep are within the range of ${[-6\sim 20,-6\sim 20]}^T$. Let $N_t=1000$ and ${\rho }_d=14$. The algorithm performance is shown in Figure 19. When the size of the sheep herd is small, i.e., $N=10$, the scattering of the sheep herd and the weak attraction between sheep will increase the difficulty for the sheepdog to gather the sheep herd. That explains why it takes longer time to complete herding. When the size of the sheep herd is big, i.e., $N=50$, the detouring distance of the sheepdog becomes longer, and the sheep at the forefront of the sheep herd will be less and less controllable by the sheepdog. In general, if the size of the sheep herd keeps increasing, it will take more and more time and control costs to herd.

6. Conclusions and Discussion

In this paper, for the social force model-based sheep herd, four behavior-based herding algorithms have been developed to solve the herding problem. By defining four critical sheep, a basic behavior-based herding algorithm is first proposed. Then, the dynamic far-end mechanism is synthesized to avoid the undesired circling behavior in the final stage of herding, and the pausing mechanism is conceived to reduce the control cost. It is shown by experiments that the circling issue would be more severe when the destination area becomes smaller. The simulation data have validated the effectiveness of the proposed herding algorithms. By Table 1, it can be further calculated that the integrated Algorithm 4 improves 23.7%, 0, 8.1% in terms of success rate; reduces −0.06%, −10.02%, 7.02% in terms of completion step and reduces 23.51%, 17.16%, 7.92% in terms of control cost compared to Algorithm 1, Algorithm 2, and Algorithm 3, respectively. Moreover, a thorough analysis has also been performed, discussing how the design parameters ${\overline{\theta }}^t$, $\upsilon$, $\vartheta$, and $\underline{r}$ will affect the performance. Finally, we check the performance of Algorithm 4 for $N=10,24$ and 50. The results indicate that it will take more and more time and control costs to herd as the size of the sheep herd increases. Nevertheless, this work might still be improved from the following aspects. First, if the size of the sheep herd is greatly enlarged, then a single sheepdog would obviously be insufficient to deal with the herding problem. In such cases, it is necessary to consider the cooperation of multiple sheepdogs. Second, we have not considered obstacles or boundaries in this paper, which are common in practice. Besides completing the herding task in such a complex environment, it would be more interesting to think about how to make use of the specific conditions of the environment to help to herd.