Trajectory Planning and Tracking Control of a Differential-Drive Mobile Robot in a Picture Drawing Application

Abstract

This paper proposes a method for trajectory planning and control of a mobile robot for application in picture drawing from images. The robot is an accurate differential drive mobile robot platform controlled by a field-programmable-gate-array (FPGA) controller. By not locating the tip of the pen at the middle between two wheels, we are able to construct an omnidirectional mobile platform, thus implementing a simple and effective trajectory control method. The reference trajectories are generated based on line simplification and B-spline approximation of digitized input curves obtained from Canny’s edge-detection algorithm on a gray image. Experimental results for image picture drawing show the advantage of this proposed method.

1. Introduction

Wheeled mobile robots are increasingly present in industrial, service, and educational robotics, particularly when autonomous motion capabilities are required on a smooth ground plane. For instance, [1] used image processing techniques to detect, follow, and semi-automatically repaint a half-faded lane mark on an asphalt road. Employing a mobile robot platform in drawing robot applications has the advantages of small size, large work space, portability, and low cost. A drawing robot is a robot that makes use of image processing techniques in input an image and draws the same image picture. A transitional x-y plotter has limits in terms of work space and becomes more expensive for large work sizes. Most portrait drawing robots/humanoid robots make use of multi-degrees of freedom for robot arms [2,3,4,5]. However, it may be too costly to use a robot arm in drawing a picture or portrait.

The basic motion tasks for a mobile robot in an obstacle-free work space are point-to-point motion and trajectory following. Trajectory tracking control is particularly useful in art drawing applications, in which a representative physical point (pen tip, for instance) on the mobile robot must follow a trajectory in the Cartesian space starting from an arbitrary initial posture. The trajectory tracking problem has been solved by various approaches, such as Lyapunov-based non-linear feedback control, dynamic feedback linearization, feed-forward plus feedback control, and others [6,7,8]. In this work, we propose a simple and effective trajectory control method for a picture drawing robot that is based on inverse kinematics and proportional feedback control in discrete time.

Omnidirectional motion capability is another important factor in fine and precise motion applications. Designing a mobile robot with isotropy kinematic characteristics requires three active wheels with a special wheel mechanism design [9]. Kinematic isotropy means the Jacobian matrix is isotropic for all robot poses, which is one of the key factors in robotic mechanism design [10,11]. Kinematic isotropy makes the best use of control degrees of freedom in Cartesian motion.

The standard construction of a mobile drawing robot is based on a classical differential drive chassis with the single castor in the back of the robot [12,13]. Some robots are driven by two stepper motors, in which the motor control is an open-loop control. Thus, the robot’s position and orientation are computed based on the input step command for stepper motors [13]. For most differential-drive mobile drawing robots, the draw pen is installed in the middle of the wheels; therefore, this configuration is not omnidirectional in kinematics, and a sharp curve is difficult and consumes more power to draw. The motivation of this work is, thus, to explore the omnidirectional motion capability for a two-control degrees of freedom differential-drive mobile robot platform for application in fine art drawing. When the draw pen does not stall at the middle of the wheels, a differential-drive mobile robot platform may be omnidirectional and even isotropic in the Cartesian space by not considering heading control. Moreover, the mobile platform has less power consumption for a given reference following path and possibly can be used for sharp turn motions.

For the first step toward an autonomous art drawing system, the authors propose an integral system consisting of a remote main controller and a drawing mobile robot. The remote main controller can be either a personal computer (PC) or a smartphone, which automatically generates line drawing picture data from a camera in real time. The picture curves are generated in the sequence of image edge detection, searching and sorting connected paths, line simplification of image space curves, and line curve smoothing by cubic B-spline approximation. The drawing mobile robot is equipped with a pen and wireless Bluetooth connection; thus, it is able to draw pictures on a paper on-line. The robot is an accurate mobile two-wheel differential driven robot controlled by a field programmable gate array (FPGA) controller. The robot is driven by two dc servo motors. The motor control is under closed-loop position proportional-integral-derivative (PID) control. The contributions of this work are as follows:

(1)

omnidirectional motion for a differential drive mobile robot platform;

(2)

autonomous art drawing system can be built to keep both mobile robot platform and control algorithm as simple as possible; and

(3)

system integration of a mobile pen drawing platform by image edge detection, line simplification and smoothing, trajectory following control, and FPGA controller.

The rest of the paper is organized as follows: Section 2 presents an omnidirectional model of a differential-drive mobile robot. Section 3 shows trajectory tracking control methods and stability analysis. Section 4 studies trajectory planning for drawing a connected digitized curve through line simplification and line smoothing. Section 5 illustrates searching and sorting picture curves of a gray image. Section 6 provides both simulation and experimental results obtained from a test-bed mobile robot. Section 7 concludes the paper.

2. Kinematic Model of a Differential-Drive Mobile Platform

This study considers a simple mobile robot platform with two independent driving wheels and one castor free wheel. A draw pen is installed along a perpendicular line between the middle of the two wheels to depict an image picture. Let $({\dot{\theta }}_1,{\dot{\theta }}_2)$ be driving the wheel angular velocity, $r$ is the radius of the driving wheel, $L$ is half of the distance between the two wheels, and D is the perpendicular distance from the pen position to the middle of the two wheels, as shown in Figure 1. We define $(x_b,y_b)$ as the position of the middle of the two wheels and $(x,y)$ as the tip position of the pen in the world frame {W}. The origin of the robot base frame {B} is assigned to the middle of two wheels, and $\theta$ is the mobile robot yaw angle and is the orientation angle from $(x,y)$ pointing to $(x_b,y_b)$ relative to the x-axis of the world reference frame {W}, as shown in Figure 1.

Let $v$ and $\omega$ be the instantaneous linear velocity of the origin and angular velocity of the robot base frame, respectively. The velocity kinematic equation of the mobile robot platform is represented by

$$\left[\begin{array}{c}{\dot{x}}_b\\ {\dot{y}}_b\\ \dot{\theta }\end{array}\right]=\left[\begin{array}{cc}\cos \theta & 0\\ \sin \theta & 0\\ 0& 1\end{array}\right]\left[\begin{array}{c}v\\ \omega \end{array}\right]$$

(1)

and

$$\left[\begin{array}{c}v\\ \omega \end{array}\right]=\left[\begin{array}{cc}\frac{r}{2}& \frac{r}{2}\\ \frac{r}{2L}& \frac{-r}{2L}\end{array}\right]\left[\begin{array}{c}{\dot{\theta }}_1\\ {\dot{\theta }}_2\end{array}\right]$$

(2)

Common to all types of mobile robots with two control degrees of freedom, this kinematic is not-omnidirectional. Suppose that the Cartesian position of the pen $(x,y)$ is the representative point of the mobile robot platform,

$$\left[\begin{array}{c}x\\ y\end{array}\right]=\left[\begin{array}{c}{x}_b\\ y_b\end{array}\right]-D\left[\begin{array}{c}\cos \theta \\ \sin \theta \end{array}\right]$$

(3)

and hence,

$$\left[\begin{array}{c}\dot{x}\\ \dot{y}\end{array}\right]=\left[\begin{array}{cc}\cos \theta & D\sin \theta \\ \sin \theta & -D\cos \theta \end{array}\right]\left[\begin{array}{c}v\\ \omega \end{array}\right]$$

(4)

The mobile robot platform kinematics without heading control is now defined by

$$\left[\begin{array}{c}\dot{x}\\ \dot{y}\end{array}\right]=J(\theta )\left[\begin{array}{c}{\dot{\theta }}_1\\ {\dot{\theta }}_2\end{array}\right]$$

(5)

where the Jacobian matrix is

$$J(\theta )=\frac{r}{2}\left[\begin{array}{cc}\cos \theta +\rho \sin \theta & \cos \theta -\rho \sin \theta \\ \sin \theta -\rho \cos \theta & \sin \theta +\rho \cos \theta \end{array}\right]$$

(6)

where ratio $\rho =D/L$. The determinant of the Jacobian matrix, $\det (J)=\rho r^2/2$, is a constant. Therefore, by not locating the pen at the middle of the wheels, i.e., $\rho >0$, the Jacobian matrix is well-conditioned and the inverse Jacobian matrix is

$$J^{-1}(\theta )=\frac{1}{r}\left[\begin{array}{cc}\cos \theta +{\rho }^{-1}\sin \theta & \sin \theta -{\rho }^{-1}\cos \theta \\ \cos \theta -{\rho }^{-1}\sin \theta & \sin \theta +{\rho }^{-1}\cos \theta \end{array}\right]$$

(7)

It is clear that the control degree in the yaw angle is now transformed to the control degree in the lateral linear motion. The bigger the $\rho$ is, the larger the mobility is in the lateral motion. Without consideration of the orientation angle, the mobile robot platform becomes an omnidirectional platform in the Cartesian space. An interesting example is a circular shape robot with a pen installed at the center position.

The singular value decomposition of the Jacobian matrix $J(\theta )$ is

$$J=U\Sigma V^T=\left[\begin{array}{cc}{u}_1& u_2\end{array}\right]\left[\begin{array}{cc}{\sigma }_1& 0\\ 0& {\sigma }_2\end{array}\right]\left[\begin{array}{c}{v}_1^T\\ v_2^T\end{array}\right]$$

where

$${\sigma }_1=\frac{r}{\sqrt{2}},v_1=\frac{1}{\sqrt{2}}\left[\begin{array}{c}1\\ 1\end{array}\right],u_1=\left[\begin{array}{c}\cos \theta \\ \sin \theta \end{array}\right]$$

and

$${\sigma }_2=\rho \frac{r}{\sqrt{2}},v_2=\frac{1}{\sqrt{2}}\left[\begin{array}{c}-1\\ 1\end{array}\right],u_2=\left[\begin{array}{c}-\sin \theta \\ \cos \theta \end{array}\right]$$

The maximum and minimum singular values of the Jacobian matrix $J(\theta )$ are ${\sigma }_{\max }=\{\begin{array}{cc}1& 0<\rho \le 1\\ \rho & \rho >1\end{array}$ and ${\sigma }_{\min }=\{\begin{array}{cc}\rho & 0<\rho \le 1\\ 1& \rho >1\end{array}$, respectively; and the condition number $cond(J)=\frac{{\sigma }_{\max }}{{\sigma }_{\min }}=\{\begin{array}{cc}{\rho }^{-1}& 0<\rho \le 1\\ \rho & \rho >1\end{array}$. When $\rho =1$, the condition number equals one, and the columns of the Jacobian matrix $J(\theta )$ are orthogonal and of equal magnitude for all robot poses. Thus, the mobile robot platform and pen system exhibit kinematic isotropy. Each wheel causes the pen’s motion to be of equal amount and orthogonal to that caused by the other wheel. In this case, it makes the best use of the degrees of freedom of the two wheels in the Cartesian motion.

3. Mobile Platform Trajectory Tracking Control Methods

3.1. Trajectory Tracking Control and Stability

As stated above, the tip position of the pen $(x(t),y(t))$ is the representative point of the mobile robot platform, which must follow a referenced Cartesian trajectory $(x_r(t),y_r(t))$, $0\le t\le T$, in the presence of initial error. Since the Jacobian matrix $J(\theta )$ is well-conditioned for all poses, a simple inverse kinematics-based resolved rate plus proportional control law is applied to meet the requirement for the pen trajectory control problem.

We define $e_1(t)=x_r(t)-x(t)$ and $e_2(t)=y_r(t)-y(t)$ as the trajectory tracking errors. The control rule of resolved rate plus P control is as follows

$$\left[\begin{array}{c}{\dot{\theta }}_1(t)\\ {\dot{\theta }}_2(t)\end{array}\right]=J^{-1}(\widehat{\theta })\left[\begin{array}{c}{\dot{x}}_r(t)+k_1{e}_1(t)\\ {\dot{y}}_r(t)+k_2{e}_2(t)\end{array}\right]$$

(8)

where $\widehat{\theta }(t)$ is the estimated orientation angle $\theta (t)$, and $k_1$ and $k_2$ are positive gains, $k_1,k_2>0$. Let $\widehat{\theta }=\theta +\delta$, and then $J(\theta )J^{-1}(\widehat{\theta })=\left[\begin{array}{cc}\cos \delta & \sin \delta \\ -\sin \delta & \cos \delta \end{array}\right]$, and the error dynamics become a linear time-varying system as follows

$$\left[\begin{array}{c}{\dot{e}}_1(t)\\ {\dot{e}}_2(t)\end{array}\right]=\left[\begin{array}{cc}-k_1\cos \delta & -k_2\sin \delta \\ k_1\sin \delta & -k_2\cos \delta \end{array}\right]\left[\begin{array}{c}{e}_1(t)\\ e_2(t)\end{array}\right]+\left[\begin{array}{cc}1-\cos \delta & -\sin \delta \\ \sin \delta & 1-\cos \delta \end{array}\right]\left[\begin{array}{c}{\dot{x}}_r(t)\\ {\dot{y}}_r(t)\end{array}\right]$$

(9)

The stability and properties of the above closed-loop system are discussed in the following two cases.

Case 1: Global exponential stability

In the case of ${\dot{x}}_r(t)={\dot{y}}_r(t)=0$ (point-to-point motion without heading control), the closed system has a globally exponentially stable equilibrium at $(e_1,e_2)=0$. It is based on the use of the Lyapunov function $V=(e_1^2+e_2^2)/2>0$, $(e_1,e_2)\ne 0$, whose time derivative, $\dot{V}=e_1{\dot{e}}_1+e_2{\dot{e}}_2=-\cos \delta (k_1{e}_1^2+k_2{e}_2^2)\le -\alpha V$, $\alpha =2\cos {\delta }_{\max }\min \{k_1,k_2\}$, provides that $\cos \delta \ge \cos {\delta }_{\max }>0$ and $k_1,k_2>0$.

Case 2: Input-to-state stability (bounded-input bounded-state stability)

Because of the bounded velocity capability of motors, the trajectory tracking velocity is limited. Assume that the derivatives of reference trajectory are limited, such that $\left|{\dot{x}}_r(t)\right|\le m$ and $\left|{\dot{y}}_r(t)\right|\le m$, where m is a constant upper boundary. Lemma 4.6 of Reference [14] states that an unforced system whose trivial solution is globally exponentially stable is input-to-state stable (ISS) if submitted to a bounded input (or perturbation). Therefore, the closed system in Equation 9 is bounded-input bounded-state (BIBS) stable. Furthermore, the bounds of trajectory tracking errors are approximately $\left|e_1\right|\le k_1^{-1}\delta m$ and $\left|e_2\right|\le k_2^{-1}\delta m$, respectively. The trajectory following errors $\left|e_1\right|$ and $\left|e_2\right|$ are directly proportional to trajectory speed m, measure error $\delta$, and gains $k_1^{-1}$ and $k_2^{-1}$.

3.2. Discrete-Time Trajectory Following Control Method

It is desirable to implement the above continuous-time trajectory control method in discrete-time form. Let $T_c$ be the trajectory control update time, and $(x_r(k),y_r(k))$ is the referenced pen trajectory at discrete time k; therefore, it is likely to generate incremental motion commands $\mathrm{\Delta }\Omega (k)$ and $\mathrm{\Delta }S(k)$, such that

$$\left[\begin{array}{c}{x}_b(k)\\ y_b(k)\end{array}\right]-D\left[\begin{array}{c}\cos \theta (k)\\ \sin \theta (k)\end{array}\right]=\left[\begin{array}{c}{x}_r(k)\\ y_r(k)\end{array}\right]$$

where

$$\left[\begin{array}{c}{x}_b(k)\\ y_b(k)\end{array}\right]=\left[\begin{array}{c}x(k-1)\\ y(k-1)\end{array}\right]+\left[\begin{array}{c}\cos \theta (k-1)\\ \sin \theta (k-1)\end{array}\right](D+\mathrm{\Delta }S(k))$$

and

$$\theta (k)=\theta (k-1)+\mathrm{\Delta }\Omega (k)$$

The above inverse kinematic equation can be rewritten as

$$\left[\begin{array}{c}\cos \theta (k-1)\\ \sin \theta (k-1)\end{array}\right](D+\mathrm{\Delta }S(k))-D\left[\begin{array}{c}\cos (\theta (k-1)+\mathrm{\Delta }\Omega (k))\\ \sin (\theta (k-1)+\mathrm{\Delta }\Omega (k))\end{array}\right]=\left[\begin{array}{c}\mathrm{\Delta }x\\ \mathrm{\Delta }y\end{array}\right]$$

(10)

where

$$\left[\begin{array}{c}\mathrm{\Delta }x\\ \mathrm{\Delta }y\end{array}\right]=\left[\begin{array}{c}{x}_r(k)\\ y_r(k)\end{array}\right]-\left[\begin{array}{c}x(k-1)\\ y(k-1)\end{array}\right]$$

Here, it is assumed that $\left|\mathrm{\Delta }x\right|\le d_{\max }$ and $\left|\mathrm{\Delta }y\right|\le d_{\max }$, in which $d_{\max }$ is an upper boundary on how far to move the pen in a single update time. If $(x_r(k),y_r(k))$ is too distant from $(x(k-1),y(k-1))$, then it needs to move the target position in closer to the current pen location by inserting one or more intermediate points. Control terms $\mathrm{\Delta }\Omega (k)$ and $\mathrm{\Delta }S(k)$ can be obtained as follows

$$\mathrm{\Delta }\Omega (k)={\sin }^{-1}\left(\frac{\mathrm{\Delta }x\sin \theta (k-1)-\mathrm{\Delta }y\cos \theta (k-1)}{D}\right)$$

(11)

and

$$\mathrm{\Delta }S(k)=D\cos \mathrm{\Delta }\Omega (k)-D+\mathrm{\Delta }x\cos \theta (k-1)+\mathrm{\Delta }y\sin \theta (k-1)$$

(12)

Motor incremental position commands are then given by

$$\left[\begin{array}{c}\mathrm{\Delta }{\Theta }_1(k)\\ \mathrm{\Delta }{\Theta }_2(k)\end{array}\right]=\frac{1}{r}\left[\begin{array}{c}\mathrm{\Delta }S(k)+L\mathrm{\Delta }\Omega (k)\\ \mathrm{\Delta }S(k)-L\mathrm{\Delta }\Omega (k)\end{array}\right]$$

(13)

It is also required that $\left|\mathrm{\Delta }{\Theta }_1(k)\right|\le \mathrm{\Delta }{\Theta }_{\max }$ and $\left|\mathrm{\Delta }{\Theta }_2(k)\right|\le \mathrm{\Delta }{\Theta }_{\max }$, where $\mathrm{\Delta }{\Theta }_{\max }$ is the maximum incremental position command of each driving motor. Here, $\mathrm{\Delta }{\Theta }_1(k)$ and $\mathrm{\Delta }{\Theta }_2(k)$ are summed up and then used as absolute reference position commands of wheel motors, which are then sent to independent PID position controllers to close the dc motor servo control loops. Figure 2 shows the system block diagram of the proposed pen trajectory following control method. In this study, the trajectory control update time is $T_c$ = 20 milliseconds, and the position servo control sampling time is $T_s$ = 1.0 milliseconds.

The above trajectory control loop requires information from the mobile robot’s position and yaw angle. The computation of the mobile robot’s position and yaw angle is based on odometric prediction from motor encoders. The odometry update time is also chosen as $T_c$ = 20 milliseconds. The mobile robot’s incremental motion changes are calculated from motor encoders reading and

$$\left[\begin{array}{c}\mathrm{\Delta }s(k)\\ \mathrm{\Delta }\theta (k)\end{array}\right]=r\left[\begin{array}{c}\frac{\mathrm{\Delta }{\theta }_1(k)+\mathrm{\Delta }{\theta }_2(k)}{2}\\ \frac{\mathrm{\Delta }{\theta }_1(k)-\mathrm{\Delta }{\theta }_2(k)}{2L}\end{array}\right]$$

(14)

where

$$\left[\begin{array}{c}\mathrm{\Delta }{\theta }_1(k)\\ \mathrm{\Delta }{\theta }_2(k)\end{array}\right]=\left[\begin{array}{c}{\theta }_1(k)-{\theta }_1(k-1)\\ {\theta }_2(k)-{\theta }_2(k-1)\end{array}\right]$$

The mobile robot base position $(x_b(k),y_b(k))$ and orientation angle are updated according to

$$\left[\begin{array}{c}{x}_b(k)\\ y_b(k)\end{array}\right]=\left[\begin{array}{c}{x}_b(k-1)\\ y_b(k-1)\end{array}\right]+\left[\begin{array}{c}\cos \theta (k-1)\\ \sin \theta (k-1)\end{array}\right]\mathrm{\Delta }s(k)$$

(15)

and

$$\theta (k)=\theta (k-1)+\mathrm{\Delta }\theta (k)$$

(16)

Robot localization using the above odometry, commonly referred to as dead reckoning, is usually accurate enough in the absence of wheel slippage and backlash.

4. Line Simplification and B-Spline Smoothing

This work represents each desired picture curve by a B-spline approximation of a digitized two-dimensional image curve ${\left\{x_t\right\}}_{t=1}^N$, $N\ge 2$, $x_t\in R^2$. Here, the B-spline curves of degree 1 and degree 3 are used most often. The problem statement is depicted as follows.

Given a digitized image curve of N connected pixels ${\left\{x_t\right\}}_{t=1}^N$, $N\ge 2$, in order to find a two-dimensional B-spline curve of degree d,

$$y_t={\displaystyle \sum _{i=1}^{n+d-1}{N}_{i,d}(t)p_i},1\le t\le N$$

(17)

for (n + d − 1) control points ${\left\{p_i\right\}}_{i=1}^{n+d-1}$, $2\le n\le N$, it is best to approximate the sequence of input data points ${\{(t,x_t)\}}_{t=1}^N$ in a least-squares sense. The functions of $N_{i,d}(t)$ are the B-spline basis functions, which are defined recursively as:

$$N_{i,d}(t)=\frac{t-t_i}{t_{i+d}-t_i}{N}_{i,d-1}(t)+\frac{t_{i+d+1}-t}{t_{i+d+1}-t_{i+1}}{N}_{i+1,d-1}(t)$$

(18)

where $N_{i,0}(t)=\{\begin{array}{cc}1& t_i\le t<t_{i+1}\\ 0& \mathrm{otherwise}\end{array}$, and $\frac{0}{0}=0$ is defined.

The B-spline curve ${\left\{y_t\right\}}_{t=1}^N$ is then the desired pen drawing reference trajectory $(x_r(t),y_r(t))$ at discrete-time t in the unit of trajectory control sampling time $T_c$.

4.1. Line Simplification and Knot Section

The above B-spline approximation problem involves knot selection and control point selection. To simplify the solution search, knot selection and control point selection are found separately in two phases. Initially, the start and end knots are set to $t_1=1$ and $t_n=N$. First, the line simplification technique is applied to select knots ${\left\{t_i\right\}}_{i=1}^n$, $2\le n\le N$, so that $1\le i<j\le n$ and $1\le t_i<t_j\le N$. The least-squares error method is then followed to search out new control points ${\left\{p_i\right\}}_{i=1}^{n+d-1}$ for the purpose of line smoothing.

Line simplification is an algorithm for reducing the number of points in a curve that is approximated by a series of points within an error of epsilon. There are many line simplification algorithms—among them, the Douglas–Peucker algorithm is the most famous line simplification algorithm [15]. The starting curve is an ordered set of points or lines and the distance dimension epsilon. The algorithm recursively divides the line. It is initially given all the points between the first and last points. It automatically marks the first and last points to be kept. It then finds the point that is furthest from the line segment with the first and last points as end points; this point is obviously furthest on the curve from the approximating line segment between the end points. If the point is closer than epsilon to the line segment, then any points not currently marked to be kept can be discarded without the simplified curve being worse than epsilon. After the line simplification algorithm, the selected knots ${\left\{t_i\right\}}_{i=1}^n$, $2\le n\le N$, are built from the index of line simplification points.

A linear spline approximation of the input image curve can be obtained based on the selected knots ${\left\{t_i\right\}}_{i=1}^n$ and control points ${\left\{x_{t_i}\right\}}_{i=1}^n$ selected from those in the input dataset ${\left\{x_t\right\}}_{t=1}^N$. The desired linear spline trajectory ${\left\{y_t\right\}}_{t=1}^N$ is then:

$$y_t=\frac{t_{i+1}-t}{t_{i+1}-t_i}{x}_{t_i}+\frac{t-t_i}{t_{i+1}-t_i}{x}_{t_{i+1}},t_i\le t\le t_{i+1},1\le i\le n$$

(19)

4.2. B-Spline Control Point Selection

This study develops the B-spline with degree d approximation of input data ${\left\{x_t\right\}}_{t=1}^N$, $N\ge 2$, with distinct knots ${\left\{t_i\right\}}_{i=1}^n$, $2\le n\le N$, obtained from line simplification for line smoothing. First, we define open integer knots ${\left\{{\tau }_i\right\}}_{i=1}^{n+2d}$ here so that ${\left\{{\tau }_i\right\}}_{i=1}^{n+2d}=\{t_1,\cdots ,t_1,t_2,\cdots ,t_{n-1},t_n,\cdots ,t_n\}$, where knots $t_1$ and $t_n$ repeat (d + 1) times, in order to guarantee that the start and end points of the B-spline curve are the first and last input data $x_1$ and $x_N$, respectively. The B-spline curves of degree 1 and degree 3 are considered in this study.

(1) B-spline of degree 1 (d = 1)

The control points ${\left\{p_i\right\}}_{i=1}^{n+d-1}$ are arranged here such that $p_1=x_1$ and $p_n=x_N$ to guarantee that the start and end points of the picture curve are unchanged, and (n − 2) control points ${\left\{p_i\right\}}_{i=2}^{n-1}$ are left to be determined.

(2) B-spline of degree 3 (d = 3)

The control points ${\left\{p_i\right\}}_{i=1}^{n+d-1}$ are arranged here such that $p_1=p_2=x_1$ and $p_{n+1}=p_{n+2}=x_N$ to guarantee zero end speeds at the start point $x_1$ and end point $x_N$, and (n − 2) control points ${\left\{p_i\right\}}_{i=3}^n$ are left to be determined.

Control points are selected by least-squares fitting of the data with a B-spline curve:

$$E=\frac{1}{2}{\displaystyle \sum _{t=1}^N{\Vert y_t-x_t\Vert }^2}=\frac{1}{2}{\displaystyle \sum _{t=1}^N{\Vert {\displaystyle \sum _{i=1}^{n+d-1}{N}_{i,d}(t)p_i}-x_t\Vert }^2}$$

(20)

For B-spline of degree 1 (d = 1), the fitting equations are:

$${\displaystyle \sum _{i=2}^{n-1}{N}_{i,d}(t)p_i}=b_t=x_t-N_{1,d}(t)x_1-N_{n,d}(t)x_N,2\le t\le N-1$$

(21)

Let $X={\left[\begin{array}{cccc}{p}_2& p_3& \cdots & p_{n-1}\end{array}\right]}^T$ and $B={\left[\begin{array}{ccccc}{b}_2& b_3& \cdots & b_{N-2}& b_{N-1}\end{array}\right]}^T$, and then we have an over-determined matrix equation:

$$A_{(N-2)\times (n-2)}{X}_{(n-2)\times 2}=B_{(N-2)\times 2}$$

(22)

where $A_{(N-2)\times (n-2)}=\left[N_{i,d}(t)\right]$, $2\le t\le N-1$, and $2\le i\le n-1$.

For B-spline of degree 3 (d = 3), the fitting equations are:

$${\displaystyle \sum _{i=3}^n{N}_{i,d}(t)p_i}=b_t=x_t-(N_{1,d}(t)+N_{2,d}(t))x_1-(N_{n+1,d}(t)+N_{n+2,d}(t))x_N,2\le t\le N-1$$

(23)

Let $X={\left[\begin{array}{cccc}{p}_3& p_4& \cdots & p_n\end{array}\right]}^T$ and $B={\left[\begin{array}{ccccc}{b}_2& b_3& \cdots & b_{N-2}& b_{N-1}\end{array}\right]}^T$, and then we also have the same dimensions for an over-determined matrix equation:

$$A_{(N-2)\times (n-2)}{X}_{(n-2)\times 2}=B_{(N-2)\times 2}$$

where $A_{(N-2)\times (n-2)}=\left[N_{i,d}(t)\right]$, $2\le t\le N-1$, and $3\le i\le n$.

The least-squares error solution can then be solved from the following normal equation:

$$SX=A^{T}AX=A^{T}B$$

(24)

or obtained from the pseudo-inverse solution:

$$X={(A^{T}A)}^{-1}{A}^{T}B$$

(25)

Since the normal equation matrix $S=A^{T}A$ is a symmetric and banded matrix, it is, therefore, desirable to search for control points through basic row eliminations and then follow with backward substitution. Once control points ${\left\{p_i\right\}}_{i=1}^{n+d-1}$ are found, the desired B-spline reference trajectory ${\left\{y_t\right\}}_{t=1}^N$ is generated through DeBoor’s B-spline recursive formulation [16].

5. Picture Curves Generation

We now describe the process from the input of the image picture to the desired picture curves’ generation. Picture curves are generated after Canny’s edge detection algorithm is performed on the input picture gray image [17]. The generation of picture draw curves involves searching and sorting picture curves in two stages. The strategy for searching out picture curves runs in three phases as stated below:

• Phase 1: remove salt points and branch points;
• Phase 2: search connected paths; and
• Phase 3: search loop paths.

The picture point decision is based on the local information of its adjacent pixels in a 3-by-3 window, $\left[\begin{array}{ccc}\mathrm{P}8& \mathrm{P}1& \mathrm{P}5\\ \mathrm{P}4& \mathrm{pixel}& \mathrm{P}2\\ \mathrm{P}7& \mathrm{P}3& \mathrm{P}6\end{array}\right]$, defined here. Let the terms 4-connect and 8-connect of a pixel point be the numbers of its 4-connected pixel equal to 1 and its 8-connected pixel equal to 1, respectively.

In phase 1, salt point and branch point are classified as below:

• salt point: 8-connect = 0; and
• branch point: 4-connect > 2.

In phase 2, start point, mid-point, and end point of a connected path are classified as below:

• start point: 4-connect = 1 or (4-connect = 0 and 8-connect = 1);
• mid-point: in the order of (P1 = 1) or (P2 = 1) or (P3 = 1) or (P4 = 1) or (P5 =1) or (P6 = 1) or (P7 = 1) or (P8 = 1);
• end point: 8-connect = 0.

In phase 3, salt point and point in a loop path are classified as below:

• salt point: 8-connect = 0;
• point in a loop path: 8-connect ≥ 2.

For the sorting of final picture curves, a simple greedy method of the nearest neighbor strategy is applied to sort picture curves in order to send out to the robot drawer. The next curve to be drawn is the one where one of its end points is closest to the end position of the drawing curve among the rest of the curves. Figure 3 shows picture curve search results; the input gray image has a size of 213 × 315 bytes. There are 5878 edge pixels after edge detection, and there are 136 connected curves in curve sorting. After line simplification, there are 2702 control points left.

6. Simulation and Experimental Results

The experimental autonomous robot image figure drawing system consists of a PC or a smartphone as a main controller and a differential-drive mobile robot platform as shown in Figure 4. The radius of the driving wheel is r = 56.25 mm, and the distance of two driving wheels is 2 L, where L = 12.25 mm. The wheel is actuated by a dc motor (motor maximum no-load speed 4290 rpm) with a 15:1 gear reducer, and the motor encoder has a resolution of 30,000 ppc. The proposed pen trajectory following control system is built on an Altera DE0-nano FPGA development board running at a system clock rate of 50 MHz. All control and series communication modules are implemented using Verilog hardware description language (HDL) and synthesized by an Altera Quartus II EDA tool. The image processing and connected curve generation program are programmed in Visual C++ for PC and in JAVA for smartphone. The mobile robot receives real-time picture curves data, which are generated from a PC or a smartphone through Bluetooth rs232 at a Baud-rate of 115,200 Hz.

The first example considers a circular path to verify the effect of $\rho =D/L$. The desired reference path trajectory is represented by $x_r(t)=R\cos (\omega t)$ and $y_r(t)=R\cos (\omega t)$, $0\le t\le t_f=2\pi /\omega$, where R = 50 mm, and the total circular length is $S={\displaystyle {\int }_0^{t_f}\sqrt{{\dot{x}}_r^2+{\dot{y}}_r^2}dt}$ = 100p mm. The control law in Equation (8) with proportional gains $k_1=k_2=20$ is used. Figure 5 shows the total displacement $\Theta ={\displaystyle {\int }_0^{t_f}\sqrt{{\dot{\theta }}_1^2+{\dot{\theta }}_2^2}dt}$ in the joint space with respect to arbitrary initial yaw angle ${\theta }_0$ for several different pen location distances D to follow the desired circular path trajectory.

The second example considers a square path of side length 100 mm, and the total length is $S={\displaystyle {\int }_0^{t_f}\sqrt{{\dot{x}}_r^2+{\dot{y}}_r^2}dt}$ = 400 mm. Figure 6 shows the total displacement $\Theta ={\displaystyle {\int }_0^{t_f}\sqrt{{\dot{\theta }}_1^2+{\dot{\theta }}_2^2}dt}$ in the joint space with respect to initial angle yaw ${\theta }_0$ for several different pen location distances D to follow the desired square path trajectory. As expected, the joint space displacement $\Theta$ decreases as the distance D increases. When $\rho =1$ (i.e., D = L), the Jacobian matrix $J$ is isotropic for all mobile robot poses; and hence, $\Theta$ is independent on the mobile robot initial yaw angle ${\theta }_0$ and $\Theta =\sqrt{2}S/r$.

In the following trajectory control experiments, a drawing pen is located at a distance of D = 50 mm. The discrete-time trajectory following control law in Equations (11) and (12) is applied with trajectory and feedback update time $T_s$ = 0.01 s. Figure 7 and Figure 8 show the experimental mobile robot’s drawn results of a circular path (radius 50 mm) and a square path (side length 100 mm), respectively. For both curves, the trajectory following error is within 3.0 mm without initial position error. It is shown that the pen tip trajectory has small trajectory tracking errors in sharp turn corners in Figure 8.

Figure 9 shows the experimental drawn results of Taiwan maps. The origin data has 1105 discrete data points, and there are 42 control points in its line simplification curve, and 44 control points in the cubic B-spline smoothing curve. Figure 10 shows tracking errors of drawn Taiwan maps in Figure 9. The drawing size is 160 × 80 mm², tracking errors are within 1.0 mm, and drawing time is 10 s. Since the step size is small, there are few differences in Figure 10.

Figure 11 shows the drawn results of a building image in Figure 3, in which there are 136 connected curves with 2702 control points in total after procedures of edge detection and line simplification. The drawing size is 150 × 120 mm² and drawing time is about 90 s.

Figure 12 shows the image drawn by the experimental mobile robot of a human portrait. The original image is 280 × 210 pixels and has 3951 pixels after edge detection, and it leaves 100 connected curves with 2267 control points after line simplification. The drawn picture size is 8.0 × 5.0 cm² and is done in about 60 s.

7. Conclusions

What is new in this present research is that by removing the control degree in yaw angle to a linear lateral motion of a differential-drive mobile robot, an omnidirectional platform can be built. Thus, we can design an autonomous art drawing platform in a closed-loop fashion through an inverse kinematics plus proportional control approach. Both simulation and experimental results show that the experimental system works in practice as well as in theory. The limitations of the current system are a lack of heading control in trajectory tracking control and slower motion of the mobile system. Further works can include integrating the vision system in the FPGA, refining the mobile robot odometry for longer traveling distance, and developing fast speed motion controller.