Development of a Worker-Following Robot System: Worker Position Estimation and Motion Control under Measurement Uncertainty

Abstract

This study proposes a sensor data process and motion control method for a mobile platform essential for transporting finished products or subsidiary materials in a smart factory. We developed a system that recognizes a fiducial marker printed on the work clothes worn by a worker, estimates the worker’s location, and follows the worker using the estimated location. To overcome the limitations of simulation-based research, gait data on a two-dimensional plane were derived through a human gait model and an error model according to the distance between the image sensor and the reference marker. The derived gait data were defined as the localization result for the worker, and a Kalman filter was used to robustly address the uncertainty of the localization result. A virtual spring-damper system was applied to follow the Mecanum wheel-based mobile platform workers. The performance of the proposed algorithm was demonstrated through comparative simulations with existing methods.

1. Introduction

With the breakthrough of robot automation technology, artificial intelligence technology, and communication technology, mobile robot-based services are being provided in various areas of society. In particular, as we enter the era of the Fourth Industrial Revolution, robots are being introduced at many production sites [1]. At a production site, the logistics regarding parts and final products play an important role in the production process. In the early stage of robot automation, AGVs (automated guided vehicles), which move only in designated areas, were used to manage logistics at the production site. However, AMRs (autonomous mobile robots) have recently replaced them in this role. These changes were made possible by the improvement of robot recognition technology based on artificial intelligence, the improvement of the self-localization functions of robots through high-speed communication, and the development of control technology. As a result, deploying AMRs at the production site makes it possible to respond flexibly to changes in the production environment and work process [2]. Among the various types of AMRs, mobile platforms that follow workers and help to transport heavy objects are becoming very important in industrial sites where workers and robots work together. Technically, it is possible to develop fully autonomous production lines. Still, industrial sites are currently maintaining the roles of workers to protect their occupational rights and to be able to respond to errors that may occur with autonomous systems. The important skills for a human-following robot are human recognition and tracking control. This technology is also considered one of the top priorities in the era of COVID-19 [3]. There have been numerous studies on technology for stably recognizing people. One category of human recognition is recognizing and following people using Lidar. Ref. [4] utilized a laser range finder (LRF) to recognize human legs. Chung et al. [5] proposed detection and tracking schemes for human legs using a single laser range finder. A marathon service robot was developed using an LRF [6]. Refs. [7,8] proposed an off-line trained full convolutional neural network to improve peoples’ leg tracking performance with a 2D Lidar. While Lidar has the advantage of enabling human recognition and tracking through a low-cost sensor and small memory, its limitation is that it is impossible to recognize a specific person using it; this makes it easy to lose the target. To complement these Lidar-based recognition technologies, image-based human recognition technologies have been developed in parallel. Bodor et al. [9] accomplished a tracking algorithm by developing a position and velocity path characteristic for each pedestrian using a Kalman filter. Ref. [10] proposed a localization method through a multi-camera and network system to recognize and track a person. Yoshimi et al. developed a vision-based target detection and avoided obstacles with ultrasonic sensors while following the person. Ref. [11] applied vision-based human tracking technology to a wheeled inverted pendulum robot. Refs. [12,13] developed a robust human tracking algorithm using a vision sensor. Image-based human recognition has the advantage of recognizing a specific person and making it possible to follow the target stably. However, it is important for a follower robot to be able to estimate the position of a subject to be followed. Still, it is difficult to estimate a worker’s position with high reliability using only an image sensor. Therefore, a recognition technology was developed through the fusion of a Lidar system with the ability to measure distance accurately and a camera with strong object recognition ability. Hoshino and Morioka [14] fused Lidar and Kinect to identify the target human and achieve human-following behavior. An omnidirectional camera and a Lidar fusion system were used to develop and improve previous studies [15]. Luo et al. [16] complemented the uncertainty of each sensor measure and enhanced the reliability of human position information by sensor fusion skills. Ref. [17] developed LRF, Vision, and ultrasound sensor-based human following robot that keeps the social distance between humans in an intuitive manner. Ref. [18] fused RGB-D vision, lasers, and thermal sensor data mounted on a mobile platform to detect and track people. Shashank et al. [19] developed an autonomous robot for an airport service with sensor fusion technology. However, recognition using camera images uses a lot of memory, and the computing load for human recognition is high. Accordingly, human recognition technology using a fiducial marker was applied to the following robot. Shan et al. [20] achieved accurate trajectory estimation and tracking control via the detection of fiducial markers using an RGBD camera. A depth camera and active IR marker set were used to improve indoor humans following the performance in [21]. Che et al. [22] proposed a tracking algorithm based on color segmentation and 2D marker detection. Low-cost cameras and ArUco marker-based human following are developed [23]. Also, ArUco marker was utilized in developing AGV (automatic guided vehicle) and tested in real-time testing environment [24]. Despite the many advantages of fiducial marker-based recognition systems, most studies do not consider the uncertainty surrounding the position estimation results derived using fiducial markers. Therefore, in this study, we propose an algorithm that improves position estimation and enables stable tracking of the worker. The contributions of this paper are as follows:

-

A Kalman filter was designed to improve the human position estimation by integrating the human walking model and the error model according to the distance measured using the ArUco marker.

-

A human-following controller for a mobile platform was designed by assuming a virtual spring damper model between a human and a robot.

This paper is structured as follows. Section 2 details the system covered in this study. This study was based on simulation. Therefore, Section 3 presents a description of the human gait model and the ArUco marker measurement probability model used to develop the algorithm in a realistic environment. The design of the Kalman filter to improve the position estimation performance and the motion control of the mobile platform based on the Mecanum wheel are described in Section 4 and Section 5, respectively. Section 6 describes the verification of the proposed algorithm’s performance through simulations. Finally, in Section 7, the discussion of the research results and conclusion are presented.

2. A Worker-Following Robot System

This study proposes a method of operating a Mecanum wheel-based mobile (MWR) robot applied to a smart factory logistics system. As shown in Figure 1, the proposed system consists of an image sensor attached to the mobile robot to recognize the fiducial marker (ArUco marker) printed on the rear side of the worker’s vest, a module that estimates the relative position of the marker through the recognized ArUco marker, and a motion control module to move the mobile robot to the estimated position.

In this research, we used an OpenCV module for ArUco marker recognition. We focused on a part where the mobile platform smoothly followed workers by using the locations of the workers as estimated through OpenCV.

In this study, a mobile robot position controller was designed using a robot [25] to which a Mecanum wheel was applied as a mobile platform. Figure 2 shows the MWR platform and its coordinate system. The rotation center of the robot is defined as the origin of the robot’s coordinate system. In addition, the direction perpendicular to the axis connecting the two pairs of Mecanum wheels while passing through the origin of the robot coordinate system is defined as the X (longitudinal) axis, and the horizontal direction, as the Y (lateral) axis. $R_w$ represents the radius of the Mecanum wheel. $l_1$, $l_2,$ $v_x, v_y$, and $w_z$ represent half of the distance between the wheels, half of the distance between the axles, the robot’s x-direction speed, the y-direction speed, and the rotational speed about the vertical axis, respectively.

The MWR is a mobile platform in which the Mecanum wheels are connected to four independent drive motors and can move in all directions through the combined force generated by each wheel. The velocity equation of the MWR can be calculated by [26]

$$v_i+R_w{w}_i\cos {\alpha }_i=v_x\mp l_1{w}_z,$$

(1)

$$R_w{v}_i\sin {\alpha }_i=v_y\pm l_2{w}_z,$$

(2)

where $v_i$ is the linear speed of each wheel, when $i=1, 2, 3, 4$ represent wheel numbers from front-left, front-right, rear-left, and rear-right, respectively. For each wheel position, the sign of $l_1$ and $l_2$ will be +/+, +/−, −/+, and −/−. And ${\alpha }_i$ is the Mecanum roller angle: ${\alpha }_1=\frac{\pi }{4}$, ${\alpha }_2=-\frac{\pi }{4}$, ${\alpha }_3=-\frac{\pi }{4}$, ${\alpha }_4=\frac{\pi }{4}$. Then the wheel speed is derived from Equations (1) and (2)

$$v_i=v_x\mp l_1{w}_z-\frac{v_y\pm l_2{w}_z}{\tan \left({\alpha }_i\right)}.$$

(3)

From $v_i=R_w{w}_i$, the angular velocity of each wheel can be derived. And substituting $i=1, 2, 3, 4$ and the corresponding Mecanum roller angle, the inverse kinematic model of the MWR under the no-slip condition is provided below [27,28]:

$$\left[\begin{array}{c}{w}_1\\ w_2\\ w_3\\ w_4\end{array}\right]=\frac{1}{R_w}\left[\begin{array}{ccc}1& -1& -\left(l_1+l_2\right)\\ 1& 1& l_1+l_2\\ 1& 1& -\left(l_1+l_2\right)\\ 1& -1& l_1+l_2\end{array}\right]\left[\begin{array}{c}{v}_x\\ v_y\\ w_z\end{array}\right]=J\left[\begin{array}{c}{v}_x\\ v_y\\ w_z\end{array}\right]$$

(4)

where $J$ is the Jacobian matrix of the inverse kinematics of the system, $L=l_1+l_2$, and $w_i \left(i=1, 2, 3, 4\right)$ means the angular velocity of each wheel. The forward kinematics of the MWR will be:

$$\left[\begin{array}{c}{v}_x\\ v_y\\ w_z\end{array}\right]=J^{+}\left[\begin{array}{c}{w}_1\\ w_2\\ w_3\\ w_4\end{array}\right]$$

(5)

where $J^{+}J=I$ and $J^{+}={\left(J^{T}J\right)}^{-1}{J}^T$ [28]. Then, Equation (5) can be rewritten as

$$\left[\begin{array}{c}{v}_x\\ v_y\\ w_z\end{array}\right]=J^{+}\left[\begin{array}{c}{w}_1\\ w_2\\ w_3\\ w_4\end{array}\right]=\frac{R_w}{4}\left[\begin{array}{cccc}1& 1& 1& 1\\ -1& 1& 1& -1\\ -1/L& 1/L& -1/L& 1/L\end{array}\right]\left[\begin{array}{c}{w}_1\\ w_2\\ w_3\\ w_4\end{array}\right]$$

(6)

That is, given a target that the mobile robot must follow, the motion controller generates control input for the robot by comparing the robot’s current position $\left(x_c, y_c, {\theta }_c\right)$ and destination $\left(x_T, y_T, {\theta }_T\right)$. Accordingly, the robot moves by controlling the angular velocity of each wheel of the MWR.

3. Human Walking Model

The worker-following robot in this study uses a Mecanum wheel as a driving mechanism and an image sensor to estimate the worker’s position. To estimate the position of a worker, a fiducial marker is used to improve the reliability of the recognition of a specific worker and object recognition through an image sensor. In this study, the widely used ArUco marker was used. As defined in Section 2, the worker-following robot considered in this study is a system that recognizes and follows the markers printed on the worker’s vest.

However, due to the limitations of the image sensor, the position estimation result produced using the marker presents uncertainty. Therefore, to ensure the successful mission performance of the worker-following robot, it is necessary to improve the reliability with which the location of the marker (the worker’s location) is identified using the image sensor. This study focused on the performance of the worker-following algorithm through simulations. To develop a more realistic algorithm, this study fused human gait model experimental data [29] and the results of a localization error experiment according to the relative distance between the image sensor and the ArUco marker [30]. Through this, the operator’s gait model with a measurement error according to the measurement distance was extracted.

3.1. Analysis of ArUco Marker Position Estimation

While the mobile robot follows the worker, the relative distance between the robot and the worker changes. Since this change in distance is caused by errors in position estimation according to the relative distance from the image sensor to the ArUco marker, this aspect is analyzed to generate gait data that closely represent reality. In [30], the percentage error and standard deviation when using the Logitech C270 Webcam were investigated, as shown in

Table 1. ArUco marker pose estimation error (Logitech C270 Webcam) [30].

Distance (cm)	Without Shadow			With Shadow
	Mean (%)	Standard Deviation	Success Rate (%)	Mean (%)	Standard Deviation	Success Rate (%)
75	2.067	0.021	97.07	2.034	0.027	95.98
100	2.204	0.028	95.94	2.184	0.028	96.03
150	2.242	0.112	96.18	2.443	0.315	96.07
175	2.315	0.229	96.10	2.620	0.197	95.99
200	3.050	0.315	96.08	3.615	0.200	96.47

, under the measurement conditions of with and without shadow.

Using the data in Table 1, the error prediction function according to distance was approximated as a quaternary function. Figure 3 shows a position error model based on the measured distance derived from the data in Table 1. The position estimation performance deteriorated as the distance between the operator and the image sensor increased.

3.2. Human Gait Model Movement Analysis

To ensure that the system proposed in this study could successfully conduct worker-following tasks, the accuracy with which the worker location was estimated needed to be improved. However, since there is no model for the movement of markers according to human gait, this study simulated the actual worker’s gait by referring to a gait data set [29]. To obtain the gait data set given in [29], as shown in Figure 4, the movements of markers (red, yellow, and green circles) attached to the human body were detected and stored using 10 cameras, as shown in Figure 4. In the reference data set [29], for healthy people moving at various speeds (C1: 0–0.4 m/s; C2: 0.4–0.8 m/s; C3: 0.8–1.2 m/s; C4: normal walking speed; C5: fast walking without running), the movement data for each marker were collected during the gait. In this study, we focused on CV7 (the coordinates of the 7th cervical vertebrae) and TV10 (the coordinates of the spinous process of the 10th thoracic vertebrae) when the gait speed was C4 (normal gait speed) from the reference data. This is because the ArUco marker was located near the center point of the two coordinate systems. Figure 5 shows the movement of CV7, TV10, and the midpoint coordinates of the two markers. That is, when the worker walked straight, the location of the marker recognized by the image sensor varied within the range shown in Figure 5.

4. Kalman Filter-Based Pose Estimation

In this study, an image sensor mounted on a mobile robot was used to follow a moving worker. Therefore, this study focused on developing the worker-following performance of the robot by simultaneously considering the position estimation error according to the worker’s gait pattern and the distance between the image sensor and the reference marker. Therefore, it was essential to post-process the worker’s position estimation results obtained from the image sensor. The Kalman filter [31] was used for this purpose. As shown in Figure 1, the walking direction is defined as the X-axis, the direction perpendicular to the walking direction as the Y-axis, and the opposite direction of gravity as the Z-axis, for the worker’s gait data. In addition, it was assumed that the worker was followed using longitudinal and lateral positional information in the six-axis coordinate system $\left(x, y, z,\Phi , \theta , \Psi \right)$ obtained through the image sensor. Therefore, the relative distance between the robot and the worker was acquired according to the measurement cycle of the image sensor, and these data were post-processed using the Kalman filter. In addition, to improve the robot’s operator follow-up control performance, the control input was calculated using the relative distance and relative speed obtained using the Kalman filter. That is, as a method of estimating speed using distance information, a Kalman filter model applicable to a linear system was used, and the applied linear system model was as follows:

$$X_{k+1}=A{X}_k+w_k$$

(7)

$$z_k=H{X}_k+v_k$$

(8)

where $X_k={\left[\begin{array}{cc}{x}_k& {\dot{x}}_k \end{array}\right]}^T$, $\mathrm{A}=\left[\begin{array}{cc}1& \mathsf{\Delta }t \\ 0& 1\end{array}\right], H=\left[\begin{array}{cc}1& 0 \end{array}\right].$ $w_k$ and $v_k$ represent the system noise and measurement noise, respectively.

In addition, if (Q, R) is defined the covariance matrix of the signal noise obtained using the trial-and-error method, and the initial estimated value ${\widehat{x}}_0$ and the initial error covariance value $P_0$ are defined, the Kalman filter gain is defined as follows:

$${\widehat{x}}_k^{-}=A{\widehat{x}}_{k-1}^{-}$$

(9)

$$P_k^{-}=A{P}_{k-1}{A}^T+Q$$

(10)

$$K_k=P_k^{-}{H}^T{\left(H{P}_k^{-}{H}^T+R\right)}^{-1}$$

(11)

$${\widehat{x}}_k={\widehat{x}}_k^{-}+K_k\left(z_k-H{\widehat{x}}_k^{-}\right)$$

(12)

$$P_k=P_k^{-}-K_{k}H{P}_k^{-}$$

(13)

Figure 6 shows the control structure of the proposed Kalman filter-based worker-following robot. From Equations (7) and (8), it can be seen that the data on the position of the worker collected by the image sensor pass through the Kalman filter, and the position and speed of the worker are simultaneously extracted. In the proposed robot control algorithm, the MWR control input is calculated through the comparison of robot tracking results (position and speed).

5. Motion Control of Mecanum Wheel Robot

As shown in Figure 6, the estimated value of the worker’s position, calculated and processed by the Kalman filter, was used as a reference value for the mobile platform to follow the worker. In this study, the worker-tracking performance was improved by simultaneously considering the position and the moving speed calculated and processed by the Kalman filter. Figure 7 shows the control structure of the Mecanum wheel-based mobile robot.

The kinematics for the motion control of a mobile robot is provided as follows [32]:

$$v_x=V\cos {\theta }_{MWR},$$

(14)

$$v_y=V\sin {\theta }_{MWR},$$

(15)

$$w_z=w,$$

(16)

where $V$ and $w$ represent the linear velocity and angular velocity of the mobile robot, respectively. For the control of the mobile robot, the linear velocity and angular velocity shown in Equations (14)–(16) must be defined. In this study, to define the linear velocity of the mobile robot, it was assumed that both the worker and the mobile platform were a point mass and that a virtual spring and damper existed between the two objects, as shown in Figure 8. Figure 8b,c shows the vertical view of the worker and mobile robot. In Figure 8c, $(x_{MWR}, y_{MWR}, {\theta }_{MWR})$ and $(x_w, y_w, {\theta }_w)$ represent the position of the mobile platform and the worker, respectively. The distance error and speed error were calculated through the error dynamics model for the relative motion of the robot and the worker, as shown in Figure 8:

$$dx=x_w\left(t\right)-x_{MWR}\left(t\right)$$

(17)

$$dy=y_w\left(t\right)-y_{MWR}\left(t\right),$$

(18)

$$e\left(t\right)=\sqrt{d{x}^2+d{y}^2}+l_{safe}$$

(19)

$$\dot{dx}\left(t\right)={\dot{x}}_w\left(t\right)-{\dot{x}}_{MWR}\left(t\right)$$

(20)

$$\dot{dy}\left(t\right)={\dot{y}}_w\left(t\right)-{\dot{y}}_{MWR}\left(t\right)$$

(21)

$$\dot{e}\left(t\right)=\dot{dx}\left(t\right)-\dot{dy}\left(t\right)$$

(22)

where the subscript w and MWR represent the worker and MWR, respectively. $l_{safe}$ means the safe distance for preventing collisions between the worker and the robot. The safety distance $l_{safe}$ could be decided according to the production environment. The two main factors to be considered to derive the safety distance are the worker’s moving speed and the road surface friction. If a work site requires high moving speed, a long safety distance will be required. However, in the opposite case, a short safety distance would be required. Another factor to consider in defining safety distance is road friction. In the proposed system, the robot moves along when the worker moves, and the robot stops when the worker stops. In such a system, the stopping distance may be different depending on the friction between the robot and the road surface. When the maximum movement speed of the worker is set as 2 m/s, and the friction of the road surface is 0.5, the braking distance of the robot is calculated to be approximately 0.27 m. If the moving speed of the worker is 0.3 m/s and the road friction is 1.0, the braking distance is calculated as about 0.04 m. Therefore, it is more efficient to set a distance suitable for the worker’s work convenience than to set a safety distance in detail according to the worker’s moving speed and road friction. In this research, we set the safety distance as 0.8 m. The linear velocity and angular velocity of the mobile robot were calculated using the following equation using the induced distance error and velocity error.

$$u\left(t\right)=Ke\left(t\right)+C\dot{e}\left(t\right)$$

(23)

$$V\left(t\right)={\displaystyle \int }u\left(t\right)dt$$

(24)

$$w\left(t\right)=\alpha \left({\theta }_w-{\theta }_{MWR}\right)$$

(25)

where u(t) represents the acceleration of the mobile robot, and K and C, represent the spring constant and damping coefficient, respectively. ${\theta }_w$ is the orientation of the worker, and $\alpha$ is a positive constant for tuning the angular velocity that represents how fast the robot faces the worker. To define the spring constant (K = 1) and damping coefficient (C = 7), the following cost function was used:

$$J={\int }_0^{t_f}{X}^{T}QX+u^{T}Ru dt,$$

(26)

where $\mathrm{X}=\left[\begin{array}{c}e\left(t\right)\\ e_{\theta }\left(t\right)\end{array}\right], e_{\theta }\left(t\right)={\theta }_w-\theta , u=\left[\begin{array}{c}V\\ w_z\end{array}\right], Q=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right], R=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]$.

Virtual forces from the spring and damper acted parallel to the line between the robot and the worker, as shown in Figure 8c. The angular velocity was set such that the robot faced the worker. Since the position and speed of the robot for calculating the control input were obtained based on the robot’s odometry and IMU information, and the position and speed of the worker were estimated and processed through the Kalman filter, the relative distance and relative speed between the two objects could be calculated. The controller was designed in such a way that a spring coefficient and a damping coefficient were applied, and an equivalent control input (acceleration) was generated. The linear velocity of the robot was derived through the integration of the induced acceleration input. The error between the target angle and the robot’s angle was used for the angular velocity of the robot, as shown in Equation (25).

6. Simulation Results

Numerical simulations were conducted using Matlab-Simulink to validate the performance of the worker-following robot proposed in this study. As the solver, the fifth order Dormand-Prince formula (ODE5) with a fixed time step (0.01 s) was used. To increase the realism of the simulation, the movement of the worker was created based on the human gait model defined in Section 3 and the measurement error model according to the measured distance of the ArUco marker. The performance of the proposed control method was confirmed by comparing the simulation results of the three control methods for two walking scenarios. The first scenario is designed to test the worker-following performance in the situation of simply following a worker walking in a straight line. This is a situation that occurs most frequently in actual work sites. The second scenario is designed to verify the proposed algorithm-based worker following the performance at a work site assuming an actual assembly line. This is a scenario in which a mobile platform loaded with parts follows a worker and transports necessary parts to each assembly station.

Among the three control methods, the first method was a simulation in which the movement of the worker on the X–Y plane was followed and controlled without Kalman filter-based signal processing. When the worker’s position derived from the gait model passed through the Kalman filter, the position information and speed information that had passed through the filter could be obtained. In the second simulation, the worker was followed based only on the position information, and in the third simulation, the speed of the mobile robot was controlled by simultaneously utilizing the position and speed information of the worker, as suggested in Section 5.

6.1. Straight-Line Walking Scenario

As can be seen from the gait data in Figure 4, the worker moved on a two-dimensional plane, and the movement patterns for the walking direction (X) and the direction perpendicular to the walking direction (Y) were different. Therefore, Kalman filters were designed separately for each coordinate axis. The Kalman filter for the X-axis data was designed as $A_x=\left[\begin{array}{cc}1& 0.1 \\ 0& 1\end{array}\right], H_x=\left[\begin{array}{cc}1& 0 \end{array}\right],{ \mathrm{Q}}_{\mathrm{x}}=\left[\begin{array}{cc}1& 0\\ 0& 5\end{array}\right], R_x=500$. The Kalman filter for the $Y$-axis data was designed as $A_y=\left[\begin{array}{cc}1& 0.1 \\ 0& 1\end{array}\right], H_y=\left[\begin{array}{cc}1& 0 \end{array}\right],{ \mathrm{Q}}_{\mathrm{y}}=\left[\begin{array}{cc}1& 0\\ 0& 5\end{array}\right], R_y=0.1$. Figure 9 shows the control performance of the simulation-based worker-following robot in a straight-line walking scenario. In the simulation results, the red line represents the movement of the worker. The case where the Kalman filter was not used is shown in pink, the case where only the distance data among the data obtained through the Kalman filter were used is shown in black, and the method proposed in this study is shown in blue. From the simulation results, it can be confirmed that the proposed method shows better tracking performance than the other methods in X-axis and Y-axis direction tracking. To quantitatively evaluate the performance, the RMSE of the worker following was investigated. For each of the three simulated cases, the RMSE was 0.4569 m, 0.1396 m, and 0.0456 m, confirming that the proposed tracking method exhibits superior performance. Figure 10 shows the longitudinal (advancing direction) velocity and angular velocity, which are the control inputs of the mobile platform.

6.2. Smart Factory Assembly Line Scenario

For a more practical application case for the developed algorithm, a worker-following robot scenario in the assembly line of a smart factory was designed. Figure 11 shows a conceptual diagram of a smart factory introduced in an online article [33] related to smart factories. The environment presented in Figure 11 is a smart factory with complete autonomy. However, this study assumed a work site in which humans and robots collaborated. In other words, the robot’s role was to carry heavy objects while following the worker. For the simulation scenario, it was assumed that a worker moved sequentially from workstation ① to ② to ③. In the case of the existing AGV, the robot moved along the red line in Figure 11b, but in the case of a human, it was assumed that the robot moved along the blue line for the efficiency of the travel path. Accordingly, the robot followed the blue line by following the worker ahead.

For the simulation, the movement path of the worker in the blue line was created through the linear segment parabolic blend (LSPB) method [34,35] as in Appendix A, a trajectory planning method. In addition, the final path was generated by applying the lateral ($Y$ direction) movement of the walking model analyzed in Section 3 and the ArUco marker-based position estimation error according to the measurement distance together. Figure 12 shows the vehicle-tracking simulation results for the smart factory scenario. It can be confirmed that all three control methods followed the worker successfully. For the simulation, the same parameters for the Kalman filter were applied as for the straight-line walking simulation. Additionally, the initial position of the robot is defined as $\left(x,y,\theta \right)=\left(9,28,-\pi /2\right)$. For each control method, the proposed virtual spring damper method produced the lowest RMSEs, showing performances of 1.0446 m, 1.0177 m, and 1.0052 m, respectively.

7. Conclusions

Industrial sites are in a transitional stage of transforming into fully autonomous systems using robotic technology. However, to protect the labor rights and income of field workers, a production system in which humans and robots collaborate is actively applied. This study was developed to increase the work convenience of industrial field logistics workers according to the demands of the industry. In this study, as a key technology for worker-following robots applied to field logistics work, a method for improving the performance deteriorated as the distance between the operator and the image sensor increased accuracy with which the pose of a worker based on an image sensor-ArUco marker is estimated, and a control method for stable worker following are proposed. In the proposed human-following robot system, a worker wears a vest with an ArUco marker printed on it that has a specific ID. Through this system configuration, the mobile robot estimates the position of and tracks the worker. The Kalman filter was applied to improve the position estimation and to enable stable control by assuming a virtual spring and damper between the robot and the worker. The performance of the proposed pose estimation and velocity control method was verified through two simulation scenarios: a straight-line movement scenario and assembly line environment scenario. For both simulation scenarios, the results demonstrate improvements in the human-following performance under the proposed pose estimation and motion control strategy, with a smaller RMSE for the worker-tracking error. However, this study has limitations, as it was based on numerical simulations. To expand the field applicability of the proposed system, we plan to conduct research for the development and improvement of algorithms in the field with the same system configuration based on actual robots in a factory environment.