A Novel Ground-to-Elevated Mobile Manipulator Base System for High-Altitude Operations

Abstract

Mobile manipulators have the potential to replace manual labor in various scenarios. However, current mobile base designs have limitations when it comes to accommodating complex movements that involve both high-altitude tasks and ground-based composite tasks. This paper presents a new design for the mobile manipulator base, which utilizes a time-sharing drive with gears and differential wheels. Additionally, a new foldable mechanical gear-track system has been developed, enabling the robot to effectively operate on both the ground and the mechanical gear-tracks. To address the challenges of power distribution and localization caused by the mechanical characteristics of the designed track, this study proposes a precise pose estimation method for the robot on the mechanical gear-track, along with a compliance control method for the gears. Furthermore, a segmented multi-sensor fusion navigation approach is introduced to meet the high-precision motion control requirements at the entrance of the designed track. Experimental results demonstrate the effectiveness of the proposed new mobile manipulator base, as well as its corresponding control methods.

1. Introduction

Elevated work environments often pose significant safety risks to human workers. A mobile manipulator, thanks to its flexible mobile base and high degree of freedom operator hand, can perform a wider range of operation tasks and replace manual work in many scenarios [1]. In the work-at-height operation situation, there are problems such as heavy manual workload, high risk factor, and complex and variable operation scenes. For instance, the environment discussed in this article is illustrated in Figure 1. In fields such as cargo transportation and storage management, height cleaning operations, and power inspection and maintenance, there is a need for mobile robots to perform operations both on the ground and at high altitudes. This would allow them to effectively handle a wide range of work-at-height operation scenarios. Although there have been several advancements in the design of mobile bases to accommodate various complex environments, there are limitations when it comes to tasks that involve both ground and height operations using the current bases.

Currently, the majority of research on the robot mobile base of mobile manipulators focuses primarily on ground motion [2,3,4], with a major characteristic being the use of wheeled mobile bases. Thakar et al. developed a base, motion planning, and coordination of the manipulator to complete specific tasks, such as indoor disinfection spraying [5], in a flat indoor environment. Piyapunsutti et al. used a base in a restaurant environment to provide food and drinks as an automated navigation service robot for customers [6], among other examples. However, as tasks and environments become more complex, conventional bases struggle to meet the requirements. Some researchers have studied the mobility and increased motion capabilities of mobile bases in more complex environments. For instance, K. Tadakuma et al. proposed a cylindrical track platform based on a caterpillar-like mechanism to achieve lateral motion [7,8]. Ernits and Hoppe [9], as well as Fiede´n and Bałchanowski [10], proposed track-type mobile platforms composed of Mecanum wheels. These solutions, to some extent, address how to extend the adaptability of mobile robots to various special ground environments, but these mobile platforms are unable to meet the requirements of work-at-height operations. In order to enable robots to perform tasks at high places, some researchers have developed a track-type robot mobile base, which has shown unique application value in scenarios such as height inspections of substations [11] and suspended powder feeding operations [12]. This type of robot mobile base successfully overcomes the limitations of traditional operating ranges, effectively extending the task execution area of the operating robot from the ground to high places. However, most of these track-type robot mobile bases fix the robot body on a pre-designed horizontal track, resulting in limited operating range and occupying a large amount of space for track deployment and installation, as well as posing significant challenges for maintenance. Xu et al. also designed a climbing robot with four-wheel magnetic adhesion for welding work on large steel surfaces [13], which has good mobility, but its operating environment is only suitable for large, flat, magnetically attachable surfaces. Liu et al. proposed a new climbing robot with spiked tracks and a double-track mechanism [14], which can climb on rough vertical surfaces and has good flexibility. However, its special structural design results in weak load capacity, making it difficult to replace humans in general load-bearing tasks. Once a large load is added, its climbing ability will be limited.

Existing academic research highlights significant challenges in achieving composite motion for ground-to-high transitions and simultaneously performing dexterous operations with mobile manipulator robot bases. In response to these challenges, this paper proposes an innovative configuration design for a mobile robot base. The core innovation of this design lies in enabling the robot to operate freely and dexterously in both ground and elevated environments. This mobile base utilizes a gear and differential wheel time-sharing drive mechanism, combined with a 6-DOF (degree-of-freedom) robotic arm, to meet the demanding requirements for flexibility and precision in complex tasks. However, this new base design introduces several challenges: at the mechanical structure level, it is essential to develop a compatible track system to facilitate climbing; at the control method level, the mobile base must ensure the robot can perform dexterous operations across a wide range of ground areas and at elevated positions, which presents significant challenges to its positioning and control methods; additionally, achieving stable and smooth climbing operations on the designed track is a major challenge. Following the presentation of this mobile robot base configuration, this paper briefly addresses the related issues arising from the reliable operation of the entire robot system based on this base and provides corresponding solutions. The specific challenges are as follows:

(1)

For the mobile base designed in this study, it is essential to thoroughly consider the feasibility of manufacturing and transportation, as well as the flexibility of adjustment and adaptation in practical applications. This ensures that its mobility and adaptability are superior to those of fixed tracks when addressing diverse work sites and task requirements.

(2)

Accurate pose estimation of robots at high altitudes serves as the foundation for flexible operations in elevated environments. However, existing height estimation methods exhibit certain limitations. For instance, the accuracy of height measurement using LiDAR is constrained [15]. Additionally, in the actual scene described in this paper, the measured plane is uneven and subject to strong vibrations caused by movement, which makes the measurement data unreliable. When using RTK (real-time kinematic) devices for height estimation, there is a problem of insufficient accuracy [16]. The utilization of infrared ranging sensors also presents disadvantages, such as interference from strong environmental light and a limited measurement range [17].

(3)

In the climbing drive section of our base configuration, we implement a rigidly connected independent drive motor architecture. However, when encountering curved tracks with substantial curvature variations, the robot’s posture remains in a continuous dynamic state. Furthermore, the designed foldable track includes sections with missing racks, presenting a significant challenge for the smooth and seamless passage of the compliant drive gear through the curved and suspended segments of the track.

(4)

These compound mobile operation robots are primarily designed for outdoor environments [18,19]. The conventional localization method used is RTK for absolute pose estimation [20], but the accuracy of RTK devices significantly decreases when encountering strong electromagnetic interference [21]. Additionally, the mechanical track in this paper has a small allowable pose error when entering the track in order to closely match the robot body. It is difficult to meet the positioning accuracy requirements solely relying on RTK [22,23,24]. Simultaneously, for the stability of outdoor work, adaptability to harsh environments should also be considered, such as working stability under strong electromagnetic interference and adverse weather conditions.

The main contributions of this article are delineated as follows:

• We present a new design for mobile manipulators, along with an innovative design for mechanical gear-tracks.
• We introduce a robot navigation algorithm that is based on the fusion of multi-sensor information and localization techniques. This algorithm significantly enhances the adaptability and accuracy of localization systems within complex outdoor environments.
• We outline a methodology for robot localization and for ensuring smooth operation on the track. This methodology addresses the challenges associated with pose estimation in a compound robot mobile base for work-at-height operations and effectively enhances the stability and smoothness of robot movement on both curved and suspended tracks.
• A series of empirical tests were conducted in real-world environments to validate the proposed robot mobile base system and methodology.

The remainder of this article is organized as follows. Section 2 introduces the design of the mobile manipulator system and its corresponding gear-climbing track system. Section 3 describes the base control method of the base, which encompasses ground multi-sensor fusion navigation control, track localization, and compliance driving methods. Section 4 presents experiments conducted with the system designed in this article. Section 5 concludes the discussion.

2. Design

2.1. Robot General Design

The mobile manipulator system designed in this study primarily consists of a robot body and a climbing track. The robot body includes a dual-drive mobile base, a manipulator arm, and its control system hardware. The mobile manipulator system is shown in Figure 2.

The total mass of the designed robotic structure is approximately 50.5 kg, with an external envelope dimension of $610\ast 930\ast 750$ mm in the retracted state of the robotic arm. Taking into account the mass of the robotic structure and the potential mass of the operational equipment it may carry, the drive system must be capable of supporting a maximum load of 58 kg, and the robotic arm can accommodate a maximum load of 7 kg. This robot is primarily utilized in high-altitude operation scenarios, where the working height is determined by the length of the mechanical track. The nominal working height is set at 15 m, and it must meet the requirement of rapid climbing, meaning the time required to ascend to the nominal height should not exceed 2 min. From the perspective of practical engineering applications, battery endurance is crucial. It should ensure that the robot can operate continuously for at least 2 h and remain on standby for over 8 h, thereby guaranteeing the robot’s availability and operational continuity in real-world working conditions.

2.2. Gear-Track and Gear-Climbing Mechanism Design

In the design of the mobile base in this paper, the mechanical gear-track and gear-driven climbing design are key elements for achieving its functionality. Firstly, the structural characteristics and design principles of the mechanical gear-track are introduced. Secondly, the focus is on the dual gear-driven design on the base, as well as the guiding and mechanical limit mechanisms designed on both sides of the drive wheels. The overall design diagram of the track and mobile base is shown in Figure 3.

2.2.1. Gear-Track Mechanism Design

For the mobile base designed in this study, with full consideration of manufacturing and transportation feasibility, as well as the flexibility for adjustment and transplantation in practical applications, a foldable mechanical gear-track has been developed. Furthermore, a suspended track interruption structure is implemented at the beginning of the vertical track, which increases the permissible error after the track is folded and unfolded, thereby reducing the precision requirements for the track control mechanism. The folding mechanism of the track is described in Figure 4.

The mechanical track is designed to ensure coordination with the mobile base, and its design primarily comprises the vertical track, 90° curved track, horizontal track, and the front flat section. The horizontal section is set to 1.5 m, and the vertical section is set to 15 m. The schematic diagram of the track structure is illustrated in Figure 5.

The overall structure of the track primarily comprises guiding plates on both sides and a metal rack. The metal rack is installed between the guiding plates and can engage with the self-locking gear on the base drive mechanism to generate driving power. The width of the guide plate is 386 mm, and the width of the guide groove is 33 mm, which is slightly larger than the 26 mm diameter of the base’s rubber wheel. The grooves on the guiding plates facilitate the uniformity of the relative position between the base movement and the track.

The arc track is responsible for directing the robot from a horizontal surface to a vertical track. The track at the entrance consists of a combination of a gradually narrowing guiding section and a gradually widening guiding section. When the robotic base reaches the beginning of the track, it initially enters the track through these two guiding sections via the guiding rods located at its bottom. The distance between two guide rods is 510 mm, whereas the stable guide section within the gradually narrowing guiding section measures 530 mm. Subsequently, it ensures that the rubber wheels of the robotic front guiding mechanism enter the gradually widening track section, as the track width increases from an initial measurement of 370 mm to 386 mm, thereby completing a full entry onto the track.

In this study, after the curved section of the track, the rack is not directly laid on the upper vertical track—instead, a certain length of suspended track is innovatively reserved, as shown in Figure 6. One advantage of this design is that it ensures the normal folding of the track and avoids interference with the lower track. The second advantage, which is more critical, is that in the actual operation of the folding mechanism, it is impossible to ensure that it unfolds from the folded state to a completely consistent vertical state with precision and accuracy. If the rack is continuously laid, these angular errors will seriously affect the climbing of the robot. This is because in order to ensure stable and reliable climbing, the robot base needs to ensure that the gear drive is fixed relative to the rack pose and is highly sensitive to deviations. For example, if the rack is too far away from the gear, the driving gear will lose driving force due to the inability to make contact. A more severe situation is that if the rack is too close to the gear, the locking force generated by the fastening mechanism on both sides of the track and the meshing force between the middle track and the gear will create a strong opposing force. This intense mutual resistance is likely to cause damage to the mobile base. Based on this, this paper designs a foldable track joint interface and removes some of the rack, while also designing the grooves on both sides of the starting point as a trumpet-shaped guide. These two innovative designs greatly increase the allowable angular error during the unfolding of the track.

2.2.2. Gear-Climbing Mechanism Design

As illustrated in the overall design diagram of the track and mobile base in Figure 5, the robot’s mobile base must first autonomously navigate from the ground into the designated mechanical track. Following this, it will transition from a horizontal to a vertical climbing state on the mechanical track until the task is completed. Consequently, the mobile base must satisfy the following requirements:

• It must possess the capability to autonomously enter the mechanical track from an open ground.
• It must incorporate a mechanical limiting mechanism to ensure stable movement on the track.
• The entire climbing process must maintain continuous power.

To address these requirements, the specific design is as follows:

• The base moves smoothly along the track, primarily utilizing the track guidance mechanism and the track drive mechanism. Upon reaching the track’s starting place, the base employs the guide rod located at its base to initiate entry into the track through the progressively narrowing guide section. Following this, the rubber tires of the front guidance mechanism engage with the gradually widening track section.
• When the base fully enters the track, the track guiding mechanisms installed on both sides of the front and rear drive motors constrain the base to the guide rail. Each set of track guiding mechanisms comprises a rubber tire, a universal ball, and a support plate, ensuring that the base moves only in a direction parallel to the guide rail. Figure 7 below illustrates the constraint method.
• As illustrated in Figure 5, a gap exists in the rack of the mechanical gear-track, resulting in a power loss issue for the base when the gear is suspended at this point. To ensure continuous and stable climbing power for the base, independent and complete drive gears have been installed at both the front and rear of the base. When one gear is suspended and loses power transmission due to the interruption of the rack, the other gear can continue to provide power. Taking the drive motor mechanism on one side as an example, it primarily consists of a drive motor, support components, bearings, a shaft, and a self-locking gear, and the composition of this mechanism is depicted in Figure 8.

Among these components, the self-locking drive gear, serving as a critical element, features a module of 2.0, 40 teeth, a shaft hole diameter of 17 mm, and is capable of transmitting a torque exceeding 78 N·m.

3. Localization and Control Methods

3.1. Localization and Control Framework

The overall localization and control framework of the robot mobile base consists of two main parts: navigation control method based on multi-sensor fusion localization and on-track localization and compliance driving method. In terms of navigation control, one approach is to use RTK and wheel odometry for pose estimation and path tracking on open ground. Another approach is to fuse RTK, wheel odometry, camera vision, and magnetic sensors in near-track areas to achieve more accurate pose estimation for precise on-track motion. In terms of on-track localization and smooth driving, the high-precision encoders of the gear motors are used, combined with the determined mechanical gear-track design parameters, to jointly calculate real-time pose. Based on this, real-time adjustment strategies are applied to allocate the driving motors, ensuring that the torque of the follow-up wheels follows the changes of the main driving wheel, thus achieving smooth control of the gear motors. The control block diagram is shown in Figure 9 below.

3.2. Ground Multi-Sensor Fusion Localization and Navigation Methods

Considering the susceptibility of RTK to electromagnetic interference and the difficulty in meeting the localization requirements in proximity areas when operating outdoors, we propose a segmented navigation and localization algorithm based on multi-sensor information fusion. Specifically, in open and spacious ground environments, it is sufficient for the system to achieve convergence in a limited time for localization. We use a fusion approach that combines RTK with wheel odometry for navigation. As shown in Figure 10, when the robot is about to enter the mechanical gear-track phase, we collect localization data from vision, magnetic, RTK, and wheel odometry. We consider the accuracy of individual sensor-based pose estimation and the validity of the data, and use an Extended Kalman Filter (EKF) to perform weighted fusion, allowing for high-precision pose estimation even when a single sensor fails to provide accurate localization.

The RTK localization system used in this study consists of a mobile station and a base station. The base station is installed in a fixed pose, while the mobile station is installed on the robot base. Real-time communication between the mobile station and the base station is used to calculate the pose of the mobile station in the Earth coordinate system. The wheel odometry uses motor encoder readings and motion models to estimate the robot’s displacement. The visual localization method identifies pre-set cooperative markers on the ground to compute the current pose of the robot. However, due to the complex and changing outdoor environment, monocular cameras often encounter problems such as marker contamination or shadows caused by sunlight, making it difficult to recognize the cooperative markers. In such cases, the magnetic sensor can effectively compensate for the localization gaps caused by the failure to recognize cooperative markers by detecting specially attached magnetic strips. However, the detection range of the magnetic sensor itself is relatively limited. The significant advantage of the wide range of localization offered by visual and RTK localization can complement the advantages of the magnetic sensor, thereby meeting the precise localization and stable navigation requirements of robots in complex outdoor environments.

3.2.1. Methods for Open Ground

During the navigation process of the robot on open ground, the first step involves using RTK pose information as a prior pose. Simultaneously, the robot integrates motion measurements from the wheel encoders to match the estimated pose with the map features. The robot’s mobile base moves on the ground through differential drive motion, which is accomplished by assigning different speeds to its two wheels. The simplified differential drive motion model is depicted in Figure 11, with the world coordinate system defined as $XOY$.

From the equation of circular motion, $R={\displaystyle \frac{v}{\omega }}$, where the base is running at v, the angular velocity of rotation is $\omega$, the center of rotation is o, the speed of the left driving wheel is $v_l$, the speed of the right driving wheel is $v_r$, and the radius of rotation of the cart is R. The sign of R is determined by the signs of v and $\omega$. When R is positive, the center of the circle o is to the left of the base’s center of control. When R is negative, the center of the circle o is on the right side of the base’s control center. When R is 0, the base rotates in place, with the center of the circle o at the base’s control center. When R is $\pm \infty$, the base moves in a straight line, with the center of the circle at infinity to the left or right.

Assuming that the base is a rigid body connection, at that moment, the radius of the left driving wheel of the base from the center of rotation o is $R_l$, the radius of the right driving wheel from the center of rotation o is $R_r$, and the distance between the two driving wheels is D. Then, according to the relationship between rotational velocity and angular velocity, we can obtain $\omega ={\omega }_r={\omega }_l$, and since $R_l-R_r=D$, we have the following:

$$\begin{array}{c}\hfill {\omega }_r={\displaystyle \frac{v_r}{R+\frac{D}{2}}},{\omega }_l={\displaystyle \frac{v_l}{R-\frac{D}{2}}}\\ \hfill v_r=v+\omega \times {\displaystyle \frac{D}{2}},v_l=v-\omega \times {\displaystyle \frac{D}{2}}\end{array}$$

(1)

Due to the symmetrical design of the two driving wheels on the base’s body, the following can be inferred:

$$\begin{array}{c}\hfill \omega ={\displaystyle \frac{v_r-v_l}{D}},R={\displaystyle \frac{\left(v_r+v_l\right)D}{2\left(v_r-v_l\right)}}\end{array}$$

(2)

By considering the above Equation (2) for circular motion, it can be shown that

$$\begin{array}{c}\hfill v=R\omega ={\displaystyle \frac{v_r-v_l}{D}}\times {\displaystyle \frac{\left(v_r+v_l\right)D}{2\left(v_r-v_l\right)}}={\displaystyle \frac{v_r+v_l}{2}}\end{array}$$

(3)

The real-time pose of the mobile base can be determined from this:

$$\begin{array}{cc}\hfill & \Delta x=X_{t+1}-X_t=cos{\theta }_t\times v\times \Delta t\hfill \\ \hfill & \Delta y=Y_{t+1}-Y_t=sin{\theta }_t\times v\times \Delta t\hfill \\ \hfill & \Delta \theta ={\theta }_{t+1}-{\theta }_t=\omega \times \Delta t\hfill \end{array}$$

(4)

where the current coordinates are $(X_{t+1},Y_{t+1},{\theta }_{t+1},{\theta }_{t+1})$, the coordinates of the previous moment are $(X_t,Y_t,{\theta }_t)$, and the time interval is $\Delta t$.

After obtaining the estimated pose of RTK and wheeled odometry, the flow block diagram illustrating the fusion localization is presented in Figure 12:

The framework combines the pose information from odometry and RTK with the EKF. The EKF consists of two components: a prediction step and an update step. The pose information obtained from wheeled odometry serves as the input value for the prediction step, while the pose information acquired from RTK is used as the observation value for the update step.

Assuming that the robot base moves in a two-dimensional plane in the ground motion area, the state vector of the system can be represented by the base’s pose given the pose ${\mathit{x}}_k=\left[X_k,Y_k,{\theta }_k\right]$ at time k, where $X_k,Y_k$ represent the geometric center position of the mobile robot and ${\theta }_k$ represents the pose of the mobile robot in the navigation coordinate system.

• State prediction
  The pose of the mobile robot at the moment $t+1$ can be expressed as

  $$\begin{array}{c}\hfill {\mathit{x}}_{k+1}=f({\mathit{x}}_k,{\mathit{u}}_{k+1})+{\mathit{w}}_{k+1}\end{array}$$

  (5)
  In Equation (5), ${\mathit{x}}_{k+1}$ represents the state variable at time $k+1$. ${\mathit{u}}_{k+1}$ denotes the control input to the system at time $k+1$, while ${\mathit{w}}_{k+1}$ represents the motion noise at time $k+1$. Finally, $f\left({\mathit{x}}_k,{\mathit{u}}_{k+1}\right)$ represents the relationship between the state variables ${\mathit{x}}_{k+1}$ and ${\mathit{x}}_k$.
  Prediction of the pose estimate at the moment $\left(k+1\right)$ from the robot motion model:

  $$\begin{array}{c}\hfill {\overline{\mathit{x}}}_{k+1}={\widehat{\mathit{x}}}_k+\left[\begin{array}{c}\Delta x\\ \Delta y\\ \Delta \theta \end{array}\right]\end{array}$$

  (6)

  where ${\overline{\mathit{x}}}_{k+1}$ is the estimated value of the positional attitude at the moment $(k+1)$ predicted by the motion model, and ${\widehat{\mathit{x}}}_k$ is the predicted value of the positional attitude at the moment k.
  The covariance matrix ${\overline{\mathit{P}}}_{k+1}$ of the a priori estimates of the predicted state vector is

  $$\begin{array}{c}\hfill {\overline{\mathit{P}}}_{k+1}=\nabla {\mathit{f}}_x{\mathit{P}}_k\nabla {\mathit{f}}_x^T+\nabla {\mathit{f}}_w{\mathit{Q}}_{k+1}\nabla {\mathit{f}}_w^T\end{array}$$

  (7)
  In Equation (7), ${\mathit{P}}_k$ and ${\mathit{Q}}_{k+1}$ represent the covariance matrices of the pose ${\mathit{x}}_k$ and process noise ${\mathit{w}}_{k+1}$, respectively. $\nabla {\mathit{f}}_x$ and $\nabla {\mathit{f}}_w$ represent the Jacobian matrices of the motion model and the Jacobian matrix of the process noise, respectively.
• State update.
  At time $(k+1)$, the observation model for the localization information provided by RTK can be obtained as follows:

  $$\begin{array}{c}\hfill {\mathit{z}}_{k+1}=\left[\begin{array}{c}{X}_{RTK}\\ Y_{RTK}\\ {\theta }_{RTK}\end{array}\right]+v_{k+1}\end{array}$$

  (8)
  In Equation (8), ${\mathit{z}}_{k+1}$ represents the system observation at time $(k+1)$, $v_{k+1}$ represents the observation noise at time $(k+1)$, and $\left[\begin{array}{c}{X}_{RTK}\\ Y_{RTK}\\ {\theta }_{RTK}\end{array}\right]$ represents the pose values calculated by RTK.

  (1)

  Calculate the Kalman gain as follows:

  $$\begin{array}{c}\hfill {\mathit{K}}_{k+1}={\overline{\mathit{P}}}_{k+1}\nabla {\overline{\mathit{h}}}_x{\left(\nabla {\mathit{h}}_x{\overline{\mathit{P}}}_{k+1}\nabla {\mathit{h}}_x^{\mathrm{T}}+{\mathit{R}}_{k+1}\right)}^{-1}\end{array}$$

  (9)

  where ${\mathit{K}}_{(k+1)}$ represents the Kalman gain at time $(k+1)$; $\nabla {\mathit{h}}_x$ represents the Jacobian matrix of the measurement model, which is known to be the identity matrix according to the observation model; and ${\mathit{R}}_{(k+1)}$ represents the covariance matrix of the observation noise, provided by the RTK pose conversion package.

  $$\begin{array}{c}\hfill {\mathit{R}}_{k+1}=\left[\begin{array}{ccc}{\sigma }_X& 0& 0\\ 0& {\sigma }_Y& 0\\ 0& 0& {\sigma }_{\theta }\end{array}\right]\end{array}$$

  (10)

  In Equation (10), ${\sigma }_X{\sigma }_Y{\sigma }_{\theta }$ is the observed noise variance of the pose $XY$ and attitude $\theta$ of the RTK output, respectively.

  (2)

  A posteriori of the state variables.

  $$\begin{array}{c}\hfill {\widehat{\mathit{x}}}_{k+1}={\overline{\mathit{x}}}_{k+1}+{\mathit{K}}_{k+1}\left(z_{k+1}-{\overline{\mathit{x}}}_{k+1}\right)\end{array}$$

  (11)

  (3)

  Update the covariance matrix to be the posterior estimate covariance matrix of the state variables:

  $$\begin{array}{c}\hfill {\mathit{P}}_{k+1}=\left({\mathit{I}}_3-{\mathit{K}}_{k+1}\right){\overline{\mathit{P}}}_{k+1}\end{array}$$

  (12)

  In the equation, ${\mathit{I}}_3$ is the third-order unit matrix.

At this stage, the data fusion process of odometry and RTK is finalized. It should be noted that the RTK system is installed in the center of the base body’s driving wheels, while the odometry calculation uses the midpoint between the two driving wheels as the reference point. The outputted localization result from the EKF can be directly utilized as the localization input for the navigation planner, allowing for subsequent path tracking. The control block diagram of the navigation planner is illustrated in Figure 13:

By converting the control information of the robot base from navigation into the corresponding speeds of the left and right wheels, precise control of the robot base can be achieved in order to follow the planned trajectory. The software for ground navigation of the robot employs movebase as the fundamental framework. In order to find the shortest effective path, the A* algorithm is utilized as the global path planner. Once a feasible and optimal global path is generated by the A* algorithm, the TED algorithm (timed elastic band algorithm) is used as the local path planner to generate smooth and safe local paths.

3.2.2. Method for Near-Track Areas

In the near-track area, high-precision localization is required for the mobile base of a robot due to the high demand of track design for track error. At the same time, RTK devices experience more severe electromagnetic interference in the near-track area, which causes localization drift. Wheel odometry is prone to slippage on smooth surfaces, and vision-based pose estimation encounters difficulties in recognizing cooperative markers due to outdoor environmental contamination or shadow caused by sunlight. Additionally, the detection range of magnetic sensors is much smaller than other systems. Given these challenges, it is proposed to integrate RTK and wheel odometry, and further enhance the accuracy of the system’s pose estimation by utilizing vision and magnetic sensors.

The robot utilizes a magnetic sensor installed on the right side of the mobile base to detect the magnetic field above a magnetic strip. By comparing the collected signals of magnetic field intensity, when the magnetic strip is located below the sensor, the robot calculates its position and orientation. The relationship between the determined magnetic field intensity signals obtained by placing the magnetic strip below the sensor and the positional differences is then fitted into a function f to estimate the deviation in position and orientation. The calculation process can be summarized as follows:

$$\begin{array}{c}\hfill \left(\begin{array}{c}{x}_{mag}\\ y_{mag}\end{array}\right)=f(S_1,S_2,\dots ,S_n)\end{array}$$

(13)

where $S_i$ represents the magnetic field intensity signal collected at the nth sampling point vertically inside the magnetic field sensor above the magnetic strip, and $(x_{mag},y_{mag})$ represents the positional deviation of the magnetic strip with respect to the magnetic field sensor. The converted positional relationship of this pose to the current position of the base can be expressed as

$$\begin{array}{c}\hfill \left(\begin{array}{c}{x}_{robot}\\ y_{robot}\\ {\theta }_{robot}\end{array}\right)=T\times \left(\begin{array}{c}{x}_{mag}\\ y_{mag}\\ 0\end{array}\right)\end{array}$$

(14)

where $(x_{robot},y_{robot},{\theta }_{robot})$ represents the current pose of the robot’s base and T represents the transformation matrix from the sensor coordinate system to the base coordinate system.

The robot utilizes visual pose estimation, which involves employing a high-definition monocular camera installed at the middle position of the front side of the base. This camera is used to recognize AprilTag cooperative markers. First, the pose of the camera in the base coordinate system is determined and represented by a transformation matrix $T_{robot}^C$. Through a visual detection algorithm, we can obtain the pose of the cooperative marker in the camera coordinate system, which is represented by a transformation matrix $T_C^{tag}$. The expressions for these transformations are as follows:

$$\begin{array}{c}\hfill T_{robot}^C=\left[\begin{array}{cc}{R}_{robot}^C& t_{robot}^C\\ 0_{1\times 3}& 1\end{array}\right],T_C^{tag}=\left[\begin{array}{cc}{R}_C^{tag}& t_C^{tag}\\ 0_{1\times 3}& 1\end{array}\right]\end{array}$$

(15)

where $R_{robot}^C$, $R_C^{tag}$ represents the rotational matrix for the corresponding posture, and $t_{robot}^C$, $t_C^{tag}$ represents the translation vector for the corresponding position.

To obtain the pose of the base in the collaborative sign coordinate system, we first calculate the pose of the camera in the collaborative sign coordinate system. Then, we use composite transformation to calculate the pose of the base in the collaborative sign coordinate system, represented by the transformation matrix $T_{tag}^{robot}$. The calculation formula is as follows:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & T_{tag}^{robot}=T_{tag}^C{T}_{robot}^C\hfill \\ \hfill & =\left[\begin{array}{cc}{R_C^{tag}}^T{R}_{robot}^C& {R_C^{tag}}^T{t}_{robot}^C-{R_C^{tag}}^T{t}_C^{tag}\\ 0_{1\times 3}& 1\end{array}\right]\hfill \end{array}\end{array}$$

(16)

Based on the use of vision and magnetic sensor to generate pose estimation, the fusion localization is then combined with RTK and wheel odometry. The schematic diagram of the fusion localization is shown in Figure 14.

This framework builds upon the use of an EKF to fuse the localization information provided by odometry and RTK. In addition, it incorporates the fusion of magnetic sensor and vision estimated pose information using a new EKF. The prediction and update steps in this framework differ from the previous framework. The previous moment’s information of the magnetic sensors and vision estimated pose are used as input values for the prediction step to generate the current prediction state estimation, while the measurements obtained from the two actual sensors at the current moment serve as the observation values for the update step. Furthermore, prior to the EKF process, the estimated pose, generated by weighted calculations based on sensor characteristics, is adjusted to minimize the impact of robot motion speed and the distance of collaborative identification on pose estimation. Subsequently, the pose estimation results are aligned to the RTK navigation coordinate system, taking into account the measurement covariance of RTK. Finally, a weighted fusion is performed to obtain the final estimated pose. The generated pose estimate is used as input for the navigation planner, allowing for path following and ultimately reaching the entrance of the mechanical gear-track. This process is consistent with the ground segment.

3.3. On-Track Localization and Compliance Driving Method

The on-track pose estimation method is essential for ensuring the safety of mobile manipulator during aerial operations and serves as the foundation for the compliance navigation of the robot through curved track segments. To simplify the issue, we abstract the gear movements of the robot base as a simple mechanical structure consisting of a thin rod connecting two pulleys at the front and rear, as illustrated in Figure 15. The following stages represent key steps that the robot must undergo for precise pose computation during the climbing process. The estimation of pose on the track can be divided into the following steps:

• When the robot enters the gear-track from the horizontal plane without entering the curved track, its current pose can be simply represented as

  $$\begin{array}{c}\hfill \begin{array}{cc}\hfill \theta & =0,y=\Delta d,z=0,\hfill \\ \hfill {\theta }_1& =\theta ,y_1=y,z_1=z\hfill \end{array}\end{array}$$

  (17)
  In this context, the base pitch angle is denoted as $\theta$, the distance that the center of mass of the base moves forward relative to the starting point of the rack is denoted as y, and the elevation height of the base center of mass is denoted as z. The arc length traversed by the gear during the time interval $\Delta t$ is denoted as $\Delta d$. The pose information for the first three stages is recorded to facilitate the calculation of poses in the subsequent stage, with the pitch angles designated as ${\theta }_1,{\theta }_2,{\theta }_3$ and the positions as $y_1,z_1;y_2,z_2;y_3,z_3$.
• Subsequently, the front drive gear enters the curved track. Due to constraints in mechanical design, the rear drive gear remains on the horizontal track. At this point, considering the continuity of motion and neglecting the minor distance discrepancies caused by the bending of the rack, let the diameter of the drive gear be denoted as r and the spacing between the two drive gears as L. The designed curved track is effectively a segment of a circle with a radius denoted as R. Thus, denoting the variable as $D=R-{\displaystyle \frac{r}{2}}$, the following relationship can be established:

  $$\begin{array}{c}\hfill \theta =arcsin\left({\displaystyle \frac{D-Dcos\frac{\Delta d}{R}}{L}}\right)\end{array}$$

  (18)

  and it can be concluded that the current pose is

  $$\begin{array}{c}\hfill \begin{array}{cc}\hfill y& =y_1+\Delta d-{\displaystyle \frac{L\left(cos{\theta }_1-cos\theta \right)}{2}},\hfill \\ \hfill z& ={\displaystyle \frac{Lsin\theta +r}{2}},\hfill \\ \hfill {\theta }_2& =\theta ,y_2=y,z_2=z\hfill \end{array}\end{array}$$

  (19)
• As the front gear traverses the curved track, the rear gear remains on the horizontal track, thereby yielding the following result:

  $$\begin{array}{c}\hfill \theta =arccos\left({\displaystyle \frac{Lcos\left({\theta }_2\right)-\Delta d}{L}}\right)\end{array}$$

  (20)

  and it can be concluded that the current pose is

  $$\begin{array}{c}\hfill \begin{array}{cc}\hfill y& =y_2+\Delta d-{\displaystyle \frac{L\left(cos{\theta }_2-cos\theta \right)}{2}},\hfill \\ \hfill z& ={\displaystyle \frac{Lsin\theta +r}{2}},\hfill \\ \hfill {\theta }_3& =\theta ,y_3=y,z_3=z\hfill \end{array}\end{array}$$

  (21)
• As the front gear traverses the suspended segment of the track, the rear gear will move onto the curved track. Similarly, by neglecting the minor distance differences caused by the slight deformation of the rack due to bending, the following result can be obtained:

  $$\begin{array}{c}\hfill \theta =arccos\left({\displaystyle \frac{D-Dsin\left(\frac{\Delta d}{R}\right)}{L}}\right)\end{array}$$

  (22)

  and it can be concluded that the current pose is

  $$\begin{array}{c}\hfill \begin{array}{cc}\hfill y& =y_3+{\displaystyle \frac{L\left(cos{\theta }_3-cos\theta \right)}{2}}\hfill \\ \hfill z& =z_3+\Delta d-{\displaystyle \frac{L\left(sin{\theta }_3-sin\theta \right)}{2}}\hfill \end{array}\end{array}$$

  (23)

In the mechanical structure design of this study, the curvature characteristics of the curved track in the bending section require the gears to travel along a curved trajectory, resulting in a longer path compared to the straight horizontal track where the other gear is located. Additionally, the gears need to overcome additional lateral forces caused by the curved track. To avoid dangerous situations such as jamming and base bending when passing through the curved track, we have developed a gear compliance driving control method. The core advantage of this method lies in its ability to allocate different control strategies and adjust the torque of the corresponding driving gears in real time based on the robot’s real-time operating status on the track, effectively addressing interference factors such as interrupted racks and dynamic changes in posture under complex working conditions.

Specifically, based on the calculation of the on-track pose, the control of the front and rear gear motors is decoupled into three primary strategies: a dual-gear position synchronization control mode, a supervisory position control for the front gear, and a supervisory position control for the rear gear. The allocation of strategies and torque control is primarily determined by the following situations:

• When the robot begins to ascend at a level where the front gear can still provide effective driving force, the front gear operates at a constant speed while the rear gear adjusts its torque through an adaptive controller. This adjustment is made in real-time, based on the curvature of the curve and the current speed of the robot, thereby altering the driving current of the rear gear.
• When the front gear of the robot transitions to a suspended state and loses driving force, the rear gear provides the necessary driving force. During this period, the front gear will undergo a process of transitioning from suspension to contact with the rack. The adaptive controller adjusts the driving current of the front gear in real-time based on the distribution of the interruption points in the track and the current speed of the robot to match the driving speed of the rear gear.
• Ultimately, the front gear will continuously operate on a vertical track, while the rear gear will experience a process of transitioning from suspension to contact with the rack. It is essential to monitor the control current of the front driving gear at a constant speed to ensure the rear driving gear matches this current appropriately.
• In the final ascent motion, both driving gears are positioned on the vertical track, and the driving current of the rear gear is adjusted in real-time to match that of the front gear, ensuring coordinated and stable rotation, thereby achieving a steady state of ascent.

The adaptive adjustment of motor torque primarily involves calculating the required torque for the motor. Subsequently, the relationship between the driving torque and driving current is roughly established through calculations. Torque control of the driving motor is then achieved by adjusting the control current of the driving motor. As illustrated in Figure 16, a simple controller has been constructed. In this control process, an initial estimation of the required torque for the driven gear is made based on the current generated when another driving gear operates at a specific position. Afterward, the control torque is mapped to the control current, and a proportional–derivative (PD) controller is designed. Through real-time monitoring and feedback adjustment, the control current of the driven gear can quickly and accurately respond to the dynamic changes required by the system, ensuring that the operational state of the driven gear remains within the optimal control range. Ultimately, the control objective is to ensure that the control torque of the driven gear closely follows the change trend of the control torque of the driving gear, achieving coordinated operation between the two and, thus, facilitating the compliance control of the geared drive system.

4. Experiments and Conclusions

4.1. Experimental Hardware Composition

As illustrated in Figure 17, the overall hardware composition of the robot primarily includes the power system, mechanical arm system, mobile base drive system, and core control system. It is evident that sensors, including the camera, magnetic sensor, and RTK system, are predominantly installed on the frontal aspect of the mobile base. The selected camera has a focal length of 3.24 mm, a field of view of H: 100° and V: 56°, and supports automatic exposure and white balance control. The magnetic sensor features an 8-point signal output, with a length of 19.2 mm and a magnetic point spacing of 10 mm. The RTK positioning system is a full-range RTK system, with a nominal horizontal accuracy of 2 cm and a vertical accuracy of 4 cm. The gear motor’s encoder has a resolution of 17 bits.

4.2. Ground Navigation Experiments

This experiment validates the effectiveness of the proposed multi-sensor fusion navigation control method for mobile base through practical navigation experiments conducted by a robot in open ground areas and in narrow spaces in the near-track area.

The experimental environment contains large signal bases, and the robotic arm mounted on the robot generates significant electromagnetic interference during operation, which substantially affects the localization accuracy of the RTK equipment. In this context, a tracking experiment for the mobile base along a specified circular trajectory with a radius of approximately 2 m was conducted. The circular trajectory possesses geometric characteristics of uniformity and periodicity, enabling a comprehensive assessment of the localization accuracy of the mobile base during continuous variation, as depicted in Figure 18. In the experiment, a comparative analysis was performed by utilizing RTK alone and by fusion of RTK with odometry information to evaluate their localization and navigation performance. Figure 19 presents the localization trajectory results obtained from the different methods. It is evident that the trajectory using the fused localization closely aligns with the reference trajectory, whereas the trajectory obtained using RTK separately exhibits significant deviations at certain points, leading to localization drift. Furthermore, due to the limited localization frequency of the RTK equipment, the pose estimation results can exhibit abrupt changes, resulting in increased instability in localization and path planning. Conversely, the fused approach, which integrates continuous odometry information, yields relatively smooth pose estimation results, minimizing the occurrence of abrupt changes.

The second experimental scenario takes place in a narrow area prior to entering the track, as shown in Figure 20. Due to the strong shielding and reflective properties of iron-based steel plates on electromagnetic waves, the localization performance of the RTK device in this environment is significantly less stable compared to typical ground environments. Additionally, the coefficient of friction between the robot’s differential wheels and the steel plate is substantially reduced, leading to an increase in odometry errors. The experiment compared two different localization fusion methods: the fusion of RTK and odometry, and the fusion of RTK, odometry, vision, and magnetic information. The experimental results are presented in Figure 21. In the RTK and odometry fusion approach, limited by the strong interference in the narrow area, the pose error can reach up to 6 cm. However, when vision and magnetic information are further integrated, they provide additional environmental perception dimensions for the robot, effectively compensating for the shortcomings of RTK and odometry. Even when the RTK signal is subjected to strong interference, the system can still assist the robot in achieving precise localization through feature matching and tracking. Experimental validation confirmed that this method has a maximum error of only 3 cm, demonstrating greater adaptability and resilience to interference.

Additionally, the effectiveness of using vision or magnetic information independently within the fusion scheme was verified. Experimental results indicate that if vision information is discarded, the limitations in pose accuracy of both the RTK and odometry, compounded by the restricted detection range of the magnetic sensor, are significant. Specifically, if the robot’s poses deviate, and it does not return to the detection range of the magnetic sensor, the robot’s deviation will progressively increase. Conversely, if magnetic information is not utilized, the reliance on a continuously placed cooperative marker on the ground may lead to localization failure if one of the markers becomes damaged or obscured by strong shadows caused by sunlight affecting the camera’s performance.

Based on these experiments, it can be concluded that the multi-sensor fusion localization and navigation method proposed in this paper demonstrates high localization accuracy and reliability in practical applications. Figure 22 illustrates the complete ground navigation planning results for a robot conducted in an outdoor environment. It can be observed that the robot accurately follows the designated path, particularly in areas requiring higher precision and stability, thereby meeting the demands of the task.

4.3. Gear-Track Climbing Experiments

In the laboratory, we constructed a simulated track system that utilizes a continuous rack as its main structural component. To test the performance of the independently designed mechanical gear-track, we selected a mobile base to execute climbing actions on the mechanical gear-track. This approach allows us to accurately verify the track’s passability during actual operations and its stability during climbing. As illustrated in Figure 23, we monitored the mobile base’s trajectory, velocity variations, and power output parameters in real time during the experiment. The results indicate that the mobile base can smoothly and reliably transition from a horizontal motion state to a vertical climbing state using the designed mechanical gear-track. Furthermore, the material strength and motion precision of the designed mechanical track and climbing gear assembly meet the climbing requirements, demonstrating good stability and adaptability throughout the entire process.

Furthermore, we specifically focus on the motion performance of the proposed track localization method and the gear compliance control method in the curved track sections. Therefore, we conducted experiments on a real outdoor track, as shown in Figure 24. Initially, we compared the height of the base measured in actual stages with calculated values. Subsequently, we observed the attitude adjustment of the mobile base during cornering and monitored the current variation of the two driving motors in real-time. Due to the use of conventional control methods, the mobile base was unable to navigate the curved track smoothly, resulting in potential irreversible mechanical damage. To ensure experimental safety, we conducted comparative experiments during the vertical ascent phase of the mobile base, employing both the gear compliance drive control method and a method without it. This allowed us to analyze the practical effects of the proposed cornering track compliance control strategy, with the experimental results presented in Figure 25.

Figure 25a presents the experimental results using the compliance control method, while Figure 25b shows the experimental results without using this method. The experimental data demonstrate significant differences. In the results without the use of compliance control for the gear, there are objective errors due to the mechanical characteristics of the gear engaging with the mechanical track and the control of the gear motor. As a result, there are slight positional deviations between the front and rear driving gears of the base when climbing the track. Furthermore, since the two gears are connected by a rigid body and the two driving motors do not have the ability to adjust torque, these small positional differences will be significantly amplified into differences in driving torque. This ultimately leads to a continuous increase in the control current difference between the two gear driving motors, severely affecting the climbing stability and reliability of the base. However, by using the compliance control method for the gear, it is possible to make the control current of the compliance gear approach that of the active gear to a certain extent, effectively ensuring a dynamic balance of torque between the two driving wheels and providing a more stable performance for the climbing process of the base.

4.4. Complete Tasks’ Experiments

After completing the ground navigation experiments and gear-track climbing experiments, we conducted a comprehensive task flow experiment outdoors, in which the robot transitioned from ground initiation to climbing to a high altitude for operational tasks, which is shown in Figure 26. Throughout this process, real-time monitoring and data collection of the robot’s operational status in a complex and variable outdoor environment allowed us to effectively validate the proposed system’s exceptional capabilities in the field of work-at-height operations. These capabilities include precise positioning and stable operational ability in complex outdoor environments, adaptive adjustment capabilities in response to complex meteorological conditions and external disturbances, as well as sustained stable operational performance during prolonged high-intensity tasks. Figure 1 illustrates the robot we designed performing actual tasks at a high altitude.

5. Conclusions

This paper presents a novel mobile manipulator base and a corresponding mechanical gear-track, equipped with a flexible operating mechanical arm. It innovatively proposes a localization and navigation control method based on multi-sensor fusion, as well as a compliance control method for gear-climbing, addressing the associated technical challenges. Ultimately, the effectiveness of the proposed system is validated through a series of diverse experiments. The successful validation of these capabilities underscores the substantial application value of the system across various scenarios, including the cleaning of building exteriors, vertical material handling in industrial automated production lines, and exploration of work-at-height terrains. This system effectively enhances the operational range and adaptability of robots, establishing a robust foundation for their future widespread application and promotion within related industries. It is anticipated to significantly improve operational efficiency, reduce labor costs, and ensure operational safety, thereby fostering technological innovation and industrial advancement in the domain of working at heights.