Unified Modulation Matrix-Based Shared Control for Teleoperated Multi-Robot Formation and Obstacle Avoidance
Abstract
1. Introduction
- (1)
- An Intent-Mediated Asymmetric Vortex Modulation (IM-AVM) strategy is proposed to resolve the local minima and saddle-point deadlocks in teleoperation. By mapping operator micro-intentions to flow-field chirality, this approach geometrically eliminates potential field local minima while maintaining high haptic transparency.
- (2)
- A multi-dimensional situational awareness and deadlock escape mechanism is established to address the entrapment issues in complex U-shaped obstacles. By integrating geometric, physical, and interaction conflict indices, the system achieves a seamless transition from assisted deadlock to autonomous escape via a maximum flux search algorithm.
- (3)
- An adaptive formation coordination system based on anisotropic flow-field modulation is developed to resolve the conflict between rigid formation constraints and narrow environmental gaps. This system utilizes a dynamic scaling factor and an adaptive compliance gain to ensure robust, collision-free navigation under lumped uncertainties. The remainder of this paper is organized as follows: In Section 2, the dynamic model of the omnidirectional mobile robot formation and the bilateral teleoperation framework are established. Section 3 elaborates on the design principles of the IM-AVM strategy and the hierarchical controller. In Section 4, rigorous proofs of system stability and convergence are provided based on Lyapunov theory. Section 5 validates the effectiveness of the proposed method through experimental analysis and discussion.
2. System Modeling and Environmental Representation
2.1. Modeling of the Mecanum-Wheeled Omnidirectional Mobile Robot
2.1.1. Kinematic Model
2.1.2. Dynamic Model
2.2. Modeling of the Bilateral Teleoperation System
2.2.1. Kinematic Mapping and Signal Preprocessing
2.2.2. Master-Side Dynamic Model
2.2.3. Velocity Mapping
2.2.4. Communication Channel Model
2.3. Enhanced Anisotropic Environmental Field
2.3.1. Anisotropic Generalized Distance Field
2.3.2. Virtual Linking Mechanism
2.3.3. Vectorized Representation of Environmental Constraints
2.4. Leader-Follower Formation Dynamic Geometric Modeling
2.4.1. Formation Transformation and Relative Pose Description
2.4.2. Kinematic Feedforward and Reference Velocity Generation
3. Shared Control and Formation Obstacle Avoidance Strategy for Teleoperation
3.1. Intent-Mediated Asymmetric Dynamic Shared Control
3.2. Multi-Dimensional Construction of an Enhanced Environmental Entrapment Index
3.2.1. Fusion Geometric Siege
3.2.2. Physical Stagnation
3.2.3. Interaction Conflict
3.3. Composite Deadlock Criterion and Escape Activation
3.3.1. Distance Potential Convolution and Maximum Flux Search
3.3.2. Dynamic Enhancement and State Reset
3.4. Dynamic Arbitration and Hierarchical Execution Architecture
3.5. Adaptive Hybrid Force Feedback Interaction Mechanism
3.5.1. Force Feedback in Assisted Deadlock State
3.5.2. Force Guidance in Escape Takeover State
3.5.3. Predictive Damping Injection for Interaction
3.6. Follower Cooperative Control Based on Flow Field Modulation
3.6.1. Perception-Based Dynamic Formation Scaling
3.6.2. Anisotropic Flow Field Modulation
3.6.3. Adaptive Nonsingular Fast Terminal Sliding Mode Controller (ANFTSMC)
4. System Stability and Convergence Analysis
4.1. Passivity Proof of Leader Interaction
4.2. Proof of Forward Invariance for Obstacle Avoidance Safety
4.3. Finite-Time Convergence Proof of Follower Dynamics
- Sliding Mode Reaching Phase
- 2.
- Sliding Mode Phase
5. Experimental Verification and Results Analysis
5.1. Teleoperation Mapping Evaluation
5.2. Core Algorithm Verification
5.3. Compliant Formation Deformation and Trajectory Adaptation
5.4. Comprehensive Formation and Long-Distance Navigation
5.5. Scalability and Generalization to Multi-Robot Formations ()
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
| IM-AVM | Intent-Mediated Asymmetric Vortex Modulation | Multidisciplinary Digital Publishing Institute |
| ANFTSM | Adaptive Nonsingular Fast Terminal Sliding Mode Controller | Directory of open access journals |
| APF | Artificial Potential Field | |
| PD | Proportional-Derivative | Three letter acronym |
| TD | Tracking Differentiator | Linear dichroism |
References
- Lima, R.; Saha, S.; Vakharia, V.; Vatsal, V.; Das, K. Augmenting Robot Teleoperation with Shared Autonomy via Model Predictive Control. In Proceedings of the 2024 IEEE Conference on Telepresence, Pasadena, CA, USA, 16–17 November 2024; pp. 42–48. [Google Scholar]
- Franchi, A.; Secchi, C.; Son, H.I.; Buelthoff, H.H.; Giordano, P.R. Bilateral Teleoperation of Groups of Mobile Robots with Time-Varying Topology. IEEE Trans. Rob. 2012, 28, 1019–1033. [Google Scholar] [CrossRef] [Scilit]
- Li, D.; Shi, H.; Cai, B.; Bai, X.; Chen, L.; Han, G.; Mi, C. A Review of Technical Advances and Applications of Intelligent Inspection Robots in Structural Health Monitoring. SmartBot 2025, 1, 92–121. [Google Scholar] [CrossRef] [Scilit]
- Shao, X.; Xie, L. A Telerobotic Shared Control Architecture for Learning and Generalizing Skills in Unstructured Environments. IEEE Trans. Autom. Sci. Eng. 2025, 22, 15953–15965. [Google Scholar] [CrossRef] [Scilit]
- Li, P.; Li, W.; Cui, Z.; Yang, P.; Chen, C.; You, B.; Wang, J. UGV Navigation in Complex Environment: An Approach Integrating Security Detection and Obstacle Avoidance Control. IEEE Trans. Intell. Veh. 2025, 10, 4727–4741. [Google Scholar] [CrossRef] [Scilit]
- Saeidi, H.; Wagner, J.R.; Wang, Y. A Mixed-Initiative Haptic Teleoperation Strategy for Mobile Robotic Systems Based on Bidirectional Computational Trust Analysis. IEEE Trans. Rob. 2017, 33, 1500–1507. [Google Scholar] [CrossRef] [Scilit]
- Lu, Z.; Si, W.; Wang, N.; Yang, C. Dynamic Movement Primitives-Based Human Action Prediction and Shared Control for Bilateral Robot Teleoperation. IEEE Trans. Ind. Electron. 2024, 71, 16654–16663. [Google Scholar] [CrossRef] [Scilit]
- Franceschi, P.; Pedrocchi, N.; Beschi, M. Human-Robot Role Arbitration via Differential Game Theory. IEEE Trans. Autom. Sci. Eng. 2024, 21, 5953–5968. [Google Scholar] [CrossRef] [Scilit]
- Piccinelli, N.; Muradore, R. Linearized Virtual Energy Tank for Passivity-Based Bilateral Teleoperation Using Linear MPC. IEEE Trans. Rob. 2025, 41, 2589–2604. [Google Scholar] [CrossRef] [Scilit]
- Wang, Y.; Shan, M.; Yue, Y.; Wang, D. Vision-Based Flexible Leader-Follower Formation Tracking of Multiple Nonholonomic Mobile Robots in Unknown Obstacle Environments. IEEE Trans. Control Syst. Technol. 2020, 28, 1025–1033. [Google Scholar] [CrossRef] [Scilit]
- Lin, J.-L.; Hwang, K.-S.; Wang, Y.-L. A Simple Scheme for Formation Control Based on Weighted Behavior Learning. IEEE Trans. Neural Netw. Learn. Syst. 2014, 25, 1033–1044. [Google Scholar] [CrossRef] [Scilit]
- Chen, X.; Huang, F.; Zhang, Y.; Chen, Z.; Liu, S.; Nie, Y.; Tang, J.; Zhu, S. A Novel Virtual-Structure Formation Control Design for Mobile Robots with Obstacle Avoidance. Appl. Sci. 2020, 10, 5807. [Google Scholar] [CrossRef] [Scilit]
- Sahu, B.K.; Subudhi, B. Flocking Control of Multiple AUVs Based on Fuzzy Potential Functions. IEEE Trans. Fuzzy Syst. 2018, 26, 2539–2551. [Google Scholar] [CrossRef] [Scilit]
- Xiao, H.; Li, Z.; Chen, C.L.P. Formation Control of Leader-Follower Mobile Robots’ Systems Using Model Predictive Control Based on Neural-Dynamic Optimization. IEEE Trans. Ind. Electron. 2016, 63, 5752–5762. [Google Scholar] [CrossRef] [Scilit]
- Ning, B.; Han, Q.-L.; Lu, Q. Fixed-Time Leader-Following Consensus for Multiple Wheeled Mobile Robots. IEEE Trans. Cybern. 2020, 50, 4381–4392. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Zhao, C.; An, L.; Zhang, L. Collisions-Free Distributed Cooperative Tracking Control for Nonlinear MASs with Limited Data Rate. IEEE Trans. Autom. Control 2025, 70, 8250–8257. [Google Scholar] [CrossRef] [Scilit]
- Grushko, S.; Vysocky, A.; Oscadal, P.; Vocetka, M.; Novak, P.; Bobovsky, Z. Improved Mutual Understanding for Human-Robot Collaboration: Combining Human-Aware Motion Planning with Haptic Feedback Devices for Communicating Planned Trajectory. Sensors 2021, 21, 3673. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Tsamis, G.; Chantziaras, G.; Giakoumis, D.; Kostavelis, I.; Kargakos, A.; Tsakiris, A.; Tzovaras, D. Intuitive and Safe Interaction in Multi-User Human Robot Collaboration Environments through Augmented Reality Displays. In Proceedings of the 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), Vancouver, BC, Canada, 8–12 August 2021; pp. 520–526. [Google Scholar]
- Khansari-Zadeh, S.M.; Billard, A. A dynamical system approach to realtime obstacle avoidance. Auton. Rob. 2012, 32, 433–454. [Google Scholar] [CrossRef] [Scilit]
- Yuan, C.; Zhang, W.; Liu, G.; Pan, X.; Liu, X. A Heuristic Rapidly-Exploring Random Trees Method for Manipulator Motion Planning. IEEE Access 2020, 8, 900–910. [Google Scholar] [CrossRef] [Scilit]
- Khansari-Zadeh, S.M.; Billard, A. Learning Stable Nonlinear Dynamical Systems with Gaussian Mixture Models. IEEE Trans. Rob. 2011, 27, 943–957. [Google Scholar] [CrossRef] [Scilit]
- Rank, M.; Shi, Z.; Mueller, H.J.; Hirche, S. Predictive Communication Quality Control in Haptic Teleoperation with Time Delay and Packet Loss. IEEE Trans. Hum.-Mach. Syst. 2016, 46, 581–592. [Google Scholar] [CrossRef] [Scilit]
- Zhang, H.; Song, A.; Li, H.; Chen, D.; Fan, L. Adaptive Finite-Time Control Scheme for Teleoperation with Time-Varying Delay and Uncertainties. IEEE Trans. Syst. Man Cybern. Syst. 2022, 52, 1552–1566. [Google Scholar] [CrossRef] [Scilit]
- Sun, W.; Liu, Y.; Gao, H. Constrained Sampled-Data ARC for a Class of Cascaded Nonlinear Systems with Applications to Motor-Servo Systems. IEEE Trans. Ind. Inf. 2019, 15, 766–776. [Google Scholar] [CrossRef] [Scilit]
- Zhang, H.; Song, A.; Li, H.; Shen, S. Novel Adaptive Finite-Time Control of Teleoperation System with Time-Varying Delays and Input Saturation. IEEE Trans. Cybern. 2021, 51, 3724–3737. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Batista, J.; Souza, D.; Silva, J.; Ramos, K.; Costa, J.; dos Reis, L.; Braga, A. Trajectory Planning Using Artificial Potential Fields with Metaheuristics. IEEE Lat. Am. Trans. 2020, 18, 914–922. [Google Scholar] [CrossRef] [Scilit]
- Gottardi, A.; Tortora, S.; Tosello, E.; Menegatti, E. Shared Control in Robot Teleoperation with Improved Potential Fields. IEEE Trans. Hum.-Mach. Syst. 2022, 52, 410–422. [Google Scholar] [CrossRef] [Scilit]
- Xing, Y.; Lv, C.; Cao, D.; Hang, P. Toward human-vehicle collaboration: Review and perspectives on human-centered collaborative automated driving. Transp. Res. Part C Emerg. Technol. 2021, 128, 103199. [Google Scholar] [CrossRef] [Scilit]
- Yao, Q.; Zheng, Z.; Qi, L.; Yuan, H.; Guo, X.; Zhao, M.; Liu, Z.; Yang, T. Path Planning Method with Improved Artificial Potential Field-A Reinforcement Learning Perspective. IEEE Access 2020, 8, 135513–135523. [Google Scholar] [CrossRef] [Scilit]
- Yang, H.; Fan, X.; Shi, P.; Hua, C. Nonlinear Control for Tracking and Obstacle Avoidance of a Wheeled Mobile Robot with Nonholonomic Constraint. IEEE Trans. Control Syst. Technol. 2016, 24, 741–746. [Google Scholar] [CrossRef] [Scilit]
- Hou, X.; Mahony, R.; Schill, F. Comparative Study of Haptic Interfaces for Bilateral Teleoperation of VTOL Aerial Robots. IEEE Trans. Syst. Man Cybern. Syst. 2016, 46, 1352–1363. [Google Scholar] [CrossRef] [Scilit]
- Li, W.; Ding, L.; Liu, Z.; Wang, W.; Gao, H.; Tavakoli, M. Kinematic Bilateral Teledriving of Wheeled Mobile Robots Coupled with Slippage. IEEE Trans. Ind. Electron. 2017, 64, 2147–2157. [Google Scholar] [CrossRef] [Scilit]
- Rodriguez-Seda, E.J.; Troy, J.J.; Erignac, C.A.; Murray, P.; Stipanovic, D.M.; Spong, M.W. Bilateral Teleoperation of Multiple Mobile Agents: Coordinated Motion and Collision Avoidance. IEEE Trans. Control Syst. Technol. 2010, 18, 984–992. [Google Scholar] [CrossRef] [Scilit]
- Huang, F.; Chen, X.; Chen, Z.; Pan, Y.-J. A Novel SMMS Teleoperation Control Framework for Multiple Mobile Agents with Obstacles Avoidance by Leader Selection. IEEE Trans. Syst. Man Cybern. Syst. 2023, 53, 1517–1529. [Google Scholar] [CrossRef] [Scilit]
- Wen, G.; Lam, J.; Fu, J.; Wang, S. Distributed MPC-Based Robust Collision Avoidance Formation Navigation of Constrained Multiple USVs. IEEE Trans. Intell. Veh. 2024, 9, 1804–1816. [Google Scholar] [CrossRef] [Scilit]
- Feng, X.; Motoi, N. Remote Shared Control with Dynamic Window Approach in Unknown Narrow Environment. IEEE Access 2025, 13, 205764–205773. [Google Scholar] [CrossRef] [Scilit]
- Franchi, A.; Secchi, C.; Ryll, M.; Buelthoff, H.H.; Giordano, P.R. Shared Control Balancing Autonomy and Human Assistance with a Group of Quadrotor UAVs. IEEE Robot. Autom. Mag. 2012, 19, 57–68. [Google Scholar] [CrossRef] [Scilit]
- McGill, S.G.; Rosman, G.; Ort, T.; Pierson, A.; Gilitschenski, I.; Araki, B.; Fletcher, L.; Karaman, S.; Rus, D.; Leonard, J.J. Probabilistic Risk Metrics for Navigating Occluded Intersections. IEEE Rob. Autom. Lett. 2019, 4, 4322–4329. [Google Scholar] [CrossRef] [Scilit]
- Schwarting, W.; Alonso-Mora, J.; Paull, L.; Karaman, S.; Rus, D. Safe Nonlinear Trajectory Generation for Parallel Autonomy with a Dynamic Vehicle Model. IEEE Trans. Intell. Transp. Syst. 2018, 19, 2994–3008. [Google Scholar] [CrossRef] [Scilit]
- Zhang, D.; Tron, R.; Khurshid, R.P. Haptic Feedback Improves Human-Robot Agreement and User Satisfaction in Shared-Autonomy Teleoperation. In Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi′an, China, 30 May–5 June 2021; pp. 3306–3312. [Google Scholar]
- Zhang, H.; Zhang, D.; Wen, J.; Shen, S.; Fu, L.; Liu, W.; Li, E.; Song, A. A Novel FTP plus d Control Approach with Event-Driven Communication for Time-Delayed Bilateral Teleoperation Systems. IEEE Trans. Syst. Man Cybern. Syst. 2026, 56, 428–442. [Google Scholar] [CrossRef] [Scilit]




















| Parameter | Description | Value |
|---|---|---|
| Repulsion field gain | 2.0 | |
| Obstacle influence radius | 1.2 m | |
| Hysteresis threshold | 0.15 | |
| Deadlock composite index threshold | 0.5 | |
| Deadlock confirmation time | 1.5 s | |
| Haptic force feedback gain | 1.0 | |
| Guidance force scaling factor | 0.25 |
| Parameter | Description | Value |
|---|---|---|
| Linear sliding surface gain | 2.0 | |
| Nonlinear sliding surface gain | 1.0 | |
| Fractional power parameters | 6, 5 | |
| Robust switching gain | 1.5 | |
| Boundary layer thickness | 0.05 | |
| Reaching law gains for X and Y axes | 0.1, 0.1 | |
| Proportional gain for formation error | 1.5 | |
| Nominal passable width | 1.6 m |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Share and Cite
Chen, R.; Zhang, Z.; Zhang, Z.; Li, J.; Zhang, H. Unified Modulation Matrix-Based Shared Control for Teleoperated Multi-Robot Formation and Obstacle Avoidance. Sensors 2026, 26, 2387. https://doi.org/10.3390/s26082387
Chen R, Zhang Z, Zhang Z, Li J, Zhang H. Unified Modulation Matrix-Based Shared Control for Teleoperated Multi-Robot Formation and Obstacle Avoidance. Sensors. 2026; 26(8):2387. https://doi.org/10.3390/s26082387
Chicago/Turabian StyleChen, Ruidong, Zhuoyue Zhang, Zhiyao Zhang, Jinyan Li, and Haochen Zhang. 2026. "Unified Modulation Matrix-Based Shared Control for Teleoperated Multi-Robot Formation and Obstacle Avoidance" Sensors 26, no. 8: 2387. https://doi.org/10.3390/s26082387
APA StyleChen, R., Zhang, Z., Zhang, Z., Li, J., & Zhang, H. (2026). Unified Modulation Matrix-Based Shared Control for Teleoperated Multi-Robot Formation and Obstacle Avoidance. Sensors, 26(8), 2387. https://doi.org/10.3390/s26082387

