Neural Network Mapping of Industrial Robots’ Task Times for Real-Time Process Optimization
Abstract
:1. Introduction
- the scheduling, the optimal intercept computation, and the trajectory execution time minimization are treated in isolation, without considering the interdependencies between them;
- the scheduling algorithms often do not consider accurately that different tasks are associated with different execution times;
- the cycle or task time maps are developed without a focus on the optimal (as opposed to feasible) times;
- the cycle time maps or tables cover only a portion of the useful workspace of the manipulator and appear tailored to a specific application;
- the methods employed for the generation of the cycle time maps are known to scale poorly to problems featuring a high number of degrees of freedom.
- the development of a systematic architecture for the representation of the task time maps as neural networks;
- the illustration of a computational procedure for the generation of the dataset needed to train the task time models;
- the demonstration of the use of the task time maps to solve the optimal intercept problem in an industrial application case.
2. Methods
2.1. Problem Statement and Solution Outline
- the kinematic and dynamic properties of the robot;
- the parameters of the actuators and transmission systems;
- the behavior of the Geometric Planning Module;
- the behavior of the Task Time Optimizer.
- the parametrization of the tasks to be executed by the robot using the GTP;
- the offline generation of a dataset of optimal execution times using the TTO;
- the generation, using deep learning models, of an accurate and efficiently queryable map of the optimal task times over the entire workspace of the robot.
2.2. Geometric Trajectory Planning
2.3. Task Time Optimization
- the actual performances of the actuation drives and of the transmissions;
- the model of the mechanical dynamics of the manipulator;
- the properties of the payload.
2.4. Task Times Map Based on Neural Networks
- sample the position and velocity boundary conditions , , , and ;
- for each set of boundary conditions, generate a geometric path using the GTP;
- for each motion path, compute the optimal execution time using the TTO and store explicitly the association between the inputs and .
- ability to sample the task space in an unstructured way;
- lower persistent memory usage (which is used not for the samples directly but rather for the parameters of the neural network);
- more straightforward adaptability to different kinds of manipulators;
- tuneability of the internal architecture for achieving the desired performances;
- application-independent software interface and generation procedure.
3. Case Study Description
- length of the proximal links mm;
- length of the distal links mm;
- frame length mm;
- mass of the proximal links kg;
- mass of the distal links kg;
- barycentric inertia of the proximal links = 5.22 × 10−2 kg m2;
- barycentric inertia of the distal links = 5.22 × 10−2 kg m2;
- mass of the screw-spline and of the end-effector kg;
- screw-spline pitch mm;
- rotational inertia of the end-effector = 6.40 × 10−6 kg m2;
- rotational inertia of the transmission system actuating the screw-spline helical joint = 1.20 × 10−6 kg m2;
- rotational inertia of the transmission system actuating the screw-spline prismatic joint = 1.20 × 10−6 kg m2.
Geometric Planning for the 5R Robot
- stationary pick and place tasks, where the initial and final velocities are null;
- on-the-fly pick and place tasks, such as those represented in Figure 1, where the initial and final velocities are non-null and aligned to the x-axis.
- an intercept motion (a), which happens in a slightly elevated plane and which terminates with an x-y position, a rotation and a velocity that match those of the pick and place target; its terminal position can be found, as will be shown, by solving the optimal intercept problem;
- a descent motion (b) through which the gripper approaches the target from above;
- a grasping/release motion (c) during which the gripping device operates to grasp or release the item;
- an ascent motion (d) through which the gripper attains the proper vertical clearance.
- assists the acceleration phase of the descent motion (b);
- opposes the deceleration phase of the descent motion (b);
- opposes the acceleration phase of the ascent motion (d);
- assists the deceleration phase of the ascent motion (d).
4. Computational Results and Discussion
4.1. Software and Hardware Setup
- an abstract class representing a generic n-DOF manipulator, from which specific implementations (among which the one relative to the 5R robot) are derived;
- an abstract class representing a generic TTO, from which specific implementations are derived according to the different optimization methods;
- an abstract class representing a generic n-DOF geometric path, with its derived implementations;
- a set of routines implementing the GTP for the 5R robot;
- a utility class dedicated to the multithreaded generation of the dataset.
- a rotation range for the end-effector equal to ;
- a range for the initial and final velocities equal to [0.05 m s−1, 0.5 m s−1];
- loss function: MSE (Mean Squared Error)
- training algorithm: SGD (Stochastic Gradient Descent)
- batch size: 2048 data points.
- , the average execution time of the entire motion planning and optimization pipeline for a single trajectory;
- , the average evaluation time of the task time map on a single CPU core;
- , the average evaluation time of the task time map on the GPU.
- CPU: Intel® Core™(Santa Clara, CA, USA) i7-8750H processor (9 cache, up to , 6 processors);
- GPU: Nvidia® (Santa Clara, CA, USA) GeForce® (Santa Clara, CA, USA) GTX 1050, 4 , GDDR5;
- RAM: 16 DDR4 SO-DIMM, 2400 .
4.2. Stationary Pick and Place Task Time Map
- on a single CPU core, using the Python-3 interpreter;
- on the GPU, using the Python-3 interpreter;
- on the CPU, using the C++ implementation of the GTP and TTO.
4.3. On-the-Fly Pick and Place Task Time Map
- ms on a single CPU core;
- on the GPU;
- ms on the CPU, using the C++ implementation of the GTP and TTO.
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Daoud, S.; Chehade, H.; Yalaoui, F.; Amodeo, L. Solving a robotic assembly line balancing problem using efficient hybrid methods. J. Heuristics 2014, 20, 235–259. [Google Scholar] [CrossRef]
- Humbert, G.; Pham, M.T.; Brun, X.; Guillemot, M.; Noterman, D. Comparative analysis of pick & place strategies for a multi-robot application. In Proceedings of the 2015 IEEE 20th Conference on Emerging Technologies & Factory Automation (ETFA), Luxembourg, 8–11 September 2015; pp. 1–8. [Google Scholar]
- Humbert, G.; Brun, X.; Pham, M.T.; Guillemot, M.; Noterman, D. Development of a methodology to improve the performance of multi-robot pick & place applications: From simulation to experimentation. In Proceedings of the 2016 IEEE International Conference on Industrial Technology (ICIT), Taipei, Taiwan, 14–17 March 2016; pp. 1960–1965. [Google Scholar]
- Ferrari, G.; Ferrarini, L.; Petretti, A.; Pizzi, E. Modeling and design of an optimal line manager of a packaging system with MILP. In Proceedings of the IECON 2015—41st Annual Conference of the IEEE Industrial Electronics Society, Yokohama, Japan, 9–12 November 2015; pp. 005050–005056. [Google Scholar]
- Pizzi, E.; Bouchrit, A.; Petretti, A.; Ferrarini, L. Performance improvement for online schedulers for packaging systems. In Proceedings of the 2016 IEEE International Conference on Automation Science and Engineering (CASE), Fort Worth, TX, USA, 21–24 August 2016; pp. 1243–1248. [Google Scholar]
- Wang, P.; Ma, H.; Zhang, Y.; Cao, X.; Wu, X.; Wei, X.; Zhou, W. Trajectory Planning for Coal Gangue Sorting Robot Tracking Fast-Mass Target under Multiple Constraints. Sensors 2023, 23, 4412. [Google Scholar] [CrossRef] [PubMed]
- Wilson, D.B.; Soto, M.A.T.; Goktogan, A.H.; Sukkarieh, S. Real-time rendezvous point selection for a nonholonomic vehicle. Proceedings of 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, 6–10 May 2013; pp. 3941–3946. [Google Scholar] [CrossRef]
- Shin, I.S.; Nam, S.H.; Roberts, R.; Moon, S. A minimum-time algorithm for intercepting an object on a conveyor belt. Ind. Robot. 2009, 36, 127–137. [Google Scholar] [CrossRef]
- Croft, E.A.; Fenton, R.G.; Benhabib, B. An on-line robot planning strategy for target interception. J. Robot. Syst. 1998, 15, 97–114. [Google Scholar] [CrossRef]
- Jasour, A.M.; Farrokhi, M. Adaptive neuro-predictive control for redundant robot manipulators in presence of static and dynamic obstacles: A Lyapunov-based approach. Int. J. Adapt. Control Signal Process. 2014, 28, 386–411. [Google Scholar] [CrossRef]
- Han, S.D.; Feng, S.W.; Yu, J. Toward fast and optimal robotic pick-and-place on a moving conveyor. IEEE Robot. Autom. Lett. 2019, 5, 446–453. [Google Scholar] [CrossRef]
- Kröger, T. Opening the door to new sensor-based robot applications—The Reflexxes Motion Libraries. In Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China, 9–13 May 2011; pp. 1–4. [Google Scholar]
- Mattone, R.; Divona, M.; Wolf, A. Sorting of items on a moving conveyor belt. Part 2: Performance evaluation and optimization of pick-and-place operations. Robot.-Comput.-Integr. Manuf. 2000, 16, 81–90. [Google Scholar] [CrossRef]
- Boschetti, G. A picking strategy for circular conveyor tracking. J. Intell. Robot. Syst. 2016, 81, 241–255. [Google Scholar] [CrossRef]
- Zhou, M.; Jiang, R. Optimal Strategy for Pick-and-Place System with two Robots. J. Phys. 2022, 2216, 012022. [Google Scholar] [CrossRef]
- Scalera, L.; Boscariol, P.; Carabin, G.; Vidoni, R.; Gasparetto, A. Enhancing energy efficiency of a 4-DOF parallel robot through task-related analysis. Machines 2020, 8, 10. [Google Scholar] [CrossRef]
- Bobrow, J.E.; Dubowsky, S.; Gibson, J.S. Time-optimal control of robotic manipulators along specified paths. Int. J. Robot. Res. 1985, 4, 3–17. [Google Scholar] [CrossRef]
- Pham, Q.C.; Stasse, O. Time-optimal path parameterization for redundantly actuated robots: A numerical integration approach. IEEE/ASME Trans. Mechatronics 2015, 20, 3257–3263. [Google Scholar] [CrossRef]
- Pham, H.; Pham, Q.C. Time-optimal path tracking via reachability analysis. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21–25 May 2018; pp. 3007–3012. [Google Scholar]
- Verscheure, D.; Demeulenäre, B.; Swevers, J.; De Schutter, J.; Diehl, M. Practical time-optimal trajectory planning for robots: A convex optimization approach. IEEE Trans. Autom. Control 2008, 53, 28. [Google Scholar]
- Pham, H.; Pham, Q.C. On the structure of the time-optimal path parameterization problem with third-order constraints. In Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, 29 May–3 June 2017; pp. 679–686. [Google Scholar]
- Fan, W.; Gao, X.S.; Lee, C.H.; Zhang, K.; Zhang, Q. Time-optimal interpolation for five-axis CNC machining along parametric tool path based on linear programming. Int. J. Adv. Manuf. Technol. 2013, 69, 1373–1388. [Google Scholar] [CrossRef]
- Zhang, Q.; Li, S.R.; Gao, X.S. Practical smooth minimum time trajectory planning for path following robotic manipulators. In Proceedings of the 2013 American Control Conference, Washington, DC, USA, 17–19 June 2013; pp. 2778–2783. [Google Scholar]
- Gasparetto, A.; Boscariol, P.; Lanzutti, A.; Vidoni, R. Path planning and trajectory planning algorithms: A general overview. Motion Oper. Plan. Robot. Syst. Backgr. Pract. Approaches 2015, 2015, 3–27. [Google Scholar]
- Braghin, F.; Cheli, F.; Melzi, S.; Sabbioni, E. Race driver model. Comput. Struct. 2008, 86, 1503–1516. [Google Scholar] [CrossRef]
- Sabelhaus, D.; Röben, F.; zu Helligen, L.P.M.; Lammers, P.S. Using continuous-curvature paths to generate feasible headland turn manoeuvres. Biosyst. Eng. 2013, 116, 399–409. [Google Scholar] [CrossRef]
- Zhao, H.; Zhu, L.; Ding, H. A real-time look-ahead interpolation methodology with curvature-continuous B-spline transition scheme for CNC machining of short line segments. Int. J. Mach. Tools Manuf. 2013, 65, 88–98. [Google Scholar] [CrossRef]
- Tang, P.Y.; Lin, M.T.; Tsai, M.S.; Cheng, C.C. Toolpath interpolation with novel corner smoothing technique. Robot. Comput. Integr. Manuf. 2022, 78, 102388. [Google Scholar] [CrossRef]
- Chen, G.; Luo, N.; Liu, D.; Zhao, Z.; Liang, C. Path planning for manipulators based on an improved probabilistic roadmap method. Robot. Comput. Integr. Manuf. 2021, 72, 102196. [Google Scholar] [CrossRef]
- Righettini, P.; Strada, R.; Zappa, B.; Lorenzi, V. Experimental set-up for the investigation of transmissions effects on the dynamic performances of a linear PKM. In Proceedings of the Advances in Mechanism and Machine Science: Proceedings of the 15th IFToMM World Congress on Mechanism and Machine Science 15, Krakow, Poland, 30 June–4 July 2019; pp. 2511–2520. [Google Scholar]
- Righettini, P.; Strada, R.; Cortinovis, F. Modal kinematic analysis of a parallel kinematic robot with low-stiffness transmissions. Robotics 2021, 10, 132. [Google Scholar] [CrossRef]
- Righettini, P.; Strada, R.; Cortinovis, F. General Procedure for Servo-Axis Design in Multi-Degree-of-Freedom Machinery Subject to Mixed Loads. Machines 2022, 10, 454. [Google Scholar] [CrossRef]
- Bourbonnais, F.; Bigras, P.; Bonev, I.A. Minimum-time trajectory planning and control of a pick-and-place five-bar parallel robot. IEEE/ASME Trans. Mechatronics 2014, 20, 740–749. [Google Scholar] [CrossRef]
- Yu, H. Modeling and control of hybrid machine systems—A five-bar mechanism case. Int. J. Autom. Comput. 2006, 3, 235–243. [Google Scholar] [CrossRef]
- Stellato, B.; Banjac, G.; Goulart, P.; Bemporad, A.; Boyd, S. OSQP: An operator splitting solver for quadratic programs. Math. Program. Comput. 2020, 12, 637–672. [Google Scholar] [CrossRef]
- Takahasi, H.; Mori, M. Double exponential formulas for numerical integration. Publ. Res. Inst. Math. Sci. 1974, 9, 721–741. [Google Scholar] [CrossRef]
Servoaxis 1 | Servoaxis 2 | Servoaxis 3 | Servoaxis 4 | |
---|---|---|---|---|
0.7 | 0.7 | 0.36 | 0.36 | |
1.4 | 1.4 | 0.72 | 0.72 | |
500 | 500 | 500 | 500 | |
64 | 64 | 10 | 10 |
Task Type | |||||
---|---|---|---|---|---|
Stationary Pick and Place | 81 | 6 | 49,419 | 610.11 | 8236.5 |
On-the-fly Pick and Place | 2310 | 73 | 56,985 | 24.7 | 780.6 |
Layer 1 | Layer 2 | Layer 3 | |
---|---|---|---|
Type | Fully connected | Fully connected | Fully connected |
Activation | ReLU | ReLU | Linear |
Dropout probability | 0.20 | 0.20 | 0.20 |
5 | 2048 | 2048 | |
2048 | 2048 | 1 |
Layer 1 | Layer 2 | Layer 3 | Layer 4 | |
---|---|---|---|---|
Type | Fully connected | Fully connected | Fully connected | Fully connected |
Activation | ReLU | ReLU | ReLU | Linear |
Dropout probability | 0.05 | 0.05 | 0.05 | 0.05 |
5 | 8192 | 8192 | 8129 | |
8192 | 8192 | 8192 | 1 |
Task Configuration Data | Task 1 | Task 2 | Task 3 | Task 4 |
---|---|---|---|---|
, [mm] | −300 | 175 | −300 | 175 |
, [mm] | 150 | 300 | 150 | 300 |
, [] | 0.0 | 0.0 | 0.0 | 0.0 |
, [m s−1] | 0.2 | 0.2 | 0.2 | 0.2 |
, [mm] | −150 | −150 | 150 | 150 |
, [mm] | 400 | 400 | 175 | 175 |
, [] | 0.0 | 0.0 | 0.0 | 0.0 |
, [m s−1] | 0.2 | 0.2 | 0.2 | 0.2 |
evaluations | 15 | 15 | 17 | 12 |
evaluations | 14 | 15 | 16 | 11 |
Solution time with GTP-TTO, [ms] | 869 | 869 | 985 | 695 |
Solution time with task time map (CPU), [ms] | 32 | 34 | 37 | 25 |
Solution time with task time map (GPU), [ms] | 1.02 | 1.09 | 1.17 | 0.80 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Righettini, P.; Strada, R.; Cortinovis, F. Neural Network Mapping of Industrial Robots’ Task Times for Real-Time Process Optimization. Robotics 2023, 12, 143. https://doi.org/10.3390/robotics12050143
Righettini P, Strada R, Cortinovis F. Neural Network Mapping of Industrial Robots’ Task Times for Real-Time Process Optimization. Robotics. 2023; 12(5):143. https://doi.org/10.3390/robotics12050143
Chicago/Turabian StyleRighettini, Paolo, Roberto Strada, and Filippo Cortinovis. 2023. "Neural Network Mapping of Industrial Robots’ Task Times for Real-Time Process Optimization" Robotics 12, no. 5: 143. https://doi.org/10.3390/robotics12050143