Modeling, Simulation, and Control of a Rotary Inverted Pendulum: A Reinforcement Learning-Based Control Approach
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe paper proposes a new reinforcement learning based modeling, simulation and control of a rotary inverted pendulum. The study is interesting but needs to be significantly revision before publication can be considered.
Give more explanation of Eq.1 in terms of model treatment. Discuss your reason why treating models like that.
It seems that the performance is only tested on LQ and PID, if possible, can MPC, for example， be supplied for simulation?
Introduction is not comprehensive. In the introduction section, please discuss and refer to important reinforcement learning based control methods, including Dois as: 10.1016/j.compchemeng.2024.108883，10.1021/acs.iecr.3c01789，10.1016/j.jprocont.2022.05.006, etc.
Clarify the parameters choice of the control method proposed in this article.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsManuscript ID: modelling-3257534
In this paper, the authors studied the modeling, simulation, and control of a rotary inverted pendulum. There are minor aspects to be improved for making the paper suitable for publication:
1. Can you explain the main advantages and novelty of the results proposed in this draft as compared to the existing ones?
2. The introduction section highlights the importance of rational modeling. The subsequent study also emphasizes the novelty and rationality of the constructed models. Consistency in the descriptions before and after is important.
3. Streamline the second section of the initial preparation to avoid unnecessary descriptions. It adequately depicts assumptions of modeling and theory.
4. Figures should be placed after they are mentioned in the text. In your manuscript, Figures are illustrated before it is mentioned in the text.
5. There are some typos and grammatical errors. The authors should check carefully and correct seriously.
6. It is recommended that the notation part be provided at the beginning of the preliminaries.
7. What are the main difficulties brought by the reinforcement learning methods in the rotary inverted pendulum?
8. How to select the value of the designed parameter such as $\eta$, $Q$ and $R$.
Comments on the Quality of English LanguageThere are some typos and grammatical errors. The authors should check carefully and correct seriously.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsA double approach, based on Multibody and Euler-Lagrange methods, for modelling a rotary inverted pendulum have been presented in this work coupled with a reinforcement learning based control system. Specifically, the rotary inverted pendulum is controlled by exploiting the reinforcement learning to improve the convergence of the control error to zero in different configurations. The paper is written in a Comprehensive way, and it is recommended for publication after minor revisions:
1) The sliding mode adopted in Section 5.4 is dependent by the dynamics of the rotary inverted pendulum especially during the reaching mode. You should describe how to find the equivalent control term of the sliding mode control depedent by the dynamics or, if it is possible, justify how the equivalent control can be neglected and, therefore, you can choose only the switch control term described in Equation (33);
2) The FL-RL control approach could be compared with another strategy based on the reinforcement learning, such as a simple proportional control coupled with the RL for improving the error convergence to zero. By this type of comparison, the results are totally justified in terms of adoption of a nonlinear control approach such as the FL control coupled with the RL.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 4 Report
Comments and Suggestions for AuthorsThe authors develop a comprehensive mathematical model of the rotary inverted pendulum (RIP), validate it through simulations, and implement various control techniques, including linear, nonlinear, and AI-based controllers. They propose a novel hybrid controller that integrates feedback linearization (FL) with reinforcement learning (RL) to compensate for unknown dynamics and enhance robustness. Simulation results demonstrate that the FL-RL approach outperforms conventional control methods, particularly in managing external disturbances. However, the following concerns should be addressed:
The reward function utilized in the RL component effectively encourages stabilization by penalizing deviations from the desired position. How might the implementation of more sophisticated reward structures or adaptive rewards based on system feedback enhance the learning capabilities and overall performance of the reinforcement learning agent in stabilizing deviations from the desired position?
What effects do different reinforcement learning hyperparameters and network architectures have on the performance of the combined RL and feedback linearization controller in stabilizing the rotary inverted pendulum?
How does the effectiveness of the FL-RL controller compare to that of more advanced adaptive controllers, such as model predictive control, in terms of managing disturbances and achieving stabilization?
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThe authors addressed my concerns well and I recommend publication.
