Next Article in Journal
Calorimetric Behaviour of Electric Cables
Previous Article in Journal
Heat Transfer in Highly Turbulent Separated Flows: A Review
Previous Article in Special Issue
Learning Feedforward Control Using Multiagent Control Approach for Motion Control Systems
Open AccessArticle

Virtual State Feedback Reference Tuning and Value Iteration Reinforcement Learning for Unknown Observable Systems Control

Department of Automation and Applied Informatics, Politehnica University of Timisoara, 300223 Timisoara, Romania
*
Author to whom correspondence should be addressed.
Academic Editor: Eduard Petlenkov
Energies 2021, 14(4), 1006; https://doi.org/10.3390/en14041006
Received: 27 December 2020 / Revised: 10 February 2021 / Accepted: 12 February 2021 / Published: 15 February 2021
(This article belongs to the Special Issue Intelligent Control for Future Systems)
In this paper, a novel Virtual State-feedback Reference Feedback Tuning (VSFRT) and Approximate Iterative Value Iteration Reinforcement Learning (AI-VIRL) are applied for learning linear reference model output (LRMO) tracking control of observable systems with unknown dynamics. For the observable system, a new state representation in terms of input/output (IO) data is derived. Consequently, the Virtual State Feedback Tuning (VRFT)-based solution is redefined to accommodate virtual state feedback control, leading to an original stability-certified Virtual State-Feedback Reference Tuning (VSFRT) concept. Both VSFRT and AI-VIRL use neural networks controllers. We find that AI-VIRL is significantly more computationally demanding and more sensitive to the exploration settings, while leading to inferior LRMO tracking performance when compared to VSFRT. It is not helped either by transfer learning the VSFRT control as initialization for AI-VIRL. State dimensionality reduction using machine learning techniques such as principal component analysis and autoencoders does not improve on the best learned tracking performance however it trades off the learning complexity. Surprisingly, unlike AI-VIRL, the VSFRT control is one-shot (non-iterative) and learns stabilizing controllers even in poorly, open-loop explored environments, proving to be superior in learning LRMO tracking control. Validation on two nonlinear coupled multivariable complex systems serves as a comprehensive case study. View Full-Text
Keywords: learning control; reference model output tracking; neural networks; state-feedback; reinforcement learning; observability; virtual state-feedback reference tuning; robotic systems; dimensionality reduction; transfer learning learning control; reference model output tracking; neural networks; state-feedback; reinforcement learning; observability; virtual state-feedback reference tuning; robotic systems; dimensionality reduction; transfer learning
Show Figures

Figure 1

MDPI and ACS Style

Radac, M.-B.; Borlea, A.-I. Virtual State Feedback Reference Tuning and Value Iteration Reinforcement Learning for Unknown Observable Systems Control. Energies 2021, 14, 1006. https://doi.org/10.3390/en14041006

AMA Style

Radac M-B, Borlea A-I. Virtual State Feedback Reference Tuning and Value Iteration Reinforcement Learning for Unknown Observable Systems Control. Energies. 2021; 14(4):1006. https://doi.org/10.3390/en14041006

Chicago/Turabian Style

Radac, Mircea-Bogdan; Borlea, Anamaria-Ioana. 2021. "Virtual State Feedback Reference Tuning and Value Iteration Reinforcement Learning for Unknown Observable Systems Control" Energies 14, no. 4: 1006. https://doi.org/10.3390/en14041006

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop