Multi-Loop Model Reference Proportional Integral Derivative Controls: Design and Performance Evaluations

Due to unpredictable and fluctuating conditions in real-world control system applications, disturbance rejection is a substantial factor in robust control performance. The inherent disturbance rejection capacity of classical closed loop control systems is limited, and an increase in disturbance rejection performance of single-loop control systems affects the set-point control performance. Multi-loop control structures, which involve model reference control loops, can enhance the inherent disturbance rejection capacity of classical control loops without degrading set-point control performance; while the classical closed Proportional Integral Derivative (PID) control loop deals with stability and set-point control, the additional model reference control loop performs disturbance rejection control. This adaptive disturbance rejection, which does not influence set-point control performance, is achieved by selecting reference models as transfer functions of real control systems. This study investigates six types of multi-loop model reference (ML-MR) control structures for PID control loops and presents straightforward design schemes to enhance the disturbance rejection control performance of existing PID control loops. For this purpose, linear and non-linear ML-MR control structures are introduced, and their control performance improvements and certain inherent drawbacks of these structures are discussed. Design examples demonstrate the benefits of the ML-MR control structures for disturbance rejection performance improvement of PID control loops without severely deteriorating their set-point performance.


Introduction
Control systems encounter unpredictable disturbances in real-world control applications. In order to maintain optimal control performance, real control systems should be designed to be robust enough against environmental disturbances, which are mainly unpredictable in character. From a practical point of view, performance degradations of classical control systems are largely caused by their dependence on predetermined system models. In the controller design phase, classical PID controllers are tuned to produce optimal control of a predetermined nominal plant model, which is indeed an idealized representation of a real-world system. The mathematical models of real systems can maintain their validity for predefined operating conditions and ranges, and only observable dynamics of the systems in these range and conditions can be represented by system models. Hence, they omit some inaccessible system dynamics and unknown disturbances, which are not rigorously investigated in the modeling stage. To enhance the representative capability of mathematical modeling, it is well-known that non-linear modeling [1], fractional-order modeling [2], and stochastic modeling [3] methods are of structure come from the fact that all adaptation efforts aim for approximation of plant or process models compared to the reference model. This property is advantageous for time delay systems. Section 6 presents a multi-loop model reference PID-PID control with plant function adaptation (ML-MR PID-PID-PFA). This structure employs a linear adaptation rule for plant function and it can improve control performance for time delay systems.
Illustrative design examples cover control the problems of an automatic voltage regulator model, the liquid level of reboilers, large time delay experimental process models, and Twin-Rotor Multi-input multi-output System (TRMS) experimental setup.

Multi-Loop Model Reference PID-MIT (ML-MR PID-MIT) Control: Analytical Tuning and Nonlinearity
The ML-MR PID-MIT control structure has been proposed to enhance the robust control performance of existing closed loop control systems. Its application has been discussed in several works [25][26][27][28]. This structure is essentially composed of two hierarchical control loops: the inner loop is a classical PID control loop that is a stable and well-tuned control loop, while the outer loop is a adaptation loop that performs the MIT rule. The outer loop is appended to the inner loop to enhance the robust control performance of the inner loop according to regulation of a reference model. Figure 1 shows a block diagram of the ML-MR PID-MIT control structure. It has two error signals per loop: (i) Control error: When the controller of the inner loop is tuned properly, the value of the control error, which is written as e c = u r − y, moves to zero, and thus ensures the control system output settles to the reference input (the signal u r is the modified reference input controlled by the MIT rule and y is the system output). (ii) Model error: Discrepancy between the reference model output and controlled system output is defined as the model error, which is written as e m = y − y m (the signal y m is the output of reference model). The objective of the outer loop is to shape the reference input signal (r) such that the model error converges to zero. The convergence of the model error to zero implies that the inner loop resembles the response of the reference model T m (s). The reference model describes the desired response of the control system that forms the inner loop.
Algorithms 2020, 13, x FOR PEER REVIEW 4 of 29 reference MIT-PID control, which employs the model reference control rule only for plant function adaptation (ML-MR MIT-PID-PFA) in contrast to ML-MR MIT-PID in Section 2. The main advantages of this type of structure come from the fact that all adaptation efforts aim for approximation of plant or process models compared to the reference model. This property is advantageous for time delay systems. Section 6 presents a multi-loop model reference PID-PID control with plant function adaptation (ML-MR PID-PID-PFA). This structure employs a linear adaptation rule for plant function and it can improve control performance for time delay systems.
Illustrative design examples cover control the problems of an automatic voltage regulator model, the liquid level of reboilers, large time delay experimental process models, and Twin-Rotor Multi-input multi-output System (TRMS) experimental setup.

Multi-Loop Model Reference PID-MIT (ML-MR PID-MIT) Control: Analytical Tuning and Nonlinearity
The ML-MR PID-MIT control structure has been proposed to enhance the robust control performance of existing closed loop control systems. Its application has been discussed in several works [25][26][27][28]. This structure is essentially composed of two hierarchical control loops: the inner loop is a classical PID control loop that is a stable and well-tuned control loop, while the outer loop is a adaptation loop that performs the MIT rule. The outer loop is appended to the inner loop to enhance the robust control performance of the inner loop according to regulation of a reference model. Figure 1 shows a block diagram of the ML-MR PID-MIT control structure. It has two error signals per loop: (i) Control error: When the controller of the inner loop is tuned properly, the value of the control error, which is written as y u e r c − = , moves to zero, and thus ensures the control system output settles to the reference input (the signal r u is the modified reference input controlled by the MIT rule and y is the system output).
(ii) Model error: Discrepancy between the reference model output and controlled system output is defined as the model error, which is written as  Let us consider a PID control system as the inner control loop. The PID controllers are three-coefficient standard industrial controllers and optimal tuning of the PID coefficients (k p , k i , Algorithms 2020, 13, 38 5 of 26 k d ) enhances step performance by reducing overshoots and settling time. The transfer function of PID controllers is widely expressed in the parallel form as: For a given plant (process) function G(s), the transfer function of the inner loop can be written by: Previous studies on the ML-MR PID-MIT control structure aim to maintain the initial well-tuned control performance of an existing closed loop control system. For this reason, the reference model is taken as the transfer function of the inner loop; that is, T m (s) = T(s). Accordingly, any performance deterioration in the inner loop leads to an increase in model errors. The recovery of temporal performance deterioration is carried out by shaping the reference input of the inner loop via u r = θr. The adaptation gain θ is determined according to the well-known MIT rule [21], which implements continuous gradient descent optimization of the cost function: The feed-forward MIT rule is expressed in the continuous domain as [18][19][20][21]: where the parameter γ is the learning rate, which is an important parameter for the convergence speed and stability of the gradient descent optimization. Taking Laplace transform of Equation (4), one obtains: To find the sensitivity derivative de m dθ , the model error is written in the form of e m = y − y m = T(s)θr − T m (s)r. Then, the sensitivity derivative of the system is obtained: Here, one can substitute the reference input with r = y m /T m (s) in Equation (6) and rearrange the sensitivity derivative as: By using it in Equation (5), the MIT rule for the update of the adaptation coefficient θ to minimize cost function J is obtained as [25,26,28]: When one configures T m (s) = T(s), the adaptation gain θ is simplified to: which is implemented in the outer loop in Figure 1 [25,26,28]. Application of ML-MR PID-MIT control for the existing control loop was defined as follows [25,26,28]: Algorithms 2020, 13, 38 6 of 26 Step 1: Identify the transfer function model of the existing closed loop PID control system by means of closed loop model identification methods and use this transfer function as a reference model.
Step 2: Enclose the existing closed loop control system via the outer loop. The outer loop employs Equation (9) and the reference model control.
To create a design from scratch for a first order system model in the form of G(s) = K τs+1 e −Ls , an analytical tuning scheme for the ML-MR PID-MIT control structure can be proposed as follows: Step 1: Design a closed loop PID control system according to the Tavakoli-Tavakoli PID tuning rule that can be rearranged for a standard PID controller function in parallel form as follows [29]: This optimal tuning rule implements ITAE and its control performance have been shown previously [29]. The optimal PID controller function of the inner loop can be written for the Tavakoli-Tavakoli optimal PID tuning rule as: Step 2: The reference model is a theoretical model that represents the optimally tuned closed loop control system. Therefore, the transfer function of reference model can be expressed as: In MATLAB and Simulink control system simulations, we implemented this reference model as a numerical model of the PID control loop, as illustrated in the diagram in Figure 2. Step 3: Implement the ML-MR PID-MIT control structure as shown in Figure 2. The learning rate i k = γ can be selected to comply with integral dynamics of the PID control loop.
It is important to note two characteristic features of this structure. Firstly, the performance of the ML-MR PID-MIT control structure is sensitive to the time delay component of the system. As the time delay of the system increases, the adaptation performance of the MIT rule decreases because large delay in the system response easily misleads gradient descent optimization of the MIT rule. Step 3: Implement the ML-MR PID-MIT control structure as shown in Figure 2. The learning rate γ = |k i | can be selected to comply with integral dynamics of the PID control loop.
It is important to note two characteristic features of this structure. Firstly, the performance of the ML-MR PID-MIT control structure is sensitive to the time delay component of the system. As the time delay of the system increases, the adaptation performance of the MIT rule decreases because large delay in the system response easily misleads gradient descent optimization of the MIT rule.  [29]). The system responses in the figure demonstrate that the ML-MR PID-MIT control structure can significantly improve the disturbance rejection performance of classical PID control for zero delay systems. However, when the time delay of the plant increases, improvement in disturbance rejection control decreases.  Secondly, due to multiplication terms u r = θr and y m e m , the ML-MR PID-MIT control structure presents a nonlinear control characteristic. The origin of the nonlinearity in the adaptation gain θ is the nonlinear cost function, which involves the square of model error in Equation (3). This yields a nonlinear term y m e m = y m (y − y m ) = y m y − y 2 m . Therefore, ML-MR PID-MIT control can exhibit some characteristic properties that do not appear in linear control systems. A noteworthy property is that the time response of the PID-MIT control shows amplitude dependence. In the case of reference input r(t) = 0, this results in y m 0; in this case, the outer loop is not functional, so that y m e m = 0 in Equation (9). As the amplitude of the reference input signal r(t) grows, the amplitude of y m increases, and this effect indeed amplifies the model error due to the term of y m e m . Accordingly, this makes the MIT rule of the outer loop more sensitive to model errors and the system responds more effectively in cases where the model mismatches between reference model and control loops. To demonstrate this effect, an illustrative example for control of the plant function G(s) = 2 s+1 is carried out. Figure 4 shows time response ML-MR PID-MIT control structure that is designed for plant function G(s) = 2 s+1 [29]. The learning rate γ is set to −4, and the PID controller is configured to k p = 4, k i = 4, and k d = 0 according to the Tavakoli-Tavakoli PID tuning rule in this simulation. When the amplitude of r(t) moves to one (r(t) = 1), the ML-MR PID-MIT control structure becomes more effective and contributes to the disturbance rejection control performance of the inner loop (classical PID control loop). However, when the amplitude of r(t) moves to zero (r(t) = 0), the ML-MR PID-MIT control structure is not functional, and its disturbance rejection control performance becomes the same as the disturbance rejection performance of the classical PID controller. The overlapping of time responses at set-point zero in the figure apparently confirms this effect.

Multi-Loop Model Reference PID-MIT with Controller Gain Modification (ML-MR PID-MIT-CGM): Adaptive RDR Adjustment for Dynamic Disturbance Rejection and the Linear Adaptation Rule
The disturbance rejection capacity of a closed loop control system can be expressed with the reference to disturbance ratio (RDR) spectrum, which presents a measure for the additive input disturbance rejection performance of a control system for each frequency component [4,30]. The analysis demonstrated that the RDR performance of a closed loop control system depends on the spectral power density of the controller function [4]. The RDR spectrum of the PID controller is expressed in the form of [4]:

Multi-Loop Model Reference PID-MIT with Controller Gain Modification (ML-MR PID-MIT-CGM): Adaptive RDR Adjustment for Dynamic Disturbance Rejection and the Linear Adaptation Rule
The disturbance rejection capacity of a closed loop control system can be expressed with the reference to disturbance ratio (RDR) spectrum, which presents a measure for the additive input disturbance rejection performance of a control system for each frequency component [4,30]. The analysis demonstrated that the RDR performance of a closed loop control system depends on the spectral power density of the controller function [4]. The RDR spectrum of the PID controller is expressed in the form of [4]: The increase of low frequency RDR values improves the disturbance rejection control for low frequency and steady disturbance signals. An increase in high frequency RDR values improves the transient disturbance rejection performance. However, as mentioned previously, the tradeoff between Algorithms 2020, 13, 38 9 of 26 disturbance rejection control and set-point control performance bounds the RDR performance of the closed loop PID control systems [4]. A large increase in RDR values deteriorates the step response performance; this results in very high overshoots and consecutive ripples, which delay the convergence of the controlled system output to the set-point. Further increases in RDR values finally lead to instability of the control systems [4]. Therefore, the unstable state of the closed loop control systems limits the improvement of their RDR performances. A solution for this problem is adaptive adjustment of RDR performance; RDR performance should be increased for disturbance incidents in the inner loop. In order to adjust the RDR performance of the PID control loop adaptively, a variable gain PID controller is proposed by multiplying the PID controller function with the adaptation coefficient θ. This modified PID controller function allows adjustment of the DC gain of the classical PID controller by means of the coefficient θ. The transfer function of this controller can be expressed as: The RDR spectrum of this RDR-adjustable PID controller can be obtained as: This expression of the RDR spectrum apparently shows that the proposed variable gain PID controller C θ (s) allows modification of the RDR spectrum via adaptation gain θ. This enables adaptive adjustment of the disturbance rejection performance of the inner loop by the outer loop. Since the gain θ amplifies the control error inside the inner loop, the value of the adaptation gain θ should be configured to remain positive, and hence is written by: Figure 5 shows the RDR spectrum of the controller C θ (s) for several values of the gain θ. This figure reveals that the proposed variable gain PID controller C θ (s) enables modification of the RDR spectrum via the gain θ, and this asset allows adaptive adjustment of disturbance rejection performance in the control loop. For θ = 1, the C θ (s) controller exhibits disturbance rejection performance equal to the performance of the classical PID controllers. When θ > 1, the disturbance rejection performance of C θ (s) increases, as illustrated in the RDR spectrum in Figure 5. It should be noted that the variable gain PID control may cause instability, particularly for time delay systems, because higher values of the gain θ result in greater amplification of the integral gain, which may cause instability in the control systems as a consequence of accumulation of integral element control errors during the delay period of the system response. This limits the application domain of the proposed method to stable and almost-zero time delay plant functions.
The application steps for ML-MR PID-MIT with controller gain modification (ML-MR PID-MIT-CGM) for existing control loops can be given as follows: Step 1: Obtain a transfer function model of an optimal control loop and use this transfer function as a reference model of the MRC loop (T m (s) = T 1 (s)).
Step 2: Enclose the existing closed loop control system by the outer loop, as shown in Figure 6. Apply the MIT rule for controller gain modification of the closed loop PID control system to adjust the RDR.
To show an application of the ML-MR PID-MIT-CGM control structure, an Automatic Voltage Regulator (AVR) control example is presented in this section. Figure 7 shows a block diagram of the proposed ML-MR PID-MIT-CGM control structure for the AVR control example. The learning rate γ is set to 10. The application steps for ML-MR PID-MIT with controller gain modification (ML-MR PID-MIT-CGM) for existing control loops can be given as follows: Step 1: Obtain a transfer function model of an optimal control loop and use this transfer function as a reference model of the MRC loop ( Step 2: Enclose the existing closed loop control system by the outer loop, as shown in Figure 6. Apply the MIT rule for controller gain modification of the closed loop PID control system to adjust the RDR. To show an application of the ML-MR PID-MIT-CGM control structure, an Automatic Voltage Regulator (AVR) control example is presented in this section. Figure 7 shows a block diagram of the proposed ML-MR PID-MIT-CGM control structure for the AVR control example. The learning rate γ is set to 10.  The application steps for ML-MR PID-MIT with controller gain modification (ML-MR PID-MIT-CGM) for existing control loops can be given as follows: Step 1: Obtain a transfer function model of an optimal control loop and use this transfer function as a reference model of the MRC loop ( Step 2: Enclose the existing closed loop control system by the outer loop, as shown in Figure 6. Apply the MIT rule for controller gain modification of the closed loop PID control system to adjust the RDR. Bendjeghaba et al. implemented an improved harmony search algorithm for optimal tuning of closed loop PID control of an AVR model. The optimal PID controller was obtained by Bendjeghaba et al. in [31] as: We used this optimal PID controller for the AVR model in Figure 7. Figure 8a,b show responses of Bendjeghaba et al.'s optimal PID control loop and ML-MR PID-MIT-CGM control structure for square waveform input disturbances. The figure clearly shows disturbance rejection performance improvements for the ML-MR PID-MIT-CGM control at simulation times of 15 and 35 s, where the set-point is equal to 1. At these instances, the adaptation gain θ of the outer loop actively responds to step-up and step-down disturbances, and it increases the RDR performance of the inner loop by increasing θ, as illustrated in Figure 9. This effect provides a better disturbance rejection control than the classical PID control loop. Due to the amplitude dependence of the MIT rule, contributions of the ML-MR PID-MIT-CGM control to the disturbance rejection performance begin to vanish in the vicinity of the set-point 0. Consequently, the system can respond slightly better for step-up and step-down disturbances at the simulation times of 65 and 85 s compared to the optimal PID loop proposed by Bendjeghaba et al. applications. The nonlinearity of the MIT rule leads to dependence of disturbance rejection performance on the level of the set-point. Another drawback for linear control applications is that the performance of the PID-MIT-CGM control structure is very sensitive to the time delay of plants. A sufficiently large time delay may easily lead to instability in the systems. Using this modified adaptation gain-where Equation (18) replaces the MIT rule in Equation (16)-produces a new ML-MR control structure, which is called multi-loop model reference PID integral (ML-MR PID-I) control. The ML-MR PID-I control structure behaves linearly and improves the disturbance rejection performance independent of the reference input amplitude. Results in Figure 10 demonstrate this effect. Although ML-MR PID-MIT-CGM is very effective around the unity set-point, its performance degrades at set-point 0, although it is slightly better than the optimal PID loop proposed by Bendjeghaba et al. However, ML-MR PID-I control presents the same performance as ML-MR PID-MIT control at the unity set-point and can maintain this performance improvement at set-point 0. Figure 11 validates this effect by showing the adaptive adjustment of the RDR spectrum at set-point levels 1 and 0. As a consequence, for linear control systems with almost zero time delay, the proposed ML-MR PID-I control structure is capable of improving the disturbance rejection performance without deteriorating the set-point performance.    Figure 9b. This results in an increase of PID coefficients, and accordingly an increase in the RDR rates in the spectrum, as shown in Figure 9b. Therefore, the RDR performance of the optimal PID control loop of Bendjeghaba et al. is adaptively modified by adaptation gain θ, while the disturbances affect the system response. This figure verifies an important asset of the proposed ML-MR PID-MIT-CGM control structure: It adaptively adjusts the disturbance rejection control performance and retains the set-point control performance of the optimal PID loops in absence of disturbance incidents. Thus, it overcomes the inherent shortcoming that is caused by the performance tradeoff between set-point control and disturbance rejection control. This property can significantly contribute to the practical robust control performance of classical control loops and can be viewed as an elegant solution for increasing the disturbance rejection performance without deteriorating the set-point performance of control systems. However, there will be a few drawbacks in the PID-MIT structures for linear control system applications. The nonlinearity of the MIT rule leads to dependence of disturbance rejection performance on the level of the set-point. Another drawback for linear control applications is that the performance of the PID-MIT-CGM control structure is very sensitive to the time delay of plants. A sufficiently large time delay may easily lead to instability in the systems.
As mentioned in the previous section, the MIT rule does not behave linearly due to the terms of y m e m . To eliminate this nonlinearity and to make structural contributions to the PID control loop independent of the level of signal amplitudes, one can update the term y m e m as ce m , where the constant c ∈ R can be typically set to 1. This modification implies that the sensitivity derivative is constant and can be shown as de m dθ = c. Hence, Equation (16) is simplified to an integral operator and the adaptation gain can be written as: Using this modified adaptation gain-where Equation (18) replaces the MIT rule in Equation (16) -produces a new ML-MR control structure, which is called multi-loop model reference PID integral (ML-MR PID-I) control. The ML-MR PID-I control structure behaves linearly and improves the disturbance rejection performance independent of the reference input amplitude. Results in Figure 10 demonstrate this effect. Although ML-MR PID-MIT-CGM is very effective around the unity set-point, its performance degrades at set-point 0, although it is slightly better than the optimal PID loop proposed by Bendjeghaba et al. However, ML-MR PID-I control presents the same performance as ML-MR PID-MIT control at the unity set-point and can maintain this performance improvement at set-point 0. Figure 11 validates this effect by showing the adaptive adjustment of the RDR spectrum at set-point levels 1 and 0. As a consequence, for linear control systems with almost zero time delay, the proposed ML-MR PID-I control structure is capable of improving the disturbance rejection performance without deteriorating the set-point performance.

Multi-Loop Model Reference PID Internal Model (ML-MR PID-IM) Control Structure (ML-MR PID-IM): Internal Model Linear Adaptation Rule
Internal model control (IMC) relies on a mathematical model of a plant or process, which is referred to as an internal model and performs the control law according to this internal model. It was reported that IMC can be an effective solution for disturbance rejection and model perturbation problems [32]. In this section, we consider IMC to design a ML-MR control structure. Previously, an IMC-PID controller was proposed for improved disturbance rejection of time-delayed processes by using the reference model of the plant [14]. IMC law can be designed by manipulating transfer function models of the control system. If the internal model is an accurate enough representation of the system, the derived control law in the s-domain becomes valid for the real system.
In this section, we assume that the internal model is the PID control loop (the inner loop) and design an internal model PID controller, such that the outer loop can perform the IMC for the inner loop. Let us consider the multi-loop control structure that is depicted in Figure 1. To improve the disturbance rejection performance without influencing the control performance of the inner loop, the reference model, which is the internal model, is assumed to be the transfer function of the inner loop.
Our design objective is to find an internal model controller ) ( 2 s C so that the whole system behaves the same as the theoretical reference model ) (s T m , which represents an optimal design of the PID control loop. In order to achieve this objective, the transfer function condition ) ( . It is used in Equation (20) to satisfy the transfer function condition, which gives: Then, the controller function ) ( 2 s C is written as:

Multi-Loop Model Reference PID Internal Model (ML-MR PID-IM) Control Structure: Internal Model Linear Adaptation Rule
Internal model control (IMC) relies on a mathematical model of a plant or process, which is referred to as an internal model and performs the control law according to this internal model. It was reported that IMC can be an effective solution for disturbance rejection and model perturbation problems [32]. In this section, we consider IMC to design a ML-MR control structure. Previously, an IMC-PID controller was proposed for improved disturbance rejection of time-delayed processes by using the reference model of the plant [14]. IMC law can be designed by manipulating transfer function models of the control system. If the internal model is an accurate enough representation of the system, the derived control law in the s-domain becomes valid for the real system.
In this section, we assume that the internal model is the PID control loop (the inner loop) and design an internal model PID controller, such that the outer loop can perform the IMC for the inner loop. Let us consider the multi-loop control structure that is depicted in Figure 1. To improve the disturbance rejection performance without influencing the control performance of the inner loop, the reference model, which is the internal model, is assumed to be the transfer function of the inner loop.
One can write the transfer function that involves the internal model controller loop C 2 (s) and the theoretical reference model T 1 (s) as: Our design objective is to find an internal model controller C 2 (s) so that the whole system behaves the same as the theoretical reference model T m (s), which represents an optimal design of the PID control loop. In order to achieve this objective, the transfer function condition T 2 (s) = T m (s) should be satisfied. By using T m (s) = T 1 (s) (Equation (19)), the transfer function condition becomes T 2 (s) = T m (s) = T 1 (s). It is used in Equation (20) to satisfy the transfer function condition, which gives: Then, the controller function C 2 (s) is written as: By using Equation (19) in Equation (22), the C 2 (s) function is found: where the L(s) function is the open loop transfer function of the inner loop, which is expressed as L(s) = C 1 (s)G(s). In this form, the C 2 (s) function can be directly implementable. Application steps for existing control loops are summarized as follows: Step 1: The transfer function model of an optimal control loop is obtained, and this transfer function is used as the reference model (T m (s) = T 1 (s)).
Step 2: The optimal closed loop control system is enclosed by the outer loop, as shown in Figure 12.
Algorithms 2020, 13, x FOR PEER REVIEW 17 of 29 By using Equation (19) in Equation (22), the ) ( 2 s C function is found: where the ) (s L function is the open loop transfer function of the inner loop, which is expressed as In this form, the ) ( 2 s C function can be directly implementable. Application steps for existing control loops are summarized as follows: Step 1: The transfer function model of an optimal control loop is obtained, and this transfer function is used as the reference model ( Step 2: The optimal closed loop control system is enclosed by the outer loop, as shown in Figure  12. To demonstrate a possible application for the ML-MR PID-IM control, an example design for level control problem is presented. The plant model is an approximate model that represents the liquid level in the reboiler of the steam-heated distillation equipment [14,33]. The liquid level is controlled by adjusting the control valve on the steam line and the process model is given as [14,33]. The optimal PID controller for improved disturbance rejection was given as [33]: The system model is unstable, and the controller function was obtained with negative coefficients. By considering Equation (23) (25) We implemented the ML-MR PID-IM control structure for this system, as shown in Figure 13, and performed control simulations for square waveform input disturbance.  To demonstrate a possible application for the ML-MR PID-IM control, an example design for level control problem is presented. The plant model is an approximate model that represents the liquid level in the reboiler of the steam-heated distillation equipment [14,33]. The liquid level is controlled by adjusting the control valve on the steam line and the process model is given as G(s) = −1.6(−0.5s+1) 3s(s+1) [14,33]. The optimal PID controller for improved disturbance rejection was given as [33]:

Inner Loop
The system model is unstable, and the controller function was obtained with negative coefficients. By considering Equation (23), the C 2 (s) function of the ML-MR PID-IM control structure is obtained as: We implemented the ML-MR PID-IM control structure for this system, as shown in Figure 13, and performed control simulations for square waveform input disturbance. Figure 14 shows responses of the ML-MR PID-IM control structure and Chen et al.'s optimal PID control loop. Results in Figure 14 clearly demonstrate that the ML-MR PID-IM control structure improves the disturbance rejection performance of Chen et al.'s optimal PID control loop without influencing its set-point performance. In the absence of disturbance, responses of the ML-MR PID-IM control structure overlap with responses of the optimal PID control loop. The set-point performance of the ML-MR PID-IM control is almost the same as the performance of the optimal PID control loop proposed by Chen et al. This is an important advantage for robust control performance so that the proposed ML-MR control structures can improve the disturbance rejection while maintaining the set-point performance of the existing PID control loops. A limitation of the ML-MR PID-IM control is that it is not an effective solution for large time delay plant dynamics and inaccurate plant models. High time delays can severely affect the performance of the ML-MR PID-IM control scheme.

Multi-Loop Model Reference MIT-PID with Plant Function Adaptation (ML-MR MIT-PID-PFA): Nonlinear Adaptation Rule for Time-Delayed Systems
To improve the disturbance rejection capacity of time delay systems, model reference control is adopted for plant functions in a similar manner to classical IMC [14]. Any mismatch in responses from the real system and mathematical models of plants is assumed to be disturbance and is rejected by contributions of the MIT rule. In this configuration, the MIT rule forms the inner loop. The outer loop deals with the stability and set-point control. Therefore, this type of ML-MR control structure can be more effective for time delay systems because the adaptation loop only handles model mismatches between real plants and their mathematical models.
Given a first order plus dead time plant function in the form of G(s) = K τs+1 e −Ls , an analytical design scheme for the ML-MR MIT-PID-PFA control structure can be suggested as follows.
Step 1: Identify a model of the plant function and take the reference model of the MRC loop as the plant function (T m (s) = G(s)).
Step 2: Enclose the MRC loop with an optimal closed loop PID controller loop, as shown in Figure 15. Use the Tavakoli-Tavakoli PID tuning rule, which is expressed by Equation (10), for optimal operation of the PID controller function C(s).

Multi-Loop Model Reference PID-PID with Plant Function Adaptation (ML-MR PID-PID-PFA): Linear Adaptation Rule for Time-Delayed Systems
To obtain a linear adaptation rule, the MIT rule, which leads to nonlinearity, is changed to classical PID controller. ML-MR PID-PID-PFA control is implemented, as illustrated in Figure 16. An example design for the first order plus time delay plant model is given in the following section.

Multi-Loop Model Reference PID-PID with Plant Function Adaptation (ML-MR PID-PID-PFA): Linear Adaptation Rule for Time-Delayed Systems
To obtain a linear adaptation rule, the MIT rule, which leads to nonlinearity, is changed to classical PID controller. ML-MR PID-PID-PFA control is implemented, as illustrated in Figure 16.
The design steps of the ML-MR PID-PID-PFA control structure for a first order time delay plant in the form of G(s) = K τs+1 e −Ls can be carried out as follows.
Step 1: Identify the model of the plant function and take the reference model of the MRC loop as the plant function (T m (s) = G(s)).
Step 2: Enclose the MRC loop with the optimal closed PID controller loop, as shown in Figure 16. Tavakoli-Tavakoli PID tuning rule, which is expressed by Equation (10), is used for the both PID controller functions.

Multi-Loop Model Reference PID-PID with Plant Function Adaptation (ML-MR PID-PID-PFA): Linear Adaptation Rule for Time-Delayed Systems
To obtain a linear adaptation rule, the MIT rule, which leads to nonlinearity, is changed to classical PID controller. ML-MR PID-PID-PFA control is implemented, as illustrated in Figure 16. Step 1: Identify the model of the plant function and take the reference model of the MRC loop as the plant function ( Step 2: Enclose the MRC loop with the optimal closed PID controller loop, as shown in Figure  16. Tavakoli-Tavakoli PID tuning rule, which is expressed by Equation (10) To demonstrate the effectiveness of the ML-MR MIT-PID-PFA and ML-MR PID-PID-PFA control structures, we used them to enhance the first order plus deadtime plant model, which was given by Monje et al. [34].
This plant function is a first order dynamical model of the Basic Process Rig 38-100 Feedback Unit (produced by Feedback Instruments Ltd) experimental platform [34]. This system presents a 50 s time delay with a time constant of 433.33 s and is considered a large time delay process. Monje et al. designed an optimal fractional order PID (FOPID) controller for this system as follows: C FOPID (s) = 0.6152 + 0.01 s 0.8968 + 4.3867s 0.4773 (27) We implemented the ML-MR MIT-PID-PFA and ML-MR PID-PID-PFA control structures for this system by following the design steps given above. For control performance comparison purposes, we implemented a classical optimal PID control system with the Tavakoli-Tavakoli PID tuning rule (k p = 1.1862, k i = 0.0026, k d = 22.8885), a classical optimal PID control loop with MATLAB optimal tuning (k p = 0.8029, k i = 0.002, k d = −35.3930), and an optimal FOPID control system with Monje et al.'s tuning rule. The set-point of this experimental system was originally 0.47 [34] and a step disturbance with an amplitude of 0.47 at 1000 s was applied to the input of the plant model. The results of the control system simulations are shown in Figure 17. The simulation results reveal that both the set-point and disturbance rejection performances of the ML-MR PID-PID-PFA structure re better than other controllers. The lowest overshoot and fastest settling results were achieved by the ML-MR PID-PID-PFA structure. It is noteworthy to observe that the control performance of the ML-MR PID-PID-PFA structure is superior to the optimal FOPID control system. This finding is a clear indication of the potential for multi-loop PID controllers to surpass the control performance of the FOPID controllers. The FOPID controllers have been considered as substitutes for classical PID control and have been extensively studied in the last decade, showing performance improvements of FOPID controllers over classical PID controllers. A major complication in practice for FOPID controllers is their realization complexity. Fractional order controllers can be implemented by using integer order models because near-ideal realization of fractional order derivatives and integral elements in digital systems is highly computationally expensive [2]. Recently, analog circuit realizations of fractional order controllers have been shown, with results promising low-cost and low-complexity solutions for practical realization of FOPID controllers [35,36]. Recent works have shown the robust control performance improvement of MR-ML control structures with FOPID controllers [25,26], indicating an avenue for robust control system research. It is useful to investigate the frequency dependence of disturbance rejection performance. For this purpose, the performances of the classical closed loop PID control structure and ML-MR PID-PID-PFA control structure were compared in the control problem of the Basic Process Rig 38-100 Feedback Unit experimental platform [34] using sinusoidal waveform disturbance signals at several frequencies. The PID controller was tuned according to the ITAE design rule proposed by Tavakoli et al. Table 1  from simulations. For step disturbance, the ML-MR control structure reduces 50% the MAE of classical PID control system. In general, environmental disturbances are altered slowly and involve low frequency components. For very low frequency disturbance signals, the MAE of the ML-MR PID-PID-PFA control structure decreases down to 25% of the MAE of the classical PID control system. As the disturbance frequency increases, the MAE performance of the ML-MR PID-PID-PFA control structure equals the classical PID control structure. Figure 18 shows some of the simulation results and confirms these findings. It is useful to investigate the frequency dependence of disturbance rejection performance. For this purpose, the performances of the classical closed loop PID control structure and ML-MR PID-PID-PFA control structure were compared in the control problem of the Basic Process Rig 38-100 Feedback Unit experimental platform [34] using sinusoidal waveform disturbance signals at several frequencies.
The PID controller was tuned according to the ITAE design rule proposed by Tavakoli et al. Table 1 shows the mean absolute error (MAE) of the control error signal ( 1 T T 0 e c (t) dt) from simulations.
For step disturbance, the ML-MR control structure reduces 50% the MAE of classical PID control system. In general, environmental disturbances are altered slowly and involve low frequency components. For very low frequency disturbance signals, the MAE of the ML-MR PID-PID-PFA control structure decreases down to 25% of the MAE of the classical PID control system. As the disturbance frequency increases, the MAE performance of the ML-MR PID-PID-PFA control structure equals the classical PID control structure. Figure 18 shows some of the simulation results and confirms these findings.  In former studies, a ML-MR control structure using the MIT rule was experimentally verified for control of an experimental magnetic levitation system [26] and control of an experimental electrical rotor [28]. We conducted an experimental study to validate the contribution of the ML-MR PID-PID-PFA control structure to the disturbance rejection performance of classical PID control systems. A twin-rotor multi-input multi-output (MIMO) system (TRMS) is a popular rotor control test platform that is preferred for electrical rotor control experiments [37,38]. The TRMS is composed of two electrical rotors [39]. These are pitch and yaw rotors, as illustrated in Figure 19. The angles of In former studies, a ML-MR control structure using the MIT rule was experimentally verified for control of an experimental magnetic levitation system [26] and control of an experimental electrical rotor [28]. We conducted an experimental study to validate the contribution of the ML-MR PID-PID-PFA control structure to the disturbance rejection performance of classical PID control systems. A twin-rotor multi-input multi-output (MIMO) system (TRMS) is a popular rotor control test platform that is preferred for electrical rotor control experiments [37,38]. The TRMS is composed of two electrical rotors [39]. These are pitch and yaw rotors, as illustrated in Figure 19. The angles of the rotors are controlled by regulating the input voltages of the DC electric motors. This adjusts the rotational speed of the propellers so that the rotors can hover.
Algorithms 2020, 13, x FOR PEER REVIEW 24 of 29 the rotors are controlled by regulating the input voltages of the DC electric motors. This adjusts the rotational speed of the propellers so that the rotors can hover. Figure 19. Experimental twin-rotor multi-input multi-output system (TRMS) setup [39]. Figure 20 shows the experimental results. After settling the pitch angle to 0.5 rad, an input disturbance with a unit step waveform was applied at 25 s. One can observe that the step-point control performances of the classical PID controller and ML-MR PID-PID-PFA control structure are almost the same. The slight differences are mainly caused by internal system noise and limitations of the reference model in representing whole dynamics in the main rotor of the TRMS experimental setup. When disturbance is applied to the system, the response of ML-MR PID-PID-PFA control structure differs from the response of the PID controller and better rejects the disturbance at the system output, as shown in the figure. One of major technical complications that was observed in the experimental study was the model mismatch problem in the real system. We used a nonlinear model of TRMS that is a good representation of experimental systems. However, inherent limitations of mathematical modeling and changes of operating conditions reduce the model consistency and lead to model mismatching problems. Since the response of reference model does not adequately match with the response of the  Figure 19. Experimental twin-rotor multi-input multi-output system (TRMS) setup [39]. Figure 20 shows the experimental results. After settling the pitch angle to 0.5 rad, an input disturbance with a unit step waveform was applied at 25 s. One can observe that the step-point control performances of the classical PID controller and ML-MR PID-PID-PFA control structure are almost the same. The slight differences are mainly caused by internal system noise and limitations of the reference model in representing whole dynamics in the main rotor of the TRMS experimental setup. When disturbance is applied to the system, the response of ML-MR PID-PID-PFA control structure differs from the response of the PID controller and better rejects the disturbance at the system output, as shown in the figure.
Algorithms 2020, 13, x FOR PEER REVIEW 24 of 29 the rotors are controlled by regulating the input voltages of the DC electric motors. This adjusts the rotational speed of the propellers so that the rotors can hover. Figure 19. Experimental twin-rotor multi-input multi-output system (TRMS) setup [39]. Figure 20 shows the experimental results. After settling the pitch angle to 0.5 rad, an input disturbance with a unit step waveform was applied at 25 s. One can observe that the step-point control performances of the classical PID controller and ML-MR PID-PID-PFA control structure are almost the same. The slight differences are mainly caused by internal system noise and limitations of the reference model in representing whole dynamics in the main rotor of the TRMS experimental setup. When disturbance is applied to the system, the response of ML-MR PID-PID-PFA control structure differs from the response of the PID controller and better rejects the disturbance at the system output, as shown in the figure. One of major technical complications that was observed in the experimental study was the model mismatch problem in the real system. We used a nonlinear model of TRMS that is a good representation of experimental systems. However, inherent limitations of mathematical modeling and changes of operating conditions reduce the model consistency and lead to model mismatching problems. Since the response of reference model does not adequately match with the response of the One of major technical complications that was observed in the experimental study was the model mismatch problem in the real system. We used a nonlinear model of TRMS that is a good representation of experimental systems. However, inherent limitations of mathematical modeling and changes of operating conditions reduce the model consistency and lead to model mismatching problems. Since the response of reference model does not adequately match with the response of the real TRMS setup, the PID controller of the adaptation loop, which generates the adaptation gain θ, can cause instability in the system for the same PID controller coefficients for the control loop. This stability problem can be solved by retuning the PID controller to the adaptation loop.

Discussions and Conclusions
This study investigated some contemporary ML-MR adaptive control structures that are applicable for classical PID control loops. These structures are the ML-MR PID-MIT, ML-MR PID-MIT-CGM, ML-MR PID-I, ML-MR PID-IM, ML-MR MIT-PID-PFA, and ML-MR PID-PID-PFA control structures. Design problems of these ML-MR structures were addressed, and straightforward design and implementation strategies were presented. The performance evaluations, advantages, and drawbacks of these ML-MR structures were discussed according to control simulation results. Simulation results indicated significant disturbance rejection performance improvements in four different control application examples. The performance comparisons with several single-loop optimal control loops theoretically show that improvement of the disturbance rejection control without affecting the set-point performance is possible with presented ML-MR control structures. Some noteworthy properties of the presented ML-MR adaptive control structures are listed in Table 2. Table 2. Some properties of the presented ML-MR control structures.

ML-MR Control
Structures Character Outer loop integrations cover the whole closed control loop to deal with disturbances, and several works have indicated that these type integrations are preferable solutions for improving fault tolerance and disturbance rejection in the whole control loop [25][26][27][28]. Inner loop integration only encloses the plant function, and hence it can be employed to deal with external disturbances or parametric perturbations that influence the controlled system. * ML-MR PID-MIT-CGM and ML-MR PID-I structures can adaptively increase the RDR performance of the closed control loops without affecting set-point performance. The ML-MR PID-IM control structure benefits from having an internal model controller design, however it should be noted that its performance is very sensitive to the accuracy of plant and controller models. * Large-time delay systems that set the point and disturbance rejection performance of ML-MR PID-PID-PFA control structures can surpass the disturbance rejection performance of a single-loop optimal FOPID control system. These results reveal that the disturbance rejection capacity of closed loop PID control loops can be exceeded by combining them with ML-MR control structures. ML-MR control structures are a feasible way to resolve performance tradeoff between set-point and disturbance rejection control structures. * Stability analysis of linear ML-MR control structures is relatively straightforward and is performed by obtaining the transfer function of the whole structure and checking for pole placements for left half-plane (LHP) stability. The satiability of a nonlinear ML-MR structure, such as ML-MR PID-MIT control, along with the stability and convergence conditions, are derived on the basis of the bounded input and bounded output stability state of the model error (see Appendix A). * Results in this study are based on simulation models. The reference models perfectly match the controlled systems. It is very important to note that model mismatches due to limitations in mathematical modeling and parametric model perturbations in real systems may have severe effects on the control performance and stability of the presented ML-MR adaptive control structures in real-world applications. The sensitivity of these structures to model mismatching should be carefully investigated in future works. The fragility of the system stability due to model imprecision should be carefully considered when designing real control systems. Moreover, the resolution of sensors and I/O ports of control cards are important in practical applications. * This study is a theoretical demonstration of six different ML-MR adaptive control structures. Future works should address experimental performance evaluations and practical realization issues of ML-MR adaptive control structures.

ML-MR PID-MIT
Proof A2. Let us consider the adaptation rule of the ML-MR PID-MIT control structure, which is given by Equation (9) as θ = −γ 1 s y m e m . The model can be written as: The output of the reference model y m can be written as y m = T m (s)r(s). For set-point control stability, one considers the step response of the system. Therefore, the reference input is assumed to be a step function by using r(s) = 1 s in Equation (A3).
According to Lemma 1, the convergence of the model error to zero (e m = 0) is possible for θ = 1 in the case of T m (s) = T(s). By applying this condition, the model error is written as: The function T(s) is the transfer function of the inner loop, which is stable due to optimal tuning of the PID control system. When the T(s) function given by Equation (2) is used in Equation (A5), one obtains The inner loop is configured to be stable, therefore the output of the inner loop is bounded for y < ξ ∈ R for a step reference input. When the model error e m of the system given by Equation (A6) is shown to be stable, the bounded input and bounded output of the ML-MR PID-MIT control structure become stable, so that e m = y − y m < ε ∈ R is satisfied. In order to ensure convergence of the model error to zero, a final value theorem can be applied as: