DYNAMIC SYSTEM MODELLING USING RULES3 INDUCTION ALGORITHM

This paper describes the use of a new induction algorithm to derive production rules for modelling dynamic systems. The algorithm, called RULES-3, is an efficient tool for extracting a compact set of IF-THEN rules from a collection of examples to classify objects into known categories. The paper summarises the operation of RULES-3 and presents the results obtained in the modelling of two linear systems and two non-linear systems.


INTRODUCTION
In general, to control a system which changes continually or for which a model cannot be derived through mathematical analysis, methods for obtaining a model of the system experimentally are required.Until recently, the most common methods have involved assuming a model structure and determining the values of the model parameters by experimentation (see, for example [1]).These methods rely on choosing a correct structure, which demands much experience and intuition from the control engineer.In the past few years, new machine learning techniques for experimental model building have been developed that do not need making assumptions about the structure of the model.These techniques can be separated into two types, neural-network-based techniques and induction-based techniques.
Neural-network-based techniques are simple and robust and have been successfully applied to the modelling of a variety of linear and non-linear systems [2;3;4; 5; 6; 7; 8; 9; 10, 11].However, a disadvantage with these techniques is that the models produced only implicitly mimic the surface behaviours of the original systems, are non-transparent and therefore cannot be easily scrutinised or modified.
Induction-based techniques are also simple and reliable, but have the additional benefit of yielding more transparent models.Models obtained using induction are either represented as decision trees or production rules.Although they essentially will only mimic the external behaviours of the original systems, these models can be readily examined and modified if necessary.
This paper describes the use of RULES-3, the latest version of RULES, for inductive model building.Following a summary of the operation of RULES-3, the paper will present the results obtained with that algorithm in modelling four different systems.

RULES-3
RULES-3 [14] is a simple algorithm for extracting a set of classification rules from a collection of examples for objects belonging to one of a number of known classes.An object must be described in terms of a fixed set of attributes, each with its own range of possible values which could be nominal or numerical.For example, attribute "speed" might have nominal values {low, medium, high} or numerical values in the range [-10, 10].
An attribute-value pair constitutes a condition in a rule.If the number of attributes is N a , a rule may contain between one and N a conditions.Only conjunction of conditions is permitted in a rule and therefore the attributes must all be different if the rule comprises more than one condition.RULES-3 extracts rules by considering one example at a time.It forms an array consisting of all attribute-value pairs associated with the object in that example, the total number of elements in the array being equal to the number of attributes.The rule forming procedure may require at most N a iterations per example.In the first iteration, rules may be produced with one element from the array.Each element is examined in turn to see if for the complete example collection it appears only in objects belonging to one class.If so, a candidate rule is obtained with that element as the condition.In either case, the next element is taken and the examination repeated until all elements in the array have been considered.At this stage, if no rules have been formed, the second iteration begins with two elements of the array being examined at a time.Rules formed in the second iteration therefore have two conditions.The procedure continues until an iteration when one or more candidate rules can be extracted or the maximum number of iterations for the example is reached.In the latter case, the example itself is adopted as the rule.If more than one candidate rule is formed for an example, the rule that classifies the highest number of examples, is selected and used to classify objects in the collection of examples.Examples of which objects are classified by the selected rule are removed from the collection.The next example remaining in the collection is then taken and rule extraction is carried out for that example.This procedure continues until there are no examples left in the collection and all objects have been classified.Figure 1 summarises the steps involved in RULES-3.

SYSTEM MODELLING EXPERIMENTS
The model building ability of RULES-3 was tested on four systems.

System 1
System 1 is a linear second-order system previously modelled using a neural network [17].Its discrete input-output equation is: . . .
1 8 0 837 0 019 0 018 Thus the system has four essential attributes: y(t), y(t-1), u(t), and u(t-1).Two cases were tested.In one case, the range of input and output variables was divided into four equal intervals or quantisation levels.In the other case, nine quantisation levels were employed.This yielded 4 values per attribute in one case and 9 values per attribute in the other.16 rules and 18 rules were extracted by RULES-3 for the first and the second cases respectively.The quantised responses of the system and the models to a step input are shown in Figures 2 and 3.As can be expected, the model was more accurate when the number of quantisation levels was higher, although in both cases the responses of the models closely followed those of the plant.Figure 4 displays the responses of models with 5 attributes (y(t), y(t-1), y(t-2), u(t), u(t-1)) and 6 attributes (y(t), y(t-1), y(t-2), u(t), u(t-1), u(t-2)).Those models were obtained to test the ability of RULES-3 to handle situations where the order or structure of the system of interest is unknown.Apart from the model based on three  attributes (y(t-1), u(t), u(t-1)) (see Figure 5 for its response), which can only represent a system of lower order than the actual plant, the other models all gave responses almost identical to it.This demonstrates that the induction algorithm can derive good models without information on the exact number of attributes, or the system order, involved.For comparison, the quantised response of a neural network using a multi-layer perceptron (MLP) with 4 input units, 19 hidden units and 9 output units is plotted in Figure 6.The neural network also modelled the plant well.However, its training time, at 23.11 minutes, was considerably longer than the model extraction time of 2.52 seconds taken by RULES-3.(number of quantisation levels = 9; number of attributes = 4).

System 2
System 2 is a non-linear plant with the following input-output equation: u(t) and y(t) are the two essential attributes for this problem.Seven quantisation levels were each defined for u(t) and y(t).13 rules were extracted by RULES-3 as the model of the plant.A series of random signals was applied to the plant as shown in Figure 7 and the responses obtained are plotted in Figure 8.Note that the RULES-3 model behaved exactly as the plant for 67 of the 74 time instants shown.Only the data corresponding to the initial 18 time instants had been employed to extract the model.This shows the excellent generalisation ability of RULES-3.As with System 1, models were also generated without assuming prior knowledge of the plant structure.The responses of models based on three attributes (y(t), y(t-1), u(t)) and four attributes (y(t), y(t-1), u(t), u(t-1)) are plotted in Figures 9  and 10, superimposed on the expected responses of the actual plant.These show that even when irrelevant attributes were used, RULES-3 could still produce accurate models.The response of a neural model using an MLP with 2 input units (for y(t) and u(t)), 11 hidden units and 7 output units is charted in Figure 11.Although it employed only relevant attributes, the neural model gave a poorer performance compared to the production rules.The training time for the neural model was 11.4 minutes while the induction time for RULES-3 was 1.92 seconds.(number of quantisation levels = 7; number of attributes = 2).

System 3
System 3 is a linear third-order plant with the following input-output equation: y(t) = 2.03786 y(t-1) -1.366 y(t-2) + 0.3012 y(t-3) + 0.003 u(t-1) + 0.0089 u(t-2) + 0.000163 u(t-3) y(t-1), y(t-2), y(t-3), u(t-1), u(t-2) and u(t-3) are the six essential attributes for this problem.50 quantisation levels were defined.78 rules were extracted by RULES-3.The quantised responses of the system and the model to a step input are shown in Figure 12.As can be seen, the responses of the model and the system are almost identical.The quantised response of an MLP with 6 input units, 15 hidden units and 13 output units and the response of the plant are plotted in Figure 13.The training time was 1 minute 48 seconds for RULES-3 and 1 hour 49 minutes for the MLP.Note that the number of output units of the MLP was 13 instead of 50 (that is the output quantisation level was assumed to be 13 rather than 50) to avoid excessive MLP training times.(number of quantisation levels = 50; number of attributes = 6).

System 4
System 4 is a non-linear plant.Its discrete input-output equation is: y(t) = 2.03786 y(t-1) -0.824 y(t-2) + 0.130667 y 3 (t-2) -0.16 u(t-2) y(t-1), y(t-2) and u(t-2) are the three essential attributes for this system.15 quantisation levels were defined for the system.25 rules were extracted by RULES-3.The quantised responses of the system and the model to a step input are shown in Figure 14.The quantised response of an MLP with 3 input units, 15 hidden units and 7 output units and the response of the plant are shown in Figure 15.The training time was 6.43 seconds for RULES-3 and 29 minutes for the MLP.Table 1 summarises the statistics for the tests on the four systems.Table 2 gives a comparison of the accuracies of models obtained using RULES-3 and MLPs.It can be seen that the former are at least equal to the latter.4. CONCLUSION The use of a new induction algorithm to extract models of dynamic systems has been described.Induction-based modelling generally has the advantage of producing explicit models.In addition to this explicitness, the models induced by RULES-3, the algorithm adopted in this work, can have better accuracies than models constructed using neural networks.RULES-3 is easy to apply.It operates efficiently and does not require making assumptions about the structure of the system to be modelled.

Step 1 .Figure 1 .
Figure 1.Induction procedure in RULES-3 (N c =number of conditions, N a =number of attributes).

Figure 4 .
Figure 4. Quantised responses of System 1 and of its model obtained by RULES-3 (number of quantisation levels = 9; number of attributes = 5 or 6).

Figure 5 .
Figure 5. Quantised responses of System 1 and of its model obtained by RULES-3 (number of quantisation levels = 9; number of attributes = 3).

3 Figure 8 .
Figure 8. Quantised responses of System 2 and of its model obtained by RULES-3 (number of quantisation levels = 7; number of attributes = 2).

Figure 9 .
Figure 9. Quantised responses of System 2 and of its model obtained by RULES-3 (number of quantisation levels = 7; number of attributes = 3).

3 Figure 10 .
Figure 10.Quantised responses of System 2 and of its model obtained by RULES-3 (number of quantisation levels = 7; number of attributes = 4).

Figure 11 .
Figure 11.Quantised responses of System 2 and of its model obtained by MLP.(number of quantisation levels = 7; number of attributes = 2).

Figure 12 .
Figure 12.Quantised responses of System 3 and of its model obtained by RULES-3 (number of quantisation levels = 50; number of attributes = 6).

Figure 13 .
Figure 13.Quantised responses of System 3 and of its model obtained by MLP.(number of quantisation levels = 50; number of attributes = 6).

Figure 14 .
Figure 14.Quantised responses of System 4 and of its model obtained by RULES-3 (number of quantisation levels = 15; number of attributes = 3).

Figure 15 .
Figure 15.Quantised responses of System 4 and of its model obtained by MLP.(number of quantisation levels = 15; number of attributes = 3).

Table 1 .
Summary of statistics for different tests.(QL = number of quantisation levels)

Table 2 .
Comparison of accuracies of RULES-3 and MLP. ( QL = number of quantisation levels y p = output of plant y m = output of model)