Due to its complex and ill-defined nature, the UML Class Diagram was selected as a domain for the ITS design. The proposed UML-ITS is implemented to teach OO analysis and design concepts, where students collaboratively construct a UML class diagram based on some given requirements. UML-ITS creates a session between two students and assigns them roles of Tutor (the one who evaluates the solution of the other student and provides suggestions/feedback) and Tutee (the one who develops the solution). For the rest of the paper, the role names of Tutor and Tutee are used. Each action performed by the student is recorded in the log file. The students develop a solution model by drawing diagrams on the workspace area and interact with each other using the chat tool. UML-ITS provides feedback to students while solving a problem through hints. Once both students agree on the modeled solution, they can submit the final solution to the ITS. The solution is evaluated against the sample/ideal solution stored in the knowledge base. If the submitted solution has errors, the hints are generated; otherwise, a new problem scenario is displayed.
Figure 1 shows the interface of UML-ITS. The same screen is displayed to both students but with different toolbar options.
3.1. XML Solutions
XML stands for eXtensible Markup Language, which is widely being accepted as a form of information representation. XML stores information in the form of tags. In UML-ITS, all the ideal solutions of UML textual problems are stored in XML documents. All XML documents begin with an XML declaration followed by a root element, which is <ClassDiagram> in our case. Each class is defined with the <Class> tag, which includes <Name>, <Attribute>, <Method>, and <Relationship> tags as child elements. The <Relationship> tag has four attributes:
WithClass: Defines the name of the class that has a relationship with the current class.
Type: Defines the type of relationship, i.e., Association, Generalization, etc.
Link: Defines Start or End of the line connector. This helps in the identification of parent-child and whole-part relationships if generalization, composition, or aggregation types are used. The Link = “End” attribute value represents the “Whole” class relationship, whereas Link = “Start” represents the “Part” class relationship. For class inheritance, the parent class is represented with the End link type and child class with the Start link type.
Multiplicity: Defines participation constraints on association relationship types.
The student’s solution is also converted to a temporary XML document that is compared against the XML solution stored in the UML-ITS’s knowledge base. The temporary XML document is updated whenever the tutee student makes changes in the solution. If the temporary XML document is different from that of XML solution, the hints are generated by the UML-ITS based on the differences found.
Figure 2 describes the comparison of both XML documents.
3.4. Peer Feedback Evaluator Design
The evaluation of tutor feedback takes place before delivering it to the tutee student. The main function of the feedback evaluator module is to evaluate tutor feedback against the ideal solution. The first step in evaluating the tutor’s feedback is to identify the type of feedback that the tutor is providing to the tutee student. The sub-components of the Feedback Evaluator are shown in
Figure 5.
It can be seen in
Figure 5 that the feedback classifier receives a feedback event, which is the hint that the tutor is providing to the tutee. The feedback classifier identifies whether it is related to classes, relationships, attributes, or methods. This identification is based on the tutor’s actions that he/she has taken on the workspace. If the tutor clicks the class component, then it is classified as ‘C’, relationships as ‘R’, attributes as ‘A’, and methods as ‘M’. For example, if the tutor finds an error in the solution related to classes, then the tutor clicks on that particular class. As soon as the tutor clicks on the class, the following message is generated:
C ClassName = “Items”
The message indicates that the tutor has clicked on the class component and its name is ‘Items’. Once the component on which the tutor is providing feedback has been identified, then further classification takes place to see if the response is related to a wrong component, a missing component, or extra classes in the tutee’s solution. This classification is based on the button activated from the tutor’s toolbar. If Suggest Missing is activated, then a token related to the missing component is added to the feedback; if the Error button is selected, then a token for the wrong component is added; and if the Delete button is activated, then an extra component token is added to the feedback message. The feedback tokenizer assigns tokens based on their classification, as shown in the following example, which indicates that the ‘Items’ class is an incorrect class in the tutee’s solution.
WrongC ClassName = “Items”
For attributes and methods, WrongA and WrongM tokens are used, respectively. Along with the class name in which the wrong attribute or method is defined, the attribute and method names are also appended in the feedback message. For example, the following feedback messages indicate that the tutor has marked the attribute with the name ‘id’ in the class ‘Items’ as a wrong attribute. Similarly, the method with name ‘Show’ in the class ‘Items’ has been marked as a wrong method.
WrongA ClassName = “Items” Attribute = “id”
WrongM ClassName = “Items” Method = “Show”
For errors in relationships, more information is added in the message to indicate the endpoints’ directions along with the type of relationships. For example, if the tutor finds an error in the composition relationship type, then the following feedback message is generated:
WrongR ClassName = “Items” WithClass = “Orders” Type = “Composition” Link = “Start”
The above message indicates that the relationship of the type Composition between the Items and Orders classes is wrong. The endpoints of relationships are identified through the Link attribute. If the endpoints of the relationship are drawn incorrectly, then it is also considered to be an error in the relationship, which is tokenized similarly, as described above.
Table 1 shows the list of tokens that are assigned to feedback messages.
The feedback evaluator compares the tokenized feedback message with an ideal solution to see if the tutor is responding correctly to the tutee. This comparison is based on following conditions:
A missing class diagram component is present in the ideal solution but not in the tutee’s solution.
A wrong class component is present in the tutee’s solution but not in the ideal solution.
An extra class diagram component is present in the tutee’s solution but not in the ideal solution. This condition also checks if the total number of classes present in the tutee’s solution is greater than the total number of class components in the ideal solution. If the number is greater, then the selected component is considered to be extra; otherwise, it is considered to be wrong.
Once the feedback evaluator compares the feedback with the ideal solution, extra information is added in the feedback message. If the tutor has correctly marked an incorrect class diagram component as the wrong class, then a plus (+) sign is added at the beginning of the message, which indicates that the feedback from the tutor is correct. Conversely, if the tutor has marked a correct class diagram component as the wrong class, then a minus (−) sign is added at the beginning of the feedback message to indicate incorrect feedback from tutor.
Table 2 shows some of the examples of correct and incorrect feedback message identification. The feedback message along with token and sign is delivered to the Hints generator to produce hints accordingly.
3.5. Tutor Feedback Evaluation Model
Tutor feedback in the UML-ITS is evaluated by the feedback evaluation component, which evaluates all feedback coming from the tutor before it is delivered to the tutee. This evaluation is beneficial for both the tutor and tutee during their learning process in many ways. Firstly, the tutors can reflect on their own knowledge about the domain when their mistakes are notified by the system. During their tutoring process, if tutors receive the same type of tutee mistake on which they provided wrong feedback, the tutors will recall and try to avoid responding incorrectly. Secondly, the tutors have an opportunity to correct themselves before their feedback is delivered to the tutee, hence upholding their faith in teaching, and preventing them from thinking of themselves as inept tutors. Thirdly, the tutor’s image in the tutee’s perceptions is retained since the tutee always expects to receive the required feedback. Lastly, the tutee receives relevant feedback and models the solution without recording extra misconceptions in their log.
Figure 6 illustrates the overall problem-solving flow, and the roles of each student and the system as a whole. Bold parts in the flow diagram are related to tutor feedback evaluation. The feedback evaluation process starts when the tutor sends a response to the tutee student. The tutee student can take any one action: they can take step to design a solution (i.e., creating a class or modifying some properties, etc.), they can indicate that they have completed the solution by clicking the Evaluate button, or they can ask for the tutor’s help by clicking the Hints button. The tutor can respond at any time after the tutee clicks the Evaluate button or Hints button. The main function of the feedback evaluator module is to evaluate the tutor feedback against the ideal solution and generate hints depending on the tutor’s error. If the tutor has marked a correct mistake/error in the solution, then positive feedback from the system is generated to appreciate the tutor. In the opposite case, where the tutor marks a correct component as a mistake/error, negative feedback is displayed to the tutor indicating that the selected component is the correct one. Once the tutor marks the correct mistake/error, then the tutor needs to suggest the changes that are not present in tutee’s solution. The feedback evaluator evaluates tutor’s feedback regarding classes, attributes, methods, and relationships in the same ways as described above.