Inheritance Coding with Gagné-Based Learning Hierarchy Approach to Developing Mathematics Skills Assessment Systems

Abstract

This study developed an inheritance coding with Gagné-based learning hierarchy approach to building systems for assessing mathematics skills and diagnosing student learning problems. The proposed Gagné-based learning hierarchy approach combines Gagné learning hierarchy theory with an inheritance coding approach. First, Gagné learning hierarchy theory is used to generate test questions and learning path diagrams for a skills assessment system. To assess learning achievement, an inheritance coding approach is used to encode the test questions according to learning hierarchy paths. The analysis, design, development, implementation, and evaluation design model is used throughout the process of developing the assessment system. Statistical analyses of the test questions for assessing student learning achievement included expert validity, internal reliability, test–retest reliability, and parallel-form reliability. System performance questionnaires were also designed to survey the opinions of the students regarding the mathematics skills assessment system. The internal reliability of the overall questionnaire was also calculated. The experimental practical application of the assessment system, developed by the Gagné-based learning hierarchy approach, showed that it can accurately diagnose student learning barriers and provide learning suggestions for students and teachers.

1. Introduction

The Gagné model identifies five major categories of learning: verbal information, intellectual skills, cognitive strategies, motor skills, and attitudes [1]. Robert M. Gagné suggested that learning tasks for intellectual skills could be hierarchically organized according to their complexity and proposed the following categories of learning tasks: stimulus recognition, response generation, procedure following, use of terminology, discriminations, concept formation, rule application, and problem solving. The alternative Gagné taxonomy worked well with student outcomes in many courses because it could distinguish among verbal information, attitudes, and psychomotor skills. Additionally, the taxonomy effectively distinguished among intellectual skills, i.e., concept learning, rule using, and problem solving [2]. The significance of the hierarchy was that it identified the prerequisites for facilitated learning at each level. Learning hierarchies provided a basis for determining the best sequence of instruction [3]. A study by Flynn [4] established that the Gagné learning theory, including its events of instruction, accurately described and supported learning in a cooperative learning environment. The findings clearly suggested that collaborative learning should also be consistent with and/or supportive of the Gagné learning hierarchy [4,5]. In addition to improving understanding of learning hierarchies and hierarchical analysis, the Gagné model introduced an important notion: learners incrementally acquire prerequisite skills before they attempt to master higher or more complex skills. According to the decomposition hypothesis, using the Gagné learning hierarchies to analyze complex tasks required an assumption that mastery of less complex units is a prerequisite for mastery of more complex units. The theory is applicable in the design of increasingly complex task levels [6,7]. Gagné [6] conceptualized learning as a change in human disposition or capability. Gagné also argued that the change could be retained and should not simply be ascribed to the growth process. Curriculum planners can apply this concept when examining the nature, depth and breadth of coverage of linear programming material such as that in a mathematics curriculum [8]. In the hierarchy proposed by Gagné, problem solving is the highest level of learning because it requires mastery of the next lower type of learning. Problem solving requires the application of principles and facts to explain and solve new phenomena or to predict consequences from known conditions. Additionally, problem solving requires the use of prediction and the analysis of facts and principles to identify cause and effect relationships among physical phenomena in the environment [9]. In the above research studies, the results were in agreement with the Gagné cognitive theory of prerequisite knowledge and learning hierarchy. Therefore, this study uses the Gagné model to develop learning hierarchy diagrams and test questions for a mathematics skills assessment system used to diagnose student learning problems.

The analysis, design, development, implementation, and evaluation (ADDIE) model is an instructional design model [10] that provides descriptive guidelines for building effective academic performance support tools in five phases. The main purpose of the ADDIE model is to improve learning efficiency by analyzing learning needs and developing a system of learning tools. Although it has been criticized in recent years, the ADDIE model has proven to be sufficiently flexible for building effective academic performance support tools [11,12]. Arkün and Akkoyunlu [13] examined the process of using the ADDIE model and student opinions to develop a multimedia learning environment. The authors concluded that, in terms of problem solving, the ADDIE model is highly effective, in a cyclical way. Hsu, Lee-Hsieh, Turton, and Cheng [14] used the ADDIE model to develop online continuing education courses on caring for nurses. The authors concluded that ADDIE is useful and developed a procedure for using the model to collect data and analyze results. Moradmand, Datta, and Oakley [15] used the ADDIE instructional system design method to guide the process of designing, implementing and evaluating educational multimedia software for learning mathematics. They concluded that this application appears to have considerable potential for use in teaching elementary school mathematics through storytelling and multimedia. Azimi, Ahmadigol, and Rastegarpour [16] evaluated the effectiveness of ADDIE instructional design and multimedia for learning key skills in futsal. Participants who were trained by an ADDIE model had higher mean scores for key futsal skills compared to students trained by traditional methods. Additionally, the authors successfully used ADDIE for developing e-materials for elementary students, including a mathematics speed unit, a complex graphics area unit, a fraction area unit, and a Mandarin character recognition learning course [17,18,19,20,21,22]. According to these studies, the ADDIE design model is sufficiently dynamic and flexible for use throughout the process of developing a mathematics skills assessment system.

For both students and educators, assessment systems developed using the ADDIE design model have proven useful for evaluating learning outputs. Researchers have developed various methods of analyzing the learning problems and barriers of students [23,24,25,26]. For example, Chen [27] developed an intelligent personalized learning system that used genetic algorithms to determine the best learning paths according to the incorrect answers of individual learners. Ketterlin-Geller and Yovanoff [28] used cognitive diagnostic assessments to provide detailed and precise information about the cognitive processes of students who had difficulty learning mathematics. Ozyurt, Ozyurt, and Baki [29] proposed a structure and improvement processes for computerized adaptive testing systems. Conventional systems for assessing the abilities of students are now being replaced by adaptive assessment systems. The objective of adaptive assessment systems is to evaluate students according to their actual ability rather than according to their test grades. Hauswirth and Adamoli [30] proposed a pedagogical approach to facilitating students in learning from their mistakes by answering a series of questions. Hwang, Panjaburee, Triampo, and Shih [31] proposed a group decision approach for developing a concept-effect relationship model with the cooperation of multiple domain experts. Low-achievement students who were taught using the group decision approach had significantly better learning achievement compared to those who were taught using the conventional approach. Yang, Hwang, Yang, and Hwang [32] proposed a two-tier test-based learning approach to enhancing learning outcomes in computer-programming courses in a web-based learning environment. Lai, Kobrin, DiCerbo, and Holland [33] described an application of the assessment triangle, and compared two studies to evaluate whether evidence of student performance is consistent. Wilkins and Norton [34] used a hierarchy of fraction scheme that charted the outline of a progression from part–whole concepts to measurement concepts of fractions. However, for elementary students struggling in mathematics, the issue of which diagnostic method is most effective for understanding their learning problems needs further study.

The aim of the proposed research is to propose effective theoretical methods, construct processes, and establish assessment systems. The research question is to diagnose student learning problems. This study proposes a Gagné-based learning hierarchy approach (GBLHA) to developing mathematics skills assessment systems that use the ADDIE model to diagnose student learning problems. Firstly, a learning hierarchy diagram for a mathematics unit is built by applying Gagné learning hierarchy theory. Secondly, an inheritance coding approach is used to encode the test questions according to the learning hierarchy paths. The level value for a lower-level skill is added to the level value for a higher-level skill. Thirdly, when the questions are used to test a student, the accuracy ratio (AR) for the skill level concept is computed to assess the learning achievement of the student. The experiments showed that students and teachers can use the mathematics skills assessment system developed by the GBLHA to identify barriers to learning and to provide suggestions for improving learning efficiency.

This paper is organized as follows. Section 2 discusses how Gagné theory is used to diagnose student learning problems. Section 3 introduces the proposed Gagné-based hierarchy approach. Section 4 describes the system development process. Section 5 discusses our observations in practical applications of the approach in diagnosing student learning problems. Finally, Section 6 concludes the study.

2. Diagnosing Student Learning Problems by Applying Gagné Theory

Gagné [6] advocated a hierarchical analysis of intellectual skills. The component skills are task-analyzed in a parts-to-whole sequence that follows the hierarchy in a “bottom-up” fashion, followed by increasingly complex combinations of the parts [35]. Gagné developed the concept of a schema, which he defined as a modifiable information structure that contains prototypical information about frequently experienced situations. The schema can be used to interpret new situations. So, understanding the situations requires prior knowledge [1]. For example, prior knowledge of mathematics can be identified by pre-test scores in mathematics. Acquisition of mathematics knowledge can then be assessed by post-tests administered after each module.

Merrill [36] applied the Gagné hierarchical task analysis by breaking down the processes of learning and teaching mathematics into bite-sized chunks. Merril [36] concluded that the best way to teach such tasks would be having a subject-matter expert demonstrate how the task should be performed and then letting the learners practice performing the task. Mathematics and science assessments based on the Gagné theory of learning would address some limitations of using tests for assessment. When Gagné theory is used to define concepts, the instructional designer or test writer must explicate the dimensions of a concept, and the learner must discriminate among examples or generate instances based on these defined or agreed-upon limits.

For learning mathematics, e.g., long division, Gagné theory is much too complex to be applied in only one session. The subparts such as multiplication and subtraction must be achieved in prior instructional sessions before the teacher can attempt to teach long division. In such a case, the subparts are often independent and functional tasks in their own right. Some paths include all the steps of other paths. Such paths have hierarchical part–whole relationships to one another. The hierarchical relationship can be used to specify an instructional or learning prerequisite sequence. Students cannot learn how to perform the task until they know how to perform the subtasks. A positive transfer from simpler tasks to more complex tasks is expected to occur when performance of the simpler tasks is required to complete the more complex tasks [36]. Gagné developed hierarchical task analysis, which is the most widely accepted task analysis approach. This approach entails identification of a hierarchy of subskills such that the acquisition of low-level skills or behaviors is followed by a positive transfer of the skills to a higher level. Gagné conceptualized intellectual skills as a hierarchy of skills organized from simple to complex. In other words, students must learn a low level of elements before moving on to a higher level of elements, which requires a cumulative learning process. The intellectual skills involved include discriminative learning, concrete concept learning, defined concept learning, rule learning, and problem solving. According to the learning hierarchy, concept learning is the foundation of problem solving [37]. Further studies are needed to address issues such as how long a unit on counting should last and to what extent children should be required to master relevant concepts before learning a mathematical procedure. Development of mathematics knowledge in children is often described as a hierarchical process during which later skills build on earlier skills. Some skills acquired earlier are considered particularly foundational for further learning of mathematics in children. Identifying mathematics skills that should be acquired at an early stage and structuring mathematics curricula around them may be an effective way to boost subsequent efficiency in learning mathematics. Questions about which specific mathematics skills are most important are often addressed through a combination of experimental studies and correlational research based on observational data. Although correlational research is free from certain logistical problems associated with experimental research, its utility for making causal inferences about the likely effects of interventions often relies on untested assumptions. For example, some researchers have hypothesized that individual differences in early acquisition of mathematics skills reflect variation in truly unique constructs [38,39,40].

3. Gagné-Based Learning Hierarchy Approach

According to the Gagné learning hierarchy theory, the acquisition of intellectual skills is cumulative, i.e., mastery of higher-level skills requires prior mastery of lower-level skills. Since intellectual skills are acquired in a hierarchical order, successful instruction requires the student to acquire lower-level skills before progressing to higher-level skills [6,41,42]. The current study uses a learning hierarchy to build learning path diagrams and test questions for a mathematics skills assessment system. To assess the knowledge of students, teachers were required to apply their expertise in designing test questions for each level. For example, integers and fractions are both related to certain learning processes even though they have different learning paths. Therefore, the learning hierarchy diagrams for integers and for fractions were integrated to enhance student learning. Figure 1 shows the learning hierarchy diagrams for integers and fractions. The a-path and b-path are the paths for integers and fractions, respectively, and the numbers in front of a and b represent the learning level.

To compute the learning achievements of students, an inheritance coding approach was used to encode the test questions according to the learning hierarchy paths. Mastering higher-level skills requires students to understand the concepts of lower-level skills. In other words, in inheritance coding, higher-level skills inherit the concepts from lower-level skills. However, higher-level skills add an additional concept, which differentiates them from lower-level skills. Therefore, each skill level inherits concepts from lower skill levels and presents a new concept. However, the lowest skill level only presents its own concept. Each test question was encoded as a vector of binary numbers (i.e., 1 and 0) with the same length as the vector of the selected skill level in the learning hierarchy diagram. A code of 1 indicated that the test question included the skill level concept. Otherwise, the code was 0. The inheritance coding representation is accurate and efficient because it represents the skill level concept in a hierarchical order. For convenience and simplicity, individuals X_j for skill level codes in the learning hierarchy diagram are shown below.

X_j = (x₁, x₂, …, x_i, …, x_ρ)

(1)

where x_i (i = 1, 2, …, ρ) denotes the code for the skill level, its value being 0 or 1, and ρ is the number of selected skill levels in a learning hierarchy diagram. j = 1, 2, …, p_s, where p_s denotes the total number of test questions.

For example, the range of skill levels from 1b to 14b in the fractions unit (Figure 1) is used to generate test questions. For a test question generated from 1b of the fractions unit, the vector for individual X is encoded as X_1b = (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1). For test questions generated from 7b and 14b of the fractions unit, the vectors for individuals Xs are encoded as X_7b = (0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1) and X_14b = (1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1), respectively. For example, the higher skill level X_7b inherits the concepts of lower-level skills 1b to 7b. To solve a test question for 7b of the fractions unit, a student must have learned the concepts for skill levels 1b to 7b.

The inheritance coding procedure uses the following steps to produce p_s individuals.

Step 1.

Design test questions from each skill level in the learning hierarchy diagram. The number of selected skill levels is ρ.

Step 2.

Encode X_j in Equation (1). The value of x_i is 0 or 1. Repeat ρ times, and produce a vector (x₁, x₂, …, x_ρ).

Step 3.

Repeat Step 2 p_s times to produce p_s feasible individuals for test questions.

The questions designed for each skill level in the learning hierarchy diagram are used to test a student, and the accuracy ratio (AR) of the skill level concept is computed to determine the learning achievement of a student. For the skill level concept, AR_i is calculated as follows.

$$A{R}_i=\frac{n_i}{m_i}\times 100\%$$

(2)

where n_i is the number of correct answers a student gave for skill level concept C_i, and m_i is the number of test questions for skill level concept C_i.

The threshold is set to evaluate the learning achievement of a student. The threshold should be set so that the teacher can determine whether a student answered a question correctly due to mastery of the concept or due to a correct guess. The threshold should also indicate whether a student answered a question incorrectly due to failure in the concept or due to a careless mistake. Based on the test data analyses and discussions with the participating teachers in the study, a threshold between 50% and 60% was considered reasonable. As the AR was higher than the threshold for the skill level concept, a teacher could determine whether the student understood the skill level concept. Furthermore, the AR trend in learning achievement for a student was shown from high to low, i.e., progressing from mastery of lower-level concepts to mastery of higher-level concepts.

4. Developing Mathematics Skills Assessment Systems

Figure 2 shows the five phases of ADDIE (analysis, design, development, implementation, and evaluation), which is widely used for instructional design. The “Analysis” phase identifies the probable causes of a performance gap. The “Design” phase verifies the desired performance level and selects appropriate testing methods. The “Development” phase generates and validates the learning resources. The “Implementation” phase prepares the learning environment and engages the students. The last phase, “Evaluation”, assesses the quality of the instructional products and processes [10]. The authors have successfully used ADDIE to develop an auto-reply system for questions and answers in a mathematics speed unit [18,22].

This study used the ADDIE design model throughout the process of developing the mathematics skills assessment system.

4.1. Analysis Phase

The analysis phase clarifies the skills assessment problems, establishes the assessment goals, and identifies the existing knowledge and skills of the learner. The aim is to ensure that the assessment problems can be used to diagnose student learning barriers. In Taiwan, textbooks from Kang-Hsuan edition, Han-Lin edition, and Nan-I edition were integrated in the database of questions for the mathematics skills assessment system. Benchmark mathematics skills for grades 1–9 were selected according to curriculum guidelines. The learning hierarchy for the mathematics unit was then integrated according to the Gagné learning hierarchy. The aim of this phase was to analyze student learning barrier points and to integrate the appropriate test questions for designing learning hierarchy diagrams.

4.2. Design Phase

Based on the results of the analysis phase, learning objectives and assessment questions were selected in the design phase. The test questions were developed and evaluated for each learning hierarchy. The aim was to ensure appropriate test questions were selected for each learning hierarchy.

The skills assessment system designed in this study includes several mathematics units. The test questions for each mathematics unit were designed by professional teachers. Figure 1 shows that the integers unit has 11 (1a–11a) levels, and the fractions unit has 14 (1b–14b) levels. Figure 3 shows that the area computation unit has 10 (1a–10a) levels.

Two course design experts and two teaching experts examined and revised the test questions. Expert validity exceeded 0.8, which indicated that expert agreement was acceptable.

The test questions designed for each skill level were evaluated in terms of their reliability for assessing mastery of each skill level concept. For all skill level concepts, the Cronbach alpha value exceeded 0.8, which indicated good internal reliability.

The test questions designed to assess student learning in mathematics were administered to a group of students two times at a 1-month interval. The Pearson correlation coefficient for test–retest reliability approximated 0.91, which indicated acceptable stability of the test questions over time.

A large set of test questions pertaining to all critical concepts was generated. The questions were then randomly split into two parallel-form sets. The parallel-form test questions were given to the same group of students. The Pearson correlation coefficient obtained for parallel-form reliability approached 0.9, which indicated that both versions had acceptable consistency for assessing student learning achievement.

4.3. Development Phase

In the development phase, the test questions and content were selected, and web-based storyboards and graphics were laid out. Software technologies, including ASP.NET, C#, and Structured Query Language (SQL) server, were integrated in the web-based mathematics skills assessment system. ASP.NET is an open-source server-side web-application framework designed for web development to produce dynamic web pages, developed by Microsoft to allow programmers to build dynamic web sites, applications, and services. The system was created by the ASP.NET Web Form web-based application with C# programming and with an SQL server database. ASP.NET, C#, and the SQL server were all developed by Microsoft, so they had good compatibility. Finally, tests of the system were performed.

The aim was to complete all content and software and to test the mathematics skills assessment system. The system was reviewed by students and teachers during development and then revised according to their feedback.

4.4. Implementation Phase

In the implementation phase, a procedure for training teachers and students was developed. Teachers were trained in using the web-based software, performing tests, and assessing learning outcomes. Students were trained in using the software and completing the testing procedures. Tests of the web-based system were also performed to ensure that it functioned properly.

The aim of the implementation phase was to ensure that the mathematics skills assessment system effectively diagnosed student learning barrier points and provided learning suggestions for students and teachers.

Purposive sampling was performed to evaluate the efficiency and compatibility of the system. The participating students included grade 5 and grade 6 students in Pingtung and Kaohsiung, Taiwan. The teachers actively participated in testing the system and provided test responses.

4.5. Evaluation Phase

The evaluation phase was performed in two parts: formative and summative. Formative evaluation was performed at each stage of the ADDIE process to improve the effectiveness of the system and the content of the test questions. Summative evaluation was performed to understand the effects of the system, to survey user satisfaction, and to survey expert opinions. The aims of the evaluation phase were to use the feedback to revise the system and judge its effectiveness. The results were then used to refine the content of the test questions and web-based storyboards.

A self-administered questionnaire with acceptable expert validity was used to assess student opinions of the mathematics skills assessment system. The overall questionnaire had a Cronbach alpha value higher than 0.785, which indicated acceptable internal reliability.

5. Actual Implementation of the GBLHA Mathematics Skills Assessment System for Diagnosing Student Learning Problems

A practical implementation of the mathematics skills assessment system developed by the GBLHA enabled evaluation of its effectiveness for diagnosing student learning problems. Eighty students used the mathematics skills assessment system to diagnose learning problems. The results are shown below for five representative students in each of the two test subjects.

5.1. Test Subject 1: Fractions Unit

Figure 1 shows that the range of tested material included 1b to 12b in the fractions unit. Therefore, twelve skill level concepts were used to diagnose student learning problems. Three test questions were designed for each skill level, which resulted in 36 questions.

Table 1. Codes for test questions (Q_j) and concepts (C_i) in each skill level for the fractions unit.

Qj\Ci	C12	C11	C10	C9	C8	C7	C6	C5	C4	C3	C2	C1
Q1	0	0	0	0	0	0	0	0	0	0	0	1
Q2	0	0	0	0	0	0	0	0	0	0	0	1
Q3	0	0	0	0	0	0	0	0	0	0	0	1
Q4	0	0	0	0	0	0	0	0	0	0	1	1
Q5	0	0	0	0	0	0	0	0	0	0	1	1
Q6	0	0	0	0	0	0	0	0	0	0	1	1
Q7	0	0	0	0	0	0	0	0	0	1	1	1
Q8	0	0	0	0	0	0	0	0	0	1	1	1
Q9	0	0	0	0	0	0	0	0	0	1	1	1
Q10	0	0	0	0	0	0	0	0	1	1	1	1
Q11	0	0	0	0	0	0	0	0	1	1	1	1
Q12	0	0	0	0	0	0	0	0	1	1	1	1
Q13	0	0	0	0	0	0	0	1	1	1	1	1
Q14	0	0	0	0	0	0	0	1	1	1	1	1
Q15	0	0	0	0	0	0	0	1	1	1	1	1
Q16	0	0	0	0	0	0	1	1	1	1	1	1
Q17	0	0	0	0	0	0	1	1	1	1	1	1
Q18	0	0	0	0	0	0	1	1	1	1	1	1
Q19	0	0	0	0	0	1	1	1	1	1	1	1
Q20	0	0	0	0	0	1	1	1	1	1	1	1
Q21	0	0	0	0	0	1	1	1	1	1	1	1
Q22	0	0	0	0	1	1	1	1	1	1	1	1
Q23	0	0	0	0	1	1	1	1	1	1	1	1
Q24	0	0	0	0	1	1	1	1	1	1	1	1
Q25	0	0	0	1	1	1	1	1	1	1	1	1
Q26	0	0	0	1	1	1	1	1	1	1	1	1
Q27	0	0	0	1	1	1	1	1	1	1	1	1
Q28	0	0	1	1	1	1	1	1	1	1	1	1
Q29	0	0	1	1	1	1	1	1	1	1	1	1
Q30	0	0	1	1	1	1	1	1	1	1	1	1
Q31	0	1	1	1	1	1	1	1	1	1	1	1
Q32	0	1	1	1	1	1	1	1	1	1	1	1
Q33	0	1	1	1	1	1	1	1	1	1	1	1
Q34	1	1	1	1	1	1	1	1	1	1	1	1
Q35	1	1	1	1	1	1	1	1	1	1	1	1
Q36	1	1	1	1	1	1	1	1	1	1	1	1
Sum of concepts (mi)	3	6	9	12	15	18	21	24	27	30	33	36

shows the inheritance codes for the test questions (Q_j) and concepts (C_i) in each skill level, where j = 1, …, 36 and i = 1, …, 12. The total number of test questions, ps, is 36. The numbers of test questions, m_i, for skill level concept C_i (i = 1, 2, …, 12) are 36, 33, …, 3, respectively. To account for incorrect answers resulting from carelessness rather than lack of understanding, the threshold was set to 50%, recognized by the teacher. An AR exceeding 50% for a skill level concept indicated that the student understood the skill level concept.

Table 2. Learning achievement results for student 1, who correctly answered Q₁–Q₁₇, Q₁₉–Q₂₀, Q₂₄–Q₂₆, Q₂₉–Q₃₂, and Q₃₆.

	C12	C11	C10	C9	C8	C7	C6	C5	C4	C3	C2	C1
No. of correct concepts	1	3	5	7	8	10	12	15	18	21	24	27
Sum of concepts	3	6	9	12	15	18	21	24	27	30	33	36
AR (%)	33%	50%	56%	58%	53%	56%	57%	63%	67%	70%	73%	75%

shows the learning achievement results for student 1, who correctly answered questions Q₁–Q₁₇, Q₁₉, Q₂₀, Q₂₄–Q₂₆, Q₂₉–Q₃₂, and Q₃₆. Student 1 correctly answered 27 of the 36 test questions. Traditionally, if the 36 test questions are worth 100 points, student 1 scored 75 points, which can be interpreted as an understanding of 75% of the tested concepts. However, according to the learning achievement analysis at a threshold of 50%, student 1 understood 92% (C₁–C₁₁) of concepts C₁–C₁₂. The teacher inferred that student 1 made many careless mistakes. The AR trend in learning achievement was from high (75%) to low (50%) from C₁ to C₁₁, respectively.

Table 3. Learning achievement results for student 2, who correctly answered Q₁–Q₁₂, Q₁₅–Q₂₃, Q₂₆, Q₂₈–Q₃₀, Q₃₂, and Q₃₅.

	C12	C11	C10	C9	C8	C7	C6	C5	C4	C3	C2	C1
No. of correct concepts	1	2	5	6	8	11	14	15	18	21	24	27
Sum of concepts	3	6	9	12	15	18	21	24	27	30	33	36
AR (%)	33%	33%	56%	50%	53%	61%	67%	63%	67%	70%	73%	75%

shows the learning achievement results for student 2, who correctly answered questions Q₁–Q₁₂, Q₁₅–Q₂₃, Q₂₆, Q₂₈–Q₃₀, Q₃₂, and Q₃₅. Student 2 correctly answered 27 of the 36 test questions. If the 36 test questions are worth 100 points, student 2 scored 75 points, which would traditionally be interpreted as an understanding of 75% of the tested concepts. However, according to the learning achievement analysis at a threshold of 50%, student 2 only understood 83% (C₁–C₁₀) of concepts C₁–C₁₂. The interpretation by the teacher was that student 2 needed an improved understanding of concepts C₁₁–C₁₂. The teacher also inferred that student 2 made several careless mistakes.

Table 4. Learning achievement results for student 3, who correctly answered Q₁, Q₃, Q₄, Q₇, Q₈, Q₁₀, Q₁₄, Q₁₅, Q₁₇, Q₁₈, Q₂₁, Q₂₃, Q₂₆, Q₂₈, Q₃₀, Q₃₃, and Q₃₆.

	C12	C11	C10	C9	C8	C7	C6	C5	C4	C3	C2	C1
No. of correct concepts	1	2	4	5	6	7	9	11	12	14	15	17
Sum of concepts	3	6	9	12	15	18	21	24	27	30	33	36
AR (%)	33%	33%	44%	42%	40%	39%	43%	46%	44%	47%	45%	47%

shows the learning achievement results for student 3, who correctly answered questions Q₁, Q₃, Q₄, Q₇, Q₈, Q₁₀, Q₁₄, Q₁₅, Q₁₇, Q₁₈, Q₂₁, Q₂₃, Q₂₆, Q₂₈, Q₃₀, Q₃₃, and Q₃₆. Student 3 correctly answered 17 of the 36 test questions. If the 36 test questions are worth 100 points, student 3 scored 47 points, which would traditionally be interpreted as an understanding of 47% of the tested concepts. However, according to the learning achievement analysis at a threshold of 50%, student 3 did not completely understand any concepts because none of the AR values for the 12 concepts exceeded 50%. Since the AR for learning achievement showed no trend from high to low from C₁ to C₁₂, the teacher inferred that student 3 did not understand any concepts and answered the test questions by guessing. This example shows the tolerance behavior of inheritance coding with AR. Even if a student correctly answers some test questions, an AR value lower than the threshold would still inform the teacher that the student does not completely understand those concepts.

Table 5. Learning achievement results for student 4, who correctly answered Q₁–Q₂₄, Q₂₆–Q₂₉, Q₃₁, Q₃₂, Q₃₅, and Q₃₆.

	C12	C11	C10	C9	C8	C7	C6	C5	C4	C3	C2	C1
No. of correct concepts	2	4	6	8	11	14	17	20	23	26	29	32
Sum of concepts	3	6	9	12	15	18	21	24	27	30	33	36
AR (%)	67%	67%	67%	67%	73%	78%	81%	83%	85%	87%	88%	89%

shows the learning achievement results for student 4, who correctly answered questions Q₁–Q₂₄, Q₂₆–Q₂₉, Q₃₁, Q₃₂, Q₃₅, and Q₃₆. Student 4 correctly answered 32 questions of the 36 test questions. Traditionally, if the 36 test questions are worth 100 points, student 4 scored 89 points, which can be interpreted as an understanding of 89% of the tested concepts. According to the learning achievement analysis at a threshold of 50%, student 4 understood 100% of concepts C₁–C₁₂. In the example, the AR trend for learning achievement was from high (89%) to low (67%) from C₁ to C₁₂, respectively. The teacher inferred that student 4 made careless mistakes.

Table 6. Learning achievement results for student 5, who correctly answered Q₁, Q₂, Q₅, Q₆, Q₉–Q₁₁, Q₁₄–Q₁₇, Q₂₀–Q₂₂, Q₂₅, Q₂₆, Q₂₉, and Q₃₀.

	C12	C11	C10	C9	C8	C7	C6	C5	C4	C3	C2	C1
No. of correct concepts	0	0	2	4	5	7	9	11	13	14	16	18
Sum of concepts	3	6	9	12	15	18	21	24	27	30	33	36
AR (%)	0%	0%	22%	33%	33%	39%	43%	46%	48%	47%	48%	50%

shows the learning achievement results for student 5, who correctly answered questions Q₁, Q₂, Q₅, Q₆, Q₉–Q₁₁, Q₁₄–Q₁₇, Q₂₀-Q₂₂, Q₂₅, Q₂₆, Q₂₉, and Q₃₀. Student 5 correctly answered 18 questions of the 36 test questions. Traditionally, if the 36 test questions are worth 100 points, student 5 scored 50 points, which can be interpreted as an understanding of 50% of the tested concepts. However, according to the learning achievement analysis at a threshold of 50%, student 5 only understood 8% of the concepts for C₁. The teacher concluded that student 5 needed an improved understanding of C₂–C₁₂ and inferred that student 5 answered the test questions by guessing. In the example, the AR trend in learning achievement was from high (50%) to low (0%) from C₁ to C₁₂, respectively.

For students 1–5, the learning achievements for the fractions unit were validated by their mathematics teachers. The assessment system accurately diagnosed student learning barriers and provided students and teachers with suggestions for improving learning efficiency.

5.2. Test Subject 2: Area Computation Unit

Figure 3 shows that concepts 1a to 10a of the area computation unit were tested. Therefore, ten skill level concepts were used to diagnose student learning problems. Three test questions were designed for each skill level, which resulted in 30 test questions.

Table 7. Codes for test questions (Q_j) and concepts (C_i) in each skill level for the area computation unit.

Qj\Ci	C10	C9	C8	C7	C6	C5	C4	C3	C2	C1
Q1	0	0	0	0	0	0	0	0	0	1
Q2	0	0	0	0	0	0	0	0	0	1
Q3	0	0	0	0	0	0	0	0	0	1
Q4	0	0	0	0	0	0	0	0	1	1
Q5	0	0	0	0	0	0	0	0	1	1
Q6	0	0	0	0	0	0	0	0	1	1
Q7	0	0	0	0	0	0	0	1	1	1
Q8	0	0	0	0	0	0	0	1	1	1
Q9	0	0	0	0	0	0	0	1	1	1
Q10	0	0	0	0	0	0	1	1	1	1
Q11	0	0	0	0	0	0	1	1	1	1
Q12	0	0	0	0	0	0	1	1	1	1
Q13	0	0	0	0	0	1	1	1	1	1
Q14	0	0	0	0	0	1	1	1	1	1
Q15	0	0	0	0	0	1	1	1	1	1
Q16	0	0	0	0	1	1	1	1	1	1
Q17	0	0	0	0	1	1	1	1	1	1
Q18	0	0	0	0	1	1	1	1	1	1
Q19	0	0	0	1	1	1	1	1	1	1
Q20	0	0	0	1	1	1	1	1	1	1
Q21	0	0	0	1	1	1	1	1	1	1
Q22	0	0	1	1	1	1	1	1	1	1
Q23	0	0	1	1	1	1	1	1	1	1
Q24	0	0	1	1	1	1	1	1	1	1
Q25	0	1	1	1	1	1	1	1	1	1
Q26	0	1	1	1	1	1	1	1	1	1
Q27	0	1	1	1	1	1	1	1	1	1
Q28	1	1	1	1	1	1	1	1	1	1
Q29	1	1	1	1	1	1	1	1	1	1
Q30	1	1	1	1	1	1	1	1	1	1
Sum of concepts (mi)	3	6	9	12	15	18	21	24	27	30

shows the inheritance codes for the test questions (Q_j) and concepts (C_i) for all skill levels, where j = 1, …, 30 and i = 1, …, 10. The total number of test questions, ps, is 30. The numbers of test questions, m_i, for skill level concept C_i (i = 1, 2, …, 10) are 30, 27, …, 3, respectively. To account for incorrect answers resulting from carelessness rather than poor understanding, the threshold was set to 50%. A teacher could confirm that the student understood the skill level concept if the student had an AR higher than 50%.

Table 8. Learning achievement results for student 1, who correctly answered Q₁–Q₁₃, Q₁₅–Q₁₇, Q₁₉, Q₂₀, Q₂₄, Q₂₅, and Q₃₀.

	C10	C9	C8	C7	C6	C5	C4	C3	C2	C1
No. of correct concepts	1	2	3	5	7	9	12	15	18	21
Sum of concepts	3	6	9	12	15	18	21	24	27	30
AR (%)	33%	33%	33%	42%	47%	50%	57%	63%	67%	70%

shows the learning achievement results for student 1, who correctly answered questions Q₁–Q₁₃, Q₁₅–Q₁₇, Q₁₉, Q₂₀, Q₂₄, Q₂₅, and Q₃₀. Student 1 correctly answered 21 questions of the 30 test questions. Traditionally, if the 30 test questions are worth 100 points, student 1 scored 70 points, which can be interpreted as an understanding of 70% of the tested concepts. However, according to the learning achievement analysis at a threshold of 50%, student 1 only understood 50% (C₁–C₅) of concepts C₁–C₁₀. The teacher concluded that student 1 needed an improved understanding of concepts C₆–C₁₀. In the example, the AR trend in learning achievement was from high (70%) to low (50%) from C₁ to C₅, respectively.

Table 9. Learning achievement results for student 2, who correctly answered Q₁–Q₁₂, Q₁₄–Q₂₀, Q₂₂–Q₂₃, Q₂₆, and Q₂₈.

	C10	C9	C8	C7	C6	C5	C4	C3	C2	C1
No. of correct concepts	1	2	4	6	9	11	14	17	20	23
Sum of concepts	3	6	9	12	15	18	21	24	27	30
AR (%)	33%	33%	44%	50%	60%	61%	67%	71%	74%	77%

shows the learning achievement results for student 2, who correctly answered questions Q₁–Q₁₂, Q₁₄–Q₂₀, Q₂₂–Q₂₃, Q₂₆, and Q₂₈. Student 2 correctly answered 23 out of 30 test questions. Traditionally, if the 30 test questions are worth 100 points, student 2 scored 77 points, which can be interpreted as an understanding of 77% of the tested concepts. However, according to the learning achievement analysis at a threshold of 50%, student 2 only understood 70% (C₁–C₇) of concepts C₁–C₁₀. The teacher concluded that student 2 needed an improved understanding of C₈–C₁₀. In this example, the AR trend in learning achievement was from high (77%) to low (50%) for C₁ to C₇, respectively.

Table 10. Learning achievement results for student 3, who correctly answered Q₁, Q₃, Q₄, Q₇, Q₉, Q₁₀, Q₁₄, Q₁₅, Q₁₇–Q₁₉, Q₂₁, Q₂₃, Q₂₅, Q₂₇, and Q₂₉.

	C10	C9	C8	C7	C6	C5	C4	C3	C2	C1
No. of correct concepts	1	3	4	6	8	10	11	13	14	16
Sum of concepts	3	6	9	12	15	18	21	24	27	30
AR (%)	33%	50%	44%	50%	53%	56%	52%	54%	52%	53%

shows the learning achievement results for student 3, who correctly answered questions Q₁, Q₃, Q₄, Q₇, Q₉, Q₁₀, Q₁₄, Q₁₅, Q₁₇–Q₁₉, Q₂₁, Q₂₃, Q₂₅, Q₂₇, and Q₂₉. Student 3 correctly answered 16 out of 30 test questions. Traditionally, if the 30 test questions are worth 100 points, student 3 scored 53 points, which can be interpreted as an understanding of 53% of the tested concepts. However, according to the learning achievement analysis at a threshold of 50%, student 3 understood 70% (C₁–C₇) of concepts C₁–C₁₀ or 90% (C₁–C₉) of concepts C₁–C₁₀. The teacher inferred that student 3 made many careless mistakes.

Table 11. Learning achievement results for student 4, who correctly answered Q₁–Q₂₅, Q₂₇, Q₂₉, and Q₃₀.

	C10	C9	C8	C7	C6	C5	C4	C3	C2	C1
No. of correct concepts	2	4	7	10	13	16	19	22	25	28
Sum of concepts	3	6	9	12	15	18	21	24	27	30
AR (%)	67%	67%	78%	83%	87%	89%	90%	92%	93%	93%

shows learning achievement results for student 4, who correctly answered questions Q₁–Q₂₅, Q₂₇, Q₂₉, and Q₃₀. Student 4 correctly answered 28 out of 30 test questions. Traditionally, if the 30 questions are worth 100 points, student 4 scored 93 points, which can be interpreted as an understanding of 93% of the tested concepts. According to the learning achievement analysis at a threshold of 50%, student 4 understood 100% of concepts C₁–C₁₂. In the example, the AR trend for learning achievement was from high (93%) to low (67%) for C₁ to C₁₂, respectively. The teacher inferred that student 4 made careless mistakes.

Table 12. Learning achievement results for student 5, who correctly answered Q₁, Q₃, Q₄, Q₆, Q₉, Q₁₀, Q₁₂, Q₁₄, Q₁₆, Q₁₈, Q₂₁, Q₂₄, Q₂₆, and Q₃₀.

	C10	C9	C8	C7	C6	C5	C4	C3	C2	C1
No. of correct concepts	1	2	3	4	6	7	9	10	12	14
Sum of concepts	3	6	9	12	15	18	21	24	27	30
AR (%)	33%	33%	33%	33%	40%	39%	43%	42%	44%	47%

shows the learning achievement results for student 5, who correctly answered questions Q₁, Q₃, Q₄, Q₆, Q₉, Q₁₀, Q₁₂, Q₁₄, Q₁₆, Q₁₈, Q₂₁, Q₂₄, Q₂₆, and Q₃₀. Student 5 correctly answered 14 out of 30 test questions. Traditionally, if the 30 test questions are worth 100 points, student 5 scored 47 points, which can be interpreted as an understanding of 47% of the tested concepts. However, according to the learning achievement analysis at a threshold of 50%, student 5 did not completely understand any concepts because none of the AR values for the 12 concepts exceeded 50%. The teacher inferred that student 5 answered the test questions by guessing and concluded that student 5 needed an improved understanding of concepts C₁–C₁₂. In this example, the AR trend for learning achievement was from high (47%) to low (33%) for C₁ to C₁₂, respectively.

For the area computation unit, the learning achievements of students 1–5 were validated by their mathematics teachers. The assessment system accurately diagnosed student learning barrier points and provided students and teachers with suggestions for improving learning efficiency.

The above results obtained in practical applications of the GBLHA indicate that it can accurately diagnose student learning barrier points and provide students and teachers with suggestions for improving learning efficiency. The GBLHA for developing mathematics skills assessment systems has three notable requirements. The test questions for each level of a mathematics unit should be carefully designed according to the Gagné learning hierarchy theory and according to the expertise of the elementary school teacher in assessing student learning achievement. Based on the learning hierarchy paths, the test questions for each level must be designed to enable inheritance coding. When a student is tested by the proposed assessment system, the AR trend in learning achievement from high to low from lower-level to higher-level concepts, respectively, is shown to depict barriers in student learning. Notably, some elementary schools in Taiwan have already adopted the proposed GBLHA for assessing student learning achievement in mathematics. Based on the test data analyses, student learning barrier points, and discussions with teachers, the threshold should be set to between 50% and 60% when evaluating the learning achievement of a student.

Figure 4 shows part of the mathematics skills assessment system. Students were required to complete the questionnaire after participating in the learning activities to survey their satisfaction with the mathematics skills assessment system, including its system interface, function, appearance, operation, stability, and response speed. Each item was scored from 1 (low satisfaction) to 10 (high satisfaction). Among the eighty students who used the system, 10% of the students gave 6 points, 28% gave 7 points, 32% gave 8 points, 20% gave 9 points, and 10% gave 10 points. The questionnaire results indicate that most students who used the system rated their satisfaction as 6 or higher. In terms of satisfaction with the system, 7 points or more can be interpreted as 90% satisfaction. Therefore, the system appears to have practical applications for diagnosing student learning achievement.

6. Conclusions

By integrating Gagné learning hierarchy theory and an inheritance coding approach, the GBLHA provided a systematic method of building a mathematics skills assessment system for diagnosing student learning problems. The ADDIE design model was used throughout the process of developing the system. The main contribution of this study is the use of the proposed GBLHA to build a system that can accurately identify learning barriers and can provide students and teachers with suggestions for improving learning efficiency in real-world classroom settings with fewer tests. The related statistical indices for the test questions used in the mathematics assessment system revealed acceptable expert validity, internal reliability, test–retest reliability, and parallel-form reliability. The function and performance of the system were rated favorably by the students. Practical applications of the system in elementary schools showed that the GBLHA is superior to conventional methods in terms of accuracy in assessing student learning achievement. Since the learning achievements of students who used the system were validated by their mathematics teachers, we conclude that the proposed GBLHA has excellent performances in mathematics skills assessment. Future research will add different mathematics units to test the learning problems of a large number of students and further improve the performance of the system.