Development of a Computerized Adaptive Assessment and Learning System for Mathematical Ability Based on Cognitive Diagnosis
Abstract
1. Introduction
2. Cognitive Diagnostic Assessment Foundation System Construction
2.1. Selection of Knowledge Modules
2.2. Construction of Cognitive Models
2.3. Developing the Q-Matrices
2.4. Developing Assessment Tools
3. Operation Mechanism of Computerized Adaptive Learning System Based on Cognitive Diagnosis
3.1. Test System
3.2. Diagnosis System
3.3. Recommended System
3.4. Case Analysis and Effectiveness Description
- (1)
- Testing process
- (2)
- Diagnostic process
- (3)
- Referral process
- (4)
- Effectiveness interview analysis
Student J1-1-3: During my study of the solid geometry part, I never had a good grasp of the problem of determining the parallel face of the surface. After completing the corresponding tests, the system pointed out that the probability of mastering this knowledge point was 35%, and pushed me a series of learning resources.
Student J1-4-11: The system has pushed me a lot of practice questions and explanations about the relationship between circles and the position of circles, which are easy to understand and I think it has helped me a lot.
Teacher J1-1: Based on the observation of the students’ answers and the diagnoses given, I think this system is very meritorious. Be able to give some valuable learning resources.
Teacher J1-4: This system may have the problem of a large number of questions in each push. On the whole, it has a great enlightening effect on teachers’ precision teaching.
4. Discussion
5. Research Limitations
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Aboagye, Emmanuel, Joseph Anthony Yawson, and Kofi Nyantakyi Appiah. 2020. COVID-19 and E-Learning: The Challenges of Students in Tertiary Institutions. Social Education Research 2: 1–8. [Google Scholar] [CrossRef]
- Arroyo, Ivon, Beverly Park Woolf, Winslow Burelson, Kasia Muldner, Dovan Rai, and Minghui Tai. 2014. A Multimedia Adaptive Tutoring System for Mathematics That Addresses Cognition, Metacognition and Affect. International Journal of Artificial Intelligence in Education 24: 387–426. [Google Scholar] [CrossRef]
- Attri, Rajesh, Nikhil Dev, and Vivek Sharma. 2013a. Interpretive Structural Modelling (ISM) Approach: An Overview. Research Journal of Management Sciences 2: 3–8. [Google Scholar]
- Attri, Rajesh, Sandeep Grover, Nikhil Dev, and Deepak Kumar. 2013b. Analysis of Barriers of Total Productive Maintenance (TPM). International Journal of System Assurance Engineering and Management 4: 365–77. [Google Scholar] [CrossRef]
- Ayotola, Aremu, and Tella Adedeji. 2009. The Relationship between Mathematics Self-Efficacy and Achievement in Mathematics. Procedia Social and Behavioral Sciences 1: 953–57. [Google Scholar] [CrossRef]
- Baker, Ryan S. J. D., and Kalina Yacef. 2009. The State of Educational Data Mining in 2009: A Review and Future Visions. Journal of Educational Data Mining 1: 3–17. [Google Scholar] [CrossRef]
- Chaachoua, Hamid, Jean-François Nicaud, Alain Bronner, and Denis Bouhineau. 2004. Aplusix, a Learning Environment for Algebra, Actual Use and Benefits. Available online: http://hal.archives-ouvertes.fr/docs/00/19/03/93/PDF/Chaachoua-h-2004.pdf (accessed on 8 May 2025).
- Chen, Enhong, Qi Liu, Shijin Wang, Zhenya Huang, Yu Su, Peng Ding, Jianhui Ma, and Bo Zhu. 2021. Key Techniques and Application of Intelligent Education Oriented Adaptive Learning. CAAI Transactions on Intelligent Systems 16: 886–98. [Google Scholar] [CrossRef]
- Choi, Kyong Mi, Young-Sun Lee, and Yoon Soo Park. 2015. What CDM Can Tell about What Students Have Learned: An Analysis of TIMSS Eighth Grade Mathematics. Eurasia Journal of Mathematics, Science and Technology Education 11: 1563–77. [Google Scholar] [CrossRef]
- Colchester, Khalid, Hani Hagras, Daniyal Alghazzawi, and Ghadah Aldabbagh. 2017. A Survey of Artificial Intelligence Techniques Employed for Adaptive Educational Systems within E-Learning Platforms. Journal of Artificial Intelligence and Soft Computing Research 7: 47–64. [Google Scholar] [CrossRef]
- Crocker, Linda M., and James Algina. 1986. Introduction to Classical and Modern Test Theory. Orlando: Holt, Rinehart and Winston. [Google Scholar]
- De la Torre, Jimmy. 2011. The Generalized DINA Model Framework. Psychometrika 76: 179–99. [Google Scholar] [CrossRef]
- DiBello, Louis V., William F. Stout, and Louis A. Roussos. 2012. Unified Cognitive/Psychometric Diagnostic Assessment Likelihood-Based Classification Techniques. In Cognitively Diagnostic Assessment. New York: Routledge, pp. 361–89. [Google Scholar]
- D’Mello, Sidney, and Art Graesser. 2012. Dynamics of Affective States during Complex Learning. Learning and Instruction 22: 145–57. [Google Scholar] [CrossRef]
- Dogan, Enis, and Kikumi Tatsuoka. 2008. An International Comparison Using a Diagnostic Testing Model: Turkish Students’ Profile of Mathematical Skills on TIMSS-R. Educational Studies in Mathematics 68: 263–72. [Google Scholar] [CrossRef]
- Dunlosky, John, and Katherine A. Rawson. 2015. Practice Tests, Spaced Practice, and Successive Relearning: Tips for Classroom Use and for Guiding Students’ Learning. Scholarship of Teaching and Learning in Psychology 1: 72–78. [Google Scholar] [CrossRef]
- Gierl, Mark J. 2007. Making Diagnostic Inferences about Cognitive Attributes Using the Rule-Space Model and Attribute Hierarchy Method. Journal of Educational Measurement 44: 325–40. [Google Scholar] [CrossRef]
- Gierl, Mark J., Cecilia Alves, and Renate Taylor Majeau. 2010. Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees’ Knowledge and Skills in Mathematics: An Operational Implementation of Cognitive Diagnostic Assessment. International Journal of Testing 10: 318–41. [Google Scholar] [CrossRef]
- Griff, Edwin R., and Stephen F. Matter. 2013. Evaluation of an Adaptive Online Learning System. British Journal of Educational Technology 44: 170–76. [Google Scholar] [CrossRef]
- He, Ningwen. 2025. Analysis of College Entrance Examination Mathematics Questions and Teaching Implications: A Case Study of the 2024 National New College Entrance Examination Mathematics Paper II. Gansu Education 3: 159–64. [Google Scholar]
- Heffernan, Neil T., and Cristina Lindquist Heffernan. 2014. The ASSISTments Ecosystem: Building a Platform That Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching. International Journal of Artificial Intelligence in Education 24: 470–97. [Google Scholar] [CrossRef]
- Henson, Robert A., Jonathan L. Templin, and John T. Willse. 2009. Defining a Family of Cognitive Diagnosis Models Using Log-Linear Models with Latent Variables. Psychometrika 74: 191–210. [Google Scholar] [CrossRef]
- Huntington, Edward V. 1919. Mathematics and Statistics, with an Elementary Account of the Correlation Coefficient and the Correlation Ratio. The American Mathematical Monthly 26: 421–35. [Google Scholar] [CrossRef]
- Khosravi, Hassan, Shazia Sadiq, and Dragan Gasevic. 2020. Development and Adoption of an Adaptive Learning System: Reflections and Lessons Learned. Paper presented at 51st ACM Technical Symposium on Computer Science Education (SIGCSE ’20), Portland, OR, USA, March 11–14; New York: ACM, pp. 58–64. [Google Scholar] [CrossRef]
- Kibuku, Njeri, Daniel Orwa Ochieng, and Agnes Nduku Wausi. 2020. E-Learning Challenges Faced by Universities in Kenya: A Literature Review. Electronic Journal of E-Learning 18: 150–61. [Google Scholar] [CrossRef]
- Koedinger, Kenneth R., Albert T. Corbett, and Charles Perfetti. 2012. The Knowledge-Learning-Instruction Framework: Bridging the Science-Practice Chasm to Enhance Robust Student Learning. Cognitive Science 36: 757–98. [Google Scholar] [CrossRef]
- Landis, J. Richard, and Gary G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics, 159–74. [Google Scholar] [CrossRef]
- Leighton, Jacqueline, and Mark Gierl. 2007a. Cognitive Diagnostic Assessment for Education: Theory and Applications. Cambridge: Cambridge University Press. Available online: http://dx.doi.org/10.1017/CBO9780511611186 (accessed on 8 May 2025).
- Leighton, Jacqueline P., and Mark J. Gierl. 2007b. Defining and Evaluating Models of Cognition Used in Educational Measurement to Make Inferences about Examinees’ Thinking Processes. Educational Measurement: Issues and Practice 26: 3–16. [Google Scholar] [CrossRef]
- Leighton, Jacqueline P., Mark J. Gierl, and Stephen M. Hunka. 2004. The Attribute Hierarchy Method for Cognitive Assessment: A Variation on Tatsuoka’s Rule-Space Approach. Journal of Educational Measurement 41: 205–37. [Google Scholar] [CrossRef]
- Liu, Dan. 2025. Characteristics Analysis and Pedagogical Implications of Mathematical Modeling in College Entrance Examination Mathematics Questions. Mathematics Learning and Research 9: 50–53. [Google Scholar]
- Liu, Jian, Lintao Fu, and Xiao Xuan Xie. 2025. Analysis of Consistency Changes between Mathematics Test Questions and the New Curriculum Standards under the Context of the New College Entrance Examination. Teaching & Administration 12: 84–88. [Google Scholar]
- Liu, Jingchen, Zhiliang Ying, and Stephanie Zhang. 2015. A Rate Function Approach to Computerized Adaptive Testing for Cognitive Diagnosis. Psychometrika 80: 468–90. [Google Scholar] [CrossRef]
- Mavroudi, Anna, Michail Giannakos, and John Krogstie. 2018. Supporting Adaptive Learning Pathways through the Use of Learning Analytics: Developments, Challenges and Future Opportunities. Interactive Learning Environments 26: 206–20. [Google Scholar] [CrossRef]
- MOE, Ministry of Education of China. 2018. High School Mathematics Curriculum Standards (2017 Edition); Beijing: People’s Education Press.
- Nicaud, Jean-FranÇois, Denis Bouhineau, and Hamid Chaachoua. 2004. Mixing Microworld and CAS Features in Building Computer Systems That Help Students Learn Algebra. International Journal of Computers for Mathematical Learning 9: 169–211. [Google Scholar] [CrossRef]
- Nichols, Paul D. 1994. A Framework for Developing Cognitively Diagnostic Assessments. Review of Educational Research 64: 575–603. [Google Scholar] [CrossRef]
- Pekrun, Reinhard. 2006. The Control-Value Theory of Achievement Emotions: Assumptions, Corollaries, and Implications for Educational Research and Practice. Educational Psychology Review 18: 315–41. [Google Scholar] [CrossRef]
- Pekrun, Reinhard, Stephanie Lichtenfeld, Herbert W. Marsh, Kou Murayama, and Thomas Goetz. 2017. Achievement Emotions and Academic Performance: Longitudinal Models of Reciprocal Effects. Child Development 88: 1653–70. [Google Scholar] [CrossRef]
- Raj, Tajinder, and Ravi Shankar. 2005. Analysis of Interactions among the Barriers of Reverse Logistics. Technological Forecasting and Social Change 72: 1011–29. [Google Scholar] [CrossRef]
- Raj, Tajinder, Ravi Shankar, and Manoj Kumar Tiwari. 2005. Productivity Improvement of a Computer Hardware Supply Chain. International Journal of Productivity and Performance Management 54: 239–55. [Google Scholar] [CrossRef]
- Raj, Tajinder, Ravi Shankar, and Mohammad Suhaib. 2008. An ISM Approach for Modelling the Enablers of Flexible Manufacturing System: The Case for India. International Journal of Production Research 46: 6883–912. [Google Scholar] [CrossRef]
- Richardson, Michelle, Charles Abraham, and Rod Bond. 2012. Psychological Correlates of University Students’ Academic Performance: A Systematic Review and Meta-Analysis. Psychological Bulletin 138: 353–87. [Google Scholar] [CrossRef]
- Rittle-Johnson, Bethany, Michael Schneider, and Jon R. Star. 2015. Not a One-Way Street: Bidirectional Relations between Procedural and Conceptual Knowledge of Mathematics. Educational Psychology Review 27: 587–97. [Google Scholar] [CrossRef]
- Rupp, André A., and Jonathan L. Templin. 2008. Unique Characteristics of Diagnostic Classification Models: A Comprehensive Review of the Current State-of-the-Art. Measurement 6: 219–62. [Google Scholar] [CrossRef]
- Ryan, Richard M., and Edward L. Deci. 2000. Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. Contemporary Educational Psychology 25: 54–67. [Google Scholar] [CrossRef] [PubMed]
- Ryan, Richard M., and Edward L. Deci. 2020. Intrinsic and Extrinsic Motivation from a Self-Determination Theory Perspective: Definitions, Theory, Practices, and Future Directions. Contemporary Educational Psychology 61: 101860. [Google Scholar] [CrossRef]
- Siemens, George, and Ryan S. J. d. Baker. 2012. Learning Analytics and Educational Data Mining: Towards Communication and Collaboration. Paper presented at 2nd International Conference on Learning Analytics and Knowledge, Vancouver, Canada, April 29–May 2; pp. 252–54. [Google Scholar] [CrossRef]
- Singh, Vandana, and Alexander Thurman. 2019. How Many Ways Can We Define Online Learning? A Systematic Literature Review of Definitions of Online Learning (1988–2018). American Journal of Distance Education 33: 289–306. [Google Scholar] [CrossRef]
- Sun, Siyu, Xiaopeng Wu, and Tianshu Xu. 2023. A Theoretical Framework for a Mathematical Cognitive Model for Adaptive Learning Systems. Behavioral Sciences 13: 406. [Google Scholar] [CrossRef]
- Talib, Faisal, Zillur Rahman, and M. N. Qureshi. 2011. Analysis of Interaction among the Barriers to Total Quality Management Implementation Using Interpretive Structural Modeling Approach. Benchmarking: An International Journal 18: 563–87. [Google Scholar] [CrossRef]
- Tatsuoka, Kikumi K. 1983. Rule Space: An Approach for Dealing with Misconceptions Based on Item Response Theory. Journal of Educational Measurement 20: 345–54. [Google Scholar] [CrossRef]
- Tatsuoka, Kikumi K. 2009. Cognitive Assessment: An Introduction to the Rule Space Method. New York: Routledge. [Google Scholar]
- Templin, Jonathan, and Laine Bradshaw. 2013. Measuring the Reliability of Diagnostic Classification Model Examinee Estimates. Journal of Classification 30: 251–75. [Google Scholar] [CrossRef]
- Templin, Jonathan, and Robert A. Henson. 2006. Measurement of Psychological Disorders Using Cognitive Diagnosis Models. Psychological Methods 11: 287–305. [Google Scholar] [CrossRef] [PubMed]
- Templin, Jonathan, and Robert A. Henson. 2010. Diagnostic Measurement: Theory, Methods, and Applications. New York: Guilford Press. [Google Scholar]
- Tseng, Judy C. R., Hui-Chun Chu, Gwo-Jen Hwang, and Chin-Chung Tsai. 2008. Development of an Adaptive Learning System with Two Sources of Personalization Information. Computers & Education 51: 776–86. [Google Scholar] [CrossRef]
- Vanlehn, Kurt. 2011. The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems. Educational Psychologist 46: 197–221. [Google Scholar] [CrossRef]
- Van Merriënboer, Jeroen J. G., and John Sweller. 2010. Cognitive Load Theory in Health Professional Education: Design Principles and Strategies. Medical Education 44: 85–93. [Google Scholar] [CrossRef]
- Von Davier, Matthias. 2008. A General Diagnostic Model Applied to Language Testing Data. British Journal of Mathematical and Statistical Psychology 61: 287–307. [Google Scholar] [CrossRef]
- Wu, Xiaopeng, Rongxiu Wu, Hua-Hua Chang, Qiping Kong, and Yi Zhang. 2020. International Comparative Study on PISA Mathematics Achievement Test Based on Cognitive Diagnostic Models. Frontiers in Psychology 11: 2230. [Google Scholar] [CrossRef]
- Wu, Xiaopeng, Yi Zhang, Rongxiu Wu, Xiuxiu Tang, and Tianshu Xu. 2022. Cognitive Model Construction and Assessment of Data Analysis Ability Based on CDA. Frontiers in Psychology 13: 1009142. [Google Scholar] [CrossRef]
- Xie, Haoran, Hui-Chun Chu, Gwo-Jen Hwang, and Chun-Chieh Wang. 2019. Trends and Development in Technology-Enhanced Adaptive/Personalized Learning: A Systematic Review of Journal Publications from 2007 to 2017. Computers & Education 140: 103599. [Google Scholar] [CrossRef]
- Yadav, Devendra K., and Akhilesh Barve. 2015. Analysis of Critical Success Factors of Humanitarian Supply Chain: An Application of Interpretive Structural Modeling. International Journal of Disaster Risk Reduction 12: 213–25. [Google Scholar] [CrossRef]
- Zakariya, Yusuf F. 2022. Improving Students’ Mathematics Self-Efficacy: A Systematic Review of Intervention Studies. Frontiers in Psychology 13: 986622. [Google Scholar] [CrossRef]
- Zhang, Heyang, Xiaopeng Wu, and Minyue Ju. 2024. Developing a Cognitive Model of Solid Geometry Based on Interpretive Structural Modeling Method. Heliyon 10: 1–12. [Google Scholar] [CrossRef]
- Zhu, Zhemin, and Hua-Hua Zhang. 2021. Application and Prospects of Cognitive Diagnostic Adaptive Testing. China Examinations 1: 41–46. [Google Scholar] [CrossRef]
Compulsory Courses | Selective Compulsory Courses |
---|---|
|
|
Code | Cognitive Attributes | Content Description |
---|---|---|
A1 | The concept of function | Understand the concept of functions; Ability to find the defined and valued domains of functions; Ability to discern the same function |
A2 | Representation of functions | Able to choose the appropriate method (image method, list method, analytic method) to represent functions according to different needs |
A3 | The monotonicity of functions and the applications | Understand the concept of functional monotonicity; Ability to judge and prove the monotonicity of functions; Ability to use monotonicity to find maximum and minimum values |
A4 | The parity of functions and the applications | Understand the concept of functional parity; Ability to judge and prove the parity of functions |
A5 | The application of the concept and properties of functions | Through concrete examples, students will learn about simple piecewise functions, and be able to solve problems by simply applying the concepts and properties of functions |
A1 | A2 | A3 | A4 | A5 | |
---|---|---|---|---|---|
A1 | 0 | 1 | 0 | 0 | 0 |
A2 | 0 | 0 | 1 | 1 | 0 |
A3 | 0 | 0 | 0 | 0 | 1 |
A4 | 0 | 0 | 0 | 0 | 1 |
A5 | 0 | 0 | 0 | 0 | 0 |
A1 | A2 | A3 | A4 | A5 | |
---|---|---|---|---|---|
Item 1 | 1 | 0 | 0 | 0 | 0 |
Item 2 | 1 | 0 | 0 | 0 | 0 |
Item 3 | 1 | 0 | 0 | 0 | 0 |
Item 4 | 1 | 1 | 0 | 0 | 0 |
Item 5 | 1 | 1 | 0 | 0 | 0 |
Item 6 | 1 | 1 | 0 | 0 | 0 |
Item 7 | 1 | 1 | 1 | 0 | 0 |
Item 8 | 1 | 1 | 1 | 0 | 0 |
Item 9 | 1 | 1 | 1 | 0 | 0 |
Item 10 | 1 | 1 | 0 | 1 | 0 |
Item 11 | 1 | 1 | 0 | 1 | 0 |
Item 12 | 1 | 1 | 0 | 1 | 0 |
Item 13 | 1 | 1 | 1 | 1 | 1 |
Item 14 | 1 | 1 | 1 | 1 | 1 |
Item 15 | 1 | 1 | 1 | 1 | 1 |
Item 16 | 1 | 1 | 1 | 1 | 0 |
Item 17 | 1 | 1 | 1 | 1 | 0 |
Item 18 | 1 | 1 | 1 | 1 | 0 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhang, Y.; Zhang, L.; Zhang, H.; Wu, X. Development of a Computerized Adaptive Assessment and Learning System for Mathematical Ability Based on Cognitive Diagnosis. J. Intell. 2025, 13, 114. https://doi.org/10.3390/jintelligence13090114
Zhang Y, Zhang L, Zhang H, Wu X. Development of a Computerized Adaptive Assessment and Learning System for Mathematical Ability Based on Cognitive Diagnosis. Journal of Intelligence. 2025; 13(9):114. https://doi.org/10.3390/jintelligence13090114
Chicago/Turabian StyleZhang, Yi, Liping Zhang, Heyang Zhang, and Xiaopeng Wu. 2025. "Development of a Computerized Adaptive Assessment and Learning System for Mathematical Ability Based on Cognitive Diagnosis" Journal of Intelligence 13, no. 9: 114. https://doi.org/10.3390/jintelligence13090114
APA StyleZhang, Y., Zhang, L., Zhang, H., & Wu, X. (2025). Development of a Computerized Adaptive Assessment and Learning System for Mathematical Ability Based on Cognitive Diagnosis. Journal of Intelligence, 13(9), 114. https://doi.org/10.3390/jintelligence13090114