Next Article in Journal
How Much g Is in the Distractor? Re-Thinking Item-Analysis of Multiple-Choice Items
Previous Article in Journal
How Approaches to Animal Swarm Intelligence Can Improve the Study of Collective Intelligence in Human Teams
Previous Article in Special Issue
Ergodic Subspace Analysis

Tracking with (Un)Certainty

Department of Psychological Methods, University of Amsterdam, 1018 WS Amsterdam, The Netherlands
Oefenweb, 1011 VL Amsterdam, The Netherlands
Information and Computing Sciences, Utrecht University, 3584 CC Utrecht, The Netherlands
ACTNext, Iowa City, IA 52243, USA
Author to whom correspondence should be addressed.
Received: 29 August 2019 / Revised: 14 February 2020 / Accepted: 19 February 2020 / Published: 3 March 2020
(This article belongs to the Special Issue New Methods and Assessment Approaches in Intelligence Research)
One of the highest ambitions in educational technology is the move towards personalized learning. To this end, computerized adaptive learning (CAL) systems are developed. A popular method to track the development of student ability and item difficulty, in CAL systems, is the Elo Rating System (ERS). The ERS allows for dynamic model parameters by updating key parameters after every response. However, drawbacks of the ERS are that it does not provide standard errors and that it results in rating variance inflation. We identify three statistical issues responsible for both of these drawbacks. To solve these issues we introduce a new tracking system based on urns, where every person and item is represented by an urn filled with a combination of green and red marbles. Urns are updated, by an exchange of marbles after each response, such that the proportions of green marbles represent estimates of person ability or item difficulty. A main advantage of this approach is that the standard errors are known, hence the method allows for statistical inference, such as testing for learning effects. We highlight features of the Urnings algorithm and compare it to the popular ERS in a simulation study and in an empirical data example from a large-scale CAL application. View Full-Text
Keywords: computerized adaptive learning systems; student modelling; tracking; statistical inferences computerized adaptive learning systems; student modelling; tracking; statistical inferences
Show Figures

Figure 1

MDPI and ACS Style

Hofman, A.D.; Brinkhuis, M.J.S.; Bolsinova, M.; Klaiber, J.; Maris, G.; van der Maas, H.L.J. Tracking with (Un)Certainty. J. Intell. 2020, 8, 10.

AMA Style

Hofman AD, Brinkhuis MJS, Bolsinova M, Klaiber J, Maris G, van der Maas HLJ. Tracking with (Un)Certainty. Journal of Intelligence. 2020; 8(1):10.

Chicago/Turabian Style

Hofman, Abe D., Matthieu J. S. Brinkhuis, Maria Bolsinova, Jonathan Klaiber, Gunter Maris, and Han L. J. van der Maas. 2020. "Tracking with (Un)Certainty" Journal of Intelligence 8, no. 1: 10.

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

Back to TopTop