# How Should I Teach from This Month Onward? A State-Space Model That Helps Drive Whole Classes to Achieve End-of-Year National Standardized Test Learning Targets

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

^{2}adjusted is 0.637. However, all students attempted to solve a similar set of items. One of the challenges is that teachers prefer to have the flexibility to adapt the exercises to their own experience. The authors did not consider the situation where teachers select or design their own set of exercises. Ref. [7] also compared, month by month, their linear predictors without a pretest with baselines. They found an increasing performance as they gathered process information of more months.

## 3. Theoretical Framework

**Research Question 1:**To what extent can teacher-designed questions (low- or no-stake quizzes) help predict students’ long-term learning as measured by end-of-year standardized state tests?

**Research Question 2:**To what extent can a hidden Markovian state of the student’s accumulated knowledge up to the current month be estimated so that, along with the probable deliberate practice on the next months, it is sufficient to predict with good accuracy the end-of-year standardized state tests?

**Research Question 3:**To what extent can a state-space model help teachers to visualize the trade-off between asking students to perform exercises more carefully or perform more exercises, and thus help drive whole classes to achieve long-term learning targets?

## 4. Materials and Methods

- Numbers and operations;
- Patterns and algebra;
- Geometry;
- Measurement;
- Data and probabilities.

#### 4.1. The Sequence of Linear Regression Models

- ${p}_{i}$: pretest of the student i (${p}_{i}\in \mathbb{R}$, normalized).
- ${\Theta}_{j}^{\left(1\right)}$: class j average for “My behavior is a problem for the math teacher”.
- ${\Theta}_{i}^{\left(2\right)}$: student i response to “Math is easy for me”.
- ${a}_{i,j,k,t}$: percentage of exercises solved on the first attempt for a student i, on the k strand, at month t (normalized for each strand month).
- ${d}_{i,j,k,t}$: the difference between exercises solved on the first attempt and those that took more than one for a student i, on the k strand, at month t (normalized for each strand month).

#### 4.2. The State-Space Model

- $x\left(t\right)$ is the hidden state that represents the long-term knowledge of a student.
- $y\left(t\right)$ are the observed measurement as a grade or score.
- $u\left(t\right)$ has historical variables at $t=0$ and dynamic ones for $t>0$.

## 5. Results

#### 5.1. Prediction Errors of Both Models

#### 5.2. Optimal Values of the Parameters

#### 5.3. Contribution of the Different Variables to the Prediction

#### 5.4. The Optimal Control Problem: Ask Students to Do Exercises More Carefully or Do More Exercises

## 6. Discussion

## 7. Conclusions

#### 7.1. Theoretical Implications

#### 7.2. Managerial Implications

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Abbreviations

GPA | Grade Point Average |

LRMOFR | Linear Regression for the Month with an Optimal Forgetting Rate |

OLS | Ordinary Least Squares |

RMSE | Root Mean Squared Error |

SD | Standard Deviation |

SIMCE | Sistema de Medición de la Calidad de la Educación (Mesurement System of the Quality of Education) |

SS | State Space |

## References

- Soderstrom, N.; Bjork, R. Learning Versus Performance: An Integrative Review. Perspect. Psychol. Sci.
**2015**, 10, 176–199. [Google Scholar] [CrossRef] [PubMed] - Bjork, R.A.; Bjork, E.L. Desirable difficulties in theory and practice. J. Appl. Res. Mem. Cogn.
**2020**, 9, 475–479. [Google Scholar] [CrossRef] - Bjork, E.; Bjork, R. Making Things Hard on Yourself, But in a Good Way: Creating Desirable Difficulties to Enhance Learning. In FABBS Foundation, Psychology and the Real World: Essays Illustrating Fundamental Contributions to Society; Worth Publishers: London, UK, 2016; pp. 56–64. [Google Scholar]
- Gervet, T.; Koedinger, K.; Schneider, J.; Mitchell, T. When is Deep Learning the Best Approach to Knowledge Tracing? J. Educ. Data Min.
**2020**, 12, 31–54. [Google Scholar] [CrossRef] - Zytek, A.; Arnaldo, I.; Liu, D.; Berti-Equille, K.; Veeramachaneni, K. The Need for Interpretable Features: Motivation and Taxonomy. arXiv
**2022**, arXiv:2202.11748. [Google Scholar] [CrossRef] - Bagaloplan, A.; Zhang, H.; Hamidieh, K.; Hartvigsen, T.; Rudzicz, F.; Ghassemi, M. The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanations. In Proceedings of the FAccT ’22: 2022 ACM Conference on Fairness, Accountability, and Transparency, Seoul, Korea, 21–24 June 2022. [Google Scholar] [CrossRef]
- Anozie, N.; Junker, B. Predicting end-of-year accountability assessment scores from monthly student records in an online tutoring system. In Proceedings of the AAAI Workshop on Educational Data Mining, Boston, MA, USA, 16–17 July 2006. [Google Scholar]
- Zheng, G.; Fancsali, S.; Ritter, S.; Berman, S. Using Instruction-Embedded Formative Assessment to Predict State Summative Test Scores and Achievement Levels in Mathematics. J. Learn. Anal.
**2019**, 6, 153–174. [Google Scholar] [CrossRef] - Namoun, A.; Alshanqiti, A. Predicting Student Performance Using Data Mining and Learning Analytics Techniques: A Systematic Literature Review. Appl. Sci.
**2021**, 11, 237. [Google Scholar] [CrossRef] - Ruipérez-Valiente, J.A.; Muñoz-Merino, P.J.; Delgado, C. Improving the prediction of learning outcomes in educational platforms including higher level interaction indicators. Expert Syst.
**2018**, 35, e12298. [Google Scholar] [CrossRef] - Heckman, J.; Zhou, J. Interactions as Investments: The Microdynamics and Measurement of Early Childhood Learning; Working paper; Center for the Economics of Human Development and Department of Economics, University of Chicago: Chicago, IL, USA, 2021. [Google Scholar]
- Lloyd, E.; Wilson, D.; Sober, E. Evolutionary Mismatch and What to Do About It: A Basic Tutorial. Evol. Appl.
**2011**, 2–4. [Google Scholar] - Geary, D. Educating the evolved mind: Conceptual foundations for an evolutionary educational psychology. In Psychological Perspectives on Contemporary Educational Issues; Carlson, J.S., Levin, J.R., Eds.; Information Age Publishing: Charlotte, NC, USA, 2007. [Google Scholar]
- Gezer, T.; Wang, C.; Polly, A.; Martind, C.; Pugaleee, D.; Lambert, R. The Relationship between Formative Assessment and Summative Assessment in Primary Grade Mathematics Classrooms. Int. Electron. J. Elem. Educ.
**2021**, 13, 673–685. [Google Scholar] [CrossRef] - Yang, C.; Luo, L.; Vadillo, M.; Yu, R.; Shanks, D. Testing (Quizzing) Boosts Classroom Learning: A Systematic and Meta-Analytic Review. Psychol. Bull.
**2020**, 147, 399–435. [Google Scholar] [CrossRef] - Araya, R.; Van der Molen, J. Causal Dependence among Contents Emerges from the Collective Online Learning of Students. In Proceedings of the International Conference on Computational Collective Intelligence, Craiova, Romania, 11–13 September 2013; Springer: Berlin, Germany, 2013; Volume 8083, pp. 641–650. [Google Scholar] [CrossRef]
- Araya, R.; Van der Molen, J. Impact of a blended ICT adoption model on Chilean vulnerable schools correlates with amount of on online practice. In Proceedings of the Workshops at the 16th International Conference on Artificial Intelligence in Education AIED 2013, Memphis, TN, USA, 9–13 July 2013. [Google Scholar]
- Araya, R.; Gormaz, R.; Bahamondez, M.; Aguirre, C.; Calfucura, P.; Jaure, P.; Laborda, C. ICT supported learning rises math achievement in low socio economic status schools. In Proceedings of the 10th European Conference on Technology Enhanced Learning, Toledo, Spain, 15–18 September 2015; Springer: Berlin, Germany, 2015. Lecture Notes in Computer Science. Volume 9307, pp. 383–388. [Google Scholar] [CrossRef]
- Araya, R. Integrating Classes from Different Schools Using Intelligent Teacher Support Systems. In Proceedings of the 1st International Conference on Intelligent Human Systems Integration (IHSI 2018), Dubai, United Arab Emirates, 7–9 January 2018; Springer: Cham, Switzerland, 2018; Volume 722, pp. 294–300. [Google Scholar] [CrossRef]
- Araya, R. Teacher Training, Mentoring or Performance Support Systems? In Proceedings of the International Conference on Applied Human Factors and Ergonomics 2018, Orlando, FL, USA, 21–25 July 2018; Springer: Cham, Switzerland, 2019; Volume 785, pp. 306–315. [Google Scholar] [CrossRef]
- Araya, R.; Diaz, K. Implementing Government Elementary Math Exercises Online: Positive Effects Found in RCT under Social Turmoil in Chile. Educ. Sci.
**2020**, 10, 244. [Google Scholar] [CrossRef] - Heckman, J.J. Skill formation and the economics of investing in disadvantaged children. Science
**2006**, 312, 1900–1902. [Google Scholar] [CrossRef] [PubMed] - Rea, D.; Burton, T. Does an Empirical Heckman Curve Exist? 2018. Available online: https://www.wgtn.ac.nz/__data/assets/pdf_file/0005/1716953/WP18-03-does-an-empirical-Heckman-curve-exist.pdf (accessed on 1 August 2022).
- Araya, R.; Gormaz, R. Revealed Preferences of Fourth Graders When Requesting Face-to-Face Help While Doing Math Exercises Online. Educ. Sci.
**2021**, 11, 429. [Google Scholar] [CrossRef] - Urrutia, F.; Araya, R. Do written responses to open-ended questions on fourth-grade online formative assessments in mathematics help predict scores on end-of-year standardized tests? Preprints, 2022; submitted. [Google Scholar]
- Araya, R.; Arias Ortiz, E.; Bottan, N.; Cristia, J. Does Gamification in Education Work? Experimental Evidence from Chile. In IDB Working Paper Series Nº IDB-WP-982 Inter-American Development Bank; IDB: Washington, DC, USA, 2019. [Google Scholar] [CrossRef]
- Van der Molen, J. Minería de Datos Educacionales: Modelos de Predicción del Desempeño Escolar en Alumnos de Enseñanza Básica; Universidad de Chile: Santiago, Chile, 2013. [Google Scholar]
- Ulloa, O. Estimación de Desempeño en Evaluación Sumativa, Con Base en Evaluaciones Formativas Usando Modelos Espacio Estado. Master’s Thesis, University of Chile, Santiago, Chile, 2021. [Google Scholar]
- Pardos, Z.; Baker, R.; San Pedro, M.; Gowda, S.; Gowda, S. Affective States and State Tests: Investigating How Affect and Engagement during the School Year Predict End-of-Year Learning Outcomes. J. Learn. Anal.
**2014**, 1, 107–128. [Google Scholar] [CrossRef] - Chen, Y.H. A revisit to the student learning problem. Optim. Control. Appl. Methods
**2016**, 12, 263–272. [Google Scholar] [CrossRef] - Lewis, D. Modeling student engagement using optimal control theory. J. Geom. Mech.
**2022**, 14, 131–150. [Google Scholar] [CrossRef] - Raggett, G.F.; Hempson, P.W.; Jukes, K.A. A Student-Related Optimal Control Problem. Bull. Inst. Math. Its Appl.
**1981**, 17, 133–136. [Google Scholar] - Teklu, S.W.; Terefe, B.B. Mathematical modeling analysis on the dynamics of university students animosity towards mathematics with optimal control theory. Sci. Rep.
**2022**, 12, 11578. [Google Scholar] [CrossRef] - Rohrer, D.; Dedrick, R.; Stershic, S. Interleaved Practice Improves Mathematics Learning. J. Educ. Psychol.
**2015**, 107, 900–908. [Google Scholar] [CrossRef] - Rohrer, D.; Dedrick, R.; Hartwig, M. The Scarcity of Interleaved Practice in Mathematics Textbooks. Educ. Psychol. Rev.
**2020**, 32, 873–883. [Google Scholar] [CrossRef] - Raviv, L.; Lupyan, G.; Green, S.C. How variability shapes learning and generalization. Trends Cogn. Sci.
**2022**, 26, 462–483. [Google Scholar] [CrossRef] [PubMed] - Geary, D.C. Principles of evolutionary educational psychology. Learn. Individ. Differ. Evol. Educ. Psychol.
**2002**, 12, 317–345. [Google Scholar] [CrossRef] - Henrich, J. The Weirdest People in the World: How the West Became Psychologically Peculiar and Particularly Prosperous; Farrar, Straus and Giroux: New York, NY, USA, 2020. [Google Scholar]

**Figure 1.**Root mean square error (RMSE) of the two models with best RMSE. The graph shows the RMSE in the test classes for each month of the school year. LRMOFR: linear regression for each month with an optimal forgetting rate model. SS: state-space model.

**Figure 6.**Pretest effect. This is the contribution on the SIMCE prediction of one standard deviation increase of the pre-test.

**Figure 7.**Effect of the class average of the degree of agreement with “My behavior is a problem for the teacher”. This is the contribution to the SIMCE prediction of an increase of one standard deviation of the student belief.

**Figure 8.**Effect of the “Math is easy for me” subjective belief. This is the contribution to the SIMCE prediction of an increase of one standard deviation of the student belief.

**Figure 9.**Effect of an accuracy one standard deviation above the mean on every month. This is the contribution to the SIMCE prediction of an increase of one standard deviation of the student accuracy.

**Figure 10.**Effect of a difference one standard deviation above the mean on every month. This is the contribution to the SIMCE prediction of the difference between number of exercises solved correctly in the first attempt and the number solved correctly on other attempts.

**Figure 11.**Iso-effect curves for the number and operations strand, in September, where the last month with student data is August.

**Figure 12.**Effect gradients and iso-effect curves for the Number and Operations strand, in September, where the last month with student data is August.

**Figure 13.**Iso-effort, effect gradients, and iso-effect curves for the number and operations strand in September, where the last month with student data is August.

**Figure 14.**Iso-effort curves and effect gradients for the data and probability strand in September, where the last month with student data is August.

Strand | Mar | Apr | May | Jun | Jul | Aug | Sep | Oct | Average |
---|---|---|---|---|---|---|---|---|---|

Numbers | 6 | 120 | 123 | 69 | 56 | 122 | 119 | 93 | 89 |

Patterns | 0 | 2 | 4 | 18 | 7 | 1 | 14 | 27 | 9 |

Geometry | 0 | 0 | 11 | 22 | 7 | 24 | 17 | 64 | 18 |

Measurement | 0 | 0 | 5 | 35 | 11 | 17 | 11 | 28 | 13 |

Data | 0 | 1 | 2 | 3 | 5 | 3 | 7 | 30 | 6 |

Total | 6 | 124 | 146 | 147 | 86 | 168 | 168 | 243 | 136 |

**Table 2.**Percentage of times in which the model had the lowest RMSE within the 24 models in the testing classes.

Model | Mar | Apr | May | Jun | Jul | Aug | Sep | Oct | Average |
---|---|---|---|---|---|---|---|---|---|

Linear regression | 20.0 | 19.0 | 4.5 | 4.0 | 1.0 | 5.5 | 28.0 | 40.5 | 15.3 |

State-Space | 19.5 | 24.0 | 61.0 | 62.5 | 69.0 | 69.0 | 35.5 | 32.0 | 46.6 |

**Table 3.**Estimated parameters for the linear regression for each month with optimal forgetting rate model obtained with 200 cross validations.

Parameter | Mar | Apr | May | Jun | Jul | Aug | Sep | Oct |
---|---|---|---|---|---|---|---|---|

$\rho $ | 0.71 (0.05) | 0.71 (0.05) | 0.71 (0.05) | 0.71 (0.05) | 0.71 (0.05) | 0.71 (0.05) | 0.71 (0.05) | 0.71 (0.05) |

${\gamma}_{\tau}$ | 33.16 (2.22) | 23.78 (2.24) | 18.77 (2.14) | 17.17 (2.16) | 14.56 (2.17) | 17.38 (1.91) | 15.87 (1.70) | 15.34 (1.59) |

${\beta}_{0}$ | 240.04 (3.45) | 240.04 (3.45) | 240.04 (3.45) | 240.04 (3.45) | 240.04 (3.45) | 240.04 (3.45) | 240.04 (3.45) | 240.04 (3.45) |

${\beta}_{1,\tau}$ | −11.80 (1.75) | −11.49 (2.04) | −11.38 (1.94) | −9.24 (1.73) | −9.26 (1.72) | −10.27 (1.76) | −8.96 (1.71) | −8.62 (1.65) |

${\beta}_{2,\tau}$ | 5.00 (1.12) | 4.83 (1.05) | 4.60 (0.95) | 3.98 (0.95) | 4.23 (0.83) | 4.01 (0.93) | 3.61 (0.95) | 3.00 (0.89) |

${\delta}_{1,\tau}$ | −1.00 (2.03) | 1.73 (3.29) | 7.93 (2.97) | 5.57 (2.99) | 6.75 (2.68) | 7.22 (3.24) | 12.21 (2.80) | 12.01 (2.49) |

${\delta}_{2,\tau}$ | −0.38 (1.73) | 3.58 (1.63) | 5.23 (1.71) | 3.98 (2.25) | 10.57 (1.85) | 7.08 (2.08) | 9.30 (1.90) | 6.70 (2.23) |

${\delta}_{3,\tau}$ | −5.69 (1.63) | −2.30 (2.12) | 0.52 (2.06) | 5.88 (2.31) | 7.18 (2.71) | 7.03 (2.69) | 6.86 (2.74) | 4.61 (2.40) |

${\delta}_{4,\tau}$ | −2.08 (1.89) | −0.66 (2.12) | 4.14 (2.24) | 8.00 (2.67) | 4.55 (2.90) | 2.61 (2.66) | 4.45 (2.35) | 3.18 (2.94) |

${\delta}_{5,\tau}$ | 1.83 (1.64) | 5.19 (1.92) | 2.28 (2.51) | 7.76 (1.48) | 13.82 (2.35) | 5.16 (2.54) | 8.07 (2.56) | 11.09 (2.30) |

${\eta}_{1,\tau}$ | 4.70 (1.69) | 14.64 (2.61) | 13.34 (2.22) | 15.76 (2.33) | 14.85 (2.07) | 9.85 (2.59) | 5.28 (2.54) | 3.03 (2.19) |

${\eta}_{2,\tau}$ | - | 1.94 (1.21) | 2.57 (0.99) | 0.71 (1.52) | −0.82 (1.87) | 1.82 (2.90) | 2.13 (2.56) | 7.64 (2.52) |

${\eta}_{3,\tau}$ | 4.10 (1.90) | 1.20 (2.74) | 1.86 (1.67) | −3.29 (3.02) | −6.90 (3.66) | −4.86 (3.17) | −3.59 (3.13) | 0.50 (3.52) |

${\eta}_{4,\tau}$ | - | 1.21 (1.58) | 2.59 (1.75) | 1.59 (2.05) | 3.49 (2.03) | 10.44 (2.89) | 5.24 (2.10) | 3.55 (2.02) |

${\eta}_{5,\tau}$ | - | −2.62 (1.83) | 1.26 (2.46) | −2.72 (1.55) | −6.25 (2.42) | −0.79 (2.08) | 1.09 (3.20) | 1.23 (2.54) |

Parameter | Mean | SD |
---|---|---|

${x}_{0}$ | 258.3901 | 3.4421 |

A | 1.0010 | 0.0006 |

K | 0.0835 | 0.0085 |

${Q}^{\frac{1}{2}}$ | 0.8982 | 0.1080 |

${R}^{\frac{1}{2}}$ | 10.4133 | 0.5388 |

${B}_{1}$ | 0.3787 | 0.2285 |

${B}_{2}$ | 0.5888 | 0.1422 |

${B}_{3}$ | 0.0770 | 0.1739 |

${B}_{4}$ | 0.1984 | 0.1855 |

${B}_{5}$ | 0.6834 | 0.1631 |

${B}_{6}$ | 1.1153 | 0.2337 |

${B}_{7}$ | 0.3170 | 0.1014 |

${B}_{8}$ | −0.0979 | 0.1365 |

${B}_{9}$ | 0.4526 | 0.1363 |

${B}_{10}$ | −0.0675 | 0.1544 |

${B}_{11}$ | 24.6766 | 2.0893 |

${B}_{12}$ | −14.0852 | 2.0651 |

${B}_{13}$ | 6.143 | 1.2942 |

Effect | LRMOFR | SS |
---|---|---|

Intercept | ${\beta}_{0}$ | $\begin{array}{c}{\scriptstyle {A}^{T-\tau -1}{(A-K)}^{\tau}\widehat{x}\left(0\right)}{\scriptstyle +{A}^{T-\tau -1}{\displaystyle \sum _{t=1}^{\tau}}{(A-K)}^{\tau -t}Ky\left(t\right)}\end{array}$ |

Pretest | ${\gamma}_{\tau}{p}_{i}$ | ${\scriptstyle {B}_{11}\left({A}^{T-\tau -1}{(A-K)}^{\tau}{u}_{11}\left(0\right)\right)}$ |

‘My behavior is a problem…’ | ${\beta}_{1,\tau}{\Theta}_{j}^{\left(1\right)}$ | ${\scriptstyle {B}_{12}\left({A}^{T-\tau -1}{(A-K)}^{\tau}{u}_{12}\left(0\right)\right)}$ |

‘Math is easy for me’ | ${\beta}_{2,\tau}{\Theta}_{i}^{\left(2\right)}$ | ${\scriptstyle {B}_{13}\left({A}^{T-\tau -1}{(A-K)}^{\tau}{u}_{12}\left(0\right)\right)}$ |

Accuracy | $\frac{{\displaystyle \sum _{t=1}^{\tau}}{\displaystyle \sum _{k=1}^{5}}{\rho}^{\tau -t}{\delta}_{k,\tau}{a}_{i,j,k,t}}{{\displaystyle \sum _{t=1}^{\tau}}{\rho}^{\tau -t}}$ | $\begin{array}{c}{\scriptstyle {\displaystyle \sum _{k=1}^{5}}{B}_{k}\left({A}^{T-\tau -1}{\displaystyle \sum _{t=1}^{\tau}}{(A-K)}^{\tau -t}{u}_{k}\left(t\right)\right.}\\ {\scriptstyle \left.+{\displaystyle \sum _{t=\tau +1}^{T-1}}{A}^{T-t-1}{\widehat{u}}_{k}(\tau ,t)\right)}\end{array}$ |

Difference | $\frac{{\displaystyle \sum _{t=1}^{\tau}}{\displaystyle \sum _{k=1}^{5}}{\rho}^{\tau -t}{\eta}_{k,\tau}{d}_{i,j,k,t}}{{\displaystyle \sum _{t=1}^{\tau}}{\rho}^{\tau -t}}$ | $\begin{array}{c}{\scriptstyle {\displaystyle \sum _{k=6}^{10}}{B}_{k}\left({A}^{T-\tau -1}{\displaystyle \sum _{t=1}^{\tau}}{(A-K)}^{\tau -t}{u}_{k}\left(t\right)\right.}\\ {\scriptstyle \left.+{\displaystyle \sum _{t=\tau +1}^{T-1}}{A}^{T-t-1}{\widehat{u}}_{k}(\tau ,t)\right)}\end{array}$ |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Ulloa, O.; Araya, R.
How Should I Teach from This Month Onward? A State-Space Model That Helps Drive Whole Classes to Achieve End-of-Year National Standardized Test Learning Targets. *Systems* **2022**, *10*, 167.
https://doi.org/10.3390/systems10050167

**AMA Style**

Ulloa O, Araya R.
How Should I Teach from This Month Onward? A State-Space Model That Helps Drive Whole Classes to Achieve End-of-Year National Standardized Test Learning Targets. *Systems*. 2022; 10(5):167.
https://doi.org/10.3390/systems10050167

**Chicago/Turabian Style**

Ulloa, Obed, and Roberto Araya.
2022. "How Should I Teach from This Month Onward? A State-Space Model That Helps Drive Whole Classes to Achieve End-of-Year National Standardized Test Learning Targets" *Systems* 10, no. 5: 167.
https://doi.org/10.3390/systems10050167