Abstract
Early-warning systems aim to identify students at risk of failing or withdrawing in time to act. On two modules of the Open University Learning Analytics Dataset, we evaluated a hybrid model in which a Transformer encoder summarises ten weeks of virtual learning environment activity and XGBoost combines this representation with demographic and early-assessment features. Preprocessing, encoder training and tuning were nested inside student-level cross-validation with equal tuning budgets, and results are reported primarily for the 12,436 registrations not withdrawn by day 70. On this cohort, the hybrid reached an area under the ROC curve (AUC) of 0.825. It outperformed static-only and representation-only models but not gradient boosting on static features with aggregated activity (AUC 0.822) or raw weekly counts (0.832); neither difference was statistically significant. With the representation fixed, a boosted head outperformed logistic and neural heads only when static features were included. Including students who had already withdrawn raised the AUC of every learned configuration by 0.062–0.087, which is more than the differences between competitive models. Predictions were well calibrated, early-assessment variables had the largest individual attributions, and error rates differed across socio-economic groups. The main contribution is a reproducible, leakage-controlled protocol for measuring what a sequence encoder adds.