Evaluating Inferential Statistics Filtering in High-Dimensional Item Feature Spaces for Predicting IRT Parameters

Predicting parameter estimates under item response theory (IRT) from expert-coded item features offers a scalable alternative to resource-intensive field testing. This study evaluates whether inferential feature selection can improve predictive accuracy for item difficulty and item discrimination using five filter methods: the Analysis of Variance (ANOVA) F-test, Kendall’s Tau, the Kolmogorov–Smirnov test, the Anderson–Darling test, and the Energy Distance test. Models were trained using K-Nearest Neighbors (KNN) and Support Vector Regression (SVR) under random split and fixed-form cold-start partitioning strategies. Results show that the distributional properties of item features, rather than train–test splitting alone, drive predictive gains: distribution-based filter approaches, particularly the Kolmogorov–Smirnov test, consistently outperformed mean-based approaches by better capturing the full probability structure of the feature-parameter relationship. KNN benefited substantially from feature selection given its reliance on Euclidean distance, while SVR showed smaller gains due to its inherent regularization. Item discrimination generalized well to previously unseen test forms that share no calibration data with the training set, whereas item difficulty prediction was considerably more sensitive to distributional shifts when predicting entirely new, operationally administered forms. The main finding is that the distributional properties of item features are more important than the quantity of features for obtaining robust IRT parameter predictions.