Feature Selection Using Nearest Neighbor Gaussian Processes

Posch, Konstantin; Arbeiter, Maximilian; Truden, Christian; Pleschberger, Martin; Pilz, Jürgen

doi:10.3390/math14030476

This is an early access version, the complete PDF, HTML, and XML versions will be available soon.

Open AccessArticle

Feature Selection Using Nearest Neighbor Gaussian Processes

by

Konstantin Posch

¹

,

Maximilian Arbeiter

^1,2

,

Christian Truden

^3,*

,

Martin Pleschberger

⁴

and

Jürgen Pilz

^1,*

¹

Department of Statistics, University of Klagenfurt, 9020 Klagenfurt, Austria

²

Infineon Technologies Austria AG, 9500 Villach, Austria

³

Department of Economics, Analytics and Operations Research, University of Klagenfurt, 9020 Klagenfurt, Austria

⁴

KAI Kompetenzzentrum Automobil- und Industrieelektronik GmbH, 9524 Villach, Austria

^*

Authors to whom correspondence should be addressed.

Mathematics 2026, 14(3), 476; https://doi.org/10.3390/math14030476

Submission received: 23 November 2025 / Revised: 26 January 2026 / Accepted: 28 January 2026 / Published: 29 January 2026

(This article belongs to the Special Issue Evolutionary Computation for Feature Selection and Dimensionality Reduction)

Download Versions Notes

Abstract

We introduce a novel Bayesian approach for feature (variable) selection using Gaussian process regression, which is crucial for enhancing interpretability and model regularization. Our method employs nearest neighbor Gaussian processes as scalable approximations to classical Gaussian processes. Feature selection is performed by conditioning the process mean and covariance function on a random set representing the indices of relevant variables. A priori beliefs regarding this set control the feature selection, while reference priors are assigned to the remaining model parameters, ensuring numerical robustness in the process covariance matrix. For model inference, we propose a Metropolis-within-Gibbs algorithm. The effectiveness of the proposed feature selection approach is demonstrated through evaluation on simulated data, a computer experiment approximation, and two real-world data sets.

Keywords: feature selection; dimensional reduction; hierarchical Bayes; nearest neighbor Gaussian process; model uncertainty; Metropolis–Hastings algorithm; variable selection

Share and Cite

MDPI and ACS Style

Posch, K.; Arbeiter, M.; Truden, C.; Pleschberger, M.; Pilz, J. Feature Selection Using Nearest Neighbor Gaussian Processes. Mathematics 2026, 14, 476. https://doi.org/10.3390/math14030476

AMA Style

Posch K, Arbeiter M, Truden C, Pleschberger M, Pilz J. Feature Selection Using Nearest Neighbor Gaussian Processes. Mathematics. 2026; 14(3):476. https://doi.org/10.3390/math14030476

Chicago/Turabian Style

Posch, Konstantin, Maximilian Arbeiter, Christian Truden, Martin Pleschberger, and Jürgen Pilz. 2026. "Feature Selection Using Nearest Neighbor Gaussian Processes" Mathematics 14, no. 3: 476. https://doi.org/10.3390/math14030476

APA Style

Posch, K., Arbeiter, M., Truden, C., Pleschberger, M., & Pilz, J. (2026). Feature Selection Using Nearest Neighbor Gaussian Processes. Mathematics, 14(3), 476. https://doi.org/10.3390/math14030476

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Article metric data becomes available approximately 24 hours after publication online.

Article Menu

Feature Selection Using Nearest Neighbor Gaussian Processes

Abstract

Share and Cite

Article Metrics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI