Bayesian Optimization for Categorical and Mixed Variables Using a Multinomial Logit Surrogate

Bayesian optimization (BO) is a widely used framework for optimizing expensive black-box functions. Most BO methods rely on Gaussian process (GP) surrogates, which perform well in continuous domains but encounter difficulties when decision variables include categorical or mixed discrete–continuous components. In particular, GP-based approaches typically require ad hoc numerical encodings of categorical variables that may fail to capture the structure of discrete decision spaces. In this work, we propose MNL-BO (Multinomial Logit Bayesian Optimization), a preference-based Bayesian optimization framework that replaces the GP surrogate with a multinomial logit (MNL) model trained from pairwise preference comparisons. The resulting surrogate provides a natural and interpretable representation of categorical alternatives while allowing continuous, discrete, and categorical variables to be handled within a unified optimization framework. The predictive utility estimates and uncertainty indicators generated by the MNL model are employed to formulate acquisition functions that reconcile exploration with exploitation. The proposed methodology is evaluated on three progressively complex optimization challenges: a purely categorical benchmark, a combinatorial Traveling Salesman problem, and a constrained mixed-variable engineering design problem concerning material selection in pressure vessel optimization. Multi-run tests provide consistent advantages over random search and exhibit stable convergence behavior across diverse random initializations. In addition to heuristic baselines such as local search and classical metaheuristics, we also compare against tree-based Bayesian optimization baselines inspired by the Sequential Model-based Algorithm Configuration (SMAC) framework. The results indicate that the proposed MNL-BO method achieves competitive performance under comparable evaluation budgets while providing an interpretable probabilistic surrogate for categorical decision spaces. These findings suggest that preference-based surrogate modeling provides a practical and flexible alternative for Bayesian optimization in categorical and mixed-variable optimization problems.