EDA with Mixtures of Probability Distributions

Estimation of distribution algorithms (EDAs) are optimization methods that search for explicit probabilistic models which are used to sample promising candidate solutions. The optimization process consists of a sequence of incremental updates to an initial probabilistic model, which is then used to sample candidate solutions for the problem to be solved. EDAs use an explicit probability distribution encoded by a Bayesian network, a multivariate normal distribution, or another model class. Although some EDAs optimize the structure of the probabilistic graph model, the distribution type at each node is typically fixed; for continuous variables, the nodes generally encode normal distributions. The current paper proposes M-EDA (mixture-based estimation of distribution algorithm)—an EDA variant based on genetic algorithms which aims to identify an optimal type of probabilistic model, encoding mixtures of probability distributions and their corresponding parameters. M-EDA optimizes such mixtures in order to fit complex landscapes. These mixtures are encoded in the genetic algorithm (GA) through heterogeneous variable-length chromosomes. M-EDA was tested on several numerical optimization problems used widely in the literature on genetic algorithms and reached near-optimal solutions. It also demonstrated multimodal optimization capabilities. Finally, M-EDA was also tested on the instance selection (IS) problem, obtaining a substantial reduction in the number of selected instances, and outperforming most of the competing techniques—in accuracy on balanced datasets, and in instance-reduction rate on imbalanced or high-dimensional ones.