Next Article in Journal
Robust H Finite-Time Control for Discrete Markovian Jump Systems with Disturbances of Probabilistic Distributions
Next Article in Special Issue
Distributed Consensus for Metamorphic Systems Using a GossipAlgorithm for CAT(0) Metric Spaces
Previous Article in Journal
Information Decomposition in Bivariate Systems: Theory and Application to Cardiorespiratory Dynamics
Previous Article in Special Issue
J.J. Thomson and Duhem’s Lagrangian Approaches to Thermodynamics
Open AccessArticle

Black-Box Optimization Using Geodesics in Statistical Manifolds

Laboratoire de Recherche en Informatique, Université Paris-Sud, 91400 Orsay, France
This paper is an extended version of our paper published in 34th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, 21–26 September 2014, Château Clos Lucé, Parc Leonardo Da Vinci, Amboise, France.
Entropy 2015, 17(1), 304-345;
Received: 8 October 2014 / Accepted: 7 January 2015 / Published: 13 January 2015
(This article belongs to the Special Issue Information, Entropy and Their Geometric Structures)
Information geometric optimization (IGO) is a general framework for stochastic optimization problems aiming at limiting the influence of arbitrary parametrization choices: the initial problem is transformed into the optimization of a smooth function on a Riemannian manifold, defining a parametrization-invariant first order differential equation and, thus, yielding an approximately parametrization-invariant algorithm (up to second order in the step size). We define the geodesic IGO update, a fully parametrization-invariant algorithm using the Riemannian structure, and we compute it for the manifold of Gaussians, thanks to Noether’s theorem. However, in similar algorithms, such as CMA-ES (Covariance Matrix Adaptation - Evolution Strategy) and xNES (exponential Natural Evolution Strategy), the time steps for the mean and the covariance are decoupled. We suggest two ways of doing so: twisted geodesic IGO (GIGO) and blockwise GIGO. Finally, we show that while the xNES algorithm is not GIGO, it is an instance of blockwise GIGO applied to the mean and covariance matrix separately. Therefore, xNES has an almost parametrization-invariant description. View Full-Text
Keywords: black-box; optimization; geodesics; Gaussian; information geometry; naturalgradient; Noether; learning rate; IGO; xNES black-box; optimization; geodesics; Gaussian; information geometry; naturalgradient; Noether; learning rate; IGO; xNES
MDPI and ACS Style

Bensadon, J. Black-Box Optimization Using Geodesics in Statistical Manifolds. Entropy 2015, 17, 304-345.

Show more citation formats Show less citations formats

Article Access Map by Country/Region

Only visits after 24 November 2015 are recorded.
Back to TopTop