Next Article in Journal
Robust H Finite-Time Control for Discrete Markovian Jump Systems with Disturbances of Probabilistic Distributions
Next Article in Special Issue
Distributed Consensus for Metamorphic Systems Using a GossipAlgorithm for CAT(0) Metric Spaces
Previous Article in Journal
Information Decomposition in Bivariate Systems: Theory and Application to Cardiorespiratory Dynamics
Previous Article in Special Issue
J.J. Thomson and Duhem’s Lagrangian Approaches to Thermodynamics
Article Menu

Export Article

Open AccessArticle
Entropy 2015, 17(1), 304-345; doi:10.3390/e17010304

Black-Box Optimization Using Geodesics in Statistical Manifolds

Laboratoire de Recherche en Informatique, Université Paris-Sud, 91400 Orsay, France
This paper is an extended version of our paper published in 34th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, 21–26 September 2014, Château Clos Lucé, Parc Leonardo Da Vinci, Amboise, France.
Received: 8 October 2014 / Accepted: 7 January 2015 / Published: 13 January 2015
(This article belongs to the Special Issue Information, Entropy and Their Geometric Structures)
View Full-Text   |   Download PDF [544 KB, uploaded 24 February 2015]   |  

Abstract

Information geometric optimization (IGO) is a general framework for stochastic optimization problems aiming at limiting the influence of arbitrary parametrization choices: the initial problem is transformed into the optimization of a smooth function on a Riemannian manifold, defining a parametrization-invariant first order differential equation and, thus, yielding an approximately parametrization-invariant algorithm (up to second order in the step size). We define the geodesic IGO update, a fully parametrization-invariant algorithm using the Riemannian structure, and we compute it for the manifold of Gaussians, thanks to Noether’s theorem. However, in similar algorithms, such as CMA-ES (Covariance Matrix Adaptation - Evolution Strategy) and xNES (exponential Natural Evolution Strategy), the time steps for the mean and the covariance are decoupled. We suggest two ways of doing so: twisted geodesic IGO (GIGO) and blockwise GIGO. Finally, we show that while the xNES algorithm is not GIGO, it is an instance of blockwise GIGO applied to the mean and covariance matrix separately. Therefore, xNES has an almost parametrization-invariant description. View Full-Text
Keywords: black-box; optimization; geodesics; Gaussian; information geometry; naturalgradient; Noether; learning rate; IGO; xNES black-box; optimization; geodesics; Gaussian; information geometry; naturalgradient; Noether; learning rate; IGO; xNES
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. (CC BY 4.0).

Scifeed alert for new publications

Never miss any articles matching your research from any publisher
  • Get alerts for new papers matching your research
  • Find out the new papers from selected authors
  • Updated daily for 49'000+ journals and 6000+ publishers
  • Define your Scifeed now

SciFeed Share & Cite This Article

MDPI and ACS Style

Bensadon, J. Black-Box Optimization Using Geodesics in Statistical Manifolds. Entropy 2015, 17, 304-345.

Show more citation formats Show less citations formats

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top