Surrogating the surrogate: accelerating Gaussian-process-based global optimization with a mixture cross-entropy algorithm

R. Bardenet 1, 2 Balázs Kégl 1, 2, 3
3 TAO - Machine Learning and Optimisation
CNRS - Centre National de la Recherche Scientifique : UMR8623, Inria Saclay - Ile de France, UP11 - Université Paris-Sud - Paris 11, LRI - Laboratoire de Recherche en Informatique
Abstract : In global optimization, when the evaluation of the target function is costly, the usual strategy is to learn a surrogate model for the target function and replace the initial optimization by the optimization of the model. Gaussian processes have been widely used since they provide an elegant way to model the fitness and to deal with the exploration-exploitation trade-off in a principled way. Several empirical criteria have been proposed to drive the model optimization, among which is the well-known Expected Improvement criterion. The major computational bottleneck of these algorithms is the exhaustive grid search used to optimize the highly multi modal merit function. In this paper, we propose a competitive ''adaptive grid'' approach, based on a properly derived Cross-Entropy optimization algorithm with mixture proposals. Experiments suggest that 1) we outperform the classical single-Gaussian cross-entropy method when the fitness function is highly multi modal, and 2) we improve on standard exhaustive search in GP-based surrogate optimization.
Complete list of metadatas

Cited literature [13 references]  Display  Hide  Download

http://hal.in2p3.fr/in2p3-00580438
Contributor : Sabine Starita <>
Submitted on : Monday, March 28, 2011 - 2:07:50 PM
Last modification on : Thursday, April 5, 2018 - 12:30:12 PM
Long-term archiving on : Wednesday, June 29, 2011 - 2:40:08 AM

File

Bardenet.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : in2p3-00580438, version 1

Collections

Citation

R. Bardenet, Balázs Kégl. Surrogating the surrogate: accelerating Gaussian-process-based global optimization with a mixture cross-entropy algorithm. 27th International Conference on Machine Learning (ICML 2010), Jun 2010, Haifa, Israel. pp.55-62. ⟨in2p3-00580438⟩

Share

Metrics

Record views

888

Files downloads

814