Collaborative hyperparameter tuning

R. Bardenet 1, 2 M. Brendel 1 B. Kégl 3, 4 M. Sebag 5
3 TAO - Machine Learning and Optimisation
LRI - Laboratoire de Recherche en Informatique, UP11 - Université Paris-Sud - Paris 11, Inria Saclay - Ile de France, CNRS - Centre National de la Recherche Scientifique : UMR8623
4 Appstat
LAL - Laboratoire de l'Accélérateur Linéaire, LRI - Laboratoire de Recherche en Informatique
Abstract : Hyperparameter learning has traditionally been a manual task because of the limited number of trials. Today's computing infrastructures allow bigger evaluation budgets, thus opening the way for algorithmic approaches. Recently, surrogate-based optimization was successfully applied to hyperparameter learning for deep belief networks and to WEKA classifiers. The methods combined brute force computational power with model building about the behavior of the error function in the hyperparameter space, and they could significantly improve on manual hyperparameter tuning. What may make experienced practitioners even better at hyperparameter optimization is their ability to generalize across similar learning problems. In this paper, we propose a generic method to incorporate knowledge from previous experiments when simultaneously tuning a learning algorithm on new problems at hand. To this end, we combine surrogate-based ranking and optimization techniques for surrogate-based collaborative tuning (SCoT). We demonstrate SCoT in two experiments where it outperforms standard tuning techniques and single-problem surrogate-based optimization.
Type de document :
Communication dans un congrès
Sanjoy Dasgupta and David McAllester. 30th International Conference on Machine Learning (ICML 2013), Jun 2013, Atlanta, United States. Acm Press, 28, pp.199-207, 2013
Liste complète des métadonnées

http://hal.in2p3.fr/in2p3-00907381
Contributeur : Sabine Starita <>
Soumis le : jeudi 21 novembre 2013 - 11:04:12
Dernière modification le : jeudi 9 février 2017 - 15:53:23
Document(s) archivé(s) le : samedi 22 février 2014 - 04:33:20

Fichier

bardenet13.pdf
Fichiers éditeurs autorisés sur une archive ouverte

Identifiants

  • HAL Id : in2p3-00907381, version 1

Collections

Citation

R. Bardenet, M. Brendel, B. Kégl, M. Sebag. Collaborative hyperparameter tuning. Sanjoy Dasgupta and David McAllester. 30th International Conference on Machine Learning (ICML 2013), Jun 2013, Atlanta, United States. Acm Press, 28, pp.199-207, 2013. <in2p3-00907381>

Partager

Métriques

Consultations de
la notice

560

Téléchargements du document

476