| Domaine : |
 |
Informatique/Apprentissage
|
|
| Titre : |
|
Algorithms for Hyper-Parameter Optimization |
|
| Auteur(s) : |
|
James Bergstra1, R. Bardenet2, 3, Yoshua Bengio4, Balázs Kégl ( ,  )2, 3, 5 |
|
| Projet(s) / laboratoire(s) : |
|
|
|
| Résumé : |
|
Several recent advances to the state of the art in image classification benchmarks have come from better configurations of existing techniques rather than novel ap- proaches to feature learning. Traditionally, hyper-parameter optimization has been the job of humans because they can be very efficient in regimes where only a few trials are possible. Presently, computer clusters and GPU processors make it pos- sible to run more trials and we show that algorithmic approaches can find better results. We present hyper-parameter optimization results on tasks of training neu- ral networks and deep belief networks (DBNs). We optimize hyper-parameters using random search and two new greedy sequential methods based on the ex- pected improvement criterion. Random search has been shown to be sufficiently efficient for learning neural networks for several datasets, but we show it is unreli- able for training DBNs. The sequential algorithms are applied to the most difficult DBN learning problems from [1] and find significantly better results than the best previously reported. This work contributes novel techniques for making response surface models P(y|x) in which many elements of hyper-parameter assignment (x) are known to be irrelevant given particular values of other elements. |
|
| Langue du document : |
|
Anglais |
|
|
| Type de publication : |
|
Communications avec actes |
|
| Date de publication : |
|
20/11/2011 |
|
| Audience : |
|
internationale |
|
| Titre conférence : |
|
25th Annual Conference on Neural Information Processing Systems (NIPS 2011) |
|
| Ville : |
|
Granada |
|
| Pays : |
|
Espagne |
|
| Date conférence : |
|
12/12/2011 |
|
| Date conférence (fin) : |
|
15/12/2011 |
|
|
| Référence interne : |
|
LAL 11-308 |
|
| Projet ANR : |
|
| Référence du projet |
ANR-2010-COSI-002 SIMINOLE |
|
|
BibTeX,EndNote,...
