1081 articles – 5277 Notices  [english version]
HAL : in2p3-00726760, version 1

Fiche concise  Récupérer au format
An apple-to-apple comparison of Learning-to-rank algorithms in terms of Normalized Discounted Cumulative Gain
Busa-Fekete R., Szarvas G., Élteto T., Kégl B.
in ECAI 2012 - 20th European Conference on Artificial Intelligence - 20th European Conference on Artificial Intelligence (ECAI 2012) : Preference Learning: Problems and Applications in AI Workshop, Montpellier : France (2012) - http://hal.in2p3.fr/in2p3-00726760
Informatique/Algorithme et structure de données
An apple-to-apple comparison of Learning-to-rank algorithms in terms of Normalized Discounted Cumulative Gain
R. Busa-Fekete, G. Szarvas, T. Élteto, B. Kégl ()1, 2, 3
1 :  LAL - Laboratoire de l'Accélérateur Linéaire
http://www.lal.in2p3.fr/
CNRS : UMR8607 – IN2P3 – Université Paris XI - Paris Sud
Centre Scientifique d'Orsay B.P. 34 91898 ORSAY Cedex
France
2 :  LRI - Laboratoire de Recherche en Informatique
http://www.lri.fr/
CNRS : UMR8623 – Université Paris Sud
LRI - Bâtiments 650-660 Université Paris-Sud 91405 Orsay Cedex
France
3 :  INRIA Saclay - Ile de France - TAO
http://tao.lri.fr/tiki-index.php
INRIA – CNRS : UMR8623 – Université Paris XI - Paris Sud
DIGITEO Bat. Claude Shannon - Université de Paris-Sud, Bâtiment 660, 91190 Gif-sur-Yvette
France
Appstat
The Normalized Discounted Cumulative Gain (NDCG) is a widely used evaluation metric for learning-to-rank (LTR) systems. NDCG is designed for ranking tasks with more than one relevance levels. There are many freely available, open source tools for computing the NDCG score for a ranked result list. Even though the definition of NDCG is unambiguous, the various tools can produce different scores for ranked lists with certain properties, deteriorating the empirical tests in many published papers and thereby making the comparison of empirical results published in different studies difficult to compare. In this study, first, we identify the major differences between the various publicly available NDCG evaluation tools. Second, based on a set of comparative experiments using a common benchmark dataset in LTR research and 6 different LTR algorithms, we demonstrate how these differences affect the overall performance of different algorithms and the final scores that are used to compare different systems.

Communications avec actes
2012
28/08/2012
ECAI 2012 - 20th European Conference on Artificial Intelligence
internationale
242
De Raedt, L., Bessiere, C., Dubois, D., Doherty, P., Frasconi, P., Heintz, F., Lucas, P.
Ios Press

20th European Conference on Artificial Intelligence (ECAI 2012) : Preference Learning: Problems and Applications in AI Workshop
Montpellier
France
27/08/2012
31/08/2012

LAL 12-311
Liste des fichiers attachés à ce document : 
PDF
07-busa-fekete.pdf(162.3 KB)