Skip to Main content Skip to Navigation
New interface
Conference papers

Introduction to multivariate discrimination

Balázs Kégl 1, 2 
1 TAO - Machine Learning and Optimisation
LRI - Laboratoire de Recherche en Informatique, UP11 - Université Paris-Sud - Paris 11, Inria Saclay - Ile de France, CNRS - Centre National de la Recherche Scientifique : UMR8623
2 Appstat
LAL - Laboratoire de l'Accélérateur Linéaire, LRI - Laboratoire de Recherche en Informatique
Abstract : Multivariate discrimination or classification is one of the best-studied problem in machine learning, with a plethora of well-tested and well-performing algorithms. There are also several good general textbooks [1-9] on the subject written to an average engineering, computer science, or statistics graduate student; most of them are also accessible for an average physics student with some background on computer science and statistics. Hence, instead of writing a generic introduction, we concentrate here on relating the subject to a practitioner experimental physicist. After a short introduction on the basic setup (Section 1) we delve into the practical issues of complexity regularization, model selection, and hyperparameter optimization (Section 2), since it is this step that makes high-complexity non-parametric fitting so different from low-dimensional parametric fitting. To emphasize that this issue is not restricted to classification, we illustrate the concept on a low-dimensional but non-parametric regression example (Section 2.1). Section 3 describes the common algorithmic-statistical formal framework that unifies the main families of multivariate classification algorithms. We explain here the large-margin principle that partly explains why these algorithms work. Section 4 is devoted to the description of the three main (families of) classification algorithms, neural networks, the support vector machine, and AdaBoost. We do not go into the algorithmic details; the goal is to give an overview on the form of the functions these methods learn and on the objective functions they optimize. Besides their technical description, we also make an attempt to put these algorithm into a socio-historical context. We then briefly describe some rather heterogeneous applications to illustrate the pattern recognition pipeline and to show how widespread the use of these methods is (Section 5). We conclude the chapter with three essentially open research problems that are either relevant to or even motivated by certain unorthodox applications of multivariate discrimination in experimental physics.
Complete list of metadata

Cited literature [54 references]  Display  Hide  Download
Contributor : Sabine Starita Connect in order to contact the contributor
Submitted on : Thursday, July 18, 2013 - 3:47:02 PM
Last modification on : Tuesday, October 25, 2022 - 4:18:29 PM
Long-term archiving on: : Monday, October 21, 2013 - 9:46:46 AM


Publisher files allowed on an open archive



Balázs Kégl. Introduction to multivariate discrimination. IN2P3 School of Statistics (SOS2012), May 2012, Autrans, France. pp.022001, ⟨10.1051/epjconf/20135502001⟩. ⟨in2p3-00846125⟩



Record views


Files downloads