Skip to Main content Skip to Navigation
Conference papers

Introduction to multivariate discrimination

Balázs Kégl 1, 2
1 TAO - Machine Learning and Optimisation
CNRS - Centre National de la Recherche Scientifique : UMR8623, Inria Saclay - Ile de France, UP11 - Université Paris-Sud - Paris 11, LRI - Laboratoire de Recherche en Informatique
2 Appstat
LAL - Laboratoire de l'Accélérateur Linéaire, LRI - Laboratoire de Recherche en Informatique
Abstract : Multivariate discrimination or classification is one of the best-studied problem in machine learning, with a plethora of well-tested and well-performing algorithms. There are also several good general textbooks [1-9] on the subject written to an average engineering, computer science, or statistics graduate student; most of them are also accessible for an average physics student with some background on computer science and statistics. Hence, instead of writing a generic introduction, we concentrate here on relating the subject to a practitioner experimental physicist. After a short introduction on the basic setup (Section 1) we delve into the practical issues of complexity regularization, model selection, and hyperparameter optimization (Section 2), since it is this step that makes high-complexity non-parametric fitting so different from low-dimensional parametric fitting. To emphasize that this issue is not restricted to classification, we illustrate the concept on a low-dimensional but non-parametric regression example (Section 2.1). Section 3 describes the common algorithmic-statistical formal framework that unifies the main families of multivariate classification algorithms. We explain here the large-margin principle that partly explains why these algorithms work. Section 4 is devoted to the description of the three main (families of) classification algorithms, neural networks, the support vector machine, and AdaBoost. We do not go into the algorithmic details; the goal is to give an overview on the form of the functions these methods learn and on the objective functions they optimize. Besides their technical description, we also make an attempt to put these algorithm into a socio-historical context. We then briefly describe some rather heterogeneous applications to illustrate the pattern recognition pipeline and to show how widespread the use of these methods is (Section 5). We conclude the chapter with three essentially open research problems that are either relevant to or even motivated by certain unorthodox applications of multivariate discrimination in experimental physics.
Complete list of metadatas

Cited literature [54 references]  Display  Hide  Download

http://hal.in2p3.fr/in2p3-00846125
Contributor : Sabine Starita <>
Submitted on : Thursday, July 18, 2013 - 3:47:02 PM
Last modification on : Wednesday, September 16, 2020 - 5:43:39 PM
Long-term archiving on: : Monday, October 21, 2013 - 9:46:46 AM

File

epjconf_sos2012_02001.pdf
Publisher files allowed on an open archive

Identifiers

Collections

Citation

Balázs Kégl. Introduction to multivariate discrimination. IN2P3 School of Statistics (SOS2012), May 2012, Autrans, France. pp.022001, ⟨10.1051/epjconf/20135502001⟩. ⟨in2p3-00846125⟩

Share

Metrics

Record views

881

Files downloads

450