BORDES Antoine

PhD graduated
Departure date : 09/30/2010

Supervision : Patrick GALLINARI

New Algorithms for Large-Scale Support Vector Machines

There exists a deep need for machine learning methods able to learn with millions of training instances so that they could enjoy the huge available data sources. In this thesis, we propose solutions to reduce training time and memory requirements of learning algorithms while keeping strong performances in accuracy. In particular, among all the machine learning models, we focus on Support Vector Machines (SVMs) that are standard methods mostly used for automatic classification. Throughout this dissertation, we propose different original algorithms for learning SVMs, depending on the final task they are destined to. First, we study the learning process of Stochastic Gradient Descent for the particular case of linear SVMs. This leads us to define and validate the new SGD-QN algorithm. Then we introduce a brand new learning principle: the Process/Reprocess strategy. We present three algorithms implementing it. The Huller and LaSVM are designed towards training SVMs for binary classi cation. For the more complex task of structured output prediction, we re fine intensively LaSVM: this results in the LaRank algorithm. We finally introduce the original framework of learning under ambiguous supervision which we apply to the task of semantic parsing of natural language. Each algorithm introduced in this thesis achieves state-of-the-art performances, especially in terms of training speed.

Defence : 02/09/2010

Jury members :

Stéphane Canu, Professeur et directeur du LITIS à l'INSA de Rouen. [Rapporteur]
John Shawe-Taylor, Professeur et directeur du CSML à l'University College London au Royaume-Uni. [Rapporteur]
Jacques Blanc-Talon, Responsable scientifique à la DGA/MRIS.
Léon Bottou, Distinguished senior researcher à NEC Labs of America aus Etats-Unis.
Matthieu Cord, Professeur au LIP6.
Patrick Galinari, Professeur et directeur du LIP6.
Bernhard Schölkopf, Professeur et et directeur du MPI for Biological Cybernetics en Allemagne.

Departure date : 09/30/2010

2007-2017 Publications

Mentions légales
Site map