Proximal boosting and its acceleration - Université de Paris - Faculté des Sciences Access content directly
Preprints, Working Papers, ... Year : 2020

Proximal boosting and its acceleration

Abstract

Gradient boosting is a prediction method that iteratively combines weak learners to produce a complex and accurate model. From an optimization point of view, the learning procedure of gradient boosting mimics a gradient descent on a functional variable. This paper proposes to build upon the proximal point algorithm when the empirical risk to minimize is not differentiable to introduce a novel boosting approach, called proximal boosting. Besides being motivated by non-differentiable optimization, the proposed algorithm benefits from Nesterov’s acceleration in the same way as gradient boosting [Biau et al., 2018]. This leads to a variant, called accelerated proximal boosting. Advantages of leveraging proximal methods for boosting are illustrated by numerical experiments on simulated and real-world data. In particular, we exhibit a favorable comparison over gradient boosting regarding convergence rate and prediction accuracy.
Fichier principal
Vignette du fichier
paper.pdf (1.9 Mo) Télécharger le fichier
Origin Files produced by the author(s)
Loading...

Dates and versions

hal-01853244 , version 1 (02-08-2018)
hal-01853244 , version 2 (22-01-2020)
hal-01853244 , version 3 (27-07-2021)
hal-01853244 , version 4 (29-11-2022)

Identifiers

  • HAL Id : hal-01853244 , version 2

Cite

Erwan Fouillen, Claire Boyer, Maxime Sangnier. Proximal boosting and its acceleration. 2020. ⟨hal-01853244v2⟩
368 View
323 Download

Share

Gmail Mastodon Facebook X LinkedIn More