In this paper we bound the risk of the Gibbs and Bayes classifiers (GC and BC), when the prior is defined in terms of the data generating distribution, and the posterior is defined in terms of the observed one, as proposed by Catoni (2007). We deal with this problem from two different perspectives. From one side we briefly review and further develop the classical PAC-Bayes analysis by refining the current state-of-the-art risk bounds. From the other side we propose a novel approach, based on the concept of Algorithmic Stability, which we call Distribution Stability (DS), and develop some new risk bounds over the GC and BC based on the DS. Finally, we show that the data dependent posterior distribution associated to the data generating prior has also attractive and previously unknown properties.

PAC-bayesian analysis of distribution dependent priors: Tighter risk bounds and stability analysis

ONETO, LUCA;ANGUITA, DAVIDE;RIDELLA, SANDRO
2016-01-01

Abstract

In this paper we bound the risk of the Gibbs and Bayes classifiers (GC and BC), when the prior is defined in terms of the data generating distribution, and the posterior is defined in terms of the observed one, as proposed by Catoni (2007). We deal with this problem from two different perspectives. From one side we briefly review and further develop the classical PAC-Bayes analysis by refining the current state-of-the-art risk bounds. From the other side we propose a novel approach, based on the concept of Algorithmic Stability, which we call Distribution Stability (DS), and develop some new risk bounds over the GC and BC based on the DS. Finally, we show that the data dependent posterior distribution associated to the data generating prior has also attractive and previously unknown properties.
File in questo prodotto:
File Dimensione Formato  
J017 - PRL.pdf

accesso chiuso

Tipologia: Documento in versione editoriale
Dimensione 580.21 kB
Formato Adobe PDF
580.21 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11567/845907
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 17
  • ???jsp.display-item.citation.isi??? 12
social impact