* Tutorial Notes Now Available Here *

aspiringtokΤεχνίτη Νοημοσύνη και Ρομποτική

15 Οκτ 2013 (πριν από 3 χρόνια και 7 μήνες)

64 εμφανίσεις

Bayesian Methods for Machine Learning

Zoubin Ghahramani

* Tutorial Notes Now Available Here *

Topic

Many topics in Machine Learning
(e.g. kernel methods, clustering, semi
-
supervised learning, feature selection, active learning, reinforcement learning)
can be addressed within the framework of Bayesian statistics. While the
proportion of work in machine learning based on statistical mode
lling has
grown over the past few years, there remains a good degree of skepticism
with respect to taking a fully Bayesian approach. This tutorial aims at
introducing fundamantal topics in Bayesian statistics as they apply to machine
learning problems, and

addressing some misconceptions about Bayesian
approaches. The tutorial will also attempt to present a balanced view of the
limitations of Bayesian approaches. Finally the tutorial will delve into some of
the practical issues in current

Bayesian machine learning

including the role of
approximation algorithms, sampling methods, and nonparameterics.

Intended audience

The tutorial is intended for the broad ICML community. Little prior knowledge
is assumed (other than basic probability theory). The participants will hopefully
get both a big picture of Bayesian approaches to machine learning and some
insight into specific state
-
of
-
the
-
art methods.

Detailed Outline

[total 180 minutes, outline subjec
t to revision]

1.

Three canonical problems


[10 minutes]

o

Linear Classification

o

Coin Toss

o

Clustering with Gaussian Mixtures

2.

Foundations


[30 minutes]

o

Representing beliefs and the Cox Axioms

o

Dutch Book Theorem

o

Asymptotic Convergence and Consensus

o

Occam's
Razor

o

Priors: Objective, Subjective, Hierarchical and Empirical Bayes

o

Exponential Family and Conjugate Priors

o

How to choose priors?

3.

Intractability


[10 minutes]

o

Bayesian inference in Gaussian mixtures and linear classifiers

o

Hidden variables, parameters
and partition functions

4.

Approximation Tools


[40 minutes]

o

BIC

o

Laplace Approximation

o

Variational Approximations

o

MCMC

o

Exact Sampling

break

5.

Feature Selection, Model Selection and Bayesian Methods


[20
minutes]

o

Do we need to select features?

o

Automatic Re
levance Determination

o

Model selection criteria and model averaging

6.

Bayesian Discriminative Modelling


[20 minutes]

o

Myth: Bayesian methods = Generative models

o

Bayes Point Machines vs Support Vector Machines

o

Bayesian Neural Networks

7.

From Parametric to
Nonparametric Bayes


[20 minutes]

o

Gaussian Processes

o

Dirichlet Processes and Infinite Mixtures

o

Other non
-
parametric Bayesian models

8.

Further Topics


[15 minutes]

o

Bayesian Active Learning and Bayesian Decision Theory

o

Bayesian Semi
-
supervised Learning

o

R
econciling Bayesian and Frequentist Views

9.

Open Discussion of Limitations and Criticisms


[10 minutes]

o

Philosophical

o

Practical

o

Computational

10.

Other Questions from the Audience

Format

The format will be data
-
projected slides; I will also occasionally use
the
whiteboard and have some simple Matlab demos to illustrate some ideas.
However, the focus won't be on algorithms but rather on concepts.

Presenter

Zoubin Ghahramani is a Reader in Machine Learning at the Gatsby Unit in
London, and an Associate Research

Professor at CALD at CMU. He has
given tutorials at NIPS, ICANN, the Machine Learning Summer School in
Canberra, and various other summer schools. He is interested in

Bayesian
machine learn
ing
, computational approaches to sensorimotor control, and
applications of machine learning to bioinformatics.

Zoubin Ghahramani

Gatsby Computational
Neuroscience Unit


University College London


17 Queen Square, Room 403


London WC1N 3AR


United Kingdom


Tel +44 (0)20 7679 1199


Fax +44 (0)20 7679 1173


Email zoubin "AT" gatsby.ucl.ac.uk


http://www.gatsby.ucl.ac.uk/~zoubin

Center for Automated Learning and Discovery

School of Computer Science

Carnegie Mellon University

Pittsb
urgh, PA

USA