CSCI 567 Machine Learning

achoohomelessAI and Robotics

Oct 14, 2013 (3 years and 9 months ago)

187 views

CSCI 567 Machine Learning



Time
: Fall 2010, Tuesdays and Thursdays 2pm

3:20pm


Instructors:
Prof.
Fei Sha (
feisha@usc.edu
, 213
-
740
-
5924)



Introduction
and Purposes


This course

introduces students to the theory, algorithms and applications of modern

statistical
m
achine learning. Topics
include parametric and nonparamet
ric methods for
supervised,
u
nsupervised learning
and other paradigms.
Particular focuses
are
on the
theoretical understanding of these methods, as well as their computational implications.


This course assumes that students are familiar and comfortable with the mathematics
concepts a
nd tools outlined in the
Prerequisite
. At
the first meeting of the class, a
special quiz
(the grade does not count towards the final grade)
will be administered for
stu
dents to examine their preparation accordingly
.
An
undergraduate level course in
Artifi
cial Intelligence may be helpful but is not
required
.



This course also assumes that students are familiar with high
-
level programming
languages for scientific computation, in particular Matlab or Octave or R.


Prerequisite


U
ndergraduate level training or coursework in linear algebra, calculus, basic probability
and statistics
.
(Example coursework
at USC includes
topics in
MATH125, MATH126,
MATH208
x,
MATH

225, MATH
226,
or
above
or

s
imilar
ones from other departments
.)



Course Requirement and Grades




Format:
cla
ssroom lectures,
assigned readings, 6
homework
assignments/
mini
-
projects
, two in
-
class quizzes, and a final project
.




Textbooks:

o

Required:



Pattern recognition and machine le
arning (by Christopher Bishop)



C
hapters of textbooks in preparation (
hardcopy wil
l be distributed
in class or if necessary, a reader is available for purchase from the
university bookstore/library
).



Original research papers (publications from journals, conference
proceedings, etc).

o

Optional:



Elements of Statistical Learning (Trevor Ha
stie, Robert Tibshirani,
and Jerome Friedman)



All of Statistics (Larry Wasserman)



Pattern Classification (Richard O. Duda, Peter E. Hart, and David
G. Stork)



Introduction to Artificial Intelligence (by Stuar Russell and Peter
Norvig)



Machine Learning (by
Tom Mitchell)





Grading

o

Homework a
ssignmen
ts/mini
-
projects: 50%

o

Final project: 15%



A 2
-
page project proposal needs to be submitted, including project
title, team members, an extended abstract of the proposed
project, key references, and other critical info
rmation for carrying
out the project.



An 8
-
page project report needs to be submitted. The report should
contain detailed information about the project. The report will be
graded based on both scientific merits and clarity.



Team projects are allowed. Howev
er, there is a restriction on the
size of the team (at most 3 members) unless advised by the
instructor, after considering the workload of the proposed team.

o

Quiz 1: 15%

o

Quiz 2: 15%

o

Class participation: 5%


Courses Readings/class sessions


The
list of the
topics below is presented chronologically. It

is a
tentative
schedule
and
might be fine tuned
as the instruction progresses through the semester. (A much more
detailed version is also attached.)


Week 1: introduction, course overview, review of basic mat
h tools

Week 2: Supervised learning algorithms (linear regression and classification)

Week 3: Supervised learning algorithms (Bayesian techniques, neural networks)

Week 4: Supervised learning algorithms (nonparametric methods)

Week 5: Supervised learning a
lgorithms (margin based methods)

Week 6: Supervised learning algorithms (generalization bounds, metric learning or
selected topics)

Week 7: Quiz 1, review of Quiz 1 solutions

Week 8: Unsupervised learning algorithms (clustering and mixture of Gaussians)

W
eek 9: Dealing with incomplete data (EM) and linear subspace method (PCA)

Week 10: Probabilistic latent variable models (Probabilistic PCA)

Week 11: Other unsupervised learning algorithms (kernel PCA, manifold learning)

Week 12: Modeling temporal sequence
of data (Markov models)

Week 13: Hidden Markov models and Bayes net, introduction to undirected graphical
model

Week 14: Conditional models, maximum entropy

Week 15: Course wrap
-
up, overview of other learning themes in statistical learning,
Quiz 2


Sampl
e Assignments


The
homework
assignment
s
/mini
-
projects consist of several types of assignments to
fortify students’ understanding of the course material. The following is a list of sample
assignments:




Prove x
-
1

log x.



The attached data file lists rec
ent house sales in the greater Los Angeles area.
Please build a linear regression model to predict a property’s sale price using its
square footage. Submit your code, plots of the regression surface, and error
analysis.



Contrast the methods we have learnt
in class for latent variable modeling. What
are the pros and cons of each method? How would you design a new algorithm
to address those “cons”?





Statement for Students with Disabilities

Any student requesting academic accommodations based on a disabili
ty is required to
register with Disability Services and Programs (DSP) each semester. A letter of
verification for approved accommodations can be obtained from DSP. Please be sure
the letter is delivered to me (or to TA) as early in the semester as possibl
e. DSP is
located in STU 301 and is open 8:30 a.m.

5:00 p.m., Monday through Friday. The
phone number for DSP is (213) 740
-
0776.


Statement on Academic Integrity

USC seeks to maintain an optimal learning environment. General principles of academic
honesty
include the concept of respect for the intellectual property of others, the
expectation that individual work will be submitted unless otherwise allowed by an
instructor, and the obligations both to protect one’s own academic work from misuse by
others as w
ell as to avoid using another’s work as one’s own. All students are expected
to understand and abide by these principles.
Scampus,
the Student Guidebook, contains
the Student Conduct Code in Section 11.00, while the recommended sanctions are
located in App
endix A:
http://www.usc.edu/dept/publications/SCAMPUS/gov/
.
Students
will be referred to the Office of Student Judicial Affairs and Community Standards for
further review, should there be a
ny suspicion of academic dishonesty. The Review
process can be found at:
http://www.usc.edu/student
-
affairs/SJACS/
.


CSCI567 Fall 2010 Schedule
Topics
Readings
Week 1
24-Aug
Introduction, review of probability and inference,
quiz 0
Quiz 0
Ch 1.2.1-1.2.4, 1.6; 2.1-2.3,
Appendix B, notes
26-Aug
continuation of inference on Gaussians, linear
regression,
Ch 2.3, ch. 1.1, Ch 3.1
Week 2
31-Aug
Bayesian linear regression, bias-variance tradeoff
Ch 3.3, 3.2, Ch 1.3, 1.4
2-Sep
linear classification; generative vs. discriminative;
naïve bayes, perceptron
HW1 released
ch 4
Week 3
7-Sep
logistic regression; bayesian logistic regression,
model comparison
ch4
9-Sep
neural networks
ch 5
Week 4
14-Sep
nonparameteric methods, nearest neighbors
ch 2.5
16-Sep
kernel methods
HW1 due;HW2
released
ch 6
Week 5
21-Sep
support vector machines
ch 7
23-Sep
boosting
ch 14.3
Week 6
28-Sep
Generalization bounds
hand-out material
30-Sep
metric learning
HW2 due;HW3
released
hand-out material
Week 7
5-Oct
Quiz 1
7-Oct
Quiz 1 Review
Week 8
12-Oct
clustering
ch 9.1
CSCI567 Fall 2010 Schedule
14-Oct
Mixture of Gaussians
HW3 due;HW 4
released
ch 9.2
Week 9
19-Oct
EM
ch 9.3
21-Oct
PCA
ch 12.1
Week 10
26-Oct
Reinforcement learning
Final Project
Proposal Due
hand-out material
28-Oct
Probabilistic PCA, factor analysis
ch 12.2
Week 11
2-Nov
Kernel PCA and kernelized other approaches
ch 12.3
4-Nov
Other latent variable models
HW4 due; HW 5
released
ch 12.4
Week 12
9-Nov
Markov models
ch 13.1
11-Nov
Active learning
hand-out material
Week 13
16-Nov
Hidden Marov models
ch 13.2
18-Nov
Bayes net/Undirected graphical model
HW5 due; HW 6
released
ch 13.3, ch 8.1, 8.3
Week 14
23-Nov
Conditional model, max-entropy
hand-out material
25-Nov
Thanksgiving
Week 15
30-Nov
Course wrap-up, other trends in learning, quiz
review
HW 6 due
2-Dec
Quiz 2
7-Dec
Final Project
Report Due