chap18

journeycartAI and Robotics

Oct 15, 2013 (4 years and 23 days ago)

127 views

David R. Musicant

Machine Learning


Definition 1


“The subfield of AI concerned with
programs that learn from experience”


Russell / Norvig, AIMA


Definition 2


“the application of induction algorithms,
which is one step in the knowledge
discovery process.”


Machine Learning definition in glossary from
Machine Learning

at
http://robotics.stanford.edu/~ronnyk/glossary.html


David R. Musicant

Supervised Learning Classification


Example: Cancer diagnosis


Use this
training set

to learn how to classify patients
where diagnosis is not known:


The
input data

is often easily obtained, whereas the
classification

is not.

Input Data

Classification

Training Set

Test Set

David R. Musicant

Classification Problem


Goal: Use training set + some learning method to
produce a
predictive model
.


Use this predictive model to classify new data.


Sample applications:

David R. Musicant

Application: Breast Cancer Diagnosis

Research by Mangasarian,Street, Wolberg

David R. Musicant

Breast Cancer Diagnosis Separation

Research by Mangasarian,Street, Wolberg

David R. Musicant

Application: Document Classification


The Federalist Papers


Written in 1787
-
1788 by Alexander Hamilton, John Jay, and
James Madison to persuade residents of the State of New
York to ratify the U.S. Constitution


All written under the pseudonym “Publius”


Who wrote which of them?


Hamilton wrote 56 papers


Madison wrote 50 papers


12 disputed papers, generally understood to be written by
Hamilton or Madison, but not known which

Research by Bosch, Smith

David R. Musicant

Federalist Papers Classification

Research by Bosch, Smith

Graphic by Fung

David R. Musicant

Application: Face Detection


Training data is a collection of Faces and NonFaces


Rotation and Mirroring added in to provide robustness

Image obtained from work by Osuna, Freund, and Girosi at

http://www.ai.mit.edu/projects/cbcl/res
-
area/object
-
detection/face
-
detection.html

David R. Musicant

Face Detection Results

Image obtained from "Support Vector Machines: Training and Applications" by Osuna, Freund, and Girosi.

David R. Musicant

Face Detection Results

Image obtained from work by Osuna, Freund, and Girosi at

http://www.ai.mit.edu/projects/cbcl/res
-
area/object
-
detection/face
-
detection.html

David R. Musicant

Nearest Neighbor


Simple effective approach for supervised
learning problems






Envision each example as a point in n
-
dimensional space


Picture with 2 of them


Classify test point same as nearest training
point (Euclidean distance)

David R. Musicant

k
-
Nearest Neighbor


Nearest Neighbor can be subject to
noise


Incorrectly classified training points


Training anomalies


k
-
Nearest Neighbor


Find
k

nearest training points (
k

odd) and
vote on which classification


Training time?


Testing time?


Works on numerical data