x

AI and Robotics

Nov 7, 2013 (4 years and 6 months ago)

74 views

Machine Learning 9
Definition of Learning
 A computer program is said to learn from experience E with respect to
some class of tasks T and performance measure P, if its performance at
tasks in T, as measured by P, improves with experience E.
 Handwriting recognition
 T: recognizing/classifying handwritten characters
 E: handwritten characters with correct answers
 P: percent of characters correctly recognized/classified
 Vehicle driving
 T: driving on public highways
 E: front/sides/back view images and steering commands
 P: distance traveled before an accident
Machine Learning 10
Issues in Designing Learning Systems
 Type of experience (training examples)
 Type of knowledge (hypotheses) to learn
 Type of learning algorithms
 Checkers playing
 T: playing checkers
 E: playing against people
 P: chance of win
Machine Learning 11
Type of Experience
 Direct or indirect feedback from teachers
 Correct move for each board state vs. final outcome
 Indirect feedback requires additional inference for credit assignment
 Learning with direct feedback is easier, but it is usually not available
 Active or passive control of training examples
 Passive : teachers provide training examples (at random)
 Active : learners ask for interesting examples
 Explore : learners explore environments
 Self learning : active & indirect learning (no teacher)
 Quality of training examples
 Difference between training examples and testing examples
 Theoretical error bounds
Machine Learning 12
Type of Knowledge to Learn
 Vehicle driving
 Relationship between view images and control commands
 Checkers playing
 Relationship between the board states and best moves
• Target functions : B M
• Evaluation functions V: B 
– Higher values, better board states
– V(b) = { +100, –100, 0, V(b) }
– V(b) = w
0
+ w
1
x
1
+ w
2
x
2
+ w
3
x
3
+ w
4
x
4
+ w
5
x
5
+ w
6
x
6
Machine Learning 13
Training Examples
 A training example is an ordered pair
 b, V
t
(b)
 x
1
=3, x
2
=0, x
3
=1, x
4
=0, x
5
=0, x
6
=0, +100
 x
1
=3, x
2
=3, x
3
=1, x
4
=0, x
5
=0, x
6
=0, ?
 x
1
=3, x
2
=3, x
3
=0, x
4
=0, x
5
=0, x
6
=1, ?
 How to generate training examples for indirect training experience
 V
t
(b) V(S(b))
 b, V(S(b))
Machine Learning 14
Training Algorithms
 Learning find w
i
’s that best fit training examples
 Minimize E =  (V
t
(b) – V(b))
2
 V(b) = w
0
+ w
1
x
1
+ w
2
x
2
+ w
3
x
3
+ w
4
x
4
+ w
5
x
5
+ w
6
x
6
 Least mean squares (LMS) algorithm
while b (V
t
(b)-V’(b))0
for each training example b,V
t
(b)
compute V’(b) using current w
i
for each w
i
w
i
 w
i
+ (V
t
(b)- V’(b))x
i
 V
t
(b) – V(b) = +  w
i

 V
t
(b) – V(b) = –  w
i

 x
i
= 0  w
i
not changed
Machine Learning 15
Trace of the LSM algorithm
 V(b) = w
0
+ w
1
x
1
+ w
2
x
2
+ w
3
x
3
+ w
4
x
4
+ w
5
x
5
+ w
6
x
6
 w
i
 w
i
+ (V
t
(b)- V’(b))x
i
 x
1
=3, x
2
=1, x
3
=0, x
4
=0, x
5
=0, x
6
=0, +75 w
1

 x
1
=1, x
2
=3, x
3
=0, x
4
=0, x
5
=0, x
6
=0, –75 w
2

Machine Learning 16
A Checkers Playing System
 T: playing checkers
 P: percent of games won in the world tournament
 E: playing against itself
 Target function (knowledge)
 V: B 
 V(b) = w
0
+ w
1
x
1
+ w
2
x
2
+ w
3
x
3
+ w
4
x
4
+ w
5
x
5
+ w
6
x
6
 Training examples (experience)
 b, V
t
(b)
 V
t
(b) V(S(b))
 Training algorithm
 LMS
Machine Learning 17
Components in Learning System – 1
 Critic
 Generates training examples
 e.g. Trace of game {b, V
t
(b), … }
 Generalizer
 Learns from the training examples
 e.g. Estimates the target function, i.e., w
i
’s
 Experiment generator
 Find new experience
 Performance system
 Performs the given task
 e.g. Checkers playing program
Machine Learning 18
Components in Learning System – 2
Machine Learning 19
Conclusions
 What types of training examples are available?
 What (approximation) functions to learn?
 How to represent the hypotheses?
 What algorithms to use to learn?
 How well the algorithms will perform?
 How much training data is needed?
Machine Learning 20
Preview – 1
 Chapter 2
 Concept learning / inductive learning
 Version spaces
 Candidate-Elimination learning algorithm
 Inductive bias
 Task : Enjoy sport?
 Chapter 3
 Information theory : entropy
 ID3 decision tree learning algorithm
 Occam’s razor
 Task : Enjoy sport?
Machine Learning 21
Preview – 2
 Chapter 4
 Neuron / perpectron
 MLP (multi-layer perceptron) : backpropagation algorithm
 Inductive bias
 Task : vehicle driving, face recognition
 Chapter 5
 Statistics
 Confidence intervals
 Comparing learning algorithms
Machine Learning 22
Preview – 3
 Chapter 6
 Bayes theorem
 Bayes optimal classifier learning algorithm
 Naive Bayes classifier learning algorithm
 Bayesian belief networks learning algorithm
 EM (expectation maximization) algorithm
 Comparison with CE/ANN/DT and Bayesian learning
 Task : Text classification
 Chapter 7
 PAC (probably approximately correct) learning model
 VC (Vapnik-Chervonenkis) dimension
 Halving algorithm
 Weighted-majority algorithm
Machine Learning 23
Assignments
 Programming assignments
 ID3
 Error backpropagationn
 Nave Bayes classifier
 Task : Speech recognition (b/d/g/k/p/t classification)
 Experiments
 Presentations