Machine Learning Introduction

elbowcheepΤεχνίτη Νοημοσύνη και Ρομποτική

15 Οκτ 2013 (πριν από 3 χρόνια και 7 μήνες)

111 εμφανίσεις

Machine Learning Introduction

Why is machine learning important?

Early AI systems were brittle, learning can improve such a
system’s capabilities

AI systems require some form of knowledge acquisition,
learning can reduce this effort

KBS research clearly shows that producing a KBS is extremely time

dozens of man
years per system is the norm

in some cases, there is too much knowledge for humans to enter (e.g.,
common sense reasoning, natural language processing)

Some problems are not well understood but can be learned
(e.g., speech recognition, visual recognition)

AI systems are often placed into real
world problem solving

the flexibility to learn how to solve new problem instances can be

A system can improve its problem solving accuracy (and
possibly efficiency) by learning how to do something better

How Does Machine Learning Work?

Learning in general breaks down into one of two forms

Learning something new

no prior knowledge of the domain/concept so no previous representation
of that knowledge

in ML, this requires adding new information to the knowledge base

Learning something new about something you already knew

add to the knowledge base or refine the knowledge base

modification of the previous representation

new classes, new features, new connections between them

Learning how to do something better, either more efficiently or
with more accuracy

previous problem solving instance (case, chain of logic) can be
“chunked” into a new rule (also called memoizing)

previous knowledge can be modified

typically this is a parameter
adjustment like a weight or probability in a network that indicates that
this was more or less important than previously thought

Types of Machine Learning

There are many ways to implement ML

Supervised vs. Unsupervised vs. Reinforcement

is there a “teacher” that rewards/punishes right/wrong answers?

Symbolic vs. Subsymbolic vs. Evolutionary

at what level is the representation?

subsymbolic is the fancy name for neural networks

evolutionary learning is actually a subtype of symbolic learning

Knowledge acquisition vs. Learning through problem solving
vs. Explanation
based learning vs. Analogy

We can also focus on what is being learned

Learning functions

Learning rules

Parameter adjustment

Learning classifications

these are not mutually exclusive, for instance learning classification is
often done by parameter adjustment

Supervised Learning

The idea behind supervised
learning is that the learning
system is offered examples

The system uses what it
already knows to respond to
an input (if the system has
yet to learn, initial values are
randomly assigned)

If correct, the system
strengthens the components
that led to the right answer

If incorrect, the system
weakens the components that
led to the wrong answer

This is performed for each
item in the training set

Repeat some number of
iterations or until the system
“converges” to an answer

Below, we see that learning is
actually a search problem

The system is searching for the
representation that will allow it to
respond correctly to every (or most)
instance in the training set

There could be many “correct”

Some of these will also allow the
system to respond correctly to most
instances in the testing set

Forms of Supervised Learning

Most ML is some form of learning a function

F(x) = y where x is the input (typically comprised of (x
, x
, …, x
) for
some n
dimensional space, and y is the output

This form of learning typically breaks down into one of two forms:


the training items are mapped to distinct elements of a set


the training items are mapped to continuous values

In supervised learning, we have a training set of {x, y} pairs

Use the training set to “teach” the ML system

Many different approaches have been developed

neural networks using backpropagation


Bayesian networks

decision trees


Usually, once the system is trained, another data set (the test set) is run on
the system to see how it performs

There is a danger in this approach, overtraining the system means that it
learns the training set too well

it overfits to the training set such that it
performs poorly on the test set

Learning a Function

One of the most basic ideas in learning is to provide
examples of input/output and have the system learn the

The system will not learn, say f(x
, x
) = x

+ 3x

5 but
instead will learn how to map f(x
, x
) to an output (hopefully

The function will be learned only
based on how
useful the training set is and the specific type of learning
algorithm applied

Consider learning the function that

fits the data points plotted to the


there are many functions that

might fit

which one is correct?

Do we need to find a precise fit? If

not, how much error should we allow?


Earliest form of neural network

given a series of input/output pairs, identify the
linear separability (a hyper

e.g., a line in 2
d, a plane in 3

If the data points are linearly separable, the
perceptron learning algorithm is

to find it

many functions, such as XOR, are not linearly
separable, in which case perceptrons fail

An n
input perceptron computes

Weights are adjusted during learning to improve the

perceptron’s performance

this amounts to learning

the function that separates the “ins” from the “outs”

Think of the points as items that

are either in a given class or not,

the perceptron learns to classify

the items

Linear Regression

Another approach is based
on the statistical method of
regression analysis

Here, the strategy is to
identify the coefficients (such
below) to fit the
equation below, given the
data set of <x, y> values


is some random element

we need to expand on this to be
an n
dimensional formula since
our data will consist of elements
X = {x
, x
, x
, …, x
}, and y

There are a variety of ways to
do regression including
applying using some sort of
distribution (e.g., Gaussian),
applying the method of least
squares, applying Bayesian
probabilities, etc

note: neural networks are a
form of


= α + β



The more common form of supervised learning is that of a

the goal is to learn how to classify the data

f(x) = y means that x describes some input and y is its proper category
(again x is actually {x
, x
, …, x

Much of ML has revolved around classifiers

Naïve bayesian classifiers

Neural networks

K nearest neighbors



version spaces

decision trees

inductive logic programming

Some of these forms of classifiers are used heavily in data mining,
so we will hold off on discussion those until the next lecture (K
nearest neighbors, boosting, decision trees)

We will skip version spaces and inductive logic programming as they are
not as common today, but you might investigate them on your own

Bayesian Learning

Recall to apply Bayesian probabilities, we must either

have an enormous number of evidential hypotheses

or must assume that evidence are independent

The Naïve Bayesian Classifier takes the latter assumption

thus, it is known as naïve

p(C | e
, e
, e
) = P(C | e
) * P(C | e
) * P(C | e

rather than the more complex chain of probabilities that we saw previously

We can learn the prior and evidential probabilities by counting
occurrences of evidence and hypotheses amongst the data in the
training set

P(A | B) = # of times that A & B both appear in the training set / # of times
that B appears in the training set

P(A) = # of times that A appears / size of the training set

in case any of these values appears 0 times, we might want to “smooth” the
probability so that no conditional probability would ever be 0.0

smoothing is done by adding some “hallucinated values” to both the
numerator and denominator based on the size of the training set and some
established constant


Consider that I want to train a NBC on whether a particular text
based article is one that I would like to read

Given a set of training articles, mark each as “yes” or “no”

Create the following probabilities:


| yes) = probability that word i appears in an article i want to read


| no) = probability that word i appears in an article i do not want to read

) = probability that word i appears in an article

this is known as the “bag of words” approach

Now, given an article, compute
P(yes | words) and P(no | words)
where words = word
, word
, word
… for each unique word in the

We can enhance this strategy by

removing common words

using phrases

making sure that the bag contains
important words

Accuracy of the NBC given

training set of size 0

Learning in Bayesian Networks

Rather than assuming evidential independence, we might prefer Bayesian nets

We cannot learn (compute) the complex probabilities in a Bayesian network

e.g., P(A | B & C & ~D

What we can do, given these probabilities (or estimates), is learn the proper (best)
structure for the Bayesian net

this is done by taking our original network, making some minor change(s) to it,
computing the result’s probability, and selecting the network with the highest
probability for that result

For instance, in the
figure to the right, we
want to know P(T | …)

We compute that
probability on several
versions of the Bayesian
net and select the
network that provides
the highest resulting
probability in which T
was found to be true

Introduction to Neural Networks

After proving perceptrons could not learn XOR,
research into connectionism died for about 15 years

A new learning algorithm, backpropagation, and a new type
of layered network, the Artificial Neural Network, led to a
revised interest in connectionism

To the right is a multi
layered ANN

I inputs

some (0 or more) intermediate
levels known as hidden layers

O outputs

Each layer is completely
connected to the next layer

Each edge has its own weight

The goal of the backprop
algorithm is to train the
ANN to learn proper weights

NN Supervised Learning


feed forward the input

most NN use a sigmoid function to compute the output of a given node but
otherwise, it is like computing the result of a perceptron node

Determine the error (if any) by examining each output node and
comparing the value to the expected value from the training set

Backpropagate the error from the output nodes to the hidden layer
nodes (formula for weight adjustment on the next slide)

Continue to backpropagate the error to the previous level (another
hidden layer or the input)

note that since we don’t know what a given
hidden layer node was supposed to be, we can’t
directly compute an error here, we have to
therefore modify our formula for adjusting the
weight (again, see the next slide)

Repeat the learning algorithm on the next
training set item

Repeat the entire training set until the
network converges (weights change less
than some

How to Adjust Weights

For the weights connecting the hidden layer to the
output, we adjust a weight w

as follows


= w

+ sf * o

* (1

) * (e


) * i

sf is the scaling factor

this controls how quickly the network learns


is the output value of node j


is the expected value for output node j (as dictated by the training set

i is the input value

We do not know e

for the hidden layer nodes, so we
have to revise the formula to adjust the weights between
hidden layer a and hidden layer b, or between the input
layer and the hidden layer


= w

+ sf * o

* (1

) * Sum (w

* v
) * i


is the weight connecting this node to node k in the next layer and v

is the value that node k provided during the feed
forward part of the

Learning Example

Assume an input = <10, 30, 20> and expected output is <1, 0> from our

training set. Use a scaling factor of 0.1.

Part 1: Feed forward

H1 receives 7, H2 receives

H1 outputs = .9990, H2 outputs .0067

O1 receives 1.0996, O2 receives 3.1047

O1 outputs .7501, O2 outputs .9571

Recall computing output uses

the sigmoid function below

Example Continued

Part 3: Compute Error for Hidden Units:

Back prop to H1: (w11*δ01) + (w12*δO2) =

0.0394) =

Compute H1’s error (multiply by h1(E)(1

0.0706 * (0.999 * (1
0.999)) = 0.0000705 = δH1

Back prop to H2: (w21*δ01) + (w22*δO2) =

0.0394) =

Compute H2’s error (multiply by h2(E)(1

0.0414 * (0.067 * (1
0.067)) =
0.00259= δH2

Part 2: Compute Error at Output

O1 should be 1.0, O2 should be 0.0

Example Continued

Part 4: Adjust weights as new weight = old weight + scaling factor * error

Over or Under Training

The scaling factor controls how quickly the network can
learn so why not make it a large value?

What the NN is actually doing is performing a task called
gradient descent

weights are adjusted based on the derivative of the cost function

the learning algorithm is searching for the absolute minimum value,
however because we are moving in small leaps, we might get stuck in a
local minima

a local minima may learn the training set well, but not the testing set

So we control just how well the NN learns to classify the
domain by

the scaling factor

the number of epochs

the training data set

But also impacting this is the structure and size of the
network (which also impacts the number of epochs that it
might take to train the network)

What a Neural Network Learns

There has been some confusion regarding what a NN can
do and what it learns

The weights that a NN learns is a form of distributed

more specifically a distributed statistical
model of what features are important for a given class

Aside from the input and output nodes, the hidden layer nodes
do not represent any single thing but instead, groups of them
represent intermediate concepts in the domain/problem being

The facial recognition NN (on the

right) has learned to recognize

what direction a face is turned:

up, right, left or straight).

The hidden layer’s three nodes,

when analyzed, are storing the

pixels that make up the three

rough images of a face turned

in one of the directions

Problems with NNs

In terms of learning, NNs surpass most of the previously
mentioned methods because they learn via non
linear regression

A NN might be stuck in a local minima resulting in excellent performance
on the training set but poor performance on the test set

The number of epochs (iterations through the training set) is extremely

it might take a few dozen epochs, in other cases, a million epochs

There is no way to predict, given the structure of a network, how well or
quickly it will learn

NNs are not understandable by us, so we can’t really tell what the
NN has learned or how the information is represented

NNs cannot generate explanations

NNs do poorly in knowledge
intensive problems (e.g., diagnosis)
but very well in
recognition problems (e.g., OCR)

NNs have a fixed sized input so problems that deal with temporal
issues (e.g., speech rec) perform problematically, but recurrent
NNs are one way to possibly get around this problem

Avoiding Some of These Problems

To avoid getting stuck in a local minima,

one strategy is to use an additional factor

called momentum which in effect changes

the scaling factor over time

One form of this is called

simulated annealing

To avoid over fitting the training set,

do not use accuracy on the training set,

instead every so often, test the testing

set and use the accuracy on that set to

judge convergence

HMM Learning

Known as the EM algorithm or Baum
Welch algorithm

Use one training set item with observations o
, o
, …, o

Work through the HMM, one observation at a time

Once you have “fed forward” this example

for each time interval t and each state transition from i at time t to j at time
t+1, compute the estimator probability of transitions from i to j

(i) * a

* b
) *

(i) =

) * b

(j) =

(i) * a

* b


is the transition from i to j

and b
) is the output probability, which is the probability of observable O

being seen at state I

Now modify each transition probability a

and output probability b
) as

New a

= estimator probability from i to j / number of transitions out of i

New b
) =
(i) *
(i) / expected number of times in j

When done with this iteration, replace the old transition
probabilities with the new probabilities and r
epeat with the next
training set example until either the HMM converges, or you have
depleted the examples

Genetic Algorithms

Learning through manipulation of a feature space

The state is a vector representing features

binary vector

feature is present or absent

valued vector

features represented by a discrete or continuous

Supervised learning requiring a method of determining how
good a given feature vector is

learning is viewed as a search problem: what is the ideal or optimal

Natural selection techniques will (hopefully) improve the
performance of the search during successive iterations (called

this form of learning can be used to learn recognition knowledge, control
knowledge, planning/design knowledge, diagnostic knowledge

The “genetics” come in by considering that the vector is a
chromosome which is mutated by various random operations,
and then evaluated

the most fit chromosomes survive to
become parents for the next generation

General Procedure for GAs

Repeat the following until either you have exceeded
the number of stated generations or you have a vector
that is found suitable

Start with a population of parent vectors

Breed children through mutation operations

Apply the fitness function to the children

Select those children which will become parents of the next


What is the fitness function? Is there a reasonable one

What mutation operations should be applied and how
randomly? Should children be very similar to the parents or
highly different?

How many children should be selected for the next
generation? How many children should be produced by the

How is selection going to take place?

Fitness and Selection

Unlike other forms of supervised learning where feedback is a
previously known classification or value, here, the feedback for
the worth of a vector is in the form of a fitness function

given a vector V, apply the function f(V)

use this value to determine this vector’s worth towards the next generation

a vector that is highly rated may be selected in forming the next generation of
vectors whereas a vector that is lowly rated will probably not be used (unless
randomly selected)

How do you determine which vectors to alter/mutate?

Fitness Ranking

use a fitness function to select the best available vector
(or vectors) and use it (them)

Rank Method

use the fitness function but do not select the “best”, use
probabilities instead

Random Selection

in addition to the top vector(s), some approaches
randomly select some number of vectors from the remaining, lesser ranked


determine which vectors are the most diverse from the top
ranked one(s) and select it (them)

Mutation and Selection Mechanisms

Standard mutation methods are


moving around values in a vector

If p1 = {1, 2, 3, 4, 5, 6}, then this might result in {1, 5, 4, 3, 2, 6}


changing a feature’s value to another value

crossover (requires two chromosomes)

randomly swap some portion of
the two vectors

If p1 = {5, 4, 3, 2, 6, 1} and p2 = {1, 6, 2, 3, 4, 5}, crossover may yield the two
children {5, 4, 2, 3, 4, 1} and {1, 6, 3, 2, 6, 5}

How do you determine which vectors to alter/mutate?

Fitness ranking

select the best available vectors

Rank Method

rank the vectors as scored by the fitness function and then
use a probabilistic mechanism for selection

if v1 is .5, v2 if .3 and v3 is .15 and v4 is .05, then v1 has a 50% chance of
being selected, v2 has a 30% chance, v3 has a 15% chance and v4 a 5% chance

Random Selection

select the top vector(s) and select the remainder by
random selection


select the top vector(s) and then select the remainder by finding
the most diverse from the ones already selected

Genetic Programming

This form of learning is most
commonly applied to
programming code

unlike the GA approach, here the
representation is some dynamic
structure, commonly a tree

the process of inversion, mutation or
crossover is applied

Since trees are formed out of
syntactic parses of programs, we
can manipulate a program using
this approach

notice that by randomly
manipulating a program, it may no
longer be syntactically valid
however if we just use crossover, the
result will hopefully remain
syntactically valid (why?)

What kind of fitness function might
be used?

Other Forms of Learning

Reinforcement learning

A variation on supervised learning

a learner must determine
what action to take in a given situation that maximizes its

it does this through trial and error rather than through
training examples

reinforcement learning is not a new learning technique but rather a type
of problem which can be solved by any of a number of techniques
including those already seen (NNs, HMMs,

Unsupervised learning

No training set, no feedback, a form of discovery

Commonly uses either a Bayesian inference to produce
probabilities, or a statistical approach and clustering to produce
class descriptions

mostly a topic for data mining, also sometimes referred to as discovery

based Learning

Back in the 1970s, machine learning mostly revolved around
learning new concepts in a knowledge base

Version spaces

offering positive and negative examples of a class to learn
the features that distinguish items that are in versus out of the class, see for

Explanation based learning

given a KB, offer one or more examples of a
concept and have the system add representations that fit the new concepts
being learned

a commonly sited example is to add to a chess program’s
capability by understanding the strategy of a fork, see for example


taking a model in one domain and applying it to another domain,
often done through case based reasoning


finding patterns in data, what we now call data mining, one
early example was pioneered in a system called BACON that analyzed data
to find laws (which also reasoned using analogy)

it was able to infer Kepler’s third law, Ohm’s law, Joule’s law, and the
conservation of momentum by analyzing data