Introduction to Machine Learning

educationafflictedΒιοτεχνολογία

4 Οκτ 2013 (πριν από 3 χρόνια και 6 μήνες)

72 εμφανίσεις


MACHINE LEARNING

1. Introduction

What is Machine Learning?

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

2



Need an algorithm to solve a problem on a
computer


An Algorithm is a sequence of instructions to
transform input from output


Example: Sort list of numbers


Input: set of numbers


Output: ordered list of numbers


Many algorithms for the same task


May be interested in finding the most efficient

What is Machine Learning?

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

3


Don’t have an algorithm for some tasks


Example: Tell the spam e
-
mail for legitimate e
-
mails


Know the input (an email) and output (yes/no)


Don’t know how to transform input to output



Definition of spam may change over the time and from
individual to individual


We don’t have a knowledge, replace it with data



Can easily produce large amount of examples


Want a computer to extract an algorithm from the
examples

What is Machine Learning?

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

4



Believe that there is a process explaining the data



We don’t know details about the process we know
it’s not random


Example: Consumer Behavior


Frequently buy beer with chips



Buy more ice
-
cream ins summer


There is certain patterns in data


Rarely can’t indentify patterns completely


Can construct good and useful approximation


Approximations to Patterns

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

5


May not explain everything


Still detect some patterns and regularities



Use patterns


Understand the process


Make a prediction



Data Mining
: Application of ML to large
databases

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

6


Retail: Market basket analysis, Customer
relationship management (CRM)


Finance: Credit scoring, fraud detection


Manufacturing: Optimization, troubleshooting


Medicine: Medical diagnosis


Telecommunications: Quality of service optimization


Bioinformatics: Motifs, alignment


Web mining: Search engines


...

Examples of Ml Applications

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

7


Learning association: Basket Analysis


If people buy X they typically buy Y


There is a customer who buys X and don’t buy Y


He/She

is a potential Y customer


Find such customers and target them for cross
-
selling


Find an association rule: P(Y|X)


D customer attributes (e.
g.age
, gender)


P(Y|X,D)

Association Rules examples

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

8



Bookseller instead of supermarket


Products are books or authors



Web portal


Links the user is likely to click


Pre
-
download pages in advance


Credit Scoring

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

9


Bank want to predict a risk associated with loan


Probability that customer will default given the
information about the customer


Income, Savings, profession


Association between customer attributes and his risk


Fits a model to the past data to be able to
calculate a risk for a new application


Accept/Refuse application


Classification Problem

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

10



Two classes of customers: low
-
risk and high
-
risk


Input: information about a customer


Output: assignment to one of two classes


Example of classification rule


IF income
>
θ
1 AND savings>
θ
2 THEN low
-
risk ELSE
high
-
risk


Discriminant: function that separates the examples
of different classes

Discriminant Rule

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

11

Discriminant Rule

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

12


Prediction: User rule for novel instances


In some instances may want to calculate
probabilities instead of 0/1


P(Y|X), P(Y=1|X=x) =0.8 , customer has an 80%
probability of being high
-
risk

Pattern Recognition

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

13



Optical character recognition (OCR) , recognizing
character codes from their images


Number of classes as many as number of images
we would like to recognize



Handwritten characters (zip code on envelopes or
amounts on checks)


Different handwriting styles, character sizes, pen or
pencil


We don’t have a formal description that covers all
A’s characters and none of non
-
A’s

OCR

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

14


All A have something in common


Extract pattern from examples



Use redundancy in human languages


Word is a sequence of characters


Not all sequences are equally likely


Can still
r?ad

some
w?rds


ML algorithms should model dependencies among
characters in a word and word in the sequence

Face recognition

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

15


Input : image


Classes: people to be recognized


Learning program: learn to associate faces to
identities


More difficult then OCR


More classes


Images are larger


Differences in pose and lightening cause significant
changes in image


Occlusions: Glasses, beard

Face Recognition

Based on
E
Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

16

Training examples of a person

Test images

AT&T Laboratories, Cambridge UK

http://www.uk.research.att.com/facedatabase.html

Medical Diagnosis

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

17



Input: Information about the patient


Age, past medical history, symptoms


Output: illnesses


Can apply some additional tests


Costly and inconvenient


Some information might be missing


Can decide to apply test if believe valuable


High price of wrong decision

Speech Recognition

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

18


Input: acoustic signal


Output: words


Different accents and voices


Can integrate language models


Combine with lips movement


Sensor fusion

Knowledge Extraction

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

19


The rule is simpler than the data


Example: Discriminant separating low
-
risk and high
risk customer helps to define low risk customer


Target low risk customer through advertising

Compression

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

20



Explanation simpler than data



Discard the data , keep the rule


Less memory


Example: Image compression



Learn most common colors in image


Represent slightly different but similar colors by single
value

Outlier detection

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

21



Find instances which do not obey rules


Interesting not in a rule but an exception not
covered by the rule


Examples: Learn properties of standard credit card
transactions


Outlier is a suspected fraud

Why “Learn” ?

Based on

E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

22


Machine learning is programming computers to optimize
a performance criterion using example data or past
experience.


There is no need to “learn” to calculate payroll


Learning is used when:


Human expertise does not exist (navigating on Mars),


Humans are unable to explain their expertise (speech
recognition)


Solution changes in time (routing on a computer network)


Solution needs to be adapted to particular cases (user
biometrics)

What We Talk About When We Talk
About“Learning”

Based on
E
Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

23


Learning general models from a data of particular
examples


Data is cheap and abundant (data warehouses,
data marts); knowledge is expensive and scarce.


Example in retail: Customer transactions to consumer
behavior:


People who bought “Da Vinci Code” also bought “The
Five People You Meet in Heaven” (www.amazon.com)


Build a model that is
a good and useful
approximation

to the data.



What is Machine Learning?

Based On
E
Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

24


Optimize a performance criterion using example
data or past experience.


Role of Statistics: Inference from a sample


Role of Computer science: Efficient algorithms to


Solve the optimization problem


Representing and evaluating the model for inference

Regression


Example: Price of a
used car


x
: car attributes


y
: price



y
=
g
(
x
|
θ

)


g
( ) model,


θ

parameters

Based on
E
Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

25

y
=
wx
+
w
0

Supervised Learning

Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

26



Regression and classification are supervised learning
problems


There is an input and output


Need to learn mapping from input to output


ML approach: assume a model defined up to a set of
parameters


y = g(x|
θ)


Machine Learning Program optimize the parameters to
minimize the error



Linear model might be two restrictive (large error)



Use more complex models


y = w
2
x
2

+ w
1
x + w
0



Supervised Learning: Uses

Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

27


Prediction of future cases: Use the rule to predict
the output for future inputs


Knowledge extraction: The rule is easy to
understand


Compression: The rule is simpler than the data it
explains


Outlier detection: Exceptions that are not covered
by the rule, e.g., fraud

Unsupervised Learning

Based On
E
Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

28


Learning “what normally happens”


No output
, only input



Statistics: Density estimation


Clustering: Grouping similar instances


Example applications


Customer segmentation in CRM


Image compression: Color quantization


Document Clustering

Reinforcement Learning

Based on
Introduction
to Machine Learning © The MIT Press (V1.1)

29


Learning a policy: A sequence of outputs


Single action is not important


Action is good if its part of a good policy


Learn from past good policies


Delayed reward


Example:
Game playing


Robot in a maze


Reach the goal state from an initial state