Introduction to ML

hesitantdoubtfulAI and Robotics

Oct 29, 2013 (3 years and 1 month ago)

150 views

Introduction to Machine
Learning

Anjeli Singh

Computer Science and Software
Engineering


April 28
th

2008

Overview


What is Machine Learning


Examples of Machine Learning


Learning Associations


Classification


Regression


Unsupervised Learning


Reinforcement Learning


Notes



What is Machine Learning?


We store and process data


Supermarket chain


Hundred of stores


S
elling thousands of good to million of customers


Record the details: date, customer ID, goods,
money


Gigabytes of data everyday


Turn this data into information to make prediction



What is Machine Learning?


Do we know which people are likely to buy a
particular product?


Which author to suggest to people which
enjoy reading?


What is Machine Learning?

Answer is : NO

What is Machine Learning?

Answer is :NO

Because if we knew, we won’t need
any data analysis.

Go ahead and write code

What is Machine Learning?


We can collect data


Try to extract answers to these similar questions


There is a process explains the data we observe


Its not completely random E.g., Customer
behavior



People don’t buy random things


E.g. when they buy beer they buy chips


There are certain pattern in the data

What is Machine Learning?


We can’t identify process completely


We can construct good approximation


Detect certain pattern and regularities


This is the niche of Machine Learning


We can use these patterns for prediction


Assuming near future won’t be much different
the past

What is Machine Learning?


Application to large database: data mining


Its an analogy for extracting minerals from
Earth


Large volume of data is processed


To construct a simple model with valuable use


Having a high predictive accuracy


Application :


Retail, finance banks, manufacturing, medicine for
diagnosis





What is Machine Learning?


Its not a database problem


To be intelligent, a system that is in changing
environment should have the ability to learn


System should learn and adapt


Foresee and provide solution for all possible
situations





Mathematics Problem


2 + 2=

Mathematics


2 + 2= 4

What is Machine Learning?



How we did that ?


Can we write a program to add two numbers
??


Faces


Who is He ??

Faces

Denzel Washington

What is Machine Learning?


How do we acknowledge him ?


Can you write a program for that???

What is Machine Learning?


Pattern Recognition Problem


Analyzing sample face images of that person


Learning captures the pattern specific to the
person


Recognize by checking this person in a given
image




What is Machine Learning?

“Machine learning is programming computers to
optimize a performance criterion using
example data or past experience”



Uses theory of statistics to build mathematical
models


Core task is to make inferences from a sample




What is Machine Learning?


Role of Computer Science


Training, need efficient algorithms to solve
optimization problems


To store and process massive data


I
ts representation and algorithmic solutions for
inferences


Eg
, the efficiency of learning and or inference
algorithm its space and time complexity may be
as imp as predictive accuracy


Examples of Machine Learning


Learning Associations


Classification


Regression


Unsupervised Learning


Reinforcement Learning


Notes


Learning Associations


In case of retail: Basket Analysis


Finding associations between product bought
by customers


If a customer buy X, typically also buy Y


To find Potential Y customer


Target them from cross selling



Learning Associations


Association Rule


P(Y|X) where Y is the product we condition on X
and X is the product which a customer has
already purchased


Eg
. P(
chips|Beer
) = 0.7 Then 70 % who buy beer
also chips


Distinction Attribute:


P(Y|X, D) D set of customer attributes


Gender, age, marital status


Classification


Credit card Example


Predict the chances of paying loan back


Customer will default /not pay the whole amount


Bank should get profit


Not inconvenience a customer over his financial
capacity

Classification


Credit Scoring


Calculate the risk given the amount and customer
information


Customer information.
Eg
., Income, savings,
profession, age, history etc.


Form a rule


Fits a Model to the past data


To calculate the risk for a new application





Classification


Classes


Low Risk




Savings


High Risk



Ѳ
2



Rule (Prediction)





Ѳ
1

Income

If income>
Ѳ
1

AND savings>
Ѳ
2 THEN low
-
risk ELSE
high
-
risk

Example of
discriminant


Low Risk

High Risk

Classification


Decision Type


0/1 (low
-
risk/high
-
risk)


P(Y|X) where Y customer attribute and X is 0/1


P(Y=1|X=x) =0.8



Classification


Pattern Recognition


Optical character Recognition


Recognizing character code from images


Multiple classes


Collection of strokes, has a regularity(not random
dots)


Capture in learning a program


Sequence of characters
eg

.
T?e

word


Face recognition

Classification


Speech recognition


Input is acoustic and classes are words



Medical Diagnosis


Classification


Knowledge Extraction


Learning a rule from data


Compression


Fitting a rule to the data


Outlier Detection


Finding the instances that do not obey rule are
exceptions


E.g. Fraud



Regression


If something can predict
price of a used car?


Input: Brand, year, engine
capacity


Output: Car Price






Regression


X denote car attribute


Y be the price


Survey past transaction


Collect training data
y:price


Fitted function


Y = wx+w
0





x:
milage

Supervised Learning


There is an input and an output


Learn mapping from input to output


Model defined up to a set of parameters:



y = g(x|
Ѳ
)



g(.) is the model and
Ѳ

are its parameters



y is the number of regressions or a class

code (0/1)



Unsupervised Learning


Only have input data


Aim is to find regularities in the input


There is a pattern


Unsupervised Learning


To find the regularities in the input


Structure in the input space


Density Estimation


Clustering


Image Compression

Reinforcement Learning

Notes


Evolution defines us


We change our behavior


To cope with change


We don’t hardwire all sort of behavior


Evolution gave us mechanism to learn


We recognize, recall the strategy


Learning has limitation also



Can we grow a third arm??

Evolution in ML


Our aim is not understand the process
underlying learning in human


To build useful systems as in domain of
engineering

Science fitting models of data


Design experiments, observe and collect data


Extract knowledge by finding out simple
models, that explains the data


Process of extracting general rules from a set
of cases is
Induction


Going from particular observation to general
description


Statistics:
inference

Learning:
estimation



Relevant Resources


Journal of Machine
Learning Research


Neural Computation


Neural Information
Processing System (NIPS)


Book:
Introduction to
Machine Learning by
Ethem

ALPAYDIN



The MIT Press