Regression and Machine Learning

achoohomelessΤεχνίτη Νοημοσύνη και Ρομποτική

14 Οκτ 2013 (πριν από 3 χρόνια και 7 μήνες)

114 εμφανίσεις

Regression and
Machine Learning
Bianca Cung
Justin Hsueh
Levon Kolesnikov
Khang Lu
Least Squares
Linear Regression
An Introduction to Regression
What is Regression?
￿
Type of Data Mining
￿
Recall from Lecture 21: Data Mining is the
analysis of large amounts of data in order to
discover meaningful patterns
￿
Regression models and analyzes the
correlation of several variables
￿
Notably models to an X-Y graph
￿
Ex: Linear Regression models a linear graph
Predictions
Minimizing Errors
￿
To make a better
model, we minimize
the errors
￿
An error is considered
the distance between
the actual data and
the model data
Variance and the Best Fit Line
￿
Variance
: is a measure of how far a set of
numbers are spread out from each other
￿
Regression Line is drawn so that there is minimal
amount of error in predictions for all of the
already known values
￿
Whether variance is large or small, as long as
the total error is optimized to its minimum, then
line is best fit
Variance Affecting Predictions
Correlation and Causation
￿
Remember: Regression is a model of correlation
between two or more variables
￿
Correlation does not imply causation
￿
Correlation: Shows a connection between 2 or
more things
￿
Causation: Second event only arises if first event
occurs
￿
Examples:
￿
Correlation: Getting an A on the final correlates with
an A in the class
￿
Causation: Getting a 90% or higher on a test causes
a grade of A on the test
Assumptions
￿
Data represents whole population
￿
Error is a random variable
￿
Constant variance
￿
(Linear independence)
￿
(Uncorrelated errors)
Other TypesB
What if your data isn’t a straight lineB?
￿
Logarithmic Regression
Y = a + b (ln x)
￿
Quadratic Regression
Y = a*x
2
+b*x+c
￿
Power Regression
Y = a*x
b
￿
Exponential Regression
Y = a*b
x
Try Other TransformationsB
￿
Divide by X
￿
An exampleB
￿
Transform the Y
￿
Transform both Y and X
Y = a*exp(bx) is equivalent toB
ln Y = ln (a) + bx
Multiple Regression
Mile Time Gender Height
(inches)
Weight
(lbs)
Age GPA
10 1 62.2 120 20 3.3
11 0 64.5 166 21 2.8
7 1 70.1 132 18 4.0
8 0 75.0 133 23 1.6
14 0 58.9 121 19 3.7
10 0 68.8 100 25 3.5
Multiple Regression
Y = β
0
+ β
1
(gender) + β
2
(height) + β
3
(weight) +
β
4
(age) + β
5
(gpa)
￿
3+ variables and 1000+ observations
￿
Use a computer!
￿
As before, you can transform these variables if
the model does not fit
￿
By cautious of independence of variables
Machine Learning
An application of Regression
What is Machine learning?
￿
Part of artificial intelligence
￿
Creating algorithms allowing computers to evolve
behaviors based on empirical data
￿
Regression
￿
Automatically learn and recognize complex
patterns and make decisions
￿
Difficulty is that the set of possible behaviors can
be to large
Supervised learning Regression Problem
Autonomous Driving
￿
Learning algorithm-gradient descent
￿
Digitizes the road ahead and records the
person’s steering directions
￿
Once learnedB
￿
digitizes the road
￿
feeds the image to its neural networks.
￿
Measure each steering direction’s confidence
Alvin-system of artificial
neural networks
Gradient Descent
￿
Similar to Directed Random
Search
￿
Find an optimal hypothesis
function
￿
Pick a random point on the graph
and go down in the direction that
gives most downward descent.
￿
Repeat until local minimum
reached
￿
Update parameters
￿
Continue until the hypothesis
function converges
￿
This function has the least overall
error
Unsupervised Learning
￿
Tries to find structure in unlabeled data
￿
Clustering
￿
Cocktail party Problem
￿
An unsupervised learning algorithm can separate the two sources of
sound
￿
In Matlab, could be done in one line
[W,s,v]=svd((repmat(sum(x.*x,1),size(x,1),1).*x)*x’)
￿
Checkers Program
￿
Program plays checkers with itself
￿
Sees which positions leads to a win
How to Make $$$ with
Machine Learning
Sentiment Analysis
Sentiment Analysis
￿
Usually used in texts
￿
(type of text classification)
￿
Popular Machine Learning application
￿
May be used in conjunction with speech
recognition as well
Relevance
￿
Social Media
￿
Twitter, facebook, youtube, yelp, etc.
￿
Track ad campaigns
Pattern Analysis
￿
Machine doesn’t “understand” text
￿
Must read random strings
Favorable Reviews
￿
The word “sweet” – favorable 46,
unfavorable 22
￿
Repeat for other words- “pleasant” (15-6)
￿
Add values up – Naïve Bayes Classifier
Location of words
￿
Algorithm not perfect – “bag of words”
￿
Modifiers – “not”
Other difficulties
￿
This ___ makes ____ look like a great
_____
￿
Some sentences are difficult even for
humans
￿
“good ___ but ____”
Process
￿
Training data
￿
Use training data to create model
￿
Test model on new data
￿
See accuracy of model
￿
Refine model
Improvements?
￿
Sets of words
￿
“this will blow your mind”
￿
Group identical words together to improve
“goodness” value (enjoy + enjoyed)
￿
Eliminate useless words such as “the”
￿
Attach words to other words (adjectives to
nouns, etc)
Where is all this useful?
￿
Business
￿
Economics, Engineering, Marketing, Computer
Science
￿
Physical Sciences
￿
Physics, Astronomy, Chemistry
￿
Health & Medicine
￿
Genetics, Clinical Trials, Epidemiology,
Pharmacology
￿
Government
￿
Census, Law, National Defense
￿
Environment
￿
Agriculture, Ecology, Forestry, Animal Populations
Some fun resourcesB
￿
Sheather, Simon – “Modern Approach To Regression W/
R”
￿
Exponential Regression Applet
￿
http://science.kennesaw.edu/~plaval/applets/ERegression.html
￿
Your Graphing Calculator
￿
“R”
￿
Support Vector Machine
￿
ftp://ftp.cs.wisc.edu/math-prog/talks/csnatt.ppt
￿
University of Illinois at Chicago
￿
http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html