Chapter-1-Intro - Info Poster

courageouscellistΤεχνίτη Νοημοσύνη και Ρομποτική

29 Οκτ 2013 (πριν από 3 χρόνια και 9 μήνες)

68 εμφανίσεις

Artificial Intelligence

Introduction Chapter

2

Data Driven Methods

What is Artificial Intelligence?


Common AI techniques


Choosing between AI techniques


Pre and post processing

Powered by DeSiaMore

What is AI?


Various definitions:


Building intelligent entities.


Getting computers to do tasks which require human
intelligence.


But what is “intelligence”?


Simple things turn out to be the hardest to automate:


Recognising a face.


Navigating a busy street.


Understanding what someone says.


All tasks require reasoning on knowledge
.

3

Powered by DeSiaMore

4

What is AI?


Machines that perceive, understand and react
to their environment


Goal of Babbage, etc.


Oldest endeavor in computer science


Machines that think


Robots: factory floors, home vacuums


Still quite impractical


Powered by DeSiaMore

Who does AI?


Many disciplines contribute to goal of
creating/modelling intelligent entities:


Computer Science


Psychology (human reasoning)


Philosophy (nature of belief, rationality, etc)


Linguistics (structure and meaning of language)


Human Biology (how brain works)


Subject draws on ideas from each discipline.

5

Powered by DeSiaMore

Typical AI Problems


Intelligent entities (or “agents”) need to be able to do
both “mundane” and “expert” tasks:


Mundane tasks
-

consider going shopping:


Planning

a route, and sequence of shops to visit!


Recognising

(through
vision
) buses, people.


Communicating (through
natural language
).


Navigating

round obstacles on the street, and
manipulating

objects for purchase.


Expert tasks are things like:


medical diagnosis.


equipment repair.


Often “mundane” tasks are the hardest.



6

Powered by DeSiaMore

7

AI vs. humans


AI applications built on Aristotlean logic


Induction, semantic queries, system of logic


Human reasoning involves more than just induction


Computers never as good as humans


In reasoning and making sense of data


In obtaining a holistic view of a system


Computers much better than humans


In processing reams of data


Performing complex calculations


Powered by DeSiaMore

8

Successful AI applications


Targeted tasks more amenable to automated
methods


Build special
-
purpose AI systems


Determine appropriate dosage for a drug


Classify cells as benign or cancerous


Called “expert systems”


Methodology based on expert reasoning


Quick and objective ways to obtain answers



Powered by DeSiaMore

9

Data Driven Methods

What is Artificial Intelligence?


Common AI techniques


Choosing between AI techniques


Pre and post processing

Powered by DeSiaMore

10

AI Technique


Intelligence requires Knowledge


Knowledge posesses less desirable properties such as:


Voluminous


Hard to characterize accurately


Constantly changing


Differs from data that can be used


AI technique is a method that exploits knowledge that should
be represented in such a way that:


Knowledge captures generalization


It can be understood by people who must provide it


It can be easily modified to correct errors.


It can be used in variety of situations


Powered by DeSiaMore

11

The State of the Art


Computer beats human in a chess game.


Computer
-
human conversation using speech
recognition.


Expert system controls a spacecraft.


Robot can walk on stairs and hold a cup of water.


Language translation for webpages.


Home appliances use fuzzy logic.


......


Powered by DeSiaMore

12

Fuzzy logic


Fuzzy logic addresses key problem in expert
systems


How to represent domain knowledge


Humans use imprecisely calibrated terms


How to build decision trees on imprecise
thresholds


Powered by DeSiaMore

13

Advantages of fuzzy logic


Considerable skill for little investment


Fuzzy logic systems piggy bank on human analysis


Humans encode rules after intelligent analysis of lots of
data


Verbal rules generated by humans are robust


Simple to create


Not much need for data or ground truth


Logic tends to be easy to program


Fuzzy rules are human understandable


Powered by DeSiaMore

14

Where not to use fuzzy logic


Do not use fuzzy logic if:


Humans do not understand the system


Different experts disagree


Knowledge can not be expressed with verbal rules


Gut instinct is involved


Not just objective analysis


A fuzzy logic system is limited


Piece
-
wise linear approximation to a system


Non
-
linear systems can not be approximated


Many environment applications are non
-
linear

Powered by DeSiaMore

15

Neural Networks


Neural networks can approximate non
-
linear
systems


Evidence
-
based


Weights chosen through optimization procedure on
known dataset (“training”)


Works even if experts can’t verbalize their
reasoning, or if there is ground truth



Powered by DeSiaMore

16

Advantages of neural networks


Can approximate any smooth function


The three
-
layer neural network


Can yield true probabilities


If output node is a sigmoid node


Not hard to train


Training process is well understood


Fast in operations


Training is slow, but once trained, the network can
calculate the output for a set of inputs quite fast


Easy to implement


Just a sum of exponential functions

Powered by DeSiaMore

17

Disadvantages of neural networks


A black box


The final set of weights yields no insights


Magnitude of weights doesn’t mean much


Measure of skill needs to be differentiable


RMS error, etc.


Can not use Probability of Detection, for example


Training set has to be complete


Unpredictable output on data unlike training


Need lots of data


Need expert willing to do lot of truthing

Powered by DeSiaMore

18

Recap:


Fuzzy logic


Humans provide the rules


Not optimal


Neural network


Humans can not understand system


Optimal


Middle ground?


Genetic Algorithms


Decision Trees

Powered by DeSiaMore

19

Genetic algorithms


In genetic algorithms


One fixes the model (rule base, equations, class of
functions, etc.)


Optimize the parameters to model on training
data set


Use optimal set of parameters for unknown cases

Powered by DeSiaMore

20

An example genetic algorithm

Sources:

http://tx.technion.ac.il/~edassau/web/genetic_algorithms.htm

http://cswww.essex.ac.uk/research/NEC/


Powered by DeSiaMore

21

Advantages of genetic algorithms


Near
-
optimal parameters for given model


Human
-
understandable rules


Best parameters for them


Cost function need not be differentiable


The process of training uses natural selection, not
gradient descent


Requires less data than a neural network


Search space is more limited

Powered by DeSiaMore

22

Disadvantages of genetic algorithms


Highly dependent on class of functions


If poor model is chosen, poor results


Optimization may not help at all


Known model does not always lead to better
understanding


Magnitude of weights, etc. may not be meaningful
if inputs are correlated


Problem may have multiple parametric solutions


Powered by DeSiaMore

23

Data Driven Methods

What is Artificial Intelligence?


Common AI techniques


Choosing between AI techniques


Pre and post processing

Powered by DeSiaMore

24

Typical data
-
driven application

Input Data

Features

f(features)

Result

How do we find f()

Which features?

AI application

in run
-
time

Powered by DeSiaMore

25

What is the role of the data?


Validation


Test known model


Technique:


Difference between model output and ground truth helps to validate the model


Calibration


Find parameters to model with desired structure


Technique:


Tuned fuzzy logic method


Genetic algorithms


Induction


Find model and parameters from just data


Technique:


Neural network methods, bagged/boosted decision trees, support vector machines,
etc.

Powered by DeSiaMore

26

What is the problem to solve?


Do you have a bunch of data and want to:


Estimate an unknown parameter from it?


True rainfall based on radar observations?


Amount of liquid content from in
-
situ measurements of temperature,
pressure, etc?


Regression


Classify what the data correspond to?


A water surge?


A temperature inversion?


A boundary?


Classification


Regression and classification aren’t that different


Classification: estimate probability of an event


A function from 0
-
1

Powered by DeSiaMore

27

Which AI technique?


Do you have expert knowledge?


Humans have a “model” in their head? Should the final f() be understandable?


Create
fuzzy logic

rules from experts’ reasoning


Aggregate the individual fuzzy logic rules


Can tune the fuzzy rules based on data


Using regression, decision trees or neural networks for RMS error criterion


Genetic algorithms for error criteria like ROC, economic cost, etc.


Many times the original rules are just fine


Do you already know the model?


A power
-
law relationship? Gaussian? Quadratic? Rules?


Just need to find parameters to this model?


If linear, just use linear regression


If non
-
linear: use
genetic algorithms


Use continuous GAs


Both of these can be used for regression (therefore, also classification)



Powered by DeSiaMore

28

Which AI technique (contd.)


Do you know nothing about the data?


Not the suspected equation/model (GA)?


Not the suspected rules (fuzzy logic)?


Use a AI technique that supplies its equations/rules


“black box”.


For classification, use:


Bagged decision trees or Support Vector Machines


If output is probabilistic, remember to apply Platt scaling


Summary statistics on bagged DTs can help answer “why”


Neural Networks


For regression, use:


Neural networks

Powered by DeSiaMore

29

Where do your data come from?


Observed data


Compute features


Choose AI technique


The 4 choices in the previous two slides


Simulated data:


Example: trying to replicate a very complex model


Throw randomly
-
generated data at model


Compute features


Choose AI technique:


GA for parametric approximations


NN when you don’t know how to approximate


Powered by DeSiaMore

30

Where do you get your inputs?


What type of data do you have?


Individual observations?


Sample them (choose at random) and use directly


Sparse observations in a time series?


Generate time
-
based features (1D moving windows)


Signal processing features from time series


Data from remotely sensed 2D grids?


Generate image
-
based features using convolution filters


Do you need:


Pixel
-
based regression/classification?

»
Use convolution features directly


Object
-
based regression/classification?

»
Identify regions using region growing

»
Use region
-
aggregate features

Powered by DeSiaMore

31

Typical data
-
driven application

Observed data

Features

f()

Result

Signal/image processing;sampling

normalize/create chromosome/

determine confidences

FzLogic/GenAlg/NN/DecTree

Platt method/region
-
average/threshold

A data
-
driven application

in run
-
time

Powered by DeSiaMore

32

Data Driven Methods

What is Artificial Intelligence?


Common AI techniques


Choosing between AI techniques


Pre and post processing

Powered by DeSiaMore

33

Preprocessing


Often can not use pixel data directly


Too much data, too highly correlated


May need to segment pixels into objects and use features computed
on the objects


Different data sets may not be collocated


Need to interpolate to line them up


Mapping, objective analysis


Noise in data may need to be reduced


Smoothing


Present statistic of data, rather than data itself


Features need to be extracted from data


Human experts often good source of ideas on signatures to extract
from data

Powered by DeSiaMore

34

Postprocessing


The output of an expert system may be grid point by
grid point


May need to provide output on objects


Storms, forests, etc.


Can average outputs over objects’ pixels


May need probabilistic output


Scale output of maximum marginal techniques


Use a sigmoid function


Called Platt scaling

Powered by DeSiaMore

35

Summary


What is Artificial Intelligence?


Data
-
driven methods to perform specific targeted tasks


Common AI techniques


Fuzzy logic, neural networks, genetic algorithms, decision trees


Choosing between AI techniques


Understand the role of your data


Do experts understand the system? (have a model)


Do experts expect to understand the system? (readability)


Pre and post processing


Image processing techniques on spatial grids

Powered by DeSiaMore

AI Introduction

END

36

Powered by DeSiaMore