CES 510 Intelligent System Design

jabgoldfishΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 5 μήνες)

55 εμφανίσεις

CES 510 Intelligent System Design





B. Ravikumar

Department of Engg Science

116 I Darwin Hall


664 3335

ravi93@gmail.com

Textbook


Chris Manning and Hinrich Shutze,
Foundations of
Statistical Natural Language Processing
, MIT Press,
1999.


Various supplementary readings.



Other Useful Books:


Jurafsky & Martin,
SPEECH and LANGUAGE
PROCESSING: An Introduction to Natural Language
Processing, Computational Linguistics, and Speech
Recognition
.

Overview of Artificial Intelligence




major applications



image processing and vision



robotics



game playing



speech recognition



natural language understanding



etc.

What is Artificial Intelligence

(
John McCarthy
, Basic Questions)



What is artificial intelligence?


It is the science and engineering of making intelligent machines,
especially intelligent computer programs. It is related to the similar task of
using computers to understand human intelligence, but AI does not have
to confine itself to methods that are biologically observable.



Yes, but what is intelligence?



Intelligence is the computational part of the ability to achieve goals in the
world. Varying kinds and degrees of intelligence occur in people, many
animals and some machines.



Isn't there a solid definition of intelligence that doesn't depend on relating
it to human intelligence?


Not yet. The problem is that we cannot yet characterize in general what
kinds of computational procedures we want to call intelligent. We
understand some

of the mechanisms of intelligence and not others.



More in: http://www
-
formal.stanford.edu/jmc/whatisai/node1.html

What is Artificial Intelligence?



Human
-
like (“How to simulate humans intellect and behavior on
by a machine.)


Mathematical problems (puzzles, games, theorems)


Common
-
sense reasoning (
if there is parking
-
space, probably
illegal to park
)


Expert knowledge: lawyers, medicine, diagnosis


Social behavior


Rational
-
like:


achieve goals, have performance measure


What is Artificial Intelligence


Thought processes


“The exciting new effort to make computers think .. Machines
with minds, in the full and literal sense” (Haugeland, 1985)


Behavior


“The study of how to make computers do things at which, at
the moment, people are better.” (Rich, and Knight, 1991)

The automation of activities that we
associate with human thinking, activities
such as decision
-
making, problem solving,
learning… (Bellman)

The Turing Test

(
Can Machine think? A. M. Turing, 1950)


Requires


Natural language


Knowledge representation


Automated reasoning


Machine learning


(vision, robotics) for full test


What is AI?


Turing test (1950)


Requires:


Natural language


Knowledge representation


automated reasoning


machine learning


(vision, robotics.) for full test


Thinking humanly:


Introspection, the general problem solver (Newell and Simon 1961)


Cognitive sciences


Thinking rationally:


Logic


Problems: how to represent and reason in a domain


Acting rationally:


Agents: Perceive and act

History of AI


McCulloch and Pitts (1943)


Neural networks that learn


Minsky (1951)


Built a neural net computer


Darmouth conference (1956):


McCarthy, Minsky, Newell, Simon met,


Logic theorist (LT)
-

proves a theorem in Principia Mathematica
-
Russel.


The name “Artficial Intelligence” was coined.


1952
-
1969


GPS
-

Newell and Simon


Geometry theorem prover
-

Gelernter (1959)


Samuel Checkers that learns (1952)


McCarthy
-

Lisp (1958), Advice Taker, Robinson’s resolution


Microworlds: Integration, block
-
worlds.


1962
-

the perceptron convergence (Rosenblatt)

The Birthplace of

“Artificial Intelligence”, 1956


Darmouth workshop, 1956:

historical meeting of the perceived founders
of AI met: John McCarthy, Marvin Minsky, Alan Newell, and Herbert
Simon.



A Proposal for the Dartmouth Summer Research Project on Artificial
Intelligence. J. McCarthy, M. L. Minsky, N. Rochester, and C.E. Shannon.
August 31, 1955. "We propose that a 2 month, 10 man study of artificial
intelligence be carried out during the summer of 1956 at Dartmouth
College in Hanover, New Hampshire. The study is to proceed on the
basis of the conjecture that every aspect of learning or any other feature
of intelligence can in principle be so precisely described that a machine
can be made to simulate it."
And this marks the debut of the term
"artificial intelligence.“





History, continued


1966
-
1974 a dose of reality


Problems with computation


1969
-
1979 Knowledge
-
based system


Expert systems:


Dendral:Inferring molecular structures


Mycin: diagnosing blood infections


Prospector: recomending exploratory drilling (Duda).


1986
-
present: return to neural networks


Machine learning theory



Genetic algorithms, genetic programming


Statistical approaches and data mining

State of the art


Deep Blue defeated the reigning world chess champion Garry
Kasparov in 1997


Proved a mathematical conjecture (Robbins conjecture) unsolved for
decades


No hands across America (driving autonomously 98% of the time from
Pittsburgh to San Diego)


During the 1991 Gulf War, US forces deployed an AI logistics planning
and scheduling program that involved up to 50,000 vehicles, cargo,
and people


NASA's on
-
board autonomous planning program controlled the
scheduling of operations for a spacecraft


Proverb solves crossword puzzles better than most humans


DARPA grand challenge 2003
-
2005, Robocup

What’s involved in Intelligence?

Intelligent agents


Ability to interact with the real world


to perceive, understand, and act


e.g., speech recognition and understanding and synthesis


e.g., image understanding


e.g., ability to take actions, have an effect



Knowledge Representation, Reasoning and Planning


modeling the external world, given input


solving new problems, planning and making decisions


ability to deal with unexpected problems, uncertainties



Learning and Adaptation


we are continuously learning and adapting


our internal models are always being “updated”


e.g. a baby learning to categorize and recognize animals

Course overview



Intelligent systems are autonomous systems (hardware / software or a
combination) that behaves as if it exhibits some form of intelligence.



Concept goes back to Alan Turing who thought about machine
intelligence and devised Turing test to distinguish a machine from a
human through interaction.



Some major areas:


Symbolic information processing


deductive systems


Game playing


chess, backgammon etc.


natural language understanding


answering queries, translation, text
classification etc.


Machine learning
-

adaptive behavior through stimulus


Neural networks


Statistical modeling


Fuzzy logic, genetic programming etc.




Course overview




In this course we will introduce statistical
techniques for inferring structure from text.
The aim of the course is to introduce existing
techniques in statistical NLP and to stimulate
thought into bettering these.



Applications of NLP



Information Retrieval



Information Extraction



Natural language interface to database


Statistical Machine Translation

Tools


Probability Theory


Information Theory


Algorithms


Data Structures


Probabilistic AI


Grammars and automata

The Steps in NLP

The steps in NLP (Cont.)


Morphology
: Concerns the way words are built up
from smaller meaning bearing units.
(come(s),co(mes))


Syntax
: concerns how words are put together to
form correct sentences and what structural role
each word has.


Semantics
: concerns what words mean and how
these meanings combine in sentences to form
sentence meanings.


Pragmatics
: concerns how sentences are used in
different situations and how use affects the
interpretation of the sentence.


Discourse
: concerns how the immediately
preceding sentences affect the interpretation of
the next sentence.

Parsing (Syntactic Analysis)


Assigning a syntactic and logical form to an input
sentence


uses knowledge about word and word meanings (lexicon)


uses a set of rules defining legal structures (grammar)


(S (NP (NAME
Sam
))


(VP (V
ate
)


(NP (ART
the
)


(N
apple
))))


I made her
duck
.

Word Sense Resolution


Many words have many meanings or senses.


We need to resolve which of the senses of an
ambiguous word is invoked in a particular use of the
word.


I made her duck. (made her a bird for lunch or made
her move her head quickly downwards?)

Reference Resolution


Domain Knowledge (banking transaction)


Discourse Knowledge


World Knowledge


U: I would like to open a
fixed deposit account
.


S: For what amount?


U: Make
it

for
800 dollars
.


S: For what duration?


U: What is the
interest rate

for
3 months
?


S: Six percent.


U: Oh good then make
it

for
that

duration
.


Why NLP is difficult?


Different ways of Parsing a sentence


Word category ambiguity


Word sense ambiguity


Words can mean more than their sum of parts (
The Times of
India
)


Imparting world knowledge is difficult ("the blue pen ate the
ice
-
cream")


Fictitious worlds ("people on mars can fly")


Defining scope ("people like ice
-
cream," does this mean all
people like ice cream?)


Language is changing and evolving


Complex ways of interaction between the kinds of knowledge


exponential complexity at each point in using the knowledge

Inferring Knowledge from text


Words


word frequencies


collocations


word sense


n
-
grams (words appear in certain order)


Grammar


word categories


syntactic structure


Discourse


Sentence meanings


Applications


Information Retrieval


Information Extraction


Natural language interface


Statistical Machine Translation

Simple Applications


Word counters (wc in UNIX)


Spell Checkers, grammar checkers


Predictive Text on mobile handsets

More significant Applications


Intelligent computer systems


NLU interfaces to databases


Computer aided instruction, automatic graders


Information retrieval


Intelligent Web searching


Data mining


Machine translation


Speech recognition


Natural language generation


Question answering




Spoken Dialogue System

Speech

Synthesis

Speech

Recognition

Semantic

Interpretation

Response

Generation

Dialogue

Management

Discourse

Interpretation


U
s
e
r

Parts of the Spoken Dialogue System


Signal Processing:


Convert the audio wave into a sequence of feature vectors.


Speech Recognition:


Decode the sequence of feature vectors into a sequence of
words.


Semantic Interpretation:


Determine the meaning of the words.


Discourse Interpretation:


Understand what the user intends by interpreting utterances
in context.


Dialogue Management:


Determine system goals in response to user utterances based
on user intention.


Speech Synthesis:


Generate synthetic speech as a response.

Levels of Sophistication in a Dialogue
System


Touch
-
tone replacement:


System Prompt:

"For checking information, press or say
one."


Caller Response:

"One."


Directed dialogue:


System Prompt:

"Would you like checking account
information or rate information?"
Caller Response:

"Checking", or "checking account," or
"rates."


Natural language:


System Prompt:

"What transaction would you like to
perform?"


Caller Response:

"Transfer Rs. 500 from checking to
savings."