Natural Language Processing

unclesamnorweiganAI and Robotics

Oct 18, 2013 (3 years and 9 months ago)

62 views




Natural Language Processing

Heshaam Feili

http://mehr.sharif.edu/~hfaili

July 2003

Natural language Processing
(Heshaam Feili


July 2003)

2

Session Agenda



Artificial Intelligence



Natural Language Processing



History of NLP



Applications of NLP


Natural language Processing
(Heshaam Feili


July 2003)

3

AI Concepts and Definitions



Encompasses Many Definitions


AI Involves Studying Human

Thought Processes


Representing Thought Processes on
Machines

Natural language Processing
(Heshaam Feili


July 2003)

4

Artificial Intelligence


Behavior

by a machine that, if
performed by a human being, would be
considered
intelligent


“…study of how to make computers do
things at which, at the moment, people
are better” (Rich and Knight [1991])


Theory of how the
human mind

works
(Mark Fox)

Natural language Processing
(Heshaam Feili


July 2003)

5

AI Objectives


Make machines
smarter

(primary
goal)


Understand what
intelligence

is
(Nobel Laureate purpose)


Make machines more
useful

(practical purpose)


(Winston and Prendergast [1984])

Natural language Processing
(Heshaam Feili


July 2003)

6

Signs of Intelligence



Learn

or
understand

from
experience


Make sense out of ambiguous or
contradictory messages


Respond quickly and successfully
to new situations


Use
reasoning

to solve problems

Natural language Processing
(Heshaam Feili


July 2003)

7

More Signs of Intelligence


Deal with perplexing situations


Understand

and
infer

in ordinary,
rational ways


Apply
knowledge

to manipulate
the environment


Think

and
reason


Recognize the relative importance
of different elements in a situation

Natural language Processing
(Heshaam Feili


July 2003)

8

Turing Test for Intelligence

A computer can be considered to be
smart

only when a human
interviewer, “conversing” with
both an unseen human being and
an unseen computer, can not
determine which is which

Natural language Processing
(Heshaam Feili


July 2003)

9

Symbolic Processing



Use
Symbols

to Represent Problem
Concepts




Apply Various Strategies and Rules
to Manipulate these Concepts

Natural language Processing
(Heshaam Feili


July 2003)

10

AI Represents Knowledge as
Sets of Symbols

A
symbol

is a string of characters that
stands for some real
-
world concept


Examples


Product


Defendant


0.8


Chocolate

Natural language Processing
(Heshaam Feili


July 2003)

11

Symbol Structures
(Relationships)


(DEFECTIVE product)


(EQUAL (LIABILITY defendant) 0.8)


tastes_good (chocolate).

Natural language Processing
(Heshaam Feili


July 2003)

12


AI Programs Manipulate Symbols to Solve
Problems



Symbols and Symbol Structures Form
Knowledge Representation



Artificial Intelligence Dealings Primarily with
Symbolic
, N
onalgorithmic

Problem
-

Solving
Methods

Natural language Processing
(Heshaam Feili


July 2003)

13

AI Computing



Based on
symbolic representation

and
manipulation


A
symbol

is a letter, word, or number
representing objects, processes, and their
relationships


Objects

can be people, things, ideas, concepts,
events, or statements of fact


Creates a
symbolic knowledge base


Natural language Processing
(Heshaam Feili


July 2003)

14

AI Computing
(cont’d
)


Manipulates symbols to generate advice


AI reasons or infers with the knowledge base
by search and pattern matching


Hunts for answers (via algorithms)

Natural language Processing
(Heshaam Feili


July 2003)

15

Major AI Areas



Expert Systems


Natural Language Processing


Speech Understanding


Robotics and Sensory Systems


Computer Vision and Scene Recognition


Intelligent Computer
-
Aided Instruction


Neural Computing

Natural language Processing
(Heshaam Feili


July 2003)

16

Additional AI Areas


News Summarization


Language Translation


Fuzzy Logic


Genetic Algorithms


Intelligent Software Agents

Natural language Processing
(Heshaam Feili


July 2003)

NLP ?


Natural Language is one of fundamental
aspects of human behaviors.


One of the final aim of human
-
computer communication.


Provide easy interaction with computer


Make computer to understand texts.


Natural language Processing
(Heshaam Feili


July 2003)

Major Disciplines Studying
Language

Discipline

Typical Problem

Linguists

How do words from phrases and
sentences?

Psycholinguists

How do people identify the
structure of sentences?

Philosophers

What is meaning and how do
words and sentences acquires
it?

Computational
Linguists

How is the structure of
sentences identified?

Natural language Processing
(Heshaam Feili


July
2003
)

Interaction Level


The level that computer and human
interact.


NL used for make Interaction level near
to human.


Human

Computer

Command
-
line

NL UI

Graphical UI

Interaction level

Natural language Processing
(Heshaam Feili


July
2003
)

20

Natural Language Processing
(NLP)


Natural language processing concerns the development of
computational models of aspects of human language
processing such as :



Reading and interpreting a textbook



Writing a letter



Holding a conversation



Translating a document



Searching for useful information

Such models are useful in order to write computer programs to
perform useful tasks involving language processing and in
order to develop a better understanding of human
communication.

Natural language Processing
(Heshaam Feili


July
2003
)

21

Other Titles


The most common titles, apart from
Natural Language Processing

include:



Automatic Language Processing



Computational Linguistics



Natural Language Understanding



Natural language Processing
(Heshaam Feili


July
2003
)

22

Computational Lingusitics


This is the application of computers to the
scientific study of human language.


This definition suggests that there are
connections with
Cognitive Science
, that is
to say, the study of how humans produce
and understand language.

Natural language Processing
(Heshaam Feili


July
2003
)

Computational Lingusitics


Historically, Computational
Linguistics has been associated with
work in
Generative Linguistics

and
formerly included the study of formal
languages (eg finite state automata)
and programming languages.



Natural language Processing
(Heshaam Feili


July
2003
)

24

Natural Language Understanding


Distinguish a particular approach to
Natural Language Processing.


The people using this title tend to lay
much emphasis on the
meaning

of the
language being processed, in
particular getting the computer to
respond to the input in an apparently
intelligent fashion.

Natural language Processing
(Heshaam Feili


July
2003
)

Natural Language
Understanding


At one time, those who belonged to
the Natural Language Understanding
camp avoided the use of any
syntactic

processing
, but textbooks that bear
this title now include significant
sections on syntactic processing,
which suggests that the edge of the
title has been rather blunted. (For
instance, see
Allen (
1987
; part
1
).


Natural language Processing
(Heshaam Feili


July
2003
)

26

NLP History (
1
)


The first recognizable NLP
application was a
dictionary look
-
up system

developed at Birkbeck
College, London in
1948
.


NLP from
1966
-
1980


Augmented Transition Networks


Case Grammar


Natural language Processing
(Heshaam Feili


July 2003)

27

NLP History (
2
)


NLP from
1966
-
1980


Semantic representations



Schank and his workers introduced the notion of
Conceptual

Dependency
, a method of expressing
language in terms of semantic primitives. Systems were
written which included no syntactic processing.


QuillianÕs work on memory introduced the idea of the
semantic

network
, which has been used in varying
forms for knowledge representation in many systems.


William Woods used the idea of
procedural semantics

to
act as an intermediate representation between a
language processing system and a database system.

Natural language Processing
(Heshaam Feili


July
2003
)

28

NLP History (
3
)


The key systems were:



LUNAR
: A database interface system that used ATNs and
Woods' Procedural Semantics.


LIFER/LADDER
: One of the most impressive of NLP
systems. It was designed as a natural language interface to
a database of information about US Navy ships.


NLP from
1980
-

1990




-

Grammar Formalisms


NLP from
1990
-

now



-

Multilinguality and Multimodality

Natural language Processing
(Heshaam Feili


July 2003)

29

NLP Applications


Applications can be classified in different
ways, e.g. medium/modality; depth of
analysis; degree of interaction



Text
-
based applications



NL Understanding



Dialogue Systems



Multimodal

Natural language Processing
(Heshaam Feili


July
2003
)

30

Text
-
based Applications

Processing of written texts such as books, news, papers,
reports:



Finding appropriate documents on certain topics from a
text database



Extracting information from messages, articles, Web
pages, etc.

Natural language Processing
(Heshaam Feili


July 2003)

Text
-
based Applications



Translating documents from one language
to another



Text summarization


Note: Not all such applications require NLP


Keyword based techniques can used for
identifying particular subject areas, e.g.
legal, financial, etc.


Natural language Processing
(Heshaam Feili


July
2003
)

32

NL Understanding


Other kinds of request require a deeper level of analysis


Find me all articles concerning car accidents involving more
than two cars in Malta during the first half of

2001


Here the system must extract enough information to
determine whether the article meets the criterion defined
by the query.

Natural language Processing
(Heshaam Feili


July
2003
)

NL
-

Understanding


A crucial characteristic of an
understanding system is that it can
compute some representation of the
information that can be used for later
inference


A crucial question for an NLP system is
how much understanding is necessary
to achieve the purpose of the system.


Natural language Processing
(Heshaam Feili


July
2003
)

34

Dialogue
-
based Applications

Dialogue
-
based applications involve man
-
machine
communication



NL database query systems



Automated customer services, e.g. banking
services

Natural language Processing
(Heshaam Feili


July
2003
)

35

Multimodal Applications

Involve two or more modalities of communication



Text



Speech



Gesture



Image




Text


speech




Speech


text


Multimodal document generation


Spoken translation systems


Spoken dialogue systems


Natural language Processing
(Heshaam Feili


July
2003
)