CHP 12 NLP - MetaLab

blabbingunequaledΤεχνίτη Νοημοσύνη και Ρομποτική

24 Οκτ 2013 (πριν από 3 χρόνια και 7 μήνες)

62 εμφανίσεις

1

CCSB354

ARTIFICIAL INTELLIGENCE

(AI)

CHAPTER 12

NATURAL LANGUAGE PROCESSING

(NLP)

Textbook (Chapter 13, & especially pages 558 & 588)

Instructor: Alicia Tang Y. C.

2

Language is a complicated
phenomenon, involving processes
as varied as the recognition of
sounds or printed letters,
syntactic parsing, high
-
level
semantic inferences,etc.

3


What is Natural Language
Processing (NLP)?



Natural

language

gives

computer

users

the

ability

to

communication

with

the

computer

in

their

native

language
.



This

technology

allows

for

conversational

type

of

interface
.

A

general

NLP

system

is

not

yet

possible,

especially

in

recognising

and

interpreting

written

sentences
.

4

Types


Natural

Language

Understanding
:

Investigate

methods

of

allowing

the

computer

to

comprehend

instructions

given

in

ordinary

English
.


Natural Language
Generation
:

Strives
to have computers produce ordinary
English language so that people can
understand computers easily.



5


NLP Software Tools




INTELLECT



SPOCK



BBN

Parlane



NaturalLink

6


Component of NLP






Parser



Lexicon



Understander



Knowledge

base



Generator

7


Stages

in

Producing

an

Internal

Representation

of

a

Sentence


8

Contextual

Knowledge Interpretation

Expanded representation
:












possess

experiencer

pet:cheetah

love

object

Person: tarzan

agent

Person:jane

kiss

object

instrument

lips

location

location

jungle

??

9

Block Diagram of a Natural Language
Understanding Program of the

Syntactic
-
Semantic Analysis Type

Input

text

String

Output

Parser

Understander

Knowledge

base

Lexicon

Generator

10

Block Diagram of a Computer
Language Translation System

Parser

Lexicon

Rule
-
based

Processor


Rule
-
based

Processor


Generalised

Intermediate

Form

(GIF)

Input

Program

Output Program in

Target Language

11

NLP

Database

Expert Systems:

where do they stand?

12

EASE OF COMMUNICATION WITH DIFFERENT TYPES OF INTEGRATED SYSTEMS

Oh no!

Well..not
bad!

Oh
yeah!!

13

Levels of analysis for natural
language


Prosody


it deals with the rhythm and intonation of language


Phonology


it examines the sounds that are combined to form
language.


Morphology


it concerns with the components that make up words.


E.g. the effect of prefixes (non
-
, un
-
) and suffixes (
-
ing,
-
ly)

14

Levels of analysis for natural language


Syntax


this involves the study of the rule for combining words
to form legal sentences


Semantics


it considers the meaning of words, phrases and
sentences


Pragmatics


this is the study of the ways in which language is used
and its effects on the listener


World Knowledge


this includes knowledge of the ‘physical’ world, the
world of our social interaction


15

Syntax


Specification and
parsing using
Context
-
free
Grammars


the rules listed
below define a
GRAMMAR for
simple transitive
sentences

1. Sentence


noun
-
phrase verb
-
phrase

2. Noun
-
phrase


noun

3. Noun
-
phrase


article noun

4. Verb
-
phrase


verb

5. Verb
-
phrase


verb noun
-
phrase

6. Article


a

7. Article


the

8. Noun


man

9. Noun


dog

10. Verb


likes

11. Verb


bites

16

Syntax


Rules 6. to 11. are
terminals


they define a lexicon
for the language


terms that describe
high level linguistic
are called
nonterminals
(sentence, verb,
noun
-
phrase, etc)

1. Sentence


noun
-
phrase verb
-
phrase

2. Noun
-
phrase


noun

3. Noun
-
phrase


article noun

4. Verb
-
phrase


verb

5. Verb
-
phrase


verb noun
-
phrase

6. Article


a

7. Article


the

8. Noun


man

9. Noun


dog

10. Verb


likes

11. Verb


bites

17

Syntax


A derivation of the sentence “
the man bites the dog

(syntactically correct but semantically wrong, we shall
see it later on)


String



Apply Rule No.


sentence



1

noun
-
phrase verb
-
phrase

3

article noun verb
-
phrase


7

the
noun verb
-
phrase

8

the man
verb
-
phrase


5

the man
verb noun
-
phrase

11

the man bites
noun
-
phrase

3

the man bites
article noun

7

the man bites the
noun

9

the man bites the dog done

Parsing algorithms fall

into two classes:


1. Bottom
-
up

2. Top
-
down

18

Draw a
parse tree

for the
sentence “the man bites the dog”




19

Top
-
down Parsing

Unrecognised I/P


the man bites the dog

the man bites the dog

the man bites the dog

man bites the dog

bites the dog

bites the dog

the dog

the dog

dog

Parse Tree















NP



A N



the

man

bites


the

dog

V

N


NP

S

VP

A

20

Bottom
-
Up Parsing

Unrecognised I/P


the man bites the dog

man bites the dog

man bites the dog

bites the dog

bites the dog

bites the dog

the dog

the dog

dog

dog

Parse Tree




the

A

NP


N


man

bites

V


VP

S

NP

N

dog


A

the

21

Transition Network Parsers


A transition network parser represents
grammars as a set of finite
-
state
machines (i.e. transition networks), like
this:

Example:
sentence (S)

S

S

Noun
-
phrase

Verb
-
phrase

initial

final

22

Transition Network Parsers

Example: Noun
-
phrase

S

S

Article

Noun

initial

final

Example: Article

S

S

a

initial

final

the

Read page 563, text

Noun

23

The Chomsky Hierarchy and
Context
-
Sensitive Grammars


A context free grammar allows rule to have a
single

nonterminal on their left
-
hand side.


Context
-
free grammar is not powerful enough
to represent rules of natural language syntax


the context
-
sensitive languages form a proper
superset of the context
-
free counterpart

24

The Chomsky Hierarchy and
Context
-
Sensitive Grammars


Here,
one or more symbols on the left
-
hand side of a
rule are allowed

that makes it possible to define a
context in which that rule can be applied.


This ensures satisfaction of a global constraints such
as number agreement and other semantic checks.


The semantic error in earlier example could be
detected in context
-
sensitive grammar if a non
-

terminal,
act_of_biting

is added to the grammar,
preventing the sentence “man bites dog” to be valid.

25

The Chomsky Hierarchy


To correctly express grammatical structure of a
language (e.g. English), rules are needed.


We can classify grammars according to the kinds
of rules that appear in it.


Having done that, we can classify the language
into families according to the kinds of rules that
are needed to express its grammars.


One such means of classifying grammars in
this manner is called Chomsky Hierarchy

26

Grammar Hierarchy


Type 0


Type 1


Type 2


Type 3

The Grammars

for the language

27

Grammar Hierarchy

Type 0


Name of Grammar: Transformation Grammar

Form of Rules: anything


anything

Computational Power: General Turing Machine

String Characteristics: Any form


28

Grammar Hierarchy


Type 1

Name of Grammar: Context
-
sensitive

Form of Rules: A B C


A D C

Computational Power: Linear Bound Automata

String Characteristics: a
n

b
n

c
n


a1 a2 a3 b3 b2 b3

Crossing dependencies

E.g.

Ali
1

helps
4
Ahmad
2


to teach
5

Aida
3

programming
6
.

29

Type 2


Name of Grammar: Context
-
free

Form of Rules: A


B C D ….

Computational Power: Push Down Stack Automata

String Characteristics: a
n

b
n



a1 a2 a3 b3 b2 b1

Nested dependencies

E.g.

Ali
1
who studies in UNITEN
2


that offers
2

quality programmes

is
1

graduating

1

1

2

1

2

2

1

1

1

30

Type 3


Name of Grammar: Right Linear

Form of Rules: A


x B (x in terminal category)

Computational Power: Finite State Automata

String Characteristics: a* b*

They have been used for grammars of Morphology

31


NATURAL LANGUAGE PROCESSING



Problems with NLP :


Ambiguity

»
multiple word meanings


the pitcher is angry


the pitcher is empty


Inaccuracy


Incompleteness

32


Imprecision

»
I’ve been waiting for you for a long time.

»
The king ruled the kingdom for a long
time.


Unclear antecedents

»
Ben hits Bill because the sympathized
with mary.


How

people

overcome

natural

language

problems?


Context

Familiarity


Expectations

33

SPEECH RECOGNITION


Advantages:

»
most natural method

»
ease of access

»
speed

»
manual freedom

»
remote access


Context

»
Isolated Word Recognition (IWR)

»
Connected Word Recognition (CWR)

»
Continuous Speech Recognition (CSR)


Analysing Speech

»
Syllable

Phonemes

Allophones

34

Speech Component: A Holistic Look


I/O



Processing



Other data


Freq. spectrum

She likes ice
-
cream

Word Sentence



She like ice
-
cream

Sentence Structure


x. Likes(x,ice
-
cream)

Partial Meaning

likes(siti, ice
-
cream).

Full Sentence Meaning

Speech

recognition

Syntactic

Analysis

Semantic

Analysis

Pragmatics

Match with other

sound frequencies

Grammar of

language

(dictionary)

Meaning of each

word (thesaurus)

Context of

utterance

35

Exercises:

36


S



NP VP



NP



<name> | <det> <noun> |PP


VP



<verb> | <verb> NP NP | <verb>

NP PP | VP NP



PP



<prep> NP



<name>



“Ben"

| “Ann"


<noun>



"morning" |

“ice
-
cream"


<verb>



"gave" | "saw"


<det>



"the"



<prep>



"to"

| "in"


S



SENTENCE

NP



NOUN PHRASE

VP



VERB PHRASE

DET



DETERMINER

PP

-

PREPOSITION


Question
:

Consider

the

grammar

defined

by

the

following

BNF
:


37


Draw

parse

trees

in

this

grammar

for

the

following

sentences
.




a)

Ben

gave

Ann

the

ice
-
cream
.



b)



Ann

gave

the

ice
-
cream

to

Ben
.





Answers:


S



NP VP



NP



<name> | <det> <noun> |PP


VP



<verb> | <verb> NP NP | <verb>

NP PP | VP NP



PP



<prep> NP


<name>



“Ben"

| “Ann"


<noun>



"morning" |

“ice
-
cream"


<verb>



"gave" | "saw"


<det>




"the"



<prep>



"to"

| "in"