Lecture 14: Formal Grammars

Transcription

1 Lecture 14: Formal Grammars Kai-Wei Chang University of Virginia kw@kwchang.net Couse webpage: CS6501: NLP 1

2 Critical review report (due 10/20) v 1 page maximum v Pick one paper from the suggested list v Summarize the paper (use you own words) v Provide detailed comments v What can be improved v Potential future directions v Other related work v Example (see reviewer 3 at: CS6501 Natural Language Processing 2

3 How to model language? v So far, all the models we saw formulate sentence as a sequence v Language models v POS-tagging v Morphological analysis CS6501: NLP 3

4 Next key concepts v In the following few weeks, we will go beyond sequence models v Syntactic parsing model language as a recursive generating process v Often use a tree structure CS6501: NLP 4

5 What is grammar? v A compact way to define and describe the structure of sentences v Why we need grammar? v Number of C++ programs? C++ standard (2014) 976 pages. ISO/IEC 14882: pages CS6501: NLP 5

6 Can we define a program that generates all English sentences? From Julia Hockenmaier, Intro to NLP CS6501: NLP 6

7 Basic sentence structure CS6501: NLP 7

8 A Markov Model v I eat shshi; I eat meat; you eat banana v Great, it covers many sentences CS6501: NLP 8

9 Words take different arguments v [Good] I eat sushi v [Bad] I sleep sushi v [Bad] I give sushi v Intransitive verbs (sleep): no object v Transitive verbs (eat): take one direct object v Ditransitive verbs (give): take an additional indirect object. CS6501: NLP 9

10 A better model CS6501: NLP 10

11 Language is recursive Adjectives can modify nouns. We can have unlimited modifiers (in theory) CS6501: NLP 11

12 We know how to model the simple one CS6501: NLP 12

13 Recursion can be more complex We can have another noun phrase in preposition CS6501: NLP 13

14 What is sentence structure v Sentence structure is hierarchical v A sentence consists of phrases (or constituents) CS6501: NLP 14

15 Can have complex constituents CS6501: NLP 15

16 Can have complex constituents v Syntactically, constituents behave like simple ones CS6501: NLP 16

17 Constituency v Groups of words that behave as a single unit or phrase v E.g., Noun phrases: the man, a girl with glasses v Prepositional phrases: with classes, on a table v Verb phrase: eat sushi, sleep, sleep soundly v Phrases has a head: v Other parts called dependents v E.g, the man, a girl with glasses CS6501: NLP 17

18 Properties of constituents v Substitution v He talks [in class] He talks [there] v It can move around in a sentence v He talks [in class] [In class], he talks. v Can be used as an answer: v Where does he talk? [In class] CS6501: NLP 18

19 Types of dependencies v Phrases has a head: v Other parts called dependents v E.g, the man, a girl with glasses v Dependents can be arguments or adjuncts v Arguments are obligatory v E.g., [John] likes [Mary] v Adjuncts are optional v E.g., John runs [fast] v Adverbs, PPs, Adjectives All arguments have to be present and cannot be occupied multiple times Can be an arbitrary number of adjuncts CS6501: NLP 19

20 How to represent the structure CS6501: NLP 20

21 Structure (syntax) corresponds to meaning CS6501: NLP 21

22 Dependency Trees v Dependency grammar describe the structure of sentences as a graph (tree) v Nodes represent words v Edges represent dependencies CS6501: NLP 22

23 Phrases structure trees v Can be modeled by Context-free grammars CS6501: NLP 23

24 Context-free grammars CS6501: NLP 24

25 Parse tree defined by CFG CS6501: NLP 25

26 Generate sentences by CFG CS6501: NLP 26

27 Example: Noun Phrases CS6501: NLP 27

28 Example: verb phrase CS6501: NLP 28

29 Sentences CS6501: NLP 29

30 Structured Prediction beyond sequence tagging Assign values to a set of interdependent output variables Task Input Output Part-of-speech Tagging They operate ships and banks. Pronoun Verb Noun And Noun Dependency Parsing Segmentation They operate ships and banks. This image cannot currently be displayed. Root They operate ships and banks. This image cannot currently be displayed. 30

31 Next few lectures v Extend HMM to deal with the tree structure v Discriminative methods for tree structure Kai-Wei Chang 31