Lecture 14: Formal Grammars Kai-Wei Chang CS @ University of Virginia kw@kwchang.net Couse webpage: http://kwchang.net/teaching/nlp16 CS6501: NLP 1
Critical review report (due 10/20) v 1 page maximum v Pick one paper from the suggested list v Summarize the paper (use you own words) v Provide detailed comments v What can be improved v Potential future directions v Other related work v Example (see reviewer 3 at: https://goo.gl/yrbxxo) CS6501 Natural Language Processing 2
How to model language? v So far, all the models we saw formulate sentence as a sequence v Language models v POS-tagging v Morphological analysis CS6501: NLP 3
Next key concepts v In the following few weeks, we will go beyond sequence models v Syntactic parsing model language as a recursive generating process v Often use a tree structure CS6501: NLP 4
What is grammar? v A compact way to define and describe the structure of sentences v Why we need grammar? v Number of C++ programs? C++ standard (2014) 976 pages. ISO/IEC 14882:2014 1358 pages CS6501: NLP 5
Can we define a program that generates all English sentences? From Julia Hockenmaier, Intro to NLP CS6501: NLP 6
Basic sentence structure CS6501: NLP 7
A Markov Model v I eat shshi; I eat meat; you eat banana v Great, it covers many sentences CS6501: NLP 8
Words take different arguments v [Good] I eat sushi v [Bad] I sleep sushi v [Bad] I give sushi v Intransitive verbs (sleep): no object v Transitive verbs (eat): take one direct object v Ditransitive verbs (give): take an additional indirect object. CS6501: NLP 9
A better model CS6501: NLP 10
Language is recursive Adjectives can modify nouns. We can have unlimited modifiers (in theory) CS6501: NLP 11
We know how to model the simple one CS6501: NLP 12
Recursion can be more complex We can have another noun phrase in preposition CS6501: NLP 13
What is sentence structure v Sentence structure is hierarchical v A sentence consists of phrases (or constituents) CS6501: NLP 14
Can have complex constituents CS6501: NLP 15
Can have complex constituents v Syntactically, constituents behave like simple ones CS6501: NLP 16
Constituency v Groups of words that behave as a single unit or phrase v E.g., Noun phrases: the man, a girl with glasses v Prepositional phrases: with classes, on a table v Verb phrase: eat sushi, sleep, sleep soundly v Phrases has a head: v Other parts called dependents v E.g, the man, a girl with glasses CS6501: NLP 17
Properties of constituents v Substitution v He talks [in class] He talks [there] v It can move around in a sentence v He talks [in class] [In class], he talks. v Can be used as an answer: v Where does he talk? [In class] CS6501: NLP 18
Types of dependencies v Phrases has a head: v Other parts called dependents v E.g, the man, a girl with glasses v Dependents can be arguments or adjuncts v Arguments are obligatory v E.g., [John] likes [Mary] v Adjuncts are optional v E.g., John runs [fast] v Adverbs, PPs, Adjectives All arguments have to be present and cannot be occupied multiple times Can be an arbitrary number of adjuncts CS6501: NLP 19
How to represent the structure CS6501: NLP 20
Structure (syntax) corresponds to meaning CS6501: NLP 21
Dependency Trees v Dependency grammar describe the structure of sentences as a graph (tree) v Nodes represent words v Edges represent dependencies CS6501: NLP 22
Phrases structure trees v Can be modeled by Context-free grammars CS6501: NLP 23
Context-free grammars CS6501: NLP 24
Parse tree defined by CFG CS6501: NLP 25
Generate sentences by CFG CS6501: NLP 26
Example: Noun Phrases CS6501: NLP 27
Example: verb phrase CS6501: NLP 28
Sentences CS6501: NLP 29
Structured Prediction beyond sequence tagging Assign values to a set of interdependent output variables Task Input Output Part-of-speech Tagging They operate ships and banks. Pronoun Verb Noun And Noun Dependency Parsing Segmentation They operate ships and banks. This image cannot currently be displayed. Root They operate ships and banks. This image cannot currently be displayed. 30
Next few lectures v Extend HMM to deal with the tree structure v Discriminative methods for tree structure Kai-Wei Chang 31