Natural Language Processing

Transcription

1 Natural Language Processing Syntax Joakim Nivre Uppsala University Department of Linguistics and Philology Natural Language Processing 1(7)

2 What is Syntax? Words do not occur in isolation they combine into sentences Snoopy hugs Woodstock Woodstock hugs Snoopy Snoopy hugs birds *Birds hugs Snoopy Birds hug Snoopy Snoopy hugs them *Them hug Snoopy They hug Snoopy Syntax the study of sentence structure Natural Language Processing 2(7)

3 Two Views of Syntax Dependency: Syntactic structure resides in relations between words Focus on the functional roles (subject, object,...) Long tradition in descriptive grammar going back to Antiquity Constituency: Syntactic structure consists in the composition of phrases Focus on structural categories (noun phrase, verb phrase,...) Introduced in structural linguistics in the 20th century Natural Language Processing 3(7)

4 Dependency Syntactic structure represented by dependency trees Words represented by tree nodes Dependencies represented by directed arcs between nodes Functional roles specified by arc labels punct obj amod subj amod pmod pobj amod Economic news had little effect on financial markets. ADJ NOUN VERB ADJ NOUN ADP ADJ NOUN PUNCT Natural Language Processing 4(7)

5 Constituency Syntactic structure represented by phrase structure trees Words represented by terminal tree nodes (leaves) Phrases represented by internal tree nodes Phrase types specified by node labels Phrase structure trees can be defined by context-free grammars S VP NP NP PP NP NP Punct Adj Noun Verb Adj Noun Prep Adj Noun Economic news had little effect on financial markets. Natural Language Processing 5(7)

6 Two Complementary Views? Dependency trees explicitly represent dependency relations (directed arcs) functional categories (arc labels) possibly some structural categories (parts-of-speech) Phrase structure trees explicitly represent phrases (internal nodes) structural categories (node labels) possibly some functional categories (grammatical functions) Both are widely used in NLP and hybrid representations exist Natural Language Processing 6(7)

7 Quiz True or false? 1. Syntax studies the internal structure or words 2. Syntax studies the order of words in sentences 3. Dependency trees show how words are related in sentences 4. Dependency trees show how sentences are composed of phrases Natural Language Processing 7(7)

8 Natural Language Processing Dependency Joakim Nivre Uppsala University Department of Linguistics and Philology Natural Language Processing 1(9)

9 Dependency Syntactic structure consists of words, related by binary, asymmetric relations called dependencies Economic news had little effect on financial markets. ADJ NOUN VERB ADJ NOUN ADP ADJ NOUN PUNCT Natural Language Processing 2(9)

10 Dependency Syntactic structure consists of words, related by binary, asymmetric relations called dependencies Economic news had little effect on financial markets. ADJ NOUN VERB ADJ NOUN ADP ADJ NOUN PUNCT Natural Language Processing 2(9)

11 Dependency Syntactic structure consists of words, related by binary, asymmetric relations called dependencies Economic news had little effect on financial markets. ADJ NOUN VERB ADJ NOUN ADP ADJ NOUN PUNCT Natural Language Processing 2(9)

12 Dependency Syntactic structure consists of words, related by binary, asymmetric relations called dependencies Economic news had little effect on financial markets. ADJ NOUN VERB ADJ NOUN ADP ADJ NOUN PUNCT Natural Language Processing 2(9)

13 Dependency Syntactic structure consists of words, related by binary, asymmetric relations called dependencies Economic news had little effect on financial markets. ADJ NOUN VERB ADJ NOUN ADP ADJ NOUN PUNCT Natural Language Processing 2(9)

14 Dependency Syntactic structure consists of words, related by binary, asymmetric relations called dependencies Economic news had little effect on financial markets. ADJ NOUN VERB ADJ NOUN ADP ADJ NOUN PUNCT Natural Language Processing 2(9)

15 Dependency Syntactic structure consists of words, related by binary, asymmetric relations called dependencies p obj pobj amod subj amod pmod amod Economic news had little effect on financial markets. ADJ NOUN VERB ADJ NOUN ADP ADJ NOUN PUNCT Natural Language Processing 2(9)

16 Terminology Higher Head Governor Regent. Lower Dependent Modifier Subordinate. Natural Language Processing 3(9)

17 Terminology Higher Head Governor Regent. Lower Dependent Modifier Subordinate. Natural Language Processing 3(9)

18 Criteria for Heads and Dependents Criteria for a syntactic relation between a head H and a dependent D in a construction C: 1. H determines the syntactic category of C; H can replace C. 2. H determines the semantic category of C; D specifies H. 3. H is obligatory; D may be optional. 4. H selects D and determines whether D is obligatory. 5. The form of D depends on H (agreement or government). 6. The linear position of D is specified with reference to H. Issues: Syntactic (and morphological) versus semantic criteria Exocentric versus endocentric constructions Natural Language Processing 4(9)

19 Some Clear Cases Construction Head Dependent Exocentric Verb Subject (subj) Verb Object (obj) Endocentric Verb Adverbial (advmod) Noun Attribute (amod) Economic news suddenly affected financial markets. ADJ NOUN ADV VERB ADJ NOUN PUNCT Natural Language Processing 5(9)

20 Some Clear Cases Construction Head Dependent Exocentric Verb Subject (subj) Verb Object (obj) Endocentric Verb Adverbial (advmod) Noun Attribute (amod) subj Economic news suddenly affected financial markets. ADJ NOUN ADV VERB ADJ NOUN PUNCT Natural Language Processing 5(9)

21 Some Clear Cases Construction Head Dependent Exocentric Verb Subject (subj) Verb Object (obj) Endocentric Verb Adverbial (advmod) Noun Attribute (amod) subj obj Economic news suddenly affected financial markets. ADJ NOUN ADV VERB ADJ NOUN PUNCT Natural Language Processing 5(9)

22 Some Clear Cases Construction Head Dependent Exocentric Verb Subject (subj) Verb Object (obj) Endocentric Verb Adverbial (advmod) Noun Attribute (amod) subj advmod obj Economic news suddenly affected financial markets. ADJ NOUN ADV VERB ADJ NOUN PUNCT Natural Language Processing 5(9)

23 Some Clear Cases Construction Head Dependent Exocentric Verb Subject (subj) Verb Object (obj) Endocentric Verb Adverbial (advmod) Noun Attribute (amod) amod subj advmod obj amod Economic news suddenly affected financial markets. ADJ NOUN ADV VERB ADJ NOUN PUNCT Natural Language Processing 5(9)

24 Some Tricky Cases Complex verb groups (auxiliary main verb) Subordinate clauses (complementizer verb) Coordination (coordinator conjuncts) Prepositional phrases (preposition nominal) Punctuation I can see that they rely on this and that. PRON AUX VERB SCONJ PRON VERB ADP PRON CONJ PRON PUNCT Natural Language Processing 6(9)

25 Some Tricky Cases Complex verb groups (auxiliary main verb) Subordinate clauses (complementizer verb) Coordination (coordinator conjuncts) Prepositional phrases (preposition nominal) Punctuation subj ccomp I can see that they rely on this and that. PRON AUX VERB SCONJ PRON VERB ADP PRON CONJ PRON PUNCT Natural Language Processing 6(9)

26 Some Tricky Cases Complex verb groups (auxiliary main verb) Subordinate clauses (complementizer verb) Coordination (coordinator conjuncts) Prepositional phrases (preposition nominal) Punctuation subj ccomp I can see that they rely on this and that. PRON AUX VERB SCONJ PRON VERB ADP PRON CONJ PRON PUNCT Natural Language Processing 6(9)

27 Some Tricky Cases Complex verb groups (auxiliary main verb) Subordinate clauses (complementizer verb) Coordination (coordinator conjuncts) Prepositional phrases (preposition nominal) Punctuation I can see that they rely on this and that. PRON AUX VERB SCONJ PRON VERB ADP PRON CONJ PRON PUNCT Natural Language Processing 6(9)

28 Some Tricky Cases Complex verb groups (auxiliary main verb) Subordinate clauses (complementizer verb) Coordination (coordinator conjuncts) Prepositional phrases (preposition nominal) Punctuation I can see that they rely on this and that. PRON AUX VERB SCONJ PRON VERB ADP PRON CONJ PRON PUNCT Natural Language Processing 6(9)

29 Some Tricky Cases Complex verb groups (auxiliary main verb) Subordinate clauses (complementizer verb) Coordination (coordinator conjuncts) Prepositional phrases (preposition nominal) Punctuation I can see that they rely on this and that. PRON AUX VERB SCONJ PRON VERB ADP PRON CONJ PRON PUNCT Natural Language Processing 6(9)

30 Treebanks Treebanks Syntactically annotated corpora are called treebanks Treebanks can be used to train and evaluate syntactic parsers Dependency treebanks Treebanks with dependency-based annotation Example: Prague Dependency Treebank of Czech Annotation schemes can vary considerably across languages Natural Language Processing 7(9)

31 Universal Dependencies (UD) Standardized framework for dependency annotation Consistent analysis across typologically different languages obl obj case det nsubj det det DET NOUN VERB DET NOUN ADP DET NOUN the dog chased the cat from the room koira jahtasi kissan huoneesta NOUN VERB NOUN NOUN Case=Nom Case=Acc Case=Ela nsubj obj obl Natural Language Processing 8(9)

32 Quiz The big bear scared the little dog. True or false 1. The word dog is a dependent of the word bear 2. The word bear is a dependent of the word scared 3. The word scared is a dependent of the word little 4. The word little is a dependent of the word dog Natural Language Processing 9(9)

33 Natural Language Processing Constituency Joakim Nivre Uppsala University Department of Linguistics and Philology Natural Language Processing 1(11)

34 Constituency Word groups can act as single units Los Angeles a high-class spot such as Mindy s three parties from Brooklyn they Such groups of words are called constituents Constituents have similar internal structure and behavior Natural Language Processing 2(11)

35 Immediate Constituency Analysis We can find constituents by recursive decomposition: The girl in the corner wears a yellow hat and dark sunglasses. Natural Language Processing 3(11)

36 Immediate Constituency Analysis We can find constituents by recursive decomposition: The girl in the corner wears a yellow hat and dark sunglasses. The girl in the corner + wears a yellow hat and dark sunglasses. Natural Language Processing 3(11)

37 Immediate Constituency Analysis We can find constituents by recursive decomposition: The girl in the corner wears a yellow hat and dark sunglasses. The girl in the corner + wears a yellow hat and dark sunglasses. [The girl + in the corner] [wears + a yellow hat and dark sunglasses]. Natural Language Processing 3(11)

38 Immediate Constituency Analysis We can find constituents by recursive decomposition: The girl in the corner wears a yellow hat and dark sunglasses. The girl in the corner + wears a yellow hat and dark sunglasses. [The girl + in the corner] [wears + a yellow hat and dark sunglasses]. [The + girl] [in + the corner] wears [a yellow hat + and + dark sunglasses]. Natural Language Processing 3(11)

39 Immediate Constituency Analysis We can find constituents by recursive decomposition: The girl in the corner wears a yellow hat and dark sunglasses. The girl in the corner + wears a yellow hat and dark sunglasses. [The girl + in the corner] [wears + a yellow hat and dark sunglasses]. [The + girl] [in + the corner] wears [a yellow hat + and + dark sunglasses]. The girl in [the + corner] wears [a + yellow hat] and [dark + sunglasses]. Natural Language Processing 3(11)

40 Immediate Constituency Analysis We can find constituents by recursive decomposition: The girl in the corner wears a yellow hat and dark sunglasses. The girl in the corner + wears a yellow hat and dark sunglasses. [The girl + in the corner] [wears + a yellow hat and dark sunglasses]. [The + girl] [in + the corner] wears [a yellow hat + and + dark sunglasses]. The girl in [the + corner] wears [a + yellow hat] and [dark + sunglasses]. The girl in the corner wears a [yellow + hat] and dark sunglasses. Natural Language Processing 3(11)

41 Test for Constituents Substitution: similar constituents can replace each other The girl in the corner wears a yellow hat and dark sunglasses The girl wears a yellow hat and dark sunglasses She wears a yellow hat and dark sunglasses *In the corner wears a yellow hat and dark sunglasses Movement: words in a constituent move together She went to Paris on Monday. On Monday she went to Paris. *Paris on Monday she went to. *On she went to Paris Monday. Natural Language Processing 4(11)

42 Constituent Types Noun phrase (NP) she the house Robin Hood and his merry men Verb phrase (VP) blushed loves Mary was told to sit down and be quiet Prepositional phrase (PP) on it with the telescope through the foggy dew Natural Language Processing 5(11)

43 Context-Free Grammar Acontext-freegrammar(CFG)consistsof a finite set of nonterminal symbols a finite set of terminal symbols a distinguished nonterminal symbol S (for Start) a finite set of rules of the form A α where A is a nonterminal and α is a (possibly empty) sequence of nonterminal and terminal symbols Natural Language Processing 6(11)

44 Example Grammar S NP VP Punct Verb had VP VP PP Noun news VP Verb NP Noun effect NP NP PP Noun markets NP Adj Noun Adj Economic PP Prep NP Adj little Adj financial Prep on Punct. Natural Language Processing 7(11)

45 Example Grammar Grammar Lexicon S NP VP Punct Verb had VP VP PP Noun news VP Verb NP Noun effect NP NP PP Noun markets NP Adj Noun Adj Economic PP Prep NP Adj little Adj financial Prep on Punct. Natural Language Processing 7(11)

46 Derivations S Natural Language Processing 8(11)

47 Derivations S NP VP Punct Natural Language Processing 8(11)

48 Derivations S NP VP Punct Adj Noun VP Punct Natural Language Processing 8(11)

49 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Natural Language Processing 8(11)

50 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Natural Language Processing 8(11)

51 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Economic news Verb NP Punct Natural Language Processing 8(11)

52 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Economic news Verb NP Punct Economic news had NP Punct Natural Language Processing 8(11)

53 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Economic news Verb NP Punct Economic news had NP Punct Economic news had NP PP Punct Natural Language Processing 8(11)

54 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Economic news Verb NP Punct Economic news had NP Punct Economic news had NP PP Punct Economic news had Adj Noun PP Punct Natural Language Processing 8(11)

55 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Economic news Verb NP Punct Economic news had NP Punct Economic news had NP PP Punct Economic news had Adj Noun PP Punct Economic news had little Noun PP Punct Natural Language Processing 8(11)

56 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Economic news Verb NP Punct Economic news had NP Punct Economic news had NP PP Punct Economic news had Adj Noun PP Punct Economic news had little Noun PP Punct Economic news had little effect PP Punct Natural Language Processing 8(11)

57 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Economic news Verb NP Punct Economic news had NP Punct Economic news had NP PP Punct Economic news had Adj Noun PP Punct Economic news had little Noun PP Punct Economic news had little effect PP Punct Economic news had little effect Prep NP Punct Natural Language Processing 8(11)

58 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Economic news Verb NP Punct Economic news had NP Punct Economic news had NP PP Punct Economic news had Adj Noun PP Punct Economic news had little Noun PP Punct Economic news had little effect PP Punct Economic news had little effect Prep NP Punct Economic news had little effect on NP Punct Natural Language Processing 8(11)

59 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Economic news Verb NP Punct Economic news had NP Punct Economic news had NP PP Punct Economic news had Adj Noun PP Punct Economic news had little Noun PP Punct Economic news had little effect PP Punct Economic news had little effect Prep NP Punct Economic news had little effect on NP Punct Economic news had little effect on Adj Noun Punct Natural Language Processing 8(11)

60 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Economic news Verb NP Punct Economic news had NP Punct Economic news had NP PP Punct Economic news had Adj Noun PP Punct Economic news had little Noun PP Punct Economic news had little effect PP Punct Economic news had little effect Prep NP Punct Economic news had little effect on NP Punct Economic news had little effect on Adj Noun Punct Economic news had little effect on financial Noun Punct Natural Language Processing 8(11)

61 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Economic news Verb NP Punct Economic news had NP Punct Economic news had NP PP Punct Economic news had Adj Noun PP Punct Economic news had little Noun PP Punct Economic news had little effect PP Punct Economic news had little effect Prep NP Punct Economic news had little effect on NP Punct Economic news had little effect on Adj Noun Punct Economic news had little effect on financial Noun Punct Economic news had little effect on financial markets Punct Natural Language Processing 8(11)

62 Derivations S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Economic news Verb NP Punct Economic news had NP Punct Economic news had NP PP Punct Economic news had Adj Noun PP Punct Economic news had little Noun PP Punct Economic news had little effect PP Punct Economic news had little effect Prep NP Punct Economic news had little effect on NP Punct Economic news had little effect on Adj Noun Punct Economic news had little effect on financial Noun Punct Economic news had little effect on financial markets Punct Economic news had little effect on financial markets. Natural Language Processing 8(11)

63 Phrase Structure Trees S NP VP Punct Adj Noun VP Punct Economic Noun VP Punct Economic news VP Punct Economic news Verb NP Punct Economic news had NP Punct Economic news had NP PP Punct Economic news had Adj Noun PP Punct Economic news had little Noun PP Punct Economic news had little effect PP Punct Economic news had little effect Prep NP Punct Economic news had little effect on NP Punct Economic news had little effect on Adj Noun Punct Economic news had little effect on financial Noun Punct Economic news had little effect on financial markets Punct Economic news had little effect on financial markets. S VP NP PP NP NP NP Adj Noun Verb Adj Noun Prep Adj Noun Punct Economic news had little effect on financial markets. Natural Language Processing 9(11)

64 Treebanks and Grammars Constituency treebanks Treebanks with constituency-based annotation Example: Penn Treebank of English Treebank grammars We can extract CFGs from constituency treebanks Treebank grammars can be used to build syntactic parsers Natural Language Processing 10(11)

65 Quiz The big bear scared the little dog. True or false 1. The substring dog is a noun phrase 2. The substring little dog is a noun phrase 3. The substring the little dog is a noun phrase 4. The substring scared the little dog is a noun phrase 5. The substring scared the little dog is a verb phrase Natural Language Processing 11(11)

66 Natural Language Processing Universal Dependencies Joakim Nivre Uppsala University Department of Linguistics and Philology Natural Language Processing 1(13)

67 Universal Dependencies (UD) Framework for multilingual grammatical annotation Morphological layer: Part-of-speech tags Morphological features Lemmas Syntactic layer: Dependency tree Natural Language Processing 2(13)

68 Predicates and Arguments nsubj obj iobj nsubj Snoopy barked Natural Language Processing 3(13)

69 Predicates and Arguments nsubj obj iobj nsubj nsubj obj Snoopy barked Snoopy hugged Woodstock Natural Language Processing 3(13)

70 Predicates and Arguments nsubj obj iobj nsubj nsubj obj Snoopy barked Snoopy hugged Woodstock obj nsubj iobj Snoopy gave Woodstock food Natural Language Processing 3(13)

71 Predicates and Modifiers advmod obl nsubj advmod Snoopy barked angrily Natural Language Processing 4(13)

72 Predicates and Modifiers advmod obl obl nsubj advmod nsubj obj case Snoopy barked angrily Snoopy hugged Woodstock in Paris Natural Language Processing 4(13)

73 Predicates and Modifiers advmod obl obl nsubj advmod nsubj obj case Snoopy barked angrily Snoopy hugged Woodstock in Paris obl nsubj obj case Snoopy gave food to Woodstock Natural Language Processing 4(13)

74 Noun Phrases amod nmod amod black coffee Natural Language Processing 5(13)

75 Noun Phrases amod nmod nmod amod black coffee case coffee with cream Natural Language Processing 5(13)

76 Noun Phrases amod nmod nmod amod black coffee case coffee with cream nmod amod case black coffee with cream Natural Language Processing 5(13)

77 Quiz Snoopy likes tasty bagels Which words are dependents of likes? 1. Snoopy 2. likes 3. tasty 4. bagels Natural Language Processing 6(13)

78 Quiz Snoopy likes tasty bagels Which dependency relation does bagels have to its head? 1. nsubj 2. obj 3. obl 4. nmod Natural Language Processing 7(13)

79 Function Words aux case det obj nsubj det Snoopy hugged the bird Natural Language Processing 8(13)

80 Function Words aux case det obj nsubj nsubj det aux obj Snoopy hugged the bird Snoopy will hug Woodstock Natural Language Processing 8(13)

81 Function Words aux case det obj nsubj nsubj det aux obj Snoopy hugged the bird Snoopy will hug Woodstock nsubj aux aux obl case det Snoopy could have slept on the roof Natural Language Processing 8(13)

82 Subordinate Clauses ccomp mark nsubj nsubj aux obj ccomp xcomp mark Snoopy promised that he would hug Woodstock Natural Language Processing 9(13)

83 Subordinate Clauses ccomp mark nsubj nsubj aux obj ccomp xcomp mark Snoopy promised that he would hug Woodstock xcomp nsubj mark obj Snoopy promised to hug Woodstock Natural Language Processing 9(13)

84 Adverbial and Adnominal Clauses advcl nsubj obj mark nsubj obj advcl acl Snoopy hugs Woodstock if Lucy hugs Linus Natural Language Processing 10(13)

85 Adverbial and Adnominal Clauses advcl nsubj obj mark nsubj obj advcl acl Snoopy hugs Woodstock if Lucy hugs Linus acl det nsubj obj the bird who hugged Snoopy Natural Language Processing 10(13)

86 Adverbial and Adnominal Clauses advcl nsubj obj mark nsubj obj advcl acl Snoopy hugs Woodstock if Lucy hugs Linus acl acl obj det nsubj obj det nsubj the bird who hugged Snoopy the bird who Snoopy hugged Natural Language Processing 10(13)

87 Adverbial and Adnominal Clauses advcl nsubj obj mark nsubj obj advcl acl Snoopy hugs Woodstock if Lucy hugs Linus acl:relcl acl:relcl obj det nsubj obj det nsubj the bird who hugged Snoopy the bird who Snoopy hugged Natural Language Processing 10(13)

88 Other Relations flat compound punct nsubj obj Charlie Brown ate a tuna bagel. Natural Language Processing 11(13)

89 Other Relations flat compound punct nsubj flat Charlie Brown ate a tuna bagel. Natural Language Processing 11(13)

90 Other Relations flat compound punct flat nsubj obj compound Charlie Brown ate a tuna bagel. Natural Language Processing 11(13)

91 Other Relations flat compound punct punct flat nsubj obj compound Charlie Brown ate a tuna bagel. Natural Language Processing 11(13)

92 Quiz Snoopy could have tried to hug Woodstock Which verb is the root of the dependency tree? 1. could 2. have 3. tried 4. hug Natural Language Processing 12(13)

93 Quiz Snoopy could have tried to hug Woodstock Which verbs have the dependency relation aux to its head? 1. could 2. have 3. tried 4. hug Natural Language Processing 13(13)