The Prague Dependency Treebank (and WS02)

Transcription

1 The Prague Dependency Treebank (and WS02) Jan Hajič Institute of Formal and Applied Linguistics School of Computer Science Faculty of Mathematics and Physics Charles University, Prague, Czech Republic

2 This Talk: an Overview The Prague Dependency Treebank The project The 3 annotation layers: morphology surface syntax (also: the lab) deep syntax Use of the deep representation: Machine translation Challenges for NL generation 5/7/2002 PreWS02 Summer School 2

3 The Prague Dependency Treebank Project (Czech Language Treebank) PDT v. 0.5 released (JHU workshop) 400k words annotated, unchecked 2001 PDT 1.0 released (LDC): 1.3MW annotated, morphology & surface syntax 2004 PDT 2.0 release planned 1.0MW annotated, underlying (deep) syntax: the tectogrammatical layer 5/7/2002 PreWS02 Summer School 3

4 Annotation Layers Morphology Tag (full morphology, 13 categories), lemma Analytical layer (surface syntax) Dependency, analytical function Tectogrammatical layer (underlying syntax) Dependency, functor (detailed), grammatemes, ellipsis solution, coreference, topic/focus (deep word order) 5/7/2002 PreWS02 Summer School 4

5 Morphological Annotation 13 categories: Category # of values Example(s) POS 10 N (noun), Z (punctuation) SUBPOS 75 P (personal pron.), U (possessive adj.) GENDER 8 I (masc. inanimate), X (any), - (N.A) NUMBER 4 P (plural), D (dual) CASE 9 1 (nominative), 6 (locative) POSSGENDER 4 M (masc. animate), F (feminine) POSSNUMBER 3 S (singular), P (plural) PERSON 5 1 (first),... TENSE 4 P (present), M (past) GRADE 5 3 (superlative) NEGATION 3 A (affirmative), N (negative) VOICE 3 A (active), P (passive) VAR 11 1 (1 st variant), 6 (colloq. style), 8 (abbrev.) 5/7/2002 PreWS02 Summer School 5

6 Layer 1: Morphology Tag: 13 categories Example: AAFP3----3N---- Adjective no poss. Gender negated Regular no poss. Number no voice Feminine no person reserve1 Plural no tense reserve2 Dative superlative base var. Lemma: unique identifier Ex.: (to) the most uninteresting Books/verb -> book-1, went -> go, to/prep. -> To-1 5/7/2002 PreWS02 Summer School 6

7 Layer 2: Analytical syntax Surface, dependency-based representation Every word gets a node, plus one (root) Interested in: dependency structure analytical function: Pred, Sb, Obj, Adv, Atr, Atv, Pnom; AuxV, AuxP, AuxC,...; Coord, Apos, parenthesis ExD 5/7/2002 PreWS02 Summer School 7

8 Layer 2: Analytical syntax Dependency + Analytical Function dependent governor The influence of the Mexican crisis on Central and Eastern Europe has apparently been underestimated. 5/7/2002 PreWS02 Summer School 8

9 Comparison: parse trees vs. dependency Compare: Lexicalized parse tree S(walks) Dependency tree walks VBZ NP(John) VP(walks) NNP John walks VBZ John NNP 5/7/2002 PreWS02 Summer School 9

10 Layer 3: Tectogrammatical Underlying (deep) syntax 4 sublayers: dependency structure, (detailed) functors topic/focus and deep word order coreference (mostly grammatical only) all the rest (grammatemes): detailed functors underlying gender, number,... 5/7/2002 PreWS02 Summer School 10

11 Analytical vs. Tectogrammatical annotation (TR: sublayer 1 only shown) Underlying verb + tense Deep function Elided Actor in Another ellipsis... Prepositions out (TR: sublayer 1 only shown) 5/7/2002 PreWS02 Summer School 11

12 Layer 3: Tectogrammatical Underlying (deep) syntax 4 sublayers: dependency structure, (detailed) functors topic/focus and deep word order coreference (mostly grammatical only) all the rest (grammatemes): detailed functors underlying gender, number,... 5/7/2002 PreWS02 Summer School 12

13 Dependency structure Similar to the surface (Analytical) layer......but: certain nodes deleted auxiliaries, non-autosemantic words, punctuation some nodes added based on word (mostly verb, noun) valency some ellipsis resolution detailed dependency relation labels (functors) 5/7/2002 PreWS02 Summer School 13

14 Tectogrammatical Functors Actants : ACT, PAT, EFF, ADDR, ORIG cannot repeat in a clause, usually compulsory Free modifications (~ 50) can repeat; optional, sometimes compulsory Ex.: LOC, DIR1,...; TWHEN, TTILL,...; RESTR, DESC; BEN, ATT, ACMP, INTT, MANN; MAT, APP; ID, DPHR, Special Coordination, Rhematizers, Foreign phrases,... 5/7/2002 PreWS02 Summer School 14

15 Tectogrammatical Example Analytical verb form:» (he) allowed would-be to-be enrolled» směl by být zapsán Collapsed Additional attributes (grammatemes): conditional + allow 5/7/2002 PreWS02 Summer School 15

16 Tectogrammatical Example Predicate with copula (state)» (the) pool has-been already filled» bazén byl již napuštěný ý 5/7/2002 PreWS02 Summer School 16

17 Tectogrammatical Example Passive construction (action)» (The) book has-been translated [by Mr. X]» Kniha byla přeložena Disappeared Added 5/7/2002 PreWS02 Summer School 17

18 Tectogrammatical Example Object» (he) gave him a-book» dal mu knihu Obj goes into ACT, PAT, ADDR, EFF or ORIG based on governor s valency frame 5/7/2002 PreWS02 Summer School 18

19 Tectogrammatical Example Relative clause (embedded) (a) house, which is expensive, (we) (to-ourselves) will-notbuy dům, který je drahý, si nekoupíme 5/7/2002 PreWS02 Summer School 19

20 Tectogrammatical Example Incomplete phrases» Peter works well, but Paul badly» Petr pracuje dobře, ale Pavel špatně Added 5/7/2002 PreWS02 Summer School 20

21 Layer 3: Tectogrammatical Underlying (deep) syntax 4 sublayers: dependency structure, (detailed) functors topic/focus and deep word order coreference (mostly grammatical only) all the rest (grammatemes): detailed functors underlying gender, number,... 5/7/2002 PreWS02 Summer School 21

22 Deep word order, topic/focus Deep word order: from old information to the new one (left-to-right) at every level (head included) projectivity by definition i.e., partial level-based order -> total d.w.o. Topic/focus/contrastive topic attribute of every node restricted by d.w.o. and other constraints 5/7/2002 PreWS02 Summer School 22

23 Deep word order, topic/focus Example: Analytical dep. tree: Baker bakes rolls. vs. Baker IC bakes rolls. 5/7/2002 PreWS02 Summer School 23

24 Layer 3: Tectogrammatical Underlying (deep) syntax 4 sublayers: dependency structure, (detailed) functors topic/focus and deep word order coreference (mostly grammatical only) all the rest (grammatemes): detailed functors underlying gender, number,... 5/7/2002 PreWS02 Summer School 24

25 Coreference Grammatical (vs. textual) Ex.: Peter moved to Iowa after he finished his PhD. move PRED Peter Iowa ACT DIR1 finish TWHEN he ACT he APP PhD PAT NB: poster about Control, this morning 5/7/2002 PreWS02 Summer School 25

26 Layer 3: Tectogrammatical Underlying (deep) syntax 4 sublayers: dependency structure, (detailed) functors topic/focus and deep word order coreference (mostly grammatical only) all the rest (grammatemes): detailed functors underlying gender, number,... 5/7/2002 PreWS02 Summer School 26

27 Grammatemes Syntactic (= detailed functors) only for some functors: WHEN: before/after LOC: next-to, behind, in-front-of Lexical, underlying number (SG/PL), tense, modality, degree of comparison strictly only where necessary (agreement!) 5/7/2002 PreWS02 Summer School 27

28 The Valency Lexicon Valency frames each verb (+ some nouns, adjectives) has slots for functor/form pairs: give: ACT(Nom) PAT(Acc) ADDR(to+Dat) Basic set prepared in advance, annotators add entries on-the-go, checking and approval process follows (consistency) Compare: Levin s Classes, Proposition Bank 5/7/2002 PreWS02 Summer School 28

29 Tectogrammatical Annotation Manual annotation 4 groups of annotators ~ 4 sublayers Special graphical tool (TrEd) Customizable graphical tree editor Preprocessing Data from Analytical Layer, preprocessed Online dependency function preassignment 5/7/2002 PreWS02 Summer School 29

30 The [Manual] Annotation Tool Perl/PerlTk based, platform-independent Linux, Windows 95/98/2000, Solaris,... Perl as the macro language unlimited online processing capability Flexibility for interactive checking split screen, graphical diff function Customization, printing, plugins 5/7/2002 PreWS02 Summer School 30

31 The TrEd Tree Editor Graphical tool TrEd Main screen: Original sentence: [This year s flu season is still quiet in Europe.] Editing window customization Run a macro Multiwindow editing/compare 5/7/2002 PreWS02 Summer School 31

32 Valency Lexicon in TrEd to write sth (about sth) 5/7/2002 PreWS02 Summer School 32

33 What Is It Good For? Machine Translation TL representation is closer to an interlingua than surface (analytical) syntax => less work in the transfer phase more work in parsing and generation...but advantage in multilingual MT application Question answering same representation for questions and answers 5/7/2002 PreWS02 Summer School 33

34 Machine Translation Architecture Typical (structural) MT system: Transfer a parse Analysis (parsing) Generation (synthesis) source sentence target sentence 5/7/2002 PreWS02 Summer School 34

35 Machine Translation Architecture Tectogrammatical layer-based system: Transfer (tectogrammatical) parsing parsing morphology (tagging) tectogrammatical layer analytical layer morphological layer generation linearization morph. synthesis source sentence target sentence 5/7/2002 PreWS02 Summer School 35

36 Comparison: analytical layer 5/7/2002 PreWS02 Summer School 36

37 Comparison: tectogrammatical l. The [Homestead s] only remaining baker bakes the most famous roll s to the north of Long River. al-xabaaz al- axiir al-baaqii [fii Homestead] yaśmacu ashhar al-kruasaanaat ilaa shimaal min Long River. 5/7/2002 PreWS02 Summer School 37

38 The Three Crucial Steps Analytical (surface) Tectogrammatical additional parsing required Transfer minimal effort: only true transformations needed (like swimming ~ schwimmen gern) Generation back from Tectogrammatical representation to Analytical (surface syntax) 5/7/2002 PreWS02 Summer School 38

39 The Devil s In... The additional three steps: (tectogrammatical) parsing parsing morphology (tagging) Transfer tectogrammatical layer analytical layer morphological layer Generation linearization (trivial) morph. synthesis (easy) source sentence target sentence 5/7/2002 PreWS02 Summer School 39

40 The Devil s In... The additional three steps: Tectogrammatical parsing (Simple) transfer tectogrammatical layer source analytical layer target Generation: - Deletions - Insertions: prepositions, conjunctions,... - Word order - Morphology 5/7/2002 PreWS02 Summer School 40

41 Components:...the Generation Deletions of nodes [rare if going into English] Insertions of nodes prepositions, conjunctions, punctuation splitting phrases/idioms/named entities Tree reorganization (numeric expressions) Surface word order (analytical tree: defined w.o.) Morphology (agreement, cases based on subcat) 5/7/2002 PreWS02 Summer School 41

42 Generation Insertion of Prepositions střed center tectogrammatical layer přitažlivost APP.sg gravity APP.sg center středu přitažlivosti.nfs2 Atr analytical layer of AuxP gravity.nn Atr 5/7/2002 PreWS02 Summer School 42

43 Surface word order přijít.past Generation come.past tectogrammatical layer včera Petr yesterday Peter TWHEN ACT TWHEN ACT přijít.vb3sp come.vbd včera Petr Adv Sb analytical layer Peter yesterday Sb Adv 5/7/2002 PreWS02 Summer School 43

44 Generation: Complex Input English translation 5/7/2002 PreWS02 Summer School 44

45 Generation: How-To (1) Statistical, (perhaps) in two steps Analytical tree reconstruction everything except word order i.e., includes morphology (tag assignment) Word Order projective trees assumed here 1 thus, it is sufficient to determine level-by-level word order 1 Additional step required for non-projective constructions [can be avoided for English] 5/7/2002 PreWS02 Summer School 45

46 Generation: How-To (2) Reconstruction: two possible ways transformation-based learning ([fn]tbl) probabilistic, by a dependency tree model: based on triplets <word,tag,afun> and dependency relation (governor,dependent) ~ Collins bilexical model, Charniak parser model, Bangalore & Rambow afun instead of nonterminals 5/7/2002 PreWS02 Summer School 46

47 Generation: How-To (3) Word order language model for a single level in the tree: <word,tag,afun> triples; includes head (no afun) come.vbd Peter.NNP yesterday.adv Sb Adv non-projective constructions (and some more) by classic n-gram LM 5/7/2002 PreWS02 Summer School 47

48 Generation: How-To (4) Data trained on WSJ: converted to analytical dependency trees adapted Jason Eisner s head assignment rules added rules for heads of base NPs added rules for analytical functions rule-based parsing to tectogrammatical layer (for now; manual annotation will follow) i.e., TR AR data available (English) 5/7/2002 PreWS02 Summer School 48

49 Some pointers Current version of PDT: v1.0 morphology + analytical level 1.3M words (train/dev test/eval test) Projects -> Treebank LDC2001T10 (PDT v1.0) Workshop 2002 Using TL for MT Generation 5/7/2002 PreWS02 Summer School 49