ANNOTATING DISCOURSE IN PRAGUE DEPENDENCY TREEBANK

Transcription

1 ANNOTATING DISCOURSE IN PRAGUE DEPENDENCY TREEBANK PDTB WORKSHOP, PHILADEPHIA APRIL 30, 2012 Lucie Poláková (Mladová), Charles University in Prague

2 PRAGUE DEPENDENCY TREEBANK & DISCOURSE 2006: Prague Dependency Treebank (PDT) 2.0 Hajič et al. : multilayer annotations of 50,000 sentences of Czech journalistic texts: - morphology - surface syntax - underlying syntax + sentence semantics (tectogrammatical tree structures) + other phenomena included: information structure, grammatically bound and textual pronominal coreference : annotation of discourse-related phenomena - Prof. Eva Hajičová

3 DISCOURSE-LEVEL PHENOMENA IN PDT Manual annotations of the whole treebank: 49,431 sentences in 3165 documents of sentences, with an average length of 15.6 sentences 1. Explicit discourse connectives + their arguments (scopes), sense tags (= PDTB II. annotation) 2. a) Extended textual coreference annotation b) Bridging relations ALL DISCOURSE PHENOMENA ANNOTATED DIRECTLY ON THE SYNTACTIC (tectogrammatical) TREES! Both projects completed 2011, currently under checking procedures

4 EXAMPLE OF ANNOTATION ON TREES 1. Searching for the possible discourse connectives in a plain text (unlike in Penn) V polovině července radní předložené rozhodnutí akceptovali. Dodnes však žádná smlouva podepsána nebyla. In middle July, the councilors accepted the proposed decision. However, no contract was signed so far. 2. Marking the discourse relation on the tectogrammatical trees:

5 In middle July, the councilors accepted the proposed decision. However, no contract was signed so far.

6 In middle July, the councilors accepted the proposed decision. However, no contract was signed so far.

7 In middle July, the councilors accepted the proposed decision. However, no contract was signed so far.

8 In middle July, the councilors accepted the proposed decision. However, no contract was signed so far. - linking of the arguments of the relation by an arrow (automatically connected with choice of semantic intepretation)

9 In middle July, the councilors accepted the proposed decision. However, no contract was signed so far. - linking of the arguments of the relation by an arrow (automatically connected with choice of semantic intepretation) - adding of the connective

10 In middle July, the councilors accepted the proposed decision. However, no contract was signed so far. - linking of the arguments of the relation by an arrow (automatically connected with choice of semantic intepretation) - adding of the connective - marking of the extent of the arguments

11 DISCOURSE REPRESENTATION COMPACT VIEW

12 SEMANTIC LABELS IN PRAGUE TEMPORAL CONTINGENCY COMPARISON (CONTRAST) EXPANSION asynchronous reason - result confrontation conjunction synchronous pragmatic reason result opposition instantiation purpose pragmatic contrast specification explication restrictive opposition equivalence condition concession generalization pragmatic condition correction (replacement) gradation conjunctive alternative disjunctive alternative

13 SEMANTIC LABELS IN PRAGUE TEMPORAL CONTINGENCY COMPARISON (CONTRAST) EXPANSION asynchronous reason - result confrontation conjunction synchronous pragmatic reason result opposition instantiation purpose pragmatic contrast specification explication restrictive opposition equivalence condition concession generalization pragmatic condition correction (replacement) gradation conjunctive alternative disjunctive alternative

14 ADDITIONAL INFORMATION ANNOTATED List structures quite a typical way of text composition Headings Alternative lexical expressions of the connectives, for example: Z toho vyplývá = tedy, takže Důvodem je = protože (This implies = so, therefore ) (The reason is that = because) Double-sense relations The so-called collections Not annotated so far: Implicit connectives, their arguments and senses (in comparison to PDTB 2.0) Attribution (partly present in some features of the syntactic trees)

15 TREEBANK STATISTICS Measured on train data = 9/10 of the treebank = 43,955 sentences Relation Intra-sentential Intersentential Intersentential Total conj opp reason cond conc preced confr spec purp corr grad Relation Intrasentential Total restr synchr explicat disjalt exempl equiv gener conjalt f_opp f_reason f_cond total

16 INTERANNOTATOR AGREEMENT MEASUREMENT treebank: 10 sections, in each of them a sample annotated by all annotators for the IAA measurement (2,084 sentences) following table: only the inter-sentential relations (assumed improvement with intra-sententials) connective-based measurement one pair of annotators average agreement on semantic types 0.77 similar measure in PDTB 0.8 (Prasad et al., LREC 2008) agreement on higher level: 4 basic semantic classes 0.89 (Penn 0.94)

17 IAA IN THE SUBSEQUENT MEASUREMENTS Measurement Connectivebased F1 measure Agreement on semantic types Kappa on sem. types train train train train train train train dtest etest train

18 WHAT HAVE WE LEARNED? Linguistically: the absolute number of explicit inter-sentential relations is low (it increases with the incorporation of "syntactic" discourse edges) coreference annotation (entity-based, EntRel) on the same data HELPS a lot things different nature of the relations (condition syntax-bound; specification text-bound) some very ambiguous connectives in Czech (ale = but) some Czech connectives have no exact English counterpart (totiž additional semantic category?) some findings correspond with Penn numbers discourse structure to some extent language independent we need genre distinction

19 WHAT HAVE WE LEARNED? Technically: trees with resolved syntactic structure help a lot (ellipsis, easy extraction of intra-sentential relations, coreference accessible ) sometimes problems to deal with larger arguments, complexity of the trees the representation offers lot of data with all the various types of linguistic information AT ONCE

20 RECENT & NEXT Completed: the annotation of explicits annotation manual in Czech and in English webpage with a browser in a sample of data intergration of the two projects to ONE layer with coreference and bridging annotations In progress: extracting relevant tectogrammatical information (intra-sentential discourse relations) release of the discourse and coreference data as PDT 2.5 Next: altlexes genre distinction of the texts interplays (information structure, Play Coref, the phenomenon of contrast...) automatic experiments

21 Thank you! This work was supported by the Grant Agency of the Czech Republic (P406/12/0658, P406/2010/0875 ) and by the Czech Ministry of Education (ME 10018).