Treebank mining with GrETEL. Liesbeth Augustinus Frank Van Eynde

Transcription

1 Treebank mining with GrETEL Liesbeth Augustinus Frank Van Eynde GrETEL tutorial - 27 March, 2015

2 GrETEL Greedy Extraction of Trees for Empirical Linguistics Search engine for treebanks

3 GrETEL Greedy Extraction of Trees for Empirical Linguistics Search engine for treebanks Treebank = syntactically annotated corpus o o o Penn Treebank (English) TüBa (German) LASSY, CGN, SoNaR (Dutch)

4 NEDERBOOMS Exploitation of Dutch treebanks for research in linguistics CLARIN project October, 2010 February, 2012 Goals: o User-friendly tools o Fast and accurate Result: o GrETEL 1.0 o

5 Update of GrETEL 1.0 CLARIN project June, 2013 July, 2014 GrETEL 2.0 Goals: o Improve GUI o Make more data accessible Result: o GrETEL 2.0 o

6 TREEBANKS CGN treebank Spoken Dutch LASSY small Written Dutch Stylistic & regional differences conversations vs read texts NL vs VL Stylistic differences Wikipedia vs legal texts ± 1M words ± 1M words 130k sentences Manually corrected 65k sentences Manually corrected

7 TREEBANKS SoNaR Written Dutch Stylistic differences Wikipedia vs legal texts ± 500M words 41M sentences Not corrected

8 GrETEL Greedy Extraction of Trees for Empirical Linguistics Search engine for treebanks Treebank = syntactically annotated corpus o o o Penn Treebank (English) TüBa (German) LASSY, CGN, SoNaR (Dutch) Parser o E.g. Alpino (Van Noord 2006)

9 ALPINO PARSER Dit is een zin. >> ALPINO parser >> This is a sentence.

10 ALPINO PARSER Dit is een zin. >> ALPINO parser >> This is a sentence. XML trees Query language: XPath

11 XPATH and and and and and

12 XPATH and and and and and

13 XPATH and and and and and

14 XPATH

15 GrETEL 2 search modes: o Example-based search o XPath search

16 GrETEL 2 search modes: o Example-based search advantage: no or limited knowledge of data structure and/or formal query languages needed o XPath search

17 the user 1. Example sentence 2. Inspect parse 3. Indicate relevant items of the sentence 4. Select treebank 5. (Adapt XPath) 6. Inspect results GrETEL Parser (Alpino) Automatically generate XPath expression Present results

18 OUTLINE GrETEL in a nutshell GrETEL demo o o Case study Search options Conclusions

19 CASE STUDY Infinitivus Pro Participio (IPP) constructions in Dutch Hij heeft Marie horen zingen. He has heard Mary sing. dat Jan niet is kunnen komen. that Jan was not able to come.

20 CASE STUDY Infinitivus Pro Participio (IPP) constructions in Dutch Hij heeft Marie horen/*gehoord zingen. He has heard Mary sing. dat Jan niet is kunnen/*gekund komen. that Jan was not able to come.

21 GrETEL ONLINE

22 INPUT

23 INPUT PARSE

24 SELECTION MATRIX

25 SELECTION GUIDELINES

26 TREEBANK SELECTION

27 TREEBANK SELECTION

28 QUERY OVERVIEW

29 IPP constructions in CGN RESULTS Hij heeft Marie horen zingen. He has heard Mary sing. 344 hits

30 RESULTS

31 RESULTS: table

32 RESULTS: data

33 greedy search RESULTS: data

34 RESULTATEN: trees

35 IPP constructions in CGN RESULTS Hij heeft Marie horen zingen. He has heard Mary sing. 344 hits dat Jan niet is kunnen komen. that Jan was not able to come. 24 hits

36 MORE RESULTS Option 1: Use different queries Hij heeft Marie horen zingen. He has heard Mary sing. 344 hits dat hij Marie heeft horen zingen. that he has heard Mary sing. 79 hits dat Jan niet is kunnen komen. that Jan was not able to come. 24 hits Jan is niet kunnen komen. Jan was not able to come. 120 hits TOTAL: 567 hits

37 MORE RESULTS Option 2: Adapt query (via XPath Search ) //node[@cat="smain" and node[@rel="hd" and node[@rel="vc" and node[@rel="hd" and node[@rel="vc" and node[@rel="hd" //node[(@cat="smain" and node[@rel="hd" and (@lemma="hebben" and node[@rel="vc" and node[@rel="hd" and node[@rel="vc" and node[@rel="hd"

38 MORE RESULTS

39 MORE RESULTS Option 2: Adapt query (via XPath Search )

40 MORE RESULTS Option 2: Adapt query (via XPath Search ) //node[@cat="smain" and node[@rel="hd" and node[@rel="vc" and node[@rel="hd" and node[@rel="vc" and node[@rel="hd" //node[(@cat="smain" and node[@rel="hd" and (@lemma="hebben" and node[@rel="vc" and node[@rel="hd" and node[@rel="vc" and node[@rel="hd" 566 hits (one sentence matches twice: fva )

41

42 OUTLINE GrETEL in a nutshell GrETEL demo o o Case study Search options Conclusions

43 ADVANCED SEARCH

44 ADVANCED SEARCH

45 ADVANCED SEARCH

46 ADVANCED SEARCH

47 SEARCH OPTIONS Below annotation matrix

48 WORD ORDER PP-over-V o V + PP o dat hij opstond met een kater.... that he woke up with a hangover. o o PP + V dat hij met een kater opstond. that he with a hangover woke-up... that he woke up with a hangover.

49 PP-over-V in LASSY small o o o V + PP WORD ORDER dat hij opstond met een kater.... that he woke up with a hangover. 2,890 hits in 2,764 sentences But: results include PP + V as well!

50 PP-over-V in LASSY small o o WORD ORDER V + PP + word order option dat hij opstond met een kater.... that he woke up with a hangover. 787 hits in 775 sentences Results only include V + PP

51 IGNORE TOP NODE

52 CONTEXT

53 CONTEXT

54 OUTLINE GrETEL in a nutshell GrETEL demo o o Case study Search options Conclusions

55 CONCLUSIONS GrETEL: search engine for Dutch treebanks Input = natural language example Output = sample of similar sentences Syntactic concordancer Available online (via Mozilla Firefox) No installation required

56 Try it yourself! Thanks for your attention!