Natural Language Processing. Lecture 27: Conclusion

Transcription

1 Natural Language Processing Lecture 27: Conclusion

2 Levels of Linguistc nowledge spoken phonetcs phonology morphology writen orthography shallower syntax semantcs deeper pragmatcs discourse

3 uygarlastramadıklarımızdanmıssınızcasına (behaving) as if you are among those whom we could not civilize

4 uygarlastramadıklarımızdanmıssınızcasına (behaving) as if you are among those whom we could not civilize uygar civilized +las become +tr cause to +ama not able +dık past partciple +lar plural +ımız frst person plural possessive ( our ) +dan second person plural ( y all ) +mıs past +sınız ablatve case ( from/among ) +casına fnite verb adverb ( as if )

5 Finite-State Automaton Q: a fnite set of states q0 Q: a special start state F Q: a set of fnal states Σ: a fnite alphabet Transitons:... qi s Σ* qj... Encodes a set of strings that can be recognized by following paths from q0 to some state in F.

6 Levels of Linguistc nowledge spoken phonetcs phonology ambiguity morphology syntax semantcs writen orthography shallower deeper pragmatcs discourse

7 Noisy Channel source What you want y What you see x channel decode

8 source Noisy Channel NN VB RB y Cats meow ofen x channel decode

9 Noisy Channel source How are you? y 你好吗? x channel decode

10 Noisy Channel source Okay, Google y x channel decode

11 Startng and Stopping Unigram model:... Bigram model:... Trigram model:...

12 Language Modeling Questons Why do we use context? What does smoothing do, and why is it necessary? What do we use to evaluate language models?

13 Tagging

14 Broad POS categories open classes closed classes nouns verbs adjectves adverbs prepositons partcles determiners numerals pronouns conjunctons auxiliary verbs

15 Syntax

16 Parsing C Y vs. Earley s Algorithm Both dynamic programming CNF vs. general forms

17 C Y Algorithm: Chart Noun, Verb - VP,S - S book Det NP - NP this Noun - - fight Prep PP through PNoun, NP Houston

18 C Y Equatons

19 Semantcs

20 Where s the beef? Sentences from the brown corpus. Extracted from the concordancer in The Compleat Lexical Tutor, htp://

21 chicken

22 Synsets for dog (n) S: (n) dog, domestc dog, Canis familiaris (a member of the genus Canis (probably descended from the common wolf) that has been domestcated by man since prehistoric tmes; occurs in many breeds) "the dog barked all night" S: (n) frump, dog (a dull unatractve unpleasant girl or woman) "she got a reputaton as a frump"; "she's a real dog" S: (n) dog (informal term for a man) "you lucky dog" S: (n) cad, bounder, blackguard, dog, hound, heel (someone who is morally reprehensible) "you dirty dog" S: (n) frank, frankfurter, hotdog, hot dog, dog, wiener, wienerwurst, weenie (a smooth-textured sausage of minced beef or pork usually smoked; ofen served on a bread roll) S: (n) pawl, detent, click, dog (a hinged catch that fts into a notch of a ratchet to move a wheel forward or prevent it from moving backward) S: (n) andiron, fredog, dog, dog-iron (metal supports for logs in a freplace) "the andirons were too hot to touch" 22

23

24 Entty Linking Mary picked up the ball. She threw it to me.

25 Semantc oles PropBank is a set of verb-sense-specifc frames with informal descriptons for their arguments. Consider the word Agree ARG0: agreer ARG1: propositon ARG2: other entty agreeing [The group] ARG0 agreed [it wouldn t make an ofer] ARG1. Usually [John] ARG0 agrees [with Mary on everything] ARG2.

26 Fall (move downward) in PropBank arg1: logical subject, patent, thing falling arg2: extent, amount fallen arg3: startng point arg4: ending point argm-loc: medium Sales fell to $251.2 million from $278.8 million. The average junk bond fell by 4.2%. The meteor fell through the atmosphere, crashing into Cambridge.

27 M L #1: First-Order Logic DressCode(ThePorch) Functon Serves(UnionGrill, AmericanFood) estaurant(uniongrill) Predicates Have(Speaker, FiveDollars) ^ Have(Speaker, LotOfTime) x Person(x) Have(x, FiveDollars) x,y Person(x) ^ estaurant(y) ^ HasVisited(x,y)

28 First Order Logic: Advantages Flexible Well-understood Widely used

29 EM We ofen have unlabeled or incomplete data EM is an for learning without labels, e.g., classifcaton without classes E-step M-step Pick random centroids! Iterate the following:! Use centroids to label the data! Compute centroids using the labeled data! Keep doing this until labels don t change

30 NLP Uses Answer questions using the Web Translate documents from one language to another Do library research; summarize Manage messages intelligently Help make informed decisions Follow directions given by any user Fix your spelling or grammar Grade exams Write poems or novels Listen and give advice Estimate public opinion Read everything and make predictions Interactively help people learn Help disabled people Help refugees/disaster victims Document or reinvigorate indigenous languages

31 More NLP... Language Technologies Minor 4 LT courses plus LT project 5 th year Masters in Language Technologies

32 More NLP Courses /692 Speech Processing Fall: Alan W Black Practcal Systems for Speech Algorithms and NLP Fall: Yulia Tsvetkov,Taylor Berg- irkpatrick esearch oriented Computatonal Semantcs Spring: Ed Hovy, Teruko Mitamura

33 More NLP Courses Neural Networks for NLP Spring: Graham Neubig Computatonal Ethics for NLP Spring: Yulia Tsvetkov, Alan W Black Advanced Multmodal ML Fall: Louis-Philippe Morency Visual, Gesture, Speech Most Neural Net Classing Always involve NLP