!"#$%#&'()$*#+','()#$(-+,./01)
|
|
- Laureen Wilson
- 6 years ago
- Views:
Transcription
1 Questions!"#$%#&'()$*#+','()#$(-+,./01) Since induction is fallible, it is necessary to be able to assess its reliability!! Typical questions:! AgroParisTech! What is the true performance of my (learned) classification rule Is my learning algorithm better than this other one? (based in part on Sebastian Thrun CMU class " and on the tutorial of Padraic Cunningham at ECML-09)! Evaluating ML algorithms 2 Outline 1. Measuring the error rate 2. Confusion matrices and various performance criteria!1&0#&'()./*).+%*)*++-+)+#.*) 3. The ROC curve Evaluating ML algorithms 3 Evaluating ML algorithms 4
2 Evaluating classification rules Various sets of data The whole available data set Large data sample Very small data sample Learning set Validation set Test set Illimited sample Evaluating ML algorithms 5 Evaluating ML algorithms 6 Asymptotic behaviour (ideal case) Over-fitting (over-learning) Erreur erreur sur base de test Sur-apprentissage! Useful for very large data sets! erreur sur base d'apprentissage Arrêt de l'apprentissage t Evaluating ML algorithms 7 Evaluating ML algorithms 8
3 Over-fitting (NNs) Why using a test set?! The control parameters of the learning algorithm E.g.: number of hidden layers, number of neurons,... Are tuned in order to reduce the error on the validation set ))2-%+3*1)4-%+)5)666)*7*04$*1)! In order to have a non optimistically biased estimate of the error, one must measure it on an independent data set: the test set!!"#$%&'(!)#$%!*!+++!','-).'(!/! Evaluating ML algorithms 9 Evaluating ML algorithms 10 Evaluating classification rules Evaluating the error rate! True error:! (Real risk) e =! y " f ( x, # ) p( x, y) dx, y D D D = the true distribution! Test error:! (Empirical risk) 1 eˆ S =! y # f ( x, $ ) m x, y " ST A lot Few m = # of test examples T = test data Evaluating ML algorithms 11 Evaluating ML algorithms 12
4 Example: Confidence intervals! We want to estimate error D (h).!! The learned hypothesis incorrectly classifies 12 out of 40 examples in the test set T.! Q : What will be the true error rate?! R :???! We estimate it by using error T (h) which follows a binomial law! With mean! And standard error!!! They are estimated using the normal law with: Mean: Standard deviation: Evaluating ML algorithms 13 Evaluating ML algorithms 14 Confidence intervals Confidence intervals! The normal law!! The normal law! With probability N%, the true error error D lies in the interval:! N% 50% 68% 80% 90% 95% 98% 99% z N Evaluating ML algorithms 15 Evaluating ML algorithms 16
5 Confidence intervals (cf. Mitchell 97) Example: If T contains m examples independently sampled m! 30 Then With probability 95%, the true error e D is within: eˆ S ± 1.96 eˆ S (1! eˆ S ) m! The learned hypothesis incorrectly classifies 12 out of 40 test examples in T.! Q: What will be the true error on unseen examples?! A: With 95% confidence, the true error will lie within [0.16;0.44] " eˆ S ± 1.96 eˆ S (1! eˆ S ) m m = eˆ ˆ eˆ S (1 " es ) S = = ! m Evaluating ML algorithms 17 Evaluating ML algorithms 18 95% confidence intervals Performance curves 95% confidence intervals Erreur de test Erreur d apprentissage Evaluating ML algorithms 19 Evaluating ML algorithms 20
6 Evaluating learned hypotheses Various sets Data Lot of data Few Learning test " error Evaluating ML algorithms 21 Evaluating ML algorithms 22 Small data sets: a dilemma Small data sets: a dilemma Evaluating ML algorithms 23 Evaluating ML algorithms 24
7 Cross validation (k-fold) Data Learn on yellow, test on rose " error 1 Learn on yellow, test on rose " error 2 Learn on yellow, test on rose " error 3 k-way split Learn on yellow, test on rose " error 4 Learn on yellow, test on rose " error 5 Learn on yellow, test on rose " error 6 The leave-one-out procedure Data! Low bias! Highvariance! Tends to underestimate the error if the data are not fully i.i.d. Learn on yellow, test on rose " error 7 Learn on yellow, test on rose " error 8 [Guyon & Elisseeff, jmlr, 03]! error = # error i / k Evaluating ML algorithms 25 Evaluating ML algorithms 26 The Bootstrap estimate Problem Data! The calculation of the confidence interval supposes the independence of the estimations.! But our estimations are not independent. # " Learn on yellow, test on rose " error " Repeat and compute the mean Estimation of the true risk for the final h Mean of the risks On the k test samples Mean of the risk on whole data set Evaluating ML algorithms 27 Evaluating ML algorithms 28
8 Types of performance criteria 2-'8%1,-')0#.+,9*1) #':)"#+,-%1)4*+8-+0#'9*)9+,.*+,#) Evaluating ML algorithms 29 Evaluating ML algorithms 30 Confusion matrix Confusion matrix 14% of the butterflies are recognized as fishes Réel! Estimé! +! -! +! VP! FP! -! FN! VN! Evaluating ML algorithms 31 Evaluating ML algorithms 32
9 Types of performance criteria Types of performance criteria Evaluating ML algorithms 33 Evaluating ML algorithms 34 Types of performance criteria Types of performance criteria Evaluating ML algorithms 35 Evaluating ML algorithms 36
10 Types of performance measures Performance measures! Sensitivity! VP FN + VP! Recall! VP VP + FN! Specificity! VN VN + FP! Precision! VP VP + FP Réel! Estimé! +! -! +! VP! FP! -! FN! VN! Evaluating ML algorithms 37 Evaluating ML algorithms 38 Performance measures Performance measures! FN-rate! FN VP + FN! FP-rate! FP FP + VN! F-measure! 2 x recall x precision Recall + precision = 2 VP 2 VP + FP + FN Réel! Estimé! +! -! +! VP! FP! -! FN! VN! Evaluating ML algorithms 39 Evaluating ML algorithms 40
11 Performance measures Performance measures Evaluating ML algorithms 41 Evaluating ML algorithms 42 Performance measures H/*)IJ2)9%+"*)!!!!!!!!!!!!"#$%! &'()#! *++,! -.,! (--:) 6;<=>) 3#:) 6;56>) 6;A<A) Evaluating ML algorithms 43 Evaluating ML algorithms 44
12 The ROC curve Types of errors Evaluating ML algorithms 45 Evaluating ML algorithms 46 The ROC curve The ROC curve ROC = Receiver Operating Characteristic Probabilité de la classe Classe '+' Faux négatifs Vrais positifs Probabilité de la classe Classe '-' Classe '+' (10%) (90%) Critère de décision Probabilité de la classe Classe '-' Vrais négatifs Faux positifs Critère de décision (50%) (50%) Evaluating ML algorithms 47 Critère de décision Evaluating ML algorithms 48
13 Classe '+' Faux négatifs Classe '- ' Vrais négatifs Faux positifs Vrais positifs Critère de décision Critère de décision Classe '+' Faux négatifs (10%) Classe '- ' Vrais négatifs (50%) (50%) Faux positifs Vrais positifs (90%) Critère de décision Critère de décision The ROC curve The ROC curve PROPORTION DE VRAIS NEGATIFS 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1,0 1,0 1,0 0,9 0,9 PROPORTION DE VRAIS POSITIFS 0,8 0,7 0,6 0,5 0,4 0,3 Courbe ROC (pertinence = 0,90) Ligne de hasard (pertinence = 0,5) 0,8 0,7 0,6 0,5 0,4 0,3 PROPORTION DE FAUX NEGATIFS 0,2 0,2 0,1 0, ,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1,0 0 PROPORTION DE FAUX POSITIFS Evaluating ML algorithms 49 Evaluating ML algorithms 50 The ROC curve The ROC curve PROPORTION DE VRAIS NEGATIFS PROPORTION DE VRAIS NEGATIFS 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1,0 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1,0 1,0 1,0 1,0 1,0 0,9 0,9 0,9 Seuil "laxiste" 0,9 PROPORTION DE VRAIS POSITIFS 0,8 0,7 0,6 0,5 0,4 0,3 Courbe ROC (pertinence = 0,90) Ligne de hasard (pertinence = 0,5) 0,8 0,7 0,6 0,5 0,4 0,3 PROPORTION DE FAUXNEGATIFS PROPORTION DE VRAIS POSITIFS 0,8 0,7 0,6 0,5 0,4 0,3 Seuil "sévère" Probabilité delaclase Probabilité delaclase Probabilité delaclase Probabilité delaclase 0,8 0,7 0,6 0,5 0,4 0,3 PROPORTION DE FAUXNEGATIFS 0,2 0,2 0,2 0,2 0,1 0,1 0,1 0, ,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1, ,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1,0 PROPORTION DE FAUX POSITIFS PROPORTION DE FAUX POSITIFS Evaluating ML algorithms 51 Evaluating ML algorithms 52
14 The ROC curve Comparaison of learning algorithms! Résumé!! Comparison on a single data sets [Dietterich, 1998] recommends using: 5 x 2 cross-validation Paired t-test The McNemar test on a validation set! Comparison on multiples (different) data sets [Demsar, 2006] recommends using: Wilcoxon Signed Ranks Test The Friedman test Evaluating ML algorithms 53 Evaluating ML algorithms 54 Résumé Specific problems! Attention à votre fonction de coût : qu est-ce qui importe pour la mesure de performance?! Données en nombre fini: calculez les intervalles de confiance! Données rares : Attention à la répartition entre données d apprentissage et données test. Validation croisée.! N oubliez pas l ensemble de validation! The distribution of the classes is very unbalanced (e.g. 1% ou 1%O for one of the two classes)! Gray zone (uncertain labels)! Multi-valued functions! L évaluation est très importante Ayez l esprit critique Convainquez-vous vous même! Evaluating ML algorithms 55 Evaluating ML algorithms 56
15 Other evaluation criteria References! Intelligibility of the learned decision function E.g. SVMs or boosting are not good! Performances in generalization Often not correlated to the previous performance criterion! Various costs Data preparation Computational cost Cost of the ML expertise Cost of the domain expertise! Dietterich, T. G., (1998). Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms. Neural Computation, 10 (7) !! JapKowicz N. & Shah M. (2011). Evaluating Learning Algorithms. A classification perspective. Cambridge University Press, (An interesting book)! Evaluating ML algorithms 57 Evaluating ML algorithms 58 The Weka ML toolkit The Weka ML toolkit! Evaluating ML algorithms 59 Evaluating ML algorithms 60
Python Machine Learning
Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled
More informationEvaluating and Comparing Classifiers: Review, Some Recommendations and Limitations
Evaluating and Comparing Classifiers: Review, Some Recommendations and Limitations Katarzyna Stapor (B) Institute of Computer Science, Silesian Technical University, Gliwice, Poland katarzyna.stapor@polsl.pl
More informationIntroduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition
Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition Todd Holloway Two Lecture Series for B551 November 20 & 27, 2007 Indiana University Outline Introduction Bias and
More informationLecture 1: Machine Learning Basics
1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3
More informationLearning From the Past with Experiment Databases
Learning From the Past with Experiment Databases Joaquin Vanschoren 1, Bernhard Pfahringer 2, and Geoff Holmes 2 1 Computer Science Dept., K.U.Leuven, Leuven, Belgium 2 Computer Science Dept., University
More informationActive Learning. Yingyu Liang Computer Sciences 760 Fall
Active Learning Yingyu Liang Computer Sciences 760 Fall 2017 http://pages.cs.wisc.edu/~yliang/cs760/ Some of the slides in these lectures have been adapted/borrowed from materials developed by Mark Craven,
More informationPredicting Student Attrition in MOOCs using Sentiment Analysis and Neural Networks
Predicting Student Attrition in MOOCs using Sentiment Analysis and Neural Networks Devendra Singh Chaplot, Eunhee Rhim, and Jihie Kim Samsung Electronics Co., Ltd. Seoul, South Korea {dev.chaplot,eunhee.rhim,jihie.kim}@samsung.com
More informationCSL465/603 - Machine Learning
CSL465/603 - Machine Learning Fall 2016 Narayanan C Krishnan ckn@iitrpr.ac.in Introduction CSL465/603 - Machine Learning 1 Administrative Trivia Course Structure 3-0-2 Lecture Timings Monday 9.55-10.45am
More informationLaboratorio di Intelligenza Artificiale e Robotica
Laboratorio di Intelligenza Artificiale e Robotica A.A. 2008-2009 Outline 2 Machine Learning Unsupervised Learning Supervised Learning Reinforcement Learning Genetic Algorithms Genetics-Based Machine Learning
More informationAcquisition vs. Learning of a Second Language: English Negation
Interculturalia Acquisition vs. Learning of a Second Language: English Negation Oana BADEA Key-words: acquisition, learning, first/second language, English negation General Remarks on Theories of Second/
More informationQuickStroke: An Incremental On-line Chinese Handwriting Recognition System
QuickStroke: An Incremental On-line Chinese Handwriting Recognition System Nada P. Matić John C. Platt Λ Tony Wang y Synaptics, Inc. 2381 Bering Drive San Jose, CA 95131, USA Abstract This paper presents
More informationLaboratorio di Intelligenza Artificiale e Robotica
Laboratorio di Intelligenza Artificiale e Robotica A.A. 2008-2009 Outline 2 Machine Learning Unsupervised Learning Supervised Learning Reinforcement Learning Genetic Algorithms Genetics-Based Machine Learning
More informationProbability and Statistics Curriculum Pacing Guide
Unit 1 Terms PS.SPMJ.3 PS.SPMJ.5 Plan and conduct a survey to answer a statistical question. Recognize how the plan addresses sampling technique, randomization, measurement of experimental error and methods
More informationEvolutive Neural Net Fuzzy Filtering: Basic Description
Journal of Intelligent Learning Systems and Applications, 2010, 2: 12-18 doi:10.4236/jilsa.2010.21002 Published Online February 2010 (http://www.scirp.org/journal/jilsa) Evolutive Neural Net Fuzzy Filtering:
More informationEMC Publishing s C est à toi! Level 3, 2 nd edition Correlated to the Oregon World Language Content Standards
EMC Publishing s C est à toi! Level 3, 2 nd edition Correlated to the Oregon World Language Content Standards Oregon Correlation C est à toi! Level 1, 2 nd edition to the: Oregon World Language Content
More informationCS Machine Learning
CS 478 - Machine Learning Projects Data Representation Basic testing and evaluation schemes CS 478 Data and Testing 1 Programming Issues l Program in any platform you want l Realize that you will be doing
More informationModule 12. Machine Learning. Version 2 CSE IIT, Kharagpur
Module 12 Machine Learning 12.1 Instructional Objective The students should understand the concept of learning systems Students should learn about different aspects of a learning system Students should
More informationIterative Cross-Training: An Algorithm for Learning from Unlabeled Web Pages
Iterative Cross-Training: An Algorithm for Learning from Unlabeled Web Pages Nuanwan Soonthornphisaj 1 and Boonserm Kijsirikul 2 Machine Intelligence and Knowledge Discovery Laboratory Department of Computer
More informationUsing dialogue context to improve parsing performance in dialogue systems
Using dialogue context to improve parsing performance in dialogue systems Ivan Meza-Ruiz and Oliver Lemon School of Informatics, Edinburgh University 2 Buccleuch Place, Edinburgh I.V.Meza-Ruiz@sms.ed.ac.uk,
More informationAlgebra 1, Quarter 3, Unit 3.1. Line of Best Fit. Overview
Algebra 1, Quarter 3, Unit 3.1 Line of Best Fit Overview Number of instructional days 6 (1 day assessment) (1 day = 45 minutes) Content to be learned Analyze scatter plots and construct the line of best
More informationSystem Implementation for SemEval-2017 Task 4 Subtask A Based on Interpolated Deep Neural Networks
System Implementation for SemEval-2017 Task 4 Subtask A Based on Interpolated Deep Neural Networks 1 Tzu-Hsuan Yang, 2 Tzu-Hsuan Tseng, and 3 Chia-Ping Chen Department of Computer Science and Engineering
More informationKnowledge Transfer in Deep Convolutional Neural Nets
Knowledge Transfer in Deep Convolutional Neural Nets Steven Gutstein, Olac Fuentes and Eric Freudenthal Computer Science Department University of Texas at El Paso El Paso, Texas, 79968, U.S.A. Abstract
More informationLecture 1: Basic Concepts of Machine Learning
Lecture 1: Basic Concepts of Machine Learning Cognitive Systems - Machine Learning Ute Schmid (lecture) Johannes Rabold (practice) Based on slides prepared March 2005 by Maximilian Röglinger, updated 2010
More informationADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY DOWNLOAD EBOOK : ADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY PDF
Read Online and Download Ebook ADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY DOWNLOAD EBOOK : ADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY PDF Click link bellow and free register to download
More informationMULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question.
Ch 2 Test Remediation Work Name MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question. Provide an appropriate response. 1) High temperatures in a certain
More informationDisambiguation of Thai Personal Name from Online News Articles
Disambiguation of Thai Personal Name from Online News Articles Phaisarn Sutheebanjard Graduate School of Information Technology Siam University Bangkok, Thailand mr.phaisarn@gmail.com Abstract Since online
More informationIssues in the Mining of Heart Failure Datasets
International Journal of Automation and Computing 11(2), April 2014, 162-179 DOI: 10.1007/s11633-014-0778-5 Issues in the Mining of Heart Failure Datasets Nongnuch Poolsawad 1 Lisa Moore 1 Chandrasekhar
More informationKnowledge based expert systems D H A N A N J A Y K A L B A N D E
Knowledge based expert systems D H A N A N J A Y K A L B A N D E What is a knowledge based system? A Knowledge Based System or a KBS is a computer program that uses artificial intelligence to solve problems
More informationOPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS
OPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS Václav Kocian, Eva Volná, Michal Janošek, Martin Kotyrba University of Ostrava Department of Informatics and Computers Dvořákova 7,
More informationOCR for Arabic using SIFT Descriptors With Online Failure Prediction
OCR for Arabic using SIFT Descriptors With Online Failure Prediction Andrey Stolyarenko, Nachum Dershowitz The Blavatnik School of Computer Science Tel Aviv University Tel Aviv, Israel Email: stloyare@tau.ac.il,
More informationSTA 225: Introductory Statistics (CT)
Marshall University College of Science Mathematics Department STA 225: Introductory Statistics (CT) Course catalog description A critical thinking course in applied statistical reasoning covering basic
More informationExemplar for Internal Achievement Standard French Level 1
Exemplar for internal assessment resource French for Achievement Standard 90882 Exemplar for Internal Achievement Standard French Level 1 This exemplar supports assessment against: Achievement Standard
More informationProbability Therefore (25) (1.33)
Probability We have intentionally included more material than can be covered in most Student Study Sessions to account for groups that are able to answer the questions at a faster rate. Use your own judgment,
More informationA Case Study: News Classification Based on Term Frequency
A Case Study: News Classification Based on Term Frequency Petr Kroha Faculty of Computer Science University of Technology 09107 Chemnitz Germany kroha@informatik.tu-chemnitz.de Ricardo Baeza-Yates Center
More informationProbability estimates in a scenario tree
101 Chapter 11 Probability estimates in a scenario tree An expert is a person who has made all the mistakes that can be made in a very narrow field. Niels Bohr (1885 1962) Scenario trees require many numbers.
More informationChapter 2 Rule Learning in a Nutshell
Chapter 2 Rule Learning in a Nutshell This chapter gives a brief overview of inductive rule learning and may therefore serve as a guide through the rest of the book. Later chapters will expand upon the
More informationApplications of data mining algorithms to analysis of medical data
Master Thesis Software Engineering Thesis no: MSE-2007:20 August 2007 Applications of data mining algorithms to analysis of medical data Dariusz Matyja School of Engineering Blekinge Institute of Technology
More informationEvolution of Symbolisation in Chimpanzees and Neural Nets
Evolution of Symbolisation in Chimpanzees and Neural Nets Angelo Cangelosi Centre for Neural and Adaptive Systems University of Plymouth (UK) a.cangelosi@plymouth.ac.uk Introduction Animal communication
More informationFor Jury Evaluation. The Road to Enlightenment: Generating Insight and Predicting Consumer Actions in Digital Markets
FACULDADE DE ENGENHARIA DA UNIVERSIDADE DO PORTO The Road to Enlightenment: Generating Insight and Predicting Consumer Actions in Digital Markets Jorge Moreira da Silva For Jury Evaluation Mestrado Integrado
More informationSemi-Supervised Face Detection
Semi-Supervised Face Detection Nicu Sebe, Ira Cohen 2, Thomas S. Huang 3, Theo Gevers Faculty of Science, University of Amsterdam, The Netherlands 2 HP Research Labs, USA 3 Beckman Institute, University
More informationPROJECT 1 News Media. Note: this project frequently requires the use of Internet-connected computers
1 PROJECT 1 News Media Note: this project frequently requires the use of Internet-connected computers Unit Description: while developing their reading and communication skills, the students will reflect
More informationAssignment 1: Predicting Amazon Review Ratings
Assignment 1: Predicting Amazon Review Ratings 1 Dataset Analysis Richard Park r2park@acsmail.ucsd.edu February 23, 2015 The dataset selected for this assignment comes from the set of Amazon reviews for
More information1. Share the following information with your partner. Spell each name to your partner. Change roles. One object in the classroom:
French 1A Final Examination Study Guide January 2015 Montgomery County Public Schools Name: Before you begin working on the study guide, organize your notes and vocabulary lists from semester A. Refer
More informationPaper: Collaborative Information Behaviour of Engineering Students
Nasser Saleh, Andrew Large McGill University, Montreal, Quebec Paper: Collaborative Information Behaviour of Engineering Students Abstract: Collaborative information behaviour is an emerging area in information
More informationTHE ROLE OF DECISION TREES IN NATURAL LANGUAGE PROCESSING
SISOM & ACOUSTICS 2015, Bucharest 21-22 May THE ROLE OF DECISION TREES IN NATURAL LANGUAGE PROCESSING MarilenaăLAZ R 1, Diana MILITARU 2 1 Military Equipment and Technologies Research Agency, Bucharest,
More informationVersion Space. Term 2012/2013 LSI - FIB. Javier Béjar cbea (LSI - FIB) Version Space Term 2012/ / 18
Version Space Javier Béjar cbea LSI - FIB Term 2012/2013 Javier Béjar cbea (LSI - FIB) Version Space Term 2012/2013 1 / 18 Outline 1 Learning logical formulas 2 Version space Introduction Search strategy
More informationPredicting Students Performance with SimStudent: Learning Cognitive Skills from Observation
School of Computer Science Human-Computer Interaction Institute Carnegie Mellon University Year 2007 Predicting Students Performance with SimStudent: Learning Cognitive Skills from Observation Noboru Matsuda
More informationOptimizing to Arbitrary NLP Metrics using Ensemble Selection
Optimizing to Arbitrary NLP Metrics using Ensemble Selection Art Munson, Claire Cardie, Rich Caruana Department of Computer Science Cornell University Ithaca, NY 14850 {mmunson, cardie, caruana}@cs.cornell.edu
More informationDetecting Wikipedia Vandalism using Machine Learning Notebook for PAN at CLEF 2011
Detecting Wikipedia Vandalism using Machine Learning Notebook for PAN at CLEF 2011 Cristian-Alexandru Drăgușanu, Marina Cufliuc, Adrian Iftene UAIC: Faculty of Computer Science, Alexandru Ioan Cuza University,
More informationA Neural Network GUI Tested on Text-To-Phoneme Mapping
A Neural Network GUI Tested on Text-To-Phoneme Mapping MAARTEN TROMPPER Universiteit Utrecht m.f.a.trompper@students.uu.nl Abstract Text-to-phoneme (T2P) mapping is a necessary step in any speech synthesis
More informationLinking the Ohio State Assessments to NWEA MAP Growth Tests *
Linking the Ohio State Assessments to NWEA MAP Growth Tests * *As of June 2017 Measures of Academic Progress (MAP ) is known as MAP Growth. August 2016 Introduction Northwest Evaluation Association (NWEA
More information9779 PRINCIPAL COURSE FRENCH
CAMBRIDGE INTERNATIONAL EXAMINATIONS Pre-U Certificate MARK SCHEME for the May/June 2014 series 9779 PRINCIPAL COURSE FRENCH 9779/03 Paper 1 (Writing and Usage), maximum raw mark 60 This mark scheme is
More information(Sub)Gradient Descent
(Sub)Gradient Descent CMSC 422 MARINE CARPUAT marine@cs.umd.edu Figures credit: Piyush Rai Logistics Midterm is on Thursday 3/24 during class time closed book/internet/etc, one page of notes. will include
More informationPractical Research. Planning and Design. Paul D. Leedy. Jeanne Ellis Ormrod. Upper Saddle River, New Jersey Columbus, Ohio
SUB Gfittingen 213 789 981 2001 B 865 Practical Research Planning and Design Paul D. Leedy The American University, Emeritus Jeanne Ellis Ormrod University of New Hampshire Upper Saddle River, New Jersey
More informationFUZZY EXPERT. Dr. Kasim M. Al-Aubidy. Philadelphia University. Computer Eng. Dept February 2002 University of Damascus-Syria
FUZZY EXPERT SYSTEMS 16-18 18 February 2002 University of Damascus-Syria Dr. Kasim M. Al-Aubidy Computer Eng. Dept. Philadelphia University What is Expert Systems? ES are computer programs that emulate
More informationRule Learning With Negation: Issues Regarding Effectiveness
Rule Learning With Negation: Issues Regarding Effectiveness S. Chua, F. Coenen, G. Malcolm University of Liverpool Department of Computer Science, Ashton Building, Ashton Street, L69 3BX Liverpool, United
More informationCourse Outline. Course Grading. Where to go for help. Academic Integrity. EE-589 Introduction to Neural Networks NN 1 EE
EE-589 Introduction to Neural Assistant Prof. Dr. Turgay IBRIKCI Room # 305 (322) 338 6868 / 139 Wensdays 9:00-12:00 Course Outline The course is divided in two parts: theory and practice. 1. Theory covers
More informationName of Course: French 1 Middle School. Grade Level(s): 7 and 8 (half each) Unit 1
Name of Course: French 1 Middle School Grade Level(s): 7 and 8 (half each) Unit 1 Estimated Instructional Time: 15 classes PA Academic Standards: Communication: Communicate in Languages Other Than English
More informationUsing Web Searches on Important Words to Create Background Sets for LSI Classification
Using Web Searches on Important Words to Create Background Sets for LSI Classification Sarah Zelikovitz and Marina Kogan College of Staten Island of CUNY 2800 Victory Blvd Staten Island, NY 11314 Abstract
More information1. REFLEXES: Ask questions about coughing, swallowing, of water as fast as possible (note! Not suitable for all
Human Communication Science Chandler House, 2 Wakefield Street London WC1N 1PF http://www.hcs.ucl.ac.uk/ ACOUSTICS OF SPEECH INTELLIGIBILITY IN DYSARTHRIA EUROPEAN MASTER S S IN CLINICAL LINGUISTICS UNIVERSITY
More informationUsing focal point learning to improve human machine tacit coordination
DOI 10.1007/s10458-010-9126-5 Using focal point learning to improve human machine tacit coordination InonZuckerman SaritKraus Jeffrey S. Rosenschein The Author(s) 2010 Abstract We consider an automated
More informationMulti-label classification via multi-target regression on data streams
Mach Learn (2017) 106:745 770 DOI 10.1007/s10994-016-5613-5 Multi-label classification via multi-target regression on data streams Aljaž Osojnik 1,2 Panče Panov 1 Sašo Džeroski 1,2,3 Received: 26 April
More informationForget catastrophic forgetting: AI that learns after deployment
Forget catastrophic forgetting: AI that learns after deployment Anatoly Gorshechnikov CTO, Neurala 1 Neurala at a glance Programming neural networks on GPUs since circa 2 B.C. Founded in 2006 expecting
More informationActivity Recognition from Accelerometer Data
Activity Recognition from Accelerometer Data Nishkam Ravi and Nikhil Dandekar and Preetham Mysore and Michael L. Littman Department of Computer Science Rutgers University Piscataway, NJ 08854 {nravi,nikhild,preetham,mlittman}@cs.rutgers.edu
More informationAustralian Journal of Basic and Applied Sciences
AENSI Journals Australian Journal of Basic and Applied Sciences ISSN:1991-8178 Journal home page: www.ajbasweb.com Feature Selection Technique Using Principal Component Analysis For Improving Fuzzy C-Mean
More informationModel Ensemble for Click Prediction in Bing Search Ads
Model Ensemble for Click Prediction in Bing Search Ads Xiaoliang Ling Microsoft Bing xiaoling@microsoft.com Hucheng Zhou Microsoft Research huzho@microsoft.com Weiwei Deng Microsoft Bing dedeng@microsoft.com
More informationApproaches for analyzing tutor's role in a networked inquiry discourse
Lakkala, M., Muukkonen, H., Ilomäki, L., Lallimo, J., Niemivirta, M. & Hakkarainen, K. (2001) Approaches for analysing tutor's role in a networked inquiry discourse. In P. Dillenbourg, A. Eurelings., &
More informationExperiment Databases: Towards an Improved Experimental Methodology in Machine Learning
Experiment Databases: Towards an Improved Experimental Methodology in Machine Learning Hendrik Blockeel and Joaquin Vanschoren Computer Science Dept., K.U.Leuven, Celestijnenlaan 200A, 3001 Leuven, Belgium
More informationThe Lexicalization of Acronyms in English: The Case of Third Year E.F.L Students, Mentouri University- Constantine
The Lexicalization of Acronyms in English: The Case of Third Year E.F.L Students, Mentouri University- Constantine Yamina BENNANE Université Frères Mentouri. Constantine 1. Algérie Abstract: The present
More informationMachine Learning from Garden Path Sentences: The Application of Computational Linguistics
Machine Learning from Garden Path Sentences: The Application of Computational Linguistics http://dx.doi.org/10.3991/ijet.v9i6.4109 J.L. Du 1, P.F. Yu 1 and M.L. Li 2 1 Guangdong University of Foreign Studies,
More informationWhat Different Kinds of Stratification Can Reveal about the Generalizability of Data-Mined Skill Assessment Models
What Different Kinds of Stratification Can Reveal about the Generalizability of Data-Mined Skill Assessment Models Michael A. Sao Pedro Worcester Polytechnic Institute 100 Institute Rd. Worcester, MA 01609
More informationMETHODS FOR EXTRACTING AND CLASSIFYING PAIRS OF COGNATES AND FALSE FRIENDS
METHODS FOR EXTRACTING AND CLASSIFYING PAIRS OF COGNATES AND FALSE FRIENDS Ruslan Mitkov (R.Mitkov@wlv.ac.uk) University of Wolverhampton ViktorPekar (v.pekar@wlv.ac.uk) University of Wolverhampton Dimitar
More informationEvidence for Reliability, Validity and Learning Effectiveness
PEARSON EDUCATION Evidence for Reliability, Validity and Learning Effectiveness Introduction Pearson Knowledge Technologies has conducted a large number and wide variety of reliability and validity studies
More informationStopping rules for sequential trials in high-dimensional data
Stopping rules for sequential trials in high-dimensional data Sonja Zehetmayer, Alexandra Graf, and Martin Posch Center for Medical Statistics, Informatics and Intelligent Systems Medical University of
More informationNotes on The Sciences of the Artificial Adapted from a shorter document written for course (Deciding What to Design) 1
Notes on The Sciences of the Artificial Adapted from a shorter document written for course 17-652 (Deciding What to Design) 1 Ali Almossawi December 29, 2005 1 Introduction The Sciences of the Artificial
More informationQuestion 1 Does the concept of "part-time study" exist in your University and, if yes, how is it put into practice, is it possible in every Faculty?
Name of the University Country Univerza v Ljubljani Slovenia Tallin University of Technology (TUT) Estonia Question 1 Does the concept of "part-time study" exist in your University and, if yes, how is
More informationHow to Judge the Quality of an Objective Classroom Test
How to Judge the Quality of an Objective Classroom Test Technical Bulletin #6 Evaluation and Examination Service The University of Iowa (319) 335-0356 HOW TO JUDGE THE QUALITY OF AN OBJECTIVE CLASSROOM
More informationRule Learning with Negation: Issues Regarding Effectiveness
Rule Learning with Negation: Issues Regarding Effectiveness Stephanie Chua, Frans Coenen, and Grant Malcolm University of Liverpool Department of Computer Science, Ashton Building, Ashton Street, L69 3BX
More informationMachine Learning and Development Policy
Machine Learning and Development Policy Sendhil Mullainathan (joint papers with Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, Ziad Obermeyer) Magic? Hard not to be wowed But what makes
More informationLinking Task: Identifying authors and book titles in verbose queries
Linking Task: Identifying authors and book titles in verbose queries Anaïs Ollagnier, Sébastien Fournier, and Patrice Bellot Aix-Marseille University, CNRS, ENSAM, University of Toulon, LSIS UMR 7296,
More informationCAVE LANGUAGES KS2 SCHEME OF WORK LANGUAGE OVERVIEW. YEAR 3 Stage 1 Lessons 1-30
CAVE LANGUAGES KS2 SCHEME OF WORK LANGUAGE OVERVIEW AUTUMN TERM Stage 1 Lessons 1-8 Christmas lessons 1-4 LANGUAGE CONTENT Greetings Classroom commands listening/speaking Feelings question/answer 5 colours-recognition
More informationTwitter Sentiment Classification on Sanders Data using Hybrid Approach
IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 17, Issue 4, Ver. I (July Aug. 2015), PP 118-123 www.iosrjournals.org Twitter Sentiment Classification on Sanders
More informationCalibration of Confidence Measures in Speech Recognition
Submitted to IEEE Trans on Audio, Speech, and Language, July 2010 1 Calibration of Confidence Measures in Speech Recognition Dong Yu, Senior Member, IEEE, Jinyu Li, Member, IEEE, Li Deng, Fellow, IEEE
More informationMemory-based grammatical error correction
Memory-based grammatical error correction Antal van den Bosch Peter Berck Radboud University Nijmegen Tilburg University P.O. Box 9103 P.O. Box 90153 NL-6500 HD Nijmegen, The Netherlands NL-5000 LE Tilburg,
More informationComputerized Adaptive Psychological Testing A Personalisation Perspective
Psychology and the internet: An European Perspective Computerized Adaptive Psychological Testing A Personalisation Perspective Mykola Pechenizkiy mpechen@cc.jyu.fi Introduction Mixed Model of IRT and ES
More informationComparison of EM and Two-Step Cluster Method for Mixed Data: An Application
International Journal of Medical Science and Clinical Inventions 4(3): 2768-2773, 2017 DOI:10.18535/ijmsci/ v4i3.8 ICV 2015: 52.82 e-issn: 2348-991X, p-issn: 2454-9576 2017, IJMSCI Research Article Comparison
More informationTeachers response to unexplained answers
Teachers response to unexplained answers Ove Gunnar Drageset To cite this version: Ove Gunnar Drageset. Teachers response to unexplained answers. Konrad Krainer; Naďa Vondrová. CERME 9 - Ninth Congress
More informationAgenda Montreal, Quebec October 17 19
Wednesday, October 17 6:30 8:00 a.m. 8:00 9:45 a.m. 9:45 10:00 a.m. Break Agenda Montreal, Quebec October 17 19 Registration 10:00 11:30 a.m. Breakouts Continental breakfast 11:30 a.m. 1:00 p.m. Lunch
More informationarxiv: v1 [cs.lg] 15 Jun 2015
Dual Memory Architectures for Fast Deep Learning of Stream Data via an Online-Incremental-Transfer Strategy arxiv:1506.04477v1 [cs.lg] 15 Jun 2015 Sang-Woo Lee Min-Oh Heo School of Computer Science and
More informationAn Empirical and Computational Test of Linguistic Relativity
An Empirical and Computational Test of Linguistic Relativity Kathleen M. Eberhard* (eberhard.1@nd.edu) Matthias Scheutz** (mscheutz@cse.nd.edu) Michael Heilman** (mheilman@nd.edu) *Department of Psychology,
More informationUsing computational modeling in language acquisition research
Chapter 8 Using computational modeling in language acquisition research Lisa Pearl 1. Introduction Language acquisition research is often concerned with questions of what, when, and how what children know,
More informationWord Segmentation of Off-line Handwritten Documents
Word Segmentation of Off-line Handwritten Documents Chen Huang and Sargur N. Srihari {chuang5, srihari}@cedar.buffalo.edu Center of Excellence for Document Analysis and Recognition (CEDAR), Department
More informationA Version Space Approach to Learning Context-free Grammars
Machine Learning 2: 39~74, 1987 1987 Kluwer Academic Publishers, Boston - Manufactured in The Netherlands A Version Space Approach to Learning Context-free Grammars KURT VANLEHN (VANLEHN@A.PSY.CMU.EDU)
More informationLarge-Scale Web Page Classification. Sathi T Marath. Submitted in partial fulfilment of the requirements. for the degree of Doctor of Philosophy
Large-Scale Web Page Classification by Sathi T Marath Submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy at Dalhousie University Halifax, Nova Scotia November 2010
More informationSTAT 220 Midterm Exam, Friday, Feb. 24
STAT 220 Midterm Exam, Friday, Feb. 24 Name Please show all of your work on the exam itself. If you need more space, use the back of the page. Remember that partial credit will be awarded when appropriate.
More informationHuman Emotion Recognition From Speech
RESEARCH ARTICLE OPEN ACCESS Human Emotion Recognition From Speech Miss. Aparna P. Wanare*, Prof. Shankar N. Dandare *(Department of Electronics & Telecommunication Engineering, Sant Gadge Baba Amravati
More informationB. How to write a research paper
From: Nikolaus Correll. "Introduction to Autonomous Robots", ISBN 1493773070, CC-ND 3.0 B. How to write a research paper The final deliverable of a robotics class often is a write-up on a research project,
More informationExperiments with SMS Translation and Stochastic Gradient Descent in Spanish Text Author Profiling
Experiments with SMS Translation and Stochastic Gradient Descent in Spanish Text Author Profiling Notebook for PAN at CLEF 2013 Andrés Alfonso Caurcel Díaz 1 and José María Gómez Hidalgo 2 1 Universidad
More informationCooperative evolutive concept learning: an empirical study
Cooperative evolutive concept learning: an empirical study Filippo Neri University of Piemonte Orientale Dipartimento di Scienze e Tecnologie Avanzate Piazza Ambrosoli 5, 15100 Alessandria AL, Italy Abstract
More information