Practical considerations about the implementation of some Machine Learning LGD models in companies
|
|
- Arleen Harris
- 6 years ago
- Views:
Transcription
1 Practical considerations about the implementation of some Machine Learning LGD models in companies September 15 th 2017 Louvain-la-Neuve Sébastien de Valeriola Please read the important disclaimer at the end of this presentation
2 Storytelling In this talk, I would like to tell you a tale: 2
3 Storytelling In this talk, I would like to tell you a tale: in a not so far away land, 3
4 Storytelling In this talk, I would like to tell you a tale: in a not so far away land, 1. An actuary was asked by his employer, a mid-sized European bank, to improve their current Loss Given Default (LGD) model, 4
5 Credit Risk Assessment under Basel II/III Under Basel II/III*, banks may calculate credit risk using internal ratings based approach (IRB). For corporate credit, the directives distinguish between two possible alternatives: o Foundation IRB: estimating ratings scales only based on probabilities of default and adequately allocating each of the corporate loans to a specific rating class, o Advanced IRB: ratings scale is established considering not only the PD s but also all other credit parameters including Loss-Given-Default (LGD), Maturity adjustments, EAD with CFs, etc. Expected Loss (EL) = Probability of Default (PD) X Loss Given Default (LGD) X Exposure at Default (EAD) The likelihood of a borrower being unable to repay The fraction of exposure at default that is lost in the case of default The exposure at risk in the case of default 5 (*) i.e. CRR & CRD IV
6 Storytelling In this talk, I would like to tell you a tale: in a not so far away land, 1. An actuary was asked by his employer, a mid-sized European bank, to improve their current Loss Given Default (LGD) model, 2. Our actuary then began to explore the LGD models, and eventually opted for a Machine Learning model, 6
7 LGD models From the modelling point of view, LGD is the «poor little brother» of PD, as literature about LGD models is rather rare when compared to the one about PD models. In particular, no «standard benchmark» exists for LGD models. One property of LGD explains maybe at least partially this poverty: the observed distribution is generally bimodal. 7
8 Choosing a Machine Learning model for its predictive ability The deviance/error scores of the tested models clearly suggest to choose a Machine Learning model. 8
9 Storytelling In this talk, I would like to tell you a tale: in a not so far away land, 1. An actuary was asked by his employer, a mid-sized European bank, to improve their current Loss Given Default (LGD) model, 2. Our actuary then began to explore the LGD models, and eventually opted for a Machine Learning model, 3. Once all the technical stuff was complete, an important question arose: how to convince management (among others ) to effectively put the chosen model into production? 9
10 Understanding the machine learning methods The «quants» of the company are generally able to understand the technical details of the chosen model, and therefore trust its outputs, as they are convinced b by cross-validation, error measures and assesment plots. However, in order to be accepted and effectively put into production, the model and its outputs should also be understood and trusted by a large set of other protagonists, which are not necessarily «quantitative people»: decision-takers «above» the technical team (e.g. management/comitee/board), which take decision based on the output of the model, other departments of the company (e.g. Pricing department), the validators and regulators, Moreover, the European Parliament has adopted the General Data Protection Regulation, which will be in force in Among the new rules introduced by this legal text is the creation of a «right to explanation»: [... a data subject has the right to] meaningful information about the logic involved. (Article 13) This sentence is not cristal clear (for details, see GOODMAN and FLAXMAN (2016)), but 10
11 The problem When we say «understanding» a model, we mean «understanding how predictions are made» (for example, it is not necessary for the management to fully understand the calibration process) In the case of regression trees, understanding how the model predicts LGD values for new data points is not a problem, as it is very intuitive. However, some intepretation may still be required, as this model (and its output) is very different from the distribution-based models which are generally in production. In the case of more complex methods such as Bagging and Random forests, even understanding how the model predicts LGD values for new data points is rather difficult. Things may be even worse for Gradient Boosting Machine, Support Vector Machine and Neural Networks In the remainging part of this talk, I will give some pointers /ideas to overcome this difficulty. 11
12 Ideas to understand/interpret Machine Learning techniques 1. Implement ML techniques on top of traditional, in-production, models: a. Use ML technique to perform variables selection, b. Use ML technique to segment data before applying traditional model. 2. Put some parameters in the hands of the user: a. Handle the complexity parameter of the pruning process, b. Show and handle the trees behind an ensemble methods such as Random Forest. 3. Mimic properties of traditional models: a. Force monotonicity in a ML technique, b. Build confidance intervals in order to stick to traditional models output. 4. Local interpretation of the ML methods: a. Explain local predictions as linear combinations, b. Locally Interpretable Model-agnostic Explanations. 5. Global interpretation of the ML methods: a. Simplified Tree Ensemble Learner. 12
13 Variable selection First solution: stick to existing (e.g. in production) model and use machine learning techniques as guiding tools. For example, one could use a random forest to explore the data and decide which features we will select in the model (using the importance score produced by the Random Forest). In particular, this could be very useful to detect the needed interactions between variables, as it is generally difficult to select cross terms in traditional models. Example of such a process: 1. Fit GLM to the data (without any interaction) and extract the residuals, 2. Train a regression tree on these residuals in order to see which couple of variable comes first in the split, 3. Assess the significance of this interaction, 4. If it is significant, then fit once again the GLM, but this time specifying the interaction that was detected. 13
14 Segmentation Another possibility is to use the output of some ML technique to segment the data before applying traditional models on each segment. 14
15 Ideas to understand/interpret Machine Learning techniques 1. Implement ML techniques on top of traditional, in-production, models: a. Use ML technique to perform variables selection, b. Use ML technique to segment data before applying traditional model. 2. Put some parameters in the hands of the user: a. Handle the complexity parameter of the pruning process, b. Show and handle the trees behind an ensemble methods such as Random Forest. 3. Mimic properties of traditional models: a. Force monotonicity in a ML technique, b. Build confidance intervals in order to stick to traditional models output. 4. Local interpretation of the ML methods: a. Explain local predictions as linear combinations, b. Locally Interpretable Model-agnostic Explanations. 5. Global interpretation of the ML methods: a. Simplified Tree Ensemble Learner. 15
16 Giving the user the hand on some parameters: the idea A nice way to give users insights about how the ML techniques work and thus give them reasons to trust them is to let them «play» with some parameters. For example, 1. We can implement the pruning process in a Shiny tool and let the user set the complexity parameter by himself, 2. We can let the user explore the forest of regression trees which is the output of a Random Forest, select subsets of them and compute predictions, etc. 16
17 Giving the user the hand on some parameters 17
18 Giving the user the hand on some parameters 18
19 Ideas to understand/interpret Machine Learning techniques 1. Implement ML techniques on top of traditional, in-production, models: a. Use ML technique to perform variables selection, b. Use ML technique to segment data before applying traditional model. 2. Put some parameters in the hands of the user: a. Handle the complexity parameter of the pruning process, b. Show and handle the trees behind an ensemble methods such as Random Forest. 3. Mimic properties of traditional models: a. Force monotonicity in a ML technique, b. Build confidance intervals in order to stick to traditional models output. 4. Local interpretation of the ML methods: a. Explain local predictions as linear combinations, b. Locally Interpretable Model-agnostic Explanations. 5. Global interpretation of the ML methods: a. Simplified Tree Ensemble Learner. 19
20 Force monotonocity One of the hard-to-understand features of the ML techniques is the non-monotonicity. Let us for example consider the case of the business size of the debtor: Globally, this variable has a positive effect on the LGD, i.e. the corresponding parameter in a traditional model (e.g. coefficient β in a LM) would be significantly positive, However, it is possible that the effect of this variable is negative on a subset of the dataset, and that a regression tree would detect this. In order to «mimic» the monotonicity of the traditional models, it is possible to force monotonicity of ML techniques. In the case of regression trees (and other tree methods), we can constrain the calibration process: every split based on the chosen variable will be such that the prediction in, say, the child node situated on the left side of the split is larger, the prediction in the child node situated on the right side of the split is smaller. Some R package allow imposing such constraints, as XGBoost (available on github). 20
21 Computing confidence intervals for random forests As the output of most of traditional models involve confidence intervals, practitioners are sometimes disappointed that ML methods do not produce such output. While it is true that one very rarely sees it, it is possible to build such confidence intervals. For example, EFRON, HASTIE and WAGER (2014) use jackknife and infinitesimal jackknife to do so: jackknife: systematically recompute the statistic estimate, leaving out one observation at a time from the sample set, infinitesimal jackknife: instead of leaving one observation out, introduce weights and affect a slightly smaller weight to one observation (i.e. weights are equal for each observations except for this particular one). The R package randomforestci (available on github) implements this method. 21
22 Computing confidence intervals for random forests 22
23 Ideas to understand/interpret Machine Learning techniques 1. Implement ML techniques on top of traditional, in-production, models: a. Use ML technique to perform variables selection, b. Use ML technique to segment data before applying traditional model. 2. Put some parameters in the hands of the user: a. Handle the complexity parameter of the pruning process, b. Show and handle the trees behind an ensemble methods such as Random Forest. 3. Mimic properties of traditional models: a. Force monotonicity in a ML technique, b. Build confidance intervals in order to stick to traditional models output. 4. Local interpretation of the ML methods: a. Explain local predictions as linear combinations, b. Locally Interpretable Model-agnostic Explanations. 5. Global interpretation of the ML methods: a. Simplified Tree Ensemble Learner. 23
24 Explain predictions as a linear combination Local interpretations focus on a limited set of data points, for which they give explanations which are only locally correct. In the case of a regression tree, it is easy to give an interpretation as a «traditional model» (weither it is additive or multiplicative) by following the branches from the root to the chosen leaf. The resulting explanation is of course only valid for one leaf of the tree. This rather naive process can be extended to other treebased methods. Linear explanation of the prediction for the sixth leaf: LGD = 0.43 (general) 0.22 (loan or credit card) (closed loan) 0.10 (large recoverable amount) (non revolving loan) =
25 Locally Interpretable Model-agnostic Explanations The Locally Interpretable Model-agnostic Explanations (LIME) have been introduced by RIBEIRO, SINGH and GUESTRIN (2016). This method is model-agnostic because it considers the explained model as a black box, so that it can explain all types of models. It gets a trade-off between interpretability and fidelity. The process is the following: 1. Choose a limited set of features (the most important ones), 2. Sample points around the point we want to explain, 3. Use the black box model to obtain predictions for these neighbour points, 4. Fit an interpretable model on these predictions using weights ~ distance to the point we want to explain. This interpretable model could be a e.g. a linear model or a regression tree. The R package lime (available on github) implements this method. 25
26 Locally Interpretable Model-agnostic Explanations Explaining a default corresponding to a LGD of using the four most important features of the Random Forest model: 26
27 Ideas to understand/interpret Machine Learning techniques 1. Implement ML techniques on top of traditional, in-production, models: a. Use ML technique to perform variables selection, b. Use ML technique to segment data before applying traditional model. 2. Put some parameters in the hands of the user: a. Handle the complexity parameter of the pruning process, b. Show and handle the trees behind an ensemble methods such as Random Forest. 3. Mimic properties of traditional models: a. Force monotonicity in a ML technique, b. Build confidance intervals in order to stick to traditional models output. 4. Local interpretation of the ML methods: a. Explain local predictions as linear combinations, b. Locally Interpretable Model-agnostic Explanations. 5. Global interpretation of the ML methods: a. Simplified Tree Ensemble Learner. 27
28 Simplified Tree Ensemble Learner Simplified Tree Ensemble Learner (STEL) can be applied to any ensemble method, and give a global intepretation of the model. The process of the InTrees method, presented in DENG (2014), is the following: 1. Transform the output of the ensemble method into a large set of rules (e.g. «Y 1 > 0 & Y 2 = yes predict 0.29») whose length can be rather large, 2. Compute rules metrics, such as importance, predictive ability, etc. 3. Prune the rules (i.e. reduce their length), 4. Select a set of efficient and non-redundant rules (building a new database and fitting a regularized random forest on it). The output of this process is a STEL, which consists in a list of rules ordered by priority. To obtain a prediction, the rules are applied, from the top to the bottom, to a new data point, until a rule condition is satisfied by the point. The prediction value corresponding to the rule then becomes the prediction for the new point. The prediction quality is of course worse that the original ensemble method, but can be rather close to it. R package InTrees (Interpretable Trees) is an implementation of this process. 28
29 Simplified Tree Ensemble Learner From a 100 trees random forest, R extract 5,292 rules. It then prunes the rules, as one can see in the length distribution: length before after n Length Frequence Error Condition Prediction Importance 1 2 0,45 0, ,11 0, ,12 0, ,03 0, ,04 0, ,06 0, ,29 0, ,27 0,05 defoffbalsheet <= & loanstatus %in% c('closed') loanrevolving %in% c('yes') & loanrecovamount > defexposureat > & loanrecovamount <= defoffbalsheet > & loanrecovamount > loannominalamount <= 9700 & loanrecovamount > loannominalamount <= 7250 & loanrecovamount > defexposureat > & loanrecovamount <= defexposureat <= & loanrecovamount > ,04 1,00 0,07 0,05 0,88 0,05 0,10 0,02 0,01 0,02 0,02 0,02 0,92 0,02 0,09 0, ,24 0,04 loanrecovamount > ,07 0, ,30 0,03 loanmaturitydate <= & loanrecovamount > ,05 0,01 29
30 End of the story? In this talk, I would like to tell you a tale: in a not so far away land, 1. An actuary was asked by his employer, a mid-sized European bank, to improve their current Loss Given Default (LGD) model, 2. Our actuary then began to explore the LGD models, and eventually opted for a Machine Learning model, 3. Once all the technical stuff was complete, an important question arose: how to convince management (among others ) to effectively put the chosen model into production? 4. Happy end? 30
31 Thank you for your attention! Some references: GOODMAN, B., and FLAXMAN, S., European Union regulations on algorithmic decision-making and a right to explanation, in ArXiv e-prints (2016). EFRON, B., HASTIE, T. and WAGER, S., Confidence Intervals for Random Forests: The Jackknife and the Infinitesimal Jackknife, in Journal of Machine Learning Research 15 (2014), p RIBEIRO, M. T., SINGH, S., and GUESTRIN, C., Why Should I Trust You? Explaining the Predictions of Any Classifier, in KDD '16 Proceedings of the 22nd ACM SIGK4D International Conference on Knowledge Discovery and Data Mining (2016), p DENG, H., Interpreting Tree Ensembles with intrees, technical report (2014), online: 31
Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition
Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition Todd Holloway Two Lecture Series for B551 November 20 & 27, 2007 Indiana University Outline Introduction Bias and
More informationMachine Learning and Data Mining. Ensembles of Learners. Prof. Alexander Ihler
Machine Learning and Data Mining Ensembles of Learners Prof. Alexander Ihler Ensemble methods Why learn one classifier when you can learn many? Ensemble: combine many predictors (Weighted) combina
More informationLearning From the Past with Experiment Databases
Learning From the Past with Experiment Databases Joaquin Vanschoren 1, Bernhard Pfahringer 2, and Geoff Holmes 2 1 Computer Science Dept., K.U.Leuven, Leuven, Belgium 2 Computer Science Dept., University
More informationPython Machine Learning
Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled
More informationRule Learning With Negation: Issues Regarding Effectiveness
Rule Learning With Negation: Issues Regarding Effectiveness S. Chua, F. Coenen, G. Malcolm University of Liverpool Department of Computer Science, Ashton Building, Ashton Street, L69 3BX Liverpool, United
More informationLecture 1: Machine Learning Basics
1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3
More informationAssignment 1: Predicting Amazon Review Ratings
Assignment 1: Predicting Amazon Review Ratings 1 Dataset Analysis Richard Park r2park@acsmail.ucsd.edu February 23, 2015 The dataset selected for this assignment comes from the set of Amazon reviews for
More informationCS Machine Learning
CS 478 - Machine Learning Projects Data Representation Basic testing and evaluation schemes CS 478 Data and Testing 1 Programming Issues l Program in any platform you want l Realize that you will be doing
More informationRule Learning with Negation: Issues Regarding Effectiveness
Rule Learning with Negation: Issues Regarding Effectiveness Stephanie Chua, Frans Coenen, and Grant Malcolm University of Liverpool Department of Computer Science, Ashton Building, Ashton Street, L69 3BX
More informationProbability and Statistics Curriculum Pacing Guide
Unit 1 Terms PS.SPMJ.3 PS.SPMJ.5 Plan and conduct a survey to answer a statistical question. Recognize how the plan addresses sampling technique, randomization, measurement of experimental error and methods
More informationSpeech Emotion Recognition Using Support Vector Machine
Speech Emotion Recognition Using Support Vector Machine Yixiong Pan, Peipei Shen and Liping Shen Department of Computer Technology Shanghai JiaoTong University, Shanghai, China panyixiong@sjtu.edu.cn,
More informationModule 12. Machine Learning. Version 2 CSE IIT, Kharagpur
Module 12 Machine Learning 12.1 Instructional Objective The students should understand the concept of learning systems Students should learn about different aspects of a learning system Students should
More informationlearning collegiate assessment]
[ collegiate learning assessment] INSTITUTIONAL REPORT 2005 2006 Kalamazoo College council for aid to education 215 lexington avenue floor 21 new york new york 10016-6023 p 212.217.0700 f 212.661.9766
More informationModel Ensemble for Click Prediction in Bing Search Ads
Model Ensemble for Click Prediction in Bing Search Ads Xiaoliang Ling Microsoft Bing xiaoling@microsoft.com Hucheng Zhou Microsoft Research huzho@microsoft.com Weiwei Deng Microsoft Bing dedeng@microsoft.com
More information(Sub)Gradient Descent
(Sub)Gradient Descent CMSC 422 MARINE CARPUAT marine@cs.umd.edu Figures credit: Piyush Rai Logistics Midterm is on Thursday 3/24 during class time closed book/internet/etc, one page of notes. will include
More informationSTA 225: Introductory Statistics (CT)
Marshall University College of Science Mathematics Department STA 225: Introductory Statistics (CT) Course catalog description A critical thinking course in applied statistical reasoning covering basic
More informationCalibration of Confidence Measures in Speech Recognition
Submitted to IEEE Trans on Audio, Speech, and Language, July 2010 1 Calibration of Confidence Measures in Speech Recognition Dong Yu, Senior Member, IEEE, Jinyu Li, Member, IEEE, Li Deng, Fellow, IEEE
More informationADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY DOWNLOAD EBOOK : ADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY PDF
Read Online and Download Ebook ADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY DOWNLOAD EBOOK : ADVANCED MACHINE LEARNING WITH PYTHON BY JOHN HEARTY PDF Click link bellow and free register to download
More informationOn-Line Data Analytics
International Journal of Computer Applications in Engineering Sciences [VOL I, ISSUE III, SEPTEMBER 2011] [ISSN: 2231-4946] On-Line Data Analytics Yugandhar Vemulapalli #, Devarapalli Raghu *, Raja Jacob
More informationA Simple VQA Model with a Few Tricks and Image Features from Bottom-up Attention
A Simple VQA Model with a Few Tricks and Image Features from Bottom-up Attention Damien Teney 1, Peter Anderson 2*, David Golub 4*, Po-Sen Huang 3, Lei Zhang 3, Xiaodong He 3, Anton van den Hengel 1 1
More informationImpact of Cluster Validity Measures on Performance of Hybrid Models Based on K-means and Decision Trees
Impact of Cluster Validity Measures on Performance of Hybrid Models Based on K-means and Decision Trees Mariusz Łapczy ski 1 and Bartłomiej Jefma ski 2 1 The Chair of Market Analysis and Marketing Research,
More informationCertified Six Sigma Professionals International Certification Courses in Six Sigma Green Belt
Certification Singapore Institute Certified Six Sigma Professionals Certification Courses in Six Sigma Green Belt ly Licensed Course for Process Improvement/ Assurance Managers and Engineers Leading the
More informationLinking Task: Identifying authors and book titles in verbose queries
Linking Task: Identifying authors and book titles in verbose queries Anaïs Ollagnier, Sébastien Fournier, and Patrice Bellot Aix-Marseille University, CNRS, ENSAM, University of Toulon, LSIS UMR 7296,
More informationExperiments with SMS Translation and Stochastic Gradient Descent in Spanish Text Author Profiling
Experiments with SMS Translation and Stochastic Gradient Descent in Spanish Text Author Profiling Notebook for PAN at CLEF 2013 Andrés Alfonso Caurcel Díaz 1 and José María Gómez Hidalgo 2 1 Universidad
More informationUniversidade do Minho Escola de Engenharia
Universidade do Minho Escola de Engenharia Universidade do Minho Escola de Engenharia Dissertação de Mestrado Knowledge Discovery is the nontrivial extraction of implicit, previously unknown, and potentially
More informationWord Segmentation of Off-line Handwritten Documents
Word Segmentation of Off-line Handwritten Documents Chen Huang and Sargur N. Srihari {chuang5, srihari}@cedar.buffalo.edu Center of Excellence for Document Analysis and Recognition (CEDAR), Department
More informationUnsupervised Learning of Word Semantic Embedding using the Deep Structured Semantic Model
Unsupervised Learning of Word Semantic Embedding using the Deep Structured Semantic Model Xinying Song, Xiaodong He, Jianfeng Gao, Li Deng Microsoft Research, One Microsoft Way, Redmond, WA 98052, U.S.A.
More informationCS 446: Machine Learning
CS 446: Machine Learning Introduction to LBJava: a Learning Based Programming Language Writing classifiers Christos Christodoulopoulos Parisa Kordjamshidi Motivation 2 Motivation You still have not learnt
More informationTwitter Sentiment Classification on Sanders Data using Hybrid Approach
IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 17, Issue 4, Ver. I (July Aug. 2015), PP 118-123 www.iosrjournals.org Twitter Sentiment Classification on Sanders
More informationAnalysis of Enzyme Kinetic Data
Analysis of Enzyme Kinetic Data To Marilú Analysis of Enzyme Kinetic Data ATHEL CORNISH-BOWDEN Directeur de Recherche Émérite, Centre National de la Recherche Scientifique, Marseilles OXFORD UNIVERSITY
More information*Net Perceptions, Inc West 78th Street Suite 300 Minneapolis, MN
From: AAAI Technical Report WS-98-08. Compilation copyright 1998, AAAI (www.aaai.org). All rights reserved. Recommender Systems: A GroupLens Perspective Joseph A. Konstan *t, John Riedl *t, AI Borchers,
More informationEdexcel GCSE. Statistics 1389 Paper 1H. June Mark Scheme. Statistics Edexcel GCSE
Edexcel GCSE Statistics 1389 Paper 1H June 2007 Mark Scheme Edexcel GCSE Statistics 1389 NOTES ON MARKING PRINCIPLES 1 Types of mark M marks: method marks A marks: accuracy marks B marks: unconditional
More informationCSL465/603 - Machine Learning
CSL465/603 - Machine Learning Fall 2016 Narayanan C Krishnan ckn@iitrpr.ac.in Introduction CSL465/603 - Machine Learning 1 Administrative Trivia Course Structure 3-0-2 Lecture Timings Monday 9.55-10.45am
More informationWE GAVE A LAWYER BASIC MATH SKILLS, AND YOU WON T BELIEVE WHAT HAPPENED NEXT
WE GAVE A LAWYER BASIC MATH SKILLS, AND YOU WON T BELIEVE WHAT HAPPENED NEXT PRACTICAL APPLICATIONS OF RANDOM SAMPLING IN ediscovery By Matthew Verga, J.D. INTRODUCTION Anyone who spends ample time working
More informationLearning Structural Correspondences Across Different Linguistic Domains with Synchronous Neural Language Models
Learning Structural Correspondences Across Different Linguistic Domains with Synchronous Neural Language Models Stephan Gouws and GJ van Rooyen MIH Medialab, Stellenbosch University SOUTH AFRICA {stephan,gvrooyen}@ml.sun.ac.za
More informationA GENERIC SPLIT PROCESS MODEL FOR ASSET MANAGEMENT DECISION-MAKING
A GENERIC SPLIT PROCESS MODEL FOR ASSET MANAGEMENT DECISION-MAKING Yong Sun, a * Colin Fidge b and Lin Ma a a CRC for Integrated Engineering Asset Management, School of Engineering Systems, Queensland
More informationThe Oregon Literacy Framework of September 2009 as it Applies to grades K-3
The Oregon Literacy Framework of September 2009 as it Applies to grades K-3 The State Board adopted the Oregon K-12 Literacy Framework (December 2009) as guidance for the State, districts, and schools
More informationMining Association Rules in Student s Assessment Data
www.ijcsi.org 211 Mining Association Rules in Student s Assessment Data Dr. Varun Kumar 1, Anupama Chadha 2 1 Department of Computer Science and Engineering, MVN University Palwal, Haryana, India 2 Anupama
More informationarxiv: v1 [cs.lg] 15 Jun 2015
Dual Memory Architectures for Fast Deep Learning of Stream Data via an Online-Incremental-Transfer Strategy arxiv:1506.04477v1 [cs.lg] 15 Jun 2015 Sang-Woo Lee Min-Oh Heo School of Computer Science and
More informationMulti-label classification via multi-target regression on data streams
Mach Learn (2017) 106:745 770 DOI 10.1007/s10994-016-5613-5 Multi-label classification via multi-target regression on data streams Aljaž Osojnik 1,2 Panče Panov 1 Sašo Džeroski 1,2,3 Received: 26 April
More informationGenerative models and adversarial training
Day 4 Lecture 1 Generative models and adversarial training Kevin McGuinness kevin.mcguinness@dcu.ie Research Fellow Insight Centre for Data Analytics Dublin City University What is a generative model?
More informationSoftware Maintenance
1 What is Software Maintenance? Software Maintenance is a very broad activity that includes error corrections, enhancements of capabilities, deletion of obsolete capabilities, and optimization. 2 Categories
More informationThe 9 th International Scientific Conference elearning and software for Education Bucharest, April 25-26, / X
The 9 th International Scientific Conference elearning and software for Education Bucharest, April 25-26, 2013 10.12753/2066-026X-13-154 DATA MINING SOLUTIONS FOR DETERMINING STUDENT'S PROFILE Adela BÂRA,
More informationBusiness Analytics and Information Tech COURSE NUMBER: 33:136:494 COURSE TITLE: Data Mining and Business Intelligence
Business Analytics and Information Tech COURSE NUMBER: 33:136:494 COURSE TITLE: Data Mining and Business Intelligence COURSE DESCRIPTION This course presents computing tools and concepts for all stages
More informationExperiment Databases: Towards an Improved Experimental Methodology in Machine Learning
Experiment Databases: Towards an Improved Experimental Methodology in Machine Learning Hendrik Blockeel and Joaquin Vanschoren Computer Science Dept., K.U.Leuven, Celestijnenlaan 200A, 3001 Leuven, Belgium
More informationFRAMEWORK FOR IDENTIFYING THE MOST LIKELY SUCCESSFUL UNDERPRIVILEGED TERTIARY STUDY BURSARY APPLICANTS
South African Journal of Industrial Engineering August 2017 Vol 28(2), pp 59-77 FRAMEWORK FOR IDENTIFYING THE MOST LIKELY SUCCESSFUL UNDERPRIVILEGED TERTIARY STUDY BURSARY APPLICANTS R. Steynberg 1 * #,
More informationLearning to Rank with Selection Bias in Personal Search
Learning to Rank with Selection Bias in Personal Search Xuanhui Wang, Michael Bendersky, Donald Metzler, Marc Najork Google Inc. Mountain View, CA 94043 {xuanhui, bemike, metzler, najork}@google.com ABSTRACT
More informationChinese Language Parsing with Maximum-Entropy-Inspired Parser
Chinese Language Parsing with Maximum-Entropy-Inspired Parser Heng Lian Brown University Abstract The Chinese language has many special characteristics that make parsing difficult. The performance of state-of-the-art
More informationOPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS
OPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS Václav Kocian, Eva Volná, Michal Janošek, Martin Kotyrba University of Ostrava Department of Informatics and Computers Dvořákova 7,
More informationTU-E2090 Research Assignment in Operations Management and Services
Aalto University School of Science Operations and Service Management TU-E2090 Research Assignment in Operations Management and Services Version 2016-08-29 COURSE INSTRUCTOR: OFFICE HOURS: CONTACT: Saara
More informationAlgebra 1, Quarter 3, Unit 3.1. Line of Best Fit. Overview
Algebra 1, Quarter 3, Unit 3.1 Line of Best Fit Overview Number of instructional days 6 (1 day assessment) (1 day = 45 minutes) Content to be learned Analyze scatter plots and construct the line of best
More informationOn-the-Fly Customization of Automated Essay Scoring
Research Report On-the-Fly Customization of Automated Essay Scoring Yigal Attali Research & Development December 2007 RR-07-42 On-the-Fly Customization of Automated Essay Scoring Yigal Attali ETS, Princeton,
More informationPredicting Student Attrition in MOOCs using Sentiment Analysis and Neural Networks
Predicting Student Attrition in MOOCs using Sentiment Analysis and Neural Networks Devendra Singh Chaplot, Eunhee Rhim, and Jihie Kim Samsung Electronics Co., Ltd. Seoul, South Korea {dev.chaplot,eunhee.rhim,jihie.kim}@samsung.com
More informationEvidence for Reliability, Validity and Learning Effectiveness
PEARSON EDUCATION Evidence for Reliability, Validity and Learning Effectiveness Introduction Pearson Knowledge Technologies has conducted a large number and wide variety of reliability and validity studies
More informationLearning goal-oriented strategies in problem solving
Learning goal-oriented strategies in problem solving Martin Možina, Timotej Lazar, Ivan Bratko Faculty of Computer and Information Science University of Ljubljana, Ljubljana, Slovenia Abstract The need
More informationNavigating the PhD Options in CMS
Navigating the PhD Options in CMS This document gives an overview of the typical student path through the four Ph.D. programs in the CMS department ACM, CDS, CS, and CMS. Note that it is not a replacement
More informationA Case Study: News Classification Based on Term Frequency
A Case Study: News Classification Based on Term Frequency Petr Kroha Faculty of Computer Science University of Technology 09107 Chemnitz Germany kroha@informatik.tu-chemnitz.de Ricardo Baeza-Yates Center
More informationDiagnostic Test. Middle School Mathematics
Diagnostic Test Middle School Mathematics Copyright 2010 XAMonline, Inc. All rights reserved. No part of the material protected by this copyright notice may be reproduced or utilized in any form or by
More informationHuman Emotion Recognition From Speech
RESEARCH ARTICLE OPEN ACCESS Human Emotion Recognition From Speech Miss. Aparna P. Wanare*, Prof. Shankar N. Dandare *(Department of Electronics & Telecommunication Engineering, Sant Gadge Baba Amravati
More informationIntermediate Computable General Equilibrium (CGE) Modelling: Online Single Country Course
Intermediate Computable General Equilibrium (CGE) Modelling: Online Single Country Course Course Description This course is an intermediate course in practical computable general equilibrium (CGE) modelling
More informationTHE PENNSYLVANIA STATE UNIVERSITY SCHREYER HONORS COLLEGE DEPARTMENT OF MATHEMATICS ASSESSING THE EFFECTIVENESS OF MULTIPLE CHOICE MATH TESTS
THE PENNSYLVANIA STATE UNIVERSITY SCHREYER HONORS COLLEGE DEPARTMENT OF MATHEMATICS ASSESSING THE EFFECTIVENESS OF MULTIPLE CHOICE MATH TESTS ELIZABETH ANNE SOMERS Spring 2011 A thesis submitted in partial
More informationOCR for Arabic using SIFT Descriptors With Online Failure Prediction
OCR for Arabic using SIFT Descriptors With Online Failure Prediction Andrey Stolyarenko, Nachum Dershowitz The Blavatnik School of Computer Science Tel Aviv University Tel Aviv, Israel Email: stloyare@tau.ac.il,
More informationTIMSS ADVANCED 2015 USER GUIDE FOR THE INTERNATIONAL DATABASE. Pierre Foy
TIMSS ADVANCED 2015 USER GUIDE FOR THE INTERNATIONAL DATABASE Pierre Foy TIMSS Advanced 2015 orks User Guide for the International Database Pierre Foy Contributors: Victoria A.S. Centurino, Kerry E. Cotter,
More informationThe Singapore Copyright Act applies to the use of this document.
Title Mathematical problem solving in Singapore schools Author(s) Berinderjeet Kaur Source Teaching and Learning, 19(1), 67-78 Published by Institute of Education (Singapore) This document may be used
More informationSINGLE DOCUMENT AUTOMATIC TEXT SUMMARIZATION USING TERM FREQUENCY-INVERSE DOCUMENT FREQUENCY (TF-IDF)
SINGLE DOCUMENT AUTOMATIC TEXT SUMMARIZATION USING TERM FREQUENCY-INVERSE DOCUMENT FREQUENCY (TF-IDF) Hans Christian 1 ; Mikhael Pramodana Agus 2 ; Derwin Suhartono 3 1,2,3 Computer Science Department,
More informationMathematics subject curriculum
Mathematics subject curriculum Dette er ei omsetjing av den fastsette læreplanteksten. Læreplanen er fastsett på Nynorsk Established as a Regulation by the Ministry of Education and Research on 24 June
More informationApplications of data mining algorithms to analysis of medical data
Master Thesis Software Engineering Thesis no: MSE-2007:20 August 2007 Applications of data mining algorithms to analysis of medical data Dariusz Matyja School of Engineering Blekinge Institute of Technology
More informationImplementing a tool to Support KAOS-Beta Process Model Using EPF
Implementing a tool to Support KAOS-Beta Process Model Using EPF Malihe Tabatabaie Malihe.Tabatabaie@cs.york.ac.uk Department of Computer Science The University of York United Kingdom Eclipse Process Framework
More informationMGT/MGP/MGB 261: Investment Analysis
UNIVERSITY OF CALIFORNIA, DAVIS GRADUATE SCHOOL OF MANAGEMENT SYLLABUS for Fall 2014 MGT/MGP/MGB 261: Investment Analysis Daytime MBA: Tu 12:00p.m. - 3:00 p.m. Location: 1302 Gallagher (CRN: 51489) Sacramento
More informationMachine Learning from Garden Path Sentences: The Application of Computational Linguistics
Machine Learning from Garden Path Sentences: The Application of Computational Linguistics http://dx.doi.org/10.3991/ijet.v9i6.4109 J.L. Du 1, P.F. Yu 1 and M.L. Li 2 1 Guangdong University of Foreign Studies,
More informationAGS THE GREAT REVIEW GAME FOR PRE-ALGEBRA (CD) CORRELATED TO CALIFORNIA CONTENT STANDARDS
AGS THE GREAT REVIEW GAME FOR PRE-ALGEBRA (CD) CORRELATED TO CALIFORNIA CONTENT STANDARDS 1 CALIFORNIA CONTENT STANDARDS: Chapter 1 ALGEBRA AND WHOLE NUMBERS Algebra and Functions 1.4 Students use algebraic
More informationCSC200: Lecture 4. Allan Borodin
CSC200: Lecture 4 Allan Borodin 1 / 22 Announcements My apologies for the tutorial room mixup on Wednesday. The room SS 1088 is only reserved for Fridays and I forgot that. My office hours: Tuesdays 2-4
More informationAUTOMATIC DETECTION OF PROLONGED FRICATIVE PHONEMES WITH THE HIDDEN MARKOV MODELS APPROACH 1. INTRODUCTION
JOURNAL OF MEDICAL INFORMATICS & TECHNOLOGIES Vol. 11/2007, ISSN 1642-6037 Marek WIŚNIEWSKI *, Wiesława KUNISZYK-JÓŹKOWIAK *, Elżbieta SMOŁKA *, Waldemar SUSZYŃSKI * HMM, recognition, speech, disorders
More informationAxiom 2013 Team Description Paper
Axiom 2013 Team Description Paper Mohammad Ghazanfari, S Omid Shirkhorshidi, Farbod Samsamipour, Hossein Rahmatizadeh Zagheli, Mohammad Mahdavi, Payam Mohajeri, S Abbas Alamolhoda Robotics Scientific Association
More informationFourth Grade. Reporting Student Progress. Libertyville School District 70. Fourth Grade
Fourth Grade Libertyville School District 70 Reporting Student Progress Fourth Grade A Message to Parents/Guardians: Libertyville Elementary District 70 teachers of students in kindergarten-5 utilize a
More informationSTT 231 Test 1. Fill in the Letter of Your Choice to Each Question in the Scantron. Each question is worth 2 point.
STT 231 Test 1 Fill in the Letter of Your Choice to Each Question in the Scantron. Each question is worth 2 point. 1. A professor has kept records on grades that students have earned in his class. If he
More informationA study of speaker adaptation for DNN-based speech synthesis
A study of speaker adaptation for DNN-based speech synthesis Zhizheng Wu, Pawel Swietojanski, Christophe Veaux, Steve Renals, Simon King The Centre for Speech Technology Research (CSTR) University of Edinburgh,
More informationUniversity of Groningen. Systemen, planning, netwerken Bosman, Aart
University of Groningen Systemen, planning, netwerken Bosman, Aart IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document
More informationGrade Dropping, Strategic Behavior, and Student Satisficing
Grade Dropping, Strategic Behavior, and Student Satisficing Lester Hadsell Department of Economics State University of New York, College at Oneonta Oneonta, NY 13820 hadsell@oneonta.edu Raymond MacDermott
More informationLahore University of Management Sciences. FINN 321 Econometrics Fall Semester 2017
Instructor Syed Zahid Ali Room No. 247 Economics Wing First Floor Office Hours Email szahid@lums.edu.pk Telephone Ext. 8074 Secretary/TA TA Office Hours Course URL (if any) Suraj.lums.edu.pk FINN 321 Econometrics
More informationSoftprop: Softmax Neural Network Backpropagation Learning
Softprop: Softmax Neural Networ Bacpropagation Learning Michael Rimer Computer Science Department Brigham Young University Provo, UT 84602, USA E-mail: mrimer@axon.cs.byu.edu Tony Martinez Computer Science
More informationUniversity Library Collection Development and Management Policy
University Library Collection Development and Management Policy 2017-18 1 Executive Summary Anglia Ruskin University Library supports our University's strategic objectives by ensuring that students and
More informationGo fishing! Responsibility judgments when cooperation breaks down
Go fishing! Responsibility judgments when cooperation breaks down Kelsey Allen (krallen@mit.edu), Julian Jara-Ettinger (jjara@mit.edu), Tobias Gerstenberg (tger@mit.edu), Max Kleiman-Weiner (maxkw@mit.edu)
More informationLecture 1: Basic Concepts of Machine Learning
Lecture 1: Basic Concepts of Machine Learning Cognitive Systems - Machine Learning Ute Schmid (lecture) Johannes Rabold (practice) Based on slides prepared March 2005 by Maximilian Röglinger, updated 2010
More informationProbability estimates in a scenario tree
101 Chapter 11 Probability estimates in a scenario tree An expert is a person who has made all the mistakes that can be made in a very narrow field. Niels Bohr (1885 1962) Scenario trees require many numbers.
More informationScienceDirect. A Framework for Clustering Cardiac Patient s Records Using Unsupervised Learning Techniques
Available online at www.sciencedirect.com ScienceDirect Procedia Computer Science 98 (2016 ) 368 373 The 6th International Conference on Current and Future Trends of Information and Communication Technologies
More informationReflective problem solving skills are essential for learning, but it is not my job to teach them
Reflective problem solving skills are essential for learning, but it is not my job teach them Charles Henderson Western Michigan University http://homepages.wmich.edu/~chenders/ Edit Yerushalmi, Weizmann
More informationAn Effective Framework for Fast Expert Mining in Collaboration Networks: A Group-Oriented and Cost-Based Method
Farhadi F, Sorkhi M, Hashemi S et al. An effective framework for fast expert mining in collaboration networks: A grouporiented and cost-based method. JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY 27(3): 577
More informationLearning Methods for Fuzzy Systems
Learning Methods for Fuzzy Systems Rudolf Kruse and Andreas Nürnberger Department of Computer Science, University of Magdeburg Universitätsplatz, D-396 Magdeburg, Germany Phone : +49.39.67.876, Fax : +49.39.67.8
More informationDetailed course syllabus
Detailed course syllabus 1. Linear regression model. Ordinary least squares method. This introductory class covers basic definitions of econometrics, econometric model, and economic data. Classification
More informationUniversity of Waterloo School of Accountancy. AFM 102: Introductory Management Accounting. Fall Term 2004: Section 4
University of Waterloo School of Accountancy AFM 102: Introductory Management Accounting Fall Term 2004: Section 4 Instructor: Alan Webb Office: HH 289A / BFG 2120 B (after October 1) Phone: 888-4567 ext.
More informationHandling Concept Drifts Using Dynamic Selection of Classifiers
Handling Concept Drifts Using Dynamic Selection of Classifiers Paulo R. Lisboa de Almeida, Luiz S. Oliveira, Alceu de Souza Britto Jr. and and Robert Sabourin Universidade Federal do Paraná, DInf, Curitiba,
More informationMathematics Success Grade 7
T894 Mathematics Success Grade 7 [OBJECTIVE] The student will find probabilities of compound events using organized lists, tables, tree diagrams, and simulations. [PREREQUISITE SKILLS] Simple probability,
More informationDiscriminative Learning of Beam-Search Heuristics for Planning
Discriminative Learning of Beam-Search Heuristics for Planning Yuehua Xu School of EECS Oregon State University Corvallis,OR 97331 xuyu@eecs.oregonstate.edu Alan Fern School of EECS Oregon State University
More informationTest Effort Estimation Using Neural Network
J. Software Engineering & Applications, 2010, 3: 331-340 doi:10.4236/jsea.2010.34038 Published Online April 2010 (http://www.scirp.org/journal/jsea) 331 Chintala Abhishek*, Veginati Pavan Kumar, Harish
More informationPractice Examination IREB
IREB Examination Requirements Engineering Advanced Level Elicitation and Consolidation Practice Examination Questionnaire: Set_EN_2013_Public_1.2 Syllabus: Version 1.0 Passed Failed Total number of points
More informationAn OO Framework for building Intelligence and Learning properties in Software Agents
An OO Framework for building Intelligence and Learning properties in Software Agents José A. R. P. Sardinha, Ruy L. Milidiú, Carlos J. P. Lucena, Patrick Paranhos Abstract Software agents are defined as
More informationA Neural Network GUI Tested on Text-To-Phoneme Mapping
A Neural Network GUI Tested on Text-To-Phoneme Mapping MAARTEN TROMPPER Universiteit Utrecht m.f.a.trompper@students.uu.nl Abstract Text-to-phoneme (T2P) mapping is a necessary step in any speech synthesis
More informationhave professional experience before graduating... The University of Texas at Austin Budget difficulties
1. Number of qualified applicants that are willing to move. 2. Pay A disconnect between what is wanted in the positions and the experience of the available pool Academic libraries move slowly. Too often
More informationCREATING SHARABLE LEARNING OBJECTS FROM EXISTING DIGITAL COURSE CONTENT
CREATING SHARABLE LEARNING OBJECTS FROM EXISTING DIGITAL COURSE CONTENT Rajendra G. Singh Margaret Bernard Ross Gardler rajsingh@tstt.net.tt mbernard@fsa.uwi.tt rgardler@saafe.org Department of Mathematics
More information