Machine Learning (Decision Trees and Intro to Neural Nets) CSCI 3202, Fall 2010

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Machine Learning (Decision Trees and Intro to Neural Nets) CSCI 3202, Fall 2010"

Transcription

1 Machine Learning (Decision Trees and Intro to Neural Nets) CSCI 3202, Fall 2010

2 Assignments To read this week: Chapter 18, sections 1-4 and 7 Problem Set 3 due next week!

3 Learning a Decision Tree We look at someone making classification decisions, and try to infer the rule that they are using.(e.g., we might look at someone choosing videos and try to predict whether they think a particular title is a good video or not.) We assume that their rule can be written as a tree in which each node represents a local decision based on an attribute.

4

5 Some Attributes for Mike s Video Choices Do things blow up? (Tends to be good.) Is the title written in script? (Tends to be bad.) Is it a sequel? (Tends to be bad.) Is there a monster? (Tends to be good.) Is it based on a TV show? (Tends to be bad.)

6 A Training Set of Examples: We Watch Mike Make Lots of Video Choices Blowup? Script? Sequel? Monster? TV? Y Yes No Yes Yes No Y Yes Yes No Yes No N Yes Yes Yes No Yes and others

7 Building a Tree from Our Examples Suppose we have 12 attributes, and 200 examples of Mike s video choices:100 positive and 100 negative. Now, the crucial question: which attribute is most important to Mike?

8 Which is More Important? Attribute A divides the set as follows: (70 yes, 30 no) for A true (30 yes, 70 no) for A false Attribute B divides the set as follows: (100 yes, 90 no) for B true (0 yes, 10 no) for B false

9 Information as a Criterion for Attribute Values: Information value for a set of probabilities: Σ(- P i )log 2 (P i ) So, for a standard coin flip, the information is 2* (-1/2) (log 1/2) = 1 bit

10 Attribute A Information at start: -1/2 * log(1/2) + -1/2 * log (1/2) = 1 Information after Attribute A: Choice 1, weighted by 0.5: -0.7*log(0.7) *(log 0.3) Choice 2, weighted by *log(0.3) *(log 0.7) Total information: = 0.881

11 Attribute B Choice 1, weighted by /19 * (log 10/19) + -9/19(log 9/19) Choice 2, weighted by * (log 0) + -1 (log 1) Total information value: The change in information by using Attribute A is greater, so A is the more informative attribute.

12 Splitting Examples into Training and Test Sets Given our initial set of examples, split it into a (randomly-chosen) training set and a test set. Once the algorithm has generated a tree for the training set, use the test set to gauge the accuracy of the tree (measure the percent of the test set that is correctly classified). For sufficiently large and sufficiently representative training sets we converge on a high accuracy for the test set.

13 Things We ve Swept Under the Rug Doesn t this strategy lead to attributes with many possibilities? (E.g., day of the year?) Are we sure that all new examples will be completely classified? Aren t there some functions that are hard to express using decision trees?

14 A Bad Concept for Decision Trees to Learn Majority function (classified as true whenever the majority of the attributes are positive, false otherwise) Each attribute is equally important, and none are very effective at dividing the set

15

16 Neural Networks: Some First Concepts Each neural element is loosely based on the structure of neurons A neural net is a collection of neural elements connected by weighted links We think of some set of neurons as input elements ; these are linked to output elements which can be interpreted as a classification of the input pattern Standard formats: perceptrons and multilayer feedforward networks

17 Structure of an (artificial) neuron Think of this as a very simple computational element: it receives numeric input values, sums those values, and compares them to a threshold. If the sum of the inputs is greater than the threshold, the neuron outputs a 1; otherwise it outputs a zero. The output of this neuron is connected (as usual, via weighted links) to subsequent neurons in the net.

18 Perceptrons: the Simplest Neural Network Perceptrons are two-layer networks: one layer of inputs directly connected to a layer of outputs. For simplicity, we can look at a perceptron with a single output node

19 Training a Perceptron by Adjusting its Weights Overall error is the (squared) value of the difference between what we wanted and what we got from our perceptron. Once we see that our perceptron is in error, we can adjust each of the weights leading to the output node. We ll adjust each weight in such a way as to make the error value smaller.

Machine Learning and Artificial Neural Networks (Ref: Negnevitsky, M. Artificial Intelligence, Chapter 6)

Machine Learning and Artificial Neural Networks (Ref: Negnevitsky, M. Artificial Intelligence, Chapter 6) Machine Learning and Artificial Neural Networks (Ref: Negnevitsky, M. Artificial Intelligence, Chapter 6) The Concept of Learning Learning is the ability to adapt to new surroundings and solve new problems.

More information

18 LEARNING FROM EXAMPLES

18 LEARNING FROM EXAMPLES 18 LEARNING FROM EXAMPLES An intelligent agent may have to learn, for instance, the following components: A direct mapping from conditions on the current state to actions A means to infer relevant properties

More information

Supervised learning can be done by choosing the hypothesis that is most probable given the data: = arg max ) = arg max

Supervised learning can be done by choosing the hypothesis that is most probable given the data: = arg max ) = arg max The learning problem is called realizable if the hypothesis space contains the true function; otherwise it is unrealizable On the other hand, in the name of better generalization ability it may be sensible

More information

CS 4510/9010 Applied Machine Learning. Evaluation. Paula Matuszek Fall, copyright Paula Matuszek 2016

CS 4510/9010 Applied Machine Learning. Evaluation. Paula Matuszek Fall, copyright Paula Matuszek 2016 CS 4510/9010 Applied Machine Learning 1 Evaluation Paula Matuszek Fall, 2016 Evaluating Classifiers 2 With a decision tree, or with any classifier, we need to know how well our trained model performs on

More information

Classification with Deep Belief Networks. HussamHebbo Jae Won Kim

Classification with Deep Belief Networks. HussamHebbo Jae Won Kim Classification with Deep Belief Networks HussamHebbo Jae Won Kim Table of Contents Introduction... 3 Neural Networks... 3 Perceptron... 3 Backpropagation... 4 Deep Belief Networks (RBM, Sigmoid Belief

More information

Foundations of Intelligent Systems CSCI (Fall 2015)

Foundations of Intelligent Systems CSCI (Fall 2015) Foundations of Intelligent Systems CSCI-630-01 (Fall 2015) Final Examination, Fri. Dec 18, 2015 Instructor: Richard Zanibbi, Duration: 120 Minutes Name: Instructions The exam questions are worth a total

More information

Assignment 6 (Sol.) Introduction to Machine Learning Prof. B. Ravindran

Assignment 6 (Sol.) Introduction to Machine Learning Prof. B. Ravindran Assignment 6 (Sol.) Introduction to Machine Learning Prof. B. Ravindran 1. Assume that you are given a data set and a neural network model trained on the data set. You are asked to build a decision tree

More information

Performance Analysis of Various Data Mining Techniques on Banknote Authentication

Performance Analysis of Various Data Mining Techniques on Banknote Authentication International Journal of Engineering Science Invention ISSN (Online): 2319 6734, ISSN (Print): 2319 6726 Volume 5 Issue 2 February 2016 PP.62-71 Performance Analysis of Various Data Mining Techniques on

More information

Course 395: Machine Learning - Lectures

Course 395: Machine Learning - Lectures Course 395: Machine Learning - Lectures Lecture 1-2: Concept Learning (M. Pantic) Lecture 3-4: Decision Trees & CBC Intro (M. Pantic & S. Petridis) Lecture 5-6: Evaluating Hypotheses (S. Petridis) Lecture

More information

Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition

Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition Introduction to Ensemble Learning Featuring Successes in the Netflix Prize Competition Todd Holloway Two Lecture Series for B551 November 20 & 27, 2007 Indiana University Outline Introduction Bias and

More information

Exploration vs. Exploitation. CS 473: Artificial Intelligence Reinforcement Learning II. How to Explore? Exploration Functions

Exploration vs. Exploitation. CS 473: Artificial Intelligence Reinforcement Learning II. How to Explore? Exploration Functions CS 473: Artificial Intelligence Reinforcement Learning II Exploration vs. Exploitation Dieter Fox / University of Washington [Most slides were taken from Dan Klein and Pieter Abbeel / CS188 Intro to AI

More information

Artificial Neural Networks for Storm Surge Predictions in NC. DHS Summer Research Team

Artificial Neural Networks for Storm Surge Predictions in NC. DHS Summer Research Team Artificial Neural Networks for Storm Surge Predictions in NC DHS Summer Research Team 1 Outline Introduction; Feedforward Artificial Neural Network; Design questions; Implementation; Improvements; Conclusions;

More information

Decision Tree for Playing Tennis

Decision Tree for Playing Tennis Decision Tree Decision Tree for Playing Tennis (outlook=sunny, wind=strong, humidity=normal,? ) DT for prediction C-section risks Characteristics of Decision Trees Decision trees have many appealing properties

More information

Artificial Neural Networks written examination

Artificial Neural Networks written examination 1 (8) Institutionen för informationsteknologi Olle Gällmo Universitetsadjunkt Adress: Lägerhyddsvägen 2 Box 337 751 05 Uppsala Artificial Neural Networks written examination Monday, May 15, 2006 9 00-14

More information

P(A, B) = P(A B) = P(A) + P(B) - P(A B)

P(A, B) = P(A B) = P(A) + P(B) - P(A B) AND Probability P(A, B) = P(A B) = P(A) + P(B) - P(A B) P(A B) = P(A) + P(B) - P(A B) Area = Probability of Event AND Probability P(A, B) = P(A B) = P(A) + P(B) - P(A B) If, and only if, A and B are independent,

More information

UNIVERSITY OF OSLO. Faculty of Mathematics and Natural Sciences

UNIVERSITY OF OSLO. Faculty of Mathematics and Natural Sciences Page 1 of 7 UNIVERSITY OF OSLO Faculty of Mathematics and Natural Sciences Exam in INF3490/4490 iologically Inspired omputing ay of exam: ecember 9th, 2015 Exam hours: 09:00 13:00 This examination paper

More information

Lecture 5: 21 September 2016 Intro to machine learning and single-layer neural networks. Jim Tørresen This Lecture

Lecture 5: 21 September 2016 Intro to machine learning and single-layer neural networks. Jim Tørresen This Lecture This Lecture INF3490 - Biologically inspired computing Lecture 5: 21 September 2016 Intro to machine learning and single-layer neural networks Jim Tørresen 1. Introduction to learning/classification 2.

More information

LEARNING AGENTS IN ARTIFICIAL INTELLIGENCE PART I

LEARNING AGENTS IN ARTIFICIAL INTELLIGENCE PART I Journal of Advanced Research in Computer Engineering, Vol. 5, No. 1, January-June 2011, pp. 1-5 Global Research Publications ISSN:0974-4320 LEARNING AGENTS IN ARTIFICIAL INTELLIGENCE PART I JOSEPH FETTERHOFF

More information

TOWARDS DATA-DRIVEN AUTONOMICS IN DATA CENTERS

TOWARDS DATA-DRIVEN AUTONOMICS IN DATA CENTERS TOWARDS DATA-DRIVEN AUTONOMICS IN DATA CENTERS ALINA SIRBU, OZALP BABAOGLU SUMMARIZED BY ARDA GUMUSALAN MOTIVATION 2 MOTIVATION Human-interaction-dependent data centers are not sustainable for future data

More information

Inventor Chung T. Nguyen NOTTCE. The above identified patent application is available for licensing. Requests for information should be addressed to:

Inventor Chung T. Nguyen NOTTCE. The above identified patent application is available for licensing. Requests for information should be addressed to: Serial No. 802.572 Filing Date 3 February 1997 Inventor Chung T. Nguyen NOTTCE The above identified patent application is available for licensing. Requests for information should be addressed to: OFFICE

More information

4 Feedforward Neural Networks, Binary XOR, Continuous XOR, Parity Problem and Composed Neural Networks.

4 Feedforward Neural Networks, Binary XOR, Continuous XOR, Parity Problem and Composed Neural Networks. 4 Feedforward Neural Networks, Binary XOR, Continuous XOR, Parity Problem and Composed Neural Networks. 4.1 Objectives The objective of the following exercises is to get acquainted with the inner working

More information

Outline. Learning from Observations. Learning agents. Learning. Inductive learning (a.k.a. Science) Environment. Agent.

Outline. Learning from Observations. Learning agents. Learning. Inductive learning (a.k.a. Science) Environment. Agent. Outline Learning agents Learning from Observations Inductive learning Decision tree learning Measuring learning performance Chapter 18, Sections 1 3 Chapter 18, Sections 1 3 1 Chapter 18, Sections 1 3

More information

Analyzing Software using Deep Learning Introduction

Analyzing Software using Deep Learning Introduction Analyzing Software using Deep Learning Introduction Subscribe to the course via Piazza: piazza.com/tu-darmstadt.de/summer2017/20000999iv Prof. Dr. Michael Pradel Software Lab, TU Darmstadt 1 About Me Michael

More information

Evolution of Neural Networks. October 20, 2017

Evolution of Neural Networks. October 20, 2017 Evolution of Neural Networks October 20, 2017 Single Layer Perceptron, (1957) Frank Rosenblatt 1957 1957 Single Layer Perceptron Perceptron, invented in 1957 at the Cornell Aeronautical Laboratory by Frank

More information

CS Machine Learning

CS Machine Learning CS 478 - Machine Learning Projects Data Representation Basic testing and evaluation schemes CS 478 Data and Testing 1 Programming Issues l Program in any platform you want l Realize that you will be doing

More information

Compacting Instances: Creating models

Compacting Instances: Creating models Decision Trees Compacting Instances: Creating models Food Chat Speedy Price Bar BigTip (3) (2) (2) (2) (2) 1 great yes yes adequate no yes 2 great no yes adequate no yes 3 mediocre yes no high no no 4

More information

I400 Health Informatics Data Mining Instructions (KP Project)

I400 Health Informatics Data Mining Instructions (KP Project) I400 Health Informatics Data Mining Instructions (KP Project) Casey Bennett Spring 2014 Indiana University 1) Import: First, we need to import the data into Knime. add CSV Reader Node (under IO>>Read)

More information

Linear Models Continued: Perceptron & Logistic Regression

Linear Models Continued: Perceptron & Logistic Regression Linear Models Continued: Perceptron & Logistic Regression CMSC 723 / LING 723 / INST 725 Marine Carpuat Slides credit: Graham Neubig, Jacob Eisenstein Linear Models for Classification Feature function

More information

Introducing Deep Learning with MATLAB

Introducing Deep Learning with MATLAB Introducing Deep Learning with MATLAB What is Deep Learning? Deep learning is a type of machine learning in which a model learns to perform classification tasks directly from images, text, or sound. Deep

More information

Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches

Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches Qandeel Tariq, Alex Kolchinski, Richard Davis December 6, 206 Introduction This paper

More information

DEEP LEARNING AND ITS APPLICATION NEURAL NETWORK BASICS

DEEP LEARNING AND ITS APPLICATION NEURAL NETWORK BASICS DEEP LEARNING AND ITS APPLICATION NEURAL NETWORK BASICS Argument on AI 1. Symbolism 2. Connectionism 3. Actionism Kai Yu. SJTU Deep Learning Lecture. 2 Argument on AI 1. Symbolism Symbolism AI Origin Cognitive

More information

CS221 Practice Midterm #1

CS221 Practice Midterm #1 CS221 Practice Midterm #1 Summer 2013 The following pages are excerpts from similar classes midterms. The content is similar to our midterm but I have opted to give you a document with more problems rather

More information

Decision Tree For Playing Tennis

Decision Tree For Playing Tennis Decision Tree For Playing Tennis ROOT NODE BRANCH INTERNAL NODE LEAF NODE Disjunction of conjunctions Another Perspective of a Decision Tree Model Age 60 40 20 NoDefault NoDefault + + NoDefault Default

More information

Artificial Neural Networks in Data Mining

Artificial Neural Networks in Data Mining IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 18, Issue 6, Ver. III (Nov.-Dec. 2016), PP 55-59 www.iosrjournals.org Artificial Neural Networks in Data Mining

More information

Session 7: Face Detection (cont.)

Session 7: Face Detection (cont.) Session 7: Face Detection (cont.) John Magee 8 February 2017 Slides courtesy of Diane H. Theriault Question of the Day: How can we find faces in images? Face Detection Compute features in the image Apply

More information

Dudon Wai Georgia Institute of Technology CS 7641: Machine Learning Atlanta, GA

Dudon Wai Georgia Institute of Technology CS 7641: Machine Learning Atlanta, GA Adult Income and Letter Recognition - Supervised Learning Report An objective look at classifier performance for predicting adult income and Letter Recognition Dudon Wai Georgia Institute of Technology

More information

Introduction of connectionist models

Introduction of connectionist models Introduction of connectionist models Introduction to ANNs Markus Dambek Uni Bremen 20. Dezember 2010 Markus Dambek (Uni Bremen) Introduction of connectionist models 20. Dezember 2010 1 / 66 1 Introduction

More information

Adaptive Activation Functions for Deep Networks

Adaptive Activation Functions for Deep Networks Adaptive Activation Functions for Deep Networks Michael Dushkoff, Raymond Ptucha Rochester Institute of Technology IS&T International Symposium on Electronic Imaging 2016 Computational Imaging Feb 16,

More information

Kobe University Repository : Kernel

Kobe University Repository : Kernel Title Author(s) Kobe University Repository : Kernel A Multitask Learning Model for Online Pattern Recognition Ozawa, Seiichi / Roy, Asim / Roussinov, Dmitri Citation IEEE Transactions on Neural Neworks,

More information

Mocking the Draft Predicting NFL Draft Picks and Career Success

Mocking the Draft Predicting NFL Draft Picks and Career Success Mocking the Draft Predicting NFL Draft Picks and Career Success Wesley Olmsted [wolmsted], Jeff Garnier [jeff1731], Tarek Abdelghany [tabdel] 1 Introduction We started off wanting to make some kind of

More information

Assignment #6: Neural Networks (with Tensorflow) CSCI 374 Fall 2017 Oberlin College Due: Tuesday November 21 at 11:59 PM

Assignment #6: Neural Networks (with Tensorflow) CSCI 374 Fall 2017 Oberlin College Due: Tuesday November 21 at 11:59 PM Background Assignment #6: Neural Networks (with Tensorflow) CSCI 374 Fall 2017 Oberlin College Due: Tuesday November 21 at 11:59 PM Our final assignment this semester has three main goals: 1. Implement

More information

Using neural nets to recognize handwritten digits

Using neural nets to recognize handwritten digits CHAPTER 1 Using neural nets to recognize handwritten digits The human visual system is one of the wonders of the world. Consider the following sequence of handwritten digits: Most people effortlessly recognize

More information

Pattern Classification and Clustering Spring 2006

Pattern Classification and Clustering Spring 2006 Pattern Classification and Clustering Time: Spring 2006 Room: Instructor: Yingen Xiong Office: 621 McBryde Office Hours: Phone: 231-4212 Email: yxiong@cs.vt.edu URL: http://www.cs.vt.edu/~yxiong/pcc/ Detailed

More information

Predicting Student Performance by Using Data Mining Methods for Classification

Predicting Student Performance by Using Data Mining Methods for Classification BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 13, No 1 Sofia 2013 Print ISSN: 1311-9702; Online ISSN: 1314-4081 DOI: 10.2478/cait-2013-0006 Predicting Student Performance

More information

Programming Assignment2: Neural Networks

Programming Assignment2: Neural Networks Programming Assignment2: Neural Networks Problem :. In this homework assignment, your task is to implement one of the common machine learning algorithms: Neural Networks. You will train and test a neural

More information

Machine Learning with Weka

Machine Learning with Weka Machine Learning with Weka SLIDES BY (TOTAL 5 Session of 1.5 Hours Each) ANJALI GOYAL & ASHISH SUREKA (www.ashish-sureka.in) CS 309 INFORMATION RETRIEVAL COURSE ASHOKA UNIVERSITY NOTE: Slides created and

More information

Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 11, 2011

Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University. January 11, 2011 Machine Learning 10-701 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 11, 2011 Today: What is machine learning? Decision tree learning Course logistics Readings: The Discipline

More information

IAI : Machine Learning

IAI : Machine Learning IAI : Machine Learning John A. Bullinaria, 2005 1. What is Machine Learning? 2. The Need for Learning 3. Learning in Neural and Evolutionary Systems 4. Problems Facing Expert Systems 5. Learning in Rule

More information

Artificial Neural Networks

Artificial Neural Networks Artificial Neural Networks Outline Introduction to Neural Network Introduction to Artificial Neural Network Properties of Artificial Neural Network Applications of Artificial Neural Network Demo Neural

More information

Predicting a Student's Performance Vani Khosla

Predicting a Student's Performance Vani Khosla CS 229 Final Report, Fall 2015 Khosla 1 Abstract Predicting a Student's Performance Vani Khosla The ability to predict a student s performance on a given concept is an important tool for the Education

More information

Neural Networks. CSC 4504 : Langages formels et applications. J Paul Gibson, D311.

Neural Networks. CSC 4504 : Langages formels et applications. J Paul Gibson, D311. CSC 4504 : Langages formels et applications J Paul Gibson, D311 paul.gibson@telecom-sudparis.eu /~gibson/teaching/csc4504/problem11-neuralnetworks.pdf Neural Networks 1 2 The following slides are a summary

More information

The Generalized Delta Rule and Practical Considerations

The Generalized Delta Rule and Practical Considerations The Generalized Delta Rule and Practical Considerations Introduction to Neural Networks : Lecture 6 John A. Bullinaria, 2004 1. Training a Single Layer Feed-forward Network 2. Deriving the Generalized

More information

Gender Classification Based on FeedForward Backpropagation Neural Network

Gender Classification Based on FeedForward Backpropagation Neural Network Gender Classification Based on FeedForward Backpropagation Neural Network S. Mostafa Rahimi Azghadi 1, M. Reza Bonyadi 1 and Hamed Shahhosseini 2 1 Department of Electrical and Computer Engineering, Shahid

More information

INTRODUCTION TO DATA SCIENCE

INTRODUCTION TO DATA SCIENCE DATA11001 INTRODUCTION TO DATA SCIENCE EPISODE 6: MACHINE LEARNING TODAY S MENU 1. WHAT IS ML? 2. CLASSIFICATION AND REGRESSSION 3. EVALUATING PERFORMANCE & OVERFITTING WHAT IS MACHINE LEARNING? Definition:

More information

CS 224N Competitive Grammar Writing

CS 224N Competitive Grammar Writing CS 224N Competitive Grammar Writing 1 Setup Instructions: Log into your home directory in corn/myth. cd mkdir -p cs224n cd cs224n cp -r /afs/ir/class/cs224n/pa-cgw. cd pa-cgw bash export PATH=$PATH:~/cs224n/pa-cgw

More information

Deep Neural Networks for Acoustic Modelling. Bajibabu Bollepalli Hieu Nguyen Rakshith Shetty Pieter Smit (Mentor)

Deep Neural Networks for Acoustic Modelling. Bajibabu Bollepalli Hieu Nguyen Rakshith Shetty Pieter Smit (Mentor) Deep Neural Networks for Acoustic Modelling Bajibabu Bollepalli Hieu Nguyen Rakshith Shetty Pieter Smit (Mentor) Introduction Automatic speech recognition Speech signal Feature Extraction Acoustic Modelling

More information

A Methodology for Creating Generic Game Playing Agents for Board Games

A Methodology for Creating Generic Game Playing Agents for Board Games A Methodology for Creating Generic Game Playing Agents for Board Games Mateus Andrade Rezende Luiz Chaimowicz Universidade Federal de Minas Gerais (UFMG), Department of Computer Science, Brazil ABSTRACT

More information

INLS 613 Text Data Mining Homework 2 Due: Monday, October 10, 2016 by 11:55pm via Sakai

INLS 613 Text Data Mining Homework 2 Due: Monday, October 10, 2016 by 11:55pm via Sakai INLS 613 Text Data Mining Homework 2 Due: Monday, October 10, 2016 by 11:55pm via Sakai 1 Objective The goal of this homework is to give you exposure to the practice of training and testing a machine-learning

More information

NoiseOut: A Simple Way to Prune Neural Networks

NoiseOut: A Simple Way to Prune Neural Networks NoiseOut: A Simple Way to Prune Neural Networks Mohammad Babaeizadeh, Paris Smaragdis & Roy H. Campbell Department of Computer Science University of Illinois at Urbana-Champaign {mb2,paris,rhc}@illinois.edu.edu

More information

Evaluation and Comparison of Performance of different Classifiers

Evaluation and Comparison of Performance of different Classifiers Evaluation and Comparison of Performance of different Classifiers Bhavana Kumari 1, Vishal Shrivastava 2 ACE&IT, Jaipur Abstract:- Many companies like insurance, credit card, bank, retail industry require

More information

Biomedical Research 2016; Special Issue: S87-S91 ISSN X

Biomedical Research 2016; Special Issue: S87-S91 ISSN X Biomedical Research 2016; Special Issue: S87-S91 ISSN 0970-938X www.biomedres.info Analysis liver and diabetes datasets by using unsupervised two-phase neural network techniques. KG Nandha Kumar 1, T Christopher

More information

The Health Economics and Outcomes Research Applications and Valuation of Digital Health Technologies and Machine Learning

The Health Economics and Outcomes Research Applications and Valuation of Digital Health Technologies and Machine Learning The Health Economics and Outcomes Research Applications and Valuation of Digital Health Technologies and Machine Learning Workshop W29 - Session V 3:00 4:00pm May 25, 2016 ISPOR 21 st Annual International

More information

Synaptic Weight Noise During MLP Learning Enhances Fault-Tolerance, Generalisation and Learning Trajectory

Synaptic Weight Noise During MLP Learning Enhances Fault-Tolerance, Generalisation and Learning Trajectory Synaptic Weight Noise During MLP Learning Enhances Fault-Tolerance, Generalisation and Learning Trajectory Alan F. Murray Dept. of Electrical Engineering Edinburgh University Scotland Peter J. Edwards

More information

Adjusting multiple model neural filter for the needs of marine radar target tracking

Adjusting multiple model neural filter for the needs of marine radar target tracking International Radar Symposium IRS 211 617 Adjusting multiple model neural filter for the needs of marine radar target tracking Witold Kazimierski *, Andrzej Stateczny * * Maritime University of Szczecin,

More information

Applied Machine Learning Lecture 1: Introduction

Applied Machine Learning Lecture 1: Introduction Applied Machine Learning Lecture 1: Introduction Richard Johansson January 16, 2018 welcome to the course! machine learning is getting increasingly popular among students our courses are full! many thesis

More information

Project: Joint Probability (AKA, multiple probabilities) Part 1. A Familiar Data set to Introduce a New Idea

Project: Joint Probability (AKA, multiple probabilities) Part 1. A Familiar Data set to Introduce a New Idea Project: Joint Probability (AKA, multiple probabilities) At the point, we re ready to start dealing with more advanced probabilistic analysis. Although the single probabilities from the first project were

More information

Intelligent Systems. Neural Networks. Copyright 2009 Dieter Fensel and Reto Krummenacher

Intelligent Systems. Neural Networks. Copyright 2009 Dieter Fensel and Reto Krummenacher Intelligent Systems Neural Networks Copyright 2009 Dieter Fensel and Reto Krummenacher 1 Where are we? # Title 1 Introduction 2 Propositional Logic 3 Predicate Logic 4 Theorem Proving, Description Logics

More information

Childhood Obesity epidemic analysis using classification algorithms

Childhood Obesity epidemic analysis using classification algorithms Childhood Obesity epidemic analysis using classification algorithms Suguna. M M.Phil. Scholar Trichy, Tamilnadu, India suguna15.9@gmail.com Abstract Obesity is the one of the most serious public health

More information

Reinforcement Learning with Randomization, Memory, and Prediction

Reinforcement Learning with Randomization, Memory, and Prediction Reinforcement Learning with Randomization, Memory, and Prediction Radford M. Neal, University of Toronto Dept. of Statistical Sciences and Dept. of Computer Science http://www.cs.utoronto.ca/ radford CRM

More information

OPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS

OPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS OPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS Václav Kocian, Eva Volná, Michal Janošek, Martin Kotyrba University of Ostrava Department of Informatics and Computers Dvořákova 7,

More information

10701/15781 Machine Learning, Spring 2005: Homework 1

10701/15781 Machine Learning, Spring 2005: Homework 1 10701/15781 Machine Learning, Spring 2005: Homework 1 Due: Monday, February 6, beginning of the class 1 [15 Points] Probability and Regression [Stano] 1 1.1 [10 Points] The Matrix Strikes Back The Matrix

More information

A Survey on Pattern Recognition Using Spiking Neural Networks with Temporal Encoding and Learning

A Survey on Pattern Recognition Using Spiking Neural Networks with Temporal Encoding and Learning A Survey on Pattern Recognition Using Spiking Neural Networks with Temporal Encoding and Learning A.Dhilipan, P.G Scholar, Department of Computer Science, Anna University Regional Centre, Coimbatore, adhilipbe@gmail.com.

More information

6.080 / Great Ideas in Theoretical Computer Science Spring 2008

6.080 / Great Ideas in Theoretical Computer Science Spring 2008 MIT OpenCourseWare http://ocw.mit.edu 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

More information

Supervised and Unsupervised Learning. Ciro Donalek Ay/Bi 199 April 2011

Supervised and Unsupervised Learning. Ciro Donalek Ay/Bi 199 April 2011 Supervised and Unsupervised Learning Ciro Donalek Ay/Bi 199 April 2011 Summary KDD and Data Mining Tasks Finding the op?mal approach Supervised Models Neural Networks Mul? Layer Perceptron Decision Trees

More information

SOFTCOMPUTING IN MODELING & SIMULATION

SOFTCOMPUTING IN MODELING & SIMULATION SOFTCOMPUTING IN MODELING & SIMULATION 9th July, 2002 Faculty of Science, Philadelphia University Dr. Kasim M. Al-Aubidy Computer & Software Eng. Dept. Philadelphia University The only way not to succeed

More information

Computer Vision for Card Games

Computer Vision for Card Games Computer Vision for Card Games Matias Castillo matiasct@stanford.edu Benjamin Goeing bgoeing@stanford.edu Jesper Westell jesperw@stanford.edu Abstract For this project, we designed a computer vision program

More information

HANDWRITTEN DIGIT RECOGNITION FOR MANAGING EXAMINATION SCORE IN PAPER-BASED TEST

HANDWRITTEN DIGIT RECOGNITION FOR MANAGING EXAMINATION SCORE IN PAPER-BASED TEST HANDRITTEN DIGIT RECOGNITION FOR MANAGING EXAMINATION SCORE IN PAPER-BASED TEST,* APICHAT SURATANEE, NANTAPORN LERTSARI, SETHAAT KAMPHASEE, KRITSADA SRIKET Department of Mathematics, Faculty of Applied

More information

Linear Regression: Predicting House Prices

Linear Regression: Predicting House Prices Linear Regression: Predicting House Prices I am big fan of Kalid Azad writings. He has a knack of explaining hard mathematical concepts like Calculus in simple words and helps the readers to get the intuition

More information

Improving Machine Learning Through Oracle Learning

Improving Machine Learning Through Oracle Learning Brigham Young University BYU ScholarsArchive All Theses and Dissertations 2007-03-12 Improving Machine Learning Through Oracle Learning Joshua Ephraim Menke Brigham Young University - Provo Follow this

More information

CSC 4510/9010: Applied Machine Learning Rule Inference

CSC 4510/9010: Applied Machine Learning Rule Inference CSC 4510/9010: Applied Machine Learning Rule Inference Dr. Paula Matuszek Paula.Matuszek@villanova.edu Paula.Matuszek@gmail.com (610) 647-9789 CSC 4510.9010 Spring 2015. Paula Matuszek 1 Red Tape Going

More information

Explanation and Simulation in Cognitive Science

Explanation and Simulation in Cognitive Science Explanation and Simulation in Cognitive Science Simulation and computational modeling Symbolic models Connectionist models Comparing symbolism and connectionism Hybrid architectures Cognitive architectures

More information

Ensembles of Multilayer Perceptron and Modular Neural Networks for Fast and Accurate Learning

Ensembles of Multilayer Perceptron and Modular Neural Networks for Fast and Accurate Learning Ensembles of Multilayer Perceptron and Modular Neural Networks for Fast and Accurate Learning R.M. Valdovinos Lab. Reconocimiento de Patrones Instituto Tecnológico de Toluca Av. Tecnológico s/n, 52140

More information

Python Machine Learning

Python Machine Learning Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled

More information

A Procedure for Classifying New Respondents into Existing Segments Using Maximum Difference Scaling

A Procedure for Classifying New Respondents into Existing Segments Using Maximum Difference Scaling A Procedure for Classifying New Respondents into Existing Segments Using Maximum Difference Scaling Background Bryan Orme and Rich Johnson, Sawtooth Software March, 2009 (with minor clarifications September

More information

Machine Learning and Data Mining. Ensembles of Learners. Prof. Alexander Ihler

Machine Learning and Data Mining. Ensembles of Learners. Prof. Alexander Ihler Machine Learning and Data Mining Ensembles of Learners Prof. Alexander Ihler Ensemble methods Why learn one classifier when you can learn many? Ensemble: combine many predictors (Weighted) combina

More information

M.tech Scholar, Dept. of CSE

M.tech Scholar, Dept. of CSE 00000000000 Modular Neural Network Approach for Data Classification 1, Divya Taneja, 2 Dr. Vivek Srivastava, ABSTRACT M.tech Scholar, Dept. of CSE Faculty of Engineering, Rama University, Kanpur Classification

More information

arxiv: v3 [cs.lg] 9 Mar 2014

arxiv: v3 [cs.lg] 9 Mar 2014 Learning Factored Representations in a Deep Mixture of Experts arxiv:1312.4314v3 [cs.lg] 9 Mar 2014 David Eigen 1,2 Marc Aurelio Ranzato 1 Ilya Sutskever 1 1 Google, Inc. 2 Dept. of Computer Science, Courant

More information

CSC-272 Exam #2 March 20, 2015

CSC-272 Exam #2 March 20, 2015 CSC-272 Exam #2 March 20, 2015 Name Questions are weighted as indicated. Show your work and state your assumptions for partial credit consideration. Unless explicitly stated, there are NO intended errors

More information

Ensemble Classifier for Solving Credit Scoring Problems

Ensemble Classifier for Solving Credit Scoring Problems Ensemble Classifier for Solving Credit Scoring Problems Maciej Zięba and Jerzy Świątek Wroclaw University of Technology, Faculty of Computer Science and Management, Wybrzeże Wyspiańskiego 27, 50-370 Wrocław,

More information

Training Connectionist Networks with Queries and Selective Sampling

Training Connectionist Networks with Queries and Selective Sampling 566 Atlas, Cohn and Ladner Training Connectionist Networks with Queries and Selective Sampling Les Atlas Dept. of E.E. David Cohn Dept. of C.S. & E. Richard Ladner Dept. of C.S. & E. M.A. El-Sharkawi,

More information

Machine Learning B, Fall 2016

Machine Learning B, Fall 2016 Machine Learning 10-601 B, Fall 2016 Decision Trees (Summary) Lecture 2, 08/31/ 2016 Maria-Florina (Nina) Balcan Learning Decision Trees. Supervised Classification. Useful Readings: Mitchell, Chapter 3

More information

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection.

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. Instructor: Herke van Hoof (herke.vanhoof@mail.mcgill.ca) Slides mostly by: Class web page: www.cs.mcgill.ca/~hvanho2/comp551

More information

A study of the NIPS feature selection challenge

A study of the NIPS feature selection challenge A study of the NIPS feature selection challenge Nicholas Johnson November 29, 2009 Abstract The 2003 Nips Feature extraction challenge was dominated by Bayesian approaches developed by the team of Radford

More information

Fusion of Multiple Handwritten Word Recognition Techniques

Fusion of Multiple Handwritten Word Recognition Techniques Fusion of Multiple Handwritten Word Recognition Techniques B. Verma' and P. Gader2 'School of Information Technology Griffith University-Gold Coast PMB 50, Gold Coast Mail Centre QLD 9726, Australia E-mail:

More information

Part-of-Speech Tagging

Part-of-Speech Tagging TDDE09, 729A27 Natural Language Processing (2017) Part-of-Speech Tagging Marco Kuhlmann Department of Computer and Information Science This work is licensed under a Creative Commons Attribution 4.0 International

More information

Explorations Using Extensions and Modifications to the Oppenheim et al. Model for Cumulative Semantic Interference

Explorations Using Extensions and Modifications to the Oppenheim et al. Model for Cumulative Semantic Interference Lehigh University Lehigh Preserve Theses and Dissertations 2015 Explorations Using Extensions and Modifications to the Oppenheim et al. Model for Cumulative Semantic Interference Tyler Seip Lehigh University

More information

Introduction to Deep Learning

Introduction to Deep Learning Introduction to Deep Learning M S Ram Dept. of Computer Science & Engg. Indian Institute of Technology Kanpur Reading of Chap. 1 from Learning Deep Architectures for AI ; Yoshua Bengio; FTML Vol. 2, No.

More information

ANALYZING BIG DATA WITH DECISION TREES

ANALYZING BIG DATA WITH DECISION TREES San Jose State University SJSU ScholarWorks Master's Projects Master's Theses and Graduate Research Spring 2014 ANALYZING BIG DATA WITH DECISION TREES Lok Kei Leong Follow this and additional works at:

More information

Lesson Plan. Preparation. Data Mining Basics BIM 1 Business Management & Administration

Lesson Plan. Preparation. Data Mining Basics BIM 1 Business Management & Administration Data Mining Basics BIM 1 Business Management & Administration Lesson Plan Performance Objective The student understands and is able to recall information on data mining basics. Specific Objectives The

More information

Test Effort Estimation Using Neural Network

Test Effort Estimation Using Neural Network J. Software Engineering & Applications, 2010, 3: 331-340 doi:10.4236/jsea.2010.34038 Published Online April 2010 (http://www.scirp.org/journal/jsea) 331 Chintala Abhishek*, Veginati Pavan Kumar, Harish

More information