Artificial Neural Networks written examination

Size: px
Start display at page:

Download "Artificial Neural Networks written examination"

Transcription

1 1 (8) Institutionen för informationsteknologi Olle Gällmo Universitetsadjunkt Adress: Lägerhyddsvägen 2 Box Uppsala Artificial Neural Networks written examination Monday, May 15, Allowed help material: Pen, paper and rubber, dictionary Telefon: Telefax: Hemsida: user.it.uu.se/~crwth Epost: Information Technology Olle Gällmo Lecturer Address: Lägerhyddsvägen 2 Box 337 SE Uppsala SWEDEN Telephone: Telefax: Web site: user.it.uu.se/~crwth Please, answer (in Swedish or English) the following questions to the best of your ability. Any assumptions made, which are not already part of the problem formulation, must be stated clearly in your answer. Write your name on top of each page. Don't forget to hand in the last page! (your answers to question 10) The maximum number of points is 40. To get the grade G (pass) a total of 20 points is required. The grade VG (pass with distinction) requires approximately 30 points, but also depends on the results on the lab course (labs + project). Your teacher will drop in sometime between and to answer questions. In this exam, some concepts may be called by different names than the ones used in the book. Here is a list of useful synonyms and acronyms: Perceptron = summation unit = SU = conventional neuron Binary perceptron = summation unit with binary (step) activation function Multilayer perceptron = MLP = Feedforward network of summation units RBF = Radial Basis Functions Standard Competitive Learning = LVQ-I without a neighbourhood function Objective function = the function to be minimzed or maximized = error function = fitness function Now, sit back, relax and enjoy the exam! Good luck!

2 21 students attended this exam, of which 9 failed, 7 passed (G) and 5 passed with 30 points or more (may become pass with distinction, depending on the results from lab course). The best result was 37 points. 1. Why is it impossible for a single binary perceptron to solve the OR problem?... (2) Because OR is not a linearly separable problem. Perceptrons solve classification tasks by adjusting a hyper plane in the input space (i.e. in 2D, as in this case, a line). Most students got this. Some failed to mention that the discriminant is a hyperplane/line, though, which is the main point here. 21 answers, 15 with max credit, Average: Neural networks require lots of data to be trained properly. If you have too little data (too few input-target pairs) the first thing to try is to get more. However, sometimes this is simply not possible and, then, to split up the few data you have in a training set and a test set might be considered wasteful. Describe how K-fold cross validation can be used to deal with this problem! (Note: This is not early-stopping!)... (3) Split the data into K-sets, of N/K patterns each, where N is the total number of patterns. Train on all but one and test on the one left out. Do that for each of the K sets. Report the average error over the K tests. Alternative: Select N/K patterns at random, train on the rest, test on the ones selected and run this K times. Report result as above. K-Fold cross validation is not an early-stopping technique, though, as was clearly pointed out in the question. 1 point was deducted for answers which did not mention what to do with the K test results. 18 answers, 5 with max credit, Average: 1.9

3 3. What is weight decay? What is it good for and how can it be implemented?... (2) Weight decay is to let each weight in a neural network strive for 0 (in addition to the change given by the training algorithm, of course). There are several reasons for wanting to do this. For example: (most common answer) So that we can remove unecessary weights after training, since they will be very close to 0. By the way, if you do this you should retrain the network afterwards. To avoid numerical problems with too large weights since the weighted sum is in the exponent of the sigmoid, large weights may quickly lead to numerical problems. (related to the previous) Large weights also means that the sigmoids are likely to bottom out in either end, where the derivative is close to 0. This makes the network rigid, since this derivative is multiplied in the weight update formula. So, weight decay gives the network more flexibility and can speed up learning, since it tends to move the weighted sums closer to the region in the sigmoid where the derivative is the largest. Implementation: After updating the weights according to the update rule, update them again by w := (1-e)w where e is the decay rate. Some students associated weight decay with Ant Colony Optimization instead of Neural Networks. Indeed, the decay of pheromones can be viewed as a form of weight decay, but since the concept has not been discussed in those term on this course, only partial credit was given for such answers. 21 answers, 7 with max credit, Average: Write down the back propagation algorithm! For full credit, your description must be clear enough for someone who knows what a multilayer perceptron is, to implement the algorithm.... (5) Considering that so much of this course focus on MLPs and backprop, I was very surprised to see how many students failed this question. Only two students got max credit, only two more were close. most got less than half of that. 20 answers, 2 with max credit, Average:1.9

4 5. How is the hidden layer of a RBF network different from the hidden layer in a MLP? Explain this difference in terms of: a) what the hidden nodes compute when feeding data to the network... (2) MLP hidden nodes compute weighted sums of the input and feed that through a sigmoid. RBF nodes compute the distance between the input vector and the weight vector and feed that through a Gaussian (or similar function). b) how this affects the shape of the discriminant when using the networks for classification... (2) MLP hidden nodes form hyperplanes, RBF nodes form hyperspheres (or hyperellipses) Sidenote: It is not the activation function which decides the shape of the discriminant. The weighted sum forms the hyperplane in MLPs (the sigmoid only decides what to output, given the distance from that hyperplane). Similarly, for RBFs, it is the distance calculation between input vector and weight vector which forms the hypersphere, not the Gaussian. c) how the hidden nodes are trained... (2) MLPs are usually trained by some form of backprop (see Q4). The hidden layer of a RBF network is usually trained by some form of unsupervised learning, e.g. competitive learning or K-Means. Some students got minor deductions for only describing RBF, not comparing to MLP. A few students described the output layer instead of the hidden layer. 21 answers, 1 with max credit, Average: 3.4

5 6. The course book suggests two ways to initialize the weights of unsupervised competitive learning networks: Either (a) initialize weights to small random values sampled from a uniform distribution, or (b) take the first input patterns as initial weight vectors One of these is actually not a very good idea. Which, and why?(3) (a) is the bad one. Initializing like this is to place all nodes (their weight vectors) in a region close to origo. That may be very far from where the data actually is, i.e. where the nodes are needed. This means that the one which happens to be closest from the start is likely to win for all patterns. This is the "winner-take-all" scenario. (b) is better, since the initial positions of the nodes then are close to the data. Some students pointed out that it would be even better to select vectors from the data at random instead of the N first ones. Indeed, the data may be sorted which could lead to all nodes getting intial positions in the same cluster (which may lead to the same problem as in (a) but is not as likely). All students who claimed (b) was better then (a) did so with arguments that either did not hold or which actually are good arguments, for (a)! For example, some students worried that drawing weight vectors from the data may give the network extreme weight values well, if the data is in extreme locations, then the nodes should also be there! 21 answers, 12 with max credit, Average: Explain how weights are updated in Kohonen's Self Organizing Feature Map!... (3) SOFM nodes are organized in a 2D grid (usually). Each node's (i) weight vector is moved towards the input vector (instead of just the winner's, k, as in competitive learning) by an amount which decreases with distance from the winner node on the grid. (i.e. it is not a function of the distance between their weight vectors, but of the distance between node i and k on the grid): w i = ηf i, k)( x w ) ( i f is usually a Gaussian, but the important point is only that it should have its maximum (=1) för node k itself (i=k) and then decrease with distance from k. Just saying that the winner and it's neighbours are updated is not enough. I wanted to know how either in words or formally. 20 answers, 10 with max credit, Average: 1.8

6 8. Most reinforcement learning algorithms include a discount factor, usually denoted by the greek letter γ. What is the main reason for having this factor?... (3) The goal of RL is to maximize the sum of all future rewards. γ is there to ensure that this sum is finite, even if we have an infinite future. Partial credit was given for the answer that γ is there to ensure that blame/credit is given more to recent actions than to earlier ones. This is indeed an effect of γ, but it is a side effect and it is not always wanted. Actually, this side effect is the main disadvantage with discounted RL methods. Whether it is important to blame late decisions more than others, or not, depends on the application and should therefore be expressed in the problem formulation (for example in how rewards are payed), not be builtin into the algorithm! In lab 2, for example, there was nothing in the problem formulation that said that we wanted it to find the shortest route. Q-Learning/Sarsa did, however, find the shortest route because they have this hidden assumption builtin. There are other ways to ensure that the sum is finite, without this side effect. R-Learning, for example. But we have not discussed these kind of algorithms on this course. 19 answers, 6 with max credit, Average: Explain and compare the two evolutionary computing concepts genotype and phenotype.... (3) A genotype is the encoding of a solution to a problem, for example a bit string. A phenotype is the interpretation of the genotype (its manifestation, you might say, or its semantics) necessary in order to evaluate its fitness. Operators ususally work on the genotypic level, while evaluation is done on the phenotypic level. 18 answers, 13 with max credit, Average: Please mark in the table on the next page, which statements are true and which are false, and hand in the table together with your other answers. Each correct answer yields +0.5 points and each incorrect answer yields -0.5 points, so be careful do not guess! (However, the total number of points on this assigment can not become less than zero.)... (10) See comments to each individual question in the table below Average score: 5.5 Max: 8.5

7 Exam ANN Your name:... Statement True False a) If a state in a Hopfield network is stable, then the inverse state is not. False. A state and its reverse (mirror) is one and the same. They have the same energy. Hopfield networks really only store correlation between nodes. 19 answers, 18 correct. b) The main difference between immediate and delayed reinforcement learning is in how often the rewards are received. False. A delayed reinforcement learning task is one where the optimal solution can only be found by associating incoming rewards with a whole sequence of previous actions, instead of just the latest one. However, the reward may very well be received in every time step. 18 answers, 14 correct. c) Temporal difference learning tries to minimize the difference between successive predictions, rather than the difference between the current prediction and the true value. True. This is the main point with TD and that which makes it most different from supervised learning. 12 answers, 9 correct. d) An advantage with gradient descent based methods, such as back propagation, is that they can not get stuck in local minima. False. Gradient descent methods are very likely to get stuck in local minima, since they only move downhill. We can hope that we'll overshoot local minima, but we cannot avoid the problem. 21 answers, 20 correct. e) Weights are update more often in pattern learning than in batch (epoch) learning. True. Pattern learning updates the weights after every pattern presentation. Batch learning only accumulates the weight changes until the whole batch of patterns has been presented once. 20 answers, 19 correct. f) A Hopfield network with symmetric weights and asynchronous updating will always converge to a stable state. True. It may not be the wanted stable state, but it will be stable. 17 answers, 14 correct. g) If a classification task is such that it is possible to separate the classes with a linear discriminant, then a perceptron trained by Rosenblatt's convergence procedure is guaranteed to find it. True. It is even guaranteed to find it in finite time. This is Rosenblatt's Convergence Theorem. 8 answers, 6 correct. h) One difference between (most) evolutionary computation algorithms and (most) neural network training algorithms, is that evolutionary computation algorithms do not require the objective function to be differentiable. True. This is since the objective function in EC is only used for selection, not to compute how to change the parameters in the system. 19 answers, 18 correct. i) Increasing the number of hidden nodes in a multilayer perceptron improves generalization. False. Increasing the number of hidden nodes increases its representational ability, which increases the risk of overtraining (learning all patterns by heart in the extreme case). Some students pointed out, though, that if the initial number of hidden nodes was too small to solve the problem at all, then increasing the number should improve generalization. Up to the point where the number of nodes is exactly the required number of nodes, that is. My mistake, not to consider that. I intended to give all student +0.5 points for this question, whether they answered the question or not, but this would not have changed the grade for any student, so I didn't. 17 answers, 16 correct. j) It is impossible to overtrain a genetic algorithm. False. Any form of learning system, which trains on a training set which is not complete, can be overtrained, i.e. become too specialized on that particular training set. 9 answers, 4 correct.

8 k) In general, radial basis function networks perform better than multilayer perceptrons for problems with many input variables. False. RBF networks have more problems in high dimensionality than MLPs. In general. 16 answers, 15 correct. l) In radial basis function networks, one hidden layer is sufficient. True. In theory this is true also for MLPs (though more hidden layers are very rarely needed). For RBF networks it's not just theory, though, but a very practical rule. 19 answers, 18 correct. m) RPROP (Resilient backprop) only considers the sign of the gradient, not its value, when computing weight changes. True. Instead it adopts the step length on-line, individually for each weight, depending on whether we seem to keep moving downhill, or whether we have overshot a minimum. 20 answers, 19 correct. n) In general, compact input representation are better than distributed ones, when training neural networks. False. Neural networks prefer distributed representations. Neural networks are distributed processing devices, after all, and why should it have to learn how to unpack data which we packed? 12 answers, 11 correct. o) Evolutionary computation algorithms are very unlikely to get stuck in local minima. True, since it is a population method. As long as the population is spread out (maintaining diversity) it is very unlikely that they all get stuck. 18 answers, 17 correct. p) The boolean function (a AND b AND c) OR d can be implemented by a single binary perceptron. True. See Figure below. 19 answers, 14 correct. q) A self organizing feature map can be trained to separate overlapping classes. False. That would be true magic. And even if it could, it would be a failure: The whole point with SOFM is that it should preserve topology, i.e. maintain statistical distributions found in the higher dimensional space. 13 answers, 6 correct. r) Any function represented by a multilayer perceptron with a linear hidden layer, can be represented by a single layer of perceptrons (i.e. without that hidden layer). True. It can be collapsed into a network without that hidden layer. Multiplying x by 2 and then by 5 is equivalent to multiplying x by answers, 12 correct. s) In general, radial basis function networks are better for on-line learning tasks (continuous learning) than multilayer perceptrons. True. This is because a basis function is local. New information from some other part of the space does not affect it and, hence, is not likely to destroy what it has learnt so far. MLPs, on the other hand, tend to destroy the old information when new information comes in. 16 answers, 15 correct. t) Randomizing the order in which patterns are presented to the network has no effect in batch (epoch) learning. True, since the weight updates are accumulated (summed up) until the end of the batch, and + is commutative. 20 answers, 15 correct. a b c d (a AND b AND c) OR d Proof by example to question p)

Lecture 1: Machine Learning Basics

Lecture 1: Machine Learning Basics 1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3

More information

Module 12. Machine Learning. Version 2 CSE IIT, Kharagpur

Module 12. Machine Learning. Version 2 CSE IIT, Kharagpur Module 12 Machine Learning 12.1 Instructional Objective The students should understand the concept of learning systems Students should learn about different aspects of a learning system Students should

More information

Python Machine Learning

Python Machine Learning Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled

More information

Lecture 10: Reinforcement Learning

Lecture 10: Reinforcement Learning Lecture 1: Reinforcement Learning Cognitive Systems II - Machine Learning SS 25 Part III: Learning Programs and Strategies Q Learning, Dynamic Programming Lecture 1: Reinforcement Learning p. Motivation

More information

SARDNET: A Self-Organizing Feature Map for Sequences

SARDNET: A Self-Organizing Feature Map for Sequences SARDNET: A Self-Organizing Feature Map for Sequences Daniel L. James and Risto Miikkulainen Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 dljames,risto~cs.utexas.edu

More information

Softprop: Softmax Neural Network Backpropagation Learning

Softprop: Softmax Neural Network Backpropagation Learning Softprop: Softmax Neural Networ Bacpropagation Learning Michael Rimer Computer Science Department Brigham Young University Provo, UT 84602, USA E-mail: mrimer@axon.cs.byu.edu Tony Martinez Computer Science

More information

A Neural Network GUI Tested on Text-To-Phoneme Mapping

A Neural Network GUI Tested on Text-To-Phoneme Mapping A Neural Network GUI Tested on Text-To-Phoneme Mapping MAARTEN TROMPPER Universiteit Utrecht m.f.a.trompper@students.uu.nl Abstract Text-to-phoneme (T2P) mapping is a necessary step in any speech synthesis

More information

An empirical study of learning speed in backpropagation

An empirical study of learning speed in backpropagation Carnegie Mellon University Research Showcase @ CMU Computer Science Department School of Computer Science 1988 An empirical study of learning speed in backpropagation networks Scott E. Fahlman Carnegie

More information

(Sub)Gradient Descent

(Sub)Gradient Descent (Sub)Gradient Descent CMSC 422 MARINE CARPUAT marine@cs.umd.edu Figures credit: Piyush Rai Logistics Midterm is on Thursday 3/24 during class time closed book/internet/etc, one page of notes. will include

More information

Human Emotion Recognition From Speech

Human Emotion Recognition From Speech RESEARCH ARTICLE OPEN ACCESS Human Emotion Recognition From Speech Miss. Aparna P. Wanare*, Prof. Shankar N. Dandare *(Department of Electronics & Telecommunication Engineering, Sant Gadge Baba Amravati

More information

Axiom 2013 Team Description Paper

Axiom 2013 Team Description Paper Axiom 2013 Team Description Paper Mohammad Ghazanfari, S Omid Shirkhorshidi, Farbod Samsamipour, Hossein Rahmatizadeh Zagheli, Mohammad Mahdavi, Payam Mohajeri, S Abbas Alamolhoda Robotics Scientific Association

More information

Generative models and adversarial training

Generative models and adversarial training Day 4 Lecture 1 Generative models and adversarial training Kevin McGuinness kevin.mcguinness@dcu.ie Research Fellow Insight Centre for Data Analytics Dublin City University What is a generative model?

More information

QuickStroke: An Incremental On-line Chinese Handwriting Recognition System

QuickStroke: An Incremental On-line Chinese Handwriting Recognition System QuickStroke: An Incremental On-line Chinese Handwriting Recognition System Nada P. Matić John C. Platt Λ Tony Wang y Synaptics, Inc. 2381 Bering Drive San Jose, CA 95131, USA Abstract This paper presents

More information

Issues in the Mining of Heart Failure Datasets

Issues in the Mining of Heart Failure Datasets International Journal of Automation and Computing 11(2), April 2014, 162-179 DOI: 10.1007/s11633-014-0778-5 Issues in the Mining of Heart Failure Datasets Nongnuch Poolsawad 1 Lisa Moore 1 Chandrasekhar

More information

The Good Judgment Project: A large scale test of different methods of combining expert predictions

The Good Judgment Project: A large scale test of different methods of combining expert predictions The Good Judgment Project: A large scale test of different methods of combining expert predictions Lyle Ungar, Barb Mellors, Jon Baron, Phil Tetlock, Jaime Ramos, Sam Swift The University of Pennsylvania

More information

Learning Methods for Fuzzy Systems

Learning Methods for Fuzzy Systems Learning Methods for Fuzzy Systems Rudolf Kruse and Andreas Nürnberger Department of Computer Science, University of Magdeburg Universitätsplatz, D-396 Magdeburg, Germany Phone : +49.39.67.876, Fax : +49.39.67.8

More information

Artificial Neural Networks

Artificial Neural Networks Artificial Neural Networks Andres Chavez Math 382/L T/Th 2:00-3:40 April 13, 2010 Chavez2 Abstract The main interest of this paper is Artificial Neural Networks (ANNs). A brief history of the development

More information

INPE São José dos Campos

INPE São José dos Campos INPE-5479 PRE/1778 MONLINEAR ASPECTS OF DATA INTEGRATION FOR LAND COVER CLASSIFICATION IN A NEDRAL NETWORK ENVIRONNENT Maria Suelena S. Barros Valter Rodrigues INPE São José dos Campos 1993 SECRETARIA

More information

I-COMPETERE: Using Applied Intelligence in search of competency gaps in software project managers.

I-COMPETERE: Using Applied Intelligence in search of competency gaps in software project managers. Information Systems Frontiers manuscript No. (will be inserted by the editor) I-COMPETERE: Using Applied Intelligence in search of competency gaps in software project managers. Ricardo Colomo-Palacios

More information

arxiv: v1 [math.at] 10 Jan 2016

arxiv: v1 [math.at] 10 Jan 2016 THE ALGEBRAIC ATIYAH-HIRZEBRUCH SPECTRAL SEQUENCE OF REAL PROJECTIVE SPECTRA arxiv:1601.02185v1 [math.at] 10 Jan 2016 GUOZHEN WANG AND ZHOULI XU Abstract. In this note, we use Curtis s algorithm and the

More information

OPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS

OPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS OPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS Václav Kocian, Eva Volná, Michal Janošek, Martin Kotyrba University of Ostrava Department of Informatics and Computers Dvořákova 7,

More information

Course Outline. Course Grading. Where to go for help. Academic Integrity. EE-589 Introduction to Neural Networks NN 1 EE

Course Outline. Course Grading. Where to go for help. Academic Integrity. EE-589 Introduction to Neural Networks NN 1 EE EE-589 Introduction to Neural Assistant Prof. Dr. Turgay IBRIKCI Room # 305 (322) 338 6868 / 139 Wensdays 9:00-12:00 Course Outline The course is divided in two parts: theory and practice. 1. Theory covers

More information

TD(λ) and Q-Learning Based Ludo Players

TD(λ) and Q-Learning Based Ludo Players TD(λ) and Q-Learning Based Ludo Players Majed Alhajry, Faisal Alvi, Member, IEEE and Moataz Ahmed Abstract Reinforcement learning is a popular machine learning technique whose inherent self-learning ability

More information

Laboratorio di Intelligenza Artificiale e Robotica

Laboratorio di Intelligenza Artificiale e Robotica Laboratorio di Intelligenza Artificiale e Robotica A.A. 2008-2009 Outline 2 Machine Learning Unsupervised Learning Supervised Learning Reinforcement Learning Genetic Algorithms Genetics-Based Machine Learning

More information

Radius STEM Readiness TM

Radius STEM Readiness TM Curriculum Guide Radius STEM Readiness TM While today s teens are surrounded by technology, we face a stark and imminent shortage of graduates pursuing careers in Science, Technology, Engineering, and

More information

Reinforcement Learning by Comparing Immediate Reward

Reinforcement Learning by Comparing Immediate Reward Reinforcement Learning by Comparing Immediate Reward Punit Pandey DeepshikhaPandey Dr. Shishir Kumar Abstract This paper introduces an approach to Reinforcement Learning Algorithm by comparing their immediate

More information

Knowledge Transfer in Deep Convolutional Neural Nets

Knowledge Transfer in Deep Convolutional Neural Nets Knowledge Transfer in Deep Convolutional Neural Nets Steven Gutstein, Olac Fuentes and Eric Freudenthal Computer Science Department University of Texas at El Paso El Paso, Texas, 79968, U.S.A. Abstract

More information

CS Machine Learning

CS Machine Learning CS 478 - Machine Learning Projects Data Representation Basic testing and evaluation schemes CS 478 Data and Testing 1 Programming Issues l Program in any platform you want l Realize that you will be doing

More information

AGS THE GREAT REVIEW GAME FOR PRE-ALGEBRA (CD) CORRELATED TO CALIFORNIA CONTENT STANDARDS

AGS THE GREAT REVIEW GAME FOR PRE-ALGEBRA (CD) CORRELATED TO CALIFORNIA CONTENT STANDARDS AGS THE GREAT REVIEW GAME FOR PRE-ALGEBRA (CD) CORRELATED TO CALIFORNIA CONTENT STANDARDS 1 CALIFORNIA CONTENT STANDARDS: Chapter 1 ALGEBRA AND WHOLE NUMBERS Algebra and Functions 1.4 Students use algebraic

More information

Unsupervised Learning of Word Semantic Embedding using the Deep Structured Semantic Model

Unsupervised Learning of Word Semantic Embedding using the Deep Structured Semantic Model Unsupervised Learning of Word Semantic Embedding using the Deep Structured Semantic Model Xinying Song, Xiaodong He, Jianfeng Gao, Li Deng Microsoft Research, One Microsoft Way, Redmond, WA 98052, U.S.A.

More information

Exploration. CS : Deep Reinforcement Learning Sergey Levine

Exploration. CS : Deep Reinforcement Learning Sergey Levine Exploration CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 4 due on Wednesday 2. Project proposal feedback sent Today s Lecture 1. What is exploration? Why is it a problem?

More information

System Implementation for SemEval-2017 Task 4 Subtask A Based on Interpolated Deep Neural Networks

System Implementation for SemEval-2017 Task 4 Subtask A Based on Interpolated Deep Neural Networks System Implementation for SemEval-2017 Task 4 Subtask A Based on Interpolated Deep Neural Networks 1 Tzu-Hsuan Yang, 2 Tzu-Hsuan Tseng, and 3 Chia-Ping Chen Department of Computer Science and Engineering

More information

Learning Structural Correspondences Across Different Linguistic Domains with Synchronous Neural Language Models

Learning Structural Correspondences Across Different Linguistic Domains with Synchronous Neural Language Models Learning Structural Correspondences Across Different Linguistic Domains with Synchronous Neural Language Models Stephan Gouws and GJ van Rooyen MIH Medialab, Stellenbosch University SOUTH AFRICA {stephan,gvrooyen}@ml.sun.ac.za

More information

Test Effort Estimation Using Neural Network

Test Effort Estimation Using Neural Network J. Software Engineering & Applications, 2010, 3: 331-340 doi:10.4236/jsea.2010.34038 Published Online April 2010 (http://www.scirp.org/journal/jsea) 331 Chintala Abhishek*, Veginati Pavan Kumar, Harish

More information

Discriminative Learning of Beam-Search Heuristics for Planning

Discriminative Learning of Beam-Search Heuristics for Planning Discriminative Learning of Beam-Search Heuristics for Planning Yuehua Xu School of EECS Oregon State University Corvallis,OR 97331 xuyu@eecs.oregonstate.edu Alan Fern School of EECS Oregon State University

More information

Analysis of Hybrid Soft and Hard Computing Techniques for Forex Monitoring Systems

Analysis of Hybrid Soft and Hard Computing Techniques for Forex Monitoring Systems Analysis of Hybrid Soft and Hard Computing Techniques for Forex Monitoring Systems Ajith Abraham School of Business Systems, Monash University, Clayton, Victoria 3800, Australia. Email: ajith.abraham@ieee.org

More information

Learning to Schedule Straight-Line Code

Learning to Schedule Straight-Line Code Learning to Schedule Straight-Line Code Eliot Moss, Paul Utgoff, John Cavazos Doina Precup, Darko Stefanović Dept. of Comp. Sci., Univ. of Mass. Amherst, MA 01003 Carla Brodley, David Scheeff Sch. of Elec.

More information

OCR for Arabic using SIFT Descriptors With Online Failure Prediction

OCR for Arabic using SIFT Descriptors With Online Failure Prediction OCR for Arabic using SIFT Descriptors With Online Failure Prediction Andrey Stolyarenko, Nachum Dershowitz The Blavatnik School of Computer Science Tel Aviv University Tel Aviv, Israel Email: stloyare@tau.ac.il,

More information

Phonetic- and Speaker-Discriminant Features for Speaker Recognition. Research Project

Phonetic- and Speaker-Discriminant Features for Speaker Recognition. Research Project Phonetic- and Speaker-Discriminant Features for Speaker Recognition by Lara Stoll Research Project Submitted to the Department of Electrical Engineering and Computer Sciences, University of California

More information

Getting Started with Deliberate Practice

Getting Started with Deliberate Practice Getting Started with Deliberate Practice Most of the implementation guides so far in Learning on Steroids have focused on conceptual skills. Things like being able to form mental images, remembering facts

More information

Evolution of Symbolisation in Chimpanzees and Neural Nets

Evolution of Symbolisation in Chimpanzees and Neural Nets Evolution of Symbolisation in Chimpanzees and Neural Nets Angelo Cangelosi Centre for Neural and Adaptive Systems University of Plymouth (UK) a.cangelosi@plymouth.ac.uk Introduction Animal communication

More information

Framewise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures

Framewise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures Framewise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures Alex Graves and Jürgen Schmidhuber IDSIA, Galleria 2, 6928 Manno-Lugano, Switzerland TU Munich, Boltzmannstr.

More information

Calibration of Confidence Measures in Speech Recognition

Calibration of Confidence Measures in Speech Recognition Submitted to IEEE Trans on Audio, Speech, and Language, July 2010 1 Calibration of Confidence Measures in Speech Recognition Dong Yu, Senior Member, IEEE, Jinyu Li, Member, IEEE, Li Deng, Fellow, IEEE

More information

Speaker Identification by Comparison of Smart Methods. Abstract

Speaker Identification by Comparison of Smart Methods. Abstract Journal of mathematics and computer science 10 (2014), 61-71 Speaker Identification by Comparison of Smart Methods Ali Mahdavi Meimand Amin Asadi Majid Mohamadi Department of Electrical Department of Computer

More information

ISFA2008U_120 A SCHEDULING REINFORCEMENT LEARNING ALGORITHM

ISFA2008U_120 A SCHEDULING REINFORCEMENT LEARNING ALGORITHM Proceedings of 28 ISFA 28 International Symposium on Flexible Automation Atlanta, GA, USA June 23-26, 28 ISFA28U_12 A SCHEDULING REINFORCEMENT LEARNING ALGORITHM Amit Gil, Helman Stern, Yael Edan, and

More information

Proof Theory for Syntacticians

Proof Theory for Syntacticians Department of Linguistics Ohio State University Syntax 2 (Linguistics 602.02) January 5, 2012 Logics for Linguistics Many different kinds of logic are directly applicable to formalizing theories in syntax

More information

Lecture 1: Basic Concepts of Machine Learning

Lecture 1: Basic Concepts of Machine Learning Lecture 1: Basic Concepts of Machine Learning Cognitive Systems - Machine Learning Ute Schmid (lecture) Johannes Rabold (practice) Based on slides prepared March 2005 by Maximilian Röglinger, updated 2010

More information

Statewide Framework Document for:

Statewide Framework Document for: Statewide Framework Document for: 270301 Standards may be added to this document prior to submission, but may not be removed from the framework to meet state credit equivalency requirements. Performance

More information

Mathematics process categories

Mathematics process categories Mathematics process categories All of the UK curricula define multiple categories of mathematical proficiency that require students to be able to use and apply mathematics, beyond simple recall of facts

More information

Evolutive Neural Net Fuzzy Filtering: Basic Description

Evolutive Neural Net Fuzzy Filtering: Basic Description Journal of Intelligent Learning Systems and Applications, 2010, 2: 12-18 doi:10.4236/jilsa.2010.21002 Published Online February 2010 (http://www.scirp.org/journal/jilsa) Evolutive Neural Net Fuzzy Filtering:

More information

WHEN THERE IS A mismatch between the acoustic

WHEN THERE IS A mismatch between the acoustic 808 IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 14, NO. 3, MAY 2006 Optimization of Temporal Filters for Constructing Robust Features in Speech Recognition Jeih-Weih Hung, Member,

More information

Improving Conceptual Understanding of Physics with Technology

Improving Conceptual Understanding of Physics with Technology INTRODUCTION Improving Conceptual Understanding of Physics with Technology Heidi Jackman Research Experience for Undergraduates, 1999 Michigan State University Advisors: Edwin Kashy and Michael Thoennessen

More information

Model Ensemble for Click Prediction in Bing Search Ads

Model Ensemble for Click Prediction in Bing Search Ads Model Ensemble for Click Prediction in Bing Search Ads Xiaoliang Ling Microsoft Bing xiaoling@microsoft.com Hucheng Zhou Microsoft Research huzho@microsoft.com Weiwei Deng Microsoft Bing dedeng@microsoft.com

More information

Device Independence and Extensibility in Gesture Recognition

Device Independence and Extensibility in Gesture Recognition Device Independence and Extensibility in Gesture Recognition Jacob Eisenstein, Shahram Ghandeharizadeh, Leana Golubchik, Cyrus Shahabi, Donghui Yan, Roger Zimmermann Department of Computer Science University

More information

AMULTIAGENT system [1] can be defined as a group of

AMULTIAGENT system [1] can be defined as a group of 156 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART C: APPLICATIONS AND REVIEWS, VOL. 38, NO. 2, MARCH 2008 A Comprehensive Survey of Multiagent Reinforcement Learning Lucian Buşoniu, Robert Babuška,

More information

A Reinforcement Learning Variant for Control Scheduling

A Reinforcement Learning Variant for Control Scheduling A Reinforcement Learning Variant for Control Scheduling Aloke Guha Honeywell Sensor and System Development Center 3660 Technology Drive Minneapolis MN 55417 Abstract We present an algorithm based on reinforcement

More information

This scope and sequence assumes 160 days for instruction, divided among 15 units.

This scope and sequence assumes 160 days for instruction, divided among 15 units. In previous grades, students learned strategies for multiplication and division, developed understanding of structure of the place value system, and applied understanding of fractions to addition and subtraction

More information

ENME 605 Advanced Control Systems, Fall 2015 Department of Mechanical Engineering

ENME 605 Advanced Control Systems, Fall 2015 Department of Mechanical Engineering ENME 605 Advanced Control Systems, Fall 2015 Department of Mechanical Engineering Lecture Details Instructor Course Objectives Tuesday and Thursday, 4:00 pm to 5:15 pm Information Technology and Engineering

More information

TABLE OF CONTENTS TABLE OF CONTENTS COVER PAGE HALAMAN PENGESAHAN PERNYATAAN NASKAH SOAL TUGAS AKHIR ACKNOWLEDGEMENT FOREWORD

TABLE OF CONTENTS TABLE OF CONTENTS COVER PAGE HALAMAN PENGESAHAN PERNYATAAN NASKAH SOAL TUGAS AKHIR ACKNOWLEDGEMENT FOREWORD TABLE OF CONTENTS TABLE OF CONTENTS COVER PAGE HALAMAN PENGESAHAN PERNYATAAN NASKAH SOAL TUGAS AKHIR ACKNOWLEDGEMENT FOREWORD TABLE OF CONTENTS LIST OF FIGURES LIST OF TABLES LIST OF APPENDICES LIST OF

More information

Probability and Game Theory Course Syllabus

Probability and Game Theory Course Syllabus Probability and Game Theory Course Syllabus DATE ACTIVITY CONCEPT Sunday Learn names; introduction to course, introduce the Battle of the Bismarck Sea as a 2-person zero-sum game. Monday Day 1 Pre-test

More information

Knowledge-Based - Systems

Knowledge-Based - Systems Knowledge-Based - Systems ; Rajendra Arvind Akerkar Chairman, Technomathematics Research Foundation and Senior Researcher, Western Norway Research institute Priti Srinivas Sajja Sardar Patel University

More information

Georgetown University at TREC 2017 Dynamic Domain Track

Georgetown University at TREC 2017 Dynamic Domain Track Georgetown University at TREC 2017 Dynamic Domain Track Zhiwen Tang Georgetown University zt79@georgetown.edu Grace Hui Yang Georgetown University huiyang@cs.georgetown.edu Abstract TREC Dynamic Domain

More information

Machine Learning and Data Mining. Ensembles of Learners. Prof. Alexander Ihler

Machine Learning and Data Mining. Ensembles of Learners. Prof. Alexander Ihler Machine Learning and Data Mining Ensembles of Learners Prof. Alexander Ihler Ensemble methods Why learn one classifier when you can learn many? Ensemble: combine many predictors (Weighted) combina

More information

An OO Framework for building Intelligence and Learning properties in Software Agents

An OO Framework for building Intelligence and Learning properties in Software Agents An OO Framework for building Intelligence and Learning properties in Software Agents José A. R. P. Sardinha, Ruy L. Milidiú, Carlos J. P. Lucena, Patrick Paranhos Abstract Software agents are defined as

More information

Analysis of Speech Recognition Models for Real Time Captioning and Post Lecture Transcription

Analysis of Speech Recognition Models for Real Time Captioning and Post Lecture Transcription Analysis of Speech Recognition Models for Real Time Captioning and Post Lecture Transcription Wilny Wilson.P M.Tech Computer Science Student Thejus Engineering College Thrissur, India. Sindhu.S Computer

More information

Laboratorio di Intelligenza Artificiale e Robotica

Laboratorio di Intelligenza Artificiale e Robotica Laboratorio di Intelligenza Artificiale e Robotica A.A. 2008-2009 Outline 2 Machine Learning Unsupervised Learning Supervised Learning Reinforcement Learning Genetic Algorithms Genetics-Based Machine Learning

More information

arxiv: v1 [cs.lg] 15 Jun 2015

arxiv: v1 [cs.lg] 15 Jun 2015 Dual Memory Architectures for Fast Deep Learning of Stream Data via an Online-Incremental-Transfer Strategy arxiv:1506.04477v1 [cs.lg] 15 Jun 2015 Sang-Woo Lee Min-Oh Heo School of Computer Science and

More information

Deep search. Enhancing a search bar using machine learning. Ilgün Ilgün & Cedric Reichenbach

Deep search. Enhancing a search bar using machine learning. Ilgün Ilgün & Cedric Reichenbach #BaselOne7 Deep search Enhancing a search bar using machine learning Ilgün Ilgün & Cedric Reichenbach We are not researchers Outline I. Periscope: A search tool II. Goals III. Deep learning IV. Applying

More information

arxiv: v1 [cs.cv] 10 May 2017

arxiv: v1 [cs.cv] 10 May 2017 Inferring and Executing Programs for Visual Reasoning Justin Johnson 1 Bharath Hariharan 2 Laurens van der Maaten 2 Judy Hoffman 1 Li Fei-Fei 1 C. Lawrence Zitnick 2 Ross Girshick 2 1 Stanford University

More information

Learning From the Past with Experiment Databases

Learning From the Past with Experiment Databases Learning From the Past with Experiment Databases Joaquin Vanschoren 1, Bernhard Pfahringer 2, and Geoff Holmes 2 1 Computer Science Dept., K.U.Leuven, Leuven, Belgium 2 Computer Science Dept., University

More information

Classification Using ANN: A Review

Classification Using ANN: A Review International Journal of Computational Intelligence Research ISSN 0973-1873 Volume 13, Number 7 (2017), pp. 1811-1820 Research India Publications http://www.ripublication.com Classification Using ANN:

More information

Syntactic systematicity in sentence processing with a recurrent self-organizing network

Syntactic systematicity in sentence processing with a recurrent self-organizing network Syntactic systematicity in sentence processing with a recurrent self-organizing network Igor Farkaš,1 Department of Applied Informatics, Comenius University Mlynská dolina, 842 48 Bratislava, Slovak Republic

More information

Mathematics subject curriculum

Mathematics subject curriculum Mathematics subject curriculum Dette er ei omsetjing av den fastsette læreplanteksten. Læreplanen er fastsett på Nynorsk Established as a Regulation by the Ministry of Education and Research on 24 June

More information

Go fishing! Responsibility judgments when cooperation breaks down

Go fishing! Responsibility judgments when cooperation breaks down Go fishing! Responsibility judgments when cooperation breaks down Kelsey Allen (krallen@mit.edu), Julian Jara-Ettinger (jjara@mit.edu), Tobias Gerstenberg (tger@mit.edu), Max Kleiman-Weiner (maxkw@mit.edu)

More information

Class-Discriminative Weighted Distortion Measure for VQ-Based Speaker Identification

Class-Discriminative Weighted Distortion Measure for VQ-Based Speaker Identification Class-Discriminative Weighted Distortion Measure for VQ-Based Speaker Identification Tomi Kinnunen and Ismo Kärkkäinen University of Joensuu, Department of Computer Science, P.O. Box 111, 80101 JOENSUU,

More information

Learning Disability Functional Capacity Evaluation. Dear Doctor,

Learning Disability Functional Capacity Evaluation. Dear Doctor, Dear Doctor, I have been asked to formulate a vocational opinion regarding NAME s employability in light of his/her learning disability. To assist me with this evaluation I would appreciate if you can

More information

A New Perspective on Combining GMM and DNN Frameworks for Speaker Adaptation

A New Perspective on Combining GMM and DNN Frameworks for Speaker Adaptation A New Perspective on Combining GMM and DNN Frameworks for Speaker Adaptation SLSP-2016 October 11-12 Natalia Tomashenko 1,2,3 natalia.tomashenko@univ-lemans.fr Yuri Khokhlov 3 khokhlov@speechpro.com Yannick

More information

Using focal point learning to improve human machine tacit coordination

Using focal point learning to improve human machine tacit coordination DOI 10.1007/s10458-010-9126-5 Using focal point learning to improve human machine tacit coordination InonZuckerman SaritKraus Jeffrey S. Rosenschein The Author(s) 2010 Abstract We consider an automated

More information

Data Integration through Clustering and Finding Statistical Relations - Validation of Approach

Data Integration through Clustering and Finding Statistical Relations - Validation of Approach Data Integration through Clustering and Finding Statistical Relations - Validation of Approach Marek Jaszuk, Teresa Mroczek, and Barbara Fryc University of Information Technology and Management, ul. Sucharskiego

More information

The Strong Minimalist Thesis and Bounded Optimality

The Strong Minimalist Thesis and Bounded Optimality The Strong Minimalist Thesis and Bounded Optimality DRAFT-IN-PROGRESS; SEND COMMENTS TO RICKL@UMICH.EDU Richard L. Lewis Department of Psychology University of Michigan 27 March 2010 1 Purpose of this

More information

Data Fusion Through Statistical Matching

Data Fusion Through Statistical Matching A research and education initiative at the MIT Sloan School of Management Data Fusion Through Statistical Matching Paper 185 Peter Van Der Puttan Joost N. Kok Amar Gupta January 2002 For more information,

More information

Syllabus for CHEM 4660 Introduction to Computational Chemistry Spring 2010

Syllabus for CHEM 4660 Introduction to Computational Chemistry Spring 2010 Instructor: Dr. Angela Syllabus for CHEM 4660 Introduction to Computational Chemistry Office Hours: Mondays, 1:00 p.m. 3:00 p.m.; 5:00 6:00 p.m. Office: Chemistry 205C Office Phone: (940) 565-4296 E-mail:

More information

Major Milestones, Team Activities, and Individual Deliverables

Major Milestones, Team Activities, and Individual Deliverables Major Milestones, Team Activities, and Individual Deliverables Milestone #1: Team Semester Proposal Your team should write a proposal that describes project objectives, existing relevant technology, engineering

More information

Modeling function word errors in DNN-HMM based LVCSR systems

Modeling function word errors in DNN-HMM based LVCSR systems Modeling function word errors in DNN-HMM based LVCSR systems Melvin Jose Johnson Premkumar, Ankur Bapna and Sree Avinash Parchuri Department of Computer Science Department of Electrical Engineering Stanford

More information

A Case Study: News Classification Based on Term Frequency

A Case Study: News Classification Based on Term Frequency A Case Study: News Classification Based on Term Frequency Petr Kroha Faculty of Computer Science University of Technology 09107 Chemnitz Germany kroha@informatik.tu-chemnitz.de Ricardo Baeza-Yates Center

More information

AUTOMATIC DETECTION OF PROLONGED FRICATIVE PHONEMES WITH THE HIDDEN MARKOV MODELS APPROACH 1. INTRODUCTION

AUTOMATIC DETECTION OF PROLONGED FRICATIVE PHONEMES WITH THE HIDDEN MARKOV MODELS APPROACH 1. INTRODUCTION JOURNAL OF MEDICAL INFORMATICS & TECHNOLOGIES Vol. 11/2007, ISSN 1642-6037 Marek WIŚNIEWSKI *, Wiesława KUNISZYK-JÓŹKOWIAK *, Elżbieta SMOŁKA *, Waldemar SUSZYŃSKI * HMM, recognition, speech, disorders

More information

Automatic Pronunciation Checker

Automatic Pronunciation Checker Institut für Technische Informatik und Kommunikationsnetze Eidgenössische Technische Hochschule Zürich Swiss Federal Institute of Technology Zurich Ecole polytechnique fédérale de Zurich Politecnico federale

More information

Testing A Moving Target: How Do We Test Machine Learning Systems? Peter Varhol Technology Strategy Research, USA

Testing A Moving Target: How Do We Test Machine Learning Systems? Peter Varhol Technology Strategy Research, USA Testing A Moving Target: How Do We Test Machine Learning Systems? Peter Varhol Technology Strategy Research, USA Testing a Moving Target How Do We Test Machine Learning Systems? Peter Varhol, Technology

More information

Language Acquisition Fall 2010/Winter Lexical Categories. Afra Alishahi, Heiner Drenhaus

Language Acquisition Fall 2010/Winter Lexical Categories. Afra Alishahi, Heiner Drenhaus Language Acquisition Fall 2010/Winter 2011 Lexical Categories Afra Alishahi, Heiner Drenhaus Computational Linguistics and Phonetics Saarland University Children s Sensitivity to Lexical Categories Look,

More information

HIERARCHICAL DEEP LEARNING ARCHITECTURE FOR 10K OBJECTS CLASSIFICATION

HIERARCHICAL DEEP LEARNING ARCHITECTURE FOR 10K OBJECTS CLASSIFICATION HIERARCHICAL DEEP LEARNING ARCHITECTURE FOR 10K OBJECTS CLASSIFICATION Atul Laxman Katole 1, Krishna Prasad Yellapragada 1, Amish Kumar Bedi 1, Sehaj Singh Kalra 1 and Mynepalli Siva Chaitanya 1 1 Samsung

More information

success. It will place emphasis on:

success. It will place emphasis on: 1 First administered in 1926, the SAT was created to democratize access to higher education for all students. Today the SAT serves as both a measure of students college readiness and as a valid and reliable

More information

An Introduction to Simio for Beginners

An Introduction to Simio for Beginners An Introduction to Simio for Beginners C. Dennis Pegden, Ph.D. This white paper is intended to introduce Simio to a user new to simulation. It is intended for the manufacturing engineer, hospital quality

More information

Seminar - Organic Computing

Seminar - Organic Computing Seminar - Organic Computing Self-Organisation of OC-Systems Markus Franke 25.01.2006 Typeset by FoilTEX Timetable 1. Overview 2. Characteristics of SO-Systems 3. Concern with Nature 4. Design-Concepts

More information

Distributed Learning of Multilingual DNN Feature Extractors using GPUs

Distributed Learning of Multilingual DNN Feature Extractors using GPUs Distributed Learning of Multilingual DNN Feature Extractors using GPUs Yajie Miao, Hao Zhang, Florian Metze Language Technologies Institute, School of Computer Science, Carnegie Mellon University Pittsburgh,

More information

Executive Guide to Simulation for Health

Executive Guide to Simulation for Health Executive Guide to Simulation for Health Simulation is used by Healthcare and Human Service organizations across the World to improve their systems of care and reduce costs. Simulation offers evidence

More information

Software Maintenance

Software Maintenance 1 What is Software Maintenance? Software Maintenance is a very broad activity that includes error corrections, enhancements of capabilities, deletion of obsolete capabilities, and optimization. 2 Categories

More information

Attributed Social Network Embedding

Attributed Social Network Embedding JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, MAY 2017 1 Attributed Social Network Embedding arxiv:1705.04969v1 [cs.si] 14 May 2017 Lizi Liao, Xiangnan He, Hanwang Zhang, and Tat-Seng Chua Abstract Embedding

More information

Mathematics. Mathematics

Mathematics. Mathematics Mathematics Program Description Successful completion of this major will assure competence in mathematics through differential and integral calculus, providing an adequate background for employment in

More information

Kamaldeep Kaur University School of Information Technology GGS Indraprastha University Delhi

Kamaldeep Kaur University School of Information Technology GGS Indraprastha University Delhi Soft Computing Approaches for Prediction of Software Maintenance Effort Dr. Arvinder Kaur University School of Information Technology GGS Indraprastha University Delhi Kamaldeep Kaur University School

More information

Experiments with SMS Translation and Stochastic Gradient Descent in Spanish Text Author Profiling

Experiments with SMS Translation and Stochastic Gradient Descent in Spanish Text Author Profiling Experiments with SMS Translation and Stochastic Gradient Descent in Spanish Text Author Profiling Notebook for PAN at CLEF 2013 Andrés Alfonso Caurcel Díaz 1 and José María Gómez Hidalgo 2 1 Universidad

More information