Learning of Open-Loop Neural Networks with Improved Error Backpropagation Methods
|
|
- Mildred Carroll
- 5 years ago
- Views:
Transcription
1 Learning of Open-Loop Neural Networks with Improved Error Backpropagation Methods J. Pihler Abstract The paper describes learning of artificial neural networks with improved error backpropagation methods. Artificial neural networks were included into the algorithm of differential protection in order to recognise patterns of transformer's inrush current, with the purpose of improving the reliability of the protection system. The quality of improved learning methods can be assessed by comparing individual methods, number of hidden neuron layers, number of neurons in each individual layer, number of learning epochs, learning time, learning error and test error are varied. Learned neural networks were tested with patterns that were used for learning, as well as with patterns that were not used in the learning process. Keywords Neural network, Error backpropagation method, Improved learning methods, Transformer inrush. A I. INTRODUCTION rtificial neural network (ANN) represents a tool for solving those real problems classical analytical methods are not sufficient or the problem cannot be further generalised. In electric power systems, which are conditioned by two additional criteria of security and reliability of operation, it is used in taking the final decisions. ANN consists of a large number of neurons that are mutually connected and process data in parallel way with regard to dynamic condition of the neural network and with regard to its external inputs. Since it is by learning able to adapt to input information and given requirements, it is classified among adaptive systems. Learning is also related to the characteristics of associativity and generalisation. ANN is a robust system, since some neurons (process units) can be removed and the network will still correctly operate; only the results will be slightly worse. Characteristics of robustness, learning, associativity and generalisation give ANN a higher degree of flexibility. In electric power engineering open-loop ANN with the belonging error backpropagation learning algorithm are most commonly used [1]. Such neural networks are after the successful learning able to provide reasonable answers to the input data they have never encountered before, to approximate functions with a finite number of discontinuities, and to sort input vectors in a user-defined way. However, the basic J. Pihler is with the University of Maribor, Faculty of Electrical Engineering and Computer Sciences, Slovenia ( joze.pihler@unimb.si). gradient method of learning algorithm with error backpropagation, i.e. gradient descent method, in many cases does not converge to the solution fast enough and often even does not reach it. In order to accelerate and to improve efficiency in learning open-loop neural networks, the authors [], [3] and others developed improved learning methods with error backpropagation, which can converge even 100-times faster. In the second section of the paper an ANN is presented. The third section gives an overview of improved learning methods. The last, fourth section brings a comparison of these methods used in an actual example of power transformer inrush current recognition with ANN [4], [6]. As it is well known, in the case of unloaded transformer switching on, current in the primary winding can rise up to several times its nominal value. Protective relay would operate, although a fault did not occur. To avoid this problem, elements of artificial intelligence artificial neural networks are included into the existing algorithms of transformer protection. II. ARTIFICIAL NEURAL NETWORK ANN is a parallel information distribution structure, composed of processing elements neurons that are mutually connected with signal connections. Limits of capabilities of artificial neuron networks are set much above the capabilities of a single neuron. In the vast majority of applications the so called "feedforward" neural networks are used. The term topology in this kind of neural networks comprises the number of network layers and the number of neurons in these layers. We distinguish between the networks having one, two or more layers of neurons. Typical representatives of open-loop ANN are single-layer and multilayer perceptrons. A single-layer perceptron can only be used for simple examples of classification of patterns in a plane. Much wider area of use has multilayer perceptrons, which actually represent an expansion of single-layer perceptron with output layer and several hidden neuron layers. Outputs from the first neuron layer are at the same time inputs to the second layer and outputs from the second layer are inputs to the third neuron layer. Outputs from the third layer can be written in the matrix form [ 3] [ 3] [ ] [ ] [ 1] [ 1] y = F W F W F W x, (1) x is vector of inputs to ANN, y is vector of outputs 153
2 from ANN, W [n] are matrices of weights of individual layers, while F [n] is matrix of transfer functions of individual neutron layers. III. LEARNING OF ANN WITH IMPROVED METHODS Learning of ANN is an optimisation of weights with regard to a certain objective function. In this learning method the desired output from the neural network is given together with each input value. Thus it is possible to exactly determine the transformation of outputs to inputs that has to be performed by the ANN. Controlled learning is in principle implementation of optimisation methods to ANN. In addition to the parameters (weights), ANN also has inputs into the networks which are also changing. These inputs represent the learning set which should have as much characteristics of the total set as possible (of all possible inputs). The function that is minimised is a multiparameter function of error of one learning repetition with outputs from the learning set. Method of error backpropagation is used like learning rule in multilayer perceptrons. This method enables calculation of errors in individual hidden layers on the basis of error of the ANN output layer. In Fig. 1 it is illustrated on the example of three-layer ANN. Fig. 1 composition of three-layer ANN with error backpropagation algorithm The first layer has R inputs, weighted with weights W [1] and connected with S 1 neurons, while the second and third layer contain S and S 3 neurons, respectively, with adequately weighted connections. Input signals x i, (i = 1,...R) and desired output signals d j, (j = 1, S 3 ) participate in the learning process. The task of the learning process is adaptation of all ANN weights W [n] (n=1,,3) in such a way that the deviation between the desired outputs d and actual outputs y in the average of all p learning patterns will be minimal. For minimisation of the function of the sum of squared errors standard gradient procedure, i.e. gradient descent method was used [3]. The total squared error is defined by the equation p j = 1 S 1 3 E = ( d y ). () The basic learning procedure with error backpropagation principle consists of the following sequence: initialisation of all [ n] weights w ij and thresholds b ij, calculation of outputs from all neurons for all input patterns (so called "feed-forward" calculation), definition of desired outputs and calculation of [ n] local error δ for all layers (back-propagation calculation), j adaptation of weights and calculation of new weights. The above described steps are repeated until the sum of squared errors E reaches pre-prescribed value and ANN converges, or the maximum number of learning epochs is reached. One learning epoch comprises one calculation of outputs from all layers of neurons, calculation of the sum of squared errors E, backpropagation calculation of partial errorsδ, calculation of changed weights, as well as new weights and thresholds for the next epoch. A. Improved method of error backpropagation momentum gradient method Weakness of the classical error backpropagation algorithm with gradient method lies in the fact that in some combinations of initial values of weights learning of the neural network can end in local instead of global minimum. To avoid the local minimum, the change of weights is defined in discrete form as [ n]( k + 1) [ n]( k ) [ n]( k ) [ n]( k ) ij c ij ( 1 c ) η δ j i w = m w + m x, (3) wij is change of weights, m c momentum constant, η learning constant, k calculation step, δ j partial error in backpropagation calculation and x i inputs to the neural network.. B. Learning with adaptive constant Learning constant in the standard learning method with gradient descent remains unchanged during learning. If too high learning constant is chosen, it is possible that oscillations of the sum of squared errors depending on the number of learning epochs occur. On the other hand, if the selected learning constant is too low, the time to converge may become too long. Therefore the most optimal approach is to vary the learning constant during the learning process. The basic idea of this method is in the use of gradient procedure for calculation of two new points instead of one [3]. The point with lower error is then used in the next iteration. C. Jumping method with error backpropagation Multilayer neural networks usually use sigmoid functions in hidden layers. These functions convert infinite values at the input in the finite area at the output. Increasing of input values must bring their gradient towards zero value. This causes a problem when sigmoid functions are used in learning of neural networks with gradient descent method, since the value of gradient may be very low. The result is too small change of weights, although they might be far from the optimum values. The principle of jumping method with error backpropagation is to remove these negative effects of partial derivatives values. Only the sign of derivative is used to define the direction in which the weights will be changed [3]. The absolute value of derivative does not have any effect on the change of weights. If the sign in two iterations remains the 154
3 same, learning is accelerated. Contrary, it is decelerated if the sign in two iterations changes. If derivative is zero, learning constant remains unchanged. D. Conjugate gradient method The basic gradient descent method changes weights in the direction of the steepest descent (in the negative direction of the gradient). This is the direction of the steepest descent of the function. Nevertheless, the fact that the function in this direction has the steepest descent, does not necessarily lead to the fastest convergence. In the conjugate gradient method [] a searching function is used alongside related (derived) paths, which enable faster convergence than in the direction of the steepest descent. E. Pseudo-Newton methods Newton method is an alternative to conjugate gradient method for quick optimisation. It belongs to the group of second order algorithms, which consider more data on the form of error function than only the size of gradient [3]. Second order methods use quadratic approximation of the error ( k ) function. If w is weight vector in k th iteration, then the new vector of weights is ( k + 1) ( k ) 1 k E = w w H E w ( w) ( w) w, (4) - Hessian matrix of partial derivatives of second order error function [3]. Unfortunately, calculation of Hessian matrix for open-loop neural network is complicated and time consuming. For this reason inverted Hessian matrix is used. Such a method is called pseudo-newton method [3]. The non-elements are all set to zero and only the diagonal elements are calculated. In this case the equation of weights vector is generalised: E w ( k + 1) ( k ) wi wi = wi E w wi. (5) Although pseudo-newton method converges in some iteration, it requires more computation in each iteration than conjugate gradient method. Simplified Hessian matrix of dimension n x n, n is number of weights and thresholds in the network, needs to be saved in individual iterations. For larger networks is therefore more convenient to use jumping method or conjugate gradient method. F. One-step secant method From the second-order information only one-dimensional minimisation steps and information on curvature E in the direction of change, obtained from current and previous partial derivative of E in this direction, are used. Secant step method is based on independent steps of optimisation of individual weights. It uses quadratic one-dimensional approximation of the error function. Change of individual weight in k th step is defined as ( k w ) E ( k ) ( k 1) wi wi = wi k 1 k E w E w wi wi, (6) it is assumed that E is calculated in steps (k-1) and k with the use of change of weight obtained from w k 1 i previous secant or standard gradient method. G. Levenberg-Marguardt method Levenberg-Marguardt method is, similarly as pseudo- Newton method, designed to reach the second-order learning speed without the necessity to compute Hessian matrix [5]. When the minimised function (equation ()) has the form of sum of squares (typical for learning of open-loop networks), it can be written in the following form: T E = e e, (7) T = e11 es31 e1 es3 e1 p e S3 p e L L L L, e = d y, k = 1, K, S, p number of learning cases, S 3 - number of 3 neurons in the output layer. e is error vector (for all p). Values of new weights are calculated using the following equation: ( k 1) ( k ) T 1 T w + ij = wij η + J J I J e. (8) J is Jacobian matrix containing first derivatives of error function with respect to weights and thresholds, while e is vector of network error function. I is identity matrix, η is learning constant. If η equals zero, the equation (8) becomes Newton method using simplified Hessian matrix. For higher values of η the equation (8) becomes gradient descent method with small step. In each iteration η is adapted in such a way that it ensures convergence. IV. COMPARISON OF ANN LEARNING WITH IMPROVED METHODS IN THE CASE OF POWER TRANSFORMER INRUSH CURRENT RECOGNITION One of essential properties of ANN is inclusion of expert knowledge that was obtained with analysis of operation of power transformer in steady-state condition, during transients and in the case of faults. This knowledge is included in the preparation of characteristic patterns for learning of ANN. In the composition of patterns for learning of ANN all the above mentioned forms of current have to be taken into consideration. An important piece of information in composition of patterns is the number of discrete values that describe one pattern. This number depends upon time of sampling of primary and secondary currents in the case of protection system operation. Fig. shows time behaviour for twelve above mentioned forms and forms of current in stationary conditions, which are used for learning of ANN. 155
4 P 1 test P 1 test S 1 3 E = E : = d y, (9) test p P p= 1 p 1 j 1 test P = = test P test is the number of all patterns that were used for testing the neural network. In contrast to learning errors, test errors are higher in the second order methods. Fig. time behaviour of primary winding currents and belonging desired outputs for learning of ANN in recognition of transformer inrush current This figure also shows desired outputs for individual forms (d = 0 no inrush current or d = 1 inrush current) that are also needed for learning of ANN. From patterns that each contain 0 samples of the pattern shown in Fig., a set of patterns, stored in a matrix with dimension 0x1300, is created using moving window principle. This means that neuron network is learned with 1300 patterns (each of them contains 0 samples). In all hidden layers tangent sigmoid functions were used, while linear transfer function was used in the output layer. To make comparison of learning results easier we decided to present the results in graphical form. Numbers from 1 to 8 on horizontal axis of all graphs represent learning methods (1 standard gradient, momentum gradient, 3 with adaptive learning constant, 4 jumping gradient, 5 conjugate gradient, 6 pseudo-newton, 7 one-step secant, 8 Levenberg- Marquardt method). In the first step we found the optimum number of neurons in hidden layers (two-layer 9-1, three-layer 0-9-1, four-layer neurons) on the basis of minimum errors at 00 learning epochs. Number of learning epochs was then increased to 1000 and for them we calculated learning error, test error and time. Learning error for all learning methods is shown in Fig. 3. Fig. 3 values of learning error for different learning methods Test error that is shown in Fig. 4 represents the error between actual output and desired output, and was calculated on the basis of equation (9) as Fig. 4 values of test errors for different learning methods Second order methods also have the longest learning times, as shown in Fig. 5. Fig. 5 learning times for different learning methods Lower learning errors, obtained by means of improved learning methods do not necessarily mean lower test errors, which is just another proof that it is very difficult to set adequate learning conditions and criteria before learning. It is the most obvious for Levenberg-Marquardt method which has the lowest learning errors, but at the same time the highest test errors. V. CONCLUSION A comparison of results of learning and testing, yielded by different improved methods, was made on the case transformer inrush current recognising with ANN. As expected before the analyses, the best learning results were obtained by secondorder methods, such as Levenberg-Marquardt, pseudo-newton and one-step secant methods. A significant progress in learning was also shown by jumping and conjugates gradient methods. Standard gradient, momentum gradient and adaptive constant methods proved to be methods with rather poor convergence rate. An important aspect in learning of ANN was to include all characteristics of transient and steady-state conditions of power transformer in a representative way. This enabled to make ANN capable to recognise well enough current patterns that 156
5 were not used in the learning process. The decisive criterion of learning was square of error between actual and desired output for a single pattern. It was the lowest for Levenberg-Marquardt method in all selected network topologies, which demonstrates that this method is undoubtedly the best method for learning open-loop neural networks at present. Nevertheless, only on the basis of this piece of information it is not possible to make adequate conclusion on operation of already learned neural network. Test error is in all network topologies for this method the highest. ANN was capable to very well remember learning patterns, but was not learned to perform generalisation on new testing patterns. This is a consequence of the so-called excessive adaptation which occurs when too many neurons are used in the hidden layers. The same problem also occurred in all other methods that were used for learning of ANN. Test error was for other methods lower because the accuracy of learning was lower, which caused higher degree of generalisation of the results. This somehow compensated the influence of excessive adaptation caused by too high number of neurons in hidden layers, and thus brought satisfactory results from the point of view of differential protection only in testing of ANN that was learned with jumping gradient method. The guidelines that are shown by the presented results and findings of other authors reveal some new comprehensions in learning that could be of a great help in understanding of a so complex problem, such as building and learning of ANN. REFERENCES [1] D. E. Rumelhart, G. E. Hinton, R. J. Williams, Learning internal representations by error propagation, MA:MIT Press, Cambridge (1986). [] Neural Network Toolbox: for use with MATLAB, ml (004). [3] R. Rojas, Neural networks: a systematic introduction, Springer, Berlin (1996). [4] J. Pihler, "Power transformer protection with neural network", doctoral dissertation, Technical Faculty of Maribor (1995). [5] B. M. Wilamowski, S. Iplikci, O. Kaynak, M. Efe, An Algorithem for Fast Convergence in Training Neural Networks, (006). [6] S. Šuster, "Learning the artificial neural networks with improved error back-propagation methods", diploma, University of Maribor, Faculty of Electrical Engineering and Computer Science (006). 157
Artificial Neural Networks written examination
1 (8) Institutionen för informationsteknologi Olle Gällmo Universitetsadjunkt Adress: Lägerhyddsvägen 2 Box 337 751 05 Uppsala Artificial Neural Networks written examination Monday, May 15, 2006 9 00-14
More informationOPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS
OPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS Václav Kocian, Eva Volná, Michal Janošek, Martin Kotyrba University of Ostrava Department of Informatics and Computers Dvořákova 7,
More informationPython Machine Learning
Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled
More informationINPE São José dos Campos
INPE-5479 PRE/1778 MONLINEAR ASPECTS OF DATA INTEGRATION FOR LAND COVER CLASSIFICATION IN A NEDRAL NETWORK ENVIRONNENT Maria Suelena S. Barros Valter Rodrigues INPE São José dos Campos 1993 SECRETARIA
More informationLearning Methods for Fuzzy Systems
Learning Methods for Fuzzy Systems Rudolf Kruse and Andreas Nürnberger Department of Computer Science, University of Magdeburg Universitätsplatz, D-396 Magdeburg, Germany Phone : +49.39.67.876, Fax : +49.39.67.8
More informationAnalysis of Hybrid Soft and Hard Computing Techniques for Forex Monitoring Systems
Analysis of Hybrid Soft and Hard Computing Techniques for Forex Monitoring Systems Ajith Abraham School of Business Systems, Monash University, Clayton, Victoria 3800, Australia. Email: ajith.abraham@ieee.org
More informationLecture 1: Machine Learning Basics
1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3
More informationQuickStroke: An Incremental On-line Chinese Handwriting Recognition System
QuickStroke: An Incremental On-line Chinese Handwriting Recognition System Nada P. Matić John C. Platt Λ Tony Wang y Synaptics, Inc. 2381 Bering Drive San Jose, CA 95131, USA Abstract This paper presents
More informationAn empirical study of learning speed in backpropagation
Carnegie Mellon University Research Showcase @ CMU Computer Science Department School of Computer Science 1988 An empirical study of learning speed in backpropagation networks Scott E. Fahlman Carnegie
More informationA Neural Network GUI Tested on Text-To-Phoneme Mapping
A Neural Network GUI Tested on Text-To-Phoneme Mapping MAARTEN TROMPPER Universiteit Utrecht m.f.a.trompper@students.uu.nl Abstract Text-to-phoneme (T2P) mapping is a necessary step in any speech synthesis
More informationTest Effort Estimation Using Neural Network
J. Software Engineering & Applications, 2010, 3: 331-340 doi:10.4236/jsea.2010.34038 Published Online April 2010 (http://www.scirp.org/journal/jsea) 331 Chintala Abhishek*, Veginati Pavan Kumar, Harish
More informationCircuit Simulators: A Revolutionary E-Learning Platform
Circuit Simulators: A Revolutionary E-Learning Platform Mahi Itagi Padre Conceicao College of Engineering, Verna, Goa, India. itagimahi@gmail.com Akhil Deshpande Gogte Institute of Technology, Udyambag,
More informationOn the Combined Behavior of Autonomous Resource Management Agents
On the Combined Behavior of Autonomous Resource Management Agents Siri Fagernes 1 and Alva L. Couch 2 1 Faculty of Engineering Oslo University College Oslo, Norway siri.fagernes@iu.hio.no 2 Computer Science
More information(Sub)Gradient Descent
(Sub)Gradient Descent CMSC 422 MARINE CARPUAT marine@cs.umd.edu Figures credit: Piyush Rai Logistics Midterm is on Thursday 3/24 during class time closed book/internet/etc, one page of notes. will include
More informationSoftprop: Softmax Neural Network Backpropagation Learning
Softprop: Softmax Neural Networ Bacpropagation Learning Michael Rimer Computer Science Department Brigham Young University Provo, UT 84602, USA E-mail: mrimer@axon.cs.byu.edu Tony Martinez Computer Science
More informationAP Calculus AB. Nevada Academic Standards that are assessable at the local level only.
Calculus AB Priority Keys Aligned with Nevada Standards MA I MI L S MA represents a Major content area. Any concept labeled MA is something of central importance to the entire class/curriculum; it is a
More informationModule 12. Machine Learning. Version 2 CSE IIT, Kharagpur
Module 12 Machine Learning 12.1 Instructional Objective The students should understand the concept of learning systems Students should learn about different aspects of a learning system Students should
More informationEvolutive Neural Net Fuzzy Filtering: Basic Description
Journal of Intelligent Learning Systems and Applications, 2010, 2: 12-18 doi:10.4236/jilsa.2010.21002 Published Online February 2010 (http://www.scirp.org/journal/jilsa) Evolutive Neural Net Fuzzy Filtering:
More informationLecture 10: Reinforcement Learning
Lecture 1: Reinforcement Learning Cognitive Systems II - Machine Learning SS 25 Part III: Learning Programs and Strategies Q Learning, Dynamic Programming Lecture 1: Reinforcement Learning p. Motivation
More informationFramewise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures
Framewise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures Alex Graves and Jürgen Schmidhuber IDSIA, Galleria 2, 6928 Manno-Lugano, Switzerland TU Munich, Boltzmannstr.
More informationNCEO Technical Report 27
Home About Publications Special Topics Presentations State Policies Accommodations Bibliography Teleconferences Tools Related Sites Interpreting Trends in the Performance of Special Education Students
More informationMathematics. Mathematics
Mathematics Program Description Successful completion of this major will assure competence in mathematics through differential and integral calculus, providing an adequate background for employment in
More informationAxiom 2013 Team Description Paper
Axiom 2013 Team Description Paper Mohammad Ghazanfari, S Omid Shirkhorshidi, Farbod Samsamipour, Hossein Rahmatizadeh Zagheli, Mohammad Mahdavi, Payam Mohajeri, S Abbas Alamolhoda Robotics Scientific Association
More informationSARDNET: A Self-Organizing Feature Map for Sequences
SARDNET: A Self-Organizing Feature Map for Sequences Daniel L. James and Risto Miikkulainen Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 dljames,risto~cs.utexas.edu
More informationStatewide Framework Document for:
Statewide Framework Document for: 270301 Standards may be added to this document prior to submission, but may not be removed from the framework to meet state credit equivalency requirements. Performance
More informationarxiv: v1 [cs.lg] 15 Jun 2015
Dual Memory Architectures for Fast Deep Learning of Stream Data via an Online-Incremental-Transfer Strategy arxiv:1506.04477v1 [cs.lg] 15 Jun 2015 Sang-Woo Lee Min-Oh Heo School of Computer Science and
More informationEvolution of Symbolisation in Chimpanzees and Neural Nets
Evolution of Symbolisation in Chimpanzees and Neural Nets Angelo Cangelosi Centre for Neural and Adaptive Systems University of Plymouth (UK) a.cangelosi@plymouth.ac.uk Introduction Animal communication
More informationAGS THE GREAT REVIEW GAME FOR PRE-ALGEBRA (CD) CORRELATED TO CALIFORNIA CONTENT STANDARDS
AGS THE GREAT REVIEW GAME FOR PRE-ALGEBRA (CD) CORRELATED TO CALIFORNIA CONTENT STANDARDS 1 CALIFORNIA CONTENT STANDARDS: Chapter 1 ALGEBRA AND WHOLE NUMBERS Algebra and Functions 1.4 Students use algebraic
More informationThe Good Judgment Project: A large scale test of different methods of combining expert predictions
The Good Judgment Project: A large scale test of different methods of combining expert predictions Lyle Ungar, Barb Mellors, Jon Baron, Phil Tetlock, Jaime Ramos, Sam Swift The University of Pennsylvania
More informationSchool of Innovative Technologies and Engineering
School of Innovative Technologies and Engineering Department of Applied Mathematical Sciences Proficiency Course in MATLAB COURSE DOCUMENT VERSION 1.0 PCMv1.0 July 2012 University of Technology, Mauritius
More informationSystem Implementation for SemEval-2017 Task 4 Subtask A Based on Interpolated Deep Neural Networks
System Implementation for SemEval-2017 Task 4 Subtask A Based on Interpolated Deep Neural Networks 1 Tzu-Hsuan Yang, 2 Tzu-Hsuan Tseng, and 3 Chia-Ping Chen Department of Computer Science and Engineering
More informationHuman Emotion Recognition From Speech
RESEARCH ARTICLE OPEN ACCESS Human Emotion Recognition From Speech Miss. Aparna P. Wanare*, Prof. Shankar N. Dandare *(Department of Electronics & Telecommunication Engineering, Sant Gadge Baba Amravati
More informationCourse Outline. Course Grading. Where to go for help. Academic Integrity. EE-589 Introduction to Neural Networks NN 1 EE
EE-589 Introduction to Neural Assistant Prof. Dr. Turgay IBRIKCI Room # 305 (322) 338 6868 / 139 Wensdays 9:00-12:00 Course Outline The course is divided in two parts: theory and practice. 1. Theory covers
More informationMathematics subject curriculum
Mathematics subject curriculum Dette er ei omsetjing av den fastsette læreplanteksten. Læreplanen er fastsett på Nynorsk Established as a Regulation by the Ministry of Education and Research on 24 June
More informationGrade 6: Correlated to AGS Basic Math Skills
Grade 6: Correlated to AGS Basic Math Skills Grade 6: Standard 1 Number Sense Students compare and order positive and negative integers, decimals, fractions, and mixed numbers. They find multiples and
More informationA GENERIC SPLIT PROCESS MODEL FOR ASSET MANAGEMENT DECISION-MAKING
A GENERIC SPLIT PROCESS MODEL FOR ASSET MANAGEMENT DECISION-MAKING Yong Sun, a * Colin Fidge b and Lin Ma a a CRC for Integrated Engineering Asset Management, School of Engineering Systems, Queensland
More informationLecture 1: Basic Concepts of Machine Learning
Lecture 1: Basic Concepts of Machine Learning Cognitive Systems - Machine Learning Ute Schmid (lecture) Johannes Rabold (practice) Based on slides prepared March 2005 by Maximilian Röglinger, updated 2010
More informationMathematics process categories
Mathematics process categories All of the UK curricula define multiple categories of mathematical proficiency that require students to be able to use and apply mathematics, beyond simple recall of facts
More informationI-COMPETERE: Using Applied Intelligence in search of competency gaps in software project managers.
Information Systems Frontiers manuscript No. (will be inserted by the editor) I-COMPETERE: Using Applied Intelligence in search of competency gaps in software project managers. Ricardo Colomo-Palacios
More informationGenerative models and adversarial training
Day 4 Lecture 1 Generative models and adversarial training Kevin McGuinness kevin.mcguinness@dcu.ie Research Fellow Insight Centre for Data Analytics Dublin City University What is a generative model?
More informationArtificial Neural Networks
Artificial Neural Networks Andres Chavez Math 382/L T/Th 2:00-3:40 April 13, 2010 Chavez2 Abstract The main interest of this paper is Artificial Neural Networks (ANNs). A brief history of the development
More informationDiscriminative Learning of Beam-Search Heuristics for Planning
Discriminative Learning of Beam-Search Heuristics for Planning Yuehua Xu School of EECS Oregon State University Corvallis,OR 97331 xuyu@eecs.oregonstate.edu Alan Fern School of EECS Oregon State University
More informationBMBF Project ROBUKOM: Robust Communication Networks
BMBF Project ROBUKOM: Robust Communication Networks Arie M.C.A. Koster Christoph Helmberg Andreas Bley Martin Grötschel Thomas Bauschert supported by BMBF grant 03MS616A: ROBUKOM Robust Communication Networks,
More informationImprovements to the Pruning Behavior of DNN Acoustic Models
Improvements to the Pruning Behavior of DNN Acoustic Models Matthias Paulik Apple Inc., Infinite Loop, Cupertino, CA 954 mpaulik@apple.com Abstract This paper examines two strategies that positively influence
More informationUsing the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees Cognitive Skills in Algebra on the SAT
The Journal of Technology, Learning, and Assessment Volume 6, Number 6 February 2008 Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees Cognitive Skills in Algebra on the
More informationSpeaker Identification by Comparison of Smart Methods. Abstract
Journal of mathematics and computer science 10 (2014), 61-71 Speaker Identification by Comparison of Smart Methods Ali Mahdavi Meimand Amin Asadi Majid Mohamadi Department of Electrical Department of Computer
More informationHonors Mathematics. Introduction and Definition of Honors Mathematics
Honors Mathematics Introduction and Definition of Honors Mathematics Honors Mathematics courses are intended to be more challenging than standard courses and provide multiple opportunities for students
More informationKnowledge Transfer in Deep Convolutional Neural Nets
Knowledge Transfer in Deep Convolutional Neural Nets Steven Gutstein, Olac Fuentes and Eric Freudenthal Computer Science Department University of Texas at El Paso El Paso, Texas, 79968, U.S.A. Abstract
More informationEvaluation of Usage Patterns for Web-based Educational Systems using Web Mining
Evaluation of Usage Patterns for Web-based Educational Systems using Web Mining Dave Donnellan, School of Computer Applications Dublin City University Dublin 9 Ireland daviddonnellan@eircom.net Claus Pahl
More informationEvaluation of Usage Patterns for Web-based Educational Systems using Web Mining
Evaluation of Usage Patterns for Web-based Educational Systems using Web Mining Dave Donnellan, School of Computer Applications Dublin City University Dublin 9 Ireland daviddonnellan@eircom.net Claus Pahl
More information*** * * * COUNCIL * * CONSEIL OFEUROPE * * * DE L'EUROPE. Proceedings of the 9th Symposium on Legal Data Processing in Europe
*** * * * COUNCIL * * CONSEIL OFEUROPE * * * DE L'EUROPE Proceedings of the 9th Symposium on Legal Data Processing in Europe Bonn, 10-12 October 1989 Systems based on artificial intelligence in the legal
More informationCOMPUTER-ASSISTED INDEPENDENT STUDY IN MULTIVARIATE CALCULUS
COMPUTER-ASSISTED INDEPENDENT STUDY IN MULTIVARIATE CALCULUS L. Descalço 1, Paula Carvalho 1, J.P. Cruz 1, Paula Oliveira 1, Dina Seabra 2 1 Departamento de Matemática, Universidade de Aveiro (PORTUGAL)
More informationarxiv: v1 [math.at] 10 Jan 2016
THE ALGEBRAIC ATIYAH-HIRZEBRUCH SPECTRAL SEQUENCE OF REAL PROJECTIVE SPECTRA arxiv:1601.02185v1 [math.at] 10 Jan 2016 GUOZHEN WANG AND ZHOULI XU Abstract. In this note, we use Curtis s algorithm and the
More informationReinforcement Learning by Comparing Immediate Reward
Reinforcement Learning by Comparing Immediate Reward Punit Pandey DeepshikhaPandey Dr. Shishir Kumar Abstract This paper introduces an approach to Reinforcement Learning Algorithm by comparing their immediate
More informationCalibration of Confidence Measures in Speech Recognition
Submitted to IEEE Trans on Audio, Speech, and Language, July 2010 1 Calibration of Confidence Measures in Speech Recognition Dong Yu, Senior Member, IEEE, Jinyu Li, Member, IEEE, Li Deng, Fellow, IEEE
More informationAUTOMATIC DETECTION OF PROLONGED FRICATIVE PHONEMES WITH THE HIDDEN MARKOV MODELS APPROACH 1. INTRODUCTION
JOURNAL OF MEDICAL INFORMATICS & TECHNOLOGIES Vol. 11/2007, ISSN 1642-6037 Marek WIŚNIEWSKI *, Wiesława KUNISZYK-JÓŹKOWIAK *, Elżbieta SMOŁKA *, Waldemar SUSZYŃSKI * HMM, recognition, speech, disorders
More informationSoftware Maintenance
1 What is Software Maintenance? Software Maintenance is a very broad activity that includes error corrections, enhancements of capabilities, deletion of obsolete capabilities, and optimization. 2 Categories
More informationHow to Judge the Quality of an Objective Classroom Test
How to Judge the Quality of an Objective Classroom Test Technical Bulletin #6 Evaluation and Examination Service The University of Iowa (319) 335-0356 HOW TO JUDGE THE QUALITY OF AN OBJECTIVE CLASSROOM
More informationUniversity of Groningen. Systemen, planning, netwerken Bosman, Aart
University of Groningen Systemen, planning, netwerken Bosman, Aart IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document
More informationCHAPTER 4: REIMBURSEMENT STRATEGIES 24
CHAPTER 4: REIMBURSEMENT STRATEGIES 24 INTRODUCTION Once state level policymakers have decided to implement and pay for CSR, one issue they face is simply how to calculate the reimbursements to districts
More informationB.S/M.A in Mathematics
B.S/M.A in Mathematics The dual Bachelor of Science/Master of Arts in Mathematics program provides an opportunity for individuals to pursue advanced study in mathematics and to develop skills that can
More informationWHEN THERE IS A mismatch between the acoustic
808 IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 14, NO. 3, MAY 2006 Optimization of Temporal Filters for Constructing Robust Features in Speech Recognition Jeih-Weih Hung, Member,
More informationUsing focal point learning to improve human machine tacit coordination
DOI 10.1007/s10458-010-9126-5 Using focal point learning to improve human machine tacit coordination InonZuckerman SaritKraus Jeffrey S. Rosenschein The Author(s) 2010 Abstract We consider an automated
More informationA student diagnosing and evaluation system for laboratory-based academic exercises
A student diagnosing and evaluation system for laboratory-based academic exercises Maria Samarakou, Emmanouil Fylladitakis and Pantelis Prentakis Technological Educational Institute (T.E.I.) of Athens
More informationSeminar - Organic Computing
Seminar - Organic Computing Self-Organisation of OC-Systems Markus Franke 25.01.2006 Typeset by FoilTEX Timetable 1. Overview 2. Characteristics of SO-Systems 3. Concern with Nature 4. Design-Concepts
More informationLearning to Schedule Straight-Line Code
Learning to Schedule Straight-Line Code Eliot Moss, Paul Utgoff, John Cavazos Doina Precup, Darko Stefanović Dept. of Comp. Sci., Univ. of Mass. Amherst, MA 01003 Carla Brodley, David Scheeff Sch. of Elec.
More informationLikelihood-Maximizing Beamforming for Robust Hands-Free Speech Recognition
MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Likelihood-Maximizing Beamforming for Robust Hands-Free Speech Recognition Seltzer, M.L.; Raj, B.; Stern, R.M. TR2004-088 December 2004 Abstract
More informationModeling function word errors in DNN-HMM based LVCSR systems
Modeling function word errors in DNN-HMM based LVCSR systems Melvin Jose Johnson Premkumar, Ankur Bapna and Sree Avinash Parchuri Department of Computer Science Department of Electrical Engineering Stanford
More informationhave to be modeled) or isolated words. Output of the system is a grapheme-tophoneme conversion system which takes as its input the spelling of words,
A Language-Independent, Data-Oriented Architecture for Grapheme-to-Phoneme Conversion Walter Daelemans and Antal van den Bosch Proceedings ESCA-IEEE speech synthesis conference, New York, September 1994
More informationProbabilistic Latent Semantic Analysis
Probabilistic Latent Semantic Analysis Thomas Hofmann Presentation by Ioannis Pavlopoulos & Andreas Damianou for the course of Data Mining & Exploration 1 Outline Latent Semantic Analysis o Need o Overview
More informationSyllabus ENGR 190 Introductory Calculus (QR)
Syllabus ENGR 190 Introductory Calculus (QR) Catalog Data: ENGR 190 Introductory Calculus (4 credit hours). Note: This course may not be used for credit toward the J.B. Speed School of Engineering B. S.
More informationAnalysis of Enzyme Kinetic Data
Analysis of Enzyme Kinetic Data To Marilú Analysis of Enzyme Kinetic Data ATHEL CORNISH-BOWDEN Directeur de Recherche Émérite, Centre National de la Recherche Scientifique, Marseilles OXFORD UNIVERSITY
More informationAssignment 1: Predicting Amazon Review Ratings
Assignment 1: Predicting Amazon Review Ratings 1 Dataset Analysis Richard Park r2park@acsmail.ucsd.edu February 23, 2015 The dataset selected for this assignment comes from the set of Amazon reviews for
More informationSecond Exam: Natural Language Parsing with Neural Networks
Second Exam: Natural Language Parsing with Neural Networks James Cross May 21, 2015 Abstract With the advent of deep learning, there has been a recent resurgence of interest in the use of artificial neural
More informationAutomatic Pronunciation Checker
Institut für Technische Informatik und Kommunikationsnetze Eidgenössische Technische Hochschule Zürich Swiss Federal Institute of Technology Zurich Ecole polytechnique fédérale de Zurich Politecnico federale
More informationE LEARNING TOOLS IN DISTANCE AND STATIONARY EDUCATION
E LEARNING TOOLS IN DISTANCE AND STATIONARY EDUCATION Michał Krupski 1, Andrzej Cader 2 1 Institute for Distance Education Research, Academy of Humanities and Economics in Lodz, Poland michalk@wshe.lodz.pl
More informationNumeracy Medium term plan: Summer Term Level 2C/2B Year 2 Level 2A/3C
Numeracy Medium term plan: Summer Term Level 2C/2B Year 2 Level 2A/3C Using and applying mathematics objectives (Problem solving, Communicating and Reasoning) Select the maths to use in some classroom
More informationMachine Learning and Data Mining. Ensembles of Learners. Prof. Alexander Ihler
Machine Learning and Data Mining Ensembles of Learners Prof. Alexander Ihler Ensemble methods Why learn one classifier when you can learn many? Ensemble: combine many predictors (Weighted) combina
More informationThe Method of Immersion the Problem of Comparing Technical Objects in an Expert Shell in the Class of Artificial Intelligence Algorithms
IOP Conference Series: Materials Science and Engineering PAPER OPEN ACCESS The Method of Immersion the Problem of Comparing Technical Objects in an Expert Shell in the Class of Artificial Intelligence
More informationTesting A Moving Target: How Do We Test Machine Learning Systems? Peter Varhol Technology Strategy Research, USA
Testing A Moving Target: How Do We Test Machine Learning Systems? Peter Varhol Technology Strategy Research, USA Testing a Moving Target How Do We Test Machine Learning Systems? Peter Varhol, Technology
More informationDeep search. Enhancing a search bar using machine learning. Ilgün Ilgün & Cedric Reichenbach
#BaselOne7 Deep search Enhancing a search bar using machine learning Ilgün Ilgün & Cedric Reichenbach We are not researchers Outline I. Periscope: A search tool II. Goals III. Deep learning IV. Applying
More informationENME 605 Advanced Control Systems, Fall 2015 Department of Mechanical Engineering
ENME 605 Advanced Control Systems, Fall 2015 Department of Mechanical Engineering Lecture Details Instructor Course Objectives Tuesday and Thursday, 4:00 pm to 5:15 pm Information Technology and Engineering
More informationKamaldeep Kaur University School of Information Technology GGS Indraprastha University Delhi
Soft Computing Approaches for Prediction of Software Maintenance Effort Dr. Arvinder Kaur University School of Information Technology GGS Indraprastha University Delhi Kamaldeep Kaur University School
More informationNeuro-Symbolic Approaches for Knowledge Representation in Expert Systems
Published in the International Journal of Hybrid Intelligent Systems 1(3-4) (2004) 111-126 Neuro-Symbolic Approaches for Knowledge Representation in Expert Systems Ioannis Hatzilygeroudis and Jim Prentzas
More informationA Reinforcement Learning Variant for Control Scheduling
A Reinforcement Learning Variant for Control Scheduling Aloke Guha Honeywell Sensor and System Development Center 3660 Technology Drive Minneapolis MN 55417 Abstract We present an algorithm based on reinforcement
More informationGiven a directed graph G =(N A), where N is a set of m nodes and A. destination node, implying a direction for ow to follow. Arcs have limitations
4 Interior point algorithms for network ow problems Mauricio G.C. Resende AT&T Bell Laboratories, Murray Hill, NJ 07974-2070 USA Panos M. Pardalos The University of Florida, Gainesville, FL 32611-6595
More informationOn Human Computer Interaction, HCI. Dr. Saif al Zahir Electrical and Computer Engineering Department UBC
On Human Computer Interaction, HCI Dr. Saif al Zahir Electrical and Computer Engineering Department UBC Human Computer Interaction HCI HCI is the study of people, computer technology, and the ways these
More informationThis scope and sequence assumes 160 days for instruction, divided among 15 units.
In previous grades, students learned strategies for multiplication and division, developed understanding of structure of the place value system, and applied understanding of fractions to addition and subtraction
More informationBUILDING CONTEXT-DEPENDENT DNN ACOUSTIC MODELS USING KULLBACK-LEIBLER DIVERGENCE-BASED STATE TYING
BUILDING CONTEXT-DEPENDENT DNN ACOUSTIC MODELS USING KULLBACK-LEIBLER DIVERGENCE-BASED STATE TYING Gábor Gosztolya 1, Tamás Grósz 1, László Tóth 1, David Imseng 2 1 MTA-SZTE Research Group on Artificial
More informationReducing Features to Improve Bug Prediction
Reducing Features to Improve Bug Prediction Shivkumar Shivaji, E. James Whitehead, Jr., Ram Akella University of California Santa Cruz {shiv,ejw,ram}@soe.ucsc.edu Sunghun Kim Hong Kong University of Science
More informationCOMPUTER-AIDED DESIGN TOOLS THAT ADAPT
COMPUTER-AIDED DESIGN TOOLS THAT ADAPT WEI PENG CSIRO ICT Centre, Australia and JOHN S GERO Krasnow Institute for Advanced Study, USA 1. Introduction Abstract. This paper describes an approach that enables
More informationCSL465/603 - Machine Learning
CSL465/603 - Machine Learning Fall 2016 Narayanan C Krishnan ckn@iitrpr.ac.in Introduction CSL465/603 - Machine Learning 1 Administrative Trivia Course Structure 3-0-2 Lecture Timings Monday 9.55-10.45am
More informationOn-Line Data Analytics
International Journal of Computer Applications in Engineering Sciences [VOL I, ISSUE III, SEPTEMBER 2011] [ISSN: 2231-4946] On-Line Data Analytics Yugandhar Vemulapalli #, Devarapalli Raghu *, Raja Jacob
More informationAnalysis of Speech Recognition Models for Real Time Captioning and Post Lecture Transcription
Analysis of Speech Recognition Models for Real Time Captioning and Post Lecture Transcription Wilny Wilson.P M.Tech Computer Science Student Thejus Engineering College Thrissur, India. Sindhu.S Computer
More informationLongest Common Subsequence: A Method for Automatic Evaluation of Handwritten Essays
IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 17, Issue 6, Ver. IV (Nov Dec. 2015), PP 01-07 www.iosrjournals.org Longest Common Subsequence: A Method for
More informationRule Learning With Negation: Issues Regarding Effectiveness
Rule Learning With Negation: Issues Regarding Effectiveness S. Chua, F. Coenen, G. Malcolm University of Liverpool Department of Computer Science, Ashton Building, Ashton Street, L69 3BX Liverpool, United
More informationGACE Computer Science Assessment Test at a Glance
GACE Computer Science Assessment Test at a Glance Updated May 2017 See the GACE Computer Science Assessment Study Companion for practice questions and preparation resources. Assessment Name Computer Science
More informationEvaluating Interactive Visualization of Multidimensional Data Projection with Feature Transformation
Multimodal Technologies and Interaction Article Evaluating Interactive Visualization of Multidimensional Data Projection with Feature Transformation Kai Xu 1, *,, Leishi Zhang 1,, Daniel Pérez 2,, Phong
More informationMachine Learning from Garden Path Sentences: The Application of Computational Linguistics
Machine Learning from Garden Path Sentences: The Application of Computational Linguistics http://dx.doi.org/10.3991/ijet.v9i6.4109 J.L. Du 1, P.F. Yu 1 and M.L. Li 2 1 Guangdong University of Foreign Studies,
More informationIssues in the Mining of Heart Failure Datasets
International Journal of Automation and Computing 11(2), April 2014, 162-179 DOI: 10.1007/s11633-014-0778-5 Issues in the Mining of Heart Failure Datasets Nongnuch Poolsawad 1 Lisa Moore 1 Chandrasekhar
More information