Artificial Neural Networks

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Artificial Neural Networks"

Transcription

1 Artificial Neural Networks

2 Outline Introduction to Neural Network Introduction to Artificial Neural Network Properties of Artificial Neural Network Applications of Artificial Neural Network Demo Neural Network Tool Box Case-1 Designing XOR network Case-2 Power system security assessment 2

3 What are Neural Networks? Models of the brain and nervous system Highly parallel Process information much more like the brain than a serial computer Learning Very simple principles Very complex behaviours 3

4 BIOLOGICAL NEURAL NETWORK Figure 1 Structure of biological neuron 4

5 The Structure of Neurons A neuron has a cell body, a branching input structure (the dendrite) and a branching output structure (the axon) Axons connect to dendrites via synapses. Electro-chemical signals are propagated from the dendritic input, through the cell body, and down the axon to other neurons 5

6 The Structure of Neurons A neuron only fires if its input signal exceeds a certain amount (the threshold) in a short time period. Synapses vary in strength Good connections allowing a large signal Slight connections allow only a weak signal. Synapses can be either excitatory or inhibitory. 6

7 7 The Artificial Neural Network Figure 2 Structure of artificial neuron Mathematically, the output expression of the network is given as + = = = N K K K b W X F S F Y 1 ) (

8 ANNs The basics ANNs incorporate the two fundamental components of biological neural nets: 1. Neurones (nodes) 2. Synapses (weights) 8

9 Properties of Artificial Neural Nets (ANNs) 9

10 Properties of Artificial Neural Nets (ANNs) Many simple neuron-like threshold switching units Many weighted interconnections among units Highly parallel, distributed processing Learning by tuning the connection weights 10

11 Appropriate Problem Domains for Neural Network Learning Input is high-dimensional discrete or realvalued (e.g. raw sensor input) Output is discrete or real valued Output is a vector of values Form of target function is unknown Humans do not need to interpret the results (black box model) 11

12 Applications Ability to model linear and non-linear systems without the need to make assumptions implicitly. Applied in almost every field of science and engineering. Few of them are Function approaximation, or regression analysis, including time series and modelling. Classification, including pattern and sequence recognition, novelty detection and sequential decision making. Data processing, including filtering, clustering, blind signal separation and compression. Computational neuroscience and neurohydrodynamics Forecating and prediction Estimation and control 12

13 Applications in Electrical Load forecasting Short-term load forecasting Mid-term load forecasting Long-term load frecasting Fault diagnosis/ Fault location Economic dispatch Security Assessment Estimation of solar radiation, solar heating, etc. Wind speed prediction 13

14 Designing ANN models Designing ANN models follows a number of systemic procedures. In general, there are five basics steps: (1) collecting data, (2) preprocessing data (3) building the network (4) train, and (5) test performance of model as shown in Fig. Fig. 3. Basic flow for designing artificial neural network model 14

15 Neural Network Problems Many Parameters to be set Overfitting long training times... 15

16 INTRODUCTION TO NN TOOLBOX The Neural Network Toolbox is one of the commonly used, powerful, commercially available software tools for the development and design of neural networks. The software is user-friendly, permits flexibility and convenience in interfacing with other toolboxes in the same environment to develop a full application. 16

17 Features It supports a wide variety of feed-forward and recurrent networks, including perceptrons, radial basis networks, BP networks, learning vector quantization (LVQ) networks, self-organizing networks, Hopfield and Elman NWs, etc. It also supports the activation function types of bi-directional linear with hard limit (satlins) and without hard limit, threshold (hard limit), signum (symmetlic hard limit), sigmoidal (log-sigmoid), and hyperbolic tan (tan-sigmoid). 17

18 Features In addition, it supports unidirectional linear with hard limit (satlins) and without hard limit, radial basis and triangular basis, and competitive and soft max functions. A wide variety of training and learning algorithms are supported. 18

19 Case-1 Problem Definition The XOR problem requires one hidden layer & one output layer, since it s NOT linearly separable. 19

20 Design Phase 20

21 NN Toolbox NN toolbox can be open by entering command >>nntool It can also be open as shown below It will open NN Network/ Data Manager screen. 21

22 Getting Started 22

23 NN Network/ Data Manager 23

24 Design Let P denote the input and T denote the target/output. In Matlab as per the guidelines of implementation these are to be expressed in the form of matrices: P = [ ; ] T = [ ] To use a network first design it, then train it before start simulation. We follow the steps in order to do the above: 24

25 Provide input and target data Step-1: First we have to enter P and T to the NN Network Manager. This is done by clicking New Data once. Step-2: Type P as the Name, and corresponding matrix as the Value, select Inputs under DataType, then confirm by clicking on Create. Step-3: Similarly, type in T as the Name, and corresponding matrix as the Value, select Targets, under DataType, then confirm. See a screen like following figures 25

26 Providing input 26

27 Providing target data 27

28 Create Network Step-4: Now we try to create a XORNet. For this click on New Network. See a screen like in the following figure. Now change all the parameters on the screen to the values as indicated on the following screen: 28

29 Defining XORNet network 29

30 Setting network parameters Make Sure the parameters are as follows: Network Type = Feedforword Backprop Train Function = TRAINLM Adaption Learning Function = LEARNGDM Performance Function = MSE Numbers of Layers = 2 30

31 Define network size Step-5: Select Layer 1, type in 2 for the number of neurons, & select TANSIG as Transfer Function. Select Layer 2, type in 1 for the number of neurons, & select TANSIG as Transfer Function. Step-6: Then, confirm by hitting the Create button, which concludes the XOR network implementation phase. 31

32 32

33 Step-7: Now, highlight XORNet with DOUBLE click, then click on Train button. You will get the following screen indicated in figure. 33

34 Training network 34

35 Defining training parameters 35

36 Step-8: On Training Info, select P as Inputs, T as Targets. On Training Parameters, specify: epochs = 1000 Goal = Max fail = 50 After, confirming all the parameters have been specified as indented, hit Train Network. 36

37 Training process 37

38 Various Plots Now we can get following plots Performance plot It should get a decaying plot (since you are trying to minimize the error). Training State Plot Regression Plot 38

39 Performance plot plots the training, validation, and test performances given the training record TR returned by the function train. 39

40 Training state plot plots the training state from a training record TR returned by train. 40

41 Regression plot Plots the linear regression of targets relative to outputs. 41

42 View weights and bias Step-8: Now to confirm the XORNet structure and values of various Weights and Bias of the trained network click on View on the Network/Data Manager window. NOTE: If for any reason, you don t get the figure as expected, click on Delete and recreate the XORNet as described above. Now, the XORNet has been trained successfully and is ready for simulation. 42

43 XORNet Structure 43

44 Network simulation With trained network, simulation is a way of testing on the network to see if it meets our expectation. Step-9: Now, create a new test data S (with a matrix [1; 0] representing a set of two inputs) on the NN Network Manager, follow the same procedure indicated before (like for input P). 44

45 Step-10: HighLight XORNet again with one click, then click on the Simulate button on the Network Manager. Select S as the Inputs, type in ORNet_outputsSim as Outputs, then hit the Simulate Network button and check the result of XORNet_outputSim on the NN Network Manager, by clicking View. This concludes the whole process of XOR network design, training & simulation. 45

46 Simulated result 46

47 Case-2 Problem Definition Power system security assessment determines safety status of a power system in three fold steps: system monitoring, contingency analysis and security control. load flow equations are required to identify the power flows and voltage levels throughout the transmission system The contingencies can be single element outage (N-1), multiple-element outage (N-2 or N-X) and sequential outage Here single only outage CIARE-2012, at IIT Mandi a time is considered 47

48 Data Collection The input data is obtained from offline Newton- Raphson load flow by using the MATLAB software. The data have matrix size [12X65]. In data collection, these input data are divided into three groups which are train data, validate data, and test data. The matrix size of train data is [12X32] while the matrix size of test data is [12X23]. 48

49 Data Collection The bus voltages V 1, V 2 and V 3 are not included in the train data and test data because they are generator buses. They will be controlled by the automatic voltage regulator (AVR) system. In train data, there are 10 train data in secure condition while 12 train data in insecure condition. For test data, there are 1 test data which is secure status while 10 test data are insecure status. 49

50 DATA COLLECTION 50

51 DATA COLLECTION 51

52 DATA PRE-PROCESSING After data collection, 3 data preprocessing procedures train the ANNs more efficiently. solve the problem of missing data, normalize data, and randomize data. The missing data are replaced by the average of neighboring values. 52

53 Normalization Normalization procedure before presenting the input data to the network is required since mixing variables with large magnitudes and small magnitudes will confuse the learning algorithm on the importance of each variable and may force it to finally reject the variable with the smaller magnitude. 53

54 Building the Network At this stage, the designer specifies the number of hidden layers, neurons in each layer, transfer function in each layer, training function, weight/bias learning function, and performance function. 54

55 TRAINING THE NETWORK During the training process, the weights are adjusted to make the actual outputs (predicated) close to the target (measured) outputs of the network. Fourteen types of training algorithms for developing the MLP network. MATLAB provides built-in transfer functions linear (purelin), Hyperbolic Tangent Sigmoid (tansig) and Logistic Sigmoid (logsig). The graphical illustration and mathematical form of such functions are shown in Table 1. 55

56 TRAINING THE NETWORK Table 1. MATLAB built-in transfer functions 56

57 Parameter setting Number of layers Number of neurons too many neurons, require more training time Learning rate from experience, value should be small ~0.1 Momentum term.. 57

58 TESTING THE NETWORK The next step is to test the performance of the developed model. At this stage unseen data are exposed to the model. In order to evaluate the performance of the developed ANN models quantitatively and verify whether there is any underlying trend in performance of ANN models, statistical analysis involving the coefficient of determination (R), the root mean square error (RMSE), and the mean bias error (MBE) are conducted. 58

59 RMSE RMSE provides information on the short term performance which is a measure of the variation of predicated values around the measured data. The lower the RMSE, the more accurate is the estimation. 59

60 MBE MBE is an indication of the average deviation of the predicted values from the corresponding measured data and can provide information on long term performance of the models; the lower MBE the better is the long term model prediction. 60

61 PROGRAMMING THE NEURAL NETWORK MODEL ANN implementation is a process that results in design of best ANN configuration. Percentages of classification accuracy and mean square error are used to represent the performance of ANN in terms of accuracy to predict the security level of IEEE 9 bus system. Steps of ANN implementation is shown in the following flow chart. 61

62 FLOW CHART 62

63 USING NN TOOLBOX First run the MATLAB file testandtrain.m. This file contains test data (input data) and target data. Name of input data is train Name of target data is target Network can be initialized from command prompt as >>nftool or by using following step 63

64 OPENING nftool 64

65 NETWORK FITTING TOOL Network fitting tool appears as show below 65

66 PROVIDING INPUT AND TARGET DATA Clicking on next button provide option to give input and target data. 66

67 VALIDATING AND TEST DATA Here we define training, validating, and test data. 67

68 DEFINING NETWORK SIZE Here we set the number of neurons in the fitting network s hidden layer. 68

69 TRAIN NETWORK 69

70 TRAINING PROCESS By clicking on train button training process starts. 70

71 PERFORMANCE PLOT 71

72 TRAINING STATE PLOT 72

73 REGRESSION PLOT 73

74 EVALUATE NETWORK 74

75 SAVE RESULTS 75

76 SIMULINK DIAGRAM Following are the simulink diagram of the network. 76

77 Query? 77

78 Epoch- During iterative training of a neural network, an Epoch is a single pass through the entire training set, followed by testing of the verification set. Generalization- how well will the network make predictions for cases that are not in the training set? Backpropagation- refers to the method for computing the gradient of the case-wise error function with respect to the weights for a feedforward network. Backprop- refers to a training method that uses backpropagation to compute the gradient. Backprop network- is a feedforward network trained by backpropagation. 78

6-2 Copyright 2011 Pearson Education, Inc. Publishing as Prentice Hall

6-2 Copyright 2011 Pearson Education, Inc. Publishing as Prentice Hall Business Intelligence and Decision Support Systems (9 th Ed., Prentice Hall) Chapter 6: Artificial Neural Networks for Data Mining Learning Objectives Understand the concept and definitions of artificial

More information

Introduction of connectionist models

Introduction of connectionist models Introduction of connectionist models Introduction to ANNs Markus Dambek Uni Bremen 20. Dezember 2010 Markus Dambek (Uni Bremen) Introduction of connectionist models 20. Dezember 2010 1 / 66 1 Introduction

More information

CS 510: Lecture 8. Deep Learning, Fairness, and Bias

CS 510: Lecture 8. Deep Learning, Fairness, and Bias CS 510: Lecture 8 Deep Learning, Fairness, and Bias Next Week All Presentations, all the time Upload your presentation before class if using slides Sign up for a timeslot google doc, if you haven t already

More information

Classification with Deep Belief Networks. HussamHebbo Jae Won Kim

Classification with Deep Belief Networks. HussamHebbo Jae Won Kim Classification with Deep Belief Networks HussamHebbo Jae Won Kim Table of Contents Introduction... 3 Neural Networks... 3 Perceptron... 3 Backpropagation... 4 Deep Belief Networks (RBM, Sigmoid Belief

More information

DEEP LEARNING AND ITS APPLICATION NEURAL NETWORK BASICS

DEEP LEARNING AND ITS APPLICATION NEURAL NETWORK BASICS DEEP LEARNING AND ITS APPLICATION NEURAL NETWORK BASICS Argument on AI 1. Symbolism 2. Connectionism 3. Actionism Kai Yu. SJTU Deep Learning Lecture. 2 Argument on AI 1. Symbolism Symbolism AI Origin Cognitive

More information

Python Machine Learning

Python Machine Learning Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled

More information

Life Time Milk Amount Prediction in Dairy Cows using Artificial Neural Networks

Life Time Milk Amount Prediction in Dairy Cows using Artificial Neural Networks International Journal of Recent Research and Review, Vol. V, March 2013 ISSN 2277 8322 Life Time Milk Amount Prediction in Dairy Cows using Artificial Neural Networks Shailesh Chaturvedi 1 Student M. Tech(CSE),

More information

Dudon Wai Georgia Institute of Technology CS 7641: Machine Learning Atlanta, GA

Dudon Wai Georgia Institute of Technology CS 7641: Machine Learning Atlanta, GA Adult Income and Letter Recognition - Supervised Learning Report An objective look at classifier performance for predicting adult income and Letter Recognition Dudon Wai Georgia Institute of Technology

More information

Artifi ifi i c l a Neur l a Networks Mohamed M. El Wakil t akil.ne 1

Artifi ifi i c l a Neur l a Networks Mohamed M. El Wakil  t akil.ne 1 Artificial i lneural lnetworks Mohamed M. El Wakil mohamed@elwakil.net 1 Agenda Natural Neural Networks Artificial Neural Networks XOR Example Design Issues Applications Conclusion 2 Artificial Neural

More information

Progress Report (Nov04-Oct 05)

Progress Report (Nov04-Oct 05) Progress Report (Nov04-Oct 05) Project Title: Modeling, Classification and Fault Detection of Sensors using Intelligent Methods Principal Investigator Prem K Kalra Department of Electrical Engineering,

More information

Machine Learning and Artificial Neural Networks (Ref: Negnevitsky, M. Artificial Intelligence, Chapter 6)

Machine Learning and Artificial Neural Networks (Ref: Negnevitsky, M. Artificial Intelligence, Chapter 6) Machine Learning and Artificial Neural Networks (Ref: Negnevitsky, M. Artificial Intelligence, Chapter 6) The Concept of Learning Learning is the ability to adapt to new surroundings and solve new problems.

More information

DEVELOPMENT OF AN ARTIFICIAL NEURAL NETWORK (ANN) FOR PREDICTING TRIBOLOGICAL PROPERTIES OF KENAF FIBRE REINFORCED EPOXY COMPOSITES (KFRE).

DEVELOPMENT OF AN ARTIFICIAL NEURAL NETWORK (ANN) FOR PREDICTING TRIBOLOGICAL PROPERTIES OF KENAF FIBRE REINFORCED EPOXY COMPOSITES (KFRE). University of Southern Queensland FACULTY OF ENGINEERING AND SURVEYING DEVELOPMENT OF AN ARTIFICIAL NEURAL NETWORK (ANN) FOR PREDICTING TRIBOLOGICAL PROPERTIES OF KENAF FIBRE REINFORCED EPOXY COMPOSITES

More information

Reverse Dictionary Using Artificial Neural Networks

Reverse Dictionary Using Artificial Neural Networks International Journal of Research Studies in Science, Engineering and Technology Volume 2, Issue 6, June 2015, PP 14-23 ISSN 2349-4751 (Print) & ISSN 2349-476X (Online) Reverse Dictionary Using Artificial

More information

Under the hood of Neural Machine Translation. Vincent Vandeghinste

Under the hood of Neural Machine Translation. Vincent Vandeghinste Under the hood of Neural Machine Translation Vincent Vandeghinste Recipe for (data-driven) machine translation Ingredients 1 (or more) Parallel corpus 1 (or more) Trainable MT engine + Decoder Statistical

More information

Neural Networks and the Brain a.k.a. Will I Graduate?

Neural Networks and the Brain a.k.a. Will I Graduate? Neural Networks and the Brain a.k.a. Will I Graduate? Laura Elisa Celis Friday, April 28th 1 Introduction Biological neurons, namely those found in the brain, are interconnected in vast and complex networks.

More information

Lecture 5: 21 September 2016 Intro to machine learning and single-layer neural networks. Jim Tørresen This Lecture

Lecture 5: 21 September 2016 Intro to machine learning and single-layer neural networks. Jim Tørresen This Lecture This Lecture INF3490 - Biologically inspired computing Lecture 5: 21 September 2016 Intro to machine learning and single-layer neural networks Jim Tørresen 1. Introduction to learning/classification 2.

More information

Functional Analysis of Artificial Neural Network for Dataset Classification

Functional Analysis of Artificial Neural Network for Dataset Classification Functional Analysis of Artificial Neural Network for Dataset Classification 49 Functional Analysis of Artificial Neural Network for Dataset Classification Rojalina Priyadarshini Deptt. Of IT C.V.Raman

More information

Fault Diagnosis of Power System Based on Neural Network

Fault Diagnosis of Power System Based on Neural Network Abstract Fault Diagnosis of Power System Based on Neural Network Jingwen Liu, Xianwen Hu, Daobing Liu Three Gorges University, College of Electrical and New energy, Yichang, 443000, China Using matlab

More information

4 Feedforward Neural Networks, Binary XOR, Continuous XOR, Parity Problem and Composed Neural Networks.

4 Feedforward Neural Networks, Binary XOR, Continuous XOR, Parity Problem and Composed Neural Networks. 4 Feedforward Neural Networks, Binary XOR, Continuous XOR, Parity Problem and Composed Neural Networks. 4.1 Objectives The objective of the following exercises is to get acquainted with the inner working

More information

ARTIFICIAL NEURAL NETWORKS

ARTIFICIAL NEURAL NETWORKS ARTIFICIAL NEURAL NETWORKS Amrender Kumar Indian Agricultural Statistics Research Institute, New Delhi-11012 akjha@iasri.res.in 1. Introduction Neural networks, more accurately called Artificial Neural

More information

Artificial Neural Networks. Andreas Robinson 12/19/2012

Artificial Neural Networks. Andreas Robinson 12/19/2012 Artificial Neural Networks Andreas Robinson 12/19/2012 Introduction Artificial Neural Networks Machine learning technique Learning from past experience/data Predicting/classifying novel data Biologically

More information

CS 2750: Machine Learning. Neural Networks. Prof. Adriana Kovashka University of Pittsburgh February 28, 2017

CS 2750: Machine Learning. Neural Networks. Prof. Adriana Kovashka University of Pittsburgh February 28, 2017 CS 2750: Machine Learning Neural Networks Prof. Adriana Kovashka University of Pittsburgh February 28, 2017 HW2 due Thursday Announcements Office hours on Thursday: 4:15pm-5:45pm Talk at 3pm: http://www.sam.pitt.edu/arc-

More information

Course Overview. Yu Hen Hu. Introduction to ANN & Fuzzy Systems

Course Overview. Yu Hen Hu. Introduction to ANN & Fuzzy Systems Course Overview Yu Hen Hu Introduction to ANN & Fuzzy Systems Outline Overview of the course Goals, objectives Background knowledge required Course conduct Content Overview (highlight of each topics) 2

More information

Machine Learning: Neural Networks. Junbeom Park Radiation Imaging Laboratory, Pusan National University

Machine Learning: Neural Networks. Junbeom Park Radiation Imaging Laboratory, Pusan National University Machine Learning: Neural Networks Junbeom Park (pjb385@gmail.com) Radiation Imaging Laboratory, Pusan National University 1 Contents 1. Introduction 2. Machine Learning Definition and Types Supervised

More information

Case Study on Classification of Glass using Neural Network Tool in MATLAB

Case Study on Classification of Glass using Neural Network Tool in MATLAB Case Study on Classification of Glass using Neural Network Tool in MATLAB Devika Chhachhiya Research Scholar Dept. of CS& IT THE IIS University, Amita Sharma Asst. Professor Dept. of CS& IT THE IIS University,

More information

Evolution of Neural Networks. October 20, 2017

Evolution of Neural Networks. October 20, 2017 Evolution of Neural Networks October 20, 2017 Single Layer Perceptron, (1957) Frank Rosenblatt 1957 1957 Single Layer Perceptron Perceptron, invented in 1957 at the Cornell Aeronautical Laboratory by Frank

More information

Soft Redundant Instrument for Metering Station in Gas Transportation System

Soft Redundant Instrument for Metering Station in Gas Transportation System Soft Redundant Instrument for Metering Station in Gas Transportation System N.S. Rosli, R. Ibrahim, I. Ismail Electrical and Electronic Engineering Department, Universiti Teknologi PETRONAS, 31750 Bandar

More information

Document Classification using Neural Networks Based on Words

Document Classification using Neural Networks Based on Words Volume 6, No. 2, March-April 2015 International Journal of Advanced Research in Computer Science RESEARCH PAPER Available Online at www.ijarcs.info Document Classification using Neural Networks Based on

More information

Intelligent Systems. Neural Networks. Copyright 2009 Dieter Fensel and Reto Krummenacher

Intelligent Systems. Neural Networks. Copyright 2009 Dieter Fensel and Reto Krummenacher Intelligent Systems Neural Networks Copyright 2009 Dieter Fensel and Reto Krummenacher 1 Where are we? # Title 1 Introduction 2 Propositional Logic 3 Predicate Logic 4 Theorem Proving, Description Logics

More information

Introduction to Computational Neuroscience A. The Brain as an Information Processing Device

Introduction to Computational Neuroscience A. The Brain as an Information Processing Device Introduction to Computational Neuroscience A. The Brain as an Information Processing Device Jackendoff (Consciousness and the Computational Mind, Jackendoff, MIT Press, 1990) argues that we can put off

More information

Gender Classification Based on FeedForward Backpropagation Neural Network

Gender Classification Based on FeedForward Backpropagation Neural Network Gender Classification Based on FeedForward Backpropagation Neural Network S. Mostafa Rahimi Azghadi 1, M. Reza Bonyadi 1 and Hamed Shahhosseini 2 1 Department of Electrical and Computer Engineering, Shahid

More information

Speech Accent Classification

Speech Accent Classification Speech Accent Classification Corey Shih ctshih@stanford.edu 1. Introduction English is one of the most prevalent languages in the world, and is the one most commonly used for communication between native

More information

Mangeron, No. 71A, , Iași, Romania,

Mangeron, No. 71A, , Iași, Romania, Nonconventional Technologies Review Romania, December, 2016 2016 Romanian Association of Nonconventional Technologies CONTRIBUTIONS REGARDING THE DESIGN OF SOME NEURO-FUZZY NEURAL NETWORKS APPLICABLE IN

More information

Artificial Neural Networks-A Study

Artificial Neural Networks-A Study International Journal of Emerging Engineering Research and Technology Volume 2, Issue 2, May 2014, PP 143-148 Artificial Neural Networks-A Study Er.Parveen Kumar 1, Er.Pooja Sharma 2, 1 Department of Electronics

More information

Deep Neural Networks for Acoustic Modelling. Bajibabu Bollepalli Hieu Nguyen Rakshith Shetty Pieter Smit (Mentor)

Deep Neural Networks for Acoustic Modelling. Bajibabu Bollepalli Hieu Nguyen Rakshith Shetty Pieter Smit (Mentor) Deep Neural Networks for Acoustic Modelling Bajibabu Bollepalli Hieu Nguyen Rakshith Shetty Pieter Smit (Mentor) Introduction Automatic speech recognition Speech signal Feature Extraction Acoustic Modelling

More information

Session 1: Gesture Recognition & Machine Learning Fundamentals

Session 1: Gesture Recognition & Machine Learning Fundamentals IAP Gesture Recognition Workshop Session 1: Gesture Recognition & Machine Learning Fundamentals Nicholas Gillian Responsive Environments, MIT Media Lab Tuesday 8th January, 2013 My Research My Research

More information

Connectionism (Artificial Neural Networks) and Dynamical Systems

Connectionism (Artificial Neural Networks) and Dynamical Systems COMP 40260 Connectionism (Artificial Neural Networks) and Dynamical Systems Part 2 Read Rethinking Innateness, Chapters 1 & 2 Let s start with an old neural network, created before training from data was

More information

Large Scale Data Analysis Using Deep Learning

Large Scale Data Analysis Using Deep Learning Large Scale Data Analysis Using Deep Learning Introduction to Deep Learning U Kang Seoul National University U Kang 1 In This Lecture Overview of deep learning History of deep learning and its recent advances

More information

Hypothetical Pattern Recognition Design Using Multi-Layer Perceptorn Neural Network For Supervised Learning

Hypothetical Pattern Recognition Design Using Multi-Layer Perceptorn Neural Network For Supervised Learning Hypothetical Pattern Recognition Design Using Multi-Layer Perceptorn Neural Network For Supervised Learning Md. Abdullah-al-mamun, Mustak Ahmed Abstract: Humans are capable to identifying diverse shape

More information

Inventor Chung T. Nguyen NOTTCE. The above identified patent application is available for licensing. Requests for information should be addressed to:

Inventor Chung T. Nguyen NOTTCE. The above identified patent application is available for licensing. Requests for information should be addressed to: Serial No. 802.572 Filing Date 3 February 1997 Inventor Chung T. Nguyen NOTTCE The above identified patent application is available for licensing. Requests for information should be addressed to: OFFICE

More information

Artificial Neural Networks

Artificial Neural Networks Artificial Neural Networks Andres Chavez Math 382/L T/Th 2:00-3:40 April 13, 2010 Chavez2 Abstract The main interest of this paper is Artificial Neural Networks (ANNs). A brief history of the development

More information

Artificial Neural Networks written examination

Artificial Neural Networks written examination 1 (8) Institutionen för informationsteknologi Olle Gällmo Universitetsadjunkt Adress: Lägerhyddsvägen 2 Box 337 751 05 Uppsala Artificial Neural Networks written examination Monday, May 15, 2006 9 00-14

More information

Neural Network and Adaptive Neuro- Fuzzy Inference System Applied to Civil Engineering Problems

Neural Network and Adaptive Neuro- Fuzzy Inference System Applied to Civil Engineering Problems 22 Neural Network and Adaptive Neuro- Fuzzy Inference System Applied to Civil Engineering Problems Mohammed A. Mashrei Thi-Qar University, College of Engineering, Civil Department Iraq 1. Introduction

More information

COMP150 DR Final Project Proposal

COMP150 DR Final Project Proposal COMP150 DR Final Project Proposal Ari Brown and Julie Jiang October 26, 2017 Abstract The problem of sound classification has been studied in depth and has multiple applications related to identity discrimination,

More information

Package ELMR. November 28, 2015

Package ELMR. November 28, 2015 Title Extreme Machine Learning (ELM) Version 1.0 Author Alessio Petrozziello [aut, cre] Package ELMR November 28, 2015 Maintainer Alessio Petrozziello Training and prediction

More information

A Neural Network GUI Tested on Text-To-Phoneme Mapping

A Neural Network GUI Tested on Text-To-Phoneme Mapping A Neural Network GUI Tested on Text-To-Phoneme Mapping MAARTEN TROMPPER Universiteit Utrecht m.f.a.trompper@students.uu.nl Abstract Text-to-phoneme (T2P) mapping is a necessary step in any speech synthesis

More information

Introduction to the Theories of Machine Learning

Introduction to the Theories of Machine Learning Introduction to the Theories of Machine Learning with Feed-Forward Artificial Neural Networks and Evolving with Genetic Algorithms Second Research Paper Bachelor course on Media Technology at St. Pölten

More information

Available online at ScienceDirect. Agriculture and Agricultural Science Procedia 3 ( 2015 ) 14 19

Available online at  ScienceDirect. Agriculture and Agricultural Science Procedia 3 ( 2015 ) 14 19 Available online at www.sciencedirect.com ScienceDirect Agriculture and Agricultural Science Procedia 3 ( 2015 ) 14 19 The 2014 International Conference on Agro-industry (ICoA) : Competitive and sustainable

More information

Synaptic Weight Noise During MLP Learning Enhances Fault-Tolerance, Generalisation and Learning Trajectory

Synaptic Weight Noise During MLP Learning Enhances Fault-Tolerance, Generalisation and Learning Trajectory Synaptic Weight Noise During MLP Learning Enhances Fault-Tolerance, Generalisation and Learning Trajectory Alan F. Murray Dept. of Electrical Engineering Edinburgh University Scotland Peter J. Edwards

More information

EAST: An Exponential Adaptive Skipping Training Algorithm for Multilayer Feedforward Neural Networks

EAST: An Exponential Adaptive Skipping Training Algorithm for Multilayer Feedforward Neural Networks EAST: An Exponential Adaptive Skipping Algorithm for Multilayer Feedforward Neural Networks R.MANJULA DEVI Research scholar and Assistant Pressor(Senior Grade) Department Computer Science and Engineering

More information

Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches

Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches Qandeel Tariq, Alex Kolchinski, Richard Davis December 6, 206 Introduction This paper

More information

Course Outline. Course Grading. Where to go for help. Academic Integrity. EE-589 Introduction to Neural Networks NN 1 EE

Course Outline. Course Grading. Where to go for help. Academic Integrity. EE-589 Introduction to Neural Networks NN 1 EE EE-589 Introduction to Neural Assistant Prof. Dr. Turgay IBRIKCI Room # 305 (322) 338 6868 / 139 Wensdays 9:00-12:00 Course Outline The course is divided in two parts: theory and practice. 1. Theory covers

More information

The Generalized Delta Rule and Practical Considerations

The Generalized Delta Rule and Practical Considerations The Generalized Delta Rule and Practical Considerations Introduction to Neural Networks : Lecture 6 John A. Bullinaria, 2004 1. Training a Single Layer Feed-forward Network 2. Deriving the Generalized

More information

Neural Network Language Models

Neural Network Language Models Neural Network Language Models Steve Renals Automatic Speech Recognition ASR Lecture 12 6 March 2014 ASR Lecture 12 Neural Network Language Models 1 Neural networks for speech recognition Introduction

More information

Neuromorphic Architectures. James Kempsell Chris Radnovich

Neuromorphic Architectures. James Kempsell Chris Radnovich Neuromorphic Architectures James Kempsell Chris Radnovich Outline What are Neuromorphic Architectures? Biological Comparison How Neuromorphic Architectures work? Example 2-input Gate Example: Two Layer

More information

Introducing Deep Learning with MATLAB

Introducing Deep Learning with MATLAB Introducing Deep Learning with MATLAB What is Deep Learning? Deep learning is a type of machine learning in which a model learns to perform classification tasks directly from images, text, or sound. Deep

More information

Big Data Analysis Using Neuro-Fuzzy System

Big Data Analysis Using Neuro-Fuzzy System San Jose State University SJSU ScholarWorks Master's Projects Master's Theses and Graduate Research Spring 2014 Big Data Analysis Using Neuro-Fuzzy System Amir Eibagi Follow this and additional works at:

More information

ARTIFICIAL NEURAL NETWORK (ANN) INSPIRED FROM BIOLOGICAL NERVOUS SYSTEM

ARTIFICIAL NEURAL NETWORK (ANN) INSPIRED FROM BIOLOGICAL NERVOUS SYSTEM ARTIFICIAL NEURAL NETWORK (ANN) INSPIRED FROM BIOLOGICAL NERVOUS SYSTEM Kendar Pratap 1, Shelja 2 1 Lecturer, Govt. National P.G.College, Sirsa, Haryana (INDIA) 2 Lecturer, Mata Sahib Kaur Khalsa College

More information

Introduction to Deep Learning

Introduction to Deep Learning Introduction to Deep Learning M S Ram Dept. of Computer Science & Engg. Indian Institute of Technology Kanpur Reading of Chap. 1 from Learning Deep Architectures for AI ; Yoshua Bengio; FTML Vol. 2, No.

More information

SIMULATION DEPTH OF BRIDGE PIER SCOURING USING ARTIFICIAL NEURAL NETWORK AND ADAPTIVE NEURO FUZZY INFERENCE SYSTEM

SIMULATION DEPTH OF BRIDGE PIER SCOURING USING ARTIFICIAL NEURAL NETWORK AND ADAPTIVE NEURO FUZZY INFERENCE SYSTEM SIMULATIO DEPTH OF BRIDGE PIER SCOURIG USIG ARTIFICIAL EURAL ETWORK AD ADAPTIVE EURO FUZZY IFERECE SYSTEM * Behboud Mansoori and Mehrdad Fereydooni Department of Civil Engineering, Larestan Branch, Islamic

More information

A study of the NIPS feature selection challenge

A study of the NIPS feature selection challenge A study of the NIPS feature selection challenge Nicholas Johnson November 29, 2009 Abstract The 2003 Nips Feature extraction challenge was dominated by Bayesian approaches developed by the team of Radford

More information

Artificial Neural Networks in Data Mining

Artificial Neural Networks in Data Mining IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 18, Issue 6, Ver. III (Nov.-Dec. 2016), PP 55-59 www.iosrjournals.org Artificial Neural Networks in Data Mining

More information

Neural Networks in Bank Insolvency Prediction

Neural Networks in Bank Insolvency Prediction 240 IJCSNS International Journal of Computer Science and Network Security, VOL.10 No.5, May 2010 Neural Networks in Bank Insolvency Prediction Qeethara Kadhim Al-Shayea, Ghaleb A. El-Refae, and Shurouq

More information

Simple recurrent networks

Simple recurrent networks CHAPTER 8 Simple recurrent networks Introduction In Chapter 7, you trained a network to detect patterns which were displaced in space. Your solution involved a hand-crafted network with constrained weights

More information

Neural Networks and Learning Machines

Neural Networks and Learning Machines Neural Networks and Learning Machines Third Edition Simon Haykin McMaster University Hamilton, Ontario, Canada Upper Saddle River Boston Columbus San Francisco New York Indianapolis London Toronto Sydney

More information

Article from. Predictive Analytics and Futurism December 2015 Issue 12

Article from. Predictive Analytics and Futurism December 2015 Issue 12 Article from Predictive Analytics and Futurism December 2015 Issue 12 The Third Generation of Neural Networks By Jeff Heaton Neural networks are the phoenix of artificial intelligence. Right now neural

More information

Adaptive Behavior with Fixed Weights in RNN: An Overview

Adaptive Behavior with Fixed Weights in RNN: An Overview & Adaptive Behavior with Fixed Weights in RNN: An Overview Danil V. Prokhorov, Lee A. Feldkamp and Ivan Yu. Tyukin Ford Research Laboratory, Dearborn, MI 48121, U.S.A. Saint-Petersburg State Electrotechical

More information

DESIGN OF ARTIFICIAL BACK PROPAGATION NEURAL NETWORK FOR DRUG PATTERN RECOGNITION

DESIGN OF ARTIFICIAL BACK PROPAGATION NEURAL NETWORK FOR DRUG PATTERN RECOGNITION DESIGN OF ARTIFICIAL BACK PROPAGATION NEURAL NETWORK FOR DRUG PATTERN RECOGNITION Abstract In recent years considerable effort s has been devoted to applying pattern recognition techniques to the complex

More information

Artificial Intelligence. CSD 102 Introduction to Communication and Information Technologies Mehwish Fatima

Artificial Intelligence. CSD 102 Introduction to Communication and Information Technologies Mehwish Fatima Artificial Intelligence CSD 102 Introduction to Communication and Information Technologies Mehwish Fatima Objectives Division of labor Knowledge representation Recognition tasks Reasoning tasks Mehwish

More information

Deep Learning and Optical Character Recognition

Deep Learning and Optical Character Recognition FAISAL SHAFAIT Deep Learning and Optical Character Recognition Artificial Neural Networks (ANNs) Goal: make computers intelligent Idea: Model human brain Synapse Dendrite Artificial Neural Network x 2

More information

Available online:

Available online: VOL4 NO. 1 March 2015 - ISSN 2233 1859 Southeast Europe Journal of Soft Computing Available online: www.scjournal.ius.edu.ba A study in Authorship Attribution: The Federalist Papers Nesibe Merve Demir

More information

Simple Evolving Connectionist Systems and Experiments on Isolated Phoneme Recognition

Simple Evolving Connectionist Systems and Experiments on Isolated Phoneme Recognition Simple Evolving Connectionist Systems and Experiments on Isolated Phoneme Recognition Michael Watts and Nik Kasabov Department of Information Science University of Otago PO Box 56 Dunedin New Zealand EMail:

More information

EVOLVING NEURAL NETWORKS WITH HYPERNEAT AND ONLINE TRAINING. Shaun M. Lusk, B.S.

EVOLVING NEURAL NETWORKS WITH HYPERNEAT AND ONLINE TRAINING. Shaun M. Lusk, B.S. EVOLVING NEURAL NETWORKS WITH HYPERNEAT AND ONLINE TRAINING by Shaun M. Lusk, B.S. A thesis submitted to the Graduate Council of Texas State University in partial fulfillment of the requirements for the

More information

Disclaimer. Copyright. Machine Learning Mastery With Weka

Disclaimer. Copyright. Machine Learning Mastery With Weka i Disclaimer The information contained within this ebook is strictly for educational purposes. If you wish to apply ideas contained in this ebook, you are taking full responsibility for your actions. The

More information

Neural Networks. CSC 4504 : Langages formels et applications. J Paul Gibson, D311.

Neural Networks. CSC 4504 : Langages formels et applications. J Paul Gibson, D311. CSC 4504 : Langages formels et applications J Paul Gibson, D311 paul.gibson@telecom-sudparis.eu /~gibson/teaching/csc4504/problem11-neuralnetworks.pdf Neural Networks 1 2 The following slides are a summary

More information

SRM University. Faculty of Engineering and Technology. Department of Electronics and Communication Engineering

SRM University. Faculty of Engineering and Technology. Department of Electronics and Communication Engineering SRM University Faculty of Engineering and Technology Department of Electronics and Communication Engineering Course code: EC0054 Course title: Neural network and fuzzy Semester: VII Course time: Jan 2011-March

More information

A LEARNING PROCESS OF MULTILAYER PERCEPTRON FOR SPEECH RECOGNITION

A LEARNING PROCESS OF MULTILAYER PERCEPTRON FOR SPEECH RECOGNITION International Journal of Pure and Applied Mathematics Volume 107 No. 4 2016, 1005-1012 ISSN: 1311-8080 (printed version); ISSN: 1314-3395 (on-line version) url: http://www.ijpam.eu doi: 10.12732/ijpam.v107i4.18

More information

Contents - MLPs & Pose/Expression Classification 1. Contents

Contents - MLPs & Pose/Expression Classification 1. Contents Contents - MLPs & Pose/Expression Classification 1 Contents Contents... 1 Abstract...3 Acknowledgements... 4 1. Introduction... 5 2. Possible Real-world Applications... 7 3. Facial Expression Analysis...

More information

In-depth: Deep learning (one lecture) Applied to both SL and RL above Code examples

In-depth: Deep learning (one lecture) Applied to both SL and RL above Code examples Introduction to machine learning (two lectures) Supervised learning Reinforcement learning (lab) In-depth: Deep learning (one lecture) Applied to both SL and RL above Code examples 2017-09-30 2 1 To enable

More information

CHAPTER 1 AN OVERVIEW OF VLSI IMPLEMENTATION OF ANN

CHAPTER 1 AN OVERVIEW OF VLSI IMPLEMENTATION OF ANN 1 CHAPTER 1 AN OVERVIEW OF VLSI IMPLEMENTATION OF ANN 1.1 INTRODUCTION A Biological Neural Network (BNN) has highly interconnected neurons that co-ordinate all the functions like breathing, thinking and

More information

Implementation of Backpropagation Algorithm: A Neural Network Approach for Pattern Recognition

Implementation of Backpropagation Algorithm: A Neural Network Approach for Pattern Recognition International Journal of Engineering Research and Development ISSN: 2278-067X, Volume 1, Issue 5 (June 2012), PP.30-37 www.ijerd.com Implementation of Backpropagation Algorithm: A Neural Network Approach

More information

Application of neural networks to the prediction of the behavior of reinforced composite bridges

Application of neural networks to the prediction of the behavior of reinforced composite bridges Application of neural networks to the prediction of the behavior of reinforced composite bridges *Abdessemed Mouloud 1) and Kenai Said 2) 1), 2) Department of Civil Engineering, Blida1, BP 270, Route Soumaa,

More information

Sapienza Università di Roma

Sapienza Università di Roma Sapienza Università di Roma Machine Learning Course Prof: Paola Velardi Deep Q-Learning with a multilayer Neural Network Alfonso Alfaro Rojas - 1759167 Oriola Gjetaj - 1740479 February 2017 Contents 1.

More information

Machine Learning with MATLAB Antti Löytynoja Application Engineer

Machine Learning with MATLAB Antti Löytynoja Application Engineer Machine Learning with MATLAB Antti Löytynoja Application Engineer 2014 The MathWorks, Inc. 1 Goals Overview of machine learning Machine learning models & techniques available in MATLAB MATLAB as an interactive

More information

Deep Learning in Customer Churn Prediction: Unsupervised Feature Learning on Abstract Company Independent Feature Vectors

Deep Learning in Customer Churn Prediction: Unsupervised Feature Learning on Abstract Company Independent Feature Vectors 1 Deep Learning in Customer Churn Prediction: Unsupervised Feature Learning on Abstract Company Independent Feature Vectors Philip Spanoudes, Thomson Nguyen Framed Data Inc, New York University, and the

More information

Isolated Speech Recognition Using MFCC and DTW

Isolated Speech Recognition Using MFCC and DTW Isolated Speech Recognition Using MFCC and DTW P.P.S.Subhashini Associate Professor, RVR & JC College of Engineering. ABSTRACT This paper describes an approach of isolated speech recognition by using the

More information

L1: Course introduction

L1: Course introduction Introduction Course organization Grading policy Outline What is pattern recognition? Definitions from the literature Related fields and applications L1: Course introduction Components of a pattern recognition

More information

Prediction of e-learning Efficiency by Neural Networks

Prediction of e-learning Efficiency by Neural Networks BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 12, No 2 Sofia 2012 Prediction of e-learning Efficiency by Neural Networks Petar Halachev Institute of Information and Communication

More information

Time Series Prediction Using Radial Basis Function Neural Network

Time Series Prediction Using Radial Basis Function Neural Network International Journal of Electrical and Computer Engineering (IJECE) Vol. 5, No. 4, August 2015, pp. xx~xx ISSN: 2088-8708 31 Time Series Prediction Using Radial Basis Function Neural Network Haviluddin*,

More information

Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition

Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition Alex Graves 1, Santiago Fernández 1, Jürgen Schmidhuber 1,2 1 IDSIA, Galleria 2, 6928 Manno-Lugano, Switzerland {alex,santiago,juergen}@idsia.ch

More information

Psychology 452 Week 1: Connectionism and Association

Psychology 452 Week 1: Connectionism and Association Psychology 452 Week 1: Connectionism and Association Course Overview Properties Of Connectionism Building Associations Into Networks The Hebb Rule The Delta Rule Michael R.W. Dawson PhD from University

More information

Machine Learning for Predictive Modelling Rory Adams

Machine Learning for Predictive Modelling Rory Adams Machine Learning for Predictive Modelling Rory Adams 2015 The MathWorks, Inc. 1 Agenda Machine Learning What is Machine Learning and why do we need it? Common challenges in Machine Learning Example: Human

More information

Training Neural Networks, Part I. Fei-Fei Li & Justin Johnson & Serena Yeung. Lecture 6-1

Training Neural Networks, Part I. Fei-Fei Li & Justin Johnson & Serena Yeung. Lecture 6-1 Lecture 6: Training Neural Networks, Part I Lecture 6-1 Administrative Assignment 1 due Thursday (today), 11:59pm on Canvas Assignment 2 out today Project proposal due Tuesday April 25 Notes on backprop

More information

Machine Learning Algorithms: A Review

Machine Learning Algorithms: A Review Machine Learning Algorithms: A Review Ayon Dey Department of CSE, Gautam Buddha University, Greater Noida, Uttar Pradesh, India Abstract In this paper, various machine learning algorithms have been discussed.

More information

Intelligent Decision Support System for Construction Project Monitoring

Intelligent Decision Support System for Construction Project Monitoring Intelligent Decision Support System for Construction Project Monitoring Muhammad Naveed Riaz Faculty of Computing Riphah International University Islamabad, Pakistan. meet_navid@yahoo.com Abstract Business

More information

Table Of Contents. Introduction To Neural Network p/g 3. Neural Network For Credit Approval p/g 9. Neural Network For Sales Forecasting p/g 27

Table Of Contents. Introduction To Neural Network p/g 3. Neural Network For Credit Approval p/g 9. Neural Network For Sales Forecasting p/g 27 Build Neural Network With MS Ecel Published by XLPert Enterprise Copyright 2009 by XLPert Enterprise. All rights reserved. No part of this book may be reproduced, stored or distributed in any form or by

More information

Intelligent Tutoring Systems using Reinforcement Learning to teach Autistic Students

Intelligent Tutoring Systems using Reinforcement Learning to teach Autistic Students Intelligent Tutoring Systems using Reinforcement Learning to teach Autistic Students B. H. Sreenivasa Sarma 1 and B. Ravindran 2 Department of Computer Science and Engineering, Indian Institute of Technology

More information

Zaki B. Nossair and Stephen A. Zahorian Department of Electrical and Computer Engineering Old Dominion University Norfolk, VA, 23529

Zaki B. Nossair and Stephen A. Zahorian Department of Electrical and Computer Engineering Old Dominion University Norfolk, VA, 23529 SMOOTHED TIME/FREQUENCY FEATURES FOR VOWEL CLASSIFICATION Zaki B. Nossair and Stephen A. Zahorian Department of Electrical and Computer Engineering Old Dominion University Norfolk, VA, 23529 ABSTRACT A

More information

Exemplifying Workflow Sequencing and Analysis in Artificial Neural Networks

Exemplifying Workflow Sequencing and Analysis in Artificial Neural Networks Volume 5, No. 2, March 2014 (Special Issue) International Journal of Advanced Research in Computer Science RESEARCH PAPER Available Online at www.ijarcs.info ISSN No. 0976-5697 Exemplifying Workflow Sequencing

More information

A SELF-LEARNING NEURAL NETWORK

A SELF-LEARNING NEURAL NETWORK 769 A SELF-LEARNING NEURAL NETWORK A. Hartstein and R. H. Koch IBM - Thomas J. Watson Research Center Yorktown Heights, New York ABSTRACf We propose a new neural network structure that is compatible with

More information