A Neural Network Model For Concept Formation
|
|
- Ferdinand Goodwin
- 6 years ago
- Views:
Transcription
1 A Neural Network Model For Concept Formation Jiawei Chen, Yan Liu, Qinghua Chen, Jiaxin Cui Department of Systems Science School of Management Beijing Normal University Beijing , P.R.China. Fukang Fang State Key Laboratory of Cognitive Neuroscience and Learning Beijing Normal University Beijing , P.R.China. Abstract Acquisition of abstract concept is the key step in human intelligence development, but the neural mechanism of concept formation is not clear yet. Researches on complexity and self organization theory indicate that concept is a result of emergence of neural system and it should be represented by an attractor. Associative learning and hypothesis elimination are considered as the mechanisms of concept formation, and we think that Hebbian learning rule can be used to describe the two operations. In this paper, a neural network is constructed based on Hopfield model, and the weights are updated according to Hebbian rule. The forming processes of natural concept, number concept and addition concept are simulated by using our model. Facts from real neuroanatomy provide some evidences for our model. 1 Introduction It is an important and difficult problem that how concept is abstracted from concrete instances with some common features. In the last 100 years, issues about concept formation and concept development are discussed by psychologist using the prevalent means of behavioral experiments [10], but the neural mechanism of these processes are not clear. Recently, neural network models [1, 2] are used to discuss the problems about concept formation from the angel of word learning. This method first needs to explain that how a concept is represented in the neural system, i.e. how a cognition state, such as concept or memory, is represented by a physical system which consists of neurons and connections between them. Hopfield successfully discussed the problem using the thoughts on emergence, and his model has been applicated frequently in the field of cognition. Hopfield network [7] indicated that an attractor which dominates substantial region around it in the phase space denotes a nominally assigned memory. The same as memory, concept is also cognition state and should be represented by attractor of the physical system. Obviously, the collective behaviors are more appropriate than individual unit to express cognition states because of more robustness and stability. Some attractor networks [6, 4] have been created to study questions about language learning. Concrete instance can be represented by many features, and maybe language is the best way to describe these features. Word meanings carve up the world in complex ways, such that an entity, action, property, or relation can typically be labeled by multiple words [13]. Language can be seen as a simple and complete projection of the real world. The early stages of word learning are often used to study the issues of concept formation in many literatures [13, 12]. Although these works mainly focus on the acquisition of concrete concepts, abstract concepts should can be discussed by using language as research object. Some Chinese characters are pictograph, and the grapheme can express more signification than alphabet language systems. So from the angle of modeling, Chinese characters would be more suitable than other languages for exploring the neural mechanisms of concept formation. In this article, the process of how features are extracted from samples is simulated and the neural mechanisms would be discussed. A model based on Hopfield network is constructed, and the connection weights are updated based on a variant of Hebb learning rule. Samples with some common features, certainly each of them has some special features, are used to train the model. The weights states and test results indicate that the common features of the samples can be extracted by the model and represented by an attractor of the system. In the first section of this paper, a model based on Hopfield network is constructed, including architecture, weight update algorithm, samples, etc. The second section, three groups of samples are used to train the model and the simulated results are illuminated. At last, the neural mechanism of concept formation is summed up and explained.
2 2 Model Let us consider a fully connected recurrent neural network, which is a variant of Hopfield model. The details of the network, such as the Architecture, weight adjustment, samples and training, etc. are described below. 2.1 Architecture Of The Network Our neural network just has one layer and is composed of N neurons. Each neuron i has two states V i = 0 or V i = 1, which denote not firing or firing at maximum rate respectively. The instantaneous state of the system is specified by listing the N values of V i, so it is represented by a binary word of N bits. The network is full connected, ie. all neurons connected with each others. The strength of the connection from neuron j to neuron i is defined as w ij. We suppose that 0 w ij 1, i, j and w ii = 0, i. How the system deals with information is decided by the current weights state. Because there is only one layer in our model, each neuron plays the roles of receiving input vector form environment and denoting the output result. Input vector X i should be represented by a binary word of N bits so as to be matched with the neuron state. For example, the ith input vector can be written as X i = [x i,1, x i,2,..., x i,n ], x i,j = 0 or 1. The output of our network can be represented by the states of all the N neurons. 2.2 Weight Update Algorithm All the neuron states should be determined before the weights are updated. Two cases will be considered for calculating the neuron state. On the one hand, when a sample is input to the network the neurons states are changed to the same as the input vector, i.e. V i = x c,i, here x c,i denotes the ith component of the current input vector; On the other hand, when no external instance were provided, the neuron state changes with time according to the following algorithm. N V i = hardlim( w ij V j θ i ) (1) j=1 where θ i denotes the threshold of the ith neuron. In a certain system, we assume that all the thresholds are equal to a constant θ. In the formal theory of neural networks the weight w ij is considered as a parameter that can be adjusted so as to optimize the performance of a network for a given task. In our model, we assume that the weight will be updated according to Hebbian learning rule [5], i.e. the network learns by strengthening connection weights between neurons activated at the same time. It can be written as following: η w ij d, if V i = 1, V j = 1 η w ij d, if V i = 1, V j = 0 w ij = η w ij d, if V i = 0, V j = 1 d, if V i = 0, V j = 0 here, 0 < η < 1 is a small constant called learning rate. The parameter d is a small positive constant that describes the rate by which w ij decays back to zero in the absence of stimulation. Of course, equation (2) is just one of the possible forms to specify rules for the growth and decay of the weights, and there are some difference with the other forms of Hebb rule [3]. From the formula (2) we can see that synaptic efficacy w ij would grow without limit if the same potentiating stimulus is applied over and over again. A saturation of the weights should be consider. On the other hand, the synaptic efficacy w ij should be non-negative. These two restrictions can be achieved by setting: 1, if w ij (t + 1) > 1 w ij (t + 1) = w ij (t + 1), if 0 w ij (t + 1) 1 (3) 0, if w ij (t + 1) < Samples For Training In our model, the features of sample are represented by the dot matrix of Chinese character. Each element of the matrix denotes one feature. The value of each element is 1 or 0, which indicates that the sample has or hasn t the feature corresponding to the element. Each neuron in the network has only two states, any input vector should be represented by a binary word so as to be matched with the neuron state. Combination of a few Chinese characters is chosen as sample of our model, such as,, etc. Each sample is presented using m dot matrix, where m denotes the number of Chinese character. The matrix element in the character s stroke is set to 1, otherwise should be set to 0. At last the input vector can be obtained by converting the dot matrix into a vector, example of an instance is shown in Figure 1. It is need to point out that different training set should be used for different purpose of experiment. Each sample of a training set has the same number of characters, but samples belong to different sets perhaps has different number of characters. The number of neurons in our network is decided by the number of characters in each sample. For examples, a network using as a sample would consist of = 512 neurons. (2)
3 is the essential feature that all horses have altogether and the color is the special feature which each sample has it s own solely. The model simulates the process in which the abstract concept horse is extracted from several samples of horse with different color by drawing out the common features and eliminating the unique characteristic. The concept should correspond to an attractor of our model. Figure 1. The representation of an instance (A)The instance includes 2 characters; (B)The instance represented by dot matrix; (C)The input vector obtained by putting the dot matrix into one column. 2.4 Training And Testing Training is a procedure of weights updated iteratively according to the external input. In a certain experiment, the training set for the network consisted of k samples with some identical properties. During each epoch, an instance randomly selected from the training set (k samples) is shown to the network. The neuron states are changed to be the same as the input vector, and the weights are adopted according with the formulae 2 and 3. The network should be trained again and again until the weight matrix is changed over a small range at last. After learning a training set in which the instances have some identical features, does the network know these features? We addressed this question by presenting the network with some input patterns and examining the output patterns of network. If the network has learned these features, then the network will evolve to a stable state witch denotes the concept when a sample with all or most of the identical features is shown to the network. In the testing procedure, all of the weights will be fixed and a test sample would be shown to the network. The output of the network can be calculated according to formulae 1. 3 Simulation Results Firstly, how the concept horse is extracted from samples is simulated. For a horse sample, we consider the following two features: the shape and the color. The shape Figure 2. The training set includes six instances. Six samples shown in figure 2 are used in the model, i.e. k = 6. The neuron number N = 512 can be determined by calculating the dots of any sample. Before training, we initialize the weights randomly from 0.2 to 1, i.e. { 0.2 < Wij (0) = W ji (0) < 1, i, j and i j; (4) W ii (0) = 0, i. the wights are shown in Figure 3(A). The other parameters are set to be η = 0.25, d = The network is trained 150 times with samples selected from the training set randomly and the weights are shown in Figure 3(B). Figure 3. The weight matrix evolve from a random initial state to a stable final state. (A)The initial state; (B)The final state. By comparing the trained weights with the initial, we can obtain the following results: 1. The weight matrix changes from random distribution to a stable state with the
4 training process, and no any obvious change will happen once the weights reach the stable state. 2. From figure 3, we can see that the number of connections between neurons is massively reduced, but the average connection strength changes from w ij = to w ij = in the process of training. The more connections indicate the more plasticity of the network, and the strong and stable connections perhaps denote certain cognition patterns. 3. Because the similarity between individual character of the samples, some elements in the top-left quarter of the weight matrix are not 0. All elements except the bottom-right quarter will change to 0 by increasing the number of samples in the training set. The stable state of the weights is a attractor of the system, and the fix point correspond to a cognition pattern of concept horse extracted from samples. In theoretical aspect, the analysis of the collective behaviors of the neurons may refer to hopfield s work. We can also exam the attractor and cognition state of the system directly using three test samples which displayed in Figure 4(A), (B), (C). Here, we set the parameter θ = 30. The network weights are fixed and the samples are input to the network respectively. The output of the network can be calculated by formulae 1 and the test results are also shown in Figure 4. In the phase space, the three samples is dominated by a attractor which is the nominally assigned concept horse and they will eventually settle into the attractor state. On the other hand, sample which is not in the domain of the attractor will not evolving to the stable state. An example is shown in Figure 4(D). Figure 4. The attractor of the network is tested though three positive examples and a negative instance. (A) A sample arbitrarily selected from the training set; (B) An incomplete sample includes a majority of, but not all, features of the concept horse; (C) A sample includes all features of horse, and individual features are given arbitrarily; (D) A sample includes only little feature of the concept horse, although its individual features are used in the training process. Our model simulates the forming process of natural concept by using the horse as an example. In fact, a class of concept which formed by extracting the common features from concrete instances can be simulated using our model, such as the concept of natural number and addition. The simulation result is shown in Figure 5. Figure 5. Two other examples of concept formation. (A) The number concept 3 is extracted from the six samples; (B)The addition concept 2+3=5 is extracted from the six samples. Certainly, the concept of number consists of many connotations include concrete concept, abstract concept, ordering concept and number structure [9]. Our model only simulates the emerge process from concrete concept to abstract concept. 4 Discussion As we mentioned above, the issues of concept formation have been discussed from the angel of word learning, and two broad classes of proposals for how word learning works have been dominant in the literature: hypothesis elimination and associative learning. we consider that the union of the two operations may be the mechanism of concept formation. On the one hand, The features that all samples have are called essential features, and the connections between neurons which represent the essential features will be strengthened during the training process. Associative learning works. On the other hand, the connections between individual features each other and between individual features and essential features become weaker and weaker gradually. Hypothesis elimination works. These two operations can be precisely described by Hebbian learning rule. So Hebbian learning rule should be the neural mechanism of concept formation in certain condition. The reliability of our model can be explained by the fact from neuroanatomy. Experiments indicate that the number of connections between neurons is massively reduced in the adult compared to the infant. In the cat, for example, there is a huge decrease in the number of callosal axons during neonatal life, and a 90% reduction in the number of synapses and branches of the axonal arbors of the remaining fibres [8, 11]. The fact is similar with out simulation results. However, the process of concept formation is very complicated and the essential of the process is emergence.
5 Hebbian learning rule perhaps can curve up the forming mechanism of some simple concept. For complex scientific concept and social concept, more kinds of factors and more complex mechanisms should be considered. Acknowledgement This work is supported by NSFC under the grant No , No and No References [1] E. Colunga, L. B. Smith, From the Lexicon to Expectations About Kinds: A Role for Associative Learning, Psychological Review 2 (2005) [2] M. Gasser, L. B. Smith, Learning nouns and adjectives: A connectionist approach. Language and Cognitive Processes 13 (1998) [3] W. Gerstner, W. M. Kistler, Spiking Neuron Models: Single Neurons, Populations, Plasticity, Cambridge University Press, Cambridge, [4] M. W. Harm, M. S. Seidenberg, Phonology, Reading Acquisition, and Dyslexia: Insights from Connectionist Models, Psychological Review 106 (1999) [5] D. O. Hebb, The Organization of Behavior, Wiley, New York, [6] G. E. Hinton, T. Shallice, Lesioning an attractor network: Investigations of acquired dyslexia. Psychological Review 98 (1991) [7] J. J. Hopfield, Neural Networks and Physical Systems with Emergent Collective Computational Abilities, Proc. NatL Acad. Sci. USA 79 (1982) [8] G. M. Innocenti, Exuberant development of connections, and its possible permissive role in cortical evolution, Trends Neurosci. 18 (1995) [9] C. Lin, The study on the development of the number concept and operational ability in schoolchildren, Acta Psychologica Sinica 3 (1981) [10] E. Machery, 100 years of psychology of concepts: the theoretical notion of concept and its operationalization, Studies in History and Philosophy of Biological and Biomedical Sciences 38 (2007) [11] B. Payne, H. Pearson, P. Cornwell, Deveopment of visual and auditory cortical connections in cat. Cerebral Cortex 7 (1988) [12] T. Regier, The Emergence of Words: Attentional Learning in Form and Meaning, Cognitive Science 29 (2005) [13] F. Xu, J. B. Tenenbaum, Word Learning as Bayesian Inference, Psychological Review 2 (2007)
OPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS
OPTIMIZATINON OF TRAINING SETS FOR HEBBIAN-LEARNING- BASED CLASSIFIERS Václav Kocian, Eva Volná, Michal Janošek, Martin Kotyrba University of Ostrava Department of Informatics and Computers Dvořákova 7,
More informationArtificial Neural Networks
Artificial Neural Networks Andres Chavez Math 382/L T/Th 2:00-3:40 April 13, 2010 Chavez2 Abstract The main interest of this paper is Artificial Neural Networks (ANNs). A brief history of the development
More informationLanguage Acquisition Fall 2010/Winter Lexical Categories. Afra Alishahi, Heiner Drenhaus
Language Acquisition Fall 2010/Winter 2011 Lexical Categories Afra Alishahi, Heiner Drenhaus Computational Linguistics and Phonetics Saarland University Children s Sensitivity to Lexical Categories Look,
More informationProposal of Pattern Recognition as a necessary and sufficient principle to Cognitive Science
Proposal of Pattern Recognition as a necessary and sufficient principle to Cognitive Science Gilberto de Paiva Sao Paulo Brazil (May 2011) gilbertodpaiva@gmail.com Abstract. Despite the prevalence of the
More informationCourse Outline. Course Grading. Where to go for help. Academic Integrity. EE-589 Introduction to Neural Networks NN 1 EE
EE-589 Introduction to Neural Assistant Prof. Dr. Turgay IBRIKCI Room # 305 (322) 338 6868 / 139 Wensdays 9:00-12:00 Course Outline The course is divided in two parts: theory and practice. 1. Theory covers
More informationA Neural Network GUI Tested on Text-To-Phoneme Mapping
A Neural Network GUI Tested on Text-To-Phoneme Mapping MAARTEN TROMPPER Universiteit Utrecht m.f.a.trompper@students.uu.nl Abstract Text-to-phoneme (T2P) mapping is a necessary step in any speech synthesis
More informationEvolution of Symbolisation in Chimpanzees and Neural Nets
Evolution of Symbolisation in Chimpanzees and Neural Nets Angelo Cangelosi Centre for Neural and Adaptive Systems University of Plymouth (UK) a.cangelosi@plymouth.ac.uk Introduction Animal communication
More informationArtificial Neural Networks written examination
1 (8) Institutionen för informationsteknologi Olle Gällmo Universitetsadjunkt Adress: Lägerhyddsvägen 2 Box 337 751 05 Uppsala Artificial Neural Networks written examination Monday, May 15, 2006 9 00-14
More informationINPE São José dos Campos
INPE-5479 PRE/1778 MONLINEAR ASPECTS OF DATA INTEGRATION FOR LAND COVER CLASSIFICATION IN A NEDRAL NETWORK ENVIRONNENT Maria Suelena S. Barros Valter Rodrigues INPE São José dos Campos 1993 SECRETARIA
More informationDegeneracy results in canalisation of language structure: A computational model of word learning
Degeneracy results in canalisation of language structure: A computational model of word learning Padraic Monaghan (p.monaghan@lancaster.ac.uk) Department of Psychology, Lancaster University Lancaster LA1
More informationAn Empirical and Computational Test of Linguistic Relativity
An Empirical and Computational Test of Linguistic Relativity Kathleen M. Eberhard* (eberhard.1@nd.edu) Matthias Scheutz** (mscheutz@cse.nd.edu) Michael Heilman** (mheilman@nd.edu) *Department of Psychology,
More informationAxiom 2013 Team Description Paper
Axiom 2013 Team Description Paper Mohammad Ghazanfari, S Omid Shirkhorshidi, Farbod Samsamipour, Hossein Rahmatizadeh Zagheli, Mohammad Mahdavi, Payam Mohajeri, S Abbas Alamolhoda Robotics Scientific Association
More informationSARDNET: A Self-Organizing Feature Map for Sequences
SARDNET: A Self-Organizing Feature Map for Sequences Daniel L. James and Risto Miikkulainen Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 dljames,risto~cs.utexas.edu
More informationThe Good Judgment Project: A large scale test of different methods of combining expert predictions
The Good Judgment Project: A large scale test of different methods of combining expert predictions Lyle Ungar, Barb Mellors, Jon Baron, Phil Tetlock, Jaime Ramos, Sam Swift The University of Pennsylvania
More informationKnowledge-Based - Systems
Knowledge-Based - Systems ; Rajendra Arvind Akerkar Chairman, Technomathematics Research Foundation and Senior Researcher, Western Norway Research institute Priti Srinivas Sajja Sardar Patel University
More informationAccelerated Learning Course Outline
Accelerated Learning Course Outline Course Description The purpose of this course is to make the advances in the field of brain research more accessible to educators. The techniques and strategies of Accelerated
More informationUsing focal point learning to improve human machine tacit coordination
DOI 10.1007/s10458-010-9126-5 Using focal point learning to improve human machine tacit coordination InonZuckerman SaritKraus Jeffrey S. Rosenschein The Author(s) 2010 Abstract We consider an automated
More informationEvolutive Neural Net Fuzzy Filtering: Basic Description
Journal of Intelligent Learning Systems and Applications, 2010, 2: 12-18 doi:10.4236/jilsa.2010.21002 Published Online February 2010 (http://www.scirp.org/journal/jilsa) Evolutive Neural Net Fuzzy Filtering:
More informationQuickStroke: An Incremental On-line Chinese Handwriting Recognition System
QuickStroke: An Incremental On-line Chinese Handwriting Recognition System Nada P. Matić John C. Platt Λ Tony Wang y Synaptics, Inc. 2381 Bering Drive San Jose, CA 95131, USA Abstract This paper presents
More informationSeminar - Organic Computing
Seminar - Organic Computing Self-Organisation of OC-Systems Markus Franke 25.01.2006 Typeset by FoilTEX Timetable 1. Overview 2. Characteristics of SO-Systems 3. Concern with Nature 4. Design-Concepts
More informationAccelerated Learning Online. Course Outline
Accelerated Learning Online Course Outline Course Description The purpose of this course is to make the advances in the field of brain research more accessible to educators. The techniques and strategies
More informationPython Machine Learning
Python Machine Learning Unlock deeper insights into machine learning with this vital guide to cuttingedge predictive analytics Sebastian Raschka [ PUBLISHING 1 open source I community experience distilled
More informationCOMPUTER-ASSISTED INDEPENDENT STUDY IN MULTIVARIATE CALCULUS
COMPUTER-ASSISTED INDEPENDENT STUDY IN MULTIVARIATE CALCULUS L. Descalço 1, Paula Carvalho 1, J.P. Cruz 1, Paula Oliveira 1, Dina Seabra 2 1 Departamento de Matemática, Universidade de Aveiro (PORTUGAL)
More informationOCR for Arabic using SIFT Descriptors With Online Failure Prediction
OCR for Arabic using SIFT Descriptors With Online Failure Prediction Andrey Stolyarenko, Nachum Dershowitz The Blavatnik School of Computer Science Tel Aviv University Tel Aviv, Israel Email: stloyare@tau.ac.il,
More informationUsing the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees Cognitive Skills in Algebra on the SAT
The Journal of Technology, Learning, and Assessment Volume 6, Number 6 February 2008 Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees Cognitive Skills in Algebra on the
More informationOn the Combined Behavior of Autonomous Resource Management Agents
On the Combined Behavior of Autonomous Resource Management Agents Siri Fagernes 1 and Alva L. Couch 2 1 Faculty of Engineering Oslo University College Oslo, Norway siri.fagernes@iu.hio.no 2 Computer Science
More informationUnsupervised Learning of Word Semantic Embedding using the Deep Structured Semantic Model
Unsupervised Learning of Word Semantic Embedding using the Deep Structured Semantic Model Xinying Song, Xiaodong He, Jianfeng Gao, Li Deng Microsoft Research, One Microsoft Way, Redmond, WA 98052, U.S.A.
More informationSystem Implementation for SemEval-2017 Task 4 Subtask A Based on Interpolated Deep Neural Networks
System Implementation for SemEval-2017 Task 4 Subtask A Based on Interpolated Deep Neural Networks 1 Tzu-Hsuan Yang, 2 Tzu-Hsuan Tseng, and 3 Chia-Ping Chen Department of Computer Science and Engineering
More informationarxiv: v1 [math.at] 10 Jan 2016
THE ALGEBRAIC ATIYAH-HIRZEBRUCH SPECTRAL SEQUENCE OF REAL PROJECTIVE SPECTRA arxiv:1601.02185v1 [math.at] 10 Jan 2016 GUOZHEN WANG AND ZHOULI XU Abstract. In this note, we use Curtis s algorithm and the
More informationUsing computational modeling in language acquisition research
Chapter 8 Using computational modeling in language acquisition research Lisa Pearl 1. Introduction Language acquisition research is often concerned with questions of what, when, and how what children know,
More informationSoftprop: Softmax Neural Network Backpropagation Learning
Softprop: Softmax Neural Networ Bacpropagation Learning Michael Rimer Computer Science Department Brigham Young University Provo, UT 84602, USA E-mail: mrimer@axon.cs.byu.edu Tony Martinez Computer Science
More informationProbabilistic Latent Semantic Analysis
Probabilistic Latent Semantic Analysis Thomas Hofmann Presentation by Ioannis Pavlopoulos & Andreas Damianou for the course of Data Mining & Exploration 1 Outline Latent Semantic Analysis o Need o Overview
More informationIntroduction to Simulation
Introduction to Simulation Spring 2010 Dr. Louis Luangkesorn University of Pittsburgh January 19, 2010 Dr. Louis Luangkesorn ( University of Pittsburgh ) Introduction to Simulation January 19, 2010 1 /
More informationWE GAVE A LAWYER BASIC MATH SKILLS, AND YOU WON T BELIEVE WHAT HAPPENED NEXT
WE GAVE A LAWYER BASIC MATH SKILLS, AND YOU WON T BELIEVE WHAT HAPPENED NEXT PRACTICAL APPLICATIONS OF RANDOM SAMPLING IN ediscovery By Matthew Verga, J.D. INTRODUCTION Anyone who spends ample time working
More informationNeural pattern formation via a competitive Hebbian mechanism
:" ' ',i)' 1" ELSEVIER Behavioural Brain Research 66 (1995) 161-167 BEHAVIOURAL BRAIN RESEARCH Neural pattern formation via a competitive Hebbian mechanism K. Obermayer a'*, T. Sejnowski a, G.G. Blasdel
More informationTwitter Sentiment Classification on Sanders Data using Hybrid Approach
IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 17, Issue 4, Ver. I (July Aug. 2015), PP 118-123 www.iosrjournals.org Twitter Sentiment Classification on Sanders
More informationLecture 1: Machine Learning Basics
1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3
More informationKnowledge Transfer in Deep Convolutional Neural Nets
Knowledge Transfer in Deep Convolutional Neural Nets Steven Gutstein, Olac Fuentes and Eric Freudenthal Computer Science Department University of Texas at El Paso El Paso, Texas, 79968, U.S.A. Abstract
More informationarxiv: v1 [cs.lg] 3 May 2013
Feature Selection Based on Term Frequency and T-Test for Text Categorization Deqing Wang dqwang@nlsde.buaa.edu.cn Hui Zhang hzhang@nlsde.buaa.edu.cn Rui Liu, Weifeng Lv {liurui,lwf}@nlsde.buaa.edu.cn arxiv:1305.0638v1
More informationFOR TEACHERS ONLY. The University of the State of New York REGENTS HIGH SCHOOL EXAMINATION PHYSICAL SETTING/PHYSICS
PS P FOR TEACHERS ONLY The University of the State of New York REGENTS HIGH SCHOOL EXAMINATION PHYSICAL SETTING/PHYSICS Thursday, June 21, 2007 9:15 a.m. to 12:15 p.m., only SCORING KEY AND RATING GUIDE
More informationAssignment 1: Predicting Amazon Review Ratings
Assignment 1: Predicting Amazon Review Ratings 1 Dataset Analysis Richard Park r2park@acsmail.ucsd.edu February 23, 2015 The dataset selected for this assignment comes from the set of Amazon reviews for
More informationCorpus Linguistics (L615)
(L615) Basics of Markus Dickinson Department of, Indiana University Spring 2013 1 / 23 : the extent to which a sample includes the full range of variability in a population distinguishes corpora from archives
More informationModule 12. Machine Learning. Version 2 CSE IIT, Kharagpur
Module 12 Machine Learning 12.1 Instructional Objective The students should understand the concept of learning systems Students should learn about different aspects of a learning system Students should
More informationSpeaker Identification by Comparison of Smart Methods. Abstract
Journal of mathematics and computer science 10 (2014), 61-71 Speaker Identification by Comparison of Smart Methods Ali Mahdavi Meimand Amin Asadi Majid Mohamadi Department of Electrical Department of Computer
More informationReinforcement Learning by Comparing Immediate Reward
Reinforcement Learning by Comparing Immediate Reward Punit Pandey DeepshikhaPandey Dr. Shishir Kumar Abstract This paper introduces an approach to Reinforcement Learning Algorithm by comparing their immediate
More informationDeveloping True/False Test Sheet Generating System with Diagnosing Basic Cognitive Ability
Developing True/False Test Sheet Generating System with Diagnosing Basic Cognitive Ability Shih-Bin Chen Dept. of Information and Computer Engineering, Chung-Yuan Christian University Chung-Li, Taiwan
More informationA Reinforcement Learning Variant for Control Scheduling
A Reinforcement Learning Variant for Control Scheduling Aloke Guha Honeywell Sensor and System Development Center 3660 Technology Drive Minneapolis MN 55417 Abstract We present an algorithm based on reinforcement
More informationPREDICTING SPEECH RECOGNITION CONFIDENCE USING DEEP LEARNING WITH WORD IDENTITY AND SCORE FEATURES
PREDICTING SPEECH RECOGNITION CONFIDENCE USING DEEP LEARNING WITH WORD IDENTITY AND SCORE FEATURES Po-Sen Huang, Kshitiz Kumar, Chaojun Liu, Yifan Gong, Li Deng Department of Electrical and Computer Engineering,
More informationCalibration of Confidence Measures in Speech Recognition
Submitted to IEEE Trans on Audio, Speech, and Language, July 2010 1 Calibration of Confidence Measures in Speech Recognition Dong Yu, Senior Member, IEEE, Jinyu Li, Member, IEEE, Li Deng, Fellow, IEEE
More informationA New Perspective on Combining GMM and DNN Frameworks for Speaker Adaptation
A New Perspective on Combining GMM and DNN Frameworks for Speaker Adaptation SLSP-2016 October 11-12 Natalia Tomashenko 1,2,3 natalia.tomashenko@univ-lemans.fr Yuri Khokhlov 3 khokhlov@speechpro.com Yannick
More informationStatewide Framework Document for:
Statewide Framework Document for: 270301 Standards may be added to this document prior to submission, but may not be removed from the framework to meet state credit equivalency requirements. Performance
More informationAn Online Handwriting Recognition System For Turkish
An Online Handwriting Recognition System For Turkish Esra Vural, Hakan Erdogan, Kemal Oflazer, Berrin Yanikoglu Sabanci University, Tuzla, Istanbul, Turkey 34956 ABSTRACT Despite recent developments in
More informationMandarin Lexical Tone Recognition: The Gating Paradigm
Kansas Working Papers in Linguistics, Vol. 0 (008), p. 8 Abstract Mandarin Lexical Tone Recognition: The Gating Paradigm Yuwen Lai and Jie Zhang University of Kansas Research on spoken word recognition
More informationSyntactic systematicity in sentence processing with a recurrent self-organizing network
Syntactic systematicity in sentence processing with a recurrent self-organizing network Igor Farkaš,1 Department of Applied Informatics, Comenius University Mlynská dolina, 842 48 Bratislava, Slovak Republic
More informationWord learning as Bayesian inference
Word learning as Bayesian inference Joshua B. Tenenbaum Department of Psychology Stanford University jbt@psych.stanford.edu Fei Xu Department of Psychology Northeastern University fxu@neu.edu Abstract
More informationGenerative models and adversarial training
Day 4 Lecture 1 Generative models and adversarial training Kevin McGuinness kevin.mcguinness@dcu.ie Research Fellow Insight Centre for Data Analytics Dublin City University What is a generative model?
More informationSoftware Maintenance
1 What is Software Maintenance? Software Maintenance is a very broad activity that includes error corrections, enhancements of capabilities, deletion of obsolete capabilities, and optimization. 2 Categories
More informationTest Effort Estimation Using Neural Network
J. Software Engineering & Applications, 2010, 3: 331-340 doi:10.4236/jsea.2010.34038 Published Online April 2010 (http://www.scirp.org/journal/jsea) 331 Chintala Abhishek*, Veginati Pavan Kumar, Harish
More informationClass-Discriminative Weighted Distortion Measure for VQ-Based Speaker Identification
Class-Discriminative Weighted Distortion Measure for VQ-Based Speaker Identification Tomi Kinnunen and Ismo Kärkkäinen University of Joensuu, Department of Computer Science, P.O. Box 111, 80101 JOENSUU,
More informationMaths Games Resource Kit - Sample Teaching Problem Solving
Teaching Problem Solving This sample is an extract from the first 2015 contest resource kit. The full kit contains additional example questions and solution methods. Rationale and Syllabus Outcomes Learning
More informationCS Machine Learning
CS 478 - Machine Learning Projects Data Representation Basic testing and evaluation schemes CS 478 Data and Testing 1 Programming Issues l Program in any platform you want l Realize that you will be doing
More informationMachine Learning and Data Mining. Ensembles of Learners. Prof. Alexander Ihler
Machine Learning and Data Mining Ensembles of Learners Prof. Alexander Ihler Ensemble methods Why learn one classifier when you can learn many? Ensemble: combine many predictors (Weighted) combina
More informationStrategy Study on Primary School English Game Teaching
6th International Conference on Electronic, Mechanical, Information and Management (EMIM 2016) Strategy Study on Primary School English Game Teaching Feng He Primary Education College, Linyi University
More informationModeling function word errors in DNN-HMM based LVCSR systems
Modeling function word errors in DNN-HMM based LVCSR systems Melvin Jose Johnson Premkumar, Ankur Bapna and Sree Avinash Parchuri Department of Computer Science Department of Electrical Engineering Stanford
More informationCooperative Game Theoretic Models for Decision-Making in Contexts of Library Cooperation 1
Cooperative Game Theoretic Models for Decision-Making in Contexts of Library Cooperation 1 Robert M. Hayes Abstract This article starts, in Section 1, with a brief summary of Cooperative Economic Game
More informationMaking welding simulators effective
Making welding simulators effective Introduction Simulation based training had its inception back in the 1920s. The aviation field adopted this innovation in education when confronted with an increased
More informationAbstractions and the Brain
Abstractions and the Brain Brian D. Josephson Department of Physics, University of Cambridge Cavendish Lab. Madingley Road Cambridge, UK. CB3 OHE bdj10@cam.ac.uk http://www.tcm.phy.cam.ac.uk/~bdj10 ABSTRACT
More informationUniversity of Victoria School of Exercise Science, Physical and Health Education EPHE 245 MOTOR LEARNING. Calendar Description Units: 1.
University of Victoria School of Exercise Science, Physical and Health Education EPHE 245 MOTOR LEARNING Calendar Description Units: 1.5 Hours: 3-2 Neural and cognitive processes underlying human skilled
More informationAn empirical study of learning speed in backpropagation
Carnegie Mellon University Research Showcase @ CMU Computer Science Department School of Computer Science 1988 An empirical study of learning speed in backpropagation networks Scott E. Fahlman Carnegie
More informationAustralian Journal of Basic and Applied Sciences
AENSI Journals Australian Journal of Basic and Applied Sciences ISSN:1991-8178 Journal home page: www.ajbasweb.com Feature Selection Technique Using Principal Component Analysis For Improving Fuzzy C-Mean
More informationhave to be modeled) or isolated words. Output of the system is a grapheme-tophoneme conversion system which takes as its input the spelling of words,
A Language-Independent, Data-Oriented Architecture for Grapheme-to-Phoneme Conversion Walter Daelemans and Antal van den Bosch Proceedings ESCA-IEEE speech synthesis conference, New York, September 1994
More information***** Article in press in Neural Networks ***** BOTTOM-UP LEARNING OF EXPLICIT KNOWLEDGE USING A BAYESIAN ALGORITHM AND A NEW HEBBIAN LEARNING RULE
Bottom-up learning of explicit knowledge 1 ***** Article in press in Neural Networks ***** BOTTOM-UP LEARNING OF EXPLICIT KNOWLEDGE USING A BAYESIAN ALGORITHM AND A NEW HEBBIAN LEARNING RULE Sébastien
More informationMalicious User Suppression for Cooperative Spectrum Sensing in Cognitive Radio Networks using Dixon s Outlier Detection Method
Malicious User Suppression for Cooperative Spectrum Sensing in Cognitive Radio Networks using Dixon s Outlier Detection Method Sanket S. Kalamkar and Adrish Banerjee Department of Electrical Engineering
More informationPhysics 270: Experimental Physics
2017 edition Lab Manual Physics 270 3 Physics 270: Experimental Physics Lecture: Lab: Instructor: Office: Email: Tuesdays, 2 3:50 PM Thursdays, 2 4:50 PM Dr. Uttam Manna 313C Moulton Hall umanna@ilstu.edu
More informationIntroduction to Causal Inference. Problem Set 1. Required Problems
Introduction to Causal Inference Problem Set 1 Professor: Teppei Yamamoto Due Friday, July 15 (at beginning of class) Only the required problems are due on the above date. The optional problems will not
More informationLearning Methods for Fuzzy Systems
Learning Methods for Fuzzy Systems Rudolf Kruse and Andreas Nürnberger Department of Computer Science, University of Magdeburg Universitätsplatz, D-396 Magdeburg, Germany Phone : +49.39.67.876, Fax : +49.39.67.8
More informationAnalysis of Hybrid Soft and Hard Computing Techniques for Forex Monitoring Systems
Analysis of Hybrid Soft and Hard Computing Techniques for Forex Monitoring Systems Ajith Abraham School of Business Systems, Monash University, Clayton, Victoria 3800, Australia. Email: ajith.abraham@ieee.org
More informationA SURVEY OF FUZZY COGNITIVE MAP LEARNING METHODS
A SURVEY OF FUZZY COGNITIVE MAP LEARNING METHODS Wociech Stach, Lukasz Kurgan, and Witold Pedrycz Department of Electrical and Computer Engineering University of Alberta Edmonton, Alberta T6G 2V4, Canada
More informationData Integration through Clustering and Finding Statistical Relations - Validation of Approach
Data Integration through Clustering and Finding Statistical Relations - Validation of Approach Marek Jaszuk, Teresa Mroczek, and Barbara Fryc University of Information Technology and Management, ul. Sucharskiego
More informationMining Topic-level Opinion Influence in Microblog
Mining Topic-level Opinion Influence in Microblog Daifeng Li Dept. of Computer Science and Technology Tsinghua University ldf3824@yahoo.com.cn Jie Tang Dept. of Computer Science and Technology Tsinghua
More informationA Bootstrapping Model of Frequency and Context Effects in Word Learning
Cognitive Science 41 (2017) 590 622 Copyright 2016 Cognitive Science Society, Inc. All rights reserved. ISSN: 0364-0213 print / 1551-6709 online DOI: 10.1111/cogs.12353 A Bootstrapping Model of Frequency
More informationEffect of Treadmill Training Protocols on Locomotion Recovery in Spinalized Rats
Short Communication Effect of Treadmill Training Protocols on Locomotion Recovery in Spinalized Rats Abstract Both treadmill training and epidural stimulation can help to reactivate the central pattern
More informationGetting Started with Deliberate Practice
Getting Started with Deliberate Practice Most of the implementation guides so far in Learning on Steroids have focused on conceptual skills. Things like being able to form mental images, remembering facts
More informationWord Segmentation of Off-line Handwritten Documents
Word Segmentation of Off-line Handwritten Documents Chen Huang and Sargur N. Srihari {chuang5, srihari}@cedar.buffalo.edu Center of Excellence for Document Analysis and Recognition (CEDAR), Department
More informationCAAP. Content Analysis Report. Sample College. Institution Code: 9011 Institution Type: 4-Year Subgroup: none Test Date: Spring 2011
CAAP Content Analysis Report Institution Code: 911 Institution Type: 4-Year Normative Group: 4-year Colleges Introduction This report provides information intended to help postsecondary institutions better
More information1 st Quarter (September, October, November) August/September Strand Topic Standard Notes Reading for Literature
1 st Grade Curriculum Map Common Core Standards Language Arts 2013 2014 1 st Quarter (September, October, November) August/September Strand Topic Standard Notes Reading for Literature Key Ideas and Details
More informationModeling function word errors in DNN-HMM based LVCSR systems
Modeling function word errors in DNN-HMM based LVCSR systems Melvin Jose Johnson Premkumar, Ankur Bapna and Sree Avinash Parchuri Department of Computer Science Department of Electrical Engineering Stanford
More informationPh.D. in Behavior Analysis Ph.d. i atferdsanalyse
Program Description Ph.D. in Behavior Analysis Ph.d. i atferdsanalyse 180 ECTS credits Approval Approved by the Norwegian Agency for Quality Assurance in Education (NOKUT) on the 23rd April 2010 Approved
More informationComputerized Adaptive Psychological Testing A Personalisation Perspective
Psychology and the internet: An European Perspective Computerized Adaptive Psychological Testing A Personalisation Perspective Mykola Pechenizkiy mpechen@cc.jyu.fi Introduction Mixed Model of IRT and ES
More informationLearning to Schedule Straight-Line Code
Learning to Schedule Straight-Line Code Eliot Moss, Paul Utgoff, John Cavazos Doina Precup, Darko Stefanović Dept. of Comp. Sci., Univ. of Mass. Amherst, MA 01003 Carla Brodley, David Scheeff Sch. of Elec.
More informationThis scope and sequence assumes 160 days for instruction, divided among 15 units.
In previous grades, students learned strategies for multiplication and division, developed understanding of structure of the place value system, and applied understanding of fractions to addition and subtraction
More informationLearning Structural Correspondences Across Different Linguistic Domains with Synchronous Neural Language Models
Learning Structural Correspondences Across Different Linguistic Domains with Synchronous Neural Language Models Stephan Gouws and GJ van Rooyen MIH Medialab, Stellenbosch University SOUTH AFRICA {stephan,gvrooyen}@ml.sun.ac.za
More informationEntrepreneurial Discovery and the Demmert/Klein Experiment: Additional Evidence from Germany
Entrepreneurial Discovery and the Demmert/Klein Experiment: Additional Evidence from Germany Jana Kitzmann and Dirk Schiereck, Endowed Chair for Banking and Finance, EUROPEAN BUSINESS SCHOOL, International
More informationProduct Feature-based Ratings foropinionsummarization of E-Commerce Feedback Comments
Product Feature-based Ratings foropinionsummarization of E-Commerce Feedback Comments Vijayshri Ramkrishna Ingale PG Student, Department of Computer Engineering JSPM s Imperial College of Engineering &
More informationThe Strong Minimalist Thesis and Bounded Optimality
The Strong Minimalist Thesis and Bounded Optimality DRAFT-IN-PROGRESS; SEND COMMENTS TO RICKL@UMICH.EDU Richard L. Lewis Department of Psychology University of Michigan 27 March 2010 1 Purpose of this
More informationFUZZY EXPERT. Dr. Kasim M. Al-Aubidy. Philadelphia University. Computer Eng. Dept February 2002 University of Damascus-Syria
FUZZY EXPERT SYSTEMS 16-18 18 February 2002 University of Damascus-Syria Dr. Kasim M. Al-Aubidy Computer Eng. Dept. Philadelphia University What is Expert Systems? ES are computer programs that emulate
More informationUnderstanding and Supporting Dyslexia Godstone Village School. January 2017
Understanding and Supporting Dyslexia Godstone Village School January 2017 By then end of the session I will: Have a greater understanding of Dyslexia and the ways in which children can be affected by
More informationPerformance Analysis of Optimized Content Extraction for Cyrillic Mongolian Learning Text Materials in the Database
Journal of Computer and Communications, 2016, 4, 79-89 Published Online August 2016 in SciRes. http://www.scirp.org/journal/jcc http://dx.doi.org/10.4236/jcc.2016.410009 Performance Analysis of Optimized
More informationEvolving Spiking Networks with Variable Resistive Memories
Evolving Spiking Networks with Variable Resistive Memories Gerard Howard david4.howard@uwe.ac.uk Department of Computer Science, University of the West of England, Bristol, BS16 1QY, UK Larry Bull larry.bull@uwe.ac.uk
More information