PERSPECTIVES IN NEURAL COMPUTING

PERSPECTIVES IN NEURAL COMPUTING

J. G. Taylor and C. L. T. Mannion (Eds.) THEORY AND APPLICATIONS OF NEURAL NETWORKS Proceedings of the First British Neural Network Society Meeting, London -i-- Springer-Verlag London Berlin Heidelberg New York Paris Tokyo Hong Kong Barcelona Budapest

J. G. Taylor, SA, SSe, MA, PhD, FlnstP Department of Mathematics, King's College, Strand, London WC2R 2LS, UK C. L. T. Mannion, SSe, PhD, MlnstP Department of Electrical Engineering, University of Surrey, Guildford, Surrey, GU2 5XH, UK Series Editors J. G. Taylor, SA, SSe, MA, PhD, FlnstP Department of Mathematics, King's College, Strand, London WC2R 2LS, UK C. L. T. Mannion, SSe, PhD, MlnstP Department of Electrical Engineering, University of Surrey, Guildford, Surrey, GU2 5XH, UK ISBN-13: 978-3-540-19650-1 e-isbn-13: 978-1-4471-1833-6 DOl: 10.1007/978-1-4471-1833-6 British Ubrary Cataloguing in Publication Data British Neural Network Society Meeting (1 st : 1990 : King's College, London) Theory and applications of neural networks. - (Perspectives in neural computing) I. Title II. Taylor, John, 1931- III. Mannion, C. L. T. IV. Series 006.3 Ubrary of Congress Data available Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms of licences issued by the Copyright Ucensing Agency. Enquiries conceming reproduction outside those terms should be sent to the publishers. Springer-Verlag London Umited 1992 The use of registered names, trademarks etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant laws and regulations and therefore free for general use. The publisher makes no representation, express or implied, with regard to the accuracy of the information contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that may be made. 34/3830-543210 Printed on acid-free paper

PREFACE This volume contains the papers from the first British Neural Network Society meeting held at Queen Elizabeth Hall, King's College, London on 18--20 April 1990. The meeting was sponsored by the London Mathematical Society. The papers include introductory tutorial lectures, invited, and contributed papers. The invited contributions were given by experts from the United States, Finland, Denmark, Germany and the United Kingdom. The majority of the contributed papers came from workers in the United Kingdom. The first day was devoted to tutorials. Professor Stephen Grossberg was a guest speaker on the first day giving a thorough introduction to his Adaptive Resonance Theory of neural networks. Subsequent tutorials on the first day covered dynamical systems and neural networks, realistic neural modelling, pattern recognition using neural networks, and a review of hardware for neural network simulations. The contributed papers, given on the second day, demonstrated the breadth of interests of workers in the field. They covered topics in pattern recognition, multi-layer feedforward neural networks, network dynamics, memory and learning. The ordering of the papers in this volume is as they were given at the meeting. On the final day talks were given by Professor Kohonen (on self organising maps), Professor Kurten (on the dynamics of random and structured nets) and Professor Cotterill (on modelling the visual cortex). Dr A. Mayes presented a paper on various models for amnesia. The editors have taken the opportunity to include a paper of their own which was not presented at the meeting. November 1990 J.G. Taylor C.L.T. Mannion

ACKNOWLEDGEMENTS We would like to thank all those people and organisations involved in the meeting. In particular the London Mathematical Society for their financial support, Mrs Marion Harris for her invaluable secretarial and administrative help, and Linda Schofield for assistance with the production of this volume.

CONTENTS Contributors... xi Invited Lecture Self-Organizing Cortical Networks for Distributed Hypothesis Testing and Recognition Learning G.A. Carpenter and S. Grossberg... 3 Tutorials Dynamical Systems and Artificial Neural Networks J.G. Tay/or... ;... 31 Neural Modelling C.L.T. Mannion............... 77 Self-Organising Neural Maps and their Applications N.M. Allinson... 101 Contributed Papers The Application of Topological Mapping in the Study of Human Cerebral Tumours G. Sieben, L. Vercauteren, M. Praet, G. Otte, L. Boullart, L. Calliauw, L. Roels... 121 Reliable Memory from Unreliable Components P. Whittle... 125 Possible Strategies for Using Sleep to Improve Episodic Memory in the Face of Overlap A.R. Gardner-Medwin... 129 Curvature-Driven Smoothing in Backpropagation Neural Networks C.M. Bishop........ 139 A Spontaneously Growing Network for Unsupervised Leaming P. Fletcher...... 149 Virtual Connectivity Through Structural Dissipation; Parallel Distributed Computation with Local Connectivity R.W. Taylor... 164 Temporally Processing Neural Networks for Morse Code Recognition D. Hill... 180

Dynamics of Binary Networks with Extended Time-Summation P. C. Bressloff... 198 Training Strategies for Probabilistic RAMs D. Gorse, J.G. Taylor...... 211 Computer Simulations of Recurrent Neural Nets for Temporal Recognition Problems G.S. Cooper, T.M. Child......... 218 Invited Papers Learning Vector Quantisation and the Self Organising Map T. Kohonen... 235 Nature of the Functional Loss in Amnesia: Possible Role for a Highly Structured Neural Network A. Mayes... 243 Activity Pattems in Cortical Minicolumns S.S. Christensen, R.M.J. Cotterill, C. Nielsen... 261 Dynamics and Memory in Random and Structured Neural Networks K.E. Kurten......... 274 Additional Paper (not given at conference) Coupled Excitable Cells C.L. T. Mannion, J.G. Taylor... 285

CONTRIBUTORS Allinson, N.M. Dr, Department of Electronics, University of York, Heslington, York Y01 5DD,UK Bishop, C.M. Dr, Culham Laboratory, Abingdon, Oxon, UK BoulIart, L. Dr, Ghent State University, Kon Albertlaan 29, 9000 Ghent, Belgium Bressloff, P.C. Dr, Long Range Research, GEC-Hirst Research, East Lane, Wembley, Middlesex, UK Calliauw, L. Dr, Ghent State University, Kon Albertlaan 29,9000 Ghent, Belgium Carpenter, GA Dr, Center for Adaptive Systems, University of Boston, 111 Cummington Street, Boston, MA 02215, USA Child, T.M. Dr, Information Technology Institute, University of Salford, The Crescent, Salford M5 4WT, UK Christensen, S.S. Dr, Department of Molecular Biophysics, The Technical University of Denmark, Building 307, DK-2800 Lyngby, Denmark Cooper, G.S. Dr, Information Technology Institute, University of Salford, The Crescent, Salford M5 4WT, UK Cotterill, R.M.J. Professor, Department of Molecular Biophysics, The Technical University of Denmark, Building 307, DK-2800 Lyngby, Denmark Fletcher, P. Dr, Department of Computer Science, University of Keele, Keele, Staffordshire ST5 5BG, UK Gardner-Medwin, AR. Dr, Department of Physiology, University College London, Gower Street, London WC1 E 6BT, UK, and The Physiological Laboratory, Cambridge CB2 3EG, UK

Gorse, D. Dr, Department of Computer Science, University College London, Gower Street, London WC1 E 6BT, UK Grossberg, S. Professor, Center for Adaptive Systems, University of Boston, 111 Cummington Street, Boston, MA 02215, USA Hill, D. Dr, Department of Mathematics and Statistics, University of Birmingham, Birmingham B15 2TI, UK Kohonen, T. Professor, Computer and Information Science Laboratory, University of Helsinki, Rakentajapaukio 2C, SF 02150 Espoo, Finland Kurten, K.E. Professor, Department of Theoretical Physics, University of Koln, Zulpichen Strasse 77, D-5000 Koln, Germany Mannion, C.L.T. Dr, Department of Electronic and Electrical Engineering, University of Surrey, Guildford, Surrey GU2 5XH, UK Mayes, A. Dr, Department of Psychology, University of Manchester, 10 Adria Road, Didsbury, Manchester M20 OSE, UK Nielsen, C. Dr, Department of Molecular Biophysics, The Technical University of Denmark, Building 307, DK-2800 Lyngby, Denmark Otte, G. Dr, Department of Neurology, Ghent State University, Bylokehof 24,9000 Ghent, Belgium Praet, M. Dr, Ghent State University, Kon Albertlaan 29, 9000 Ghent, Belgium Roels, L. Dr, Ghent State University, Kon Albertlaan 29,9000 Ghent, Belgium Sieben, G. Dr, Department of Pathology, Ghent State University, Kon Albertlaan 29, 9000 Ghent, Belgium Taylor, J.G. Professor, Department of Mathematics, King"s College London, Strand, London WC2R 2LS, UK Taylor, R.W. Dr, Department of Electronics, University of York, Heslington, York Y01 5DD, UK

Vercauteren, L. Dr, Automatic Control Laboratory, Ghent State University, Grote Steenweg Noord, 9710 Ghent-ZWijnaarde, Belgium Whittle, P. Professor, Statistical Laboratory, University of Cambridge, 16 Mill Lane, Cambridge CB2 2SB, UK