A Neuro-Fuzzy Synergism to Intelligent Systems. For book and bookstore information.

Transcription

1 Neural Fuzzy Systems A Neuro-Fuzzy Synergism to Intelligent Systems Chin-Teng Lin Department of Control Engineering National Chiao-Tung University Hsinchu, Taiwan C.S.George Lee School of Electrical and Computer Engineering Purdue University West Lafayette, Indiana For book and bookstore information Prentice Hall PTR, Upper Saddle River, NJ 07458

2 PREFACE xiii 1 INTRODUCTION Motivation Fuzzy Systems Neural Networks Fuzzy Neural Integrated Systems Organization of the Book References 9 PART I FUZZY SYSTEMS 2 BASICS OF FUZZY SETS Fuzzy Sets and Operation on Fuzzy Sets Extensions of Fuzzy Set Concepts Other Kinds of Fuzzy Sets, 22 " Further Operations on Fuzzy Sets, Extension Principle and Its Applications Operations oftype-2 Fuzzy Sets, Consistency Degree of Two Fuzzy Sets, Concluding Remarks Problems 34

3 3 FUZZY RELATIONS Basics of Fuzzy Relations Operations on Fuzzy Relations Various Types of Binary Fuzzy Relations Similarity Relations, Resemblance Relations, Fuzzy Partial Ordering, Fuzzy Relation Equations Concluding Remarks Problems 60 4 FUZZY MEASURES Fuzzy Measures Belief and Plausibility Measures, Probability Measures, Possibility arid Necessity Measures, Fuzzy Integrals Measures of Fuzziness Concluding Remarks Problems 86 5 POSSIBILITY THEORY AND FUZZY ARITHMETIC Basics of Possibility Theory Fuzzy Arithmetic Interval Representation of Uncertain Values, Operations and Properties of Fuzzy Numbers, Ordering of Fuzzy Numbers, Concluding Remarks Problems FUZZY LOGIC AND APPROXIMATE REASONING Linguistic Variables Fuzzy Logic Truth Values and Truth Tables in Fuzzy Logic, Fuzzy Propositions, Approximate Reasoning Categorical Reasoning, Qualitative Reasoning, Syllogistic Reasoning, Dispositional Reasoning, Fuzzy Expert Systems MILORD, Z-II, Concluding Remarks Problems 140 vi

4 7 FUZZY LOGIC CONTROL SYSTEMS Basic Structure and Operation of Fuzzy Logic Control Systems Input-Output Spaces, Fuzzifier, Fuzzy Rule Base, Inference Engine, Defuzzifier, Design Methodology of Fuzzy Control Systems Stability Analysis of Fuzzy Control Systems Applications of Fuzzy Controllers Concluding Remarks Problems, APPLICATIONS OF FUZZY THEORY Fuzzy Pattern Recognition Classification Methods Based on Fuzzy Relations, Fuzzy Clustering, Fuzzy Mathematical Programming Fuzzy Databases Fuzzy Relational Databases, Fuzzy Object-Oriented Databases, Human-Machine Interactions Concluding Remarks Problems 203 ARTIFICIAL NEURAL NETWORKS INTRODUCTION TO ARTIFICIAL NEURAL NETWORKS Fundamental Concepts of Artificial Neural Networks Basic Models and Learning Rules of ANNs Processing Elements, Connections, Learning Rules, Distributed Representations Concluding Remarks Problems FEEDFORWARD NETWORKS AND SUPERVISED LEARNING Single-Layer Perceptron Networks Perceptron Learning Rule, Adaline, Multilayer Feedforward Networks Back Propagation, Learning Factors of Back Propagation, Time-Delay Neural Networks, 250 vii

5 10.3 Other Feedforward Networks Functional-Link Networks, Tree Neural Networks, Wavelet Neural Networks, Concluding Remarks Problems SINGLE-LAYER FEEDBACK NETWORKS AND ASSOCIATIVE MEMORIES Hopfield Networks Discrete Hopfield Networks, Continuous Hopfield Networks, Associative Memories Recurrent Autoassociative Memory Hopfield Memory, Bidirectional Associative Memory, Temporal Associative Memory, Optimization Problems Hopfield Networks for Optimization Problems, Boltzmann Machines, Concluding Remarks Problems UNSUPERVISED LEARNING NETWORKS Unsupervised Learning Rules Signal Hebbian Learning Rule, Competitive Learning Rule, Differential Hebbian Learning Rule, Differential Competitive Learning Rule, Hamming Networks Self-Organizing Feature Maps Adaptive Resonance Theory The Instar-Outstar Model Shunting Activation Equations, Adaptive Resonance Theory, Counterpropagation Networks Radial Basis Function Networks Adaptive Bidirectional Associative Memories 331 ) 12.8 Hierarchical Networks Neocognitron Concluding Remarks Problems RECURRENT NEURAL NETWORKS Feedback Backpropagation Networks Recurrent Backpropagation Networks, Partially Recurrent Networks, Fully Recurrent Networks Real-Time Recurrent Learning, 350 viii

6 Time-Dependent Recurrent Backpropagation, Second-Order Recurrent Networks, The Extended Kalman Filter, Reinforcement Learning Associative Reward-Penalty, REINFORCE Algorithms, Temporal Difference Methods, Concluding Remarks Problems GENETIC ALGORITHMS Basics of Genetic Algorithms Further Evolution of Genetic Algorithms Improved Selection Schemes, Advanced Operators, Hybrid Genetic Algorithms Applications of Genetic Algorithms Genetic Algorithms for Neural Network Parameter Learning, Genetic Algorithms for Path Planning, Genetic Algorithms for System Identification and Controls, Genetic Programming Concluding Remarks Problems STRUCTURE-ADAPTIVE NEURAL NETWORKS Simulated Evolution for Neural Network Structure Learning Genetic Algorithms for Designing Neural Networks, Evolutionary Programming for Designing Neural Networks, Pruning Neural Networks Weight Decay, Connection and Node Pruning, Growing Neural Networks 427 ; ; Input Space Partitioning, Prototype Selection, Growing and Pruning Neural Networks Activity-Based Structural Adaptation, Function Networks, Concluding Remarks Problems APPLICATIONS OF NEURAL NETWORKS NeuraLNetworks in Control Systems Dynamic Backpropagation for System Identification and Control, Cerebellar Model Articulation Controller, Neural Networks in Sensor Processing Neural Networks in Communications 468 ix

7 16.4 Neural Knowledge-Based Systems Concluding Remarks Problems 475 PART III FUZZY NEURAL INTEGRATED SYSTEMS 17 INTEGRATING FUZZY SYSTEMS AND NEURAL NETWORKS Basic Concept of Integrating Fuzzy Systems and Neural Networks General Comparisons of Fuzzy Systems and Neural Networks, Choice of Fuzzy Systems or Neural Networks, Reasons for Integrating Fuzzy Systems and Neural Networks, The Equivalence of Fuzzy Inference Systems and Neural Networks Fuzzy Inference Systems as Universal Approximators, Equivalence of Simplified Fuzzy Inference Systems and Radial Basis Function Networks, Stability Analysis of Neural Networks Using Stability Conditions of Fuzzy Systems, Concluding Remarks Problems NEURAL-NETWORK-BASED FUZZY SYSTEMS Neural Realization of Basic Fuzzy Logic Operations Neural Network-Based Fuzzy Logic Inference Fuzzy Inference Networks, Fuzzy Aggregation Networks, Neural Network Driven Fuzzy Reasoning, Neural Network-Based Fuzzy Modeling Rule-Based Neural Fuzzy Modeling, Neural Fuzzy Regression Models, Neural Fuzzy Relational Systems, Concluding Remarks Problems NEURAL FUZZY CONTROLLERS Types of Neural Fuzzy Controllers Neural Fuzzy Controllers with Hybrid Structure-Parameter Learning Fuzzy Adaptive Learning Control Network, Fuzzy Basis Function Network with Orthogonal Least Squares Learning, Parameter Learning for Neural Fuzzy Controllers Neural Fuzzy Controllers with Fuzzy Singleton Rules, Neural Fuzzy Controllers with TSK Fuzzy Rules, Fuzzy Operator Tuning, 559

8 19.4 Structure Learning for Neural Fuzzy Controllers Fuzzy Logic Rule Extraction from Numerical Training Data, Genetic Algorithms for Fuzzy Partition of Input Space, On-Line Structure Adaptive Neural Fuzzy Controllers FALCON with On-Line Supervised Structure and Parameter Learning, FALCON with ART Neural Learning, Neural Fuzzy Controllers with Reinforcement Learning FALCON with Reinforcement Learning, Generalized Approximate Reasoning-Based Intelligent Controller, Concluding Remarks Problems FUZZY LOGIC-BASED NEURAL NETWORK MODELS Fuzzy Neurons Fuzzy Neuron of Type I, Fuzzy Neuron of Type II, Fuzzy Neuron of Type III, Fuzzification of Neural Network Models Fuzzy Perceptron, Fuzzy Classification with the Back-propagation Network, Fuzzy Associative Memories, Fuzzy ART Models, Fuzzy Kohonen Clustering Network, Fuzzy RCE Neural Network, Fuzzy Cerebellar Model Articulation Controller, Neural Networks with Fuzzy Training Neural Networks with Fuzzy Teaching Input, Neural Networks with Fuzzy Parameters, Fuzzy Control for Learning Parameter Adaptation, Concluding Remarks Problems 658 ) 21 FUZZY NEURAL SYSTEMS FOR PATTERN RECOGNITION Fuzzy Neural Classification Uncertainties with Two-Class Fuzzy Neural Classification Boundaries, Multilayer Fuzzy Neural Classification Networks, Genetic Algorithms for Fuzzy Classification Using Fuzzy Rules, Fuzzy Neural Clustering Fuzzy Competitive Learning for Fuzzy Clustering, Adaptive Fuzzy Leader Clustering, Fuzzy Neural Models for Image Processing Fuzzy Self-Supervised Multilayer Network for Object Extraction, 684 "^ 21.3:2 Genetic Algorithms with Fuzzy Fitness Function for Image Enhancement, A Fuzzy Neural Networks for Speech Recognition 692 xi

9 21.5 Fuzzy-Neural Hybrid Systems for System Diagnosis Concluding Remarks Problems 702 A MATLAB FUZZY LOGIC TOOLBOX 704 A. 1 Demonstration Cases 706 A. 1.1 A Simple Fuzzy Logic Controller, 706 A. 1.2 A Neural Fuzzy System ANFIS, 707 A. 1.3 Fuzzy c-means Clustering, 708 A.2 Demonstration of Fuzzy Logic Applications 709 A. 2.1 A Fuzzy Controller for Water Bath Temperature Control, 709 A.2.2 A Neural Fuzzy Controller for Water Bath Temperature Control, 712 B MATLAB NEURAL NETWORK TOOLBOX 715 B. 1 Demonstration of Various Neural Network Models 716 B.I.I Hebbian Learning Rule, 716 B.I.2 Perceptron Learning Rule, 717 B.I.3 Adaline, 718 B.I.4 Back Propagation, 719 B.I.5 Hopfield Network, 722 B.I.6 Instar Learning Rule, 723 B.I.7 Competitive Learning Rule, 724 B.I.8 LVQ Learning Rule (Supervised Competitive Learning Rule), 726 B.1.9 Self-Organizing Feature Map, 726 B.I.10 Radial Basis Function Network, 727 B.I.11 ElmanNetwork, 729 B.2 Demonstration of Neural Network Applications Adaptive Noise Cancelation Using theadaline Network, 730 B.2.2 Nonlinear System Identification, 731 BIBLIOGRAPHY 734 INDEX 783 xii,