FUZZY ENGINEERING EXPERT SYSTEMS WITH NEURAL NETWORK APPLICATIONS

FUZZY ENGINEERING EXPERT SYSTEMS WITH NEURAL NETWORK APPLICATIONS ADEDEJI B. BADIRU Department of Industrial Engineering University of Tennessee Knoxville, TN JOHN Y. CHEUNG School of Electrical and Computer Engineering University of Oklahoma Norman, OK JOHN WILEY & SONS, INC.

This book is printed on acid-free paper. Copyright 2002 by John Wiley & Sons, New York. All rights reserved. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4744. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 605 Third Avenue, New York, NY 10158-0012, (212) 850-6011, fax (212) 850-6008, E-Mail: PERMREQ@WILEY.COM. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold with the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional person should be sought. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. For more information about Wiley products, visit our web site at www.wiley.com. Library of Congress Cataloging-in-Publication Data: Badiru, Adedeji Bodunde, 1952 Fuzzy engineering expert systems with neural network applications / Adedeji B. Badiru, John Y. Cheung. p. cm. ISBN 0-471-29331-8 1. Expert systems (Computer science) 2. Fuzzy systems. 3. Neural networks (Computer science) I. Cheung, John Y. II. Title. QA76.76.E95 B24 2002 006.3 3 dc21 2002003706 Printed in the United States of America. 10987654321

To Our Spouses, Iswat Amori Badiru, and Rose Kwan-Fun Cheung

CONTENTS Preface Acknowledgments xv xvii 1 Artificial Intelligence 1 1.1 Origin of Artificial Intelligence 1 1.2 Human Intellligence versus Machine Intelligence 3 1.3 The First AI Conference 5 1.4 Evolution of Smart Programs 6 1.5 Branches of Artificial Intelligence 8 1.6 Neural Networks 9 1.7 Emergence of Expert Systems 10 1.7.1 Embedded Expert Systems 12 2 Fundamentals of Expert Systems 13 2.1 Expert Systems Process 13 2.2 Expert Systems Characteristics 14 2.2.1 Domain Specificity 15 2.2.2 Special Programming Languages 16 2.3 Expert Systems Structure 16 2.3.1 The Need for Expert Systems 19 2.3.2 Benefits of Expert Systems 19 2.3.3 Transition from Data Processing to Knowledge Processing 20 2.4 Heuristic Reasoning 21 2.4.1 Search Control Methods 22 2.4.2 Forward Chaining 22 2.4.3 Backward Chaining 22 2.5 User Interface 22 2.5.1 Natural Language 23 2.5.2 Explanations Facility in Expert Systems 23 2.5.3 Data Uncertainties 23 vii

viii CONTENTS 2.5.4 Application Roadmap 23 2.5.5 Symbolic Processing 24 3 Problem Analysis 28 3.1 Problem Identification 28 3.2 Problem Analysis 29 3.2.1 Scope of the Problem 30 3.2.2 Symbolic Nature of the Problem 30 3.2.3 Solution Time 30 3.2.4 Frequency of Problem Occurrence 30 3.2.5 Optimization versus Satisficing 31 3.2.6 Data and Knowledge Availability 31 3.3 Data Requirement Analysis 34 3.4 Expert System Justification 36 3.4.1 Problem-Selection Guidelines 47 4 Knowledge Engineering 49 4.1 Knowledge-Acquisition Phases 49 4.1.1 The Knowledge Engineer 49 4.1.2 Knowledge Characteristics 50 4.1.3 Choosing the Expert 51 4.1.4 Knowledge Extraction versus Knowledge Acquisition 53 4.2 Methods of Extracting Knowledge from Experts 54 4.2.1 Interviews 54 4.2.2 Open-Ended Interviews 55 4.2.3 Advantages and Disadvantages of Interviews 56 4.2.4 Task Performance and Protocols 56 4.2.5 Analyzing the Expert s Thought Process 57 4.2.6 Constrained Task 57 4.2.7 Tough Case Method 58 4.2.8 Questionnaires and Surveys 58 4.2.9 Documentation and Analysis of Acquired Knowledge 58 4.2.10 Expert s Block 59 4.3 Knowledge-Acquisition Meetings 59 4.4 Group Knowledge Acquisition 61 4.4.1 Brainstorming 62 4.4.2 Delphi Method 63 4.4.3 Nominal Group Technique 64

CONTENTS ix 4.5 Knowledge-Acquisition Software 66 4.5.1 Knowledge Elicitation Tools 66 4.5.2 Induction-by-Example Tools 67 4.6 Characteristics of Knowledge 68 4.6.1 Types of Knowledge 68 4.7 Knowledge-Representation Models 69 4.7.1 Semantic Networks 70 4.7.2 Frames 71 4.7.3 Scripts 73 4.7.4 Rules 74 4.7.5 Predicate Logic 76 4.7.6 O-A-V Triplets 77 4.7.7 Hybrids 78 4.7.8 Specialized Representation Techniques 78 4.8 Concept of Knowledge Sets 87 4.8.1 Properties of Knowledge Sets 88 4.9 Reasoning Models 97 5 Probabilistic and Fuzzy Reasoning 103 5.1 Human Reasoning and Probability 103 5.2 Bayesian Approach to Handling Uncertainty 104 5.2.1 Logical Relations 106 5.2.2 Plausible Relations 106 5.2.3 Contextual Relations 106 5.3 Decision Tables and Trees 106 5.4 Dempster Shafer Theory 113 5.4.1 Support Function 116 5.4.2 Plausibility 116 5.4.3 Uncertainty of A 116 5.4.4 Belief Interval 117 5.4.5 Focal Elements 117 5.4.6 Doubt Function 117 5.5 Certainty Factors 119 5.6 Fuzzy Logic 122 5.6.1 Definition of Fuzzy Set 124 6 Fuzzy Systems 133 6.1 Crisp Logic versus Fuzzy Logic 133 6.1.1 Crisp Sets 133

x CONTENTS 6.1.2 Fuzzy Sets 136 6.1.3 Fuzzy Set Construction 140 6.1.4 Fuzzy Set Operations 141 6.1.5 -Cuts 143 6.1.6 Extension Principle 145 6.2 Fuzzy Operations 146 6.2.1 Fuzzy Complement 147 6.2.2 Fuzzy Intersection 148 6.2.3 Fuzzy Union 151 6.2.4 Duality 153 6.2.5 Fuzzy Implication 154 6.2.6 Fuzzy Aggregation 156 6.3 Fuzzy Numbers 157 6.3.1 Fuzzy Number Representation 157 6.3.2 Fuzzy Number Operation 158 6.3.3 Fuzzy Ordering 161 6.4 Fuzzy Relation 161 6.4.1 Definition of a Fuzzy Relation 161 6.4.2 Binary Relation 162 6.4.3 Operations with Relations 164 6.5 Evidence Theory 165 6.5.1 Believability and Plausibility 165 6.5.2 Uncertainty 168 6.5.3 Fuzziness 168 6.5.4 Nonspecificity 170 6.6 Fuzzy Logic 171 6.6.1 Multivalued Logic 172 6.6.2 Unconditional Fuzzy Propositions 172 6.6.3 Conditional Fuzzy Propositions 173 6.6.4 Selection of Implication Operator 176 6.6.5 Multiconditional Reasoning 177 6.7 Summary 179 7 Neural Networks 180 7.1 Introduction 180 7.1.1 Definition of a Neurode 181 7.1.2 Variations of a Neurode 182 7.2 Single Neurode 183

CONTENTS xi 7.2.1 The McCulloch Pitts Neurode 183 7.2.2 McCulloch Pitts Neurodes as Boolean Components 183 7.2.3 Single Neurode as Binary Classifier 184 7.2.4 Single-Neurode Perceptron 186 7.3 Single-Layer Feedforward Network 187 7.3.1 Multicategory SLP 187 7.3.2 Associative Memory 187 7.3.3 Correlation Matrix Memory 189 7.3.4 Pseudoinverse Memory 190 7.3.5 Widrow Hoff Approach 190 7.3.6 Least-Mean-Squares Approach 191 7.3.7 Adaptive Correlation Matrix Theory 191 7.3.8 Error-Correcting Pseudoinverse Method 192 7.4 Self-Organizing Networks 192 7.4.1 Principal Components 193 7.4.2 Clustering by Hebbian Learning 194 7.4.3 Clustering by Oja s Normalization 195 7.4.4 Competitive Learning Network 197 7.5 Multiple-Layer Feedforward Network 199 7.5.1 Multiple-Layer Perceptron 199 7.5.2 XOR Example 200 7.5.3 Back-Error Propagation 201 7.5.4 Variations in the Back-Error Propagation Algorithm 202 7.5.5 Learning Rate and Momentum 203 7.5.6 Other Back-Error Propagation Issues 205 7.5.7 Counterpropagation Network 205 7.6 Radial Basis Networks 206 7.6.1 Interpolation 207 7.6.2 Radial Basis Network 208 7.7 Single-Layer Feedback Network 211 7.7.1 Single-Layer Feedback Network 212 7.7.2 A Discrete Single-Layer Feedback Network 213 7.7.3 Bidirectional Associative Memory 215 7.7.4 Hopfield Network 216 7.8 Summary 219 7.9 References 220

xii CONTENTS 8 Neural-Fuzzy Networks 221 8.1 Technology Comparisons 221 8.2 Neurons Performing Fuzzy Operations 223 8.2.1 Neurons Emulating Fuzzy Operations 224 8.2.2 Neurons Performing Fuzzy Operations 226 8.3 Neural Network Performing Fuzzy Inference 227 8.3.1 Regular Neural Network with Crisp Input and Output 227 8.3.2 Regular Neural Network with Fuzzy Input and Output 228 8.3.3 Fuzzy Inference Network 229 8.3.4 ANFIS 230 8.3.5 Applications 231 8.4 Clustering and Classification 232 8.4.1 Classification 233 8.4.2 Multilayer Fuzzy Perceptron 235 9 Evolutionary Computing 237 9.1 Introduction 237 9.2 Binary Genetic Algorithm 239 9.2.1 Genetic Representation 241 9.2.2 Population 242 9.2.3 Fitness Check and Cost Evaluation 242 9.2.4 Mating Pool 243 9.2.5 Pairing 244 9.2.6 Mating 246 9.2.7 Mutation 248 9.2.8 The Next Generation 249 9.2.9 Performance 250 9.2.10 Enhancements 251 9.3 Continuous Genetic Algorithm 252 9.3.1 Genetic Representation 253 9.3.2 Mating 254 9.3.3 Mutation 256 9.3.4 Performance 256 9.3.5 Enhancements 257 9.4 Evolutionary Programming 259 9.4.1 Evolutionary Strategies 259

CONTENTS xiii 9.4.2 Evolutionary Programming 260 9.5 Summary 263 10 Intelligent Strategy Generation in Complex Manufacturing Environments 264 10.1 Introduction 264 10.2 Model Description 265 10.3 Evolutionary Algorithm 267 10.4 Process Simulation 269 10.5 Fuzzy Logic Evaluation 271 11 Product Demand Forecasting Using Genetic Programming 274 11.1 Introduction 274 11.2 Algorithm Description 276 11.2.1 Chromosome Structure 276 11.2.2 Fitness Evaluation 276 11.2.3 Reproduction and Generation Evolution 277 11.3 Experiments and Results 278 11.4 Conclusion 281 References 282 Index 289