Springer Texts in Statistics. Series Editors: G. Casella S. Fienberg I. Olkin

Transcription

1 Springer Texts in Statistics Series Editors: G. Casella S. Fienberg I. Olkin

2 Springer Texts in Statistics For other titles published in this series, go to

3 V.G. Kulkarni Introduction to Modeling and Analysis of Stochastic Systems Second Edition ABC

4 Prof. V.G. Kulkarni University of North Carolina Department of Statistics and Operations Research Chapel Hill, NC USA ISBN e-isbn DOI / Springer New York Dordrecht Heidelberg London Library of Congress Control Number: PCN applied for c Springer Science+Business Media, LLC 2011 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (

5 To my Sons Milind, Ashwin and Arvind

6

7 Preface to the Second Edition What Is New in the Second Edition? Students and instructors will notice significant changes in the second edition. 1. The chapters on probability have been removed. Abbreviated versions of these chapters appear as Appendices A, B, C, and D. 2. A new chapter (Chapter 3) on Poisson processes has been added. This is in response to the feedback that the first edition did not give this important class of stochastic processes the attention it deserves. 3. A new chapter (Chapter 7) on Brownian motion has been added. This is to enable instructors who would like the option of covering this important topic. The treatment of this topic is kept at a sufficiently elementary level so that the students do not need background in analysis or measure theory to understand the material. 4. The chapters on design and control of stochastic systems in the first edition have been deleted. Instead, I have added case studies in Chapters 2, 4, 5, and6. The instructor can use these to talk about the design aspect of stochastic modeling. The control aspect is entirely deleted. This has necessitated a change of title for the new edition. 5. Several typos from the first edition have been corrected. If there are new typos in the second edition, it is entirely my fault. I would appreciate it if the readers would kindly inform me about them. A current list of corrections is available on the Web at vkulkarn/ugcorrections2.pdf. Who Is This Book For? This book is meant to be used as a textbook in a junior-or senior-level undergraduate course on stochastic models. The students are expected to be undergraduate students in engineering, operations research, computer science, mathematics, statistics, business administration, public policy, or any other discipline with a mathematical core. Students are expected to be familiar with elementary matrix operations (addition, multiplication, and solving systems of linear equations, but not eigenvalues or vii

8 viii Preface to the Second Edition eigenvectors), first-year calculus (derivatives and integrals of simple functions, but not differential equations), and probability. The necessary material on probability is summarized in the appendices as a ready reference. What Is the Philosophy of This Book? As the title suggests, this book addresses three aspects of using stochastic methodology to study real systems. 1. Modeling. The first step is to understand how a real system operates and the purpose of studying it. This enables us to make assumptions to create a model that is simple yet sufficiently true to the real system that the answers provided by the model will have some credibility. In this book, this step is emphasized repeatedly by using a large number of real-life modeling examples. 2. Analysis. The second step is to do a careful analysis of the model and compute the answers. To facilitate this step, the book develops special classes of stochastic processes in Chapters 2, 3, 4, 5, and7: discrete-time Markov chains, Poisson processes, continuous-time Markov chains, renewal processes, cumulative processes, semi-markov processes, Brownian motion, etc. For each of these classes, we develop tools to compute the transient distributions, limiting distributions, cost evaluations, first-passage times, etc. These tools generally involve matrix computations and can be done easily in any matrix-oriented language (e.g., MATLAB). Chapter 6 applies these tools to queueing systems. 3. Design. In practice, a system is described by a small number of parameters, and we are interested in setting the values of these parameters so as to optimize the performance of the system. This is called designing a system. The performance of the system can be computed as a function of the system parameters using the tools developed here. Then the appropriate parameter values can be determined to minimize or maximize this function. This is illustrated by case studies in Chapters 2, 4, 5,and6. How Is This Book Intended to Be Used? Typically, the book will be used in a one-semester course on stochastic models. The students taking this course will be expected to have a background in probability. Hence, Appendices A through D should be used to review the material. Chapters 2, 3, 4, 5, and6 should be covered completely. Chapter 7 should be covered as time permits. There are many running examples in this book. Hence the instructor should try to use them in that spirit. Similarly, there are many running problems in the problem section. The instructor may wish to use a running series of problems for homework.

9 Preface to the Second Edition ix What Is So Different about This Book? This book requires a new mind-set: a numerical answer to a problem is as valid as an algebraic answer to a problem! Since computational power is now conveniently and cheaply available, the emphasis in this book is on using the computer to obtain numerical answers rather than restricting ourselves to analytically tractable examples. There are several consequences of this new mind-set: the discussion of the transient analysis of stochastic processes is no longer minimized. Indeed, transient analysis is just as easy as the limiting analysis when done on a computer. Secondly, the problems at the end of each chapter are designed to be fairly easy, but may require use of computers to do numerical experimentation. Software to Accompany the Book A software package called MAXIM is available for use with this textbook. It is a collection of over 80 programs written in MATLAB. These programs can be used directly as function files from MATLAB. The user s manual describing these programs is in a file called readme. These programs can be accessed via a graphical user interface (GUI). The GUI is designed to run on PCs with Windows software. Since the software is an evolving organism, I have decided not to include any information about it in the book for fear that it will become outdated very soon. The software and any relevant information can be downloaded from vkulkarn/maxim/ maximgui.zip. The user will need to have MATLAB installed on his or her machine in order to use the software. Vidyadhar G. Kulkarni University of North Carolina Department of Statistics and Operations Research Chapel Hill, NC USA vkulkarn@ .unc.edu

10

11 Contents Preface to the Second Edition... vii 1 Introduction What Is a Stochastic Process? Discrete-Time Stochastic Processes Continuous-Time Stochastic Processes What Do We Do with a Stochastic Process? Discrete-Time Markov Models Discrete-Time Markov Chains Examples of Markov Models Transient Distributions Occupancy Times Limiting Behavior Cost Models Expected Total Cost over a Finite Horizon Long-Run Expected Cost Per Unit Time First-Passage Times Case Study: Passport Credit Card Company Problems Poisson Processes Exponential Random Variables Poisson Random Variables Poisson Processes Superposition of Poisson Processes Thinning of a Poisson Process Compound Poisson Processes Problems Continuous-Time Markov Models Continuous-Time Markov Chains Examples of CTMCs: I xi

12 xii Contents 4.3 Examples of CTMCs: II Transient Analysis: Uniformization Occupancy Times Limiting Behavior Cost Models Expected Total Cost Long-Run Cost Rates First-Passage Times Case Study: Cars-R-Us Used Car Dealer Appendix 4A: Proof Of Theorem Appendix 4B: Uniformization Algorithm to Compute P.t/ Appendix 4C: Uniformization Algorithm to Compute M.T/ Problems Generalized Markov Models Introduction Renewal Processes Cumulative Processes Semi-Markov Processes: Examples Semi-Markov Processes: Long-Term Analysis Mean Inter-visit Times Occupancy Distributions Long-Run Cost Rates Case Study: Healthy Heart Coronary Care Facility Problems Queueing Models Queueing Systems Single-Station Queues: General Results Birth and Death Queues with Finite Capacity M/M/1/K Queue M/M/s/K Queue M/M/K/K Queue Birth and Death Queues with Infinite Capacity M/M/1 Queue M/M/s Queue M/M/1 Queue M/G/1 Queue G/M/1 Queue Networks of Queues Jackson Networks Stability Limiting Behavior Case Study: Healthy Heart Coronary Care Facility Problems...236

13 Contents xiii 7 Brownian Motion Univariate Normal Random Variables Multivariate Normal Random Variables Standard Brownian Motion Brownian Motion First-Passage Times in an SBM First-Passage Times in a BM Cost Models Black Scholes Formula Problems Appendix A Probability A.1 Probability Model A.2 Conditional Probability A.3 Law of Total Probability A.4 Bayes Rule A.5 Independence Appendix B Univariate Random Variables B.1 Discrete Random Variables B.2 Continuous Random Variables B.3 Functions of Random Variables B.4 Expectations Appendix C Multivariate Random Variables C.1 Multivariate Discrete Random Variables C.2 Multivariate Continuous Random Variables C.3 Marginal Distributions C.4 Independence C.5 Sums of Random Variables C.6 Expectations Appendix D Conditional Distributions and Expectations D.1 Conditional Distributions D.2 Conditional Expectations D.3 Random Sums Answers to Selected Problems Further Reading Index...311

14