Probability Methods and measurement
OTHER STATISTICS TEXTS FROM CHAPMAN AND HALL The Analysis of Time Series C. Chatfield Statistics for Technology C. Chatfield Introduction to Multivariate Analysis C: Chatfield and A.J. Collins Applied Statistics D.R. Cox and E.J. Snell An Introduction to Statistical Modelling A.J. Dobson Introduction to Optimization Methods and their Application in Statistics B.S. Everitt Multivariate Statistics - A Practical Approach B. Flury and H. Riedwyl Multivariate Analysis of Variance and Repeated Measures D.J. Hand and c.c. Taylor Multivariate Statistical Methods - a primer Bryan F. Manley Statistical Methods in Agriculture and Experimental Biology R. Mead and R.N. Curnow Elements of Simulation B.J.T. Morgan Essential Statistics D.G. Rees Foundations of Statistics D.G. Rees Decision Analysis: A Bayesian Approach J.Q. Smith Applied Statistics: A Handbook of BMDP Analyses E.J. Snell Elementary Applications of Probability Theory H.C. Tuckwell Intermediate Statistical Methods G.B. Wetherill Further information of the complete range of Chapman and Hall statistics books is available from the publishers.
Probability Methods and measurement Anthony O'Hagan University of Warwick London New York CHAPMAN AND HALL
First published in 1988 by Chapman and Hall Ltd 11 New Fetter Lane, London EC4P 4EE Published in the USA by Chapman and Hall 29 West 35th Street, New York NY 10001 1988 A. O'Hagan ISBN-13: 978-94-010-7038-6 e-isbn-13: 978-94-009-1211-3 001: 10.1 007/978-94-009-1211-3 This title is available in both hardbound and paperback editions. The paperback edition is sold subject to the condition that it shall not, by way of trade or otherwise, be lent, resold, hired out, or otherwise circulated without the publisher's prior consent in any form of binding or cover other than that in which it is published and without a similar condition including this condition being imposed on the subsequent purchaser. All rights reserved. No part of this book may be reprinted, or reproduced or utilized in any form or by any electronic, mechanical or other means, now known or hereafter invented, including photocopying and recording, or in any information storage and retrieval system, without permission in writing from the publisher. British Library Cataloguing in Publication Data O'Hagan, Anthony Probability: methods and measurement. 1. Probabilities I. Title 519.2 QA273 Library of Congress Cataloging in Publication Data O'Hagan, Anthony. Probability: methods and measurement. Bibliography: p. Includes index. 1. Probabilities. I. Title. QA273.034 1988 519.2 87-29999
To Anne
Contents Preface xi 1 Probability and its laws 1 1.1 Uncertainty and probability 1 1.2 Direct measurement 4 Exercises l(a) 8 1.3 Betting behaviour 8 1.4 Fair bets 10 1.5 The Addition Law 12 Exercises l(b) 15 1.6 The Multiplication Law 16 1.7 Independence 18 Exercises 1 ( c ) 19 2 Probability measurements 21 2.1 True probabilities 21 Exercises 2(a) 26 2.2 Elaboration 26 Exercises 2(b) 30 2.3 The disjunction theorem 31 Exercises 2( c) 33 2.4 The sum theorem 33 Exercises 2( d) 36 2.5 Partitions 38 2.6 Symmetry probability 40 Exercises 2( e) 44 3 Bayes' theorem 45 3.1 Extending the argument 45 Exercises 3(a) 50 3.2 Bayes'theorem 51
viii Contents 3.3 Learning from experience 53 Exercises 3(b) 56 3.4 Zero probabilities in Bayes' theorem 57 3.5 Example: disputed authorship 59 4 Trials and deals 62 4.1 The product theorem 62 4.2 Mutual independence 63 Exercises 4( a) 66 4.3 Trials 67 4.4 Factorials and combinations 69 Exercises 4(b) 71 4.5 Binomial probabilities 72 Exercises 4( c) 74 4.6 Multinomial probabilities 75 Exercises 4( d) 76 4.7 Deals 77 4.8 Probabilities from information 79 Exercises 4( e) 82 4.9 Properties of deals 82 4.10 H ypergeometric probabilities 87 Exercises 4(f) 89 4.11 Deals from large collections 89 Exercises 4(g) 92 5 Random variables 93 5.1 Definitions 93 5.2 Two or more random variables 95 Exercises 5(a) 99 5.3 Elaborations with random variables 100 5.4 Example: capture-recapture 104 5.5 Example: job applications 107 Exercises 5(b) 109 5.6 Mean and standard deviation 110 Exercises 5(c) 115 5.7 Measuring distributions 115 5.8 Some standard distributions 120 Exercises 5( d) 130 6 Distribution theory 132 6.1 Deriving standard distributions 132 6.2 Combining distributions 136 Exercises 6( a) 138
Contents 6.3 Basic theory of expectations 138 6.4 Further expectation theory 141 Exercises 6(b) 145 6.5 Covariance and correlation 146 Exercises 6( c ) 149 6.6 Conditional expectations 149 6.7 Linear regression functions 152 Exercises 6( d) 156 7 Continuous distributions 157 7.1 Continuous random variables 157 7.2 Distribution functions 161 Exercises 7(a) 163 7.3 Density functions 164 7.4 Transformations and expectations 168 Exercises 7(b) 171 7.5 Standard continuous distributions 171 Exercises 7(c) 181 7.6 Two continuous random variables 182 7.7 Example: heat transfer 190 Exercises 7( d) 193 7.8 Random variables of mixed type 194 Exercises 7 ( e ) 197 7.9 Continuous distribution theory 197 Exercises 7(f) 203 8 Frequencies 204 8.1 Exchangeable propositions 204 8.2 The finite characterization 208 Exercises 8(a) 210 8.3 De Finetti's theorem 211 8.4 Updating 213 Exercises 8(b) 216 8.5 Beta prior distributions 217 Exercises 8( c) 219 8.6 Probability and frequency 220 8.7 Calibration 225 9 Statistical models 227 9.1 Parameters and models 227 9.2 Exchangeable random variables 231 Exercises 9(a) 235 ix
x Contents 9.3 Samples 236 9.4 Measuring mean and variance 240 Exercises 9(b) 243 9.5 Exchangeable parametric models 243 9.6 The normal location model 246 Exercises 9( c) 252 9.} The Poisson model 253 9.8 Linear estimation 256 Exercises 9( d) 261 9.9 Postscript 261 Appendix - Solutions to exercises 263 Index 287
Preface This book is an elementary and practical introduction to probability theory. It differs from other introductory texts in two important respects. First, the personal (or subjective) view of probability is adopted throughout. Second, emphasis is placed on how values are assigned to probabilities in practice, i.e. the measurement of probabilities. The personal approach to probability is in many ways more natural than other current formulations, and can also provide a broader view of the subject. It thus has a unifying effect. It has also assumed great importance recently because of the growth of Bayesian Statistics. Personal probability is essential for modern Bayesian methods, and it can be difficult for students who have learnt a different view of probability to adapt to Bayesian thinking. This book has been produced in response to that difficulty, to present a thorough introduction to probability from scratch, and entirely in the personal framework. The practical question of assigning probability values in real problems is largely ignored in the traditional way of teaching probability theory. Either the problems are abstract, so that no probabilities are required, or they deal with artificial contexts like dice and coin tossing in which probabilities are implied by symmetry, or else the student is told explicitly to assume certain values. In this book, the reader is invited from the beginning to attempt to measure probabilities in practical contexts. The development of probability theory becomes a process of devising better tools to measure probabilities in practice. It is then natural and useful to distinguish between a true value of a probability and a practical measurement of it. They will differ because of the inadequacy or inaccuracy of the measurement technique. Although natural, this distinction between true and measured probabilities is not found in other texts. I should emphasize that the notion of a true probability is not intended to be precisely or rigorously defined, in the way that philosophers of science would demand. It is merely a pragmatic device, in the same way as any other 'true measurement'. For instance, attempts to define length to absolute precision will run into difficulties at the atomic level: atoms, and the particles within them, do not stay in rigid positions. Yet in practice we are undeterred from referring to 'the' length of something. I have assumed only a very basic knowledge of mathematics. In particular, for the first six chapters no calculus is used, and the reader will require only some proficiency in basic algebra. Calculus is necessary for Chapter 7, and to some extent for the last two chapters, but again only an elementary knowledge is required.
Two devices are used throughout the text to highlight important ideas and to identify less vital matters. Displayed material All the important definitions. theorems and other conclusions reached in this book are displayed between horizontal lines in this way. {.... Asides This indented format. begun and ended with brackets and trailing dots. is used for all proofs of theorems. many examples. and other material which is separate from the main flow of ideas in the text.... } There are over 150 exercises. almost every one of which is given a fully worked solution in the Appendix. You should attempt as many of the exercises as possible. Do not consult the solutions without first tackling the questions! You will learn much more by attempting exercises than by simply following worked examples. Many people have generously given me help and encouragement in preparing this book. To thank them all would be impossible. I must mention Dennis Lindley and Peter Freeman. whose critical reading was greatly appreciated. Nor must I forget my family and their long-suffering support. I am sincerely grateful to all these people. University a/warwick, Coventry August 1987