Probability. Methods and measurement

Transcription

1 Probability Methods and measurement

2 OTHER STATISTICS TEXTS FROM CHAPMAN AND HALL The Analysis of Time Series C. Chatfield Statistics for Technology C. Chatfield Introduction to Multivariate Analysis C: Chatfield and A.J. Collins Applied Statistics D.R. Cox and E.J. Snell An Introduction to Statistical Modelling A.J. Dobson Introduction to Optimization Methods and their Application in Statistics B.S. Everitt Multivariate Statistics - A Practical Approach B. Flury and H. Riedwyl Multivariate Analysis of Variance and Repeated Measures D.J. Hand and c.c. Taylor Multivariate Statistical Methods - a primer Bryan F. Manley Statistical Methods in Agriculture and Experimental Biology R. Mead and R.N. Curnow Elements of Simulation B.J.T. Morgan Essential Statistics D.G. Rees Foundations of Statistics D.G. Rees Decision Analysis: A Bayesian Approach J.Q. Smith Applied Statistics: A Handbook of BMDP Analyses E.J. Snell Elementary Applications of Probability Theory H.C. Tuckwell Intermediate Statistical Methods G.B. Wetherill Further information of the complete range of Chapman and Hall statistics books is available from the publishers.

3 Probability Methods and measurement Anthony O'Hagan University of Warwick London New York CHAPMAN AND HALL

4 First published in 1988 by Chapman and Hall Ltd 11 New Fetter Lane, London EC4P 4EE Published in the USA by Chapman and Hall 29 West 35th Street, New York NY A. O'Hagan ISBN-13: e-isbn-13: : / This title is available in both hardbound and paperback editions. The paperback edition is sold subject to the condition that it shall not, by way of trade or otherwise, be lent, resold, hired out, or otherwise circulated without the publisher's prior consent in any form of binding or cover other than that in which it is published and without a similar condition including this condition being imposed on the subsequent purchaser. All rights reserved. No part of this book may be reprinted, or reproduced or utilized in any form or by any electronic, mechanical or other means, now known or hereafter invented, including photocopying and recording, or in any information storage and retrieval system, without permission in writing from the publisher. British Library Cataloguing in Publication Data O'Hagan, Anthony Probability: methods and measurement. 1. Probabilities I. Title QA273 Library of Congress Cataloging in Publication Data O'Hagan, Anthony. Probability: methods and measurement. Bibliography: p. Includes index. 1. Probabilities. I. Title. QA

5 To Anne

6 Contents Preface xi 1 Probability and its laws Uncertainty and probability Direct measurement 4 Exercises l(a) Betting behaviour Fair bets The Addition Law 12 Exercises l(b) The Multiplication Law Independence 18 Exercises 1 ( c ) 19 2 Probability measurements True probabilities 21 Exercises 2(a) Elaboration 26 Exercises 2(b) The disjunction theorem 31 Exercises 2( c) The sum theorem 33 Exercises 2( d) Partitions Symmetry probability 40 Exercises 2( e) 44 3 Bayes' theorem Extending the argument 45 Exercises 3(a) Bayes'theorem 51

7 viii Contents 3.3 Learning from experience 53 Exercises 3(b) Zero probabilities in Bayes' theorem Example: disputed authorship 59 4 Trials and deals The product theorem Mutual independence 63 Exercises 4( a) Trials Factorials and combinations 69 Exercises 4(b) Binomial probabilities 72 Exercises 4( c) Multinomial probabilities 75 Exercises 4( d) Deals Probabilities from information 79 Exercises 4( e) Properties of deals H ypergeometric probabilities 87 Exercises 4(f) Deals from large collections 89 Exercises 4(g) 92 5 Random variables Definitions Two or more random variables 95 Exercises 5(a) Elaborations with random variables Example: capture-recapture Example: job applications 107 Exercises 5(b) Mean and standard deviation 110 Exercises 5(c) Measuring distributions Some standard distributions 120 Exercises 5( d) Distribution theory Deriving standard distributions Combining distributions 136 Exercises 6( a) 138

8 Contents 6.3 Basic theory of expectations Further expectation theory 141 Exercises 6(b) Covariance and correlation 146 Exercises 6( c ) Conditional expectations Linear regression functions 152 Exercises 6( d) Continuous distributions Continuous random variables Distribution functions 161 Exercises 7(a) Density functions Transformations and expectations 168 Exercises 7(b) Standard continuous distributions 171 Exercises 7(c) Two continuous random variables Example: heat transfer 190 Exercises 7( d) Random variables of mixed type 194 Exercises 7 ( e ) Continuous distribution theory 197 Exercises 7(f) Frequencies Exchangeable propositions The finite characterization 208 Exercises 8(a) De Finetti's theorem Updating 213 Exercises 8(b) Beta prior distributions 217 Exercises 8( c) Probability and frequency Calibration Statistical models Parameters and models Exchangeable random variables 231 Exercises 9(a) 235 ix

9 x Contents 9.3 Samples Measuring mean and variance 240 Exercises 9(b) Exchangeable parametric models The normal location model 246 Exercises 9( c) } The Poisson model Linear estimation 256 Exercises 9( d) Postscript 261 Appendix - Solutions to exercises 263 Index 287

10 Preface This book is an elementary and practical introduction to probability theory. It differs from other introductory texts in two important respects. First, the personal (or subjective) view of probability is adopted throughout. Second, emphasis is placed on how values are assigned to probabilities in practice, i.e. the measurement of probabilities. The personal approach to probability is in many ways more natural than other current formulations, and can also provide a broader view of the subject. It thus has a unifying effect. It has also assumed great importance recently because of the growth of Bayesian Statistics. Personal probability is essential for modern Bayesian methods, and it can be difficult for students who have learnt a different view of probability to adapt to Bayesian thinking. This book has been produced in response to that difficulty, to present a thorough introduction to probability from scratch, and entirely in the personal framework. The practical question of assigning probability values in real problems is largely ignored in the traditional way of teaching probability theory. Either the problems are abstract, so that no probabilities are required, or they deal with artificial contexts like dice and coin tossing in which probabilities are implied by symmetry, or else the student is told explicitly to assume certain values. In this book, the reader is invited from the beginning to attempt to measure probabilities in practical contexts. The development of probability theory becomes a process of devising better tools to measure probabilities in practice. It is then natural and useful to distinguish between a true value of a probability and a practical measurement of it. They will differ because of the inadequacy or inaccuracy of the measurement technique. Although natural, this distinction between true and measured probabilities is not found in other texts. I should emphasize that the notion of a true probability is not intended to be precisely or rigorously defined, in the way that philosophers of science would demand. It is merely a pragmatic device, in the same way as any other 'true measurement'. For instance, attempts to define length to absolute precision will run into difficulties at the atomic level: atoms, and the particles within them, do not stay in rigid positions. Yet in practice we are undeterred from referring to 'the' length of something. I have assumed only a very basic knowledge of mathematics. In particular, for the first six chapters no calculus is used, and the reader will require only some proficiency in basic algebra. Calculus is necessary for Chapter 7, and to some extent for the last two chapters, but again only an elementary knowledge is required.

11 Two devices are used throughout the text to highlight important ideas and to identify less vital matters. Displayed material All the important definitions. theorems and other conclusions reached in this book are displayed between horizontal lines in this way. {.... Asides This indented format. begun and ended with brackets and trailing dots. is used for all proofs of theorems. many examples. and other material which is separate from the main flow of ideas in the text.... } There are over 150 exercises. almost every one of which is given a fully worked solution in the Appendix. You should attempt as many of the exercises as possible. Do not consult the solutions without first tackling the questions! You will learn much more by attempting exercises than by simply following worked examples. Many people have generously given me help and encouragement in preparing this book. To thank them all would be impossible. I must mention Dennis Lindley and Peter Freeman. whose critical reading was greatly appreciated. Nor must I forget my family and their long-suffering support. I am sincerely grateful to all these people. University a/warwick, Coventry August 1987