CRITICAL APPRAISAL OF MEDICAL LITERATURE

CRITICAL APPRAISAL OF MEDICAL LITERATURE

CRITICAL APPRAISAL OF MEDICAL LITERATURE David Marchevsky Mi/ton Keynes Community NHS Trust Mi/ton Keynes, England Foreword by Dr. Naum Marchevsky SPRINGER SCIENCE+BUSINESS MEDIA, LLC

Library of Congress Cataloging-in-Publication Data Marchevsky, David. Critical appraisal of medical Iiterature/David Marchevsky; foreword by Naum Marchevsky. p. ; cm. Includes bibliographical references and index. ISBN 978-1-4613-6887-8 ISBN 978-1-4615-4205-6 (ebook) DOI 10.1007/978-1-4615-4205-6 1. Medicalliterature-Evaluation. 2. Medicine-Research-Evaluation. 3. Clinical medicine-decision making. I. Title. [DNLM: 1. Biometry-methods. 2. Meta-Analysis. 3. Research Design. 4. Statistics-methods. 5. Writing. WA 950 M317c 2000] R1l8.6.M37 2000 610'.7'2-dc21 00-135008 Front cover: Synthesis and Analysis ISBN 978-1-4613-6887-8 2000 Springer Science+Business Media New York Originally published by Kluwer AcademicIPlenum Publishers in 2000 Softcover reprint of the hardcover 1 st edition 2000 AII rights reserved No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording, or otherwise, \Vithout written permission from the Publisher

To Corrinna, Natalie, Eduard and Tobias v

FOREWORD I am pleased and proud to have been asked to write the Foreword for this book. I am especially proud because, as you can see, the author's surname and mine are the same. It is not a coincidence; it happens that Dr. David Marchevsky is a nephew of mine (a fact for which, at any rate, he is not responsible). I am very pleased to have participated in the extraordinary feat accomplished by the author, although in a very modest way, by reviewing the manuscript and by making a few comments and suggestions. Let me tell you why I consider this book an important contribution to the medical community in general, and the psychiatric establishment in particular (you may have noticed that the author is a psychiatrist, a fact for which, at any rate, he is responsible). When I worked in the Pan American Zoonosis Center, in Buenos Aires, Argentina, I used to give short courses in statistics for groups of different specialities of health sciences. As a norm, I liked to use examples taken from material related to the groups' particular interests, to make it easier for them to understand the rationale of using different statistical tools. I used to look in different journals for some well-designed works that had sound methodology and data analysis. For some medical specialities, notably psychiatry, I would only find a handful of papers that complied with these conditions. Most of them would contain dogmatic statements, based only on the more or less clever interpretation of the observations made, and in many cases, from one single patient. Things have changed since then. Some journals adhere to strict rules and have bodies of expert statistical and research advisors who help to keep these rules going and to stop faulty information slipping through. Unfortunately, many medical journals cannot afford to employ these professionals, so the clinician is left to 'guess' the soundness of what he reads. Another important factor is the rapid increase in the number of journals and published articles year in year out, which makes the clinician's task even more difficult. Some papers and books that contain didactic material on how to evaluate medical literature, offer enough guidance for weighing clinical significance and some aspects of their practical use, what the author calls importance and usefulness respectively. But these sources do not give much away when it comes to guiding the clinician in the difficult art of assessing what the author calls validity. Why is this? It is simply because the amount of knowledge required comes from many disciplines and the task to condense it is huge. This book goes beyond others to fill the gap. I see this book as an important aid to the practitioners who do not accept dogma or statements made without the support of welldesigned research. The author gives a synthesis of fundamental topics that will help practitioners to better judge the soundness of what they read. To this end, this book introduces basic concepts of logic, probability, research design, statistics and epidemiology. All of this is well integrated Vll

viii Critical Appraisal of Medical Literature with the guides to the critique of articles on different medical topics, such as treatment, diagnosis, prognosis and etiology. This book also offers additional details on many statistical methods used in published articles. Even allowing for the understandable 'familial bias', let me recommend this book to the medical practitioners who are always trying to give the best there is to their patients and to those who are curious about learning the difficult art of critical evaluation of medical literature. Dr. Naum Marchevsky M.D., M.Sc. in Biostatistics

PREFACE The first phrase I heard in medical school was medicine is a life-long learning process. We learn from our personal professional experience and from processing medical information that is available to us. This book is concerned with the latter. I believe that neither stubborn scepticism nor uncritical acceptance of the medical information that surrounds us contribute much to our personal learning process. It is essential that we maintain a healthy level of scepticism before the information put to us and that we also keep an open-minded attitude towards new ideas and developments. On the other hand, we should also be critical of unwarranted generalisations and unsupported statements. I see systematic critical appraisal of medical information as a sort of platform that integrates these attitudes harmoniously. The development of critical appraisal skills implies having a theoretical and practical knowledge of topics that belong to different disciplines. As with many other skills, critical appraisal must be practised consistently and regularly. I have written this book to help the reader who is unfamiliar with systematic critical appraisal of medical literature to achieve a good level of proficiency in this area. I have used material that I have compiled for some years in order to give the reader a guide to this subject that is as comprehensive as possible. This book is intended to reach those interested in developing critical appraisal skills, such as psychiatrists preparing for the Critical Review Paper of the MRCPsych Examination in the UK, other practitioners as part of their preparation for examinations that include components similar to the Critical Review Paper, medical trainers as part of their teaching material, and other professionals and students as part of their introduction to aspects of systematic critical appraisal of medical information. I would like to express my sincere gratitude to Dr Naum Marchevsky, who read previous drafts and gave me so many useful pointers. His contribution goes further than his invaluable help with the statistical and epidemiological content of my book because he has also broadened my knowledge and curiosity on these and other related topics. I would like to thank my parents and Terri Sklar for their encouragement, and especially my wife Corrinna, for her invaluable help and her unconditional and neverending support without which I would never have completed this project. I would also like to thank Dr Horacio Fischer, Dr Phil Harrison-Read, Dr Brian Kaufman, Dr Lawrence Measey, Dr Rajan Thavasothy and Dr Elizabeth Wilmot for their inspiring positive influence on me. David Marchevsky ix

CONTENTS INTRODUCTION 1. Critical Appraisal in the United Kingdom 2. How To Improve Critical Appraisal Skills 3. How To Use This Book 1 2 2 3 PART] CHAPTER 1. BASIC CONCEPTS 1. Argument 2. Appraising Medical Evidence CHAPTER 2. PROBABILITY 1. Set and Subset 2. Random Experiment 3. Sample Space 4. Event and Probability 5. Classical Approach To Probability 6. Frequentist Approaches to Probability 7. Conditional Probability 8. Bayes' Theorem 9. Subjective Approaches To Probability 10. Proportions and Percentages 11. Odds, Odds Ratio and Log Odds 12. Summary CHAPTER 3. V ARlABLES AND MEASUREMENT 1. Variable 2. Measurement 3. Summary CHAPTER 4. MEASUREMENT INSTRUMENTS 1. Case 2. Qualitative Aspects of Measurement 3. Quantitative Aspects of Measurement 4. Receiver Operating Characteristic (ROC) Analysis 5. Summary 7 8 9 11 11 12 12 13 16 16 17 19 19 21 21 23 25 25 28 30 31 31 32 34 40 41 Xl

xu CHAPTER 5. SAMPLING AND DESIGN 1. Population and Sample 2. Sampling 3. Study Design 4. Observational Studies 5. Experimental Design 6. Summary CHAPTER 6. BIAS 1. General Points 2. Bias 3. Confounding 4. Summary CHAPTER 7. DISTRIBUTIONS 1. Frequency Distributions 2. Functions 3. Central Tendency 4. Shape 5. Partition 6. Spread, Variability or Variation 7. Sampling Distribution 8. Summary Critical Appraisal of Medical Literature 43 43 44 50 51 52 54 57 57 58 60 60 63 63 64 66 67 70 71 73 74 CHAPTER 8. BINOMIAL AND NORMAL DISTRIBUTIONS 75 1. Binomial Distributions 75 2. Normal Distributions 77 3. Summary 81 CHAPTER 9. HYPOTHESIS TESTING 1. General Principles: Statistical Models 2. Steps in Hypothesis Testing 3. Final Comments 4. Summary CHAPTER 10. ERRORS AND SAMPLE SIZE 1. Errors 2. Sample Size 3. Summary CHAPTER 11. ESTIMATION 1. General Points 2. Sampling Distribution and Standard Error 3. Estimators 4. Bias 5. Precision 6. Confidence Intervals 7. Summary 83 83 85 91 92 93 93 101 103 105 105 106 108 109 109 110 112

Contents CHAPTER 12. EPIDEMIOLOGY 1. General Concepts 2. Measures of Disease Frequency 3. Measures of Association 4. Standardisation 5. Epidemiological Studies 6. Summary 115 115 115 118 120 124 128 xiii PART II CHAPTER 13. INTRODUCTION TO CRITICAL APPRAISAL OF MEDICAL LITERATURE 133 1. General Points 133 2. How To Practice Critical Appraisal 135 3. Specific Points 136 4. Other Considerations 138 CHAPTER 14. CRITICAL APPRAISAL OF DIFFERENT STUDY DESIGNS 139 1. Descriptive or Observational Studies 139 2. Experimental Studies 141 CHAPTER 15. PAPERS ON TREATMENT 143 1. More Issues on Validity 143 2. Importance 144 3. Usefulness 147 CHAPTER 16. PAPERS ON DIAGNOSIS 149 1. More Issues on Validity 149 2. Importance 150 3. Usefulness 153 CHAPTER 17. PAPERS ON PROGNOSIS 155 1. Preliminary Concepts 155 2. Description and Analysis 158 3. More Issues on Validity 159 4. Importance 160 5. Usefulness 161 CHAPTER 18. PAPERS ON AETIOLOGY 163 1. More Issues on Validity 163 2. Importance 164 3. Usefulness 166 CHAPTER 19. REVIEW PAPERS 169 1. Notes on Reviews 169 2. Meta-analysis 171 3. Appraisal 176 4. Validity 176 5. Importance 176 6. Usefulness 177

xiv CHAPTER 20. PAPERS ON GUIDELINES 1. Notes on Guidance 2. Validity 3. Importance 4. Usefulness CHAPTER 21. PAPERS ON AUDIT 1. Notes on Audit 2. Validity 3. Importance 4. Usefulness Critical Appraisal of Medical Literature 179 179 179 180 180 183 183 184 185 185 CHAPTER 22. PAPERS ON QUALITATIVE RESEARCH 1. Notes on Qualitative Research 2. Appraisal CHAPTER 23. SUMMARY 1. General Points 2. Specific Points 187 187 189 191 191 192 PART II! CHAPTER 24. THE z- TEST AND THE t-test 1. The z-test 2. The (-test 3. Examples 4. Summary CHAPTER 25. ANALYSIS OF VARIANCE 1. General Points 2. One-way Anova 3. Two-way Anova 4. Three-way Anova 5. Multivariable Analysis of Variance 6. Related Measures Anova and Manova 7. Example 8. Summary CHAPTER 26. NON-PARAMETRIC TESTS I. The Chi-square Test 2. Fisher's Exact Probability Test 3. The Mann-Whitney Test For Two Independent Samples 4. The Wilcoxon Test 5. Kruskal-Wallis H-test For More Than Two Samples 6. Examples 7. Summary CHAPTER 27. CORRELATION, REGRESSION AND PREDICTION I. Correlation 2. Regression and Prediction 3. Multiple Prediction 199 199 202 208 210 211 211 211 216 220 220 220 221 222 223 223 227 228 230 231 232 233 235 235 240 243

Contents 4. Example 247 5. Summary 248 CHAPTER 28. MULTIV ARIABLE ANALYSIS 249 1. Principal Component Analysis and Factor Analysis 249 2. Discriminant Analysis 256 3. Examples 261 4. Summary 262 CHAPTER 29. OTHER STATISTICAL TECHNIQUES 265 1. Analysis of Covariance (Ancova) 265 2. Logistic Regression Analysis 271 3. Examples 276 4. Summary 278 REFERENCES 281 INDEX 285 xv