CRITICAL APPRAISAL OF MEDICAL LITERATURE

Transcription

1 CRITICAL APPRAISAL OF MEDICAL LITERATURE

2 CRITICAL APPRAISAL OF MEDICAL LITERATURE David Marchevsky Mi/ton Keynes Community NHS Trust Mi/ton Keynes, England Foreword by Dr. Naum Marchevsky SPRINGER SCIENCE+BUSINESS MEDIA, LLC

3 Library of Congress Cataloging-in-Publication Data Marchevsky, David. Critical appraisal of medical Iiterature/David Marchevsky; foreword by Naum Marchevsky. p. ; cm. Includes bibliographical references and index. ISBN ISBN (ebook) DOI / Medicalliterature-Evaluation. 2. Medicine-Research-Evaluation. 3. Clinical medicine-decision making. I. Title. [DNLM: 1. Biometry-methods. 2. Meta-Analysis. 3. Research Design. 4. Statistics-methods. 5. Writing. WA 950 M317c 2000] R1l8.6.M '.7'2-dc Front cover: Synthesis and Analysis ISBN Springer Science+Business Media New York Originally published by Kluwer AcademicIPlenum Publishers in 2000 Softcover reprint of the hardcover 1 st edition 2000 AII rights reserved No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording, or otherwise, \Vithout written permission from the Publisher

4 To Corrinna, Natalie, Eduard and Tobias v

5 FOREWORD I am pleased and proud to have been asked to write the Foreword for this book. I am especially proud because, as you can see, the author's surname and mine are the same. It is not a coincidence; it happens that Dr. David Marchevsky is a nephew of mine (a fact for which, at any rate, he is not responsible). I am very pleased to have participated in the extraordinary feat accomplished by the author, although in a very modest way, by reviewing the manuscript and by making a few comments and suggestions. Let me tell you why I consider this book an important contribution to the medical community in general, and the psychiatric establishment in particular (you may have noticed that the author is a psychiatrist, a fact for which, at any rate, he is responsible). When I worked in the Pan American Zoonosis Center, in Buenos Aires, Argentina, I used to give short courses in statistics for groups of different specialities of health sciences. As a norm, I liked to use examples taken from material related to the groups' particular interests, to make it easier for them to understand the rationale of using different statistical tools. I used to look in different journals for some well-designed works that had sound methodology and data analysis. For some medical specialities, notably psychiatry, I would only find a handful of papers that complied with these conditions. Most of them would contain dogmatic statements, based only on the more or less clever interpretation of the observations made, and in many cases, from one single patient. Things have changed since then. Some journals adhere to strict rules and have bodies of expert statistical and research advisors who help to keep these rules going and to stop faulty information slipping through. Unfortunately, many medical journals cannot afford to employ these professionals, so the clinician is left to 'guess' the soundness of what he reads. Another important factor is the rapid increase in the number of journals and published articles year in year out, which makes the clinician's task even more difficult. Some papers and books that contain didactic material on how to evaluate medical literature, offer enough guidance for weighing clinical significance and some aspects of their practical use, what the author calls importance and usefulness respectively. But these sources do not give much away when it comes to guiding the clinician in the difficult art of assessing what the author calls validity. Why is this? It is simply because the amount of knowledge required comes from many disciplines and the task to condense it is huge. This book goes beyond others to fill the gap. I see this book as an important aid to the practitioners who do not accept dogma or statements made without the support of welldesigned research. The author gives a synthesis of fundamental topics that will help practitioners to better judge the soundness of what they read. To this end, this book introduces basic concepts of logic, probability, research design, statistics and epidemiology. All of this is well integrated Vll

6 viii Critical Appraisal of Medical Literature with the guides to the critique of articles on different medical topics, such as treatment, diagnosis, prognosis and etiology. This book also offers additional details on many statistical methods used in published articles. Even allowing for the understandable 'familial bias', let me recommend this book to the medical practitioners who are always trying to give the best there is to their patients and to those who are curious about learning the difficult art of critical evaluation of medical literature. Dr. Naum Marchevsky M.D., M.Sc. in Biostatistics

7 PREFACE The first phrase I heard in medical school was medicine is a life-long learning process. We learn from our personal professional experience and from processing medical information that is available to us. This book is concerned with the latter. I believe that neither stubborn scepticism nor uncritical acceptance of the medical information that surrounds us contribute much to our personal learning process. It is essential that we maintain a healthy level of scepticism before the information put to us and that we also keep an open-minded attitude towards new ideas and developments. On the other hand, we should also be critical of unwarranted generalisations and unsupported statements. I see systematic critical appraisal of medical information as a sort of platform that integrates these attitudes harmoniously. The development of critical appraisal skills implies having a theoretical and practical knowledge of topics that belong to different disciplines. As with many other skills, critical appraisal must be practised consistently and regularly. I have written this book to help the reader who is unfamiliar with systematic critical appraisal of medical literature to achieve a good level of proficiency in this area. I have used material that I have compiled for some years in order to give the reader a guide to this subject that is as comprehensive as possible. This book is intended to reach those interested in developing critical appraisal skills, such as psychiatrists preparing for the Critical Review Paper of the MRCPsych Examination in the UK, other practitioners as part of their preparation for examinations that include components similar to the Critical Review Paper, medical trainers as part of their teaching material, and other professionals and students as part of their introduction to aspects of systematic critical appraisal of medical information. I would like to express my sincere gratitude to Dr Naum Marchevsky, who read previous drafts and gave me so many useful pointers. His contribution goes further than his invaluable help with the statistical and epidemiological content of my book because he has also broadened my knowledge and curiosity on these and other related topics. I would like to thank my parents and Terri Sklar for their encouragement, and especially my wife Corrinna, for her invaluable help and her unconditional and neverending support without which I would never have completed this project. I would also like to thank Dr Horacio Fischer, Dr Phil Harrison-Read, Dr Brian Kaufman, Dr Lawrence Measey, Dr Rajan Thavasothy and Dr Elizabeth Wilmot for their inspiring positive influence on me. David Marchevsky ix

8 CONTENTS INTRODUCTION 1. Critical Appraisal in the United Kingdom 2. How To Improve Critical Appraisal Skills 3. How To Use This Book PART] CHAPTER 1. BASIC CONCEPTS 1. Argument 2. Appraising Medical Evidence CHAPTER 2. PROBABILITY 1. Set and Subset 2. Random Experiment 3. Sample Space 4. Event and Probability 5. Classical Approach To Probability 6. Frequentist Approaches to Probability 7. Conditional Probability 8. Bayes' Theorem 9. Subjective Approaches To Probability 10. Proportions and Percentages 11. Odds, Odds Ratio and Log Odds 12. Summary CHAPTER 3. V ARlABLES AND MEASUREMENT 1. Variable 2. Measurement 3. Summary CHAPTER 4. MEASUREMENT INSTRUMENTS 1. Case 2. Qualitative Aspects of Measurement 3. Quantitative Aspects of Measurement 4. Receiver Operating Characteristic (ROC) Analysis 5. Summary Xl

9 xu CHAPTER 5. SAMPLING AND DESIGN 1. Population and Sample 2. Sampling 3. Study Design 4. Observational Studies 5. Experimental Design 6. Summary CHAPTER 6. BIAS 1. General Points 2. Bias 3. Confounding 4. Summary CHAPTER 7. DISTRIBUTIONS 1. Frequency Distributions 2. Functions 3. Central Tendency 4. Shape 5. Partition 6. Spread, Variability or Variation 7. Sampling Distribution 8. Summary Critical Appraisal of Medical Literature CHAPTER 8. BINOMIAL AND NORMAL DISTRIBUTIONS Binomial Distributions Normal Distributions Summary 81 CHAPTER 9. HYPOTHESIS TESTING 1. General Principles: Statistical Models 2. Steps in Hypothesis Testing 3. Final Comments 4. Summary CHAPTER 10. ERRORS AND SAMPLE SIZE 1. Errors 2. Sample Size 3. Summary CHAPTER 11. ESTIMATION 1. General Points 2. Sampling Distribution and Standard Error 3. Estimators 4. Bias 5. Precision 6. Confidence Intervals 7. Summary

10 Contents CHAPTER 12. EPIDEMIOLOGY 1. General Concepts 2. Measures of Disease Frequency 3. Measures of Association 4. Standardisation 5. Epidemiological Studies 6. Summary xiii PART II CHAPTER 13. INTRODUCTION TO CRITICAL APPRAISAL OF MEDICAL LITERATURE General Points How To Practice Critical Appraisal Specific Points Other Considerations 138 CHAPTER 14. CRITICAL APPRAISAL OF DIFFERENT STUDY DESIGNS Descriptive or Observational Studies Experimental Studies 141 CHAPTER 15. PAPERS ON TREATMENT More Issues on Validity Importance Usefulness 147 CHAPTER 16. PAPERS ON DIAGNOSIS More Issues on Validity Importance Usefulness 153 CHAPTER 17. PAPERS ON PROGNOSIS Preliminary Concepts Description and Analysis More Issues on Validity Importance Usefulness 161 CHAPTER 18. PAPERS ON AETIOLOGY More Issues on Validity Importance Usefulness 166 CHAPTER 19. REVIEW PAPERS Notes on Reviews Meta-analysis Appraisal Validity Importance Usefulness 177

11 xiv CHAPTER 20. PAPERS ON GUIDELINES 1. Notes on Guidance 2. Validity 3. Importance 4. Usefulness CHAPTER 21. PAPERS ON AUDIT 1. Notes on Audit 2. Validity 3. Importance 4. Usefulness Critical Appraisal of Medical Literature CHAPTER 22. PAPERS ON QUALITATIVE RESEARCH 1. Notes on Qualitative Research 2. Appraisal CHAPTER 23. SUMMARY 1. General Points 2. Specific Points PART II! CHAPTER 24. THE z- TEST AND THE t-test 1. The z-test 2. The (-test 3. Examples 4. Summary CHAPTER 25. ANALYSIS OF VARIANCE 1. General Points 2. One-way Anova 3. Two-way Anova 4. Three-way Anova 5. Multivariable Analysis of Variance 6. Related Measures Anova and Manova 7. Example 8. Summary CHAPTER 26. NON-PARAMETRIC TESTS I. The Chi-square Test 2. Fisher's Exact Probability Test 3. The Mann-Whitney Test For Two Independent Samples 4. The Wilcoxon Test 5. Kruskal-Wallis H-test For More Than Two Samples 6. Examples 7. Summary CHAPTER 27. CORRELATION, REGRESSION AND PREDICTION I. Correlation 2. Regression and Prediction 3. Multiple Prediction

12 Contents 4. Example Summary 248 CHAPTER 28. MULTIV ARIABLE ANALYSIS Principal Component Analysis and Factor Analysis Discriminant Analysis Examples Summary 262 CHAPTER 29. OTHER STATISTICAL TECHNIQUES Analysis of Covariance (Ancova) Logistic Regression Analysis Examples Summary 278 REFERENCES 281 INDEX 285 xv