Using R for Statistics. Sarah Stowell

Transcription

1 Using R for Statistics Sarah Stowell

2 Using R for Statistics Copyright 2014 by Sarah Stowell This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. ISBN-13 (pbk): ISBN-13 (electronic): Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Publisher: Heinz Weinheimer Lead Editor: Steve Anglin Development Editor: Matthew Moodie and Chris Nelson Technical Reviewers: Myron Hlynka, Wen Sui Liu, and Larry Pace Editorial Board: Steve Anglin, Mark Beckner, Ewan Buckingham, Gary Cornell, Louise Corrigan, Jim DeWolf, Jonathan Gennick, Jonathan Hassell, Robert Hutchinson, Michelle Lowman, James Markham, Matthew Moodie, Jeff Olson, Jeffrey Pepper, Douglas Pundick, Ben Renow-Clarke, Dominic Shakeshaft, Gwenan Spearing, Matt Wade, Steve Weiss Coordinating Editor: Anamika Panchoo Copy Editor: Laura Lawrie Compositor: SPi Global Indexer: SPi Global Artist: SPi Global Cover Designer: Anna Ishchenko Distributed to the book trade worldwide by Springer Science+Business Media New York, 233 Spring Street, 6th Floor, New York, NY Phone SPRINGER, fax (201) , orders-ny@springer-sbm.com, or visit Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation. For information on translations, please rights@apress.com, or visit Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use. ebook versions and licenses are also available for most titles. For more information, reference our Special Bulk Sales ebook Licensing web page at Any source code or other supplementary material referenced by the author in this text is available to readers at For detailed information about how to locate your book s source code, go to

3 Contents at a Glance About the Author...xiii About the Technical Reviewer... xv Acknowledgments... xvii Introduction... xix Chapter 1: R Fundamentals...1 Chapter 2: Working with Data Files...19 Chapter 3: Preparing and Manipulating Your Data...29 Chapter 4: Combining and Restructuring Datasets...47 Chapter 5: Summary Statistics for Continuous Variables...59 Chapter 6: Tabular Data...73 Chapter 7: Probability Distributions...87 Chapter 8: Creating Plots...99 Chapter 9: Customizing Your Plots Chapter 10: Hypothesis Testing Chapter 11: Regression and General Linear Models Appendix A: Add-On Packages Appendix B: Basic Programming with R Appendix C: Datasets Index iii

4

5 Contents About the Author...xiii About the Technical Reviewer... xv Acknowledgments... xvii Introduction... xix Chapter 1: R Fundamentals...1 Downloading and Installing R...1 Getting Orientated...2 The R Console and Command Prompt...3 Functions...4 Objects...6 Simple Objects... 6 Vectors... 7 Data Frames... 9 The Data Editor...13 Workspaces...14 Error Messages...15 Script Files...16 Summary...18 v

6 Contents Chapter 2: Working with Data Files...19 Entering Data Directly...19 Importing Plain Text Files...20 CSV and Tab-Delimited Files DIF Files Other Plain Text Files Importing Excel Files...24 Importing Files from Other Software...25 Using Relative File Paths...26 Exporting Datasets...27 Summary...28 Chapter 3: Preparing and Manipulating Your Data...29 Variables...30 Rearranging and Removing Variables Renaming Variables Variable Classes Calculating New Numeric Variables...33 Dividing a Continuous Variable into Categories...34 Working with Factor Variables...36 Manipulating Character Variables...37 Concatenating Character Strings Extracting a Substring Searching a Character Variable Working with Dates and Times...39 Adding and Removing Observations...41 Adding New Observations Removing Specific Observations Removing Duplicate Observations vi

7 Contents Selecting a Subset of the Data...42 Selecting a Subset According to Selection Criteria Selecting a Random Sample from a Dataset Sorting a Dataset...45 Summary...45 Chapter 4: Combining and Restructuring Datasets...47 Appending Rows...47 Appending Columns...49 Merging Datasets by Common Variables...50 Stacking and Unstacking a Dataset...53 Stacking Data Unstacking Data Reshaping a Dataset...56 Summary...57 Chapter 5: Summary Statistics for Continuous Variables...59 Univariate Statistics...59 Statistics by Group...61 Measures of Association...63 Covariance Pearson s Correlation Coefficient Spearman s Rank Correlation Coefficient Hypothesis Test of Correlation...66 Comparing a Sample with a Specified Distribution...67 Shapiro-Wilk Test Kolmogorov-Smirnov Test Confidence Intervals and Prediction Intervals...70 Summary...71 vii

8 Contents Chapter 6: Tabular Data...73 Frequency Tables...73 Creating Tables Displaying Tables Creating Tables from Count Data Creating a Table Directly Chi-Square Goodness-of-Fit Test...79 Tests of Association Between Categorical Variables...80 Chi-Square Test of Association Fisher s Exact Test Proportions test...84 Summary...86 Chapter 7: Probability Distributions...87 Probability Distributions in R...87 Probability Density Functions and Probability Mass Functions...89 Finding Probabilities...90 Finding Quantiles...93 Generating Random Numbers...95 Summary...97 Chapter 8: Creating Plots...99 Simple Plots...99 Histograms Normal Probability Plots Stem-and-Leaf Plots Bar Charts Pie Charts Scatter Plots Scatterplot Matrices viii

9 Contents Box Plots Plotting a Function Exporting and Saving Plots Summary Chapter 9: Customizing Your Plots Titles and Labels Axes Colors Plotting Symbols Plotting Lines Shaded Areas Adding Items to Plots Adding Straight Lines Adding a Mathematical Function Curve Adding Labels and Text Adding a Grid Adding Arrows Overlaying Plots Adding a Legend Multiple Plots in the Plotting Area Changing the Default Plot Settings Summary Chapter 10: Hypothesis Testing Student s T-Tests One-Sample T-Test Two-Sample T-Test Paired T-Test Wilcoxon Rank-Sum Test Analysis of Variance ix

10 Contents Kruskal-Wallis Test Multiple Comparison Methods Tukey s HSD Test Other Pairwise T-Tests Pairwise Wilcoxon Rank-Sum Tests Hypothesis Tests for Variance F-Test Bartlett s Test Summary Chapter 11: Regression and General Linear Models Building the Model Simple Linear Regression Multiple Linear Regression Interaction Terms Polynomial Terms Transformations The Intercept Term Including Factor Variables Updating a Model Stepwise Model Selection Procedures Assessing the Fit of the Model Coefficient Estimates Plotting the Line of Best Fit Model Diagnostics Residual Analysis Leverage Cook s Distances Making Predictions Summary x

11 Contents Appendix A: Add-On Packages Viewing a List of Available Add-on Packages Installing and Loading Add-On Packages Windows Users Mac Users Linux Users Appendix B: Basic Programming with R Creating New Functions Conditional Statements Conditions If Statement If/else Statement The switch Function Loops For Loop While Loop Summary Appendix C: Datasets apartments bigcats bottles brains CIAdata1, CIAdata coffeeshop concrete CPIdata customers endangered fiveyearreport xi

12 Contents flights fruit grades people people powerplant pulserates resistance supermarkets vitalsigns WHOdata Index xii

13 About the Author Sarah Stowell is a contract statistician based in the UK, who has worked with Mitsubishi Pharma Europe, MDSL International, and GlaxoSmithKline previously. She holds a Master of Science degree in Statistics. xiii

14

15 About the Technical Reviewer Dr. Larry Pace is a statistics author and educator as well as a consultant. He lives in the upstate area of South Carolina in the town of Anderson. He is a professor of statistics, mathematics, psychology, management, and leadership. He has programmed in a variety of languages and scripting languages including R, Visual Basic, JavaScript, C##, PHP, APL, and, in a long-ago world, Fortran IV. He writes books and tutorials on statistics, computers, and technology. He has also published many academic papers, and made dozens of presentations and lectures. He has consulted with Compaq Computers, AT&T, Xerox Corporation, the U.S. Navy, and International Paper. He has taught at Keiser University, Argosy University, Capella University, Ashford University, Anderson University (where he was the chair of the behavioral sciences department), Clemson University, Louisiana Tech University, LSU in Shreveport, the University of Tennessee, Cornell University, Rochester Institute of Technology, Rensselaer Polytechnic Institute, and the University of Georgia. xv

16

17 Acknowledgments First, I would like to thank the Apress team, in particular: Lead Editor Steve Anglin, for getting me on board and giving me the chance to work with Apress; Coordinating Editors Anamika Panchoo and Mark Powers for keeping me on track; Development Editor Chris Nelson for teaching me a lot about writing; Technical Editor Larry Pace for making many valuable suggestions to improve the quality of the book; and the many others whom I have not met but I can see have done a great job helping to create the finished product. I would also like to thank to Andrés Barnett, James Sedgwick, and Therese Stukel for providing data for the examples, and my husband Timothy Baldock and friends Jemma-Kay Johnstone, Christopher Gilmour, Nina Farrell, Chris Brown, Artur Kyral, and Eddie Chung, who have all helped with the project in its early stages. xvii

18

19 Introduction Welcome to Using R for Statistics. This book was written for anyone who wants to use R to analyze data and create statistical plots. It is suitable for those with little or no experience with R, and aims to get you up and running quickly without having to learn all the details of programming. About R R is a statistical analysis and graphics environment and also a programming language. It is command-driven and very similar to the commercially produced S-Plus software. R is known for its professional-looking graphics, which allow complete customization. R is open-source software and free to install under the GNU general public license. It is written and maintained by a group of volunteers known as the R core team. The base software is supplemented by over 5,000 add-on packages developed by R users all over the world, many of whom belong to the academic community. These packages cover a broad range of statistical techniques including some of the most recently developed and niche purpose. Anyone can contribute add-on packages, which are checked for quality before they are added to the collection. At the time of writing, the current version of R is What You Will Learn This book is designed to give straightforward, practical guidance for performing popular statistical methods in R. The programming aspect of R is explored only briefly. After reading this book you will be able to: navigate the R system enter and import data manipulate datasets calculate summary statistics create statistical plots and customize their appearance perform hypothesis tests such as the t-test and analysis of variance build regression models access additional functionality with the use of add-on packages create your own functions xix

20 Introduction Knowledge Assumed Although this book does include some reminders about statistics methods and examples demonstrating their use, it is not intended to teach statistics. Therefore, you will require some previous knowledge. You should be able to select the most appropriate statistical method for your purpose and interpret the results. You should also be familiar with common statistical terms and concepts. If you are unsure about any of the methods that you are using, I recommend that you use this book in conjunction with a more detailed book on statistics. No prior knowledge of R or of programming is assumed, making this book ideal if you are more accustomed to working with point-and-click style packages. Only general computer skills and a familiarity with your operating system are required. Conventions Used in This Book This book uses the following typographical conventions: Fixed width font is used to distinguish all R commands and output from the main text. Normal fixed width font is used for built-in R function names, argument names, syntax, specific dataset and variable names, and any other parts of the commands that can be copied verbatim. Slanted fixed width font is used for generic dataset and variable names and any other parts of the commands that should be replaced with the user s own values. Often it has not been possible to fit a whole command into the width of the page. In these cases, the command is continued on the following line and indented. Where you see this, the command should still be entered into the console on a single line. Text boxes, which are separate from the main text, contain reminders of statistical theory or methods. Practical examples are presented in separate numbered sections. Datasets Used in This Book A large number of example datasets are included with R, and these are available to use as soon as you open the software. This book makes use of several of these datasets for demonstration purposes. There are also a number of additional datasets used throughout the book, details of which are given in the Appendix C. They are available to download at Contact the Author If you have any suggestions or feedback, I would love to hear from you. You can me at s.stowell@instantr.com. xx