Chapter 2: Descriptive and Graphical Statistics

Transcription

1 Chapter 2: Descriptive and Graphical Statistics Section 2.1: Location Measures Cathy Poliak, Ph.D. Office: Fleming 11c Department of Mathematics University of Houston Lecture 5 - Math 3339 Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

2 Outline 1 Describing Distributions by Graphs 2 Numerical Descriptions 3 Mean, Median and Mode 4 Measurements of Spread 5 Percentiles 6 Quartiles 7 The 1.5IQR Rule Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

3 A Data Set: Course Grades From Previous Semesters Student Score Grade Tests Quiz HW Opt-out Session A yes Sp B yes Sp F no Sp A yes Sp D no Sp A yes Sp A no Sp B yes Sp A yes Sp F no Sp A yes Sp F no Sp F no Sp F no Sp F no Sp A yes Sp B yes Sp B yes Sp D no Fal B yes Fal D no Fal C no Fal F no Fal A no Fal F no Fal B no Fal B no Fal C no Sum B yes Sum A no Sum16 Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

4 Distributions When observing a data set, one of the first things we want to know is how each variable is distributed. The distribution of a variable tells us what values it takes and how often it takes these values based on the individuals. The distribution of a variable can be shown through tables, graphs, and numerical summaries. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

5 Describing distributions An initial view of the distribution and the characteristics can be shown through the graphs. Then we use numerical descriptions to get a better understanding of the distributions characteristics. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

6 Distributions for categorical variables Lists the categories and gives either the count or the percent of cases that fall in each category. One way is a frequency table that displays the different categories then the count or percent of cases that fall in each category. Then we look at the graphs (bar or pie) to determine the distribution of a categorical variable. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

7 Frequency Tables Oup-out Percent Yes 40% No 60% Grade Percent A 30% B 26.67% C 6.67% D 10% F 26.67% Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

8 Describing Data By Graphs Graphs are an easy and quick way to describe the data. Types of graphs that we use depends on the type of data that we have. Graphs for categorical variables. Bar graphs: Each individual bar represents a category and the height of each of the bars are either represented by the count or percent. Pie charts: Helps us see what part of the whole each group forms. Graphs for quantitative variables. Dotplot Stemplot Histogram Boxplot Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

9 Bar Graph of Letter Grades A B C D F Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

10 Pie Chart of Letter Grades A B C D F Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

11 R code First create a table: counts = table(grades$grade) For bar graph: barplot(counts) For pie chart: pie(counts) Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

12 Describing distributions of quantitative variables The distribution of a variable tells us what values it takes and how often it takes these values. There are four main characteristics to describe a distribution: 1. Shape 2. Center 3. Spread 4. Outliers Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

13 Describing a distribution Shape A distribution is symmetric if the right and left sides of the graph are approximately mirror images of each other. A distribution is skewed to the right if the right side (higher values) of the graph extends much farther out than the left side. A distribution is skewed to the left if the left side (lower values) of the graph extends much farther out than the right side. A distribution is uniform if the graph is at the same height (frequency) from lowest to highest value of the variable. Center - the values with roughly half the observations taking smaller values and half taking larger values. Spread -from the graphs we describe the spread of a distribution by giving smallest and largest values. Outliers - individual values that falls outside the overall pattern. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

14 Dot plots A dot plot is made by putting dots above the values listed on a number line. Price of Basketball Shoes Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

15 Stem - and - leaf plot 1. Separate each observation into a stem consisting of all but the final rightmost digit and a leaf, the final digit. Stems may have as many digits as needed, but each leaf contains only a single digit. 2. Write the stems in a vertical column with the smallest at the top, and draw a vertical line at the right of this column. 3. Write each leaf in the row to the right of its stem, in increasing order out from the stem. Rcode: stem(dataset name$variable name) Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

16 Stem-and-leaf Plot This is the number of wins out of the 2015 baseball season that each pitcher won. > stem(era$wins) The decimal point is 1 digit(s) to the right of the Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

17 Stem-and-leaf Plot of ERA > stem(era$era) The decimal point is at the Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

18 Example of Stem-and-leaf Plot > stem(grades$score) The decimal point is 1 digit(s) to the right of the Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

19 Better Plot > stem(grades$score,scale=0.5) The decimal point is 1 digit(s) to the right of the What is the "shape" of this distribution? a) skewed left b) skewed right c) symmetric d) uniform 2. What is the aprroximate center of this distribuiton? a) 50 b) 82 c) 8.5 d) 4 Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

20 Frequency Table of Scores Score Tally Frequency (count) Percent Cathy Poliak, Ph.D. Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

21 Histograms Bar graph for quantitative variables. Values of the variable are grouped together. Bars are touching. The width of the bar represents an interval of values (range of numbers) for that variable. The height of the bar represents the number of cases within that range of values. Cathy Poliak, Ph.D. Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

22 Histogram of Course Score Histogram of Course Scores Frequency Course Scores Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

23 Cumulative Frequency Polygon Plot a point above each upper class boundary at a height equal to the cumulative frequency of the class. Connect the plotted points with line segments. A similar graph can be used with the cumulative percents. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

24 Cumulative Percent Polygon Cumulative Frequency Chart Cumulative Proportion Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

25 Describing Quantitative Variables with Numbers Center - mean, median or mode Spread - range, interquartile range, variance, or standard deviation Location - percentiles or standard scores Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

26 Parameters and Statistics A parameter is a number that describes the population. A parameter is a fixed number, but in practice we usually do not know its value. A statistic is a number that describes a sample. The value of a statistic is known when we have taken a sample, but it can change from The purpose of sampling or experimentation is usually to use statistics to make statements about unknown parameters, this is called statistical inference. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

27 Notation of Parameters and Statistics Name Statistic Parameter mean x µ mu standard deviation s σ sigma correlation r ρ rho regression coefficient b β beta proportion ˆp p Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

28 Example A carload lot of ball bearings has a mean diameter of centimeters. This is within the specifications for acceptance of the lot by the purchaser. The inspector happens to inspect 100 bearings from the lot with a mean diameter of centimeters. This is outside the specified limits, so the lot is mistakenly rejected. Is each of the bold numbers a parameter or a statistic? Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

29 Presidential Approval Rating On January 25, 2017 by Gallup.com, 46% of Americans approved of how Trump is doing as President. Gallup tracks daily the percentage of Americans who approve or disapprove of the job Donald Trump is doing as president. Daily results are based on telephone interviews with approximately 1,500 national adults; Margin of error is ± 3 percentage points. Is this 46% a statistic or parameter? Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

30 Measuring center: The mean Most common measure of center. Arithmetic average. To calculate the mean of a set of observations x 1, x 2,..., x n, add their values and divide by the number of observations n. Denoted: x called x-bar if the data is from a sample, µ, called "mu" if the data is from the entire population. x = x 1 + x x n n = 1 n n i=1 x i µ = x 1 + x x N N = 1 N Where n is the size of the sample and N is the size of the population. n i=1 x i Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

31 Measuring center: The Median The median M is the midpoint of a data set such that half of the observations are smaller and the other half are larger. 1. Arrange all observations in order of size, from smallest to largest. 2. Find the middle value of the arranged observations by counting (n + 1)/2 from the bottom of the list. If the number of observations n is odd, the median M is the the center observation in the ordered list. If the number of observations n is even, the median M is the mean of the two center observation in the ordered list. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

32 Measuring Center: The Mode The mode of a data set is the numerical value that appears the most frequently. The data set can have one mode, two or more modes. A data set may not have any mode. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

33 Cacluate the mean, median and mode The following is a stem-and-leaf plot of the course scores. Determine, the mean, medain and mode of the course scores. The decimal point is 1 digit(s) to the right of the Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

34 Finding mean and median in R scores=c(8,12,17,22,40,43,49,51,67,68,68,72,75,80,81, 83,84,84,84,85,87,90,91,92,93,98,101,101,101,104) mean(scores) [1] median(scores) [1] 82 Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

35 Example: Test Scores The test scores of a class of 20 students have a mean of 71.6 and the test scores of another class of 14 students have a mean of Find the mean of the combined group. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

36 Example The following are ages of automobiles Determine the mean, median and mode of this set. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

37 Mean vs. Median If the mean and the median are both numbers that describe the center of the values then why do we have different values? If the data has values that are outliers values that are beyond the range of the others, the mean is going toward these outliers. Cathy Poliak, Ph.D. Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

38 Mean vs. Median x(score) = 71.1 and M(score) = 82.3 If the mean and the median are both numbers that describe the center of the values then why do we have different values? If the data has values that are outliers values that are beyond the range of the others, the mean is going toward these outliers. The median is resistant to extreme values (outliers) in the data set. The mean is NOT robust against extreme values. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

39 Basketball Team Cathy Poliak, Ph.D. Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

40 Average Test Scores? What is the mean and median for each of these sections test scores? Section A Section B Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

41 Types of Measurements for the Spread Range Percentiles Quartiles IQR; Interquartile range Variance Standard deviation Coefficient of Variation Cathy Poliak, Ph.D. Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

42 The Range The range is the difference between the highest and lowest values. Section A: Range = = 12 Section B: Range = = 58 Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

43 Percentiles The pth percentile of data is the value such that p percent of the observations fall at or below it. The use of percentiles to report spread when the median is our measure of center. If you are looking for the measurement that has a desired percentile rank, the 100P th percentile, is the measurement with rank (or position in the list) of np + 0.5, where n represents the number of data values in the sample. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

44 The 90th percentile of Section A test scores 1. Arrange the scores in order from lowest to highest n = 10, P = 0.90, so the 90 th percentile for this list is at np = 10(0.9) = 9.5, the mean of the 9th and 10th place values. 3. The 90th percentile is = 77 Find the 35th percentile. Find the 75th percentile. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

45 Determine the 25th percentile of the Course Scores Another way to determine percentiles is using the cumulative frequency polygon to estimate percentiles. Cumulative Frequency Chart Cumulative Proportion Scores Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

46 Determining Percentiles Suppose you know the position (order) of a value and want to know what percentile it is ranked at. If you have n data measurements, x i represents the 100(i 0.5)/n th percentile. Example: Determine the percentile of the 4 th order statistic for a sample size of n = 15. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

47 Examples of percentiles Suppose you want to know what percentile you are in a certain class. You know there are 200 students in this class and that 20 of the students have scores above you. What is your percentile? Suppose your percentile came out to be 90th percentile, how many students scored the same as or below you? What about at the 50th percentile? Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

48 The Quartiles The first quartile is 25th percentile, Q 1. The second quartile is the median and the 50th percentile, Q 2. The third quartile is the 75th percentile, Q 3. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

49 Determining Q 1 for Basketball Shoe Prices Arrange in order n = Q 1 : P = 0.25 np = 15(0.25) = Since we do not get an integer, we find the mean of the 4th and 5th element in the ordered dataset. Q 1 = = 130. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

50 Determine Q 2 for Basketball Shoe Prices Arrange in order n = Q 2 : P = 0.5 np = 15(0.5) = 8. So Q 2 is the 8th element of the ordered data. Q 2 = 150. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

51 Determine Q 3 for Basketball Shoe Prices Arrange in order n = Q 3 : P = 0.75 np = 15(0.75) = Again since we did not get and integer, the third quartile is the mean of the 11th and 12th elements in the ordred data. Q 3 = = 215. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

52 R-code for finding Q 1, Q 2, & Q 3 The values: Minimum, Q 1, Median (Q 2 ), Q 3, and Maximum are called the Five Number Summary > shoeprice=c(100,110,120,120,140,140,140,150, 185,185,215,215,250,250,290) > fivenum(shoeprice) [1] Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

53 Interquartile Range Interquartile range, IQR, is the difference between Q 3 and Q 1 IQR = Q 3 Q 1 Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

54 Example Twelve babies spoke for the first time at the following ages (in months): Find Q 1, Q 2, Q 3, the range and the IQR. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

55 Detecting Outliers: 1.5IQR Rule An outlier is an observation that is "distant" from the rest of the data. Outliers can occur by chance or by measurement errors. Any point that falls outside the interval calculated by Q 1 1.5(IQR) and Q (IQR) is considered an outlier. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

56 Outliers for Basketball Shoe Prices? Recall: Q 1 = 130, Q 3 = 215, So IQR = = 85. Q 1 1.5(IQR) = (85) = 2.5 Q (IQR) = (85) = Any price that is below $2.50 or above $ is considered an outlier. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

57 Outliers? The following is information from 91 pairs of basketball shoes: > fivenum(shoes$price) [1] The highest four numbers in the dataset is..., 170, 225, 250, 250. Are there any prices that are considered an outlier? Cathy Poliak, Ph.D. Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

58 Example of Outliers Twelve babies spoke for the first time at the following ages (in months): Using the 1.5 IQR rule, give the boundaries of the outliers. Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

59 A Graph of the Five Number Summary: Boxplot A central box spans the quartiles. A line inside the box marks the median. Lines extend from the box out to the smallest and largest observations. Asterisks represents any values that are considered to be outliers. Boxplots are most useful for side-by-side comparison of several distributions. Rcode: boxplot(dataset name$variable name) Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

60 Boxplot of Prices boxplot(shoes$price,horizontal = T) Cathy Poliak, Ph.D. cathy@math.uh.edu Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

61 Boxplot of Course Scores Cathy Poliak, Ph.D. Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

62 Boxplot of Course Scores by Session Fal15 Sp16 Sum boxplot(grades$score~grades$session,horizontal=true) Cathy Poliak, Ph.D. Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63

63 Question about the Graphs Given the first type of plot indicated in each pair, which of the second plots could not always be generated from it? a) dot plot, histogram b) stem and leaf, dot plot c) histogram, stem and leaf d) dot plot, box plot Cathy Poliak, Ph.D. Office: Fleming 11c (Department 2.1 of Mathematics UniversityLecture of Houston 5 - Math ) / 63