Globally Challenged: Are U. S. Students Ready to Compete?

Transcription

1 Globally Challenged: Are U. S. Students Ready to Compete? The latest on each state s international standing in math and reading by Paul E. Peterson Ludger Woessmann Eric A. Hanushek and Carlos X. Lastra-Anadón Prepared under the auspices of Harvard s Program on Education Policy and Governance & Education Next Taubman Center for State and Local Government Harvard Kennedy School Program on Education Policy & Governance PEPG Report No.: August 2011 Websites: hks.harvard.edu/pepg educationnext.org Harvard University EN Education Next

2 Table of Contents Table of Contents Executive Summary... v Introduction... 3 Comparing U.S. Students with Peers in Other Countries... 4 Proficiency in Math... 6 Proficiency in Reading Performance of U.S. Ethnic and Racial Groups White students Students from college-educated families Are the Proficiency Standards the Same for Math as for Reading? What Do These Findings Mean? Appendix: Differences in Math Performance of the High School Classes of 2009 and Performing the Crosswalk Identifying the Class of Critiques of PISA References Biographical Sketches Figures Figure 1 Percentage of students in the Class of 2011 at the proficient level in math in U.S. states and foreign jurisdictions participating in PISA Figure 2 Percentage of students in the Class of 2011 at the proficient level in reading in U.S. states and foreign jurisdictions participating in PISA Figure 3 Percentage of students in the Class of 2011 at the proficient level in math and reading, by race and ethnicity Photography: Cover: Getty Images, Echo P. iv: Charles Vest, AP Images, Paul Sancya P. vii: Getty Images, Jack Hollingsworth P. viii: Getty Images, Wealan Pollard P. 3: President Obama, AP Images, Pablo Martinez P. 5: Vinton Cerf, AP Images, Lai Seng Sin P. 7: Getty Images, Wealan Pollard P. 11: Getty Images, Peter Cade P. 17: Getty Images, Image Source P. 18: Michael Brown, Rich-Joseph Facun, The National newspaper Figure 4 Estimated increase in U.S. annual economic growth from improved mathematics proficiency Figure A.1 Percentage of white students in the Class of 2011 in U.S. states at the proficient level in math and percentage of all students at that level in foreign jurisdictions participating in PISA Figure A.2 Percentage of white students in the Class of 2011 in U.S. states at the proficient level in reading and percentage of all students at that level in foreign jurisdictions participating in PISA ii educationnext.org hks.harvard.edu/pepg

3 Table of Contents Figure A.3 Percentage of students in the Class of 2011 with at least one college-educated parent in U.S. states at the proficient level in math and percentage of all students at that level in foreign jurisdictions participating in PISA Figure A.4 Percentage of students in the Class of 2011 with at least one college-educated parent in U.S. states at the proficient level in reading and percentage of all students at that level in foreign jurisdictions participating in PISA Figure A.5 Percentage of students in the Class of 2011 at the advanced level in math in U.S. states and foreign jurisdictions participa ting in PISA Tables Table 1 Percentages of all students in the Class of 2011 at the proficient level in math per state. Foreign jurisdictions with similar and higher percentages at the proficient level in math in overall student population Table 2 Percentages of all students in the Class of 2011 at the proficient level in reading per state. Foreign jurisdictions with similar and higher percentages at the proficient level in reading in overall student population Table A.1 Percentages of white students in the Class of 2011 at the proficient level in math per state. Foreign jurisdictions with similar and higher percentages at the proficient level in math in overall student population Table A.2 Percentages of white students in the Class of 2011 at the proficient level in reading per state. Foreign jurisdictions with similar and higher percentages at the proficient level in reading in overall student population Table A.3 Percentages of students in the Class of 2011 with at least one college-educated parent at the proficient level in math per state. Foreign jurisdictions with similar and higher percentages at the proficient level in math in overall student population Table A.4 Percentages of students in the Class of 2011 with at least one college-educated parent at the proficient level in reading per state. Foreign jurisdictions with similar and higher percentages at the proficient level in reading in overall student population Table A.5 Percentages of all students in the class of 2011 at the advanced level in math per state. Foreign jurisdictions with similar and higher percentages at the advanced level in math in overall student population GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? iii

4 America faces many challenges but the enemy I fear most is complacency. We are about to be hit by the full force of global competition. If we continue to ignore the obvious task at hand while others beat us at our own game, our children and grandchildren will pay the price. We must now establish a sense of urgency. Charles Vest, Former President Massachusetts Institute of Technology

5 Executive Summary Executive Summary At a time of persistent unemployment, especially among the less skilled, many wonder whether our schools are adequately preparing students for the 21st-century global economy. This is the second study of student achievement in global perspective prepared under the auspices of Harvard s Program on Education Policy and Governance (PEPG). In the 2010 PEPG report, U.S. Math Performance in Global Perspective, the focus was on the percentage of U.S. public and private school students performing at the advanced level in mathematics. 1 The current study continues this work by reporting the percentage of public and private school students identified as at or above the proficient level (a considerably lower standard of performance than the advanced level) in mathematics and reading for the most recent cohort for which data are available, the high-school graduating Class of Proficiency in Mathematics U.S. students in the Class of 2011, with a 32 percent proficiency rate in mathematics, came in 32nd among the nations that participated in PISA. Although performance levels among the countries ranked 23rd to 31st are not significantly different from that of the United States, 22 countries do significantly outperform the United States in the share of students reaching the proficient level in math. In six countries plus Shanghai and Hong Kong, a majority of students performed at the proficient level, while in the United States less than one-third did. For example, 58 percent of Korean students and 56 percent of Finnish students were proficient. Other countries in which a majority or near majority of students performed at or above the proficient level included Switzerland, Japan, Canada, and the Netherlands. Many other nations also had math proficiency rates well above that of the United States, including Germany (45 percent), Australia (44 percent), and France (39 percent). Shanghai topped the list with a 75 percent math proficiency rate, well over twice the 32 percent rate of the United States. However, Shanghai students are from a prosperous metropolitan area within China, with over three times the GDP per capita of the rest of that country, so their performance is more appropriately compared to Massachusetts and Minnesota, which are similarly favored and are the top performers among the U.S. states. When this comparison is made, Shanghai still performs at a distinctly higher level. Only a little more than half (51 percent) of Massachusetts students are proficient Globally Challenged: Are U. S. Students Ready to Compete? The latest on each state s international standing in math and reading Paul E. Peterson Ludger Woessmann Eric A. Hanushek Carlos X. Lastra-AnadÓn 1. Hanushek, Peterson, and Woessmann (2010). GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? v

6 Executive Summary in math, while Minnesota, the runner-up state, has a math proficiency rate of just 43 percent. Only four additional states Vermont, North Dakota, New Jersey, and Kansas have a math proficiency rate above 40 percent. Some of the country s largest and richest states score below the average for the United States as a whole, including New York (30 percent), Missouri (30 percent), Michigan (29 percent), Florida (27 percent), and California (24 percent). Proficiency in Reading The U.S. proficiency rate in reading, at 31 percent, compares reasonably well to those of most European countries other than Finland. It takes 17th place among the nations of the world, and only the top 10 countries on PISA outperform the United States by a statistically significant amount. In Korea, 47 percent of the students are proficient in reading. Other countries that outrank the United States include Finland (46 percent), Singapore and New Zealand (42 percent), Japan and Canada (41 percent), Australia (38 percent), and Belgium (37 percent). Within the United States, Massachusetts is again the leader, with 43 percent of 8th-grade students performing at the NAEP proficient level in reading. Shanghai students perform at a higher level, however, with 55 percent of young people proficient in reading. Within the United States, Vermont is a close second to its neighbor to the south, with a 42 percent proficiency rate. New Jersey and South Dakota come next, with 39 and 37 percent of the students identified as proficient in reading. Students living in California (about oneeighth of the U. S. school-age population) are statistically tied with their peers in Slovakia and Spain. Data and Approach Increasingly, states, and the federal government itself, have established performance levels that students are asked to reach. A national proficiency standard was set by the board that governs the National Assessment of Educational Progress (NAEP), which is administered by the U.S. Department of Education and generally known as the nation s report card. We provide information on student performance in both reading and mathematics, but our main concern is the relative performance of U.S. students in mathematics. That information is obtained by comparing student performance on NAEP math and reading tests with the performance of students from across the world on similar examinations. If the NAEP exams are the vi educationnext.org hks.harvard.edu/pepg

7 Executive Summary nation s report card, the world s report card is assembled by the Organization for Economic Co-operation and Development (OECD), which administers the Program for International Student Assessment (PISA) to representative samples of 15-year-old students in 65 of the world s school systems. Since the United States participates in the PISA examinations, it is possible to make direct comparisons between the average performance of U.S. students nationwide and that of their peers elsewhere. But because PISA exams do not set proficiency standards in the same way that NAEP exams do, one cannot calculate the percent proficient in the various countries of the world without performing a crosswalk between NAEP and PISA. Once that crosswalk has been performed, it is possible not only to provide estimates of the percentage of students who are proficient in various countries but also to view from an international perspective the performance of students from particular social groups as well as those living in each state. A crosswalk is made possible by the fact that representative (but separate) samples of the high-school graduating Class of 2011 took both the NAEP and PISA math and reading examinations. NAEP tests were taken in 2007 when the Class of 2011 was in 8th grade and PISA tested 15-year-olds in 2009, most of whom are members of the Class of Given that NAEP identified 32 percent of U.S. 8th-grade students as proficient in math, the PISA equivalent is estimated by calculating the minimum score reached by the top-performing 32 percent of U.S. students participating in the 2009 PISA test. U.S. students in the Class of 2011, with a 32 percent proficiency rate in mathematics, came in 32nd among the nations that participated in PISA. Performance of U.S. Ethnic and Racial Groups The percentage proficient in the United States varies considerably across students from different racial and ethnic backgrounds. While 42 percent of white students were identified as proficient in math, only 11 percent of African American students, 15 percent of Hispanic students, and 16 percent of Native Americans were so identified. Fifty percent of students with an ethnic background from Asia and the Pacific Islands, however, were proficient in math. In reading, 40 percent of white students and 41 percent of those from Asia and the Pacific Islands were identified as proficient. Only 13 percent of African American students, 5 percent of Hispanic students, and 18 percent of Native American students were so identified. Given the disparate performance among students from various cultural backgrounds, it may be worth inquiring as to whether differences between the United States and other countries are attributable to the substantial minority population within the United States. To examine that question, we compare GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? vii

8 Executive Summary While the 42 percent math proficiency rate for U.S. white students is much higher than the averages for students from African America and Hispanic backgrounds, U.S. white students are still surpassed by all students in 16 other countries. 2. Quoted in the STEM Education Coalition s website Accessed June 13, U.S. white students to all students in other countries. We do this not because we think this is the right comparison, but simply to consider the oft-expressed claim that comparisons do not take into account the fact that the United States is a much more diverse society than many of the high-performing countries. While the 42 percent math proficiency rate for U.S. white students is much higher than the averages for students from African American and Hispanic backgrounds, U.S. white students are still surpassed by all students in 16 other countries. A better than 25-percentage-point gap exists between the performance of U.S. white students and the percentage of all students deemed proficient in Korea and Finland. White students in the United States trail well behind all students in countries such as Japan, Germany, Belgium, and Canada. In reading, the picture looks better. As we mentioned above, only 40 percent of white students are proficient, but that proficiency rate would place the United States at 9th in the world. What Do These Findings Mean? The United States could enjoy a remarkable increment in its annual GDP growth per capita by enhancing the math proficiency of U.S. students. Increasing the percentage of proficient students to the levels attained in Canada and Korea would increase the annual U.S. growth rate by 0.9 percentage points and 1.3 percentage points, respectively. Since long-term average annual growth rates hover between 2 and 3 percentage points, that increment would lift growth rates by between 30 and 50 percent. When translated into dollar terms, these magnitudes become staggering. If one calculates these percentage increases as national income projections over an 80-year period (providing for a 20-year delay before any school reform is completed and the newly proficient students begin their working careers), a back-of-the-envelope calculation suggests gains of nothing less than $75 trillion over the period. That averages out to around a trillion dollars a year. Even if you tweak these numbers a bit in one direction or another to account for various uncertainties, you reach the same bottom line: Those who say that student math performance does not matter are clearly wrong. Charles Vest, former president of the Massachusetts Institute of Technology, has warned, America faces many challenges...but the enemy I fear most is complacency. We are about to be hit by the full force of global competition. If we continue to ignore the obvious task at hand while others beat us at our own game, our children and grandchildren will pay the price. We must now establish a sense of urgency. 2 viii educationnext.org hks.harvard.edu/pepg

9 Program on Education Policy & Governance Harvard University EN Education Next

10 Sixty-five countries participated in the math and reading examinations administered by the Program for International Student Assessment (PISA). 2 educationnext.org hks.harvard.edu/pepg

11 Introduction Globally Challenged: Are U.S. Students Ready to Compete? The latest on each state s international standing in math and reading By Paul E. Peterson, Ludger Woessmann, Eric A. Hanushek, and Carlos X. Lastra-Anadón At a time of persistent unemployment, especially among the less skilled, many wonder whether our schools are adequately preparing students for the 21st-century global economy. Despite high unemployment rates, firms are experiencing shortages of educated workers, outsourcing professional-level work to workers abroad, and competing for the limited number of employment visas set aside for highly skilled immigrants. As President Barack Obama said in his 2011 State of the Union address, We know what it takes to compete for the jobs and industries of our time. We need to out-innovate, out-educate, and out-build the rest of the world. 1 The challenge is particularly great in math, science, and engineering. According to Internet entrepreneur Vinton Cerf, America simply is not producing enough of our own innovators, and the cause is twofold a deteriorating K 12 education system and a national culture that does not emphasize the importance of education and the value of engineering and science. 2 To address the issue, the Science, Technology, Engineering, and Math (STEM) Education Coalition was formed in 2006 to raise awareness in Congress, the Administration, and other organizations about the critical role that STEM education plays in enabling the U.S. to remain the economic and technological leader of the global marketplace. 3 Tales of shortages of educated talent appear regularly in the media. According to a CBS News report, 22 percent of American businesses say they are ready to We know what it takes to compete for the jobs and industries of our time. We need to out-innovate, out-educate, and out-build the rest of the world. President Barack Obama 1. Office of the Press Secretary, White House Office, Remarks by the President in the State of the Union Address, January 25, 2011 ( the-press-office/2011/01/25/ remarks-president-state-union-address) 2. Vinton G. Cerf: How to fire up US Innovation, Wall Street Journal, April , ( SB html) 3. STEM Education Coalition website, STEM Ed Coalition Objectives, accessed June 13, 2011 at about-us/ GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 3

12 Comparing U.S. Students with Peers in Other Countries hire if they can find people with the right skills. As one factory owner put it, It s hard to fill these jobs because they require people who are good at math, good with their hands, and willing to work on a factory floor. 4 According to a Bureau of Labor Statistics report, of the 30 occupations projected to grow the most rapidly over the next decade, nearly half are professional jobs that require at least a college degree. 5 On the basis of these projections, McKinsey s Global Institute estimates that over the next few years there will be a gap of nearly 2 million workers with the necessary analytical and technical skills. 6 In this report, we examine the capacity of American schools to meet these needs. 4. Cynthia Bowers, CBS News Report: Skilled Labor Shortage Frustrates Employees, August 12, 2010 ( eveningnews/main shtml) 5. Bartsch (2009). 6. McKinsey Global Institute (2011). 7. Hanushek, Peterson, and Woessmann (2010). 8. NAEP has three levels: basic, proficient, and advanced. We report here the rates of those who are at or above the proficient level. 9. See Appendix for an international comparison of the math performance of the Class of 2011 at the advanced level. Comparing U.S. Students with Peers in Other Countries This is the second study of student achievement in global perspective prepared under the auspices of Harvard s Program on Education Policy and Governance (PEPG). In the 2010 PEPG report, U.S. Math Performance in Global Perspective, the focus was on the percentage of U.S. public and private school students performing at the advanced level in mathematics. 7 Specifically, the study compared the math performance of students in the high-school graduating Class of 2009 with that of their peers around the world. The current study continues this work by reporting the percentage of public and private school students identified as at or above the proficient level (a considerably lower standard of performance than the advanced level) in mathematics and reading for the most recent cohort for which data are available, the high-school graduating Class of Just as it is critical that the United States produce a segment of students who perform at the very highest level, so is it essential that a much larger portion of the next generation be proficient enough in math and reading to perform effectively in an economy that requires ever-increasing technical skill. 9 At one time it was left to teachers and administrators to decide exactly what level of math proficiency should be expected of students. But, increasingly, states, and the federal government itself, have established performance levels that students are asked to reach. A national proficiency standard was set by the board that governs the National Assessment of Educational Progress (NAEP), which is administered by the U.S. Department of Education and generally known as the nation s report card. In 2007, just 32 percent of 8th graders in public and private schools in the United States performed at or above the NAEP proficiency standard in mathematics, and 31 percent performed at or above that level in reading. When more than two-thirds of students fail to reach a proficiency bar, it raises serious questions: Are U.S. schools failing to teach their students adequately? 4 educationnext.org hks.harvard.edu/pepg

13 Comparing U.S. Students with Peers in Other Countries Percentages and Scores for Proficient Students In the United States, in 2007, the share of 8th-grade students identified as proficient on the NAEP math examination was percent. The minimum math score on the PISA examination obtained in 2009 by the highest-performing percent of all U.S. students was estimated to be To cover a broad content area while ensuring that testing time does not become excessive, the tests employ matrix sampling. No student takes the entire test, and scores are aggregated across students. For individual student observations, results are thus estimates of performance obtained by averaging five plausible values, as PISA and NAEP administrators recommend. Comparable numbers for the other categories are as follows: Reading proficiency: percent of U.S. students are proficient on the NAEP, which corresponds to a score of on PISA. Advanced math: percent of U.S. students scored at the advanced level on the NAEP, which corresponds to on PISA. Advanced reading: percent of U.S. students scored at the advanced level on the NAEP, which corresponds to on PISA. Or has NAEP set its proficiency bar at a level beyond the normal reach of a student in 8th grade? One way of tackling such questions is to take an international perspective. Are other countries able to lift a higher percentage or even a majority of their students to or above the NAEP proficiency bar? Another approach is to look at differences among states. What percentage of students in each state is performing at a proficient level? How does each state compare to students in other countries? Those are the questions we shall explore in this report. We provide information on student performance in both reading and mathematics, but our main concern is the relative performance of U.S. students in mathematics. This emphasis is based on prior research that has identified numeracy or math skills as primary determinants of advances in a nation s economic productivity. 10 It is also, as we shall see, the subject area in which the United States performs well below many other countries in the industrialized world. That information is obtained by comparing student performance on NAEP math and reading tests with the performance of students from across the world on similar examinations. If the NAEP exams are the nation s report card, the world s report card is assembled by the Organization for Economic Co-operation and Development (OECD), which administers the Program America simply is not producing enough of our own innovators, and the cause is twofold a deteriorating K 12 education system and a national culture that does not emphasize the importance of education and the value of engineering and science. Vinton Cerf, Internet entrepreneur 10. For a discussion of this literature, see Hanushek and Woessmann (2008). In addition, it seems probable that math performances are more precisely calibrated across different languages and cultures than reading performances are. GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 5

14 Proficiency in Math 11. In the 2007 edition of the other largescale international study that compares performance of students across countries, the Trends in Mathematics and Science Study (TIMSS 2007), the United States as well as the individual states of Massachusetts and Minnesota participated. This enables the comparison of those two particular states with other countries performance for TIMSS. See note 19, below. For a further comparison of TIMSS and PISA, see Hanushek, Peterson, and Woessmann (2010), Appendix. TIMSS 2007 surveyed fewer countries in the industrialized world than PISA does, concentrating instead on gathering similar information from among developing countries. The total number of countries surveyed by TIMSS 2007 was 48, while PISA surveyed 65 jurisdictions in TIMSS content has a less applied emphasis than PISA does. Nonetheless, at the country level, scores on PISA and TIMSS are highly correlated (Hanushek and Woessmann, 2008). 12. Development of the Assessment, NAEP: available online at nationsreportcard/mathematics/howdevelop. asp, accessed June 13, for International Student Assessment (PISA) to representative samples of 15-year-old students in 65 of the world s school systems (which, to simplify the presentation, we shall refer to as countries; Hong Kong, Macao, and Shanghai are not independent nations but are nonetheless included in PISA reports). Since its launch in 2000, the PISA test has emerged as the yardstick by which countries measure changes in their performance over time and the level of their performance relative to that of other countries. Since the United States participates in the PISA examinations, it is possible to make direct comparisons between the average performance of U.S. students nationwide and that of their peers elsewhere. But because PISA exams do not set proficiency standards in the same way that NAEP exams do, one cannot calculate the percent proficient in the various countries of the world without performing a crosswalk between NAEP and PISA. Once that crosswalk has been performed, it is possible not only to provide estimates of the percentage of students who are proficient in various countries but also to view from an international perspective the performance of students from particular social groups as well as those living in each state. 11 A crosswalk is made possible by the fact that representative (but separate) samples of the high-school graduating Class of 2011 took both the NAEP and PISA math and reading examinations. NAEP tests were taken in 2007 when the Class of 2011 was in 8th grade and PISA tested 15-year-olds in 2009, most of whom are members of the Class of Given that NAEP identified 32 percent of U.S. 8th-grade students as proficient in math, the PISA equivalent is estimated by calculating the minimum score reached by the top-performing 32 percent of U.S. students participating in the 2009 PISA test. (See methodological sidebar on page 5 for details of these scores and Appendix for further discussion of the crosswalk.) Proficiency in Math According to the National Center for Education Statistics (NCES), which administers NAEP, the determination of proficiency in any given subject at a particular grade level was the result of a comprehensive national process [which took into account] what hundreds of educators, curriculum experts, policymakers, and members of the general public thought the assessment should test. After the completion of the framework, the NAEP [subject] Committee worked with measurement specialists to create the assessment questions and scoring criteria. 12 In other words, NAEP s concept of proficiency is not based on any objective criterion, but instead reflects a consensus on what should be known by students who have reached a certain educational stage. NAEP 6 educationnext.org hks.harvard.edu/pepg

15 Proficiency in Math NAEP Definition of Math Proficiency at the 8th Grade Level and PISA s Definition of Proficiency Level Three Eighth-graders performing at the proficient level should be able to conjecture, defend their ideas, and give supporting examples. They should understand the connections between fractions, percents, decimals, and other mathematical topics such as algebra and functions. Quantity and spatial relationships in problem solving and reasoning should be familiar to them, and they should be able to convey underlying reasoning skills beyond the level of arithmetic. These students should make inferences from data and graphs, apply properties of informal geometry, and accurately use the tools of technology. Students at this level should be able to calculate, evaluate, and communicate results within the domain of statistics and probability. i Roughly comparable is PISA s Level 3 standard, defined as follows: At Level 3 students can execute clearly described procedures, including those that require sequential decisions. They can select and apply simple problem solving strategies. Students at this level can interpret and use representations based on different information sources and reason directly from them. They can develop short communications reporting their interpretations, results and reasoning. ii Sample NAEP Question at 8th Grade Proficiency Level Three tennis balls are to be stacked one on top of another in a cylindrical can. The radius of each tennis ball is 3 centimeters. To the nearest whole centimeter, what should be the minimum height of the can? Explain why you chose the height that you did. Your explanation should include a diagram. If you chose 18 cm from the list of five choices, you are in the company of the 28 percent of U.S. 8th graders from the Class of 2011 who answered correctly. iii The U.S. proficiency rate in reading, at 31 percent, compares reasonably well to those of most European countries other than Finland. Sample PISA Question at Proficiency Level Three Mark (from Sydney, Australia) and Hans (from Berlin, Germany) often communicate with each other using chat on the Internet. They have to log on to the Internet at the same time to be able to chat. To find a suitable time to chat, Mark looked up a chart of world times and found the following: Greenwich 12 Midnight Berlin 1:00 am Sydney 10:00 am At 7:00 pm in Sydney, what time is it in Berlin? The answer is 10 am. iv i. NAEP s definitions of the different levels of math achievement nationsreportcard/mathematics/achieveall.asp. Accessed on June 13, ii. OECD (2009a). iii. Question come from NAEP s online past questions database, nationsreportcard/itmrlsx/ search.aspx?subject=mathematics. Accessed on June 13, iv. Shiel, Perkins, Close, and Oldham (2007). GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 7

16 Proficiency in Math Percentage of students in the class of 2011 at the proficient level in math in U.S. states and foreign Percentage Proficient Shanghai Singapore Hong Kong Korea Finland Taiwan Liechtenstein Switzerland Massachusetts Japan Canada Netherlands Macao Belgium New Zealand Germany Australia Minnesota Vermont Estonia North Dakota Iceland New Jersey Kansas South Dakota France Slovenia Denmark Pennsylvania New Hampshire Austria Montana Virginia Colorado Wisconsin Maryland Slovakia Wyoming Washington Norway Sweden Ohio Luxembourg Iowa Indiana Oregon Connecticut Poland Texas Nebraska North Carolina Hungary Maine Czech Rep Idaho U.K. Portugal 13. NAEP s definitions of the different levels of achievement nationsreportcard/mathematics/achieveall. asp. Accessed on June 13, OECD (2009a). says that 8th graders, if proficient, understand the connections between fractions, percents, decimals, and other mathematical topics such as algebra and functions. 13 PISA does not set a proficiency standard. Instead, it sets different levels of performance, ranging from one (the lowest) to six (the highest). A student who is at the proficiency level in math set by NAEP performs moderately above level three on the PISA, which includes students who can execute clearly described procedures, including those that require sequential decisions. They can select and apply simple problem-solving strategies. 14 (See sidebar for a detailed statement of the 8th-grade proficiency standard and sample questions from PISA and NAEP that proficient students are expected to pass.) Given the above definition of math proficiency, U.S. students in the Class of 2011, with a 32 percent proficiency rate, came in 32nd among the nations that participated in PISA. Although performance levels among the countries ranked 23rd to 31st are not significantly different from that of the United States, 22 countries do significantly outperform the United States in the share of students reaching the proficient level in math. In six countries plus Shanghai and Hong Kong, a majority of students performed at the proficient level, while in the United States less than one-third did. For example, 58 percent of Korean students and 56 percent of Finnish students were proficient. Other countries in which a majority or near majority of students performed at or above the proficient 8 educationnext.org hks.harvard.edu/pepg

17 Proficiency in Math jurisdictions participating in PISA (Figure 1) U.S. average Utah Alaska U.S. Ireland South Carolina Delaware Italy Spain Illinois New York Missouri Michigan Rhode Island Latvia Florida Lithuania Kentucky Arizona Georgia Arkansas California Greece Tennessee Nevada Dubai Russia Israel Croatia Oklahoma Hawaii Louisiana West Virginia Turkey Alabama New Mexico Serbia Bulgaria Mississippi Uruguay Trinidad and Tobago Romania Chile Thailand District of Columbia Mexico Qatar Kazakhstan Argentina Azerbaijan Montenegro Brazil Albania Jordan Peru Colombia Panama Tunisia Indonesia Kyrgyzstan level included Switzerland, Japan, Canada, and the Netherlands. Many other nations also had math proficiency rates well above that of the United States, including Germany (45 percent), Australia (44 percent), and France (39 percent). Figure 1 presents a detailed listing of the scores of all participating countries as well as the performance of individual states within the United States. Shanghai topped the list with a 75 percent math proficiency rate, well over twice the 32 percent rate of the United States. However, Shanghai students are from a prosperous metropolitan area within China, with over three times the GDP per capita of the rest of that country, so their performance is more appropriately compared to Massachusetts and Minnesota, which are similarly favored and are the top performers among the U.S. states. When this comparison is made, Shanghai still performs at a distinctly higher level. Only a little more than half (51 percent) of Massachusetts students are proficient in math, while Minnesota, the runner-up state, has a math proficiency rate of just 43 percent. 15 Only four additional states Vermont, North Dakota, New Jersey, and Kansas have a math proficiency rate above 40 percent. Some of the country s largest and richest states score below the average for the United States as a whole, including New York (30 percent), Missouri (30 percent), Michigan (29 percent), Florida (27 percent), and California (24 percent). (See Table 1 for a comparison of each state with performances abroad.) 15. Our results are qualitatively similar to those reported by TIMSS 2007, which tested a representative sample of students in Massachusetts and Minnesota. Five countries that had higher average scores than Massachusetts on TIMSS 2007 also took the PISA test. Four of those countries Taiwan, Korea, Singapore, and Hong Kong are identified in Figure 1 of this report as outperforming these states on PISA. Japan also outperformed Massachusetts on TIMSS, but we found Japan s performance to be statistically indistinguishable from the Massachusetts one. Minnesota (whose performance was consistently lower than that of Massachusetts) trailed all five of the above-named countries on both tests, but it outperformed Australia, Sweden and Norway on TIMSS 2007, even though we identified it as not having done that well. In sum, the Massachusetts and Minnesota performances reported here resemble those reported by TIMSS, though Minnesota students seem to have done modestly better on TIMSS than reported here, while the reverse is true for Massachusetts students (Mullis, Martin and Foy, 2008, p. 38). GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 9

18 Proficiency in Math Table 1 Percentages of all students in the class of 2011 at the proficient level in math per state. Foreign jurisdictions with similar and higher percentages at the proficient level in math in overall student population. Percent Significantly Countries with similar percentages State proficient outperformed by* of proficient students 1 Massachusetts Canada Japan Netherlands New Zealand Switzerland 2 Minnesota Australia Belgium France Germany Netherlands 3 Vermont Australia Denmark Estonia France Germany 4 North Dakota Denmark Estonia France Iceland 5 New Jersey Australia Austria Denmark France Germany 6 Kansas Austria Denmark Estonia France Slovenia 7 South Dakota Austria Denmark France Hungary Sweden 8 Pennsylvania Austria Denmark France Hungary Sweden 9 New Hampshire Austria Denmark France Hungary Sweden 10 Montana Austria France Hungary Poland Sweden 11 Virginia Czech Rep France Hungary Poland Sweden 12 Colorado Austria France Hungary Poland Sweden 13 Wisconsin Czech Rep France Hungary Poland Sweden 14 Maryland Czech Rep France Hungary Poland U.K. 15 Wyoming Czech Rep France Poland Portugal U.K. 16 Washington Czech Rep France Hungary Poland U.K. 17 Ohio Czech Rep France Poland Portugal U.K. 18 Iowa Czech Rep France Poland Portugal U.K. 19 Indiana Czech Rep France Poland Portugal U.K. 20 Oregon Czech Rep Hungary Poland Portugal U.K. 21 Connecticut France Poland Portugal Spain U.K. 22 Texas Czech Rep Hungary Poland Portugal U.K. 23 Nebraska Czech Rep Hungary Poland Portugal U.K. 24 North Carolina Czech Rep Hungary Poland Portugal U.K. 25 Maine Czech Rep Hungary Poland Portugal U.K. 26 Idaho Czech Rep Hungary Poland Portugal U.K. 27 Utah Italy Poland Portugal Spain U.K. 28 Alaska Italy Poland Portugal Spain U.K. United States Italy Latvia Poland Spain U.K. 29 South Carolina Italy Poland Portugal Spain U.K. 30 Delaware Hungary Italy Portugal Spain U.K. 31 Illinois Czech Rep Italy Portugal Spain U.K. 32 New York Hungary Italy Portugal Spain U.K. 33 Missouri Hungary Italy Portugal Spain U.K. 34 Michigan Ireland Italy Lithuania Portugal Spain 35 Rhode Island Latvia Lithuania 36 Florida Greece Latvia Lithuania 37 Kentucky Latvia Lithuania 38 Arizona Greece Latvia Lithuania 39 Georgia Greece Latvia Russia 40 Arkansas Croatia Greece Israel Latvia Russia 41 California Greece Russia 42 Tennessee Croatia Greece Israel Russia Turkey 43 Nevada Croatia Greece Israel Russia 44 Oklahoma Croatia Greece Israel Russia Turkey 45 Hawaii Croatia Israel Russia Turkey 46 Louisiana Bulgaria Croatia Israel Serbia Turkey 47 West Virginia Bulgaria Turkey 48 Alabama Bulgaria Croatia Israel Serbia Turkey 49 New Mexico Bulgaria Serbia Turkey 50 Mississippi Bulgaria Trinidad and Tobago Uruguay 51 District of Columbia Kazakhstan Mexico Thailand *Number of countries whose percent proficient was statistically significantly higher Note: List of countries performing at a level that cannot be distinguished statistically are limited to those 5 with the largest population. 10 educationnext.org hks.harvard.edu/pepg

19 Proficiency in Reading Proficiency in Reading According to NAEP, students proficient in reading should be able to make and support inferences about a text, connect parts of a text, and analyze text features. 16 According to PISA, students at a proficiency level four, a level of performance set very close to NAEP s proficient level, should be capable of difficult reading tasks, such as locating embedded information, construing meaning from nuances of languages critically evaluating a text. 17 (See sidebar on page 7 for more specific definitions and sample questions.) As can be seen in Figure 2, the U.S. proficiency rate in reading, at 31 percent, compares reasonably well to those of most European countries other than Finland. It takes 17th place among the nations of the world, and only the top 10 countries on PISA outperform the United States by a statistically significant amount. In Korea, 47 percent of the students are proficient in reading. Other countries that outrank the United States include Finland (46 percent), Singapore and New Zealand (42 percent), Japan and Canada (41 percent), Australia (38 percent), and Belgium (37 percent). Within the United States, Massachusetts is again the leader, with 43 percent of 8th-grade students performing at the NAEP proficient level in reading. Shanghai students perform at a higher level, however, with 55 percent of young people proficient in reading. Within the United States, Vermont is a close second to its neighbor to the south, with 42 percent proficiency. New Jersey and South Dakota come next, with 39 and 37 percent of the students identified as proficient in reading. The District of Columbia, the nation s worst, performs at a level that cannot be distinguished statistically from that of Turkey and Bulgaria. Students living in California (about one-eighth of the U. S. school-age population) are statistically tied with their peers in Slovakia and Spain. See Table 2 for a comparison of how each state fares internationally. Performance of U.S. Ethnic and Racial Groups The percentage proficient in the United States varies considerably across students from different racial and ethnic backgrounds (see Figure 3). While 42 percent of white students were identified as proficient in math, only 11 percent of African American students, 15 percent of Hispanic students, and 16 percent of Native Americans were so identified. Fifty percent of students with an ethnic background from Asia and the Pacific Islands, however, were proficient in math, placing them at a level comparable to all students in Belgium, Canada, and Japan, if lower than that of all students in Korea and Taiwan. In reading, 40 percent of white students and 41 percent of those from Asia and the Pacific Islands were identified as proficient. Only 13 percent of African Increasing the percentage of proficient students to the levels attained in Canada and Korea would increase the annual U.S. growth rate by 0.9 percentage points and 1.3 percentage points, respectively 16. NAEP s definitions of the different levels of reading achievement is available at nces.ed.gov/nationsreportcard/reading/ achieveall.asp. Accessed on June 13, OECD (2000). GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 11

20 White Students Percentage of students in the class of 2011 at the proficient level in reading in U.S. states and foreign Percentage Proficient U.S. average Shanghai Korea Finland Hong Kong Massachusetts Singapore Vermont New Zealand Japan Canada New Jersey Montana Australia New Hampshire Connecticut Belgium Maine South Dakota Minnesota Pennsylvania Ohio Iowa Kansas Nebraska Colorado Netherlands Washington Oregon Virginia Wisconsin Wyoming Maryland France New York North Dakota Switzerland Idaho Germany Norway Iceland U.S. Indiana Missouri Delaware Poland Sweden Utah Liechtenstein Ireland Illinois U.K. Hungary Estonia Michigan Florida North Carolina American students, 5 percent of Hispanic students, and 18 percent of Native American students were so identified. White Students Given the disparate performance among students from various cultural backgrounds, it may be worth inquiring as to whether differences between the United States and other countries are attributable to the substantial minority population within the United States. To examine that question, we compare U.S. white students to all students in other countries. We do this not because we think this is the right comparison, but simply to consider the oft-expressed claim that education problems in the United States are confined to certain segments within the minority community. This is equivalent to the claim that the overall performance of the United States in international comparisons does not take into account the fact that the United States is a much more diverse society than many of the high-performing countries. While the 42 percent of math proficiency rate for U.S. white students is much higher than the averages for students from African American and Hispanic backgrounds, U.S. white students are still surpassed by all students in 16 other countries. A better than 25-percentage-point gap exists between the performance 12 educationnext.org hks.harvard.edu/pepg

21 White Students jurisdictions participating in PISA (Figure 2) Kentucky Texas Taiwan Rhode Island Alaska Italy Denmark Israel Oklahoma Georgia Tennessee Arkansas Portugal Slovenia Greece South Carolina Arizona Luxembourg Austria Czech Rep West Virginia Slovakia Spain Nevada California Alabama Latvia Dubai Macao Croatia Hawaii Louisiana Lithuania Mississippi New Mexico Turkey Russia Bulgaria District of Columbia Trinidad and Tobago Chile Uruguay Serbia Brazil Qatar Argentina Romania Mexico Montenegro Colombia Kazakhstan Panama Jordan Thailand Albania Tunisia Peru Kyrgyzstan Indonesia Azerbaijan of U.S. white students and the percentage of all students deemed proficient in Korea and Finland. White students in the United States trail well behind all students in countries such Japan, Germany, Belgium, and Canada (see Figure A.1). White students in Massachusetts outperform their peers in other states; 58 percent are at or above the proficient level in math. Maryland, New Jersey, and Texas are the other states in which a majority of white students is proficient in math. Given recent school-related political conflicts in Wisconsin, it is of interest that only 42 percent of that state s white students are proficient in math, a rate no better than the national average. In reading, the picture looks better. As we mentioned above, only 40 percent of white students are proficient, but that proficiency rate would place the United States at 9th in the world. This proficiency rate does not differ significantly from that for all students in Canada, Japan, and New Zealand, but white students trail in reading, by a significant margin, all students in Korea, Finland, and Singapore. In no state is a majority of white students proficient, although Massachusetts comes close with a 49 percent rate. The four states with the next highest levels of reading proficiency among white students are New Jersey, Connecticut, Maryland, and Colorado. (See Figure A.2 for the ranking of all the states.) GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 13

22 NAEP Definition of Reading Proficiency at the 8th Grade Level Eighth-grade students performing at the proficient level should be able to provide relevant information and summarize main ideas and themes. They should be able to make and support inferences about a text, connect parts of a text, and analyze text features. Students performing at this level should also be able to fully substantiate judgments about content and presentation of content. Sample NAEP Question What is an acceptable way to place a $1 Bargain Basement ad in this newspaper? 1. Phone in the ad, pay by credit card 2. Phone in the ad, pay by money order 3. Mail the ad, pay by cash 4. Mail the ad, pay by check If you chose answer four, you, along with 31 percent of 8th graders, got the question correct. Question from PISA corresponding to the NAEP proficiency level in reading: R236: New Rules BARGAIN BASEMENT Use this coupon for items over $25 but not more than $100 3DAYS FOR $1 We ll insert your classified ad for 3 consecutive days in the BARGAIN BASEMENT section. To qualify, the advertised item must be over $25 but not more than $100 and each item must be priced. Enclose check or money order with coupon. 1. Print one (1) letter in each space. 2. Allow one (1) space between words. 3. Include punctuation marks within the appropriate letter space. 4. ALL ADS MUST HAVE PRICE AND PHONE NUMBER IN THEM. 5. No businesses, individuals only qualify for this rate. 6. Maximum of 3 orders (9 insertions) per item. FIRST LINE Comparable PISA Question Question: Underline the sentence that explains what the Australians did to help decide how to deal with the frozen embryos belonging to a couple killed in the plane crash. i (Answer underlined in red in text to the right.) i. Cosgrove, Sofroniou, Kelly, and Shiel (2003). EDITORIAL Technology Creates the Need for New Rules Science has a way of getting ahead of law and ethics. That happened dramatically in 1945 on the destructive side of life with the atomic bomb, and is now happening on life s creative side with techniques to overcome human infertility. Most of us rejoiced with the Brown family in England when Louise, the first testtube baby, was born. And we have marvelled at other firsts most recently the births of healthy babies that had once been embryos frozen to await the proper moment of implantation in the mother-to-be. It is about two such frozen embryos in Australia that a storm of legal and ethical questions has arisen. The embryos were destined to be implanted in Elsa Rios, wife of Mario Rios. A previous embryo implant had been unsuccessful, and the Rioses wanted to have another chance at becoming parents. But before they had a second chance to try, the Rioses perished in an airplane crash. What was the Australian hospital to do with the frozen embryos? Could they be implanted in someone else? There were numerous volunteers. Were the embryos somehow entitled to the Rioses substantial estate? Or should the embryos be destroyed? The Rioses, understandably, had made no provision for the embryos future. The Australians set up a commission to study the matter. Last week, the commission made its report. The embryos should be thawed, the panel said, because donation of embryos to someone else would require the consent of 3DAYS SECOND LINE Name Address SPECIAL OFFER the producers, and no such consent had been given. The panel also held that the embryos in their present FOR state had FREE no life or rights Items and must thus be $25 or less could be destroyed. The item commission must be priced. members were conscious of treading on slippery legal and Times. The Times reserves the right to limit quantity of free ads in any given publication. ethical grounds. Therefore, they urged that three months be allowed for public opinion to respond to the commission recommendation. Should there be an overwhelming outcry against destroying the embryos, the commission would reconsider. Couples now enrolling in Sydney s Queen Victoria hospital for in vitro fertilization programmes must specify what should be done with the embryos if something happens to them. This assures that a situation similar to the Rioses won t recur. But what of other complex questions? In France, a woman recently had to go to court to be allowed to bear a child from her deceased husband s frozen sperm. How should such a request be handled? What should be done if a surrogate mother breaks her child-bearing contract and refuses to give up the infant she had promised to bear for someone else? Our society has failed so far to come up with enforceable rules for curbing the destructive potential of atomic power. We are reaping the nightmarish harvest for that failure. The possibilities of misuse of scientists ability to advance or retard procreation are manifold. Ethical and legal boundaries need to be set before we stray too far. Follow the above instructions & mail us this coupon to insert your free ad for 3 consecutive days in the BARGAIN BASEMENT section. The advertised item must be $25 or less and each $1 ads and free ads accepted only on this coupon. $1 ads and free ads will not be accepted by phone. No cancellations or refunds. coupons also available at the CLASSIFIED Counter of The Mail to: The Times Newspaper, BARGAIN BASEMENT, P.O. box 847, Trenton, NJ educationnext.org hks.harvard.edu/pepg

23 Students from College-educated Families Percentage of students in the class of 2011 in the U.S. at the proficient level in math and reading, by race and ethnicity. (Figure 3) 50% Math Reading Percentage Proficient White African -American Hispanic Native American Asia and the Pacific Islands White African -American Hispanic Native American Asia and the Pacific Islands Students from College-educated Families An elite segment of the U.S. population that the NAEP data allow us to isolate consists of students who have at least one parent who has attended college. Given the benefits that accrue to most of those who live in bettereducated families, that segment can be expected to outrank all students in other countries. It may be helpful to think of it as the upper bound of what the U.S. education system has delivered in terms of student performance. Significantly, not even among students from college-educated families can we find a majority of students crossing the proficiency bar in math (see Figure A.3). Only 44 percent of such students did so. In Massachusetts, 61 percent of students from college-educated families are proficient in math. Seven other states have a majority of students from college-educated families performing proficiently in math: Vermont, Minnesota, Kansas, Pennsylvania, Virginia, New Jersey, and Colorado. In reading, 42 percent of U.S. students from college-educated families in the Class of 2011 are proficient. In two states a majority of these students are proficient in reading: Massachusetts with 57 percent and Vermont with 53 percent. Other high-ranking states include New Jersey, Connecticut, Pennsylvania, Oregon, and Ohio. (See Figure A.4 for the ranking of all the states.) GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 15

24 Students from College-educated Families Table 2 Percentages of all students in the class of 2011 at the proficient level in reading per state. Foreign jurisdictions with similar and higher percentages at the proficient level in reading in overall student population. Percent Significantly Countries with similar percentages State proficient outperformed by* of proficient students 1 Massachusetts Canada Finland Japan Korea Singapore 2 Vermont Canada Japan Korea New Zealand Singapore 3 New Jersey Australia Belgium Canada Japan Netherlands 4 Montana Australia Belgium Canada Japan Netherlands 5 New Hampshire Australia Belgium Liechtenstein Netherlands 6 Connecticut Australia Belgium France Japan Netherlands 7 Maine Australia Belgium Liechtenstein Netherlands 8 South Dakota Australia Canada France Japan Netherlands 9 Minnesota Australia Belgium Liechtenstein Netherlands 10 Pennsylvania Australia Belgium France Liechtenstein Netherlands 11 Ohio Australia Belgium France Liechtenstein Netherlands 12 Iowa Australia Belgium France Liechtenstein Netherlands 13 Kansas Belgium France Liechtenstein Netherlands 14 Nebraska Belgium France Liechtenstein Netherlands 15 Colorado Australia Belgium France Germany Netherlands 16 Washington France Germany Netherlands Norway Switzerland 17 Oregon Australia France Germany Poland Switzerland 18 Virginia Belgium France Germany Netherlands Poland 19 Wisconsin France Germany Hungary Netherlands Poland 20 Wyoming France Germany Netherlands Norway Switzerland 21 Maryland France Germany Netherlands Poland Sweden 22 New York France Germany Hungary Netherlands Poland 23 North Dakota France Germany Hungary Netherlands Poland 24 Idaho France Germany Netherlands Poland U.K. United States France Germany Netherlands Poland U.K. 25 Indiana France Germany Netherlands Poland U.K. 26 Missouri France Germany Netherlands Poland U.K. 27 Delaware France Germany Netherlands Poland U.K. 28 Utah France Germany Netherlands Poland U.K. 29 Illinois France Germany Poland Taiwan U.K. 30 Michigan Germany Italy Poland Taiwan U.K. 31 Florida Italy Netherlands Poland Taiwan U.K. 32 North Carolina Israel Italy Netherlands Poland U.K. 33 Kentucky Greece Italy Netherlands Poland U.K. 34 Texas Greece Italy Netherlands Portugal U.K. 35 Rhode Island Greece Hungary Italy Portugal U.K. 36 Alaska Greece Hungary Italy Portugal U.K. 37 Oklahoma Greece Hungary Italy Portugal Slovenia 38 Georgia Greece Hungary Israel Italy Portugal 39 Tennessee Greece Hungary Israel Italy Portugal 40 Arkansas Greece Hungary Israel Italy Portugal 41 South Carolina Greece Hungary Israel Italy Portugal 42 Arizona Denmark Greece Israel Portugal Spain 43 West Virginia Croatia Czech Rep Greece Portugal Spain 44 Nevada Austria Croatia Czech Rep Slovakia Spain 45 California Austria Croatia Czech Rep Slovakia Spain 46 Alabama Austria Croatia Czech Rep Slovakia Spain 47 Hawaii Croatia Latvia Slovakia 48 Louisiana Croatia Latvia Lithuania 49 Mississippi Bulgaria Croatia Lithuania Russia Turkey 50 New Mexico Bulgaria Lithuania Turkey 51 District of Columbia Bulgaria Chile Trinidad and Tobago Turkey *Number of countries whose percent proficient was statistically significantly higher Note: List of countries performing at a level that cannot be distinguished statistically are limited to those 5 with the largest population. 16 educationnext.org hks.harvard.edu/pepg

25 Are the Proficiency Standards the Same for Math as for Reading? Are the Proficiency Standards the Same for Math as for Reading? Has NAEP set a lower proficiency standard in math than in reading? If so, is the math standard too low or the reading bar too high? At first glance it would seem that the standard is set at pretty much the same level. After all, 32 percent of U.S. students are deemed proficient in math and 31 percent are deemed proficient in reading. But that coincidence is quite misleading. When compared to peers abroad, the U.S. Class of 2011 performed respectably in reading, trailing only 10 other nations by a statistically significant amount. Admittedly, the United States trails Korea by 16 percentage points, but it s only 9 percentage points behind Canada. Meanwhile, U.S. performance in math is seriously disappointing. It significantly trails that of 22 countries. Korean performance is 26 percentage points higher than that of the United States, while Canadian performance is 17 percentage points higher. Judged by international standards, the U.S. Class of 2011 was clearly doing worse in math than in reading, despite the fact that NAEP reports similar percentages proficient in the two subjects. A direct comparison of NAEP s proficiency standard with PISA s levels three and four (out of a total of six proficiency levels) also indicates that a lower NAEP bar has been set in math than in reading. To be proficient, one needs to perform at or near the fourth level on PISA s reading exam, but only modestly above the third level on its math exam. From these findings, we infer that the NAEP experts set an 8th-grade math proficiency standard at a level lower than the one set in reading. Perhaps this is an indication that American society as a whole, including the experts who design NAEP standards, set lower expectations for students in math than in reading. If so, it is a sign that low performance in mathematics within the United States may be deeply rooted in the nation s culture. Those who are setting the common core standards under discussion might well take note of this. Of course, it could be argued that the math proficiency standard is correct but the reading standard has been set too high. In no country in the world does a majority of the students reach the NAEP proficiency bar set in 8th-grade reading. NAEP experts set an 8th-grade math proficiency standard at a level lower than the one set in reading. What Do These Findings Mean? Many have concluded that the productivity of the U.S. economy could be greatly enhanced if a higher percentage of U.S. students were proficient in mathematics. As Michael Brown, Nobel Prize winner in medicine, has declared, If America is to maintain our high standard of living, we must continue to innovate. GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 17

26 What Do These Findings Mean? If America is to maintain our high standard of living, we must continue to innovate. Math and science are the engines of innovation. With these engines we can lead the world. Michael Brown, Nobel Prize winner in medicine 18. Quoted at the STEM Education Coalition s website stemedcoalition.org/, Accessed June 13, Hanushek. and Woessmann (2008). Math and science are the engines of innovation. With these engines we can lead the world. 18 But others have argued that the overall past success of the U.S. economy suggests that high-school math performance is not that critical for sustained growth in economic productivity. After all, U.S. students trailed their peers in the very first international survey undertaken nearly 50 years ago. But that is the wrong message to take away. Other factors contributed to the relatively high rate of growth in economic productivity during the last half of the 20th century, including the openness of the country s markets, respect for property rights, low levels of political corruption, and limited intrusion of government into the operations of the marketplace. The United States, moreover, has always benefited from the in-migration of talent from abroad. Furthermore, the United States has historically had far higher levels of educational attainment than other countries, with many more students graduating from high school, continuing on to college, and earning an advanced degree. It appears that in the past the country made up for low quality in elementary and high school by educating students for longer periods of time. As we proceed into the 21st century, none of these factors remain as favorable to the United States. While other countries are lifting restrictions on market operations, the opposite has been occurring within the United States. The U.S. has also placed sharp limits on the numbers of talented workers that can be legally admitted into the country. Our higher education system, though still perceived to be the best in the world, is recruiting an ever-increasing proportion of its faculty and students from outside the country. Meanwhile, educational attainment rates among U.S. citizens now trail the industrial-world average. Even if some of these trends can be reversed, that hardly gainsays the desirability of enhancing the mathematical skills of the U.S. student population, especially at a time when the nation s growth in productivity is badly trailing growth rates in China, India, Brazil, and many smaller Asian countries. Eric Hanushek and Ludger Woessmann have shown elsewhere that student performance on international tests such as those we consider here is closely related to long-term economic growth. 19 Assuming that past trends continue, the country could enjoy a remarkable increment in its annual GDP growth per capita by enhancing the math proficiency of U.S. students. Increasing the percentage of proficient students to the levels attained in Canada and Korea would increase the annual U.S. growth rate by 0.9 percentage points and 1.3 percentage points, respectively (see Figure 4). Since longterm average annual growth rates hover between 2 and 3 percentage points, that increment would lift growth rates by between 30 and 50 percent. 18 educationnext.org hks.harvard.edu/pepg

27 What Do These Findings Mean? Estimated increase in U.S. annual economic growth from improved mathematics proficiency. (Figure 4) 2.5 Shanghai Added annual GDP growth Current U.S. 0 0 U.K. France Germany Canada Korea Finland Increased % of students who are proficient in math Note: Country points indicate the additional % of students proficient in the United States in order to reach the country s level (X-axis) and corresponding additional growth to be expected on reaching that level (Y-axis). When translated into dollar terms, these magnitudes become staggering. If one calculates these percentage increases as national income projections over an 80-year period (providing for a 20-year delay before any school reform is completed and the newly proficient students begin their working careers), a back-of-the-envelope calculation suggests gains of nothing less than $75 trillion over the period. 20 That averages out to around a trillion dollars a year. Even if you tweak these numbers a bit in one direction or another to account for various uncertainties, you reach the same bottom line: Those who say that student math performance does not matter are clearly wrong. Given the integration of the world economy, a global perspective is needed for assessing the performance of U.S. schools, districts, and states. High-school graduates in each and every state compete for jobs with graduates from all over the world. Charles Vest, former president of the at Massachusetts Institute of Technology, has warned, America faces many challenges...but the enemy I fear most is complacency. We are about to be hit by the full force of global competition. If we continue to ignore the obvious task at hand while others beat us at our own game, our children and grandchildren will pay the price. We must now establish a sense of urgency For a thorough explanation of this calculation, see Hanushek and Woessmann (2011). 21. Quoted in the STEM Education Coalition s website stemedcoalition.org/, Accessed June 13, GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 19

28 Differences in the Math Performance of the High School Classes of 2009 and 2011 Appendix Differences in the Math Performance of the High School Classes of 2009 and 2011 Students are identified as advanced by NAEP only if they score well above the proficient level. Seven percent of U.S. students in the Class of 2011 performed at the advanced level in math (See Figure A.5, Table A.5), a gain of 1 percentage point over the 6 percent identified as advanced in the Class of That slight gain is a modest accomplishment, especially given the decline in performance in many other countries. While most changes were small, the percentage of advanced students declined by 2 or more points in the Czech Republic, Austria, Korea, Finland, United Kingdom, Ireland, and Lithuania. Only two of the higher-performing countries, Denmark and Portugal, showed improvement of 2 percentage points or more. Given all these changes, the relative position of the U.S. Class of 2011 improved from the 31st place held by the U. S. Class of 2009 to a tie for 26th place (with Poland, Hungary, Norway, the United Kingdom, Portugal, Italy, and Sweden), despite the inclusion of two new high-scoring PISA participants, Shanghai and Singapore. Within the United States, little change could be observed between the Class of 2009 and the Class of 2011, apart from the astonishing shift upward in the already high-performing state of Massachusetts, where the percentage advanced rose from 11 percent to 15 percent, a gain unequalled by any other state. Minnesota s performance came in second place in both years, but its students performance budged northward by only 0.7 percentage points to 11.5 percent for the Class of In four other states, scores improved by 2 percentage points or more: Vermont, Maine, North Dakota, and Wyoming (which made a 3.0 gain, the largest gain outside of Massachusetts). It is remarkable how concentrated in certain parts of the country these gains are to be found. If teaching to the talented is a skill, the teachers getting better at the task seem to be concentrated in a few states in New England and the northern plains. Indeed, the picture that we see of little change in the relative performance of the United States is one that is consistent with the broader trajectory of the United States in international comparisons, which is at best flat and at worst in slight decline over time. Regardless of whether the United States is actually improving in its performance, it is clear that its relative standing with respect to other developed countries is in the bottom half of the OECD countries. 20 educationnext.org hks.harvard.edu/pepg

29 Performing the Crosswalk At the same time, we have noted above how a number of other countries, most notably Asian countries such as Taiwan, Singapore, and Japan but also others such Finland and Switzerland or Canada, are significantly ahead of the United States. Performing the Crosswalk Our aim is to compare how students in the different states in the United States are doing with respect to their peers internationally. To obtain this information, we perform a crosswalk between NAEP and the Program for International Student Assessment (PISA), which was administered by the Organization for Economic Co-operation and Development OECD, to representative samples of 15-year-old students in 65 of the world s school systems, which, to simplify the presentation, we shall refer to as countries. (Hong Kong, Macao, and Shanghai are not independent nations but nonetheless are included in PISA reports.) The crosswalk is performed by looking at the percentage of U.S. students who reach the proficient level on the NAEP assessment and at the equivalent cutoff score in PISA for that percentage of U.S. students. This gives us the equivalent of the PISA proficiency threshold, allowing us to estimate comparable proficiency rates for all countries and to compare student performance in each of the states within the United States with that of their international peers. Our analysis relies on test-score information from young adults collected by NAEP and PISA. 1 NAEP is a large, nationally representative assessment of student performance that has been administered periodically since the early 1970s to U.S. students in 4th grade and 8th grade, and at the age of seventeen. Since 2001, it has provided achievement data for students in each of the 50 states and a select number of urban school districts. PISA is an internationally standardized assessment of student performance in mathematics, science, and reading established by OECD. It was administered in 2000, 2003, 2006 and 2009 to representative samples of 15-year-olds in all OECD countries as well as in many others. 2 NAEP is governed by the National Assessment Governing Board (NAGB), which consists of 26 educators and other public figures appointed by the U.S. Secretary of Education. In 2007, NAEP tested representative samples of 8th-grade public and private school students in each of the Data for NAEP come from the official website [accessed May 15, 2011], ed.gov/nationsreportcard/. NAEP has also tested periodically a representative sample of students in several other subjects. 2. The OECD, which administers PISA, is an international economic organization encompassing most of the high-income, developed countries of the world. In 2009, it had 30 members; three new members (Chile, Israel, and Slovenia) were added in Sixty-five countries/economies participated in PISA in 2009 (up from 57 in 2006). Data for PISA 2009 come from the PISA microdata ( The PISA assessments build upon earlier international testing, most importantly those of the International Association for the Evaluation of Educational Achievement (IEA) now known as Trends in Mathematics and Science Survey (TIMSS). IEA has conducted assessments since the mid-1960s and is responsible for the TIMSS testing that is discussed below. See Historical PISA scores and those of TIMSS are summarized in Provasnik, Gonzales, and Miller (2009), which also contains references to the original publications for TIMSS. GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 21

30 Identifying the Class of 2011 states, in 10 large public school districts, and in the United States as a whole in math, science, and reading. For each of these jurisdictions, NAEP 2007 calculates the percentage of students who perform at three levels: basic, proficient, and advanced. Using the NAEP and PISA data for the United States as a whole, the crosswalk exercise identifies an estimated PISA score of for math proficiency, as defined by NAEP, and a score of for reading proficiency, as defined by NAEP. With the PISA data, we can obtain an estimate of the percentage of students in those countries above the cutoff, i.e., those who reach the level equivalent to the proficient level in 8th-grade math on NAEP The shares of students who reach the proficient level in 8th-grade math in each U.S. state are taken directly from NAEP It is assumed that both NAEP and PISA tests randomly select questions from a common universe of mathematics knowledge. Given that assumption, it may be further assumed that students who scored similarly on the two exams will have similar math knowledge, i.e., students who scored points or better on the PISA test would have been identified as proficient had they taken the NAEP math test. The scaling of PISA straightforwardly reveals that a score of points is 31 percent of one standard deviation above the average OECD student score on the PISA, indicating that a somehow accomplished group has been found. Some of the calculated differences in performance across countries may simply reflect sampling uncertainty or measurement error. We therefore calculate whether the observed differences among states and countries are statistically significant (at the 5 percent level). The requisite standard errors are computed using the methodology described by the OECD. 3 These standard errors account for both sampling uncertainty (including the two-stage sampling design employed by PISA) and test unreliability (as captured by the five plausible values that represent the underlying probability distribution). NAEP 2007 standard errors are obtained from the NAEP website OECD (2009b). See Chapters accessed May 18, National Center for Education Statistics, Report , Table C-1. Identifying the Class of 2011 This crosswalk identifies the relative performance of the Class of NAEP examinations are given to 8th graders, January through March, when most students are 13 years of age. PISA examinations are given to a random sample of students at the age of 15, the age at which approximately 70 percent of U. S. students are in 10th grade. 5 To track the Class of 2011 we rely upon the 2007 NAEP test and the 2009 PISA test. In comparing the performance of the Class 22 educationnext.org hks.harvard.edu/pepg

31 Critiques of PISA of 2011 on the NAEP and PISA tests at these two different points in time, we assume that no event happened between 8th and 10th grade that significantly altered the performance of American students relative to that of students in other countries. Our previous report relied on a similar crosswalk between the PISA 2006 results and the NAEP 2005, a less preferable cohort match but the best possible given the information available in November With the appearance of the latest PISA wave in December 2010, we were able to match cohorts more closely, as NAEP data was available for 8th graders in 2007 while PISA data was available in 2009 for 15-year-olds who are typically in 10th grade. While we are pleased that the cohort match reported in this report is more precise, the findings from this report are consistent with those presented in the previous report. Inasmuch as there are only small differences from one year to the next in the performance of students in particular countries, small variations in the years used to execute a particular cross-walk do not alter findings materially. Critiques of PISA Questions have been raised as to whether any cross-country comparison using PISA data can be meaningful. Prais and others point out that PISA s testing focuses on real-life circumstances and on students capacity to enter the labor market with core skills, such as literacy and numeracy, more than the testing of any specific curriculum. 7 This inherently favors some countries specific sequencing of items in the curriculum. For example, Germany s introduction of supplementary questions in the PISA math test (that focused more on arithmetic skills) presents a case of a country that thought that the basic PISA math test was not providing an accurate evaluation of its students math skills. But too much emphasis on specific questions ignores the commonality of welldesigned tests of student achievement. The TIMSS and PISA tests have quite different designs, but the performance of countries on TIMSS and PISA are highly correlated and both are strongly correlated with a country s economic performance. (See our previous report for further discussion.) 8 Further, the U.S. National Center for Education Statistics (NCES) is deeply involved in the development of PISA, as it is an active participant in OECD discussions of PISA design; presumably, that office is satisfied that the test is not unfair to U.S. students. Beyond the appropriateness of the specific PISA questions, worries have been voiced about whether excessive focus on such exams can take away from other educational values that arguably cannot be so readily measured 6. Hanushek, Peterson, and Woessmann (2010). 7. Prais (2003). 8. Hanushek, Peterson, and Woessmann (2010), Appendix. GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 23

32 Critiques of PISA 9. Lingiard and Grek (2007). (i.e., democratic participation, artistic talents, understanding of politics, history, etc). 9 Goldstein argues that too much importance is attached to PISA: Perhaps the major [reservation about PISA] centers around the narrowness of its focus, which remains concerned, even fixated, with psychometric properties of a restricted class of conceptually simple models. There is almost no reference [in the official PISA reports] to debates about Percentage of white students in the class of 2011 in U.S. states at the proficient level in math and Percentage Proficient U.S. white student average Shanghai Singapore Hong Kong Massachusetts Korea Finland Taiwan Liechtenstein Switzerland Maryland Texas New Jersey Japan Canada Netherlands Minnesota Macao Colorado Virginia Belgium New Zealand North Carolina Kansas Germany North Dakota South Carolina Pennsylvania Australia Connecticut Alaska Delaware South Dakota Wisconsin Vermont U.S. Washington Ohio Estonia Montana Nebraska Illinois Iceland Indiana Arizona California New York France Oregon New Hampshire Wyoming Slovenia Denmark Iowa Idaho Austria Florida Percentage of white students in the class of 2011 in U.S. states at the proficient level in reading and Percentage Proficient U.S. white student average Shanghai Massachusetts New Jersey Korea Connecticut Finland Hong Kong Maryland Colorado Texas New York Singapore Ohio Vermont Montana New Zealand Japan Pennsylvania Delaware Minnesota Canada Virginia U.S. Kansas Washington South Dakota North Carolina Nebraska Illinois Iowa Australia Wisconsin Georgia Maine New Hampshire Missouri Arizona Oregon Belgium Alaska Florida Wyoming South Carolina Indiana Rhode Island Idaho Netherlands Michigan California North Dakota Utah France Arkansas Tennessee Switzerland Germany 24 educationnext.org hks.harvard.edu/pepg

33 Critiques of PISA the appropriateness of these models, nor is there reference to methodological and substantive critiques the usefulness [of such international surveys] must remain in doubt and their value for money somewhat questionable. 10 Whatever the legitimacy of such concerns, there is little doubt that the acquisition of mathematical and reading skills are fundamental to effective performance in contemporary industrial societies. 10. Goldstein (2004). percentage of all students at that level in foreign jurisdictions participating in PISA (Figure A1) Norway Iceland Oklahoma Hawaii Poland Sweden Liechtenstein Nevada Ireland Kentucky New Mexico U.K. Mississippi Louisiana Alabama Hungary Estonia Taiwan Italy Denmark Israel Portugal Slovenia Greece Luxembourg West Virginia Austria Czech Rep. Slovakia Spain Latvia Dubai Macao Croatia Lithuania Turkey Russia Bulgaria Trinidad and Tobago Chile Uruguay Serbia Brazil Qatar Argentina Romania Mexico Montenegro Colombia Kazakhstan Panama Jordan Thailand Albania Tunisia Peru Kyrgyzstan Indonesia Azerbaijan Georgia Missouri Utah Slovakia Norway Sweden Michigan Luxembourg Rhode Island Poland Maine Hungary Czech Rep. U.K. New Mexico Portugal Nevada Ireland Italy Spain Arkansas Tennessee Kentucky Louisiana Hawaii Latvia Lithuania Alabama Oklahoma Mississippi Greece Dubai Russia Israel Croatia West Virginia Turkey Serbia Bulgaria Uruguay Trinidad and Tobago Romania Chile Thailand Mexico Qatar Kazakhstan Argentina Azerbaijan Montenegro Brazil Albania Jordan Peru Colombia Panama Tunisia Indonesia Kyrgyzstan percentage of all students at that level in foreign jurisdictions participating in PISA (Figure A2) GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 25

34 Percentage of students in the class of 2011 with at least one college-educated parent in U.S. states at the participating in PISA (Figure A.3) Percentage Proficient U.S. average for students of college educated parents Shanghai Singapore Hong Kong Massachusetts Korea Finland Taiwan Liechtenstein Vermont Minnesota Switzerland Kansas New Jersey Colorado Pennsylvania Virginia Japan Canada Texas Netherlands Indiana Macao Maryland Wisconsin New Hampshire Ohio Washington Montana North Carolina Oregon South Dakota North Dakota Connecticut Belgium Idaho New Zealand Wyoming Iowa Germany Illinois Nebraska U.S. Maine Australia South Carolina Utah Delaware Estonia Rhode Island New York Iceland Missouri California France Michigan Arizona Percentage of students in the class of 2011 with at least one college-educated parent in U.S. states at participating in PISA (Figure A.4) Percentage Proficient U.S. average for students of college educated parents Massachusetts Shanghai Vermont New Jersey Connecticut Pennsylvania Oregon Ohio Korea Montana New Hampshire Finland Maine Kansas Hong Kong Virginia Minnesota Maryland Colorado Iowa South Dakota Washington Nebraska Singapore U.S. Texas New Zealand Japan Indiana Wisconsin New York Canada Illinois Missouri Wyoming Idaho Delaware Utah Rhode Island North Dakota Australia North Carolina Michigan Belgium Arizona Kentucky Florida Oklahoma California Netherlands Tennessee South Carolina France Georgia Arkansas Switzerland Percentage of students in the class of 2011 at the advanced level in math in U.S. states and foreign Percentage Proficient Shanghai Singapore Hong Kong Taiwan Korea Switzerland Japan Belgium Finland Massachusetts Netherlands New Zealand Canada Germany Macao Liechtenstein Australia Minnesota Slovenia New Jersey Vermont Maryland Iceland France Slovakia Colorado Austria Washington Virginia Czech Rep Kansas Oregon Connecticut Estonia Denmark Sweden Luxembourg North Carolina Wisconsin Pennsylvania New Hampshire Nebraska Indiana South Carolina Poland Montana Maine Alaska Iowa Hungary Norway Note: Excludes participating countries below 1 percent. 26 educationnext.org hks.harvard.edu/pepg

35 proficient level in math and percentage of all students at that level in foreign jurisdictions the proficient level in reading and percentage of all students at that level in foreign jurisdictions U.S. Illinois Texas South Dakota North Dakota U.K. Delaware New York Portugal Wyoming Ohio Italy Idaho Utah Michigan Florida Spain Missouri Kentucky Rhode Island California Arizona Lithuania Dubai Ireland Turkey Georgia Israel Greece Tennessee Arkansas Nevada Russia Latvia Croatia Oklahoma Hawaii New Mexico Bulgaria Alabama Serbia Louisiana West Virginia Trinidad and Tobago Mississippi Uruguay Qatar District of Columbia Thailand Kazakhstan Chile Azerbaijan Romania Montenegro Germany Norway Iceland West Virginia Poland Sweden Liechtenstein Ireland U.K. Alabama Hungary Estonia Nevada Hawaii Taiwan Italy Denmark Israel New Mexico Portugal Slovenia Greece Louisiana Luxembourg Austria Czech Rep. Mississippi Slovakia Spain Latvia Dubai Macao Croatia District of Columbia Lithuania Turkey Russia Bulgaria Trinidad and Tobago Chile Uruguay Serbia Brazil Qatar Argentina Romania Mexico Montenegro Colombia Kazakhstan Panama Jordan Thailand Albania Tunisia Peru Kyrgyzstan Indonesia Azerbaijan Slovenia Denmark Austria Kentucky Florida Slovakia Norway Sweden Luxembourg Poland Nevada Hungary Czech Rep. Georgia U.K. Arkansas Portugal Ireland Tennessee Oklahoma Italy Spain New Mexico Hawaii West Virginia Latvia Lithuania Alabama Louisiana Greece Dubai Russia Israel Croatia Mississippi Turkey Serbia Bulgaria Uruguay District of Columbia Trinidad and Tobago Romania Chile Thailand Mexico Qatar Kazakhstan Argentina Azerbaijan Montenegro Brazil Albania Jordan Peru Colombia Panama Tunisia Indonesia Kyrgyzstan jurisdictions participating in PISA (Figure A.5) U.S. average GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 27

36 Appendix Table A.1 Percentages of white students in the class of 2011 at the proficient level in math per state. Foreign jurisdictions with similar and higher percentages at the proficient level in math in overall student population. Percent Significantly Countries with similar percentages State proficient outperformed by* of proficient students 1 Massachusetts Finland Korea Lichtenstein Netherlands 2 Maryland Canada Japan Korea Netherlands Switzerland 3 Texas Canada Japan Korea Netherlands Switzerland 4 New Jersey Belgium Canada Japan Korea Netherlands 5 Minnesota Australia Canada Germany Japan Netherlands 6 Colorado Belgium Canada Germany Japan Netherlands 7 Virginia Australia Canada Germany Japan Netherlands 8 North Carolina Australia Canada Germany Japan Netherlands 9 Kansas Australia Belgium Germany Japan Netherlands 10 North Dakota Australia Belgium Germany Netherlands New Zealand 11 South Carolina Australia Belgium France Germany Netherlands 12 Pennsylvania Australia Belgium Germany Netherlands New Zealand 13 Connecticut Australia Belgium France Germany Netherlands 14 Alaska Australia Belgium France Germany Netherlands 15 Delaware Australia Estonia Germany Netherlands 16 South Dakota Australia Denmark France Germany Netherlands 17 Wisconsin Australia Estonia France Germany Netherlands 18 Vermont Australia Denmark France Germany Netherlands United States Estonia 19 Washington Australia Denmark France Germany Netherlands 20 Ohio Australia Austria France Germany Netherlands 21 Montana Austria Denmark France Iceland Slovenia 22 Nebraska Australia Austria France Germany Netherlands 23 Illinois Australia France Germany Netherlands Poland 24 Indiana Australia Austria Denmark France Germany 25 Arizona Australia France Germany Hungary Poland 26 California Austria Denmark France Hungary Sweden 27 New York Austria France Hungary Poland Sweden 28 Oregon Austria Denmark France Hungary Sweden 29 New Hampshire Austria Denmark France Slovakia Slovenia 30 Wyoming Austria France Hungary Poland Sweden 31 Iowa Czech Rep. France Hungary Poland Sweden 32 Idaho Austria France Hungary Poland Sweden 33 Florida Czech Rep. France Poland Portugal U.K. 34 Georgia Czech Rep. France Poland Portugal U.K. 35 Missouri Czech Rep. France Poland Portugal U.K. 36 Utah Czech Rep. France Hungary Poland U.K. 37 Michigan Czech Rep. France Poland Portugal U.K. 38 Rhode Island Czech Rep. Hungary Poland Portugal U.K. 39 Maine Czech Rep. Hungary Poland Portugal U.K. 40 New Mexico France Italy Poland Spain U.K. 41 Nevada Italy Poland Portugal Spain U.K. 42 Arkansas Czech Rep. Italy Portugal Spain U.K. 43 Tennessee Hungary Italy Portugal Spain U.K. 44 Kentucky Ireland Italy Lithuania Portugal Spain 45 Louisiana Greece Italy Portugal Spain U.K. 46 Hawaii France Italy Russia Turkey U.K. 47 Alabama Greece Italy Portugal Russia Spain 48 Oklahoma Croatia Greece Israel Lithuania Russia 49 Mississippi Croatia Greece Israel Russia Turkey 50 West Virginia Turkey *Number of countries whose percent proficient was statistically significantly higher Note: List of countries performing at a level that cannot be distinguished statistically are limited to those 5 with the largest population. 28 educationnext.org hks.harvard.edu/pepg

37 Appendix Table A.2 Percentages of white students in the class of 2011 at the proficient level in reading per state. Foreign jurisdictions with similar and higher percentages at the proficient level in reading in overall student population. Percent Significantly Countries with similar percentages State proficient outperformed by* of proficient students 1 Massachusetts Finland Korea 2 New Jersey Finland Korea 3 Connecticut Finland Korea 4 Maryland Canada Finland Japan Korea Singapore 5 Colorado Canada Finland Japan Korea Netherlands 6 Texas Canada Finland Japan Korea Netherlands 7 New York Canada Finland Japan Korea Singapore 8 Ohio Australia Canada Japan Korea Netherlands 9 Vermont Canada Japan Korea New Zealand Singapore 10 Montana Canada Japan Korea Netherlands Singapore 11 Pennsylvania Canada Japan Korea Netherlands Singapore 12 Delaware Canada Japan Korea Netherlands Singapore 13 Minnesota Australia Canada Japan Netherlands Singapore 14 Virginia Australia Canada Japan Korea Netherlands United States Canada Japan New Zealand 15 Kansas Australia Canada Japan Netherlands Singapore 16 Washington Australia Belgium Canada Japan Netherlands 17 South Dakota Australia Belgium Canada Japan Netherlands 18 North Carolina Australia Belgium Canada Japan Netherlands 19 Nebraska Australia Belgium Canada Japan Netherlands 20 Illinois Australia Belgium Canada Japan Netherlands 21 Iowa Australia Belgium Japan Netherlands 22 Wisconsin Australia Canada France Japan Netherlands 23 Georgia Australia Canada France Japan Netherlands 24 Maine Australia Belgium Japan Liechtenstein Netherlands 25 New Hampshire Australia Belgium Liechtenstein Netherlands 26 Missouri Australia Belgium Liechtenstein Netherlands 27 Arizona France Germany Japan Korea U.K. 28 Oregon Australia Canada France Germany Japan 29 Alaska Australia Belgium France Liechtenstein Netherlands 30 Florida Australia Belgium France Germany Netherlands 31 Wyoming Australia Belgium France Liechtenstein Netherlands 32 South Carolina Australia France Germany Netherlands Poland 33 Indiana Australia Belgium France Germany Netherlands 34 Rhode Island Australia Belgium France Germany Netherlands 35 Idaho Belgium France Germany Netherlands Switzerland 36 Michigan Belgium France Germany Netherlands Poland 37 California France Germany Netherlands Norway Switzerland 38 North Dakota France Germany Netherlands Poland Sweden 39 Utah France Germany Netherlands Poland Sweden 40 Arkansas France Germany Netherlands Poland Sweden 41 Tennessee France Germany Netherlands Poland Sweden 42 Oklahoma France Germany Netherlands Poland U.K. 43 Hawaii France Germany Italy Poland U.K. 44 Nevada France Germany Italy Poland U.K. 45 Kentucky France Germany Netherlands Poland U.K. 46 New Mexico France Germany Italy Poland U.K. 47 Mississippi France Germany Italy Poland U.K. 48 Louisiana France Germany Italy Poland U.K. 49 Alabama Germany Italy Netherlands Poland U.K. 50 West Virginia Austria Czech Rep. Greece Portugal Spain *Number of countries whose percent proficient was statistically significantly higher Note: List of countries performing at a level that cannot be distinguished statistically are limited to those 5 with the largest population. GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 29

38 Appendix Table A.3 Percentages of students in the class of 2011 with at least one college-educated parent at the proficient level in math per state. Foreign jurisdictions with similar and higher percentages at the proficient level in math in overall student population. Percent Significantly Countries with similar percentages State proficient outperformed by* of proficient students 1 Massachusetts Korea Liechtenstein Singapore 2 Vermont Finland Korea Liechtenstein Switzerland 3 Minnesota Liechtenstein Switzerland 4 Kansas Japan Liechtenstein Netherlands Switzerland 5 New Jersey Japan Liechtenstein Netherlands Switzerland 6 Colorado Canada Japan Liechtenstein Netherlands Switzerland 7 Pennsylvania Canada Japan Liechtenstein Netherlands Switzerland 8 Virginia Canada Japan Liechtenstein Netherlands Switzerland 9 Texas Canada Japan Liechtenstein Netherlands 10 Indiana Belgium Canada Japan Netherlands New Zealand 11 Maryland Belgium Canada Japan Netherlands New Zealand 12 Wisconsin Belgium Canada Netherlands New Zealand 13 New Hampshire Belgium Netherlands New Zealand 14 Ohio Belgium Netherlands New Zealand 15 Washington Belgium Canada Netherlands New Zealand 16 Montana Belgium Netherlands New Zealand 17 North Carolina Belgium Canada Germany Japan Netherlands 18 Oregon Belgium Canada Germany Netherlands New Zealand 19 South Dakota Belgium Netherlands New Zealand 20 North Dakota Belgium Netherlands New Zealand 21 Connecticut Belgium Netherlands New Zealand 22 Idaho Belgium Germany Netherlands New Zealand 23 Wyoming Australia Belgium Germany Netherlands New Zealand 24 Iowa Belgium Germany New Zealand 25 Illinois Australia Belgium Germany New Zealand 26 Nebraska Australia Germany United States Australia Germany Netherlands 27 Maine Australia Germany 28 South Carolina Australia Estonia Germany 29 Utah Australia Germany 30 Delaware Australia Estonia Germany Iceland 31 Rhode Island Estonia Iceland 32 New York Estonia France Iceland Slovenia 33 Missouri Denmark Estonia France Iceland Slovenia 34 California Denmark France Slovenia 35 Michigan Austria Denmark France Iceland Slovenia 36 Arizona Austria Denmark France Slovakia Slovenia 37 Kentucky Austria Denmark France Slovakia Sweden 38 Florida Austria France Hungary Poland Sweden 39 Nevada Czech Rep. Hungary Poland Sweden U.K. 40 Georgia Czech Rep Hungary Poland U.K. 41 Arkansas Ireland Portugal U.K. 42 Tennessee Ireland Italy Portugal Spain U.K. 43 Oklahoma Ireland Italy Portugal Spain 44 New Mexico Italy Spain 45 Hawaii Spain 46 West Virginia Portugal 47 Alabama Latvia Lithuania 48 Louisiana Lithuania 49 Mississippi Turkey 50 District of Columbia Trinidad and Tobago Uruguay *Number of countries whose percent proficient was statistically significantly higher Note: List of countries performing at a level that cannot be distinguished statistically are limited to those 5 with the largest population. 30 educationnext.org hks.harvard.edu/pepg

39 Appendix Table A.4 Percentages of students in the class of 2011 with at least one college-educated parent at the proficient level in reading per state. Foreign jurisdictions with similar and higher percentages at the proficient level in reading in overall student population. Percent Significantly Countries with similar percentages State proficient outperformed by* of proficient students 1 Massachusetts Shanghai 2 Vermont Korea 3 New Jersey Finland Korea 4 Connecticut Finland Korea 5 Pennsylvania Finland Korea 6 Oregon Finland Japan Korea New Zealand Singapore 7 Ohio Finland Japan Korea New Zealand Singapore 8 Montana Finland Korea 9 New Hampshire Finland Korea 10 Maine Finland Japan Korea New Zealand Singapore 11 Kansas Finland Japan Korea 12 Virginia Australia Canada Japan Korea Netherlands 13 Minnesota Finland Japan Korea New Zealand Singapore 14 Maryland Canada Finland Japan Korea Netherlands 15 Colorado Australia Canada Japan Korea Netherlands 16 Iowa Canada Finland Japan Korea Singapore 17 South Dakota Australia Canada Japan Korea Netherlands 18 Washington Canada Finland Japan Korea Netherlands 19 Nebraska Canada Finland Japan Korea Netherlands United States Japan New Zealand Singapore 20 Texas Australia Canada Japan Korea Netherlands 21 Indiana Australia Canada Japan Korea Netherlands 22 Wisconsin Australia Canada Japan Korea Netherlands 23 New York Canada Japan Korea Netherland Singapore 24 Illinois Australia Canada Japan Korea Netherlands 25 Missouri Australia Canada Japan Korea Netherlands 26 Wyoming Australia Canada Japan Korea Netherlands 27 Idaho Australia Belgium Canada Japan Netherlands 28 Delaware Australia Canada Japan Korea Netherlands 29 Utah Australia Belgium Canada Japan Netherlands 30 Rhode Island Australia Belgium Canada Japan Netherlands 31 North Dakota Australia Canada France Japan Netherlands 32 North Carolina Australia Belgium Canada Japan Netherlands 33 Michigan Australia Canada France Japan Netherlands 34 Arizona Canada France Germany Japan Poland 35 Kentucky Australia Canada France Germany Japan 36 Florida Australia Belgium France Germany Netherlands 37 Oklahoma Australia Belgium France Germany Netherlands 38 California Belgium France Germany Netherlands Switzerland 39 Tennessee Australia France Germany Poland U.K. 40 South Carolina Australia France Germany Poland U.K. 41 Georgia France Germany Netherlands Poland U.K. 42 Arkansas France Germany Netherlands Poland U.K. 43 West Virginia France Germany Italy Poland U.K. 44 Alabama France Germany Italy Poland U.K. 45 Nevada France Germany Italy Spain U.K. 46 Hawaii Germany Italy Netherlands Poland U.K. 47 New Mexico Greece Italy Netherlands Poland U.K. 48 Louisiana Italy Netherlands Poland Spain U.K. 49 Mississippi Czech Rep. Greece Italy Portugal Spain 50 District of Columbia Bulgaria Croatia Lithuania Slovakia Turkey *Number of countries whose percent proficient was statistically significantly higher Note: List of countries performing at a level that cannot be distinguished statistically are limited to those 5 with the largest population. GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 31

40 Appendix Table A.5 Percentages of all students in the class of 2011 at the advanced level in math per state. Foreign jurisdictions with similar and higher percentages at the advanced level in math in overall student population. Percent Significantly Countries with similar percentages State advanced outperformed by* of advanced students 1 Massachusetts Belgium Canada Germany Japan Netherlands 2 Minnesota Australia Liechtenstein Slovenia 3 New Jersey France Iceland Liechtenstein Slovakia Slovenia 4 Vermont Austria France Iceland Slovakia Slovenia 5 Maryland Austria Czech Rep France Slovakia Slovenia 6 Colorado Austria Czech Rep France Iceland Slovakia 7 Washington Austria Czech Rep Denmark France Sweden 8 Virginia Austria Czech Rep Denmark France Sweden 9 Kansas Austria Czech Rep Denmark France Sweden 10 Oregon Austria Czech Rep Denmark France Sweden 11 Connecticut Czech Rep Denmark France Hungary Sweden 12 North Carolina Czech Rep Denmark Hungary Poland Sweden 13 Wisconsin Czech Rep Denmark Hungary Poland Sweden 14 Pennsylvania Czech Rep Denmark Hungary Poland Sweden 15 New Hampshire Czech Rep Denmark Hungary Poland Sweden 16 Nebraska Denmark Hungary Norway Poland Sweden 17 Indiana Hungary Poland Portugal Sweden U.K. 18 South Carolina Hungary Poland Portugal Sweden U.K. 19 Montana Hungary Norway Poland Portugal U.K. 20 Maine Hungary Poland Portugal Sweden U.K. 21 Alaska Hungary Norway Poland Portugal U.K. 22 Iowa Hungary Norway Poland U.K. United States Hungary Italy Poland Portugal U.K. 23 Illinois Hungary Poland Portugal Sweden U.K. 24 Texas Hungary Norway Poland Portugal U.K. 25 South Dakota Hungary Norway Poland Portugal U.K. 26 North Dakota Hungary Italy Poland Portugal U.K. 27 Delaware Italy Poland Portugal Turkey U.K. 28 New York Hungary Italy Poland Portugal U.K. 29 Wyoming Hungary Italy Poland Portugal U.K. 30 Ohio Hungary Italy Poland Portugal U.K. 31 Idaho Hungary Italy Portugal U.K. 32 Utah Hungary Italy Portugal U.K. 33 Michigan Hungary Italy Portugal U.K. 34 Florida Lithuania Spain Turkey 35 Missouri Lithuania Spain Turkey 36 Kentucky Lithuania Spain Turkey 37 Rhode Island Lithuania Spain Turkey 38 California Lithuania Turkey 39 Arizona Ireland Lithuania Spain Turkey 40 Georgia Greece Ireland Israel Russia Turkey 41 Tennessee Greece Ireland Israel Russia Turkey 42 Arkansas Greece Ireland Israel Russia Turkey 43 Nevada Greece Ireland Israel Russia Turkey 44 Oklahoma Bulgaria Croatia Russia Turkey 45 Hawaii Bulgaria Croatia Turkey 46 New Mexico Bulgaria Serbia 47 Alabama Bulgaria Serbia 48 Louisiana Bulgaria Serbia 49 West Virginia Bulgaria Serbia 50 Mississippi Trinidad and Tobago Uruguay 51 District of Columbia Dubai *Number of countries whose percent advanced was statistically significantly higher Note: List of countries performing at a level that cannot be distinguished statistically are limited to those 5 with the largest population. 32 educationnext.org hks.harvard.edu/pepg

41 References References Bartsch, Kristina (2009): Employment projections, , Monthly Labor Review November. Cosgrove, Judith, Nick Sofroniou, Amy Kelly, and Gerry Shiel (2003): A teacher s guide to the reading literacy achievements of Irish 15-year-olds. Educational Research Center: Dublin. Goldstein, Harvey (2004): International comparisons of student attainment: some issues arising from the PISA study, Assessment in Education McKinsey Global Institute (2011): Growth and renewal in the United States: Retooling America s economic engine, February (available at mckinsey.com/mgi) Hanushek, Eric, Paul Peterson, and Ludger Woessmann (2010): U. S. Math Performance in global perspective: How well does each state do at producing high-achieving students? Harvard s Program on Education Policy and Governance Report No.: Hanushek, Eric and Ludger Woessmann (2008): The role of cognitive skills in economic development. Journal of Economic Literature, 46:3. Hanushek, Eric A. and Ludger Woessmann (2011): How much do educational outcomes matter in OECD countries? Economic Policy, no. 67 (July), Lingiard, Bon and Grek, Sotiria (2007): The OECD, indicators and PISA: An exploration of events and theoretical perspectives ESRC/ESF Research Project on Fabricating Quality in Education Working paper 2, PDF%20Files/FabQ_WP2.pdf Mullis, Ina, V. S., Michael O. Martin and Pierre Foy, (2008): TIMMS 2007 International Mathematics Report: Findings from IEA s Trends in International Mathematics and Science Study at the Fourth and Eighth Grades. International Association for the Evaluation of Educational Achievement. Shiel, G., R. Perkins, S. Close,, and E. Oldham (2007): PISA Mathematics: A Teacher s Guide Ireland s Department of Education and Science. OECD (2009a): PISA 2009 assessment framework: Key competencies in reading, mathematics and science. Paris: OECD. OECD (2009b): PISA Data Analysis Manual. Paris: OECD. OECD (2000): Knowledge and skills for life: First results from PISA Paris: OECD. Prais, Sig (2003): Cautions on OECD S recent educational survey (PISA). Oxford Review of Education, Vol. 29, No. 2. Provasnik, Stephan, Patrick Gonzales, and David Miller (2009): U. S. performance across international assessments of student achievement: Special supplement to the Condition of Education Washington, DC: National Center for Education Statistics. GLOBALLY CHALLENGED: ARE U.S. STUDENTS READY TO COMPETE? 33

42 Biographical Sketches Biographical Sketches Paul E. Peterson is the director of the Program on Education Policy and Governance in the Taubman Center for State and Local Government at Harvard s Kennedy School of Government. He is the Henry Lee Shattuck Professor of Government in the Faculty of Arts and Sciences at Harvard University and a senior fellow at the Hoover Institution at Stanford University. He is the editor-in-chief of Education Next: A Journal of Opinion and Research. His most recent publication is Saving Schools: From Horace Mann to Virtual Learning (Harvard University Press, 2010). Websites: hks.harvard.edu/pepg educationnext.org savingschools.com Ludger Woessmann is professor of economics at the University of Munich, head of the Department of Human Capital and Innovation at Ifo Institute for Economic Research, and coordinator of the European Expert Network on the Economics of Education (EENEE). His most recent book, School Accountability, Autonomy and Choice around the World, considers sources of international differences in student achievement. Website: cesifo.de/woessmann Eric A. Hanushek is the Paul and Jean Hanna Senior Fellow at the Hoover Institution of Stanford University. He is chairman of the Executive Committee for the Texas Schools Project at the University of Texas at Dallas, a research associate of the National Bureau of Economic Research, a member of the Koret Task Force on K-12 Education, and the area coordinator for Economics of Education of the CESifo Research Network. He currently serves as a commissioner on the Equity and Excellence Commission of the U.S. Department of Education and was the past chair of the Board of Directors of the National Board for Education Sciences. His most recent book, Schoolhouses, Courthouses, and Statehouses, describes how improved school finance policies can be used to meet our achievement goals. Website: hanushek.net Carlos X. Lastra-Anadón is a research fellow with the Program on Education Policy and Governance in the Taubman Center for State and Local Government at Harvard s Kennedy School of Government. He recently earned his Masters in Public Policy from Harvard s Kennedy School. In addition to contributing to PEPG s U.S. Math Performance in Global Perspective report, he is co-organizing a conference at PEPG on that subject in August of His most recent paper, State Standards Rise in Reading, Fall in Math, co-authored with Paul E. Peterson, was published in the Fall 2010 issue of Education Next. Before that he worked in international education reform with McKinsey and Company in the Middle East and Europe and worked on the launch of Empieza por Educar, the Spanish version of Teach for America and a member of the Teach for All network. 34 educationnext.org hks.harvard.edu/pepg