PUBPOL 639 Quantitative Methods for Program Evaluation Winter 2010 Instructor: Kevin Stange (kstange@umich.edu) Office hours: Monday & Wednesday 2:40-4:00, Weill Hall 5236 GSI: Yann Toullec (ytoullec@umich.edu Office Hours: Tuesday 4-5 & Thursday 10-11, Weill Hall 3207 Lecture: s: Final Exam: Monday & Wednesday 1:00-2:30, 1110 Weill Hall Friday 2:30-4:00, 1110 Weill There is no final exam Overview and Objectives This course introduces students to multiple regression analysis and other tools of causal inference and program evaluation. The course will focus on applying these tools to real data on various policy topics. Applications will be drawn from a wide range of policy areas including education, welfare, unemployment, discrimination, health, immigration, the environment, and economic development. The course has two highly related objectives: 1) Train students to thoughtfully produce their own empirical research. With the wide availability of data and statistical software, there are very few technical barriers to conducting empirical research. All you need is an internet connection and Excel. However, producing good and convincing empirical research is another matter entirely. In this course, we will develop the core analytical tools of single and multi-variable regression and also discuss fixed effects, difference-in-difference, natural experiment, instrumental variables, regression discontinuity, event study, and matching approaches. Throughout the focus will be on real world applications, understanding the strengths and weaknesses of each approach, and communicating methods and findings in plain English. 2) Train students to critically consume empirical research done by others. We will teach you to read and understand empirical research and to judge whether it constitutes a firm basis for policy. This should serve you in your future role as a policy or business analyst, researcher, policy-maker, manager, or voter. Readings Since the course is primarily a methods course, the majority of readings will be from one of the course textbooks. Both are required and should serve as a useful reference in your future work: 1) Stock and Watson, Introduction to Econometrics 2nd edition (1 st or brief edition OK). 2) Angrist and Pischke, Mostly Harmless Econometrics. Paperback edition. The textbook readings will be supplemented with additional readings including academic journal articles and policy reports. All readings should be done before lecture. 1
Prerequisites: PUBPOL 529 (statistics) or equivalent. Topics The course is divided into three parts. In Part I, we will discuss causal inference as distinct from statistical inference and contrast evidence from observational data with that from randomized trials. We will also review core statistical inference concepts, though this review will be brief because the course assumes that you know this. In Part II, we will develop the core analytic tool of linear regression. We will cover single and multi-variable regression models, hypothesis testing, dummy variables, heteroskedasticity, model fit, multicollinearity, joint hypothesis testing, and transformations (logs, exponentials, polynomials, interactions). We will also briefly discuss a few alternatives to linear regression, such as Logit and Probit models (when outcomes are binary) and matching. A key limitation of regression is that it requires that all relevant factors can be adequately measured and controlled for. Part III introduces fixed effects and panel data, differences-in-differences, event study, instrumental variables, and regression discontinuity models. These are all techniques to control for some unobserved factors that may confound estimates from linear regression. A GSI will be leading section every Friday. s will mostly be used to demonstrate how to put quantitative methods into practice using Stata and to provide guidance on the problem sets. The GSI may occasionally use the time to clarify material covered in lecture or the readings. Course Components In-class Quizzes (7) 20% Quizzes will test material from both the reading and lectures. Quizzes cannot be made up, so plan your schedule accordingly. Your lowest quiz score will be dropped. The quizzes are closedbook. You may consult a single index card of notes during the quizzes. Homework Assignments (8-10) 40% Homework assignments consist of data analysis and short essays (< 1 page) that interpret your findings. At least one will ask you to contrast and evaluate the methods used by a set of papers on a specific topic. They are graded on a scale of 0 to 10. You may (and are encouraged to) discuss the assignments in groups of three or four, but your answers must be written up individually, in your own words. So that we may confirm that you have written up answers in your own words, list your study group members on your problem set. Problem sets should be typed and uploaded to the course website. Class Participation 15% During each class, I will ask questions of randomly-selected students. This is intended to generate compulsory democratic participation. Questions will be based on the reading, assignments, problem sets and lectures. Final Paper 25% There will be a final paper/project due Friday April 23 rd. You will be asked to critically evaluate the empirical evidence on a specific policy topic and make a recommendation based on the weight of this evidence. 2
Software We will do analysis in Stata, a software program used widely by policy analysts. We provide links to online Stata tutorials and offer training in sections. Stata is available in the Ford School computer lab and from the UM computing resources. It is also available for purchase. I recommend you buy it so that you can use it freely and often, the best way to learn any language. 3
DETAILED COURSE SCHEDULE (Note: More supplemental readings will be added) PART I: CAUSAL INFERENCE BASICS WEEK 1 Mon 1/11 Lecture 1: Overview and Introduction No readings Wed 1/13 Lecture 2: Causal Inference I 1. Angrist & Pischke Ch 1 & 2. 2. Paul W. Holland (1986). "Statistics and Causal Inference." Journal of the American Statistical Association 81:396 (Dec), pp. 945-960. Fri 1/15 WEEK 2 Mon 1/18 No class (MLK day) Wed 1/20 Lecture 3: Causal Inference II Stock & Watson, Ch. 1 Fri 1/22 PART II: REGRESSION (BIVARIATE AND MULTIVARIATE) WEEK 3 Mon 1/25 Lecture 4: Randomized Trials 1. Bertrand and Mullainathan, 2004. Are Emily and Greg More Employable than Lakisha and Jamal? Evidence on Racial Discrimination in the Labor Market from a Large Randomized Experiment, September 2004, American Economic Review. 2. Stock & Watson, Chs. 2 & 3 (to review t-tests, p-values, confidence intervals, hypothesis testing, all of which we will use in class today) Wed 1/27 Fri 1/29 Lecture 5: Observational Analysis & Introduction to Bivariate Regression Stock and Watson Ch. 4.1-4.4, Appendix 4.1 WEEK 4 Mon 2/1 Lecture 6: Bivariate Regression & Testing Hypotheses Stock and Watson Ch 4.5; 5.1-5.2 Wed 2/3 Fri 2/5 Lecture 7: Dummy Variables, Heteroskedasticity Stock and Watson Ch 5.3, 5.4 WEEK 5 Mon 2/8 Lecture 8: Measures of Fit, Interpreting Output No new readings Wed 2/10 Fri 2/12 Lecture 9: Introduction to Multiple Regression, Omitted Variable Bias Stock & Watson Ch 5.7, 6.1-6.6 WEEK 6 Mon 2/15 Lecture 10: Multiple Regression Baiker and Chandra, 2004. Cooper 2008. Baiker and Chandra 2008. Wed 2/17 Fri 2/19 Lecture 11: Multiple Regression & Joint Hypothesis Tests Stock & Watson Ch. 7 4
WEEK 7 Mon 2/22 Lecture 12: Multiple dummies, multicollinearity Stock & Watson Ch. 6.7 Wed 2/24 Lecture 13: Multiple Regression & Causality Angrist & Pischke Ch 3 through 3.2.3 (skim very technical stuff, get the gist) Fri 2/26 WEEK 8 Mon 3/1 Winter Break Wed 3/3 Fri 3/5 Winter Break Winter Break WEEK 9 Mon 3/8 Lecture 14: Multiple Regression & Causality (continued) Wed 3/10 Fri 3/12 WEEK 10 Mon 3/15 Lecture 15: Nonlinearity: Logs, Polynomials, and "Non-parametrics" Lecture 16: Nonlinearity: Logs, Polynomials, and "Non-parametrics" (continued) Stock & Watson Chs. 8.1-8.2 Wed 3/17 Lecture 17: Interaction Terms Stock & Watson Ch. 8.3 Fri 3/19 WEEK 11 Mon 3/22 Wed 3/24 Lecture 18: Brief Introduction to Matching Methods Lecture 19: Binary Dependent Variables: Linear Probability Model Angrist & Pischke Ch 3.3 Dynarski, Susan (2003). Does Aid Matter? Measuring the Effect of Student Aid on College Attendance and Completion. American Economic Review 93:1, pp. 279-288. Fri 3/26 PART III: ADDRESSING UNOBSERVABLES WEEK 12 Mon 3/29 Lecture 20: Probit and Logit Stock & Watson Ch. 11.1-11.3 Wed 3/31 Lecture 21: Fixed Effects, Panel Data & Natural Experiments 1. Currie, Janet and Duncan Thomas, (1995). Does Head Start Make a Difference? American Economic Review 85(3): 341-364. 2. Angrist & Pischke Ch 5 (through 5.1) Fri 4/2 5
WEEK 13 Mon 4/5 Lecture 22: Differences in Differences; Pooled Cross s; Event Study 1. Card and Krueger, 1994. "Minimum Wages and Employment: A Case Study of the Fast Food Industry in New Jersey and Pennsylvania." American Economic Review 84 (September 1994). 2. Eissa & Liebman, 1996. "Labor Supply Response to the Earned Income Tax Credit," The Quarterly Journal of Economics, vol. 111(2), pages 605-37, May. 3. Angrist & Pischke, Ch 5.2 Wed 4/7 Lecture 23: Instrumental Variables 1. Angrist & Pischke, Ch 4 (through 4.1.2) 2. Stock & Watson Ch 12.1-12.3 3. Angrist, Joshua and Alan Krueger (1991). Does Compulsory Schooling Attendance Affect Schooling and Earnings? Quarterly Journal of Economics 106:4, pp. 979-1014. Fri 4/9 WEEK 14 Mon 4/12 Lecture 24: IV in Randomized Trials Krueger, Alan (1999). Experimental Estimates of Education Production Functions. Quarterly Journal of Economics 114:2, pp. 497-532. Wed 4/14 Lecture 25: Regression Discontinuity 1. Angrist & Pischke Ch. 6 2. DiNardo & Lee, 2004. "Economic Impacts of New Unionization on Private Sector Employers: 1984-2001," in Quarterly Journal of Economics, 119(4), 1383-1441. 3. Carpenter & Dobkin, 2009. The Effect of Alcohol Access on Consumption and Mortality: Regression Discontinuity Evidence from the Minimum Drinking Age, American Economic Journal: Applied Economics, Vol. 1, Issue 1, pp. 164 82 Fri 4/16 WEEK 15 Mon 4/19 Wed 4/21 Fri 4/23 Final Lecture No Lecture FINAL PAPER DUE 6