Summary of the assessment of the Economics undergraduate program goals for

Summary of the assessment of the Economics undergraduate program goals for 2012 13 For each of the core courses listed in the Learning Outcomes Table, the detailed course by course results for academic year 2012 13 appear in the Learning Outcomes document for that course. What follows is a summary of the key points from these and other assessment activities. The six broad learning goals are considered in turn, combining the assessment results from the Junior and Senior level courses. Economics 200 is summarized separately, partly because the assessments for its three broad learning goals are less clearly distinguished. Critical Thinking Taken as a group, the instructors of the eight core courses consider this to be the most important goal of the undergraduate program. It is an important part of assessment for all six courses that are required for the economics major. Critical thinking encompasses: the understanding and correct application of basic economic models such as supply and demand and profit maximization by firms; the mathematical skill required to use those models properly; careful evaluation of evidence; and a general facility in analyzing problems from the viewpoint of costs and benefits. Senior level: The trio of senior core courses provides a well balanced set of assessments. Economics 407 assesses Critical Thinking in the context of economic theory and business strategy, Economics 453 evaluates students capacity to complete a statistical modeling exercise, and Economics 479 evaluates their ability to write a clear and well structured economics essay that employs sound arguments. Students were generally successful in Economics 453 and 479. In 453 about 2/3 of the students earned the highest mark of 4/4. The instructor wrote that, despite minor caveats, "Students were very successful in learning how to estimate a logistic regression model." He also noted that in the Spring he gave extra instruction on aspects of statistical inference that had shown weakness in Fall s assessment; the fraction earning the highest mark rose from 65% to 69% from the Fall to the Spring semester. In Economics 479, 70% of students earned the highest mark for organization and clarity, but performance in using logical and consistent arguments was weaker. On that criterion only 45% earned the highest mark in the Fall, rising to 64% in the Spring. Such rapid improvement may reflect the assessment program s success but also underscores the importance of monitoring the consistency of assessment standards. In Economics 407, the planned future assessments of Critical Thinking, which are based on a written exam question, an individual report, and a group report, may be more demanding than in any other course. In 2012 13 the assessment score was weaker (2.9) than in 453 and 479, but the

assessment process was in its initial development, under two different instructors. The upcoming result for 2013 14 should be more meaningful. Junior level: At the junior level, the Critical Thinking score was much higher in Economics 332 (Macroeconomics) than in Economics 361 (Microeconomics). In 332 students earned 3.1, while in 361 they earned the minimum score of 1.0. The score in 332 was based on students responses to four written exam questions, concerning the workings of the IS/LM model and historically competing theories of macroeconomic behavior. (The fraction of answers that earned the highest score of 4/4 on those questions rose from 35% to 61% from the Fall to the Spring.) Students generally earned higher scores on the IS/LM questions than on the historical controversies. The score in Economics 361 was based on responses to two written exam questions that required mathematical and graphical problem solving, concerning the maximization of firm s profit and consumer s utility. Unlike the assessment questions in 332, these questions required solving a new problem, and the average score across all the questions was only 39%. The questions were reasonable given the expectations for Economics 361, which sets the foundation for most senior level courses in the undergraduate major. These results are thus concerning. The questions were on exams prior to the final exam and so may not reflect the mastery that students achieved by the end of the course. Business Knowledge The assessment of business knowledge places more emphasis on facts and occurs mostly before the senior year. The subgoals that the department has chosen to assess, within this broad topic, are: Explain the relationships among business, government, and markets. Explain the determination of prices in a market economy. Apply profit maximizing principles to common business decisions. Explain the determinants of international trade and the workings of international markets. The assessment of Business Knowledge is based entirely on answers to examination questions. Senior level: Among upper division classes, the department assesses Business Knowledge through Economics 407, but in 2012 13 the assessment on this goal was thin. It comprised three written questions, one in the Fall semester and two in the Spring. One was an algebraic question about oligopoly equilibrium, one concerned collusion, and the third question concerned optimal information structure in an organization. All required a reasonably high level of economic analysis. The performance (2.9 overall) was weaker than it should be, especially on the question that required simple math.

Following an internal review, the assessments for the academic year 2013 14 are broader and more extensive. Junior level: In Economics 332, Business Knowledge was assessed through a final exam question comparing Keynes s and Hayek s views on business cycles. The average performance across the semesters was: 36% Excellent (i.e., 4), 16% Good (i.e., 3), 31% Fair and 17% Poor. The percentage earning the highest mark rose from 9% in the Fall to 63% in the Spring, which makes these results hard to interpret. In Economics 361, the median score was 50% on a question requiring discussion of graphical cost curves and identifying a competitive firm s short run supply curve and also about 50% on a problem requiring detailed analysis of a monopolist's problem, including the calculation of elasticity. These were appropriate assessment questions and the results are concerning, because the program relies on 361 to teach the basic principles and methods of profit maximization. (The scores on the monopoly problem rose from 29% in the Fall to 72% in the Spring, which is encouraging.) The discrepancy between the topics assessed in 361 and in 332 indicates that it may be appropriate for the department to review the goals for these courses. Economics 300 is the simpler version of 361, for the general Eller major, and in 2012 13 the 20 multiple choice assessment questions focused on supply and demand, demand elasticity, costs, pricing strategy, the properties of different market structures, entry and exit, market power and antitrust, and regulation. On average, 80% of the students answered these questions correctly, a strong performance. The weakest score, by far, was on a question concerning two part pricing. Fourteen of the 20 questions (or very similar questions) were asked also in 2011 12. Student performance generally improved year over year; the three questions for which performance declined concerned optimal pricing and the conditions leading firms to exit an industry. In Economics 330, the simpler version of 332, the seven multiple choice questions assessing Business Knowledge focused on banking, monetary policy, and international trade and exchange rates. The average performance was again about 80%. The worst performance was on two questions concerning comparative advantage and Federal Reserve s required reserve rate. Only four of the seven questions either appeared in the 2011 12 assessment or had close counterparts, and for most of those questions the average performance improved year over year. Overall student performance in these courses was fairly strong, but the results indicate that, except for Economics 300, the department may in various cases want to review which topics in Business Knowledge are assessed or whether students are learning the material adequately.

Social Responsibility The Economics Department implements the Social Responsibility goal through learning about economic policy issues. It has chosen the following specific subgoals under Social Responsibility: Describe how alternative courses of action affect various individuals and social groups. Describe, evaluate, and apply criteria for weighing competing interests, for the purpose of making policy decisions. Compare and evaluate the arguments supporting various government policies. The department currently assesses students mastery of the Social Responsibility goal only at the sophomore and junior levels, and all assessment occurs through examination questions. In Economics 361, two written questions on the first midterm assessed Social Responsibility: one simple question concerning the efficiency of trade, and a second question concerned Pareto efficiency and required the calculation of consumer surplus. Students average score was 87% on the first question and 51% on the second. The average score on each question, as for most assessment questions in this course, increased more than 15% from the Fall semester to the Spring semester. The relatively low scores on the consumer surplus question again raise the issue of whether students come out of Economics 361 with enough mastery of the technical material to be well-prepared for subsequent courses in the Economics major. In Economics 332, three written questions form the assessment of Social Responsibility. One question concerns Keynes s view of uncertainty and how it affects economic decisions; the second and third questions concern the IS and LM curves. Altogether, 53% of students gave answers judged Excellent or Good (4 or 3) and 47% gave answers judged Fair or Poor. The instructors of 332 and 361 have chosen good assessment questions concerning policy issues, but the department should work with these instructors jointly to ensure that the questions are chosen holistically. The complete package of assessment questions should closely reflect the department s subgoals, be complementary across courses, and provide a fairly clear picture of undergraduate majors mastery of the basic elements of economic policy analysis. In Economics 330, the simpler version of 332, the 13 multiple choice questions assessing Social Responsibility comprised four on trade and exchange rates, six on stabilization policy, one on the aggregate supply curve, and two on the public debt. The average performance on these questions was 72% correct; the score on nine of the 13 questions was at least 80%, but very weak performance on questions concerning comparative advantage, the definition of a recessionary gap, and the public debt brought the average down. Economics 330 seems to be fulfilling fairly well its responsibility, within the Eller undergraduate major, to give students basic fluency in international trade and macroeconomic policy issues.

Communication The department assesses Communication in its core courses only at the senior level. Students did well in Economics 453 and 479 and no immediate adjustments are indicated. The assessment process also seems strong in those courses. The department will however review student comments on 479, in light of the enrollment drop. In Economics 479, three of the four criteria for evaluating the writing assignment designated for assessment focus on writing skills: organizational structure, professional tone, and English. On those criteria, 61% of the students received 4/4 and 16% received 3/4. In the empirical project for Econ 453, 82% of the students received 4/4 for Communication, a score that the instructor characterizes as sufficient to serve as a writing sample. The instructor also said that Students were extremely successful in demonstrating the ability to present statistical data in tables and graphs and also to write about statistical results. Economics 407 based its Communication assessment on one exam question in the Fall and on a group presentation in the Spring. Student performance was assessed lower than in the other two courses, but assessment for this course has already been restructured, going forward. The new plan for assessment of Communication (described in the Learning Outcomes document for 407) is more comprehensive and detailed than anything that the Economics Department has previously done for this learning goal. Collaboration The department assesses Collaboration in its core courses only in the senior level courses 407 and 453. The assessment in Economics 407 showed effective collaboration, and those assessment instruments have been improved for the next year. The assessment in Economics 453 produced a high score but it is unclear whether the instrument used provides an adequate measure of collaboration: any group that completes the project requirements automatically receives 4/4. That assessment also does not cover every student, because in 453 group participation is voluntary. In 2012 13, 80% of the students chose to work in a group. The department and the instructor will review this assessment instrument, with the possibility of making adjustments. Technology The department assesses the Technology goal only in Economics 453, where it focuses on capacity to use correctly a statistical software package. The department believes that this is the most challenging Technology task facing undergraduate majors, and the assessment shows strong student performance: 92% received 4/4 and the rest received 3/4. The instructor wrote:

Students were extremely successful in learning how to use SAS software for statistical analysis and for graphical presentation of data. The instructor of Economics 453 makes adjustments year by year to correct any weaknesses in students use of the software. These adjustments are described in Learning Outcomes document for the course. General education (Economics 200) Since Fall 2011 Economics 200 students have answered a fixed battery of questions (originally 35 but reduced to 25 in Spring 2012) at the start of the course and again at the end of the course. The purpose is to test the adequacy of their understanding and the contribution of the course. Economics 200 is the only course for which the department uses both a pre test and a post test. Among the 25 questions, 6 assess Critical Thinking, 13 assess Business Knowledge, and 6 assess Social Responsibility. In 2012 13: average performance in Critical Thinking rose from 33% to 57% over the course of the semester; average performance in Business Knowledge rose from 39% to 64%; and average performance in Social Responsibility (i.e., policy issues) rose from 31% to 57%. More complete results appear in the Learning Outcomes document for the course, but historical comparisons are questionable because 2012 13 was the first year in which all sections of Economics 200 took the assessment tests. Expectations are generally lower for Economics 200, which is taught in large sections to freshmen and sophomores who mostly do not become Economics majors. Much of the enrollment in Economics 200 represents students who hope to enter the Eller College. The pre test and post test data show that students learn much in Economics 200, but their overall performance on the post test is still somewhat below expectations. The department may wish to review whether the questions chosen to assess Business Knowledge provide the best representation of the department s four Business Knowledge subgoals. It may also make sense to adjust the composition of the 25 questions, to place greater weight on Social Responsibility. Students in the small Honors section of Economics 200 showed considerably less improvement between the pre test and post test. Their average score on the post test was only 65%, only slightly better than the average of 63% in the regular sections.

University wide goals for General Education In addition to covering several of the Business Knowledge subgoals for Eller s undergraduate major, Economics 200 should help students to achieve the learning goals for Tier 1 General Education in Individuals and Societies. The university has adopted, at two different times, two separate sets of goals for those courses. General goals adopted in 2006 are that the student should be able to: think critically, communicate effectively, understand and value differences, and use information effectively, with each general goal elaborated through subgoals. An earlier set of goals for Tier 1 courses in Individuals and Societies require (in full) that the student: understand more clearly issues of self identity, social difference and social status, and the effects of major institutions on individual experiences; demonstrate knowledge of the formal and informal structures and processes that make social systems, governments, and economies work; have an informed opinion about socio cultural problems and issues, which can be expressed orally or in writing, and based on knowledge about social, cultural, political, economic, philosophical, and religious theory; demonstrate a well developed critical faculty for distinguishing among the various theoretical and ideological interpretations of world events as they are presented in the media. The department's current assessment process has not addressed the scope and depth of Economics 200's contribution to these university wide learning goals.