Student Learning Outcomes Assessment Report Rutgers Business School Newark and New Brunswick April 2012 Rutgers Business School Newark and New Brunswick (RBS) follows the AACSB International Standards of Assurance of Learning, and continuously assesses student learning outcomes. RBS Assessment Committee, consisted of faculty, students, administrators, and a Board member, met once every semester. The Committee reviews the assessment plans, tools, results and provides improvement suggestions. Assessment Plan and Learning Goals Under the leadership of the co-chairs of the Assessment Committee, each academic program at RBS has developed a five-year assessment plan in 2009 (see Appendix A). During the plan development process, program directors reviewed the curriculum and linked assessment learning goals with the curriculum. The Plan includes program learning goals, assessment tools, assessment cycle, and the implementation plan. In the assessment plan, each learning goal is scheduled to be assessed at least twice within the five-year period. The Assistant Dean for Academic Programs and Assessment has worked with the program directors to implement this plan. Program learning goals and objectives were reviewed and revised to fit the specialty of each academic program in Fall 2010. Faculty, students, alumni, board members and other stakeholders were invited to review the program learning goals and objectives via emails, online surveys, and meetings by the program directors. They provided valuable feedback which assisted the program directors to revise the learning 1
goals and objectives. The learning goals and objectives were clearly defined and publicly posted on the program websites (see Table 1). Table 1. Program Learning Goals URLs Academic Program Learning Goals URL Undergraduate Programs http://www.business.rutgers.edu/sites/default/files/user_files/learnin g-goals-undergraduate.pdf MBA Programs Master of Accountancy in Financial Accounting Master of Accountancy in Governmental Accounting Master of Accountancy in Taxation Master of Quantitative Finance Program Master of Information Technology Program Ph.D. Program http://www.business.rutgers.edu/sites/default/files/user_files/learnin g-goals-mba.pdf http://www.business.rutgers.edu/sites/default/files/user_files/learnin g-goals-maccy-fa.pdf http://www.business.rutgers.edu/sites/default/files/user_files/learnin g-goals-maccy-ga.pdf http://www.business.rutgers.edu/sites/default/files/user_files/learnin g-goals-maccy-tax.pdf http://www.business.rutgers.edu/sites/default/files/user_files/learnin g-goals-mqf.pdf http://www.business.rutgers.edu/sites/default/files/user_files/learnin g-goals-mit.pdf http://www.business.rutgers.edu/sites/default/files/user_files/learnin g-goals-phd.pdf Courses at RBS are offered in two formats case-oriented and lecture-oriented. The lecture-oriented courses are designed principally to convey theoretical knowledge and concepts to students. Thus, they tend to be focused on quizzes and exams, with an occasional term paper thrown in. The learning goals here are designed to fill in knowledge gaps and to ensure that students acquire the basic competence for the program. Most of the core courses in a program are of this nature. The case-oriented courses are 2
designed to reinforce the basic concepts covered in the lecture (core) courses and are mostly elective courses. The learning goals emphasize presentation skills, written and oral communication skills, and demonstration of the ability to apply the basic concepts to real-world situations. Assessment Tool, Implementation and Results We have adopted standardized tests, developed course-embedded tests and our own rubrics to assess student learning outcomes across the undergraduate and graduate programs in Newark and New Brunswick. Below are selected examples of direct assessments. We administered the Major Field Test in Business (developed by the Educational Testing Service ETS) to undergraduate students in Business Policy and Strategy course in Spring 2012 and Fall 2011 to assess our student learning outcomes in Business Knowledge. The results of these tests were compared to the performance of other students in accredited business schools across the country. Multiculturalism and Diversity Learning Goal was assessed in the Principles of Management course in Fall 2011. Professor Michael Gordon coordinated the assessment process across multiple sections on both Newark and New Brunswick campuses. Multiple-choice questions were embedded in the mid-term and final exams. RBS has also developed internal tests to assess MBA students Business Knowledge. The course-embedded tests were designed by our MBA core curriculum faculty members. The assessments were conducted while students were enrolled in the MBA core courses in Fall 2011 and Spring 2012. The core courses are Accounting for 3
Managers, Marketing for Decision Making, Financial Management, Organization Behavior and Information Technology for Managers. Carter Daniel, the RBS Business Communication Director, developed a Written Communications assessment rubric. This rubric was applied in writing assignments in the MBA Business Communications course in Fall 2011. Pre- and post-assessments were conducted at the beginning and at the end of the semester. The improvement in results provided a measure of the value added by the course. Each assessment practice was implemented as scheduled. The assessment implementation and results were reported accordingly. An annual assessment report was developed at the end of each academic year. It summarized the program assessment results by each learning goal as well as the improvement suggestions (see Appendix B). The report was distributed to the Assessment Committee, Program Directors, and Department Chairs at RBS. In addition to the direct assessments, we have also applied indirect assessment tools. We have been implementing the use of the EBI exit survey for the undergraduate and MBA programs on both campuses. In addition, we have begun to implement exit surveys for the Master of Accountancy programs, the Master of Quantitative Finance program and MBA in Professional Accounting programs. These indirect assessment results assisted program directors to understand and meet students expectations. Response to Assessment Results and Successful Improvement The annual assessment report included not only the assessment results but improvement suggestions as well (see Appendix B). The Assessment Committee, 4
Program Directors and Department Chairs reviewed the assessment results and made decisions on improvement activities. Undergraduate Major Field Test Administration Reforms and Performance Improvement RBS used to administer the Major Field Test in Business to the Business Policy and Strategy students in every spring semester to assess our students Business Knowledge. The results of the Major Field Test in Business were compared to the performance of other students in accredited business schools across the country. We analyzed the reports and conducted additional statistical analyses to evaluate student learning in order to improve our program quality. During a recent analysis of results in Spring 2011, we found out that we needed to enlarge the sample size because test takers in the Spring semester were not representative of the entire population of the undergraduate student body. Hence, we decided to administer the test to the Business Policy and Strategy students in the Fall and Spring semesters on both Newark and New Brunswick campuses. In addition, a new assessment process was implemented in Spring 2011. We created a plan to address the scheduling, motivation, communication concerns, and encouragement. First, the exam administration was scheduled three months in advance. Students on Newark and New Brunswick campuses were informed that the exam was scheduled in April. Second, the course instructors were invited to participate in the planning process to determine a motivational technique that will help students participate and take the exam seriously. The course instructors agreed to administer the ETS test 5
during class meeting time and agreed to give a five-point extra credit based on student performance. In February, the facilitators visited each Business Policy and Strategy classroom on both campuses to discuss the exam, its breakdown, provide sample questions, and explain the extra credit points. During the explanation, students were informed of the significance of the assurance of learning and the importance of their participation. Newark students were contacted via email a few weeks after their in-class test overview to inquire if they had additional questions. A week prior to the exam, the students received an email from the Associate Dean of the Undergraduate Program, to remind them of the date of the exam and the importance of their participation. In Spring 2011, the Newark average score was 152.63 (175 students completed the exam). The highest score earned was 184 and the lowest score was 120 (MFT total score range is from 120 to 200). The New Brunswick average score was 167.10 (186 students completed the exam). The highest score was 194 and the lowest score was 128. Table 2 shows that average scores of both Newark and New Brunswick increased from the previous year. The percentile indicates the percentage of institutions at or below the lower limit of the score, comparing with the national data provided by ETS. The percentiles for both campuses were increased from the previous year as well. Table 2. Undergraduate 2010 and 2011 New Brunswick and Newark Mean Percent Correct 2010 Mean (Percentile) 2011 Mean (Percentile) Newark Total Test Scaled Score 147 (20%) 153 (50%) New Brunswick Total Test Scaled Score 159 (85%) 167 (95%) 6
MBA Program Course-Embedded Assessment and Improvement Actions MBA Academic Director, Alex Sannella, introduced the approach of developing course-embedded questions to assess Finance, Accounting, Marketing, Organization Behavior, and Information Systems knowledge in the MBA core courses in Fall 2011 and Spring 2012. Most of the results were satisfactory (see Table 3). However, Accounting and Finance areas needed improvements. Table 3. MBA Business Knowledge Assessment Percent Correct in Fall 2011 and Spring 2012 Accounting Assessment Indicator Percent Correct Financial Accounting 85% Cost Volume Profit 78% Product Costing 40% Finance Assessment Indicator Capital Structure 58% Risk and Return 98% Capital Budgeting 90% Investments 85% Marketing Assessment Indicator Knowing the customer 93% Setting goals 92% Marketing mix plan 86% Organization Behavior Assessment Indicator Leadership 95% Motivation 96% Ethics and Decision making 98% Team Effectiveness 97% Personality and Values 97% Information Technology Assessment Indicator Information Systems for Competitive Advantage 97% Information Systems within Organizations 89% System Development 86% Information Systems Management 92% The results of the Accounting assessment indicated that students mastered financial accounting knowledge very well, but needed more solid managerial accounting knowledge. Due to the limited time of the Accounting for Managers course, an elective 7
course Managerial Accounting is to be offered to, not only students in Accounting concentration, but also all MBA students who need it. Finance assessment results indicated that capital structure area was not satisfactory. Upon reviewing the assessment question, Finance and Economics Department Chair, Professor Ivan Brick, suggested revise the assessment question to clarify the question which is more appropriate for the assessment purpose. The original question was: Bryan invested in Bryco, Inc. stock when the firm was financed solely with equity. The firm is now utilizing debt in its capital structure. What does Bryan need to do to unlever his position? The revised question is: Bryco, Inc. is financed solely with equity. The firm is considering utilizing debt in its capital structure. Delineate two factors that favor utilization of debt? This revised question will be used for assessment next time. Professional Accounting Program Student Exit Interviews and Curriculum Revision The MBA in Professional Accounting program conducted exit interviews with the Class of 2010. Thirty-six percent of the class participated. Table 4 summarizes the exit interview comments as well as the responses regarding the curriculum. The program curriculum was reviewed and revised based on the feedback. Table 4. Summary of Professional Accounting Class 2010 Exit Interview Comments and Reponses Comments Responses Too CPA focused This is the purpose of the program and cannot be changed Have more electives for individuals with We are looking into the possibility of advance standing in required PA classes. having the MBA PA students register for online courses offered in the Master of Accountancy Program including Valuation and Forensic Accounting. 8
Increase EXCEL applications throughout the program. After the computer workshop, EXCEL is hardly used and EXCEL is very important in day to day work in the accounting profession. The program needs a financial statement analysis course. Teach more theory in the Introduction to Financial Accounting course. EXCEL assignments were added in several key accounting courses beginning in the Summer 2010 semester. We just received the Chairperson's approval to add a Financial Statement Analysis course. We will offer one tax course in the Fall that will cover both individual and corporate taxation. This will remove the separate corporate tax class and free up the three credits for Financial Statement Analysis course in the second Summer term. The accounting instructor this summer integrated more theory into the course. RBS periodically reviews assessment plans, process and tools, as well as learning goals and objectives to align with not only the School s mission and vision, but AACSB International Standards of Assurance of Learning as well. We continuously make efforts to improve student learning across all academic programs in Newark and New Brunswick. 9
Appendix A Academic Program Assessment Plans The program core courses and assessment tools are used to assess learning goals. The assessment committee co-chairs, the senior academic program manager, and program directors worked together, and developed a five-year assessment plan for each academic program. Rutgers Business School has two undergraduate level programs, nine master s level programs, and one doctoral level program. Each program developed an assessment plan according to the AACSB Assurance of Learning standards, and RBS mission. Undergraduate Programs Newark and New Brunswick Campuses Undergraduate programs at Newark and New Brunswick campuses provide students with a strong foundation in the broad functional areas of business. These programs are intended to equip graduates with the skills needed to enter the work force as professionals and deal ethically and effectively with the myriad of complex problems that arise in business. Since these two undergraduate programs have the same learning goals and similar curricula, they developed the assessment plan together. 10
Table A.1. Undergraduate Programs Assessment Plan Learning Goals Communication Business Knowledge Quantitative Critical Thinking AY 09/10 Major Field Test - Spring AY 10/11 Business Forum - Fall Major Field Test - Spring AY 11/12 Major Field Test - Spring AY 12/13 Business Forum - Fall Major Field Test - Spring AY 13/14 Major Field Test - Spring Production and Operations Mgmt Spring Production and Operations Mgmt Spring Managerial Accounting -Fall Managerial Accounting - Fall IT Ethics Multiculturalism and Diversity Management Business Principles of Information Forum Management - Systems - Fall Spring Spring Management Information Systems Business Forum Fall Business Forum Fall Principles of Management 11
MBA Program Rutgers Business School s MBA program offers innovative curricula across the board, with concentrations in finance, information technology, management and global business, marketing, pharmaceutical management and supply chain management. Students opt to enroll full time or part time, and courses are offered at Newark and New Brunswick campuses. The MBA program has four learning goals and the five-year plan is developed to assess these learning goals. Table A.2. MBA Program Assessment Plan Learning Goals AY 09/10 Communication Business Communications PT MBA - Fall Business Knowledge Major Field Test FT MBA AY 10/11 Major Field Test FT MBA AY 11/12 Integrated Business Applications FT MBA Fall and Spring Major Field Test FT MBA AY 12/13 Major Field Test FT MBA AY 13/14 Business Communications PT MBA - Fall Major Field Test FT MBA Critical Thinking Major Field Test FT MBA Operations Analysis FT MBA Fall Major Field Test FT MBA Major Field Test FT MBA Operations Analysis FT MBA Fall Major Field Test FT MBA Major Field Test FT MBA Ethics Business Ethics FT MBA - Fall Business Ethics FT MBA - Fall 12
MBA in Professional Accounting Program The MBA in Professional Accounting Program is specifically designed to train graduates holding degrees in the Arts and Sciences for careers in accounting. The program integrates management skills with accounting theory and practice. Table A.3. MBA in Professional Accounting Program Assessment Plan Learning Goals AY 09/10 AY 10/11 AY 11/12 AY 12/13 AY 13/14 Communication Business Communication - Summer 2009 Business Communication - Summer 2010 Business Knowledge Accounting Research Course: Capstone Course. ETS Major Field Test - Summer 2010 Accounting Research Course: Capstone Course. ETS Major Field Test - Summer 2011 Accounting Research Course: Capstone Course. ETS Major Field Test - Summer 2012 Accounting Research Course: Capstone Course. ETS Major Field Test - Summer 2013 Critical Thinking Intermediate Accounting Fall 2009 Intermediate Accounting Fall 2009 Intermediate Accounting Fall 2009 Ethics Business Ethics - Spring 2010 Business Ethics - Fall 2012 13
Executive MBA and International Executive MBA Programs The Executive MBA program s mission is 1) to re-engineer course material every year to reflect changes in the global business environment, and 2) to guarantee Executives an MBA education that is relevant, practical, and timely. International Executive MBA classes are offered in China (Beijing and Shanghai), and Singapore. These two identical programs share four learning goals and one assessment plan. Table A.4. EMBA and IEMBA Programs Assessment Plan Learning Goals AY 09/10 AY 10/11 Communication Executive Skills Business Strategy Business Knowledge Critical Thinking Aggregate Economic Analysis; Managerial Economics; Analytical Techniques Ethics AY 11/12 AY 12/13 AY 13/14 Executive Skills Business Strategy Aggregate Economic Analysis; Managerial Economics; Analytical Techniques Aggregate Economic Analysis; Managerial Economics; Analytical Techniques Aggregate Economic Analysis; Managerial Economics; Analytical Techniques Aggregate Economic Analysis; Managerial Economics; Law & Legal Reasoning Law & Legal Reasoning 14
Analytical Techniques Master of Accountancy in Financial Accounting Program This graduate degree program was designed for graduating college accounting majors who intend to become CPAs and need to satisfy the new requirement for professional accounting certification. The underpinning of the program includes the value of the role of the accountant and accounting in the U.S. and global business environment. It provides students knowledge of the skills necessary to be successful in a career in accounting. Table A.5. Master of Accountancy in Financial Accounting Program Assessment Plan Learning Communication Business Critical Thinking Ethics Goals Knowledge AY 09/10 22:835:628-- Advanced Accounting Research Summer 2009 22:835:628-- Advanced Accounting Research Summer 2009 22:835:628-- Advanced Accounting Research Summer 2009 AY 10/11 22: 372:619-- Ethics in Business - Fall 2010 AY 11/12 22:835:628-- Advanced Accounting Research Summer 2011 22:835:628-- Advanced Accounting Research Summer 2011 22:835:628-- Advanced Accounting Research Summer 2011 AY 12/13 22: 372:619-- Ethics in Business Fall 2012 AY 13/14 22:835:628-- Advanced Accounting Research Summer 2013 22:835:628-- Advanced Accounting Research Summer 2013 22:835:628-- Advanced Accounting Research Summer 2013 15
Master of Accountancy in Governmental Accounting Program This program is designed to prepare professionals seeking to advance in the rapidly expanding field of government financial management, accounting, and auditing. This program focuses on developing graduates with strong communications and decision-making skills, experienced in leveraging and designing technology and information systems, with broad insight into the accounting, financing, and operational needs of governmental entities. Table A.6. Master of Accountancy in Governmental Accounting Program Assessment Plan Learning Communication Goals AY 09/10 010:512 Ethical Issues in Public Financial Management Business Knowledge AY 10/11 010:604 Design and Development of Information Systems AY 11/12 010:525 Government Budgeting AY 12/13 010:654 Public Sector Auditing Systems Critical Thinking 010:525 Government Budgeting Systems AY 13/14 010:551 Governmental Accounting and Auditing Ethics 010:512 - Ethical Issues in Public Financial Management 010:654 Public Sector Auditing 16
Master of Accountancy in Taxation Program The Master of Accountancy in Taxation imparts a basic knowledge of the major technical, conceptual, and research issues in the area of taxation, and provides training in the practical and procedural aspects of taxation. Students explore the ethical and legal obligations of practicing in the area of taxation. Table A.7. Master of Accountancy in Taxation Program Assessment Plan Learning Communication Business Critical Ethics Goals Knowledge Thinking AY 09/10 Business Ethics Final Paper Business Ethics AY 10/11 Cumulative Exam AY 11/12 Business Ethics Final Paper Business Ethics AY 12/13 Cumulative Exam AY 13/14 Business Ethics Final Paper Business Ethics Master of Quantitative Finance Program The Rutgers Master of Quantitative Finance Program is designed to prepare graduates for employment in this interdisciplinary, technologically sophisticated, and specialized field. Students opt to enroll part-time or full-time. Courses are offered in Newark and New Brunswick campuses. Table A.8. Master of Quantitative Finance Program Assessment Plan Learning Goals AY 09/10 Communication Business Knowledge Critical Thinking Investment Analysis & Management (22:839:603) Ethics Investment Analysis & Management (22:839:603) AY 10/11 Financial Modeling I Options 17
(22:839:571) (22:390:609) AY 11/12 AY 12/13 AY 13/14 Investment Analysis & Management (22:839:603) Investment Analysis & Management (22:839:603) Financial Modeling II (22:839:662) Numerical Analysis (22:839:510) Investment Analysis & Management (22:839:603) Investment Analysis & Management (22:839:603) Master of Information Technology Program This program is intended primarily for students who will later pursue doctoral study in information technology or accounting information systems. The program also prepares students to manage significant software development projects and lead teams of information technology professionals engaged in design, analysis, implementation, and service delivery. Table A.9. Master of Information Technology Program Assessment Plan Learning Goals AY 09/10 AY 10/11 AY 11/12 AY 12/13 Communication Information Systems Security (198:643) - Spring 2010 Business Knowledge Advanced Auditing and Accounting Information Systems (835:626) - Spring 2010 Critical Thinking Capstone Project (198:686) - Summer 2012 Capstone Project Ethics Professional Development Seminar - Fall 2010 Professional Development Seminar - Fall 2011 18
AY 13/14 Information Systems Security (198:643) - Spring 2014 Advanced Auditing and Accounting Information Systems (835:626) - Spring 2014 (198:686) - Summer 2013 Ph.D. Program The Ph.D. program trains leaders in business scholarship and allows doctoral students to collaborate with world-class professors who have expertise in a broad range of research areas. Due to the program s unique academic requirements, the PhD program developed its plan to assess the following six learning goals. Table A. 10. Ph.D. Program Assessment Plan Learning Goals Goal 1: The acquisition of advanced knowledge in areas of specialization. Goal 2: The development of advanced theoretical or practical research skills for the area of specialization. Goal 3: Explicit attention to the role of the specialization areas in managerial and organizational contexts. Goal 4: Preparation for teaching responsibilities in higher education. Demonstration of Achievement The curriculum for our doctoral students normally includes 5 major courses. We maintain and publicly post on the web a syllabus for each course offered through the program. Students learn research skills in methodology courses (each major requires students to take 4 methodology courses) and through working with faculty on their first and second early research requirements and their dissertation proposal. Syllabi for methodology courses taught through the program are posted on our web site, and the program office maintains a file of early research papers and dissertation proposals. This differs by major. Each of our students, even those who are not teaching assistants, is expected to teach at least one section in the school before they graduate. Students are required to attend a 19
Goal 5: Dissertation, or equivalent, demonstrating personal integration of, and original intellectual contribution to, a field of knowledge. Goal 6: Our doctoral students make original research contributions. teacher training seminar during at least year. The student must prepare a teaching portfolio that documents their teaching experience and includes student and peer teaching evaluations. The program office maintains a file of the teaching portfolios. The program office maintains a file of completed dissertations. We document publications by current students and alumni. Assessment Cycle of Exams and Surveys In addition to the program-based assessment plan, Rutgers Business School plans to administer direct and indirect assessment practices. The undergraduate seniors (both Newark and New Brunswick campuses) and MBA second year students are scheduled to take the Major Field Test in Business every year. The test results will be compared with those of the ETS comparison groups. The exit surveys will be used to assess student learning experience. The undergraduate seniors and MBA graduating students will be asked to respond to the EBI exit surveys prior to their graduation. The survey results will be compared with those of EBI selected comparison groups. Graduating students in other programs will also be asked to complete an self-developed online exit survey. Rutgers Business School is developing an alumni database. RBS plans to survey its alumni every three years. EBI alumni surveys for undergraduate and MBA will be used and the results will be compared with selected groups. A self-developed alumni survey will be used for other programs. The results of exams and surveys will be used for program and student learning improvement. 20
Table A. 11. Assessment Cycle of Exams and Surveys AY 09-10 AY 10-11 AY 11-12 AY 12-13 AY 13-14 Major Field Test in Business X X X X X (Undergraduate) Major Field Test in Business (MBA) X X X X X EBI Exit Survey (Undergraduate) X X X X X EBI Exit Survey (MBA) X X X X X Exit Survey (other programs) X X X X X EBI Alumni Survey (Undergraduate) X X EBI Alumni Survey (MBA) X X Alumni Survey (other programs) X X Implementation of Assessment Plans and Continuous Improvement The senior academic program manager will coordinate with program directors to implement this assessment plan. Program directors will work with instructors to develop assessment tools for all learning goals. Program directors will summarize assessment activities and results, and improvement activities and evidence to support these activities every semester. A semi-annual assessment report will be written and submitted to the Assessment Committee in spring and fall semesters. The Committee will review the report and make suggestions for continuous improvement. Program directors will take actions according to improvement suggestions and revise assessment plans or tools if necessary. Academic programs will be reviewed periodically. Learning goals, curricula, assessment plans, and assessment tools shall be reviewed comprehensively during the program review period. Each academic program reviews its program at least once every five years. The program review may occur as often as needed. 21
The program review process may start with the formation of the review committee. The committee is composed of faculty, students, administrators, alumni, employers, advisory board members or business community. The committee members can be appointed, voluntary, or invited. The program director leads the review process. The program review committee evaluates, develops, or revises learning goals, curricula, assessment practices and tools, and resources. The following should also be included in the review process: 1) the admission standards to the program, 2) individual course structure and content, 3) alignment of learning goals and RBS mission, 4) student learning achievement and experience, 5) feedback from employers and alumni, and 6) career placement statistics. A report and recommendations shall be submitted in writing to the senior associate dean for assessment. The program director shall ensure the improvement recommendations are implemented upon the completion of the review. The following graphic is the closing the loop flowchart. The RBS mission statement is the center of this flowchart. Assessment activities are developed to assure student learning at RBS. 22
Student Learning Assessment Plan Development/Review/Revise of Learning Goals, Curricula, and Assessment Tools RBS Mission Assessment Data Collection and Analysis Assessment Evidence and Reports Figure 1. Assessment Flowchart 23
24
Appendix B Annual Assessment Report Rutgers Business School Learning Assessment Summary Summer 2010, Fall 2010, and Spring 2011 Undergraduate Programs Program Learning Goal Assessment Results Improvement Suggestions Undergraduate- Newark Business Knowledge Written Communication ETS Major Field Test total mean score was 153, at the 50 th percentile compared to national results. The scores in all areas increased compared to those in previous years. Finance assessment indicator scored the highest at 90 th percentile. Legal and Social Environment scored lowest at 35 th percentile. The results indicated that the average scores of the four items assessed in the first assignment in Newark were between 2.0-2.7, but the average scores in the last assignment were between 4.3 and 4.9. The results indicated that all students in Newark met or exceed expectations in the last assignment. 25 After reviewing the performance data, there are several standards that need to be implemented annually: the 5- point extra credit for performance, splitting the exam over two days, clear exam organization, and student encouragement. The MFT needs to be clearly organized from its introduction to the students, to test administration, and to student follow up upon exam completion. A plan and calendar needs to be arranged that details when the exam will be given, when the facilitators will visit the classrooms to introduce the exam to the students and give sample questions with answer key, and when students will be notified of their extra credit bonus points. In the future, the instructors will spend more time on these two items and provide more in-class practices to improve macro-organization and quality of writing techniques. Ethics The results showed that 91% In the future, the lecture will simplify the ethical
Undergraduate- New Brunswick Critical Thinking Quantitative Skills Business Knowledge students meet the business ethics learning goal. The overall mean score is 8.6 (Proficient). The result is satisfactory. The overall mean score of all six questions is 2.38 (out of 4), which corresponds with the evaluation category of satisfactory. The overall mean score of all five questions was 3.17 (out of 5). The result met our expectation. ETS Major Field Test total mean score was 167, at the 95 th percentile compared to national results. The scores in all areas increased compared to those in previous years. Finance and Information Systems assessment indicators scored the highest at 95 th decision making model to make it more user-friendly for students as they apply it to the case study. Although the result is satisfactory, suggestions of improvement were made. First, the questions will be reviewed and modified/re-worded to be clearer, so that it is easier to understand the questions without confusion. Second, students need to be given more exercises to apply in-depth critical thinking skills in the class. In the future, quantitative questions may be considered to test critical thinking skills. For example, giving students quantitative questions involving revenues, costs, volumes, and break-even points that require them to analyze, work with numbers, apply some logic. The following suggestions may be considered for continuous improvement. First, the course syllabus of all sections should be consistent, so that all instructors cover the same materials. Second, the five questions should be covered in the textbook, so that students are able to know the answers. Third, students who need improvement may need to take additional quantitative courses to meet this learning goal. After reviewing the performance data, there are several standards that need to be implemented annually: the 5- point extra credit for performance, splitting the exam over two days, clear exam organization, and student encouragement. The MFT needs to be clearly organized from its introduction to the students, to test administration, and to student follow up upon exam completion. A plan and calendar needs to be arranged that details when the exam will be given, when the 26
Written Communication Critical Thinking percentile. Legal and Social Environment scored lowest at 45 th percentile. In New Brunswick, the average scores were between 2.4 and 3.0 in the first assignment, but the average scores were between 3.7 and 4.5 in the last assignment. Ninety-one percent students in New Brunswick meet or exceed expectation in the last assignment. The overall mean score of all six questions is 2.04 (out of 4), which corresponds with evaluation category of satisfactory. facilitators will visit the classrooms to introduce the exam to the students and give sample questions with answer key, and when students will be notified of their extra credit bonus points. The average scores of macro-organization and quality of writing on Wednesday section in New Brunswick were relatively lower than other sections. Therefore, in the future, the instructors will spend more time on these two items and provide more in-class practices to improve macro-organization and quality of writing techniques. Although the result is satisfactory, a suggestion of improvement is to make the questions more clear in the future. Many students reported confusion of the questions. It causes difficulty of understanding the questions. Hence, Professor Sannella will review the questions and modify/re-word the questions to be clearer. 27
Graduate Programs Program Learning Goal Assessment Results Improvement Suggestions MBA Business Knowledge Critical Thinking Ethics ETS Major Field Test total mean score was 252, at the 55 th percentile compared to national results. The total mean score decreased compared to the score in previous years. The Accounting assessment indicator scored lowest at 30 th percentile compared to the national data, while Marketing indicator scored highest at 70 th percentile. The average scores for the first learning objective was 28.63 (out of 35), and the second learning objective was 28.85 (out of 40). Hence, on average, our students exceeded our expectation in the first learning objective, and met our expectation in the second learning objective. The overall mean score is 3.18 (out of 4). More than 93 percent students ethics projects are graded as proficient or accomplished. The results Since the test contents do not align with our MBA curriculum, Assessment Committee decided to develop our own measures of learning, which we believe will lead to better measurement of more cutting edge content, based on the latest management literature and on corporate feedback. Starting fall 2011, the following five assessment indicators will be assessed in the course embedded questions. Accounting Finance Management Marketing Information Systems In the future, the instructor will make more efforts on a) case-study based problem solving; and b) class projects developed upon real life operations analysis problems to improve student s critical thinking skills. The results indicate that a majority of part-time MBA students meet the RBS ethics learning goal. Students are meeting the expectation in the areas of case analysis, scientific research and ethical development. 28
indicate that a majority of part-time MBA students meet the RBS ethics learning goal. Program Learning Goal Assessment Results Improvement Suggestions MBA Professional Accounting Written Communication Ethics The results indicates student performance improved to none below expectations, 19 at expectations and 32 exceeding expectations, with one student dropping out of the program. The overall mean score is 3.19 (out of 4) which is above the target score of 3. Although there was significant improvement throughout the semester, we will consider including more initial writing skills instruction at the beginning of the semester in order to reduce the significant percentage of the class performing below expectations. This year the objective multiple choice was used for assessment for the first time. In the future, the new objective system for assessment will be applied. It will allow valid comparisons and improvement plans year-to-year that were less possible under the subjective essay grades which was used in the previous year. Program Learning Goal Assessment Results Improvement Suggestions MACCY Written Taxation Communication Three students exceeded expectations, three students met expectations, and there were no students who failed to meet our expectations. These results are positive, as 100% of the students met or exceeded the quality of writing expectations held at Rutgers Business School. Based on the results of this report, it is evident that students at MACCY-Tax program are writing at or above the level in which we expect. They have very strong research skills, and produce thoughts in an effective and professional manner. Their papers are organized according to a logical thought process and their paragraphs are structured appropriately. Ethics Three essays meet expectations of all four traits; twelve essays meet expectations of three traits; and 16 essays meet expectations It is the first time that this course is offered in the Taxation program. In the future, professors teaching this course can focus on making students 29
of two traits. The average score is 2.58 (out of 4). Ideally, the target for the program in would be in the 3-plus range. examine ethical issues from several different vantage points. More specifically, on a regular basis the instructor should inform students that virtually every tax issue will involve issues that pertain to the taxpayer, IRS, and the tax practitioner. Sometimes the tax issue involved will also involve a taxpayer and their inter-relationship with other taxpayers (e.g., terms of a buy-sell agreement). Given the fact that multiple parties are involved in virtually tax issue that engenders ethical issues, students must be able to identify the inter-play between parties and the bearing this has on resolving difficult ethical issues. Program Learning Goal Assessment Results Improvement Suggestions MACCY Financial Accounting Written Communication Business Knowledge No students exceeded expectations, 11 students met expectations, and there were 4 students who failed to meet our expectations. These results are encouraging, as 73% of the students met the writing communication expectations held at Rutgers Business School. Average scores of Economics Financial Factors, Business Environment, Strategic Relevant Issues, and Accounting Knowledge were 3.09, 3.09, 2.74, and 2.93, respectively, on a 4-point scale. Scores of Economics Financial Factors and Business Environment meet RBS Based on the results, it is evident that the majority of the students enrolled in the MACCY Financial Accounting program are writing at the level in which we expect. They are able to produce thoughts in an effective and professional manner. Their papers are organized according to a logical thought process and their paragraphs are structured appropriately. The quality of writing received the lowest average score in the rubric thus demonstrating a potential area of need. The analysis of the results shows that there is room for improvement, as expected. Many of the items above suggest that the students should be given more detailed instructions to much more explicitly address the business knowledge areas listed in developing and presenting their work. MACCY s Advanced Accounting Research class brings into it 30
Critical Thinking expectation. Average scores of Use Factual Knowledge, Use Analytical Tools, Identification of Relevant Issues, and Innovative Ideas were 3.13, 2.89, 2.29, and 1.81, respectively, on a 4-point scale. Score on the use of factual knowledge is acceptable. a great deal of information that the students were expected to have learned in prior semesters. Next summer, steps should be taken to more heavily emphasize the importance of these background factors, thereby fostering greater student efforts to acquire any information that they may not have thought to provide, and with possession of that information, lead them to integrate it more thoroughly in the development of their project. This will foster their ability to integrate these factors more thoroughly in their future professional lives. Scores on the use of factual knowledge and analytical tool use are acceptable. In the future, student s performance could be enhanced further by requiring them to justify the information (factual knowledge) that they bring in to their projects, and be clear in specifying how that information is relevant to their work outputs. They should also be required to justify their choice of analytical tools, and justify why they chose not to use other commonly used analytical tools. The goal here is to promote their thinking through the usefulness, or lack of same, of various tools used in performing relevant analyses for projects of the kind used in this course. As to the identification of relevant issues, the instructor found that the scores were lower than expected. For next year s class, the instructor suggests that relevant literatures on critical thinking and problem analysis be even more deeply consulted with the goal of developing a lecture for the students on how best to identify relevant issues, how to separate the important 31
Ethics The overall mean score of for all graded activities was 3.37 (out of 4). The mean scores for specific assessed skills were 3.49 for case analysis, 3.47 for ethical development and 3.11 for ethical theory. Again, these scores are on a 4-point scale. This means that on average, students exceeded proficient performance in all assessed skills. information from the huge amounts of less relevant information that may confuse the students. Finally, the innovative idea scores were disappointing. The instructor believes, though, that it is difficult to foster the development of innovative ideas unless you have a very broad knowledge of the field and can understand what would be new, and what is just a repetition of old concepts and ideas, using different words. To address this issue, the instructor believes that an outside speaker should be brought in to help the students learn how to develop innovative ideas and how to apply these ideas in relevant accounting and auditing work settings. Taken together, the instructor believes that the suggested improvements will make the course even more effective than it currently is and will further enhance the probability that the students will go on to richly rewarding careers in accounting. The core aspects of the course will be maintained and supplemented with additional material designed to enhance the level of proficiency of students. Modifications planned include: 1) assigning discussion threads to groups, which are subsets of the entire section, 2) incorporating additional taped video material and other multimedia material into the course to complement the online asynchronous discussions which the students responded to so well, and 3) refining the module that specifically addresses CPA exam specific subjects. 32
Program Learning Goal Assessment Results Improvement Suggestions MACCY Governmental Accounting Critical Thinking Governmental Accounting Knowledge The average scores of learning objective 1 and learning objective 2 are 89.7 and 83.7, respectively. All students are proficient and accomplished in the learning objective 1. Almost 82% students are proficient and accomplished in the learning objective 2. Overall, the results are satisfactory. 72.4% of the students achieved satisfactory results and are classified as accomplished or proficient, indicating that a decent majority of the students meet learning objective one of the business knowledge learning goal. 86.2% of the students achieved satisfactory results and are classified as accomplished or proficient, demonstrating that for learning objective The course instructor will use the assessment results to revise upcoming syllabus in terms of learning goals and to add new materials to address two needed enhancements. The instructor plans to enhance student proficiency in reading critically and writing more organized conclusions from this reading. In addition, the instructor plans to add background materials on organized writing in the syllabus and on the webliography. The weekly threaded discussions will provide immediate feedback on how well these enhancements are working and provide me with opportunities each week to provide feedback on these two points. The instructor will also review the syllabus for any opportunities to limit existing material where students demonstrate clear understanding of that material. In order to improve the proportion of students who achieve satisfactory performance in future online classes, an informal assessment will be performed early in the semester and a personal email will be sent to targeted students, offering additional guidance and encouragement, and seeking feedback from these students. 33
two of the business knowledge learning goal the vast majority of the students meet the objective. Program Learning Goal Assessment Results Improvement Suggestions Master of Quantitative Finance Critical Thinking The average score of the problem was 10 (out of 12). Hence, on average, our students exceeded our expectation. Overall, the results are very satisfactory. Finance Knowledge The average score is 24 out of 30. Based on this result, students have obtained this learning goal. Approximately 83% students meet or exceed our expectation. Overall, the results are satisfactory. In the future, students need to be encouraged to address more questions which require critical thinking and develop quantitative skills. Professor Zuckerman believes that developing critical thinking skills is essential for MQF students in order to better prepare them for the job market. The contemporary finance job market requires not only quantitative skills, but also independent and creative thinking. In the future, the instructor suggests that students read business periodicals or newspapers and collect realtime data to do their own in their course work to further improve their finance knowledge. 34
Rutgers Business School Learning Assessment Summary Summer 2011, Fall 2011 Undergraduate Programs Program Learning Goal Assessment Results Improvement Suggestions Undergraduate- Newark Business Knowledge Multiculturalism and Diversity ETS Major Field Test total mean score was 148 in Fall 2011, at the 34 th percentile compared to national data. The scores in all areas decreased compared to those in Spring 2011. Finance assessment indicator scored the highest at 60 th percentile. Legal and Social Environment scored lowest at 21 th percentile. Learning Objective 1. increasing perspective, sensitivity and understanding of the impact of demographic 35 Newark Percent Correct 77% During the analysis of Spring 2011, we found out that we need to enlarge the sample size because test takers in the Spring semester are not representative. Hence, we decide to administer the test in the Fall and Spring semesters. The decision of offering the test in the Fall semester was made in October. Instructors of Business Policy and Strategy courses were involved in the test arrangement and student motivation. The test was given in November. In the Spring 2011, we provided the students with the handouts in February and the exam was in April. It is possible that students did not have enough time to prepare for the test in Fall 2011. Ample time may make the difference. The performance gap between Newark and New Brunswick campuses was discussed at the department chair s meeting. It is evident that students in the undergraduate programs in Newark and New Brunswick meet the learning goal of multiculturalism and diversity. In the future, the instructor will introduce more class exercises coupled with a classroom