Manual for the Assessment of Curricular and Co-Curricular Student Learning

Transcription

1 Manual for the Assessment of Curricular and Co-Curricular Student Learning 1

2 Iowa Wesleyan University Assessment Manual PART Membership of the Committee on Assessment 1.1 The Committee on Assessment will consist of the Vice President for Academic Affairs, Director for Adult and Graduate Studies, Dean of Students, the Assistant Dean for Wesleyan Studies, Director for Campus Ministry and Service Learning, Director for Career Development and Internships, Assistant Dean of Academic Resources/Library Director, divisional faculty representatives, and one (1) student nominated by the Committee on Assessment and approved by the Representative Student Government. 1.2 Divisional faculty representatives will be appointed by the Executive Committee with the approval of the Vice President for Academic Affairs. Each division shall be represented. The faculty terms will be for three years with one replaced each year. The chair of the Committee on Assessment will be a faculty member, who is appointed by the Vice President for Academic Affairs, in consultation with the President. 2.0 Duties of the Committee on Assessment 2.1 Direct the academic and co-curricular assessment process 2.2 Integrate the assessment standards of the Higher Learning Commission of the North Central Association with the Iowa Wesleyan College assessment program; 2.3 Establish an Assessment Calendar 2.4 Conduct assessment of student learning within the Wesleyan Studies curriculum and the Institutional Learning Outcomes (ILOs) (i.e. Life Skills ) 2.5 Serve as a resource for assessment activities of the major academic and co-curricular programs 2.6 Guide the use of the assessment findings for program planning 2.7 Guide the integration of assessment findings with strategic planning 2.8 Guide the linking of assessment process with the budget process 2.9 Maintain documentation of the academic and co-curricular assessment activity 3.0 Institutional Learning Outcomes to be assessed: A. Communication: Students will show proficiency in acquiring, processing, and transferring information in a variety of ways, including written communication, oral communication, and information literacy. a. Written Communication: Students will develop and express ideas in writing, including working with various genres, styles, texts, technologies, data, and/or images. b. Oral Communication: Students will deliver a prepared, purposeful presentation designed to increase knowledge, to foster understanding, or to promote change in the listeners' attitudes, values, beliefs, or behaviors. 2

3 c. Information Literacy: Students will show the ability to know when there is a need for information, to be able to identify, locate, evaluate, and effectively and responsibly use and share that information for the problem at hand. B. Critical Reasoning: Students will strategically apply critical thinking and problem-solving skills. a. Problem Solving: Students will design, evaluate and implement a strategy to answer open-ended questions or achieve desired goals. b. Critical Thinking: Students will comprehensively explore issues, ideas, artifacts, and events before accepting or formulating an opinion or conclusion. C. Civic Engagement: Students will develop the knowledge, skills, values, and motivation to actively engage in communities to promote social justice and human welfare. a. Civic Engagement: Students will demonstrate their ability to make a difference in the civic life of communities and develop the combination of knowledge, skills, values and motivation to make a difference in the quality of life of those communities. b. Global Learning: Students will critically analyze complex, interdependent global systems and legacies (such as natural, physical, social, cultural, economic, and political) and their implications for people s lives and the earth s sustainability. 4.0 Curricular Assessment Calendar: Institutional Learning Outcomes Information Assess Review Implement Assess Review Implement Literacy & Civic Engagement Written -- Assess Review Implement Assess Review Communication & Critical Thinking Oral Communication, Problem Solving, & Global Learning Assess Review Implement Assess 5.0 Curricular Assessment of Institutional Learning Outcomes Overview 5.1 Direct Assessment Course-Embedded Assessment: Assessment of Institutional Learning Outcomes in Courses A. Each academic program including Wesleyan Studies is responsible for the assessment of at least three Institutional Learning Outcomes as identified on program curriculum maps. B. Assessment Rubrics recommended for use: (see Appendix A) Written Communications -- AAC&U Written Communication rubric Oral Communications -- AAC&U Oral Communication rubric Information Literacy -- AAC&U Information Literacy rubric Critical Thinking -- AAC&U Critical Thinking rubric Problem Solving -- AAC&U Problem Solving rubric 3

4 Civic Engagement -- AAC&U Civic Engagement rubric Global Learning AAC&U Global Learning rubric C. Student artifacts and assessment data are collected and stored in LiveText when appropriate to the assessment method. I. In consultation with Program Assessment Coordinators, Assessment committee identifies courses and outcomes for a semester based on Academic Program Assessment Charts (see Appendix C). II. LiveText (LT) coordinator sends shell assignment to all identified courses. III. Faculty will create an assignment in each class that will be used to assess the identified LO. IV. Students upload assignment in LT Assessment of Institutional Learning Outcomes using Standardized Testing A. Spring Assessment Day One day dedicated to assessment of ILOs using standardized tests. B. Assessment Committee plans and executes testing. C. Assessment Committee reviews data and makes recommendations. Reports will be shared with appropriate parties. 5.2 Indirect Assessment FSSE and NSSE will be administered during the same year every three years Course Evaluations (see Appendix B) A. Distributed to Division Chairs by the VPAAs office. B. All instructors will give course evaluations each term. C. Relevant learning outcome data for the ILOs is reported in the Academic Program Report (see Appendix D). 6.0 Direct Assessment Activities of the Major Academic Programs and Wesleyan Studies 6.1 Definitions Program assessment coordinator: This person is responsible for program assessment. This includes submitting updated Curriculum Maps, Assessment Charts, and Assessment Reports to the Assessment Committee and VPAA in a timely manner and coordinating with program faculty to ensure that all are involved in the assessment process and that LiveText is used to collect assessment data Program Learning Outcomes are specific to the program s field of expertise and defined by the program Institutional Learning Outcomes refer to those identified in Program refers to both program and/or divisional matters. Each academic entity will determine at which level assessment will occur. 6.2 Process Programs are responsible for assessment of Institutional and Program learning outcomes as identified in program curriculum maps. Programs are required to 4

5 assess at least three institutional learning outcomes as identified by the program. Programs are required to assess at a minimum one program learning outcome Programs offered through AGS will be assessed. If appropriate, the same learning outcomes and assessment methods are used regardless of modality Programs are responsible for using assessment data to improve student learning within programs The Assessment Committee serves to facilitate program assessment by providing recommendations and guidelines. The committee reviews program assessment processes to offer advice and guidance LiveText is to be used to collect and assess course embedded assessment when appropriate All academic programs submit an updated Curriculum Map (Appendix E) at the end of every three year assessment cycle (see 4.0). If assessment processes change mid-cycle, a revised document is to be submitted to the Assessment Committee All academic programs submit an Academic Program Assessment Report (Appendix D) and an updated Academic Program Assessment Chart (Appendix C) at the end of every academic year. 6.3 Curriculum Maps (Appendix E) Course Title and Course Number: List of all program courses, regardless of modality It is possible that Programs may not teach and/or assess every LO. Each individual course may not teach and/or assess any LO Teaching LOs: Identify in which course(s) each Learning Outcome is specifically introduced (Beginning), reinforced (Developing), or additionally reinforced (Proficient) by marking the appropriate letter in each cell. A. (B) -- Beginning level: At this level, the instructor introduces new material including concepts, strategies, and vocabulary of the discipline. Student s work begins to demonstrate basic application and comprehension of material. B. (D) -- Developing level: At this level, the instructor reinforces material introduced in prerequisite or beginning level courses, including concepts, strategies, and vocabulary. Additional material that is more complex builds upon that previously learned. Student s work develops beyond the basic level to demonstrate clear application and comprehension of material. C. (P) -- Proficient level: At this level, the instructor continues reinforcing previously introduced material while building upon it with advanced information pertinent to the discipline. Student s work increasingly demonstrates facility with concepts, strategies, and vocabulary of the discipline in consistent and effective ways Assessing LOs: Identify in which course(s) each Learning Outcome is assessed by marking (A) in the appropriate cell. 6.4 Academic Program Assessment Chart (Appendix C) 5

6 6.4.1 Assignment type can be a standardized test or course embedded assessment as designated by the instructor in coordination with the Program; however, the assignment should directly assess the LO Assessment methods will align with LOs. A. Programs are encouraged to use the designated AAC&U rubrics to assess ILOs, though this is not obligatory and rubrics can be modified to fit the program s needs. All programs will assess ILOs using a 5-point scale with anchors: 0-Unsatisfactory, 1-Emerging, 2-Progressing, 3- Satisfactory, 4-Exemplary. Although consistency across programs is encouraged, the goal is not to compare programs on level of performance. Rather, each program is to evaluate level of performance against Program specified Expected Level of Performance within the program. B. Program Learning Outcomes will be assessed using methods deemed appropriate by that program. C. Rubrics, unless AAC&U rubrics, and other assessment methods will be submitted to the Assessment Committee for review Assessment Cycle identifies the year in which assessment data will be collected, reviewed, and improvements implemented. Programs are encouraged to use the institutional assessment cycle that follows a three years assess, review, implement structure (see 4.0). It is not necessary to assess all LOs every academic year Course level corresponds that the Curriculum Map (see Appendix E). Classes are designated as Beginning, Developing, or Proficient (see 7.3.3) Assessment location at a minimum will identify the course in which each LO is assessed. If assessments occur outside courses, this needs to be described here. Additional information may include the faculty s name Expected level of performance corresponds to the benchmark or standard established prior to collecting data for the respective course and LO. This benchmark may be the percent of students expected to meet a certain level on a rubric or indicate a certain response on an indirect assessment. This can also be an average score on a test or survey Summary of Data Analysis provides an overview of direct and indirect assessment data collected during the previous year (if following the institutional assessment cycle) as reported in the Academic Program Assessment Report (see Appendix D) Action plan for improvement summarizes action plans outlined in the Academic Program Assessment Report. Action Plans are to be implemented the subsequent academic year. 6.5 Academic Program Assessment Report (Appendix D) Data A. Direct Assessment Data I. If data were collected in LiveText, provide LiveText graphs for each course. If data were not collected in LiveText, provide appropriate descriptive statistics. 6

7 II. Met: Identify which rubric dimensions and/or learning outcomes met the expected level of performance goal as identified in the Academic Program Assessment Chart. III. Not Met: Identify which rubric dimensions and/or learning outcomes did not meet the expected level of performance goal as identified in the Academic Program Assessment Chart. B. Indirect Assessment Data I. Course Evaluation Data: Report mean and standard deviations for student ratings of each appropriate ILO from Course Evaluations Data Analysis: Provide a reflection and interpretation of assessment data. Use the guiding questions provided Action Plan for Improvement: Provide a detailed description of changes to be implemented during the subsequent academic year, including budget implications. 7.0 Assessment of Non-Academic Program Graduation Requirements 7.1 Direct Assessment of Writing Intensive Courses: Course Embedded Assessment Institutional Learning Outcome to be assessed: Written Communication Assessment responsibility: Assistant Dean of Wesleyan Studies Courses to be assessed: Writing Intensive courses Assessment Cycle: Assessment follows the institution-wide assessment cycle (See 4.0) Assessment Rubric to be used: AAC&U Written Communication rubric Process: Assessment procedures outlined in 7.0 will be followed as appropriate. 7.2 Direct Assessment of Service Learning (S-L) Courses: Course Embedded Assessment Institutional Learning Outcome to be assessed: Civic Engagement Assessment responsibility: Director of Service Learning Courses to be assessed: All Service Learning-designated courses Assessment Cycle: Assessment follows the institution-wide assessment cycle (See 4.0) Assessment Rubric to be used: AAC&U Civic Engagement rubric Process: Assessment procedures outlined in 7.0 will be followed as appropriate. 7.3 Internship Requirement Direct Assessment: No institutional process currently A. NOTE: Programs may use internship courses to directly assess institutional and/or program learning outcomes. Responsibility for collecting assessment data, reviewing data, and implementing changes lies with the respective program Indirect Assessment A. All Institutional Learning Outcome are be assessed (see 3.0) B. Assessment responsibility: Director of Internship C. Courses to be assessed: All Internship-designated courses 7

8 D. Assessment Method: Self-report of mastery using likert-type scale by the student and site supervisor at completion of the internship E. Assessment Cycle: I. Assess: Every semester II. Review: The Director of Internship will examine assessment data every May and report to each program? III. Implement: Programs will implement change in coordination with the Director of Internship? F. Procedure I. All students and site supervisors complete assessments in LiveText (LT). II. LiveText platform sends midpoint and final assessment ratings to supervisor and student to complete. 8

9 APPENDIX A: ASSESSMENT RUBRICS 9

10 10

11 11

12 12

13 13

14 14

15 15

16 APPENDIX B Course Evaluation: Indirect Measure of ILOs Please read each statement carefully, and then select one of these 5 alternatives: A = strongly agree, B = agree, C = disagree, D = strongly disagree, E = not addressed Life Skills This course helped develop my ability to: 10. Express ideas in writing (Written Communication) 11. Prepare and deliver oral presentations, such as class discussion, debates, prepared contributions to class discussion, posing questions (Oral Communication) 12. Identify, locate, evaluate, and/or use information (Information Literacy) 13. Design, evaluate, and/or answer open-ended questions or problems (Critical Thinking) 14. Explore issues, ideas, and events before accepting opinions or conclusions (Problem-solving) 15. Make a difference in the quality of life of communities (Civic Engagement) 16. Analyze how global systems are interconnected and impact people s lives (Global Learning) NOTE: If ILOs are not assessed in a class, students have the option to indicate that the learning outcomes was not addressed. Means will not be calculated for questions in which three or fewer students indicate a rating other than not addressed. 16

17 Program: Program Assessment Coordinator: APPENDIX C IW Academic Program Assessment Chart Year: Institutional Learning Outcome #1: Communication: Students will show proficiency in acquiring, processing, and transferring information in a variety of ways. Learning Outcome Assignment Type Assessment Method Assessment Cycle Course Level Assessment Location Written Communication Expected Level of Performance Summary of Data Analysis Action Plan for Improvement Oral Communication: Information Literacy 17

18 Institutional Learning Outcome #2: Critical Reasoning: Students will strategically apply critical thinking and problem-solving skills. Learning Outcome Assignment Type Assessment Method Assessment Cycle Course Level Assessment Location Summary of Data Analysis Problem Solving Expected Level of Performance Action Plan for Improvement Critical Thinking Institutional Learning Outcome #3: Civic Engagement: Students will develop the knowledge, skills, values, and motivation to actively engage in communities to promote social justice and human welfare. Learning Outcome Assignment Type Assessment Method Assessment Cycle Course Level Assessment Location Summary of Data Analysis Civic Engagement Expected Level of Performance Action Plan for Improvement Global Learning 18

19 Program Learning Outcomes: Learning Outcome PLO #1 Assignment Type Assessment Method Assessment Cycle Course Level Assessment Location Expected Level of Performance Summary of Data Analysis Action Plan for Improvement PLO #2 PLO #3 19

20 Learning Outcome PLO #4 Assignment Type Assessment Method Assessment Cycle Course Level Assessment Location Expected Level of Performance Summary of Data Analysis Action Plan for Improvement PLO #5 PLO #6 20

21 NOTES: Please explain any additional information about your assessment plan that you would like assessment committee to know. 21

22 APPENDIX D IW Academic Program Assessment Report Program: Year Reviewed: LEARNING OUTCOME: Beginning Course Level: [list course(s)] DATA Direct Assessment Data: LiveText or Other MET: NOT MET: Indirect Assessment Data: Course Evaluations Mean: Standard Deviation DATA ANALYSIS Analyze the data and reflect on why students did not meet the expected level of performance. Consider each rubric dimension where students did not meet the expected level of performance. The questions below are meant as prompts to guide your interpretation of your data. Not all questions will be applicable or addresses. Guiding Questions: 1. Could low student ratings be due to problems with assessment? a. Was the assignment or artifact used appropriate for the rubric or assessment tool? Did the assignment capture most of the dimensions on the rubric or assessment tool? b. Were the rubrics used appropriately by the raters? (e.g., were they trained well enough on use of LiveText and how to interpret and use each dimension of the rubric) c. Is this the right course to teach and assess this learning outcome? 22

23 d. Is the expected level of student performance too high or too low? 2. Could low student ratings be due to teaching methods? Remember to consider what IS working so you can continue that, as well as consider what IS NOT working and needs changed. a. Consider what you are doing now to teach this learning outcome. What methods of instruction, tools, scaffolding are offered to students? b. What change could be implemented to improve instruction? State what will be done, by whom, and when. Things to consider: What scaffolding or supports do students need to improve in this LO? (These could come from within or outside your program) What supports or training will faculty need to improve ability to teach this LO? What resources (equipment, supplies, space, technology) are needed? 3. Could low student ratings be due to problems with curriculum or program requirements? a. Do you need to change how a course is taught, or what content is included in the course? b. Do you need to change which courses are required, or the sequence of courses taken (prerequisites), or perhaps add a course? c. Are students sufficiently prepared for this course, or do previous courses need to be changed to provide scaffolding or develop skills in preparation for this course? ACTION PLAN FOR IMPROVEMENT Summarize changes to be implemented in the coming academic year (Consider possible curriculum, personnel, equipment or materials needed, training, academic supports, or assessment changes): Budget requested to make these changes: Developing Course Level: DATA Direct Assessment Data: LiveText or Other MET: 23

24 NOT MET: Indirect Assessment Data: Course Evaluations Mean: Standard Deviation DATA ANALYSIS Analyze the data and reflect on why students did not meet the expected level of performance. Consider each rubric dimension where students did not meet the expected level of performance. See above for question prompts to guide your interpretation of data. Not all questions will be applicable or addressed. ACTION PLAN FOR IMPROVEMENT Summarize changes to be implemented in the coming academic year (Consider possible curriculum, personnel, equipment or materials needed, training, academic supports, or assessment changes): Budget requested to make these changes: Proficient Course Level: DATA Direct Assessment Data: LiveText or Other MET: NOT MET: Indirect Assessment Data: Course Evaluations Mean: Standard Deviation 24

25 DATA ANALYSIS Analyze the data and reflect on why students did not meet the expected level of performance. Consider each rubric dimension where students did not meet the expected level of performance. See above for question prompts to guide your interpretation of data. Not all questions will be applicable or addressed. ACTION PLAN FOR IMPROVEMENT Summarize changes to be implemented in the coming academic year (Consider possible curriculum, personnel, equipment or materials needed, training, academic supports, or assessment changes): Budget requested to make these changes: LEARNING OUTCOME: Beginning Course Level: [list course(s)] Direct Assessment Data: LiveText or Other DATA MET: NOT MET: Indirect Assessment Data: Course Evaluations Mean: Standard Deviation DATA ANALYSIS 25

26 Analyze the data and reflect on why students did not meet the expected level of performance. Consider each rubric dimension where students did not meet the expected level of performance. See above for question prompts to guide your interpretation of data. Not all questions will be applicable or addressed. ACTION PLAN FOR IMPROVEMENT Summarize changes to be implemented in the coming academic year (Consider possible curriculum, personnel, equipment or materials needed, training, academic supports, or assessment changes): Budget requested to make these changes: Developing Course Level: DATA Direct Assessment Data: LiveText or Other MET: NOT MET: Indirect Assessment Data: Course Evaluations Mean: Standard Deviation DATA ANALYSIS Analyze the data and reflect on why students did not meet the expected level of performance. Consider each rubric dimension where students did not meet the expected level of performance. 26

27 See above for question prompts to guide your interpretation of data. Not all questions will be applicable or addressed. ACTION PLAN FOR IMPROVEMENT Summarize changes to be implemented in the coming academic year (Consider possible curriculum, personnel, equipment or materials needed, training, academic supports, or assessment changes): Budget requested to make these changes: Proficient Course Level: DATA Direct Assessment Data: LiveText or Other MET: NOT MET: Indirect Assessment Data: Course Evaluations Mean: Standard Deviation DATA ANALYSIS Analyze the data and reflect on why students did not meet the expected level of performance. Consider each rubric dimension where students did not meet the expected level of performance. See above for question prompts to guide your interpretation of data. Not all questions will be applicable or addressed. ACTION PLAN FOR IMPROVEMENT 27

28 Summarize changes to be implemented in the coming academic year (Consider possible curriculum, personnel, equipment or materials needed, training, academic supports, or assessment changes): Budget requested to make these changes: 28

29 APPENDIX E Curriculum Map Template Program: Institutional Learning Outcome #1: Communication: Students will show proficiency in acquiring, processing, and transferring information in a variety of ways. Course Title Course Number Written Communication: Students will develop and express ideas in writing, including working with various genres, styles, texts, technologies, data, and/or images. Oral Communication: Students will deliver a prepared, purposeful presentation designed to increase knowledge, to foster understanding, or to promote change in the listeners' attitudes, values, beliefs, or behaviors. Information Literacy: Students will show the ability to know when there is a need for information, to be able to identify, locate, evaluate, and effectively and responsibly use and share that information for the problem at hand. 29

30 Program: Institutional Learning Outcome #2: Critical Reasoning: Students will strategically apply critical thinking and problem-solving skills. Course Title Course Number Problem Solving: Students will design, evaluate and implement a strategy to answer openended questions or achieve desired goals. Critical Thinking: Students will comprehensively explore issues, ideas, artifacts, and events before accepting or formulating an opinion or conclusion. 30

31 Program: Institutional Learning Outcome #3: Civic Engagement: Students will develop the knowledge, skills, values, and motivation to actively engage in communities to promote social justice and human welfare. Course Title Course Number Civic Engagement Students will demonstrate their ability to make a difference in the civic life of communities and develop the combination of knowledge, skills, values and motivation to make a difference in the quality of life of those communities. Global Learning: Students will critically analyze complex, interdependent global systems and legacies (such as natural, physical, social, cultural, economic, and political) and their implications for people s lives and the earth s sustainability. 31

32 Program: Program Learning Outcomes: Course Title Course Number Program LO #1 Program LO #2 Program LO #3 Program LO #4 Program LO #5 Program LO #6 Program LO #7 Program LO #8 Please identify in which course(s) each Learning Outcome is specifically introduced (Beginning), reinforced (Developing), or additionally reinforced (Proficient). Programs may or may not address every Learning Outcome or every Outcome dimension. For Institutional Learning Outcomes, please identify in which course each Learning Outcome is specifically assessed using Institutional Learning Outcome rubrics. (B) -- Beginning level: At this level, the instructor introduces new material including concepts, strategies, and vocabulary of the discipline. Student s work begins to demonstrate basic application and comprehension of material. (Emergent skills) (D) -- Developing level: At this level, the instructor reinforces material introduced in prerequisite or beginning level courses, including concepts, strategies, and vocabulary. Additional material that is more complex builds upon that previously learned. Student s work develops beyond the basic level to demonstrate clear application and comprehension of material. (Benchmark to milestone skills) (P) -- Proficient level: At this level, the instructor continues reinforcing previously introduced material while building upon it with advanced information pertinent to the discipline. Student s work increasingly demonstrates facility with concepts, strategies, and vocabulary of the discipline in consistent and effective ways. (Capstone skills) (A) Learning Outcome is specifically assessed in this course 32

33 Writing Goals and Learning Objectives ASSESSMENT RESOURCES *As described by the Assessment Office at the University of Connecticut a) Goals are broad, general statements of what the program, course, or activity intends to accomplish. Goals describe broad learning outcomes and concepts (what you want students to learn) expressed in general terms (e.g., clear communication, problem-solving skills, etc.) Goals should provide a framework for determining the more specific educational objectives of a program, and should be consistent with the mission of the program and the mission of the institution. A single goal may have many specific subordinate learning objectives. b) Learning Objectives are brief, clear statements that describe the desired learning outcomes of instruction; i.e., the specific skills, values, and attitudes students should exhibit that reflect the broader goals. There are three types of learning objectives, which reflect different aspects of student learning: Cognitive objectives: What do you want your graduates to know? Affective objectives: What do you want your graduates to think or care about? Behavioral Objectives: What do you want your graduates to be able to do? Objectives can also reflect different levels of learning: Mastery objectives are typically concerned with the minimum performance essentials those learning tasks/skills that must be mastered before moving on to the next level of instruction. Developmental objectives are concerned with more complex learning outcomes those learning tasks on which students can be expected to demonstrate varying degrees of progress. Instructional Objectives describe in detail the behaviors that students will be able to perform at the conclusion of a unit of instruction such as a class, and the conditions and criteria which determine the acceptable level of performance. What are the differences between Goals and Objectives? Both goals and objectives use the language of outcomes the characteristic which distinguishes 33

34 goals from objectives is the level of specificity. Goals express intended outcomes in general terms and objectives express them in specific terms. c) Learning Outcomes are statements that describe significant and essential learning that learners have achieved, and can reliably demonstrate at the end of a course or program. Learning Outcomes identify what the learner will know and be able to do by the end of a course or program the essential and enduring knowledge, abilities (skills) and attitudes (values, dispositions) that constitute the integrated learning needed by a graduate of a course or program. The learning outcomes approach to education means basing program and curriculum design, content, delivery, and assessment on an analysis of the integrated knowledge, skills and values needed by both students and society. In this outcomes-based approach to education, the ability to demonstrate learning is the key point. What are the differences between Objectives and Outcomes? Objectives are intended results or consequences of instruction, curricula, programs, or activities. Outcomes are achieved results or consequences of what was learned; i.e., evidence that learning took place. Objectives are focused on specific types of performances that students are expected to demonstrate at the end of instruction. Objectives are often written more in terms of teaching intentions and typically indicate the subject content that the teacher(s) intends to cover. Learning outcomes, on the other hand, are more student-centered and describe what it is that the learner should learn Outcomes Pyramid *As described by the Assessment Office at the University of Connecticut 34

35 35

36 Iowa Wesleyan University Assessment Manual PART Co-Curricular Areas 8.1 Student Development Residential Life Student Engagement Campus Ministry Career Development New Student Orientation 8.2 Athletics All Individual Sports Teams 8.3 OASIS Peer Tutoring Academic Coach Writing Center 8.4 International Student Services 8.5 Library 8.6 Academic Advisement Co-Curricular Assessment Cycle Programming Learning Outcomes Program Learning Assess Implement Assess Implement Assess Implement Outcome #1 Review Review Review Program Learning -- Assess Implement Assess Implement Assess Outcome #2 Program Learning Outcome #3 Review Assess Review Review Implement Assess Review Review Implement 9.0 Co-Curricular Assessment of Institutional Learning Outcomes Overview 9.1 Direct Assessment Program-Embedded Assessment: Assessment of Institutional Learning Outcomes in Co-Curricular Programs A. Each identified co-curricular program is responsible for the assessment of at least three Program Learning Outcomes that align with ILOs during the 2 year assessment cycle. B. Assessment Rubrics recommended for use when applicable: (see Appendix A) Written Communications -- AAC&U Written Communication rubric 36

37 Oral Communications -- AAC&U Oral Communication rubric Information Literacy -- AAC&U Information Literacy rubric Critical Thinking -- AAC&U Critical Thinking rubric Problem Solving -- AAC&U Problem Solving rubric Civic Engagement -- AAC&U Civic Engagement rubric C. Student artifacts and assessment data are collected and maintained in Live Text. A secondary program-level collection site should be established through Google Drive for artifacts and data that are not easily compatible with Live Text. Assessment data should be organized and readily accessible. 9.2 Indirect Assessment Surveys A. Surveys can be developed and administered to provide rich, quantitative and qualitative data. Relevant learning outcome data is reported in the Co-Curricular Program Report (see Appendix B) Focus Groups A. Focus Groups provide qualitative data in a group setting assembled to participate in a guided discussion. Relevant learning outcome data is reported in the Co-Curricular Program Report (see Appendix B) Personal Interviews A. Personal interviews provide qualitative data in an individual setting through a guided discussion or question-and-answer. Relevant learning outcome data is reported in the Co-Curricular Program report Assessment Activities of Co-Curricular Programs 10.1 Definitions Program assessment coordinator: This person is the director of the department /program being assessed. This includes submitting Assessment Plans and Reports to the Assessment Committee and VPSD in a timely manner and coordinating with program staff to ensure that all are involved in the assessment process Program Learning Outcomes (PLO) are specific to the program s field of expertise and defined by the program Institutional Learning Outcomes (ILO) refer to those identified in Program refers to both departmental and division levels. Each division will determine at which level assessment will occur Process 11.1 Each Co-curricular department director, or designee, is responsible for designing assessment plans, executing those plans, collecting and analyzing the data, providing reports, and determining next steps within the department. These efforts should be aligned with departmental goals and learning outcomes and the mission, vision, strategic priorities, and learning outcomes of the university. 37

38 11.2 Each Co-curricular department assessment plan should span 2 academic years. See the Co-Curricular department assessment plan template below (Appendix A). Within those 2 years, a departmental learning outcome aligned with each institutional learning outcome should be assessed. For example, the goal would be that in year 1, a departmental learning outcome directly aligned with communication is assessed. In year 2, a departmental learning outcome directly aligned with critical reasoning is assessed. And, in year 3, a departmental learning outcome directly aligned with civic engagement is assessed. In doing so, all three institutional learning outcomes are assessed in the co-curricular on a 3 academic year cycle Each Co-curricular department/sub-department will gather and analyze assessment data through designed assessment plans to ensure goals are being met and program improvement is occurring. Each department/sub-department will maintain records on the status of the development, modification, and implementation of these assessment plans Each Co-curricular department/sub-department, should set student learning outcomes to inform departmental goals and practices. Those departmental learning outcomes must align with and support the institutional learning outcomes as described in section 3.0. Those departmental learning outcomes will be reviewed by the Assessment Committee prior to implementation Each Co-curricular department/sub department should also determine measurement tools to assess the stated student learning outcomes. If appropriate, national assessments allowing cohort comparisons is encouraged (i.e. National Survey of Student Engagement) The Co-curricular departments listed in 1.0 will submit their 3 year assessment plan on April 15, 2016 and every 3 years following to the Dean of Students who will share those plans with the Assessment Committee for feedback and recordkeeping Each year, every Co-Curricular Department/Sub-Department will file an assessment report annually to the Vice President for Student Development who will share it with the Assessment Committee. That report will attend to only the learning outcomes being assessed during that particular academic year (See Appendix B) Each Co-curricular department is responsible for sharing assessment information with constituencies Expected level of performance corresponds to the rubric or other method for the respective activity/event and LO Summary of Data Analysis provides an overview of direct and indirect assessment data collected during the previous year (if following the institutional 38

39 assessment cycle) as reported in the Co-Curricular Program Assessment Report (see Appendix B) Action plan for improvement summarizes action plans outlined in the Co- Curricular Program Assessment Report (Appendix B). Action Plans are to be implemented the following fiscal year. 39

40 Appendix A IW Co-Curricular Department Assessment Plan Department: Years: Department Assessment Plan Goals: 1.) 2.) 3.) 4.) 5.) Departmental Learning Outcomes Institutional Learning Outcome Connection Assessment Method Assessment Location Year being Assessed Supporting Goal Expected Level of Performance 40

41 Appendix B IW Co-Curricular Department Assessment Report Assessment Year: Departmental Learning Outcome #1: What institutional learning outcome(s) does DLO #1 support? Communication Critical Reasoning Civic Engagement Did this Objective Support the Institutional Learning Outcome as envisioned? Yes No Brief Explanation Expected Level of Performance Met? Yes No Summary of Data Analysis (Discuss Data: Include charts, tables, graphics, etc.): What Goal did this Outcome support? Is the Goal being met? Yes No Brief Explanation Recommendations for Departmental Improvement based on Assessment Data and Analysis 1) 2) 3) 41

42 SUMMARY OF DATA ANALYSIS Analyze the data and reflect on why students did not meet the expected level of performance. The questions below are meant as prompts to guide your interpretation of your data. Not all questions will be applicable or addressed. Guiding Questions: 4. Could low student ratings be due to problems with assessment? a. Was the assessment method appropriate for the learning outcome? Did the assessment capture information relevant to the learning outcome? b. Were the rubrics or evaluations used appropriately by the raters? (e.g., were they trained well enough on use of LiveText and how to interpret and use each dimension of the rubric) c. Is this the right location to teach and assess this learning outcome? d. Is the expected level of student performance too high or too low? 5. Could low student ratings be due to teaching methods? Remember to consider what IS working so you can continue that, as well as consider what IS NOT working and needs changed. a. Consider what you are doing now to teach this learning outcome. What methods of instruction, tools, scaffolding are offered to students? b. What change could be implemented to improve instruction? State what will be done, by whom, and when. Things to consider: What scaffolding or supports do students need to improve in this LO? (These could come from within or outside your program) What supports or training will staff and instructors need to improve ability to teach this LO? What resources (equipment, supplies, space, technology) are needed? 6. Could low student ratings be due to problems with program requirements? a. Do you need to change how a program is implemented, or what content is included in the program? b. Are students sufficiently prepared for the program? 42

43 IW Co-Curricular Department Assessment Report Assessment Year: Departmental Learning Outcome #2 (if applicable): What institutional learning outcome(s) does DLO #2 support? Communication Critical Reasoning Civic Engagement Did this Outcome Support the Institutional Learning Outcome as envisioned? Yes No Brief Explanation Expected Level of Performance Met? Yes No Summary of Data Analysis--Discuss Data: Include charts, tables, graphics, etc. See Subsequent page for prompts to guide interpretation of your data. What Goal did this Outcome support? Is the Goal being met? Yes No Brief Explanation Recommendations for Departmental Improvement based on Assessment Data and Analysis 1) 2) 3) 43

44 IW Co-Curricular Department Assessment Report Assessment Year: Departmental Learning Outcome #3 (if applicable): What institutional learning outcome(s) does DLO #3 support? Communication Critical Reasoning Civic Engagement Did this Outcome Support the Institutional Learning Outcome as envisioned? Yes No Brief Explanation Expected Level of Performance Met? Yes No Summary of Data Analysis (Discuss Data: Include charts, tables, graphics, etc.): What Goal did this Outcome support? Is the Goal being met? Yes No Brief Explanation Recommendations for Departmental Improvement based on Assessment Data and Analysis 1) 2) 3) 44

45 IW Co-Curricular Department Assessment Report Assessment Year: Departmental Learning Outcome #4 (if applicable): What institutional learning outcome(s) does DLO #4 support? Communication Critical Reasoning Civic Engagement Did this Outcome Support the Institutional Learning Outcome as envisioned? Yes No Brief Explanation Expected Level of Performance Met? Yes No Summary of Data Analysis (Discuss Data: Include charts, tables, graphics, etc.): What Goal did this Outcome support? Is the Goal being met? Yes No Brief Explanation Recommendations for Departmental Improvement based on Assessment Data and Analysis 1) 2) 3) 45

46 Appendix C EXAMPLE: IW Co-Curricular Department Assessment Plan Department: Residential Life Years: Department Assessment Plan Goals: 1.) Residential Students will develop a deeper appreciation of the personal and communal benefits of being an active member of a community. 2.) Residential Students will demonstrate the qualities that a person is expected to have as a responsible member of a community. 3.) Residential Students will enhance their life skills as defined as "psychosocial abilities for adaptive and positive behavior that enable individuals to deal effectively with the demands and challenges of everyday life." Departmental Learning Outcome Students will be able to identify multiple forms of diversity in their residential community. Students will be able to describe at least three ways they embody a considerate and inclusive community. Institutional Learning Outcome Connection Civic Engagement Civic Engagement / Critical Reasoning Assessment Method Student Experience Survey Focus Groups w/ AACU Oral Communication Rubric Student Experience Survey Assessment Method Operational? Built Fall 2015 Focus Groups Prompts being developed in Spring 2016 Assessment Location Director of Student Engagement Director of Student Engagement Year being Assessed Supporting Goal Expected Level of Performance At least 75% of survey respondents will identify 2 or more forms At least 75% of respondents will describe at least 3 ways at the satisfactory level (rubric dimension) and appropriately within survey. 46

47 Students will be able to complete and submit work orders. Students will be accountable for their actions. Students will be able to appropriately address conflict. Critical Reasoning Communication/ Critical Reasoning Communication/ Critical Reasoning # of Work Order Submissions Student Experience Survey Conduct Meeting Due Process Prompts Rubric # of Conduct Situations Focus Groups w/ AACU Oral Communication Rubric Student Experience Survey On-Line system is operational Due Process prompts Rubric built Fall 2015 Tracking Conduct is operational Focus Groups Prompts being developed in Spring 2016 National or more annual submissions Dean of Students Dean of Students Director of Student Engagement At least 90% of respondents will indicate user ability of work order system At least 75% of respondents will hold themselves somewhat accountable (according to Rubric Standards) At least 90% of respondents will indicate appropriate conduct resolution in survey and at least 75% of respondents at the satisfactory level (rubric dimension) 47