KPI SUMMARY REPORT Assessment for Student Learning: -level Assessment Board of Trustees Meeting, August 23, 2016 BACKGROUND Assessment for Student Learning is a key performance indicator aligned to the College category of Educational Quality. As a measure of Educational Quality, this category is aligned to the Illinois Board of Higher Education (IBHE) Public Agenda Goal to raise the number of people with quality postsecondary credentials and to improve transitions along the educational pipeline. Assessment for student learning is conducted at the institutional, program or discipline, course, and class level. (ISLO) describe what a student should know or be able to do upon completion of their college experience. As such, NOTE: Details about the assessment process and RVC can be found on the College website through the featured link, Assessment for Student Learning. learning outcomes at the institutional level are general enough to apply to all students regardless of credential earned and program or discipline studied. The RVC ISLO are representative of the national expectations of associate s degree completers described within the Lumina Foundation s Degree Qualifications Profile. Figure 1 displays the ISLO expected of each student completing a degree or certificate program at Rock Valley College. Figure 1: Rock Valley College Analytic Reasoning Students will form logical inferences, judgments, or conclusions from facts or premises related to topics encountered in the classroom, workplace, and daily life. Communication Global Awareness and Responsibility Personal Responsibility Students will exchange ideas effectively in a variety of settings. Students will develop the knowledge and skills required to responsibly interact with social and natural communities, both locally and globally. Students will accept responsibility for their personal and professional wellness and development, positioning themselves for life-long learning. PROCESS For several years, reading days have been used to assess the RVC ISLO. In preparation for a reading day, student work samples are collected from a variety of courses and organized into electronic forms for scoring. These work samples come from assignments embedded into students regular coursework to provide a direct measure of student learning. During reading day, faculty from multiple programs and disciplines gather to evaluate student work samples with respect to the ISLO. Community College Survey of Student Engagement As an indirect measure of student learning, Rock Valley College participates in the Community College Survey of Student Engagement (CCSSE) every other year. CCSSE is a nationally normed survey that measure institutional practices and student behaviors that are associated with learning and retention. The College receives five benchmark scores that are compared to all participating Page 1 of 5
community colleges and community colleges of a similar size. The five CCSSE benchmarks are Active and Collaborative Learning, Student Effort, Academic Challenge, Student-Faculty Interaction, and Support for Learners. RESULTS ISLO scores were calculated by comparing checklist ratings to learning expectations based on credit hours earned. This method of scoring allows for students who have completed their program to be held to a higher standard than those with 30 or more credits and student with 30 or more credits to be held to a higher standard than those with fewer than 30 credits. Individual student scores are converted to the percentage of points earned and averaged to create an institutional score for each student learning outcome component. If the average score is 80% or Achieving 80% or more higher, then the outcome is generally considered to be achieved. If the average score is at least 60% but less Approaching at least 60% but than 80%, students are generally approaching less than 80% achievement. Learning outcomes with an average score lower than 60% have been identified as areas in Needs less than 60% need of improvement. Improvement Communication Table 1 summarizes student achievement with respect to Communication. Table 1: Achievement of : Communication C1: Students will create texts that are clear (e.g., coherent and concise). 77% C2: Students will create texts that are substantially error-free. 23% C3: Students will identify the perspective (e.g., purpose or audience) of texts. 67% C4: Students will use appropriate formats and technologies to exchange ideas. C5: Students will comprehend a variety of texts. 40% C6: Students will present texts to a group. Alternate Method* *This learning outcome was not assessed and requires an alternate assessment method. Data presented in Table 1 indicate: Students are generally achieving with respect to using appropriate formats and technologies. Students generally are approaching achievement with respect to clarity of communication and the ability to identify perspective (e.g., purpose or audience). Students generally need improvement with respect to reducing errors (grammatical and mechanical) and comprehension. The ability to present texts to a group was not assessed. In the future, recorded speeches could be collected and evaluated to address this component. Analytic Reasoning Table 2 summarizes student achievement with respect to Analytic Reasoning. In general, students are achieving all four components of this learning outcome. Page 2 of 5
Table 2: Achievement of : Analytic Reasoning AR1: Students will identify the ideas, theories, or methods relevant to various topics, tasks, or problems. AR2: Students will select appropriate relevant information, resources, or technologies necessary to address various topics, tasks, or problems. 93% AR3: Students will apply an appropriate method, strategy, or plan of action to perform a task, resolve a problem, or draw a logical conclusion. AR4: Students will analyze information, resources, technologies, or data. 81% Global Awareness and Responsibility Table 3 summarizes student achievement with respect to Global Awareness and Responsibility. Table 3: Achievement of Students Learning Outcomes: Global Awareness and Responsibility GAR1: Students will recognize their own cultural rules and biases. GAR2: Students will describe diverse values and perspectives. 99% GAR3: Students will distinguish between what is and what ought to be in social and natural contexts. GAR4: Students will analyze social and environmental issues in a variety of contexts. GAR5: Students will analyze the reciprocal impact of individual and group behavior on the local, national, and global communities. GAR6: Students will identify behaviors related to civic engagement. GAR7: Students will work effectively in groups. Alternate Method* = Not available because the checklist item was not applicable for the assignment or raters could not agree on a rating *This learning outcome was not assessed and requires an alternate assessment method Data presented in Table 3 indicate: Students are generally achieving the learning outcome for describing diverse values and perspectives. Evaluators had trouble rating most components of Global Awareness and Responsibility. Since this could indicate poor alignment between the learning outcome statements and the assignments evaluated, assignments will be more purposefully selected in the future to better provide students with an opportunity to demonstrate the knowledge and skills required to responsibly interact with social and natural communities. The ability to work effectively in groups was not assessed. In the future, observation of group work or intragroup evaluations could be used to assess this learning outcome. Personal Responsibility Table 4 summarizes student achievement with respect to Personal Responsibility. Data presented in Table 4 indicate: Evaluators had trouble rating most components of Personal Responsibility. Alternate methods, such as behavioral observation or self-reflection, will need to be developed to assess this learning outcome. Checklist items were not developed for PR2 in anticipation that this component could not be assessed with a writing sample. Page 3 of 5
Table 4: Achievement of : Personal Responsibility PR1: Students will recognize personal biases. PR2: Students will demonstrate civil, ethical, and professional behavior. Alternate Method* PR3: Students will recognize consequences of their behavior. PR4: Students will identify behaviors associated with greater well-being (e.g., physical, emotional, intellectual). PR5: Students will exhibit adaptability to changing circumstances and environments. = Not available because the checklist item was not applicable for the assignment or raters could not agree on a rating *This learning outcome was not assessed and requires an alternate assessment method Community College Survey of Student Engagement Table 5 shows RVC CCSSE benchmark scores compared to the national cohort and other mediumsized community colleges. Table 5: RVC Benchmark s Compared to the National Cohort and Other Medium Colleges Rock Valley College Medium Colleges 2016 National Cohort Decile Difference Difference Benchmark Active and Collaborative Learning 45.8 2 49.8-4.0 50.0-4.2 Student Effort 48.9 4 49.8-0.9 50.0-1.1 Academic Challenge 49.5 5 50.0-0.5 50.0-0.5 Student-Faculty Interaction 47.2 2 50.2-2.9 50.0-2.8 Support for Learners 47.6 3 49.6-2.0 50.0-2.4 Data in Table 5 indicate: RVC scored below the national cohort on all five benchmarks. RVC scored below the medium college group on all five benchmarks. RVC falls in the second decile for Active and Collaborative Learning and Student-Faculty Interaction, meaning that the College only scored better than 20% of the other participating schools. These are the areas of most concern based on decile scores. FINDINGS AND CONCLUSIONS As a measure of Educational Quality, institutional assessment for student learning ensures that the credentials awarded by the College represent a quality postsecondary education and facilitate transitions along the educational pipeline. Findings from ISLO assessment suggest some areas of success and others in need of improvement. For example, students are approaching or achieving most expectations related to Communication but need some additional support with reducing writing errors and comprehension. The data available also suggest that students are achieving expectations related to Analytic Reasoning. Global Awareness and Responsibility and Personal Responsibility elements were difficult to rate, suggesting the need for development of alternative assessment methods or more intentional alignment between student assignments and checklists. Page 4 of 5
Community College Survey of Student Engagement According to the CCSSE 2016 results, RVC students are scoring below the national average on Active and Collaborative Learning, Student Effort, Academic Challenge, Student-Faculty Interaction, and Support for Learners. The areas of most concern are Active and Collaborative Learning and Student-Faculty Interaction. Improvements in these institutional practices and student behaviors could lead to improvements in student learning and retention. Page 5 of 5