Assessment Report Department/Program: Sciences/FOS major Chair: Dr. Kobilinsky Degree /Minor/Certificate/or other Program: BS Time Period Covered for this Assessment Review: Fall 2014 Spring 2015 Assessment occurred in the following courses: Course Semester and Year CHE 104 General Chemistry II --- laboratory report Spring 2015 CHE 220 Quantitative Analysis --- laboratory report Spring 2015 CHE 321 Instrumental Analysis II --- laboratory report Spring 2015 TOX 416 Analytical Toxicology --- laboratory report Spring 2015 Direct Assessments of Learning Goals In this academic year (Fall 2014 Spring 2015), student learning assessment was conducted in the scaffolding manner. One department learning goal (Practical skills) was assessed. The Practical Skills learning goal is consisted with 3 different sub-areas: (1) Inquiry and Analysis; (2) Quantitative Literacy; and (3) Problem solving. The rubrics were set up for these 3 different sub-areas based on the rubrics of Association of American Colleges and Universities (AAC&U). 1
Learning goal assessed SCORE = (4) Exceeds expectations; (3) Meets expectations; (2) Approaches expectations; and (1) Does not meet expectations Mean of CHE 104 Exceeds expectations (%) CHE 104 CHE 220 CHE 321 TOX 416 Meets Approaches expectations (%) expectations (%) Does not Mean of meet CHE 220 expectations (%) Exceeds expectations (%) Meets Approaches expectations (%) expectations (%) Does not Mean of meet CHE 321 expectations (%) Exceeds expectations (%) Meets Approaches expectations (%) expectations (%) Does not Mean of meet TOX416 expectations (%) Exceeds expectations (%) Meets Approaches expectations (%) expectations (%) Does not meet expectations (%) Existing knowledge, research and/or view 3.30 25.0 70.0 5.0 0.0 3.21 33.3 42.9 19.0 4.8 3.60 40.0 50.0 10.0 0.0 3.28 20.0 75.0 5.0 0.0 Inquiry and Analysis Analysis 3.26 15.0 75.0 10.0 0.0 3.22 33.3 42.9 23.8 0.0 3.65 40.0 60.0 0.0 0.0 3.19 15.0 80.0 0.0 5.0 Conclusion 3.28 25.0 65.0 10.0 0.0 3.25 33.3 38.1 28.6 0.0 3.58 40.0 60.0 0.0 0.0 3.23 20.0 75.0 5.0 0.0 Limitations and implications 3.30 25.0 70.0 5.0 0.0 3.18 33.3 33.3 33.3 0.0 3.60 40.0 60.0 0.0 0.0 2.98 15.0 55.0 30.0 0.0 Practical Skills Quantitative literacy Interpretation 3.36 30.0 55.0 15.0 0.0 3.14 28.6 38.1 33.3 0.0 3.85 70.0 30.0 0.0 0.0 3.16 20.0 75.0 0.0 5.0 Representation 3.43 45.0 40.0 15.0 0.0 3.06 28.6 19.0 52.4 0.0 3.83 70.0 30.0 0.0 0.0 3.10 15.0 80.0 0.0 5.0 Calculation 3.40 35.0 50.0 15.0 0.0 3.11 28.6 28.6 42.9 0.0 3.80 70.0 30.0 0.0 0.0 3.11 15.0 80.0 0.0 5.0 Communication 3.44 45.0 40.0 15.0 0.0 3.18 28.6 42.9 28.6 0.0 3.73 70.0 20.0 10.0 0.0 3.13 15.0 80.0 0.0 5.0 Define problem 3.33 30.0 60.0 10.0 0.0 3.18 33.3 47.6 14.3 4.8 3.73 60.0 40.0 0.0 0.0 3.20 15.0 80.0 5.0 0.0 Identify strategies 3.26 20.0 70.0 10.0 0.0 3.28 33.3 52.4 14.3 0.0 3.73 60.0 40.0 0.0 0.0 3.14 15.0 80.0 0.0 5.0 Problem solving Proposes, evaluates and implements potential solution Evaluate outcomes relative to the problem and professional ethics in science 3.24 20.0 65.0 15.0 0.0 3.28 33.3 52.4 14.3 0.0 3.63 50.0 50.0 0.0 0.0 3.16 15.0 80.0 0.0 5.0 3.21 20.0 65.0 15.0 0.0 3.18 33.3 33.3 33.3 0.0 3.60 50.0 50.0 0.0 0.0 2.90 15.0 50.0 30.0 5.0 2
Assessment Process The assessment was set up to evaluate one of Departmental learning goals --- Practical skills across 4 different levels (CHE 104, CHE 220, CHE 321, and TOX416). The knowledge and skills of these courses are built upon on the course of the previous level. Three different sub-areas of practical skills (Inquiry and Analysis; Quantitative Literacy; and Problem Solving) were focused. During Fall 2014, instructors of these courses and Departmental outcomes assessment committees built assessment rubrics (Appendix A) for these three sub-areas using Association of American Colleges and Universities (AAC&U) assessment rubrics (https://www.aacu.org/value-rubrics) as temples and also set up the lab report writing guidelines (Appendix B E) for the chosen lab exercises. The criteria for lab exercise selection were: (1) need some analytical skills and calculation, and (2) need to draw conclusion based on the experiment. At the beginning of Spring 2015, students were provided with the rubric and the guidelines of lab report writing for one chosen lab exercise by instructors and Departmental outcomes assessment committees. There were total 71 students (CHE 104 20 students; CHE 220 21 students; CHE 321 10 students; and TOX416 20 students) participating in this assessment activity. Their reports were scored based on the scoring rubrics with the following scales. 4 Above standards (Exceeds Expectations) 3 Meet standards (Meets Expectations) 2 Approaching standards (Approaches Expectations) 1 Below standards (Does Not Meet Expectations) 3
Conclusions A. Finding: Students submitted a report after finishing their laboratory assignment which is designed to test students for all of the knowledge and the techniques they learned in the past 2 semesters. The reports were scored by using scoring rubric (Appendix A). The statistic data were summarized in Appendix F - H (Appendix I: the samples of the reports). A.1. Practical Skills: Inquiry and Analysis A.1.1. Existing knowledge, research and/or view This data indicates student s performance of laboratory report in this skill was above the satisfactory level for all participated students from those 4 courses (CHE104 mean=3.30 and 85.0% meet expectations; CHE220 mean=3.21 and 76.2% meet expectations; CHE321 mean=3.60 and 90.0% meet expectations; TOX416 mean=3.28 and 95.0% meet expectations). A.1.2. Analysis This data indicates student s performance of laboratory report in this skill was above the satisfactory level for all participated students from those 4 courses (CHE104 mean=3.26 and 90.0% meet expectations; CHE220 mean=3.22 and 76.2% meet expectations; CHE321 mean=3.65 and 100.0% meet expectations; TOX416 mean=3.19 and 95.0% meet expectations). A.1.3. Conclusion This data indicates student s performance of laboratory report in this skill was above the satisfactory level for all participated students from those 4 courses (CHE104 mean=3.28and 90.0% meet expectations; CHE220 mean=3.25 and 71.4% meet expectations; CHE321 mean=3.58 and 100.0% meet expectations; TOX416 mean=3.23 and 95.0% meet expectations). A.1.4. Limitations and implications This data indicates student s performance of laboratory report in this skill was above the satisfactory level for the participated students from CHE104, CHE220, and CHE321 (CHE104 mean=3.30and 95.0% meet expectations; CHE220 4
mean=3.18 and 66.6% meet expectations; CHE321 mean=3.60 and 100.0% meet expectations). However, TOX416 students performances in this skill were just approaching the expectation (TOX416 mean=2.98 and 70.0% meet expectations). A.1.5. Summary The students performances in these 4 courses met the expectations (the means were greater than 3), except TOX416 in one of the sub-areas (Limitation and implications) with average of 2.98 and only 70% of TOX416 students met the expectations. Even though CHE220 students performances were above satisfactory (the means were greater than 3), 24-34% of students in CHE220 did not meet the expectation. This indicates that students performances in CHE220 were not consistent with a group of students was performing very well and a group of students was performing underexpectations. A.2. Practical Skills: Quantitative literacy A.2.1. Interpretation This data indicates student s performance of laboratory report in this skill was above the satisfactory level for all participated students from those 4 courses (CHE104 mean=3.36 and 85.0% meet expectations; CHE220 mean=3.14 and 66.7% meet expectations; CHE321 mean=3.85 and 100.0% meet expectations; TOX416 mean=3.16 and 95.0% meet expectations). A.2.2. Representation This data indicates student s performance of laboratory report in this skill was above the satisfactory level for all participated students from those 4 courses (CHE104 mean=3.43and 85.0% meet expectations; CHE220 mean=3.06 and 48.6% meet expectations; CHE321 mean=3.83 and 100.0% meet expectations; TOX416 mean=3.10 and 95.0% meet expectations). A.2.3. Calculation 5
This data indicates student s performance of laboratory report in this skill was above the satisfactory level for all participated students from those 4 courses (CHE104 mean=3.40 and 85.0% meet expectations; CHE220 mean=3.11 and 57.2% meet expectations; CHE321 mean=3.80 and 100.0% meet expectations; TOX416 mean=3.11 and 95.0% meet expectations). A.2.4. Communication This data indicates student s performance of laboratory report in this skill was above the satisfactory level for all participated students from those 4 courses (CHE104 mean=3.44 and 85.0% meet expectations; CHE220 mean=3.18 and 71.5% meet expectations; CHE321 mean=3.73 and 90.0% meet expectations; TOX416 mean=3.13 and 95.0% meet expectations). A.2.5. Summary The students performances in these 4 courses met the expectations (the means were greater than 3). Even though CHE220 students performances were above satisfactory (the means were greater than 3), 29-52% of students in CHE220 did not meet the expectation. This indicates that students performances in CHE220 were not consistent with a group of students was performing very well and a group of students was performing underexpectations. A.3. Practical Skills: Problem solving A.3.1. Define problem This data indicates student s performance of laboratory report in this skill was above the satisfactory level for all participated students from those 4 courses (CHE104 mean=3.33 and 90.0% meet expectations; CHE220 mean=3.18 and 80.9% meet expectations; CHE321 mean=3.73 and 100.0% meet expectations; TOX416 mean=3.20 and 95.0% meet expectations). A.3.2. Identify strategies This data indicates student s performance of laboratory report in this skill was above the satisfactory level for all participated 6
students from those 4 courses (CHE104 mean=3.26 and 90.0% meet expectations; CHE220 mean=3.28 and 85.7% meet expectations; CHE321 mean=3.73 and 100.0% meet expectations; TOX416 mean=3.14 and 95.0% meet expectations). A.3.3. Proposes, evaluates and implements potential solution This data indicates student s performance of laboratory report in this skill was above the satisfactory level for all participated students from those 4 courses (CHE104 mean=3.24and 85.0% meet expectations; CHE220 mean=3.28 and 85.7% meet expectations; CHE321 mean=3.63 and 100.0% meet expectations; TOX416 mean=3.16 and 95.0% meet expectations). A.3.4. Evaluate outcomes relative to the problem and professional ethics in science This data indicates student s performance of laboratory report in this skill was above the satisfactory level for the participated students from CHE104, CHE220, and CHE321 (CHE104 mean=3.21and 85.0% meet expectations; CHE220 mean=3.18 and 66.6% meet expectations; CHE321 mean=3.60 and 100.0% meet expectations). However, TOX 416 students performances in this skill were just approaching the expectation (TOX416 mean=2.90 and 65.0% meet expectations). A.3.5. Summary The students performances in these 4 courses met the expectations (the means were greater than 3), except TOX416 in one of the sub-areas (Evaluate outcomes relative to the problem and professional ethics in science) with average of 2.90 and only 65% of TOX416 students met the expectations. Even though CHE220 students performances were above satisfactory (the means were greater than 3), 15-34% of students in CHE220 did not meet the expectation. This indicates that students performances in CHE220 were not consistent with a group of students was performing very well and a group of students was performing underexpectations. 7
B. Summary of laboratory report assessment: Overall student s performance for the selected lab exercise report was above the departmental standards (3.00), except TOX416 in two of the sub-areas (Inquiry and Analysis: Limitations and implementations; Problem Solving: Evaluate outcomes relative to the problem and professional ethics in science). This could be due to (1) these sub-areas were much more challenges for TOX416 students, (2) the standards of the rubrics for TOX416 were too high, or (3) the standards of the rubrics for CHE104, CHE220, and CHE321 were too low. The Quantitative Literacy showed the largest discrepancy among CHE220 students. Even though CHE220 students performances were above satisfactory (the means were greater than 3), a large percentage of students in CHE220 did not meet the expectation. This indicates that students performances in CHE220 were not consistent with a group of students was performing very well and a group of students was performing under-expectations. This indicates that students who have great performances in CHE104 might not perform well in CHE220 and students who have bad performances in CHE220 could do much better in CHE321. This could be due to (1) CHE220 students were not well prepared, (2) this exercise was too difficult for CHE220 students, (3) the standards of the rubrics for CHE220 were too high, or (4) the standards of the rubrics for CHE104, CHE321, and TOX416 were too low. Revising the courses, the lab exercises, the guidelines for lab report writing, and the rubrics are needed for the future assessment. 8
Actions Taken Timeframe for Actions To Be Taken and By Whom implementation and intermediate steps Revise the rubrics by Department outcomes assessment committees Fall 2015 Choose a lab exercise as an assessment tool for Biology related courses (BIO104, CHE315, BIO315, and Fall 2015 BIO413) by lab instructors and Department outcomes assessment committees Revise the curriculum of CHE220 and TOX416 Fall 2015/Spring 2016 Develop the lab report writing guidelines for Biology related courses by lab instructors and Department Fall 2015 outcomes assessment committees Implement the scaffolding assessment for Biology related courses by lab instructors and Department Spring 2016 outcomes assessment committees Close loop for 2013-2014 assessment (please see 2013/2014 assessment report) 2015-2016 Were last year s actions implemented as planned? Please explain. Last year (2013/2014) assessment has suggested to revise the rubrics for TOX416 lab report and oral presentation and re-implement the assessment again. However, Department outcomes assessment committees decided to evaluate the student learning in the scaffolding manner. By doing this scaffolding assessment, it will help to understand the problem with TOX416 is due to underpreparation of the lower courses or due to the course itself. Therefore, the proposed action was postposed and will be conducted in 2015/2016. Assessment data and conclusions were discussed in a Department or Program meetings on TBA. [dates] 9
Attachments: Appendix A: Scaffolding rubric Page 12 Appendix B: Guideline for CHE 104 lab exercise Page 17 Appendix C: Guideline for CHE 220 lab exercise Page 18 Appendix D: Guideline for CHE 321 lab exercise Page 19 Appendix E: Guideline for TOX 416 lab exercise Page 20 Appendix F: The statistic data of the student s lab report assessment: Inquiry and Analysis Page 21 Appendix G: The statistic data of the student s lab report assessment: Quantitative Literacy Page 23 Appendix H: The statistic data of the student s lab report assessment: Problem Solving Page 25 10
Appendix A: Scoring rubric 11
12
13
14
15
16
17
18
19
Appendix F: The statistic data of the student s lab report assessment --- Inquiry and Analysis CHE104 Existing knowledge, research and/or view Analysis Conclusion Limitations and implications Mean 3.30 3.26 3.28 3.30 median 3.00 3.00 3.00 3.00 mode 3.00 3.00 3.00 3.00 Range 2.00 4.00 2.00 4.00 2.00 4.00 2.00 4.00 CHE220 Existing knowledge, research and/or view Analysis Conclusion Limitations and implications Mean 3.21 3.22 3.25 3.18 median 3.00 3.00 3.00 3.00 mode 4.00 3.00 4.00 4.00 Range 1.00 4.00 2.00 4.00 2.00 4.00 2.00 4.00 20
CHE321 Existing knowledge, research and/or view Analysis Conclusion Limitations and implications Mean 3.60 3.65 3.58 3.60 median 3.75 3.50 3.50 3.63 mode 4.00 3.50 4.00 4.00 Range 2.75 4.00 3.00 4.00 3.00 4.00 3.00 4.00 TOX 416 Existing knowledge, research and/or view Analysis Conclusion Limitations and implications Mean 3.28 3.19 3.23 2.98 median 3.00 3.00 3.00 3.00 mode 3.00 3.00 3.00 3.00 Range 2.00 4.00 1.75 4.00 2.00 4.00 2.00 4.00 21
Appendix G: The statistic data of the student s lab report assessment --- Quantitative Literacy CHE104 Interpretation Representation Calculation Communication Mean 3.36 3.43 3.40 3.44 median 3.50 3.50 3.63 3.63 mode 4.00 4.00 4.00 4.00 Range 2.00 4.00 2.00 4.00 2.00 4.00 2.00 4.00 CHE220 Interpretation Representation Calculation Communication Mean 3.14 3.06 3.11 3.18 median 3.00 3.00 2.90 3.00 mode 3.00 4.00 2.65 3.00 Range 2.00 4.00 2.00 4.00 2.00 4.00 2.00 4.00 22
CHE321 Interpretation Representation Calculation Communication Mean 3.85 3.83 3.80 3.73 median 4.00 4.00 4.00 4.00 mode 4.00 4.00 4.00 4.00 Range 3.25 4.00 3.25 4.00 3.25 4.00 2.75 4.00 TOX 416 Interpretation Representation Calculation Communication Mean 3.16 3.10 3.11 3.13 median 3.00 3.00 3.00 3.00 mode 3.00 3.00 3.00 3.00 Range 1.00 4.00 1.00 4.00 1.00 4.00 1.00 4.00 23
Appendix H: The statistic data of the student s lab report assessment --- Problem Solving CHE104 Define problem Identify strategies Proposes, evaluates and implements potential solution Evaluate outcomes relative to the problem and professional ethics in science Mean 3.33 3.26 3.24 3.21 median 3.25 3.25 3.25 3.00 mode 3.00 3.00 3.00 3.00 Range 2.00 4.00 2.00 4.00 2.00 4.00 2.00 4.00 CHE220 Define problem Identify strategies Proposes, evaluates and implements potential solution Evaluate outcomes relative to the problem and professional ethics in science Mean 3.18 3.28 3.28 3.18 median 3.00 3.00 3.00 3.00 mode 3.00 3.00 3.00 4.00 Range 1.00 4.00 2.00 4.00 2.00 4.00 2.00 4.00 24
CHE321 Define problem Identify strategies Proposes, evaluates and implements potential solution Evaluate outcomes relative to the problem and professional ethics in science Mean 3.73 3.73 3.63 3.60 median 4.00 4.00 3.88 3.88 mode 4.00 4.00 4.00 4.00 Range 3.00 4.00 3.00 4.00 3.00 4.00 3.00 4.00 TOX 416 Define problem Identify strategies Proposes, evaluates and implements potential solution Evaluate outcomes relative to the problem and professional ethics in science Mean 3.20 3.14 3.16 2.90 median 3.00 3.00 3.00 3.00 mode 3.00 3.00 3.00 3.00 Range 2.00 4.00 1.50 4.00 1.75 4.00 1.00 4.00 25