Report of Critical Thinking Assessment,

Transcription

1 Report on Assessment of Critical Thinking October Report of Critical Thinking Assessment, Submitted October 2016 Prepared by Dr. Rebecca Ertel, Office of Academic Planning and Assessment for the Senate General Education and Writing-Across-the-Curriculum Committee General Education Critical Thinking Outcome Overview The critical thinking component of CSU s general education curriculum has the following learning outcome and sub-outcomes: Outcome: Students will think critically Students will apply the habits of inquiry and analysis to various situations to Accurately interpret evidence (statements, graphics, questions, etc.) Identify relevant arguments (reasons, claims, pros and cons, etc.) Analyze and evaluate alternative points of view Justify key results or procedures and explain assumptions and reasons Fair-mindedly follow where evidence and reason lead Critical Thinking Assessment Description: Spring 2015-Spring 2016 Signature Assignments Writing samples (Signature Assignments) were collected from 129 first-year students enrolled in English 1100 and 1101 during fall semester 2015 and from 54 senior students enrolled in upper-division writing intensive (WI) courses during spring semester Samples were collected from four (4) sections of English 1100 offered during Fall 2015; one section did not submit samples for scoring due to late scheduling of the course. Samples were collected from five (5) sections of English 1101 offered during Fall 2015; four (4) sections did not submit samples due to instructor oversight or late scheduling. Samples were collected from eleven (11) sections of writing intensive courses offered during Spring 2016; samples were not returned from six (6) writing intensive courses. Writing Intensive courses were not offered by fourteen (14) programs, primarily because the WI courses are offered in alternate years. The samples were scored by the instructor of the section from which they were taken and a member of the Written Communication Faculty Learning Community. The two scores were averaged to produce a final score for the sample. In cases in

2 Report on Assessment of Critical Thinking October which the scores differed by more than one level on the rubric, the sample was scored by a third reader and the two closest scores were averaged to produce the final score. The samples were assessed using a modified version of the American Association of Colleges and University s VALUE rubric for written communication. A calibration session was held for the scorers prior to scoring the samples. The percent of agreement on the category scores was.72, above the.7 thresh hold generally accepted to establish reliability (Jonsson and Svingby, 2007). Discussion of Scores for English 1100 and English 1101 As expected, the Fall 2015 assessment of Signature Assignments found that more than 80% of students in English 1100 and 1101 begin their first-year performing at the Benchmark (1) or Milestone 1 (2) levels on the rubric; the scores are about equally divided between 1 s and 2 s. Critical Thinking Scores English 1100/1101, Fall 2015 Table1. Students scored highest in the area of selecting evidence and weakest in the drawing conclusions and assessing consequences. Comparison of English 1100 and 1101 While students in both courses scored primarily 1 or 2, more than a quarter of English 1101 students scored 3 or 4 while only about 7% of students in 1100 scored 3 or 4. Student samples from 1100 did not reflect large differences among category scores. Student samples from English 1101 showed students tended to score lower in the categories of Explanation of Issues and Evidence.

3 Report on Assessment of Critical Thinking October English 1100 Critical Thinking Scores, Fall 2015 Table 2. English 1101 Critical Thinking Scores, Fall 2015 Table 3. English 1100/1101 Score Distribution Table 4.

4 Report on Assessment of Critical Thinking October Discussion of Scores for Writing Intensive Courses The Spring 2016 assessment of Signature Assignments found that the samples submitted from senior students in upper-division writing intensive courses are evenly distributed between samples scored 1 or 2 and samples scored 3 or 4. Students scored lowest in the area of drawing conclusions and highest in the areas of Explanation of Issues and Influence of Context and Assumptions. Critical Thinking Scores Writing Intensive Courses, Spring 2016 Table 5. Comparison of English 1100 and 1101and Writing Intensive Courses The data suggest most students make measurable gains in critical thinking between their first and senior years. Fifty percent (50%) of samples from senior students enrolled in upper-division writing intensive courses had a composite score of 3 or 4. In comparison, only 18.6% of samples from students enrolled in English 1100 or 1101 had composite scores of 3 or 4. Samples were scored lowest on drawing conclusions from evidence at both the first-year and senior levels. Distribution of Scores in English 1100/1101 Table 6.

5 Report on Assessment of Critical Thinking October Distribution of Scores in Writing Intensive Courses Table 7. Recommendations of the GenEd/WAC Committee The GenEd/WAC committee recommends the University Senate direct academic programs that offer general education courses supporting the learning outcome for Critical Thinking to identify teaching strategies and learning activities to incorporate into these courses with the goal of improving student learning in the area of analyzing evidence and drawing conclusions. All degree programs should similarly strengthen instruction in these abilities in their curricula. Programs should report their actions to the GenEd/WAC Committee by the end of March. The GenEd/WAC Committee will provide a summary report to the Senate by the end of the academic year.

6 Report on Assessment of Critical Thinking October References Jonsson, A. & Gunilla, S. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2, Stoddart, T., Abrams, R., Gasper, E., & Canaday, D. (2000). Concept maps as assessment in science inquiry learning A report of methodology. International Journal of Science Education, 22,

7 Report on Assessment of Critical Thinking October Catalog Descriptions: Appendix A Central State University English 1100: Introduction to Writing and Reading for College (5) Course Syllabus ENG Introduction to Writing and Reading for College (5 credits, I, II): An intensive introduction to writing and reading for College. Students will read literary and non-literary texts and compose essays that demonstrate proficiency in college level writing and mechanics. At least one paper will be a readerresponse essay based on a literary text. Students will also be introduced to the basic principles of documentation and write one essay using documentation. Lecture/discussion periods will focus on discussions of assigned readings and introducing the conventions of academic prose, including elements of Standard English grammar and mechanics. Individualized and small-group work on the writing process included. Required of students whose performance on the English Placement Test indicates a need for intensive writing instruction. Equivalent to English 1101 for general education requirements. Textbooks: Maimon, Elaine P. and Janice H. Peritz. Writing Intensive: Essentials for College Writers. Boston: McGraw Hill, 2009, Print. Connect Writing 2.0. Boston: McGraw Hill, Online. Learning Outcomes 1. Recognize elements that contribute to rhetorical situations, including purpose, audience, stance, and appropriate conventions. 2. Critically examine personal views in comparison/contrast with those explored in texts. 3. Identify, analyze, and discuss themes and structures of texts. 4. Apply the process of writing through drafting, revising, and editing, at times in collaboration with others. 5. Apply writing process to produce effective and fully developed essays. 6. Use correct grammar, sentence structure, and punctuation. 7. Explore a diverse range of authors, cultures, and perspectives. 8. Use electronic environments to draft and publish work where possible. Policies/Requirements: 1. Write a combination of essays, annotated bibliographies, and/or shorter compositions of formal writing that is roughly equivalent to twenty pages. 2. Write one source-supported essay following the MLA format.

8 Report on Assessment of Critical Thinking October Complete all assignments according to scheduled due dates, including all assignments from Connect Writing 2.0. The Connect Writing 2.0 assignments should total at least 10% of the final grade. 4. Participate in all class discussions, activities, and assignments. 5. Take the departmental final examination at the prescribed time. 6. Avoid plagiarism. Plagiarism, the submitting of another student s work or material from other printed sources as one s own work, may result in the grade of "F" for the paper or for the course itself. 7. Attend class on time regularly. Students are responsible for obtaining materials, assignments, and information given on missed class days. 8. Instructors may develop and enforce other policies consistent with the current CSU course catalog, departmental guidelines, and standard academic practices. Veteran instructors may pick an alternate supplemental textbook. Criteria for Grading: The grade for the class will be determined by the student s level of competence in class assignments and activities, homework, tests, quizzes, and essays. Instructors may develop and enforce attendance and other policies consistent with the current CSU course catalog and Departmental guidelines. Each instructor will develop a course section syllabus that discusses specific methods of delivery, topics, activities, and assignments. The final exam will count as at least 10% of the final grade. ENG1100 uses the university s A-F and I grading scale. Americans with Disabilities Compliance: Central State University is committed to including students with disabilities as full participants in its programs, services, and activities through compliance with Section 504 of the Rehabilitation Act of 1973 and the Americans with Disabilities Act (ADA) of If you are a student with a documented disability please call Dr. Wanda Hadley at to discuss your course accommodations. Knowledge, Skills, and Dispositions At the completion of the course the student will demonstrate knowledge, skills, and dispositions as follows: Knowledge: 1. Explain the theme, structure, and meaning of literary and non-literary texts. 2. Know how to develop a thesis. 3. Explain the relationship between details and themes of texts. 4. Understand what constitutes evidence and how to determine the most effective form. 5. Develop the ability to produce edited writing according to the conventions of Standard American English. 6. Understand how to document primary and secondary sources in essays.

9 Report on Assessment of Critical Thinking October Skills: 7. Identify, analyze and discuss themes and structures of literary and non-literary texts. 8. Defend themes by identifying valid supporting ideas and evidence. 9. Write effective thesis statements. 10. Write effective, fully developed, and organized essays. 11. Synthesize ideas from readings into other activities and writing. 12. Use and document secondary sources. 13. Use a more extensive, sophisticated vocabulary. 14. Write for clarity, organization, and development. 15. Use correct grammar, sentence structure, and punctuation. Dispositions: 16. Understand and appreciate a variety of literary and non-literary texts. 17. Examine personal views in comparison/contrast with those explored in texts. 18. Explore a diverse range of authors, cultures, and perspectives. 19. Learn the relationship between literature and life/universal truths. 20. Understand the importance of effective written communication. 21. Develop confidence in responding to oral /written prompts. 22. Understand the importance of documentation.

10 Report on Assessment of Critical Thinking October Catalog Descriptions: Central State University English 1101: Introduction to Writing for College (4) Course Syllabus ENG (4 credits, 1, II, III) Introduction to Writing for College: Informative writing based on literary and non-literary texts. Students will compose essays that demonstrate proficiency in college-level writing and mechanics. At least one paper will be a reader-response essay based on a literary text. Students will also be introduced to the basic principles of documentation and write one essay using documentation. Individualized and small-group work on the writing process included. Textbooks: Maimon, Elaine P. and Janice H. Peritz. Writing Intensive: Essentials for College Writers. Boston: McGraw Hill, 2009, Print. Kennedy, X. J., Bedford Guide for College Writers with Reader, Ninth Edition. New York: Bedford/St. Martin s, Print. Learning Outcomes 1. Recognize elements that contribute to rhetorical situations, including purpose, audience, stance, and appropriate conventions. 2. Critically examine personal views in comparison/contrast with those explored in texts. 3. Identify, analyze, and discuss themes and structures of texts. 4. Apply the process of writing through drafting, revising, and editing, at times in collaboration with others. 5. Apply writing process to produce effective and fully developed essays. 6. Use correct grammar, sentence structure, and punctuation. 7. Explore a diverse range of authors, cultures, and perspectives. 8. Use electronic environments to draft and publish work where possible. Policies/Requirements: 1. Write a combination of essays, annotated bibliographies, and/or shorter compositions of formal writing that is roughly equivalent to twenty pages. 2. Write one source-supported essay following the MLA format. 3. Complete all assignments according to scheduled due dates. 4. Participate in all class discussions, activities, and assignments. 5. Take the departmental final examination at the prescribed time. 6. Avoid plagiarism. Plagiarism, the submitting of another student s work or material from other printed sources as one s own work, may result in the grade of "F" for the paper or for the course itself.

11 Report on Assessment of Critical Thinking October Attend class on time regularly. Students are responsible for obtaining materials, assignments, and information given on missed class days. 8. Instructors may develop and enforce other policies consistent with the current CSU course catalog, departmental guidelines, and standard academic practices. Veteran instructors may pick an alternate supplemental textbook. Criteria for Grading: The grade for the class will be determined by the student s level of competence in class assignments and activities, homework, tests, quizzes, and essays. Instructors may develop and enforce attendance and other policies consistent with the current CSU course catalog and Departmental guidelines. Each instructor will develop a course section syllabus that discusses specific methods of delivery, topics, activities, and assignments. The final exam will count as at least 10% of the final grade. ENG1100 uses the university s A-F and I grading scale. Americans with Disabilities Compliance: Central State University is committed to including students with disabilities as full participants in its programs, services, and activities through compliance with Section 504 of the Rehabilitation Act of 1973 and the Americans with Disabilities Act (ADA) of If you are a student with a documented disability please call Dr. Wanda Hadley at to discuss your course accommodations. Knowledge, Skills, and Dispositions At the completion of the course the student will demonstrate knowledge, skills, and dispositions as follows: 1. Knowledge: 2. Explain the theme, structure, and meaning of literary and non-literary texts. 3. Know how to develop a thesis. 4. Explain the relationship between details and themes of texts. 5. Understand what constitutes evidence and how to determine the most effective form. 6. Develop the ability to produce edited writing according to the conventions of Standard American English. 7. Understand how to document primary and secondary sources in essays. 8. Identify, analyze and discuss themes and structures of literary and non-literary texts. Skills: 1. Defend themes by identifying valid supporting ideas and evidence. 2. Write effective thesis statements. 3. Write effective, fully developed, and organized essays. 4. Synthesize ideas from readings into other activities and writing. 5. Use and document secondary sources. 6. Use a more extensive, sophisticated vocabulary.

12 Report on Assessment of Critical Thinking October Write for clarity, organization, and development. 8. Use correct grammar, sentence structure, and punctuation. Dispositions: 1. Understand and appreciate a variety of literary and non-literary texts. 2. Examine personal views in comparison/contrast with those explored in texts. 3. Explore a diverse range of authors, cultures, and perspectives. 4. Learn the relationship between literature and life/universal truths. 5. Understand the importance of effective written communication. 6. Develop confidence in responding to oral /written prompts. 7. Understand the importance of documentation.

13 Report on Assessment of Critical Thinking October Central State University English 1102: Writing and Researching the Essay Course Syllabus Catalog Description ENG 1102 (4 credits, I, II, III) Writing and Researching the Essay: Research-based argumentative and analytical writing using library and Internet sources. Students will continue to develop proficiency in college-level writing and mechanics. They will write brief essays and at least one longer research paper, properly formatted, using both print and electronic sources. The research paper will be oriented toward the student s major field or area of interest. Course includes review for the English Proficiency exam. Prerequisite: Grade of C or above in ENG 1100 or 1101 or equivalent. Textbooks Maimon, Elaine P. and Janice H. Peritz. Writing Intensive: Essentials for College Writers. Boston: McGraw Hill, 2009, Print. Lunsford, Andrea A. And John J. Ruszkiewicz. Everything s an Argument. Boston: Bedford/St. Martin s, Print. Learning Outcomes 1. Analyze argumentative strategies and rhetorical appeals. 2. Develop appropriate argumentative strategies and rhetorical appeals in essays. 3. Find, evaluate, and use sources as recommended by academic major or area of interest. 4. Write unified and coherent analytical and persuasive essays. 5. Apply the process of writing through drafting, revising, and editing, at times in collaboration with others. 6. Employ textual conventions for incorporating ideas from sources. 7. Achieve proficiency in the tools of research, including print documents, scholarly databases, general databases, and informal Internet sources. 8. Use correct grammar, sentence structure, and punctuation. Policies/Requirements 1. To enroll in ENG 1102, students must have earned a grade of "C" or above in ENG 1101 or equivalent 2. Write a combination of essays, annotated bibliographies, and/or shorter compositions of formal writing that is roughly equivalent to twenty pages. 3. Write a fully developed, fully documented research paper, proceeding from rough drafts to final form. 4. Consult with faculty in the major departments for recommended sources. 5. Study sample essays for discussion of ideas and writing models. 6. Schedule at least two conferences with the instructor. Conferences can be individual or in small groups. 7. Attend class on time. Students are responsible for finding out, on their own, the material for classes missed.

14 Report on Assessment of Critical Thinking October Avoid plagiarism. Plagiarism, the submitting of another's work or material from other printed sources as one's own work may result in the grade of "F" for the paper or for the course itself. 9. Take the required departmental final exam at the prescribed time. 10. Achieve skills and complete all work by the end of the semester. 11. Instructors may develop and enforce other policies consistent with the current CSU catalog, departmental guidelines and standard academic practices. Veteran instructors may pick an alternate supplemental textbook. Criteria for Grading: The final grade will be determined by the level of competence in the writing and discussion required in the course, mastery of proper documentation(s) as required by major area, and class attendance and participation. Each instructor will develop a course section syllabus that discusses specific methods of delivery, topics, activities, and assignments. The final exam will count as at least 10% of the final grade. ENG1102 uses the university s A-F and I grading scale. Americans with Disabilities Compliance: Central State University is committed to including students with disabilities as full participants in its programs, services, and activities through compliance with Section 504 of the Rehabilitation Act of 1973 and the Americans with Disabilities Act (ADA) of If you are a student with a documented disability please call Dr. Wanda Hadley at to discuss your course accommodations. Knowledge, Skills, and Dispositions At the completion of this course the student will demonstrate knowledge, skills, and dispositions as follows: Knowledge (what you can explain and/or teach to others) 1. Understand the various modes of writing (exposition, description, and argumentation) used in research writing. 2. Explain the relationship between the thesis and the rest of the research paper. 3. Describe the techniques of argumentation. 4. Refute adversarial positions. 5. Describe proper documentation as required by major area. 6. Communicate awareness of issues. Skills (tasks you can perform): 1. Read and think critically. 2. Analyze argumentative issues. 3. Relate information in essays to one s own academic major or areas of interest. 4. Apply the techniques of argumentation to research-based writing. 5. Read and think critically. 6. Use approved documentation forms in research-based writing.

15 Report on Assessment of Critical Thinking October Use primary and secondary sources properly. 8. Write unified and coherent analytical and persuasive essays. 9. Choose and limit a research topic. 10. Find information in the library and on the Internet. 11. Formulate a thesis for the research paper. 12. Use MLA or APA formatting and documentation conventions. Dispositions (attitudes and patterns of behavior) 1. Value careful reading and effective written communication. 2. Value critical awareness of differing arguments. 3. Maintain interest in current topics. 4. Confidence in discussing issues. 5. Evaluate and organize ideas. 6. Achieve proficiency in the tools of research. 7. Recognize rhetorical modes used in research writing.

16 Report on Assessment of Critical Thinking October Appendix C First-Year Signature Assignment Prompt Name: Student ID Number: English 1100/1101 You will have the rest of the class period to write a well-developed essay in response to the following directions. Be sure to plan your ideas carefully, include specific examples and details from the attached article in your paragraphs, and leave sufficient time for editing. Directions: Read Style Showdown: $1,000 Sweater Faces $100 Rival, then write an essay that addresses the following questions. You do not have to answer the questions in order, but you should discuss each question in your essay. The finished essay should be at least four to six paragraphs in length. 1. In your own words, describe two similarities and two differences between the two sweaters and then discuss what is significant about the similarities and/or differences between the two sweaters. 2. In your opinion, is there anything that the author could have explained in more detail or considered more carefully? Why? 3. How would you design and manufacture a sweater given what you have learned from the reading? Feel free to combine features of both or to suggest new ideas

17 Report on Assessment of Critical Thinking October Appendix D

18 Report on Assessment of Critical Thinking October Appendix E

19 Report on Assessment of Critical Thinking October