ASSESSMENT REPORT for EMORY ACADEMIC PROGRAMS

Transcription

1 ASSESSMENT REPORT for EMORY ACADEMIC PROGRAMS Art History Department September 25, 2015 The Art History Department met on September 16, 2015, to discuss the results of its annual assessment. Led by the Director of Undergraduate Studies, the meeting involved thirteen of the sixteen members of the department (two are on leave this semester; one was unable to attend), ranging in rank from assistant to full professor, some of whom have taught at Emory for thirty years or more. Since 2009, our department has used two methods to measure student achievement of our designated learning outcomes: the assessment of seminar papers and the analysis of a Senior Exit Survey administered to all graduating Art History majors. Direct Assessment Methodology. Our primary method is the direct assessment of 400-level seminar papers submitted by senior Art History majors during the academic year. The seminars focus on a particular area of study within art history, and although they have no prerequisites and are open to non-majors, they are designed with advanced art history students in mind; our majors are required to take at least one. All seminars are writingintensive and require a substantial research paper. Each faculty member was assigned two or three papers to assess. To keep their evaluation as unbiased as possible, we ensured that faculty read papers from courses other than their own, and that the papers were read blind. Each was assessed according to a rubric that was developed and is periodically revised by the department, which identifies six components of a successful research paper in art history (see attachment). The rubric was designed to ensure some degree of consistency in evaluation, allowing faculty members to judge the papers according to a shared set of criteria, ranking each component on a four-point scale. To correct for subjective judgment, each paper was evaluated by two readers and their scores were averaged. Findings. Taking the papers as a whole, the average score was out of a possible 24 points (four points for each of the six components), or 81%. As the chart below attests, ratings were higher in every category this year, with all falling within the acceptable ( good ) range. The overall percentage rose from 65% to 81%, a considerable improvement. Component Average score Percentage Thesis/ argument Mechanics/quality of writing Organization/development Written sources Visual sources Citation TOTAL

2 The larger sample of papers we had to assess this year seventeen, as opposed to eleven in may have yielded more reliable results. This year s papers were particularly strong in incorporating written and visual sources and organization and development of an We also noticed a degree of enthusiasm and originality in this set of papers that we don t always see, and that is not easy to measure. These scores, which are the highest they have been in three years, suggest to us that students in Art History are improving their ability to write thesis-driven papers. This accords with their self-evaluations in the Indirect Assessment, which often noted the value of art history courses in learning to write. The results of the Direct Assessment inspired a spirited discussion about the scholarly apparatus of a research paper. Some faculty were bothered by the inconsistency in citation style, for instance, and others by the absence in some papers of a bibliography or reference list. We decided to require students in 400-level courses to always include a list of sources, even when they are noted in full in footnotes, to assist in our evaluation of the papers. Indirect Assessment Methodology. Our primary indirect method of assessment entailed the tabulation, summary, and discussion of the Senior Exit Survey, which all majors are required to complete before graduation. Recognizing that students often possess remarkable insight into their own learning, we wanted to see how they viewed their own achievement of our three Student Learning Goals, or core competencies for art history: 1. Graduates should be able to describe and analyze works of art and architecture, taking into account form, function, and meaning. 2. Graduates should be able to recognize and interpret a wide range of artistic traditions and cultural monuments, as well as to situate them in chronological order. 3. Graduates should be able to write a research paper combining primary and secondary sources into a persuasive In the first part of the survey, students are asked to assess the degree to which they felt they had achieved each of these three competencies in art history. In the second half, they are asked to respond to two broad, openended questions: 1. What aspects of your education in this department helped you learn effectively, and how were they helpful? 2. What might the department have done differently that would have helped you learn more effectively, and why would these actions have helped? All seventeen graduating art-history majors completed the Senior Exit Survey. The DUS tabulated the students self-assessments and analyzed the results in relation to last year s responses. Answers to the survey questions were gathered and summarized for discussion by the department. A package of these materials was provided to the faculty, with instructions, one week before the assessment meeting. Findings. Student Learning Goals. A majority of majors considered themselves very strong in all three proficiencies, with the weakest scores falling in Student Learning Goal 2: 2

3 Student Learning Goal 1. Graduates should be able to describe and analyze works of art and architecture, taking into account form, function, and meaning. 2. Graduates should be able to recognize and interpret a wide range of artistic traditions and cultural monuments, as well as to situate them in chronological order. 3. Graduates should be able to write a research paper combining primary and secondary sources into a persuasive Extremely well Very well Adequately Not very well Comparing these results with last year s, we found that the percentage of students who felt entirely confident of their abilities in all three competencies rose an average of seven percent. More than half the students felt extremely well prepared to describe and analyze works of art; the remainder felt very well prepared to do so. Less than half of the majors this year felt extremely well prepared to recognize and interpret a wide range of artistic traditions and cultural monuments; but sixty-five percent felt that they could perform that task very well, and a small number six percent felt that they could do so adequately. More than half of the students were extremely confident of their ability to write a strong research paper, with a smaller percentage claiming to be able to do so very well. Goal Extremely well Very well Adequately Not very well No. % No. % No. % No. % No. % No. % No. % No. % NB: No students in either year of the survey ranked their accomplishment lower than not very well, so the lowest possible category has been eliminated from this graph. Percentages have been rounded off for clarity. Open-ended survey questions. Answers to the questions on the second part of the Exit Survey reveal that our students have an overwhelmingly positive experience with the faculty in the Art History Department. Students commended the access to and knowledge of the professors an amazing privilege and recognized that many of their teachers were in the forefront of their fields and the authors of important scholarly texts. They regarded their professors as dedicated to teaching and excited about their fields of expertise, which made even slide-based courses lively, and considered them helpful and insightful: They fully understand your problems and needs. Students frequently commented on the emphasis given to writing and research. In their art history courses they had learned to bring together sources into an original argument, and also to read critically and analyze scholarly texts. They appreciated the opportunity to work with primary sources and valued reading responses and class discussions as a means of practicing critical thinking. Many mentioned the rigorous writing assignments in upper-level courses, which had helped them hone their writing skills. Writing research papers seems to have been particularly important as it allowed students to synthesize their own ideas with information learned in class; it also taught them to think critically about where information comes from. As one student put it, I believe that I have acquired the ability to translate what is visual into words in a concise and compelling way. Moreover, I have learned to not take anything at face value, but instead, to dive deeper to understand the meaning behind the things I see. 3

4 The most valued courses, it seems, were small and discussion-based: Learning how to discuss works of art in an educated way as opposed to simply reading about them and memorizing information is a skill that I hoped to develop, and through these higher-level, smaller classes, I felt that I did. Some students would have liked more group discussions about works of art and readings. On the whole, they prefer hands-on learning to lectures, which accounts, in part, for the nearly universal enthusiasm for museum visits. Many students cited the close connection between the department and the Carlos Museum as an asset to the program. Visiting the galleries made the study of art more comprehensive, they said, and allowed them to experience art firsthand: Seeing works in person is absolutely crucial for getting a real appreciation for and understanding of art. Some students expressed appreciation for the variety of classes and teaching styles offered by the department. One wrote of the importance of approaching art from various perspectives from learning about Shamanism, to taking a class cross-listed with classics, to learning about conservation. These many ways of looking at art were able to broaden my art horizons, and not limit myself to any one favorite type of art. The distribution requirements for the major helped students explore a wide breadth of topics and periods in art history. Yet the most common dissatisfaction by far was with what they saw as the limited curriculum. Students want more courses offered every semester so that they so they could, if they wanted, take several at once. They felt that the course offerings were heavily weighted toward ancient art and would have liked more courses in modern art. One common theme much more prevalent than in previous years was the perceived lack of community among Art History majors. One student recommended a mandatory session for majors to talk about future course offerings and graduate school prospects, something to get the major/minor students together. Another suggested a meeting with faculty to discuss upcoming courses and events, and to find out more about when certain classes would be offered to help with planning. I think more contact with the undergraduates would have helped facilitate this, because we are such a small department. Analysis Our assessment this year reveals that in AY , every art history major graduated with the ability to describe and analyze works of art more than adequately, a primary objective of our department. One hundred percent believed that they could write a better-than-average research paper, and ninety-four percent considered themselves able to recognize and interpret most artistic traditions and cultural monuments. These numbers suggest that students are attaining a sound education in the most important skills of our discipline. The analysis of this data did raise the question of when and how in the course of the major these three goals are specifically engaged. To make sure that every goal is being addressed within the curriculum, the faculty members present at the meeting completed a matrix (attached) indicating which goals they assessed at which level. The DUS then aggregated the data and found that goals 1 and 2 predominate in 100-, 200-, and 300-level courses; goal 3 the research paper is added by the majority of our professors in the upper-level courses. Because our students are required to take at least three courses at the 300-level, they should be well-prepared for the required senior seminar. At the 400-level, attention to the first and second goals varies from professor to professor, but all require a substantial research paper or project. This exercise documented what we hoped was true that despite our varying teaching styles and topics, we all have the same achievements in mind, and we all work toward them in courses at every level. Year after year, our assessment findings reveal that the department s greatest strength lies in its faculty s expertise, teaching, and accessibility. Students consider our program rigorous, and they regard the Carlos Museum as a major asset. 4

5 Action plan Our department will address the findings of the assessment in the following ways: 1. To ensure that every major feels fully integrated in the department and is aware of all it has to offer, we have planned a pizza party in the fall semester that will give us the opportunity to introduce a range of art history programs, prizes, and opportunities, and to encourage our students to actively participate in these events. In addition, we are organizing a range of activities to bring interested students together. This semester we have scheduled a screening of an exhibition-on-film; a private tour with the curator of Indigenous Beauty, the current special exhibition in the Carlos Museum; and a coffee hour with a visiting museum curator. We will continue scheduling events such as these, including an annual careers forum in February. 2. To address our concerns about student writing, we are starting at the top, with the honors students. We have restructured the program to include a series of writing workshops, including sessions on conducting research, proper citation and attribution, and the art of self-editing. This year, these will be overseen and organized by the Honors Coordinator, but in the future they may be taught by advanced graduate students in the department. We will also make more of an effort to publicize the fall symposium when honors students present their research-in-progress to all students in the department in hopes that more will consider enrolling in the program. 3. Our newly redesigned departmental course evaluation form is providing us with more specific and useful information about student learning and ensuring greater participation in the evaluation process. In keeping with our concern for more data to help us refine our teaching approaches and strengthen our curriculum, we are also revising our Senior Exit Survey to include two additional questions: a. How has the visual-arts requirement been relevant and useful to your study of art history? b. How have you been able to integrate your major with your liberal arts education as a whole? As always, we look forward to hearing what our students have to say. Submitted by Linda Merrill, Director of Undergraduate Studies Lmerri2@emory.edu 5

6 Grading Rubric for Art History Research Papers Goal 3: Graduates should be able to write a research paper combining primary and secondary sources into a persuasive Components Excellent (4) Good (3) Needs improvement (2) I. Thesis/ Argument Paper has a clearly stated and consistently developed thesis, marked by originality. Paper has a thesis, but reader must reconstruct it from the text. Vague thesis, not central to the Argument is discursive, tends to merely narrate or digress from one topic to another. Poor (1) No thesis is articulated. (Paper is mostly a list of facts.) II. Mechanics/ Writing Quality III. Organization/ Development All sentences are grammatically correct and clearly written, with good transitions, precise wording, and no spelling errors. Tone is scholarly, non-colloquial. Paper contains an introduction, main body, and conclusion. Introduction lays out the main argument (thesis) and gives an outline of what to expect in the paper. The argument is well anchored and developed in the main body of the text. The conclusion brings everything together, acknowledges possible shortcomings of the paper, and suggests what further work might be done to advance the subject matter of the paper. Most sentences are grammatically correct and clearly written. Occasional imprecise word or misuse of a word, or spelling error, which do not adversely affect scholarly tone. Paper contains an introduction, main body, and conclusion. Introduction lays out the main argument (thesis), but gives little idea of what to expect in the paper. The argument is poorly anchored and barely developed in the main body of the text. The conclusion summarizes the main argument and evidence, but does not move beyond what was presented in the paper. Several sentences are grammatically incorrect or not clearly written. Several words are misused and misspelled, compromising the scholarly tone. Paper contains an introduction, main body, and conclusion. Introduction gives an idea of what to expect in the paper, but does not effectively lay out the main argument (thesis). It may begin with a set of rhetorical questions, or an anecdote that is never fully explained. The argument is not developed in the main body of the text. The conclusion does little more than restate the problematic introduction. Introduction and/or conclusion may be too wordy or too short. Paper is full of grammatical errors, unclear writing, misuse of words, and spelling errors. No scholarly tone. Paper has no clear organizational pattern. 6

7 IV. Written Sources sophisticated use of both primary (when applicable) and secondary written sources to support a persuasive sufficient, but unsophisticated use of primary (when applicable) and secondary written sources to support a persuasive Paper ineptly deploys minimal written sources. Argument is not persuasively supported. Paper deploys few, if any, secondary written sources or only historical generalities as evidence. No support of V. Visual Sources sophisticated use of visual evidence in the conceptual and physical structure of the paper to support a persuasive sufficient, but unsophisticated, use of visual evidence in the paper to support a persuasive Paper deploys insufficient use of visual evidence in the paper. Argument is not persuasively supported. Paper deploys no analysis of visual evidence. No support of VI. Citation All evidence is properly cited in footnotes or endnotes. All evidence is cited in footnotes or endnotes, but there are some minor problems with completeness or format of some citations. Some pieces of evidence are unreferenced or inaccurately referenced, and there are problems with completeness and format of citations. little or no citing of evidence. Rev. Sept

8 LEARNING GOAL ASSESSMENT Instructor s name: For each undergraduate course that you are teaching this year, check the goals that you will address and assess (with an exam or major paper or project). Leave the other cells blank. Goal 1: Graduates should be able to describe and analyze works of art and architecture, taking into account form, function, and meaning, while demonstrating a command of arthistorical language. Goal 2: Graduates should be able to recognize and interpret a wide range of artistic traditions and cultural monuments, as well as to situate them in chronological order. Goal 3: Graduates should be able to write a research paper combining primary and secondary sources into a persuasive Course Semester Goal 1 Goal 2 Goal 3 e.g., ARTHIST 101 Fall 2015 X X 8