Utah State University Library Instruction Program Annual Report for 2012-2013 I. Introduction Major Accomplishments Kacy Lundstrom Coordinator of Library Instruction 1 The Merrill-Cazier library accomplished numerous goals this year, including taking a more curricular approach to instruction. Rather than relying on existing relationships and historic patterns to determine when and where to teach IL skills in the majors, subject librarians engaged in curriculum analysis so that they better understand how students move through the majors and how to better sequence IL skills in the curriculum so that students develop a progressively more sophisticated and deeper experience with IL. Steps towards curricular integration include the following: Nearly all subject librarians completed curriculum maps, identifying course sequences for IL instruction and outreach. Librarians selected Mindomo as the platform for creating web-based maps. LPMs created base maps to standardize formatting of curriculum maps. The Coordinator of Library Instruction shared curriculum maps with Norm Jones, Director of General Education Curricular Integration, and with HASS advisors. They are exploring using Mindomo to work with faculty and students to better represent how general education and other skills are sequenced through a curriculum. Subject librarians began to look at data from IDEA evaluations, Outcome #9, to see where faculty are identifying IL as learning outcome for their course. Subject librarians will use this preliminary data to reach out to faculty who have identified IL as a major learning outcome. Another major accomplishment is the continuation of our highly course-integrated program. Overall, librarians taught the same number of classes as last year (862 versus 861 in FY12). While there was a significant decrease in sessions between FY11 and FY12, that decrease, which may reflect the use of new methods like flipping the classroom to help with increased issues of library instruction room availability, has stabilized. This number may also reflect curriculum mapping efforts, which have shifted some of the instruction to online mediums, or which have been discontinued in order to reach another, more IL related course outcome, resulting in about the same number of face to face sessions as last year. The total number of student contact hours rose to 19, 398 (up from 18,282 in FY12). The total number of students reached also increased from 11,983 in FY12 to 12, 044 in FY13. 1 Wendy Holliday served as Coordinator of Instruction through the fiscal year, drafting and collecting much of the data in this report. Merrill-Cazier Library Instruction Program Annual Report 2012-2013 1
We continue to rigorously assess our contributions to students learning and to use those assessments to improve practice. The VALUE rubric assessment analysis was completed, including final demographic and statistical analysis. Librarians shared this data and the process of this intensive assessment in a poster session at the ACRL conference. Librarians use these results with faculty on campus to articulate the learning outcomes students seem to struggle with the most. For example, after sharing the data with the writing department, instructors were supportive of collaborating closely with librarians to revise learning outcomes and assignments to reflect the needs we saw from these shared assessments. The History Department also completed two semesters of gathering data for the Understanding Library Impacts study. Librarians and faculty have plans to gather data for another year, and are looking at ways to use this research study with other departments. II. English 1010 and 2010 English 1010 and 2010 continues to comprise a major portion of our information literacy efforts. In the summer of 2013, five lecturers received funding through the library to participate in an Information Literacy Fellows Workshop. The goal of this workshop was to revise the learning outcomes to reflect recent assessment results that indicated a need to focus specifically on certain outcomes and to better articulate where instruction efforts would be focused (and differentiated) in Engl1010 and 2010. Every section of English 1010 and 2010 has a librarian assigned to teach, design assignments and lesson plans and assist students specifically with their research assignments in those courses. An increased number of sessions were taught (248 versus 242 in FY12). Participation rates in our ENGL 1010 curriculum remain high. Revisions to this year s ENGL1010 curriculum included introducing more accountability with flipped classroom videos, through the addition of quizzes and discussion prompts in Canvas. Not all instructors used these discussions prompts, so more follow-up may be needed with individual instructors. Library integration into the professional review assignment had mixed results. It was unclear how that assignment fit into library instruction. Most instructors dropped the library instruction component in the Spring and shifted to an in-class visit where librarians discussed the research process with students. The review assignment was eliminated for the Fall 2013 semester. The number of sessions for ENGL2010 courses also increased this year (314 versus 302). As librarians learned more ways of using unique methods like flipping the classroom and other in-the-writing-classroom visits, the bottleneck for library room usage has become less of a barrier to continuing to provide effective library instruction. Also, librarians reiterated to instructors that no more than two in-the-library visits were possible with the current number of sections and library rooms available. Like last year, most ENGL 2010 instructors included one introductory lesson on the research process and/or library resources followed by a research day in the library dedicated to hands-on time with individual help from a librarian. Very few other kinds of sessions (such as on evaluation or citation) were taught, suggesting that instructors and Merrill-Cazier Library Instruction Program Annual Report 2012-2013 2
librarians have reached a consensus about the sequence of library instruction for ENGL 2010. There was some variety in lessons, including brainstorming sessions, and session focusing on evaluating information for usage; however, librarians continue to describe the content of sessions in very general terms. A large portion of the Information Literacy Fellows Workshop was spent creating new lesson plans, many which are suited to ENGL2010 outcomes, including helping students learn how to synthesize and explore existing conversations in the research. The coordinator for library instruction in writing courses experimented with a synthesis activity in four sections of ENGL2010 and assessed the usefulness of that activity by surveying participating students. The data indicates students particularly found the collaborating effort of extracting main ideas together useful. There are plans to use elements of this activity for both 1010 and 2010 in FY2013. III. Other Course-Related Instruction Subject-specific library instruction sessions increased slightly from 180 in FY12 to 183 in FY13, continuing last year s efforts to target classes where it is most effective. Participation rates by college has remained the same. Subject librarians continue to integrate with courses in ways that that focus on IL student learning outcomes. For example, librarians are integrating with subject-specific courses in the following ways: Assisting English faculty with research assignments to identify literary analysis on specific works and incorporating primary sources focused on Early American literature. Working with the History faculty to identify learning outcomes throughout the history curriculum, which included information literacy outcomes. Collaborating with Sociology faculty to incorporate JSTOR and ArtSTOR in the Museums Studies program. Creating with Engineering faculty online research guides to be used in a Hydraulic Structure Design course. Developing learning goal assignments with a Biology faculty for USU 1350, Integrated Life Sciences which is a breadth life science, general education class. Introducing research sources in to the Chemistry capstone project, such as SciFinder Scholar, Web of Science, Scopus, and the American Chemistry Society journal collection. Integrating legislative resources to track the history of various laws related to agriculture and the natural environment for an assignment in an Applied Sciences, Technology and Education course. Incorporating into multiple art courses the Art Book Room and Design collection. College Number of Sessions Number of Departments Percent of departments participating in library instruction College of Agriculture 4 6 50% Merrill-Cazier Library Instruction Program Annual Report 2012-2013 3
Caine College of the Arts 9 4 75% Huntsman School of Business 23 4 100% College of Education and Human Services 56 7 100% College of Engineering 6 6 40% College of Humanities, Arts, and Social Sciences 53 6 83% College of Natural Resources 19 3 33% College of Science* 7 6 67% *Including the WSU-USU Nursing program Table 1: Library Instruction by College Librarians taught 72 Connections courses using the same approach, the Cephalonian method, as in previous years. See: http://www.cardiff.ac.uk/insrv/educationandtraining/ infolit/cephalonianmethod/index.html. Surveys conducted post the Connections experience reported that 69% of students agreed or strongly agreed that Because of the library workshop, I am more likely to ask library staff for assistance. Librarians taught slightly fewer ENGL 1010 concurrent enrollment classes offered through local high school (38 versus 44 in FY12). The sessions combined lessons on brainstorming research questions, using library databases to search for information, and proper citation. We receive little feedback on the effectiveness of this approach. These coordination duties will transition from Betty Dance to Anne Hedrich. As part of that transition, Anne will have the opportunity to revise and reconsider approaches. We continue to have a goal to conduct some assessments of student learning in these courses. Librarians taught 14 sessions of PSY 1730, a scenario based lesson that requires students to compare anecdotal and empirical evidence of the value of a college degree. The teaching assistant and library peer mentors took over most of this instruction. In-class assessments (of student presentations and sources summarized and evaluated during class debate or vote) suggest that scenario on value of college degree might be getting stale. The PSY departments new subject librarian and library peer mentors might re-work the lesson in the coming year. IV. LibGuides & Online Materials Librarians work with faculty to create these customized course guides such as this one for an English 2010 course: http://libguides.usu.edu/2010king and guides to assist distance students with accessing and using library resources and conducting research: http://libguides.usu.edu/rcde. These course guides highlight library sources for class projects and disciplines, search strategies, and other research trips. Currently there are 320 published guides in the system. This is a slight decrease from last year (337 in FY2012), but is likely a result of re-organization and clean-up which attempted to makes sure published guides only existed for active courses. Librarians continue to look to best Merrill-Cazier Library Instruction Program Annual Report 2012-2013 4
practice and usability strategies, like MIT s 2 recent study, to improve the design and impact of these guides. In 2012, the language on the homepage to link to these guides was changed from LibGuides to Library Research Guides, in hopes that the language would be more clear for students and faculty. Total electronic views has increased from 128,037 to 145,171, indicating that guides are receiving higher usage perhaps as a result of this change. Using enrollment figures from each class, we calculated the number of hits per student enrolled. The most intensively used guides, based on hits per students, are listed in Table 2. In some cases, students probably continue to use the LibGuide throughout their coursework, even after the initial course is over, suggestion that students find the guides valuable portals for accessing library research tools. Guide Total Hits Hits/Student EDUC 6570 Susan Friedman 2032 156.3 ENGL 2010: Russ Beck 3236 147.0 ENGL 2010: Whitney Olsen 2554 116.0 ENGL 2010: Susan Andersen 1781 80.9 HIST 4990: Political Culture (Jones) 982 75.5 ENGL 2010: Park 1226 55.7 ENGL 2010: Kristal Lindheimer 1222 55.5 ENGL 2010: Bryan Johnson 1216 55.2 Public Health 5500: Research Sources 1074 48.8 PSY 3500: Johnson 542 45.1 Table 2: Top LibGuides based on hits per student Librarians have also created a number of valuable online tutorials outside of the libguide system. One tutorial, Understanding Plagiarism (https://usu.instructure.com/courses/152916), was built within a module that can be easily placed directly within the Canvas system. Another extensive module is nearly complete, which was created at the request of the Family, Consumer and Human Development Department on conducting a literature review. Erin Davis, Regional Campus and Distance Education Coordinator recently completed an in-depth, basic tutorial in response to RCDE faculty s request for a basic tutorial introducing students to the research process. See Doing your Research A-Z : https://library.usu.edu/instruct/restut/begin.html. In the next year, librarians plan to continue to assess the needs and effectiveness of these types of tutorials, particularly in comparison to LibGuide usage. V. Statistics A new system of recording statistics was created for FY13. Librarians met to discuss the usefulness of the data currently collected and revised new categories, including accounting for online consulting time and broadcast courses. System and details were clarified with 2 Henning, N. (2008). Research Guides (LibGuides) Usability Results: MIT Libraries. Retrieved from http://libstaff.mit.edu/usability/2008/libguides-summary.html Merrill-Cazier Library Instruction Program Annual Report 2012-2013 5
subject librarians. The new workflow will begin in Fall 2013 and will be assessed at the end of the semester. The distribution of sessions across the curriculum was similar to last year, with a slight increase in all categories except for concurrent enrollment and other (which mainly represents specific workshops on citation managers that tend to have low attendance). See Table 3. The number of distance classes taught through interactive broadcast was consistent with last year (26 compared to 25 in FY12). Departments from the Huntsman School of Business and the College of Education and Human Services continued to top the list of sessions per department. The departments of History and English are also heavy users of the instruction program. See Table 4. Concurrent Total English 1010 English 2010 Connections Subject Enrollment/ HS Other FY 2013 248 314 72 183 38 6 861 FY 2012 242 302 67 180 44 27 862 FY 2011 320 298 71 198 44 68 999 FY 2010 413 277 73 150 25 18 956 FY 2009 430 269 81 167 22 29 998 Table 3: Summary of Library Instruction Classes Department College Number of classes in FY 2013 Number of classes in FY 2012 TEAL ED 16 13 MGT BUS 15 9 ENGL HASS 13 12 WILD NR 11 10 PSY ED 10 7 FCHD ED 9 13 MIS BUS 8 15 Table 4: Most frequent library instruction sessions, by department One important change in the content of the instruction sessions was a shift from skills based sessions to process concept. In the past, skills-bases sessions (such as search techniques) were the most frequently taught. This reflects a larger, pedagogical shift in the IL Program to focus more on concepts, such as synthesis, exploration and evaluation instead of focusing mainly on teaching students how to find things. See Table 5. Our goal is to continue to increase concept-based sessions, especially as the new lesson plans developed by the Information Literacy Fellows Workshop are marketed and integrated into the classroom. The major format for these courses continues to be hands-on or active learning, but there is still a large number of sessions with demonstration components. As librarians continue to experiment and improve our flipping the classroom efforts, the number of demonstrations is likely to continue to decrease. See Table 6. Merrill-Cazier Library Instruction Program Annual Report 2012-2013 6
Librarians use the Association of College and Research Libraries (ACRL) Information Competency Standards as a guide. Standard Two (locating needed information) is most often covered, followed by Standard One (defining information needs) and Standard Three (evaluates information). See Table 7. Many librarians report these figures inconsistently, so we changed the wording in the documentation to define each competency, which we will assess to see if this improves reporting. Content # of classes Format # of classes Skill 243 Hands-On/Active 262 Process/concept 295 Demonstration/lecture 194 Disciplinary Tools 86 Tours 16 Orientation 132 Other 49 Table 5: Content of Instruction Table 6: Format of Instruction # of ACRL Standards classes One 314 Two 367 Three 232 Four 131 Five 1 Table 7: ACRL Standards Covered Fulltime librarians in the Reference Services Department taught an average of 74 sessions last year (versus 80 in FY12). However, librarians from other departments taught an increase in classes (160 in FY 13 versus 131 classes in FY12). Librarians recorded 670 hours prepping for library instruction sessions, or about 45 minutes of prep time for each hour of in-class time. Librarians recorded 106 hours spent on research consultations, a decrease from the year before (124). VI. Assessment A. With the adoption of the IDEA system for faculty evaluation, faculty chose from a list of 12 objectives which objectives they wanted students to use to evaluate their course. Outcome 9 relates to information literacy, stating focusing whether students are Learning how to find and use resources for answering questions or solving problems. This outcome was valuable in helping librarians know which faculty on campus, and in which classes, are identifying IL as an outcome. Subject librarians were given preliminary data and as faculty learn more about how to choose the most effective outcomes, we will continue to use this data to target our integrated instruction with the most relevant courses. Librarian Evaluation Merrill-Cazier Library Instruction Program Annual Report 2012-2013 7
Librarians participate in evaluations using these three identified IL outcomes: Learning how to find and use resources for answering questions or solving problems Learning to analyze and critically evaluate ideas, arguments, and points of view Acquiring an interest in learning more by asking my own questions and seeking answers The aggregate results for all librarians showed that 73% of students (n=569) reported exceptional or substantial gains for the outcome: learning how to find and use resources for answering questions or solving problems (a 1% increase from the previous year). Fifty-two percent of students made the same level of progress for the second goal: learning to analyze and critically evaluate ideas, arguments, and points of view (compared to 55% reported last year). Sixty-two percent of students reported exceptional or substantial progress for acquiring an interest in learning more by asking my own questions and seeking answers. See Figure 1. These results confirm other assessment results that suggest that students struggle with evaluating information based on critical reading and analysis. Figure 1: Librarian Effectiveness 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Learning how to find and use resources for answering questions or solving problems Learning to analyze and critically evaluate ideas, arguments, and points of view Acquiring an interest in learning more by asking my own questions and seeking answers Exceptional progress; I made outstanding gains on this objective. Substantial progress; I made large gains on this objective. Moderate progress; I made some gains on this objective. Slight progress; I made small gains on this objective. No apparent progress B. VALUE and ULI For our ongoing rubric-based assessment project, we completed the scoring and analysis of the HIST 4990 papers, completed demographic analysis for each course we assessed and finalized and presented the results at the ACRL National Conference in April 2013. We reported results to the Writing Program Director and to the Psychology Department faculty. We are using the results to address weaknesses in student performance, including a Merrill-Cazier Library Instruction Program Annual Report 2012-2013 8
new assignment in PSY 2010 that requires students to summarize the main questions and findings of an empirical research article, rather than only locating an article. See this online library guide for more VALUE assessment details: http://libguides.usu.edu/snapshot. The Instruction Program also participated in the Understanding Library Impacts project. This assessment instrument uses the critical incident technique to elicit student perceptions of how library collections and services support or inhibit learning activities associated with student assignments (http://www.derekrodriguez.net/understandinglibrary-impacts.html). We collected data from four sections of the HIST 4990 capstone course. The sample was small, but preliminary analysis suggests that students find library instruction and individual research consultations especially helpful as they are getting oriented, developing a thesis and gathering evidence. The study also linked the instructor s rubric scores of their final papers to their responses. Students who scored higher on their final papers reported more experience doing history research papers before the capstone course and greater confidence. They also used more library resources and services, including attending a library instruction session. The lowest scoring student reported little experience conducting historical research, high levels of anxiety, and used limited library resources and services, including library instruction. This supports the History Department s efforts to revise their curriculum to provide students with more research and library experiences earlier in their coursework and that library instruction might play a role in improved students performance and confidence. Librarians will continue to analyze the final results this summer and work with the History Department to collaborate on curriculum and assignment development. We will use the results in the next few years to address weaknesses in student learning: Implement revisions of learning outcomes (and accompanying assignments and lesson plans) into practice in ENGL1010 and 2010. Assess new assignment in PSY 2010 to see if it is meeting goals. Discuss possible approaches to research proposal assignment in PSY 3500 to see if additional instruction beyond a single session is needed and to see if assignment needs to be scaffolded better to address weaknesses in information synthesis. Work with history program to examine curriculum so that students get earlier introduction to research skills. VIII. Goals Our primary focus for 2013-2014 will be to continue to share subject curriculum maps with departments in order to continue integrating in strategic, curricular-focused ways. We also need to continue using methods that are sustainable, but also effective. Our flipping the classroom efforts have begun to help us use our time in class with students more efficiently, but creating quality, engaging videos, holding students accountable for watching them, and framing our approach for students more clearly are needed. Part of our efforts in utilizing our expertise in a efficient way has also lead us to take on a larger design role, such as with the Information Literacy Fellows Workshop. This requires us to find new Merrill-Cazier Library Instruction Program Annual Report 2012-2013 9
ways to track and measure those efforts other than simply counting consult and classroom session hours. The following are our specific goals for FY2014: Organize, revise and market new assignments and lessons created from Info Literacy Learning Fellow Workshop according to revised 1010/2010 IL outcomes. Complete curriculum map formatting and share with departments. Identify learning outcomes for disciplines/majors and then tie to curriculum maps. Discuss possibility of ULI assessment project in PSYC department. Continue to assess and refine our instructional videos for flipping the classroom. Assess LibGuide usage patterns and modify how we use and integrate this tool into our instructional practices. Evaluate librarian teaching loads and our use of face-to-face classroom time and technology in order to address class size and enrollment issues. Survey ENGL 1010 concurrent enrollment faculty about library instruction and the quality of student work in their courses. Use this information to develop an assessment plan for these courses. Establish better ways of documenting our design role with faculty rather than just documenting class sessions and consult hours. Adapt face to face activities and create resources to better meet needs of RCDE students, including creating Canvas modules, LibGuides and tutorials, and assessing the effectiveness of each in helping students learn. Collaborate with faculty in the majors to identify and better define information literacy learning outcomes for their programs. Improve librarians frequency and consistency in recording instruction and consultation statistics, with more frequent reminders, individual mid-year reports, and more specific examples of information that we need to capture. Merrill-Cazier Library Instruction Program Annual Report 2012-2013 10