Indiana University East Unit Assessment System

Indiana University East Unit Assessment System

APPENDICES Appendix A (1-3) Field Director Reports 2001, 2002, 2003 Appendix B Appendix C Appendix D Appendix E Appendix F Appendix G Appendix H Appendix I (1-4) Appendix J Appendix K Appendix L Appendix M Appendix N Appendix O (1-4) Appendix P Graduate Survey With Results Framework for Incorporating Conceptual Framework into UAS Alignment of IPSB Developmental Standards With INTASC Principles and Division Themes Pre-Service Teacher Program Interview and Writing Sample Rubric Portfolio Process Changes Portfolio Rubic Changes Field Experience Evaluation Forms M201, M301, M401, Student teaching Updated Portfolio Guidelines Assessment of Professional Potential (APP), Student Alert Student Professional Agreement Form Sample Data Base Pages Elementary & Secondary Program Cover Sheets Capstone Syllabi Results of Admissions Criteria 2

Indiana University East Unit Assessment System Final Report Introduction The Indiana University East (IUE) Division of Education seeks to develop professionals in the field who: are well grounded in knowledge essential to being informed decision makers, have the ability and desire to make reflective decisions on issues critical to education, use the best available research to guide their practice, and bring a pluralistic perspective to all dimensions of education. These goals are reflected in the three interactive themes that form the foundation of our Teacher Education conceptual framework: Teacher as Reflective Scholar, Teacher as Instructional Leader, and Teacher as Global Citizen. Our Unit Assessment System builds upon this conceptual framework incorporating INTASC principles and IPSB content and development standards. Criterion 1 Stakeholders have been involved in Unit Assessment System development and management since inception and continue to contribute to the process. The stakeholders involved are: Division of Education faculty, education students, cooperating teachers, program alumni, Board of Advisors, and Liberal Arts faculty. Division of Education faculty began discussions regarding transformation into a performancebased program Fall 1996. For the past five years, one Division meeting per month has been devoted to assessment issues with committee work occurring during the interim weeks. Each year Division retreats have been utilized for design, implementation and analysis of our UAS process. Two additional ways faculty have been involved as stakeholders are with participation in the division retreats and our annual goal setting process. 3

Pre-service teachers have primarily been involved through structured discussions initiated during class and field experience seminars. Discussions focused on what it means to be a performance-based program, portfolio organization and content, gaining feedback on rubrics and the scoring process. During Summer 2000, a select group of elementary and secondary pre-service teachers worked with faculty to extensively revise portfolio guidelines. Annually, student teachers review the portfolio process and provide feedback to the division for consideration. The primary involvement of Cooperating teachers is through required workshops where the UAS is explained and their feedback sought. They also provide input on the end of semester program evaluation form. Appendix A: Cooperating Teacher Workshop section of the Field Director Annual Reports Feedback from Education Alumni is beneficial in reviewing our unit assessment system. Summer 2000 the Beginning Teacher Mentor interviewed 35 first year teachers regarding their teacher preparation program. This data, along with graduate survey data collected Fall 1999, were used in establishing divisional goals. A survey more closely aligned with our conceptual framework and INTASC principles is being developed for future program review. This newly developed graduate survey was distributed and responses evaluated Summer & Fall 2002. Appendix B The Board of Advisors participated in the March 2000 NCATE site visit preparation, which included discussions on the implementation of INTASC and IPSB standards. They meet twice a year and continue to be involved in Unit Assessment System review. The spring 2003 meeting focused Board of Advisors input on our division themes (conceptual framework). In January 2000, the Division sponsored a campus wide meeting during which Liberal Arts Faculty were introduced to IPSB content and developmental standards. They continue to meet with content specific education faculty to determine and implement effective integration and assessment of IPSB content standards within the IUE curriculum and the Division s UAS. 4

Criterion 2 Our conceptual framework integrates INTASC principles, Indiana University East Learning Objectives and Division Themes that are foundational to program reflection, design and implementation. To establish the correlation between our conceptual framework and UAS, the grid in Appendix C (formerly Appendix A) was developed. A second grid demonstrates the alignment between INTASC and IPSB standards with programmatic integration. (Appendix D, formerly Appendix B) Division of Education faculty carefully reviewed INTASC principles and aligned course outcomes within syllabi. The addition of IPSB standards in course syllabi were finalized by Spring 2002. Three program decision points (described in Criterion 3) were established assessing pre-service teacher progress: admission to teacher education, admission to student teaching, and graduation/licensing. To ensure quality implementation, each decision point was piloted. The mechanisms used to assure on-going review of effective INTASC and IPSB integration with the conceptual framework include: annual goal setting by the Division of Education, annual report by the Director of Field Experiences, and IU East campus assessment report. Criterion 3 When UAS development began, we committed to a performance-based program, incorporated INTASC into our conceptual framework, and determined the entry and exit points for the program. Then we focused on expectations for each. The Pre-Service Teacher Program Assessment Chart provides detailed information regarding multiple forms of performance assessment that comprise the UAS, delineation of points in the program at which formative and summative decision are made, the processes through which decisions are made and the information used in those decisions. It also provides evidence that the decision point performance assessments provide qualitative discriminations among (and within) candidate performances. (Appendix E, formerly C, is the Updated: Pre-Service Teacher Program Assessment Chart). 5

During the two-year pilot, guidelines and policies were developed, implemented and refined with considerable stakeholder input. A key component was creating rubrics used to judge teacher candidates performance at each decision point. The Admissions Committee developed rubrics to evaluate performance in on-site writing samples and interviews. Appendix F provides the current interview and writing sample rubric. The Assessment Committee developed the pre-student teaching and graduation portfolio rubrics. These rubrics have two sections one includes our Divisional themes and INTASC principles and the other section examines such things as organization, explanations, selection of and variety in documentation. The rubric developed for graduation recommendation is the same as the one used for student teaching admission except that students must attain a higher minimum score and include artifacts from student teaching. Since submission of the June 2002 UAS final report, the area of greatest change has been with the portfolio process. Simply stated, the goal was to make it more humane for both students and faculty. The new process is detailed in Appendix G: Portfolio Process Changes. Current rubrics are in Appendix H: Portfolio Rubric Changes Working to inform pre-service teachers about INTASC principles starts in our first education course and continues through graduation. These standards are an integral component of student learning. For example, in the pre-student teaching field experience and in student teaching pre-service teachers use an INTASC based assessment form as a means of self-evaluation. Appendix I: Field Experience Evaluation Forms. Elementary and secondary pre service teachers prepare a portfolio demonstrating the Division conceptual framework and INTASC principles for admission to student teaching. This portfolio is revised and updated following student teaching and is submitted for review prior to licensing and graduation. Division of Education faculty identify INTASC principles, IPSB standards, Division themes and IUEast learning objectives in course syllabi. These are discussed with pre-service teachers so that the relationship between course content and the identified standards is understood. The Division web page now serves as the Student Advising Handbook. http://www.iue.indiana.edu/departments/doe/. 6

At the first decision point, Admissions Committee faculty complete writing sample and interview rubrics for each candidate, along with other criteria detailed in Appendix E. At the second and third decision points, teacher candidates complete an INTASC based portfolio, a key component of our Unit Assessment System. We ask that students demonstrate Who they are as Teachers in other words, we want the portfolio to be a personal document rather than prescribed. Our requirements are limited; he/she must have a video with accompanying lesson plan and reflection of the videotaped lesson and he/she must demonstrate a working knowledge of and ability to implement the INTASC principles and Division themes through the selection of artifacts, explanations and reflections. The preservice teacher determines everything else. The Portfolio Guidelines on our website contain suggestions for organization, artifact selection, and layout. There are several appendices that include the INTASC principles, our conceptual framework, the rubrics, and some sample excerpts from outstanding portfolios. Appendix J: Updated Portfolio Guidelines Fall 2000 the Assessment Committee planned and implemented a workshop for developing interrater reliability. Each Faculty read a portfolio designated as the target the best pilot portfolio that had been submitted. The outstanding parts of the target portfolio were identified and discussed. Faculty individually read and scored a second portfolio. The ensuing discussion focused on scoring differences and resolution of discrepancies. To maintain inter-rater reliability, the assessment committee monitors preparation of new faculty and reviews policies and procedures each semester. All Division of Education faculty are involved with evaluation of portfolios with a member of the assessment committee as one team participant for evaluating each portfolio. The Director of Field Experiences is permanent chair of this committee. As the number of education students has increased it has become impossible for a member of the assessment committee to always be one team participant for evaluating each portfolio. However, with on- going inter-rater reliability training each semester and at least annual analysis of the data and the process relative to portfolios all division faculty are 7

equally well prepared for portfolio evaluation. Efforts are now focused on insuring that newer faculty evaluate portfolios with more experienced faculty. Another form of assessment used throughout our program is the Assessment of Professional Performance (APP). Pre-service teachers use it for self-evaluation at various points in our program, and faculty have the opportunity to evaluate pre-service teachers as needed. Cooperating teachers use it after each field experience, and at the conclusion of student teaching. Appendix K: Assessment of Professional Potential (APP) The APP is comprised of four areas. First is personal and professional responsibility that includes such items as meeting deadlines and keeping commitments. The second area is social/emotional/physical well being, third is interpersonal relationships, and the final area is an overall judgment regarding the pre-service teacher s professional qualities. Data concerning the APP continues to be collected and analyzed. Early data indicates that it meets the goals for which it was developed. This form and process is available to all university faculty as well as the division for addressing inappropriate dispositions and exceptional performance at any time that they occur. In addition, a form for students to sign so that they would know the expectations for professional behavior and for dismissal from the Division was developed and first implemented spring 2003. Appendix L: Student Professional Agreement Formative and summative data for pre-service teachers is maintained in clinical files, advising files and are recorded in our Education Advisory database. This Microsoft Access database includes a place for comments, when the student was admitted; their major, endorsements, test scores, GPA and other pertinent information. We continue to add data needed for effective evaluation of student progress and of our program. Our database specialist is responsible for maintaining the database. (Appendix M, formerly D,: Sample Data- Base) 8

Pre-service teachers are consistently provided feedback regarding program progress. Pre-service teachers meet at least twice per year with an assigned advisor to register for classes and review their program sheet that clearly delineates all program expectations and requirements. (Appendix N: Elementary and Secondary Program Cover Sheet). Pre-service teachers are audited at the three entry points and receive information in a timely fashion so issues can be addressed as needed. When preservice teachers apply for student teaching, they have an interview with the Director of Field Experiences who reviews program status. Praxis I and II scores, semester grades and other relative data are addressed at each entry point audit. Professional growth is addressed through cooperating teachers feedback on the APP, student selfassessment with the APP and education and liberal arts faculty input, as needed, regarding professional issues. Feedback regarding field experiences comes from cooperating teacher INTASC based evaluation forms for M201, M301, M401 and student teaching and from faculty and university supervisor site visits. Completed rubrics from pre student teaching and post student teaching portfolios provide information regarding how effectively pre-service teachers are able to demonstrate their ability to meet division themes, INTASC principles and IPSB standards. An appeals process is in place for each decision point. The student must write a letter to the Admissions Committee, who may prepare a plan for reacceptance. If an appeal is rejected, the student may appeal to the Division of Education Chair. Should the appeal fail there, it then goes through the regular University process, which is available on our division website. http://www.iue.indiana.edu/departments/doe/retention.htm Multiple performance assessments of an individual are aggregated to make a final summative decision regarding candidacy. Program sheets are used for tracking pre-service teacher progress. As they continue through the program, the program sheet is regularly updated and kept in the cumulative advising files. Data from our data base can be analyzed in many different ways. 9

Evidence of validity of decision resulting from the use of summative decision point rubrics is an on-going process. Appendix A, formerly C identifies how data is used to improve all aspects of our program. Criterion 4: Every semester data is collected regarding numbers of students who apply for admission and results on admission criteria. Appendices A & P delineate the percent of candidates who passed, passed with remediation, or failed criteria at each summative decision point. To assure that IPSB standards are appropriately, adequately and cumulatively incorporated and assessed throughout our program, a chart (Appendix D) was developed that correlates IPSB standards with INTASC principles, Division conceptual framework and IU East learning objectives. A grid was developed to establish where standards are incorporated into education courses and how certification areas will be met. Each education faculty is responsible for identifying appropriate IPSB standards within their courses. Summer and Fall 2001, the Division Chair and Assessment Committee Chair conducted two meetings with adjuncts during which we discussed our UAS, worked further on the integration of divisional themes, INTASC principles and IPSB standards in their courses and identified assignments from their classes that might be appropriate for the portfolio. Cooperating teachers are introduced to INTASC principles and IPSB standards during a workshop required to become cooperating teachers. They use INTASC based forms to evaluate preservice and student teachers. Each semester faculty involved with field experiences meet with cooperating teachers to discuss the integration of INTASC and its relationship to pre-service teacher assessment. Feedback provided by cooperating teachers on how effectively pre-service teachers are implementing the INTASC principles is analyzed and will be included in the Field Director s annual report. (Appendix A) 10

The delineation of the percent (proportion) of the candidates who successfully completed the State required licensing exams is shown in the Title II tables included at the end of this report. IU East Division of Education ranked in the first quartile for the State of Indiana. Praxis score data is, also, reported in the Field Director s Annual Report Appendix A. While our Beginning Teacher Mentor Program works with first and second year teachers, we have not yet had candidates who entered and successfully completed the two- year induction period. A Mentor Training proposal to IPSB (submitted summer 2003) is in the approval process. The annual Field Director s Report provides data that enables us to aggregate individual candidate assessments that are used to evaluate the quality of courses and the program. Data considered includes pre-service teachers portfolio scores, cooperating teacher evaluations, and Praxis results. Faculty use this, and other data, to reflect on and revise the program. Criterion 5: Data collected on candidate performances is used to make decisions on curriculum and program practices. First, data is used to track growth in understanding and integration of Division conceptual framework and INTASC Principles. Using our database, we determine individual progress as well as identify patterns within our elementary and secondary program. Several examples are provided below. Redesign Courses: Data indicated that students need greater content related to being global citizens so liberal arts requirements were adapted. Now pre-service teachers select from more course options that respond to cultural diversity. M300 students (Teaching in a Pluralistic Society), engage in a service learning activity that often includes diverse populations. Improve Field Placements: Data indicated that increased time in special education classes would be beneficial so the M201 field experience was changed to address this need. Other changes based on preservice teacher and faculty input involve how student teaching time is distributed. Pre-service teachers 11

will now have one week of observation, one phase-in week and then 9-10 weeks of total teaching. Finally, we revised field experience evaluation forms so that each included our conceptual framework, INTASC principles and APP criteria. Thus, we will be better able to track pre-service teacher professional growth over time. Improve Preparation of Portfolios: Through data analysis we learned where our pre-service teachers were struggling on some aspects of portfolio development and how to better teach certain concepts. For example, we incorporated teaching how to write explanations and reflective entries into the prestudent teaching field experience seminar when this was identified as a weak area in pilot portfolios. Capstone syllabi course changes were designed to assist students in preparation of their portfolios. Appendix O. Changes in the portfolio process were previously noted in Appendix G. Analyze Praxis Scores: This year our Director of Field Experiences analyzed praxis scores such that this data will be shared with our liberal arts divisions. This will allow further discussion among faculty regarding how to best improve our program. This analysis continues to be a part of the annual Field Director s Report. Identify Remediation Resources: We work with our Tutoring Center and the Writing Program Director to identify courses that will help students needing remediation. Students now have access to the PLATO system and work with tutoring center staff to prepare for Praxis retakes. Changes in Conceptual Framework: Student feedback and aggregated portfolio scores reveal two conceptual framework themes as more strongly understood and practiced (Reflective Scholarship and Instructional Leadership) by students. The third theme, Global Citizenship, appears to be a little less practiced and not as well understood. While this has led to much analysis and attention at division meetings and retreats during recent semesters, Global Citizenship remains an important theme in our conceptual framework. Efforts are in process to clarify the definition of the Global Citizenship theme for the purpose of better facilitating students understanding and ability to put this theme into practice. At the pre-student teacher and student teaching level, this theme has been a focus of portfolio 12

preparation discussions which includes examples from portfolios of previous semesters as well as how Global Citizenship applies to the students current experiences. This will continue to be an area where we work to achieve balance in our students understanding and practice of the conceptual framework themes at an equally high level of success. Changes in the program, including general education, professional education, and field-work have resulted directly from analyses of candidate performances and evaluative assessments. In Fall 2000, the Division of Education initiated a program review to ensure a closer match between our conceptual framework and elementary program. Our primary goals were to provide more flexibility so that students could pursue areas of interest and to provide increased depth in content areas. Criterion 6: To assure that the unit assessment system is continuously managed, the Chair of the Division of Education is in charge of overseeing the UAS. This involves monitoring and reviewing the conceptual framework, program and curricula and then initiating faculty discussion regarding needed revisions. The Division Committee Structure includes Assessment and Admissions Committees. Following a two- year pilot, the UAS plan has been successfully in place for two years. Implementation and review continues to date. The Division Assessment Committee is responsible for developing, revising, and analyzing assessment and sharing information on a regular basis at Division meetings. The Portfolio Committee is a component of the Assessment Committee. They update the portfolio guidelines and other issues related to the portfolios including training of faculty in inter-rater reliability. An Admissions Committee, comprised of Division of Education faculty, is responsible for determining when pre-service teachers are accepted into the Teacher Education program. The Beginning Teacher Mentor is responsible for ensuring coherent integration of INTASC principles with first year teachers. A Division member serves on the IU East Assessment Committee to ensure 13

congruence between Divisional goals and university requirements. Advisory Board, Steering Committee and Cooperating Teachers provide yearly feedback on the program. Data is managed in three different ways for each pre-service teacher. Data is recorded on students Division of Education record and it is collected and stored in students Clinical Files. This data is collected and recorded in our database for retrieval and analysis for pre-service teacher and program review. Clinical files are accessible only to Division of Education faculty, professional staff and our secretary. The database is accessible through a password system, available to the Division of Education Chair, members of the Assessment Committee and Licensing Officer. The Data Specialist returns hard copies of all data for filing in student files assuring that data is not repeated. The Data Specialist keeps back-up copies of computerized data and is responsible for making changes in the database. The assessment committee has responsibility for collecting data and assuring that changes in the database are made. To assure that the Division assessment system is linked to the IU East campus assessment plan we examined our IU East objectives for interconnections with our Division assessment system (including Division conceptual framework, INTASC principles, and IPSB standards). From this examination we determined that all campus objectives are met and exceeded by the unit assessment system in place. Appendix D displays this interconnectedness. Criteria 7: The UAS provides for review and revision of the assessment system on a regularly planned schedule. Each fall the Division of Education has a retreat at which data from the previous year is analyzed, areas for change are identified and annual goals are determined based on this information. Division meetings regularly focus on discussion of the Unit Assessment System and assessing progress toward annual goals. The responsibility for reviewing and revising the program rests with faculty as a unit. 14

All stakeholders have opportunity for input. The annual Field Director s Report, described earlier, is the data repository. Also carefully considered is information gleaned from course evaluations, graduate surveys, student teaching surveys and anecdotal conversations with students, graduates and cooperating teachers. The quality of student outcomes from year to year is the key criteria for UAS revision. This is both formative and summative as we use the information gained from requirements for entrance into the teacher education program PRAXIS I scores, data from interview and writing along with the summative data from Portfolios, PRAXIS II scores, and cooperating teacher evaluations for guidance in regard to course and program changes. Assessment is integrated with the Division s conceptual framework and relevant standards and includes ongoing discussion with constituents. 15

EVIDENCE FOR EACH CRITERION: next Board of Examiners visit. Criterion 1: Compilation of INTASC pre and post and student teaching evaluation forms to provide evidence of student professional growth and development Compilation of Cooperating Teacher INTASC forms to provide evidence of their judgments relative to student professional growth and development. Division minutes where feedback was discussed and decisions made. Division Meeting and faculty retreat minutes where faculty discussed changes. Any documentation (e.g., advising sheets, student handbook, curricular changes) providing evidence of program changes. Board of Advisors and task force minutes. Criterion 2: Integrated Conceptual framework and UAS grid Alignment of IPSB standards with INTASC principles and conceptual framework grid Programmatic grid showing how developmental standards are integrated. Minutes from meetings with IU content faculty Course syllabi that reflect conceptual framework, INTASC principles, IPSB standards, and IU East learning objectives Criterion 3: Program Assessment chart indicating 3 decision points Criteria 6: Written policies regarding the procedures for data management. Copies of the Division s annual campus assessment report Grid connecting IPSB Standards with INTASC principles, conceptual framework, and IU East learning objectives (attach to report as well) Criteria 7: Division Goals Copies of the Division s annual campus assessment report Copies of the Annual Student Teaching Report Division Meeting and Retreat minutes Program Sheet 16