Washington State Teacher and Principal Evaluation Project Preparing and Applying Formative Multiple Measures of Performance Part A: Applying Multiple Measures of Performance Facilitator Guide June 2013 Office of Superintendent of Public Instruction Old Capitol Building, P.O. Box 47200 600 Washington St. S.E. Olympia, WA 98504-7200 360-725-6000 TTY 360-664-3631 www.k12.wa.us 1269_06/13
Contents Overview of the Training Module Series... 1 Audience... 1 Timing and Structure... 1 List of Training Modules... 2 Preparing for the Preparing and Applying Formative Multiple Measures of Performance Module... 4 Module Overview... 4 Context... 4 Intended Outcomes... 5 Agenda for Part A: Applying Multiple Measures of Performance Module... 5 Facilitation Notes for Module... 6 Equipment and Materials... 7 Materials to Prepare... 7 Materials to Bring... 7 Facilitator Guide... 8 Welcome... 8 Connecting... 10 Connecting Content... 10 Connecting Activity: Give One, Get One... 11 Connecting Wrap-Up... 12 Learning Segment... 12 Learning Content 1: Why Measure Effectiveness?... 12 Learning Activity 1: Types of Measure Jigsaw... 14 Learning Content 2: Year Long Evaluation Cycle... 15 Learning Activity 2-A: Measure the evidence for teacher evaluations!... 17 Learning Activity 2-B: Measure the evidence for principal evaluations... 18 Learning Activity 3: Multiple Measures Carousel... 19 Optional Activity... 21 Implementing Segment... 21 Implementing Content... 21 Implementing Activity: Mapping Criteria to Measures... 22 Implementing Debrief... 24 Reflecting... 24 Wrap-Up... 25
Overview of the Training Module Series This series of seven training modules is designed to prepare school leadership teams for the implementation of new educator evaluation systems through the following intended outcomes: Make the components of educator evaluation concrete and actionable for educators and their evaluators. Support district and school leadership teams in developing a common understanding of the new educator evaluation legislation, criteria, and frameworks, and the opportunities for professional growth and development. Provide participants with implementation tips and strategies to help schools make educator evaluation meaningful and doable. The training module series will accomplish these goals through the use of consistent, standardized training materials, detailed facilitator guides, and participant handouts. Audience The audience for each module is district-level leadership teams of four to six people. Districts may bring some or all school leadership teams together for a single training session, or training sessions may consist of teams from several districts within a region. These modules can also be used individually or with small groups of educators. The Office of Superintendent of Public Instruction (OSPI) recommends that each district leadership team include the district administrators, principals, and two to three current classroom teachers representing a variety of subject areas, grade levels, and/or student needs. The modules are designed so that school leadership teams can facilitate abbreviated or complete versions of each module to other school staff (see the Timing and Structure section for more details). For districts that are interested in bringing these trainings back to their school sites, team members should be comfortable presenting information to a group of adult learners. Timing and Structure The Preparing and Applying Formative Multiple Measures of Performance module is divided into four sessions (A, B, C, and D) designed to last two hours each. Sessions include interactive learning activities, with some activities being optional depending on the length of time allotted. Suggested homework assignments described at the conclusion of each module are intended to help participants extend and apply their learning, and are designed to take about an hour. The modules are organized into a four-part structure to help facilitators and participants pace the content appropriately. The four segments of each session are as follows: Connecting Builds community, prepares the team for learning, and links to prior knowledge, other modules, and current work Facilitator Guide: Applying Multiple Measures of Performance 1
Learning Describes key concepts and highlights various implementation scenarios; supports teams in applying knowledge and sharing ideas Implementing Supports teams in problem-solving and planning next steps for schools and districts Reflecting Engages participants in providing feedback, reflecting on learning, and closing the session List of Training Modules Introduction to Educator Evaluation in Washington. The basics of educator evaluation reforms, the evaluation criteria for teachers and principals, the four-level rating system, state and local decision matrix and a preview of the remaining modules. Using Instructional and Leadership Frameworks in Educator Evaluation. An orientation to the components of instructional and leadership frameworks, how they are different from previous evaluation tools, and how they support identification of practice across a continuum. This module will provide a jumpstart into the three instructional and two leadership frameworks. Preparing and Applying Formative Multiple Measures of Performance: An Introduction to Self-Assessment, Goal Setting, and Criterion Scoring. An overview of the types of measures required and supported by RCW 28A.405.100 and WAC 392-191A, the differences between measures and evidence, how to move beyond an observation-only evaluation system, and the benefits to the types of measures used in educator evaluation. This module includes criterion scoring guidance that has been informed by the instructional and leadership framework authors. Including Student Growth in Educator Evaluation. An overview of student growth goal setting, selecting classroom-based, school-based, district-based, and state-based tools, and using student learning data in educator evaluation. This module will offer a process for establishing student growth goals, examples of student growth goals, and a process for determining the change in student achievement between two points in time. Conducting High-Quality Observations and Maximizing Rater Agreement. An overview of high-quality observation practices with special emphasis on collecting evidence, strategies for maximizing rater agreement, and strategies for districts and school administrators to learn about and plan for achieving maximum rater agreement. Providing High-Quality Feedback for Continuous Professional Growth and Development. An overview of examples of, and protocols for, how to provide feedback to teachers and leaders so that they continue to grow and improve in their practice; how to engage faculty in these conversations; and strategies for connecting professional development planning with evaluation outcomes. Combining Multiple Measures Into a Summative Rating. While an evaluation rating is often viewed as a measure of a single point in time, it is actually the culmination of a Facilitator Guide: Applying Multiple Measures of Performance 2
comprehensive process of self-assessment, goal-setting, plan implementation, dialogue, and reflection that unfolds over months. This module provides an understanding of how to assess practice using multiple types of evidence and performance rubrics in a thoughtful, comprehensive, reliable manner and follow the process from OSPI to create an overall performance rating. Facilitator Guide: Applying Multiple Measures of Performance 3
Preparing for the Preparing and Applying Formative Multiple Measures of Performance Module Module Overview This module is designed to provide an overview of using measures and evidence in a comprehensive evaluation system for teachers and leaders. This module provides an orientation to the basic concepts of different types of measures, how to gather evidence, and how to use each in alignment with the Washington state evaluation criteria (i.e., criterion scoring) and for ongoing reflection about teaching and leadership practice. This module supports the notion that districts have flexibility to determine which types of measures (beyond observations and student growth) will best suit their context and that there is not one uniform way to create a multiple measures evaluation system. This module does not go into great depth about evidence in relationship to any of the specific instructional or leadership frameworks and instead leaves it up to the districts. This module has been informed by the ongoing collaboration with the framework authors. The training provided by the ICFFs and LCFFs offer examples of evidence related to the rubrics and guides evaluators to be intentional and thoughtful about the multiple measures of teacher or principal performance. This module does not address student growth as one of the measures because the module on Including Student Growth in Educator Evaluation is designed to address student growth measures. This module is broken into four parts (A-D) and each part is 2 hours in length. Context Participants will have varying levels of knowledge about the use of multiple measures in an evaluation system. Many participants may be familiar with observations as the main measure of teacher performance; participants may have also begun learning about the need for additional measures to address all of the criteria. Some may have attended meetings, participated in other training sessions, talked to neighboring districts, or watched recorded webinars, while others may have only a passing knowledge of multiple measures of performance and what is involved in a more comprehensive evaluation system. This module has been created so that a facilitator can deliver the content in a variety of ways. Since the module is broken into four parts (A, B, C, and D), each lasting two hours, a facilitator can choose to conduct four two-hour sessions or two four-hour sessions, or may arrange the learning and implementing components into another configuration of time that works for the participants. Facilitator Guide: Applying Multiple Measures of Performance 4
Intended Outcomes At the end of this session, participants will be able to: Understand the legislative requirements for using multiple measures as part of educator evaluation in Washington Determine specific measures to support implementation of evaluation in their districts Understand a sample evaluation cycle across a school year and implications of multiple measures Articulate the differences between evidence and measures and ways to integrate both into educator evaluation to move beyond an observation-only evaluation system Support effective self-assessment, articulation of strong goals, and gathering of evidence to support evaluation in alignment with the Washington state criteria and approved frameworks Agenda for Part A: Applying Multiple Measures of Performance Module 1. Entry Task and Welcome (5 minutes) a. Entry Task (2 minutes) b. Welcome, Introductions, and Norms (3 minutes) 2. Connecting (15 minutes) a. Connecting Content (3 minutes) b. Connecting Activity: Give One, Get One (10 minutes) c. Connecting Wrap Up/Debrief (2 minutes) 3. Learning (80 minutes) a. Learning Content 1: Why Measure Effectiveness (10 minutes) b. Learning Activity 1: Types of Measures Jigsaw (10 minutes) c. Learning Content 2: Year Long Evaluation Cycle (10 minutes) d. Learning Activity 2-A: Measure the Evidence for Teacher Evaluations (15 minutes) e. Learning Activity 2-B: Measure the Evidence for Principal Evaluations (15 minutes) f. Learning Activity 3: Multiple Measures Carousel (20 minutes) Optional Activity: An optional activity is included at this point on the agenda, if there is flexibility to add 60 minutes to the agenda. Participants will need Facilitator Guide: Applying Multiple Measures of Performance 5
laptop computers or tablets with Internet access to conduct this activity. Alternatively, it can be assigned as homework. 4. Implementing (40 minutes) a. Implementing Content (5 minutes) b. Implementing Activity: Mapping Criteria to Measures (30 minutes) c. Implementing Wrap Up/Debrief (5 minutes) 5. Reflecting (5 minutes) a. What s Next and Homework Options Facilitation Notes for Module This module is divided into four two-hour sessions (Parts A, B, C, and D). Each session has its own introduction, connecting activity, and reflection, making them almost like mini-modules. If you are doing each of the sessions separately, you will want to follow the facilitation guides for each of these mini-modules. If you are doing two or more of the mini-modules together, use your judgment about how best to transition from each section (for example, deleting the Connecting activity for the next session and going straight into the Learning segment, combining the two reflection activities into one at the end of the entire session with your audience, etc.). This facilitator guide provides the script and notes for Part A. There is a separate facilitator s guide, slide deck, and handout packet for parts B, C, and D. Facilitator Guide: Applying Multiple Measures of Performance 6
Equipment and Materials Equipment: Laptop computer, projector Materials to Prepare Make a copy of the Participant Handout packet for each participant. Prior to the session starting, create one chart paper labeled, Instructional Framework and the other Principal Framework Prior to the session starting, create one chart paper that states: Measure Formative Summative Prior to the session starting, create a set of chart papers, each labeled with the eight criteria and with three columns underneath the criterion heading: Criteria Practice Evidence Measure Create a chart-paper list with the session norms on it and post where everyone can see it. Make copies of the Debrief Cards (cardstock is best). Cut apart and give each participant a card at the end of the session. Prepare Plus/Delta Chart for use at the conclusion of the session. If inserting the optional activity and adding 60 minutes to the agenda, participants need a laptop or tablet and Internet access. Materials to Bring Put the following materials on each table: Markers (several per table) Standard-size sticky notes (at least one pad per table) Highlighters Chart paper Bring the following materials for use by you: Chime/bell or some predetermined signal (clapping three times) that will be used at several points during the meeting to get participant attention Chart paper, easel, and markers Facilitator Guide: Applying Multiple Measures of Performance 7
Facilitator Guide Welcome Slide 2 is the entry task slide. During this slide, remind participants to complete the instructions on the slide (teams talk and then post a sticky note with their district name on the readiness continuum posted on the wall). Ensure all district teams complete this task, as it will lead off the implementing section and ensure districts engage in the right activities. Slide 2 Welcome participants, introduce yourself, and ask participants to briefly do the same with their first name and role. If the group is large, ask for a quick identification of district teams by district. Explain the logistics for today s sessions (restrooms, food/beverages, etc.) This slide lists the agenda for the session. Explain: As in the previous modules, the agenda for this session is divided into four main sections. We will begin with some connecting activities to help build community, prepare everyone for learning, and link to prior knowledge, other modules, and your current work. Then we will transition to the learning segment, where the key concepts for this module are explained. The implementing portion of the day focuses on what you ll need to do to implement the content from this session after leaving here today. We ll end with reflection time so that you can provide feedback and reflect on your learning, and then we ll close the session. Slide 3 Facilitator Guide: Applying Multiple Measures of Performance 8
Explain: This module is part of a series of modules that OSPI has packaged together to support statewide implementation of new educator evaluation systems. Section 5 of Senate Bill 5895 states that the Office of the Superintendent of Public Instruction must develop and make available a professional development program to support the implementation of the evaluation systems required by RCW 28A.405.100. The program components may be organized into professional development modules for principals, administrators, and teachers. These modules aren t intended to be the only training that you receive, but are meant to build awareness and a common foundation across the state. Slide 4 Explain: Let s have a brief review of the norms for today s sessions. These were the norms we discussed during previous modules. [Read the norms on the slide and reference the chart paper posted on the wall with the norms listed.] You also have a copy of these in your participant packet to refer to throughout the session. Any comments about these norms? Any to add? Now, with these norms in mind, let s get started with the module content. Slide 5 Explain: This module is a little bit different than the Introduction to Educator Evaluation in Washington module and the Using Instructional and Leadership Frameworks in Educator Evaluation modules, in that it is divided into four smaller parts, each with a specific focus within the larger topic of multiple measures. Slide 6 Facilitator Guide: Applying Multiple Measures of Performance 9
Connecting Slide 7 is the transition slide to the Connecting segment of the module. Remind the group of the purposes of the Connecting segment (listed on the slide). Slide 7 Connecting Content Explain: The outcomes for today s session are as follows: [read slide]. Each segment of the session is aligned to one or more of these outcomes, and the divider transition slide between segments articulates those outcomes. In the Using Instructional and Leadership Frameworks in Educator Evaluations Module, we focused on the instructional frameworks and their alignment to the eight criteria. If you haven t gotten the message already through that module and any of the homework activities suggested at the end of it, the eight criteria are the bottom line. The frameworks are a means to that end. As we have done in previous sessions, when the criteria are really explored and unpacked and analyzed, we begin to understand that collecting evidence about performance for each criterion for every teacher and/or principal feels very daunting. There is much to consider: the goals of the educator and his/her students (in the classroom, the team, the grade level, the specific group, the whole school), the self-assessment, the evidence of their practice, the impact or outcomes, their reflection and selfassessment. It is the eight criteria we are trying to assess performance against, and you will need multiple measures and solid evidence to do that. This module provides an overview of these concepts. The modules on Conducting High-Quality Self- Assessments, Goal-Setting in Educator Evaluation, and Gathering Evidence (Parts B through D) go into much more depth. Slide 8 Facilitator Guide: Applying Multiple Measures of Performance 10
Throughout the module you will see slides marked with a capital G or a RCW 28A.405.100 lowercase g. The capital G indicates guidance from OSPI that draw directly from the Revised Code of Washington (RCW) or the Washington Administrative Code(WAC). The lowercase g indicates guidance that is good practice but is up to districts to determine how they want to proceed on that specific point. Slide 9 Connecting Activity: Give One, Get One Explain: In the past (or maybe still in the present) what is the most common way we measure teacher performance? [Get audience responses most people should say observation.] What is the most common way we measure principal performance? [Response to this question will be very mixed.] We re going to start thinking beyond observation and about multiple measures. In your handout packet there is a sheet labeled Give One, Get One. You may have done an activity like this before. You re going to collect nine different people s signatures. You re going to sign nine different papers in only one box based on whether this statement applies to you. This is a way to get to know the others in the room and start thinking about multiple measures of educator practice. Slide 10 Give participants eight minutes to complete their Give One, Get One signatures collection. Call the group to attention and ask them to sit down. Facilitator Guide: Applying Multiple Measures of Performance 11
Connecting Wrap-Up Explain: Let s do a quick show of hands. How many people got all nine squares filled? How many of you signed for the following box: Meaningful selfassessment? Collecting data from students about the classroom learning environment? Assembling, reviewing evidence of practice? Were there any squares difficult to find a signature for? Each of these squares mentions some of the possible measures you can consider building into your evaluation system. Slide 11 Learning Segment Slide 12 is the transition slide to the Learning segment. Explain: We transition to the Learning segment of the session, where we will cover the main content and objectives for this module. The Learning segment focuses on a SAMPLE evaluation cycle during a school year, how different types of measures and evidence fit into that, and how to move beyond an observation-only evaluation system. While it is required that 2 observations are conducted during a year, districts may consider more than 2 observations. Slide 12 Remind participants about the purposes of the Learning segment and the specific outcomes aligned to this part of the session. Learning Content 1: Why Measure Effectiveness? Explain: There are lots of reasons why new evaluation systems in many states are being created. In Washington, the foundational reason for the new system is to improve teaching and learning. According to OSPI, There is no point in creating a new system that does not improve teaching and learning. Slide 13 Facilitator Guide: Applying Multiple Measures of Performance 12
Explain: A different sort of evaluation system for both teachers and leaders requires several shifts in thinking. At the state level OSPI supports a shift from a gotcha approach to focus on improvement. There is a refocus from worrying about the 1% in trouble to the 99% interested in continually improving their practice. In 2009 10, 59,022 teachers were Satisfactory and 459 were Unsatisfactory. In the same year, 2,578 principals were Satisfactory and 41 were Unsatisfactory. There are many other shifts in thinking. We re moving beyond quality to effectiveness (or a shift from inputs to outcomes). There is a shift to using data to improve instruction and a shift away from dead data data that is accessible too late to make any difference at the classroom or school level. The data needs to be diagnostic. There is a shift from gathering evidence for the purpose of punishment to gathering evidence to support growth. And also a shift in responsibility from the evaluator to something closer to what we expect in the classroom: the evaluator and learner developing a shared experience to support growth. Slide 14 Explain: When you see these pictures what do they make you think of? [Gather audience responses.] Each of these measures something quantifiable and with a basis of comparison. And each of these measures is appropriate for different functions. When we think about teacher evaluation or principal evaluation measures, we must think about the basis of comparison and which measures are appropriate depending on the function. Slide 15 Facilitator Guide: Applying Multiple Measures of Performance 13
Explain: As we develop more sophisticated performance evaluation systems, we need to ensure the measures selected and implemented will truly help educators grow and develop in service of their students. We want measures that (read through the list of bullets). Because we want measures that help teachers grow, we need to think of the types of measures that can be used. What are some potential measures that can be used within the evaluation process? Think about how they can be used for both formative and summative purposes. According to Popham, when talking about classroom assessment, states that In a very direct sense, formative uses of classroom assessment results are improvement-oriented, but summative uses of classroom assessment results are evaluationoriented. [Use chart paper to discuss how measures can be used both formatively and summatively] Slide 16 Learning Activity 1: Types of Measure Jigsaw We have 8 potential measures that can be used for teacher evaluations, and 4 types of measures for principal evaluations [read slide with the 12 types]. We are going to read about them in a bit more detail. [Count them off by 12. Have them read the corresponding measure in Handout 3, Multiple Ways to Measure Teacher Effectiveness, and Handout 4, Selecting Principal Measures. Give 3 minutes] Slide 17 Facilitator Guide: Applying Multiple Measures of Performance 14
Once you have read about your measure, share with the whole group about that measure. What do you need in order for that measure to be able to be used as a measure of professional growth? [Give 7 minutes. Some responses you may want to get: trained evaluators; detailed data collection process, scoring rubric] Note: There is an optional activity/resource that they can use if they want more information about measures found later in the document. Slide 18 Note: For the group that does Observations, remind them that observations do not have to just be during classroom instruction. Evaluators can also observe teachers during other aspects of teacher practice, including interactions with families, colleagues, and community. Learning Content 2: Year Long Evaluation Cycle Explain: Because we have a set of measures that do these things mentioned on the previous slide and have started making those mind shifts, the evaluation systems are not business as usual anymore. Read this slide to yourself and note the directive about measures in it. Give the audience time to read the quote. We must know the what and the how when it comes to measures of educator practice. Slide 19 RCW 28A.405.100 Explain: So when we think about the school year and how the evaluation cycle fits into it. The evaluation process can be mapped onto the OSPI process model. The evaluation process is a singular analysis of all 8 criteria: instructional frameworks (criteria 1-6), professional practice (7&8), and student growth (state rubrics for criterion 3,6, and 8). Starting at the beginning of the model, educators need to consider a deep self-assessment and then Slide 20 Facilitator Guide: Applying Multiple Measures of Performance 15
based on that some carefully developed goals. Goals should have the dual focus of improving practice and supporting student growth. Some systems require educators to set at least one practice goal and at least one student growth/learning goal. Often these two activities are discussed collaboratively, peer-to-peer, administrator-to-teacher, and/or within teams. In Washington, 3 student growth goals are required in a comprehensive evaluation and 1 student growth goal is required in a focused evaluation (either criterion 3 or 6). It will be up to the district to determine if they will have an instructional and/or professional practice goal. These goals are developed within the evaluation framework, in which the 8 criteria guide educators into thinking about specific areas of professional practice. Formal observations follow, as does collecting evidence, which are ongoing activities throughout the year. We will further discuss evidence collection later on in this module, in part D, and two other modules Conducting High Quality Observations and Maximizing Rater Agreement, and Combining Multiple Measures into a Summative Rating. Throughout the year, educators can continually selfassess and reset goals based on data, assessment results, conversations, reflection, and continuous improvement. The conclusion of this cycle is the summative score or rating, which is discussed in a future module as well. Explain: Let s focus in on step 2 and 3, selecting and gathering evidence. Within these steps, the principal continues to monitor all aspects of performance, including student growth in a variety of ways. Teachers have the opportunity to offer evidence during the course of the year. Criterion and summative scoring take place at the end of the year after two observations and multiple other evidence-collecting events. This process should continuously inform educator growth within and between the school years. Slide 21 Often, the burning question is whether or not a selfassessment or specific goals set should count in the teacher s evaluation as a measure of his/her Facilitator Guide: Applying Multiple Measures of Performance 16
performance or practice. Similar questions come up for principals. To help answer that question, we need a short vocabulary lesson: What is the difference between measures and evidence? First you talk about it. Explain: Take two minutes and talk with your elbow partner about the kinds of evidence a teacher might collect throughout the year and how that evidence is different from a measure. Slide 22 Explain: A measure is compared with a standard and determines something specific, like dimensions, quantity, or capacity. Evidence makes something more clear and is an indicator or sign leading to a conclusion. Slide 23 Learning Activity 2-A: Measure the evidence for teacher evaluations! Explain: Let s use criterion 5 from teacher evaluation as an example to help us think about the difference between measure and evidence. Criterion 5 focuses on developing an effective learning environment. In your handout packet, you have the alignment between your chosen instructional framework and criterion 5, Handout 5. After that, Handout 6, you have a student behavior plan for one student, including a progress chart of how student is meeting that behavior plan. Would this behavior plan serve as evidence, meaning Slide 24 Facilitator Guide: Applying Multiple Measures of Performance 17
would it help make it clear and give the evaluator an indication of a teacher s capacity to create an effective learning environment? How would we measure, or compare, that teacher s classroom environment against a standard? Here s where the instructional frameworks are handy because they set forth some descriptive language for different aspects of practice and help us measure against that standard. Now, unlike using a measuring tape, yardstick, scale, odometer, or thermometer, we cannot compare against a standard without using some level of professional judgment about whether the standard is met or exceeded or not met. This is where evidence is useful. How would you align this piece of evidence with your instructional framework? [Each framework has some form of student/classroom behavior that it should align to]. Now look at that specific rubric row. Does this student behavior plan provide you sufficient evidence to measure the teacher on their ability to handle student behavior? Does it provide you sufficient evidence to create a score for Criterion 5? What additional evidence could we use to measure that dimension of your instructional framework? Of Criterion 5? If we used only classroom observations, would that be sufficient? Why or why not? Slide 25 Learning Activity 2-B: Measure the evidence for principal evaluations Explain. We will do a similar process as above for principals but slightly shorter. It will focus on Criterion 3, Ensuring School Safety. First In Handout 7, you have the descriptors from the two leadership frameworks for school safety. Second in Handout 7, you have three pieces of evidence 1. You have a working conditions survey completed by the teachers which assesses the degree to which teachers feel safe and supported in schools. Slide 26 Facilitator Guide: Applying Multiple Measures of Performance 18
2. Second, you have a conditions for learning survey, which is completed by students. It assesses the degree to which students feel safe in school, and they get along with other students in their school. 3. Finally, you have school-level student behavior. This is the percentage of students who were involved with each type of disciplinary action. [Note: ASWP Frameworks, and Marzano Frameworks state that School Safety is Criterion 2] Slide 27 Use the questions on slide 27 to guide the conversation Learning Activity 3: Multiple Measures Carousel Explain: We re going to engage in activity that asks you to think about the connections between the eight criteria, what that might look like in practice, and the types of evidence you might see for that aspect of practice. Then we ll consider how that aspect of practice might be measured. I will divide you into eight groups and give each group a chart paper labeled with one of the eight criteria. Your first task is brainstorm five to eight educator practices that would demonstrate this criterion. Then we will all pass our chart papers to the table on the right, and that table will brainstorm possible types of evidence of those practices. Then we will pass our chart papers to the table on the left (so they are back with their original owners) and that table will brainstorm possible measures. When you are done, please post your chart paper on the wall nearby your table. Slide 28 Give groups 15 20 minutes to complete this activity. Call the group back together and engage them briefly in the following discussion: Ask: As we look around the room at the chart papers, what do you notice? How would you synthesize the evidence described? Facilitator Guide: Applying Multiple Measures of Performance 19
Explain: This graphic represents SOME BUT NOT ALL of the measures and evidence discussed for evaluation purposes in Washington. Pilot districts selected the measures to use during their pilot implementation year this year and OSPI learned from their experiences before making recommendations to the Legislature. Slide 29 Explain: When we think about the context of multiple measures, the legislative requirements for teachers RCW 28A.405.100 are as follows [read slide]. Notice that student growth data and observation are required. Student input (such as from surveys) is optional. However, even if not included as a measure, conducting self-assessment and goal-setting activities is extremely valuable to supporting the purposes of an evaluation system: continuous professional growth and an impact on student learning and their outcomes. Slide 30 Explain: And for principals, the multiple measures referenced in the legislative requirements are as follows [read slide]. RCW 28A.405.100 Notice that student growth data is required. Staff input (such as from surveys) is optional. However, as with teachers, even if not included as a measure, conducting self-assessment and goal-setting activities is extremely valuable to supporting the purposes of an evaluation system: continuous professional growth and an impact on student learning and their outcomes. Slide 31 Facilitator Guide: Applying Multiple Measures of Performance 20
Optional Activity The following is an optional activity, if there is flexibility to add 60 minutes to the agenda. Participants will need laptop computers or tablets with Internet access to conduct this activity. Alternatively, it can be assigned as homework. Slide 32 Implementing Segment Slide 33 is the transition slide between the Learning and Implementing segments. The Implementing segment provides time for districts to engage in activities specific to their needs focused on determining which of the measures beyond observations they will use in their evaluation systems. Slide 33 Implementing Content Explain: To start us off thinking about multiple measures that will be best suited to your district, we are going to go over a few critical decisions about how best to select measures. Overall, we need to ensure that there is an alignment between the criteria and instructional frameworks and the measures that you select. Slide 34 Facilitator Guide: Applying Multiple Measures of Performance 21
Explain. When deciding which measures to use, there are some key messages that you need to consider. [Have participants read the 5 different messages] Slide 35 Explain: You are going to spend the remaining time making some decisions critical to your evaluation system with an emphasis on teachers Again, these activities are applicable to principal evaluation and can be accomplished by your team following these same steps after you leave today s session. You will go through a stepwise process of thinking about the alignment between the specific dimensions within your instructional framework that match each criteria and different measures or instruments. Implementing Activity: Mapping Criteria to Measures Explain: Turn to in your handout packet Handout 8 if you want to work on your teacher alignment, and Handout 9 if you want to work on your principal alignment. We want you to think about the criteria and the instructional frameworks that are aligned to each criteria. Think about which measures will help you gather the evidence you need to accurately measure a teacher (or principal). Don t think of the measures just in terms of creating a score, but think of them in how they can be used for teacher or principal growth as well. Slide 36 [Give participants about 15-20 minutes] Facilitator Guide: Applying Multiple Measures of Performance 22
Explain: As you can see from your Xs, some criteria match up with two, three, or more different measures. It would be RCW 28A.405.100 easy to say Great, we ll use them all. But an evaluation system must also be feasible to implement. So you have some additional decisions you should talk through as a district team. Before you talk through those questions, tally up the number of Xs in each column for the various measures. And put the total number in the first row of the table on Handout 10 of the packet. Explain: Spend the rest of the implementing time discussing with your team the questions on Handout 10, indicating a Y or N response for each question for each measure. If you aren t sure because you need to do additional research into some of these measures, feel free to reference the Guide to Evaluation Products now or later on to understand the benefits and limitations of each of these in an evaluation system. http://resource.tqsource.org/gep/ Explain. Before making these critical decisions, there are a few things we need to remember Slide 37 } To review the strength and weaknesses of each measure. (Refer back to Handout 3, Multiple Ways to Measure Teacher Effectiveness. ) } To think about how the measures accurately assess teacher practice. } To make explicit how the measures can be used to help with professional growth [Give participants 15-20 minutes to complete] Slide 38 Facilitator Guide: Applying Multiple Measures of Performance 23
Implementing Debrief Call the groups back together and ask each team to share out on the following three questions: What conclusions have you drawn about using multiple measures in your evaluation system? What measures are you considering in your evaluation system design? What are your next steps? Slide 39 Each team shares out. Reflecting Slide 41 provides the transition to the Reflecting segment of the session. The Reflecting segment is intended to be a time for participants to share feedback about the session and plan for the next session. Slide 40 To conclude the session, provide each participant one of the Debrief Cards. Call on at least one person at each table to respond to their card s prompt on the slide. If time, give participants time to go around their table and share their response to the prompt with their tabletop group and/or district team. Slide 41 Facilitator Guide: Applying Multiple Measures of Performance 24
Explain: The Part B: Conducting High-Quality Self- Assessments Module is next in the series, and it will provide an overview of conducting high-quality selfassessments. Some possible ideas for how you can extend the learning you engaged in today to those from your district who cannot be here. Slide 42 Wrap-Up Thank the participants for coming and make sure you have inserted your e-mail address and other contact information on this slide so they can get in touch with any follow-up questions. Slide 43 Facilitator Guide: Applying Multiple Measures of Performance 25