DO YOU SEE WHAT I SEE?

DO YOU SEE WHAT I SEE? DISTRICT DESIGNS LEARNING PLAN TO DEVELOP A CLEAR VISION OF EFFECTIVE INSTRUCTION As Larry Gerardo, a principal in For Wayne (Indiana) Communiy Schools, sa in fron of a compuer, he had no idea how he new projec in which he had been asked o paricipae would affec his work and he work of oher principals. Ye he knew ha For Wayne Communiy Schools had decided ha he disric would approach iner-raer reliabiliy as professional learning, saring wih he principals. Though he was uncerain of he oucome of his work, he was inrigued wih he power of principals working and learning ogeher on he RISE Indiana Teacher Effeciveness Rubric he disric s insrumen for evaluaing insrucional pracice and improving his pracice in supporing eacher learning. 12 JSD www.learningforward.org December 2015 Vol. 36 No. 6

STEPS IN THE PROCESS By Kay Psencik, C. Todd Cummings, and Larry Gerardo The disric leadership eam of For Wayne Communiy Schools, Indiana s larges school disric, has focused on becoming a learning organizaion for many years, due primarily o he leadership of Superinenden Wendy Robinson (Hirsh, Psencik, & Brown, 2014). Valuing professional learning, she parnered wih organizaions such as The Wallace Foundaion and Learning Forward and consuled Michael Fullan s work o build leadership capaciy. She waned o ensure he disric focused on developing a skilled and commied disric and principal leadership core o achieve he disric s moral purpose. In 2010, as one of is major iniiaives, he disric implemened he RISE Indiana Teacher Effeciveness Rubric a principal and eacher evaluaion sysem o clarify for eachers and adminisraors wha highly effecive, rigorous insrucion really looks like. The disric uses he rubric, which was developed by Indiana Deparmen of Educaion and guided by Charloe Danielson s Framework for Teaching, o evaluae classroom eachers insrucional pracice. The rubric s 24 measures cover four major domains: purposeful planning, effecive insrucion, eacher leadership, and core professionalism. A he same ime, he disric received a Teacher Incenive Fund gran o provide sipends for eachers based on suden performance daa and heir evaluaion. The evaluaion carried 60% of he weigh in deermining sipends. As a resul, he disric paid nearly $8 million in eacher effeciveness sipends in 2012-14. Disric leaders began o analyze he eacher evaluaions o deermine wheher principals were raing eachers across he disric wih he same lenses. They waned o be sure ha principals were observing insrucion in he same way and in agreemen on raings eachers received. Daa from five years of implemenaion of he RISE Indiana Teacher Effeciveness Rubric showed principals were all over he map in scoring insrucion. Through daa analysis, hey found ha no all principals had a clear or common undersanding of he rubric s elemens. They also realized ha he disric had lile professional learning in place for principals ha focused on eacher evaluaion. Esablish a heory of change and logic model. Esablish a leadership eam. Develop clear definiion of erms from he RISE Indiana Teacher Effeciveness Rubric. Esablish exemplars. Hire exernal parners o provide video and manage eacher raings. Hire exernal parner o make 12 videos for he disric of exemplar eachers. Engage he enire principal corps and disric leaders in defining erms. Esablish iner-raer agreemens among he leadership eam members. Esablish norms for videos. Begin conversaions. Esabilish proocols o guide conversaions and bring principals o raer agreemen. Tes o idenify areas of agreemen and discrepancies. Suppor. December 2015 Vol. 36 No. 6 www.learningforward.org JSD 13

DEVELOPING CLARITY Disric leaders deermined ha principals needed professional learning wih an emphasis on iner-raer reliabiliy especially in purposeful planning and effecive insrucion, he firs wo of he rubric s domains. Modeling heir work afer he Measures of Effecive Teaching projec (Bill & Melinda Gaes Foundaion, 2012), disric leaders sough a process ha would mee he unique needs of he disric and work oward ensuring principal raer agreemen. They realized ha he mos effecive way o do his work is o increase principals conversaions abou high-qualiy insrucion. This focus deepens he disric s effors a becoming a learning sysem. The disric leadership eam has esablished a clear vision and definiion of sandards-driven professional learning o ensure ha all in he organizaion are learning in powerful ways. Team members know ha if hey are o achieve heir moral purpose educaing all sudens o high sandards hey need o engage eachers and principals in a cycle of coninuous improvemen. The disric superinenden and disric leadership eam believe professional learning is he cenral process for coninuous improvemen. Leaders focused heir work on iner-raer reliabiliy o esablish effecive approaches o engaging principals in deep conversaions around insrucion and key elemens of he RISE Indiana Teacher Effeciveness Rubric. Disric leaders knew hey mus sar wih a clear vision of iner-raer reliabiliy and build an effecive change process ha made sense o everyone in he organizaion in order o develop his process wih fideliy, so hey did heir homework and lined up sraegic parners o buress he work. THE RESEARCH As a saring poin, disric leaders relied heavily on he work of he Measures of Effecive Teaching projec and one of is principal auhors, Tom Kane. Kimball & Milanowski (2009) and Graham, Milanowski, & Miller (2012) found ha qualiy observaion verified by a well-rained observer added validiy o he evaluaion process and ha adding even a second observer creaes even sronger raings. Consequenly, iner-raer reliabiliy is an essenial learning design o suppor principal and eacher learning ha resuls in highly effecive insrucion every day for every child. In addiion o he research, disric leaders drew on suppor from he Harvard Universiy Sraegic Daa Projec in he Cener for Educaional Policy Review. Having access o Measures of Effecive Teaching projec s principal auhors Tom Kane and Andrew Ho helped guide early heoreical underpinnings. PARTNERSHIPS Realizing ha he process would need key parners o ensure success, he disric seleced Empirical Educaion s Calibraion GUIDING ASSUMPTIONS Esablishing clear guidelines and horough, inense pracice hrough experience wih peers srenghens iner-raer reliabiliy and observer agreemen. If observers and raers have clear and concise insrucions abou how o rae behavior and can come o agreemen abou his raing, his agreemen increases consisen raings across he disric. When using qualiaive daa using wo or more observers, developing iner-raer reliabiliy and observer agreemen ensures ha resuls generaed will be useful in undersanding he effeciveness of all eachers based on common vocabulary and can be used o design professional learning. If even one of he observers is erraic on his or her scoring sysem, he enire sysem may be jeopardized as percepions of ohers may inerfere wih is effeciveness. Developing iner-raer reliabiliy and observer agreemen is more abou having clear disinguishing descripors, exemplars, and conversaions han abou simple agreemen. and Cerificaion Engine as he vehicle o hos videos and he calibraion ess. The disric also chose Edivae from School Improvemen Nework o creae videos of a diverse group of eachers o highligh eachers eaching a a highly effecive level. Learning Forward senior consulan Kay Psencik provides disric leadership wih he ools o build an iner-raer reliabiliy process grounded in he principles of professional learning and guided by a framework ha includes KASAB (knowledge, aiudes, skills, aspiraions, and behavior), heory of change, and logic model. ESTABLISHING EXPECTATIONS The disric leadership eam considered several saisical approaches during he design phase. Wha he disric really waned was absolue agreemen among all principals when hey observed lessons. The eam esablished he expecaions and percenage of agreemen as a sandard all principals mus mee. Those expecaions include: In order o become a rainer, principal leaders had o develop 90% agreemen on all elemens in planning and insrucion (domains 1 and 2 of he rubric). To receive cerificaion, principals and raers mus agree a a minimum of 85% of all elemens in planning and insrucion. For example, here are five componens in he purposeful planning caegory. The group of observers of a common eacher plan and observed video lesson mus agree on an exac raing wih he norm esablished by he leader- 14 JSD www.learningforward.org December 2015 Vol. 36 No. 6

ship eam on four of hose five elemens. The eam will discuss any discrepancy for any elemen. Team members will share heir hinking and, using heir observaion noes, he definiions, and he rubric, hey will work oward coming o agreemen abou is raing. Ulimaely, every principal will mee he sandard hrough aking a es and maching he norms esablished by he principal leadership eam. ASSESSMENTS Principals preesed for agreemen before engaging in he learning process o deermine significan areas of agreemen and disagreemen. Faciliaors monior progress by giving assessmens regularly hroughou he learning sessions so ha hey focus on he needs of he learners and differeniae insrucion. Principals are required o be cerified o rae eachers. If a principal is unable o reach he sandards of agreemen required, he disric provides inense coaching and suppor and a cerified second evaluaor for he school. Principals who don mee he sandard have muliple opporuniies o learn and mee he cerificaion requiremen. THE LEARNING PROCESS A he firs meeing of he leadership eam, Gerardo became excied abou he work. He knew i would be a challenge o do he work well, bu he believed ha if hey could do a grea job, i would have a significan impac on eaching and learning in he disric. As he became clear abou he work o be done, he and his eammaes jumped righ in. He realized ha he firs ask was o analyze he erms in he rubric and consider hose ha migh be roublesome. The eam found many words ha migh be easily inerpreed in differen ways and some ha had muliple definiions. Furhermore, he knows ha he eam s work was o develop a definiion of erms ha would mirror he disric s purpose and definiion of rigorous insrucion. Firs, he disric esablished a clear purpose for he learning process and worked o ensure ha everyone undersood i. The purpose of his program is o esablish a professional learning and cerificaion sysem for all principals and assisan principals o ensure reliable use of he rubric. The process includes six seps: 1. Ensure everyone knows he purpose and process of he work. 2. Develop precise and clear definiions of erms unclear in he disric s rubric. 3. Develop a raining manual for a group of rainers o ensure consisency. 4. Esablish ongoing disricwide collaboraion and suppor. 5. Clarify cerificaion procedures. 6. Esablish pos-cerificaion suppor and monioring. The superinenden, he cabine, disric leaders for his projec, and he faciliaor developed he disric s proposed iniiaives o improve he iner-raer reliabiliy of classroom observaion raing. These included a rain-he-rainer approach and developing a user s manual for he rubric. In a vision saemen, disric leaders laid ou he program design and described he implemenaion of he program. The disric s goal was o ensure ha every principal engaged in conversaion abou he rubric, he definiions, and heir observaions of insrucion by July 2015 and was cerified by June 2016. Because of he shor imeline, several componens of he projec needed o be managed a he same ime. The disric needed o hire a consulan o guide he process, idenify an effecive cerificaion calibraion engine and use i effecively, and creae videos showing highly effecive eachers, as well as launch a sringen professional learning program for all disric adminisraors. The disric esablished a disric leadership eam of principals from all school levels. The eam also included disric leaders responsible for eacher evaluaion as well as hose responsible for curriculum, assessmen, and insrucion. The disric se crieria for his eam, approved ha crieria hrough he superinenden s cabine, and requesed he principals o join he group. This leadership eam defined he erms in he rubric and sough exemplars o be sure all were seeing wih he same eyes. As he leadership eam became proficien in idenifying erms and recognizing hose indicaors in video of lessons, he group urned o ways o engage he enire adminisraive eam. The eam sough feedback from he larger communiy of adminisraors, hen used he feedback o make revisions. The goal was o develop as clearly ariculaed definiions as possible so ha people could see he definiion in he same way. The principals became engaged in he process and could see he value of he work hey were doing ogeher. As Gerardo repored, I shared his process wih my eachers, and hey are so excied abou his work. They are eager o deepen heir undersanding of he definiion because we all wan o improve our pracice. A he same ime ha principals were working on definiions, disric leaders conraced wih School Improvemen Nework o creae 12 eaching videos mirroring he erms being defined by he principal leadership eam. The leadership eam se crieria for he selecion of hese eachers, balancing he lis by race, gender, and sexual orienaion in order o capure he wides view of he disric. Mos imporanly, he eachers had o be raed highly effecive on he rubric. Once he principal leadership eam was saisfied wih is definiions, he work of calibraion began. This eam began observing videos and, using he rubric and heir definiions, hey raed eachers in he videos on each elemen in he firs wo domains of he rubric. The process of viewing he videos, scoring he elemens, discussing he raionale for he raings, and working oward consensus proved o be ime-consuming. The leadership eam 16 JSD www.learningforward.org December 2015 Vol. 36 No. 6

spen one day a monh viewing he videos and hosing he essenial conversaions around heir observaions. The leadership eam had o mee ha sandard of agreeing on 90% of he elemens in each domain. As he principals in he disric leadership eam began o use he definiions while viewing video of classroom eachers insead of jus he rubric, hey had an aha momen. One principal repored, When we jus used he rubric, we were all over he place in our raings here was no agreemen. We have used jus he rubric for five years, and we were in a habi of jus using he rubric. Our faciliaors had o remind us o pay aenion o he rubric erm definiions ha we had been working on for almos eigh monhs. When we used he definiions, we realized we were in closer agreemen on our raings. The disric principal leadership eam viewed video afer video, sopped and discussed each elemen, working oward agreemen, and coninued he process unil he eam me he sandards of agreemen. Afer celebraing heir success, he eam began viewing video and esablishing he norms all oher principals would have o mee. They realized hey were sill learning. Once he videos were normed, he principal leadership eam began o hos afernoon sessions wih all principals o give hem he opporuniy o work hrough he same processes and o have he same conversaions he eam had been having. They began wih observing video, scoring ha video in domains 1 and 2 using he definiions as well as he rubric, and hosing conversaions. They held muliple afernoon meeings a elemenary, middle, and high school levels. Two principals worked ogeher a each level. They creaed a proocol o engage all principals in he conversaion and work oward agreemen. They were all working oward heir firs assessmen dae 12 monhs laer. Everyone began o deepen heir undersanding of he elemens and wha hey looked like. They were beginning o wear he same glasses. On July 14, 2015, almos 12 monhs afer he sar of he work, all principals and assisan principals me o ake heir preassessmen and engage in meaningful conversaions around he insrucion hey were observing. All principals in he disric were a 65% absolue agreemen on all elemens. Sessions for he nex school year will focus on areas where hey were no in agreemen. Afer 10 monhs of deep conversaions, hey will ake heir full es. PERCEIVED INITIAL IMPACT Paricipans say ha having collegial conversaions around definiions, consrucing common meaning regarding insrucion and he rubric, and engaging wih verically aligned eams have already impaced heir sysem of suppor for eachers. They repor: Increased precision and qualiy of feedback commens; More consisen raings across all forms of feedback; Greaer clariy and undersanding in he relaionship beween domains 1 and 2; and Beer undersanding on he par of eachers and coaches of he erms and vocabulary in he rubric. CHALLENGES Paricipans encounered several challenges: Saying focused on his process as professional learning and no cerificaion. The disric did no choose o jus cerify he principals, bu o ensure here was ample ime for learning from each oher. However, when principals know hey will be esed and cerified hrough he process, hey naurally concern hemselves wih ha process raher han wha hey are learning. The leadership eam is key o ensuring ha principals say focused on heir own learning and he learning of heir peers. Ensuring ha all principals can disinguish beween evaluaion and raer agreemen. Evaluaion of eacher effeciveness has many componens. Principals do muliple drop-in observaions and view arifacs in he classroom, such as uni designs and suden work, o make a final raing. The classroom formal observaion is only one componen. Iner-raer agreemen is a focus on he lenses he principal uses o see he rubric in he classroom. Developing precision in observaions and descripive language o disinguish differences in observaions and move oward agreemen. LESSONS LEARNED By engaging in his process, paricipans came o undersand a few key conceps. Definiion of erms maers. The rubric gives principals and eachers clariy around qualiy insrucion, bu erms in he documens ofen have muliple meanings and lack clariy of vision. When principals come o agreemen abou wha he erms mean, and hen have muliple opporuniies o discuss wha hose erms look like while viewing lessons, hey begin o see ogeher. Collaboraion ime maers. Principals spend many hours evaluaing eachers. They really value ime ogeher o discuss heir observaions and work ogeher oward common agreemen abou heir observaions. The leadership eam is adaman abou ensuring ha all principals have exensive ime o work wih each oher, discuss video lessons ogeher, and learn from each oher. They requesed and were graned longer ime han planned o engage heir peers in he same level of conversaion and dialogue ha he leadership eam had experienced. Principals were concerned ha he process would be rushed and hey would no have he same rich experience as he leadership eam. They appealed o he cabine for longer working ime and muliple windows o cerify. This reques led o many Coninued on p. 23 18 JSD www.learningforward.org December 2015 Vol. 36 No. 6

Words maer es and curiosiy o validae and encourage. Then ask eachers o describe ideas in more deail so you can picure how i would ranspire in he classroom. When ime permis, have eachers use you as a mock audience o each he conen or skill. This shor rial run can help uncover vague language or plans ha lack specifics. In many cases, i will also reveal criical sequences in he eaching process ha were missing alogeher. RICHER OPPORTUNITIES FOR LEARNING Diligenly and consisenly modeled and implemened, pracical unpacking sraegies can help an educaor communiy develop shared undersanding of underlying ideas, uncover gaps in grasp of insrucional pracices, and prepare lessons wih improved clariy and richer opporuniies for suden learning. A cenral goal of communicaion is o cohere o coalesce fragmens of informaion back ogeher ino a single undersanding (Akinson, 2003). This definiion describes well one of he mos difficul asks of eaching. And i s acually he origin of he word communicaion: o make common or bring ogeher. Vague words produce underdeveloped concepions, limiing eacher growh and undersanding of pracice and leaving sudens wih ambiguous ideas. Well-defined and specified language paves he way for purposeful classroom ineracion, minimizes unproducive sruggle, and creaes opporuniies o learn. REFERENCES Akinson, C. (2003, April 22). Bulle poins kill (effecive communicaion) [Web log pos]. Available a www. markeingprofs.com/3/akinson6.asp. Ermeling, B.A. & Graff-Ermeling, G. (in press). Teaching beer: Igniing and susaining insrucional improvemen. Thousand Oaks, CA: Corwin Press. Genevieve Graff-Ermeling (genevieve.ermeling@ lhsoc.org) is chief academic officer a Orange Luheran High School in Orange, California. Bradley A. Ermeling (brad.ermeling@gmail.com) is principal research scienis a Pearson Research and Innovaion Nework. Ronald Gallimore (ronaldg@ucla.edu) is disinguished professor emerius a Universiy of California, Los Angeles. Do you see wha I see? Coninued from p. 18 afernoons of rich conversaions wih heir peers. Leadership maers. The final, mos imporan lesson from he projec was how principals ook over he leadership and faciliaion. From wriing he proocol and implemenaion o planning for he disricwide assessmen, leadership eam members were vocal advocaes for he power of a houghful, reflecive, conversaional process. NEXT STEPS As he disric moves closer oward raer agreemen among all principals, i plans o ake oher approaches. Firs, he disric will work wih eachers o undersand he definiions and use hem wih precision in heir collaboraion o design curriculum maps, unis of sudy, assessmens, and lessons o mach he descripors in he firs wo domains. The disric will also work o develop iner-raer agreemen among hose who evaluae principals and program direcors. Finally, he disric will work o ensure ha he conversaions principals are having around qualiy insrucion coninue hrough ongoing professional learning and disric leadership meeings. One principal sums up he impac of he professional learning on his work: I learned oday ha I need o pay more aenion o he rubric and he definiions when I do my observaions, said Chad Hasong, principal of Norh Side High School. I had begun o make assumpions abou wha his rubric says, and his work is going o reshape he way I observe eachers and give hem feedback. REFERENCES Bill & Melinda Gaes Foundaion. (2012). Gahering feedback for eaching: Combining high-qualiy observaions wih suden surveys and achievemen gains. Seale, WA: Auhor. Graham, M., Milanowski, A., & Miller, J. (2012, February). Measuring and promoing iner-raer agreemen of eacher and principal performance raings. Cener for Educaor Compensaion Reform. Available a hp://files.eric.ed.gov/ fullex/ed532068.pdf. Hirsh, S., Psencik, K., & Brown, F. (2014). Becoming a learning sysem. Oxford, OH: Learning Forward. Kimball, S. & Milanowski, A. (2009). Examining eacher evaluaion validiy and leadership decision making wihin a sandards-based evaluaion sysem. Educaional Adminisraion Quarerly, 45(1), 34-70. Kay Psencik (kay.psencik@learningforward.org) is a Learning Forward senior consulan. C. Todd Cummings (c.odd.cummings@fwcs.k12.in.us) is manager of he Sysem of Suppor and Compensaion and Larry Gerardo (larry.gerardo@fwcs.k12.in.us) is principal of he Career Cener a Anhis in For Wayne Communiy Schools in Indiana. December 2015 Vol. 36 No. 6 www.learningforward.org JSD 23