Teacher and Leader Keys Effectiveness System TLE Electronic Platform Administering Surveys of Instructional Practice The contents of this document were developed under a grant from the U. S. Department of Education. However, those contents do not necessarily represent the policy of the U. S. Department of Education, and you should not assume endorsement by the Federal Government.
Table of Contents Introduction... 3 Figure 17: Example of Survey Items... 3 Purpose of the Survey Protocol... 4 Survey Security... 4 Survey Protocol... 4 Preparing for the Survey... 6 Administering the Survey... 6 Grades 3-5 Surveys of Instructional Practice Protocol... 7 Grades 6-8 Surveys of Instructional Practice Protocol... 9 Grades 9-12 Surveys of Instructional Practice Protocol... 11 Following Survey Administration... 13 Use of Survey Data... 13 Survey Results... 13 Figure 18: Survey Results Summary Sheet (Sample for Grade 7 Teacher)... 14 Figure 19: Survey Results for Each Standard by Mean... 14 Appendices... 15 Checklists for Survey Administration... 16 TLE Electronic Platform Survey Quick Reference Guide- School Administrator... 17 TLE Electronic Platform Survey Quick Reference Guide- Student... 19
Introduction Another measure of the Teacher Keys Effectiveness System consists of Student Surveys of Instructional Practice. Surveys are an important data collection tool used to gather client (in this instance, student) data from individuals regarding the students perceptions of teacher performance and will be administered through the GaDOE TLE Electronic Platform. Advantages of using a survey design include the rapid turnaround in data collection, the limited cost in gathering the data, and the ability to infer perceptions of a larger population from smaller groups of individuals. In the Teacher Keys Effectiveness System, surveys will be used as a measure of teacher effectiveness and documentation to support four of the TAPS standards. These four standards: Standard 3-Instructional Strategies, Standard 4-Differentiated Instruction, Standard 7-Positive Learning Environment, & Standard 8-Academically Challenging Environment reflect the direct experience of students in classrooms. Multiple data sources enable the evaluator to obtain a more accurate picture of performance and assist the teacher in increasing student success. These data sources do not stand alone but are complementary to each other and should be integrated into the process of evaluation to provide a richer portrait of teacher performance. The flaws of one data source are often the strengths of another, and, by combining multiple methods, evaluators can make more solid judgments regarding teacher performance and make decisions that are supported by multiple types of data. Student surveys may help the teacher set goals for continuous improvement (i.e., for formative evaluation). Surveys provide feedback directly to the teacher for the purpose of professional growth and development. Student surveys may also be used to provide information to evaluators that may not be accurately obtained during observation or through other types of documentation. The surveys ask students to report on items they have directly experienced. Three different versions of the student survey (Grades 3-5, 6-8, and 9-12) will be provided. The versions are designed to reflect developmental differences in students ability to provide useful feedback regarding their teacher. All surveys are to be completed anonymously to promote honest feedback. In addition, all surveys are examined to ensure they are written at an appropriate readability level using the Flesch-Kincaid Readability Scale. An example of survey items is shown in Figure 17. Figure 17: Example of Survey Items Please note that responses in grades 3-5 are Yes, Sometimes, and No. Strongly Agree Agree Disagree Strongly Disagree My teacher encourages me to participate in class, rather than just sitting and listening. 3 2 1 0 My teacher encourages me to ask questions in class. 3 2 1 0 September 15, 2014 Page 3 of 19
Purpose of the Survey Protocol Georgia Department of Education This Survey Protocol provides procedural information as well as detailed instructions for administering the student surveys. It is imperative that the surveys be administered according to the directions in this manual. If you have any questions, contact your District Teacher Keys Effectiveness System Site Coordinator or your Georgia Department of Education Evaluation System Specialist. Uniform survey administration is needed to ensure high-quality data. To ensure this uniformity, the Survey Administrator s responsibilities are clearly outlined and detailed in this protocol. Step-by-step instructions for administering the surveys are designed to protect the integrity and security and must be followed exactly. It is recommended that Survey Administrators read this manual and become thoroughly familiar with all survey administration procedures before administering the surveys. Survey Security Surveys may be administered by anyone the Site Administrator deems appropriate. This includes any personnel such as the regular classroom teacher, paraprofessionals, instructional coaches, media specialists, and or counselors. The classroom teacher can administer their own surveys by following the survey protocol. Survey conditions, especially the supervision and seating arrangements of students, should be designed to maximize anonymity and encourage students to respond honestly. The Survey Administrator must supervise the survey administration at all times. All instances of survey irregularities must be reported to the building administrator immediately. If questions arise, or if any situations occur that could cause any part of the survey administration to be compromised, the district coordinator should be notified. The appropriate district staff should contact the GaDOE Division of Teacher and Leader Effectiveness for guidance as needed or appropriate. Administration of the Survey If a common media center or computer lab is not available, the Survey Administrator will need to identify a location where the survey can be administered to small groups of students based on the number of available computers. All appropriate accommodations need to be made for students with disabilities and English Language Learners. Students are able to comprehend at a higher level when listening to the survey items read aloud. Therefore, it is considered appropriate for the readability of 3-5 surveys to be written at a slightly higher readability level. All students in Grades 3-5 will have the surveys read aloud. Survey items for all students have read aloud capability within the electronic platform. All appropriate accommodations need to be made for students with disabilities and English Language Learners, based on Individual Education Plans (IEPs) or language instruction education plans (extended time, read aloud, September 15, 2014 Page 4 of 19
dual language dictionaries, etc.). Students with severe/profound disabilities, if sampled for participation in the surveys, may or may not participate, with needed accommodations, as determined to be appropriate by the IEP committee. Surveys will be read to Visually Impaired students. A secure protocol for entering the student responses from a Braille survey into the electronic platform is provided for educators. Auditory devices may also be utilized. The use of a toggle switch within the electronic platform will allow the survey to be read through headphones for any student requiring this accommodation. Additionally, the electronic platform provides translation into other languages through use of a drop down menu allowing the student to select from a list of multiple languages. Districts will have multiple options for selecting survey windows. From October to March an open survey window will be available for schools to select a time frame that does not interfere with testing or other uses of computer labs, etc. The multiple survey options will accommodate courses taught only during first semester, only during second semester, all year, or for shorter segments within the school year. The appropriate survey window for a course and/or teacher sample will be selected by the district or principal. Surveys may also be administered multiple times during the year at the district s or principal s discretion. All surveys will be administered using the GaDOE TLE Electronic Platform. Surveys will be administered in the following manner. In self-contained classes (e.g., elementary teachers, special education teachers) all students will be surveyed unless otherwise determined by the IEP committee including the classroom teacher or case manager, a school administrator, and the parent. In departmentalized settings (e.g., some upper elementary, middle and high school teachers, elementary PE and music teachers), the principal will select students to be surveyed by class periods. There is a possibility that students may be selected to complete surveys on more than one teacher, but no student should be sampled to respond to surveys on more than two teachers in any given survey administration period. In non-departmentalized elementary staff and self-contained teachers, all students will be surveyed. In departmentalized elementary and multi-class (art, music, PE, etc.) teachers, the principal shall choose at least two class periods per year consisting of different students during which all students in these class periods will complete the survey. In middle school and high school classes, the principal shall choose at least two class periods per year consisting of different students during which all students will complete the survey so that those surveyed are representative of the students the teacher is teaching. In resource or self-contained special education, ESOL, etc., classes, the principal shall schedule a time when all appropriate students taught by these teachers can complete the survey. September 15, 2014 Page 5 of 19
Preparing for the Survey Georgia Department of Education Before administering the surveys, the Survey Administrator ensures the following: Survey login key and passcode information is created for students to access the survey. (See appendices for GaDOE TLE Electronic Platform Quick Reference Guide - School Administrators.) All computers are ready for students to access the survey. The seating arrangement allows for complete anonymity. There is a planned activity for those students who complete the survey more quickly (return to their classroom, read a book, etc.). Administering the Survey Administration time: 20-30 minutes anticipated NOTE: The survey is NOT timed. Students may take as much time as necessary to complete the survey. Preparations Required: Each student must have unique survey key and passcode to access the survey in the GaDOE TLE Electronic Platform. Room arrangement must ensure student privacy. Log onto https://tle.gadoe.org. The bold portions below are to be read exactly as they are written in order to ensure consistency of implementation. Today we are asking you to tell us about how your teacher, <insert name>, teaches and what happens in your classroom. You should think carefully about what each statement means. Remember your responses do not indicate whether you like your teacher or not. They indicate what is happening in your classroom. Your teachers and school will not know the answers you marked, but they will know how many students marked each answer. Please log into the website https://tle.gadoe.org. On the home login screen, you will click on Anonymous Survey Login to begin the survey. Please use the survey key and passcode provided. (This step may be completed in advance by the survey administrator for those students who may struggle with logging on to the site.) (Assist children to ensure all students see the survey.) September 15, 2014 Page 6 of 19
Grades 3-5 Surveys of Instructional Practice Protocol Now look at the computer screen. You should see the survey items and the name of the teacher for whom you are completing the survey. You will also see choices across the top that are yes, sometimes, and no. Raise your hand if you don t see these choices. [Pause.] (Assist children who do not see the statement and/or the boxes.) The word yes means this happens in your classroom. The word sometimes means this happens in your classroom sometimes. The word no means this does NOT happen in your classroom. Raise your hand if you do NOT understand what each of the answers mean. [Pause.] Repeat and clarify the meaning of each answer choice as necessary. Remember your responses do not indicate whether you like your teacher or not. They indicate what is happening in your classroom. You will see the speaker next to the question. Please click on it to hear the question read to you. Please listen to the question and mark your answer. Raise your hand if you do NOT understand how to use the speaker to hear the question read to you. [Pause.] You also can read the questions in another language. Please click on the drop down menu on the right side of the screen to select a different language. Raise your hand if you do NOT understand how to choose a different language to read the questions. [Pause.] (Please note, the questions are only read aloud in English.) We are ready to begin. This survey is not timed. Please carefully consider each survey item and respond thoughtfully. Your input is important. Read Aloud Option: I will read each survey item twice. Please mark yes, sometimes, or no depending on how often this happens in your classroom. (Please read each item twice. Survey items are found on the next page.) Once you finish the survey, please click on the Save & Mark Complete button found at the bottom or top left side of the screen. You should see the message The End. The Survey is now over. We are finished with the survey. Thank you for your attention and your thoughtful responses. Please click Sign Out on the top right corner of the screen to exit the survey. September 15, 2014 Page 7 of 19
Grades 3-5 Surveys of Instructional Practice Georgia Department of Education Grades 3-5 Standard 3: Instructional Strategies 1. My teacher wants me to ask questions about what we are learning. 2. My teacher gives us different kinds of activities to make class more interesting. 3. If I don t understand something, my teacher explains it in a different way. 4. My teacher helps me understand what I still need to work on at school. 5. My teacher shows me how I can use what I learn at home and in the community. Grades 3-5 Standard 4: Differentiated Instruction 6. The work my teacher gives me is at the right level for me. 7. My teacher understands where I need to improve. 8. My teacher has me work in class with different groups of students. 9. My teacher gives students as much individual help as they need. 10. My teacher wants us to share our own ideas in class. Grades 3-5 Standard 7: Positive Learning Environment 11. My teacher cares about my learning. 12. My teacher is good at getting students to behave well in class. 13. My teacher explains the instructions if I don't understand them. 14. I feel comfortable asking my teacher questions. 15. My teacher treats all students with respect. Grades 3-5 Standard 8: Academically Challenging Environment 16. The work I am assigned in this class challenges me. 17. My teacher encourages me to do the work even when it is hard for me. 18. My teacher doesn t let me give up when the work gets hard. 19. My teacher says it is OK for me to make mistakes, as long as I try my best. 20. My teacher wants me to try new things, even if they are hard for me. September 15, 2014 Page 8 of 19
Grades 6-8 Surveys of Instructional Practice Protocol Please read the following statements carefully and thoughtfully consider your response to each statement. Remember, your responses do not indicate whether you like your teacher or not. They provide information about classroom instruction and environment. Now look at the computer screen. You should see the survey items and the name of the teacher for whom you are completing the survey followed by four possible answer choices Strongly Agree, Agree, Disagree, and Strongly Disagree. Please raise your hand if you do NOT see the survey and the answer choices. [Pause.] The choices indicate your level of agreement with each statement. Please raise your hand if you do not understand the answer choices. [Pause.] Repeat and clarify the meaning of each response as necessary. You will see the speaker next to the question. Please click on it to hear the question read to you. Please listen to the question and mark your answer. Raise your hand if you do NOT understand how to use the speaker to hear the question read to you. [Pause.] You also can read the questions in another language. Please click on the drop down menu on the right side of the screen to select a different language. Raise your hand if you do NOT understand how to choose a different language to read the questions. [Pause.] (Please note, the questions are only read aloud in English.) We are ready to begin. This survey is not timed. Please carefully consider each survey item and respond thoughtfully. Your input is important. Read Aloud Option: I will read each survey item twice. Please mark Strongly Agree, Agree, Disagree, or Strongly Disagree depending on how often this happens in your classroom. (Please read each item twice. Survey items are found on the next page.) Once you finish the survey, please click on the Save & Mark Complete button found at the bottom or top left side of the screen. You should see the message The End. The Survey is now over. We are finished with the survey. Thank you for your attention and your thoughtful responses. Please click Sign Out on the top right corner of the screen to exit the survey. September 15, 2014 Page 9 of 19
Grades 6-8 Surveys of Instructional Practice Georgia Department of Education Grades 6-8 Standard 3: Instructional Strategies 1. My teacher encourages me to participate in class, rather than just sitting and listening. 2. My teacher encourages me to ask questions in class. 3. My teacher frequently checks to see if we understand what is being taught. 4. My teacher takes time each day to summarize what we have learned. 5. My teacher gives me assignments that require me to connect knowledge from several different sources. Grades 6-8 Standard 4:Differentiated Instruction 6. My teacher chooses activities and assignments based on what students need to learn. 7. My teacher gives students as much individual attention as they need to be successful. 8. When I don t understand something, my teacher tries to figure out why I don t understand it. 9. The work my teacher gives me is at the right level for me. 10. My teacher allows me to work with different groups of students depending on the activity we are doing. Grades 6-8 Standard 7: Positive Learning Environment 11. I feel comfortable asking my teacher questions. 12. My teacher cares about my learning. 13. My teacher holds students responsible for their behavior. 14. My teacher explains the instructions if I don't understand them. 15. My teacher treats all students with respect. Grades 6-8 Standard 8: Academically Challenging Environment 16. The work assigned in this class challenges me. 17. My teacher expects me to do my best. 18. When I am confused by something, my teacher will not let me give up until I understand it. 19. Because of my teacher, I push myself to learn as much as I can. 20. My teacher encourages me to try new things, even if they are difficult for me. September 15, 2014 Page 10 of 19
Grades 9-12 Surveys of Instructional Practice Protocol Please read the following statements carefully and thoughtfully consider your response to each statement. Remember, your responses do not indicate whether you like your teacher or not. They provide information about classroom instruction and environment. Now look at the computer screen. You should see the survey items and the name of the teacher for whom you are completing the survey followed by four possible answer choices Strongly Agree, Agree, Disagree, and Strongly Disagree. Please raise your hand if you do NOT see the survey and the answer choices. [Pause.] The choices indicate your level of agreement with each statement. Please raise your hand if you do not understand the answer choices. [Pause.] Repeat and clarify the meaning of each response as necessary. You will see the speaker next to the question. Please click on it to hear the question read to you. Please listen to the question and mark your answer. Raise your hand if you do NOT understand how to use the speaker to hear the question read to you. [Pause.] You also can read the questions in another language. Please click on the drop down menu on the right side of the screen to select a different language. Raise your hand if you do NOT understand how to choose a different language to read the questions. [Pause.] (Please note, the questions are only read aloud in English.) We are ready to begin. This survey is not timed so there is no need to rush. Please carefully consider each survey item and respond thoughtfully. Your input is important. Read Aloud Option: I will read each survey item twice. Please mark Strongly Agree, Agree, Disagree, or Strongly Disagree depending on how often this happens in your classroom. (Please read each item twice. Survey items are found on the next page.) Once you finish the survey, please click on the Save & Mark Complete button found at the bottom or top left side of the screen. You should see the message The End. The Survey is now over. We are finished with the survey. Thank you for your attention and your thoughtful responses. Please click Sign Out on the top right corner of the screen to exit the survey. September 15, 2014 Page 11 of 19
Grades 9-12 Surveys of Instructional Practice Georgia Department of Education Grades 9-12 Standard 3: Instructional Strategies 1. My teacher encourages me to be an active participant in class, rather than just sitting and listening. 2. My teacher uses a variety of activities and strategies to help me be interested in class. 3. My teacher frequently checks to see if I understand what is being taught. 4. My teacher takes time each day to make sure we summarize what we have learned. 5. My teacher asks me to use what I learn to solve problems or relate to real world topics. Grades 9-12 Standard 4: Differentiation 6. My teacher gives students as much individual attention as they need to be successful. 7. My teacher teaches in different ways to meet the needs of the students. 8. When I don t understand something, my teacher tries to figure out why I don t understand it. 9. The work my teacher gives me meets my academic needs. 10. My teacher gives me opportunities to use what I learned in creative ways. Grades 9-12 Standard 7: Positive Learning Environment 11. My teacher cares about my learning. 12. The instructions my teacher gives are clear. 13. My teacher ensures the rules and procedures are followed in class. 14. My teacher treats students with respect. 15. I feel comfortable asking my teacher questions. Grades 9-12 Standard 8:Academically Challenging Environment 16. My teacher makes productive use of class time. 17. The work assigned in class challenges me. 18. When confused by something, my teacher will not let me give up until I understand it. 19. Because of my teacher, I push myself to learn as much as I can. 20. My teacher encourages me to try new things, even when they are difficult. September 15, 2014 Page 12 of 19
Following Survey Administration Use of Survey Data Georgia Department of Education A building administrator must log on to the GaDOE TLE Electronic Platform and verify that the number of surveys completed for each teacher is consistent with the number of students intended to be surveyed. All surveys must be complete by March 31st. The Survey Administrator and any scheduled proctors should refrain from discussing any of the items and associated student responses. Surveys of Instructional Practice will provide documentation in the form of data that is required for Standards 3, 4, 7, and 8. This documentation may be used to inform formative evaluations and must be used to inform summative assessment ratings for those standards. Prior to the closing of the survey window on March 31 st, but as soon as the completion of 15 surveys for a particular teacher is complete, the results will be accessible in the teacher s evaluation plan on the GaDOE TLE Electronic Platform. If the TAPS rating on any of the standards for which survey data is provided differs significantly from the rating that would be indicated by those data, the evaluator is required to provide written justification within the GaDOE TLE Electronic Platform to explain why the performance rating on the standard is not aligned with the survey data. To gain valid survey results, a minimum of 15 students shall complete the survey for their teacher; student data cannot be disaggregated for groups smaller than 15 for confidentiality purposes to protect the anonymity of respondents. Teachers (i.e. collaborative gifted teacher), who cannot utilize the Surveys of Instructional Practice based on the procedures established for the students he or she teaches, will not have this type of data to supplement the documentation or inform the ratings on the TAPS performance Standards 3, 4, 7, and 8. Survey Results Survey results will be compiled within the GaDOE TLE Electronic Platform and must be utilized as documentation to support annual performance ratings. All surveys must be completed by March 31st. A summary of results for each item will be provided to individual teachers. Figure 18, The Survey Results Summary Sheet will include: The number of students with valid responses for each survey item. The percentage of rating for each response scale level for each survey item. (Yes, Sometimes, No for Grades 3-5; Strongly Agree, Agree, Disagree, Strongly Disagree for Grades 6-12). The teacher, school, district, and state mean, the median, and the standard deviation compared to all other teachers at that grade level band (3-5, 6-8, and 9-12) for each survey item. September 15, 2014 Page 13 of 19
Number of Valid Responses Strongly Agree (3 pts) Agree (2 pts) Disagree (1 pts) Strongly Disagree (0 pt) Mean School Mean District Mean State Mean Median Standard Deviation Georgia Department of Education Figure 18: Survey Results Summary Sheet (Sample for Grade 7 Teacher) Survey Results Summary Percentage of Ratings Item My teacher encourages me to participate in class, rather than just sitting and listening. My teacher encourages me to ask questions in class. 30 3% 50% 47% 0% 2.57 2.1 2.2 2.1 3 0.57 28 0% 25% 68% 7% 2.18 2.3 2.3 2.2 2 0.55 Evaluators and teachers will also be provided with a summary chart for each standard by mean score. Figure 19 shows a partial Survey Results table for each standard by mean. Figure 19: Survey Results for Each Standard by Mean 3. Instructional Strategies 4. Differentiated Instruction 7. Positive Learning Environment 8. Academically Challenging Environment 0.9 2.1 3.0 1.7 September 15, 2014 Page 14 of 19
Appendices September 15, 2014 Page 15 of 19
Checklists for Survey Administration Determine location and number of available computer stations for survey administration. Generate the unique survey key and passcodes for each class in the GaDOE TLE Electronic Platform as required for students to take survey. Schedule student groups according to the Survey Protocol. Select/Confirm proctor(s) to assist with preparation and administration of the surveys as necessary. Arrange the seating in the computer lab/media center or other appropriate space to provide students privacy, ensure anonymity, and promote honesty of responses as necessary. Ensure computers are functional and the survey is accessible at all stations. Access the survey on the electronic platform. Administer the survey to all students to which it is assigned. Verify survey completion for each class assigned via the GaDOE TLE Electronic Platform. September 15, 2014 Page 16 of 19
GADOE TLE Electronic Platform Survey Quick Reference Guide- School Administrator Logging In Log in to the TLE Electronic Platform by going to: https://tle.gadoe.org. Click the Administration tab, as seen below. Click the Anonymous Surveys link. Printing Student Logons Click on the tab, All Accounts. Fifty student logons were prepopulated based on the teacher s demographic. If the number of accounts or lock out date needs adjusting, please click on the one that needs updating and make the change. (Please note the Lockout Date is the day you can no longer complete surveys.) September 15, 2014 Page 17 of 19
To print the codes, click on the Teacher s Name, under the Assessed Person column. Once the lists of unique logins are generated, they can be exported or printed for distribution to each student to complete the survey. Survey codes can be printed for multiple teachers by clicking the box above the teacher s name to select all, scroll to the bottom and click View Accoutns. This will allow you to print the accounts for all the teachers selected. Creating Student Logons If a teacher needs survey codes generated, click on the tab, Select Accounts. You can select one teacher or multiple teachers to create the accounts. You can also add filters to narrow down the teachers listed. Once the teacher or teachers are selected, scroll to the bottom and click Create Accounts. Select the survey from the drop down menu, enter the lockout date, enter the number of accounts, and click create accounts. The accounts will be generated and can be printed by clicking on the tab All Accounts to print. September 15, 2014 Page 18 of 19
GaDOE TLE Electronic Platform Survey Quick Reference Guide- Student The student logs into the TLE Electronic Platform at https://tle.gadoe.org and clicks on Anonymous Survey Login and enters the Survey Key and passcode. For the 3-5 Survey, the Survey Administrator may log in the students so they are ready to begin with the survey. The student takes the survey. The student may choose to read the question in a different language by selecting it from the dropdown menu or hearing the question read aloud by clicking on the microphone below the question. Once the student completes the survey, the student must click Save & Mark Complete to record his/ her responses. If the student fails to complete all survey items, it will prompt the student to complete all the questions. Once the survey is complete, the student will receive the following notification on the screen. Click Sign Out to exit the survey. Do Not Disturb Sign September 15, 2014 Page 19 of 19