Department of Evaluation and Assessment

Transcription

1 Department of Evaluation and Assessment

2

3 Department of Evaluation and Assessment Additional At-a-Glance Report information may be located at:

4

5 Board of Trustees Mike Morath District 2: North and Near East Dallas Miguel Solis President District 8: Love Field, Northwest Dallas, and Central Dallas Lew Blackburn, Ph.D. First Vice President District 5: Oak Lawn, West Dallas, Wilmer, Hutchins, and portions of East Oak Cliff Elizabeth Jones Second Vice President District 1: Northwest Dallas Dan Micciche Secretary District 3: Northeast Dallas Nancy Bingham District 4: Southeast Dallas, Seagoville, Balch Springs Joyce Foreman District 6: Southwest Dallas Eric Cowan District 7: North Central Oak Cliff and portions of West Dallas Bernadette Nutall District 9: South Dallas and portions of Downtown Dallas, Pleasant Grove, Deep Ellum, Uptown, and East Dallas Mike Miles Superintendent of Schools The Dallas Independent School District, as an equal opportunity educational provider and employer, does not discriminate on the basis of race, color, religion, sex, national origin, age, disability, sexual orientation, gender identity, gender expression, genetic information, or any other basis prohibited by law in educational programs or activities that it operates or in employment decisions. The district is required by Title VI and Title VII of the Civil Rights Act of 1964, Title IX of the Education Amendments of 1972, Section 504 of the Rehabilitation Act of 1973, the Americans with Disabilities Act, and the Age Discrimination Act of 1975, as amended, as well as board policy not to discriminate in such a manner. (Not all prohibited bases apply to all programs.) If you suspect discrimination please contact: Mary McCants, Title VII or Title IX, at (972) ; Daphne LaMontagne, Section 504, at (972) ; Diedrae Bell-Hunter, Americans with Disabilities Act, at (972) ; or Employee Relations at (972) General questions about the district should be directed to Customer Service at (972) Ross Avenue Dallas, TX (972) /07/14

6

7 At-a-Glance Table of Contents Title Report Number Page ASSESSMENT - LOCAL... 1 Assessment of Course Performance (ACP): EA ASSESSMENT STATE AND NATIONAL... 7 State and National Assessments, EA National Assessment of Educational Progress (NAEP)/Trial Urban District Assessment (TUDA): EA OFFICE OF INSTITUTIONAL RESEARCH (OIR) The Office of Institutional Research: Classroom Effectiveness Indices Analysis and Applications... EA PROGRAM EVALUATIONS Postsecondary Outcomes of Dallas ISD Graduates, EA Evaluation of the Dual Language and ESL Programs: EA ACHIEVE3000: EA The Margaret and Gilbert Hernandez Intake Center: EA AVANCE Dallas Program: EA Reconnection Center Program: EA Title II, Part A Teacher and Principal Training and Recruiting: EA Youth and Family Centers Program: EA Fellows Academy Program: EA Texas Title I Priority Schools Grant: EA Evaluation of the Office of Family and Community Engagement Program EA African American Success Initiative: EA The Dallas ISD On-Track Indicator, EA The Summer Learning Program, EA The Summer Learning Lab Program, EA Title I: Comparative Measures of Student Achievement EA The Translation Services Program: EA The Neglected and Delinquent Children and Youth Program: EA i

8 At-a-Glance Table of Contents Title Report Number Page National Summer Learning Demonstration Project and Study: EA E&A Internal Service Fund: EA Reading Results for Kindergarten to Grade 2... EA Student Success Initiative: EA Evaluation of the Home Instruction for Parents of Preschool Youngsters (HIPPY) Program: EA McKinney-Vento Homeless Children and Youth Education Program: EA American Indian Education Program: EA Career Education and Workforce Partnerships, EA Adult Education and Workforce Literacy, EA Refugee School Impact Grant Program: EA State Compensatory Education Program: EA Evaluation of the Reasoning Mind Mathematics Program EA Special Education Program: EA College Access Program: EA Academic Facilitators Program: EA After-School Programs, EA Imagine 2020 Initiative: EA Parent Resource Specialists: EA ii

9 1

10 2

11 Assessment of Course Performance (ACP): Kathy Williams-Palmer, MSAE EA At-a-Glance For the school year, Local Assessment, in conjunction with the Department of Teaching and Learning (T&L), developed the Assessments of Course Performance (ACPs) as standardized final semester tests in grades kindergarten through twelve for Reading/Language Arts, Mathematics, Science, Social Studies, World Languages, English-as-a-Second-Language (ESL), Instructional Technology, Health, Physical Education and Visual & Performing Arts courses. Overall, 233 tests were developed and 12 were trans-adapted in Spanish. With seven single semester tests, a total of 252 ACPs were administered during the school year. ACP Development Prior to the test development phase, T&L provided outlines, called blueprints, detailing the Student Expectations (SEs) that required assessment items. Test blueprints were published on MyData Portal and Local Assessment websites at the beginning of each semester. In addition, T&L provided item blueprints, which directed item writers to measure specific aspects of SEs with special content. The processes of item writing, editing, item analyses, and test approvals were managed through the Office of Test Development. Tests comprised approximately four-fifths of the well-performing items from the previous year and one-fifth new or revised items. Item writing during the summer generally focused on expanding and revising test item banks. All tests were aligned to the Texas Essential Knowledge and Skills (TEKS) using the Curriculum Map developed by each content area. After test administration began, Curriculum Map alignment pages were also published on the MyData Portal and Local Assessment websites. These pages assured that each test item s SE was in the Curriculum Map. Valid and Reliable Tests The development of valid and reliable assessments is crucial for the evaluation of student knowledge and mastery of instructional material. Therefore, it is essential that the data provided by tests be valid and reliable. Content validity was demonstrated through the use of item and test blueprints, developed by T&L, and Curriculum Map alignment pages, ensuring each item was aligned with the curriculum. Cronbach s alpha, a coefficient used to provide a general indicator of test reliability, ranges from 0 to 1, with 0 indicating no test reliability. Coefficients approaching 1 indicate high test reliability. Of the 252 ACPs administered, 230 ACPs had a reliability of.70 or higher. Of the 22 ACPs whose reliability was less than.70, four tests had a reliability of less than.60. Kindergarten PE ACPs had too few items (10) for an appropriate reliability score; it was not calculated. Curriculum Alignment The ACPs were designed as measures of the taught curriculum. Although every SE in the curriculum was not measured, significant SEs were identified and included in the test design process. Teachers were surveyed and asked to indicate the extent to which the ACPs successfully matched what was taught in the course. The results are presented in Table 1. About a third of the teachers reported a 100% match. Over half of the teachers indicated that courses and ACPs had a 75% match. Given that there were more SEs taught than can be appropriately measured on a two-hour test, this level of matching was acceptable. Question Table 1 ACPs Aligned with Taught Curriculum and Student Performance Response Category Sem 1 % Sem 2 % To what extent does the ACP 100% match successfully assess (match) About 75% match what is taught in this course? About 50% match About 25% match No match at all Scaled Scores All ACP tests used scaled scores rather than raw scores. Tests aim to have 55% correct set to equal a scale score of 70, or passing. However, some tests have lower or higher passing percentages based on test difficulty. On 27 of 128 first semester tests and 48 of 124 second semester tests, the criterion was raised to reflect similar passing rates seen on their respective STAAR. The 55% criterion was lowered on 66 of 128 first semester tests and 48 of 124 second semester tests. Maintaining the 55% standard would have led to unreasonably low passing rates. Semester One Results The mean percent correct ranged from a low of 34.7% in Beginning Language Arts 7 ESL to a high of 91.7% in Grade 1 Spanish Reading, while the median percent correct was 55.4%. Mean scale scores ranged from 67.1 to 92.4, with a median scale score of Mean course grades were higher, ranging from 75.4 (Chemistry) to 95.9 (Grade 3 PE). 3

12 The percent of students passing the first-semester ACPs ranged from 37.8% (Grade 4 Social Studies and Physics) to 97.9% (Grade 1 Spanish Reading). {See Table 2} In contrast, the percent of students passing the first-semester courses ranged from 81.4% (Algebra I) to 100% (Grade 2 Music and Grade 3 PE). Semester Two Results The mean percent correct ranged from 37.2% to 92.7%, with a median percent correct score of 59.9%. Mean scale scores ranged from 67.8 in Grade 4 Spanish Social Studies to a high of 92.7 in Kindergarten Spanish Reading, while the median scale score was Mean course grades tended to be higher, ranging from 75.3 (Chemistry) to 96.0 (Grade 3 PE). The percent of students passing the second-semester ACPs ranged from 34.2% (Beginning Language Arts 6 for ESL) to 94.5% (Kindergarten PE). {See Table 2} In contrast, the percent of students passing the second-semester courses ranged from 78.4% (English IV) to 100% (Grade 2 Music and Grade 3 PE). Students in Algebra I and Biology, and their Pre- AP courses took the STAAR EOC in lieu of the ACP. Test Table 2 Percent of Students Passing First- and Second-Semester ACPs % Passing Sem 1 % Passing Sem 2 Kindergarten Mathematics Grade 1 Mathematics Grade 2 Mathematics Grade 3 Mathematics Grade 4 Mathematics Grade 5 Mathematics Mathematics Mathematics 6 Pre-AP Mathematics Mathematics 7 Pre-AP Mathematics Algebra I Algebra I Pre-AP Algebra II Algebra II Pre-AP Geometry Geometry Pre-AP Pre-Calculus Pre-Calculus Pre-AP Mathematical Models Kindergarten Reading Kindergarten Spanish Reading Grade 1 Reading Grade 1 Spanish Reading Grade 2 Reading Grade 2 Spanish Reading Grade 3 Reading Grade 3 Spanish Reading Grade 3 Language Arts Grade 3 Spanish Language Arts Grade 4 Reading Grade 4 Spanish Reading Grade 4 Language Arts Grade 4 Spanish Language Arts Table continues Test % Passing Sem 1 % Passing Sem 2 Grade 5 Reading Grade 5 Spanish Reading Grade 5 Language Arts Grade 5 Spanish Language Arts Reading Language Arts Language Arts 6 Pre-AP Reading Mastery Reading Language Arts Language Arts 7 Pre-AP Reading Mastery Reading Language Arts Language Arts 8 Pre-AP English I English I Pre-AP English II English II Pre-AP English III English IV Grade 3 Science Grade 3 Spanish Science Grade 4 Science Grade 4 Spanish Science Grade 5 Science Grade 5 Spanish Science Science Science 6 Pre-AP Science Science 7 Pre-AP Science 8 & Science 8 Pre-AP Integrated Physics and Chemistry Biology Biology Pre-AP Chemistry Chemistry Pre-AP Physics Physics Pre-AP Environmental Systems Grade 3 Social Studies Grade 3 Spanish Social Studies Grade 4 Social Studies Grade 4 Spanish Social Studies Grade 5 Social Studies Grade 5 Spanish Social Studies Social Studies Social Studies 6 Pre-AP Texas Studies Texas Studies 7 Pre-AP United States Studies United States Studies 8 Pre-AP U.S. History Government Economics World History World History Pre-AP World Geography World Geography Pre-AP Beginning Reading 6 for ESL Beginning Language Arts 6 for ESL Interm Reading 6 for ESL Interm Language Arts 6 for ESL Beginning Language Arts 7 for ESL Interm Language Arts 7 for ESL Beginning Language Arts 8 for ESL Interm Language Arts 8 for ESL Table continues 4

13 Test % Passing Sem 1 % Passing Sem 2 Reading II for ESL Reading III for ESL English I for ESL English II for ESL Technology Applications Computer Science I Pre-AP Web Design MS Health Education HS Health Education Grade 1 Art Grade 4 Art Art Art I Kindergarten PE Grade 3 PE PE PE 7& Grade 2 Music Grade 5 Music HS Band MS Band French I French II Spanish I Spanish II Spanish for Native Speakers I Spanish for Native Speakers II Course Grades Scale score results were used in the calculation of the final semester grade for grades three through 12. For kindergarten through grade two, as this was a pilot year for ACPs; ACP scores were not factored into final semester grades. Other ACPs that were pilot tests include Physical Education and Visual & Performing Arts ACPs. For grades three through five, ACP scale score results accounted for 5% of the final semester grade. Middle school results accounted for 10% of the final semester grade, while high school ACP scale score results accounted for 15% of the final semester grade. For more information, see EA , available at or contact Kathy Williams-Palmer at

14 6

15 7

16 8

17 State and National Assessments, Robert Mount, Ph.D., Karen Byers, M.S., Kiesha Pearson, M.Ed., Fred Washington, M.A., Jorge Aguirre, B.S., Rafael Fontalvo, B.A. & Jose Alonzo EA At-a-Glance State and National Assessments (SNA) coordinates the administration, distribution and processing of all state- and district-mandated criterion- and norm-referenced tests in the Dallas ISD. These tests included TAKS, STAAR, Texas English Language Proficiency Assessment System (TELPAS), ITBS/ Logramos, WMLS-R and Stanford 10. In addition, SNA supports Magnet School Testing, NAEP/TUDA, Credit by Examination (CBE), and Credit by Examination for Acceleration (CEFA). SNA is also responsible for test security, training of Campus Test Coordinators, accountability of locally developed tests and the reporting of testing irregularities to the Texas Education Agency (TEA) and the Department of Human Capital Management. State-Mandated Assessments The graduating class of was the last group of students required to meet State standards outlined for the Texas Assessment of Knowledge and Skills (TAKS). The State of Texas Assessments of Academic Readiness (STAAR ) replaced the TAKS at all assessment levels. In addition, House Bill 5 (HB 5) reduced the number of courses required for graduation from 12 to 5. These assessments include Algebra I, English I (combined reading/writing), English II (combined reading/writing), Biology, and U.S. history. Students must pass all five End-of- Course assessments to be eligible to graduate from a Texas public high school. The school year also saw the elimination of STAAR- M as an assessment measure for students with disabilities. The following presents an overview of state mandated testing; Across grades 1-12, a total of 2,977 students took the Texas Assessment of Knowledge and Skills (TAKS) Exit retest. At grades 3-8, a total of 69,979 students took STAAR Reading, 23,561 students took STAAR Writing (grades 4 & 7), 67,989 took STAAR Math, and 22,446 students took STAAR Science (grades 5 & 8) and 10,338 to STAAR Social Studies. A total of 73,400 students took STAAR/EOC with about 12,000 tested in each of the following subjects: Algebra I, Biology, English I, and English II. A total of 57,242 students were rated on the TELPAS. The majority of students rated Beginning was in grades K-3, while those rated Intermediate, Advanced, and Advanced High were in grades At grades K-2, 18,653 students were tested with the ITBS (22,204 in reading and 40,726 in mathematics), and 18,653 with the Logramos reading. Woodcock Munoz and Stanford 10 Assessments of LEP Students School districts are required to identify LEP students through the use of a home language survey. They are then required to assess those students with a Texas Education Agency (TEA) approved oral language proficiency test (Woodcock-Munoz Language Survey). Students in grade Pk-12 who were new to the district and spoke a language other than English were assessed with the WMLS and placed in an appropriate instructional program within four weeks of enrollment. Students who scored 4 on the WMLS were assessed with a TEA approved norm-referenced test (Stanford 10). A total of 40,389 students were tested with the WMLS (27,466 on the English version and 12,064 on the Spanish version) and 859 with the Stanford 10. The highest concentrations of student tested were in grades PK (12,946), K (4,861) and 5 (6,596). Magnet School Application Process and Assessment Results State and National Assessments provided testing with the ITBS, Stanford 10 or Logramos for students who did not have norm- or criterion- 9

18 referenced achievement test scores from the previous school year. Specifically, these services were provided to students who were applying for admission to the district s Academies (elementary), Vanguards (middle), and Magnet High Schools who met the following criteria: (1) currently enrolled in private, parochial, or other schools (grades K- 10), and (2) currently enrolled Dallas ISD students (grades K-2 and 7-10). For the school year, STAAR results replaced ReadiStep and other norm-referenced tests as part of the Magnet Application Process. Students with previous STAAR results were not allowed to take a norm-referenced test as a substitute for STAAR results. This change in admission criteria resulted in a large decrease in the number of students tested for admission into the Magnet Program. A total of 439 students were tested for admission into the Magnet Program in (234 with ITBS, 47 with Logramos and 158 with Stanford 10) compared to 15,308 students who were tested in (12,326 with ITBS, 2,493 with Logramos and 489 with Stanford 10). Credit by Examination for Acceleration and Credit by Examination with ACP Results Credit by Examination for Acceleration (CEFA) was developed by Texas Tech University to assess mastery of the Texas Essential Knowledge and Skills (TEKS) for students in grades It is a means by which highachieving elementary students may advance one grade level by passing all four tests in the common core areas (language arts, mathematics, science, and social studies), while middle and high school students may be awarded credit for a particular course(s). The Iowa Tests of Basic Skills (ITBS) was used to determine acceleration for students in kindergarten. Credit-by-Examination with Assessment of Course Performance (CBEwACP) is another means by which students entering the Dallas ISD from home-study or non-accredited schools may receive course credit(s). Students who entered the district in the second semester of grade 9 or in either semester (1 st or 2 nd ) of grades 10-12, were eligible to submit an application for CBEwACP. Students entering the district at the elementary or middle school levels were placed at the discretion of the principal and school personnel or through the use of other diagnostic measures. The following results were found for student who tested for acceleration and course credit: Four students were tested for acceleration at the Kindergarten level with 2 (50%) scoring at or above the 70 th percentile. At the elementary level, 11 students were tested for acceleration with 1 student achieving a mastery score of 80% or above on all four core subject areas (reading, math, science, and social studies). At the middle and high school levels, more than 1,500 students were tested for course credit (CEFA) with an 85% success rate at achieving course credit. A total of 438 students were tested for Credit-by-Examination with Assessment of Course Performance (CBEwACP) with 223 students (51%) receiving course credit. Test Security and Test Monitor Trainings The Department of State and National Assessments provided Campus Test Coordinator training on test security and test administration procedures. Counselors and Core classroom teachers/teachers who received Classroom Effectiveness Indices (CEIs) were not used as Campus Test Coordinators. In addition, a new directive suspended the use of counselors as Campus Test Coordinators. A total of 59 training sessions were held with 395 CTCs trained. To determine the quality of training and to get ideas about how to improve the training, CTCs were asked to complete an online survey at the end of the school year. A total of 172 campus staff responded to the survey, resulting in a 44% response rate. Figure 1 shows the percent of respondents by school level and Figure 2 shows the years of experience of Campus Test Coordinators responding to the survey. Seventy-eight percent (78%) of all respondents were from elementary schools, 11% from middle schools and 8% from high schools. Forty-two percent (42%) of the respondents indicated that they were new Campus Test Coordinators, while only 14% had 1-2 years of experience and 17% had 3-5 years experience. Survey results indicated that more than 90% of the CTCs felt that the trainer was knowledgeable, presented the information in an organized fashion, 10

19 emphasized how the information could be used in a school setting, and met the objectives of the training. Figure 1: Number and percent of Campus Test Coordinators responding to the survey by school level. High School 14 8% Middle School 19 11% Elementary School % Alternative School 3 2% Other 2 1% Figure 2: Number and percent of Campus Test Coordinators responding to the survey by years of experience. New Test Coordinator 73 42% 1-2 Years Experience 24 14% 3-5 Years Experience 29 17% More Than 5 Years Experience 43 25% Other 3 2% State and National Assessments also trained Test Monitors that went out to campuses to ensure proper test administration and test security procedures were being followed. At the end of each observation, test monitors were required to complete a number of observation forms documenting all aspects of the testing environment and testing irregularities observed. Test monitors were allowed to debrief or discuss any perceived test administration issue(s) with the Campus Test Coordinator or principal prior to leaving the school. All observation forms were submitted to the Office of Test Security for review. If a testing irregularity was observed, the test monitor had to also submit a Testing Irregularity/Security Violation Report Form to the Office of Test Security. The district s Test Security then filled a Testing Irregularity Incident Report with TEA immediately and scheduled a follow-up investigation to gather as much information as possible. A total of 220 test monitors were trained to monitor 221 schools 1 to 2 times during each testing window. School assignments were done through an online self-selection process. Testing Irregularities Testing irregularities fell into two distinct categories, Procedural and Serious. Procedural irregularities are less severe and most common. They typically involved minor deviations in testing procedures such as uncovered instructional displays or hall monitors not effectively monitoring bathroom breaks. Serious irregularities constituted severe violations of test security and/or confidentiality that may result in the involvement of the TEA s Educator Certification and Standards Division for consideration of disciplinary action (including suspension or revocation of educator certification credentials). If a Serious testing irregularity was identified, the district s Test Security Officer worked collaboratively with the Office of Professional Responsibility (OPR), Human Capital Management, principals and/or Division Executive Directors to conduct further investigations. A total of 265 procedural testing irregularities were reported to State and Nation Assessments. Of these, most (225) were Procedural Irregularities associated with the STAAR/EOC while the other 40 were associated with the TAKS (10), TELPAS (10), and ITBS/Logramos (20). Serious Testing irregularities were not cited in this report due to confidentiality issues. Only 1 Serious testing irregularity was forwarded to the Office of Professional Responsibility (OPR). Follow-up investigations by OPR and the Office of Test Security found the allegation of cheating associated with the Serious irregularity to be unfounded. However, OPR found that an ethics violation had occurred and took appropriate actions. Assessment of Course Performance (ACP) Accountability Process Assessments of Course Performance (ACPs) are locally developed tests that are used in grade K-12. These are secure test materials that are distributed and collected from all 11

20 schools in the district during a testing window. From year to year, about 60% of the items on the tests are recycled; therefore the retention of these tests by schools jeopardizes the integrity and security of the tests. To address this issue, a continuous effort was instituted to monitor the return of ACP test booklets to the Assessment Service Center. From the spring test sessions, a random sample of 15 elementary, 5 middle, and 5 high schools were selected from across the 5 Divisions for accountability purposes. A total of 86,184 test booklets were shipped to the various schools. Of those shipped, 44,169 were shipped to elementary schools, 20,105 were shipped to middle schools and 21,910 were shipped to high schools. More than 99% of all ACP test booklets shipped were returned to the Assessment Service Center (<1% retention rate). Across school levels, 5 elementary, 3 middle and 3 high schools returned all test booklets (100%) shipped to those schools. Of the schools that retained test booklets, a total of 23 test booklets were retained across all elementary schools with no more than 5 test booklets retained at any given school. At the middle school level, 165 test booklets were retained. Of those, 164 were retained at Florence Middle School and 1 at DESA. Follow-up phone calls and s failed to recover any test booklets from the schools. The retention rates fell below a ±1% accountability error for all school levels indicating that Campus Test Coordinators are doing a reasonably good job at accounting for ACP test booklets. However, failure to return a single secure test booklet constitutes a threat to the security of the ACP testing program. To decrease retention rates, it is recommended that Campus Test Coordinators do the following: 1. Sign for all test materials shipped to the school. 2. Sign and date the Delivery Ticket indicating that you, the Campus Test Coordinator, have counted and verified the quantity of materials received. 3. Return a copy to the signed and dated Delivery Ticket to the Assessment Service Center. 4. Send a follow-up to the Assessment Service Center as a backup indicating the quantity of materials received. 5. Enclose a copy of the signed and dated Delivery Ticket with ACP returns to show a 1-to-1 correspondence with the quantity of materials received and the quantity of materials returned. For more information see EA at 12

21 National Assessment of Educational Progress (NAEP)/Trial Urban District Assessment (TUDA): 2014 Frederick L. Washington, M.A., Robert Mount, Ph.D. EA N AEP is a congressionally mandated project of the National Center for Education Statistics. Located within the United States Department of Education and the Institute of Education Sciences, NAEP provides a comprehensive measure of students learning at critical junctures in their school experiences. It is the largest nationally representative and continuing assessment of what America's students know and can do in various subject areas. Assessments are conducted periodically in mathematics, reading, science, writing, the arts, civics, economics, geography, and U.S. history. In 2014, Technology and Engineering Literacy (TEL) were added to the subject areas assessed by NAEP. Since NAEP assessments are administered uniformly across the nation, NAEP results serve as the nation s report card and a common metric for all states and selected urban districts. This permits NAEP to provide a clear picture of students academic progress over time ( TUDA is designed to explore the feasibility of using NAEP to report on the performance of public school students at the district level. In accordance with federal law, NAEP administered the mathematics, reading, science, and writing assessments to samples of fourth and eighth grade public school students in urban districts across the country ( Eligibility Urban school districts had to meet the following criteria to participate in the TUDA project: 1. Must be located in large cities having a population of 250,000 or more. 2. Must be large enough to support a three-subject assessment cycle for NAEP in grade levels included in the state assessment program. 3. Must have a majority (50 percent or more) of students meeting at least one of the following criteria: (a) Either African American or Hispanic (b) Eligible for participation in the free and reducedlunch program (or other appropriate indicators of poverty status). At- a-glance According to the U.S. Census Bureau, Dallas, Texas had more than one million residents in During the school year, approximately 147,588 students were enrolled in the Dallas ISD. Of these, 69% were Hispanic and 89% were eligible to participate in the free or reduced lunch program. Based on the city s population and the district s student characteristics, Dallas ISD was invited to participate in the TUDA project for the first time. Urban Districts Participating in TUDA In 2011, the National Assessment Governing Board invited 21 urban school districts to participate in TUDA project. The same 21 schools participated in the project for the school-year (See Table 1). Table 1: Urban Districts Participating in the 2013 TUDA Project Urban Districts Albuquerque Public Schools Atlanta Public Schools Austin ISD Baltimore City Public Schools Boston Public Schools Charlotte-Mecklenburg Schools Chicago Public Schools Cleveland Metropolitan School District Dallas ISD Detroit Public Schools District of Columbia Public Schools Fresno Unified School District Hillsborough County Public Schools Houston ISD Jefferson County Public Schools Los Angeles Unified School District Miami-Dade County Public Schools Milwaukee Public Schools New York City Department of Education School District of Philadelphia San Diego Unified School District School/Student Selection and Participation A random sample of schools was selected from each participating urban district. These schools were thought to be representative of all schools in the district. From each sampled school, a random sample of students was selected for the TUDA assessment. These students represent an extension of the sample of students who would have been selected by NAEP as part of their state and national sample. Extended samples allow reliable reporting of student group results within districts. 13

22 Students at participating TUDA schools took the same NAEP assessments as those participating in the main NAEP assessment program. Results for students in the TUDA samples were included in state and national samples with appropriate weighting. As with the main NAEP assessments, TUDA does not provide individual scores for students or schools. The National Assessment Governing Board (NAGB) set guidelines to attain comparable inclusion rates across states and districts. The goal was to ensure inclusion of 95% of all students, 85% of Students with Disabilities (SDs) and 85% of English Language Learners (ELLs). NAGB also offered a decision tree for the exclusion of SDs and ELLs because any modification to an assessment changes the standard administration and alters the construct being measured. TUDA and the Dallas ISD During the school year, the Dallas ISD participated in the TUDA project for the second time. A sample of 92 Dallas ISD schools was selected to participate. Of these, 53 were elementary, 36 were middle, and 3 were high schools. Student characteristics at these schools were considered to be representative of all students in the district. The NAEP State Coordinator served as a liaison between the Texas Education Agency (TEA) and the Dallas ISD, while the Dallas ISD TUDA Coordinator served as the liaison between the district and the NAEP State Service Center (NSSC). The Dallas ISD TUDA Coordinator and the Department of State and National Assessments provided TUDA Campus Testing Coordinators (CTC) training designed to increase their knowledge of TUDA and the policies outlined by NAEP. During the pre-assessment visit, NAEP field staff collaborated with CTCs to determine the inclusion/exclusion of SD and ELL students. During the assessment visit, NAEP field staff was responsible for all assessment activities, including paper, pencils, dictionaries, calculators, etc. Campus Test Coordinators provided a work area for NAEP field staff, secured rooms for testing, and ensured that students were in their designated location at the appropriate time. Results As a district, Dallas ISD had a 100% participation rate (all schools). About 2,800 4 th and 2,800 8 th graders from 92 Dallas ISD schools (3 high schools, 36 middle, and 53 elementary) participated in the project. Inclusion and exclusion rates by demographic category are presented in Table 2 & 3, respectively. At grade 4, Dallas ISD had a satisfactory inclusion rate in math for ELLs (98%) but did not meet the guideline in reading (70%). SD inclusion rates failed to meet guidelines in both math (77%) and reading (60%). Exclusion rates in reading failed to meet guidelines for both ELLs (30%) and SDs (40%). Math exclusion rates failed to meet guidelines for SDs (23%) but met guidelines for ELLs (2%). At grade 8, Dallas ELL inclusion rates met guidelines in math (95%) and reading (92%), while SD inclusion rates failed to meet guidelines in either math (77%) or reading (74%). Reading exclusion rates failed to meet guidelines for SDs (26%), but met guidelines for ELLs (8%). Math exclusion rates failed to meet guidelines for SDs (23%), but met guidelines for ELLs (5%). Table 2: Inclusion Rates (%) by Demographic Category and Grade Grade 4 Grade 8 Category Readin Math Readin Math g g ELL 70** 98* 92* 95* SD 60** 77** 74** 77** Note. * indicates met guidelines and ** indicates failed to meet guidelines. Table 3: Exclusion Rates (%) by Demographic Category and Grade Grade 4 Grade 8 Category Readin Math Readin Math g g ELL 30** 2* 8* 5* SD 40** 23** 26** 23** Note. * indicates met guidelines and ** indicates failed to meet guidelines. In Figures 1-4, Dallas ISD s math and reading scores for grades 4 and 8 are compared to the nation, large cities, and the state of Texas by student groups. Figure 1: Comparison of NAEP Grade 4 Math Performance by Jurisdiction and Student Group, 2011 and

23 Dallas ISD grade 4 students 2013 math average scale score increased by one point from 2011 to 2013 for Black or African American and Hispanic students. This increase was similar to gains observed in Texas and the nation for all students, but lower than that observed for large cities. Dallas ISD NSLP and ELL grade 4 students showed an increase of two to three points, respectively, from 2011 to 2013 which were better than or equal to all students in the nation, large cities and Texas. Dallas ISD students overall, and in all student subgroups, showed gains in their math average scale score from 2011 to Dallas ISD did not have enough White students in the 2013 sample to be included in the analyses. Figure 2: Comparison of NAEP Grade 4 Reading Performance by Jurisdiction and Student Group, 2011 and 2013 Dallas ISD ELL, Hispanic, and NSLP grade 4 students showed gains in their average scale score from 2011 to Dallas ISD White, Black or African American grade 4 students 2013 performance was slightly below that posted in Figure 3: Comparison of NAEP Grade 8 Math Performance by Jurisdiction and Student Group, 2011 and 2013 Dallas ISD grade 4 students 2013 reading average scale score increased by one point from 2011 to 2013, which was similar to all students in large cities and the nation but higher than Texas. Dallas ISD Hispanic, ELL, and NSLP grade 4 students 2013 reading average scale score increased from 2011 to 2013, which was better the all students in the nation, large cities, and Texas. Dallas ISD White and Black or African American grade 4 students 2013 reading average scale score decreased from 2011 to Dallas ISD grade 4 students showed a one point gain in their reading average scale score from 2011 to 2013 for all students. Dallas ISD grade 8 Hispanic and ELL students 2013 math average scale scores increased from 2011 to 2013, which was similar to large cities and the nation. Dallas ISD White, Black or African American grade 8 students 2013 average scale score decreased from 2011 to However, in Texas, all students average scale score decreased from 2011 to Dallas ISD grade 8 students overall, and in all student subgroups, performed relatively the same from 2011 to Figure 4: Comparison of NAEP Grade 8 Reading Performance by Jurisdiction and Student Group, 2011 and

24 Dallas ISD grade 8 students 2013 reading average scale score increased by three points from 2011 to 2013, which was similar to or better than large cities, Texas and the nation. Dallas Black or African American grade 8 students reading average scale score held steady from 2011 to Overall, Dallas ISD grade 8 students showed an increase in their 2013 reading average scale score. Dallas ISD ELL, Hispanic, and NSLP grade 8 students showed an increase in their reading average scale score from 2011 to 2013, while Black or African American students held steady. Dallas ISD did not have enough White students in the sample to be included in the analyses. For more information, contact State and National Assessments at or see 16

25 17

26 18

27 The Office of Institutional Research: Classroom Effectiveness Indices Analysis and Applications Namrita Chawla EA At-a-Glance A Classroom Effectiveness Index, or CEI, is a valueadded measure of the amount of academic progress that a teacher afforded his or her students after a year of instruction. CEIs evaluate a student s performance on select summative, standardized tests by comparing his or her performance to that of similar students in the district. Similar students have the same demographic characteristics (i.e. gender, limited English proficient (LEP) status, special education status, talented and gifted status, socioeconomic status, and three neighborhood characteristics) and the same level of achievement on the same p rior-year tests. An important feature of the CEIs is the use of a comparison group to measure relative progress. This measure of performance is in contrast to a system, such as the Academic Excellence Indicator System (AEIS), which evaluates students absolute progress against a pre-determined score criterion. CEIs are computed at three levels. A teacher receives a CEI for each section, course, and division (or content area) he or she teaches. Reports containing all CEIs, as well as student-level information, are provided annually via MyData Portal to teachers, principals, and executive directors as a tool to guide instructional and professional development planning. Background During the school year, 5,661 teachers throughout the district taught CEI-eligible courses. Among those eligible, 5,128 teachers received CEIs during initial production. After the publication of CEI reports in September 2013, teachers had the opportunity to formally request investigations into their CEIs. Investigations were based on data quality issues including demographic, attendance, and assessment results. The adjustment request process was available for specific, documented adjustment requests. While the adjustment request process is transitioning away from post-production corrections to student rosters, requests to include or exclude students were accepted if submitted with explanation and the principal s approval CEI Adjustment Process Teachers used a standard form to request adjustments to their CEIs, which involved the verification of data used to compute individual CEIs, and where appropriate, the recalculation of CEIs. The Office of Institutional Research (OIR) staff reviewed each Table 1. CEI Teachers* and CEI Recipients by School Type School Type CEI-eligible Teachers CEI Teachers N % N % Elementary School** 3, , Middle School 1, , High School 1, , Total 5, , *Data from the student information system describing teacher course assignments were used to compute teacher counts. **Prekindergarten and kindergarten teachers do not receive CEIs. request by examining appropriate information sources, such as the district s student information system, preliminary CEI rosters maintained in OIR databases, CEI training documents, and databases used in the calculation of CEIs and School Effectiveness Indices, or SEIs. CEIs were recalculated when data and roster discrepancies were confirmed. Additionally, an updated report containing the new or adjusted CEI was posted on MyData Portal. Teachers who submitted adjustment requests received memoranda explaining the outcome of the investigations. Principals were notified when teachers from their schools received any correspondence regarding their adjustment requests. During the CEI adjustment period for CEIs, 18 traditional adjustment requests were submitted by teachers from 14 schools throughout the district. There were three main types of issues presented in the CEI adjustment requests: 1) Roster Corrections related to continuous enrollment: Submitted by teachers who felt that their report did not accurately reflect the students they taught during the school year. This type of request generally came from teachers who did not correct or improperly corrected their rosters during the spring CRV process. Additionally, there was a discrepancy between continuous enrollment status for some students during the CRV process and the CEI reports. This discrepancy 19

28 was due to the fact that rosters were created before the end of the school year and did not use test-specific attendance requirements. 2) Policy Issues: Submitted by teachers who did not understand the policy-related requirements for CEI-eligibility such as the appropriate combination of tests, retained students, and/or special education students. 3) Calculation of CEIs: Submitted by teachers who questioned the validity of the methods and data used to compute CEIs. Figure 1 Adjustment requests are differentiated by type based on the major issue described by the requester Eight CEIs were adjusted during the CEI. adjustment process. All of the adjustments involved linking (or adding) students to the teacher s course. In seven of the cases, the students appeared to be continuously enrolled during the CRV Verification Process in spring, but were not included in the CEI reports once the final continuous enrollment requirements were applied. In one of the cases, the teacher updated her rosters during the spring CRV period but did not submit them. Of the 18 adjustment requests submitted, ten (55.5 percent) were not adjusted. Teachers who did not receive an adjustment were provided memoranda explaining why an adjustment was not necessary or appropriate. Principals received copies of the memoranda sent to teachers currently employed at their campuses. There were three predominant issues addressed in the memoranda sent to teachers: 1. Red Bar/Green Bar: Provided guidance on the interpretation of students relative gain scores in relation to their passing rates. It was reiterated that the CEI is a measure of students relative gain, not passing rate. In order for a student to have high relative gain scores, he must outperform the average score of his comparison group, whether that score is a passing value or a failing value. 2. Attendance Requirements: Addressed concerns regarding the inclusion of students with excessive absences in the computation of CEIs. Teachers were reminded of the maximum number of days a student can be absent from a teacher s course and still be included in the computation of the teacher s CEI. They were also reminded that the CEIs only take into account excused and unexcused absences. Students who were included in the teacher s CEI were listed in this memorandum. 3. Continuous Enrollment: Explained the criteria for a student to be considered continuously enrolled in a year-long course. The students who were excluded from the teacher s CEI as a result of not being continuously enrolled were listed in this memorandum. Recommendations CEI Adjustment Process The CEI adjustment process should continue to provide an easily accessible and fair platform for teachers to dispute their CEIs. To accomplish this goal, it is recommended that OIR continues to refine the adjustment process to increase its overall efficiency by: Working with appropriate district leaders and departments to streamline the dispute process related to the all of the metrics in the TEI, including CEIs. Implementing a CEI question/answer period before accepting formal adjustment requests. Many of the adjustment requests OIR receives are questions about the process or calculations and could be addressed without a formal adjustment request. Encouraging digital submission of questions and adjustment requests to better track submissions Sending districtwide reminders to reduce the number of late submissions Collaborating with appropriate district leaders and departments to determine the continued suitability and usefulness of certain CEIrelated policies Refining the CEI rosters so that teachers are more easily able to view their student s continuous enrollment status. 20

29 21

30 22

31 Postsecondary Outcomes of Dallas ISD Graduates, L. Shane Hall, Ph. D. EA At-a-Glance Dallas ISD has tracked the college enrollment and completion status of its graduates with data from the National Student Clearinghouse (NSC), based in Herndon, Virginia, since NSC collects college enrollment and completion information from more than 3,000 institutions of higher education across the country and has cooperative agreements with the majority of public and private two-year and four-year colleges and universities. Although schools allow students to opt out of sharing their enrollment records, few actually do so. Methodology NSC has provided data to Dallas ISD for all students reported as high school graduates for the school years from through NSC provides data and reports twice a year. November 2013 reports provided preliminary college enrollment information on the Dallas ISD graduating class of A small number of colleges and universities do not provide information to the NSC; however, at present, no in-state institutions enrolling significant numbers of Dallas ISD graduates are nonparticipants. Institutions that enroll smaller numbers of graduates vary their participation from year to year, meaning that data reported may differ from prior-year reports. Postsecondary Enrollment The NSC reports three enrollment statistics for each high school graduating class: students enrolled in the fall immediately following graduation (also known as seamless enrollers), students enrolled in the first year following graduation, and students enrolled in the two years following graduation. Students who enroll in college one or two years after completing high school are known as delayed enrollers. Seamless and delayed enrollment rates are shown in Figures 1-3 by institutional level (two-year or four-year colleges). As shown in Figure 1, slightly less than half (49%) of the class of 2013 enrolled in college during the fall immediately after high school graduation, a small decline from 52 percent for the class of More 2013 graduates enrolled in two-year colleges than in four-year colleges, which is consistent with prior Dallas ISD graduating classes. Compared with the class of 2012, two-year college enrollment rates remained the same (27% of graduates from both years), while four-year college enrollment declined three percentage points. 100% 80% 60% 40% 22% 32% 28% 26% 20% 27% 22% 25% 27% 26% 0% 24% 25% % Year 2-Year Figure 1. Students enrolled in college in the fall immediately following graduation, by high school graduating class. As shown in Figure 2, the percentage of students from the class of 2012 who enrolled in college reached 59 percent, up from 52 percent for that year s seamless enrollers (Figure 1). The growth in enrollment occurred mainly among students who enrolled in two-year colleges. 100% 80% 60% 40% 27% 33% 33% 20% 22% 26% 0% 27% Year 25% 25% Year 32% 34% 2012 Figure 2. Students enrolled in college within the first year after graduation, by high school graduating class. Meanwhile, college enrollments within two years after high school graduation reached 63 percent among members of the class of 2011 (see Figure 3). Students enrolling in two-year colleges accounted for 37 percent, compared to 26 percent for students who 23

32 enrolled in four-year colleges and universities within two years after graduation from high school. 100% 80% 60% 40% 29% 36% 38% 20% 26% 24% 0% 27% % 28% % 26% 2011 El Centro (DCCCD) Mountain View (DCCCD) Eastfield (DCCCD) Brookhaven (DCCCD) Richland (DCCCD) Univ. of Texas - Arlington Univ. of N. Texas University of Texas Texas Woman's University Cedar Valley (DCCCD) Prairie View A&M Texas A&M University Year 2-Year Figure 3. Students enrolled in college within the first two years after graduation, by high school graduating class. NSC data indicate that the majority of Dallas ISD graduates who go on to college enroll in public institutions located in the state of Texas. Among the class of 2013, for example, more than 90 percent of the students who entered higher education enrolled in a public, in-state institution. Fewer than 10 percent enroll in private or out-of-state institutions of higher education. Further, most Dallas ISD graduates stay close to home when enrolling in college. As shown in Figure 4, campuses in the Dallas County Community College District (DCCCD) were among the institutions enrolling the largest numbers of Dallas ISD graduates for the graduating classes of Meanwhile, among four-year universities, the University of Texas at Arlington enrolled the most Dallas ISD graduates, followed by the University of North Texas, Texas Woman s University, and the University of Texas in Austin. Figure 4. Most common institutions of enrollment in the fall following high school graduation for all classes, Persistence in College Another significant measure of college success is the percentage of students who enroll in college for a second year, following their initial enrollment after high school. NSC uses the percentage of students who enroll in college for a second year as a measure of persistence in college. Persistence in college has become a national concern because large numbers of high school graduates drop out of college without completing a degree or certificate. Research shows that many of these students are initially required to enroll in remedial courses and become discouraged when they spend one or more semesters taking (and paying for) courses that are not credit-bearing; others may be unable to continue in college for financial reasons or because of family obligations. Figure 5 demonstrates college persistence rates, measured by the percentage of high school graduates who returned for a second year of college following their initial enrollment after high school. Although persistence rates for students in two-year colleges increased over the previous year, from 66 percent to 70 percent, students in four-year colleges have much higher persistence rates, averaging 88 percent between 2007 and Overall persistence rates (two- and four-year colleges combined) have averaged 78 percent between 2007 and

33 Figure 5. First- to second-year college persistence rates, Dallas ISD classes of College Graduation In addition to college enrollment and persistence, NSC reports six-year college graduation rates, or the percentage of students who complete a college degree within six years of graduating from high school. Completion includes all levels of college degrees, from associate s degrees through postgraduate degrees. Figure 6 reports college enrollment and graduation rates for the Dallas ISD classes of 2005, 2006, and Overall college enrollment rates increased from 53.4 percent for the class of 2005 to 55.0 percent for the class of In addition, graduation rates have shown steady increases across the three graduating classes, from 16.2 percent in 2005 to 19.2 percent for the class of The 2007 rate shows an increase over college completion rates for previous years. Previous Dallas ISD research found a college completion rate of about 15 percent between 1998 and 2004 (Hall and Johnson, 2011) % 80.0% 60.0% 40.0% 20.0% 0.0% 100% 80% 60% 40% 20% 0% All Colleges 77% 80% 80% 76% 77% 4-Year 88% 90% 88% 88% 87% 2-Year 67% 72% 74% 66% 70% 53.4% 53.7% 55.0% 2005 (N=6731) 16.2% 18.6% 19.2% 2006 (N=6302) 2007 (N=6341) Enrolled in College Graduated Figure 6. Six-year college graduation rates for Dallas ISD classes of 2005 and Although NSC reports do not include the number of degrees awarded, the organization does provide student-level data on Dallas ISD graduates, including whether a student graduated from college. Table 1 displays the top 15 colleges by the number of degrees and certificates awarded to Dallas ISD graduates who completed high school between 2007 and Types of college degrees included in this count range from one-year certificates in various occupational fields to undergraduate and graduate degrees. As shown in Table 1, these schools included four-year universities in Texas and campuses in the Dallas County Community College District. Three universities the University of North Texas, Texas Woman s University, and the University of Texas at Austin, accounted for 20 percent of the total degrees awarded. Overall, the 15 schools listed in this table accounted for nearly 57 percent of the degrees and certificates awarded to Dallas ISD graduates. Table 1 Colleges Granting the Most Degrees to Dallas ISD Graduates, Degrees granted College or University N % University of North Texas Texas Woman s University University of Texas, Austin University of Texas, Arlington Everest College, Dallas Mountain View College, DCCCD Eastfield College, DCCCD El Centro College, DCCCD Prairie View A&M University University of Texas, Dallas Texas A&M University Southern Methodist University Texas A&M University, Commerce Stephen F. Austin State University Cedar Valley College, DCCCD Next Steps Total 3, As shown in this report, more than half of Dallas ISD high school graduates enroll in college sometime after high school. NSC does not provide data on Dallas ISD graduates who do not enroll in college. The district is working with the Texas Workforce Commission on an agreement to access that agency s data, to better understand the occupations and careers pursued by Dallas ISD graduates, regardless of whether they went to college. For more information, contact Shane Hall at

34 26

35 EVALUATION OF THE DUAL LANGUAGE AND ESL PROGRAMS: Rosemary Garcia-Rincón EA At-a-Glance The Multi-Language Enrichment Program (M-LEP), in accordance with federal, state and local policies and mandates, provided Bilingual Education and English-as-a-Second Language (ESL) programs in grades prekindergarten through 12 to meet the affective, linguistic and academic needs of English Language Learners (ELLs). The goal of the M-LEP is to build a strong instructional program for all ELLs to prepare students to be college and workforce ready. Students Served In , 64,711 (41%) of the total Dallas ISD student population in grades prekindergarten through 12 were identified as English Language Learners. The majority of identified ELLs were in grades prekindergarten through five. 0ver 50 percent of ELLs were in grades prekindergarten, three, four, and five. Grades six (47%) and nine (25%) had the largest enrollments of identified ELLs among secondary school grades. Programs The program models in elementary schools included Dual Language (DL) one- and two-way, Newcomer for recent immigrant students and content-based ESL. Program models implemented in secondary schools included ESL, Sheltered content courses and English Language Institute (ELI) for recent immigrant students. In , the DL one-way instructional model was implemented in 155 elementary schools and the two-way model in 19 elementary schools. The one-way model consisted of one language group (Spanish speaking only) who received instruction in two languages (English and Spanish). The two-way model consisted of two language groups (Spanish and English) who received instruction through two languages (English and Spanish). Students must achieve proficiency (perform at or above grade level in both languages) in reading, writing and mathematics, by the end of grade five. Of the total identified ELLs, 68 percent were served in the DL program (PK-5); five percent were served in ESL (grades PK-12); and 26 percent were served in sheltered content courses (grades 6-12). Woodcock Muñoz Language Survey-Revised (WMLS-R) Spring 2014 WMLS-R results were used to determine the English acquisition level at the end of the school year for grades five through 12. The WMLS-R broad ability levels data for all grades (5-12) showed that 36.6% of ELLs performed at the English proficient level and met one of the criteria for reclassification from ELL to non-ell status. (Figure 1). The data also showed that most ELLs performed at the broad ability level three (limited English proficient) for four continuous years. Figure 1: Comparison of WMLS-R broad ability levels for all grades (5-12) for Broad Ability Percent In general, fifth grade ELLs who performed at the English proficient levels in two-way instruction outperformed ELLs in one-way and ESL instruction. ELLs in all secondary grades in sheltered instruction outperformed ELLs in ESL instruction. WMLS-R: Spanish Of the total number of identified opted-in non-ell students (1,353) in grades pre-kindergarten through five, 699 students were tested with the Spanish WMLS-R in the fall of 2013 and 113 in the spring of Fall assessment data showed that a higher percent of Hispanic non-ell students (42.7%) in grades pre-kindergarten and kindergarten (51.3% and 73.6%, respectively) were Spanish proficient. Spring 2014 WMLS-R data showed that almost 71 percent of all tested non-ell students (PK-5) performed at the Spanish proficient level. TELPAS In August 2013, new standards were set for grades two through 12 on TELPAS Reading to meet the increased rigor of the STAAR. Consequently, the new TELPAS reading proficiency level standards were higher in comparison to the original standards and the percentage of students attaining the advanced high level were lower than previous years. TELPAS was administered to all ELLs in grades kindergarten through 12. Fifth grade ELLs with Advanced High (AH) ratings outperformed ELLs in all other grades while eighth and eleventh grade ELLs outperformed all other ELLs at middle and high school levels. Yearly progress indicated that 59 percent of the 27

36 students in grades one through12 had a composite rating lower or the same as the previous year. Grade five ELLs showed the greatest growth with 55 percent having a rating one or more levels higher than the previous year. The data showed that a higher percentage of ELLs in ESL instruction performed at the AH composite rating with the exception of grade three, which had a higher percentage of ELLs in two-way DL instruction. At the secondary school level, a higher percentage of ELLs in general education (program denials) classes performed at the AH composite rating than ELLs in other instructional models. ITBS and Logramos Overall, the data showed increases in student performance on the Logramos Vocabulary and Reading Comprehension tests for ELLs in the DL programs in comparison with last year s data. ELLs who received two-way instruction in grades kindergarten through second outperformed ELLs in the one-way instructional model for all grades. ELLs in first grade who received two-way instruction performed better than ELLs in other instructional models on the ITBS Mathematics Total. At grade two, ESL and DL students performed at similar levels. Comparisons with the previous two year s data showed that ELLs performed at a higher level in kindergarten and grade one in two-way. In addition, data showed that non- ELLs outperformed ELLs for all grades. STAAR Non-ELL students did better this year than last year on 24 percent of the STAAR tests administered while ELLs increased on 29 percent of the tests. Exited ELLs outperformed both non-ell and ELL students in all subjects except grade eight in mathematics. ELLs had a higher percentage at the Satisfactory level than non-ells on fifth grade reading and grades three through six and grade eight mathematics. Seventh grade reading, writing and mathematics were lowest for ELLs. Science was lowest for ELLs at grade eight and social studies decreased by 13 percentage points from the previous year. Figures 2 and 3 showed two year STAAR outcome comparison data for reading and mathematics. Figure 2: STAAR Reading: ELLs Percent Satisfactory Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Total Difference Figure 3: STAAR Mathematics: ELLs Percent Satisfactory Grade 3 Grade 4 Grade 5 Grade 6 ELL students did better on all tests in Spanish, with the exception of science, than those ELLs tested in English. Grade five students tested on STAAR Reading in Spanish (83.2%) outperformed students at other grade levels and those ELLs tested in English. In general, grades three, four and five showed higher percentages of Satisfactory for DL programs than the general education program. Students who opted-in one- and two way programs outperformed students in all other programs including general education. ELL performance in reading at grades six, seven and eight was less than non-ell students while sheltered student performance was higher in grades six and eight in mathematics. Exited ELL students outperformed non-ell, ELL, district and state students on STAAR End-of-Course (EOC) tests. The one exception was Algebra l, where state students performed six percentage points above exited ELL students. Non-ELL students showed higher percentage rates on all EOC tests than ELL students. Recommendations Grade 7 Grade 8 Total Difference Based on the findings from the evaluation, the following recommendations should be considered: Train parents whose children receive two- way instruction to become DL program advocates for recruitment purposes. Conduct campus audits in to monitor for the fidelity of program implementation. Ensure that WMLS test administrators are properly trained and proper testing and scoring procedures are adhered to throughout the school year. Place more emphasis on testing all opted-in non-ells with the Spanish WMLS-R in the spring to determine progression of Spanish language proficiency. Ensure the accuracy of identifying potential ELLs within limited English categories and programs and the proper placement in the corresponding programs. Ensure that students are tested with the appropriate assessments and in the appropriate language. For more information, see EA at 28

37 ACHIEVE3000: Rosemary Garcia-Rincón EA At-a-Glance A CHIEVE3000 is an online software program that provides differentiated online instruction to improve reading and writing proficiency. The Multi-Language Enrichment Program (M-LEP) purchased the Achieve3000 in English and Spanish for the school year as a supplemental resource to address the reading and writing needs of English as a Second Language (ESL) students in the Dallas ISD. This was in response to the low level performance of ESL students on the reading and/or writing STAAR or EOC in middle and high school. The software was also made available to ESL students in grades 3-5. Approximately 9,000 software licenses were purchased for a cost of $495, This software program had been used previously at Comstock Middle school with seventh and eighth graders and showed positive results according to an ACHIEVE3000 report. 1 ACHIEVE3000 s differentiated online instruction in reading and writing gives students the capacity to improve their performance in these and other content areas. Three software programs were used in the district: KidBiz3000 (grades 3-5), TeenBiz3000 (grades 6-8), and Empower3000 Differentiated Literacy Solution (grades 9-12). All students received the same topics and lessons, while they were taught one-on-one at their individual levels. The program delivers differentiated assignments at 12 different reading levels, along with formative assessments linked to state and Common Core standards. The program uses a five step literacy routine that helps students: acquire knowledge from informational texts, develop strong content knowledge, use higher-order thinking skills, make reading strategies automatic, understand that information influences opinions and must be validated with evidence (grades 3-5), argue effectively with supporting evidence (grades 8-12), and communicate effectively when writing and speaking (grades 8-12). Students reading levels are continually assessed and immediate results are provided for possible intervention. By the time students finish high school they need a Lexile level 2 of 1300 in order to be successful in college and beyond. The Achieve3000 s online literacy solutions uses the Lexile Framework for reading to measure students reading ability and to track reading progress throughout the year. Outcomes Program software licenses were available for 9,000 English language learners (ELLs) in grades 3 through 8 who did not pass the STAAR Reading and/or Writing or English EOC in high school. Each campus had the option of providing the program during, after, and/or Saturday school and was also accessible from home. The program was to have started in March 2014; however, due to delays, it officially began May 2014 and terminates in December 2014 when the software licenses expire. A total of 432 elementary, 5,823 middle and 3,535 high school level ESL students were identified who met the eligibility criteria. Enrollment data, however was not available to report. The program started out with problems due to low rate of teacher training participation and student use. In addition, there were questions as how to implement the program where only a portion of the classes had licenses. Reading and writing outcome data will not be available until STAAR and EOC data become available in June As a result of the issues confronted during , there are plans to change the student eligibility selection process for Recommendations The ACHIEVE3000 program was implemented for the first time late in the second semester (May) of the school year. The information presented in this report is only to provide an overview of the ACHIEVE3000 program. Other than to continue the program through December 2014 according to contract specifications, there are no specific recommendations at this time. However, before committing to longer term use of the program, data analysis should be conducted to determine if students who used the program showed improvement in test scores. For more information, contact Program Evaluation at Source ACHIEVE3000 End of Year Implementation Highlights Comstock Middle School, May An educational tool that uses a measure called a Lexile to match readers of all ages with books, articles and other appropriate leveled reading resources. 29

38 30

39 THE MARGARET AND GILBERT HERNANDEZ INTAKE CENTER: Rosemary Garcia-Rincón EA At-a-Glance T he Margaret and Gilbert Hernandez Student Intake Center was opened on July 20, This was in response to a need reported by campus administrators to re-open a central intake center for newly arrived immigrants and refugees that were never enrolled in a school in the United States. The center s staff provides support to families and students to begin the path toward academic success. The main purpose of the intake center was student registration, orientation and assessment of all immigrants and refugees in grades kindergarten through 12 who have never been schooled in the United States. Each child is assessed with the Woodcock Muñoz Language Survey-Revised (WMLS-R) to determine English proficiency status and for program placement decisions made by the Language Proficiency Assessment Committee (LPAC). Once the registration process was completed, each child received a backpack filled with school supplies and literacy materials. Additional services include Newcomer parent enrichment, literacy and language enrichment, campus support, and community outreach. A six week summer enrichment program at several apartment complexes (with large number of recent immigrants) which feed into Skyline, Conrad, and Thomas Jefferson high schools was started June 9, Intake Center Services In the student intake center was operated by a staff that included an intake center supervisor, one parent/community liaison, two LPAC/intake specialists, two ELL/immigrant academic specialists (one elementary and one secondary level) and two support staff. Immigrant families were referred to the intake center by non-profit agencies, Dallas Independent School District schools and word-of-mouth. The following services were provided; Intake Services parents and students were guided through the registration process, parents were educated about the Bilingual and English-as-a Second Language (ESL) programs offered in the Dallas ISD, campus support was provided in order to maintain state, federal, and district compliance with all bilingual and ESL program guidelines, and campus audits were conducted for LPAC and Bilingual/ESL program compliance, as needed. The intake center registered a total of 1,601 immigrants and refugees during the school year. According to the Home Language Survey completed by parents, a total of 32 languages were spoken among the enrollees. The following table shows some demographic information. Although English was listed as one of the top five languages spoken, it is highly probable that some parents intentionally answered in this manner thinking it would help their child receive English instruction instead of bilingual instruction. A second explanation might have been that a child was born in the U.S. and later moved to Mexico and then returned to the U.S. Table 1. Student Demographics Gender N % Female Male Language N % Spanish 1, Arabic Burmese Nepali English Source. M-LEP dept. Figure 1 shows the enrollees by grade. Enrollment by grade was consistent through grade eight and then peaked at grade nine to 31 percent. Figure 1. Percent enrollees registered by grade KN Other services provided by the intake center included; Newcomer Parent Enrichment empowered parents and students by providing information to help them navigate through the school system, 31

40 encouraged and maintained relationships with all stakeholders involved in supporting and meeting the needs of the English Language Learners (ELLs), worked closely with families to ensure they Accor According to TEA regulations, all districts are required understood and were willing to utilize all district and community services available to them, and guided parents and children in the use of instructional materials and the Parent Literacy Center. Parent meetings, held every other week, were conducted at eight host schools and included parents from neighboring schools. Literacy and Language educated parents about the Bilingual and ESL programs offered in the Dallas ISD, ensured that parents understood their child s academic path, including assessment requirements, monitored students progress, including campus visits and meeting with their teachers, provided campuses with professional development to support students linguistic and academic development, and addressed parent concerns and questions throughout the school year. The following table describes the community outreach services provided to parents and students. Table Community Outreach Activities Parents Students Activity N N Dallas Mayor s Back to School Fair Intake Center Opening Celebration W. T. White Health Fair Rosemont Parent Academy 20 - Dallas Fiesta Latino-Americana Las Llaves al Exito Fair: College Fair 20 - District 5 Academic Pep Rally 25 - Parents as Leaders (Rosemont Elem.) 30 - Parent Appreciation and Resource Fair Dallas Children s International Day 50 - Total Source. Margaret and Gilbert Herrera Intake Center Supervisor. A total count of participants by activity held during the school year included the following: o o o Family Literacy Fairs N=90, Student Enrichment classes N=449, Parent Enrichment sessions N=328, o o Multi-Cultural event N=29, and Language and Literacy Summer Outreach- N=27 parents/65 students. to have policy and procedures on file for the selection, appointment, and training of Language Proficiency Assessment Committee (LPAC) members. The intake center supervisor is charged with these responsibilities, as well as overseeing the daily operations of the center. Training involved the use of a yearly revised LPAC manual that integrates state and federal Title III, of Public Law (No Child Left Behind) requirements regarding the identification, program placement, parent notification, annual review, and assessment of ELLs as they reach academic proficiency. Four training sessions were offered to all campus LPAC chairpersons in (beginning of year, mid semester, mid-year and end of year). In the spring of 2014, a fifth training was provided in preparation for the school year. The purpose of the additional training was to ensure that the ELL identification process begins at the start of the upcoming school year, as this is crucial for the leveling and staffing process of teachers. Summary and Recommendations The Margaret and Gilbert Herrera Intake Center has played an important role in serving immigrant families in our community during the school year. The services provided are greatly needed to offer guidance and support to families not familiar with our culture and educational process and to help the students make the transition into our schools easier. The following recommendations should be considered: continue providing the existing services for the school year, and per funding availability, hire additional full-time staff to audit campuses districtwide and help the staff address LPAC and BE and ESL compliance issues as were observed during the Texas Education Agency audit during the school year. For more information, contact Program Evaluation at

41 AVANCE DALLAS PROGRAM: Rosemary Garcia-Rincón EA At-a-Glance AVANCE-Dallas, a non-profit organization has provided foundational education services through the Parent-Child Education Program to at-risk children and their families in Dallas ISD campuses since The program implements two major components, early childhood development and parent education. The program is offered free of charge to parents and is in 32 elementary campuses; however, 10 of the campuses receive Title lll funding (a total of $300,000) and are the focus of this report. Overview The AVANCE-Dallas Parent-Child Education Program served low-income, Hispanic children, ages zero through three, and their parents. Mothers met weekly for four hours while their children were cared for in an age-appropriate educational setting. AVANCE s nationally recognized program, teaches parents that they are their children s first and most important teachers and that the foundation for education begins at home. Parent participation in the Parent Education program can lead to better parenting skills and an increased appreciation for education that leads them to improve their own literacy and professional skills. The children served in the AVANCE program are not registered Dallas ISD students so recruitment for the program includes door-to-door solicitation within campus boundaries, public service announcements through Univision (Spanish television station), presentations at Parent Teacher Association (PTA) meetings and pre-kindergarten and kindergarten round-ups. Eligibility criteria include ethnicity, age, and low income status; however, verification of income is not required. Although children ages 0-3 are the target group for the program, exceptions are made to include four year olds who were not accepted into a Dallas ISD prekindergarten program. In AVANCE program services were provided at ten elementary level campuses selected by the Multi-Lingual Education (M-LEP) department: David G. Burnet, F.P. Caillet, Esperanza Hope Medrano, Obadiah Knight, Maple Lawn, James Bowie, Anson Jones, John F. Kennedy, Urban Park, and Annie Webb Blanton. A total of 340 parents and 394 children at the participating campuses were registered to receive services in Of these totals, 88.5 percent of parents and 89.8 percent of children completed the program. Early Childhood Development The purpose of the Early Childhood Development program is to advance Hispanic children s learning early on so that they can succeed in school and for the rest of their lives. Classes are provided during the foundational years of birth to age three. Teaching environments are developmentally appropriate for infants, toddlers and preschoolers and were created with the development of children s language abilities, cognitive and social skills, and an interest in learning in mind. Six learning centers are used in the classroom that make learning and discovery an enjoyable experiences for the children as they explore math, art, dramatic play, blocks, writing, and reading. Several activities are used to strengthen phonological awareness, social skills, and literacy development including; alliteration exercises, rhyme challenges, word games, teacher-child conversations, peer-to-peer games, alphabet cards, and story time. Thus, children are given an educational foundation before entering school, providing them confidence and the ability to succeed. Parent Education Teaching parents the importance of their role in their child s education is the best way to ensure their children succeed later in school. Parents are taught by parent educators to develop creative, instructive relationships with their children during the fundamental years of life. The parent education program provided one day of training per week per campus with the exception of Burnet, Bowie, and Kennedy which provided an additional day of training based on need. A total of 32 sessions were offered through the year; however, parents were required to attend 27 sessions within a nine-month period for a total of 150 hours. The weekly sessions covered: parenting, toy making, and community resources. 33

42 In addition, important components of the parent education program included: intensive early childhood classes, parent education, home visitation, and family literacy (parents become active readers with their children). Each AVANCE program participant was to receive an average of seven home visits by a trained toy-making instructor/home visitor. Home visits were designed to work with each participating parent and their child. Home visitors were to observe and document the mother-child interactions. Outcomes Children were administered a pre- and post-cognitive and development test that included; Early Learning Accomplishment Profile (E-LAP) for ages zero through 2, and Learning Accomplishment Profile (LAP-3) for ages three and four. Testing was administered in September and again in May. Children ages three and four, who showed a delay in development were tested in February and results reported to the admitting campus. Outcome data for were not available for this report. Parents were administered a pre- and post-avance Dallas Parent Questionnaire to determine levels of success in parenting knowledge and skills. Parents also kept track of the number of times they read to their child and recorded this on the Weekly-Sign in Form. The goal was for parents to increase reading from one day or less per week to five times per week by the end of the program. Outcome data for were not available for this report. Summary and Recommendations The AVANCE-Dallas Parent-Child Education Program served low-income, Hispanic children, ages zero through three, and their parents at ten elementary campuses in Through the program s implementation of two components, childhood development and parent education, children were given an educational foundation before entering school and parents were taught that they are their children s first and most important teachers and that the foundation for education starts at home. This report provides only general information about the program, as outcome data were not made available for evaluation. It is recommended for that AVANCE and/or participating schools collect participant data and provide it to program evaluators for analysis to determine program effectiveness. For more information, contact Program Evaluation at

43 Reconnection Center Program: Dorothea Weir, Ph.D. EA At-a-Glance TThe Reconnection Center Program (Reconnect) was designed to help students recover credits for incomplete or failed courses. The program allowed students to remain enrolled in courses in the traditional classroom setting on their campus while recovering credit in Reconnection Centers during assigned periods. Credit recovery allowed students to remain on track to be promoted or graduate with their cohort. School counselors and Reconnect staff worked together to identify students who were good candidates for the Reconnect program. Once students were identified, they were assigned to the program for a certain class period, including before- and after-school periods at some campuses. The Reconnect program provided students with online coursework. The online work was prescriptive, meaning that the student pre-tested over the objectives in a module, using the NovaNET system. The pre-test determined which objectives the student needed to master. The student worked through only those modules and was tested again at completion. As students mastered a module, they moved to the next module. With successful completion of prescriptive online work and the district Assessment of Course Performance (ACP) test, students were awarded credit using district grading standards. Three selected campuses (Lincoln, Madison, and Pinkston) piloted the Apex Learning system in the spring of Apex was subsequently selected to replace NovaNET for use in all schools for the school year. The Reconnection Centers had a school-within-aschool environment in which the staff, generally two teachers, formed an educational team. This team was responsible for facilitating student learning and mastering of objectives in core subjects using online, self-paced coursework. Each center contained 25 to 30 networked computers. At the time of this report, the district was operating Reconnection Centers at 30 high schools receiving Title I funds. The Title I workscope allocation for this program was $2,608,763. Student Selection Criteria Criteria and procedures for student selection were modified for the school year. A Path of Documentation was established, and relevant personnel at each campus received information and training on the new requirements. Student selection was initiated by the counselor, who met individually with students who had failed a core course. The purpose of the meeting was to determine (a) whether the failure was in a course required by the student s graduation plan, and (b) if so, whether the Reconnection Center was the most viable option for credit recovery, versus other alternatives (summer school, evening school, or in-class instruction during the regular semester). The counselor presented academic expectations and a timeline for those expectations to the student. The student was informed about the grading structure and student responsibilities, and was required to agree to the expectations and responsibilities in a signed contract. In order to complete the online work required in the Reconnection Center, the student was also required to have passed either the 8 th grade STAAR Reading test or the English I End-of-Course (EOC) test. If the student had passed neither of those, a reading level of 6.0 on the Basic Assessment Skill Inventory (BASI) was necessary for enrollment. Characteristics of Students Served in Reconnect Across grades 9-12, 7.9% of enrolled students (3,044 of 38,639) were registered for one or more periods of Reconnect. Figure 1 shows the percent of students at each grade level who were enrolled in the program during either or both semesters of the school year. The majority of students served were in Grade 12 (N=1,796); more than one-fifth of district seniors had at least one period in Reconnect. In comparison to districtwide percentages, a larger proportion of Reconnect students were male and African American, and a smaller proportion were LEP or economically disadvantaged. 100% 80% 60% 40% 20% 1.6% 4.0% 7.5% 98.4% 96.0% 92.5% 21.9% 78.1% 0% Grade 9 Grade 10 Grade 11 Grade 12 Reconnect 1.6% 4.0% 7.5% 21.9% Other 98.4% 96.0% 92.5% 78.1% Figure 1. Students in Reconnect and not in Reconnect by grade. 35

44 Outcomes of the Reconnect Program ACP. Reconnection Center students were required to take the district s ACP final examination to receive course credit for each completed semester. Figure 2 shows the percentage of tests taken in Reconnect by content area, and Table 3 shows the results of those tests. All tests taken were in subjects required for graduation under the student s graduation plan. Passing rates by content area varied from 39% to 75%, and mean scaled test scores ranged from 67 to 77. The majority of Reconnect students did not pass semester final examinations in their Reconnect courses. Social Studies, 20.1% Science, 24.7% Language Other Than English, 2.7% Mathematics, 17.2% Health, 4.0% Economics, 3.8% English, 27.5% Figure 2. ACP exams taken in Reconnect by content area. Table 1 Results of ACP Exams Administered in Reconnection Centers Content Area N Tested Mean Score Percent Passing English/Lang. Arts 1, Mathematics Science 1, Social Studies Lang. Other Than English Economics Health Credit Recovery in Grades The workscope Key Result for students in grades 9-11 was stated as, 80% of the under-credited 9 th, 10 th, and 11 th graders in the Reconnection Center will recover credits to advance to the next grade level by July Table 2 shows credit gains of Reconnection Center students between the end of the first six weeks and the end of the second semester of the school year. Grade 12 is included in this table for comparison purposes. The year s credit accumulation was lower at all grade levels than the credit gains expected for students enrolled in regular classes, which generally must average six credits per year to graduate in four years. Table 2 Credit Gains of Reconnect Students Credits Grade N Students BOY EOY Gain , Table 3 shows the percentages of Reconnection Center students who attained the next grade level by the end of the second semester of Only grade 11 students reached the 80% promotion target. Table 3 Promotion Rates of Reconnect Students Promoted Grade N Students N Percent , A second Reconnect workscope goal was that 90 percent of grade 12 Reconnect students would recover credits sufficiently to graduate with their class by August, By the specified date, 81% of 12 th grade Reconnect students had graduated, and an additional 7% had completed all course requirements for graduation, although they had not passed one or more sections of the TAKS (Figure 3). Graduates, 81% Figure 3. Graduation Status of Grade 12 Reconnect Students. Recommendations Completed course requirements, 7% Possible dropouts, 2% Continuers, 10% A recommendation from the past four school years which was addressed in was to create clear criteria for selecting students for the program, including setting the BASI reading level score at a sixth grade equivalency. As well, specific counselors have been identified to work with Reconnection Center facilitators, and have been given a larger role in identifying and enrolling students in the program. 36

45 Documentation of coursework and attendance continues to be inconsistently and inadequately maintained. These problems may be alleviated by better information available through the Apex Learning system, which is coordinating with Dallas ISD Information and Technology Services to track students course progress using district-verified student names and ID numbers. It is further recommended that the existing student grading system be used to record attendance and grades. With new limitations on Reconnect courses offered, and an increased emphasis on providing instruction in the five EOCtested courses, one solution would be to repurpose the existing Reconnection course numbers to identify the course being taken. Before- and after-school enrollment should be also recorded in GradeSpeed at every campus. Electronic documentation of student progress wll allow more accurate, thorough, and informative evaluation of the impact of these programs on credit recovery. Prior achievement results, course grades, attendance data, and course completion records of current Reconnect students could then be analyzed to refine identification criteria, in order to assist schools in assigning students to Reconnect who are most likely to be successful in the program. For more information, see EA , which can be found at 37

46 38

47 Sarina Rapini, MPA Title II, Part A Teacher and Principal Training and Recruiting: At-a-Glance EA Dallas Independent School District (Dallas ISD) received federal funds for training and recruiting through Title II, Part A as authorized under the No Child Left Behind Act, Public Law The two-fold purpose of Title II, Part A is to increase student achievement through improved teacher and principal quality and to hold local educational agencies and schools accountable for student academic improvement. Program Description Dallas ISD used grant funds to assist in the areas of certified teacher recruitment, class-size reduction, instructional support services (ISS), expansion of the Alternative Certification (AC) program, distance learning, arts based professional development, advanced academic teacher training, International Baccalaureate (IB) training, and campus-specific supplemental training. Recruitment activities were part of class-size reduction and certified teacher recruitment efforts, whereas staff training was the focus of the remaining grant activities. Full-scale program evaluations of two additional Title II components (Reasoning Mind and the Leadership Development Fellows Academy) were conducted separately. Methodology The workscopes for Title II, Part A were reviewed to note the amount of funding allotted for each area, program plans for implementation, and goals. In addition, formal and informal interviews were held with program managers to find out details related to implementation and to collect program documentation. Follow-up information was collected via phone and . Amounts of Funding The district received $11,862,964 in Title II, Part A funds. As seen in Table 1, almost six million ($5,848,616) was allotted to the Class Size Reduction Program. Approximately $1.4 million was budgeted for the Certified Teacher Recruitment Program ($1,432,729) and Instructional Support Services Training ($1,343,309). Other large amounts included the Principal Fellows Academy ($1,196,470), Expansion of the Alternative Certification (AC) Program ($697,331) and Reasoning Mind Professional Development ($525,000). Another $321,000 was allotted to Advanced Academic Teacher Training. The remaining allocations were for specialized campus training and ranged from $15,900 (Montessori Professional Development) to $170,216 (Distance Learning). Program Activities and Implementation Certified Teacher Recruitment Program Human Capital Management (HCM) teacher recruitment activities included attending recruitment events, advertising, conducting information sessions, screening applicants, processing candidates for H-1B visa applications, and documenting recruitment activity success. Between September 2013 and April 2014, 66 recruitment activities took place in Texas (N=47; 71%) and other locations within the United States (N=19; 29%). Upon completion of recruitment events, staff members wrote short follow-up reports, which will be used to guide future recruitment events. HCM staff processed 23 new teacher visas and 64 extensions. Class Size Reduction Program Grant money was used to fund teacher positions in core subject areas to reduce class sizes and better meet the instructional needs of students. Grant positions were filled based upon grant guidelines and a prescribed district formula. HCM staff members worked with principals to identify campus needs and to hire highly qualified teachers. Positions were filled on an as needed basis throughout the year. During the school year, there were 64 grant-funded teacher positions across 44 campuses. Positions were spread across Division 1 (N=15), Division 2 (N=11), Division 3 (N=10), Division 4 (N=17), and Division 5 (N=11). Instructional Support Services Training ISS staff utilized Title II funds to provide supplemental professional development and resources for teachers, principals, and other instructional staff. Professional development included face-to-face and online core content training, as well as 67 training sessions on the Response to Intervention (RtI) framework and the Exceed Student Achievement Management System. ISS staff coordinated four Region 10 Student Assessment of Academic Readiness (STAAR)/End of Course (EOC) Academy training sessions for staff. The purpose was to provide the latest information on STAAR and to discuss instructional adjustments necessary for student success. Through the sessions, resources and strategies 39

48 were shared to assist campus staff with planning, instructional rigor, and student engagement. Expansion of the Alternative Certification Program Grant funds were used to expand the AC program by supplementing recruitment, training, and retention activities. AC staff participated in 52 recruitment events between July 2013 and June There were 527 interns placed in 165 campuses during More than half (59%) were in the district s critical need areas of bilingual, math, and science. The AC hiring goals of 150 bilingual, 50 mathematics, and 50 science interns were met for bilingual (N=198), mathematics (N=50) and science (N=63). Of the 317 candidates that were certified in , 220 (67%) continued to teach in the district during Distance Learning Distance Learning staff members provided training and technical support for core content specialists in the development of online training courses. They also helped instructional staff create virtual Professional Learning Communities (vplcs), which allowed for teacher and administrator collaboration. As part of each vplc session, teachers were required to complete a scaffolding field experience in their classrooms. Distance Learning staff also facilitated a blended learning initiative in Imagine 2020 secondary schools. To prepare teachers for blended learning instruction, Distance Learning staff developed and presented 19 Google Drive modules throughout the school year. Improved Arts Based Professional Development Arts based professional development was provided to instructional staff to improve the quality of art instruction and align the curriculum to the new Texas Essential Knowledge and Skills. In the fall, program staff conducted 45 professional development sessions for 1,387 participants. During the spring, 1,212 participants attended the program s 45 professional development sessions. During the school year, Dallas ISD students competed in more University Interscholastic League events and earned more good, excellent, and superior ratings than in the previous two school years. Students also received fewer poor and unsatisfactory ratings. Advanced Academic Teacher Training Title II funds were budgeted for advanced academic teacher training to increase the number of highly qualified advanced academic teachers. Advanced academic teachers, including Advanced Placement (AP), Pre-AP, and IB teachers, attended professional development to improve their effectiveness. The National Math and Science Initiative (NMSI) and International Baccalaureate Organization (IBO) hosted and conducted the training. NMSI provided two workshops for AP teachers during the school year and two sessions for Pre-AP and AP teachers in June Three teachers were sent to IB workshops in Florida and Dallas in June International Baccalaureate Professional Development Grant funds were used to fund professional development for three Dallas ISD IB campuses. In December 2013, J.L. Long became an authorized Middle Years Program (MYP) school. Harry Stone remained in the candidacy phase to become an IB MYP campus, pending the outcome of an authorization visit made in May Wilson was in its third year as an authorized International Baccalaureate Diploma (IBD) campus and in its second year to offer the International Baccalaureate Career-Related Certificate (IBCC). Harry Stone. Stone sent 29 staff members to IB training between July 2013 and October To comply with IBO requirements, sixth grade was moved into middle school and given a middle school Texas Education Agency (TEA) code. All but four of the 69 eighth grade students (94%) were accepted into their selected high school. Long. Long sent 20 IB staff members to out-of-state IB training sessions between September 2013 and June Wilson. Wilson used Title II funds to send 10 staff members to IB training sessions. In all, 182 students in ninth and tenth grade took pre-ib courses, and 38 eleventh- and 28 twelfth-grade students participated in the IBD program. By the end of the school year, all 28 IB seniors had graduated and were accepted into at least one college. Of those 28 IB students, 25 met requirements to receive the IB diploma. An additional 28 eleventh and 19 twelfth grade students took IBCC courses. All 19 IBCC seniors were accepted into college, and two completed the IBCC program. Wilson will not offer the IBCC program in Campus-Based Professional Development Title II grant funds were used to provide supplemental training to meet the unique instructional needs of Barack Obama Leadership Academy, Irma Rangel Young Women s Leadership School and George Bannerman Dealey Montessori. Barack Obama used Title II funds to send staff to workshops, bring in speakers for staff training, and purchase books for a summer book study. Irma Rangel sent nine staff members to five conferences and training sessions. Dealey Montessori sent four teachers to the downtown Dallas Montessori 40

49 Academy to receive training on how to use the Montessori teaching methods and materials. Recommendations A majority of Title II funds were spent on professional development. Although attendance information was collected, in some cases, feedback was not gathered to determine the quality and usefulness of the training. Watson, C. E. (August 2012). Dallas ISD Evaluation Report: Final Report of Title II, Part A Professional Development. The full Title II report can be found at For more information, contact Program Evaluation at Programs that include training and utilize Title II funds should include an end-of-course survey to gather feedback from participants. The survey data would be useful for future planning and quality control. ISS staff members collected survey data for online and face-to-face training sessions, but the survey did not specify whether training was online or in person. It would be best to add an item to the survey that asks participants to specify the training type attended or collect the data separately. That way ISS staff can gauge the quality of training by training type. The Response to Intervention program implemented a survey to gather feedback from staff for It is recommended that the survey be administered again in , so that changes can be tracked over time and so that RtI staff can continue to pinpoint needs. Distance Learning program staff created a usage report of student and staff Chromebook activity. However, the usage report did not include enough detail to evaluate the quality of blended learning in the classroom. The report did not specify whether a staff member was using the Chromebook for training or instructional purposes. Similarly, the report did not include information about the length of activities; for example, students and staff that used the Chromebook for one minute appeared the same on the report as those that were online for one hour. Adjustments should be made to capture more detailed information about specific Chromebook activity, so that the quality of blended learning can be better evaluated References Bush, J. and Hernandez, D. (August 2011). Dallas ISD Evaluation Report: Title II, Part A Teacher and Principal Training and Recruiting Bush, J. (August 2013). Dallas ISD Evaluation Report: Title II, Part A Teacher and Principal Training and Recruiting

50 42

51 Youth and Family Centers Program: Leonardo R. Ledezma, Ph.D. EA At-a-Glance The Youth and Family Centers (YFC) program formally began offering behavioral health services to students in the Dallas Independent School District in The main objectives of the YFC program included, but were not limited to, (1) providing school-based health care, behavioral health care, and support services to Dallas children and families, (2) reducing the barriers to academic success so children can learn and teachers can teach, and (3) promoting the wholeness of the family by engaging families in their children s health care and education. The Title I allocation for this component was $863,717. In , the focus of the YFC program evaluation was to describe the general characteristics of the YFC program, clients, and staff; determine the effectiveness of the program at increasing service delivery compared to prior years, as well as the number of YFC services delivered and program growth. YFC Program Growth YFC has shown an increase in the number of students served over the past five years (2008 to 2013). The number of students served and the number of visits per student at YFC decreased during the school year (Table 1). This decrease was probably attributable to the fact that critical staff experienced turnover. The lack of full-time positions made it difficult to recruit and retain qualified Spanish speaking therapists. Several therapists in part-time positions left for full-time employment elsewhere. Two psychiatrists left the district for full-time positions in other institutions. Table 1 presents the number of students served and the number of visits for psycho-social services from the school year through the school year. Table 1 Number of YFC Clients and Services Number of Students Mean Number of Visits/Student School Year Number of Visits ,211 24, ,388 20, N/A N/A N/A ,599 28, ,244 26, ,829 31, ,493 40, ,817 35, ,087 54, ,504 40, In center locations were critical with new centers being opened such as Wilmer-Hutchinson. Several of the old centers were moved to strategically better locations. The director of the YFC made it a top priority to train principals and other school administrators about the program and students needs. Parent awareness has increased regarding program services that are available to meet the needs of their children. Lastly, YFC and the Homeless Education/McKinney-Vento programs have formed a partnership with external agencies to provide a more comprehensive service to the student population. Increased productivity standards and improved billing processes resulted in clinical services being billed to sources such as Value Options/Medicaid, private insurance, or a family cost share sliding scale. These improvements generated revenue of about a million dollars for the year. Client Satisfaction Data collected at the service sites revealed a moderate improvement in the overall level of satisfaction of students and parents. Responses to the Satisfaction Survey for Services Provided by YFC indicated that students and their families had similar levels of satisfaction when the information was collected, in 2013 and Students and their families reported that the staff of the YFC was friendly and helpful. They also responded that it was easy to get an appointment with a counselor, counselors were able to relate to their needs and that the suggestions given by the counselors were helpful. Students and their families were satisfied with the counseling services received at the YFC. According to their responses, students and their families agreed that the counseling they received had a positive impact on students and their families. Students and their families responded that they made changes as a result of the counseling received at the YFC. They also felt their goals were met and that they would recommend the YFC services to friends or family members. Staff Overview The YFC program employed 95 staff members in Staff included support and administrative personnel (27), full-time and part-time behavioral 43

52 health professional employees (68). The part-time staff consisted mostly of therapists who provided behavioral health services to students and their families in the evening hours. Staff Educational Levels. Data showed that a large proportion of YFC behavioral health staff attained master s degrees, with 54 of 68 (79.4%) staffers holding the Master of Arts or Master of Science degree. Most of these were therapists providing services to students and their families. Thirteen staffers held a Ph.D. degree. Gender. Administrative and behavioral health staff members were all females. Client Characteristics Ethnicity, Gender and Grade. Although the total number of client contacts reported by the program was 40,896, only 39,906 contacts had student I.D., grade, ethnicity and gender information needed. Grades 1 through 5 (elementary level) had 56.4 percent of the client contacts. More than half of all client contacts occurred in elementary school level. Hispanic students made up 62 percent of all client contacts while, African Americans accounted for 26.2 percent of client contacts. Males, both African American and Hispanic, accounted for 56.1 percent of the client contacts, compared to 32 percent for females. Hispanic males were responsible for largest percentage (38.6%) of client contacts. Third grade students had the most (10.8%) client contacts. YFC clients were more heavily distributed in grade 1 through grade 7. The number of client contacts increased steadily from grades PK through 2 and decreased from 4 through 12 (see Figure 1). The largest number of client contacts occurred at grade 2 (4,382) and grade 4 (4,417). Figure 1. YFC Client Contacts by Grade, PKKN Summary and Recommendations During the school year, the YFC program provided 40,896 mental health service contacts and served 5,504 students. Increased productivity standards and improved billing processes resulted in clinical services being billed to sources such as Value Options/Medicaid, private insurance, or a family cost share sliding scale. These improvements generated revenue of over a million dollars for the year. The YFC staff consisted of behavioral health professionals (N=68) most of them with a master s level of education (N=54, 79.4%). In general, the demographic characteristics of YFC clients were representative of the overall Title I schools student population. The YFC program worked in coordination with the Homeless Education/McKinney- Vento program within the district and agencies outside the district. This coordination allowed the program to increase the efficiency of the use of resources and maximize the services to students. Recommendations for the YFC program improvement include (1) continue increasing students awareness of the services provided by the program, (2) continue exploring additional services or interventions that could contribute to increase the behavioral health of students and their families, (3) continue to recruit bilingual therapists and Child and Adolescent Psychiatrists, (4) the district should provide the funds to hire full-time therapists to prevent the personnel turnover. The increased demand for services created a waiting list, which means that students would have to wait for services. The nature of the student s needs require prompt service and attention. The district should increase funding to hire more personnel and attend to the behavioral needs of the students and their families. For more information, see EA , which can be found at Client Contacts by Grade 44

53 Fellows Academy Program: Nora E. Douglas, Ph.D. EA T he Dallas Independent School District created the Leadership Development/Fellows Academy in response to the Destination 2020 plan and tasked the program with increasing leadership capacity and supporting principals in campus efforts to improve quality of instruction, raise student achievement, and create a positive school culture and climate. The program operated during the and school years. There were 57 fellows (cohort I) selected for the school year and 48 (cohort II) selected for the school year. Cohort II fellows participated in training and professional development sessions, a residency experience with a mentor principal, and supplemental work and activities. Strengths and Areas of Growth Cohort I and II Cohort I and II fellows participated in online surveys and in-person interviews, and identified areas of strength within the Fellows Academy program and areas of potential improvement. Two major areas of strength in the Academy training were: the network of fellows and the departmental presentations/rotations. The hands-on practice during the residency in a safe environment was the most common strength identified. Figures 1 and 2 show some of the responses to cohort II survey items. Fellows were asked to rate the quality of their relationship with their mentor principal. Figure 1: What was the quality of your relationship with your mentor principal (Cohort II)? Neutral, 9.8% Negative, 4.9% Very Negative, 0.0% At-a-Glance (see Figure 2). Two other common themes were making sure the Fellows Academy training curriculum was fully planned before training began and providing more training on operational topics such as attendance, truancy, discipline, Special Education policies, and test coordination. Figure 2: During the school year, how often would additional support (from Academy staff, mentor principals, or other fellows) have been beneficial (Cohort II)? Almost Never 3.4% Rarely 10.3% Sometimes 41.4% I did not need any support 10.3% Almost Always 20.7% Frequently 13.8% Cohort I Principals, Assistant Principals and Other Leadership Positions In April and May 2014, the evaluator conducted interviews with cohort I fellows employed as principals on Dallas ISD campuses for the school year. In addition, the evaluator administered an online survey to campus staff and teachers. Positive, 29.3% Very Positive, 56.1% Three areas of potential focus were: the order of training topics, mentor principal accountability (place fellows with mentor principals who were receptive to having a fellow on their campuses), and sufficient feedback and in-person support from Academy staff Results of the survey and interviews indicated that cohort I fellows set clear expectations for staff, built teams and increased collaboration, and gained staff trust through consistency and availability to improve campus culture. With regard to quality of instruction, fellows conducted spot observations and coached teachers on alignment of standards, demonstrations of learning (DOLs) and learning objectives (LOs). Fellows were less likely to report that they had made significant changes to student achievement on their campuses, but indicated that they used data to make decisions for their campus and expected to 45

54 see positive academic results. Fellows also created new procedures for arrival and dismissal, discipline, passing periods, lunch, and recess to improve campus operations. Almost all of the fellows repeatedly commented on the necessity to focus first on campus procedures and building infrastructure, then on quality of instruction and campus culture, with student achievement being an indirect result of the preceding work. Figure 3 shows how campus staff rated their relationship with their cohort I principal fellow. The majority of staff rated their relationship with their principal as Very Positive or Positive (77.5%). Figure 3: Staff relationship with cohort I principal 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% What is your relationship like with your current principal? 42.0% Very Positive 35.5% 15.4% No statistically significant differences were detected between fellow and non-fellow campuses with new principals for the school year on Climate Survey or STAAR results. Campus Placements 4.8% 2.4% Positive Neutral Negative Very Negative As of July 24, 2013, there were 19 cohort I fellows in principal positions, 30 in assistant principal positions and 6 in other leadership positions. Two fellows had left the district. As of July 17, 2014, there were 21 fellows in principal positions, 24 in assistant principal positions and 4 in other leadership positions. Another 6 fellows had left the district (Figure 4 and 5). Figure 4: Cohort I Fellow Principal Placements from July 24, 2013 to July 17, Fellow Promotions Resignations Principals, July 24, Fellow Principals, July 17, 2014 Figure 5: Cohort I Fellow AP and Other Staff Placements from July 24, 2013 to July 17, As of July 21, 2014, six cohort II fellows had been placed in principal positions, 36 in assistant principal positions, and 3 in other leadership positions (i.e., IB coach, instructional coach and librarian). Two fellows positions were yet to be determined and one fellow had resigned. Recommendations Fellows in Promoted Resigned Promoted Fellows in Position, July 2013 from Other to AP Position, July 2014 Asst. Principal Other Positions There were six recommendations resulting from the evaluation. Fellows Academy program staff have already begun addressing these recommendations. 1. Provide more opportunities for hands-on training experience (departmental rotations, online database training, follow-up training sessions for graduates). 2. Improve mentor principal selection process by selecting principals with coaching mindset, selecting campuses with appropriate characteristics, and providing mentor principal training and continuous oversight. 3. Hire additional Fellows Academy staff members in order to provide more oversight and in-person support of fellows, mentor principals, former fellow graduates. 4. Continue to refine the Fellows Academy training curriculum to cover the most relevant training topics in the most appropriate order to account for residency timing and fellows who leave the program for mid-year employment opportunities. 5. Conduct more Fellows Academy staff visits to campuses during residency and during first year out of the program. 6. Provide more guidance and training regarding the principal selection process as well as support for fellow graduates still going through the process. Additional information may be obtained by consulting the Fellows Academy Program Report, EA , at:

55 Texas Title I Priority Schools Grant: C. Elizabeth Watson, MPA EA At-a-Glance The Texas Title I Priority Schools (TTIPS) grant is a program funded under the Title I Federal School Improvement Programs grants. The overall goal of the TTIPS grant is to assist some of Texas most underperforming schools to make the necessary changes to undergo drastic improvement, including improvement of students academic performance, teacher quality, parent and community engagement, and campus culture. Campuses selected for the TTIPS grant receive funds in order to make needed improvements. Program Description The TTIPS grant program, under the direction of the Texas Education Agency (TEA), described four intervention models designed to improve low performing schools. In order to receive the grant, applicants had to agree to undergo one of the four intervention models. Franklin D. Roosevelt, H. Grady Spruce, and North Dallas agreed to the Transformation model and A. Maceo Smith New Tech agreed to the Turnaround model. The Turnaround Model required the school s principal and 50 percent of the current staff to be replaced, improvement in the use of data to guide instruction, and strengthening community- and social-supports for the students. The Transformation Model required the principal to be replaced unless the principal had been with the school less than two years. In addition, the school was to implement a staff evaluation system tied to student achievement and other factors such as classroom observations. The school was also required to implement practices aimed at improving use of data to modify teaching and providing staff with professional development. Campuses receiving the grant were measured by how well that campus improved across seven critical success factors (CSFs): 1. Academic Performance 2. Increase the Use of Quality Data to Drive Instruction 3. Increase Leadership Effectiveness 4. Increase Learning Time 5. Increase Parent/Stakeholder Involvement 6. Improve School Climate 7. Increase Teacher Quality These success factors were measured using various tests, surveys, and other district data. The schools and districts set goals for each of the three years of the grant. In the summer of 2010, TEA awarded Smith New Tech, Roosevelt, and Spruce high schools the TTIPS grant. TEA awarded Roosevelt a total of $5,246,825 (100% of requested amount) and Spruce a total of $5,205,909 (100% of requested amount) to be spent over three years. Smith New Tech received permission for a delayed implementation of the grant for the school year, and was awarded $5,056,869. In the summer of 2011, TEA awarded North Dallas High School the TTIPS Cycle 2 grant. North Dallas received $5,981, (100% of requested amount. For the school year, although Spruce and Roosevelt had completed the three years of their grant implementation, they were permitted an extension of the period of availability of funds until September 30, TAP System The TTIPS grant called for a system that rewarded teachers, principals, and staff who, in implementing the Transformation Model, had increased student achievement and high school graduation rates. The Turnaround Model also had the option of implementing this system. The district selected the TAP system, a comprehensive research-based reform model that provided differentiated compensation for teachers and principals based on teachers performance in the classroom and the academic performance of their students. The teaching observations are based on TAP s Skills, Knowledge, and Responsibilities (SKR) rubric, which measures teaching pedagogy. The TAP system was not funded past the original three years of the grant award, and therefore Roosevelt and Spruce High Schools did not participate in TAP in , as they were in an extension year which allowed them the opportunity to utilize unspent grant funds. While TAP participation was optional, Smith had 100 percent participation in the TAP program this year, while North Dallas had 92 percent. Participating teachers were separated into three categories -- career, mentor, and master teachers -- to distribute instructional leadership and fulfill TAP s goal of providing multiple career paths to teachers that keep them in the classroom. Every TTIPS campus had a leadership team that consisted of administrators and teachers to conduct SKR observations for the instructionally focused accountability component. SKR observations were completed several times throughout the school year by a member of the leadership team. At the end of each 47

56 school year, results of SKR observations would be one component of the performance based compensation (PBC) that was calculated by the National Institute for Excellence in Teaching (NIET). The other half of the PBC was based on student achievement value-added growth, which is based on two parts: 20 percent on school-wide value-added growth and 30 percent based on value-added growth of an individual classroom teacher s students. If the teacher does not teach a TAKS/STAAR tested course, such as art or physical education, the 50 percent value-added PBC is based solely on the school-wide growth. Teachers at extension campuses, Roosevelt and Spruce High Schools, will receive financial awards based solely on their CEI quintile. Those in the first quintile will receive $1,000; those in the second quintile will receive $2,000; those in the third quintile will receive $3,000, and so forth. While this system necessarily excludes teachers without CEI s, program management determined it to be the most appropriately comparable system to the one in place at North Dallas and Smith New Tech High Schools for the year. Outcomes The grant provided funds for schools to improve across seven areas of the school: academics, increase use of quality data to drive instruction, increase leadership effectiveness, increase learning time, increase parent/stakeholder involvement, improve school climate, and increase teacher quality. As part of the application process, campuses and the district had to set a series of measures and goals for each area. Roosevelt and Spruce completed implementation in , but were permitted an extension of the period of availability of School Improvement Grant (SIG) funds that were awarded for Cycle 2 schools until September 30, 2014; North Dallas and Smith New Tech had one final year of TTIPS granted funding. In , Smith New Tech and North Dallas reached the 70 point requirement. In North Dallas was the only campus that achieved the 70 points required. In , Smith New Tech earned 91.7 points, while North Dallas earned 58.7 points, and missed the 70 point requirement for the first time in its grant history. TTIPS schools do a good or very good job in talking with them about student situations. Three Dallas ISD campuses applied for Cycle 3 TTIPS grants in the summer of 2014, but none were awarded funding. Both A. Maceo Smith New Tech and North Dallas High Schools also applied for an extension year to spend their fund balances through September of 2015, in the same manner Roosevelt and Spruce High Schools did in At the time of this report, these applications were still under review, with no decision returned. Recommendations Campuses and the district should have a sustainability plan in place for campuses, especially regarding personnel: Campuses should have a long-term strategy that considers what the impact will be without these grant funded activities, and how the district and campuses can moderate that impact. More importantly, as part of the TTIPS activities, many highly qualified personnel were hired to enact various aspects of the grant (e.g. master teachers for the TAP program, campus coordinators). These highly skilled employees will no longer have positions at the campus after the final year of implementation since their positions are grant funded. The campus and district should make every effort to identify these valuable employees and find other positions for them after the TTIPS grant expires. For more information, please refer to the full report EA found at In a survey of TTIPS teachers, respondents generally rated Teacher Quality, the Use of Quality Data, and Leadership highly. Two domains in which a significant variance between campuses can be seen are Parent/Stakeholder Involvement and School Climate. The two campuses at which teachers rated Parent/Stakeholder involvement the lowest were also the two who rated School Climate the lowest: Roosevelt and Spruce High Schools. A survey of parents of students attending TTIPS campuses found that the large majority of parents responded that the 48

57 Evaluation of the Office of Family and Community Engagement Program Michael Lucas, Ph.D., Yaohua Wang EA At-a-Glance The Office of Family and Community Engagement (OFCE) has been operating in the district for more than a decade, though the name has changed and the mission has been adjusted in order to meet the district s evolving needs. The OFCE was staffed by eight staffers, one director, one manager and six coordinators, specialists and trainers whose primary task was to bring parents into active involvement in their children s education and schools. The OFCE s webpage, provides extensive information and links for parents. Evaluation Component The evaluation was guided and informed by the Title I, Part A Federal Funding workscope. The workscope was prepared by the Project Manager and staff and clearly outlined the project(s) to be undertaken, the approach to implementation, the goals and expected results of the program, and metrics by which program success may be measured and evaluated. The workscope requested $490,392 to fund the program. Program Components PREP University Family Workshop Series One of the key programs the OFCE offered to help foster parental involvement was the PREP University Family Workshop series (PREP U). The family workshops were divided into two versions, the evening series and the Saturday series, called Super Saturday. The Super Saturday events presented parents with the opportunity to attend up to three different sessions in one day. Overall, the Office of Family and Community Engagement presented a variety of workshops in 41 sessions. Parents were surveyed about the workshops they attended. The OFCE scored 90 percent or above on approval ratings in virtually all categories. Parent Portal Registrations The OFCE was responsible for training parents how to access and effectively use the district s Parent Portal website. The Parent Portal system was a gateway into a comprehensive look at student performance that included classroom assignments, test scores, and attendance records. The website also offered an opportunity for parents to communicate with teachers. Video Library The OFCE made a goal to develop a training video library consisting of at least three videos in As of April, 2014, the library was available at and contained three videos, all addressing some aspect of registration for Parent Portal. The videos appear to be professionally produced and were visually appealing, and that may have encouraged viewers to view the entire video. PREP Leaders Program The PREP Leaders program trained parents to be leaders and skilled parent-educators as they took a significant role in fostering their children through the educational processes. The immediate goals of the program were to increase parental involvement and to empower parents to become leading advocates in their children s education. 1 In , the program expanded to seven feeder patterns. The program collaborated with community liaisons and school officials from 76 target schools to select one or two parents to invite into the PREP Leaders program. At the end of the school year, the district had 76 participating campuses, but only 42 campuses had active PREP Leaders, as some campuses had multiple leaders. District Parent Advisory Council The District Parent Advisory Council (DPAC) was made up of parent and community leaders that met monthly November through June. The DPAC served as an advisory council to the Office of Family and Community Engagement. The subject of interest to the DPAC was barriers to substantive or meaningful parental involvement and ways to overcome those barriers districtwide. Processes for determining needs were presented as part of the annual Title I Comprehensive Needs Assessment conducted by the DPAC. In , DPAC was formed by volunteers selected from each feeder pattern. Training Academy The Dallas ISD Training Academy began operations in The Academy offered both required courses and courses of interest for district community liaisons. Some of the required courses were Parent Portal, School Messenger, and the Uniform Program. 1 Program Management Strategy and Implementation,

58 The Academy also helped with courses required to operate other programs housed with the Office of Family and Community Engagement. SUMMARY AND RECOMMENDATIONS Summary The Office of Family and Community Engagement has successfully executed the requirements of their Title I workscope. The workscope s Key Results section, goals, and the program s performance are summarized here. The Project Key Results section of the workscope specified an increase in Parent Portal registration of 15 percentage points. They needed to reach 35 percent registered to achieve that. As of May, 2014, the OFCE had registered 52 percent of Dallas ISD households. The next workscope goal was to increase PREP University Attendance by 10 percent. The baseline attendance was 2,092. The program needed to register 2,301 parents for the workshops to achieve that. At the end of May 2014, 3,213 parents had attended workshops, a 54 percent increase. The Office of Family and Community Engagement signed up to recruit and train a minimum of 60 PREP Leaders by the end of the year. A total of 65 parents completed the training and remain active at the end of This figure was cumulative and included any persons trained prior to the school year. The PREP Leader training goal is the least successful aspect of the program. By the end of May, each PREP Leader was to have hosted a minimum of four parent workshops or meetings. Toward the end of the year, there were not enough workshops left in the year for all of the parents who trained to handle a minimum of four workshops as the lead trainer. However, the OFCE made sure that all of the PREP Leaders took part in the production and presentation of a minimum of four workshops. According to the workscope, one of the key program goals was for at least 65 percent of parents to indicate the district offered adequate workshops and resources. Eighty-eight percent of the evening sessions and 91 percent of the Super Saturday sessions participants either agreed or strongly agreed that: The district offers adequate workshops and resources to support the child s education in the home. These results indicate that parent approval is much higher than the target goal. The OFCE was unable to document meeting the workscope goal that says 60% of community/parent liaisons/other designees will have attended a minimum of three training sessions through the Training Academy on Parent Portal, School Messenger, Six Types of Parent Involvement, successful PTA/PTO, Launching a PREP U Campus Series, etc. and will be able to develop and launch parent engagement initiatives at their campus by hosting and leading parent workshops and events. This is an overly complicated goal that could be greatly simplified into two or three less complex and more easily documented goals. That would likely help with compliance. The OFCE was able to document at least 47% of schools in compliance with another 14% documenting two out of the three required sessions, narrowly missing the 60 percent goal. We strongly recommend breaking this goal into at least three separate goals and we recommend positive documentation processes in place at the beginning of the year. Recommendations We see two areas where significant gains can be made. First, many of the program s goals were difficult to document, or were documented at the end of the year from data collected along the way. We recommend avoiding overly complicated goals, making several simpler goals instead. Simplified and clearly written goals make it easier to develop and maintain recordkeeping and allow program staff to monitor progress throughout the semester. Second, this department has the opportunity for significant growth over the next two to three years. In , Parent Portal registrations were reported as exceeding 52 percent, using the number of students with at least one parent registered for Parent Portal as of June 1, 2014 as the representative measure. This means the district now has access to slightly over half the households in the district. PREP University, other programs hosted by the Office of Family and Community Engagement, and other district service groups must continue to reach out and find more innovative and engaging ways to attract and serve the families of students, if the district is to reap the positive effects of parental involvement so much of the literature suggests. The growth rate for this and other programs suggests that the district should expect more parental involvement and should prepare to support parental demands for services and to leverage opportunities to harness parental volunteerism to help in the schools and promote their goals. The full potential for impact on student lives is difficult to imagine, though surely it is enormous. The full Office of Family and Community Engagement evaluation report can be found at 50

59 African American Success Initiative: Anduamlak Meharie, Ph.D. EA A frican American Student Success Initiative (AASI) provides support to feeder patterns with a high concentration of African American students and those with the widest achievement gaps. The initiative focuses on academic support, student advocacy, and other support services designed to improve student academic success and prepare students for college, career, and the workforce. Program Background and Goals Beginning late in SY , the program employed a more decentralized approach to implementation, in which campuses were given the discretion to design and implement activities most appropriate for their individual needs. The first phase of the redesign at the end of SY included the Lincoln, Madison, Pinkston, and South Oak Cliff feeder patterns (Cohort 1). In , the Carter, Samuell, North Dallas, and Wilmer Hutchins feeder patterns (Cohort 2) were added. The AASI had a budget of $503,946 Title I, Part A funding for SY All schools (N = 70) in the eight feeder patterns, with the exception of magnet schools, were eligible to receive AASI funds. The AASI has four performance goals: 1) Decrease the achievement gaps on State of Texas Assessments of Academic Readiness (STAAR) between African American students in targeted feeder patterns and statewide white students by 25 percent in reading and mathematics. 2) Increase graduation rate by three percentage points. 3) Increase ACT scores by 1.5 points 4) Increase SAT scores by 120 points. Baseline data for the four performance goals were established using data from SY (cohort 1) and SY (cohort 2). Cohort 1 feeder patterns are expected to achieve the goals by the end of SY , whereas cohort 2 feeder patterns are At-a-Glance expected to reach the same goals a year later, end of SY The AASI proposed the following main components for SY : focused action planning, professional development, student advocacy and case management, parent academies, and academic enrichment camps. Purpose and Methods of the Evaluation The focus of AASI evaluation was to 1) describe the characteristics of AASI participants and program activities; 2) examine the degree to which the various components of the initiative were implemented; 3) explore stakeholders perception of the professional development they received on Culturally Responsive Teaching and Learning Strategies (CLRT); and 4) provide a status update on current achievement gaps. This evaluation was primarily based on review of available documents submitted by participating campuses and personal communication with AASI staff. Post-training survey data from educators who participated in one or more of the professional development sessions on CLRT were also used to supplement data from document reviews. Posttraining surveys solicited training participants feedback on: 1) the quality, relevance, and usefulness of the trainings; 2) their level of understanding of the basic framework of CLRT; 3) the degree to which they applied the knowledge and skill-set they gained from the trainings; 4) any obstacles or barriers they encountered during implementation; and 5) additional resources needed. Implementation Since the redesign of the initiative a total of seven professional development sessions on Culturally and Linguistically Responsive Teaching and Learning (CLRT) were conducted by Dr. Sharroky Hollie, executive director of the Center for Culturally Responsive Teaching and Learning. Overall, 428 individuals participated in at least one of the seven training sessions. The initiative employed a trainthe-trainer model in SY , and campuslevel training data submitted to the evaluator indicated that a total of 872 school staff were introduced to the CLRT framework at the school 51

60 level by Campus Instructional Coaches and Academic Facilitators. District- and campus- level training sessions received very positive feedback from participants. The majority of respondents believed changes to current approaches to teaching African-American students are necessary and indicated that, training sessions were timely, specific and content-focused, with clearly stated objectives, and relevant to their work. Generally, responses indicated while the majority of participants have the general understanding of CLRT, they nevertheless lacked the depth of knowledge required for everyday application of the framework. Campus-level training participants particularly requested additional training sessions and materials. Overall, respondents suggested they may benefit from refresher training, but more importantly, they indicated that a more in-depth and practical training on how to apply the CLRT concepts in a classroom setting is essential. Review of available campus action plans indicated that key actions related to improving the achievement of African-American students (particularly literacy) and providing support to the implementation of professional development on CLRT and student advocacy and case management. Indicators of success associated with the key actions were appropriate, specific, and measurable. According to available documents on the student advocacy and case management component of AASI, at least 850 students were identified for the mentorship component. On average, mentors scheduled seven meetings to discuss various topics or issues including academic or STAAR prep/tutoring, college/career prep, leadership, problem solving/critical thinking, etiquette/social skills/character building, positive self-image, and communication. Activities or field experiences included trips to local, museums, libraries, or the arboretum. Types of service projects were mostly comprised of the following: volunteer work at homeless shelters, nursing homes, and church; campus beautification projects; food drives and fundraisers; and community clean-up activities. Available documents revealed that only two AASI sponsored parent academy workshops were conducted by AASI personnel and Principal Fellows in two feeder patterns during SY Forty-nine parents completed post-workshop questionnaires and their responses were generally very positive. Responses from the 25 principals who completed the survey indicated that more than half did not provide any AASI parent workshops during the school year, suggesting this component of the initiative was not implemented as planned. Academic Enrichment Camps were implemented in four of the eight feeder patterns in June These eight-day Enrichment Camps targeted at-risk students at the elementary and high school levels. Participating elementary teachers received a twoday pre-camp training as well as half day training and planning sessions during the eight-day enrichment camps. A total of 369 elementary and 28 high school students were invited to camp from various campuses but close to a third (30.6%, N = 113) of invited elementary school students did not attend camp. Overall, attendance for summer camps was better in the two high schools compared to the six elementary schools. Summary While implementation of AASI during SY appeared to be significantly better compared to the previous year, the AASI can benefit from improved levels of implementation, monitoring, and proper and timely documentation. Due to the late start to implementation of AASI during SY , fidelity of implementation of nearly all components of the initiative were significantly and negatively affected. This is in part reflected by the proportion of the available federal funding that was spent during the school year (24.9% or $125,435.77). Feedback from relevant stakeholders also indicated that the late notice and start of the initiative greatly hampered their ability to plan ahead, allocate resources, and implement various components of the initiative. Furthermore, fragmentation of responsibility for the implementation, monitoring, and documentation of AASI activities for the majority of the school year hampered the evaluator s ability to document and assess implementation fidelity. Relevant implementation documents were not available for all participating campuses. Moreover, some available documentation for various components of the initiative were incomplete or otherwise unusable for evaluation purposes. As a result, the evaluator cannot, with any degree of confidence, speak to the fidelity of implementation of many of the components of AASI. Readers are cautioned to reserve judgment on the potential impact of the AASI program until the program has been fully implemented for at least two consecutive academic years and rigorous analysis of implementation and achievement data conducted. Recommendations The workscope indicates that the AASI plans to continue with only two of the main 52

61 components from SY , professional development and student advocacy. A District symposium on closing the Achievement Gap is additionally proposed. In light of this information, the following relevant recommendations are offered for ongoing implementation, monitoring, and evaluation of the AASI. Clearly and effectively communicate the goals, objectives, and expectations of the AASI to all relevant stakeholders. Be sure administrators are notified early to allow participating campuses enough time to plan an implementation strategy. Be sure administrators are strong advocates of the initiative so they can set expectations prior to the school year. Begin implementing all components of AASI at the start of the school year. Student advocates, for example, should be identified early so they can begin working with students at-risk of dropping out earlier during the school year. Ensure student advocates have adequate training to provide support to at-risk students. Design differentiated training sessions for CICs and AFs new to CLRT training and others who already have attended one or more sessions prior to the upcoming school year. Future training sessions on CLRT for those with previous exposure to the framework need to focus primarily on deeper understanding and the application of the framework. Make sure campus-level trainers have resources necessary to provide adequate training to teachers. It may be prudent to conduct one or two focus groups with CICs and AFs to gather a more nuanced feedback on the type of support they require. Ensure implementation of all components of the initiative are electronically documented. Ensure the program evaluator has timely and on-going access to implementation documents. Without full and timely access to relevant documents a meaningful evaluation cannot be conducted. For more information, please refer to the full report EA found at 53

62 54

63 Dorothea Weir, Ph. D. The Dallas ISD On-Track Indicator, At-a-Glance EA Ahigh school diploma is rapidly becoming the minimum qualification for most occupations in the American economy. Consequently, addressing graduation and dropout rates has taken on greater urgency in school districts across the country. The first year of high school is increasingly recognized as a transition period that may be make-or-break for many students in terms of eventual high school success and graduation. The Consortium on Chicago School Research (CCSR) developed an On-Track indicator in 1999 to gauge whether students in their first year of high school were making enough progress to have a reasonable expectation of graduating in four years. Unlike other indicators, it provides information about progress and performance from a relatively early point in a student s high school career. Dallas ISD developed its own On-Track indicator in From through , the Dallas On-Track Indicator was incorporated into the Principal Appraisal System. Like the Chicago indicator, the Dallas indicator combines two factors: the number of credits earned during freshman year and the number of semester Fs in core subjects. A student is considered to be On- Track at the end of ninth grade if he or she has met both of the following criteria: The student has accumulated five full course credits, and The student has no more than one semester F (that is, one-half of a full credit) in a core subject (English, math, science, or social studies). Methodology The model was developed using data for students entering high school for the first time in The cohort consisted of 8,008 first-time, regular education ninth graders who were enrolled for the full school year. The cohort was 35% African American, 58% Hispanic, and 16% LEP. Analyses were conducted in June, 2008, to construct a baseline measure districtwide. The analysis was repeated annually in subsequent years. Only nonspecial-education grade nine students who were enrolled in the same school for both fall and spring semesters were included. Results were calculated only for schools receiving an accountability rating; alternative schools were excluded from the analyses. In , On-Track documentation was improved, as registrars at all high schools were asked to provide prior grade-level information for all students entering ninth grade from outside the district. District forms were changed to require future collection of this information for all entering students at the time of enrollment. In , new state testing required identification of all first-time ninth grade students, and this information was made available through the Chancery student data system. Results Figure 1 shows freshman On-Track percentages for all district high schools, for all schools other than magnets (i.e., comprehensive and Early College high schools), and for magnets only for the through school years. The On-Track rate dropped considerably in , increased slowly through , and decreased slightly in Total Non-Magnet Magnet Figure 1. Percent of first-time ninth grade students On-Track, 2009 through Figure 2 shows On-Track percentages by student ethnicity. In , the districtwide On-Track rate of African American students was 73.6%; Hispanic students, 72.8%; and White students, 90.4%. Historically, the On-Track rate of all ethnic subgroups fell in , although the rates of African American and White students fell more sharply than that of Hispanic students. The On-Track rate of White 55

64 students has increased gradually each year since The rates for African American and Hispanic students increased from 2010 through 2013, but in 2014 fell slightly below 2009 rates. African American Hispanic White Figure 2. Percent of first time ninth-grade students On-Track by ethnicity, Table 1 shows On-Track percentages by school and school group for the through school years, and the percentage change over the three-year period. As a group, comprehensive high schools had an On-Track rate in of 72.0%. In 2009, the first year in which the indicator was used, all non-magnet high schools had On-Track percentages of 60% or more. In 2014, four schools (Molina, Pinkston, Roosevelt, and Spruce) fell below 60%, and five more below 70% (Bryan Adams, Conrad, South Oak Cliff, Pinkston, and White). Over the period from 2012 through 2014, three schools had On-Track increases of 10 percentage points or more; these were Kimball, South Oak Cliff, and Gilliam Academy. Schools with decreases of more than 10 percentage points were Conrad, Madison, Molina, Roosevelt, Spruce, and Wilmer-Hutchins. Magnet high school On-Track rates changed only 1.4 percentage points, from 88.7% in 2012 to 90.1% in Rangel Young Women s Leadership Academy and the Talented and Gifted Magnet at Townview both had On-Track rates of 100% for the second and third consecutive years, respectively. The largest increase was at Gilliam Collegiate Academy, which improved its On-Track rate by 30.8 percentage points. Table 1 Freshman On-Track Percentage by School, 2012 through 2014 Percent On-Track School Change Comprehensive High Schools B. Adams Adamson Carter Conrad Hillcrest Jefferson Kimball Lincoln Madison Molina N. Dallas Pinkston Roosevelt Samuell Seagoville Skyline S. Oak Cliff Spruce Sunset White Wilson Wilmer-Hutchins Garza Early College Gilliam Collegiate Academy Lassiter Early College Comprehensive total Magnet High Schools Arts Magnet Obama HS Rangel HS Smith New Tech Townview Business Townview ESSM Townview Health Townview Law Townview SEM Townview TAG Magnet total Recommendations District On-Track rates have varied little since 2010, despite large year-to-year differences at some campuses rates for comprehensive and magnet school groups, and for the district overall, were marginally lower than in One factor influencing recent On-Track rates may be state accountability requirements for End-Of-Course (EOC) tests in core courses, which fall most heavily on ninth-grade students. Progress of freshman cohorts since should be examined to determine whether new measures of course completion and EOC performance should be incorporated, as well as to examine the reasons for large fluctuations at specific high schools. For more information, contact Program Evaluation at

65 The Summer Learning Program, Nolan Rett Mickelson EA At-a-Glance The Dallas Independent School District s (Dallas ISD) Department of Summer Learning and Extended-Day Services provided summer programming for thousands of elementary, middle, and high school students across the district. The Thriving Minds Summer Camp, a partnership between the district and Big Thought (a non-profit organization devoted to integrating core-subject curriculum with artistic exploration), extended traditional and bilingual instruction to elementary students at eight sites through core curriculum integrated with studies in the arts. The summer of 2014 marked the fifth year implementing this program. It began district-wide in 2010 and has served between 5 and 10 sites each summer. During 2013 and 2014, the Thriving Minds Summer Camp hosted the National Summer Learning Demonstration Study and Project. The RAND Corporation conducted the study with funding from the Wallace Foundation. The Dallas ISD provided a remedial Summer Learning Program for elementary students at three traditional elementary sites and one Summer Learning Lab. The learning lab did not differ from the traditional sites in either duration or curriculum. Teachers at these sites received additional observations, coaching, and professional development. Six traditional middle schools and one alternative school provided credit-recovery in core courses; three additional middle schools hosted Summer Learning Labs. Junior Players, a non-profit organization promoting creative expression particularly through drama, provided enrichment courses at two of the learning lab campuses. The High School Summer Learning Program remained similar to previous years and offered opportunities for credit recovery and test preparation at nine campuses and one alternative school. Several other enrichment programs were offered across the district, focusing on various ages and needs. These programs presented opportunities in music, dance, science, and karate among other subjects. Elementary School Summer Learning The district provided three elementary extendedyear programs in summer 2014 The district partnered with Big Thought to provide the Thriving Minds Elementary Summer Camp at eight elementary schools. These campuses were identified from an existing partnership with Big Thought for after-school programming. Three elementary schools provided the Traditional Elementary Summer Learning program and were managed by the district. One campus Kennedy hosted a Summer Learning Lab, which afforded teachers one-to-one coaching for the entire day and professional development every afternoon. Students at Kennedy attended Monday through Thursday from 8:00 to 12:00 while teachers stayed until 3:00. The traditional sites operated only from 8:00 to 12:00. The Thriving Minds program operated Monday through Friday from 8:00 to 3:45. Teachers delivered instruction in core classes in the morning and community artists delivered arts enrichment programming in the afternoons. Evaluation staff observed classrooms across all grade levels at 7 Traditional Elementary program locations, 5 Thriving Minds Summer Camp locations, and the Summer Learning Lab during the third week of the programs, visiting 97 classrooms in all. Evaluators took notes in the following areas: grade level, subject, number of teachers and students, classroom setting, class lesson and instruction, and use of technology in lesson. These observations revealed that the traditional sites averaged 9.2 students per class, Thriving Minds sites averaged 10.4 per class, and the learning lab at Kennedy averaged 12.4 students per class. Each learning lab class had one teacher paired with one Instructional Coach. Further, the instruction in learning lab classrooms was likely to have students working in pairs or groups of three or more and use technology such as laptops and projectors. Nearly one-fourth (24.5%) of observed classrooms at traditional sites were not observed in a lesson. In general, evaluators reported that these classes tasks were to complete a worksheet or read a book or text. In most of these classes, the students sat and did nothing. Teachers were not instructing and students were not producing. However, among the three programs observed classrooms, the depth of instruction increased from the previous year. The current year s instruction demonstrated activities that illustrated subjects in ways that students could easily recognize, relate, become engrossed in, and expand upon. 57

66 Exactly 981 students attended at least one day of the Traditional Elementary Summer Learning Program; 3,702 students attended the Thriving Minds Summer Camp, and 371 students attended the Kennedy learning lab. Across all types of programs 3,606 students were recommended to attend because they were at risk for retention. To be promoted, these students must have attended at least 90 percent of the instructional days. At Thriving Minds sites, 65 percent of the recommended students met this criterion (see Figure 1). The percentages were lower at the traditional sites (45%) and the learning lab (56%). The Thriving Minds Camp lasted four more days than the traditional summer school. Figure 1: Percent of Recommended Students Meeting and not Meeting the 90% Attendance Criterion for Promotion Middle School Summer Learning District middle school students could attend one of two types of credit-recovery programs during the summer: traditional or Summer Learning Lab. As with the elementary learning lab, the duration and curriculum did not differ between the traditional and learning lab programs for students. The teachers at Dade, Edison, and Medrano received additional coaching and professional development. These 3 sites, 6 conventional middle schools and LACEY, an alternative school, provided instruction from June 30 to July 30. Classes began at 9:00 and ended at 1:00. Students attended enrichment courses from 1:00 to 3:00 at Dade and Edison. Junior Players, a non-profit organization promoting artistic expression, provided courses in various subjects, including: the performing arts, dance, modeling, business, and art. The Summer Learning Program was offered to qualifying students in grade 6 through 8. Exactly 2,905 students accounted for 5,786 enrollments in 12 courses. Eighth-grade students attended fewer days than seventh- and sixth-grade students, on average. Learning lab students attended at lower rates than students at traditional sites. The course passing rates were comparable among the two middle school programs; however, students in the sixth grade at learning labs passed their courses at higher rates than their traditional counterparts. Also, students at learning labs who did not meet the 90 percent attendance criterion to pass the course tended to have high proportions of passing rates. Nearly 47 percent of eighth-grade language arts students, for example, did not attend enough days to pass, but received passing scores. Every U.S. Studies 8 student at the learning labs passed the course, regardless of attendance. Evaluator observations during the third week of the middle school summer programs indicated that the traditional sites averaged more students per class (M = 14.4) than the Dade and Edison learning labs (M = 10.7) and the Medrano learning lab (M = 12.8). In nearly 71 percent of the classrooms across all middle school programs, students were working alone. But, students were working in pairs or groups in over 60 percent of the Medrano learning lab classes. Dade and Edison, as well as the traditional classrooms, were more likely to have most to all students engaged in learning. At Medrano, where a teacher and Instructional Coach were paired in every classroom, only half of the students were engaged in 60 percent of the classrooms observed. Almost 74 percent of the middle school classrooms experienced classroom management problems, whether they were isolated to a few students or pervasive across the entire class. In traditional sites, the problems were significant enough to disrupt the lesson in 71 percent of the observations. High School Summer Learning The Dallas ISD s High School Summer Learning Program primarily provided credit recovery at 9 high schools and 1 alternative campus from June 30 to July 30. Classes were held Monday through Thursday from 8:00 to 12:00. The program was intended to allow students in grades 9-12 an opportunity to recover credit for courses not successfully completed during the academic year. To obtain credit for a course, the student must have attended at least 90 percent of the instructional days and received a course grade of at least 70 percent. 58

67 A team of evaluators visited five high school campuses during the third week of the program. Observation protocols were the same as for elementary and middle school sites. Hillcrest High School had the highest average observed class size with 14.4 students per class. The average observed class size at Madison High School was 4.3. Madison had the smallest enrollment overall. Indeed, when the evaluator visited Madison, three classes only had one student. The overall average for the observed high schools was 8.7 students per class. Over 63 percent of the summer middle school classrooms had students who were working alone. Notes from observers indicated that students working alone were usually quietly completing assigned problems from a workbook. While the students worked independently, they were engaged in the task. Exactly 86.8 percent of the high school classrooms had most or all students engaged. Indeed, observation notes described students as focused while completing the assigned work, but doing nothing once that work was completed. The evaluator observed several classes where the learning expectations were very low and tasks accomplished very easily. Once completed, the students and teacher sat in the rooms and chatted. Precisely 3,329 high school students attended the Summer Learning Program at one of 9 sites and 1 alternative campus. At every grade level, 50 percent or less of the students met the 90 percent attendance criterion for passing a course. Students enrolled in over 5,368 courses. Low passing rates were found in both Pre-Calculus courses (66.7% and 76.7%, respectively). Every student enrolled in the English IV courses passed despite over 33 percent of the English IV 1 and 48 percent of English IV 2 students not meeting the attendance criterion. Recommendations Enrolling students recommended for retention was the responsibility of the students home school staffs. Despite the Department of Summer Learning s clear and early communication to the home school principals, counselors, and data controllers that all students at risk for failure or retention be enrolled by May 23 rd, this responsibility was ignored. Recommended students should be enrolled earlier. Waiting until the last few days of school is confusing for all involved: students, parents, home schools, summer sites, and district administrators. Moving forward, this rule must be enforced. Nearly all of the middle students attended the summer session because they failed a course during the regular school year: The instruction delivered during the regular school year was not sufficient for these students. For these struggling students to learn and achieve at higher rates, the instruction during the summer must be of higher quality than during the regular school year. Despite improvements in the quality of instruction from previous years, the lessons offered little in the way of activities that students could engage with authentically. The Summer Learning Program is short in duration and every instructional day weighs heavily. The more involved instruction at the Summer Learning Labs sites than at traditional sites due to the influence of Academic Facilitators, Instructional Coaches, and professional development indicates that this type instruction and engagement can be achieved with some investment guiding teachers during the summer. The evaluator recommends the expansion of summer learning programs that include direct teacher coaching and professional development to more sites across the district. Taking the learning lab concept to high school sites would increase the rigor and quality of instruction at that level. For more information, contact Program Evaluation at (972) Information about other Dallas ISD summer learning programs may be found in EA and EA

68 60

69 The Summer Learning Lab Program: Amber McEnturff and Nolan Rett Mickelson EA At-a-Glance The Summer Learning Lab (SLL) Program partnered teachers with instructional coaches to build teachers instructional capacity. SLL was part of the Dallas Independent School District s (Dallas ISD) plan to support teachers professional growth as part of the Teacher Excellence Initiative (TEI). SLL was implemented during the 2014 Summer Learning Program in three middle schools (Dade, Edison, and Medrano) and one elementary school (Kennedy). The two key components of SLL were instructional coaching and professional development (PD). Each campus had student instruction in the morning, PD in the afternoon, and coaching at various times during the day. The combination of coaching and PD was intended to make SLL an intensive period of professional growth for teachers. The purpose of this report is to describe implementation of SLL and outcomes for SLL participants. 1 Methodology To summarize participant characteristics, frequency analyses were conducted using information from the SLL staff rosters and the district personnel database. Information about SLL program implementation came from observations of classroom instruction and professional development sessions, and interviews with the four summer learning principals. Teachers and coaches also responded to an online survey about SLL, including their perceptions of the coaching and professional development offered. Percentages reported from the survey were based on the number of participants providing a valid answer, omitting those who skipped the question or selected I don t know. Limitations to this evaluation included (a) potential bias in survey responses, (b) differences in implementation, which impeded comparability of survey results between campuses, and (c) small numbers of coaches at Dade (n = 3) and Edison (n = 2), which led to the omission of demographics for these coaches to protect their confidentiality in this report. Participants According to staff rosters, there were 85 teachers at SLL campuses, including 18 at Dade, 20 at Edison, 21 at Medrano, and 26 at Kennedy. Table 1 summarizes teacher demographic characteristics. There were 40 coaches at SLL campuses, including three at Dade, two 1 SLL participants included teachers and coaches. More information about student outcomes and classroom observations can be found in EA at Edison, 12 at Medrano, and 23 at Kennedy. Table 2 summarizes coach characteristics for Medrano and Kennedy. Table 1: Teacher Demographic Characteristics Dade (n = 18) Edison (n = 20) Medrano (n = 21) Kennedy (n = 26) Race/Ethnicity Asian 0% 0% 0% 4% AA 83% 50% 52% 12% Hispanic 6% 5% 10% 31% White 0% 15% 19% 8% Not reported 11% 30% 19% 46% Gender Female 78% 80% 57% 77% Male 22% 20% 43% 23% Highest Degree Completed BA/BS 72% 65% 24% 62% MA/MS 17% 25% 38% 27% PhD 6% 0% 0% 4% Not reported 6% 10% 38% 8% Alternative Certification Intern Yes 89% 70% 86% 77% No 11% 30% 14% 23% Taught at Home Campus during Summer Learning Yes 44% 20% 33% 27% No 56% 80% 67% 73% Note: Some percentages do not total 100 due to rounding. Table 2: Coach Demographic Characteristics Medrano (n = 12) Kennedy (n = 23) Race/Ethnicity Asian 0% 4% AA 42% 65% Hispanic 8% 22% White 25% 4% Not reported 25% 4% Gender Female 83% 91% Male 17% 9% Highest Degree Completed BA/BS 42% 35% MA/MS 42% 52% PhD 0% 0% Not reported 17% 13% Coached at Home Campus during Summer Learning Yes 17% 4% No 83% 96% Note: Coach refers to campus instructional coaches (CICs), master teachers, or academic facilitators. Coach demographics are not reported for Dade (n = 3) or Edison (n = 2) due to small number of respondents. Some percentages do not total 100 due to rounding. 61

70 Of the 85 SLL teachers, 75 (88%) completed the survey. Of the 40 SLL coaches, 32 (80%) completed the survey. Table 3 shows the average years of experience teaching and coaching for SLL participants based on teacher and coach survey responses. Teachers at Medrano reported roughly the same number of years of teaching experience (10.2 years) as coaches (9.3 years). Coaches at Kennedy reported three times the number of years of teaching experience (15.7 years) than teachers (5.0 years). Comparisons were not reported for Dade and Edison due to small numbers of coaches. Table 3: Years of Experience for Teachers and Coaches Dade Edison Medrano Kennedy Teacher (n = 14) (n = 17) (n = 21) (n = 18) Years of teaching 7.6 (6.4) 7.5 (8.4) 10.2 (7.4) experience 5.0 (4.7) Coach (n = 3*) (n = 1*) (n = 10) (n = 15) Years of teaching experience (4.5) 15.7 (8.5) Years of coaching experience (2.7) 2.4 (2.6) Note: Cells include mean years of experience with standard deviations in parentheses. *Coach demographics are not reported for Dade (n = 3) Edison (n = 1) due to small number of respondents. It is unclear if years of coaching and teaching experience overlapped for Medrano and Kennedy coaches. Implementation Hiring and Leveling. For all four campuses, the Teaching and Learning Department hired principals and coaches for SLL. SLL principals hired SLL teachers for their campuses. SLL principals located potential participants by asking other principals to identify teachers that wanted professional growth. They also accessed Summer Learning applicants from Oracle but had to ensure applicants knew SLL was different from traditional Summer Learning before considering them for hire. At Dade and Edison, teacher hiring targeted teachers at those schools for the summer session. However, once hiring began, many of the teachers had already scheduled other professional development or vacations which would conflict with teaching the Summer Learning Program. The summer principals then turned to other Oracle applications for potential summer teachers. Teachers from outside those schools were hired because the summer principals believed they were the best candidates for the teaching positions, not because they either needed or desired professional development. Principals related that the teachers that taught at Dade and Edison during the regular school year applied because they thought that they would benefit from teaching during summer and receiving PD. According to open-ended responses on the survey, about 36 percent of teachers on all four campuses decided to participate in SLL for the professional growth opportunity. About 22 percent applied to teach during Summer Learning, not specifically for SLL, but still elected to participate. The rest of teachers were motivated to participate by a principal, instructional coach, or colleague referral, the opportunity to help students improve, or the extra pay SLL teachers received because the work day was longer than traditional Summer Learning. Coaching. According to SLL documentation from Teaching and Learning, coaches were instructed to use an I do, we do, you do co-teaching model. The model describes a gradual release of instructional responsibility, where the coach models instruction ( I do ), the coach co-teaches with the progressing teacher ( we do ), and eventually, the progressing teacher teaches independently ( you do ). The actual implementation of coaching varied by campus. According to teacher survey responses, teachers at Kennedy had the highest percentage (89%) reporting that the I do, we do, you do co-teaching model was used, followed by Edison (69%), Dade (57%), and Medrano (57%). Coaches at Medrano and Kennedy were originally intended to work in the classroom alongside teachers in a 1:1 ratio throughout SLL. Kennedy had the highest model fidelity when it came to the 1:1 coach to teacher ratio. All teachers had 1:1 coaching except a few bilingual teachers who had a 1:2 coach to teacher ratio because there were not enough bilingual coaches. At Medrano, student enrollment was higher than expected. While new teachers were recruited to accommodate enrollment, coaches taught class during the first week of summer learning. Then, coaches were assigned to groups of teachers in 1:3 or 1:4 ratios. Dade and Edison coaches were assigned to the campus at large, not to individual teachers. For Dade and Edison, coaches were in charge of afternoon professional development and were less frequently in the classroom providing support and coaching to individual teachers. At Dade, coach observations took place at least once weekly. At Edison, conversations with teachers indicated that some teachers were never observed by a coach. Other teachers reported that coaches observed only in conjunction with the summer principal s spot observations. Additionally, Dade and Edison coaches were only on campus during the first two weeks of summer school. Therefore, a key difference in implementation between the campuses was the coachto-teacher ratio and time spent in classrooms by coaches. While observing classrooms, an evaluator noticed coach/teacher pairs teaching small groups of students separately. This was an unintended use of the 1:1 coaching ratio. On the survey, teachers reported the approximate percentage of instruction time with the coach spent teaching separate groups. A higher percentage of Kennedy teachers reported spending at least 50 percent of instructional time with the coach teaching separate groups (61%), followed by Dade (46%), Edison (44%), and Medrano (24%). According to principal interviews, the role of coaches was not clear to SLL staff, which may have led to the unintended use of separate group instruction. 62

71 Teachers indicated their level of satisfaction with the coaching they received during the traditional school year and during SLL. Table 4 shows the percentage of teachers selecting either satisfied or very satisfied with coaching. Nearly double the percentage of Kennedy teachers reported satisfaction with SLL coaching (83%) compared with school year coaching (44%), while percentages for Dade, Edison, and Medrano were more similar. Table 4: Teachers Satisfied With School Year versus SLL Coaching Dade Edison Medrano Kennedy (n = 13) (n = 15) (n = 19) (n = 18) Satisfied with school year coaching 77% 60% 53% 44% (n = 13) (n = 14) (n = 19) (n = 18) Satisfied with SLL coaching 85% 50% 58% 83% Note: Compare percentages for Edison with caution due to differing number of respondents. Coaches also reported benefiting from SLL participation. According to open-ended responses on the survey, coaches reported increasing their coaching capacity through collaboration with other coaches, frequent interactions with teachers, and leading professional development sessions. Two coaches stated that SLL helped them understand the new teacher evaluation system under TEI. Collaboration. Teachers had the opportunity for three general types of collaboration, including one-on-one with a coach, with a coach and a group of teachers, and with a group of teachers alone. Table 5 shows the percentage of teachers who reported frequencies of at least once per day for each type of collaboration. A greater percentage of teachers at Kennedy reported receiving one-on-one collaboration time with a coach at least once per day (76%) than the other campuses, which was expected since most Kennedy teachers had a 1:1 coaching ratio. Table 5: Percentage of Teachers Collaborating At Least Once Per Day Dade (n = 13) Edison (n = 15) Medrano (n = 21) Kennedy (n = 17) One-on-one collaboration 23% 20% 38% 76% with coach Collaboration with coach and 31% 40% 38% 47% other teachers Collaboration with other teachers without a coach 31% 40% 33% 41% Furthermore, teachers were asked to select which of the three types of collaboration was most helpful. Table 6 shows the percentage of teachers selecting each type of collaboration as the most helpful. In general, teachers at Dade preferred collaboration with coach and other teachers (42%), teachers at Medrano (57%) and Kennedy (63%) preferred one-on-one collaboration with a coach, and teachers at Edison were split between preferring one-on-one collaboration with a coach (40%) and collaboration with other teachers without a coach (40%). Table 6: Percentage of Teachers Reporting the Type of Collaboration That Was Most Helpful Dade (n = 12) Edison (n = 15) Medrano (n = 21) Kennedy (n = 16) One-on-one collaboration 25% 40% 57% 63% with coach Collaboration with coach and 42% 20% 24% 31% other teachers Collaboration with other teachers without a coach 33% 40% 19% 6% Note: Most helpful components for each campus are highlighted in green. Professional Development. Professional development (PD) for all four SLL campuses was developed by the Teaching and Learning Department. Coaches administering the PD accessed the modules via Google Drive and made any desired edits. PD at Kennedy consisted of six one-hour sessions and hour sessions. Five were facilitated by reading/language arts campus instructional coaches (CICs), one by a guest facilitator, and the rest by SLL coaches. Kennedy PD topics included classroom environment, classroom management, instructional strategies, TEKS, routines and transitions, and student engagement. PD at Kennedy was primarily based on Domains Two and Three of the Teacher Excellence Initiative (TEI). According to the TEI Teacher Guidebook, Domain Two, instructional practice, includes standards such as establishing lesson objectives, measuring student mastery through a demonstration of learning, and engaging students. Domain Three, classroom culture, includes standards such as maximizing instructional time and maintaining high student motivation. In addition to classroom instruction time and formal PD, Kennedy teachers were given time daily to reflect and plan with their coaches. Most teachers at Kennedy viewed the PD favorably and thought it was helpful that they could immediately implement the new strategies and receive feedback the following day. PD criticisms from Kennedy teachers included that sometimes too much information was presented in a short amount of time or that some information could have been conveyed via rather than formal PD. PD at all three middle schools was based on the book Engaging Students with Poverty in Mind (Jensen, 2013). Middle school PD topics included seven engagement factors, setting the atmosphere, motivating through connection, five rules of engagement, and building cognitive capacity. At Medrano, PD was delivered to all teachers on Monday afternoons, coach and teacher 63

72 groups planned and implemented the PD Tuesday and Wednesday afternoons, and all teachers and coaches evaluated and reflected on student artifacts and PD together on Thursday afternoons. Most Medrano teachers found the PD helpful; one stated PD was helpful but not as helpful as working with the instructional coach in the classroom. Teachers received formal PD at Dade and Edison daily. At the conclusion of each day s PD session during the last two weeks, the Dade and Edison principals allowed teachers 15 to 30 minutes to plan for the next day s lessons. Teachers at Dade and Edison reported positively about the PD received from the coaches during the first two weeks, especially when it used artifacts and specific instances from their classes that day. The principals at those learning labs, however, felt that they did not have enough time to thoroughly plan for each day s PD session. In particular, the Edison principal related that the expectations for the PD during the last two weeks were never clearly communicated to her by either the Department of Summer Learning and Extended Day Services or the Teaching and Learning Department. Indeed, one evaluator observed a PD session at Edison during the next to last week which consisted of a 30- minute activity about the district s core beliefs followed by a series of video clips with little to no discussion of each clip s relevance. The principals offered some insight as to why PD implementation suffered at Edison: The Dade summer principal was the assistant principal during the regular school year and was periodically informed about the learning lab program by Dade s regular school year principal throughout the program s planning process. The Edison summer principal received no such updates and, as she stated, received very little communication about the program s goals and her role as principal in an SLL prior to the first days of the summer session. The Dade and Edison programs also hosted enrichment provided by the Junior Players in the afternoons while teachers were in PD sessions. During the last two weeks, after the coaches had left campus, the summer administration delivered PD. Principals reported that, although Junior Players should have handled any discipline matters with students, they felt it was their duty to deal with those concerns. Oftentimes this involved departing in the middle of a PD session, leaving other administrators to improvise the PD without preparing. Other times principals reported giving teachers a quick reading assignment to pass time until the principal could return. Taking on these tasks created a conflict of roles for the summer administrators at the sites with enrichment in the afternoons. Teachers rated the effectiveness of PD in aiding improvement in each of the TEI standards in Domains Two and Three. The effectiveness of SLL PD in these areas was summarized by averaging the percentage of teachers selecting each effectiveness rating across all items within each TEI domain. The results are shown in Table 7. Ninety percent or more of teachers in all four schools rated PD as effective or very effective in helping them improve in both Domains Two and Three of TEI. Table 7: Average Percentage of Teachers by School Indicating Levels of Effectiveness of PD in TEI Domains Two and Three Domain Two: Instructional Practice Dade Edison Medrano Kennedy Very ineffective 7% 1% 0% 0% Ineffective 0% 1% 3% 0% Neutral 3% 7% 7% 2% Effective 21% 39% 46% 14% Very effective 69% 52% 46% 84% Domain Three: Classroom Culture Dade Edison Medrano Kennedy Very ineffective 7% 0% 0% 0% Ineffective 0% 1% 1% 0% Neutral 0% 4% 7% 3% Effective 22% 40% 49% 16% Very effective 72% 54% 43% 81% Note: Number of respondents per question varied as follows: Dade (n = 15), Edison (n = 17), Medrano (n = 20 to 21), Kennedy (n = 17 to 18). Some percentages do not total 100 due to rounding. General Questions. Survey participants answered general questions about their overall experience with SLL. Participants were asked to rank three SLL components (coaching, PD, and gaining teaching experience) from most helpful to least helpful. Gaining teaching experience was included to account for the possibility that any additional teaching practice was helpful regardless of SLL-specific PD and coaching. Table 8 shows the highest percentage of teachers at Dade (50%) and Edison (56%) found gaining teaching experience to be the most helpful part of SLL, while teachers at Medrano (43%) and Kennedy (50%) thought coaching was the most helpful. The lowest percentage of teachers at Dade (14%) thought coaching was the most helpful, while the lowest percentage of teachers at Edison (19%) thought PD was most helpful. Teachers at Medrano (24%) and Kennedy (22%) were least likely to select teaching as the most helpful component. Table 8: Percentage of Teachers Selecting SLL Component As Most Helpful School Highest % Middle % Lowest % Dade Teaching (50%) PD (36%) Coaching (14%) Edison Teaching (56%) Coaching (25%) PD (19%) Medrano Coaching (43%) PD (33%) Teaching (24%) Kennedy Coaching (50%) PD (29%) Teaching (22%) Note: Number of respondents per question varied as follows: Dade (n = 14), Edison (n = 16), Medrano (n = 21), Kennedy (n = 17 to 18). Figure 1 shows the percentage of teachers that selected agree or strongly agree when asked if they received the support needed to be a successful teacher during the traditional school year and SLL. In all four SLL campuses, teachers felt more supported during SLL than 64

73 they did during the traditional school year. This is important given the TEI goal to build an effective support system for teachers. Figure 1: Percentage of Teachers Receiving Support Needed During the Traditional School Year versus SLL Dade (n = 14) Edison (n = 17) Medrano (n = 21) Kennedy (n = 18) Traditional Year Support 50% 64% 67% Teachers rated their agreement with four statements about SLL overall, shown in Table 9. Survey results showed that 100 percent of teachers at Dade and Kennedy agreed that their teaching improved during SLL, compared with 71 percent at Edison and 90 percent at Medrano. The greatest percentage of teachers at Dade (92%) agreed that coaching helped them improve, while only 50 percent of teachers at Edison thought coaching helped. The lowest percentage of teachers at Edison M (81%) felt that PD helped them improve. All teachers at Dade and Kennedy would recommend SLL participation, compared with 88 percent of Edison and 81 percent of Medrano teachers. Table 9: Percentage of Teachers Agreeing With the Following Statements Dade Edison Medrano Kennedy Teaching improved during 100% 71% 90% 100% summer school IC helped me improve 92% 50% 81% 78% PD helped me improve 100% 81% 95% 94% I would recommend SLL participation to other teachers 100% 88% 81% 100% Note: Number of respondents per question varied as follows: Dade (n = 13 to 15), Edison (n = 14 to 17), Medrano (n = 21), Kennedy (n = 18). Recommendations 77% 94% 90% 100% 94% SLL Support Designate Summer Learning Lab sites earlier so that principals may be hired earlier. Involve SLL principals in hiring of coaches to help ensure good coach-teacher matches. Moreover, include principals in developing the PD modules, especially if the principals will be delivering the PD themselves. Consider adding PD for instructional coaches. An unintended secondary benefit of SLL was professional development for coaches, suggesting it is possible to use SLL to develop both teachers and coaches. Additionally, classroom observations indicated that the SLL classrooms demonstrated more involved instruction for the students, perhaps due to the influence of coaches and PD. This indicates better instruction and engagement can be achieved with some investment guiding teachers during the summer. The evaluator recommends the expansion of summer learning programs that include direct teacher coaching and professional development to more sites across the district. (See full Summer Learning Program report, EA , referenced below.) Follow-up with participants to determine whether benefits of SLL carried over into school year. References Jensen, E. (2013). Engaging students with poverty in mind: Practical strategies for raising achievement. Alexandria, VA: Association for Supervision & Curriculum Development. TEI Teacher Guidebook (Draft revision August 4, 2014). Downloaded from: The full Summer Learning Program final evaluation report can be found at For more information, contact Program Evaluation at

74 66

75 Title I: Comparative Measures of Student Achievement Robert J. Costello, Ph.D. EA At-a-Glance The purpose of this report was to examine the academic achievement and progress of Title I students. This report contains comparisons of Title I students by grade and divisions, with district and State results. Comparisons were also made by major student subgroups. These subgroups were All Students, African American, Hispanic, white, Economically Disadvantaged, English Language Learners and Special Educations students. In all cases, comparisons were made for a two year period and change scores were calculated. The following variables were used to determine the progress of Title I students: Iowa Tests of Basic Skills (ITBS) reading and mathematics for grades kindergarten, one and two; Logramos (Spanish version of ITBS) reading for grades kindergarten, one and two; State of Texas Assessments of Academic Readiness (STAAR) reading and mathematics for grades 3 through 8, writing grades 4 and 7, science grades 5 and 8 and social studies grade 8. STAAR End-of-Course (EOC) examinations were given primarily in grades 9 and 10. The five EOC examinations were Algebra I, Biology, English I, English II and U.S. History. Student and teacher demographic were also presented, along with attendance, dropout and graduation rates. Program Description The purpose of Title I was to ensure that all students have a fair, equal, and significant opportunity to obtain a high-quality education and reach, at a minimum, proficiency on challenging state academic achievement standards and state academic assessments. The Title I program budget for the school year was $94,381, from Special Revenue Fund. This was an increase of approximately 4 million dollars from the school year ($90,361,970.00). Monies were given to both individual projects and individual schools to be used at the discretion of principals and executive directors. Program Demographics A total of 159,760 students were enrolled in Dallas ISD schools. Hispanic (70%) and African American (23%) students made up the majority of the student population (93%), while white (5%), Asian (1%), and Native American (0.4%) students combined accounted for approximately six percent (6%). Student characteristics showed that 88 percent of Title I students were economically disadvantaged and seven percent were special education students. Approximately 41 percent of the Title I students were classified as English language learners (ELL). ITBS Reading and Mathematics The ITBS Reading and Mathematics tests were administered to kindergarten through second grade students. Kindergarteners maintained a comparable level of performance between 2013 and 2014 on both tests. In contrast, first and second graders had a decline in performance of between four and seven percentage points between the two years on both tests. This decline held true across all subgroups. Logramos Reading Logramos results echoed the ITBS results. Kindergarteners maintained their performance levels between 2013 and 2014 while the performance of first and second graders declined. Again, performance levels decreased for all subgroups in these two grades. STAAR Reading Of the six grades that were required to take the STAAR Reading examination (grades 3-8), five had lower percentages of students with Satisfactory performance when compared to Only sixth grade showed an increase. Among the subgroups in each of the five lower performing grade levels, African Americans, Hispanics, Economically Disadvantaged and ELL students all had decreases in performance. White students decreased in two of the grades (3 and 4) and special education students decreased in three grades (3, 5, and 7). Among sixth graders, all subgroups except African Americans had higher percentages. Of the 42 reading measures that were analyzed (percentages for seven subgroups across six grades), district students had higher performance levels on 11 (26%) of the 42 measures. District performance across all grade levels fell below State levels. The discrepancy was the greatest among fourth graders with a 15 percentage point gap and smallest among eighth graders with an eight point gap. Similar to the district, the State had a lower percentage of students achieving Satisfactory levels compared to ; however, there were eight instances where district measures decreased from 2013 levels while State levels increased. STAAR Mathematics The district as a whole performed better this year relative to All grades except grade 7 had increases in the percentage of students achieving Satisfactory scores. All subgroups within the higher performing grades had increases with the exception of 67

76 whites in third grade and African Americans in eighth grade. All subgroups in grade 7 had lower performances this year. Of the 42 measures analyzed on this examination (percentages for seven subgroups across six grades), 33 (79%) showed improvement from The district had Satisfactory rates below the State rates. The gap between the district and State ranged from 15 percentage points in grades 6 and 8 to 10 percentage points in grade 5. Increases and decreases in the district rates mirrored the changes in State rates except among the two subgroups mentioned above (white third graders and African American eighth grades). STAAR Writing Performance on the STAAR Writing examination decreased this year for both fourth and seventh graders. This was true across all subgroups except white students in both grades and special education students in grade 7. There were three improvements (21%) across 14 writing measures (percentages for seven subgroups across two grades). At both the district and State levels, students in grade 4 had higher performance levels than seventh graders. District fourth graders performed below State levels but district seventh graders were on par. STAAR Science Fewer fifth and eighth grade students achieved Satisfactory scores on the STAAR Science examination this year compared to The only improvement was among white students in grade 5. This improvement accounts for seven percent of this examination s 14 measures. District performance levels fell below State levels. In four cases, the district did not follow trends at the state level. Grade 5 All Students, African American, Hispanic, and Special Education subgroups had decreasing performance levels while the State had increasing levels. STAAR Social Studies Overall, eighth graders did not perform as well on the STAAR Social Studies examination compared to Of the seven measures analyzed, only one measure (14%) for the white subgroup, showed improvement. District performance levels were below State levels. subgroups showed gains from to at both the State and district levels. Six of the seven subgroups showed larger gains at the district level than the State on Algebra I. Two subgroups gains (All Students and Special Education) were the same or greater than State gains for Biology EOC. Attendance and Graduation Rates The overall attendance rates for Dallas ISD, as well as the attendance rates for each ethnic group (African American, Hispanic, White, Asian, and Native American), met or exceeded the State s 90 percent standard. The overall attendance for each year examined was 95 percent. The percentage of students who graduated in four years, high school continuers, or acquired a GED was 89.8 percent (88.7% in ). There was a decrease in the percentage of student dropout in (10.2%) from (11.3%). Additional Measures of Title I Success Additional measures of Title I student success including district students academic preparation for college, as measured by participation rates and performance of students on the ReadiStep, the Preliminary SAT/National Merit Scholarship Qualifying Test (PSAT/NMSQT), the American College Test (ACT) and the Scholastic Aptitude Test (SAT), Pre-Advanced Placement (Pre-AP) courses, Advanced Placement (AP) courses/exams, Dual Credit courses, International Baccalaureate (IB) courses, and advanced levels on STAAR EOCs can be found in the College and Career Readiness evaluation report EA Additional measures of ELL student success can be found in the Evaluation of the Dual Language and ESL Program evaluation report EA Measures of Title I Special Education students success can be found in the Evaluation of Special Education Program report EA Measures of early childhood success can be found in the Reading Results for Kindergarten to Grade 2 report EA For more information, see EA , which can be found at org/page/888. STAAR End-of-Course (EOC) The five STAAR EOC examinations (English I, English II, Algebra I, Biology, and U.S. History) were administered primarily to ninth and tenth graders. The district had a lower percentage of students achieving a Satisfactory score compared to the State on three of the examinations English I, English II, and Algebra I. Dallas ISD ELL students outperformed State ELLs on all five EOCs. For both Algebra I and Biology all 68

77 The Translation Services Program: Leonardo R. Ledezma, Ph.D. EA At-a-Glance The Translation Services program of Dallas ISD, funded by Federal Title I Part A grants, provides translation and interpretation services to schools for English language learner (ELL) students and parents, to assist in communicating academic information and promote participation in all school related activities. During the school year , there were a total of 64,452 ELL students in the Dallas ISD. Based on a needs assessment, it was determined that there was a need to increase the involvement of ELL student parents in academic learning and other school activities of their children. This may be helped by providing document translations or oral interpretations in their native language. Language support for evening and weekend events such as family nights and parent workshops was also needed to assist ELL parents to become active participants in their children s education. Furthermore, tutoring in languages other than Spanish was needed in order to contribute to the academic improvement of ELL students. The Title I program work scope allocation for this component was $111,295. The goals of the Translation Services program in were: 1. Assist parents in understanding the state s academic content standards, student academic achievement standards, and academic assessment: 2. Provide materials and training in the parent s native language to help parents improve their children s achievement, such as literacy training and using technology, to foster parent involvement; 3. Ensure that information related to schools and parents programs, meetings, and other activities was sent to parents in a format and language that parents can understand; 4. Provide support for parent involvement activities as parents may request; 5. Provide materials and training to help parents to work with their children to improve their children's achievement, such as literacy training and using technology to foster parental involvement. The purpose of this report was to evaluate the effectiveness of the Translation Services program based on data collected at the end of school year Data were collected and provided by Translation Services Department of Dallas ISD. Data sources include logs, records, translation request forms, and interpretation request forms. Services Provided The Translation Services program provided document translations and oral interpretations at Dallas ISD Title I schools involving student academic and school-related activities. While the translation services translated written documents (e.g. Letter/Flyers, Newsletters/Calendars, ARD, Forms/Surveys, Student Manual/Handbooks) into requested languages such as Spanish, Cambodian, Vietnamese, etc., the interpretation services provided oral interpretations and language support for meetings (e.g., Parent Conference, PTA, Parent Training/Workshop, etc.), testing, and tutoring. Table 1 summarizes the services provided in and the previously evaluated year. Table 1 Summary of Translation Services Type of Services Written Document Translation Oral Interpretation* Student Tutoring (Hours) 1, *Including language support to parents at Trainings/Workshops During , Translation Services provided 392 written document translations, and 539 oral interpretation services. A total of 378 hours of tutoring were provided to those ELL students who needed additional academic assistance in the areas of reading, mathematics, language arts, and social studies. The reason for the reduction in tutoring hours is that the only tutoring position was split funded and this allowed only half-time tutoring. Services Satisfaction Survey To determine areas of strength and weakness of the program, user satisfaction surveys were conducted during February-March A survey with 16 questions was distributed to 229 staff who requested services during the school year. Also, a brief five-question parent survey was distributed to 500 parents. The survey questions focused on perceptions and satisfactions of the services provided. A total of 229 staff surveys and 138 parent surveys were collected; the response rates were 44 percent and 57 percent, respectively. Staff Survey. The most requested service type was Document Translation (63.7%) followed by ARD Interpretation (46.9%) and Interpretation at School Meetings/PTA (36.3%). While 82.6 percent of the respondents felt parent conferences are more successful by having an interpreter, 45.1 percent said that they were still able to communicate with parents of 69

78 ELL students if language interpreters were not available. Approximately 48 percent of the staff noticed an increase in attendance when parents of ELL students know that the meeting will be offered in their language, and 70.9 percent said that having interpreters at meetings/workshops helped parents become more involved in their child s education. Overall, 75.2 percent of the respondents were satisfied with the availability of language interpreters for their meetings. Concerning school and district documents, over 80 percent of the respondents felt that having documents available to parents of ELL students in their native language impacts their involvement in school activities. Most of the staff felt the necessity of translated documents (70.8%). Only 41.8 percent of the respondents requested the services often, and 48.0% requested the services sometimes. Overall, 55.8 percent were satisfied with the turnaround time of documents submitted for translation. As for the tutoring services, 32.2 percent of the respondents felt that tutoring contributed toward the student s academic progress compared to 53 percent in the previous survey. If a language tutor was not available, only 28.3 percent said that they would have easy access to another way of communicating information to parents. A new service called Spanish Interpreter hotline (SI) was offered and 49.6 percent of the parents used this service. Users learned about the service through a Translation Services letter or (37.6%), a coworker (32.7%) and District online newsletter (29.7%). About 97 percent of the users considered that SI is a useful resource in helping to communicate with English language learner parents. Parent Surveys. Two separate surveys were administered to parents; one survey was given to Spanish speaking parents and the other to Vietnamese speaking parents. Among the 115 respondents to the Spanish survey, 98 percent of the parents stated that the interpreter helped them to participate in meetings about their children s education. Also, 93 percent stated that the information the school sent home in their native language helped them to better understand and be part of their child s education. When asked what they would do if an interpreter was not available to help, 33 percent said they would rely on teachers, 29 percent said that their daughter or son would help, 29 percent said they would rely on office personnel and 9 percent said that a neighbor would help. In the Vietnamese survey, parents responded that the information in their native language helped them to take part in their children s education (100%) and that having a tutor who spoke Vietnamese had helped their child improve in school (82.6%). Ninety-six (96%) percent stated that being able to speak to the tutor in Vietnamese has helped them to become more active in their children s education. Summary The Translation Services program of the Dallas ISD provided document translation and oral interpretation services to schools, ELL students and their parents. The purpose of the program was to provide professional services and expertise in communicating with the diverse communities, to facilitate school communication with limited or non-english speaking parents, and to promote participation in their children s education. During the school year , Translation Services provided more translation and interpretation services but fewer tutoring services than in According to the staff and parent surveys, the program enabled families of ELL students to participate in academic activities and school functions, and fostered cooperation between home and school As there was only one Vietnamese tutor, which limits the scope of the services, the academic progress result was based on very limited case numbers (N=10). The tutoring service was an effort made to meet the educational needs of ELL students and assist in closing the achievement gap for students in need of additional academic assistance. Recommendations Based on the survey results, it is clear that having interpreters available and information translated in the parents native language increased participation in academic programs and activities, and it promoted parent s collaboration in supporting their children s education. It is recommended that the staff increase the utilization of the translation and interpretation services. Similarly, it is recommended that more effort be made to fund and recruit qualified tutors. For more information, contact Program Evaluation at

79 The Neglected and Delinquent Children and Youth Program: Leonardo R. Ledezma, Ph.D. EA At-a-Glance The Elementary and Secondary Education Act required educational agencies to conduct Title I programs in Institutions for Neglected or Delinquent Children and Youth. It defines the term institution for neglected children as a public or private residential facility, other than a foster home, that is operated for the care of children who have been committed to the institution or voluntarily placed in the institution under applicable State law, due to abandonment, neglect, or death of their parents or guardians. Program Characteristics The goal of the Title I program for Neglected or Delinquent Children and Youth was to improve the educational performance of the children in the institutions, regardless of their home school or economic deprivation level. Specifically, the program was to help the neglected or delinquent children and youth to succeed in the regular education program, through supplemental instruction in reading/language arts and mathematics. The purpose of this report is to provide an overview of the Title I program conducted in Institutions for Neglected or Delinquent Children and Youth during the school year. The report presents demographic characteristics of the students, number of students served, and achievement data used to measure student progress. There were 219 Title I students in two institutions for Neglected or Delinquent Children and Youth. Promise House served 122 students aged 14 through 18 in grades 8 through 12 and Cottrell House served 97 students aged 6 through 16 in grades 1 through 11 (Table 1). Table 1 Number and Percent of Students Served in Institutions for Neglected or Delinquent Children and Youth, School N % Promise House Cottrell House Total Note: Data were obtained from Institution Program Coordinators on 07/21/2014. Data indicated that there were more males (65.1%) than females (34.9%) served. Of the total students served, 106 (51.5%) were African American, 62 (30.1%) were Hispanic, 38 (18.5%) were White (Table 2). Table 2 Number and Percent of Students Served in Institutions for Neglected or Delinquent Children and Youth by Gender and Ethnicity, African American Hispanic White Total N % N % N % N % Male Female Total Note: Difference due to missing data (N=10) and multiple ethnicity students not reported (N=3). Program Outcomes Reading and mathematics entry and exit course grades were obtained for Promise House and Cottrell House students. Table 3 shows a comparison of average entry and exit cumulative grades in reading and mathematics (all grades combined). The data from Promise House showed an increase from entry to exit scores for both reading and mathematics, while Cottrell House students did not experience any gain. Students at Promise House were not promoted even though their grades improved. The institution reported students as Not Applicable (NA) for promotion. Cottrell House, on the other hand, did not show any improvement in grades but promoted 17 students. Table 3 Average Entry and Exit Scores of Neglected and Delinquent Students for Entry Exit Entry Exit School Reading Reading Math Math Promise House Cottrell House Note. Data were obtained from Institution Program Coordinators on 7/21/2014. Data indicated that most Promise House students, upon release, went to another institution (51.6%), to a foster home (37.7%), to unknown destinations (9.8%), or home (0.9%). Meanwhile, most Cottrell House 71

80 students went home after release (84.3%), while a smaller group went to another institution (13.3%) and an even smaller group was incarcerated (2.4%). Table 4 Destination Upon Release by Institution Foster Care Destination Upon Release Another Institution Incarceration Other Total Institution Home Promise House Cottrell House Total Note: Total differences due to missing data. The number of special education students was small. Less than a quarter of the students were classified as special education students (Table 5). Table 5 Number and percent of Special Education Students by Institution Institution Special Education No Yes Total Promise House 83 (85.6%) 14 (14.4%) 97 Cottrell House 68 (71.6%) 27 (28.4%) 95 Total 151 (78.7%) 41 (21.3%) 192 Note: Differences due to missing data. Most of the students served attended school between 0 and 105 days in a school year of 177 days, as shown in Figures 1 and 2. However, the number of days of service to students may not be under the control of the institutions since they receive students at all times of the year. Most Promise House students (84.4%) were served between 0 to 35 days, while 40% of Cottrell House students were served between 0 and 35 days and about 50% were served between 36 and 105 days Cottrell House Percent Days Served Figure 2. Percent of Students by Days Served at Cottrell House Recommendations Results indicated that there was an improvement in classroom performance made by Title I school students in reading and mathematics at Promise House, but none for those at the Cottrell House. It is recommended that the district, along with support from the evaluation team, study the context and implementation of this program to ensure that these students are served in the best way possible. It is also recommended that the two institutions work to improve data recording to reduce the occurrence of missing data. Similarly, it is recommended to explore the possibility that the Youth and Family Program could provide behavioral services to the children in the two institutions. Some of the population in these institutions might require more specialized treatment and could benefit from the services provided by the Youth and Family program. For more information contact Program Evaluation at Promise House Percent Days Served Figure 1. Percent Students by Days Served at Promise House 72

81 National Summer Learning Demonstration Project and Study: Nolan Rett Mickelson EA At-a-Glance Big Thought, a non-profit organization in partnership with the Dallas Independent School District (Dallas ISD), provided arts-integrated creative learning to students. The collaboration was an expansion of several partnerships among the Dallas ISD, the City of Dallas, community organizations, and Big Thought to integrate resources and increase creative learning in schools and neighborhoods. In , Big Thought managed the Thriving Minds 21 st Century after-school program and partnered with the district to support a summer program for students both at risk and not at risk of retention. In 2012, the RAND Corporation selected this partnership to participate in a study that examines the effects of summer learning loss. Summer learning loss is the general decline in students academic skills during summer vacation. The magnitude of learning loss generally varies by school subject, family income level, and grade level. This study, which has now concluded its second year, is funded by the Wallace Foundation. Thriving Minds Summer Camp Program In the summer of 2014, the Dallas ISD partnered with Big Thought to host the Thriving Minds Summer Camp program for Early Childhood, Pre-Kindergarten, and first- through fifth-grade students. Elementary students from these locations that were in danger of retention, that attended 21 st Century after school programs, and students selected to be part of the National Summer Learning Demonstration (primarily rising fifth-grade students) were targeted for participation. Table 1 gives the number of students at each of the eight summer camp sites. Students at risk of retention were classified as recommended. All other students 21 st Century and Wallace were classified as enrichment. Overall, almost 30 percent of the students at the summer camps attended for the enrichment program. Edward Titche Elementary School had the largest enrollment at 444 students, but also enrolled Early Childhood through fifth-grade students; other sites hosted only Pre-Kindergarten through fifth. Few enrichment students at J. N. Ervin Elementary School attended at least one day (12.3%). Nearly 43 percent of the students at Julian T. Saldivar and Highland Meadows attended for enrichment. Table 1: Thriving Minds Summer Camp Enrollment by Student Type Recommended Enrichment Site N % N % Allen Bethune Brashear Ervin Kiest Highland Meadows Saldivar Titche All 2, , Note. Does not include 2 students recorded as enrolled and attending a summer camp as part of the Extended-School Year for students with disabilities. Thriving Minds Summer Camps did not host this program. The Thriving Minds Summer Camp was provided Monday through Friday from 8:00 a.m. to 3:45 p.m. from June 30 th to July 31 st, a total of 23 instructional days. Transportation was provided from the student s home school for all students who attended any of the 91 campuses feeding into the eight summer camp sites. Students who were part of the National Summer Learning Demonstration received transportation even if they had transferred to a campus that was not part of the Thriving Minds program to preserve the integrity of the study. Students were provided breakfast, and lunch each day. Students completed coursework in English/Language Arts, Mathematics, Science, and Social Studies with certified teachers. Core subject teachers teamed up with certified fine arts teachers and campus integration specialists to create an integrated curriculum in all subject areas. Students would rotate through several enrichment programs each week that included physical education, dance, music, drama, and visual arts, among others. National Summer Learning Demonstration Students In February 2013, all rising fourth-grade students at 81 elementary schools received an opportunity to apply for the Thriving Minds Summer Camp. Applications were received by the Evaluation and Assessment department to determine student eligibility. Students at risk for grade retention or those who participated in the 21 st Century after-school program were ineligible for participation in the program. Over 2,000 applications 73

82 were received from parents of eligible students who were interested in the voluntary summer program. A random lottery was conducted to determine who was selected to attend. Students that were selected to participate were granted the opportunity to attend the summer program for two years: the summers of 2013 and All students, attending the program or not, would complete assessments in the fall of 2013 and again in the fall of Evaluation and Assessment will continue to provide additional information to RAND Corporation s staff to further evaluate the effects of summer learning loss in the following years. Results The Thriving Minds Summer Camp program was one of five summer enrichment programs across the country to participate in this study. The RAND Corporation has provided the district with a report every October that included a program evaluation with recommendations to strengthen the summer program. Final results of the study will not be released until the study has concluded. For more information, contact Program Evaluation at (972) Information about other Dallas ISD summer school programs may be found in EA , EA , and EA summer school reports. 74

83 E&A Internal Service Fund: Nancy Kihneman, Ph.D. EA At-a-Glance The Internal Service Fund (ISF) of Evaluation and Assessment was established to fund equivalent positions in the Evaluation and Assessment Department utilizing grant or other district funding set-asides. With monthly billing against the set-asides, the department provides products and services to fulfill evaluation requirements of specific grants and initiatives where a portion of that grant or initiative funding has been allocated to evaluation services and placed in the ISF. Billed amounts from the fund set-asides cover a substantial portion of ISF personnel salaries, equipment, and supplies. However, demands on ISF personnel that are not specifically grant funded require those services to be subsidized by the General Operating Fund. ISF Practices and Characteristics For , the cost of evaluation services per person-day remained at $500. The department is required to review the daily rate on an annual basis and make changes, as appropriate. The rate of $500 per day was established in , following a review of market rates for evaluation services. At that time, the rate was increased from $400 per day to reflect increases in cost of living from (the year of the last rate increase) to A memo sent to the Chief Financial Officer and the Director of Grants Management on August 8, 2013 documented evaluation daily rates for evaluations. Typically, evaluation activity runs from October through September of the following school year. Some projects are billed monthly over the course of the evaluation cycle, while others are billed in one or two payments, at the discretion of the project manager. Due to discrepancies among grant beginning and ending cycles, evaluation beginning and ending cycles, and the district's closure of accounting books for a period at the close of the fiscal year, larger projects typically bill the entire work's effort from October through May each year. Evaluation work continues from June through the end of the evaluation cycle as outlined in the evaluation plan. Time and Effort Billing The ISF time and effort billing form attests to the total number of person days billed during each month or billing period. Each month or billing period during the billing cycle, the assigned evaluator submits the form to the program manager for signature, and returns it to ISF fund management. The ISF manager completes accounting journals to bill all current ISF-funded projects and submits the journals to grants accounting or the general ledger accountants for posting of charges Evaluations There were 24 active accounts in the ISF, including Titles I and II. Title III was evaluated by ISF staff; however, funding was provided through an FTE in the Language and Literacy Department. Four high schools had TTIPS (Texas Title I Priority School) grants (Smith, Roosevelt, Spruce, North Dallas). Three accounts were associated with Adult Basic Education and two accounts with HIPPY. Grant account names are listed below: Title I HIPPY (2) Title II TTIPS (4) Title III Special Education UIL Student Activities ABE/TANF EL Civics (3) College/Career Ready Summer School Native American Indian Reconnect Program Refugee Social Services Carl D. Perkins EC Education K-2 After School Programs EC/Community Partners SSI Big Thought As of August 2014, a total of $868,763 was billed for evaluation services provided by the fund. The total includes $350,000 for Title I evaluation services. Grant set-asides were significantly lower than in prior years, but demands for services from funded projects and unfunded projects were higher. This reduction reflects continued constraints facing grantors at the state and federal levels, resulting in reduced grant awards and/or elimination of certain types of grants. In many cases, services for a particular program exceeded the amount placed in the fund; however, evaluation charges did not exceed the amount of the grant set-aside and evaluators continued to provide services without billing. UIL Student Activities contracted with the ISF for evaluation services in the amount of $60,000. However, program management was unable to provide the necessary data and information required 75

84 to complete a final evaluation for Services in the amount of $30,000 were billed, but the remaining $30,000 will not be billed unless the evaluation can be continued and completed during the school year. Title I evaluation included an overall Title I data report as well as reports associated with programs funded under Title I such as Youth and Family Services, Academic Facilitators and Reasoning Mind, among others. Unfunded Services As discussed in prior reports, evaluation staff are often called upon by district administrators and the Board of Trustees to conduct evaluations and/or provide data for district initiatives that have not set aside funding. In addition, evaluators are asked to increase the scope of evaluations beyond the amount of the set-aside. During the school year, unfunded services included evaluation of State Compensatory Education, Accelerated Instruction, Research Review, PBM Integrated Interventions/PBMAS submissions, Class Size Waiver report, Commit! data requests, Teach for America report, the Districtwide Campus Staff Survey and tasks associated with the pilot of Teacher Excellence Initiative (TEI). In addition there were numerous ad hoc requests fulfilled by evaluation staff. Evaluations under Titles I and II for the Leadership Fellows Academy and Reasoning Mind far exceeded the grant set-asides in terms of resources expended on these projects. the daily rate for services in be increased to $600. This is still below market, but better reflects the true value of services provided. This will most likely not result in additional funds for the ISF because project managers frequently allocate evaluation funds based on overall grant funding rather than an accurate estimate of the number of days of services they will be requesting and/or the actual number of days it takes to complete an evaluation. Finally, with respect to billing issues, a new market study should be conducted in Recommendations Increase daily rate for evaluation services to $600 per day for Conduct a market study to determine current conditions regarding rate structures for services similar to those provided by E&A. For more information on this report, contact Evaluation and Assessment at Billing Issues in Evaluation funds in the amount of $64,800 were initially set aside for the continuation of the Wallace/Big Thought Summer Learning project. Evaluation services were provided and will continue into the fall of However, because the grant was overspent, the district withdrew evaluation funding, with the commitment from ISF management that services would continue as originally planned. An informal market analysis was conducted in 2011 to determine if the daily rate for services should be increased. At that time and for the next several years, it was decided not to increase the rate. However, even in 2011, district ISF rates were well below market for similar services (see EA ). Rates have not been increased due to the sluggish economy, reduced grant funding for projects, and the necessity for grant-funded projects to cut the overall amount allocated to evaluation so that more money from the grant could go toward direct student services. ISF management now recommends that 76

85 Reading Results for Kindergarten to Grade 2 Myoungsook Kim, Ph.D. EA At-a-Glance During the school year, the Early Childhood: K-2 Curriculum and Instruction department coordinated a range of programs and initiatives designed to help students in grades kindergarten (K) through two read on grade level by the end of second grade. These included Istation reading, iread backpack project, centralized literacy library, and read aloud modules. The total budget for the department was $2,767,126, which included $166,010 of Title I, Part A funds. The purpose of this report is to summarize student progress in reading on the Iowa Tests of Basic Skills (ITBS), Logramos, the Texas English Language Proficiency Assessment System (TELPAS), Istation s Indicators of Progress Early Reading (ISIP), and the Assessment of Course Performance (ACP). Performance on ITBS In , about 62 percent of kindergarten students performed at or above grade level on ITBS, whereas less than 50 percent of students in grades one and two were at grade level (45% to 48%). Thus, a sizable percentage of students in grades K through two performed below grade level (38% to 55%). Compared to , the percentage for kindergarten students was about the same but the percentages for first- and second-grade students dropped (8% and 6% drop, respectively) (See Figure 1). Figure1: Percentage of Grades K-2 Students At or Above the 40 th Percentile on ITBS Reading P e r c e n t K G1 G2 G Source: PEIMS district database: 10/29/2012 and 10/28/2013. Note. Statistics are based on Reading Profile Total results for kindergarten students and Reading Total results for grades 1 and 2 students. Statistics include magnet or alternative schools. Performance on Logramos For all grade levels, over 85 percent of students performed at or above grade level on Logramos in (85% to 92%). The percentage at grade level was lower than in The pattern was consistent for all student groups except for White students in kindergarten and second grade, which showed slight increases (3.5 and 1.7 percentage points, respectively) (See Figure 2). When compared to ITBS, results indicated that limited English proficient (LEP) students performed at a higher level than their English counterparts in reading when administered a test in their native language. Figure 2: Percentage of Grades K-2 Students At or Above the 40 th Percentile on Logramos Reading P e r c e n t K G1 G2 G K Source: PEIMS district database: 10/29/2012 and 10/28/2013. Note. Logramos is a Spanish language achievement test; as such, few if any African American students take this test. White students who take this test are more likely in the Dual Language two-way program. Statistics are based on Reading Total results. Effectiveness of the iread Backpack Project The iread backpack project, provided for kindergarten students, was first implemented in December 2012 at nine elementary schools. The project was expanded to 25 schools (the Imagine 2020 schools, the South Oak Cliff feeder, and the Wilmer-Hutchins feeder) in to help kindergarten students read on grade level and engage in reading at home every day. Attendance rates and ITBS/Logramos results were compared for kindergarten students who participated in the project and for those who did not. Results showed that overall attendance rates were high and about the same for iread schools and non-iread schools (95% to 96%) across school years (from to ). Over 65 percent of the kindergarten students in iread schools performed at grade level on ITBS (66% to 69%), while about 89 percent of the kindergarten students did on Logramos across school years. The percentage of students at grade level on ITBS was higher in iread schools than non-iread schools, while the percentage on Logramos was about the same between iread schools and non-iread schools (See Figure 3). 77

86 Figure 3: Percentage of Kindergarten Students with an ITBS/Logramos Reading Percentile Rank At or Above the 40 th Percentile P e r c e n t P e r c e n t Kindergarten ITBS Reading Profile Total At or Above the 40 th Percentile iread Schools Non iread Schools Kindergarten Logramos Reading Total At or Above the 40 th Percentile Table 1: Percentage of Students Performing At or Above Grade Level on ISIP BOY MOY EOY Grade N (%) N (%) N (%) English Version Kindergarten 7,667 (36) 7,821 (32) 7,339 (35) Grade 1 8,325 (27) 8,504 (27) 8,042 (28) Grade 2 9,412 (25) 9,864 (25) 9,410 (24) Spanish Version Kindergarten 6,170 (59) 6,223 (52) 5,981 (58) Grade 1 6,105 (61) 6,357 (63) 6,195 (65) Grade 2 5,574 (64) 5,877 (62) 5,672 (61) English and Spanish Combined Kindergarten 12,735 (48) 12,652 (44) 12,187 (49) Grade 1 13,313 (44) 13,485 (46) 12,942 (48) Grade 2 12,723 (45) 12,813 (45) 12,258 (44) Source: PEIMS district database: 10/28/2013. Note. BOY=beginning of the year. MOY=middle of the year. EOY=end of the year. N=numer tested. %=% at or above the 40 th percentile. If a student took the test more than once, the student s latest score was used. For the English and Spanish versions combined, if a student took the test in both languages, the Spanish version was used. Relationship between Time Spent on Istation Reading and ITBS/Logramos Scores Results of correlation analyses indicated no meaningful relationship between total time spent on Istation reading and ITBS/Logramos reading scores (r = to 0.03). However, this does not mean that the Istation reading program is not effective. Rather, it may indicate that time alone is not a good measure to examine the effectiveness of the program. iread Schools Non iread Schools Performance on ACP 1 Performance on TELPAS Results on TELPAS reading showed that about 73 percent of second-grade students performed at or above grade level (at the Intermediate level or higher). Between and , the percentages of students on grade level for all high school feeder patterns, with the exception of Madison, dropped. Declines ranged from five percentage points to 25 percentage points; it should be noted that higher cut scores were used in Performance on ISIP Results showed that across ISIP administrations in for grades K through two, over 50 percent (52% to 65%) of students tested in Spanish were at grade level, whereas less than 40 percent (24% to 36%) of students tested in English were at grade level. The overall percentages of students at grade level for the English and Spanish versions combined were less than 50 percent (44% to 49%) (See Table 1). ISIP results by division and feeder pattern varied. There was no significant progress in the percentage of students at grade level throughout the year across divisions and student groups for the most part. About 86 percent of students in grades K through two passed the fall 2013 reading ACP, while 83 percent of students passed the spring 2014 reading ACP. When reviewed by grade, in fall 2013, first-grade students had a higher passing rate (95%) than kindergarten (89%) or second-grade (72%) students, while in spring 2014, kindergarten students had a higher passing rate (94%) than first- (84%) or second-grade (72%) students. Across grade levels, White students had higher passing rates (84% to 98%) than Hispanic (72% to 94%) or African American (60% to 92%) students. Differences in Second-Grade Reading Scores by Prekindergarten Enrollment Status Results showed differences in second-grade reading achievement scores by prekindergarten enrollment status. Across test types (ISIP, TELPAS, ACP), the percentages at grade level in were higher for the Dallas ISD prekindergarten group (DISD PK) than the No DISD PK group. Regarding the longitudinal ITBS/Logramos results for the kindergarten cohort from kindergarten through second grade, the same pattern was present. That is, although the gap 1 ACPs for grades K through two were newly developed and administered for the first time in fall 2013 and spring As a result, results should be reviewed with caution. 78

87 between the two groups narrowed slightly as students progressed from kindergarten through second grade, students who attended district prekindergarten were more likely to be reading on grade level than those who did not attend. Recommendations Encourage more kindergarten through second-grade teachers to utilize district resources and participate in professional learning opportunities. The K-2 Early Childhood department plans to provide resources and supports around language and literacy development in a variety of formats (e.g., face-to-face workshops, virtual professional learning communities (vplc), podcasts, webinars, and online training modules). It will be helpful for teachers to know the opportunities in a timely manner, utilize resources better, and thus provide high quality instruction to students. Magnuson, K. A., Meyers, M., Ruhm, C., & Waldfogel, J. (2004). Inequality in preschool education and school readiness. American Educational Research Journal, 41, Mitchell, L., Wylie, C., & Carr, M. (2008). Outcomes of early childhood education: Literature review. Report to the Ministry of Education. Wellington: New Zealand Council for Educational Research. The full K-2 Early Childhood Education final report can be found at For more information, contact Program Evaluation at Make sure that teachers who use Istation reading are able to utilize data summary statistics and provide individualized feedback based on data interpretation. Results of correlation analyses indicated that time alone may be not a good measure to examine the effectiveness of the Istation reading program. It will be important that teachers use the data summary to pinpoint student needs and help students master areas of weakness in a timely manner. Encourage and enable parents to be involved to help their children increase their literacy skills through the iread backpack project. Given that the iread backpack program involves parent engagement, motivating and educating parents to learn and develop the skills to nurture and support their children s literacy skills will be essential (The Annie E. Casey Foundation, 2010). Continue the district focus on expanding prekindergarten enrollment. Results showed significant differences in reading achievement scores by prekindergarten enrollment status. The result is in line with previous research that reported benefits of early childhood education programs (Howes, Phillipsen, & Peisner-Feinberg, 2000; Magnuson, Meyers, Ruhm, & Waldfogel, 2004; Mitchell, Wylie, & Carr, 2008). It will be important to make sure that the K-2 Early Childhood efforts are aligned to prekindergarten efforts (birth to PreK) through streamlined communication and vertical planning. References The Annie E. Casey Foundation (2010). Early warning! Why reading by the end of third grade matters. A Kids Count Special Report. Baltimore, MD. Howes, C., Phillipsen, L., & Peisner-Feinberg, E. (2000). The consistency and predictability of teacher-child relationships during the transition to kindergarten. Journal of School Psychology, 38,

88 80

89 Anduamlak Meharie, Ph.D. Student Success Initiative: EA At-a-Glance The Student Success Initiative (SSI) was created by the 76 th Texas Legislature in 1999 to ensure all fifth and eighth grade students receive the instruction and support they need to be academically successful in reading and mathematics. The SSI grade advancement requirements also specify that grade advancement requirements apply to enrolled fifth and eighth grade students who take the State of Texas Assessment of Academic Readiness (STAAR) mathematics and reading tests. As per these requirements, a student may advance to the next grade level only by passing these tests or by unanimous decision of his or her Grade Placement Committee (GPC) that the student is likely to perform at grade level given additional accelerated instruction during the course of the year. According to the SSI manual, required accelerated instruction must occur after each administration of the assessment for which a student did not meet the passing standard, including the third administration. House Bill 5, 83 rd Legislative Session, added new TEC to require each school district to provide accelerated instruction in the applicable subject area each time a student fails to perform satisfactorily on an end-of-course (EOC) assessment instrument. According to these requirements, a student must pass the English I, English II, Algebra I, Biology, and U.S. History EOC exams in order to graduate from high school. Accelerated instruction may be provided before or after normal school hours and may include participation at times of the year outside normal school operations. House Bill 5 also amended TEC to require school districts to offer, without cost to a student, additional accelerated instruction in any subject if the student failed to perform satisfactorily on an EOC assessment. Districts or charter schools are required to separately budget and prioritize state compensatory education (SCE) funding and any other funding necessary to sufficiently support the cost of additional accelerated instruction for students who fail to perform satisfactorily on an EOC assessment instrument 1. According to a letter published on Texas Education Agency (TEA) 1 State compensatory education funds may not be used for any other purpose until districts or charter schools have sufficiently funded additional accelerated instruction. website, sufficient funds have been appropriated to support implementation of accelerated instruction programs for grades three through eight as well as for high school for the and school years as specified under Texas Education Code (TEC) and , the Student Success Initiative. Additionally, TEC as amended by HB 5 requires each school district to evaluate the effectiveness of accelerated instruction programs for high school students and to hold an annual public hearing to consider the results of the evaluation. The SSI evaluation was to describe the context of SSI and evaluate the effectiveness of services provided to at-risk students and those who fail to perform satisfactorily on STAAR reading and mathematics as well as STAAR EOC assessments. This evaluation was based on review of relevant program and funding documents, descriptive statistics, and a series of paired sample t-test. Program Characteristics Evaluation and Assessment requested data on fifth and eighth grade students from each campus who received tutoring services between the first and second administration of the STAAR. In sum, 155 campuses submitted the names of 4,283 fifth-grade and 2,995 eighth-grade students who failed to meet the satisfactory standard on the April administration of the STAAR mathematics and reading tests and received tutoring support 2. On average, students received days of tutoring for an average of total hours. Most campuses provided tutoring during the regular school day, although a considerable number of campuses utilized a combination of strategies including before, during, and after school, as well as Saturday sessions. In most schools (N= 151) services were provided largely by campus staff including teachers, campus instructional coaches, administrators, and 2 Four schools indicated that no students required tutoring. 81

90 counselors; some campuses (N= 20) also utilized tutoring contractors to provide support. Targeted students participated in reading or mathematics tutoring services designed to increase students mathematics and reading skills so that they were no longer considered at-risk and could meet or exceed the passing standards on the STAAR. The evaluator did not request data on tutoring services provided to high school students who failed or were at-risk of failing STAAR EOC tests during the regular school year. Data provided in this report, therefore, is largely based on all students who participated in December and May STAAR EOC administrations. Additionally, this report presents the passing rates of students who enrolled in mathematics and science Applications and Study Skills courses (for local credit only) during SY , designed to deepen conceptual understanding and provide students who need instructional intervention with additional learning opportunities focused on mathematics [and science] TEKS SEs with which the student is struggling. Only eight campuses had students enrolled in one or both of these courses. Table 1 shows amount of expenditure 3 on personnel for services provided during the school year (excludes summer programs). In-school tutoring is a contracted expense where consultants provided tutoring during the day. Table 1: Funding Expenditure by Programmatic Level In-School Tutoring Programmatic Extra-duty Pay Program Level Actuals Actuals High 63, , Middle 55, , Elementary 695, , Total 813, , In addition to the expenditures on personnel, the district also spent money on resources purchased to help students who did not pass the STAAR between the first and second administrations and also for all the content EOC areas during the regular school year in 2014 (Table 2). Table 2: Expenditure on Curricular Resources by Subject Area Tested and Grade Level Grade Level STAAR Reading/ English I & II EOC STAAR Math/ Algebra I EOC Biology EOC Total 5 49, , n/a 82, , , n/a 27, , , , , Total 179, , , , *Data for US History EOC was unavailable Unlike previous years in which selected campuses hosted the SSI summer tutoring program for all students across the district, during SY each campus provided intensive, accelerated instruction to their own students who failed the April or May 2014 STAAR reading or mathematics tests or any of the STAAR EOC assessments in May. Elementary and middle school students participated in an eight-day SSI summer program held June High school students participated in an 11-day program held June According to data obtained from Office of Budget Services summer SSI utilized $2,830, on personnel for services provided to students. The evaluator was unable to obtain verified rosters of students for all campuses who participated in SSI summer tutoring program. To evaluate the effectiveness of the summer SSI program for elementary and middle students, the evaluator compared scale score gains and passing rates for all students who failed either or both mathematics and reading STAAR tests on the second administration (May) and retook the relevant test during the third administration (June). For high school students the evaluator was able to obtain usable attendance data for 2,619 students from 18 high schools who participated in the 11-day summer program in June. Program Outcomes The results of the evaluation are provided below in two main sections. The first focuses on fifth and eighth grade students who took the STAAR reading and mathematics assessments. The second section is on high school students who took the STAAR EOC assessments. Each main section is then further divided by STAAR/EOC administration time periods. STAAR Reading & Mathematics: May Administration 3 Data on expenditures should be read with caution. The nature of funding allocation for SSI is complex and dynamic and it is beyond the scope of this brief report to adequately capture the nuances of this process and provide an accurate estimate of all expenditures. Using data from 4,283 fifth-grade and 2,995 eighthgrade students who failed to meet the satisfactory standard on April administration of the STAAR mathematics and reading tests and received tutoring support, a series of paired samples t-test were 82

91 conducted to determine whether there were significant differences between the average scale scores of STAAR mathematics and reading tests between the April and May administrations. Analysis of the STAAR results revealed that students who received tutoring services on each of the content area they failed to pass on the first administration of the STAAR (April) did significantly better on the second (May) administration (Table 3). Table 3: Mathematics and Reading STAAR Test Mean Scale Score Gains and Passing Rates: April and May Administrations Mean STAAR Exam Grade (N) Scale Score Gain SD t d Passing Rate (May) English Version Reading 5 (1,294) * (1,142) * Mathematics 5 (2,795) * (1,855) * Spanish Version Reading 5 (1,096) * Mathematics 8 (66) * *p<0.01 Of the 1,294 fifth graders who took the April 2014 English version of the STAAR reading test, 30.3% passed (met standard) and showed an average mean scale score gain of 60 points (Negative t values indicate that the first administration in the comparison had a smaller mean that the second). The result of the paired samples t-test was statistically significant (p<0.01) and, more importantly, Cohen s effect size value (d=.37) suggests a small to moderate practical significance 4. Although the mean scale score gain in the English version of STAAR reading for eighth grade students (N=1,142) between April and May was significantly lower compared to fifth graders (M= 31.53), only a slightly lower percentage of students had a passing rate (27.3%) on the May administration. The paired t-test results were similarly statistically significant (p<0.01) and Cohen s effect size value (d=.37) suggests a small to moderate practical significance. The biggest gain was found among fifth grade students who retook the Spanish version of the STAAR reading test (N= 1,096). Their mean scale score gain was points. The difference was statistically significant (p<0.01), but more revealing is the effect size (d = 0.70) which suggests a moderate to large practical significance. 4 An effect size is an objective measure of the magnitude or importance of observed effect. Results of paired samples t-tests conducted to examine differences between the average scale scores of the English version of the STAAR mathematics tests for fifth- and eighth- grade students between the April and May administrations were similarly encouraging. Fifth-grade students (N = 2,795) had a higher mean gain score compared to 1,855 eighth graders (67.93 vs , respectively) and a higher passing rate in April (41.3 vs. 26.0, respectively). While differences for both grades were statistically significant, the effect size, d, for fifth graders was 0.52 which is a moderate one, whereas a 0.28 effect size for eighth graders suggests a small practical significance. Although a considerably smaller sample size (N = 66), eighth-grade students who retook the Spanish version of STAAR mathematics in May had 16.7% passing rate and a mean scale point gain with an effect size of 0.58, which is a moderate practical significance. STAAR Reading & Mathematics: June Administration Since the evaluator was unable to obtain verified rosters of fifth and eighth grade students who participated in tutoring services during the summer portion of the SSI, what is provided below and in Table 4 (next page) is based on all students who failed either or both mathematics and reading STAAR tests on the second administration (May) and retook the relevant test during the third administration (June). Therefore, the outcome of the analysis cannot solely be attributed to the eight-day SSI summer tutoring program. Mean scale score for the 1,372 fifth-grade students who retook the English version of the STAAR reading test in June improved by points with a 25.1% passing rate. The result of the paired t-test was statistically significant (p<0.01) and, more importantly, Cohen s effect size value (d =.45) suggests a moderate practical significance. Those fifth-grade students who retook the Spanish version of the STAAR reading test in June (N = 615) showed similar results; mean scale scores improved by points and 23.1% achieved a passing rate. The results were both statistically significant (p<0.01) and Cohen s effect size value (d=.41) suggests a moderate practical significance. On the other hand, of 1,422 eighth-grade students who retook the English version of the STAAR reading test in June, 19.5% met the minimum standard and passed. Mean scale score for these students was only 0.47, which was not statistically significant with 83

92 the effect size, d, at 0.01, indicating no practical significance. Table 4: Mathematics and Reading STAAR Test Mean Scale Score Gains and Passing Rates: May and June Administration Mean STAAR Exam Grade Scale Score Gain SD t d Passing Rate (June) English Version Reading 5 (1,372) * (1,422) Mathematics 5 (1,842) * (1,618) * Spanish Version Reading 5 (615) * Mathematics 5 (54) ** *p<0.01, **p<0.05 Results of paired samples t-tests conducted to examine differences between the mean scale scores of the English version of the STAAR mathematics tests for fifth- and- eighth grade students between the May and June administrations were as follows. Fifth-grade students (N = 1,842) had a higher mean gain score (M = 32.72) compared eighth graders (N = 1,618, M = 51.93) but a slightly lower passing rate in June (27.1 vs. 29.9, respectively). Scale score differences for both grades were statistically significant at the 0.01 level and the effect size, d, were 0.52 and 0.53 for fifth grade and eighth grade students, respectively, which suggest a moderate practical significance. The smallest gain in mean mathematics scale score was found among a comparatively smaller number of fifth grade students who retook the Spanish version of the STAAR mathematics test (N= 54). Their mean scale score gain was points. The difference was statistically significant (p<0.05) but effect size of 0.28 suggesting a small practical significance. STAAR EOC: December Administration STAAR EOC results (Table 5) for the December administration 5 are for all students who took one or more of EOC assessments. Table 5: STAAR EOC Passing Rates: December Administration (continued on next page) STAAR EOC Passing Rates (Num. Tested) N % Algebra I (2,551) Biology (1,468) December EOCs should be conceived as the third administration of a testing cycle, with the first and second administrations taking place in May and July, respectively, of the previous school year. Table 5: STAAR EOC Passing Rates: December Administration (continued from next page) STAAR EOC Passing Rates (Num. Tested) N % US History (14) Reading I (4,227) 2, Reading II (1,722) Writing I (5,741) Writing II (3,339) 1, The evaluator also examined the passing rates of students who were enrolled in the mathematics and science Applications and Study Skills courses. Although the number of students enrolled in these courses was considerably low for rigorous comparison purposes (see table 6), passing rates for them were higher by a significant margin in Algebra I (+16.8%) and a small margin in Biology (3%). Table 6: December STAAR EOC Passing Rates of Students Enrolled in Applications and Study Skills Courses Students with Final Course Grade for Semester 1 Percent Passing Rates Algebra I (25) 44.0 Biology (22) 36.4 STAAR EOC: May Administration Beginning in SY , EOC Reading and EOC Writing tests were replaced with EOC English tests. Table 7 presents the results of the first administration of the new cycle of testing for SY Table 7: STAAR EOC Passing Rates: May Administration Passing Rates STAAR EOC N % Algebra I (10,537) 7, Biology (10,080) 8, US History (7,800) 7, English I (13,789) 7, English II (10,943) 10, STAAR EOC passing rates for English I (50.9%) and English II (54.8%) were considerably lower compared to the other EOC results. Passing rates for US History were highest (92.0%), followed by Biology (87.7%), and Algebra I (73.2%). Table 8: May STAAR EOC Passing Rates of Students Enrolled in Applications and Study Skills Courses Students with Final Course Grade for Semester 2 Percent Passing Rate Algebra I (302) 81.1 Biology (341) 90.9 Once again, although the number of students enrolled in mathematics and science Applications 84

93 and Study Skills courses during the second semester were considerably low for rigorous comparison purposes (see table 8), passing rates for these group of students were higher for Algebra I (+7.9%) and Biology (3.2%). The evaluator also examined whether students enrolled in mathematics and science Applications and Study Skills courses in both semesters performed significantly better on Algebra I and Biology EOC exams (Table 9). The result of the analysis showed that students enrolled in the mathematics course during both semesters (N = 59) had a passing rate of 98.3%. The passing rate for their counterparts enrolled in the biology course was less dramatic at 81.5%. Table 9: May STAAR EOC Passing Rates of Students Enrolled in Applications and Study Skills Courses Percent Students with Final Course Passing Rate Grades for Both Semesters (May) Algebra I (59) 98.3 Biology (297) 81.5 STAAR EOC: July Administration Using attendance data on 2,619 students from 18 high schools who participated in the 11-day summer program in June, the evaluator conducted a series of paired sample t-tests to determine whether there were significant differences between the average scale scores on each of the five STAAR EOC tests between the May and July administrations. In order to retain the statistical power of the analysis EOC results were not disaggregated by grade levels (Table 10). Table 10: STAAR EOC Mean Scale Score Gains and Passing Rates: May to July Administration STAAR EOC Exam Mean Scale Score Gain SD t d Percent Passing Rate N Algebra I * English I 1, * English II * Biology * US History *p<0.01 English I and English II EOC exams showed the largest mean gain scores between May and July administrations ( and , respectively) as well as the largest effect sizes (0.62 and 0.63, respectively) suggesting a moderate to large practical significance. Mean scale score gain for Biology EOC test was points, with a moderate effect size (d = 0.54). With a small to moderate effect size (d = 0.38), the average scale score gain for Algebra I was points. The smallest mean scale score gain was attributed to US History EOC test (29.03) with a small effect size (d = 0.17). All results, with the exception for US History EOC test, were statistically significant at the 0.01 level. A third of students (33.1%) who retook the Algebra I and English I STAAR EOC tests in July met or exceeded expectations. Passing rates were slightly higher for Biology EOC (37.6%) and slightly lower for English II EOC (27.1%). Students who retook US History EOC had the lowest passing rate among the group at 20.8%. Summary and Recommendation High students who were enrolled in the mathematics and science Applications and Study Skills courses for one or two semesters performed significantly better, particularly in Algebra I, compared to those who were not enrolled in these courses. While the generalizability of the results is a concern due to the comparatively small sample size of students enrolled in these courses during SY (only eight campuses had students enrolled), the district should consider conducting a more in-depth feasibility study of participation in these courses by more campuses and a greater number of at-risk students. It is likely that semester long supplemental courses designed to deepen and strengthen conceptual understanding of subjects, particularly with differentiated instruction focusing on specific TEKS SEs with which students are struggling, lead to better student outcomes compared to intensive instructions provided in a short period of time. A rigorous evaluation of the effectiveness of accelerated instruction programs is highly dependent on the availability of high quality student attendance data and other program related information. The district should strongly consider providing the necessary infrastructure to campuses to properly document implementation data related to accelerated instruction programs (e.g., attendance, number of hours tutored by content area, service provider, and when service was provided) and making mandatory the documentation implementation. For more information, contact Program Evaluation at

94 86

95 Evaluation of the Home Instruction for Parents of Preschool Youngsters (HIPPY) Program: Amber McEnturff, M.S. At-a-Glance EA Home Instruction for Parents of Preschool Youngsters (HIPPY) is a home visiting program for parents of three-, four-, and fiveyear-olds targeting school readiness and parent involvement. HIPPY Dallas ISD had a total budget of $1,682,767 for program year , including a general operating budget from Dallas ISD of $548,419, $518,748 from Title I funding, $100,000 from Title III funding, and $515,600 from external sources. The HIPPY program in Dallas began in 1988 and is the largest of 12 HIPPY programs in the state of Texas, serving 879 children and 874 parents in , with 661 children and 614 parents successfully completing the program year. HIPPY Dallas ISD will expand to serve approximately 1,000 families in The purpose of this report is to summarize implementation and outcomes for HIPPY participants. Methodology Implementation and Participants. Information about HIPPY Dallas ISD program implementation came from an interview with the HIPPY Dallas ISD program manager. To summarize participant characteristics, frequency analyses were conducted using information from the Texas HIPPY Efforts to Outcomes database. School Readiness. HIPPY participants completed the Bracken School Readiness Assessment (BSRA-3) during the first two weeks (pre-test) and last two weeks (post-test) of the program year. Standard scores were reviewed to determine participants levels of school readiness relative to a national sample. Paired sample t-tests and Cohen s d effect sizes were conducted to show whether changes were statistically and practically significant. 1 Parent Involvement. First-year HIPPY parents completed a parent survey as a pre-test during the first two weeks and as a post-test during the last two weeks of the program year. Frequency analyses were conducted comparing HIPPY pre-test and post-test averages to the national average. 2 The comparison group from the national database was created by selecting families with income levels less than $40K that lived in zip codes where greater than 40 percent of the population was Hispanic or African American, 1 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd. ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. 2 Comparison data was from the 2007 National Household Education Survey by the Institute for Education Sciences National Center for Education Statistics at the U.S. Department of Education. The data can be downloaded here: reflecting the income range and racial composition of the HIPPY Dallas ISD group. Academic Achievement. Former HIPPY participants in grades kindergarten through eight were tracked in the district database, which allowed for analyses of academic outcomes as measured by the Iowa Test of Basic Skills (ITBS), Logramos, and State of Texas Assessments of Academic Readiness (STAAR). Propensity score matching was used to create control groups from the pool of non-participants in the district database. The percentage of students in each group that met the standard for each assessment was calculated. For ITBS and Logramos, the standard was defined as a score at or above the 40 th percentile. For STAAR, the standard was defined as a satisfactory rating. The difference in percentage points between the HIPPY and control groups was computed. Attendance and promotion rates were also compared between HIPPY participants and control groups. Implementation and Participants Figure 1 provides a brief overview of HIPPY child and parent demographic characteristics. Of the 661 child participants, most were enrolled in the HIPPY 3 curriculum 3 (57%), with 31 percent in HIPPY 4 and 12 percent in HIPPY 5. Most were Hispanic (82%) with a primary language of Spanish (79%). Most parent participants (95%) were mothers, Hispanic (85%), had one child enrolled in the program (92%), were not employed outside of the home (83%), and had an annual household income of less than $20,000 (64%). Figure 1: HIPPY Child and Parent Demographic Snapshot Enrollment by HIPPY Curriculum Year HIPPY 3's 57% Child Race/Ethnicity Hispanic 82% HIPPY 4's 31% Adult Participant Relationship to HIPPY Child Mother 95% Employment Status Not Employed Outside Home 83% Annual Household Income $0 to < $20K 64% HIPPY 5's 12% Af. Amer. 14% Note: *PT and FT refer to part-time and full-time employment, respectively. 3 Throughout this report, the term HIPPY 3 s refers to participants using the HIPPY 3 curriculum, meant for children approximately age three. The same applies to HIPPY 4 s and 5 s. Thus, curriculum year roughly coincides with age but is not an exact indicator of age. PT* 9% $20K to < $40K 33% FT* 8% Other 4% Other 5% > $40K 3% 87

96 School Readiness HIPPY participants enrolled in curriculum years 3, 4, and 5 made statistically and meaningfully significant improvements in school readiness raw scores from fall 2013 to spring Figure 2 shows the average school readiness standard scores for English and Spanish test-takers for HIPPY 3 s, 4 s, and 5 s. All subgroups made significant gains in standard scores, except English speaking HIPPY 5 s, who started and ended the program year with an above average standard score. A majority of HIPPY 4 s (82% of English speakers and 68% of Spanish speakers) and HIPPY 5 s (85% of English speakers and 79% of Spanish speakers) were classified as ready for school compared to their same-aged peers. 4 Figure 2: Average Standard Scores on BSRA-3 at Pre-Test and Post-Test with School Readiness Classifications Ready for School HIPPY 3's Standard Score HIPPY 4's Standard Score HIPPY 5's Standard Score English (n = 44)* Spanish (n = 183)* English (n = 34)* Spanish (n = 109)* English (n = 13) Spanish (n = 29)** Pre Post Very Delayed Delayed Average Advanced Very Advanced School Readiness Classifications Based on Standard Scores Note. Standard scores were categorized according to the publisher. *p <.001. **p =.012. Cohen s d effect sizes for Spanish in all curriculum years were in classified as medium in magnitude. Cohen s d effect sizes for English in curriculum years 3 and 4 were classified as large in magnitude. Parent Involvement HIPPY participants increased the average number of minutes spent reading to the child from 13.6 minutes to 20.1 minutes per day (Figure 3). This exceeds the national average of 16.6 minutes per day. The average number of books in the home increased from 17.5 books to 28.3 books, which is slightly lower than the national average of 29.7 books (Figure 4). Participants made substantial gains in frequency of four literacy enrichment activities (such as pointing out letters) after one year of HIPPY participation and exceeded the national averages in all four activities. Figure 3: Average Number of Minutes Per Day Read to Child HIPPY Pre (n = 236) 13.6 HIPPY Post (n = 236) 20.1 National Average (n = 336) 16.6 Figure 4: Average Number of Books the Child Had of His or Her Own HIPPY Pre (n = 237) 17.5 HIPPY Post (n = 237) 28.3 National Average (n = 336) 29.7 Note. Figure 3 and 4 national averages derived from the 2007 National Household Education Survey Academic Achievement Table 1 shows the percentage of HIPPY participants and propensity score matched control group participants that met the standard for ITBS, Logramos, and STAAR mathematics and reading. The table also shows the percentage point difference between HIPPY and control groups. HIPPY participants outperformed the control group in 15 of the 21 possible comparisons (green text). The control group outperformed the HIPPY group in three comparisons (red text), and the groups performed about the same in three comparisons (black text). There were no notable differences between the HIPPY and control groups for attendance or promotion rates. Table 1: Percentage that Met Standard on ITBS, Logramos, and STAAR for Former HIPPY Participants and Matched Control Groups Percentage that Met Standard Percentage Points Grade n HIPPY Control Difference ITBS Mathematics K % 52.1% % 43.2% % 51.7% 5.8 ITBS Reading K % 68.4% % 52.4% % 53.1% 0.0 Logramos Reading K % 77.1% % 77.6% % 93.6% 3.8 STAAR Mathematics % 58.6% % 56.0% % 68.1% % 68.5% % 55.9% % 60.2% 8.6 STAAR Reading % 60.0% % 51.2% % 62.2% % 71.3% % 61.6% % 73.3% 0.7 Recommendations Prepare for increased data collection by improving efficiency of data processes and/or obtaining additional support to assist the program with data collection and case file management. This will be important as the site continues to expand in the coming program year. Include more detailed information about HIPPY participants in the district database to facilitate further analyses of student outcomes. Information such as length of program participation and successful completion status are not currently collected in the district database. The full HIPPY final evaluation report can be found at For more information, contact Program Evaluation at Ready for school was defined as a standard score of 85 or higher, including categories of Average, Advanced, and Very Advanced. 88

97 McKinney-Vento Homeless Children and Youth Education Program: Leonardo R. Ledezma, Ph.D. EA At-a-Glance The McKinney-Vento Homeless Education program is a federally funded program. The purpose of the McKinney-Vento Homeless Education Program is to facilitate the enrollment, attendance and success in school of homeless children and youths. It does this by providing enrichment and support services to homeless children and their families. The program also acts as an advocate on behalf of homeless families by educating staff and the community about the rights and issues of homeless children, and their parents. In the program period , the Homeless Education Program received $264,412 from McKinney-Vento funds. The amount of Title I, Part A funds for students in homeless situations was $70,000, and Foundations provided $15,000 of funding. 1 McKinney-Vento Homeless Children and Youth Education Program Context and Goals The McKinney-Vento Homeless Assistance Act (Subtitle B-Education for Homeless Children and Youths) guarantees educational rights and protections for children and youths experiencing homelessness. Section 725 (2) of the McKinney-Vento Act defines homeless children and youths as individuals who lack a fixed, regular, and adequate nighttime residence due to loss of housing or economic hardship. This includes children and youths living in shared housing with other persons; living in cars, hotels, motels, bus or train stations, trailer parks, abandoned buildings, camping grounds; living in emergency or transitional shelters; abandoned in hospitals or waiting for foster care placement. The Dallas Independent School District (Dallas ISD) responded to the homeless population of children and youths within the district with the McKinney-Vento Homeless Education Program to provide direct services to homeless school age children. These services included tutoring, homework help, referrals, school supplies, food, backpacks, and school uniforms. The McKinney-Vento Homeless Education Program has also served the children, families, and youths indirectly by providing training to Dallas ISD personnel and shelter personnel. The McKinney-Vento Homeless Education Program stated goals are: improve attendance rates of the McKinney-Vento Homeless Education Program students, and increase directly served homeless students academic success. Program Participants The total number of homeless students enrolled in Dallas ISD increased from 2,586 in to 3,030 students enrolled as homeless in Of the 3,030 enrolled as homeless, 1,893 were served by the McKinney-Vento program as compared to 1,252 in the previous year. Table 1 shows that homeless students were represented at all grades. There were more students in the lower grades than in prior years. Table 1 Number of McKinney-Vento and Temporary Arrangement Students by Grade Grade McKinney- Vento Temporary Arrangement Total PK KN Total 1,893 1,136 3,030 Approximately 46 percent of all McKinney-Vento students were African American, 45 percent were Hispanic, and six percent were white. Twenty-four percent (24%) of the McKinney-Vento students were classified as English language learners (ELLs) and nine percent were special education students. About seventy-one percent (70.9%) of all homeless students were in elementary schools. The elementary schools with the largest homeless populations were: Cabell (83, 2.7%), North Dallas (63, 2.1%), Blanton (47, 1.6%), Bayles (36, 1.2%), Field (26, 0.9%), and Rusk (24, 0.8%). Homeless students were present at 190 Dallas ISD schools. The number of students enrolled at a particular school varied throughout the school year. 1 Source: District Budget Summary. 89

98 Program Activities The Homeless Education Program manager worked in conjunction with the Psychological and Social Services Department coordinating efforts with psychologists, teachers, and social workers to provide services to homeless students. The program also collaborated with various local agencies including University of Texas at Dallas, West Lake YMCA, Rainbow Days, Captain Hope s Kids, Church of the Incarnation, Focus on Teens, Promise House, the North Texas Food Bank, and Camp Hoblitzelle in Midlothian to provide enrichment and camps throughout the year. Attendance Program Outcomes Figure 1 shows the percent of attendance by days enrolled for both the McKinney-Vento and Temporary Arrangement students. Due to the transient nature of the population, some students were enrolled for a short time, while others attended school for the whole year. There were higher rates of attendance for McKinney-Vento students compared to Temporary Arrangement students across all enrollment brackets. Efforts by the McKinney-Vento program manager to support parents and students to remain at their school of origin for the school year, accounts, in part, for the increased attendance. McKinney-Vento students stayed enrolled longer and had more regular attendance. Less than one percent (0.8%) of the total homeless students had no attendance data. The percent attendance for homeless students enrolled for the full year (176+) was 63.5 percent for McKinney-Vento students and 50.6 percent for Temporary Arrangement students. The percent attendance for homeless students enrolled between 141 and 175 days was 32.9 percent for McKinney- Vento students and 22.8 percent for Temporary Arrangement students. Almost half of homeless students (1,431 or 45%) remained in school for the entire school year (more than 175 days). Figure 1. Percent of Attendance by Number of Days Enrolled for Temporary Arrangement and McKinney-Vento Students Iowa Tests of Basic Skills (ITBS) Results Temporary Arrangement students outperformed McKinney-Vento students on the ITBS Reading Total at the Kindergarten level. However, McKinney-Vento students outperformed Temporary Arrangement students on reading at grades 1 and 2 and mathematics at both kindergarten and grades 1 and 2 (Table 2). ITBS Reading Kindergarten Grades 1 and 2 Mathematics Kindergarten Grades 1 and 2 Table 2 Grades K-2 ITBS Percent of Students at or Above 40 th Percentile McKinney-Vento Temporary Arrangement State of Texas Assessments of Academic Readiness (STAAR) Results The percentage of students meeting the Satisfactory level of STAAR for grades 3-8 are presented in Table 3. Results showed that McKinney-Vento students outperformed Temporary Arrangement students in all subjects. Table 3 Percentage of Grades 3-8 STAAR with Satisfactory Results by Group and Subject McKinney- Vento Temporary Arrangement % % Reading Mathematics Writing Social Studies Science Source. Student data files STAAR administrations through May. STAAR End of Course (EOC) Results McKinney-Vento students were outperformed on all STAAR End-of-Course (EOC) tests when compared to all district students. Algebra I and Biology EOCs tests were the same as last year, allowing comparisons with results. McKinney-Vento students showed losses of 3 percentage points in Algebra I and 12 percentage points in Biology from Comparisons with Temporary Arrangement students were not possible, due to the small number of Temporary Arrangement students in grades 9 and 10 (Table 4). 90

99 Table 4 Percentage at Satisfactory Level on STAAR EOC Tests for Homeless Students Student Group Satisfactory Satisfactory English I McKinney-Vento 32.9 N/A District 50.2 N/A English II McKinney-Vento 44.7 N/A District 54.3 N/A Algebra I McKinney-Vento District Biology McKinney-Vento District US History McKinney-Vento 83.8 N/A District 91.4 N/A Source: Student Database 10/28/2013. Temporary Arrangement students outperformed McKinney-Vento students on the ITBS Reading Total at the kindergarten level. However, McKinney-Vento students outperformed Temporary Arrangement students on reading at grades 1 and 2 and mathematics at both kindergarten and grades 1 and 2. It is recommended that the McKinney-Vento program continue their outreach activities. It is further recommended that if the Dallas ISD wishes to continue its McKinney-Vento program it needs to fund this program at a higher level. Currently, the program has a single full-time staff member who is unable to physically meet the needs of the homeless population in spite of enormous efforts. Increasing the staff of the program would result in more services being delivered to more homeless students. For more information, contact Program Evaluation at Summary and Recommendations The total number of homeless students increased from 2,586 enrolled in Dallas ISD in to 3,030 students enrolled as homeless in Of the 3,030 enrolled as homeless 1,893 were served by the McKinney-Vento program as compared to 1,252 in the previous year. The major goals for the school year were to improve school attendance rates and to increase student achievement among program participants. Attendance of McKinney-Vento students improved and was consistently higher than Temporary Arrangement students. McKinney-Vento students remained in school longer than in prior years. Efforts by the program manager to support parents and students to remain at their school of origin for the school year accounts, in part, for the increased attendance. Homeless students were distributed over all grades rather than concentrated in a few upper grades, as was the case in previous years. This suggests that the program manager s efforts may have increased awareness among principals about the needs of homeless students. It also shows schools willingness to enroll homeless students, regardless of any perceived possible ill effects on school performance. The percentage of McKinney-Vento students who obtained a Satisfactory level on the various STAAR EOC examinations was less than the percentage of all district students in all subjects. The program has attained the goal of increasing the student achievement of the McKinney-Vento students compared to Temporary Arrangement students on all STAAR tests grades

100 92

101 American Indian Education Program: Sarina Rapini, MPA EA At-a-Glance The American Indian Education Program (AIEP) was designed to provide enrichment and support services to American Indian and Alaska Native students and their families. It also promoted understanding throughout the Dallas Independent School District (Dallas ISD) by educating staff and the community about the culturally diverse populations within the district. The AIEP aimed to address educational and culturally related academic needs of American Indian and Alaska Native students so that they may meet the same performance standards expected of all Dallas ISD students. The program received federal funds to enhance and supplement core educational services for American Indian students. AIEP offered opportunities and activities to help students improve academic skills and become more proficient in core content areas. The program also encouraged postsecondary readiness by exposing students to postsecondary options through presentations and fairs. Program Goals In , AIEP received a federal grant of $123,759 and served 653 students. AIEP outlined the following five goals: Increase American Indian student participation in cultural education, Support academic achievement in reading and develop reading habits among American Indian students in grades PK-8, Decrease dropout rates among American Indian students, Prepare high school students for college and the workforce, and Provide opportunities for students to explore postsecondary options through college field trips. Program Activities Throughout the year, AIEP sponsored reading clubs for program participants to help students improve reading habits and achievement. Students were encouraged to discuss and share cultural themes as found in the books written by American Indian authors. To encourage an interest in college, AIEP took 20 high school students to Southern Methodist University (SMU) on March 28, Students took a campus tour and attended a question and answer session. The event was hosted by Steve Denson, a professor at SMU s Cox School of Business. AIEP also partnered with the American Indian Parent Advisory Committee (PAC) to provide college and career information at two events. The PAC Royalty Pageant occurred on November 1, 2013, and the American Indian Talent Showcase followed on December 13, At both events, AIEP staff members set up a table to distribute college and career resources to American Indian students. AIEP representatives were also available throughout the year to meet with secondary students and provide academic and career guidance. AIEP staff hosted an End of the Year Awards Ceremony on June 6, During June 9 to June 19, AIEP staff conducted a summer enrichment program at Jack Lowe Elementary. The program was open to American Indian students in kindergarten through eighth grade. A total of 18 students participated in science, writing, and reading enrichment courses as well as American Indian tribal history and language activities. The summer program also included speakers from the Environmental Protection Agency, Burlington Northern Santa Fe Railway, Fort Worth ISD, and Urban Inter-Tribal Center of Texas. On the last day, students took a field trip to the Trinity River Audubon Center. Program Participants In , a total of 653 students participated in the program. 1 Demographic data are presented in Table 1. A majority (83.2%) of students were economically disadvantaged and more than half (56.5%) were considered at risk. Fewer were English Language Learners (4.7%) or special education students (10.5%). As shown in Table 2, the percentages of AIEP students varied by grade level. A total of 229 (37%) students were in high school, 162 (26.2%) were in middle school, and 228 (36.8%) were in elementary school. Students from 145 Dallas ISD elementary, middle and high schools across the district participated in the AIEP program. The largest portion of students attended Sunset High School (N=29), Wilson High School (N=20), and Quintanilla Middle School (N=19). 1 It should be noted that there were a total of 658 American Indian students in Dallas ISD for the school year. Students that participate in AIEP may not identify themselves as American Indian but claim an affiliation with one of the American Indian tribal nations. Out of the 653 AIEP students, 249 (38%) identified themselves as American Indian and 261 (40%) identified themselves as Hispanic. 93

102 Table 1: Demographics of AIEP Students* Demographics N % Gender Female Male Special Populations Econ. Disadvantaged At Risk Special Education English Language Learner Source: Chancery file, May 27, *Data were missing for 34 students. Table 2: AIEP Students by Grade* Grade N % Early Childhood 1.2 PK K *Data were missing for 34 students. Student Performance Table 3 presents the average attendance of AIEP students for the fall, spring, and entire school year. Of the 606 students with valid data, 29 (4.8%) were enrolled for only one semester. Of the students enrolled for both semesters, attendance between semesters declined for 335 (58.1%) students and increased for 207 (35.9%) students. The remaining students (N=35; 6.1%) had no change between semesters, and 33 of the 35 students had perfect attendance each semester. Table 3: Attendance Rates of AIEP Students* Semester N % Fall Spring School Year *Data were missing for 47 students. Table 4 shows the State of Texas Assessments of Academic Readiness (STAAR) passing rates for students in grades three through eight. Passing rates ranged between 61 and 72.9 percent. The passing rates for AIEP students surpassed the district in all subject areas by two (Writing) to 19.7 (Social Studies) percentage points. Table 4: STAAR Passing Percentages, Grades 3-8 AIEP District Subject N % N % Reading , Writing , Mathematics , Social Studies , Science , Table 5 presents STAAR End Of Course (EOC) passing rates for AIEP students and the district. AIEP students passing rates were slightly lower than the district in all subject areas except United States History, which had a 100 percent passing rate. Like the district, less than half of AIEP students (44.8%) passed the English I STAAR EOC. Table 5: STAAR EOC Passing Percentages AIEP District Subject N % N % Algebra I , Biology , English I , English II , U.S. History , Summary and Recommendations The American Indian Education Program (AIEP) received federal funds to provide enrichment and support services to American Indian students. During the school year, AIEP staff provided academic support, college and career resources, and enrichment activities, including field trips and guest speaker presentations. Student outcomes were mixed. Attendance declined between the fall and spring semesters but remained above 90 percent for both semesters. On STAAR 3-8, AIEP passing rates were above the district in all subject areas. On the STAAR EOC exam, AIEP passing rates were slightly below the district in all subject areas except United States History. It is recommended that AIEP staff members consider implementing a survey to gather feedback from student participants about their experiences in the program. The survey data would be useful for evaluation purposes and future planning. For more information contact Program Evaluation at

103 Career Education and Workforce Partnerships, Jinhai Zhang, Ph.D. EA At-a-Glance The Carl D. Perkins Vocation and Technical Education Act of 2006 helped support the Dallas Independent School District s Career Education and Workforce Partnerships Department, known as Career and Technical Education (CTE). CTE s goals were to prepare students for college and career success, provide them with the skills to be competitive in a global economy, and recognize and meet the unique needs of a diverse student population. Since the introduction of Perkins IV, a nationwide effort under the national States Career Clusters Initiative defined 16 Career Clusters, or broad groupings of occupations and industries based on related disciplines, including recommended sequences of academic coursework. Texas has adopted the national 16 Career Clusters Model and is implementing all Career Clusters. The Dallas ISD has followed the model and emphasized the term Career Pathways to distinguish these groupings from the career clusters offered at each high school. Local Progress of the Radical Transformation During , the High School Redesign program worked with high schools to identify and develop one or more career pathways at each campus and redesignate the pathway as an academy with enrollment of students. High School Redesign projected a total of 132 pathways representing each of the 16 federally defined Career Clusters to be located throughout the district. The pathways were initiated in The number of pathways reached 102 in , but fell to 90 in , 87 in , and 72 in One factor contributing to the decline in pathways was the loss of CTE teachers qualified to teach necessary courses due to statewide budget cuts. A second factor was the de-emphasis on CTE due to increased emphasis on core courses following new graduation requirements introduced in In addition, decisions about the location of specific pathways were ultimately made by campus principals and were likely to be reexamined if a change in the principalship resulted in a change of direction for the school. Beginning in , TEA organized CTE courses into 16 career pathways (Table 1). In , courses aligned with all 16 pathways were offered in Dallas ISD schools. As part of the High School Redesign program, comprehensive high schools offered five satellite magnet programs, 30 career academies, and 72 career pathways. Academy and pathway programs were offered at district comprehensive high schools and magnet schools. Table 1: Career Pathways Projected and Implemented Projected Career Pathway (09-10) Implemented (13-14) Agriculture, Food & Natural Resources 2 3 Architecture & Construction 6 5 Arts, A/V Technology & Communication 12 5 Business, Management & Admin Education & Training 8 4 Finance 9 8 Government and Administration 5 1 Health Science 4 3 Hospitality & Tourism 10 4 Human Services 4 6 Information Technology 9 6 Law, Public Safety, Corrections & Security Manufacturing Marketing, Sales & Service 11 5 S.T.E.M 12 9 Transportation, Distribution & Logistics 12 2 Total Level of Student Participation Career and technical education (CTE) students were classified into two levels. High school students who had taken two or more courses for three or more credits within a specified pathway were classified as Career Pathways students. Elective students were all those who were enrolled in one or more CTE courses. The latter group included prospective Career Pathways students who had not yet acquired three credits. All middle school (grades 7-8) CTE students were elective students, while high school students were either elective or pathway students. More than half of all students in grades 7-12 were enrolled in at least one CTE course, as shown in Table 2. In grades 7-8, 6,703, or 31.7 percent of 21,122 students, were enrolled in elective CTE courses. In high school, 23,562 students, or 62.9 percent of 37,483, were enrolled in CTE courses. Among students in grade 12, nearly 70 percent were enrolled in CTE courses (see Table 2) 95

104 Table 2: Percent of CTE Student by School Level Grade Total Students CTE Students Middle School CTE % 7 10,915 2, ,207 4, Total 21,122 6, High School 9 10,959 6, ,969 5, ,571 5, ,984 5, Total 37,483 23, All Secondary 7-12 Total 58,605 30, Career and technical education (CTE) courses, administered by the Career Education and Workforce Partnerships Department, were offered in all district high schools. Nearly 35 percent of CTE course enrollment among high school students was in the Business, Management, and Administration (19.9%), and Information Technology (14.3%) pathways. Health Science had 8.7% of CTE students, while Arts, Audio/Visual (A/V) Technology & Communication had 7.6% of CTE students. Hospitality and Tourism had 7.4% of students, and Human Services had 7.1% of enrollment. The level of CTE course enrollment varied across schools. Among comprehensive high schools, the largest percentages of CTE students were found at North Dallas (86.4%), Skyline (80.9%), Pinkston (79.1%), and Conrad (77.8%). Skyline had the largest percentage of Career Pathways students (36.8%), followed by Conrad (31.9%), Spruce (27.5%) and Lincoln (27.1%). Among magnet schools, all students at the Education and Social Services, Health Professions, and Public Service magnets were CTE students, and 99.8% of Business Magnet and 99.7% Smith New Tech High School were CTE students. More than 95 percent of students at the Health Magnet (99.8%), the Business Magnet (98.3%) and Education and Social Services Magnet (96.7%) were Career Pathways students. Academic Effectiveness and College Readiness Based on certain qualifications, high school CTE students, including elective students, could earn college credit in three ways: through content-enhanced articulated courses, Dual Credit courses, and/or College Board Advanced Placement (AP) Program courses. Thirty-three high schools offered courses included in an articulated tech prep program with the Dallas County Community College District in Student enrollment totaled 19,052 courses taken that could potentially provide college credit towards degree and certificate programs. CTE Career Pathways students accounted for 57 percent of district enrollment in Dual Credit courses and 44 percent of enrollment in Advanced Placement courses. Recommendations High School Redesign projected a total of 132 pathways representing each of the 16 federally defined Career Clusters to be located throughout the district in when pathways were initiated. The number of pathways dropped from 102 in to 72 in To meet the High School Redesign projections, the district should install additional programs and enroll more students, particularly in the areas of Marketing, Sales and Service; Business, Management, & Administration; Law, Public Safety, Corrections & Security; Education and Training; and Transportation, Distribution, & Logistics. In addition, plans to develop career academies at some high schools continued to be affected by staffing cutbacks necessitated by previous state reductions in funding. Another finding of the program was that pathways offered in district high schools did not sufficiently match local workforce needs. The current academy and pathway programs at district high schools should focus on business services, construction, leisure and hospitality, education and health services, and government. For example, business services and leisure and hospitality have increased 6.2% and 6.3%, respectively, from April 2013 through April 2014 in the Dallas-Fort Worth area. The district has only 7 programs in business and 4 programs in hospitality. Emphasis should be placed on matching CTE programs to local employment needs as well as to student and community interests. For more information, see EA , available at 96

105 Adult Education and Workforce Literacy, Nolan Rett Mickelson EA At-a-Glance TThe Adult Education and Workforce Literacy (AEWL) program, formerly the Adult Basic Education program, is a federal- and state-funded program which provided free education and literacy services to adults throughout Dallas and Rockwall Counties. The AEWL Department of the Dallas Independent School District (Dallas ISD) managed these programs. The AEWL program provided eligible adults with classes that would allow them to acquire functional skills that could enhance the quality of their lives as family members, workers, and citizens, as well as take a General Educational Development test and receive a State of Texas Certificate of High School Equivalency. The program received funding through the Temporary Assistance to Needy Families (TANF) federal grant. The purpose of the TANF is to increase the flexibility of states in operating a program assisting needy families to end the dependence of parents on government long-term welfare by promoting job preparation, work, and marriage. The bulk of the funds supporting the program, however, came from the Adult Education and Family Literacy Act (AEFLA). The grant period for the Adult Education federal fund was from September 1, 2013 to August 31, Prior to the school year, the Texas Education Agency guided adult basic education programs. In May of 2013, Governor Rick Perry signed Texas Senate Bill (SB) 307 transferring oversight to the Texas Workforce Commission (TWC). This transition marked a change in adult education programs from providing English language, general literacy, and numeracy instruction to courses designed to develop career readiness, workforce skills, and a foundation for post-secondary education or career and technical education. Enrollment Requirements The AEWL program welcomed all residents who met enrollment requirements. Residents 18 or older were eligible if they were not enrolled in school and were not high school graduates. For 17-year-olds to be eligible, they must have been officially withdrawn from school, and had parental approval. Only in the case of a court mandate could 16-year-olds enroll. High school seniors who had completed all classes and final exams, but had not passed the state required tests were eligible for General Educational Development (GED) instruction. Assessment State-mandated adult literacy assessments were used to determine a participant s baseline level and progress. According to TEA s Adult Education Assessment policy, students must be assessed before the student may be enrolled in the adult education management information system called Texas Educating Adults Management System. The approved standardized assessments in Texas include the Test of Adult Basic Education (TABE), General Assessment of Instructional Needs (GAIN), and the Basic English Skills Test (BEST). Program Staff To accomplish the purposes of the program and to attain federal and state targets, the AEWL Department employed qualified personnel as director, teachers, and administrators. There were 88 staff members serving the program in , down from 104 staff members the year prior. Nearly 82 percent the staff had obtained bachelor s degree or higher. In addition to the employment qualifications, professional development was a requirement for most of the staff; all of the AEWL staff completed all required in-service hours. At the conclusion of the school year, the AEWL Department reduced its staff by 48 percent because the Dallas ISD will no longer continue its role as either the fiscal agent or the lead organization for adult education in Dallas County. Instead, the AEWL Department will be a contracted service provider reimbursed by the Workforce Solutions of Greater Dallas (the fiscal agent) in cooperation with the Dallas County Community College District (the lead organization) and many other local organizations. Instructional Programs The AEWL Department collaborated with various local agencies to offer several instructional programs. The classes were held in schools, recreational centers, Texas Workforce Commission centers, and other community-based organization centers across Dallas County. Classes were held Monday through Thursday with three different time slots (morning, afternoon and evening) to accommodate students. Registration was in person only, on a first-come first-served basis, at the class site during the scheduled registration/orientation dates. Staff with the AEWL Department actively recruited parents of Dallas ISD students for programs by visiting campuses near the four major learning 97

106 centers and passing out informational flyers at the beginning of semesters. The focus of AEWL program included initiatives in integrated career awareness/workforce development, college transitions, distance learning, and individualized goal setting with each student. The instructional programs included: Adult Basic Education (ABE) offered to Englishspeaking adult students with limited reading and writing skills, English as a Second Language (ESL) offered at six levels: Beginning (Literacy, Low and High), Intermediate (Low and High), and Advanced. The ESL component also offered computer training, Adult Secondary Education (ASE), also known as General Educational Development, provided five subjects for instruction: Mathematics, Science, Social Studies, Writing Skills and Reading, EL Civics/Citizenship delivered civics and citizenship instruction for recent immigrants to empower them to actively participate in their communities or to prepare for U.S citizenship, Distance Learning (DL) provided distance education in some areas. Participant Characteristics Table 1 presents the characteristics of program participants. A total of 5,983 students participated in the program. The enrollment decreased by 392 participants, just over 6 percent, from the previous year. The majority of the participants were Hispanic (87.5%), female (71.4%), and ranged in age from 25 to 44 (62.74%). Table 1 Demographics of Program Participants 1 Ethnicity N % Asian American African American Hispanic 5, White Other Gender N % Male 1, Female 4, Age N % , , and above Total 5, Includes 673 students not meeting minimum contact hour requirement. 2 Includes American Indian, Alaskan Native, Native Hawaiian or Other Pacific Islander, and Two or More Races. Program Outcomes Table 2 shows the results of educational gains and attendance of the participants, as the program performance was measured upon student retention and student academic progress. Retention is the percentage of students enrolled with baseline assessment and those who participated in a progress assessment. Educational gains are demonstrated by students gains in scores on standardized tests and the percentages of students who completed at least one level from baseline assessment to progress assessment. All students with 12 hours or more of attendance are held accountable for education gain goals. Table 2 Attendance and Educational Gains by Program Enrolled Number Completed one level Program N Left Pgm N % Adult Basic Ed. 1, English Language 3, , Adult Secondary Ed Total 5, , The student retention rate was high; only 17 students left the program during the year. As for educational gains, over 68 percent of students completed at least one level, higher than the 60 percent from the year prior. The English language classes are noteworthy: The percentages completing at least one level were 69 percent or above for all ESL levels. The percentage completing ASE was 59 percent. At every educational functioning level, the AEWL program met or exceeded the state target despite target increases of as much as 4 percent across all educational functioning levels. The GED testing program changed on January 1, 2014 to a format more rigorous than previous versions. Anyone that had not passed all of the GED section tests prior to that date was required to start the entire process from the beginning. This meant a rush of participants hurrying to pass sections in November and December of 2013, many of whom did not meet the 12 contact hour requirement to be considered an AEWL program participant. Budget and Expenditures Table 3 shows the total budget and actual expenses of the Adult Education and Workforce Literacy program by grant. The largest source of funding came from the AEFLA federal grant, about $3.1 million, more than two-thirds of the total funding. Table 3 Adult Basic Education Fund by Grant Grant Budget Expense 1 AEFLA/TANF Federal $ 4,475, $2,178, El Civics State 112, , Total $4,588, $3,203, Estimates of expenses through August 31,

107 Summary and Recommendations Overall, the AEWL program met all of the grant required performance measures for the EL Civics and Adult Education grants. With the transition of program oversight from the TEA to the TWC, the program managed to continue focusing academic performance while transitioning the students from ESL to GED, then to community college and the work force the newer goals under the TWC. Furthermore, the number of students served by the program are assuredly underestimated due to students who prepared to take the GED exam prior to the changes on January 1, Students who had not passed the GED exam were required to retake all subject tests, even those they had previously passed, when transitioning to the new exam. The transition from TEA to TWC created goals that were misaligned with the intentions of the program. For example, the pre- and post-tests used to determine educational functioning levels (ABE, ESL, and ASE) and the state targets remained focused on basic language, literacy, and numeracy skills while the TWC s desired outcomes included gaining employment, retaining employment, completing the GED exam, and entering post-secondary education or career training. Moreover, the focus on employability requires that participants in the program be employable. That is, they must have a valid Social Security Number. This, in effect, nullified the Dallas ISD s desired secondary outcomes of giving parents a foundation to become involved in their children s education and the community at large. The evaluator does not offer any recommendations for program improvement because the Dallas ISD has ceased to be either the fiscal agent or the lead organization for these grants, though the AEWL Department will remain a contracted service provider serving the majority of adult basic education students in Dallas County, reimbursed through the Workforce Solutions of Greater Dallas. For more information, see EA , available at 99

108 100

109 Sarina Rapini, MPA Refugee School Impact Grant Program: The purpose of the Refugee School Impact Grant (RSIG) program was to provide support services, instructional support services, school orientation for refugee parents, and professional development for Dallas ISD staff. The RSIG program received a federal grant of $142,034 for the school year. In order to evaluate the outcomes of the grant, the RSIG program supervisor provided evaluators with program activity information and related documents. The evaluators accessed student achievement, school attendance and demographic data using the district s Public Education Information Management System (PEIMS) database. Program Participants During the school year, 660 refugee students received either instructional or supportive services. This evaluation focuses on RSIG s instructional support, which included tutoring, instruction in the student s native language, and a four week summer enrichment program. According to data provided by the program supervisor, a total of 361 students received instructional support. Of the 361 students, 241 received tutoring support, 31 received instruction in their native language, and 89 participated in the summer enrichment program. Student data extracted from the district s PEIMS database are presented in Tables 1, 2, and 3. During the school year, 28 refugee students withdrew from Dallas ISD. More than one third (37.7%) of student participants attended Conrad High School, as seen in Table 1. Most students (N=272; 75.3%) were concentrated in Conrad High School, Tasby Middle School, and McShan Elementary School while the remaining students were dispersed across six other schools. Table 1: Refugee Students Receiving Instructional Support by School School N % Conrad High School Tasby Middle School McShan Elementary School Hotchkiss Elementary School Lowe Elementary School Gill Elementary School Hill Middle School Jill Stone Elementary School Dan Rogers Elementary School Total At-a-Glance EA As shown in Table 2, more than half of the student participants were in secondary school (N=204; 56.8%). The largest number of participants were in ninth grade. Table 2: Refugee Students Receiving Instructional Support by Grade* School N % K Total *Data were missing for three students. Table 3 presents the characteristics of student participants. In , there were more male students than female, and a majority of the students (67.1%) came from Nepal, Burma, or Thailand. Almost all of the students were considered at-risk (94.7%), economically disadvantaged (93.1%) or English Language Learners (95.6%). Table 3: Characteristics of Refugee Students Receiving Instructional Support Demographics N % Gender* Female Male Special Populations At Risk** Econ. Disadvantaged*** English Language Learner Birth Country Nepal Burma Thailand Other *Data were missing for three students. **Data were missing for 19 students. ***Data were missing for 25 students. Other countries include Bhutan, Iraq, Congo, Malaysia, Somalia, Ethiopia, Iran, Kenya, and 15 other countries. 101

110 Program Activities The RSIG program provided services to refugee students and their families to improve academic performance and social adjustment as they transitioned into Dallas ISD. The grant focused on four main services: support services for students, school orientations for newcomer parents, professional development for Dallas ISD staff, and instructional support services for students. Support services for refugee students included: Social gatherings, including a social skills group, multi-cultural club, and an art club Academic and career counseling Translation and interpreter services for school-related purposes In the fall semester, RSIG staff members conducted an art club on October 25 for 43 refugee students. In the spring semester, 19 students attended a social skills group on March 6. The multi-cultural club was hosted five times as shown in Table 4. The RSIG program also provided various school supplies, including backpacks and uniforms (N=46), calculators (N=9), and dictionaries (N=4) to refugee students who were particularly low income. RSIG staff members also took refugee students on two college visits during the school year. A total of 33 students visited the University of Texas at Arlington on October 12, 2013, and eight students visited Richland College on March 22, Table 4: Refugee Student Attendance for Multi-Cultural Club Date N September 17, October 3, December 19, February 5, February 14, The RSIG program also included a four week summer program to teach literacy, language, and mathematics skills based on state standards. RSIG staff provided enrichment through thematic learning, cultural lessons, and field trips. The program was open to currently enrolled refugee students and students that were expected to enroll for the school year. There were 89 students that attended the summer program. Table 5 displays parent attendance for RSIG s newcomer parent orientations. RSIG staff conducted ten school orientations that introduced newcomer parents to the American education system and to Dallas ISD. Parents received a handbook, a school calendar, an example of a written absence excuse, and an application for a free school uniform. Table 5: Newcomer Parent Attendance for Orientation Date N August 14, October 10, December 3, January 21, January 31, February 11, Table 6 presents Dallas ISD staff attendance for RSIG s professional development sessions. RSIG staff conducted 11 sessions that covered a variety of topics, including cultural awareness, the refugee settlement process, refugee student support services, and instructional objectives. RSIG staff also assisted Dallas ISD s Teen Pregnancy Preventive Program with professional development. Table 6: Staff Attendance for Professional Development Date N August 13, August 21, August 29, September 24, October 7, October 15, October 22, October 28, November 5, November 12, January 17, 2014* 9 June 7, 2014** 12 *Indicates training for the Teen Pregnancy Preventative Program. **Indicates training for the Summer Enrichment Program In order to provide a wide variety of support and instructional services, RSIG staff partnered with external organizations. Throughout the school year, RSIG staff met with a total of 29 organizations. The majority of meetings were with the International Rescue Committee, Catholic Charities of Dallas, Catholic Charities of Fort Worth, and the Vickery Coalition Action Team. External organizations provided RSIG with additional translation services, teacher training, school supplies, and refugee student opportunities. Student Performance RSIG provided instructional support in an effort to increase the number of refugee students promoted to the next grade level, to increase attendance, and to improve academic achievement. Table 7 presents the promotion rates for students who received instructional support. Most of the students (84%) were promoted to the next grade. Of the 35 retained students, 20 (83%) were in ninth grade. 102

111 Table 7: Promotion Rates of Refugee Students* Promoted N % Yes No Total *Based on students with valid data Table 8 presents the average attendance of students for the fall, spring, and entire school year. Of the 358 students with valid data, 45 (12.6%) were enrolled for only one semester. Of the students enrolled for both semesters, attendance between semesters declined for 121 (42%) students and increased for 141 (45%) students. The remaining students (N=41; 13%) had perfect attendance and experienced no change between semesters. Table 8: Attendance Rates of Refugee Students* Semester N Rate Fall Spring School Year *Data were missing for three students. Tables 9 and 10 present State of Texas Assessments of Academic Readiness (STAAR) and STAAR End of Course (EOC) data for student participants. Passing rates ranged between 8.1 percent and 42 percent. Most students did not meet state standards on the STAAR 3-8 or STAAR EOC. Passing rates were highest in mathematics for both exams. Table 9: STAAR 3-8 Passing Rates Subject Tested (N) Passed (N) Passing Rate (%) Reading Writing Mathematics *Based on students with valid data Table 11: Grade Point Averages of Refugee Students* Grade N Mean Minimum Maximum Summary and Recommendations The RSIG program was created to ease refugee families and students into the American school system and Dallas ISD. Refugee families were provided with school supplies, support services, and guidance. Refugee students received instructional support to improve academic achievement, attendance, and promotion rates. The RSIG program met or exceeded projections for increased services. Student outcome measures were mixed. On average, most students were promoted to the next grade (84%) and had high attendance rates (97.2% mean attendance rate). Except for ninth grade, most students had GPAs within the passing range. STAAR results showed that most students did not meet the state standards for reading and mathematics. For more information contact Program Evaluation at Table 10: STAAR EOC Passing Rates Subject Tested (N) Passed (N) Percent Rate (%) Algebra English I English II *Based on students with valid data The evaluator also analyzed grade point averages (GPA) for high school student participants. Table 11 presents the GPAs for the 127 students with valid data by grade. The mean GPA for all 127 students was The mean was lowest for ninth grade students and highest for twelfth grade students. Ninth grade also had the largest range with the lowest minimum and highest maximum of the four grades. 103

112 104

113 State Compensatory Education Program: Dorothea Weir, Ph.D. EA At-a-Glance Compensatory education is defined in law as programs and/or services designed to supplement the regular education program for students identified as at risk of dropping out of school (At-Risk). The purpose of the State Compensatory Education (SCE) program is to increase academic achievement and reduce the dropout rate of At-Risk students through direct instructional services. The goal is to reduce any disparity in (1) performance on state-mandated assessments, and (2) high school graduation/ completion rates between At-Risk students and all other district students. Texas Education Code (TEC) defines the state criteria for identifying At-Risk students. 1 Students who meet any of the criteria are to be reported through the Public Education Information Management System (PEIMS) in the fall of each school year. Among the 13 criteria are students who are limited English proficient, students who failed a state assessment, or students who were retained. The district is required to submit the district improvement plan, two campus improvement plans, and the local evaluation of state compensatory education strategies, activities and programs to the Texas Education Agency (TEA) annually. The scope of the evaluation includes the program context, implementation, and results of the SCE Program It examined and summarized how SCE direct funds were used for supplemental services/programs, as shown in the district/campus improvement plans, project workscopes, and expenditures. 2 The report also identified the At-Risk population and examined how their TAKS/STAAR performance and graduation/completion rates compared to their Not-At-Risk peers. 3 Student Demographics In school year , a total of 95,137 students were identified as At-Risk (i.e., met one or more of the state criteria), representing 59.9 percent of the district s grades EC-12 enrollment (N=158,932). As shown in Figure 1, the percentage of At-Risk students 1 Texas Education Code (TEC) , available at m# Budget and expenditures information provided by Dallas ISD Budget Services. 3 The report summarized the performance results of At-Risk and Not-At-Risk students; it did not link student performance to specific programs or services. remained at about the same level from through , but decreased in and Percent At-Risk and Not-At-Risk 100% 80% 60% 40% 20% 0% 33.0% 34.4% 33.6% 38.2% 40.1% 67.0% 65.6% 66.4% 61.8% 59.9% % Not At-Risk 33.0% 34.4% 33.6% 38.2% 40.1% % At-Risk 67.0% 65.6% 66.4% 61.8% 59.9% Figure 1. SCE expenditures by Major Function. Table 1 presents the demographic composition of the At-risk student population. The majority of At-Risk students are male, Hispanic, and LEP. Table 1 At- Risk Student Demographic Characteristics, District. At-Risk. Subgroup N Percent N Percent Gender Male 81, % 50, % Female 77, % 44, % Ethnicity Hisp./Lat. 110, % 77, % Afr. Am. 37, % 14, % White 7, % 1, % Other 4 3, % 1,612 17% Eng. Prof. Non-LEP 95, % 32, % LEP 61, % 62, % Total 158, % 95, % SCE Programs/Services The primary records supporting expenditures attributed to the SCE program are the district and 4 Other includes Asian, American Indian or Alaska Native, Native Hawaiian or Other Pacific Islander, Two or More Races, and Not Available. 105

114 campus improvement plans, which are required by state law. SCE programs/services must be described in the Campus Improvement Plan (CIP) if implemented at the campus level, or in the District Improvement Plan (DIP) if implemented districtwide. District Improvement Plan The District Improvement Plan (DIP) was reviewed for SCE components. The DIP includes Accountability System, Needs Assessment, Action Plans, and Resources. Some SCE required components were present to some extent in the DIP: (1) Comprehensive needs assessment; (2) total amount of SCE funds allocated for resources and staff; (3) strategies to help At-Risk students; and (4) summative evaluation. 5 SCE funds were included in the Action Plans and Resources sections. The Board of Trustees did not set performance targets for because of the complete revision of the state testing and accountability system and the unavailability of passing standards for the STAAR for all grades. The district plan contained some general statements related to At-Risk students; it serves as the underlying framework for all Campus Improvement Plans. Campus Improvement Plans The most frequent strategies/activities identified for At-Risk students in the CIPs were tutoring and Summer School/Extended Year Programs. Other strategies and actions mentioned for assisting At-Risk students were mentoring, class size reduction, credit recovery, and instructional coaches to assist teachers. SCE Budget and Expenditures The SCE allocation for Dallas ISD was $149,054,435. The Direct Cost requirement was 52%. The district budgeted $91,080,386 to support programs/services for At-Risk students (Table 2). Actual expenditures totaled $86,080,025. Table 2 Summary of SCE Funding and Budget Description Amount SCE Allocation received $ 149,054, Required Min. Direct Cost (52%) $ 77,508, Budget for SCE PICs 24, 26, 28, 29, 30 $ 57,420, Budget for PIC 32 (Pre-K Pgm.) 6 $ 33,659, Total SCE Direct Cost Budget $ 91,080, Some individual evaluation reports for programs serving certain At- Risk students are available at Dallas ISD s website 6 The state requires that all PreK expenditures be coded in Program Intent Code 32, so the PreK budget and expenditures were not coded with the five SCE PICs. SCE programs/services are coded using five Program Intent Codes (PICs): PIC 24 - Accelerated Education PIC 26 - Non-Disciplinary Alternative Education Programs AEP Basic Services PIC 28 - Disciplinary Alternative Education Program DAEP Basic Services PIC 29 - Disciplinary Alternative Education Program DAEP SCE Costs PIC 30 - Title I, Part A Schoolwide Activities related to SCE costs on campuses with 40% or more economically disadvantaged students Prekindergarten (PIC 32) is not an SCE code, although SCE funds may be used under this code. Beginning in , a new PIC code, PIC 34, will be used for SCE Prekindergarten funds. Figures 2 through 4 show expenditures by SCE Major Function, Object, and Program Intent Codes. The major expenditures were Instruction & Instruction Related Services (81%), Payroll Costs (97%), and Title I Part A Schoolwide Activities (84%). Instruction & Related Svcs, $44,129,677, 81% Figure 2. SCE expenditures by Major Function. Payroll Costs, $52,640,278, 97% Figure 3. SCE expenditures by Object. Inst & School Leadership, $351,035, 1% Support Svcs Student, $9,693,133, 18% Support Svcs - Non-Student Based, $39,228, 0% Ancillary Svcs, $41,404, 0% Adm Support Svcs, $112, 0% Intergovernmental Chgs, $2,622, 0% Prof and Contracted Svcs, $681,432, 1% Capital Outlay, $52,606, 0% Supplies & Materials, $612,866, 1% Other Operating costs, $270,029, 1% 106

115 Title I Schoolwide Activities, $45,442,419, 84% Accelerated Ed, $3,765,554, 7% AEP Basic, $2,243,639, 4% DAEP Basic, $2,796,027, 5% Grade 11, TAKS: ELA and Math STAAR/TAKS Performance Figures 5 and 6 present the results for STAAR/TAKS Reading and Mathematics, respectively. Not-At-Risk students outscored At-Risk students at all grade levels for both Reading and Mathematics Figure 4. SCE expenditures by PIC. The district budgeted 1,653 Full-Time Equivalent (FTE) positions, including 767 Prekindergarten positions, with SCE funds, and filled 1,418. Most FTEs were core content teachers at elementary and secondary campuses or Prekindergarten teachers and teacher assistants. Table 3 SCE Full-Time Equivalents (FTEs) Organization Type Funded Filled Campus K Campus Pre-K Campus Total 1, , Central Organizations Total 1, , Table 4 SCE Filled Positions by PIC Program Intent Code (PIC) Campus Central 24 Accelerated Education Non-Disciplinary Alt. Ed. Pgm - Basic Disciplinary Alt. Ed. Pgm. - DAEP Title I Part A Schoolwide Activities* Prekindergarten Total 1, * On campuses with 40% or more economically disadvantaged students STAAR/TAKS Passing Rates Beginning in spring 2012, the TEA began to phase in the State of Texas Assessments of Academic Readiness (STAAR) to replace the Texas Assessment of Knowledge and Skills (TAKS). For the purpose of assessing SCE program results pertaining student performance during the transitional period of TAKS and STAAR, the following were analyzed according to grade levels for Grades 3-11: DAEP SCE, $9,572, 0% Grades 3-8, STAAR: Reading and Math Grade 9, STAAR EOC: English I Reading and Algebra I Grade 10, STAAR EOC: English II Reading and Geometry Passing Rate Not At-Risk% At-Risk% Figure 5. Reading/ELA passing rates by grade and At-Risk status. Passing Rate Not At-Risk% At-Risk% Figure 6. Mathematics passing rates by grade and At-Risk status. Graduation/Completion Rates Four-year graduation rates, the percentage of students who graduate on-time in four years, are used for federal accountability. For state accountability, districts were evaluated using an indicator known as completion rate. Completers include not only students who graduate in four years but also students who continue in high school for a fifth year. Figures 7 and 8 show longitudinal graduation and completion rates for the cohorts of Class of 2008 through Class of 2012 by At-Risk status. 7 Graduation and completion rates of At-Risk students are lower than those of Not-At-Risk students, resulting in gaps between the two groups. Nevertheless, graduation and completion rates of At-Risk students have steadily 107

116 increased, keeping pace with Not-at-Risk students over the last four years Figure 7. Graduation rates by cohort and At-Risk status Class of 2008 Class of Not At-Risk Class of Class of 2010 Not At-Risk Class of 2009 Class of 2010 Class of 2011 At-Risk Figure 8. Completion I rates by cohort and At-Risk status. Summary and Recommendations Class of Class of 2011 At-Risk Class of 2012 The goal of the SCE Program is to reduce disparities in academic performance and high school completion between At-Risk and Not-At-Risk students. The District Improvement Plan outlined improvement strategies that were broadly aligned with the purposes of SCE, targeting assistance to schools and student groups with the largest numbers of At-Risk students. In practice, however, SCE appears to be addressed primarily as a compliance program rather than a strategic instructional program. SCE funds seemed to be used most extensively as an additional resource for increasing teacher and staff positions. The same trend was found in the Campus Improvement Plans (CIPs). The majority of SCE funds expended at the campus level were directed toward Title I, Part A Schoolwide Activities. SCE funds can be used to reduce class size by hiring core content teachers, but only after the generally required studentteacher ratio has been met using non-sce funds. The CIPs should be more specific on whether and how SCE funds were used for class size reduction. As recommended in past reports, available SCE funds should be focused on providing more direct services to At-Risk students. 8 Beginning in , new legislation passed in House Bill 5 mandates that districts are required to budget and prioritize SCE funding to support accelerated instruction each time a student fails to pass an EOC assessment. 9 The bill also prohibits use of the funds for any other purpose until the district adopts a budget to support AI. Consequently, AI will necessarily become a primary service in grades At the other end of the grade sequence, there is increased district and state emphasis on Early Childhood education, particularly Prekindergarten programs. Thus, it is recommended that the district pay close attention to the effective use of potentially more limited SCE funds for grades 1-8. With the majority of SCE funds budgeted and spent at the campus level, Campus Improvement Plans necessarily must be more specific on strategies and activities delivering direct services to At-Risk students. These strategies should be aligned to the particular profile of At-Risk students at each campus and the reasons for their At-Risk classification (e.g., achievement, LEP status, prior grade retention, etc.). Since the district has a Dropout Early Warning System (DEWS) 10 for grades 6-12 in place, it is recommended that campus staff, especially principals and counselors, make best use of this system to aid selection of specific strategies aligned with the needs of their At-Risk students. At its current level of funding, with more than $57.9 million available in in indirect cost allocations intended primarily to defray program administrative costs, the SCE program should well be able to meet the expense of a designated program manager assigned the responsibility of overseeing program implementation and data collection, as well as evaluation services necessary to assess program effectiveness. It is recommended, as in previous years, that the district designate a full-time SCE program manager with responsibilities including, but not limited to: 8 The purpose of the SCE program is to improve student performance through direct instructional services to students at risk of dropping out of school. The more removed services are from the student, the more the resources are diluted and the more difficult it becomes for the school district to defend the use of the SCE funds and justify the effectiveness of the program in improving student performance. (ACET, 2012) 9 House Bill 5, 83 rd Legislative Session, added new TEC and amended TEC The Dropout Early Warning System is provided to alert principals to students who are at risk of dropping out. Students listed are currently failing two or more state credit core courses. A student may also have been referred for discipline or truancy problems, and number of TAKS/STAAR/EOC failures. Cf. /PORT/DEWS/Default.jsp 108

117 Familiarity with SCE program requirements and the TEA Financial Accountability System Resource Guide (FASRG). 11 Access to current information and requirement updates for SCE from the annual Association of Compensatory Educators of Texas (ACET) conference The information should be reviewed and provided to district department heads impacted by SCE. Oversight of the intent, purposes, and requirements of the SCE program, including compliance with budget allocation and expenditure requirements. Knowledge of effective interventions sufficient to provide assistance to campuses and central office program managers in selecting appropriate services and programs to meet At-Risk student needs. Responsibility and authority to coordinate all aspects of the SCE program and act as a liaison with other district functions to collect, maintain, and provide data for records and reports. Maintenance and provision of required documentation at the central level. Monitoring of implementation and student performance, and recommending funding changes for programs and services not effectively meeting student needs. Reporting of program results to appropriate administrative levels. It is recommended that the existing student records system be expanded to track student participation in specific programs and services. Documenting and tracking student program participation will allow more accurate, thorough, and informative evaluation of the impact of these programs on the academic performance and high school completion of At-Risk students. For more information, see EA , available at 11 TEA Financial Accountability System Resource Guide (FASRG) is available at /index4.aspx?id=

118 110

119 Evaluation of the Reasoning Mind Mathematics Program Joan Bush, Ph.D., Myoungsook Kim, Ph.D. EA At-a-Glance In , 35,965 Dallas Independent School District (ISD) students were enrolled in the Reasoning Mind (RM) computer-based mathematics curriculum. RM was available to second-grade students in , expanded to third grade in , and added to fourth grade in RM curriculum was supplemental to students regular mathematics instruction. The purpose of this report is to summarize context, implementation, and outcome findings. Methodology The evaluators developed the Dallas ISD Reasoning Mind Theory of Change Model to guide the evaluation. 1 Context. Information was gathered from internal documents, the RM web site, and interviews. Student data was extracted from the October 2013 Public Education Information Management System file. Implementation. Qualitative analyses were conducted to analyze the School Leadership Information Form and RM weekly report comments. Frequency and descriptive analyses were carried out to examine teacher training completion and student use of RM. 2 As for spring campus staff surveys, quantitative and qualitative analyses were conducted. Outcomes. Frequency and descriptive analyses were conducted for the Assessment of Course Performance (ACP), Iowa Test of Basic Skills (ITBS), and State of Texas Assessments of Academic Readiness (STAAR). Correlation coefficients were computed to note the strength of relationships between test scores and RM indicators. Crosstab analyses were used to study the link between time, accuracy, and achievement. Two-level HLM analyses were conducted to find out whether student- and teacher-level RM implementation indicators were significant predictors of students test scores when students (Level 1) were nested within teachers (Level 2). Multiple regression and Cohen s f analyses were computed to determine statistically or practically significant differences between RM and non-rm fourth-grade students 2014 STAAR mathematics raw scores when controlling for prior 1 The model posits that if RM components are implemented with fidelity, students will improve on mathematics achievement. 2 RM s fall student data extract did not match the end of year extract. As a result, data in this report differ from data in the interim report. Approximately 1,300 students were missing from the fall file. achievement (STAAR 2013) and economic status. Frequency and descriptive analyses were carried out to review differences between fourth-grade students with one, two, and three years in RM. Limitations of outcome analyses included (a) lack of a control group for grades two and three and (b) size differences and possible selection bias between fourth-grade RM and comparison groups. RM Context for In , second and third graders at 147 schools and fourth graders at 130 schools were enrolled in RM. There were 35,965 RM students, which included 12,980 (95%) of the district s second-grade students, 12,537 (95%) of the district s third-grade students, and 10,448 (83%) of the district s fourth-grade students. The budget of $1,944,320 included $1,419,320 of Title I, Part A funds and $525,000 of Title II, Part A funds. One RM teacher per campus was supported. All teachers completed the Qualification Course (QC); supported teachers received added training (12 hours), observations, and support. Central administrators created implementation tools and a training schedule that ensured most teachers completed the QC at the onset of the school year or before. Three implementation targets for were set: (a) Teachers will complete the RM QC before or at the beginning of the school year, (b) Students will spend at least two hours a week on RM (60 hours for the year), and (c) Students will answer 75 percent or more of RM Guided Study Level A problems correctly. Staff Participation in RM Training The teacher completion rate of the QC course was greatly improved for fall 2013 (90%) in comparison to fall 2012 (47%). Likewise, most supported teachers (90%) met the professional development requirement. There were 192 campus administrators from 136 elementary campuses that attended RM training. Results of RM classroom observations showed that most supported teachers improved over time and reached proficiency by the third observation. Reasoning Mind Student Use Results of the School Leadership Information Form yielded three barriers: (a) technology access issues, (b) teacher difficulty meeting the two-hour weekly requirement, and (c) lack of alignment between RM 111

120 and district curriculum. The most-cited barriers on RM weekly reports related to RM and district testing time that interfered with RM time, technology issues, technology access/scheduling problems, schedule changes (holiday weeks/school events), and teacher turnover/leaves of absence. Computer on Wheels (COW) carts were ordered for all campuses; however, most of the rollout occurred during second semester. Overall implementation improved from to ; however, analyses showed variation within schools and within grade levels at the same schools. As seen in Figure 1, mean hours increased from fall (18.19) to spring (30.84), but the average (49.03) was roughly 11 hours below the 60-hour target. About a quarter (28%) met the district target, whereas a sizeable percentage logged (41%) or (25%) hours (see Figure 2). Figure1: Mean Hours Students Spent on RM (Target=30 Hours per Semester and 60 Hours per Year) M e a n Figure 2: Percentage of Students in RM Hour Categories for by Grade (Target=60 Hours) P e r c e n t a g e nd Grade 3rd Grade 4th Grade All Fall Spring Year 2nd Grade 3rd Grade 4th Grade All Hours Hours Hours Hours Hours Hours Hours Hours Most third (72%) and fourth graders (67%) met the accuracy target (75% or higher) but few second graders (19%) did so; however, second-grade mean accuracy increased from fall (56.75%) to spring (67.86%). Campus Staff Perceptions of RM Survey results were mostly positive, and overall, supported teachers were more positive than non-supported teachers and campus administrators. More than half of campus staff agreed or strongly agreed that they would like to continue using RM next year (59% to 83%) and that students benefited from RM (62% to 92%); however, notably fewer administrators (42% to 49%) than teachers (52% to 82%) indicated RM helped teachers be more effective or would recommend RM to others (see Figure 3). Figure 3: Percentage of RM Staff Members that Agreed or Strongly Agreed with Selected Survey Satisfaction Items Most campus staff members viewed RM training and resources as helpful or very helpful (91% to 100%) and were satisfied or extremely satisfied (72% to 100%) with RM, campus administrator, mathematics campus instructional coach, and campus-based technician support (60% to 81%). Results were mixed for other support. Other findings follow. Campus administrators reported more frequent administrator to teacher and supported to non-supported teacher communication (54% to 79%) than supported (34% to 63%) and non-supported teachers (35% to 49%). A fifth or more frequently or very frequently faced problems with COW carts (39% to 49%), computers not holding a charge (26% to 33%), wireless connectivity (26% to 35%), network (26% to 32%), and student logins (19% to 25%). Some noted frequently or very frequently having limited access to computers (22% to 25%), problems with computers in a wired lab (22% to 24%), and schedule conflicts (18% to 20%). P e r c e n t a g e Would like RM on campus next year On open-ended items, the most-cited successes were increased mathematics/problem solving skills (n=153), greater student engagement (n=122), new STAAR modules (n=116), and RM being a helpful resource (n=64). The most frequent challenges were computer malfunctions (n=186), difficulty meeting time requirements (n=112), RM not aligned with state/district curriculum (n=79), scheduling conflicts (n=71), and limited computer access (n=66). The two main suggestions were to increase the number of computers (n=151) and to align RM to district/state curriculum (n=104). Student Mathematics Achievement Outcomes The percentage of students that performed at grade level or above on spring mathematics assessments varied. More second (76.1%) than third (56.5%) or fourth graders (62%) passed the spring 2014 ACP. 52 RM helps teachers be more effective Students benefit from RM Would recommend RM to others Administrator Supported Teacher Non Supported Teacher 112

121 About half of second graders (52.9%) performed at the 40 th percentile or higher on ITBS. Almost 60 percent of third- (59.6%) and fourth-grade (58.4%) students met the standard for STAAR. Results of correlation analyses showed that students spring mathematics assessment scores were highly correlated with Guided Study Level A accuracy (0.57 to 0.72). Correlations between objectives completed and achievement scores were medium in size (0.37 to 0.49), whereas correlations between hours online and test scores were small (0.12 to 0.22). Although accuracy was most important, time online was significant as well. The majority (68% to 97%) in all time categories with accuracy rates at 75 percent or higher performed at grade level or above on 2014 mathematics assessments. Of district students that met the time target of 60 hours or more (28%), most (63% to 67%) performed at grade level, and of students with between 40 and hours (41%), over half (52% to 60%) performed at grade level. Results of preliminary two-level HLM analyses showed that teacher assignment was influential on student test scores; 17 to 20 percent of total variance in students mathematics achievement was accounted for by teacher-level factors. At the teacher level, after controlling for student-level variables, classroom-level accuracy was a positive predictor of test scores; the number of objectives completed was a positive predictor for third grade. As for the student level, after controlling for teacher-level variables, the three largest predictors were accuracy, number of objectives completed, and prior achievement (see Table 1). Table 1: HLM Results for Predicting RM Students Achievement Predictors Grade 2 Gamma Grade 3 Gamma Grade 4 Gamma Intercept Teacher-Level Predictors Aggregated Accuracy Aggregated Objectives Completed Student-Level Predictors Accuracy Objectives Completed Prior Achievement Gender Economic Status Ethnicity (African American) Ethnicity (Hispanic) Grade 2 Variance Grade 3 Variance Grade 4 Variance Level 2 (u0j) Level 1 (eij) Note. Number of students=11,610, 11,087, and 9,617 (grades 2, 3, and 4, respectively); Number of teachers=477, 378, and 271, respectively. ITBS NCE scores were used for second grade. STAAR raw scores were used for third and fourth grade. Reference groups were female for gender, not economically disadvantaged for economic status, and White for ethnicity. 1 p<.05 2 p<.001. Results of fourth-grade regression analyses showed that, when accounting for prior achievement and economic status, students in non-rm schools outperformed students in RM schools. Differences were statistically significant (β=-0.02, p<.01) but not practically significant (Cohen s f=0.03). Fourth-grade longitudinal results showed that average STAAR raw scores increased slightly as years in RM increased. Means were closest for students in RM for two (29.7) versus three (30.6) years; on average, three-year students answered one more item correctly than two-year enrollees. Differences between students in RM for one year (26.3) versus two (29.7) were larger; the lower one-year student averages could be due to students being new to Dallas ISD and not solely due to having only one year in RM. For three-year students that logged at least 10 hours each year, most (62.6%) met the STAAR Level 2 Satisfactory standard. Recommendations Continue to monitor student hours and accuracy. Like last year, outcome analyses showed that accuracy rates were more strongly related to student achievement than other RM indicators. This does not mean that time spent online is unimportant. Rather it means that when students spend quality time that allows for mastery of material (completion of objectives with high accuracy), they have the best chance of doing well on mathematics achievement measures. Work with RM to include year-to-date information on weekly and monthly reports; continue to have RM include the teacher metrics that were added to weekly reports. To help administrators monitor implementation, ensure that year-to-date data is included for each school overall and by grade for district hour and accuracy targets. Year-to-date information for each school should include (a) the percentage of students by grade level that attained the hour target and (b) the percentage of students by grade level that met the accuracy target. Teacher-level year-to-date data should be added as well. Partner with RM to improve fall semester startup delays and to help campuses work through barriers that affect students time online. Analyses of RM hour data showed that average hours were below the district target of two hours per week for most weeks in and that it took about nine weeks before student RM hours began to stabilize. Review of staff surveys, the School Leadership Information Form, and RM weekly reports revealed several barriers that precluded implementation. As a result, RM and district staff members need to work together to help schools be RM ready earlier in the fall and to overcome barriers that negatively affect RM implementation. Increase in-person communication between RM, Teaching and Learning, School Leadership, and Information Technology staff members. An initial meeting was held in the fall that included staff 113

122 members from RM, Teaching and Learning, and School Leadership; however, follow-up meetings were not held. It will be important that staff members involved with RM communicate regularly, so that more efficient planning, campus support, and joint problem solving takes place all year. Strengthen in-person campus communication (administrator to RM teachers, supported to non-supported teachers, RM Implementation Coordinators to all campus staff) to ensure teachers review RM data, receive ongoing feedback, share training information, and brainstorm solutions to challenges. On surveys, many more administrators than teachers reported the frequent occurrence of monthly campus meetings, discussions of RM weekly and monthly reports, administrator walkthroughs, and supported to non-supported teacher collaboration. Ongoing communication will be important to overcome challenges and to ensure student success. Continue to ensure that first-year RM teachers complete the RM Qualification Course prior to or at the onset of the school year. By working with RM to develop a summer/early fall semester training schedule and requiring that teachers complete the QC in a timely manner, the majority of district teachers were able to use RM with students at the beginning of the school year. This emphasis on providing and requiring early QC training will be important as Dallas ISD strives to implement RM effectively. Provide teacher guidance for how to simultaneously meet RM and district curriculum requirements. Results of the School Leadership Information Form and campus staff surveys showed that some district staff members were concerned that RM and district/state requirements were not aligned and that time spent on RM kept teachers from covering the district curriculum. As a result, it will be important that teachers receive suggestions for meeting both RM and district/state curriculum requirements. Disseminate ongoing, systemic RM campus staff updates that emphasize ways to increase RM implementation fidelity and effectiveness. For most RM teachers, no formal training occurs after the initial course. As a result, campus staff members could benefit from follow-up information such as reminders of important information taught in training, tips, and updates. Topics could include student data monitoring, intervention strategies, ways to motivate students, tips for helping students develop grit (rather than giving up), reminders of RM and district resources available to support RM implementation, the importance of meeting time and accuracy targets, and so forth. Prior to future expansion of RM, ensure adequate time to budget, order, and roll out additional technology. Based on campus survey results, the most frequent implementation barrier was computer malfunctions/ outdated technology, and the main suggestion was to increase the number of working computers. COW carts were ordered for all campuses and rolled out during ; however, most of the rollout occurred during the second semester. It will be important that Information Technology staff members are included in expansion planning meetings, so that administrators can adequately budget, order, and get new technology in place prior to the start of the school year. References Bobko, P. (2001). Correlation and regression: Applications for industrial organizational psychology and management. Thousand Oaks, CA: Sage Publications. Bush, J., & Kim, M. (March 2013). Preliminary evaluation findings for the third-grade Reasoning Mind Mathematics Program: First semester implementation and outcome data. Dallas, TX: Dallas Independent School District. Bush, J., & Kim, M. (September 2013). Evaluation of the Reasoning Mind Mathematics Program: Final report. Dallas, TX: Dallas Independent School District. Bush, J., & Kim, M. (April 2014). Evaluation of Reasoning Mind: Implementation and outcome findings. Dallas, TX: Dallas Independent School District. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2 nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Hox, J. J. (1998). Multilevel modeling: When and why. In I. Balderjahn, R. Mathar & M. Schader (Eds.). Classification, data analysis, and data highways, New York: Springer Verlag. Maas, C.J.M. & Hox, J.J. (2004). Robustness issues in multilevel regression analysis. Statistica Neerlandica, 58, Raudenbush, S. W., Bryk, A. S., Cheong, Y. F., Congdon, R., & du Toit, M. (2011). HLM 7: Hierarchical linear and nonlinear modeling. Lincolnwood, IL: Scientific Software International. Raudenbush, S.W., & Bryk, A.S. (2002). Hierarchical Linear Models, (1 st ed.) Thousand Oaks, CA: Sage Publications. Reasoning Mind. Web site: Snijders, T., & Bosker, R. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling, 1st edition. Thousand Oaks, CA: Sage. Tabachnick, B.G., & Fidell, L.S. (2001). Using multivariate analysis. Boston: Allyn and Bacon. Waxman, H.C., & Houston, W.R. (February 2012). Evaluation of the Reasoning Mind Program in Beaumont ISD. Unpublished manuscript. The full RM final evaluation report can be found at For more information, contact Program Evaluation at

123 Special Education Program: Robert Costello, Ph.D., Jiangang Xia, Ph.D. EA At-a-Glance The Special Education program provides support and services for students with disabilities. The academic performance of these students is monitored primarily by two systems of accountability, the Performance-Based Monitoring Analysis System (PBMAS) and Adequate Yearly Progress (AYP). In PBMAS, a performance level (PL) ranging from 0 (met standard) to 3 (significantly below standard) is assigned to each indicator. This report evaluated trends, in three main areas related to PBMAS and AYP: the over-representation of African American students in special education, disproportionate placements of special education students to out-of-school suspension, and the performance and participation of special education students on state assessments. Program Participants The special education population for the district, as of October 28, 2013, was 11,937 students or 7.5 percent of the total population of 159,760. The special education representation of the two most populous ethnic groups within the district, Hispanic and African American, was monitored through PBMAS. Hispanic students have typically been under represented in special education. That is, fewer Hispanic students were classified as special education students than were expected based on the population of the district as a whole, while African American students typically have been over represented in special education. The district received a PBMAS performance level of 3 for the school year on this measure. In addition, although males and females were equally represented in the district total population, the majority of special education students were male (67.6%). Thirty-four percent (34%) of the special education students were also English language learners (ELL), while 41 percent of the district students were ELLs. The out-of-school suspension (OSS) indicator for special education students was first reported through PBMAS in with performance levels first assigned in (Table 1). The difference rate steadily increased from to , at which point the district received a performance level of 2 for this indicator. Since then the difference rate declined each succeeding year from 19.5 to 10.2 in , lowering the district s performance level to 1. However, data indicated a difference rate of 17.2 which raised the district performance level to a 2. Table 1. PBMAS Performance Level of Out-of-School Suspensions Special Ed. District Difference Rate Perf Level N/A N/A Trends in Academic Performance ITBS/Logramos District students showed losses from one year to the next for grades KN, 1 and 2 on ITBS Reading Profile and Total. Special education students showed gains in kindergarten and grade 1. Kindergarten special education students showed a ten percentage point gain on ITBS Reading Profile. District students showed losses from one year to the next for grades KN, 1 and 2 on ITBS Mathematics Total. Special education students showed increases in grades KN and 2. Logramos is the Spanish language version of the ITBS. District students showed losses from 2013 to 2014 for all grades, while special education students showed a gain in kindergarten. STAAR Special education students did not reach state special education averages across all grades for STAAR Reading. Dallas ISD special education students were outperformed on STAAR Mathematics test when compared to comparable state groups. STAAR Writing scores for special education students showed lower percentages than comparable State groups. Fourth grade special education students performed better than seventh grade special education students in writing. Comparisons with State STAAR Science percentages showed that district scores were lower than State scores for special education grade 5 students. However, grade 8 special education students performed at approximately the same level as State special education students (31% vs. 32%). STAAR Social Studies results showed Dallas ISD grade 8 special education students outperforming State special education students. Another way to compare STAAR results was to examine progress from one year to next year. In grades 4, 6 and 8 special education students show year-to-year progress on STAAR Reading. Special 115

124 education students showed progress in mathematics across all grades with the exception of grade 7. In four cases, the special education students showed a decrease from one year to the next when State scores showed an increase (grades 5 and 7 reading, grade 4 writing and grade 5 science). Overall, 57 percent of the STAAR test measures examined showed improvement from to for special education students. Figure 1 shows the percent of special education students who took the various versions of the STAAR test. Almost 50 percent of the special education students took the STAAR Modified versions of the reading, mathematics, writing, science and social studies tests. Approximately 35 percent took the regular STAAR tests and 15 percent took the STAAR Alternate. Figure 2 showed a greater percentage of special education students reached the Satisfactory level 2 on the STARR Modified version of the tests than on the regular STAAR and/or STAAR Alternate tests. Special education student performance was highest on STAAR Modified Social Studies (60.4%) and lowest on STAAR Writing (18.4%) and STAAR Social Studies (17.4%). Figure 1: Percent of Special Education Students Tested on STAAR, STAAR Modified and STAAR Alternate Science Social Stu Writing Math Read Percentage of Students Tested on STAAR, STAAR M, and STAAR ALT STAAR STAAR Mod STAAR ALT Figure 2. Percent of Special Education Students who Attained Satisfactory on STAAR, STAAR Modified and STAAR Alternate STAAR STAAR Modified STAAR Alternate STAAR End-of-Courses (EOC) STAAR EOC results showed gains in Algebra I and Biology for special education students from to District special education students performed at a lower level than special education students statewide (Figure 3). English I, English II and U.S. History were given for the first time in 2014; therefore, no past year comparisons could be made. The district and State students were closest on the Biology EOC and within eight to nine percentage points on English I and II EOCs. History showed a three point difference and Algebra had a five point difference. Figure 3. Special Education Percent Satisfactory EOC STAAR EOC % Satisfactory English I English II District U.S. History State Alg I Biology RECOMMENDATIONS Examine the use of the Instructional Setting/Assessment report showing participation rates on STAAR and STAAR-M and the reasons for the increase in students taking STAAR-M, in particular, whether there were changes in students instructional arrangement or curriculum requirements to justify the increase. Given the high percentage of special education students (approximately 50%) who took the STAAR Modified version of the tests, the special education department needs to prepare teachers, LPAC members and principals to the fact that the modified versions of the STARR tests will not be available next year. As with disproportionality, response treatment interaction (RTI) training completed by campuses should be documented with state testing results serving as an additional outcome measure. Within the framework of RTI, teachers should be trained to respond to environmental needs of children with specific disabilities. For more information, see EA , which can be found at 116

125 College Access Program: Nora E. Douglas, Ph.D. EA At-a-Glance The College Access Program (CAP) was created to meet Dallas ISD needs for increased student college applications, scholarship applications, participation and performance on college entrance exams, and parental involvement in the college admissions and financial aid process. The three College Access (CA) program vendors selected for the school year were Education is Freedom (EIF), Academic Success Program (ASP), and The Princeton Review (TPR). The program served students in high schools receiving Title I funds. Vendors collaborated with campus counselors, other college access advisers, and other college access program staff to ensure a seamless implementation for students. Vendors provided supplemental, individual, and small group assistance to students and parents. During the school year, Education is Freedom (EIF) served 20 Dallas ISD campuses and reported serving 6,487 students on an individualized basis, Academic Success Program (ASP) provided one-onone services to 1,352 students across seven campuses, and The Princeton Review (TPR) served seven campuses and 2,233 students. Figure 1 shows the allocation of campuses by provider. Figure 1: Percentage of Campuses by Provider TPR, 21% ASP, 21% EIF, 59% Title I program cost requested for this component was $2,300,000. Evaluation Purpose and Scope The scope of this evaluation was to evaluate the effectiveness of the program based on outlined program goals and objectives, including the extent to which vendors served students and parents individually and in small groups, and the extent to which vendors met program goals. Program Goals The expected program goals were that 98 percent of seniors should have an active Career Cruising account and have taken an approved aptitude test; 80 percent of seniors would submit at least one college application; 50 percent of seniors would complete a financial aid application; 90 percent of juniors and seniors would take the SAT or ACT; 80 percent of seniors would receive college or scholarship essay assistance; 50 percent of seniors would attend at least one college fair; and six Postsecondary Counselor Training meetings (including vendors) would be held during the school year. The expected outcomes were for counseling staff and vendor staff. Vendors and counselors were expected to work collaboratively to meet program goals. The goals stated above were aligned with the goals set by the Counseling Services department. Vendor staff and campus counselors met at the beginning of the school year to divide up the work necessary to meet campus goals. They completed and signed a College Access Program Vendor Goal Form as a result of that meeting. Student Services Individual and Small Group Table 1 shows the percentage of all students and seniors served in CAP during the school year. Table 1: Number and Percentage of Students Served Individually, All Students and Seniors Only Total Student Population (9 th - 12 th ) Seniors Only Served in CAP N % N % Not Served 25, , Served 10, , Total 35, , ASP students were most often seen for Apply Texas application assistance, followed by information about financial aid. EIF students were seen for Apply Texas application assistance, followed by information about applying to college. TPR students were most often seen for registering for the SAT or ACT, followed by assistance with Apply Texas applications. On average, EIF and TPR students were served about three times per student. ASP students were served almost nine times each, though ASP had a much higher maximum number of visits, which skewed their average. Most EIF and TPR students had one or two visits with a CAP advisor. About 45 percent of ASP students had one or two visits. Another 23 percent of students were seen 10 or more times. In previous school years, CAP vendors were permitted to hold large student and parent workshops. For the school year, vendors were instructed to 117

126 provide only small-group sessions. About 87 percent of ASP workshops had 25 participants or fewer. About 85 percent of EIF workshops had 25 or fewer participants. About 97 percent of TPR workshops had 26 participants or fewer. Some workshops had participation of up to 502 students and parents. These data indicate that vendors were not always adhering to this policy. Parent Services Small Group Workshops included parent only (n=5) and parentstudent workshops (n=26). ASP most often provided college and financial aid information (about 73% of workshops), while EIF and TPR provided assistance with college and financial aid applications (about 63% and 60%, respectively). The results of a vendor survey indicated that reaching parents to provide College Access Program information seemed to be an area for future focus (Figure 2). Vendors commented that parents often did not attend college access information sessions. In addition, language barriers needed to be addressed. Some vendors indicated that they had had more success working with parents in small groups while others reported one-on-one assistance was more successful. Figure 2: Vendor Quotes: Individual Student Assistance Vendor Quotes: Small Group Parent Assistance Parents have been really great at digesting information and asking questions. I have seen much more success explaining information, such as financial aid, to parents in small groups. Many of our parents do not participate, so small groups with parents during financial aid nights were a success because I was able to offer more one-on-one attention, which they appreciated. We are working tirelessly to try to reach as many parents as possible but sometimes the language barrier makes that difficult. It is very hard to get parents to attend parent sessions and or/workshops so that has been a big issue for me this year. Meeting Program Goals Obama Male Leadership Academy and A. Maceo Smith New Tech High School were not included because they did not serve seniors during the school year. Career Cruising: With the exception of two campuses, all campuses (94%) met the goal of 98 percent of seniors having a Career Cruising account. The Career Cruising database showed no evidence of students taking the available aptitude test. College Applications: All but one campus (Townview Business) met the goal of 80 percent of seniors submitting an application (94% met). Data for Patton Academic Center were not available through ApplyTexas. Self-report data indicated 56 percent of Patton seniors submitted a college application. Financial Aid Applications: Twelve of the 32 campuses (38%) met the goal of 50 percent of the senior class submitting a FAFSA. College Entrance Exams: During the school year, Dallas ISD funded school-day testing on the SAT for all juniors. Dallas ISD administered the SAT to juniors on February 26, As a result, the percentage of juniors who took the SAT or ACT was higher than for seniors. Twenty-one of the 34 campuses (62%) had 90 percent or more of their juniors take the SAT or ACT exams. Two campuses had 90 percent of more of their seniors take the SAT or ACT exams: Irma Lerma Rangel Young Women s Leadership School (92%) and Dr. Wright L. Lassiter Jr. Early College High School at El Centro College (98%). Scholarship Assistance: None of the campuses reached the goal of providing college and scholarship essay assistance to 80 percent of seniors; however, Lincoln High School had a higher percentage of students served compared with other campuses (58% assisted with scholarship essays). Lincoln counselors and vendors implemented a targeted approach to scholarship essay assistance during the school year. College Fairs: Due to the method in which data is collected for the College Access Program, the evaluator was unable to determine whether 50 percent of seniors attended at least one college fair. The vendors are not required to record individual attendance at college fairs. Instead, they record overall attendance and fair dates. Postsecondary Counselor Training Meetings: The Counseling Services department held six training meetings during the school year covering college placement exam updates, college readiness goals, graduation requirements, Special Education updates, House Bill 5 information, scholarship reports, financial aid information, and dual credit updates. Recommendations The following are recommendations based on the results of the evaluation: identify which aptitude test students should take and develop procedures to ensure students have access to the test, review best practices with regard to college and scholarship essay assistance to increase the percentage of students served, clarify policies and procedures regarding small-group workshops, and further clarify expectations of the College Access Program Goal Form meeting. For more information, see EA , which can be found at 118

127 Academic Facilitators Program: Leonardo R. Ledezma, Ph.D. EA At-a-Glance The Academic Facilitators (AF) program, as well as the Campus Instructional Coaches (CIC) received Title I and general operation funding during this school year. The Academic Facilitators program budget consisted of Title I and general operation funds designated as payroll costs. Budget information is found in Table 1. Table 1. Budget for Instructional Coaches Program, Budget Source and Code Academic Facilitators (6100) 8,070,216 5,271,665 4,658,765 CICs Stipend (6100) 824,334 Total 8,070,216 5,271,665 5,483,099 Note: Budget quantities for Academic Facilitators include professional development. There were 54 AFs and 390 CICs. The report focused on AFs primarily. The role of the AFs was determined by several organizational factors. According to division coordinators, the AFs role evolved during the school year. At first, AFs established communication with principals and CICs to share information, arrange schedules, and outline possible support. Principals determined what teachers were to get CICs support. According to the guidelines, AFs interventions were monitored and directed by the principals, who drove communication with both AFs and CICs. Later on, campuses were tiered and this determined AFs schedules. Any further requests, as well as ongoing support, were coordinated and supervised by campus leadership. Changes or adjustments in support were always a product of consultation with CICs and campus leadership. In the current school year, , the program has the following main aspects: Coaching: refers to the process in which the AFs participated in the instruction by performing classroom observations with the CICs to advise teachers about instructional techniques. This aspect of the program also included observing the coach coaching. The Coaching aspect also required observing the coach providing professional development. Teacher Teams: AFs worked with CICs who in turn worked with teachers to form teams. Sometimes, depending on the characteristics of the teachers and the student population in a school, teachers participated in co-teaching. AFs also supported CICs in meeting facilitation, team planning and monitoring team effectiveness when they interacted with teachers in their schools. Professional Development: Professional development included coplanning professional development and co-facilitating professional development with CICs. Support Planning: This aspect referred to helping CICs and teachers in lesson planning and included support planning, monitoring support plan, and helping access additional resources for instruction. Data Driven Instruction: The FAs provided support to CICs in assessment development, data conversations and analysis, instructional technology support, identifying target areas based on data analysis, and other content specific support. Other Duties: these refer to any other duties necessary to support instruction. AFs and CICs kept records of their activities during the year. Records reflected the number of hours spent in each component. AFs summary of the days they spent at schools by six-week periods were presented in Figures 1 and 2. The number of days AFs devoted to schools varied according to division and school needs. Divisions 1 and 2 AFs seemed to work more intensely during the first two six-week periods and recorded lower number of days during the rest of the year. Divisions 3, 4 and 5 appeared to have a more even spread of days during the school year. Figure 1: Summary of Days at Schools of AFs by Six Weeks Period Division Figure 2: Summary of Days at Schools of AFs by Six Weeks Period Division

128 When Academics Facilitators (AFs) efforts were observed by component of the program it was clear the largest number of hours in all five divisions was Professional Development. Figure 3. Summary of Yearly Days by Program Component Other Duties Data Driven Instruction Support Planning Professional Teacher Teams Coaching This evaluation attempted to tease out the effects of various components of the Academic Facilitators program on student achievement. The difficulty with this approach was in determining the effect that Academic Facilitators had on Campus Instructional Coaches, who in turn coached the classroom teachers and who then had an effect on the students performance. Standardized coefficients were used to allow for the direct comparison of effect size for each component of the program on the student scores in the different content areas for STAAR and STAAR EOC testing. The five major components of the program included Coaching, Data Driven Instruction, Instructional Technology, Professional Development, and Teacher Teams. Comparisons were made to determine the contribution of each component to the students STAAR performance, based on the size of the coefficient. It is important to highlight that statistically significant standardized coefficients were due in part to large sample sizes, and that all coefficients presented were significant Also, the adjusted R 2 which indicates the amount of variance in the STAAR test scores attributed to that component of the program was very small. Due to these factors conclusions about the effects of the various program components on student achievement could not be made. It was found that different components of the Academic Facilitators program can have both positive and negative effects. This was apparent when comparisons were made across divisions. Since the same component had greater or lesser effects on student achievement by division, one is led to conclude that the program varied greatly by division and in some cases by grade level. For STAAR EOC results indicated that Coaching had positive effects on English I, English II and US History, but had negative effects on Algebra I. Data Driven Instruction had negative effects on English I and English II but positive effects on Algebra I. Instructional Technology appears to affect Algebra I only. The overall Title I report presents results for STAAR and STAAR EOC. It is clear that, in general, the percentage of students in grades 3-8 meeting Satisfactory level on the reading portion of STAAR decreased in , while mathematics increased. The program goal of increasing scores in reading, mathematics and writing by a minimum of 5% was not met. In all fairness, the district mirrored the State performance on STAAR with a few exceptions. This could explain why the AFs program appeared to have very little impact on students achievement. Summary The program was implemented following new guidelines during A new structure and reporting line was created to serve teachers. The CICs were added to the process and the AFs were removed from the direct contact with the teachers. The campus leadership was in charge of overseeing and coordinating the work of AFs and CICs. Academic Facilitators performed more of a support role to the coaches than to teachers, as opposed to what had been done in previous years. Multiple regression analysis showed the effects of the different program aspects on student achievement. In most cases, for STAAR grades 3 through 8, the effects were small and varied by division. The size of the Adjusted R 2 makes it difficult to discern a clear pattern of effects. Recommendations Better supervision should be applied to the record keeping process. It is recommended to maintain accurate records of all activities and record relevant information for a proper program evaluation. There is a disconnection between AFs and CICs in information record keeping. Furthermore, according to AFs, it is expected that in each division will put into practice a different version of the program. This situation will create even more disarray in its implementation eliminating standardization and comparability. Uniformity and fidelity of implementation are keystones for the success of any program. This case is not an exception and it is important that the guidelines of the program design are followed. The district should consider defining standard guidelines to maintain uniformity of the role of AFs and CICs as well as the goals of the program. Additional information may be obtained by consulting the Instructional Coaching Program report, EA , which can be found at 120