www.wiscape.wisc.edu www.avidtops.org 2012-2013 DISTRICT FINDINGS: ANNUAL REPORT CONTENTS SECTION 1. MADISON METROPOLITAN SCHOOL DISTRICT (MMSD) REPORT... 2 Program Implementation and Certification Status as of 2012-13... 2 Program Demographic and Enrollment Information... 3 Program Retention Rates... 3 SECTION 2. BOYS AND GIRLS CLUB (BGC) OF DANE COUNTY REPORT... 4 TOPS Out-of-School Programming... 4 College Continuation Activities... 5 SECTION 3. WISCONSIN CENTER FOR THE ADVANCEMENT OF POSTSECONDARY EDUCATION (WISCAPE) EVALUATION OF STUDENT OUTCOMES... 6 Methodology... 7 High School Impacts, 2012-13 Cohort... 8 Variation in High School Impacts by Duration of Participation, 2009 and 2010 Cohorts... 12 Middle School Analysis, 2012-13 Cohort... 15 L139 EDUCATION BUILDING 1000 BASCOM MALL MADISON, WI 53706-1326 SCHOOL OF EDUCATION UNIVERSITY OF WISCONSIN-MADISON 2012-2013 REPORT 1
SECTION 1. MADISON METROPOLITAN SCHOOL DISTRICT (MMSD) REPORT Program Implementation and Certification Status as of 2012-13 Advancement Via Individual Determination (AVID) is a national college readiness system consisting of two components: An elective course aimed at preparing a selected group of students for entrance and enrollment in postsecondary education. AVID aims to serve students in the academic middle (2.00 3.50 GPA) who may be first generation, low-income, historically underrepresented in postsecondary education and/or students with special circumstances. Best-practice instructional and student strategies that help all students access curriculum and become independent learners on a pathway to college, career, and community readiness. MMSD began AVID as a pilot program first implemented at East High School in 2007. In 2008, MMSD and the Boys and Girls Club (BGC) of Dane County created a partnership entitled AVID/Teens of Promise (TOPS). TOPS provides: Summer career opportunities for AVID students. Community mentors to work with AVID students during non-school hours. Financial, staff, and programmatic resources to support the implementation of college field trips, guest speakers, and tutorials within the AVID elective class. In 2012-13, MMSD expanded the implementation of both components of the AVID College Readiness System to 11 middle schools. Additionally, MMSD and BGC expanded their partnership to include AVID/ College Club at Cherokee and Wright Middle Schools. College Club, similar to TOPS, provides supplemental services for AVID students as well as financial and staffing supports to implement the AVID elective class. Currently, the AVID TABLE 1. AVID STUDENT DEMOGRAPHIC CHARACTERISTICS, GRADES 7-12, 2012-13* MMSD Data Categories Total Middle and High School Enrollment Percent of Middle and High School Enrollment Total Middle and High School AVID Enrollment Percent of Middle and High School AVID Enrollment Percent of Middle and High School Enrollment Served by AVID Total Enrollment 12706 938 7% Free or Reduced Price Lunch (F/R) 6055 48% 657 70% 11% English Language Learners (ELL) 1724 14% 188 20% 11% Special Education 2232 18% 56 6% 3% Black/African 2487 20% 263 28% 11% Hispanic/ 2043 16% 300 32% 15% Asian 1197 9% 74 8% 6% White 5974 47% 215 23% 4% Multi-racial 947 7% 84 9% 9% Native ** Less than 1% Less than 1% Female 6734 53% 525 56% 8% Male 5972 47% 413 44% 7% *Table 1 provided by MMSD. **Due to small sample size, numbers have been suppressed per district standards. 2 2012-2013 REPORT
elective course is offered at four high schools and 11 middle schools. In 2012-13, MMSD offered 43 sections of AVID in grades 7 through 12, serving 938 students. This information is disaggregated by student demographic characteristics (Table 1) and by grade level (Table 2). The implementation of AVID is monitored by AVID Center through an extensive process in which schools become AVID certified sites. To become a certified school the following criteria must be met: TABLE 2. NUMBER OF AVID SECTIONS AND TOTAL ENROLLMENT BY GRADE, 2012-13* Grade Level AVID Enrollment Sections per Grade 7 88 5 8 280 12 9 192 8 10 152 7 11 132 6 12 94 5 Total 938 43 *Table 2 provided by MMSD. Schools have implemented AVID for at least two years. Schools show evidence through data, practices, protocols, classroom observations, and student work of meeting or exceeding the national AVID implementation criteria, which are based on a series of rubrics that focus on student selection, use of AVID strategies, student access to rigorous course work, and effective site team practices. Schools show evidence of using AVID strategies beyond the AVID elective classroom. In spring 2013, La Follette, Memorial, and West High Schools were certified by AVID Center. Additionally, East High School was certified and designated as a National Demonstration School, becoming the first school in the state of Wisconsin to receive such distinction. MMSD middle schools will be certified in 2014-15, with preliminary certification occurring in 2013-14. Program Retention Rates MMSD and BGC partnership schools seek to retain all students who ever enter its program during middle and high school. However, some students choose to exit the AVID elective. Others are exited for academic or behavioral reasons by the AVID site team at each school. Common reasons for program exits include: Self-exits: For various reasons, students decide to drop the elective from their course schedules. TABLE 3. AVID ONE-YEAR RETENTION AND EXIT RATES FROM 2012-13 TO 2013-14, BY GRADE AND STUDENT DEMOGRAPHICS* MMSD Data Categories Percentage of Students Retained in AVID Percentage of Students Who Exited AVID Total Enrollment 88% 12% Grade 7 88% 12% Grade 8** 96% 4% Grade 9 87% 13% Grade 10 87% 13% Grade 11 94% 6% Grade 12 88% 12% F/R 89% 11% Non F/R 87% 13% ELL 84% 16% Special Education 84% 16% Black/African 89% 11% Hispanic/ 92% 8% Asian 94% 6% White 82% 18% Multi-racial 87% 13% Native *** Female 87% 13% Male 89% 11% *Table 3 provided by MMSD. **Does not account for the number of students who did not apply for 9th grade AVID. ***Due to small sample size, numbers have been suppressed per district standards. 2012-2013 REPORT 3
Exits due to low GPA: Students are exited from the AVID elective course if their cumulative GPAs fall below 2.00 for more than one semester. Exits due to behavioral referrals: Students are exited from the AVID elective for significant behavioral issues. Exits due to student mobility: Students move or transfer to another school. Trends in 2012-13 retention rates: Overall, 88% of students who participated in middle school AVID and/or / College Club during the 2012-13 school year re-enrolled in the program for the 2013-14 school year. The remaining 12% exited during the 2012-13 school year or summer and did not participate in the program the following year. Of the students enrolled in 8 th grade AVID, 65% applied for and were accepted into 9 th grade AVID. (This number is not included in retention or exit calculations.) Relative to the 2011-12 academic year, the 2012-13 program retention rates increased by 5.5%. Retention rates at high schools increased by 5%. Of all grade level advancements, the transition from 9 th grade to 10 th grade AVID has the lowest retention rate. Table 3 (on previous page) displays grade level to grade level retention rates. It does not reflect persistence within AVID across all four years of high school enrollment. SECTION 2. BOYS AND GIRLS CLUB (BGC) OF DANE COUNTY REPORT Teens of Promise (TOPS) Out-of-School Programming In 2012-13, the BGC of Dane County provided 117 of 365 Advancement Via Individual Determination (AVID)/TOPS eligible students with TOPS out-of-school programming. This is approximately 30% of eligible students based on grade level, academic standing, and completion of other TOPS program requirements. BGC is responsible for the recruitment and training of mentors and employers as well as the development of the Career Exploration curriculum. TABLE 4. TOPS STUDENT DEMOGRAPHIC CHARACTERISTICS, 2012-13* Mentoring Partnerships (After 9th Grade) Career Exploration (After 10th Grade) Summer Internships (After 11th or 12th Grade) Number of Eligible Students 365 139 226 Number of Participants 59 26 32 Participation Rate (%) 16% 19% 14% East High (#) 24 5 10 Memorial High 3 7 9 La Follette High 21 13 7 West High 11 1 6 Black/African (#) 16 6 7 Hispanic/ 21 6 9 Asian 3 2 2 White 10 7 9 Multi-racial 9 5 5 Male (#) 24 12 14 Female 35 14 18 *Table 4 provided by BGC of Dane County. 4 2012-2013 REPORT
Mentoring BGC operates the TOPS mentoring program open to students who have successfully completed freshman year. Matches are made twice a year until the first semester of junior year. Interested students must submit an application before being matched with college-educated adults who hope to positively influence college-bound students. Mentors and mentees spend time together outside of school, with the majority spending roughly two hours a month together. They are supported by TOPS student coordinators who work at the school of the student. Summer Internships and Career Exploration BGC runs TOPS summer programming for students. Career exploration is open to students who are in good standing and have successfully completed sophomore year. Summer internships are open to students who are in good standing and have successfully completed junior or senior years. Both programs run for six weeks. Students are exposed to a variety of careers that require college degrees. To be considered for placement, TOPS student interns must apply, interview, and complete job-readiness training. Interns are paid a minimum of $8.00/hour for 10-20 hours a week and gain direct experience working with a supervisor in a company. Students who participate in career exploration go on seven career-focused field trips. They are also required to research a variety of careers and make a presentation to a panel of invited guests and their peers. Successful completion of all program requirements also results in monetary compensation provided by the BGC. TOPS College Continuation The purpose of TOPS College Continuation services is to ensure that high school graduates from the College Readiness System obtain four-year college degrees. TOPS College Continuation staff use a combination of best-practices in academic advising, coaching, and mentoring to successfully guide students through the transition from high school graduation to college degree completion. College Continuation services began in 2011-12 and BGC undertakes the following: Disseminate information about college campuses, financial aid, and scholarships to assist high school seniors and their families to make more informed decisions about college selection. They also disburse and process TOPS scholarships. As of fall 2013, 72 alumni received TOPS scholarships totaling $61,775. Provide additional resources to recent high school graduates to ensure their readiness for college and that they matriculate in the fall semester immediately following graduation. TABLE 5. COLLEGE ENROLLMENT AND PARTICIPATION RATES AMONG HIGH SCHOOL GRADUATES, BY GRADUATION YEAR* Class Size Full-Time College Part-Time College For-Profit College Joined the Military Not Enrolled in College Percent Enrolled in College Class of 2010 10 4 1 2 0 3 70% Class of 2011 21 15 0 0 0 6 71% Class of 2012 18 12 3 0 1 2 83% Class of 2013 84 69 7 1 1 6 90% Total 133 100 11 3 2 17 86% *Table 5 provided by BGC of Dane County. Enrollment data verified with the National Student Clearinghouse and TOPS documents for scholarship processing. 2012-2013 REPORT 5
Track enrollment each semester, conduct campus visits with students, communicate monthly via various communication formats, and plan alumni gatherings. WISCONSIN CENTER FOR THE ADVANCEMENT OF POSTSECONDARY EDUCATION (WISCAPE) EVALUATION OF STUDENT OUTCOMES For the past four years, WISCAPE has partnered with the Madison Metropolitan School District (MMSD) and the Boys and Girls Club (BGC) of Dane County to conduct an annual assessment of the Advancement Via Individual Determination/Teens of Promise () program. is designed to increase college preparation, postsecondary education access and degree attainment, and academic achievement for under-represented students TABLE 6. COLLEGE DESTINATIONS OF GRADUATES (CLASS OF 2010, 2011, 2012, 2013) ATTENDING COLLEGE* 6 2012-2013 REPORT College Destination (defined as low-income students and students of color) in the academic middle (students with a GPA between 2.0 and 3.5). The 2012-13 WISCAPE report analyzes measurable outcomes for students that reflect the program s stated goals. The report evaluates impacts of the program for high school students enrolled in the program during 2012-13, considers the relationship between time spent in the program and student outcomes among the 2009 and 2010 freshman cohorts of participants. The guiding research questions regarding high school include: How do the academic, behavioral, and attendance outcomes of students who participate in during high school compare to those of their peers who do not participate in the program? Are program Number of AVID/ TOPS Alumni Percent at Campus Madison Area 75 66% Madison College 48 42% UW-Madison 15 13% Edgewood College 9 8% Madison Media Institute, ITT Technical Institute, Argosy 3 3% Wisconsin (Not in Madison) 32 28% UW-Milwaukee 10 9% UW-Platteville 5 4% UW-Whitewater 4 4% UW-Oshkosh 3 3% UW-Eau Claire & UW-LaCrosse 4 4% UW-Parkside & UW-Richland Center 2 2% Alverno College, Viterbo College, Marian University 3 3% Morraine Park Technical College 1 < 1% Out-of-State Colleges 7 6% Minnesota 2 2% Illinois, Michigan, Kentucky 3 3% New York, Texas 2 2% *Table 6 provided by BGC of Dane County. outcomes favorable for the program s focal groups: students from low-income families and students of color? How does the effect of high school AVID/ TOPS on academic, behavioral, and attendance outcomes change with duration of participation in the program? In particular, do students who spend more time in high school have better outcomes than students with less exposure? How does the program affect student outcomes during the critical senior year, especially the likelihood of high school graduation?
Methodology In this year s evaluation, we took additional steps to increase the rigor of the methods used to isolate program effects by carefully constructing a group of students who did not participate in but could be reasonably compared to those who did. Specifically, for each component of the evaluation we used an extensive MMSD administrative database to match students to students who did not participate in but had similar academic, attendance, and behavioral records prior to high school, attended the same high school, and shared the same demographic and family background characteristics (e.g. gender, race/ ethnicity, free-or-reduced price lunch receipt, English Language Learner status, Exceptional Education Needs status). As seen in the graph below, the demographic characteristics of the comparison group are nearly identical to that of the students, thereby allowing our analysis to better isolate program effects. After matching students to this comparison group, we then estimated impacts of the program on focal outcomes of interest. For the 2012-13 analysis, student outcomes of interest were: cumulative GPA, GPA in core courses, enrollment and GPA in Advanced Placement/Honors courses, attendance rates, and behavioral referrals. For the 2010 freshman cohort analysis, student outcomes of interest were: junior year cumulative GPA, enrollment in AP/Honors courses, and junior year attendance rate. For the 2009 freshman cohort analysis, student outcomes of interest were: high school graduation, senior year cumulative GPA, and enrollment in AP/Honors courses. Summary of High School Results, 2012-2013 The results suggest that the program positively boosts college readiness for high school students, particularly students from low-income families and students of color. For example, appears to boost the chances of high school graduation, improve cumulative GPAs, increase participation in AP/ Honors courses, improve attendance records, and result in fewer behavior referrals. Together, these improvements suggest that prospects for college participation and success may be enhanced as well. DEMOGRAPHIC CHARACTERISTICS OF STUDENTS SAMPLED, 2012-13 70% 60% 50% 40% 65% 63% 44% 44% 8th Grade GPA: 3.01 Number of Students Sampled: 533 COMPARISON GROUP 8th Grade GPA: 3.01 Number of Students Sampled: 533 30% 20% 10% 24% 24% 33% 33% 20% 20% 23% 23% 7% 6% 6% 6% 0% Low-Income Male African Other White EEN ELL 2012-2013 REPORT 7
DISTRICT FINDINGS 2012-2013 SCHOOL YEAR IMPACT 1 000 increases the cumulative and core GPAs of high school students, particularly among low-income students and students of color. Core GPA is the grade point average for core academic subjects: math, English, science, and social studies. CUMULATIVE GPA 3 2.5 2 2.74 2.61 2.66 2.42 2.53 2.36 2.76 2.48 2.67 2.42 2.63 2.43 1.5 COMPARISON GROUP 1 0.5 0 District Low-Income African Low-Income Male DIFFERENCE IS STATISTICALLY SIGNIFICANT AT THE 90% LEVEL CORE GPA 3 2.5 2 1.5 2.54 2.39 2.46 2.18 2.34 2.18 2.52 2.22 2.47 2.18 2.43 2.17 1 0.5 0 District Low-Income African Low-Income Male 8 2012-2013 REPORT
DISTRICT FINDINGS 2012-2013 SCHOOL YEAR IMPACT 2 000 increases rates of enrollment in Advanced Placement (AP) and Honors courses for some students, including low-income students. The program also increases the grades earned by some students in those courses. AP/HONORS ENROLLMENT 70% 60% 50% 40% 30% 58% 51% 54% 45% 52% 42% 59% 52% 55% 45% 54% 48% 20% 10% COMPARISON GROUP 0% District Low-Income African Low-Income Male AP/HONORS GPA 3 2.5 DIFFERENCE IS STATISTICALLY SIGNIFICANT AT THE 90% LEVEL 2 1.5 2.46 2.47 2.4 2.23 2.13 2.28 2.47 2.34 2.4 2.23 2.31 2.27 1 0.5 0 District Low-Income African Low-Income Male 2012-2013 REPORT 9
DISTRICT FINDINGS 2012-2013 SCHOOL YEAR IMPACT 3 000 The program improves students' attendance records, reducing the number of days they miss school. The effects are especially pronounced for low-income, African, and students. There are 180 days in the school year. MMSD s attendance goal is 94% or 169 days. ATTENDANCE RATE 100% 80% 95% 94% 95% 93% 95% 93% 95% 93% 95% 93% 96% 94% 60% COMPARISON GROUP 40% 20% DIFFERENCE IS STATISTICALLY SIGNIFICANT AT THE 90% LEVEL 0% District Low-Income African NUMBER OF DAYS ABSENT Low-Income Male 14 12 10 8 6 8.71 11.52 9.49 13.27 8.69 12.38 8.69 12.78 9.16 12.87 7.42 10.12 4 2 0 District Low-Income African Low-Income Male 10 2012-2013 REPORT
DISTRICT FINDINGS 2012-2013 SCHOOL YEAR IMPACT 4 000 The program reduces the number of behavioral referrals received, especially for low-income students and students of color. BEHAVIORIAL REFERRALS 3.5 3.0 2.5 2.0 1.5 2.01 2.96 1.99 3.31 2.14 3.13 2.26 1.91 3.35 1.88 3.31 1.0 0.5 0.0 District Low-Income African 1.34 Low-Income Male COMPARISON GROUP DIFFERENCE IS STATISTICALLY SIGNIFICANT AT THE 90% LEVEL 2012-2013 REPORT 11
DISTRICT FINDINGS IMPACT BY PROGRAM DURATION IMPACT 5 EFFECTS OF ON JUNIOR YEAR CUMULATIVE GPA ACCORDING TO LENGTH OF TIME SPENT IN PROGRAM 3.0 2.5 2.0 1.5 1.0 0.5 0.0 2.01 2.36 2.7 The positive impacts of on junior year cumulative GPA appear to accrue for students who spend a longer time in the program. The greatest benefits occur after a student spends at least two years in AVID/ TOPS. EFFECTS OF ON RATE OF AP/HONORS PARTICIPATION ACCORDING TO LENGTH OF TIME SPENT IN PROGRAM 2.26 1 Year 2 Years 3 Years CG Time Time Spent spent in in Among 2010 Freshman Cohort COMPARISON GROUP (JUNIORS MATCHED USING 8TH GRADE DATA, DID NOT EVER PARTICIPATE IN ) 75% 50% 25% 0% 37% 62% 45% 32% 1 Year 2 Years 3 Years CG Time Spent Time in spent in Among 2010 Freshman Cohort increases rates of enrollment in Advanced Placement (AP) and Honors courses. The positive impacts appear to accrue for students who spend a longer time in the program. DIFFERENCE IS STATISTICALLY SIGNIFICANT AT THE 90% LEVEL IMPACT 6 12 2012-2013 REPORT
DISTRICT FINDINGS IMPACT BY PROGRAM DURATION IMPACT 7 EFFECTS OF ON ATTENDANCE RATE AND NUMBER OF DAYS ABSENT DURING JUNIOR YEAR ACCORDING TO LENGTH OF TIME SPENT IN PROGRAM JUNIOR YEAR ATTENDANCE RATE COMPARISON GROUP (JUNIORS MATCHED USING 8TH GRADE DATA, DID NOT EVER PARTICIPATE IN ) 100% 80% 60% 40% 20% 0% 91% 96% 93% 93% 1 Year 2 Years 3 Years CG Time Spent Time spent in in Among 2010 Freshman Cohort DAYS ABSENT DURING JUNIOR YEAR The program improves junior year attendance records, reducing the number of days they miss school. The positive impacts appear to accrue for students who spend a longer time in the program. There are 180 days in the school year. MMSD s attendance goal is 94% or 169 days. DIFFERENCE IS STATISTICALLY SIGNIFICANT AT THE 90% LEVEL 18 15 12 9 12.51 13.43 17.06 6 7.81 3 0 1 Year 2 Years 3 Years CG Time Spent in Time spent in Among 2010 Freshman Cohort 2012-2013 REPORT 13
DISTRICT FINDINGS IMPACT BY PROGRAM DURATION* IMPACT 8 EFFECTS OF ON SENIOR YEAR CUMULATIVE GPA ACCORDING TO LENGTH OF TIME SPENT IN PROGRAM COMPARISON GROUP (SENIORS MATCHED USING 8TH GRADE DATA, DID NOT EVER PARTICIPATE IN ) The positive impacts of on cumulative GPA appear to accrue for students who spend a longer time in the program. The greatest benefits occur after a student spends at least three years in. 3.0 2.5 2.0 1.5 1.0 0.5 0.0 EFFECTS OF ON HIGH SCHOOL GRADUATION RATES ACCORDING TO LENGTH OF TIME SPENT IN PROGRAM 2.15 2.49 2.68 2.71 2.54 1 Year 2 Years 3 Years 4 Years CG Time Time Spent spent in in Among 2009 Freshman Cohort DIFFERENCE IS STATISTICALLY SIGNIFICANT AT THE 90% LEVEL IMPACT 9 The program increases the chances that a student will graduate from high school, with the likelihood of graduation particularly strong for students who remain in the program for at least three years. 100% 80% 60% 40% 20% *This is the first of the annual AVID/ TOPS reports to include senior year outcomes. 0% 75% 96% 91% 71% 57% 1 Year 2 Years 3 Years 4 Years CG Time Time Spent spent in in Among 2009 Freshman Cohort 14 2012-2013 REPORT
WISCAPE Middle School Analysis, 2012-13 In addition to conducting the high school analysis, WISCAPE also reviewed baseline findings from the first year of AVID middle school implementation at MMSD. Using a similar methodology as the high school evaluation, the middle school analysis matched 8th grade students who participated in AVID during the 2012-13 school year with those who did not participate but had similar characteristics. Doing so helps distinguish AVID impacts from selection into the program. Specifically, we used an extensive MMSD administrative database to match students with similar academic, attendance, and behavioral records prior to 8th grade, who attended the same middle school, and who shared the same demographic and family background characteristics (e.g. gender, race/ ethnicity, free or reduced price lunch receipt, English Language Learner status, Exceptional Education Needs status). The guiding research question was the following: As the AVID middle school program begins implementation, how do academic outcomes for students in AVID compare to those who do not participate in the program? So far, the results show few significant impacts from the first year of AVID implementation at MMSD middle schools. On average, 8th grade AVID students and comparison group students have a similar 8th grade cumulative GPA of 2.90 and a 96% attendance rate. However, AVID students do receive fewer behavioral marks (1.46) than their counterparts (2.14). This finding is significant at the district-wide level but not significant at every middle school. As the AVID program continues to develop at the middle school level, we can better evaluate its impact. WISCAPE aims to inform and improve postsecondary education policy, research, and practice through the creation and exchange of knowledge. The Wisconsin Center for the Advancement of Postsecondary Education (WISCAPE) was established in 2001 on the University of Wisconsin Madison campus to study the challenges confronting postsecondary education while bringing together key stakeholders to discuss issues and solutions. The center is housed in the School of Education. MISSION WISCAPE recognizes that colleges and universities make significant contributions to society. To help these institutions better serve the public good, the center aims to inform and improve postsecondary education policy, research, and practice through the creation and exchange of knowledge. WISCAPE S WORK In pursuit of this goal, WISCAPE conducts and supports research projects; sponsors public programs; produces and distributes publications; and convenes and fosters communication among key stakeholder groups. For more information on this report, contact Eleni Schirmer at eschirmer@wisc.edu. L139 EDUCATION BUILDING 1000 BASCOM MALL MADISON, WI 53706-1326 SCHOOL OF EDUCATION UNIVERSITY OF WISCONSIN-MADISON www.wiscape.wisc.edu 2012-2013 REPORT 15