Massachusetts Institute for College & Career Readiness KNOW THE SIGNS: USING DATA TO IDENTIFY AND SUPPORT AT -RISK STUDENTS
Who We Are The Massachusetts Institute for College and Career Readiness builds capacity for districts and communities to engage in evidence based strategies to improve college and career readiness outcomes. This is achieved by providing districts with a national senior researcher who is interested in learning how to make their research relevant to educators and policy-makers. Currently, MICCR is supporting 15 school districts to develop and implement a variety of college and career readiness activities - http://sites.bu.edu/miccr/.
Who Are We (cont.) MICCR was established using funding from the U.S. Department of Education, Institute for Education Sciences. Principal Investigators and Directors include: V. Scott Solberg, Boston University Chad d Entremont, Rennie Center Ben Forman, MassINC Carrie Conaway, Massachusetts Department of Elementary and Secondary Education
Understanding Early Warning Systems Susan Therriault Director & Principal Researcher College and Career Readiness and Success Center, American Institutes of Research http://www.ccrscenter.org/ Image Source: http://www.air.org/person/susan-bowles-therriault
What is an Early Warning System? An early warning system (EWS) relies on readily available data housed at the school to accomplish the following: Rely on early warning indicators Predict which students are at risk of missing key education milestones Target resources to support off-track students early Implement systemic and routine strategy for identifying, diagnosing, supporting, and monitoring at-risk students Examine patterns and identify specific school and districts issues and needs
EWS Theory of Action EWS is grounded in research because indicators and thresholds are validated. EWS is not another label or accountability measure. An outcome measure must be meaningful and actionable by adults in the system.
Dropout Prevention Research: Institute of Education Sciences Practice Guide Recommendations Source: Dynarski et al., 2008
EWS Implementation Pathway Synthesize Research Validate Indicators Customize and Develop Tools and Supports Launch and Implement EWS Assess and Improve Processes
Synthesize Research/Validate Indicators: Early Warning Indicators for High School Graduation Outcome Sources: Allensworth & Easton, 2005; 2012; Balfanz, Herzong, MacIver, 2007; Balfanz, et.al., 2011; Jerald, 2006; Heppen & Therriault; 2008
Launch and Implement: 7-Step Early Warning Intervention Monitoring System (EWIMS) Step 7: Evaluate and Refine the EWIMS Process Step 5: Assign and Provide Interventio ns Step 4: Interpret the EWS Data Step 1: Establish Roles and Responsibil ities Step 3: Review the EWS Data Step 2: Use the EWS Tool Sources: Therriault, et.al., 2010; Therriault et.al., 2013
Resources College & Career Readiness & Success Center: www.ccrscenter.org Early warning indicators for postsecondary readiness Early warning system resources, research and guidance: www.earlywarningsystems.org U.S. Department of Education Institute for Education Research Learning Series http://ies.ed.gov/ncee/edlabs/projects/ews.asp
References Allensworth, E. M., & Easton, J. Q. (2005). The on-track indicator as a predictor of high school graduation. Chicago, IL: Consortium on Chicago School Research at the University of Chicago. Allensworth, E. M., & Easton, J. Q. (2007). What matters for staying on-track and graduating in Chicago Public High Schools: A close look at course grades, failures, and attendance in the freshman year. Chicago, IL: Consortium on Chicago School Research at the University of Chicago. Balfanz, R. (2009). Putting middle grades students on the graduation path (Policy and Practice Brief). Baltimore, MD: Johns Hopkins University, Everyone Graduates Center. Retrieved from http://www.amle.org/portals/0/pdf/research/research_from_the_field/policy_brief_balfanz.pdf Balfanz, R., & Herzog, L. (2005). Keeping middle grades students on-track to graduation: Initial analysis and implications. Presentation at the second Regional Middle Grades Symposium, Philadelphia, PA. Dynarski, M., Clarke, L., Cobb, B., Finn, J., Rumberger, R., & Smink, J. (2008). Dropout prevention: A practice guide (NCEE 2008 4025). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Heppen, J., & Therriault, S. B. (2008). Developing early warning systems to identify potential high school dropouts. Washington, DC: National High School Center. Retrieved from http://www.betterhighschools.org/pubs/documents/issuebrief_earlywarningsystemsguide.pdf Therriault, S., Heppen, J., O Cummings, M., Fryer, A., & Johnson, A. (2010). Early warning system implementation guide. Washington, DC: National High School Center, U.S Department of Education. Available at www.earlywarningsystems.org. Therriault, S., O Cummings, M., Yerhot, L., & Hauenstein, M. (2013). Implementing an early warning system in the middle grades. Washington, DC: National High School Center, U.S. Department of Education. Available at www.earlywarningsystems.org.
A State Example: Massachusetts Kate Sandel Senior Policy Analyst - Office of Planning and Research Massachusetts Department of Elementary and Secondary Education http://www.doe.mass.edu/ Image Source: https://www.linkedin.com/in/kate-sandel-5090305
What is the EWIS? Massachusetts Early Warning Indicator System (EWIS) Precursor: Indicator for rising 9 th graders Support from State Longitudinal Data System grants Identifies students in grades 1 12 as high, moderate and low risk of risk of missing key academic milestones (without intervention) Transitional Indicator: Provides student risk levels at the beginning of each school year Reports are in Edwin Analytics secure portal for school/district personnel. EWIS implementation is method of using the EWIS information with other resources to study students risk, explore underlying causes, deploy programming and monitor results of interventions
EWIS Key Concepts Provides risk level information for all students based on prior year data Only includes data from existing statewide collections The risk levels are not based on a student s relative likelihood for reaching an academic goal when compared with other students Individual indicators in the risk model are not reflective of the level of risk - taken together, the indicators (data points) collectively form a risk level A student s Risk Level indicates the likelihood of missing the academic milestone without interventions High Risk Moderate Risk Low Risk At Risk for not reaching the upcoming academic milestone ~25% of students at high risk meet the college success milestone Moderately at risk for not reaching the upcoming academic milestone ~60% of students at moderate risk meet the college success outcome Likely to reach the upcoming academic milestone ~90% of students at low risk meet the college success outcome
Postsecondary Milestones K-12 Milestones Model EWIS Seven Academic Milestones Grade Level Groupings Grades 1-3 Academic Milestones Reading by the end of third grade Reaching proficiency or higher on 3 rd grade ELA State Assessment State Results 60% Grades 4-6 Middle school ready Reaching proficiency or higher on 6 th grade ELA and math State Assessments 55% Grades 7-9 Grades 10-12 High school ready Passing grades on all 9 th grade courses High school graduation Meeting all local and state graduation requirements College Enrollment Enrolling in postsecondary education Academic Readiness Enrolling in credit-bearing courses without developmental education College Persistence Persisting in second year of postsecondary education 79% 87% 76% 65% 86%
The Massachusetts EWIS is = a method to better target interventions and supports at the individual, small group, and whole school levels The Massachusetts EWIS is not a label for students ( not college-track ) a systematic way to identify students for further review to determine if additional supports are necessary related to measures that are included in the state s accountability system (PPI) a system that ESE reexamines each year to continually improve helpful to use with other data, especially throughout the year a diagnostic tool with reasons why a student is at risk an accountability or college admissions measure a stagnant system that will remain the same in future years a real time indicator of risk
Early Warning Implementation Cycle
Image Source: http://www.huffingtonpost.com/stacey-alcorn/no-walls-no-limits---an-i_b_8826086.html Early Warning Systems in Practice: Worcester Public Schools Maureen Binienda Superintendent Worcester Public Schools
How are EWIS data are used to reduce dropout? The WPS uses EWIS data, along with other district diagnostic data and indicators, in order to target and support students. EWIS reports available through Edwin allow school administrators and instructors to easily identify students at risk. By triangulating this data with formative assessment results, classroom observations, and other district indicators to explore underlying causes, educators are able to develop and implement appropriate intervention strategies for high risk students.
Six step process 1. Review EWIS data (summer/beginning of school year) 2. Incorporate local data 3. Explore relationships between the outcome of interest and other data such (i.e. attendance, discipline, test scores, program participation) 4. Take action 5. Monitor students and interventions 6. Repeat
Connecting the data to the needs of students EWIS provides a solid starting point for their analyses. School staff are able to determine which students need focused attention in certain areas and further their investigation using data they have access to within their schools. EWIS data do not tell schools what interventions are needed. School staff have a better of understanding of the student s individual needs and develop interventions and support strategies for groups of students as well as individuals. Graduation Improvement Teams have been established in all high schools and there has been a re-alignment of alternative programs to enhance the district s capacity to support those at risk of dropping as well as returning students.
Getting help from community partners Worcester Public Schools College & Career Readiness Network 22 Community-Based Organizations (CBOs) in Worcester some with a primary mission of supporting at-risk student populations Regular meetings with district and school-based staff; Data sharing made possible by obtaining parent permissions and agreements between CBO s the WPS to ensure FERPA compliance External programs are often able to spend more one-on-one time with their students in comparison to guidance counselors and educators given the high caseloads and large class sizes. Community-based partners work collaboratively with the district so students are working toward the same goals in and outside of school. Communication between the internal staff and external partners is essential to avoid duplication of efforts and to ensure services are complementary
MICCR is a three-year project co-led by Boston University s School of Education, MassINC s Gateway Cities Innovation Institute (GCII), and the Rennie Center for Education Research & Policy that was developed in close partnership with the Massachusetts Department of Elementary and Secondary Education (MA ESE) For more information on MICCR, please contact LaVonia Montouté, MICCR Program Director, at lmontoute@renniecenter.org. Made possible by a grant from: 24