LONGITUDINAL DATA SYSTEMS

A DISTRICT S S APPROACH TO LONGITUDINAL DATA SYSTEMS July 30, 2009 NCES Summer Data Conference

Objectives Characteristics of the District s Longitudinal Data System (LDS) Effective Use of LDS Issues Discovered While Using LDS Recommendations for the Future ** (**A Eff ti LDS b t b d l d d d t (**An Effective LDS can best be developed and managed at the school/site/district level rather than at the state level)

District Characteristics District Demographics Architecture of Our LDS Business Rules

Western Heights: Intent on 10% Western Heights Public Schools is a diverse, multicultural, high-challenge school district consisting of six school sites located in the southwest quadrant of Oklahoma City, serving 3,400 students. African American 23.5% Low Socio-Economic i 79.3% Asian/Pacific 4.0% Eng. Lang. Learners 13.2% Hispanic 19.5% Special Services 14.8% Caucasian 43.9% Male 51.7% Native American 9.0% Female 48.3% Other.1% Mobile (4 year rate) 70.0%+

Western Heights SIF Infrastructure Zone Integration Server (ZIS) SIF Agents Applications SIF Data Objects Network Account (EduStructures) Transportation System (Route Point) State Student ID System (Mizuni) Student Information System (Pearson SMS) Library Automation (Follett Destiny) Food Service (Data Futures) Data Warehouse (Mizuni) Coming Soon: Human Resources & Finance Data Analysis & Reporting (Mizuni) Instructional Services (Microsoft Class Server & Renaissance Place) Instructional Management (Campusware Grade book, EZ IEP, & EZ Planner)

Business Processes that Drive the LDS Single Source of Data Entry Schools Interoperability Framework (SIF) Compliant SIF Agents Present to Support Zone Integration of Disparate Applications (i.e., Allows Deployment of Best-of-Breed Solutions) Historical Capability Within the Student Information System (SIS) & Other Systems (Avoid Annual Dumping of Data) Family Information Management Capability in Place (Especially in the SIS) Capability to Support Flexible Insertion of New Data Packages as Federal & State Reporting Requirements Change (e.g., English Language Learner Parameters)

Effective Use of LDS How Does LDS Impact Parents and Students? t How Does LDS Impact Teachers? How Does LDS Impact Administrators?

Parent Version of Dashboard 8

Student Report Card

Student Standards-Based Report

Individual Student Results: Norm-Referenced Test Data

Individual Student Performance on Each CRT Standard, Compared to Site/District Results

Teacher Version of the Dashboard

Teacher Reporting: Class Summary, CRT Data by State Instructional Standards

Teacher Reporting: Class Summary, CRT Data by State Instructional Standards

Teacher Reporting: Assessments with Grading Alignment

Teacher Reporting: Class Summary, Student Performance by State Standards

Example of Teacher Trend Data and Current Problems with LDS Data Management

Administrator s Version of the Dashboard

Site Level Reporting: Trend Data

Current Administrative Reporting: The Analytical Scope/Trend-Based

Site Level Reporting: NCLB Validation of Testing Requirements

District Level Reporting: NCLB Validation of Testing Requirements

District Level Reporting: Samples of Trend Data

District Level Reporting: Mobility Trend Data

In the beginning, what were we seeking? Data Warehousing Vertical Report Architecture/Student Progress Data Over Time Expanding Zone Integration Between Disparate Applications Re-Engineered School Business Operations and Processes Professional Development to Support the New Paradigm at Every Level of School Operations

Entry/Gain/Loss (Enrollment) Codes: District vs. State 27 Pupil Accounting Codes: Reasons Students Enter and Exit Schools State has 10 codes; WH District has 56 codes Detailed District Coding: Creates a Culture of Caring by Improving Staff Awareness of Students Current Status Promotes Trend Reporting Processes Streamlines Federal and State Reporting Enhances Data Management Capabilities

What Did We Discover? Trend Data All Schools Fifth Grade Math 2004 Compared to 2006 45 40 35 30 25 20 15 10 5 0 44.4 41.3 32.3 27.3 14.5 14.1 17.4 8.9 Below Base Basic Proficient Advanced 2004 2006

What Did We Discover? 45 40 35 30 25 20 Trend Data All Schools Fifth Grade Reading 2004 Compared to 2006 40.4 32.5 38.5 25.8 32.9 2004 15 12.8 9.2 8.3 2006 10 5 0 Below Basic Basic Proficient Advanced

Additional Considerations for an LDS Approach Bring Cohort-Based Reporting into the Analytics Move to a Longitudinal Reporting Architecture While Maintaining Existing Trend-Data ed aa Reporting

Student Cohort Assignment

District Level Longitudinal Reporting USDOE s Uniform Graduation Rate Reporting Tool 32

District Level Longitudinal Reporting

Mobility Comparisons between 10 th -12 th Grade and 12 th Grade Only 10 th 12 th Grade 12 th Grade Only Total Students 994 339 Original Students 630 206 Transfers In 364 133 Transfers Out W/Request for Record Transfers In/Transfers Out 357 146 225 83 *Mobility Rate 49.9% 9% 57.8% Dropouts 130 (13.1%) 59 (17.4%) *Mobility Rate = (Transfers In + Students who Transfer Out Transfer In/Transfers Out)/ Total Students

Academic Success/Failure: 10 th -12 th Grade versus 12 th Grade Only 10 th -12 th Grade Success Rate 12 th Grade Only Success Rate Total Students 576/994= 58% 208/339 = 61.4% Original Students 430/630 = 68% 144/206 = 69.9% Transfers In 167/364 = 46% 64/133 = 48.1% Transfers Out W/Request 195/357 = 54.6% 86/146 = 58.9% for Record Transfers In/Transfers Out 126/225 = 56% 45/83 = 54.2% *Mobility 236/496 = 47.7% 105/196 = 53.6%

SES Comparison between 10 th -12 th Grade and 12 th Grade Only 10 th 12 th 10 th -12 th 12 th Only 12 th Only Normal SES Low SES Normal SES Low SES Total Students 553 451 222 117 Original Students 276 354 116 90 Transfers In 267 97 106 27 Transfers Out W/Request for Record Transfers In/Transfers Out 306 51 137 9 207 18 82 1 *Mobility 66.2% 28.8% 8% 57.8% 29.9% 9% Dropouts 72 (13%) 58 (12.9%) 57 (25.7%) 22 (18.8%) *Mobility Rate = (Transfers In + Students who Transfer Out Transfer In/Transfers Out)/ Total Students

Academic Success Comparison by Ethnicity and Gender: 10 th 12 th Grade African-American Caucasian Hispanic Female Male Female Male Female Male Total Students 74/138 = 49/115 155/208= 138/217 52/90 = 52/96 = 53.6% = 42.6% 74.5% = 63.6% 6% 57.8% 54.2% Original Students 49/78 = 62.8% Transfers In 25/60 = 41.7% Transfers Out W/Request for Record Transfers In/Transfers Out 33/59 = 83.6% 16/56 = 28.6% 113/139 = 81.3% 42/69 = 60.9% 105/153 = 68.6% 33/64 = 51.6% 33/55 = 60% 19/35 = 54.3% 41/62 = 66.1% 11/34 = 32.4% 36/63 = 21/54 = 42/65 = 45/77 = 15/24 = 12/37 = 57.1% 38.9% 64.6% 6% 58.4% 62.5% 32.4% 21/39 = 53.8% 12/37 = 32.4% 29/43 = 67.4% 30/47 = 63.8% 10/15 = 66.7% 9/19 = 47.4% *Mobility Rate 62.3% 67.8% 43.8% 29.5% 48.9% 52%

Academic Success Comparison by Ethnicity and Gender: 12 th Grade Only Total Students 16/39 = 41% Original Students African-American Caucasian Hispanic Female Male Female Male Female Male 16/22 = 72.7% 15/42 = 35.7% 10/21 = 47.6% 59/73 = 80.8% 43/47 = 91.5% 40/64 = 62.5% 29/39 = 71.8% 21/32 = 65.6% 11/18 = 61.1% 19/38= 50% 15/25 = 60% Transfers In 10/17 = 5/21 = 16/21 = 12/25 = 10/14 = 4/13= 58.8% 23.8% 76.2% 48% 71.4% 30.8% Transfers Out W/Request for Record Transfers In/Transfers Out 16/22 = 7/20 = 19/25 = 20/32 = 8/14= 7/16 = 72.7% 7% 35% 76% 62.5% 57.1% 43.8% 8/12= 66.7% 2/11 = 18.2% 10/16 = 62.5% 10/19 = 52.6% 5/5 = 100% 4/8 = 50% *Mobility Rate 69.2% 71.4% 47.9% 59.4% 62.5% 55.3%

Early Elementary Math: Overall Mean Performance Score (State Test) for 2007-2008 3rd Graders Students who begin their education early (Pre-K) and stay in the same school tend to do well academically.

Descriptive Statistics: Early Elementary Math 2007-2008 2008 Mean Performance Scores (State Test) Continuous Enrollment Mean Std. Deviation N PreK-3rd 729.8095 67.53132 63 K-3rd 712.8810 63.76527 42 1st-3rd 719.8077 62.98636 26 2nd-3rd 710.3784 72.98301 37 3rd Only 695.4490 83.40715 49 Students who begin their education early (Pre-K) and stay in the same school tend to do well academically.

Early Elementary Reading: Overall Mean Performance Score (State Test) for 2007-2008 3rd Graders Students who begin their education early (Pre-K) and stay in the same school tend to do well academically.

Descriptive Statistics: Early Elementary Reading 2007-2008 Mean Performance Scores (State Test) Continuous Enrollment Mean Std. Deviation N PreK-3rd 759.2857 72.03682 63 K-3rd 751.8571 54.33257 42 1st-3rd 734.34623462 51.30843 26 2nd-3rd 733.4054 59.20936 37 3rd Only 727.9388 65.37310 49 Students who begin their education early (Pre-K) and stay in the same school tend to do well academically.

8 th Grade Math: All Students (State Test) 2007-2008 Mean Performance Score

8th Grade Reading: All Students (State Test) 2007-2008 2008 Mean Performance Score

8th Grade Math: All Students (State Test) 2007-2008 2008 Mean Performance Score by Mobility

8th Grade Reading: All Students (State Test) 2007-2008 2008 Mean Performance Score by Mobility

Academic Performance of Durational Cohorts at the Middle School Level (State Test) Students who begin their education early and stay in the same school tend to do better academically.

What does the Longitudinal Data Tell Us? Mobility has a Huge Impact on Academic Achievement and Student t Success The Negative Impact of Mobility is Evident at all Levels of Education (Elementary, e Middle, High School)

Is Mobility a Problem Specific to Western Heights Alone? 10 th 12 th Grade 12 th Grade Only Total Students 994 339 *Mobility Rate 49.9% 57.8% Dropouts 130 (13.1%) 59 (17.4%) Do you know your mobility rate and its relationship to academic success at all school levels?

How Do We Address the Problem of Mobility in Our Schools? Establish Intervention Strategies at Site Levels that Effectively Identify and Serve Mobile Students Set Standards for the Articulation of Real-Time Data Between Districts and States: Student Performance Data Teaching and Learning Resources Data Demographic Data

Survey Tool for Early Identification of Mobile Students

A Common Language is Necessary to Support Instruction and Demonstrate Accountability in Understandable Terms Scheduling Lesson Planning Course (Reading) 25007.1 Course (Reading) 25007.1 Term 00 Quarter 0 Period 00 Learning Styles 000 Section 000 Critical Thinking Skills 000 Unit 0 Grade Level 00 Site 0000 Student Learner Characteristics Teacher 000 Student 00000000 Instructional Code (Reading Comprehension / Sequencing) 25007.1.5.2 Specific Student Learning Styles 00000

Why Do We Need a Common Language Initiative (CLI)? Current nomenclature for state standards of instruction are not synchronized over time to support cohort analysis within a LDS. This condition is NOT an Oklahoma-specific ao aspec c issue, but is prevalent in most, if not all, state standard descriptors.

Elements of a Common Language Initiative (CLI) Samples of Families of Knowledge

CLI Development Processes

Instructional Report Process Mapping: Course An Example Using CLI Creation of Modified Lesson Plan(s) 25007.1 Reading Instructional Goal(s)/District and State Standard(s) 25007.1.5 Comprehension 5 Comprehension Formative Assessment(s) 25007.1.5.2 Sequencing Creation of Lesson Plan(s) 25007.1.5.2.2 Analyze and explain the causes, motivations, sequences, and results Identification of Appropriate Instructional Content of events from a text. Bloom s, Gardner s, IEP, ELL, 504, Gifted, etc. Delivery of Lesson Plan(s) and Instructional Content Student/Teacher Schedules and Student/Parent Info Summative Assessment(s) Any/All of the Above Remediation Acceleration Start Process Again

The Future of Reporting: Establishing a Common Language Iowa Test of Basic Skills Course Reading Comprehension 25007.1.5 Comprehension Oklahoma Core Curriculum Tests Reading Comprehension/Critical Literacy 25007.1.5 Comprehension Stanford 9 Reading Reading Comprehension Action/Reason/Sequence 25007.1.5.2 Sequencing State Learning Standard d Language Arts Comprehension/Critical Literacy A.3.4.d. Analysis and Evaluation: Analyze and explain the causes, motivations, sequences, and results of events from a text. 25007.1.5.2.2 Sequencing Language Arts 25007 Reading 25007.1

Teacher s Response to Instructional Needs Curriculum Planning by Class, Group, or Individual Student Teachers have access to the Curriculum Resource Library. They can search for, share, adapt and store lesson plans and content from multiple sources. (This open-source system was created by an inhouse developer.) Easy Planner, Amy s Lesson Plans Search by Topic Search by Grade Level Search by Research-Based Instructional Strategies Search by State Standards, etc. 5858

Site Level Response to School Improvement: Professional Learning Communities (PLCs) Professional Learning Communities (PLCs for Each School) A Place For Educators to Collaborate and Communicate Design, Acquire, and Disseminate Instructional Support Activities

Current Administrative Reporting: Analytical Scope/Trend-Based

Recommendations: National Level Raise awareness of the problem of mobility through public relations campaigns to all stakeholders Create effective parameters for developing state & national standards of instruction. In other words, set common industry standards (e.g., ASTM) for developing state & national instructional standards Develop more effective guidance regarding the identification of mobile students in districts Bring in other social service providers to discuss ways of addressing mobility Create a Common Language Initiative to provide uniform reporting of student learning needs Develop well defined, universal enrollment status codes for all schools Incorporate SIF-based reporting

Recommendations: State Level Acknowledge the problem of mobility and raise awareness of all stakeholders Develop/Adopt metrics to recognize districts/schools/teachers who improve learning achievement for mobile students Join the Common Language Initiative to improve uniform reporting across schools/districts/states Use SIF technology for all educational software to support exchange of educational information in near real time Adopt p a well-defined, adequately-detailed enrollment coding system that is uniform across schools within the state and across other states States should use LDS aggregates of student performance data rather than managing individual academic performance data at the state level.

Recommendations: School District and Site Levels Know site mobility rates and inform all stakeholders of this issue and its impact on learning Develop survey intakes to identify mobile families/students in order to provide targeted intervention/assistance Adopt effective longitudinal metrics that acknowledge teachers who improve students academic performance Use common language(s) to enhance longitudinal reporting Use SIF-based technology on all educational software needs for ease of data exchange between disparate systems Involve the community (business and other social service providers) to assist y( p ) in maximizing the impact of educational efforts

Contact Information Joe Kitchens Western Heights Superintendent joe.kitchens@westernheights.k12.ok.us Dr. Mwarumba Mwavita Western Heights Director of School Improvement mwarumba.mwavita@westernheights.k12.ok.usmwavita@westernheights us Dr. Lisa McLaughlin Western Heights Assistant Superintendent lisa.mclaughlin@westernheights.k12.ok.us Dr. John Steffens Assistant Vice-Provost Emeritus Executive Director, Public Service Institute University of Oklahoma steffens@ou.edu