LONGITUDINAL DATA SYSTEMS

Transcription

1 A DISTRICT S S APPROACH TO LONGITUDINAL DATA SYSTEMS July 30, 2009 NCES Summer Data Conference

2 Objectives Characteristics of the District s Longitudinal Data System (LDS) Effective Use of LDS Issues Discovered While Using LDS Recommendations for the Future ** (**A Eff ti LDS b t b d l d d d t (**An Effective LDS can best be developed and managed at the school/site/district level rather than at the state level)

3 District Characteristics District Demographics Architecture of Our LDS Business Rules

4 Western Heights: Intent on 10% Western Heights Public Schools is a diverse, multicultural, high-challenge school district consisting of six school sites located in the southwest quadrant of Oklahoma City, serving 3,400 students. African American 23.5% Low Socio-Economic i 79.3% Asian/Pacific 4.0% Eng. Lang. Learners 13.2% Hispanic 19.5% Special Services 14.8% Caucasian 43.9% Male 51.7% Native American 9.0% Female 48.3% Other.1% Mobile (4 year rate) 70.0%+

5 Western Heights SIF Infrastructure Zone Integration Server (ZIS) SIF Agents Applications SIF Data Objects Network Account (EduStructures) Transportation System (Route Point) State Student ID System (Mizuni) Student Information System (Pearson SMS) Library Automation (Follett Destiny) Food Service (Data Futures) Data Warehouse (Mizuni) Coming Soon: Human Resources & Finance Data Analysis & Reporting (Mizuni) Instructional Services (Microsoft Class Server & Renaissance Place) Instructional Management (Campusware Grade book, EZ IEP, & EZ Planner)

6 Business Processes that Drive the LDS Single Source of Data Entry Schools Interoperability Framework (SIF) Compliant SIF Agents Present to Support Zone Integration of Disparate Applications (i.e., Allows Deployment of Best-of-Breed Solutions) Historical Capability Within the Student Information System (SIS) & Other Systems (Avoid Annual Dumping of Data) Family Information Management Capability in Place (Especially in the SIS) Capability to Support Flexible Insertion of New Data Packages as Federal & State Reporting Requirements Change (e.g., English Language Learner Parameters)

7 Effective Use of LDS How Does LDS Impact Parents and Students? t How Does LDS Impact Teachers? How Does LDS Impact Administrators?

8 Parent Version of Dashboard 8

9 Student Report Card

10 Student Standards-Based Report

11 Individual Student Results: Norm-Referenced Test Data

12 Individual Student Performance on Each CRT Standard, Compared to Site/District Results

13 Teacher Version of the Dashboard

14 Teacher Reporting: Class Summary, CRT Data by State Instructional Standards

15 Teacher Reporting: Class Summary, CRT Data by State Instructional Standards

16 Teacher Reporting: Assessments with Grading Alignment

17 Teacher Reporting: Class Summary, Student Performance by State Standards

18 Example of Teacher Trend Data and Current Problems with LDS Data Management

19 Administrator s Version of the Dashboard

20 Site Level Reporting: Trend Data

21 Current Administrative Reporting: The Analytical Scope/Trend-Based

22 Site Level Reporting: NCLB Validation of Testing Requirements

23 District Level Reporting: NCLB Validation of Testing Requirements

24 District Level Reporting: Samples of Trend Data

25 District Level Reporting: Mobility Trend Data

26 In the beginning, what were we seeking? Data Warehousing Vertical Report Architecture/Student Progress Data Over Time Expanding Zone Integration Between Disparate Applications Re-Engineered School Business Operations and Processes Professional Development to Support the New Paradigm at Every Level of School Operations

27 Entry/Gain/Loss (Enrollment) Codes: District vs. State 27 Pupil Accounting Codes: Reasons Students Enter and Exit Schools State has 10 codes; WH District has 56 codes Detailed District Coding: Creates a Culture of Caring by Improving Staff Awareness of Students Current Status Promotes Trend Reporting Processes Streamlines Federal and State Reporting Enhances Data Management Capabilities

28 What Did We Discover? Trend Data All Schools Fifth Grade Math 2004 Compared to Below Base Basic Proficient Advanced

29 What Did We Discover? Trend Data All Schools Fifth Grade Reading 2004 Compared to Below Basic Basic Proficient Advanced

30 Additional Considerations for an LDS Approach Bring Cohort-Based Reporting into the Analytics Move to a Longitudinal Reporting Architecture While Maintaining Existing Trend-Data ed aa Reporting

31 Student Cohort Assignment

32 District Level Longitudinal Reporting USDOE s Uniform Graduation Rate Reporting Tool 32

33 District Level Longitudinal Reporting

34 Mobility Comparisons between 10 th -12 th Grade and 12 th Grade Only 10 th 12 th Grade 12 th Grade Only Total Students Original Students Transfers In Transfers Out W/Request for Record Transfers In/Transfers Out *Mobility Rate 49.9% 9% 57.8% Dropouts 130 (13.1%) 59 (17.4%) *Mobility Rate = (Transfers In + Students who Transfer Out Transfer In/Transfers Out)/ Total Students

35 Academic Success/Failure: 10 th -12 th Grade versus 12 th Grade Only 10 th -12 th Grade Success Rate 12 th Grade Only Success Rate Total Students 576/994= 58% 208/339 = 61.4% Original Students 430/630 = 68% 144/206 = 69.9% Transfers In 167/364 = 46% 64/133 = 48.1% Transfers Out W/Request 195/357 = 54.6% 86/146 = 58.9% for Record Transfers In/Transfers Out 126/225 = 56% 45/83 = 54.2% *Mobility 236/496 = 47.7% 105/196 = 53.6%

36 SES Comparison between 10 th -12 th Grade and 12 th Grade Only 10 th 12 th 10 th -12 th 12 th Only 12 th Only Normal SES Low SES Normal SES Low SES Total Students Original Students Transfers In Transfers Out W/Request for Record Transfers In/Transfers Out *Mobility 66.2% 28.8% 8% 57.8% 29.9% 9% Dropouts 72 (13%) 58 (12.9%) 57 (25.7%) 22 (18.8%) *Mobility Rate = (Transfers In + Students who Transfer Out Transfer In/Transfers Out)/ Total Students

37 Academic Success Comparison by Ethnicity and Gender: 10 th 12 th Grade African-American Caucasian Hispanic Female Male Female Male Female Male Total Students 74/138 = 49/ /208= 138/217 52/90 = 52/96 = 53.6% = 42.6% 74.5% = 63.6% 6% 57.8% 54.2% Original Students 49/78 = 62.8% Transfers In 25/60 = 41.7% Transfers Out W/Request for Record Transfers In/Transfers Out 33/59 = 83.6% 16/56 = 28.6% 113/139 = 81.3% 42/69 = 60.9% 105/153 = 68.6% 33/64 = 51.6% 33/55 = 60% 19/35 = 54.3% 41/62 = 66.1% 11/34 = 32.4% 36/63 = 21/54 = 42/65 = 45/77 = 15/24 = 12/37 = 57.1% 38.9% 64.6% 6% 58.4% 62.5% 32.4% 21/39 = 53.8% 12/37 = 32.4% 29/43 = 67.4% 30/47 = 63.8% 10/15 = 66.7% 9/19 = 47.4% *Mobility Rate 62.3% 67.8% 43.8% 29.5% 48.9% 52%

38 Academic Success Comparison by Ethnicity and Gender: 12 th Grade Only Total Students 16/39 = 41% Original Students African-American Caucasian Hispanic Female Male Female Male Female Male 16/22 = 72.7% 15/42 = 35.7% 10/21 = 47.6% 59/73 = 80.8% 43/47 = 91.5% 40/64 = 62.5% 29/39 = 71.8% 21/32 = 65.6% 11/18 = 61.1% 19/38= 50% 15/25 = 60% Transfers In 10/17 = 5/21 = 16/21 = 12/25 = 10/14 = 4/13= 58.8% 23.8% 76.2% 48% 71.4% 30.8% Transfers Out W/Request for Record Transfers In/Transfers Out 16/22 = 7/20 = 19/25 = 20/32 = 8/14= 7/16 = 72.7% 7% 35% 76% 62.5% 57.1% 43.8% 8/12= 66.7% 2/11 = 18.2% 10/16 = 62.5% 10/19 = 52.6% 5/5 = 100% 4/8 = 50% *Mobility Rate 69.2% 71.4% 47.9% 59.4% 62.5% 55.3%

39 Early Elementary Math: Overall Mean Performance Score (State Test) for rd Graders Students who begin their education early (Pre-K) and stay in the same school tend to do well academically.

40 Descriptive Statistics: Early Elementary Math Mean Performance Scores (State Test) Continuous Enrollment Mean Std. Deviation N PreK-3rd K-3rd st-3rd nd-3rd rd Only Students who begin their education early (Pre-K) and stay in the same school tend to do well academically.

41 Early Elementary Reading: Overall Mean Performance Score (State Test) for rd Graders Students who begin their education early (Pre-K) and stay in the same school tend to do well academically.

42 Descriptive Statistics: Early Elementary Reading Mean Performance Scores (State Test) Continuous Enrollment Mean Std. Deviation N PreK-3rd K-3rd st-3rd nd-3rd rd Only Students who begin their education early (Pre-K) and stay in the same school tend to do well academically.

43 8 th Grade Math: All Students (State Test) Mean Performance Score

44 8th Grade Reading: All Students (State Test) Mean Performance Score

45 8th Grade Math: All Students (State Test) Mean Performance Score by Mobility

46 8th Grade Reading: All Students (State Test) Mean Performance Score by Mobility

47 Academic Performance of Durational Cohorts at the Middle School Level (State Test) Students who begin their education early and stay in the same school tend to do better academically.

48 What does the Longitudinal Data Tell Us? Mobility has a Huge Impact on Academic Achievement and Student t Success The Negative Impact of Mobility is Evident at all Levels of Education (Elementary, e Middle, High School)

49 Is Mobility a Problem Specific to Western Heights Alone? 10 th 12 th Grade 12 th Grade Only Total Students *Mobility Rate 49.9% 57.8% Dropouts 130 (13.1%) 59 (17.4%) Do you know your mobility rate and its relationship to academic success at all school levels?

50 How Do We Address the Problem of Mobility in Our Schools? Establish Intervention Strategies at Site Levels that Effectively Identify and Serve Mobile Students Set Standards for the Articulation of Real-Time Data Between Districts and States: Student Performance Data Teaching and Learning Resources Data Demographic Data

51 Survey Tool for Early Identification of Mobile Students

52 A Common Language is Necessary to Support Instruction and Demonstrate Accountability in Understandable Terms Scheduling Lesson Planning Course (Reading) Course (Reading) Term 00 Quarter 0 Period 00 Learning Styles 000 Section 000 Critical Thinking Skills 000 Unit 0 Grade Level 00 Site 0000 Student Learner Characteristics Teacher 000 Student Instructional Code (Reading Comprehension / Sequencing) Specific Student Learning Styles 00000

53 Why Do We Need a Common Language Initiative (CLI)? Current nomenclature for state standards of instruction are not synchronized over time to support cohort analysis within a LDS. This condition is NOT an Oklahoma-specific ao aspec c issue, but is prevalent in most, if not all, state standard descriptors.

54 Elements of a Common Language Initiative (CLI) Samples of Families of Knowledge

55 CLI Development Processes

56 Instructional Report Process Mapping: Course An Example Using CLI Creation of Modified Lesson Plan(s) Reading Instructional Goal(s)/District and State Standard(s) Comprehension 5 Comprehension Formative Assessment(s) Sequencing Creation of Lesson Plan(s) Analyze and explain the causes, motivations, sequences, and results Identification of Appropriate Instructional Content of events from a text. Bloom s, Gardner s, IEP, ELL, 504, Gifted, etc. Delivery of Lesson Plan(s) and Instructional Content Student/Teacher Schedules and Student/Parent Info Summative Assessment(s) Any/All of the Above Remediation Acceleration Start Process Again

57 The Future of Reporting: Establishing a Common Language Iowa Test of Basic Skills Course Reading Comprehension Comprehension Oklahoma Core Curriculum Tests Reading Comprehension/Critical Literacy Comprehension Stanford 9 Reading Reading Comprehension Action/Reason/Sequence Sequencing State Learning Standard d Language Arts Comprehension/Critical Literacy A.3.4.d. Analysis and Evaluation: Analyze and explain the causes, motivations, sequences, and results of events from a text Sequencing Language Arts Reading

58 Teacher s Response to Instructional Needs Curriculum Planning by Class, Group, or Individual Student Teachers have access to the Curriculum Resource Library. They can search for, share, adapt and store lesson plans and content from multiple sources. (This open-source system was created by an inhouse developer.) Easy Planner, Amy s Lesson Plans Search by Topic Search by Grade Level Search by Research-Based Instructional Strategies Search by State Standards, etc. 5858

59 Site Level Response to School Improvement: Professional Learning Communities (PLCs) Professional Learning Communities (PLCs for Each School) A Place For Educators to Collaborate and Communicate Design, Acquire, and Disseminate Instructional Support Activities

60 Current Administrative Reporting: Analytical Scope/Trend-Based

61 Recommendations: National Level Raise awareness of the problem of mobility through public relations campaigns to all stakeholders Create effective parameters for developing state & national standards of instruction. In other words, set common industry standards (e.g., ASTM) for developing state & national instructional standards Develop more effective guidance regarding the identification of mobile students in districts Bring in other social service providers to discuss ways of addressing mobility Create a Common Language Initiative to provide uniform reporting of student learning needs Develop well defined, universal enrollment status codes for all schools Incorporate SIF-based reporting

62 Recommendations: State Level Acknowledge the problem of mobility and raise awareness of all stakeholders Develop/Adopt metrics to recognize districts/schools/teachers who improve learning achievement for mobile students Join the Common Language Initiative to improve uniform reporting across schools/districts/states Use SIF technology for all educational software to support exchange of educational information in near real time Adopt p a well-defined, adequately-detailed enrollment coding system that is uniform across schools within the state and across other states States should use LDS aggregates of student performance data rather than managing individual academic performance data at the state level.

63 Recommendations: School District and Site Levels Know site mobility rates and inform all stakeholders of this issue and its impact on learning Develop survey intakes to identify mobile families/students in order to provide targeted intervention/assistance Adopt effective longitudinal metrics that acknowledge teachers who improve students academic performance Use common language(s) to enhance longitudinal reporting Use SIF-based technology on all educational software needs for ease of data exchange between disparate systems Involve the community (business and other social service providers) to assist y( p ) in maximizing the impact of educational efforts

64 Contact Information Joe Kitchens Western Heights Superintendent Dr. Mwarumba Mwavita Western Heights Director of School Improvement us Dr. Lisa McLaughlin Western Heights Assistant Superintendent Dr. John Steffens Assistant Vice-Provost Emeritus Executive Director, Public Service Institute University of Oklahoma