Curriculum-Based Measurement What is it? How can it be useful?

Curriculum-Based Measurement What is it? How can it be useful? Dr. Michelle Hosp University of Utah January 15 th, 2007

Acknowledgements Drs. Lynn & Doug Fuchs American Institutes for Research (AIR) 2

Overview Development of Curriculum-Based Measurement (CBM) How CBM is Different from Other Assessments How to Conduct Reading CBM CBM and IEPs 3

Development of CBM

History of CBM Developed at the University of Minnesota by Stan Deno and Phyllis Mirkin in the late 70 s & early 80 s An expansion of the Data-Based Program Modification (DBPM) model based on mastery measurement 5

History of CBM (cont) History of CBM (cont) Out of this came CBM which focused on Efficiency for teachers Reliable and valid ways to measure student progress Provide assessment information to help teachers make instructional decisions Provide information about student growth within an instructional program 6

Leaders in the Area of CBM Stan Deno Lynn & Doug Fuchs Mark Shinn 7

CBM Research CBM research has been conducted over the past 25+ years Research has demonstrated that when teachers use CBM for instructional decision making: Students learn more Teacher decision making improves Students are more aware of their performance 8

How CBM is Different from Other Assessments

What is the Difference Between Traditional Assessments and CBM? Traditional assessments: Lengthy tests Not administered on a regular basis Teachers do not receive immediate feedback Student scores are based on national scores and averages and a teacher s classroom may different tremendously from the national student sample 10

What is the Difference Between Traditional Assessments and CBM? (cont) CBM provides an easy and quick method to gathering student progress Teachers can analyze student scores and adjust student goals and instructional programs Student data can be compared to teacher s classroom or school district data 11

What is the Difference Between Curriculum-Based Assessments (CBA) and CBM? Curriculum-Based Assessment Measurement materials aligned with school curriculum Measurement is frequent Assessment information is used to formulate instructional decisions CBM is one type of curriculum-based assessment 12

Different Types of CBA 13

Salient Features of Mastery Measurement Curriculum is broken down into specific subskills or short-term instructional objectives Assess specific skill that is being taught Example r controlled vowels in reading Single digit addition, without regrouping Skills usually assessed using teachermade tests or tests in curriculum 14

Downsides to Mastery Measurement Skill Hierarchies Teacher-Made Tests Reliability & validity are unknown Retention & generalization of skills are not usually measured Measurement of Short-Term Instructional Objectives Measurement shifts occur making it difficult to monitor overall progress because: 1. different skills are measured at different points in time 2. different skills are not of equal difficulty and do not represent equal curriculum units 15

Basic Math Facts 100 Addition Facts Subtraction Facts Multiplication Facts 90 80 70 Accuracy 60 50 40 30 20 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Weeks 16

Most Forms of Classroom Assessment Are Mastery Measurement CBM is NOT Mastery Measurement CBM is a General Outcome Measure

Salient Features of General Outcome Measurement General domains, not subskills Keeps global curriculum outcomes intact and uses long-term goals Makes no assumptions about instructional hierarchy for determining measurement (i.e., CBM fits with any instructional approach) No measurement shifts 18

Salient Features of General Outcome Measurement (cont) Incorporates automatic tests of retention retention and generalization Measurement of Long-Term Curricular Goal Performance Test Construction Standardized procedures used to assess performance on the long-term goal Reliability & validity can be determined 19

Downsides to General Outcome Measurement Often lacks information on specific subskills If interested in identifying specific skills to teach, GOM not appropriate Need to use a diagnostic measure Fidelity of implementation is important 20

25 Instructional Change 20 Correct Digits 15 10 5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Weeks 21

The Basics of CBM CBM monitors student progress throughout the school year Students are given reading probes at regular intervals Weekly, bi-weekly, monthly Teachers use student data to quantify short- and long-term goals that will meet end-of-year goals Estimate rates of student improvement 22

The Basics of CBM CBM tests are brief and easy to administer All tests are different, but assess the same skills and the same difficulty level CBM scores are graphed for teachers to use to make decisions about instructional programs and teaching methods for each student Identify students who are not demonstrating adequate progress 23

What Does CBM Oral Reading Fluency Look Like?

Phonics Grade 2 Reading Curriculum cvc patterns cvce patterns cvvc patterns... Sight Vocabulary Comprehension Identification of who/what/when/where Identification of main idea Sequence of events Fluency 25

Grade 2 Reading CBM Each week, every student reads aloud from a second-grade passage for 1 minute Each week s passage is the same difficulty As student reads, teacher marks errors Count number of words read correctly Graph scores 26

Ron's class was studying horses. They learned what horses eat. Horses eat oats. They eat apples. Horses also need a great deal of water to drink. The students also learned how to care for horses. Horses need to be groomed. To groom a horse you have to brush its coat until it shines. You must also clean out a horse's stall each day. Ron's class was going on a field trip. They were going to visit a horse farm. They were even going to get to ride a horse! 27

Ron's class was studying horses. They learned 7 what horses eat. Horses eat oats. They eat apples. 16 Horses also need a great deal of water to drink. The 27 students also learned how to care for horses. Horses 36 need to be groomed. To groom a horse you have to 47 brush its coat until it shines. You must also clean out a 59 horse's stall each day. 63 Ron's class was going on a field trip. They were 73 going to visit a horse farm. They were even going to 84 get to ride a horse! 89 28

CBM Oral Reading Fluency 29

READING CBM Not interested in making kids read faster Interested in kids becoming better readers The CBM score is an overall indicator of reading competence Students who score high on CBM Are better decoders Are better at sight vocabulary Are better comprehenders Correlates highly with other global measures of reading (e.g. high stakes testing; commercially available tests; teacher made tests) 30

What We Look For in CBM Students whose scores are going up Indicates they are becoming better readers Students whose scores are flat Indicates they are not profiting from instructional program and require a change in their instructional program 31

Sarah s Progress on Words Read Correctly Sarah Smith Reading 2 Words Read Correctly 180 160 140 120 100 80 60 40 20 0 Sep Oct Nov Dec Jan Feb Mar Apr May 32

Jessica s Progress on Words Read Correctly Words Read Correctly 180 160 140 120 100 80 60 40 20 0 Jessica Jones Reading 2 Sep Oct Nov Dec Jan Feb Mar Apr May 33

STEPS TO CONDUCTING READING CBM

Steps to Conducting Reading CBM Step 1: How to Place Students in a Reading CBM Task Step 2: How to Identify the Level of Material for Monitoring Progress for Oral Oral Reading Fluency Step 3: How to Administer and Score CBM Oral Reading Fluency 35

Steps to Conducting Reading CBM (cont) Step 4: How to Graph Scores Step 5: How to Set Ambitious Goals Step 6: How to Apply Decision Rules to Graphed Scores to Know When to Revise Programs and Increase Goals 36

Step 1: How to Place Students in a Reading CBM Task Kindergarten: Letter-sound fluency First Semester Grade 1: Word-identification fluency Second Semester Grade 1 and Grades 2-3: Oral reading fluency Grades 4-8: Maze fluency 37

Kindergarten CBM Letter-Sound Fluency Student presented with page of 26 random letters on LSF Student Copy Student reads the letter sounds for 1 minute Teacher marks errors on LSF Teacher Score Sheet 38

CBM Letter Sound Fluency (LSF) Student copy of LSF Letters in the box are practice 39

CBM Letter Sound Fluency (LSF) LSF Teacher Score Sheet Errors are marked with a slash (/) Score is adjusted if student completes in less than 1 minute 40

Kindergarten CBM Letter-Sound Fluency Alternate-passage stability (3 weeks):.92 -.94 Criterion validity with WRMT:.58 -.71 Predictive Validity with CBM (Fall 1 to Spring 1):.68 Predictive Validity with CBM (Fall K to Spring 1):.54 Predictive Validity with TerraNova (Fall 1 to Spring 1):.53 Predictive Validity with TerraNova (Fall K to Spring 1):.43 41

First-Semester Grade 1 CBM Word-Identification Fluency Student presented with a list of 50 words Student reads words for 1 minute Teacher marks errors on WIF Score Sheet 42

CBM Word Identification Fluency (WIF) CBM WIF Student list 43

CBM Word Identification Fluency (WIF) WIF Teacher Score Sheet Words read correctly marked as 1 Words read incorrectly marked as 0 44

First-Semester Grade 1 CBM Word-Identification Fluency Alternate-passage stability (3 weeks):.97 Criterion validity with WRMT:.75 -.93 Predictive Validity with CBM (22-30 weeks):.68 -.87 Predictive Validity with TerraNova (22-30 weeks):.62 -.76 45

Second Semester Grade 1 and Grades 2-3 CBM Oral Reading Fluency For students in grades 1-8 Student reads grade-appropriate passage for 1 minute from ORF Student copy Teacher marks errors on ORF Teacher copy 46

CBM Oral Reading Fluency (ORF) ORF Student copy 47

CBM Oral Reading Fluency (ORF) ORF Teacher copy Numbers along margin allow for easy calculation of words attempted 48

Second Semester Grade 1 and Grades 2-3: CBM Oral Reading Fluency Alternate-passage stability (3 weeks):.92 Criterion validity with WRMT:.70 -.89 Predictive validity with CBM (22-30 weeks):.72 -.86 Predictive validity with TerraNova (22-30 weeks):.65 -.72 49

Grades 4 8 CBM Maze Fluency For students in grades 1-8 Administered to a group of students at one time Students read passage and circle correct word for each blank Tests lasts for 2.5-3 minutes Teacher grades each test later 50

CBM Maze Fluency Maze Student copy Students receive 1 point for each correct answer Scoring is discontinued if 3 consecutive errors are made 51

CBM Maze Fluency Juan s CBM Maze 10 correct answers before he made 3 consecutive mistakes Juan s score = 10 52

Grades 4 8 CBM Maze Fluency Alternate-passage stability (3 weeks):.94 Criterion validity with WRMT:.71 -.93 Predictive Validity with CBM (22-30 weeks):.70 -.84 Predictive Validity with TerraNova (22-30 weeks):.67 -.74 53

Step 2: How to Identify the Level of Material for Monitoring Progress Generally, students use the CBM materials prepared for their grade level However, some students may need to read from a different grade level if they are well below grade-level expectations 54

Step 2: How to Identify the Level of Material for Monitoring Progress To find the appropriate CBM level: Determine grade level text for student Administer 3 CBM Passage Reading Fluency passages If student reads 10-50 words correct in 1 minute but with less than 85-90% accuracy, move to next lower CBM level If student reads more than 50 words correct in 1 minute, move to the highest level of text where he/she reads between 10-50 words correct with 85-90% accuracy 55

Step 3: How To Administer and Score CBM ORF A word must be pronounced correctly, in accordance with the context of the sentence The word r-e-a-d must be pronounced reed when the context is: He will read the book not He will red the book Repetitions are ignored Bill jumped high. Read as: Bill jumped jumped high 56

Words Read Correctly CBM ORF Scoring Criteria (cont) Self Corrections (within 3 seconds) The dog licked Kim. Read as: The dog liked (2 seconds) licked Kim. Insertions are ignored The big dog ran home. Read as: The big black dog ran home. Dialect/Articulation I need a pen to sign my name. Read as: I need a pin to sign my name. 57

Errors for CBM ORF Scoring Criteria Mispronunciations/ Word Substitutions The house was big. Read as: The horse was big. Mother went to the store. Read as: Mom went to the store. Omissions Juan went to a birthday party. Read as: Juan went to a party. 58

Errors for CBM ORF Scoring Criteria (cont) Hesitations (over 3 seconds) Leslie is moving to Miami. Read as: Leslie is moving to MmmIiiiAaa (3 seconds) Reversals (words not read in the correct order) The fat cat walked past us. Read as: The cat fat walked past us. 59

Special Rules for CBM ORF Scoring Criteria Numerals are counted as words August 6, 2003 Should be read as August sixth, two thousandthree. Not August six, two zero zero three. John is in grade 3. Should be read as John is in grade three. Not John is in grade third. 60

Special Rules for CBM ORF Scoring Criteria (cont) Hyphenated Words are counted as separate words if they can stand alone son-in-law (3 words) forty-five (2 words) bar-b-que (1 word) re-evaluate (1 word) Abbreviations are counted as words Dr., Mrs., Ms., Mr. Should be read as: doctor, mis iz, miz, mister Not D-R, M-R-S, M-S, M-R. 61

Directions for Conducting Reading CBM Give the child a passage to read Place your copy in front of you Say to the child When I say please begin start reading aloud at the top of the page. Read across the page. Try to read each word. If you come to a word you don t know, I ll tell it to you. Be sure to do your best reading. Do you have any questions? Please Begin. Start your timer 62

Conducting Reading CBM Follow along and mark any errors on your paper After 1 minute say please stop Mark the last word read ] Count the number of errors and subtract it from the total number of words read (this will give you the total of Words Read Correct) Chart their score on the CBM graph If 4 consecutive scores fall below the goal line, a change in instruction is recommended 63

CBM Oral Reading Fluency Practice 64

ORF Practice Scoring -7 71 64 WRC 65

Step 4: How to Graph CBM Scores Graphing student scores is vital Graphs provide teachers with a straightforward way of Reviewing a student s progress Monitoring the appropriateness of student goals Judging the adequacy of student progress Comparing and contrasting successful and unsuccessful instructional aspects of a student s program 66

Step 4: How to Graph CBM Scores Teachers can use computer graphing programs OR Teachers can create their own graphs Create template for student graph Use same template for every student in the classroom Vertical axis has range of student scores Horizontal axis has number of weeks 67

Step 4: How to Graph CBM Scores The vertical axis is labeled with the range of student scores. Correctly Read Words Per Minute 100 90 80 70 60 50 40 30 20 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Weeks of Instruction The horizontal axis is labeled with the number of instructional weeks. 68

Reading CBM Graph W ords Read Correct 48 45 42 39 36 33 30 27 24 21 18 15 12 9 6 3 0 Instructional Change 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Weeks CBM Data Goal Line 69

CBM Benchmarks K: 40 letter sounds per min 1: 60 words correct from list or 50 words correct from text per min 2: 75 words correct from text per min 3: 100 words correct from text per min 4: 20 replacements to text per 2.5 min 5: 25 replacements to text per 2.5 min 6: 30 replacements to text per 2.5 min Fuchs & Fuchs, 2004 70

CBM Risk 1: < 15 sounds/min 2: < 15 words in text/min 3: < 50 words in text/min 4: < 70 words in text/min 5-7: < 15 maze replacements/2.5 min Fuchs & Fuchs, 2004 71

Norms for ORF CBM AIMSweb (2005) Hasbrouck & Tindal (2006) Grade Percentile Fall Winter Spring Fall Winter Spring 1 90% 75% 50% 25% 10% 42 19 7 2 0 74 44 23 12 6 104 78 50 27 14 N/A N/A N/A N/A 81 47 23 12 6 111 82 53 28 15 2 90% 75% 50% 25% 10% 100 75 51 24 13 126 101 75 50 24 143 117 91 67 39 106 79 51 25 11 125 100 72 42 18 142 117 89 61 31 3 90% 75% 50% 25% 10% 130 102 75 47 28 147 122 94 65 39 163 138 110 81 51 128 99 71 44 21 146 120 92 62 36 162 137 107 78 48 72

Norms for ORF CBM- Con t AIMSweb (2005) Hasbrouck & Tindal (2006) Grade Percentile Fall Winter Spring Fall Winter Spring 4 90% 75% 50% 25% 10% 5 90% 75% 50% 25% 10% 6 90% 75% 50% 25% 10% 148 122 98 72 47 168 140 111 84 59 181 158 131 101 70 167 140 113 89 62 182 156 126 98 73 198 170 144 113 83 183 154 125 100 72 198 171 140 108 81 211 183 155 125 94 145 119 94 68 45 166 139 110 85 61 177 153 127 98 68 166 139 112 87 61 182 156 127 99 74 195 167 140 111 82 180 152 123 98 72 194 168 139 109 83 204 177 150 122 93 73

Norms for ORF CBM- Con t AIMSweb (2005) Hasbrouck & Tindal (2006) Grade Percentile Fall Winter Spring Fall Winter Spring 7 90% 75% 50% 25% 10% 184 158 131 104 82 197 170 140 114 89 208 182 154 125 99 180 156 128 102 79 192 165 136 109 88 202 177 150 123 98 8 90% 75% 50% 25% 10% 185 163 140 109 78 194 171 148 117 86 200 180 156 129 98 185 161 133 106 77 199 173 146 115 84 199 177 151 124 97 74

Step 5: How to Set Ambitious Goals Once a few scores have been graphed, the teacher decides on an end-of-year performance goal for each student Two options for making performance goals: End-of-Year Benchmarking National Norms 75

Step 5: How to Set Ambitious Goals End-of-Year Benchmarking For typically developing students, a table of benchmarks can be used to find CBM end-of-year performance goal Grade Kindergarten Benchmark 40 letter sounds per minute (LSF) 1 st Grade 50 words correct (WIF) or 60 words correct (ORF) per minute 2 nd Grade 75 words correct per minute (ORF) 3 rd Grade 100 words correct per minute (ORF) 4 th Grade 20 correct replacements per 2.5 minutes (CBM Maze) 5 th Grade 25 correct replacements per 2.5 minutes (CBM Maze) 6 th Grade 30 correct replacements per 2.5 minutes (CBM Maze) 76

Step 5: How to Set Ambitious Goals National Norms For typically developing students, a table of average rates of weekly increase can be used to find end-of-year performance goal Grade ORF Norms Maze Norms 1 st Grade 2.00 0.40 2 nd Grade 1.50 0.40 3 rd Grade 1.00 0.40 4 th Grade 0.90 0.40 5 th Grade 0.50 0.40 6 th Grade 0.30 0.40 77

Step 5: How to Set Ambitious Goals Median: 29 4 th Grade ORF Norm: 0.90 Multiply by weeks left: 16 0.90 = 14.4 Added to median: 14.4 + 29 = 43.4 43.0 is end-of-year performance goal Grade ORF Norms Maze Norms 1 st Grade 2.00 0.40 2 nd Grade 1.50 0.40 3 rd Grade 1.00 0.40 4 th Grade 0.90 0.40 5 th Grade 0.50 0.40 6 th Grade 0.30 0.40 78

Step 5: How to Set Ambitious Goals Once the end-of-year performance goal has been created, the goal is marked on on the student graph with an X A goal-line is drawn between the median of the student s scores and the X 79

Step 5: How to Set Ambitious Goals WIF: Correctly Read Words Per Minute 100 90 80 70 60 50 40 30 20 10 0 X 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Weeks of Instruction 80

Step 6: How to Apply Decision Rules to Graphed Scores After goal-lines have been drawn, teachers use graphs to evaluate student student progress and formulate instructional decisions Standard decision rules help with this process 81

Step 6: How to Apply Decision Rules to Graphed Scores Based on 4 most recent consecutive scores: If scores are above goal-line, end-ofyear performance goal needs to be increased If scores are below goal-line, student instructional program needs to be revised 82

Step 6: How to Apply Decision Rules to Graphed Scores WIF: Correctly R 90 80 70 60 50 40 30 20 10 0 ead Words Per Minute100 most recent 4 points 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Weeks of Instruction goal-line X 83

Step 6: How to Apply Decision Rules to Graphed Scores 90 80 70 60 50 40 30 20 10 0 WIF: Correctly Read Words Per Minute100 Most recent 4 points most recent 4 points goal-line 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Weeks of Instruction goal-line X X 84

Other Ways to Use the CBM Database How to Incorporate Decision-Making Frameworks for Planning How to Use CBM to Identify Non-Responders Within a Response-to-Intervention Framework to Identify Disability 85

How to Incorporate Decision- Making Frameworks for Planning CBM graphs can provide the teacher with information about the class: Student CBM raw scores (screening) Graphic representation of every student Rate of growth for individuals and class (progress monitoring) List of students who may need additional intervention Additional Information: Grade, Schools, Districts 86

How to Use PM to Identify Non-Responders to Identify Disability Traditional assessment for identifying students with learning disabilities relies on intelligence and achievement tests Alternative framework is conceptualized as nonnon-responsiveness to otherwise effective instruction Dual-discrepancy: Student performs below level of classmates Student s learning rate below classmates 87

How to Use PM to Identify Non-Responders to Identify Disability All students do not achieve same degree of reading competence Just because reading growth is low, student doesn t automatically receive special education services If learning rate is similar to other classmates, student is profiting from the regular education environment 88

How to Use PM to Identify Non-Responders to Identify Disability If a low-performing student is not demonstrating growth where other students are thriving, special intervention should be considered Alternative instructional methods must be tested to address mismatch between student s learning requirements and requirements in conventional instructional program 89

CBM & IEPs Time Learner Behavior Level Content Material Criteria 90

IEP Goals & Objectives Time (The amount of time the goal is written for) In 30 weeks Learner (The student for whom the goal is written)..jose will Behavior (The specific skill the student will demonstrate) read aloud Level (The grade the content is from) 2 nd grade Content (What the student is learning about) reading Material (What the student is using) passages from ORF CBM progress monitoring material Criteria (The expected level of performance including time and accuracy)..90 words correctly in one minute with greater than 95% accuracy 91

Goals and Objectives Writing goals and objectives use the same principles but goals are typically written for 1 year and objectives are on a shorter time frame. 92

Reading Goals ORF Goal In one year, Edgar will read aloud a 2nd grade passage from ORF CBM progress monitoring material at 90 words correctly in 1 minute with greater than 95% accuracy. Mazes Goal In 30 weeks, Devin will correctly restore missing words on a 4th grade maze passage from Maze CBM progress monitoring material at 20 words correct in 3 minutes with greater than 95% accuracy. 93

Reading Objectives ORF Objective In 10 weeks, Edgar will read aloud a 2nd grade passage from OR CBM progress monitoring material at 50 words correctly in 1 minute, with greater than 95% accuracy. Maze Objective In 10 weeks, Devin will correctly restore missing words on a 4th grade maze passage from Maze CBM progress monitoring material at 8 words correct in 3 minutes, with greater than 95% accuracy. 94

CBM Materials AIMSweb / Edformation DIBELS Edcheckup McGraw-Hill Pro-Ed, Inc. Vanderbilt University 95

Questions 96