Curriculum-Based Measurement What is it? How can it be useful?

Transcription

1 Curriculum-Based Measurement What is it? How can it be useful? Dr. Michelle Hosp University of Utah January 15 th, 2007

2 Acknowledgements Drs. Lynn & Doug Fuchs American Institutes for Research (AIR) 2

3 Overview Development of Curriculum-Based Measurement (CBM) How CBM is Different from Other Assessments How to Conduct Reading CBM CBM and IEPs 3

4 Development of CBM

5 History of CBM Developed at the University of Minnesota by Stan Deno and Phyllis Mirkin in the late 70 s & early 80 s An expansion of the Data-Based Program Modification (DBPM) model based on mastery measurement 5

6 History of CBM (cont) History of CBM (cont) Out of this came CBM which focused on Efficiency for teachers Reliable and valid ways to measure student progress Provide assessment information to help teachers make instructional decisions Provide information about student growth within an instructional program 6

7 Leaders in the Area of CBM Stan Deno Lynn & Doug Fuchs Mark Shinn 7

8 CBM Research CBM research has been conducted over the past 25+ years Research has demonstrated that when teachers use CBM for instructional decision making: Students learn more Teacher decision making improves Students are more aware of their performance 8

9 How CBM is Different from Other Assessments

10 What is the Difference Between Traditional Assessments and CBM? Traditional assessments: Lengthy tests Not administered on a regular basis Teachers do not receive immediate feedback Student scores are based on national scores and averages and a teacher s classroom may different tremendously from the national student sample 10

11 What is the Difference Between Traditional Assessments and CBM? (cont) CBM provides an easy and quick method to gathering student progress Teachers can analyze student scores and adjust student goals and instructional programs Student data can be compared to teacher s classroom or school district data 11

12 What is the Difference Between Curriculum-Based Assessments (CBA) and CBM? Curriculum-Based Assessment Measurement materials aligned with school curriculum Measurement is frequent Assessment information is used to formulate instructional decisions CBM is one type of curriculum-based assessment 12

13 Different Types of CBA 13

14 Salient Features of Mastery Measurement Curriculum is broken down into specific subskills or short-term instructional objectives Assess specific skill that is being taught Example r controlled vowels in reading Single digit addition, without regrouping Skills usually assessed using teachermade tests or tests in curriculum 14

15 Downsides to Mastery Measurement Skill Hierarchies Teacher-Made Tests Reliability & validity are unknown Retention & generalization of skills are not usually measured Measurement of Short-Term Instructional Objectives Measurement shifts occur making it difficult to monitor overall progress because: 1. different skills are measured at different points in time 2. different skills are not of equal difficulty and do not represent equal curriculum units 15

16 Basic Math Facts 100 Addition Facts Subtraction Facts Multiplication Facts Accuracy Weeks 16

17 Most Forms of Classroom Assessment Are Mastery Measurement CBM is NOT Mastery Measurement CBM is a General Outcome Measure

18 Salient Features of General Outcome Measurement General domains, not subskills Keeps global curriculum outcomes intact and uses long-term goals Makes no assumptions about instructional hierarchy for determining measurement (i.e., CBM fits with any instructional approach) No measurement shifts 18

19 Salient Features of General Outcome Measurement (cont) Incorporates automatic tests of retention retention and generalization Measurement of Long-Term Curricular Goal Performance Test Construction Standardized procedures used to assess performance on the long-term goal Reliability & validity can be determined 19

20 Downsides to General Outcome Measurement Often lacks information on specific subskills If interested in identifying specific skills to teach, GOM not appropriate Need to use a diagnostic measure Fidelity of implementation is important 20

21 25 Instructional Change 20 Correct Digits Weeks 21

22 The Basics of CBM CBM monitors student progress throughout the school year Students are given reading probes at regular intervals Weekly, bi-weekly, monthly Teachers use student data to quantify short- and long-term goals that will meet end-of-year goals Estimate rates of student improvement 22

23 The Basics of CBM CBM tests are brief and easy to administer All tests are different, but assess the same skills and the same difficulty level CBM scores are graphed for teachers to use to make decisions about instructional programs and teaching methods for each student Identify students who are not demonstrating adequate progress 23

24 What Does CBM Oral Reading Fluency Look Like?

25 Phonics Grade 2 Reading Curriculum cvc patterns cvce patterns cvvc patterns... Sight Vocabulary Comprehension Identification of who/what/when/where Identification of main idea Sequence of events Fluency 25

26 Grade 2 Reading CBM Each week, every student reads aloud from a second-grade passage for 1 minute Each week s passage is the same difficulty As student reads, teacher marks errors Count number of words read correctly Graph scores 26

27 Ron's class was studying horses. They learned what horses eat. Horses eat oats. They eat apples. Horses also need a great deal of water to drink. The students also learned how to care for horses. Horses need to be groomed. To groom a horse you have to brush its coat until it shines. You must also clean out a horse's stall each day. Ron's class was going on a field trip. They were going to visit a horse farm. They were even going to get to ride a horse! 27

28 Ron's class was studying horses. They learned 7 what horses eat. Horses eat oats. They eat apples. 16 Horses also need a great deal of water to drink. The 27 students also learned how to care for horses. Horses 36 need to be groomed. To groom a horse you have to 47 brush its coat until it shines. You must also clean out a 59 horse's stall each day. 63 Ron's class was going on a field trip. They were 73 going to visit a horse farm. They were even going to 84 get to ride a horse! 89 28

29 CBM Oral Reading Fluency 29

30 READING CBM Not interested in making kids read faster Interested in kids becoming better readers The CBM score is an overall indicator of reading competence Students who score high on CBM Are better decoders Are better at sight vocabulary Are better comprehenders Correlates highly with other global measures of reading (e.g. high stakes testing; commercially available tests; teacher made tests) 30

31 What We Look For in CBM Students whose scores are going up Indicates they are becoming better readers Students whose scores are flat Indicates they are not profiting from instructional program and require a change in their instructional program 31

32 Sarah s Progress on Words Read Correctly Sarah Smith Reading 2 Words Read Correctly Sep Oct Nov Dec Jan Feb Mar Apr May 32

33 Jessica s Progress on Words Read Correctly Words Read Correctly Jessica Jones Reading 2 Sep Oct Nov Dec Jan Feb Mar Apr May 33

34 STEPS TO CONDUCTING READING CBM

35 Steps to Conducting Reading CBM Step 1: How to Place Students in a Reading CBM Task Step 2: How to Identify the Level of Material for Monitoring Progress for Oral Oral Reading Fluency Step 3: How to Administer and Score CBM Oral Reading Fluency 35

36 Steps to Conducting Reading CBM (cont) Step 4: How to Graph Scores Step 5: How to Set Ambitious Goals Step 6: How to Apply Decision Rules to Graphed Scores to Know When to Revise Programs and Increase Goals 36

37 Step 1: How to Place Students in a Reading CBM Task Kindergarten: Letter-sound fluency First Semester Grade 1: Word-identification fluency Second Semester Grade 1 and Grades 2-3: Oral reading fluency Grades 4-8: Maze fluency 37

38 Kindergarten CBM Letter-Sound Fluency Student presented with page of 26 random letters on LSF Student Copy Student reads the letter sounds for 1 minute Teacher marks errors on LSF Teacher Score Sheet 38

39 CBM Letter Sound Fluency (LSF) Student copy of LSF Letters in the box are practice 39

40 CBM Letter Sound Fluency (LSF) LSF Teacher Score Sheet Errors are marked with a slash (/) Score is adjusted if student completes in less than 1 minute 40

41 Kindergarten CBM Letter-Sound Fluency Alternate-passage stability (3 weeks): Criterion validity with WRMT: Predictive Validity with CBM (Fall 1 to Spring 1):.68 Predictive Validity with CBM (Fall K to Spring 1):.54 Predictive Validity with TerraNova (Fall 1 to Spring 1):.53 Predictive Validity with TerraNova (Fall K to Spring 1):.43 41

42 First-Semester Grade 1 CBM Word-Identification Fluency Student presented with a list of 50 words Student reads words for 1 minute Teacher marks errors on WIF Score Sheet 42

43 CBM Word Identification Fluency (WIF) CBM WIF Student list 43

44 CBM Word Identification Fluency (WIF) WIF Teacher Score Sheet Words read correctly marked as 1 Words read incorrectly marked as 0 44

45 First-Semester Grade 1 CBM Word-Identification Fluency Alternate-passage stability (3 weeks):.97 Criterion validity with WRMT: Predictive Validity with CBM (22-30 weeks): Predictive Validity with TerraNova (22-30 weeks):

46 Second Semester Grade 1 and Grades 2-3 CBM Oral Reading Fluency For students in grades 1-8 Student reads grade-appropriate passage for 1 minute from ORF Student copy Teacher marks errors on ORF Teacher copy 46

47 CBM Oral Reading Fluency (ORF) ORF Student copy 47

48 CBM Oral Reading Fluency (ORF) ORF Teacher copy Numbers along margin allow for easy calculation of words attempted 48

49 Second Semester Grade 1 and Grades 2-3: CBM Oral Reading Fluency Alternate-passage stability (3 weeks):.92 Criterion validity with WRMT: Predictive validity with CBM (22-30 weeks): Predictive validity with TerraNova (22-30 weeks):

50 Grades 4 8 CBM Maze Fluency For students in grades 1-8 Administered to a group of students at one time Students read passage and circle correct word for each blank Tests lasts for minutes Teacher grades each test later 50

51 CBM Maze Fluency Maze Student copy Students receive 1 point for each correct answer Scoring is discontinued if 3 consecutive errors are made 51

52 CBM Maze Fluency Juan s CBM Maze 10 correct answers before he made 3 consecutive mistakes Juan s score = 10 52

53 Grades 4 8 CBM Maze Fluency Alternate-passage stability (3 weeks):.94 Criterion validity with WRMT: Predictive Validity with CBM (22-30 weeks): Predictive Validity with TerraNova (22-30 weeks):

54 Step 2: How to Identify the Level of Material for Monitoring Progress Generally, students use the CBM materials prepared for their grade level However, some students may need to read from a different grade level if they are well below grade-level expectations 54

55 Step 2: How to Identify the Level of Material for Monitoring Progress To find the appropriate CBM level: Determine grade level text for student Administer 3 CBM Passage Reading Fluency passages If student reads words correct in 1 minute but with less than 85-90% accuracy, move to next lower CBM level If student reads more than 50 words correct in 1 minute, move to the highest level of text where he/she reads between words correct with 85-90% accuracy 55

56 Step 3: How To Administer and Score CBM ORF A word must be pronounced correctly, in accordance with the context of the sentence The word r-e-a-d must be pronounced reed when the context is: He will read the book not He will red the book Repetitions are ignored Bill jumped high. Read as: Bill jumped jumped high 56

57 Words Read Correctly CBM ORF Scoring Criteria (cont) Self Corrections (within 3 seconds) The dog licked Kim. Read as: The dog liked (2 seconds) licked Kim. Insertions are ignored The big dog ran home. Read as: The big black dog ran home. Dialect/Articulation I need a pen to sign my name. Read as: I need a pin to sign my name. 57

58 Errors for CBM ORF Scoring Criteria Mispronunciations/ Word Substitutions The house was big. Read as: The horse was big. Mother went to the store. Read as: Mom went to the store. Omissions Juan went to a birthday party. Read as: Juan went to a party. 58

59 Errors for CBM ORF Scoring Criteria (cont) Hesitations (over 3 seconds) Leslie is moving to Miami. Read as: Leslie is moving to MmmIiiiAaa (3 seconds) Reversals (words not read in the correct order) The fat cat walked past us. Read as: The cat fat walked past us. 59

60 Special Rules for CBM ORF Scoring Criteria Numerals are counted as words August 6, 2003 Should be read as August sixth, two thousandthree. Not August six, two zero zero three. John is in grade 3. Should be read as John is in grade three. Not John is in grade third. 60

61 Special Rules for CBM ORF Scoring Criteria (cont) Hyphenated Words are counted as separate words if they can stand alone son-in-law (3 words) forty-five (2 words) bar-b-que (1 word) re-evaluate (1 word) Abbreviations are counted as words Dr., Mrs., Ms., Mr. Should be read as: doctor, mis iz, miz, mister Not D-R, M-R-S, M-S, M-R. 61

62 Directions for Conducting Reading CBM Give the child a passage to read Place your copy in front of you Say to the child When I say please begin start reading aloud at the top of the page. Read across the page. Try to read each word. If you come to a word you don t know, I ll tell it to you. Be sure to do your best reading. Do you have any questions? Please Begin. Start your timer 62

63 Conducting Reading CBM Follow along and mark any errors on your paper After 1 minute say please stop Mark the last word read ] Count the number of errors and subtract it from the total number of words read (this will give you the total of Words Read Correct) Chart their score on the CBM graph If 4 consecutive scores fall below the goal line, a change in instruction is recommended 63

64 CBM Oral Reading Fluency Practice 64

65 ORF Practice Scoring WRC 65

66 Step 4: How to Graph CBM Scores Graphing student scores is vital Graphs provide teachers with a straightforward way of Reviewing a student s progress Monitoring the appropriateness of student goals Judging the adequacy of student progress Comparing and contrasting successful and unsuccessful instructional aspects of a student s program 66

67 Step 4: How to Graph CBM Scores Teachers can use computer graphing programs OR Teachers can create their own graphs Create template for student graph Use same template for every student in the classroom Vertical axis has range of student scores Horizontal axis has number of weeks 67

68 Step 4: How to Graph CBM Scores The vertical axis is labeled with the range of student scores. Correctly Read Words Per Minute Weeks of Instruction The horizontal axis is labeled with the number of instructional weeks. 68

69 Reading CBM Graph W ords Read Correct Instructional Change Weeks CBM Data Goal Line 69

70 CBM Benchmarks K: 40 letter sounds per min 1: 60 words correct from list or 50 words correct from text per min 2: 75 words correct from text per min 3: 100 words correct from text per min 4: 20 replacements to text per 2.5 min 5: 25 replacements to text per 2.5 min 6: 30 replacements to text per 2.5 min Fuchs & Fuchs,

71 CBM Risk 1: < 15 sounds/min 2: < 15 words in text/min 3: < 50 words in text/min 4: < 70 words in text/min 5-7: < 15 maze replacements/2.5 min Fuchs & Fuchs,

72 Norms for ORF CBM AIMSweb (2005) Hasbrouck & Tindal (2006) Grade Percentile Fall Winter Spring Fall Winter Spring 1 90% 75% 50% 25% 10% N/A N/A N/A N/A % 75% 50% 25% 10% % 75% 50% 25% 10%

73 Norms for ORF CBM- Con t AIMSweb (2005) Hasbrouck & Tindal (2006) Grade Percentile Fall Winter Spring Fall Winter Spring 4 90% 75% 50% 25% 10% 5 90% 75% 50% 25% 10% 6 90% 75% 50% 25% 10%

74 Norms for ORF CBM- Con t AIMSweb (2005) Hasbrouck & Tindal (2006) Grade Percentile Fall Winter Spring Fall Winter Spring 7 90% 75% 50% 25% 10% % 75% 50% 25% 10%

75 Step 5: How to Set Ambitious Goals Once a few scores have been graphed, the teacher decides on an end-of-year performance goal for each student Two options for making performance goals: End-of-Year Benchmarking National Norms 75

76 Step 5: How to Set Ambitious Goals End-of-Year Benchmarking For typically developing students, a table of benchmarks can be used to find CBM end-of-year performance goal Grade Kindergarten Benchmark 40 letter sounds per minute (LSF) 1 st Grade 50 words correct (WIF) or 60 words correct (ORF) per minute 2 nd Grade 75 words correct per minute (ORF) 3 rd Grade 100 words correct per minute (ORF) 4 th Grade 20 correct replacements per 2.5 minutes (CBM Maze) 5 th Grade 25 correct replacements per 2.5 minutes (CBM Maze) 6 th Grade 30 correct replacements per 2.5 minutes (CBM Maze) 76

77 Step 5: How to Set Ambitious Goals National Norms For typically developing students, a table of average rates of weekly increase can be used to find end-of-year performance goal Grade ORF Norms Maze Norms 1 st Grade nd Grade rd Grade th Grade th Grade th Grade

78 Step 5: How to Set Ambitious Goals Median: 29 4 th Grade ORF Norm: 0.90 Multiply by weeks left: = 14.4 Added to median: = is end-of-year performance goal Grade ORF Norms Maze Norms 1 st Grade nd Grade rd Grade th Grade th Grade th Grade

79 Step 5: How to Set Ambitious Goals Once the end-of-year performance goal has been created, the goal is marked on on the student graph with an X A goal-line is drawn between the median of the student s scores and the X 79

80 Step 5: How to Set Ambitious Goals WIF: Correctly Read Words Per Minute X Weeks of Instruction 80

81 Step 6: How to Apply Decision Rules to Graphed Scores After goal-lines have been drawn, teachers use graphs to evaluate student student progress and formulate instructional decisions Standard decision rules help with this process 81

82 Step 6: How to Apply Decision Rules to Graphed Scores Based on 4 most recent consecutive scores: If scores are above goal-line, end-ofyear performance goal needs to be increased If scores are below goal-line, student instructional program needs to be revised 82

83 Step 6: How to Apply Decision Rules to Graphed Scores WIF: Correctly R ead Words Per Minute100 most recent 4 points Weeks of Instruction goal-line X 83

84 Step 6: How to Apply Decision Rules to Graphed Scores WIF: Correctly Read Words Per Minute100 Most recent 4 points most recent 4 points goal-line Weeks of Instruction goal-line X X 84

85 Other Ways to Use the CBM Database How to Incorporate Decision-Making Frameworks for Planning How to Use CBM to Identify Non-Responders Within a Response-to-Intervention Framework to Identify Disability 85

86 How to Incorporate Decision- Making Frameworks for Planning CBM graphs can provide the teacher with information about the class: Student CBM raw scores (screening) Graphic representation of every student Rate of growth for individuals and class (progress monitoring) List of students who may need additional intervention Additional Information: Grade, Schools, Districts 86

87 How to Use PM to Identify Non-Responders to Identify Disability Traditional assessment for identifying students with learning disabilities relies on intelligence and achievement tests Alternative framework is conceptualized as nonnon-responsiveness to otherwise effective instruction Dual-discrepancy: Student performs below level of classmates Student s learning rate below classmates 87

88 How to Use PM to Identify Non-Responders to Identify Disability All students do not achieve same degree of reading competence Just because reading growth is low, student doesn t automatically receive special education services If learning rate is similar to other classmates, student is profiting from the regular education environment 88

89 How to Use PM to Identify Non-Responders to Identify Disability If a low-performing student is not demonstrating growth where other students are thriving, special intervention should be considered Alternative instructional methods must be tested to address mismatch between student s learning requirements and requirements in conventional instructional program 89

90 CBM & IEPs Time Learner Behavior Level Content Material Criteria 90

91 IEP Goals & Objectives Time (The amount of time the goal is written for) In 30 weeks Learner (The student for whom the goal is written)..jose will Behavior (The specific skill the student will demonstrate) read aloud Level (The grade the content is from) 2 nd grade Content (What the student is learning about) reading Material (What the student is using) passages from ORF CBM progress monitoring material Criteria (The expected level of performance including time and accuracy)..90 words correctly in one minute with greater than 95% accuracy 91

92 Goals and Objectives Writing goals and objectives use the same principles but goals are typically written for 1 year and objectives are on a shorter time frame. 92

93 Reading Goals ORF Goal In one year, Edgar will read aloud a 2nd grade passage from ORF CBM progress monitoring material at 90 words correctly in 1 minute with greater than 95% accuracy. Mazes Goal In 30 weeks, Devin will correctly restore missing words on a 4th grade maze passage from Maze CBM progress monitoring material at 20 words correct in 3 minutes with greater than 95% accuracy. 93

94 Reading Objectives ORF Objective In 10 weeks, Edgar will read aloud a 2nd grade passage from OR CBM progress monitoring material at 50 words correctly in 1 minute, with greater than 95% accuracy. Maze Objective In 10 weeks, Devin will correctly restore missing words on a 4th grade maze passage from Maze CBM progress monitoring material at 8 words correct in 3 minutes, with greater than 95% accuracy. 94

95 CBM Materials AIMSweb / Edformation DIBELS Edcheckup McGraw-Hill Pro-Ed, Inc. Vanderbilt University 95

96 Questions 96