CINDY GOLDSWORTHY: My name's Cindy Goldsworthy and you see that the two folks that

Transcription

1 CINDY GOLDSWORTHY: My name's Cindy Goldsworthy and you see that the two folks that are supposed to be here today are Cindy Goldsworthy and Kristen Lewald. Kristen at the very last minute could not be here, but one of her colleagues, Jenn Ross, volunteered, didn't she Jenn, to help with this session. So Jenn will be here also co-presenting. And Jenn, I'll let Jenn introduce herself, but one of the reasons that I'm thrilled to have Jenn here with me today is she's been with the PVAAS core team for five years. So we have the resident expert sitting here on PVAAS since Kristen can't be here. Some of you, I know some of you, I certainly don't know all of you. But just to kind of give you some background, my interest in this has been that I've been both involved in PVAAS and also involved with RTII. So what's nice is really to be able to talk with all of you about the connection between PVAAS and RTII. Because one of the things that I think we all struggle with is how do we pull all these things together that are happening in our field? And how do we help teachers understand how all of this fits together? So today's session is about that. Today's session is about connecting PVAAS and RTII. Neither one of those two things, PVAAS or RTII, are we going to go into in great depth. We're really going to be talking about the connection. So I know we've probably got a variety of folks here in the audience with various levels of knowledge about both of those. Feel free to ask questions and we'll do what we can to help you. But what I want to emphasize from the beginning is, we're real -- this is really not an overview of PVAAS. We're hoping that you have some knowledge and background about PVAAS as we move through this. Nor is this an overview about RTII. So we're again hoping that you've got some background and knowledge about RTII and we can really take this more or less to the next level where we can talk about how these two fit together. Now having said that, if you don't feel like you're in that category, don't leave because 1

2 we will do our best to explain things and answer any questions that you have and so forth. And I think this is probably a beginning step to lots more discussions about how these two really important initiatives connect and how you can use them together. So having said that, basically what we're going to be talking about today is what is the role of PVAAS and RTII. How does PVAAS aid in the whole decision-making process at various points in the RTII process? And also to help you zero in on what are the specific reports that PVAAS offers and how might you use those reports in those different decision points along the way in the RTII process. So in a way, to kind of short-circuit to okay what do you need, which of those reports in RTII -- in PVAAS, excuse me, will really help you in your response to instruction and intervention program. So those are the three basic questions or the three basic areas that we want to focus on today. I did a little bit of kind of trying to get to know you with the beginning, but let's have a sense of who's here in the audience because that will also help us gear our next two hours. Having said that, let me do a little bit of background. You all know that there's a keynote at 10:45. So we're going to end our session even though at the beginning slide it said 10:45, our session actually ends at 10:25. And then you'll have about 20 minutes to get some refreshments and coffee and head to the keynote. And we can take a stretch break between 8:30 and 10:25 if need be, but certainly feel free to take breaks whenever you need to do that as well. So who's here in the session? Let's do a little bit of this is who I am. How many teachers do we have here in the session? Okay, how many principals do we have? All right, how many central office folks do we have? Good, IU folks? All right, excellent. And counselor, we have a counselor. Did I leave anybody out? PaTTAN folks, PaTTAN folks, yes I knew that. 2

3 Hello PaTTAN folks. How many of you -- if you did a one, two, three. One, don't know anything about the topic. Two, have some good background knowledge. Three, I really feel like I could be up here talking about this. PVAAS, let's just do PVAAS first. PVAAS, one, two, three. Very nice, okay, very good. RTII, one, two, three. Okay, excellent, all right, good. So looks like everybody feels like they've got some background knowledge in both of those topics and that will be very helpful. This is always difficult to do at the beginning before you get people talking, but it would be really helpful if you shared with us some thoughts about what you hope to learn today in this session. Why'd you pick this session? What are you hoping to get out of this session? Anybody willing to share that? And since I know a few names, you may want to just volunteer before I call your name, but okay Jenn? AUDIENCE MEMBER: Working with a lot of schools, especially secondary RGI schools, we do a lot with PVAAS and I feel like I need a little bit more oomph so I can compare to gap [inaudible]. CINDY GOLDSWORTHY: Okay, okay. So as a consultant, trying to help schools pair the two together. Good, anybody else? Yes, in the back. AUDIENCE MEMBER: We have [inaudible] teams and so it's just another act through data that I can bring in besides the other pieces, the other RTII [inaudible]. CINDY GOLDSWORTHY: Good, good. And that really is the essence of the talk today, is really how do you add that to the other pieces that you're using, the other assessment tools or the other assessment information. Other thoughts? AUDIENCE MEMBER: I think anytime I hear anything about PVAAS I learn just a little more, a little more and another nugget. So just to sure up my [inaudible]. 3

4 CINDY GOLDSWORTHY: Good, good. So you know, maybe you're a two and you want to get to a three, right. That kind of thing, good. Somebody else had their hand up. AUDIENCE MEMBER: My school didn't make AYP last year and I'm one of the teacher members of the team to draft a plan. And every time PVAAS gets brought up, a lot of teachers go ooh. So I need to -- CINDY GOLDSWORTHY: Okay, excellent. So typical responses and I think that does give us some sense of where to go. And again, as I said, please ask questions. This is a small enough group that I'd rather you ask the questions and Jenn is here to answer those questions if I can't. And actually we'll just have her answer those questions and we can hopefully get you leaving here feeling like you -- this was time well spent. So we're going to start with a little PVAAS quiz just to see where you are and kind of sure up some of your knowledge. So number one, PVAAS provides individual growth and projection reports for each student. So if you believe that's true, thumbs up. If you believe it's false, thumbs down. Okay so everybody, true or false? Okay so we've got a mixture here of true and false. So the answer to that is actually false. Somebody who said false, why did you say false? Yes, in the back. AUDIENCE MEMBER: In third grade there are no growth and projection reports for third grade students. CINDY GOLDSWORTHY: Okay, let me hear another reason why it's false. Yes? AUDIENCE MEMBER: It can't measure individual growths and things. CINDY GOLDSWORTHY: Okay, the answer to that is false because you cannot measure individual growth. Growth for individual students, let me say it that way. You can't get growth for individual students. You can get projections for individual students but not growth for individual students. And we're going to talk in just a bit about the difference between the 4

5 growth side of PVAAS and the projection side of PVAAS. So that'll hopefully be a review, but it'll help clarify that. Any questions on that? Let's look at number two. PVAAS growth calculations for all grades and subjects use the same methodology. True, false, or I have no idea. Okay, all right, that's fair. True, false, and I have no idea. The answer to that one is also false. And again, you'll see an explanation of this in just a bit when Jenn goes over some of the PVAAS reports with you. There is a different methodology used for grades, one methodology for grades four and eight, four through eight, excuse me. Another methodology for grade 11. And Jenn will explain that to you when we get there. Number three, high-achieving schools hit a ceiling and cannot show growth. So meaning that if you're a high-achieving school, at some point you can't show any more growth because you're up there. True or false? True, false. Very good, answer to that one is also false. All schools can show growth. And that's actually a misconception that a lot of folks have. So you may, in fact, have, especially those of you that are -- that said part of your role is to work with school districts and work with teachers. You may have folks that are saying, well can we get any better? We're already doing really well. We're already getting all greens. Can we really be still showing growth? And the answer to that is yes. All schools can show growth. Value added reports provide lists of student growth values. And you say false because? AUDIENCE MEMBER: You just said you can't show growth. CINDY GOLDSWORTHY: Okay, so you can't show growth. All right, how many people said false to that? All right, good. Number five, performance diagnostic reports display the growth of groups of students predicted to be in each PSSA category. Is that -- first of all, how many of you know what the performance diagnostic reports are or remember what they are? Okay, so 5

6 only a couple of people. So this is probably going to be a guess for most of you. True or false? The performance diagnostic reports display the growth of groups of students projected to be in each PSSA category, meaning kids projected to be proficient, the performance diagnostic report shows the growth. True, false. The answer to that is true. You could get growth with groups of students. And again, we'll look at the performance diagnostic reports in a bit and look at how that works. PVAAS projection summaries provide a guarantee about future PSSA performance. True, false. Even if you probably had no background knowledge, this is a good example of one of those sentences on an assessment that's really not a good question, well-crafted question and why is that? What's the operative word there? Guarantee, right. Very few guarantees about anything. So no, it does not provide a guarantee. It provides a what? Not a guarantee, but what? What do the projections provide? Probability or likelihood. PVAAS can and should play an important role in the development and implementation of a multilevel system of prevention in a school system. True? Absolutely true. Good answer, everybody. Yes, that was a true. Number eight, you are missing an important piece if you are not using PVAAS in the screening process of RTII. What do you think? Absolutely true. Okay, now many of you had different answers to some of these questions. So if you can keep those questions in mind, we'll get more clarification about the answers as we move into a review, an overview of some of the PVAAS reports and PVAAS concepts, which is what Jenn's going to do for us. Aren't you? JENNIFER ROSS: There we go. Is it on? Okay, good. Two types of information that PVAAS provides. And I know that Cindy already kind of gave you a quick review of that with some of the questions that we went over. We're sitting here today, knowing with kids in our classrooms 6

7 -- not today physically, I mean it's the summer. But you know, we're pretending here we're in September, PVAAS data comes out. We've got kids in our classrooms. You know, fourth, fifth, sixth grade and so on. PVAAS is going to provide you when it comes out in September information looking back. What happened with groups of students last year? Helping you to evaluate what happened with the programmings that you have in your school. Did those students make a year's worth of growth for that year that they were in school and through your intervention programs or whatever you have. It's all about groups of students and helping you evaluate did they work. Also provides you though, a look to the future. So if you're sitting here in September, you're going to be able to get projection information about, on individual students and groups of students, about where they likely to be in March when they take that PSSA. Key difference between a lot of other assessments that you have, may be Foresight, Study Island, things like that. While those, and particularly Foresight say, you know, it's more of like a predicted scale score, what Foresight's really providing you is this line in the middle. It's saying if we give a student the Foresight in September, that's how they're predicted to do if they would've taken the PSSA in September. It's about where they are right then, which is very good information that you do have to have. Have to know where they are right now so you can guide your instruction from that point. What PVAAS though is telling you, we're actually projecting based on their entire history, where are they likely to be in March. So this way, you get information about everything. Looking back, you have other assessment data telling you about today and you also have information looking towards the future as well, okay. And each one of these pieces is important in that process, that RTII process, okay. And again, I always just like to remind 7

8 folks right, currently no individual measures of student growth. Question? AUDIENCE MEMBER: Well would you say that Study Island is also doing that today? JENNIFER ROSS: Yes, yes. Really, PVAAS is the only one that's a true projection to the future. And I know we tend to use that word prediction and projection interchangeably, but there is a subtle difference. Really that prediction, what you know, Foresight, Study Island, anything like that give, is really about today. If they did something today, this is how we would predict that they would do right now. That projection is actually a projection into the future, okay. So I'll let Cindy take this next piece. Sorry. CINDY GOLDSWORTHY: So PVAAS is a tool that aids in those decisions. As we know in RTII, there's a number of decision points along the way. For example, we know that we have to decide in RTII who gets intervention. What type of intervention do they need? We know that we have to decide about movement between tiers. The student's getting tier two level support, does the student need tier three level support? All of those kinds of decision points all along the way, there are various reports in PVAAS that can be added to the array of other information that you're looking at in order to make those decisions. In the whole universal screening process as you saw on the last question of the quiz, there PVAAS plays a role. Various reports in PVAAS can help with that. So just to get a feel from all of you as a group, where are some of the places -- is there anybody already using PVAAS as part of their decision-making in their RTII process? And if so, where are you using it? Anybody in the group doing that currently? No? What I hope to show with you as we go through this today are some examples, particularly from one district that I worked with two, three years ago. I left that district, but I'm still continuing to work with them and that's Derry Township School District in Hershey. So I want to -- as we go through this, I'll give you some examples of where we used PVAAS all 8

9 along the way in the RTII process. And maybe as we get started, some of you will say oh we are doing that. And we'll add that to it as we go along. So when we connect PVAAS and RTII, there are a number of questions that we can think of. And I think this may help frame this for you a bit more. So for example, to what degree is our tier one working? That's a big question that we want to ask in RTII. Are there PVAAS reports that you could add to? Other assessment data and other information that you have? And the answer is yes, and we'll talk about what those are. Who needs intervention? Big question that folks have. How do we figure that out? What's the universal screening process that's in place? Does PVAAS play a role there? The answer is yes. There are particular specific PVAAS reports that can be used to help you figure that out. What's our universal screening process? And then from there, how are we going to whittle down and figure out who it is that needs our interventions? And we don't want tier two or tier three support to be something that students do for life. We want to have entrance criteria as well as exit criteria. What PVAAS reports can help us there? That's another decision-making point. Overall, we want to look at, now that we've figured out who those students are that need supports through our tier two and tier three intervention, is it working? And are there reports in PVAAS that we can use, added to other data and other information, that will help us decide that. So at that level, it becomes not only how is the individual student doing? But also in a sense, a program evaluation. Is what we're doing, is our treatment, are the interventions that we're using, is the instruction that we're engaging students in, is that working? And then a big picture is how does the school plan as a whole system? So again, you've probably heard through this conference and other RTII work and sessions and readings et cetera, that it really is a systems approach. So how do we use some of the 9

10 PVAAS reports to come at this from a systems level? And Karen, you had a question. AUDIENCE MEMBER: I can wait until -- CINDY GOLDSWORTHY: No, that's good. AUDIENCE MEMBER: Going back to the tier two and three, are students responding to the intervention. Because we only have PVAAS once a year. That's really, there are other things you're going to check in beforehand, right, to make sure that intervention's working. That's really looking at those protocols that are in place, the standard treatment protocols, right? To see if they're effective for groups of students. CINDY GOLDSWORTHY: Yeah, yeah, yeah. We would be looking, exactly. And when we get to that, we'll go through that in more detail. Actually each one of these we're going to be talking about in a little more detail. But Karen's question was since we have that information in PVAAS once a year, we're also going to be doing, for example, progress monitoring along the way. Looking at individual students and how they're responding to the intervention. We're really looking with this particular question, at a treatment protocol or an intervention or a program as a whole. Is this working as a whole for a group of students? How does PVAAS assist in scheduling the intervention period, especially at the secondary level? This is a huge issue particularly at the secondary level. But how can you use PVAAS in a way that gives you, I like to think of it almost as a leg up in figuring out what your schedule's going to be. Because again, as many of you know that work in secondary schools, you're putting those schedules together very early in the year for the upcoming school year. So can we be using PVAAS to help us put together a potential list of students who may be in need of intervention? And then can a school system use PVAAS to help determine how to allocate resources? And the answer to that is yes, absolutely. There are PVAAS reports, again, added 10

11 with other information and data that can be used from a central office perspective, perhaps a buildings perspective as well, to say how are we going to allocate our resources? Time, money, people? Where are our greatest needs and what might we need to adjust based on what our data is telling us? So what you have in this PowerPoint and what we're going to refer to as we go through the morning here is a chart that really takes those questions that we just covered, is tier one working, who needs intervention, et cetera, and then links to that the PVAAS reports that will help you in answering those questions and in making decisions relative to those questions. And actually, there are, it's on two slides here when you download the PowerPoint if you haven't already. But each of those reports can be of course found in the PVAAS system. And just in case you're not familiar with this, we thought we would show you, if you can be patient with us for a second, where you find these reports. How many of you have spent time on the PVAAS website? Okay, some of you, most of you have. So you know that across the top there's going to be a bar that has a colored bar that says reports. Everybody know what I'm talking about, okay? So we're going to show you that in a second because again, when you go back and pour over this PowerPoint, as reviewing from your session this morning, you'll want to perhaps go and look and make sure you know how to find those reports because we're not going to be able to spend a lot of time with that today. But there's the list of reports. And you can see how they're organized. At the top, we have custom reports. Then district reports, and under the district reports you'll see the value added and performance diagnostic. Those are two of the reports that we're going to be covering today. Under school reports, you'll see value added and performance diagnostic. We're going to be talking about those reports today. Under summary reports, you'll see value added and performance diagnostic. And again, we'll be mentioning those reports today. And 11

12 then towards the bottom of the bar is where you'll see the projection reports. So once you become familiar with the organization of this, it's a little bit easier to go and find the reports and see where they are. Questions or thoughts or comments? Is this clear so far, everybody? So you've got the questions, you've got the chart with where the reports are, you go to the site, you find the reports, okay. All right, so then let's take a look at the first one. We're going to start with projections. Just so this isn't confusing, we're not going to go through that chart question by question because what you will see -- let me back up for a minute and show you this. What you'll see is that many of those reports are used multiple times to answer different questions. So we're going to really talk about the reports themselves and then tie it back to the questions that they answer. So it's not that we're going to go through each one in a linear fashion. So you'll see, for example, projection reports can be used under who needs intervention. And then we also would be able to use projection reports about how does the district and school plan as a system. We can use projection reports on how do we schedule students for the intervention period and how should the district allocate resources. So those are some big questions that the projection reports can help us with. So how important are the projection reports? They are really, really, really, really important and they play a huge role in the whole RTII process. Jenn is going to talk with you about the projection reports, why they're important, some background on them, and make sure everybody's clear about that. And then we'll come back to let's talk about how this ties in with RTII. JENNIFER ROSS: I think we're going to try to use different microphones. Is this one on? That'll be easier I think than switching back and forth. Okay just to give you a refresher, I know many of you are familiar with PVAAS, but just to remind you of who can you get 12

13 projections on, okay. You can get projections out to basic, proficient, and advanced levels. What we mean by that is PVAAS will tell you in an individual student level as well as for groups of students, what's their probability of reaching that basic level or higher on a future PSSA. What's the probability of them reaching proficient or higher? And finally, what's their probability to reach an advanced level? For each one of those subject areas, okay, you can get projections. Math, reading, science, and writing. Underneath that, the number there on the left would be the grade level in which the student last tested, okay. Because we know this data doesn't come out until September, kind of like the PSSA. You give the PSSA in March, but it takes a while to get all of the analysis and scoring and everything done. So whenever you have a student maybe sitting there that was last tested in fourth grade, they're now in fifth grade when you have that data. You can project out to their fifth or sixth grade math or reading. So it's like you're looking out to that current year that you have them and even the following year. When you finally get down to you know, a student last testing in seventh grade, you can look out to their eighth grade year. Can't quite look out to 11th grade because that's four years into the future. It's just a little too far to make that projection. However, for students last testing in eighth grade, it'd be like your current ninth graders, you can project out to their 11th grade year. Where are they likely to be at that point in time knowing that you've got all of ninth grade, all of tenth grade, and a good portion of 11th grade to intervene. Yes? AUDIENCE MEMBER: When a student gets to the next level, meaning they're in ninth or tenth grade. They're no longer in the same building, so the people who need the scores can no longer get them. In other words, so how -- JENNIFER ROSS: There's, yeah there's a couple ways to deal with that. But you're right. 13

14 That was always a challenge for schools because most schools, middle school goes up to eighth grade. By the time they're in ninth grade, the high school doesn't have that information. Some schools allow folks at the high school to have access to the middle school data. That is one solution that some schools and districts have done. Sometimes it's a challenge though, if you're a bigger district and have multiple middle schools. And then you have to kind of, you know, be searching. One of the things that we did last year that we'll be continuing, that SAS who does the analysis, is offering at no charge for districts, is to submit current enrollment data. And so districts submit about ten pieces of information for a student. We had about almost 200 districts who did it last year found it very easy to do. In the fall you can submit data to say here's who I currently have in my buildings. So even if you have kids that are might be moving within your district, from one school to another such as that case, or they're moving from one elementary to another, maybe you have someone brand new to your district. You can tell them their PA secure ID, their number - - or I mean their name, PA secure ID number, their name, some basic demographic information. They will change those demographics within the system so that you now have access to their projections. AUDIENCE MEMBER: We did that. I was on the phone with Kristen Lewald and she could see those data but we could not. CINDY GOLDSWORTHY: And where are you? What -- AUDIENCE MEMBER: Excuse me? CINDY GOLDSWORTHY: You're in a district? AUDIENCE MEMBER: Correct. JENNIFER ROSS: Okay, so you're, I mean, well I can talk with you after this. I'm not real sure 14

15 right, it might have to do with the access that you have. That your district would have to give you access to some student level, I'm not real sure why. I mean, we can kind of work with you about that. But yeah, you should have access to that data, okay. Any other questions with that, yes? AUDIENCE MEMBER: Under science chart, when you say grade last tested. Because the science test is going to be in fourth, not third. So you take, you're saying that whatever data you have for your math between third grade [inaudible]. JENNIFER ROSS: Yes, yes. They've found the reliability studies that there is a high correlation between how students score in math and reading and how they do on science. That they are able to, yes, use those third grade math and reading scores to project how a student would do in science at the fourth grade, okay. And you'll notice, right, that NA there. That's because coming out of fourth grade again, it would be projecting to the next time they would take the science test which would be eighth grade. Again, it's a little too far in the future with four years, so that's really the only projection that's not available. The other grade levels, it's just to the next grade level in which they're tested for that, okay. Any other questions related to those projections on who you can get those on? Okay, just a refresher on what an individual student projection report looks like and just to give you a quick preview here of what's really important on this report when it comes especially to what Cindy talked about, that entrance and exit criteria that you would use in an RTII process. Just as a reminder, those red dots that you see on an individual student projection report are the student's academic history. How did they do at individual points in time on the PSSA? If you wanted to know what year or what grade level, you can find that down here on the x-axis. But you'll notice there are no lines because it's not about growth. We know those are individual points in time on what a 15

16 student did at that point in time in March on that PSSA. Looking at that history, academic history though, and both math and reading, if it's for science they also include science, they then get a projection score. That yellow square is the projected score. Knowing that those dashes that you see here that are in red, blue, and green are the cut points on the PSSA for that given year, that grade level that you're projecting to. For this student, it happens to be that we're projecting out to 11th grade reading, okay. So this is the child's reading history taking also into account their math, history. This is where we project the child's going to be. Looks like okay, above proficiency. But remember like Cindy said, there's no guarantee, okay. Just because that projection's there, there's going to be some measurement error, okay. There's going to be some projection error. What's really important is the middle part of that report. Kind of tried to blow it up here so you can see it a little better, but I know you probably can't read the numbers. What this says is for this child, given their projection, they've got about a 90% probability of reaching basic or higher. Pretty good probability. I'd feel good with that, that they're at least going to reach basic. Here, in the middle, then it says well what's their probability of reaching proficient or better? About a 68%. So it's saying, if you think about what that 68% probability means, it means if they take that test 100 times, they're probably going to be proficient 68 out of those 100 times. There's no magic number. It's kind of like where do you want that, where do you want that goal for that student to be. For me personally, I'd probably want that probability to be a little higher to feel real comfortable that I've got that child at likely to being proficient or not. Also gives you the advanced. In this case, we're saying with that projected score, got about a 20% chance of reaching advanced level, okay. The bottom of the report gives some general guidelines across the state. Generally we 16

17 tend to say if that probability to be proficient is 70% or higher, pretty likely that the student will be proficient. Maybe consider continuing the path that student's on; 40%-70% probability of being proficient, might want to think about, we might need to alter, maybe add to, supplement what they're getting with that core curriculum. We need to add some kind of possibly intervention for that child. Less than 40% probability, probably pretty unlikely. Probably some intensive intervention might be needed. Again though, this is one indicator using it with the other data that you have on that child. Question. AUDIENCE MEMBER: Okay [inaudible]. We have this projection for Isabella from eighth grade onto 11th grade reading and it's based on her past testing history in reading and math, science and writing also if she has -- JENNIFER ROSS: No, no. Math and reading are projected out for reading. Math and reading out to math, okay. If it's a science projection, they're using math, reading, and science. It all goes back into what did they find to be the best predictor. What had the highest correlation? What gives you the best prediction? And that's why they don't just use reading to project out on reading. They have found that it's actually a more precise, accurate projection if you use both the math and reading data. AUDIENCE MEMBER: Someone has a question. JENNIFER ROSS: Mm-hmm, go ahead. AUDIENCE MEMBER: So Isabella's projection is calculated. Now Isabella's in a school where notoriously, the groups of students do not perform as well in 11th grade as they do in eighth grade. It's consistently low. Does that have an effect on the projection that's made for us here? JENNIFER ROSS: Yes. And here's, and I'll repeat the question. I know we're videotaping so I'm going to repeat for the larger audience as well. Question was historically Isabella is likely 17

18 to go to a school that historically that school has not seen growth with 11th graders. Does that affect her projection? The answer is yes, as long as SAS has that information about where she's likely to go. And across the state we've worked with PDE. We know feeder patterns for schools. This -- and there's actually a statement right here in the middle to remind folks of that. That this projection does take into account after you know, you've taken into account the child's history, okay. That gets a projection. Then it might be adjusted slightly based on where that student's likely to go. If they re in a -- they're likely to attend a school that has historically been getting more than a year's worth of growth with 11th graders, that projection's probably going to be bumped up a little bit because they know this is a child who's going to go to a school that has high growth. If historically it's been a school that hasn't been seeing you know, growth with students, less than a year's worth of growth. Projection is probably going to be adjusted down slightly. And if they're a school who gets about a year's worth of growth, probably no adjustment. That's about, you know, where that child's projected to be. Does that make sense for everyone, okay? There are a few districts where there isn't, and I will be quite honest right, a feeder pattern. For example, I'm you know, from the western part of the state, Pittsburgh Public. Reason is is because there are so many magnet high schools. Those kids can really go anywhere within the district, okay. So that in that case, it's more of using an average district, average school kind of effect, okay. Because we're just not sure where students are going to go, okay. Any other questions about key part of that report? Really, it's that probability in the center there, probability to be reaching basic, proficient, or advanced. It's the most important information, okay. We always this question so I want to make sure that I addressed it. We always have 18

19 questions about how accurate is that projection, how reliable is it? It is very accurate, reliable, robust measure on a student, okay. Many reliability studies have been done, okay. You can find the most recent reliability study and all kinds of information about that projection model on the US DOE website. I know you can't read it here, but hopefully on that handout you can you know, that you can download. That reliable study was published with our growth model proposal in Because it is that projection that's used as part of that growth model to make AYP, okay. Four different review panels nationally have looked at those projections and those projection studies, the reliability studies. These are national experts in statistics and psychometrics, so you know, we've had some top-level people look at those and approve those. Bottom line though about what those reliability and research are saying is that that PVAAS projection even looking out three years into the future. So say you're looking out from eighth grade to 11th grade. It's much more reliable predictor of a student's future performance than if we just look at that last PSSA score, okay. Because we know that's one point in time on a child. There's going to be some error around it. Kids can have good days, they can have bad days. What it's doing is using an entire history on a child to say really this is how the child is likely to do, okay. Just real quick, just to remind you what those projection summaries are that I know Cindy's going to be really making that connection for you as well. You can look at all those individual student projection reports and look at okay and try to add up how many students do we have with the probability of reaching proficiency from 70% to 100%. Well to save you time so you don't have to sit there and tally them, they provide a projection summary report that literally tells you well how many kids do I have with the 70% probability or higher to reach proficiency. That's in the green piece of the pie. It's kind of like a stoplight, you know, that's 19

20 pretty likely. How many have between a 40% and 70% probability? Well that's in the yellow piece of the pie. How many have less than a 40%, that's in the pink piece of the pie, okay. You may get a few students who don't have a projection. That maybe is because they don't have enough data to get a projection. You need at least two years worth of data. You have to have at least two years of history for them to be able to project, okay. Sometimes you have kids that came in from a private school, they're home-schooled, came in from a different state. They just may not have that history. But at least you'll be able to know who those students are, okay. Because what you're able to do is not only see the proportion of students and their probabilities to proficiency, you could actually click on any number of those numbers, get who those students are. So you right away, you know immediately that list of students that maybe only have a 40% to 70% probability of reaching proficiency, okay. You can get that on districtlevel report, you can get it as a school-level report, okay. Whatever's more you know applicable for you. So how do we apply that? CINDY GOLDSWORTHY: Okay, can you hear me? All right, good. So let's talk now about how you would apply, as Jenn said, the projection reports both on individual students and the projection reports that are summary reports for a building or a district. Do you have any thoughts as you've been sitting listening to this? Anybody have any, let's do the individual student reports. How might you use those in an RTII process? Thoughts, ideas? Yes? AUDIENCE MEMBER: We're not using RTII in our middle school right now, but we are right now looking at placements for algebra for eighth grade. And if we simply use the teacher scores and their suggestions, we have many, many less children that we're looking at algebra. So we have started to look at the PVAAS projections to put more kids in. 20

21 CINDY GOLDSWORTHY: I'm not sure if everybody could hear her, but basically what she said is we're looking at placements for students in middle school into algebra. We don't have RTII yet, but we're looking at which kids should be in algebra in eighth grade. And if we're relying solely on teacher recommendations, what they're finding in their building is they are not getting as many students placed into algebra as they believe might be. And then looking at the PVAAS data has been helpful to say we've got perhaps a fair number of students who are projected to be proficient and/or advanced, because that's the other thing that you would want to look at. And then say, well should we be really encouraging those students to be in an algebra class in eighth grade? So there is a good use of PVAAS to use for course scheduling for kids, to make projections about where students should be. For example, one of the things that we've emphasized in my school district has been looking at the projections coming out of eighth grade and not only using that for universal screening for who's going to need tier support in ninth and tenth grade, but also to say in the counseling department, and you're a counselor. In the counseling department, what course selections are kids making? So if you're predicted to be advanced in 11th grade and you've decided as a young eighth grade student that you're really going to take the low road in ninth grade because you know, you're in this, that, and the other and you want to work and you've got a girlfriend and you know, 50,000 other things are going on. We might have a different conversation with you. There are some courses we think you ought to be taking in ninth grade, there are some courses we think you ought to be taking in tenth grade, so the projections really can play a role with all kids, not just the kids in tiers two and three particularly at the secondary level in course selection. So absolutely looking at how you're going to schedule kids is one way to use projection 21

22 scores. Who is at risk for universal screening? Who are the kids that we need to make sure that we've got on some kind of watch list? Now here's one of the dilemmas that we run into, particularly again at the secondary level. You are building that student's schedule prior to having the PVAAS scores, right. And I see some heads shaking, those of you that work in secondary schools can relate to this. One of the things that we often recommend and we've certainly done in our school district is that we have certainly identified those kids. Let's use eighth grade going into high school. We've certainly identified those kids based on lots of other assessment data, their past history of intervention, et cetera et cetera. Kids who need ninth grade tier two or Tier three intervention. But then crosscheck that list with the projection scores as soon as we have those available, and also use seventh to eighth grade projections to build that initial list. Because you want to have as much data as you can in front of you in making those decisions about those kids that are transitioning, for example from middle school to high school. Conversely at the fifth grade level, same issue. You could be looking at the projection scores in sixth grade for those students that you had in fifth and making sure you didn't miss anybody when those projections come out. Is this making sense to folks? So the whole universal screening process is a huge area in RTII where you can be using not only individual student reports, but you could be using the summary reports. Yes? AUDIENCE MEMBER: Cindy, that raises the question when do the PVAAS things get released? I've forgotten. CINDY GOLDSWORTHY: Last year I think it was October one, wasn't it? JENNIFER ROSS: Yeah, this year we're looking at probably the third week of September. We don't have absolute confirmation, but around the third week of September. CINDY GOLDSWORTHY: Okay? So universal screening. Big issue in RTII, right. Are you 22

23 using your PVAAS scores? Are you not only looking at individual student projections, but in terms of universal screening are you also using the summary projection reports to generate that list of kids who are in that range that you consider at risk. Here, for PVAAS, it's 70% to 100%, 40% to 70%, and below 40%. So those kids in the 40% to 70% range, those kids in the below 40% range, are we making sure that we have those kids at least on a radar screen. You'll see a lot -- and I'm sure all of you that are involved in this see this. You'll see some things that are sometimes puzzling. For example, you may see a student coming -- let's use eighth grade again. You may see a student coming out of eighth grade. Grades look pretty good. Student has maybe had intervention tier two support in seventh grade but didn't have it in eighth grade. But eighth grade grades still look pretty good. Student may but yet, and may have been proficient on the seventh grade PSSA. And you don't yet have the eighth grade PSSA score. You're not sure what to do with the student. You look at the projection, though, from seventh to eighth and/or in September of their ninth grade year, you're looking at the projection for 11th grade and it doesn't look so good. So the student has a 50% probability of being proficient. What we often have to do then is look at individual students to make some decisions about those kinds of students. For example, why did they do so well in eighth grade? What's our grading system like? Do we have a flawed grading system that may be because there was lots of extra credit. Maybe there were lots of bonus points, maybe everybody in that science class gets As. We can't really just rely on grades. That's why that extra piece of information can be helpful. Maybe this is a student whose parents worked diligently with that student every single night and that's how that student was able to look okay in eighth grade. So that's why those projections can be very 23

24 helpful, because it's another piece of information. Doesn't mean necessarily in our RTII process that we're going to bump that student into our high school tier two support. But it does mean that we want to put that student on some sort of monitor list because we know that that probability is really where things are headed unless we do something differently. So that may be for example in our high school classes, maybe having the teachers be aware that this is a student who we need to keep an eye on. And if you've got a building where you've got team meetings happening and student discussions happening, that's where those, that kind of monitoring can take place. So again, universal screening individual students. That's how the projections can fit. Entrance and exit criteria, another place where you can start to look at that. Now again, Karen brought this up earlier. You're going to want to be careful about using a projection for exit criteria because you have whole, hopefully in your RTII process, you have a whole lot of other data that you want to look at as well. But it's one piece of information to be talking about in a meeting. And I'm going to show you an example of that in a minute. Certainly looking at again the pie chart is a great way to be thinking about how you're going to allocate your resources. So now there were some central office folks here. Now you're sitting in a district and you're trying to figure out we only have x-number of intervention specialists here to help us with this. Where's our greatest need? How are we going to do this? Well taking a look at this, both from the district level and from the building level is going to really help us then at each grade level. It's really going to help us figure out where is our greatest need? Where do we have a lot of students who are in need of intervention and perhaps we're going to have to reallocate some of our resources to those particular grade levels. And likewise, that can be very helpful for the principal. So you've got a person in your 24

25 building and you've been perhaps giving more of your intervention time at the fourth and fifth grade level or the fifth grade level let's say. You look at your -- the kid's coming to you that are sitting in your building right now. Where do we need to put our time and efforts? So again, you can use that to determine how to allocate resources. Planning for -- yes Karen? AUDIENCE MEMBER: I was going to add something to that. I've seen school districts use that as an alignment tool as well. If they see consistently that they do not have a large percentage of their students on track to proficiency and they have need for intervention because they are somewhere in the middle, may or may not you know, be proficient or not likely to be proficient, then they know to focus their time and energy on aligning their core curriculum, getting better at that in addition to [inaudible]. CINDY GOLDSWORTHY: Yeah. And that's an excellent point, and could you hear Karen? Let me repeat that for the back and also for the taping. What Karen was talking about was that she has worked with districts who have used this as an alignment tool. So for example, not just saying where do we need to place our intervention supports? Do we have more intervention needs at third grade than fifth grade, et cetera. But for a tier one level, where do we need to beef up what we're doing for all kids because we know we've got a group of kids at this grade level who are projected to be at lower levels than we want them to be is I think basically what you were saying. Did I get that correct, Karen? Yep. AUDIENCE MEMBER: They get more, I think get more return on their investment if they complain of poor alignment. Do a better job on all students and then they can intervene more effectively for smaller [inaudible]. CINDY GOLDSWORTHY: Yeah. And so what Karen's basically saying is that you're getting a bigger return on your investment because you're looking at your tier one, and we're going to get to how do you use PVAAS for tier one. But you're looking at your tier one and you're 25

26 saying, wow we've got, we don't want to put all of our efforts in looking at what we need to do with tiers two and three because we've got a lot of kids that need that. Therefore we really should be looking at what do we need to sure up in aligning our curriculum, instruction, and assessment at tier one so that we have fewer kids who are in need of tiers two and three and therefore we can invest what we do have available to us in terms of resources at the tiers two and three level. Make sense, everybody? So again, another powerful way to take a look at that from as that third bullet says, from a systems approach. Now you're not looking at individual students and individual grade levels, but you're backing up and you're looking at the bigger picture. How are we doing with all our of groups of kids? What do we want to sure up at tier one so that we have the most efficient approach to dealing with kids who need supports in tiers two and three. And you had a question back there. AUDIENCE MEMBER: Is that what you would suggest for our higher performing students. Our greatest loss of growth is our advanced students. And those aren't kids for whom we would normally be seeking intervention. CINDY GOLDSWORTHY: Correct. AUDIENCE MEMBER: So would you suggest using it as Karen suggested as far as alignment as opposed to an intervention-type strategy because they're already in their high levels. CINDY GOLDSWORTHY: Yes, yes. And when we get to tier one, we'll talk more about that. But the question here from the back, just to make sure everybody heard that was in this particular school district what they're seeing, and this would be in the performance diagnostic reports. They're seeing not as much growth at their higher levels with their higher level students. Meaning their students predicted to be proficient and/or advanced. So they're not 26