PROGRESS IN ENGLISH 7-11 DIGITAL Guidance and information for teachers Introduction The digital versions of Progress in English comprise the short form of the tests for 7 to 11 year olds. The short form exercises are: Exercise 1: Spelling Students are asked to spell the words missing in a passage. The words have been selected to exemplify spelling patterns that are particularly important for students at different ages. For Progress in English 7, 8 and 9, students use an on-screen alphabet to give their responses to the items in Exercise 1. They click on a letter and it appears in the answer box. An on-screen eraser can be clicked to remove a letter or letters and the student can then re-select letters until the word has been completed.
Exercise 2: Grammar The exercise assesses the ability to correct mistakes in a passage. The items cover a variety of grammatical rules, appropriate to the age of the students taking the test. For PIE 7, 8 and 9 this exercise is multiple-choice with students presented with four answer options. Exercise 3: Reading comprehension (narrative) In this exercise, students read a story and answer questions by selecting one of four multiple-choice options. This format makes it possible to assess literal and inferential comprehension, together with some understanding of authorial techniques, in an accessible way. Exercise 4: Reading comprehension (non-narrative) This exercise complements Exercise 3 by presenting an information text linked to the story. Once again, multiple-choice questions are used to assess students understanding of what they have read. Exercises 1 and 2 are presented together as the first part of the test and exercises 3 and 4 are presented together to complete the test. All exercises are untimed. The development of these versions of GL Assessment s major English assessment series was undertaken by the National Foundation for Educational Research to provide teachers and students with an alternative format to the paper tests with the added benefits of immediate scoring and reporting. The content of these digital tests is the same as their paper equivalents. However, methods of answering questions in exercises one and two are different (see above). An equating study was carried out during autumn of 2005 to ensure that the digital tests were equivalent to the paper tests. Use of Progress in English tests Progress in English is particularly suitable for use during the second half of the academic year. It can also be used as a start-of-year test, in which case it is recommended that students be given the test intended for the year below, as set out below: 2
Year group Autumn term Spring/summer term Year 2 in England & Wales Paper version of Progress in English 7 Year 3 in Northern Ireland Primary 3 in Scotland Progress in English 6 Year 3 in England & Wales Progress in English 7 Progress in English 8 Year 4 in Northern Ireland Primary 4 in Scotland Year 4 in England & Wales Progress in English 8 Progress in English 9 Year 5 in Northern Ireland Primary 5 in Scotland Year 5 in England & Wales Year 6 in Northern Ireland Progress in English 9 Progress in English 10 Primary 6 in Scotland Year 6 in England & Wales Year 7 in Northern Ireland Primary 7 in Scotland Administration Progress in English 10 Progress in English 11 When adding a student to a Progress in English register, all four exercises are allocated to the student. The four exercises may be taken over two test sessions. Once students have been added to a register, students can log in using the Register ID. Students should click on the student icon (see below) and they will then be asked to enter the Register ID. If the children are very young, the teacher may prefer to log on for each student. 3
Students will then see the following screen and they should select their name. The following screen will appear and students should click on Take Test next to the exercises they are due to take. 4
Please note that after an exercise has been completed it may remain on the list of assessments available to a student until results have been stored. The latter process can take some time at particularly busy test periods. A holding screen similar to the one below will appear. This enables students to start each pair of exercises at the same time. Each exercise starts with an explanation of what is required of students and some practice questions. The content of the latter has been kept very simple: the items are there to enable students to become familiar with ways of answering different questions. These include clicking in an answer box and typing with the keyboard, selecting an answer from a series of options, and selecting letters from an onscreen alphabet. All of the instructions are given via the headphones to reduce the amount of reading required. 5
Practice item for Progress in English 7, Exercise 1: Spelling The practice section has been constructed to enable students to go through these items at their own pace. At the end of the practice section, the following screen acts as a holding screen. It is possible, but not necessary, to wait until all students have reached this point so that they start the exercise simultaneously. Once students have completed the first pair of exercises and clicked on the End Test button, they will be returned to the following screen, at which point they can select the second pair of exercises or log off. 6
Students with Special Assessment Needs Having read the description of the content of the test, you should carefully consider whether the test is appropriate for students with severe reading difficulties, or for those whose English language fluency is very limited. These students will obviously be at a disadvantage in any test such as this, and compensation cannot be made by simply allowing extra time for the test or by reading out the test material. Therefore, the test should be used with discretion, if at all. If it is necessary to give students extra help, it is recommended that they be tested in a separate group so as to minimise disruption to the rest of the class. Unexpected incidents during a test session As with the paper test, should anything unexpected occur during the test session, the incident should be recorded and appended to the group report for this specific group of students. This will allow the incident to be taken into account when scores are being considered. If there is a failure in your computer system while students are taking an exercise it will not be possible to re-enter the test at the point at which the failure occurred. In this instance, students will need to re-take the complete exercise. If students complete an exercise and results are stored (ie they have clicked the end test button) and then the system fails, it will be possible to retrieve results, and therefore reports, from the GL Assessment back-up server. Should this happen, please contact GL Assessment Customer Support Team on 0845 602 1937 and you will be connected to a Testwise adviser. 7
Testwise Progress in English Report Only students who have completed both pairs of Progress in English exercises will be included in the analysis presented in the group report. Such students will be highlighted in Section A with a note of which exercises they have completed and their raw score. They will not be included in the csv report. A sample Progress in English group report may be viewed at: http://www.gl-assessment.co.uk/pie The report is in three sections: Section A This section summarises each student s attainment on the test as a whole, giving their age at the time of test, their raw score, standardised score, stanine, percentile rank and group rank (based on standardised score). The standardised score is particularly useful as it shows the student s attainment in relation to a nationally representative sample of students of the same age. The national average standardised score is 100, and two thirds of students will score between 85 and 115. The student s standard score is also shown as a vertical line with a horizontal line showing the 90% confidence band. It is recognised that any test score represents a performance on a particular day, and the score should therefore be placed within such a confidence band. If the test were taken again, nine times out of ten one would expect the score to fall within this range. The stanine score shows the standard nine score the student achieves in comparison with the national sample, with 9 being the highest score and 1 being the lowest. The national percentile shows the percentage of students in the national sample whose scores were lower than the student. Final columns of this section give scores against curriculum categories: spelling, grammar, reading comprehension (narrative) and reading comprehension (nonnarrative). 8
Section B The first table in this section shows the mean score for the class/group by gender against the national average. This allows the group s attainment to be evaluated as below, at, or above the national average. The information displayed on this table is illustrated by the graph beneath the table, which as well as displaying the group and national means, shows the 90% confidence bands. Descriptive comments are also given to explain whether the average and the range of scores for the class/group are significantly different from the national average. A second table shows the distribution of students across nine score bands (stanines) compared to the national distribution. This data is presented in a graph to show the distribution for the class/group separately for boys, girls, all students and the national sample. There can be many reasons why the attainment of students might be significantly higher or lower than the national average. Factors might relate to the students motivation, to levels of support at home, to the quality of their previous experience of education, etc. Whatever the reasons, the first step to improving students attainment is to know accurately where they are now. This report helps by giving an overview of the current performance of the group as a whole against national standards. The final table in Section B shows results analysed by Process Area. All the test questions in the short form of Progress in English series can be grouped into one or more of the following categories: Curriculum Content Category Writing Spelling Grammar Reading Reading comprehension narrative Reading comprehension nonnarrative Process Category Writing Spelling Punctuation Consistency Reading Simple inference Complex inference Retrieval Authorial techniques The bar charts show the percentage success rates for each of the categories for the class/group against the national average. In some cases, the profile for the 9
class may be above the national average, or indeed below the national average, in all process areas. In other cases the results may reveal strengths in one particular process area, but a relative weakness in another. The results presented are based on raw test scores that have not been adjusted for age. If the average age of the group is more than 3 months older/younger than the national standardisation sample, then their mean raw scores may be slightly higher/lower respectively than the national average because of this. The purpose of this part of the report is to focus on the relative strengths and weaknesses in different areas and on different questions, rather than a simple comparison to the national norms. Section C There is a wealth of information available from a detailed analysis of students performance on each of the questions within the tests. This data can also be compared with the difficulty of the questions as established during the national standardisation. Question by Question Graph This graph gives a quick overview of the success rates for each question for the class/group (bars) compared to the national average (thick dark line). The questions are sorted from left to right according to their difficulty, as indicated by the percentage of students answering the question correctly at the time of the national standardisation. For example, the question on the extreme left of the graph is the easiest question in the test (usually answered correctly by over 90% of the national sample), and the question on the extreme right is the most difficult question in the test. Question by Question Listing This data is also presented in the form of a table that includes a brief description of each question alongside the question number. The questions are listed in the order they are presented in the Question by Question Graph, that is, from left to right across the graph. 10
Question Listing by School-National Difference This table presents the Question by Question Listing in a slightly different order. The difference between the group/class and the national success rate is calculated and shown in the group national difference column. The table is sorted so that the questions where the class is most ahead of the national average are listed at the top, and the questions where the class is most behind the national average at the bottom. Some of the questions that can be answered using the Question by Question reports are: Which questions did the class find most difficult, and which relatively easy? Are there any common elements in the questions the class found most easy/difficult? For example, do they all relate to a particular aspect of the curriculum? Are there any questions where no student in the class answered correctly? What implications might this have for teaching? Have these areas of the curriculum been covered? Do such areas need to be reviewed? Individual Scores An individual s scores can be viewed by selecting CSV report in Testwise. This will produce an Excel spreadsheet with one or more students raw score and standardised score, item by item, with process categories for writing (spelling, punctuation and consistency) and reading (simple inference, complex inference, authorial techniques and retrieval). 11