Section 1 Data held at the School and the Development of a Pupil Tracking System

Transcription

1 Cwrt Sart Report Cwrt Sart is a small Comprehensive of 680 pupils. The school was originally a Secondary Modern and dates back to the 1920s. It became fully comprehensive in Attached to the school is a unit for Hearing Impaired pupils and a Speech and language unit. The school also has a significant number of pupils in mainstream who have learning difficulties. This means that the school has 57 statemented pupils. The area served is socially deprived with 31% of the pupils being in receipt of free school meals. The community served is tight knit and many parents attended the school when it was a secondary modern. The school has always enjoyed a good reputation as a caring school but, until recently, there has been no tradition of academic excellence. The provision of sporting opportunities and enhanced learning opportunities has, in the past, been a higher priority than academic success. Low expectations from parents, pupils and the staff contributed to this position. A new Headteacher was appointed in February 1996 with a clear school improvement agenda. This coincided with the involvement in the Highly Reliable Schools Project headed by Professor David Reynolds. The school improvement figures listed below are testimony to the change in culture that has taken place in the school. There has been no change in the nature of the catchment area or the intake during this period. In comparison with Welsh Office Benchmarking Data, Cwrt Sart School is above the upper quartile for their performance given the free meal rate. One of the key school improvement factors has been their use of data.

2 GCSE entry rate 90% 87% 90% 94% 93% 96% 91% 5 or more A* to C Grades 21% 25% 27% 42% 41% 44% 42% 5 or more A* to G Grades 71% 69% 67% 79% 75% 82% 75% 1 or more A* to G Grades 89% 88% 88% 92% 90% 93% 89% YELLIS 5 or more A* to C Grades 29% 36% 22% 37% 27% Progress has been made at Key Stage 3 but not as significant as that of pupils at Key Stage 4. Cwrt Sart s priority has been Key Stage 4. The school is actively working to a 50% 5 or more A* to C grade success level which, given prior attainment data and CATs etc., is a very challenging target. Section 1 Data held at the School and the Development of a Pupil Tracking System Cwrt Sart is a data rich school. In addition to the Key Stage 2 and 3 results they have access to CATs data, NFER and Suffolk Reading Tests and YELLIS predictors. All this data is collated and used to establish predictors for pupils. In addition, children are targeted specifically at Key Stage 4 using estimated grades. These pupils are monitored by individual mentors. It became apparent that the use of data at Key Stage 4 was a significant factor in the schools improvement. However it was also recognised that whilst the school had the data available at Key Stage 3, it was not systematic in its use of it. The result was the development of the pupil tracking system. Tracking Pupil Performance at Departmental Level Data on pupil performance is usually gained through assessments and examination results. Most teachers and Heads of department keep a record of these tests and store them in mark books or on a departmental spreadsheet. However there are inherent difficulties when using such a

3 system to track pupils performance at both departmental and school level. These include:- Difficulties in interrogating a large database Variation in the demands of the test Different mark ranges for different tests Variation in teacher marking To overcome such difficulties, a standardised system of pupil tracking has been developed. This has proved helpful in identifying possible pupil underperformance at an early stage. In addition it has raised the issue about continuity of assessment procedures and grade criteria both within departments and between departments. Cwrt Sart identified two essential determinants to consider when tracking pupil performance: Pupils efforts Pupils attainments Determination of Pupil Effort An effort grade may be calculated by determining pupil effort against an agreed set criteria. This can vary between departments depending upon their individual curricular requirements. Examples are: 1. time on task 2. keeping work up to date 3. completion of homework The following criteria were established in Heads of Department meetings. Each department has a combination of factors which are listed and the following grades awarded.

4 Effort Score Criteria 5 Always 4 Usually 3 Mostly 2 Infrequently 1 Rarely/never Determination of Pupil Attainment This is a far more complex issue. It is not possible to link the grading system to National Curriculum levels due to the differences in degree of difficulty per module of work and the different attainment levels available to different departments. It was agreed that each department would identify an assessment opportunity each term. The criteria for the assessment would be established and the results or material produced moderated within the department. Cwrt Sart are also moving towards a system of keeping examples of moderated data in a portfolio both as exemplars for performance management purposes and as an aid to standardisation within departments. The most straightforward model relates to a standardised test..after a test a standardised attainment score may be calculated by dividing the actual mark range into appropriate bands. e.g. for a test out of 80 with the highest score 74 and the lowest 34 Attainment Score Mark Range e.g. for a test out of 45 the highest score was 39 and the lowest mark was 16

5 Attainment Score Mark Range This information can be collated by each Head of Department or centrally. However this does not reflect the differentiation needed for different ability levels. It has been decided to limit available grades to certain ability groups. Set 1 will have access to 1 to 5 Set 2 will have access to 2 to 5 Set 3 will have access to 3 to 5 Recording the data A simple proforma is used. The pupils names etc. are extracted from assessment manager and prepared class lists sent out to staff. The results are then collated into an excel spreadsheet. Year Class Teacher Assessment description Date Pupil Effort Attainment Pupil Targets This information can then be used against the CATs and SATs data currently held by the school. An indication of potential could be identified by the school and pupils tracked against this level.

6 Average CATs score Expected attainment For every pupil the average attainment score can be calculated and compared with their standard CAT. Pupil with average score 4.4 and CAT score 107 is attaining within the expected range. Pupil with average score 4.4 and CAT score 91 is achieving significantly better than expected. Pupil with average score 3.4 and CAT score 123 is significantly underperforming. Interrogating the data Once the information is entered on a spreadsheet it can be interrogated to determine: the consistency of pupil performance the individual s performance relative to other pupils an individuals performance relative to their potential (CATs of SATs). In addition the data may be interrogated to determine: Relative class performance Gender performance differences Performance differences within specific areas of a subject e.g. within different attainment targets. One of the most effective uses of the system is to determine the most improved and the most underperforming pupils. Whilst this can be done at any time it is sensible to do this at fixed periods. The most improved can than

7 be commended whilst underperforming pupils can be targeted for interview/monitoring. The system has been easily adapted for whole school use. Standardised data from each department is entered into a whole school spreadsheet and then interrogated in similar ways. Determine pupil performance across all areas Relative performance in each subject area Pupils relative strengths and weaknesses Future targets for pupils Variation between departments How the data is used The data is used initially in year 7, alongside a wide range of other performance data, to establish sets for a range of subjects. Subsequently the data generated is used to evaluate the performance of pupils relative to their group and their individual potential. Letters of commendation have been sent together with letters expressing concern about pupils performance. In some instances parents have been invited to the school. Tutors have used the data to set short-term targets for pupils although the reliability of the data has led to some concern about whether tutors should have this type of access. An internal evaluation of the number and range of grades allocated by departments. Benefits to the school There is little doubt that the development of the system has enhanced the schools ability to move towards a self-critical organisation. This is in line with the new Inspection requirements outlined by ESTYN. Coupled with monitoring systems and peer observation, the information provided by the pupil tracking process is invaluable both as an aid to predicting performance but also as a tool for monitoring effectiveness. In addition the following benefits have also been identified:

8 1. The pupil tracking data can be used to replace monitoring reports thus reducing the bureaucratic load on teachers. 2. The school is now in a position to pass judgements on pupil progress and set requirements using reliable data as a basis. 3. Target setting and the evaluation of examination results is far more effective. 4. Pupils who are beginning to under perform are identified early and support is provided. 5. Parents welcome the letters of commendation and of concern. 6. Tutors have a far more direct involvement in target setting. 7. Formal school reports are far more meaningful. 8. Portfolios of work are being established that will present useful data for performance management. 9. Value added data is useful for staff during the threshold process. 10. Data is shared with partner primary schools thus encouraging the raising of expectations at Key Stage 2. Areas for concern There have been a number of issues that the school has had to resolve. These are as follows: 1. Clarification has been sought over the nature of the assessment opportunity. It has been agreed that this can be at the discretion of the department. It need not simply be a test but could also be a combination of events i.e. in languages an oral, listening piece of work and a written exercise. 2. Departments tried to link their grading to National Curriculum levels. This has proved totally impractical. French can only obtain a maximum level 4 because the children join us working towards level 1. Consequently the perception of pupils is that they have not made as much progress as in other subjects where higher levels are available. The link between tracking criteria and levels has been broken and this has proved very useful. 3. There was inconsistency regarding the number of level 5s awarded etc.. Some departments allocated a higher percentage of top grades

9 than others. This was a problem when pupils choose their options. This has now been resolved and is monitored carefully. Certain departments still remain conservative in their grade distribution. This is not a cause for concern now that the school is aware of these discrepancies. 4. Tutors took the initial data at face value. Consequently concerns were raised by some parents and pupils regarding the conservative allocation of grades by some departments. 5. The software has not been readily available to develop effective data processing systems. This has involved more staff time than initially anticipated. However the results justify the investment made. 6. There has been a significant improvement in Key Stage 2 results but the CATs and NFER data does not mirror the SATs results. If schools are to be measured, in value added terms, against SATs then most schools will have a difficulty. Analysis of the data shows significant discrepancies. However the pupil tracking system does generate powerful data that can be used in that debate. Areas for further development 1. Portfolios need to be developed further. 2. The criteria for assessment needs to be more fully documented. 3. Further work needs to be done to ensure greater consistency between departments. Cwrt Sart will need to narrow the gap. 4. Software will need to be adapted to make the process more user friendly and to be able to generate monitoring reports on a more regular basis. 5. Tutors will need further guidance on how to use the data more effectively as a motivational tool. 6. Greater links need to be made to the rewards system.

10 Section 2 Evaluation of the CATS data by component parts and review of curriculum areas in the light of findings Introduction The development of the schools expertise in the use of CATs data led Cwrt Sart to explore the potential of interrogating the results by section. It was anticipated that an analysis of the part marks of the verbal and quantative sections of the CATs would fulfil a diagnostic function. This information could then be used to compare the relevant schemes of work and amendments could be made. Action taken INSET was provided by Pat O Brien who explained fully the background to the interpretation of the CATs results. In particular he identified the potential of linking the Verbal to Non Verbal to identify two different types of pupil, those with literacy difficulties and those with organisation difficulties. He felt that there was a link between the quantitative scores and Mathematical potential but the diagnostic potential was restricted. Use of CATs with Literacy Intervention Cwrt Sart uses the verbal/nonverbal links to identify pupils needing support for literacy. Additional materials have been purchased and reading and literacy support is being provided by the English Department and support assistants. A number of packages have been bought for the network but these are of limited use. Greater emphasis has been placed on developing pupils literacy skills in subject areas. Reading intervention has been shown to have short term gains but proved to be of limited value in the long term. Of greater value has been the review of the English scheme of work at both Key Stage 3 and 4 and the significant change in focus to a skill based approach rather than a literature based approach to teaching English.

11 Cwrt Sart found that the reading tests that they used (and that their partner primary schools used) are more effective in identifying specific weaknesses in pupils literacy skills. This is now the model that they use rather than the evaluation of the CATs specific data in relation to meeting pupils specific literacy needs. The CATs data is used to identify pupils with potential difficulties and the reading tests then identify the specific need. The changes in the curriculum provision have resulted in an improvement in results at both Key Stage 3 and 4. Cwrt Sart still has much to do to improve their literacy support. Use of Quantative Results The evaluation of CATS quantitative part marks proved inconclusive. Cwrt Sart have researched alternatives and, in partnership with their feeder primary schools, now use NFER Mathematics tests. These give a topic by topic evaluation of individual and pupils skill levels. The data is currently used to evaluate gaps in pupil knowledge and to assess the effectiveness of pupils learning. Changes have been made to the scheme of work for Mathematics as a result of the information gathered. This is an area that Cwrt Sart will seek to develop further next academic year. Once again there has been a significant improvement in results at both Key Stages and this identification of strengths and weaknesses has played a significant role in this process. Use of Non-Verbal Scores Cwrt Sart School is able to identify pupils with low non-verbal and high verbal scores quite easily. They tend to need help with their organisational and conceptual skills. Cwrt Sart has used the Somerset Thinking skills packages to improve pupils performance but has failed to find an appropriate delivery mechanism given the curricular constraints that the school operates under. As an alternative Cwrt Sart has launched Thinking Science but this again has proved difficult to implement fully given the nature of the demands on the department. The school has just purchased a new Key Stage 3 Science scheme and the priority has been to establish this and not the development of

12 thinking Science. The school will be able to re-visit this area in about 18 months. Tutors are made aware of the pupils who have potential but also have low non-verbal scores. They are mentored to ensure that they bring their books to school, use their journals effectively and organise their homework effectively. Conclusion The evaluation of CATs has proved useful in relation to the identification of pupils needing support but not in terms of the identification of specific learning needs. CATs is a good predictor because it is skills based rather than diagnostic. Section 3 Evaluation of CATs Data and Dissemination of Results to Primary Schools This is an area that has been developed effectively. Partner primary schools have been given a copy of the summative data used to evaluate the performance of last years cohort. A full explanation of results and links to CATs etc. has been provided. As a result improved links have been developed between English and Mathematics. This is in addition to those already in existence at SMT level. This year the school identified significant improvement in Key Stage 2 SATs but not in pupils CATS and NFER reading scores. The data is contradictory. Because of the culture of sharing data that has been developed, this potentially sensitive issue can be raised in a spirit of openness and partnership. It is critical that the schools have an open dialogue since Key Stage 2 performance is being used as a value added measure for both Key Stage 3 and 4 performance. If the data being using is unreliable, subsequent evaluations are meaningless. As a result of the partnership that has developed Cwrt Sart is able to address these issues.

13 Joint INSET has been facilitated to develop the skills of interpretation of NFER results. This has led to the sharing of good practice in relation to developing strategies for improving pupils Numeracy skills. The transference of data between Key Stage 2 and Key Stage 3 is excellent. Cwrt Sart School has SATs levels, SATs part marks, NFER Mathematics scores and NFER group reading results in time to formulate their year 7 teaching groups. This is not the norm in the area. In addition the cluster has also standardised the reading schemes used. Cwrt Sart has met some of these costs. Conclusion The spirit of partnership and understanding has been greatly enhanced by the sharing of data. It is not perceived by either sector as threatening. It has also provided a firm footing for further curricular developments.