Moving o n: Principals as Literacy Leaders in South Australia

Transcription

1 School of Education Fogarty Learning Centre Moving o n: Principals as Literacy Leaders in South Australia FINAL REPORT Assoc Prof Deslea Konza Dr Leanne Fried Dr Kevin McKennariey

2 ISBN This project was supported by funding from the South Australian Department of Education and Child Development The views expressed here are those of the authors and do not necessarily represent the views of the South Australian Department of Education and Child Development Edith Cowan University, Fogarty Learning Centre and the South Australian Department of Education and Child Development. First published on the Fogarty Learning Website in This work is copyright. You may download, display, print and reproduce this material in unaltered form only (retaining this notice) for your personal, non-commercial use and use within your organisation. All other rights are reserved. Requests and inquiries concerning reproduction and rights should be addressed to The Director, Fogarty Learning Centre, Edith Cowan University, 270 Joondalup Drive, Joondalup, WA 6027.

3 Principals as Literacy Leaders (PALL) in South Australian Department of Education and Child Development Schools FINAL REPORT Associate Professor Deslea Konza Edith Cowan University Dr Leanne Fried Edith Cowan University Dr Kevin McKennariey Assisting Schools Education Consultancy

4 Executive Summary The second extension of the Principals as Literacy Leaders (PALL) project was implemented in in 176 Department of Education and Child Development (DECD) schools in South Australia. Modules were developed and delivered by academics from Griffith University and Edith Cowan University, but also drew on material developed originally by the Australian Catholic University. The project was evaluated by examining a variety of data sources: evaluations of the five professional learning modules; principal discussions of their perceived growth in leadership capabilities and literacy knowledge; pre- and post-surveys of principals literacy knowledge; principal, teacher and Regional Leadership Coordinator Evaluation Questionnaires; and school Intervention Evaluation reports. Major findings include the following: Different data sources support the conclusion that both the leadership capabilities and literacy knowledge of principals who engaged in the PALL project developed strongly throughout its implementation. Their new knowledge and skills led to increased confidence in working with staff, governing bodies and parents. These new capabilities constitute an ongoing resource for the individuals themselves, and for the system. The Big Six was a useful and accessible model that enhanced principals learning about reading development and facilitated communication with staff and parents. It provided a focus for school improvement, and assisted schools in aligning data collection and teaching around the elements required to build effective reading skills. The literacy knowledge of teachers and to a lesser extent SSOs also developed, and was having an impact on their classroom practice. There was, not unexpectedly, some reduction in effect as information was transferred from leaders to classroom teachers, but the model whereby the school leader received the content and was responsible for passing the message on, increased leader credibility, and provided a cost-efficient way of building teacher knowledge. These teachers should feel empowered by their development, and through their greater capability and professional orientation, contribute to the sustainability of effective practice wherever they are appointed. Whole-school changes around literacy teaching occurred across a significant number of schools as a result of the PALL project. Many of these changes related to data collection and analysis procedures, and an increased focus on using data to inform planning and teaching. Adoption of the waves approach to literacy intervention, the implementation of literacy blocks, and a greater emphasis on explicit teaching were powerful influences on whole-school change. Consistent practices also provide the context for greater collaboration, and a stronger, more cohesive and more stable learning environment. Sharing leadership with individuals and collaborative groups increased throughout the project, resulting in greater collective responsibility for planning at the school level. In some cases this was very much informal rather than titled leadership as the particular expertise of individuals or groups was utilised. Both perception and achievement data support the view that student literacy outcomes were enhanced as a result of the PALL project. All but three of the 76 submitted evaluation reports referenced improvements in students reading achievement, although the data provided to support these statements varied widely. If genuine acceleration of literacy development has occurred, a range of positive short and long-term outcomes should follow, not only for the children themselves, but also for their families and the broader community. Consistent with evaluations of the PALL Pilot and the first DECD extension, this research found that the contribution of the Regional Literacy Coordinators (RLCs) to the project was critical to its success. 3

5 Feedback suggests that overall, those filling these positions possessed the right combination of professional experience, credibility and personal qualities that was required to maintain momentum over the two-year life of the project. The strong evidence base underpinning the PALL project strengthened the credibility of the leadership and literacy positions put forward in the first two modules, and facilitated acceptance and use of the frameworks that were used throughout. Active engagement with parents and the broader the community continues to present challenges for many schools. Whilst there were some positive outcomes reported, changes in this area were difficult to achieve. Little reference was made to work being undertaken with other schools or with community agencies or business groups. This appears to be the final frontier, and one that should be explored further with programs that include this as a more direct focus. Improving the literacy knowledge and leadership capacity of principals can have a positive impact on the knowledge and instructional practice of teachers, and therefore on student achievement. This model can be a cost-effective approach to change at a system level. The following recommendations emerged from the evaluation: Data collected for this evaluation were largely perception data, and need to be followed up by systematic collection of student achievement data at the system level over at least five years to determine whether differences in achievement exist between PALL and non-pall schools. This analysis needs to be cross-referenced with information from the RLCs and project managers about which schools engaged deeply in the process to fairly assess the impact of the PALL project. Some PALL schools stood out in the progress made by their students. A closer analysis of a sample of these schools to determine any commonalities could provide important information about the combination of factors that lead to the best outcomes for students. This could relate to some specific element of PALL, but could also depend on the ways in which leaders transfer knowledge to their staff, or to the existing experience and knowledge of staff that were simply tweaked by PALL input, or even to the use of particular resources. It may eventuate that PALL was most successful in interventions that targeted a particular stage of reading development, or students with more entrenched difficulties. Alternatively, the most successful schools could be those where PALL operated in combination with one or more other programs or initiatives. Much could be learnt from a closer analysis. If PALL continues to have some traction, particularly across Australia as a result of the Train the Trainer program and other iterations of the project, a conference held on a negotiated basis could present stories of successful practice that are grounded in the core business of principals, thus maintaining the momentum of the project, strengthening the network of leaders involved, and inducting new leaders. Alternately, conferences of relevant professional associations (for example, APPA) could have a PALL stream where success stories are shared. At a more local level, regular check-in sessions could support the sustainability of PALL practices. In individual or clusters of school communities, much of the learning from PALL could be packaged to help parents support their children s literacy development. More informed parents would lead to a better preparation and ongoing support for many children. Updated materials, tools and frameworks should be distributed through an established website such as that operated by the SA DECD or APPA. Materials and resources that have been developed and implemented successfully by schools could also be more widely distributed. 4

6 Table of Contents Executive Summary... 3 List of Tables... 6 List of Figures... 6 Introduction Major components of the PALL project in SA DECD schools in Professional learning modules The Regional Leadership Coordinators (RLCs) Frameworks and tools School contexts Concerns around student literacy achievement Concerns around the skills and expertise of some staff Lack of school-wide approach to literacy instruction Inefficient use of resources Research questions and methodology Aims Data collection procedures Summary of research questions and data sources Outcomes of the PALL project Enhanced capabilities in leading literacy learning Enhanced school literacy processes Enhanced student achievement in reading Effectiveness and challenges of project components Project components that enhanced principal capabilities Challenging components of the PALL project Conclusions and Implications Recommendations Final Comments References Appendices Background Information Teacher Questionnaire PALL Teacher Questionnaire Principal s Role in Leading Literacy Learning Background Information

7 List of Tables Table 1: Summary of research questions and data sources Table 2: Normed Data Comparison Time 1 and Time 2 (Category 3 Schools) Table 3: Principal views on project outcomes Table 4: Combined RLC responses to questions on project outcomes Table 5: Principal knowledge of literacy learning and teaching Table 6: Leading literacy data gathering and analysis Table 7: Teacher perceptions of principal s role in leading literacy learning Table 8: Teacher knowledge of literacy learning and teaching Table 9: Teacher responses to PALL project Table 10: Strategies used to build teacher knowledge and practice Table 11: Example of data provided by school to demonstrate achievement in early literacy Table 12: Sample of participants highlights in Modules 1 and Table 13: Principal evaluation of the Literacy Practices Guide Table 14: Principal evaluation of the RLC role Table 15: Reasons for changes in aspects of Parent and Community Support 52 List of Figures Figure 1: Group results of pre and post-project literacy knowledge Figure 2: Principal perceptions of their growth in literacy knowledge Figure 3: Teacher perceptions of their growth in literacy knowledge Figure 4: Principal perceptions of teacher growth in literacy knowledge Figure 5: Teacher perceptions of their growth in literacy knowledge Figure 6: Principal perceptions of teacher growth in evidence based practice Figure 7: Teacher perceptions of their growth in evidence based practice Figure 8: Principal perceptions of changes in student literacy achievement Figure 9: Teacher perceptions of changes in student literacy achievement

8 Introduction This document reports on the second extension of the South Australian Principals as Literacy Leaders (PALL) project, which was conducted from as a continuation of the national PALL Pilot project ( ) and the South Australian Department of Education and Child Development (SA DECD) PALL Project Extension 1 ( ). It was funded by SA DECD, managed by the Australian Primary Principals Association (APPA) and involved academics from Edith Cowan and Griffith Universities, but also drew on material developed by the Australian Catholic University. Griffith University was responsible for the evaluation of the first extension (see Dempster & McKennariey, 2012) and Edith Cowan University was responsible for the evaluation of the second extension, which is the focus of this report. In order for the report to be read as an independent document, brief explanations of its various components are included, although these are described in more detail in the previous reports. This is followed by a section on the school contexts, and provides a summary of the issues identified by participating principals as those that they wanted to target as part of their involvement in the PALL project. Part three explains the research procedures used in evaluating the project. The broad areas of focus and the precise research questions are outlined, and the data collection instruments are described. Part four presents the major outcomes, and Part five offers some analysis of the most effective components of the project, and some challenges that emerged for both presenters and participants. This is followed by conclusions and implications as Part six; and recommendations in Part seven. The main frameworks used in the project, and a sample of individual school evaluation reports, are included as appendices. 1. Major components of the PALL project in SA DECD schools in PALL is a two-year professional learning program aimed at building principals understanding of how children learn to read, and supporting leadership of literacy teaching in their schools. A total of 176 school principals began the project in March The precise number who completed the full two-year project is difficult to determine, but 117 principals completed the project evaluation questionnaire at the conclusion of Module 5 in May A total of 100 principals completed the Personal Leadership Profile at the beginning of the project, and in May of Seventy-six principals submitted Intervention Evaluation reports, although more implemented a literacy intervention. The Evaluation reports were due at a particularly busy time in the school year, which would have affected some principals capacity to complete this final task. 1.1 Professional learning modules Five one-day professional development modules were delivered, each of which included instruments or frameworks designed to support professional learning. Approximately 110 principals and leaders participated in Adelaide, and approximately 65 attended sessions in the regional town of Bordertown. Following each module, principals engaged in school-based tasks designed to consolidate understandings and facilitate the dissemination of knowledge throughout their schools as they collaboratively planned and implemented a targeted literacy intervention with their staff. In most schools, a specific area of reading development with a particular cohort of students was targeted; for example, development of letter-sound knowledge in the junior primary years; or improved comprehension in the upper primary years. Principals had access to all presentation materials, readings, frameworks, non-commercial assessments and tools developed for the project to support them in these endeavours. Module 1 presented key findings from research into leadership, with particular reference to how school leaders can contribute to improved student learning. This module introduced the Leading Literacy Learning Blueprint (Dempster, 2009, see Appendix A), which provided a framework for leading literacy improvement 7

9 in schools. Important concepts included the establishment of moral purpose and a shared mission, broad distribution of leadership, and the notion of disciplined dialogue to support the building of professional conversations with teachers about effective classroom teaching. Module 2 presented the research evidence regarding best practice in the teaching of reading. The Big Six (Konza, 2010, see Appendix B) of (1) oral language (2) phonological awareness (3) letter/sound knowledge (4) vocabulary development (5) fluency and (6) comprehension were presented as the core elements required for skilled reading to develop. The need for each of these elements to be explicitly taught was also highlighted. This input was designed to provide principals with the capacity to have informed professional conversations with their staff. Module Two also introduced participants to the Literacy Practices Guide (Konza, 2012), which provides checklists of what effective reading instruction looks like in practice. This instrument is described further in section 1.3. Module 3 examined the role of both qualitative and quantitative data in informing planning, and measuring intervention success. Specific assessments for each of the Big Six were discussed, including whole class screeners of some skills, and fine-grained individual assessments for individual students. Principals were provided with a framework for gathering data on student achievement and classroom practice, and strategies for engaging staff in analysis of data, and planning subsequent interventions. Module 4 incorporated the concept of intervention waves, with first wave teaching referring to effective whole class teaching as the most efficient way to ensure that most students succeed; second wave teaching for students who need additional instructional time and intensity to maintain age-appropriate progress; and third wave teaching for students who are significantly behind their peers, and who need a separate and more intensive program in order to make progress. The need for teachers to have a large repertoire of strategies to teach students according to their learning needs was highlighted. Suggested strategies for supporting students at the whole class, small group and individual level in the junior, middle and upper primary years were included. Module 5 provided principals with guidelines and frameworks to support evaluation of their school-based literacy interventions. Principals were asked to gather both quantitative and qualitative data on the impact of the interventions on teaching and learning activities, and on the effect of the intervention(s) on student achievement. Reminders were given about the range of data sources available to them, and of the need to consider how newly developed practices could be embedded in school structures and processes in order to maximise the potential for sustainability. 1.2 The Regional Leadership Coordinators (RLCs) After each of the modules, which were delivered between March 2011 and May 2012, principals were asked to carry out school-based activities designed to reinforce key concepts and connect the content with what they were experiencing in their school settings. Each principal was supported by a Regional Leadership Coordinator (RLC) who maintained contact through school visits and throughout the two-year project. The role of the RLC was to follow up questions after module delivery; support the principals in completing the between-session tasks; and provide support for the intervention planning, implementation and evaluation. An important aspect of the RLC role was that there was no line management involved. The load for different RLCs differed considerably in the DECD project, with some having a small number of principals to support while others had over 20. 8

10 1.3 Frameworks and tools In addition to the Leading Literacy Learning Blueprint and the Big Six framework, other tools and proformas were used throughout the PALL project. The Literacy Practices Guide (LPG), developed as part of the PALL pilot, was designed to support principals understanding of effective reading instruction and the collection of classroom-based data. The LPG provides a structured way of looking at five different dimensions of the teaching/learning environment as they relate to the teaching of reading: (1) the classroom environment, (2) student work, (3) planning documentation, (4) reading instruction, and (5) reading across the curriculum. Indicators within each dimension were drawn from the past several decades of research into the components of an effective reading program. Separate two-page guides are provided for the first two years of formal schooling, the junior primary school, and the senior primary years. The LPG was distributed as a word document so that principals could use it as a working document, and adjust it to meet their own needs. Principal feedback contributed to significant changes in its format, such as including a column for the teacher s own ratings of different elements of the classroom literacy environment, and broadening the principal s input to potentially include feedback from a peer or another school leader. As part of the third module, simple proformas were also developed to guide principals through the processes of analysing the data sets that were collected at their schools, assessing their usefulness, and determining if fewer, more or different assessments were required. An Evaluation Guide was also developed for Module Five to help principals align the purpose of their intervention, the key questions that needed to be asked to determine intervention effectiveness, the data they needed to answer those questions, and how that data could be collected. 2. School contexts The principals involved in the PALL project in came from schools ranging in size and across all socio-economic areas. In the Intervention Evaluation reports, principals were asked to nominate the major issue(s) needing attention in their schools that were relevant to the aims of PALL. Most principals nominated more than one area of concern. This section discusses those that were mentioned most frequently by the 76 principals who submitted final reports, and provides some contextual information relating to the schools involved. 2.1 Concerns around student literacy achievement A total of 78 comments referred to student achievement in some area of literacy as being a major source of concern, which means that some principals mentioned it in more than one context. Forty-nine comments (64.5%) included specific reference to NAPLAN results, with 15 referring to comprehension results; 3 to writing results; 5 to spelling; 9 to reading; 13 to overall literacy problems as revealed by NAPLAN; and 4 to flattening trends in NAPLAN results across the students year levels. The progress of children in the early years of formal schooling, particularly in oral language and phonological awareness, was specifically mentioned by eight principals; another two principals mentioned more widespread oral language problems; four referred to vocabulary problems; two made reference to a general lack of letter-sound knowledge; and one referred to reading problems in Years 3 and 4. Such a strong focus on literacy problems would be expected in a project designed to address leadership capabilities in this area. The emphasis on NAPLAN results perhaps reflects their acute awareness of the public nature of, and importance placed on these results by some parents, the media, and educational systems. 9

11 2.2 Concerns around the skills and expertise of some staff Less anticipated was the number of principals who identified issues relating to staff as requiring their close attention in this project. Seventeen leaders (22%) mentioned difficulties associated with teacher performance, and the need to increase the accountability of teachers when students did not make progress. A further 14 principal comments (18%) related to the lack of differentiated teaching, and the fact that the needs of many struggling students were not being met. There was reference to intervention that was haphazard and undocumented, without consistent data to inform the content of the intervention and its impact on students. The lack of teacher knowledge of explicit teaching and other strategies designed to support students with significant learning difficulties was mentioned by 11 leaders (14%), and the low expectations held by some teachers was mentioned by one leader. 2.3 Lack of school-wide approach to literacy instruction Thirty-one principals (41%) made reference to the lack of a school-wide approach to literacy instruction, with an additional six specific responses referring to the lack of collaboration and coordination in literacy teaching, even across the same year level. Relating to this were an additional 35 comments (46%) regarding either a lack of diagnostic data or inconsistent approaches to data collection, little use of data to inform teaching, and inconsistencies in intervention approaches. 2.4 Inefficient use of resources Four leaders referred to the ineffective use of SSOs; another two responses related to the need to evaluate and update their existing school literacy resources; and one referred to their school s lack of easy to administer, whole-class assessments. 3. Research questions and methodology A mixed-methods approach to data gathering was considered the most useful for the participants in the project, thus there was a need for data to be drawn from multiple perspectives on the key research questions. Mixed-methods research combines both quantitative and qualitative methods in single study design (Gall, Gall, & Borg, 2007) and is well supported in the literature (Brannen, 1992; Flick et al., 2007; Punch, 2005; Thomas, 2003). This approach was considered to best match the project s purposes because, as Wiersma and Jurs (2005, p. 277) argue, mixed methods provide a more complete understanding of the phenomenon being investigated than individual methods. This section aligns the specific research questions with the broad areas of focus within the PALL project, and outlines the tools used to gather the relevant data. 3.1 Aims To enhance principal leadership capabilities and literacy knowledge The primary aim of the project was to develop the capabilities of principals to lead literacy learning and teaching in their schools. In order to investigate this, the following research questions were framed: To what extent were the leadership capabilities of principals enhanced through the PALL project? To what extent was the literacy knowledge of principals enhanced through the PALL project? 10

12 To improve school-wide literacy processes As inconsistent literacy practices and data collection processes across the school were identified as areas of concern by over 80% of principals, any changes in school-wide processes were worthy of investigation. The following research question supported this process: To what extent were evidence-based, school-wide literacy processes implemented as a result of principals involvement in the PALL project? To enhance the literacy knowledge and teaching practice of teachers One responsibility that principals had as part of the PALL project was to pass on information about reading development and effective teaching practices to their staff. The following question sought to explore how effectively this was done. To what extent was teachers knowledge of the reading process, and the effective teaching of reading, enhanced as a result of the principals involvement in the PALL project? What strategies did principals use to build the literacy knowledge and practice of their teachers and how effective did they believe them to be? To improve student literacy achievement The ultimate goal of the PALL project was to improve the reading outcomes of students, which led to the following question: To what extent did student reading outcomes improve throughout the duration of the PALL project? 3.2 Data collection procedures Data were collected from the following sources: School Profiles School Profiles, completed by principals before and after the project, included broad information on the schools missions and values, but also specific information on literacy teaching and learning priorities, and levels of student and staff satisfaction. The profiles contained information such as the demography of the schools, their missions and values, literacy teaching and learning priorities, staff numbers and experience, and student and staff satisfaction data. These data enabled comparisons to be made about changes in the profiles influenced by the project, together with reasons for those changes. The changes were discussed by principals in partnership with their RLCs. The Personal Leadership Profile (Appendices C1, C2, C3) The Personal Leadership Profile (PLP) was originally developed for the PALL Pilot and was completed by principals in the second extension to assess perceptions of their own leadership capabilities. The PLP had 36 statements on it, randomly placed, which reflected the eight dimensions of the Leadership for Literacy Learning Blueprint (LLLB), and which have been linked in the literature to student learning: (1) moral purpose (2) disciplined dialogue (3) a sound evidence base (4) active involvement of school leaders in professional development with their teachers (5) enhancing the conditions for learning (6) planning and coordinating the curriculum across the school (7) shared leadership and (8) connecting with parents and the community. Principals were asked to grade their level of knowledge and competence in each of the statements. There were four levels: limited, sound, very good and excellent. 11

13 Appendix C1 provides an example of the PLP instrument, which was initially completed by principals in March After data had been entered, a personal PLP report was returned to each principal for discussion with their RLC (see Appendix C2). Principals were asked to complete the same document in May 2012 (see Appendix C3). During the intervening period, principals participated in the five modules, planned their own school literacy intervention and were almost six months into the implementation phase. Regional Literacy Coordinators had also worked with principals during this time. One hundred principals completed the PLP on both occasions, and received a second PALL report that reflected their own assessment of changes in their leadership over that 14 month period. A website was created and principals were asked, after reflecting on and discussing the scores with their RLC, to comment briefly on any changes in their profiles. Comments were originally received from 25 of approximately 150 principals who were still actively participating in the project. The busyness of principals, concerns about the validity of self-assessment testing, the fact that personal data were being sought from them or even the relevance of the task for a number of them may explain the failure of some principals to complete this task. At a final meeting of a group of the participants in late November to celebrate the success of individual interventions a further effort was made to collect comments from as many additional principals as possible. As a result, a further 24 responses were received, making 49 responses in total. This source of data provided useful information regarding whether principals believed that they had progressed in their leadership capacities through their involvement in the professional learning modules, their mentoring links with the RLCs and their engagement with teaching staff and others throughout the literacy intervention. A content analysis of principal comments on changes in their PLPs was conducted. Within each dimension, key ideas were developed based on responses by principals. In order to make reporting more manageable, the schools were classified into three groups. These were based on the seven DECD indices where schools with an index of 1 were the most disadvantaged, and those with an index of 7 were most advantaged. Those who were grouped into Indices 1 and 2 were categorised as Group 1, those who were in Indices 3 and 4 were categorised as Group 2 whilst those who were classified as 5, 6 and 7 were categorised as Group 3. The Literacy Knowledge and Beliefs Survey (Appendix D) Originally a very comprehensive literacy knowledge survey (Joshi et al, 2009) was to be included in the data collection. A brief trial with a number of principals in Western Australia led to the decision to construct a much simpler survey, as the original was seen to be too time-consuming, in addition to being intimidating and confronting. While the information gained from the simpler survey was much more limited, it did provide some evidence about changes in principals understanding of the key components required for skilled reading to develop, effective teaching practices at different stages of learning to read, and the level of confidence they had in their understanding of the reading process. The survey contained 22 statements, responses to which were possible along a five-point scale: strongly disagree, disagree, not sure, agree and strongly agree. The most correct answers scored five, and the least correct scored one point, with not sure scoring three. Some questions were phrased so that strongly disagree was the most correct response. The principals were asked to complete the survey in March 2011, before delivery of Module 2, which focused on literacy knowledge, and in May of 2012 during delivery of Module 5. The surveys were completed anonymously, but principals were asked to record the school code (unknown to the researchers) on pre-and post-surveys so responses could be matched. A total of 82 post surveys were collected, two of which had no identification and one of which could not be matched with a pre-survey. Thus 79 matched surveys were available for analysis. 12

14 Module Feedback Sheets (Appendix E) At the conclusion of each module, participants briefly evaluated the usefulness of each session along a fivepoint scale, from not helpful to extremely worthwhile, and were given the opportunity to comment on any highlights and ways in which the session could be improved. These responses provided additional insight into the components of the program that principals found useful. All evaluations were reviewed to inform the format and content of future modules. Changes made in response to suggestions included additional opportunities for principals to discuss their schools responses to various aspects of the project, provision of materials that exemplified recommended practices, and the modelling of certain recommended strategies. Principal Evaluation Questionnaire (Appendix F) The Principal Evaluation Questionnaire was distributed during delivery of Module 5 in May 2012 to maximise the return rate. Responses were received from 117 principals. The questionnaire sought information about the focus of their literacy intervention, explored the strategies they used to build teacher knowledge and practice and how effective they believed the strategies were, and sought their overall response to the project. Principals were able to respond anonymously. Principal Evaluation Questionnaire - Final (Appendix G) Eight RLCs and 36 principals presented a showcase of their results to regional personnel in November The opportunity was taken on this occasion to gain some final reflections from this subset of the larger group, who would have represented those who were most engaged in the project. A total of 26 final questionnaires were returned from these principals, which explored their views on the role of the RLC and other elements of the project as it concluded. Teacher Evaluation Questionnaire (Appendix H) At the conclusion of the project, principals were asked to pass on an ed copy of the Teacher Evaluation Questionnaire to one or two teachers who had been involved in the school s implementation of the PALL literacy intervention. This questionnaire explored the extent to which the teachers believed the school s involvement had built their knowledge of literacy teaching and learning; how they viewed the principal s role in leading literacy teaching and learning in their school; and their overall response to the project. Responses were received from 70 teachers. RLC Evaluation Questionnaire (Appendix I) An Evaluation questionnaire was also ed to the Regional Leadership Coordinators. It investigated their views on the usefulness of different aspects of their role in supporting the aims of the PALL project; the schools use of the Literacy Practices Guide; and on the overall project outcomes. Five responses (50%) were received. RLC Focus Group After the showcase event in November of 2012, an opportunity was taken to meet with the RLCs for some final comments. Seven of the RLCs were able to join a focus group discussion. They were asked to reflect on the most significant aspects of the project, and ways in which they believed it could be improved. School Intervention Evaluation Reports (see Appendices J-P for examples) Intervention evaluation reports were received from 76 schools. These reports varied in length and detail, but generally provided information regarding the school context, the main issues of concern at the school, the intervention plan or plans they put into place, and some evaluation of the results. Some principals were unable to report specific outcomes by the time the reports were due because they felt not enough time had passed to do so. 13

15 3.3 Summary of research questions and data sources The focus areas of research, research questions and sources of data used to answer the questions have been summarised in Table 1. Table 1: Summary of research questions and data sources Research Focus and Questions 1. To enhance principal capabilities in leading literacy learning To what extent were principals leadership capabilities enhanced through the PALL project? To what extent was principal literacy knowledge enhanced through the PALL project? 2. To improve school literacy processes To what extent were school-wide literacy processes implemented as a result of principals involvement in the PALL project? 3. To enhance the literacy knowledge and skills of teachers To what extent was teachers knowledge of the reading process, and the effective teaching of reading, enhanced as a result of the principals involvement in the PALL project? What strategies did principals use to build the literacy knowledge and practice of their teachers and how effective did they believe them to be? Sources of data collection Personal Leadership Profile Principal Evaluation Questionnaire RLC Evaluation Questionnaire Personal Leadership Profile Principal Evaluation Questionnaire Survey of Literacy Knowledge and Beliefs Module Feedback sheets Personal Leadership Profile School Intervention Evaluation Reports Principal Evaluation Questionnaire Teacher Evaluation Questionnaire School Intervention Evaluation Reports Principal Evaluation Questionnaire 4. To improve student literacy outcomes To what extent was student reading achievement improved throughout the duration of the PALL project? Principal Evaluation Questionnaire Teacher Evaluation Questionnaire School Intervention Evaluation Reports 4. Outcomes of the PALL project This section of the report responds to the questions in each of the research focus areas. It draws on data from module evaluations, the PLP discussions, project evaluation questionnaires completed by principals, teachers and mentors, and the school evaluation reports which were submitted in October, In some cases this was only around eight months after an intervention had been implemented, and two schools reported that they did not have any hard data to report. Other schools had started making some changes quite early in 2011, and were able to report in a variety of ways. To support statements made in each section, extracts from different sources of data are provided. 14

16 4.1 Enhanced capabilities in leading literacy learning To what extent were principals leadership capabilities enhanced through the PALL project? From a comparison of the pre and post group results of the Personal Leadership Profile the normed data it is clear that the principals believed that their leadership capabilities had developed, with each Time 2 result being higher than Time 1 across the dimensions. The normed data for Category 3 schools for the two occasions on which the profiles were administered have been tabled below. The only increased standard deviation (SD) was in the area of shared leadership, which went from 1.0 to 1.2. The remainder of the SDs were reduced, which means that there were few outliers, or extreme responses. Principals therefore were largely consistent in their perceptions of personal growth in the leadership area. Similar results were found for both Category 1 and Category 2 schools. Table 2: Normed Data Comparison Time 1 and Time 2 (Category 3 Schools) Scale 1=low; 10=high Time 1 Normed Data Time 2 Normed Data Strong Shared Strong Conditions Curriculum Parent and Shared Prof. Devt. Evidence/ Moral Evidence for & Teach. Comm. Leadership (M/SD) Moral Purpose Mean/SD Purpose (M/SD) Base (M/SD) Learning (M/SD) (M/SD) Support (M/SD) (M/SD) 6.3 (1.7) 6.6 (1.6) 6.0 (2.0) 7.0 (1.3) 6.2 (1.6) 5.5 (1.9) 7.0 (1.0) 6.3 (1.5) 7.4 (1.1) 7.5 (1.3) 7.3 (1.2) 7.9 (1.1) 7.5 (1.1) 6.6 (1.4) 7.9 (1.2) 7.4 (1.3) This belief was consistent with responses to statements in the Principal Evaluation Questionnaire regarding the overall project outcomes as presented in Table 3. The mean score for the principals perception of their growth in leadership was 3.6 out of a possible Of the 112 principals who responded to this question, 107 believed their capacity to lead literacy learning had improved to a moderate or great extent. No responses reflected the belief that no growth had occurred, and 11 principals believed their leadership had improved to a slight extent. This reflects a high level of satisfaction on the part of the target audience. Table 3: Principal views on project outcomes As a result of the school s participation in the PALL project, I perceive improvement in: 1. My personal literacy knowledge. 1 Not at all No. of Respondents 2 To a slight extent 3 To a moderate extent 4 To a great extent Mean My professional capacity in leadership for literacy learning Our school s capacity to address students difficulties in literacy learning Our students literacy achievement

17 Additional support for their views was provided by the RLCs, five of whom completed the RLC Evaluation Questionnaire. All rated the item relating to the development of principals professional capacity in leadership for learning at 4 to a great extent - the maximum rating possible. Other items relating to leadership of literacy learning, such as reviewing literacy assessment practices, promoting changes to interventions for struggling students, engaging in focussed discussions about students reading development and providing additional resources to support literacy teaching also scored highly, with mean responses of between 3.3 and 3.6, or between a moderate and great extent (see Table 4). The evidence from these sources suggests some success in achieving a core aim of the project. Table 4: Combined RLC responses to questions on project outcomes Number of Respondents As a result of participation in the PALL project, I believe principals have: Learnt more about how children learn to read. Promoted the Big Six model to provide a framework for the teaching of literacy. Reviewed literacy assessment practices. Promoted changes to the nature of reading interventions for struggling students. Promoted more explicit teaching of reading strategies at classroom and individual levels. Encouraged conversations with parents about student literacy development. Engaged in focussed discussions (disciplined dialogue) with staff about students reading development. Provided additional resources to support literacy teaching in classrooms. Shared leadership of the literacy intervention. Developed their professional capacity in leadership for learning. 1. Not at all 2. To a slight extent 3. To a moderate extent 4. To a great extent Mean Further evidence to support the development of leadership capability comes from analysis of changes in whole school practices and processes, in teacher knowledge and practice, and in enhanced student achievement, which are discussed in sections 4.2, 4.3 and 4.4. To what extent was principals literacy knowledge enhanced through the PALL project? In addition to growth in leadership capability, the principals also believed that their literacy knowledge had grown considerably as a result of their involvement in PALL, with a combined mean of 3.6 (from a possible 4) for the question in their Evaluation Questionnaire relating to this area (see Table 3 in section 4.1). Of the 112 principals who responded to this question, 101 believed their literacy knowledge had improved to a moderate or great extent. This mirrors almost precisely the responses of the subset of 26 principals who completed an additional questionnaire at the project s conclusion, which explored this question in a little more depth (see Table 5). 16

18 Table 5: Principal knowledge of literacy learning and teaching As a result of participating in the PALL project I have: 1 Not at all No. of Respondents 2. To a slight extent 3. To a moderate extent 4. To a great extent Mean Learnt more about how children learn to read. Understood more about the relationship between decoding and comprehension. Understood more about the importance of automaticity and fluency to reading comprehension. Understood more about the importance of oral language and vocabulary to the development of reading. COMBINED MEAN Discussion of changes in the pre and post Personal Leadership Profiles, and analysis of the comments posted on the website, also reflected this belief. Principals made at least 62 specific references across the eight PLP dimensions to development of their literacy knowledge: to such things as a deeper understanding and knowledge about reading and the acquisition of skills ; growth and awareness of the links and importance in literacy sections through the Big Six ; and as being much more aware of the gaps in student reading that I did not recognise prior to the PALL training. One principal stated: Learning from PALL caused a change in my thinking [with] aha moments regarding the underpinning skills to high level comprehension. Developing their own understanding of the reading process also helped some principals reflect on the pedagogy of their teachers. One commented that he became aware for the first time of practices that were not consistent with the evidence or analysed deeply by teachers. For another, PALL was the catalyst for my knowledge of teacher growth. An important outcome of this development in literacy knowledge was an increased confidence in having professional discussions about reading development with staff, with governance boards, and with parents: principals were sharing learning with staff ; taking a lead role in professional development for staff as an outcome of attempts to improve student literacy ; and now leading by example. One principal stated that his new knowledge allowed me to more confidently approach staff members to discuss and challenge their thinking and practices. The following comments are further evidence of their increased confidence: PALL, and in particular the Big Six, has given me the knowledge and background information that allows me to make recommendations to staff in regards to content and pedagogy that are making a real difference to student learning. [I have a] better understanding of reading elements (Big Six) and am able to speak more effectively to teachers about aspects of teaching and learning. PALL has made me more confident to lead literacy with my staff, talk to teachers about how and what they teach, and has made me much more confident in my own knowledge about how children learn to read. 17

19 [I have taken] opportunities to speak to Governing Council and parent groups about the Big Six and reading programs. The main change has been to work with Governing Council and outline the PALL and the Big Six and teach members about our project. A secondary-trained principal made the following comment: Without PALL I would not have known the questions or conversations that needed to be had with primary teachers, nor would I have felt remotely comfortable taking a year 1/2 class for a literacy lesson each week. (And most certainly would not have timetabled myself into taking each of the primary classes for a lesson in 2013.) Results from the RLC Evaluation Questionnaire, which asked them to rate the development of the principals on various aspects of literacy, also support the principals views. The five RLCs who completed the survey scored the principals at the maximum rating for the item learnt more about how children learn to read, revealing that they believed this had occurred to a great extent (see Table 4 in section 4.1). Perceptions that principals literacy knowledge had increased were also confirmed by the pre and post results of the Survey of Literacy Knowledge and Beliefs. A total of 79 matched pre and post surveys were available for analysis. Most questions in this survey were clustered under established literacy component headings, with combinations of questions addressing changes in beliefs about reading development and confidence in understanding the theory behind reading development and capacity to teach reading effectively. Cautious interpretation of these results is necessary, as some categories contained only one question, and most categories contained only a few. The overall trend however, as displayed in Figure 1, is of development in most areas. Statistically significant growth in knowledge was evident in five of the seven categories. Further discussion of the results follows Figure Phonological Awareness Q5 Letter Sound Knowledge Q2, Q7, Q9, Q12, Q13, Q15 Vocabulary Q1 Fluency and Automaticity Q4, Q6, Q9, Q11, Q13 Assessment Q3, Q17, Q20 Evidence Based Beliefs Q1, 2, 4, 6, 7, 8, 9, 10, 11, 12, 13, 15, 20 Confidence Q21, Q22 Pre Intervention Average Post Intervention Average Figure 1: Group results of pre and post project literacy knowledge and beliefs survey 18

20 Phonological awareness With group average means of 1.8 for both pre and post intervention responses, it was evident that there was no overall growth in principals understanding of the term phonological awareness. The single question in this area targeted a common misunderstanding about the difference between phonological awareness and phonics. Only five leaders scored the maximum score of five in the post survey, one of whom had also scored five in the pre survey. One leader went from a score of one to five, and three others went from two to five, thus growth occurred in only a tiny percentage of the participants who completed both pre and post surveys. Confusion about these terms is common amongst teachers at all levels (Joshi et al., 2009; Louden & Rohl, 2006; Lyon & Weiser, 2009; Sallinger et al., 2010; Walsh, Glaser & Wilcox, 2006), and it appears that the project did little to address this. Many schools targeted this area in their junior primary years, and one would expect some growth in knowledge. This reveals the difficulty of developing a clear understanding of this material. While it could be argued that principals generally do not require deep knowledge of every concept, an understanding of such foundational information would better equip principals to ensure appropriate teaching is occurring in their schools. A great deal more could be done in developing the knowledge of principals in this area. Letter-sound knowledge Responses to items 2, 7, 9, 12, 13 and 15 were averaged to obtain a score that represented principals understanding of the role of letter-sound knowledge in reading before and after involvement in the project. Because the pre and post responses of each principal were matched, a paired sample t test could be administered. A statistically significant difference (t = 5.42; p <.0001) was found between the pre test scores (M=3.4; SD=.48) and post test scores (M=3.8; SD=.50), suggesting that understanding did increase, and that this was unlikely to have happened by chance. The effect size was then calculated. Effect sizes go beyond the question of did it work to how well did it work in a range of contexts (Coe, 2002). The effect size (Cohen s d) was.81, which is considered large, and further supports the view that principals understanding of the importance of teaching letter-sound knowledge developed throughout the program. This effect size also suggests that the development in knowledge was broadly based; that is, it was not the result of a relatively small number of principals increasing their understanding a great deal, while others did not. Vocabulary Pre and post responses to item 1 from the survey were analysed to assess changes in principal understanding of the role of vocabulary in reading acquisition. Using a paired sample t test, it was found that there was a statistically significant difference (t = 2.66; p =.009) between the scores principals obtained for the pre test (M=4; SD=.93) and post test (M=4.4; SD=.86). The effect size (Cohen s d) was calculated at.44, which is moderate, and suggests slightly less growth than was found in letter-sound knowledge. Fluency and automaticity Responses to items 4, 6, 9, 11 and 13 were averaged to obtain a score that represented changes in principals understanding of the role of fluency and automaticity in the reading process. Using a paired sample t test, it was found that there was a statistically significant difference (t = 6.32; p <.0001) between the scores principals obtained for the pre test (M=3.4; SD=.40) and post test (M=3.7; SD=.32). The effect size (Cohen s d) of 0.83 suggests strong growth across the group in understanding how fluency and automaticity affect the development of meaningful reading. The consistency of growth is also reflected in the relatively low standard deviations, meaning there were few outliers or widely varying responses. 19

21 Assessment Responses to items 3, 17 and 20 were averaged to determine any changes in principals understanding of assessment. Using a paired sample t test, it was found that there was no significant difference (t =.97; p =.33) between the scores principals obtained for the pre test (M=3.5; SD=.48) and post test (M=3.6; SD=.48). Growth in this area was slight, although two of the three questions tapped into an understanding of assessment in general, and its role in informing teaching. Principals would be expected to have a reasonably secure grasp of this before their involvement in the project. These questions would need to be modified in future studies to explore specific knowledge of literacy assessment. Evidence-based beliefs Responses to many of the items reflected the principals general beliefs about how reading develops and how it should best be taught, and whether or not these beliefs were consistent with the research into reading acquisition. Combined pre and post responses to items 1, 2, 4, 6, 7, 8, 9, 10, 11, 12, 13, 15 and 20 were analysed to assess any movement in the group towards a more evidence-based set of beliefs. Using a paired sample t test, it was found that there was a statistically significant difference (t=6.29; p<.0001) between the scores principals obtained for the pre test (M=3.2; SD=.32) and post test (M=3.4; SD=.28). This supports the view that throughout the duration of the project, the principals beliefs about how reading develops, and how it should be taught, became more consistent with the research evidence. The small standard deviations reflect a general consistency in this development; that is, there were no widely varying responses. Cohen s d in this case was.66, a moderately strong effect, which also supports the conclusion that principals moved noticeably towards beliefs that were more consistent with the research evidence. This is an encouraging outcome of the project, as changing beliefs is not usually easy. Confidence Pre and post responses to items 21 and 22 were analysed to assess changes in principals confidence in their understanding of reading development and ability to teach reading. Using a paired sample t test, it was found that there was a statistically significant difference (t = 3.58 p <.0001) between the scores principals obtained for the pre test (M=3.2; SD=.28) and post test (M=3.4; SD=.24). The effect size of.66 reflects moderate growth in confidence, which is consistent with the results of other data sources. Concluding comment These results reveal that moderate or strong growth occurred in five of the seven categories over the duration of the project. While many other projects and initiatives would have been taking place in the schools concerned, the fact that this growth occurred in 79 different principals, all of whom were participating in PALL, suggests that the project was responsible to some extent for this development. 4.2 Enhanced school literacy processes What changes occurred at the school level as a result of principals involvement in the PALL project? References to changes at the school level across a range of structures and processes permeated the PLP responses, the Intervention Evaluation reports, and the Project Evaluation Questionnaires. These developments are discussed in general terms before specific changes relating to data collection and use, the implementation of evidence-based literacy practices, professional development, shared leadership and the use of resources are discussed in more detail. 20

22 A whole school approach to literacy instruction In discussions of the PLPs, there was an overwhelming number of references to the establishment of schoolwide approaches to teaching reading, and of what could be referred to as a collective vision around literacy teaching as a result of involvement in the PALL project: Shared vision directs work across school and classroom learning programs and teacher practice. I have been chipping away and moving staff towards a common goal/purpose in the delivery of curriculum. Developing a whole school approach identified as a priority. Actively involved in whole school approaches, programs and agreements. We have developed a structured intervention program across the school. This is because of PALL. As a result of the project, many principals were actively overseeing the school s curriculum development and were coordinating and managing the teaching and learning program. For many, this was a major development in their thinking and skills. In the words of one principal, an agreed literacy focus across the school each term and common agreements within year levels were developed which led to greater consistency of teaching methods and learning for students. Another indicated that she had developed a stronger focus on the principal leading a whole site approach with teachers ; and another, that the school had reviewed school curriculum areas and now connect the year levels and their themes. Principals wrote of being all on the same page, of newly established common understandings around objectives, common agreements and goals, and of all staff involved in clear understandings and setting goals. One principal made the following statement about her school s site agreement: There has been a significant change in teachers thinking as a result of whole school agreement about our intervention model. Teachers are clear about their responsibility for all students in their class and that they will not be fixed elsewhere. Another stated: I led staff through a site audit of current methodologies and resources used to teach comprehension. This revealed that teaching practices were not aligned and there were few common assessment tools to track learner progress. Involving staff in this process enabled them to see the rationale for change and empowered them through shared decision making. Other relevant comments from the PLP discussions included: We have developed literacy blocks, which have the same structure in each year level across the school. We have now developed whole school agreements about literacy teaching. This is because of my new learning from PALL. Confirming evidence of these changes emerged from the Intervention Evaluation reports. Thirty-five principals reported the development of a school-wide approach to literacy instruction, with many references to the use of agreed benchmarks, common language and agreed terminology. One commented that it is exciting to see staff using the same language and working together ; and another explained that the school 21

23 had worked hard at using the same language of the Big Six. Another principal maintained that PALL language had supported and enhanced her site s strategic goals, giving structure and purpose to data. Additional comments included: All staff use a common language approach and methods to deliver explicit literacy teaching on a daily basis [There are now] more cohesive approaches and new groups of teachers using the same language. Twenty-eight principals mentioned use of the Big Six framework, or parts of it, across the school, with others referring to staff discussions of the Big Six documents available on the DECD Literacy Secretariat website, and the Big Six podcasts. The development of a school-wide approach, as reported by so many principals, is a significant outcome of the project because lack of consistency across schools in relation to teaching literacy was a commonly identified issue of concern. In some cases, the discussions that led to these developments had more profound implications: a culture of learning through operating with teams and across classrooms was reported by one principal, and a culture of trust was reported by another, as the result of ensuring that all teachers had the opportunity to have their voice heard and their ideas, issues or concerns acted upon. One other principal reported a dramatic reduction in well-being targets as the school moved from behaviour controlling the learning to learning leading behaviour. An uplifting comment in one Intervention Evaluation report summed up changes in the following words: NAPLAN results are showing improvement in reading, but the biggest change has been in the culture of the school. More effective data collection to inform planning and teaching Changes to data collection and use in planning were important parts of the development that occurred, and were common themes in the Evaluation reports. Twenty-six reports included reference to a whole-school approach to assessment as being a major outcome of the PALL project, with another 13 references to better or more coherent use of data. Comments referring to data timelines, data collection schedules, the establishment of a literacy assessment cycle, looking at data as a collective group much more closely, and ongoing monitoring of reading progress indicated a new focus on the use of data to inform planning and teaching. The discussion of the PLP dimension of a strong evidence base also provided information about changes in the collection and use of data at the school level. We now have a data collection policy and student intervention meetings to analyse data fortnightly. Seeing huge changes in data cycles and what to use to assess, and then decide on next actions. [There are now] discussions around the importance of having good evidence and the need to gather data and interpret them. Needs more than just professional judgement. We have developed a comprehensive picture of the needs, progress and achievement of each student through an agreed data regime. Agreed targets and data collection are shared during primary and secondary meetings each term. Data are gathered and shared. 22

24 We went back and looked more closely at our data in relation to the Big Six, how students were progressing and what it means for us spent staff meeting time doing disciplined dialogue looked at resources from PALL to underpin a whole school approach. Fifteen comments (or 35.71% of the total on the strong evidence base dimension) were related to analysis of data leading to improved student achievement. The in depth diagnostic testing clearly shows what needs to be developed and the intervention programs using this testing have shown great progress. Assessment is very diagnostic. It is more around assessment for learning. Students needs are being diagnosed earlier and programmes are being adapted more frequently to ensure progress is being achieved. Teasing the individual reading skills and their impact on students reading development was very useful. The accountability is in the student outcomes. This is where we have the most productive performance development discussions and where the greatest changes happen. We are much more specific about student achievement across all areas of the Big Six and have developed ways of incorporating them into our student achievement books. There were nine comments (18.36% of comments on this dimension) from principals about school interventions and the links they had with a strong evidence base. One stated that strong data allowed the school to create and use an intervention model. Others saw the data as having a huge impact on Waves 1, 2 and 3 structures. Another principal maintained that using evidence led to increased efficiency in directing student interventions and that staff had been won over by the data. Support for this view came from a leader who said that teachers have a stronger sense of purpose as a result of reflecting on evidence and seeing the need to change their approaches. One other commented: Setting the seed in teachers minds around Hey, how about trying this is difficult. Slowly, the idea of new data and its potential to drive support got people on board. Some comments reflected changes in data collection procedures. One principal commented that he had become more aware of the need to be explicit about the information collected and the purpose for it. This then led to the introduction of a broad phonological awareness screening at school entry. Another said that SmartData was being compiled, saved and examined on a much more regular basis. Another stated: Using online assessments has resulted in easier access to data. Staff can access and analyse the data more easily and this has made a big difference to how data inform teaching at the classroom level. Principals also acknowledged that they had learned much by being exposed to this dimension. PALL has highlighted the importance of the analysis of data for me, and has led me to actively improving my skills in this area. PALL has made me more aware of the importance of collecting as much useful data as possible to inform any decision-making and change. 23

25 One leader stated that data from a strong evidence base enabled me to ask teachers for more at our performance management meetings. Another phrased it thus: Biggest area of personal growth is leadership of learning and working with staff who have a strong belief in their skill and ability, but little evidence of success. Data, data, data! The subset of principals who completed the final Project Evaluation Questionnaire confirmed a high level of school engagement with data. Table 6 summarises the responses related to collecting and analysing literacy data and suggests that the project had contributed particularly to principals having focused discussions with staff around literacy data and encouraging staff recognition of the links between data and literacy learning. Table 6: Leading literacy data gathering and analysis As a result of participating in the PALL project I have: Applied my knowledge about the usefulness and limitations of different types of data. Engaged in focussed discussions about data related to literacy learning (disciplined dialogues) with staff. Encouraged recognition of the links between sets of data to enhance literacy learning. COMBINED MEAN 1 Not at all Number of Respondents 2 To a slight extent 3 To a moderate extent 4 To a great extent Mean Six different principals, however, expressed some concerns about their schools data collection procedures. One referred to the fact that we talk about it, but it is not always happening in classrooms ; another that we still have work to do around analysing our data to use for evidence, while a third commented that, we have a lot of data but we need to ensure we use this data to inform our programming. Another expressed some concerns, but was optimistic that progress was being made: Data is not the issue, rather it is gathering the support to analyse data and move forward. There is some evidence that this is now taking place more consistently and purposefully. Evidence-based literacy instruction There were 20 references in Intervention Evaluation reports to the school-wide implementation of the waves approach to literacy instruction, an approach to differentiating instruction with a strong research base. Nine schools reported specifically on the progress made by wave two students, and nine (not all the same schools) also reported on progress made by wave three students. Seven additional principals referred to increased differentiation of instruction. Comments included: Waves of support gave us a framework to think about children s needs and SSOs time. Also, ensured more celebrations of our success as this showed up in my leadership data. The wave approach has brought an increase of strategies into the classroom. The practice, the oral work, repetition and links with spelling and AL are very clear. Small group support and bringing families into the activities has been powerful. 24

26 Reference to the introduction of a waves approach was also mentioned in the PLP discussion of the use of a strong evidence base. One principal said that data collection and analysis were being used in Waves 1, 2 and 3 and that the professional learning team was looking at connecting with and planning around mid-year data sets. The central challenge was to improve on these sets before the end of the year. Reading needs, for example, need to be more clearly established through PALL evaluation tools. The wave model of intervention has been really successful. A number of the reports referred to increased time allocations for special education teachers or SSOs to support third wave interventions; to the appointment of case managers for wave three students; and to increased access to specialist services in psychology, disability, social learning, attendance, behaviour and speech pathology for wave three students. Related to the waves approach was the introduction or reorganisation of literacy blocks. One principal stated that we have developed literacy blocks which have the same structure in each year level across the school - this is because of my new learning from PALL. Another commented that literacy blocks were working well in his school. Comments in the Principal Evaluation Questionnaire regarding the most significant changes that had occurred as a result of PALL included establishment of Literacy Agreement and Literacy Blocks ; whole school agreements - content and format of literacy block R-7 - whole school timetable which supports literacy block/intervention ; and staff agreements about what is important to include in literacy blocks. Fifteen reports made reference to an increased staff commitment to explicit teaching strategies, evidenced by statements about the use of synthetic phonics, very early exposure to blending with Reception students, the development of a reading scope and sequence and to the introduction of more explicit literacy programs such as Jolly Phonics, Reading Eggs and MULTILIT. One principal stated that Year 3-7 teachers teach reading rather than supervise tasks and another succinctly summed up his school s change as a move from providing reading activities to teaching reading. Other comments in Evaluation reports included: (There is) a focus on explicit, intentional teaching and practices. Developed resources and practices in classrooms that impact on explicit teaching and learning of skills, and strengthened phonic development in the early years. Greater need to be explicit and consistent. Focus on literacy has seen the resurgence in digital learning to support literacy. Evidence from the Teacher Evaluation Questionnaire referred to similar changes, with 60 teachers responding that they were now teaching reading more explicitly to a moderate or great extent. Teachers referred to the Big Six being explicitly used ; on planning activities and lessons with more specific goals, and paying more attention to detail. The following points were made when teachers were asked to identify the most significant change in their teaching as a result of the PALL project: Differentiation of students & explicit teaching of skills. More explicit teaching of both oral reading & comprehension. Explicit teaching of comprehension skills (strategies). Decodable readers for struggling readers. More explicit teaching, strong comprehension focus. 25

27 Professional development targeting staff knowledge of literacy development One of the responsibilities of principals participating in the PALL project was to pass on their knowledge to staff: to build their understanding of evidence-based literacy practices, and therefore have a positive impact on student learning. The PALL model was viewed as supportive of this process. One principal stated: PALL PD was particularly influential because it was sustained over a period of time and allowed me to take back ideas and trial them at my site, then be clearer about what I needed to learn at the next training day. In discussions of the PLP dimension on professional development, there were 13 individual comments (about 25% of total comments in this area) regarding changes to professional development at their schools. References were made to professional development now being ongoing or sustained ; to having a focus on consistent practice in relation to reading ; how it had high participation rates for staff both in and out of school hours/programs and how it was aligned to support classroom practice. Other principals referred to their new learning being a strength of our site, and the inclusion of latest research and practical strategies. Consistent with their responsibility to support staff in their development of literacy knowledge and practice was evidence that many principals did take on a leading role in the delivery of professional development in their schools. One principal s view was that: PALL has made it very clear that as the leader it is vitally important to play an active role in the professional development of staff. In a small school, this means that our whole staff is one PLC and therefore work as one, with one purpose, towards one goal. Without the information and background from PALL, I don t think I would be as confident in doing (these things). Another principal said that in the previous twelve months, she had managed to take a lead role in professional development for staff as an outcome of attempts to improve student literacy. Her role was not only to provide professional development, but also to lead it by example. Fourteen Intervention Evaluation reports referred to the opportunities the PALL project afforded and/or the momentum it gave to staff development in the area of literacy. For one principal, it prompted a realisation of just how much we did not know what we did not know ; and in another school, the staff had to accept the challenge to unlearn and relearn what quality practice actually was. Two different principals reported that implications for classroom teaching that emerged from PALL had become a standing item on staff meeting agendas. Three principals indicated literacy and the Big Six had become foci for pupil free days. Some principals did not lead the professional learning, but coordinated, participated in and shared the professional learning experiences. One commented that he ensured teachers had more access to uplifting professional development in literacy (e.g. the Big Six workshop). Others also tapped into other resources, not only those associated with PALL, as they engaged in the language of meaningful dialogue and increased reflection on teaching literacy. Training and development in the Big Six, Steven Graham and other specific T&D has enhanced the teaching and learning programs. Accessed ongoing support from the region s Literacy Coach, Curriculum Consultant, the Literacy Secretariat, Speech Pathologists and Hearing agencies. Increased teacher knowledge through combined programs Big Six, Stephen Graham. 26

28 At least five principals made comments about the professional development of support staff. One spoke of specific training of SSOs. Another said the non-teaching staff accessed all PD and are expected and supported to undertake PD to continuously improve to meet student learning needs. A third indicated that PD and readings around Big Six have been very successful PD for SSOs to support the intervention programs. Two others stated: [There has been a] very strong focus on professional learning for SSOs and the intervention team in comprehension strategies and vocabulary building. Involved staff and SSOs in T&D in the Big Six. Upskilling SSOs that support agreed approaches and networking with other schools was valuable. One leader commented that SSOs [were] competently assessing children s skills and another, that SSOs were becoming proficient in implementing specific intervention programs such as Multilit. Supporting evidence for changes in professional development was also gathered from the Teacher Evaluation Questionnaire, which in part explored the role that principals played in supporting classroom practice. The teachers strongly agreed that as a result of participation in the project, the principals were engaging in focused discussions with them about literacy teaching (see Table 7). The teachers felt less strongly about the principals participation in discussions about evaluation of reading interventions, with the mean of 2.8 for this item sitting between to a slight extent and to a moderate extent. Overall, the teachers believed that the principals were leading literacy in the schools to a moderate extent. Table 7: Teacher perceptions of principal s role in leading literacy learning Number of Respondents As a result of the school s participation in the PALL project, I have: 1 Not at all 2 To a slight extent 3 To a moderate extent 4 To a great extent Mean Received additional resources to support literacy teaching. Changed my classroom environment to support literacy more effectively. Participated in professional development about how students learn to read. Participated in discussions of reading achievement data to identify students for intervention. Engaged in focussed discussions with the Principal and/or colleagues about literacy teaching. Participated in discussions about different levels (waves) of reading intervention. Participated in discussions about how to evaluate the reading intervention. COMBINED MEAN Challenges around the professional development of staff remained for some principals, particularly in schools where there was a relatively rapid turnover of principals. One commented that: 27

29 There is still much work to be done with regards to supporting, evaluating and developing teacher quality. In a site where there has been significant turnover of principals, I am beginning to put processes in place to further develop teacher practice. Another wrote: As a principal, I have set very high expectations. However, it is taking time to bring all staff on board as most of the staff have been here for a considerable length of time and are comfortable with the status quo. In four months time, I will have been the longest serving principal in the past 8 years (2.6 years) and the fact that I am not leaving and am developing confidence as a leader is a reason for a growth in all domains. There is, of course, much work to be done but as my sense of purpose develops my staff will respond more positively to my leadership. Shared leadership of literacy learning Comments regarding shared leadership also reflected the impact of the PALL project at the whole-school level. Seventeen of the 40 PLP comments in this dimension related to building leadership. One principal wrote: This change has been achieved through a better understanding of the value of school leadership and making better use of the skills of other staff members. Building capacity has been a focus at our school. References were made to staff having more control over development plans and a collective responsibility developing. Other comments included: PALL has led me to feel it is OK to work collaboratively and take a back seat in some aspects of the work in order to encourage others to lead as well. [I have] come to value more highly the importance of shared leadership. Teamwork, collaborative decision-making and common agreements ensure focussed direction across classrooms and whole school based programs. Four principals mentioned structural changes within the school as ways in which leadership was shared. One saw re-defining the teacher librarian position into one that involved literacy as important. Another saw the reading support teacher becoming part of the literacy leadership team as worthwhile. Two others saw changes in structures as important. These involved using the reading support teacher to promote action research around reading instruction and identifying mentor teachers to support explicit reading instruction. Twelve of the 40 comments in this dimension related to collaborative and team work approaches. Some related to a particular group of staff within larger schools: a core intervention team or a professional learning team. Comments included: [The] core intervention team takes responsibility for many aspects of the program. Teachers have strong input when advocating for students. More opportunities for shared leadership through professional learning teams undertaking their own action research. This process is also creating greater accountability. New leadership with a common approach on-going meetings with this group. 28

30 The view that these new leaders were in turn leading the learning of staff was also prevalent. One principal stated that there was strong evidence in performance management meetings that all leaders had embraced the concept that they needed to lead the leading and get ownership from others. Another suggested that new leaders were guiding the analysis of NAPLAN data. A third maintained that our leadership is now broader and we include all of our Step 9 teachers. Further comments included: I have tried to provide more opportunities to take on school leadership. My staff member has shared excellent practices through PLCs. Have allowed teachers to be experts in their particular areas and encouraged them to run workshops for their peers and to access appropriate professional development. Encourage colleagues to step up and lead in their expertise areas and passions. Staff have engaged in many self-led professional practice discussions and research sessions to further the teaching of literacy. Engagement with all staff across the school was also prominent. One leader indicated that he met regularly with leadership and staff to examine whole school processes and had teachers plan together to align practices and data collection. A second principal stated that she worked on the PALL Big Six with the whole school staff and embedded the principles into the Site Improvement Plan and professional development planning. At least two other principals combined some of the PALL structures and processes with other key system developments to capitalise on strengths of individual staff members. One commented: I utilized teachers involved in other focus areas (Regional Literacy Program - Stephen Graham; Literacy Secretariat - Reading Focus). Both were empowering and collaborative in the early years structure. Another principal in a small school stated: A small school such as ours (5 teaching staff, including me) means that the opportunity is there to have everyone involved. The PAC comprises the whole staff, NEP and IEP reviews involve the whole staff. Staff are willing to and do take on many roles because there is no one else. In this situation, I feel a leader cannot stand aloof, the sharing of leadership and responsibilities is a must for continued smooth operation of the school. A number of comments, however, revealed concerns in this area, particularly in small schools. One principal maintained that developing leadership in a small school has been a struggle, but we are now much stronger in this area. A second said that in small schools, it is imperative that there is shared leadership; however, in a school that has seen much upheaval, shared leadership is only now emerging as a consistent practice. A third stated that, being in a very small school means minimal opportunity for leadership with a title. Opportunities for extending staff learning exist but are taken up spasmodically. Two principals identified other challenges for leadership within individual schools. One asserted that I overestimated myself in the first survey. PALL was a great knowledge and reality stimulation. The other stated that she feels there is further to go now talking to others about their leadership opened my eyes. 29

31 Improved provision and utilisation of resources Comments regarding changes to the resourcing of literacy teaching were also prominent, indicating additional ways in which the PALL project influenced operations at the school level. There were 18 PLP comments (or 42% of those in the conditions for learning dimension) about resourcing, with many of those referring to a new emphasis on aligning financial resources to literacy. Comments included: An audit of resources resulted in them being reallocated and gaps filled. We have purchased new resources related to teaching literacy. Targeted improvement in resources included testing resources. Curriculum priorities are supported by leadership and financially funded to achieve success for students. One principal summed up the influence of PALL in this area in the following way: Understanding the importance of creating the optimal conditions for students in which to learn (through PALL), I was able to have discussions with staff, parents and students about what they saw as areas of greatest need and, if it needed it, a significant amount of funding and resources was allocated. Changes to classroom structures and staffing allocations were also included in discussions of resourcing. One principal said that she had made big changes in the arrangement of classes where teachers are now working closely together. Other comments referred to review of classroom routines to assess use of resources, and a realignment of staffing resources during the year: By modifying class structures and school budgets we have provided additional resources to students. A new wave of resources developed for classrooms. Changes in staff had an influence on teaching. Another summed up his school s response in this way: Reviewed resources, targeted skill development and ensured depth of resources to support teaching and learning at a range of levels R-7. Used tips and tools from other sites acquired at PALL training days. Changes in the way SSOs were utilised were reported by 10 principals in their Intervention Evaluations, with one principal commenting that SSO hours were now allocated to meet the needs of specific students, rather than to teachers, and SSO time was much more accountable. Another principal referred to extra and better use of SSO time, and another had increased SSO time by an extra 250 hours to support wave two interventions. Changes in the duties of SSOs had an additional benefit in one school: Use of SSO time is now much more targeted and SSOs are reporting they feel valued as paraprofessionals. Resourcing, however, was an on-going challenge for some principals. Developing a more evidence-based approach not only required additional teaching and assessment resources in some schools, but also additional time for teachers to build a new knowledge base, analyse data with which they were unfamiliar, 30

32 and collaborate with their peers. Ensuring that the school could support staff as they developed these new skills and maintain(ing) teacher release time to look at data placed further pressure on school resources. Noting the variety of changes that were evident in the conditions for learning dimension (e.g. school budgets, literacy blocks, use of literacy practice guides, the wave model or data analysis), and the fact that most of these changes did not cost the school financially, these findings suggest that many schools have adopted altered structures and processes that will be sustained in the future. 4.3 Enhanced literacy knowledge and practice of teachers One responsibility that principals had as part of the PALL project was to pass on information about reading development and effective teaching practices to their staff. This was investigated through the following question: To what extent was teacher knowledge of the reading process, and the effective teaching of reading, enhanced as a result of the principals involvement in the PALL project? Data gathered from the PLP discussions across three of the dimensions - active involvement of school leaders in professional development with their teachers; enhancing the conditions for learning; and, planning and coordinating the curriculum across the school - reveal that most principals believed that teacher knowledge and practice had been enhanced as a result of the schools involvement in PALL. Specific comments relating to teacher development included the following: Teacher knowledge of the importance of the Big Six in reading vastly improved and was evident in their programmes and teaching. Improved teacher skills and confidence in teaching literacy. From practices that were not consistent or analysed deeply, teachers have worked together to develop consistent and coherent practices. Teachers have a better understanding of intervention and the planning needed to improve student reading. Teaching buddies modelled lessons to each other and shared across year levels. Teachers and SSOs have engaged in specific literacy development sessions broadening the skill base for our site and students, while aligning to the Australian curriculum. PALL has contributed to further development in literacy teaching and learning including teacher training, pedagogical change, new data sets and extended intervention. The Intervention Evaluation reports also contained references to the development of teacher knowledge, which impacted on different levels of their practice. One leader reported More (and deeper) professional conversations at staff meetings; more literacy specific language used in professional conversations; more literacy specific information given in reports to parents. Teacher views on the growth of their knowledge and skills about reading development and instruction were also canvassed. One or two teachers from each school who were directly involved in the literacy intervention being implemented as part of the PALL project were invited to complete the Teacher Evaluation Questionnaire. This instrument explored the extent to which teachers believed their knowledge of literacy learning and teaching had developed as a result of their school s involvement in the PALL project. Because teachers were not directly involved in the project, development in these areas would reflect to some degree 31

33 how successful the principals were in passing on their knowledge. A total of 70 teachers returned questionnaires, although not all teachers answered all questions. Table 8 combines items related to the teachers knowledge of reading development and instruction from two sections of the questionnaire. Sixty of the 69 teachers who responded believed their knowledge of how children learn to read had developed to a moderate or great extent. Sixty of 66 respondents (91%) said they were now teaching reading more explicitly to a moderate or great extent. The combined mean for this section was 3.1, reflecting an overall belief that their knowledge and skills had developed to a moderate extent. Table 8: Teacher knowledge of literacy learning and teaching Number of Respondents As a result of the school s participation in the PALL project, I have: Developed my personal knowledge of how children learn to read. Understood more about the relationship between decoding and comprehension. Understood more about the importance of automaticity and fluency to reading comprehension. Understood more about the importance of oral language and vocabulary to the development of reading. Used aspects of the Big Six model for the teaching of literacy. 1 Not at all 2 To a slight extent 3 To a moderate extent 4 To a great extent Mean Changed reading assessment practices Changed reading interventions for struggling students. Used more explicit teaching of reading strategies at classroom and individual levels. Developed my ability to diagnose student needs in literacy. Developed my professional capacity to address students literacy difficulties COMBINED MEAN 3.1 The final section of the Teacher Evaluation Questionnaire asked the teachers to rate how the school involvement in the project had impacted on them in relation to several important project outcomes. As the combined mean for this section was 3.0, this meant that overall the teachers considered that the schools involvement in the project had resulted in changes to a moderate extent in relation to the main project outcomes (see Table 9). 32

34 Table 9: Teacher responses to PALL project Number of Respondents As a result of the school s participation in the PALL project, I have: Developed my personal knowledge of how children learn to read. Developed my ability to diagnose student needs in literacy. Increased attempts to support parents in assisting their child s literacy development. Developed my professional capacity to address students literacy difficulties. Noticed improved student attitudes to literacy learning. Noticed increased student achievement in literacy. COMBINED MEAN 1 Not at all 2 To a slight extent 3 To a moderate extent 4 To a great extent Mean Comparisons of principal and teacher project responses Surveys rely on self-reported evidence, which is of limited value without some corroboration or triangulation from other data sources. To address this issue, comparisons were undertaken to determine the extent to which principals views on the impact of their involvement in the PALL project were consistent with teachers views. Categories in which principals and teachers were asked the same or very similar questions included: development of personal literacy knowledge (reflecting each respondent s view of his or her own development); views on the extent to which teacher literacy knowledge had developed; an increase in teachers evidence-based practice; and perceived changes in students literacy achievement. Responses were aggregated and then graphed. The rating scales were collapsed into two categories as follows: responses of not at all and to a slight extent were combined to make a minor extent category; responses of to a moderate extent and to a great extent were combined to make a major extent category. An important consideration here is that the principals and teachers who completed the questionnaires may not have been directly connected; that is, a principal may have completed the questionnaire while none of the teachers from his/her school did, and some schools may have only been represented by teachers completing the questionnaire. The anonymous nature of the questionnaires made it impossible to match principals with teachers. It is likely however, that the principals who did respond were more likely to encourage their staff to respond. It was also considered that some general comparisons of a sample of principals and a sample of teachers views could be made. The representation of these results as pie graphs makes similarities and differences in responses immediately apparent. Development of personal literacy knowledge Figures 2 and 3 show that in terms of literacy knowledge, principals were somewhat more positive than teachers about their own growth in this area as a result of their involvement in the project. The overwhelming majority (90%) of principals considered their literacy knowledge had developed to a major 33

35 extent while 79% of teachers placed their personal knowledge development in the major extent category. These are both high percentages, and suggest that overall the project was very successful in building literacy knowledge, although it must be remembered that these figures relate to perceptions only. These figures also suggest that there was some dilution of effect as knowledge was transferred from principal to teacher. This could be explained by the processes used by the principals to transfer their expertise, or by the time required for people to process new knowledge before they are able to successfully teach others. It may also be unrealistic to expect anyone with relatively newly acquired knowledge to deliver messages as effectively as an expert who had been immersed in and delivering the material for many years. Minor Extent Major Extent Minor Extent Major Extent 10% 21% 90% 79% Figure 2: Principal perceptions of their growth in literacy knowledge Figure 3: Teacher perceptions of their growth in literacy knowledge Comparison of principal and teacher perceptions of growth in teacher knowledge It was interesting to compare the level of agreement between the principals perceptions of teacher growth in literacy knowledge with the teachers own perceptions. They were remarkably similar, as demonstrated in Figures 4 and 5, and in fact principals were slightly more positive about this growth. The fact that more than 75% of both groups believed that teacher knowledge had grown as a result of PALL suggests that this model of professional development can have a significant effect. Minor Extent Major Extent Minor Extent Major Extent 23% 21% 77% 79% Figure 4: Principal perceptions of teacher growth in literacy knowledge Figure 5: Teacher perceptions of their growth in literacy knowledge 34

36 Increase in evidence-based practice by teachers The questions in this area related to principal and teacher perceptions of development in teacher practice. Seventy-four per cent of the principals considered that as a result of their involvement in PALL, teachers had increased their use of evidence-based practice to a major extent (see Figure 6). The percentage of teachers who stated that they had increased their use of evidence-based practice to a major extent was higher at 80% (see Figure 7). The discrepancy could be the result of the larger sample of principals, which would have included both highly engaged and less engaged principals, whereas only teachers actively involved in the PALL literacy intervention were surveyed. These percentages also suggest that building the skills and knowledge of principals and supporting them in their school contexts to develop teacher capacity can have a strong impact on the teachers involved. It should be remembered, however, that only one or two teachers from each school were asked to complete the questionnaire, and that no independent data were gathered on the capabilities of teachers across the schools apart from the views of the RLCs. Minor Extent Major Extent Minor Extent Major Extent 26% 20% 74% 80% Figure 6: Principal perceptions of teacher growth in evidence based practice Figure 7: Teacher perceptions of their growth in evidence based practice Perceived increase in student literacy achievement The percentages of principals and teachers who said that student achievement in literacy had been enhanced to a major extent as a result of PALL were very similar, with 83% of principals and 84% of teachers reporting this level of change (see Figures 8 & 9). As analysing data was a focus of the project and many of the principals reported undertaking this with teachers, it is understandable that a similar proportion of teachers and principals perceived changes in student achievement. It is encouraging that there was such a high level of agreement about the impact of the project on this important outcome, and that the student data included in the School Intervention reports support these views. A more systematic analysis of student achievement across all the schools who engaged in the project, however, would be required before definitive claims about the benefits of PALL can be made. 38

37 Minor Extent Major Extent Minor Extent Major Extent 17% 16% 83% 84% Figure 8: Principal perceptions of enhanced student literacy achievement Figure 9: Teacher perceptions of enhanced student literacy achievement What strategies did principals use to build the literacy knowledge and practice of their teachers and how effective did they believe them to be? Within PALL, development of teacher knowledge and practice around reading instruction relied on the principals, who were the direct participants in the project, passing on the relevant information. All principals had access to the presentation materials, frameworks and non-commercial assessments used in the modules. Principals, in collaboration with their staff, decided on the nature of the literacy intervention to be implemented, and the target group of students. Some schools addressed whole-class teaching across the school, while others targeted interventions with specific year levels, or with specific groups of students. Because the interventions were implemented by the teachers and SSOs working with the students, the ways in which principals managed the transfer of knowledge and skills, and the effectiveness of their strategies, had a major impact on the outcomes. One section of the Principal Evaluation Questionnaire explored the resources used by the principals, and the extent to which principals felt they were effective. Table 10 reveals that the Big Six framework was considered to be most effective tool (of those listed) in building teacher knowledge, with project readings and the Literacy Practices Guide also seen to be useful. In terms of developing the classroom practice of teachers, the Big Six model was perceived to be the most effective. What is not evident from these data is just how principals used these tools. Further investigation of this, and perhaps of the different combinations of tools used, would be helpful in determining the most effective ways of transferring the relevant knowledge and skills. 38

38 Table 10: Strategies used to build teacher knowledge and practice Effectiveness of different PALL components/strategies in building the literacy knowledge and practice of teachers. 1 Not at all No. of Respondents 2 To a slight extent 3 To a moderate extent 4 To a great extent Mean 1. Powerpoint Presentations 2. The Big Six Podcasts 3. The Big Six Framework 4. Project Readings 5. The Literacy Practices Guide Building Teacher Knowledge Changing Teacher Practice Building Teacher Knowledge Changing Teacher Practice Building Teacher Knowledge Changing Teacher Practice Building Teacher Knowledge Changing Teacher Practice Building Teacher Knowledge Changing Teacher Practice Enhanced student achievement in reading To what extent was student reading achievement enhanced by the principals involvement in the PALL project? While the perceptions of both principals and teachers regarding student reading achievement as a result of their schools involvement in PALL was very positive, it is critical to examine the actual data provided by schools. School leaders were given complete freedom in deciding how they would report student outcomes in order to maximise their ownership of the reporting process, and to minimise the additional work that would be required of them. The suggested report length of three-to-five pages also meant that the data provided would be minimal, although a number of schools submitted reports that were considerable longer. It is therefore difficult to report on anything apart from broad trends in the outcomes relating to student achievement. It must also be acknowledged that many, if not all schools were involved in a range of practices that would have had an impact on student achievement, and therefore the direct impact of the PALL project is difficult to determine. Nevertheless, positive responses in terms of student achievement were certainly attributed to the PALL project by many of the school leaders. This section provides an overview of some of the ways in which student progress was reported. All but three of the 76 intervention evaluation reports referred to improved student achievement, and included data ranging from school-based tests to NAPLAN results, and from quite general statements such as Increase in NAPLAN mean scores from 2011 to1012 for Years 3 and 7 and NAPLAN data for this year shows increased 39

39 achievement with the mean score and reflects the more rigorous, consistent whole school approaches, to comparisons of scores for reading, writing spelling and grammar over two years. It was not possible within the time frame of the project to compare results for the same cohort of students, but assuming there were broad similarities between student cohorts at the same school, some comparison is legitimate. NAPLAN results Some schools reported on the numbers of students above National Minimum Standard, increase in points, or percentages. Other reports compared student progress to regional and state benchmarks. The following points reveal some of the ways in which seven different schools reported NAPLAN results: A significant improvement in NAPLAN scores, and a larger proportion of students moving into the higher proficiency bands. These results have been acknowledged across the Northern Adelaide Region. NAPLAN data from 2012 testing indicates that 84.9% of students were above the National Minimum Standard, which is an increase of 6.9% from the previous year. NAPLAN growth - Reading grew 95 points; spelling grew 115 points Reading Year 3: 78.6% above NMS Reading Year 3: 80% above NMS Reading Year 5: 52.8% above NMS Reading Year 5: 92.6% above NMS. Yr 3 significant increase in reading, grammar and spelling. Yr 5 increase of 3-10% in all areas except writing. Yr % in all areas except writing which was equal for the first time in NAPLAN testing, all year levels (3, 5 & 7) met the N.A.R. standard in Reading. Year 3 NAPLAN reading mean score Increased 38 points from 2010 to One school provided comparative data to support their focus on literacy in the early years. Table 11: Example of data provided by school to demonstrate achievement in early literacy YEAR 3 Comparison 2011/ or - NUMERACY READING WRITING YEAR 5 Comparison 2011/ or - NUMERACY READING

40 WRITING YEAR 7 Comparison 2011/ or - NUMERACY READING WRITING Other standardised assessments Some schools reported progress measured by other assessments such as the Waddington, the PROBE, the South Australian Spelling Test (SAST), or specific intervention programs such as Mulitlit: 100% of R-2 classes are at or above CA (Waddington). The 30% of students with reading age below CA was reduced to 25%. The 25% of students below CA for spelling was reduced to 20%. Waddington Reading Test: Reading Ages for JP students have improved by 7 months or more between February and July testing. Upper Primary students have improved on average their score on the PROBE by 38%. 50% of Year 6 students showed more than 12 months gain in SAST spelling ages; all junior students improved spelling ages by at least 12 months. 55 students completed MULTILIT and passed the placement test. Running records At least 10 schools cited improvements in Running Record results: 2012 Running Record data: Year 2 students showed an increase from 40% to 100% of students achieving the regional average. Running Records data now indicates that 76.2% of our students in Years 1 and 2 have reached the School and Regional benchmarks. 86% of Year Two students reached our 2012 running records target: an improvement from last year with only 54% reaching the target. 72% of Year One students reached our 2012 targets compared to 61% last year. Running record data showed the following improvement in students achieving the district benchmarks between 2011 & 2012: Reception: 40% to 63% Year 1: 69% to 79% Year 2: 72% to 86% Upper Primary students moved up in their Reading Levels by at least 4 levels. After two and a half terms of intervention, 48/54 Wave 2 JP students showed an increase of at least 10 reading levels. 41

41 Phonological awareness In the early years of schooling, phonological awareness was successfully targeted for intervention by two schools: P.A identification and targeted program led to 28/34 students no longer needing intervention. PAA tests show 82% of students now scoring % on retest. Progress measured by unidentified assessments One report noted improved reading results, although the assessment instrument used was not recorded: The UP class went from 33% to 91% now reading at chronological age. The MP class went from 38% to 78% now reading at chronological age. The JP class went from 14% to 57% reading at chronological age. General comments Other schools reported more generally on student progress, or told individual student stories. Majority of Junior Primary students are at, or above, their chronological age in Reading. On average our Upper Primary students Reading Ages have improved by 6 months or more. Wave three students are able to retain the words they have learned and able to use them in routine life. It has also helped them to use words in writing. After 2 terms of intervention, 28/32 Wave 3 JP students made progress in reading. One JP student who did not know single sounds is now sounding out 3 letter words. Two leaders were not able to report enhanced student achievement as a result of their involvement in PALL. One reported inconsistent NAPLAN results, and another believed that Multilit was not having a significant enough effect to warrant continued use. One principal commented that: Our school was already progressing down a path of achievement in the area of reading and was well on its way in the other aspects of the literacy area. PALL was of limited benefit for the school. Apart from these instances, the results provided by the vast majority of the principals suggest that increased student achievement in different areas of literacy was, at least in part, due to the impact of the PALL project. 5. Effectiveness and challenges of project components 5.1 Project components that enhanced principal capabilities While it was clear from principal comments that confidence in their increased literacy knowledge encouraged them to take on a new leadership role with regard to literacy learning and teaching, further content analysis was required to identify the particular components of PALL that appeared to contribute most to these changes. Review of the module evaluations, discussions around principal growth in different dimensions of the PLP, and the different Evaluation Questionnaires, revealed that the following elements appeared to attract a particularly positive response from the participants. The evidence base supporting the PALL positions There was a strong emphasis in all professional learning modules on the research evidence that supported the positions taken in the PALL project. This aspect of the project resonated powerfully with many principals, and could be seen as one of the major reasons for the traction that the project gained from its initial stages. It was 42

42 during the first sessions of Modules 1 and 2 that the research behind the PALL leadership and literacy positions was presented. Principal feedback on these sessions was extensive. Of the combined 194 positive comments on the first sessions of these two modules, 100 comments referred specifically to the research and/or the evidence base that was presented. The strong emphasis on evidence, and the fact that the research was giving a consistent message over time, appeared to give the leaders confidence in the material being presented, providing a foundational credibility that touched the right chord with this audience. For some, this was because the research confirmed what they believed or intuitively understood: it made sense ; reinforced thinking ; was reaffirming and affirming of the practices we employ. It would also appear that principals appreciated having the synthesis of research provided for them: there were references to the concise or succinct summary of important factors. This seemed to increase the accessibility of the information: it deepened understanding and provided clarification. A number of comments also referred to the relevance, practicality or usefulness of these sessions, words not always associated with sessions that essentially encapsulate research findings. Comments such as absolutely relevant to our core business, information to move forward, linking theory with practice and putting it all together and making sense of the components suggest that the links between research and application were appropriately made. It is extremely encouraging that the principals recognised the importance of the evidence-based approach, and responded so positively to it. It provides a timely reminder of the fact that despite their many responsibilities and distractions, principals are drawn to content that will support their core business of leading learning. This orientation towards valuing evidence augurs well for the leadership of South Australian schools. Further examples of the comments listed under Highlights in the feedback on Modules 1 and 2 are listed below. Table 12: Sample of participants highlights in Modules 1 and 2 Module 1 Module 2 The different research sources and the collation of The evidence of research helps consolidate and common elements. challenge my own learning/understanding. The bringing together of latest research and analysis Allows us to go back into sites to share the of the commonalities. evidence/research as a basis of conversations. I loved hearing the summary of the research in a We must have an understanding of the evidence nutshell and being able to clearly see the crossovers (from research reports) to be basing decisions about and connection. teaching and learning on. Great to hear what research tells us about The Big Six clear! Evidence based! Thank you. leadership. Sharing of research information from a range of Clear and concise. The logical and clear presentation sources. of summaries of previous research. The strong research base information to move The presentation of research and evidence to forward. provide a basis for understanding excellent. Bringing together of research in a concise manner. Research background very useful. Interesting to hear research to reinforce what we ve Research based consensus to provide confidence to heard or believed. engage in discussion with teachers. Good overview and useful connections between Again the connection of the research and the main various research documents. elements. Great to have summary of considerable evidence Put a lot of known knowledge into context. presented succinctly. Rich range of research data stimulated interesting discussion. Supplied plenty of evidence. Great theory gave cause to think the literacy was interesting and a reminder of the historical context. 43

43 The provision of tools and frameworks Different sources of data confirmed the value of the frameworks, data collection tools, and organising matrices that were provided in the project. This reflects the need for professional learning for this very busy group of people to have immediate and practical application: if principals are to expend time and energy, they expect prompt pay-off or reward. This section discusses the feedback regarding different tools used within the project. The Leading Literacy Learning Blueprint (LLLB) The LLLB was seen to be a powerful overarching framework to support the work undertaken throughout the project. From its introduction in Module 1, it was seen to be exciting ; an excellent basis for focussed change ; a clear framework - simple and made sense ; and a useful tool for developing a leadership framework. It provided a chance to reflect on my leadership and have discussions with colleagues ; the reawakening that I need to reconnect to staff learning ; and really focused me on the elements I need to develop. For another principal, the Blueprint highlighted the need to build a vision and articulate it, plus the expectation of staff to implement it. Again, the evidence base, the depth of the Blueprint and the practical nature of the tool were valued by the principals. Its potential use in other contexts and across other learning areas was also commented on multiple times. Comments from Module 1 evaluations included: The Blueprint very important document for leadership within my school. Leadership for learning first and foremost. Application to all forms of leadership. Possible multiple uses of Blueprint with school community. Really liked this blueprint have thought about applying this in other areas. Clarity (easily understood) and ability to apply it across a range of contexts and school community groups; e.g. Governing Council, teachers. Useful tool to analyse what underpins the leadership and to discuss this further with staff. The LLLB s clarity and wide potential application increased its perceived usefulness. The eight dimensions of the LLLB also underpinned the Personal Leadership Profile, which allowed principals to monitor their growth in the different areas of leadership throughout the project. This instrument was critical in promoting reflection on changes in different dimensions of the principals leadership profiles, and was also an integral component of the project. Reflections on changes in the PLPs have provided significant data for this report. The Big Six framework The Big Six model, when introduced during Module 2, was seen to be explicit, focussed, clear and sound ; very clear, logical and accessible ; and assisted principals in putting it all together and making sense of the components/skills needed to be independent readers. For some, it refreshed knowledge ; put a lot of knowledge into context and was affirming of the practices we employ. For others, it was a good way to set the scene made me think back. Helped me reflect on my own beliefs and how they were formed over the years ; reinvigorated my thinking ; and was thought provoking the need to ensure students have the right building blocks in place. The Big Six framework thus facilitated the learning of the principals by clarifying the links between different components of the reading process, and the steps required to gaining comprehension. Module evaluation responses included the following comments: Interactive model was excellent and has given me a great resource to share with my Early Years Team. The Big Six loved it! Loved it! Loved it! Loved it! So much stuff to talk to my teachers about. 44

44 Enjoyed hearing the snippets about research that contributed to identifying and developing the Big Six concept want to share the research with staff to backup what should/is happening and why. Found this session (Big Six) really interesting and affirming of the practices we employ - ordered and systematic. Enjoyed it. Heaps of information. Thank you. Positive initial impressions are of little significance if the material is not useful in practice. For some principals, the model was instrumental in building the confidence in their literacy knowledge that was required for them to pass on the information, as reflected in the following comments drawn from PLP discussions: PALL, and in particular the Big 6, has given me the knowledge and background information that allows me to make recommendations to staff in regards to content and pedagogy that are making a real difference to student learning. [I have a] better understanding of reading elements (Big 6) and am able to speak more effectively to teachers about aspects of teaching and learning. Other PLP contributions suggested that the Big Six framework was used extensively in staff development, with references to lots of work on the Big Six ; learning about the Big Six has provided a good foundation for developing the model ; and the Big Six has been taught through staff meetings and professional learning communities. Other comments suggested that the framework could contribute to sustainability of actions implemented as part of PALL: With staff we have developed a greater understanding of the Big Six. Have tested students and this has become part of our school testing cycle. Working on PALL Big Six with the whole staff and embedding the principles in our site plan. Other principals responded that they used the Big Six to help parents support their children s reading development through workshops on topics such as how to read to your child or developing parent support by listening to reading and supporting those who need intervention. Thus the Big Six framework appeared to facilitate the transfer of knowledge to school sites. This statement is supported by comments made in the Teacher Evaluation Questionnaire, which included opportunities for teachers to comment on the most significant components of the PALL project for them. There were references to using the Big Six model, and the Big Six explicitly used. More extensive teacher comments included the following: Focusing on the Big Six and reviewing our phonological assessments has had a big impact on my program focus this year. The Big 6 has been great in looking at literacy from a different angle. My teaching in both comprehension and fluency, especially fluency, has been modified. Being more aware of making sure all aspects of the Big 6 are in my program. The biggest change has been in the different activities I set that focus on one or more aspects of the Big 6. The clarity and connection between the Big Six principles starting with oral language. The Big Six clarified the Big Picture for me in regards to development stages, strategies and outcomes a great framework to plan lessons. Principal introducing the Big Six /Wave 2. I used this as my base, which helped me differentiate my literacy program. 45

45 The Literacy Practices Guide (LPG) Initial responses to the LPG (as revealed in the module evaluations) reinforced the view that tools that had immediate and practical application in their role as instructional leaders were valued by most leaders. The LPG was seen as an excellent starting point ; a brilliant starting resource ; a highly useful document which I am excited about implementing with my staff ; and useful to develop common understanding of what literacy looks like in classroom. While many walkaround tools provide headings under which observations can be made, few actually provide checklists or indicators of best practice. The list of basic literacy practices ; the examples of R/1, 2-4, 5-7 for teachers ; the level of detail ; or more globally, the précis of the research, was seen as helping leaders know what to look for, hear and see. The descriptions of best practice added an educative aspect to the tool, particularly for principals who were not aware from their training or experience of the practices that were consistent with the research. A number of principals commented that the LPG would provide opportunities to open dialogue ; to develop discussions ; target conversations ; and help teachers reflect. Other comments included: I ve been looking for something of this standard for use with staff. It has some exciting possibilities for reflective practices. A straightforward improvement tool and process can t wait! Really like this it will fit in well with our PD and Performance Management. The level of detail included I m really keen to share it with staff (R-7). This tool will be extremely useful in SSI review and in performance building with staff. Thank you, thank you this is the tool I needed to begin the hard conversation needed. We have been looking for a checklist that can be used in performance management by all the leadership team. This will help. Models I can take back to school; stuff to start dialogue with teachers, especially rattle the cage (supporting of course) of one particular recalcitrant whom I ve recently discovered doesn t actually strategically explicitly teach reading to the individual child. Audit will provide evidence for further discussion re literacy practices and help site move towards consensus about what needs to be taught / provide performance management focus for discussion and planning targeted conversation. Some comments reflected principals caution about using the LPG. It was seen as a useful tool not sword. Another commented: This will be a powerful and confronting (for some) tool ; and another said some scary aspects but what the heck! The above comments were recorded when leaders were first introduced to the LPG, and suggest that most principals thought it would be useful. Again, determining their opinions after they had used it was more important. Comments made in PLP discussions and feedback sessions generally confirmed the perceived value of the LPG, with references to how it had aided observations of teachers, been strongly used ; and opened up discussions and supported deprivatising of classrooms. Two principals stated that it provided a useful frame for classroom observations and reflection, which was a focal point for performance and development conversations. Another said that it was used as a base for classroom observation and teacher self-reflection. A fourth stated that the Literacy Practices Guide underpinned what they were doing in their school in providing descriptive feedback to staff. 46

46 The LPG allowed principals to gather information about what was occurring in classrooms. One indicated that she had previously accepted teacher practices in passing, rather than investigating them. Another stated that by the end of the project, observations suggest a reduction in unproductive practices across the school. The LPG had facilitated one principal s active involvement in class programs with teachers and students. One principal, in discussing growth in his own learning, commented that the LPG helped him see that he had overestimated my knowledge about good practice. Others referred to the LPG being used in professional learning and training and development sessions ; and being agendered into staff meetings throughout each term. It exposed staff to effective literacy practices ; allowed principals to see regular work in classrooms ; facilitated student book audits ; and developed stronger performance management processes. One maintained that looking at student work across the board has provided a great deal of useful data. Two other principals made the following observations: Coming to a small school, it was the ideal time to make opportunities to observe teachers in action directly, not only so I had a better idea of how my new staff operated, but to see their strengths and where I thought they could make improvements. Staff are beginning to accept that observation of classroom practice is a normative practice as opposed to an indication of lack of trust, punishment etc. I expect that over the next twelve months, this domain will show an even greater growth. The final Principal Evaluation Questionnaire also explored the views of this subset of principals about the the LPG. The mean response to the five questions in this section was 3.3 of a possible 4 (see Table 14), indicating that the principals rated the impact of the Literacy Practices Guide on their leadership practice as between moderate and great. Table 13: Principal evaluation of the Literacy Practices Guide The Literacy Practices Guide enabled me to: Recognise effective literacy practices in classrooms. Support the set-up of classroom environments that facilitate student learning. Encourage more explicit teaching of readingrelated skills. Promote professional learning to develop teacher knowledge about how students learn to read. Promote a discussion about classroom teaching. COMBINED MEAN 1 Not at all Number of Respondents 2 To a slight extent 3 To a moderate extent 4 To a great extent Mean The RLCs, in their Evaluation Questionnaire, rated the extent to which principals promoted more explicit teaching of reading strategies at classroom and individual levels at 3.6 out of a possible 4, which could also be seen as a reflection of the value of the LPG with its descriptors of effective practices at different year levels. The RLCs who attended the focus group also mentioned the LPG as the second most effective element of the PALL project (after 47

47 the Big Six framework) in building their capacity to lead literacy learning. The LPG opened the door by describing what classrooms should look like. By providing a means by which leaders could access classroom practice with a defined focus, the LPG facilitated professional conversations, provided data about teaching practice, and informed both principals and teachers about best practice in the teaching of reading. Other data collection frameworks Modules 3 and 5 included a number of templates and grids to help principals identify gaps in data collection, organise data collection timelines, and plan evaluation procedures. These simple and clear documents were seen to be very valuable in helping leaders enact the PALL principles in their schools. Once again, the provision of practical tools was seen to be a strength of the project. In the words of one principal, Having some useful tools to help with planning was a positive outcome of the professional learning session; and for another, the proformas helped him re-think about what data we ve got/need and don t have/need. Other comments regarding the grids provided in Module 3 included the following: The data identification sheet certainly helped to identify gaps. I now have a clear idea of what data I need to collect and how to collect it. I will use chart to plot the data we use for Big Six. Practical application to own school to identify gaps of data collection areas. Useful framework for establishing a data schedule. Great model to use with staff to build comprehensive picture of the data we need to collect and the systems and processes. Using framework for planning to ensure that there are no gaps in the plan. Similarly, the planning documents provided in Module 5 received a positive response. The fact that these were colour-coded, and provided on USB, increased their usefulness. Module evaluations included the following comments regarding the frameworks: Good for organising our thoughts in readiness for evaluation. Framework for work in schools - simplified process. Planning tool useful plus examples given. Clarified some thinking found this would be useful in my planning. Very good to have scaffolding for evaluation of our intervention strategy. The Role of the Regional Leadership Coordinators The support of the RLCs between modules, and as principals led the implementation of literacy interventions, was crucial to the perceived success of PALL. The range in experience and the varying roles of leaders within their schools meant that every leader s context was different and presented differing challenges. A major factor in leaders effectiveness was their ability to align the project with all the other demands of their roles. This was one of the most challenging aspects of the RLC role: helping principals maintain momentum within the project and focus on the project s goals while understanding and empathising with leaders regarding the demands they faced from other quarters. Dealing with such difficulties required a persistent but pragmatic approach. 48

48 The Principal Evaluation Questionnaire collected data concerning the extent to which the RLC role supported the principals across several aspects of the project. The overall mean for this section was 3.2 from a possible 4.0, indicating that leaders felt that the role of the RLC was of moderate to great benefit to them in the project. The most positive responses from principals were in the areas of understanding the use of data; understanding of the Big Six; and understanding of literacy interventions. Each of these items had a group mean of 3.4. Table 14: Principal evaluation of the RLC role 1. The RLC s support improved my understanding of: a. The Leadership for Literacy Learning Blueprint. b. The Big Six. c. The Literacy Practices Guide. d. The use of data. e. Literacy interventions. f. Evaluation of interventions. 2. The Mentor supported my professional learning (including readings). 3. The Mentor supported my use of Disciplined Dialogue with staff. COMBINED MEAN 1 Not at all No. of Respondents 2 To a slight extent 3 To a moderate extent 4 To a great extent Mean Rating The Principal Evaluation Questionnaire also provided opportunities for principals to add comments about the contribution made by the RLCs. The combination of support and keeping principals on task was evident in many of the comments. Knowing someone was there if needed ; talking about improvement journey ; keeping me on track ; exploring data ; providing a sounding board to discuss strategies ; and time to discuss, reflect, clarify 1:1 were some of the benefits recorded. Other principals referred to the RLC monitoring, highlighting progress and sharing good practice, and providing active support in aspects of school program e.g. data analysis, resources, checking up on my tasks! Additional comments regarding the various ways in which principals benefited from the contribution of the RLC include the following: Support with organisation and opportunities to talk and reflect. Greater understanding & knowledge from a leader s perspective. Arranging follow up sessions with other leaders involved in the course, reminders of tasks to complete and support offered. 49

49 The design of the professional learning in our region has been outstanding. Our RLC has provided me with excellent support. Providing support and helping develop my literacy leadership and confidence - I believe RLC s make this programme work - without Colleen s input I think I would not have completed all tasks nor have the confidence to tackle staff literacy practices. Maintaining project momentum over the two years, particularly once the modules had been delivered and principals were no longer meeting together as a group, would have been very difficult without the continued involvement of these highly credible sources of information, encouragement and practical support. This outcome mirrors that of both the Pilot PALL project and the first extension, and also of smaller PALL projects conducted in Western Australia. The continued mentoring was integral to the success of the project. Opportunities for networking As with much professional learning, the opportunity to network with colleagues who faced similar challenges was a strongly valued feature of the PALL project. Sharing with colleagues, discovering commonalities, and discussions with others re practices that are occurring in their sites were very beneficial in clarifying beliefs/practices/further site development. The module presentations usually began with feedback sessions where school leaders shared their journey since the last meeting. Hearing other people s strategies ; listening to colleagues stories ; and finding out about the journey of others, their approaches and challenges were regarded as highlights. In the words of one principal, sharing dialogue is invaluable hearing about other leaders journeys is vital and enriching. A small selection of the many other comments regarding the value of networking included: In a diverse region the opportunity to meet face to face is rare and highly valued. Hearing what others are doing sharing different assessments and successes/issues. Extremely useful to share with colleagues chance to see what they are doing clarify my own work. Getting more ideas and support to implement within my site. Additional networking also occurred within the clusters served by different RLCs, a number of whom arranged sessions where leaders could meet and share their experiences and expertise. 5.2 Challenging components of the PALL project This section examines elements of the project that presented challenges for those delivering content and/or for participants. It also refers to a less effective outcome of the project. Delivery to large groups Finding the right balance of participant numbers relative to cost is difficult when attempting to implement largescale change. Delivery to large numbers of participants is challenging, and becomes even more complex when some are enthusiastic, on board and keenly anticipating the professional learning; and others have been conscripted, are too preoccupied, or are resistant to the message. The larger the group, the greater the differences in levels of knowledge, and the harder it is to meet each individual s needs. There were challenges associated with trying to meet the learning and delivery expectations of a very large and diverse professional audience when their needs and expectations were in some cases diametrically opposed. Often the same session attracted comments such as learnt nothing new, or points laboured unnecessarily ; while others found the content overwhelming and too much covered in too short a time ; and still others found it just right. Ironically, differentiating instruction is one of the principles espoused in the PALL project, but knowledge of the students is 50

50 required to do that. Even if that were possible through more orientation meetings or preliminary activities, differentiating to meet the needs of more than 150 participants would remain a challenge. For some participants, the opportunity to hear from experts in the field meant that they wanted maximum input from the presenters, rather than talking with colleagues who were no better informed than I was. Others needed to have regular opportunities to talk amongst themselves, and to relate material to their own context. Mindful of the limited time available, and with a large volume of core content to be delivered, the presenters on occasions erred on the side of too much presenter talk and too few opportunities for interaction for some participants. This is difficult to resolve unless different versions could be offered perhaps differing in the amount of table talk included, or in the level of knowledge assumed. Matching the development of knowledge with the desire for action The two-year duration of the PALL project had considerable advantages: extended engagement with the participants for presenters and RLCs; time for new knowledge to settle ; opportunities to bring staff on board and build their knowledge; time to trial assessments, collect data and plan the intervention, including putting in place any new structures that may be required; and time for the participants to coalesce into a community of learners who could support each other. Some participants, however, expressed impatience a desire to get on with it. It seemed that for some, a sense of purpose, even excitement, had already been established after delivery of the first two days. As one participant expressed it: still need/want the what s next. Many wanted assessment information and implementation strategies at that point to take advantage of the momentum that had been generated. While understanding the imperative for action the urgent needs of some of their students being paramount in their minds - the danger was that principals would take a collection of strategies without fully understanding how, when and why they should be used, and for partial understandings to affect implementation in the longer term. Combining the desire of some participants for action ; that is, information about assessment and intervention strategies early in the project, with the need to build relatively deep understanding of new concepts, presented a dilemma. Condensing the time taken for module delivery to perhaps the second half of the year, with implementation planned for the following year, may be an option, especially as the level of knowledge deepens across the state. Engaging parents and the community While not a focus of the research for the second extension, data were collected on perceptions of change in parent and community support generated by PALL through the Principal and Teacher Evaluation Questionnaires and the PLP discussions. Consistent with results of the PALL Pilot and the first extension, this was the area where least gains were made, although some schools reported some important outcomes in this area. In the PLP discussion of this dimension, 37 comments were made by principals regarding changes in their leadership (see Table 16). Most comments related to communication with parents. Five related simply to information provided through the school newsletter, but others referred to parent information evenings or open nights designed to develop parents interest in literacy, or specific parent workshops about, for example, how to read to their children and sharing knowledge with them. Almost 25% of comments in this dimension related to the active engagement of parents in individual schools: Ongoing engagement of a significant number of parent and senior student mentors in our key school reading program. Keep parents informed through class blogs. Other key parent involvement is with students in classrooms and involvement in active Autism group. Involvement by parents in assisting with reading in the classroom. 51

51 Encouraging more parents to be involved in reading in the class. Parent reading/volunteers and support at home has increased. Have run some parent workshops and developed a brochure for parents. Building parent capacity. Parent workshops held to assist parents to be more involved with their children. Developed a reading club which parents are noting and seeing. Parents involved in supporting literacy groups across the school. Table 15: Reasons for changes in aspects of Parent and Community Support Dimension aspect Include parents as integral to the school s learning program (Item 10) Are active in the local community and the professional communities (Item 14) Seek the input of professionals beyond the school (Item 18) Involve wider community support to improve learning (Item 27) Network with other schools and teachers on good practice (Item 30) Total number of comments made by principals on this dimension* Frequency of principal comments Only three comments from principals related to support for literacy from outside school groups. One principal spoke of connections with another school. A second indicated that they have formed partnerships with local clubs who provide assistance and funding to the school. Another said that working with other schools was probably not helpful, as it opened the expectations, yet resulted in only a small gain. A small number of comments related to the work being undertaken with Governing Councils. Broadcasting to the community about the direction and distance travelled through Governing Councils and newsletters were ways of ensuring parents understood the amount and level of support children were receiving. Strong membership and active direction of the new Governing Council... Opportunities to speak to Governing Council and parent groups and outline PALL and the Big 6 and teach members about our project. More than a quarter of the principals who responded, however, indicated that they had concerns in this area, or that there had been improvement to a small degree, but that more effort was needed. Two principals spoke of their concerns with negative parents and also the problems of working with poorer families. The first stated that there had been a group of parents who had been very negative and expected me to achieve a lot in a short time. The other, when speaking of socio-economic challenges, stated that they had been trying to improve communication through social media, and were investigating the use of Skype for parent interviews and Governing Council meetings. Other comments that reflected disappointment with this area of their leadership included: This is one area where we don t do well. I still need to do a lot of work here. I have organised workshops but don t get many attending. Have not consolidated in this area. 52

52 Trying to find time to create these partnerships. Our community is very difficult to engage, given we work very hard to do so. On a positive note, one leader responded that PALL has strengthened my commitment for more parent involvement. As mentioned previously, these results would be regarded as positive outcomes in many studies, but were relatively muted compared to development in other areas of leadership. 6. Conclusions and Implications This section summarises the major conclusions that can be drawn from the evaluation data. While the study has relied heavily on contributions from the principals directly involved, these have been supported by data from other key stakeholders, such as the RLCs, and teachers. Quantitative data was also derived from the Survey of Literacy Knowledge and Beliefs. Overall, a number of strong and consistent messages emerged from the data. 1. Growth in leadership capability occurred as a result of the PALL project The overwhelming majority of principals believed that their leadership capabilities had grown through their involvement in the PALL project. This view was strongly supported by questionnaire data from the RLCs and the teachers. As a result of the PALL project, many principals were actively coordinating and managing the school s literacy program from a position of greatly enhanced credibility. The extended nature of the project and the continued support offered by the RLCs contributed to this development, the impact of which was quite profound for some. These new capabilities constitute an ongoing resource for the individuals themselves, and for the system. 2. Growth in principals literacy knowledge occurred as a result of the PALL project Principals also believed that their knowledge about the process of learning to read, the different elements that must combine for this to occur in a timely manner, and the practices that support this most effectively for children who learn at different rates, had increased. This conclusion is also supported by data from the RLC and teacher questionnaires, and the Survey of Literacy Knowledge and Beliefs. The Big Six framework and the Literacy Practices Guide provided major support in this area. As a result of this increased knowledge, the learning environment for teachers and students should be enriched, and better able to cater for the diverse range of abilities that children bring with them. This could make a critical difference to struggling students, who need highly skilled and knowledgeable teachers if they are to make progress. 3. Growth in principal s literacy knowledge led to increased confidence in leading literacy learning Principals maintained that their own improved knowledge about reading gave them increased confidence in working with staff, governing bodies, and to a lesser extent with parents. This was particularly important for principals who had little or no previous knowledge about reading development as it opened up new opportunities to grow professionally, and to lead learning in their schools. Personal growth and increased job satisfaction can reduce stress and enhance well being, which could be seen as positive secondary effects of the PALL project, and lead to an even greater contribution by these leaders. 53

53 4. Whole-school changes around literacy teaching occurred across a significant number of schools as a result of the PALL project Many school-wide changes that were consistent with the effective teaching of reading were documented. This appeared to be a powerful outcome of the PALL DECS 2 project. The use of common language and whole school agreements around literacy, and focussed professional learning sessions were examples of changes that occurred relatively early in some schools. New or revised data gathering and analysis procedures across the school, and an increased focus on using data to inform planning and teaching, were integral to the change process in many schools, resulting in class restructuring, realignment of staff, and provision of new resources consistent with the PALL message. Adoption of the waves approach to instruction, the implementation of literacy blocks, and a widespread commitment to more explicit teaching were also key indicators of changes that were reported. The fact that many of these new practices and processes did not cost the school financially, and had become embedded in site plans, greatly increases the likelihood of sustained change. Established school-wide practices result in a critical mass of people who can contribute to the professional learning of new staff, supporting further sustainability. Consistent practices also provide the context for greater collaboration, and a stronger, more cohesive and more stable learning environment. 5. The Big Six was a useful and accessible model that facilitated learning The Big Six model of reading represents a vast body of research in a manner that most principals found accessible. Within the project, it facilitated communication with staff and parents, and provided a focus for school improvement. The framework helped schools align data collection and teaching around the elements required to build effective reading skills. The Survey of Literacy Knowledge and Beliefs revealed that while principal knowledge increased throughout the project, there was room for much greater development. Because the model of the Big Six has been widey accepted, and many leaders now have an understanding of the core components of the model, hubs of likeminded schools could work together to further develop their knowledge of reading. 6. Staff knowledge of reading also developed as a result of principal involvement in the PALL project There was evidence from different sources that the literacy knowledge of teachers and to a lesser extent, SSOs had developed, and was having an impact on their classroom practice. There was, not unexpectedly, some reduction in effect as information was transferred from leaders to classroom teachers, but the model whereby the school leader receives the content, and is responsible for passing the message on, acted to both increase leader credibility, and provide a cost-efficient way of building teacher knowledge. These teachers should feel empowered by their development, and through their greater capability and professional orientation, contribute to the sustainability of effective practice wherever they are appointed. 7. An increase in shared leadership resulted from many schools involvement in the PALL project Sharing leadership with individuals and collaborative groups increased throughout the project in many schools, resulting in greater collective responsibility for planning at the school level. In some cases this was very much informal rather than titled leadership as the particular expertise of individuals or groups was utilised. This broadened the skill base in the school, shared the workload, and further developed the professional expertise of the people involved. 54

54 8. Growth in student achievement occurred as a result of school involvement in the PALL project Both perception and achievement data support the view that student achievement in literacy was enhanced as a result of the PALL project. There was a very high level of agreement between principals and teachers about this point, with 83% of principals and 84% of teachers attributing reading improvements to involvement in PALL. All but three of the 76 submitted evaluation reports referenced improvements in students reading achievement, although the data provided to support these statements varied widely. If genuine acceleration of literacy development has occurred, a range of positive short and long-term outcomes should follow, not only for the children themselves, but also for their families and the broader community. 9. The role of the Regional Leadership Coordinator was integral to the success of the PALL project The positive contribution of the RLCs to the project cannot be overstated. The role demanded a complex set of skills and sensitivities: the ability to respond to widely differing levels of understanding and need; to facilitate the incorporation of PALL tasks into leaders core business without overwhelming them; and to maintain the fine balance between support and intrusion. Feedback suggests that overall the RLCs performed these duties outstandingly well, and that those filling these positions possessed the right combination of professional experience, credibility and personal qualities that was required to maintain momentum over the two-year life of the project. The independence of the role and the lack of any line management responsibility were crucial to the level of communication required for a sustained relationship to develop. Such a role should always incorporate these elements. 10. The strong evidence base underpinning the PALL project was an important determinant of its success The strong and consistent research base that supported the leadership and literacy frameworks appeared to have a strong impact on the participants. It strengthened the credibility of the positions put forward in the first two modules, and facilitated acceptance and use of the frameworks that were used throughout, such as the Leading for Literacy Learning Blueprint, the Big Six model, the Literacy Practices Guide and the Waves model of intervention. The fact that so many of the school leaders responded positively to this aspect appears to reflect an increasingly reflective and considered approach on their part. There will always be the next big thing presented to schools as the solution to one or more of the complex issues they face. Based on the leaders response to this aspect of the project, there will be greater scrutiny of such offerings. A more discriminating approach should increase the effectiveness of programs selected by schools and enhance the learning outcomes of their students. 11. Active engagement with parents and the broader the community continues to present challenges for many schools It is not always simple to find ways to involve parents in students learning. Many schools recognised this and many noted it as an area they needed to further address in 2013 and beyond. Whilst there were some positive outcomes reported, this research suggests that changes in this area were difficult to achieve. Little reference was made to work being undertaken with other schools or with community agencies or business groups. This appears to be the final frontier, and one that should be explored further with programs that include this as a more direct focus. 55

55 12. Enhancing classroom practice, and therefore student achievement, by improving the knowledge and leadership capacity of principals can be a cost-effective approach to change at a system level Implementing PALL in so many schools within a relatively small period of time was an ambitious attempt to improve the teaching of reading across a system. The implementation of PALL in benefitted from the experiences and evaluations of both the PALL Pilot and the first extension. While there were some difficulties in implementation, and maintaining the engagement of all school leaders across the two year period was always going to be difficult, the fact that 76 leaders submitted evaluation reports that included positive reports of changed teacher practice and improved student achievement means that large numbers of schools embraced the PALL message. When combined with those schools involved in the first PALL extension and the Pilot, a critical mass of schools should now be incorporating systems and practices that will result in improved classroom practice and enhanced student outcomes. This should provide the momentum for more widespread and sustained improvement across the system. 7. Recommendations Despite the perceived successes of the second extension of PALL in DECD schools, further work needs to be done to both assess its longer term impact, and inform future professional learning in the area of literacy development. 1. Systematic collection of quantitative data over several years. The data collected for this evaluation was largely perception data, and although triangulated from different sources, needs to be followed up by rigorous and systematic collection of student achievement data at the system level over at least five years to determine whether differences in achievement exist between PALL and non-pall schools. Because of the greatly differing levels of engagement or buy-in by different leaders, this analysis needs to be cross-referenced with information from the RLCs and project managers about which schools engaged deeply in the process to fairly assess the impact of the PALL project. If true change occurred, and sustainable processes and practices were put in place, the effects of leadership and other staff changes should be minimal. 2. Further case analysis of showcase schools It was impossible within this evaluation to tease out the direct impact of the PALL project because of the number and enormous complexity of factors that operate in school sites, and the variety of ways in which PALL interventions were implemented. While all data sources used in this evaluation attributed positive outcomes to the PALL project, other programs and enterprises were also mentioned. Longer-term DECD initiatives such as SILA, regional emphases on particular skills such as oral language, and other specific programs such as that offered by Stephen Graham were almost certainly part of the improvements noted at some sites. Much of the success of the PALL initiative could well be attributed to the fact that the principles espoused by PALL supported many existing practices, and so fell upon fertile ground. In some cases, PALL could simply have provided a framework to facilitate communication and provide support for changes that were already underway in many sites. Much has been learnt in other countries from analysis of schools that punch above their weight in terms of student outcomes. Some PALL schools stood out in the progress made by their students. A closer analysis of a sample of these schools to determine any commonalities could provide important information about the combination of factors that lead to the best outcomes for students. Important factors could include some specific element of PALL, such as the overarching framework provided by the LLLB or the support of the RLC; or to a certain body of knowledge such as that encapsulated by the Big Six, or a particular tool such as the LPG. Alternately, enhanced progress could relate more to the ways in which leaders transferred knowledge to their staff; or to the existing experience and knowledge of staff that was simply tweaked by the PALL experience; or even to the use of particular resources. It could eventuate that PALL was most successful in interventions that 56

56 targeted a particular stage of reading development, or students with more entrenched difficulties. It may also be that the most successful schools are those where PALL operated in combination with one or more other programs or initiatives. Much could be learnt from a closer analysis. 3. Establishment of communities of learners Schools could be paired with a buddy school or grouped in small hubs to further develop their learning, using the Leading Learning Blueprint and other PALL frameworks. Leaders could request to be matched with a school that has demonstrated excellence in an area they want to develop. This model, of course, could apply to any curriculum area, and would already exist on an informal basis. The RLCs could contribute to the establishment of these learning communities with their knowledge of exemplar schools in different curriculum areas, and support the learning and the relationships with the skills they used so effectively in PALL. This could be one way in which PALL principals could continue their learning in the critical area of reading. 4. A periodic PALL conference If PALL continues to have some traction, particularly across Australia as a result of the Train the Trainer program and other iterations of the project, a conference held on a negotiated basis could present stories of successful practice that are grounded in the core business of principals, thus maintaining the momentum of the project, strengthening the network of leaders involved, and inducting new leaders. Alternately, conferences of relevant professional associations (for example, APPA), could have a PALL stream where success stories are shared. At a more local level, regular check-in sessions could support the sustainability of PALL practices. 5. Parent involvement In individual or clusters of school communities, much of the learning from PALL could be packaged to help parents support their children s literacy development. The lack of emphasis on explicit teaching which permeated teaching and teacher education courses for many years has also resulted in many parents being unfamiliar with languagebased information that most people once learnt in primary school. More informed parents would lead to a better preparation and ongoing support for many children. 6. Continued dissemination of updated materials Updated materials, tools and frameworks should be made available through an established website such as one operated by the SA DECD or APPA. Materials and resources that have been developed and implemented successfully by schools could also be more widely distributed. 8. Final Comments The declining literacy achievement of Australian students over the past several decades continues to provoke significant comment in the public arena, and much political energy and financial outlay at state and federal levels has been devoted to finding a solution. While many factors contribute to a school s capacity to provide the best learning opportunities for its students, the knowledge and skills of the teacher, and what happens in the classroom has the greatest effect on student learning aside from factors located within the students themselves. This evaluation suggests that the PALL project has been largely successful in increasing the literacy knowledge and pedagogical skills of teachers at a system level in an affordable way by working with principals to ensure credible leadership support. It also supports the paradigm shift that is gradually gaining momentum across Australia as the need to address the low literacy levels of so many students has become apparent, and the role of explicit and systematic teaching has been acknowledged. There is no doubt that the PALL project has had an impact in the short term: attention has been focused on an evidence-based approach to the teaching of reading, and there are tantalising signs of upward trends in teacher 57

57 knowledge and student reading achievement in schools that have embraced the PALL messages. Results over the next few years will provide a clearer indication of its true impact, but there is cause for optimism. This report concludes with a principal s comment offered within the school s Intervention Evaluation report: The Pall project has been one of the most powerful agents for school change in all of my experience as a principal and teacher. Its emphasis on improving the knowledge of the leaders and then providing the resources to pass this on at the school level has been both empowering and enlightening, and has lifted the confidence of principals to engage in leading literacy improvement in schools. 58

58 References Brannen, J. (1992). Mixing methods: Qualitative and quantitative research. Thousand Oaks, CA: Avebury. Dempster, N., Konza, D., Robson, G., Gaffney, M., Lock, G. and McKennariey, K. (2012). Principals as Literacy Leaders, Kingston: Bowden. Dempster, N., & McKennariey, K. (2012). Principals as Literacy Leaders SA DECS: Final Report. Available at Coe, R. (2002). It s the effect size, stupid: what effect size is and why it is important. Paper presented at the Annual Conference of the British Educational Research Association, university of Exeter, England, September. Available at Flick, U., Kvale, S., Angrosino, M. V., Barbour, R. S., Banks, M., Gibbs, G., et al. (2007). The Sage qualitative research kit (B. Jenner, Trans.). London: SAGE. Gall, M. D., Gall, J. P., & Borg, W. R. (2007). Educational research: An introduction (8 th ed.). Boston: Pearson/Allyn & Bacon. Joshi, R., Binks, E., Hougen, M., Dalhgren, M., Ocker-Dean, E., & Smith, D. (2009). Why elementary teachers might be inadequately prepared to teach reading. Journal of Learning Difficulties, 42(5), Konza, D. (2010). Understanding the reading process. Research into Practice, SA Department of Education and Children s Services. Konza, D. (2012). The Literacy Practice Guide: A smart tool for principals. Paper presented at the national conference of the Australian Council for Educational Leaders, Brisbane, October 3-6. Lyon, G., & Weiser, B. (2009). Teacher knowledge, instructional expertise, and the development of reading proficiency. Journal of Learning Difficulties, 42(5), Punch, K. F. (2005). Introduction to social research: Quantitative and qualitative approaches. London: SAGE Publications. Sallinger, T., Mueller, L., Song, M., Jin, Y., Zmanch, C., Toplitz, M., Partridge, M., & Bickford, A. (2010). Study of teacher preparation in early reading (NCEE ). Washington, DC: The National Center for Education, Evaluation and Regional Assistance, Institute of Educational Sciences, U.S. Department of Education. Thomas, R. M. (2003). Blending qualitative and quantitative research methods in theses and dissertations. Thousand Oaks, CA: Corwin Press. Walsh, K., Glaser, D., & Wilcox, D. (2006). What education schools aren t teaching about reading and what elementary teachers aren t learning. Retrieved from Wiersma, W., & Jurs, S. G. (2005). Research methods in education: An introduction (8 th ed.). Boston, MA: Allyn and Bacon. 59

59 Appendices Appendix A: Leading Literacy Learning Blueprint Neil Dempster: Griffith University:

60 Appendix B: The Big Six Model School of Education Fogarty Learning Centre 61

61 Limited Sound Very good Excellent Appendix C 1: Personal Leadership Profile (PLP) Your name: School: Please indicate below your time with the PALL Project. Tick the relevant box(es). A.) I have been involved from the beginning (early 2011). OR B.) I joined part way through the Project. I received some induction. I received no induction. How to complete the Profile: You are asked to rate the extent of your knowledge and skill about each of the leadership actions listed. The questions focus on aspects of leadership known to be linked with learning. Please tick the point on the scale that reflects your judgment. The status of my knowledge and skill to: 1. Actively oversee the school s curriculum program 2. Promote skills in data analysis and interpretation through PD amongst teachers 3. Coordinate and manage the teaching and learning program 4. Observe teachers in action directly and provide specific feedback 5. Ensure that both school and system data are gathered 6. Encourage team work amongst teachers 7. Set high expectations 8. Build vision and set directions collaboratively 9. Plan school organisation structures to support improved learning 10. Include parents as integral to the school s learning programs 11. See that goals are embedded in school and classroom routines 12. Ensure that teachers engage in extended learning about school priority areas 13. Manage resources strategically 14. Be active in the local community and the professional communities 15. Align financial resources to priorities 16. Participate as leading learners with teachers in professional development 62

62 Limited Sound Very good Excellent The status of my knowledge and skill to: 17. Pursue systematic data gathering across the school s responsibilities 18. Seek the input of professionals beyond the school 19. Provide a safe and pleasant physical environment 20. Support, evaluate and develop teacher quality 21. Ensure consensus on goals 22. Maintain commitment to curriculum priorities 23. Support collaborative work cultures 24. Ensure social and emotional support for learners 25. Celebrate teacher and student successes 26. Display a keen interest in students classroom work and achievements 27. Involve wider community support to improve learning 28. Share leadership systematically with teachers 29. Plan for student learning based on data 30. Network with other schools and teachers on good practice 31. Monitor student learning based on data 32. Share accountability tasks with teachers based on classroom, school and system data 33. Apply resources to the conditions of learning 34. Participate actively in curriculum decision making 35. Concentrate on the development of deep knowledge about key learning areas 36. Play an active hands on role in professional development 63

63 Appendix C2: PLP Report (T1) 64

64 Appendix C3: PLP Report Time 2 65

65 Appendix D: Survey of Leaders Literacy Knowledge and Beliefs 1. Vocabulary knowledge on school entry is one of the strongest predictors of future reading ability. Strongly Disagree Disagree Not Sure Agree Strongly Agree 2. The teaching of phonic elements of reading should always be based within meaningful text. SD D NS A SA 3. Assessment should primarily be carried out to inform future planning for student learning. SD D NS A SA 4. Students must attain automaticity of the basic elements of reading if they are to be successful in comprehending text. SD D NS A SA 5. Phonological awareness refers to an awareness of the relationship between letters and sounds. SD D NS A SA 6. Books with predictable text are a useful resource for students to practise early reading skills. SD D NS A SA 7. The use of context is more helpful than letter-sound knowledge from the earliest stages of learning to read. SD D NS A SA 8. Children learn to read in much the same way as they learn to talk. SD D NS A SA 66

66 9. Fluent readers do not need precise decoding skills as they are able to make meaning from other cues. SD D NS A SA 10. Effective teaching of reading requires specific study of separate skills such as vocabulary, fluency, phonics and comprehension. SD D NS A SA 11. Sustained silent reading is a vital part of every reading program as it models best practice. SD D NS A SA 12. Teaching spelling is not important because the English language is too inconsistent. SD D NS A SA 13. Decodable readers are a useful resource for students to practise early reading skills. SD D NS A SA 14. Students who are significantly behind in reading benefit from being withdrawn from most literacy lessons for a different program because they are gaining very little from being in the mainstream class. SD D NS A SA 15. Most beginning readers need explicit and systematic teaching of phonics. SD D NS A SA 67

67 16. Teachers must give more time to struggling students if they are to succeed. SD D NS A SA 17. Schools should have standardised assessments for all year levels in reading. SD D NS A SA 18. Daily lesson planning is essential in literacy. SD D NS A SA 19. Each school should have a literacy expert to teach students with severe reading problems. SD D NS A SA 20. Teacher judgement is not as valuable as standardised assessment of reading ability. SD D NS A SA 21. I have a strong grasp of the theory of reading development. SD D NS A SA 22. I am confident in my ability to teach reading to every child in my class. SD D NS A SA 68

68 Appendix E: Example of Module Feedback Sheet Feedback Sheet: Module 3 Leadership for Learning Leading Literacy Data Gathering and Analysis Session 2 Revisiting the Literacy Practices Guide: What did we find? What was the highlight? Not helpful Extremely Worthwhile What would you change? Session 3 Using data to establish priorities for reading development: Positions and starting points What was the highlight? Not helpful Extremely Worthwhile What would you change? Session 4 Leading Literacy Assessment for diagnostic teaching What was the highlight? Not helpful Extremely Worthwhile What would you change? 69

69 Session 5 Using data to explore reading development priorities at the classroom and individual student level What was the highlight? Not helpful Extremely Worthwhile What would you change? Session 6 Developing a plan to collect and explore data in your school related to establishing reading development priorities What was the highlight? Not helpful Extremely Worthwhile What would you change? 70

70 APPENDIX F: Principal Evaluation Questionnaire PRINCIPALS AS LITERACY LEADERS PRINCIPAL QUESTIONNAIRE You are invited to complete the following questionnaire about your experience of the PALL DECD project. It should take no more than 10 to 15 minutes to complete. This is anonymous unless you choose to be contacted for a follow up interview towards the end of the year (see last page). Before commencing the questionnaire, please complete the following by ticking the box appropriate to your situation. In my school the PALL intervention is concentrating on: (tick all that apply) Oral language Phonological awareness Phonics/word study Vocabulary Fluency Comprehension Other The intervention focussed on the following groups (tick all that apply) R-1 Wave Wave Wave 3 Thank you for your participation. Yours sincerely Deslea Konza PhD Edith Cowan University May

71 Broader aims of the PALL project in SA DECD were to build classroom teacher knowledge of reading and to develop the classroom practice of teachers. We are interested in the strategies used to do this, and the extent to which you believe these aims have been achieved. Please tick any of the following activities undertaken at your school to build teacher knowledge and develop classroom practice, and the extent of their effectiveness for in achieving both aims. 1. Passed on workshop information using the same or all of the Powerpoint presentations provided. This strategy was successful in building teacher Strongly disagree Disagree Agree Strongly Agree Unsure knowledge. in changing teacher practice. 2. Showed and discussed with staff one or more of the Big Six podcasts available on the APPA website. This strategy was successful in building teacher Strongly disagree Disagree Agree Strongly Agree Unsure knowledge. in changing teacher practice. 3. Used the Big Six to provide a framework for the teaching of literacy. This strategy was successful in building teacher Strongly disagree Disagree Agree Strongly Agree Unsure knowledge. in changing teacher practice. 4. Shared and discussed one or more of the readings provided as part of the PALL project. This strategy was successful in building teacher Strongly disagree Disagree Agree Strongly Agree Unsure knowledge. in changing teacher practice. 72

72 5. Involved staff in additional PD to reinforce key messages. This strategy was successful in building teacher Strongly disagree Disagree Agree Strongly Agree Unsure knowledge. in changing teacher practice. 6. Used the Literacy Practices Guide to engage staff in a Disciplined Dialogue about their classroom practice. This strategy was Strongly disagree Disagree Agree Strongly Agree Unsure successful in building teacher knowledge. in changing teacher practice. 7. Purchased and used different assessment instruments to assess and monitor student knowledge. This strategy was successful in building teacher Strongly disagree Disagree Agree Strongly Agree Unsure knowledge. in changing teacher practice. 8. Engaged with teachers in more frequent discussions of student achievement based on data. This strategy was successful in building teacher Strongly disagree Disagree Agree Strongly Agree Unsure knowledge. in changing teacher practice. 9. Purchased and used new resources that were consistent with the key messages of the PALL program (e.g. synthetic phonics program, magnetic letters, decodable readers). This strategy was Strongly disagree Disagree Agree Strongly Agree Unsure successful in building teacher knowledge. in changing teacher practice. 73

73 10. Involved more staff in literacy leadership roles. This strategy was successful in building teacher Strongly disagree Disagree Agree Strongly Agree Unsure knowledge. in changing teacher practice. Overall, as a result of participating in the PALL project I perceive improvement in: Overall Project Outcomes Strongly disagree Disagree Agree Strongly Agree Unsure My personal literacy knowledge. My professional capacity in leadership for literacy learning. Our school s capacity to address students difficulties in literacy learning. Our students literacy achievement. 74

74 APPENDIX G: Principal Evaluation Questionnaire - Final PRINCIPALS AS LITERACY LEADERS Principal Questionnaire You have recently participated in the Principals as Literacy Leaders (PALL) project, with funding provided by the SA Department of Education and Child Development. I invite you to complete the following questionnaire about your experience of the project to this point. This is an anonymous questionnaire. Please do not write your name, or any other comments that will identify you or your school. Thank you for your time and consideration in reflecting on your experience of the PALL project. Yours sincerely Dr Deslea Konza Associate Professor of Language and Literacy Director Fogarty Learning Centre Edith Cowan University 75

75 Instructions Please complete the questions by ticking the appropriate box or inserting short answers (e.g. dot points) for open-ended questions. Background Information 1. What is the size of your school? a. Less than 100 students b. Between 101 and 300 students c. Between 301 students and 500 students d. More than 500 students 2. How long have you been a principal or literacy leader a. Up to three years b. Four to seven years c. Eight to fifteen years d. More than fifteen years 3. In your position as a school leader do you undertake classroom teaching as part of the school s timetable? a. Yes b. No 4. In my school the literacy interventions concentrated on: Year Level(s) 5. Focus e.g. Phonics, Oral Language, Vocabulary, Comprehension, Spelling, Other (please specify) 76

76 1. Roles of the Mentor 1. The Mentor s support improved my understanding of: a. The Leadership for Literacy Learning Blueprint b. The Big Six c. The Literacy Practices Guide d. The use of data e. Literacy interventions f. Evaluation of interventions 2. The Mentor supported my professional learning (including readings). 3. The Mentor supported my use of Disciplined Dialogue with staff. Not at all To a slight extent To a moderate extent To a great extent 4. What has been the single most useful aspect of the role of the Mentor? 5. What aspect of the role of the Mentor could be improved? 2. Use of the Literacy Practices Guide The Literacy Practices Guide enabled me to: Not at all To a slight extent 1. Recognise effective literacy practices in classrooms. To a moderate extent To a great extent 2. Support the set-up of classroom environments that facilitate student learning. 3. Encourage more explicit teaching of reading-related skills. 4. Promote professional learning to develop teacher knowledge about how students learn to read. 5. Promote a discussion about classroom teaching. 77

77 3. Knowledge of Literacy Learning and Teaching As a result of participating in the PALL project I have: Not at all To a slight extent 1. Learnt more about how children learn to read. To a moderate extent To a great extent 2. Understood more about the relationship between decoding and comprehension. 3. Understood more about the importance of automaticity and fluency to reading comprehension. 4. Understood more about the importance of oral language and vocabulary to the development of reading. 5. Promoted the Big Six model to provide a framework for the teaching of literacy. 6. Reviewed assessment practices based on the Big Six model. 7. Promoted changes to the nature of reading interventions for struggling students. 8. Promoted more explicit teaching of reading strategies at classroom and individual levels. 9. Encouraged conversations with parents about student literacy development. 10. What do you regard as the single most significant impact in your school of the PALL Project? 11. Can you give an example from your school? 78

78 4. Leading Literacy Data Gathering and Analysis As a result of participating in the PALL project I have: Not at all To a slight extent 1. Applied my knowledge about the usefulness and limitations of different types of data. 2. Engaged in focussed discussions about data related to literacy learning (disciplined dialogues) with staff. 3. Encouraged recognition of the links between sets of data to enhance literacy learning. To a moderate extent To a great extent 5. Literacy Interventions As a result of participating in the PALL project I have: Not at all To a slight extent To a moderate extent To a great extent 1. Discussed different levels (waves) of literacy interventions with staff. 2. Provided additional resources to support literacy teaching in classrooms. 3. Monitored the different levels of literacy interventions. 4. Worked with staff on data to identify different target groups for intervention. 79

79 6. Evaluation of Intervention and Future Planning As a result of participating in the PALL project I have: Not at all To a slight extent To a moderate extent To a great extent 1. Shared the development of questions, criteria and approaches to evaluating literacy interventions with staff.* 2. Set up processes to evaluate our reading intervention. 3. Analysed outcomes to determine the efficacy of literacy interventions in collaboration with staff. 7. Project Outcomes As a result of the school s participation in the PALL project, I have: Not at all To a slight extent To a moderate extent To a great extent 1. Developed my personal knowledge of how children learn to read. 2. Developed my professional capacity in leadership for learning. 3. Developed my capacity to conduct Disciplined Dialogue about students reading development. 4. Developed my school s capacity to support students reading development. 5. Noticed improved student attitudes to literacy learning. 6. Noticed increased student achievement in literacy. 80

80 APPENDIX H: Teacher Evaluation Questionnaire EDITH COWAN UNIVERSITY PRINCIPALS AS LITERACY LEADERS (PALL SA DECD) Teacher Questionnaire As you know, your principal is a participant in the Principals as Literacy Leaders (PALL), with funding provided by the South Australian Department for Education and Child Development. Edith Cowan University is evaluating the effectiveness of the project for principals and teachers. Your principal has nominated you as a staff member who has been involved in the literacy intervention associated with the project. We invite you to complete the following short questionnaire about your experience of the project to this point. As this is an anonymous questionnaire, please do not write your name, or any other comments on the questionnaire that will identify you or your school. By completing this questionnaire you are consenting to take part in this research. Thank you for your time and consideration in reflecting on your experience of the PALL project. Yours sincerely Deslea Konza Associate Professor of Language and Literacy Director Fogarty Learning Centre Edith Cowan University 81

81 PALL Teacher Questionnaire Instructions Please complete the questions by ticking the appropriate box or inserting short answers (e.g. dot points) for open-ended questions. Background 1. Your role {more than one response may be appropriate for this item] a. Teacher b. Coordinator (e.g. literacy leader, curriculum co-ordinator) c. School executive member (e.g. Assistant Principal, Deputy Principal) 2. Current area of teaching responsibility a. Lower primary (including pre-compulsory) b. Middle primary c. Upper Primary d. All of the above 3. Years of teaching experience a. Up to 3 years b. 4 to 7 years c. 8 to 15 years d. More than 15 years 82

82 1. Knowledge of Literacy Learning and Teaching As a result of the school s participation in the PALL project, I have: 1. Learnt more about how children learn 1 Not at all 2 To a slight extent 3 To a moderate extent 4 To a great extent to read. 2. Understood more about the relationship between decoding and comprehension. 3. Understood more about the importance of automaticity and fluency to reading comprehension. 4. Understood more about the importance of oral language and vocabulary to the development of reading. 5. Used aspects of the Big Six model for the teaching of literacy. 6. Changed reading assessment practices. 7. Changed reading interventions for struggling students. 8. Used more explicit teaching of reading strategies at classroom and individual levels. 9. Talked with parents about student literacy development. 10. What has been the most significant change in your teaching as a result of the PALL project? 11. What do you believe has brought about this change? 83

83 As a result of the school s participation in the PALL project, I have: 1. Received additional resources to support literacy teaching. 2. Principal s Role in Leading Literacy Learning 1 Not at all 2 To a slight extent 3 To a moderate extent 4 To a great extent 2. Changed my classroom environment to support literacy more effectively. 3. Participated in professional development about how students learn to read. 4. Participated in discussions of reading achievement data to identify students for intervention. 5. Engaged in focussed discussions with the Principal and/or colleagues about literacy teaching. 6. Participated in discussions about different levels (waves) of reading intervention. 7. Participated in discussions about how to evaluate the reading intervention. 3. Project Outcomes As a result of the school s participation in the PALL project, I have: 1 Not at all 2 To a slight extent 3 To a moderate extent 4 To a great extent 1. Developed my personal knowledge of how children learn to read. 2. Developed my ability to diagnose student needs in literacy. 3. Increased attempts to support parents in assisting their child s literacy development. 4. Developed my professional capacity to address students literacy difficulties. 5. Noticed improved student attitudes to literacy learning. 6. Noticed increased student achievement in literacy. 84

84 APPENDIX I: RLC Evaluation Questionnaire EDITH COWAN UNIVERSITY PRINCIPALS AS LITERACY LEADERS (PALL SA DECD) Regional Leadership Consultant Questionnaire You have recently participated in the Principals as Literacy Leaders (PALL) project, with funding provided by the South Australian Department for Education and Child Development. Edith Cowan University is evaluating the effectiveness of the project for principals and teachers. I invite you to complete the following questionnaire about your experience of the project. This is an anonymous questionnaire. Please do not write your name, or any other comments that will identify you or your school. Thank you for your time and consideration in reflecting on your experience of the PALL project. Yours sincerely Dr Deslea Konza Associate Professor of Language and Literacy Director Fogarty Learning Centre Edith Cowan University 85

85 Instructions Please complete the questions by ticking the appropriate box or inserting short answers (e.g. dot points) for open-ended questions. Background Information 1. The principals I supported were from schools in a. an Adeliade metropolitan area b. a regional area 2. I supported the following number of schools 1. Roles of the Regional Leadership Coordinator (RLC) I believe the following aspects of my role were useful in supporting the principals: 1. Clarifying aspects of the PALL project to improve principals understanding of the: a. Leadership for Literacy Learning Blueprint b. The Big Six c. Literacy Practices Guide d. Use of data e. Literacy interventions f. Evaluation of interventions 2. Organising professional learning (including readings). 1 Not at all 2 To a slight exte nt 3 To a mod erat 4 To a great extent 3. Providing support for the use of Disciplined Dialogue with staff. 4. What do you believe has been the single most useful aspect of the role of the RLC? 5. What aspect of the role of the RLC could be improved? 86

86 2. Use of the Literacy Practices Guide I believe the Literacy Practices Guide enabled principals to: 1 Not at all 2 To a slight extent 3 To a moderate extent 4 To a great extent 1. Recognise effective literacy practices in classrooms. 2. Support the set-up of classroom environments that facilitate student learning. 3. Encourage more explicit teaching of reading-related skills. 4. Promote professional learning to develop teacher knowledge about how students learn to read. 5. Promote a discussion about classroom teaching. 87

87 3. Project outcomes As a result of participating in the PALL project I believe principals have: 1 Not at all 2 To a slight extent 3 To a moderate extent 4 To a great extent 1. Learnt more about how children learn to read. 2. Promoted the Big Six model to provide a framework for the teaching of literacy. 3. Reviewed literacy assessment practices. 4. Promoted changes to the nature of reading interventions for struggling students. 5. Promoted more explicit teaching of reading strategies at classroom and individual levels. 6. Encouraged conversations with parents about student literacy development. 7. Engaged in focussed discussions (disciplined dialogue) with staff about students reading development. 8. Provided additional resources to support literacy teaching in classrooms. 9. Shared the leadership 10. Developed their professional capacity in leadership for learning. 11. What do you believe has been the most significant impact of the PALL project in the schools you have supported? 12. Can you provide an example? 88

88 APPENDICES J-P: Examples of School Intervention Reports Principals as Literacy Leaders Evaluation Report 2012 School: Palmer Primary School Principal: Lynne Noll Context Statement Palmer Primary School is a small rural town approximately 60kms from Adelaide. It is situated about 15kms from the River Murray, with its nearest regional centre is Murray Bridge. The school has 23 students from Year 1 to Year 6 (no year 7s in 2012). We have a JP and an UP class. It has been assigned a Level 3 Index of Disadvantage, this year after previously being assigned a Level 4). Many families live out of town, travelling in by bus from nearby farms or by car from the township of Mannum. The school s priorities in 2012 have been Literacy and Numeracy, Student Engagement, and Well being for Learning. These will continue to be priorities in 2013, with particular focus on Reading Comprehension. Issue for Palmer Primary School (2010) No whole school literacy approach exists. Each teacher coming into the school employs his/her own pedagogy and designs his/her own scope and sequence of literacy teaching. Data is collected as a whole school, but it is not interrogated through a whole school lens. Approximately 30% of students reading ages were more than 6 months below their chronological age. Approximately 25% of students spelling ages were more than 6 months below their chronological ages. These students have become our Wave 2 and Wave 3 students. Primary Purpose 1 To explore the impact of a whole-school approach to the teaching of literacy. Key Questions Can we impact more positively on student literacy learning by creating a whole school literacy statement that aligns literacy practice across the school? What pedagogies and structures do we need to include in our whole school plan? Primary Purpose 2 To explore ways of raising the reading levels of students who are in Wave 2 and 3 Interventions? Key Questions What structures can we implement to support these students? How can we make best use of our resources to support these programmes? How can we work better with the families of these students to support the development of reading skills? 89

89 Data Sources Data is collected from the following sources: Student achievement data Perception data sets Classroom Environment and demographic data Performance Management Meetings Staff meetings Process data sets Data Collection Methods Term One Week 2 Weeks 3 and 8 Week 7 Ongoing Each term Term Two Weeks 3 and 8 Ongoing Ongoing Term Three Weeks 3 and 8 Ongoing Ongoing Term Four Weeks 3 and 8 Week 6 Ongoing Ongoing Recount writing sample JP Letter/sound relationships (ongoing) Student Literacy Goal review Westwood Spelling Pat-R / Reading Progress Test Pat-Maths/ I can Maths JP Edu-check as required PM Benchmark Running Records to Level 30 UP Superspell diagnostic tests Student Work Folios Parent Feedback at organised 3 way interviews Staff surveys and self plotting on TfEL JP Letter/sound relationships (ongoing) Student Literacy Goal review PM Benchmark Running Records to Level 30 NAPLaN UP Superspell diagnostic tests Student/Staff Engagement surveys JP Letter/sound relationships (ongoing) Student Literacy Goal review PM Benchmark Running Records to Level 30 UP Superspell diagnostic tests Parent Feedback at organised 3 way interviews JP Letter/sound relationships (ongoing) Student Literacy Goal review Westwood Spelling Pat-R / Reading Progress Test Pat-Maths/ I can Maths PM Benchmark Running Records to Level 30 UP Superspell diagnostic tests Annual school driven Parent survey 90

90 Evaluation Overview As a result of our involvement with the P.A.L.L programme the following has been achieved: Staff Knowledge Increased teacher knowledge through T&D (topic examples, Giving meaningful feedback to students, Australian Curriculum (A.C.), Jolly Phonics) Increased awareness of the Literacy - General Capability in the A.C. More (and deeper) professional conversations at staff meetings More literacy specific language used in professional conversations More literacy specific information given in reports to parents Staff Practice Teachers/staff requesting specific literacy resources to support student learning and diagnostic data collection (eg. PM Benchmark materials from Levels 25 to 30, Probe Testing materials, Reading Box programme) Students being allocated levelled reading materials from Level instead of being able to choose unlevelled materials after level 25. Staff allocating levels to existing school reading resources, to make more structured use of them. Staff choosing T&D according to their personal learning needs. Staff using common processes/language for literacy learning More literacy charts/posters displayed in classrooms More explicit teaching of literacy skills ( The Big 6) Lots of sharing of ideas and resources Staff networking with teachers in other school to share practice. Teacher time targeted to Wave 2 and 3 students Authentic feedback being given to students Additional parent meetings for Wave 3 students Staff using Literacy Practices Guide for self review and professional development planning Wave 2 and 3 students receiving SSO or small group support (some with appropriately modified curriculum) Review of data collection practice with Wendy Featherston Data Analysis and Reporting Consultant) School wide changes Literacy structures included in whole school activities (eg. Spelling Demon Challenge at weekly assembly, Word of the Week added to Weekly bulletin) Emergence of future literacy priorities for Site Improvement Planning Timetabling blocks established and modified to maximize teacher/student interaction time Whole School Literacy Statement developed outlining school beliefs and practices Literacy statement used in Induction of new staff to ensure sustainability Data collection review Literacy Resources chosen to support the specific interests of Wave 2 and 3 students Weekly information about helping children with reading included in school newsletters (2011) 91

91 Student Learning Outcomes Over the 2011 and 2012 school years we were able to note an improvement in our Feb/March literacy data results. Approximately 30% of students had reading ages that were more than 6 months below their chronological age, in Mar In March 2102 this percentage had reduced to 25%. Approximately 25% of students had spelling ages that were more than 6 months below their chronological ages. In March 2012, this percentage had reduced to 20%. One of the Wave 2 students had increased their reading age from 1 year below C.A to 1 year 4 months above. One of the JP Wave 3 students, is now spelling 3 letter words, unaided, when previously, he was not recognising any of the letter sounds. Most of the improvement has been evident at the Wave 1 Level, with three students improving their reading age by more than 3 years in 1. Secondary Purpose Students borrowing more and accessing resources from local Community Library More enthusiasm (and parental interest) in being a part of the Premier s Reading Challenge Students (in Waves 2 and 3) wanting to bring work to the office to show other staff Use of the Leadership Profile for staff feedback to Principal Challenges The main challenge for Palmer Primary School has been permanency of teacher placement. None of the teachers who are here no (2012), were here when we agreed to be a part of the PALL project (2010). We have, each year, had to look at ensuring that what had been achieved and set up the previous year, could be maintained and built upon. We have been fortunate to have quality teachers in the classrooms, with a willingness to work with the school s structures. Despite not having been a part of the whole journey, our current staff are committed to improving the way we support our students with their literacy learning, and contribute greatly to the future literacy planning for our school. 92

92 Where to from here for Palmer? Palmer will look to use the on-line PAT-R testing in We will do this once per year (possibly midyear) as a means of annual data collection, whilst teachers will continue to use some of our present screening tools to measure effectiveness of teaching programmes. Our work with students has indicated that Reading Comprehension is an issue for our students and we will explore some of the Stephen Graham work in this area. We will also use the PROBE tests (in Terms 3 and 4 in 2012) and intend to use gathered data for planning in We will have planned T&D in the ACARA English for Terms 3 and 4 of 2012, ready for its implementation in 2013, along with T&D about implementing the Literacy Standards from the Quality, Improvement and Effectiveness Unit. Conclusion My involvement in PALL has been one of the most useful T&D exercises in which I have participated. It has been very useful to be able to engage in professional conversations at the Bordertown get-together. I have also found the regional cluster meetings to be encouraging and helpful and though these meetings, I have been able to establish professional networks, which are useful, not just in literacy, but in every area of our work as educators. Seeing the Regional Office staff attending the Bordertown meetings, reinforced how important rigorous literacy practice is, and how it is the business of everyone to ensure that we are offering the best possible chances for young learners. Having the expertise of Deslea and Neil has been invaluable and the comment from Deslea If it s the third day at school, then we are up to blending three letter words, was a big Aha moment for me a practical marker of how far we have come, from the literacy thinking of my early teaching days!!! Thank you to everyone involved, including Noel, whose work with us as individuals and in co-ordinating our cluster is very much appreciated. I would recommend PALL to all school leaders!! 93

93 CADELL PRIMARY SCHOOL Intervention Evaluation Report 1. Context: A brief description of the school context drawn from the PALL context statement. Cadell Primary School is located 180km from the Adelaide GPO in a small rural community. The school is situated about 2km from the township of Cadell and is very much the heart of the community. Enrolments are at 30 for 2012 with two full time classes and a third (middle Primary) group two mornings a week. Cadell Primary School has a strong focus on Literacy and Numeracy providing intervention programs for students. We have focused on Literacy and in particular writing over the last three years. We have instituted a writing process which is implemented in both classes and is part of our school approach to Literacy. This year our Site Improvement Plan has been focused on Reading Comprehension. Through targeted Training and Development, rigorous discussion around Professional Readings and observations there has been a change in pedagogy, lesson structure and explicit teaching of specific comprehension skills. I have participated in the PALL program and shared my learning, readings and tasks to help develop our data profile, and intervention plan. 2. Description of the problem : The evidence used (the reasons for adopting the intervention or taking a number of intervention actions for the target groups or group) and the nature of the intervention. We indentified that our students were good at decoding but their comprehension was low. This was supported by overall low PAT-R and NAPLaN results in After an observation to Courteney Gardens Primary School in Victoria in 2011, we were impressed with their Reading Comprehension results. We purchased three levels of Strategies To Achieve Reading Success (STARS) for teachers which had strategies for explicitly teaching the twelve Reading Comprehension skills. A focus on Blooms Questioning in all curriculum areas was implemented. Students on an ILP (Individual Learning Plan) were given extra support through MultiLit. 3. Purpose: The Primary and Secondary Purposes and the key questions which place the focus on the interrogation of the intent of the intervention and its effectiveness. Primary purpose: To ascertain whether explicitly teaching the twelve Reading Comprehension skills would have a positive impact on student understanding and whether to continue this long term. Secondary purpose: 94

94 4. Data collection methods: The selection of data collection methods and the presentation of the data relevant to the questions posed. Each of the twelve Reading Comprehension skills were taught using a variety of texts in line with the writing genre that was being taught at the time with reviews after three skills. Staff shared ideas and resources to value add to the STARS books. PAT-R was carried out in February and then again in August. The results are significant. The data shows that 50% of students improved and some significantly, 28% stayed the same and 22% fell. The data also indicates that in term 1, 50% were below stanine 5 but in term 3, only 28% were below stanine with 72% at or above stanine 5. Only one of the four students identified in the Literacy Intervention Plan did not make significant improvement in the PAT-R test. o Student A stanine 2 to 3 o Student B stanine 3 to 8 o Student C stanine 1 to 5 o Student D stanine 4 to 4 School review processes have indicated that the explicit teaching of Reading Comprehension is a positive experience for both staff and students. Students are able to verbalise the skills they are using, and staff are confident to use other sources of literature. A student survey indicated that 90% of students were more confident with comprehension, with comments such as: I understand more about what I m reading I am reading more I have been doing well in class and know more I ve gone up lots in my reading levels On how we think of things from a different point of view 95

95 5. Conclusion commendations and recommendations: future planning and action. Commendations Through the PALL program I have been able to share with staff the knowledge, resources and varieties of testing according to the Big Six. We have watched the videos, read some of the provided readings and the Literacy Secretariat Resource Papers. With this knowledge we have been able refine our Whole School Approach to Literacy, Intervention Plan, and our Data Plan. Teachers are using common language around the big six and wave concept. Most students made greater gains in reading levels (Lexile and Running Records) and Reading Comprehension (PAT-R) There is a whole school commitment to improving Reading Comprehension and not just of identified students Teaching interventions that are based on diagnostic assessments Teachers actively seeking research based practices the big six evident in teacher programs Recommendations Continue to maintain focus on common strategies to ensure they are fully embedded. Continue to develop wave 1, 2 and 3 intervention strategies. Continue and review the intervention plan and program. Continue the routine of teaching the twelve skills of Reading Comprehension with review weeks in between to determine which skills need to be revisited. Continue to encourage the students and all staff to use the Bloom s questions across the curriculum. Continue our Professional Readings discussion around good practice. We will not invest in any more STARs books as staff are confident about the skills to be covered and using their own more relevant resources. 96

96 P.A.L.L - INTERVENTION EVALUATION REPORT Elizabeth Downs Primary School, 2012 CONTEXT SCHOOL: Elizabeth Downs Primary School (EDPS), a Pre-school to Year 7, Category 1 Index of Disadvantage School, opened in Elizabeth Downs is a changing community with growing cultural diversity and complexity. This change is characterised by a growing cohort of students from non English speaking backgrounds (NESB) and increased transience of the student population. The enrolment of 237 students includes approximately 85% School Card holders, 17% students with disabilities, 7% Aboriginal students and 20% ESL students. As of 2012 the school started as an Intensive English Learning Centre (IELC) catering for refugee and new arrival students in 3 classes, R-7. There are 12 mainstream classes. A SSO is allocated to each class to support the Literacy programs being taught as part of the Literacy Block, five days a week. DESCRIPTION OF PROBLEM (Evidence used, reasons for adopting the intervention, nature of the intervention) Analysis of the NAPLaN data over the last few years has shown minimal growth in Reading, especially for year 3 students. Cohorts of students vary greatly and a high percentage starts their school life with limited language skills which impacts on all areas of their learning. Our Wave 3 students have been our main focus to date through small group work, targeted teaching, the development of a Phonics Awareness Screening test for data collection and directed support. As part of ongoing monitoring and analysis of teaching and learning, teachers select three targeted students from the top, middle and lower abilities. Testing data, strategies identified for teaching these students, scaffolding, and target setting are documented and used as the basis for differentiating learning for the whole class and as part of Professional Chats. These students are our Wave 1, 2 and 3 students. Aim/ Set Directions: The aim for the Catch-up Reading Wings Online Intervention Program is to target Wave 2 students (those who need supplementary instruction with the expectation that they will catch-up to the Wave 1 students) by providing focussed teaching around reading and comprehension matched to student needs. The intervention program will aim to move students closer to the regional reading standards, raise expectations for student achievement and include strategies relating to the Big 6. Focus of the Intervention: Focussing on year levels 3 to 5: three classes Yr 3/4 to Yr 4/5. Initial target group: 3 x groups of 4 Groups meet 3 times a week for 30min. 1 term (10 weeks) for intervention program to run. Criteria for selecting students: Students identified as Wave 2; working in the middle ability group(s) in the class. Selection will be based on student running record data and in consultation with the class teacher. Initially aim to have selected students at similar reading levels or within the same range. E.g. all between levels

97 Intervention Program: Use of program to select texts and use linked activities. Students work on one text each week to build knowledge of story/ context for comprehension and fluency, as well as develop word recognition and language comprehension. Comprehension skills developed: literal comprehension, inferential comprehension, visual/ creative comprehension, plot sequence, vocabulary/ sentence building, word building We continued the intervention program into term 3 and 4 as growth in student reading levels, according to Running Record data, was significant for most students accessing this program as well as teachers noting an improvement in confidence and willingness to read independently. We have also been developing a whole school approach to guided reading which is structured, explicit and consistent across the school. Strategies being explicitly taught in classes and the language teachers are using are also being modelled in the intervention sessions, reinforcing student learning and applying skills outside their classroom experiences. PURPOSE & DATA COLLECTION METHODS (Primary & Secondary purposes and the key questions which place the focus on the interrogation of the intent and effectiveness of the intervention) Primary purpose: To find out about changes in the teaching and learning experiences in which children are engaging and their effects in reading. KEY QUESTIONS DATA SOURCES DATA METHODS What reading strategies are students employing when reading? What are the student s confidence and engagement levels in reading? Are students developing a love of reading and independence? Are there ways we can link comprehension and still maintain an enjoyment of reading? Is differentiation employed to tailor teaching and learning in reading to student s needs. Class teachers - testing data and anecdotal comments Deputy and Principal Early Years Reading Focus Teacher (EYRF) teacher leading the whole school approach to guided reading SSO coordinating the intervention. Running Record data Discussions and observations Research by EYRF Daily quizzes associated with the online reading program. 98

98 Secondary purpose: To ascertain if there are any changes being seen in children s achievement I reading. KEY QUESTIONS DATA SOURCES DATA METHODS How can we move students forward in their ability to read? Can we improve student engagement with their reading which should lead to improved confidence and independence? How can we ensure class programs and intervention can provide continuous improvement? Can we develop skills and see application in classroom work? Class teachers - testing data and anecdotal comments Deputy and Principal Early Years Reading Focus Teacher (EYRF) teacher leading the whole school approach to guided reading SSO coordinating the intervention. Students participating in the program Running Record data Discussions and observations Research by EYRF Daily quizzes associated with the online reading program. Comments from students reflecting on their skills, reading levels and reasons for participating n the Catch-up Reading intervention (see below). SSO setting targets with the students for the term. Comments from the students participating in Wings Online I feel brave to read in front of others now. Has helped me to concentrate. Helped me with my spelling and sight words. Learn new words. Helped me understand. Helped my reading and learning new words. Learn how to be a better reader Help me read. Comments from the SSO co-ordinating the Wings Online Program Over the past term I have been running an intervention program with a group of students around reading; working with children that are around level 19/20. It is an online program -Wings Online; making learning to read easy, fun by combining books with online reading and activities. The aim is to improve reading and comprehension levels for those students who are just below their expected year level standard. Just by giving students the opportunity to try a different medium has given them the confidence to read with enthusiasm and commitment. We all set goals right from the beginning around reading levels and all students wanting to achieve high goals. We celebrate achievements on a daily and weekly basis. We highlight daily, those whom achieve perfect scores and weekly achievement to those who attend all sessions by sticker chart. We also congratulate students that have nearly made it to a perfect score encouraging them to want to do better. I do not need to ask them now to do the quiz again, they ask me Can I do the quiz again to try to get a better score? In the first session we select a text that we will work through over the 3 sessions in the week. All students are on the same text. Session 1: Students are self guided; they listen to the text, read the text and complete the quiz. 99

99 Session 2: We re-visit the text, reading the text together, taking in turns and reading out loud. Students are guided through the text talking about meaning, structure and visual components and then complete the quiz again. This session is also a great one to work on fluency, as we can all hear each other. All records of quiz activities are printed out and kept in file so we can keep track of student progress. Session 3: Students can read or listen to the text, do the quiz and or we complete an extension activity. Running Record Data for students participating in the Catch-up Reading Program: Term 2, STUDENT BEGINNING OF TERM 2 END OF TERM 2 MB BI SJ JK 18 IP BN CS SW KW-E 22 LG SK JT AG TG Running Record Data for students participating in the Catch-up Reading Program: Term 3, STUDENT STARTED TERM RUNNING RECORD TERM 2 RUNNING RECORD END TERM 2 RUNNING RECORD END TERM 3 TERMS LEVELS MB BI

100 SJ JK IP BN CS SW KW-E LG SK JT AG P A S R J S T B CONCLUSIONS (Commendations & recommendations, future planning) The Catch-up Reading Intervention Program has been very successful and there have been significant gains made in student reading levels and noticeable improvements in confidence and attitude towards reading, as noted by classroom teachers. Students are excited about sessions and independently remember to attend lessons, often reminding their class teacher it is time to go! Every session students monitor their own progress and every success is celebrated by completing the quizzes and extension activities set by the SSO. It is a very positive atmosphere in these sessions which encourage the students to keep striving for their best. We 101

101 have found that having small achievable targets keep the students on track and wanting to keep improving. They set their own targets at the beginning of the term and the SSO keeps referring to them and together they monitor their progress to achieving them. It is important to ensure students are aware of success, to actively engage them in their learning and offer reinforcement of their success which will encourage positive habits and give a sense of achievement. We have further developed the level of intervention being provided by: Increasing the number of students attending from 4 students per group, 3 times a week to 8/9 students per group twice a week. Purchasing 10 laptops and headphones to support the program and setting up a larger and more permanent space for the groups to work. Purchasing the subscription for Raz-Kids another online program to extend the number of books and activities we can provide and also encourage the work to be followed up on-line at home.. Allocated a room for the reading intervention to happen so students have a quiet space to work and can be set up as an attractive teaching space. The SSO leading the intervention has put in a lot of time and effort to develop it and make it a positive experience for the students. Things put in place are: Developing the teaching space arrangement of furniture for laptops, displays, resources. Linking with the whole school guided reading focus by using the language, putting up strategy posters (as used by the class teachers) for students to refer to and models when reading together. Developing personal skills and understanding of pedagogy and teaching strategies by working with the Principal, Deputy and Early Years Reading Focus Teacher. Developing target setting and reward strategies help students monitor and celebrate success. Based on the success experienced in the second half of the year we are looking to further resource and expand this reading intervention strategy into In consultation with the SSO and class teachers we will devise further programs to build on student improvement in line both the school s Site improvement Plan and our reading comprehension strategies. In conjunction with the learning gained from the PALL course and the collaborative leadership of a SSO our site has been able to incorporate an intervention program that does engage and drive reading improvement for targeted students at a pace which does help quickly close the gap for these students. We will build on this momentum gained into

102 PALL REPORT November 2012 David Loveband Principal Norwood Primary School CONTEXT: Norwood Primary is a Category 7 school with a current cohort of 365 students. The school opened in 1877 and as such, is proud of its heritage. The school enjoys close links to the local community. Parents are strong supporters of the school and volunteer across a broad range of governance, sport and curriculum areas. There is an ongoing arts focus with performing arts, music tuition and graphic arts being highlighted. The school population reflects the cultural and social diversity of the community with about thirty cultures represented. More than 35% of our students come from a non English speaking background. Greek, Italian and students from Asian backgrounds predominate. Of late there has been a growth in student enrolments from the Sub Continent. R-2 students are provided with LOTE Greek and Yr. 3-7 students experience LOTE Italian. Mother tongue is provided for Chinese, Greek and Italian students. The school has a 0.5 ESL teacher. The school has a Leadership Team comprising the Principal and a Senior Leader. Team leader reps from 4 Learning Teams have regular input to the Leadership Team. Teachers collaborate very successfully in Learning Teams at the Early Years, Primary Years and Middle School levels as well as a Language/ESL Learning Team. Well trained and committed SSO s provide strong learning support programs to students. Twenty percent of our students are School Card holders. SIP priorities include excellence in learning pedagogy and achievement, manifested through an inquiry approach. Improved learning outcomes in Literacy, Numeracy and Science have the main focus. Improved learner inclusion is a focus of student Health and Well being. ICT is an overarching area, influencing all SIP priorities. Support for students (ESL, Mother Tongue, Coordination, intervention programs and special needs) is provided through a range of intervention programs. Considerable resources (Finances and HR) are allocated to this support program, which is overseen by the Student Support Team. This is comprised of representative groups within the school. Significant school data is collected & analysed to identify students (approximately 28% of the student population) to be provided with additional learning support. 33% Non English Speaking background and 30% English as a second language School is zoned 28% of students involved in specific intervention/student support program 7 Aboriginal students. 6 students with disability Description of the issues Through a high profile focus on Literacy, outlined in the school Site Improvement Plan, staff have consolidated and continued to develop pedagogical approaches to the successful teaching and learning of literacy skills across the R-7 context. This has been in response to various professional development initiatives, research findings specific to the testing of Literacy skills and a more 103

103 professional approach to data collection and analysis in order to drive targeted student intervention programs. Close alignment to the Eastern Adelaide Regional targets for Literacy and efficient Curriculum Committee and year level Learning Team foci have combined to have positive impacts regarding the support of student learning. In the early years particularly, a strong commitment to literacy learning along with effective teaching methodologies exists. Teachers strive for consistency of practice. Jolly Phonics has proved to be a very successful vehicle to help bring consistency to the extent that an R-2 spelling program is now underway (modified & improved in 2012) that is directed at specific stages of literacy learning. Student placement into specific spelling classes is based on results from PAA testing. Focus is on: Initial Sounds, Blending, Segmentation, Syllables etc. Learning Teams work closely together to develop shared practices around literacy. Processes include analysing data, developing shared agreements and understandings, developing site based targets in literacy developed in 2011, based on a regime of diagnostic testing, NAPLAN results, Running Records, Regional Targets and Curriculum Committee decisions. Common understandings and practices in the area of literacy have been established through very successful engagement in the regional Stephen Graham coordinated i-lit program and implemented across the site. Focus on Learning Focus 1. Legitimate testing regimes would provide data indicating specific areas of literacy learning that could be supported by classroom teaching, specific literacy programs and targeted/focussed intervention programs across R-3 classes. The Consolidation of the Early Intervention program to support the development of phonological awareness in early years students, thus presenting a strong base in literacy skills. Eight classes targeted. Using student data to plan differentiated learning and assessment regimes to further challenge students. Consolidate effective role of Student Support team in its role of managing and monitoring student literacy improvement outcomes as well as developing consistent expectations of student achievement across the school Focus 2. All staff are on the same page re whole school literacy approach and previous, as well as ongoing regional i-lit Professional Development is consolidated and enriched. Consistent teaching, learning and assessment strategies throughout mainstream classes. Data collection linked to East Adelaide Regional emergent literacy targets and instructional reading level targets. Strong commitment to high quality literacy teaching across all R-3 classes. Committed Learning Team commitment to Big 6, Running Records, Oral description skills, Jolly Phonics and graded spelling program including 4-5 letter sounds per week to new Reception students. 104

104 Roles and Responsibilities; Reception/Year 1 class teacher; Alex Paluszek: Early Years i-lit coordinator. Manages and coordinates Early Years testing regimes and data analysis. Directs Wave 2 & Wave 3 support program and hence manages SSO student intervention programs. Supports staff with pedagogical development via i-lit program. Reception/Year 1 class teacher; Di Harrington: Reading Support Teacher. Shared role with Alex to coordinate Early Years Literacy program and intervention support. Data collection linked to Regional emergent literacy targets and instructional reading level targets. Include the management of phonological skill development and focus on enhancement of oral description skills. Focus 1: Data Collection procedures As a school we have tried to consolidate and regularly apply effective diagnostic testing in order to better target support for students who require intervention A solid testing regime would enable us to do this. Reliable, suitable and effective Diagnostic tests have been an issue for year 2-7. Through a combination of information via the Eastern Adelaide Region i-lit program and the PALL program, new tests were added specifically to the early years and in particular Reception and year 1 students. These were introduced in term In week 7 & 8 Term , diagnostic testing regime across all Primary and Middle School classes including year 2 students. Data transferred to all new class rolls week so teachers have a clear and recent indication of student capabilities at the end of the previous year. For early years classes, teachers consolidate decisions around student placement into Wave 1, 2 or 3 across first 2-3 weeks of term 1. Decisions are supported by class work, early test results and anecdotal summations. Across week 1 & 2: All new Receptions are checked for their understandings of the 25 Concepts of Print (CAP). This provides a basis for all teachers to begin their reading programme for their students and these skills are specifically taught through teacher s guided reading programmes. Teachers used the SPA test on Reception students who did not seem to be progressing as quickly through the initial sounds (Jolly Phonics programme), offered in the combined spelling program. All Year 1 students have their Phonological skills assessed using the PAA test. If they score a 3 or below, they are incorporated into our Intervention programme and work on the specific area they are having difficulty with. Identified at risk Year 2 students also tested with PAA. Base levels established by Early Year s staff and Wave 1, 2 & 3 students identified (and areas of support), using data sets from PAA. The data sets for the PAA test are included as an attachment. The data became the most relevant component over the year that allowed us to fully determine student progress, assessment and thus learning outcomes. The test has a total score of 100 and is broken into 5 major sections. Each section has sub-groups with marks out of 5 given. Any student who received a mark of 3 or below in any sub-group were then identified and grouped with similar students in small groups for explicit intervention. A 5 week program was developed specifically focusing on developing the area required where explicit instruction and engagement occurred. At the end of the 5 weeks students were then reassessed for 105

105 their level of understanding and if they scored 4 or 5 out of 5 they were released from the program. These sessions were run by SSO s, and it is important to acknowledge their role in the success of this program. It must be recognised that one of our SSO s is a qualified teacher whose understanding of student development ensured the implementation of a quality support program. In all, 41 Year 1 students were assessed at the beginning of the year, with 34 students identified as requiring extra assistance. Some students only required intervention in 1 sub-group area, while others were identified as requiring intervention in multiple areas or across all assessed areas. Of the 34 students identified, all required further assistance in the 3 areas of sound manipulation. Due to the large numbers, all teachers in the Early Years decided to make this a focus within their classroom programs (wave 1 approach). The intervention program worked so successfully that by the end of term 2 28 children had been released from the program. This allowed time to be allocated to year 2 students who were struggling with letter formation and the structure of descriptive writing. Reception teachers nominated students they knew required extra assistance, with 4 students identified as lacking basic phonological awareness skills. The success of the testing regime and the specific clarity of the data obtained has clearly set a precedent for the ongoing monitoring and support of students challenged by phonological awareness issues. Coupled with the explicit teaching methodology, we feel sure that students learning needs will be supported. Focus 2: Securing whole school literacy approach Linked to Focus 1, has been a second tier of literacy support utilising the role of a Reading Support teacher. Two teachers working in tandem with a strong literacy focus have impacted of R-3 literacy teaching and learning consolidated through the positive support and interest of 8 teachers and 4 SSO s. Classes have worked through oral and written recounts, oral and written descriptions, oral and written information reports and oral and written procedures. Using the Stephen Graham s assessment models, staff chose 9 target children to assess in oral language. Explicit teaching around oral description via the assessment of 3 struggling readers, 3 middle readers and 3 top readers occurred. The whole class became a wave 1 level in this sense with explicit teaching of oral description. This was followed up with reassessment of the 9 students after the explicit teaching. Comparative data collected and analysed to ascertain if explicit teaching activity was effective. In almost all classes this proved to be the case. To support our ESL programme and further our understanding of the ESL Scope and Scales, staff also undertook recording unscaffolded oral descriptions from all their Reception & Year 1 ESL children and written descriptions from all ESL Year 2s and Year 3s.These were used for scaling and were included in the Census for ESL funding. After a fortnight of explicitly teaching oral descriptions, staff then reassessed their target children s oral description skills. Considering our generally articulate and highly literate clientele, staff were surprised at many children experiencing difficulty using descriptive language skills prior to the explicit teaching. 106

106 There has been a professional expectation that teachers would put into practice the new learning offered to them. This has happened on a variety of levels, some choosing to work collaboratively to support each other and combine it with their peer triangulated feedback as part of their commitment to TfEL. Resources have been made and shared to support the teaching of these genres. Discussions at team meetings and specific reading focus release days have shown that staff have been very open to new learning and are excited by their students high level of engagement and progress. Information shared from our regular PAA testing alerted teachers to the need to concentrate on sound manipulation which was a common problem area for all children tested. It was decided that staff needed to address this themselves in their own classes to improve overall scores and hence, as mentioned, a Wave 1 approach was adopted for all year 1 and 2 students. Early in the year all staff were required to align their students into waves on a literacy needs basis. Teachers then met with senior staff, ESL teacher and appropriate SSOs to plan and describe the differentiated curriculum that will be offered to support that student in their learning. This is reviewed each term. Teachers were able to use data from the intervention testing, oral skills assessment, running records as well as their own observations to identify short term goals and activity ideas to direct SSO support time. The Reading Support teacher, working with the principal, was able to provided staff with up-to-date readings on literacy learning and to lead professional discussions at team meetings regarding assessing reading levels, reading comprehension skills, home reading and importance of oral language skills. Initially all R-3 teachers were involved in a reading audit to share amongst peers the variety of stimulating practices offered in all classrooms to support the development of reading. Team time has also been used to level home reading books which were donated to the school; purchase new home reading material & teacher resources, IWB books and paper big books. Early Years staff collaborate to produce weekly talking topics to direct their daily oral news sharing. All new Receptions are checked for their understandings of the 25 Concepts of Print. This provides a basis for all teachers to begin their reading programme for their students and these skills are specifically taught through teacher s guided reading programmes. Teachers were provided time to maintain up-to-date running records data with a particular focus on comprehension. Teachers have been working on the 3 main areas of comprehension questioning literal, inferential and response questions. Four mornings a week, the 6 early years classes divide the children into ability groups to access a spelling programme based around Jolly Phonics and Jolly Grammar. This has proved very successful in targeting student s sounding abilities and teaching spelling rules, with a particular emphasis on aural and oral sounding skills. On Wednesdays, all children stay in their own classes to allow classroom teachers to gauge individual progress. This has complimented the reading programme within the school. Staff are constantly sharing teaching ideas, student skill acquisition and assessments and see the groups as a continuum, allowing movement of students every term between the groups where applicable. Teacher professional development has included four members of staff completing Running Record training with Stephen Graham; one member being part of i-lit throughout 2012 and all R-3 teachers going to Guided Reading training run by Margaret Menner in November. Staff acknowledge the huge impact the SSOs have on supporting teachers and students in their literacy learning. They have a vital role in running our Intervention programme 3 mornings a week and also supporting our Jolly Phonics spelling programme. They have also been crucial in gathering data every 5 weeks to direct our Intervention programme and also provide baseline data on new Receptions students. All SSOs willingly meet with teachers to plan and give feedback on specific programmes. Each 107

107 term SSO s review student support programs as part of the Student Support team role. The team meets with teachers and SSO s to monitor the intervention program. Students Through PALL awareness, areas of focus this year have been: Oral language skills Developing the appropriate language and structure of genres Fostering independence in students borrowing own take home books and library books Students as literacy leaders speaking and reading at Assemblies; weekly sharing topics; oral presentations Increased SSO support to allow for consistent and regular targeted intervention Wave teaching group planning (Principal, Senior Leader, ESL teacher, classroom teacher & SSOs) to provide differentiated curriculum. Our data is clearly showing that our students Are borrowing a greater number of library books through a more flexible library borrowing procedure. 100% of our students completed the Premier s Reading Challenge PAA testing demonstrated 82% of our Year 1 students are now scoring % on retests 28 out of 34 Year 1 Students have successfully graduated from the Intervention programme. This has now allowed us to pick up 4 older Receptions and 8 Year 2students who have scored poorly on the PAA test and written genre assessments Running Records data collected in term 3 met with district benchmarks and showed an improvement from last year s results, however staff still felt that because data was collected at 90% accuracy rate that a focus within classes was still needed to aim for better fluency, phrasing and comprehension % of Reception Students 69% of Year 1s 71% of Year 2s % of Reception Students 79% of Year 1 s 86% of Year 2 s (50% for region) (70% for region) (80% for region) Are having great success with Jolly phonics spelling and sounding philosophy. Have increased confidence in writing and speaking using genre structures Teachers Over 2011 & 2012, PALL has meant that a far greater emphasis and intensity has been placed on the professional development of teachers particularly at the Early Years (R-3). This focus has been both deliberate and intended as there is a very important requirement that we get literacy right very early in students learning experiences. With a philosophy of self determination, independent and proactive learning opportunities backed with high quality, functional, practical and very relevant professional development, then the recipe is there for exciting, collaborative teaching and learning. The conduit for this is the provision of opportunities for quality learning team planning time for colleagues to meet, collaborate and hone skills. With the provision of relevant, specific detailed data reflecting student learning needs and teacher pedagogical effectiveness, then the collaboration and collegiality is enhanced more. Through PALL, many of the conduits for data success were provided. The regime of testing that I brought to the early years group through PALL sessions and resources provided much stimulus, so to the nurturing of individuals in regard to the vision that I was able to share with colleagues re the improvements we could bring about in Early years literacy enhancement. The new focus is to ensure successful early years literacy learning` crosses the threshold into the Primary and Middle school 108

108 domains. This will be a 2013 focus but already there are many elements that these teachers are incorporating as a result of shared Early Years excitement gleaned from Staff meeting workshops and shared teaching experiences. There is no doubt that PALL and a combination of insightful Early Years teachers who have been encouraged to take opportunities via i-lit and Reading Support Teacher roles, have allowed us, as a staff, to work towards improving the literacy, reading and comprehension skills of our students. Staff have had more opportunities to work collaboratively to plan, teach and assess literacy teaching using the data collected to inform practice. While wanting to maintain Jolly Phonics spelling, Intervention and guided reading practices, the aim now is to improve in the areas of * student comprehension (NAPLAN results) and *improving parents understanding of the reading process via parent workshops/regular newsletter articles in In reflecting back, the original 2011 Blue Print strategies defined our focus: Accurate, specific, essential data sets that allow us to address specific literacy learning areas A developing whole school literacy curriculum that has grown out of incredible valued dialogue from all stake holders working collegiately in highly focussed Professional Learning Teams The development of a legitimate student intervention program, managed via a student support team functioning to clearly support student literacy learning. David Loveband Principal. Norwood Primary School. November

109 CONTEXT STATEMENT E ELIZABETH GROVE PRIMARY SCHOOL Principal: Moya Wellman lizabeth Grove Primary School is part of the Elizabeth Grove Community Campus, consisting of Elizabeth Grove Primary School, Elizabeth Grove Out of School Hours Care, Elizabeth Grove Children s Centre, Kids n You Family Services and Good Beginnings Australia (TAP). We also have a campus canteen, an internet café and a second hand shop. In 2011 Elizabeth Grove was reclassified as a category one school under the index of disadvantage. The Campus is committed to the wellbeing, health, care and education of all children, families, staff and community members. We value, respect and promote a sense of ownership, collaboration and participation in all community endeavours, which will maximise positive learning outcomes for all people accessing the services. Characteristics of student population At the end of 2011 there were 285 students (208 families) at Elizabeth Grove Primary School School DECS English as a second Language (ESL) 23.6% 14.2% Disability 15.2% 9.1% School card 80.0% 27.8% Aboriginal/ Torres Strait (ATSI) 14.1% 5.2% Non English Speaking Background (NESB) 12.3% 10.7% Transience is high. 110 students came into the school and 80 students left during the year. This did not include receptions starting school or year sevens leaving. INTERVENTION PROBLEM We had been monitoring spelling using mainly Waddington, Holborn and Westwood diagnostic testing and to some extent NaPLAN. Teachers also used other means usually checklists around phonics. Staff agreed that spelling was an issue so the challenge was taken to staff. We worked to develop a whole school spelling program and published a scope and sequence in spelling. 110

110 CONSIDERATIONS Even though we were working towards a consistent approach published in a spelling scope and sequence, some children needed more intense intervention to make progress.. At the same time as this was happening we were thinking about changing our support practices for our students with difficulties. Some students have just not had the opportunity to learn because of transience and trauma while others have well documented learning difficulties/disabilities. In our research about what works the Special Ed teacher had come across a programme for Primary aged children called MULTILIT (Making Up for Lost Time in Literacy), so we had several staff trained and implemented the programme. The data (reading levels, sight words, reading ages) was showing success so in 2012 we have implemented MINILIT (Meeting Initial Needs in Literacy) for JP students. PURPOSE To improve student learning outcomes, particularly in the area of spelling. The following is an overview of our intentions in setting up our intervention program. Teachers must have training in the skills necessary to provide a differentiated curriculum to meet the needs of their students. Data will be collected and analysed to identify these students A team is set up (Intervention Team) to review and oversee the process for referral and delivery of programs Regular evaluation and monitoring of students progress will be done by the Intervention team and class teachers YEARS 1 and 2 Selection criteria for students According to whole school data, students who had wide gaps in literacy or low reading levels were referred by classroom teacher. Intervention for these students is referred to as WAVE 3 intervention. WAVE 3 The following strategies were implemented for these students Phonological Awareness test was administered by the Speech Pathologist assisted by an SSO, to identify Year 1 and 2 students early in Term 1. Individual programs were drawn up to meet the students needs. students accessed Phonological Awareness WAVE 3 intervention 4-5 times a week for 15 minute blocks.intervention meetings with classroom teacher, SSO, Special Education teacher and a member of Leadership occurred at least once a term to review progress. WAVE 2 This intervention is slightly less intense than WAVE 3. These students were identified as having significant gaps in their learning. Some students who have finished the WAVE 3 intervention also went into WAVE 2. Intervention for Years 1 and 2 was provided in small groups by two teachers The focus was alphabet knowledge, decoding letters into words, blending and reading. 111

111 YEARS 3-7 Prior to the selection of Primary aged students for intervention, two staff members were trained in the MULTILIT (Making Up Lost Time in Literacy) program in November 2010.Resources were made so the program could be up and running at the start of MULTILIT is suitable for struggling readers in Years 2-7 who struggle to decode on syllable words. The lessons are explicit, structured and systematic in approach, teaching phonics and sight words This program made up the WAVE 3 component of our intervention. Selection criteria for students in Years 3 7 The data used for identifying students for the 2011 MULTILIT program was students who were significantly below their appropriate age level according to the Westwood spelling test results. WAVE 3 Strategies used WAVE 2 Students were tested on the MULTILIT placement test to determine eligibility. Students who failed the test were placed in Wave 3 intervention, with individual sessions 4-5 days a week, working on strategies for sounding and blending words students worked through the MULTILIT lessons then sat a post test to determine their level of understanding of the decoding process After completion of WAVE 3 a consultation process with the classroom teacher is held to decide whether a student returns to the class or accesses WAVE 2 intervention for a short time. If a student does not pass the placement test, further 1:1 or small group support is offered using Reading Freedom, Phonics First or Reading Doctor Programs, recommended by the Specific Learning Difficulties Association (SPELD). Selection criteria for students for WAVE 2 Students significantly behind their expected level based on their Westwood spelling test results. Strategies used Students were removed from class in groups of up to six, for 2-3 lessons a week focussing on spelling, reading and comprehension. Students in Year 6/7 who completed MULTILIT but based on teacher judgement required further support with reading worked with an SSO on the Reading Freedom computer program Intervention meetings with classroom teacher, SSO, Special Education Teacher and a member of Leadership occurred at least once a term. At these meetings, data was reviewed, and it was decided which students came off intervention and returned to class and which students i.e. new enrolments, would undergo placement testing. Reporting A letter is sent to parents informing them that their child is part of the intervention programme. Reports are written for parents at the end of the year. Review of intervention and strategies used for children was undertaken regularly. Children who fail to respond and move forward in their learning will be referred to the Student Review Team and appropriate Regional Personnel. 112

112 Outcomes and Recommendations for the Future 54 students identified as requiring WAVE 2 support in 2011 worked with the Intervention teacher. 55 students completed MULTILIT and passed the placement test Most students who completed and passed the WAVE 3 MULTILIT placement test were recommended for at least one term of WAVE 2 Intervention. 14 students were still working on MULTILIT at the end of 2011 and will continue WAVE 3 literacy intervention at the start of 2012 Some Aboriginal students received extra small group and 1:1 support through the Aboriginal Programs Assistance Scheme The intervention programme continued in MINILIT, (Meeting Initial Needs in Literacy) the Junior Primary version of MULTILIT was introduced in the school in Training and Development to support the intervention program MULTILIT and MINILT training Phonemic Awareness Training Oral Language Reading /spelling programs (SPELD) INTERVENTION at ELIZABETH GROVE SCHOOL 2012 for Year 1-7 students 2012 Year 1 2/3 MiniLit placement test was administered to students identified by the classroom teacher in December 2011 by SSO who had been trained in delivering the MiniLit program groups of 4 students were assigned an SSO/teacher to work on this intervention program which is made up of 80 lessons to address alphabet names and sounds, blending of sounds into words, digraphs, handwriting and reading 22 Year 1/2 students, plus 5 Year 3 students to access MiniLit intervention 4 times a week this Wave 2 program to occur throughout the day with students taken out of the classroom for 45 minute lessons 3 or 4 SSOs/teachers will deliver the MiniLit program Intervention meetings with classroom teacher, SSO, Special Education teacher and a member of Leadership will occur at least once a term new groups to be added during the year as students complete the MiniLit program and new students placement tested 2012 Year 3 7 The progress of students who received Wave 2 and Wave 3 Intervention support in 2011 was reviewed at an Intervention meeting at the end of Term 4 with classroom teacher, SSO and Special Education teacher. Some students moved from Wave 2 to Wave 1 classroom learning and it was recommended that some students still struggling with decoding skills to remain on Wave 2 intervention for another term. Most students who completed and passed the Wave 3 MULTILIT placement test were recommended for at least one term of Wave 2 Intervention in new Intervention teacher employed fulltime 35 students identified as requiring Wave 2 support in Students will be removed from class in groups of up to six, for 3 lessons a week focussing on spelling, reading and comprehension 113

113 21 students to continue with the MULTILIT Wave 3 literacy intervention with SSO, ACEO or Special Education teacher. They were re tested on placement test and started MULTILIT where the first errors occurred. At the completion of MULTILIT, students sit a placement test to determine their level of understanding the decoding process. If they pass there is a consultation process with the classroom teacher to decide whether a student returns to the class or accesses Wave 2 intervention for a short time. If a student does not pass the placement test, further 1:1 or small group support is offered using Reading Freedom, Phonics First, Literacy Planet, Reading Doctor or Ipad programs, recommended by SPELD. Intervention meetings with classroom teacher, SSO, Special Education teacher and a member of Leadership will occur at least once a term at these meetings, where data will be reviewed, it will be decided which students will come off intervention and return to class and which students i.e. new enrolments, will undergo placement testing, if identified according to data. The following data collected so far this year is showing pleasing results Group one Year level Chronological Age Spelling Age Feb Spelling Age July student 1 4 9y8m 9 y 10y3m Student y8m Multilit 8y8m Student y1m 8 y 5 m 9y7m Student y10m 7 y10 m 9y Student y10m 6y 7 m 9y3m Student 6 4 9y9m 9y 1 m 9y10m Student 7 5 9y9m 6 y 9 m 8y3m Student y7m 7 ys 4 m 8y5m Student 9 4 9y9m 6 y 6 m 7y7m Group Two Student y6m 9y10m 9y10m Student y5m 6y 6y4m Student y11m 6y5m 6y8m Student y6m 7y1m 8y9m Student y1m 10y1m Student 6 5 9y7m 8y3m 9y Student y2m 9y5m Group Three Student y10m 7y2m 8y7m Student y 8y7m 114

114 Student y9m 9y1m 9y5m Student y1m 12y3m Student y4m 9y5m 10y2m Student y 8y8m 10y1m Student y9m 9y5m 9y8m Student y 9y3m 11y Student y5m 10y3m 11y Student y7m 8y11m 9y5m Year level Chronological Age Spelling Age Feb Spelling Age July Group Four Student y4m 7y1m 7y1m Student y10m 10y7m 11y1m Student y3m 8y8m 9y11m Student y4m 8y5m 10y6m Student y6m 9y5m 10y1m Student 6 11y5m 8y8m Student y11m 9y9m Group one Core word list end of 2011 Core word list July read spell read Student Student 2 no data available no data available 674 Student Student Student Student Student Student Student Group Two Student no data available

115 Student 2 no data available no data available 82 Student Student Student 5 no data available no data available 717 Student completed Student 7 no data available no data available 359 Group Three Student 1t Student 2t Student Student 4 no data available no data available 714 Student 5 Student completed Student 7 no data available no data available 358 Student 8 Student Student Core word list end of 2011 Core word list July Group Four read spell read Student completed 353 completed Student completed Student completed 445 completed Student completed 116

116 Intervention Evaluation Report October 2012 School: Renmark Junior Primary School Principal: Trevor Broughton 1. Context: A brief description of the school context drawn from the PALL context statement. Renmark Junior Primary School is located next to the Renmark Primary School and they are currently in the process of amalgamating from Amalgamation has been on the agenda for a number of years and the uncertainty with regard to this has had a significant impact on the direction of the school with three different Principals over the last three years. The school has had an intense focus on Literacy for the last 6 years, with Guided Reading as their main focus. Our key focus over the last two years is to develop a learning framework that maps a genre sequence and to focus on the development of explicit teaching of comprehension skills as outlined in our Site Improvement Plan. Staff have worked together in PLC s to review and develop whole of school literacy and common language agreements. The Reading Support Teacher has worked closely with the leadership team to facilitate this process. We have participated in the PALLS program and used our learning to provide instructional leadership for our Professional Learning Communities, reflect on our practices and guide our future development. We have worked with our Regional Leadership Consultant who has provided ongoing support between units and has helped us reflect on practice and provide challenges for us. 2. Description of the problem : The evidence used (the reasons for adopting the intervention or taking a number of intervention actions for the target groups or group) and the nature of the intervention. NAPLAN data for year 3 students indicates that over 54% of our students are in the bottom three bands for reading while our Running Records data indicates that many (70%) of our students are leaving year two with the ability to read at level 26 or above. A detailed review of the 2011 NAPLAN data indicates that while our students have the ability to read at the required level they are not able to effectively understand what they are reading. This has required us to review the nature of our intervention and look closely at which students are targeted for intervention and how the intervention is deployed in order for students to receive the maximum benefit. Our intervention included staff professional development, reflection and analysis of data and the ongoing review of student progress to ensure that the intervention is effective. Extra support has been provided for the students in Wave 2 and Wave

117 3. Purpose: The Primary and Secondary Purposes and the key questions which place the focus on the interrogation of the intent of the intervention and its effectiveness. Primary purpose: To review the processes involved in Guided Reading and to ascertain the effectiveness of the program in providing the instructional content required to improve comprehension skills for students and to provide teachers with the knowledge and skills to effectively teach comprehension skills to students. What are we teaching students? How are we teaching students? Are ALL students engaged with their learning, or is it only the students working in the teacher group? Are we providing students with the skills to understand what they are reading? How can we measure their level of understanding? Secondary purpose: To measure whether there is a significant change in student Running Records progress and NAPLAN results as a consequence of this focus. To emphasise the theory that quality teaching has a strong impact on student achievement. 4. Data collection methods: The selection of data collection methods and the presentation of the data relevant to the questions posed. Running Records Data collected and analysed in week 10 of every term for all students and in weeks 5 and 10 for at risk students. The data is used to track student progress R-2 as well as identifying students requiring intervention. Extra support provided to ensure that all students requiring intervention are catered for. Support staff provided with training and development enabling them to target the specific needs of individual students. The data is graphed and used in PLC s for deeper analysis and discussion. Student Review Team to meet regularly to assess needs of students. Provides evidence of successful implementation of Professional Learning and used to set future directions. PLC s are utilised to analyse data at a deeper level and to discuss effectiveness of current program for each student. Provide evidence of the effectiveness of the structure through staff feedback. The Leadership team has made a concerted effort to spend time in the classrooms so that they are able to listen to every child read at least once per term. 118

118 5. Conclusion commendations and recommendations: future planning and action. Through the PALL program we have been able to provide staff with Instructional Leadership and we have gained an understanding of the analysis of student data to inform our teaching. We have been able to target the specific needs of individual students and ensured that resources are matched to these needs. Through the program we have reinforced the importance of the need to begin intervention at the earliest possible opportunity. Early Years / Intervention The impact of the targeted support has been very positive over the first term of implementation with 60% of the students improving by three reading levels or more, 20% improving by two levels and 20% improving only one level. The school s Running Records data now indicates that (76.2%) percentage of our students in Years 1 and 2 have reached the School and Regional benchmarks. NAPLAN data from 2012 testing indicates that 84.9% of students were above the National Minimum Standard which is an increase of 6.9% from the previous year. Resourcing has included the resourcing of an SSO for 20 hours per week to specifically work with the targeted students on reading intervention. Staff Professional Development Professional development for staff has been specific to the improving the quality of instruction and the importance of comprehension within the reading program and the learning cycle. Ensuring that students are effectively engaged during the Guided Reading sessions has resulted in teachers developing student centred activities linked to the Blooms Taxonomy, enabling open ended learning to take place within a structured reading program. Teachers have also revised their methodologies and reviewed their book introductions to provide a greater depth of understanding for the students. This approach is consistent with the recommendations from the SSI Review. Recommendations Continue with the PLC s as a vehicle to continuously improve explicit teaching, reflect on teaching practice and improve learning outcomes for students. Continue intervention program through the targeting of resources to identified, at risk students. Continue focussed Professional Development in Literacy, in particular the focus on comprehension strategies and skills through quality teaching. Performance management to include observations of classroom practice and discussions around implementation of learning from Professional Development and addressing the questions identified earlier. Continue to develop wave 1, 2 and 3 intervention strategies. 119

119 PALL 2 Reporting and Evaluation of Literacy Interventions Principal Deb O Neil SchoolGilles Street PS Context: A brief description taken from the PALL context statement or your website! (no more than ½ a page) Two main components to the school- Mainstream Programme offering primary education form reception to year 7 students and an Intensive English Language programme (IELP) that prepares students who have been in Australia for less than a year and have minimal or no English language competence for mainstream education. It is a category 6 school and has enrollments of approx 270 in mainstream and 105 in IELC. While IELP classes undertake academic lessons separately, their rooms are located along side mainstream classrooms and IELC and mainstream classes join together for many activities, such as sport, the Arts, excursions, camps, the Student representative Council and social events. All Intensive English Language students are exempt from NAPLAN and Running records Data collection. Mainstream classes are characterised with high levels of English as Second language students. In mainstream section the demographics are 27.2% Transience, 38% Non English Speaking background and30% English as a second language support learners 27% residents of the C.B.D. (5000 postcode) 9 Aboriginal students. 6 Students with disability. High number of exemptions (61 children/ 1038 days MS) The Middle Years International Baccalaureate Programme is offered for year 6 and 7 students. Description of the Issue(s): What evidence/data that lead you to look at this issue(s)? Who is it an issue for? What other issues are related to it i.e. if you improve this issue what related things will also improve? Whole School Review The school has worked hard over the past 5 years to develop a consistent and cohesive approach to literacy teaching and assessment practices. This was particularly important in the early years where there was very little school coordinated data collection until NAPLA data in year 3. In the review of our early years literacy programme staff identified that there were a number of different programmes operating across the site and although there were some consistencies in teaching practice there was no whole site agreements in relation to how and what we taught within the early years literacy programme. A small group was established and began collecting resources including Deselea Konza Big 6 and research sheets from the DECD Literacy secretariat. It was noted that the collection of data was also loosely monitored with some whole school agreements but no baseline analysis or comparisons. The identification of learning difficulties often relied on data collection made by the class teacher before wave 2 intervention processes were recommended. Aside from running records data we had no shared agreements or defined benchmarks in the early years of what we expected students to achieve particularly in relation to spelling and sight word recognition. 120

120 Running records were only collected for year 1 and 2 students and staff with older year levels using screening tests such as Waddington s to determine reading ages. A data base was used and children were identified from this and NAPLAN data and directed into various in class and school intervention programmes. Historically the school has achieved good means scores in NAPLAN, with little or no children achieving under the National benchmarks, however on deeper analysis we were concerned that perhaps we were not moving or extending those children in the middle bands to achieve higher, and there was an over representation of students in the lower bands. Were our students just coasting and could we In sum we really didn t have a clear target of what we as a group expected at year levels in spelling and sight word recognition. We also identified the lack of oral language as a whole school focus. Although many teachers incorporated this into their learning programme there were no whole school agreements about what and how it is developed. Data Analysis. We began collecting and analysing the data we did have. In relation to our data our year 3 NAPLAN results in spelling showed that we were not really making any progress from year to year and it appeared that any improvement relied on the particular cohort rather than any definite purposeful actions on our part. There was also a disproportionally large % of students represented in the bottom 2 bands in Reading and Spelling in NAPLAN results and this increased from , in Reading an increase of 9.2%, and in Spelling and increase of 8%. In spelling this comparison also reflected a decrease in the number of students in the top 2 bands by 23%. It was also noted that although our year 2 running records data showed a high percentage of students achieving above level 26 our year one data showed a decline in the number of year ones achieving above recommended benchmark RR16. Our benchmarks for year one was also considered low in comparison to our region and the State recommendations that came out of the literacy secretariat. It became evident that we needed to ensure there was a consistent pedagogical base, which directed the teaching, learning and assessment throughout Mainstream. I n order to do this we need to develop some consistent shared understandings about what we would teach and develop targets that were consistent as well as aspirational. Purpose: Purposes are framed as inquiry questions. Primary purposes are the key reasons you are collecting the data what is it you want to show? Secondary purposes refer to information that also came out of your inquiry. For example if your primary purpose is What is the effect of explicitly teaching comprehension skills in a half hour daily block on achievement of Wave 1 students, then a secondary purpose could be To what extent did teachers feel that their literacy training and development built their capacity to explicitly teach comprehension skills? Primary Purpose; Has the development of common understanding and shared agreements across staff impacted on literacy improvement in the early years at 121

121 Axis Title GSPS? Has there been an increase in attainment as a result of setting of site benchmarks in spelling and reading improved? Our Secondary purpose questions were; Has the shared collection and analysis of student achievement data informed intervention practices and improved the early identification and intervention process for students? What impact has staff training and development in literacy impacted on teacher pedagogy? Data Collection Methods: Depending on the purpose, collection methods could range from surveys to standardised testing. Mention what data was collected, by whom, when and why Year 3 NAPLAN mean scores Numeracy Reading Writing Grammar Spelling There has been a gradual increase in all areas from 2011 to 2012 data. The largest increase has been in the area of spelling, which increased 21% since This area has shown varied results over the past 5 years and no real trends have emerged. We expect with the results gathered from our Oxford benchmark data that a more sustained growth over time will develop. There has been a gradual increase in year 3 reading over the past 5 years which has been pleasing as it has been a focus since lower scores were recorded in

122 Year 3 Spelling Proficeincy band comparisons 2011/ For band growth comparisons we have removed the exempted students from the data. Exempted students are part of the Intensive English Language unit of the school and are automatically exempted from the NAPLAN tests. These students have arrived from a non-english speaking Country with minimal English within the past year. Spelling results show that there has been little change in the % of students in the lower 2 proficiency bands but an increase in the higher bands. Proficiency Band 1 Proficiency Band 2 Proficiency Band 3 Proficiency Band 4 Proficiency Proficiency Band 5 Band Year 3 Reading Proficiency band comparisions2011/2012 Year 3 Reading band comparisons has seen a slight decrease in the number of students in the lower bands and an increase in the % of students in the upper bands Proficiency Band 1 Proficiency Band 2 Proficiency Band 3 Proficiency Band 4 Proficiency Proficiency Band 5 Band 6 123

123 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Year 2 Running Records < Year 2 data over the last 3 years has always indicated that over 70 % of our students have achieved over RR 21. (over 50% at 26 or above) The reduction in the number of students in the bottom levels has been a pleasing aspect with only 21% below level 21. We anticipate that this will continue to grow based on our year 1 data this year. 60% 50% 40% 30% 20% 10% 0% Year 1 Running Records > The year one data has shown a significant improvement over the last 3 years with a 20% growth in students achieving over RR level 21 in There has also been over 20% decrease in the students achieving level 20 or below and a decrease in children recoding RR levels below 15 over the past 3 years. 124

124 Oxford benchmarks. In 2011 we set benchmarks for achievement using the Oxford word list. We could not find any data on what was expected per year level so made some assumptions based on what we thought students should know in various year levels. The benchmarks were based on how many words the students could recognise and read and how many they could spell. Data was collected throughout 2012 Initial Oxford Benchmarks Reading Spelling Reception 45 Year Year Year The teachers collected data on a one to one basis as students progressed through the word list as an integral part of their spelling programme. In October whole school data was collected. The data collected showed that we had underestimated the benchmarks for reading of the words. Many of the students became very proficient in reading the words and could identify them easily. This also transferred to their reading with teachers reporting greater fluency and improvements in RR levels throughout. The spelling of the words proved to be problematic, considerably time consuming and not really giving us accurate data about spelling proficiency. Teacher Survey s All R-3 teachers were asked to respond to a short survey about the literacy work this year. There results showed that; (still getting them in) Training and development Staff training became more focussed and a number of staff participated in state and regional training. 5 staff members attended I-Lit training run across the Eastern Adelaide region over 3 terms and were involved in collection of baseline data, exploring practice and measuring growth. These mini action research processes provided valuable information and the opportunity to share data and practice. % Staff attended training on levelled texts and a short series of writing workshops were run by the assistant principal. 2 staff received training in THRASS as a means of teaching phonemes and regular opportunities were provided for staff to meet in year level teams and discuss practice. 125

125 Conclusions/ Commendations/ Recommendations: What worked for whom and why and what didn t work? If something didn t work suggest what else would be needed eg more time, more support, maybe you suspect there were outcomes that you haven t measured that you d like to look into more The development of our literacy programme for the early years has been a long and well thought process. We believed it was essential to have whole school commitment and shared agreements. This can only be achieved if the process is truly collaborative, consultative and based on sound research and data. In 2011 we established a series of agreements and commitments and implemented during 2011 and We knew the work on the PALLS before our involvement and we were aware of the research on which it was based. The involvement in the project has enabled us to further develop and refine our practices. In particular it has assisted us in organising and developing our waves of intervention, providing focus for training and development and the opportunity to examine tools we were and were not using to collect, monitor and diagnose student learning across the site. A number of classes implemented a literacy time where human resources were pooled and provided smaller teacher to student ratio. Staff were able to assess student needs, plan for learning and continually monitor and refine their practice to meet the needs. Team meetings were used to discuss individual and group needs and plan for change. As a result of our learning staff are collecting and analysing data, monitoring individual student growth and tracking students within and across year levels. This information is recorded and shared across the group and specific pathways and programmes are developed on a more personalised and individualised basis. There is a closer alignment with what is happening in each classroom and often children are shared across classes as a means of better extending their learning or providing more intensive re teaching. Oral language in the library. One of the initiatives to come out of our work has been Literacy in the Library. This happens every morning from 8.25am until A group of year 5, 6 and 7 students attended a training run by our assistant principal covering how to help young children read, questioning skills and basic oral language games and activities. A teacher is on duty and students from the years 5-7 s are rostered on each day. We have also engaged volunteers from the community to be in the library to assist. It is a wonderful opportunity for all students to read, play games and talk before school starts particularly those who don t get chance to read at home and the new arrivals students whose home language is not always English. Co-ordinated effort and mapping wave 2 intervention processes by Deputy Principal and Intervention support teacher. This will be done on a termly basis in line with the Intervention waves. We believe this will enable us to have a better process for early identification. Refinement and implementation of whole school assessment processes. A defined whole school agreement has been extended and implemented from R-7 in relation to all of the elements of literacy teaching. This includes share agreements for all aspects of English language and consistent data collection tools are being added to ensure that there is a consistent data collection across mainstream. For 2013 whole school assessment practices will be finalised and processes for collecting sharing 126

126 and monitoring this data will be developed. Staff will also engage in moderation exercises using the Australian Curriculum English. Review of Benchmark standards. Running records Our benchmarks in Running records will be reviewed so they are in line with the regional and Literacy secretariat levels. We feel these may be aspirational for our year one students, however they will give us a better picture of where our students sit-in line with the state. Oxford word lists. The Oxford word list was selected as part of the teaching of Spelling and benchmarks were established. Although they proved to be a valuable teaching tool there were some problems that arose in when using them as a tool for data collection? This process proved to be considerably time consuming, particularly in relation to spelling of the words. We also doubted whether that it was the best way to identify spelling levels or flag potential gaps as the Oxfords list is based on fluency not difficulty of word nor phonemic understandings. Although the learning of sight words is essential and will still form a part of our programme we believe that using it as a data collection tool for spelling may not be the best and we will re look at this for Staff training. Staff identified their needs through the survey and THRASS training. Literacy course Ginny is running. 127

127 Mt Compass Area School Literacy Priority Report 2012 Convenor/Leader: Pat Maloney Principal Members: Emma Adams, Inge Barlow, Velma Beaglehole, OrnellaCamplin, Denise Martin, Hayley Phillips, John Scholz Key Priority; Primary Purpose Embed an effective whole site common approach to literacy teaching, thereby achieving measurable student improvement ( Effective literacy teaching is systematic and explicit and requires a planned classroom literacy program that is balanced and integrated all teachers to provide quality literacy instruction in each curriculum area. 1 ) SA DECD Literacy Secretariat, 2012 Strategy 1 Whole school direction and planning Developing an agreed approach to teaching Literacy R-10. This includes o All Junior Primary classes using Jolly Phonics in Literacy sessions. o Reading Support Teacher overseeing R-2 teaching and learning of reading. A focus on developing Phonological Awareness has been implemented in all Reception classes with support from School Support Officers and Reading Support Teacher. o Literacy Coach working with Yr 3-9 teachers by modelling and training in persuasive writing modelling and training in reading comprehension strategies using PAT-R comprehension data to inform intervention strategies using NAPLAN data to inform intervention strategies o Students R-9 filled in a reading survey, which is being used to inform reading practice o A tactical teaching reading facilitator for the Middle School has been trained but is currently undertaking other duties Literacy in all learning areas o Staff meetings have highlighted the importance of literacy in curriculum areas other than English o The programming and planning template, Understanding by Design, has undergone several revisions to incorporate literacy, along with other general capabilities The LNNP (Literacy and Numeracy National Partnerships) evaluation rubric was given to all staff in May 2012 to ascertain the perceived performance of the school in literacy. This was readministered in October Table 17 shows the results for both periods. It is pleasing to note staff perceptions moved from functioning to strategic in most categories. The rubric is an aid for further planning. Interventions o Students who have been identified as being at risk in their learning attend Rainbow Reading, conducted by SSOs, on a daily basis. o Multilit is used to boost the reading competency of these students through the work of the Teaching and Learning Coordinator. o Reading Doctor is used for identified students in Reception- year 2 who have poor phonetic and sight word knowledge. These students get two plus sessions a week. o Small group intervention based around specific recommendations from Psychologists and Speech Pathologists (i.e. Articulation, language intervention etc.) o Support-a-Talker implemented in the new Reception class twice a week to boost oral language skills. 128

128 Strategy 2 Professional learning Principal undertaking PALL course (Principals as Literacy Leaders) and Deputy Principal undertaking SPALL (Secondary Principals as Literacy Leaders) and sharing strategies and information at group and whole staff gatherings. Literacy Coach provides 1:1 professional development for teachers of Yr 3-7 and two Yr 9 teachers on a fortnightly basis. Evidence-based practice strategies are being implemented by teachers. Literacy Coach and Reading Support Teacher conducted a reading workshop for parents in Term 3. The workshop was modified and presented to Governing Council and then to a whole staff meeting. As a result of the LNNP evaluation rubric, a Literacy Professional Learning Community was instituted in Term 2 with ongoing fortnightly meetings. Reception teachers have attended professional development to build a deeper understanding of the importance of Phonological Awareness in the Early Years. This ensured successful implementation of a Phonological Awareness program in all Reception classes. There has been a focus on incorporating Guided Reading into teacher s Literacy programs. Two teachers will have the opportunity to attend a Guided Reading workshop presented by Margaret Menner in Term 4. Strategy 3 Collection and use of data Agreed data to be collected is: running records R-6, PAT-R comprehension Yr 3-9, Lexiles Yr 1-9, Westwood spelling Yr 1-10 and Phonological Awareness Rec. Data is recorded in EDSAS, enabling it to be passed on to the next teacher in an organised, coherent manner. PAT-R comprehension data collected and analysed in Term 1 and Term 4. The data is being used to inform effective teaching interventions, particularly with inferential comprehension. The data helped to identify students at risk in reading. This was matched against students already getting support so that those not receiving support could be helped. A reading survey collected information about students attitudes to reading and is being used to inform practice. NAPLAN data will be rigorously analysed from October 2012 and used to inform teaching practice. Strategy 4 Resourcing Librarian, Chris Corey, purchased Springboard into Comprehension, which is being used across multiple classrooms. Several key texts were purchased to be used during literacy coaching sessions. The Fountas and Pinnell Benchmark Assessment 2 was purchased for use with older students in order to address their needs more effectively. Primary Comprehension A-G is being used intensively in one classroom. They cover a variety of genres and both literal and inferential comprehension. Maybe more copies should be ordered so other teachers can use them, too (cost $37.50 each). Possible resources to purchase from the Scholastic catalogue include Primary Grammar CDs for use on whiteboards (cost $37.95 each) and parts of speech/grammar posters at $17 each. Purchasing 8 ipads would be an asset to Literature Circles as books can be downloaded very cheaply, when compared to purchasing multiple copies of the same text in paper format. Older students may find reading using an ipad very motivating. 129

129 Mean Result Evidence NAPLAN Results Figure 10 illustrates changes in the mean scores for the school over the past three years. The Year 3 mean scores for numeracy, reading and writing were greater in 2012 than in both 2011 and Year 5 mean scores for numeracy, reading and writing were lower in 2012 than in 2011 but the numeracy and reading scores were higher in 2012 than in Year 7 mean scores were higher in 2012 for reading and writing when compared with Year 9 mean scores for numeracy, reading and writing were higher in 2012 when compared to the preceding two years. Mean scores over time Numeracy Year 3, Reading Year Writing 3, Writing Numeracy Year Reading 5, Reading Writing Numeracy Year 7, Reading Year Writing 7, Writing Numeracy Year 9, Reading Reading Writing Numeracy Year 3 Year 5 NumeracyYear 7 Year Figure 10 School summary report for Figure 11 shows Mt Compass Area School means are closer to the national means in all tested areas when compared with the State, Region and like schools Difference From National Mean Numeracy Reading Writing Grammar Spelling Mount Compass Area School DECD Fleurieu and Kangaroo Island Category 5 Figure 11 School comparison of means for

130 Figure 12 shows the percentage of Mt Compass Area School students above the State mean in all tested areas is greater than that for the rest of the State and greater than that for the Region. This is a positive outcome. Likewise, Figure 13 shows a positive outcome as the percentage of Mt Compass Area School students below the State mean is less than that for the State and also for that of the Region. Percentages Of Students Above DECS Mean 90% 80% 70% 60% 50% Reading Writing Spelling Grammar Numeracy 40% 30% 20% 10% 0% Mount Compass Area School DECD Fleurieu and Kangaroo Island Figure 12 Percentages of students above the State mean across all tested areas 70% 60% Percentages Of Students Below DECS Mean 50% 40% 30% Reading Writing Spelling 20% 10% 0% Mount Compass Area School DECD Fleurieu and Kangaroo Island Figure 13 Percentages of students below the State mean across all tested areas Table 16 shows the progress of cohorts of students. The pink cells indicate lower than expected percentages given a 25:50:25 ratio for L:M:H. The green cells indicated higher than expected percentages in the H division. Year 9s made good progress in grammar, writing, reading and numeracy. However, there is room for improvement in spelling (L to M, and M to H). Grammar could also be improved (M to H). 131

131 Year 7s made good progress in numeracy. Writing shows movement from M to H. There is room for improvement in writing, spelling and grammar, i.e. move from L to M for these three areas, and move M to H for latter two areas. Year 5s made good progress in numeracy and reading. Writing reflects the normal expectations. There is room for improvement in spelling (M to H) and grammar (L to M). Table 16 School Progress 2012 Year 9 Year 7 Year 5 L M H L M H L M H Spelling Grammar Writing Reading Numeracy Note. Percentage progress Table 17 shows the perceptions of staff regarding literacy in the school in May 2012 and October 2012, as measured using the LNNP Evaluation Rubric. Table 17 LNNP Evaluation Rubric: Literacy focus 132

132 Note. Broadly each level could be described as - Undeveloped Developing Functioning Strategic Embedded Little attention paid to, or engagement with, the criterion Aware of, and beginning to engage with, the criterion Operating effectively Planned, deliberate, consistent action Sustained activity producing sustainable outcomes, an integral part of the school culture Reading Survey Younger students were more likely to report they liked reading than older students, see Figure

133 Do you like reading? % R-2 Yr 3-6 Yr Yes Sometimes No Figure 14 Percentage of students reporting on their attitude towards reading Younger students were also more likely to report they read at home than older students, see Figure 15. Furthermore, students R-6 were more likely to report they read to family members (such as mum, dad, brother, sister or grandparent) than students in Years 7-9. Students in Years 3-9 were more likely to report that they read to themselves than students in R-2, and this seems developmentally appropriate. R-2 students were less likely to report they read online ( s, websites and chatlines) whereas students in Years 3-6 reported reading s and websites more frequently, even more frequently than the Years 7-9. Whilst Year 7-9 students reported using Facebook more frequently than students in the younger years, overall Year 7-9 students reported less reading at home both online and using more traditional reading methods. A comparison of Figure 14 with Figure 15 shows a similar percentage of Year 7-9 students reporting they do not like reading (22%) and not reading at home (28%). As research has shown students who fail to read in their own time eventually lose academic ground (Anderson, Wilson, & Fielding, 1988; Cunningham & Stanovich, 1986) our school needs to address this reported trend. 134

134 94 89 Do you read at home? % R-2 Yr 3-6 Yr Yes Sometimes No Figure 15 Percentage of students reporting reading at home Figure 16 shows over 80% R-2 students spend minutes per day reading for enjoyment, whilst this decreases to 72% for Year 3-6, with a further decrease to 52% for Year 7-9 students. It is concerning that 35% of Year 7-9 students report they do not spend any time reading for enjoyment. Reading for enjoyment, with its accompanying academic benefits, should be pursued by the whole school. Students who read more are proficient in reading fluency, comprehension and general vocabulary development (Worthy, 2002). Time spent reading for enjoyment % 66 R-2 Yr 3-6 Yr min 60 min 2 hrs > 2 hrs Figure 16 Percentage of students reporting time they spend reading for enjoyment When asked, What helps you with reading? R-2 students reported a variety of strategies such as sounding out, looking at the pictures and sight words. R-6 students recognised other people helped them with 135

135 reading, as did reading a lot. Having a quiet place to read was cited at all year levels as being helpful, but with the numbers citing this reason increasing with the year level, as shown in Table 18. Interestingly, nine students from Year 7-9 cited enjoyment as helping them with reading, whereas this reason was not given at all by the R-6 cohort. Table 18 Numbers of Students Citing a quiet place Helps Them with Reading R The reading survey showed older students tended not to read for enjoyment. Consequently, in order to address this matter, steps will be made to encourage students to read for enjoyment. An enthusiastic speaker from the bookshop Bees Knees will be invited to speak to the Years 7-9 in single gender groups. She has already spoken to the Years 5-6 classes. The reading survey also noted older students prefer quietness when reading, so steps will be taken to address this preference. Evidence from Running Records By the end of the year our Junior School students are expected to reach the following targets. Rec minimum of level 5 Year 1 minimum of level 14 Year 2 minimum of level 21 Year 3 minimum of level 28 Table 19 shows the percentage of students who have reached our school s target by the end of Term 2 and Term 3. Table 19 Percentage of students reaching school target. Term 2 Term 3 Year level % % Rec It is great to see so much improvement across the board. Year 6s show a slight decline only due to missing data from Ideally we would like all the year 6 s to be at level 30 and above. 136

136 R-6 Mean Scores PM levels Rec Dec Sept Figure 17 Average scores for reading levels R-6 Reading levels in the Early Years We are expected to gather Running Record data for all year 1 and 2 students at the end of Term 1 (optional) and Term 3. Figure 18 comparison between school and state Figure 19 show comparisons to the State and Region from our Term 1 data. Term 3 data has not yet been received to show further comparisons. Year 1 Figure 18 comparison between school and state Figure 19 comparison between school and region It is very pleasing to see a high percentage of our Year 1 students reading at a higher level in comparison to the state and the region, as is the low percentage of students reading levels

137 Year 2 In Figure 20 and 12 the School is compared to the Region and State. Although the spread across the reading levels is greater, it is still pleasing to see a higher percentage of students reading at a higher level compared to the Region. Figure 20 comparison between school and region Figure 21 comparison between school and state Regional Targets for Year 1 and 2 students Year 1 30% to reach a minimum level of 20 by the end of term 3 Year 2 66% to reach a minimum level of 21 by the end of term 3 Figure 22 shows our Year 1 students have reached the Region s Target at the end of Term 3. 55% of our Year 2 students have reached the Region s Target. 70 Year 2 Target Year 1 Target Apr Sept 10 0 min 20 min 21 Figure 22 shows year 1 and 2 reading levels compared to Region s targets 138