School Size and the Quality of Teaching and Learning

School Size and the Quality of Teaching and Learning An Analysis of Relationships between School Size and Assessments of Factors Related to the Quality of Teaching and Learning in Primary Schools Undertaken by the Inspectorate of the Department of Education and Skills Evaluation Support and Research Unit Inspectorate Department of Education and Skills 1

Contents 1 RESEARCH QUESTION AND CONTEXT 3 1.1 Background to this study 3 1.2 Inspection data from Incidental Inspections and Whole-school Evaluations 4 1.3 School sizes 4 1.4 Data from Parent and Pupil Questionnaires used in Whole-school Evaluations 5 2 LEARNING 6 2.1 Data from Incidental Inspections - Quality of Learning 6 3 TEACHING 7 3.1 Data from Incidental Inspections - Quality of Teaching 7 3.2 Data from Whole-school Evaluations - Quality of Teaching and Learning 8 3.3 Data from Parent Questionnaires - Quality of Teaching 9 4 QUALITY OF ENGLISH LESSONS DURING WSE 10 4.1 Data from Whole-school Evaluations - Quality of English Lessons 10 4.2 Data from Parent Questionnaires - Reading 11 4.3 Data from Pupil Questionnaires - Reading 11 5 QUALITY OF MATHEMATICS LESSONS DURING WSE 13 5.1 Data from Whole-school Evaluations - Quality of Mathematics Lessons 13 5.2 Data from Parent Questionnaires - Mathematics 14 5.3 Data from Pupil Questionnaires - Mathematics 15 6 QUALITY OF PLANNING 16 6.1 Data from Incidental Inspections - Quality of Planning 16 6.2 Data from Whole-school Evaluations - Quality of Planning 17 7 QUALITY OF ASSESSMENT 18 7.1 Data from Incidental Inspections - Quality of Assessment 18 7.2 Data from Whole-school Evaluations - Quality of Assessment 19 8 CONCLUSIONS 20 Appendix I - Information on the statistical tests and limitations of this study 21 Appendix II - Mean and standard deviation for all dependent variables 23 Appendix III Between-school variances for pupil and parent questionnaires 24 Appendix IV Results of statistical tests on incidental inspection data 25 Appendix V - Results of statistical tests on whole-school evaluation data 28 Appendix VI - Results of statistical tests on parent questionnaires 32 Appendix VII - Results of statistical tests on pupil questionnaires 38 2

1 RESEARCH QUESTION AND CONTEXT This report has been produced by the Inspectorate s Evaluation Support and Research Unit as a contribution to the Small Primary Schools Value-for-Money Review being conducted by the Department of Education and Skills. The Inspectorate wishes to gratefully acknowledge the advice provided by the ERC and its staff in contributing to the final report. The focused research question was: to investigate if relationships exist between the schoollevel quality data that were collected by the Inspectorate in a sample of primary schools and school size as given by a school s enrolment. 1.1 Background to this study The Department of Education and Skills in undertaking a value-for-money review of small schools requested the Inspectorate to analyse the data it collects to see if there were possible relationships between school size and the quality of teaching and learning. This question is of considerable interest as, in the context of a value-for-money review, it is important to know if the quality of students learning experiences may be positively affected by the different environments that would be found in schools of different sizes. For example, a key area of interest is whether students in smaller schools have better learning experiences than students in larger schools. The Inspectorate is charged under the Education Act, 1998 with operating a quality assurance programme, which it does through an extensive programme of inspections. During visits to schools and classrooms, inspectors make judgements about the factors that relate to outcomes for students, and they have a very specific focus on the quality of teaching and learning. The data that are recorded by inspectors were used in this report to search for relationships between school size and factors related to the quality of teaching and learning. Four different data sources were used in compiling this report and they were: 1. Data based on inspectors judgements arising from incidental 1 (unannounced) oneday inspections of primary schools 2. Data based on inspectors judgements arising from whole-school evaluations 2 (WSEs) in primary schools 3. Data from questionnaires completed by parents during whole-school evaluations 4. Data from questionnaires completed by pupils during whole-school evaluations. The tables below show the total number of inspections/questionnaires relating to each of the data sources. Data source Number of inspections Incidental inspections 625 Whole-school evaluations 361 Schools in which parent questionnaires 187 were administered Schools in which pupil questionnaires were 199 administered Data source Number of questionnaires Parent questionnaires 14,874 Pupil questionnaires 11,864 1 Incidental inspections are unannounced one-day inspections that inspectors carry out in primary schools to evaluate the quality and effectiveness of aspects of the education provided in schools under the normal conditions of a regular school day. The focus is on the quality of the education experienced by the learner. 2 Whole-school evaluations in primary schools involve evaluating the work of the school under the areas of management, planning, curriculum provision, teaching and learning, and student support. 3

These data sources incorporate a significant set of numerical data that have been collected by the Inspectorate. The collection of questionnaire data, as part of whole-school evaluations (WSE), began in September 2010. Data relating to incidental inspections and whole-school evaluations have a longer timeframe. October 2009 was when data first became available for incidental inspections, while data for whole-school evaluations span a period of approximately three years. This study has a number of limitations and these are described more fully in Appendix I. The main limitations are that the data wasn t collected originally for the purpose of this study and so a range of assumptions needed to be made about the data and their nature. Thus, this report should be read as providing supplemental guidance for the Department s Value-for- Money Review Committee on Small Schools and should be interpreted in the context of the literature review and other research available to the committee. 1.2 Inspection data from Incidental Inspections and Whole-school Evaluations The data used arise from the processes that take place during regular inspection activities and are maintained by the Inspectorate as a record of the quality of educational provision in the schools evaluated and are used for planning and reporting purposes. In making judgements about the quality of practice in schools the Inspectorate employs a four-point quality continuum as outlined in Looking at our School: An aid to self-evaluation in primary schools. The four points of the quality continuum are described here: Significant/major weaknesses (uniformly weak) Weaknesses outweigh strengths (more weaknesses than strengths) Strengths outweigh weaknesses (more strengths than weaknesses) Significant strengths (uniformly strong). 1.3 School sizes The schools were divided into two groups based on their enrolment. The enrolment data were sourced from the Department s primary schools database. The schools were grouped according to two groups as follows: Group A schools were those with between 1 and 49 pupils, and Group B schools were those with 50 or more pupils. The table below shows the number of data sources (inspections/questionnaires) according to school size. Data source Number of inspections for schools with 49 or less pupils Number of inspections for schools with 50 pupils or more Total Number of inspections Incidental inspections 119 506 625 Whole-school 74 287 361 evaluations Parent questionnaires Pupil questionnaires Schools with 49 23 26 Number of schools in which questionnaires were administered or less pupils Schools with 50 or more pupils Total 164 187 173 199 Number of questionnaires Completed Schools with 49 432 358 or less pupils Schools with 50 or more pupils 14,442 11,506 Total 14,874 11,864 4

This analysis focuses primarily on the areas most directly connected to pupils learning as assessed by inspectors: quality of planning, quality of teaching; quality of learning and quality of assessment. The data from the incidental inspections and from the whole-school evaluations were segregated according to school size. These data are presented according to the four quality levels of the Inspectorate s quality continuum referred to above. In addition, the data include, for whole-school evaluations, an analysis of the quality of English lessons and the quality of mathematics lessons as assessed by inspectors during whole-school evaluations. 1.4 Data from Parent and Pupil Questionnaires used in Whole-school Evaluations The questionnaires that were completed by parents and pupils were analysed according to school size. The results that were analysed were from the following questions: Parent questionnaires Q12. Teaching is good in the school. Q18. The school is helping my child to progress with reading. Q19. The school is helping my child to progress in Maths. Pupil questionnaires Q12. I think I am doing well at reading. Q13. I think I am doing well at Maths. This report presents findings of the analysis from the data sources that are described in this introduction. The analysis consists of three parts: 1. Raw percentage scores for quality indicators for incidental inspection data and wholeschool evaluation data. 2. Raw percentage scores for the views expressed by parents and pupils in the questionnaires that were administered as part of whole-school evaluations. 3. Statistical analysis of the data from each of the four data sources. Further information on the tests that were run may be found in the Appendices of this report. 5

2 LEARNING This section of the report deals with an analysis of the data in respect of indicators of the quality of learning. During incidental inspection visits, inspectors provide separate assessments of quality for learning and for teaching but, during WSE a combined assessment is given for teaching and learning. These combined WSE assessments appear in section 3, below, on teaching. 2.1 Data from Incidental Inspections - Quality of Learning The table below shows inspectors judgements on the quality of learning in the schools where an incidental inspection took place. Quality of Learning - Incidental Inspections School size Group A Schools with 49 or less pupils [112 inspections] Group B Schools with 50 or more pupils [504 inspections] Quality of learning Significant/major weaknesses (uniformly weak) Weaknesses outweigh strengths (more weaknesses than strengths) Strengths outweigh weaknesses (more strengths than weaknesses) 2.8% 3.3% 14.2% 11.1% 74.5% 75.1% Significant strengths (uniformly strong) 8.5% 10.5% It is evident from the table that while there are differences in the quality of learning these are very small. To establish the significance of these differences statistical testing was undertaken. The results of statistical testing confirmed that for incidental inspection data there is no statistically significant relationship between the quality of learning indicator data and school size. Overall observation(s) regarding the quality of learning: From the statistical tests it is indicated that school size is not associated to a statistically significant degree with the quality of learning. 6

3 TEACHING This section of the report deals with an analysis of the data in respect of indicators of the quality of teaching. 3.1 Data from Incidental Inspections - Quality of Teaching The table below shows inspectors judgements on the quality of teaching in the schools where an incidental inspection took place. Quality of Teaching - Incidental Inspections School size Group A Schools with 49 or less pupils [112 inspections] Group B Schools with 50 or more pupils [504 inspections] Quality of teaching Significant/major weaknesses (uniformly weak) Weaknesses outweigh strengths (more weaknesses than strengths) Strengths outweigh weaknesses (more strengths than weaknesses) 3.7% 3.2% 14.7% 12.8% 76.1% 72.7% Significant strengths (uniformly strong) 5.5% 11.3% It is evident from the table that there are differences in the quality of teaching across the different school sizes. For example, it is noticeable that in a greater percentage of the larger schools teaching was rated as having significant strengths, when compared with the smaller schools. To establish the significance of these differences statistical testing was undertaken. The results of statistical testing confirmed that for incidental inspection data there is no statistically significant relationship between the quality of learning indicator data and school size. 7

3.2 Data from Whole-school Evaluations - Quality of Teaching and Learning The table below shows inspectors judgements on the quality of teaching and learning in the schools where a whole-school evaluation took place. It should be noted that for WSE, inspectors provide an assessment of quality that encompasses teaching and learning. Quality of Teaching and Learning - Whole-school Evaluations School size Group A Schools with 49 or less pupils [74 WSEs] Group B Schools with 50 or more pupils [287 WSEs] Quality of Teaching & Learning Significant/major weaknesses (uniformly weak) Weaknesses outweigh strengths (more weaknesses than strengths) Strengths outweigh weaknesses (more strengths than weaknesses) 3.3%.0% 6.6% 9.5% 75.4% 72.8% Significant strengths (uniformly strong) 14.8% 17.7% While the table shows that there are differences in the quality of teaching and learning between small schools and larger schools, as observed during WSEs, these differences appear to be small. To establish the significance of these differences statistical testing was undertaken. The results of statistical testing confirmed that for whole-school evaluation data there is no statistically significant relationship between the quality of teaching and learning indicator data and school size. 8

3.3 Data from Parent Questionnaires - Quality of Teaching The table below shows parents views on the quality of teaching in the schools where a whole-school evaluation took place. These views were gathered using a confidential questionnaire during the evaluation process. The figure for overall agreement is obtained by adding the percentages for strongly agree and agree categories. Adding the percentages for disagree and strongly disagree categories gives a figure for overall disagreement. Parent Questionnaires - Quality of Teaching School size Group A Schools with 49 or less pupils [432 questionnaires] Group B Schools with 50 or more pupils [14,442 questionnaires] Teaching is good in the school Strongly agree 67.6% 58.8% Agree 27.3% 37.8% Disagree 2.3% 1.2% Strongly Disagree.5%.3% Don't know 2.3% 1.9% Overall agreement 94.9% 96.6% Overall disagreement 2.8% 1.5% It is noteworthy, from the responses to the questionnaires, that parents of pupils in larger schools hold marginally more positive views on the quality of teaching than do parents of pupils in smaller schools. To establish the significance of these differences statistical testing was undertaken. The results of statistical testing confirmed that for the data from parent questionnaires there is no statistically significant relationship between parents views on the quality of teaching and school size. Overall observation(s) regarding the quality of teaching: From the statistical tests it is indicated that school size is not associated to a statistically significant degree with the quality of teaching. 9

4 QUALITY OF ENGLISH LESSONS DURING WSE 4.1 Data from Whole-school Evaluations - Quality of English Lessons The table below shows inspectors judgements on the quality of English lessons in the schools where a whole-school evaluation took place. Quality of English Lessons - Whole-school Evaluations School size Group A Schools with 49 or less pupils [74 WSEs] Group B Schools with 50 or more pupils [287 WSEs] Quality of English Lessons Significant/major weaknesses (uniformly weak) Weaknesses outweigh strengths (more weaknesses than strengths) Strengths outweigh weaknesses (more strengths than weaknesses) 2.7%.0% 19.2% 14.5% 63.0% 66.8% Significant strengths (uniformly strong) 15.1% 18.7% It appears from the data that the quality of English lessons is marginally better in larger schools when compared with smaller schools. To establish the significance of these differences statistical testing was undertaken. The results of statistical testing confirmed that there is no statistically significant relationship between the quality of English lessons and school size. 10

4.2 Data from Parent Questionnaires - Reading The table below shows parents views on how well the school is helping their child in their reading. These data were collected during whole-school evaluations. The views were gathered using a confidential questionnaire during the evaluation process. The figure for overall agreement is obtained by adding the percentages for strongly agree and agree categories. Adding the percentages for disagree and strongly disagree categories gives a figure for overall disagreement. Parent Questionnaires - Reading School size Group A Schools with 49 or less pupils [432 questionnaires] Group B Schools with 50 or more pupils [14,442 questionnaires] The school is helping my child to progress with reading Strongly agree 65.3% 62.6% Agree 29.8% 33.5% Disagree 2.0% 1.3% Strongly Disagree.7%.4% Don't know 2.2% 2.2% Overall agreement 95.1% 96.1% Overall disagreement 2.7% 1.7% The data suggest that parents of pupils in smaller schools hold very similar views to parents of pupils in larger schools, in relation to whether the school is helping their child to progress with their reading. Statistical testing showed that there is no statistically significant relationship between parents views in relation to whether the school is helping their child to progress with reading and school size. 4.3 Data from Pupil Questionnaires - Reading The table below shows pupils views on how well they are doing at reading. These data were collected using a confidential questionnaire during whole-school evaluations. Pupil Questionnaires - Reading School size Group A Schools with 49 or less pupils [358 questionnaires] Group B Schools with 50 or more pupils [11,506 questionnaires] I think I am doing well at reading Yes 84.7% 83.0% No 3.7% 4.4% Don't know 11.6% 12.6% It appears from the data that pupils in smaller schools hold marginally more positive views than pupils in larger schools about their reading. Statistical testing was undertaken to examine these differences and it confirmed that there is no statistically significant relationship between the views held by pupils about how well they are doing at reading and the size of school in which they are learning. 11

Overall observation(s) regarding the quality of English lessons: From the statistical tests it is indicated that school size is not associated to a statistically significant degree with the quality of English lessons or reading. 12

5 QUALITY OF MATHEMATICS LESSONS DURING WSE 5.1 Data from Whole-school Evaluations - Quality of Mathematics Lessons The table below shows inspectors judgements on the quality of mathematics lessons in the schools where a whole-school evaluation took place. Quality of Mathematics Lessons - Whole-school Evaluations School size Group A Schools with 49 or less pupils [74 WSEs] Group B Schools with 50 or more pupils [287 WSEs] Quality of Mathematics Lessons Significant/major weaknesses (uniformly weak) Weaknesses outweigh strengths (more weaknesses than strengths) Strengths outweigh weaknesses (more strengths than weaknesses) 2.7%.4% 10.8% 9.9% 70.3% 72.9% Significant strengths (uniformly strong) 16.2% 16.9% The data suggest that while there is little difference between the quality of mathematics lessons in smaller schools compared with larger schools, there is nevertheless a proportionately greater percentage of smaller schools where there are significant weaknesses in the quality of mathematics lessons. To examine these differences statistical testing was undertaken. It confirmed that there is no statistically significant relationship between school size and the quality of mathematics lessons. 13

5.2 Data from Parent Questionnaires - Mathematics The table below shows parents views on how well the school is helping their child in Mathematics. These data were collected during whole-school evaluations. The views were gathered on a confidential questionnaire during the evaluation process. The figure for overall agreement is obtained by adding the percentages for strongly agree and agree categories. Adding the percentages for disagree and strongly disagree categories gives a figure for overall disagreement. Parent Questionnaires - Mathematics School size Group A Schools with 49 or less pupils [432 questionnaires] Group B Schools with 50 or more pupils [14,442 questionnaires] The school is helping my child to progress in Maths Strongly agree 62.3% 58.3% Agree 31.4% 35.8% Disagree 2.7% 1.7% Strongly Disagree 1.5%.4% Don't know 2.2% 3.8% Overall agreement 93.6% 94.1% Overall disagreement 4.2% 2.1% The data indicate that there are marginal differences in the views of parents when asked about whether the school is helping their child to progress in Mathematics, with parents of pupils in smaller schools very slightly more likely to be more negative in their views. Statistical testing confirmed that there are no statistically significant differences between the views expressed by parents of pupils in smaller schools compared with parents of pupils in larger schools with respect to the school helping their child to progress in Mathematics. 14

5.3 Data from Pupil Questionnaires - Mathematics The table below shows pupils views on how well they are doing at Mathematics. These data were collected using a confidential questionnaire during whole-school evaluations. Pupil Questionnaires - Mathematics School size Group A Schools with 49 or less pupils [358 questionnaires] Group B Schools with 50 or more pupils [11,506 questionnaires] I think I am doing well at Maths Yes 80.3% 77.9% No 6.5% 7.2% Don't know 13.2% 14.9% It appears from the data that pupils in smaller schools hold marginally more positive views about how well they are doing at Mathematics. It is interesting that this contrasts with the views held by their parents. Statistical testing confirmed that there is no statistically significant relationship between school size and the views held by pupils on how well they are doing at Mathematics. Overall observation(s) regarding the quality of mathematics lessons: From the statistical tests it is indicated that school size is not associated to a statistically significant degree with the quality of mathematics lessons or the views of parents or pupils on progress in Mathematics. 15

6 QUALITY OF PLANNING 6.1 Data from Incidental Inspections - Quality of Planning The table below shows inspectors judgements on the quality of planning in the schools where an incidental inspection took place. Quality of Planning - Incidental Inspections School size Group A Schools with 49 or less pupils [112 inspections] Group B Schools with 50 or more pupils [504 inspections] Quality of planning Significant/major weaknesses (uniformly weak) Weaknesses outweigh strengths (more weaknesses than strengths) Strengths outweigh weaknesses (more strengths than weaknesses) 9.0% 7.4% 34.2% 30.5% 48.6% 56.0% Significant strengths (uniformly strong) 8.1% 6.2% The data show that there are variations between the different school sizes in respect of the quality of planning. A larger percentage of the larger schools was judged to have better quality planning than was the case for smaller schools. To examine these differences statistical testing was undertaken. The statistical testing confirmed that there is no statistically significant relationship between school size and the quality of planning. 16

6.2 Data from Whole-school Evaluations - Quality of Planning The table below shows inspectors judgements on the quality of planning in the schools where a whole-school evaluation took place. Quality of Planning - Whole-school Evaluations School size Group A Schools with 49 or less pupils [74 WSEs] Group B Schools with 50 or more pupils [287 WSEs] Quality of Planning Significant/major weaknesses (uniformly weak) Weaknesses outweigh strengths (more weaknesses than strengths) Strengths outweigh weaknesses (more strengths than weaknesses) 1.4% 1.8% 25.7% 27.0% 64.9% 60.4% Significant strengths (uniformly strong) 8.1% 10.9% The data show very small differences in the judgements on the quality of planning in small schools compared with larger schools. To examine these differences statistical testing was undertaken. It confirmed that there is no statistically significant relationship between school size and the quality of planning. Overall observation(s) regarding the quality of planning: From the statistical tests it is indicated that school size is not associated to a statistically significant degree with the quality of planning. 17

7 QUALITY OF ASSESSMENT 7.1 Data from Incidental Inspections - Quality of Assessment The table below shows inspectors judgements on the quality of assessment for the schools where an incidental inspection took place. Quality of Assessment - Incidental Inspections School size Group A Schools with 49 or less pupils [112 inspections] Group B Schools with 50 or more pupils [504 inspections] Quality of assessment Significant/major weaknesses (uniformly weak) Weaknesses outweigh strengths (more weaknesses than strengths) Strengths outweigh weaknesses (more strengths than weaknesses) 2.8% 5.3% 29.9% 29.8% 59.8% 59.6% Significant strengths (uniformly strong) 7.5% 5.3% The data indicate that, when compared with larger schools, smaller schools are marginally more likely to have fewer weaknesses at the lower end of the quality continuum and more strengths at the upper end of the quality continuum in respect of assessment. In assessing these differences, statistical testing confirmed that there is no statistically significant relationship between school size and the quality of assessment. 18

7.2 Data from Whole-school Evaluations - Quality of Assessment The table below shows inspectors judgements on the quality of assessment in the schools where a whole-school evaluation took place. Quality of Assessment - Whole-school Evaluations School size Group A Schools with 49 or less pupils [74 WSEs] Group B Schools with 50 or more pupils [287 WSEs] Quality of Assessment Significant/major weaknesses (uniformly weak) Weaknesses outweigh strengths (more weaknesses than strengths) Strengths outweigh weaknesses (more strengths than weaknesses) 3.7% 1.7% 27.8% 33.9% 59.3% 51.7% Significant strengths (uniformly strong) 9.3% 12.6% The data indicate that, when compared with larger schools, smaller schools are marginally more likely to have fewer weaknesses at the lower end of the quality continuum and more strengths at the upper end of the quality continuum in respect of assessment. In assessing these differences, statistical testing confirmed that there is no statistically significant relationship between school size and the quality of assessment. Overall observations regarding quality of assessment: From the statistical tests it is indicated that school size is not associated to a statistically significant degree with the quality of assessment. 19

8 CONCLUSIONS The data indicate that, in the sample sets that were examined, there appear to be differences between school categories in some aspects of the quality of provision for pupils. However, statistical analysis indicates that there is no significant relationship between the school size categories and these apparent differences. This means that while there are, in some instances, differences between small schools and larger schools, in respect of the areas that were examined, these differences are not directly associated with the size of the school. Notwithstanding the limitations acknowledged in respect of factors not examined in the analysis and the overall limitations inherent to this report it appears that school size is not a significant variable that gives rise to effects in the areas that were examined in this report. 20

Appendix I - Information on the statistical tests and limitations of this study The statistical tests that were used are described below and they are contained in these appendices: 1. All statistical testing was run using SPSS PASW18. 2. The categorisation of schools into two groups based on a cut-off enrolment of fifty pupils was undertaken on the basis of the terms of reference of the Value-for-Money Review Committee. From a statistical perspective, it may be the case that a binary categorisation is insufficient to capture effects related to school size. For example, a five-level categorisation, although not within the compass of the terms of reference, may have had greater facility with respect to identifying school-size effects. 3. The data from incidental inspections and from whole-school evaluations consisted of one independent ordinal variable (school size) and a number of dependent variables relating to quality indicators (For incidental inspection: planning; teaching; learning; assessment. For WSE: planning; teaching and learning; English; Mathematics, assessment). For the purposes of this testing the dependent variables were defined as scale variables. On the basis of advice from SPSS the analysis was performed using an Independent Samples-Mann Whitney U Test for non-parametric data, which is based on the assumption of independent, random samples and the distribution of each sample being the same shape. It should be noted that other statistical tests such as chi-square tests may be applied to the data. 4. The questionnaire data for parents and pupils was transformed by calculating a score for each question for each school. This produced a score for each school for each question. These data were treated as a continuous scale variable. Q-Q plots were constructed to test for normality and based on these plots and the basis of the central limit theorem, relating to the number of data points; it was assumed that the data were normally distributed. 5. It should be noted that the sample sizes vary depending on the available data and that the report refers to percentages of the actual sample size used in the computations. So, statements should be read as meaning the percentage of schools for which there is data. 6. Inherent to the statistical tests that were performed are underlying assumptions about the data, their variance from a normal distribution, the relative sizes of the sample sets, and the validity of the data as statistical measures. 7. A limitation inherent to treating school size as a categorical variable is that it does not then allow for the possibility of testing for curvilinear effects associated with school size. However, as noted in point 2, the number of school size categories was constrained by the terms of reference of the review committee. 8. It should be noted that the data were not created originally for the purpose of generating this statistical report and thus no validity testing or reliability testing has been undertaken on the instruments. This means that the data, which are based on outcomes of inspection and are derived from the rubric used for inspections, may provide different measures in relation to the outcomes of teaching and learning than those which would be provided by the use of standardised tests of cognitive ability in Mathematics and English. Therefore, the results of the statistical analysis may be taken as indicative and subject to the underlying assumptions holding. 9. Inherent to the analysis of data in this report is the reasonable assumption that the data that have been gathered comes from a sample of schools randomly selected for WSE and incidental inspection and that a normal distribution holds for the various datasets. 10. The data that were analysed consist, in the case of data from WSE and incidental inspection, of school-level ratings by inspectors, while the data from parent and pupil questionnaires are composed of aggregated school values based on the total responses to the individual questions. The table in Appendix II shows the mean and standard deviation of all outcome variables used in this report. The table in Appendix III shows the between-school variance for the data from the pupil and parent questionnaires. The variance components were calculated in HLM 6.0 using maximum likelihood estimation, without sampling weights. It is evident that regardless 21

of school size or any other school characteristics, schools do not vary appreciably on any of the five data items from the pupil and parent questionnaires. 11. The datasets consist of qualitative data comprising ordinal variables and this places further limitations on the analysis and conclusions that may be drawn from that analysis. The purpose of this report is to provide a top-level indicative analysis of the data held by the Inspectorate and so supplement the work undertaken by the Department s Value-for-Money Review Committee on Small Primary Schools. The analysis, its results and conclusions should therefore be interpreted in the context within which this report was generated. 12. It should be noted that factors for which a control was not possible for this report are likely to have an impact on teaching and learning. These factors are likely to include the socio-economic status of the pupils and the school; gender factors; the percentage of pupils with special educational needs; and factors related to the organisation of the school. This report cannot provide advice on the conflation of these factors. Conflation can arise, for example, where elements that affect teaching and learning do not present in the statistical analysis because the factors with which they are associated are not present in this analysis. It may be possible if other data were included in the model that some link could be shown to exist. An examination of such factors is beyond the scope of this report. An example of such factors could be the possibility that school size may be associated with factors such as the density and type of social interaction that is experienced by children, which in turn may be associated with elements that relate to small schools, such as supportive networks of families and teachers, tightly knit communities, and strong personal identities. In consequence, these factors may have positive effects on learning outcomes for pupils and thus there could be links between school size and learning outcomes for pupils in respect of factors not studied in this analysis. Therefore, this analysis should only be used as a supplemental indicator to the wider studies and literature review used by the Department s Value-for-Money Review Committee. 22

Appendix II - Mean and standard deviation for all dependent variables Data from Incidental Inspections Mean Std. Deviation Schools with 49 or less pupils Quality of planning 2.56.771 Quality of assessment 2.72.641 Quality of teaching 2.83.569 Quality of learning 2.89.574 Schools with 50 or more pupils Quality of planning 2.61.714 Quality of assessment 2.65.663 Quality of teaching 2.92.604 Quality of learning 2.93.586 Data from Whole-School Evaluations Mean Std. Deviation Schools with 49 or less pupils Quality of Planning 2.80.596 Quality of Teaching & Learning 3.02.591 Quality of English Lessons 2.90.670 Quality of Mathematics Lessons 3.00.619 Quality of Assessment 2.74.678 Schools with 50 or more pupils Quality of Planning 2.80.642 Quality of Teaching & Learning 3.08.516 Quality of English Lessons 3.04.576 Quality of Mathematics Lessons 3.06.528 Quality of Assessment 2.75.690 Data from Parent Questionnaires Mean Std. Deviation Schools with 49 or less pupils Teaching is good in the school. 1.37231738218109 265475601172637 The school is helping my child to 1.41033254110850.455323543630509 progress with reading. The school is helping my child to 1.48479625766326.527792971417585 progress in Maths. Schools with 50 or more pupils Teaching is good in the school. 1.43196860227525.183657857992635 The school is helping my child to 1.40677274435174.166228485256021 progress with reading. The school is helping my child to 1.44766408860731.192836476948461 progress in Maths. Data from Pupil Questionnaires Mean Std. Deviation Schools with 49 or less pupils I think I am doing well at reading. 1.375482103367.4733197909285 I think I am doing well at Maths. 1.381749403186.2886135321346 Schools with 50 or more pupils I think I am doing well at reading. 1.357574199653.1689548244198 I think I am doing well at Maths. 1.451827917972.2033277497425 23

Appendix III Between-school variances for pupil and parent questionnaires Question Percent (%) Between-School Variance Parent Questionnaire Data Teaching is good in the school 8.0 The school is helping my child to progress with reading 5.0 The school is helping my child to progress in Maths 5.2 Pupil Questionnaire Data I think I am doing well at reading 1.6 I think I am doing well at Maths 1.4 24

Appendix IV Results of statistical tests on incidental inspection data Ranks School size N Mean Rank Sum of Ranks Quality of planning Group A: Schools with 49 or less pupils Group B: Schools with 50 or more pupils 111 296.75 32939.50 502 309.27 155251.50 Total 613 Test Statistics a Quality of planning Mann-Whitney U 26723.500 Wilcoxon W 32939.500 Z -.750 Asymp. Sig. (2-tailed).453 a. Grouping Variable: School size 25

Ranks School size N Mean Rank Sum of Ranks Quality of teaching Group A: Schools with 49 or less pupils Group B: Schools with 50 or more pupils 109 284.76 31038.50 494 305.80 151067.50 Total 603 Test Statistics a Quality of teaching Mann-Whitney U 25043.500 Wilcoxon W 31038.500 Z -1.470 Asymp. Sig. (2-tailed).141 a. Grouping Variable: School size Ranks School size N Mean Rank Sum of Ranks Quality of learning Group A: Schools with 49 or less pupils Group B: Schools with 50 or more pupils 106 286.64 30383.50 485 298.05 144552.50 Total 591 Test Statistics a Quality of learning Mann-Whitney U 24712.500 Wilcoxon W 30383.500 Z -.821 Asymp. Sig. (2-tailed).412 a. Grouping Variable: School size 26

Ranks School size N Mean Rank Sum of Ranks Quality of assessment Group A: Schools with 49 or less pupils Group B: Schools with 50 or more pupils 107 311.41 33321.00 493 298.13 146979.00 Total 600 Test Statistics a Quality of assessment Mann-Whitney U 25208.000 Wilcoxon W 146979.000 Z -.824 Asymp. Sig. (2-tailed).410 a. Grouping Variable: School size 27

Appendix V - Results of statistical tests on whole-school evaluation data 28

Ranks School size N Mean Rank Sum of Ranks Quality of Planning Group A: Schools with 49 or less pupils Group B: Schools with 50 or more pupils 74 179.53 13285.00 285 180.12 51335.00 Total 359 Test Statistics a Quality of Planning Mann-Whitney U 10510.000 Wilcoxon W 13285.000 Z -.051 Asymp. Sig. (2-tailed).959 a. Grouping Variable: School size Ranks School size N Mean Rank Sum of Ranks Quality of Teaching & Learning Group A: Schools with 49 or less pupils Group B: Schools with 50 or more pupils 61 143.24 8737.50 232 147.99 34333.50 Total 293 Test Statistics a Quality of Teaching & Learning Mann-Whitney U 6846.500 Wilcoxon W 8737.500 Z -.503 Asymp. Sig. (2-tailed).615 a. Grouping Variable: School size 29

Ranks School size N Mean Rank Sum of Ranks Quality of English Lessons Group A: Schools with 49 or less pupils Group B: Schools with 50 or more pupils 73 164.97 12042.50 283 181.99 51503.50 Total 356 Test Statistics a Quality of English Lessons Mann-Whitney U 9341.500 Wilcoxon W 12042.500 Z -1.503 Asymp. Sig. (2-tailed).133 a. Grouping Variable: School size Ranks School size N Mean Rank Sum of Ranks Quality of Mathematics Lessons Group A: Schools with 49 or less pupils Group B: Schools with 50 or more pupils 74 174.41 12906.00 284 180.83 51355.00 Total 358 Test Statistics a Quality of Mathematics Lessons Mann-Whitney U 10131.000 Wilcoxon W 12906.000 Z -.606 Asymp. Sig. (2-tailed).545 a. Grouping Variable: School size 30

Ranks School size N Mean Rank Sum of Ranks Quality of Assessment Group A: Schools with 49 or less pupils Group B: Schools with 50 or more pupils 54 143.32 7739.50 230 142.31 32730.50 Total 284 Test Statistics a Quality of Assessment Mann-Whitney U 6165.500 Wilcoxon W 32730.500 Z -.091 Asymp. Sig. (2-tailed).928 a. Grouping Variable: School size 31

Appendix VI - Results of statistical tests on parent questionnaires Q-Q plots to establish normality Teaching is good in the schools 32

The school is helping my child to progress with reading The school is helping my child to progress in Maths 33

Group Statistics School size N Mean Std. Deviation Std. Error Mean Teaching is good in the school. Group A: Schools with 49 or less pupils Group B: Schools with 50 or more pupils 23 1.37231738218109.265475601172637.055355489424962 164 1.43196860227525.183657857992635.014341269291557 Levene' s Test for Equality of Varianc es Independent Samples Test t-test for Equality of Means Sig 95% Confidence Interval of. the Difference (2- F Si g. t df tail ed) Mean Difference Std. Error Difference Lower Upper Teach Equal 5.1.0-185.17 -.0434608488 -.0260913824 ing is varia 28 25 1.3 2.0596512200 56414.1453938226 51455 good nces 73 94158 39771 in the assu schoo med l. Equal - 25.0.30 -.0571830588 -.0581106265 varia 1.0 37 7.0596512200 05647.1774130667 27016 nces 43 94158 15332 not assu med 34

Group Statistics School size N Mean Std. Deviation Std. Error Mean The school is helping my child to progress with Group A: Schools with 49 or less pupils 23 1.41033254110850.455323543630509.094941521906506 reading. Group B: Schools with 50 or more pupils 164 1.40677274435174.166228485256021.012980263937739 Independent Samples Test Levene's Test for Equality of Variance s t-test for Equality of Means Sig 95% Confidence Interval of. the Difference (2- F Si g. t df tail ed) Mean Difference Std. Error Difference Lower Upper The Equal 11.4.0.0 185.94.0035597967.0492871539 -.1007969414 scho varia 74 01 72 3 56757 17490.0936773479 54821 ol is nces 41307 helpi assu ng med my child to progr ess with readi Equal varia nces not assu med.0 37 22.8 29.97 1.0035597967 56757.0958247349 79112 -.1947509368 38079.2018705303 51594 ng. 35

Group Statistics School size N Mean Std. Deviation Std. Error Mean The school is helping my child to progress in Group A: Schools with 49 or less pupils 23 1.48479625766326.527792971417585.110052442178580 Maths. Group B: Schools with 50 or more pupils 164 1.44766408860731.192836476948461.015057999017193 Independent Samples Test Levene's Test for Equality of Variance s t-test for Equality of Means Sig 95% Confidence Interval of. the Difference (2- F Si g. t df tail ed) Mean Difference Std. Error Difference Lower Upper The Equal 19.9.0.6 185.51.0371321690.0571539637 -.1498895064 scho varia 29 00 50 7 55953 65402.0756251683 75513 ol is nces 63608 helpi assu ng med my child to progr ess in Math Equal varia nces not assu med.3 34 22.8 30.74 1.0371321690 55953.1110778257 07346 -.1927443299 23852.2670086680 35757 s. 36

ANOVA Sum of Squares df Mean Square F Sig. Teaching is good in the school. Between Groups Within Groups.072 1.072 1.884.172 7.049 185.038 Total 7.120 186 The school is helping my child to progress with reading. The school is helping my child to progress in Maths. Between.000 1.000.005.943 Groups Within Groups 9.065 185.049 Total 9.065 186 Between.028 1.028.422.517 Groups Within Groups 12.190 185.066 Total 12.218 186 37

Appendix VII - Results of statistical tests on pupil questionnaires Q-Q plots to establish normality I think I am doing well at reading 38

I think I am doing well at Maths ANOVA Sum of Squares df Mean Square F Sig. I think I am doing well at reading. Between Groups.007 1.007.136.713 Within Groups 10.511 197.053 Total 10.518 198 I think I am doing well at Maths. Between Groups.111 1.111 2.379.125 Within Groups 9.193 197.047 Total 9.304 198 39

Group Statistics School size N Mean Std. Deviation Std. Error Mean I think I am doing well at reading. I think I am doing well at Maths. Group A: Schools with 49 or less pupils Group B: Schools with 50 or more pupils Group A: Schools with 49 or less pupils Group B: Schools with 50 or more pupils 26 1.375482103367.4733197909285.0928256480813 173 1.357574199653.1689548244198.0128453974337 26 1.381749403186.2886135321346.0566017704708 173 1.451827917972.2033277497425.0154587225533 40

Independent Samples Test Levene's Test for Equality of Variance s t-test for Equality of Means Sig. 95% Confidence Interval of F Si g. t df (2- tail ed) Mean Difference the Difference Std. Error Difference Lower Upper I Equal 14.6.00.36 197.71.017907903.048584599 -.113720573 think varian 65 0 9 3 7137 1513.077904765 0645 I am ces 6371 doin assu g med well at readi ng. Equal varian ces not.19 1 25.9 65.85 0.017907903 7137.093710219 1703 -.174728753 5870.210544561 0144 assu med I Equal 2.62.10-197.12 -.045438017 -.019528846 think varian 0 7 1.5 5.070078514 3000.159685876 4676 I am ces 42 7861 0399 doin assu g med well at Math s. Equal varian ces not - 1.1 94 28.8 45.24 2 -.070078514 7861.058674803 1391 -.190109918 5363.049952888 9640 assu med 41