Hessisches Kultusministerium School Inspection in Hesse/Germany Contents 1. Introduction...2 2. School inspection as a Procedure for Quality Assurance and Quality Enhancement...2 3. The Hessian framework for school quality (HRS) as a guideline for the external evaluation...3 4. The Purpose of the Education Inspectorate in Hesse...4 5. The Staff the Inspectorate...4 6. Methods and Instruments for Collecting Data...5 7. The School Inspection Procedure...9 8. Quality assurance...13 9. External Evaluation of Charter Schools ( Selbstständige Schulen SES)...14
1. Introduction School inspection as an obligatory external evaluation procedure was introduced throughout Hesse in 2006. By the end-of-year 2010, all approximately 2,000 Hessian schools had been evaluated for the first time. Since January 2011, all public schools are being inspected for the second time, along with evaluations of adult education colleges and centres for students with special educational needs. School inspection as a key element of quality assurance and enhancement encourages the quality processes within schools and places an emphasis on the critical self-evaluation of the schools own endeavours to become better. 2. School inspection as a Procedure for Quality Assurance and Quality Enhancement External school inspection is supposed to complement the evaluations of school achievement such as central final examinations, large scale assessments and centralised as well as internal comparative tests based on educational standards. School inspection focuses on the quality of strategic, administrative, professional, cultural and educational processes at school, in addition to the abovementioned evaluations. It thus allows for a complete view of school performance. School inspection helps to give schools impetus to develop more quality by: referring to collected data concerning the quality of the teaching and learning process, objectives, school management, pedagogic professionalism and cultural activities as outlined in the Hessian framework of school quality (HRS) giving evidence of strengths and weaknesses confirming the validity of internal quality assessment from an external viewpoint encouraging within-school debate and development processes with reference to the inspection report creating a stronger commitment to sticking to plans and agreed development targets With regard to the growing self-responsibility of schools - for example concerning human resources development, budgeting or internal curricula - the results of school inspection serve as a basis for accountability. Hessian schools undergo an external evaluation for at least two up to four days referring to specific quality criteria of the HRS-framework. Data are collected by standardised evaluationtools. The results are presented in an individual report given to the school. The Hessian school inspection procedure is based on standards for educational evaluation, which focus on four major principles: Fairness: Usefulness: Feasibility: Precision: The evaluation should be carried out according to legal and ethical premises. The well-being of the persons affected by the results of the evaluation has always to be kept in mind. The evaluation should focus on the specific information requirements of the intended users. The evaluation should be planned and executed realistically in a goal-oriented, cost-conscious way. The evaluation should produce competent and professional information based on reliable data. May 2014 2
3. The Hessian framework for school quality (HRS) as a guideline for the external evaluation The Hessian framework for school quality - developed by scientists, teachers, headmasters and inspectors is the central reference guideline for determining the quality level of Hessian schools, not just in terms of external inspection but also for each single school to evaluate its quality internally. All instruments used in school inspection are based on the HRS. The framework consists of three main components: input (premises and conditions), processes and output (achievements and effects). Seven quality domains, which are regarded as fundamental characteristics for school quality, refer to these three components of the framework. The second school inspection at Hessian schools concentrates on analysing those five of the above mentioned seven quality domains that deal with processes that schools can strongly determine themselves. These are: Quality domain II: Objectives and Strategies for Quality Enhancement This quality domain focuses on quality management, school program activities, internal evaluations and derivation of further courses of action. Quality domain III: Leadership and Management The head of a school requires management knowledge and leadership skills in order to promote quality enhancement. These management qualities are evaluated here. Quality domain IV: Professionalism The quality of educational processes at school depends on the motivation and competence of the teaching staff and its commitment to lifelong learning. Appropriate learning opportunities should be created for all staff. Quality domain V: School culture The quality of relationships among and between teachers and students, cooperation with parents and partners, student activities, advice and additional work groups in the afternoon are taken into consideration here. Quality domain VI: Teaching and Learning This domain is in the centre of interest during school inspections. Lessons are regarded as a social process in which teachers and students work responsibly together to increase knowledge and competences according to individual demands and necessities. Lesson observations build a major source for gaining insight here. School inspection in Hesse does not collect, analyse or assess data concerning general conditions and determining factors of school life such as political and legal requirements or human and material resources (quality domain I). Similarly, the results of central tests and final examinations (quality domain VII) are also not taken into consideration by the Hessian school inspection - official data are provided by the KultusDataWarehouse for each single school. They serve as reference screen for the schools in comparison to others. During the second school inspection, 26 out of 81 quality criteria of the domains II to VI of the HRS were compiled in order to collect data on schools. The underlying focus of each single criterion is analysed to produce the so called core information that describe possible indicators of school quality and thus make them generally measurable at all different schools. May 2014 3
4. The Purpose of the Education Inspectorate in Hesse The major objective of the Hessian Education Inspectorate is to maintain and improve the quality of education in schools in Hesse. The focus is on supporting individual schools, as well as the Hessian education system as a whole with respect to the effectiveness of their development efforts. Based on the results of regular school inspections, the schools receive feedback concerning the quality of their current educational practice, as well as stimulus for their future efforts. These results are communicated in an accessible way to all target groups and stakeholders by means of school reports. In cooperation with the local education authorities, the schools develop target agreements based on the information of the school report in order to enhance their quality of education. The Education Inspectorate offers an aggregated feedback of school quality at regional state level as well as institutional knowledge (knowledge of the organisation and its structure) for monitoring decisions. Aside from the feedback to individual schools, the Education Inspectorate offers information at the higher level in the form of aggregated data (i.e. summarized and anonymised data). This data provides an overview of typical quality profiles of schools in Hesse and focuses on the following points at regional level: internal evaluation, orientation of the school management actions on the development of the school, individual support and, implementation of the German educational standards and the core curriculum. These results can be used to make monitoring decisions in terms of educational policy, for example regarding future measures and programs. Furthermore, they can be used to identify needs for training or qualification. School inspections allow a deeper insight into the school development on individual and regional state level (longitudinal design). The process of school inspection is intended for a full evaluation (all public schools in Hesse) and to cover a broad spectrum (all quality areas of the HRS with respect to educational processes). This comprehensive and binding evaluation of all process areas enables school inspections to be a reliable component in the quality assurance system and to give differentiated feedback to the individual schools. With respect to the results of the first inspection report, the second inspection indicates how the school has progressed. Moreover, the longitudinal collection and processing of data in the second school inspection provide the opportunity to track developments, to highlight major changes and to check the success of measures on the system level. 5. The Staff the Inspectorate The Inspectorate (supervisory authority) is a department of the Provincial School Office and Teachers Academy (LSA, ) and consists of all persons involved in school inspections. These include primarily inspectors and the staff at the headquarters in Wiesbaden. The latter are both the contact persons for the school management regarding organisational issues and also in charge of the preparation for, and follow-up to, the school visit. The Inspectorate consists furthermore of experts in empiricism, May 2014 4
statistics and information technology, who are responsible for the maintenance of the collection instruments and analysis tools, as well as for the data analysis and for the yearly reports. Depending on the size of the school, a school inspection is carried out by a team of two to four inspectors, who have no personal or official connection to the evaluated school. The members of these teams are pedagogues and psychologists who are experienced in school evaluation and quality management, and who have proven to be successful in school practice. In being, for example, a former headteacher, teacher trainer or school psychologist, each school inspector contributes a certain degree of experience. Usually, at least one team member has experience in the same type of school as that to be evaluated. The school inspectors become qualified via theoretical and practical training modules. A school inspection team may occasionally be supplemented by so-called externals. Externals are representatives of parents or student associations, industry or science, enriching the process from an extracurricular perspective. These people undergo a brief training for this function. 6. Methods and Instruments for Collecting Data In the overall process of school inspection, various methods and instruments are used. They are consistent, coordinated and complement each other in terms of the triangulation principle. This means to view a characteristic from different points of view (i.e., from the perspective of the students and from the perspective of teachers) and with various methods (i.e., with interviews or surveys). The evaluation of every criterion is verified by different collection methods and includes the perspective of different target groups. Document analysis Observation Online surveys Interviews Figure 1: Collection methods of school quality. The following figure illustrates the triangulation principle in the context of a school inspection. A selected criterion of the HRS and its related information serve as an example. May 2014 5
There are various working agreements for learning groups or school levels (contents of teaching, standards, education) Exams are taken simultaneously at this school. overall evaluation of the criterion = average of the core information Evaluation of core information 1 There are binding agreements on performance requirements and criteria for performance evaluation. There are binding agreements with the staff on the promotion of interdisciplinary skills. Evaluation of core information 2 Evaluation of core information 3 There are binding agreements with the staff for a common approach in educational issues. Evaluation of core information 4 Instruments of the school inspection Interviews Online surveys Document analysis Class observations Figure 2: The Principle of Triangulation. Document analysis Prior to the school inspection, the inspection team analyses the school documents according to the list of document requirements. This document analysis is carried out with regard to the HRS defined quality criteria as well as the core information, which has been defined in terms of the criteria. This includes: the report of the first school inspection, the target agreements following the first school inspection, the school program, concept papers, decisions, regulations (e.g. educational plans or support frameworks, school rules), the interpretation of results of internal evaluations and central performance reviews, the training plan, information about the school, additional documents relevant to the evaluation of quality criteria, the website. May 2014 6
During the school inspection additional documents are examined, such as: meeting minutes class tests (sample) promotion plans (sample) teaching reports (sample) schedules, examples of written communications of the school management press reports The documentation should be informative regarding current school progress and should allow an assessment of the school quality. Online surveys By means of standardised online surveys, the following groups are asked for their individual opinions: pupils (from the third grade), teachers (except the school management and trainee teachers), trainers (at vocational schools), selected schools management, teachers and parents of special care students. For the second school inspection, the school management is questioned separately in the form of an online survey, which also allows the management to comment on the quality criteria of the school in written form, and to mention current processes at the school in order to underpin their own assessments of the school quality. The Inspectorate provides codes (TANs) for the access of online surveys, according to the number of members of the group to be interviewed. Thus, an exhaustive data collection of the assessments of all school community members is possible. The higher the participation in the survey, the more meaningful is the result for the school. The school administration is responsible for the communication of information relating to the online survey and TANs to the addressees. There are various versions of surveys for students: Surveys for pupils of primary schools, special schools, students at secondary school level, students of vocational schools and students at adult education colleges. The scope and content of the questionnaires depend on the groups to be interviewed. There are so-called closed questions, based on a four-point rating scale, that record the answers of the various groups to the criteria of the HRS, identified in the school inspection. Additionally, these questions include I don t know or not applicable responses. There are model questionnaires available on www.schulinspektion.lsa.hessen.de. Does not apply Does rather not apply Does rather apply Does apply I don t know (1) (2) (3) (4) (5) Figure 3: Example of a response scale of the online survey. May 2014 7
A new form of the second school inspection includes five additional questions to the online survey for teachers, parents or students. These questions can be formulated to the specific needs of a school s internal evaluation. The result of this additional school internal survey is reported to the school. Yet, the results are not subject to an evaluation by the Inspectorate and are not included in the report. Class Observations Class observations are the focus of the school inspection on-site. In this case, the inspectors select a representative sample of school classes on the basis of schedules of all classes and teachers. The samples are representative of all subjects, types of schools, teachers and grades. Usually, at least 50% of the permanent teaching staff is observed. In small schools there are at least 12 class observations. The observations take place without prior notice. The observation is not an evaluation of individual teachers or groups of learners, but rather gives an interdisciplinary overview about the teaching quality and the teaching culture in the school. The observed lesson sequences are not discussed with the teachers. The classroom observations are limited to 20 minutes, and the observed section may be at the beginning, in the middle or at the end of a lesson. The observations are reported in a lesson observation form ( Unterrichtsbeobachtungsbogen ) which consists of rating scales from 1 to 4. It refers to the following aspects of the quality domain VI teaching and learning in the HRS: Interviews development of professional and interdisciplinary skills, structured and transparent teaching and learning processes, dealing with heterogeneous learning conditions, a climate and environment conducive to learning. As part of the school visit, the inspection team interviews the following groups: members of the school management teachers, parents (not at vocational schools, adult education colleges or upper secondary schools), students, administrative staff and maintenance (not educational staff) the school social workers (if any) representatives of the vocational schools. The guideline-based interviews (based on HRS) take between 30 and 90 minutes. The number of interviewees per group is usually six. The interviews should allow the expression of the interviewees opinion on aspects of the school life. The statements of the interviewees are documented by a member of the inspection team for later analysis, but not for archiving. May 2014 8
7. The School Inspection Procedure The school inspections consist of three main phases: the preparation, the implementation and the evaluation phase. 1. Preparation phase: Written information provided to the school by the Inspectorate (about 11 weeks prior to the school visit): announcement of the date of the school inspection and the date of delivery of the access credentials to the Internet-based electronic exchange platform. provision of information and materials to prepare for the school inspection. Confirmation of the working arrangements. Preliminary talk by telephone between the inspection team with the school management (about 8-9 weeks before the school visit) to: clarify unresolved issues and agreement of the inspection process, decide whether to evaluate a school-specific core area include school-specific online questions. Providing the access code for the online survey (about 7 weeks prior to the school visit). Data collection prior to the school inspection (approximately 4 to 6 weeks prior to the school visit). Online questionnaire for the school management, parents, teachers, students and trainers and provision of documents and information by the school. Analysis of school documents. 2. Implementation phase: School visit (duration: 2 to 4 days) Orientation tour, information about the core area, class observations, interviews, document analysis, feedback. 3. Evaluation phase: Data analysis. Preparation of the report. Presentation of results (optional) followed by the evaluation conference continuing work with the inspection results of the school target agreements with the State Office of Education Prior to the school visit: Preparation An electronic exchange platform is used to exchange documents and information between the Inspectorate and schools. It provides secure and largely paperless communication and ensures the confidentiality of data collection and analysis. School principals have exclusive access to their individual school s electronic portal in order to upload preliminary information ahead of the inspection. In addition, the school administration has access to the same central statistical data on staff and students that is also available to the Inspectorate. These need only be supplemented by a few details. In the school s portal, the May 2014 9
school administration announces the evaluation of a core area respectively decides to add school-specific questions to the online survey (if so desired). The school receives information of how to the use the electronic exchange platform. This platform serves to exchange documents between the Inspectorate and the school. The school administration can download from the exchange platform a series of informational documents about the second inspection and the TANs for the online survey. Furthermore, the exchange platform includes a list of requirements that specifies which school documents are of interest for the inspection team prior to the school visit. Prior to the inspection, the inspection team and the school management clarify all organisational and substantive questions during a one-hour telephone conversation. It is important to come to an agreement on the following programme points regarding schedule, content and staff: optional choice of the evaluation of a core area respectively optional school-specific supplements to the online survey, schedule regarding the school tour, introduction to the teaching staff, information about the core area (optional), interviews, document analysis,feedback, composition of the interview groups, organisation of the online survey, setting up school documents on the exchange platform, working conditions of the inspection team. The school and the inspectorate conclude a written working arrangement, which is signed by both parties. This working arrangement clarifies all responsibilities, services and the procedure of the school inspection. The working arrangement is commits both parties to act transparently and responsibly. The school visit: Implementation A school visit lasts for a minimum of two and a maximum of four days. The duration of the visit as well as the number of inspectors (at least two) depends on the size of the system to be evaluated. The school visit usually starts with the introduction of the inspection team to the school management. Subsequently, the inspection team sets up in an office, to be used exclusively by the inspection team during the school visit. The inspection team is given a guided orientation tour of the school building and, if necessary, the grounds. Usually in the first break, the members of the inspection team introduce themselves to the teaching staff and briefly outline the inspection s objectives and procedures. Normally in the third lesson, the inspection team will conduct the first lesson observation together. This first observation is used for the convergence of practices of the inspection team, which as a group is often newly composed for each inspection. Thereafter, each inspection team member carries out classroom observations individually, usually in the morning. The teacher interviews take place depending on the school mainly in the afternoon. Interviews with the non-teaching staff and the pupils mostly take place in the morning. The inspection team analyses the school documents in the afternoon and discusses perceptions and the current status of the inspection process within the team. In the afternoon, the school can also provide a half-hour presentation in order to explain its chosen core area for assessment. The school visit ends with a feedback session in the presence of (at least) the extended school May 2014 10
management. The feedback round may be supplemented by interested members of the staff, parents or pupils. During these feedback sessions, the results of the inspection are not yet discussed. The main points are rather observations and impressions of the working methods of the inspection team and the willingness of the school community to cooperate with the team. Example of the schedule of a two-day school visit: First Day 07:30 Official Welcome 08:00 Orientation tour 09:30 Introduction to the staff 09:45 13:00 Classroom visits 13:00 Lunch 13:30 Interview with pupils (1 hour) 15:00 Presentation of core area (1/2 hour, optional) 16:30 Interview with parents (1 hour) 17:30 Team meeting Second Day 08:00-13:00 Classroom visits 13:00 Lunch 13:30 Interview with the non-teaching staff (1/2 hour) 14:30 Interview with the teachers (1.5 hours) 16:00 Interview with the school administration (1 hour) 17:00 Feedback on the inspection procedure 17:45 Team meeting Figure 4: Example of the procedure of a two-day school visit Preparation for the school inspection by the school management. In preparation for the inspection, the following hints should be taken into account: 1. Establishment of an information board in the staff room showing: the Hessian school quality frame of reference, the lesson observation form ( Unterrichtsbeobachtungsbogen ), May 2014 11
a schedule for the school visit, a compilation of the interview groups, organisational information, material for the inspection process, space for open questions, a link to the website of the Inspectorate: www.schulinspektion.lsa.hessen.de. 2. Communication with the Inspectorate or the management of the inspection team: The school management may ask questions or express concern to the management of the inspection team at any time by contacting the team headquarters. 3. Preparatory staff conference and information from the school committees The whole staff should be informed about the upcoming school inspection and appropriate material should be handed out during a conference. During this conference, open questions may be collected and clarified later with the management of the inspection team. After the school visit: Evaluation and report: After the school visit, all the collected data will be summarised and the core information will be evaluated in form of an overview of results. On this basis, the inspection team creates the school inspection report. The report provides the schools with detailed feedback on their quality profile in terms of the evaluated criteria of HRS and includes the following content: Foreword: 1. Summary of the results 2. Differentiated feedback on the quality profile of the school 2.1. Quality Area II "Objectives and strategies of quality development" 2.2. Quality Area III: "Leadership and Management" 2.3. Quality Area IV: "Professionalism" 2.4. Quality Area V: "School culture" 2.5. Quality Area VI: "Teaching and Learning" 2.6. Profile focus of the school 3. Annex 3.1. Collection methods and instruments 3.2. Explanation of the methods and the presentation of the evaluations 3.3. Results of the class observations 3.4. Results of the online survey Provision of the report, presentation of results and target agreements Prior to its completion, the report is given to the school management. Within three working days, the school management has the opportunity to correct factual errors or to add a supplementary statement to the report. The report is then provided to the school management on the exchange platform and as a printed version. The presentation of results (EP = Ergebnispräsentation ) is the final part of the inspection process. As part of the EP, the management of the inspection team optionally offers a presentation and explanation of selected central results. This presentation serves to pave the May 2014 12
way for the further school development. With the provision of the report the work of the inspection team is finished. This is followed by discussions and conferences at school, and with a representative of the State Office of Education, which conclude with agreed targets for the focus of further education development. This is mainly a matter of deciding the appropriate measures needed to improve areas where weaknesses have been revealed. Before completion, the agreed targets are usually discussed in various school committees and are open to the public in this context. During the implementation of the common target and performance agreements, the school will be advised by the State Office of Education. Follow-up inspection Schools that turn out to be "below standard" will be inspected again two years later. The criterion for a "school needing re-inspection" is reached, if at least half the criteria of the quality areas II to V or at least at half the criteria of the quality area VI have been rated with 2.0 or lower. In the follow-up inspection, taking place about two years after the regular inspection, the Inspectorate provides a focused evaluation of the areas classified as weak. 8. Quality assurance In order to develop further the process of school inspection, systematic evaluation within an internal quality management is required. The inspectorate uses procedures and methods for the internal quality assurance and the development of the school inspection, which are presented below. School management survey After the inspection the various school managements are interviewed in writing, and from a sample via structured telephone interviews, regarding the procedure and the results of the external evaluation. The findings from these surveys are discussed through conferences and closed-door meetings of the inspectorate and are used for the optimisation of the process. School conference survey Along with the provision of the written school inspection report, the school management receives TAN (access code) letters for each member of the school conference. In online surveys, the members may provide detailed feedback on the school inspection from their point of view. Examination of reports After its completion by the management of the inspection team the Inspectorate cross-reads and checks the reports for plausibility and compliance of the quality criteria. Peer Reviews As part of a transnational cooperation (between Berlin, Brandenburg, Hamburg, Hesse, Lower Saxony, North Rhine-Westphalia and the Netherlands), the involved partners have agreed to allow colleagues from other states to participate in inspections within their own state. Following this, feedback reports containing opinions on the procedure, the instruments used May 2014 13
and the professionalism of the inspectors are produced. These peer reviews provide important information for the further development of the entire inspection process. Scientific reports and support The design, the frame of reference, the procedures and instruments are continually evaluated and developed both by external experts as well as by the research assistants of the Inspectorate. In examinations, hearings or working groups, findings and experiences are exchanged and used for the further development and optimisation of methods and instruments. Conferences and closed-door meetings The members of the Inspectorate meet for regular conferences and closed door meetings, mainly in order to reflect and further develop the inspection process. The inspectors are also involved in continuous internal training. This process is documented in portfolios. 9. External Evaluation of Charter Schools ( Selbstständige Schulen SES) Charter schools acknowledged by the Hessian Ministry of State have got more scope to decide on using the budget for staff and case management, the teaching organisation and teaching contents than regular schools. Provided that the state-wide standards of education are respected, variations in educational organisation are allowed in particular in the composition of learning groups, forms of outer and inner differentiation, certificates of achievement, school-specific curricula and class schedules. The greater scope demands accountability on the other hand. Required is an internal quality management system as well as a school management that stimulates and supports methods for quality assurance and enhancement. The impact should be visible in school development priorities and the improvement of teaching and learning procedures. Approximately six months after registration Charter Schools undergo an initial external analysis by the Hessian Inspectorat, which is repeated within a couple of years. Again the HRS serves as the major reference guideline for the evaluation. But the procedure is in part different to the regular inspection described above. The external evaluation of Charter Schools concentrates on three major elements: An assessment of the school-based quality management system along with the school management s endeavor to encourage and support the process, a chosen school-specific core area, the teaching and learning process in the same way as in the regular school inspections. The procedure can be regarded as a meta-evaluation of the school s own quality management system. The evaluation report provides central criteria of an effective quality management system and gives feedback for further optimisation of the already implemented strategies for assuring and enhancing school quality. May 2014 14