Report of External Evaluation and Review Rotorua English Language Academy Ltd trading as Rotorua English Language Academy Highly Confident in educational performance Highly Confident in capability in self-assessment Date of report: 11 January 2018
Contents Purpose of this Report... 3 Introduction... 3 1. TEO in context... 3 2. Scope of external evaluation and review... 5 3. Conduct of external evaluation and review... 5 Summary of Results... 7 Findings... 8 Recommendations... 15 Appendix... 16 MoE Number: 9497 NZQA Reference: C26050 Dates of EER visit: 18 and 19 October 2017 2
Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: Type: Rotorua English Language Academy Ltd trading as Rotorua English Language Academy Private training establishment (PTE) First registered: 1 March 1994 Location: Delivery sites: Courses currently delivered: 1460 Pukuatua Street, Rotorua As above Certificate in General English for Speakers of Other Languages (Level 3) Tour Group Programme Code of Practice signatory: Number of students: Yes Domestic: nil International: 25 130 Number of staff: Scope of active accreditation: Distinctive characteristics: Seven 10 full-time staff and six 14 part-time staff depending on demand As above Learners at Rotorua English Language Academy (RELA) mostly enrol in the Certificate in General English for Speakers of Other Languages (Level 3). This runs continuously, with enrolments taken every Monday. The programme is individualised for each learner, with some enrolling for one week 3
and others for up to two years. The Tour Group Programme is offered to groups of international learners. This programme offers English language development and a cultural and social experience of New Zealand. RELA is located in Rotorua, a significant tourist destination which appeals to short-term learners. It also results in seasonal variations in the number of learners enrolled and staff employed. Recent significant changes: Previous quality assurance history: There have been two changes to the governing members of RELA: the withdrawal of a director, and change in a trustee. NZQA has been notified of and has approved the changes. The previous external evaluation and review (EER) was conducted in October 2013. At that visit, NZQA was Highly Confident in RELA s educational performance and Highly Confident in its capability in self-assessment. RELA is a member of English New Zealand and has undergone two English New Zealand audits since the previous EER: in September 2015 and August 2017. The September 2015 audit report found that RELA met all standards; there was one commendation and one suggestion for improvement. The August 2017 audit report found that RELA met all standards and there were no suggestions for improvement. Other: RELA offers some accommodation on site to longterm adult learners. RELA is a member of Quality English and one of four New Zealand schools who are members. 4
2. Scope of external evaluation and review Two focus areas were included in this evaluation. Together they cover all the learner enrolments and the mandatory focus area for international providers. RELA s request for a focus on English language teaching and learning was agreed to by the evaluators. The previous two EERs in 2009 and 2013 selected two programmes Certificate in General English for Speakers of Other Languages (Level 3) and General English as focus areas. For this EER the focus areas were: Focus Area 1: International Students: Support and Wellbeing Focus Area 2: English language teaching and learning. 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: http://www.nzqa.govt.nz/providers-partners/registration-andaccreditation/external-evaluation-and-review/policy-and-guidelines-eer/introduction. The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. A pre-scoping meeting, via phone, was held with the principal before the on-site visit. A self-assessment summary document was supplemented with additional educational performance data and outcomes reports. Two evaluators conducted the on-site visit over two days. Personnel interviewed or otherwise contacted during this EER included: the principal, school manager, director of studies, assistant director of studies, teachers, administration and homestay staff, current learners and some homestay host families. An observer from English New Zealand also attended for much of the visit. In addition to the comprehensive self-assessment information and documents provided for the EER scoping exercise, the EER team sighted and was provided with various documents, including a number pertaining to governance and management including the Education (Pastoral Care of International Students) Code of Practice self-review. Performance and outcomes data included learner progressions, reports and feedback. Programme information included the syllabus, descriptors, moderation and tests. Handbooks for teachers, homestays and agents were also viewed. A number of learner and homestay files were selected at random for review. 5
Disclaimer The findings in this report have been reached by means of a standard evaluative process: http://www.nzqa.govt.nz/providers-partners/external-evaluation-andreview/policy-and-guidelines-eer/introduction/. They are based on a representative selection of focus areas, and a sample of supporting information provided by the TEO under review or independently accessed by NZQA. As such, the report s findings offer a guide to the relative quality of the TEO at the time of the EER, in the light of the known evidence, and the likelihood that this level of quality will continue. For the same reason, these findings are always limited in scope. They are derived from selections and samples evaluated at a point in time. The supporting methodology is not designed to: Identify organisational fraud 1 Provide comprehensive coverage of all programmes within a TEO, or of all relevant evidence sources Predict the outcome of other reviews of the same TEO which, by posing different questions or examining different information, could reasonably arrive at different conclusions. 1 NZQA and the Tertiary Education Commission (TEC) comprehensively monitor risk in the tertiary education sector through a range of other mechanisms. When fraud, or any other serious risk factor, has been confirmed, corrective action is taken as a matter of urgency. 6
Summary of Results Statement of confidence on educational performance NZQA is Highly Confident in the educational performance of Rotorua English Language Academy Ltd trading as Rotorua English Language Academy. Statement of confidence on capability in self-assessment NZQA is Highly Confident in the capability in self-assessment of Rotorua English Language Academy Ltd trading as Rotorua English Language Academy. RELA is a small niche school providing quality specialist English language programmes, comprehensive support for learners, and an authentic experience of New Zealand culture which strongly matches and meets the needs of learners. RELA has robust processes and practices that systematically monitor the performance of the whole organisation. There is a focus specifically on priority areas, such as the review and development of the curriculum and curriculum descriptors to improve teaching and target the individual language needs of learners. Comprehensive self-assessment provides useful information which is used to guide programmes, teaching and resource development. As a result, RELA has made a number of continuous changes to existing processes and has recently initiated others, such as surveys, to gather specific feedback in areas RELA has identified could be improved, for example orientation. This is part of an ongoing and continuous review process. The success and impact of highly effective self-assessment is evident in the consistent gains of learners English language proficiency compared with the learners own goals and formal assessment results. RELA not only reviews learners individual progress but takes an organisational view of achievement benchmarking data and achievement internally and against international exam results. Overall, RELA is comprehensively meeting the needs of learners by systematically determining their English language and personal goals and delivering quality English language programmes in conjunction with individual pastoral care and support. Effective processes contribute to the strong outcomes for learners, and robust and meaningful self-assessment ensures the PTE has a thorough understanding of its own performance, which is used for ongoing improvement. 7
Findings 2 1.1 How well do students achieve? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. RELA has comprehensive oversight of achievement. Data pertaining to achievement individual and organisational is collected and collated. RELA staff reliably review and benchmark this data using the PTE's criteria and international testing results. RELA is using this data and the resulting information effectively to understand learners progress in English language proficiency, to inform ongoing development of curriculum descriptors for assessment, and to support the development of teaching staff. Individual achievement and gains of learners who study for six weeks or longer are systematically tested and monitored at placement, every six weeks and at exit. Reports provide a clear indication of progress which includes bar graphs for easy visual comparison for each area of English language proficiency. RELA uses this individual learner data to develop understanding of achievement across the organisation. Benchmarking across the organisation commenced in 2014. RELA takes into account the length of enrolment as well as the gains and to ensure that learning outcomes are achievable within the timeframe of the programme. There is a significant amount of data collected that RELA uses to understand a range of factors that influence achievement such as the age of learners and their level of English language ability at entry. Over the three years of data, only nine of the 194 learners enrolled for six weeks or more had not made a gain in English language proficiency, which is negligible in terms of the number of learners represented in the data. RELA is using the data to better understand areas where learners consistently make progress and areas where progress is more variable. RELA benchmarks international exam results, both IELTS 3 and Cambridge English exams, and compares the results year to year. The indicator for achievement is the learners own exam result goal, regardless whether or not RELA considers the goal as achievable. Attempts to benchmark RELA globally, by country, are limited as RELA numbers are so small that extracting any significant information or meaning 2 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 3 International English Language Testing System 8
is difficult. Overall, for the years 2015, 2016 and year-to-date 2017, the majority of learners who sat international exams achieved their goals. 1.2 What is the value of the outcomes for key stakeholders, including students? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Good. Learners recognise RELA as a small niche school providing quality English language programmes, located in one of New Zealand s prime tourist destinations. RELA is convincingly meeting learners needs for English language development and a desire to experience New Zealand adventure activities and New Zealand culture. RELA's own self-assessment clearly demonstrates the quality and value of learners outcomes following their time at RELA. Data collected at exit via surveys from 2014 2017 reflect very positive ratings and responses regarding the school, course, teachers and homestays. These indicate that expectations were exceeded and the students were recommending the school to others. The i-graduate surveys in 2014 and 2016 showed RELA as performing very well in the English language barometer compared with other New Zealand schools. RELA also collects and reviews data on re-enrolment, extension, upgrade and withdrawal (prior to arranged finish date) as a proxy for achievement. The information and analysis shows for the past three years, from 2014 2016, 17 per cent, 9 per cent and 18 per cent of students respectively extended or upgraded their study while at RELA, or returned (re-enrolled) after a period of time away. RELA has disaggregated data and looks at this in more detail to extrapolate further useful information to understand and meet learners needs in this highly competitive sector. Withdrawals are closely reviewed, and over the three years all withdrawals bar two, who were dissatisfied, have been due to extenuating personal reasons. Learners have a high level of satisfaction with the quality of their outcomes, which supports the feedback RELA already gathers in its formal and informal processes. Other stakeholders, agents and homestay families confirm that RELA responds quickly to any issues and they observe learners gaining in confidence and improvement in their use of English. RELA has recently introduced a survey for agents which has had a good response and indicates that RELA communicates well with agents. The results from the survey also provided RELA with information on gaps in agents knowledge of the Code of Practice, and RELA now has an opportunity to address this. Systematic feedback from homestay hosts as important stakeholders does not occur. It may be useful for RELA to gain another perspective in regard to the information and support provided to the host families. 9
This is important as RELA recognises that learners experience of homestay has an impact on their outcomes and overall experience at RELA. 1.3 How well do programme design and delivery, including learning and assessment activities, match the needs of students and other relevant stakeholders? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. RELA s programmes effectively match the needs of learners. English New Zealand audits in 2015 and 2017 validate the academic standards and recognise the quality of RELA s staff and curriculum and course delivery, which ensures that the integrity of the programmes is maintained. All RELA permanent and relief teachers are well qualified and experienced. Most significantly, eight staff have extensive exam experience and seven have experience teaching younger learners, ensuring that the needs of the learners are met and the programmes offered are understood. Comprehensive course documentation covers specific goals, learning outcomes and expected standards of achievement for all programmes, including exam preparation classes, to ensure learning activities and assessment tasks are purposefully aligned with learning outcomes. The syllabus of the afternoon integrated skills classes is negotiated with content based on learner need and feedback. The curriculum design allows flexibility and is structured to meet the immediate needs of learners. Extensive work has been undertaken over recent years to improve the curriculum descriptors to support a flexible teaching and learning approach. A comprehensive curriculum document guides teachers to ensure consistency in teaching in the classroom. Curriculum descriptors are systematically and effectively used in reporting and planning. Teachers identify the descriptors as providing more focus in their planning and attribute improved consistency in assessing English language proficiency and clearer reporting to learners as a resulting significant benefit. Through this ongoing review process, programmes are constantly reviewed and updated to effectively meet the needs of learners. The director of studies monitors and approves changes to maintain the integrity of the assessment. A range of activities, such as surveys, teacher observation and regular staff meetings monitor and provide feedback on the teaching and quality of programmes. 10
1.4 How effectively are students supported and involved in their learning? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. The majority of RELA staff are permanent and have been at the PTE for many years. As a result, all staff have a clear understanding of and are experienced in addressing the language, academic and pastoral support needs of international students. The support structures and feedback mechanisms are designed to effectively identify and meet these needs and encourage learners to establish effective social and academic support networks while they study. Relevant and accurate programme and activity information and advice are provided prior to and on arrival at RELA. Orientation is provided and extracurricular activities including many that are no or low cost are offered most days. Learners value the opportunity to socialise, explore the community, and practise speaking English. Academic and language support for learners is enhanced through the offering of classes that match the proficiency level of the learner, even when learner numbers are low. In addition, learners are supported to determine for themselves on a weekly basis how well they are progressing. They can use a grid with standardised descriptors to guide their judgements. Self-study sessions offered to learners after hours are directed by qualified teachers and tailored to the individual needs of learners. These self-study sessions are popular prior to the six-weekly testing which gives learners formal and useful feedback on their progress, which they discuss one-to-one with the teacher. These activities improve the students learning ability. RELA understands its commitments under the Code of Practice. Pastoral care is the responsibility of all staff. As the PTE is small, staff and students are well known to each other and engage daily, providing informal opportunities to monitor the frame of mind of the learners. Any issues are dealt with directly by RELA staff or referred to an external professional. RELA staff carefully select and monitor homestays. Feedback from learners about their homestay experience is sought after the first four days and evaluated regularly during study and at exit. There are clear guidelines and expectations outlined to homestays around their responsibilities for the pastoral care of learners, especially those learners under 18 years. RELA often accepts enrolments from learners with physical or psychological difficulties which can have an impact on their learning. The small school environment and pastoral care is successful in supporting these learners to remain 11
in study and participate in the experience of studying English in New Zealand. This is valued by all learners. 1.5 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. Governance and management leadership is effective and described by staff as strong. The principal, with support from the senior management team, continues to anticipate future demand and effectively navigate the ongoing changes within and impacting on the sector, such as changes to legislation and compliance. The PTE s direction is clear and the continued delivery of quality programmes is evident through the independent reviews undertaken by English New Zealand, Quality English and NZQA over the past two decades. RELA s business plan reflects the priority areas of the PTE, and management is paying close attention to learners needs. Investment and resourcing target and meet these needs, the most significant being the review of curriculum, development of descriptors, and supporting staff in their development. Teacher-focused professional development meetings occur reliably and are aligned to the curriculum. RELA also develops teachers individually by supporting professional development based on the individual needs and interests of teachers. Succession within the PTE has been planned. Manuals and guidelines have been developed to ensure continuity in practice and processes. Shared administrative and teaching roles also ensure the sustainability and consistency of the delivery and performance of the PTE. This is important in a small PTE where changes in staff can have a significant impact. Self-assessment is comprehensive and effective. Feedback from staff and learners is captured in the organisational self-assessment working document which identifies areas for improvement and ideas to implement. The comprehensive document tracks changes and the impact of the changes, and findings are used insightfully. There is a formal annual review to re-evaluate priorities which guides the PTE s use of its resources. 12
1.6 How effectively are important compliance accountabilities managed? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. RELA s compliance accountabilities are well managed. There is a clear process to guide review and this is occurring reliably. This is the responsibility of the principal who, with the support of senior staff, monitors the PTE s compliance responsibilities. RELA sees its primary compliance accountabilities as being NZQA requirements for maintaining registration, and specifically the Code of Practice. The new Health and Safety at Work Act (2015) has also been a priority for governance to understand. A number of practices at the school have been enhanced or changed, including a review and change of trustees and the accountability that now sits with these responsibilities. In respect to managing compliance with NZQA requirements, samples of learner and homestay files were selected and appraised by the evaluators and indicated good record-keeping. Learner information was complete and there is evidence that the wellbeing and safety of learners in homestays is monitored, although some improvements in the recording of information would make this more robust. 13
Focus Areas This section reports significant findings in each focus area, not already covered in Part 1. 2.1 Focus area: International Students: Support and Wellbeing The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 2.2 Focus area: English language teaching and learning The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 14
Recommendations There are no recommendations arising from the external evaluation and review. 15
Appendix Regulatory basis for external evaluation and review External evaluation and review is conducted according to the External Evaluation and Review (EER) Rules 2013, which are made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. Self-assessment and participation in external evaluation and review are requirements for maintaining accreditation to provide an approved programme for all TEOs other than universities. The requirements are set through the NZQF Programme Approval and Accreditation Rules 2013, which are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. In addition, the Private Training Establishment Registration Rules 2013 require registered private training establishments to undertake self-assessment and participate in external evaluation and review, in accordance with the External Evaluation and Review Rules (EER) 2013, as a condition of maintaining registration. The Private Training Establishment Registration Rules 2013 are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the External Evaluation and Review (EER) Rules 2013. The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website (www.nzqa.govt.nz). The External Evaluation and Review (EER) Rules 2013 are available at http://www.nzqa.govt.nz/assets/about-us/our-role/rules/eer-rules.pdf, while information about the conduct and methodology for external evaluation and review can be found at http://www.nzqa.govt.nz/providers-partners/external-evaluation-andreview/policy-and-guidelines-eer/introduction/. NZQA Ph 0800 697 296 E qaadmin@nzqa.govt.nz www.nzqa.govt.nz 16