Report of External Evaluation and Review

Similar documents
Report of External Evaluation and Review

Higher Education Review (Embedded Colleges) of Navitas UK Holdings Ltd. Hertfordshire International College

Programme Specification

BSc (Hons) Banking Practice and Management (Full-time programmes of study)

Navitas UK Holdings Ltd Embedded College Review for Educational Oversight by the Quality Assurance Agency for Higher Education

Qualification handbook

Higher Education Review of University of Hertfordshire

Mandatory Review of Social Skills Qualifications. Consultation document for Approval to List

Nottingham Trent University Course Specification

Programme Specification. MSc in International Real Estate

Programme Specification

Practice Learning Handbook

Practice Learning Handbook

Programme Specification. BSc (Hons) RURAL LAND MANAGEMENT

PROGRAMME SPECIFICATION

CONSULTATION ON THE ENGLISH LANGUAGE COMPETENCY STANDARD FOR LICENSED IMMIGRATION ADVISERS

Institutional review. University of Wales, Newport. November 2010

Bachelor of International Hospitality Management, BA IHM. Course curriculum National and Institutional Part

HARPER ADAMS UNIVERSITY Programme Specification

Programme Specification. MSc in Palliative Care: Global Perspectives (Distance Learning) Valid from: September 2012 Faculty of Health & Life Sciences

Overview. Contrasts in Current Approaches to Quality Assurance of Universities in Australia, the United Kingdom and New Zealand

Programme Specification

Student Assessment Policy: Education and Counselling

PROGRAMME SPECIFICATION

P920 Higher Nationals Recognition of Prior Learning

Introduction 3. Outcomes of the Institutional audit 3. Institutional approach to quality enhancement 3

PERFORMING ARTS. Unit 2 Proposal for a commissioning brief Suite. Cambridge TECHNICALS LEVEL 3. L/507/6467 Guided learning hours: 60

Personal Tutoring at Staffordshire University

SURVEY RESEARCH POLICY TABLE OF CONTENTS STATEMENT OF POLICY REASON FOR THIS POLICY

Higher Education Review (Embedded Colleges) of Kaplan International Colleges UK Ltd

Accreditation of Prior Experiential and Certificated Learning (APECL) Guidance for Applicants/Students

Document number: 2013/ Programs Committee 6/2014 (July) Agenda Item 42.0 Bachelor of Engineering with Honours in Software Engineering

ASSISTANT DIRECTOR OF SCHOOLS (K 12)

ROLE DESCRIPTION. Name of Employee. Team Leader ICT Projects Date appointed to this position 2017 Date under review Name of reviewer

Business. Pearson BTEC Level 1 Introductory in. Specification

Programme Specification (Postgraduate) Date amended: 25 Feb 2016

Foundation Certificate in Higher Education

THE QUEEN S SCHOOL Whole School Pay Policy

Qualification Guidance

Programme Specification

APAC Accreditation Summary Assessment Report Department of Psychology, James Cook University

Initial teacher training in vocational subjects

Quality in University Lifelong Learning (ULLL) and the Bologna process

Providing Feedback to Learners. A useful aide memoire for mentors

BILD Physical Intervention Training Accreditation Scheme

Chiltern Training Ltd.

Student Experience Strategy

An APEL Framework for the East of England

REGULATIONS FOR POSTGRADUATE RESEARCH STUDY. September i -

2007 No. xxxx EDUCATION, ENGLAND. The Further Education Teachers Qualifications (England) Regulations 2007

PROJECT DESCRIPTION SLAM

Developing an Assessment Plan to Learn About Student Learning

PROGRAMME SPECIFICATION: MSc International Management (12 month)

School Leadership Rubrics

University of Cambridge: Programme Specifications POSTGRADUATE ADVANCED CERTIFICATE IN EDUCATIONAL STUDIES. June 2012

BSc (Hons) Property Development

STUDENT AND ACADEMIC SERVICES

Henley Business School at Univ of Reading

Referencing the Danish Qualifications Framework for Lifelong Learning to the European Qualifications Framework

Thameside Primary School Rationale for Assessment against the National Curriculum

PROGRAMME SPECIFICATION KEY FACTS

SELF-ASSESSMENT EXTREMISM & RADICALISATION SELF-ASSESSMENT AND RISK ASSESSMENT

Rules of Procedure for Approval of Law Schools

Recognition of Prior Learning (RPL) Procedure - Higher Education

MSc Education and Training for Development

Unit 7 Data analysis and design

Assessment of Generic Skills. Discussion Paper

Minutes of the one hundred and thirty-eighth meeting of the Accreditation Committee held on Tuesday 2 December 2014.

GCSE English Language 2012 An investigation into the outcomes for candidates in Wales

1 Use complex features of a word processing application to a given brief. 2 Create a complex document. 3 Collaborate on a complex document.

Course Specification Executive MBA via e-learning (MBUSP)

Chapter 2. University Committee Structure

General study plan for third-cycle programmes in Sociology

CORE CURRICULUM FOR REIKI

Pharmaceutical Medicine

Graduate Diploma in Sustainability and Climate Policy

BSc Food Marketing and Business Economics with Industrial Training For students entering Part 1 in 2015/6

Inquiry Learning Methodologies and the Disposition to Energy Systems Problem Solving

Special Educational Needs & Disabilities (SEND) Policy

ACADEMIC AFFAIRS GUIDELINES

Pearson BTEC Level 3 Award in Education and Training

Guidance on the University Health and Safety Management System

Orientation Workshop on Outcome Based Accreditation. May 21st, 2016

Exam Centre Contingency and Adverse Effects Policy

Chapter 9 The Beginning Teacher Support Program

POST-16 LEVEL 1 DIPLOMA (Pilot) Specification for teaching from September 2013

Irtiqa a Programme: Guide for the inspection of schools in The Emirate of Abu Dhabi

CAUL Principles and Guidelines for Library Services to Onshore Students at Remote Campuses to Support Teaching and Learning

School Inspection in Hesse/Germany

CELTA. Syllabus and Assessment Guidelines. Third Edition. University of Cambridge ESOL Examinations 1 Hills Road Cambridge CB1 2EU United Kingdom

A GENERIC SPLIT PROCESS MODEL FOR ASSET MANAGEMENT DECISION-MAKING

Information Sheet for Home Educators in Tasmania

Internship Department. Sigma + Internship. Supervisor Internship Guide

STUDENT HANDBOOK ACCA

Assessment Pack HABC Level 3 Award in Education and Training (QCF)

Digital Media Literacy

MASTER S COURSES FASHION START-UP

Bilingual Staffing Guidelines

ELEC3117 Electrical Engineering Design

Standards and Criteria for Demonstrating Excellence in BACCALAUREATE/GRADUATE DEGREE PROGRAMS

Transcription:

Report of External Evaluation and Review Regent International Education Group Not Yet Confident in educational performance Confident in capability in self-assessment Date of report: 1 December 2015

Contents Purpose of this Report... 3 Introduction... 3 1. TEO in context... 3 2. Scope of external evaluation and review... 5 3. Conduct of external evaluation and review... 6 Summary of Results... 7 Findings... 10 Recommendations... 19 Appendix... 20 MoE Number: 7453 NZQA Reference: C18575 Dates of EER visit: 26-28 May 2015 2

Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: Type: Location: Delivery sites: Courses currently delivered: Regent International Education Group Private training establishment (PTE) 520 Queen St, Auckland Central As above National Certificate in Horticulture (Landscape Strand) (Level 4) First registered: 18 December 2001 Diploma in Horticulture (Landscaping) (Level 5) Diploma in Business (Advanced) (Level 7) Diploma in Information Technology (with strands in IT Support and Multimedia) (Levels 6 and 7) General and Academic English (Levels 1-4) New Zealand Certificate in English Language (NZCEL) (Levels 1-4) Code of Practice signatory: Number of students: Number of staff: Yes International: 116 at the time of the external evaluation and review (EER); the majority of students are enrolled in the business and IT programmes. 15 full-time (mainly management and support staff) and 13 part-time staff (most of the teaching staff 3

are part-time contractors) Scope of active accreditation: In addition to courses currently delivered: National Certificate in Horticulture (Introductory) (Level 2) Certificate in Aged Care (Level 3) Certificate in Foundation Skills for Health Studies (Level 3) National Certificate in Social Services (Level 4) National Diploma in Computing (Level 5) Distinctive characteristics: Recent significant changes: Students come from a variety of countries; most are from India, the Philippines or China; a smaller number are from Russia, Saudi Arabia and other countries. The operations director resigned in late 2014. The position was filled by a new director of studies, who left during April 2015. At the time of the EER, the business programme leader was also actingdirector of studies. A part-time job search coordinator was appointed in late 2014. During April 2015, a new appointment was made to the position of IT programme leader. A temporary role of assistant to the director was established in January 2015. The marketing and support team of four includes three staff who have been appointed during the last year. Regent International made substantial changes to the Diploma in Business (Advanced) (Level 7) programme in 2012. At the time, NZQA was informed of the changes to the assessment practice for this diploma. Regent International did not, however, inform NZQA of other changes it was making regarding the programme s content or structure. As a result of this, and following a programme monitoring visit in December 2014, Regent International was required to make an 4

application for programme change in the Diploma in Business (Advanced) (Level 7). This application was approved in April 2015. Previous quality assurance history: Due to Regent International s ongoing failure to comply with national external moderation requirements, NZQA withdrew accreditation for the New Zealand Diploma of Business (Level 6) and the NZIM Diploma in Management (Level 5) during 2014. In July 2014, Regent International received approval from NZQA to deliver the New Zealand Certificate in English (NZCEL) (Level 1-4). At the most recent EER conducted in December 2013 (with the EER report published in July 2014), NZQA was Not Yet Confident in both the educational performance and capability in selfassessment of Regent International. Other: Regent International receives no external funding support from government agencies and is fully funded by international students fees. 2. Scope of external evaluation and review The scope of the EER included the following mandatory focus areas: Governance, management and strategy International students Other focus areas selected were: Diploma in Information Technology (with strands in IT Support and Multimedia) (Levels 6 and 7) Diploma in Business (Advanced) (Level 7) These two programmes provide a good representation of overall organisational performance, as they have approximately 75 per cent of all the students enrolled at the PTE. Both programmes were selected as focus areas in the previous EER, which provides a basis for comparison and review of developments during the previous 18 months. 5

3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: http://www.nzqa.govt.nz/providers-partners/registration-andaccreditation/external-evaluation-and-review/policy-and-guidelines-eer/introduction. The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. Regent International was not due for another EER until early 2016. The organisation requested an earlier EER and NZQA agreed to this. The lead evaluator met with the director, acting-director of studies and marketing manager to: discuss the self-assessment materials submitted by Regent International agree on the focus areas finalise the agenda for the site visit. The lead evaluator was provided with an overview of the information systems that underpin Regent International s operations. The evaluation team of three visited Regent International over three days. During the visit, the evaluation team interviewed: The director Acting director of studies (who is also the business programme leader) Marketing manager Four academic staff from the business department Four academic staff from the computing department (including the IT programme leader) Five administrative and support staff 11 current business students 13 current IT students Three external stakeholders. The evaluators reviewed a wide range of management, academic and support materials during and after the visit, both online and in hard copy. 6

Summary of Results Statement of confidence on educational performance NZQA is Not Yet Confident in the educational performance of Regent International Education Group. The reasons for this judgement are as follows: Regent International has extensive data on graduate destinations which shows that learners are completing courses and gaining qualifications. This enables many students to achieve their employment and immigration objectives. Educational achievement is variable across the organisation. Qualification completion rates for the business diploma have been consistently high for several years (95 per cent in 2013 and 2014). However, inconsistent assessment practices during this period reduce the reliability of results data for this programme. The average qualification completion rate for the information technology diplomas for the period 2012 to 2014 is 81 per cent. This is considered adequate when benchmarked against comparable tertiary providers. NZQA is not satisfied that Regent International s assessment practices for the compulsory internship are appropriate, given that this is a capstone course for a level 7 qualification. Regent International has increased the focus on the development of students soft skills through targeted learning activities and assessments (such as presentation and communication skills and managing diversity). Regent International has also recently revised the assessment methodology and materials for the business programme to provide stronger evidence of individual student achievement. It is too early to determine the impact of these initiatives on learner outcomes. Regent International s students report high rates of satisfaction with many aspects of their study experience, including the quality of teaching and support. Monitoring of individual progress and wellbeing is facilitated by Regent International s information systems. The evaluation team identified several instances of non-compliance (related to contact hours and English language entry requirements) and inaccurate information being provided to students, which reduces NZQA s confidence in Regent International s organisational performance. NZQA finds that, as a result of inconsistent assessment practices across the organisation, some achievement data prior to 2015 is not reliable. Serious weaknesses remain in the assessment of the IT internships, which are not being effectively managed. Initiatives to address other areas of weakness in educational performance have been recently implemented or are being actively developed by 7

Regent International. There is insufficient evidence, as yet, of the improved outcomes needed to justify a rating of confident. Statement of confidence on capability in self-assessment NZQA is Confident in the capability in self-assessment of Regent International Education Group. The reasons for this judgement are as follows: Regent International has detailed operational documents which describe processes to ensure consistency and to optimise the collection and use of information using Regent International s information systems. Processes that contribute to teaching and learning are facilitated by Regent International s information systems. Academic staff monitor the commitment levels of students, measured by levels of activity on Whiteboard 1, particularly for poorly performing students. Other examples of improvements include new digital tools for identifying plagiarism and improved management of at-risk students. Regent International s data appears valid, and ongoing systems enhancements have the potential to strengthen the use of data in decisionmaking by governance and management. Stronger processes for understanding and using information to improve educational performance, particularly at organisational level, would enhance capability in selfassessment. Regent International has a framework to systematically determine and address the needs of students and other stakeholders and to evaluate programmes and activities on a regular basis, although some prescribed processes are not consistently applied and the impact on outcomes is not always evident. Student feedback is gathered and analysed regularly. Regent International demonstrates a genuine concern about meeting student needs. However, there is no mechanism to systematically advise students of the outcomes of their feedback. There are some examples of the findings of moderation being used to make changes, and the organisation has provided some professional development on assessment practice. Teachers respond to feedback on their teaching from a variety of sources. However, there are limited opportunities for contracted academic staff to gather regularly to reflect on their teaching effectiveness. 1 Whiteboard is Regent International s learning management system. 8

Processes have been established for module reviews as each part of a programme is concluded. However, these are not always taking place and/or the prescribed evidence is not always gathered. There is no evidence of annual programme reviews. Overall, Regent International has made some meaningful improvements since its previous EER. For example, assessment and moderation practice in the business programme has been reviewed during 2014 and early 2015; staff demonstrate a better understanding and use of the PTE s information systems; there is evidence that advisory members input is valued; and data is analysed and used to inform proposed changes to business programmes. While some of these new initiatives are implemented only recently and are yet to have significant organisational impact, the evaluation team found Regent International is on track to further improve the quality of its internal systems, processes and decision-making. Therefore, NZQA is confident in Regent International s capability in self-assessment. 9

Findings 2 1.1 How well do learners achieve? The rating for performance in relation to this key evaluation question is Adequate. The rating for capability in self-assessment for this key evaluation question is Adequate. Regent International has evidence of students completing programmes, gaining qualifications and acquiring useful skills and knowledge which enable many of them to gain employment. Employment outcomes are strong, although educational achievement is variable across the organisation. Furthermore, the reliability of some achievement data is reduced by inadequate assessment and moderation practices. Qualification completion rates for the business diploma have been consistently high for several years (93 per cent in 2012 3, 95 per cent in 2013 and 2014). However, prior to 2015, assessment was weighted towards group, rather than individual, assignments. A sampling of assessments and moderation reports for 2014 also suggests that some assessment practices were inadequate to ensure the authenticity of assignments or the accuracy of individual assessments. This reduces the reliability of the above results data for this programme. During 2014 and 2015 Regent International has sought to address identified weaknesses in assessment methodology. Changes include increasing the individual assessment weighting to improve measurement of individual learner achievement. These changes should strengthen the reliability of results data for the business diploma. An increased focus on the development and assessment of soft skills (particularly communication and managing diversity) is also being addressed through the revisions, although it is too soon to determine the impact of these changes on learner outcomes. The average qualification completion rate for the information technology diploma (levels 6 and 7) for the period 2012 to 2014 is 81 per cent. This is considered adequate when benchmarked against performance at other PTEs offering IT programmes to international students. Regent International explained that a decline in qualification achievement from 2013 (84 per cent) to 2014 (80 per cent) was due to four (out of 15) students failing to secure a placement for their compulsory internship in the 2014 academic year. NZQA is satisfied that the 2 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 3 The 2012 achievement data is based on six months data only, due to the change in assessment practice which occurred in July 2012. 10

assessment and moderation practices for the taught modules of the IT diploma provide adequate assurance of the validity of results. However, weaknesses in the assessment of the internship, and the lack of internal and external post-assessment moderation for this module, reduces confidence in the internship results and the educational value of the capstone project for the IT diploma. Regent International has data on retention rates, paper pass rates and average marks. It is noted that in 2015 (possibly as a result of the new assessment approach) there is a greater number of re-sits required by business students. The organisation provides additional tuition for these students, and in some instances require students to re-enrol. Regent International has demonstrated correlations between academic success and high attendance rates, and also students use of Whiteboard. Regent International monitors these aspects of student commitment closely, particularly for poorly performing students. The organisation also analysed student achievement in relation to age, country of origin, source of enrolment, etc. It is not apparent what value this type of analysis has, or how it has been used to identify meaningful improvements to educational performance. 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Regent International has gathered extensive data on graduate destinations as evidence of the value of its educational provision for students. This data (including immigration and employment outcomes at six, 12 and 24 months post-graduation) has since 2014 been analysed on the basis of a ratings system to demonstrate the relevance of qualifications for subsequent employment. According to this data, 91 per cent of IT graduates contacted (171 out of a total of 184) are employed in relevant jobs. More than 50 per cent of these are in support roles. For business graduates, 88 per cent (133 out of a total of 151) are employed in relevant jobs, and 29 per cent of those are in supervisory or managerial roles. While this analysis relies on subjective assessment of the relevance and level of positions, it is apparent that many of Regent International s students are achieving their employment and immigration objectives. In addition, students interviewed by the evaluation team stated that they valued the opportunity to broaden their knowledge (including New Zealand-related content) and apply their learning. Regent International seeks external stakeholder feedback and advice in a variety of ways. During 2014 the organisation sought input from industry bodies and 11

employers on what skills employers required. This information has contributed to programme development (for future delivery) (see 1.3). Also in 2014, Regent International convened several meetings of its combined advisory board, although, most recently, independent advisors met separately with each department. In the case of business, most engagement with external stakeholders (primarily external subject and educational consultants) has been focused on academic issues. However, there is broader representation for the IT industry and some evidence that Regent International has followed advice from external stakeholders. This is an improvement from the findings of the previous EER. 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Adequate. The rating for capability in self-assessment for this key evaluation question is Adequate. Regent International s performance in matching the needs of students was inconsistent since the previous EER. During 2014, the business diploma underwent significant review, in part associated with NZQA programme compliance requirements. Regent International has sought external support in redesigning and moderating its assessment tools and has made changes to improve the assessment of individual achievement. Regent International has also increased the focus on soft skills development (see 1.1) and incorporated more New Zealand content. In one module, the class (of more than 50 students) was split into two to provide more opportunity for meeting individual learning needs. Staff manage the composition and size of groups for group assignments to encourage equal participation by all members and to foster cultural diversity. There is still insufficient evidence of the impact of changes made to content and teaching practice, but they have the potential to strengthen the programme and benefit students in the future. The IT diploma remains largely unchanged since the previous EER. The programme requires students to apply their knowledge and skills in a variety of contexts. The compulsory internship provides opportunities for students to develop job search skills and gain confidence and experience in a New Zealand workplace. In some cases students gain ongoing employment through the internship. Regent International has made some changes in response to issues raised at the previous EER in relation to the internship. These changes include: 12

Clarification of the requirements for a proposal A minimum of three meetings by staff with internees 4 (although Regent was unable to provide complete evidence of this occurring) A new marking schedule (although marking criteria are unclear and do not match learning outcomes and there were no criteria indicating how marks were awarded). However, the evaluation team is not convinced that sufficient weighting is placed on the quality or level of the final report in the overall assessment schema. The quality of employer feedback is quite variable and, in some cases, very basic, adding little to an understanding of student progress. Taken together, these issues potentially reduce the relevance and value of the internship for students and employers and reflect an inadequate response to the serious issues raised during the previous EER. During the site visit, the evaluation team noted several inconsistencies related to attendance. In March 2014, Regent International identified non-compliance with the New Zealand Immigration Service s requirements requirement for students enrolled for the business diploma to attend the training provider for 20 contact hours per week and revised its teaching timetable. The evaluation team found a discrepancy between the timetabled hours for students enrolled in the level 7 IT diploma during term 1 of 2015 and tutors reported contracted hours. While this diploma is compliant with immigration requirements, the inconsistency remains of concern. Overall, these examples suggest that Regent International should pay closer attention to attendance requirements. Regent International undertakes to evaluate programmes and activities on a regular basis and to systematically determine and address the needs of students and other stakeholders. Student feedback (primarily, but not only, gathered through regular surveys) is analysed and discussed by management and staff. Students are generally satisfied with their learning experience. Recent improvements include the provision of increased computer and library resources. However, there is no mechanism to systematically inform students of what the organisation does with the feedback they provide. Regent International has established processes for academic staff to review modules after they are delivered. However, these reviews are not always taking place and/or the prescribed evidence is not always gathered. There is no evidence of annual programme reviews leading to systematic improvement. 4 The evaluation team was originally advised during an interview with programme leaders for IT that staff were meeting with internees on a weekly basis and could provide logs of the meetings. 13

It is noted that Regent International has recently developed 12 new business modules (for future delivery, subject to NZQA approval) to better meet stakeholder expectations for graduate skills. 1.4 How effective is the teaching? The rating for performance in relation to this key evaluation question is Adequate. The rating for capability in self-assessment for this key evaluation question is Adequate. Teaching at Regent International is generally effective. Teachers are well qualified in their discipline and most are experienced educators. Classes involve an appropriate variety of activities, including lectures, class discussions and group tasks which engage students in the learning. All teaching and learning resources are available on Whiteboard. Teachers and students relate well to each other and students are generally engaged and motivated. Students receive feedback on their progress and have access to their achievement record on Whiteboard. Tutors are available to answer questions, and additional one-to-one or small group tuition is available, if required, from a staff member appointed for that purpose. Recently, Regent International has focused on improving assessment practice. The evaluation team found that some progress has been made in developing appropriate assessments which provide students and teachers with useful feedback on progress and in using the findings of moderation to make changes. The business diploma master package includes the development of assessment tools, particularly tests and exams to replace group assignments, which have been externally pre-moderated. Regent International has indicated its intention to undertake external post-assessment moderation of business diploma modules. However, there is no schedule to ensure all modules are systematically externally post-moderated. An example of recent post-assessment moderation showed that the template did not provide for the moderator to comment on the accuracy of assessor judgements, although Regent International reported that some discussion had taken place between the programme leader and the moderator. An error in reviewing model answers brings into doubt the thoroughness of the external post-assessment moderation. The evaluation team noted that despite reported changes identified earlier in this report, little improvement is evident in the assessment of the internship in the IT diploma. This is a serious weakness which is not being managed effectively. The staff showed little awareness of the issues identified in 1.3 above, and stated that completion of an internship assured students of a pass grade. Furthermore, an example of a recently assessed internship proposal showed evidence of undetected plagiarism. 14

The horticulture programme is subject to external moderation by Primary ITO (the relevant industry training organisation). Regent International has achieved inconsistent moderation results in recent times, particularly in relation to assessor judgements. Taken together, the evidence shows that further work is required to meet NZQA expectations in regard to assessment and moderation. Regent International monitors teaching effectiveness in a variety of ways. Information gathered from student surveys at the end of each module is discussed with teachers. Examples of changes made in response to student feedback include using video clips to illustrate key concepts and setting a two-week turnaround for marking assessments. Teachers receive adequate induction into the organisation, are provided with all teaching materials and receive a reasonable level of oversight and support. Regular observations from colleagues and performance appraisals provide useful feedback for improving their teaching. There are limited opportunities for academic staff (most of who are part-time contractors) to gather regularly to reflect on their teaching effectiveness. 1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Academic and support staff at Regent International work together to provide students with effective guidance and support to achieve their study goals and to enhance their well-being. The evaluators found that Regent International has established systems for managing attendance and is identifying and responding appropriately to students who are at risk (for personal or academic reasons). Recent software enhancements for the detection of plagiarism, together with tutorial sessions on APA referencing and consistent instructions on all assessments, appear largely effective in minimising academic misconduct. The recently appointed part-time job search coordinator provides individual coaching and support for skills development, as well as contributing to taught modules. It is still too early to make any evaluation of the effectiveness and impact of this service. Documented student feedback and interviews during the EER confirmed high rates of student satisfaction overall with support services although some students indicated they would like more social activities. Regent International has recently established a student council charged with developing ideas for events and out-ofclass activities. Regent International provides prospective students with information on the school and programmes online and through agents. On arrival, students complete 15

appropriate induction and enrolment processes. Information on study objectives gathered at this time is available to teaching staff. Regent International ensures students have access to first-language support staff and a 24/7 contact. A very small number of under-18 students enrol at Regent International (approximately one per year). Appropriate processes appear to be in place to provide suitable support. The evaluation team found Regent International is providing a good level of support and guidance overall. Adequate evidence of annual review of compliance with the Code of Practice for the Pastoral Care of International Students was provided, although the evaluation team noted several minor errors in course handbooks. This suggests that further care should be exercised in the review process. During the site visit, the evaluators found seven instances during July 2014 where Regent International made offers of place based on performance in an English language proficiency test which was not permitted by Regent International s status as a Category 3 education provider at that time. Although Regent International gives advice to students about who they should take concerns to, Regent International does not have a policy and procedure for the PTE to follow in dealing with complaints. These examples indicate that a more systematic approach is required to ensure compliance with relevant regulations and the requirements for ongoing registration with NZQA. 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Adequate. The rating for capability in self-assessment for this key evaluation question is Adequate. Regent International s director and two senior managers work closely together to lead and manage the organisation. They do not have scheduled meetings, but hold frequent ad hoc and issues-based discussions, the outcomes of which are recorded in online logs. They also attend minuted monthly meetings with the programme leaders where they review student progress and discuss programme issues. There is patchy evidence of monthly department reports. In some departments, weekly or fortnightly meetings are also held to share information and address day-to-day issues. The evaluation team observed that accessing records of discussions and decision-making was sometimes problematic for Regent International and selfassessment processes are not always transparent. A realistic business plan reflects Regent International s current situation accurately and sets out a clear direction for the future. It is not clear how this direction is shared with the wider organisation or how staff contribute to change and 16

improvements. Many of the staff are part-time contractors and there are few, if any, opportunities for whole staff gatherings. Regent International has extensive documentation which describes processes to ensure consistency and to optimise the collection and use of information, using Regent s information systems. There is some evidence that the information systems contribute to the quality of teaching and support services. 5 For example, teachers described using Whiteboard activity logs for monitoring poor performers and support staff get regular reminders regarding visa and insurance dates. Both groups of staff identify and provide support to students categorised as at risk. The evaluators identified several instances of non-compliance during the EER (refer 1.3 and1.5). Further attention is required to ensure processes are consistently and accurately completed. Improvement of processes and extending the data capture and analysis has been a key part of Regent International s approach to self-assessment since the previous EER. There is abundant data available on student achievement and graduate destinations. The analysis of Regent International s educational performance is generally descriptive. There is limited evidence of management using these findings insightfully to make comprehensive improvements. Efforts to establish data compatibility as a basis for benchmarking with other providers have been unsuccessful to date. Enhanced data on graduate outcomes has generated some information which may have value for future development of the business diploma. Overall, a greater focus on reflection and evaluation is required for increasing capability in self-assessment and improving educational performance. 5 Access to Regent International s operations system (Regent CRM) is currently restricted to management and support staff. 17

Focus Areas This section reports significant findings in each focus area, not already covered in Part 1. 2.1 Focus area: Governance, management and strategy The rating in this focus area for educational performance is Adequate. The rating for capability in self-assessment for this focus area is Adequate. 2.2 Focus area: International student support The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. 2.3 Focus area: Diploma in Information Technology (with strands in IT Support and Multimedia) The rating in this focus area for educational performance is Adequate. The rating for capability in self-assessment for this focus area is Adequate. 2.4 Focus area: Diploma in Business (Advanced) The rating in this focus area for educational performance is Adequate. The rating for capability in self-assessment for this focus area is Good. 18

Recommendations NZQA recommends that Regent International: Review and improve the management, academic rigour and assessment of the IT internship (including internal and external moderation) as a high priority Sustain the improvement initiatives in the business programmes and extend them to the IT and horticultural programmes, with a particular focus on assessment and moderation practices Consider how the relevance and value of data-gathering and analysis can be improved, particularly through a greater focus on reflection and evaluation, to inform decision-making for enhanced organisational performance Consider how staff with different employment arrangements can fully utilise the benefit of its information systems Ensure records of discussions and decision-making are purposeful and easily accessible by all relevant staff. 19

Appendix Regulatory basis for external evaluation and review External evaluation and review is conducted according to the External Evaluation and Review (EER) Rules 2013, which are made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. Self-assessment and participation in external evaluation and review are requirements for maintaining accreditation to provide an approved programme for all TEOs other than universities. The requirements are set through the NZQF Programme Approval and Accreditation Rules 2013, which are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. In addition, the Private Training Establishment Registration Rules 2013 require registered private training establishments to undertake self-assessment and participate in external evaluation and review, in accordance with the External Evaluation and Review Rules (EER) 2013, as a condition of maintaining registration. The Private Training Establishment Registration Rules 2013 are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the External Evaluation and Review (EER) Rules 2013. The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website (www.nzqa.govt.nz). The External Evaluation and Review (EER) Rules 2013 are available at http://www.nzqa.govt.nz/assets/about-us/our-role/rules/eer-rules.pdf, while information about the conduct and methodology for external evaluation and review can be found at http://www.nzqa.govt.nz/providers-partners/external-evaluation-andreview/policy-and-guidelines-eer/introduction/. NZQA Ph 0800 697 296 E qaadmin@nzqa.govt.nz www.nzqa.govt.nz 20