Report of External Evaluation and Review

Report of External Evaluation and Review New Zealand Institute of Sport Limited trading as New Zealand Institute of Sport (NZIS) Highly Confident in educational performance Highly Confident in capability in self-assessment Date of report: 27 October 2016

Contents Purpose of this Report... 3 Introduction... 3 1. TEO in context... 3 2. Scope of external evaluation and review... 5 3. Conduct of external evaluation and review... 5 Summary of Results... 7 Findings... 9 Recommendations... 22 Appendix... 23 MoE Number: 8490 NZQA Reference: C22680 Dates of EER visit: 20-22 June 2016 2

Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: Type: New Zealand Institute of Sport Limited trading as New Zealand Institute of Sport Private training establishment (PTE) First registered: 20 March 1998 Location: Delivery sites: (i.e. campuses) Courses currently delivered: 105 Waterloo Quay, Wellington (Westpac Stadium) 382 Manukau Road, Greenland, Auckland 66B Wharenui Road, Riccarton, Christchurch Foundation level: NZIS Certificate in Sport and Recreation (SportsZone) (Level 2) NZIS Certificate in Sport Studies (Level 3) Other certificate level: NZIS Certificate in Personal Training (Level 3) NZIS Pre-police Proficiency Certificate (Level 3) Diploma level: NZIS Diploma in Sports Management and Exercise Prescription (Level 5) NZIS Diploma in Sport (Advanced Sport Management) (Level 6) 3

NZIS Diploma in Sport (Advanced Exercise Prescription) Level 6) Code of Practice signatory: Number of students: Yes Domestic: 704 equivalent full-time students 32 per cent Māori 36 per cent Pasifika 93 per cent under 25 years of age. International: 10 equivalent full-time students Number of staff: Scope of active accreditation: Distinctive characteristics: 42 full-time equivalents, and seven part-time staff. http://www.nzqa.govt.nz/providers/ details.do?providerid=849044001 NZIS delivers qualifications for the sport, fitness and recreation industry. It is registered with NZQA as a Pasifika provider, and has a strong focus on providing education to Pasifika and Māori learners. NZIS is also one of New Zealand s larger private training establishments, with three main campuses. Recent significant changes: Previous quality assurance history: In December 2015, NZIS was purchased by Intueri Education Group. At the time of the evaluation the previous management team remained in charge of most aspects of the PTE. NZQA undertook an external evaluation and review of NZIS in 2012. At that evaluation NZQA was Highly Confident in NZIS s educational performance and Highly Confident in the PTE s capability in self-assessment. 4

2. Scope of external evaluation and review The scope of the external evaluation and review was fourfold: Governance, management and strategy International learners Diploma programmes (levels 5 and 6 combined) Youth Guarantee programmes offered. This selection ensured coverage of mandatory focus areas and all the main programmes of NZIS, covering the bulk of its student population. All three main campuses were visited to ensure coverage of the selected scope. 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: http://www.nzqa.govt.nz/providers-partners/registration-andaccreditation/external-evaluation-and-review/policy-and-guidelines-eer/introduction. The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. The evaluation team consisted of three evaluators and an active observer. The team visited NZIS over 20-22 June 2016. (On 20 and 22 June the team was based at the Wellington head office of NZIS. On 21 June two evaluators visited the Christchurch campus, while the third evaluator and the observer visited the Auckland campus.) Interviews (in person, groups, or via telephone) were held with: Senior leadership team (chief executive, deputy chief executive, and chief executive of Intueri Education Group) Academic advisory board (internal) Academic advisory board (external member) Tutors (including the site leaders) (total in excess of 20) Youth Guarantee learners (total in excess of 30) Diploma learners (total in excess of 30) International learner and international pastoral support officer 5

Stakeholders (including church pastor, police officer, gym manager, Canterbury rugby representative, Canterbury rugby league representative, iwi organisation representative) Graduates (interviews and telephone calls). The evaluation team is confident that a sufficient range of staff, stakeholders and students were interviewed for evaluation and review purposes. During the site visit, NZIS provided the evaluation team with a range of operational documentation, including strategic planning materials, management meeting minutes, quality management policies, enrolment information and policies, student achievement records, student evaluations, collated analysis of student satisfaction levels, and course design, delivery and assessment materials. This documentation complemented the self-assessment information that had been pre-submitted by the trust. The evaluators undertook a sample review of all materials tabled. 6

Summary of Results Statement of confidence on educational performance NZQA is Highly Confident in the educational performance of New Zealand Institute of Sport Limited trading as New Zealand Institute of Sport. This evaluation has found strong evidence that NZIS is meeting the most important needs of the students and other stakeholders. Primarily, this is demonstrated by the high percentage of learners who complete formal qualifications and go on to higher study or employment. (In 2015, the qualification completion rate for Youth Guarantee enrolled learners was 76 per cent, and for learners enrolled in higherlevel qualifications it was 80 per cent. Overall, 75 per cent of graduates from 2015 either gained employment or went on to further study at NZIS.) In addition, NZIS has also been able to demonstrate that its programmes meet the other needs of students, including enhanced educational confidence, leading to improved wellbeing. NZIS has a paramount focus on learner guidance and support. A family focus permeates NZIS, reflecting the ethos that every learner can learn. This means learners feel safe and included within their classes. Further to this, NZIS has provided clear evidence that its educational outcomes are valued by its stakeholders, including learners and their families, employers and the sports industry, and targeted communities (including Māori and Pasifika groups). Cultural competency and connectivity is highly valued at NZIS, and a range of engagement mechanisms are used to uphold the cultural wellbeing of learners. In addition, NZIS has good evidence that its delivery processes are of a high quality, and this positively contributes towards educational outcomes. In particular, the practical learning experiences and positive ethos of the organisation are highly valued by learners, some of whom have not experienced significant success in previous formal education settings. The competencies of tutors (knowledge, experience, and rapport) is a major contributing factor in ensuring high delivery quality, alongside of clear and robust managerial planning and systems. NZIS provides a positive working environment for its staff, and draws upon clear workplace policies and procedures, including a quality management system to guide its tertiary education provision. Strategic planning is also evident, and appropriately focused on measured learning and wellbeing outcomes. 7

Statement of confidence on capability in self-assessment NZQA is Highly Confident in the capability in self-assessment of New Zealand Institute of Sport Limited trading as New Zealand Institute of Sport. NZIS has a useful and comprehensive range of self-assessment systems in place, which collectively allow for the evaluation of its programmes and delivery. This includes an ongoing series of student, employment and stakeholder surveys and tutor to management reporting. One strong example of practice in this area is the annual bringing together of staff from multiple campuses to discuss past delivery and develop best-practice methodologies for future educational provision. Ongoing reflection against teaching and delivery plans is then catered for in weekly staff meetings which are specifically orientated around programme delivery and student needs. Through such fora there is clear evidence that meaningful information is collated and used to make programme improvements. In particular, NZIS has been able to reflect on how well its programmes lead learners to further education and employment outcomes over an extended period of time. NZIS has large cohorts of Māori and Pasifika learners, and accordingly very consciously also monitors learning outcomes for these two groups of students. To boost the achievement of Māori and Pasifika learners, NZIS uses its analysis of learning outcomes to provide targeted support, including pastoral care services to students at risk of under-achievement. Overall, the self-assessment summary and supporting evidence provided for this evaluation demonstrates high-level analytical review and reflection by NZIS, covering all areas of its business. 8

Findings 1 1.1 How well do learners achieve? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. In evaluating how well learners achieve, this evaluation has taken into consideration the extent to which learners complete formal qualifications, acquire useful skills and knowledge (developing their cognitive abilities), and improve their overall wellbeing with enhanced abilities and attributes. Through this evaluation, NZIS has presented strong evidence that the significant majority of its learners are achieving the formal qualifications they seek. Summary details of such outcomes within Youth Guarantee and Student Achievement Component-funded (SAC) programmes delivered by NZIS are shown in the tables below. The salient feature noted by the evaluation team is the high qualification completion rate. For Youth Guarantee programmes, the qualification completion rate was 74 per cent, 81 per cent and 76 per cent for 2013, 2014 and 2015 respectively. This is consistently above the contractual target of 65 per cent. For SAC-funded programmes, the overall qualification completion rate was 81 per cent, 82 per cent, and 80 per cent for 2013, 2014 and 2015 respectively. This is consistently above the Tertiary Education Commission contract target of 70 per cent. In sum, around four out of five learners who enrol with NZIS will successfully complete their qualification of choice by the end of their year of enrolment. This is a high level of qualification completion, above sector averages, and consistent throughout course levels (from New Zealand Qualifications Framework level 2 to level 6). Data presented by NZIS is also consistent with that of the Tertiary Education Commission sighted for this evaluation. 1 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation s activities. 9

Table 1. Focus Area 3 level 5 diploma qualifications completion and progression results, 2012-2015 New Zealand Institute of Sport - summary results 2012 2013 2014 2015 Total SAC placements level 5 142 136 156 140 Course completions 85% 87% 85% 82% Qualification completion 78% 85% 82% 75% Retained in study 83% 79% 84% 81% Table 2. Focus Area 3 level 6 diploma qualifications completion and progression results, 2012-2015 New Zealand Institute of Sport - summary results 2012 2013 2014 2015 Total SAC placements level 5 91 69 78 86 Course completions 86% 85% 83% 90% Qualification completion 81% 79% 81% 86% Retained in study 79% 79% 77% 80% Table 3. Focus Area 4 Youth Guarantee programmes qualification completion and progression results, 2013-2015 2 New Zealand Institute of Sport - summary results 2013 2014 2015 Total Youth Guarantee placements 190 277 273 3 Course completions 80% 81% 79% Qualification completion 74% 81% 76% Retained in study 66% 76% 76% In considering its qualification completion rates, course completion rates, and retention levels, NZIS sets outcome targets and consciously benchmarks its performance against other, similar providers and sector averages. This is good practice and means the organisation is continuously aware not only of the results it is achieving (including tracking current cohorts of learners), but is also aware of how well the sector overall is performing. Data is also disaggregated by campus 2 2012 data is not included as Youth Guarantee was a new funding stream from 2013 onwards. 3 This is the Certificate in Sport and Recreation (Level 2), offered in Wellington, Christchurch and Auckland. 10

and course, giving the management team a very detailed purview of student achievements. Achievement of Māori and Pasifika learners NZIS has large cohorts of Māori and Pasifika learners. Accordingly, it very consciously monitors learning outcomes for these two groups of students. (NZIS notes that around one-third of its learners in 2015 identified as Pasifika, and nearly one-third identified as Māori.) Learning outcomes for these two cohorts are high. Average qualification attainment rates for Māori learners (across all funding streams) were 70 per cent, 76 per cent, and 73 per cent for the years 2013, 2014 and 2015 respectively. Average qualification attainment rates for Pasifika learners (across all funding streams) were 73 per cent, 78 per cent, and 73 per cent for the years 2013, 2014 and 2015 respectively. Qualification achievement rates were also high in the highest areas of diploma study offered by NZIS, with Māori learners having qualification completion rates in 2015 of 80 per cent, and 89 per cent at diploma levels 5 and 6 respectively. For Pasifika learners, the qualification completion rates in 2015 were 70 per cent, and 88 per cent at diploma levels 5 and 6 respectively. Further evidence of the personal educational gains made by individual learners, particularly within the Youth Guarantee stream of funding, was also presented for this evaluation (discussed under Focus Area 2.4). On this basis, the evaluation team accepts the evidence of NZIS that these positive results reflect specific Māori and Pasifika engagement activities used in delivery (discussed under Findings 1.5). Other outcomes In addition to formal qualification outcomes, learners interviewed consistently stated that their study with NZIS was providing them with useful skills and knowledge for desired work opportunities (discussed further under Findings 1.2). They also commented that they considered the programmes enhanced their overall wellbeing, and in particular they valued the progressive learning levels available within NZIS. These comments were consistent across students in different programmes, and correlated with the views of learners who had already graduated from NZIS. Student evaluations (including aggregated information) sighted by the evaluation team also triangulates this viewpoint from current and past learners. This evaluation, therefore, finds that educational performance in relation to learner achievement is strong, and there are no significant gaps or weaknesses. Self-assessment of learner achievement occurs at a number of levels. Foremost, tutors work with learners one-to-one to set personalised individual learning plans. NZIS also allocates a home tutor for learners, and this person is responsible for learner outcome monitoring. Weekly meetings are held between tutors to discuss and report on student achievement matters, and ensure all learners are progressing appropriately through their studies. This approach provides a 11

comprehensive picture of direct and indirect issues to address and mitigate in order to ensure successful learner outcomes. (Learners are also aware of their progress to date, and next achievement milestone.) Results are then tracked through to managerial level, so that NZIS has ongoing oversight of both individuals and cohorts. This data was very clearly articulated within the organisation s summary of self-assessment. This is followed by formal and regular board reporting of cohort outcomes. Accordingly, in this context, this evaluation finds strong evidence of effective self-assessment that leads to improvements. 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. In evaluating the value of outcomes for key stakeholders, including learners, this evaluation has taken into consideration knowledge dissemination beyond NZIS, and engagement strategies and outcomes with targeted communities and stakeholders such as employers. Through this evaluation, NZIS has demonstrated strong evidence of the value of its educational offer to stakeholders. The evaluation team has identified three main groupings of stakeholders: learners and their families, employers and industry, and community/iwi groupings. In regard to learners and their families, beyond qualification success these stakeholders desire future study and employment opportunities. In addition to this, many seek to be able to progress their own sporting careers, both during study and beyond. NZIS has demonstrated that its programmes can be customised to accommodate the individual sporting aspirations of learners across a wide range of sporting codes (for example bodybuilding, rugby, netball, etc). NZIS deliberately employs staff with diversity in sporting experiences to accommodate this. In addition, given the popularity of rugby, a rugby academy operates on some campuses (which supplements regular learning activities). NZIS, however, also recognises that employment for many of its graduates may not be in sports-related fields (as the career of professional athletes can be short, and only a few may reach such levels). Accordingly NZIS places a strong emphasis on pathways to further study, and employment outcomes. In regard to Youth Guarantee learners, NZIS focuses on ensuring initial success and inclusiveness, noting that many of these students have come from other educational settings where they felt disconnected and disengaged from learning. 12

The mix of practical experiences in courses assists in providing this hook into education and higher learning. (Students commented that inter-campus sport exchanges and field trips were consistent highlights that helped to motivate their overall attendance and participation in NZIS.) In regard to diploma learners, NZIS has kept records of former students (as far as possible) who have either enrolled or completed degree-level study at the Open Polytechnic of New Zealand or Victoria University in Melbourne. NZIS has active relationship agreements with these institutions, which formally recognise the learning outcomes students achieve at NZIS. At the time of the evaluation, 17 former NZIS students had graduated at degree level, and a further 11 were enrolled in degree-level study. NZIS monitors the employment outcomes of its learners. 4 As with its other selfassessment practices, the information gathered is disaggregated into courses, campuses, targeted learner cohorts (Māori and Pasifika), level of study, and industry/non-industry work. This is good practice. A report on labour market outcomes was provided as supplementary material to the self-assessment summary. This report highlights that, overall, 75 per cent of graduates from 2015 are known to be in employment or further study. Accordingly, based on the labour market report, this evaluation finds post-study sports sector engagement is a strong outcome that NZIS has rightly identified as one of its successes. The evaluation team was also able to meet with (or call via telephone) employers and sport sector representatives. Interviewees outlined that the training provided by NZIS met their needs and provided them with graduate-level employees with the necessary prerequisite knowledge and skills to work in their sector/organisations. NZIS was considered to be a quality provider whose graduates had appropriate theoretical and practical knowledge for employment. NZIS also works consciously to foster and develop relationships among its wider community. To this end, the evidence shows that diverse stakeholders provide linkages between NZIS and the community. This included, for example, a community police officer (who valued a free community sports programme developed and run by students at NZIS Christchurch), a church pastor in Wellington (who valued the health aspects of the service NZIS was delivering to the Pasifika community), and rugby and rugby league representatives (who valued the self-confidence NZIS was able to provide to its learners within their sports codes). In addition, NZIS makes its facilities available for community use, and its staff and students contribute to community events, which promote a healthy lifestyle. The evaluation team also accepts the NZIS claim that it is one of only a few providers of its type and focus within its communities of interest, and notes that it engaged with 4 Typically via an annual telephone survey conducted in February/March of the following year. 13

59 secondary schools in 2015 to delivery STAR and Gateway programmes. 5 Based on these findings, NZIS has provided clear evidence that its education programmes are highly valued within its community. From an academic perspective, NZIS maintains an external academic committee, (a member of which was interviewed and advised that they can and do contribute to course design). The PTE has also recently participated in sector research focused on best practice in support of Pasifika learners, and has developed linkages with two degree-level providers within its sector. 6 (Evidence showing a list of 10 learners who had progressed their studies towards degree-level study was also provided. 7 ) NZIS also makes its facilities (and staff) available to graduates who are studying higher-level qualifications at other institutions, to support their ongoing success. NZIS has also actively participated in sector reforms, including the current review of qualifications within the sports domain. This means NZIS is highly familiar with the changes occurring within its sector relating both to qualification offers and to funding systems. Accordingly, this evaluation finds that educational performance in relation to stakeholder outcomes is strong. The evaluation also finds evidence of effective self-assessment of stakeholder outcomes, as provided within the summary of selfassessment, supplementary materials on labour market outcomes, further education datasheets, and academic advisor committee minutes. 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. In evaluating the matching of programmes and activities with the needs of learners and stakeholders, this evaluation has taken into consideration the extent of ongoing needs analysis, the maintaining of relevance, developments in subject content, the incorporation of relevant teaching practice and technologies, and the adequacy and appropriateness of resources. 5 These are secondary/tertiary education sampling programmes. 6 The research project was the 2014 Le So otaga (Bridging the Divide), conducted by Ako Aotearoa. 7 At Victoria University (Melbourne). 14

Learners highly value the positive learning environment and the practical hands-on type of learning offered by NZIS, whereby theory and practice are integrated. Learners commented that this type of environment allows them to achieve. For Youth Guarantee learners in particular, NZIS appeared to offer learners their first experience of educational success. Course and tutor evaluations completed by learners support this finding. 8 (NZIS undertakes such surveying on a regular basis, and collates data for managerial discussions.) The founding philosophy that every learner can learn remains central to NZIS, whereby the challenge is on educators to find the right hooks to engage learners and ensure they can succeed in their endeavours. That is, there is a strong match between the educational philosophy and ethos, mode of delivery and curriculum offered, with the needs and aspirations of learners. The Tertiary Education Commission numeracy and literacy tools are used to preassess the levels of learners on entry. This is good practice. Written learning materials typically comprise worksheets developed by NZIS. These are held on a server, allowing students ready access. Students and staff consider that these materials are appropriate for the learning outcomes being sought. Textbooks are available on site. Physical resources are a significant feature of NZIS delivery. The redeveloped Christchurch campus ensures that a large gym and field areas are readily accessible to learners. For Auckland and Wellington-based learners, physical resource areas are away from the main campus (a vehicle is available to transport learners). Returning students in Wellington commented that the organisation of the campus had improved since mid-2015, and timetabling issues they had raised had been addressed. To maintain currency, programmes are reviewed after delivery. All NZIS staff meet annually to discuss programme design and incremental improvements. This is good self-assessment practice. 1.4 How effective is the teaching? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. In evaluating the effectiveness of the teaching, consideration has been given to whether learning environments are planned and structured for the benefit of students, whether activities engage learners, and whether they provide 8 Regular means for all courses offered within overall programmes. 15

opportunities to apply knowledge and skills. The evaluation has also considered whether assessment practices are valid, sufficient, fair and transparent. The evaluation has found the teaching to be highly effective. This is primarily due to the quality of the core tutors and their passion for their learners and their sector. The evaluation has also found that programmes are well planned and learning is appropriately staggered, allowing for natural progressions. Notwithstanding, there is scope for NZIS to have its programme delivery (and assessments) reflected on by an impartial third party. Tutors have relevant industry knowledge and teaching knowledge/qualifications. The specialist sporting knowledge of individual tutors is also drawn upon to enhance delivery. Tutors maintain a strong rapport with students, which was confirmed in interviews and student surveys. (In particular, the pastoral support offered by tutors involved in Youth Guarantee, and by other staff with dedicated support roles, was highly effective in ensuring learners remained on track with their educational goals.) NZIS requires its tutors to hold a qualification in adult education, and the majority have such qualifications already (Certificate in Adult Education or similar); if not, they hold higher qualifications, such as a Bachelor of Teaching. This evaluation has found evidence of a strong induction programme for staff, and a focus on ensuring effective delivery to Māori and Pasifika learners. Professional development opportunities are also routinely made available to staff; and organisational-wide development is also offered. As an example, at the end of last year (2015), training on managing student self-harm was provided, noting that NZIS works with a large number of vulnerable youth. For this evaluation, samples of student assessments were sighted, including internal moderation. The exemplars provided and explanations of assessment processes were sound. Internal moderation of around 25 per cent of course assessments is undertaken. External moderation outcomes are generally positive. Evidence was provided showing that the Skills Active Industry Training Organisation considers NZIS met their requirements in 2015, and there were no significant matters arising in relation to NZQA moderation of other unit standards. This evaluation finds that overall educational performance in relation to teaching effectiveness is strong across all campuses. This evaluation also finds that selfassessment is strong, which has resulted in the delivery of useful and relevant professional development for staff, and continuous improvements in educational delivery. 16

1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. In evaluating how well learners are guided and supported, this evaluation has taken into consideration whether learners are provided with comprehensive and timely study information, continued support appropriate to their needs, an inclusive learning environment, and minimal barriers to learning. NZIS has a paramount focus on learner guidance and support. Key to this is a focus on understanding the family context (particularly for younger learners), and to this end families are involved in interview processes. Gaining this extra information about learners home lives is good practice, and allows tutors to have a wider appreciation, perspective and, where appropriate, responsiveness to external issues learners may face (such as transport or accommodation issues). NZIS also engages staff specifically in pastoral support roles, to assist learners overcome direct and indirect barriers to learning. An internal family focus permeates NZIS. Actions and events such as an orientation programme, sporting exchanges, graduation ceremonies, wearing a uniform, cultural groups (such as kapa haka groups), and competitions between campuses and classes are used successfully to foster a family atmosphere at NZIS. Learners feel safe and included in their classes. While there are no learner complaints, those interviewed knew who to talk to, and how to broach matters should any issues arise. NZIS maintains clear information for learners on course requirements and ensures all learners are given appropriate information on learning modules and related assessment progress. This was supported by evidence from learners who indicated they knew exactly what they needed to do to next, and through the year, to complete their qualifications. NZIS has identified that, for some learners, meals and a place to study can be barriers for their learners. To mitigate this, one NZIS campus runs a breakfast club, and others provide homework clubs. (There are also individual study spaces, computer labs and WiFi available on campuses.) In addition, NZIS has both Māori and Pasifika leaders to hand to provide supplementary pastoral and cultural support. Learners indicated that they felt a strong sense of family and community demonstrated at NZIS, and were enjoying their studies. This evaluation therefore finds that educational performance in relation to student guidance and support is strong, and there are no significant gaps or weaknesses. The evaluation also finds good evidence of effective self-assessment that leads to improvements in this area that is, the ethos and culture of inclusiveness at NZIS 17

is deliberate, and led by the senior leadership team, and that strategies to enhance this are a focus in self-assessment and leadership meetings. 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Excellent. This evaluation has considered how well managers and governors respond to change, use results of self-assessment for improvements, have a clear organisational purpose, balance innovation and continuity, provide effective leadership, allocate resources, ensure policies are legal and ethical, and value staffing contributions. NZIS provides a positive working environment for its staff, meaning tutors indicated that they feel valued and that they are aware their work makes a positive difference to the learners. Staff job descriptions and appropriate performance reviews processes are also in place. NZIS has clear workplace policies and procedures, including a quality management system for its tertiary education provision. The chief executive attests that there are no legal or ethical issues arising. There is also a robust strategic plan, and the educational component is appropriately focused on measured learning and wellbeing outcomes. The chief executive, deputy chief executive, and three site leaders are also highly regarded by their staff and students; and demonstrate a high level of competency in ensuring NZIS meets the expectations of internal and external stakeholders. Organisational self-assessment shows strengths in process management and strategic planning. For example, there are regular surveys of students, stakeholders and staff; and information is gather and considered, resulting in nuanced changes in programme delivery. NZIS also employs a senior academic manager to strengthen data management and interpretation for self-assessment and educational performance purposes. In December 2015, NZIS was sold to another company. This ownership change means both the chief executive and the deputy chief executive who have been leading NZIS since establishment in 1998 are presently scheduled to conclude their duties in March 2017. This will be a significant change for NZIS. This evaluation has sought to inquire about the planning relating to this transition. The evaluation team notes that the transition plan outlined in interviews with management focuses largely on maintaining and enhancing the status quo in regards to educational delivery. Both the existing and incoming managerial team 18

consider NZIS is on track, and therefore at this time planning reflects the utilisation of present systems and processes. This includes maintaining a focus on present cohorts youth, Māori and Pasifika learners. The evaluation team is satisfied that at the date of the evaluation the leadership of NZIS remains strong, and that suitable planning is underway to manage future transition arrangements. This evaluation finds that educational performance in relation to governance and management is highly effective and there are no significant gaps. Self-assessment in this area is also effective in bringing about a wide range of improvements across NZIS s operations. 19

Focus Areas This section reports significant findings in each focus area, not already covered in Part 1. 2.1 Focus area: Governance, management and strategy The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 2.2 Focus area: International learners The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. NZIS has only a small number of international learners (up to 10 people in 2015). These learners attend primarily because of a strong focus on a particular sporting area, and the desire to extend their knowledge relating to that sport; for example, an interest in rugby. One evaluator reviewed materials relating to international learners, and held a discussion with current international learners and with the pastoral liaison officer for international learners. There are no issues arising at this time. NZIS, however, has indicated that growing international enrolments may be a future option for the organisation, which is currently being explored. Should this occur, further evaluative emphasis on this focus area may be warranted. 2.3 Focus area: Diploma programmes The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. 2.4 Focus area: Youth Guarantee The rating in this focus area for educational performance is Excellent. The rating for capability in self-assessment for this focus area is Excellent. Youth Guarantee learners have limited prior success in formal education settings, and the focus of the programmes funded is the acquisition of unit standards resulting in NCEA or other formal achievements, allowing for retention in education, and ultimately progression to higher levels of study. (There is no labour market 20

outcome sought from the Tertiary Education Commission in the provision of funding for these programmes.) One of the key Youth Guarantee programmes of NZIS is the Certificate in Sport and Recreation (SportsZone stream). NZIS data shows that 80 per cent of 2015 graduates from this programme were retained in further studies (well above contractual commitment levels). This positive finding further correlates with interviews with specific groups of these students, who indicated that through the programme their social and educational confidence and self-esteem had been enhanced, and that they feel engaged and purposeful in education (and in some cases, life more generally). Learners also expressed a desire to continue studying in the sports and recreation sector, in order to ultimately gain employment. 21

Recommendations NZQA recommends that NZIS: In addition to industry inputs, consider also engaging a third party as a critical friend with specialist subject knowledge to review programme design and delivery (including student resources) Explain to its stakeholder groups how it intends to maintain its current ethos and nature as a Pasifika provider under the new ownership structure. 22

Appendix Regulatory basis for external evaluation and review External evaluation and review is conducted according to the External Evaluation and Review (EER) Rules 2013, which are made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. Self-assessment and participation in external evaluation and review are requirements for maintaining accreditation to provide an approved programme for all TEOs other than universities. The requirements are set through the NZQF Programme Approval and Accreditation Rules 2013, which are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. In addition, the Private Training Establishment Registration Rules 2013 require registered private training establishments to undertake self-assessment and participate in external evaluation and review, in accordance with the External Evaluation and Review Rules (EER) 2013, as a condition of maintaining registration. The Private Training Establishment Registration Rules 2013 are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the External Evaluation and Review (EER) Rules 2013. The report identifies strengths and areas for improvement in terms of the organisation s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website (www.nzqa.govt.nz). The External Evaluation and Review (EER) Rules 2013 are available at http://www.nzqa.govt.nz/assets/about-us/ourrole/rules/eer-rules.pdf, while information about the conduct and methodology for external evaluation and review can be found at http://www.nzqa.govt.nz/providers-partners/external-evaluation-and-review/policyand-guidelines-eer/introduction/. 23

NZQA Ph 0800 697 296 E qaadmin@nzqa.govt.nz www.nzqa.govt.nz 24