Napier University APRIL Enhancement-led institutional review

Transcription

1 Napier University APRIL 2006 Enhancement-led institutional review

2 Preface The Quality Assurance Agency for Higher Education (QAA) exists to safeguard the public interest in sound standards of higher education (HE) qualifications and to encourage continuous improvement in the management of the quality of HE. To do this, QAA carries out reviews of individual higher education institutions (HEIs) (universities and colleges of HE). In Scotland this process is known as Enhancement-Led Institutional Review (ELIR). The Agency operates equivalent but separate processes in Wales, England and Northern Ireland. Enhancement-led approach Over the period 2001 to 2003, QAA, the Scottish Higher Education Funding Council, Universities Scotland and representatives of the student body worked closely together on the development of the enhancement-led approach to quality in Scottish HE. This approach, which was implemented in academic year , has five main elements: a comprehensive programme of review at the subject level, managed by the institutions improved forms of public information about quality, based on addressing the different needs of the users of that information including students and employers a greater voice for student representatives in institutional quality systems, supported by a national development service (known as the student participation in quality scotland - sparqs - service); a national programme of enhancement themes, aimed at developing and sharing good practice in learning and teaching in HE ELIR involving all of the Scottish HEIs over a four-year period, from to The ELIR method embraces a focus on: the strategic management of enhancement; the effectiveness of student learning; and student, employer and international perspectives. QAA believes that this approach is distinctive in a number of respects: its balance between quality assurance and enhancement; the emphasis it places on the student experience; its focus on learning and not solely teaching; and the spirit of cooperation and partnership which has underpinned all these developments. Nationally agreed reference points ELIR includes a focus on institutions' use of a range of reference points, including those published by QAA: the Scottish Credit and Qualifications Framework (SCQF) the Code of practice for the assurance of academic quality and standards in higher education subject benchmark statements, which describe the characteristics of degrees in different subjects Guidelines on preparing programme specifications, which are descriptions of what is on offer to students in individual programmes of study. Programme specifications outline the intended knowledge, skills, understanding and attributes of a student completing that programme. They also give details of teaching and assessment methods and link the programme to the SCQF.

3 Conclusions and judgement within ELIR ELIR results in a set of commentaries about the institutions being reviewed. These commentaries relate to: the ability of the institution's internal review systems to monitor and maintain quality and standards at the level of the programme or award. This commentary leads to a judgement on the level of confidence which can reasonably be placed in the soundness of the institution's current and likely future management of the quality of its programmes and the academic standards of its awards. The expression of this judgement provides a point of tangency between the ELIR method and other review methods operating in other parts of the UK. The judgement is expressed as one of: broad confidence, limited confidence or no confidence the institution's arrangements for ensuring that the information it publishes about the quality of its provision is complete, accurate and fair the effectiveness of the institution's approach to promoting an effective learning experience for students the combined effect of the institution's policies and practices for ensuring improvement in the quality of teaching and learning the effectiveness of the institution's implementation of its strategy for quality enhancement. The ELIR process The ELIR process is carried out by teams comprising three academics, one student and one senior administrator drawn from the HE sector. The main elements of ELIR are: a preliminary visit by QAA to the institution in advance of the review visit a Reflective Analysis document submitted by the institution three months in advance of the second part of the review visit a two-part review visit to the institution by the ELIR team; Part 1 taking place five weeks before Part 2, and Part 2 having a variable duration of between three and five days depending on the complexity of matters to be explored the publication of a report, 20 weeks after the Part 2 visit, detailing the commentaries agreed by the ELIR team. The evidence for the ELIR In order to gather the information on which its commentaries are based, the ELIR team carries out a number of activities including: reviewing the institution's own internal procedures and documents, as well as the Reflective Analysis institutions prepare especially for ELIR asking questions and engaging in discussions with groups of relevant staff talking to students about their experiences exploring how the institution uses the national reference points.

4 The Quality Assurance Agency for Higher Education 2006 ISBN X All QAA's publications are available on our website Printed copies are available from: Linney Direct Adamsway Mansfield NG18 4FN Tel Fax qaa@linneydirect.com Registered charity number

5 Contents Introduction 1 Style of reporting 1 Method of review 1 Background information about the institution 2 Institution's strategy for quality enhancement 3 Internal monitoring and review of quality and standards and public information 3 Overview of the institution's internal arrangements for assuring the quality of programmes and maintaining the standards of its academic awards and credit 3 Internal approval, monitoring and review 4 External examining 8 Collaborative provision 9 Research degrees 10 Overview of the use made of external reference points for assuring quality and standards 10 Commentary on the ability of the institution's internal review systems to monitor and maintain quality and standards 11 Overview of the institution's approach to ensuring that the information it publishes about the quality of its provision is complete, accurate and fair 12 Commentary on the institution's arrangements for ensuring that the information it publishes about the quality of its provision is complete 13 The student experience 13 Overview of the institution's approach to engaging students in the assurance and enhancement of the quality of teaching and learning 13 Overview of the institution's approach to the promotion of effective student learning 15 Overview of the institution's approach to the promotion of the employability of its students 18 Commentary on the effectiveness of the institution's approach to promoting an effective learning experience for students 19 Effectiveness of the institution's strategy for quality enhancement 20 Overview of the institution's approach to managing improvement in the quality of teaching and learning 20 Overview of the linkage between the institution's arrangements for internal quality assurance and its enhancement activity 20 Overview of the institution's approach to recognising, rewarding and implementing good practice in the context of its strategy for quality enhancement 21 Commentary on the combined effectiveness of the institution's policies and practices for ensuring improvement in the quality of teaching and learning 22 Commentary on the effectiveness of the institution's implementation of its strategy for quality enhancement 23 Summary 23 Background to the institution and ELIR method 23 Overview of the matters raised by the review 23 Commentary on the ability of the institution's internal review systems to monitor and maintain quality and standards 24 Commentary on the institution's arrangements for ensuring that the information it publishes about the quality of its provision is complete,

6 accurate and fair 25 Commentary on the effectiveness of the institution's approach to promoting and effective learning experience for students 25 Commentary on the combined effect of the institution's policies and practices for ensuring improvement in the quality of teaching and learning 25 Commentary on the effectiveness of the institution's implementation of its strategy for quality enhancement 26

7 Enhancement-led institutional review Introduction 1 This is the report of an Enhancement-led Institutional Review (ELIR) of Napier University (the University) undertaken by the Quality Assurance Agency for Higher Education (QAA). QAA is grateful to the University for the willing cooperation provided to the ELIR team. 2 The review followed a method agreed with Universities Scotland, student bodies and the Scottish Higher Education Funding Council (SHEFC), and informed by consultation with the Scottish higher education sector. The ELIR method focuses on: the strategic management of enhancement; the effectiveness of student learning; and the use of a range of reference points. These reference points include: the Scottish Credit and Qualifications Framework (SCQF), the Code of practice for the assurance of academic quality and standards in higher education (Code of practice), published by QAA, subject benchmark information, and student, employer and international perspectives. Full detail on the method is set out in the Handbook for Enhancement-Led Institutional Review which is available on QAA's website. Style of reporting 3 ELIR reports are structured around three main sections: internal monitoring and review of quality and standards and public information; the student experience; and the effectiveness of the institution's strategy for quality enhancement. Each section contains a sequence of 'overviews' and 'commentaries' in which the ELIR team sets out its views. The first commentary in the first main section of the report leads to a single, formal judgement included within the ELIR reports on the level of confidence which can be placed in the institution's management of quality and standards. The judgement is intended to provide a point of tangency with the methods of audit and review operating in other parts of the UK where similar judgements are reached. In the second and third main sections of the report, on the student experience, and the effectiveness of the institution's quality enhancement strategy, there are no formal judgements, although a series of overviews and commentaries are provided. These are the sections of the ELIR report which are particularly enhancement-focused. To reflect this, the style of reporting is intended to address the increased emphasis on exploration and dialogue which characterises the team's interaction with the institution on those matters. The reader may, therefore, detect a shift in the style of reporting in those sections, and this is intended to emphasise the enhancement-led nature of the method. Method of review 4 The University submitted a Reflective Analysis (RA) which set out the University's strategy for quality enhancement, its approach to the management of quality and standards and its view of the effectiveness of its approach. Other documents available to the ELIR team with the RA included: the institutional profile at 16 February 2006; the University's Strategic Plan: Creating the Future, The University towards 2010 (Summary); the undergraduate prospectus (2006); the postgraduate prospectus (2006); Enhancing reputation and supporting growth: The University's Strategy for Quality Enhancement; Promoting Learning for Achieving Potential: The University's Strategy for Learning, Teaching and Assessment; Delivering academic quality: A guide to the quality framework (2005); The Educational Development Staff Development Programme ; Enhancement themes programme of assessment workshops: a compendium of reports and case studies by staff from Napier University's supporting quality in Scottish higher education; and the Napier Students' Association programme representatives handbook. The RA provided the focus for the review and was used to develop a programme of activities by the ELIR team to provide a representative illustration of the way the University approaches the management of quality, enhancement and academic standards. 5 The University submitted four case studies with its RA. These took the form of descriptions page 1

8 Napier University of the University's approach to: the student contribution to the life of the University; student retention; technology supported learning; and professional development in support of learning and teaching. 6 A working group, chaired by the Director of Quality Enhancement Services, led the preparation for ELIR, including the production of the RA. The working group consisted of: the chairs of faculty quality committees; assistant faculty managers (quality); the directors of Lifelong Learning, Communications and Information Technology, Educational Development and Research; a sabbatical officer from the Students' Association; and two staff members with experience of external review activity. The working group was also responsible for engaging staff with the ELIR process across the University. 7 The ELIR team visited the University on two occasions: the Part 1 visit took place on the 1 and 2 March 2006, and the Part 2 visit took place between the 3-6 April During Part 1, senior staff gave a number of presentations to the ELIR team in relation to: recent institutional developments; the University's approach to quality assurance and enhancement; the University's implementation plan for learning, teaching and assessment, and quality enhancement strategies; the University's research degrees provision; supporting student success; and student involvement in quality assurance and enhancement. In the remainder of Part 1, the team met senior staff with responsibility for managing quality assurance and enhancement activity across the University, a group of staff involved in internal subject review and a group of students including those with a representational role at programme and University levels. These meetings enabled the team to explore a range of matters, many of which had been raised by the University in the RA. The ELIR team considered that the RA tended to be descriptive rather than evaluative, but that nevertheless it provided a useful starting point for the review. 9 During Part 1, the University made available a set of documentation which had been identified within the RA and a small amount of supplementary information identified during the course of the visit. This enabled the ELIR team to develop a programme of meetings and to identify a set of documentation for the Part 2 visit in order to provide a representative view of the University's approach to assuring and enhancing quality, and maintaining the standards of its awards. 10 The ELIR team comprised: Dr R Condie; Professor T Cryer; Dr A Eadie; Mr C Pirie (Reviewers); and Mr P Probyn (Review secretary). The review was coordinated on behalf of QAA by Dr J Ross, Assistant Director, QAA Scotland. Background information about the institution 11 The University was described in the RA as a 'modern and vibrant higher education institution dedicated to providing educational services which are relevant to the needs of today's students and employers'. The University began as Napier Technical College in 1964, becoming Napier College of Commerce and Technology in 1974, Napier Polytechnic in 1987, and finally Napier University in The provision of over 250 programmes of study is managed and delivered by schools within a faculty structure. At the time of the ELIR, the University consisted of four faculties: Arts and Social Science; Engineering and Computing; Health and Life Sciences; and Napier Business School. In November 2005, the University's Academic Board approved a new three faculty academic structure which is due to take effect from September 2006: Engineering, Computing and Creative Industries; Health, Life and Social Sciences; and Napier Business School. Educational provision is based on three main, and four smaller, campuses in the city of Edinburgh. In , Napier University had a student population of approximately 12,600, of whom 79 per cent were full time and 21 per cent part-time. Of the student population, some 82 per cent were undergraduates, 16 per page 2

9 Enhancement-led institutional review cent taught postgraduates and 2 per cent postgraduate research students. In the same year, there were 1,645 staff, of whom 784 were on academic or research contracts. Institution's strategy for quality enhancement 12 The University describes the aims of its quality enhancement strategy as being to: strengthen the University's reputation for providing high quality learning that: is student-focused and flexible; enhances student employability; and is valued by students, employers, professional bodies and the community; and contributes to the Scottish vision for a high quality, higher education sector support the achievement and delivery of the University's strategic goals further embed within the University a culture of critical self-evaluation leading to continuous quality enhancement and management of risk assure standards through rigorous self-regulation and review, the effective use of external reference points and appropriate peer and student involvement in review processes at all levels. Internal monitoring and review of quality and standards and public information Overview of the institution's internal arrangements for assuring the quality of programmes and maintaining the standards of its academic awards and credit 13 The RA emphasised the integration of the University's approach to quality assurance and quality enhancement, indicating that there is a developmental relationship between the two, with enhancement being firmly embedded in the institution's assurance mechanisms. A key source of guidance on quality assurance and enhancement is the University's Quality Framework, which sets out the institution's procedures for maintaining academic quality and standards, and for quality enhancement, covering the quality cycle from module and programme design and delivery to programme review and evaluation. As such, the Quality Framework is a central source of academic guidance for staff and it also provides information for students, such as the arrangements for the recognition of prior learning for credit, and the research students' handbook. 14 The RA confirmed that over the last decade the University has devolved authority for assuring the quality of its taught programmes of study and for maintaining the standards of its awards, to the faculties, through the faculty boards. The arrangements for overseeing research degrees provision are an exception to this, with the RA indicating that these arrangements had become more centralised since 2002 (see below, paragraph 56). 15 The RA indicated that the University's mechanisms to monitor and evaluate the effectiveness of the devolved system are inherent within its structures. These structures include its committees, and a number of individuals who have key roles in managing assurance and enhancement in teaching and learning, including vice principals and deans, assistant faculty managers (quality), and programme and module leaders. The remit of the committees and these individuals is supported strategically and operationally by two central services: Quality Enhancement Services and Educational Development. 16 The Academic Board has responsibility for overseeing the academic work of the University. The University's Quality Committee has specific responsibility and devolved authority from the Academic Board for the processes of assuring quality and academic standards. A Learning, Teaching and Assessment sub-committee of the Quality Committee was established in A Regulations Sub-Committee of the Academic Board is accountable for ensuring that the regulations remain 'valid, reliable and up to date', while the Quality Forum has authority page 3

10 Napier University to determine, on behalf of the Academic Board, whether requests from faculty boards for regulatory exemptions in relation to taught provision can be approved. 17 The four faculty boards each report to the Academic Board. Reporting to each faculty board, there is a hierarchy of faculty quality committees and school committees (or equivalent) which link programme and University level quality management responsibilities. Each academic programme has a board of studies which is responsible to the relevant faculty board for the development, planning, management, operation and enhancement of the programme. In addition, each programme has a student-staff liaison committee. The RA indicated that cross membership between boards of studies, school, faculty and University level committees facilitates discussion on relevant matters including the sharing of good practice, and that student representation is a feature of the committees at all levels (see below, paragraph 78). 18 Quality Enhancement Services provides professional support for the Quality Committee and its associated sub-committees and working groups, for example through QES staff membership of a range of approval and review teams operating within the Quality Framework. It fulfils this role by developing, monitoring and reviewing institutional advice, guidance, policy and procedures for the use of faculties, schools and individuals. 19 The Planning and Intelligence Unit supports schools in their quality assurance activities through the provision of management information, and the ELIR team learned that the quality of this information has been recently improved, and is recognised by schools to make a positive contribution to their quality assurance work. The Planning and Intelligence Unit also supports the work of the governing body in identifying the key performance indicators necessary to monitor the University's progress in implementing its Strategic Plan. 20 The ELIR team was able to confirm that the University engages in a regular evaluation of its quality processes. Thus, year on year modifications are made, some of which are internally driven (for example, a review of module and programme approval processes) and some of which are in response to the changing external context (for example, the Scottish Funding Council's expectations relating to internal review at the subject level). 21 The ELIR team learned that, following the appointment of a new Principal in 2003, the University is undertaking a programme of significant review and change with respect to its governance, senior management arrangements and responsibilities, and committee structure. A fundamental academic portfolio review is also being undertaken. Such changes include a new faculty structure with a greater reliance on a matrix of responsibilities and management duties across three new faculties (see above, paragraph 11). It is planned that each new faculty will have associate deans with specific responsibility for academic development, academic quality and customer service, and research and knowledge transfer. A review of the Academic Board and its sub-committees, in order to provide better support for the management of assurance and enhancement and the attainment of the University's strategic objectives, was in progress at the time of the ELIR visit, and scheduled to report in May Internal approval, monitoring and review Module approval 22 Module approval is carried out at school level, with heads of school overseeing processes undertaken by the schools' quality, learning, teaching and assessment committees (or their equivalent). Module approval events involve peer academic judgement of the proposals, and ensure their alignment with school and faculty plans. A key outcome of each event is a definitive module descriptor, presented in a University-approved format. The ELIR team was able to confirm that the module approval process, as described in the RA, was implemented in accordance with the requirements of the University, and adheres to the relevant section of the Code of practice, published by QAA. page 4

11 Enhancement-led institutional review Programme approval 23 Programme approval is devolved to faculty boards and is a two-phase process. The first phase is a formal scrutiny of any proposal by a school panel. The second phase involves consideration of the proposal by a further panel at the faculty level. Either panel may impose conditions before further progress, and can make recommendations for further development to the programme team. The responses to such conditions and recommendations are built into a final programme document, which includes a programme specification, before being signed off by the validation panel convener. 24 From its consideration of sample programme approval material, the ELIR team considered that the documentation provided by the programme team was comprehensive, and comprehensive briefing notes were provided to the validation panel members. The resulting validation panel reports addressed each of the areas defined in the checklist produced by QES. The ELIR team was able to confirm that the operation and nature of the programme validation process was sound, and that it involved the appropriate use of external reference points, and included external involvement on the validation panel. Annual module and programme review 25 The process of annual module and programme review and evaluation is undertaken by module and programme leaders, respectively. The Quality Framework provides templates with prompts to support module and programme leaders in their work, and the process leads to the production of an action plan at module and programme level. These are submitted to the head of school who takes their content into account in an annual school report, which is submitted to the faculty board and the faculty executive. A faculty summary report is considered annually by the Quality Committee. 26 The RA acknowledged that the annual review process could be considered to be bureaucratic. The expectation of the current academic restructuring (see above, paragraph 21) is that the current management of devolved authority and associated reporting mechanisms will be strengthened through the implementation of a matrix structure at faculty level. The University anticipates that these changes should bring greater clarity and value to the operation of the review process. 27 The ELIR team was able to study information relevant to the annual review of a particular module, and of a specific programme at master's level. The module review was undertaken by the module leader, using University produced pro-forma documentation. The review included consideration of student feedback, and the comments of the external examiner. The approach invited confirmatory responses and the proforma's prompt to provide a critical appraisal received only limited response. The module report was incorporated into the school summary report of the process, as were the annual reviews of all the modules delivered by the school. The school report did not have an emphasis on critical analysis, or on the specific actions to be taken in order to ensure continuous improvement. 28 Annual programme monitoring reports are approved at the level of the school quality enhancement committee. While programme teams are encouraged, in completing the proforma, to evaluate the operation of the programme to date, and in doing so, reflect on successes achieved and plans for the future, those reports considered by the ELIR team were characterised by descriptive and confirmatory responses. Periodic programme review 29 The Quality Framework requires that each approved programme of study is subject to formal periodic review within a five-year cycle. This process is described as offering programme teams the opportunity to discuss with a team of peers the appropriateness and relevance of the design, learning, teaching and assessment approaches, student support mechanisms, and general management of the programme. Such review teams include internal members from another faculty, and members with relevant page 5

12 Napier University expertise from outside the University. The RA described the outcomes of the periodic review process as mirroring those of the validation process. 30 The ELIR team considered the documentation relating to the review of a Master's programme. This illustrated that the twophase approach adopted for programme approval was also a key feature of programme review. At the scrutiny stage, the team comprised University staff, who were provided with guidance produced by the faculty in question. This guidance was explicit in its requirements, including on the nature of the external reference points that were to be addressed through the process. The scrutiny team's report was restricted to the provision of a decision, a set of conditions and a list of recommendations. The programme review documentation was, however, comprehensive, including a wide range of appendices. It was noted in this case that the requirements of the relevant professional and statutory body were particularly emphasised by the programme review documentation, which based much of its confidence on the inclusion of the University in an inter-institution consortium focussed on a specific range of academic provision. The internal reference points, and some of the expected sectoral reference points, were given less prominence. The ELIR team noted that external benchmarking of academic standards did not feature significantly in the documentation. After the phase two review event the report produced was again limited to the team's decision with its conditions and recommendations. 31 In broad terms, the requirements of the University processes for the periodic review of programmes were confirmed by the documentation seen by the ELIR team. However, the team formed the view that the documentation concentrated on the assurance aspects of the process and lost some of the opportunities it offered for the evaluation and development of the provision. Internal annual quality audits 32 The University undertakes internal audits of its processes and practices on a thematic basis. Themes for the annual cycle of internal audit activity are suggested currently by the Quality Forum, for the approval of the Quality Committee, with the latter taking responsibility for overseeing the audit outcomes. Quality Enhancement Services has a key role in managing and administering the internal audits. Audit teams are drawn from academic and support staff with appropriate expertise. The outcomes of the audits are reported to the Quality Committee and faculty boards in the form of an annual overview of audit activity. The RA indicated that the Quality Committee monitors the action taken at faculty level in response to the audit by receiving regular progress reports. At the time of the ELIR visit, the University had recently undertaken two internal audits, and the outcomes of these were still to be reported. References to previous audits were among the committee proceedings seen by the ELIR team. While being able to confirm the regular audit activity described in the RA, the team was not clear, from the documentation made available to it, what influence the activity had on the development of provision or processes. School review 33 Following the implementation of a revised school review methodology in 2002, and its refinement in the handbook of school review in 2005, the University had completed nine school reviews up to December The stated primary purposes of the school review process are to: support quality assurance and enhancement at the subject level; support the achievement of the University's strategic goals; identify, celebrate and share areas of good practice; and assist schools in developing an appropriate response to identified areas for further development. The process, undertaken by a review team, involves meetings with students and considers a school report and action plan, both of which are prepared for the exercise. The outcomes of the process, in the form of full and summary reports and the action plan, are considered by the relevant faculty board and student-staff liaison committees (see below, paragraphs 80-81). page 6

13 Enhancement-led institutional review 34 Sample school review documentation made available to the ELIR indicated that the key purpose of the process was to address the Scottish Funding Council's guidelines for internal reviews at the subject level. The school review handbook is an extensive document describing a two-phase process that focuses in phase one on academic standards, delivery, management and research and scholarship and, in phase two, on future strategy and developments in the area. 35 Phase one of the school review takes place over one and a half days during which the review team, including three external members, meets with staff and students and tours the relevant facilities. A self-evaluation document is prepared by the school which, in the case seen by the ELIR team, was comprehensive although dominated by description rather than evaluation. The team noted the sample report provided was also descriptive in nature, but it listed review findings in terms of strengths and further developments. 36 The phase two focus on future strategy and developments, together with the response of the school to phase one, are considered during a further event at which one of the original external members attends. A further report is produced together with an action plan. 37 The reports and action plan are received and noted by the Quality Committee and the Academic Board respectively. From its consideration of the core documentation relating to a number of school reviews, the ELIR team formed the view that the process was variable in relation to the rigour of the action plans produced, and retained a focus on assurance rather than placing the emphasis on continuous improvement for students. 38 The school reviews undertaken in any session become the subject of an overview report and schedule for future events, which are presented to the Quality Committee. The ELIR team noted that the overview report is in the main confirmatory in style, but also identifies key matters considered to be of institutional importance. It provides suggested actions for the following year but, in the main, these relate to matters of process rather than relating to key planning issues. The actions expected of schools and the University, and the monitoring of any actions taken, were not obvious to the team even where the issues raised related to the need for certain key planning decisions. Evaluation of the University's processes 39 It was evident to the ELIR team that the extensive range of quality assurance processes in operation are recognised and adhered to by staff at module, programme, school and faculty levels. The extent to which staff members are able to discern the specific and distinctive purpose served by each of the processes, and the nature of the developmental outcomes to be expected from such processes, are less clear. 40 From its consideration of all the available documentation, the ELIR team was able to confirm the University's description that the outcomes of its quality procedures pass through a multi-layered process before being considered at University level. Because reports are summarised at each stage, the information available at the institutional level about its provision is limited, and this is illustrated by the committee minutes, which provide only limited detail of the discussion and evaluation on which decisions are based. This formal reporting is complemented by the outcomes of other formal and informal groups and networks which add relevant information, albeit on a less systematic basis. It was also evident, from the documentation produced at school and faculty levels, that although the processes used to assure quality and standards are the same across the institution, the ways in which they are recorded and therefore demonstrated, differed between the schools and faculties, thus providing modest but noticeable variation in the level of detail provided to University-level bodies. 41 The ELIR team formed the view that the University's current processes are focussed on quality assurance rather than enhancement. It seemed to the team that the Quality Framework provides clear guidance and page 7

14 Napier University assistance on the processes, and encourages engagement by staff with the confirmatory nature of quality assurance, but this does not always lead to evaluation or self-critical analysis. The documentation seen by the team illustrated a wide range of quality assurance processes that yielded extensive description based on input measures (for example, programme content), and focused much less on developmental analysis of outputs (for example, analysis of student attainment), and the identification of opportunities for improvement. However, the team was able to identify, particularly from discussions with staff, that continuous improvements in the quality of provision are achieved, but that the quality system and its operation is not always central to that process; rather quality enhancement relies upon the work of relatively informal groups and networks operating mainly at school and faculty levels. 42 The ELIR team noted the overall context of significant change with structural, management and strategic goals undergoing revisions, only some of which were in process at the time of ELIR. The team concluded that for the proposed developments to have their anticipated impact, specific mechanisms for the management of change are needed in order to build capacity in those areas defined as being of priority in the University's Strategic Plan. External examining 43 The University appoints external examiners for modules and programmes, to report and comment on assessments which contribute to awards at SCQF levels 7 to 11. The operation of the criteria for their appointment, approved by the Academic Board in March 2004, is the responsibility of Quality Enhancement Services. The RA indicated that an annual one-day induction event is provided for new external examiners, together with an external examiner's handbook, which is available on the University's website and in hard copy. 44 The responsibility for dealing with external examiners' reports is devolved by the Academic Board to Quality Enhancement Services which monitors that external examiners receive a response to those matters the University describes as 'major issues'. The online report form contains a specific section which invites external examiners to raise any major issues. Action taken in response is addressed through the annual monitoring process. The Quality Committee receives a report on the themes arising from external examiners' reports. 45 From the sample documentation provided, the team considered that the information provided to external examiners was adequate in order for them to be fully informed of their expected role. The external examiners' report template was designed to elicit the necessary information from examiners in order that they can discharge their duties in relation to the protection of the standards of awards. The external examiners' reports seen by the team were completed by examiners with the expected level of detail and examiners had taken the opportunity to make their views clear. 46 The ELIR team noted that the responses to external examiners' reports were restricted to those issues identified in the section of the report identified as 'major issues' and other aspects of the external examiners' reports and the comments they contained were not subject to systematic response. 47 The flowchart of the process provided by the University covering the receipt and actions arising from external examiner reports included steps up to, but not beyond, the provision of responses to external examiners' reports. The mechanisms through which the reports passed in order to formulate the responses varied between the cases provided to the ELIR team. In some cases the responses were considered at school and/or faculty level committees before their approval and dispatch to the external examiners, whereas in other cases the responses were reported to the relevant committee after being sent to the external examiner. The team noted that faculty quality committees considered all responses and, in some cases, these were collated on a school basis. 48 At University level, the Quality Committee receives periodic reports highlighting the 'major page 8

15 Enhancement-led institutional review issues' raised by external examiners. The ELIR team noted that those issues arising from the external examiners reports for were still being followed up at the meeting of the Quality Committee held in February 2006 when matters had been overtaken by the identification of major issues arising from the reports. In recognition of the difficulties produced by the process, the Quality Committee is considering ways in which closure of the annual cycle could be expedited in future, and the University is encouraged to progress this. 49 The ELIR team considered that, in general, the external examining arrangements are in line with the Code of practice, although the University's oversight of responses to the reports, and to the actions taken by schools in response to the reports, and the institution's own analysis of the key issues emerging, is somewhat slow. The responses to external examiner comments seen by the team were variable in quality, and were restricted to the major issues identified by the external examiners. The University's external examining process is assurance-led and is not able, in its current form, to provide timely institutional perspectives upon which enhancements to programmes might be based. 50 Through discussion and the consideration of regulatory matters it became clear to the ELIR team that the University did not exercise close scrutiny of the activities of examination boards: for example, there is no central administrative attendance at examination boards; examination board minutes are not collected by the centre; there is no institution level committee at which examination board recommendations are considered; and responses to external examiners' reports are considered retrospectively by the University. The University does not undertake an institutional overview and evaluation of the outcomes achieved by students across the University's provision, in relation to, for example, the distribution of outcomes at programme, school or faculty levels, or in relation to degree classifications achieved. The University is encouraged to consider how the further security of the academic standards of its awards might be achieved through closer University-level scrutiny of its examination arrangements. Collaborative provision 51 The University has 22 collaborative agreements in place, 11 of which are with overseas partners. The University requires that all such agreements, and associated programme information, are formally approved by the relevant dean of faculty, the Director of Quality Enhancement Services, the Director of Finance, and the University Secretary, before finally being approved by the Principal. Since 1999, the Quality Committee has undertaken a full review of the University's overseas provision, which has led to the introduction of stringent requirements for the approval of such arrangements. These include a strengthened process of partner evaluation, greater clarity of the responsibilities of the respective partners, senior staff involvement in the scrutiny of programme documentation prior to approval, and the introduction of a University Code of Practice for collaborative provision as part of the Quality Framework. The RA indicated that the creation of the Collaborative Partnerships Committee in , with devolved authority from the Quality Committee to oversee the overall management of collaborative partnerships, is expected to further strengthen the University's identification and dissemination of good practice in this area of activity. 52 The University's arrangements for approval, monitoring and review (see above, paragraphs 22 to 38) apply to collaborative provision, with first cohort review as an additional requirement. The ELIR team considered documentation relating to the approval process for an international collaborative programme from initial consideration to first cohort review. 53 The first cohort review process is undertaken by an internal team, which receives relevant documentation, and meets with programme staff of the University and from the overseas partner institution. The process page 9

16 Napier University involves meetings with students, and is based firmly on clear documentation and analysis provided by the programme team. The review confirms the regular attendance of University staff at the overseas partner institution and the performance of the students is considered. The review team's report includes an overall conclusion and a number of recommendations. The ELIR team formed the view that the first cohort review process provides a timely and robust means by which the University could be assured of the success, or otherwise, of any new overseas collaborative provision. 54 The ELIR team was assured by the relevant staff that there was strong control exercised by the partners and the University over the promotional and other materials produced and published in relation to collaborative provision. Overall, the team formed the view that academic, and other, risks to the University arising from collaborative activity were being well managed. 55 In addition to the 22 formal collaborative agreements, the University has a number of collaborative arrangements with local and other further education colleges in the UK. These relate primarily to the promotion of opportunities for further education students to progress through defined pathways into higher education at the University, subject to the successful attainment of the entry qualifications expected of all applicants. The ELIR team considered that these arrangements were managed securely. Research degrees 56 The RA stated that responsibility for the standards of research degree awards, and the quality of research students' learning experience, rests with the University's Higher Degrees Committee, a subcommittee of the Research and Knowledge Transfer Committee. The Higher Degrees Committee is supported by the Research Office, and is chaired by the University's Director of Research. Each faculty has a research degrees committee, and a faculty director of research/research committee chair who leads activities and developments, liaises with schools and reports on student progress to the University Committee as appropriate. The RA indicated the University's expectation that the appointment of a Vice- Principal (Research, Commercialisation and Knowledge Transfer), together with the anticipated appointment of associate deans (research and knowledge transfer), would help to ensure the consistent application of processes for monitoring the progress of research students. Overview of the use made of external reference points for assuring quality and standards 57 The RA indicated that the University has embraced the Academic Infrastructure, including the Code of practice, subject benchmark statements, the SCQF, programme specifications and the requirements of professional, statutory and regulatory bodies. In this overall context, the RA stated that, by ensuring account has been taken of the various parts of the academic infrastructure within its quality processes, the University can be confident that it has made effective use of external reference points in order to inform and influence its management of quality and standards. The ELIR team noted that both the Quality Framework, and the procedures it sets out, take account of a variety of external and internal reference points. Code of practice 58 In , the University undertook a mapping exercise of its arrangements against the Code of practice. The exercise identified a small number of instances where the University could not fully demonstrate that account had been taken of the intention of the precepts. These precepts covered the areas of postgraduate research programmes (see below, paragraphs 82; 95 to 96) and students with disabilities (see below, paragraphs 93 to 94), and the University is continuing to progress work to address these matters. page 10

17 Enhancement-led institutional review Programme specifications 59 The RA indicated that programme specifications were introduced in in preparation for QAA academic (subject) review, and further updated in to meet the Scottish Funding Council's expectations in relation to teaching quality information. The University's programme development process includes the production of a programme specification, and guidance on this is contained in the University's Quality Framework. Programme specifications are contained in student handbooks, and are lodged centrally although, as yet, they have not been made available online. 60 The programme specifications seen by the ELIR team were broadly in line with University and sector expectations. The team considered that the style and content of the programme specifications made them an accessible and useful source of information for students. Progress files 61 The RA recognised that the University has been comparatively slow in implementing personal development planning (PDP). PDP is due to be implemented University-wide in (see below, paragraph 107). Progress files are not currently a feature of undergraduate provision and are only in use in a restricted area of the University's postgraduate research provision. Professional, statutory and regulatory bodies and employers 62 At the time of the ELIR, almost 100 of the University's programmes were accredited by professional, statutory or regulatory bodies (PSRBs). The University's Quality Framework contains an expectation that, when planning a new programme, programme teams will consider whether professional body accreditation is appropriate. The Quality Framework does not contain any requirement for programme submissions to PSRBs to be approved by the University, or for PSRB reports to be received by University level committees. The ELIR team formed the view that this was a lost opportunity for institutional learning and self-evaluation. Arising from its discussions with staff, and consideration of documentation, the team came to the view that, in many cases, PSRB accreditation is used by programmes and schools as a reference point for confirming academic standards, and as an alternative to a clear articulation of academic standards by the University. Commentary on the ability of the institution's internal review systems to monitor and maintain quality and standards 63 The University's internal review systems are appropriate to enable it to monitor and assure the standards and quality of the University's provision. The University, through its Quality Framework, has comprehensive and fully documented processes for programme approval, monitoring and review which cover all taught provision, and an additional subjectrelated process of school review. All of these processes make use of, and encourage, adherence to a range of external reference points. Collectively the processes making up the Quality Framework adhere to the precepts of the Code of practice. The University is aware of a small number of cases where adherence to all the precepts offered by the Code has not been achieved, and is addressing this. 64 There is a risk that the operation of the processes set out in the Quality Framework could become mechanistic and that the collective burden of the processes may be heavy with overlaps and repetitions. The distinctive purpose of each process is insufficiently clear for each of the processes to be recognised as bringing its own added value to the assurance of quality and standards. There would be benefit, as the University progresses its wider reforms, in undertaking a critical analysis of the purpose and significance of the components of the Framework. 65 The approach taken by module and programme leaders, schools and faculties to their reporting of the application of the processes produces documentation that is mainly confirmatory in nature. As such the page 11