University of Essex NOVEMBER Institutional audit

Transcription

1 University of Essex NOVEMBER 2003 Institutional audit

2 Published by Quality Assurance Agency for Higher Education Southgate House Southgate Street Gloucester GL1 1UB Tel Fax Web Quality Assurance Agency for Higher Education 2004 ISBN All the Agency's publications are available on our web site Printed copies are available from: Linney Direct Adamsway Mansfield Nottinghamshire NG18 4FN Tel Fax qaa@linneydirect.com

3 Preface The Quality Assurance Agency for Higher Education (the Agency) exists to safeguard the public interest in sound standards of higher education (HE) qualifications and to encourage continuous improvement in the management of the quality of HE. To do this the Agency carries out reviews of individual HE institutions (universities and colleges of HE). In England and Northern Ireland this process is known as institutional audit. The Agency operates similar but separate processes in Scotland and Wales. The purpose of institutional audit The aims of institutional audit are to meet the public interest in knowing that universities and colleges are: providing HE, awards and qualifications of an acceptable quality and an appropriate academic standard; and exercising their legal powers to award degrees in a proper manner. Judgements Institutional audit results in judgements about the institutions being reviewed. Judgements are made about: the confidence that can reasonably be placed in the soundness of the institution's present and likely future management of the quality of its programmes and the academic standards of its awards; the reliance that can reasonably be placed on the accuracy, integrity, completeness and frankness of the information that the institution publishes, and about the quality of its programmes and the standards of its awards. These judgements are expressed as either broad confidence, limited confidence or no confidence and are accompanied by examples of good practice and recommendations for improvement. Nationally agreed standards Institutional audit uses a set of nationally agreed reference points, known as the 'academic infrastructure', to consider an institution's standards and quality. These are published by the Agency and consist of: The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ), which include descriptions of different HE qualifications; The Code of practice for the assurance of academic quality and standards in higher education; subject benchmark statements, which describe the characteristics of degrees in different subjects; guidelines for preparing programme specifications, which are descriptions of the what is on offer to students in individual programmes of study. They outline the intended knowledge, skills, understanding and attributes of a student completing that programme. They also give details of teaching and assessment methods and link the programme to the FHEQ. The audit process Institutional audits are carried out by teams of academics who review the way in which institutions oversee their academic quality and standards. Because they are evaluating their equals, the process is called 'peer review'. The main elements of institutional audit are: a preliminary visit by the Agency to the institution nine months before the audit visit; a self-evaluation document submitted by the institution four months before the audit visit; a written submission by the student representative body, if they have chosen to do so, four months before the audit visit; a detailed briefing visit to the institution by the audit team five weeks before the audit visit; the audit visit, which lasts five days; the publication of a report on the audit team's judgements and findings 20 weeks after the audit visit. The evidence for the audit In order to obtain the evidence for its judgement, the audit team carries out a number of activities, including: reviewing the institution's own internal procedures and documents, such as regulations, policy statements, codes of practice, recruitment publications and minutes of relevant meetings, as well as the self-evaluation document itself; reviewing the written submission from students; asking questions of relevant staff; talking to students about their experiences; exploring how the institution uses the academic infrastructure. The audit team also gathers evidence by focusing on examples of the institution's internal quality assurance processes at work using 'audit trails'. These trails may focus on a particular programme or programmes offered at that institution, when they are known as a 'discipline audit trail'. In addition, the audit team may focus on a particular theme that runs throughout the institution's management of its standards and quality. This is known as a 'thematic enquiry'. From 2004, institutions will be required to publish information about the quality and standards of their programmes and awards in a format recommended in document 02/15 Information on quality and standards in higher education published by the Higher Education Funding Council for England. The audit team reviews progress towards meeting this requirement.

4 Contents Summary 1 Introduction 1 Outcome of the audit 1 Features of good practice 1 Recommendations for action 1 Undergraduate programmes in computing, history, law and sociology 1 National reference points 2 Main report 4 Section 1: Introduction: the University of Essex 4 Collaborative provision 5 Background information 5 The audit process 5 Developments since the previous academic audit 6 Section 2: The audit investigations: institutional processes 6 The institution's view as expressed in the SED 6 The institution's framework for managing quality and standards, including collaborative provision 7 The institution's intentions for the enhancement of quality and standards 9 Internal approval, monitoring and review processes 10 External participation in internal review processes 12 External examiners and their reports 13 External reference points 14 Programme-level review and accreditation by external agencies 15 Student representation at operational and institutional level 15 Feedback from students, graduates and employers 16 Progression and completion statistics 17 Assurance of the quality of teaching staff, appointment, appraisal and reward 18 Assurance of the quality of teaching through staff support and development 18 Assurance of the quality of teaching delivered through distributed and distance methods 20 Learning support resources 20 Academic guidance, support and supervision 21 Personal support and guidance 22 Collaborative provision 23 Section 3: The audit investigations: discipline audit trails 24 Section 4: The audit investigations: published information 31 Student's experience of published information and other information available to them 31 Reliability, accuracy and completeness of published information 31 Findings 34 The effectiveness of institutional procedures for assuring the quality of programmes 34 The effectiveness of institutional procedures for securing the standards of awards 35 The institution's use of the academic infrastructure 36 The effectiveness of institutional procedures for supporting learning 36 The outcomes of the discipline audit trails 37 The utility of the SED as an illustration of the institution's capacity to reflect upon its own strengths and limitations and to act upon these to enhance quality and standards 39 Commentary on the institution's intention for the enhancement of quality and standards 39 The reliability of information 39 Features of good practice 40 Recommendations for action 40 Appendix 41 The University of Essex's response to the audit report 41

5

6 Institutional Audit Report: summary Summary Introduction A team of auditors from the Quality Assurance Agency for Higher Education (the Agency) visited the University of Essex (the University) from 24 to 28 November 2003 to carry out an institutional audit. The purpose of the audit was to provide public information on the quality of the opportunities available to students and on the academic standards of the awards that the University offers. To arrive at its conclusions the audit team spoke to members of staff throughout the University, to current students, and read a wide range of documents relating to the way the University manages the academic aspects of its provision. The words 'academic standards' are used to describe the level of achievement that a student has to reach to gain an academic award (for example, a degree). It should be at a similar level across the UK. Academic quality is a way of describing how well the learning opportunities available to students help them to achieve their award. It is about making sure that appropriate teaching, support, assessment and learning opportunities are provided for them. In institutional audit, both academic standards and academic quality are reviewed. Outcome of the audit As a result of its investigations the audit team's view of the University is that: broad confidence can be placed in the soundness of the University's current and likely future management of the quality of its programmes and the academic standards of its awards. In coming to this judgement, the team considered that the validity of the statement is dependent on the University continuing to strengthen its ability to ensure that permitted variations in local practice do not limit the Senate's capacity to assure the academic standards of all its awards and the quality of all its provision. Features of good practice The audit team identified the following areas as being good practice: the Teaching and Learning Innovation Fund for supporting innovation in teaching and learning; the support offered to international students; and the ability of the administrative services review process to focus on aspects of the student experience. Recommendations for action The audit team also recommends that the University should consider further action in a number of areas to ensure that the academic quality and standards of the awards it offers are maintained. The team advises the University to: continue to rationalise the rules of assessment and the use of academic discretion to ensure parity of treatment of students; ensure that the respective responsibilities of the Senate and the boards of examiners in the award of degrees are secure; ensure that the Senate can be assured that appropriate independent external guidance has been sought on all degree schemes recommended for approval; continue to monitor the processes of periodic review and annual monitoring to ensure that they generate rigorous outcomes and foster enhancement; strengthen the ability of the University to use The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ) as a reference point for specifying academic standards; promote the good practice demonstrated in some areas to encourage greater consistency in obtaining feedback from students; increase the effectiveness of the institutionallevel structure for the preparation and support of graduate teaching assistants; develop an explicit institutional-level strategy for staff development to include the management of departmental-level staff development activity; and exercise caution in the future development of its collaborative arrangements in order to ensure an appropriate experience of higher education for all students engaged in its collaborative provision. Undergraduate programmes in computing, history, law and sociology To arrive at these conclusions, the audit team spoke to staff and students, and was given information about the University as a whole. The team also looked in detail at the programmes listed above to find out how well the University's systems and procedures were working at programme level. The University provided the team with documents, page 1

7 University of Essex including student work and, here too, the team spoke to staff and students. As well as supporting the overall confidence statement given above, the team was able to state that the standard of student achievement in these programmes was appropriate to the titles of their awards and their place in the FHEQ. The team considered that the quality of learning opportunities available to students in each of the programmes was suitable for a programme of study leading to the named award. National reference points To provide further evidence to support its findings, the audit team also investigated the use made by the University of the academic infrastructure which the Agency has developed on behalf of the whole of UK higher education. The academic infrastructure is a set of nationally agreed reference points that help to define both good practice and academic standards. The audit found that the University was making effective use of the academic infrastructure to inform its framework for the management of quality and standards. From 2004, the Agency's audit teams will comment on the reliability of the information about academic quality and standards that institutions will be required to publish, and which is listed in the Higher Education Funding Council for England's document 02/15, Information on quality and standards in higher education. The audit found that the University was preparing appropriately for the publication of the required information. page 2

8 Main report

9 University of Essex Main report 1 An institutional audit of the University of Essex (the University) was undertaken during the period 24 to 28 November The purpose of the audit was to provide public information on the quality of the University's programmes of study and on the discharge of its responsibility as an awarding body. 2 The audit was carried out using a process developed by the Quality Assurance Agency for Higher Education (the Agency) in partnership with the Higher Education Funding Council for England (HEFCE), the Standing Conference of Principals (SCOP) and Universities UK (UUK), and has been endorsed by the Department for Education and Skills. For institutions in England, it replaces the previous processes of continuation audit, undertaken by the Agency at the request of UUK and SCOP, and universal subject review, undertaken by the Agency on behalf of HEFCE, as part of the latter's statutory responsibility for assessing the quality of education that it funds. 3 The audit checked the effectiveness of the University's procedures for establishing and maintaining the standards of academic awards; for reviewing and enhancing the quality of the programmes of study leading to those awards; for publishing reliable information; and for the discharge of its responsibility as an awarding body. As part of the audit process, according to protocols agreed with HEFCE, SCOP and UUK, the audit included consideration of examples of institutional processes at work at the level of the programme, through four discipline audit trails (DATs), together with examples of those processes operating at the level of the institution as a whole. The scope of the audit encompassed all of the University's provision leading to its awards. Section 1: Introduction: the University of Essex 4 Undergraduate students were first admitted to the University in October 1964, with the University receiving its Royal Charter in The University, which has undergraduate and postgraduate degree awarding powers, characterises itself as expanding and research-intensive. 5 The University has four undergraduate schools: Humanities and Comparative Studies; Science and Engineering; Law; and Social Sciences. In addition there is one university-wide Graduate School. Each of the 17 departments belongs to a school, with some departments belonging to more than one school. All departments are based at the Colchester campus, with the exception of East 15, which is located at Loughton, some 50 miles from Colchester. East 15 became a department in 2001 when the University merged with the East 15 School of Acting. 6 The University has grown by 25 per cent over the past four years to reach a student population in of some 8,700 full-time equivalents (FTEs), including those studying through collaborative arrangements. Approximately 180 undergraduates and 50 postgraduates are based at the East 15 Loughton campus. The majority of the University's undergraduate students come from the south-east of England, East Anglia and London. The University has a substantial postgraduate population, with some 30 per cent of the student population at the Colchester and Loughton campuses being in this category, the majority of them coming from outside the UK. Overall, some 40 per cent of the University's students come from outside the UK. 7 The University's self-evaluation document (SED) explained that the University 'has a tradition of admitting students, including mature students, who do not have standard entry qualifications'. Some 14 per cent of the University's UK/EU undergraduate students and 29 per cent of overseas students are mature. The University has a commitment to widening participation, and this informs the expansion of its collaborative activity in south-east Essex. To underpin this activity, the University has developed a sub-degree framework for Foundation degrees, and has a working party which is considering entry qualifications to Foundation degrees. 8 The University's mission statement is expressed as follows: 'The aim of the University is to equip students, employers and the wider community with the knowledge, skills and ideas for living and working successfully in an international world of rapid social and technical change, by means of teaching, training, expert advice and research of a world-class standard'. Among the strategic aims through which it will support its mission, the University plans 'to increase substantially the number of students studying for University of Essex awards by building on existing strengths, embracing innovation and developing new partnerships'. The audit team learnt from senior staff of a proposal, in the early stages of development, to engage with the University of East Anglia to provide a joint higher education (HE) presence in Ipswich. page 4

10 Institutional Audit Report: main report Collaborative provision 9 The University's strategy for local collaborative partnerships is for particular partner colleges to offer degree schemes that are not offered by the University itself. The University has three collaborative partnerships in the region: Writtle Agricultural College, South East Essex College of Arts and Technology (SEEC) and the Tavistock and Portman NHS Trust. These collaborative arrangements are included in this institutional audit. Background information 10 The published information available for this audit included: the information on the University's web site; five Agency quality assessment reports and four subject review reports; the report of the Higher Education Quality Council's overseas partnership audit of the University of Essex and the British Council Teaching Centre, Athens (Dec 1997); the report of the Agency's quality audit of the University (Aug 1998); the report of the Agency's developmental engagement in accountancy and business management (May 2003). 11 The University provided the Agency with: an institutional self-evaluation document SED; four discipline self-evaluation documents (DSEDs) for the areas selected for DATs; documentation as listed in the SED. 12 During the briefing and audit visits, the audit team was given access to the University's internal documents and to its intranet. The team appreciated the unrestricted access it was given to these sources of information. The audit process 13 Following a preliminary meeting at the University in February 2003 between an Agency officer and representatives of the University and students, the Agency confirmed that four DATs would be conducted during the audit visit. On the basis of the SED and other published information, the audit team confirmed that the DATs would focus on taught programmes in: Computing; History; Law; Sociology. 14 The University provided the Agency with DSEDs in September The DSED for Computing included a report (October 2000) by the British Computer Society (BCS) on the accreditation of the University's degrees schemes in computing. 15 At the preliminary meeting for the audit, the students of the University were invited, through their Students' Union, to submit a separate document expressing views on the student experience at the University, and identifying any matters of concern or commendation with respect to the quality of programmes (degree schemes) and the academic standards of awards. They were also invited to give their views on the level of representation afforded to them, and the extent to which their views were noted and acted upon. In generating their written submission, the Students' Union conducted a student survey, working with a member of the University's staff who provided advice on questionnaire design and data interpretation. The questionnaire was ed to every student at the Colchester campus, as well as paper copies being distributed at the main lecture theatres. Two visits were made to SEEC to gather students' views. In total, just over 1,200 responses were received. The key matters addressed in the survey were: resources (access, availability of staff, books, common room, computers, course materials); representation (including extenuating circumstances, appeals); quality of teaching (usefulness of lectures and classes, development of key skills, feedback). 16 On the basis of this survey, the Students' Union generated a document which was submitted to the Agency, and to the University, in July The audit team is grateful to the students of the University for preparing this helpful document. 17 The audit team visited the University on 13,14 and 15 October 2003 for the purpose of exploring with the Vice-Chancellor, senior members of staff of the University and student representatives matters of institutional-level management of quality and standards raised by the University's SED, the students' written submission (SWS), and published documentation. At the close of the briefing visit, a programme of meetings for the audit visit was agreed with the University. The team did not select any area for a thematic enquiry. 18 The audit visit took place from 24 to 28 November Eleven meetings were held during the visit with groups of staff and students from the University and staff of its partner colleges. Meetings page 5

11 University of Essex were also held with staff and students in the four subject areas selected for the DATs. The audit team comprised Professor J L Beeby, Ms S Blake, Dr R Davison, Professor B Gower, Dr M Stowell, auditors, and Mr D Stannard, audit secretary. The audit was coordinated for the Agency by Dr D J Buckingham, Assistant Director. Developments since the previous academic audit 19 The University received a continuation audit by the Agency in November 1997, the report of which was published in August Since then, the University has seen a substantial increase in student numbers, a widening of its portfolio via the incorporation of East 15 Acting School as an academic department and the rapid development of the partnership with SEEC. Since 1997, the University has also established its partnership with the Tavistock and Portman NHS Trust and has closed its collaborative partnership with the British Council in Athens. 20 The 1998 continuation audit report highlighted the lack of a clear, explicit quality assurance strategy. In response to this comment, the University produced a teaching quality strategy, the latest version of which was approved by the Senate in June 2003 for implementation in Since 1998, six subject areas have undergone Agency subject reviews. In all cases the provision was approved and, in most cases, the University was awarded the maximum grading. There was a Developmental Engagement in Accountancy and Business Management in May 2003 which reported confidence in both the standards and quality of the provision. 21 The 1998 continuation audit report contained several comments related to approval, monitoring and review of provision. It noted that 'external participation in the process is not mandatory' for the approval of new degree schemes. The report also commented that, due to the review method then employed by the University, there could be long periods before issues are identified and addressed. The five-yearly degree schemes review process was replaced by the current periodic review scheme in which, although also on a fiveyear cycle, is supplemented by a system for monitoring provision annually. 22 The 1998 continuation audit report identified some areas which echoed the findings of the 1992 report of the Academic Audit Unit's first quality audit of the University. One such area concerned the use of postgraduate students in teaching. The 1998 report suggested greater support for graduate teaching assistants (GTAs), with a compulsory element of training before postgraduate students could teach. The University has now established a code of practice on the training and monitoring of GTAs. 23 Concerns raised in the 1998 audit report included those associated with weakness in the procedures with respect to collaborative provision, the report noting that 'initiatives with external partners have largely emerged as ad hoc arrangements without forming part of any strategy for accreditation, franchising or progression'. Given the rapid development of the strategic partnership with SEEC, the University has recognised a need for the development of a procedural framework and a set of principles for collaborative provision, and has been collating the procedures and practice for validation and review in relation to all its collaborative provision into a single document. At the time of the audit visit, however, this validation handbook was still in draft form. 24 At the time of the continuation audit, the rules for progression and classification varied from school to school, and the 1998 audit report commented on the difficulty of achieving parity of treatment across the University. A working party on rules of assessment reported in autumn 1999, with undergraduate schools subsequently revising rules which were approved in summer The University's responses to the continuation report are set out in a paper Continuation Audit Report - final report on the follow-up to issues raised. The audit team found this clear in its description of the actions taken, although there was little evaluation of the effectiveness of these actions. Several of the matters which emerged from the report of the continuation audit, and to which the University has responded, have been followed up in the present audit. Section 2: The audit investigations: institutional processes The institution's view as expressed in the SED 26 The SED outlined the processes by which the University assures itself of the quality and standards of its provision. It stated that the use of external examiners and their annual reports is pivotal in the assurance of standards. 27 Approval, monitoring and review processes are described in detail in the SED, including the rationale for changes to the systems. Annual monitoring and periodic review procedures have page 6

12 Institutional Audit Report: main report been introduced too recently for an effective evaluation of these processes yet to have taken place. Overall, the SED gave a clear description of the process of assurance, but commentary on evaluation of the processes was largely absent. 28 The quality strategy of the University is contained in the teaching quality strategy as approved by the Senate in June The implementation of the quality strategy involves significant devolvement to departmental level, with support and monitoring from the deans of the schools. Quality and standards of collaborative provision are managed through the recently formed Board of Studies for Learning Partnerships (previously the Board of Collaborative Education), chaired by the Dean of Learning Partnerships. 29 The SED made frequent reference to the Agency's academic infrastructure. Significant effort across the University has been focused on the production of programme specifications. It explained that work with partner colleges has been guided by the Code of practice for the assurance of academic quality and standards in higher education (Code of practice), published by the Agency, in particular the section relating to collaborative arrangements. The institution's framework for managing quality and standards, including collaborative provision 30 The executive governing body of the University is its Council, described in the SED as 'responsible for the management and administration of the revenue and property of the University'. The Vice-Chancellor is the chief academic and executive officer of the University. In carrying out his work he is assisted by a management team consisting of four Pro-Vice- Chancellors, the Registrar and Secretary, and the Director of Finance. They meet as the Vice- Chancellor's Advisory Group (VAG) and also as the Budget Sub-Committee, a sub-committee of the Council's Finance and Strategy Committee. In order to support good communication and discussion of policy, the Vice-Chancellor holds weekly meetings with heads of department and deans. The SED describes this as a 'relatively flat management structure with VAG/Budget Sub-Committee at the core, surrounded by strong and trusted departments', and with short lines of communication. 31 Overall responsibility for the management of academic provision rests with the Senate. The powers of the Senate are set out in the Statutes of the University, and include directing teaching and examinations and all aspects of degree schemes. In carrying out this task, the Senate receives reports and recommendations for approval from school boards and from its Academic Standards Committee (ASC). Other committees of the Senate involved in managing quality and academic standards are the Learning and Teaching Committee (LTC) (see below, paragraph 33) and the Board of Studies for Learning Partnerships (see below, paragraph 34). 32 ASC is chaired by the Pro-Vice-Chancellor (Academic Standards), and is required to make recommendations to the Senate 'on the enhancement of the quality of education and the maintenance and monitoring of academic standards'. It is also required to consider reports on periodic reviews of degree schemes (see below, paragraph 55), and to consider relevant documents from national bodies. Membership of the ASC includes the Pro-Vice-Chancellor (Academic Development) and the Vice-President (Academic and Welfare) of the Students' Union. ASC has responsibility for putting the University's teaching policy into practice. It approaches this formally by issuing guidelines within which, for example, can be included means of alignment with the Code of practice. Guidance on the University's procedures for quality assurance takes the form of a series of documents on the University's intranet. These form in essence a 'quality manual', but there is no single printed document with that title. 33 LTC is responsible for the maintenance and implementation of the learning and teaching strategy and the learning support infrastructure. According to the SED, the committee 'actively promotes teaching innovation in departments' (see also below, paragraph 104) and 'oversees new developments in teaching and learning and related matters'. 34 There are four school boards, each chaired by a dean. The Board of the Graduate School and the Board of Studies for Learning Partnerships are also chaired by deans. The deans have substantial devolved responsibility for the management of quality, academic standards and quality enhancement processes in the University. Their duties include matters relating to student progress, the setting and conduct of examinations and, where appropriate, reporting to the Senate. They are exofficio members of the ASC. Schools oversee student record keeping and examinations, and their boards consider proposals for new degree schemes and such other matters as deans refer to them. Each school has an administrator and other support staff. 35 The role of the Graduate School and its Board is particularly important in quality assurance, quality enhancement and security of academic standards for page 7

13 University of Essex postgraduate research students, to whom the University's annual monitoring and periodic review processes do not apply. The Board has established guidelines for research student supervision, and annual reports on student progress are received by the Dean of the School. 36 A significant amount of responsibility for the management of quality and academic standards is vested in the departments. The SED explained that 'within a range of University and school frameworks, guidelines and codes of practice, departments are given considerable autonomy in academic matters'. Specifically, heads of department have responsibility for all aspects of the provision of teaching in their departments. The principal line of communication for matters associated with teaching is to the school board, of which all heads of department in the school are members. 37 Responsibility for approving the rules of assessment for progression between years, and the classification of degrees, lies with the schools. This led to comments in the 1998 continuation audit report about the consistency of the rules of assessment across the University. ASC subsequently established a working party to consider the situation, and this reported in The first recommendation of the working party was that all discretionary clauses, other than those relating to documented extenuating circumstances, should be removed from the rules of assessment. This proposal was not adopted, in part because of the views of external examiners, some of whom favoured the use of discretion while others did not. As a result, discretion is still used in various ways, leading the audit team to conclude that inconsistency was still possible in the treatment of students studying different schemes, echoing an issue identified in the 1998 continuation audit report about parity of treatment in classification of degrees. The 1999 report of the working party envisaged that problems with consistency and comparability of the rules could be overcome if 'degree classes throughout the University [were] determined in an algorithmic fashion as a function of overall weighted averages and class credits'. Some progress towards this end has been achieved in that there are now three sets of rules of assessment in use on the Colchester campus, and each school is expected to choose and use one set of rules for all the degree schemes for which it is responsible. Different rules of assessment may, however, apply at collaborating partners (see below, paragraph 134). In discussions with members of the University, it was made clear to the team that although progress towards greater uniformity is continuing, staff see strong academic arguments for maintaining the different rules of assessment that the University permits. 38 The rules for progression and degree classification have also been modified in response to the guidance given by The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ), published by the Agency. The award of a Pass degree has been discontinued, and boards of examiners have been given new powers to allow more flexibility for students to 'retrieve failure through reassessment'. The audit team noted, in particular, that students were now given more opportunity to resit failed courses. However, the use of this option is not uniform across the University, adding to the possibility of inconsistent treatment. 39 The University defines a complete programme of study leading to the award of a degree as a 'scheme'. The components from which a scheme is constructed are termed 'courses'. A board of examiners is selected by the dean of school for each scheme within the school. At least one member of the board is the scheme external examiner, and the chair of the board is the dean of the relevant school or another senior member of academic staff. The SED explained that the external examiner has 'overarching responsibility for the standards of the scheme and for arbitration, where necessary, on the degree class of particular candidates'. The audit team noted, however, that the full membership of boards of examiners is not minuted by the school boards, nor is it reported to the Senate, and considered this to be a potentially serious omission. The team also noted that boards of examiners, including those at partner institutions, determine classifications and award degrees without further reference to school boards or to the Senate, and found no reference in the University's Statutes, Ordinances and Regulations that formalised the authority of boards of examiners. The University is advised to review the formal relationship between boards of examiners and the Senate to ensure that the respective responsibilities of the Senate and the boards in the award of degrees are secure, and that the Senate has the ability to maintain an effective institutional-level overview of the work of the boards. 40 The SED explained that the University's strategy for collaborative provision was to 'develop a range of partnership provision which is complementary to provision at its main campus'. Responsibility for overseeing the quality and academic standards of collaborative provision, and of lifelong learning programmes, rests with the recently formed Board page 8

14 Institutional Audit Report: main report of Studies for Learning Partnerships, which is the successor of the Board for Collaborative Education, and which functions in the same way as a school board. To facilitate communication and strengthen links, there is significant committee crossrepresentation between the University and the collaborating partners at all levels, including the level of the University Council. 41 The University manages the assurance and enhancement of quality and academic standards through guidelines set out by the senior level committees, primarily ASC, and endorsed by the Senate. The Pro-Vice-Chancellor (Academic Standards) who chairs ASC, has oversight of the operation of the University's quality assurance processes. It is the responsibility of the deans to ensure that the University's guidelines are followed, and for this they have the appropriate level of delegated authority. On some matters, for example, consideration of annual monitoring reports, they are only required to report to ASC on those items which they judge to have institution-wide significance. On the other hand, the reports of periodic review panels are presented to ASC for discussion and for consideration of any actions. Taken as a whole, this gives a complex structure in which, while deans are required to ensure that departments in their schools act in accordance with the agreed quality assurance processes, the heads of department are responsible not to them but to the Senate. Similarly, decisions of school boards are usually reported to the Senate and not necessarily to the ASC. Although there are additional lines of communication, such as the link Pro-Vice-Chancellor associated with each department, and the fact that deans are ex-officio members of ASC, it is nevertheless possible for consideration of issues at institutional level to be delayed by these indirect reporting lines. Overall, the audit team concluded that the University has in place an appropriate structure for managing quality and standards but that it might, on occasion, be slow to respond to problems and developments. The institution's intentions for the enhancement of quality and standards 42 The SED did not identify separately the University's intentions for the enhancement of quality and standards, but rather identified these as consequences of the development of policies, procedures and practices as these appeared in the text of the document. The audit team therefore has chosen to generally comment on these intentions within the body of the audit report, but identifies here just three important examples of matters relating to enhancement that the University has in hand. 43 The SED reported that the University intended to administer an annual student satisfaction survey in A working party is currently developing the proposed survey, which is designed to align with the requirements for student evaluation set out in HEFCE's document, Information on quality and standards in higher education (HEFCE's document 02/15). The University also intends that the survey should provide it with data which is amenable to analysis to support the enhancement of the student experience. Although the proposal is to conduct the survey on-line, the University has expressed some reservations about the response rate using such a method because some departmental experiences suggested that the transfer of student course survey instruments from paper to on-line led to a decreased response rate. The audit team would encourage the University in its efforts to develop an institution-wide student satisfaction survey in the interest of regularly gaining valuable information about the students' perception of their experience, such as that gained from the survey conducted as part of the SWS to this audit (see above, paragraph 15). 44 The expansion of student numbers in recent years has put greater demands on the University's teaching buildings, with some students commenting to the audit team on over-crowding in classrooms. The University is engaged in a programme of capital building, with a Networks Centre at the Colchester campus to provide additional space for teaching and research for the Departments of Computer Science and Electronic Systems Engineering, and a new lecture theatre building planned for use from October The University is also planning a new campus in Southend, adjacent to SEEC's new building in Southend town centre. Some library facilities will be shared between the University and the College. The audit team considered that the University's plans for capital building, coupled with the implementation of new timetabling software to make more effective allocation of existing teaching space would be an appropriate enhancement of resources for teaching and learning. 45 Later in this report (paragraph 98 et seq), the audit team comments on the University's approach to the assurance of quality of teaching through staff support and development. In , the University is introducing annual developmental reviews for all teaching and administrative staff (see below, paragraph 94). At the time of the audit visit, the University was also considering a proposal for a further significant enhancement of staff support in the form of a development programme for teaching staff, which will provide opportunities for academic page 9

15 University of Essex staff to achieve a formal teaching qualification (see below, paragraph 93). In particular, the team would encourage the University to take this opportunity to ensure that its expanding cohort of GTAs participates in relevant aspects of the staff development programme so that departments, and students, can be assured that all GTAs have experience of good practice in teaching and learning. These initiatives should, together, allow the University to provide a more systematic approach to meeting the development needs of staff and enhancing quality in teaching and learning. Internal approval, monitoring and review processes 46 The SED pointed to the University having 'an integrated approach to the approval, monitoring and review of its taught programmes'. It stated that the formal processes 'are interlocking and are informed by other mechanisms such as staff/student liaison committees (SSLC) and external examiners' reports'. These processes are monitored by the Quality Enhancement Office which reports annually to ASC. The audit team saw the annual report of May 2003, and considered it to be a usefully evaluative document which included a review of approval processes, consideration of efforts to raise awareness of the purpose of the annual monitoring reviews, proposals to modify the periodic review process, and discussions of external examiners reports and programme specifications. One specific item was the consideration of e-learning with a timetable for producing guidance for distancelearning developers (see below, paragraph 108). Programme (scheme) approval 47 Proposals for new degree schemes are submitted by departments to the appropriate school board. The SED explained that all relevant issues on a centrally-provided checklist must be addressed, and that a draft programme specification must be included. The checklist requires formal confirmation that the programme specification 'has been developed with due consideration for, and acknowledges, external reference points'. The programme specification is required to identify learning outcomes and the relationship of the award to the FHEQ. The guidelines for scheme proposers, available on the intranet, give detailed explanations of the information required for the proposal. Scrutiny of a proposal is undertaken by a sub-committee of the relevant school board, appointed by the dean. The sub-committee may meet departmental representatives, if required, to address any issues arising. The dean may refer a proposal to the ASC if difficult quality matters are raised. Finally, the proposal is considered by the school board and, if approved, a recommendation for approval is made by the board to the Senate. School boards are central to the approval process, but the audit team saw little evidence of an institutional overview of the process other than the formal recommendation for approval to the Senate. In its consideration of the effectiveness of its scheme approval procedures, the University might wish to consider if it is satisfied that its institutional-level overview of scheme approvals is able to ensure equivalence of practice across the schools. 48 Provision is made for considering the views of external experts during the scheme approval process. External input is usually undertaken, according to the SED, 'where there is a strong vocational emphasis or where a new curriculum area is being developed'. Examples are given of such consultation, but the SED explained that in practice 'informal consultation with subject peers, who may be external examiners for existing cognate schemes' occurs in most cases. Where a scheme is not significantly changed from an existing scheme, no external view need be taken. The University's degree schemes on the main campus are not regarded as primarily vocational, and there is therefore limited attention to the views of employers in the approval process, the exception being for those schemes for which accreditation by a professional or statutory body (PSB) is sought. The University's approach does not, in the view of the audit team, reliably ensure that the Senate receives an appropriate level of external advice in scheme approval. The University is referred to the comments made in the 1998 continuation audit report (see above, paragraph 21), and might wish to consider revising its guidelines for scheme approval so that the Senate can always be assured that appropriate independent external guidance has been sought on the quality and standards of schemes recommended to it for approval. 49 At the level of the courses that contribute to a degree scheme, the approval of new courses is the responsibility of the relevant school board, although the SED explained that this responsibility is 'normally delegated to the dean and reported as dean's action'. Proposals for new courses and for changes to existing courses must be supported with a detailed rationale and outline. The SED gave examples of proposals which deans had felt it appropriate to refer to their school board for consideration. 50 Validation of new scheme proposals involving collaborative partners is the responsibility of the Board of Studies for Learning Partnerships, which acts in place of a school board in making page 10

16 Institutional Audit Report: main report recommendations for approval to the Senate. Again, there is a centrally generated checklist, specific to collaborative provision. Validation panels include external members and representatives of the University and the partner institution. Panels for vocational degrees at partner institutions normally include employer representation. At SEEC, internal panel members are now included in sessions which were previously restricted to University members only, marking a move toward greater partnership in these matters. Annual monitoring 51 The annual monitoring process has been considerably modified since 1998, when the report of the continuation audit commented on some weaknesses. The SED explained that annual monitoring now consists of an 'evaluative summary of the delivery of a scheme over a year'. This summary is prepared by departments for each degree scheme they teach, using a report pro forma which, according to the SED, 'is intended to encourage a forward looking approach to monitoring activity'. Guidelines for the process include good practice examples to encourage inclusion of adequate information rather than insubstantial responses. The University recognises the need to ensure that annual monitoring is embedded in departmental processes, and is encouraging departments to use their existing internal review mechanisms to support the annual monitoring process and the resulting follow-up actions. It is also 'exploring focusing an element of periodic review on this specific area' to encourage the effective use of departmental processes. 52 Annual monitoring reports are considered by the appropriate dean. The University recognised in the first year of operation of the new process, , that monitoring reports and the subsequent feedback to departments 'did not meet the overall objectives of the process'. Accordingly, in , additional staffing resources were provided to manage the process centrally, and departments were given detailed feedback on their reports. A broadly similar process applies to collaborative provision. 53 The University intends that each annual monitoring report, together with the dean's response, will be used as an additional means of providing feedback to external examiners. Following the first year of operation, and a review of the process by the Quality Enhancement Officer, monitoring report deadlines have been moved to allow departments sufficient time following the end of the external examining timetable and receipt of statistical information. The University emphasised that departments are still required to respond to external examiners following receipt of their reports, and that sending annual monitoring reports to external examiners is 'an enhancement to existing practice'. 54 The University wishes to ensure that the new process for annual monitoring will be 'a helpful reflective tool with which departments are fully engaged' and that it will be 'an integral part of a department's process of review and development'. The audit team formed the view from the evidence seen during the DATs that there is some way to go before this is aim is fully satisfied, with most teaching staff appearing to have only limited involvement in the monitoring process. Meanwhile, the University is continuing to evaluate the process and to adjust it as necessary to integrate it with the periodic review scheme. Periodic review 55 The current periodic review process was introduced across the University in , following piloting in the previous year. It is designed to take account of the appropriate section of the Code of practice, and is applied to each scheme, or group of schemes, on a quinquennial basis. Each review is conducted by a panel, typically of nine members, chaired by the dean of school. Of the members, there are up to four staff plus a student from the department, an external member who is not a current external examiner and a member of the staff of the Academic Section who acts as secretary of the review. Periodic review panels include two members from outside the department but within the School. Joint schemes are reviewed with the single honours schemes of the lead department but with an additional review panel member from the partner department. The review is paper-based, and documentation is provided by the department according to a specified schedule. This includes a reflective document written by the department, annual monitoring reports, external examiners' reports, subject benchmark statements and student survey results. The panel will meet students registered on the schemes under review. The main meeting of the panel follows informal discussions between the non-departmental panel members. 56 Detailed guidelines are provided to panels setting out the key questions to be explored, and for which the outcomes should form part of the panel report. Reports may conclude with a summary of recommendations for consideration by the department. The report is formally presented to ASC, and includes a recommendation stating whether the degree scheme(s) should be continued page 11