Certa. Access Validating Agency relicensing review by the Quality Assurance Agency for Higher Education

Transcription

1 Certa Access Validating Agency relicensing review by the Quality Assurance Agency for Higher Education May 2015

2 Contents About this review... 1 About the AVA... 1 Summary of outcomes... 2 Required actions... 2 Governance... 2 Strategic planning... 2 Management... 2 Course recognition... 3 Moderation, monitoring and certification... 3 Recommended actions... 4 Governance... 4 Course recognition... 4 Moderation, monitoring and certification... 4 Features of good practice... 4 Explanation of the outcomes Governance... 6 Legal and constitutional arrangements... 6 Governance Strategic planning... 8 Planning and monitoring... 8 Development, promotion and enhancement Management... 9 Resources and financial management... 9 Staffing... 9 Self-assessment and risk management Operations Data management and transfer Communications Complaints and appeals Course recognition Diploma development (and transfer) Validation processes Validation criteria Validation panels Diploma approval Provider/centre approval Criteria for provider/centre approval... 16

3 Course recognition Modifications and amendments Revalidation and confirmation of provider/centre approval Moderation, monitoring and certification Moderation processes Moderation responsibilities Moderator recruitment, selection and appointments Moderator induction and training Moderation reports Course monitoring and review Standardisation Award of Access to HE Diplomas Issue and dispatch of Access to HE Diplomas Conclusions Appendix: Risk judgement guidance Glossary... 24

4 About this review This is a report of an Access Validating Agency (AVA) relicensing review conducted by the Quality Assurance Agency for Higher Education (QAA) 1 at Certa (the AVA). The AVA relicensing review process is described in the publication AVA Relicensing: Operational Description. 2 Other parts of The QAA Recognition Scheme for Access to Higher Education can be found on the Access to HE website. 3 The main purposes of the review were to assess the AVA's compliance with QAA's AVA licensing criteria, 4 and on the basis of this assessment, make a judgement about its risk level and a decision about whether its AVA licence should be renewed. These judgements were based on evidence derived from a number of sources, principally: documentary evidence provided by the AVA discussions between the review team and AVA staff, and members of its Board of Trustees and Access to HE Committee, that met on the review visit that took place from 12 to 13 May 2015 the views of AVA stakeholders, gathered through a survey conducted by QAA, which informed the review team's analysis of documentary evidence and its agendas for face-to-face meetings. This report: identifies features of good practice identifies required actions, where the AVA is not fully compliant with the licensing criteria identifies recommended actions, where the AVA can enhance its practice states QAA's judgement of the AVA's risk level: low, medium, high or very high (see appendix for descriptions of each category of risk) states QAA's AVA licensing renewal decision and any follow-up activity. A summary of the outcomes is given in the section starting on page 2. Explanations of the outcomes are given in the section starting on page 6. About the AVA Background information about Certa is given in the AVA's profile on the Access to HE website: 1 Access to HE website: 2 Available at: 3 A full list of Access to HE publications is available here: 4 Available at: 1

5 Summary of outcomes Required actions Certa's compliance with each of the five AVA licensing precepts at the time of the review visit is summarised below, followed by the actions QAA required Certa to take to achieve full compliance. Governance Certa met some of the AVA licensing criteria. Seven criteria were not fully met. To meet licensing criterion 1.10, Certa is required to: revise the Memorandum and Articles of Association to remove the contradiction of how trustees are elected. To meet licensing criteria 1.11, 1.14 and 1.16, Certa is required to: revise the terms of reference for the Board of Trustees and the Access to HE Committee to ensure they are consistent and clear about where formal approval of Access to HE Diplomas takes place. To meet licensing criteria 1.12, 1.14 and 1.16, Certa is required to: specify in Board documentation those matters that cannot be delegated by the Board. To meet licensing criteria 1.18 and 3.6, Certa is required to: ensure minuting for both the Board of Trustees and Access to HE Committee meetings records the key items from reports and minutes received to allow internal reference and direction in the development of the AVA's policy and operation, and for administrative effectiveness. To meet licensing criterion 1.19, Certa is required to: document clearly that the minutes of the governing body are retained for the lifetime of the organisation. Strategic planning Certa met nearly all of the AVA licensing criteria. One criterion was not fully met, which also relates to criteria 5.2 and 5.4, which were also not met. To meet licensing criteria 2.9, 5.2 and 5.4, Certa is required to: produce and regularly update documents that provide expert advice on the delivery of Access to HE Diplomas, in order to ensure that providers meet the AVA's expectations and the requirements of the Access to HE Diploma and credit specifications, and that they meet all of the sub-parts of the licensing criterion 2.9. Management Certa met most of the AVA licensing criteria. Three criteria were not fully met. To meet licensing criterion 3.4, Certa is required to: review its current staffing structure to ensure the level and structure of staffing can deliver the requirements involved in the holding of an AVA licence, and that the staffing structure ensures that: there is appropriate expertise among staff to carry out the requirements of the AVA licence, including expertise to advise providers on the development, delivery, assessment and quality assurance of Access to HE provision staff with responsibilities for Access to HE are appropriately supported within the structures and systems of the AVA training and development opportunities are made available to staff, including opportunities to enhance the quality of leadership and organisational 2

6 management, to ensure that staff remain competent for the demands of their roles. To meet licensing criteria 3.6, Certa is required to address the required action under licensing criterion 1.18 above. To meet licensing criterion 3.13, Certa is required to: revise its complaints and appeals procedures to ensure that there are no conflicts of interest within the policies, in line with best practice. Course recognition Certa met many of the AVA licensing criteria. Thirteen criteria were not fully met. To meet licensing criterion 4.10, Certa is required to: make the Learning Programme Plan master template explicit that information should be provided on assessment methods for the Diploma. To meet licensing criteria 4.12, 4.13, 4.14, 4.15, 4.16 and 4.17, Certa is required to: review its recognition panel membership to ensure that: it is an expert, external panel that meets sub-parts (a) to (d) of licensing criteria 4.13; AVA officers are not members but are in attendance; no more than one member of the designated body that approves Diplomas is a member; providers are not panel members at events to approve a Diploma they wish to deliver; and all panels are constituted in accordance with these revised guidelines. To meet licensing criterion 4.19, Certa is required to: amend the Diploma panel report template to make clear that the panel's responsibility is to make a recommendation to the designated body and not to formally make the decision itself. To meet licensing criterion 4.23, 4.32 and 4.33, Certa is required to: ensure that a Diploma has been approved by the designated body before course recognition is confirmed, and that after formal approval the provider is informed of the date of course recognition and when delivery may start. To meeting licensing criterion 4.30, Certa is required to: revise the Centre Agreement Application Form to include the relevant AVA licensing criteria in addition to those of Ofqual. To meet licensing criterion 4.43, Certa is required to: include an analysis of the outcomes of moderation and course review within the revalidation of Diplomas. Moderation, monitoring and certification Certa met most of the AVA licensing criteria. Five criteria were not fully met. To meet licensing criteria 5.2 and 5.4, Certa is required to: provide information on the moderation processes in time for providers and external moderators to prepare for the forthcoming year. To meet licensing criteria 5.7 and 5.12, Certa is required to: clearly state in documentation relating to the appointment of external moderators (Terms of Moderation Agreement) that those appointed to review particular Diplomas are appropriately experienced or qualified to do so in the subject. To meet licensing criterion 5.32, Certa is required to: develop and document administrative checks for the issue and reissue of Diplomas and achievement transcripts. 3

7 Recommended actions QAA recommended to Certa the following actions to enhance practice. Governance Revise formal documentation to state how members of the Access to HE Committee are appointed and to set a maximum number of consecutive terms of office for both Board and Committee members (licensing criterion 1.10). Course recognition Improve the clarity and completeness of guidelines for providers on the process and procedures for the transfer of Access to HE Diplomas (licensing criterion 4.5). Update the Guidelines for Panel Members document so that it effectively summarises sub-parts (a) to (f) of licensing criterion Make clear in its documentation the approval process for major amendments to programmes (licensing criterion 4.37). Moderation, monitoring and certification Continue endeavours to work with higher education providers to seek feedback about the performance of Access to HE students and the appropriateness of the preparation provided by its recognised Access to HE courses (licensing criterion 5.24). Features of good practice QAA identified the following feature of good practice at Certa. The AVA's interaction with Access to HE students, including student representative meetings, surveys of students and biannual newsletters, which informs students of AVA developments and changes, and allows student feedback to inform the AVA's quality assurance processes and Diploma development (licensing criterion 3.10). 4

8 Risk judgement Certa's level of risk: high AVA licence decision Certa's AVA licence: temporarily extended Follow up QAA required Certa to: report the actions taken in response to the recommended actions in its next annual self-evaluation report submit an action plan in response to the required actions by 1 September 2015 submit progress reports with documentary evidence to demonstrate the completion of required actions by 1 December 2015 and 1 March 2016 undergo a revisit by a QAA review team by May Please note: QAA publishes changes to Certa's risk level, the current status of its licence and any required or recommended actions in the AVA profile. 5 5 Available at: 5

9 Explanation of the outcomes This section explains the outcomes of the relicensing review in five main sections corresponding to the five licensing precepts. Paragraph numbers and numbered references correspond to the AVA licensing criteria. A glossary of terms is provided on page Governance Legal and constitutional arrangements 1.1 Certa's constitutional basis is set out in its Memorandum and Articles of Association. These specify its legal identity, function, aims and principal governance structures. 1.2 Certa's company objects are congruent with the aims of The QAA Recognition Scheme for Access to Higher Education. 1.3 Certa's constitutional structure requires the involvement of member organisations. 1.4 Certa is not itself a provider of Access to HE courses or a receiver of Access to HE students. 1.5 Member organisations include organisations providing Access to HE courses and higher education institutions that receive Access to HE students and are subscribers to QAA. 1.6 Certa operates formal and transparent processes for approving organisations as members, and for agreeing any other formal organisational relationships. 1.7 Certa's legal identity, constitutional and governance arrangements provide formal accountability to member organisations; protection from the undue influence of any one of, or a minority of, its member organisations, or their representatives; independent decision making and operation as an AVA; and an identified basis on which it is able to enter into legally binding agreements. 1.8 Certa has appropriate agreements in place with the suppliers of goods or services that are integral to the operation of the AVA. Governance 1.9 Certa has clear descriptions of its governance and deliberative structures set out in its formal documentation The review team explored whether Certa's governance structures include relevant experience and expertise in relation to governance and AVA matters. The process document for the appointment of trustees identifies that the Board approves the appointment of new trustees, and Companies House is notified of the appointment before they are formally appointed at the AGM. In effect, this means that the Board appoints its own members. This contradicts the Articles of Association, which state that seven to 10 trustees must be elected by members, and the election results are received and reported at the AGM. To meet licensing criterion 1.10d, that the process and criteria for the appointment or election of representatives to decision making bodies is clear and transparent, Certa is required to revise the Memorandum and Articles of Association to 6

10 remove the contradiction of how trustees are elected. According to the terms of reference of the Access to HE Committee, appointments are made to the Committee via the Board, however, there are no details of duration of office or criteria for appointment, or for the requirement of Access to HE experience. Similarly, the terms of reference for the Finance and Staff Committee give no indication of how appointments are made or criteria for appointment, other than needing to be a trustee. With regard to the criteria for the appointment of trustees, these are available but there is no limit to the number of times members can be reappointed. As a consequence, the review team recommends that Certa revise formal documentation to state how members of the Access to HE Committee are appointed and to set a maximum number of consecutive terms of office for both Board and Committee members Formal committees should operate according to terms of reference that detail a clear remit, responsibilities and reporting relationships through which their accountability to the organisation is defined. The review team found that the terms 'ratification' and 'approval' are used interchangeably by the Board and the Access to HE Committee with regard to the approval of Access to HE Diplomas, which made it unclear where formal approval was taking place. The terms of reference for the Access to HE Committee state that the Committee confirms formal ratification of all Access to HE Diplomas following validation panels and receipt of panel reports, and then makes recommendations of final approval to the Board. The Board terms of reference states that the Board ratifies the approval of all qualifications. To meet licensing criterion 1.11, Certa is required to revise the terms of reference for the Board of Trustees and the Access to HE Committee to ensure they are consistent and clear about where formal approval of Access to HE Diplomas takes place The Board of Trustees is the ultimate authority for the AVA licence and is responsible for ensuring that the organisation meets its responsibilities as an AVA. The Board has direct responsibility, which cannot be delegated for the formal approval of reports and responses to QAA in relation to the AVA licensing requirements. However, this requirement is not identified in either the Board terms of reference nor in the Memorandum and Articles of Association. In order to meet this licensing criterion, Certa is required to specify in Board documentation those matters that cannot be delegated by the Board The Board also has ultimate responsibility for the company's legal and financial affairs, policy development and strategic direction, as well as the approval and quality assurance and enhancement of the AVA's recognised Access to HE courses, monitoring and approval of academic standards, and the process for the award of Access to HE Diplomas to students Paragraph 1.11 noted that it was unclear where formal approval of Access to HE Diplomas is taking place within Certa's committee structure, and in paragraph 1.12 that Board documentation does not specify those matters that cannot be delegated by the Board for formal approval. Responsibilities are not therefore properly specified and appropriately located within the governance structure. Licensing criterion 1.14 is therefore not fully met and required actions are given in paragraphs 1.11 and Monitoring and reporting procedures are in place through which the Board can ensure that delegated activities are properly undertaken Paragraphs 1.11 and 1.12 indicate that currently the extent and limits of authority of bodies or individuals holding delegated responsibilities are not adequately defined and required actions are given in these paragraphs. 7

11 1.17 The separation and links between Certa's governance and management structures are clear, and the separation is demonstrated in the AVA's operation Full and generally accurate records of committee's decision making are maintained and are sufficient for purposes of public accountability, however, at Board level, there rarely appears to be any details or summary of discussion of the Access to HE Committee minutes documented. Furthermore, the Access to HE Committee meeting minutes do not appear to document any details or summary of discussion of lead external moderator reports. While the Chair of the Access to HE Committee speaks to the minutes of the Committee at Board meetings, this is not documented in any minutes. It was agreed by both Board and Committee members that the review team met that minutes need to be more detailed to enable discussions to take place and for issues to be followed up at later meetings. The current mode of documentation allows no opportunities for committees to follow-up on issues discussed, but not minuted, at previous meetings. Certa is therefore required to ensure minuting for both the Board of Trustees and Access to HE Committee meetings records the key items from reports and minutes received to allow internal reference and direction in the development of the AVA's policy and operation, and for administrative effectiveness While Certa officers advised the review team that minutes of the Board are retained for the lifetime of the organisation, this is not formally stated in the organisation's documentation. To meet the criterion, Certa is required to document clearly that the minutes of the governing body are retained for the lifetime of the organisation. 2 Strategic planning Planning and monitoring 2.1 Certa operates a regular process for strategic planning relating to its Access to HE activity, which takes account of all areas expected by this licensing criterion. 2.2 The AVA's strategic plan for Access to HE specifies strategic objectives, with linked operational targets, and with responsibilities assigned and deadlines set for meeting the targets. 2.3 The strategic objectives include objectives relating to the development, promotion and enhancement in the area(s) in which the AVA operates. 2.4 The strategic plan is formally approved through Certa's governance structures. 2.5 The strategic plan is regularly monitored; the AVA's achievement of its strategic objectives is assessed; and action is taken in light of the outcomes of the monitoring process. 2.6 The strategic planning process is considered through the AVA's self-assessment processes. Development, promotion and enhancement 2.7 Certa provides a focus for the development, promotion and enhancement of Access to HE provision in its area(s) of operation. 2.8 Certa gathers and analyses information relevant to the development, promotion and enhancement of Access to HE, and communicates its analyses to its providers and other stakeholders on a regular basis. 8

12 2.9 Certa provides advice on the delivery of the Access to HE Diploma to ensure that providers meet the AVA's expectations and the requirements of The Access to Higher Education Diploma and Credit Specifications; 6 and it arranges events and opportunities for practitioners to support and develop good practice. Previously, the AVA provided advice on the delivery of the Access to HE Diploma through a range of documentation, primarily located in the Access to HE Handbook. However, since when the new Diploma specification came into force, the AVA has not produced this documentation, largely due to staff constraints (see paragraph 3.4) and instructed providers and moderators to use the Access to HE Handbook together with supplements uploaded to the Certa website. As a result, the advice of the AVA on the delivery of the Diploma is out of date and, in many instances relating to the new specification, erroneous. The review team concludes that, presently, the AVA does not meet this criterion, and this also has implications for other criteria. In order to meet criteria 2.9, 5.2 and 5.4, Certa is required to produce and regularly update documents that provide expert advice on the delivery of Access to HE Diplomas, in order to ensure that providers meet the AVA's expectations and the requirements of The Access to Higher Education Diploma and Credit Specifications, and that they meet all of the sub-parts of criterion Certa promotes Access to HE through its activities and publications. 3 Management Resources and financial management 3.1 Certa currently has sufficient financial resources to exercise its AVA responsibilities in full. 3.2 The AVA's facilities, administrative systems and allocation of funds are sufficient to manage its AVA responsibilities, including, in particular, responsibilities relating to assuring the quality and standards of its recognised Access to HE provision. 3.3 The AVA operates a responsible, systematic and rigorous approach to the management of its financial affairs. Staffing 3.4 In meetings during the review visit, the review team discussed the staffing arrangements for Access to HE to confirm whether the level and structure of staffing can deliver the requirements involved in holding an AVA licence. The team noted that the AVA staffing structure has changed considerably during the last 18 months and the AVA has accepted a number of voluntary redundancies. At the time of the review visit the Access to HE and Training Centre Manager had been appointed Interim Chief Executive Officer and was now covering both roles. The team noted that the number of continuing professional development tutor assessors has been reduced and the position of Awards Team Leader was made redundant. The Awards Team is now managed and led by the Finance and Operations Manager. After the review visit the AVA clarified that an additional Access to HE Administrator was also in post at the time of the review. One member of staff from the AVA team, the Access Development Officer, had been required to work extensively on the awarding organisation side of the business for the previous twelve months due to the demands of developing its non-access qualifications. 6 Available at: 9

13 This led directly to the lack of an Access to HE Handbook, as detailed in paragraph 2.9 above, and is likely to have contributed to the undercurrent of dissatisfaction from a small minority of stakeholders in the stakeholder survey. The Board stated that they believed staffing on the AVA side was currently insufficient and recruitment would need to take place, particularly if the stated aim to grow the business was to be realised. The review team thus concluded that at the present time the AVA staffing is under-resourced. Certa is therefore required to review its current staffing structure to ensure the level and structure of staffing can deliver the requirements involved in the holding of an AVA licence, and that the staffing structure ensures that: there is appropriate expertise among staff to carry out the requirements of the AVA license, including expertise to advise providers on the development, delivery, assessment and quality assurance of Access to HE provision staff with responsibilities for Access to HE are appropriately supported within the structures and systems of the AVA training and development opportunities are made available to staff, including opportunities to enhance the quality of leadership and organisational management, to ensure that staff remain competent for the demands of their roles. Self-assessment and risk management 3.5 Certa has procedures for regularly monitoring and assessing the quality, effectiveness and security of its management and operations. Operations 3.6 Minutes of committees' proceedings and other formal records are produced, are retrievable, and are held for an agreed period. As noted in paragraph 1.18 the minutes of the Board of Trustees and the Access to HE Committee, while detailed in some ways, lack any discussion or detail of issues arising from either minutes of subcommittee or papers tabled. They thus lack any record for members to raise under matters arising for following meetings or any audit trail. The required action under paragraph 1.8 should enable the AVA to address this requirement. 3.7 The AVA develops, maintains and documents procedures, including indications of specific actions for AVA staff, in relation to key aspects of AVA operations for Access to HE to ensure that the work of the AVA is not unduly reliant on the knowledge, practices or efforts of individual staff. 3.8 The AVA's documented operational procedures for Access to HE are clear and readily available, and include processes expected by this licensing criterion. Data management and transfer 3.9 Certa has adequate and effective structures and systems for collecting, recording and holding data about Access to HE providers, courses, students and awards. Communications 3.10 Certa has a range of mechanisms through which it communicates to, and gathers feedback from, member organisations and other stakeholders, about matters relating to Access to HE. 10

14 Certa has a very proactive relationship with Access to HE students. The AVA holds a student representative meeting once per year, conducts two surveys of Access to HE students towards the end of the academic year, and produces a biannual newsletter for students. The minutes of the student representative meetings show that they are well attended, deal with a number of important topics and provide useful feedback to the AVA. Both AVA officers and the Access to HE Committee members confirmed that the feedback from students is useful in both a general sense but also were able to give specific examples as to matters that have changed as a result of student feedback. The interaction with Access to HE students, including student representative meetings, surveys of students and biannual newsletters, which informs students of AVA developments and changes, and allows student feedback to inform the AVA's quality assurance processes and Diploma development, is a feature of good practice Certa monitors providers' published information about Access to HE to verify its accuracy and currency, and its consistency with QAA requirements The AVA makes use of the Access to HE logo in its own publications and ensures that its use of the logo is consistent with QAA's published guidance. Complaints and appeals 3.13 Certa has transparent and accessible procedures to enable complaints and appeals to be received from students and providers in relation to its role as an awarding body. There is both a complaints procedure and an appeals procedure. Both the complaints and appeals procedures require students (if that is the appellant) to go through the centre procedures first, where appropriate, in line with standard practice. The complaints procedure has appropriate stages but lacks any statements around conflict of interest (for example, what happens if the complaint is about someone who would be involved in the investigation). The appeals procedure is split into two sections depending on who is appealing about what. Appeals by students (or providers on behalf of students) are dealt with by a panel comprising a trustee, a member of the Quality and Standards Committee and someone entirely external to the AVA. Appeals by centres (for example, against validation decisions) are dealt with by a panel consisting of the Chair of the Access to HE Committee, one impartial member of the Access to HE Committee and one other representative of a member organisation. Since the Access to HE Committee is the body that ratifies such decisions this is a conflict of interest. In order for complaints and appeals to be considered and resolved fairly, the AVA is required to revise its complaints and appeals procedures to ensure that there are no conflicts of interest within the policies, in line with best practice. 4 Course recognition Diploma development (and transfer) 4.1 Certa has a systematic and transparent approach to developing new Access to HE Diplomas (and considering requests for course transfer). 4.2 Certa considers key information at an early stage in the development of new Diplomas (or consideration of transfer requests) and confirms the areas in this licensing criterion are met. 4.3 During the development phase for a new Diploma, individuals currently delivering higher education provision in a subject relevant to the progression route(s) are involved in advising on the expectations and requirements for entry to the progression route(s) and 11

15 the appropriateness of the Diploma proposals in meeting those expectations. 4.4 In considering a request by a provider for the transfer of responsibility for course recognition from another AVA, Certa formally confirms, before the transfer is agreed, those areas expected by this licensing criterion. 4.5 Certa publishes clear guidance for those involved in the development (and transfer) of Access to HE Diplomas. The guidance includes the information expected by this licensing criterion. Details of requirements for the development of a new Diploma are contained within the Learning Programme Plan Master Template and the out-of-date Access to HE Handbook (see paragraph 2.9). Guidance is also given in the Process Checklist. For course transfer, the core document for the transfer of courses is the Learning Programme Plan, and in a recent example of the process examined by the review team the completion of the form was supported by bespoke guidance from the AVA. Certa has generic guidelines for providers wishing to transfer a course to Certa, although the review considered that these guidelines could be improved to make it clearer to providers what is required of them and what documentation they are required to submit. The review team recommends that Certa improves the clarity and completeness of guidelines for providers on the process and procedures for the transfer of Access to HE Diplomas. 4.6 The AVA's guidance also includes explicit statements that the design of Diplomas should be appropriate to the primary purpose of Access to HE courses, and that Access to HE Diplomas are intended to provide a preparation for study in UK higher education, but the award of a Diploma does not provide guaranteed entry to UK higher education programmes. Validation processes 4.7 The AVA operates a thorough, transparent and consistent approach to the validation of Access to HE Diplomas. 4.8 The AVA's validation process involves the scrutiny of Diploma proposals by an appointed panel, which assesses the proposals against a set of clear, standard criteria, including those listed in licensing criterion Both the individual units and the totality of each named Access to HE Diploma are considered as part of the Diploma validation process. Validation criteria 4.10 The Learning Programme Plan, which is completed for the approval of a new Diploma, includes details of the curriculum and its delivery but not explicitly the assessment methods for the Diploma. Units are listed with their credit value but no detail of assessment methods are provided. While guidance is given in the Access Handbook for , it is not explicit in the Learning Programme Plan considered by the recognition panel. In order for the AVA's validation process and criteria to ensure that, for a Diploma to be recommended for approval it meets the requirements of this licensing criterion, Certa is required to make the Learning Programme Plan master template explicit that information should be provided on assessment methods for the Diploma The outcomes of the validation process for each Diploma, including recommended conditions of approval and textual amendments to documentation, are recorded. 12

16 Validation panels 4.12 Panel reports show that the panels for new and transferred courses are constituted according to the AVAs guidelines. For major modifications panels comprise the same members as for the consideration of a new or transferred Diploma. Panel membership is specified in the AVA's Guidelines for Panel Members, which states that a panel will aim to consist of: an independent panel chair, representatives from provider(s) who has/have submitted an Approval to Run form, a minimum of two external representatives from other organisations with experience of Access to HE provision, at least one representative from an appropriate Higher Education Institute that offers progression opportunities, an 'HE within FE' representative, and the existing or intended external moderator. In the view of the review team, the 'aim' to constitute a panel of certain types of representatives is not a definitive statement of the members required to constitute a panel. In the panel reports examined by the review team, panel membership is broad and does not distinguish between the members whose role is to consider and approve a submission and those from the providers wishing to deliver the Diploma. In the case of the Access to HE Diploma (Art and Design), which had been recently approved, there were no independent external panel members of the panel considering the proposal. Validation panels have both representatives from Certa and providers wishing to deliver the Diploma as panel members who could be involved in the approval. The team concluded that Certa did not meet the criterion that the AVA appoints an expert, external panel to scrutinise proposals for new, transferred or major modifications to Access to HE Diplomas. Certa is therefore required to review its recognition panel membership to ensure that: it is an expert, external panel that meets sub-parts (a) to (d) of licensing criterion 4.13; AVA officers are not members but are in attendance; no more than one member of the designated body that approves Diplomas is a member; providers are not panel members at events to approve a Diploma they wish to deliver; and all panels are constituted in accordance with these revised guidelines As noted in paragraph 4.12, the AVA's panel membership guidelines state that a panel will 'aim' to consist of certain representatives, which, in the view of the review team, could lead to different constitutions of panels. In the reports studied by the team one definitely did not meet the guidance while four others were missing representation from key sectors specified in the guidance. AVA officers drew attention to an example of where they had managed this situation, where the membership had not followed their guidelines, by deferring the process. The team concluded that the criterion for the composition of panels to ensure that panel members jointly provide current, relevant experience and expertise in the delivery and assessment of Access to HE courses; curriculum knowledge relevant to the Diploma(s) and all the units being considered; and the delivery of higher education programmes, in areas indicated as intended progression routes for the Diploma(s) being considered, is not met. See also paragraphs 4.12, 4.14, 4.15, 4.16 and As paragraph 4.12 noted, membership and composition of validation panels for Access to HE Diplomas, and the criteria for selecting panel members, are not clearly specified and consistently applied, therefore this licensing criterion is not met in full The Validation Panel report has a statement confirming whether there are any registered conflicts of interest. The Guidelines for Panel Members specifies that 'representatives from provider(s) who has/have submitted an Approval to Run an AVA Access to HE Diploma Application Form' are members of the panel. This does not meet the licensing criterion that the Chair and members of the validation panel have no conflicts 13

17 of interest in respect of the outcome of the validation event, as panels include providers who are applying to deliver the Diploma and these staff are involved in the decision making of the panel. In the case of the Access to HE Diploma (Art and Design) Panel there were no independent members yet it was recorded that there was no registered conflict of interest on the panel report. The membership of validation panels requires review to ensure that providers who wish to deliver a Diploma do not attend as full panel members and are not involved in making validation decisions. A required action is given in paragraph While records indicate that no validation panel has been attended by more than one member of the Access to HE Committee, the requirement that no more than one member of the validation panel is also represented on the body that has been designated as responsible for the approval of Diplomas within the AVA requirement is not made explicit in the AVA's Access to HE Diploma Validation Panel Membership document. A required action is given in paragraph Panel reports show that AVA officers are members of recognition (validation) panels with the Access Development Officer being listed as a member at all panels. Representatives from Certa are listed in the Access to HE Diploma Validation Panel membership in the Guidelines for Panel Members. The review team concluded that this licensing criterion is not met. A required action is given in paragraph Before the panel event, panel members are informed about their roles and responsibilities; the purpose and conduct of validation panel events; QAA's current requirements about the Access to HE Diploma; the criteria for successful validation; and the possible outcomes of the process and the post-panel process. The review team found that this criteria is met through the now out-of-date Access to HE Handbook, and summarised but not explicitly stated in the Guidelines for Panel Members document. The review team recommends that Certa updates its Guidelines for Panel Members document so that it effectively summarises sub-parts (a) to (f) of licensing criterion The panel makes a recommendation for approval, including any recommended conditions of approval, and the panel's recommendations are referred to the designated body within the organisation's governance structure for formal approval. Panel reports clearly list conditions and recommendations, which have to be addressed within 28 days, this being checked by the Business Development Officer. Once signed by the Access to HE and Training Centre Manager to confirm all the conditions have been met this becomes the Definitive Panel Report. The Panel Report includes a 'Panel Decision' to 'approve' for one or five years. However, it is the responsibility of the Board to formally approve new Diplomas and the panel should therefore only recommend Diplomas for approval. The Access to HE Committee is responsible for confirming the formal ratification of all Access to HE Diplomas and making recommendations of final approval to the Board. There is inconsistency in the terminology used in the Access to HE Committee minutes on the action they take with respect to the consideration and approval of Diplomas. The minutes of the Access to HE Committee formally record that it 'ratifies' Recognition Panel Reports and in another case 'approving' them. With respect to the responsibility of the Board, the minutes of the Access to HE Committee are noted but ratification of Access to HE Diplomas is not consistently made explicit in the Board's minutes, with only one example being seen by the review team where new Diplomas were approved. The review team concluded that criterion 4.10 is not met. Certa is therefore required to amend the Diploma panel report template to make clear that the panel's responsibility is to make a 14

18 recommendation to the designated body and not to formally make the decision itself. Diploma approval 4.20 The Access to HE Committee considers the validation panel report and the panel's recommendations, and confirms that the AVA's validation processes and criteria have been properly and consistently applied The Access to HE Committee recommends to the Board to grant or withhold approval (with or without conditions) and, with regard to any conditions on the approval of a Diploma, it confirms that conditions will ensure that the Diploma meets the AVA's validation expectations; that there are clear timescales for meeting conditions; the date by which any conditions must be met; and the process for confirming that conditions have been met The AVA monitors whether and when conditions have been met, ensuring that the process is followed for confirming this When Certa is satisfied that the conditions have been met, the approval of an Access to HE Diploma is formally recorded, and providers wishing to deliver the Diploma are informed that they may proceed. The Access Administrator advises the Quality Team once a Diploma has been ratified by the Access to HE Committee. According to the Panel Process Flow Chart, an Approval to Run letter is then sent to the provider. In the evidence available to the review team, either a Confirmation Letter or Approval to Run letter are sent to the provider; both of which give approval to run prior to formal ratification by the Access to HE Committee and approval by the Board. Evidence was supplied to the team of one instance where a letter was sent to the provider following ratification of the Diploma by the Access to HE Committee but this, from the evidence supplied, is not consistent practice. The April Monthly Bulletin 2014 listed a number of Diplomas that had not been formally ratified by the Access to HE Committee. The review team concluded that the criterion is not met because providers are informed that they may proceed before the approval of the Diploma has been formally recorded. A required action is given in paragraph See also paragraph The withdrawal of approval from a Diploma before the end of its validation period is considered by the Access to HE Committee and ratified by the Board with reference to standard criteria; procedures for withdrawal are applied consistently and transparently; and any necessary amendments to the Access courses database are made promptly The AVA holds full records of all validation and approval decisions. It maintains definitive Diploma documentation, making amendments, as necessary, to reflect approved modifications The Access to HE Committee maintains oversight of the validation and approval process and criteria, and is responsible for ensuring that processes are robust and operated consistently. Provider/centre approval 4.27 The process taken by Certa to approve a new provider to deliver an AVA Diploma is explained in detail in the operation Panel Process Checklist. The Access to HE Committee ratifies the recognition panel reports in accordance with its terms of reference, however, due to lack of transparency in the Board minutes it is not possible to determine 15

19 which centres were ratified by the Board at any meeting (see paragraph 1.18). The review team examined a number of audit trails and was satisfied that Certa operates a transparent, systematic and consistent approach to the approval of providers for the delivery of Access to HE courses, through which it confirms that the provider meets its criteria for centre approval, thus meeting this licensing criterion Certa formally confirms centre approval for the delivery of named Access to HE Diplomas (or Access to HE provision) before a provider is permitted to deliver any part of any individual Access to HE Diploma The AVA reconsiders (and may withdraw) approval if the provider fails to continue to meet the terms of its approval. The withdrawal procedure meets the requirements of this licensing criterion. Criteria for provider/centre approval 4.30 Certa's process and criteria for provider approval are met through the Approval to Run AVA Diploma Application Form and the Centre Recognition form. The review team noted that where criteria are addressed through the Centre Recognition Application (the permanent and main base of the provider being located in the UK, a single named point of accountability for quality assurance and strategic decision making, and clear arrangement for the day-to-day operational management of courses), the form makes reference to Ofqual criteria throughout and not the relevant AVA licensing criteria. Certa is required to revise the Centre Agreement Application Form to include the relevant AVA licensing criteria in addition to those of Ofqual Certa's process and criteria for centre approval ensure that providers explicitly commit to cooperating with the AVA's moderation, monitoring and standardisation procedures. Course recognition 4.32 Certa operates a standard process to confirm that Diploma approval and centre approval have been satisfactorily completed before course recognition is confirmed. The confirmation letter confirming approval of the Diploma sent to providers gives approval to run prior to formal ratification of the new Diploma by the Access to HE Committee. The review team, in the evidence supplied by the AVA, only saw one example where a letter was sent to the provider after ratification of the Diploma by the AVA. Advising all providers of ratification of a Diploma that they wish to deliver is not undertaken consistently from the evidence made available to the review team. The team concluded that this licensing criterion is not fully met. A required action is given in paragraph See also paragraph The Panel Process Flow Chart indicates that after ratification of the recognition panel report the Approval to Run letter is sent to the provider, however, the Approval to Run letter informs the provider of approval subject to ratification by the Access to HE Committee. The audit trails show the Approval to Run/confirmation letter being sent to the provider prior to ratification of the relevant recognition panel report by the Access to HE Committee, with only one example provided of a letter being sent to the provider subsequent to ratification. Certa is required to ensure that a Diploma has been approved by the designated body before course recognition is confirmed, and that after formal approval the provider is informed of the date of course recognition and when delivery may start When the AVA has confirmed recognition of a new course, it uploads the required 16

20 course information to the Access courses database. It revises course records, as necessary, to maintain the currency of information available on the database. Modifications and amendments 4.35 Certa provides mechanisms through which providers, moderators and partner higher education institutions can suggest modifications to units or Diplomas, enabling those suggestions to be systematically considered Certa operates standard procedures for regulating modifications to units and Diplomas, and ensures that any feedback about the unit/diploma received from providers/centres, moderators or receiving higher education institutions is taken into account The AVA defines minor modifications and major changes to Diplomas but it is not made clear where authority lies for approval or the process involved within the AVA's documentation. AVA officers confirmed that major amendments are considered through the validation process. Modifications to units are agreed in principle by the Access to HE Manager and then they are considered by an amendment panel, with recommendations for approval being sent to Access to HE Committee. The review team recommends that Certa makes clear in its documentation the approval process for major amendments to programmes When minor modifications are considered (including modifications to units included in more than one Diploma), the AVA confirms that affected Diplomas will continue to comply with the AVA's validation criteria before approving the modification If modifications are made to a unit or Diploma which is delivered by more than one provider, all providers involved in the delivery of the unit or Diploma are informed of the amendments The Access to HE Committee has final approval of unit and Diploma modifications. Revalidation and confirmation of provider/centre approval 4.41 Certa's standard validation period for Diplomas is five years from the date of initial approval. The AVA maintains a forward schedule of revalidation for Diplomas Periodic Diploma revalidation and confirmation of provider/centre approval are requirements of continued course recognition The Learning Programme Plan is used as a key document for revalidation. The Revalidation Learning Programme Plan template asks providers to provide a critical review of their Diploma over the last five years. In reviewing the Plans in the audit trails, the moderation, evaluation and review process is described but the outcomes are not analysed. Consequently, Certa is required to include an analysis of the outcomes of moderation and course review within the revalidation of Diplomas The constitution of revalidation panels is the same as for validation panels. Outcomes are confirmed by the same designated body as for Diploma approvals. However the same issues apply for the constitution of revalidation panels as validation panels, which have led to required actions in criteria 4.12 to 4.15, and If a Diploma is to be discontinued at, or before the end of, its validation period, providers are informed of the final date on which new starters may be enrolled, allowing 17