Referring concerns to reviews Introduction 1 This guide outlines how the Quality Assurance Agency for Higher Education (QAA) can refer concerns identified through its Concerns Scheme to review teams for consideration. Background 2 The Concerns Scheme allows QAA to investigate concerns about the standards and quality of higher education provision, and the information that higher education providers produce about their learning opportunities. Where there is evidence of weaknesses that go beyond an isolated occurrence, and where the evidence suggests broader failings in the management of quality and standards, we can investigate. These concerns may be raised by students, staff, organisations, or anyone else in a formal submission to QAA. 3 The Concerns Scheme has four phases: Screening: to determine if the issues raised in the submission meet the remit of the Scheme Initial inquiry: to test the submission, gather further information, and invite a response from the provider Full investigation: to examine the evidence and determine if it reflects serious systemic or procedural shortcomings, resulting in a published report and recommendations Action plan: to monitor and sign off the provider's action plan in response to the recommendations. Referring concerns to reviews 4 Where a higher education provider has a QAA review visit scheduled within the next nine months, we may investigate the issues raised within that review and not produce a separate report about it. If we choose to investigate through a review, we will pass the information and accompanying evidence to the reviewers. We will explain the nature of the concerns to the provider and invite them to provide a response to the reviewers. The reviewers' subsequent view of the seriousness and validity of the concern may affect the review outcome. 5 The Concerns Scheme is a reactive method and we are unable to predict when concerns will arise. QAA handles concerns differently in review depending on the time they are raised. This guide is structured around how to handle concerns arising at any time: before the review visit during the review visit after the review visit (but before the publication of the review report). 1
6 In addition, some review teams will need to consider during the course of reviews action plans related to concerns and provide advice to QAA on whether the action plan has been successful completed. 7 This guide applies to reviews carried out under the following methods: Higher Education Review, Higher Education Review: Wales; Higher Education Review (Alternative Providers); Higher Education Review (Embedded Colleges, Higher Education Review (Foreign Providers); Annual Monitoring for Educational Oversight; Annual Monitoring for Specific Course Designation and Reviews of Transnational Education. Concerns arising before a review visit 8 Through the process of screening and initial inquiry, the Concerns team may identify evidence of potential systemic weaknesses that it should draw to the attention of the review team. QAA will decide, depending on the level of risk indicated, whether to begin an initial inquiry straight away, or to refer matters to the scheduled review, which may be up to nine months away. In determining the level of risk, QAA will consider whether the concerns represent low, moderate or high risk. 9 Low risks may relate to: minor omissions or oversights a need to amend or update details in documentation, where the amendment will not require or result in major structural, operational or procedural change completion of activity that is already underway in a small number of areas that will allow the provider to meet the expectations of the UK Quality Code for Higher Education more fully. 10 Moderate risks may relate to: weaknesses in the operation of part of the provider's governance structure (as it relates to quality assurance) or lack of clarity about responsibilities insufficient emphasis or priority given to assuring standards or quality in the provider's planning processes quality assurance procedures are broadly adequate, but have some shortcomings in terms of the rigour with which they are applied problems identified are confined to a small part of the provision. 11 High risks may relate to: ineffective operation of parts of the provider's governance structure (as it relates to quality assurance) significant gaps in policy, structures or procedures relating to the provider's quality assurance breaches by the provider of its own quality assurance management procedures information in the public domain which is seriously inaccurate or has the potential to mislead. 12 If the concerns represent a low risk, the Concerns team can refer the issues to the next scheduled review visit if it is taking place within the next nine months. If the next scheduled review is more than nine months away QAA should investigate the concerns through the normal Concerns Scheme process. 2
13 If the concerns represent a moderate or serious risk then QAA will begin an initial inquiry, in order to draw the matter to the provider's attention and gather evidence. At the end of the initial inquiry, QAA will decide whether to: close the case refer any issues to the review team - either concerns or actions identified by the provider to resolve issues or begin a separate full investigation. 14 Issues that require urgent investigation or multiple, highly specialised or complex concerns could justify a full investigation separate from the next review. Alternatively, review teams could take on specialised or complex concerns by extending the visit to allow the team time dedicated to the concerns, or by adding extra reviewers to the team. 15 QAA may refer concerns to a review team at any time in the months, weeks or days before the review visit where a provider's review visit is scheduled within the next nine months. QAA may consider bringing the review forward if the risks represented by the concerns are substantial. QAA expects review teams and providers to be flexible in accommodating what would become a new line of enquiry. QAA will write to the provider advising that the review team will investigate the concerns. The letter will specify the nature of the concerns and invite the provider to make a written response to the review team. QAA will also inform those raising the concerns of this action. 16 In instances where the person making a submission to the Concerns Scheme asks to remain anonymous, QAA will consider the feasibility of investigating the issues through review while maintaining this anonymity. Protecting a person's anonymity might mean evidence should be redacted. 17 The review team will respond to the concerns by evaluating the evidence provided and identifying any additional evidence needed. Teams may also need to revise their meeting agendas, and their list of key staff to meet on the visit. 18 The concerns referred to the review could lead to recommendations or affirmations, and may have an impact on judgement areas. Concerns arising immediately before or during a review visit 19 Where concerns become known to QAA immediately before or during a review visit, the concerns may be investigated during the review visit. This may require an extension of the visit in some review methods and we may add extra reviewers to the team. Alternatively, QAA may investigate the concerns after the visit, using the process described below for 'Concerns arising after a review visit' (see paragraph 25). 20 The Concerns team will act quickly to screen submissions to the Concerns Scheme that coincide with a review visit so that where possible issues can be referred and resolved through review. In instances where the person making a submission asks to remain anonymous, QAA will consider the feasibility of investigating the issues through review while maintaining this anonymity. Protecting a person's anonymity might mean that evidence cannot be shared with a team, or should be redacted. 21 QAA will ensure that concerns are evidenced before referring them to the review team. QAA will not refer unsubstantiated allegations to review teams. 22 When QAA decides to refer concerns arising during a visit to a review team, QAA will invite the provider to respond to the concerns during the meetings and support its assertions with evidence. 3
23 It is possible that the investigation of concerns leads to recommendations or affirmations, and may have an impact on judgement areas. 24 It is not possible to refer concerns to a review team that is already at its judgement meeting, as the team has no opportunity to explore the issues with the provider. In such cases, the concerns should be handled as 'Concerns arising after a review visit' (see paragraph 25). Concerns arising after a review or monitoring visit (but before publication of the review report) 25 QAA will put the publication of review or monitoring visit reports on hold if concerns arise after a review visit to allow them to be investigated. QAA will determine whether the concerns have already been captured by the review team in their report, or whether they represent new issues of which the team were unaware. 26 If the review team has taken account of the issues in their report, QAA will take no further action. 27 If the concerns represent new issues that the review or monitoring team have considered or reflected in their draft report, QAA will take forward the concerns through the normal Concerns Scheme method, during which time the draft report will be put on hold. Full investigation action plans 28 Following any concerns full investigation, QAA requests an action plan from the provider setting out how it will meet QAA's recommendations. Occasionally providers share action plans with QAA as the result of Initial Inquiries. 29 Normally QAA monitors completion of the action plan and signs it off. However, when a provider is due a scheduled review or monitoring visit within nine months, the next review or monitoring team may consider progress against the action plan. In such cases, QAA should ensure the provider is aware of this and that the review or monitoring team receives the Concerns full investigation report and the provider's action plan. 30 The review team will evaluate the completion of the action plan and formulate a distinct statement about progress in completing the action plan for the report. 31 Ultimately, the review team need to check if the concerns action plan can be signed off as complete, and if the team have confidence in the way the recommendations have been met. 32 If actions are ongoing, the team would have to: indicate whether the team is confident the actions taken are relevant and appropriately address the recommendations indicate if delays in the implementation of the action points are justifiable indicate how much time would still be required to complete these before QAA can sign off the action plan consider whether to reiterate, reinforce or revise any recommendations in the review report. 33 Action plans that cannot be signed off on the recommendation of review teams will continue to be monitored by QAA's Concerns team. 4
QAA1633 - June 16 The Quality Assurance Agency for Higher Education 2016 Southgate House, Southgate Street, Gloucester GL1 1UB Registered charity numbers 1062746 and SC037786 Tel: 01452 557050 Website: www.qaa.ac.uk 5