Matching external agency evaluations and institutional quality culture: A Belgian illustration C. Duykaerts - F. Coignoul Promoting Quality Culture in Higher Education Institutions The Role of QA Agencies ENQA - EUA Symposium Brussels September 12-13, 2013 1
And before anyone asks Belgium exists and we can prove it! 2
outline 1. background data 2. presentation of AEQES 3. presentation of the University of Liège and its IQA system 4. promoting quality culture? 5. concluding remarks 3
background data (i) key features of Belgium Independence in 1830 Constitutional parliamentary monarchy 3 communities, 3 regions and a federal governement Population of 11 million inhabitants Brussels as a national, a European and an international capital [further information : www.belgium.be] 4
background data (ii) higher education in the Frenchspeaking community Long academic tradition, great variety of HE providers and study programmes HE is a public good : social dimension appr. 200.000 students 79% in Bachelor; 16% in Master and 4% in PhD Missions of HE : high-level training, personal growth and fulfilment, service to society, fundamental and applied research (Bologna Act, 31 March 2004) 5
background data (iii) quality assurance in the Frenchspeaking community HE universities 6 HEIs university colleges 20 HEIs adult education 106 HEIs art schools & conservatoires 16 HEIs 6
challenge : find the right balance within the QA system EQA IQA
AEQES brief history & legal framework AEQES 1 : Act of November 2002 all components of higher education evaluation of study programmes self-assessment reports, peer reviews and confidential reports AEQES 2 : Act of February 2008 a more independent agency, evaluation of programmes based on a 10-year planning, selfassessment reports, external reviews, publication of results and follow-up procedures Agency review every 5 years (ESG compliance) ENQA full member in 2011, on EQAR 2012 8
AEQES key features 1. Independent public sector agency 2. formative QA evaluation process, hence no formal effects in terms of institution fundings or authorization (AEQES is not an accreditation agency) 3. programme-based quality assurance (AEQES standards) scope : 1st and 2 nd cycle degrees of the four sectors (i.e. universities, university colleges, art schools and conservatoires, adult education centres) all similar programmes are evaluated simultaneously (clusters approach) system-wide analysis 4. no ranking, no scores 9
AEQES methodology (i) AEQES reference framework (revised in 2012) Five criteria + guidelines 1.HEI implements a policy for maintaining its programmes quality 2. HEI implements a policy for ensuring the relevance 3. HEI implements a policy for ensuring the internal coherence 4. HEI implements a policy for ensuring the efficiency & equity of its study programme 5. HEI comes up with an action plan for continuous improvement 10
AEQES methodology (ii) 1 2 Appointement of the chair of the panel and of its experts Writing and sending of the SER 3 Pre meeting of the chair of the panel 4 Visit of the expert panel with AEQES staff. Meetings with HEI staff, present and former students, employers 5 Observations of the Academic authorities Writing of the draft report 6 Writing of the final report* 7 8 Writing of the system-wide analysis * Writing of the follow-up action plan and its calendar* 9 Implementation of the follow-up action plan BEFORE THE VISIT VISIT FEEDBACK FOLLOW-UP * Published on the website www.aeqes.be By the HEIs By the Agency By the experts with the help of AEQES staff 11
The University of Liège Internal quality system at the University of Liège to solve 3 emerging problems No long term strategy Pressures on transparency and accountability Increased number of faculties and geographic spread in 3 provinces 12
9 Faculties, 1 School, 1 Institute Philosophy and Letters Law - Political Science - Criminology Medicine Applied Sciences Veterinary Medicine Psychology and Education Gembloux Agro-Bio Tech Architecture HEC Management School Institute for Human and Social Sciences 13
and 20,000 students 14
The internal quality System of ULg SMAQ «Service de Management et d Accompagnement de la Qualité» Structure in charge of internal evaluations and supportive of external evaluations and quality related initiatives Set up in October 2009 Headed by a vice-rector in charge of quality Three full time administrative staff members 15
The internal quality System of ULg CATQUAR «Commission d Aide Technique à la Qualité et d Analyse des Référentiels» SMAQ internal support and advisory committee Composed of representatives of education, research, and support : 5 members from the academic scientific fields 5 members from the administrative staff 5 members representing students (including doctorate students) 5 members of the support staff 16
ULg standpoint on quality Internal quality in universities is unique and different for each institution with different priorities sustained by strategic choices requiring a large degree of autonomy with procedures adapted to expected results 17
ULg standpoint on evaluations Self evaluation is the cornerstone of the process The main and maybe unique purpose of evaluations is self improvement Values and objectives that sustain evaluations need to be discussed and agreed upon The «consumer» standpoint must be considered Evaluations must be recognized as beneficial by the evaluated structure 18
helped by the «Bologna decree» in 2004 Higher education institutions are bound to ensure the quality monitoring and management of all their missions; [...] in the field of education, institutions must comply with AEQES procedures. 19
helped by similar procedures Self evaluation External evaluation Action plan 20
helped by shared values «the Agency emphasizes the values that are the warrants of the diversity and richness of our higher education provision rather than focuses on norms and control dialogue and creativity take the lead on formalism.. control belongs to the public authority which mission is to guarantee the independence of higher education and to maintain it in public hands.» AEQES Chairman 2009-2011 21
SMAQ values - ambitions Critical thinking culture to become familiar with strategy and objectives Active participation to promote collaborative decision processes Negociation and dialogue with the leadership to discuss and share a vision Transparency to strengthen mutual trust Planification of changes to improve coherence of decisions 22
SMAQ evaluates study programmes, research, and administrations 23
An emphasis on action plans Departments action plans are crucial for the management to be accurately informed and to build a coherent strategy along the lines of existing strengths and priorities They are given a specific attention during both internal and external evaluations 24
Quality is a roadmap for the leadership 25
is AEQES promoting quality culture in higher education institutions? AEQES is an independent public sector agency, practising formative evaluation based on a dialogue between all stakeholders within the French-speaking Community. Fully embedded in the European context, the Agency is responsible for assessing the quality of higher education and working for its continuous improvement. [ ] from AEQES mission statement See http://www.aeqes.be/english_about_us.cfm 26
AEQES : link between values & procedures formative approach no ranking, no scores, no formal consequences context-sensitive evaluations, follow-up revised reference framework : self-reflexion against bureaucracy and window-dressing participative approach composition of the Internal QA commission composition of the panels, interviews & oral feedback debriefing with the HEIs on the whole process AEQES steering committee and its WG 27
towards quality culture? Source: Facteurs humains et organisationnels de la sécurité industrielle : un état de l art. Les Cahiers de la sécurité industrielle 2009
AEQES next developments A new decree is needed : AEQES 3? see AEQES position paper dated June 18th 2012 http://www.aeqes.be/documents/20121004%20aeqes%20po sition%20paper%20of%20june%2018.pdf among others: improve the periodicity of the evaluations strenghten the follow-up site visit focus on action, after the diagnosis done by the evaluation process complete the quality loop P D C A explore the impact studies adapt methods & procedures to the evolving needs of HE 29
to conclude : AEQES expected impact and challenges Overall quality enhancement Collaborative work between HEIs and other stakeholders Implementation of shared values Development of purposeful QA systems Handle diversity across higher education Communicate (results) in the best manner Fight bureaucracy and help spread quality culture 30
Thank you for your attention! Freddy COIGNOUL f.coignoul@ulg.ac.be Caty DUYKAERTS caty.duykaerts@aeqes.be 31