Situation Assessment. Co-funded by:

Transcription

1 Referencing of the Latvian Education System to the European Qualifications Framework for Lifelong Learning and the Qualifications Framework for the European Higher Education Area Situation Assessment 2013 Riga, December 2013 Co-funded by: 1

2 The Assessment was prepared by the Academic Information Centre. Experts involved in preparing the assessment and collecting information: Baiba Ramiņa, Rihards Blese, Juris Dzelme, Andra Šenberga, Academic Information Centre Academic Information Centre Higher Education Quality Evaluation Centre State Service of Education Quality This Report was developed with the support of the European Commission projects LLP LV-KAI-EQF_NCP Activities of the EQF National Coordination Points with a view to implement the EQF at national level. The opinions expressed are those of the author(s) only and should not be considered as representative of the European Commission s official position. 2

3 Contents Abbreviations... 4 Introduction Assessment of Level 1 4 Descriptors in Latvian Qualifications Framework and Proposals for Improvement Reforms in the Quality Assessment of Higher Education Implementation of Learning Outcomes Recognition of Knowledge and Skills Acquired outside Formal Education Appendix 1. Proposed amendments regarding the first and second qualifications level to Amendments to the Cabinet of Ministers Regulations of 2 December 2008 No. 990 Regulations on the classification of the Latvian education Appendix 2. Proposals regarding compatibility of Latvia s formal education credentials with EQF and LQF levels Appendix 3. Proposed wording of the descriptions of knowledge, skills, competences and education credentials included in LQF Appendix 4. The outcome of ESF Project Development of a sectoral qualifications system and improvement of the efficiency and quality of vocational education and training - evaluation of the compatibility of 1 st, 2 nd, 3 rd level sector qualifications with EQF/LQF level Appendix. Regulations on the classification of Latvian education Appendix. Questionnaire for interviews

4 Abbreviations ACHEP Accreditation Commission of Higher Education Programmes AIC Academic Information Centre CEEN Central and Eastern European Networking Association CoM Cabinet of Ministers of the Republic of Latvia CP credit points EAQAN Euroasian Quality Assurance Network ECTS European Credit Transfer System EHEA European Higher Education Area ENQA European Association for Quality Assurance in Higher Education EQANIE European Quality Assurance Network in Engineering Education EQF European Qualifications Framework ESF European Social Fund HEC Higher Education Council HEQEC Higher Education Quality Evaluation Centre INQAAHE International Network for Quality Assurance Agencies in Higher Education LQF Latvian Qualifications Framework MoES Ministry of Education and Science of the Republic of Latvia NCP National Coordination Point NQF National Qualifications Framework QF-EHEA Qualifications Framework of the European Higher Education Area AR Accreditation Rules SAC Study Accreditation Commission 4

5 Introduction The general aim of the European Qualifications Framework (EQF) as a system of common references is to facilitate the development of the principle of life-long learning and to promote inhabitant mobility between states. EQF offers to Latvia the possibility to describe its system of education, to make it and the documents of education more understandable to other countries. Simultaneously, also inhabitants of Latvia can better understand the systems of education in their own and in other countries, i.e., documents of education, which have been referenced to EQF. Referencing the qualifications of national education to EQF offers a number of advantages: the implementation of Qualifications Framework will, definitely, improve the international comparability for the qualifications of Latvian education, as well as help employers better understand the process of education and its outcomes. A student, a pupil and a trainee will develop greater clarity about the learning/ study process and better life-long learning opportunities will be provided. The descriptors of qualification levels, which are based upon learning outcomes, can be used and are already used as guidelines for elaborating education programmes. However, the process of referencing contains also risks, which must be taken into consideration in referencing the qualifications of national education to EQF. To prevent hasty and ill-considered organisation of the referencing procedure, it is implemented in two stages in Latvia. The first stage, , was the elaboration of the Latvian Qualifications Framework, as well as referencing the system of formal education, in place in Latvia, to the EQF for life-long learning and the Qualifications Framework of European Higher Education Area (EHEA). The second stage was planned for , as an update of the self-assessment report, on the basis of the new Vocational Education Law, Law on Institutions of Higher Education, as well as the results of a number of existing pilot-projects, for example, ESF project Development of sectoral qualifications system and increasing the efficiency and quality of vocational education ( ). The planned achievements, on which the scheduled review of the self-assessment report had to be based, have not been fully attained (the laws have not been drafted, the project has been extended), therefore the National Coordination Point in 2013 conducted an assessment what kind of changes had occurred in Latvia s system of education with regard to aspects of EQF implementation and what was needed for successful implementation of EQF. In terms of ESF project Development of sectoral qualifications system and increasing the efficiency and quality of vocational education ( ), which is implemented by an institution under MoES the State Education Development Agency, it is planned to explore occupations in 12 sectors, as well as tasks, knowledge, skills and competences relevant to the professional activity of these occupation, and to elaborate the structure of qualifications, grouping the occupations related to 5

6 the sector: basic occupations, related occupations and specialisations. Appendix 4 comprises recommendations for this project. All stakeholders must be involved in the referencing process, therefore the Academic Information Centre in cooperation with the Ministry of Education and Science, also following the submission of the self-assessment report to the European Commission, continues organising various seminars, conferences, discussions to inform society about European and Latvian qualifications frameworks and to identify deficiencies in elaboration and implementation of qualifications infrastructure. The referencing procedure in Latvia has resulted in the elaboration of eight-level Latvian Qualifications Framework (LQF). The descriptors of the qualification levels are based upon learning outcomes, and the qualifications in formal education are linked to the aforementioned levels. The level descriptors were drafted in accordance with the national standards of education and occupations, as well as EQF level descriptors. LQF covers the fields of formal higher, vocational and general education. The table below contains the placement of qualifications of Latvian formal education in eight LQF and EQF levels. Table 1. The placement of the Latvian formal qualifications in the LQF and EQF Latvian education documents (qualifications) Certificate of general basic education (for students in special educational programmes for students with severe mental development disorders or several severe development disorders) Certificate of general basic education (for students in special educational programmes for students with mental development disorders) Certificate of general basic education Certificate of vocational basic education Certificate of general secondary education Certificate of vocational education Diploma of vocational secondary education Diploma of first level professional higher education (1 st level professional higher (college) education, the length of full-time studies 2-3 years) Bachelor s diploma Professional Bachelor s diploma Diploma of professional higher education, diploma of higher professional qualification (2 nd level professional higher education, the length of full-time studies at least 4 years) LQF and EQF level

7 Master s diploma Professional Master s diploma 7 Diploma of professional higher education, diploma of higher education, diploma of higher professional qualification (2 nd level professional higher education, the total length of full-time studies at least 5 years) Doctor s diploma 8 7

8 When developing the LQF levels, they were made compatible with the QF-EHEA levels (see figure below). Figure 1. The compatibility of LQF/EQF levels with the QF-EHEA levels EQF levels LQF levels QF-EHEA levels rd cycle nd cycle st cycle 5 5 short cycle While preparing the referencing report, experts agreed that by referencing programs of professional secondary education and vocational education the existing inequality and artificially created differences are reduced. As illustrated in Figure 3 below, at present five Latvian professional qualification levels impart qualifications only up to professional higher education level, i.e. not including qualifications 8

9 corresponding to the LQF level 8. When developing the LQF in the future, it will be taken into account that both a Doctor s degree and vocational education qualifications, which do not belong to higher education, may be referred to LQF level 8. Yet currently such vocational education qualifications do not exist in the Latvian education system. Figure 3. The placement of five Latvian professional qualification levels 1 on eight LQF/EQF levels 2 LQF/EQF levels (2010) 8 7 Latvian professional qualification levels (1999) The project, implemented with ESF support, for assessing study programs in higher education and elaborating recommendations to improve the quality of education Improvement of study programmes content and implementation and development of academic personnel s competence envisaged conducting detailed assessment of study programs in 40 institutions of higher education throughout Latvia within 2 years, to improve the quality, effectiveness and international competitiveness of higher education and to verify the compatibility of study programs with the needs of economy. In major changes were introduced to the system of accreditation of higher education in Latvia, implementing a new type of accreditation of study programs, which replaces the current accreditation of individual programs with the accreditation of study directions. Section 2 of the Report presents the changes in greater detail. 1 Vocational Education Law ( ) 2 Amendments to CoM Regulation of 2 December 2008 No.990 Regulation on the classification of Latvian education ( ) 9

10 The use of learning outcomes is an important component of the European Qualifications Framework. To identify the understanding of learning outcomes in the field of education and their implementation, a study on the use of learning outcomes in Latvia s institutions of education was conducted; Section 3 of this Report provides information on it. 10

11 1. Assessment of Level 1 4 Descriptors in Latvian Qualifications Framework and Proposals for Improvement. The level descriptors for Levels 1 4 in Latvian Qualifications Framework (hereinafter LQF) were examined to assess, whether the compatibility of professional qualifications with EQF should be reexamined, and, if necessary, to draft proposals for amendments to these level descriptors in regulatory enactments. Currently two different level descriptors are binding in vocational education: 1) The one defined in Section 5 of the Vocational Education Law, which provides descriptors of five levels of vocational qualifications 3 ; 2) The levels of European Qualifications Framework (hereinafter EQF) and their descriptors included in Table 2 of the Cabinet Regulation of 2 December 2008 No. 990 Regulation on the Classification of Latvian Education. Whereas Table 1 of Annex 1 to this Regulation defines the compatibility of a concrete EQF level with the level of education and type of education program. Assessment of the classification included in Table 1, Annex 1 to the Cabinet of Ministers Regulation of 5 October 2010 No.931 Amendments to the Cabinet Regulation of 2 December 2008 No.990 Regulation on the Classification of Latvian Education The First and the Second Qualification Level and its Comparison with the International Standard Classification of Education (ISCED-97) and the European Qualifications Framework (EQF) shows that the following amendments are needed: 1) Section First stage of basic education ; education included in 10T Continuing vocational education (to obtain 1 st level professional qualification) should be implemented among adults without restrictions regarding former education. Such programs of education are implemented to obtain very simple qualifications (for example, construction worker, cook s assistant, etc.), 3 Vocational Education Law. Section 5. Levels of Vocational Qualification There are five levels of vocational qualifications specified in the education system of Latvia (hereinafter qualification levels): 1) the first qualification level theoretical and practical training, which provides an opportunity to perform simple tasks in a specific sphere of practical operation; 2) the second qualification level - theoretical and practical training, which provides an opportunity to perform independently qualified artisan work; 3) the third qualification level higher theoretical preparedness and professional skill, which provides an opportunity to perform specific artisan duties, which also include planning and organising of the work to be implemented; 4) the fourth qualification level theoretical and practical training, which provides an opportunity to perform complicated artisan work, as well as to organise and manage the work of other specialists; and 5) the fifth qualification level higher qualification of a specific sector, which provides an opportunity to plan and also perform scientific research work in the relevant sector. 11

12 acquiring basic knowledge and skills for performing previously known, simple tasks under supervision, therefore this education should be defined as complying with Level 2 of EQF/LQF. 2) Section Second stage of basic education 21 General education, basic education (Grades 1-9) programs, proposal to change compliance with EQF/LQF from 1 2 level. In accordance with provisions of Cabinet Regulation of 19 December 2006 No Regulations Regarding the State Basic Education Standard and Basic Education Subject Standards, the primary objectives of basic education programmes include providing an educatee with the basic knowledge and basic skills necessary for social and personal life; creating a basis for further education of an educatee; encouraging harmonious formation and development of an educatee; promoting responsible attitude of an educatee towards himself or herself, the family, society, surrounding environment and the State. Whereas the primary tasks include: forming preconception and understanding regarding primary natural and social processes, moral and aesthetic values; providing an opportunity to acquire basic knowledge of and basic skills in language and mathematics; providing an opportunity to acquire basic learning abilities and basic skills in using information technologies, the knowledge and democratic values necessary for a citizen of Latvia, to gain experience in creative activity; to form a basic concept of the cultural heritage of Latvia, Europe and the world, as well as cultivating communication and co-operation skills. A comparison with the European level descriptors, based upon learning outcomes (see publication Referencing of the Latvian Education System to the European Qualifications Framework for Lifelong Learning and the Qualifications Framework for the European Higher Education Area, second version, p.56), allows concluding that the knowledge of the basic facts of the field of study, various basic skills, which are needed in order to use information, perform tasks and solve routine problems, as well as competences, which ensure work or studies, conducted under supervision with certain autonomy, ensure maximum compliance of the knowledge, skills and competences acquired by a person with the 2 nd EQF level. The level descriptors of the 3 rd European learning outcomes based level contain higher requirements, which comprise already the knowledge of facts, principles and general concepts. At the same time Latvia must take into consideration the outcomes for this level of education that the national qualifications frameworks of other countries comprise, as well as the results of international research on education (for example, OECD PISA 2009), which shows that Latvian young people aged 15 demonstrate outcomes in mathematics below the average EU member state level, but in literary and natural science the level complies only with the EU average (see In view of the above-mentioned, Latvia has no grounds to classify the knowledge, skills and competence outcomes acquired in basic education as EQF/LQF 3 rd level, which is higher than the classification of other EU member states. 3) In section Second stage of basic education ; 22 Vocational basic education, to be implemented without restrictions regarding prior education, compliance with 2 nd EQF/LQF level 12

13 should be defined, since the majority of vocational basic education programmes are implemented in special institutions of education (from among the vocational basic education programmes licensed in % were licensed for implementation at special institutions of education, for persons with special needs), inter alia, for persons with mental health disorders, mental development disorders, multiple mental development disorders, etc. following acquisition of special basic education programmes. In these vocational basic education programmes simple knowledge and skills in the field are acquired, the lowest qualification in the sector is obtained (see Annex 4) and the acquired knowledge, skills and competence do not reach the level defined by the descriptors of 3 rd EQF/LQF level, thus decision must be taken regarding the classification of this kind of education as 2 nd EQF/LQF level. 4) Section Secondary education level 32 Vocational education (acquisition of 2 nd level professional qualification), it is proposed to express the last phrase in the following wording: Vocational education (acquisition of 2 nd level professional qualification, to be implemented following acquisition of basic education partially or fully) and to define compliance with 2 nd 3 rd EQF/LQF level, according to the duration of implementation of the concrete vocational education program and rules of enrolment, since: - As the descriptions of 14 sectors and qualifications structures show, the differences between the knowledge, skills and competences of the 2 nd qualification level, ensured by vocational education, and the 3 rd vocational qualification level, which is ensured by vocational secondary education (see Appendix 4), are significant, therefore the competence ensured by vocational education cannot be classified on the same level as the competence obtained in vocational secondary education (see - Section 27(1) of Vocational Education Law provides that admission is equal with vocational basic education, i.e., persons are admitted to programmes of vocational basic education and vocational education without any restriction regarding prior education and not earlier than in the calendar year when the person becomes 15 years old. Since the law has a higher legal force compared to CM regulations, Latvia may encounter the practice of preparing vocational education programmes, planning to admit persons without previously acquired basic education. Such education programmes should be classified as the 2 nd EQF/LQF level; - Only vocational education, which has been implemented in compliance with the scope defined in the 1 st line of Table 1, Appendix 3 to the Cabinet Regulation of 2000 No. 211 Regulations on the state standard of vocational secondary education and state standards of vocational education - programmes, which the educatee starts after having obtained 13

14 basic education and with 3 year duration of studies should be classified at the 3 rd EQF/LQF level. 5) Section Secondary education level 35a Vocational secondary education (acquisition of 2 nd level professional qualification), to be implemented following acquisition of basic education, there are no grounds to define compatibility of this education with 4 th EQF/LQF level, since the structure of qualifications of sectors does not envisage an approach that different levels of knowledge, skills and competences can be set for one qualification (for example, tailor, carpenter, cook, etc.). Thus, in this instance 3 rd 3 EQF/LQF level should be defined. 6) Section Secondary education level 30T, to express the 4 th column in the following wording: Continuing vocational education (acquisition of 3 rd level professional qualification), to be implemented following the acquisition of general or vocational secondary education, since: - Section 27(7) of Vocational Education Law provides that a person is admitted for vocational further education programmes, which provide an opportunity to acquire a third level professional qualification, after the acquisition of a vocational basic education or secondary education. - According to the qualifications structures of various sectors, in view of the significant differences in the descriptors of knowledge, skills and competence levels for the 2 nd professional qualification level (for example, tailor, cook, hair-dresser, etc.) and for the 3 rd professional qualification level (for example, architectural technician, utilities technician, etc.) (see Appendix 4), the occupations of the 2 nd and 3 rd professional qualifications levels cannot be classified as one EQF/LQF level. Thus, only occupations of the 3 rd professional qualifications level can be classified as EQF/LQF 4 th level. Following the assessment of the compatibility of documents certifying acquisition of formal education with EQF/LQF levels, resulting from the elaboration of Latvian Qualifications Framework and the first stage of referencing the existing qualifications in formal education to EQF (see proposals for wording of document descriptions are provided in Appendix 2, inter alia, it is proposed to move the document of education included in the descriptor of the 4 th EQF/LQF level certificate of vocational education to the 3 rd EQF/LQF level, since the qualifications included in the descriptors and qualifications structures of 14 sectors point to significant differences between qualifications/ occupations, acquired in vocational education (2 nd level of professional qualifications) and qualifications acquired in vocational secondary education (3 rd level of professional qualifications) (see Appendix 4). At the same time additional entries must be made regarding the certificates of professional qualifications in the descriptors of the 2 nd, 3 rd and 4 th EQF/LQF levels, since programmes of professional continuous education are implemented for 14

15 the acquisition of occupations/ qualifications of the 1 st, as well as 2 nd and 3 rd level of professional qualifications. Appendix 3 contains proposals on the wording of Latvian documents certifying acquisition of learning outcomes. Appendix 4 comprises the summary of information included in the qualification structures of 14 sectors, necessary for creating this work, to be transposed to the proposals for amending descriptors of 1 st - 4 th EQF/LQF levels. 15

16 2. Reforms in the Quality Assessment of Higher Education In 2013 a new kind of accreditation of study programs was introduced, replacing the accreditation of individual programs with accreditation of study directions. The previous accreditation of programs of higher educational establishments envisaged that a graduate could receive a document of higher education, recognised by the state, if both the institution of higher education and the programme had been accredited. All new programmes initially received a licence, which functioned as a permission for enrolling applicants in the programme under the condition that the institution of higher education accredited the programme no later than within 3 years following the receipt of the licence. Thus, at the time when the first students graduated, the programme was already accredited. A programme was granted accreditation for 2 years (if a need to improve the programme was identified) or 6 years. In accordance with the new procedure, since 2013 institutions of higher education obtain accreditation for study directions. This means that by accrediting a study direction, all programmes included in this study direction are accredited. A study direction is accredited for 6 years, as previously. For 2 years, if deficiencies that can be eliminated are identified. The programmes that the study direction comprises are listed and described in detail in the Annex to the Accreditation Sheet of the study direction issued to the institution of higher education. On 25 September 2012 the Cabinet of Ministers (CoM) Regulation No. 668 Procedure for Accreditation of Higher Education Institutions, Colleges and Study Directions (AR) was adopted. The main changes, compared to the previous CoM Regulation of 3 October 2006 No.821 Procedure for Accreditation of Higher Education Institutions, Colleges and Higher Education Programmes", are the following 1) transition from assessment and accreditation of study programmes to the assessment and accreditation of study directions; 2) since 2013 the assessment and accreditation is organised by the Ministry of Education and Science (MoES); 3) experts are selected according to the principle of randomness. To relfect the respective changes, adjustmens must be made to the self-assessment report, as indicated in Figure 5 (SR, p. 50) replacing HEQEC with MoES and CAHEP with SAC. Moreover, error on p.51 of SR must be eliminated, noting that a study programme must be compared with similar study programmes in Latvia and at least 2 study programmes in various EU countries. 16

17 As previously, in order to receive a state recognised diploma, both the institution of higher education (and all its separate branches) and the programme (separately in branches) must be accredited. The accreditation already granted to the institutions of higher education is valid and it will not be necessary to re-accredit it, however, those new institutions of higher education, which thus far had not yet obtained accreditation, will be able to receive it only if at least half of the study directions will be accredited. The list of institutions of higher education, which previously have been accredited, is available from the homepage of HEQEC: The list of newly accredited study directions (and the programmes they comprise) is available from Excel table on the homepage of the Ministry of Education and Science: (see Accredited Study Directions and programmes ). The main changes in Regulation on Licencing Study Programmes, adopted on 25 April 2013, (compared with the previous) are as follows: 1) the composition of the licencing commission is identical with the composition of the accreditation commission, therefore the study accreditation commission (SAC) can perform the tasks of the licencing commission and actually does it; 2) a new study programme is added to an accredit study direction on the basis of a decision by the accreditation commission and, in fact, the new study programmes are immediately accredited; 3) the decision on creating a new study direction at an institution of higher education or college is adopted by the CoM; 4) an expert for assessing a new study direction or study programme is selected according to the principle of randomness. Accordingly, these changes must be reflected by introducing amendments into the Self-Assessment Report, prepared by AIC (Riga 2011) (SR), inter alia, supplement Figure 4 (SR p. 49) with the procedure for selecting the expert and organising work, as well as the procedure for adopting CoM decision on creating a new study direction at an institution of higher education or college. (The overlapping of the procedures for licencing and accreditation in transiting to accreditation of study directions was extensively discussed while the CoM Regulations were drafted, and the decision to merge SAC and licencing commission was adopted). The application of the licencing and accreditation rules that are currently in force to the assessment of study directions started in May The majority of study directions in Latvia were accredited by using the evaluation results of study directions and study programmes from the European Social Fund project Evaluation of Higher Education Study Programmes and Proposals for Quality Improvement (Agreement No.2011/0012/1DP/ /11/IPIA/VIAA/001) 17

18 (hereinafter ESF Project). Three different systems for evaluating higher education have been used in the recent years: 1) Evaluation and accreditation of higher education (institutions of higher education, colleges and programmes thereof) in accordance with CoM Regulation No. 821 adopted on Procedure for Accreditation of Higher Education Institutions, Colleges and Higher Education was conducted until March The evaluation was organised by HEQEC. 2) All Latvian study programmes, divided into 28 study directions, were evaluated in the framework of ESF Project from September 2011 to June Almost all Latvian study programmes, divided into 29 study directions were accredited, using the ESF Project outcomes. (Due to various reasons (an institution of higher education did not participate in ESF Project or did not register all study programmes, dissatisfaction with the evaluation received in the ESF Project, new study programmes created recently, etc.), approximately 30 study programmes, which need accreditation, have not been evaluated yet.) In the ESP Project, the implementation of which was organised by HEC, the evaluation of all study programmes, divided into 28 study directions, was organised by HEQEC (selected all expert candidates, provided transportation and accommodation, organised the visits, provided methodological assistance). 3) The evaluation (and accreditation) of higher education (institutions of higher education, colleges and programmes thereof) in accordance with the Cabinet of Ministers (CoM) Regulation adopted on 25 September 2012 No. 668 Procedure for Accreditation of Higher Education Institutions, Colleges and Study Directions commenced in May Some of 30 study programmes, needing additional evaluation, outside the ESF Project, have been evaluated, to solve the problems of accreditation and licencing. Even though formally all 3 methods of evaluation applied recently differ as to their methods and are organised in a different way, in all cases great focus is placed upon the study outcomes, and the compliance with European principles and guidelines for higher education was verified. The composition of experts changed very little, the same database of experts and similar principles for selecting experts and expanding the database (searching for new experts) were used. The new CoM Regulation Procedure for Accreditation of Higher Education Institutions, Colleges and Study Directions sets out clear and very detailed criteria and requirements for elaborating the self-assessment report, for evaluating study directions and study programmes. Appendix 6 and 7 to AR define the criteria to be used by experts: The consistency of the study programmes appropriate for the study direction and separate parts thereof with the requirements regarding the development of Latvian and common European education area, inter alia, comparison of each study programme of all study directions with at least one study programme of the same level and corresponding to the same study direction in Latvia (if a similar study programme is 18

19 implemented in Latvia) and at least two study programmes of institutions of higher education or colleges recognised by the European Union member sates. (Criteria defined in Appendix 6 to AR). While evaluating study programmes experts must use the document elaborated by European Association for Quality Assurance in Higher Education (ENQA) "Standards and Guidelines for Quality Assurance in the European Higher Education Area (hereinafter ESG), must assess the aims and objectives of study programmes, how clear, attainable and verifiable these are, consistency of the study programme and its separate parts with the requirements for creating Latvian and European common education area (inter alia, comparison of the study programme with at least two study programmes of the European Union member states). Experts in their opinion must include answers to the questions: Are the expected study outcomes of the studies clearly presented, are problem solving skills developed in practice? Do the students, who are enrolled in the study programme, have appropriate knowledge, skills and competence to attain the planned study outcomes within the envisaged time? Does the testing of interim-results during the course of studies ensures reaching the study outcomes of the study programme? (requirements to experts defined in Appendix 7 to AN). The experts, in evaluating study programmes and study directions, use holistic, integral approach, paying more attention to the functioning of quality management and assurance system in institutions higher education and colleges. If these internal systems function, then during the external inspection there is no need to conduct an in-depth study of details of the operation, for example, exact analysis of the wording of the study outcomes and checking the compliance with Latvian and European recommendations and regulatory enactments, then it is possible to focus only upon discussions about various development options and future outlooks, opportunities and risks. Therefore, the experts reports have rather different level of detailed analysis of the use of study outcomes, but the problem in general has been evaluated in all cases. If the experts agree that the compliance with the requirements regarding development of Latvian and European common education area has been attained, it is not always specially highlighted; thus, the experts reports sometimes lack more detailed analysis, but contain only general positive assessment. A recommendation could be made to the experts to provide a more detailed description of the situation, even in the absence of significant deficiencies, since the issue of using study outcomes, EQF and NQF gains importance with the growing diversity in the content of study programmes and methodological solutions. Improving Organisation of Quality Assessment Standard 3.6 of European Standard and Guidelines (ESG) elaborated by ENQA provides: Agencies should be independent to the extent both that they have autonomous responsibility for their operations and that the conclusions and recommendations made in their reports cannot be influenced by third parties such as higher education institutions, ministries or other stakeholders. 19

20 the definition and operation of its procedures and methods, the nomination and appointment of external experts and the determination of the outcomes of its quality assurance processes are undertaken autonomously and independently from governments, higher education institutions, and organs of political influence. Currently in Latvia, the only one among the EU states, this standard is not implemented, therefore the informative report prepared by MoES (submitted to CoM on ) On the course and procedure of accreditation of study directions, according to which starting with 1 January 2014 the accreditation of higher education institutions and colleges, as well as study directions will be organised and conducted, using European quality assurance agencies registered in the European Quality Assurance Register (EQAR) analyses the possibility of decreasing the direct interference of MoES and CoM in the quality assessment of higher education. MoES wants to involve foreign specialists and organisations in the quality assessment of higher education and accreditation, to renew compliance with ESG requirements and to ensure that the evaluation is international. The main part, and the one most visible to society, of quality assessment of higher education is expert group visits to the institution of higher education (IHE) and the following report on the quality of the higher education institution, college or study program. In Latvia this part is international, since in almost all commissions part of the experts come from abroad (1 st level vocational programmes and colleges are exceptions). The second part of quality assessment is its organisation: searching for experts and their training, planning and implementation of the visit, summarizing and publishing the results. This work must be done by an independent institution, which could be both Latvian and foreign. The foreign institutions, which comply with ESG requirements, have developed procedures and methods appropriate for their own states. Aligning these with Latvia s requirements would be a difficult process, in view of the fact that the current AR contains numerous rather specific elements of methodology intended for Latvia, among them, the ones elaborated in the ESF Project Assessment of study programmes of higher education and recommendations for improving the quality. Moreover, it should be taken into consideration that approximately half of the EU states do not have EQAR registered quality assurance institutions, thus, the choice could be rather complicated. The Use of Distance Learning and Prior Experience in Assessment Distance learning, the Internet resources and systems like MOOC are more frequently used in the study process. The use of combined methods, combining of virtual space with direct studies ( blended learning ), the use of the Internet, combining formal, non-formal and informal learning, and the use of shared programmes is very effective. The quality assurance system must pay special 20

21 attention to these new forms; parallel use of diverse methods of learning should be facilitated with the assistance of NQF and the system of learning outcomes. Special attention should be paid to the assessment of the operations of the branches and contact points of higher education institutions and colleges, ensuring that equal study outcomes are attained. Appropriate criteria should be used in assessing the branches and shared programmes. The use of NQF and the study outcomes should facilitate comparison and merging of parts of education obtained under various circumstances, integrating these in united, internationally aligned and recognisable qualifications. Criteria for using previous experience, assessing recognition of the study outcomes acquired in non-forma and informal learning should be developed and used in quality assessment. Experts should present to students, faculty members and society the best practice examples, should help to select and apply the most appropriate, effective methods of study. Assessing the changes achieved in study process Extensive use of life-long learning involves students with very diverse levels of previous training in the study process. In assessing the study outcomes experts should pay attention not only to the final, but also the initial level and the path for reaching the planned study outcomes from various points of departure. The experts should especially support greater changes in the level of knowledge, skills and competences, should help popularise and distribute methods that ensure students fast progress in the study process. Changing the System of Occupational Standards In Latvia two systems of standards define the structure and the study outcomes of a study programme: 1) education standards define requirements with regard to the formation of the structure of the study programme, regulate the general objectives of the study programme, existence and minimal scope of the main parts, define the framework for the content of the study programme; 2) occupational standards are oriented towards defining the content of professional study programmes, as these define the outcomes to be achieved knowledge, skills and competence to be acquired in order to obtain qualification. 21

22 The system of occupational standards established in Latvia has a number of deficiencies, which could be eliminated by amending the CoM Regulation: - the occupation standards are erroneously oriented towards job descriptions and Classification of Occupations, the terminology used is not exact, since the occupation standards must be used for regulating the content of education; - the drafting of occupation standards and their approval by CoM is too long and too bureaucratic, it lacks sufficient support, therefore occupation standards poorly reflect the relevant needs of society and often become outdated before approval, are oriented towards the past, not the future; - in the regulated occupations the occupation standards unnecessarily duplicate international legal acts. Proposals for possible improvements to the system of occupational standards: - the system of occupation standards should be oriented towards descriptors of qualifications and Classification of Education, first of all towards NQF, clearly separating the learning outcomes and qualifications descriptors to be used in the regulatory enactments within the system of education from job descriptions (as regards terminology, specify the use of the word occupation ). - significantly reduce the number of occupation standards, using approximately from 8 to 29 occupation standards (no more than one standard for one direction of study); - in regulated occupations adopt internationally recognised standards, amending and/ or supplementing them in Latvia only in case of exceptional necessity; - to add to the regulation on the use of occupation standards the requirement to discuss mandatorily the study outcomes set for obtaining the qualification with all stakeholders, using Sectoral Expert Councils and other similar structures; - to add to the quality assurance system (internal and external) the requirement to pay special attention to study outcomes, updating them and defining and applying a part on competences and attitude (formation of motivation); - to ensure sufficient, also financial, state support for the elaboration of occupation standards. 22

23 3. Implementation of Learning Outcomes In Europe learning outcomes are used increasingly more often as the basis for standards of occupations and education, curricula, assessment criteria and descriptors of qualifications and levels of education in national qualifications frameworks (NQF). All these tools define the learning outcomes on various levels of specificity and envisage them for a number of goals, for example: to define the abilities to be expected from a person, who has obtained a concrete qualification of education; for managing the learning process, for managing the assessment process. 4 Legal regulation comprising the learning outcomes approach has been developed also in Latvia. Already since 1 June 1999 the Law on Education clearly defines education as a process of systematic acquisition of knowledge and skills and development of attitudes, and result thereof, i.e., in Latvia learning outcomes have been defined on the level of regulatory enactments already for a decade. The use of learning outcomes in education is ensured by the state standards of education and occupations, the state system of education, as well as accreditation of institutions and programs of education. As regards general education, the state standard of general basic and secondary education, as well as standards of subjects, set requirements for each study subject, i.e., by defining the necessary competences that must be acquired in order to obtain basic or secondary education. Institutions of education elaborate education programmes, but teachers subject programmes in accordance with the state education standards. If the education programmes implemented by a school do not comply with the requirements of state education standard, it cannot be accredited. Upon concluding general basic and secondary education programme students take centralised state exams, the content of which has also been elaborated in compliance with the state standards of education. The content of vocational education is defined by the state standards of vocational education, occupation standards and vocational education programmes. The state standards of vocational education define the strategic aims of programmes of education, the mandatory content of education, as well as the basic principles and procedure for evaluating the acquired education. Occupational standards define the basic objectives and responsibilities, appropriate for the occupation, basic requirements regarding professional qualifications, the general and professional 4 A recent Cedefop study on the content of vocational education revealed that on the level of the content of education and study programmes learning outcomes have two main functions: regulatory and didactic. In the rgulatory function the content of education is a tool, which ensures equally high standard in the education provided throughout the territory in this case learning outcomes must ensure a stable basis for assessment; they must be measurable. In the didactic function the content of education provides a frameowrk for directing the study process in this case the learning outcomes can be broadly defined, including competences that cannot be measured; they reflect values and the role, for which the study process prepares pupils/ trainees/ students. 23

24 knowledge necessary for performing them, skills, attitudes and competences. Institutions of vocational education use state standards of education and occupations to elaborate their own programmes of education; otherwise the respective programme of education cannot receive accreditation. Trainees/ students demonstrate the acquired learning outcomes at the centralised state exams in general subjects (in the programmes of vocational secondary education) and examinations of professional qualifications, the content of which is elaborated in compliance with respective occupational standards. In higher education the institutions of higher education are reviewing the study programmes that their implement in order to implement in them learning outcomes and to pass the accreditation procedure. Moreover, the programmes of higher education must be elaborated in compliance with the appropriate state standards of education and/ or occupation standards (professional higher education programmes). On 22 February 2011 the CoM regulation No.146 Procedure for assessing professional competence acquired outside the system of formal education was approved, it establishes the procedure for assessing the professional competence, acquired outside the system of formal education, which complies with 1 3 Level of Latvian professional qualifications, i.e., 3 rd 4 th level of LQF. As regards higher education (5 8 level of LQF), on 10 January 2012 CoM Regulation No. 36 Regulation on the recognition of study outcomes acquired in previous learning or professional experience, was adopted, issued in compliance with the Law on Higher Education Institutions (1995, amendments in force since ). The Regulation defines the procedure for evaluating and recognising the learning outcomes (in higher education) achieved in previous education or professional experience, as well as criteria for recognition. Credit points are used in higher education, in Latvia these are defined as the amount of work of one-week full-time studies. The amount of credit points envisaged for one study year in full-time studies is 40 credit points. In recalculating into ECTS credit points, the number of Latvian credit points must be multiplied by 1.5. The methodology of study In the framework of European Qualifications Framework project, it was planned to conduct 66 structured interviews in all regions of Latvia and Riga in October and November 2013, to identify the knowledge and understanding of learning outcomes among the administrations and faculty members of institution of education, as well as their use, questions were also asked about expectations in connection with the implementation of learning outcomes approach. 24

25 The interviews were conducted at institutions of education of all levels and types, the qualifications offered by whom are found in Latvian qualifications framework. It was planned to survey 16 institutions of education, among them: 3 in Riga 2 secondary schools 1 secondary school with Russian as the language of instruction 13 in regions 6 basic schools; 3 secondary school; 1 primary school; 1 evening school with two parallel languages of instruction; 1 basic school with Russian as the language of instruction; 1 secondary school with two parallel languages of instruction It was also planned to conduct interviews at 10 institutions of vocational education, of which: 3 in Riga 1 institution of vocational secondary education, under the supervision of the Ministry of Culture; 1 institution of vocational secondary education, which operates within a college; 1 institution of vocational secondary education 7 in regions - 1 institution of vocational education, under the supervision of the Ministry of Culture, 1 institution of vocational secondary education, which operates within a college; 5 institutions of vocational education It was planned to conduct 8 structured interviews at the institutions of higher education, of which: 6 in Riga 1 college established by the state; 1 college established by private persons; 2 institutions of higher education established by the state; 2 institutions of higher education established by private persons 2 in regions 1 college established by the state and one institution of higher education established by the state. The number, type and the proportion (Riga region) of the surveyed institutions in three groups was selected according to the proportional distribution of institutions of education as to their number. This proportion, however, is not observed between the three groups (general, vocational and higher education), since the differences among the total number of institutions of education as to the levels and types of education are too great. Two structured interviews were conducted at every institution of education. One of the interviews was with a representative of the administration, but the other with a faculty member. Interview was chosen as the method for collecting data, since it allows obtaining more accurate answers, by explaining the concept of learning outcomes. Structured interview, with the possibility of commenting answers was selected. This method was chosen to obtain comparable data, which could be used in quantitative analysis; but the rather extensive possibilities for commenting on answers gave the possibility to conduct qualitative analysis and to define recommendations for improving the system of education in the context of learning outcomes. 25

26 Since the sample is not too extensive, wherever possible, the acquired quantitative data were examined in the cross-section of all levels and types of education. Whereas the recommendations and the results of qualitative analysis were defined according to individual levels and types of education, alongside providing shared proposals and conclusions regarding the whole system of education in general. While conducting the study, 30 interviews were performed at institutions of general education, 17 interviews at institutions of vocational education and 14 interviews at institutions of higher education, of which 4 at colleges. In total, 39 interviews were conducted in regions and 22 in Riga. Knowledge about learning outcomes and the main obstacles in implementing them Accessibility of information In general the outcomes of interviews show that the majority of respondents assess the accessibility of information on learning outcomes as good or rather good. 28.3% of the surveyed believe that the accessibility of information is rather good, but 37.7%, see it as good. 26.4% of the respondents assess the accessibility of information as average, but only 5.7% of the respondents as rather poor. Differences in answers according to the levels and types of education can be observed. The staff of institutions of general education has the most positive assessment of the accessibility of the information, but the representatives of institutions of higher education and colleges are the most critical in their assessment of the accessibility of information. Please assess and characterise the accessibility of information on learning outcomes Poor Rather poor Average Rather good Good Hard to say No answer A number of insights on the accessibility of information and ways of improving it were expressed during the interviews. Even though the employees of the institutions of higher education expressed the most critical opinion on the accessibility of information, the representatives of this group were the ones, who mentioned a couple of times the seminars organised by EQF national contact point and other institutions, the seminar materials on learning outcomes, available on the Internet, as a good source of information. The informative seminars organised thus far have been 26

27 mainly targeting the employees of higher education institutions, therefore the data assessment leads to the conclusion that, even though the accessibility of information is not given high evaluation, nevertheless the greatest amount of most extensive information had been available to this concrete group of respondents. Institutions of higher education frequently mention experience of other countries and sources of information in foreign languages as important means for gaining knowledge on learning outcomes. Moreover, occasionally it is emphasized that information from the local sources of information is not sought at all, which means that the local providers of information should increase their authority and credibility, in order to become a full-fledged partner in implementing the learning outcomes approach. At institutions of vocational and general education learning outcomes are predominantly linked with the standard of education, subject standards, and in the context of institutions of vocational education also with the standard of occupations, which explains why the accessibility of information is evaluated positively, since all of the abovementioned is conveniently available from the Internet. The following restricting factors are mentioned during the interviews that the standards are rather difficult and sizeable, that learning outcomes are given different interpretation among institutions of education, which causes the main problems in order for the system to take a uniform approach to this issue. Moreover, the cases, when an institution of education considers in a planned way the interdisciplinary skills and competences in general, if these are not listed in one of the standards, are rather infrequent. It was mentioned during the interviews that it was easy to find enumeration of learning outcomes to be reached, however, it was more difficult to find information on them as a concept and understand their meaning and place in the whole system of education. It can be concluded that the institutions of higher education are better informed, but, at the same time, being aware of the scope of the concept, understand that more information could be made accessible. Whereas the institutions of vocational education, due to the specific features of their work, are more oriented towards labour market and the practical application of knowledge, which facilitates thinking in the framework of reaching learning outcomes; however, not all vocational schools, especially in regions, are sufficiently involved in the circulation of information. At schools learning outcomes are often perceived as implementation of the Cabinet of Ministers regulations, not as a tool for developing and assessing a student s knowledge, skills and competences, therefore the development of interdisciplinary skills and competences is often neglected. Sources of information It was often emphasized during the interviews that, it would possible to find all the necessary information, if only a person wished to do it, since nowadays the most extensive range of 27

28 information can be obtained by using all the possibilities offered by the Internet; this is also reflected in answers to the question Where do you find information on learning outcomes? Where do you find information on learning outcomes? Number Internet Publications Seminars Communicating with institutions From colleagues The Internet has been indicated as the most popular answer, followed by seminars and sharing information among colleagues. Publications and communication with institutions, which deal with the respective issues, were mentioned considerably less frequently. The representatives of higher education institutions, especially, mentioned various web sites, seminar materials, as well as twitter accounts as useful sources of information. The Internet as the source of information was the most popular answer also among the employees of the sector of general education; however, they noted that these sources could not always be trusted and that they lacked knowledge on where to look for necessary information. It is often emphasized that seminars and other types of interpersonal exchange of information are very important for getting to know best practice and for implementing positive changes. The representatives of institutions of higher education not only participate in various seminars and experience sharing events, but also organise them, since the employees of these institutions are often members of working groups dealing with the respective issues on national or international level. The employees of institutions of general and vocational education have high evaluation of experience sharing seminars; however, these are not always accessible to everybody. 28

29 Which would be the preferable way of obtaining information on learning outcomes? Number of responses The institutions of secondary and vocational institutions, surveyed in the regions, expressed the opinion that the administrative territorial reform has resulted in decreasing of information and best practice sharing among faculty members, since active communication, predominantly, takes place within the framework of a district, which often means a smaller number of schools, compared to the time when such sharing of information took place on the regional level. Moreover, an argument with regard to the principle money follows the student was expressed, which does not facilitate the wish of city schools to share their best practice with smaller schools, because due to the decreasing number of children schools actively strive to maintain a stable number of children in all groups of grades. The institutions of vocational education often emphasize the employers opinion and participation in the elaboration of occupation standards, as well as qualification examinations as an excellent way of obtaining recent information on learning outcomes, since the employers satisfaction is an important assessment of the work done by the institution of vocational education. Moreover, it is also a way of establishing professional contacts in order to share best practice also in further work. HEIs Internet Institutions of vocational education Internet Publications Seminars Communicating with institutions From colleagues As regards the preferable way of obtaining information, the general preferable situation does not differ much from the existing one, in terms of preferred channels. Only the demand for publications has increased, but the need to share information with colleagues decreased. Even though minor differences according to the levels and types of education can be observed, it is obvious that the accessibility of information on the Internet and seminars are the preferable channels for transmitting information to institutions of education of all types and levels. Irrespectively of the method for obtaining information, the representatives of institutions of all levels and types of education note that it is important to share best practice and success stories, thus inspiring and supporting changes at other institutions of education. Moreover, these examples should be sufficiently detailed to ensure that the best practice is adopted in a quality way. Schools Internet 29

30 The institutions of higher education point out the need for more extensive use of the Internet tools in providing information on learning outcomes. It is noted that it would be valuable if the What would be the preferable way of obtaining information on learning outcomes? Internet Internet Publications Internet Seminars Internet responsible agencies collected topical information, for example, by creating regular news pages on the relevant topic, then institutions of education would not have to spend so much of their own time looking for credible topical information. During the interviews the seminars that had been held and the accessibility of materials on the Internet were praised, as well as communication on twitter. It was suggested that wider public should be informed about learning outcomes and the qualifications framework in general, using the media. The need to present the main topical issues to the vicerectors of higher education institutions in charge of studies/ learning was pointed out. Whereas the institutions of vocational education and of general education emphasized the need for seminars and experience sharing visits, as well as the need to improve their quality. It is important to see the best practice directly, by visiting other schools, as well as, in the case of vocational schools, by allowing the faculty members to spend internship periods in the companies of the sector, thus ensuring better understanding of the best practice to be adopted. It is also important to explain learning outcomes from the perspective of didactics, on the basis of diagnostic work. It should also be clearly defined, where the teachers of various subjects should turn for advice, if they wish to receive methodological support in order to reach learning outcomes in classes. It can be concluded that the representatives of institutions of higher education are satisfied with the quality of seminars held thus far, but wish to have more extensive possibilities of finding various materials on the Internet. Whereas the respondents from the institutions of general and vocational education note the need to improve the quality of seminars and to develop additional networking tools, not forgetting to ensure availability of the necessary information on the Internet. Knowledge about learning outcomes Communicating with institutions From colleagues Internet Interviewers observed that in the majority of institutions of vocational and general education learning outcomes are interpreted as the evaluation that the students receive in the study process. Of course, when the interviewers explained the aim of the learning outcomes, the respondents adjusted their answers, however, the initial response usually was linked with the grades. Following more 30

31 extensive explanations, the learning outcomes were usually linked with the existing standards. Especially at schools of general education the standards are perceived as direct reflection of learning outcomes. Due to this view, broader view on the learning outcomes approach is often lacking, to quote one of the respondents: there is an opinion that learning outcomes is the general minimum, but not a tool for developing every child s individual abilities and for assessing progress, which is incorrect. Whereas the respondents at institutions of higher education predominantly immediately recognised learning outcomes and mentioned a seminar or a process for improving internal quality, which the institution of higher education was applying or intended to apply. What is the general level of knowledge about learning outcomes in the country? Low Rather low Average Rather high High Hard to say When asked about the general level of knowledge about learning outcomes in the country, over a half (54.7%) of respondents assessed it as average. None of the respondents assessed it as high, but only 13.2% evaluated it as rather high. 1.9% and 26.4% of the respondents evaluated knowledge on learning outcomes as, respectively, low or rather low. Two kinds of approaches can be discerned in the comments. First of all, it is noted that the faculty members and society often have intuitive understanding of the learning outcomes concept, however, would rarely define it in the way required for this interview, as well as, most probably, they would be unable to explain the way learning outcomes fit into the total system of education. Secondly, it is emphasized that the learning outcomes approach and its meaning should be explained to pupils starting with primary school, since currently young people focus more upon the result the grades and the diploma that they get, not upon the content. What is your colleagues level of knowledge about learning outcomes at your institution of education? Low Rather low Average Rather high High Hard to say 31

32 Whereas, when asked about their colleagues level of knowledge at the institution of education represented by the respondent, the situation changes and the assessment is higher compared to society in general. 50.9% respondents state that knowledge is rather high, but 24.5 % assess it as high. 22.6% assess their colleagues knowledge as average, but only 1.9% respondents as rather low. The situation differs, if the data are analysed according to the level and type of education. The employees of the institutions of vocational education have the most critical opinion of themselves and their colleagues, but the employees of the institutions of general education have the highest assessment of their colleagues, 90% of the answers are high and rather high. The representatives of the institutions of higher education note in the comments that the main problem is promoting the understanding of the concept itself, especially among the professionals of the field, who are involved in developing and ensuring the study process. Learning outcomes have been defined for many courses; however, not all faculty members are familiar with the specific terminology of the learning outcomes approach. Respondents at the institutions of vocational education note that, although the understanding of the term learning outcomes is not always high, since the studies are very practical, every member of the faculty has a rather clear understanding about what kind of knowledge, skills and competences he or she should impart to the educatees, so that they would be able to pass the qualifications exam. Whereas the employees of the institutions of general education note that the faculty members have very good knowledge of the standard; however, this does not mean that the concept of learning outcomes is also understood. Schools experience particular difficulties in understanding and introducing interdisciplinary skills and competences. Obstacles to introducing learning outcomes During the interviews respondents were asked about obstacles in working with learning outcomes, both on the state and the institutional level. Six obstacles were proposed (lack of knowledge about learning outcomes among the employees of the institutions of education; lack of uniform understanding of learning outcomes; lack of information; unclear (fragmented) policy for introducing learning outcomes; lac of resources for dealing with this issue; unwillingness of the faculty members to accept change), it was possible to assess these on the scale from 1 to 5, in which 1 meant is not an obstacle, but 5 a significant obstacle, it was also possible to add other significant obstacles. 32

33 The trend shows that more criticism is addressed to obstacles for working with learning outcomes on the level of the state compared to the level of an institution. The data show that the respondents, in assessing the work of their own institution of education, assess the obstacles as insignificant more than 50% of respondents choose the assessment 1 and 2, which points to the fact that the employees of the institution of education blame the state for deficiencies in implementing learning outcomes. The following should be mentioned as the major obstacles on the What are the main obstacles in working with learning outcomes in Latvian education in general? (Please assess on the scale from 1 to 5, 1 denoting is not an obstacle, but 5 a significant obstacle. Lack of knowledge among the staff Lack of uniform understanding Lack of information Fragmented policy of implementation Lack of resources Faculty members unwillingness to accept changes Hard to say state level: unclear (fragmented) policy for implementing learning outcomes, lack of uniform understanding of learning outcomes, as well as lack of resources, which received the assessment 4 and 5, respectively, in 54.7%, 45.3% and 45.3% of cases. Whereas the following have been assessed as the least significant of the obstacles offered lack of knowledge about learning outcomes among the employees of institutions of education, as well as lack of information, which have received assessment 4 and 5 in, respectively, 18.9% and 20.7% of cases. 33

34 On the level of institutions lack of knowledge about learning outcomes among the employees of the institutions of education, as well as lack of information, have also been recognised as being the least significant obstacles, receiving the assessment 4 and 5 in, respectively, 3.8% and 3.8% cases. Whereas lack of resources, unwillingness of the faculty members to accept changes, as well as lack of uniform understanding of learning outcomes have been recognised as the most significant What are the main obstacles to working with learning outcomes at your institution? Please assess on the scale from 1 to 5, 1 denoting is not an obstacle, but 5 significant obstacle. Lack of knowledge among the staff Lack of uniform understanding Lack of information Fragmented policy of implementation Lack of resources Faculty members unwillingness to accept changes Hard to say deficiencies, which were assessed with 4 and 5 in, respectively, 26.4%, 13.2% and 11.3% of the cases, which, compared to the assessment of these obstacles on the state level, is not much. Obstacles to working with learning outcomes on the state level Since a number of comments were given with regard to all obstacles both on the level of the state and the institution, a summary of these will be provided below, examining every proposed obstacle separately. This will give better understanding of the reasons, why respondents have recognised a proposed obstacle to working with learning outcomes as significant or, on the contrary, as insignificant. Lack of knowledge about learning outcomes among the employees of the institutions of education on the level of the state Even though this was assessed as one of the insignificant obstacles, the comments show that the situation is not that unambiguous. Employees of institutions of higher education, especially in professional programmes, note that professionals of the field and also young faculty members are not always able to understand the concept of learning outcomes in full. At the same time it is stated that these professionals of the field are able to provide intuitively to their students the necessary knowledge, skills and competences, without always understanding how these fit into the common concept of learning outcomes. Whereas the representatives of institutions of general education highlight the fact that faculty members have good knowledge of knowledge, but not as good of skills and competences. 34

35 Lack of uniform understanding of learning outcomes on the level of the state During the interviews this was seen as one of the main obstacles to working with learning outcomes on the state level, which means that explanatory work should be continued. The institutions of higher education point to the lack of common understanding and vision on the application of this concept. Certain difficulties in differentiating skills and competences exist. The employees of institutions of vocational education note that the general education and vocational education should be differentiated more, as rather different understandings of learning outcomes exist, but the pupils have to take uniform centralised examinations. The employees of the institutions of general education also note the various understanding of learning outcomes among them; moreover, request reviewing of standards, since, for example, the transition from primary school to basic school has not been sufficiently aligned. Lack of information on the level of the state During the interviews this obstacle was assessed as one of the least significant. Even though the respondents note in their comments that information is not always offered proactively, yet admit that, if there is a wish to find it, there are not too many obstacles to this. Such comments are especially made by the employees of institutions of higher education. Whereas the representatives of institutions of general education note that there is little information abut skills and competences, especially as regards methodological material, since the existing listing of knowledge, skills and competences included in the Cabinet of Ministers Regulation is too general and insufficient to serve the teachers needs. Therefore schools have to elaborate their own methodological materials, this may lead to an erroneous interpretation of a regulatory enactment. Unclear (fragmented) policy for implementing learning outcomes in the state During the interviews this was emphasized as one of the main obstacles in working with learning outcomes. The employees of institutions of higher education note that too little explanations are provided on the meaning and application of learning outcomes on the level of the state, as well as note that the legal basis has been aligned, but there are problems in implementation. Employees of the institutions of vocational and general education in their comments draw attention to the fact that the curriculum of the natural sciences subjects is not always synergetic, moreover, sometimes the standard is changed, but the adjusted textbooks appear only afterwards. Employees of schools note the lack of diagnostic work regarding skills, therefore the general level in the state is not known (except in financial literacy), which causes the implementation of erroneous policy regarding learning outcomes in the state. Lack of resources in order to focus upon this issue on the state level Lack of resources is also noted as one of the main obstacles in working with learning outcomes in the state level. It is worth noting that it is not the lack of financial resources, but the lack of human resources, which is emphasized in the comments. The employees of the institutions of higher 35

36 education note that the resources exist, but the ability to find and use them is needed, whereas the employees of the institutions of vocational and general education draw attention to such issues as motivation of the faculty members, knowledge and workload, as well as the capacity of NCE. Unwillingness among the faculty members to accept changes When touching upon the faculty members resistance to change as an obstacle in working with learning outcomes, the employees of institutions of higher education note that this practice is to a lesser extent observed in private institutions of higher education, because the faculty members are replaced more often, on the basis of students assessment and according to other criteria. Likewise, there are more professionals of the field in private institutions of higher education, who become involved in the implementation of study programmes and who, perhaps, do not have as good understanding of the learning outcomes approach, but the content of studies that they offer is oriented towards application. In some of the interviews it was noted that as regards accepting changes, the generation of faculty members does matter; however, the employees of all level and type of educational institutions emphasize that this issue requires examination on case by case basis, therefore it would be difficult to foreground a special group having particular problems with accepting change. Other significant obstacles to working with learning outcomes When asked about other obstacles, employees of institutions of higher education note such factors as natural wish to resist bureaucratic activities, as well as the limited possibility for policy implementers to take a distanced look at the processes of implementation. Whereas the representatives of the institutions of vocational education point out the low level of knowledge about the concept of learning outcomes among employers, which hinders exchange of opinions and development of quality standards of occupations. Employees of institutions of both vocational and general education note lack of motivation among youth, which does not permit the faculty members to implement their ideas. Schools also draw attention to the fact that as the result of competition between schools and the regional reform, the methodological associations have become less active. The fact that there is little interest in society regarding implementation of the learning outcomes concept is also noted. Mass media and non-governmental organisations show little interest in issues of education quality, especially in general education, thus understanding of these issues is not developing in society. Obstacles to working with learning outcomes on the level of institutions Lack of knowledge about learning outcomes among the staff of the institution of education As mentioned above, only 3.8% of the respondents assess this obstacle with 4 or 5. It must be noted that a comparatively smaller number of respondents see the lack of knowledge among their colleagues as an obstacle, compared to the same assessment with regard to the whole sector of education. In the comments, similarly to the discussions of the state level, it is noted that the faculty 36

37 members have an intuitive understanding of the learning outcomes concept, however, if they were asked to provide the definition of the learning outcomes approach and to explain various terms, then they would not be able to provide the correct answer always. Lack of uniform understanding of learning outcomes among colleagues Even though this obstacle was frequently noted as problematic (grade 4 and 5 in 11.3% of cases), it is also noted that with regard to some standards such interpretation is possible, which allows adjusting the programmes to the possibilities at the disposal of the institution of education, at the same time retaining the learning outcomes defined in the standard. Lack of information Also this obstacle is noted as significant only occasionally (grade 4 and 5 only 3.8% of cases). Employees of higher education institutions see as a solution seminars for employees with the possibility of working jointly on issues of methods and didactics, but they also draw attention to the fact that such additional activities might scare away professionals, who are preoccupied with their everyday work. Unclear (fragmented) policy for introducing learning outcomes in your institution of education Even though some institutions of education admit having problems in the process of policy implementation, the majority, nevertheless, emphasize that as much information as can be obtained from state institutions is forwarded to the employees, using various mechanisms. Institutions of higher education are elaborating clear course descriptions, whereas the institutions of vocational and general education deal with these issues during meetings of the teachers council, placing special emphasis upon the implementation of the standard. Lack of resources in your institution for dealing with this issue Similarly to the assessment of lack of resources on the state level, also on the level of institutions the lack of human resources is mentioned. Lack of time (work-load) and lack of knowledge are mentioned, as well as lack of resources fore exhaustive analysis of the situation in the particular institution of education. The fact that the number of educatees on all levels and in all types of education is decreasing was also noted. Unwillingness among the faculty members to accept changes Even though 13.2% of the respondents assess this obstacle with grades 4 and 5, they give very brief comments, mentioning the arguments that additional work and the change of previous approach face certain resistance. However, a number of the surveyed respondents, especially employees of the institutions of higher education note that, although initially there had been certain problems, following repeated explanations the faculty members had understood and accepted the changes. 37

38 Other significant obstacles in working with learning outcomes When considering additional factors, which hinder proper work with learning outcomes, employees of institutions of education note a number of them, which relate to the hindering factors on the state level. The level of previous training and motivation of the educatees is mentioned. Schools in the regions in particular mention the issue of human resources, and also note that it is more difficult to work in combined classes. Since reforms are constantly discussed, faculty members are anxious about their jobs, which creates additional stress in everyday life. It is also noted that some faculty members do not want to know what their colleagues are doing and how to create synergy in the study process. The existing system of evaluation is also discussed, in it the grades are more important than the achieved outcomes, especially on the state level (in the context of centralised examinations) the grade is valued, but not the progress achieved in working with a particular child. And, finally, the respondents are not convinced that the proposed changes will facilitate higher level of knowledge among pupils. It can be concluded that the respondents, in assessing the obstacles to introducing the learning outcomes approach, predominantly see them on the national level, much fewer on the local level. Uniform understanding of the need for learning outcomes approach, its meaning and significance is lacking. Likewise, there is no shared vision on the development of the system of education and the contribution of each stage of education to it, especially among institutions of education of various types and levels. The following are highlighted as the main problems fragmented policy for implementing learning outcomes, as well as lack of resources, especially, as emphasised in the comments, lack of human resources. Lack of knowledge among staff and lack of information are highlighted as less significant obstacles. Use of learning outcomes within the system of education in general Elaboration of study programmes, based on clearly defined and valid learning outcomes The majority of respondents (34.0%) hold that elaboration of study programmes, which are based on clearly defined and valid learning outcomes, can be rather considered to be a common practice in Latvia. This is the most popular choice, however, in general there is no consensus among respondents. The trend can be assessed as positive, since more than a half of respondents (52.9 %) hold that it is or rather is a common practice, but only 18.9 % of the respondents hold that it not or rather is not a common practice. 38

39 It is noted that that programmes of education do not mention learning outcomes and the achievements of particular individuals, only the study subjects are discussed, and the programmes, basically, are elaborated by focusing upon the content, not the estimated outcome. The institutions of education of all surveyed levels emphasized a significant deficiency of the existing system, i.e., that the study programmes are not made in a way that would ensure mastering of interdisciplinary skills and that the possibilities to modify the programme within the framework of the curriculum are very restricted. Is the elaboration of study programmes, which are based on clearly defined and valid learning outcomes, a common practice? No Rather no Average Rather yes Yes Hard to say No answer The representatives of the institutions of vocational education note that the standards of occupation have a positive impact upon the elaboration of such study programmes, however, there is also the view that the standards of occupations promote study process, which is isolated from the general system, which causes difficulties in combining professional subjects and subjects of general education, at the same time defining skills, which are not used in practice within the particular sector. The respondents representing the institutions of general education quite frequently stated that the study programmes of basic and secondary school are oriented towards the admission requirements of institutions of higher education, thus, the amount of material that must be mastered in each separate subject is too big and the common curriculum of the programme is connected neither with the content of the possible study programme, nor the content of vocational secondary education. Best practice examples in linking the learning process and content with learning outcomes (3.2) In discussions about best practice examples, which particularly successfully link the study process and content with learning outcomes, ESF projects for improving natural sciences subjects, cooperation with employers in developing and improving programmes are the most frequently mentioned examples. Science subjects in general was the most popular answer among the institutions of higher education, in particular, vocational study programmes (regulated study programmes), which have 39

40 clearly defined and exact standards of occupations. The College of Culture mentioned one particularly successful example 1 programme offering 12 specializations, which means specially adjusted modules for reaching specific learning outcomes. Institutions of higher education and vocational education note as best practice examples applied character of programmes, development of successful cooperation with employers and consultations with employers (particularly, Latvian Employers Confederation) in the process of developing new programmes and updating programmes, as well as the involvement of the employees of educational institutions in elaboration of standards of occupations. The programs of continuous education, developed by the State Employment Agency and which already have known employers, are also mentioned as a best practice example. Institutions of vocational education and higher education mention foreign experience as significant source of information and Latvian experts, who ensure accessibility to foreign experience and are the first ones to implement it in practice; Sandra Kraže, expert in the implementation of the Bologna process in Latvia, is mentioned in particular. A number of respondents point out the value of ESF projects (for example, ESF project Improving the knowledge and professional competence of teachers of vocational subjects and internship supervisors ); however, the representatives of institutions of vocational education note that, even if the funding exists, the real possibilities to create something new are limited, since the existing work-load is an obstacle to it. It is also noted that the programmes are not flexible and that the procedure for updating them is long and complicated. Likewise, the representatives of institutions of vocational education note that in the majority of cases best practice examples are borrowed from abroad and that it is easier to mention foreign examples, since Latvian institutions of education seldom share their best practices. The respondents on the level of institutions of general secondary education mention as the best example the interactive programs of natural sciences, for which the study aids were created after the programme was elaborated (mathematics programme for grade 7 9, etc.) Other subjects are less frequently mentioned as best practice examples, for example, the Latvian language, in which the study programme in secondary school links grammar with creative tasks. A very individualised method has been implemented in a number of institutions, where each student in each subject has a note-book, created by the teacher, and study materials are placed in it, at the beginning of each topic a note listing the knowledge, skills and competences that the student will acquire, is pasted; also information on the structure of tests and their linkage with knowledge and skills, an extensive interpretation of results is pasted following the test. Some institutions have developed also a plan for introducing inter-disciplinary skills. A special group of programmes offered by some institutions of secondary education are the programmes made especially and adjusted for children with special needs. Some institutions have implemented a system, which embraces children with learning disorders, by a referral they can be 40

41 sent to a special class and, according to exactly defined learning outcomes, the child can return to the general class only when the respective level has been reached and the teacher has ascertained that the child will be able to keep pace with others in the class. The importance of using learning outcomes in various fields: curriculum development 45.3% of the respondents hold that learning outcomes are very significant in creating curriculum, 71.7% of the respondents hold that they are of great or rather great importance. What is the importance of learning outcomes in the curricula development? Little Rather little Average Rather great Great Hard to say No answer Notwithstanding the opinion of the majority, a couple of times a totally opposite opinion was expressed, i.e., that the curriculum defines learning outcomes, not learning outcomes - the curriculum, and since the curriculum is overcrowded, in the successive levels of education things that have been taught before must be retaught, and the use of learning outcomes is only fictitious and not relevant. evaluating educatees 41.5 % hold that the use of learning outcomes in evaluating educatees is of rather great importance; slightly less respondents (32.1%) hold that it is of average importance. How great is the importance of learning outcomes in evaluating educatees? Little Rather little Average Rather great Great Hard to say No answer It is noted that learning outcomes are used most effectively in qualifications exams and that it is also the easiest to verify them on this occasion. A number of respondents note that learning 41

42 outcomes are not properly applied; since it is impossible to evaluate within the existing system of evaluation the individual growth, nor the skills or competences. This is most acutely felt by schools attended by children with special needs, as well as evening schools, where often every student has his or her individual programme. In situations like these it is impossible to evaluate pupils properly. One of the offered solutions is to define also the sufficient skills, not only the maximum level, so that children with special learning needs could receive higher evaluation. quality assessment process No consensus is observed as regards the importance of learning outcomes in the quality assessment process % hold that it is of rather great importance, 26.4 % hold that it is of great importance and 18.9 % hold that it is of average importance. The employees of the surveyed institutions note that much more still needs to be done in this respect and draw attention to the fact that learning outcomes used to be more significant in the quality assessment process, whereas currently it is only formal. Respondents note that the outcomes have no significance in the assessment process and they are not verified in practice, which points to deficiencies in the system of assessment, since the accreditation of institutions is only formal. How important are learning outcomes in the quality assessment process? Little Rather little Average Rather great Great Hard to say No answer It is also noted that the external quality assessment process is oriented towards numerical indicators (school ratings, reports on grades, which are included in the self-assessment report), and no attention is paid to individual learning outcomes. Recognition of outcomes attained in prior learning or professional experience An equal number of respondents (30.2%) consider that the recognition of outcomes attained in prior learning or professional experience is of great or rather great importance. Only 5.7% consider that learning outcomes are of little importance. 42

43 Institutions of higher education admit that there are both good and bad examples; however, the importance of learning outcomes grows in recognising outcomes attained in prior learning or professional experience. It is noted that recognition of formal education develops much faster than How great is the importance of recognising the outcomes attained in prior learning or professional experience? Little Rather little Average Rather great Great Hard to say No answer recognition of non-formal education. The financial aspect of recognising prior outcomes (setting the fee, granting tuition fee discounts) creates problems for institutions of higher education. Institutions of vocational education note that it is very important, in connection with recognition, to ensure the possibility to resume studies to those educatees, who had dropped out previously. Institutions of vocational education have comparatively numerous examples of successfully conducted recognition. A trend can be observed among institutions of general education that the representatives of these institutions do not always trust the evaluation granted by the previous institution of education, in particular, this is typical of gymnasiums. Representatives of an institution characterised the existing situation by noting that pupils are evaluated upon moving from pre-school to the first grade, they have to take entrance exams to be admitted to a state gymnasium, upon returning from abroad they have to take tests to be allocated to a grade. Representatives of many institutions note that the transition between different levels of education is not effective; in reality each successive level is not based upon the previous one, the standard defines the succession, but due to lack of individual approach it is impossible to ensure it in reality. It can be concluded that representatives of institutions of education hold that the study subjects and programmes already now are based on learning outcomes, however, asked about some particularly good examples, the majority of respondents name only the programmes in science subjects for secondary schools, elaborated with the ESF support, thus revealing insufficient awareness of best practice examples in other institutions. The use of learning outcomes is of special importance in curricula development and recognition of outcomes attained in prior learning or professional experience, and only rather great importance in evaluating educatees and in the quality assessment procedures. 43

44 The importance of learning outcomes in curricula development/ implementation of programmes and study subjects Developing study subjects and programmes subordinated to learning outcomes 45.3% of the representatives of surveyed institutions hold that the development of study subjects and programmes at their institution to a large extent is subordinated to learning outcomes and to reaching them. Comparatively significantly less respondents hold that the development of subjects or programmes is subordinated to learning outcomes and to reaching them to a rather large extent (24.5%) or to an average (20.8%) extent. To what extent the development of study subjects and programmes is subordinated to learning outcomes. Small Rather small Average Rather large Large Hard to say No answer The representatives of those institutions that offer to educatees programmes of vocational education of various levels note that since the programmes are vocational it is easy to introduce learning outcomes, concrete skills are defined in the standards of occupations and learning outcomes are defined by using the standard of occupation and employers recommendations. The representatives of institutions of general education note that this linkage depends upon each study subject, since for some subjects it is easier to define learning outcomes (for example, mathematics) and for some subjects it is more difficult (for example, visual art). The majority of Latvian institutions of general secondary education do not elaborate new programmes and subjects, since they work according to the standard. If a new programme or a subject is developed, then they try to make the content interesting both from the perspective of the teacher and the pupils. 44

45 The importance of factors in developing new study subject/ programme experience of foreign institutions of general education Views on the significance of the experience of foreign institutions of general education are very diverse. The opinion that foreign experience is of rather little importance received the highest evaluation (28.8%). The options that the foreign experience is of rather great importance (28.8%), of great importance or of rather little importance (each 18.9%) have received similar support. How important is the experience of other countries in the elaboration of a new subject/ programme? Little Rather little Average Rather great Great Hard to say No answer This lack of consensus can be explained by the fact that institutions of different levels have different traditions of cooperation with institutions in other countries. Institutions of higher education and institutions of first level vocational education have more cooperation and transpose more experience than institutions of secondary education. Institutions of vocational education have concrete twinning schools, exchange of pupils and faculty members takes place. Germany s dual system is one of the general examples, a specific example would be the experience transposed from Dresden as part of ERAF project a mini workshop was equipped at Olaine Technical College for conducting study experiments and test in practice the competences needed for producing medications. experience of Latvian institutions of education The majority of representatives from the surveyed institutions note that the experience of Latvian institutions of education is of rather great importance (39,6%). Similar support has been granted to the opinions that the experience of Latvian institutions of education is of average or great importance (22.6% each). There are leading institutions in each vocational sector, whose experience is transposed. Institutions of first level vocational education have good cooperation with centres of competence, they also share among themselves the elaborated study programmes. Cooperation among Latvian institutions of education is more typical of institutions of vocational education; institutions of general education note that the competitions among schools is too tough and that schools with 45

46 specific target audience, for example, evening schools, have limited possibilities of cooperation, because it is difficult to involve educatees in the process. How important is the experience of Latvian institutions in elaborating a new study subject/ programme? Little Rather little Average Rather great Great Hard to say No answer faculty members experience (3.5.3) The majority of respondents (50.9%) note that the faculty members experience is of great importance in the elaboration of new study subjects and programmes, the opinion that the faculty members experience is of rather great importance has received similar support (43.4%). It is typical of institutions of higher education to take into consideration the experience of the sector s professionals. In the majority of cases the faculty members are directly responsible both for the content of the course and the teaching, thus the content is elaborated on the basis of their own experience and opinions, seeking the advice of the professionals in the sector. Many institutions of higher education invite the professionals from the sector to elaborate and to teach study courses. How important is the experience of Latvian institutions in elaborating a new study subject/ programme? Little Rather little Average Rather great Great Hard to say No answer 46

47 national qualifications infrastructure The opinions on the significance of the national qualifications framework in the elaboration of new study subjects/ programmes differ, and when answering to this question a comparatively large number of respondents note (22.6%) that it is difficult to answer this question. The majority of those, who did respond, (30.2%) holds that the national qualifications framework is of great importance. What is the significance of the national qualifications infrastructure in the elaboration of a new study subject/ programme? learning outcomes Exactly a half of respondents notes that learning outcomes are significant in the elaboration of a new study subject/ programme. How important are learning outcomes in the elaboration of a new study subject/ programme? 47

48 standard of education /subject standards The absolute majority of respondents (97.7%) notes that the standard of education and subject standards have rather great or great significance in the elaboration of a new study subject or programme. 60.5% of these respondents note that the standard of education and subject standards are of great significance. How important is the standard of education/ subject standards in the elaboration of a new study subject/ programme? standards of occupation Institutions of vocational education acknowledge that standards of occupations are very important and note that the programmes of vocational education would be unable to exist without them. Numerically, 73.3% respondents note that standards of occupations are of great importance, and 20.0% of respondents note that standards of occupations are of rather great importance, thus, in total 93.3% respondents hold that the standards of occupations are of great or rather great importance in the elaboration of new study subjects/ programmes. How important are the standards of occupation in the elaboration of a new study subject/ programme? other factors Representatives of the surveyed institutions of education mention also a number of other factors, which are of great importance in the elaboration of new study subjects or programmes. A couple of 48

49 factors are typical of all levels of education, however, some are typical of the institutions of education belonging to a particular level. One of the most important factors mentioned both with regard to higher and vocational education are employers recommendations and the general labour market trends (sectors that experience lack of specialists, technological development, skills that are in demand). The representatives of the institutions of higher education note the importance of the councils of study programmes, which regularly examine issues linked with the content and implementation of a study programme and adopt decisions on introducing amendments to the content of study programmes or the process of implementation The internal need of administration and the faculty to update the programme/ improve the subjects is another factor mentioned by the representatives of a number of educational institutions. Institutions of secondary education mention various resources as a very important factor. It is noted that the accessibility of information, textbooks and various study aids is important (often the study aids that are available are not appropriate for the programme, and it is not worth teaching a new programme, using outdated text-books), as well as resources that are available for the faculty members salaries. One of the most important resources, which are often mentioned, is the internal environment and the administration s attitude and support. Institutions of secondary education also note that the current system gives very restricted possibilities of sharing best practice examples and get familiarised with them, the system, when the institutions of education were allocated to districts, functioned much better. Another factor, which is frequently mentioned by the representatives of institutions of education, is the fact that in those subjects that do not have centralised examinations, updating and amending is easier compared to subjects that have centralised examinations. Representatives of schools note that the existence of centralised examinations hinder creative approach to elaborating the content of the subject and to teaching it. A general conclusion can be made that the faculty members experience and expected learning outcomes, as well as the standard of education and standards of occupations are of greatest importance in the elaboration of new study subjects and programmes. The experience of Latvian and foreign institutions of education is not evaluated unambiguously. Respondents note that also national qualification frameworks are of great importance; however the representatives of institutions of secondary education note that they do not see direct influence of qualifications infrastructure upon the content of the programme and process of its elaboration. 49

50 Learning outcomes based implementation of study subjects and programmes The absolute majority (84.9%) holds that the implementation of study subjects and programmes is based or is rather based upon learning outcomes. To what extent the implementation of study subjects/ programmes is learning outcomes based? The faculty members note that the books and methodological materials elaborated in recent years are structured directly upon learning outcomes, the descriptions of study subjects/ courses are updated and improved. Representatives of those institutions, which implement vocational education programmes of various levels, note that the practical training is more learning outcomes based. The majority of representatives from institutions of secondary education note that, obviously, not all educatees are able to apply knowledge acquired during lessons for solving various tasks and to search information independently and adapt it to their needs. The interviews allow concluding that one of the reasons why study subjects and programmes are insufficiently learning outcomes based is the fact that young people not always believe that education is the foundation for their success, and it is difficult to work with young people like them and to think about learning outcomes; therefore the study work is analysed in order to define new goals, in particular for pupils with special needs. The representatives of those institutions that in their work encounter educatees with special learning needs, note that the work is targeting individual aims, the ability of a particular child to do certain things is rather considered an outcome, not what the standard prescribes. 50

51 The harmonisation of study methods and content within the framework of study programmes The respondents opinion on this issue differs. 67.9% of the respondents note that study To what extent the study methods and content are harmonised in the framework of study programmes? Not Rather not Average Rather is Is Hard to say No answer methods and content are harmonised or rather are harmonised, however, a comparatively large share of respondents (26.4%) note that the harmonisation of study methods and content is average. Various methods are used to harmonise study methods and content within the framework of programmes, after the results of various surveys (faculty members, educatees, employers) are obtained, discussions among colleagues are held. It is important to avoid both horizontal and vertical overlapping. The vertical overlapping is dealt with by succession in studies, the horizontal overlapping is dealt with by faculty members meeting. Many institutions hold that harmonisation should happen more consistently, actively and directly, they emphasize that the harmonisation of study methods and content should be included as an element in the quality management system. Predominantly the most popular method for preventing overlapping is faculty members meetings of various levels (programme, module, department, institution). This method is popular among institutions of all types. On the level of secondary education overlapping is dealt with in methodological meetings; frequently the teachers of one subject or teachers of related subjects jointly elaborate the content and methods, as well as the totality of materials to be used in classes. The opinion on interdisciplinary skills differs, some schools coordinate it, other do not; it deserves more attention on the state level than it has received thus far. Harmonisation often depends upon the stance adopted by administrative staff and management, thus it is an important factor; in some schools management does not pay proper attention to it, thus no harmonisation processes take place. 51

52 Pupils knowledge and understanding of learning outcomes More than a half of respondents (53.8%) hold that pupils have average knowledge and understanding of learning outcomes % of the respondents evaluate pupils knowledge and understanding as complete. First of all, in the majority of institutions the term learning outcomes are not used vis-à-vis educatees. It is noted that in the majority of cases educatees have short-term thinking (do not see the whole picture, do not understand the significance of separate components) and perceive grades as being more important than the acquired skills. Due to the economic situation in the state a large number of educatees lack motivation for acquiring higher-level education, therefore they also have no motivation to understand learning outcomes. In particular, lack of motivation is typical of graduates from institutions of basic and secondary education. At the institutions of higher education the expected/ desirable learning outcomes are often highlighted during lectures, as knowledge, skills and competences, however, the students not always understand them or understand them only after completion of the course. To deal with this situation, it is worth introducing a practice, in which the students evaluate their learning outcomes from another vantage point, for example, by applying their knowledge in practice, performing real tasks from work environment as part of the study process It is typical of programmes of vocational education that the educatees become aware of the learning outcomes only during the practical training, linking their knowledge with the actual work environment, therefore the fact that the practical training for qualification, following which the educatees might develop better understanding, takes place only during the last year of studies, significantly worsens the situation. What is the educatees knowledge and understanding of learning outcomes? Incomplete Rather incomplete Average Rather complete Complete Hard to say No answer In the majority of cases the understanding of learning outcomes among the students of institutions of secondary education depends upon the subject teachers, since this issue is not focused upon centrally, on the scope of the whole school. The best practice examples show that teachers jointly with pupils analyse tests and, depending upon previously acquired skills, define the content of the test individually for each educatee. 52

53 Accessibility of learning outcomes For institutions of higher education learning outcomes are accessible in ECTS catalogues, course and programme descriptions, in e-study environment, learning outcomes are included in selfassessment reports, learning outcomes are presented to students during the introductory lectures of courses. Institutions of vocational education note that it is possible to familiarise oneself with learning outcomes in the programme descriptions, qualifications descriptions, electronic register; the majority of institutions of education understand learning outcomes ad programme descriptions and standards, and publish these on their home pages, whoever, these are nowhere called learning outcomes. Educatees are informed during the subjects Introduction to the field of study, Introduction the occupation, printed information is freely available at the institution of education from subject teachers and administration (study/ learning department and/ or heads of study/ learning programme/ /director of studies). In secondary education learning outcomes are rather understood as the numerical achievements of students, and it is noted that these are published in e-register and hat study plans and descriptions of evaluation are available; in many institutions, but not in all, also descriptions of skills are available in the electronic system alongside grades. Learning outcomes are discussed at meetings on methodologies, parents meetings, upon parents request they can familiarise themselves with them individually. Those institutions of education, which understand learning outcomes literary as skills, competences and knowledge, note that such data as a whole are not available from the institutions of secondary education, since the standard sets out that only grades should be granted, learning achievements differ from learning outcomes, and representatives of institutions note that learning outcomes will have no significance until the state defines the level, on which a particular skill should be mastered for obtaining a certain grade. At the interviews the representatives of institution clearly expressed the view that more attention should be paid to connections with the students actual skills, however, the standards do not offer this possibility. The use of learning outcomes in the elaboration of educational curriculum on the national level The role of the standard of education in defining learning outcomes (CC) This question was put to the representatives of institutions of secondary education (also vocational secondary schools). 41.0% of the respondents hold that the standard of education to a large extent makes defining learning outcomes easier. To a similar extent the importance of the standard of education is assessed as rather great (28.2%) and average (25.6%). In general it can be observed that the representatives of administration use the standard of education more than faculty 53

54 members, and the representatives of administration have more positive assessment of the significance of the standard. Representatives of institutions of education note that the standard is rather static, general and it can be successfully used as the basis, which gives indications and outlines directions for work, however, methodological material, providing additional value, is not available. Representatives of institutions of vocational secondary education admit that the standard of education enumerates the skills necessary for obtaining education, however the representatives of institutions of vocational secondary education not always have the time to work with them, since the framework of vocational education is more important for acquiring vocational education. Little Rather little Average Rather great Great Hard to say No answer To improve the quality of vocational education, it would be important not to have united centralised examinations for institutions of general secondary education and institutions of vocational secondary education, since there are valid differences in the examination results. The representatives from institutions of secondary vocational education note that it is very important for the faculty members to be flexible and able to interpret the standard in order to achieve the best results. Representatives from the institutions of secondary vocational education note that the standard of education insufficiently considers interdisciplinary skills and link between subjects. In professional higher education programmes a very large number of credit points is allocated for the block of subjects of general education, subjects, which are not necessary for acquiring occupation are studied, thus little time remains for contact hours in the specialism. The importance of subject standard in defining learning outcomes (CE) for secondary school level (including vocational) This question was put to representatives of institutions of secondary education (also vocational). 44.7% of the respondents note that the subject standards to a large extent alleviate defining of learning outcomes. However, more than one third of respondents (34.2%) notes that subject standards alleviate defining of learning outcomes only to a rather large extent. The institutions of vocational education note that this standard cannot be applied 1:1, since the number of classes envisaged for mastering subjects of general education is smaller and that there 54

55 are no standards for subjects of vocational education. The institutions also note that a subject standard for vocational subjects would be necessary, it would significantly alleviate work. Representatives from institutions of secondary education note that the subject standard partially alleviates the process, but partially encumbers it, since the standard is very sizeable and in some subjects the transition from one level to the next has not been harmonised; for example, the transition in the standard of the Latvian language from basic school to secondary school for minority schools. The subject standards have the main aim of preparing pupils for examinations, but the subject standards offer to teachers the totality of topics to be mastered, to be included in the programme, thus ensuring that such topics are studied at school, which otherwise, were they not included in the standard, would not be mastered. The role of the standard of education for first level higher professional education in defining learning outcomes This question was put to the representatives of those institutions, which implement first level professional study programs, and 75% of the respondents noted that the standards of occupations were of great or rather great importance. The standard of professions provides a framework for expression, however, this framework is rather specific and does not allow any deviations, therefore 25% of respondents indicate that the standard of first level higher professional education is of little importance in defining learning outcomes. 55

56 Absence of standard for academic education This question was put to representatives from institutions of higher education. The representatives were asked to what extent the lack of standard in academic education complicated defining of learning outcomes. The majority of representatives from institutions of higher education noted that was possible to do without this standard quite successfully and assessed its importance as average. None of the representatives of higher education noted that the absence of standard for academic education to a large or rather large extent complicated defining of learning outcomes. Representatives from institutions noted that the standard would be necessary as a framework for working and underlined that it was easier to define the expected outcomes if one had a standardised sample, since faculty members lacked resources for defining everything themselves. The second predominant opinion was that the possible standard would be so general, that it would be totally possible to do without it. Standard of occupations This question was put to representatives from institutions of higher education, institutions of secondary vocational education and institutions of first level higher professional education. Half of 56