Annex 1.6 Indicators and Data for VET

Similar documents
PROGRESS TOWARDS THE LISBON OBJECTIVES IN EDUCATION AND TRAINING

The European Higher Education Area in 2012:

Summary and policy recommendations

National Academies STEM Workforce Summit

Educational Indicators

Department of Education and Skills. Memorandum

INSTRUCTION MANUAL. Survey of Formal Education

SOCRATES PROGRAMME GUIDELINES FOR APPLICANTS

ESTONIA. spotlight on VET. Education and training in figures. spotlight on VET

The Survey of Adult Skills (PIAAC) provides a picture of adults proficiency in three key information-processing skills:

Twenty years of TIMSS in England. NFER Education Briefings. What is TIMSS?

DISCUSSION PAPER. In 2006 the population of Iceland was 308 thousand people and 62% live in the capital area.

Educational system gaps in Romania. Roberta Mihaela Stanef *, Alina Magdalena Manole

Overall student visa trends June 2017

The development of national qualifications frameworks in Europe

The development of ECVET in Europe

ANALYSIS: LABOUR MARKET SUCCESS OF VOCATIONAL AND HIGHER EDUCATION GRADUATES

UPPER SECONDARY CURRICULUM OPTIONS AND LABOR MARKET PERFORMANCE: EVIDENCE FROM A GRADUATES SURVEY IN GREECE

BASIC EDUCATION IN GHANA IN THE POST-REFORM PERIOD

Students with Disabilities, Learning Difficulties and Disadvantages STATISTICS AND INDICATORS

EXECUTIVE SUMMARY. TIMSS 1999 International Science Report

Impact of Educational Reforms to International Cooperation CASE: Finland

UNIVERSITY AUTONOMY IN EUROPE II

Introduction Research Teaching Cooperation Faculties. University of Oulu

EQE Candidate Support Project (CSP) Frequently Asked Questions - National Offices

Challenges for Higher Education in Europe: Socio-economic and Political Transformations

National Pre Analysis Report. Republic of MACEDONIA. Goce Delcev University Stip

The recognition, evaluation and accreditation of European Postgraduate Programmes.

Referencing the Danish Qualifications Framework for Lifelong Learning to the European Qualifications Framework

Assessment and national report of Poland on the existing training provisions of professionals in the Healthcare Waste Management industry REPORT: III

Science and Technology Indicators. R&D statistics

OECD THEMATIC REVIEW OF TERTIARY EDUCATION GUIDELINES FOR COUNTRY PARTICIPATION IN THE REVIEW

COMMISSION OF THE EUROPEAN COMMUNITIES RECOMMENDATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL

Modern Trends in Higher Education Funding. Tilea Doina Maria a, Vasile Bleotu b

ehealth Governance Initiative: Joint Action JA-EHGov & Thematic Network SEHGovIA DELIVERABLE Version: 2.4 Date:

May To print or download your own copies of this document visit Name Date Eurovision Numeracy Assignment

TIMSS Highlights from the Primary Grades

Educating for innovationdriven

Australia s tertiary education sector

Study on the implementation and development of an ECVET system for apprenticeship

PIRLS. International Achievement in the Processes of Reading Comprehension Results from PIRLS 2001 in 35 Countries

Financiación de las instituciones europeas de educación superior. Funding of European higher education institutions. Resumen

School Inspection in Hesse/Germany

THE ECONOMIC IMPACT OF THE UNIVERSITY OF EXETER

DEVELOPMENT AID AT A GLANCE

CALL FOR PARTICIPANTS

Lifelong Learning Programme. Implementation of the European Agenda for Adult Learning

The EQF Referencing report of the Kosovo NQF for General Education, VET and Higher Education

Dual Training at a Glance

Tailoring i EW-MFA (Economy-Wide Material Flow Accounting/Analysis) information and indicators

The development of ECVET in Europe

NA/2006/17 Annexe-1 Lifelong Learning Programme for Community Action in the Field of Lifelong Learning (Lifelong Learning Programme LLP)

A comparative study on cost-sharing in higher education Using the case study approach to contribute to evidence-based policy

teaching issues 4 Fact sheet Generic skills Context The nature of generic skills

LAW ON HIGH SCHOOL. C o n t e n t s

SECTION 2 APPENDICES 2A, 2B & 2C. Bachelor of Dental Surgery

EUROPEAN UNIVERSITIES LOOKING FORWARD WITH CONFIDENCE PRAGUE DECLARATION 2009

Cross-case Analysis of Measures in Alternative Learning Pathways

VOCATIONAL EDUCATION AND TRAINING THROUGH ONE S LIFETIME

PROJECT PERIODIC REPORT

COMMISSION OF THE EUROPEAN COMMUNITIES. COMMISSION STAFF WORKING DOCUMENT Accompanying document to the

IAB INTERNATIONAL AUTHORISATION BOARD Doc. IAB-WGA

eportfolios in Education - Learning Tools or Means of Assessment?

HIGHLIGHTS OF FINDINGS FROM MAJOR INTERNATIONAL STUDY ON PEDAGOGY AND ICT USE IN SCHOOLS

Universities as Laboratories for Societal Multilingualism: Insights from Implementation

Summary Report. ECVET Agent Exploration Study. Prepared by Meath Partnership February 2015

Innovative e-learning approach in teaching based on case studies - INNOCASE project.

Norway. Overview of the Vocational Education and Training System. eknowvet Thematic Overviews

Annual Implementation Report 2010

Programme Specification. MSc in International Real Estate

Qualification Guidance

e) f) VET in Europe Country Report 2009 NORWAY e) f)

2001 MPhil in Information Science Teaching, from Department of Primary Education, University of Crete.

MODERNISATION OF HIGHER EDUCATION PROGRAMMES IN THE FRAMEWORK OF BOLOGNA: ECTS AND THE TUNING APPROACH

Self Awareness, evaluation and motivation system Enhancing learning and integration and contrast ELS and NEET

Teaching Practices and Social Capital

Department: Basic Education REPUBLIC OF SOUTH AFRICA MACRO INDICATOR TRENDS IN SCHOOLING: SUMMARY REPORT 2011

D.10.7 Dissemination Conference - Conference Minutes

Preprint.

International House VANCOUVER / WHISTLER WORK EXPERIENCE

AUTHORITATIVE SOURCES ADULT AND COMMUNITY LEARNING LEARNING PROGRAMMES

EXECUTIVE SUMMARY. TIMSS 1999 International Mathematics Report

international PROJECTS MOSCOW

The Isett Seta Career Guide 2010

NCEO Technical Report 27

03/07/15. Research-based welfare education. A policy brief

Introduction. Background. Social Work in Europe. Volume 5 Number 3

ROA Technical Report. Jaap Dronkers ROA-TR-2014/1. Research Centre for Education and the Labour Market ROA

The International Coach Federation (ICF) Global Consumer Awareness Study

PROJECT DESCRIPTION SLAM

I set out below my response to the Report s individual recommendations.

EMPIRICAL RESEARCH ON THE ACCOUNTING AND FINANCE STUDENTS OPINION ABOUT THE PERSPECTIVE OF THEIR PROFESSIONAL TRAINING AND CAREER PROSPECTS

BENCHMARK TREND COMPARISON REPORT:

Research Update. Educational Migration and Non-return in Northern Ireland May 2008

Sector Differences in Student Learning: Differences in Achievement Gains Across School Years and During the Summer

Conditions of study and examination regulations of the. European Master of Science in Midwifery

Rethinking Library and Information Studies in Spain: Crossing the boundaries

Modern apprenticeships: filling the skills gap?

Summary. Univers Emploi. Editorial : The Univers Emploi project. Newsletter n 2 February 2012

A European inventory on validation of non-formal and informal learning

Transcription:

1 Annex 1.6 Indicators and Data for VET Friederike Behringer & Harald Pfeifer, BIBB Introduction At the Lisbon European Council in spring 2000, the European Union set itself not only strategic goals, but also acknowledged the need for regular discussion and assessment of the progress made in achieving these goals on the basis of commonly agreed Structural Indicators. In order to allow for a monitoring of progress in achieving identified targets reliable data and adequate indicators are necessary. The need (and possibility) exists for improving information on VET (IVET and CVET), because of the lack of indicators/unbalanced coverage indicators are not well related to one another of the lack of easy access to existing data. When monitoring trends further problems are caused by methodological and technological changes in the surveys over time. In this annex we will not discuss in full detail all the indicators related to the topics of this report. Rather, we will focus on five issues: Expenditure on VET The role of VET in upper secondary education Indicators on Life-long learning Methodological changes of surveys and their consequences Authors gratefully acknowledge support generously provided by Eurostat.

2 Pitfalls in comparing results from different surveys. Expenditure on VET Information about financial input is a prerequisite to assess efficiency of VET or simply progress towards increase in investment in human resources in this area. There is very little information on expenditure for initial vocational education and training at European level, and what we have is not satisfying. Public expenditure on VET is not shown separately (as % of GDP and of total public spending). A presentation only by ISCED level (UOE) is not sufficient at European level, as long as information on programme orientation is missing. At least in some countries information is available from national sources, cf. ETF 2003, Table 4, p. 85). It is suggested to collect this information at European level. Private expenditure on IVET is included in private expenditure on educational institutions (UOE), this should be shown separately. Educational expenditure at the workplace is a difficult case (OECD 2004a, p. 65; cf. Green et al., 2000). Aiming at identifying expenditure on training per se the figures provided include the staff costs for all staff involved in training (full- and part-time trainers), training materials and equipment, training and teaching aids, examination fees, external courses and administration. Expenditure for salaries of the trainees and apprentices (wages, social security contributions, fringe benefits) is deducted, as this is not expenditure for education per se, but rather for cost of living of the trainees. Net training costs of companies are lower than gross expenditures since trainees are involved in productive activity. Accounting for the economic benefits of productive work reduces the gross costs substantially. Taking the German example gross costs of apprenticeship training (including wages for apprentices) are almost halved after deduction of returns (Beicht, Walden & Herget, 2004). On average, staff costs for apprentices exceed economic benefits of productive work during apprenticeship by some 7 %, with pronounced differences according to occupation, branch and size of company. Educational expenditure at the workplace as collected for UOE equals companies gross costs of apprenticeship training, deducting expenditure on staff costs of apprentices. However, state subsidies or tax incentives contingent on companies involvement in apprenticeship training are treated like the benefits through apprentices productive work: they are not deducted from expenditures. CVTS shows information on companies expenditure of CVET, but information on other forms of CVT (outside internal/external courses) needs more methodological development. So far coverage was restricted to companies with at least 10 employees, excluding micro enterprises, and excluding the public sector and some branches of private enterprises (e.g. agriculture, forestry and fishing, public administration, health and education, and households employing domestic staff), this should be extended in the next CVTS. 1 In the case of company provided VET there are expenditures directly related to the provision of VET, and fictive costs, labour costs of internal personnel temporarily acting as trainers etc. that are difficult to define and to record. For the next CVTS 1 Germany and Austria realised an estimation of the total costs of CVT for all enterprises and all economic sectors on the basis of CVTS 2. In Germany BIBB estimates a total of direct and indirect costs of continuing training which is 0.85 % of GDP. For Austria companies total costs for continuing training are estimated at 0.43 % of GDP (Markowitsch & Hefler 2003, p. 109). At European Level (EU15) it is estimated (cf. European Commission 2003, Annex 4) that the expenditure of companies covered by CVTS 2 represents about 0.6 % of GDP, and expenditure of all companies on CVT represents about 1 % of GDP, thus providing an approximation of the magnitude of companies expenditures not covered by CVTS so far.

3 more methodological work is needed, but has to take care of not jeopardizing comparability over time. As noted by OECD (OECD 2004a), companies expenditure for initial vocational training is included in private expenditure, but CVT in companies is excluded. Private investment in adult training thus is underestimated. Not much information on expenditure of participants and their families on IVET/CVET is available. Inclusion of these issues in the upcoming AES could be a way forward. VET in upper secondary education Indicators At national and international level there are several programmes assessing outcomes of educational systems (e.g. PISA, TIMSS 2 ). PISA is an internationally standardised assessment of 15-year old students. Most persons in this age group are still enrolled in initial education, which predominantly is general, not vocational education. Students in vocational programmes have been included in PISA, but for analysis of this subgroup it is likely that disproportionate sampling is a prerequisite. There are some initiatives at national level to assess the skills of the adult population 3, and at international level there are IALS and ALL. These assessments focus mostly on literacy and numeracy. Attempts to incorporate the assessment of teamwork, practical knowledge and other soft skills and competencies have not yet been successful. It is an exercise of assessing the stock of skills and competencies of the adult population, but so far the contribution of VET in relation to general education and work and life experience is not clear, and we have not been able to find comparative analysis taking into account the strand of education as one of the factors relating to literacy. There are no international studies allowing the assessment of the specific outcomes of VET programmes, as it is the case for the outcomes of educational systems (PISA, TIMMS). Assessment of the performance of the adult population (IALS, ALL) does not specify the contribution of VET in relation to general, at least not at international level. In the absence of performance indicators for those graduating from vocational programmes other indicators of outcomes will be used. Several goals and benchmarks for the Lisbon strategy are influenced by Life-long learning, education and training performance. Starting with youth education attainment, there is one structural indicator as well as the two benchmarks set by the European Council: By 2010, an EU average rate of no more than 10 % Early school leavers 4 should be achieved (benchmark) Structural indicator on youth education attainment 5 By 2010, at least 85 % of 22 year olds in the European Union should have completed upper secondary education. 2 Acronyms PISA (Programme for International Student Assessment), TIMSS (Third International Mathematics and Science Study), IALS (International Adult Literacy Survey), ALL (Adult Literacy and Life-skills Survey). 3 A synthesis is provided in Danish Technological Institute/Rand Europe/SKOPE 2004. 4 Defined as 18 to 24 year olds with only lower secondary education and not in education and training. 5 Percentage of 20-24 year olds completing at least upper secondary education.

4 The last indicator is of only limited validity, due to sample size of the cohort defined as just one year of birth (warning issued by Eurostat). All three indicators refer to the completion of educational programmes, and to some extent they duplicate one another. We suggest dropping the latter benchmark completion of upper secondary education because of limited validity due to small sample size as part of the Labour Force Survey. Data The monitoring of developments on upper secondary education mainly relies on two data sources: First, there are the data collected by UOE (Unesco, OECD, Eurostat). They rely on counts of enrolments and graduates provided by the countries administrations. Second, there are data recorded in surveys. Information on educational attainment is provided through the Labour Force Survey on the basis of interviews or proxy interviews of samples. The indicator on highest successfully completed level of education has been in LFS for very long time. There have been major revisions of the variables on education and training, with a complete restructuring in 1998. It is self reported educational attainment, liable to reporting bias or misunderstandings by the persons interviewed and to errors introduced by sampling. The main shortcoming, however, is that the information is not reported according to general or vocational programmes, thus waiving much information content. Data collected by UOE face different problems. The most important ones are the relation of graduates (persons), graduations (successful exams) and enrolments and the strive for providing unduplicated numbers of graduates. The unduplicated number of graduates is calculated by netting out students with multiple upper secondary graduation (graduating in a previous year and/or earning more than one qualification during the reference period). In some countries it is deemed to be impossible to provide unduplicated numbers. For provision of graduation rates all graduates regardless of age are calculated as a percentage of population of typical age. Typical age refers to the ages at the end of a cycle of education, starting from typical age of entry, theoretical duration of a cycle and assuming full-time attendance and no repetition of a year. Age at graduation may be higher than typical graduation age caused by repetition of years and/or by interrupting education and resuming later in life. This will cause overestimation of graduation rates achieved at typical age 6, but not necessarily overestimation of graduation achieved sometime in the life cycle. Demographic changes and varying sizes of age cohorts add another element contributing to bias. The main advantage of these graduation rates is the differentiation according to strand of education, providing graduation from general, vocational and pre-vocational programmes. UOE is the basic source of information on attainment in VET. Because of individuals graduating from general programmes as well as from vocational programmes 7 the sum of the two graduation rates is higher than the total rate, the size of the difference indicating the extent of multiple graduation. Unfortunately, the two sources of information on educational attainment lack comparability. UOE does not provide information on attainment in a well-defined age group. For European OECD countries typical graduation ages in upper secondary education vary mainly between 16 and 20 years, with Spain making an exception (typical age for graduation from prevocational programmes is 15-17 years). As mentioned above, longer duration of studies than assumed, and graduation at adult ages will result in an upward bias of graduation at 6 Overestimation is due to inclusion of all graduates in the nominator, but restricting the denominator to population of typical age. 7 Including pre-vocational programmes.

5 typical age. There are major discrepancies when comparing UOE indicator Upper secondary graduation rates at typical age with youth educational attainment based on LFS (cf. Figure 1). These differences accrue to some 18 percentage points or more in the cases of Germany, Denmark and Island (UOE 2001 higher than LFS). The differences are not unidiectional; e.g. for Sweden UOE graduation rates are 14 percentage points lower than LFS, and these cases are harder to explain. When ranking countries according to these indicators, results vary substantially. The problem of apparent contradictions needs to be addressed in more detail. Introducing separate treatment of graduation from general vs. vocational and prevocational programmes in the LFS indicator on educational attainment would be a great advantage. Figure 1: Youth education attainment level and upper secondary graduation rates 25 20 15 10 5 0-5 -10-15 -20-25 Difference (percentage points) between LFS Youth Education Attainment Level and OECD Upper Secondary Graduation Rates 2001 EL* SK SE LU* IE CZ HU BE ES FR PL FI DK DE IS Diff 22.5 21.4 14.5 11.5 8.6 5.5 1.4 0.4-1.5-3.2-3.4-4.5-17.5-18.4-20.6 LFS Youth Education Attainment Level (Percentage of the population aged 20 to 24 having completed at least upper secondary education) and OECD Upper Secondary Graduation Rates (Ratio of upper secondary graduates to total population at typical age of graduation (multiplied by 100) in public and private institutions) LFS 80.5* 94.4 85.5 77.5* 84.6 90.5 84.4 79.4 65.5 81.8 88.6 86.5 78.5 73.6 49.4 OECD 58.0* 73,0 71.0 66.0* 76.0 85.0 83.0 79.0 67.0 85.0 92.0 91.0 96.0 92.0 70.0 Source: For LFS data on Youth Education Attainment Level: Eurostat New Cronos data base, Structural Indicator II.9., extracted on the 24.08.2004; For OECD data on Upper Secondary Graduation Rates: OECD - Education at a Glance 2003 (p. 39), 2003 and OECD - Education at a Glance 2002 (p. 34), 2002.* Data for 2000. Both data sources have a problem in common: They use information of educational attainment/graduation classified according to ISCED levels. However, the mapping of qualifications by level of education is contested, pointing out inconsistencies as to what is counted as an ISCED level 3 qualification in different countries. The mapping seems to be more the result of political negotiations than underpinned by research. A remedy would be to go back to the original national surveys and their information on named qualifications. International teams of jurors should benchmark these qualifications by both level and programme orientation (general or vocational) to produce an international dataset that distinguishes qualifications by level and orientation. This is the only way a detailed time series dataset of adult qualifications across Europe that specifies the vocational qualifications can be obtained, and reclassifying data already recorded through LFS would provide adequate data to answer many questions on skills acquisition and social and economic impact of skills.

6 Life-long learning So far, participation in Life-long learning (LLL) is covered regularly by the LFS and the CVTS; the new AES should provide additional information. LFS provides information on LLL on a yearly basis, and allows analysis of those participating. Unfortunately, in the LFS participation in LLL is only referring to a period of four weeks prior to the survey. This obviously underestimates the extent of adult participation. OECD has provided comparisons of LFS with IALS, ECHP, ESWC and OECD/INES data for different years, showing a substantially lower participation rate in LFS than in all other surveys (OECD 1999, Ok/Tergeist 2003). The underestimation of the participation rate is most likely biased. Two examples: First, in previous years LFS has been carried out mostly in spring term - seasonal factors probably affect participation rates in different sectors of the economy unevenly. Second, another bias is connected with the duration of training programmes. The share of longer measures of LLL will be overestimated. If longer measures are unevenly distributed across educational attainment levels, with low skilled persons participating more often in longer measures than high skilled persons 8, than with a reference period of 4 weeks the participation rate of low skilled persons will be upward biased, compared with the participation rate of high skilled persons. Verisimilar a reference period of 4 weeks underestimates the gap of participation rates according to level of education and misleadingly implies a more equal distribution of participation. Expanding the reference period for participation in LLL could be a remedy, without loosing out of sight the problems of the interviewed people in recalling correctly all events related to Life-long learning for a period of 12 months. CVTS is restricted to company provided continuing training, and it is restricted to the employed. CVTS so far focused on private enterprises thus excluding the public sector. However the coverage of NACE categories has been further narrowed down, excluding some sectors because of expected difficulties with data collection (e.g. agriculture, forestry and fishing, and households employing domestic staff). So far coverage was further restricted to companies with at least 10 employees. Given the fact that a substantial share of work force is employed in small firms, with an expected upward trend in the future, this is a real hiatus. This size cut-off potentially leads to some distortion in the sense of over- or under-stating CVT efforts of individual countries, as the proportion of employees in micro enterprises differs strongly across countries. For example, in Italy, Spain and Portugal at least 40% of all employees in the NACE sectors covered in CVTS worked in enterprises with less than 10 employees, compared to 10% or less in Lithuania, Hungary and Romania (Behringer/Käpplinger forthcoming). CVTS did not collect any information on individual characteristics of participants in 1999, with the exception of gender. In order to allow for a monitoring of progress in achieving identified targets it seems necessary to collect some information on individual characteristics of those receiving company provided CVT. 8 For example, detailed analysis of national surveys for Germany has shown for several years much longer duration of participation of the low skilled, strongly influenced by their high share of measures of active labour market policy.

7 Methodological changes of surveys and their consequences: The example of the Structural Indicators Early school leavers and Life-long learning Early school leavers and Life-long learning have been two of an initial list of 35 Structural Indicators chosen for the purpose of regular discussion and assessment of the progress made in achieving the goals set at the Lisbon European Council in spring 2000. Definition Both structural indicators are calculated using data from the European Labour Force Survey (EU-LFS), which is obligatory in all EU-member states. Early school leavers: persons aged 18 to 24 highest level of education or training attained is ISCED 0, 1 or 2 have not received any education or training in the four weeks preceding the survey 9. Life-long learning: persons aged 25 to 64 have received education or training in the four weeks preceding the survey. The denominator for both consists of the total population of the same age group, excluding no answers to the question participation to education and training (and in the case of Early school leavers also excluding no answers to the questions 'highest level of education or training attained ). The definition of education and training is part of the respective commission regulations; cf. Box. Adaptation of variables With the strive for input harmonisation across member countries a number of modifications have been adopted, mostly following the corresponding regulations and leading to adjustments in the technological and methodological organisation of the national LFS. The aim of improving the adequacy of measurement by modifying the questionnaire as well as the intention to gain comparability across countries and across time are important issues. However, two types of problems occur in the short run. First, a comparison over time is difficult due to breaks in series caused by technological and methodological changes. Second, comparisons between countries are facing the problem that, due to different stages of the implementation process, some countries have already implemented the change while others have not. 9 This short reference period might cause an upward bias of the indicator, especially if explanatory notes are not followed properly during the interview. Another problem emerges for countries that formerly have not used a reference period of four weeks. This is the case for the Netherlands (until 1999), Portugal (until 2000) and France (until 2003). The reference period for the participation (and non-participation) in education and training in those countries has been only one week.

8 With reference to Early school leavers and Life-long learning, the latest adjustment concerns the reference year of 2003, when some of the EU-member countries for the first time implemented new education variables in their national surveys to take note of the distinction between formal education and other forms of learning activities (Commission Regulation (EC) No 2104/2002 (1); see box for details). In this case, both of the problems stated above apply. First, the comparison with reference year 2003 is extremely difficult, due to the inclusion of new variables widening the coverage of so-called taught learning activities, thereby most likely increasing the positive reply of respondents to the question of participation in education or training compared to previous years. Entering the definition of Life-long learning and Early school leavers, a wider coverage of participation in education and training influences both Structural Indicators, although in opposite directions. As a consequence, comparisons of both Structural Indicators across time will contain a bias, which as such does not need to reverse the trend that would have been observed under unchanged conditions, but at least brings in an important component of uncertainty concerning the actual development between the years 2002 and 2003.

9 Box: Legal Base for the implementation of new education and training variables The Commission Regulation (EC) No 2104/2002 of 28 November 2002 defines the following education and training variables to be surveyed, starting in 2003: Participation in formal education or training during previous four weeks - level, - field; Participation in courses and other taught learning activities during previous four weeks - total length, - purpose of the most recent course or other taught activity, - field of the most recent taught activity, - participated in most recent taught activity during working hours; The list of variables included in the Commission Regulation describes taught learning activities as attending any courses, seminars, conferences or receiving private lessons or instructions outside the regular education system. Council Regulation (EC) No 577/98 of 9 March 1998 on the organisation of a labour force sample survey in the Community addresses the following education and training variables: Participation in education or training during previous four weeks - purpose, - level, - type, - total length, - total number of hours, - highest successfully completed level of education or training, - year when this highest level was successfully completed, - non-tertiary vocational qualification obtained; The description of variables contained in the annex of the Commission Regulation (EC) No 1575/2000 differentiates type of education or training as - Class-room instruction - Instruction in a working environment (without complementary class-room instruction at a school or college including conferences, seminars, workshop courses) - Instruction combining both work experience and complementary class-room instruction (including any form of dual system or sandwich courses ) - Other type of instruction

10 Second, some countries have not yet implemented the new variables in 2003, since its inclusion has been optional. This is the case for Belgium, France, Italy, Netherlands, Lithuania, Malta, Poland, Portugal and United Kingdom (until second quarter 2003) 10. In other words, for these countries Early school leavers and Life-long learning are based on the old concept of Participation in Education or Training. Hence, cross-country comparisons are problematic for the year 2003. During the period of optional inclusion of the new variables this problem remains. Consequences As described above two issues need to be addressed, when drawing conclusions based on structural indicators LLL and ESL: The comparison over time on the one hand and the crosscountry comparison during the phase when not all of the countries survey according to the same definition on the other hand. The 2005 Progress Report towards the Lisbon Objectives in Education and Training (European Commission 2005) is one example where the underlying problems become obvious. In this report, the development of the structural indicators Lifelong learning and Early school leavers over the years 2000 until 2004 is discussed. Concerning the indicator on Life-long learning the report rightly addresses the problem of data inconsistency in more than one occasion (e,g, p. 5 and p. 18). It is acknowledged that the increase of the indicator is not an adequate estimate of real improvement, as a part of the increase was, however, due to a break in series in 2003 (p. 5). Nevertheless, the authors calculate a yearly rate of increase in LLL required to reach the benchmark of 12.5% in 2010. This ignores the fact that the level of LLL participation will probably be very different from the current level as soon as the new way of measuring LLL participation will be introduced in all countries. Further, in the report the best performing countries for this indicator are named, not recalling the fact that a number of countries have not implemented the new definition for the time being (which, as shown above, would most likely lead to an increase of LLL participation in those countries). Ranking countries that are measuring by using different concepts and definitions is, however, a problematic approach. On Early school leavers the authors note that the average rate of early school leaving in the EU is still high but decreasing steadily and that there was an improvement in the average EU ratio of almost 1.5% over the period 2000-2003 (p. 14). Although the new definition of the participation in education and training also affects the rate of Early school leavers, not more than a footnote is hinting towards the existing problem of data inconsistencies. 10 For some countries (Spain, Czech Republic, Estonia, Latvia) no clear information about the implementation process at the time of writing this annex has been available.

11 Table 1a: Life-long learning (LLL) in countries that have implemented the new education and training variables in 2 nd 2003 quarter Country LLL 2002 LLL 2003 Difference Sweden 18.4 34.2 15.8 Finland 18.9 25.3 6.4 Slovenia 9.1 15.1 6.0 Austria 7.5 12.5 5.0 Cyprus 3.7 7.9 4.2 Greece 1.2 3.7 2.5 Ireland 7.7 9.7 2.0 Denmark 18.4 18.9 0.5 Luxemburg 7.7 6.3-1.4 Weighted Country Mean 5.57 Table 1b: Life-long learning (LLL) in countries that have not implemented the new education and training variables in 2 nd quarter 2003 Country LLL 2002 LLL 2003 Difference Belgium 6.5 8.5 2.0 Lithuania 3.3 4.5 1.2 Portugal 2.9 3.7 0.8 Poland 4.3 5.0 0.7 Italy 4.6 4.7 0.1 Netherlands 16.4 16.5 0.1 Malta 4.4 4.2-0.2 United Kingdom 22.3 21.3-1.0 Weighted Country Mean 0.05 Source: Eurostat, Structural Indicator, Employment I.5.1, extracted on 12.05.2005. Life-long learning: Percentage of the population aged 25 to 64 participating in education and training over the four weeks prior to the survey. So far it has been argued that implementing a wider coverage of Life-long learning exerts a positive effect on the level of the indicator. Moreover, it also affects the structure of measured participation in LLL. In the following Box the upper figure depicts LLL by level of educational attainment for those countries in EU15 that maintained the concept of measurement in 2003 11. It shows stable differences between the different educational levels. Implementing the new methodology with its wider coverage of taught learning activities increases measured 11 Only for some of for the new member states data are available for 2000 and 2001, thus analysis had to be restricted to EU15.

12 participation in all groups, but by widening the gap between the educational groups (cf. Box, lower figure). Box: Life-long learning and measurement effects in 2003 30.0 25.0 Lifelong-learning by level of educational attainment (Group 1: Countries maintaining the old concept of LLL in 2003) 20.0 15.0 10.0 High Medium Low 5.0 0.0 2000 2001 2002 2003 Lifelong-learning by level of educational attainment (Group 2: Countries introducing a wider coverage of LLL in 2003) 30.0 25.0 20.0 15.0 10.0 High Medium Low 5.0 0.0 2000 2001 2002 2003 Source: Eurostat Data Sheet, 22.06.2004. Group1: Countries maintaining the old concept of LLL in 2003 are BE, IT, PT, UK. Group2: Countries having introduced the wider coverage of LLL in 2003 are DK, EL, IE, LU, AT, FI, SE. Four countries (DE, FR, ES, NL) are left out of the analysis due to only partial coverage of taught learning activities in 2003 (DE), changing of the reference period from one week to four weeks in 2003 (FR), LLL-data by educational attainment not available for 2003 (NL) or no clear information about the implementation of the new education variables available (ES). Life-long learning: Percentage of the population aged 25-64 participating in education and training over the four weeks prior to the survey. Level of educational attainment: Low: ISCED 0,1,2; Medium: ISCED 3,4; High: ISCED 5,6. Regarding Early school leavers, the implementation of the new questions on participation in Life-long learning is expected to exert a lessening effect on the indicator (cf. definition above). In line with this expectation in five out of the nine countries which fully implemented the new definition of Life-long learning the indicator of Early school leavers decreased between 0.5 percentage points (Austria and Slovenia) up to 2.6 percentage points (Ireland). Contrary to the expected effect, in Denmark, Finland and Cyprus the indicator for Early school leaving increased between 2002 and 2003. 12 For Denmark, the effect of the implementation of the 12 For Luxemburg, 2003 Data were not yet available.

13 new concept of measuring Life-long learning was comparatively low; hence the effect on ESL might be low, too 13. In addition, Denmark as well as Cyprus are amongst the countries with high degree of variation of results over time partly influenced by a low sample size. In Finland, the trend of decreasing participation in LLL since 2000 was not reversed by the implementation of the new variables, and it fits into the picture that the indicator of ESL has increased. Still, contrary to expectations, in countries that implemented the new concept for measurement of LLL the indicator of ESL decreased on average by 0.3 percentage points 14, compared to an overall decrease for EU25 of 0.5 percentage points. Other problematic aspects of Early school leavers as measured in the LFS stem from the definition applied. This group is calculated by filtering out those not having attained an educational level above lower secondary and not having been in education or training in the four weeks prior to the survey. There are two problems connected with this procedure: On the one hand, persons who have left school but participated during the reference period in some kind of learning activity outside formal education are not defined as Early school leavers, even though their taught learning activity will not necessarily lead to acquiring a qualification. It is estimated that restricting the definition to participation in regular education would result in an increase of the rate of Early school leavers in EU25 of more than 1 percentage point. On the other hand, there is an upward distortion, too. The procedure might result in defining persons as Early school leavers who, for one reason or another, had a longer break in their participation in education and training before the time of the interview. The reference explanatory notes say that interviewees being enrolled in a programme within the regular education system but on vacation should be considered in regular education, but still there might be longer breaks before enrolment causing over-estimation of Early school leavers. 15 One approach of approximating the magnitude of this group is to cross Early school leavers with the so called Main Status. Main status records whether the person considers him/herself as being e.g. employed, unemployed, a pupil or student or being in further training or unpaid work experience. Although this variable is optional in the LFS, many countries include it in their national surveys. The result of this exercise shows for some countries a remarkably high share of people, which by definition are classified as Early school leavers but consider themselves to be pupils, students, in further training or unpaid work experience. In Sweden and Finland more than 10% of those who have been identified as Early school leavers in 2003 claim that they are not. In the Czech Republic, Belgium and Austria the corresponding values were at 6.1%, 5.5% and 2.3%, respectively, whereas in countries like Italy, Denmark, Greece, Malta and Slovenia this number was around 1.5% (Germany, Spain and the UK have not implemented this variable). Looking back on data available for the years 2002 and 2001 shows firstly that this problem has been present also in previous years and as such is not a new phenomenon. Secondly, also in Ireland (4.1% in 2002) and the Netherlands (around 1% for both years) some people do not consider themselves as Early school leavers although being classified as such 16. Subtracting these people from the group of Early school leavers would consequently lead to lower national rates of Early school leavers, reducing this rate by up to one percentage point. It would be worthwhile to address in more detail the reasons for disparities between self-reported status and classification as Early school leaver, with special regard to the element unpaid work experience. It might be that unpaid work experience is increasing as a prerequisite to get company provided training, as a prerequisite for graduation, or for getting a regular job. 13 Due to the diverging age groups there is no direct relationship between the two indicators, but if a country experiences an increase in participation in LLL, the chances are that this will encompass younger age groups, too. 14 Unweighted country mean. 15 The implementation of quarterly surveys adds to this problem, because of the period of long vacations in Quarter 3. Proposals to solve this problem are discussed, but not yet decided. 16 Data for Belgium, Greece, France, Portugal, Finland (for 2003), Ireland (for 2002) and the Netherlands (for 2002 and 2001) is flagged as not reliable by Eurostat.

14 In recent years, also other modifications in the national LFS are to be taken into account, when comparing data of the two Structural Indicators over time and across countries. In 2003 France adjusted the reference period for participating in education or training from initially one week to four weeks common in all other EU-member states. The participation in Life-long learning increased from a rate that since 1992 has always been between 2.6% and 3.0% to 7.4 %, with the rate of Early school leavers, however, remaining at the old level. In the Netherlands (1999) and in Portugal (2000) the reference period was also changed from one to four weeks but showing no clear picture of sudden changes in the data. In Switzerland that with its LFS is also included in the Eurostat data collection system, the shortening of the reference period from 12 months to 4 weeks in 2003 and the implementation of the wider coverage of taught learning activities was accompanied by a decrease in Life-long learning from 34.4% to 24.8% and an increase in Early school leavers from 5.4% to 7.7%. Apart from these modifications and their effects, Eurostat publishes additional breaks for several countries, warning to compare the subsequent data with former years. This is the case for Italy (1993), Portugal (1998), Belgium (1999), Finland (2000), Sweden and Bulgaria (2001), Latvia and Lithuania (2002) and Luxemburg and Hungary (2003), where, according to Eurostat, changes in the survey characteristics cause a lack of comparability. For Early school leavers, Eurostat issues an additional warning that high degree of variation of results over time is partly influenced by a low sample size in the case of Denmark, Luxemburg, Island, Norway, Estonia, Latvia, Lithuania, Cyprus, Malta, Slovenia. All in all, the discussion above implies a fundamental problem, when employing time series of the Structural Indicators Life-long learning and Early school leavers to assess the progress in the achievements of the Lisbon Goals. Certainly, the harmonisation of the way different national LFS are conducted is the best way to guarantee the supply of high quality data in the long run. Paradoxically, this causes the lack of comparable data in the short run, both crosssectional and longitudinal. Policy conclusions based on data referring to recent years, however, have to bear in mind the difficulties resulting from this process. Pitfalls in comparing results from different surveys The discussion so far has been on difficulties and problems when comparing data from a survey which is conducted in the EU countries and which underlies changes over time. Most of the indicators used were based on European surveys. When trying to incorporate Non- European countries, there is not so much data available besides UOE. Taking again the example of participation in LLL, our starting point - Eurostat - indicates that comparable data for US or JP are not available. OECD provided information on participation in continuing education and training among the adult population in 2002 (OECD 2002), making use of different surveys referring to years between 1994/1995 and 2001. Most of the data have been collected in the different cycles of IALS (between 1994/95 and 1998/99). This introduces another element of uncertainty in the comparison, because of different timing of the reference period, with the uncertainty being more severe when comparing results that are (hoped to be) not so stable over time. The fact that some data have been collected earlier than others is probably not posing a problem when analysing rather stable variables normally expected to be changing only slowly over time, like literacy, for example. This is not the case with LLL, where across EU political action is taken to reach the benchmarks set by the council. While IALS has been conducted in a common framework, the comparison of data stemming from different national and international surveys is even more problematic. While in theory the intention often is to measure the same incidence, a number of indicators lack comparability

15 not only because of methodological and technical differences between surveys, but also due to the lack of commonly accepted definitions. For reasons of brevity we focus on reference periods, different methods of data collection and different sampling methods. One example illustrating this problem is the data on participation in continuing education and training among the adult population published by OECD in Education at a Glance 2001 (Indicator C6). Where available results from recent national sources are shown 17, supplemented with data stemming from the IALS. Thereby the time of conducting the survey varies from 1994/95 in Netherlands and Poland to 1999 in US. The table summarising the sources for the indicator points out not only differences in the reference periods (12 months in most countries, but also 6 months), but also in the data collection method (face-to-face vs. telephone) as well as in the sampling method (random-route in most countries, but also use of registers; cf. OECD 2001, p.360). Most likely these inconsistencies have an impact on the comparability of the results. On top of these arguments, another problem arises as a consequence of changes in the national surveys over time. With reference to the above discussion on modifications in internationally conducted surveys and their impact on time series, the same issue needs to be addressed when using national surveys for international comparisons. For OECD s indicator C6 (2001; participation rate in continuing education and training for 25-64 year olds) data for the United States are provided by the National Household Education Survey. The US National Centre for Education Statistics (NCES) notes that between 1995 and 2001 the changing of questionnaire structure, wording and response options could affect the measurement of course participation (NCES 2003). Thus, in cases where national surveys are used as substitutes for internationally conducted surveys, those aspects also need to be considered. 17 This is the case for Australia, Canada, Finland, Germany, Sweden, Switzerland and the US.

16 List of abbreviations ALL: CVET: CVTS: Adult Literacy and Life-skills Survey Continuing vocational training Continuing Vocational Training Survey ECHP: European Community Household Panel ESL: Early school leavers ESWC: European Survey of Working Conditions GDP: IALS: Gross Domestic Product International Adult Literacy Survey ISCED: International Standard Classification of Education INES: IVET: LFS: LLL: PISA: International Indicators of Education Systems Initial vocational education and training Labour Force Survey Life-long learning Programme for International Student Assessment TIMSS: Third International Mathematics and Science Study UOE: VET: Unesco-OECD-Eurostat Vocational education and Training

17 Bibliography Behringer, F. Käpplinger, B. (2005). Enterprise-provided training and its national context: Towards an explanatory model. Paper prepared for ECER 2005, Forthcoming. Beicht, U., Walden, G., Herget, H. (2004). Costs and Benefits of In-Company Vocational Education and Training in Germany. Bonn: BIBB. BIBB Forschung (2003). Kosten der betrieblichen Weiterbildung in Deutschland auf der Basis von CVTS II. Bonn: BIBB, Issue 3. Danish Technological Institute/Rand Europe/SKOPE (2004) Defining a Strategy for the Direct Assessment of Skills. Synthesis of Findings of Review of Initiatives (Phase 1). European Training Foundation ETF (2003). Thirteen Years of Cooperation and Reforms in Vocational Education and Training in the Acceding and Candidate Countries. Luxembourg: Office for Official Publications of the European Communities European Commission (2003), Final Summary Report. Directorate-General for Education and Culture, Implementation of Education & Training 2010 Work Programme, Working Group Making the best use of resources, Progress Report. European Commission (2005). Progress Towards the Lisbon Objectives in Education and Training - 2005 Report, Commission Staff Working Paper. Brussels. Green, A.; Hodgson, A.; Sakamoto, A., Spours, K. (2000). Financing vocational education and training. In: P. Descy, M. Tessaring (Eds.): Training in Europe. Second report on vocational training research in Europe 2000 (Vol 1). Luxembourg: Office for Official Publications of the European Communities, p. 71-113. Markowitsch, J., Hefler, G. (2003). Weiterbildung in Österreich im europäischen Vergleich I. Ergebnisse und Analysen der 2. Europäischen Erhebung zur betrieblichen Weiterbildung (CVTS II), Vienna. National Center for Education Statistics, U.S. Department of Education (2003). The Condition of Education 2003. Washington, DC: U.S. Government Printing Office. OECD (1999). Employment Outlook 1999. Paris. OECD (2001). Education at a Glance 2001. Paris OECD (2002). Education at a Glance 2002. Paris OECD (2004). OECD Employment Outlook 2004. Paris. OECD (2004a). Handbook for Internationally Comparative Education Statistics (). Paris. Ok, W., Tergeist, P.: Improving Workers Skills (2003). Analytical Evidence and the Role of the Social Partners. Paris: OECD Social, Employment and Migration Working Papers Nr. 10.