The role of indicators in education and training policies Claude SAUVAGEOT Workshop Lisbonne 26/1/2006
The availability of objective data for the planning, the monitoring of the education system, for the public debate and for policy-makers : a necessity
VET Specificities 4 domains - data on education and training : initial, continuing, Life Long Learning (LLL) - data on transition between education/training and employment - data on employment and labour market - data on human capital
Indicators and Assessment : a leverage for change in the education system In addition, VET is a leverage to change the labour market
National and international indicators projects A strong link between a national project and an international project They must support each other Without a national project, no international project An international project can stimulate a national project
How can we build a pertinent set of indicators at the country level? First step : to build an efficient and timely information system For VET, it is a big challenge because we need data on 4 domains (supply and demands sides, internal and external evaluation)
Second step Very important to clarify the objectives of the planning, of the monitoring and of the country For VET, important to clarify its role and its link with economic changes, employment changes, employers needs. For an international project, we need to identify some shared objectives
The set of indicators has to be adapted to - the target - the level of the analyse For an international project, it can be a subset of the national set It must be a small set to be useful It must be one of the tools we can use to develop the culture of evaluation that we need to improve the education and training system
What is an indicator? An indicator is a synthetic information item used to measure, evaluate and direct. It always refers to an objective to be reached. What is not an indicator? A basic information raw data
A good indicator allows to - Measure how far or how close we are from an objective - Identify problems or unacceptable situations - Meet policy concerns - Answer clearly to questions leading to its choice
Some indicators for policy makers Many examples for general education: At the country level: Publications prepared by Québec, France, Denmark, but also : Key Indicators on vocational education for Central Asian Countries (prepared by EFT). For international comparisons: Publications made by OECD (Education at a glance) or by UNESCO for EFA World education Indicators : Unesco/Oecd Eurostat Eurydice ETF indicators In process for TVET: TVET in Jordan
Some examples of objectives Overall Improving the qualifications of the working population Increasing the rate of employment of young people who have completed an initial level of vocational education Increasing the rate of vocational education graduates. (see EFA)
Some examples of objectives Development of capacity (Algeria, Morocco) Development of apprenticeship (Algeria, Morocco) Improving the Labour Market and raising its effectiveness (Egypt) Encouraging the private sector towards active participation in the training process (Egypt)
Some examples of objectives Meet the needs of the industry and the market (Israel) Reduce unemployment and welfare dependence (Israel) Development of private sector in VET (Morocco)
Jordan Objectives (1) 6 Strategic Goals : Enhancing the relevance between outputs of the education system and the requirements of social and economic development. Continuing the development of the infrastructure of general and vocational education. Developing the qualitative aspects and dimensions of general and vocational education. Improving the efficiency and effectiveness of the education system, with special emphasis on the economics of education, including its inputs, operations and outputs. Promoting the democracy of education, and fostering its role in social mobility. Given the chance to the individual for enrolled and complete his growth to the best of his motivation, abilities and readiness.
Jordan Objectives (2) 5 major objectives: Equipping Jordanians for the world of work either waged or self-employment. Enhancing upper and horizontal mobility in the labour market. Achieving equitable outcomes in TVET Rationalisation investment in training Maximising the value of public TVET expenditure.
Some examples of objectives: UNESCO Education For All 6 Dakar Objectives 1 Expanding and improving comprehensive early childhood care and education, especially for the most vulnerable and disadvantaged children. 2 Ensuring that by 2015 all children, particularly girls, those in difficult circumstances and those belonging to ethnic minorities, have access to and complete free and compulsory primary education of good quality. 3 Ensuring that the learning needs of all young people and adults are met through equal access to appropriate learning and life skills programmes.
Education For All 4 Achieving a 50 per cent improvement in levels of adult literacy by 2015, especially for women, and equal access to basic and continuing education for all 5 Eliminating gender disparities in primary and secondary education by 2005, and achieving gender equality in education by 2015, with a focus on ensuring girls full and equal access to and achievement in good quality basic education. 6 Improving all aspects of the quality of education and ensuring excellence so that recognised and measurable learning outcomes are achieved by all, especially in literacy, numeracy and essential life skills.
Some examples of objectives: European Union (1) A great goal «To become the most competitive and dynamic knowledge-based economy in the world capable of sustainable economic growth with more end better jobs and greater social cohesion.» Three strategic objectives: 1 Increasingthe qualityand effectivenessof education and training systems in European Union ; 2 - Facilitating the access of all to the education and training systems; 3 - Opening up education and training systems to the wider world.
Some examples of objectives: European Union (2) Each strategic objective is divided in several subobjectives. Objective n 1 is divided in 5 sub-objectives, objective n 2 in 3 and 3 in 5. We obtain the following 13 sub-objectives Strategic Objective n 1 Increasing the quality and effectiveness of education and training systems in European Union Objective n 1.1 Improving education and training for teachers and trainers Objective n 1.2 Developing skills for the knowledge society Objective n 1.3 Ensuring access to ICTs for everyone Objective n 1.4 Increasing the recruitment to scientific and technical studies Objective n 1.5 Making the best use of resources
A very very important step Moving from objectives to indicators
EDUCATION FOR ALL 18 indicators Early childhood and pre-primary Primary : access Primary : finances Primary : teachers Primary : efficiency Primary : assessment of basic learning competencies Literacy
European Union Indicators (1) Objective 1.1 - Improving education and training for teachers and trainers Ind. 1 Distribution of teachers by age. This is an approximation of the shortage/surplus of qualified teachers and trainers on the labour market. Ind. 2 Number of young people under 15 in the total population (replaces development of the number of candidates for training programmes (teachers and trainers)). Ind. 3 Pupil (or trainee) to teacher (or trainer) ratio (replaces the percentage of teachers and trainers on continuing vocational training courses).
European Union Indicators (2) Objective 1.2 - Developing skills for the knowledge society - Ind. 4: Percentage of the population aged 22 that completes higher secondary education; 2010 objective: 85%. - Continuing training of teachers in fields that demonstrate a need for skills (abandoned). Ind. 5: Percentage of pupils at proficiency level 1 or below on the PISA reading scale; objective: reduce this percentage by 20% by 2010. Ind. 6: Levels reached in reading/writing (PISA Programme for International Student Assessment coordinated by OCDE). Ind. 7: Levels reached in numeracy/mathematics (PISA). Ind. 8: Levels reached in science (PISA). Levels reached for Learning-to-learn skills (abandoned). Ind. 9: Percentage of adults who did not complete higher secondary education but who have been involved in some kind of education or training initiative, by age group.
European Union Indicators (3) Objective 1.4 - Increasing recruitment to scientific and technical studies The total number of graduates in mathematics, science and technology must increase by at least 15% by 2010 with a reduction in gender imbalance. Ind. 10: Increase in the percentage of enrolments in mathematics, science and technology courses (ISCED levels 5A, 5B and 6, by gender) in relation to all enrolments in higher education. Ind. 11: Increase in the percentage of graduates in mathematics, science and technology courses (ISCED levels 5A, 5B and 6, by gender) in relation to all graduates in higher education. Ind. 12: Increase in the number of graduates on these courses, by gender. Increase in the number of scientists and engineers, by gender (abandoned). Ind. 13: Number of mathematics, science and technology higher education graduates per 1 000 inhabitants aged 20-29 (distribution according to ISCED 5A, 5B and 6). Increase in the number of qualified teachers in mathematics, science and technology fields (secondary level) (abandoned).
European Union Indicators (4) Objective 1.5 - Making the best use of resources Increase in the investment in human resources by inhabitant (structural indicator) measured by five indicators. Ind. 14: Proportion of public expenditure on education to GDP. Ind. 15: Proportion of private expenditure to GDP. Ind. 16: Company expenditure on continuing vocational training as a percentage of the wage bill. Ind. 17: Total expenditure by pupil/student according to level of education. Ind. 18: Total expenditure by pupil/student compared with GDP per capita.
European Union Indicators (5) Objective 2.1 Open learning environment Ind. 19 Percentage of people aged 25 to 64 involved in education and training initiatives (structural indicator); objective at least 12.5% by 2010. Objective 2.2 - Making learning attractive Ind. 20: Number of hours devoted to continuing vocational training for 1 000 working hours for companies with continuing vocational training courses or traineeships per economic activity. Ind. 21: Number of hours devoted to continuing vocational training for 1 000 working hours for all companies by economic activity. Ind. 22: Rate of participation by age and level of education. Objective 2.3 - Supporting active citizenship, equal opportunities and social cohesion - Ind. 23: Proportion of people aged between 18 and 24 who did not complete the first cycle of secondary education and who are not continuing their education or training (structural indicator); objective: no more than 10% in 2010. In the planning stage: an indicator measuring active citizenship.
European Union Indicators (6) Objective 3.4 - Increasing mobility and exchanges Ind. 26: Internal and external mobility of teachers and trainers in the context of Socrates (Erasmus, Comenius, Lingua and Grundtvig) and Leonardo da Vinci programmes. Ind. 27: Internal and external mobility of students in the context of Socrates and Leonardo da Vinci programmes. Ind. 28: Percentage of foreign students in higher education (ISCED 5 and 6) in relation to all students enrolled in the destination country, by nationality (EU countries or other countries). Ind. 29: Percentage of students (ISCED 5 and 6) from the country of origin who are enrolled abroad (EU country or another country). Proportion of students or people in training from a country who carry out part of their studies in another EU country or a third country (abandoned). Proportion of teachers, researchers and academics from other EU countries employed at different levels of the education system (abandoned).
Jordan Indicators (1) Context indicators (context) Ind.1 Educational attainment for population (16 years+) by age and sex. Ind.2 Labour force participation rates by educational level and sex. Ind.2-a Employment rates by educational level and sex. Ind.2-b Unemployment rates by age, sex and educational levels. Ind.3 Gross domestic product per capita.
Jordan Indicators (2) Equipping Jordanians for the world of work (participation) ( A ) All TVET Ind.4 Participation rates by age, gender and educational stream. Ind.5 Participation rates in school-based TVET as a percentage of all participants. Ind.6 Participation rates in initial vocational education and training by programme. ( B ) Apprenticeship Ind.7 Percentage of apprentices to total TVET participants. Ind.8 Distribution of apprentices by gender and specialization (number or percentage) to total participants.
Jordan Indicators (3) ( C ) Vocational Education Ind.9 Percentage of vocational education students to total TVET participants. Ind.10 Distribution ( number or percentage ) of vocational education students by gender and type of education. (D) Applied secondary education Ind.11 Percentage of applied secondary education participants to total TVET participants. Ind.12 Distribution ( number or percentage ) of applied secondary education participants by gender and specialisation.
Jordan Indicators (4) ( E ) Technical/Technician Education Ind.13 Percentage of students in technical/ technician education to total TVET participants. Ind.14 Distribution ( number or percentage ) of participants in technical / technician education by gender, age and specialisation.
Jordan Indicators (5) Achieving equitable outcomes (outputs and outcomes) (A) Access for equity groups Ind.15 Gross enrolment rate in TVET by gender, and stream. Ind.16 Net enrolment rate in TVET by region and stream. Ind.17 Gross enrolment rate in TVET by region, and stream. Ind.18 Net enrolment rate in TVET by region, and stream. (B) Performance/ Outcomes indicators Ind.19 Completion rates by programme. Ind.20 Dropout rates by age, sex, programme and semester. Ind.21 Percentage of graduates by age, sex, and programme.
Jordan Indicators (6) Rationalization investment in training ( resources ) Ind.22 Percentage of current public educational ( TVET ) expenditure to gross domestic product ( GDP ). Ind.23 Public educational expenditure of ( TVET ) to the total public expenditure. Ind.24 public expenditure per student by type of education and training Ind.25 Current public expenditure ( TVET ) per student as a percentage of gross domestic product per capita Ind.26 Distribution of public expenditure ( TVET) by type of education and training. Ind.27 Relative proportions of public and private investment in educational institutions. Ind.28 Funds of TVET by sources and type of education and training.
Jordan Indicators (7) Maximising the value of public TVET expenditure (outputs) Ind.29 Cost per contact hour. Ind.30 Graduate cost per programme. Ind.31 Teaching load per teacher. Ind.32 Teachers qualifications by gender, educational level and years of experience. Ind.33 Cost of in service training for trainers by programme.
Comments (1) European Union and Jordan indicators can be used as a starting point. Some countries have already calculated many indicators (Morocco, Turkey ) But some other areas have to be analysed: non-formal education, supply side.
Comments (2) Concerning non formal competencies, Unesco started a project for this domain. Unesco helps to build a MIS for non-formal education. A handbook and a software can be provided. Some indicators are suggested..
Indicators - questions A good indicator answer to a least one (good) question
Examples of questions Did we succeed to improve the level of qualification for the active population?
For France, the increase of the upper secondary level population is higher than the increase for many other countries Population with a ISCED level >= 3A or 3B : rate between the proportion for 25-34 years old and 55-64 years old 4,5 4 3,5 3 2,5 2 1,5 1 0,5 0 Spain Korea Greece Portugal Mexico Italy Turkey Hungary Ireland Belgium Finland France Poland Iceland Mean Japan Australia Luxembourg Sweden Austria Canada Norway N.Zealand Denmark Un.Kingdom Switzerland Czech Germany Un.States Higher than : Germany, USA, UK, Japan ; Lower than : Spain, Korea, Italy
Where are we on education expenditures? Expenditures on education as a percentage of GDP 8 7 6 5 4 3 2 1 0 Denmark Korea Norway Sweden Austria Canada Un.States France Switzerland Finland OECD total Portugal Count.mean Australia Spain Un.Kingdo Italy Belgium Hungary Mexico Japan Ireland Belgium Netherlands Czech Greece Turkey Higher than : Netherlands, Japan, UK A little bit lower than : Canada, USA Lower than : Sweden, Korea, Norway, Denmark
Are the young population (or the school leavers) more unemployed than the overall population? (by Isced level? By gender?))
Which impact for vocational training? - on unemployment - on qualification - on promotion
Did we increase enrolment ratios on VET? By age-group Isced level - gender
Did we increase the pass rate for vocational diplomas? Did we increase the number of graduates?
Additional question Did we increase the competencies in literacy, foreign language, computer skills?
Another question Is the transition from school to employment is easier for students with an industrial speciality than for students with a service speciality?
Did we succeed to improve relationships between schools and firms?
Is the school level linked with unemployment rate? For the overall population For young people For school leavers by gender
50 45 40 35 30 25 20 15 10 5 0 Aucun diplôme CAP/BEP Supérieur Brevet seul Baccalauréat
How to use indicators in vocational education and training? Monitoring an effective link between training and employment Monitoring human resources management in firms and its impact on the recruitment of young people Observation and forecast of the training employment relationship Lifelong learning
How to organise such a project at a country level? A small team but with a steering committee involving all the departments of the ministry of education and all the ministries involved in education The steering committee has to defined the set of the indicators. It must be a shared project The team has to look after all the sources of data available inside the Ministries and outside : population data, economic data (GNP,GDP ) A target : to produce quickly a synthetic document Important : a good layout easy now with computers and softwares
The different steps in work on indicators (1) Census of available sources Surveys, censuses Formal non formal and sometimes informal training Certification Funding Transition between training and work Forecasts
The different steps in work on indicators (2) Calculation Glossary Consistency of results Analysis and choices Using graphs
How to analyse the set of indicators? To write comments to do the document easy to read and to use Glossary with clear definitions In some cases : formula of calculations
The different steps in work on indicators (3) Internal and external assessment Updating the data and the document
Comparative analysis : a different purpose Choose a subset of national indicators shared by all the countries involved in the project Needs for international classifications and common definitions But for instance Jordan is in a good position to do it easily because they use ISCED (CITE) and ISCO (CITP) On labour market, we can use Key indicators of Labour Market (KLM) provided by ILO with standardised definitions INES Project for OECD countries World Education Indicator (WEI) : UNESCO / OECD project for some countries UNESCO : follow-up of EFA, ETF..
How to organise such a project at an international level (1)? Define a shared list of objectives and indicators (to be validated by national authorities) Discuss on definitions and classification used Define a international set which can be quickly available because the needed information is already available
How to organise such a project at an international level (2)? Define a medium-term strategy on shared objectives and indicators strongly linked with a medium-term strategy on data collection Try to take into account and to use the differences between countries to improve all the participating countries Take the sufficient time and provide a lot of effort to reach a good level of comparability
How to diffuse the results at the national level? Education and training involve more or less everybody It means diffusion must be as large as possible to feed the public debate All the kind of media and supports have to be used : Paper, CD-Rom, Websites..
How to diffuse the results at the international level? Need some discussion between ETF and the countries
When can we start? Right now, even with a small set of shared indicators To show to everybody and to policy makers how useful it is To convince about transparency To improve your information system : the virtuous circle Very important When the first document is finished Prepare immediately the second one
Just do it And good trip