The Effects of Upper Secondary Education and Training Systems on Literacy and Numeracy Skills Levels and Distributions Andy Green and Nicola Pensiero LLAKES CENTRE, UCL Institute of Education Third International PIAAC Conference, Madrid, November 7-9 th, 2016,
Research on Education System Effects on Skills Comparative research tells us much about the effects of education systems on skills during compulsory education (eg: Hanushek & Woßmann, 2006, 2010; OECD, 2010; Woßmann, 2008; Werfhorst and Mifs,2010). Much less of the contributions of upper secondary education and training systems. This is partly due to: lack of comparable longitudinal data for skills changes for a range of countries; little time series cross-sectional data for adult skills; Lack of comparability of qualifications across countries. This paper seeks to rectify this by employing quasi-cohort analysis of data on tested scores of 15 year olds in PISA 2000 and 27 year olds in the 2012 PIAAC survey. The two surveys ask different questions and use different scales so we can only compare relative changes in skills levels across the life course. However, using skills Gini measures, which take account of scale, we can estimate absolute changes in skills distributions. The analyses suggests that countries/systems vary considerably in changes in skills over this phase of the life course.
Comparative Research on System Effects on Skills Levels in Compulsory School The most consistent findings from the more curriculum-oriented IEA studies over the years is that variation across countries in performance in particular subjects is associated with the total time spent on studying that subject (see also OECD 2013). Research using the more competence-oriented PISA surveys has focussed on the effects of school choice and competition (OECD, 2013); school autonomy (OECD, 2013; Bol and Van de Werfhorst, 2013) and accountability measures, including the use of centralised exist examinations (Bol and Van de Werfhorst, 2013, Woßmann, 2005), but the results are sometimes inconclusive. On the other hand, research finds fairly consistent associations across systems between skills inequality and systems with: early selection to different tracks or types of school; a high proportion of privately funded schools; a lack of standardization in curricula and assessment; federal systems where funding is devolved to the regional level (Hanushek and Woßmann, 2006, 2010; Schuetz et al., 2008; OECD, 2010)
Theories of Effects of Upper Secondary Education and Training on Skills Inequality Effects of branching points. Boudonian positional theory (1974) suggests that the more the branching points in an education system the more likely students from different social backgrounds will make differential choices about educational pathways which will tend to increase inequalities. Previous education research on upper secondary E and T (Lasonen and Young,1998; Raffe et al.,1998, 2001) suggests that where there is greater of parity of esteem between academic and vocation tracks this is likely to reduce skills inequality. Comparative political economy suggests that apprenticeship systems incentivise learning through prospects of skilled employment (Hall & Soskice, 2001).
Typologies of Upper Secondary Education and Training systems The literature on upper secondary education systems in the early 2000s identifies four broad types of upper-secondary education and training systems in OECD countries (OECD, 1985; Lasonen & Young,1998; Raffe et al., 2001; Green, 2000, 2003; Hodgson & Spours, 2014). The main overall dimensions of the classification relate to: 1) institutional integration/fragmentation and 2) Curricula standardisation/differentiation. Country groupings emerging closely resemble classifications of economies and welfare systems in the comparative political economy literature (see Esping-Andersen, 1990; Hall & Soskice, 2001; Pontussen, 2008; Green & Janmaat, 2011).
Type 1: Differentiated Dedicated Upper Secondary Predominantly school-based systems Institutions with general academic and vocational provision in different types of dedicated upper secondary institution and with apprenticeships representing separate but residual systems. (Czech Republic, Denmark, Estonia, France, Finland, Greece, Italy, Japan, Korea, Poland and Russia.) Typically 3 yr programs organised according to subject specialisms > full ISCED L3 quals with common core curriculum, including maths and national language Externally examined, grouped awards requiring passes in core subjects.
Type 2: Comprehensive Comprehensive school-based provision with general and vocation provision in one institution. Similar to Type 1 but with greater integration of institutions and programs. Work-based learning built into vocational programs. Much greater cross-institutional institutional variation in North American than Scandinavian systems, so subdivided into: Type 2a (US and Canada) Type 2b (Sweden and Norway)
Type 3: Separate Academic and Apprenticeship Systems Tracked School-based general education and Dual Systems of Apprenticeship (Austria, Germany, Switzerland). Generally 3 yr programs > full level 3 quals with common core subjects but highly differentiated across academic tracks in terms of subject specialisms and forms of assessment and regulation Social partner organisation: apprenticeships, closely integrated with labour markets Allow smooth transitions into skilled jobs which may create positive feedback loop for learning during apprenticeship (Hall and Soskice, 2001).
Type 4: Mixed Systems Mixed Systems have a high diversity of school- and employmentbased programs of variable length and quality but with dominant academic tracks. (Australia, England, Northern Ireland, Ireland, Scotland, Spain and New Zealand). Programs generally organised on flexible modular basis with competence-based vocational programs of no fixed duration No common core Maths and national language not mandatory - Market-oriented, with diversity providers, including private training organisations and private awarding bodies (UK).
Hypotheses Regarding Literacy and Numeracy 1. High rates of completion of full ISCED level 3 upper secondary education and training programs will reduce skills inequality. 2. Compulsory core curricula including study of Maths and national language will reduce skills inequalities. 3. Greater parity of esteem between the general and academic tracks will reduce skills inequality. This is most likely in upper secondary E&T systems with either a) Dual Systems of apprenticeship or b) integrated school-based general and vocational institutions. 4. HE participation rates will have non-linear and rather small effect on skills inequality. 5. The same mechanisms will increase mean levels on literacy and numeracy skills by raising the skills levels of those at the bottom end of attainment.
Independent Variables for System Characteristics for both Analyses Inclusiveness Variables: Rates of full upper secondary completion 2 yrs or more Rates of secondary and tertiary participation at 17/18 Social gradient of Upper secondary completion (effects of parental education level) Parity of Esteem Variables: Vocational Prevalence Social Mix of the Vocational Curriculum Standardisation Variables: - Mandatory Maths and Language Learning - Maths Prevalence Control Variables HE Participation Rates Youth Unemployment Rate/Utilisation of skills at work
Methodology for Estimating Effects of System Characteristics on Changes in Skills Levels and Distributions Changes in literacy and numeracy skills after lower secondary schooling are estimated using a pseudo cohort derived from 15 year olds in PISA 2000 and 27 year olds in PIAAC, conducted 12 years later (proxied by 25-29s). We use a DID strategy to estimate the effects of systems types and characteristics (Hanushek and Woessmann 2006, Card and Krueger 1994). Estimates for changes in skills levels are based on relative changes since we cannot make comparisons in absolute terms. Relative changes are measured in terms of changes in quintile position in rank country rank order on the two tests. The estimates for changes in skills distributions use a skills Gini measure which allows comparison of absolute changes.
Findings on Changes between 15 and 27 in Skills Inequality
Change in Literacy Skills Ginis between 15 and 27
Change in Numeracy Skills Ginis between 15 and 27
Changes in Inequality of Opportunity in Literacy Skills
Changes in Inequality of Opportunity in Numeracy Skills
Effects of System Types on Inequality of Numeracy and Literacy Outcomes The DID regressions show that compared with the Type 1 systems, Type 2 systems do not have a consistently different effect on skills inequality. Type 2a systems (Norway and Sweden) show for both domains a non-significant negative effect on inequality of skills outcomes but a significant positive (p < 0.2) effect on inequality of numeracy skills opportunities. Type 2b systems (US, CAN) show a positive effect on inequality of outcomes (only significant for numeracy at the p<0.05 level) and no significant effects on inequalities of skills opportunities.
Effects of System Types on Inequality of Numeracy and Literacy Outcomes However, Type 3 and Type 4 systems do differ significantly from the reference case. Type 3 systems (Germany; Austria) have significant negative effects on inequality of outcomes in literacy (p < 0.05) and numeracy (p < 0.1). They also have negative effects on inequalities of opportunity for numeracy and literacy skills (only at the p < 0.3 level). Type 4 Mixed systems have significant positive effects on inequality of outcomes in both literacy (p < 0.1) and numeracy (p < 0.05) and on inequality of skills opportunities in both literacy and numeracy (at the p < 0.05 level).
What Are the Upper Secondary Education and Training System Characteristics Associated with Mitigation of Skills Inequality?
Effects of System Types on Inequality of Numeracy and Literacy Outcomes The DID regressions show that compared with the Type 1 systems, Type 2 systems do not have a consistently different effect on skills inequality. Type 2a systems (Norway and Sweden) show for both domains a non-significant negative effect on inequality of skills outcomes but a significant positive (p < 0.2) effect on inequality of numeracy skills opportunities. Type 2b systems (US, CAN) show a positive effect on inequality of outcomes (only significant for numeracy at the p<0.05 level) and no significant effects on inequalities of skills opportunities.
Effects of System Types on Inequality of Numeracy and Literacy Outcomes However, Type 3 and Type 4 systems do differ significantly from the reference case. Type 3 systems (Germany; Austria) have significant negative effects on inequality of outcomes in literacy (p < 0.05) and numeracy (p < 0.1). They also have negative effects on inequalities of opportunity for numeracy and literacy skills (only at the p < 0.3 level). Type 4 Mixed systems have significant positive effects on inequality of outcomes in both literacy (p < 0.1) and numeracy (p < 0.05) and on inequality of skills opportunities in both literacy and numeracy (at the p < 0.05 level).
DID Regression Results on System Characteristics The DID regressions show a non-linear and quite small association between participation rates and changes in skills inequality. Upper secondary education seems to be more important. There are only four variables which are significantly associated with changes in inequality of skills outcomes (distributions) for both literacy and numeracy. Only four variables are significantly associated with relative changes in inequality of opportunity for skills The indicators for parity of esteem generally have weaker effects than the indicators for standardisation of curricula. The scatterplots to follow show the significant relationships.
HE Participation Rates and Literacy Skills Mitigation
Effects of Standardisation of Curricula The strongest effects we find on the mitigation of skills inequalities come from variables for the prevalence of Maths and national language learning and completion rates for full upper secondary education. Mandatory provision of both Maths and national language has a significant negative effect on inequality of skills outcomes in both literacy (p<0.05) and numeracy (p < 0.01). It also has a highly significant negative effect (relatively) on inequalities of skills opportunities for both literacy (p < 0.01) and numeracy (p < 0.01). Prevalence of Maths learning (see Figure 7) also has a significant negative effects on inequality of skills outcomes in literacy and numeracy (both at the p < 0.01 level) and on inequalities of skills opportunities in both domains (both at the p < 0.01 level). Completion of full upper secondary education (>2 yrs) has significant negative effects on inequalities in skills outcomes both in literacy (p < 0.05) and numeracy (p < 0.1) and on inequalities of skills opportunities both in literacy (p < 0.1) and numeracy (p < 0.1).
Prevalence of Maths Study and Mitigation of Inequality in Numeracy
ISCED 3 Completion and Mitigation of Inequality in Literacy Skills
or inequality of opportunities for skills. Effects of Relative Parity of Esteem Our second hypothesis was that greater parity of esteem between the vocational and academic tracks would be likely to mitigate inequalities of skills. Our analysis only partially confirms the hypothesis. Vocational prevalence is positively associated with inequality mitigation of literacy skills outcomes (p < 0.1, Model 2). Countries where the proportion of students in vocational supper secondary programmes is higher tend to see greater mitigation in inequality of literacy skills outcomes, as, for instance, in Austria, Germany and Norway. The social mix of vocational programmes is also positively associated with mitigation of inequality of skills outcomes in literacy. Countries where vocational tracks are more prone to include children of graduate parents, such as Germany, Japan and the Scandinavian countries (except Sweden), do tend to show greater inequality mitigation in literacy skills whereas Anglophone countries, with less social mixing, tend to mitigate inequalities less. However, neither of these variables have significant effects on mitigating inequality of numeracy skills outcomes
Vocational Prevalence and Changes in Inequality of Literacy Skills
Conclusions Countries vary considerably in how far they mitigate skills inequality during the life course. This seems to have little to do with HE participation rates and more to do with upper secondary education and training systems. The system characteristics most associated with inequality mitigation are: - High rates of completion at the full ISCED Level 3; - Mandatory Maths and national language learning on all programs; - Relative parity of esteem between vocational and academic programs. Countries with Dual Systems (Austria and Germany) which combine all of these appear best at mitigating skills inequality (though from quite high base line). Central and eastern European countries with high level 3 completion and mandatory core learning also seem relatively successful, whatever their other systems characteristics. Countries with mixed systems with low level 3 completion, diverse program lengths and without mandatory maths and language learning are least successful.
Findings on Changes in Mean Skills Levels across Countries/Systems
Reading age 15 (quint) Literacy age 27 (quint) Change over pseudo-cohorts Math age 15 (quint) Numeracy age 27 (quint) Change over pseudo-cohorts AT 3 3 0 3 4 1 BE (Flanders) 5 4-1 5 5 0 CA 5 4-1 4 3-1 CZ 2 3 1 2 3 1 DE 1 3 2 1 3 2 DK 2 3 1 3 4 1 EN 4 2-2 4 2-2 ES 1 1 0 1 1 0 FI 5 5 0 4 5 1 FR 3 2-1 3 2-1 IR 4 1-3 2 1-1 IT 1 1 0 1 2 1 JP 4 5 1 5 5 0 KR 4 4 0 5 3-2 N IR
Patterns of Change Across Countries/Systems No change in ranking for most country/domains. Twelve instances (out of 40) where a country changes by two or more positions in the quintile ranking for either literacy or numeracy. Countries showing relative improvement or relative declines tend to do so in both domains and in the same direction. Germany goes up two places on both scales; Ireland goes down three places in literacy and one place in numeracy; Korea stays the same in literacy and goes down two places in numeracy; Norway and Sweden go up two places in both; England goes down two places in both. The other notable pattern is the uni-directional change across domains in two particular groups of countries. Most Anglophone countries (including Canada, Ireland, England and Northern Ireland) show relative declines in both domains. USA shows a relative decline in numeracy but no change in literacy. All Nordic countries (except Finland) show relative improvements in ranking in both literacy and numeracy.
Systems Types and Changes in Relative Skills Levels The system types associated with the largest positive improvement in literacy and numeracy rankings relative to the reference group are: the Type 2b (Nordic) comprehensive systems which improve their rank position in both numeracy and literacy relative to the reference group by almost two positions on the five point scale (p < 0.01 in both cases). the Type 3 systems with Dual Systems of apprenticeship which significantly (p<0.05) improve their rank order position for numeracy by 1.4 units on the five point scale relative to the reference group but show no significant improvement for literacy. In contrast, two system types have significant negative effects on changes in rank position in numeracy and literacy. Type 2a (North American) systems show a decline in rank order position of 0.6 units in numeracy and 1.1 units in literacy (significant at the p<0.3 and p<0.05 levels respectively). Type 4 mixed systems show a decline of 1.6 units in numeracy and 1.85 units in literacy (significant in both cases at the p<0.05 level).
PISA PIAAC change in rank ordering - Numeracy The Effects of HE Participation on Numeracy 3 2 DE NO SE 1 CZ AT IT DK PL FI 0 BE JP ES 28 38 48 58 68 78 88-1 FR IE NL US -2 KR EN -3 N IE -4 Tertiary level entry rate
PISA PIAAC change in rank ordering - Numeracy The Effects of the Social Gradient of Level 3 Completion on Numeracy 3 NO DE 2 SE DK FI 1 CZ PL AT JP 0 ES BE -1 NL US IE FR -2 KR EN -3 N IE -4 ISCED 3 social gradient 1 2 3 4 5 6 7 8 9
PISA PIAAC change in rank ordering - Numeracy The Effects of Vocational Prevalence on Numeracy 3 2 DE SE NO 1 0 ES JP DK FI PL CZ AT -1 US IE NL FR -2 KR EN -3 N IE -4 0,1 0,15 0,2 0,25 0,3 Vocational 0,35 prevalence 0,4 0,45 0,5 0,55 0,6
PISA PIAAC Change in rank ordergin - Numeraccy The Effects of Studying Maths in Upper Secondary Programmes 3 2 1 0 ES BE SE PL FI NO DE CZ AT IT JP -1 IE NL FR US CA -2 EN KR -3 N IE -4 0 1 2 3 4 5 6 % Students studying Maths
Conclusions: Effects of System Types Our analysis suggests that some types of US E and T systems have a significant impact on whether these countries show relative improvement in the mean literacy and numeracy competence of young people between 15 and 27 years of age. Countries with Type 2b comprehensive upper secondary education and training systems with little between-school variation (Sweden and Norway) or with Type 3 Dual systems (Austria and Germany) seem to be particularly effective. On the other hand, countries with Type 4 mixed systems (England, Ireland, Northern Ireland and Spain) all show relative decline in both literacy and numeracy.
What Characteristics of Different Systems Explain This? Countries with relatively low literacy and numeracy gain (England, Ireland, Northern Ireland and Spain) have mixed systems with: low levels of inclusiveness, including low rates of participation at 17/18, high social gradients of level 3 completion and, in most cases, higher rates of enrolment in private schools. low esteem for vocational learning, with lower rates of enrolment on vocational programmes, particularly in apprenticeships. least likely to require students to study maths and the national language at the upper secondary level.
What Characteristics of Different Systems Explain This? Countries which do relatively well (Nordic countries and Germanspeaking) rate higher across all of these dimensions. relatively inclusive at the upper secondary level, with high participation at 17/18, lower social gradients of level 3 completion, and lower proportions in private schools; exhibit greater esteem for vocational programmes, with high a high proportion of upper secondary students in vocational programmes or apprenticeships. All require the study of maths and the national language as part of the upper secondary curriculum.
What Characteristics of Different Systems Explain This? Tracking per se does not explain why some systems fare better than others at the US level. Relative improvements in literacy and numeracy skills are possible in systems with strong tracking (Austria and Germany) and in systems with low levels of tracking (Sweden and Norway) providing that: participation rates in long cycle L3 are high vocational education is prevalent and relatively highly esteemed study of maths and national language form mandatory part of curriculum,
Policy Implications We should worry less about the existence of tracking in US E&T than in earlier phases of education. What matters more is the relative quality of the general and vocational tracks. Countries that do not currently mandate the teaching and assessment of Maths and the national language on all programs in US E&T should consider doing so. Ensuring that all tracks in US E&T last 2 or more yrs and lead to full ISCED L3 qualifictions should be a priority.
Relevant Publications Green, A. and Pensiero, N. (2016) The Effects of Upper Secondary Education and Training Systems on Skills Inequality. A Quasi-Cohort Analysis using PISA 2000 and the OECD Survey of Adult Skills. British Education Research Journal. Green, A, Green, F. and Pensiero, N (2015) Cross-Country Variation in Adult Skills Inequality: Why are Skill Levels and Opportunities so Unequal in Anglophone Countries? Comparative Education Review, 59, 4 (featured article).
The Effects of System Types on Inequality of Numeracy and Literacy Outcomes Literacy DID estimate (γ 1 Y. age27) Numeracy DID estimate (γ 1 Y. age27) Model 1 (N: 21) England, Ireland, N. Ireland, Spain Education system (Ref.: Differentiated) Mixed 0.0103847 *** (0.0050311) 0.0200053 **** (0.0081531) Germany, Austria Dual -0.0173784**** (0.0082183) -0.0128706** (0.0075919) Sweden, Norway Comprehensive (Nordic) -0.0066884 (0.0062861) -0.0008654 (0.004107) US, Canada Comprehensive(North America 0.0049557 (0.005184) 0.0206856 ***** (0.0044668)
Table 2: Effects of System Types on Inequality of Opportunities in Literacy and Numeracy Literacy Numeracy DID estimate (γ 1 Y. age27) DID (γ 1 Y. age27) Model 1 (N: 21) Education system (Ref.: Differentiated) England, Ireland, N. Mixed 0.0257956 ** (0.0153532) 0.0414056 **** (0.018667) Ireland, Spain Germany, Austria Dual -0.0174881 * (0.0151912) -0.0246045 * (0.0197374) Sweden, Norway Comprehensive(Nordic) 0.0269098 ** (0.0177136) 0.0303923 ** (0.0200365) US, Canada Comprehensive(North America) -0.0120852 (0.0130983) 0.0070817 (0.0144121)
The Effects of System Characteristics on Inequality of Numeracy and Literacy Outcomes Model 2 Vocational prevalence -0.0385963 *** (0.0185493) -0.0122034 (0.0314484) (N: 18) Model 3 ISCED3 completion -0.0486412 **** (0.0215021) -0.0722444 *** (0.0382777) (N: 20) Model 4 Social mix vocational track -.0003294 * (0.0002874) -0.0002594 (0.0002596) (N: 17) Model 5 ISCED3 social gradient 0.0024947 ** (0.0017545) 0.003043 *** (0.0014741) (N: 19) Model 6 (N: 20) Math and language (0: ref cat) 1-0.0050595 (0.0088002) -0.0234435 **** (0.0089809) 2-0.0124188 **** (0.0052845) -0.0262719 ***** (0.0062045) Model 7 No Math -0.0025277 ** (0.0016167) -0.0047466 **** (0.0021075) (N: 21) Model 8 Youth unemployment (15-24, 2004) -0.009342 (0.025895) -0.0553222 ***** (0.0267629)
Effects of System Characteristics on Inequality of Opportunities in Literacy and Numeracy ***** p < 0.01, **** p<0.05, *** p<0.1, ** p < 0.2, * p < 0.3 Model 2 (N: 17) Vocational prevalence 0.0478344 (0.0561742) 0.0408097 (0.0778425) Model 3 (N: 19) ISCED3 completion -0.1168168 ** (0.0495715) -0.1763447 *** (0.0894423) Model 4 (N: 16) Social mix vocational track 0.0002567 (0.0007144) 0.0001697 (0.0009159) Model 5 (N: 18) ISCED3 social gradient 0.003513 * (0.0031732) 0.0056699 ** (0.0040376) Math and Language (0: ref. category) Model 6 (N: 20) Mandatory in 1 core subject -0.0043835 (0.01339) -0.0122578 (0.0129844) Both core subjects mandatory -0.0406949 ***** (0.0088289) -0.0600176 ***** (0.011541) Model 7 (N: 21) Maths prevalence -0.0087195 **** (0.0036245) -0.0141271 ***** (0.0042384) Model 8 (N: 21) Youth unemployment (15-24, 2004) -0.048641 (0.0982997) -0.0928815 (0.0993261) Model 9 (N: 20) HE enrollment rate 0.0031892 (0.0520461) 0.0192176 (0.0590414)
Estimates of Effects of System Types DID estimate (γ 1 Y. age27 ) -0.6* (0.49) 1.9***** (0.26) 0.9 (0.87) -1.85**** (0.69) Numeracy (quintiles) Literacy (quintiles) DID estimate (γ 1 Y. age27) Education system (Ref.: Model 1 (N: 20) Differentiated. CZ, DK, FI, BE, FR, IT, JP, KR, NL, PL) Type 2a Comprehensive America. CA, US) (North -1.1**** (0.39) Type 2b Comprehensive (Nordic. NO, SE) 1.9***** (0.39) Type 3 Dual (AT, DE) 1.4**** (0.57) Type 4 Mixed (EN, ES, IR, N IR) -1.6**** (0.76) ***** p < 0.01, **** p<0.05, *** p<0.1, ** p < 0.2, * p < 0.3
DID Estimates of Effects of System Characteristics Numeracy (quintiles) Literacy (quintiles) DID estimate (γ 1 Y. age27) DID estimate (γ 1 Y. age27) Model 1 (N: 19) Model 2 (N: 18) Model 3 (N: 19) Model 4 (N: 20) Model 5 (N: 17) Model 6 (N: 20) Enrolment age 17-18 1.03***** (0.25) 0.83***** (0.23) ISCED3 social gradient -0.56***** (0.13) -0.54***** (0.13) HE entry rates 0.31** (0.19) 0.32** (0.20) % secondary private schools -0.39* (0.34) 0.06 (0.35) Vocational prevalence 0.65***** (0.19) 0.34* (0.26) Vocational specificity 0.45***** (0.14) 0.39***** (0.13)