Sampling outcomes. Population coverage School and student response rates Teacher response rates

Size: px
Start display at page:

Download "Sampling outcomes. Population coverage School and student response rates Teacher response rates"

Transcription

1 Sampling outcomes Population coverage School and student response rates Teacher response rates effects and effective sample sizes Variability of the design effect The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law. PISA 2015 TECHNICAL REPORT

3 estimate based on the student sample of the proportion of the PISA-eligible 15-year-old population that is represented by the non-excluded PISA-eligible 15-year-old. Multiplying these two proportions together (P/(P + E) and (ST7b_3/ST7a_2.1)) gives the overall proportion of the NEP that was covered by the non-excluded portion of the student sample. Coverage index 1 and coverage index 2 will differ when countries have excluded geographical areas or language groups apart from other school-level exclusions. In these cases coverage index 2 will be less than coverage index 1. Coverage index 3: Coverage of the national 15-year-old population, calculated by P/ST7a_1: The national population of 15-year-olds, defined by sampling task 7a response box [1] and denoted here as ST7a_1 (and called all 15-year-olds in Table 11.1), is the entire population of 15-year-olds in each country (enrolled and not enrolled), based on national statistics. The value P is the weighted estimate of PISA-eligible non-excluded 15-year-old from the student sample. Thus (P/ST7a_1) indicates the proportion of the national population of 15-year-olds covered by the non-excluded portion of the student sample. It therefore also reflects the proportion of 15-year-olds excluded or not at school. Coverage index 4: Coverage of the estimated school population, calculated by (P + E)/S: The value (P + E) provides the weighted estimate from the student sample of all PISA-eligible 15-year-old in each country, where P is the weighted estimate of PISA-eligible non-excluded 15-year-old and E is the weighted estimate of PISA-eligible 15-year-old who were excluded within. The value S is an estimate of the 15-year-old school population in each participating country (called estimate of enrolled from frame in Table 11.1). This is based on the actual or (more often) approximate number of 15-year-old enrolled in each school in the sample, prior to contacting the school to conduct the assessment. The S value is calculated as the sum over all sampled of the product of each school s sampling weight and its number of 15-year-old (ENR) as recorded on the school sampling frame. Thus, (P + E)/S is the proportion of the estimated school 15-year-old population that is represented by the weighted estimate from the student sample of all PISA-eligible 15-year-old. It is influenced by the accuracy of the school sample frame, fluctuations in the target population size and the accuracy of the within-school sampling process. Its purpose is to check whether the student sampling has been carried out correctly, and to assess whether the value of S is a reliable measure of the number of enrolled 15-year-olds. This is important for interpreting coverage index 5. Coverage index 5: Coverage of the school sampling frame population, calculated by S/ST7b_3: The value (S/ST7b_3) is the ratio of the enrolled 15-year-old population, as estimated from data on the school sampling frame, to the size of the enrolled student population, as reported on sampling task 7b and adjusted by removing any additional excluded in the field. In some cases, this provided a check as to whether the data on the sampling frame gave a reliable estimate of the number of 15-year-old in each school. In other cases, however, it was evident that ST7b_3 had been derived using data from the sampling frame by the NPM, so that this ratio may have been close to 1.0 even if enrolment data on the school sampling frame were poor. Under such circumstances, coverage index 4 would differ noticeably from 1.0, and the figure for ST7b_3 would also be inaccurate. SCHOOL AND STUDENT RESPONSE RATES Tables 11.3 to 11.8 present school and student-level response rates at the national and regional levels. Tables 11.3 and 11.4 (by adjudicated regions) indicate the rates calculated by using only original and no replacement. Tables 11.5 and 11.6 (by adjudicated regions) indicate the improved response rates when first and second replacement were accounted for in the rates. Tables 11.7 and 11.8 (by adjudicated regions) indicate the student response rates among the full set of participating. PISA 2015 TECHNICAL REPORT

4 Table 11.1 [Part 1/2] PISA target populations and samples All 15-yearolds Enrolled 15-yearolds Target population School-level exclusions Target minus school level exclusions School level exclusion rate (%) Estimation of enrolled from frame participating Weighted number of participating excluded Australia Austria Belgium Canada Chile Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Latvia Luxembourg Mexico Netherlands New Zealand Norway Poland Portugal Slovak Republic Slovenia Spain Sweden Switzerland Turkey United Kingdom United States Albania Algeria Argentina Brazil B-S-J-G (China)* Bulgaria Colombia Costa Rica Croatia Cyprus Dominican Republic FYROM Georgia Hong Kong (China) Indonesia Jordan Kazakhastan Kosovo Lebanon Lithuania Macao (China) Malaysia Malta Moldova Montenegro Peru Qatar Romania Russian Federation Singapore Chinese Taipei Thailand Trinidad and Tobago Tunisia United Arab Emirates Uruguay Viet Nam * B-S-J-G (China) refers to the four PISA-participating China provinces: Beijing, Shanghai, Jiangsu and Guangdong. 1. Note by Turkey: The information in this document with reference to Cyprus relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the Cyprus issue. Note by all the European Union Member States of the and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus PISA 2015 TECHNICAL REPORT

5 Table 11.1 [Part 2/2] PISA target populations and samples Weighted number of excluded ineligible Weighted number of ineligible Withinschool exclusion rate (%) Overall exclusion rate (%) Percentage of ineligible / withdrawn Coverage Index 1 Coverage Index 2 Coverage Index 3 Coverage Index 4 Australia Austria Belgium Canada Chile Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Latvia Luxembourg Mexico Netherlands New Zealand Norway Poland Portugal Slovak Republic Slovenia Spain Sweden Switzerland Turkey United Kingdom United States Coverage Index 5 Albania Algeria Argentina Brazil B-S-J-G (China) Bulgaria Colombia Costa Rica Croatia Cyprus Dominican Republic FYROM Georgia Hong Kong (China) Indonesia Jordan Kazakhastan Kosovo Lebanon Lithuania Macao (China) Malaysia Malta Moldova Montenegro Peru Qatar Romania Russian Federation Singapore Chinese Taipei Thailand Trinidad and Tobago Tunisia United Arab Emirates Uruguay Viet Nam Note by Turkey: The information in this document with reference to Cyprus relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the Cyprus issue. Note by all the European Union Member States of the and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. PISA 2015 TECHNICAL REPORT

6 Table 11.2 [Part 1/2] PISA target populations and samples, by adjudicated regions All 15-yearolds Enrolled 15-yearolds Target School-level population exclusions Target minus school level exclusions School level exclusion rate (%) Estimation of enrolled from frame participating Weighted number of participating Belgium (Flemish community) Spain (Andalusia) Spain (Aragon) Spain (Asturias) Spain (Balearic Islands) Spain (Basque Country) Spain (Canary Islands) Spain (Cantabria) Spain (Castile and Leon) Spain (CastileLaMancha) Spain (Catalonia) Spain (Extremadura) Spain (Galicia) Spain (La Rioja) Spain (Madrid) Spain (Murcia) Spain (Navarra) Spain (Valencia) United Kingdom (Scotland) United States (Massachusetts (public)) United States (North Carolina (public)) United States (Puerto Rico) excluded Argentina (CABA) United Arab Emirates (Abu Dhabi) United Arab Emirates (Dubai) Puerto Rico is an unincorporated territory of the United States. As such, PISA results for the United States do not include Puerto Rico. Table 11.2 [Part 2/2] PISA target populations and samples, by adjudicated regions Weighted number of excluded ineligible Weighted number of ineligible Withinschool exclusion rate (%) Overall exclusion rate (%) Percentage of ineligible / Coverage withdrawn Index 1 Coverage Index 2 Coverage Index 3 Coverage Index 4 Belgium (Flemish community) Spain (Andalusia) Spain (Aragon) Spain (Asturias) Spain (Balearic Islands) Spain (Basque Country) Spain (Canary Islands) Spain (Cantabria) Spain (Castile and Leon) Spain (CastileLaMancha) Spain (Catalonia) Spain (Extremadura) Spain (Galicia) Spain (La Rioja) Spain (Madrid) Spain (Murcia) Spain (Navarra) Spain (Valencia) United Kingdom (Scotland) United States (Massachusetts (public)) United States (North Carolina (public)) United States (Puerto Rico) Coverage Index 5 Argentina (CABA) United Arab Emirates (Abu Dhabi) United Arab Emirates (Dubai) Puerto Rico is an unincorporated territory of the United States. As such, PISA results for the United States do not include Puerto Rico. For calculating school response rates before replacement, the numerator consisted of all original sample with enrolled age-eligible who participated (i.e., assessed a sample of PISA-eligible, and obtained a student response rate of at least 50%). The denominator consisted of all the in the numerator, plus those original sample with enrolled age-eligible that either did not participate or failed to assess at least 50% of PISA-eligible sample. Schools that were included in the sampling frame, but were found to have no age-eligible, or which were excluded in the field were omitted from the calculation of response rates. Replacement do not figure in these calculations PISA 2015 TECHNICAL REPORT

7 Table 11.3 Response rates before school replacement Weighted school participation rate before replacement (%) (SCHRRW1) Weighted number of responding (weighted also by enrollment) (NUMW1) Weighted number of sampled (responding + non-responding) (weighted also by enrollment) (DENW1) Unweighted school participation rate before replacement (%) (SCHRRU1) responding (unweighted) (NUMU1) responding and non-responding (unweighted) (DENU1) Australia Austria Belgium Canada Chile Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Latvia Luxembourg Mexico Netherlands New Zealand Norway Poland Portugal Slovak Republic Slovenia Spain Sweden Switzerland Turkey United Kingdom United States Albania Algeria Argentina Brazil B-S-J-G (China) Bulgaria Colombia Costa Rica Croatia Cyprus Dominican Republic FYROM Georgia Hong Kong (China) Indonesia Jordan Kazakhastan Kosovo Lebanon Lithuania Macao (China) Malaysia Malta Moldova Montenegro Peru Qatar Romania Russia Singapore Chinese Taipei Thailand Trinidad and Tobago Tunisia United Arab Emirates Uruguay Viet Nam See note 1 under Table PISA 2015 TECHNICAL REPORT

8 Table 11.4 Response rates before school replacement, by adjudicated regions Weighted school participation rate before replacement (%) (SCHRRW1) Weighted number of responding (weighted also by enrollment) (NUMW1) Weighted number of sampled (responding + non-responding) (weighted also by enrollment) (DENW1) Unweighted school participation rate before replacement (%) (SCHRRU1) responding (unweighted) (NUMU1) responding and non-responding (unweighted) (DENU1) Belgium (Flemish community) Spain (Andalusia) Spain (Aragon) Spain (Asturias) Spain (Balearic Islands) Spain (Basque Country) Spain (Canary Islands) Spain (Cantabria) Spain (Castile and Leon) Spain (CastileLaMancha) Spain (Catalonia) Spain (Extremadura) Spain (Galicia) Spain (La Rioja) Spain (Madrid) Spain (Murcia) Spain (Navarra) Spain (Valencia) United Kingdom (Scotland) United States (Massachusetts (public)) United States (North Carolina (public)) United States (Puerto Rico) Argentina (CABA) United Arab Emirates (Abu Dhabi) United Arab Emirates (Dubai) Puerto Rico is an unincorporated territory of the United States. As such, PISA results for the United States do not include Puerto Rico. For calculating school response rates after replacement, the numerator consisted of all sampled (original plus replacement) with enrolled age-eligible that participated (i.e., assessed a sample of PISA-eligible and obtained a student response rate of at least 50%). The denominator consisted of all the in the numerator, plus those original sample that had age-eligible enrolled, but that failed to assess at least 50% of PISA-eligible sample and for which no replacement school participated. Schools that were included in the sampling frame, but were found to contain no age-eligible, were omitted from the calculation of response rates. Replacement were included in rates only when they participated, and were replacing a refusing school that had age-eligible. In calculating weighted school response rates, each school received a weight equal to the product of its base weight (the reciprocal of its selection probability) and the number of age-eligible enrolled in the school, as indicated on the school sampling frame. With the use of probability proportional to size sampling, where there are no certainty or small, the product of the initial weight and the enrolment will be a constant, so in participating countries with few certainty school selections and no oversampling or undersampling of any explicit strata, weighted and unweighted rates are very similar. The weighted school response rate before replacement is given by the formula: 11.1 weighted school response rate = before replacement i Y i (Y N) WE i i WE i i where Y denotes the set of responding original sample with age-eligible, N denotes the set of eligible non-responding original sample, W i denotes the base weight for school i, W i = 1/P i where P i denotes the school selection probability for school i, and E i denotes the enrolment size of age-eligible, as indicated on the sampling frame PISA 2015 TECHNICAL REPORT

9 Table 11.5 Response rates after school replacement Weighted school participation rate after all replacement (%) (SCHRRW3) Weighted number of responding (weighted also by enrollment) (NUMW3) Weighted number of sampled (responding + non-responding) (weighted also by enrollment) (DENW3) Unweighted school participation rate after all replacement (%) (SCHRRU3) responding (unweighted) (NUMU3) responding and non-responding (unweighted) (DENU3) Australia Austria Belgium Canada Chile Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Latvia Luxembourg Mexico Netherlands New Zealand Norway Poland Portugal Slovak Republic Slovenia Spain Sweden Switzerland Turkey United Kingdom United States Albania Algeria Argentina Brazil B-S-J-G (China) Bulgaria Colombia Costa Rica Croatia Cyprus Dominican Republic FYROM Georgia Hong Kong (China) Indonesia Jordan Kazakhastan Kosovo Lebanon Lithuania Macao (China) Malaysia Malta Moldova Montenegro Peru Qatar Romania Russia Singapore Chinese Taipei Thailand Trinidad and Tobago Tunisia United Arab Emirates Uruguay Viet Nam See note 1 under Table PISA 2015 TECHNICAL REPORT

10 Table 11.6 Response rates after school replacement, by adjudicated regions Weighted school participation rate after all replacement (%) (SCHRRW3) Weighted number of responding (weighted also by enrollment) (NUMW3) Weighted number of sampled (responding + non-responding) (weighted also by enrollment) (DENW3) Unweighted school participation rate after all replacement (%) (SCHRRU3) responding (unweighted) (NUMU3) responding and non-responding (unweighted) (DENU3) Belgium (Flemish community) Spain (Andalusia) Spain (Aragon) Spain (Asturias) Spain (Balearic Islands) Spain (Basque Country) Spain (Canary Islands) Spain (Cantabria) Spain (Castile and Leon) Spain (CastileLaMancha) Spain (Catalonia) Spain (Extremadura) Spain (Galicia) Spain (La Rioja) Spain (Madrid) Spain (Murcia) Spain (Navarra) Spain (Valencia) United Kingdom (Scotland) United States (Massachusetts (public)) United States (North Carolina (public)) United States (Puerto Rico) Argentina (CABA) United Arab Emirates (Abu Dhabi) United Arab Emirates (Dubai) Puerto Rico is an unincorporated territory of the United States. As such, PISA results for the United States do not include Puerto Rico. The weighted school response rate, after replacement, is given by the formula: 11.2 weighted school response rate = after replacement i (Y R) i (Y R N) WE i i WE i i where Y denotes the set of responding original sample, R denotes the set of responding replacement, for which the corresponding original sample school was eligible but was non-responding, N denotes the set of eligible refusing original sample, W i denotes the base weight for school i, W i = 1/P i, where P i denotes the school selection probability for school i, and for weighted rates, E i denotes the enrolment size of age-eligible, as indicated on the sampling frame. For unweighted student response rates, the numerator is the number of for whom assessment data were included in the results less those in with between 25 and 50% student participation. The denominator is the number of sampled who were age-eligible, and not explicitly excluded as student exclusions. For weighted student response rates, the same number of appears in the numerator and denominator as for unweighted rates, but each student was weighted by its student base weight. This is given as the product of the school base weight for the school in which the student was enrolled and the reciprocal of the student selection probability within the school. In countries with no oversampling of any explicit strata, weighted and unweighted student participation rates are very similar. Overall response rates are calculated as the product of school and student response rates. Although overall weighted and unweighted rates can be calculated, there is little value in presenting overall unweighted rates. The weighted rates indicate the proportion of the student population represented by the sample prior to making the school and student nonresponse adjustments PISA 2015 TECHNICAL REPORT

11 Table 11.7 Response rates, within after school replacement Weighted student participation rate after second replacement (%) (STURRW3) assessed (Weighted) (NUMSTW3) sampled (assessed + absent) (weighted) DENSTW3) Unweighted student participation rate after second replacement (%) (STURRU3) Number of assessed (unweighted) (NUMSTU3) sampled (assessed + absent) (unweighted) (DENSTU3) Australia Austria Belgium Canada Chile Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Latvia Luxembourg Mexico Netherlands New Zealand Norway Poland Portugal Slovak Republic Slovenia Spain Sweden Switzerland Turkey United Kingdom United States Albania Algeria Argentina Brazil B-S-J-G (China) Bulgaria Colombia Costa Rica Croatia Cyprus* Dominican Republic FYROM Georgia Hong Kong (China) Indonesia Jordan Kazakhastan Kosovo Lebanon Lithuania Macao (China) Malaysia Malta Moldova Montenegro Peru Qatar Romania Russia Singapore Chinese Taipei Thailand Trinidad and Tobago Tunisia United Arab Emirates Uruguay Viet Nam * See note 1 under Table PISA 2015 TECHNICAL REPORT

12 Table 11.8 Response rates, within after school replacement, by adjudicated regions Weighted student participation rate after second replacement (%) (STURRW3) assessed (weighted) (NUMSTW3) sampled (assessed + absent) (weighted) (DENSTW3) Unweighted student participation rate after second replacement (%) (STURRU3) assessed (Unweighted) (NUMSTU3) sampled (assessed + absent) (unweighted) (DENSTU3) Belgium (Flemish community) Spain (Andalusia) Spain (Aragon) Spain (Asturias) Spain (Balearic Islands) Spain (Basque Country) Spain (Canary Islands) Spain (Cantabria) Spain (Castile and Leon) Spain (CastileLaMancha) Spain (Catalonia) Spain (Extremadura) Spain (Galicia) Spain (La Rioja) Spain (Madrid) Spain (Murcia) Spain (Navarra) Spain (Valencia) United Kingdom (Scotland) United States (Massachusetts (public)) United States (North Carolina (public)) United States (Puerto Rico) Argentina (CABA) United Arab Emirates (Abu Dhabi) United Arab Emirates (Dubai) Puerto Rico is an unincorporated territory of the United States. As such, PISA results for the United States do not include Puerto Rico. TEACHER RESPONSE RATES Unweighted response rates for both science and non-science teachers were created using similar methods to those for unweighted student and school response rates that is, ineligible teachers are not used in the denominator for the rate calculation. These rates are presented in Table 11.9 for science teachers and in Table for the non-science teachers. In addition to these rates, unweighted response rates were calculated also for each sampled school in each country which implemented the Teacher Questionnaire. These rates were created as quality indicators for the questionnaire team who would use the Teacher Questionnaire data to create derived variables to help provide context about PISA. Table 11.9 Science teacher response rates Country Science teacher unweighted response rate (%) Science teacher numerator Science teacher denominator ineligible science teachers Australia Chile Czech Republic Germany Italy Korea Portugal Spain United States United States (Massachusetts (public)) United States (North Carolina (public)) Brazil B-S-J-G (China) Colombia Dominican Republic Hong Kong (China) Macao (China) Malaysia Peru Chinese Taipei United Arab Emirates United Arab Emirates (Abu Dhabi) United Arab Emirates (Dubai) PISA 2015 TECHNICAL REPORT

13 Table Non-science teacher response rates Country Non-Science teacher unweighted response rate (%) Non-Science teacher numerator Non-Science teacher denominator ineligible non-science teachers Australia Chile Czech Republic Germany Italy Korea Portugal Spain United States United States (Massachusetts (public)) United States (North Carolina (public)) Brazil B-S-J-G (China) Colombia Dominican Republic Hong Kong (China) Macao (China) Malaysia Peru Chinese Taipei United Arab Emirates United Arab Emirates (Abu Dhabi) United Arab Emirates (Dubai) DESIGN EFFECTS AND EFFECTIVE SAMPLE SIZES Surveys in education and especially international surveys rarely sample by simply selecting a random sample of (known as a simple random sample, or SRS). Rather, a sampling design is used where are first selected and, within each selected school, classes or are randomly sampled. Sometimes, geographic areas are first selected before sampling and. This sampling design is usually referred to as a cluster sample or a multi-stage sample. Selected attending the same school cannot be considered as independent observations as assumed with a simple random sample because they are usually more similar to one another than to attending other. For instance, the are offered the same school resources, may have the same teachers and therefore are taught a common implemented curriculum, and so on. School differences are also larger if different educational programmes are not available in all. One expects to observe greater differences between a vocational school and an academic school than between two comprehensive. Furthermore, it is well known that within a country, within sub-national entities and within a city, people tend to live in areas according to their financial resources. As children usually attend close to their home, it is likely that attending the same school come from similar social and economic backgrounds. A simple random sample of is thus likely to cover the diversity of the population better than a sample of 100 with 40 observed within each school. It follows that the uncertainty associated with any population parameter estimate (i.e., standard error) will be larger for a clustered sample estimate than for a simple random sample estimate of the same size. In the case of a simple random sample, the standard error of a mean estimate is equal to: 11.3 σ ( ˆ )= μ σ n 2 where σ 2 denotes the variance of the whole student population and n is the student sample size. PISA 2015 TECHNICAL REPORT

14 For an infinite population of and infinite populations of within, the standard error of a mean estimate from a cluster sample is equal to: σ σ σ ( μˆ )= + n n n within where σ 2 denotes the variance of the school means, σ2 within denotes the variances of within, n denotes the sample size of, and n denotes the sample size of within each school. The standard error for the mean from a simple random sample is inversely proportional to the square root of the number of selected. The standard error for the mean from a cluster sample is proportional to the variance that lies between clusters (i.e. ) and within clusters and inversely proportional to the square root of the number of selected and is also a function of the number of selected per school. It is usual to express the decomposition of the total variance into the between-school variance and the within-school variance by the coefficient of intraclass correlation, also denoted Rho. Mathematically, this index is equal to: 11.5 Rho = σ σ σwithin This index provides an indication of the percentage of variance that lies between. A low intraclass correlation indicates that are performing similarly while higher values point towards large differences between school performance. To limit the reduction of precision in the population parameter estimate, multi-stage sample designs usually use supplementary information to improve coverage of the population diversity. In PISA the following techniques were implemented to limit the increase in the standard error: (i) explicit and implicit stratification of the school sampling frame and (ii) selection of with probabilities proportional to their size. Complementary information generally cannot compensate totally for the increase in the standard error due to the multi-stage design however but will greatly reduce it. Table provides the standard errors on the PISA 2015 main domain scales, calculated as if the participating country sample was selected according to (i) a simple random sample; (ii) a multi-stage procedure without using complementary information (unstratified multi-stage sampling, with sampling weights ignored) and (iii) the unbiased BRR estimate for the actual PISA 2015 design, using Fay s method. It should be mentioned that the plausible value imputation variance was not included in these computations, which thus only reflect sampling error. Note that the values in Table for the standard errors for the unstratified multi-stage design are overestimates for countries that had a school census (Cyprus 1, Iceland, Luxembourg, Macao (China), Malta, Trinidad and Tobago, and Qatar) since these standard error estimates assume a sample of was collected. Also note that in some of the countries where the BRR estimates in Table are greater than the values for the unstratified multi-stage sample, this is because of regional or other oversampling (The countries with oversampling were: Argentina, Australia, Belgium, Brazil, B-S-J-G (China), Canada, Colombia, the Czech Republic, Denmark, Italy, Malaysia, Portugal, the United Arab Emirates, the United Kingdom). The BRR estimates in Table are also greater than the values for the unstratified multi-stage sample for almost all countries since nearly every country undersamples very small. As described in the sampling design chapter, some countries have a substantial proportion of attending that have fewer than the target cluster size (TCS). When small school undersampling was done, very small were undersampled while all other were slightly oversampled in compensation. In such cases, very small with at most 0, 1, or 2 age-eligible PISA expected to be enrolled were typically undersampled by a factor of 4 while the very small with between 3 and TCS/2 age-eligible PISA expected to be enrolled were undersampled by a factor of 2. This takes the allocation of to strata slightly away from proportional allocation, which can add slightly to weight variability and therefore to sampling variance. This is done though, to help countries minimise the operational burden of having too many small in their sample. For the other instances of countries in Table that have BRR estimates that are somewhat greater than estimates based on an unstratified multi-stage design it is unclear why the BRR variance should be larger, though it is possible that the stratification undertaken possibly did not explain enough between-school variance in these countries. 1. See note 1 under Table PISA 2015 TECHNICAL REPORT

15 It is usual to express the effect of the sampling design on the standard errors by a statistic referred to as the design effect. This corresponds to the ratio of the variance of the estimate obtained from the (more complex) sample to the variance of the estimate that would be obtained from a simple random sample of the same number of sampling units. The design effect has two primary uses in sample size estimation and in appraising the efficiency of more complex sampling plans (Cochran, 1977). In PISA, as sampling variance has to be estimated by using the 80 BRR replicates, a design effect can be computed for a statistic t using: 11.6 Var Deff() t = Var BRR SRS () t () t where Var BRR (t) is the sampling variance for the statistic t computed by the BRR replication method, and Var SRS (t) is the sampling variance for the same statistic t on the same data but considering the sample as a simple random sample. Based on Table 11.11, the unbiased BRR standard error on the mean estimate in science in Australia (for example) is equal to 1.46 (rounded from ). As the standard deviation of the science performance is equal to , the design effect in Australia for the mean estimate in science is therefore equal to: 11.7 VarBRR t Deff() t = () ( ) 2 VarSRS() t = = / The sampling variance on the science performance mean in Australia is about 2.94 times larger than it would have been with a simple random sample of the same sample size. Note that the participating are as this number were assessed for science. Another way to express the reduction of precision due to the complex sampling design is through the effective sample size, which expresses the simple random sample size that would give the same sampling variance as the one obtained from the actual complex sample design. The effective sample size for a statistic t is equal to: 11.8 n n Var Effn t SRS (t) () = = Deff() t Var BRR (t) where n is equal to the actual number of units in the sample. The effective sample size in Australia for the science performance mean is equal to: 11.9 n Effn() t = Deff() t = = In other words, a simple random sample of in Australia would have been as precise as the actual PISA 2015 sample for the national estimate of mean science proficiency. VARIABILITY OF THE DESIGN EFFECT Neither the design effect nor the effective sample size is a definitive characteristic of a sample. Both the design effect and the effective sample size vary with the variable and statistic of interest. As previously stated, the sampling variance for estimates of the mean from a cluster sample is proportional to the intraclass correlation. In some countries, student performance varies between. Students in academic usually tend to perform well while on average student performance in vocational is lower. Let us now suppose that the height of the was also measured, and there are no reasons why in academic should be of different height than in vocational. For this particular variable, the expected value of the betweenschool variance should be equal to zero and therefore, the design effect should tend to one. As the segregation effect differs according to the variable, the design effect will also differ according to the variable. The second factor that influences the size of the design effect is the choice of requested statistics. It tends to be large for means, proportions, and sums but substantially smaller for bivariate or multivariate statistics such as correlation and regression coefficients. PISA 2015 TECHNICAL REPORT

16 effects in PISA for performance variables The notion of design effect as given earlier is extended and gives rise to five different design effect formulae to describe the influence of the sampling and test designs on the standard errors for statistics. The total errors computed for the international PISA initial reports (, 2016a,b) that involves performance variables (scale scores) consist of two components: sampling variance and measurement variance. The standard error of proficiency estimates in PISA is inflated because the were not sampled according to a simple random sample and also because the estimation of student proficiency includes some amount of measurement error. For any statistic r, the population estimate and the sampling variance are computed for each plausible value and then combined as described in Chapter 9. The five design effects and their respective effective sample sizes are defined as follows: Effect VarSRS() r + MVar() r Deff1( r) = Var () r SRS where MVar(r) is the measurement variance for the statistic r. This design effect shows the inflation of the total variance that would have occurred due to measurement error if in fact the samples were considered as a simple random sample. Effect VarBRR() r + MVar() r Deff2( r) = Var () r + MVar() r SRS shows the inflation of the total variance due only to the use of a complex sampling design. Effect Var Deff3( r) = Var BRR SRS () r () r shows the inflation of the sampling variance due to the use of a complex design. Effect VarBRR() r + MVar() r Deff4( r) = Var () r BRR shows the inflation of the total variance due to measurement variance. Effect Deff () VarBRR() r + MVar() r 5 r = Var () r SRS shows the inflation of the total variance due to the measurement variance and due to the complex sampling design. The product of the first and second design effects equals the product of the third and fourth design effects, and both products are equal to the fifth design effect. Tables through present the values of the different design effects and the corresponding effective sample sizes for each of the major domains PISA 2015 TECHNICAL REPORT

17 Table Standard errors for the PISA 2015 main domain scales Collaborative problem solving Financial literacy Mathematical literacy Reading literacy Science literacy Country Simple random sample Unbiased BRR Simple random sample Unbiased BRR Simple random sample Unbiased BRR Simple random sample Unbiased BRR Simple random sample Australia Austria Belgium Canada Chile Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Latvia Luxembourg Mexico Netherlands New Zealand Norway Poland Portugal Puerto Rico (United States) Slovak Republic Slovenia Spain Sweden Switzerland Turkey United Kingdom United States Unbiased BRR Albania Algeria Argentina Brazil B-S-J-G (China) Bulgaria Colombia Costa Rica Croatia Cyprus Dominican Republic FYROM Georgia Hong Kong (China) Indonesia Jordan Kazakhstan Kosovo Lebanon Lithuania Macao (China) Malaysia Malta Moldova Montenegro Peru Qatar Romania Russia Singapore Chinese Taipei Thailand Trinidad and Tobago Tunisia United Arab Emirates Uruguay Viet Nam Puerto Rico is an unincorporated territory of the United States. As such, PISA results for the United States do not include Puerto Rico. 2. See note 1 under Table PISA 2015 TECHNICAL REPORT

18 Table effects and effective sample sizes for scientific literacy Country effect 1 Efect 2 effect 3 effect 4 effect 5 Australia Austria Belgium Canada Chile Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Latvia Luxembourg Mexico Netherlands New Zealand Norway Poland Portugal Puerto Rico (United States) Slovak Republic Slovenia Spain Sweden Switzerland Turkey United Kingdom United States size 1 size 2 size 3 size 4 size 5 Albania Algeria Argentina Brazil B-S-J-G (China) Bulgaria Colombia Costa Rica Croatia Cyprus Dominican Republic FYROM Georgia Hong Kong Indonesia Jordan Kazakhstan Kosovo Lebanon Lithuania Macao Malaysia Malta Moldova Montenegro Peru Qatar Romania Russia Singapore Chinese Taipei Thailand Trinidad and Tobago Tunisia United Arab Emirates Uruguay Viet Nam Puerto Rico is an unincorporated territory of the United States. As such, PISA results for the United States do not include Puerto Rico. 2. See note 1 under Table PISA 2015 TECHNICAL REPORT

19 Table effects and effective sample sizes for mathematical literacy Country effect 1 Efect 2 effect 3 effect 4 effect 5 Australia Austria Belgium Canada Chile Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Latvia Luxembourg Mexico Netherlands New Zealand Norway Poland Portugal Puerto Rico (United States) Slovak Republic Slovenia Spain Sweden Switzerland Turkey United Kingdom United States size 1 size 2 size 3 size 4 size 5 Albania Algeria Argentina Brazil B-S-J-G (China) Bulgaria Colombia Costa Rica Croatia Cyprus Dominican Republic FYROM Georgia Hong Kong Indonesia Jordan Kazakhstan Kosovo Lebanon Lithuania Macao Malaysia Malta Moldova Montenegro Peru Qatar Romania Russia Singapore Chinese Taipei Thailand Trinidad and Tobago Tunisia United Arab Emirates Uruguay Viet Nam Puerto Rico is an unincorporated territory of the United States. As such, PISA results for the United States do not include Puerto Rico. 2. See note 1 under Table PISA 2015 TECHNICAL REPORT

20 Table effects and effective sample sizes for reading literacy Country effect 1 Efect 2 effect 3 effect 4 effect 5 Australia Austria Belgium Canada Chile Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Latvia Luxembourg Mexico Netherlands New Zealand Norway Poland Portugal Puerto Rico (United States) Slovak Republic Slovenia Spain Sweden Switzerland Turkey United Kingdom United States size 1 size 2 size 3 size 4 size 5 Albania Algeria Argentina Brazil B-S-J-G (China) Bulgaria Colombia Costa Rica Croatia Cyprus Dominican Republic FYROM Georgia Hong Kong Indonesia Jordan Kazakhstan Kosovo Lebanon Lithuania Macao Malaysia Malta Moldova Montenegro Peru Qatar Romania Russia Singapore Chinese Taipei Thailand Trinidad and Tobago Tunisia United Arab Emirates Uruguay Viet Nam Puerto Rico is an unincorporated territory of the United States. As such, PISA results for the United States do not include Puerto Rico. 2. See note 1 under Table PISA 2015 TECHNICAL REPORT

21 Table effects and effective sample sizes for collaborative problem solving Country effect 1 Efect 2 effect 3 effect 4 effect 5 Australia Austria Belgium Canada Chile Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Latvia Luxembourg Mexico Netherlands New Zealand Norway Poland Portugal Puerto Rico (United States) Slovak Republic Slovenia Spain Sweden Switzerland Turkey United Kingdom United States size 1 size 2 size 3 size 4 size 5 Brazil B-S-J-G (China) Bulgaria Colombia Costa Rica Croatia Cyprus Hong Kong Lithuania Macao Malaysia Montenegro Peru Russia Singapore Chinese Taipei Thailand Tunisia United Arab Emirates Uruguay Puerto Rico is an unincorporated territory of the United States. As such, PISA results for the United States do not include Puerto Rico. 2. See note 1 under Table PISA 2015 TECHNICAL REPORT

22 Table effects and effective sample sizes for financial literacy Country effect 1 Efect 2 effect 3 effect 4 effect 5 Australia Chile Italy Netherlands Poland Slovak Republic Spain United States size 1 size 2 size 3 size 4 size 5 Brazil B-S-J-G (China) Lithuania Peru Russia To better understand the design effect for a particular country, some information related to the design effects and their respective effective sample sizes are presented in Annex C. In particular, the design effect and the effective sample size depend on: The sample size, the number of participating, the number of participating and the average withinschool sample size, which are provided in Table C.2 (Annex C); The school variance, school variance estimates and the intraclass correlation, which are provided respectively in Tables C.3 and C.4 (Annex C); The stratification variables, the intraclass correlation coefficient within explicit strata and the percentage of school variance explained by explicit stratification variables, which are provided respectively in Tables C.5 and C.6 (Annex C). Finally, the standard errors on the mean performance estimates are provided in Table C.1 (Annex C). References Cochran, W. (1977), Sampling Techniques (3 rd ed.), John Wiley and Sons. (2016a), PISA 2015 Results (Volume I): Excellence and Equity in Education, PISA, Publishing, Paris, org/ / en. (2016b), PISA 2015 Results (Volume II): Policies and Practices for Successful Schools, PISA, Publishing, Paris, PISA 2015 TECHNICAL REPORT

Overall student visa trends June 2017

Overall student visa trends June 2017 Acronyms Acronyms FSV First-time student visas The number of visas issued to students for the first time. Visas for dependants and Section 61 applicants are excluded

15-year-olds enrolled full-time in educational institutions;

CHAPTER 4 SAMPLE DESIGN TARGET POPULATION AND OVERVIEW OF THE SAMPLING DESIGN The desired base PISA target population in each country consisted of 15-year-old students attending educational institutions

Department of Education and Skills. Memorandum

Department of Education and Skills Memorandum Irish Students Performance in PISA 2012 1. Background 1.1. What is PISA? The Programme for International Student Assessment (PISA) is a project of the Organisation

Twenty years of TIMSS in England. NFER Education Briefings. What is TIMSS?

NFER Education Briefings Twenty years of TIMSS in England What is TIMSS? The Trends in International Mathematics and Science Study (TIMSS) is a worldwide research project run by the IEA 1. It takes place

Introduction Research Teaching Cooperation Faculties. University of Oulu

University of Oulu Founded in 1958 faculties 1 000 students 2900 employees Total funding EUR 22 million Among the largest universities in Finland with an exceptionally wide scientific base Three universities

National Academies STEM Workforce Summit September 21-22, 2015 Irwin Kirsch Director, Center for Global Assessment PIAAC and Policy Research ETS Policy Research using PIAAC data America s Skills Challenge:

Measuring up: Canadian Results of the OECD PISA Study

Measuring up: Canadian Results of the OECD PISA Study The Performance of Canada s Youth in Science, Reading and Mathematics 2015 First Results for Canadians Aged 15 Measuring up: Canadian Results of the

The Rise of Populism. December 8-10, 2017

The Rise of Populism December 8-10, 2017 The Rise of Populism LIST OF PARTICIPATING SCHOOL Byron College B Arsakeio Tositseio Lykeio Ekalis A Tositseio Arsakeio Lykeio Ekalis QSI Tbilisi Ionios School

The Survey of Adult Skills (PIAAC) provides a picture of adults proficiency in three key information-processing skills:

SPAIN Key issues The gap between the skills proficiency of the youngest and oldest adults in Spain is the second largest in the survey. About one in four adults in Spain scores at the lowest levels in

TIMSS Highlights from the Primary Grades

TIMSS International Study Center June 1997 BOSTON COLLEGE TIMSS Highlights from the Primary Grades THIRD INTERNATIONAL MATHEMATICS AND SCIENCE STUDY Most Recent Publications International comparative results

Welcome to. ECML/PKDD 2004 Community meeting

Welcome to ECML/PKDD 2004 Community meeting A brief report from the program chairs Jean-Francois Boulicaut, INSA-Lyon, France Floriana Esposito, University of Bari, Italy Fosca Giannotti, ISTI-CNR, Pisa,

PIRLS. International Achievement in the Processes of Reading Comprehension Results from PIRLS 2001 in 35 Countries

Ina V.S. Mullis Michael O. Martin Eugenio J. Gonzalez PIRLS International Achievement in the Processes of Reading Comprehension Results from PIRLS 2001 in 35 Countries International Study Center International

PROGRESS TOWARDS THE LISBON OBJECTIVES IN EDUCATION AND TRAINING

COMMISSION OF THE EUROPEAN COMMUNITIES Commission staff working document PROGRESS TOWARDS THE LISBON OBJECTIVES IN EDUCATION AND TRAINING Indicators and benchmarks 2008 This publication is based on document

HIGHLIGHTS OF FINDINGS FROM MAJOR INTERNATIONAL STUDY ON PEDAGOGY AND ICT USE IN SCHOOLS

HIGHLIGHTS OF FINDINGS FROM MAJOR INTERNATIONAL STUDY ON PEDAGOGY AND ICT USE IN SCHOOLS Hans Wagemaker Executive Director, IEA Nancy Law Director, CITE, University of Hong Kong SITES 2006 International

Improving education in the Gulf

Improving education in the Gulf 39 Improving education in the Gulf Educational reform should focus on outcomes, not inputs. Michael Barber, Mona Mourshed, and Fenton Whelan Having largely achieved the

Impact of Educational Reforms to International Cooperation CASE: Finland

Impact of Educational Reforms to International Cooperation CASE: Finland February 11, 2016 10 th Seminar on Cooperation between Russian and Finnish Institutions of Higher Education Tiina Vihma-Purovaara

Universities as Laboratories for Societal Multilingualism: Insights from Implementation

Universities as Laboratories for Societal Multilingualism: Insights from Implementation Dr. Thomas Vogel Europa-Universität Viadrina vogel@europa-uni.de The Agenda 1. Language policy issues 2. The global

May To print or download your own copies of this document visit Name Date Eurovision Numeracy Assignment

1. An estimated one hundred and twenty five million people across the world watch the Eurovision Song Contest every year. Write this number in figures. 2. Complete the table below. 2004 2005 2006 2007

DEVELOPMENT AID AT A GLANCE

DEVELOPMENT AID AT A GLANCE STATISTICS BY REGION 2. AFRICA 217 edition 2.1. ODA TO AFRICA - SUMMARY 2.1.1. Top 1 ODA receipts by recipient USD million, net disbursements in 21 2.1.3. Trends in ODA 1 Ethiopia

EXECUTIVE SUMMARY. TIMSS 1999 International Mathematics Report

EXECUTIVE SUMMARY TIMSS 1999 International Mathematics Report S S Executive Summary In 1999, the Third International Mathematics and Science Study (timss) was replicated at the eighth grade. Involving

RELATIONS. I. Facts and Trends INTERNATIONAL. II. Profile of Graduates. Placement Report. IV. Recruiting Companies

I. Facts and Trends II. Profile of Graduates III. International Placement Statistics IV. Recruiting Companies mir.ie.edu After the graduation of our 4th intake of the Master in International Relations

EXECUTIVE SUMMARY. TIMSS 1999 International Science Report

EXECUTIVE SUMMARY TIMSS 1999 International Science Report S S Executive Summary In 1999, the Third International Mathematics and Science Study (timss) was replicated at the eighth grade. Involving 41 countries

International House VANCOUVER / WHISTLER WORK EXPERIENCE

International House VANCOUVER / WHISTLER WORK EXPERIENCE 2 3 work experience At IH Vancouver, we understand that language acquisition is only the first step in achieving your career goals. With this in

Challenges for Higher Education in Europe: Socio-economic and Political Transformations

Challenges for Higher Education in Europe: Socio-economic and Political Transformations Steinhardt Institute NYU 15 June, 2017 Peter Maassen US governance of higher education EU governance of higher

AACSB Accredited Business Programs Business Students Study Abroad Office: 32 Sayre Drive, Coxe Hall, 1 st Floor Phone: 610-758-4877 Fax: 610-758-5156 Website: www.lehigh.edu/studyabroad Email: incis@lehigh.edu

SOCRATES PROGRAMME GUIDELINES FOR APPLICANTS

SOCRATES PROGRAMME GUIDELINES FOR APPLICANTS The present document contains a description of the financial support available under all parts of the Community action programme in the field of education,

GHSA Global Activities Update. Presentation by Indonesia

GHSA Global Activities Update Presentation by Indonesia as the GHSA chair in 2016 2016 Global Activities JEE Process Action Packages Coordination Jakarta Call for Action A Systemic Network Model : Coordination

Science and Technology Indicators. R&D statistics

2014 Science and Technology Indicators R&D statistics Science and Technology Indicators R&D statistics 2014 Published by NIFU Nordic Institute for Studies in Innovation, Research and Education Address

Students with Disabilities, Learning Difficulties and Disadvantages STATISTICS AND INDICATORS

Students with Disabilities, Learning Difficulties and Disadvantages STATISTICS AND INDICATORS CENTRE FOR EDUCATIONAL RESEARCH AND INNOVATION Students with Disabilities, Learning Difficulties and Disadvantages

The European Higher Education Area in 2012:

PRESS BRIEFING The European Higher Education Area in 2012: Bologna Process Implementation Report EURYDI CE CONTEXT The Bologna Process Implementation Report is the result of a joint effort by Eurostat,

Teaching Practices and Social Capital

D I S C U S S I O N P A P E R S E R I E S IZA DP No. 6052 Teaching Practices and Social Capital Yann Algan Pierre Cahuc Andrei Shleifer October 2011 Forschungsinstitut zur Zukunft der Arbeit Institute

CHAPTER 3 CURRENT PERFORMANCE

CHAPTER 3 current 3-1 3. Current Performance The examination of the performance of the n education system begins with an analysis of how students have fared over time, and in comparison with other countries,

SECTION 2 APPENDICES 2A, 2B & 2C. Bachelor of Dental Surgery

Cardiff University College of Biomedical and Life Sciences School of Dentistry Entry 2017 SECTION 2 APPENDICES 2A, 2B & 2C Bachelor of Dental Surgery Admissions Policy for Undergraduate Courses Entry 2017

The development of national qualifications frameworks in Europe

European Centre for the Development of Vocational Training WORKING PAPER No 8 The development of national qualifications frameworks in Europe Luxembourg: Publications Office of the European Union, 2010

Summary and policy recommendations

Skills Beyond School Synthesis Report OECD 2014 Summary and policy recommendations The hidden world of professional education and training Post-secondary vocational education and training plays an under-recognised

How to Search for BSU Study Abroad Programs

How to Search for BSU Study Abroad Programs Ways to Research Your BSU Options: Visit our website at http://studyabroad.bsu.edu Browse the print brochures outside of our office Speak to students who have

Advances in Aviation Management Education by Dr. Dale Doreen, Director International Aviation MBA Program John Molson School of Business Concordia University 15 th Annual Canadian Aviation Safety Seminar

OHRA Annual Report FY15

Contents Director s Statement... 3 Our Organization... 4 Institutional Review Board Operations... 5 Quality Improvement Program... 6 Program Metrics... 7 Highlights... 14 2 P a g e Director s Statement

Berkeley International Office Survey

Berkeley International Office Survey 1. What is your gender? Male 64.8% 308 Female 35.2% 167 2. What is your age? 17-20 0.0% 0 21-24 17.9% 85 25-30 56.6% 269 31-35 19.2% 91 36+ 6.3% 30 1 of 40 3. What

EQE Candidate Support Project (CSP) Frequently Asked Questions - National Offices

EQE Candidate Support Project (CSP) Frequently Asked Questions - National Offices What is the EQE Candidate Support Project (CSP)? What is the distribution of Professional Representatives within EPC member

The International Coach Federation (ICF) Global Consumer Awareness Study

www.pwc.com The International Coach Federation (ICF) Global Consumer Awareness Study Summary of the Main Regional Results and Variations Fort Worth, Texas Presentation Structure 2 Research Overview 3 Research

Target 2: Connect universities, colleges, secondary schools and primary schools

Target 2: Connect universities, colleges, secondary schools and primary schools with ICTs Target 2: Connect universities, colleges, secondary schools and primary schools with ICTs 1 Introduction Governments

international PROJECTS MOSCOW

international PROJECTS MOSCOW Lomonosov Moscow State University, Faculty of Journalism INTERNATIONAL EXCHANGES Journalism & Communication Partners IHECS Lomonosov Moscow State University, Faculty of Journalism

Eye Level Education. Program Orientation

Eye Level Education Program Orientation Copyright 2010 Daekyo America, Inc. All Rights Reserved. Eye Level is the key to self-directed learning. We nurture: problem solvers critical thinkers life-long

The development of ECVET in Europe

European Centre for the Development of Vocational Training WORKING PAPER No 14 The development of ECVET in Europe (2011) Luxembourg: Publications Office of the European Union, 2012 The development of

HARVARD GLOBAL UPDATE. October 1-2, 2014

HARVARD GLOBAL UPDATE October 1-2, 2014 All photos are part of the Office of International Education s annual photography contest and were taken by Harvard undergraduates engaged in study, work, internship,

The recognition, evaluation and accreditation of European Postgraduate Programmes.

1 The recognition, evaluation and accreditation of European Postgraduate Programmes. Sue Lawrence and Nol Reverda Introduction The validation of awards and courses within higher education has traditionally,

National Pre Analysis Report. Republic of MACEDONIA. Goce Delcev University Stip

National Pre Analysis Report Republic of MACEDONIA Goce Delcev University Stip The European Commission support for the production of this publication does not constitute an endorsement of the contents

DISCUSSION PAPER. In 2006 the population of Iceland was 308 thousand people and 62% live in the capital area.

Increasing Employment of Older Workers through Lifelong Learning Discussion Paper Jón Torfi Jónasson Institute of Social Science Research, University of Iceland Introduction This Peer Review is concerned

OHRA Annual Report FY16

Contents Director s Statement... 3 Our Organization... 4 Institutional Review Board Operations... 5 Quality Improvement Program... 6 Program Metrics... 7 Highlights... 133 What's Ahead......16 2 P a g

PISA 2015 Results STUDENTS FINANCIAL LITERACY VOLUME IV

ISA P r o g r a m m e f o r I n t e r n a t i o n a l S t u d e n t A s s e s s m e n t PISA 2015 Results STUDENTS FINANCIAL LITERACY VOLUME IV PISA PISA 2015 Results (Volume IV) STUDENTS FINANCIAL LITERACY

HAAGA-HELIA University of Applied Sciences. Education, Research, Business Development

HAAGA-HELIA University of Applied Sciences Education, Research, Business Development Finnish Education System 24 Universities of Applied Sciences 15 Universities Professional Master Degrees 1 1,5 5 4 3

Language. Name: Period: Date: Unit 3. Cultural Geography

Name: Period: Date: Unit 3 Language Cultural Geography The following information corresponds to Chapters 8, 9 and 10 in your textbook. Fill in the blanks to complete the definition or sentence. Note: All

OCW Global Conference 2009 MONTERREY, MEXICO BY GARY W. MATKIN DEAN, CONTINUING EDUCATION LARRY COOPERMAN DIRECTOR, UC IRVINE OCW

OCW Global Conference 2009 MONTERREY, MEXICO BY GARY W. MATKIN DEAN, CONTINUING EDUCATION LARRY COOPERMAN DIRECTOR, UC IRVINE OCW 200 institutional members in the OCWC Over 8,200 courses posted Over 130

RECOMMENDED CITATION: Pew Research Center, October, 2014, People in Emerging Markets Catch Up to Advanced Economies in Life Satisfaction

NUMBERS, FACTS AND TRENDS SHAPING THE WORLD FOR RELEASE OCTOBER 30, 2014 FOR FURTHER INFORMATION ON THIS REPORT: Katie Simmons, Senior Researcher Richard Wike, Director, Global Attitudes Research Russ

COST Receiving Site Locations (updated July 2013)

COST Receiving Site Locations (updated July 2013) American Samoa Summer Semester Australia Brisbane Elementary, Secondary, Special Ed, Early Childhood, and Physical Ed Resident Halls or * All students

International Branches

Indian Branches Chandigarh Punjab Haryana Odisha Kolkata Bihar International Branches Bhutan Nepal Philippines Russia South Korea Australia Kyrgyzstan Singapore US Ireland Kazakastan Georgia Czech Republic

IAB INTERNATIONAL AUTHORISATION BOARD Doc. IAB-WGA

GROUP A EDUCATION, TRAINING AND QUALIFICATION MINUTES OF THE MEETING HELD ON 28 AUGUST 2006 IN QUÉBEC CANADA 1. Welcome and Apologies Christian AHRENS opened the meeting welcoming everyone. Apologies had

UNIVERSITY AUTONOMY IN EUROPE II

UNIVERSITY AUTONOMY IN EUROPE II THE SCORECARD By Thomas Estermann, Terhi Nokkala & Monika Steinel Copyright 2011 European University Association All rights reserved. This information may be freely used

REFLECTIONS ON THE PERFORMANCE OF THE MEXICAN EDUCATION SYSTEM

DIRECTORATE FOR EDUCATION REFLECTIONS ON THE PERFORMANCE OF THE MEXICAN EDUCATION SYSTEM DAVID HOPKINS 1, ELPIDA AHTARIDOU, PETER MATTHEWS, CHARLES POSNER AND DIANA TOLEDO FIGUEROA 2 LONDON CENTRE FOR

TESL/TESOL Certification

TESL/TESOL Certification Teaching English as a Second Language Certificate inlingua Victoria TESL/TESOL Courses are recognized by TESL Canada inlingua Victoria College of Languages 101-910 Government Street

SOCIO-ECONOMIC FACTORS FOR READING PERFORMANCE IN PIRLS: INCOME INEQUALITY AND SEGREGATION BY ACHIEVEMENTS

Tamara I. Petrova, Daniel A. Alexandrov SOCIO-ECONOMIC FACTORS FOR READING PERFORMANCE IN PIRLS: INCOME INEQUALITY AND SEGREGATION BY ACHIEVEMENTS BASIC RESEARCH PROGRAM WORKING PAPERS SERIES: EDUCATION

ehealth Governance Initiative: Joint Action JA-EHGov & Thematic Network SEHGovIA DELIVERABLE Version: 2.4 Date:

ehealth Governance Initiative: Joint Action JA-EHGov & Thematic Network SEHGovIA DELIVERABLE JA D4.1.1 Strategy & Policy Alignment Documents I WP4 (JA) - Policy Development and Strategy Alignment Version:

Supplementary Report to the HEFCE Higher Education Workforce Framework

Supplementary Report to the HEFCE Higher Education Workforce Framework based on the international Changing Academic Profession (CAP) Study William Locke and Alice Bennion Centre for Higher Education Research

Information needed to facilitate the clarity, transparency and understanding of mitigation contributions

Climate Change Expert Group Paper No.2017(1) Information needed to facilitate the clarity, transparency and understanding of mitigation contributions Sara Moarif (IEA) May 2017 Unclassified COM/ENV/EPOC/IEA/SLT(2017)1

JAMK UNIVERSITY OF APPLIED SCIENCES

WELCOME ON EXCHANGE TO JAMK UNIVERSITY OF APPLIED SCIENCES SCHOOL OF TECHNOLOGY WWW.JAMK.FI 1 JYVÄSKYLÄ WHERE? Central Finland in the lake district of Finland 270 kilometres north of Helsinki (capital

(English translation)

Public selection for admission to the Two-Year Master s Degree in INTERNATIONAL SECURITY STUDIES STUDI SULLA SICUREZZA INTERNAZIONALE (MISS) Academic year 2017/18 (English translation) The only binding

Rethinking Library and Information Studies in Spain: Crossing the boundaries

Rethinking Library and Information Studies in Spain: Crossing the boundaries V IRGINIA O RTIZ- R EPISO U NIVERSIDAD C ARLOS III DE M ADRID D EPARTAMENTO DE B IBLIOTECONOMIA Y D OCUMENTACIÓN Barcelona,

North American Studies (MA)

North American Studies (MA) Master Discover the world at Leiden University Type Language City Master English Leiden Taking an integrated approach to the study of the United States, the MA North American

GEB 6930 Doing Business in Asia Hough Graduate School Warrington College of Business Administration University of Florida GENERAL INFORMATION Instructor: Linda D. Clarke, B.S., B.A., M.B.A., Ph.D., J.D.

APPLICATION GUIDE EURECOM IMT MASTER s DEGREES

APPLICATION GUIDE EURECOM IMT MASTER s DEGREES 2018-2019 OUTLINE DURATION... 3 APPLICATION DEADLINES... 5 APPLICATION PROCESS... 6 DETAILED APPLICATION PROCESS... 7 REQUIRED DOCUMENTS... 9 FINANCIAL MATTERS...

2001 MPhil in Information Science Teaching, from Department of Primary Education, University of Crete.

Athanasia K. Margetousaki Nikolaou Plastira 100, Vassilika Vouton GR 700 13 Heraklion, Crete Greece Phone. +302810391828 Fax: +30 2810 391583 e-mail amarge@iacm.forht.gr, amarge@edc.uoc.gr STUDIES 1995

The ELSA Moot Court Competition on WTO Law

The ELSA Moot Court Competition on WTO Law SWOT Analysis and Strategy 2012/2013 Table of Contents 1. A SWOT Analysis Basic information 2. The past decade of EMC2 a review 3. Present Situation the product

16-17 NOVEMBER 2017, MOSCOW, RUSSIAN FEDERATION OVERVIEW PRESENTATION

16-17 NOVEMBER 2017, MOSCOW, RUSSIAN FEDERATION OVERVIEW PRESENTATION Tuberculosis is the leading infectious killer Current actions and investments are falling far short ENDING TB IN THE SUSTAINABLE DEVELOPMENT

Information Session on Overseas Internships Career Center, SAO, HKUST 1 Dec 2016

Information Session on Overseas Internships 2016-17 Career Center, SAO, HKUST 1 Dec 2016 Agenda Mailing lists subscription Overseas Internship Programs (summer and year-round) Sponsorship Schemes Things

Conversions among Fractions, Decimals, and Percents

Conversions among Fractions, Decimals, and Percents Objectives To reinforce the use of a data table; and to reinforce renaming fractions as percents using a calculator and renaming decimals as percents.

Name Partner University Country Partners with THUAS Faculty Partners with Programme Student exchange Staff exchange European University of Tirana

Name Partner University Country Partners with THUAS Faculty Partners with Programme Student exchange Staff exchange European University of Tirana Albania Faculty of Management & Organisation B European

RECOGNITION OF THE PREVIOUS UNIVERSITY DEGREE

RECOGNITION OF THE PREVIOUS UNIVERSITY DEGREE You must request for the recognition of your bachelor diploma at the Czech University which provides similar study modules. If you studied in economic field,

The Achievement Gap in California: Context, Status, and Approaches for Improvement

The Achievement Gap in California: Context, Status, and Approaches for Improvement Eva L. Baker, EdD - University of California, Los Angeles, Center for Research on Evaluation, Standards, and Student Testing

The development of ECVET in Europe

European Centre for the Development of Vocational Training WORKING PAPER No 10 The development of ECVET in Europe Luxembourg: Publications Office of the European Union, 2010 The development of ECVET in

CALL FOR PARTICIPANTS

CALL FOR PARTICIPANTS TRAINING OF TRAINERS FOR EUROPEAN ERASMUS+: YOUTH IN ACTION PROJECTS 2017/18 CALL FOR PARTICIPANTS YOU HAVE...already gained experience as a trainer within the field of Non-Formal

06-07 th September 2012, Constanta Romania th Sept 2012

Cerintele actuale pentru pregatirea specialistilor din industria alimentara din Europa si strategii pentru viitorul acestei cariere - rezultate ale proiectului european fp7 Track Fast Training Requirements

Setting the Scene and Getting Inspired

Setting the Scene and Getting Inspired Inclusive Education and Schools Sheldon Shaeffer Save the Children Learning Event Inclusive Education: From Theoretical Concept to Effective Practice Bangkok, Thailand

Financiación de las instituciones europeas de educación superior. Funding of European higher education institutions. Resumen

Financiación de las instituciones europeas de educación superior Funding of European higher education institutions 1 Thomas Estermann Head of Unit Governance, Autonomy and Funding European University Association

83 Fellows certified in 2016! Currently 161 Fellows registered Global Online Fellowship In Head & Neck Surgery and Oncology

83 Fellows certified in 2016! Currently 161 Fellows registered 2017 Global Online Fellowship In Head & Neck Surgery and Oncology The Fellowship There are no current global standards of training, practice

The Conference Center. of the Americas. at the Biltmore Hotel. Miami, Florida

The Conference Center of the Americas at the Biltmore Hotel Miami, Florida March 7-8, 2001 B R I E F I N G B O O K Acknowledgments This Briefing Book was prepared by the Partnership for Educational Revitalization

Tailoring i EW-MFA (Economy-Wide Material Flow Accounting/Analysis) information and indicators

Tailoring i EW-MFA (Economy-Wide Material Flow Accounting/Analysis) information and indicators to developing Asia: increasing research capacity and stimulating policy demand for resource productivity Chika

No. 11 Educational Assessments in Latin America: Current Progress and Future Challenges by Laurence Wolff* June 1998 Laurence Wolff is an education consultant at the Inter- American Development Bank in

Current changes in Finnish academic profession Timo Aarrevaara Professor, HEGOM University of Helsinki Academic profession in Europe The academic profession is a critical part of the future of knowledge-based

L APPLICATION DE LA CONVENTION APOSTILLE AUX DIPLOMES NOTAMMENT DELIVRES PAR LES «USINES A DIPLOMES» établi par le Bureau Permanent * * *

APOSTILLE / NOTIFICATION / PREUVES / ACCES A LA JUSTICE APOSTILLE / SERVICE / EVIDENCE / ACCESS TO JUSTICE Doc. prél. No 5 Prel. Doc. No 5 décembre / December 2008 L APPLICATION DE LA CONVENTION APOSTILLE

The Junior Community in ALICE. Hans Beck for the ALICE collaboration 07/07/2017

The Junior Community in ALICE Hans Beck for the ALICE collaboration 07/07/2017 2 Contents 1. Structure of ALICE & its junior community 2. Juniors activities within their group 3. Impact of juniors beyond

Integrating Grammar in Adult TESOL Classrooms

Applied Linguistics 29/3: 456 482 ß Oxford University Press 2008 doi:10.1093/applin/amn020 Integrating Grammar in Adult TESOL Classrooms 1 SIMON BORG and 2 ANNE BURNS 1 University of Leeds, UK, 2 Macquarie

Collaborative Partnerships

Collaborative Partnerships Building Effective Sponsored Student Programs By Craig E. Hastings A publication of NAFSA: Association of International Educators, 2012 This publication is available on the NAFSA

CONSULTATION ON THE ENGLISH LANGUAGE COMPETENCY STANDARD FOR LICENSED IMMIGRATION ADVISERS Introduction Background 1. The Immigration Advisers Licensing Act 2007 (the Act) requires anyone giving advice

Analysis of European Medical Schools Teaching Programs

44(1):26-31,2003 STUDENT CMJ Analysis of European Medical s Teaching Programs Tina Dušek, Tamara Bates Zagreb University of Medicine, Zagreb, Croatia Aim. To compare the teaching programs of European medical

APPENDIX 2: TOPLINE QUESTIONNAIRE

183 PEW SOCIAL & DEMOGRAPHIC TRENDS APPENDIX 2: TOPLINE QUESTIONNAIRE PEW RESEARCH CENTER ASIAN-AMERICAN SURVEY FINAL TOPLINE JAN. 3-MARCH 27, Chinese Filipino Indian Japanese Korean Vietnamese N 3,511

In reviewing progress since 2000, this regional

United Nations Educational, Scientific and Cultural Organization EFA Global Monitoring Report 2 0 1 5 Regional overview: East Asia and the Pacific United Nations Educational, Scientific and Cultural Organization

TERTIARY EDUCATION BOOM IN EU COUNTRIES: KEY TO ENHANCING COMPETITIVENESS OR A WASTE OF RESOURCES?

Studia Ekonomiczne. Zeszyty Naukowe Uniwersytetu Ekonomicznego w Katowicach ISSN 2083-8611 Nr 271 2016 Uniwersytet Warszawski Wydział Nauk Ekonomicznych Katedra Makroekonomii i Teorii Handlu Zagranicznego