NEQMAP HKPISA. Large-scale International/Regional Assessment What we learned from PISA. 24 September 2014 UNESCO, Bankgkok. Esther Sui Chu HO Director

Transcription

1 HKPISA NEQMAP Large-scale International/Regional Assessment What we learned from PISA Esther Sui Chu HO Director HKPISA Center 24 September 2014 UNESCO, Bankgkok 1 1

2 Overview of PISA What is PISA? Outline How to build capacity to participate? How students perform? How to make Use of PISA result for policy making and educational practices Practices on analyzing PISA Items Overview of PISA assessment frameworks Have a taste of PISA items Analysis PISA items Discussion on Learning Points, Insights for national assessments & Implications for curriculum Concluding remarks

3 What is PISA? Age-based target population (15-year-olds) National samples of 150 schools with 5,000 students Two hours of testing time for each student Context questionnaires for the students, parents and schools Sample 475,000 students 65 participating countries (2012) -> 72 (2015) 3

4 Content of Assessment 3 Literacy Domains Rotating major domains Mathematical Literacy Reading Literacy Scientific Literacy R Instruments Test booklets Questionnaires ERA (2009)

5 DeSeCo (OECD, 2003) Definition and Selection of Competencies Three broad categories of key competencies 1. Using tools interactively 2. Interacting in socially heterogeneous groups 3. Acting autonomously 5

6 Key competencies for successful life and sustainable society (Adopted by HO based on Amartya Sen, 2014) 2. New way of Being Using Tools Interactively: Using language, math, science, problem solving symbols, and text, technology..etc 1. New way of Doing Act Autonomously & take responsibility Acting within the larger context; Forming and conducting life plans; and Defending and asserting one s rights, interests, limits, and needs Interact in heterogeneous groups: Relating well to others with different background; Cooperating; and Managing an resolving conflict 3. New Way of Inter-being Amartya Sen: What to do? How to live? (individually & collectively) Who to be?

7 65 Participating countries/economies (PISA 2012 Asia Pacific regions) OECD Countries Australia Hungary Poland Austria Iceland Portugal Belgium Ireland Slovak Republic Canada Israel Slovenia Chile Italy Spain Czech Republic Japan Sweden Denmark Korea Switzerland Estonia Luxembourg Turkey Finland Mexico United Kingdom France Netherlands United States Germany New Zealand Greece Norway Partner Countries (Non-OECD Countries / Regions) Albania Kazakhstan Shanghai-China Argentina Latvia Singapore Brazil Liechtenstein Thailand Bulgaria Lithuania Tunisia Chinese Taipei Macao-China United Arab Emirates Colombia Malaysia Uruguay Costa Rica Montenegro Vietnam Croatia Peru Cyprus Qatar Hong Kong-China Romania Indonesia Russian Federation Jordan Serbia 7 7

8 72 Participating countries/economies (PISA 2015 Asia Pacific regions) OECD Countries Partner Countries (Non-OECD Countries / Regions) Australia Iceland Slovak Republic Albania Indonesia Peru Austria Ireland Slovenia Algeria Jordan Qatar Belgium Israel Spain Argentina Kazakhstan Romania Canada Italy Sweden Brazil Kosovo Russian Federation Chile Japan Switzerland Bulgaria Latvia Serbia Czech Republic Korea Turkey Chinese Taipei Lebanon Singapore Denmark Luxembourg United Kingdom Colombia Lithuania Thailand Estonia Mexico United States Costa Rica Macao-China Trinidad and Tobago Finland Netherlands Croatia Malaysia Tunisia France New Zealand Dominican Republic Malta United Arab Emirates Germany Norway Former Yugoslav Republic of Macedonia Moldova Uruguay Greece Poland Georgia Montenegro Vietnam Hungary Portugal Hong Kong-China People s Republic of China

9 How to build capacity? Learning by doing Building of Professional Research Team Establishing HKPISA center Assessing the local relevancy of the assessment Design representative sampling frame Training of Test administrators (TAs) Training of Test Coders (TCs) Training of Data analysts (local and international) Reporting of results to different stakeholders Sharing in international conference, seminars and workshops

10 Research Team of HKPISA 2012 Principal Investigator Prof. Ho Sui-chu, Esther, Department of Educational Administration and Policy, CUHK Project Advisor Prof. Lo Nai-kwai, Leslie, Director of Hong Kong Institute of Educational Research, CUHK Prof. J. Douglas Willms, University of New Brunswick, Canada Project Leaders Prof. Chung Yue-ping & Prof. Tsang WK, Department of Educational Administration and Policy, CUHK Experts in Language Education Prof. Lau Kit-ling, Dinky, Department of Curriculum and Instruction, CUHK Prof. Man Ying-ling, Department of Chinese, Hong Kong Institute of Education Prof. Chun Ka-wai, Cecilia, Department of Curriculum and Instruction, CUHK Dr. Tong Choi-Wai, Quality School Improvement Project, HKIER, CUHK Prof. Man Yee-fan, Evelyn, Department of Curriculum and Instruction, CUHK 10 10

11 Research Team of HKPISA 2012 Experts in Mathematics Education and Problem Solving Prof. Au Kwok-keung, Department of Mathematics, CUHK Prof. Shiu Ling-po, Department of Educational Psychology, CUHK Prof. Tse Chi-shing, Department of Educational Psychology, CUHK Prof. Wan Yau-heng, Department of Mathematics, CUHK Dr. Lau Tai-shing, Chung Chi College, CUHK Mr. Wong Ka-lok, Faculty of Education, HKU Prof. Wong Ka-ming, Faculty of Engineering Technologies, North Glasgow College, UK Experts in Science Education Prof. Cheung Sin-pui, Derek, Department of Curriculum and Instruction, CUHK Prof. Ng Pun-hon, Department of Curriculum and Instruction, CUHK Prof. Yip Din-yan, Department of Curriculum and Instruction, CUHK Prof. Lau Kwok-chi, Department of Curriculum and Instruction, CUHK Experts in Policy Studies & Survey Prof. Chung Yue-ping, Department of Educational Administration and Policy, CUHK Prof. Lo Nai-kwai, Leslie, Director of Hong Kong Institute of Educational Research, CUHK Prof. Ho, Sui Chu, Department of Educational Administration and Policy, CUHK +IT Expert Team in

12 MANAGEMENT STRUCTURE OF THE HKPISA CENTER MANAGEMENT COMMITTEE 1. Director of HKIER 2. HKPISA Center Director 3. HKPISA Center Manager 4. Representatives of Faculty of Education 5. Representative(s) from Funding Body(ies) (2003-present) HKPISA CENTER 1. Center Director 2. Center Manager (1) 3. Research Officer (1) 4. Project Coordinator (1) 5. Research Assistant (1) 6. Project Assistant (1) SUBJECT EXPERT COMMITTEES 1. Language and reading 2. Mathematics 3. Sciences 4. Problem solving 5. IT TRANSLATORS 2 Translators 1 Verifier TEST ADMINISTRATORS (TAs) 12 TAs for trial study 33 TAs for main study TEST MARKERS (TMs) 8 TMs for trial study 24 TMs for main study SCHOOL COORDINATORS (SCs) 39 SCs for trial 148 SCs for main (1 SCs per school)

13 HKPISA2012 Sampling Explicit Strata Implicit Strata Total Number of Schools Number of Participating Schools Government High Ability 15 6 Medium Ability 8 2 Low Ability 7 2 N/A 1 0 Aided High Ability Medium Ability Low Ability N/A 1 0 Independent # Local (DSS*) International 32 3 Total # There is no implicit stratification for independent schools. *DSS refers to schools under the Direct Subsidy Scheme

14 Student Background Number of participating students Proportion (%) Grade/Form 7 / S / S / S / S / S Total Sex Female Male Total

15 OVERALL PERFORMANCE IN READING, MATH AND SCIENTIFIC LITERACY IN PAPER BASED & COMPUTER BASED ASSESSMENT (PBA & CBA)

16 PISA 2012 Top 10 countries/economies Mathematics Science Reading Countries/Regions Mean S.E. Countries/Regions Mean S.E. Countries/Regions Mean S.E. Shanghai-China 613 (3.3) Shanghai-China 580 (3.0) Shanghai-China 570 (2.9) Singapore 573 (1.3) Hong Kong-China 555 (2.6) Hong Kong-China 545 (2.8) Hong Kong-China 561 (3.2) Singapore 551 (1.5) Singapore 542 (1.4) Chinese Taipei 560 (3.3) Japan 547 (3.6) Japan 538 (3.7) Korea 554 (4.6) Finland 545 (2.2) Korea 536 (3.9) Macao-China 538 (1.0) Estonia 541 (1.9) Finland 524 (2.4) Japan 536 (3.6) Korea 538 (3.7) Ireland 523 (2.6) Liechtenstein 535 (4.0) Vietnam 528 (4.3) Chinese Taipei 523 (3.0) Switzerland 531 (3.0) Poland 526 (3.1) Canada 523 (1.9) Netherlands 523 (3.5) Canada 525 (1.9) Poland 518 (3.1) 16

17 PISA 2012 Top 10 countries/economies (Computed based assessment, CBA) CBA Problem Solving CBA Mathematics Digital Reading Countries/Economies Mean S.E. Countries/Economies Mean S.E. Countries/Economies Mean S.E. Singapore 562 (1.2) Singapore 566 (1.3) Singapore 567 (1.2) Korea 561 (4.3) Shanghai-China 562 (3.4) Korea 555 (3.6) Japan 552 (3.1) Korea 553 (4.5) Hong Kong-China 550 (3.6) Macao-China 540 (1.0) Hong Kong-China 550 (3.4) Japan 545 (3.3) Hong Kong-China 540 (3.9) Macao-China 543 (1.1) Canada 532 (2.3) Shanghai-China 536 (3.3) Japan 539 (3.3) Shanghai-China 531 (3.7) Chinese Taipei 534 (2.9) Chinese Taipei 537 (2.8) Estonia 523 (2.8) Canada 526 (2.4) Canada 523 (2.2) Australia 521 (1.7) Australia 523 (1.9) Estonia 516 (2.2) Ireland 520 (3.0) Finland 523 (2.3) Belgium 512 (2.5) Chinese Taipei 519 (3.0) 17

18 PISA 2009 vs 2012 Top 10 countries/economies (digital Reading) Digital Reading (2012) Digital Reading (2009) Countries/Economies Mean S.E. Countries/Economies Mean S.E. Singapore 567 (1.2) Korea 568 (3.0) Korea 555 (3.6) New Zealand 537 (2.3) Hong Kong-China 550 (3.6) Australia 537 (2.8) Japan 545 (3.3) Japan 519 (2.4) Canada 532 (2.3) Hong Kong-China 515 (2.6) Shanghai-China 531 (3.7) Iceland 512 (1.4) Estonia 523 (2.8) Sweden 510 (3.3) Australia 521 (1.7) Ireland 509 (2.8) Ireland 520 (3.0) Belgium 507 (2.1) Chinese Taipei 519 (3.0) Norway 500 (2.8) 18

19 EQUALITY OF GIRLS VS BOYS

20 Gender Differences (2000+ to 2012) * 數值有顯著差異 20 20

21 Jordan Qatar Thailand Malaysia Iceland United Arab Emirates Latvia Singapore Finland Sweden Bulgaria Russian Federation Albania Montenegro Lithuania Kazakhstan Norway Macao-China Slovenia Romania Poland Indonesia United States Estonia Chinese Taipei Shanghai-China Belgium Turkey Greece France Hungary Serbia Slovak Republic Vietnam Canada Netherlands OECD average Portugal Uruguay Croatia Israel Czech Republic Australia United Kingdom Switzerland Germany Argentina Denmark Mexico New Zealand Tunisia Ireland Hong Kong-China Spain Brazil Japan Korea Italy Peru Austria Liechtenstein Costa Rica Chile Luxembourg Colombia Score point difference Gender Difference in Math Literacy across countries (PISA 2012) Singapore Korea Shanghai Hong Kong Japan Macao Girls perform better Boys perform better OECD average 11 scorepoint

22 EQUALITY OF HIGH VS LOW SES (SOCIAL ECONOMIC STATUS)

23 Mathematics Mean Score Social Gradients in 2012 (Top Ten) 700 Shanghai-China Hong Kong-China Chinese Taipei Japan Macao-China Korea Singapore Finland Sweden United States United Kingdom Canada Shanghai Level Hong Kong Korea Level 5 Level Level 3 Level Index of Economic, Social and Cultural Status (ESCS) 23

24 Quality and Equality (PISA 2012) 24

25 Use of PISA results For Policy Making and Educational Practices of all stakeholders

26 Policy for Education Equality Socioeconomic Status (a) Socioeconomic Status (c) Socioeconomic Status (b) Figure 2. A social policy with positive effects could (a) raise outcome levels evenly across the SES distribution, (b) raise outcomes more for those with high SES than for those of low SES, or raise outcomes more for those low SES than for those of high SES. 26

27 Data and Evidence for Policy Makers School Teachers & Administrators Regional Reports & Result leaflets estherho

28 Leaflet for general public (HK PISA 2012)

29 HKPISA 2012 Feedback to Schools Part 1: HKPISA 2012 School Report Part 2: HKPISA School Data Enquiry System 學校數據查閱系統 (SDES)

30 For School Seminar for - -school administrator -school teachers

31

32 To inform schools -School Report

33 School Reports with statistics and analysis Literacy Performance Math, Science, Reading Questionnaire Index Self Related Cognition Drive and Motivation Engagement with and at School and School Climate

34 HKPISA School Data Enquiry System Objective Provide PISA schools with accumulated, longitudinal PISA data Support data-driven school planning and self-evaluation

35 HKPISA School Data Enquiry System Content Student factors Self-related Cognitions (e.g. self concepts, ) Personal Values & Attitudes Parent factors Involvement at home in the child s education Participation in the school Literacy Performance in Reading, Math, and Science Data from 2012, 2009, 2006, 2003, 2000+

36 Theme: Environmental Issues Index: Awareness of Environmental Issues Index: Environmental Optimism Index: Perception of Environmental Issues Index: Responsibility for Sustainable Development Theme: Literacy Performance Index: Percentage Correct for Math Index: Percentage Correct for Reading Index: Percentage Correct for Science Theme: Parental Factors Index: Cultural Possession Index: Economic, Social and Cultural Status Index: Home Educational Resources Index: Home Based Involvement Index: Material Resources Index: Parental Satisfaction Index: Parental Arrangement of Science Activities Index: School Based Involvement Theme: Science Teaching & Learning Index: Focus on Model or Application Index: Hands-On Activities Index: Interaction Index: Student Investigations Theme: Science Values & Activities Index: General Value of Science Index: Personal Value of Science Index: Science Activities Theme: Self-Related Cognitions Index: Instrumental Motivation in Science Index: General Interest in Learning Science Index: Enjoyment of Science Index: Science Self-Efficacy Index: Future-Oriented Science Motivation Index: Science Self-Concept

37 Focus group discussion with from frontline educators about PISA (Sept 2014) Principals said the feedback from HKPISA center is useful for improving the instruction of Chinese & English which match with the New Secondary School Reform (public examinations) Librarian said PISA helps to improve reading strategies of students and reading climate of schools Panel head of Science said I integrate PISA items in regular Integrated Science lessons and assessments, the open ended Qs are more challenging for students Vice-chairman of Math and Science Professional Association said, we collaborated with CDI to initiate a new science project focusing on process skills of scientific investigation based on PISA and TIMSS Math Teacher focused on the NON-cognitive outcomes of math in PISA and Math team in school plan to improve the affective and motivation aspects of learning math in school. (especially for girls) HKPISA website provide detailed information (e.g. parent background, choice and involvement over time) beyond the regular information collected from the Internal and External School Review by the government Combination of information PISA and other benchmarking from school is useful for school plan every 3 years and so on

38 Further Analysis to share with international academic community Books, Papers and Special Issue in Educational Journal, Thematic Reports

39 Multilevel Analysis of PISA Data: Insights for Education Policy and Practice Chapters Policy Issues Chapter 1 Introduction Chapter 2: Overall Quality an d Equality of Hong Kong Basic Education System from PISA to PISA 2006 Chapter 3: Characteristics of East Asian Learners: What We Learned from PISA Chapter 4: Reading Habits, Reading Attitude and Reading Performance Chapter 5. Self-related Cognition and Mathematics Performance Chapter 6. Affective Domain of Scientific Literacy and Gender Differences Chapter 7. Self Regulated Learning of Hong Kong s 15-Year Old Students Chapter 8 Effect of Parental Involvement and Investment on Reading, Mathematics and Science Chapter 9 Effect of School Decentralization and School Climate on Student Performance Chapter 10 Student Performance in Chinesemedium (CMI) and English-medium (EMI) schools Chapter 11 Critical Review of Assessment of Problem Solving in PISA and Its Implication for Curriculum Reform Chapter 12 Conclusions and Implications

40 Thematic Monograph Ho, S. C. (2012). Hong Kong Students on Line: Digital Technologies and Reading in PISA Education Policy Studies Series. Number 75. Hong Kong: Faculty of Education & HKIER, CUHK. Ho, S. C. & Man, E.Y.F. (2007). Student performance in Chinese medium of instruction and English medium of instruction schools: What we learned from the PISA study. Education Policy Studies Series. Number 64. Hong Kong: Faculty of Education & HKIER, CUHK. Ho, S. C. (2005). Can basic education system in Hong Kong be equal and excellent: Results from PISA Education Policy Studies Series. Number 57. Hong Kong: Faculty of Education & HKIER, CUHK. 何瑞珠 (2009) 從國際視域看東亞社會的影子教育 教育政策研討系列 之 72 香港 : 香港中文大學教育學院, 香港教育研究所 何瑞珠 (2009) PISA 2003 解難能力評估及啟示 教育政策研討系列 之 70 香港 : 香港中文大學教育學院, 香港教育研究所 何瑞珠 (2007) 從 PISA 看香港中學生的閱讀表現 習慣及態度 教育政策研討系列 之 66 香港 : 香港中文大學教育學院, 香港教育研究所 何瑞珠 (2004) 從 PISA 剖析香港中學生的學習策略與學習成效的關係 教育政策研討系列 之 56 香港 : 香港中文大學教育學院, 香港教育研究所 何瑞珠 (2004) 從國際視域剖析香港教育的素質與均等 教育政策研討系列 之 54 香港 : 香港中文大學教育學院, 香港教育研究所 湯才偉 (2004) 從國際學生閱讀評估計劃看本港中文科的讀文教學 教育政策研討系列 之 55 香港 : 香港中文大學教育學院, 香港教育研究

41 Nurturing Young Scholars LAM, Y.P. (2006) The Effect of Family Social Capital on Student Literacy Performance. A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Education in Education. The Chinese University of Hong Kong, (CUHK) August (In Chinese) MAK, Hok Kiu Edward( 2011). Gender differences in learning Mathematics in Hong Kong: PISA 2003 study. A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Education in Education. CUHK, August 2011 WONG, Kwan-Yin Leo ( 2012). Gender Differences in Scientific Literacy of HKPISA 2006: A Multidimensional Differential Item Functioning and Multilevel Mediation Study. A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Education in Education. CUHK, Feb 2012

42 Impact of PISA (Breakspear, 2012)

43 Impact of PISA

44 @estherho

45 Body and Heart Engagement: Learning vs Caring Challenge

46 Concluding Remarks PISA can provide data to inform if our education reform is on a right track but Hong Kong need to keep it low-stake to have reliable assessment. PISA promote the value of quality and equality of basic education that is very important PISA can be used to inform different stakeholders -Reports Each cycle (2000+ to 2012); Website Feedback to school; Conferences, Seminars, Workshops and Publications Most important PISA has been used in Hong Kong for teacher professional development so that we can improve our curriculum, pedagogies and assessment Hong Kong also use PISA to identify our weaknesses and we attempt to pursuit a balance of learning and caring climate at home and in school

47 Teachers Professional Development Seminar (over 200 teachers) How PISA measures reading/mathematical/scientific literacy of students An analysis of Hong Kong students performance in the relevant literacy domain An examination of the released assessment items

48 Pedagogical Practice Guide

49 Workshops for Local & International Scholars

50 You can find more information form HKPISA website:

51 Part II: Workshop Making use of PISA released Items 51

52 Workshop rundown About 5-6 participants in a group 45 mins - a taste of PISA items 1. answer the item (start from reading, math and science) 2. analyze the item type with the worksheet (WS - Part A) 3. coding of your answer (WS Part B) 4. Group Discussion on PISA Items and Learning points from students performance Insights for National Assessments Implications for Improvement 15 mins sharing your insights

53 Table 1: Summary of the assessment areas in PISA 2012 Reading Literacy Mathematics Literacy Science Literacy Practices: Study the Assessment Framework of PISA 2012 Definition Knowledge Domain Competencies involved The capacity of an individual to understand, use, reflect on and engage with written texts in order to achieve one s goals, to develop one s knowledge and potential, and to participate in society. The form of reading materials: Continuous texts including different kinds of prose such as narration, exposition, argumentation Non-continuous texts including graphs, forms and lists Mixed texts including both continuous and non-continuous formats Multiple texts including independent texts (same or different formats) juxtaposed for specific purposes. Types of reading task or process: Access and retrieve Integrate and interpret Reflect and evaluate Complex - e.g. finding, evaluating and integrating information from multiple electronic texts Mathematical literacy is an individual s capacity to formulate, employ and interpret mathematics in a variety of contexts. It includes reasoning mathematically and using mathematical concepts, procedures, facts and tools to describe, explain and predict phenomena. It assists individual to recognize the role that mathematics plays in the world and to make well-founded judgments and decisions needed by a constructive, engaged and reflective citizen. Clusters of relevant mathematical areas and concepts: Quantity Space and shape Change and relationships Uncertainty and data Types of mathematical task or process: Formulate (formulating situations mathematically) Employ (employing mathematical concepts, facts, procedures, and reasoning) Interpret (Interpreting, applying and evaluating mathematical outcome) The extent to which an individual: Possesses scientific knowledge and uses that knowledge to identify questions, acquire new knowledge, explain scientific phenomena and draw evidence-based conclusions about science-related issues. Understands the characteristic features of science as a form of human knowledge and enquiry. Shows awareness of how science and technology shape our material, intellectual and cultural environments. Engages in science-related issues and with the ideas of science, as a reflective citizen. Knowledge of science, such as: Physical systems Living systems Earth and space systems Technology systems Knowledge about science, such as: Scientific enquiry Scientific explanations Types of scientific task or process Identifying scientific issues Explaining scientific phenomena Using scientific evidence Context and situation The use for which the text is constructed: Personal Educational Occupational Public The area of application of mathematics, focusing on uses in relation to personal, social and global settings such as: Personal Occupational Societal Scientific The area of application of science, focusing on uses in relation to personal, social and global settings such as: Health Natural resources Environment Hazard Frontiers of science and technology Adapted from PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving and Financ Literacy (OECD 2013)

54 Group Discussion and Sharing Learning points from students performance: how students performed on those specific items and what we can conclude or learn from that? Insight for National Assessments: what could be the possible implications of those specific items for assessment? how could they be reflected or used in our national assessment (NA) or school based assessment? Implication for Curriculum and Pedagogy: what could be the possible implications of those items on designing curriculum and pedagogy of your subject?

55 Context Item Analysis for Reading Part A: Analysis of reading materials Note: Fill up the square below to indicate your selection. Personal Public Occupational Educational Text format Continuous text Non-continuous text Text type Narration Exposition Description Argumentation Persuasion Instruction Charts Forms Maps Advertisements Certificates Part B: Analysis of questions Note: Fill up the square below to indicate your selection. Question format Reading ability/ aspect to be assessed Item difficulty level Multiple choice Complex multiple choice Closed-constructed response Short response Open-constructed response Retrieving information Interpreting: Forming a broad understanding Interpreting: Developing an interpretation Reflecting on and evaluating the content of a text Reflecting on and evaluating the form of a text Level 5 Level 4 Level 3 Level 2 Level 1

56 Item Analysis for Mathematics Part A: Analysis of questions Note: Fill up the square below to indicate your selection Question type Process Domain of knowledge Application area Item difficulty level % correct Comment Complex Multiple Choice Multiple Choice Closed Constructed Response (i.e. short answer) Open Response (i.e. give explanation on something) Formulate Employ Interpret/Evaluate Quantity Space and shape Change and relationships Uncertainty and data Personal Societal Occupational Scientific Level 6 Level 5 Level 4 Level 3 Level 2 Level 1

57 Item Analysis for Science Part A: Analysis of questions Note: Fill up the square below to indicate your selection Question type True & False Multiple Choice Closed Constructed Response (i.e. short answer) Competencies Open Response (i.e. give explanation on something) Explaining phenomena scientifically Identifying scientific issues Domain of knowledge Using scientific evidence Knowledge of science: Physical systems Earth and space systems Living systems Knowledge about science: Scientific enquiry Application area Item Focus Scientific explanations Health Hazards Environment Scientific Frontiers Global Personal Social Item difficulty level Level 6 Level 5 Level 4 Level 3 Level 2 Level 1 % correct Comment

58 HKPISA Further information OECD/PISA HKPISA