PISA-based OECD Test for Schools: A Blueprint for School Improvement

NWEA.org 2016 Northwest Evaluation Association. Partnering to help all kids learn PISA-based OECD Test for Schools: A Blueprint for School Improvement Presented by Dr. Terri L. Breeden, Shaker Heights City SD and Rob Johnson, NWEA Regional Manager 2016 Northwest Evaluation Association. NWEA.org Partnering to help all kids learn

Thoughtful Questions to Consider about our Schools OECD: Organisation for Economic CO-operation and Development PISA

Know how students feel about their own learning, their confidence for learning and how their belief systems have a direct correlation to their performance?

Know where your school fits nationally and internationally regarding students processing, analyzing and thinking?

5 PISA in brief Over half a million students representing 28 million 15-year-olds in 72 countries/economies took an internationally agreed upon 2-hour test Goes beyond testing whether students can reproduce what they were taught Mathematics, reading, science, problem-solving, financial literacy and responded to a 30 minute questionnaire on their personal background, their schools and their engagement with learning and school Parents, principals and system leaders provided data on school policies, practices, resources and institutional factors that help explain performance differences.

6 The structure of the PISA assessment 2000 2003 2006 2009 2012 2015 Reading Reading Reading Reading Reading Reading Mathematics Mathematics Mathematics Mathematics Mathematics Mathematics Science Science Science Science Science Science Problem Solving Digital Reading Problem Solving Collaborative Problem Solving

PISA helps Countries Learn The tests are designed to assess to what extent students at the end of compulsory education, can apply their knowledge to real-life situations and be equipped for full participation in society.

PISA-based OECD Test for Schools Requested by schools Spearheaded by America Achieves Developed with grant funds 2012 Paper/Pencil pilot started Taken by over 400 US schools

Includes International PISA PISA-Based OECD Test for Schools Utilizes the common PISA scale Administration Country level; online via thumb drive School-level & Online Sample size at each school 35 students No more than 85 students Test intervals Every three years On-demand Includes student survey questions on engagement, school culture and climate Country level comparable data to Participating PISA countries comparison data School level reporting None 159-Page reports for schools

OECD Test for Schools (based on PISA) Participation Through 2015 In The United States 5 31 34 5 8 1 6 8 16 2 4 15 10 20 6 12 5 21 48 40 1 14 3 1 10 2 50 2 1 1-9 schools 10-19 schools 1 53 > 20 schools 12

What are schools learning from the OECD Test for Schools?

Each School Receives Actionable Information LEADING COMPANY 159 page report for each school Detailed analyses of how your school(s) compare on multiple measures- math, reading, science Not just scores, but context including climate and culture

14 Overview of Results Report Student Engagement and the Learning Environment at Your School Disciplinary climate Teacher/student relations Students reading habits Instrumental motivation Self-efficacy Your School Compared With Similar Schools in Your Country Your School s Results in an International Context

Below International Baseline Medium-High Performers Your school Top Performers Internationally US average

Are my students critical thinkers? Are they able to conceptualize, generalize, utilize and link information sources and flexibly translate between them? OECD VS Consortia Proficiency Levels Comparison.xlsx

OECD Sample Test Item

2016 Northwest Evaluation Association. NWEA.org Partnering to help all kids learn

Sample Question: Science Reverse Osmosis One way of producing drinking water from seawater is by reverse osmosis. Reverse osmosis is a type of filtration. Seawater is pushed through a semi-permeable membrane. Pressure is applied to the seawater. Semi-permeable means salt is trapped on one side of the membrane, but water can pass through. The trapped salts form a seawater concentrate on one side of the membrane. Question Where do the trapped salts go? A. Ocean B. Storage tank C. Delivery pipeline D.Water distribution system See more sample questions at: www.oecd.org/pisa/pisabasedtestforschools.

Why do schools care? Evolution of employment in occupational groups defined by PIAAC problem-solving skills % 25 20 15 10 5 0 Employment of workers with advanced problem-solving skills Employment of workers with poor problem-solving skills -5-10 Employment of workers with medium-low problemsolving skills (PIAAC) -15-20 Source:PIAAC 2011

2016 Northwest Evaluation Association. NWEA.org Partnering to help all kids learn

How We Compare to our Peers Nationally

2016 Northwest Evaluation Association. NWEA.org Partnering to help all kids learn

Exploring Reader Profiles How is your reading Instruction addressing the deep and wide?

What types of readers are my students?

Comparison of Reader Profiles

2016 Northwest Evaluation Association. NWEA.org Partnering to help all kids learn

What Students Say It challenged my mind The test was simple, and the navigating through them was easy. The test to solve real-world problems that I felt like were relevant. also had some interesting articles and facts I enjoyed reading Some questions allowed for a broad typed out open-ended answer. This allowed me to better portray my thoughts through each answer.

How can we learn from other schools? Access to the Global Learning Network Facilitated by America Achieves Participate in Virtual, Regional and National Convenings Engage in peer-to-peer learning

GOALS FOR THE CONVENING OF WORLD-LEADING SCHOOLS Create a sense Participants Build of community Improve identify understanding of how this among high- participants concrete work fits in achieving and understanding actions that with other committed of their OECD will further this efforts to individuals to Test for work in their improve establish a Schools results schools and student peer network districts outcomes globallearningnetwork.americaachieves.org

Let s hear from a school

Mean score 580 570 560 Singapore Hong Kong-China Chinese Taipei Korea 550 High mathematics performance Shanghai-China performs above this line (613) Average performance of 15-year-olds in Mathematics Fig I.2.13 540 Macao-China Japan Liechtenstein 530 Switzerland 520 510 Massachusetts Germany Poland Belgium Netherlands Estonia Canada Viet Nam Finland 500 490 New Zealand Czech Republic Luxembourg Austria Slovenia Denmark France Latvia Australia Ireland United Kingdom Iceland Norway 480 Slovak Republic Portugal Connecticut Spain United States Hungary Italy Russian Fed. Lithuania Sweden 470 Florida Israel Croatia 460 26% of American 15-year- 450 Greece Romania Serbia Turkey olds do not reach PISA 440 430 420 Chile Bulgaria U.A.E. Kazakhstan Thailand Malaysia Level 2 (OECD average 23%, Shanghai 4%, Japan 11%, Canada 14%, Some estimate long-term economic cost to be US$72 trillion ) US 410 12 countries perform below this line Mexico Low mathematics performance

Are my students engaged and motivated? North Star Academy, Newark NJ Math & Science below average Student interest in math & science below average Revamped curriculum One year later: >50% of seniors planned to major in STEM http://hepg.org/hel- home/issues/30_3/helarticle/quietly,-a-new-test-gains-

Thank You Questions?