Andreas Schleicher (National Speaker Series 2014) by Houston A+ Challenge

PISA 2012 Evaluating school systems to improve education

OECD EMPLOYER BRAND Playbook

Houston, 25 March 2014 Andreas Schleicher

Skills transform lives and drive economies

Increased likelihood of positive outcomes for adults with higher literacy skills (scoring at PIAAC Level 4/5 compared with those scoring at Level 1 or below) Odds ratio 3.0

2.5

2.0

1.5

1.0 Being Employed

High wages

Good to Participation in High levels of excellent health volunteer political efficacy activities

High levels of trust

Changes in the demand for skills

Trends in different tasks in occupations (United States) Mean task input in percentiles of 1960 task distribution

Routine manual

Nonroutine manual Routine cognitive

Nonroutine analytic Nonroutine interpersonal

1960

1970

1980

1990

2000

2006

2009

Source: Autor, David H. and Brendan M. Price. 2013. "The Changing Task Composition of the US Labor Market: An Update of Autor, Le vy, and Murnane (2003)." MIT Mimeograph, June.

PISA in brief •  Over half a million students… –  represen4ng 28 million 15-‐year-‐olds in 65 countries/economies

… took an interna4onally agreed 2-‐hour test… –  Goes beyond tes4ng whether students can reproduce what they were taught… … to assess students’ capacity to extrapolate from what they know and crea4vely apply their knowledge in novel situa4ons –  Mathema4cs, reading, science, problem-‐solving, ﬁnancial literacy –  Total of 390 minutes of assessment material

… and responded to ques4ons on… –  their personal background, their schools and their engagement with learning and school

•  Parents, principals and system leaders provided data on… –  school policies, prac4ces, resources and ins4tu4onal factors that help explain performance diﬀerences .

Each year OECD countries spend 200bn$ on math educa?on in school

What do 15-‐year-‐olds know… …and what can they do with what they know? Mathema?cs (2012)

High mathematics performance Mean score … Shanghai-‐China performs above this line (613)

Average performance of 15-year-olds in Mathematics

580

Singapore

570

Chinese Taipei

560

Macao-China Japan Liechtenstein Switzerland

540 530

510 500 490 480 470

Fig I.2.13

Korea

550

520

Hong Kong-China

Poland Belgium Germany Austria Slovenia New Zealand Denmark France Czech Republic LuxembourgLatvia Portugal Spain Slovak RepublicUnited States Connec?cut Hungary

MassachuseJs

Florida

Israel

Netherlands Estonia Finland Canada Viet Nam Australia Ireland United Kingdom Iceland Norway Italy Russian Fed. Lithuania Sweden Croatia

460 450 440

Greece Romania

430 420

Chile

410 … 12 countries perform below this line

Serbia Turkey Bulgaria U.A.E. Kazakhstan Thailand Malaysia Mexico

Low mathematics performance

26% of American 15-‐year-‐olds do not reach PISA Level 2 (OECD average 23%, Shanghai 4%, Japan 11%, Canada 14%, Some es?mate

long-‐term economic cost to be US$72 trillion )

High mathematics performance Singapore Chinese Taipei

Hong Kong-China

Average performance of 15-year-olds in mathematics

Korea Macao-China Japan Liechtenstein Switzerland

Strong socio-economic impact on student performance

Poland Belgium Germany Austria Slovenia New Zealand Denmark France Czech Republic LuxembourgLatvia Portugal Spain Slovak RepublicUnited States Hungary Israel Greece Romania

Chile

Netherlands Estonia Finland Canada Viet Nam Australia Ireland United Kingdom Iceland Norway Italy Russian Fed. Lithuania Sweden Croatia

Serbia Turkey Bulgaria U.A.E. Kazakhstan Thailand Malaysia Mexico

Low mathematics performance

Socially equitable distribution of learning opportunities

2012 Singapore Hong Kong-China

Chinese Taipei Korea

Japan

Switzerland Poland

Liechtenstein Estonia

Netherlands

Canada Belgium Finland Viet Nam MassachuseJs Germany Strong socio-economic Austria Australia impact on student New Zealand Denmark Slovenia Ireland Iceland Czech Rep. performance 26 24 22France 20 18 16 14 12 10 8 6 UK Latvia Luxembourg Norway Portugal Italy Russian Fed. Connec?cut US Spain Lithuania Sweden Slovak Rep. Hungary Croatia Israel Florida

Romania Bulgaria

Chile

Greece Turkey

Macao-China

Serbia

United Arab Emirates Kazakhstan Thailand Malaysia Mexico

Socially equitable distribution of learning opportunities 4 2 0

Australia Austria Belgium Canada Chile Czech Rep. Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel socio-economic Strong Italy impact on student Japan performance Korea Luxembourg Mexico Slovak Rep. Netherlands New Zealand Norway Poland Portugal Slovak Rep. Slovenia Spain Sweden Switzerland Turkey UK US

2012

Korea

Switzerland Poland

Netherlands Belgium

Germany

Japan Estonia Canada Finland

Socially equitable

Austria Australia New Zealand Denmark Ireland distribution of learning Slovenia Iceland Czech Rep. opportunities France UK Luxembourg Norway Portugal Italy US Spain Sweden Hungary Israel Greece Turkey

Chile Mexico

Australia Austria Belgium Canada Chile Czech Rep. Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Israel Italy Japan Korea Luxembourg Mexico Slovak Rep. Netherlands New Zealand Norway Poland Portugal Slovak Rep. Slovenia Spain Sweden Switzerland Turkey UK US

Korea

Switzerland Poland

Netherlands Belgium

Germany

Japan Estonia Canada Finland

Austria Australia New Zealand Denmark Ireland Slovenia Iceland Czech Rep. France UK Luxembourg Norway Portugal Italy US Spain Sweden Hungary Israel Greece Turkey

Chile Mexico

Contribution of various factors to upper secondary teacher compensation costs, per student as a percentage of GDP per capita (2004)

Salary as % of GDP/capita

Instruction time

1/teaching time

1/class size

Difference with OECD average 15

Percentage points

-5

Slovak Republic

Poland

United States

Sweden

Finland

Mexico

Ireland

Iceland

Norway

Hungary

Czech Republic

Austria

Italy

Denmark

Netherlands

France

New Zealand

United Kingdom

Australia

Japan

Greece

Germany

Luxembourg

Korea

Belgium

Switzerland

Spain

Portugal

-10

Korea

Switzerland Poland

Netherlands Belgium

Germany

Japan Estonia Canada Finland

Austria Australia New Zealand Denmark Ireland Slovenia Iceland Czech Rep. France UK Luxembourg Norway Portugal Italy US Spain Sweden Hungary Israel Greece Turkey

Chile Mexico

Shanghai

2003 - 2012

Singapore Singapore

Switzerland Poland

Netherlands Belgium

Germany

Korea

Japan Estonia Canada Finland

Austria Australia New Zealand Denmark Ireland Slovenia Iceland Czech Rep. France UK Luxembourg Norway Portugal Italy US Spain Sweden Hungary Israel Greece Turkey

Chile Mexico

The American dream of social mobility In some countries it is close to a reality

Mexico Chile Greece Norway Sweden Iceland Israel Italy United States Spain Denmark Luxembourg Australia Ireland United Kingdom Hungary Canada Finland Austria Turkey Liechtenstein Czech Republic Estonia Portugal Slovenia Slovak Republic New Zealand Germany Netherlands France Switzerland Poland Belgium Japan Macao-‐China Hong Kong-‐China Korea Singapore Chinese Taipei Shanghai-‐China

300 325 350 375 400 425 450 475 500 525 550 575 600 625 650 675

Mathematics performance by decile of social background The children of room cleaners in Shanghai outperform the children of professionals in the US

High impact on outcomes

16 16 Lessons from high performers

Must haves

Quick wins

Catching up with the top-‐performers Low feasibility

High feasibility

Money pits

Low hanging fruits

High impact on outcomes

17 17

Quick wins

Must haves

Lessons from high performers

Commitment to universal achievement Capacity at point of delivery

Resources where they yield most Gateways, instruc?onal systems

Coherence Low feasibility

A learning system High feasibility

Incen?ve structures and accountability

Money pits

Low hanging fruits

High impact on outcomes

18 18 Lessons from high performers

❒

Quick wins Must htaves A commitment o educa?on and the belief that Commitment to universal competencies can be learned and atchievement herefore a ll children can achieve Capacity

educa?onal standards and Resources personaliza?on as at Universal point of delivery yield most the approach to heterogeneity where in the tshey tudent body… … as opposed to a belief that students have different Gateways, instruc?onal des?na?ons to be met with different expecta?ons, and systems selec?on/stra?fica?on as the approach to Coherence heterogeneity A learning system l  Clear ar?cula?on who is responsible for ensuring Low feasibility High feasibility student success and to whom l

Incen?ve structures and accountability

Money pits

Low hanging fruits

Countries where students have stronger beliefs in their abilities perform better in mathematics

Fig III.4.5

OECD average

650

Mean mathematics performance

600

550

500

450

400

350

300 -0.60

Shanghai-China

Singapore Hong Kong-China Korea RÂ˛ = 0.36 Chinese Taipei Macao-China Japan Switzerland Netherlands Estonia Canada Liechtenstein Finland Germany Poland Belgium Viet Nam Slovenia Denmark New Zealand Latvia Portugal Italy Austria Australia Russian Fed. Hungary Luxembourg Croatia Slovak RepublicSpain Greece Israel Norway Turkey Sweden Serbia Czech Republic Lithuania U.A.E. Iceland Romania United Kingdom Thailand Malaysia United States Ireland Bulgaria Kazakhstan Chile Montenegro France Costa Rica Mexico Uruguay Albania Brazil Argentina Tunisia Colombia Qatar Jordan Indonesia Peru

-0.40

-0.20

0.00

0.20

0.40

0.60

Mean index of mathematics self-efficacy

0.80

1.00

1.20

Perceived self-responsibility for failure in mathematics

Fig III.3.6

Percentage of students who reported "agree" or "strongly agree" with the following statements: France

Shanghai-China

OECD average

Sometimes I am just unlucky The teacher did not get students interested in the material Sometimes the course material is too hard

This week I made bad guesses on the quiz My teacher did not explain the concepts well this week Iâ&#x20AC;&#x2122;m not very good at solving mathematics problems 0

100

The parent factor Students whose parents have high educa4onal expecta4ons for them tend to report more perseverance, greater intrinsic mo?va?on to learn mathema?cs, and more conﬁdence in their own ability to solve mathema?cs problems than students of similar background and academic performance, whose parents hold less ambi?ous expecta?ons for them.

Parentsâ&#x20AC;&#x2122; high expectations can nurture studentsâ&#x20AC;&#x2122; enjoyment in learning mathematics

Fig III.6.11

Change in the index of intrinsic motivation to learn mathematics that is associated with parents expecting the child to complete a university degree

0.50 0.45

0.35 0.30 0.25 0.20 0.15 0.10 0.05

Germany

Mexico

Macao-China

Croatia

Hungary

Portugal

Chile

Hong Kong-China

Italy

Korea

0.00

Belgium (Flemish)

Mean index change

0.40

High impact on outcomes

23 23

Quick wins

Must haves

Lessons from high performers

Commitment to universal achievement ❒

Clear ambi?ous goals that are shared across the

Capacity system and aligned wResources ith high s takes gateways and at point of delivery where they yield most instruc?onal systems l

Coherence Low feasibility

l

Well established delivery chain through instruc?onal which Gateways, curricular goals translate into instruc?onal systemss ystems, instruc?onal prac?ces and student learning (intended, implemented and achieved) A learning system High level of metacogni?ve content of instruc?on … High feasibility

Incen?ve structures and accountability

Money pits

Low hanging fruits

High impact on outcomes

❒ 24Capacity at the point of delivery Must haves

Lessons from high performers

l

l  l

Quick wins

AJrac?ng, developing and retaining high quality Commitment universal achievement teachers and school leaders and tao w ork organisa?on in which they can use their poten?al Capacity Instruc?onal leadership and human resource Resources at point of delivery management in schools where they yield most Keeping teaching an aJrac?ve profession Gateways, instruc?onal System-‐wide career development … systems Coherence A learning system

Low feasibility

High feasibility

Incen?ve structures and accountability

Money pits

Low hanging fruits

High impact on outcomes

25 25 Lessons from high performers

❒

Must haves

Quick wins

Incen?ves, accountability, knowledge management

Commitment to universal achievement Aligned incen?ve structures For students Capacity Resources gateways clarity and nature of the incen?ves at point oHow f delivery aﬀect the strength, direc?on, opera?ng on students at each where stage of tthey heir eyduca?on ield most l

l

l  l

Degree to which students have incen?ves to take tough courses and study hard Gateways, Opportunity costs for staying in school and performing well instruc?onal

systems For teachers Coherence Make i nnova?ons in pedagogy and/or organisa?on A learning system l  l

Low feasibility

l  l

l

Improve their own performance and the performance of their colleagues Pursue professional development opportuni?es that lead to stronger pedagogical prac?ces

High feasibility

Incen?ve structures and A balance between ver?cal and lateral accountability accountability Eﬀec?ve instruments to manage and share knowledge and spread innova?on – communica?on within the system and with stakeholders around it Money pits Low A capable centre with authority and legi?macy to hanging act fruits

26 26

Countries that grant schools autonomy over curricula and assessments tend to perform be`er in mathema4cs

Lessons from high performers

650

Fig IV.1.15

Shanghai-China

Mathematics performance (score points)

600

550

500

450

400

Chinese Taipei

Viet Nam

Korea

Estonia

Singapore

Hong Kong-China Japan

Poland Latvia Slovenia Czech Rep. Belgium Switzerland Canada Germany Finland New Zealand Lithuania Portugal Hungary Austria Croatia Italy Spain Serbia France Australia Macao-China Turkey Norway Iceland Denmark Slovak Rep. Bulgaria Greece Romania Kazakhstan Israel Malaysia Chile Uruguay USA Sweden Jordan Costa Rica Indonesia Brazil Luxembourg Tunisia Albania Colombia UAE Argentina Peru

Netherlands UK Thailand R² = 0.13

350

Qatar 300 -1.5

-1

-0.5

0.5

Index of school responsibility for curriculum and assessment (index points)

1.5

Schools with more autonomy perform better than schools with less autonomy in systems with standardised math policies

Fig IV.1.16

School autonomy for curriculum and assessment x system's extent of implementing a standardised math policy (e.g. curriculum and instructional materials)

Score points 485

480

475

470

465 460

Standardised math policy

455

No standardised math policy

Less school autonomy More school autonomy

Schools with more autonomy perform better than schools with less autonomy in systems with more collaboration

School autonomy for resource allocation x System's level of teachers participating in school management Across all participating countries and economies Score points 485

480

475

470

465

460

Teachers participate in management

455

Teachers don't participate in management

Less school autonomy More school autonomy

Fig IV.1.17

Schools with more autonomy perform better than schools with less autonomy in systems with more accountability arrangements

Fig IV.1.16

School autonomy for curriculum and assessment x system's level of posting achievement data publicly

Score points 478 476 474 472 470 468 466

School data public

464

School data not public

Less school autonomy More school autonomy

Quality assurance and school improvement

Fig IV.4.14

Percentage of students in schools whose principal reported that their schools have the following for quality assurance and improvement: Singapore

OECD average

Implementation of a standardised policy for mathematics Regular consultation with one or more experts over a period of at least six months with the aim of improving Teacher mentoring Written feedback from students (e.g. regarding lessons, teachers or resources) External evaluation Internal evaluation/self-evaluation Systematic recording of data, including teacher and student attendance and graduation rates, test results and Written specification of student-performance standards Written specification of the school's curriculum and educational goals 0

100

-50

Chinese Taipei Hong Kong-China Thailand Viet Nam Luxembourg Switzerland Indonesia Italy Kazakhstan Japan Czech Republic Netherlands Estonia Albania Ireland United States Hungary Sweden Korea United Kingdom Finland Denmark OECD average France Shanghai-China Australia Spain Slovak Republic Mexico Germany Austria Colombia Chile Canada Poland Jordan Argentina United Arab Emirates Portugal Peru Costa Rica Brazil New Zealand Malaysia Slovenia Uruguay Qatar

Score-point difference

Differences in mathematics performance between private and public schools shrink considerably after accounting for socio-economic status

Fig IV.1.19

Observed performance difference

After accounting for studentsâ&#x20AC;&#x2122; and schoolsâ&#x20AC;&#x2122; socio-economic status

Performance advantage of public schools

-25

Performance advantage of private schools

-75

-100

-125

School competition and mathematics performance

Fig IV.1.18

Adjusted by per capita GDP 650

Shanghai-China

There is no relationship between the prevalence of competition and overall performance level

Mathematics performance (score points)

600

Viet Nam

Korea

550 Estonia Germany Slovenia Portugal

Poland

Switzerland

Finland

500 Norway

Iceland

450

Montenegro

400

RÂ˛ = 0.030

Japan

Chinese Taipei Hong Kong-China

Netherlands Latvia Singapore Czech Rep. Belgium Lithuania New Zealand Slovak Rep. Spain Italy France Serbia Macao-China Ireland Hungary Romania Austria UK Bulgaria USA Sweden Australia Turkey Thailand Greece Chile Uruguay Kazakhstan Malaysia Jordan Costa Rica Mexico Argentina Albania Brazil Tunisia Indonesia UAE Luxembourg Colombia Peru

350

Qatar 300 30

Percentage of students in schools that compete with at least one other school

100

High impact on outcomes

33 33 Lessons from high performers

Must haves

Quick wins

to universal achievement ❒  Commitment Inves?ng resources where they can make most

of a diﬀerence

Capacity Resources l  Alignment of resources with key c hallenges (e.g. at point of delivery where they ytield mostt o the most aJrac?ng the most talented eachers challenging classrooms) Gateways, instruc?onal l  Eﬀec?ve spending choices that priori?se high quality systems teachers over smaller classes Coherence A learning system Low feasibility

High feasibility

Incen?ve structures and accountability

Money pits

Low hanging fruits

Countries with better performance in mathematics tend to allocate educational resources more equitably 700

Adjusted by per capita GDP

650

Mathematics performance (score points)

Fig IV.1.11

30% of the varia?on in math performance across OECD countries is 600 explained by the degree of similarity of educa?onal resources between 550advantaged and disadvantaged schools 500

450

Mexico Costa Rica

400

Shanghai-China

Chinese Taipei Korea R² = 0.19 Viet Nam Singapore Hong Kong-China Estonia Japan Poland Slovenia Switzerland Latvia Finland Belgium Canada Germany Macao-China Slovak Rep. New Zealand IrelandIceland France Austria UK Spain Denmark Australia Israel Croatia Hungary Romania Sweden Bulgaria Portugal USA Turkey Greece Norway Italy Serbia Thailand Malaysia Chile Uruguay Kazakhstan Jordan Brazil Indonesia UAE Montenegro Colombia Tunisia Argentina Luxembourg

Peru 350

Qatar 300 1.5

Less equity

0.5

OECD countries tend to allocate at least an equal, if not a larger, number of teachers per student to disadvantaged schools; but disadvantaged schools tend to have great difficulty in attracting -0.5 qualified teachers.0

Equity in resource allocation (index points)

Greater equity

High impact on outcomes

35 35

Quick wins

Must haves

Lessons from high performers

Commitment to universal achievement Capacity at point of delivery ❒

A learning system l

Coherence l

Low feasibility

Resources where they yield most

Gateways, instruc?onal An outward orienta?on to keep the system learning, technology, interna?onal benchmarks systems as the ‘ eyes’ and ‘ears’ of the system A learning system Recognising challenges and poten?al future threats to current success, learning from them, designing High feasibility responses and implemen?ng these Incen?ve structures and accountability

Money pits

Low hanging fruits

High impact on outcomes

36 36

Quick wins

Must haves

Lessons from high performers

Commitment to universal achievement

❒

Capacity at point of delivery

Resources where they yield most

Coherence of policies and prac?ces Alignment of policies across all aspects of the system Coherence l  Coherence of policies over sustained periods of ?me Low feasibility l  Consistency of implementa?on l  Fidelity of implementa?on (without excessive control) l

Money pits

Gateways, instruc?onal systems A learning system High feasibility

Incen?ve structures and accountability

Low hanging fruits

High impact on outcomes

37 37

Quick wins

Must haves

Lessons from high performers

Commitment to universal achievement Capacity at point of delivery

Resources where they yield most Gateways, instruc?onal systems

Coherence Low feasibility

A learning system High feasibility

Incen?ve structures and accountability

Money pits

Low hanging fruits

Find out more about PISA at www.pisa.oecd.org •  All na?onal and interna?onal publica?ons •  The complete micro-‐level database

Thank you !

Email: Andreas.Schleicher@OECD.org TwiJer: SchleicherEDU

and remember: Without data, you are just another person with an opinion