THE ECONOMIC BENEFITS OF CLOSING THE STUDENT ACHIEVEMENT GAP IN BULGARIA
2012 EDUCATION Conference: Equal Access to Quality Education 25 September 2012 Sofia, Bulgaria
1|Pa ge
Teach For Bulgaria (TFB) is a non-profit organization whose mission is to ensure that every child has equal access to quality education regardless of which school they attend, where they live, or their parents’ financial resources. To accomplish this mission, TFB recruits high-caliber young graduates and professionals to work as teachers for two years. In their classrooms, these teachers dramatically impact their students’ lives by improving the quality of education they receive. Additionally, TFB helps the teachers develop their leadership skills and pursue their desired career path after the program. TFB believes that the young leaders we recruit will lead a fundamental educational change – starting with the two years at school and then continuing as alumni who work towards achieving our mission from the various professional fields they join. This model has been proven successful internationally in over twenty countries, including the United States and the United Kingdom (where it has been implemented for 20 and 10 years respectively).
The TFB team consists of 13 staff members from diverse educational and professional backgrounds. TFB operates in 18 municipalities across central and Southwest Bulgaria with the hope to expand across the country. Currently, TFB has 55 teachers placed in 31 schools. Teach For Bulgaria is a member of the Teach For All network along with twenty four other country organizations around the world who share the same goals, methods and best practices.
This report was prepared on behalf of Teach For Bulgaria by Peter Stoyanov, freelance economist. Evgeni Kanev, Ph.D. (Maconis LLC) and Trayan Trayanov (Teach For Bulgaria) provided invaluable advice and comments.
2|Pa ge
CONTENTS
Contents ..........................................................................................................................................................3 Figures and Tables ...........................................................................................................................................4 1.
Introduction ............................................................................................................................................5
2.
Summary of findings ...............................................................................................................................6
3.
Methodology...........................................................................................................................................8
4.
5.
3.1.
Conceptual framework ...................................................................................................................8
3.2.
Technical overview of the estimates ..............................................................................................9
3.2.1.
The projection methodology .................................................................................................9
3.2.2.
Assumptions.........................................................................................................................10
3.2.3.
How to interpret the projections .........................................................................................11
Dimensions of the education gap in Bulgaria: some preliminaries.......................................................13 4.1.
Quantity does not mean quality ..................................................................................................13
4.2.
What is PISA?................................................................................................................................13
4.3.
Choosing a peer group .................................................................................................................14
Dimensions of the education gap in Bulgaria: Estimates ......................................................................16 5.1.
5.1.1.
Dimensions of the gap .........................................................................................................16
5.1.2.
Can the gap be closed? ........................................................................................................21
5.1.3.
Estimated benefits of closing the gap ..................................................................................21
5.2.
The Language Gap ........................................................................................................................22
5.2.1.
Dimensions of the gap .........................................................................................................22
5.2.2.
Can the gap be closed? ........................................................................................................24
5.2.3.
Estimated benefits from closing the gap .............................................................................24
5.3.
6.
The international gap ...................................................................................................................16
The School Community Type Gap ................................................................................................26
5.3.1.
Dimensions of the gap .........................................................................................................26
5.3.2.
Can the gap be closed? ........................................................................................................27
5.3.3.
Estimated benefits from closing the gap .............................................................................29
Note on accuracy & recommendationS for further research ...............................................................30 Call for further research ............................................................................................................................31
7.
References.............................................................................................................................................32
8.
Glossary of term / key concepts ...........................................................................................................33
3|Pa ge
FIGURES AND TABLES
Figure 1: Education quantity vs. quality: PISA 2009 test scores and average years of schooling .................14 Figure 2: Average PISA 2009 scores, EU and peer group countries ..............................................................17 Figure 3: The importance of the economic, social and cultural status gradient in explaining mathematics scores ............................................................................................................................................................17 Figure 4: Resilient students and disadvantaged low achievers .....................................................................18 Figure 5: Distribution by proficiency levels (mathematics, PISA 2009).........................................................20 Figure 6: Between- and within-school variance in reading performance .....................................................21 Figure 7: Closing the international gap: Impact of the reform on GDP (in million 2011 BGN) ....................22 Figure 8: Closing the international gap: Impact of the reform on the real growth rate ..............................22 Figure 9: The difference in mean reading scores of students based on language spoken at home .............23 Figure 10: Closing the language gap: Impact of the reform on GDP (in million 2011 BGN) ........................25 Figure 11: Closing the language gap: Impact of the reform on the real growth rate...................................25 Figure 12: Mathematics mean score and 95% confidence intervals, 2009: Bulgaria vs. OECD ....................28 Figure 13: Science mean score and 95% confidence intervals, 2009: Bulgaria vs. OECD .............................28 Figure 14: Mathematics mean score and 95% confidence intervals, 2009: Hungary and Turkey.................28 Figure 15: Science mean score and 95% confidence intervals, 2009: Hungary and Turkey..........................28 Figure 16: Mathematics mean score and 95% confidence intervals, 2009: Greece and Serbia ...................28 Figure 17: Science mean score and 95% confidence intervals, 2009: Greece and Serbia ............................28 Figure 18: Closing the school community gap: Impact of the reform on GDP (in million 2011 BGN) .........29 Figure 19: Closing the school communitygap: Impact of the reform on the real growth rate ....................29
4|Pa ge
1. INTRODUCTION
It is widely recognised in economic literature that the quality of a country’s labour force (a.k.a. the country’s human capital) is of crucial importance to its wellbeing. A number of important questions pertaining to human capital are discussed and debated in this study: why are there differences in the distribution of human capital both among countries and within countries; how can these differences be closed; how much do we stand to gain if they are closed; and alternatively, how much are we foregoing by not addressing the issue. In this report, we set out to underline the importance of educational outcomes to human capital formation and distribution, as well as the potential measurable economic benefits from improving these outcomes. Without a doubt, the size of the potential benefits to be enjoyed is simply enormous, and they could far exceed any conceivable costs of implementation. We also underline the fact that the investment in education is a long-term policy, as (a) it takes a long time to reform the education system, and (b) once reformed, the benefits take a long time to trickle down to the economy. Using individual-level data from the 2009 PISA tests, we provide a measure of the size of underachievement of three specific groups of students, corresponding to the three major achievement gaps which could be addressed by policy. We also apply a widely accepted simulation framework to estimate the potential benefits to the country if these gaps are closed. The three gaps we discuss are: (i)
(ii) (iii)
The gap in performance on cognitive ability assessments between Bulgarian students and students in developed countries (using the OECD as the benchmark); The gap in performance between students who speak mostly Bulgarian at home and those who do not; The gap in performance between schools based on size of the community where the school is located.
In this report, we sidestep the issue of whether, how, or at what cost the problems can be fixed. We do, however, present our estimate of the potential benefits, and hope that by showing the vast size of the potential gains from reform, this can serve as a starting point for a discussion. We address in more detail some of the constraints of our work in the last section of the report.
5|Pa ge
2. SUMMARY OF FINDINGS
This report provides an estimate of the potential benefits that the Bulgarian economy could enjoy if three types of educational achievement gaps are closed. We use data from the PISA’2009 tests to quantify the respective gap, and an established simulation procedure to estimate the potential future benefits. Our main results are as follows. More details are available in Table 1 and the main text of the report. The international achievement gap: We find that Bulgarian 15year-olds1 lag significantly behind their OECD peers. This is the largest gap of the three discussed. If this gap were closed, Bulgaria could increase its long-term GDP annual growth rate by 1.09 percentage points. The language gap: Students who do not speak Bulgarian at home lag significantly behind their peers who do speak Bulgarian at home. If this gap were closed, Bulgaria could increase its longterm annual growth rate by 0.2 percentage points, the smallest of the three in terms of economic impact and number of students affected (13.6%). The school community type gap: In Bulgaria, 15-year-olds living in villages significantly under-perform compared to those living in small towns and towns, and they, in turn, significantly underperform compared to 15-year-olds living in cities and in Sofia, Bulgaria’s capital. If this gap were closed, Bulgaria could increase its long-term annual growth rate by 0.73 percentage points. Closing the gap would affect to some extent around 77% of the 15-year-olds in the country. There are other internal gaps that are important, but are not discussed here: gaps based on socio-economic and family status, and gaps by type of school (public vs. private, general schools vs. professional schools, etc.). The reader should keep in mind that, in our exercise, closing the international gap involves an increase of the national average scores only, and no distributional consequences are implied or analysed. In the case of the language gap and the school community gap, the increase in the national average comes from bringing the average of some students to the level of their betterperforming peers. In addition to the estimated economic benefits, such intervention has important non-economic implications and moral dimensions, as it implies a redistribution of resources and opportunities.
1
The term ’15-year-olds’ is used interchangeably with ‘students’ throughout the report.
6|Pa ge
Table 1: Summary of benefits from closing the achievement gaps
The international gap:
Closing this gap increases the national mean score by 58.7 PISA points, affecting 100% of the 15-year-olds.
Increase in annual growth rate by the end of the phase (percentage points) Value of the reform from 2012 until the end of phase, as % of the 2012 GDP Percentage addition to GDP due to the reform (*)
end Phase 1 2027
end Phase 2 2052
end Phase 3 2067
end Phase 4 2092
0.22 4.62 1.23
0.90 122.21 16.57
1.09 315.23 35.82
1.09 828.34 77.61
The language gap:
Closing this gap increases the national mean score by 10.9 PISA points and affects 13.6% of the 15-year-olds.
Increase in the annual growth rate by the end of the phase (percentage points) Value of the reform from 2012 until the end of phase, as % of the 2012 GDP Percentage addition to GDP due to the reform (*)
end Phase 1 2027
end Phase 2 2052
end Phase 3 2067
end Phase 4 2092
0.04
0.17
0.20
0.20
0.86 0.23
21.97 2.90
54.61 5.89
133.70 11.34
The school community type gap:
Closing this gap increases the national mean score by 39.2 PISA points and affects 77% of the 15-year-olds.
Increase in the annual growth rate by the end of the phase (percentage points) Value of the reform from 2012 until the end of phase, as % of the 2012 GDP Percentage addition to GDP due to the reform (*)
end Phase 1 2027
end Phase 2 2052
end Phase 3 2067
end Phase 4 2092
0.15
0.60
0.73
0.73
3.08 80.36 204.12 520.57 0.82 10.78 22.70 46.79 (*) Calculated as (GDP with reform / GDP without reform) – 1 NB: The gaps should be viewed as independent of each other. The benefits should not be compared or added to each other. Details on the calculations are available later in the text. Source: Author’s calculations, based on Hanushek and Woessmann (2011) and data from PISA 2009.
7|Pa ge
3. METHODOLOGY
The calculation of potential economic benefits in this report is based on the research of Eric Hanushek 2 and Ludger Woessmann3, published in several academic papers (please see the bibliography for concrete references). The methodology is widely accepted and used, including in the official OECD reports on the PISA surveys. This section presents a brief overview of the conceptual framework and technical aspects of the calculation.
3.1.
C ONCEPTUAL
FRAMEWORK
4
The process of economic growth has always been one of the central issues discussed by economists, and the past couple of decades have brought a successful marriage of economic theory with empirical work. This report is based on the work done by Hanushek and Woessmann, who, like many others, concentrate on the role of human capital. In particular, they model a country’s growth rate as a function of the skills of workers (human capital) and other factors (traditionally including initial levels of income and technology, economic institutions and other systematic factors). The underlying model combines features of neoclassical growth models and endogenous growth models. In particular, one central feature of the model is that the impact of improved human capital may have a permanent effect on the rate of economic growth, i.e. the impact will not dissipate over time, bringing the growth rate down to its initial level. Hanushek and Woessmann focus on quality rather than quantity of education
The main contribution of Hanushek and Woessmann is shifting the focus from the traditional measures of human capital (proxied by some measure of the quantity of education, such as attainment) to a measure which focuses on the quality of education (cognitive skills, in their terminology). They develop a measure of cognitive skills by combining the results of 12 different international tests of math, science and reading, administered between 1964 and 2003 and covering a variety of countries. The results of these surveys are used to construct a measure of average cognitive skills at the national level, which the authors interpret (under certain assumptions) as measures of the human capital of the population and workforce of the respective country. The next step of the analysis links the constructed measure of cognitive skills5 to long-term economic performance. The main 2
Hoover Institution, Stanford University, CESifo, and NBER Department of Economics, University of Munich, Ifo Institute for Economic Research, CESifo, and IZA 4 The presentation in this section is based on Hanushek and Woessmann (2011) 5 Cognitive skills in this context include math and science skills, as it is easier to test for a common set of expected skills in these disciplines 3
8|Pa ge
model (used in our report) links the average annual growth rate of the PPP GDP per capita from 1960 to 2000 to cognitive skills, initial years of schooling, and initial GDP per capita in 24 OECD countries. The estimated equation is: ℎ = +1.864 ∗
+ 0.046 ∗ ℎ − 0.303 ∗
where the coefficients on
and are statistically significant while the coefficient on ℎ is not. The estimates above conform to the usual thinking – a higherquality workforce increases your growth rate (positive coefficient), as does the initial level of education (in terms of quantity), while in countries which were rich initially, the addition to the growth rate is smaller (they grow more slowly than countries which started out as poor). We use the estimated coefficient for cognitive skills (the measure of which corresponds to PISA points) in our modelling, implying that an increase of one standard deviation (100 PISA points) yields an average of 1.864 percentage points higher (annual) longterm economic growth rate.
3.2.
T ECHNICAL
OVERVIEW OF THE ESTIMATES
We use a two-step approach. In the first step, we describe and quantify the respective gap and ‘translate’ it to a change in the mean PISA score for Bulgaria. The second step replicates the estimating methodology used by Hanushek and Woessmann in their work on OECD and European Union countries – we assume that a reform is implemented such that the mean score of the country is raised by the respective number of points on the PISA scale, i.e. the education gap is closed by the end of the reform. The projection described below is then used to estimate the expected future benefits of the improved skillset of the population (and workforce). We also attempt to provide (in the respective sections below) background information on whether other countries have managed to close similar gaps, but we would like to stress again that the duration of the reform is chosen for illustrative purposes, rather than relying on a strict feasibility analysis.
3.2.1. The projection methodology The methodology is based on a forward-looking projection and measures (in present value terms) the benefits of a reform which
than in reading. See Hanushek and Woessmann (2009) for a more detailed discussion.
9|Pa ge
leads to an improvement in cognitive skills (i.e. increase in the country’s mean score on PISA tests) that would occur over the lifetime of a child born in the year 2012, when the reform is initiated. The projection assumes a life expectancy of 80 years (which broadly corresponds to the OECD average), that the working life is 40 years, and that over the period there is no change in the structure and size of the country’s population. There are four phases that are identified on the basis of the actual economic impact of the reform: Phase 1: Implementing the reform
Phase 1, 15 years (2013-2027): During this phase, the reform is carried out. For simplicity, it is assumed that the reform happens in a linear fashion, i.e. each year 1/15thof the total is completed. During this phase, the impact on GDP growth is minimal, as the majority of the workforce has acquired skills under the prereform system, and those currently graduating have only partially benefited from the reform. The size of the impact grows, as each successive cohort leaving the school system has fractionally higher skills, until the full expected impact of the reform is realised at the end of the phase.
Phase 2: Workers educated before the start of the reform leave the workforce
Phase 2, 25 years (2028-2052): The reform is completed in full; all future members of the workforce have higher cognitive skills. Older workers with lower skills are gradually replaced by workers with higher skills. By the end of this phase, all workers who were educated before the reform have left the workforce.
Phase 3: Workers who only partially benefited from the reform (were educated while the reform was carried out) leave the workforce
Phase 3, 15 years (2053-2067): During this phase, all workers have at least partially benefited from the reform. By the end of the phase, all workers who were educated during Phase 1 (the reform period) have left the workforce.
Phase 4: The full long-term impact is achieved. The growth rate is permanently increased.
Phase 4, 25 years (2068-2092): Full, long-run growth effect is achieved; the workforce is comprised only of workers who were educated under the fully-reformed system. The structure of the projection model emphasises a very important feature of education reform – the benefits of education reform are realized in the long run. First, it takes time to reform the education system; and second, it takes time for the benefits of the reformed education system to trickle down to the actual economy.
3.2.2. Assumptions This section provides the full list of assumptions we have used in the projections. Some were mentioned in the previous section and are included here for the sake of completeness or are discussed in more detail. We assume that the reform program is completed over 15 years and that progress is linear. Since in this report we do not provide an analysis of how the reform should be carried out, and what are its instruments and channels of influence, we do not include any estimate of the cost of the reform in the projections.
10 | P a g e
The projection period is 80 years, which is the average life expectancy for the OECD in 2012.6 The average person is assumed to work for 40 years, and there are no changes in the size or structure of the population over this period.7 Following Hanushek and Woessmann, we assume that, in the absence of reform, the country will grow (in real GDP per capita terms) at a rate of 1.5% per year,8 which broadly corresponds to the long-term OECD experience. The cumulative effects of the reform are calculated as the sum of the discounted values of the difference in GDP with and without the reform over the analysed period. A 3% social discount rate is used, the same as in Hanushek and Woessmann’s work, where it is jmore or less standard in this type of analysis. Our starting point is the GDP for Bulgaria, as per the latest available data from the NSI. As of writing this report, the 2011 GDP is estimated at BGN 75,265 million, which is the base figure we use in the projections. All future GDP estimates are in 2011 BGN at unchanged purchasing power parity. All variables are in real terms, i.e. no adjustment for inflation is needed.
3.2.3. How to interpret the projections The projections in this report estimate the GDP of Bulgaria along two paths – with and without reform. The GDP without reform is the same in all cases – it is calculated using the constant 1.5% (real annual) growth rate for the whole 80-year period. The GDP with reform uses the same 1.5% base growth with an added increment coming from the education reform and its effect on the workforce. The size of the addition to the growth rate depends on the state of completeness of the reform and the extent to which the effects have trickled down to the economy, i.e. how much of the workforce has been replaced by more qualified workers. For all gaps discussed in this report, we report the following pieces of data.9
6
Life expectancy was 73.5 years in Bulgaria in 2010, having grown from 69.25 in 1960. 7 The ‘no changes in the size of the population and its structure’ assumption is unlikely to hold in the case of Bulgaria and has been made for convenience alone. Under it, the dynamics of GDP per capita are the same as those for the whole economy. A more detailed analysis of education reform would have to include a detailed and more realistic demographic projection. 8 At a first glance, 1.5% growth rate may be considered to be low for the former socialist countries, the historic experience of which is different from that of OECD countries. Hanushek and Woessmann argue that since the former socialist countries have had more than 20 years of transition to a market-based economy, their future experience should be closer to the OECD model than to their own past experience. 9 In recognizing that the reader may also want to see the near-term benefits of what is essentially a long-term program; the reporting is done in phases as well as at the end of the full 80-year period. These
11 | P a g e
First, we report the addition to the no-reform growth rate at the last year of the four phases. Note that the figure for the last year of Phase 3 includes the full impact, and the same growth rate continues throughout Phase 4. We also remind the reader that the underlying growth model is an endogenous growth model, and the reform changes the growth rate of the GDP of the economy in equilibrium, not only the level of the GDP. There are two important implications: (a) the growth rate will not go down in the future, i.e. the higher growth rate is permanent, and (b) the higher the growth rate in the no-reform case, the higher the expected benefits of the education reform as they amplify the base-case growth. Second, we report the ‘value of the reform’ for the respective period. The value of the reform is estimated as the difference between the present value of the GDP produced during the respective period with the reform, and the present value of the GDP produced during the respective period without the reform. The GDPs for future years are discounted using the (constant) social discount rate, as discussed in the assumptions section. The value of the reform thus estimated is presented in two forms – as an absolute value (in 2011 BGN) and as a percentage of Bulgaria’s GDP in the base year (2012). Third, as an illustration, we provide the percentage gain in GDP which is due to the education reform.
are interim results; the full impact is felt only at the end of Phase 3 when the growth rate reaches its new long-term value.
12 | P a g e
4. DIMENSIONS OF THE EDUCATION GAP IN BULGARIA: SOME PRELIMINARIES
4.1.
Q UANTITY
DOES NOT MEAN QUALITY
Bulgaria has made substantial progress in educating its population since 1950. Starting with 82.4% of the population with at least primary education in 1950, education attainment improved substantially to reach over 90% by 1965, and over 95% by 1985. The progress did not stop after the collapse of socialism in 1989, and the percentage of the population with at least primary education reached around 99% in 2010. Additionally, there has been a substantial increase in the share of people attaining secondary and tertiary education levels. There is a disconnect between the number of years of schooling and the quality of skills in Bulgaria.
Bulgaria has a low share of people lacking at least primary education and a fairly high number of average years of schooling per person, but at the same time there is a disconnect between the number of years of schooling and the quality of skills (as measured by PISA). Figure 1 shows that there are a number of countries (in the upper-left region – Singapore, Finland, Portugal, Croatia, Serbia, Turkey and several others) which achieve better PISA results with the same or fewer average years of education. We recognize that there is no direct relationship between the PISA test results (which focus on the skills of 15-year-olds only) and the average years of schooling of the whole population, but the issue is still worth exploring in more depth, not least of all because PISA-type tests indicate the quality of the future workforce.
4.2.
W HAT
IS
PISA?
The OECD Programme for International Student Assessment (PISA) assesses the extent to which students near the end of compulsory education have acquired the knowledge and skills that are essential for full, effective participation in modern societies, with a focus on reading, mathematics and science.10 The surveys are done on a triennial basis, with the first survey carried out in 2000 focusing on reading. The subsequent surveys focused on mathematics (2003) and science (2006), and in 2009 the cycle returned to reading. Bulgaria participated in the 2000, 2006 and 2009 rounds, and therefore fully comparable data on reading performance is available for the 2000-2009 period, while for mathematics and science performance it is only available for the shorter 2006-2009 period.
10
PISA 2009 Results - Learning Trends Changes in Student Performance Since 2000 (Volume V), p.17
13 | P a g e
Figure 1: Education quantity vs. quality: PISA 2009 test scores and average years of schooling
Source: Based on data from the Barro & Lee dataset (2010) and PISA 2009 test scores.
Individual student scores are grouped according to proficiency levels, which indicate the mastery of specific sets of skills. Levels begin with Proficiency Level 1 (subdivided into 1a and 1b for reading), indicating the lowest skillset, and increase to Level Lev 6. A note on OECD averages: we use OECDtotal, which is a weighted average (the OECD is treated as a single entity).
Note that the OECD uses two types of averages in their reports. One is the “OECD Average,”” which is a simple average of the mean scores for the individual countries, i.e. each membermember country has an equal weight in the final number. The second type of average is “OECD Total,”” which treats the OECD as a single entity comprised of all member countries. In this case the countries have weights corresponding to the number of students, i.e. larger countries have larger weights. In our report we use the latter number (OECD Total).
4.3. We compare Bulgaria to the OECD (treated as a single entity)…
C HOOSING
A PEER GROUP
When doing a comparison, choosing a comparison group is an important issue.. In this report we have taken the following approach – we use two country groups for comparison. The first is the OECD countries, which comprise a group representative of the rich or developed countries. This is is where wher we would like to be, and we use these numbers as our benchmark, even though 14 | P a g e
we recognise that the OECD is made up of countries which can differ substantially. … and a group of countries similar to Bulgaria (Serbia, Romania, Slovakia, Hungary and Greece).
The second comparison group consists of five countries which we have identified in our analysis to appear similar to Bulgaria in at least three of the following five dimensions: • • • • •
geographic location expenditures per student socio-economic status (Gini index) size (population) political background and recent history
These countries are our neighbours Serbia, Romania and Greece, plus former socialist Slovakia and Hungary. The experiences of these countries differ substantially, and when making comparisons we try to look at individual countries rather than going for the ‘average’ which could prove misleading.
15 | P a g e
5. DIMENSIONS OF THE EDUCATION GAP IN BULGARIA:
ESTIMATES
5.1.
T HE
INTERNATIONAL GAP
5.1.1. Dimensions of the gap In 2009, Bulgarian 15-year-olds scored an average of 429 points on the reading test, substantially (67 points) below the OECD average of 496 points. The same is true for the performance in math and science –Bulgarian students scored 60 points lower in mathematics and 57 points lower in science. Figure 2 shows the mean scores for Bulgaria, the peer group countries, and the other countries in the EU. Bulgaria is at the bottom of the league-tables, together with Romania. Bulgaria lags 58.7 points, or one and a half school years, behind the OECD. The distance to the top performers in the world is two to three times larger.
For the purpose of our report, we estimate the gap to be 58.7 points (the average of the difference in mathematics and science scores of Bulgaria and OECD). How large is this? One possible way to interpret the number is to use the PISA estimates, which say that an additional year in school is equal to approximately 38 points – meaning that, on average, Bulgarian students are more than a year and a half behind their peers from the OECD. The improvement needed to reach the level of the best performers internationally would require an improvement of almost double the size of the gap vis-à-vis the OECD average (more than 100 points, i.e. over one standard deviation for the PISA test). The size of the gap in average test performance alone does not tell the whole story; there are other very important aspects.
Economic, social and cultural status strongly affects outcomes in Bulgaria.
First, the gap in achievement between students from poor and rich families is more pronounced in Bulgaria than in most other countries (see Figure 3). Bulgaria is well above the OECD average in both the score point difference associated with a one-unit change in the PISA economic, social, and cultural status index (ESCS) and the percentage of the variance in performance explained by the ESCS. Only in Hungary and Slovakia do socioeconomic factors affect educational achievement (with respect to both mathematics and science scores) to such an extent. Greece, Serbia and Romania perform better than the OECD average on both measures.
16 | P a g e
Figure 2: Average PISA 2009 scores, EU and peer group countries
Source: PISA 2009.
Figure 3: The importance of the economic, social and cultural status gradient in explaining mathematics scores
Source: Calculations based on PISA 2009. The black lines represent the OECD average (as a single entity).
17 | P a g e
Bulgaria has a lower share of resilient students, and a higher share of disadvantaged low achievers than the OECD and the countries in the peer group.
Another illustration of the importance of the economic, social and cultural status in Bulgaria is given by the fact that relatively few Bulgarian students manage to overcome the disadvantage of their background (classified by OECD as resilient students11), while a relatively high percentage of disadvantaged students in Bulgaria score in the bottom quarter across all countries (a.k.a. disadvantaged low achievers 12 ). As illustrated in Figure 4, Bulgaria is in the unfavourable top-left corner, with only countries such as Peru, Azerbaijan, Kazakhstan, Qatar and Kyrgyzstan being in a worse position.
Figure 4: Resilient students and disadvantaged low achievers
Source: Calculations based on PISA 2009.
11
The formal definition is: “A student is classified as resilient if he or she is in the bottom quarter of the PISA index of economic, social and cultural status (ESCS) in the country of assessment and performs in the top quarter across students from all countries after accounting for socio-economic background.” 12 The formal definition is: “A student is classified as disadvantaged low achiever, if he or she is in the bottom quarter of the PISA index of economic, social and cultural status (ESCS) in the country of assessment and performs in the bottom quarter across students from all countries after accounting for socio-economic background.”
18 | P a g e
Almost half of the Bulgarian students in 2009 only demonstrated basic skills (Level 1 or below), and there are just a few top performers. In the case of the top performer, China (Shanghai), more than half of the students were top performers (Level 5 or Level 6).
Second, the poor average performance of Bulgaria is not due to a small number of very poor performing students. Almost a quarter of the 15-year-olds tested in 2009 scored below the lowest Level 1 in mathematics (see Figure 5), and a further 22.7% scored at Level 1 – which means that almost half of Bulgarian students demonstrated only basic level skills. At the other end of the spectrum, in Bulgaria there are very few top performers – just 3% of students tested scored at Level 5, and only 0.8% at the highest Level, 6. As a stark comparison, in China (Shanghai) more than half of the students scored at Level 5 or 6. In Slovakia, the top performer in our peer group with respect to this criterion, 9.1% scored at Level 5 and 3.6% at Level 6. Hungary also scored quite high (8.1% and 2.0% respectively). Greece (4.9% and 0.8%) and Turkey (4.4% and 1.3%) performed marginally better than Bulgaria, and only Romania performed worse than Bulgaria. Lastly, the variance in performance in Bulgaria is large compared to most countries in the world (Figure 6). There is substantial variation both within schools and between schools, suggesting the interplay of factors acting at the student level (e.g. family income, parental education and employment status) and factors acting at the school level (e.g. allocation of students to schools, by policy or by parental choice).
19 | P a g e
Figure 5: Distribution by proficiency roficiency levels (mathematics, PISA 2009)
Source: PISA 2009. Sorted by the sum of Level 5 and Level 6 performers.
20 | P a g e
Figure 6: Between- and within-school variance in reading performance
Source: PISA 2009
5.1.2. Can the gap be closed? Using our assumed duration of the reform of 15 years, closing a gap of 58.7 points (in the mathematics and science scores) seems ambitious but possible. We would need to achieve approximately a 3.9 point improvement per year. Mexico and Brazil managed to improve their mathematics scores at an annual rate of 5.5 and 5.0 points respectively between 2003 and 2009. It is also true that they started from relatively low average scores (385 and 356 respectively), and their experience may not be directly applicable (progress at higher scores is slower). Geographically closer, and closer to the Bulgarian starting score, Turkey improved by 3.7 points per year, Greece and Portugal by 3.5, and Italy by 2.9 points.
5.1.3. Estimated benefits of closing the gap In terms of its economic impact, the international gap is the largest of the three discussed in this report. Our simulation shows that closing this gap would increase the long-term growth rate of the economy by 1.09 percentage points. At the end of the 80-year period, the percentage gain to the economy would reach an amazing 77.6 per cent, and it would be a meaningful 16.6 per cent as early as the middle of the period (2052). In monetary terms, this translates to a value of the education reform over the full 80-year period of more than eight times Bulgarian GDP in 2012. Table 2, Figure 7 and Figure 8 provide more details and a graphical representation of the benefits.
21 | P a g e
Table 2: Estimated benefits from closing the international gap
Addition to the annual growth rate at the last year of the phase (percentage points) Value of the reform from 2012 until the end of phase (2011 BGNm) as % of the GDP in 2012 Percentage addition to GDP due to the reform (*)
end Phase 1 2027
end Phase 2 2052
end Phase 3 2067
end Phase 4 2092
0.22
0.90
1.09
1.09
3 528 93 359 240 820 632 804 4.62 122.21 315.23 828.34 1.23 16.57 35.82 77.61 (*) Calculated as (GDP with reform / GDP without reform) - 1 Source: Author’s calculations, based on Hanushek and Woessmann (2011) and data from PISA 2009.
Figure 7: Closing the international gap: Impact of the reform on GDP (in million 2011 BGN)
Figure 8: Closing the international gap: Impact of the reform on the real growth rate
Source: Author’s calculations, based on Hanushek and Woessmann (2011) and data from PISA 2009.
5.2.
T HE L ANGUAGE G AP
5.2.1. Dimensions of the gap Insufficient mastery of the language of instruction is one of the main constraints on the mastery of subject knowledge and cognitive skills in school. The underlying logic is straightforward – if you cannot properly understand what you are being taught because of the language barrier, you have lower chances to do well. Figure 9 shows that different achievement in school according to the language spoken at home is observed in most countries around the world, and in most cases, students who speak the language of the test at home (a.k.a. the ‘linguistic majority’) perform better than their peers who do not (the ‘linguistic minority’). Countries in the upper right-hand region of the figure are the ones who performed well overall, and those in the lower left-hand region underperformed in the 2009 PISA test. The black line indicates equal performance – along it there is no difference (within the given country) in the mean scores between students who speak the language of the test at home, and those who speak another language. The closer to the line of equality a 22 | P a g e
country is on the figure, the smaller the difference between the two groups of students. Figure 9: The difference in mean reading scores of students based on language spoken at home
Note: The figure shows the point estimates of the respective means. For example, in Hungary, the 95% confidence intervals for the two means are large, and the difference in scores may be quite different than apparent from the figure – it could be as low as 18 points or as large as 168 points. In Greece and Romania the differences in the means are respectively between 42 and 117 points, and 35 to 101 points. In Bulgaria, the difference in the means is between 70 and 133 points, and only Mexico (76) and Peru (86) have a higher minimum difference. In the Czech Republic, Croatia and Latvia, among others, the two means are not statistically different from each other. Source: Based on data from PISA 2009.
Bulgaria has one of the largest differences within PISA countries in the scores of the linguistic majority and the linguistic minorities, and has the lowest-scoring linguistic minority in our peer group. In 2009, Bulgarian linguistic minority students, who comprised about 11% of the total number of 15-year-olds, scored on average 342 points on the reading test, 372 points on the mathematics test and 369 points on the science test. This means that their scores were a staggering 102 points lower on the reading test, and respectively 66 and 82 point lower on the mathematics and science tests compared to the linguistic majority students. With a mean score of only 342 points on the reading test, the ‘average’ student from a linguistic minority group in Bulgaria barely reaches proficiency at Level 1a on the reading scale – something which over 94% of OECD students achieve. Essentially, this score means that the mean student is only capable of just the basic skills in reading, mathematics and science. Table 3 provides a fuller description of the proficiency levels reached by the mean Bulgarian student from a linguistic minority. 23 | P a g e
Table 3: Descriptions of the Proficiency Levels for the mean student from a linguistic minority in Bulgaria (PISA 2009)
Proficiency at Level 1a (scores higher than 335 but lower than or equal to 407 points) Students proficient at level 1a on the reading literacy scale are capable of locating pieces of explicitly stated information that are rather prominent in the text, recognising a main idea in a text about a familiar topic, and recognising the connection between information in such a text and their everyday experience. Tasks at this level require students to locate one or more independent pieces of explicitly stated information, recognise the main theme or author’s purpose in a text about a familiar topic, or make a simple connection between information in the text and common, everyday knowledge. Typically the required information in the text is prominent and there is little, if any, competing information. Students are explicitly directed to consider relevant factors in the task and in the text. Mathematics Proficiency Level 1 (357.8 to 420.1 points) At Level 1 students can answer questions involving familiar contexts where all relevant information is present and the questions are clearly defined. They are able to identify information and to carry out routine procedures according to direct instructions in explicit situations. They can perform actions that are obvious and follow immediately from the given stimuli. Science Proficiency Level 1 (334.9 to 409.5 points) At Level 1, students have such a limited scientific knowledge that it can only be applied to a few, familiar situations. They can present scientific explanations that are obvious and follow explicitly from given evidence. Source: PISA 2009, Assessment Framework: Key competencies in reading, mathematics and science; PISA 2009 Results: What Students Know and Can Do, Vol.I
5.2.2. Can the gap be closed? There is mixed evidence internationally on whether the language gap can be closed.
Looking globally, countries have had mixed results in addressing their language gaps. On the one hand there are countries which have a broadly similar share of students from linguistic minorities (7% to 18%), such as Israel, Latvia, Australia and Kazakhstan, where there is no (statistically distinguishable) difference in the performance between the linguistic minorities and the linguistic majority; and there are countries such as Spain and Canada where the difference is minimal. On the other hand, looking at the OECD as a single entity, the difference in scores is between 29 and 45 points. In Italy, the difference is between 53 and 70 points, in Austria between 47 and 83 points, between 40 and 75 points in Germany, and between 51 and 92 points in Sweden. There is certainly a lot of room for examining the different experiences and policies to establish whether and how the language gap can be closed in Bulgaria.
5.2.3. Estimated benefits from closing the gap The size of the language gap is the equivalent of 11 points in the national mean score.
Despite the large gap between the linguistic majority and the linguistic minority in Bulgaria, even if the gap were immediately fully closed, 13 the impact on the national average would be 13
Technically, ‘closing the gap’ in this case means bringing the national mean score in mathematics (428 points) and science (439 points) to the respective means for the linguistic majority group (438 points for mathematics and 451 points for science). This effectively means increasing the mean scores for the students who reported
24 | P a g e
relatively small – an increase of only 11 points, a value dwarfed by the magnitude of the international gap discussed above. Closing the language gap would be expected to increase the longterm growth rate of the economy by 0.20 percentage points. The value of the reform over the full 80-year period would be 133.7% of the GDP of Bulgaria in 2012, a sizeable gain despite the fact that it would occur over decades. Even as soon as 2027 Bulgaria could see a boost of 0.23 percentage points to its GDP. This might seem miniscule, but applied to the size of the economy it comes to almost 140 million BGN in present value terms. In addition to the purely economic benefits, it should be noted that closing this gap entails not only raising the national mean score by a certain amount of points, but also an important change in the internal distribution. Achieving a state of equality for linguistic minority students has social value in addition to the purely economic benefit, as well as an important moral aspect that is beyond the scope of this paper.
Table 4: Estimated benefits from closing the language gap
Addition to the annual growth rate at the last year of the phase (percentage points) Value of the reform from 2012 until the end of phase (2011 BGNm) as % of the GDP in 2012 Percentage addition to GDP due to the reform (*)
end Phase 1 2027
end Phase 2 2052
end Phase 3 2067
end Phase 4 2092
0.04
0.17
0.20
0.20
655 16 784 41 715 102 141 0.86 21.97 54.61 133.70 0.23 2.90 5.89 11.34 (*) Calculated as (GDP with reform / GDP without reform) - 1 Source: Author’s calculations, based on Hanushek and Woessmann (2011) and data from PISA 2009.
Figure 10: Closing the language gap: Impact of the reform on GDP (in million 2011 BGN)
Figure 11: Closing the language gap: Impact of the reform on the real growth rate
Source: Author’s calculations, based on Hanushek and Woessmann (2011) and data from PISA 2009.
using another language at home and those who, for various reasons, have no answer to this question. These two groups comprise 13.6% of the 15-year-olds.
25 | P a g e
5.3.
T HE S CHOOL C OMMUNITY T YPE G AP
5.3.1. Dimensions of the gap Disparities in performance are also commonly observed across communities with different population sizes. Specifically, students from smaller communities tend to perform worse than their peers in larger cities. In the OECD (again seen as a single entity comprising all member countries), students living in villages14 (8.7% of all students) perform worse on average than students from small towns (16.5%), who in turn perform worse than students living in average-sized towns (31.2%). The performance of students living in towns, cities (27.8%) and large cities (15.7%) does not differ statistically at the 95% confidence level. In other words, approximately a quarter of OECD students suffer from a school community gap, with the difference in performance varying between 6 to 34 points. In Bulgaria, 15-year-olds living in villages perform significantly worse than those living in small towns and towns, who in turn perform significantly worse than 15year-olds living in cities and large cities…
In Bulgaria, by contrast, there are three distinct groups in terms of achievement scores. The first group, the top performers, are students living in cities (23%) and large cities (15.3%), the difference between whom is not statistically different. The second group, again with no statistically meaningful difference in performance, is comprised of students living in towns (38.6%) and small towns (17.7%), who performed, on average, 57 points worse on the reading test, 51 points worse on the math test, and 46 points worse on the science test. The last group of students – those living in villages (5.3% of all 15-year-olds) – performed 68 points below their peers from small towns in reading, 63 points lower in mathematics and 52 points lower in science, with the difference in the science scores being statistically significant at the 94% level and the rest at 95%.
… and the differences are both statistically and economically significant. On average, living in a village sets you almost three school years back compared to peers living in cities.
These differences are not only statistically significant, they are economically significant. PISA estimates that one school year is ‘worth’ about 38 points, suggesting that on average in Bulgaria the ‘privileged’ group (about 38.4% of the population) is more than a year and a half ahead of the ‘less privileged’ group (56.3%), who in turn are more than one school year ahead of the ‘disadvantaged’ group living in villages (5.3%). To put things in another perspective – the difference in performance between the best and worst performers based on their school community size in Bulgaria on the reading test is 17% greater than the difference between Bulgaria and the bestperforming country in the world, China (Shanghai), and 39% greater than the difference between Bulgaria and the bestperforming European country, Finland. The gap between Bulgarian students living in villages and those living in cities is
14
In PISA’s classification, “village” means fewer than 3,000 people; “small town” is between 3,000 and 15,000 people; “town” is between 15,000 and 100,000 people; “city” is between 100,000 and 1,000,000 people; and “large city” is over 1,000,000 people.
26 | P a g e
16% larger in math and 12% larger in science than the gap between Bulgaria and Finland. If we look at the data in more detail – including both the linguistic and the school community type dimensions – the segmentation observed above holds for the linguistic majority group alone. In the case of the linguistic minority students, the reported means for the school community types are not statistically different (even though we do observe a similar tendency in the point estimates – the linguistic minority 15-year-olds who live in villages scored lower than those living in small towns, who in turn scored lower than those in towns, etc., but the difference is within the error margins of the test). In this report we do not attempt to provide a model of student behaviour and the relevant factors, and consequently cannot comment on whether this apparent breaking of the relationship for linguistic minority students is due to addressable underlying factors or due to the low number of minority students tested (e.g. only 46 15-year-olds linguistic minority students living in villages were tested by PISA in 2009). We estimate the gap to be an equivalent of a 39 PISA point difference in the national mean score. This is calculated by bringing the mean scores for all groups (i.e. all community types) to those of the highest-performing group, 15-year-olds living in cities. That is equivalent to raising the national mean score in math from 428 to 468 points, and the science score from 439 to 478 points.
5.3.2. Can the gap be closed? It is difficult to distil the possible disparities within any given country into a single number. A visual inspection of the differences within the OECD, however, suggests that it is indeed possible to eliminate the disadvantage faced by students living in villages and small towns (see Figure 12 and Figure 13). In our peer group, Hungary and Turkey seem to have similar issues to Bulgaria (Figure 14 and Figure 15). Greece, on the other hand, has a very equal distribution – a situation worth exploring in detail (Figure 16 and Figure 17). Serbia is somewhere in the middle, with some inequality in the mean score point estimates, but the differences are mostly not statistically significant, and only 10 students from villages were tested, so no mean data is available for villages.
27 | P a g e
Figure 12: Mathematics mean score and 95% confidence intervals, 2009: Bulgaria vs. OECD
Figure 13: Science mean score and 95% confidence intervals, 2009: Bulgaria vs. OECD Science mean score and 95% confidence intervals
Mathematics mean score and 95% confidence intervals 550
550
500
500
450
450
400
400
350
350 300
300 Village Small Town Town
City
Bulgaria
Large Village Small Town City Town
City
Village Small Town Town
Large City
City
Bulgaria
OECD Total
City
Large City
OECD Total
Source: PISA 2009
Figure 14: Mathematics mean score and 95% confidence intervals, 2009: Hungary and Turkey
Large Village Small Town City Town
Source: PISA 2009
Figure 15: Science mean score and 95% confidence intervals, 2009: Hungary and Turkey Science mean score and 95% confidence intervals
Mathematics mean score and 95% confidence intervals 600
550
550
500
500
450 450
400
400
350
350 300
300 Village Small Town Town
City
Hungary
Large Village Small Town City Town
City
Village Small Town Town
Large City
City
Hungary
Turkey
City
Large City
Turkey
Source: PISA 2009
Figure 16: Mathematics mean score and 95% confidence intervals, 2009: Greece and Serbia
Large Village Small Town City Town
Source: PISA 2009
Figure 17: Science mean score and 95% confidence intervals, 2009: Greece and Serbia Science mean score and 95% confidence intervals
Mathematics mean score and 95% confidence intervals 550
550
500
500
450
450
400
400
350
350 300
300 Village Small Town Town Greece
City
Large Village Small Town City Town
City
Large City
Serbia
Source: PISA 2009
Village Small Town Town Greece
City
Large Village Small Town City Town
City
Large City
Serbia
Source: PISA 2009
28 | P a g e
5.3.3. Estimated benefits from closing the gap Ensuring equal performance of children regardless of where they live, i.e. closing the school community type gap, would increase the long-term growth rate of the Bulgarian economy by 0.73 percentage points. In terms of magnitude, this falls between the impact of closing the international gap and closing the language gap. Over the full 80-year period, i.e. the lifetime of a child born in 2012, the value of the reform would reach 5.2 times the GDP in 2012, and by 2092 the economy would enjoy 46.8% higher GDP. Lastly, similar to the language gap, providing equal-quality education to students regardless of where they live carries noneconomic benefits in excess of the purely economic impact. Under our scenario of closing the gap, some 77% of 15-year-olds will increase their scores, even if by a small amount.
Table 5: Estimated benefits from closing the school community type gap
Addition to the annual growth rate at the last year of the phase (percentage points) Value of the reform from 2012 until the end of phase (2011 BGNm) as % of the GDP in 2012 Percentage addition to GDP due to the reform (*)
end Phase 1 2027
end Phase 2 2052
end Phase 3 2067
end Phase 4 2092
0.15
0.60
0.73
0.73
2 352 61 394 155 935 397 688 3.08 80.36 204.12 520.57 0.82 10.78 22.70 46.79 (*) Calculated as (GDP with reform / GDP without reform) - 1 Source: Author’s calculations, based on Hanushek and Woessmann (2011) and data from PISA 2009.
Figure 18: Closing the school community type: Impact of the reform on GDP (in million 2011 BGN)
Figure 19: Closing the school community type: Impact of the reform on the real growth rate
Source: Author’s calculations, based on Hanushek and Woessmann (2011) and data from PISA 2009.
29 | P a g e
6. NOTE ON ACCURACY & RECOMMENDATIONS FOR FURTHER RESEARCH
This report is not intended to be a rigorous, scientific exploration of the field, but rather to provide a few basic data points as to the size of the gaps in education achievement internationally and within Bulgaria, and their potential economic dimensions. As such, we have often taken shortcuts that make our results easier to obtain. The biggest shortcut we have taken is in foregoing a model of the education system, i.e. we do not claim to be able to explain behaviour or cause and effect relationships. As a consequence, we do not provide solutions, or that the problems identified can be solved. We simply assume that it is possible to live in a situation where the identified problem does not exist and we provide real-world examples where the respective gaps do not exist, but without claiming that other countries’ experience can be replicated in Bulgaria. Second, we use a modelling approach based on a class of economic models – endogenous growth theory – which is not universally accepted as the definitive economic growth model among experts. The methodology we use is widely accepted – and has been used in academic research and OECD publications – but should not be viewed as the only possible approach. Third, the concrete econometric treatment of the projections is based on academic research on the experience of OECD economies over 1960-2000. In line with the authors of the main strand of research that we use – Eric Hanushek of Stanford University and Ludger Woessmann of University of Munich – we have assumed that the OECD experience is applicable to Bulgaria as well. This may (Bulgaria is on the path of market-based development) or may not be true (the world as a whole may have changed substantially since the period over which the econometric analysis was done by the authors whose research we use in our report). Fourth, we would like to draw the reader’s attention to the fact that despite being widely used, this methodology offers a somewhat simplistic view of the world. A major deficiency of the methodology is that it disregards the possibility of lifelong learning – the ‘quality’ of a country’s labour force is proxied by the cognitive skills of students (in the case of our report, of 15year old Bulgarian students), and there is no possibility to upgrade one’s skills after finishing school. Another deficiency of the model is that it does not include any allowance of potential positive-sum game effects – e.g. the idea that the combined benefit to a country in which everyone is educated is bigger than the sum of the individual benefits to each citizen.
30 | P a g e
Fifth, we explicitly treat the three main types of achievement gaps that we describe and quantify as independent of each other. We do not claim to have produced a logical model which explains the existence of these gaps, nor do we analyse in detail the linkages between the different factors.
C ALL
FOR FURTHER RESEARCH
The text above outlines a number of simplifying assumptions in our report which suggest the issue can be explored in significantly more detail. There is an enormous scope for academic and policy research on the topic of education outcomes, growth, and development. Possibly the most important line of further research would be a more detailed and rigorous analysis of the linkages between the different dimensions of the achievement gap. We have only looked at three independent ‘orthogonal’ dimensions, i.e. dimensions which cannot and should not be compared or ranked. One cannot claim that, for example, the language gap is more or less important than the school community gap. On the other hand, it is likely that in reality they have a combined influence in addition to their individual influences, and a policy designed to address both issues simultaneously may have to be quite different from policies designed to address the issues separately.
31 | P a g e
7. REFERENCES
Barro, Robert and Jong-Wha Lee (April 2010), A New Data Set of Educational Attainment in the World, 1950-2010. NBER Working Paper No. 15902. Dataset downloaded from http://www.barrolee.com (2011.09.04 Update). Hanushek, Eric A. and Ludger Woessmann (2008), The Role of Cognitive Skills in Economic Development, Journal of Economic Literature 2008, 46:3, 607–668, http:www.aeaweb.org/articles.php?doi=10.1257/jel.46.3 .607 Hanushek, Eric A. and Ludger Woessmann (2009), Do Better Schools Lead to More Growth - Cognitive Skills, Economic Outcomes, and Causation (NBER w14633) Hanushek, Eric A. and Ludger Woessmann (2012 forthcoming), Do Better Schools Lead to More Growth? Cognitive Skills, Economic Outcomes, and Causation, Forthcoming: Journal of Economic Growth, version: June 14, 2012 Hanushek, Eric A. and Ludger Woessmann (2012), The Economic Benefit of Educational Reform in the European Union, CESifo Economic Studies, Vol. 58, 1/2012, 73–109 doi:10.1093/cesifo/ifr03, Advance Access publication 24 January 2012 Hanushek, Eric A. and Ludger Woessmann (2011), How much do educational outcomes matter in OECD countries?, Economic Policy July 2011, pp.427-491 PISA 2006: The High Cost of Low Educational Performance - The Long-Run Economic Impact of Improving PISA Outcomes, ISBN 978-92-64-07748-5 PISA 2009 Results: Overcoming Social Background - Equity in Learning Opportunities and Outcomes (Volume II), ISBN 978-92-64-09150-4 PISA 2009 Results: Learning Trends Changes in Student Performance since 2000 (Volume V), ISBN 978-92-6409158-0
32 | P a g e
8. GLOSSARY OF TERM / KEY CONCEPTS
Achievement gap – the observed disparity (greater than the margin of error of the respective test) in the performance of groups of students. In this report, we measure performance as the score on the PISA tests, but a number of other measures are used (grade point average; dropout rates; college enrolment rates; education completion rates, etc.) Groups of students may be defined across a number of attributes – race/ethnicity, nationality or citizenship, gender, socioeconomic status, family/household status, etc. BGN (or 2011 BGN) – Bulgarian Leva. Throughout this report BGN is used interchangeably to mean “constant purchasing power 2011 Bulgarian Leva”, i.e. we assume no changes in the Lev’s purchasing power over the period discussed. Cognitive skills / cognitive abilities – skills utilized in the process of obtaining knowledge through thought, experience and the senses. Economic, social, and cultural status (ESCS) – The Programme for International Student Assessment (PISA) index of economic, social and cultural status was created on the basis of the following variables: the International Socio-Economic Index of Occupational Status (ISEI); the highest level of education of the student’s parents, converted into years of schooling; the PISA index of family wealth; the PISA index of home educational resources; and the PISA index of possessions related to “classical” culture in the family home. (http://stats.oecd.org/glossary/detail.asp?ID=5401) Education quality – a general measure of the quality of skills and knowledge acquired in an individual’s education. Often measured by some form of test of cognitive skills. Education quantity – a general measure of the quantity of education received by an individual. Usually measured as average years of schooling or as educational attainment. Educational Achievement – refers to the quality of skills and knowledge mastered by an individual, and does not necessarily correspond to having completed a particular level of schooling or degree. Educational Attainment – the highest degree of education (or the highest level of schooling) an individual has attended and successfully completed. Enrolment – the number of students attending a given school, education level, or other educational measurement. Gap – the observed disparity on a number of educational measures between the performance of groups of students. GDP – Gross Domestic Product. The market value of final goods and services produced in a country within a given period of time. 33 | P a g e
Human Capital – an aggregate view of a human being acting within an economy measuring competencies, knowledge, social and personality attributes embodied in the ability to perform labour so as to produce economic value. Linguistic Majority – citizens of a country speaking the official language of the country at home. Linguistic Minority – citizens of a country speaking a language at home other than the officially recognized language. OECD – Organization for Economic Cooperation and Development. A forum where the governments of 34 democracies work together to address the economic, social and environmental challenges of globalization. PISA – Launched in 1997 by the OECD, the Programme for International Student Assessment is an international study meant to evaluate educational systems worldwide every three years by assessing 15-year-olds' competencies in three subjects: reading, mathematics, and science.
34 | P a g e