Insight Note

RISE Vision Document 2: Ambitious Learning Goals Need Audacious New Approaches

Key Points

  • There is a growing global consensus that many developing countries face a learning crisis and that national and international learning goals can help focus attention on that crisis.

  • Developing countries will not meet OECD learning levels in a generation if they sustain current levels of learning progress. To reach these levels, they would require an acceleration in the pace of improvement from an average gain of 0.9 points per year to 3.8 points per year on international assessments (or similar accelerations on national assessments).

  • Strategies like requiring students to stay in school longer or providing infrastructure or recurring inputs alone will not improve learning. Any quality- improvement strategy should be embedded in an effective system that focuses on learning. Countries will need to take audacious measures to accelerate learning progress.

The pivot to focus efforts on learning goals

The right to education in the Universal Declaration of Human Rights was an education goal, not a schooling goal. The education objectives of schooling were emphasised in the 1990 World Declaration on Education for All at Jomtien, Thailand, which stated:

“Every person—child, youth and adult—shall be able to benefit from educational opportunities designed to meet their basic learning needs. These needs comprise both essential learning tools (such as literacy, oral expression, numeracy, and problem solving) and the basic learning content (such as knowledge, skills, values, and attitudes) required by human beings to be able to survive, to develop their full capacities, to live and work in dignity, to participate fully in development, to improve the quality of their lives, to make informed decisions, and to continue learning.”

The 1990 Education for All declaration also recognised that schooling isn’t learning as “millions ... satisfy the attendance requirements but do not acquire essential knowledge and skills” and that learning within schools “must be... qualitatively improved.”

The Sustainable Development Goals will almost certainly reflect a refocus of efforts onto learning. The July 2014 draft of Goal 4 includes targets for universal schooling leading to “relevant and effective learning outcomes,” the elimination of illiteracy and innumeracy among youth by 2030, and the acquisition of relevant skills for employment and entrepreneurship.

There are numerous efforts to turn these broad learning goals into concrete, measurable, and achievable targets. All of these effortsrecognise that neither education nor learning can be reduced to a single goal but will require measures of mastery at various stages and assessment of different domains of skills and competencies. For instance, the Learning Metrics Task Force (LMTF) divided areas of measurement into stages. Children should enter school ready to learn, master foundational reading skills by Grade 3, achieve proficiency by the end of primary school, and demonstrate a set of skills by the end of compulsory schooling (Table 1).

In this vision document, we are not prejudging the outcome of the specific goals that will be adopted either globally or nationally, but rather want to ask: if goals are adopted, what will it take to reach them?

Table 1: Learning Metrics Task Force proposed areas of measurement and indicators.

Areas of Measurement Description of Indicators
Learning for All Combine measures of completion and learning (reading proficiency at the end of primary school) into one indicator.
Age and Education Matter for Learning Measure timely entry, progression and completion of schooling, and population-based indicators to capture those who do not enter or those who leave school early.
Reading Measure foundational skills by Grade 3 and proficiency by the end of primary school.
Numeracy Measure basic skills by end of primary and proficiency by lower secondary school.
Ready to Learn Measure acceptable levels of early learning and development across a subset of domains by the time a child enters primary school.
Citizen of the World Measure among youth the demonstration of values and skills necessary for success in their communities, countries and the world.
Breadth of Learning Opportunities Track exposure to laerning opportunities across all seven domains of learning

Source: Learning Metrics Task Force, 2013, Figure 6, pg.23

 

Developing countries need to significantly accelerate current rates of progress to reach international learning goals

Perhaps nothing special is required to meet learning goals. Perhaps the natural, if not inevitable, result of the general path of development (improving health status of children, expansion in parental income and education, greater democracy) and ongoing “business as usual” approaches to schooling (eg better educated teachers, higher spending per student, better inputs) will be acceptable learning outcomes. But “perhaps” is not a good plan.

There is substantial evidence that reaching meaningful learning goals in a reasonable time frame will require a very substantial increase in learning gains over the “business as usual” pace of progress of recent decades for most developing countries.

Not all countries are even making positive progress. In a survey of rural India which assesses over half a million children a year the percentage of Grade 5 students who can read a simple story fell from 54 percent to 48 percent from 2010 to 2014, and the percentage of Grade 5 students who could do a simple division problem fell from 36 percent in 2010 to just 26 percent in 2014 (ASER, 2015).

The lack of learning data that is comparable over time makes comprehensive and definitive answers impossible, but what news there is, isn’t good. Beatty and Pritchett (2012) calculate the rate of progress in developing countries on indicators for reading, mathematics, and science from various assessments. While the results vary across instrument and domain, there are three general findings. First, on a common scale of one student standard deviation equalling 100 points, the typical observed progress is only one to two points per year. Second, a substantial number of countries exhibit retrogression as the performance of the tested population is falling. Third, the pace of learning improvement in most developing countries will need to accelerate dramatically to reach an international learning goal (a mean score of 500 or improving one standard deviation on international standardised assessments) in one generation (25 years).

Table 2 illustrates this point using the most recent data from the Programme for International Student Assessment (PISA). It shows the pace of progress averaged across reading, mathematics, and science (column II) and the pace that would be needed to reach a country average score of 500 (the OECD mean) in one generation of 25 years (column III).1 This is an ambitious agenda, but not impossible for poor countries, as Vietnam already scores above 500 in reading, mathematics, and science.

Table 2: The pace of improvement in learning outcomes would have to increase dramatically to reach an ambitious goal in one generation in many developing countries.

Country I II III IV V=IV-II
 

Average score across Reading, Mathematics and Science (OECD average=500)

Average annual progress in points per year across the three domains

Years to move from country average to 500 at current pace

Progress in points per year to reach 500 in 25 years

Acceleration to reach learning goal of average PISA of 500 in 25 years
Average (of these countries) 404 0.9 91 3.8 2.9
Peru 375 2.5 50 5 2.5
Indonesia 384 0.4 317 4.6 4.3
Colombia 393 1.9 55 4.3 2.4
Tunisia 397 3 35 4.1 1.1
Argentina 397 0.7 155 4.1 3.5
Jordan 398 -0.7 Forever 4.1 4.8
Brazil 402 2.5 39 3.9 1.4
Uruguay 412 -1.8 Forever 3.5 5.3
Malaysia 413 -0.3 Forever 3.5 3.8
Mexico 417 1.7 49 3.3 1.6
Costa Rica 426 -0.9 Forever 3 3.9
Thailand 437 2 31 2.5 0.5

Source: OECD, 2014. Data for in-school 15-year-olds

 

Table 2 shows the average across the eleven non-OECD developing countries which have participated in PISA in multiple years:2

  • In 2012, developing countries were about 100 points behind the OECD mean of 500.
  • The average gain was only 0.9 points a year; four countries show worse outcomes over time (column II).
  • Extrapolation of the “business as usual” gain in learning implies it would take four generations - 100 years - for students to reach the current OECD level of 500 (column III).
  • To reach a PISA score of 500 in a generation would require countries to accelerate learning by 2.9 points a year (column V or column IV minus column II) to 3.8 points per year (column IV) - four times the current pace of 0.9 points per year.

There are of course vast differences across countries - Brazil, if it could maintain its current progress of 2.5 points per year gain, would only need an acceleration of 1.4 points, whereas Indonesia, at a measured progress of only 0.4 points a year, would need an acceleration of 4.3 points.

More of the same will not allow countries to meet learning goals

The global movement for schooling has been fantastically successful in expanding schooling, and this creates momentum for “more of the same.” However, the evidence strongly suggests that neither merely expanding exposure nor more traditional inputs will be enough to accelerate learning progress sufficiently.

More of the same: just more time in school isn’t enough

Nearly every country that reached universal primary schooling reset their goal to universal basic education, and nearly every country that reached universal basic education reset their goal to universal secondary education. But this cannot be the only way to reach universal command of competencies.

If the learning profile is shallow - children are learning little each year - then a “business as usual” expansion of the number of years in school produces only the same learning gains per year. Pritchett (2013) calculates the gains to the fraction of children with Trends in International Mathematics and Science Study (TIMSS) mastery of mathematics above a level of 4203 for various countries by (a) moving to universal completion of Grade 9 by all children or (b) raising the learning profile so that learning is higher by a student standard deviation at Grade 9 or (c) both.4

Figure 1 shows that the gains in a learning goal—that all children in a cohort leave their schooling with at least a minimal standard of competence - from expanding completion are limited when learning is low. In Ghana at baseline, only 10 percent of the cohort aged 15 to 19 reached a learning goal. Expanding schooling such that every child completes Grade 9 would only add an additional 10 percent of learners who achieve that goal. Thus, even with universal Grade 9 completion, only 20 percent of a cohort would be above a learning threshold. In contrast, if students learned more per year, even without expanding grade completion, the fraction reaching a learning goal would increase by 22 percent.

Figure 1: Reaching universal learning goals will require both more schooling and more learning per year of schooling.

Bar chart that shows five countries and their universal completion of Grade 9, a steeper learning profile, and both universal Grade 9 and a steeper learning profile

Source: Adapted from Pritchett, 2013, table 2.4.

Therefore, helping cohorts leave their schooling prepared for the 21st century will require both expansions in the time in school and substantial improvements in learning progress per year of schooling.

More of the same: just more spending isn’t enough

More emphasis on schooling as a means of preparing children for their adult life means a shift in focus from the logistics of access to providing learning and learning environments.

One important part of the learning environment is access to infrastructure and recurrent inputs. Children should be able to learn in safe and comfortable places, and can learn more with access to age- and subject-appropriate learning materials. But reviews of hundreds of studies in dozens of countries have shown that while inputs can matter, very little of the observed learning variation across students, across regions, or across countries can be accounted for by the difference in access to inputs (McEwan 2014; Glewwe et al. 2014). A plan for expanding inputs is only part of a plan for improving learning. Unfortunately, when many countries prepare plans to improve the quality of schooling, they only focus on the inputs, not an entire plan for learning.

Effective teaching and teachers are also central to accelerating the pace of learning. Better trained teachers and a structure of compensation adequate to attract, retain, and motivate a quality teaching force are all key to supporting learning. But effective teaching does not come about through simple recipes like better qualifications, more training, or higher wages. Narrow bureaucratic measures of formal qualifications or exposure to in-service trainings explain some, but not much, about why some teachers are effective and others are not (Bruns and Luque 2014; Mehtabul and Kingdon 2014; Slater et al. 2012; Rivkin et al. 2005).

For example, it might be the case that globally the best systems, like Finland, have high pay for teachers. However, that higher pay is embedded in an effective system of education and isn’t necessarily isolatable as the key causal factor of performance that, changed alone, would lead to better outcomes. The recent experience in Indonesia with the Teacher Law of 2005, a plan to raise teacher pay integrated with other mechanisms to improve teacher accountability and performance, is instructive. The law was stripped down essentially to just higher teacher pay, due to last-minute politics and over the strong objections of the technical staff working in education. As a result, Indonesia now has an annual budget higher by billions of dollars, and rigorous experimental evidence shows that, not surprisingly, teachers with higher pay and nothing else changed do no better at promoting student learning than they did before the pay increase (Chang et al. 2014). Perhaps many reforms - teacher pay, training, school and teacher autonomy, etc - showed mixed performance because they work best (or at all) only when embedded in an effective system that supports and values high performance.

Conclusion: a case for audacious steps to meet learning goals

In order for children in developing countries to receive an education equal to the challenges they will face as adults in the 21st century, a dramatic improvement in learning is required. The existing pace of improvement in learning is far from adequate and a decisive acceleration in the pace of improvement in learning outcomes is needed to reach ambitious learning goals in one generation. While the plans for better learning outcomes almost certainly will involve “more of the same” - more spending, more inputs, more qualifications and training of teachers - there is no evidence that simply “more of the same” can add up to the changes in learning that are needed. Thus, if developing countries want to meet 21st century challenges they will need to search for system actions that accelerate the overall pace of progress in learning.

Footnotes

1 These calculations are illustrative and do not suggest that all countries adopt any particular goal of improvement or goals based on PISA scores. Instead, they inform understanding of what it would take for developing countries to reach global learning levels. Calculations with other assessments and other ambitious goals would likely produce similar results.

2 This table excludes many non-OECD countries that participated in PISA, including oil rich countries like Qatar and UAE and all former Soviet sphere countries. These calculations of course starkly understate the global problem as only the more middle-income countries even participate in PISA. When two Indian states (thought to be more educationally advanced) participated in PISA, their scores were well below any countries in the table. Other South Asian countries and many African
countries are likely performing at
or below that level

3 We use 420 since it equates to one student standard deviation below the OECD mean of 500.

4 To stress once again, these calculations are not meant to propose specific goals for countries to adopt or that mathematics is more important than other education competencies like creativity or ability to work with others or character—they are just illustrative of reaching goals in any domain.

References

ASER (2015). Annual status of education report (rural): 2014. New Delhi India: ASER Centre.

Beatty, A. and Pritchett, L. (2012) “From Schooling Goals to Learning Goals: How Fast Can Student Learning Improve?” CGD Policy Paper 012. Washington DC: Center for Global Development. http://www. cgdev.org/content/publications/detail/1426531.

Bruns, B., & Luque, J. (2014). Great teachers: How to raise student learning in Latin America and the Caribbean World Bank Publications.

Chang, M. C., Shaeffer, S., Al-Samarrai, S., Ragatz, A. B., de Ree, J., & Stevenson, R. (2014). Teacher reform in Indonesia: The role of politics and evidence in policy making World Bank Group.

Glewwe, P., Hanushek, E. A., Humpage, S., & Ravina, R. (2014). School resources and educational outcomes in developing countries: A review of the literature from 1990 to 2010. In P. Glewwe (Ed.), Education policy in developing countries (pp. 13-64) University of Chicago Press.

Inter-Agency Commission (1990). World declaration on education for all and framework for action to meet basic learning needs.

Learning Metrics Task Force (2013). Toward universal learning No. 2). Montreal and Washington: UNESCO Institute for Statistics and Center for Universal Education at the Brookings Institution.

McEwan, P. (2014). Improving learning in primary schools of developing countries: A meta-analysis of randomized experiments. Review of Educational Research.

Mehtabul, A., & Kingdon, G. (2014). Assessing teacher quality in India. IZA Discussion Paper.

OECD (2014). PISA 2012 results: What students know and can do.

Pritchett, L. (2013). The Rebirth of Education: Schooling Ain’t Learning. CGD Books.

Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417.

Slater, H., Davies, N. M., & Burgess, S. (2012). Do teachers matter? Measuring the variation in teacher effectiveness in England. Oxford Bulletin of Economics and Statistics, 74(5), 629.

Universal Declaration of Human Rights (1948).