Which African country is making the most progress in teaching kids to read by the time they reach third grade? Which language-of-instruction policies are most effective in early literacy teaching? Which country is getting the most children to complete primary school equipped with basic literacy, numeracy, and critical thinking skills? Which countries in Africa are managing to expand secondary school coverage rapidly while still raising learning outcomes? 

The answers? We don’t know. We don’t know because there is no way to know. Comparable data on learning outcomes over the past 10 years in sub-Saharan Africa simply does not exist. There is a small exception: by 2020, the regional test serving Africa’s francophone countries, PASEC (Programme d’Analyse des Systèmes Educatifs de la CONFEMEN) will produce a second round of data on learning outcomes at the second and sixth grade levels for 10 countries that is fully comparable to 2014 results. But how will the progress in these countries compare to that of the region’s other 40 countries, or to the largest countries: Nigeria, Ethiopia, South Africa? And what is happening at the secondary level? Impossible to say. Only Senegal and Zambia have participated in any cross-national learning measurement at that level, joining PISA for Development for one round.

Figure 1. Regional Assessment Coverage of African Countries: PASEC and SACMEQ

What Latin America tells us about cross-national testing 

It doesn’t have to be this way. Latin American countries have followed a different path, as we document in a new paper with Nancy Birdsall. Over the past 20 years, virtually all countries in the region have collaborated on a regional test of math and language skills in third and sixth grade, and many have also joined PISA to benchmark the learning of 15-year-olds. When the region launched ERCE in the late 1990s, most of its participating countries had not developed national assessment systems; the “horizontal” collaboration among countries in comparing curriculum standards and learning domains, identifying different levels of learning, generating test items, and agreeing on sampling and testing protocols contributed directly and efficiently to the growth of national testing capacity. What has been called “a culture of testing” today is broadly diffused across the Latin American region, with virtually all countries measuring learning nationally and cross-nationally at regular intervals through technically robust programs of assessment. 

Equally importantly, the region’s experience—and indeed, that of the OECD—shows that cross-national learning results resonate with policymakers and the public in ways that national assessments cannot. Seeing a neighboring country make faster learning progress is a wake-up call. It is impossible to assign causality to cross-national learning measurement alone, but a striking number of Latin American countries have implemented large-scale reforms aimed at raising learning over the past 15 years (Chile, Ecuador, Peru, Colombia, Mexico, Brazil), and in several cases have seen truly impressive learning gains. In a sense, even those cases where reforms proved politically unsustainable attest to how ambitious Latin America’s efforts to improve education quality have been. It is difficult to imagine politicians assuming the risks of major reforms without public awareness of, and dissatisfaction with, student learning levels. In case studies we profile, presidents and education ministers actively used cross-national test results in their communications campaigns to sell and sustain big reforms.

To improve learning, Africa needs comparable data 

African countries are making strong efforts to promote region-wide integration in other areas, such as trade. But the education community seems to throw up its hands and assume that two separate, disconnected, sub-regional testing efforts that grew out of colonial ties and never reached all countries in the region are destiny. And while one of those sub-regional tests, SACMEQ, had a very promising start serving 15 countries in Southern and Eastern Africa from 1999–2008, it has not released results since then. 

It is time—indeed, past time—to support a region-wide test that serves all countries in sub-Saharan Africa. It makes sense to build on the current institutional capacity of PASEC, which is already expanding across the region. But it is equally important to integrate technical capacity, test items, sampling frames, and experience from SACMEQ, to ensure that new test results are comparable with prior results and provide crucial perspective on countries’ recent progress. It also makes sense to align this expanded regional effort more closely with other regional and international tests, by building in common “linking” items to make results comparable internationally and harmonizing the grades tested. 

For example, PASEC currently uses an oral test to measure reading fluency and basic math skills in second grade. Oral tests present higher costs, time, and standardization challenges than paper-based tests and thus are harder to scale up. A decision to align around a paper-based test in third grade would lower costs and generate robust, internationally comparable results at both the first two measurement points recommended for monitoring the Sustainable Development Goals (SDGs): Grades 2 or 3, and Grade 6. Latin America’s ERCE, which uses a paper-based test to measure learning at Grades 3 and 6, has already signaled its willingness to collaborate with African partners in this area, to lower the costs of item development and test design.

At the SDG’s third recommended point of learning measurement (lower secondary), PISA is likely to remain dominant. This has many advantages—a larger number of countries already participate in PISA than in any other cross-national test, and the number is also growing faster than for any other test. Even much lower-income countries see the PISA countries as exemplars to learn from and aspire to. Our case study of Peru found that PISA has influenced the country’s education policies through multiple channels over the past 20 years. No regional test has had policy influence equal to that of PISA. But countries require a high level of technical capacity to participate in PISA and it is expensive (US$ 1–2 million per country per year). The OECD is very aware of these issues; PISA-D was specifically designed to help lower-capacity, low-income countries join. Substantial support from the donor community will be needed to help low-income countries in Africa and other regions make this high-return investment in better education systems. 

Four years after the SDG targets were adopted, there is still no baseline measure of learning levels at third grade, sixth grade, and the end of lower secondary for about half of the countries in the world and for most of the poorest countries in the world—the countries whose children will benefit most from schools that deliver learning. Even a goal of establishing such a baseline by 2021 seems audacious, given the slow progress of the past four years. Latin America’s experience provides a clear example of how coordinated and efficient application of resources can build countries’ capacity, strengthen their focus on learning, support better research, and stimulate countries to adopt and diffuse reforms that raise learning. Raising learning raises children’s lives and potential. This can happen in Africa too.

This blog was originally posted on the Center for Global Development website on 12 September 2019 and has been cross-posted on the RISE website as outlined under the Creative Commons Attribution-NonCommercial 4.0 International License.