A challenge with comparative education research is that we lack a globally comparable measure of the real outcomes of education – actual learning or skills. Instead, we make do with a patchwork of international and regional assessments, each with different country coverage and especially few that allow for comparisons between high-income and low-income countries. Justin Sandefur makes the case for a global standardized assessment here.

So naturally researchers get pretty excited whenever new comparative assessments do come out.  Recently Lant Pritchett blogged about the latest round of the OECD international assessment of adult skills (PIAAC), which included for the first time measures for Jakarta, showing the dismally low levels of skills even in the capital city of a typical middle income country like Indonesia (Indonesia is 127 of 216 countries by PPP GDP per capita).

This prompted me to look at the World Bank’s new skill survey of working age adults in urban areas of developing countries (STEP) that includes a literacy assessment calibrated to the same scale as PIAAC, thus allowing for comparisons.

Two striking findings emerge;

First the good news. Low-income Vietnam which is a huge positive outlier in other international tests, also performs very well here (Vietnam achieves OECD levels on PISA, and on the TIMMS-scaled questions from the Young Lives survey). Vietnam (urban) performs just below the OECD average overall, but better than expected for a country at that income level. One of the first research programmes funded by RISE is focused on better understanding of the superior performance of Vietnam. 

Literacy correlates with GDPPC, but the deviations from that association are large and interesting

Source: PIAAC and STEP [marked with a (U) for urban]

Second, the bad news.  Many other low-income countries are around the same shockingly low level as Jakarta. University graduates from Jakarta, Ghana, and Kenya, have lower levels of literacy than the average junior secondary graduate from OECD countries.  Secondary school graduates from urban Kenya and Ghana countries have substantially worse literacy on this assessment than people with only primary school completion in OECD countries. 

University graduates from poor countries have worse literacy skills than junior secondary school graduates from rich countries

But are developing countries catching up?

One way of looking at this is by comparing the skill level of different age cohorts. Generally, across almost all countries younger adults have better literacy skills than older adults. This partly reflects the deterioration of skills over time, but partly also reflects real improvement in school quality over time. If we make the heroic and unrealistic assumption of zero deterioration in skills, then looking at the improvement in skill of younger cohorts can give us an over-generous upper bound estimate of the rate of improvement in school quality. The next chart shows that some progress is being made, but even with these over-optimistic assumptions, low-income countries are still many decades away at current rates of progress from catching up even with current OECD levels (and OECD countries are likely to continue improving also). The chart shows literacy levels just for adults with upper secondary school, to avoid confounding improvement in school quality with the general increase in enrolment. Vietnam has already caught up with OECD levels for 15-24 year olds, but Bolivia, Ghana, and Kenya are still lagging way behind. 

Younger adults have better literacy, implying that schools may be improving, but not nearly fast enough 

Note: This chart shows literacy level for upper secondary graduates only within each age group. 

There is one more twist that we can squeeze out of the data. Hanushek, Piopiunik, and Wiederhold have previously used the PIAAC data to look at the skill level of teachers in rich countries, relative to other professionals. We can do the same here for low-income countries with the PIAAC data. The sample size of teachers is small, but just about enough (100-150 teachers from each country from an overall sample of ~2,000 adults per country).  And the results are not great.

This last chart takes people identified as primary, secondary, or “other” teaching professionals, excluding university teachers. Finnish teachers, perhaps unsurprisingly, come top. In general, the average teacher in most OECD countries has around the same level of literacy (or better) as the average OECD tertiary graduate. Down at the other end, the average teacher in Kenya, Bolivia, and Ghana, has literacy skills lower than the average OECD junior secondary school graduate. It’s also notable here that the average Vietnamese teacher is only at the level of an OECD senior secondary graduate, so Vietnam is able to achieve its high overall skill level without having tertiary skill level teachers. 

How literate are teachers?

The conclusions from these data support earlier findings, namely that there is a crisis in learning levels in low-income countries, but that this crisis extends beyond just early grades, to adults, and even university graduates and school teachers. Furthermore, at present rates of progress, it will take many years for low-income countries to catch up to OECD skill levels. Yet as Vietnam demonstrates, it is possible to do better, even on a low-income. The challenge is finding out how to make that happen.