Measuring progress in education is no simple matter. With the shift from the Millennium Development Goals (MDGs) to the Sustainable Development Goals (SDGs) in 2015, the global community put new emphasis on improving the quality of education rather than only tackling access to education. Accordingly, SDG 4 captures learning outcomes, including foundational literacy and numeracy.

Target 4.1 frames the overall goal:

“By 2030, ensure that all girls and boys complete free, equitable and quality primary and secondary education leading to relevant and effective learning outcomes”

and indicator 4.1.1 sits below this:

"Proportion of children and young people (a) in grades 2/3; (b) at the end of primary; and (c) at the end of lower secondary achieving at least a minimum proficiency level in (i) reading and (ii) mathematics”

Developing a methodology for measuring this indicator across countries has been a central focus for the UNESCO Institute for Statistics (UIS) as the custodian agency, and for the wider global education community.

To determine the composition and methodology of all SDG indicators, an Inter-Agency Expert Group on Sustainable Development Goals Indicators (IAEG-SDGs) meets twice a year. Ahead of the 10th meeting in October 2019, the IAEG opened up a public consultation on several proposals to revise, remove, replace and add indicators. Among many suggestions put forward, one suggested replacing SDG indicator 4.1.1 as cited above with a more general indicator, combining it with a slightly revised version of indicator 4.6, capturing literacy and numeracy of the general population:

"Proportion of the population achieving at least a minimum level of proficiency in (a) literacy and (b) numeracy skills, by sex, age and educational level”

RISE responded by publishing a blog calling on the community to make their voices heard in the public consultation process.

RISE wasn’t alone. A plethora of global and grassroot organizations, multilateral institutions, Civil Society Organisations, think tanks, and academics came out in support of maintaining SDG 4.1.1. As the consultation was public, the responses are openly accessible (via Excel file download). It was encouraging to read responses from organisations across the global education community who were standing up for the need to measure early years learning and progress through schooling. The comments were remarkably unified around three main reasons for supporting the preservation of 4.1.1:

1. Look at the system: 4.1.1 shows what’s going on inside the education system

“4.1.1 is the most important measure of system capability to deliver the competencies identified by national curricula” – Economist, Global Partnership for Education (GPE) ¹

“[I]ndicator 4.1.1 is very important to measure the goal in terms of proficiency on each grade of the primary and secondary education. By distinguish[ing] between levels of education you could formulate policy programs more effective and well targeted” - Director’s Advisor and Coordinator of SDG Indicators Working Group, National Administrative Department of Statistics of Colombia (DANE)

“The concepts of learning outcomes (target 4.1) and functional literacy (target 4.6) are different. While the first concept is related to the functioning of the education systems, the curriculums established there and the results of the teaching and learning processes in schools, the concept of functional literacy is defined regarding the necessary reading and numeracy skills sufficient to function in particular contexts.” - Senior Research and Learning Advisor, USAID

The overwhelming conviction of respondents was that 4.1.1 is critical as a quality measure of primary and lower secondary education and, by extension, an indicator measuring the functionality of an education system. It tells us how well education systems do what they are supposed to do—produce learning—and is sensitive to more rapid changes between cohorts (for instance, improvements or deteriorations in learning as a response to reform). It holds valuable discriminatory power on learning and changes in learning at different stages of schooling, and so it can inform targeted policies. 4.1.1 is called a “flow” indicator because it measures learning at a specific point and for a different sample of children as they “flow” through the system. By contrast, the proposed indicator is about “stock” because it captures the skill level of adults which is much more static due to slow changes to the population. Impact of reforms on this indicator would be very small and only observable with a considerable time lag.

The comments on the proposed change showed strong support for the SDGs to continue sending a clear signal that measuring learning through stages of school is imperative. As Silvia Montoya and Jordan Naidoo put it: 4.1.1 “determines whether education systems are fit for purpose in delivering the skills needed to ensure lifelong learning for all – the overarching ambition of SDG 4.”

2. Look at the early grades: 4.1.1. measures learning early on

“It is vital that the indicator in the early years of schooling (grades 2 and 3) is maintained. Evidence is clear - inequalities in education start from the early years. Unless identified and tackled at that stage, they will continue to widen, and the SDGs will fail” – Professor, University of Cambridge

“The proposal “risks making it impossible to intervene early to help children master foundations of learning (literacy and numeracy) when still in lower elementary school grades.” – Regional Coordinator, Twaweza East Africa

“We are committed to the continued use of this measure due to the importance of capturing early foundational skills as children without these become left behind in education and cannot learn.” – Statistics Adviser, DFID

It was equally relevant to respondents that the current indicator specifically emphasizes early years measurement. Research shows that when children fall behind early, they often stay behind. That’s why reliable data is needed on this pivotal stage for governments to make informed decisions on interventions in the early years. The proposed change would risk losing this focus, and respondents even voiced concern over not achieving SDG 4 altogether if stakeholders fail to acknowledge the importance of foundational skills. In 2019, RISE introduced “universal, early, conceptual, and procedural mastery of basic skills” as a vital concept that education systems need to prioritize if they are to become coherent for learning.

3. Look at the progress: 4.1.1 now has a feasible methodology

“[S]o many 'learning' experts have developed a consensus on how to measure SDG 4.11. (...) even on how to map against citizen led assessments (...) There has been a powerful culture of collaboration created spearheaded by UNESCO-UIS-GEM report and so many allies of learning around the world -where voices and technical views have been heard” – CEO of ITA, PAL Network and ITA

“Target 4.1.1 is arguably the most important indicator with the SDG 4 framework as it addresses the fundamental purpose of education and schooling. (…) The work that has gone into measuring and reporting indicator 4.1.1 has been substantial” – Program Officer, BMGF

“There has been much work done on making 4.1.1 internationally comparable and this has gained tremendous momentum and is turning the tide of the focus of education toward learning” – Education Policy Manager, ONE Campaign

Strides have been made, by UIS and others (such as the Technical Cooperation Group on the Indicators for SDG 4, the Global Alliance to Monitor Learning and the Global Education Monitoring Report), to develop a feasible methodology for learning at the different stages of schooling, balancing conflicting priorities of international comparability and (national) capability for data collection. In particular, there has been considerable progress in harmonizing methodologies in national and cross-national assessments to make the indicator comparable (by linking assessments that use different definitions of “minimum proficiency”), as well as in developing a common protocol for data reporting. However, there have also been differing views on this, including suggestions that measurement is arguably something better designed nationally than imposed from a global level.

Several respondents commended on the progress that has been made on the methodology, the momentum, and the culture of collaboration in the past years. In fact, only in 2018, the early-grade indicator, SDG 4.1.1a, measuring learning in grades 2/3, was upgraded from Tier III to Tier II, acknowledging substantial improvements to the suggested methodology and highlighting its value among the indicators.

The outcome

During the subsequent IAEG-SDGs meeting in November in Addis Ababa, in part as a result of the many organisations supporting 4.1.1, the IAEG made two decisions to strengthen the indicator. Not only was the above proposal rejected, but SDG 4.1.1a was further upgraded from Tier II to Tier I, meaning that it is recognised as an indicator that is now “conceptually clear, has an internationally established methodology and standards are available, and data are regularly produced by countries for at least 50 percent of countries and of the population in every region where the indicator is relevant” (as per the IAEG’s Tier classification).

It is promising to see so many organizations successfully rallying around the need to preserve SDG 4.1.1. This not only underlines the need to continue to strengthen learning assessments of all kinds, but also powerfully illustrates the role that the education community plays in advocating for quality education, and their wider role in supporting a shift to align education systems around learning outcomes. It is important to acknowledge that everyone has a stake in ensuring the development of systems that deliver learning for all—and that the measurement of learning early on and across schooling is an important part of achieving this goal. Setting up such systems will not be easy; the proposal to replace 4.1.1 reflects these tensions. It will take the sum of lessons learned across countries, communities on the ground, and within the global community, as part of a continued dialogue to build coherent systems and to ensure we don’t miss measuring learning when it counts most.

¹ All quotes taken from IAEG’s public consultation (Excel file download).