Conceptualising ‘coherence’ within an education system

The question posed by Pritchett – “Why is it that (some) education systems have succeeded so impressively at some aspects and yet failed so dismally at others?” (2015:39) – is particularly relevant for Ethiopia. Access to primary education in Ethiopia has expanded dramatically over the past 20 years, but learning outcomes have not kept pace over the same period – in spite of the explicit focus of GEQIP reforms to improve learning outcomes since 2009 (Woldehanna et al., 2016), and of O-Class reforms to improve school readiness since 2010. Pritchett’s proposed explanation for such an outcome is that, although an education system may “proclaim the goal of achieving uniformly high levels of learning,” it can become:

“for a variety of reasons, coherent only to the purpose of schooling. That is, systems of relationships of accountability [are] adequately coherent to produce continued progress in enrolment and grade attainment expansion [...] but [are] incoherent for learning both within, between, and across elements of accountability”. (Pritchett, 2015:39)

The RISE and ELP research further hypothesises that a reason for learning outcomes not keeping pace is due to the increased diversity of children within school as it has moved from an elite to a mass system within the period of two decades. This raises a further question about whether there is coherence with respect to improving learning for those who face the greatest disadvantages due to their background. As our analysis of Young Lives data shows, for example, first generation learners are less likely to be learning (Iyer et al., forthcoming). The challenge of achieving equitable improvements in learning is of particular importance to Ethiopia, which has a very diverse population, among which over 90 different languages are spoken.

Pritchett (2015) conceptualises an education system as comprised of a series of accountability relationships between four different sets of actors: citizens, parents and students; the state; organisational providers of schooling; and teachers (2015:15–see Figure 1). The accountability or ‘principal-agent’ relationships between each group of actors (politics, compact, management, and voice/client power relationships) consist of four ‘design elements’: delegation, finance, information, and motivation (Pritchett, 2015:16).

Pritchett (2015) combines these sets of actors and design elements into a four-by-four ‘diagnostic for systems of basic education’, and proposes that “governmental (sub) systems work when there is an adequate flow of accountability in the system” (2015:18). Pritchett argues that ‘coherence’ within, between and across accountability relationships is a key concept for the analysis of education systems – in particular, to consider whether a system is ‘coherent for learning’, and if not, to identify the nature of incoherence within the system, and how this inhibits the achievement of quality learning outcomes.

Figure 2 provides an annotated version of Pritchett’s four-by-four diagnostic, including the three types of in/coherence described by Pritchett that may be identified through a systems approach. In the following section, we consider how we can adapt Pritchett’s model of accountability and four-by-four diagnostic to understand in/coherence within the Ethiopian education system.

While Pritchett’s (2015) model provides a useful framework to understand accountability relationships and to analyse in/coherence in an education system, its starting point is from the perspective of a system in which decisions are made at the centre. As the Ethiopian education system is largely decentralised, with the transfer of expenditure and decision-making authority from upper (federal and regional) to lower (woreda) tiers of government, some adaptations are required before Pritchett’s model can be applied in such a context. Service providers in the Ethiopian system are strictly accountable to woreda governments for producing results and in turn, the woreda authorities are held accountable by the regional and federal governments for delivering basic services and achieving targets (Khan et al., 2014). Di Gropello’s (2004) work on decentralised education systems in Latin America, and in particular, her ‘sub-national shared responsibility model’ (Figure 3) provides a more suitable model for this structure of accountability relationships. Di Gropello’s model places two different political actors at the centre of the decentralisation process, and distributes the main responsibility for service delivery among them.

This model reflects the way in which the Ethiopian education system was structured following the country’s 1994 Education and Training Policy (ETP), and recent early learning reforms (which are the focus of ELP) have been implemented according to this decentralised model. Since the 1994 ETP, regions have in principle been responsible for developing their education plans and for the majority of education spending. By contrast, the GEQIP reforms of interest to RISE have been designed and implemented in a more centralised way, with engagement of key donors in collaboration with the Federal Ministry of Education. The potential mis-match between the intended decentralised structure of the education system and the centralised nature of the GEQIP reforms is a key area of interest within the RISE Ethiopia research. We hope to compare and contrast the implications of this approach with the more decentralised reforms associated with ELP. It is also important to note that there are considerable regional variations in capacity and the extent of decentralisation within the education system (particularly between ‘established’ and ‘emerging’ regions) in Ethiopia; the reasons for this variation is also a key area of interest for both RISE and ELP.

The three levels of government (national/federal, regional, and local/woreda) are organised hierarchically, and are seen as having mostly complementary roles in service delivery. Kebele (‘village’) councils are also involved in planning for service delivery at the local level, and are a conduit through which the voice of communities is passed upwards to woredas. Similarly, coordination mechanisms such as Parent, Student, and Teacher Associations (PSTAs) influence the flow of information from clients to front line organisations. These actors are implied in our stylised accountability framework with respect to ‘Students, Parents, and Communities’. An understanding of their roles, responsibilities, and relationships to education service delivery is a key part of our research, and will be included in our stakeholder interviews at local levels.

Di Gropello’s model of sub-national shared responsibilty reflects systems which can take advantage of the skills and specificities at each level and the relationships between them. This model particularly recognises the importance of identifying the implications of ‘compact’ relationships between the centre and regional level and between regional and local levels, as well as the ‘voice’ relationship between citizens at the local level and between citizens and the regional level (di Gropello, 2004). According to di Gropello (2004), the complexities inherent in systems which involve “more than one political sub-national actor in service delivery with complementary functions makes the creation of an effective accountability system particularly difficult” (2004:18). Our aim has therefore been to develop a model of accountability relationships within the Ethiopian system which will allow us to capture some of these complexities.

In recognition of the complexity of the different layers of accountability, we have adapted di Gropello’s sub-national shared responsibility model to develop a stylised accountability framework for Ethiopia’s education system (Figure 4). This includes all the relationships and design elements of Pritchett’s model, but re-configures the relationships following di Gropello’s decentralised structure, and includes groups of actors which make sense in the Ethiopian context. This also includes adding ‘compact’ relationships directly from the state to Regional Education Bureaus (REBs), and from REBs to Woreda Education Offices (WEOs); and adding ‘voice’ relationships from service users to REBs and to WEOs.

It should be noted that we have only included one ‘sub-system’ within the education system in Figure 4 – namely, the education administration. Actor mapping work for RISE and ELP points to three sub-systems in the Ethiopian education system: the education administration, the finance sub-system, and the political sub-system (see Figure 5). Responsibilities for education are divided between these three sub-systems; for example, while the education administration is responsible for policy development, it does not have control over teacher recruitment, nor the local allocation of funds; these activities are covered by the political and financial sub-systems respectively. Additionally, there are ‘design elements’ (e.g., finance) of accountability relationships in which the education administration is not directly involved. For example, for early learning, the formula used to allocate block grants across regions is decided by the House of Federations and cascaded to regions (the finance sub-system). Further, while allocations from regions to woredas might be based on a similar formula, within-woreda allocations to education are determined locally, by negotiation between the woreda finance office, woreda council, and cabinet (local levels of the finance and political sub-systems respectively). While the funding structure for the GEQIP reforms works slightly differently from early learning reforms (for example, the political sub-system is not as obviously implicated in the implementation of GEQIP), it is again the finance sub-system, rather than the education administration, through which funds are disbursed.

We have also made an important addition to our stylised accountability framework for Ethiopia’s education system by including donors as a key set of actors. Donors are not included as a group of actors in Pritchett’s model, nor in di Gropello’s approach. However, donors such as the World Bank, DFID, and others have been instrumental in the design and implementation of the GEQIP reforms in Ethiopia over the past 10 years. It is therefore crucial to understand the nature of accountability relationships between donors and other groups of actors, including those at the federal, regional, woreda, and school levels, in order to understand the nature of in/coherence within the system. We anticipate that NGOs are less directly involved in the design and implementation of national reforms in the same way that donors are in Ethiopia, but rather involved in the delivery of reforms at regional level. As this remains an empirical question (to be explored through further actor mapping and key stakeholder interviews), we do not include NGOs as a separate group in our model at this stage. As discussed in the following section, Kingdon et al.’s (2014) political economy framework provides a useful starting point to consider key actors beyond those identified in the Pritchett and di Gropello models.

The extension of Pritchett’s model to reflect accountability relationships within a decentralised system, and to include donors as a key group of actors, also has implications for the four-by-four diagnostic presented in Figure 2. As evident from Figure 6 below, an equivalent diagnostic for our model of the Ethiopian education system is much more complex than Pritchett’s original. Firstly, with additional sets of actors (donors) and ‘levels’ of actors within the system (regional stakeholders; woreda-level stakeholders), the number of accountability relationships grows. Secondly, we have extended the structure to permit one principal (e.g., the state) to hold one type of accountability relationship with multiple agents (e.g., from the state to the Ministry of Education, and from the state to Regional Education Bureaus).

Identifying in/coherence within, between, and across this expanded number of accountability relationships will inevitably be a more complex process than via Pritchett’s original four-by-four diagnostic. Illustrating this complexity, we have prepared a version of our adapted matrix annotated with an example of questions that could be explored (see Appendix 1). As discussed in the concluding section, this complexity means that we have to make practical considerations in focusing our approach to systems research in RISE and ELP. Before this, in the following section, we discuss how insights from a political economy approach can add nuance to the ways in which we examine accountability relationships and in/coherence within our adapted framework for the Ethiopian education system.

Figure 6: An adapted system diagnostic for the Ethiopian basic education system

* Zonal Education Office is important in some regions (e.g., SNNP, where a zone typically overlaps with a language/ethnic group) but has no role in others.

In their rigorous review of the political economy of education systems in developing countries, Kingdon et al. (2014) argue that:

“Education reform does not take place in a vacuum, but under specific constraints and opportunities, many of which are politically driven, shaped by the interests and incentives facing different stakeholders, the direct and indirect pressures exerted by these stakeholders, and by formal and informal institutions. Each of these factors influences different aspects of education reform, whether policy design, financing, implementation or evaluation.” (Kingdon et al., 2014:5)

The ‘direct and indirect pressures’ described here, as exerted by stakeholders and institutions, can be seen as analogous to Pritchett’s accountability relationships between different actors, while the ‘interests and incentives’ of different stakeholders described by Kingdon et al. can be understood as the ‘delegation’ and ‘motivation’ design elements of accountability relationships in Pritchett’s framework (see Table 1). Table 1 includes an attempt to align the five themes in Kingdon et al.’s framework (each with a series of sub-questions to guide analysis) with Pritchett’s conceptualisation of accountability relationships within education systems. It further indicates the planned methods which RISE and ELP will use to explore these questions.

Kingdon et al.’s description of an education system as a ‘value chain’ in which “stakeholders both [make] decisions and [operate] at various levels within both national and international environments” also includes what Pritchett might describe as ‘incoherence’: “there are numerous opportunities along the entire [value] chain for unfavourable leakages and corrupt behaviours to undermine efforts to achieve the ultimate goals: delivery of educational services in an equitable, efficient and effective manner” (Kingdon et al., 2014:5-6).

The synergies between Kingdon et al.’s framework and Pritchett’s model of accountability relationships indicate the value of using a political economy approach to examine in/coherence within an education system. Moreover, there are elements of Kingdon et al.’s political economy framework which are not explicitly included in Pritchett’s model; incorporating these elements can add nuance to our understanding of the education reforms of interest to RISE and ELP in several ways.

By asking “who are the key stakeholders with an interest in the sector?” (Kingdon et al., 2014:8), we are able to go beyond Pritchett’s four defined groups of actors to include international donors and development partners as a key group of actors in the Ethiopian education system, as discussed above. Questions about “direct and indirect mechanisms available to different groups to exercise their power” (Kingdon et al., 2014:8) also encourage us to consider stakeholder relationships not only according to a unidimensional ‘principal-agent’ structure, but also the multidimensional nature of power dynamics between different actors. For example, rather than understanding the ‘delegation’ element of a ‘compact’ accountability relationship exclusively in the terms described by Pritchett – as a “specification of what the principal [i.e., the Ministry of Education] wants from the agent [i.e., a Regional Education Bureau]” (2015:19), a political economy approach would encourage us to consider what these actors might want from each other, the direct and indirect mechanisms by which different (and potentially competing) interests are prioritised, and the eventual implications of these dynamics for in/coherence in the system.

Table 1: Linking Kingdon et al.’s (2014) political economy framework with Pritchett’s (2015) education system diagnostic

It is important to acknowledge here that we will not treat accounts given by key stakeholders as purely ‘factual’ information during interviews or during analysis. Instead, our understanding of accountability relationships within the education system will consist of potentially competing perspectives (which are themselves influenced by particular motivations and experiences) of different stakeholders. Considering in/consistency between stakeholders’ accounts (and the implications of this for our wider research questions) will be a key focus for our analysis. Additionally, certain elements highlighted by Kingdon et al.’s political economy framework – such as corruption, rent-seeking, and patronage – are clearly sensitive topics to discuss within key stakeholder interviews. Careful planning is required to consider whether, how, and with whom these topics are appropriate to be explored during fieldwork. Our proposed approach is not to ask about these issues directly during the interviews.

Finally, Kingdon et al.’s questions about the interests and incentives of different actors – and how these actors, interests, and incentives have varied over time – are particularly useful to guide our interest in change over time within the Ethiopian education system. Pritchett’s model supports a cross-sectional understanding of in/coherence within a system, but incorporating Kingdon’s approach allows us to consider, for example, if any current in/coherence within the education system can be attributed to conflicting interests and incentives of different actors from the design phase to the implementation phase of education reforms.

This document outlines a conceptual basis for adopting a systems approach in the RISE and ELP projects, which aim to understand the extent to which the Ethiopian education system is coherent for equitable learning. In order to develop a stylised model of accountability relationships in the Ethiopian system, we have adapted Pritchett’s (2015) model of accountability relationships and di Gropello’s (2004) ‘sub-national shared responsibility’ model. Additionally, we have drawn on Kingdon et al.’s (2014) political economy framework in order to consider change over time and power dynamics within accountability relationships of interest.

Following the adapted system diagnostic presented in Figure 6, in order to gain a comprehensive understanding of whether the Ethiopian education system is coherent for equitable learning, we would need to examine the four design elements of each of the eight accountability relationships (politics, management, voice, client power, and four different ‘levels’ of compact relationships) within the decentralised system. However, in practical terms, this would be extremely challenging for both RISE and ELP, and a series of choices must therefore be made. This is particularly important so that we can prioritise areas for attention in order to achieve a key objective of both RISE and ELP – to provide policy makers with actionable information to help guide the delivery of quality, equitable learning at scale. We will not explore all of our specific logistical concerns in detail here, but it may be useful for others adopting a systems approach to be aware of some of the key decisions the Ethiopia research team will be considering:

1. Sub-systems. Understanding the accountability relationships within and between the education administration, finance, and political sub-systems is undoubtedly important in order to fully understand coherence within the Ethiopian education system. In both RISE and ELP, while we will first examine linkages between these three sub-systems from the perspective of stakeholders within the education administration (Figure 4), we will need to consider whether and how to engage with the finance and political sub-systems (Figure 5). Although it does not have sole responsibility for GEQIP or early learning reforms, the education administration remains the primary sub-system of interest in order to understand the design and implementation of these reforms, and to understand system in/coherence for equitable learning.
2. Accountability relationships. Informed by the literature around decentralisation and our adapted framework for Ethiopia’s education system, our key stakeholder interviews will focus on the compact, management, voice, and client power accountability relationships. Given responsibilities for design and implementation of the programmes, more attention in the first stage will be given at the national level for RISE (as GEQIP is a federal project) and at the sub-national level for ELP. Given the structure of the Ethiopian system, the politics relationship (from citizens to the state) will initially receive less attention. However, we will identify empirically the extent to which this relationship should be included further; for example, this relationship may be explored in more depth at local levels (i.e., between parents and woreda councils).
3. Adopting a multi-stage approach to fieldwork. In light of the exploratory nature of this work, we plan to conduct multiple stages of fieldwork. This seems particularly important since we anticipate that processes affecting the education system will evolve over the period of the research; this is already apparent with the shift from the second to third phase of GEQIP, which has implications both for the emphasis on ‘equity’ within the system and for ‘O’-Class reforms. This staged approach will involve starting at national and regional levels and then progressing to woreda levels at a later stage for both RISE and ELP. Adopting multiple research methods (including actor mapping, policy analysis, and key stakeholder interviews) over several stages of fieldwork will support an iterative approach to data collection and analysis, in which findings from initial stages allow us to refine the focus of later stages of the work.

This approach also allows us to engage with stakeholders throughout the research process, which has already proved beneficial. At a country engagement event in October 2017, key education stakeholders advised that areas of interest for the RISE research could include both horizontal misalignments in the system (for example between curriculum and assessment reforms) and vertical misalignments (with respect to information and resources flowing from federal to school levels) (Woldehanna and Yorke, 2017). Overall, we anticipate that this multi-stage approach will ultimately lead to a richer and more nuanced understanding of in/coherence around equitable learning in the Ethiopian education system.

There is a growing interest in adopting a systems approach to education research, particularly in order to understand and address the causes of the ‘learning crisis’ in low and middle-income countries. We hope that these reflections on our approach to adapting existing conceptual frameworks to understand a specific education system, and on moving from conceptual to more practical considerations, will be of help to others conducting systems research. In turn, it is hoped that adopting a systems approach will ultimately enable education systems to effectively provide equitable, quality learning outcomes for all.

¹ This note has been prepared in collaboration with the RISE and ELP Ethiopia teams to inform the design of the research. The authors would like to thank Belay Hagos, Alula Pankhurst, Pauline Rose, and Louise Yorke for their feedback and suggestions on earlier versions of the note.

² For a summary of RISE Ethiopia research objectives, see https://www.riseprogramme.org/countries/ethiopia/rise-ethiopia-research-overview-technical.

³ Using diagnostic tools and policy information, the Systems Approach for Better Education Results (SABER) produces comparative data and knowledge about education system policies and institutions. It evaluates the quality of those education policies against evidence-based global standards, with the aim of helping countries systematically strengthen their education systems. For more information, see: http://saber.worldbank.org/.

Beyene, G. and Tizazu, Y. (2010). Attitudes of teachers towards inclusive education in Ethiopia. Ethiopian Journal of Education and Sciences, 6 (1): 89 – 96.

Di Gropello, E. (2004). Education Decentralisation and Accountability Relationships in Latin America. World Bank Policy Research Working Paper 3453. Washington, D.C.: World Bank.

Khan, Q., Faguet, J-P., Gaukler, C., Mekash, W. (2014). Improving Basic Services for the Bottom Forty Percent: Lessons from Ethiopia. World Bank Study 90430. Washington, D.C.: World Bank.

Kingdon, G. G., Little, A., Aslam, M., Rawal, S., Moe, T., Patrinos, H., Deteille, T., Banerji, R., Parton, B. and Sharma, S. K. (2014). A rigorous review of the political economy of education systems in developing countries. Final Report. Education Rigorous Literation Review. London: Department for International Development.

Moore, M. (2015). Creating efficient, effective, and just educational systems through multi-sector strategies of reform. RISE Working Paper 15/004. Oxford: Blavatnik School of Government.

Pritchett, L. (2015). Creating education systems coherent for learning outcomes: making the transition from schooling to learning. RISE Working Paper 15/005. Oxford: Blavatnik School of Government

Rose, P., Sabates, R., Alcott, B. and Ilie, S. (2016). Overcoming inequalities within countries to achieve global convergence in learning. Background Paper, The Learning Generation. New York: The Education Commission.

Tesfay, N. and Malmberg, L-E. (2014). Horizontal inequalities in children’s educational outcomes in Ethiopia. International Journal of Educational Development, 39: 110 – 120.

Woldab, Z. E. (2012). School-intrinsic impediments in the provision of primary education for nomadic pastoralist Afar children in Ethiopia. Journal of Educational and Social Research, 2 (3): 347 – 358.

Woldehanna, T. and Jones, N. (2006). How pro-poor is Ethiopia’s education expansion? A benefit analysis of education since 1995/6. Working Paper 23. London: Save the Children.

Woldehanna, T., Rose, P. and Rolleston, C. (2016). A rising tide of access: what consequences for inclusive learning and sustainable development in Ethiopia? [Blog] UKFIET. Available at: https://www.ukfiet.org/2017/a-rising-tide-of-access-what-consequences-for-inclusive-learning-and-sustainable-development-in-ethiopia/ (Accessed 22 January 2018).

Woldehanna, T. and Yorke, L. (2017). Launch event highlights the need for alignment in Ethiopia’s education system reform programme. [Blog] RISE. Available at: https://www.riseprogramme.org/node/540 (Accessed 01 March 2018).

Woodhead, M., Rossiter, J., Dawes, A., and Pankhurst, A. (2017). Scaling up early learning in Ethiopia: exploring the potential of O-Class. Young Lives Working Paper 163. Oxford: Young Lives.

World Bank (2017). Program Appraisal Document, General Education Quality Improvement Program for Equity (GEQIP-E). Washington, D.C: World Bank.