“What can education systems research learn from health systems research?” Kara Hanson of the London School of Hygiene and Tropical Medicine asks in a new working paper for the RISE series.

Hanson traces a complex set of issues around researching complex adaptive systems. My three main takeaways are:

1. Do study systems
2. Don’t get too caught up on a single model or ranking
3. Be question-drive: Focus on how and why outcomes change, beyond whether a reform has impact

The fields of health and education face some similar challenges, including that of delivering services with often-universal commitments. Both require getting workers the needed support and supervision to deliver a quality service, including in remote, poor, and/or rural areas.

Despite the temptation for academics in these fields to take on a smaller, more manageable object of study—“very narrowly defined questions and impact evaluations of policy interventions with relatively short causal chains”— the gains to systems-level research can be far greater.

1. Do study systems

The good news: health systems research built up a field from scratch, and that research matters. Even if it’s not as flashy to do something better as to invent something new, the payoff can be much greater. For instance, 62.5% of child deaths in low-income countries could be averted by improving the delivery and utilization of health services, without any new technologies at all (Leroy, JP et al. 2007 in Hanson). Crucially, “the way the health system is organised and financed has been shown to influence system-level performance, and is amenable to change regardless of the absolute level of resources available” (OECD 2004 in Hanson). In other words, system characteristics do affect health outcomes, beyond how rich the country is and how much it spends.

2. Don’t get too caught up on a single model or ranking

The bad news, in case you were waiting for it: there is not a definitive model of health systems (or how to research them) that we can copy-paste over to education. Common to education and health systems research is the problem of representing actors and the relationships between them in a way that adds analytical value, and the challenge of studying systems change at scale. Hanson recommends bringing together a broad range of disciplines and methods “which permit an understanding of how and why interventions work, not just whether they work. Systems research should therefore be strongly theory-of-change led.”

Using a theory of change to evaluate the Affordable Medicines Facility–malaria. See Tougher S, the ACTwatch Group et al. 2012

When mapping complex systems, there’s a tricky tendency to add lines and arrows until PowerPoint spaghetti results. But models are supposed to simplify, and to identify which linkages are critical. Otherwise, “building blocks” inventory approaches can become stumbling blocks by leaving out the systemic elements necessary to understand outcomes.

Hanson 2015, UKFIET presentation. How models of systems add on to each other

Indeed, what distinguishes system research “is its recognition of the linkages and interactions among system components, the variety of forms of signal that induce change (incentives, information, accountability) and the responses (intended and otherwise) that these changes provoke.”

But fitting these elements into indicators to rank national systems in a league table, as the World Health Organization tried to do in 2000, turned out to be a near-impossible task—a tip for what education systems research should perhaps avoid.

3. Be question-driven

After the Ebola outbreak in 2015, “health systems strengthening” received sudden renewed attention in West Africa. Amanda Glassman of the Center for Global Development commented in DevEx: “Each country’s situation will be different. But we should start from the quality and coverage problem that needs to be fixed, rather than broad categories of health system inputs on their own.”

The same is true for education—and that’s exactly why a better understanding of education systems is so urgently needed in order to improve learning outcomes.

In his own new RISE working paper, Lant Pritchett draws an analogy to a car. Cars that work can go further with more gas, “but if I have a car whose transmission has failed then adding more gas will not add to miles travelled… if the system does not add up to a functional whole, the causal impact of augmenting individual elements is completely unpredictable.”

Of course, the way cars produce miles travelled is more straightforward than the way education systems produce learning. Hanson urges researchers to “develop a set of models adapted to the specific actors, functions and relationships in the education sector.  But the field will also benefit from efforts to make such models more analytical, and particularly to focus on the policy signals (incentives, information flows, structures of accountability) that will stimulate and sustain system change.”

Hanson’s full paper is available here.