Understanding whether wide-ranging education reforms effectively improve learning outcomes is of crucial importance in many low and middle-income countries where, in spite of gains in school enrolments, learning outcomes remain poor. Ethiopia is one country experiencing the global ‘learning crisis’, and since 2008 the Government of Ethiopia has embarked on an ambitious package of reforms – the General Education Quality Improvement Program (GEQIP) – in order to improve the quality of basic education in the country. In this paper, we consider a suitable quantitative strategy for investigating the impact of large-scale, complex education reforms, with particular reference to the latest phase of GEQIP in Ethiopia. Quantitatively, randomised control trials (RCTs) have been considered the ‘gold standard’ to evaluate interventions; however, in the field of education, many scholars have acknowledged the limitations of RCTs particularly with respect to their external and construct validity. In light of these and other concerns, we outline our approach to assessing the impact of GEQIP-E on learning outcomes – a longitudinal design which incorporates variations across time and space that we are likely to observe over the course of reform implementation. This design allows us to understand both the impact and, importantly, the processes of implementation, which is essential if we are to understand not only whether but why certain elements of large-scale reforms may – or may not – lead to improved learning outcomes.