In Ethiopia, teachers are expected to deliver a grade-specific curriculum, set out in textbooks and teacher guides. The curriculum is designed to be cumulative in grades, with higher-order skills building on their respective foundations. However, due to slow learning progress in early primary grades, language-based transitions in upper primary grades, and automatic promotion policies alongside socio-economic disadvantage and limited support at home, many children reach Grade 7 or 8 lacking substantially in the prerequisite skills. It is reasonable to argue, therefore, that the development of learning for these pupils is limited by the mis-match between curriculum content and their developing skills base (i.e., excessive distance from the zone of proximal development in Piagetian terms).

Some teachers, of course, will be able to make adaptations in the classroom, to address and mitigate the gap between students’ skills and curriculum expectations. These might include providing extra support or remediation for early low-achievers to enable them to ‘catch up’ and to access the expected grade- specific content. Some interventions focused on this kind of supportive remediation have shown large effects¹ /a> in other contexts where curricula have been described as ‘over-ambitious² . Other teachers will be much less able to make appropriate adaptations in the classroom, owing to weaker pedagogical skills or training, lack of resources, or other challenges in the face of which adaptation may seem impossible or futile, such as very large and heterogeneous classes and lack of knowledge of students’ actual learning levels in the absence of adequate assessment.

Moreover, incentives and training for teachers are to some extent aligned with ‘delivering the prescribed curriculum’ and teachers may perceive adaptation as risking the progress of high achievers whose skills ordinarily keep pace with expectations. ‘Teaching to the top’, however, risks increasing dispersion of pupils’ learning outcomes, raising both inequality and inequity, while preparing a small proportion of pupils for progression to the next stage of education. This last aim is arguably most consonant with the historical structure of many education systems globally, which were not designed to equip children with skills en masse, but to select and prepare an elite group of students. In Ethiopia, under the imperialist system until 1974, more than 90 percent illiteracy prevailed.

The crisis of low learning levels in Ethiopia, as in many sub-Saharan African contexts, affects a majority of pupils. While reducing inequality (dispersion) in learning levels is desirable, it is desirable only to the extent that this reduction comes from ‘raising the floor³, not at the expense of pupils who are on target to succeed. Enabling teachers to raise learning outcomes for low and middle performers is key. Where data are available, as is rare in Africa, estimates of value-added and especially of ‘differential value-added’ may serve as a diagnostic tool to help identify which teachers add more value to pupils’ learning and to the learning of particular groups of pupils; and under what conditions.

Figure 1 below uses data from the Young Lives primary school survey in Ethiopia conducted in 2012/13  in Grades 4 and 5 with pupils typically aged 11.⁴ Young Lives conducted a benchmarking exercise to map national curricular competencies to test data from the survey and created five indicative skill groupings as shown below. The curriculum target or expectation in terms of skills for these grades is benchmarked here as ‘basic’ or at the very minimum ‘emerging basic’ skills. A large proportion of pupils have clearly not reached these levels, especially in numeracy, with only a tiny minority having unambiguously mastered the intended skills.

The Young Lives Upper Primary School Survey, conducted in 2016-17 in the same sites in Ethiopia, following pupils in the same age cohort (typically aged 15 by this time) includes beginning- and end- of-year mathematic assessment data for 9,434 students in Grades 7 and 8 in 271 classes across 63 schools. Figure 2 shows a similar picture to Figure 1 regarding learning levels four years later. This time we report the number of school years behind curricular expectations which corresponds to pupils actual assessed learning levels in Young Lives tests. The vast majority of pupils’ learning is found to be at least three years behind expectations, both in urban and rural areas of the country.

With the upper primary data, it is possible to investigate questions of school- and teacher- effectiveness using very recent data. We focus on the questions of:

1. Does differential teacher effectiveness appear to play a key role in explaining the progress of groups of pupils with different starting points?
2. Do teachers of different levels of ‘overall effectiveness’ tend to achieve this effectiveness by raising learning outcomes of particular groups of pupils?

In relation to question 1, Figure 3 below represents two schools with somewhat different groups of pupils and somewhat different patterns of learning progress. School A demonstrates substantial progress for students with above average prior achievement and limited progress for students with below average prior achievement. In contrast, School B shows a more homogenous group of students (in terms of prior achievement), with more progress among those with below average prior achievement–perhaps at the cost of high-early-achievers’ progress. School A is in fact a school with greater than average ‘value-added’ (effectiveness) while School B is closer to average while it is clear that the two schools generate effectiveness by improving learning among rather different groups of pupils – school A ‘raises the roof’ while School B adds value more evenly across the range of prior learning levels.​​​​​​

In relation to question 2, we examine the question of who teachers are effective for, as classrooms are not groups of homogenous average learners. Specifically, we look at whether ‘apparent’ teacher effectiveness (as measured by overall effectiveness such as a single teacher value-added estimate) links to the prior learning levels, or by heterogeneity in prior learning levels, among the students that a teacher faces.

In Figure 4 above, we contrast the least effective teachers with teachers of average effectiveness. Students are divided into quintiles of prior performance, Q1 to Q5, where Q1 is the lowest performing 20 percent of students from the beginning of the year. We then order teachers, from most- to least-effective and plot how each teacher’s overall value-added is shared across these five student groups; note that teacher value-added estimates are standardised with a mean of 0 and a standard deviation of 1. This means that the least effective teachers show ‘negative’ value- added, but that should be interpreted relative to their peers, not to any other expectation.

A first interpretation is that there is a lot of variation in whom teachers are effective for. This  is most obvious in the middle of the distribution of teacher effectiveness (left hand chart in Figure 4). Here, some teachers are more effective for lowest performers at the cost of higher performers, and vice versa. In contrast, the least effective teachers are perhaps especially ineffective for the most able pupils (i.e., among the 50 least effective teachers, negative columns are dominated by students in Q4 and Q5).Figure 5 presents the picture for the fifty most effective teachers. In order to reach the highest levels of average effectiveness, of course, teachers had to be effective for all the groups of students within their classes (note the relatively equally divided bars for the ‘top 5’ teachers). Among this group of teachers, there is no evidence of ineffective teaching for any particular group, where prior attainment is concerned.

School and teacher effectiveness in the literature is computed based on average progress for the relevant student group; it says nothing about differential effectiveness within classrooms or schools. But if teachers are differentially effective, as Figure 3 implies, then a teacher’s effectiveness (i.e., the residual⁵ that is treated as the teacher’s value- added) may depend on the composition of students in the classroom and on their teaching strategies employed, which will in turn be influenced by their training and by how performance is judged. In classrooms with extreme heterogeneity in achievement–such as in Ethiopia Grade 7/8, where variation in prior achievement can be equivalent to five or more grades in one classroom–it may be instructive to review teacher effectiveness for different sub-groups and in relation to classroom composition.

For example, high-early-achievement might indicate ‘preparedness’ to learn. It might be an effective strategy for a teacher to focus attention on the highest-achieving and plan for the advances of this group to carry the rest of the classroom. This teacher then presents as effective even though a majority of students make little progress, which raises the question: Is the teacher that ‘raises the floor’ more or less desirable than the teacher that ‘raises the roof’?

And what if progress is highest among low-early-achievers but modest among those with high prior achievement? Even if overall effectiveness is lower, a classroom in which the mean achievement at the end of the year depends little on a student’s beginning of year score may be said to show greater equity because it has decreased differences in outcomes across its intake spectrum. Could it be that these classrooms better-represent something about teacher effectiveness/value added that is sought for in the achievement of universal basic skills–or is this simply an artefact of the group’s composition?

This idea of differential effectiveness based on prior achievement—or distribution of prior achievement in a classroom— is linked to the Teaching at the Right Level (TaRL) literature. TaRL research indicates large gains when students are grouped according to skills rather than based on age or number of grades completed. Typical TaRL approaches include some adjustment in the content that teachers deliver and the processes that they use, but they rely on substantial adjustment in the matching between content and student capability. This suggests that there may be ‘untapped effectiveness’ among teachers insofar as a (simple) reallocation of students to classes produces large learning gains, without much else (i.e., although there may be some orientation on how to deliver the new material, there is no teacher replacement up-skilling, retraining, or major resource addition). The TaRL approach may hold the greatest potential in a context of curriculum/skills mismatches that drive differential effectiveness, such as might exist in Ethiopia’s upper-primary grades.

Research along these lines may shed some light on whether Ethiopia’s teachers are ‘ineffective’ for many students because they lack skills or motivation to teach or whether they are ‘ineffective’ for many students because they are being asked to teach material that most children are under-prepared to learn. What if teachers are ‘ineffective’ simply because the students they are tasked with teaching are under-prepared? Is there evidence that teachers might be more effective where the student group is homogenous and/or closer to the level of the curriculum? How then does variation in teacher effectiveness relate to variation in prior achievement? Using the Ethiopia school survey data it might be possible for us to investigate some of these relationships for classes and/or teachers.

Teachers in much of sub-Saharan Africa face large and heterogenous classes in challenging and poorly resourced conditions. Such conditions are particularly demanding in terms of teacher skill and effectiveness, yet in many contexts, the ‘best’ teachers are not found where they are most needed. Further, teachers may be more or less effective for particular groups of students, depending on, for example, the curriculum they are required to teach or the extent to which a teacher is able to adapt this curriculum to pupils’ needs. These issues are rarely examined at scale in developing countries, owing to a lack of data. But, where data is available, there is much diagnostic potential in analysing which teachers add more value to pupils’ learning and under what conditions. Specifically, this kind of analysis may help to understand the importance of competing explanations of low levels of learning outcomes such as:

1. Over-ambitious curricula
2. Absence of ‘teaching at the right level’
3. Within class heterogeneity and pupil grouping strategies
4. Teaching and learning strategies–such as ‘differentiation’ or ‘mastery’

¹ See Snilstveit, B., Stevenson, J., Menon, R., Phillips, D., Gallagher, E., Geleen, M., & Jimenez, E. (2016). The impact of education programmes on learning and school participation in low-and middle-income countries. London: International Initiative for Impact Evaluation (3ie).

² Pritchett, L., & Beatty, A. (2012). The negative consequences of over-ambitious curricular in developing countries: Harvard Kennedy School.

³ See Crouch, L. and Rolleston, C. (2017). “Raising the Floor on Learning Levels: Equitable Improvement Starts with the Tail.” An Insight note from the RISE Programme. Available  at  https://www.riseprogramme.org/publications/raising-floor-learning-levels-equitable-improvement-starts- tail

⁴ Young Lives is an international study which is tracing the changing lives of 12,000 children in two cohorts. It is following two groups of children, with approximately equal numbers of girls and boys, from poor households in Ethiopia, Peru, Vietnam, and the states of Andhra Pradesh and Telangana in India. The first group were born in 2001-02 and the second in 1994–95.

⁵ In statistical modelling, the teacher-level residual, understood as ‘teacher value-added’ is the gain in test scores associated with a particular teacher after conditioning on appropriate explanatory variables which may include prior test scores and socio-economic indicators.