Joseph Bullough
Blavatnik School of Government, University of Oxford
Blog
40 New Insights From RISE and Its Community at vCIES 2021
The RISE community came together to showcase an array of exciting new research work at last month’s CIES (Comparative and International Education Society) 2021 Conference, one of the largest gatherings of comparative education researchers and experts in the world.
Over 4 days at this year’s virtual CIES Conference, more than 30 RISE researchers organized or took part in more than a dozen panels spanning a range of themes including education inequality, teacher reforms, management and bureaucracy, instructional coherence, information, community voice, political economy, and systems research.
Here’s a quick roundup of the highlights from 10 of these sessions:
1. “Mind the gap”: Understanding education inequality
- In cases studied in the Democratic Republic of Congo, Egypt, Kenya, Malawi, the Philippines, and Uganda, interventions aimed at improving foundational skills (using several measures of inequality) raises average achievement and reduces inequality across socioeconomic groups, with the biggest improvements seen for those at the poorest group (Slade and Crouch; slides; paper forthcoming)
- While the drivers of inequality vary, gaps between groups are modest compared to the gap between existing learning levels and the SDG learning goal, even children from relatively better off households fail to reach the SDG, pointing to the need to pursue a global equity agenda by “raise the floor” on learning performance and targeted low learning (Pritchett and Viarengo; paper)
- While schooling may be a necessary condition for learning, it is certainly not sufficient for it; little attention is paid to low learning as a driver of student dropout. Students who perform poorly (one standard deviation below the mean) at age 8 are 53 percent more likely to drop out of school by age 12 (Sobol et al; paper)
- In Ethiopia, responding to new inequalities created (and old ones exacerbated) by the COVID-19 pandemic means both supporting parents and caregivers to cater for those disadvantaged and working to strengthen local-level capacity (Woldehanna et al; paper forthcoming)
2. Examining challenges in teacher policy and practice
- Teacher careers structures should be considered from a systems perspective, and reform efforts should consider a set of five policy principles: “Choose and Curate toward Commitment to Capable and Committed teachers” (Hwa and Pritchett; slides; blog; paper forthcoming)
- In Indonesia, poor teaching quality persists despite repeated attempts to reform teacher recruitment and quality; dysfunctions in the teacher recruitment process point to the need to better understand system-level barriers and start with policies that support teachers to perform at their best as professional educators (Huang et al; paper).
- Two different approaches to measuring teacher quality in a performance pay intervention (one based on test scores, and the other based on a subjective appraisal by school principals) both raised test scores when trialled in Pakistan. But objective test score measures resulted in more punitive discipline and test preparation time, whereas subjective measures resulted in improvements to pedagogy. (Brown and Andrabi; slides; paper)
- Detailed classroom observations and teacher interviews in Vietnam, an education system that has been a successful outlier, find that high-performing classrooms used teaching practices that incorporated metacognitive strategies and socioemotional engagement (Dao et al; paper)
3. “Beyond the front(line) and centre”: Rethinking agents and agency in education reform
- A combination of narrative and systematic literature reviews finds a lack of focus on “middle managers” in learning-oriented reforms, even though they are critical actors. This flags the need to better understand and “unpack the practices” of middle managers in reforms (Asim et al; paper forthcoming)
- An analysis of the role of bureaucracy in managing and implementing an education reform effort in Delhi, India, finds it was hampered by bureaucratic norms that perpetuated hierarchy, jargon, and sanction-based accountability, which adversely distorted its goals (Davis et al)
- An examination of six country case studies of interventions focused on improving instructional leadership involving middle-tier administrators suggests the need for open, non-hierarchal communication and ensuring middle managers fully understand the importance of what the reforms mean (Jones et al.; slides; paper)
- An analysis of the structure and interactions of teacher unions in Indonesia finds that the dominant union has significantly influenced some teacher policy changes, and that the strength of the union leverage depends on strategic political partnerships and the ability to mobilise teacher strikes (Chambers-Ju et al; paper forthcoming)
4. Instructional Coherence: How greater alignment between curriculum, instruction and children’s level can improve children’s learning
- A curriculum reform in Tanzania that simplified the curriculum and shifted more focus (80 percent) to instructional time devoted to foundational literacy and numeracy in Grades 1 and 2 produced large, significant gains to learning in Kiswahili and Math (Mbiti and Rodriguez-Segura; paper forthcoming)
- Application of a formal, structural model of the learning process to learning dynamics in developing countries shows that improving instruction and slowing curricular pace produces significant long-term learning gains, with expansion of schooling having little to no effect (Belafi et al; presentation)
- A diagnostic study using data on learning outcomes from in and out-of-school children in Nigeria, observes that most children are being instructed at a level above their skill level and shows that focusing instruction at the learning level of the most disadvantaged children could have substantial effects on the proportion of children who meet foundational skills targets (Adeniran et al.; slides; paper)
- A set of four common principles links a number of approaches for bringing instruction into better alignment with children’s learning levels, nesting them under programmes that align levels of instruction with the goals and needs of students (“ALIGNS”) (Silberstein et al; slides; paper)
- Together, these insights underline how coherence of instruction with children’s learning levels is itself an essential component of a well-functioning education system and illustrate the cumulative losses that can occur from trying to cover too much, too fast.
5. How information shapes, goals, behaviours and motivation
- A study from India that uses representative samples of administrative census data from two states to measure the magnitude of manipulation on a low-stakes setting. This study finds that official learning outcomes data are extensively overstated, but that computer-based tests, compared to the conventionally administered paper-based tests, significantly reduce the possibility of inflating results (Singh; paper)
- Similarly, in Indonesia, which has seen widespread cheating in national exams, computer-based testing makes cheating more difficult and refocuses classroom efforts on learning rather than “shortcuts of cheating”, catalysing a shift in local norms (Berkhout et al.; paper; blog)
- A comparative study that looks at the reliability of three nationally representative surveys on learning outcomes in India concludes that the National Achievement Survey (NAS) do not accurately reflect states’ relative performance, calling attention to need for improved data and the ways in which (for administrative and political reasons) data can be inflated (Johnson and Parrado; slides; paper)
- A study of a community-based accountability intervention in Indonesia found that the most effective treatment arm had a combination of monitoring mechanisms and teacher incentives that not only generated better information but also shifted the power dynamic between the school and the community (Susanti et al.; slides; paper forthcoming)
- A government-run programme in Tanzania that provided schools with diagnostic information on the quality of teaching, learning and management and combined this with measures to improve follow-up with frontline providers resulted in changed beliefs, improvements to teaching practice, and modest gains to learning in Kiswahili (Cilliers and Habyarimana; slides; paper forthcoming)
6. Opportunities and constraints for community voice to improve learning in schools
- A review of 48 studies that provided information to communities to strengthen their voice finds that more information is not always better – outcomes improve only if the recipient sees the information as relevant, has the power to act on it, and has the incentive to act on it (Kosec & Wantchekon; slides; paper)
- An RCT of two interventions aimed at strengthening parental involvement in Mexico (doubling financing to parent associations and giving parents information on ways to be involved in schools) finds no impact on teacher practices or test scores, and highlights that giving more power to parents – especially over school finances – can erode trust and upset the delicate social relationship between community and teachers (Barrera-Osorio et al.; slides; paper)
- An RCT called “KIAT Guru” introduced a ‘social accountability contract’ for teachers that was monitored by local communities in 200 rural primary schools in Indonesia, leading to improvements in learning outcomes but with significant variation between treatment arms that may have depended on how the program affected the social relationship between communities and teachers (Pradhan et al.; slides; paper).
- A study that reconstructs the historical exposure of different communities to Western education in Nigeria finds that access to schools shaped families’ aspirations for education across generations, highlighting the role that local role models and information about the returns to education play in determining the demand for education (Okoye et al; paper forthcoming)
7. Political economy research to improve systems of education
- A policy deliberation initiative in Nigeria (aimed at better connecting decision-makers and key stakeholders in the education process and creating social contract for education) illustrates how constructive dialogue can influence the formulation of shared goals and accountability structures around education investment, learning outcomes, and support for learning-oriented reforms (Wantchekon et al; paper forthcoming)
- In India, an analysis looks at stakeholders’ roles and how these shape reforms around education, pointing to the influential role that civil society organisations play in the country and their ability to build coalitions which reshape the narrative and refocus it around learning (Aiyar)
- A political economy study of Vietnam that examines the unique political and social conditions for Vietnam’s continued success to improve learning. A combination of political commitment, public governance and societal buy-in shape the country’s accountability relations around schooling and learning. This highlights the importance of coherence around learning goals across different actors of the system (London; paper)
- In Pakistan, system frictions place competing demands for performance on “good” teachers, indicating the unintended consequences of having many different measures on which they are judged (Siddiqi; paper forthcoming)
8. Learning from research on implementation at scale
- A study that uses an innovative methodology (the Surveys of Enacted Curriculum) to map (mis)alignments between curriculum components in Uganda and Tanzania—including standards, exams, and instruction—finds incoherences in coverage (Kaffenberger and Atuhurra; slides; paper)
- An analysis of the “Millions learning real-time scaling lab” initiative in motion presented by the Center for Universal Education at Brookings, emphasises the importance of supporting and elevating strong leaders at the core of effective instructional delivery (Perlman Robinson et al)
- A cross-comparative review of 8 highly effective instructional programmes at scale by RTI International flags important implications for pedagogy (a systematic approach, time on the foundations—literacy and numeracy, a teaching of concepts), support (quality teaching aids, training, and materials), and systems (alignment with government plans) (Stern et al; blog)
9. Innovations in research methods to study education systems
- Presentation reviewed four methods for studying education systems that have been deployed by researchers across the RISE Programme: structural modeling, qualitative observation, RCTs to generate system insights, and new methods to measure emerging concepts (Pritchett and Spivack; slides)
- RISE Ethiopia’s research methods have been able to adapt as the pandemic evolved and information continued to be provided to policymakers throughout the period, highlighting the importance of research flexibility and the resilience of actors and researchers (Sabates et al.; blog)
- A project in South Africa to understand performance gaps in the education system discusses how trust and accountability interact and illustrates to prevent or promote improvement, highlighting the importance of accountability loops and the “stocks and flows” of system dynamics (Ehren)
- An RTI International (earlier mentioned) analysis of 8 successful projects which sets out to examine what ingredients, methods and system supports are needed to improve learning levels at scale flags the potential of using causal chain analyses and mixing methods (less commonly employed) to better understand how education systems work (Jukes et al)
10. Delivering education during COVID-19
- New reflections on a study which looked at the impacts of the 2005 Pakistan earthquake, points to learning losses of 1.5-2 years, based on 14 weeks out of school—and that schools’ closures only mediated 10 percent of the loss, with much more lost after children returned to school, presumably as they fell behind the curriculum (Das et al; paper; insight note)
- A study by the RISE Ethiopia team examines the flow of information across the Ethiopian education system during the pandemic, highlighting the critical need for prioritisation and inclusion of ground-level actors and communities, particularly concerning information and empowering their actions (Yorke et al; slides; paper forthcoming)
- A survey drawing from the Oxford COVID-19 Government Response Tracker at Oxford’s Blavatnik School of Government documents patterns in the Brazilian education system that point to the likelihood of widened inequalities as income predicts access to materials and support (Petherick et al.; paper)
RISE blog posts and podcasts reflect the views of the authors and do not necessarily represent the views of the organisation or our funders.