Firstly, Rukmini taught me that a commitment to education needs to start from love for the child. The other day Karen Mundy, another member of the RISE ILT (and the new Director of the UNESCO International Institute for Educational Planning), was pointing out that RISE discussions often start with statistics and facts that document the learning crisis and show just how low learning levels in developing countries are. Karen contrasted this with the times she had recently heard Rukmini speak at global education meetings. Rukmini, while justly famous for the ASER reports (and the spread of citizen-led learning assessment globally, see below) that document learning levels, she doesn’t lead start with that. She leads with love for the child, then, from that expressed love for the child she works to concern that children’s needs for solid foundational learning are not being met. Anyone who has had the privilege of seeing Rukmini working with children or just hanging out in a village knows the infectious joy she radiates.

Second, Rukmini’s work has shown me that even though it takes a long time, successful reform at scale requires both effective action and advocacy that drives an education system into a receptivity to adopting and implementing effective actions.

Pratham first came to my attention via an RCT impact evaluation of their in-school volunteer tutoring program (balsakhi), which began in 2001 (this was also one of the first high-profile RCTs of the now famous Nobel Prize-winning duo, Abhijit Banerjee and Esther Duflo [along with Shawn Cole and Leigh Linden]). This RCT produced evidence (published in a top tier academic journal) that in-school tutoring of children who were lagging behind their grade level by volunteers produced substantial learning gains and that this was a cost-effective way of improving learning.

However, in contradiction to the aphorism “build a better mousetrap and the world will beat a path to your door,” Rukmini Banerji and the co-founder of Pratham, Madhav Chavan, did not find that having rigorous evidence of an effective action that raised learning in and of itself created a heavily trodden path to their door. In fact, they generally found that the reaction from the mainstream education establishment, both in government and in civil society, was indifference, if not outright hostility. The “Made in India” innovation of what has now become “Teaching at the Right Level” (TaRL) was a “disruptive” innovation (in the original Clay Christensen sense of an innovation that meets user needs but is not at the cutting edge as perceived by the profession) and hence did not fit with the reigning agendas of the time of enrollment expansion and upgrading of the physical school infrastructure (e.g. boundary walls) and quantifiable inputs.

Rukmini and Madhav realized the Indian system of basic education was not “aligned” or “coherent” for learning. They realized that unless and until they could shock the system—including politicians, civil servants, educationists, and ultimately parents and the public at large—out of their complacency about learning, there was no appetite or space for the adoption of learning promoting innovations. Hence the Annual Status of Education Report (ASER)—with the added twist that this means “impact” in Hindi—was born. Aware that the government of India would never allow “outsiders” to do in-school assessment of children at national scale, Pratham devised another entirely “Made in India” simple tool for assessing a child’s literacy and numeracy. They then carried out a “citizen-led” assessment done by volunteers through a huge number of partnerships with organizations all across India, which allowed national scale at amazingly low cost.

The first ASER report went into the field in late 2005 and the results were launched in early 2006 (establishing a pattern of very quick turnaround of data into reports so the results were up-to-date and relevant). The first report was greeted with great fanfare and widespread acceptance of the results and implications of the report showing very low levels of student mastery.

Just kidding. While it did get some attention in the press, the first report was met with hostility by the education establishment. No one from the government agreed to be present at the ASER launch. I was living in India at the time, working for the World Bank, and used the ASER results in a report, and we received the official comment from the Government of India that since the ASER results were not generated by the government they were not “official” data and hence the World Bank should not mention these results in our report (advice we graciously acknowledged and then politely ignored).

The point is that what has eventually led to the acceptance and adoption of TaRL was the combination of repeated evidence that this pedagogic innovation was an effective action that could work, at scale with government teachers and a sustained public information and advocacy campaign to get the system to accept the evidence of a learning crisis and hence accept the need to act to address that crisis. ASER was done year after year after year and, combined with other assessment results (such as the Education Initiatives student learning assessment in 2009 and the PISA 2009+ participation of two Indian states), eventually led to an acceptance that there was a need to focus on (re)making the Indian education system from a “selection” or “filtration” system into a true “education” system.

Third, successful implementation at scale requires “connective tissues” that reach all the way up and all the way down. For me to type this little appreciation essay on my computer requires that the ideas in my brain be effectively connected to the actions of my fingers. That requires a set of functional connective tissues: nerves, muscles, tendons, bones that work in at least reasonable harmony.

I have elsewhere described a challenge that India faces is that it has a “flailing state”: the schemes, plans, visions, policies, programs formulated in the very big and impressive brains at the top of the Indian hierarchy are not effectively connected to the ground-level reality. Hence India has amazing capability for some tasks—like holding the free and fair elections in the world’s largest democracy, like being able to send a rocket to Mars—but at many “implementation intensive” tasks (like clinic level health care [here and here], like policing, like sanitation, like basic education [here for the 1999 pioneering PROBE report]) it flails along with schemes that can deliver on logistics and easily quantifiable inputs that come and go without much impact on outcomes.

Two recent RISE studies illustrate what can be learned from Rukmini on this score.

On the “flailing” side: a recent RCT impact evaluation of a scaled program to implement “school improvement plans” was titled “Improving public sector management at scale?” The answer to the question is “No.” They find that although the program was implemented and schools did in fact produce the required plans (the “logistical” part) and hence there was “process compliance” (a feature of a “flailing” versus “failed” state is that they produce an elaborate fiction masquerading as fact), after that nothing happened. The authors conclude: “Our results illustrate how ostensibly well-designed programs, that appear effective based on administrative measures of compliance, may be ineffective in practice.”

In contrast, let me just quote from the abstract of a recent qualitative study of Pratham by Masooda Bano and Zeena Oberoi that puts into better words what I have observed Rukmini and Pratham doing:

The case study suggests that while a combination of factors, including evidence of success, ease of method, the presence of a committed bureaucrat, and political opportunity are key to state adoption of an innovation, exposure to ground realities, hand holding and confidence building, informal interactions, provision of new teaching resources, and using existing lines of communication are core to ensuring the co-operation of those responsible for actual implementation. The Pratham case, however, also confirms existing concerns that even when NGO-led innovations are successfully implemented at a large scale, their replication across the state and their sustainability remain a challenge. Embedding good practice takes time; the political commitment leading to adoption of an innovation is often, however, tied to an immediate political opportunity being exploited by the political elites. Thus, when political opportunity rather than a genuine political will creates space for adoption of an innovation, state support for that innovation fades away before the new ways of working can replace the old habits. In contexts where states lack political will to improve learning outcomes, NGOs can only hope to make systematic change in state systems if, as in the case of Pratham, they operate as semi-social movements with large cadres of volunteers. The network of volunteers enables them to slow down and pick up again in response to changing political contexts, instead of quitting when state actors withdraw. Involving the community itself does not automatically lead to greater political accountability. Time-bound donor-funded NGO projects aiming to introduce innovation, however large in scale, simply cannot succeed in bringing about systematic change, because embedding change in state institutions lacking political will requires years of sustained engagement.

The “hand holding” and “confidence building” and “informal interactions” described by Bano and Oberoi come from having structures of Pratham that intertwine with the government organizational structures at every level. If I may propose a semi-gruesome personal comparison, I recently ruptured my Achilles tendon, which as, as my surgeon put it, was “like setting off a firecracker inside a thick nylon rope.” While the tendon was not torn all the way through, it had enough of the connective tissue ruptured that my foot was failing because the tendon was not taut. My rupture was severe enough that just stitching the tendon endings back together was not enough; rather, the surgeon had to borrow another tendon and wrap that tendon around the Achilles to strengthen it. Pratham has learned that you cannot fix flailing by just blasting out more signals from the brain, just as with a ruptured tendon I could not force myself to walk just by commanding my foot to work.

Pratham understands that you cannot fix deeply human problems like the education of a child with stronger and stronger, tech-enabled, command and control. You don’t make Pinocchio into a real boy by adding more strings. Pratham has also learned (from hard experience) that “bottom-up,” school-by-school and district-by-district improvement eventually has to reach and change the “top” to be sustained. Pratham has learned that during transitions there has to be extra strengthening connective tissue all along the pathway from fingers to brain until it can work on its own.

Fourth, I learned from Rukmini the importance of learning from doing and doing what has been learned.

The very first time I met Rukmini, we were traveling around Uttar Pradesh visiting the sites of a Pratham project that was being done with “treatment” and “control” villages in order to do an RCT (which eventually became this published paper). While we were in one village, the young PhD student who was supervising the implementation of the experiment on behalf of the RCT researchers arrived. Rukmini introduced him by saying: “Lant, this is Dan, his job is to make sure we don’t help any children.” The very flustered research assisted eventually thought to say, “I am just protecting the integrity of the experiment.”

(I should point out Rukmini’s amused and amusing [dis]respect for academics extends to me as well. One day I visited her and she said, “I love your new paper ‘The Negative Consequences of Overambitious Curricula in Developing Countries’.” I was very flattered, but this led to the mistake of asking her exactly what she liked about it. “I loved that I didn’t have to read it, everything important about the paper is in the title.”)

The context was that the “experiment” had two treatment arms and it was already obvious that one treatment arm was having no impact and the other working quite well. Pratham, naturally, wanted to stop implementing the ineffective thing and do the effective thing in all villages, whereas the researchers wanted to continue implementing the ineffective treatment arm to reach the sample size they needed to give good (adequate statistical power) results.

The point was that the collaboration of Pratham with the Nobelists Banerjee and Duflo (and thereafter JPAL) has been a long and fruitful partnership. But Rukmini points out that while they have learned a great deal from doing the experiments, they have never changed what they were doing due to the published results of the experiment. That is, Pratham has learned from doing experiments in three ways.

One, the interaction with the geniuses in the design of the treatments was useful as it helped them be precise about what the “treatment” was, what the hypothesized causal mechanisms of effect were, and helped them design the implementation.

Two, the implementation of the experiment served as a disciplined pilot, and much was learned of the teething troubles about what would be encountered at scale.

Third, often the impact of the program was so obvious (either positive or in the lack of an impact) during implementation it passed the FO test and hence on did not need to wait for the completion and computations needed for publication to learn the right lessons.

Equally important to “learning by doing”—but not always followed by many organizations in practice—was “doing what was learned.” A recent presentation of the TaRL approach traced its evolution over 20 years from the very first experiences with the balsakhi program in 2001, to implementation with volunteers from village meetings, to implementation in summer camps with government teachers, to implementation at a statewide level. TaRL is now being implemented by various states of India and across Africa in large part drawing on lessons from learning by doing.

Fifth, although I cannot say I learned this first from Rukmini, her example hammers home the adage: “There is no end to what can be accomplished if you don’t care who gets the credit.” It is great that the Yidan Prize will bring Rukmini and Pratham’s “Made in India” innovations even more global attention. But to those of us lucky to know Rukmini, the relevant saying isn’t “Failure is an orphan but success has many fathers” but rather that “Many successes have the same mother: Rukmini Banerji.”