The recent publication of a report evaluating the family of citizen led assessments has led to a number of blogs that are looking closely at the learnings from these efforts and trying to understand the implications of what has been achieved and what has not.[1]

The flurry of interest is welcome. This family of assessments now stretches around the world from south Asia across eastern and western Africa and all the way to Veracruz in Mexico. Facilitated by Pratham, ASER was the first of its kind in the citizen led assessment movement.[2] Initiated in India by Pratham ten years ago, this is a genuine model of South-South learning and cooperation and has been referred to as “one of the most internationally influential educational initiatives of the decade”.[3]

At the core of the work is a basic question: Enrollment levels are very high in many countries. Children are going to school, but are they learning? To search for answers, ordinary people fan out into villages and communities, to reach households and children. In the presence of their families and others, children read simple text and recognize numbers and do basic computations. Each child in a sampled household gets one-on-one attention. And over the heads of the children, between parents and neighbours there are conversations about the status of schooling and learning in the community. This mammoth exercise, covering more than a million children a year, happens annually across India, Pakistan, Kenya, Uganda, Tanzania, Mali, Senegal and Mexico (soon to be joined by Nigeria and maybe Bangladesh).

While applauding the role of the citizen led assessments in building awareness around the issue of children’s learning, observers point out that these measurements have not been successful in raising children’s learning.  They argue that information is not enough and that “reporting on woeful learning levels will not automatically galvanize action”.[4]

Well before ASER started in India, we knew that only providing information does not lead to action for improving learning. In 2004, JPAL conducted a randomized control trial experiment on a Pratham intervention in Jaunpur district in rural Uttar Pradesh. We learned several fundamental lessons from that work.  First, availability of information led to extensive local participation in heated discussions, but no action (collective or individual) was taken either in the community or in the school to improve children’s learning. However, when live “demonstrations” were provided about how to teach children, many villages and volunteers came forward to “act”. The lesson: along with seeing a problem and realizing that there is a need to act, people seemed to need a clear demonstration of what can be done and how to do it. In a context where outcomes have been low and “stuck”, first hand experience of how   children’s learning could improve was essential to make people believe that change was possible.      

By the time ASER was launched in 2005, Pratham had evolved a frugal and fast method for enabling children to learn to read and to do basic maths. The method was effective with children who were seven or eight or older. Community volunteers could do it and teachers too, in any Indian language. We had confidence in the assessment and in the intervention as we had already implemented this approach on significant scale in India. Learning from our past experiences, we ensured that ASER – the evidence gathering effort and the intervention (now called the Read India campaign) to fed into one another.

Over time, Pratham has developed two models to improve children’s learning – one for communities and a similar one suited for government systems. In both, the “catalytic” element incorporates elements of live demonstrations, participation and learning by doing.  Recently rigourous independent impact evaluations have been done on both models with very promising results.[5] In the first model, a Pratham team member works with village volunteers to run “learning camps” with children in Grades 3 to 5. A learning camp is an intensive burst of activity that lasts 8-10 days and is repeated several times over the next few months. Using the ASER reading tool, children are assessed and then grouped by level rather than by grade and taught for a few hours a day. Each group uses appropriate methods and materials for instruction. As children progress, they move into the next group. The ASER tool is used to explain the goals of the intervention to parents and community members and it is also used to track and review children’s progress. Within a duration of 100 hours, over 75% of locations, where this has been tried, have succeeded in getting more than 70% children able to read simple texts.  (This has happened even where baselines were low and more than 85% children could not read sentences before the intervention began). In the last two years itself, the ASER method of assessment has been used as the first step to initiate local action to benefit over 1 million children in India alone. These large scale demonstrations have helped many others see what is possible.

Pratham’s second model is one in which this “teaching-at-the-right-level” approach is used to catalyze action with government school systems. Seeing actual examples of how children are learning, several districts and states in India decided to move on this direct path from assessment to action. Using a variety of Pratham-government collaborations, close to 5 million government school children participated in such programs in the last school year in India.

The links between ASER and Read India are direct; both initiatives are integrally linked to each other. Whether at the village level or at the state level, ASER helps people understand the problem and Read India enables village volunteers and teachers find a solution. To make parents and communities believe that children can learn, large scale demonstrations are done so that people can see for themselves that significant and substantial change is possible in a relatively short period of time. To shake up a huge lethargic education system that is just about waking up to the importance of learning outcomes, a massive effort is needed to catalyse energy and commitment. Fuelling the changes at the local level and at the system level requires gradual building up of conviction and capability. For the last ten years and more, Pratham has been very busy doing this on scale.

In the development world, there is a lot of talk around the need for “transparency” and “accountability”. But the lens with which we viewed the situation in India when we started the ASER initiative was a different. Like many other countries, India had been focussed for many years on the “visible” issues of access and enrollment.  ASER, and other citizen led assessments which followed in country after country, enabled common people to feel and “see” a new problem – the challenge of children who are in school but not learning.  If you do not know there is a problem, you do not need solutions. It is only when the problem is visible, when you can see it, that it becomes real and alive and demands a solution. At the ground level, it is by involving hundreds of ordinary people in uncovering the problem, by engaging with thousands of parents (many of whom are illiterate) that large scale awareness began to grow. At this stage, it was visibility rather than transparency; engagement and demonstrations rather than accountability that seemed to be the key levers for moving learning into the centre of the stage.

At a higher level, many cogs are blocking the wheel. First, to date, outcomes have not been the focus of policy or practice. Moving learning from the shadows of the sidelines to the centre of the stage is no mean feat. Second, and perhaps more important, in India, and in many other countries, a culture of measurement is to be built. Historically, data and evidence have not been the basis on which decisions are taken. The capability to generate reliable information using systematic and rigourous methods is low; the capacity to analyse data is lower still. Weakest of all is the ability and the appetite for digesting evidence and translating it into meaningful action. It is in this kind of challenging scenario that citizen led assessments have made substantial headway. They have succeeded in drawing national attention to a new problem based on information collected in a new way. It is unprecedented to have this “climate change” in such a short period of time. Much much more needs to be done to shift mindsets and to develop mechanisms for evidence led decision-making and action. 

The movement of citizen-led assessments has only just begun. But even early on this journey we are already beginning to see what is needed for change. We need simple tools that help ordinary people understand problems. Engaging leads to understanding. The tools need to lend themselves easily to action. The momentum generated by engagement can be translated into local action. It has been done in India. Pakistan also has linked its ASER to action. But large scale demonstrations of how to bring about change and extensive hand-holding is needed. This is what leads to the conviction that change is possible. This is also what leads to the building of capabilities to make change happen.  

 Citizen led assessments have a very strong potential to lead to citizen-led action for improving learning. Together, in the next few years, let us explore its power.

[1] Dana Schmidt of the Hewlett Foundation wrote about the citizen led assessments (http://www.hewlett.org/blog/posts/how-ordinary-citizens-transformed-education-agenda) Also from the Hewlett Foundation, Pat Scheid reported from Senegal (http://www.hewlett.org/blog/posts/what-data-revolution-looks-ground). R4D’s Nathaniel Heller wrote about the recent evaluation report of the citizen led assessments (http://www.opengovpartnership.org/blog/nathaniel-heller/2015/07/02/why-citizen-led-assessments-educational-outcomes-matter#sthash.zrwYx99O.dpuf). ACER in Australia talking about ASER in India (Also see http://rd.acer.edu.au/article/citizen-led-educational-monitoring-shows-promise).

[2]Pratham is a non government organization working with children and youth in India (www.pratham.org). ASER stands for Annual Status of Education Report and is conducted by ASER Centre, the autonomous research and evaluation unit of Pratham (www.asercentre.org).

[3] Colin Bangay June 2, 2015. Why are citizen led learning assessments not having an impact on home soil – and how can we change that? World Education Blog. EFA/GMR. EFA report. http://efareport.wordpress.com/2015/06/02/why-are-citizen-led-learning-assessments-not-having-an-impact-on-home-soil-and-how-can-we-change-that/#more-5890

[4] July 9, 2015. Colin Bangay. What role for Citizen Led Learning Assessments? – Moving beyond Measurement EFA report. http://efareport.wordpress.com/2015/07/09/what-role-for-citizen-led-learning-assessments-moving-beyond-measurement/​​​​​​​

[5] See povertyactionlab.org