The magnitude of the cheating problem in Indonesia before the introduction of CBT becomes clear when we look at suspicious patterns on answer sheets. According to a so-called ‘integrity index’, which is generated by the Government of Indonesia based on previously developed algorithms that detect those patterns, there is substantial evidence of cheating on the national exam in 2015 for about a third of all junior secondary schools (Grades 7-9). These ‘low integrity schools’ are locally concentrated, as shown on the map below. In red districts (118 out of 514 districts), between 75 and 100 percent of the junior secondary schools, with 532,946 students or 13 percent of all exam takers, have suspicious answer patterns on their 2015 exam. This large number of cheating students and teachers point to the existence of a ‘cheating culture’ in these regions.

Each year since 2015, schools have applied to participate in CBT once they have the required resources (i.e., computers and stable electricity). By 2019, 39,379 out of 50,124 junior secondary schools implemented CBT. Using administrative data of all junior secondary schools between 2015 and 2019, we find a drop in average exam scores as soon as schools switched to computers.

You might wonder if a decrease in exam scores is driven by students who are unfamiliar with computers. As shown in the figure below, we find that when schools switch from exams on paper to CBT, their average exam scores drop only if they had suspicious answer patterns (i.e., low integrity) before the intervention. The drop in the average exam scores of low integrity schools is almost 10 points on a scale from 0 to 100. We also find only a small difference in the effect between schools that already had a computer lab before the intervention (41 percent of the schools) and schools that did not. These findings provide supporting evidence for our assumption that the drop in exam scores is due to reducing the possibility of cheating, not due to a lack of computer skills.

It is still important for students to achieve high grades because the result determines the quality of their subsequent education. With higher grades, they can get access to higher ranked senior secondary schools. Also, local governments and schools gain prestige from high grades on the national exam, so they continue to pressure teachers to perform well. Accordingly, we find that exam scores improve after the first year of CBT implementation. In the third year of CBT implementation, the effect for schools with computer labs in 2015 was less than half the size of the effect in the first year of CBT implementation (-3.9 points compared to -8.9 points, respectively). Since cheating is virtually impossible, this improvement can only be achieved through students and teachers exerting more effort into learning outcomes.

Schools within the same district switched to computers in different years. The computers not only changed the exam scores of the CBT schools; the exam scores of schools that continued their exams on paper also decreased as more schools surrounding them switched to computers. As shown in the table below, when all schools surrounding a school that takes the paper-based exam switched to computers, the average paper-based exam score of this school decreased by 9 points, and the integrity index increased by 21 points in a district with a cheating culture (dark orange and red on the map). In districts where less than 50 percent of schools had low integrity in 2015 (light orange on the map), there was no significant change in the average score or integrity index on paper-based exams as the percentage of CBT schools in the district increased.

Our results imply a norm change with respect to cheating. We have a few explanations for why cheating on paper-based exams becomes more difficult when surrounding schools cannot cheat anymore. It might be harder for them to obtain answer sheets. Teachers and students that take the computer-based exam have no reason to try to obtain the answer sheets and, therefore, they cannot forward them either. In addition, teachers from CBT schools might be stricter when proctoring the exams of schools that still take the paper-based exam, to which they are assigned by the district government. (The national exam proctoring procedure is that teachers never proctor students from the school in which they teach.) If their students cannot cheat, to ensure a fair competition, they might also prevent cheating in schools that conduct paper-based exams.

Note: Table includes schools that still took the exam on paper in 2019: 4,861 schools in columns 1 and 2 and 5,841 schools in columns 3 and 4. Standard errors between parentheses and corrected for clustering at the district level. Regression includes year and school fixed effects, such that coefficients present changes within schools. *** p-value < 0.01

Taking our findings together, the transition from paper-based exams to computer-based exams has helped Indonesia to move from a cheating culture towards a learning culture. Cheating on national exams in Indonesia was concentrated in certain regions, pointing to the existence of a local cheating culture. In many districts, there was sufficient evidence for substantial cheating on the paper-based exams for more than 75 percent of the schools. We find that the computers induce a norm change with respect to cheating practices in these regions, while learning outcomes improve over time when cheating is no longer possible.

• Emilie Berkhout