An intellectual history of the experimental approach in development economics
By Rashna Mahzabin
Billionaire, humanitarian, businessman, and influencer promoted chicken ownership in Africa to combat poverty and Professor Blatman pointing out that cash transfer may be more cost-effective than chicken added that the best method to fight poverty is using the right method to study the competing program and design elements of chickens versus cash transfers. This case of “chicken vs cash” tells us how the randomization movement has taken momentum in development.
In the midst of the momentum, this year Abhijit Banerjee, Esther Duflo, and Michael Kremer were awarded the Nobel prize in economics sciences for their experimental approach to alleviating poverty. The root of poverty runs deep and to examine and analyze different aspects and variables and which intervention works in the contexts, the trio used an experimental approach.
The trio asked rather small questions than trying to understand the big picture of poverty which eventually got them the Nobel prize by analyzing smaller and manageable questions. By pioneering an approach to empirical research for providing such answers, the 2019 Laureates ― Abhijit Banerjee, Esther Duflo, and Michael Kremer ― have transformed development economics. Their approach remained guided by microeconomic theory and the use of microeconomic data. But it shifted focus towards identifying workable policies, for which one can make causal claims of Impact.
Banerjee and Duflo were able to use these studies to explain why some businesses and people in less developed countries do not take advantage of the best available technologies. They highlighted the significance of market imperfections and government failures. By devising policies to specifically address the root of problems, they have helped make possible real contributions to alleviating poverty in these countries. Banerjee, Duflo, and Kremer also took significant steps towards applying specific findings to different contexts. This brought economic theories of incentives closer to direct application, fundamentally transforming the practice of development economics, by using practical, verifiable and quantitative knowledge to isolate causes of poverty and to devise adequate policy based on behavioral responses. The impact of these developments upon real-world development outcomes is immensely significant. Their work and substantial amounts of research that followed it established evidence on fighting poverty in many developing countries. And they are continuously expanding their horizon of contributions, which now also includes the climate and environmental policy, social networks and cognitive science.
As a result, we now have a large number of concrete results on specific mechanisms behind poverty and specific interventions to alleviate it. For example, on schooling, strong evidence now shows that the employment of contract teachers is generally a cost-effective way to improve student learning, while the impact of reduced class size is mixed, at best. On health, poor people’s investment in preventive care has been shown to be very sensitive to the prices of health products or services, giving a strong argument for generous subsidies to such investments. On credit, growing evidence indicates that microfinance programs do not have the development effects that many had thought when these programs were introduced on a large scale.
The corroboration of fighting poverty in the developing countries traces the intellectual history of the experimental approach in development economics, focusing on a set of thematic areas: education, health, behavioral biases, gender and politics, and credit. A separate subsection for each of these five areas highlights the substantive contributions by the Laureates. It elucidates how the experimental approach pioneered by Banerjee, Duflo and Kremer have substantially changed our factual knowledge about economic, social and political phenomena in developing countries. This article will be covering three of the highlighted areas of their work; education, health, and behavioral biases.
The following is by no means an exhaustive presentation of the Laureates’ research, and even less a literature review of the three thematic areas. However, it elucidates how the experimental approach pioneered by Banerjee, Duflo and Kremer have substantially changed our factual knowledge about economic, social and political phenomena in developing countries, as well as the methodological direction of the field.
In the mid-1990s, Kremer and his co-authors initiated the transformation of development economics. To investigate how supply and demand factors interact and to determine educational outcomes, they launched a series of field experiments in collaboration with a nongovernmental organization (NGO) in western Kenya. Two of the experiments estimated the impact of additional school inputs. Two other experiments estimated the effects of health interventions, including deworming of children. One experiment, begun in 1998, provided teachers with financial incentives tied to students’ test scores. These early studies illustrated the power and feasibility of focused field experiments.
But they also offered substantive lessons. Given the context, simply providing more resources had a limited impact on school quality. More textbooks per student did not improve average test scores but did improve test scores of the ablest students. Giving flip charts to schools had no effect on student learning. The two health interventions reduced school absenteeism but did not improve test scores. In theory, the incentive program could lead teachers either to increase effort to stimulate long term learning or, alternatively, to teach to the test. The latter effect dominated. Teachers increased their efforts in test preparation, which raised test scores on exams linked to the incentives, but left test scores in unrelated exams unaffected.
Duflo and Kremer conducted another early multiple-treatment experiment in Kenya, starting in 2005 that also was motivated by the challenges of a large influx of new students with varying academic preparation in response to the introduction of free primary education. Specifically, Duflo, Dupas, and Kremer took advantage of a program that gave school committee funds to hire extra contract teachers, in order to reduce first-grade class sizes. But they added two experimental variations: tracking students by prior achievement and training school committees to monitor the extra teachers. This design allowed them to analyze a range of important questions, including the impact of class-size reduction without changing pedagogy, the impact of contract teachers working under a dynamic incentive scheme versus tenured civil-servant teachers, the impact of empowered school committees, and the impact of tracking by achievement in primary schools.
A common response to overcrowded classrooms is to add more teachers. The idea is simple: lowering the student-teacher ratio increases the number of times teachers can spend per individual student, which could have a direct effect on learning. If students benefit from higher-achieving peers, then sorting students into separate classes based on their preparedness or their ability could disadvantage low-achieving students while benefiting high-achieving students, thereby exacerbating inequality. For that reason, tracking is a controversial practice that many oppose. But as Duflo, Dupas, and Kremer stressed, tracking also allows teachers to better target their teaching to student needs.
Duflo, Dupas, and Kremer provided evidence suggesting that all the students in the study benefited from tracking/ The findings from the first field experiments in Kenya provided a starting point for an early randomized controlled trial regarding education in India, which started in 2000. Reviewing the findings from Kenya, Banerjee, Duflo, and their co-authors concluded that students appeared to learn nothing from additional days at school. Neither did spending on textbooks seem to boost learning, even though the schools in Kenya lacked many essential inputs. Moreover, in the Indian context, Banerjee and Duflo intended to study, many children appeared to learn little: in results from field tests in the city of Vadodara, fewer than one in five third-grade students could correctly answer first-grade curriculum math test questions. In a series of papers in the early 2000s, Duflo and Banerjee, along with their various co-authors, began a systematic exploration of how to address teacher absenteeism. Duflo, Hanna, and Ryan initiated a field experiment in 2003 that examined high powered incentives linked to attendance. Working with an NGO that operated single-teacher schools in rural India, they randomly selected some schools where teachers received an additional bonus per day attended, as verified by school cameras at the start and end of the school day. They found that teacher absence dropped by half in treatment schools relative to control schools. Moreover, student learning improved. This work by Duflo, Hanna, and Ryan is one of the first examples of how randomized evaluations can shed light not only on the impacts of specific interventions but also help estimate behavioral parameters that are of more general interest. One way of doing so is to combine experimental evidence with structural modeling.12 Specifically, Duflo and her co-authors estimated a structural model using treatment-group data, and they validated the model using a control sample. The study provided convincing evidence of important behavioral parameters. One example is the wage elasticity of the teacher labor supply, which is required to design policies for better teacher performance.
In the past 20 years, more than 100 randomized controlled trials on education have been implemented across the developing world. The growing number of high-quality studies is also mirrored by a growing number of systematic reviews of the evidence. A clear message from these meta-studies is that some of the early interventions tested by Banerjee, Duflo and Kremer are seen as the most cost-effective interventions to improve student learning. For example, Glewwe and Muralidharan concluded that “interventions that focus on improved pedagogy (especially supplemental instruction to children lagging behind grade level competencies) are particularly effective, and so are interventions that improve school governance and teacher Accountability,”
Miguel and Kremer (2004) estimated the direct effects and the externalities of deworming. They argued that with externalities, studies that randomize disease control at the individual level will underestimate effect sizes, as they do not incorporate the positive externalities. While spillovers are likely to be of first-order importance — especially in countries where infectious diseases still account for a large share of the disease burden — they have received limited empirical attention in public health and epidemiological research. In contrast, Miguel and Kremer (2004) designed their study specifically to measure these spillovers. They examined an NGO program for school-based mass treatment with deworming drugs and health education. The order of treatment phase-in to 75 primary schools was determined by a list that first grouped schools geographically and then alphabetically within locations. In Miguel and Kremer’s econometric model, the effect of deworming is conditional on the total density of the local-school population within a particular geographic distance. Holding constant the total number of children attending primary school who live within a certain distance from the school, the number of these children attending schools assigned to treatment should be uncorrelated to other local observables and non-observables. The exposure to treatment spillovers is thus (quasi-) experimentally designed. Miguel and Kremer (2004) found evidence for large external effects on worm infection rates, as well as on subsequent school-participation rates, extending about 2 miles (at least 3 km) away from treatment schools.14,15 The empirical approach proposed by Miguel and Kremer (2004) has been utilized in a large number of studies within economics — of both health and non-health issues — to estimate the magnitude and spatial scope of treatment externalities Some infrastructure is a public good in the sense that additional customers can be served at low marginal cost once the infrastructure is in place, even though the service is excludable. Such goods are natural monopolies. If households heterogeneously value the infrastructure (and the supplier cannot perfectly price discriminate), there will be static deadweight losses. Society may then be better off by regulating prices to reduce those static inefficiencies, even if this may reduce the incentives to invest in infrastructure.
Kremer and his co-authors (2011) examined these issues in the context of water infrastructure technology. In the study area, many people collected water from naturally occurring springs, which can be contaminated by feces from humans or other animals. The authors evaluated a program that protected a random subsample of springs from fecal contamination. The intervention reduced the presence of Escherichia coli (bacteria used as indicators of fecal matter) by two-thirds in the water at the source, and households reported that children had about 25-percent lower incidences of diarrhea in the treatment versus control groups.
Kremer and Miguel (2007) provided the first experimental assessment of how prices affect the adoption of health products in a low-income setting. Among 50 primary schools enrolled in the free-deworming program discussed above, they randomly selected 25 of those to participate in a cost-sharing program, where parents had to pay a fee for their children’s deworming pills. They found an uptake of 75 percent in schools with free deworming pills, but only 18 percent with a fee of US$0.40 (which is still a heavily subsidized price). While this result shows that demand is very sensitive to price, and potentially raises questions about the maintained assumptions of the rational human-capital model, the evidence is not conclusive. As Kremer and Miguel hypothesized, the perceived private value of deworming may be lower than the fee charged, simply because of the treatment externalities they documented. While their paper did not disentangle this externality effect from other effects of the positive price, subsequent experiments with alternative designs have pushed the research frontier significantly forward and helped distinguish different mechanisms. Understanding why health-service quality is so low and what policies could improve it has long been a very active research area in development economics. The early absenteeism studies discussed above provided significant impetus for this agenda, and much of the early work focused on the effort channel.
High rates of absenteeism and more generally poor public-service provision served as motives for an experimental study by Banerjee, Duflo and their co-authors of ways to improve immunization coverage in rural India. In the study area, only 2 percent of children between one and two years old had received the recommended package of basic immunizations. In this study, the researchers discussed several reasons for the low uptake rates, including poor public-service provision. For example, in the year preceding the intervention, they documented that almost half of the health staff in charge of immunizations were absent from their health centers and could not be found anywhere in their villages. The intervention utilized mobile vaccination clinics (“camps”), where the care staff were always on site. In a random subsample of these camps, small incentives were offered to households that brought their children to be immunized. Full immunization rates reached 39 percent in communities served by “camps with incentives,” compared to 6 percent in control communities, and 18 percent in communities with “camps but no incentives.” However, regular camps were sufficient to raise the percentage of children receiving at least one shot to levels comparable with those in the camps with incentives (78 and 74 percent, respectively). The incentives were particularly effective at encouraging families to stay the course and reach full immunization. Still, even with good access, reminders of the benefits of immunization, and small nonfinancial rewards (1 kilogram of lentils valued at about US$1) for each immunization, 61 percent of the households did not get their children fully immunized.
Duflo, Kremer, and Robinson (2011) reached an important milestone in integrating behavioral and development economics. The researchers started a series of experiments in 2000 to shed light on a big puzzle: why do so many smallholder farmers, especially in sub-Saharan Africa, fail to take up relatively simple modern technologies, such as fertilizer, despite evidence of very high returns from agricultural trials? To answer this question, they set up a long-term sequence of field experiments with farmers in western Kenya. Their first set of findings suggested that it is not necessarily easy to use fertilizer in the correct way. Farmers may thus not use it because it is unprofitable unless the right quantity is applied. But these findings also suggested a substantial scope for learning. The next set of experiments looked at whether a lack of information could explain low adoption rates. The results suggested it cannot. Duflo, Kremer, and Robinson (2011) instead asked whether present bias can explain farmers’ behavior. They proposed a model where some farmers are statistically present-biased — in the sense of being hyperbolic discounters — and naive, such that they underestimate the likelihood that they will be present-biased in the future. Because purchasing fertilizer has a small fixed cost, hyperbolic discounting implies that farmers who plan to buy fertilizer will defer their purchase until close to a deadline. But at that point, they will be impatient again and choose not to buy. Using this model, Duflo, Kremer, and Robinson compared two alternative policy interventions: a relatively large subsidy and a small time-limited discount on fertilizer bought at harvest time, when farmers have some money. They implemented both interventions in a field experiment and showed that farmers purchased 50 percent more fertilizer when offered the small time-limited subsidy, which took the form of free delivery. Moreover, and consistent with the theory, this effect was greater than that of offering free delivery plus a 50-percent subsidy on fertilizer later in the season. The results are in line with the present bias being an important driver of low uptake among smallholder farmers. Evidence from additional experiments that explored alternative hypotheses further strengthened this interpretation.
Over the past 20 years, we have seen major changes in development economics research. Several scholars have played a vital role in this endeavor. However, the broad contributions by this year’s Laureates have been essential for bringing development research to its current standing. Kremer and his co-authors pursued a set of early experiments in western Kenya that showcased the promise of splitting up the daunting global-poverty question into smaller more manageable topics, each of which could be rigorously studied via a designated field experiment. Banerjee and Duflo, often together with Kremer or other researchers, broadened and expanded the set of topics, and articulated to the research community how pieces from such microeconomic studies can help us get closer to solving the broad development puzzle. All three Laureates expanded the experimental approach to basically all branches of the field. They were also at the forefront in addressing legitimate challenges to this experimental approach and in presenting solutions to these challenges. The contributions by Banerjee, Duflo and Kremer have encouraged and inspired a new generation of researchers to follow their lead.