Experimental economics fights global poverty

The 2019 Nobel Prize in economics was awarded to three researchers for “their experimental approach to alleviating global poverty”, one which has “transformed development economics”.

What are randomised experiments? And why have they became so influential in development economics?

Improving the quality of life, particularly for the poor, is considered to be one of the main objectives of modern societies. Doing so requires a certain level of wealth. Economists have been preoccupied for centuries with understanding why some nations have “developed” economically and others have not.

But a more immediate question is: what can be done in the present? More specifically, what policies should less-developed countries adopt to improve the lives of their citizens?

What policies should less-developed countries adopt to improve the lives of their citizens?

Development economics as a subdiscipline was born in the 1940s and 1950s. For pioneers such as Paul Rosenstein-Rodan, W. Arthur Lewis and Albert Hirschman, development economics meant studying ways in which poor countries could attain large-scale societal transformation. They used a variety of analytical approaches, which pointed towards industrialisation as the driver of development.

Arthur Lewis, a native of Saint Lucia, was awarded the 1979 Nobel Prize in economics. This was in recognition of his study of the ways in which countries with unlimited supplies of labour – as is the case for many developing countries – could attain economic development.

One view of the challenge of development is that it is fundamentally about answering causal questions. If a country adopts a particular policy, will that cause an increase in economic growth, a reduction in poverty or some other improvement in the well-being of citizens?

In recent decades economists have been concerned about the reliability of previously used methods for identifying causal relationships. In addition to those methodological concerns, some have argued that “grand theories of development” are either incorrect or at least have failed to yield meaningful improvements in many developing countries.

Two notable examples are the idea that developing countries may be caught in a poverty trap that requires a “big push” to escape and the view that institutions are key for growth and development.

These concerns about methods and policies provided a fertile ground for randomised experiments in development economics. The surge of interest in experimental approaches in economics began in the early 1990s. Researchers began to use “natural experiments”, where for example random variation was part of a policy rather than decided by a researcher, to look at causation.

Randomised trials are not a new research method. They are best known for their use in testing new medicines.

But it really gathered momentum in the 2000s, with researchers such as the Nobel awardees designing and implementing experiments to study a wide range of microeconomic questions.

Randomised trials

Proponents of these methods argued that a focus on “small” problems was more likely to succeed. They also argued that randomised experiments would bring credibility to economic analysis by providing a simple solution to casual questions.

These experiments randomly allocate a treatment to some members of a group and compare the outcomes against the other members who did not receive treatment. For example, to test whether providing credit helps to grow small firms or increase their likelihood of success, a researcher might partner with a financial institution and randomly allocate credit to applicants that meet certain basic requirements. Then a year later the researcher would compare changes in sales or employment in small firms that received the credit to those that did not.

Randomised trials are not a new research method. They are best known for their use in testing new medicines. The first medical experiment to use controlled randomisation occurred in the aftermath of the second world war. The British government used it to assess the effectiveness of a drug for tuberculosis treatment.

In the early 20th century and mid-20th century American researchers had used experiments like this to examine the effects of various social policies. Examples included income protection and social housing.

The introduction of these methods into development economics also followed an increase in their use in other areas of economics. One example was the study of labour markets.

Randomised control trials in economics are now mostly used to evaluate the impact of social policy interventions in poor and middle-income countries. Work by the 2019 Nobel awardees – Michael Kremer, Abhijit Banerjee and Esther Duflo – includes experiments in Kenya and India on teacher attendance, textbook provision, monitoring of nurse attendance and the provision of microcredit.

The popularity, among academics and policymakers, of the approach is not only due to its seeming ability to solve methodological and policy concerns. It is also due to very deliberate, well-funded advocacy by its proponents.

Big funders

A key actor is the Abdul Latif Jameel Poverty Action Lab (J-PAL), which was founded by Duflo and Banerjee. Since its creation in 2003, J-PAL has conducted 876 policy experiments in 80 countries. One estimate suggests that it received around $300 million between 2003 and 2018, from a range of institutions. These include the World Bank, Britain’s Department for International Development and the Bill and Melinda Gates Foundation.

The idea is that policy decisions should be based on “objective”, “rigorous” and “rational” information and analysis.

J-PAL appears to have been influential in the World Bank’s decision in 2005 to establish a dedicated impact evaluation unit composed of former J-PAL associates to conduct randomised trials. The number of such experiments used in World Bank evaluations increased from zero in 2000 to just over two thirds of all evaluations in 2010.

These changes have taken place in a broader international context where there is increased emphasis on “evidence-based policy”. The idea is that policy decisions should be based on “objective”, “rigorous” and “rational” information and analysis. Although this may seem obvious, there are many debates about what this actually means in practice.

In this context, the advocates of randomised trials have sought to position their preferred methods as the most reliable form of evidence.

But there is strong opposition to the approach. This argues that experiments are not as reliable or useful as they are made out to be. Some even say that many such experiments fail to satisfy ethical principles and may actually harm development efforts.

This is the first article in a two-part series on randomised trials. The next will argue that the usefulness and appropriateness of randomised trials for development questions has been severely overstated.

Seán Mfundza Muller, Senior Lecturer in Economics and Research Associate at the Public and Environmental Economics Research Centre (PEERC), University of Johannesburg; Grieve Chelwa, Senior Lecturer -- Economics, University of Cape Town; and Nimi Hoffmann, Lecturer in International Education, University of Sussex