The limitations of Abhijit Banerjee’s methods

As we continue to struggle with using evidence-based research for urban policy there is bound to be some interest in whether the answer lies in the method suggested by Abhijit Banerjee. The Nobel prize is a confirmation of its intellectual rigour. An affirmation of its value has come from the man who implemented the most fundamental economic reforms in India — Manmohan Singh. The true value, and difficulties, of this method, however, emerge when we take it to the challenge of Indian urban policy. The method does help highlight some of the serious lacunae in urban policy, even as the experience points to limitations of the method.

The support for the method is best understood when we place it in the context of the larger challenges to economic thinking. If the ideal of a completely free market was confronted by the Great Depression in the first half of the twentieth century, the collapse of communism ensured scepticism over all ideological theories. The resultant rise of pragmatism meant that policymakers could take virtually any step on the expectation that it would work. Since the success or failure of a policy was only known after implementation, there was no way of knowing beforehand whether it would work. We could never be certain policymakers were not pursuing policies due to vested interests, while all the while claiming they would be beneficial. And if the policies did not work, we could never be sure whether it was the result of corruption or due to more legitimate causes. As the line between genuine failure and corrupt practices blurred, we tended to tar all failures with the same brush of corruption. This made the honest policymaker averse to the risks of taking a decision, and hence contributed to policy paralysis.

It is here that the method proposed by Abhijit Banerjee, and others like him, became attractive. In this method, specific policy options are tested under experimental conditions. Using randomised control trials (RCT), not unlike the ones used to test drugs, the method throws up results that are objectively arrived at. This rules out the possibility of individual policymakers only doing what they find expedient.

In an urban policy environment in India where decisions are based on anything from popular beliefs to the fancies of individual policymakers, any movement towards objective evidence is undoubtedly a good thing. Apart from keeping out expediency and its contribution to corruption, the evidence generated by this method provides a better view of what actually works. And this could include several unexpected results. Valuable as this method could be in increasing evidence-based research in urban policy-making, it falls well short of providing an effective policy framework. Its focus is entirely on testing specific policy options. It tells us little or nothing about how to arrive at the options that are to be tested. There could easily be policy options that fall outside the purview of current beliefs and hence slip under the radar. These include not just entirely new options but also previously prominent ones. Would anyone spend the money required for these trials to find out whether nationalisation was an effective urban policy option today?

Much also depends on when a policy is believed to work. An experiment in an urban school could show that teachers in one that requires them to swipe a card when they enter tend to be more on time than those in a school that has no such requirement.

But a mother who is a teacher in the school where they are forced to come on time may do so by cutting the time she spends on the pre-school learning of her young daughter at home. An enforcement of this policy could then result in the more sensitive mothers, and teachers, dropping out of the workforce. Is there any objective way of judging the overall impact of such a policy, of when we can say it works?

Even if we arrive at a consensus on what works, there are other difficulties. What do we do when the results of an experiment at one place and time contradict those of another done elsewhere at another time? Banerjee’s suggestion would be to do more experiments for each local situation. But policy cannot always wait endlessly for experiments to be done in a way that suits academic rigour. As another Nobel laureate, Angus Deaton, has pointed out, RCT is neither necessary nor sufficient for evidence-based policy-making.

Urban policymakers should thank Abjijit Banerjee for his much-needed reminder of the importance of evidence-based approaches, but they should refuse to be shackled by the narrowness of randomised control trials.

The writer is a professor at the School of Social Science, National Institute of Advanced Studies, Bengaluru

COMMENT NOW