The Hindu Business Line : Meditations on forecasting and the improbable

We have been brought up on the belief that acquiring more and more data will at the end of it all bring us to light at the end of the tunnel. On the contrary, explosion of competing media channels has made accumulation of daily news as knowledge almost useless. The more magazines and TV shows one is exposed to, the less the additional value one gains from the next one.

For any student of economics or management, success in forecasting ought to be a matter of great interest. Though Taleb half-jokingly quotes Yogi Berra, the baseball coach, that "it is tough to make predictions, especially about the future", he is not alone. One of the authors whom few conventional courses in the subjects would mention is Paul Ormerod of The Economist who has written some equally thought-provoking books on the complexity and non-linear ways of human society and therefore economies. His most recent book is provocatively titled: Why Most Things Fail.

Paul Ormerod echoes Taleb's views on the impossibility of any kind of reasonable forecast where aggregates are concerned. He too draws attention to biological phenomena as more appropriate parallels to economic behaviour. Not that companies, governments and organisations act in a random manner, but the net result of millions of decision making units acting upon one another produces an unpredictable end result like the GDP which is an abstraction, far too complicated to be susceptible to any kind of prediction.

What is more, the use of Gaussian distribution (bell curve) to consider the probability distributions would in his view be absurd because, as he says "the breakdown of simple connections between the size of events and the scale of their causes means that successful short-term prediction in such systems becomes at best very difficult and usually impossible. And an inability to predict removes the potential for precision, engineering-like control which has motivated a great deal of social and economic policy in the West since the war."

Economic forecasting, a treacherous field

In a technical paper published in Physica A, he argues along with his co-author, that "by scientific standards, the accuracy of short-term economic forecasts has been poor. Based on a rigorous analysis of over 120 years of macro-economic data, Ormerod suggests that "the genuine information content in economic growth data is low", and in summary, the poor forecasting record of GDP growth by economists appears to be due to "inherent characteristics of the data, and cannot be improved substantially no matter what economic theory or statistical technique is used to generate them".

You might as well forget fancy econometrics where markets and companies are concerned about sums up his advice.

So, then, where does this leave us ordinary mortals? We have to find the numbers to feed appetites of CEOs and the finance department everyday. We have to guess what the sales, collection or inventory will be next week or what will happen this time round when we run a price-off or a promo. The fact is, no one knows.

It is useful to beware of another typical managerial pitfall - the proneness to overdose on knowledge. We have been brought up on the belief that acquiring more and more data will at the end of it all bring us to light at the end of the tunnel. On the contrary, explosion of competing media channels, propped up by advertisers, has made accumulation of daily news as knowledge almost useless. The more magazines and TV shows one is exposed to, the less the additional value one gains from the next one. They repeat themselves, in words and images. So, one thing we can do is to be selective in our TV watching and reading of papers and magazines. We can free up our own time for deeper reflection - and some peace and quite, of course. Knowledge can actually be toxic to the system!

As for the imponderables, if we listened to Taleb, we must do the opposite - focus on the outliers that could destroy our plans. It is important to be mentally prepared for the eventuality, knowing that it can hit us any time. This is advice for life rather than professional management alone. Choose the games you will play, stand above the rat race and decide in advance so to speak how much you are willing to take as the worst case. Face the downside risk, in other words, and think about mitigating it - not estimating its probability so much as assessing its likely impact. As a friend told him, don't ever run to catch a train, because missing a train is painful only if you run after it. I for one find his wisdom irresistible in one respect: an aggressively stoic prior disdain of the grapes is ever more rewarding than running after them and then say they are sour!

Positive Black Swans

Taleb also introduces us to the notion of the positive Black Swans - the lottery ticket, the first customer who could fall in love with our product and become its champion, or a breakthrough innovation with enormous potential. With these, one must bet the maximum that we can afford, since the downside is known and can be limited. We must also be watchful and look out for the weak signals and amplify them and spend time considering their implications. Also ensure that we do not overlook evidence against our case, in our eagerness to prove ourselves right, which is the confirmation bias. It would be a salutary discipline to learn to be an empirical sceptic; and be on our guard whenever someone gives us a simple "because" statement e.g. "oh, the next year's economy will be good because the monsoons have been good". This might make for impressive chit chat but hardly any scientific discourse or serious discussion on business planning.

Never was there a greater logic for adopting a tentative approach to planning - keeping scenarios in mind and adjusting them when required without falling into the trap of ownership of ideas. Single point estimates are really no more than a ritual game - as anyone can see if only we would look back over several years and see how far we actually were from the forecast.

Of all the managerial pitfalls the greatest is, when faced with missing the numbers, is to find a number of possible external factors that explain it, while celebrating the `achievement' of the target as a personal triumph. Alas this is true of all sales people and CEOs.

However, both these authors are hardly upstarts. Rather, they are reviving and carrying on an unpopular school of thought to which they refer glowingly, which owes its origins to two towering stalwarts among economic thinkers of the first half of the 20th century, von Hayek and Schumpeter.

In my previous instalment on this page, I had argued that there is growing evidence that the laws of physics and the methods of the exact sciences are inappropriate to describe and understand the reality of social systems and human lives. One of the examples mentioned was the idea of perfect competition.

One of the earliest and original thinkers in economics, Schumpeter argued against this in the early decades of the last century but has never been acknowledged till now by mainstream courses. Schumpeter saw this kind of competition as relatively unimportant. He wrote: "[What counts is] competition from the new commodity, the new technology, the new source of supply, the new type of organization... competition which... strikes not at the margins of the profits and the outputs of the existing firms but at their foundations and their very lives."

Schumpeter in particular is credited with having developed two other Nobel laureates in Economics (Tobin and Paul Samuelson) and is known for bringing in a real life flavour to economic theory for the first time. He almost single-handedly developed the notions of entrepreneurship and innovation along with the `gales of creative destruction' to explain the way capitalism really works out there in the market place - and not in terms of mathematical models. For him periodic failure and disappearance through M&A of many corporate entities was to be expected.

In the end, Taleb warns us of how we are driven by the impulse to prove ourselves right all the time, and to feel in control of the environment. We would do better to be grateful for what we have, and remember that if one were ruthlessly objective about it, a good deal of life is about luck. And no amount of clever spin can take that away.