The Hindu Business Line : Beware the Black Swan

Think back and you will probably find turning points that changed the course of your life — a chance meeting with a stranger, an interview, a phone call, a sudden request to step into someone else's shoes, an idea from a book that you happened to browse through randomly, or someone coming into money or needing your help in some way.

For a long time now, I have had a sneaking suspicion that the so-called social science really doesn't qualify as a science at all. And where it seemingly does, by virtue of numbers, formulae and regression equations, it is so artificial and bound by qualifying clauses as to become irrelevant to the real world. A game played according to rules no doubt, but of dubious relevance to everyday life.

Take the notion of perfect competition on which all of classical economics is based. An infinite number of buyers and sellers meet in a marketplace to transact, with no power to affect the price. All products are substitutable and there is no differentiation. Price is therefore determined entirely by supply and demand.

Millions of students have been fed on the ways of this mythical world in school and college. The current strain of neo-classical economists still deals with competition and the free markets with similar assumptions of information symmetry, equilibrium conditions and efficient markets hypothesis and so on. Only the mathematics is more impressive — the basic assumptions are not. One has been taught to treat all other real life economic and marketing issues, for example imperfect competition, state-controlled prices, product differentiation, advertising effects and so on as departures from a norm — a frame of reference that paints an ideal world. The truth, however, is that the so-called exceptions are more the norm and the textbook economy doesn't happen at all.

The world of quantitative finance is further removed from reality in that it takes many aggregates and feeds them into equations creating a magic world of not just stocks and bonds but orders of derivative securities based on the estimates of various people on the future prices of these instruments, in themselves a kind of abstraction. Nearly all of this depends on some form of speculative extrapolation, in which typically the future is expected to be an extension of the past.

As every intelligent investor and trader knows, there are two separate influences on security values — something called the fundamentals or the earning power of the underlying assets (such as the company and its products) on the one hand, and a purely emotional, perception- or euphoria-led element called sentiment on the other. Nevertheless, the variations from the expected projections are estimated by the standard measure of uncertainty namely the standard deviation, familiar to most undergraduates.

The breakthrough that the brilliant book The Black Swan (Random House, New York, 2007) by Nassim Nicholas Taleb achieves is to show how limited the usefulness of this measure is in estimating anything that can vary over a wide range of values — such as the sales of a new book or future prices of stocks or indices.

The author is a fully accredited academic and member of the `quant' profession that rules over Wall Street. He has been an options trader and taught statistics and has a PhD as well.

He divides phenomena into those where the traits of the central tendency and variation within known limits apply. This he calls Mediocristan. This is the world inhabited by experts and knowledge.

Then there is another world where tsunamis and earthquakes, meltdowns, political upheavals and stock market collapses happen quite often, yet completely unpredictably, which he calls Extremistan. In this world, knowledge is of no use because experts (so called) do not really know what they do not know.

Black Swan is a metaphor for the apparently improbable events that do happen from time to time, which are impossible to predict until the moment before they do, and have huge impacts, and which seem somehow explicable after the event.

The events that are predictable by the statistics of correlation and regression and the standard deviation are those that (like heights and weights of humans) are known to range over certain values.

For instance, the sales of a Harry Potter book or the wealth of a Warren Buffet or Bill Gates bears no relationship to the average person, and can vitiate all attempts at prediction or averaging. Also, they represent an enormous chunk of the total wealth or book sales of all those being considered, such that any sort of normal distribution of the variable would be pointless. These latter are called scaleable quantities, where the distribution does not trail away into a thin tail but can have a very fat tail indeed.

Taleb exhorts us to pay attention to the apparently improbable and include it in our reckoning, in practically all our daily lives. Failure to do so would entail being caught totally unawares and being devastated by events like the Black Thursday or the May Mayhem in the US and Indian stock markets respectively.

Our private lives too are moulded by black swans. Think back and you will probably find turning points that changed the course of your life — a chance meeting with a stranger, an interview, a phone call, a sudden request to step into someone else's shoes, an idea from a book that you happened to browse through randomly, a friend you ran into at a most unexpected time or someone coming into money or needing your help in some way.

N. R. Naryana Murthy of Infosys recounts how his life was changed, and he was cured of his idealistic sympathy for communist regimes, after being manhandled and abused by the Bulgarian security police, on a baseless charge while back-packing, as a young man, through Europe.

Much the same unpredictability holds for the course of history too: from invasions to climate changes, from fall of dynasties to wars and pestilences. One might feel after the event that we could have `predicted' many of the disasters, such as the plague in Surat or the violence of Partition. "It was a disaster waiting to happen" is a familiar judgement, post-facto.

A Black Swan event is derived from the scientifically valid statement "all swans are white" which was true for centuries — as long as no one could produce evidence of a black swan. With the discovery of the new species of the bird when Australia was colonised, the black swan became reality! The philosophical truth behind this is one of the many remarkable insights of this book by an author who has specialised in the interpretation of uncertainty, chance and luck. His earlier work, Fooled by Randomness also pointed out the many fallacies in the way even professionally qualified statisticians estimate probabilities.

Why is this book important for the manager? Managers as a breed hate uncertainty and ambiguity, the engineering and accounting trained ones more so. They are hungry for explanations, and final causes, which make them feel in command. Their thinking is fashioned by their training in linear logic, whereas nearly all of social sciences describe non-linear events with complex causal relationships.

This is why few such brilliant `straight thinkers' have been of much help in solving problems that have festered for ages such as in the Middle East, Ireland or Kashmir. Taleb also points out the inbuilt bias amongst professionals to look for evidence only to confirm what one has already decided is the explanation. We tend to ignore the silent evidence that is not adduced in favour of an argument. It is all too easy to shift the argument from say, "most terrorists are Muslim" to "most Muslims are terrorists" or the `absence of evidence' as equivalent to `evidence of absence'. There are many such tricks in debate and argument, which we run into in our daily lives mainly because our nature seems to rebel against the phrase "I don't know".