The Hindu Business Line : Poll vault

The very first reaction from a senior Delhi-based journalist, Raman Swamy, on the utter failure of psephologists and an overwhelming majority of media commentators in the recently concluded UP State Assembly polls was "dishonest studio arithmetic".

Sadly enough, high-profile psephologists still claim that they stick to a scientific methodology of sample surveys in poll predictions (especially exit polls). This lack of honesty or courage was seen when they hopelessly failed to predict the results of the 2004 parliamentary polls. No wonder, an editorial comment in a multi-edition north Indian morninger bluntly advised "retirement for pollsters till they improve their methodology".

Mahesh Rangarajan's afterthought that the BSP's success was due to a high share of votes among the Mandal or lower backward classes is but a quibble. In the same daily, he previously wrote that BSP's polling strength was often underestimated. Sociologist Dipankar Gupta's suggestion that pollsters ask "what grounds members of different castes coalesce politically" is apparently judicious but hard to justify as `null hypothesis' (or beginning with no preconceived notion) is a must for any sample survey, based on principles of probability. His point that the UP poll has showed that "jati loyalty is not the key" is at best a hypothesis, subject to ground check.

Any large-scale sample survey should have a pilot survey before the actual survey to test the questionnaire/schedule and examine whether proposed stratification (in case of stratified sampling) or identification of sizes correspond to the ground reality. Pilot surveys were a must in the path-breaking national sample surveys under the Indian Statistical Institute in the 1950s (NSS was then a unit of ISI). After the shameful failure during the 2004 parliamentary polls, pollsters or the organisations they worked for should have undertaken a pilot trial before pre-poll studies for the UP elections — given that it is a State with the largest legislature and elects the largest number of MLAs. Perhaps pilot surveys would depress the monetary gains for pollsters, especially the market survey-wallahs and might even compel them to drastically change sample designs — pushing the cost further.

In the late-1950s, Anish Maitra, a young and bright lecturer at the Asutosh College in south Kolkata, often cited sample survey findings to drive home the point that such surveys were often misused in the interest of bureaucrats and ministers. Undertaken by the State Statistical Bureau, West Bengal (subsequently re-christened as Bureau of Applied Economics and Statistics), in the earlier part of the decade, it said that a clerk heading a four-member household with a monthly income of Rs 100 could, apart from meeting filial expenses, have a small bank balance and maintain a telephone. Worse was a readership survey on behalf of a large-circulation Bengali daily by the market research section of an advertising agency headed by some ex-Communists. The area chosen was Metiabruz where 80 per cent of residents were Hindi- or Urdu-speaking. Each of the surveys was supervised by a master's in Statistics from the University of Calcutta.

Psephology — a derivative of the Greek psephos or pebble, used as ballots — was coined by the British historian R.B. McCallum 55 years ago to scan past election data, unlike poll forecasts by money-spinner psephologists in India.

Preference of sampling techniques in any quantitative estimation to complete counts is unquestionable. Random sampling is much more precise as sampling theory and practice is based on stochastic models. Statisticians have two formidable theoretical weapons in defence of reposing faith in sample surveys — Law of Large Numbers (LLN) and Central Limit Theorem (CTL). Psephologists claim that their pre-poll or exit-poll surveys are very large and hence scientifically sound — as if, for that reason, they cannot be blamed if results prove wrong. If we go by this argument, the sampling theory is to be blamed. There lies the catch.

CTL states that the sampling distribution of means will increasingly approximate a normal distribution as sample size increases. Actually, the larger the sample size, the smaller is the standard error. That means the range of estimate gets narrower and precise. Hence sample size must be large enough to make reliable estimation. But more important is that it has to be adequately representative. We can judge how a kilogram of rice will be taking a handful of it from a sacking. Even a sample size of 5,000 can give us realistic results, remaining within the CTL and LLN.

We saw how 10,000-plus sample sized exit-poll surveys, with a pre-determined multi-stage sampling method make disappointingly erroneous forecasts. Statisticians normally go in for thorough post-mortem, preferably by external statistical experts. Psephologists are not known to have done alike. In 1976, Austin Ranney wrote a review article, `Thirty years of psephology' in the British Journal of Political Science, focusing on the well-known Nuffield general election studies under political scientist David Butler. That had no connection with poll forecasting.

David S. Broder in his column in Austin American Statesman in mid-September, 1989 sarcastically wrote: "The science of interpreting elections has a fancy name: psephology. A shorter, simpler and more accurate title for much election analysis is: fiction."

More Stories on : Politics | Newspapers & Publishing | Other States

Article E-Mail :: Comment :: Syndication :: Printer Friendly Page

Stories in this Section
Nihari a la Mexican style

Sensory aid

Tax incentives for adoption