In 1966, Akira Endo, a Japanese biochemist who was then working for the pharmaceutical company Daiichi Sankyo, was travelling in the US when an epiphanic realisation dawned on him. America was then grappling with very high mortality rates from heart disease, and Endo established a correlation between that and the rampant obesity he saw all around him. In fact, he observed, many overweight people resembled the sumo-wrestlers of his native Japan. He attributed much of this to the rich American diet, which led to a build-up of cholesterol; barely a couple of years earlier, two scientists had received a Nobel prize for explaining how cholesterol is created and processed inside cells. Endo reasoned that as Japan became more ‘Westernised’, heart disease would become more common. Upon his return to Japan, therefore, Endo set about trying to find a drug that would lower cholesterol.

As a child, Endo had gone walking in the woods with his grandfather, and was at home in the world of moulds, mushrooms and other classes of fungi, and was familiar with their characteristics. He began screening fungi, and struck pay dirt in 1972, when he stumbled on a mould, Penicillium citrinum , which could block a key enzyme needed to make cholesterol. Within a year, he had extracted it, and labelled the drug ML-236B. But in lab tests, it failed on rats; similar experiments elsewhere on drugs to lower cholesterol failed too — in some cases, they actually increased the overall death rate. Even within his own company, he was losing battles for budget — and losing face as well.

Eventually, Endo capitulated — and retired to the world of academia — but his work was picked up and taken forward by scientists Michael Brown and Joseph Goldstein (who would then go on to win the Nobel for physiology in 1985 “for their discoveries concerning the regulation of cholesterol metabolism”). Other drug trials succeeded, leading to the formulation of what we know today as statin, the cholesterol-lowering drug that is today a $300-billion market. In many ways, it is the most widely prescribed drug franchise in history, and by some estimates has saved millions of lives. But Endo, the man who set off the fungi research that would yield the drug, has seen his work go largely unrecognised.

The killing of great ideas

Author Safi Bahcall, a second-generation physicist and a biotech entrepreneur, recounts the sobering story of Endo as a launchpad for his celebration of “loonshots” — which he defines as ideas or projects that most scientific or business leaders think won’t work, or won’t make money. “The most important breakthroughs … are surprisingly fragile. They pass through long dark tunnels of scepticism and uncertainty, crushed or neglected, their champions often dismissed as crazy,” he writes.

It needs large groups of people to translate those “loony” ideas into products that work, but in many cases, even teams with the means to develop those ideas reject them. That thought leads him to explore why even good teams kill great ideas.

Illustratively, in the early 2000s the Finnish company Nokia was selling half the smartphones on earth, and its management was celebrated for its success and vision. In 2004, its innovative engineers had created a new kind of internet-ready phone with a touchscreen; a phone which would be complemented by an online app store. Yet, the same “visionary” management shot down both the projects. Barely three years later, Steve Jobs unveiled the iPhone, which ran on much the same principles. From then on, Nokia went rapidly downhill, and sold its mobile business in 2013. From its peak, Nokia’s value had dropped by about $250 billion.

So why did Nokia “turn”?

Bahcall attributes this as a strange quirk of matter called “phase transition”. Fill a bathtub with water; if you hit the water surface with a hammer, it slips through the liquid. But if the water is frozen, and you hit the surface, the ice shatters. The same molecule behaves like a liquid in one context and as a rigid solid in another.

The same holds true for teams and companies, argues Bahcall. When groups are small, everyone’s stake in the outcome of the group project is high. At a small biotech firm, if the drug works, everyone will be a millionaire. If it fails, every one will be looking for a job. But as companies grow larger, the stakes in outcome decrease, while the perks of rank increase. “When the two cross, the system snaps… Those same groups — with the same people — begin rejecting loonshots,” he reasons.

Providing support

While such “phase transitions” are inevitable, they can be managed if we understand the forces that operate. Bahcall dissects case studies of companies and countries to draw up rules for nurturing loonshots, and then to hold on to the edge for the long term.

Illustratively, from about 500-1500 CE, China and India were very advanced economic powers: between them, they accounted for over half the world’s GDP. The five largest nation-states of Western Europe, by comparison, averaged at best 2 per cent. And yet, it was the powers of Western Europe, which became an incubatory lab for loonshots, that won the Scientific Revolution.

What we know as the scientific method is arguably the mother of all moonshots, reckons Bahcall. And missing loonshots, in the way China and India did, can be fatal.

In marshalling his argument for nurturing “crazy” ideas, Bahcall lines up a galaxy of case studies from the worlds of science, business and entertainment. And although he is occasionally prone to get carried away with scientific minutiae, his narrative style is free-flowing and makes for enchanting reading.

Bahcall’s world-view encompasses many universes — from biochemistry and behavioural economics to business. And his anecdotal accounts explain and help us understand everything from how wildfires spread, how traffic flows and why snarls happen, how to hunt for terrorists online, to why the James Bond franchise initially failed to take off. As Bahcall himself acknowledges, his theories may sometimes verge on the “loony”, but for that reason, they are worth nursing: in the end, they are educative — and hugely entertaining.