As data consumption skyrockets, tech majors like Oracle, IBM and SAP are fighting it out for a piece of the pie.
Rachna Sharma, a 23 year-old medical student is a big fan of this popular-chain of ice cream. She frequently updates her Facebook status and posts on Twitter after visits to the ice-cream parlour. One day, she started receiving messages and even e-mails about new offers and monthly schemes from the ice cream vendor.
The use of big data and social analytics is allowing companies to track individual habits and make sense of un-structured data available on social platforms.
Big data refers to the enormous amounts of data generated, from for example social networks or data that is available in the public space, that it becomes difficult to process manually. Consider this. It took humankind roughly 24,000 years to produce 5 exabytes of data ( 250,000 years of DVD quality video). We now globally generate that much content every 2 days. In 2010 there were 255 million websites globally. This has grown to 511 million by 2011.
Although data uptake in India is still at an early stage the numbers are staggering. A study done by NetApp reveals that information deluge has become the order of the day among Indian enterprises, with 40 per cent of organizations in verticals like BFSI, Media & Entertainment, Telecommunications, and Government having more than 100 terabytes of data currently.
Corporations are therefore now looking to deploy serious computing power to make sense of these huge amounts of data. “About 90 per cent of the data has been created in the last two years. It is being created at a great velocity and is mostly unstructured information. This data can be a good revenue generating tool for companies as it helps them track consumer behaviour by analysing the data,” says Tim Young, Executive, Big Data Strategy, IBM.
The NetApp study estimates that Big Data solutions market opportunity in India is set to grow to $153.1 million by 2014. This represents a compounded annual growth rate of 37.8 per cent for the period 2011-2014.
Anil Valluri, President, NetApp India Marketing & Services Pvt Ltd, said, “Data by itself has no value unless it is translated into insight that can help drive business results. Verticals like retail, telecom, and manufacturing can immensely benefit from Big data solutions that dramatically help organizations get control of data and reduce time-to-insight.”
Tech firms companies such as IBM, HP Oracle and SAP are therefore seeing a huge opportunity in helping businesses deal with all this data. “Big Data implementations have just begun rolling out globally with Indian firms in the consideration phase. In a globalized economy, Big Data possess strong growth potential worldwide including in the Indian market which will match steps with other countries in adopting Big Data for driving business” says Venkatesh Krishnan, Vice President, Systems Business at Oracle India. Oracle’s Big Data Platform includes Oracle Big Data Appliance, which in conjunction with Oracle Exadata, and Oracle Exalytics help customers acquire and organize diverse data types.
To counter this IBM has expanded its PureSystems portfolio by launching the PureData System. The company is expecting growth markets like India and other ASEAN countries to increasingly adopt big data and social analytics in a big way.
The big challenger to Oracle and IBM is the German software major SAP whose big data platform has become its bestselling product in a short time. Core to its analytics strategy is HANA, an in-memory database appliance that processes records at incredible speeds. “The real driver for this technology is the increasing need to connect with the customer. How can I take the feedback from customer reviews and reach it to my engineers in the shortest possible time,” Anirban Dey, head of the HANA development at SAP Labs India recently said at a media event in Thailand. SAP is taking the big data analytics to a whole new level by trying to figure how to understand sentiments and emotions from tweets and Facebook posts.
Some of the sectors likely to immediately benefit from Big Data adoption in India include computer & electronic products, financial & insurance and government. In addition, sectors such as Healthcare and Telecom are also likely to benefit a lot from Big Data.
For example, in the delivery of healthcare services, management of chronic or long-term conditions is expensive. Use of in-home monitoring devices to measure vital signs, and monitor progress is just one way that sensor data can be used to improve patient health and reduce both office visits and hospital admittance.
Manufacturing companies deploy sensors in their products to return a stream of telemetry. Sometimes this is used to deliver services that delivers communications, security and navigation services. Perhaps more importantly, this telemetry also reveals usage patterns, failure rates and other opportunities for product improvement that can reduce development and assembly costs.
The proliferation of smart phones and other GPS devices offers advertisers an opportunity to target consumers when they are in close proximity to a store, a coffee shop or a restaurant. This opens up new revenue for service providers and offers many businesses a chance to target new customers.
Retailers usually know who buys their products. Use of social media and web log files from their ecommerce sites can help them understand who didn’t buy and why they chose not to, information not available to them today. This can enable much more effective micro customer segmentation and targeted marketing campaigns, as well as improve supply chain efficiencies.
Finally, social media sites like Facebook and LinkedIn simply wouldn’t exist without big data. Their business model requires a personalized experience on the web, which can only be delivered by capturing and using all the available data about a user or member.
“It feels as if data now rivals oxygen as the most vital human element. This flood of information creates a wealth of opportunities for businesses,” says Oracle’s Krishnan.
But all this will also need professionals who know how to handle the data deluge. EMC Corporation, another major vendor for Big Data solutions, has launched professional training and certification courses in cloud computing and big data analytics courses in India. The NYSE-listed company aims to train over 30,000 engineering students in the first year itself as part of its EMCAcademic Alliance (EAA) programme.
“While India has the necessary technical manpower, it is important to skill them in these new areas in order to leverage the power of analytics and deliver benefits,” says EMCIndia & SAARC President Rajesh Janey.
What is Big Data
Data becomes "big data" when it basically outgrows your current ability to process it, store it, and cope with it efficiently. Storage has become very cheap in the past decade, which means it has become easy to collect mountains of data. However, our ability to actually process the mountains of data quickly has not scaled as fast. Traditional tools to analyse and store data -- SQL databases, spreadsheets, the Chinese abacus -- were not designed to deal with vast data problems.
The amount of information in the world is now measured in zettabytes. A zettabyte, which is 1021 bytes (that is 1 followed by twenty-one zeroes), is a big number. Imagine you wrote three paragraphs describing your favorite movie - that's about 1 kilobyte. Next, imagine you wrote three paragraphs for every grain of sand on the earth -- that amount of information is in the zettabyte range.
You may "only" have some number of terabytes in your databases, but you still have a lot of data to work with. And that number is only going to balloon in size every year.