There are scores of companies itching to make money from the Big Data Revolution. Some have developed products that can compile reams of data from numerous servers and applications into manageable formats. Other services enable the complex number crunching that’s required to make sense of the data. Several companies help businesses use the data that’s harvested for future projections. And a few customize the applications they’ve developed for their own businesses to resell to their clients.
New applications are arising almost as fast as the amounts of data being generated. Data is multiplying by the second. According to IDS, 1.8 zettabytes (1.8 trillion gigabytes) of data was created in 2011—enough data to fill 57.5 billion 32-gigabyte iPads. That’s a lot of bits and bytes!
Data has just about taken over the world:
• Every single hour, Walmart processes more than one million customer transactions. They are stored in databases estimated to contain 2.5 petabytes (2.5 million gigabytes) of data. In case you’re wondering just how large that is, consider that all of this data would fill 167 times the number of books in the U.S. Library of Congress.
• According to IDS, 1.8 zettabytes (1.8 trillion gigabytes) of data was created in 2011—enough data to fill 57.5 billion 32-gigabyte iPads.
• Facebook users have posted 50 billion photos on the site
This data is coming from every device we own. Between 2008 and 2009, the number of devices connected to the Internet exceeded the 7 billion people on this planet! Furthermore, IMS Research predicts, that by 2020, up to 22 billion embedded systems and other portable devices will be connected to the Internet. And those devices will produce more than 2.5 quintillion bytes of new data every single day. It’s estimated that data is growing at 50% annually, and doubling every two years.
The problem is this. Until recently, most of that data was just collected and stored. Sure, some of it was used to program simple functions, like geolocation for your automobile or synching your iPhone and iPad. But most of it just sat around in big server warehouses because the computing capacity just hadn’t yet caught up.
But some very smart people decided that this trove of data just had to contain some great information that could be harnessed and massaged into value. However, the database management systems that were in vogue just couldn’t handle this incredible volume of information.
That has all changed now. Utilizing what is called “massively parallel software” running on thousands of servers, and powered by super-processing speeds and smarter chips, governments and corporations around the world are throwing money at this new approach to data mining. They’ve discovered that in gathering the data and by looking for correlations, they have found patterns that can predict the future. And the results have been startling, including:
• A blog on the New York Times website knew (not guessed)—months ahead—that Obama would win reelection by crunching data
• Searching words in tweets on Twitter has resulted in predictions of earthquakes, the movement of the Dow Jones Industrial Average, and election outcomes
• Scientists used Google search to forecast the exact location of an influenza outbreak.
• Mining keywords in pre-release critiques of movies has accurately predicted the success or failure of a film.
• Looking at economic data from hundreds of sources, some economists predicted deflation well ahead of the government’s 2008 release affirming it.
Imagine what you could do with those predictive capabilities! Companies are frothing at the mouth thinking of all the possibilities: precise ad placement to your target market, so your ad dollars are never wasted; forecasting flight delays so air traffic can be rerouted without untimely delays; predicting which drugs may have deadly side effects; forecasting who will get breast cancer and who won’t; knowing likely crime “hot spots” in advance—the applications are infinite.
And growing. It’s no surprise that big companies like Oracle (ORCL), IBM (IBM), Microsoft (MSFT) and Hewlett-Packard (HPQ) have spent more than $15 billion scooping up software firms who specialize in big data management and analysis.
Big Data is an Investor’s Dream
The potential to make money from Big Data is tremendous. Whether a compiler or user of big data, there are scores of companies who will make their fortune from being in the right place at the right time in this burgeoning industry. One good example is our Spotlight pick this month, which is using its expertise in big data crunching to make a name for itself in the global digital advertising arena.
