There is an apocryphal story about a nuclear scientist at the Los Alamos proving grounds in the late 1940s. When informed that the Soviet Union had sent spies to infiltrate the complex and steal the secrets of the atomic bomb, he replied that the U.S. should give the Soviet Union all the data on those first atomic bomb tests. His reasoning was, of course, that the Soviets would take years to get through the data regardless what their spies told them and thus not actually build a bomb for many more years.
The Easy Part – Collection & Storage
The prescient scientist’s point was that the Soviets would be dealing with more data than they were prepared to analyze. Twenty-first century companies face the very same problem, albeit at much lower stakes. Most large national and multinational companies efficiently collect and store petabytes or even exabytes of data. In turn this data is “mined” to spot business trends, consumer buying habits and other more esoteric but still practical information.
It Gets More Difficult – Search & Analysis
Increasingly large amounts of data are collected and stored but to no discernible advantage. While the data is actually there, the in-house relational database tools are simply not equipped to deal with such a massive amount of data in anything like a realistic time frame. Instead, companies are turning to solutions that combine in-memory computing engines with the distributed parallel processing power of multiple servers.
The capabilities of these new tools allow managers to make informed decision and garner new insights based on the entire library of information and not just portions. It is, indeed, a significant leap forward when a company can make intelligent use of its data and not just trust “informed guesses.”
The Hardest Part – Action & Results
The newest generation of “big data” solutions is just a part of the answer to fully utilizing big data. A fundamentally new approach must be undertaken to turn the insights provided by big data into actionable results. With the newest computers and database analytics, both structured and unstructured data can be combined from multiple sources to yield unprecedented and tangible results.
In short, the best type of big data analysis can establish the who, what and where of your business enterprise. The process, however, requires a solid combination of people, processes and analytical tools. Only in this way can your company fully realize the insights and benefits of big data analysis.
