When is Big Data Technology Appropriate?

Big data has the ability to help achieve new mission areas by harnessing the power of vast and rapidly changing data sets, as noted in our previous blog, Big Data Enables New Missions. As TechAmerica stated in its report, Demystifying Big Data, “the impact of big data has the potential to be as profound as the development of the Internet itself.”

However, big data technology is not appropriate for use on all data sets. So, when do you use big data technology?

• When you have a high volume of data that is growing fast
• When your data is rapidly changing (big data tools are well suited for data sets with a high velocity, which means data is constantly going in and out of the set)
• When you have data from multiple sources or in a variety of formats (big data technology can be used to integrate data from a variety of sources, enabling analysts to reach across a range of data points that might not conventionally go together)

It is in vogue to wave the Big Data wand at all analytical problems, but there is an added cost in building tools, training, and infrastructure for an effective Big Data platform. Organizations achieve the best results by aligning technology and tools to the mission problems at hand. Today, many problems are still solvable using traditional data analysis platforms.