The term “Big data” has been around for a while already (almost 15 years now) and the hype is almost over. So is it not relevant anymore?
Over 90% of the data that exists today was created in the last two years. Looking at emerging technologies like IoT this trend won’t stop. We have to find ways to make use of it.
Big data delivers big insights, with seemingly endless use cases and benefits — from improving how companies market to customers to stopping attempted financial fraud, from tailoring cancer treatment for better outcomes to reducing the number of traffic accidents.
How to handle the huge amount of data
It’s not only the use cases that are expanding, the infrastructure and the environment where big data is collected and analyzed are expanding too. In fact, big data is literally everywhere today - streaming from sensors and devices, moving across the internet and of course in data centers.
Many organizations are struggling to find the right way to use and store data. While many started with on-premises systems, more and more cloud providers have started offering comprehensive solutions. Today, many organizations have already shifted parts of their big data project to the cloud.
There’s no disputing that the cloud is one of the best things that ever happened in terms of advancing and accelerating innovation across nearly every industry. For big data analytics, cloud computing and cloud data storage give organizations a flexible, scalable and cost-effective way to store and analyze massive volumes of data.
Keeping in mind that Big Data also contains valuable, often very sensitive data – organizations struggle to use the cloud for “everything”. Therefore, we see a lot of hybrid environments, where parts of the data are on premises and parts of it in the cloud.
Another concern is that many organizations don’t want a vendor lock-in. Imagine a retailer using AWS for cloud projects, which also happens to be the service of one of their strongest competitors in the market. With that we see a lot of companies using multiple cloud services to make sure they can’t run into a trap.
But that’s not the only challenge with adopting cloud for big data analytics.
Security concerns
While cloud computing comes with a lot of benefits, it also dramatically increases companies’ attack surface. In the end, it’s just someone else’s data center – and you have to be very careful with trusting vendors here. Some of that data that organizations send to cloud providers ends up in the hands of attackers who sell it for use in malicious activities.
Data is constantly moving between different environments, databases, and applications, which can be on-premises, in the cloud or a combination of both. Often, data is not protected well and too frequently it’s not protected at all. Using hybrid or multicloud environments only complicates your infrastructure, and the more complex the infrastructure gets, the harder it becomes to protect it.
Many enterprises don’t realize that they ultimately must take steps to protect their data in the cloud beyond turning on the minimum viable security provided by cloud vendors and other third parties.
No matter where the data goes and resides, its security represents one of the biggest obstacles to companies looking to reap the benefits of big data analytics. When enterprises cannot reliably and consistently secure sensitive data used in big data analysis, it often leads to refused access, risk of non-compliance or even breaches.
