Since time immemorial, data has been the key to success. Traders who understood their customers’ needs and emotions could sell more goods at a higher price. Craftsmen who understood what their customers wanted were able to sell more products, and kings who knew their enemies’ secrets were able to win more battles. In the past, the amount of data at hand was vastly smaller, yet people used what little of it they had to gain leverage whenever possible.
Today, not much has changed.
People still need data to make decisions, and as before, sometimes that data isn’t available to the decision maker. It’s still often the case that decisions are made solely on the basis of a gut instinct. While deciding on instinct might not be a problem when choosing the color of your car, it can be a huge problem for enterprises who are making strategic decisions and investing large amounts of capital.
The good thing is, today, one key factor has changed. Today we have virtually limitless data at our fingertips. And we can do much more with it. Big data isn’t bad; if applied carefully, massive data sets can reveal important trends that would otherwise go undetected. This applies in every situation you could imagine, from hunger to healthcare to energy to law enforcement and so on. Data is used to create insights and helps people better understand reality and grasp the size of the problems around them.
Data is everywhere… and most companies use it.
Many organizations expand their use of data analytics to save on cost and increase revenues. It’s not about buzzwords and trends; it’s about making informed decisions. As of 2018, 84% of enterprises reported they have launched advanced analytics and big data initiatives to improve accuracy and improve decision making. And now we’re seeing more and more live production environments, and it’s still growing, so big data strategy in some shape or form has to be a part of every digital strategy.
However, analysis creates its own set of challenges. First, there’s the volume of data. Where do you begin? How do you know if the data’s any good? Is there data subject to data protection regulations in there? Then there’s the dearth of available talent that makes data analysts seem like a rare and endangered species. Fortunately, advances in AI and machine learning are promising to reduce the manual work that is still needed to scour and gain insights from big data.
But there’s another challenge, perhaps the biggest big data challenge of all, and that is irresponsible usage, which can lead to disaster. And unfortunately, that’s exactly what “big data” as a catchphrase has come to represent.
So what exactly is so bad about mismanagement of big data?
Well, we talked about how data is everywhere, which can be great, however…
Only 6% of companies say they aren’t exposed to any cyber security risks and 62% say digital security threats have increased in the past 12 months.
Now let’s be real:
If you answered "no" to any of the above, then you know your data is at risk.
And therein lies the rub: data is everywhere, but security isn’t.
So why is that? And why are hackers so interested?
Data itself is useless, it’s what you do with it that makes the difference. The job of data analysts is to gain insights from data in order to create value and essentially monetize data. Yet, the better they do their job, the more sensitive the data becomes. Regardless of whether the input data is sensitive or not, if the analyst is doing a good job, the output will be more valuable than the input and subsequently more sensitive. And with that, the data becomes increasingly interesting for bad actors.
There are people out there who know exactly what that data is worth in the underground economy so the threats are real. And the more valuable the data you have and the more value you derive by analyzing that data, the bigger the impact of a breach will be.
Someone recently asked me, “should data scientists, engineers, and analysts care about data protection?” I said “only if they’re any good at what they do.”
So we know what the stakes are, but what's stopping organizations from adequately protecting their data? Is it really that hard to secure a big data analytics environment? Stay tuned for my next post to find out!