Artificial intelligence (AI) is already embedded deep into the economic and social fabric of the world. It does everything from operating website chatbots to authenticating users with their bank. It keeps planes in the sky and cars on the road. It identifies criminals and reviews mortgage and job applications. But as it becomes more important to our daily lives, regulators are getting nervous.
In Europe, they have produced the world’s first comprehensive AI regulation: the EU AI Act. Among many other things, it will mandate providers of general-purpose AI (GPAI) and high-risk AI to maintain high levels of cybersecurity. Whatever the AI, these plans should begin with protecting the data it's trained on.
What the AI Act says
The EU AI Act classifies AI technology according to risk. Systems labelled as posing “unacceptable risks,” such as social scoring and real-time facial recognition, are prohibited. High-risk systems are subject to strict compliance requirements, as are certain GPAI systems. Limited-risk AI developers must only alert users to the presence of the tech in their systems.
High-risk systems include those deployed in biometrics, critical infrastructure, public services, education and democratic processes. According to legal experts, the providers of such systems must deploy risk management processes, carry out data governance, and ensure they “achieve appropriate levels of accuracy, robustness, and cybersecurity.”
GPAI models are thus named because they can perform a wide range of tasks in downstream systems and applications. Providers must draw up detailed technical documentation about training, testing, evaluation and more. However, GPAIs that pose “systemic” risks must also have measures in place to assess and mitigate these risks, track, document, and report serious incidents, as well as ensure an “adequate level” of cybersecurity protection.
What could go wrong?
Any cybersecurity measures such as these should begin with data protection. Why? Because it is data that makes AI models what they are. This data is also potentially valuable to threat actors. It might be financial or personal information that could be used to commit identity theft. It could be intellectual property, which can give rival organizations a competitive advantage. Or it could be corporate or personal data that malicious actors could use to extort victims with.
Hackers could also look to access, delete and alter that data, or even insert new data into it, in order to manipulate or sabotage the model’s operation. These are known as “data poisoning” attacks.
These scenarios could cause major financial and/or reputational damage to the AI provider and any downstream corporate users. In the case of data poisoning, there could even be a physical safety risk, depending on the context. There’s also a compliance risk from unauthorized third parties accessing sensitive training data.
How data-centric security works
That’s why mitigating AI risk should always begin with a data-centric security strategy. This starts with continuous, automated data discovery, classification, and protection that are in line with policy. It means that, wherever sensitive data resides in the organization, it is protected in a way (ie tokenization, encryption) that will render it useless to any third-party hacker.
Next should come additional protections such as:
- Robust role-based access controls and multi-factor authentication (MFA)
- Data loss prevention (DLP)
- Continuous monitoring and real-time auditing
- Regular security training for employees
- A comprehensive data governance framework
The EU AI Act is a first-of-its-kind piece of regulation. But it certainly won’t be the last. Organizations with a presence inside the bloc, and those which want to meet the highest standards of AI safety and security, are already working hard to see how they can comply. Is your organization?
