You have definitely used GenAI already. Maybe you asked ChatGPT to write an email or created a stunning logo with Midjourney. You could even check your code with GitHub Copilot. It’s all GenAI, and it has become a usual thing in our daily lives. Honestly, we are all much faster with AI.
On the surface, GenAI looks innocent. But it has a serious drawback – AI models can be used with wrong intentions and help spread cyber threats. The algorithms that have been created to help us can easily find weak spots in security. So, how is this happening, and what do the security risks of generative AI mean for us all?
What is generative AI?
This is a branch of artificial intelligence that can create new content or even write code. It analyzes information in the same way as traditional AI models, but then it learns from this data and produces something new/ And this new content doesn’t look as if machines made it. In other words, GenAI teaches machines to work in a creative way.
This creativity is achieved thanks to a special neural network that operates like the human brain. The networks study patterns in massive amounts of data, process them and give fresh outputs. People train these models on large datasets and call them foundation models because they can be adapted for many tasks.
Main cybersecurity risks of generative AI
GenAI opens up great opportunities for people. But it also brings new safety challenges. The thing is, cybercriminals use it, too . Now, their attacks are even better masked. Let’s look at the main risks and how they can impact businesses and individuals.
Adversarial attacks
You know well how hard it may be to get what you want from Google. Often, you must be really inventive and try different wording and phrasing until you get what you need. Hackers use this trick, often called prompt injection. So how do they use it? They intentionally mislead AI, and a machine may easily forget about its safety guidelines.
Let’s look back at what happened with Microsoft’s Tay chatbot, which the company designed for X (formerly Twitter). Users attacked it with malicious prompts for fun after its launch, and the chatbot immediately responded in an offensive way. It looked like innocent fun, but it showed how you can affect AI behavior.
Data leaks
Machines remember everything you tell them. So, you must carefully share personal details, as they will stay in the system for a long time if you don’t delete them. Others may also accidentally access this data.
And it does happen in real life. A few years ago, ChatGPT mistakenly revealed other people’s conversation histories. Other users simply found them in their chats. In the other situation, AI disclosed data from individual health records. Such information should never become public. It can bring unbelievable harm to the financial industry and others.
Malicious software
Individuals can now build software, even if they don’t have strong technical skills. AI will do the technical part for them. So, attackers can create viruses and other threats and send them out.
Security experts are warning about WormGPT and FraudGPT, which appear to be malicious versions of ChatGPT and were actually created for cybercriminals. These tools can write realistic phishing emails in perfect English. Moreover, they can generate powerful malware that constantly changes its code, and traditional antivirus software fails to detect it.
Bias issues
Bias is not the issue of the algorithm, but the result of poor training. If people have shared biased opinions during machine training, the AI will only make it worse over time. And it happens much more often than you may think.
There are enough real-world examples to demonstrate it. Amazon AI for their hiring process. Soon, it was found that this system rejected resumes that included the word “women’s.” And this is not a single example of that kind. Unfortunately, people lose their trust in AI because of it.
Harmful AI training
Hackers often train AI intentionally incorrectly. They add harmful or tricky data at the learning stage and create hidden problems. Later, it encouraged AI to provide incorrect answers or behave in ways it shouldn’t.
Researchers demonstrated that only a few hundred malicious images added to a training set of millions can cause problems. For example, facial recognition systems can sometimes identify people incorrectly. If a security company’s AI is trained on poisoned data, it could be programmed to ignore the face of a specific hacker.
Fake content
You should never trust the content you get with the help of AI. Machines can now create very realistic fake videos and audio clips. They have even received a special name – deepfakes.
The UK firm Arup had a very painful and costly experience in 2024. Criminals made a video with their senior executive, and then used this video for a call with a request for a transfer. An employee had no doubts that the call was real and sent $25 million to the wrong account. This is how AI can be used to fool people and bring serious losses.
Privacy and compliance
Generative AI uses an unimaginable amount of data. And sometimes, it does not strictly follow GDPR, HIPAA, and CCPA. Where is the data stored? Who can access it? Is it being used to train public models? Those who use AI to process customer data are at risk, as only one lost detail can lead to massive fines and reputation damage.
Clearview AI was fined in several countries for taking billions of facial images from social media without permission to train its AI. This showed an important rule: you can’t use any online data to train AI.
How to make AI safer
GenAI is a great tool, but you should use it carefully. The first rule is to protect all your data with strong passwords and encryption. If some of your information is super sensitive, it’s better not to share it all. Another rule is to test AI models to find any problems before attackers use them as an advantage. You may test AU for discriminatory results or use tools to recheck AI-generated images, video, or audio. Don’t trust all the results you receive from machines – such information must always be reviewed by humans.
Conclusion
The task of people is to make GenAI creative, not destructive. It must not turn into a new safety weapon. When potential threats are clear, we should act and warn others about what to look out for. The future of cybersecurity depends on all of us. Together, we can protect our data and stop or at least substantially reduce potential attacks.
