The UK’s cybersecurity agency has warned that chatbots can be manipulated by hackers to cause scary real-world consequences.
The National Cyber Security Centre (NCSC) has said there are growing cybersecurity risks of individuals manipulating the prompts through “prompt injection” attacks.
To continue reading this report in The Guardian, go to:
theguardian.com/technology/2023/aug/30/uk-cybersecurity-agency-warns-of-chatbot-prompt-injection-attacks
Also see:
Exercise caution when building off LLMs: Large Language Models are an exciting technology, but our understanding of them is still ‘in beta’.
Since the release of ChatGPT in late 2022, Large Language Models (LLMs) have attracted global interest and curiosity. Whilst initially this saw unprecedented numbers of user signups to ChatGPT1, in recent months we’ve seen products and services built with LLM integrations for both internal and customer use. Organisations in all sectors report they are investigating building integrations with LLMs into their services or businesses.
As a rapidly developing field, even paid-for commercial access to LLMs changes rapidly. With models being constantly updated in an uncertain market, a startup offering a service today might not exist in 2 years’ time. So if you’re an organisation building services that use LLM APIs, you need to account for the fact that models might change behind the API you’re using (breaking existing prompts), or that a key part of your integrations might cease to exist.
In addition to the risks of working in a very rapidly changing and evolving market, LLMs occupy an interesting blind spot in our understanding. Only a few years ago, when asked to think of machine learning (ML) and artificial general intelligence (AGI), people could understand the difference. ML was perceived as ‘good at things like classifying whether an image had a cat in it’. AGI as a concept didn’t really exist yet (but whatever it was, ‘we’d know it when we saw it because it would act like us’ and then possibly lock the pod bay doors).
So amongst the understandable excitement around LLMs, the global tech community still doesn‘t yet fully understand LLM’s capabilities, weaknesses, and (crucially) vulnerabilities. Whilst there are several LLM APIs already on the market, you could say our understanding of LLMs is still ‘in beta’, albeit with a lot of ongoing global research helping to fill in the gaps.
ncsc.gov.uk/blog-post/exercise-caution-building-off-llms
Thinking about the security of AI systems: Why established cyber security principles are still important when developing or implementing machine learning models.
If you’re reading this blog, there’s a good chance you’ve heard of large language models (LLMs) like ChatGPT, Google Bard and Meta’s LLaMA. These models use algorithms trained on huge amounts of text data which can generate incredibly human-like responses to user prompts.
In our previous blog (ChatGPT and large language models: what’s the risk?) we discussed some of the cyber security considerations to be aware of when using LLMs. This time we’ll dig a bit deeper into a couple of the specific vulnerabilities in LLMs, namely:
- ‘prompt injection’ attacks
- the risk of these systems being corrupted by manipulation of their training data
However, LLMs aren’t the only game in town. The cyber security fundamentals still apply when it comes to machine learning (ML), and we’ll also highlight a few other things to be aware of if you’re involved in scoping, developing or implementing an ML project.
ncsc.gov.uk/blog-post/thinking-about-security-ai-systems