Les Bell
Blog entry by Les Bell
Welcome to today's daily briefing on security news relevant to our CISSP (and other) courses. Links within stories may lead to further details in the course notes of some of our courses, and will only be accessible if you are enrolled in the corresponding course - this is a shallow ploy to encourage ongoing study. However, each item ends with a link to the original source.
News Stories
The Risks of Artificial Intelligence
With AI all over the news in recent weeks, I thought it was time to do a bit of a deep dive on some of the risks posed by artificial intelligence. I'll cover just a few of the stories published over the last few days, before concluding with some deeper thoughts based on recent research.
OpenAI Prepares to Study "Catastrophic Risks"
OpenAI, the company behind the GPT-3 and -4 large language models, ChatGPT, and the Dall-E AI image generator, has started to assemble a team, called "Preparedness", which will
"tightly connect capability assessment, evaluations, and internal red teaming for frontier models, from the models we develop in the near future to those with AGI-level capabilities. The team will help track, evaluate, forecast and protect against catastrophic risks spanning multiple categories including:
- Individualized persuasion
- Cybersecurity
- Chemical, biological, radiological, and nuclear (CBRN) threats
- Autonomous replication and adaptation (ARA)
"The Preparedness team mission also includes developing and maintaining a Risk-Informed Development Policy (RDP). Our RDP will detail our approach to developing rigorous frontier model capability evaluations and monitoring, creating a spectrum of protective actions, and establishing a governance structure for accountability and oversight across that development process. The RDP is meant to complement and extend our existing risk mitigation work, which contributes to the safety and alignment of new, highly capable systems, both before and after deployment."
OpenAI, Frontier risk and preparedness, blog post, 26 October 2023. Available online at https://openai.com/blog/frontier-risk-and-preparedness.
Google Adds AI to Bug Bounty Programs
Google, which has a long history of AI research, has announced that it is adding its AI products to its existing Bug Hunter Program. Based on the company's earlier research and Red Team exercises, it has tightly defined several categories of attacks, such as prompt attacks, training data erxtraction, model manipulation, adversarial perturbation, model theft or extraction, etc. and have further developed a number of scenarios, some of which will be in scope for the Bug Hunter Program, and so of which will not.
Interestingly. jailbreaks and discovery of hallucinations, etc. will not be within scope, as Google's generative AI products already have a dedicated reporting channel for these content issues.
The firm has already given security research a bit of a boost with the publication of a short report which describes Google's "Secure AI Framework" and provided the categorisation described above, along with links to relevant research.
Vela, Eduardo, Jan Keller and Ryan Rinaldi, Google’s reward criteria for reporting bugs in AI products, blog post, 26 October 2023. Available online at https://security.googleblog.com/2023/10/googles-reward-criteria-for-reporting.html.
Fabian, Daniel, Google's AI Red Team: the ethical hackers making AI safer, blog post, 19 July 2023. Available online at https://blog.google/technology/safety-security/googles-ai-red-team-the-ethical-hackers-making-ai-safer/.
Fabian, Daniel and Jacob Crisp, Why Red Teams Play a Central Role in Helping Organizations Secure AI Systems, technical report, July 2023. Available online at https://services.google.com/fh/files/blogs/google_ai_red_team_digital_final.pdf.
Handful of Tech Firms Engaged in "Race to the Bottom"
So we are gently encouraged to assume, from these and other stories, that the leading AI and related tech firms already programs to mitigate the risks. Not so fast. One school of thought argues that these companies are taking a proactive approach to self-regulation for two reasons:
- Minimize external regulation by governments, which would be more restrictive than they would like
- Stifle competition by increasing costs of entry
In April, a number of researchers published an open letter calling for a six-month hiatus on experiments with huge models. One of the organizers, MIT physics professor and AI researcher Max Tegmark, is highly critical:
"We’re witnessing a race to the bottom that must be stopped", Tegmark told the Guardian. "We urgently need AI safety standards, so that this transforms into a race to the top. AI promises many incredible benefits, but the reckless and unchecked development of increasingly powerful systems, with no oversight, puts our economy, our society, and our lives at risk. Regulation is critical to safe innovation, so that a handful of AI corporations don’t jeopardise our shared future."
Along with other researchers, Tegmark has called for governments to licence AI models and - if necessary - halt their development:
"For exceptionally capable future models, eg models that could circumvent human control, governments must be prepared to license their development, pause development in response to worrying capabilities, mandate access controls, and require information security measures robust to state-level hackers, until adequate protections are ready."
Milmo, Dan and Edward Helmore, Humanity at risk from AI ‘race to the bottom’, says tech expert, The Guardian, 26 October 2023. Available online at https://www.theguardian.com/technology/2023/oct/26/ai-artificial-intelligence-investment-boom.
The Problem is Not AI - It's Energy
For many, the threat of artificial intelligence takes a back seat to the other existential threat of our times: anthropogenic climate change. But what if the two are linked?
The costs of running large models, both in the training phase and for inference once in production, are substantial; in fact, without substantial injections from Microsoft and others, OpenAI's electricity bills would have rendered it insolvent months ago. According to a study by Alex de Vries, a PhD candidate at VU Amsterdam, the current trends of energy consumption by AI are alarming:
"Alphabet’s chairman indicated in February 2023 that interacting with an LLM could “likely cost 10 times more than a standard keyword search. As a standard Google search reportedly uses 0.3 Wh of electricity, this suggests an electricity consumption of approximately 3 Wh per LLM interaction. This figure aligns with SemiAnalysis’ assessment of ChatGPT’s operating costs in early 2023, which estimated that ChatGPT responds to 195 million requests per day, requiring an estimated average electricity consumption of 564 MWh per day, or, at most, 2.9 Wh per request. ...
"These scenarios highlight the potential impact on Google’s total electricity consumption if every standard Google search became an LLM interaction, based on current models and technology. In 2021, Google’s total electricity consumption was 18.3 TWh, with AI accounting for 10%–15% of this total. The worst-case scenario suggests Google’s AI alone could consume as much electricity as a country such as Ireland (29.3 TWh per year) [my emphasis], which is a significant increase compared to its historical AI-related energy consumption. However, this scenario assumes full-scale AI adoption utilizing current hardware and software, which is unlikely to happen rapidly."
Others, such as Roberto Verdecchia at the University of Florence, think de Vries' predictions may even be conservative, saying, "I would not be surprised if also these predictions will prove to be correct, potentially even sooner than expected".
A cynic might wonder: why does artificial intelligence consume so much power when the genuine article - the human brain - operates on a power consumption of only 12W? It really is quite remarkable, when you stop to think about it.
de Vries, Alex, The growing energy footprint of artificial intelligence, Joule, 10 October 2023. DOI:https://doi.org/10.1016/j.joule.2023.09.004. Available online at https://www.cell.com/joule/fulltext/S2542-4351(23)00365-3.
The Immediate Risk
Having canvassed just some of the recent news coverage of AI threats and risks, let me turn now to what I consider the biggest immediate risk of the current AI hype cycle.
Current large language models (LLM's) are generative pretrained transformers, but most people do not really understand what a transformer is and does.
A transformer encodes information about word position before it feeds it into a deep learning neural network, allowing the network to learn from the entire input, not just words within a limited distance of each other. Secondly, transformers employ a technique called attention - particularly a derivative called self-attention - which allows the output stages of the transformer to refer back to the relevant word in the input sentence as it produces output.
The result - which leads to the key risk - is the impressive performance in conversational tasks, which can seduce non-technical business users into thinking they are a general artificial intelligence - but this is far from the case. In fact, most LLM's work on the statistical properties of the text they are trained on and do not understand it in any way. In this respect, they are actually very similar to the compression algorithms used to encode text, speech, music and graphics for online transmission (Delétang, et. al., 2023). In the same way as the decompression algorithm predicts the colour of the next pixel in a graphics image, so an LLM predicts the mostly likely next word, based upon the statistical properties of the text it has been trained upon. However, this is not always le mot juste.
For example, when Princeton researchers asked Open AI's GPT-4 LLM to multiply 128 by 9/5 and add 32, it was able to give the correct answer. But when asked to multiply 128 by 7/5 and add 31, it gave the wrong answer. The reason is that the former example is the well-known conversion from centigrade to fahrenheit, and so its training corpus had included lots of examples, while the second example is probably unique. GPT-4 simply picked a likely number - it did not perform the actual computation (McCoy et. al., 2023).
Another example found by the researchers was a simple task of deciphering text encrypted using the Caesar Cipher; GPT-4 easily performed the task when the key was 13, because that value is used for ROT-13 encoding on Usenet newsgroups - but a key value of 12, while returning recognizable English-language text, gave the incorrect text.
In short, large language models do not understand the subject matter of the text they process.
Many managers either have never realized this, or forget it in their enthusiasm. And right now, that is the real risk of AI.
Delétang, Grégoire, et. al., Language Modeling Is Compression, arXiv preprint, 19 September 2023. Available online at https://arxiv.org/abs/2309.10668.
McCoy, R. Thomas, Shunyu Yao, Dan Friedman, Matthew Hardy and Thomas L. Griffiths, Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve, arXiv preprint, 24 September 2023. Available online at https://arxiv.org/abs/2309.13638.
Upcoming Courses
- SE221 CISSP Fast Track Review, Virtual/Online, 13 - 17 November 2023
- SE221 CISSP Fast Track Review, Sydney, 4 - 8 December 2023
- SE221 CISSP Fast Track Review, Sydney, 11 - 15 March 2024
- SE221 CISSP Fast Track Review, Virtual/Online, 13 - 17 May 2024
- SE221 CISSP Fast Track Review, Virtual/Online, 17 - 21 June 2024
- SE221 CISSP Fast Track Review, Sydney, 22 - 26 July 2024
These news brief blog articles are collected at https://www.lesbell.com.au/blog/index.php?courseid=1. If you would prefer an RSS feed for your reader, the feed can be found at https://www.lesbell.com.au/rss/file.php/1/dd977d83ae51998b0b79799c822ac0a1/blog/user/3/rss.xml.
Copyright to linked articles is held by their individual authors or publishers. Our commentary is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License and is labeled TLP:CLEAR.