White House officials worried about the potential of AI chatbots for social harm and the Silicon Valley powerhouses rushing them to market invested heavily in the three-day competition that ended Sunday at the DefCon hacker convention in Las Vegas.
About 3,500 competitors tapped laptops trying to uncover the flaws of the eight top models in the big language representative of the next big thing in technology. But don’t expect quick results from this first-ever independent “red-teaming” of multiple models.
The findings won’t be made public until around February. And even then, fixing the flaws in these digital constructions – whose inner workings are not fully reliable or fully understood even by their creators – will take time and millions of dollars.
Current AI models are not very usable, brittle and malleable, academic and corporate research shows. Security is a consideration in their training as data scientists assemble impressively complex collections of images and text. They are prone to racial and cultural prejudices, and are easily manipulated.
“It’s tempting to pretend we can sprinkle some magic security dust on these systems after they’re built, patch them into submission, or bolt special security apparatus on the side,” said Gary McGraw, a cybsersecurity veteran and co-founder of the Berryville Institute of Machine Learning. Competitors at DefCon “are more likely to leave in search of new, difficult problems,” said Bruce Schneier, a Harvard public-interest technologist. “This was computer security 30 years ago. We were just destroying things left and right. Michael Sellitto of Anthropic, which provided one of the AI test models, acknowledged in a press briefing that understanding of their capabilities and safety issues “is an open area of scientific research.”
Conventional software uses well-defined code to issue clear, sequential instructions. OpenAI’s ChatGPT, Google’s Bard and other language models are different. Trained largely by entering – and classifying – billions of data points in internet crawls, they are perpetual works-in-progress, a daunting prospect given their ever-changing potential for humanity.
After the public release of chatbots last fall, the generative AI industry has had to repeatedly plug security holes exposed by researchers and tinkerers.
Tom Bonner of AI security firm HiddenLayer, a speaker at this year’s DefCon, tricked a Google system into marking a piece of malware as harmless by simply inserting a line that said “it safe to use.”
“There are no good guardrails,” he said.
A team including Carnegie Mellon researchers found leading chatbots vulnerable to automated attacks that also produce malicious content. “It is possible that the very nature of deep learning models makes such threats inevitable,” they wrote.
It’s not like the alarms aren’t sounding.
The attacks trick artificial intelligence logic in ways that aren’t even obvious to their creators. And chatbots are especially vulnerable because we interact with them directly in plain language. That interaction can change them in unexpected ways.
Researchers have found that “poisoning” a small collection of images or text in the vast sea of data used to train AI systems can be harmful — and easily overlooked.
A study co-authored by Florian Tramér of the Swiss University ETH Zurich determined that damaging just 0.01% of a model is enough to damage it – and cost $60. Researchers are waiting for some websites used to crawl the web for both models to expire. Then they buy the domains and post bad data on them.
Hyrum Anderson and Ram Shankar Siva Kumar, who red-teamed AI while partners at Microsoft, call the state of AI security for text- and image-based models “pathetic” in their new book that “Not with a Bug but with a Sticker.” An example they cited in live presentations: The AI-powered digital assistant Alexa was tricked into interpreting a Beethoven concerto clip as a command to order 100 frozen pizzas.
Surveying more than 80 organizations, the authors found that most do not have a response plan for a data poisoning attack or data theft. Most of the industry “won’t even know it’s happening,” they wrote.
Andrew W. Moore, a former Google executive and Carnegie Mellon dean, says he faced attacks on Google’s search software more than a decade ago. And between late 2017 and early 2018, spammers hacked Gmail’s AI-powered detection service four times.
Big AI players say security and safety are top priorities and made voluntary commitments to the White House last month to submit their models — often “black boxes” whose inside tightly held – outside check.
Tramér expects search engines and social media platforms to play for financial gain and disinformation by exploiting weaknesses in AI systems. A skilled job applicant can, for example, know how to convince a system that they are the right candidate.
Ross Anderson, a computer scientist at Cambridge University, worries that AI bots will destroy privacy as people interact with them in hospitals, banks and employers and malicious actors use them to extracting financial, employment or health data from allegedly closed systems.
Another concern is company secrets being eaten and spat out by AI systems. After a Korean business news outlet reported such an incident at Samsung, corporations including Verizon and JPMorgan banned most employees from using ChatGPT at work.
While the big AI players have security staff, many smaller competitors likely don’t, meaning poorly secured plug-ins and digital agents can proliferate. The startup is expected to launch hundreds of offerings built on licensed pre-trained models in the coming months.
Don’t be surprised, researchers say, if one escapes your address book.
Photo: People attend the DefCon conference August 5, 2011, in Las Vegas. White House officials are concerned about the potential of AI chatbots for societal harm and Silicon Valley powerhouses rushing them to market are investing heavily in the three-day competition that ends on Sunday, August 13, 2023 at DefCon hacker convention in Las Vegas. About 3,500 competitors tapped laptops trying to expose the vulnerabilities of eight top models in the big language that is representative of the next big thing in technology. (AP Photo/Isaac Brekken)
Copyright 2023 Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or distributed.
Topics
InsurTech Artificial Intelligence
