My Experience with ARIA

In October I competed in an online LLMs security competition as part of the Assessing Risks and Impacts of AI program. It was hosted by Humane Intelligence, which was supported by a number of federal technology standards agencies. After the Trump administration came into office these agencies have required us not to name them (This article has been updated to reflect that). They put out the call for people across the nation to break production LLMs submitted by model creators. Humane Intelligence covered the flights, hotel, and conference costs for the top 30 scoring participants to attend CAMLIS and do a similar exercise working together in person. I was one of the 30 who beat out the competition (there were over 500 participants) and was able to attend the conference in Washington DC (technically Arlington, Virginia but you could see the capitol building from the room we were red teaming in so I call it what I want).

I got a lot out of this opportunity. I was able to be a part of a team from around the country, (including security engineers at Dropbox, Amazon, and Microsoft), which attacked real life software platforms used by fortune 50 companies.

We were literally writing the book on AI safeguards.

The event was sponsored by the National Institute for Standards in Technology, and we had people from NIST and CISA (Cybersecurity and Infrastructure Security Agency) there to watch and talk with us, as well as industry leaders in the generative AI space. I found multiple vulnerabilities in productivity and office software from multiple vendors.

The conference itself was also an amazing opportunity, I watched talks from AI security experts from leading institutions such as Microsoft, Google Cloud, Booze Allen and Hamilton, and Meta, letting me learn the bleeding edge of security in AI. The talks that I watched gave me a few ideas of projects and research I want to do at UWEC.

After the conference, we were given a chance to work together and write a paper out of the results from the event, the folowing is our finished report. https://humane-intelligence.org/wp-content/uploads/2025/09/camlis-report.pdf