Smashing Security podcast #470: This AI security flaw might be impossible to fix

A website that purported to assist travelers with UK visa applications has been found to have collected sensitive personal data, including passport scans and selfies, from thousands of users. The data was reportedly stored in an unsecured Amazon storage bucket, making it accessible to unauthorized individuals. When a journalist attempted to alert the company operating the website, they were met with a legal response rather than immediate action to address the security vulnerability.

The issue came to light through reporting by Zack Whitaker at TechCrunch. The website, known as UK Visa Portal, collected information that users believed they were submitting through official government channels. This data included passport copies, selfies, and precise GPS location data from the time the selfies were taken. The discovery that this information was stored in an open Amazon storage bucket raises significant privacy concerns for those who used the service.

In parallel, research from Cornell University suggests that a prevalent technique used to manipulate artificial intelligence models, known as prompt injection, may be inherently difficult or even impossible to fully resolve. Prompt injection involves crafting specific inputs to trick AI agents into performing unintended actions or revealing sensitive information. This poses a challenge as organizations increasingly integrate AI into critical systems, including email, file management, and corporate networks.

The podcast episode also featured an interview with Andrea Sivieri of CoreView. Sivieri discussed a scenario where attackers could potentially lock an entire organization out of its Microsoft 365 environment. This could be achieved without resorting to tricking users into executing malicious code or compromising their passwords.

During the discussion, Tanya Janca, a special guest on the podcast, highlighted the Cornell paper on prompt injection, noting its potential long-term implications for AI security. Graham Cluley, the host, brought attention to the UK visa portal incident, emphasizing the importance of verifying the legitimacy of websites when submitting personal information.

The podcast also included a promotional segment for CoreView's free Microsoft 365 Security Posture Check tool. This tool is designed to identify security drift, policy exceptions, and overlooked permissions within Microsoft 365 environments that could be exploited by attackers. The tool runs locally and does not transmit tenant data to CoreView.

Janca also shared news about her upcoming book signing event at Black Hat, where she will be signing copies of "Alice and Bob Learn Secure Coding." The book is aimed at individuals who write code or work with AI models that generate code, providing guidance on ensuring code safety. ESET is sponsoring the podcast episode and will be distributing copies of Janca's book at their booth during the event.