An engineer at the software development firm Softjourn narrowly avoided installing a malicious software package after an AI programming assistant recommended it. The incident highlights a new supply chain attack vector where threat actors register packages with names "hallucinated" by AI models, a practice dubbed "slopsquatting."
According to Sergiy Fitsak, Softjourn's managing director, an engineer requested a package recommendation from an AI agent for a routine task. The AI suggested a package with a name that sounded legitimate and resembled a familiar library.
Softjourn's internal policy, however, requires developers to verify any software recommendations from AI. Following this procedure, the engineer reviewed the recommended package's source code on GitHub. The review revealed that the package had very few downloads and had only been created a few days prior, raising immediate suspicion.
Fitsak explained that AI models occasionally generate plausible-sounding package names that do not correspond to existing software. Attackers have begun to exploit this by registering actual malicious packages under these invented names, anticipating that developers might install them without thorough verification, especially under time pressure.
Had Softjourn's engineer not followed the verification protocol, the company could have inadvertently installed malware, potentially creating a backdoor into their systems, enabling data theft, or other forms of compromise. The specific payload of the identified malicious package was not detailed.
Fitsak emphasized the importance of human oversight in the software supply chain, particularly when integrating AI-generated recommendations. He noted that Softjourn's established habit of checking download counts and reviewing source code on GitHub for AI-recommended packages, even for seemingly routine ones, was crucial in preventing a potential supply chain compromise. This process, while taking a few extra minutes, is seen as a vital safeguard against such emerging threats.
