From prompt injection to Promptware: Evolution of attacks against LLM applications
Ben Nassi
VideoAbstract:
In this talk, we trace the evolution of attacks against LLM-powered applications in production and show how they have grown increasingly sophisticated and damaging—progressing from simple prompt injections into a new class of malware we call Promptware. We review this evolution through real-world test cases and incidents, and recent studies that we published including Invitation Is All You Need! (published at BHUSA'25) and Here Comes The AI Worm (to appear at CCS'25).
We analyze how Promptware has transformed over the past three years across multiple dimensions:
Attack vectors: shifting from direct prompt injection to indirect injection and full supply chain attacks.
Severity: moving from trivial manipulations (e.g., making a chatbot curse a user) to high-impact exploits (e.g., forcing an assistant to start a live video stream).
Sophistication: evolving from straightforward injections to sophisticated injections intended to bypass guardrails (e.g., delayed tool invocations).
Outcomes: escalating from confidentiality violations (e.g., data exfiltration) to direct financial losses (e.g., unauthorized Ethereum transfers).
Domains: extending from purely digital effects to physical consequences (e.g., activating a boiler or opening windows in a smart home).
Lateral Movement: including: (1) off-device lateral movement between different LLM applications (2) in-device lateral movement, breaking free from a single agent or application to compromise adjacent systems.
Persistence: shifting from transient manipulations to long-lived compromises controlled via a remote C2 channel.
Finally, we look ahead to emerging Promptware variants that feature pure 0-click execution, applications in zero-knowledge of the target applications, and discuss an ongoing study intended to secure against non-textual Promptware variants.
Bio:
Ben Nassi is a faculty member at the School of Electrical and Computer Engineering at Tel Aviv University (since 2025) and a Black Hat board member (Asia & Europe) since 2022. Before, he was a research fellow at the Faculty of Electrical and Computer Engineering at the Technion, did a postdoc at Cornell Tech (hosted by Tom Ristenpart) and a PhD at The Ben-Gurion University of the Negev (supervised by Yuval Elovici).