Not all attacks on AI happen in a single, perfectly crafted prompt. Many unfold like a conversation, where attackers probe, adapt and exploit the system’s responses over multiple turns.
This iterative process, known as multi-turn attacks, manipulates AI models by reframing prompts, adding constraints or leveraging their helpful tendencies.
Over time, these small adjustments can lead to serious failures such as leaking sensitive information, violating policies or drifting away from safe behavior.
In this blog, we’ll explore why multi-turn attacks are a growing concern and how Fortify is tackling them with advanced, realistic testing strategies.
How context window limitations make AI vulnerable to multi-turn attacks
This growing concern over multi-turn attacks is further compounded by the limitations of AI context windows. A context window is the text the model can “read” at once: your prompt plus recent conversation. A bigger window lets you include more, but it doesn’t guarantee the model will consistently use the most important details as the conversation gets long.
When there’s a lot of text, models often rely on shortcuts. They tend to focus more on what was said most recently (and sometimes what was said first), and they can miss key details buried in the middle or surrounded by distracting content.
NeedleBench (Li et al., 2024) evaluates retrieval and reasoning at different context lengths and information densities, and shows models can struggle when relevant details are buried among distractors or spread throughout the input. Salvatore et al. (2025) describe “lost-in-the-middle” behavior, where performance degrades when crucial information sits in the middle of a long context.
Multi-turn attacks: an iteration strategy, not just more prompts
Understanding the differences between single-shot and multi-turn testing is key to creating strong AI security. Single-shot testing gives quick, broad insights, while multi-turn testing shows how attackers can exploit weaknesses over multiple interactions.
Research shows multi-turn dynamics can increase risk. Agarwal et al. (2024) study prompt leakage specifically in multi-turn interactions, showing that iterative strategies can dramatically increase successful leakage compared to simpler baselines. Ha et al. (2025) add a useful perspective: once attackers discover a working multi-turn jailbreak, they can sometimes compress that winning sequence into a short, reusable prompt.
That’s why single-shot and multi-turn testing are complementary: multi-turn helps you discover what a persistent adversary can do, and single-shot helps you scale and regression-test the resulting attack artifacts.
Fortify’s multi-turn testing approach
Fortify’s Multi-Turn Attack Support is designed to simulate a persistent attacker, not a one-liner.
In multi-turn mode, Fortify runs a five-turn adversarial conversation. Each step is a distinct attempt, and follow-ups are grounded in the full conversation history rather than repeating the same idea with minor rewrites.
A key capability is on-the-fly technique generation: if initial approaches fail, the attacker can generate a new technique (an “instruction”) informed by the conversation and continue probing. This mirrors how human red-teamers work: observe the failure mode, adapt, and try a different approach.
Fortify now automates this process, making it easier than ever to identify and address vulnerabilities. This new feature ensures that your AI models are resilient against sophisticated, multi-turn attacks.
[INSERT DEMO HERE]
Single-shot vs. multi-turn testing: when to use each
While single-shot and multi-turn testing serve different purposes, they are both essential for a comprehensive AI security strategy.
- Single-shot testing is ideal for quick, broad coverage. It’s cost-effective and works well for baseline checks, broad vulnerability scans, and regression testing.
- Multi-turn testing focuses on depth and realism. It simulates persistent adversaries who adapt and probe over time, making it especially valuable for high-risk scenarios, such as securing sensitive workflows or protecting AI systems against determined attackers.
Since multi-turn testing involves multiple interactions and evaluations, it’s more resource-intensive. That’s why Fortify combines both approaches: single-shot testing for broad, low-cost coverage and multi-turn testing for high-impact, realistic threat simulations. Together, they provide a well-rounded defense against evolving AI threats.
The future of AI security testing
In short, single-shot is breadth, multi-turn is depth, and you need both to understand real risk. With Fortify’s new Multi-Turn Attack Support, you can ensure your AI models are thoroughly tested and secured against the most persistent adversaries.
References (2024–2025)
- Agarwal et al. (2024). Prompt Leakage effect and mitigation strategies for multi-turn LLM Applications. EMNLP 2024 Industry Track. ACL Anthology · PDF
- Li et al. (2024). NeedleBench: Evaluating LLM Retrieval and Reasoning Across Varying Information Densities. arXiv:2407.11963. arXiv · PDF
- Ha et al. (2025). M2S: Multi-turn to Single-turn jailbreak in Red Teaming for LLMs. arXiv:2503.04856. arXiv · PDF
- Salvatore et al. (2025). Lost in the Middle: An Emergent Property from Information Retrieval Demands in LLMs. arXiv:2510.10276. arXiv · PDF
