Connecting an AI model to external tools creates a system whose security depends on more than the model’s answer. NCSC guidance and NIST experiments explain risks at the boundary between instructions, data and actions.
AI orchestration security starts with the connected system
The NCSC says no model exists in isolation and gives rules-based limits around a model as an example of whole-system design. If model output can trigger actions in other applications, those applications and their access controls form part of the security boundary. This is a design consideration, rather than proof that an orchestration framework makes every workflow safe.
https://www.ncsc.gov.uk/blog-post/thinking-about-security-ai-systems
External data can carry hostile instructions
NIST describes agent hijacking as indirect prompt injection through material such as an email, file or website. Its evaluation extends simulated tasks to risks including data exfiltration and unintended command execution. The work illustrates why a connected agent must not treat every instruction embedded in task data as authorised by its user. These are evaluation scenarios, not a measured incident rate for every UK deployment.
https://www.nist.gov/news-events/news/2025/01/technical-blog-strengthening-ai-agent-hijacking-evaluations
Evaluate the task and the surrounding controls
NIST discusses adaptive tests, task-specific consequences and repeated attack attempts, which can change the picture given by a single average success rate. The NCSC separately highlights supply-chain security, updates and appropriate access controls. Together these sources support examining the actual workflow and its authority; they do not establish a universal winning product or a guarantee of secure automation.
https://www.nist.gov/news-events/news/2025/01/technical-blog-strengthening-ai-agent-hijacking-evaluations
https://www.ncsc.gov.uk/blog-post/thinking-about-security-ai-systems
Sources and further reading
https://www.ncsc.gov.uk/blog-post/thinking-about-security-ai-systems
https://www.nist.gov/news-events/news/2025/01/technical-blog-strengthening-ai-agent-hijacking-evaluations