AI and LLM Security Testing
for Chatbots, RAG
and AI Agents
AI and LLM security testing
Adversarial Testing for the AI Features You Are Putting in Front of Users
Large language models follow instructions from any text they read, which makes chatbots, retrieval-augmented generation and AI agents a new kind of attack surface. iSpyCyber tests your AI applications the way attackers do, from direct and indirect prompt injection to data leakage, excessive agency and insecure output handling, and tests the application, data and model supply chain around them. Every finding is reproduced, risk-rated and delivered with practical guardrails.
Why organisations choose iSpyCyber
Application, not just model
We test the full system, including prompts, tools, retrieval, plugins, APIs and the data they can reach.
Prompt injection depth
Direct, indirect and multi-turn injection through user input, documents, web pages, emails and tool output.
Agent and tool safety
Excessive agency, tool misuse and privilege boundaries in agents that can act on your systems.
Reproducible results
Non-deterministic behaviour is tested repeatedly, and every finding includes the prompts and conditions to reproduce it.
Practical guardrails
Fixes span system design, access control, output handling and monitoring, not just prompt wording.
Continuous validation
Retesting confirms every fix, and iefyx, our AI-native VAPT platform, keeps the surrounding exposure visible between engagements.
Coverage
What We Test in Your AI and LLM Applications
Our testers work through the OWASP Top 10 for LLM Applications and MITRE ATLAS techniques, then test the application and infrastructure around the model.
Prompt injection
Direct and indirect injection, jailbreaks and multi-turn manipulation that override instructions or policy.
Sensitive information disclosure
Leakage of personal data, secrets, other users' data and confidential training or context data.
System prompt leakage
Extraction of system prompts, hidden instructions, credentials and business rules embedded in them.
RAG and vector stores
Retrieval poisoning, document-level access control, embedding weaknesses and cross-tenant retrieval.
Agents and excessive agency
Tool and plugin permissions, unsafe actions, confirmation bypass and chained tool abuse.
Improper output handling
XSS, SQL, command and SSRF risks when model output reaches browsers, databases, shells or APIs.
Model and data supply chain
Third-party models, adapters, datasets, serialisation formats and ML dependencies.
Misinformation and abuse
Harmful, misleading or off-policy output and abuse of the application for unintended purposes.
Unbounded consumption
Cost and resource abuse, rate limiting, context flooding and denial of wallet.
Methodology
A Proven Six-Phase Testing Methodology
Aligned to the OWASP Top 10 for LLM Applications, MITRE ATLAS and the NIST AI Risk Management Framework, and tailored to your use cases, data and users.
- OWASP Top 10 for LLM Applications
- MITRE ATLAS
- NIST AI RMF
- ISO/IEC 42001
- CVSS 3.1
2–4
weeks for a typical engagement
6
standards-aligned phases
100%
findings manually validated
01
- Days 1–2
Scope and plan
AI features, models, tools, data sources, users and acceptable-use policies agreed up front.
Key activities
- Confirm AI features, models and integrations in scope
- Agree test accounts, usage limits and data handling
- Document intended behaviour and policy boundaries
- Output: Signed rules of engagement
02
- Days 2–4
Map and model
Prompts, retrieval sources, tools, permissions and data flows mapped and threat-modelled.
Key activities
- Map system prompts, tools and retrieval sources
- Identify trust boundaries and untrusted inputs
- Threat-model injection, leakage and agent abuse
- Output: AI threat model and test plan
03
- Days 3–10
Discover
Manual adversarial testing supported by curated attack libraries and repeated trials.
Key activities
- Direct, indirect and multi-turn prompt injection
- Data leakage and RAG access control testing
- Agent, tool and output handling testing
- Output: Candidate vulnerability list
04
- Days 7–12
Exploit and validate
Impact safely proven, attack chains built and results reproduced across repeated runs.
Key activities
- Prove data access or unsafe actions with evidence
- Measure reproducibility across repeated attempts
- Rate every finding by impact and likelihood
- Output: Validated, risk-rated findings
05
- Days 12–15
Report and debrief
Executive summary, technical findings and a walkthrough with product, ML engineering, security and leadership.
Key activities
- Executive summary for leadership
- Findings mapped to OWASP LLM Top 10 and ATLAS
- Live walkthrough and Q&A with your teams
- Output: Final report and debrief
06
- After remediation
Fix and retest
Guardrail and design guidance, and retesting of each issue against the updated system.
Key activities
- Engineering Q&A while guardrails are built
- Retest every remediated finding
- Updated report confirming closure
- Output: Retest report
Effort distribution
Where the testing time goes
Most of every engagement is spent on hands-on adversarial testing and repeated validation by skilled testers. Percentages are indicative for a typical grey box LLM application test.
Quality built into every phase
Every finding reproduced
Prompts, context and success rates are recorded for each issue.
Risk rated in context
Severity reflects the data, tools and users each feature can reach.
Agreed rules of engagement
Usage limits and test data protect live services and costs.
Closure confirmed
Retesting proves each guardrail actually works.
Engagement models
Choose the Testing Approach That Fits Your Risk
Every approach uses the same methodology. The difference is how much context our testers start with, and how deep they can go.
Black box
Only the public AI feature, simulating an external user or attacker. Best for customer-facing chatbots and validating public exposure.
Grey box
Accounts for each role and architecture notes on tools and retrieval to test in depth. Best for most AI applications, copilots and RAG systems.
White box
System prompts, tool definitions, code and configuration for the deepest coverage. Best for autonomous agents and AI handling regulated data.
Not sure which approach you need?
Our specialists will recommend a scope based on your AI use cases, data sensitivity and governance obligations.
Deliverables
Reports Built for Leadership, Engineering and Auditors
Clear business context for decision-makers, precise technical detail for engineers and evidence your auditors can rely on.
Executive summary
Overall AI risk posture, key themes and business impact in plain language for leadership, risk and AI governance teams.
Technical findings
Every issue with severity, affected feature, the prompts and context to reproduce it and observed success rate.
Remediation roadmap
Prioritised guardrails across design, access control, output handling and monitoring so the highest-impact issues are resolved first.
Retest report
Independent confirmation of which findings are closed, ready to share with customers, risk committees and auditors.
Supports your compliance and assurance requirements
- OWASP Top 10 for LLM Applications
- MITRE ATLAS
- NIST AI RMF
- ISO/IEC 42001
- ISO/IEC 27001
- EU AI Act
- SOC 2
- DPDP Act 2023
- CERT-In guidelines
- GDPR
iSpyCyber provides independent security testing that supports these requirements. Certification and attestation are issued by accredited bodies and licensed auditors.
AI and LLM security testing for every industry
FAQ
AI and LLM security testing questions, answered
Can’t find what you need? Talk to our security team.
The basics
What is LLM security testing?
LLM security testing is a hands-on adversarial assessment of applications built on large language models, covering prompt injection, data leakage, agent and tool misuse, output handling and supply chain risks, with guidance on how to fix each issue.
Do you test the model itself or our application?
Primarily your application: the prompts, retrieval, tools, permissions and data around the model, because that is where most exploitable risk sits. Where you host or fine-tune models, we also assess the model supply chain and serving infrastructure.
Which platforms do you test?
Applications built on hosted models such as OpenAI, Anthropic, Google and Azure OpenAI, and on self-hosted open-weight models, using frameworks such as LangChain, LlamaIndex and custom agent code.
How do you handle non-deterministic results?
We repeat attacks across multiple runs and record prompts, context and success rates, so findings can be reproduced and fixes measured.
Does this help with the EU AI Act or ISO/IEC 42001?
Our testing provides independent evidence for AI risk management and governance programmes. Conformity assessment and certification are performed by the appropriate bodies.
Working with us
How long does an AI security test take?
Most engagements take two to four weeks, depending on the number of AI features, tools, data sources and roles in scope.
Do you retest after we fix the issues?
Yes. We retest remediated findings against the updated system and issue an updated report confirming which issues are closed.
Get started
Ship AI Features Safely.
Without Slowing Innovation.
Talk to our AI security specialists about a scoped test of your chatbots, RAG systems and AI agents.
- ISO/IEC 27001 and ISO 9001 certified
- 40+ enterprise and government clients
- 15+ countries