AI Red Teaming: attack it before someone else does
Goal-driven campaigns against your deployed assistant, agent or knowledge base. We think like the attacker, document every route we find, and hand you the evidence to fix it.
- 2–3 week campaigns
- Manual + automated
- Retest included
Systems where a bad day has a number attached
Red teaming is for AI that can move money, expose records, send messages or make decisions about people. It is also the right answer when a customer or regulator asks "has anyone independently tried to break this?"
- Customer-facing agents in fintech, insurance, healthcare and legal.
- Internal copilots with access to HR, finance or customer systems.
- Multi-tenant SaaS products that added an AI feature on top of shared data.
- Teams that have fixed audit findings and want to know what is left.
| Security audit | Red team | |
|---|---|---|
| Starts from | Checklist and design | Attacker goals |
| Covers | Breadth across the system | Depth on what matters |
| Needs | Code and config access | A deployed target |
| Finds | Gaps and exploits | Chained exploits |
| Best when | First review, pre-launch | After hardening, annually |
How a campaign runs
Scope and rules
Agree targets, goals, accounts, environments, what is out of bounds and how to stop. Written rules of engagement, signed by both sides.
Threat scenarios
Three to six attacker goals ranked by business impact, such as "read another tenant's data" or "make the agent send a payment".
Attack
Manual campaigns per goal, backed by automated payload generation. Every attempt is logged, successful or not.
Triage
Reproduce each success twice, rate severity, write the fix. Critical findings are reported the day they are confirmed.
Report and retest
Report, debrief with engineering, and a retest window once fixes land. Probes join your regression suite.
Each goal becomes a tree of routes
We work backwards from the goal. Each branch is a different way in: a content channel, a tool, a retrieval path, an output sink. Branches end in one of two states: a reproducible success with a severity, or a block with the name of the control that stopped it.
Both outcomes are reported. Knowing which controls held is as useful as knowing which failed, because it tells you what to keep when the next feature ships.
Example: a multi-tenant support assistant with a ticket-lookup tool and a vector index of past tickets. Three branches, two of them open.
What we try
The categories below map to the OWASP Top 10 for LLM Applications. Campaigns weight them by the goals you care about.
| Category | Attacker goal | Example techniques | Typical finding |
|---|---|---|---|
| Direct prompt injection | Bypass rules, extract the system prompt | Role play, instruction nesting, encoding, multilingual payloads, many-shot pressure | Policy bypass, leaked business rules |
| Indirect prompt injection | Hijack the model through content | Hidden text in PDFs and web pages, crafted emails and tickets, poisoned tool results | Unintended tool calls, data exfiltration |
| Tool and agency abuse | Make the model act beyond its task | Argument manipulation, scope creep, chained calls, confirmation fatigue | Writes or lookups outside the user's rights |
| Retrieval and tenancy | Read data the user may not see | Cross-tenant queries, ACL gaps, metadata filter bypass, embedding probes | Cross-tenant disclosure |
| Output exploitation | Turn output into an exploit | Markdown and HTML injection, link smuggling, SQL or shell fragments in output | Stored XSS, data exfil via rendered links |
| Privacy and leakage | Extract personal or secret data | Memory probing, log inspection, summarisation of restricted records | PII in answers or logs |
| Resource exhaustion | Cost or availability damage | Long-context floods, recursive tool loops, parallel sessions | Denial of wallet, degraded service |
| Safety and brand | Produce harmful or off-policy output | Jailbreaks, persona drift, competitor and legal-advice scenarios | Reputational and compliance exposure |
How we report
Reproducible findings
Exact payloads, accounts, steps and timestamps. Each success was reproduced twice before it entered the report, so your engineers can see it on their first attempt.
Severity you can argue with
Likelihood and impact are justified separately, using the same matrix as our audit. Business context from your side can move a rating; we record why.
Controls that held
Blocked branches are listed with the control responsible. This becomes your "do not remove" list and your evidence for questionnaires.
Probes for CI
Every payload becomes a test in your pipeline, with expected behaviour. Model upgrades and prompt edits run against the full set before release. See how we evaluate LLM outputs.
Fixing what a campaign finds
Frequently asked questions
How is red teaming different from an LLM security audit?
An audit reviews design, code and configuration for coverage. A red team starts from attacker goals and tries to reach them by any route, in the deployed system. Do the audit first if you have never had a review; red team once the obvious gaps are closed.
Do you use automated attack tools?
Yes, for breadth: payload libraries, mutation and model-assisted generation of variants. The findings that matter most usually come from manual work that chains a weakness in retrieval, a tool and an output path. See red teaming in our glossary.
Is a red team safe to run against production?
We prefer staging with production-like data. If production is unavoidable, we agree read-only goals, test accounts, rate limits and a stop condition, and we never target real customers' data.
How often should we run a campaign?
After any major change to tools, content sources or model, and at least annually. Between campaigns, the regression probes we hand over run in CI.
Can you test safety and brand behaviour as well as security?
Yes. Campaigns can include harmful-content, off-policy advice and competitor or reputational scenarios. We separate those findings from security findings so each lands with the right owner.