Agentic AI penetration testing

Your AI agents can act.
We test what happens when someone else is steering.

Agents read untrusted content, call tools, hold credentials, and make decisions. River Gum Security performs in-depth, hands-on penetration tests of agentic applications, showing you how an attacker turns an agent's autonomy into access. Our preferred approach is full white-box testing: we work with your source code, prompts, and tool definitions to find what black-box testing alone would miss.

  • OWASP LLM Top 10
  • OWASP Agentic AI
  • MITRE ATLAS
  • NIST AI RMF
attack-chain.trace — illustrative
  1. INPUT Support ticket contains hidden instructions in a quoted email thread
  2. AGENT Summarization agent ingests the ticket as trusted context
  3. TOOL search_crm(query="*") invoked outside user intent
  4. TOOL send_email(to=attacker) accepted with no approval gate
  5. IMPACT Customer records exfiltrated through an approved business workflow
One injected sentence, four trust decisions, zero exploits of the underlying infrastructure. This is the class of failure we hunt for.

A model that can be talked into things, wired to systems that can do things, is a new kind of attack surface. Scanners and generic app tests were not built for it.

  • Non-deterministic behavior that defeats one-shot test cases
  • Natural-language input that doubles as an attack channel
  • Delegated authority that outlives the user's original intent

What we test

The full agentic attack surface, not just the prompt box

We assess the model, the orchestration around it, and every system the agent is allowed to touch, end to end, the way a real adversary would.

01

Prompt injection, direct & indirect

Instructions smuggled in through documents, web pages, email, tickets, images, and tool output, tested against every content source your agent reads.

02

Tool abuse & excessive agency

Whether an agent can be driven to misuse the tools it has: unauthorized actions, parameter tampering, missing approval gates, and over-broad permissions.

03

MCP servers, plugins & skills

Tool servers and extensions as supply chain: poisoned tool descriptions, unsafe server implementations, rug-pulls, and confused-deputy flaws.

04

Identity, auth & privilege

How agents authenticate and what they inherit: credential exposure, token scope, cross-tenant access, and escalation through delegated authority.

05

Memory, RAG & context poisoning

Persistent state as a foothold: poisoned vector stores, long-term memory manipulation, and retrieval paths that cross data-classification boundaries.

06

Multi-agent trust boundaries

Agent-to-agent messaging, orchestrator hijack, and trust that spreads further than it should when one component in the chain is compromised.

07

Data exfiltration channels

Covert egress through rendered markdown, link previews, tool calls, and logging, including paths no one thought to monitor.

08

Code execution & sandboxing

Code-interpreter and shell-capable agents: sandbox isolation, filesystem and network reach, secrets in the environment, and escape paths.

09

Guardrails & the surrounding app

Bypass testing of filters and policy layers, plus conventional web, API, and cloud weaknesses in the application that hosts the agent.

Our preferred approach

Full white-box testing, with your source code

An agent's behavior is the product of its prompts, tool code, permissions, and data paths. Seeing all of them lets us test what is actually there, instead of guessing at it through the chat window.

WHY

Find root causes, not just symptoms

With the code in hand we trace a misbehaving agent to the exact tool handler, missing authorization check, or prompt design that allows it.

WHY

Deeper coverage in the same time

System prompts, tool schemas, and permission models are visible from day one, so effort goes into attacking real weaknesses rather than reverse-engineering the design.

WHY

Fixes your developers can apply

Findings point to specific files, functions, and configuration, with a recommended change instead of a general warning.

What we review alongside live testing

  • Agent orchestration logic, system prompts, and guardrail code
  • Tool and MCP server implementations, including parameter handling
  • Authentication, authorization, and credential and token handling
  • RAG, vector store, and memory pipelines
  • Sandbox, container, and infrastructure-as-code configuration
  • Dependencies and third-party plugins or skills

Share the code. We strongly recommend it.

The best results come when you share your source code, and we strongly recommend it. Gray-box and black-box engagements are available when code truly can't be shared, but coverage is narrower and real vulnerabilities are more likely to go undiscovered. We'll tell you plainly what a given level of access can and cannot show.

Services

One focus, with the support around it

Core engagement

Agentic AI Application Penetration Test

A scoped, hands-on adversarial assessment of your agent or multi-agent system in a realistic environment. Testing is manual and creative, with tooling used to extend reach rather than replace judgment, and every finding is reproduced and demonstrated with real impact, not theoretical risk.

  • Architecture and trust-boundary review before testing begins
  • Full white-box with source code, strongly recommended for the best results
  • Safe-testing rules of engagement for production-adjacent agents
  • Prioritized findings with exact reproduction steps
  • Remediation guidance your engineers can act on
  • Retest of fixes included

AI Threat Modeling & Risk Assessment

Structured analysis of AI-enabled designs before or alongside build: threat scenarios, control gaps, and a ranked view of where to invest first.

Security Training & Workshops

Custom technical workshops for developers and security teams on attacking and defending AI-enabled systems, built around your stack.

Virtual CISO Advisory

Experienced program review and guidance that keeps AI adoption and the wider security roadmap aligned with business reality.

How an engagement runs

Clear scope, real attacks, usable results

  1. 1

    Scope & model

    We map the agent's inputs, tools, data, identities, and trust boundaries, and agree on objectives and rules of engagement.

  2. 2

    Attack

    Manual adversarial testing across the attack surface above, chaining small weaknesses into end-to-end compromise paths.

  3. 3

    Report

    An executive summary for leadership and a technical report with evidence, reproduction steps, and prioritized fixes.

  4. 4

    Debrief & retest

    A walkthrough with your engineers, then verification that remediations hold against the original attacks.

About

Senior, hands-on, and accountable

River Gum Security LLC is led by Joff Thyer. Every engagement is performed by the person you speak with during scoping, not handed to a junior team after the contract is signed.

Joff brings years of consulting and penetration testing experience, along with professional instruction experience that includes the SANS Institute and active faculty membership with IANS Research. Earlier in Joff's career, building and securing enterprise and academic networks at scale shaped how Joff evaluates the environments agents are deployed into.

14+
years in cyber security
1
accountable senior tester on every engagement
US
based, working with teams everywhere

Questions

Before we talk

How is this different from a traditional penetration test?

A conventional test targets deterministic code and infrastructure. Agents add a reasoning layer that can be manipulated through language, and they act through tools with real permissions. We test that layer and its consequences, and we also cover the conventional web, API, and cloud weaknesses around it.

Do you test agents running in production?

We prefer a staging environment that mirrors production. When production testing is necessary, we agree on strict rules of engagement up front: scoped accounts, rate limits, non-destructive payloads, and defined stop conditions.

Which platforms and models do you support?

Testing is model- and framework-agnostic. Whether you build on a hosted model API, self-hosted models, an agent framework, or MCP-based tooling, the work centers on how your system behaves under attack.

Why do you prefer white-box testing?

Agent behavior depends on prompts, tool code, and permissions that a black-box tester can only infer. With source access we review those directly, confirm findings against the code, and give your developers precise fixes. The result is deeper coverage and fewer false assumptions for the same engagement time.

Do I have to share our source code?

Sharing is not mandatory, but we strongly recommend it: the best results come from white-box testing with source access. Where code can't leave your environment, we can review it on-site or in a repository or workspace you control. A gray-box or black-box engagement is the fallback, and you should expect it to find less.

How is my data and source code handled?

Client data, source code, and findings are treated as confidential under a signed agreement. We access only what the scope requires, can work inside your own repositories and environments, and can use synthetic or sanitized data where you prefer it.

What do I need to prepare?

For white-box: read access to the relevant repositories, a short architecture overview, system prompts and tool definitions, and a test environment with accounts. We help with the rest during scoping.

Put your agents under adversarial pressure before someone else does.

Tell us what you're building. We'll respond with a scoping conversation and a proposal tailored to your architecture.

consulting@rivergumsecurity.com Please don't send sensitive details in the first message.