2026-07-18 · AI red teaming
AI security yellow teams: why SaaS startups need both red and blue thinking
When we run AI red teaming engagements for European SaaS companies, we hear the same question almost every time: can't we just run an automated scan? The answer is no, and the reason has gotten more specific as frontier models like Anthropic's Claude Mythos and OpenAI's GPT-5.5 enter security testing.
A third discipline is showing up alongside the usual red team and blue team split, and SaaS engineering leaders should understand it before they buy an AI security tool. Security press has started calling it the yellow team: the builder-engineers who create the frameworks, harnesses, and execution environments that make AI-powered security testing actually work.
Yellow teams don't run the tests and they don't defend against them. They build the infrastructure that decides whether your AI red teaming finds real vulnerabilities or just generates expensive noise.
What yellow teams actually do in AI security
The term isn't new. Security researcher April C. Wright introduced it at Black Hat USA 2017 in a talk called "Orange Is the New Purple," arguing that developers, architects, and DevOps engineers, the people who build the software, deserved a seat at the table alongside red teams (attackers) and blue teams (defenders). She called them the Yellow Team: the builders.
In AI security, that concept has picked up a much narrower job description: engineers who wrap frontier AI models in operational constraints so the models are actually useful for security work.
A model like Mythos or GPT-5.5 can, in principle, find vulnerabilities in your authentication flow, your tenant isolation logic, or your RAG implementation. Without constraints, it will also flag every intentionally unauthenticated internal endpoint, every expected behavior it doesn't understand, and every architectural decision it misreads as a flaw. The gap between "the model found something" and "the model found something that matters" is exactly what a harness closes: defining what the model can access, what counts as a valid test credential, which endpoints are in scope, and how to tell intentional architecture apart from an exploitable bug.
The yellow team builds that harness. Nobody else does.
Why SaaS companies can't skip the harness
Frontier models aren't plug-and-play security tools. They're reasoning engines that need real operational scaffolding before they're safe to point at a production application, and that scaffolding is specific to your codebase, not generic.
A harness for AI security testing typically covers four things. Scope and boundaries, so the model doesn't waste time (or cause an incident) testing admin panels and debug endpoints the same way it tests customer-facing APIs. Architecture context, so the model knows how your tenant isolation, authentication, and data access patterns actually work instead of flagging every deliberate design choice as a bug. Credential and permission management, so test accounts carry the right privilege levels and the model can't accidentally escalate past its test boundary. And output triage, so a human reviewer isn't stuck sorting through every raw finding before knowing which ones are real.
Building that infrastructure takes someone who understands both your SaaS architecture and what the model can and can't reason about reliably. That's yellow team work, and it's why AI security testing isn't something you switch on.
The dual-purpose mindset
What separates yellow teams from typical DevOps or security engineering is that they build for both sides at once: attack infrastructure for the red team, and detection or defensive capability for the blue team, from the same underlying work.
In our AI red teaming engagements, that shows up directly. When we build a harness to test for prompt injection in a SaaS company's AI features, we're documenting the exact attack patterns we use as we go. Those patterns become the starting point for the detection rules and input validation logic the client's own team deploys afterward.
That dual-purpose output matters for compliance. SOC 2 Type II and EU AI Act readiness both want two things: evidence that testing happened, and documented controls that came out of it. A harness configuration, a test execution framework, and validated finding criteria satisfy both asks from a single body of work.
It also sharpens a build-versus-buy decision a lot of SaaS teams are facing right now. When you're evaluating an AI security tool, ask whether it lets you define your own harness and customize the execution framework for your architecture. If it doesn't, you're buying generic scanning, and generic scanning won't find the risks specific to how your product is actually built.
What this means for your security program
If you're a CTO or engineering lead at a European B2B SaaS company, a few things follow directly from this.
AI red teaming isn't a product you install. It takes custom engineering work to make a frontier model effective against your specific application, and that work needs its own budget line, not a footnote in a security tools budget. Someone on your team, or your testing partner's team, needs to own the harness on an ongoing basis: maintaining execution constraints and updating the model's understanding of your architecture as it changes. That documentation work has value independent of AI testing. Detailed records of your tenant isolation model and authentication flows are foundational for any security program, AI-assisted or not.
Build in reproducibility from the start. A finding an AI model surfaces needs to be reproducible by a human tester, which means logging, recording, and replay capability belong in the harness, not as an afterthought. And treat the harness as a living system. It isn't a one-time build. As your application changes, the testing framework built around it has to change too.
At Faultline Security, we bring that harness-building discipline to every AI red teaming engagement we run for European SaaS companies. We build the constraints specific to your application, document the testing framework for your team to keep using, and make sure every finding is reproducible before it reaches you. If you're evaluating AI security testing for a SOC 2 audit, an EU AI Act obligation, or investor due diligence, get in touch.