Every risk leader understands the logic of the Swiss Cheese Model, whether they have ever used the name or not.
Developed by psychologist James Reason and recently revisited by Alex Johnson in a thoughtful Fintech Takes essay on AI risk, the framework rests on a deceptively simple idea: every control has weaknesses. Failure occurs only when those weaknesses align across multiple layers. Resilience comes from building enough independent safeguards that no single gap can cascade into catastrophe. It is the principle that underpins the three lines of defense and one of the reasons modern risk management has remained effective despite the inevitability of human error.
GenAI deserves the same treatment, because a GenAI pipeline is itself a stack of layers, and risk can emerge in any one of them.
Every layer of your pipeline is a slice with holes
A production GenAI application is not one system. It is a chain: the data it retrieves from, the prompts that shape it, the foundation model underneath, the orchestration and tool calls around it, the guardrails on top, and the third-party components you do not control. Each layer can fail independently.
Bad retrieval produces confident answers grounded in the wrong document. A prompt change made for tone quietly weakens a refusal behavior. A vendor model update shifts outputs three weeks after your signoff. An agent's tool call does something no individual component was ever tested for. Jailbreaks, data leakage, bias, hallucination, each enters through a different hole, in a different slice.
Which means a single control, applied at a single point, can never be the answer. Testing only the model ignores the pipeline. Testing only the final output ignores where failures originate. The holes will eventually line up.
That divide plays out as two different teams asking two different questions about the same system. Business teams ask whether the AI does what it is supposed to do, and their holes look like a trust gap (no hands-on way to validate behavior before it reaches customers), reputation risk (hallucinations and logic errors that turn into public liabilities the moment they surface), and unclear readiness (no objective proof the system is actually ready for production).
Risk teams ask a different question: whether the AI is reliably blocked from doing what it should not. Their holes look like novel risks (bias, data leakage, jailbreak attempts that do not map to any control the organization has tested before), a missing evidence trail (testing too subjective to defend to legal, audit, or a regulator who asks how you know), and no thresholds (no measurable, agreed-upon definition of what "safe enough" actually means). Neither list fails loudly on its own. The breach shows up when a hole in one list lines up with a hole in the other, layer by layer, across the same pipeline.
A fraud case summarizer: watch the two holes line up
Take a GenAI agent that summarizes flagged transactions for fraud analysts and recommends escalate or close. Map it onto the diagram above and the holes stop being abstract.
At the orchestration layer, the business-team hole is a trust gap: analysts have no practical way to spot-check the agent's reasoning before acting on its recommendation, so they either rubber-stamp it or ignore it, and either habit defeats the point of having it. The risk-team hole at that same layer is emergent agent behavior showing up as hallucination: the model fabricates a transaction detail that never happened, and the case reads as more damning than it is.
At the outputs layer, the business-team hole is unclear readiness: there is no baseline showing the summarizer performs at least as well as a human analyst on a representative set of past cases, so no one can actually say it is ready to sit in the workflow. The risk-team hole is a missing evidence trail: there is no defined false-negative threshold for the agent's escalate-or-close calls, and no logged record of testing against it, so no one can state the model's actual miss rate.
Line up the trust gap with the hallucination, and here is what happens: an analyst accepts a fabricated summary because there was no way to verify it, closes a case that should have been escalated, and the loss does not surface until an audit finds it months later, filed as a control failure that "should have been visible." Neither team failed on its own. The holes just lined up.
The GenGuardX framework: identify, measure, and mitigate at every layer
Building GenGuardX meant rethinking AI risk as an ongoing management discipline, not a one-time assessment. That is why we structured it around four continuous activities: Identify, Measure, Mitigate, Monitor.
- 01 · Identify, map applicable risks. Select use-case-specific risk categories such as accuracy, bias, toxicity, privacy leakage, groundedness, prompt injection, and agent tool use, alongside business-side gaps like trust, adoption readiness, and reputational exposure, layer by layer, so no slice on either stack goes unexamined.
- 02 · Measure, run standardized evaluations. Use repeatable tests against curated datasets, expected outputs, policies, and thresholds, not one-off scripts. If a control cannot be re-run, it cannot be defended.
- 03 · Mitigate, track fixes and retest. Apply guardrails, prompt changes, routing logic, or workflow controls, then prove the gap was closed, with an evidence trail that survives audit.
- 04 · Monitor, watch for drift and regressions. Detect new failure modes, threshold breaches, and model behavior changes after deployment.
Because with GenAI, the slices do not hold still: inputs drift, vendors update models, agents shift behavior. A pipeline that was safe at approval is not guaranteed to be safe in month three.
Run as a cycle, not a checklist: what monitoring finds feeds the next round of identification and testing.
Seeing it on the platform
GenGuardX is built as one interconnected platform, not a set of separate tools handed to separate teams. Business and risk sit inside the same workspace, working off the same evaluation runs, so the identify-measure-mitigate-monitor cycle becomes a shared feedback loop instead of two teams checking each other's homework after the fact.
In practice, that means the same platform lets people iterate, test, experiment, and validate together. Someone can adjust a prompt or a routing rule, immediately see how it performs against the business's own quality bar, and see in the same place whether that change opened up a new risk. Nobody is waiting on a handoff or a separate report to find out.
That same shared foundation is what lets risk go deeper without leaving the business context behind. Instead of a one-off audit bolted on at the end, the platform continuously checks whether the system's answers agree with themselves across independent methods, and whether they hold up when the same question is asked a dozen different ways. Every experiment a business owner runs is automatically stress-tested the same way.
Zooming into individual use cases turns that ongoing check into a shared evidence trail: everyone, business and risk alike, can see exactly which parts of the pipeline are holding steady and which still need work, logged and re-runnable rather than living in a slide deck or someone's personal notes.
The question worth asking your team
Not "did our GenAI system pass testing?" but "can we name the risks at every layer of this pipeline, and show the control, the measurement, and the evidence for each one?"
If any layer draws a blank, that is the hole. And holes, eventually, line up.