Google Mantis: What AI-Assisted Bug Finding Needs to Work

TL;DR: AI security tools need a verification loop

Google Cloud’s Mantis project is a useful example of where AI-assisted application security is heading. Instead of asking a model to scan a repository and return a long list of possible problems, Mantis is described as a harness that can coordinate vulnerability discovery, triage, reproduction, deduplication, and patch verification. The important idea is not that an agent can replace a security engineer. It is that an agent becomes more useful when it has access to repository history, a threat model, isolated execution, repeatable tools, and an explicit test for whether a proposed fix actually works.

For developers, this changes the buying question. The best security assistant is not the one that produces the most warnings. It is the one that helps a team move from a suspicious line of code to a reproducible finding, a reviewable patch, and evidence that the regression is gone.

What Mantis is trying to solve

The Google Cloud overview of Mantis presents an open-source harness for finding and fixing bugs with AI agents. The Mantis repository includes workflows and skills for security-oriented analysis. In practical terms, the harness gives an agent a structured environment instead of a blank chat box.

That environment matters because security findings are rarely just text-generation problems. A useful investigation may need to understand how a feature changed over time, identify the trust boundary around an input, construct a small proof of concept, run tests in a sandbox, compare a candidate patch with the original behavior, and remove duplicate reports. Each step has different evidence requirements. A model that is good at explaining code may still be poor at proving exploitability or checking that a patch did not break an unrelated path.

Why repository history and threat models matter

A static scan sees the code that exists today. An agent with repository history can also see why a permission check was added, which input was considered unsafe, or whether a previous fix was incomplete. That context can reduce false positives and help prioritize changes that affect a real boundary.

A threat model adds another layer. It asks who controls an input, what the attacker can reach, which identity or data can be affected, and what a successful exploit would change. Without those questions, an agent may label every unusual pattern as critical or miss a subtle issue because the vulnerable code looks ordinary in isolation.

This is why AI security tools should expose their reasoning as evidence, not as a confident score. A report should show the relevant file and path, the assumptions used, the reproduction steps, the expected impact, and the limits of the analysis. Teams can then decide whether the finding deserves a patch, a compensating control, or no action.

Reproduction is the dividing line

The most valuable part of an agentic security workflow is often the attempt to reproduce a problem safely. A sandbox can limit network access, credentials, filesystem scope, and destructive commands while allowing the agent to test a narrow hypothesis. This makes the result more concrete than a pattern match.

Reproduction also improves communication. A developer can review a small test case and understand what failed. A security reviewer can see whether the behavior crosses the claimed trust boundary. A release owner can ask for the same test to pass after the patch. The process creates an artifact that survives beyond the original conversation.

That does not mean every vulnerability can be proven automatically. Some issues depend on production configuration, timing, identity policy, or a chain of several weak controls. In those cases, the right output is a bounded hypothesis with explicit uncertainty, not an invented exploit.

What to check before choosing an AI security tool

Before downloading or adopting an agentic code-security product, check five practical boundaries:

1. Permissions: Can the agent read secrets, write to production branches, or call external services?
2. Isolation: Does reproduction run in a disposable environment with a clear network and filesystem policy?
3. Evidence: Are findings linked to code, tests, logs, and reproducible steps?
4. Recovery: Can a human pause, roll back, or reject a patch before it is merged?
5. Data handling: Are repository contents, prompts, and generated patches retained or used for training?

A software directory is useful here because the workflow matters more than the word “AI” on a feature page. If you are comparing coding assistants, security scanners, test tools, or repository utilities, browse the BTTC software directory and compare permissions, integrations, export options, and review controls. You can also find more practical technology explainers in the BTTC blog.

FAQ: using agents for application security

Can an AI agent replace a security engineer?

No. It can accelerate repetitive investigation, generate test cases, and organize evidence, but humans still need to define risk, validate assumptions, approve changes, and understand production context.

Is a reproduced bug automatically a critical vulnerability?

No. Reproduction proves that a behavior can occur under the tested conditions. Severity still depends on exposure, attacker control, affected assets, required privileges, and business impact.

Should an agent be allowed to merge its own patch?

Usually not by default. A safer boundary is to let the agent create a proposed change and a regression test, then require human review and an independent check before merging.

Conclusion

Mantis points toward a more credible model for AI-assisted security: agents should not merely describe suspicious code; they should gather context, test a bounded hypothesis, produce reviewable evidence, and verify a fix inside controlled limits. That workflow will make security software more useful, but it will also make permissions, sandboxing, audit trails, and human approval part of the product itself.

For teams evaluating the next generation of developer tools, the durable question is simple: does the product help you prove and safely resolve a problem, or does it only produce another list to review?