Coding agents are excellent at finding evidence that their work succeeded.

Tests passed.  Build exited 0.  Healthcheck returned 200.  No ERROR lines found.

Then they say done.

But a passing signal is not automatically a passing outcome. Prove It makes your coding agent actively search for evidence that its conclusion is wrong before it is allowed to trust it.

Use it to challenge bug fixes, tests, pull requests, CI results, logs, deployments, and any confident “done” claim before you trust it.

Don't try to prove it works. Try to prove it doesn't.

Who needs Prove It?

You need it if you let an AI coding agent:

This is for the moment between “the AI finished” and “you believe it.” You do not need to be a testing expert. Invoke one skill and give it the claim you want challenged.

Install in 30 seconds

Prove It is listed on skills.sh. The installer detects Claude Code, Codex, Cursor, and other supported agents:

npx skills add Pablo-aps/prove-it

Then ask it directly:

# Claude Code / Cursor
/prove-it review your last implementation and try to find a case where it still fails

# OpenAI Codex
$prove-it does this CI output actually prove the issue is resolved?

The skill can also activate automatically when you ask an agent to prove, verify, validate, confirm, or double-check a claim.

Four rules. One verdict.

CLAIM
All production replicas run abc123 and the migration is applied.

VERDICT
NOT PROVEN

WHY
The deploy job passed, but only four of six replicas are represented.

FALSIFICATION ATTEMPTS
- checked rollback events -> none found
- compared runtime image SHAs -> evidence exists for 4/6 replicas
- checked migration version -> no production schema evidence supplied

NEXT PROOF
1. read the image SHA from every replica
2. read the production migration version through an approved read-only path

The verdicts mean something

PROVEN is deliberately scoped. Prove It does not claim mathematical proof, universal correctness, or future safety.

It catches the expensive kind of “done”

tests green             ≠ original bug fixed
build green             ≠ deployment works
deploy succeeded        ≠ every replica runs the new version
healthcheck 200         ≠ workers and dependencies are healthy
HTTP 200                ≠ asynchronous operation completed
no errors in these logs ≠ no relevant error occurred
one successful request  ≠ race condition fixed
agent confidence        ≠ evidence

It also watches for counterfeit proof: skipped tests, weakened assertions, ignored exit codes, empty catches, hardcoded results, behavior mocked away, and timeouts increased without reproducing the timing failure.

These patterns are not automatically wrong. They become evidence against a claim when they hide, bypass, or redefine the behavior being verified.

See EXAMPLES.md for deployment, logs, async work, payment races, positive proof, and blocked verification.

Vibe verification

Vibe verification is when an agent sees enough positive signals to feel confident and stops searching for ways the conclusion could be false.

Vibe verification:  “Tests pass. Looks good.”
Prove It:            “What would let these tests pass while the bug still exists?”

That shift is small, memorable, and useful across stacks because it changes the verification objective—not the test framework.

Evidence, not prompt theater

The repository includes a reproducible 12-case benchmark with:

• misleading success signals that should end in FAILED;
• incomplete evidence that should end in NOT PROVEN;
• inaccessible evidence that should end in BLOCKED;
• positive controls that should end in PROVEN.

One directional run with Codex CLI 0.147.0 · gpt-5.6-luna · low reasoning · 2026-08-18:

Prove It changed three decisions in this run: two explicitly unavailable proof paths were correctly labeled BLOCKED, and one fully covered, scoped deployment claim was accepted as PROVEN instead of remaining indefinitely unproven.

Every case uses the same model, prompt artifact, output schema, and reasoning setting. The treatment changes one thing: it installs and explicitly invokes Prove It. Read the 24 raw outputs and full benchmark/ methodology.

This is one run per cell on cases authored while developing the skill. It is evidence that the skill changed this model's behavior on this fixture set—not an independent study, a model ranking, or a universal performance claim.

The design is also grounded in a broader finding: language models can favor confirming evidence, and counterexample-oriented prompting can reduce that bias. See Failing to Falsify, Large Language Models Cannot Self-Correct Reasoning Yet, and Anthropic's work on sycophancy in language models. These sources motivate the mechanism; they do not prove this skill works for every model or task.

Frequently asked questions

What is Prove It?

Prove It is a small open-source Agent Skill that makes an AI coding agent define a falsifiable claim, search for counterevidence, verify the real outcome, and return an evidence-backed verdict.

Which AI coding agents does it support?

The skill follows the open Agent Skills specification and works with Claude Code, OpenAI Codex, Cursor, and other compatible coding agents. The repository keeps the behavior in one portable SKILL.md.

How is it different from testing or code review?

Tests and review produce evidence. Prove It changes how the agent challenges and interprets that evidence. It does not replace either one.

Can Prove It prove that software is correct?

No. It returns a scoped verdict from the checks and evidence that are actually available. It never claims formal correctness, universal safety, or certainty about unobserved behavior.

What this is not

Prove It is intentionally one readable SKILL.md. It has no runtime dependencies, hooks, background process, telemetry, MCP server, or orchestration layer.

It is not:

• a testing framework or security scanner;
• a replacement for domain-specific tests, observability, or human review;
• permission to run destructive experiments against production;
• a claim that an AI can prove arbitrary software correct.

It is a behavioral guardrail: define the claim, attack it, verify the outcome, and earn the verdict.

Small on purpose

prove-it/
├── README.md          # understand it in two minutes
├── README.*.md        # five localized guides
├── EXAMPLES.md        # six concrete uses
├── skills/prove-it    # the entire behavior
├── benchmark/         # reproducible evidence
└── LICENSE

The skill follows the open Agent Skills specification. Contributions are welcome when they make the core behavior sharper without turning the project into a framework.