Specor

AI-Generated Test Cases for Every Merge: Verifying Intent, Not Just Structure

Daniel Ciolfi ·

A merge request in Specor can resolve every conflict, pass every approval, and still ship a regression nobody catches — because conflict resolution only proves the spec is internally consistent. It says nothing about whether the actual implementation still does what the merged spec now says it should do. That gap between “the model merges cleanly” and “the product still works” is exactly where regressions hide.

It’s the same gap Git for product teams doesn’t close by itself. Branching, diffing, and merging give product definitions the version control they were missing — but version control was never the whole story for code either. The other half is CI: something that actually exercises the change before you trust it. Specs needed their own version of that.

What a clean merge doesn’t tell you

Say a branch removes mandatory signup from checkout and adds a rule that Pix payment links expire after 15 minutes. The merge resolves without conflicts — nothing else in the target branch touched those nodes. Structurally, it’s a clean merge.

But “no conflicts” only means no two branches edited the same field. It doesn’t mean the implementation of guest checkout was actually built to match the new rule, and it doesn’t mean nothing two or three hops away in the graph — the broader checkout flow, the payment-link validation, a related persona permission — quietly stopped lining up with what the spec now claims. Those are the regressions that conflict detection structurally cannot see, because they’re not conflicts. They’re drift between the model and reality, surfacing right at the moment the model changes.

Reading the diff, not guessing at risk

So instead of asking a human to remember everything that might be affected, Specor generates the test plan from the merge itself. When you trigger generation on an open merge request, it computes the real structural diff between the source and target branches — the same diff engine behind live diffs — and walks the graph outward from every node that actually changed, up to two hops, to find the “blast radius”: features, rules, and flows close enough to the change that they’re a plausible regression risk.

For the changed nodes themselves and for each blast-radius neighbor, an AI agent writes focused Gherkin-style test cases — Given/When/Then, in plain language, plus a one-sentence “why” explaining what change or dependency the case is covering. A direct change might read:

Given a guest user with items in their cart When they reach checkout without signing in Then they can pay and receive an order confirmation

A blast-radius case two hops away gets a lighter touch and is labeled as a regression check rather than a feature test, so reviewers know at a glance which kind of risk they’re looking at:

Regression check: this Rule depends on the changed Checkout flow. Given a guest checkout generates a Pix payment link When 15 minutes pass without payment Then the link is rejected as expired

The AI writes the test. A human verifies the build.

This is a deliberate split, and it’s worth being explicit about it. The agent’s job stops at writing a test case that’s specific and checkable — it never marks anything passed or failed itself. A developer or QA engineer reads each case, checks it against the real, running implementation, and marks it pass or fail by hand. The model can tell you what to verify with far more precision than a human staring at a diff; it has no way to know whether your code actually does it.

That also makes generation one of the few places in Specor that spends AI credits deliberately. The six Spec Quality agents and Product Intelligence checks run automatically in the background and never touch your balance — but test case generation is a user-clicked action with a real cost, so it checks your organization’s token balance up front and deducts what it actually used, success or partial failure alike. You’re not paying for a guess; you’re paying for a test plan grounded in your real diff.

Test cases as a merge gate

Once test cases exist, they’re more than documentation. Branch protection can require every test case on a merge to be marked passed before the merge is allowed to confirm — turning “we think this is fine” into an explicit, visible gate, the same way a required CI check gates a code merge. Skip generation entirely and the gate (if enabled) blocks until someone runs it; generate and leave one case failing, and the merge stays open until it’s resolved or regenerated against a fixed implementation.

Verifiable from wherever you’re already working

Because the whole point is connecting the spec’s intent to the real implementation, the loop closes through the same channel your engineers already use to read that intent: MCP. An AI coding tool connected to Specor can pull a merge’s test cases alongside its diff, and — for users with editor or admin access — mark them pass or fail directly, without a context switch back to the Specor UI. Viewers can still read everything; marking stays restricted to people with write access, in the MCP tools exactly as it is in the app.

A merge that resolves cleanly was never proof the product still works. Now there’s a test plan, generated from the actual change, sitting right next to the merge request that needs it.