The Green Bar That Proves Nothing
There is a moment every team knows and nobody questions: the tests pass, the bar turns green, and everyone breathes out. Green means done. Green means safe. Green means somebody thought about it and the system agreed. That small splash of colour carries an astonishing amount of trust — and that trust rests on a quiet assumption that held for decades: that the person who wrote the test and the person who wrote the code arrived at the same result independently. Two paths, one destination. When they meet in the middle, that meeting is proof.
That assumption breaks the moment a machine writes both. The model produces the function, and in the same breath it produces the tests that check that function. Both come from the same understanding of the task, the same reading of the requirement, the same blind spots. The test no longer checks whether the code is correct. It checks whether the code does what the model thought it should do — and that is an entirely different, far weaker claim.
The Circular Validation Trap
Picture the simplest case. A model is asked to write a function that calculates a discount. It reads the requirement, mildly misunderstands it — say it applies the discount before tax when it should come after — and writes clean, plausible code containing exactly that error. Then we ask the same model to write the tests. It writes them against its own understanding. The test expects the discount before tax. The code delivers the discount before tax. Green.
The test worked — in the sense that it ran and passed. But it verified nothing. It didn't find the bug, because it came from the same source as the bug. It didn't expose the misunderstanding; it froze it, sealed it, and knighted it with a green tick. That is the circular validation trap: when the same party supplies both the claim and the proof of the claim, the proof is worthless. You cannot stand as your own witness to your own correctness.
The heart of it fits in a single sentence: green is not the same as correct. Green only means that two artefacts agree. When both artefacts come from the same source with the same misreading, their agreement is not news. It is a tautology. You can effortlessly reach 100% coverage of the wrong behaviour this way — every line exercised, every branch tested, every assertion satisfied, and the software still calculates the discount wrong. The metric gleams while the product lies.
The insidious part is that from the outside it looks exactly like functioning quality assurance. The same test suite, the same green pipeline, the same percentages in the report. Except the check has lost its independence — the one property that makes a test a test at all. A test that can never catch anything its author didn't already believe is not a safety net. It's a mirror.
Tests as Executable Specification
If the model can no longer be its own examiner, the independence has to come from somewhere else. It has to come from a human. And the point at which it enters the process is not the implementation — the machine is welcome to that — but the specification: the precise, binding statement of what "correct" means in this particular case.
The shift in perspective is fundamental. A test stops being an after-the-fact proof that the code works and becomes the up-front definition of what working even means. The discount comes after tax. A negative amount is invalid and must be rejected. A basket with no items yields no discount — not null, not an error, but exactly zero pounds. These are not implementation details. They are business decisions — and business decisions belong to the human, because only the human answers for the consequences.
In practice this does not necessarily mean a human types every line of every test by hand. It means a human owns the behavioural contract. There is a spectrum, and every point on it is legitimate as long as authorship stays clear:
- The human writes the core assertions themselves — the handful of lines that pin down what must happen — and lets the machine build the surrounding test machinery: fixtures, mocks, setup.
- The human states the contract in prose, precise and complete, and lets the model translate it into tests — then reviews that translation line by line, as strictly as if they had written it themselves.
- The human defines the examples that matter: the concrete input-output pairs the behaviour hangs on. The model may flesh them out, but the pairs themselves come from human judgement about the domain.
The common thread is always the same: the definition of correct must not come from the same source as the implementation. The moment it does, the check is circular and the green bar is hollow. The test suite then becomes what it always should have been and rarely was in the human era: the executable, independent specification of the system. The code is an answer to it, not its co-author.
Properties, Not Examples: Property-Based and Invariant Tests
There is a class of tests that is structurally more resistant to the circular trap — because it is harder to "game", whether deliberately or by accident. Example-based tests say: "For input X, I expect output Y." A model writing both the code and the test picks X and Y from the same understanding and meets itself. Property-based tests say something fundamentally different: "For any possible input, this property must hold."
That is the idea behind tools in the style of Hypothesis or QuickCheck: you don't describe a single example, you describe an invariant — a truth about the system that must always hold, whatever data comes in — and the framework hurls hundreds of randomly generated inputs at it, deliberately including the ugly ones: the empty set, the enormous number, the Unicode chaos, the negative boundary. When a case breaks the invariant, it shrinks it down to the smallest counter-example and drops it at your feet.
Why is this harder to game? Because an invariant is stated at a higher level than the implementation. "The amount returned is never negative." "Encrypting and then decrypting yields the original." "The sorted list has the same elements as the unsorted one." "Running the same operation twice changes nothing compared to running it once" — idempotence. Sentences like these catch whole classes of bugs nobody thought about explicitly, because they don't query one example, they assert a truth. The model cannot simply tailor its code to the test cases, because it doesn't know the test cases — they are rolled at runtime.
The contrast with the other end of the spectrum could not be sharper. Snapshot tests simply freeze the current actual behaviour: on the first run the test records what the code outputs, and from then on it fires whenever the output changes. That is useful against accidental regressions — but it verifies zero correctness. A snapshot of a bug is a frozen bug, solemnly sealed and defended against any correction. When a model produces both the code and the snapshot, you have the circular test in its purest form: "the code does what the code does." Snapshots are a tool, not a proof, and mistaking them for verification is one of the most expensive errors in an AI-heavy codebase.
From Coverage Percentage to Real Requirement
The most important shift, in the end, is not a technical one but a change in the question being scored. For decades the guiding metric was: what percentage of the code does the test suite cover? That number was never especially meaningful, but in the age of AI it becomes actively misleading, because a model will effortlessly produce tests that touch every line without a single one of them checking a real requirement. Coverage measures what was executed, not what was asserted. 100% of a lie is still a lie.
The question that matters is no longer "how much does the test cover?" but "does this test encode a real requirement?". Is there a statement behind this assertion about the business, about expected behaviour, about something that would hurt if it flipped — or does it merely touch a line so the statistics look right? A single test that pins down a real invariant is worth more than a hundred that manufacture coverage. And when the metric changes, so does where humans point their scarce attention.
That attention belongs first at the edges the model skips. Generated code describes the happy path eloquently and omits the corners: the empty list, the negative amount, the time zone at the day boundary, the concurrent access, the timeout mid-transaction. These are precisely the cases no example in the training material emphasised — and precisely the cases in which systems die in production. The human who knows the domain knows where the bodies are buried. The model does not.
One effective move is to have the model generate adversarial tests against its own code: "Here is the implementation — write the nastiest test cases you can think of to break it." This is surprisingly productive, because a model in the destructive role often sees edges it skipped in the constructive one. But — and this is the warning that carries straight over from the discipline of code review — it shares the blind spots. The same model with the same training bias, cast as attacker, overlooks the same class of bugs it built in as author. It finds the edges it knows and walks straight past the ones it has never seen. Adversarial AI tests are an extra layer, never the last one. The last one is a human with a threat model no model possesses.
And there is a tool for checking the tests themselves — for measuring whether they can catch anything at all. Mutation testing deliberately alters the production code: turns a > into a >=, deletes a line, negates a condition — and watches whether any test goes red as a result. If everything stays green despite the smuggled-in mutation, the suite verifies nothing at that spot: the test runs through the code without nailing down its behaviour. Mutation testing is the most honest answer to the question "do our tests actually test?" — and in a world where machines churn out tests by the yard, it is the natural check on their quality.
What AI Tests Are Good for Anyway
Now the honest other side, or the argument tips into dogma. None of this means AI-generated tests are worthless or ought to be banned. That would be expensive nonsense.
As a first layer they are genuinely useful. They catch regressions — when a later change breaks behaviour that used to be right, the generated test fires, no matter who wrote it. They handle the dull boilerplate: the setup, the fixtures, the twenty tedious cases around the one interesting one. They reach a baseline coverage in minutes that would take a human hours. And — this is the real prize — they free the human up for the meaningful invariants. Someone no longer typing fixtures has their head clear for the one assertion that actually counts. The division of labour is not "human or machine" but "machine for volume, human for meaning".
The mistake is not using AI tests. The mistake is confusing them with verification — treating the first layer as the last and stopping the moment it goes green. You don't ban them; you simply refuse to hand them a responsibility they cannot carry.
And one more caveat, so the lesson doesn't get stretched too far: not everything needs heavy testing. The care should scale with the blast radius — with what happens, worst case, if this code is wrong. A throwaway script that reformats a CSV once needs no invariant suite. An internal dashboard for five people is not the payment flow. Full rigour — hand-owned specifications, property-based tests, mutation testing — belongs where a mistake genuinely hurts: money, customer data, authentication, anything irreversible. Testing everything at maximum is its own form of waste. The art is steering scarce human attention to where correctness is not negotiable.
Conclusion
For decades the test was the tick that came after the real work of writing the code. That order is reversing. When the machine writes the code, the test is no longer the proof that it worked — it is the up-front, human-owned definition of what working even means. The test suite is the specification, and the specification does not belong to the machine that satisfies it.
This is precisely where we start at NH Labs. We let AI write what writes well: the implementation, the boilerplate, the twenty tedious cases, the baseline coverage in minutes. But we own the behavioural contract — the assertions that pin down what "correct" means in this system, with this data, for this customer. A model can produce green all it likes; what green is allowed to mean, a human defines. That is why our clients can trust the green bar: not because a machine agreed with itself, but because a human decided beforehand what agreement has to prove.