The signal that no longer measures anything
Picture the classic software interview, the way it has run millions of times. A whiteboard or a shared editor, a problem with a tidy algorithmic twist, a clock ticking. The candidate is asked to type, under observation, a function that reverses an array, inverts a tree, finds the shortest path. What gets scored is how quickly and how cleanly she gets from the empty file to a working solution. For decades this was a reasonable signal: someone who could do it fluently could usually produce code on the job too.
That very signal now measures the wrong thing. When a model writes most of the code – and in many teams it already does – "typing clean code fast from your head" is no longer the scarce skill. It has become the cheapest ingredient in the whole process. What has stayed scarce is precisely what the classic interview never tests: judgement. Reading, decomposing, verifying – and above all knowing what to build in the first place.
So we go on hiring against a signal that measures a skill the tool on the other side of the table wields better than any applicant. This is not a small calibration problem. It is a broken instrument.
Why the classic interview breaks
Let's start with the uncomfortable truth: LeetCode and HackerRank are trivially solvable by AI. Not "with effort", not "on the easy problems" – trivially. The entire genre of algorithmic puzzle questions, the hard tier included, is exactly the sort of well-defined craft, present a million times over in the training data, at which models excel. A current model solves the median interview problem correctly, commented, with optimal runtime, in seconds. A test whose questions the cheapest tool in the room completes flawlessly no longer measures the human's talent. At best it measures whether the human has memorised the tool's ability.
That collapses the tacit assumption the whole format rested on: that coding speed is the bottleneck. It was, for a long time. Whoever typed clean code faster shipped faster, and the interview was a reasonable proxy for that. But the bottleneck has moved. Production is no longer the constriction – it is plentiful, fast, cheap. What slows a team down is no longer how quickly someone writes the first version, but whether someone spots that the version does the wrong thing, contains a security hole, or solves a problem nobody had.
It gets worse: the classic interview actively rewards the wrong instinct. It prizes the reflex to start typing at once – "show me you can produce code". In the AI era that reflex is an anti-pattern. The valuable first move is not the typing but the pause: interrogating the task, prodding the edges, deciding whether the problem is even framed correctly. A process that rewards people for skipping that step selects for the wrong turn of mind.
And the whiteboard ritual has always tested a skill that barely occurs in the real job: flawless production from nothing, no looking things up, no iteration, without the tools you actually use every day. That artificiality was once a forgivable compromise. Today it is absurd: we forbid the candidate the very tool she will genuinely use to do the work, then grade her in a discipline that tool has already taken over.
What to hire for instead
When typing gets cheap, a person's value shifts onto everything that surrounds the typing. Not the ability to produce code, but the ability to conduct, scrutinise and stand behind the code that gets produced. Concretely that comes down to a handful of qualities the classic interview never seriously tested:
- Taste. The ability to see, between two working solutions, which is the better one – the simpler, the more robust, the one the team will still understand in two years. Taste is not a luxury; it is the filter that separates a stream of generated code from rubbish.
- Systems thinking. Models build locally sensible and globally inconsistent. They smuggle in a second, competing pattern without noticing. What you are hiring for is the person who sees how a change fits the whole – or doesn't.
- Reviewing generated code without mercy. The core skill. Reading plausible, clean-looking, confidently wrong code and finding the subtle gap: the edge case never handled, the invented API call, the silent assumption that doesn't hold.
- Debugging someone else's broken code. Not your own fresh code, but a system somebody else – or something else – built and that falls over at three in the morning. That is the real work, and it demands the exact opposite of the whiteboard reflex.
- Product judgement. Knowing what to build. Code can be immaculate and still solve the wrong problem. The person who realises a feature isn't needed at all saves more than any fast typist ever produced.
- Specification and communication. A model is only as good as the intent you hand it. Someone who can translate a fuzzy requirement into a precise, testable specification has already done the real engineering – the rest is generation.
You can see the pattern: every one of these is a judgement skill, not a production skill. None can be delegated to the model, because they are precisely the point at which "the code runs" becomes "we understand what it does and we stand behind it".
How you actually test for it
The good news: once you know what you are hiring for, the assessment formats are obvious – they simply resemble the real work. Three formats replace the whiteboard cleanly.
Have them review a flawed PR live. You give the candidate a realistic pull request of two or three hundred lines, into which you have deliberately built a handful of faults: an off-by-one at the boundary, an authorisation check that misses exactly one path, an assumption that the response is sorted. Then you watch how she reads. Does she find the subtle things or only the obvious typo? Does she prioritise by blast radius – payment logic first, variable name last? Can she explain why something is wrong, not merely that it feels off? This pair-review interview measures more about actual fitness in forty minutes than any algorithmic question.
Have them debug a broken system. You sit her in front of a small, running application with a genuine bug – one that won't yield to staring, only to hypotheses, logs, decomposition. How does she proceed when she didn't write the code and doesn't understand it? Does she form a theory and test it, or guess about at random? Debugging is the most honest test of judgement there is, because you can't memorise your way through it.
A take-home WITH AI allowed. This is the decisive inversion, the "AI-allowed interview". Instead of banning the tool the candidate will use anyway, you hand it to her explicitly – and then grade not the code but the judgement. What prompts did she write? Where did she overrule the model? What did she throw away from the first generated version, and why? You have her talk through her decisions. Two candidates hand in the same working code; one took it on blind, the other caught three silent bugs and made an architectural call. The classic interview couldn't tell them apart. This format sees nothing else.
The common denominator: you stop testing empty-file production and start testing the interaction with code that already exists – reading, judging, improving, owning it. That is the job.
The junior problem
Here it gets uncomfortable, and honest analysis mustn't flinch from it. Everything said so far comes down to a quality we call judgement – and judgement is acquired, not innate. The obvious question is: how does a junior acquire it if they never write the first draft?
The traditional path by which engineers built judgement was brutal but effective: you wrote a thousand lines of bad code yourself, watched them get torn apart in review, understood why they were bad, and out of that built an intuition over years. The writing-and-failing was the curriculum. When a model takes over the first draft, that very path disappears. The junior now sees only finished, plausible-looking code – and plausible-looking code teaches you nothing, because it hides the mistakes you'd have learned from.
This is the looming collapse of the junior pipeline. If teams only hire for judgement, but judgement only grows through years of writing-and-failing, then nobody hires the people in whom that judgement could grow in the first place. You consume a stock of seniors that an earlier, slower system produced, and at the same time you shut down the factory that builds the supply. It doesn't bite this quarter, but it bites.
The way out is not nostalgia but a deliberately built apprenticeship. A few contours are emerging:
- Enforced deep reading. If juniors no longer learn by writing, they must learn by structured, guided reading. Not waving generated code through, but taking it apart: why this structure? What happens on empty input? Where would this break? Reading-as-training, deliberate and accompanied.
- Taking apprenticeship seriously again. The senior looking over a junior's shoulder while they dismantle a generated PR is no longer a luxury – it is the only remaining channel for transmitting judgement. Teams that do this systematically will have seniors in five years. The others will have to buy them in, expensively.
- Manufacturing friction on purpose. Sometimes apprenticeship means having the junior write the first draft themselves – no model – not because it's efficient but because the failure is the lesson. Efficiency and training pull in different directions here, and you have to hold the tension knowingly.
Ignore the junior problem and you optimise yourself into a dead end: maximum short-term productivity, a dried-up talent pipeline.
The other side: fundamentals matter more, not less
Now the honest counter-argument, or the thesis tips into its opposite and becomes just as wrong. The temptation to conclude from all this that "being able to program doesn't matter, good prompts are what count" is real – and it is a trap.
The reason is simple and hard: you cannot review code you do not understand. Every single judgement skill listed above presupposes deep technical foundations. To spot the invented API call, you have to know how the real API behaves. To see the silent race condition, you need concurrency in your gut. To sense that a data structure is the wrong one, you must have seen a thousand data structures in use. The "merciless reviewer" is not an alternative to the deep engineer – she is the deep engineer, only her leverage is no longer the typing but the judgement.
That is why the pure prompt jockey – someone who works models deftly but can't really read the code they emit – is not an acceptable substitute but the most dangerous profile of all. He produces plausible-looking code fast and has no way of telling when that code is confidently wrong. He turns the organisation into a machine that piles up comprehension debt faster than anyone can pay it down.
So you mustn't overcorrect into the opposite and hire people who have never built anything. Someone who has never carried a system from the empty file into production has never collected the scars that judgement is made of. The right reading is not "fundamentals have become unimportant" but a more uncomfortable one: the bar for depth rises, it doesn't fall. A mediocre engineer with high output used to be useful – he produced a lot, after all. That niche disappears, because the model takes over the output. What remains is the demand to understand deeply enough to overrule the model. That is a higher bar, not a lower one.
Conclusion
For decades the classic interview measured a real skill: producing clean code fast. That skill has not become worthless, but it is no longer scarce – the cheapest tool in the room wields it better than any applicant. Keep hiring against it and you are measuring with a broken instrument and wondering at the results.
What has become scarce is what the whiteboard could never test: taste, systems thinking, the ability to review generated code without mercy, to debug other people's broken systems, to spot the right problem and frame a precise specification. You assess it by having people do real work – review a flawed PR, debug a system, think through a take-home with AI allowed – and grading not the code but the judgement behind it. And you rebuild the apprenticeship, or you dry up the very pipeline that produces judgement in the first place.
That is exactly what we hire for at NH Labs. Not for typing speed and not for the number of puzzles solved, but for judgement and the willingness to carry responsibility – for every line submitted, no matter who or what typed it. We look for the people who read generated code the way an editor reads a manuscript: fast, sceptical, with a feel for where the smooth surface hides a fault. That is why our clients can trust the result – not because an AI was fast, but because a human with good judgement stands behind it.