Technical interviews have begun measuring verification instead of output. That is real progress, but it stops one layer short of what matters.
I had the pleasure of corresponding via email with Andy Nian, cofounder and head of growth at PracHub, a site that catalogs technical interview questions reported by candidates. He wrote to me in August asking whether I would cite his AI-assisted question set in a piece I had written about why interviews should measure judgment rather than output.
I came back asking for his data instead, and he graciously answered a few questions on the record: figures, clear caveats about what those figures could not support, and three public examples. Almost no founder does that. One of his answers has stayed with me since, and most of what follows is an argument with it, in the good sense.
What Changed
The economics are two lines. Before capable AI, the path ran from skill to implementation to output, so output was a lossy but honest proxy for skill. With capable AI, the path runs from judgment to AI execution to output, and the artifact stops carrying information about whoever commissioned it. Two candidates of very different competence now produce work that looks similar at a glance.
The measurement industry noticed first, which makes sense given lead times. CodeSignal shipped AI-assisted assessments in May 2025 with session replay and full transcripts of the candidate’s exchange with the assistant. HackerRank’s July 2026 release is structured as Plan, Build, Review, with the plan as an artifact before any code and a diff view that lets the interviewer compare the finished work against its starting state, including everything the model contributed. CoderPad’s 2026 survey of more than 650 developers and hiring leaders concluded that hiring should assess whether someone can review, edit, adapt, and correct AI-generated code. A concentrated set of employers followed: Meta replaced one of two onsite coding rounds in October 2025, Canva redesigned its questions to be deliberately more ambiguous, and Coinbase rebuilt in phases through early 2026.
Nian’s catalog suggests how early this still is. As of September 1, his AI-assisted tag covered 104 of 10,239 public question records, a little over one percent. The earliest tagged examples date to April 2025, but meaningful volume does not begin until January 2026. The records are concentrated rather than broad: Meta accounts for 33 of the 104, Shopify for 15 and Rippling for 13. The tag also appears to span two different things, rounds where a candidate works with an assistant and questions about designing AI systems, which is worth knowing before anyone reads the count as a measure of one phenomenon. Nian was careful to say these are catalog records and not a claim about the share of interviews in the market.
What These Interviews Actually Measure
Nian’s read, on the record: the format is moving from bounded AI-assisted coding toward repository debugging, feature implementation, code review, system design and defending the workflow. The recurring signal, he says, is verification. Candidates have to define the task, inspect unfamiliar code, test what gets generated, catch plausible mistakes, explain trade-offs, and remain accountable for the result.
I think he is right, and it matches what I see inside my own company. The question is no longer how to instruct an agent to produce what you need. It is now how to confirm it produced the right thing, and how to read that code six months later well enough to understand what it does and why when something breaks. That is a genuine shift, and it is the correct response to a contaminated proxy.
It is also the cheapest form of judgment on offer.
Verification Is Necessary and Insufficient
Look at what every one of these formats hands the candidate. The ticket is already written. The repository is already chosen. The scope is already set. The candidate directs the agent, checks its work, and explains the outcome.
That is quality control. It is harder than it looks; most engineers are worse at it than they think, and testing it is a real improvement over testing whether someone can produce a binary search from memory. But it is not the decision that determines whether the work was worth doing. Nobody in these interviews is asked whether the ticket should have been written that way, whether the constraint everyone accepted is the one that actually binds, which trade-off is tolerable in this system and which is not, or whether the agent’s framing of the problem should have been accepted at all.
Those decisions are where engineering judgment lives, and the market has not found a way to price them.
That is not an oversight. It is a constraint. Measurement requires observability, and verification is observable. You can watch someone catch a plausible mistake inside forty-five minutes. You cannot watch someone decide, over three weeks, not to build something. You cannot watch the argument that reframed a requirement before any code existed. The market has moved upstream of output and stopped at the last point where judgment still leaves a visible trace inside an interview slot.
So the honest description of where we are is this. Output stopped proving anything. Verification became the proxy. Verification is a better proxy than output, which is why the shift is worth taking seriously. It is still a proxy, and it captures the part of judgment that happens after the problem has already been defined.
And Proxies Do Not Stay Clean
There is a second reason not to settle here, visible in the preparation market.
An interview that grades verification should be hard to rehearse. The preparation industry’s product is rehearsal, so it has the strongest incentive of anyone to turn an unrehearsable test into a rehearsable one, and what candidates say they want suggests the demand is there. Testimonials advertise recognizing most of a loop in advance, memorizing a ranked question list for a target company, walking in with the effective equivalent of a cheat sheet.
Those are marketing testimonials describing a range of interview experiences, most of them traditional rounds. They are not measured evidence about how candidates fare in AI-assisted interviews, and I am not presenting them as such. What they show is what candidates believe they are buying, which is the relevant thing: the appetite for recognition persists regardless of what the newer formats are built to reward.
When I put that tension to Nian, he did not deflect. He agreed the gap exists and described the more durable practice as clarify, inspect, test, explain, revise, while acknowledging that what candidates want is a repeatable pattern. He deserves credit for saying so, and the problem is not his to solve. It is what happens to any signal once it becomes the thing being scored. What is unusual is the compression: the format is barely eighteen months old, and the countermeasure is already a product category.
Where This Leaves Us
AI made output abundant, output stopped working as evidence, and the interview that selects the people producing that output has had to move upstream to find something that still carries information. It landed on verification, the right first move, but still a long way from the destination.
The organizations that get this right will not be the ones with the best verification rubric. They will be the ones that find a way to observe the decision before the ticket: what a candidate chooses not to build, which requirement they refuse to accept as given, where they say the model’s framing of the problem is wrong.
I have not seen anyone test that yet. I would like to.
Figures and quotations from Andy Nian, cofounder and head of growth at PracHub, provided on the record in September 2026. The full AI-assisted collection is public at prachub.com/ai-assisted.