Source-derived protocol · AI Engineer
Encode Verification Criteria and Feed Them to Agents
Work out how you would automatically tell whether a thing is good, encode that judgment in specs, rubrics and evals, and feed the criteria to your agent so you are not figuring out the review process manually.
4 actions
AI Engineer's principle
Encode your best people's judgment into specs, rubrics and evals, and feed that automatic verification to your agent rather than reviewing manually.
Your version
4 total
Turn your review judgment into automatic verification criteria and hand those criteria to your agent instead of reviewing manually.
The source states the practice but not a schedule or a specific artifact; the user supplies both.
Decide what [thing will you build automatic verification criteria for] is and write down how you would build an automatic way of telling whether it is good or not.
- Spend a lot of time thinking about how you would build an automatic way of telling whether the thing is good or not.
Setup · onceDone when: A written description of automatic verification criteria for [thing will you build automatic verification criteria for] exists.
Put your best people's judgment into specs, rubrics, and evals for [thing will you build automatic verification criteria for].
Setup · onceDone when: Specs, rubrics, and evals encoding your best people's judgment are written down.
Feed the automatic verification criteria to your agent instead of trying to figure out the review process manually.
Recurring · on your cadenceDone when: The agent has received the verification criteria and is applying them instead of a manual review process.
Review whether the verification criteria still capture your best people's judgment and whether the agent is applying them instead of a manual review process.
Review · on your cadenceDone when: A decision is recorded on whether the criteria still capture your best people's judgment and whether the agent is applying them instead of a manual review process.NuthinButta scaffolding
Why it works
What AI Engineer says this builds. Their framing, not ours — every line links to where they said it.
Extracting requirements requires talking to the customer directly — the job is figuring out what they actually mean
Voss argues that a customer saying 'I need to keep track of my orders' is not a sufficient requirement — the real job is figuring out what that means for their specific business. The only way to extract a business owner's requirements is to talk to them directly. Requirements differ completely depending on whether you run a bakery, an auto parts store, or a giant manufacturing concern.
View in the full analysis →Code review is the bottleneck when AI generates code at scale
Voss argues that speeding up code generation without speeding up code review does not increase shipping speed — the constraint is the bottleneck. A 10,000-line diff takes about a week to review at ~450 LOC/hour, so teams should plan merge cadence around that reality.
View in the full analysis →The junior engineer problem: the ladder is gone and nobody teaches product engineering
Voss argues the profession needs to fix the junior engineer problem — the junior developer career ladder has disappeared before the industry has figured out how to train new juniors. Universities teach data structures and boot camps teach React, but nobody teaches 'go sit with a baker for a week and come back with a spec' — which is the person we need. There is a whole cohort of unemployed juniors, and the industry needs to figure out how to create product engineers on purpose instead of accidentally, and quickly, or everyone will be screwed.
View in the full analysis →
Source-derived protocol
Based on “AI Writes the Code. So What's Left for Engineers? - Laurie Voss, Arize AI” by AI Engineer.
- Source fidelity verifiedPassed · Oct 25, 2026
- Protocolability reviewedPassed · configurable template
- Editorially reviewedNot yet
- Supported by independent sourcesNot evaluated
Source fidelity means each creator-worded element was checked against the exact statements it cites. It says nothing about whether the approach works.