Creator perspectiveAmeisen argues that the two visions of language models — next-token predictor vs. planner — aren't as in conflict as you'd think. Models are trained by repeatedly asking 'what happens next?', and once good enough on hard tasks, the best way to know what happens next is to think about what happens way down the line. To predict a story character's next action well, a model must build a good understanding of the character's goals and what might happen in the future. A next-word predictor that plans and refers back to its plan can perform higher-order tasks — executing a plan slowly over many words.
Lenny's Podcast · Inside the black box of AI: How AI models actually think | Emmanuel Ameisen (Anthropic)
Creator perspectiveAI capabilities appear as big external jumps but are actually smooth internal progressions that cross usefulness thresholds. As a model smoothly improves at next-token prediction, it first masters easy words (sentence structure), then medium words (basic facts), then hard words (domain expertise). Each task has a different reliability threshold at which it becomes 'solved', producing apparent big jumps despite smooth underlying improvement. The phenomenon of abilities appearing suddenly at thresholds is called emergence.
Lenny's Podcast · Inside the black box of AI: How AI models actually think | Emmanuel Ameisen (Anthropic)
Creator perspective · MoneyThe speakers argue that writing good error messages matters more now because agents are the primary consumers of error output. Agents need detailed error messages in order to self-correct, making error message quality a first-class engineering concern.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Creator perspective · MoneyThe speakers argue that when you have verifiers in place, your source code doesn't matter as much — you can write it again (e.g., convert to a different language) and prove that it works. Without verifiers, you cannot prove a rewrite works and source code matters more than it should.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Creator perspective · MoneyNick Nisi discourages relying too much on manual code review of agent output: agents write code faster than humans can review it, creating a signal-to-noise problem. Reviewing agent-generated code like a normal code reviewer also leaves engineers further abstracted away from the codebase, since they are no longer writing code themselves and become less familiar with it.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Creator perspective · MoneyThe speakers emphasize that the end goal is proving back to the human, not just passing output to the LLM for interpretation. Screenshots make this easier by popping up visual proof, rather than relying solely on the LLM's interpretation of JSON.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Creator perspective · MoneyNick Nisi describes setting up agents so they know how to fix themselves and stay on track while you're off doing something else, so you don't come back to a PR at the end of the day and find the agent didn't self-correct — which would mean finishing the work yourself and slowing the team down. He identifies having to turn-by-turn nudge agents that ran in the background as a significant slowdown that prevents going faster.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Creator perspective · MoneyThe speakers endorse the view that the way you verify, test, and prove your software works might be even more important than the actual application code.
hedged (might be more important)
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Creator perspective · MoneyNick Nisi observes that teams preserving clean datasets of 'this is what we're trying to get to' will have a specific advantage when self-evolving tooling becomes the default. If you're not sitting on a mountain of clean target-state datasets and your competitors are — with similar apps — they will go significantly faster in their development cycle.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Creator perspective · MoneyZack Proser describes how running many parallel agent sessions with diffracted attention causes subtle bugs to creep in: the agent claims 'yeah, I did what you said' but the result is still wrong. This motivates verification tooling as the safety net for parallelized work.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Report · MoneyWhen the MLCR eval was released, the highest accuracy seen on the levels of reasoning was 20%; a few months later it reached 75% with very high reasoning levels.
AI Engineer · Insurance Claims Are AI's Messiest Data Problem — Denys Linkov, Wisedocs
Principle · MoneyEvery prompt change can affect every kind of input your users send, and you cannot test all of those by hand. Prompt regressions are often discovered only when a customer hits them.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Report · MoneyLaurie Voss argues that the path an agent takes is non-deterministic by design: you can't predict the output from the input, which is why you need evals. Two runs of the same agent might both be fine, or one might be great and one might be wrong.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Creator perspective · MoneyAgents make evals harder than single LLM calls because agents chain many steps together. A single LLM call is relatively simple — input in, output out. In agents, an early error in the chain cascades through all remaining steps, each getting progressively more off the rails. Being confidently wrong is worse than being obviously wrong. Example: a financial analysis agent searched for Tesla, got results about the 18th-century inventor, and wrote a beautiful, perfectly formatted report about the wrong subject — and none of the checks failed.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Creator perspective · MoneyThe common failure mode: an LLM app works in testing, ships, then breaks when a user uses it in an unexpected way. Changing a prompt to fix one thing creates a nagging feeling that something else broke with no way of knowing. Changing a system prompt to fix tone can cause the bot to start making up product features, discovered only when a customer hits it. Every prompt change can affect every kind of input users send, and you can't test all of those by hand. Without evals you end up playing whack-a-mole: fix one thing, something else breaks, fix that, something else breaks.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Report · MoneyLaurie Voss describes the dominant but flawed practice of shipping AI features by eyeballing a few outputs, and why unit tests and human review fail for LLM outputs.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Principle · MoneyAgents can be creative and do things in unexpected ways, so design outcome-level evals that don't care how the agent found its data, only that the right result is present — that is the right level for a lot of evals.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Principle · MoneyMultiply failure frequency by severity to get your priority — the most common failure is not necessarily the first thing you fix.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Report · MoneyLaurie Voss instructs running 'npm run batch' to run 15 queries in a row. The batch runs five at a time and takes three or four minutes, optimistically, on Wi-Fi. If your batch run fails for whatever reason, don't worry about it — use the Phoenix 101 Golden seed project instead, which has 16 runs done earlier on reliable Wi-Fi.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Principle · MoneyBreak tasks down in a way that you can reason about the issue and the way they connect.
AI Engineer · Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo