PrinciplePamela Fox notes that a lot of the time you realize your evaluator itself is incorrect, so with LLM-as-judge you have to figure out whether the problem is your app or your judge. You also want to calibrate your judges with human input, and it helps to work with a domain expert who can label your data and check whether they agree with the LLM judge on what was a pass or a fail.
Commercial context: The creator describes this as their own product.
AI Engineer · The Model Swap Workshop: GPT-5.5 vs Kimi vs DeepSeek vs Mistral — Pamela Fox, Microsoft
Creator perspective · MoneyNick Nisi connects the verification approach back to basic unit-testing practice: bring deterministic checks to the models in a way that they can self-heal and self-improve. This includes moving manual developer verification habits — like flexing the app in Chrome while debugging late at night — into code and a deterministic CLI so agents get the same experience, and giving agents visibility into all aspects of performance such as networking, traffic, and rendering.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Creator perspective · MoneyNick Nisi argues that the real value of local verification is the rapid development loop: give the agent a task, let it go, and verify locally without waiting on CI — CI is a feature of it, but not the actual tool. The faster feedback loop allows running faster and parallelizing oneself across multiple agents; he reports running three agents on a low day (at a conference) and normally 10 to 15. He also notes that when delegating, you tell the agent where you want to end up — e.g., add an email to the waitlist, persisted and saved — and how to verify it, and that telling the agent exactly what you want gets there without generating a bunch of additional useless code.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Principle · MoneyEvery eval has holes but the holes are not in the same place; stack as many evals as you can on top of each other so it's hard for an error to get through all of them. Code evals catch formatting problems but miss meaning; LLM judges catch reasoning gaps but miss subtle hallucinations; humans catch the subtle stuff but can't look at every trace. Cramming every failure into one giant LLM-as-judge eval is an unreliable, complicated way of not solving the problem — build several small evals instead.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Principle · MoneyWhen presenting analysis or doing financial analysis on an investment, you need to show where your numbers came from and attribute your decision to the body of evidence you had. 'Trust me, I ran the numbers' without a trace should not be trusted.
AI Engineer · Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo
Principle · MoneyIn data and finance you need a separate point of reference, a counterfactual you can check against, without redoing your plan entirely. Have something review and give you a rough answer or challenge if something doesn't look right.
AI Engineer · Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo
Principle · MoneyStart from the outcome you want and how you'll measure success, rather than adopting AI because it can do something. The specific tool matters less than the result.
AI Engineer · Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo
Principle · MoneyIf an organization isn't measuring token spend, start with measurement before anything else, because the biggest ROI comes simply from measuring. Once spend is visible, you can right-size which teams and workloads spend on what, and the data informs workload decisions.
AI Engineer · The AI Bill Is Too Damn High: Tokenomics 101 — Eugene Cheah, Featherless AI
Creator perspective · MoneyThe controls that could have stopped all three incidents discussed are probably already inside the organization, just waiting to be configured. Many controls are already owned today but haven't been set up for the AI world. The audience is urged to take protective steps right now, as soon as this week, rather than keeping an eye out down the line.
AI Engineer · Your AI Isn’t the Problem. Your Infrastructure Is. — Jiachen Jiang, Coder
Principle · MindThe host urges listeners not to let their 'no' and their 'I love you' stand in opposition; both are necessary for a healthy relationship. He recommends practicing the phrases 'I love you, no' or 'No, and I love you' out loud to internalize this, and shares that not wanting to cut off love in order to draw a boundary kept him in relationships that didn't feel good for an incredibly long time. He also recounts firing someone after many violations and how starting the call with 'I love you. This is so hard.' made it flow much easier.
Charisma on Command · I’m 39. Here’s What I Wish I Knew at 30
Principle · SocialThe creator's highest-rated technique: whatever topic comes up, find a playful way to make it about you and the woman together, because men typically fail by never doing this.
Coach Kyle · What Women Actually Want to Talk About (TIER LIST)
Principle · MoneyThe way to stand out in a world that wants to blend in is to be really fucking good and be fully you. The meta of building a personal brand is just being you — wear what you want to wear and change it when you want to change it. To have X factor, you have to be able to put other things in the bucket — add something spice and everything nice beyond your stock traits.
Alex Hormozi · If I Started A Personal Brand In 2026, I'd Do This - Dolly Parton
Principle · MoneyNnenna Ndukwe recommends defining what code quality means to you and your team before building agentic workflows, then codifying those standards so AI agents can extract and adhere to them, and making them distributable across CLI, IDE, and CI workflows. The same standards should be checked against across planning, code generation, and code review phases.
Strongly endorsed by the host; no stated uncertainty.
AI Engineer · How to Build Quality Gates into Agentic Coding Workflows — Nnenna Ndukwe, Qodo AI
Principle · MoneySecurity measures should be put in from the beginning of app creation, and designing with threats in mind during the prompt phase makes it easier to build an app that is harder to hack. When creating something with AI, also instruct the tool to design a testing process and a security review, not just the app.
AI Engineer · AI Security Engineer Foundations + Certificate — Javier Garza, Snyk
Principle · MoneyWhen engaging business stakeholders like a CFO, the very first question they ask is ROI. A common fear when deploying agents is that the only measurable thing about an agent is whose job it takes away from, and that fear survives because we mostly haven't measured the alternatives. The reframe: ask what did the agent take off your plate, and what did that freed time allow your business to do? A good intern lightens your workload: routine work moves off your desk and you get sharper at the work that's left. Apply the same test to an agent. When an agent handles routine tasks (like balance checks at 7 and 9), the banker spends the next morning with the customer who's rethinking their mortgage. The agent made the banker better at banking — that is the column that belongs on your scorecard. So
AI Engineer · https://www.youtube.com/watch?v=PwCSYy-oGsM
PrincipleFuad Ali emphasizes that if you're not collecting tracing, nothing else works — getting tracing started is step one.
AI Engineer · Building Self-Learning Loops for Your Agent — Fuad Ali, Arize AI
Principle · MoneyMeasure your adoption of the tools, because getting adoption numbers up is what gets you the last part of the way; every capability you can get from better tools is gated by the tool adoption number. An elegant tool that gets reached for a third of the time is just a third of a tool.
endorsed
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Principle · MoneyIt's quite important to run evals multiple times because it's not about per-shot accuracy, it's about your determinism.
endorsed
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Principle · MoneyGo into your traces (e.g., LogPire or wherever you send your traces) and actually read them. If you just look at your scoreboard, you might get answers that look directionally correct but are burying underlying issues that are much larger problems inside your architecture.
endorsed
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Principle · MoneyOutput guardrails cost money per request, even when the output is redacted, so consider stopping bad prompts at the input instead. Accepting a prompt, processing it, and catching bad output on the back end costs money you probably didn't have to spend. If a hacker discovers you only sanitize output on the back end, they will likely DDoS you to run up your costs. The speaker states that, legally, he would never actually DDoS anyone — the DDoS scenario is hypothetical. With the input guardrail on, the jailbreak prompt was blocked before reaching the LLM, and it was fast.
AI Engineer · Build a Platform and Watch It Burn — Michael Forrester, Accenture & Whitney Lee, Datadog