RecommendationPamela Fox recommends Anthropic for RAG because citations are built into the models: provide sources in structured format, enable citations, and the model returns blocks of text with citations that can be extracted programmatically to link citations in frontends.
AI Engineer · The Model Swap Workshop: GPT-5.5 vs Kimi vs DeepSeek vs Mistral — Pamela Fox, Microsoft
RecommendationPamela Fox advises that when evaluating tool calling, check whether the model is calling the right tool and whether it is passing in the arguments you expect. If a model calls a function with the wrong arguments, that means there's ambiguity in the function description — go back into the description and clarify it (e.g., 'don't include me or you, just named attendees', 'first and last if provided'). After clarifying, evaluate all the different inputs and see how the model is doing.
AI Engineer · The Model Swap Workshop: GPT-5.5 vs Kimi vs DeepSeek vs Mistral — Pamela Fox, Microsoft
RecommendationPamela Fox recommends applying prompt engineering when a model fails a task like counting, e.g., 'break the sentence into words, count per word, and show your reasoning', to improve the output. The counting test prompt is: 'How many times does the letter E appear in the following sentence? The elderly gentleman eagerly entered the elevator. Count carefully.' Testing the same prompt across multiple models immediately shows capability differences (e.g., counting E's: GPT-5.5 got 13, Kimi got 13, DeepSeek got 6, Mistral got 8).
AI Engineer · The Model Swap Workshop: GPT-5.5 vs Kimi vs DeepSeek vs Mistral — Pamela Fox, Microsoft
RecommendationPamela Fox advises that to build a proper agent, use an agent framework after establishing tool calling and tool calling in loops. She provides three agent frameworks in the demo — LangChain, Pydantic AI, or the (Microsoft) agent framework — so you can pick your favorite.
AI Engineer · The Model Swap Workshop: GPT-5.5 vs Kimi vs DeepSeek vs Mistral — Pamela Fox, Microsoft
Recommendation · MoneyGet an NDA signed early in the deal cycle to get on legal's radar and start vendor review earlier. Spin up a trial portal or demo instance early to kick off a lightweight security review and start the IT review sooner than required.
30 Minutes to President's Club | No-Nonsense Sales · #613: Master Parallel Selling Strategies to Speed Up Every Deal | Sam Mckenna | 30MPC Hall of Fame
Recommendation · MoneyWhen designing a system to detect adverse media for counterparty due diligence or acquisition news, backtest it against snapshots from before the event — e.g., one day, one week, and one month before the acquisition — to check whether the search would have revealed the signal at the time.
AI Engineer · Backtest Your Agent Against Yesterday's Web — Ryan Ahern, Exa
Recommendation · MoneyPrathik Naidu recommends integrating with systems of record — CRM, ticketing systems, document repositories with SOPs, contracts, and email threads — as a really valuable source of information for AI deployments.
AI Engineer · Why Buying Companies Beats Selling Them AI — Prathik Naidu, Long Lake
Recommendation · MoneyThe second offensive option is to charge the agents — figure out how to charge AI agents and make it easy for them to get access to your tool. There are now two parallel tracks for SEO: how humans find things and how agents find things.
MicroConf · SaaS Moats in the AI Era: Sell the Outcome, Not the Tool
Recommendation · MoneyJan Curn advises that if there is non-trivial cost, like compute, you should charge before you actually do the service, to avoid losing money if payment fails to arrive.
endorsed
AI Engineer · Why AI Agents Still Won't Pay for Anything — Jan Curn, Apify
Recommendation · MoneyYou can run these verification checks regularly, as part of CI, or as various hooks. A git hook is probably the last option the speaker would reach for. A Claude Code hook can be used to run verification checks. How you run verification checks depends on what your team is using and how they're doing it. Hooks are relevant for local development for the whole team because you can share hooks across the board.
endorsed
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Recommendation · MoneyTake screenshots and record videos and put them on the pull request as proof. But make sure the screenshot actually shows the state you expect and that it actually ran the commands to get to that state, running start to finish.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Recommendation · MoneyAvoid vague verification prompts like 'always verify the change in a browser' because they leave too much to interpretation. Instead write specific instructions like 'Remember to reload and check', 'Check that the data persists', and 'Look at the console for errors'. If you forget instructions or things change, prompts alone leave too much to interpretation.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Recommendation · MoneyDynamically remove tools from an agent based on what the brain is currently doing — e.g., disable all web search and external egress tool calls while an email-writing task is in progress, then re-enable them when the task completes. Intent systems ensure the right capabilities are given to the agent at the right time, preventing context bloat and reducing token spend because the agent spends less time on off-pathway actions. Nudge agents toward the goal state and ensure they cannot do the wrong thing — that combination is how you get the outcome you want in this era.
None stated.
AI Engineer · An AI Agent Got Its User Banned From Resy — Keycard, JPMorgan Chase & AAuth
Recommendation · MoneyLaurie Voss notes you can go into the web searches in a trace to see exactly where the agent got its data and how it combined it.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Recommendation · MoneyLaurie Voss recommends testing your rubric with examples and without examples and seeing which one does better. In his test, with examples the judge agreed with him 15 out of 16 times in three different runs; without examples, it agreed 16 times twice and 15 times once. The specific problem with examples was that one report didn't include a list of risks — Laurie marked it actionable as a human, but the examples said 'you must include risks,' so the judge marked it not actionable.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Recommendation · MoneyLaurie Voss demonstrates that in Phoenix you can filter your traces by any eval, sort by score to find your worst runs, and click straight from a failing score into the step that caused it — from score to cause in one click.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Recommendation · MoneyLaurie Voss explains that the trace records which tools the agent called, but not which ones it could have called, so you have to write that list yourself. The tool selection eval runs with 'npm run eval tools' and takes about 15 seconds.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Recommendation · MoneyLaurie Voss recommends suppressing tracing on the judge model so the judge stays out of the agent's traces, and giving every evaluator in the repo its own telemetry setting. He shares an anecdote of forgetting this, which filled his traces with the judge running itself.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Recommendation · MoneyLaurie Voss sets his standard of actionability for financial reports: the report should say whether to buy, sell, or hold, rather than leaving it ambiguous.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Recommendation · MoneyClaude Code has a feature that allows you to gauge how much you want to be involved in the process, and financial harnesses can guide users in the same way. With money, some things can be left for the agent to figure out — for example, if there are a few possible values for something, ask it to do three scenarios. When the agent generates multiple scenarios, review them side by side before making any decision. You wouldn't hand over entirely the keys to the kingdom to something you don't necessarily trust is going to have your best interest in mind.
Also discussed by Figma (unclassified source — not counted until classified). Not counted as independent sources.
AI Engineer · Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo