Principle · MoneyTo verify a web app as a human developer: refresh the page and check the application is still visible; check the network tab to see if it's healthy; check the response codes coming back from the API match what you expected based on what you sent it; check local storage for persistence, check console logs, and read what's actually on the page. Verifying the app was part of the core responsibility of human developers when hand-chiseling code.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Principle · MoneyDon't give the agent just a vague paragraph like 'hey, do this, always verify the change in a browser, whatever that means' — that's too vague. Give progressively more specific verification instructions: 'remember to reload and check', 'check that the data persists', 'look at the console for errors'.
AI Engineer · Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Creator perspective · MoneyMedical records get commingled between patients (e.g., Bob Joe and Bob Joe Jr.), which violates HIPAA. Organizations without duplicate-detection keep receiving the same bad-quality duplicate data, and claims adjusters processing around 200 claims at a time cannot keep every follow-up in mind, so things get lost without technological support.
claim-count figure flagged as possible numeric error
AI Engineer · Insurance Claims Are AI's Messiest Data Problem — Denys Linkov, Wisedocs
Principle · MoneyMartone argues production AI requires far higher reliability than academia's ~90% standard: at EliseAI's volume (~5 million calls/month, ~60 million/year), 90% reliability would mean millions of failed calls — an unbearable failure rate. Every additional 'nine' of reliability is another hill to climb.
endorsed claim; some numeric details flagged as possible errors
The a16z Show · Why Real-World AI Is Harder Than the Benchmark | EliseAI’s Mario Martone
Creator perspective · MoneyDick explains that when people started looking at how to authorize agents, the tool sitting on the shelf was OAuth, and people grabbed it assuming it should solve the problem. But OAuth was not designed for the agentic world; it was designed originally for letting one website talk to another website. OAuth 2 added scopes — before that it was just 'do anything' — but it required preregistration and expected a browser involved in a redirect.
AI Engineer · An AI Agent Got Its User Banned From Resy — Keycard, JPMorgan Chase & AAuth
Creator perspective · MoneyThe creator describes that making a payment is not a scope; it's 'can I do this one thing in this particular context'. Just because a payment is right in context by itself doesn't mean it's the right thing within the broader mission — e.g., it may be a double payment made five seconds ago. You need granularity at the transaction layer to know exactly what is happening, not just 'I'm allowed to go and make payments'.
AI Engineer · An AI Agent Got Its User Banned From Resy — Keycard, JPMorgan Chase & AAuth
Creator perspective · MoneyIan frames the problem of how an agent shows up to a website, interacts with it, and establishes that it can be trusted. Relying on CAPTCHAs as an agent-defense mechanism fails because agents can now solve them. Agents face the shared-IP collateral-damage problem far worse than Vercel's: how do you let unregistered traffic you've never met before show up at your front door and decide whether to let it in? The proposed direction is to create agreeable rails so downstream resources can decide which agents to allow and whom to trust based on the users those agents represent.
None stated.
AI Engineer · An AI Agent Got Its User Banned From Resy — Keycard, JPMorgan Chase & AAuth
Principle · MoneyWhen improving an agent, Voss says the important thing to check is whether anything that used to pass has broken. You can do this by filtering down to the passes split in the eval results.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Creator perspective · MoneyTool use evals check three things: did the agent pick the right tool, pass sensible arguments, and do something sensible with the result. Tool selection takes what was asked, which tools were available, and which were picked.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Principle · MoneyThe same test suite demonstrated locally can be run in GitHub Actions, where the workflow starts Phoenix as a service container.
AI Engineer · How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Principle · MoneyApproving every calculation — the other side of the spectrum — doesn't feel great. Reviewing every PR line by line that your agents ship is also an extreme to avoid. Both lead to a lot of toil and unnecessary work.
AI Engineer · Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo
Principle · MoneyFor code, you don't necessarily need to ask before every decision because the tests will speak for themselves. Reviews come in handy in coding because you only need to check that everything is passing, and you will have a lot of confidence because of it.
AI Engineer · Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo
Principle · MoneyOne of the easiest ways to check whether a node in your decision-making process needs to be implemented as code: if it's a calculation, likely it has to be implemented as code. Calculations implemented as code should also have testing — unit testing, golden examples, etc.
AI Engineer · Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo
Principle · MoneyUnstructured thoughts in a chat need to be labeled as an interpretation. Agent reasoning or pulling information from memory needs to be treated as an assumption.
AI Engineer · Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo
Principle · BodyGrubbs discourages changing an athlete's entire training plan based on a single down force plate reading. Instead, practitioners should check other data sources — travel context, wellness questionnaires — and only build an action plan when three or four flags stack up, asking 'how can I help this athlete out?'
Garage Strength · How The Best Pro Baseball Players Actually Train w/ MLB Strength Coach Stephanie Grubbs
PrincipleIf you can get away with linear regression or decision trees for your application, you should use those instead of neural networks; try the simplest method first, then try something more complicated and see if the performance lift justifies the additional complexity; going straight to neural networks and transformers is often not the right way to go.
Y Combinator · AI Safety Fact vs. Fiction | YC Paper Club
PrincipleMichael argues that beyond technical tools, proper safeguards at the human level are needed, citing the cybersecurity lesson that the typical way of breaking into a system is through human (social) engineering. A hypothetical attack scenario is described: a caller who has been at the company one month, fearing mistakes with an angry customer, is rushed by a fake crying baby into giving up access. A possible formal spec follows: people at the company less than six months who haven't passed a fake social hacking scenario or safety training cannot give passwords or resets to new email addresses.
Y Combinator · AI Safety Fact vs. Fiction | YC Paper Club
Creator perspectiveWe need to borrow more ideas from the aerospace industry for AI safety. Creating something like the NTSB for AI has actually been discussed and could be a good idea. Commercial aviation is 80 times safer now than in the 1970s. The key thing for an AI safety oversight body is independence. The NTSB was set up as an independent agency in 1967, and other aviation safety reporting systems like ASRS were put in place as non-punitive reporting systems. Non-punitive reporting systems incentivized manufacturers and airlines to share safety information and to take every single accident seriously, drawing as much scientific knowledge as possible from what went wrong. In 2007 there was also the Aviation Safety Information Analysis and Reporting System, which had some form of immunity, and we need to
Y Combinator · AI Safety Fact vs. Fiction | YC Paper Club
PrincipleThe core question in AI safety is: if we build a system, maybe an AI system, how do we know it's going to be safe? For a system with learned components, such as an auto landing system for an aircraft, you need evidence to justify deploying it. Commercial air traffic aims for about a 10^-9 probability of catastrophic failure per flight hour, and the ACAS-X aircraft collision avoidance system was held to that standard.
Y Combinator · AI Safety Fact vs. Fiction | YC Paper Club
Creator perspectiveIn the new world where it's easy to write code and hopefully soon easy to verify code, the thing that's left that's hard is the specification. Some specs are easy and universal (memory safety for C code, writable once and proved for everyone's code); others are domain-specific (for a bank: no loss of funds — the total across a sequence of transactions should stay the same).
Y Combinator · AI Safety Fact vs. Fiction | YC Paper Club