35,595 results · Most relevant
When a graduate student or team member fails an exam or makes a big mistake, intervene personally and convince the department to keep them rather than expel them, weighing their potential future contribution — they may turn out to be an outstanding researcher. Expelling someone for a single exam failure can cost you an outstanding researcher.
Prathik Naidu evaluates acquisition targets on three key questions: whether there is a customer base that trusts the business with real end-to-end outcomes, whether there is human judgment in the service being provided, and whether there are regulatory reasons that govern accountability when making decisions.
The easiest way by far to get verification across the board is to put an agents.md (not a claud.md) in your app. In agents.md, write instructions like: on change, verify things by running this; every time you change significant things, do this; or before you open a PR, do this. Agents have to go out of their way to ignore or avoid the instructions in agents.md.
endorsed
Only mark a workflow node satisfied after the verifier confirms all checks pass. Use a verification gate that requires all pull requests are merged before workflow completion. Reference the actual repository checked and the PR in verification status output. When an agent fixes code from PR reviews, verify it still matches the overall goal of working software.
Use the red-green refactor TDD loop with your agent within the verification piece itself: verify this, reproduce it, show me, fix it, and then show me the proof of that. Asking the agent to show you the proof is just as important as the fix that's going to go in, because that's what's going to leave you alone and allow you to trust the model to actually do the work.
Insurers can analyze which claims people are making, in which geographies, what types of injuries, and which doctors are involved to understand loss ratio; track how much patients travel within a network and whether they get local treatment from the right provider; map networks of which lawyers, doctors, chiropractors, and physiotherapists work together; analyze which conditions co-occur by mapping diagnoses to ICD codes; compare a doctor's write-up diagnoses against another's; analyze consistent factors in treatment costs across claims and how to underwrite them; and run counterfactual analysis on network changes, billing changes, and care direction to estimate potential savings.
Consequential creator advice requires independent verification before Today can promote it.
The baseline for structuring claims data is OCR and images to extract information from underlying records; preprocessing PDFs is hard and large files contain rare edge cases. After extraction, a series of models is layered on top — NLP and extraction models, sometimes VLLMs to structure data (standard entity extraction also works well) — then various classifiers, then a summary layer whose output is human-verified for correctness. The abstraction pattern concentrates information density up the stack, with verification by models and human intervention at various points.
When you separate hands and brain, you can dynamically change which hands (tools) are available based on what the brain is doing — e.g., when an agent is writing an email, disable all web search and external egress tool calls until the email task is complete, then re-enable them.
The creator recommends propagating intent information immutably through the system, which gives you properties that short-circuit forensic investigation of a rogue agent. Trying to figure out what a rogue agent was trying to do versus what the intention of the user was is very hard without propagated intent information. This also lets companies more quickly scope and identify whether an error was human error or agent software error, and handle that blast radius appropriately, and use intent to diagnose whether a failure was caused by a mal-aligned user or a mal-aligned agent, and respond accordingly.
Use intent-based auth in agentic commerce so users can ensure their agents buy the right item within their personal budget. As merchants, capture purchase intent data to understand why a purchase is being made and whether the agent should be trusted to visit your site. Davis suggests extending know-your-customer practices to know-your-agent: couple the agent's identity with the intent of the user, operator, and processor. Intent-based auth also helps simplify regulatory compliance operations in commerce, which enables more agents doing good work for users.
None stated.
To build the golden dataset with splits, find every report with a human label, and each one becomes an example in a Phoenix dataset — the input is the report, the expected output is the human's label. Sort the labeled examples by span ID, then every fourth example goes into the test split and the rest go into dev. Sorting by span ID first means a report always lands in the same split however many times you run the script. Span ID links each example back to the trace it came from, so you can always click through to the original run.
After counting categories, Laurie Voss recommends prioritizing by frequency × severity: the most common failure is where you look first but isn't necessarily the first thing you fix — a catastrophic rare failure (e.g., bomb instructions 1% of the time) gets fixed immediately.
Laurie Voss advises you shouldn't run your whole agent on every single commit because it takes a really long time and costs a whole bunch of money. Once you're live, keep evaluating: run your evals on a sample of real traffic and alert when scores drop and stay down — but don't alert on every single failure, because one failure would be noise; alert if you get a trend in failures. Production monitoring is incredibly useful, and evals can get you that as well.
To wire up the evaluator, attendees create a classification evaluator, which does all the plumbing, and give it a name, the model, and their template. The evaluator's options are given out of band: actionable with a score of 1 and not actionable with a score of 0. Running the rubric evaluation takes three or four minutes.
When chatting with an agent for financial purposes, you may reason about something after a decision has been made, and you are well in your right to correct information and add more context. When creating a harness, build not only gates for validating things and staying accountable, but also for correcting the course and making sure you can do so safely. Without course-correction support, typing one sentence in the chat can wipe out your whole analysis — the speaker says this has happened many times, including while dictating to their phone.
A good way to showcase that you need to design a harness that gives you accountability is making sure you can trace all the decisions that have been made, who made them, and whether it was an agent assumption you let carry on or something you provided a revised input to.
A locally coherent in-season lifting template: lower body day one pairs a squat pattern supersetted with a jump, plus a knee-dominant hamstring exercise (Nordic or Razor Curl) supersetted with a 3D lunge attacking the groin (lateral lunge), and optional mobility or core work. Upper body day one uses a horizontal push (dumbbell bench, push-up, floor press) supersetted with a dynamic horizontal push (band push-up or med ball throw), an upper back movement (face pull, band pull apart, YTWs) supersetted with a vertical pull (chin-up, pull-down), and optional grip work. The second lower body day uses a split squat pattern or trap bar deadlift per athlete preference — chosen based on age, injury history, and testing profile — supersetted with a split squat jump, then hip-dominant posterior chain
Consequential creator advice requires independent verification before Today can promote it.
The guest endorses creating something like the NTSB for AI, emphasizing that the key to aviation-style oversight is independence (the NTSB was set up as an independent agency in 1967), and that non-punitive reporting systems like ASRS incentivized manufacturers and airlines to share safety information and take every accident seriously.
CJ calls visibility into what the agent is doing table stakes: maintain a durable event stream with full trajectory across the entire lifecycle, listen to it in real time, and immediately interrupt any sessions going in a direction you don't like. You can also build a custom UI on top of the event stream to show tool calls and messages, and when accessing a flow programmatically you see every tool call, result, and sub-agent handoff.
When kicking off a big multi-agent project, the speaker wrote a short high-level prompt ('Ken Thompson made Unix in a week. Can you try and make an operating system...') and let the agents run rather than over-specifying; gave it a GitHub repo so output is inspectable and versioned; asked it to have a team of docs writers writing documentation about what each phase implemented so progress is monitorable via the repo; and instructed the flow to use a specific model for documentation writing ('use Sonnet 5.5').
Choose sources → to get a library of your own and a daily brief.