417 results · Most relevant
When you ask an agent where a number came from, ask specifically for the paper trail rather than accepting a verbal explanation; make 'where did this number come from?' a standard check in every conversation.
Open a spreadsheet and look at a few cells you know will give you a ballpark total. If it doesn't feel right, say so. When something looks plausible but doesn't feel right, look into the calculations behind it.
If a decision influences the final decision, it definitely comes with accountability. Design a harness that gives you accountability by making sure you can trace all the decisions that have been made and who made them.
Correcting an input — e.g., saying 'actually it's not this number, we will get no fewer than 150 guests' — gets your harness to rethink the plan, see the consequences of that input into other stuff, and suggest either a modification of your plan or just changing the input. You are well within your right to correct information and add context after a decision has been made.
Design something with the intent that 'this really needs to be an aside' when you have too many questions you don't want contaminating your main analysis; the speaker notes that having too many 'stupid questions' leads to opening a separate chat because you don't trust it won't contaminate what you're doing.
When there are a few possible values for something, ask the agent to do three scenarios and review them side by side before making any decision.
Use the 'build your own' file in the repo — it has all the principles baked in, in order, and will guide you on building a similar harness.
Add a curveball to the agent by typing it in the chat and see how some of the evidence lines or inputs that go into the rest of your process change; the evidence lines should trace back to you correcting a fact.
Anekha Sokhal recommends that when handling customer data you avoid sending it to third-party processors if you can build in-house — Moshi built a vision language model to understand clients' messy PDF data, taking uploaded files, breaking them down, and rebuilding them in its own database (some data types require connecting directly to client systems instead). If you don't have a team of engineers, off-the-shelf document processing solutions are a great option; if you do have engineers, they will likely want to build the models. If you do use a third party, ensure you have agreement on how they process the data and what they do with it.
Endorsed guidance conditioned on team capability; no measured outcomes cited.
If you're getting golden examples of how the data should look, a good argument is: why would your process give you something different to that? Why don't you just check with your golden examples? Golden examples should be checked against the process output — if your process gives something different, check why against the golden examples.
The speaker showed a colorful example to senior leadership; when asked where a number and assumption came from, they couldn't answer ('I don't know where that came from'), which damaged trust in their harness and its results.
To start a harness session, describe the problem you want to achieve in the chat window — the chat window on the right-hand side starts essentially empty and is the place to start with a financial worry or analysis you want to do. Walk from the goal you want to achieve and try your best to give an explanation to the agent, and you can keep adding clarifying information to the chat as the planning conversation continues. If you want to run the harness for a problem you care about, start by describing that problem in the chat; if you pick your own direction, you can enter things like planning your own wedding, a trip, or a new investment in your company or personal life — anything relating to financial life where the same principles apply.
You can open a branch on a tagged challenge — a separate agent tasked with reviewing the plan, reviewing the assumptions, and doing some healthy challenge: does this read reasonable? Click the tagged challenge open, and you will see how Felipe did — he will join back into the trunk and change the plan, so there is a consequence. You can try opening another branch and just clicking on dismiss — some conversations don't actually change anything or aren't material to your process, and you will know better.
A person using a shared master file didn't know its embedded assumptions had changed; they did nothing wrong themselves, but their results became entirely wrong. Traceability for changes in shared files prevents this.
Anekha Sokhal describes breaking a market down into its component data types rather than treating it as one blob: structured data (futures time series plus technical indicators, and fundamentals time series covering demand and supply), unstructured data (news, X/Twitter, podcast audio), and the user's private data (trusted analysts, subscriptions, internal data).
Described as her own approach with moderate confidence; the atom carries a possible_numeric_error provenance flag.
Approving every calculation — the other side of the spectrum — doesn't feel great. Reviewing every PR line by line that your agents ship is also an extreme to avoid. Both lead to a lot of toil and unnecessary work.
For code, you don't necessarily need to ask before every decision because the tests will speak for themselves. Reviews come in handy in coding because you only need to check that everything is passing, and you will have a lot of confidence because of it.
One of the easiest ways to check whether a node in your decision-making process needs to be implemented as code: if it's a calculation, likely it has to be implemented as code. Calculations implemented as code should also have testing — unit testing, golden examples, etc.
Unstructured thoughts in a chat need to be labeled as an interpretation. Agent reasoning or pulling information from memory needs to be treated as an assumption.
Break tasks down in a way that you can reason about the issue and the way they connect.
No published protocol matches this selection.
The creators behind most of what cleared.
344 cleared insights match
71 cleared insights match
Choose sources → to get a library of your own and a daily brief.
