35,848 results · Most relevant
Sharing an MD file, PDF, or other output with another agent does not mean you have shared knowledge. The receiving agent loses the dead ends, failures, and full context, and must re-derive everything at token cost.
described
If there is context that Agent A can share with Agent B, it closes the lost context problem because they're working on the same context. This is the positive counterpart to the output-only handoff failure.
The hosts argue that when prospecting underperforms, the root cause is usually targeting the wrong accounts rather than weak copy or phone skills. Cold calls on the wrong accounts are bad calls no matter how skilled the rep is, and a turnaround often comes from removing waste from the prospect list rather than increasing activity or improving phone technique.
Use the adversarial piece in the planning phase: have Claude make a plan, then pass the artifacts to Codex for review asking what makes sense and what doesn't; Codex often pares the plan down a lot, and Claude is very amenable to influence — it concedes Codex is right and the end result is a better plan. Adversarial review isn't the only option: you can also scope work out until it's very clear, then divvy up the pieces at the clean seam so both models can work more rapidly (which, jokingly, burns more of your money faster). The speaker makes HTML plans so he can follow along.
Loops and similar primitives are going to be one of the key unlocks for higher levels of productivity that are less demanding to monitor; there are ways to let your agent go unattended for a bit and make decisions on its own. Nick highly recommends looking into loops and goals.
Beyond understanding the power base, the exercise tests whether the champion can actually connect you to the people you need; a real champion strategizes with you on how to reach key stakeholders.
As you move closer to production, switch the judge prompt from pairwise ('which is more helpful?') to pointwise ('is this good? give a verdict, pass or fail'). A pointwise judge prompt should ask for a pass/fail verdict based on how closely the answer adheres to the policy, generating only the verdict with no punctuation or extra text.
When going to production, don't identify your judge by a model name alias like 'GPT-4o-mini' because if the underlying version changes, you won't know and your evals will fail. Pin the specific version of the judge model rather than relying on a model name alias.
A one-shot judge example is not an eval; to move toward production you need a dataset. To move toward production, you need a sample dataset of about 30 examples. An eval is not a one-time thing; it is a data set — a collection of input, output, and a judgment on that output.
Tool use can be evaled without any AI: if you persist tool calls as JSON in an array, you can write deterministic code to check whether a tool was called and with what argument. You can still use unit tests for that — 'in the list of tool calls was this tool called? With what argument?' Exact match is very easy, free, costs nothing to run, and anyone can do it. Schema validation is an eval technique: if you have JSON blobs or message envelopes, you'll want to validate them, and you can have fuzzy rules. You should be at least examining your message envelopes.
Pienaar argues production debugging requires doubt, uncertainty, and eliminating theories, while LLMs are trained to be decisive, confident, and answer quickly. He cites an OpenAI system card showing confidence and accuracy deviated in the RLHF process, partly because post-training optimizes for human preferences, and concludes overconfidence is part of the model itself that builders must counteract.
AssemblyAI did not want to build a general support bot that just deflects conversations from customers — the kind anyone can build. Instead they wanted to dogfood their own product: since they work in voice and their customers build voice agents, they built a voice agent. Talking to your own voice agent about how to build a voice agent is itself a live product test — a meta layer where you're already using the product. Running through the same walls as your customers (latency, turn-taking, barging, interruption handling) builds better empathy and lets you give customers better advice because you've actually shipped something with the same API they build on. It's not enough to just teach customers how to get started; you need to be dogfooding your own product all the time, and building the
The creator's core idea: use a widely available but little-known shrimp that looks and tastes like lobster but is much easier to cook and much cheaper, to recreate the taste and texture of a New England-style lobster roll.
You can load custom sources into the sandbox, including custom dependencies, .env files, and helper files. Supported source types are Google GCS buckets, GitHub repositories, and inline files.
Gabe's core heuristic: with exhaustive verification you can ship code at the speed your agents write it. Without it, someone in the organization spends time verifying code, which becomes the new bottleneck, and teams are forced to trade off velocity against bugs and user experience.
: none
Seema Amble argues the case for AI-native startups centers on incumbents being limited to their system of record and not completing the end-to-end job; the startup's opportunity is to own the entire arc of a workflow (e.g., legal brief through trial).
Seema Amble states what she looks for in durable vertical AI companies: owning end-to-end work and building a data asset enabling something not done before. Moats are extremely hard to forecast — the best early-stage businesses focused on winning customer trust and selling more, not a predetermined moat sequence. Defensibility comes from customer dependency: when the agent owns more of the work, the customer becomes dependent on the product — the signal of reaching a moat. Stickiness and network effects are downstream of initial customer use and product value.
Alex Zhang explains that the original problem RLMs were designed to solve was harnesses having a really bad time dealing with long context; the deeper point is compositionality — giving language model systems more control over their actions at every step, with a central context they can always draw from. Standard tool calls are limited because they must be invoked sequentially every turn (tool A, then B, then C) with no central context to draw back from — a key weakness RLMs are designed around. Most harnesses keep appending the trajectory as a prompt, which makes tasks out-of-distribution unless the lab trained on user trajectories (Anthropic, OpenAI).
Alex Zhang argues there is still a lot of alpha in being knowledgeable in a domain even when AI can solve problems: companies still hire mathematicians for data labeling and steering models even with recent math-proof AI systems. People who know how to solve problems also know how to use AI to solve them because they act as a strong verifier — knowledge plus AI beats either alone. He adds that while throwing enough compute at a problem can sufficiently explore solutions, a knowledgeable person can uncover something for the model that would erase a trillion-token spend, so bringing in domain expertise is more efficient than brute-force compute; and since there are still many problems in the wild that need solving, we can't afford to always throw maximum compute at them — efficiency remains
Alex Zhang (with the host agreeing) identifies the core trick for agentic systems as externalizing state to a file system — 'that is the big trick' — and forcing everything to run through code, trusting the model to write the code, so it harnesses itself.
Choose sources → to get a library of your own and a daily brief.