35,595 results · Most relevant
Clone the workshop repo, cd into the research desk directory, run npm install or bun install, then spin up the server with bun run dev. The first prerequisite step is copying .env.example into .env.local.
A demo shows the basic agent creation flow: prompt Claude with 'build me an agent that tells me jokes every time given a topic' and hit enter; Claude generates the system prompt, name, description, and model. Give the agent access to the agent tool set, then hit create agent, which calls the v1 agents endpoint. Configure the environment (the template for the sandbox) by creating a new one, choose packages to install, then start the session and test run it with a prompt like 'Tell me jokes about cats'.
Type 'workshop next' to have Claude implement the provisioning scheme to upgrade the agent with sub-agents and an SEC skill. There should be a button in the setup that says 'staff to desk' — when clicked, it runs the provisioning. You can also delete the skill and ask Claude to reprovision it so it recreates all the resources, including the sub-agents, the skill, and a memory store.
Give the agent a skill that lets it talk to the SEC, describing how to use the Python package and that it has to use the Edgar identity to call. The SEC requires you to provide your name and email when calling their API so they know who's calling; put your Edgar identifier (name and email) into .env.local.
Use a memory store that allows sharing knowledge across sessions, so agents can reuse what was learned previously.
In the demo UI, type 'build me an agent that tells me jokes every time given a topic' and hit enter. Claude automatically generates a system prompt, name, description, and model choice; you can change the model before creating. Then give the agent access to the agent tool set and hit create agent, which calls the v1 agents endpoint.
Buy, don't build, when a third-party product can do the job — even if you've already built your own version. If a vendor could have delivered the same capability in an hour, buy it instead of building — but only if you evaluate the vendor before building. Before building an internal agent, check whether the vendor supports your full stack — quirky integrations (QuickBooks, Bill.com, PandaDocs, Canadian entities) are a common reason third-party products won't work. A third-party agent built by a strong team is less likely to catastrophically break your application than your own vibe-coded build. Vibe coding your own CRM is moronic — don't build your own CRM even though edge cases sometimes justify building.
The first takeaway from the case study is to prioritize user experience. Jake tanked while focusing on technical SEO basics, then switched to user experience and giving people what they want — which produced massive growth. He also turned down ads significantly because he believes they deliver a bad user experience, accepting a direct revenue hit, and says delivering high quality and getting information to users quickly makes people remember you.
The revenue-hit-was-a-blessing framing is Jake's personal belief ('I feel like that hit was probably the best thing that could have happened to my business').
The host repeatedly recommends automating tax collection because nobody wants to get in trouble for not knowing the tax rules. In the EU specifically, digital subscriptions are subject to VAT, so she recommends using Stripe Tax when selling digital subscriptions there.
Kim Maida recommends using a security token service to manage agent access control with delegated, task-based policy governance, and routing every agent request through the STS and policy evaluation before a token is granted and the agent is authorized to act.
Step one: create an environment via the Environments API (V1 environments) to define the sandbox template, including networking (unrestricted or restricted to specific domains/MCPs) and packages installed on the container. Step two: create the agent, giving it a name (e.g., 'joke teller'), choosing a model (Opus 4A, Haiku, or Sonnet), providing a system prompt (e.g., 'You're a comedian. Find a fresh joke on the topic. Deliver it well.'), and providing tools (e.g., the agent tool set: bash, read, write, grep, etc.). Step three: create the session, which takes an agent ID and an environment ID.
Nnenna Ndukwe recommends starting a quality-driven agentic engineering process by defining what code quality means to you and your team. The standards then need to be codified somewhere your AI can extract them so it can adhere to your code standards regardless of interface, and made distributable across all interfaces: CLI, IDE, and CI workflows.
Instead of vague prompts like 'create a user login screen in Python', specify all security requirements the agent should implement, such as a login function taking email and password, rate limiting, and parameterized queries.
The creator recommends building for agents as well as people if you want to win in the next era of consumers, arguing it might today be more important to build for agents than for people. Going forward, 'make something people want' (from Y Combinator, the startup factory from Paul Graham) should also mean making something that agents want. If you don't build for the agent economy, your product might not win or be successful.
The evaluation is probably the most crucial part of an AutoResearch system: you want a very simple way of allowing the system to evaluate every solution it generates, and you want it to output a single scalar metric. Ask yourself two questions for the metric and goal: what do I want to improve, and is higher or lower better? Make your eval fast, cheap, and low-noise, because this leads to your ability to run more steps under a fixed budget or time constraint, and low noise makes both you and the agent more confident while hill climbing.
Thor Schaeff recommends using a native audio model like Gemini Live instead of a cascading voice agent pipeline when fluid multilingual switching is needed, because the native model can switch very fluidly between different languages.
Thor Schaeff recommends sending real-time text, audio, and video frames to the Gemini Live model over a WebSocket connection, and streaming the audio chunks returned by the Gemini Live API into the application.
All the agent's loop information should be available in the PR itself so a human can read it, validate the code, do code review, and ship. Keep a human in the loop for code review on agent-proposed fixes — the speaker recommends this as a limit on how far to take self-healing software.
endorsed
Get tracing started first: if you're not collecting tracing, nothing else works. It takes sub-ten minutes on your own agent.
endorsed
Evaluation is multifaceted, so you can't just use LLM-as-a-judge alone and expect it to give you everything — checking a dataset for a hundred problems only tells you those hundred issues don't exist; there could be a thousand other issues. Use LLM-as-a-judge as one generic layer, send judge outputs to human reviewers who cross-check the judgments, and for search data evaluate against real-time information as an additional layer. Find a union of all the different layers of evaluation to gain enough confidence in your dataset.
Choose sources → to get a library of your own and a daily brief.