35,595 results · Most relevant
Consider running speculative decoding when you have extra GPU space. If your workload is high concurrency, your GPUs are probably already busy running requests, and it would not make sense to implement speculative decoding. Smaller batch sizes would benefit from it.
Speculative decoding benefits workloads that are highly structured, such as coding use cases, writing JSON, or SQL prompts. It likely benefits less for highly creative use cases, like writing poetry or brainstorming, where there is more variety in how the next tokens are generated. High temperature settings in creative use cases also reduce draft-model acceptance rates.
moderate
Dru Knox recommends giving agents access to internal services (requiring governance and API access decisions), production logs (raising compliance questions), and an environment where the agent can execute the code it runs. He warns this Agent IT work will be harder than anticipated.
Eugene Teo says he is really, really big on people spending a good amount of time building up the personal side of their brand.
Thariq warns to be cautious of prompt injection attacks in enterprise agent setups — a prompt-injected agent can exfiltrate a codebase — and recommends implementing security measures for agent surface areas, especially when connecting to external data sources like Slack; he likens the attack surface to an iceberg with most risk hidden below the surface.
Lenny and Anish Acharya recommend starting to think about every function in your company as an agent loop, going from input to some kind of impact. The job is to figure out where the loop gets blocked or goes wrong, and give it more context, more insight, and more direction.
The big opportunity is building products around the loop 'make me happier' rather than just productivity, because more people want to spend time than save time. Apply AI to basic consumer needs — feeling more connected, more loved, making progress, having fun — and build AI loops for human goals like 'loop improve my health' or 'loop make me a better friend', including an app around improving connections with friends and family.
Anish Acharya says the most useful question to ask yourself as a founder or CEO is: if we assume these AI things are infinitely intelligent and astonishingly cheap, how would we reorganize the company?
Anish Acharya recommends focusing on the basics of consumer need — feeling more connected, more loved, making progress, having fun — rather than saving time, since more people want to spend time than save time.
Anish Acharya distinguishes between using AI (swapping in tools) and reorganizing the entire company around AI (redesigning processes from scratch). Less ambitious or earlier-stage companies give existing employees access to AI tools, while the most ambitious companies rethink everything around the models.
The creator recommends focusing on collaboration as a moat: software where value grows as human collaboration or agent collaboration grows is where there is more value.
The creator argues operational excellence is a viable moat: pressure will shift from developing code to operating it, so the better your operational and technical excellence, the better chance your SaaS company has to survive.
Gabriel lays out the workshop's core recipe: step one is giving Codex your context — from documents, your brain, conversations, email, Notion, and Linear, including a large code base with Git commits and history — because that context is a really important ingredient; step two is to just build things. He also notes you can collaborate with Codex on long-running tasks and work over time to make tasks repeatable, ensure consistency, and build loops.
Kent C. Dodds recommends validating that a problem exists and is interesting enough before building a solution. Because of AI, you can prototype things really quickly to validate the problem, build a simple solution with paper cuts and see if people resonate with it, and find something important enough for somebody to want to try it out.
Laurie Voss warns that the goal isn't 'make the evals pass', because otherwise the coding agent will cheat by including all of your test data into its eval and automatically pass all of your evals. Make sure your requirements to Claude Code are clear: you want it to do the specific thing, which then makes the evals pass.
Hand a pile of eval explanations to a coding agent and ask what it could do to get better — e.g., 'here is all the reasons that you messed up in the last test. What could you do that would make you better at this?' — and it will improve your application so it gets better next time.
Set up tracing in Arize AX to capture the raw data you need to run evals in the first place. When running evals, pull the input and output attributes out of these spans and feed them to your evaluators — the input becomes the evaluator's input, the output becomes what gets graded.
The source recommends that when a voice agent handles sensitive actions like refunds, verify the exact parameters in the tool call trace, and that when the agent mishears a number, verify the tool call executed with the correct value.
The source recommends keeping time to first audio under 2.4 seconds to avoid dead air, and setting time-to-first-audio SLAs for customer service voice agents.
The source recommends that when a user corrects a voice agent, verify the correction was actually processed before the call ends, and that when a transcript shows the agent 'correcting itself', verify the correction in the audio.
Choose sources → to get a library of your own and a daily brief.