34,064 results · Most relevant
Jesse Hall recommends that when a caller asks a question requiring a backend lookup (transaction, payment status, dispute), the agent should acknowledge the delay verbally — saying something like 'give me a second, I'm looking that up' — and multitask by continuing the conversation while the tool runs in the background. When latency cannot be hidden, the agent should ask another question to keep the conversation moving.
Karthic Rao recommends designing each LLM call as a small, typed, schema-checked unit so the same one-line migration change applies to almost all of them.
Karthic Rao recommends pointing every model call at a privately hosted LLM as a configuration change rather than a rewrite, because each call is small, typed, and schema-checked; one line changes and the same answer comes back.
After the first batch of sales calls, if people have pull, you now have customers — time to deliver, start repeating, and figure out how to automate and systematize. Once a couple of customers say yes, do not just send them a login and pray: go deliver the product firsthand yourself. You will learn a ton from it, and passive onboarding causes you to miss critical learning opportunities.
The first step of defining an eval is defining what good means. Avoid vague criteria like 'helpful, accurate, good response' — the more specific and concrete you can be, the better your eval will be. Write concrete criteria such as 'answer that follows the days' and 'keeps totals within the stated budget'.
Sofia Jakovcevic says a regression dataset should be representative of the average workflow users push into systems, and should capture known failures (moments where the agent often errors or fails eval metrics), edge cases even though they are less common, and successful workflows you want to protect, because a fix must not break what was already working. She notes that forgetting to capture successful workflows is a common user mistake.
Sofia Jakovcevic recommends pushing the experiment into CI, for example in GitHub Actions, so each pull request runs through an experiment workflow against team-deemed-important evaluations. Merging PRs without meeting regression criteria means pushing on vibes instead of with confidence in positive performance impact.
Dr. Scott Forbes recommends taking 5 grams of creatine per day for muscle benefits, or using a relative dosing strategy of 0.1 grams of creatine per kilogram of body weight per day — calculate your daily intake by multiplying your weight in kilograms by 0.1. Bigger people may require a slightly higher dose because they have more muscle and more creatine turnover. Brain-focused creatine studies often use 20–25 grams, but that high dose is probably not necessary for cognitive benefits; taking 20–25 grams daily is still safe but unnecessary — you're just wasting money on double or triple the required amount.
Consequential creator advice requires independent verification before Today can promote it.
When search complaints arise, respond by proposing to benchmark it rather than arguing: collect questions and find the best documents by hand, including ones that looked helpful but weren't. About 200 examples took less than a week and was very helpful despite not being fun or glamorous. The key search metric is precision at 10: of the top 10 results, how many are the right documents.
Kumar describes a process Contour used successfully over the past few months: run live A/B tests for research workflows. Track a single analyst's workflow on one specific name across a three-month earnings cycle (one earnings report to the next) with no AI tools, documenting the data sources the analyst finds and the conclusions they reach. Then run the same research process again with the PM involved and AI tools introduced one by one, comparing outputs against the documented baseline. When evaluating the results, ask whether the analyst would have come to the same type of conclusion with or without the AI tools.
Contour developed metrics for analyst productivity: insights generated by an analyst per day, time spent on each name, and the eventual P&L outcomes from each decision. They also track cost per useful insight — a metric Contour came up with — to understand where ideas eventually land, which ones were interesting, and which tools were used to come up with them. Kumar emphasizes that understanding where analyst time goes is really valuable when measuring ROI of AI and tech spend.
Figure out who your avatar is — ideally a market that is growing, in pain, has a problem to solve, can afford your stuff, and is easy to find.
Set a public goal with a date and announce it to the world — a shot across the bow — to incite energy and alignment within your team; the public goal cannot be cancelled.
Once you know who your audience is, figure out what they like and associate your brand through it: content about the thing they like, products they like (free and paid), and appearing next to the people they like.
Farza recommends working backwards from the launch of whatever you're working on: decide the launch video and copy before building the feature.
Jarmak recommends that after deploying changes you have measurement systems in place to capture all relevant data in the actual environment the system is deployed in, otherwise you cannot validate the changes.
Jarmak recommends decoupling what an agent or other system flags from actually enforcing that gate within the code itself and how it is escalated to a human.
Jarmak recommends setting up internal benchmarks within the organization that reflect actual workloads as closely as possible and running an iterative flywheel process to hill-climb toward optimal configurations.
Jarmak recommends coming up with explicit boundaries for when it is safe for the agent to deploy autonomously versus when it should stop and escalate to a human, to avoid unsafe autonomous deployment.
The host recommends that founders should talk to their users, and suggests asking them 'What do you think of this website?' to get feedback.
Use saved training trajectories and a controlled intervention to explain a failure, separating equilibrium theory from observed empirical stability.
Demonstrate all 3 repaired bugs and defend why a passed gradient check is necessary but not sufficient for generalization.
Use the tractable fixture and posterior predictive results to separate approximation, sampling and model errors; justify one revised modeling assumption.
Choose sources → to get a library of your own and a daily brief.
Map each implemented component to the original paper and explain 2 reduced-model failure modes without claiming full-scale reproduction.
Use calibration and shift measurements to justify a monitoring or abstention threshold, and give an explicit confounding counterexample to causal feature-attribution claims.
Defend a causal claim or refusal using the DAG, numerical estimates and a counterexample showing why predictive model fit cannot establish identification.
