35,684 results · Most relevant
Jesse Hall advises against trusting dashboard-reported time-to-first-byte as a proxy for perceived responsiveness and against using time-to-first-token for LLMs, because tokens do not equate to audio. When TTS needs an entire sentence before speaking, the metric that matters is time to first sentence.
Rob Snyder recommends using the theory of constraints from manufacturing to look at your case study factory and find the bottleneck to your growth.
Start small with evals: 20 traces, one dimension, one judge. Read the explanations and iterate over them, and then you can expand to more and more evals.
Find trace data by clicking the traces option on the nav, then opening the demo project from recently viewed. Open 10-20 traces and start thinking about what you'd want to measure as you look through the data. When exploring traces in the Arize UI, you can expand the time range to see more data. Walk down the trace tree one step at a time to see how the steps connect; sometimes you have to go down the trace to come back up when analyzing spans. Switch to a span to see the data passed in and out of a step, such as the OpenAI input and output. Additional attributes and data for a trace can be viewed in the attributes section. Take inventory: what are users asking for, what did the agent do, which tools, what are the answers — and note whether it's an answer you would ship.
To build an eval, go to your traces and select a few choices to add to your dataset. Use filters to identify data — the best way to identify data. Filter root spans by setting parent ID equals null (or use default traces), and filter tool calls by name to identify specific tools. If you want to test a tool call, add all of those tool calls to your dataset.
LLM-as-judge research shows it only does well handling one criterion at a time, so it's good practice to judge one quality per eval and add more evals over time. Exception: for decision models with agent-as-judge, you can handle multiple criteria in one eval.
speaker cites 'lots of research' without specifics
Sofia Jakovcevic advises that when a team wants to switch to a new model that is five times more expensive, you should prove ROI with a numeric set of measurements before switching, using the experiment suite.
Supporting hybrid architectures is not trivial: it is not enough to add a different memory pool per layer type — you must also design cache-sharing strategies across pools.
Dr. Forbes advises parents that before starting children on supplements, they should ensure regular exercise, good sleep, and whole foods with lots of fruits and vegetables, which are much more important; if a child is athletic, active, and doing the basics, definitely give them creatine — it is safe and can improve performance. He adds that parents often worry about a 14-year-old on creatine while allowing Burger King before football.
age detail flagged as possible numeric error
Coach Kyle advises that a woman being with her family or a crowd is not a reason to skip the approach — he models going up and saying something anyway, noting that most guys talk themselves out of it and that nobody around actually cares.
Sam Green advises that unstructured data such as news, research, transcripts, policy documents, and internal memos needs the same careful treatment as structured data: cleaned, governed, and made ready for point-in-time research and systematic study over time.
Sam Green advises that when building a data platform, data should be made ready for both point-in-time research questions today and systematic study over time.
Aaron Linsky advises that when building AI research tools, you should train custom models to reason over your proprietary data and use them together with foundation models to feed systematic decision-making processes.
On the trading side, Contour measures entry and exit times and thinks about how a position is spread across multiple industries and multiple names within an industry. AI agents track names across the portfolio — including positions spread across industries — to make sure traders are on top of interesting insights.
Kumar describes Contour's goal of marrying all parts of the ops and compliance workflow into one specific tool that those functions can use, and consolidating financial operations work into one platform instead of going into many different platforms for different pieces of the order flow. The aim is to minimize repetitive work and avoid being reliant on external providers for piecemeal solutions like order reconciliation or vendor info tracking.
Dane Knecht argues that when AI transforms how a team works, you should redesign the whole process rather than making the existing multi-team process faster — sometimes the process shouldn't involve those teams at all. He treats the org like a factory floor that needs rebuilding, which may require hard structural changes including restructuring. As an example, Cloudflare replaced a multi-team workflow with a single AI-powered tool where a product manager describes the cohort, writes the email, it rewrites in company voice, sends to a manager for one-click approval, then sends. The principle: collapse multi-team processes into one tool that gives an individual a superpower.
creator interpretation and advice, illustrated with a Cloudflare example
Create stakes within the company — if you don't hit the goal, you'll be humiliated, miss your goals, could lose a lot of money, and everyone's jobs are at risk.
Farza describes his opening pattern: a casual 'Hey, everyone. So this is...' then zoom in and move the product around so people can see there's something moving there, prompting the viewer to ask 'what is that?'
Farza describes rehearsing each script line in his head over and over — in the shower, going to work — asking how to say each line, so that by the time he's in front of the camera he has already visualized the delivery about 10 times.
Farza recommends launching demos almost every single day rather than saving content for polished productions. He describes how his company now does build-update videos every two weeks and it's been really fun, and that he forces himself to just make videos and make stuff every day in the early days. He also notes that spending lots of money on fancy produced videos is an effort to seem polished and further along than you are, which he takes the opposite approach to.
Choose sources → to get a library of your own and a daily brief.