Recommendation · MoneyThe speakers recommended having Kubernetes network policy — at L4, L7, and wherever possible — to solve network-level challenges, which is directly relevant to blocking data exfiltration attempts.
AI Engineer · Build a Platform and Watch It Burn — Michael Forrester, Accenture & Whitney Lee, Datadog
Recommendation · MoneyThe speakers recommended giving your agent an identity using Spiffe or Spire, then making policy around what it is and is not allowed to do.
AI Engineer · Build a Platform and Watch It Burn — Michael Forrester, Accenture & Whitney Lee, Datadog
Method · MoneyA code-based evaluator can catch user frustration via all-caps usage — if the presenter gets frustrated with tools like Claude or Codex, they might just use all caps to get their point across. All-caps usage is a good way of catching user frustration in chat applications. As an alternative, a frustration evaluator could also catch curse words or certain terms you wouldn't usually use in a chat but might use with an agent. The code-based evaluator is a Python snippet where the evaluator function targets a JSON path, extracts a user message, checks if there's all caps, and returns a Boolean score of true if so. Copy only the bottom part of the Python snippet when wiring it up, because the upper part is static. Langfuse also supports TypeScript code for code-based evaluators. Name a Boolean-s
Method, not a verified protocol.
AI Engineer · Continuously Improving Agents with Langfuse — Annabell Schäfer & Lotte Verheyden, ClickHouse
Method · MoneyRun evaluators on live observations to know what's going on in production, targeting the observation type 'agent' again. Use the test run panel on the right side of the evaluator page to verify the evaluator — in the demo it returned false because the sample trace only asked about turning Bluetooth on and off.
Method, not a verified protocol.
AI Engineer · Continuously Improving Agents with Langfuse — Annabell Schäfer & Lotte Verheyden, ClickHouse
RecommendationOmar recommends treating unsatisfactory output on a plugged-in document as a signal to revise the schema — plug in a real document, observe the output, and adjust the schema description accordingly.
AI Engineer · Parsing the Epstein Files for JMail — Palak Agarwal & Omar Alhait, Reducto
MethodThe speaker advises that as with most projects, gather a simple dataset, do some vibe evals by looking in Studio to see if outputs make sense, then scale to 100 or 1,000 documents for something more systematic. You can also work with Reducto's team, who have people dedicated to helping customers do their evals.
Method, not a verified protocol.
AI Engineer · Parsing the Epstein Files for JMail — Palak Agarwal & Omar Alhait, Reducto
Recommendation · MoneyUse the highest-end models for your evals even if you don't use those models in production, because your agent is only as good as its evals. You can connect any model provider you want for evals, including local open source models, rather than being limited to one provider.
AI Engineer · Let Your Agent Cook: Using Skills to Evaluate and Improve Your App — Ankur Duggal, Arize AI
Method · MoneyUse the Arize dataset skill, the Arize evaluator skill, and the Arize experiment skill together to build evals and run experiments. Tell the agent to make the changes it recommended and run the experiments so you can compare against where it failed to make sure it passes against that.
Method, not a verified protocol.
AI Engineer · Let Your Agent Cook: Using Skills to Evaluate and Improve Your App — Ankur Duggal, Arize AI
Recommendation · MoneyTest fixes against a dataset built from the spans that caused the initial failure, then run experiments to compare against where it failed to make sure it passes. In the cross-market case, the agent found the failing areas, built a dataset from them, and iterated fixes until the passing rate improved.
AI Engineer · Let Your Agent Cook: Using Skills to Evaluate and Improve Your App — Ankur Duggal, Arize AI
Prediction · MoneyRemy predicts the next 12 months will be marked by 'ROI maxing' rather than token maxing: not just cutting costs but proving value, and maxing every opportunity for choice at each layer of the AI deployment. ROI maxing is all about maximizing choice to prevent lock-in.
prediction
AI Engineer · An AI Future Without the Lock-In — Remy Guercio, Tailscale
Recommendation · MoneyPeter Werry recommends adding another LLM step on top to scale across multiple collections, where schema collections have tiny summaries and descriptions, then load the right schema for the query.
AI Engineer · Beyond RAG: Build a Relational Context Engine from Scratch — Peter Werry, Unblocked
Recommendation · BodyWhen a client says they want to deadlift 200+ kilos but their back squat is significantly lower, treat raising the back squat as step one.
Garage Strength · How To Get Strong and Still Be Athletic w/ Sika Strength
Recommendation · BodyFor field sport athletes, keep back squat volume moderate to low during the competitive season.
Garage Strength · How To Get Strong and Still Be Athletic w/ Sika Strength
Recommendation · BodyFor a 5K time goal, run approximately twice per week.
Garage Strength · How To Get Strong and Still Be Athletic w/ Sika Strength
Method · BodyFor a 90K guy who can train five days a week wanting a good deadlift and a good 5K, the guest outlines a clear scheme: first look at the client's current deadlift technique and strength level; second, look at the back squat, since it's back squat driven deadlift programming all the time. If someone deadlifts 180 kilos but back squats 140, they're at step one — the back squat needs to be bumped up before the deadlift can progress. Work on the back squat for three or four months, practice the deadlift the entire time you're back squatting, and expect a reasonable transition between the two in the two months after the squat block. Do two to two-and-a-half or three gym sessions per week with deadlifting or strength training as part of the hybrid equation.
Method, not a verified protocol.
Garage Strength · How To Get Strong and Still Be Athletic w/ Sika Strength
Recommendation · BodyTaking more steps in the first 5-10 meters of acceleration improves sprint mechanics.
Garage Strength · How To Get Strong and Still Be Athletic w/ Sika Strength
Method · BodyBefore starting a running training block, check whether the person has trained enough to actually train running — training to train is very different from doing a running training block. If someone hasn't run in two years, starting day one of a 5K progression program will put them in a world of pain because the program assumes their ankles, knees, and hips are ready for the mileage. Start with very basic baseline plyometrics and very short distance running. Don't think about running in terms of pace — think about it in terms of exposure. If you want somebody running a 20 minute 5K (four minute kilometer pace), most of the runs should look like four minute kilometer pace, and in the initial phase with very little running volume, the stride should look like the target race pace stride. There
Method, not a verified protocol.
Garage Strength · How To Get Strong and Still Be Athletic w/ Sika Strength
Recommendation · MoneyThe speaker advises running certain agents (especially maintenance tasks) in the repository on the repo's budget rather than your own budget, and using the repository's identity rather than your own identity when the task is on behalf of the project.
AI Engineer · From Your Laptop to the Pipeline: Scaling Custom Agents with GitHub Copilot
Method · MoneyWhen live-pulling a customer's full set of tickets, summarize them and store the summary in your structured stored context so that future common questions don't require re-looking-up and re-summarizing everything. With a local (synced) lookup, answering 'which customers were upset last week' becomes really simple: gather all customers' number of tickets, summarize their tickets, and give a status on each one. An agent checking Zendesk can answer 'why was customer A upset last week' by finding the tickets filed by that customer, seeing the API sync was broken, and noting the customer filed three tickets total.
Method, not a verified protocol.
AI Engineer · Why Your Company Needs a Context Graph (and How to Build It) — Gil Feig, Merge
Method · MoneyFor an example prompt — 'Give me a one paragraph status update on our customer Acme. Include renewal risk and last support interaction.' — first check the cache for the account summary (assuming no skills match). If the cached account summary is stale or missing, call the CRM using a live API lookup. Query the ticketing system for the last support interaction, using cache within the SLA, otherwise a live lookup. Pull only five tickets rather than the full history to avoid timeout risk from live lookups. Generate a derived summary, possibly store it, and pass it back to the client.
Method, not a verified protocol.
AI Engineer · Why Your Company Needs a Context Graph (and How to Build It) — Gil Feig, Merge