Recommendation · MoneyWhen building a resume-matching model, source jobs data that includes which companies are posting, how many jobs are available, salary, and location — and ensure data flows to you consistently because performance depends on continuous delivery.
stated
AI Engineer · https://www.youtube.com/watch?v=IK8cC4FScps
Recommendation · MoneyThor says the easiest way to try the Gemini Live API is through Google AI Studio: go to ai.studio/live (or ai.dev/live) to talk to the model and explore its capabilities. In AI Studio you can select Gemini 3.1 Flash Live, choose a different voice (e.g., Puck), and enable grounding with Google Search. The Live Translate model can also be tried there.
AI Engineer · Can Your Agent Hear You Now? Building Live Voice Agents with Gemini — Thor Schaeff
RecommendationRun custom harness judges in CI on agent-proposed PRs, experiment with them, and optimize from there.
endorsed
AI Engineer · Building Self-Learning Loops for Your Agent — Fuad Ali, Arize AI
RecommendationHarness-as-a-judge can connect to external systems and call them to verify whether the user was allowed to do the action and whether the trace reflects the access they had.
endorsed
AI Engineer · Building Self-Learning Loops for Your Agent — Fuad Ali, Arize AI
RecommendationA price-correctness rubric passes when every returned product's price is within the requested min/max range and fails when any result exceeds the budget. The rubric can go into the code, check filters work correctly, and catch the issue on every subsequent run — not just a pre-coded test case.
endorsed
AI Engineer · Building Self-Learning Loops for Your Agent — Fuad Ali, Arize AI
RecommendationEnable Signal to run on top of your agents to get clustering analysis of issues. Configure it to trigger alerts off a specific eval threshold (e.g. an eval that must stay below a 1% trigger rate), a latency spike, or a cost spike. It takes a few minutes to get started once traces flow in.
endorsed
AI Engineer · Building Self-Learning Loops for Your Agent — Fuad Ali, Arize AI
RecommendationWith Harness-as-a-judge, write the rubric as a plain-language sentence; it reads the actual trace directly and runs in a sandbox against real traces, instead of writing column mapping and eval template boilerplate.
endorsed
AI Engineer · Building Self-Learning Loops for Your Agent — Fuad Ali, Arize AI
RecommendationWith Arize skills, you can ask your coding agent (e.g. Claude Code) to go read your traces from within the repo, seeing what the agent is doing from trace data and the actual code at the same time. Example prompt: 'Hey, check my traces for any errors or anything I can fix. Cross-reference with the code for the agent in this repo and use the agent skills.'
endorsed
AI Engineer · Building Self-Learning Loops for Your Agent — Fuad Ali, Arize AI
Recommendation · MoneyTo use Data Designer for RL, skip the golden solution generation step and end up with a dataset full of prompts. Use NeMo Gym, which is essentially an RL reward function tool: you write a function that gives you a reward, then use any RL training framework to execute in real time with those tools and provide a reward.
AI Engineer · Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA
Recommendation · MoneyAlways use the cheapest correct column type when building data generation pipelines. Using an LLM to create a sampler is a complete waste of resources — write samplers in Python instead — but using an LLM to generate text data makes a lot of sense.
AI Engineer · Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA
Recommendation · MoneyUse pass-fail checks on actual output plus LLM-as-judge for impact analysis tasks when evaluating agentic systems.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Recommendation · MoneyTo build a code-based graph analysis agent, you can leverage call graphs at SCIP to understand who's calling this and who references this. It can read the comments inside and outside of the code, and fold in prose from outside of the code, including GitHub issues, linear issues, and tickets.
endorsed
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Recommendation · MoneyYoni Michael recommends mining gold data from the graph but cross-referencing it back to the source code so the gold isn't cheating — i.e., not polluting gold context with graph output.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Recommendation · MoneyYoni Michael recommends hooking the code graph in as an MCP to the tools you're already using — e.g., having the code graph in addition to Claude Code and asking Claude Code to call the graph — and then writing skills so the agent uses the graph properly. To engineer an agent to use these things in actual coding, promote the prompt-engineering practices into skills and give those to your coding agent. He notes an engineer receiving such a PR for the first time can get a half-decent first pass from being more deterministically guided, and that the deterministic part is helpful because you can incrementally diff over what's changed and analyze it deterministically, getting a fast conclusion about what's happening as opposed to waiting for the agent to read everything.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Recommendation · MoneyThe host recommends using a classification layer in front of an agent to check for prompt injection. For such a layer, you can either use an LLM-based layer, or alternatively a semantic search layer (e.g., vector search via Quadrant or LanceDB), or an in-memory process.
AI Engineer · How Secrets Leak Through AI Agents (and How to Stop It) — Venice AI
Recommendation · MoneyTo decrease risk when protecting agent-handled data, the host recommends ensuring narrow agent scope: if you pull tools from an MCP server that may have 15-20 different tools, you don't really want to use all of them — you actually only really need one. He adds that tight context loops caused by narrow agent scope can also improve context window performance, because fewer tools means the agent may perform much better.
AI Engineer · How Secrets Leak Through AI Agents (and How to Stop It) — Venice AI
Recommendation · MoneyThe host says using sandboxes and VMs, for example Firecracker and AWS EC2 in a VPC, can significantly upgrade the security of your agent. However, sandboxing does not act as a cure-all for agentic systems: if your agent has internet access enabled, it can still accidentally exfiltrate data through external API access if you're not careful, and therefore monitoring permissions is essential.
AI Engineer · How Secrets Leak Through AI Agents (and How to Stop It) — Venice AI
Recommendation · MoneyFor an LLM call observation, attach data like duration, token cost, token usage, model used, and prompt version used — everything you can think of that will help you assess the quality of your LLM call. For tool calls, the attached data can be completely different; for example, a web search observation can include the query that was used.
AI Engineer · Continuously Improving Agents with Langfuse — Annabell Schäfer & Lotte Verheyden, ClickHouse
RecommendationOmar Alhait recommends that when layout is hard to read, such as in a construction blueprint, you spend a lot of time in the layout prompt tailoring it to understand how well your current extraction solution maps across all of your different documents.
AI Engineer · Parsing the Epstein Files for JMail — Palak Agarwal & Omar Alhait, Reducto
RecommendationOmar urges the audience to send the Epstein cover sheet to ChatGPT and try to get results out of it to see how accurate it is, noting the cover sheet is a very complex parsing problem with hard-to-understand layouts and somewhat subjective reading order.
AI Engineer · Parsing the Epstein Files for JMail — Palak Agarwal & Omar Alhait, Reducto