Report · MoneyMCP capabilities include search engine queries, Google results, scraping pages as Markdown files or HTML, targeting specific sites like Amazon, Reddit, LinkedIn, and Instagram, and browser automations like clicking and scrolling.
stated
AI Engineer · https://www.youtube.com/watch?v=IK8cC4FScps
Report · MoneyNVIDIA's search dataset training took Nemotron from 0% to 30% on the BrowseComp benchmark.
AI Engineer · Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA
Creator perspective · MoneyData Designer has a concept of model providers; the presenter used vLLM for local inference on an NVIDIA GPU cluster with GPT-OS 120B available, and any model or open model can be used to create synthetic data. Structured columns let you generate JSON, Markdown, or custom formats instead of plain text. You can also add LLM-as-a-judge to score outputs, e.g., a second model judging the quality of questions the first model created.
AI Engineer · Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA
Principle · MoneyYoni Michael describes writing an eval that examines all tools in the trace and measures how often the agent uses your target tools, plus a deterministic check that verifies the agent cites all the evidence you expect it to cite, comparing what the agent said to what you expect for each given task.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Principle · MoneyIf the speaker was actually building this and testing it, they would pick a specific snapshot of the code base — that lets you have a deterministically set of gold questions when you run your evals. The speaker acknowledges that the snapshot approach is kind of a cop-out answer.
endorsed
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Report · MoneyFrom about a year and a half of experience, Yoni Michael identifies three concrete areas where today's coding agents fail on data: impact analysis, join recommendations, and safe code/transformation logic changes. These are not fringe tasks — they are the day-to-day workflows of data teams getting enterprise data to work with AI.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Principle · MoneyYoni Michael describes the eval execution setup: each task runs multiple times, on isolated clones of the actual data environment so there is no leakage across environments, with pass-fail checks running against the actual output of agent tasks, plus LLM-as-judge for impact analysis evaluation.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Report · MoneyYoni Michael reports the eval comparison between the graph agent and a grep-based control agent: on pass@k (ever correct across five runs) they are very similar with grep having a slight edge; on pass^k (correct on every run) the graph agent is twice as good; and the graph approach took about 3.1 times fewer tokens on average across all runs. He also notes the first k=1 run showed the graph agent only edging out control slightly and inconsistently.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Creator perspective · MoneyBusiness definitions are often nested in the SQL itself. An agent asked to tidy up a metric can remove a predicate; syntactically the SQL will still compile, but the predicate may have been a very important part of the business definition. Example: an agent removing a predicate silently redefines the metric to include trial churn, propagating across three downstream dashboards representing that metric. The context layer catches changes in definitions and business metrics before they are actually implemented.
hypothetical example
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Principle · MoneyBrandon Callender describes building out the graph by transforming all the extracted symbols and information into a bunch of nodes and edges that are populated into a Neo4j graph. Once loaded, you can go into Neo4j and see a graph of references to all the different parts of the codebase.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Report · MoneyIn Typedef's benchmark, holding models and task constant and varying only access to the data context layer produced a task success rate almost 35 points higher when the agent was armed with the data context layer, on complex data changes.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Creator perspective · MoneyYoni Michael recommends nudging the prompt so agents use graph tools instead of defaulting to grep, and notes agents sometimes poke the graph, get confused, and return to grep. If an agent gets no results from a tool, it thinks the tool is broken and won't return to it.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Report · MoneyTypedef's product builds a context graph and ontology layer over a data architecture, giving agents a tool set to answer questions and make changes across the data system.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Report · MoneyYoni Michael says building an amazing graph context for a codebase is something you can do in an afternoon. If actually building this, he recommends using something like SCIP to trace call graphs inside the code base — SCIP lets you see what is calling what and how it interacts, with outputs available for Rust, TypeScript, and Python. Tracking symbols and how they move across the codebase is straightforward, but tracking what is calling what requires a slightly augmented approach, another rung of deterministic parsing. He adds there is definitely more structure that can be extracted from a given code base, and an audience member asked about higher levels of granularity beyond symbols.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Principle · MoneyBrandon Callender describes building a blast radius tool that takes a qualified canonical name, walks down the graph edges, and figures out exactly what depends on that thing and what will break if it gets changed. He also suggests writing a tool that executes arbitrary Cypher, such as one that finds all of the symbols within a given file path and lays them out, and building tools that specifically harness the capabilities of the graph to give the agent easier access to what it needs.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Report · MoneyTask success rate for complex data changes is almost 35 points higher when the agent is armed with the data context layer compared to when it is not, per Typedef's internal eval tasks on real production data engineering use cases with multiple runs per task on isolated clones.
publisher_reported
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Report · MoneyYoni Michael reports that in Typedef's benchmark evals, task success rate on complex data engineering changes goes up almost 35 percentage points when the agent is armed with the data context layer, and that the benchmark will keep growing and evolving as they work with more teams. He adds that richer context changes outcomes tremendously compared to just swapping models or adding generic MCP tools.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Creator perspective · MoneyYou can verify join safety against the actual data by profiling the tables in your warehouse to check whether a join key is really unique. Profiling allows the agent to not guess about things like cardinality.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Principle · MoneyYoni Michael describes building up a corpus of questions where each question is a real task, creating real production data engineering use cases, tasks, and workflows ranging in difficulty from easy to blind. Questions are categorized as good for the graph, good for grep, or good for both.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Principle · MoneyYoni Michael describes setting up a control agent that has access to the same model and a set of file system probing tools maintained by Pydantic, so the graph-armed agent can be compared against a baseline.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef