Principle · MoneyTraditional software engineering assumes deterministic systems, but AI systems are non-deterministic and probabilistic, with multimodal context dependency in responses — asking an AI for a login page three times will likely yield three different implementations. This is why AI-centric threat modeling is needed: what works for traditional software doesn't work anymore.
AI Engineer · AI Security Engineer Foundations + Certificate — Javier Garza, Snyk
Principle · MoneyThreats in AI systems must be prioritized based on impact, likelihood of exploitation, and business and safety impact.
AI Engineer · AI Security Engineer Foundations + Certificate — Javier Garza, Snyk
Principle · MoneyWhen training an LLM, don't include data that can be used against you later, such as registry data from the internet that has a planted indirect attack.
AI Engineer · AI Security Engineer Foundations + Certificate — Javier Garza, Snyk
Principle · MoneyAdi argues we might be jumping the gun on what agent identity looks like. Identity isn't something you can define from scratch; identity is formed. Human identity systems are based on recognition of who we are, how much trust we place on each node in the system, and how that has accumulated and evolved over time — including through errors, fallbacks, and mishaps. Agent identity is not something agents get issued; it's something they accumulate.
AI Engineer · https://www.youtube.com/watch?v=mBNVKkyfnE0
Principle · MoneyAdi's key insight from the experiment is that agents actually want to live on the same surfaces as humans do. Agents don't have to adopt a new network because they're joining the ones we're already on, where we've built the trust layers, reputation, and engage in two-way transactions every day.
AI Engineer · https://www.youtube.com/watch?v=mBNVKkyfnE0
Creator perspective · MoneyThe new specs focus on security over adoption. The better the key card and the more hinge on security, the less the agent can do, the less access, and the less adoption you leave room for. Identity specs tend to be too restrictive: scoping tighter, expiring faster, revoking sooner — all sound, but they don't help drive adoption or bring agents into surfaces. Agent identity specs focus on the end state but don't account for the rest of the world where agents need to communicate with existing surfaces of today.
AI Engineer · https://www.youtube.com/watch?v=mBNVKkyfnE0
Principle · MoneyGovernance is many times forgotten to be approached from the very beginning; teams focus on technical aspects and only observe business effects afterwards. With this framework, teams start with governance from the beginning, which helps in the envisioning phase of choosing what functional area to approach with AI. In Dan's projects, departments literally compete over who should come first for AI adoption because they each have a board mandate to absorb AI, and the framework helps them decide.
AI Engineer · https://www.youtube.com/watch?v=PwCSYy-oGsM
Report · MoneyThor Schaeff demos the 'Gemini Nano Banana Story Canvas', a vibe-coded example where Gemini Live comes up with a story (an audience-suggested 1980s Taipei tale culminating in the birth of bubble tea) and uses Nano Banana Lite to illustrate it, generating images on the fly via tool calls and the Gemini Interactions API.
AI Engineer · Can Your Agent Hear You Now? Building Live Voice Agents with Gemini — Thor Schaeff
Report · MoneyData Designer lets you validate your generations; the whole idea is to build a DAG or graph describing how your dataset should be, then execute it with one click to get a dataset out.
AI Engineer · Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA
Report · MoneyNVIDIA created NeMo Data Designer, an open-source tool for building data pipelines that can add functionality to a model or agent, installable via pip and backed by a large NVIDIA team.
AI Engineer · Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA
Creator perspective · MoneyIf you give an agent the chance to cheat, it will cheat — it will just give the answer instead of using the search tool calls you want it to use.
AI Engineer · Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA
Creator perspective · MoneyModels have a knowledge cutoff because you have to stop collecting and creating data at some point to start training. Anything after the data freeze date is not part of the model's training data, so the model cannot answer questions about current events from its training data. Learning about current events is as important, if not more important, than knowing about things from the past — that's why models need to learn how to do search, and the model needs access to real-time information to do useful things.
AI Engineer · Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA
Creator perspective · MoneyThe proposed alternative to curating internet data is to synthetically generate data, either from scratch or from a seed dataset, which is what Data Designer is for.
AI Engineer · Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA
Report · MoneyThe workshop teaches how to take a target agent behavior, turn it into a data problem, and train a model to learn that task; the target use case is teaching a model to do search, and the framework is reusable for other agentic behaviors.
AI Engineer · Teaching Agents to Search with NVIDIA Data Designer — Dhruv Nathawani, NVIDIA
Report · MoneyYoni Michael describes the core idea behind the data context layer: an LLM-enabled layer that sits between business people (accountants, analysts) and data engineers so both sides can understand each other, and a unified vision of how everything works from runtime through lineage so agents can answer questions across layers.
AI Engineer · The Data Context Layer for Agents — Yoni Michael & Brandon Callender, Typedef
Report · MoneyLotte Verheyden categorizes AI engineering processes into five steps (tracing, monitoring, building data sets, experimenting, evaluating) and recommends starting with the online part (tracing and monitoring) before the offline part.
AI Engineer · Continuously Improving Agents with Langfuse — Annabell Schäfer & Lotte Verheyden, ClickHouse
Report · MoneyLotte Verheyden recommends setting up aggregate metrics to continuously see the overall health of the system over time, covering both cost and quality.
AI Engineer · Continuously Improving Agents with Langfuse — Annabell Schäfer & Lotte Verheyden, ClickHouse
Report · MoneyAnnabell Schäfer demonstrates verifying evaluators by going back to the IT support agent, asking the Bluetooth question again, then disagreeing with the agent (e.g., 'no, there is no such Bluetooth') and checking how it turns out in the traces; the agent confirmed she was right, and the disagreement was captured in traces with the 'Okay, dad, no worries, let's try the other way first' reply.
AI Engineer · Continuously Improving Agents with Langfuse — Annabell Schäfer & Lotte Verheyden, ClickHouse
Principle · MoneyWolfram Ravenwolf explains that changing the dataset makes it a new benchmark, an agent harness version update makes it a different benchmark, and an inference engine software update can change benchmark results without you being able to detect it.
AI Engineer · From Zero to Leaderboard: Agent Evaluation — Wolfram Ravenwolf, Weights & Biases
Principle · MoneyWhen scores are very close, the price/performance perspective may make the cheaper agent the better choice even if another is ranked best. If scores are very close to each other, you would have to do 10 runs or more to be very sure which model is better. But you have to make a choice at some point — pick one model and go with it.
AI Engineer · From Zero to Leaderboard: Agent Evaluation — Wolfram Ravenwolf, Weights & Biases