34,785 results · Most relevant
An MCP server lacking hard tenant isolation allowed users to access any other user's data, and the flaw ran unnoticed for months. The speaker recommends hard tenant isolation at the AWS-account/GCP-project level, tightly scoped workload identity, provenance tagging, and audit trails.
Andrey Rybka states that if a system combines untrusted input, sensitive data, and external action, it needs human sign-off.
A sandbox blocked web access but forgot DNS, so a hidden prompt instructed the agent to smuggle data out through DNS lookups, which were readable in DNS logs. The speaker recommends default-deny egress including DNS and placing egress controls outside the sandbox.
Figure out who your avatar is: ideally a market that is growing, in pain, has a problem to solve, can afford your stuff, and is easy to find. The 'easy to find' criterion mattered less in the past because algorithms were worse — you had to put out a beacon saying 'this is for you.' Now AI knows who content is for and serves it directly.
There are two ways to win at building credibility: accomplish something notable, or do so much work that people want to bargain on that work. Credibility can come from volume of work or volume of accomplishments — either works. If you haven't accomplished anything, show the work: do a thousand outreach attempts or knock on 10,000 doors and document it.
Farza explains he doesn't care about view counts on launch videos — he cares about actual numbers like downloads and retention. After the Clicky video, 7,000 people downloaded the product, and about 100 of them kept using it every day for five days straight. That retention — not virality — inspired him to keep going.
Because many variables are involved, evaluate the closest configuration possible that reflects the actual work and environment you will be measuring. A law firm, for example, should not look at Terminal Bench results unless it is incorporating coding agents with its developers, because the benchmark configuration must reflect the actual environment.
Tanmay Sah explains that each core dimension is divided into sub-dimensions, forming a framework of 13 sub-dimensions and 3 core questions. Technical strength includes data quality, task design fit, predictive validity/outcome analysis, fairness checks, failure-mode analysis, and explainability to regulators. Deployment includes decision authority given to the AI, human-in-the-loop effectiveness, exposure control (who uses the system), and monitoring controls. Business impact includes financial impact, legal consequences, and scale (how many users are affected).
one atom flagged possible_numeric_error on the count of technical-strength sub-dimensions; framework counts are as stated by the guest
The guest endorses 'quantity breeds quality' in creative volume, but says there needs to be a quality standard when approving. Every submitted video is watched start to finish (now by his team), and if it doesn't genuinely have all the parts and components of a great ad that could do well, it's denied. A quality ad must have a hook, body, call to action, and unique value propositions, and must meet production quality standards: no echoing, good lighting, good editing — if it doesn't have any of that, it's denied; if it's great, it's approved.
The guest shares his saying: it's a lot easier to teach an actor how to act like an athlete than it is to teach an athlete to act like an actor. Instead of going for someone you think could be good, you have to find creators who already know how to act — who have already started making content and know the basics. TikTok is still an amazing place to find amazing creators, even today with Tribe's 280,000 creators. Don't go for creators making a ton of money — go for ones making a couple thousand a month, so they're excited and your brand can be the life-changing opportunity for them. After finding the right creators, sell them on why your brand is the best opportunity.
Technique 4: change things that should not affect the result — the supplier's name, the supplier's address, reordering clauses — and verify the number of flags stays the same; also run the contract again and again to check output consistency. In the invariance technique, everything can be checked using code — no LLM-as-judge needed; the team stays away from LLM-as-judge as much as possible because it's a very hard task (as the previous speaker showed).
Technique 3: take an example of a contract that has been signed, remove a clause, and the agent is supposed to say 'Hey, there's something missing here.' The missing-clause technique is very hard because different suppliers have different templates and content, so knowing what should be in which contract requires thinking; these were mixed with existing contracts that should not show any flags. The missing-clause approach is the most powerful approach they had: it generates a lot of contracts, and it revealed the agents were missing most of the clauses that had been removed — a very strong benchmark. When an agent is missing most of the clauses that have been removed, that finding itself is a very strong benchmark result.
Technique 5: turn an example into many using the concept of templates — it's the most powerful technique but very hard; Air Canada's experts have different kinds of templates with blanks that should be filled by the expert. Use expert templates with blanks as the basis for generating synthetic test contracts. Generate an array of fill values for each template blank instead of filling blanks manually, enabling generation of many contracts. Deliberately generate some tuples that break rules and some that do not, so you know exactly what should be flagged as a breach. Check template-generated contract evaluations with code, without involving any LLM. Reuse templates that experts already use in their normal workflow to generate test contracts, reducing friction. Expect roughly a four-week
Technique 2: start from rules instead of creating an agent that provides flags — take a known rule (e.g., a cap above $1 million that every Air Canada contract should follow), create a clause that breaks the rule, generate the contract around it, and check the agent finds the flag back. The rule-based technique's problem is that you need to know the rules; coverage of everything cannot be guaranteed, so the team also came up with their own rules. Using rules from experts and documentation plus their own rules, the team generated 275 tests where they know the test can run and know the answer. When you know a specific rule, create a clause that breaks the rule, generate the contract around it, and test whether the agent finds the flag. When you have limited access to domain experts, use
The biggest impact of AI on the speaker's content is especially with captions — their captions now are so different from before because the AI can tell them which hook types aren't working for interviews and suggest trying different styles, then report which performed well and which didn't. To make A/B testing more apples-to-apples: take a longer video, create several cut-downs, test different caption experiments across them, and use the API to compare engagement, since the base content is more similar.
Personal experience plus a recommended method.
Adit Abraham argues that if you have a harness where the agent is not only extracting information but also correcting itself and reviewing its outputs, you can achieve higher accuracy than human annotation — he calls this the first time agents can be better than humans at structured extraction (via Reducto's Deep Extract). Human-annotated ground truth often contains errors (missing content, dropped rows, lazy partial fills) that go undetected because there's no comparison point.
The speaker recommends comparing a creative's organic views to your account's norm: if a post performs well relative to your norm, that is the creative to put ad money behind, because content cannot get views unless the creative is relevant. He claims this should save an extraordinary amount of money, noting many people waste money on social ads whose creative will not convert.
GaryVee advises leaders to empower people underneath them: instead of suppressing a direct report with bigger natural strengths in parts of the job, become an accelerator of them. He frames suppression as an anti-pattern and suggests auditing whether you are a suppressor or an accelerator.
GaryVee describes his practice of 'kind candor' — putting the word kind in front of candor — which he says completely reactivated his organization. He observed his executives using radical candor as a weapon to suppress people or be jerks with their words, and advises not using candor as an excuse to be a jerk. He notes most companies need kind candor for the kind part, while his organization weirdly needed it for the candor part.
GaryVee describes his operating model: he works for the 2,800 people his company employs — they don't work for him — and he asks how he can bring them value, believing that serving their value first gets him everything he wants.
Use saved training trajectories and a controlled intervention to explain a failure, separating equilibrium theory from observed empirical stability.
Demonstrate all 3 repaired bugs and defend why a passed gradient check is necessary but not sufficient for generalization.
Use the tractable fixture and posterior predictive results to separate approximation, sampling and model errors; justify one revised modeling assumption.
Choose sources → to get a library of your own and a daily brief.
Map each implemented component to the original paper and explain 2 reduced-model failure modes without claiming full-scale reproduction.
Use calibration and shift measurements to justify a monitoring or abstention threshold, and give an explicit confounding counterexample to causal feature-attribution claims.
Defend a causal claim or refusal using the DAG, numerical estimates and a counterexample showing why predictive model fit cannot establish identification.
