Principle · MoneyA prospect may agree the product solves pain points and offers great value, but still won't drop their own priorities to purchase — a key reason value-based selling fails.
Lenny's Podcast · How to find product-market fit (but, actually) | Rob Snyder (Author, The Power of Pull)
Principle · MoneyRob Snyder says the case study factory is the final step that takes you from things starting to work to creating a repeatable system that helps you grow, and he thinks it's the most important thing in all of this.
Lenny's Podcast · How to find product-market fit (but, actually) | Rob Snyder (Author, The Power of Pull)
Principle · MoneyRob Snyder says all delivery is about figuring out what X is and how to design a bullet train so every customer gets there.
Lenny's Podcast · How to find product-market fit (but, actually) | Rob Snyder (Author, The Power of Pull)
Principle · MoneyThe urgency marker of genuine pull is the prospect expressing immediate need, not future intent.
Lenny's Podcast · How to find product-market fit (but, actually) | Rob Snyder (Author, The Power of Pull)
Principle · MoneyTeams use P-U-L-L as a fill-in template for a target customer, making the abstract framework concrete and repeatable.
Lenny's Podcast · How to find product-market fit (but, actually) | Rob Snyder (Author, The Power of Pull)
Principle · MoneyRob Snyder explains that you don't try to sell the prospect immediately in outreach — you just want to get a call, and on the call you'll see if they have pull. If the prospect has pull, they will turn the call into a sales call themselves; if they don't, nothing the founder does will change that.
None stated.
Lenny's Podcast · How to find product-market fit (but, actually) | Rob Snyder (Author, The Power of Pull)
Principle · MoneyRob Snyder advises that your outreach message doesn't need to convince them to buy your product — it just needs to get them to talk with you. He recommends sending a short message that shares something interesting about yourself to cause them to meet with you, rather than pitching the product in the message, and keeping cold outreach to a sentence or two.
Lenny's Podcast · How to find product-market fit (but, actually) | Rob Snyder (Author, The Power of Pull)
Principle · MoneyRob Snyder describes using the PULL framework as a template — 'like a Mad Libs thing where it's just like P-U-L-L and you fill it out' — for a target prospect. Many people keep the framework in a doc during the sales call and fill it out live, while Rob himself keeps it in his mind during calls.
Lenny's Podcast · How to find product-market fit (but, actually) | Rob Snyder (Author, The Power of Pull)
Creator perspective · MoneyLLM as a judge takes a secondary LLM and asks it to grade some aspect of your agent, producing a label and an explanation; explanations give you the reasoning behind a label so you don't have to take it at face value. Decision models are a newer, cheap and fast eval type returning a probability and an output without an explanation. Code-based evals are great for deterministic tests like checking if a word is in an output or making sure something is in JSON; ROUGE and BLEU scores are popular examples. Annotations are a form of evaluation with golden truths on golden datasets, sometimes synthetic. Agent as a judge is a newer type (last ~6 months) where you take a full agent and evaluate something in a different agent, good for super complex success criteria. User feedback and businessmetrics
AI Engineer · How to Write Your First AI Eval - SallyAnn DeLucia, Arize AI
Principle · MoneyThe trick to doing well on filtered vector search with permissions is to do both pre-filtering and post-filtering, combined into what they call native filtering.
AI Engineer · Can Your AI Analyst Keep a Secret? - Aaron Linsky, Bridgewater & Nikhil Benesch, turbopuffer
Report · MoneyEvery number in Kepler is clickable, showing not only the 10K, 10Q, 8K, or investor presentation it came from, but the actual calculations done behind the scenes. This is what makes numbers truly citable: showing how you got it and where you got it from.
AI Engineer · Do Not Let Models Invent Numbers - Vinoo Ganesh, Kepler
Creator perspective · MoneyLLMs don't actually read your documents. They chunk things together, do a semantic search, and retrieve information that may or may not be relevant. This is why attaching sources after the fact is problematic.
AI Engineer · Do Not Let Models Invent Numbers - Vinoo Ganesh, Kepler
Creator perspective · MoneyThe model itself can make up things from its own corpus or training set that are really difficult to work around, especially with historical data. Looking up a revenue number from 2005 likely lives within the model's training corpus. If retrieved from the training corpus, you can't guarantee the number was pulled correctly from the right year and scope, and can't trace it back. Models like saying stuff after the fact — that is what they do.
AI Engineer · Do Not Let Models Invent Numbers - Vinoo Ganesh, Kepler
Creator perspective · MoneyMost people in the industry right now are wrapping a lot of financial research with an LLM, saying as long as things are ragged properly, they should be good to go. Hooking up Claude, OpenAI, or any of these models to just a native RAG tool or a vector database actually isn't enough — this is the scariest thing if you live in financial analytics. If you're looking at anything that requires numerical accuracy and auditability, you actually can't trust regular RAG. There are really cool things you can do if you're looking at just semantic search, and that's it.
AI Engineer · Do Not Let Models Invent Numbers - Vinoo Ganesh, Kepler
Creator perspective · MoneyAnytime the model is forced to do math or can't reason through something, Kepler makes that impossible. By having the post-processing step, the model can't write something it can't cite — the platform is structurally incapable of producing hallucinated numbers. The model can still hallucinate, and it definitively does — the structural guarantee applies to the platform, not the model.
AI Engineer · Do Not Let Models Invent Numbers - Vinoo Ganesh, Kepler
Report · MoneyTenant isolation is critically important in Kepler's architecture to prevent one desk from seeing another desk's research (e.g., Citadel Desk 1 seeing Citadel Desk 2's research). Every builder adds a new filter, building up complex filters over time to prevent data leaking inadvertently across different sources. Every hit is re-authorized in Postgres at the document level.
AI Engineer · Do Not Let Models Invent Numbers - Vinoo Ganesh, Kepler
Creator perspectiveSuyeol Yun argues that existing benchmarks use short, simple tasks that only capture fragments of the full unit of analyst work, so they fail to test real capability.
AI Engineer · Why Frontier Models Fail as Buy-Side Analysts - Janie Gu, Google DeepMind & Suyeol Yun, LinqAlpha
Creator perspectiveEach benchmark harness ships with its own default reasoning effort, so using defaults without configuration compares models at different reasoning efforts (e.g., one on high, one on low). Benchmark builders should configure reasoning effort explicitly to ensure a head-to-head comparison; many original benchmark papers did not, and their scores were flawed.
AI Engineer · Why Frontier Models Fail as Buy-Side Analysts - Janie Gu, Google DeepMind & Suyeol Yun, LinqAlpha
Creator perspectiveBenchmark verification is broken because labs run their own harnesses, report their own scores, and build self-reported implementations that produce wildly different numbers. These differences are mostly caused by infrastructure differences (scaffolding, tool-calling, input serialization), not actual differences in model quality. Kaggle spent two months reproducing 65 different industry benchmarks, recreating and running all of them and reading every trajectory trace. Running the exact same benchmark multiple times on the same infrastructure produced score variance of up to plus or minus 20% across all 65 benchmarks, which reflects instability in the harness or the benchmark design itself.
AI Engineer · Why Frontier Models Fail as Buy-Side Analysts - Janie Gu, Google DeepMind & Suyeol Yun, LinqAlpha
Creator perspective · MoneyJay Sahnan advocates building a single agent that can interact with and act across thousands of different portals, rather than maintaining per-portal scripts.
AI Engineer · How to Build Financial Agents That Work Across the Web - Jay Sahnan & Arik Bird, Browserbase