Treat wide AI estimate ranges as failures when the recalculated actual falls far outside them
When multiple AI models produce wide estimate ranges (e.g., $0.5B to $6B) around a recalculated actual (e.g., $450M), treat all of them as failed.
35,595 results · Most relevant
When multiple AI models produce wide estimate ranges (e.g., $0.5B to $6B) around a recalculated actual (e.g., $450M), treat all of them as failed.
Security failure one: forgetting to filter the tenant means Citadel could search all of Millennium's documents. Failure two: a caller claims to be another firm, impersonating an entity to query documents. Failure three: search returns information that's out of date — a problem seen globally, even in Google search results. Failure four: a direct read skips the search rules, so you're not querying the full corpus — there's no allow list, and you only get back the single document you searched. Failure five: hitting an index that isn't populated. People building these systems themselves often miss the security piece associated with querying information.
Kepler's architecture assigns pointers to unique identifiers (facts, like a number in a 10-K) in a giant preprocessing and contextualization step. The preprocessing step converts unstructured data into a structured data store. Kepler stores information deterministically that can be stored deterministically, and uses hybrid storage (TurboPuffer plus other mechanisms) for information that can't be. Kepler partners with TurboPuffer, which the speaker describes as awesome.
The way software is now being built in startups is spec-driven development: write very in-depth specifications, get clear prompts or specs, and build sub-agents at the right parts of the codebase.
Live, as tokens move forward during inference, filters and references are given to an LLM that has to resolve them. A registry is used where the sources themselves are looked up, and information is pulled from that registry to give traceability. When there is no match, the number is stripped — a very aggressive thing to do. A post-processing step also tries citation repair: if information can't be reliably traced, an on-the-fly agentic determination says this information came from this source; in certain spaces Kepler does do on-the-fly agentic citation repair.
When a number cannot be matched to a source, Kepler strips it. The speaker acknowledges this is aggressive, but it ensures the platform cannot produce hallucinated numbers. The tradeoff is that the platform cannot be as magically powerful as people want — stripping unverifiable numbers is the price of trustworthiness for numerical work.
Browserbase forked Chromium and produced a verified browser that they run in the data center, making many changes to the Chromium binary. The first change was removing the automation flag: when driving a headless browser through automation, the webdriver flag is set to true, and they swapped it to false everywhere. The second change was aligning on identity: instead of being called headless Chrome, the browser is called Chrome, a change that goes deep into the Chromium code in many different places. The third change was keeping the browser process alive: instead of killing Chromium instantly when the task is done, they leave Chromium running and close the windows out.
Dr. Scott Forbes reports that creatine can help with cognitive performance after sleep deprivation, citing roughly four or five studies showing brain improvements particularly when the brain is stressed. One study showed a single dose got creatine into the brain and helped preserve cognitive function in sleep-deprived participants; sleep deprivation studies often use severe deprivation of around 21 hours.
Consequential creator advice requires independent verification before Today can promote it.
Dr. Scott Forbes describes emerging research showing creatine can get into the brain and help memory and cognitive function, particularly in older adults or in stressed situations. Cognitive domains shown to improve include overall cognition, processing speed, memory (short-term and recall), and attention span.
hedged
Coach Kyle advises asking for a woman's Instagram when ending an approach on public transit — before the bus takes off — because the transit window is limited, and warns that failing to ask, or failing to message or text her afterward, means the close goes nowhere.
There is a financing mismatch: hardware financiers want very long terms, but customers can only afford or don't want long-term financing beyond 36 months. CoreWeave has quoted a median prepayment of 15% to 25% up front on a $100M TCV cluster — meaning $15M to $25M a startup has to put up front, which might be their entire round. Illustrative scenario: raising $40M, needing a $100M contract over three years, and putting down a 30% prepayment leaves only $10M to run the business, creating a step function where contracted capacity exceeds demand. When contracted capacity is tied to equity rounds, a startup may have to cap out and turn away customers until its next equity round funds another prepayment for more capacity.
Carmen Li describes how last year the implied forward curve for GPU compute was completely downward sloping, meaning longer-term contracts were cheaper per chip per hour. Starting early this year, the whole curve shifted up and became contango, implying people assume prices will rise down the line and aren't getting enough discount for longer-term contracts. The term structure has now flattened, meaning people will either get penalized or awarded by a longer-term contract. For B200, last year the curve was contango and this year the whole curve shifted up — every single term is more expensive. In platform rates, people similarly think the price is going to go up.
Market observations and interpretations from the speaker; some atoms were flagged as possible numeric errors.
Ben Horowitz identifies reward hacking — where an AI does something it wasn't told to do as a means to get its reward — as probably the biggest safety problem in AI, and suggests it would be exciting if every PhD student, professor, and undergraduate in every university worked on ending it.
asserted
The episode recaps Sam McKenna's top four takeaways: one, three steps every time you book a meeting (send the thanks, send an invite with your name first in the subject, and connect on LinkedIn); two, show a prospect that you know them in the first five minutes of a call; three, use typically questions when prospects are hesitant to share, since it allows them to latch on to an answer; and four, when you get to a problem, tell a story that sounds like: you make me think of a customer where XYZ was the problem, really similar to you, and ABC is what we did to solve it.
He realized that as the one guy making TikTok Shop videos all day, if he could make 10 other people six figures a year at 10% commission, that would mean $10 million a year just from them.
illustrative math, not a measured outcome
The guest built a model showing that as creative uploads increase, revenue goes up and CAC goes down at predictable rates — e.g., going from 1,000 to 2,000 ads has expected revenue and CAC reduction. He got up to 400-500 creatives a month and still wasn't seeing CAC reduction until Hudson told him he wasn't uploading anywhere near enough ads and needed to double, triple, or 5X that.
The guest describes the spec writer's job: take what is known about the product strategy, design psychology, users, and user research and compact it into a very efficient prompt. User research is still very necessary and in some ways almost more necessary in AI-driven development.
Reducto released harnesses around its model work, with Deep Extract as a leading example. Adit Abraham says it is the first time he finds agents better than humans at structured extraction on many tests, since human annotators miss content, drop rows, or fill in only part of the information. A harness where the agent extracts, corrects itself, and reviews its own outputs can tip the balance toward perfect accuracy.
based on the speaker's own evaluation on 'many tests'; not an independent benchmark
Adit Abraham describes Reducto's original two-part approach: use traditional computer vision to preserve structure and use VLMs as a correction loop, given accuracy-cost and latency trade-offs. Over time, as multimodality advanced, they built their own VLMs for document tasks, culminating in R1 — a unified foundation model with better accuracy, higher speed, and lower cost than off-the-shelf models, still providing bounding boxes and consistency.
Adit Abraham tells of a healthcare customer where an OCR misread of 'pain' instead of 'chest pain' got a patient's claim rejected because the doctor's search showed no relevant history. He cites the stat that in 80% of Medicare prior-auth appeals, the original decision was overturned, indicating systemic inefficiency in how the original decision was made, and that millions of Medicare claims are rejected every year for prior auth. Medical patient packets can run to tens of thousands of pages, making them impossible for even the best doctors to fully review, and human reviewers miss critical details.
Choose sources → to get a library of your own and a daily brief.