Report · MoneyOptiver implemented an MCP tool search to reduce context bloat, gave agents the option to have their own CLI wrapping a bunch of the MCPs rather than calling MCPs directly (much more efficient), and built optimizations into the process that helped get AI spend back under control after big frontier model releases of the summer shot spend up.
AI Engineer · How a Trading Firm Ships New AI Models the Same Day - Noyan Tokgozoglu, Optiver
Creator perspective · MoneyWhen re-running evals, run the tool through a proxy around the MCPs requesting a particular snapshot.
AI Engineer · How a Trading Firm Ships New AI Models the Same Day - Noyan Tokgozoglu, Optiver
Report · MoneyTo optimize a skill for a specific repository, write code instructing: learn a skill from the trajectory such that when the coding agent is presented with similar problems, the skill should be helpful. Starting with mini SWE agent with GPT-5 mini (budget constrained), skill optimization took Go repository issue resolution performance from 24% to 93%, an almost 3x jump.
AI Engineer · Beating RL With Reflection: GEPA and Optimize Anything — Lakshya A. Agrawal, GEPA
Principle · MoneyRoman Ugarte recommends collecting a rich set of real user tasks, quantifying them, and tracking week-over-week improvement across those categories of tasks as very important dimensions to making the product work behind the scenes.
Lenny's Podcast · How we built Grok Bot in a month | Roman Ugarte (SpaceXAI)
Principle · MoneyRoman Ugarte recommends a recruiting approach: look at the biggest problems at the company, find who would be the best in the world to solve them, and ruthlessly go after them. Being on the job market is not a precondition for trying to hire someone. He also describes using the bot for sourcing: every morning, go to a conference website, download new PDFs, find new names not yet tracked, add them to a spreadsheet, research them, check for direct connections at the company, and if connected, send a Slack message asking for an introduction.
Lenny's Podcast · How we built Grok Bot in a month | Roman Ugarte (SpaceXAI)
Report · MoneyTejas Bhakta describes anti-patterns where agents make one kernel faster while disabling a bunch of speedups: they may disable CUDA graphs, which can make end-to-end inference 20 times slower, or only test on small context windows, producing kernels that are not viable for real workloads.
AI Engineer · Autoresearch Made Our Models 3x Faster — Tejas Bhakta, Morph
Principle · MoneyPractical tidying moves include removing or consolidating before adding something new, revisiting flows rather than just individual screens, and asking whether something still serves a clear purpose.
NNgroup · UX Debt
Creator perspective · MoneyThe model took 15% for the executive share assumption, but the citation was real while the number was made up by AI; AI also drew the boundaries as 5% to 30%, yet the real number disclosed in the filings was 45%, so AI failed to identify that number from the filing.
AI Engineer · Right Answer, Wrong Reasons: AI Forecasts Costco - Hassan El Mghari & Omar El Mghari, Together AI
ReportWhen richer undo operations were introduced, recovery improved significantly from 0 to 142 out of 143 cases (recovery expressivity).
AI Engineer · Your Coding Agent Can't Undo Its Own Mistakes - Tanmay Sah & Dolly Sah
ReportWhen precise information about what had changed and where was provided, recovery improved from 0 to 38 out of 48 cases (state grounding).
AI Engineer · Your Coding Agent Can't Undo Its Own Mistakes - Tanmay Sah & Dolly Sah
ReportPrompting agents to repair their own recovery failures (verifier-guided repair strategies) recovered none of the 197 naturally occurring failures under the original recovery language.
AI Engineer · Your Coding Agent Can't Undo Its Own Mistakes - Tanmay Sah & Dolly Sah
ReportWith the original recovery language, none of the 197 recovery-defective mutations could be restored; after extending the recovery language, 180 of 197 cases (~92%) were recovered.
AI Engineer · Your Coding Agent Can't Undo Its Own Mistakes - Tanmay Sah & Dolly Sah
ReportUnder normal conditions agents achieved an 84% success rate, but when failures were introduced, only 47% of the normally successful trials recovered successfully.
AI Engineer · Your Coding Agent Can't Undo Its Own Mistakes - Tanmay Sah & Dolly Sah
ReportIn EvoUndo, 600 self-evolution tasks were evaluated; 478 produced capability improvements, but 197 of those improvements had recovery defects — around 42% of capability-positive cases.
AI Engineer · Your Coding Agent Can't Undo Its Own Mistakes - Tanmay Sah & Dolly Sah
Creator perspective · MoneyKepler uses ANN plus BM25 to retrieve an initial set of up to 200 documents, then reciprocal ranking to narrow to about 60 documents, then uses cross-encoders, chunking the query and document together to get results. This allows searching really wide and then narrowing — something LLMs are not good at doing on their own. Kepler frequently throws candidate documents away, deciding not only which source is right but which documents to stop looking at globally.
AI Engineer · AI Can't Make Up a Number It Can't Look Up - Vinoo Ganesh, Kepler
Creator perspective · MoneyIn Kepler's architecture, code writes every number — the model never generates numbers directly. Kepler uses a combination of re-ranking with old-school pointer dereferencing (dot-get methods) to pull things into the system.
AI Engineer · AI Can't Make Up a Number It Can't Look Up - Vinoo Ganesh, Kepler
Report · MoneyWrong numbers or wrong facts come from three sources: retrieving the wrong document entirely; getting the right source but pulling the wrong number out of it (e.g., reading 2023 revenue from a 2025 10-K without confirming the figure corresponds to 2023); and models doing mental math, since models don't do math — they predict the next token.
AI Engineer · AI Can't Make Up a Number It Can't Look Up - Vinoo Ganesh, Kepler
Creator perspective · MindRyo Lu uses a hypothetical: asking an AI to build a life-logging tool may return a finished product in two minutes, with explanations for features like mood-score trend charts, daily reminders to build habits, and weekly reports for reflection. The justifications can sound reasonable enough that you simply accept them with 'oh okay, fine'.
AI Engineer · When We Can Do Anything, Do We Still Know What We Want? — Ryo Lu, ex-Cursor | AI Engineer Shanghai
Report · MoneyRamp took a step back and trained a retrieval / financial question-answering model handed off from frontier orchestrators to do financial QA much faster; they partnered with Prime Intellect to post-train a QAN 3.5 35-billion model, which beat Opus 4.6 (the frontier at the time) on exact task accuracy while serving at high QO-level latencies.
AI Engineer · Our Smaller Model Beat Opus on Finance Tasks - Alexander Shevchenko, Ramp
Report · MoneyAamir Shakir states that using multimodal search reduces LLM token spend by 3x, because most tokens are used to find, parse, and read information.
AI Engineer · Better Search Took This AI Agent From 36% to 98% Accuracy - Aamir Shakir, Mixedbread