34,950 results · Most relevant
Remember that a bar is a claim and a trace is evidence — click any bar on WolfBench to go to the Weave page and inspect all the traces behind that score. You can see all five runs that composed the score, then pick any run to inspect the task name, reward (success or not), and all steps including the initial request, the LLM's first command, its analysis, and each command it sent.
Check the refusal count before trusting a top-ranked model — Fable had 13 refusals and failed the benchmark because it refused so many tests. Dismissing a benchmark because a top model failed without checking the traces is a mistake; the benchmark has to be transparent so you can look at the traces and see what happened, and then it makes sense suddenly.
Duggal recommends connecting all the pulled data into a coding agent — whether Claude Code, Cursor, or Codex — so it can automatically do the debugging work for you.
Setup takes just a couple steps: set up the CLI and skills so the coding agent can pull down traces, understand what's going wrong, look at existing evaluations and annotations, do its own analysis, and come back with fix suggestions. Get your code instrumented so all traces from your agent flow into your chosen platform, like Arize. From within the coding agent you can then pull down traces, build datasets, run experiments, and run evaluations — the full gamut of the experience.
Bostrom proposes a decision criterion: to justify slowing or pausing AI development, the rate at which AI risk is falling must exceed the annual human death rate. If risk falls fast, a few months suffices and long waits are unnecessary; if risk falls slowly, waiting costs more expected life-years than the research gains. Only an intermediate case supports long pauses of many years.
conditional
Everyone has to get good at trying a bunch of stuff with the latest models to figure out what is now possible before competition does. Get to the frontier both by playing with the models and by watching what types of experiences have just become possible. Frontier team members should have many AI subscriptions ($100-200/month on everything) to stay current.
Keep a list of what's 'almost possible' and track when things cross over into possible. When something crosses from almost possible to possible, it feels like a phase shift (like solid to gas or liquid to gas). When you see something become possible, think about how to get a team around it and just go. When something surprising and interesting emerges, the response should be 'we got to just do something with this.'
Gabriel Martinez recommends repeating the human processes his team found themselves using to fight slop: break down the work, manage context, weigh out your options, and make sure the output enters the codebase in a way the team can still reason about.
Creator advice attributed to Gabriel Martinez, endorsed by him.
Laskin advises enterprises to start with closed models at scale, then move to open models by customizing the agentic harness first; fine-tuning comes later once they are sophisticated enough. He warns against attempting a straight-shot sprint from closed models to open-model ownership — enterprises must go through a rental stage first. Enterprises spending $100M+ per year on closed models should start evaluating a more optimal (open/self-hosted) approach, and when hyperscalers can't supply enough compute at good rates, heavy consumers should consider on-prem bare metal with infrastructure providers like Dell.
You can automatically approve and merge PRs that are green, and you can define rules and conditions that control when PRs get automatically approved and merged by the agents.
Design the agent pipeline so that building the pipeline, generating the schema, and pulling in the right context happens where the human is present, since that is where mistakes are cheapest.
Davis Palmy advises not jumping into having the agent take on all this work at once: it's too heavy an undertaking, and teams won't have time to build trust in the system. Start with narrow and verifiable tasks like incident triage — let the agent read a Sentry alert, pull traces and other context, and post in Slack — while engineers review the agent's output and decide what to do. As more correct diagnoses roll in, the team builds trust and understanding of that pillar. Each step in the software factory needs careful ownership and monitoring before you even think about letting the pipeline loop. Once you can tie the system's output back to the input signal, that's when you unlock organizational self-improvement.
Teams building a software factory need to first build trust and understanding, which is why the rollout should be incremental. Don't jump into automating all pillars at once — it's too heavy an undertaking and teams won't have time to build trust. Start with narrow, verifiable tasks like incident triage: let the agent read a Sentry alert, pull traces and context, then post in Slack, keeping output reviewable so engineers can decide what to do. As correct diagnoses roll in, the team builds trust in the pillar.
Placing an animated frame inside another frame keeps the animation working while allowing the outer frame to be animated too: set the transform on a frame to rotate it, make the rotation go one way and then back, and loop it so it animates continuously; a container frame can also be scaled to grow bigger and then smaller again.
Matt Owens says the secret to working with AI is deliberate practice, a specific outcome in mind, and giving yourself the time to do it right. When using AI, be deliberate and have an outcome in mind — that's been the most productive way of using it. At Athletics they use Claude Code connected to Figma MCP only when they have a specific idea and outcome in mind. The question with AI is intent: automating a workflow like a new business pipeline is fine, but trying to make a presentation really sick and amazing without touching it will never work.
Tempo sprinting at about 90% of max speed is easy to understand, do, and verify, unlike maximum velocity running. For young people, up to 150 meters a week of 90% max speed running can be interesting (he has no science to support it). Players who get less than ~120-150m of 90% speed work seem to have problems more often, and players who get more need to be really adapted or it's too much. As a starting point, use 120 meters of tempo sprinting per week at 90% of maximum speed.
he explicitly says he has no science to support the 150m figure; the 120-150m range is from his observations
Use MCP as a distribution layer for skills: instead of installing skills locally, create a remote MCP server that becomes the source of truth across your whole company. Clients can connect, search for the right skills, and the server can enforce policy so everyone gets the best version of the skill depending on the query and their assigned role. This fixes the dev-centric and client-support problems because it's unified and works in almost every client, so distributing skills to non-technical people is essentially solved. A centralized MCP skill API also gives a VP of AI one place to govern what knowledge gets shared.
Eli Cohen recommends running pen testing on every code change — every PR, every delta — instead of only two or three times a year, and telling developers what they need to fix and how to do it differently, rather than handing over an undifferentiated report.
Eli Cohen argues you need to prepare your production system for the unknown-unknowns, which requires a new paradigm: AI pen testing capabilities, plus agent red teaming — simulating multi-step attacks such as prompt injection, data exfiltration, and goal hijacking — combined with dynamic testing, which is good at identifying authorization issues.
To build regulatory compliance from scratch: read every single regulation — federal and state — refactor the entire legal code base into an abstract framework, put it into your architecture, and then test it. The regulatory licensing process ultimately requires aggregating expertise across a very large set of domains.
anecdotal
Choose sources → to get a library of your own and a daily brief.