35,595 results · Most relevant
Mario Martone says these conversations impact housing and healthcare and are consequential — mistakes actually impact people's lives. Building evaluations that let about 10 people gather an understanding every day of how the conversations are going is a really hard task, which the team has largely cracked.
Mario Martone says the team built a way to experiment in a closed loop where something happens and they can test it internally, so they can see the performance of every model on their own internal benchmarks versus external ones that cannot be trusted. The team has some Kubernetes loop for experiments but is working on automating it fully.
The 2023 paper 'Thousands of AI Authors on the Future of AI' surveyed 1,200 AI professionals about when certain enhancements will happen. The extinction-probability questions were about T-infinity, all time, and the next 100 years — none of that is in the next 10 years, yet this is the paper most people quote. The narrator adds there is very little signal in asking 1,200 AI professionals hard forecasting questions — it's just a really hard question — and that with infinity time, everything happens, so he is not sure that timeframe framing is useful.
The narrative around the Hugging Face incident was that the AI went haywire, went rogue, escaped from the servers, made copies of itself, and is now solving math and hacking things — which is not what happened. The Slashdot headline 'OpenAI says its AI models acted on its own' misrepresents what OpenAI actually said; OpenAI never said that in one statement. OpenAI released a 60-page technical report, the OpenAI Hugging Face Incident, that can be read.
Simple methods such as trying admin-admin as a default credential actually work to get you into legacy systems. Failing to change the admin-admin password is why almost all social security numbers have been breached.
Turn-based auto routing must be locked down to at most two to three models, otherwise the cache will be extremely inefficient. If you put 30 models into turn-based routing and it keeps changing the model every turn, your low cache hit rate will start to wreck your optimization. Turn-based routing between JRM 5.3 and JRM 5.3 flash makes sense as a two-model setup. An audience member found that starting with request-level routing didn't make sense, and noted two models tend to be okay for turn-based routing while task and organization level routing makes the most sense.
Oliver Kharraz describes the Blue Shield of California partnership: on their own member directory, members can now book a doctor's appointment directly, powered by ZocDoc. He frames payer partnerships as a natural integration point where payers can serve patients better, be there when a patient decides to get care, and keep patients out of high-cost modalities.
In Claude Code, dynamic workflows let you type a request like 'hey Claude, go and do X, Y, and Z, use UltraCode' and it spins up a complicated web of agents that has fully orchestrated itself. Instead of hand-drawing a classify-and-act or fan-out-and-synthesize topology with each agent and its sequencing specified, you just tell Claude to use UltraCode to achieve its goal. Under the hood it uses a JavaScript engine supporting MapReduce-esque activities deep within Claude Code, and Claude knows how to orchestrate itself for harder work based on internal model training and its own general capabilities.
Dynamic workflows shipped within Claude Managed Agents and make it much easier to get power out of increasingly capable models. A lead agent creates a plan for how to accomplish a task and can automatically fan out to its own set of agents, spinning them up and down based on its own capability. Across multiple phases, agents accomplish discrete, verifiable units of work without you managing interconnections — for example, phase one might spin up Explorer agents to read files, and phase two might identify something about those files.
Users are building their own complicated multi-agent harnesses to solve single-agent problems, but hand-rolling orchestration is non-trivial: you must handle container restarts (e.g., one of 100 agents dies two hours in — how do you pick up where it left off and understand the state of the whole topology?), a single coordinator keeping track of even 60 agents hits a cap because, like Linus, there's a limit to how much one agent can keep in its head, and you have to figure out how to inspect the graph of agents — poke at each node, run everything through the coordinator, or otherwise understand the state of the graph at any point.
Developer tools like Clerk and Resend blew up when AI apps started recommending them — the apps themselves drove adoption without the vendors knowing it was happening. The lesson: getting AI agents to recommend your product can drive explosive growth, so optimize for agent recommendations.
None; cited as observed examples.
For one piece of data, it might come from 20 different sources and there's only one that's actually right, so the Rowspace team spends time with portfolio management and finance teams to understand calculations, identify correct data, and override other sources. Trust in the AI system comes from having your own team go in and validate that the information is correct and can be relied on, sourced from trade order management, performance systems, Bloomberg, or pricing indexes.
Redding Ridge partnered with Rowspace to have the vendor sit with teams, understand the actual calculations and information, identify the golden source of truth, and enhance AI engines to repeat that. With Rowspace, portfolio construction that took days now takes minutes, allowing PMs to build a portfolio to 80-90% completion and then use judgment to refine the last 10%. Warehouse purchase activity trade recs that historically took many hours per week for two to three individuals are now automated and repeatable in the span of a minute, with the team QAing and checking the output each day as a rinse-and-repeat routine process.
To prevent memorization, the team clustered building blocks by similarity and excluded entire clusters from overlapping between test and train sets. When they did this, model performance dropped — evidence that prior performance relied partly on memorization or cheat modes.
Because DNA-encoded libraries (DELs) are combinatorial, models can cheat by learning each building block independently and decomposing predictions into the sum of three building blocks. When train and test sets share building blocks, performance is better — which the team suspects is memorization. Even similar-but-not-identical building blocks can be close enough for memorization or other cheat modes to convert into apparent predictive power.
suspected cause, not confirmed
Quigley reports Leash Bio went directly from chemical library hits to good PK/PD in a mouse in six months with very few med-chem cycles. He attributes this partly to optimizing the chemical library for drug-likeness from the start — a good starting point compounds downstream speed — and recommends optimizing library design for drug-likeness at the start of a program.
reported as Leash's own result and attribution
Hermes sees cross-attention, so you can interpret where Hermes thinks the binding pocket is. When they first examined attention maps, they expected the model to look at binding pockets, and a more measured hope was that it would look at all residues. Instead, Hermes was looking at really short sequences of amino acids away from the functional part of the protein and away from conserved regions. In molecular evolution, a stretch of a protein that does a good job often gets shuffled among other proteins, creating a conserved region called a conserved protein domain; biologists use these to infer protein families, relationships, and functions. The non-conserved short stretches Hermes attends to have a lot more variation and are typically thought not to do much. Hermes was effectively comingup
In a hypothetical document-exception assistant, security asks which environment will host the deployment, data owners ask who agreed to the use of the documents, and risk (RISC) asks who reviews a wrong exception before it reaches the next system. Each of these questions can change what the team is allowed to build.
Paden Gayle describes growing the hypothetical system one step at a time. First, it summarizes a document the analyst already has — still confirm whether existing access covers this use. Next, it reads from a second system, creating another data path somebody needs to approve. Next, it drafts a response a customer will see, requiring someone to check the content and review which controls apply. Finally, it sends a response automatically, requiring an owner for that action, a plan for what happens if it sends something wrong, and a way to stop and recover it. Those are the four different engineering scopes.
Three to four months in advance is the sweet spot for booking travel to get good flight prices; travel and lodging typically consume about 40% of the kickoff budget, but booking only two months or a month and a half out pushes travel costs toward 60% of the full kickoff budget, which eliminates the ability to fund other fun kickoff activities.
Choose sources → to get a library of your own and a daily brief.