33,339 results · Most relevant
Gary Vaynerchuk describes his '1585' system: involve yourself in the first 15% of a project to ideate and contribute strategy, let the team completely own the next 80%, then come back in for the final 5% to add a finishing touch.
When a GitHub issue comes in, write a Prose program for how the agent will try to address it — as complex or simple as you want, including multi-agent workflows, specific skills, or specific models. Use pull requests for event-driven adversarial review of agent-built code, letting agents receive back pressure from the app itself. From the trigger you triage workflows, so you can build out code factories from creating standing jobs of the Prose programs. The agent running cannot consider its job complete until its developer agents have taken the reviewer feedback into account.
Al Chen endorses a simple disaster recovery pattern for ClickHouse: run ClickHouse in a primary region, a job that backs up to a bucket, and another job that restores that bucket to a DR region, where you might have multiple pods with replicas of the data. The procedure relies on ClickHouse backup, an open source project maintained by Altinity (itself a competitor to ClickHouse for managing data failover), which shares logs and a status table of whether backup and restore happened. As an extra step, check row counts between tables and run a check that replicas within the region have converged.
endorsed
Al Chen endorses a flow for handling judge reasoning safely: let the judge read the raw trace data but do not persist the judge's reasoning; the score always persists through the UI because you want it for evals. In application code, a check tells the agent whether the record contains sensitive information — if it does, suppress that reasoning from the UI. Most banks he works with want to see the LLM-as-judge's reasoning on sensitive information, so an extra upgrade is a second LLM-as-judge whose input is redacted of sensitive information, giving confidence its output is clean and can show up in the UI.
endorsed
Al Chen describes a locally coherent method for verifying CVE remediation with Harbor: instruct the agent that version 2.6.2 of a package supposedly remediates the CVE and ask it to write an exploit based on the older version to see if the new version still has that exploit; then have the agent write a test to see if that exploit actually passed; and run many Docker containers or Modal or Daytona sandboxes in parallel to get many trials of how the agent performed.
described
A method for automated quality gates: anchor experiments on datasets, set up CI so any agent or prompt change triggers a GitHub CI job running evals against the dataset, define minimum score thresholds per evaluator, and treat failing scores as a block on production promotion.
A method for assembling evaluation datasets: create use-case-specific datasets (including red-teaming sets for prompt injection), source items from team spreadsheets, production traces, and curated good examples, and keep domain experts in the loop defining golden datasets.
A method for scoring agent outputs: extract common categories from domain expert feedback to define dimensions of quality, build a library of evaluators that automatically score incoming traces (optionally in real time from the client application), and blend LLM-as-a-judge with deterministic checks — relying 100% on LLM judges risks inaccurate outputs.
A method for getting domain expert feedback: build an annotation queue routing agent outputs to experts, give them simple scores (yes/no, 1-10), allow free-text comments, and keep the process very simple since experts have day jobs — ask for about an hour of their time on a curated batch.
Set up a Sales Navigator search for executives who leave your current customers and join accounts in your territory, even if you don't know them, because they've likely had a good experience with your company. Move fast on these alerts — speed to lead matters before competitors reach out — and existing customers have higher odds of taking you with them to their new company.
Every time a deal closes, save all stakeholders — procurement, legal, champions, moles, skeptics — into a LinkedIn list (not a saved search) so you get alerted when and where they change jobs. The list tracks when a person leaves a company regardless of destination, not just moves into your territory.
Block time on the calendar to work alerts or it won't get done — a daily triage in the morning (when it's quiet, before Slack gets busy) or before closing out the day. During the block, check alert reports and take action on the results. Setting up alerts without a way to work them leads to forgetting, no booked meetings, and sadness.
When a profile viewer is tied to an open opportunity, reach out to capitalize on the data, create a conversation, and possibly accelerate or expand the opportunity — a fantastic way to multi-thread — without calling attention to their tie to the deal. For an unknown viewer with no existing opportunity, send a connection request framing the specific challenge you solve for their industry peers, or spark dialogue by asking what made them swing by. Don't sit quietly waiting for them to go inbound.
Wisedocs' baseline preprocessing uses OCR and images to extract information from underlying records; a series of models (NLP, extraction, VLLMs, entity extraction) then structures the data; a summary layer sits on top and is human-verified. The abstraction pattern concentrates information density as you go up the stack, verified at various points with models and human intervention.
The baseline data-structuring approach uses OCR and images to extract information from the underlying records; each step of the AI data-extraction pipeline must be calibrated (build good evals, calibrate judges) because incorrect upstream data used downstream gains nothing; experts who can verify the information must work across teams; with good claim-level extraction you can flag conflicting diagnoses, deduplicate, and search for missing records; hallucinations show up when people assume something is in a medical record but it's not.
Wisedocs built the MLCR benchmark with Artificial Analysis to measure AI effectiveness on domain-specific extraction: tiered question difficulty from simple (when did the injury happen, what was the injury) to harder (how many surgeries, patient capacity); an open-source dataset for the easy tiers plus a private hosted set to stop benchmark-maxing; measurement on three axes — conciseness, accuracy, completeness — against a human expert-validated answer, with triple-judge verification; calibration methods were described in a blog post and follow-up.
Consequential creator advice requires independent verification before Today can promote it.
Wisedocs structures benchmark questions in six tiers from simple factual extraction to complex tasks, releasing an open-source dataset for the first three tiers while keeping a private hosted set (with Artificial Analysis) to stop bench-maxing. AI answers are measured on conciseness, accuracy, and completeness against a human expert-validated answer, with triple judge verification, and each step of the extraction process is kept well calibrated.
Consequential creator advice requires independent verification before Today can promote it.
Run the judge against the humans as an experiment: the experiment takes a dataset, a task to run on every example (the judge), and evaluators to score the results (checking whether the judge's label matches the human's). Only run the meta-evaluation on the dev split, not the test split. Run the meta-evaluation yourself with the command 'npm run meta'. The rubric file 'evals actionability.ts' takes three or four minutes to run.
A multi-part method for constructing a rubric: mark data clearly with XML tags, keep boundaries between input/output/instructions, mark examples as examples, show examples to clarify criteria, test with and without examples, and constrain the judge's answer to a fixed set of options.
The eval workflow being built: trace your agent, read what it did, write evals for what matters, check the evals can be trusted, change the agent, prove the change worked, then lock it in with CI. Lock in agent fixes by putting the change into CI so that it stays fixed.
Choose sources → to get a library of your own and a daily brief.