Explore

Why bubbles form: technology, adoption, and manic-depressive investors

Bubbles form from the gap between technology advancement, consumer adoption that moves in fits and starts, and manic-depressive investor sentiment — investors get overexcited and then decide it's never going to work. During the internet bubble, investors decided the internet was the biggest thing ever and put all their money in believing it would never go down in price; but with only about 50-60 million people online, half on dial-up, the market wasn't big enough to make money on many ideas and the thing crashed.

Host's framework and historical account

The a16z Show · What Makes a Great Entrepreneur? | Ben Horowitz on Vintage x Vintage

Avoid the two symmetric investing mistakes

Don't pass on a great entrepreneur because of something you dislike about the idea, and don't back a great entrepreneur whose idea is actually no good — both mistakes are costly.

The a16z Show · What Makes a Great Entrepreneur? | Ben Horowitz on Vintage x Vintage

Pick the entrepreneur over the idea, and judge the magnitude of their strength

What ends up mattering in picking winners is the entrepreneur — given a perfect rating system, you would take the entrepreneur over the idea every time. But you need the idea to help you evaluate the entrepreneur: what they went through in their life to get to that idea, how good the idea is, and how much original work they did to get there. What matters is whether someone's strengths are world-class — are they the best in the world at something? Look at the magnitude of the strength. Every entrepreneur who gets to the firm is pretty good, but the question is whether they're outstanding — better than everybody else at something. The question to ask when evaluating a person is: what are they truly great at?

The a16z Show · What Makes a Great Entrepreneur? | Ben Horowitz on Vintage x Vintage

RAG handles semantic content questions but not structural questions

RAG handles semantic content questions like finding code, the PR that changed it, and the Slack thread debating it. Wrapping semantic retrieval in an agent loop with file search tools is very powerful and can go really far. But RAG does not handle structural questions like PRs merged last week, who reviewed payments the most, or open PRs.

described

AI Engineer · Beyond RAG: A Relational Context Engine That Cuts Token Burn — Unblocked

Keep hidden fields and tenant IDs away from the LLM; inject them server-side

Keep hidden fields away from the LLM entirely; don't let the LLM see or derive and execute them. For tenant isolation, don't let the LLM interact with tenant ID metadata fields; inject those fields behind the scenes and constrain queries to a specific tenant.

endorsed

AI Engineer · Beyond RAG: A Relational Context Engine That Cuts Token Burn — Unblocked

Structural questions are queries, not semantic searches, so they need a query engine

Structural questions are queries, not semantic searches, so they require a query engine rather than a vector store. Generating queries is the easy part; the hard part is the scaffolding around it.

endorsed

AI Engineer · Beyond RAG: A Relational Context Engine That Cuts Token Burn — Unblocked

CPT-style post-decay multimodal training degrades text capability

The most common approach — adding multimodal training after the pre-training decay stage (CPT-style) — tends to interfere with learned text capability and cause degradation, because the model has already largely converged.

AI Engineer · How MiniMax M3 Was Built: Sparse Attention and Native Multimodality — Olive Song

Four bottlenecks in a multimodal data pipeline: concurrency, prefetching, data copies, framework optimizations

The speaker lays out four bottlenecks to address in a multimodal data pipeline: concurrency (how fast one image can be processed before feeding the model), prefetching, minimizing data copies across processes, and framework optimizations. Training clusters usually have more CPUs than GPUs, yet the GPU is what starves in the pipeline.

AI Engineer · From 15% to 90% GPU Utilization: Fix the Data Pipeline, Not the Model

Wait time ratio: measuring time blocked on data acquisition

The speaker introduces a wait time ratio metric: the time it takes to prep data and transfer it to the GPUs divided by the total pipeline time including training. The baseline pipeline spends 85% of its time blocked on data acquisition, reduced to under 10% after optimizations.

AI Engineer · From 15% to 90% GPU Utilization: Fix the Data Pipeline, Not the Model

Scaling up re-exposed the bottleneck: wait time ratio rose with 4x ranks, 200 streams, 4x batch

When scaling up, the same pipeline's wait time ratio went up even though only numbers were scaled (4x data parallel ranks, 200 data streams, batch size x4). When the data generator and worker pool are co-located for fast data transfer, the node's network interface card becomes a bottleneck: while packing processing into specific nodes, their network throughput cannot keep up with the demand of the training pipeline.

AI Engineer · From 15% to 90% GPU Utilization: Fix the Data Pipeline, Not the Model

Large multimodal tensors copied by value across processes cause transport costs and OOMs

After adding concurrency and prefetching, data transport costs still linger and have gone up. Every sample transfers large image arrays across process boundaries multiple times: pickled on the worker, copied to the coordinator, then re-pickled onto the trainer. Multimodal tensors are huge (megabytes per sample), and by default every sample is returned by value, so expensive arrays are copied from process to process. As a result the data generator can often OOM because of repeated mem copies of large multimodal tensors.

AI Engineer · From 15% to 90% GPU Utilization: Fix the Data Pipeline, Not the Model

Interactive prompts get agents stuck

Pedro Lopez describes the anti-pattern of CLIs waiting for interactive prompts: with no human present, agents get stuck.

AI Engineer · Designing CLIs for Agents, Not Humans — Pedro Lopez, Airbyte

Scratchpads, grep, and bigger context windows don't scale as memory

Anders Swanson argues against common memory shortcuts: relying on scratchpads (markdown files, file hierarchies, wikis, prompts) leads to ungoverned recall and does not scale in large multi-tenant environments; relying on grep for memory retrieval scales poorly and lacks complex hybrid retrieval capability; and using only larger context windows leads to token waste, compaction, and loss of useful signal in irrelevant information.

endorsed

AI Engineer · From Context to Memory: Your Agents Need a Real Memory Layer — Anders Swanson, Oracle

Loss functions: defining error, empirical loss, and matching loss to problem type

To train a network you must define what a wrong prediction means — an error or loss function: the closer the prediction to the actual value, the lower the loss. The empirical loss is the mean of individual losses across the dataset, and training minimizes this loss on the available data. For binary classification (e.g., pass/fail) use softmax cross entropy; for regression predicting a real number (e.g., a grade) use a loss like mean squared error.

Alexander Amini · MIT Introduction to Deep Learning (2022) | 6.S191

Users progress toward full automation as trust builds

Users show a clear trajectory toward fully automated PRs as they build trust in the AI's accuracy: they first get reviews and build trust in the accuracy of the reviews, then give the agent teeth by allowing it to start blocking PRs, then use autofix capabilities to automatically turn PRs green and fix issues raised by code review or CI failures.

AI Engineer · AI Writes More PRs. Who Validates Them? — Ali-Reza Adl-Tabatabai, Sonar

Combining agentic validation with program analysis yields better precision and coverage

There is a ton of opportunity in combining an agentic approach to validation with program analysis now that Guitar is part of Sonar and integrating with SonarQ; combining agents with taint analysis, control flow analysis, data flow analysis, and software composition analysis yields much better precision and coverage at a great price point; the combination of agentic validation plus algorithmic program analysis will be much better than either alone.

AI Engineer · AI Writes More PRs. Who Validates Them? — Ali-Reza Adl-Tabatabai, Sonar

Three principles for internal app infrastructure: invisible, secure by default, discoverable

When building Atlas, WorkOS maintained three principles: (1) invisible infrastructure — no one should need to know where the deployment target is, and it should be transparent and swappable (Cloudflare Workers today; Vercel or AWS later shouldn't matter); (2) secure by default — no one should access sensitive internal tools outside the company, since many contain PII and customer/sales data; (3) discoverability — when someone shares a tool link in Slack, anyone should be able to click it and view the tool.

described

AI Engineer · Let Anyone at Your Company Ship Internal Apps with AI — Garrett Galow, WorkOS

Secure-by-default details: Okta SSO, token auth for APIs, audit logs, and managed secrets

All internal apps are automatically protected by the company's Okta SSO account — users cannot log in without Okta access. For APIs and Slackbots, token-based authentication can be enabled to provide API access. All requests are logged and auditable so a leak or security issue can be detected. Secrets for internal and third-party API access are stored via a secrets manager (WorkOS uses Doppler), so builders never push secrets into a Git repo.

described

AI Engineer · Let Anyone at Your Company Ship Internal Apps with AI — Garrett Galow, WorkOS

The most important success metric: non-engineers building and deploying without engineers

The most important success metric for an internal app platform is enabling non-engineers to build and deploy apps. A marker of true success is eliminating the need for engineers to migrate or manage tools on behalf of marketing or sales. 37% of WorkOS's 60 internal apps were built without any engineers in any of the commits — totally built between a person and Claude without engineering involvement.

described

AI Engineer · Let Anyone at Your Company Ship Internal Apps with AI — Garrett Galow, WorkOS

Missing context, details, or permissions leads to hallucinations

Matt Gibiec explains that if the data provided to an agent is missing context, details, or the right permissions to pull data from tools in the environment, hallucinations result.

AI Engineer · Your AI Agent Has No Nervous System — Matt Gibiec, Dynatrace