The GPU cost model nobody writes down before buying hardware
Self-hosting beats the API on cost per token only above a utilisation threshold almost nobody reaches. Below it you are paying for idle silicon and engineer attention — and only o…
Field Notes
All notes
Fine-tune or retrieve: the decision is about what changes, not what you prefer
Fine-tuning encodes behaviour; retrieval supplies facts. Asking which is better is asking the wrong question — ask how often the underlying knowledge changes and whether you must…
Multi-tenant isolation: the requirement that arrives after the demo
Every multi-tenant RAG eventually discovers the same thing: filtering results after retrieval is not isolation. The filter has to constrain the search, not trim its output.
Choosing a vector store: four questions that settle it
Published benchmark tables compare recall on public datasets, which is not your dataset, your filters, or your query distribution. Four operational questions decide it instead.
Before an agent touches production data: the guardrail checklist
The risk is not that the model says something wrong. It is that the agent does something wrong — with credentials, in a system that does not have an undo.
Context engineering is the new prompt engineering
For two years the advice was to write better prompts. That advice stopped paying off somewhere around the point where the model became reliably good at following instructions. Wha…
Why most enterprise RAG pilots stall in week three
The demo works on a handful of clean PDFs, the room is impressed, and then the real document set arrives: scans, merged cells, ten-year-old contracts and three languages. This is…
How to run an AI pilot that survives procurement
The demo goes well. The users are happy. Then legal asks where the data went, security asks who can see what, and finance asks what it costs per month. If you have not prepared fo…
Building 50 evaluation questions that predict production quality
Teams that measure tend to measure the easy cases, because those are the ones where someone already knows the answer. The questions that will cost you money are the ones nobody th…
Permission models: the part of enterprise RAG nobody budgets for
Retrieval quality is a solved-enough problem. Permissions are the reason enterprise RAG projects stop: not because they are hard, but because they arrive late and force a rewrite.
Self-hosting vs. managed inference: where the cost curve actually crosses
"Self-hosting is cheaper" is only true above a certain utilisation. Below it, you are paying for an idle GPU and an engineer's attention — and that second line item is the one nob…
Where the milliseconds go: a latency budget for RAG
A RAG request feels slow long before it is slow. Knowing which of the five stages is responsible is the difference between a targeted fix and three weeks of replacing the wrong co…
Model routing: sending easy questions to cheap models
Routing is the highest-leverage cost control in an LLM system, and it is almost always implemented last, after eighteen months of paying premium prices for questions a smaller mod…
Agent evaluations are the new unit tests
Every team that runs an agent in production eventually builds the same thing: a suite of scenarios that must not regress. The only question is whether you build it before or after…
What a forward-deployed engineer actually does all day
The title sounds like research. The job is mostly data plumbing, security questionnaires, and convincing a department head that the pilot will not embarrass them — which is exactl…
Nothing here yet. The next crawl fills this up.