Notes from the Trenches // Token budget
I set out to spend tokens like a budget instead of a faucet. I built a worker to edit a resume rather than forge each one from scratch. It failed twice, and the second failure was the one worth keeping.
The job forge is the expensive part of the machine. It reads a candidate record, reads a posting, and writes a tailored two-page resume plus a landing page. Measured over fourteen runs it costs about a hundred and seventy three thousand tokens each, a little over a dollar a bundle. The scorer that grades every posting, by contrast, costs about twenty cents for a hundred rows. Same machine, two engines, a hundred to one apart in price.
When the bill is that lopsided, there is only one place worth optimizing, and only one honest question to ask of it: am I paying premium rates for work that is actually cheap?
The postings cluster by shape. Forward deployed roles look like each other. Applied AI roles look like each other. Once one strong resume exists in a cluster, the rest are not new documents. They are edits of the first: keep the spine, keep every claim, swap the company and the title, re-point which lines lead so they answer this posting instead of that one.
So I built an edit worker. Find the highest-scoring built bundle in the target's cluster, call it the seed, hand its resume and the new posting to the model, and ask for the edited resume back. In theory a third of a full forge, because it reads no reference library and only changes what the new role demands.
The first failure was stupid and fast. I named a model that does not exist, and the function threw a five hundred before it did any thinking. One line to fix. The kind of error that is embarrassing precisely because it is cheap.
The second failure was slow, and it was expensive, and it was the interesting one.
HTML.
Result504 FUNCTION_INVOCATION_TIMEOUT. The model was still writing the document when the sixty second function clock ran out.
The edit that was supposed to be cheap had timed out doing the one thing I told it to do: write the whole page again.
I had framed the edit as a small generation. It is not. Returning the whole resume is work proportional to the resume, whether one line changed or forty did. The size of the output has nothing to do with the size of the change.
The cheap thing was never a smaller model or a tighter prompt. It was a different shape. Do not return the document. Return the difference: the new target line, which bullets lead, the handful of reworded hooks. Returning the whole thing is proportional to the document. Returning only what moved is proportional to the change.
Efficiency is not a smaller model or fewer tokens. It is changing the shape of the work so you pay for the delta, not the document.
Once the answer is a patch and not a page, three things fall out at once, and they are the three levers everyone reaches for and usually applies to the wrong layer.
Caching and batching were never the fix on their own. They are what a diff-shaped workload makes possible.
The route stops returning a page. It returns a small structured patch. The driver applies that patch to the seed on the machine that already holds it, then renders the PDF locally where there is no sixty second clock. Raising the timeout would have worked too, and it would have missed the point. A longer clock buys you the right to keep paying for the whole document. The diff stops you paying for it at all.
The rule I am keeping: when work looks repetitive, do not ask which model is cheap enough to do it. Ask what the smallest true unit of the work is, and refuse to send anything larger. The budget was never about the price of a token. It was about the shape of the thing you were buying.