The model proposes.
A tool call is structured output: a tool name and arguments. It does not run itself.
LESS MYSTERY. BETTER MENTAL MODELS.
The model thinks.
The harness makes it act.
An LLM generates messages. The software around it supplies context, runs tools, and decides what happens next. That software is the harness.
See the loop◎Youset the goal
HARNESS · THE CONTROL LAYER
Build context
LLM generate a message
Route the result⌘Toolsread, search, execute
THE 10-SECOND VERSION
Model = message generator. Harness = context + tools + control loop. Cache = reused prompt computation, not a saved answer.
01 / THE CONTROL LOOP
A tool call is a request.
Not an action.
The model asks to use a tool. The harness validates the request, enforces permissions, and executes it.
Usersets the goal
Harnessorchestrates
LLMgenerates messages
Toolruns via harness
The harness is the hub: the LLM requests tools; the harness runs them.
The harness controls.
It can approve, reject, or ask you. A model requesting a shell command is not permission to run it.
The loop has limits.
It stops on a final answer, cancellation, error, or a step / token budget—not just when the task is done.
02 / WHAT THE MODEL SEES
A new call.
A growing context.
The model does not automatically remember a previous API call. The harness sends the relevant history again—or uses an API that manages it.
Token: a chunk of text, often part of a word. Requests consume input tokens; generated messages consume output tokens.
01
Instructions + tool definitionsRules, tool names, argument schemas
stable02
Reference materialProject notes, documents, selected files
often stable03
Conversation so farUser messages, model replies, tool results
grows04
Latest message or tool resultThe new information for this call
new LLM generates output tokens
The tool’s output becomes the model’s input.
A file read is not knowledge until its result reaches the next model call.
Context is a working set, not a hard drive.
It has a token limit. Harnesses may select files, truncate results, or summarize old turns to fit.
The pricing connection: history is input again on later calls. Prompt caching can make repeated prefixes cheaper.
03 / PROMPT CACHING
Same beginning.
Less repeated work.
Providers can reuse computation for a matching prompt prefix. You still send the context; the model still generates a new response.
Call 1
Instructions
Tools
Document
History
Message A
Call 2
Matching prefix: potentially reusable From first change onward: fresh computation
Conceptual blocks, not token counts. A matching prefix is necessary, not a guarantee: minimum length, cache lifetime, routing, model, and provider rules also matter. Some APIs require explicit cache configuration.
DO
Stable first. Variable last.
Keep instructions, tool definitions, and reusable documents consistent. Append new turns after them.
AVOID
Churn at the front.
A changing timestamp or reordered tools near the start can break reuse for everything after it.
Cache ≠ memory.It does not add context, increase the context window, or replay an old answer. It reuses computation for input already present.
04 / FOLLOW THE TOKENS
A cache hit is cheaper.
Not free.
Separate the bill into fresh input, cache writes, cache reads, and generated output. Rates differ by model and provider.
Fresh input
New or uncached input, billed at the regular input rate.
Cache write
Creating a reusable prefix. Some providers charge a premium.
Cache read
Reusing an eligible prefix, usually at a discounted input rate.
Output
Newly generated tokens. Prompt caching does not discount them.
One prefix, repeated requests.
Call 1 writes the prefix. Later calls read it. Each call has fresh input and output.
Edit example rates USD / 1M tokens
WITH PROMPT CACHING
$0.2340
Fresh input Write Read Output
Assumptions: fixed shared prefix; all later reads hit before expiry; no growing history. Each input token is counted once as fresh, write, or read. Actual agents often have growing context and partial hits. Tool fees, cache storage fees, taxes, and other provider charges are excluded.
Show the math
Let N = calls, P = prefix tokens, F = fresh tokens per call, O = output tokens per call. Rates are dollars per million tokens.
No cache = [N × (P + F) × input rate + N × O × output rate] / 1,000,000With cache = [P × write rate + (N − 1) × P × read rate + N × F × input rate + N × O × output rate] / 1,000,000For automatic caching without a write premium, set the write rate equal to the regular input rate.
Before using real prices: check the exact model, cache lifetime, minimum prefix length, and billing rules. OpenAI caching ↗Anthropic caching ↗Gemini caching ↗
05 / MAKE IT STICK
Three ideas. One quick check.
The harness runs the loop.
The model generates the next message.
The cache saves repeated input work.