Concepts

Four ideas — runs, steps, structure, outcomes. Each exists because the optimizer, standing outside your process, cannot work it out for itself.

Runs

A run is one execution of a workflow — a request handled, a job processed, a conversation turn served.

The edge infers run membership by hashing a message prefix, on the reasoning that an agent re-sends its whole thread each turn, so turn N+1 contains turn N. That is true of a simple loop and false of nearly everything else:

• Concurrency. Two runs of one agent in flight simultaneously interleave at the edge and share a process-local tracker.
• Compaction. The moment an agent trims history to fit a context window, turn N+1 no longer contains turn N, and one run silently becomes two.
• Summarizing agents. Anything that condenses rather than appends never builds a prefix at all.
• Replicas. The tracker lives in one process. Behind a load balancer, a five-step run can fragment into five one-step runs.

Scoping a run removes all four at once, and you do not have to name it: run.id resolves itself from an active trace or a fresh uuid. What matters is that the boundary is declared rather than guessed from message content.

Steps

A step is one call within a run, named by you.

The alternative is a fingerprint computed from the masked system prompt. It works, and it has a cost that surfaces at the worst time: editing a prompt mints a new identity. The per-step model search that had converged on the cheapest model that holds quality resets to zero, and re-converges one probe per occurrence.

So ordinary prompt tuning quietly discards accumulated optimization. A declared step id is content-independent: reword the prompt behind classify_intent as much as you like and the step keeps everything it has learned.

Two ids, two different questions

run.id and step.id sound alike and do opposite jobs. The distinction is worth thirty seconds because it is the one thing people get backwards:

Read it as: run.id is different every time on purpose. step.id is the same every time on purpose.

A classifier called every night has one step.id — classify_intent — for its entire life, across thousands of runs. That is what lets the model search accumulate evidence about it. Each night’s execution gets its own run.id, because grouping tonight’s calls with last night’s would be wrong.

Structure

Steps nest. A step opened inside another names it as parent, and that single field is the difference between a tree and a chain.

Consider three sub-tasks running in parallel. At the edge they arrive interleaved, in completion order. Inferring from that order produces:

• edges between siblings that never called one another, and
• a cycle that does not exist, when the siblings share a step shape — a summarizer mapped across ten documents is the common case.

Declared, they are siblings: one parent, three children, no order implied.

Outcomes

An outcome is what actually happened to a call’s output.

The quality gate has to know whether a cheaper model held up. Standing outside your process, its only option is to dispatch a second, more expensive call and pay an LLM judge to compare them. That is real money, and it reaches you as exploration_cost.

Your program already knows, for free:

• Did the JSON parse and validate?
• Did the tool call actually execute?
• Did the agent retry this step?
• Did a person accept, edit or reject the answer?

Every one is a stronger signal than a judge’s opinion, because a judge is approximating exactly these facts. Reporting them replaces bought evidence with free evidence — and once a (step, model) pair has enough verified evidence, the gate stops dispatching a baseline call at all.

What is declared, and what is only suggested

Not all annotations carry equal weight, deliberately.

Discovery mode

Routing answers “which tier should this call use?” with a guess. Discovery finds out: for each step of the workflow, it searches the whole reachable ladder and converges on the cheapest model that holds the step’s quality bar — in a handful of agent runs, not weeks of traffic.

How a step’s search runs:

1. The first call anchors. The step’s first occurrence is served by a frontier model (or your pin, if you sent model alongside a discover annotation — the pin becomes the ceiling instead). That answer is graded once; its score and cost are what every candidate is measured against.
2. Later calls run paired rounds. Each occurrence dispatches every surviving candidate against the same live prompt, graded by the same judge — which is what makes their scores comparable — and successive halving eliminates the weaker half by cost-per-quality-point. One graded failure eliminates a candidate on the spot.
3. The step settles. When one survivor clears the bar (three rounds at a 90% match rate, by default), it becomes the answer and serves judge-free from then on. If every cheaper model fails, the answer is the anchor itself — “keep the frontier model” is a legitimate result, and the report says so.

Two things make the economics different from ordinary routing:

• The platform pays for the search. Anchors, probes and gradings run on our keys, debited from a per-agent discovery credit (a free grant to start). While a step searches, the meter claims nothing — a subsidy is not a saving.
• The winner may not be yours to serve. A settled step serves on your keys when you hold one for its provider — that is when the saving becomes real. Until then it serves on platform keys, still drawing the credit. The report on the agent’s dashboard page prices the gap and names the provider to add.

Ask for discovery three ways, most specific first: a step.mode: "discover" annotation, the model: "discover" sentinel on the request, or "auto" with discovery enabled on the agent. step.mode: "off" excludes a step even when the agent has discovery on.

The asymmetry is on purpose. Believing a mis-declared kind costs at most one graded exploration; disbelieving a correct one costs overspend on every call forever. But a caller must never be able to switch the quality gate off, which is why the score is clamped in one direction only.

Next

• Instrument an agent — runs and steps in practice.
• Report outcomes — the loop that pays for itself.