Shimmy

Tell the optimizer what your program already knows.

You may not need this

Pointing an OpenAI client at the optimizer already works and already saves money:

const openai = new OpenAI({
  baseURL: 'https://rfa-labs.com/v1',
  apiKey: process.env.SHIMMY_API_KEY,
});

That stays supported and always will. It is the fastest way to start, and for plain request/response traffic it is most of the benefit.

Shimmy exists for what that cannot give you.

What a proxy cannot see

The optimizer sees a stream of independent HTTP requests. Everything else it has to reconstruct — and three of those reconstructions are guesses.

How your calls nest. Run membership is inferred by hashing a message prefix: turn N+1 usually contains turn N. That holds for a strictly sequential agent and breaks for everything else — concurrent branches interleave into one linear run, an agent that compacts its own history stops containing the prior turn, a summarizing agent never builds a prefix, and a second replica has no memory of the first one’s runs.

Worse than losing information, it invents some. Parallel sub-calls get numbered in arrival order, so the graph draws edges between siblings that never called each other — and when those siblings share a step shape (a summarizer mapped across ten documents), the repeat also registers as a loop that does not exist.

What a step is for. Difficulty is inferred from ten English keywords, a character count, and whether there is a code fence — then rounded up when unsure. Every ambiguity is billed to you as overspend, and a non-English workload has almost no signal at all. "Analyze the sentiment and label it positive or negative" trips the word analyze and routes to a frontier model to pick one of three labels.

Whether the answer was any good. The quality gate dispatches a second, more expensive call and pays an LLM judge to compare them. That spend reaches you as exploration_cost. Your code already parsed the JSON.

Adopt it in the order that pays

Each step is independently useful, and each makes the next one worth more.

1. wrap() — routing, plus cost metadata you can actually read.
2. run() / step() — correct topology; fixes the fan-out and phantom-loop cases above.
3. kind — replaces keyword-guessed difficulty with fact.
4. report() — replaces bought quality signal with free observation.

A call with no annotations behaves exactly as it did before the SDK existed, so you can stop at any rung.

Install

npm install @rfa-labs/shimmy

Everything here

• Start
  • Overview — What Shimmy is, and why a proxy needs telling.
  • Quickstart — Install, wrap, scope, and make one annotated call.
  • Concepts — Runs, steps, outcomes — and what the edge cannot see.
• Guides
  • Instrument an agent — Runs and steps, including the parallel case inference gets wrong.
  • Report outcomes — Replace a paid LLM judge with what your code already knows.
  • Framework adapters — LangChain, Vercel AI SDK, LlamaIndex.
  • Config as code — Routing ladders and guardrails in git, not a dashboard.
• Reference
  • Client — run, step, wrap, report, verify.
  • Control plane — Savings, calls, agents, settings — every endpoint.
  • Types — Step kinds, outcome signals, annotations.
  • Admin API — Model catalog and custom providers, managed at runtime.
• Direct API
  • Integration — The raw HTTP path — a stock OpenAI client, or curl. No SDK.
  • SDK vs direct API — What the SDK adds, and what is identical over the wire.
• Moving over
  • Migrating — From a base_url swap — and when not to bother.