Talk to us
Shimmy docs

SDK vs direct API

The SDK gates nothing. Every optimization — routing, caching, the quality gate, the learning memory, the savings meter — runs inside the gateway and reaches you over plain HTTP whether or not you use a client library. So the honest question is not “what can the SDK do that the API can’t”, it is “what does the SDK save you from doing by hand”.

What is identical

Over raw HTTP you get, in full:

  • All routing and optimization. model: "auto" and "discover", the five-tier ladder, semantic and exact caching, context pruning, plan caching — all of it, on every call.
  • The savings meter. The optimizer block on every response; the same figures from GET /v1/dashboard/savings and GET /v1/calls.
  • Every annotation. run, step, intent, source — you write the optimizer object yourself (see integration).
  • Outcomes. POST /v1/outcomes is the same endpoint the SDK’s report() calls.
  • The whole control plane. Every settings write and read is a plain /v1 request.

None of that is a reason to adopt the SDK. These are:

What the SDK does for you

The SDK addsBy hand, over the API
Ambient run/step topology — run() / step() scopes track nesting automatically, including sibling detection for parallel workYou compute and attach run.id / run.seq / run.parent on every request yourself, and get the fan-out-vs-chain distinction right by hand
Automatic source capture — file, line, function and release attached from the call siteYou populate source.* yourself, or forgo the code-linked dashboard rows
Per-call idempotency key — set on every completion, so a retry is dedupedYou set Idempotency-Key yourself, and remember to reuse one key across your own retries
Config as code — plan() / apply() with local validation that refuses a misspelled tier key before it orphans your evidenceThe raw settings write accepts a typo and silently retrains routing from zero
Framework adapters — LangChain, Vercel AI SDK, LlamaIndex emit annotations without touching call sitesYou thread annotations through the framework’s request path yourself
Response-meta recovery — reads the optimizer block a typed client would otherwise dropYou read it off model_extra (Python) or an untyped cast (TS) yourself

When to use which

  • Raw HTTP / stock OpenAI client — request/response traffic, a language with no Shimmy SDK, or a team that wants zero dependencies. You still get routing, caching, quality-gated savings, and the meter. Declare run/step by hand when you want correct topology and per-step learning.
  • The SDK — agentic or parallel workloads where topology is easy to get wrong, teams that want source-linked evidence and config-as-code for free, or anyone on TypeScript, Python or Rust who would rather not hand-maintain the optimizer block. It is a superset, adopted one rung at a time.

Next