How the Optimizer compares.
LiteLLM, Portkey, and Cloudflare AI Gateway are real tools that solve real problems. Here is an honest, feature-by-feature look at what each does — and the layer only the Optimizer was built to do.
The capabilities, side by side.
✓ = built in · ~ = partial / basic · — = not offered. We try hard to keep the competitor columns accurate; if we got one wrong, tell us.
| Capability | The Optimizer | LiteLLM | Portkey | Cloudflare AI Gateway |
|---|---|---|---|---|
| OpenAI/Anthropic-compatible proxy (one-line base_url swap) | ✓ | ✓ | ✓ | ✓ |
| Exact + semantic response caching | ✓ | ~ | ✓ | ~ |
| Model routing + fallback chains | ✓ | ✓ | ✓ | ~ |
| Quality-bounded routing (LLM judge scores every prose downshift) | ✓ | — | — | — |
| Tool-call verification (structural: declared tool + valid args) | ✓ | — | — | — |
| Test-time-compute pruning (verify-and-stop) | ✓ | — | — | — |
| Agentic plan caching (multi-step run templates) | ✓ | — | — | — |
| Tool-call memoization + context pruning | ✓ | — | — | — |
| Routing memory (cold classes warmed on nearest graded analogs) | ✓ | — | — | — |
| Learned router (trained on your own graded outcomes, data-gated) | ✓ | — | — | — |
| Auditable savings ledger (baseline-vs-actual metering) | ✓ | ~ | ~ | ~ |
| Exploration cost clamped (never billed for unmade savings) | ✓ | — | — | — |
| Opt-in anonymized cross-tenant learning | ✓ | — | — | — |
| Self-hosted VPC deployment | ✓ | ✓ | — | — |
| Open source | — | ✓ | ~ | — |
| Pricing model | 20% of measured savings (free tier: $5) | Free, open source | Subscription + usage tiers | Per-request / Cloudflare plan |
| You pay only when you save | yes | n/a (free) | no | no |
When to pick them, when to pick us.
LiteLLM
Excellent free, open-source proxy for routing and basic caching — and its newer auto-router adds heuristic tiers with Thompson-sampling exploration within a tier. Two things it deliberately does not do: verify whether the cheap answer was actually good (it trusts the tier; its bandit learns from proxy rewards, never a ground-truth judge), and learn from agent tool-call traffic. If you want a self-hosted gateway and you’ll wire your own quality checks, LiteLLM is a great default. The quality-guarantee layer is what we built.
Portkey
Strong observability and gateway features — logging, analytics, guardrails. If your primary need is to *see* spend, Portkey is solid. The Optimizer is built to *cut* spend: every downshift verified (judged prose, structurally-verified tool calls), verify-and-stop, and a memory substrate that compounds savings over time.
Cloudflare AI Gateway
Edge caching, DDoS protection, and a gateway in one — great at the CDN layer. Deep agentic optimization (plan caching, trace optimization, verified downshift routing) is not what an edge gateway does. They’re complementary, not exclusive: you can put Cloudflare in front of us.
The savings ledger is auditable to the dollar.
Point a slice of your traffic at the Optimizer and watch baseline-vs-actual in the dashboard. If we don’t save, you don’t pay.