OpenAI-compatible LLM gateway · For teams running LLM APIs in production

Find repeated LLM work you can run for less.

ModelGate prices every LLM request, finds the tasks you pay for again and again, and tests a cheaper model, plain code or a small open model against your own recorded answers. What passes arrives as a pull request you review.

Demo video: a fictional company and synthetic data.

A 14-second product demo with synthetic data, in four scenes: a quality-checked switch from claude-opus-5-5 to claude-sonnet-5-5; an LLM call replaced by generated code that matched 200 of 200 held-out calls; a paid model replaced by the free open model all-MiniLM-L6-v2, verified on 132 held-out tickets; and a prompt injection in a retrieved document scored 75 and blocked with a 403 before reaching the model.
  • OpenAI-compatible base URL
  • OpenAI, Anthropic, Gemini, Azure
  • Changes arrive as pull requests
  • Prompts not stored by default
Works withOpenAIAnthropicGoogle GeminiAzure OpenAIGitHubn8n

How it works

From observed usage to a change you can review.

Your code changes only when you merge. Routing and caching stay off until you switch them on.

  1. Step 1

    Observe usage

    Each request is logged — failures included — and priced from the provider’s token counts against a versioned rate table.

    One base URL change. Prompts aren’t stored unless you opt in.

  2. Step 2

    Find a candidate

    Repeated requests are grouped into tasks, ranked by spend, and flagged where a cheaper option may fit.

    Code candidates need 50+ recorded calls of one task.

  3. Step 3

    Test an alternative

    A cheaper model, generated code or a small open model is checked against your recorded answers first.

    Needs prompt and response storage, and a trial or paid plan.

  4. Step 4

    Review and install

    On GitHub, the change opens as a pull request on a new branch. ModelGate never merges it.

    Not on GitHub? Copy the change from the dashboard.

  5. Step 5

    Check the result

    ModelGate compares cost per call before and after: measured for model and prompt changes, estimated for code.

    Quality after the change is yours to monitor.

Not available yet: running a replacement inside the gateway. Verified code runs in your own app; Python versions are ports that ModelGate doesn’t execute.

Capabilities

Three ways to cut the cost of a repeated task, and one guard.

  1. 1Optimize

    Check a cheaper model before you switch.

    ModelGate flags oversized prompts and premium models on simple tasks, replays real requests through a cheaper model to score agreement before you switch, and measures cost per call after you change it.

    Prompt changes are recommendations you apply in your code.

    ModelGate dashboard: claude-sonnet-5-5 scored 93.3% agreement on 24 samples, with a Re-run eval button; a prompt change shows $556.31 saved so far, measured.
    See it in the demo
  2. 2Replace with code

    Some AI calls don’t need AI.

    For narrow tasks — fixed labels, extraction, JSON reshaping — with 50+ recorded calls, ModelGate generates code from 20 of them and tests it on the rest. It’s suggested only if it matches 95% of 30 or more calls it never saw. You review it as a pull request.

    The code runs in your app, so hosting, maintenance and monitoring are yours. Open-ended writing doesn’t qualify.

    Generated TypeScript replacement function that maps webhook fields into a fixed JSON shape, with the result: 100% match on 200 unseen calls.
    See it in the demo
  3. 3Go free

    Use free models when they fit.

    When code can’t do the task, ModelGate tests small open-weight models — permissive licences, pinned revisions — against the same held-out answers before suggesting one.

    No per-call model API fee or key. You host it, so compute and latency are yours — and ModelGate’s 30% fee applies to the estimated saving for 90 days.

    Task ticket-triage on claude-sonnet-5-5 marked could be an open model, and: open model verified (Xenova/all-MiniLM-L6-v2), 100% match on 132 unseen calls.
    See it in the demo
  4. 4Protect

    Stop malicious instructions before they reach your model.

    User messages, retrieved documents and tool results are scored for injection patterns. In enforce mode a high-risk request gets a 403 before any provider call; every incident is logged with masked context.

    A heuristic: it catches common attack families, not every phrasing.

    Security settings: prompt-injection shield in ENFORCE, 4 requests blocked, 8 flagged; a request log row with status blocked on gemini-2.5-pro.
    See it in the demo

Also in the gateway: request-level cost logs, an exact cache and model routing, spend limits and secret/PII scans, and reliability checks. Caching, routing, code generation and enforce mode are paid-plan features. Figures in the images come from the demo workspace, not a customer.

Up close

Some LLM calls should be an if { statement }

Ticket triage on gpt-4o, replaced by generated code — checked against the model’s own answers first.

Before$0.00102per call, on gpt-4oopenai.chat.completions.create(…)
After$0.00per call, keyword rules in your codeconst priority = replacement(ticket.body);

Labelling tickets “urgent” or “normal”: generated from 20 recorded calls, then run in a sandbox on 150 it never saw — 146 matched (97.3%). Below 95% on 30+ unseen calls, nothing is suggested. Illustrative counts.

See the actual generated code
Before · LLM callillustrative
const res = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [
    { role: "system", content: "Reply with one word: urgent or normal." },
    { role: "user", content: ticket.body },
  ],
});
const priority = res.choices[0].message.content;
After · generated code$0.00 per call
export function replacement(input: string): string {
  const text = input.toLowerCase();
  const rules = [
    {
      "label": "urgent",
      "any": ["outage", "is down", "can't log in", "charged twice", "data loss"],
      "none": ["not urgent"],
    },
  ];
  for (const r of rules) {
    if (r.any.some((k) => text.includes(k)) && !r.none.some((k) => text.includes(k))) return r.label;
  }
  return "normal";
}

// where the LLM call was:
const priority = replacement(ticket.body);

TypeScript is what gets tested. The Python version is a port ModelGate doesn’t run.

Only narrow tasks qualify. More examples: image checks, a free open model, a blocked injection →

Pricing

A base fee, plus a share of what it saves.

14 days free from the day you add a card. Cancel anytime.

Pro

$99/ month

1 project included · +$29/month per extra project

Connect your app

Agency

$199/ month

5 clients included · +$29/month per extra client

For agencies

On both plans

  • + 30% of realized savings from caching and routing, measured per request.
  • + 30% of savings from changes made from our suggestions, for 3 months each (estimated for code replacements; can be turned off).
  • Reliability add-on, optional: $49/month + $0.50 per active test above 20 a month.

Every saving is labelled: measured when it happened at the gateway, estimated for code replacements and suggestions you haven’t made yet. Full pricing and billing terms →

See what ModelGate finds in your traffic.

Connect with a base URL change. You see the cost per task first; nothing changes until you approve it.

Connect your app

Not ready? Watch the 77-second demo.

Raz Dagan

Built by Raz Dagan — system engineer with a background in physics, mathematics and AI. LinkedIn · info@modelgatehq.com