
model-relay
Multi-provider router and cache-safe 429 failover for Claude Code (Claude, GLM, Gemini, DeepSeek, Kimi, MiniMax, OpenRouter)
Install with your AI
Paste into Claude Code, Cursor, or any agent — it reads the repo and wires the tool into your project.
Install and set up model-relay (claude-plugin project) into my current project. Found on https://claudeers.com/model-relay Repo: https://github.com/cablate/model-relay Homepage/docs: — Detected install method: claude-plugin → /plugin install model-relay@cablate/model-relay Category: plugins. Platforms: cli, api. Read the repo's README for exact setup and env vars, then install it and wire it into my project. Claudeers Health Verdict: active; community-verified: false. Confirm the source before running anything.
/plugin marketplace add cablate/model-relay /plugin install model-relay@cablate/model-relay
git clone https://github.com/cablate/model-relay
// compatibility
| Platforms | cli, api |
|---|---|
| Operating systems | — |
| AI compatibility | claude |
| License | MIT |
| Pricing | open-source |
| Language | JavaScript |
Keep working when Claude hits a 5-hour usage limit, offload subagents,
and switch any window from an interactive panel—without resending a long chat's whole prompt cache.
English · 繁體中文 · Install · Features · Configuration · Changelog
Why
Long Claude Code sessions run into four costly routing problems. model-relay solves them on localhost (127.0.0.1:8950) with zero external proxies.
| When… | Without model-relay | With model-relay |
|---|---|---|
| Claude hits a 5-hour usage limit (HTTP 429) mid-turn | Your turn fails and the conversation stops until quota resets | The same request automatically fails over to your backup provider and completes |
| Claude quota recovers 10 minutes later | Switching a 620k-token chat back to Claude resends all 620k tokens and burns fresh quota | Sticky sessions stay on the backup provider for 7 days; only new chats return to Claude |
You want haiku subagents on a fast/cheap endpoint | Changing global env vars forces your main Opus/Sonnet thread onto that endpoint too | matchModels routes haiku subagents to the provider you choose while your main thread stays on Claude |
| You want to switch just one window to another provider | Editing settings.json switches every open terminal at once | Type /provider to open the interactive panel and pin only the current window |
Third-party gateways choke on ToolSearch or cache_control | All tools vanish or requests fail with HTTP 400 | Per-provider stripToolSearch and stripCacheControl clean requests on the fly |
Who this is for
model-relay is built for Claude Code users who combine a Claude subscription with backup or specialized endpoints (z.ai, DeepSeek, Kimi, MiniMax, OpenRouter, or a gateway on your own machine). Each provider has to accept Anthropic's /v1/messages format; a service that only speaks OpenAI Chat Completions needs a local translator (see Limitations). If you need a multi-tenant cloud proxy for an entire organization, use LiteLLM instead—model-relay is a single-user localhost daemon.
Install
Requires Node.js 20+ and Claude Code 2.1.287 or later (tested on Node.js 22.23.2 and Claude Code 2.1.302).
Upgrading from 0.6 or earlier? This version reads only a config in the current format (
"format": 2). Until you convert yours, the plugin starts no router for the older file and leaves a 0.6 router that is still running alone, so your chats keep routing as before. To convert it, run/provider setup upgrade my model-relay configand Claude does it with you, or follow UPGRADING.md by hand (node tools/setup.mjs migrate, thenmigrate --go, then/provider restart).
claude plugin marketplace add cablate/model-relay
claude plugin install model-relay@model-relay
-
Set your provider's API key before starting Claude Code. The relay reads it from the environment of Claude Code, so a variable set in a shell that is already running Claude Code is not seen. Make it permanent, for example
export BACKUP_API_KEY="your-api-key"in~/.zshrc(macOS/Linux) orsetx BACKUP_API_KEY "your-api-key"(Windows), then open a new terminal. -
Open Claude Code and let it set things up with you. Run
/reload-pluginsfirst if Claude Code was already open./provider setupClaude runs the setup tool for you, never asks you to paste a key into the chat, and asks before it changes
~/.claude/settings.json. Add what you want, such as/provider setup use deepseek for haiku. -
Check that it works.
/provider doctor --pingYou should see
Config OK, your provider underProviders, and a✓for it underUpstream ping:Config OK (~/.claude/model-relay.json) Port: 8950 Providers: backup -> your-model-id Failover: backup (back to Claude after 10 min) Retry: on Aliases: none (run "node tools/setup.mjs alias haiku <provider>" to route haiku) Claude settings: ✓ settings.json ANTHROPIC_BASE_URL=http://127.0.0.1:8950 Router health: ✓ up on 127.0.0.1:8950 (pid 38780, <version>, pinned sessions: 0, 429 failover: idle, proxy off (0 routed / 0 tunnelled)) Upstream ping: ✓ backup (your-model-id) -> answers, first byte after 640 msAnything
doctorfinds is listed with a⚠and what fixes it;/provider setupcan run those fixes with you.doctor --jsonprints the same findings for an assistant or a script, each with thesetup.mjsarguments that fix it (fix) and whether that command changes something you should agree to first (needsConsent).
Nothing goes to a provider unless you say so. A new config keeps haiku and every other request on Claude; the first provider you add becomes the one that takes over on a 429 (failover.chain). To send Claude Code's haiku calls (subagents, WebFetch summaries, internal helpers) to a provider, run node tools/setup.mjs alias haiku <name>.
Setting it up by hand: run /provider init (creates ~/.claude/model-relay.json with no provider yet, so every request still goes to Claude, and points ANTHROPIC_BASE_URL in ~/.claude/settings.json at the relay). Add a provider with node tools/setup.mjs add <name> --preset <preset> --model <model id> --token env:<VAR> (presets lists the built-in ones; --url takes any other Anthropic-compatible address), restart Claude Code, then run /provider doctor --ping.
Where node tools/setup.mjs runs: the node tools/setup.mjs ... commands later in this README are the same tool. They run from the plugin's install directory, which is ~/.claude/plugins/cache/model-relay/model-relay/<version>/ ($CLAUDE_CONFIG_DIR/plugins/cache/model-relay/model-relay/<version>/ when you set CLAUDE_CONFIG_DIR; ls that folder for your version). Elsewhere, use the full path to tools/setup.mjs inside it. The command acts on the config directory in $CLAUDE_CONFIG_DIR, or ~/.claude if unset. /provider setup runs them for you from anywhere.
A key exported later is not seen: env:VAR tokens are read from the router process's own environment, which it inherits from whichever Claude Code session (or setup command) started it. A variable exported after the router started is not seen until the router restarts: stop the router process (doctor prints its pid) and start a new Claude Code session. doctor reads your shell, so it can report the key as set while the router cannot see it.
Updating: run claude plugin update model-relay@model-relay. The plugin then restarts the relay by itself once no request is in flight through it; a relay from 0.6 or earlier cannot report that, so the status line asks you to run /provider restart when no Claude Code window has a reply in progress.
What's inside
Interactive AbovePrompt control panel and provider test
Type /provider to open a panel right above your prompt input box. It does one thing: pick which model this window uses, one button per row. Once a switch takes, the panel closes and the status line shows the model now in use. Click a button, or press ctrl+x tab and then the key shown on it (1–9 to switch, p to test every provider, and r to try Claude again while a 429 cooldown runs). Switching a pinned window back to Claude takes a second press, so you never rebuild a long chat's prompt cache by accident. The switches every window shares stay folded under Settings [s]: press it to show 429 failover, retry and Remote Control (f / t / x) and the model aliases. They turn on or off without editing JSON: failover and retry apply from the next request, while Remote Control asks first because it rewrites settings.json, and it only reaches new sessions.
Each provider row shows two things apart: the last real request (last 200 · 45.0s) and the last test (test ok 640ms). The test sends one 1-token request and stops at the first byte (15 seconds at most), so a long reply never reads as a slow provider, and a failure says what is wrong in words: the key was refused, the model name is wrong, the quota is used up, the local gateway is not running, or there was no answer in time. The router keeps the provider's key out of anything it shows. When the router skips a provider (its env: variable is unset for the router), the panel says so.
Cache-safe sticky 429 failover
When Anthropic returns a 429 usage-limit response, the router immediately forwards the request to the first provider in failover.chain and pins that session ID for 7 days. After failover.cooldownMs (default 10 minutes), new sessions probe Claude again while existing sticky sessions stay on their backup provider. failover.enabled switches the whole thing off.
Transparent provider retries
A provider answering 429 or 5xx is retried on the SAME provider with exponential backoff (retry.attempts/delayMs/backoff), capped by retry.maxDelayMs; a longer Retry-After keeps polling at that cap. Set retry.attempts: 0 to retry forever (keep hammering until it succeeds — the client connection is watched, so retries stop once the caller disconnects). Claude Code never sees the intermediate failures. retry.enabled turns retrying off everywhere in one go. Turn on retry.failover and a request that still fails once its retries run out is handed to the next provider in failover.chain (or to that provider's own fallback list), never back to one it already tried.
Built-in provider presets and local gateway auto-launch
Built-in presets (zai, deepseek, kimi, minimax, openrouter) fill in a service's address and the compatibility switches it needs. You always choose the model, because model IDs go out of date, and give the token (or "env:VAR_NAME"). node tools/setup.mjs presets lists them. For local gateways on 127.0.0.1, set launch: { command, args } and the router will automatically spawn the gateway process whenever its port is closed.
Model alias routing (pointing haiku / sonnet / opus at a provider)
model-relay separates three routing layers so they never get mixed up:
- Per-request model alias (
matchModels+ANTHROPIC_DEFAULT_*_MODEL): Routes only requests whosemodelmatches (such ashaikusubagents orWebFetchsummaries) without pinning the main conversation. - Per-conversation pin (
/provider <name>): Pins only the current conversation ID (x-claude-code-session-id) for 7 days; other windows stay on Claude. - 429 failover chain (
failover.chain): Takes over only when Anthropic returns a 429 usage-limit error.
Pointing a Claude tier like haiku at a provider (for example backup, with model your-model-id[1m]) coordinates two places at once:
~/.claude/settings.json→env.ANTHROPIC_DEFAULT_HAIKU_MODEL = "your-model-id[1m]"so Claude Code itself resolves thehaikualias to that model (including the[1m]context window) and sends"model": "your-model-id".~/.claude/model-relay.json→providers.backup.matchModelsgains"haiku", so the router also catches internal Claude Code calls that hardcode"claude-haiku-*", rewriting the wiremodelfield to"your-model-id". A request for a provider's own model always reaches that provider.node tools/setup.mjs alias haiku claudeundoes both.
Use setup.mjs alias (or /provider alias) to inspect or set both files in one step:
node tools/setup.mjs alias # inspect haiku / sonnet / opus / subagent default
node tools/setup.mjs alias haiku backup # point haiku at backup in both settings.json and model-relay.json
node tools/setup.mjs alias haiku claude # reset haiku back to Anthropic Claude
node tools/setup.mjs alias subagent sonnet # set CLAUDE_CODE_SUBAGENT_MODEL=sonnet in settings.json
For per-call subagent routing, model-relay also dynamically registers a full-capability Agent type model-relay:<name> for each provider the router serves (such as model-relay:backup) via $.agent.register with all parent tools and model set to that provider's model ID, and locks that model in on('agent.spawn')—so you can spawn subagent_type: "model-relay:backup" on the fly without creating .claude/agents/*.md files or restarting the session.
Claude Code can still run such a subagent on another model without saying so (a model missing from availableModels is swapped, with a line only in its debug log). The plugin checks twice: when the subagent starts, against the model Claude Code reports it started on, and when it finishes, against the models the router actually received. Either mismatch shows a notice that points to /provider doctor.
Edit the config from the command line
add, set, providers, remove and migrate change the file without hand-editing JSON. Every edit keeps a .model-relay-backup copy next to the file, tokens are never printed, and the running router picks the change up on its next request. On an older config (see format under Configuration), add, set, providers, remove, alias, init, restart, rc and proxy enable refuse and tell you to run migrate first; migrate is the only command that understands it. doctor also warns about an older config, a failover name that matches no provider, a model name another provider claims first, an availableModels list in ~/.claude/settings.json that leaves a provider's model out, and a running router older than the plugin.
node tools/setup.mjs providers
node tools/setup.mjs add local --url http://127.0.0.1:4000 --model your-local-model-id --token env:LOCAL_GATEWAY_TOKEN
node tools/setup.mjs set failover.chain backup,local
node tools/setup.mjs set providers.local.fallback none
node tools/setup.mjs migrate --go
Remote Control without giving up the relay
Remote Control only starts when Claude Code talks to api.anthropic.com, which a base URL pointing at the relay breaks. HTTPS proxy mode keeps that address and makes the relay the proxy instead: Claude Code connects through HTTPS_PROXY, the relay answers for api.anthropic.com with a certificate that Claude Code trusts through NODE_EXTRA_CA_CERTS, and model calls are routed exactly as before (providers, sticky failover, retries). Everything else, token refresh and Remote Control traffic included, passes to Anthropic unchanged.
node tools/setup.mjs proxy enable --go
node tools/setup.mjs rc
- Contained trust: The CA the relay creates can only vouch for
api.anthropic.com(a critical name constraint), its private key is discarded right after signing, and only Claude Code processes that carryNODE_EXTRA_CA_CERTSload it. The system certificate store is never touched. - Safe fallback: Without a usable certificate the relay tunnels
api.anthropic.comuntouched, so Claude Code keeps working, just unrouted. A stopped router is different: the plugin's automatic direct fallback only works inANTHROPIC_BASE_URLmode, so in proxy mode Claude Code cannot connect until the router is restarted (node tools/setup.mjs rc). - Reversible:
proxy disable --gorestoresANTHROPIC_BASE_URL, and the previoussettings.jsonis kept assettings.json.model-relay-backup.
HTTPS_PROXY lives in settings.json env, so it also applies to commands Claude Code runs (git, curl, gh): they go through the relay as blind tunnels, and fail while the router is down.
Configuration
Configuration lives in ~/.claude/model-relay.json. The directory defaults to $CLAUDE_CONFIG_DIR when set, falling back to $USERPROFILE/.claude or $HOME/.claude. model-relay.example.json shows a provider from a preset and a local gateway.
| Field | Default | Meaning |
|---|---|---|
format | (none) | 2 for a config in the current format (init, add and migrate --go set it), the only format the router reads. A file without it, or one named custom-agent.json, is an older config: until migrate --go rewrites it, the plugin starts and replaces no router for it, so a 0.6 router still running keeps using it, and with none answering the session talks to Claude directly. The plugin says so in a notice, the status line and the panel; see UPGRADING.md |
port | 8950 | Localhost port for the relay HTTP server (1025–65535). Change it with node tools/setup.mjs set port <n>, which also updates settings.json; the router already running keeps the old port until you stop it (the command prints its pid), and it refuses a port that already answers |
claudeUrl | "https://api.anthropic.com" | Upstream Anthropic API base URL |
proxy.enabled | false | Accept HTTPS-proxy connections for Remote Control. Off by default. When on, a CONNECT to any host other than api.anthropic.com is a blind tunnel that any local process can use; node tools/setup.mjs proxy enable --go turns it on and proxy disable --go off |
failover.enabled | true | Automatically switch to the failover chain on Claude 429 usage limits |
failover.chain | [] | Ordered providers that take over when Claude is out of quota or a provider keeps failing; unknown names are skipped. Empty means a 429 from Claude is passed back as it is (add puts the first provider of a new config here) |
failover.cooldownMs | 600000 | Milliseconds before probing Claude again for new sessions (10 minutes, minimum 60000) |
retry.enabled | true | Master switch for retrying a failing provider; false keeps the other retry settings for later |
retry.attempts | 3 | Times the SAME provider is retried with backoff when it answers 429/5xx; 0 means retry forever |
retry.delayMs | 2000 | First retry delay; multiplied by retry.backoff each attempt |
retry.backoff | 2 | Multiplier between retries (2s → 4s → …), capped by retry.maxDelayMs |
retry.maxDelayMs | 10000 | Longest single wait; a longer Retry-After keeps polling at this interval |
retry.failover | false | Opt-in: after retries are exhausted, forward the original request to the next provider in the chain instead of returning the error |
providers.<name>.retry | true | Per-provider switch: false never retries this provider (failure returns at once); true follows the global retry; or a partial object like { "attempts": 5 } overrides just those fields |
providers.<name>.fallback | the chain | Who takes over from this provider: false for nobody, or a list like ["backup"] |
providers.<name>.preset | <name> | Optional preset (zai, deepseek, kimi, minimax, openrouter): the address and compatibility switches, never a model |
providers.<name>.baseUrl | (from preset) | Anthropic-compatible upstream URL for this provider |
providers.<name>.token | (required) | Literal token or "env:ENV_VAR_NAME". The variable is read from the router process's environment (inherited from the Claude Code session or setup command that started it), so one exported later is not seen until the router restarts |
providers.<name>.model | (required) | Target model ID when rewriting requests (suffixes like [1m] are stripped on wire) |
providers.<name>.matchModels | (from preset, else the model's first word) | Regex patterns matched against incoming request model, besides the provider's own model, which always matches |
providers.<name>.stripToolSearch | true | Strip ToolSearch and defer_loading: true tools from requests |
providers.<name>.stripCacheControl | false | Recursively remove cache_control fields before forwarding |
providers.<name>.authHeader | "both" | How to send credentials: "bearer", "x-api-key", or "both" |
providers.<name>.launch | null | { "command": "...", "args": [...] } to auto-spawn if localhost port is closed |
Commands
| Command | Description |
|---|---|
/provider | Open the panel above the prompt to pick this window's model |
/provider <name> · /provider claude | Switch this window without the panel, answered in one line (e.g. /provider backup); back to Claude, the next turn rebuilds the prompt cache |
/provider status | The status as text: where this window goes, the switches every window shares, and anything to fix |
/provider help | Every command, grouped by what it reaches: this window, every window, setup and checks |
/provider setup [goal] | Let Claude walk you through setup with the setup tool; keys stay out of the chat |
/provider init · /provider doctor [--ping] | First-run setup and health check; work from any folder, with or without a config file |
/provider restart | Restart the relay so it runs the installed plugin's code. After claude plugin update the plugin does this by itself once the relay answers no request; a relay left by an older plugin cannot report that, so the status line asks you to run it when no window has a reply in progress |
/provider ping | Test every provider: does it answer, how soon does the answer start, and what is wrong if it does not |
/provider probe | Reset the 429 cooldown timer so the next new session tries Claude immediately |
/provider alias [haiku|sonnet|opus] [<name>|claude] | Inspect or point a Claude model alias at a provider (updates both settings.json and matchModels) |
/provider failover [on|off|<a,b>] | Turn 429 failover on/off or update failover.chain |
/provider retry [on|off] | Turn automatic provider retries on or off |
/provider rc [on|off] | Turn Remote Control HTTPS proxy mode on or off (applies to new sessions) |
Cost, privacy, and permissions
- Network and data flow: The router listens only on
127.0.0.1:<port>. Requests go strictly toclaudeUrl(https://api.anthropic.com) or thebaseUrlendpoints you configure./provider pingsends amax_tokens: 1request to each configured provider when you trigger it. No telemetry or analytics exist. - Local files: Reads
~/.claude/model-relay.jsonand writes session sticky state to~/.claude/model-relay-state.jsonplus local logs (model-relay.logandmodel-relay-requests.log, each emptied once it passes 5 MB). The old file names (custom-agent.json,custom-agent-state.json,custom-agent-router.log,custom-agent-requests.log) are no longer read;migrate --gorenames them. - Local certificate (proxy mode only):
~/.claude/model-relay-pki/holds a public CA file and one leaf certificate with its key (restricted to your user). The CA is limited toapi.anthropic.comand trusted only throughNODE_EXTRA_CA_CERTS; delete the folder or runproxy disable --goto stop using it. - Plugin permissions: Inspect all hooks, state keys, and env access at any time by running
claude plugin validate /path/to/model-relay.
Supported environments
| Environment | Status |
|---|---|
| Windows 11 + Node.js 22 + Git Bash / PowerShell | Tested |
macOS / Linux + Node.js 20+ (sh + curl) | Per official Node.js & POSIX specs. CI runs only node tools/check.mjs on Node.js 22 (Ubuntu, Windows, macOS); Node.js 20, the plugin hooks and the Claude Code integration are not tested there |
Limitations
- Requires an Anthropic-compatible
/v1/messagesendpoint: Providers that only speak OpenAI Chat Completions require a local translation proxy (and can be supervised viaproviders.<name>.launch). - Switching back to Claude mid-session rebuilds cache: Because prompt caches do not transfer across providers,
/provider claudeon a long conversation will resend the full conversation history to Anthropic. - No upstream proxy: The relay connects to Anthropic and your providers directly, so a network that only lets traffic out through a (company) proxy is not supported yet.
Troubleshooting
Start with /provider doctor --ping: it lists each problem with a ⚠ and the command that fixes it, and /provider setup can run those fixes with you.
| Symptom | What to check |
|---|---|
/provider says config is missing or invalid | Run /provider doctor (or node tools/setup.mjs doctor from the install directory) to verify JSON syntax and check if any env:VAR variables are unset in your shell |
Status bar shows Relay offline: connected directly to Claude | In ANTHROPIC_BASE_URL mode the plugin bypassed a stopped router so your chat keeps working. In proxy mode there is no such bypass: Claude Code cannot connect until the router is restarted. Run /provider doctor or check ~/.claude/model-relay.log (a router that finds its port taken by another process or another config directory logs that there and exits) |
Remote Control says it is only available via api.anthropic.com | Claude Code still uses the relay as its API address; run node tools/setup.mjs proxy enable --go and start a new session |
proxy enable refuses because NODE_EXTRA_CA_CERTS is already set | Something you use (often a company network that inspects TLS) already relies on that certificate, and Claude Code accepts only one file there, so proxy mode would make it stop trusting yours. Remote Control through the relay cannot be combined with it yet; nothing was changed |
| Claude Code cannot connect after enabling proxy mode | The router is probably down: run node tools/setup.mjs proxy status, start it with node tools/setup.mjs rc, or go back with node tools/setup.mjs proxy disable --go |
| Tools are missing on a third-party provider | Set "stripToolSearch": true on that provider in model-relay.json so ToolSearch does not hide other tools |
A model-relay:<name> subagent runs on Claude instead of the provider | The plugin shows a notice when this happens. Usually an availableModels list does not include the provider's model, so Claude Code runs the subagent on CLAUDE_CODE_SUBAGENT_MODEL or the parent's model; add the model ID exactly as configured. node tools/setup.mjs doctor checks the list in ~/.claude/settings.json only |
The status line says the config is from an older version, and doctor says it is outdated | The file is named custom-agent.json or has no "format": 2, so this version's router does not read it. node tools/setup.mjs migrate shows what it would write, and migrate --go writes it, with the router then routing as the older version did (it refuses if the result would route differently); then run /provider restart when no Claude Code window has a reply in progress. See UPGRADING.md |
Uninstall
- Run
claude plugin uninstall model-relay@model-relayand removeANTHROPIC_BASE_URLfrom~/.claude/settings.json(in proxy mode removeHTTPS_PROXY,NODE_EXTRA_CA_CERTSandNO_PROXYinstead; runningnode tools/setup.mjs proxy disable --gofirst does that and putsANTHROPIC_BASE_URLback, which you then remove). - Delete optional state files (
~/.claude/model-relay.json,~/.claude/model-relay-state.json,~/.claude/model-relay-pki/,~/.claude/model-relay.log,~/.claude/model-relay-requests.log, and any*.model-relay-backupcopies).
Run from source
node tools/check.mjs
claude plugin validate .
claude plugin test .
node tools/smoke.mjs # one real Claude Code request through this checkout's router; add --provider <name> for a provider
tools/smoke.mjs needs a logged-in claude, so CI does not run it. Run it before a release and after a Claude Code update.
See CONTRIBUTING.md for architecture and release steps.
More
// faq
What is model-relay?
Multi-provider router and cache-safe 429 failover for Claude Code (Claude, GLM, Gemini, DeepSeek, Kimi, MiniMax, OpenRouter). It is open-source on GitHub.
Is model-relay free to use?
model-relay is open-source under the MIT license, so it is free to use.
What category does model-relay belong to?
model-relay is listed under plugins in the Claudeers registry of Claude-compatible tools.
// embed badge
[](https://claudeers.com/model-relay)
// retro hit counter
[](https://claudeers.com/model-relay)
// reviews
// guestbook
// related in Claude Plugins
A single CLAUDE.md file to improve Claude Code behavior, derived from Andrej Karpathy's observations on LLM coding pitfalls.
Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing routine tasks, explainin…
"CLI-Anything: Making ALL Software Agent-Native" -- CLI-Hub: https://clianything.cc/
financial-services — a Claude ecosystem project on GitHub.