token-optimizer

Cut the tokens you waste. Keep the work you'd lose.

Optimize every session, then watch the dollars add up on the dashboard.

Most token tools only touch one slice of the problem.

They compress command output, which covers 15-25% of your context on a good day. The other 75-85% (bloated configs, unused skills, duplicate system prompts, stale memory, plus the 60-70% you lose on every compaction) goes untouched.

Token Optimizer covers all of it, keeps your work alive across compactions, measures whether the optimization actually helped, and gives you a live dashboard that shows every token, every dollar, and every turn, auto-updated after every session. Runs fully local. Zero baseline context overhead. Zero runtime dependencies.

Works on Claude Code (CLI and VS Code), OpenCode, OpenClaw, Codex, Hermes, and GitHub Copilot (CLI and VS Code, beta) today. Windsurf, Cursor, and more on the way.

TL;DR

Install the plugin. Most of it runs automatically from that point:

Install, run the audit once, everything else just works.

Install

Recommended on every platform (macOS, Linux, Windows):

/plugin marketplace add alexgreensh/token-optimizer
/plugin install token-optimizer@alexgreensh-token-optimizer

Then in Claude Code: /token-optimizer

Please enable auto-update after installing. Claude Code ships third-party marketplaces with auto-update off by default, and plugin authors cannot change that default. So you won't get bug fixes automatically unless you turn it on. In Claude Code: /plugin → Marketplaces tab → select alexgreensh-token-optimizer → Enable auto-update. One-time, 10 seconds, and you'll never miss a fix again. Token Optimizer also prints a one-time reminder on your first SessionStart so you don't forget.

tmp="$(mktemp -d)"
release_json="$(curl -fsSL https://api.github.com/repos/alexgreensh/token-optimizer/releases/latest)"
tag="$(python3 -c 'import json,sys; print(json.load(sys.stdin)["tag_name"])' <<<"$release_json")"
checksums="$(python3 -c 'import json,sys; data=json.load(sys.stdin); print(next(a["browser_download_url"] for a in data["assets"] if a["name"]=="CHECKSUMS.sha256"))' <<<"$release_json")"
git clone --branch "$tag" --depth 1 https://github.com/alexgreensh/token-optimizer.git ~/.claude/token-optimizer
curl -fsSL -o "$tmp/CHECKSUMS.sha256" "$checksums"
install_sum="$(grep '  install.sh$' "$tmp/CHECKSUMS.sha256")"
(cd ~/.claude/token-optimizer && (printf '%s\n' "$install_sum" | sha256sum -c - --quiet 2>/dev/null || printf '%s\n' "$install_sum" | shasum -a 256 -c - --quiet))
bash ~/.claude/token-optimizer/install.sh
rm -rf "$tmp"

codex plugin marketplace add alexgreensh/token-optimizer

TOKEN_OPTIMIZER_RUNTIME=codex python3 skills/token-optimizer/scripts/measure.py codex-install --project "$PWD"
TOKEN_OPTIMIZER_RUNTIME=codex python3 skills/token-optimizer/scripts/measure.py setup-daemon

opencode plugin token-optimizer-opencode

{
  "plugin": ["token-optimizer-opencode"]
}

git clone https://github.com/alexgreensh/token-optimizer.git
token-optimizer/install.sh --opencode

# From GitHub (recommended)
openclaw plugins install github:alexgreensh/token-optimizer

# From ClawHub
openclaw plugins install token-optimizer

git clone https://github.com/alexgreensh/token-optimizer.git
token-optimizer/install.sh --hermes

token-optimizer/install.sh --hermes --dry-run   # preview
python3 token-optimizer/skills/token-optimizer/scripts/measure.py hermes-doctor

git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
cd token-optimizer
bash install.sh --copilot

bash install.sh --copilot --dry-run   # preview without writing anything
TOKEN_OPTIMIZER_RUNTIME=copilot python3 skills/token-optimizer/scripts/measure.py copilot-doctor

TOKEN_OPTIMIZER_RUNTIME=copilot python3 ~/.claude/skills/token-optimizer/scripts/measure.py copilot-install

Full Visibility: See Every Token, Every Dollar, Every Turn

Most tools tell you your context is full. Token Optimizer shows you exactly where every token went, how much each turn cost, which skills and MCP servers actually fired, and which ones are just sitting there eating your budget.

One single-file HTML dashboard. Auto-regenerates after every session via the SessionEnd hook. Bookmark http://localhost:24842/token-optimizer and it's always current. Zero baseline context overhead, no telemetry, zero setup after install.

What the dashboard tracks

• Per-turn token breakdown for every API call: input, output, cache-read, cache-write, with spike detection highlighting context jumps
• Cache analysis: stacked bars showing input vs output vs cache-read vs cache-write split, with TTL mix (1h vs 5m) and hit rate alongside
• Pacing metrics between calls so you can see whether a thread was steady or stop-start
• Cost across 4 pricing tiers: Anthropic API, Vertex Global, Vertex Regional, AWS Bedrock. Set your tier once and every session updates
• Color-coded quality scores overlaid on every session: green healthy, yellow degrading, red trouble
• Subagent cost breakdown: orchestrator vs worker spend, top offenders ranked by cost, flags when subagents consume over 30%
• Top 5 costliest prompts per session, pairing each user message with the cost of the response
• Skill adoption trends: which skills you actually invoke vs just having installed
• Model mix over time: Opus, Sonnet, Haiku breakdown across every session
• CLAUDE.md and MEMORY.md health cards on the Overview tab with line count, orphan count, and status at a glance
• Drift detection: config snapshots compared across time so you catch creep before it costs you
• Savings tracker: cumulative dollars saved from optimizations, checkpoint restores, and tool-output replacements

/context shows a capacity bar. Proxy compressors print a terminal report. Token Optimizer shows the receipts, auto-updated, with zero baseline context overhead.

Launch it

python3 measure.py setup-daemon           # Bookmarkable URL at http://localhost:24842/token-optimizer
python3 measure.py dashboard --serve      # One-time serve over HTTP

By default the dashboard binds to localhost, reachable only from your own machine. On a headless or shared box where you want LAN access, set TOKEN_OPTIMIZER_DASHBOARD_HOST=0.0.0.0 before running setup-daemon:

TOKEN_OPTIMIZER_DASHBOARD_HOST=0.0.0.0 python3 measure.py setup-daemon

setup-daemon now honors this variable (the older dashboard --serve path already did). Because the daemon runs under launchd / systemd / Task Scheduler with an empty environment, the host you choose is persisted to a small dashboard-host file next to the daemon, so the background service binds it correctly. This setting is per-runtime: each daemon (Claude, Codex, Hermes, Copilot) persists its own host, so run setup-daemon under each runtime you want to expose. The setting is sticky: re-running setup-daemon without the variable (for example during an upgrade) keeps your last choice. It is cleared only when you change it or run setup-daemon --uninstall. Allowed values are 127.0.0.1, localhost, and 0.0.0.0; anything else is rejected with a warning and falls back to localhost. In network mode the dashboard is view-only for LAN visitors: the token endpoint is loopback-locked, so the per-install token that gates every toggle can only be fetched from the machine itself. LAN visitors can view the dashboard; skill and MCP toggles work only from the machine running the daemon.

Throughout this README, whenever a feature mentions it's also visible on the dashboard, that means it lives inside this same HTML page. One place, everything tracked.

What Makes This Different

Three kinds of token waste, and most tools fix one

Structural waste: bloated CLAUDE.md, unused skills, duplicate system reminders, stale MEMORY.md, invisible entries past line 200, dead MCP servers. Often the biggest share in high-waste setups, and it compounds, since a leaner prefix means a smaller cache-read bill on every turn that follows. Almost nobody touches this.

Runtime waste: verbose command output, oversized MCP results, and re-read files that flood your context mid-session. The slice proxy compressors handle.

Behavioral waste: the habits that quietly burn tokens. Letting the cache expire, compacting too late, looping on a failing approach, running Opus where Haiku would do, switching models mid-session and killing your cache. Token Coach analyzes 30 days of your session history to surface patterns no single session reveals: quality trending down, sessions creeping longer, cache hit rates falling, cost per session climbing. It even distinguishes when a cache drop is caused by a model switch (expected) vs. a config change (fixable).

Token Optimizer covers all three. And because it checkpoints your session before compaction fires and restores what the summary dropped, the savings stick instead of vanishing the moment auto-compact kicks in.

Fully local, zero dependencies, zero telemetry

Pure Python stdlib on Claude Code and Codex. TypeScript with zero runtime deps on OpenCode and OpenClaw. Nothing to pip install, no analytics endpoint, no phone-home. Every measurement is a local SQLite write to a file you own under your runtime home. You can inspect it, export it, or delete it.

Zero baseline context overhead

Token Optimizer runs as an external process. It doesn't inject always-on instructions into your context and it doesn't add MCP overhead. Optional quality nudges and checkpoint restore hints are short, event-triggered messages when they are useful; idle overhead stays zero.

/context shows the dashboard light. Token Optimizer opens the hood.

/context tells you that your context is 73% full. Token Optimizer tells you which 12K are wasted on skills you never use, flags 47 orphaned MEMORY.md topic files Claude can't see, checkpoints your decisions before compaction destroys them, and gives you a quality score that tracks how much dumber your AI is getting as the session wears on.

How it compares

Every claim in this table is tested against real sessions and a 57-fixture compression suite you can run yourself. See the full benchmark methodology and results.

What It Saves

Savings come from all three waste types. Some are measured from real token deltas, some are estimated from your baselines and current pricing, and some are enabled (a setting Token Optimizer recommends, like an MCP output cap, that does the saving once you apply it). The dollar figures are API-equivalent: literal cash if you pay per token (API, Bedrock, Vertex), and capacity plus quality if you are on a flat subscription (more work per session, fewer compaction wipeouts, less drift before you hit a limit).

Pricing uses the current published Anthropic rates used by the modeled Opus profile: $5 input, $25 output, $0.50 cache-read, $6.25 5m cache-write, and $10 1h cache-write per million tokens.

What different users can expect (modeled estimates)

These are modeled from the volume, structural waste trimmed, cache-hit rate, and model mix, not guarantees. Your number is your own.

Estimated savings for a heavy user

Modeled on a real heavy-Opus usage profile from local trends data: 428 sessions, 12,430 API calls, and 2.23B total input mix over 30 days. The measured split is 8.7M fresh input, 2.13B cache-read input, 88.5M cache-write input, and 9.65M output. The structural line uses a historical starting-context baseline of ~45K tokens before the first user message, with ~15K-20K prefix tokens trimmed by Token Optimizer. Estimated potential value across the three pillars:

In that historical-baseline scenario, structural is the giant: the diagram models how a leaner prefix can reduce future input, cache-write, and cache-read work. Runtime compression and behavioral coaching add on top. Estimated potential, not a guarantee. On a flat subscription it shows up as capacity and quality rather than cash.

Trust & Safety FAQ

# Download checksums from the latest release
curl -sL $(gh release view --json assets -q '.assets[] | select(.name=="CHECKSUMS.sha256") | .url') -o /tmp/checksums.sha256

# Verify your install
cd ~/.claude/token-optimizer && shasum -a 256 -c /tmp/checksums.sha256

Why install this first

Every Claude Code session starts with invisible overhead: system prompt, tool definitions, skills, MCP servers, CLAUDE.md, MEMORY.md. A typical power user burns 50-70K tokens before typing a word.

With Opus 4.6 and Sonnet 4.6 now at 1M context, that feels like breathing room. The problems still compound:

• Quality degrades as context fills. MRCR drops from 93% to 76% between 256K and 1M. Your AI gets measurably dumber as the window fills.
• Rate limits hit faster. Ghost tokens count toward your plan's usage caps on every message, cached or not. 50K overhead times 100 messages is 5M tokens burned on nothing.
• Compaction is catastrophic. 60-70% of your conversation gone per compaction. After 2-3 compactions, you've lost 88-95%. And each compaction means re-sending all that overhead again.
• Higher effort means faster burn. More thinking tokens per response means you hit compaction sooner, which means more total tokens across the session.
• You can't fix what you can't see. Without per-turn visibility into cache hits, model mix, and subagent spend, every "it feels slow" guess costs money. The dashboard shows exactly which turn was the expensive one.

Token Optimizer tracks all of this. Quality score, degradation bands, compaction loss, drift detection, per-turn cost across four pricing tiers, and skill-and-MCP attribution for every session. Zero baseline context overhead.

"But doesn't removing tokens hurt the model?" No. Token Optimizer only touches what's safe to touch. Structural optimization removes genuinely unused components (duplicate configs, unused skill frontmatter, orphaned memory entries), never the conversation itself. Active Compression works on new content entering your window (smart re-reads, credential-safe command summaries) and on the compaction boundary (checkpoints before auto-compact, restore after). Nothing already in your context gets edited or removed, which means your prompt cache stays intact. The context-quality scoring system tracks degradation in real time, and most users see scores improve after optimization because the model has more room for real work.

Smart Compaction and Session Continuity

When auto-compact fires, 60-70% of your conversation vanishes. Decisions, error-fix sequences, agent state, all gone.

Smart Compaction catches all of it as checkpoints before compaction fires, then restores what the summary dropped. It also injects a digest of large tool outputs the model previously processed, so after compaction the model knows what it already saw without re-reading everything from scratch. Sessions pick up where you left off, even after a crash or /clear. Checkpoint history and compaction loss per session are also visible on the dashboard.

Compression savings only stick if your session survives the compaction. Saving tokens on git status doesn't help if the next auto-compact wipes out the decision that made you run git status in the first place. Smart Compaction closes that loop: checkpoint your decisions, restore them after compaction, and remind the model what outputs it already processed so it doesn't waste tokens re-reading them.

python3 measure.py setup-smart-compact    # checkpoint + restore hooks

Progressive Checkpoints

Instead of waiting for emergency compaction, Token Optimizer captures session state at multiple thresholds: 20%, 35%, 50%, 65%, and 80% context fill, plus quality drops below 80, 70, 50, and 40. It also snapshots before agent fan-out and after large edit batches. On restore, it picks the richest eligible checkpoint, not just the most recent one.

Background guards handle one-shot threshold capture, cooldown suppression, and deterministic extraction. No LLM calls in the checkpoint path.

Tool Result Archive (model-aware, no manual lookups)

Large MCP tool results (>4KB) get archived to disk automatically and replaced with a short preview plus an inline hint like [Full result archived (12,400 chars). Use 'expand abc123' to retrieve.]

That hint is visible to Claude, not just you. So after a compaction (when the original tool result has been summarized away), if the model needs the full output again to answer your next question, it invokes expand abc123 itself and the archived content comes back through the CLI. No command re-run, no lost output, no context cost in the meantime.

Native Bash/Read/Grep-style outputs are archived for continuity too, but Claude's current PostToolUse API cannot replace those already-returned results. Those durability archives are not counted as token savings.

You can run expand yourself too when you want to see a specific archived result, but the primary flow is automatic: the model sees the hint, the model asks for the bytes, the bytes come back.

python3 measure.py expand --list                 # List all archived tool results
python3 measure.py expand <tool-use-id>          # Retrieve a specific archived result manually

Session Continuity

Sessions auto-checkpoint on end, /clear, and crashes. On a fresh session, Token Optimizer drops a short in-context pointer to the most recent relevant checkpoint, so Claude can pull the right prior state on its own if the new conversation needs it. No auto-replay of stale context, no user action required, just a breadcrumb the model can follow when it matters.

Enable optional local-only checkpoint telemetry to see whether checkpoints are firing and which triggers are active:

TOKEN_OPTIMIZER_CHECKPOINT_TELEMETRY=1 python3 measure.py checkpoint-stats --days 7

Cold-Resume-Lean: reopen a stale session for free

Long sessions degrade. The usual move is to keep grinding in a bloated, low-quality context or to --resume, which re-loads the entire transcript at full cost. Token Optimizer adds a third option: let the old session go, then reopen it lean.

Open a fresh session and just ask for it in plain language:

"Let's continue the auth refactor — check what we discussed last session."

Token Optimizer reconstructs a lean context for the right same-project prior session (active task, where you left off, key decisions, modified files, git state) straight from its checkpoint — no LLM call, no full-transcript cold-resume. Name a topic and it picks that session; stay vague ("where we left off") and it grabs the most recent one in the project. It never pulls another project's context. The only cost is your new session's normal first turn reading the injected summary, typically ~160K cache-write tokens cheaper than a --resume.

You can also drive it manually:

python3 measure.py resume-lean                       # list reopenable cold sessions
python3 measure.py resume-lean <#|session_id> --print # emit the lean context block
claude "$(python3 measure.py resume-lean 1 --print)"  # seed a fresh session with it

Fresh-session nudge. When a session runs long (high context fill) and quality drops below 70, Token Optimizer nudges you once: it tells you how many tokens (and the API-$ equivalent) you'd reclaim by starting fresh right now, and reassures you that your task, decisions, and files are already checkpointed so a new session picks up exactly where you stopped. The reclaimed savings are tracked as a realized Lean resumes line in the Savings view. Tune or disable via TOKEN_OPTIMIZER_FRESH_NUDGE_QUALITY (default 70) and TOKEN_OPTIMIZER_FRESH_NUDGE_MIN_FILL (default 50).

Quality Scoring

The quality score reports two composites: Resource Health for monotonic session risk, and Session Efficiency for behavior that can improve or regress during the session.

Efficiency Grades

Every quality score includes a letter grade for quick triage. The status line shows something like ContextQ:A(82), and the same grade appears in the dashboard, coach tab, and CLI output.

Degradation Bands

The status bar shifts color as your context fills:

• Green (<50% fill): peak quality zone
• Yellow (50-70%): degradation starting
• Orange (70-80%): quality dropping
• Red (80%+): severe, consider /clear

What Degradation Actually Looks Like

Real session. 708 messages, 2 compactions, 88% of the original context gone. Without the quality score, you'd have no idea.

Active Compression (v5)

Token Optimizer no longer just measures context bloat. It actively reduces it. Seven features target specific waste patterns, each with honest risk assessment and dashboard toggles.

On by default: Quality Nudges, Loop Detection, Delta Mode, Structure Map, Bash Compression (16 handlers), Activity Mode Detection, Decision Extraction.

Five compression controls are toggleable from the Manage tab in the dashboard, via CLI (measure.py v5 enable|disable <feature>), or with environment variables. Activity Mode and Decision Extraction are hook-managed continuity features.

Privacy note: Every feature runs on your machine. No analytics endpoint, no phone-home, no cloud sync. "Measurement" and "beta telemetry" always mean local-only SQLite writes to a file you own, and you can inspect, export, or delete that file at any time. Network use is limited to install/update checks against GitHub or package registries for the install method you chose.

Quality Nudges (ON by default, fully automatic)

Watches your context quality in real time. When the score drops 15+ points or crosses below 60, an inline system note enters the context that reads something like [Token Optimizer] Quality dropped to 58. Consider /compact to protect context.

Claude sees that note on the next turn and surfaces the warning to you naturally, or adjusts behavior on its own. You don't have to watch a dashboard or remember thresholds. The nudge shows up right where decisions get made, with zero setup after install.

Value: catches context rot early so /compact lands at the right moment, before you lose decisions to compaction.

How it works: runs inside the existing quality-cache hook on every UserPromptSubmit. Cooldown of 5 minutes between nudges, max 3 per session. Suppressed on the first check after a compaction, so you don't get warned about quality you just fixed.

Risk: none. Only adds a short note to context, never removes anything.

Loop Detection (ON by default, fully automatic)

Catches the AI getting stuck on a retry loop before it burns through tokens. When similarity crosses the threshold, a short inline note lands in the context flagging the loop so the model breaks out of it, with no user action needed. Savings are measured from the actual content of the looping turns, not estimated.

Value: post-hoc detectors found that loop sessions average 47K wasted tokens. Real-time detection prevents this. Every caught loop logs the measured token cost of the loop turns to your local telemetry.

How it works: compares the last 4 user messages and last 5 tool results for similarity. Fires at confidence ≥0.7 with a session cap of 2 notes. Uses fixed message templates and never echoes user content back.

Risk: none. Only adds a short note.

Delta Mode (ON by default, your biggest single win)

When the AI re-reads a file after editing it, the Read call returns only what changed instead of the whole file. Fully automatic, no configuration, no user action. 65%+ of Read calls in real sessions are re-reads, which makes this the highest-impact v5 feature.

Value: typical sessions re-read the same file 2-5 times. Delta mode sends only the diff. A 2,000-token file re-read becomes a 50-token diff, for 97% savings on that specific read.

…view the full README on GitHub.