Skip to content
Documentation

Measure & evaluate

Measure reduction without overstating it.

Token counts, estimated cost and measured provider usage answer different questions.

AvailableReviewed September 2026

Define the comparison

Token reduction compares a source representation with the delivered representation under a stated tokenizer. The result depends on the source, read mode, cache state, output processing and recovery requests.

There is no universal percentage for every project or task.

Inspect your own usage

lean-ctx gain
lean-ctx gain --by-tool --json
lean-ctx spend --json

A fresh installation may have no useful measurement yet. Capture a representative workflow before drawing conclusions.

Keep the units separate

Metric What it supports
Tokens before/after A representation-size comparison
Estimated USD A calculation using selected prices and assumptions
Provider-reported usage/cost Observed usage on a covered provider path
Task completion and quality A separately defined evaluation outcome

Provider caching, input/output rates, reasoning, retries and overhead affect billing. Token reduction alone cannot establish invoice savings.

Account for recovery

If an agent retrieves omitted content later, include that delivery in the task total. A small first read followed by several full recoveries can tell a different story from the first-call percentage.

Evaluate with evidence

Local benchmark and scorecard commands inspect representation behavior. They are not automatically matched-workload benchmarks or quality proofs. Declare the task and quality gate before publishing a result.

For integrity records, continue with the savings ledger.

Sources & versions3 references Reviewed
Core checkout
0ce2207ee4
Installed runtime
3.10.2
SDK release
1.1.0

Separate baselines for source, CLI/configuration and SDK contracts. Review does not certify every platform or integration.

Versions & compatibility