On this page
Measure & evaluate
Measure reduction without overstating it.
Token counts, estimated cost and measured provider usage answer different questions.
Define the comparison
Token reduction compares a source representation with the delivered representation under a stated tokenizer. The result depends on the source, read mode, cache state, output processing and recovery requests.
There is no universal percentage for every project or task.
Inspect your own usage
lean-ctx gain
lean-ctx gain --by-tool --json
lean-ctx spend --json
A fresh installation may have no useful measurement yet. Capture a representative workflow before drawing conclusions.
Keep the units separate
| Metric | What it supports |
|---|---|
| Tokens before/after | A representation-size comparison |
| Estimated USD | A calculation using selected prices and assumptions |
| Provider-reported usage/cost | Observed usage on a covered provider path |
| Task completion and quality | A separately defined evaluation outcome |
Provider caching, input/output rates, reasoning, retries and overhead affect billing. Token reduction alone cannot establish invoice savings.
Account for recovery
If an agent retrieves omitted content later, include that delivery in the task total. A small first read followed by several full recoveries can tell a different story from the first-call percentage.
Evaluate with evidence
Local benchmark and scorecard commands inspect representation behavior. They are not automatically matched-workload benchmarks or quality proofs. Declare the task and quality gate before publishing a result.
For integrity records, continue with the savings ledger.
Sources & versions
- 11-analytics-and-insights.mddocs/reference/11-analytics-and-insights.mdReviewed checkout
- config_cmd.rsrust/src/cli/config_cmd.rsReviewed checkout
- mod.rsrust/src/core/stats/format/mod.rsGitHub
- Core checkout
0ce2207ee4- Installed runtime
- 3.10.2
- SDK release
- 1.1.0
Separate baselines for source, CLI/configuration and SDK contracts. Review does not certify every platform or integration.
Versions & compatibility