Token cost and savings
Read what your LLM traffic actually cost, what it would have cost unoptimized, and which of the three savings levers recovered the difference.
- Version: 0.4
- Feature: cost.calculator
- Role: admin_user, viewer
- Type: task
Token Cost & Savings reports what your LLM traffic cost, what the same traffic would have cost
with no optimization, and where the difference came from. Every figure is aggregated from
ai_usage_logs — one row per completion, written by the gateway's own chat pipeline.
What do I need?
- Licence
- Any
- Role
admin_userviewer
Prerequisites
- Traffic has gone through the gateway's chat path. Nothing else on the appliance produces a usage row.
- The models in use have prices on the Models page, or every cost reads as zero.
- You are signed in as an admin_user, the superadmin, or a viewer. A normal_user is refused by the API.
Three levers produce the savings on this page, and only these three:
- Prompt compression reduces the number of input tokens sent to the provider.
- Model routing sends an easy request to a cheaper model — the same tokens at a lower rate.
- Prompt caching bills a repeated prompt prefix below the input rate, using the provider's own cache accounting.
Warning
Only the two chat paths write usage rows. Inline-proxy/ICAP traffic, the browser extension, the endpoint agent and image or document analysis record detections but no usage, so they never appear here. Session summarization is not a lever on this page either — it changes what a later prompt contains, and its effect arrives as compression or as fewer tokens, never as its own figure.
/assets/screenshots/cost-savings@0.4.pngChoose what the page covers
Open the page
Select Administration → Token Cost & Savings.
The page loads the last 30 days across every LLM source you may see.
Narrow to one source
Open the first dropdown and pick a source, or leave it on All sources.
The list is built from the sources that actually produced traffic in the window, so an unused provider is not offered. The choice narrows the tiles, the trend, the models chart and the tables together.
Choose a lever to look at
Open the second dropdown: All levers, Compression, Model switching or Prompt caching.
This is a view filter in the browser — it changes which savings figure each tile and column shows, not which rows were fetched. On Model switching and Prompt caching the Tokens saved tile reads an em dash with the sub-line saves on rate, not tokens, because neither lever removes a token.
Set the period
Pick Last 24 hours, Last 7 days, Last 30 days, Last 90 days, or Custom range… and then a start and end date in the calendar, and select Apply.
A custom range is inclusive of both dates, cannot reach into the future, and is capped at 366
days by the gateway. The header dropdown shows 2026-08-01 → 2026-08-31 for a custom range.
Read the figures
The four tiles across the top answer the whole question:
| Tile | What it is |
|---|---|
| Cost before optimization | The baseline — what the same calls would have cost on the requested model, uncompressed, uncached |
| Cost after optimization | What was actually billed, with the percentage saved, the request count and the number of users |
| Total saved (or Saved — compression / model routing / prompt caching) | Baseline minus actual, for the lever selected |
| Tokens saved | Input tokens compression removed. An em dash when the selected lever saves on rate |
Under them, Daily trend switches between Savings (a stacked bar per lever per day) and Cost vs baseline (two lines: Actual cost against Baseline — no optimization). Days with no traffic are filled in as zero so the axis stays continuous. Savings by lever is the same totals as a donut, Savings by model ranks the top five models by amount saved, and By source and By team list the same per-provider and per-tenant-group figures.
Note
Costs below one cent are printed to six decimal places ($0.000412); anything larger is printed
to two. Per-1M-token pricing makes single calls genuinely that small.
Drill into a user and their projects
Find the user
Scroll to By user & project and select a row.
The table is one row per account, sorted by net cost. Saved % draws the share of baseline recovered.
Read the project list
The dialog lists every project that account owns in the period, with the same token, cost and savings columns.
A project is a chat session. Stateless /chat/completions calls carry no session, so they
belong to no project — the dialog shows that remainder as its own row rather than letting the
projects fail to add up to the user total.
Warning
A user row appears only when the caller is an admin. A By team tile and the Users export
need the same. The viewer role counts as an admin here, deliberately, so the page and its PDF
agree.
Export the report
Open the export menu
Select Export in the page header.
The menu offers PDF report plus CSV slices: Daily breakdown, Models, Projects, and — for an admin — Users. A slice with no rows is disabled.
Take the PDF
Select PDF report.
The gateway re-assembles the whole report server-side under your own scope and renders it through the report renderer, so the file cannot show more than the screen did. Its cover carries the subject, the source filter, the sessions-tenant filter and the period.
Or take a CSV slice
Select one of the CSV entries.
The CSV is written from the rows already loaded in the browser. The filename and the scope line carry the period, the source, and the lever — the lever labelled Lever (view filter), because it narrowed the view and not the data.
Warning
The PDF report action sends the period and the source, and nothing else. Opening one user's
project dialog does not narrow the exported PDF — the gateway supports a tenantId drill on the
export, but the dashboard's menu does not send it.
Fields reference
Header controls
| Control | What it does |
|---|---|
| All sources | Filters every figure to one LLM source. Server-side |
| All levers / Compression / Model switching / Prompt caching | Chooses which savings figure the tiles, charts and columns show. Browser-side view filter |
| Period dropdown | Last 24 hours, Last 7 days, Last 30 days, Last 90 days, Custom range… |
| Export | PDF report (server-rendered) and the four CSV slices |
By user & project columns
| Column | Shows |
|---|---|
| User / project | The account name, or the project name when the table is ungrouped |
| Requests | Completions billed in the period |
| Input / Output / Total | Prompt, completion and total tokens; Total carries a −n saved sub-line |
| Baseline | What the same calls would have cost unoptimized |
| Net cost | What was billed |
| Saved | Baseline minus net, for the selected lever |
| Saved % | That saving as a share of baseline, capped at 100% |
| Last activity | Relative time of the newest call |
What the gateway records per call
Column in ai_usage_logs | Meaning |
|---|---|
cost_usd | What the call was billed |
baseline_cost_usd | What it would have cost on the requested model, uncompressed |
saved_usd | Baseline minus actual — the realized reduction |
input_saved_usd / output_saved_usd / cache_saved_usd | That saving split across compression, routing and provider caching |
projected_saved_usd | What a shadow-mode optimization would have saved. Not counted as a saving |
tokens_saved | Input tokens compression removed |
cache_read_tokens / cache_write_tokens | The provider's own prompt-cache accounting |
Verify
- Send a chat message through the gateway, then reload the page with the period set to Last 24 hours. Cost after optimization moves and the request count rises by one.
- Compare a user row's Requests with the sum of that user's project rows plus the No project row in the dialog. They agree; a difference means stateless calls, which is what the remainder row exists to show.
- Select Export → PDF report and confirm the cover's period matches the period on screen.
- This page writes no event log rows. There is nothing to look for in Event logs — reading a cost report is not audited, in line with every other read endpoint in the gateway.
If it fails
- This feature (cost.calculator) is not included in the installed license tier. Contact your CID
representative to upgrade. — a 403 with the code
FEATURE_NOT_LICENSED. The key ships in all three tiers, so this normally means no licence is installed at all. See Licence. - Could not load the report. Please try again. — the summary request failed. The most common cause is the gateway being unreachable; check it before reading anything into the empty page.
- Every cost reads
$0.00while requests are counted — the models in use have no price recorded. Prices come from the model catalogue; see Models. - No savings in this period on the donut, and No usage in this period on the trend — the window holds no chat traffic, or none of it was optimized. Widen the period before concluding the levers are off.
- The page is empty although the appliance is busy — the traffic is not arriving through the chat path. Inline-proxy, extension and agent traffic produce no usage rows.
- Failed to render cost-savings PDF: … — a 503. The report renderer is down; the JSON views and the CSV exports still work.
- This is a read-only demo account (viewer role) — actions and changes are disabled. — a
READ_ONLY_ROLErefusal. Reads on this page are allowed for a viewer, so this points at some other action in the same session.
Next steps
- AI usage — usage captured outside the chat path, by the extension and the proxy, where no cost is recorded.
- Tenant analytics — the same tokens and costs for one account, with detections and compliance alongside.
- Models — where per-model pricing comes from.
Last updated on