Token Usage and Caching

Providers disagree about whether cached tokens are part of the prompt count. Summing the fields blindly over-reports on some providers and under-reports on others — so skyl normalises to one rule that every adapter obeys. This page is that rule.

You will learn

The inclusion rule, and why it exists
What each provider reports on the wire, before normalisation
How to accumulate usage across a conversation
Which token counts skyl cannot give you

The rule#

Field	Type	Description
InputTokens	int	Every token of input, including any served from or written to a cache. This is what you are billed for. Zero value: not reported
OutputTokens	int	Every token the model generated. Reasoning tokens are inside this figure — except on Gemini, where they are excluded entirely and this under-reports. Zero value: not reported
CacheReadTokens	int	Tokens served from a prompt cache, usually at a large discount. Part of InputTokens, not additional to it — so this is how much of your bill was discounted. Zero value: not reported
CacheWriteTokens	int	Tokens written to a prompt cache, usually at a premium. Part of InputTokens. Only Anthropic reports this; elsewhere it is always zero. Zero value: not reported

Stated once, plainly:

InputTokens is the total input, cached tokens included. It is what you are billed for.
CacheReadTokens and CacheWriteTokens are a breakdown OF InputTokens, not an addition to it. CacheReadTokens is how much of your bill was discounted.

Therefore:

func (u Usage) TotalTokens() int {
	return u.InputTokens + u.OutputTokens   // cache is NOT added again
}

func (u Usage) TotalTokens() int {
	return u.InputTokens + u.OutputTokens   // cache is NOT added again
}

Why this needed normalising#

Deep diveWhat each provider actually sends

OpenAI and Gemini report a cache figure that is a subset of the prompt count. Their prompt_tokens already includes the cached tokens.

Anthropic reports cache figures that are disjoint from its input count. Its input_tokens excludes them entirely.

So before normalisation, the same cached conversation reported a different billable input depending on which provider served it. Naively adding InputTokens + CacheReadTokens over-reported on OpenAI and Gemini by the size of the cache, while InputTokens alone under-reported on Anthropic by the same amount.

The adapters now converge: Anthropic's adapter adds the cache figures in, the others copy the prompt count as-is. One rule, four adapters.

Accumulating#

goCompiles

var total skyl.Usage

for {
	resp, err := client.Complete(ctx, req)
	if err != nil {
		return err
	}
	total = total.Add(resp.Usage)
	// …
}

fmt.Printf("%d in / %d out / %d total\n",
	total.InputTokens, total.OutputTokens, total.TotalTokens())
fmt.Printf("%d of the input was served from cache\n", total.CacheReadTokens)

var total skyl.Usage

for {
	resp, err := client.Complete(ctx, req)
	if err != nil {
		return err
	}
	total = total.Add(resp.Usage)
	// …
}

fmt.Printf("%d in / %d out / %d total\n",
	total.InputTokens, total.OutputTokens, total.TotalTokens())
fmt.Printf("%d of the input was served from cache\n", total.CacheReadTokens)

Add sums all four fields, so the inclusion rule survives accumulation.

Zero means "not reported"#

Streaming usage#

Usage arrives on the terminal EventDone:

goCompiles

for stream.Next() {
	if ev := stream.Event(); ev.Type == skyl.EventDone && ev.Usage != nil {
		total = total.Add(*ev.Usage)
	}
}

for stream.Next() {
	if ev := stream.Event(); ev.Type == skyl.EventDone && ev.Usage != nil {
		total = total.Add(*ev.Usage)
	}
}

For a stream the caller abandoned, the stream_end hook still fires with whatever usage arrived — usually nothing. Those tokens were generated and billed regardless, so reporting nothing would make that spend invisible. See Hooks.

What skyl cannot tell you#

Reasoning-token counts are not surfaced. They sit inside OutputTokens on Anthropic and the OpenAI family.

On Gemini it is worse: thoughtsTokenCount is excluded from candidatesTokenCount, so OutputTokens genuinely under-reports what you are billed. If you run reasoning models on Gemini and need accurate cost, read the counter from Response.Raw:

goCompiles

var raw struct {
	UsageMetadata struct {
		ThoughtsTokenCount int `json:"thoughtsTokenCount"`
	} `json:"usageMetadata"`
}
if err := json.Unmarshal(resp.Raw, &raw); err == nil {
	billed := resp.Usage.OutputTokens + raw.UsageMetadata.ThoughtsTokenCount
	_ = billed
}

var raw struct {
	UsageMetadata struct {
		ThoughtsTokenCount int `json:"thoughtsTokenCount"`
	} `json:"usageMetadata"`
}
if err := json.Unmarshal(resp.Raw, &raw); err == nil {
	billed := resp.Usage.OutputTokens + raw.UsageMetadata.ThoughtsTokenCount
	_ = billed
}

Recap

InputTokens includes cached tokens; the cache fields break it down.
TotalTokens() is input plus output — never add the cache figures again.
Providers disagree on the wire; the adapters normalise so you do not have to.
Zero means "not reported". CacheWriteTokens is Anthropic-only.
Streaming usage on OpenAI needs the host to honour stream_options.include_usage.
Reasoning tokens are not surfaced, and on Gemini OutputTokens under-reports.

Try out some challenges

Each of these is solvable with what this page covered. Run them against the sandbox — no API key needed.

Compute a cache hit rate

Report what fraction of your input tokens were served from cache, across a run.

Show hint

The inclusion rule makes this simpler than it looks — no subtraction needed.

Show solution

goCompiles

func cacheRate(u skyl.Usage) float64 {
	if u.InputTokens == 0 {
		return 0
	}
	// CacheReadTokens is part of InputTokens, so this is a direct ratio.
	return float64(u.CacheReadTokens) / float64(u.InputTokens)
}

func cacheRate(u skyl.Usage) float64 {
	if u.InputTokens == 0 {
		return 0
	}
	// CacheReadTokens is part of InputTokens, so this is a direct ratio.
	return float64(u.CacheReadTokens) / float64(u.InputTokens)
}

If the inclusion rule were the other way round, this would be CacheRead / (Input + CacheRead) — and you would have to know which provider served it to get it right. That is exactly the difference normalisation buys.

Detect silently missing streaming usage

Warn when a streaming call reports zero usage, so a compatible host that ignores include_usage does not quietly zero your cost report.

Show hint

A completed stream that produced text but reports zero input tokens is suspicious.

Show solution

goCompiles

skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	if ev.Operation == skyl.OpStreamEnd && ev.Completed && ev.Usage.InputTokens == 0 {
		log.Printf("warning: %s reported no usage for a completed stream; "+
			"the host may not honour stream_options.include_usage", ev.Provider)
	}
})

skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	if ev.Operation == skyl.OpStreamEnd && ev.Completed && ev.Usage.InputTokens == 0 {
		log.Printf("warning: %s reported no usage for a completed stream; "+
			"the host may not honour stream_options.include_usage", ev.Provider)
	}
})

Gating on ev.Completed avoids a false warning for streams the caller abandoned, where zero usage is expected.