Skip to content
skyl

Hooks

Four operations, and the one field that will leak your users' prompts.

A hook observes every attempt against a provider — including retried ones, and including streams the caller abandoned. It is how you get metrics, logging and cost accounting without touching any call site.

You will learn

  • The four operations and when each fires
  • Why stream_end fires even for an abandoned stream
  • The field that carries the prompt, and what that means for your logs
  • Why hooks must be cheap

Registering one#

goCompiles
client := skyl.New(p,
	skyl.WithHook(func(ctx context.Context, ev skyl.HookEvent) {
		metrics.Record(ev.Provider, ev.ResponseModel, ev.Duration, ev.Err)
	}),
)
client := skyl.New(p,
	skyl.WithHook(func(ctx context.Context, ev skyl.HookEvent) {
		metrics.Record(ev.Provider, ev.ResponseModel, ev.Duration, ev.Err)
	}),
)

Hooks accumulate — calling WithHook twice registers both.

The four operations#

OperationWhen it firesCarries usage?
completeOnce per attempt of a non-streaming requestYes, on success
streamThe streaming handshake — as soon as the provider accepts, before any token existsNo
stream_endOnce when a stream finishes or is closedYes, if it completed
modelsOnce per attempt of a model-listing requestNo

stream carries no usage because it fires before generation starts. That is why stream_end exists.

stream_end fires for abandoned streams too#

goCompiles
skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	if ev.Operation == skyl.OpStreamEnd && !ev.Completed {
		metrics.Inc("skyl.stream.abandoned", "provider", ev.Provider)
	}
})
skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	if ev.Operation == skyl.OpStreamEnd && !ev.Completed {
		metrics.Inc("skyl.stream.abandoned", "provider", ev.Provider)
	}
})

Two mechanics worth knowing:

  • It fires from whichever of the terminal event or Close comes first. On the Close path the hook runs inside the caller's defer, so its latency lands there.
  • The ctx it receives is the one the stream was opened with, which is frequently already cancelled — a client hanging up is the ordinary reason a stream is abandoned. A hook that needs to record the event must not depend on that context being live.

The event#

FieldTypeDescription
ProviderstringThe adapter that was called.
ModelstringThe model that was *asked for*. See ResponseModel for the one that answered.
OperationstringOne of complete, stream, stream_end, or models.
AttemptintThe zero-based retry attempt this event reports.
Durationtime.DurationHow long the attempt took. For stream that is the handshake alone; for stream_end it is the whole stream, handshake included.
ErrerrorThe attempt's error, or nil. A non-nil Err on a non-final attempt was retried.
UsageUsageToken consumption. Populated for successful complete calls, and for stream_end when the stream ran to completion.
ResponseIDstringThe provider's identifier for the response, when it gave one.
ResponseModelstringThe model that actually served the request.
StopReasonStopReasonWhy generation ended, for complete and a completed stream_end.
CompletedboolWhether a stream ran to its terminal event. Meaningful only for stream_end. False means the caller closed the stream early — those tokens were still generated and still billed, which is why the event fires anyway.
Request*RequestThe request that produced this event. **It carries the prompt.** Anything a hook does with it is a decision about user data: logging it verbatim ships conversation content wherever the logs go. Treat it as read-only; skyl reuses it across retries.

Note Model versus ResponseModel: the first is what you asked for, the second is what answered. Group metrics by the second, or you merge snapshots. See Which Model Actually Answered.

The prompt problem#

Deep diveWhy give hooks the prompt at all?

Because the OpenTelemetry GenAI conventions ask for temperature, top_p and max_tokens on every span, and there are only two ways to provide them: add a field to HookEvent for every sampling parameter, forever, or hand the hook the request.

Handing over the request keeps the struct stable as skyl grows, and lets a hook report anything about the request's shape. The cost is that the prompt comes with it — which is a real hazard, so it is documented on the field itself rather than buried.

The otel module, which is the reference consumer of this, deliberately records no prompt content: only model, sampling parameters and token counts. A span is a durable record shipped to a third-party backend, and putting user conversations there by default is not a decision a library should make silently.

Treat Request as read-only. skyl reuses it across retries.

Hooks must be cheap#

Hooks run synchronously on the calling goroutine, so a slow hook slows the request. Do metrics and logging; do not do I/O without a timeout.

// DON'T: this adds a network round trip to every model call.
skyl.WithHook(func(ctx context.Context, ev skyl.HookEvent) {
	db.Exec("INSERT INTO usage …")
})
// DON'T: this adds a network round trip to every model call.
skyl.WithHook(func(ctx context.Context, ev skyl.HookEvent) {
	db.Exec("INSERT INTO usage …")
})

If you need to persist, buffer:

goCompiles
events := make(chan skyl.HookEvent, 1024)

skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	select {
	case events <- ev:
	default:
		// Drop rather than block a model call on a full buffer.
		metrics.Inc("skyl.hook.dropped")
	}
})
events := make(chan skyl.HookEvent, 1024)

skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	select {
	case events <- ev:
	default:
		// Drop rather than block a model call on a full buffer.
		metrics.Inc("skyl.hook.dropped")
	}
})

The default branch is the point: a hook that blocks when your buffer fills turns a persistence problem into a latency problem for every user.

Recap

  • Four operations: complete, stream, stream_end, models.
  • stream fires before any token exists, so it carries no usage.
  • stream_end fires even for abandoned streams, with Completed: false.
  • Its context is often already cancelled — do not depend on it being live.
  • HookEvent.Request carries the prompt. Log shape, never content.
  • Hooks run synchronously; buffer with a non-blocking send if you must persist.

Try out some challenges

Each of these is solvable with what this page covered. Run them against the sandbox — no API key needed.

Build a cost tracker

Accumulate spend per model across a process, from streaming and non-streaming calls alike.

Show hint

Non-streaming usage arrives on complete; streaming usage arrives on stream_end. Counting both without double-counting is the whole trick.

Show solution
type costs struct {
	mu sync.Mutex
	by map[string]skyl.Usage
}

func (c *costs) hook(_ context.Context, ev skyl.HookEvent) {
	// Only these two carry usage. `stream` fires before generation, and a
	// failed attempt has nothing to add.
	if ev.Operation != skyl.OpComplete && ev.Operation != skyl.OpStreamEnd {
		return
	}
	if ev.Err != nil && ev.Usage.TotalTokens() == 0 {
		return
	}

	key := ev.ResponseModel
	if key == "" {
		key = ev.Model // the provider did not report; fall back
	}

	c.mu.Lock()
	defer c.mu.Unlock()
	c.by[key] = c.by[key].Add(ev.Usage)
}
type costs struct {
	mu sync.Mutex
	by map[string]skyl.Usage
}

func (c *costs) hook(_ context.Context, ev skyl.HookEvent) {
	// Only these two carry usage. `stream` fires before generation, and a
	// failed attempt has nothing to add.
	if ev.Operation != skyl.OpComplete && ev.Operation != skyl.OpStreamEnd {
		return
	}
	if ev.Err != nil && ev.Usage.TotalTokens() == 0 {
		return
	}

	key := ev.ResponseModel
	if key == "" {
		key = ev.Model // the provider did not report; fall back
	}

	c.mu.Lock()
	defer c.mu.Unlock()
	c.by[key] = c.by[key].Add(ev.Usage)
}

Two details that matter. Keying on ResponseModel means two snapshots behind one alias are billed separately, which is what you want. And not returning early on ev.Err != nil when usage is non-zero captures the abandoned-stream case, where tokens were billed despite the failure.

Edit this page on GitHub