WithHook — skyl

WithHook registers a function called after every attempt against a provider — including retried ones, and including streams the caller abandoned. It is how you get metrics, logging and cost accounting without touching any call site.

Reference#

func WithHook(h Hook) Option

type Hook func(ctx context.Context, ev HookEvent)

Parameters

h — the observer. A nil hook is ignored rather than causing a panic later.

Caveats

Hooks accumulate. Calling WithHook twice registers both, in order.
They run synchronously on the calling goroutine, so a slow hook slows the request. Do metrics and logging; do not do I/O without a timeout.
HookEvent.Request carries the prompt. Logging it verbatim ships conversation content wherever your logs go.
For stream_end, the hook fires from whichever of the terminal event or Close comes first — so on the Close path it runs inside the caller's defer, and its latency lands there.
That event's ctx is the one the stream was opened with, which is frequently already cancelled. A hook that must record it cannot depend on a live context.

Usage#

Metrics

goCompiles

skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	// Group by ResponseModel, not Model — two snapshots behind one alias
	// should not be merged into a single line.
	metrics.Record(ev.Provider, ev.ResponseModel, ev.Duration, ev.Err)
})

skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	// Group by ResponseModel, not Model — two snapshots behind one alias
	// should not be merged into a single line.
	metrics.Record(ev.Provider, ev.ResponseModel, ev.Duration, ev.Err)
})

Logging shape, never content

goCompiles

skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	log.Printf("%s %s attempt=%d msgs=%d dur=%s in=%d out=%d err=%v",
		ev.Provider, ev.Operation, ev.Attempt, len(ev.Request.Messages),
		ev.Duration, ev.Usage.InputTokens, ev.Usage.OutputTokens, ev.Err)
})

skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	log.Printf("%s %s attempt=%d msgs=%d dur=%s in=%d out=%d err=%v",
		ev.Provider, ev.Operation, ev.Attempt, len(ev.Request.Messages),
		ev.Duration, ev.Usage.InputTokens, ev.Usage.OutputTokens, ev.Err)
})

Message count is usually enough to debug, and carries nothing private.

Cost accounting across both call styles

goCompiles

skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	// Only these two carry usage. `stream` fires before generation starts.
	if ev.Operation == skyl.OpComplete || ev.Operation == skyl.OpStreamEnd {
		costs.Add(ev.ResponseModel, ev.Usage)
	}
})

skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	// Only these two carry usage. `stream` fires before generation starts.
	if ev.Operation == skyl.OpComplete || ev.Operation == skyl.OpStreamEnd {
		costs.Add(ev.ResponseModel, ev.Usage)
	}
})

Buffering work that must not block a call

goCompiles

events := make(chan skyl.HookEvent, 1024)

skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	select {
	case events <- ev:
	default:
		// Drop rather than block a model call on a full buffer.
		metrics.Inc("skyl.hook.dropped")
	}
})

events := make(chan skyl.HookEvent, 1024)

skyl.WithHook(func(_ context.Context, ev skyl.HookEvent) {
	select {
	case events <- ev:
	default:
		// Drop rather than block a model call on a full buffer.
		metrics.Inc("skyl.hook.dropped")
	}
})

OpenTelemetry in one line

goCompiles

client := skyl.New(openai.New(key), otel.Hook())

client := skyl.New(openai.New(key), otel.Hook())

The otel module is the reference hook consumer, and records no prompt content by design.

Troubleshooting#

Every request got slower after I added a hook

Hooks run synchronously. If yours writes to a database or makes an HTTP call, it adds that latency to every model call. Buffer with a non-blocking send.

My logs contain users' prompts

You logged ev.Request. Log the shape — provider, model, attempt, message count, duration, usage — and never the messages themselves.

stream_end reports zero usage

Two possibilities. If ev.Completed is false, the caller abandoned the stream and little or no usage arrived — expected. If it is true, the provider did not report usage: on OpenAI-family hosts that means stream_options.include_usage was not honoured.

My hook's context is already cancelled

Expected on the stream_end path — a client hanging up is the ordinary reason a stream ends early. Use a background context for any work that must complete.