Hook — skyl

Hook is the function type WithHook registers. It fires after every attempt against a provider — including retried ones, and including streams the caller abandoned.

Reference#

type Hook func(ctx context.Context, ev HookEvent)

Parameters

ctx — the context the operation was started with. For stream_end it is frequently already cancelled.
ev — the HookEvent describing the attempt.

Caveats

Hooks run synchronously on the calling goroutine. A slow hook slows the request. Do metrics and logging; do not do I/O without a timeout.
ev.Request carries the prompt. Logging it verbatim ships conversation content wherever your logs go.
Hooks accumulate — registering two runs both, in order.
For stream_end, the hook fires from whichever of the terminal event or Close comes first, so on the Close path it runs inside the caller's defer and its latency lands there.
Treat ev.Request as read-only; skyl reuses it across retries.

Usage#

Metrics

goCompiles

var hook skyl.Hook = func(_ context.Context, ev skyl.HookEvent) {
	metrics.Record(ev.Provider, ev.ResponseModel, ev.Duration, ev.Err)
}

client := skyl.New(p, skyl.WithHook(hook))

var hook skyl.Hook = func(_ context.Context, ev skyl.HookEvent) {
	metrics.Record(ev.Provider, ev.ResponseModel, ev.Duration, ev.Err)
}

client := skyl.New(p, skyl.WithHook(hook))

Composing several

goCompiles

func chain(hooks ...skyl.Hook) skyl.Hook {
	return func(ctx context.Context, ev skyl.HookEvent) {
		for _, h := range hooks {
			h(ctx, ev)
		}
	}
}

func chain(hooks ...skyl.Hook) skyl.Hook {
	return func(ctx context.Context, ev skyl.HookEvent) {
		for _, h := range hooks {
			h(ctx, ev)
		}
	}
}

Equivalent to registering them individually, but useful when a package wants to export one hook value assembled from several concerns.

Doing work off the request path

events := make(chan skyl.HookEvent, 1024)

var hook skyl.Hook = func(_ context.Context, ev skyl.HookEvent) {
	select {
	case events <- ev:
	default:
		// Drop rather than block a model call on a full buffer.
		metrics.Inc("skyl.hook.dropped")
	}
}

events := make(chan skyl.HookEvent, 1024)

var hook skyl.Hook = func(_ context.Context, ev skyl.HookEvent) {
	select {
	case events <- ev:
	default:
		// Drop rather than block a model call on a full buffer.
		metrics.Inc("skyl.hook.dropped")
	}
}

Use a background context in the consumer: the hook's own context is often cancelled by the time it fires.

Troubleshooting#

Every request got slower

Your hook is doing I/O on the request path. Buffer it with a non-blocking send.

My hook's context was already cancelled

Expected on the stream_end path — a client hanging up is the ordinary reason a stream ends early. Use a background context for work that must complete.

My logs contain user prompts

You logged ev.Request. Log the shape instead: provider, model, attempt, len(ev.Request.Messages), duration, usage.