Skip to content
skyl

Provider Options

The merge is shallow on three of four adapters, and that will bite you.

Request.ProviderOptions sends arbitrary vendor-specific fields, overriding anything skyl set. It reaches the wire on all four adapters — but by two different mechanisms, and the difference is the single most important thing on this page.

You will learn

  • How to send a field skyl does not model
  • Why the shallow merge destroys sibling keys
  • Why Anthropic works differently, and what that enables
  • What ProviderOptions cannot do

The simple case#

goCompiles
req.ProviderOptions = map[string]any{
	"top_k":            40,
	"presence_penalty": 0.4,
	"seed":             7,
}
req.ProviderOptions = map[string]any{
	"top_k":            40,
	"presence_penalty": 0.4,
	"seed":             7,
}

Top-level keys, merged over the payload skyl built. skyl does not validate the contents — that is the point.

The shallow merge#

This is entry 14 in the silently ignored list, and it is the one most likely to look like it worked — the request succeeds, it just quietly has no token cap.

Anthropic uses JSON paths#

// Anthropic only: sets one nested field without disturbing its siblings.
ProviderOptions: map[string]any{
	"thinking.budget_tokens": 4096,
	"system.0.cache_control": map[string]any{"type": "ephemeral"},
}
// Anthropic only: sets one nested field without disturbing its siblings.
ProviderOptions: map[string]any{
	"thinking.budget_tokens": 4096,
	"system.0.cache_control": map[string]any{"type": "ephemeral"},
}
Deep diveWhy the two mechanisms differ

It is a consequence of how each adapter is built rather than a design choice anyone would make deliberately.

The OpenAI-format and Gemini adapters construct a map[string]any payload and serialise it, so merging another map over it is natural — and shallow, because that is what map assignment does.

The Anthropic adapter builds a typed SDK params struct. There is no map to merge into. So options are applied to the encoded body by JSON path, which turns out to be strictly more capable.

The cost of that capability is a sharp edge: on Anthropic, a top-level key that happens to contain a . is interpreted as a path. That is the only place in skyl where a key's spelling changes its meaning.

Before this was implemented, provider/anthropic silently ignored ProviderOptions entirely — in violation of the rule that every adapter must honour it — leaving cache_control, top_k and every beta feature unreachable on Anthropic with no workaround at all. The shared contract suite now asserts that every adapter honours it, so the rule cannot be met by one adapter and quietly missed by another.

What it cannot do#

Common uses#

GoalProviderOption
Prompt cachinganthropic"system.0.cache_control": {…}
A thinking token budgetanthropic"thinking.budget_tokens": 4096
Turn reasoning offopenai"reasoning_effort": "minimal"
Native structured outputopenai"response_format": {…}
Native structured outputgeminithe whole generationConfig, restated
top_kall"top_k": 40

Keeping it safe#

Because your keys override skyl's, a stray option can silently disable something you rely on. Set them in one place:

goCompiles
// Options are provider-specific by definition, so branch once, centrally,
// rather than sprinkling map literals through your call sites.
func withCaching(req *skyl.Request, provider string) {
	if provider != "anthropic" {
		return // no portable equivalent; do not pretend otherwise
	}
	if req.ProviderOptions == nil {
		req.ProviderOptions = map[string]any{}
	}
	req.ProviderOptions["system.0.cache_control"] = map[string]any{"type": "ephemeral"}
}
// Options are provider-specific by definition, so branch once, centrally,
// rather than sprinkling map literals through your call sites.
func withCaching(req *skyl.Request, provider string) {
	if provider != "anthropic" {
		return // no portable equivalent; do not pretend otherwise
	}
	if req.ProviderOptions == nil {
		req.ProviderOptions = map[string]any{}
	}
	req.ProviderOptions["system.0.cache_control"] = map[string]any{"type": "ephemeral"}
}

Recap

  • ProviderOptions sends anything skyl does not model, overriding what skyl set.
  • On openai, openaicompat and gemini the merge is shallow — nested objects are replaced wholesale.
  • Restate every sibling key, or you silently lose maxOutputTokens, temperature and the rest.
  • Anthropic applies options by JSON path, so nested fields can be set surgically.
  • On Anthropic, a top-level key containing a . is treated as a path.
  • It never sets headers — use the adapter's WithHeader option for those.

Try out some challenges

Each of these is solvable with what this page covered. Run them against the sandbox — no API key needed.

Set a Gemini seed without breaking the request

Add seed: 7 to a Gemini request that already sets MaxTokens, Temperature and Stop, without losing any of them.

Show hint

The shallow merge replaces generationConfig entirely. Rebuild it.

Show solution
goCompiles
gen := map[string]any{"seed": 7}

// Restate everything skyl would have put in generationConfig, or the merge
// silently drops it.
if req.MaxTokens > 0 {
	gen["maxOutputTokens"] = req.MaxTokens
}
if req.Temperature != nil {
	gen["temperature"] = *req.Temperature
}
if req.TopP != nil {
	gen["topP"] = *req.TopP
}
if len(req.Stop) > 0 {
	gen["stopSequences"] = req.Stop
}

req.ProviderOptions = map[string]any{"generationConfig": gen}
gen := map[string]any{"seed": 7}

// Restate everything skyl would have put in generationConfig, or the merge
// silently drops it.
if req.MaxTokens > 0 {
	gen["maxOutputTokens"] = req.MaxTokens
}
if req.Temperature != nil {
	gen["temperature"] = *req.Temperature
}
if req.TopP != nil {
	gen["topP"] = *req.TopP
}
if len(req.Stop) > 0 {
	gen["stopSequences"] = req.Stop
}

req.ProviderOptions = map[string]any{"generationConfig": gen}

Note this has to be built after the rest of the request, and has to be kept in sync if you later add a field. That fragility is exactly why the shallow merge is on the silently-ignored list rather than being described as a feature.

Detect an accidental override

Write a check that warns when a ProviderOptions key would override something skyl set.

Show hint

You know which top-level keys each adapter produces. A small allow-list is enough.

Show solution
goCompiles
// Keys skyl itself sets on the OpenAI-format payload. Overriding one of these
// is legal — but it is almost never what someone meant to do.
var oaiManaged = map[string]bool{
	"model": true, "messages": true, "max_completion_tokens": true,
	"temperature": true, "top_p": true, "stop": true,
	"tools": true, "tool_choice": true, "stream_options": true,
}

func warnOverrides(opts map[string]any) {
	for k := range opts {
		if oaiManaged[k] {
			log.Printf("warning: ProviderOptions[%q] overrides a field skyl manages", k)
		}
	}
}
// Keys skyl itself sets on the OpenAI-format payload. Overriding one of these
// is legal — but it is almost never what someone meant to do.
var oaiManaged = map[string]bool{
	"model": true, "messages": true, "max_completion_tokens": true,
	"temperature": true, "top_p": true, "stop": true,
	"tools": true, "tool_choice": true, "stream_options": true,
}

func warnOverrides(opts map[string]any) {
	for k := range opts {
		if oaiManaged[k] {
			log.Printf("warning: ProviderOptions[%q] overrides a field skyl manages", k)
		}
	}
}

The tools and messages entries are the important ones: overriding either replaces your entire conversation or tool set with whatever you put in the map, and the request will still succeed.

Edit this page on GitHub