Health and metrics

Three endpoints sit outside the auth boundary, because the things that call them — a kubelet, a load balancer, a scrape job — cannot reasonably hold a bearer token.

The endpoints#

Path	Purpose	Auth
`GET /healthz`	Liveness. 200 while the process is running.	None
`GET /readyz`	Readiness. Reports whether the server is accepting traffic.	None
`GET /metrics`	Prometheus metrics. Served only when SKYL_METRICS is on.	None

Liveness versus readiness#

Metrics#

Set SKYL_METRICS=true. That does two things: it serves /metrics in Prometheus format, and it installs the telemetry hook on every provider client — so the metrics cover all providers rather than whichever one happened to be wired first.

The instrumentation follows the OpenTelemetry GenAI semantic conventions, so model traffic appears in your observability stack the same way any other dependency does, and looks the same whichever provider served it — which is the point of the library.

Metric	Meaning
`gen_ai.client.token.usage`	Tokens consumed, split by `gen_ai.token.type` (input/output)
`gen_ai.client.operation.duration`	How long an operation took

Attributes include gen_ai.provider.name, gen_ai.request.model, gen_ai.response.model, gen_ai.operation.name and error.type.

Usage#

Kubernetes probes

livenessProbe:
  httpGet: { path: /healthz, port: 8080 }
  periodSeconds: 10

readinessProbe:
  # Distinct from liveness, so SIGTERM drains cleanly instead of dropping
  # in-flight paid requests.
  httpGet: { path: /readyz, port: 8080 }
  periodSeconds: 5

livenessProbe:
  httpGet: { path: /healthz, port: 8080 }
  periodSeconds: 10

readinessProbe:
  # Distinct from liveness, so SIGTERM drains cleanly instead of dropping
  # in-flight paid requests.
  httpGet: { path: /readyz, port: 8080 }
  periodSeconds: 5

Prometheus scrape

scrape_configs:
  - job_name: skyl-gateway
    static_configs:
      - targets: ['skyl-gateway:8080']
    metrics_path: /metrics

scrape_configs:
  - job_name: skyl-gateway
    static_configs:
      - targets: ['skyl-gateway:8080']
    metrics_path: /metrics

A cost alert

# Tokens per minute, by the model that actually answered — not the one asked
# for, so two snapshots behind an alias are not merged.
sum by (gen_ai_response_model) (
  rate(gen_ai_client_token_usage_sum[5m])
) * 60

# Tokens per minute, by the model that actually answered — not the one asked
# for, so two snapshots behind an alias are not merged.
sum by (gen_ai_response_model) (
  rate(gen_ai_client_token_usage_sum[5m])
) * 60

An error-rate alert

sum by (gen_ai_provider_name, error_type) (
  rate(gen_ai_client_operation_duration_count{error_type!=""}[5m])
)

sum by (gen_ai_provider_name, error_type) (
  rate(gen_ai_client_operation_duration_count{error_type!=""}[5m])
)

Troubleshooting#

/metrics returns 404

SKYL_METRICS is not set to a true value. It is off by default.

Requests were dropped during a rolling deploy

Your load balancer is probing /healthz rather than /readyz. Only the latter reports draining.

Should I put /metrics behind auth?

It is unauthenticated so a scrape job does not need a token. It exposes no prompt content and no credentials — only counts and durations. Restrict it at the network level if your threat model requires it; do not expose the gateway to the internet regardless.

Token metrics are missing for streaming calls

Streaming usage is reported on the stream_end event, and requires the upstream host to report it. On OpenAI-family hosts that means stream_options.include_usage must be honoured.