Security advisory — Malicious litellm versions 1.82.7 and 1.82.8 were removed from PyPI (potential API key exfiltration). Uninstall them, rotate exposed credentials, and upgrade to a safe release (e.g. 1.82.9+). Run pip show litellm to verify. PyPI · README

Gateway observability

The gateway exposes Prometheus text metrics for latency, cache, and upstream errors. Off by default: GET /metrics returns 404 unless GATEWAY_METRICS is on, so counters are not left on an unauthenticat

Gateway observability

The gateway exposes Prometheus text metrics for latency, cache, and upstream errors. Off by default: GET /metrics returns 404 unless GATEWAY_METRICS is on, so counters are not left on an unauthenticated port.

Metrics

Prometheus endpoint: GET /metrics (404 when GATEWAY_METRICS=off).

MetricMeaning
agx_gateway_ttft_secondsTime to first token
agx_gateway_tokens_per_secondGeneration throughput
agx_gateway_cache_hits_totalL1/L2 hits
agx_gateway_cache_lookups_totalCache lookups
agx_gateway_channel_healthChannel health
agx_gateway_active_streamsIn-flight streams
agx_gateway_upstream_error_totalUpstream errors

Cache switches and pricing: architecture/cache-and-pricing.md.

Grafana

Import enterprise/docs/observability/grafana-ai-gateway.json from the main repo and point the datasource at Prometheus. This is a dashboard template, not a hosted Grafana.

Audit is not metrics

Compliance records for LLM / MCP calls live in gateway_audit_events + JSONL, not /metrics. Query them from admin /audit. See architecture/data-flow.md and runbooks/audit-pg-backfill.md.

Made-with: Damon Li