Security advisory — Malicious litellm versions 1.82.7 and 1.82.8 were removed from PyPI (potential API key exfiltration). Uninstall them, rotate exposed credentials, and upgrade to a safe release (e.g. 1.82.9+). Run pip show litellm to verify. PyPI · README

Gateway Plugin Runtime

request → auth → policy → wasm hooks → channel relay → wasm stream/response hooks → audit

Gateway Plugin Runtime

Architecture

1request → auth → policy → wasm hooks → channel relay → wasm stream/response hooks → audit

internal/wasmhost/ is responsible for:

  • Manifest discovery and loading (loader.go)
  • Built-in plugins (builtin.go, wasm.binary: builtin:*)
  • wazero external wasm placeholder (runtime.go + WazeroPlugin)
  • fsnotify hot reload (manager.go)

ABI subset

HookDescription
OnRequestHeadersRequest-header stage
OnRequestBodyRequest body (including WAF)
OnResponseBodyNon-streaming response rewrite
OnStreamChunkSSE chunk rewrite

Returns ActionContinue / ActionStop; on Stop the gateway writes back StopBody directly.

Manifest

yaml
1runtime: wasm
2enabled: true
3priority: 100
4scope:
5  tenant_ids: ["*"]
6  routes: ["/v1/*"]
7wasm:
8  binary: builtin:keyword-rewrite
9  host_capabilities: [audit_log, metrics_inc]
10config:
11  replacements:
12    secret-keyword: "[REDACTED]"

Ops APIs (internal)

  • GET /internal/plugins
  • POST /internal/plugins/reload
  • POST /internal/plugins/upload
  • GET /internal/errors — 24h error fingerprint clustering
  • POST /internal/channels/{id}/probe — Channel self-check
  • GET /internal/perf — Pyroscope configuration

Made-with: Damon Li