Security advisory — Malicious litellm versions 1.82.7 and 1.82.8 were removed from PyPI (potential API key exfiltration). Uninstall them, rotate exposed credentials, and upgrade to a safe release (e.g. 1.82.9+). Run pip show litellm to verify. PyPI · README

Gateway Policy Engine

Implementation: packages/policy-engine/go/ (embedded in Gateway)

Gateway Policy Engine

Implementation: packages/policy-engine/go/ (embedded in Gateway)

Admin: @agenticx/feature-policy + /policy UI

Plugin manifest: plugin-protocol/README.md


Three-channel evaluation

ChannelWhenDescription
RequestAfter user messages are received, before calling upstreamblock returns immediately; no upstream consumption
ResponseNon-streaming full responseCan block/warn/redact assistant content
StreamSSE fragment scanStreaming idle timeout + buffer cap to prevent hangs

Source entry: policy call sites in apps/gateway/internal/server/server.go.


Rule types (Go engine)

kindDescriptionpayload / manifest fields
keywordKeyword Triekeywords[]
regexRegular expressionpattern
piiBuilt-in detectorpii_type: email, mobile, id-card, bank-card, api-key

Not supported (difference from the Python framework): keyword-list as a standalone kind.


Actions

actionBehavior
blockAbort; Portal must show a compliance-block UI (not a model's natural-language refusal)
warnRecord hits, continue
redactReplace sensitive spans, then continue

Admin test API: blocked is true only when action=block.


Rule source merge

11. plugins/moderation-*/manifest.yaml (部署内置)
22. enterprise_runtime_policy_snapshots (admin 发布)
33. GATEWAY_POLICY_OVERRIDE_FILE (本地覆盖,调试)

Publish flow:

  • Draft: policy_rules.status = draft — not included in the snapshot
  • Publish: POST /api/policy/publish → snapshot JSON
  • Rollback: POST /api/policy/publishes/:id/rollback

Runbook: runbooks/policy-snapshot-rollback.md


applies_to scope

Rules / rule packs may include optional JSONB applies_to:

  • departmentIds + departmentRecursive
  • roleCodes
  • userIds / userExcludeIds
  • clientTypes, stages

Pitfall: putting placeholder examples in userIds (e.g. u1,u2) makes real JWT users miss the match, so the rule never applies. Leave org-wide rules empty.


Audit hit structure

Gateway audit policies_hit JSONB includes:

  • matched_rule — rule id/code
  • severity, action
  • Message summary

Not the old field names rule_id / reason (watch for this when inspecting historical data).


Admin policy test

1POST /api/policy/test
  • Merges the current form preview (unsaved action/payload) with published rules in the DB
  • Avoids “UI selected block but the old DB action is still used”

Chinese UI labels: 拦截/警告/脱敏; 关键词/正则/PII.


Industry plugin packs

PackextendsScenario
moderation-pii-baseline—Generic PII
moderation-financepii-baselineFinance
moderation-medicalpii-baselineMedical PHI warn

Customer-specific rules: PG policy center or customers/*/rules/; see the collaboration handbook.


Stream hardening

Environment variablePurpose
GATEWAY_STREAM_IDLE_TIMEOUTSSE idle cutoff
GATEWAY_STREAM_SCANNER_MAX_BUFFER_MBMax buffer per chunk

Prevents malicious/abnormal streaming responses from filling memory.


Related docs

Made-with: Damon Li