AI Gateway Architecture

Source: enterprise/apps/gateway/

AI Gateway Architecture

Source: enterprise/apps/gateway/

Module: github.com/agenticx/enterprise/gateway

Responsibilities

OpenAI-compatible API — chat completions + embeddings
Three-route routing — local / private-cloud / third-party
Channel relay — multi-upstream weight/priority/retry
Policy engine — request / response / stream three-channel evaluation
Audit — JSONL must succeed + PG best-effort + checksum chain
Metering — usage_records / jsonl
Quota — tenant-level tracker (TPM/QPM/concurrency, config-driven)

Package layout

1apps/gateway/
2├── cmd/gateway/main.go
3└── internal/
4    ├── server/           # HTTP entry, handlers
5    ├── config/           # YAML + env
6    ├── provider/         # OpenAICompatibleProvider
7    ├── adaptor/          # Channel adaptor factory
8    ├── channel/          # Registry, Picker, Affinity, Stats
9    ├── relay/            # Retry executor
10    ├── keypool/          # Multi-key pool
11    ├── billing/          # Token settlement
12    ├── routing/          # Decider (header/model/default route)
13    ├── runtimeconfig/    # Remote/local providers polling
14    ├── quota/            # Tracker
15    ├── metering/         # PG / jsonl sink
16    ├── audit/            # Writer, backfill, chain
17    └── gatewayinternal/  # HTTP GET + Bearer

Chat request processing order

Diagram: JWT → quota → request policy → route or channel → upstream → response/stream second pass → audit → metering.

Streaming path performs stream-phase policy evaluation and segmented audit during SSE scanning.

Routing decision

routing.Decider priority (simplified):

Explicit provider request header
local_route_header configured header value
Model config Models[].route
default_route

Route values: local, private-cloud, third-party (aligned with provider table route field).

Channel relay

Enable with: GATEWAY_CHANNEL_REGISTRY=on

Diagram: admin CRUD writes gateway_channels; the gateway refreshes the registry, picks by weight, and retries the upstream.

Runbook: runbooks/gateway-channel-relay.md

Key resolution priority

PG / remote providers decrypt api_key_cipher
<PROVIDER>_API_KEY env var
LLM_API_KEY
Mock placeholder (policy/audit/metering still run)

Audit checksum chain

Each gateway_audit_events row has checksum, prev_checksum
Algorithm: Blake2b (see Go struct for field layout)
Admin GET /api/audit/chain-verify validates full table
PG write failure → .pg-pending, backfill on startup via GATEWAY_AUDIT_BACKFILL_DAYS

Difference from Python AgenticX gateway

Capability	Go Enterprise Gateway	Python AgenticX
17 secret detectors	partial in plugins	more built into framework
LiteLLM routing	no, OpenAI client	yes
keyword-list kind	no	may exist
extends manifest	single string	depends on loader

Do not conflate the two lines in customer-facing docs.

AI Gateway Architecture

Responsibilities

Package layout

Chat request processing order

Routing decision

Channel relay

Key resolution priority

Audit checksum chain

Difference from Python AgenticX gateway

Related docs