Primary question: Is per-prompt routing with a fail-closed scorer worth operating an extra Go service plus Postgres, measured against your own routing-decision data?
RepoDaily adoption score
RepoDaily rates this as 91/100 (strong) for adoption: evidence, installation path, production risk, differentiation, license clarity, and AI/agent fit are scored from the article sources and adoption notes.
6 source(s) across 4 source category/categories, plus a RepoDaily-specific evidence module when available.
5 workflow step(s), 5 next-action step(s), and 3 command/install signal(s) were detected.
Trending momentum is +284 stars, with maintenance/release/issue signals counted when present.
Risk is marked medium, with 5 security note(s) and 4 explicit skip condition(s).
3 opportunity lens item(s), 4 alternative(s), and 4 type-specific section(s) support differentiation.
License source or license wording is present.
8 AI/agent-related signal(s) were detected in the article text and metadata.
Project overview
workweave/router is a Go service that sits between your coding agent and your model providers, choosing a model for every prompt. The repository pulled 284 stars this period at trending rank 13, and its own description makes a specific promise: route each prompt in under 50 milliseconds and cut costs 40-70% with nothing more than an endpoint change. The topic list — claude-code, codex, anthropic, openai-compatible, ai-gateway — tells you exactly whose bill it targets: agentic coding tools that happily spend frontier-model tokens on trivial subtasks.
What stands out in the source is architectural discipline most young infrastructure repos never develop. CONTRIBUTING.md defines three concentric layers where imports flow inward only. The inner ring (internal/auth, internal/proxy, internal/router, internal/providers, internal/translate) holds pure domain logic with no I/O outside Service methods; internal/router and internal/providers are interface-only and never import each other. Adapters — internal/postgres, internal/router/cluster, and provider packages for anthropic, openai, google, and openaicompat — depend only on the inner ring and never on each other. HTTP wiring lives in internal/api/* and internal/server*, and cmd/router/main.go is the single file allowed to construct concrete adapters. Cross-layer violations are review-blocking, and the maintainers explicitly reject DI containers, reflection-based wiring, and service locators.
The routing engine itself is documented as a work in motion. HMM_GO_SELECTION.md lays out a target architecture, a flag ladder, and a rollback story for moving HMM roster ownership and deterministic arm selection into Go. go.mod supports the in-process-scoring reading: direct dependencies include knights-analytics/hugot and dmitryikh/leaves — Go libraries for ONNX and gradient-boosted-tree inference — plus golang-lru for caching, tink-go for key encryption, and a full OpenTelemetry stack (otelgin, OTLP trace and metric exporters). State lives in Postgres behind pgx/v5, with queries generated by SQLC from db/queries/.
Read the LICENSE before anything else. This is Elastic License 2.0, copyright 2025 Workweave, Inc.: you may run and modify it internally, but you may not offer it as a hosted or managed service, and you may not circumvent the license-key functionality. That makes router source-available rather than OSI open source — fine for most internal adopters, disqualifying for anyone whose product is itself a routing service.
Why it is trending now
- 284 stars this period at rank 13, with topics aimed squarely at agentic coding: claude-code, codex, anthropic
- The pitch requires no client rewrite — any OpenAI- or Anthropic-compatible SDK just points at a different base URL
- Vendor-claimed <50ms routing and 40-70% cost cuts attack the largest line item in agentic tooling spend (verify against the analytics export, not the README)
- Fail-closed by policy: when the cluster scorer cannot run, the router returns HTTP 503 instead of silently downgrading; PRs reintroducing fail-open behavior are rejected
- Docs-first repo: SEMANTICS.md defines canonical terms for sessions, rounds, turns, actions, and steps, and instructs readers to start there
Problem it solves
- Agentic coding tools send trivial subtasks — file reads, formatting, boilerplate diffs — to the most expensive model on the roster
- Each provider ships its own request format, so a team spanning Anthropic, OpenAI, Google, and OpenAI-compatible endpoints maintains several integrations; TRANSLATION_COMPATIBILITY.md exists precisely for cross-format translation and rollout modes
- Cost attribution is opaque: without row-level routing decisions you cannot say which requests hit which model or why
- Cheap-model-first fallbacks fail silently — quality drops with no signal about which request got downgraded, the exact regression pattern the maintainers say made them retire the old heuristic
How it works
- A client — Claude Code, Codex, or any OpenAI/Anthropic-compatible SDK — sends its request to the router endpoint instead of a provider
- The router scores the prompt and runs the cluster scorer to pick a model arm; HMM_GO_SELECTION.md describes the migration of roster ownership and deterministic arm selection into Go behind a flag ladder
- The request is translated to the target provider's format in internal/translate and forwarded through the matching adapter: anthropic, openai, google, or openaicompat
- Every routing decision is persisted to Postgres (pgx/v5, SQLC-generated queries from db/queries/) and exportable through /v1/analytics/* with read-only keys and cursor paging
- If the scorer cannot run, the request fails with HTTP 503 — no silent fallback to a default model
Architecture read: three rings, one composition point
The layering rules in CONTRIBUTING.md are the most concrete thing in this repo. The inner ring — internal/auth, internal/proxy, internal/router, internal/providers, internal/translate — is pure domain logic with no I/O outside Service methods. internal/router and internal/providers are interface-only and forbidden from importing each other, which keeps selection policy decoupled from provider specifics.
Adapters (internal/postgres, internal/router/cluster, internal/providers/{anthropic,openai,google,openaicompat}) may depend only on the inner ring and never on each other. Presentation code under internal/api/* and internal/server* adapts HTTP to Services and is banned from importing internal/postgres or concrete provider packages. cmd/router/main.go is the only file that constructs concrete adapters — composition is plain function calls, no DI container, no reflection, no service locator. Cross-layer violations are review-blocking.
Command surface: the make targets that matter
- make full-setup — boots the entire stack after a fresh clone
- make check — regenerates code, builds, and tests; this is the CI required-checks gate, paired with git diff --quiet after make generate to catch uncommitted generated code
- make generate — regenerates SQLC output under internal/sqlc/ from db/queries/; hand-edits to generated files are reverted on sight
- make build / make test — typecheck the whole module / run all tests
- Hot-reload loop: make db (Postgres only, port 5433), make setup (init schema + migrate + seed an rk_ API key), make dev (server under CompileDaemon hot reload)
Try-it path: from clone to a routed request
- Prerequisites: Go 1.25+ (go.mod pins go 1.25.0 with toolchain go1.25.9), golang-migrate for migrations, CompileDaemon for hot reload
- Run make db to start Postgres on port 5433
- Append DATABASE_URL=postgresql://router:router@localhost:5433/router?sslmode=disable to .env.local
- Run make setup — it initializes the schema, runs migrations, and seeds an rk_ key
- Run make dev, point one Claude Code or Codex project at the endpoint with the seeded key, and work normally for a day
- Pull the day's decisions from /v1/analytics/* with a read-only key and compare per-model spend against an un-routed control project
Maintenance risk: what to monitor
- Elastic License 2.0 forbids offering the software as a hosted or managed service and forbids circumventing license-key functionality — disqualifying if your own product is an AI gateway
- Every commit needs DCO sign-off (git commit -s adds the Signed-off-by trailer); unsigned PRs are asked to amend
- The HMM-to-Go selection migration ships behind a flag ladder with a documented rollback story — a live transition, so pin your version and read HMM_GO_SELECTION.md before upgrading
- The fail-closed 503 posture is enforced in review as much as in code (re-introducing fail-open behavior gets a PR rejected), so retries and scorer health checks belong on your side
- Provider and model names must use constants like providers.ProviderAnthropic — magic strings are rejected, which also constrains how config-driven your forks can get
Who should pay attention?
Good fit if
- Teams standardizing Claude Code or Codex across many repos that want per-prompt model choice without rewriting clients
- Platform groups that need routing audit data: /v1/analytics/* exports raw decisions with row-level grain, cursor paging, and read-only keys
- Shops already operating Go and Postgres — pgx/v5, golang-migrate migrations, and OTel exporters match a standard internal toolkit
- Anyone who prefers fail-closed behavior: a 503 is observable; a silent downgrade to a cheaper model is not
Skip for now if
- Anyone planning to resell routing as a hosted service — the license prohibits it explicitly
- Teams with no Go or Postgres operational capacity; the source pack documents no single-binary or embedded-database mode
- Policies requiring OSI-approved licenses only; Elastic License 2.0 is not OSI-approved
- Workloads where one fixed cheap model is sufficient — if prompts do not vary in difficulty, the router adds a network hop and a state store
Risks and cautions
Internal self-hosting sits squarely inside the license, but the hosted-service ban, the Postgres and Go 1.25 operational footprint, and a routing engine mid-migration into Go add real adoption friction.
- Elastic License 2.0: no hosting as a managed service, no circumventing license-key functionality, no removing licensor notices
- Runtime prerequisites: Go 1.25+, golang-migrate, CompileDaemon, and a Postgres instance (dev default port 5433)
- HMM roster ownership and deterministic arm selection are moving into Go behind a flag ladder — expect interface churn during rollout
- Fail-closed design returns HTTP 503 when the cluster scorer cannot run; callers need retry logic and scorer health monitoring
- BYOK encryption and key-pair auth are first-class configuration concerns in CONFIGURATION.md, backed by tink-crypto/tink-go/v2 in go.mod
- The /v1/analytics/* export supports dedicated read-only keys, so reporting consumers never need write-capable credentials
- Custom routing policy runs out-of-process under the POLICY_ROUTER_HARNESS.md sidecar contract, isolating org-specific rules from the proxy path
- Provider and model identifiers must come from constants (providers.ProviderAnthropic), reducing misrouting from string typos
- The license forbids removing or obscuring license-key-protected functionality when self-hosting, so any modification plan must respect that clause
Alternatives to compare
| Approach | When to use | Trade-off |
|---|---|---|
LiteLLM | You want a Python-based gateway with broad provider adapter coverage and simple fallback rules | MIT-licensed, free to self-host |
Portkey AI Gateway | You want a lightweight gateway focused on load balancing, guardrails, and edge deployment | MIT-licensed, free to self-host |
RouteLLM | You want framework-style cheap-vs-strong routing for chat workloads rather than a full gateway service | Open source, free to self-host |
OpenRouter | You would rather operate no infrastructure and accept a per-token routing markup | Commercial, usage-based per-token pricing |
What this trend reveals
Turn the analytics export into cost attribution
/v1/analytics/* already emits raw routing decisions with cursor paging, a documented row grain, and a field reference. Nothing in the docs index covers dashboards or per-team chargeback on top of it, and SEMANTICS.md supplies the canonical session/round/turn/action/step vocabulary to group by.
Read ANALYTICS_EXPORT.md, pull one day of rows with a read-only key, and check whether a per-session cost breakdown answers a question your finance side currently cannot.
Ship one policy sidecar
POLICY_ROUTER_HARNESS.md defines a contract for out-of-process policy sidecars — the sanctioned route to org-specific rules (for example, pinning one model for a regulated project) without forking the proxy path.
Write a sidecar that forces a single model for one tagged project, run it against make dev, and confirm the harness contract holds inside your latency budget.
Fill the provider matrix
Adapters ship for anthropic, openai, google, and openaicompat under internal/providers, and CONTRIBUTING.md points to AGENTS.md recipes for adding providers. Any provider without an OpenAI-compatible endpoint is a candidate contribution.
Pick one provider you already pay for, follow the AGENTS.md adapter recipe, and verify the rule that adapters never import each other survives review.
RepoDaily verdict
An unusually disciplined Go router — fail-closed, strictly layered, with a row-level analytics export — that earns a trial from agentic-coding teams already running Postgres, while its Elastic License 2.0 hosted-service ban and in-flight HMM-to-Go migration set the boundaries of who should adopt.