Phoenix Internal API (PAI)

PAI is the privileged internal API plane. It is not publicly reachable and is protected by:

• PAI keys (X-PAI-Key)
• Companion token (X-PAI-Token) / session-JWT second factor (PAI_REQUIRE_PAIRED_TOKEN, PAI_SESSION_TOKENS_ENABLED)
• Optional in-app IP allowlist (PAI_IP_ALLOWLIST) — skipped when empty; restrict at the network/system layer (WAF / load balancer / security group) instead
• Optional per-key IP/CIDR and domain restrictions
• Optional WAF / load balancer allowlist

Base URL

/internal/v1

Swagger

Internal Swagger UI (requires a PAI key, or a session JWT when PAI_SESSION_TOKENS_ENABLED is on; IP allowlist enforced only when configured):

• GET /internal/docs
• GET /internal/openapi.json

Authentication

Provide a PAI key via one of:

• X-PAI-Key: phx_pai_...
• X-API-Key: phx_pai_...
• Authorization: PAIKey phx_pai_...

Key Generation

POST /api/v1/admin/pai/keys

Generated Global Admin only, no email step. Returns the api key (phx_pai_...) and companion token (phx_pait_...) once — both shown on screen; the companion token is also emailed best-effort. Manage with GET /api/v1/admin/pai/keys (list, no secrets) and POST /api/v1/admin/pai/keys/{key_id}/revoke (revoke). To rotate, generate a new pair and revoke the old key. POST /api/v1/admin/pai/keys accepts optional per-key restrictions:

{
  "name": "Internal Analytics",
  "ip_allowlist": ["10.0.0.0/8", "192.168.1.10"],
  "domain_allowlist": ["internal.example.com", "*.svc.example.com"]
}

For backwards compatibility, mixed entries submitted in ip_allowlist are split into IP/CIDR and domain restrictions by the backend. Domain restrictions are matched against X-Forwarded-Host, Host, Origin, and Referer; use them as an additional guard with the global IP allowlist, not as a replacement for network controls.

Email delivery is best-effort and is not required before PAI key issuance (GA-only generation succeeds even when SMTP is unavailable; the companion token is shown on screen and, when SMTP is configured, also emailed). MFA verification is planned but not yet available.

IP allowlist enforcement mode (PAI_IP_ALLOWLIST_MODE, added 2026-08-23)

PAI_IP_ALLOWLIST_MODE decides how every PAI IP allowlist is applied. It is an environment variable — it cannot be changed at runtime or from the admin UI.

Any unrecognised value resolves to enforce: a typo must never silently switch an access control off.

The mode applies to all three IP layers — the system-level PAI_IP_ALLOWLIST, a key's own ip_allowlist, and the per-key list carried inside a session JWT. It covers the IP axis only: domain allowlists, companion tokens, scope checks and rate limits are unaffected, so a request that fails any of those is still rejected in monitor mode.

An empty allowlist remains a no-op in both modes — “not configured” has always meant “not applied”, and monitor does not invent misses for a layer that was never restricted. While monitor is active a configured allowlist is not a control, so the process-start “PAI is unrestricted” warning still fires.

GET /api/v1/admin/pai/ip-allowlist

Global-admin only. Read-only report of how the allowlist is currently applied, backing the IP Allowlist panel in the admin dashboard. Returns no key material.

{
  "mode": "monitor",
  "raw_mode": "monitor",
  "effective": "monitoring",
  "system_allowlist": ["10.0.0.0/8"],
  "trusted_proxy_hops": 2,
  "require_paired_token": false,
  "key_mode": "db",
  "caller_ip": "203.0.113.5",
  "caller_allowed": false,
  "observations": [
    {
      "observed_at": "2026-08-23T18:05:11.412000+00:00",
      "layer": "system",
      "client_ip": "203.0.113.5",
      "allowlist": ["10.0.0.0/8"],
      "path": "/internal/v1/cve/CVE-2024-0001",
      "method": "GET"
    }
  ],
  "observation_total": 1,
  "observation_retained": 1,
  "observation_limit": 100
}

effective is not_configured when the allowlist is empty, otherwise enforcing or monitoring. caller_allowed is null when nothing is configured, since “allowed” would imply a check ran. layer is one of system, key, or session_key.

Observations are an in-memory, per-worker ring buffer capped at observation_limit — a rollout aid, not an audit log. A multi-worker deployment shows only the worker that served the request, and a restart clears it. The WARNING log line is the durable record.

DELETE /api/v1/admin/pai/ip-allowlist/observations

Global-admin only. Clears the retained observation buffer and returns {"cleared": true, "dropped": <n>}. Application logs are unaffected. The action is written to the admin audit log.

Paired companion token (x-pai-token, added 2026-07-21)

Database-mode PAI keys (pai_key_mode=db) can carry an optional companion token — an independent secret (phx_pait_...) minted alongside the api key by POST /api/v1/admin/pai/keys (returned inline as pai_token in that response, and also emailed to the requesting admin). Send it as a second header alongside the key:

• x-pai-token: phx_pait_...

The companion is required — a missing or mismatched value returns 401 — when either:

• the global setting pai_require_paired_token is true (env PAI_REQUIRE_PAIRED_TOKEN, default false), or
• the specific key has a companion token hash stored (a key issued with a companion keeps requiring it regardless of the global setting).

This is a backward-compatible, per-key rollout: existing db-mode keys issued before this feature (no stored companion hash) keep working with the api key alone unless pai_require_paired_token is turned on globally. Env-mode PAI keys (pai_key_mode=env, the single shared PAI_KEY) never require a companion token — there is no per-key record to hang a hash off of.

Session tokens (Authorization: Bearer <jwt>, added 2026-07-21)

Once minted via POST /internal/v1/pai/session (below), a short-lived session JWT can be presented as Authorization: Bearer <token> in place of the api-key + companion-token pair on subsequent internal API calls, until it expires or the key's revocation epoch advances.

Session Tokens (added 2026-07-21)

POST /internal/v1/pai/session

Exchanges a PAI api-key + companion-token pair for a short-lived session JWT, so subsequent calls can send Authorization: Bearer <token> instead of resending the raw key pair on every request.

Flag: pai_session_tokens_enabled (default false, env PAI_SESSION_TOKENS_ENABLED). Returns 404 when off.

Auth: Same as any PAI call — the router-level require_pai_access dependency runs first, so the IP allowlist, api key, and (if required for this key) companion token are already validated before this handler runs. If that validation fails, the usual 401/403 responses apply and the request never reaches the session-minting logic below.

Request

No request body. Send the pair as headers:

POST /internal/v1/pai/session
X-API-Key: phx_pai_...
x-pai-token: phx_pait_...

(Omit x-pai-token if the key doesn't require a companion — see Paired companion token above.)

Response 200

{
  "access_token": "<jwt>",
  "token_type": "Bearer",
  "expires_in": 900
}

expires_in is seconds until expiry, from pai_session_ttl_seconds (default 900, i.e. 15 minutes; env PAI_SESSION_TTL_SECONDS).

Using the session token

Present it on subsequent internal API calls in place of the key pair:

Authorization: Bearer <access_token>

The minted JWT embeds the same per-key IP/domain allowlist as the underlying key, so the session path is never broader than what the raw api-key + companion pair could already reach. It is also invalidated early if the key's revocation epoch advances (e.g. the key is revoked/rotated) — verify_session_token checks the JWT's embedded epoch against the key's current epoch on every call and rejects a stale one.

Notes

• A session JWT cannot itself be exchanged for another session JWT — presenting a Bearer session token to this endpoint returns 401 ("Cannot mint a session from an existing session token..."). You must always present the api-key + companion pair to mint a session; this prevents a leaked short-lived token from self-refreshing indefinitely.
• Standard PAI auth errors (401 Missing PAI key, 401 Invalid PAI key, 401 Missing PAI companion token, 401 Invalid PAI companion token, 403 Client IP not allowed) apply as usual if the pair fails validation before reaching this endpoint.

Core Endpoints

The /internal/v1/cves* group (added 2026-07-08) mirrors the public /api/v1/cves read surface, unredacted and with higher limits, gated only by PAI auth (require_pai_access + pai_enabled).

POST /internal/v1/cves/batch (added 2026-08-20) mirrors the public POST /api/v1/cves/batch (Purple R1): same id-normalisation (uppercase, dedupe, sort, silently drop malformed ids — all-malformed is a 422, not an empty 200) and the same explicit not_found[] list, but returns records unredacted (no tier projection) and accepts up to a flat 5000 ids per call instead of the public route's per-tier cap. Gated on the same enable_cve_batch_lookup flag as the public route (default false, returns 404 when off) so both transports move together.

not_found semantics. An id lands in not_found when the corpus has no NVD record for it — determined by the returned record carrying no NVD-derived content (no description, published, modified or cvss). This matters because the underlying get_enriched_cve() never signals a miss: it returns a populated-looking envelope (cve_id plus an enrichment block) for any id, including one that does not exist. Treat items as authoritative and not_found as "not in Blue's corpus", rather than assuming every requested id comes back in items.

One known imprecision, stated so it is not mistaken for corruption: a CVE that genuinely exists in NVD but has no description, no dates and no CVSS — a freshly reserved or rejected entry — is reported in not_found. No fabricated record is ever returned in items.

Package Intelligence

POST /internal/v1/packages/intel

Returns the internal package-version intelligence payload for a package coordinate. This is the PAI twin of the premium public package intelligence endpoint.

Gated by enable_package_intel_api (default false). When the flag is off the endpoint returns 404.

Request

{
  "ecosystem": "Maven",
  "name": "com.h2database:h2",
  "version": "1.3.176"
}

Response fields

Advisory Endpoints

Gated by ENABLE_ADV_FETCH (default false). All return 404 when the flag is off.

CWE / OWASP Top 10 / Threat-Frequency Analysis Endpoints

Added 2026-07-10. PAI mirrors of the five public /api/v1/analysis/* endpoints (see PUBLIC_API.html). These routes are not themselves gated by ENABLE_CWE_OWASP_THREAT_TIERED_API — that flag only governs whether the public routes shape their payload. The PAI mirrors always return the full, unshaped payload (AccessMode.FULL), regardless of the flag's state, because PAI access is itself the Enterprise gate (require_pai_access) and there is no end-user tier context on a PAI call.

No additional tier parameter or header is required beyond the PAI key itself. Missing/unavailable underlying data returns 404 (this router's existing convention — see GET /internal/v1/eol-intelligence), not the public routes' 503.

Merged CWE-Intelligence Mirror (added 2026-08-23)

GET /internal/v1/analysis/cwe-intelligence is the PAI mirror of the public merged CWE-intelligence table (backend/app/routers/analysis.py). Same outer-join merge over /cwe, /kev-cwe, /owasp-top10 and CWE_MASTER, keyed by bare CWE id.

Unlike the CWE/OWASP/threat-frequency mirrors above (not flag-gated — always full, gated only by require_pai_access), this route is flag-gated: enable_cwe_intelligence_endpoint (default false, absent from site_config.json), same flag as the public route. Returns 404 on this PAI plane when off, same as the public route (both are new routes with no legacy unflagged behavior to preserve).

Always AccessMode.FULL, passed explicitly at this one call site rather than baked into the shaper: raw nvd_count and raw kev_count are present on every row. This is the only plane where either raw count is ever returned — the public route's own tier ceiling (Enterprise → PERCENT_ONLY) never reaches FULL; see PUBLIC_API.html for why the public route uses a dedicated resolver instead of the shared resolve_access_mode.

Same response envelope as the public route: schema_version: "cwe-intel-v1", per-source source_provenance list ({"name", "generated_at"}, named to avoid colliding with each row's own differently-shaped sources: List[str] field), cwes (dict keyed by bare CWE id), total_unique_cwes (== len(cwes)), dropped_keys (count of raw source keys that failed CWE-id normalisation and were therefore excluded from the merge — 0 in the steady state). Same caching contract: ETag is a sha256 over the shaped (AccessMode.FULL) bytes — computed the same way as the public route's ETag over its own lower-tier bytes, so the two planes never collide on the same digest for different content — plus Cache-Control: private, max-age=0, must-revalidate and If-None-Match → 304 support.

Unified Intelligence Query Surface Mirror

Added 2026-07-25. PAI mirror of the public Unified Intelligence Query Surface (backend/app/routers/intel_query_pai.py), over the same resolver registry (backend/app/services/intel_query/registry.py) as every other adapter for this feature. Gated by the same flag, enable_intel_query_surface (default false) — returns 404 when off, in addition to the standing require_pai_access dependency on the whole /internal/v1/intel/* router.

Final whole-branch review fix-round (2026-07-25): the limit ceiling was lowered from 1000 to 500 to match Enterprise's real max_page_size (tier_limits.py) — PAI access is itself the Enterprise gate, so a caller could previously ask for double what even an Enterprise public caller's own tier cap allows. The resolver call is also now offloaded to a worker thread (anyio.to_thread.run_sync), matching the public route's own fix (see PUBLIC_API.html's "Known rollout gates" section) — some resolvers (CveResolver in particular) run unindexed O(N) scans over in-memory data and must never block the event loop.

Full fidelity — no data_shield pass — for a phx_pai_ (pai_internal) key. Unlike the public route, which runs exactly one filter_response_for_tier pass keyed to the caller's resolved tier, this router runs no field-level shaping for an internal PAI key. Every field a resolver emits is returned as-is. provenance.redaction_profile is stamped "pai" (not a tier name) so a deterministic consumer can tell which shaping pass — none — produced the response. As on every other PAI endpoint, PAI's advantage is field-level fidelity, never internal business data: LLM model names and per-call costs are absent because no resolver emits them, not because they were redacted.

A phx_gintel_ (api_global_intel) BUNDLE key IS shaped, at the tier its licence grants (fix C1, 2026-08-14). The "PAI access is itself the Enterprise gate" premise above holds for the internal key family (pai_internal maps to ENTERPRISE); it was falsified for this router when Plan B admitted a customer key sold at two levels. A global_intel:pro key was receiving the ENTERPRISE-classified deterministic.signals / chains that the identical query at Pro on /api/v1/intel/* strips. A bundle key's response — including the 403 body — now goes through exactly one filter_response_for_tier pass at enterprise / pro, floored at registered when the key carries no valid stamp, and provenance.redaction_profile names that tier instead of "pai". Same licence, same fields, whichever door. Its limit is likewise clamped to its own tier's max_page_size rather than the Enterprise ceiling of 500.

Bundle-key admission is flag-gated (same fix). With enable_global_intel_license off, the accepted-scope tuple collapses to PAI_DEFAULT_SCOPES (pai_internal only) and a phx_gintel_ key gets 401 here too — so the documented rollback kill switch genuinely closes this door instead of leaving already-issued bundle keys admitted on the one plane where the tier demotion had no effect. An unreadable flag store means no admission.

Whether a caller can resolve an unconfirmed/under-review (SUSPECT-band) malware package at all is a completely separate mechanism from the field-shaping described above (see PUBLIC_API.html's malware verdict access gate section). This router never passes allow_clean=True for the malware entity — it relies entirely on resolve_entity/search_entity's own default (False), identical to how an anonymous/non-admin caller is treated on the public route. A PAI caller sees confirmed-malicious verdicts exactly the way a non-admin public caller does; clearing the PAI dependency does not, by itself, grant visibility into scanned-but-not-confirmed packages.

This was an explicit, human-reviewed security decision made during implementation, reversing an earlier draft that forced allow_clean=True unconditionally for PAI on the theory that backend/app/routers/mpi_artifacts.py (a PAI-only, no-admin-stacking read router with no verdict gate at all) was sufficient precedent that PAI-alone is this platform's ceiling of trust for MPI data. That reasoning was rejected: loosening who can see unconfirmed/under-review SUSPECT-band malware verdicts is an authorization-policy change requiring explicit approval before weakening any access gate (per AGENTS.md Hard Rules), and backend/app/routers/malware_intel_internal.py stacks an extra require_global_admin check on top of require_pai_access specifically for other sensitive malware operations (force-rescan, raw repo_analysis blob) — direct evidence that this codebase's convention does not treat PAI alone as admin-equivalent for sensitive malware data.

If a future task needs PAI to see SUSPECT-band packages, it requires a distinct, higher-trust PAI signal (e.g. a specific key scope or attribute) — not simply restoring the unconditional allow_clean=True this router originally shipped with.

Advisory block opt-in — ?include=advisory (Plan D, added 2026-07-26)

GET /internal/v1/intel/malware/{entity_id}?include=advisory (and the equivalent on GET /internal/v1/intel/malware?include=advisory search) populates the envelope's advisory block with MPI LLM reasoning — {"analyst": {"verdict", "confidence_class", "reasoning"}, "judge": {"verdict", "reasoning"}, "reproducible": false}, built by the shared build_reasoning_view() (app/services/mpi_reasoning.py). See PUBLIC_API.html's "Advisory block opt-in" section for the full field-by-field detail and the confidence-class vocabulary it uses.

PAI honors this unconditionally whenever the caller passes it — no additional tier check, unlike the public REST mirror, which additionally requires the caller's resolved tier to be enterprise. This is consistent with this route's "full fidelity, no data_shield pass" behavior for an internal PAI key described above: PAI access is already this platform's highest trust boundary, so there is nothing narrower to check.

For a phx_gintel_ bundle key the gate is still not applied, but the shield is (fix C1, 2026-08-14). The call still returns 200 and still needs no bundle — advisory is not an expanded token and never triggers the licence gate — but advisory is ENTERPRISE in data_shield, so it survives only for a global_intel:enterprise key. Below that it comes back null, exactly as it does on /api/v1/intel/* at the same tier.

This does not widen the allow_clean verdict-visibility gate described immediately above. include_advisory is only ever consulted by the malware resolver after an allow_clean-gated row load has already succeeded — a PAI caller requesting ?include=advisory for an unconfirmed/under-review (SUSPECT-band) package still gets the same 404 any other non-privileged caller gets.

Requires both enable_intel_query_surface and enable_mpi_reasoning_enterprise to be on; advisory stays null if either is off.

Expanded intelligence composition — ?include= (Plan B, added 2026-08-13)

GET /internal/v1/intel/library_version/{purl}?include=vulnerabilities,malware,exploitation,campaign,licensing returns the library, its vulnerabilities with per-CVE detail, its malware intelligence, and its exploitation / campaign / licence vectors in one call, under a new top-level expanded envelope block. GET /internal/v1/intel/library/{purl} (unversioned) accepts the same tokens and additionally returns a cross-version rollup (any_version_compromised, compromised_versions[], safe_versions[], worst verdict).

PAI first, then everywhere (Plan D, 2026-08-14). PAI shipped first because it skipped data_shield entirely and stamped redaction_profile="pai", so the composition shape could be proven before being locked behind a tier matrix. Plan D added the same ?include= surface to public REST (/api/v1/intel/*), the MCP intel_get tool and phoenix-cli, all four sharing the registry's single token gate. See PUBLIC_API for the tier matrix — which since fix C1 applies to a phx_gintel_ caller on this router too, and is not applied only to an internal phx_pai_ key.

SCHEMA_VERSION moves 1.0.0 → 1.1.0. The bump is MINOR because expanded is opt-in: a request with no include returns a byte-identical response — the expanded key is absent, not null, and provenance keeps exactly its four original keys.

Authentication. Both phx_pai_ (pai_internal) and phx_gintel_ (api_global_intel) keys authenticate on this router, through the same validate_pai_request path: same system IP allowlist, same per-key IP/domain allowlists, same companion-token requirement, same rate limiter. A Global Intel key is a licence, not a way around PAI's network controls.

include accepts CSV (?include=a,b) or repetition (?include=a&include=b). include=all expands server-side to every token the caller's tier permits and never errors on an over-privileged token. Token validation and the token→tier matrix live in backend/app/services/intel_query/registry.py — one gate for four transports, not one per adapter.

Several envelope field names are deliberately not the source's own. data_shield's DEFAULT_FIELD_CLASSIFICATIONS is a flat map keyed on bare field names and filter_response recurses by bare key, so a nested key here shares one classification with every dataset on the platform. Two distinct hazards force the renames, and both are live.

The second hazard is the less obvious one and is worth stating plainly: adding a classification is not free either. Under fail-closed, an unclassified name is dropped for every tier, so classifying a name that another data_shield-filtered producer also emits is a live behaviour change to that unrelated, already-shipped endpoint. tests/test_data_shield_intel_coverage.py (D-T2c) checks this mechanically against every module reachable from a filter_response_for_tier call site. None of these renames is cosmetic.

Metering and quota (Plan C, added 2026-08-14). Expanded composition is metered separately from a plain lookup because it is materially more expensive — it fans out to MPI, OSV/library-intel and up to 500 per-CVE exploitation lookups.

?include=advisory alone is not an expanded call: it needs no bundle, does not increment intel_expanded_lookups, and is not subject to the intel_expanded quota. Only the tokens in EXPANDED_TOKENS (plus all) trigger either — the classifier consults registry.wants_expanded(), the same gate the licence check uses, so the two cannot drift.

All three composing transports are billed identically (Plan D, 2026-08-14; corrected 2026-08-14 by fix I2). The rule is "a transport bills intel_expanded_lookups exactly when it can actually serve the composition". For the two HTTP transports that is usage_metering.COMPOSING_PATH_PREFIXES = ("/api/v1/intel/", "/internal/v1/intel/"). MCP is the third: it has no billable path of its own (classify_request decides /api/v1/mcp in an earlier branch), so IntelToolHandler records the same two counters itself, from the same usage_metering.intel_usage_keys() definition the path classifier uses. Until that fix MCP enforced the intel_expanded quota against a counter only REST and PAI ever wrote — composition served, billing zero, the 2,000/day Pro cap unreachable through that door. An expanded call now costs the same fan-out and the same counter through any of the three.

MCP bills intel_query_lookups on intel_get/intel_search and adds intel_expanded_lookups only when a licence was actually resolved and the composition served; api_calls_total/api_calls_mcp continue to come from usage_tracking_middleware, which does not skip /api/v1/mcp, so nothing is double-counted.

Metering has a hard prerequisite, and the surface now refuses without it. Both counters are columns added by db/migrations/123-global-intel-usage-counters.sql. With 123 unapplied the read path (SELECT *) silently returns 0 and the write path fails wholesale into a swallowed exception — the whole surface unmetered and unquota'd, with only a WARNING. app/services/usage_schema.py now checks the columns directly: logged loudly at startup, and every licence-gated expanded call returns 503 (naming the migration) rather than serving the platform's most expensive read unbilled. Deploy order is 122 → 123 → code.

One consequence on the public route, and the shape of it matters: an anonymous caller sending a licence-gated include is refused by the route, not by _enforce_anonymous_limits, so the 403 still carries the base envelope. Plan C had put intel_expanded in ANONYMOUS_ENFORCEABLE_FEATURES so the middleware would refuse it before any resolver work; once the public router actually composed, that became a regression — the middleware has no envelope, licence or tier context, so all it could return was a bodiless {"detail": …}, deleting data the caller was entitled to. anonymous_feature_key now falls the anonymous enforcement back to search, so the call is still metered and still 429-able while the route owns the refusal. ?include=advisory alone is unaffected: it needs no bundle and was never an expanded call.

Daily intel_expanded caps: Pro 2,000, Enterprise unlimited; free/registered are 0, though an unlicensed caller is refused by the 403 licence gate before the quota is ever consulted. Source of record is the tier_features DB table (db/init/07-user-tiers.sql), mirrored by the fallback map in backend/app/services/tier_limits.py; tests/services/test_tier_limits.py pins the two together.

/entities and /vocabularies are static vocabulary listings and are not metered. /internal/* is skipped entirely by usage_tracking_middleware, so this router records its own usage — there is no second writer and nothing to double-count. If the metering backend is unreachable the quota check fails open (logged at WARNING): the licence gate is the authorization control, and a storage blip must not deny a paying customer access they hold.

Provenance. When composition runs, provenance gains three keys, and a fourth when there is something to disclose:

• block_sources — per-block source attribution. vulnerabilities is attributed to library-intel, not osv: its rows come from the precomputed artifact, whereas deterministic.vulnerability.cve_ids in the same envelope is a live OSV read. Only malware is ever live.
• block_freshness — "live" for a block genuinely served from a live MPI read; the library_intelligence.json artifact's _metadata.generated_at for anything precomputed, so a stale enrichment lane is visible instead of presented as live.
• block_errors — {block: reason} for any omitted block. A failed or timed-out block is omitted with a reason, never returned empty (an empty block reads as "we looked and found nothing" when the truth is "we could not look") and never fails the whole call.
• block_notes — {block: caveat}, present only when a block that did land carries one. Today the sole entry is vulnerabilities, disclosing that it is a precomputed snapshot which may diverge from deterministic.vulnerability.cve_ids. When the two disagree, treat deterministic as the live answer and expanded.vulnerabilities as a snapshot of the age given in block_freshness.

provenance.intel_license is stamped {"product": "global_intel", "tier": "pro|enterprise", "source": "api_key:global_intel_tier"} on a licensed response.

Truncation is never silent. expanded.vulnerabilities carries total (the real count), returned, and truncated. Selection under the per-tier cap is by severity then EPSS, so a cap never drops the most exploitable rows; presentation order is then imposed by sort_deterministic() (determinism rule D5).

Array ordering (D5), precisely. sort_deterministic() orders a list of objects by the first present key in envelope.STABLE_ID_KEYS (id, signal_id, malware_signal_id, cve_id, cwe_id, purl, cpe_uri, technique_id); lists of scalars sort naturally. expanded.campaign.campaigns[] carries none of those keys and is therefore ordered by its producer instead — ascending by (date, name) — which is stable, tier-independent and reproducible, but is not the stable-ID rule the sentence above describes. It was unordered entirely until 2026-08-14. advisory is exempt from D5 altogether: LLM output is not reproducible and ordering it would imply that it is.

Budget. Each block has its own timeout, INTEL_EXPANDED_BLOCK_TIMEOUT_MS (default 3000, capped at 60000). The whole resolve is offloaded via anyio.to_thread.run_sync so composition never runs on the event loop.

Tenant-Scoped Research Triggers Mirror

Added 2026-07-26. PAI mirror of the public Tenant-Scoped Research Triggers (backend/app/routers/research_pai.py), over the same job store and dispatcher as the REST surface. Gated by the same flag, enable_intel_research_triggers (default false); returns 404 when off, in addition to the standing require_pai_access dependency on the whole /internal/v1/research/* router.

• POST /internal/v1/research — trigger research. Body: {"kind": "...", "target": "..."}. Returns 202.
• GET /internal/v1/research/{job_id} — get one job (org-scoped; another org's job reads as 404).
• GET /internal/v1/research — list this organisation's jobs (state, limit 1–200 default 50, offset).

org_id comes from the PAI key (PAIKeyInfo.org_id), never from a request field — a PAI caller may not name an arbitrary org, or PAI becomes a cross-tenant write primitive. A PAI key with no resolvable org_id gets 403.

No separate Pro/Enterprise tier check. A PAI key has no subscription tier to resolve; quota is pinned at the Enterprise daily ceiling (250/day) rather than gated on a lookup that does not exist for this transport — the same precedent as this file's own Unified Intelligence Query Surface mirror capping its limit at Enterprise's max_page_size instead of inventing a separate PAI-only ceiling.

This 250/day ceiling shares the SAME per-org daily counter as REST — it is not an independent PAI-only allowance. create_job()'s quota check counts every row in intel_research_jobs for the org created that UTC day, regardless of which transport created it, so a Pro org's own REST quota is 25/day but a PAI key acting on that same org can create up to 250/day against the identical counter — heavy PAI usage can exhaust the shared counter and 429 the org's own subsequent REST calls for the rest of the day. This is a deliberate, already-tracked rollout gate, not an oversight.

job_state, not state, in the response — the same rename, and the same field-classification-collision rationale, as the REST mirror (see PUBLIC_API.html). This router duplicates a small, route-local _to_response() rather than importing research.py's, matching this codebase's established PAI-adapter-router convention.

Feature doc: 2026-07-25-tenant-scoped-research-triggers.md.

Bulk Export

Flag: enable_pai_bulk_export (default false). Every endpoint in this section returns 404 when the flag is off.

Serves pre-built, per-tier NDJSON snapshots of Phoenix datasets (backend/app/routers/bulk_export_pai.py). Artifacts are produced out-of-band by pipeline/build_bulk_exports.py and served as-is — the backend does not filter response data at request time, because the producer already wrote base and premium as physically separate files (data_shield cannot scale to millions-of-record exports).

Authentication. This surface admits PAI_INTERNAL keys only (PAI_EXPORT_SCOPES in pai_auth.py) — narrower than the rest of PAI. A phx_gintel_ (Global Intel bundle) key is rejected here even though it is admitted on /internal/v1/intel/*: global_intel is a paid customer entitlement, and this surface's licensing rationale holds only because every consumer is a first-party Phoenix platform. global_intel may lift the tier of an already-admitted key (resolve_export_tier); it never grants admission to this router.

Phase 1 has no delta endpoint. Every manifest entry carries "delta": {"supported": false} — there is no incremental/changed-since feed yet. Consumers must re-pull the full snapshot.

GET /internal/v1/export/manifest

Lists every dataset this export generation produced, tier-shaped for the caller. Free — it does not consume the bulk_export usage counter or the per-key concurrency slot described below, so discovery never costs a consumer anything (a metered discovery call would push consumers toward guessing dataset names instead of listing them).

{
  "schema_version": "1.0",
  "export_version": "v1",
  "generated_at": "2026-08-19T02:14:07Z",
  "tier": "base",
  "datasets": [
    {
      "dataset": "kev",
      "tier_required": "base",
      "accessible": true,
      "delta": {"supported": false},
      "source_freshness": {},
      "snapshot": {
        "watermark": "2026-08-19T02:00:00Z",
        "url": "/internal/v1/export/kev/snapshot",
        "records": 1284,
        "bytes": 483920,
        "sha256": "deadbeef...",
        "sha256_uncompressed": "c0ffee...",
        "cursor": "eyJkIjoia2V2IiwidCI6ImJhc2UiLCJzIjo0MTJ9",
        "snapshot_id": "kev-base-000412"
      }
    },
    {
      "dataset": "malware_packages",
      "tier_required": "premium",
      "accessible": false,
      "delta": {"supported": false},
      "source_freshness": {}
    }
  ]
}

malware_packages above is an ILLUSTRATIVE Phase 2 entry, not current output. Phase 1 registers exactly six datasets — cve_core, kev, epss, ransomware, exploitation_evidence, library_intel (pipeline/bulk_export/datasets.py) — and malware_packages is not among them, so no manifest produced by this build contains it. It is shown to illustrate the shape of an inaccessible entry: a premium dataset stays listed for a base caller with accessible: false and no snapshot block. Note source_freshness is present-but-empty rather than absent: shape_manifest_for_tier emits that key for every dataset entry, accessible or not.

• tier — the caller's resolved export tier (base or premium; see resolve_export_tier). Starts at base and is max-at-resolution — never lowers, never persists a derived row. On the live request path only a global_intel stamp lifts a caller to premium. resolve_export_tier also reads a per-key premium export-tier stamp and the org's mpi entitlement, but neither is reachable today: PAIKeyInfo has no export_tier field and nothing sets one, and bulk_export_pai._caller_and_tier calls the resolver without an entitlement_lookup. Both branches are unit-tested and dormant until a caller supplies them — do not document them as live inputs.
• accessible — whether the CALLER's tier satisfies tier_required for that dataset. A premium-only dataset stays listed for a base caller with accessible: false rather than being hidden — hiding it would make the 403 on a snapshot request (below) incomprehensible, and the dataset's existence is a published product, not a secret.
• snapshot — present only when accessible is true; snapshot.url is always /internal/v1/export/{dataset}/snapshot.
  • snapshot.cursor — the same opaque, resumable position as the X-Phoenix-Export-Cursor response header on a direct snapshot request for this dataset+tier (kept in sync). This is what a consumer persists; snapshot.watermark is informational only (R2 — never a wall-clock range key).
  • snapshot.snapshot_id — human-readable label for this build ({dataset}-{tier}-{seq:06d}), matching X-Phoenix-Export-Snapshot-Id. Convenient for logs/support, not a substitute for cursor.
• source_freshness — per-source freshness metadata for this dataset (DatasetSpec.source_freshness()), most relevant for a dataset that merges multiple upstream artifacts of differing age (e.g. exploitation_evidence). library_intel is the first dataset to populate it, returning {"generated_at": <library_intelligence.json's own _metadata.generated_at>}; every other dataset still returns {} — no serialiser has wired one yet — since the field exists so each producer can populate it independently without another manifest-shape change.
• 503 (with Retry-After: 3600) when this export generation has not finished building yet (manifest.json absent) — never a bare empty 200, which would read as "no datasets exist."

GET /internal/v1/export/{dataset}/snapshot

Streams the full NDJSON snapshot for one dataset at the caller's resolved tier, gzip-compressed on disk and served with Content-Encoding: gzip (the body is not re-compressed by the app-wide GZipMiddleware — the pre-set Content-Encoding header short-circuits it).

Response headers:

Conditional requests. Send If-None-Match: "<etag>"; a match returns 304 with an empty body and no re-transfer. Only If-None-Match is honoured — Last-Modified is informational and If-Modified-Since is not implemented.

Verifying integrity — read this before comparing digests. Two digests are published because no single one is reproducible by every client.

The response carries Content-Encoding: gzip, so a standard client decompresses before your code sees the bytes, and hashing what you received can never reproduce the ETag (which covers the compressed artifact). Comparing the wrong pair looks exactly like corruption. Both digests are watermark-independent: an unchanged dataset rebuilt tomorrow yields identical values, which is what makes 304 revalidation worthwhile.

Range requests. Lets a consumer resume a dropped multi-gigabyte transfer instead of restarting it. Support comes from the installed Starlette's FileResponse, and was verified live against the deployed runtime (Starlette 1.6.0): a Range: bytes=0-99 request returned 206 with exactly 100 bytes and a Content-Range header. Range handling was added to FileResponse well before 1.0, so the exact earliest supporting release is deliberately NOT stated here — only the version actually tested is. backend/requirements.txt pins starlette>=0.27.0, which does permit resolving a version older than the one verified, so an unchecked deployment must be treated as "may not support it": a Range request answered 200 with no Content-Range/Accept-Ranges has no range handling. If you must support both, treat 206 as the fast path and fall back to a full re-fetch when the response is 200.

Status codes:

Metering. /internal is exempt from the global usage-tracking middleware (main.py), so this router meters itself: every snapshot request (200 or 304 — the check runs before the conditional-request branch) increments the caller's exports counter one time via UserTierService().record_usage(subject, {"exports": 1}) — the same rail user_tier_service.FEATURE_USAGE_FIELDS already maps the bulk_export feature to. The subject is the key's admin_id (falling back to key_id), matching every other PAI usage-metering call site (intel_query_pai._usage_subject). GET /manifest records nothing — discovery stays free. A metering failure (e.g. storage unavailable) is logged and swallowed; it never fails the export request itself.

Per-key concurrency guard. PAI's standard rate limit (1000 calls/minute, config.py) is the wrong shape for full-corpus streams — one consumer looping a multi-gigabyte snapshot download would saturate the backend. This router additionally caps each key to EXPORT_CONCURRENT_STREAMS_PER_KEY (currently 1) snapshot streams in flight at a time, tracked as a cache-backed counter (CacheService.increment / CacheService.increment_with_expire / CacheService.decrement) that is incremented when a stream starts and released once the response has actually finished sending. A second request for the same key while one is still in flight gets 429 Export already in progress for this key.

The guard fails closed on a cache outage: if a concurrency lease cannot be established authoritatively, the snapshot route returns 503 Export concurrency control unavailable; retry shortly. This is a deliberate divergence from the fail-open convention every interactive PAI route follows, and it is scoped to this one streaming route — GET /internal/v1/export/manifest still serves normally during an outage, so a consumer can always discover state and back off.

The reasoning: PAI's ordinary per-key rate limiter depends on the same cache, and the exports counter above is accounting-only (this router calls no UserTierService.enforce_feature_limit against it), so failing open did not degrade one control — it removed every control at once, on the only route that streams multi-hundred-megabyte to multi-gigabyte bodies. One valid or stolen key could then open unlimited concurrent streams and exhaust disk and egress capacity precisely while the platform was already degraded. Refusing bulk egress that cannot be accounted for is the correct trade here. Note this remains a per-key concurrency cap, not a volume budget: no per-day export quota is enforced in this phase.

Cross-dataset consistency — NOT guaranteed

Datasets are built independently by separate producer runs and carry independent watermarks. A kev snapshot may reference a CVE not yet present in a cve_core snapshot; a malware record may reference a package version absent from a base dataset that has not rebuilt yet.

Consumers must tolerate dangling cross-dataset references. Do not build a foreign-key constraint across two mirrored datasets — it will hold in testing and fail in production.

This is a deliberate design trade-off, not an oversight: enforcing cross-dataset consistency would require a global build barrier across every producer, serialising the pipeline. The contract is eventual convergence — any dangling reference resolves within one build cycle of the referenced dataset.

Related: docs/plans/2026-08-19-pai-bulk-export-sync-design.md (design, §7-§9), docs/plans/2026-08-19-pai-bulk-export-phase1-implementation.md (implementation plan).

Org/Team Admin Tenancy

Flag: enable_org_team_admin_scoping (default false). All endpoints in this section return 404 when the flag is off.
Auth: Global admin or org manager (is_org_manager Cognito group) for the target org — enforced by resolve_admin_scope + ensure_target_org_in_scope.
Related: docs/Individual_Feature/2026-06-29-org-team-tenancy-plan-3-tenancy-router.md

Organizations

Teams

Members

GET /internal/v1/admin/organizations/{org_id}/members

Auth: Global admin or tenant_admin of that org. team_admin callers are auto-scoped to their own teams.

Query param: team_id (optional uuid) — filter by team. Overridden by auto-scoping for team_admin callers.

Response 200: { "members": [...] }

Each member carries the membership row plus the identity resolved from users (2026-08-26):

The identity join is a LEFT JOIN: a membership row whose sub has no users row is still listed, with the three identity fields null, rather than disappearing from its own org.

POST /internal/v1/admin/organizations/{org_id}/members

Auth: Global admin or tenant_admin of that org.

Add an existing user (by user_sub) to an org.

Errors: 400 invalid role; 403 TENANT_ADMIN_ESCALATION (tenant_admin caller cannot assign tenant_admin role); 409 seat limit.

Response 201: { "added": bool, "already_member": bool }

PATCH /internal/v1/admin/organizations/{org_id}/members/{user_sub}

Auth: Global admin or tenant_admin of that org.

Update a member's role. Request body: { "role": "member|viewer|team_admin|tenant_admin" }

Errors: 400 invalid role; 403 TENANT_ADMIN_ESCALATION; 404 member not found; 409 LAST_ADMIN (cannot demote the last tenant_admin).

DELETE /internal/v1/admin/organizations/{org_id}/members/{user_sub}

Auth: Global admin or tenant_admin of that org.

Remove a member. 409 LAST_ADMIN when removing the last tenant_admin. 204 on success.

PUT /internal/v1/admin/organizations/{org_id}/members/{user_sub}/team

Reassign a member to a team. Enterprise orgs only (403 ENTERPRISE_REQUIRED for non-enterprise).

Request body: { "team_id": uuid }

Invitations

POST /internal/v1/admin/organizations/{org_id}/users — Org-scoped user creation

Create a brand-new user and enroll them in the org. Provisions the user in Cognito (sends invite email) and inserts a users DB row.

Auth: Global admin or tenant_admin of that org. team_admin callers always receive 403 TEAM_ADMIN_DIRECT_CREATE_FORBIDDEN.

Request body

Response 201

{ "user_sub": "cognito-sub-uuid", "role": "member", "org_id": "...", "team_id": null }

Errors

Caller Scope Helpers

Admin Registrations (org-scoped)

Auth: require_admin (admin or global-admin Cognito group).

These are /api/v1/admin/ endpoints (not PAI). Documented here because their scoping behaviour changed as part of the org/team tenancy feature.

GET /api/v1/admin/registrations

List user registrations.

Scope: Global admin sees all rows. Non-global admins see only rows belonging to their primary org (get_user_primary_org). Non-global admins with no org membership receive an empty list.

Query parameters: status (optional), limit (default 100, max 1000), offset (default 0).

Response 200: Array of UserRegistrationListItem.

GET /api/v1/admin/registrations/pending

Convenience alias — equivalent to GET /registrations?status=pending. Same org-scoping rules.

Response 200: Array of UserRegistrationListItem with status=pending.

POST /api/v1/admin/registrations/{registration_id}/approve

Approve a pending registration.

Auth: require_global_admin. Unlike the GET endpoints above this is not org-scoped — only global admins may approve.

Query parameters: notes (optional str) — free-text note recorded with the approval.

Response 200: MessageResponse — { "success": true, "message": "..." }. The message is composed at runtime and names what actually happened, including the organization the user was bound to and any binding warning. Treat it as human-readable prose, not a parseable contract.

Errors:

• 404 — no registration with that id.
• 400 — registration is not pending (Cannot approve registration with status: <status>).
• 502 — Cognito account provisioning failed. The registration is left pending and the call is safe to retry.

Side effects:

• Sets the registration to approved and persists it via save_user_registration, which also writes the tenancy columns (tenant_id, requested_team_id, requested_role, intent) added by migration 102.
• Creates the Cognito account, mirrors a users row, and clears the stored password.
• Organization binding (flag enable_approval_org_binding, default off): creates scf_tenants + a default team + tenant_admin/team_admin membership in one transaction and sets users.primary_tenant_id, so the org licence resolves. Every approved registrant gets their own org; a slug clash creates a numbered variant rather than joining an existing tenant. Binding is best-effort — a failure is reported in the response message and recorded on the registration, but never fails the approval. With the flag off, no tenancy rows are written.
• Mints a core firewall key when scf_seat_licensing_enabled is on. The licence key is the bound tenant's canonical org_key when binding succeeded and resolved one. When binding did not run or did not bind (flag off, no organization on the registration, or a binding failure), the key falls back to normalize_org_slug(registration.organization) — the same normalized form the tenancy tables use, never the raw form entry. Licensing is skipped entirely in two cases: the organization has no canonical slug (empty or reserved), or binding succeeded but the bound tenant's org_key could not be resolved — deriving one from the form entry there would licence a different key than the tenant actually uses.
• Writes an audit entry via log_admin_action, and queues background notifications.

Neutral Defaults

PAI calculation endpoints apply neutral defaults when parameters are omitted:

• PS-HP: missing optional inputs default to neutral scoring (0.0 for components).
• PS-OSS (PSS): missing lists/counts default to 0; missing criticality uses the calculator's neutral 0.5.
• PS-PHS / PS-PVS: missing signals are treated as null and excluded via renormalization.
• ADQE: missing modifiers resolve to neutral (0).
• EOL SLR: missing category/vendor/status fall back to neutral multipliers and unknown status.