Skip to main content

Evaluations and watermarks

State machine​

pending  → processing → complete
                    ↘ error      (batch-level failure only)
  • pending — row created, worker not yet started. Usually milliseconds.
  • processing — worker running.
  • complete — all scenarios finished; per-scenario results populated.
  • error — batch-level failure before any results were produced. Top-level error field carries the reason.

Per-scenario failures do not flip the batch to error. The batch completes; the failing scenario's entry carries its own error string.

Idempotency​

  • POST /evaluate is not idempotent — each call creates a new evaluation. Use Idempotency-Key? Not currently supported; deduplicate client-side if you need at-most-once submission.
  • GET /evaluations/{id} and GET /scenarios/{id}/results are idempotent and cacheable.
  • GET /violations/{id}/remediation is idempotent and cacheable per scenario_id:policy_code until the next re-evaluation.
  • POST /results/fetch is not idempotent — the first successful call consumes the token.

Watermarks​

Every authenticated response carries a meta.watermark of the form wm_<8 hex>. The watermark is logged server-side against the calling API key.

"meta": { "watermark": "wm_a1b2c3d4" }
  • Watermarks are opaque to clients — do not parse them.
  • They are unique per response.
  • They are used for leak tracing: if a sanitised result leaks externally, the watermark identifies the calling key and the request that produced it.
  • Safe to include in support tickets to speed up correlation.

Anomaly detection​

The middleware tracks per-key activity and surfaces internal observability for suspicious patterns (request rate spikes, IP churn, unusual error mix). Today this is platform-internal — partners do not see anomaly events.

Enabling PUBLIC_API_ANOMALY_AUTO_SUSPEND=true causes the platform to auto-suspend keys exceeding configured thresholds. This flag is off by default in production.

Result tokens (single-use JWT)​

POST /evaluate returns a result_token JWT alongside the evaluation_id. Properties:

  • TTL: 900 seconds (15 minutes).
  • Single-use: consumed on the first successful POST /results/fetch.
  • Bound to one evaluation_id.
  • Sent in the request body, not the URL — POST /results/fetch with {"result_token":"..."} — so it never lands in access logs.
  • Still authenticated: like every endpoint except /governance/health, the call carries the Bearer key and X-FB-Signature. The token is an additional single-use credential, not a replacement for signing.

If the backing evaluation is not yet complete when the token is presented, the response returns { ok: true, data: { status: "processing", message: "Results not yet available" } } and the token is not consumed. Wait until the evaluation reports complete, then fetch once.

Last updated on