> ## Documentation Index > Fetch the complete documentation index at: https://docs.vainona.ai/llms.txt > Use this file to discover all available pages before exploring further. # Read the events feed > Every event the organization recorded in the last 30 days, oldest first in the order they were recorded, limited to the key's scope. Without `cursor` it starts at the oldest. `next_cursor` is always present, even on an empty page, so a poller resumes exactly where it stopped. `cursor` also takes an event id, to resume after an event a receiver got by push. A cursor or event older than 30 days is `invalid_request` with `details.reason: "cursor_expired"`: re-read the current state (for a subscription, its `query`) and start again without one. ## OpenAPI ````yaml /api-reference/openapi.yaml get /events openapi: 3.1.0 info: title: Vainona API version: '1' license: name: Proprietary identifier: LicenseRef-Proprietary description: | The `/v1` contract from main spec §6. It is locked: changes within `/v1` are additive only, and anything breaking is `/v2`. - **Base URL.** `https://api.vainona.ai/v1`. - **Evolution.** Response objects may gain fields and enums may gain values. Clients must ignore what they do not know. Request objects reject unknown fields. - **Path segments.** Namespace names (`{ns}`) and document ids (`{id}`) may contain `/`. Send it percent-encoded as `%2F`, so that `acme/prod/tenant_123` is `acme%2Fprod%2Ftenant_123`. - **Templates (v1.5).** The judgment routes also take a namespace prefix ending in `/*`, such as `acme%2Fprod%2F*`. A judgment created there is a template: every namespace under the prefix inherits it, including ones created later. The most specific matching prefix wins (§7.7). - **Thresholds are settings (v1.5).** Named thresholds belong to the judgment, not to a version. They are evaluated at read time against the raw fields, so a change applies at once to every answer, with no recompute and no new version (§6.5). - **Entity judgments (entities, E1).** A judgment can read the documents that point at the judged one, through `context.related`, and apply only to documents matching `applies_to` (§6.5.2). Its answers carry a `watermark`, and creating one that runs `on_change` returns a replay estimate of its monthly cost until you confirm (§6.9). - **Referenced documents (entities, E3a).** A relation can also read the one document the judged document points at, through `join: {theirs: "id", mine: "attributes."}`. A change to what the relation renders of that document re-judges the judged documents that point at it and are inside `freshness.fanout.scope`: a fan-out. A large fan-out is a `fanout` job that waits for confirm (§6.5.2, §6.9). - **Idempotency.** Every write operation is idempotent by construction: a retried append adds nothing while its values are still in the array, and a retried upsert or patch sets the same content again (§6.4). `Idempotency-Key` is accepted on the mutating routes that declare it and, while the node still caches the original response, returns it verbatim. Marking or unmarking a non-production prefix, and starting this month's recipe-tuning run, change nothing when repeated, so they take none. - **Errors.** Every error is the `Error` envelope; the HTTP status follows the code (see each response). - **Rate limits.** Every response carries `X-RateLimit-Limit` and `X-RateLimit-Remaining`; a `rate_limited` response adds `Retry-After`, except a daily allowance that is used up, which says when it renews in `details.resets_at` instead. - **Usage.** Every response that bills carries `usage`. Judging is billed in engine-neutral judgment units: one judgment answered, per started 1,000 tokens of its compiled context and its question together, at least 1 (§9). No response ever carries an engine's price. servers: - url: https://api.vainona.ai/v1 security: - apiKey: [] tags: - name: Namespaces description: Namespaces, their settings and cache warming (§6.2, §6.11). - name: Documents description: Writes, point reads and queries (§6.3 to §6.8). - name: Judgments description: >- Judgment definitions, versions, settings, activation, backfill and template detach (§6.5, §6.9, §7.7). - name: Outcomes and calibration description: | Post what actually happened to judged documents, and read how well answers match it: the calibration report, calibrated answers and threshold recommendations (§6.10). See [calibration](/concepts/calibration). - name: Jobs description: >- Backfill, shadow, periodic, deletion, reference index, fan-out and evaluation export jobs (§6.2, §6.9, §7.10.5). - name: Engines description: The engine registry (§6.5, §7.4.5). - name: Organization description: | Settings of the whole organization: prefixes marked non-production, which the tenant fee and template calibration pools leave out (§9). - name: Subscriptions description: | Saved queries that send an event when a document starts or stops matching: the query's filter grammar (§6.8), on a namespace or a template prefix, with the fields each event carries. - name: Webhooks description: | Webhook endpoints, the events feed, the delivery log, redelivery and recovery. Every event type and its body is under `webhooks`. Events are delivered at least once, signed as Standard Webhooks, and kept for 30 days. Webhooks are on every plan, and deliveries are not billed; plans set how many endpoints an organization has (§9). paths: /events: get: tags: - Webhooks summary: Read the events feed description: | Every event the organization recorded in the last 30 days, oldest first in the order they were recorded, limited to the key's scope. Without `cursor` it starts at the oldest. `next_cursor` is always present, even on an empty page, so a poller resumes exactly where it stopped. `cursor` also takes an event id, to resume after an event a receiver got by push. A cursor or event older than 30 days is `invalid_request` with `details.reason: "cursor_expired"`: re-read the current state (for a subscription, its `query`) and start again without one. operationId: listEvents parameters: - $ref: '#/components/parameters/Cursor' - name: limit in: query description: Events per page. schema: type: integer minimum: 1 maximum: 1000 default: 100 - name: types in: query description: >- Only these event types; `*` after a dot matches any, as in `subscription.*`. style: form explode: false schema: type: array uniqueItems: true items: $ref: '#/components/schemas/WebhookEventPattern' - name: namespace_prefix in: query description: >- Only events of namespaces whose name starts with this prefix, such as `acme/prod/`. schema: type: string maxLength: 256 responses: '200': description: One page of events. headers: X-RateLimit-Limit: $ref: '#/components/headers/RateLimitLimit' X-RateLimit-Remaining: $ref: '#/components/headers/RateLimitRemaining' content: application/json: schema: $ref: '#/components/schemas/EventList' '400': $ref: '#/components/responses/InvalidRequest' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '413': $ref: '#/components/responses/TooLarge' '429': $ref: '#/components/responses/RateLimited' '500': $ref: '#/components/responses/Internal' components: parameters: Cursor: name: cursor in: query description: The `next_cursor` of the previous page. schema: type: string schemas: WebhookEventPattern: type: string description: >- An event type such as `job.completed`, or a family with `*` after the dot, such as `job.*`. pattern: ^[a-z_]+\.([a-z_]+|\*)$ maxLength: 64 EventList: type: object required: - events - next_cursor properties: events: type: array maxItems: 1000 items: $ref: '#/components/schemas/WebhookEvent' next_cursor: type: string description: Where the next page starts. Always present, even on an empty page. WebhookEvent: type: object description: | An event, as pushed (the request body) and as the feed lists it: `id`, `type`, `timestamp` and `data`, discriminated on `type`. Types may be added, so ignore the ones you don't know. oneOf: - $ref: '#/components/schemas/SubscriptionEnteredEvent' - $ref: '#/components/schemas/SubscriptionExitedEvent' - $ref: '#/components/schemas/SubscriptionSyncedEvent' - $ref: '#/components/schemas/JobAwaitingConfirmEvent' - $ref: '#/components/schemas/JobCompletedEvent' - $ref: '#/components/schemas/JobFailedEvent' - $ref: '#/components/schemas/NamespaceBudgetPausedEvent' - $ref: '#/components/schemas/NamespaceBudgetResumedEvent' - $ref: '#/components/schemas/WebhookTestEvent' discriminator: propertyName: type mapping: subscription.entered: $ref: '#/components/schemas/SubscriptionEnteredEvent' subscription.exited: $ref: '#/components/schemas/SubscriptionExitedEvent' subscription.synced: $ref: '#/components/schemas/SubscriptionSyncedEvent' job.awaiting_confirm: $ref: '#/components/schemas/JobAwaitingConfirmEvent' job.completed: $ref: '#/components/schemas/JobCompletedEvent' job.failed: $ref: '#/components/schemas/JobFailedEvent' namespace.budget_paused: $ref: '#/components/schemas/NamespaceBudgetPausedEvent' namespace.budget_resumed: $ref: '#/components/schemas/NamespaceBudgetResumedEvent' webhook.test: $ref: '#/components/schemas/WebhookTestEvent' Error: type: object required: - error properties: error: type: object required: - code - message properties: code: $ref: '#/components/schemas/ErrorCode' message: type: string details: type: object description: >- Code-specific detail, such as the scan estimate on a refused query. Every `internal` error has `request_id`. properties: suggested_context: description: >- On a create without a context recipe, the recipe suggested from the namespace's documents, or null when there are none (D-v15-612). oneOf: - $ref: '#/components/schemas/SuggestedContext' - type: 'null' SubscriptionEnteredEvent: type: object required: - type - data allOf: - $ref: '#/components/schemas/WebhookEventCommon' properties: type: const: subscription.entered data: $ref: '#/components/schemas/SubscriptionTransition' SubscriptionExitedEvent: type: object required: - type - data allOf: - $ref: '#/components/schemas/WebhookEventCommon' properties: type: const: subscription.exited data: $ref: '#/components/schemas/SubscriptionExitedData' SubscriptionSyncedEvent: type: object required: - type - data allOf: - $ref: '#/components/schemas/WebhookEventCommon' properties: type: const: subscription.synced data: $ref: '#/components/schemas/SubscriptionSyncedData' JobAwaitingConfirmEvent: type: object required: - type - data allOf: - $ref: '#/components/schemas/WebhookEventCommon' properties: type: const: job.awaiting_confirm data: $ref: '#/components/schemas/JobEventData' JobCompletedEvent: type: object required: - type - data allOf: - $ref: '#/components/schemas/WebhookEventCommon' properties: type: const: job.completed data: $ref: '#/components/schemas/JobEventData' JobFailedEvent: type: object required: - type - data allOf: - $ref: '#/components/schemas/WebhookEventCommon' properties: type: const: job.failed data: $ref: '#/components/schemas/JobEventData' NamespaceBudgetPausedEvent: type: object required: - type - data allOf: - $ref: '#/components/schemas/WebhookEventCommon' properties: type: const: namespace.budget_paused data: $ref: '#/components/schemas/NamespaceBudgetData' NamespaceBudgetResumedEvent: type: object required: - type - data allOf: - $ref: '#/components/schemas/WebhookEventCommon' properties: type: const: namespace.budget_resumed data: $ref: '#/components/schemas/NamespaceBudgetData' WebhookTestEvent: type: object required: - type - data allOf: - $ref: '#/components/schemas/WebhookEventCommon' properties: type: const: webhook.test data: $ref: '#/components/schemas/WebhookTestData' ErrorCode: type: string description: | HTTP status by code: `invalid_request` 400, `unauthorized` 401, `budget_exceeded` 402, `plan_required` 402, `forbidden` 403, `not_found` 404, `conflict` 409, `too_large` 413, `engine_version_unavailable` 422, `insufficient_labels` 422, `rate_limited` 429, `internal` 500, `engine_unavailable` 503, `unavailable` 503. enum: - invalid_request - unauthorized - forbidden - not_found - conflict - too_large - rate_limited - budget_exceeded - plan_required - engine_unavailable - engine_version_unavailable - unavailable - insufficient_labels - internal SuggestedContext: type: object description: | The recipe suggested for a create without `context` (D-v15-612), in the refusal's `details.suggested_context`. From up to 100 of the documents the definition applies to: short text fields, numbers and small attributes are kept; ids, timestamps, URLs, base64 and long values are left out; arrays longer than 5 keep their last 5; and `max_tokens` caps the largest context at a multiple of 500, at most 2,000. The same documents always give the same recipe. required: - recipe - cost_per_answer - whole_state_cost_per_answer - sampled_documents - excluded properties: recipe: $ref: '#/components/schemas/ContextRecipe' cost_per_answer: type: - number - 'null' minimum: 0 description: >- Mean judgment units per answer over the sample with this recipe and the definition's question. whole_state_cost_per_answer: type: - number - 'null' minimum: 0 description: The same with the whole `state`. sampled_documents: type: integer minimum: 1 excluded: type: array description: >- Each path left out, and why, so you can put back what the question needs. items: type: object required: - path - reason properties: path: type: string reason: type: string enum: - id - timestamp - url - binary - long - rare - too_many WebhookEventCommon: type: object required: - id - timestamp properties: id: $ref: '#/components/schemas/EventId' timestamp: $ref: '#/components/schemas/Timestamp' description: When the event was recorded. test: type: boolean description: >- True on every event a test send sends, `webhook.test` or a sample of a named type. Absent otherwise, and never in the feed. SubscriptionTransition: type: object description: | A document that started matching (`subscription.entered`), and the body of `subscription.exited`. It carries the included attributes and answers, and those the filter reads, as the get returns them, never `state`. The answers are `fresh`, or absent where the filter tests absence. Past 64 KB they are dropped and `truncated` is true: follow `url`. required: - subscription - namespace - document - cause - sequence - url properties: subscription: $ref: '#/components/schemas/SubscriptionRef' namespace: $ref: '#/components/schemas/NamespaceName' description: >- The namespace the document is in, even for a template's subscription. document: $ref: '#/components/schemas/SubscriptionDocument' attributes: $ref: '#/components/schemas/Attributes' answers: $ref: '#/components/schemas/Answers' cause: type: string description: | What changed the document: `change` (a write, or an answer after one), `fanout` or `periodic`. With `bulk: "deliver"`, also `backfill` (with `job_id`) and `resync`. enum: - change - fanout - periodic - backfill - resync job_id: type: string description: The backfill job, when `cause` is `backfill`. sequence: type: integer format: int64 minimum: 0 description: >- Grows with every change the namespace's subscriptions record. For one subscription and document, a higher `sequence` is newer, so drop anything older than what you have; pushes can arrive out of order. url: type: string format: uri description: The document's get, to call with your own key. truncated: type: boolean description: >- True when `attributes` and `answers` were dropped to keep the body under 64 KB. SubscriptionExitedData: type: object description: >- A document that stopped matching, with its current values, which show why; or one that was deleted, with no values. required: - reason allOf: - $ref: '#/components/schemas/SubscriptionTransition' properties: reason: type: string enum: - no_longer_matches - deleted SubscriptionSyncedData: type: object description: >- Membership changed in bulk without an event per document. Re-read the members with the subscription's `query`. required: - subscription - namespace - reason - job_ids - entered - exited properties: subscription: $ref: '#/components/schemas/SubscriptionRef' namespace: $ref: '#/components/schemas/NamespaceName' reason: type: string description: >- `backfill`, `created` (the first sync in the namespace), `thresholds_changed` (a threshold edit or an activation) or `weights_changed` (new composite weights). enum: - backfill - created - thresholds_changed - weights_changed job_ids: type: array description: The backfill jobs, for `backfill`. items: type: string entered: type: integer format: int64 minimum: 0 description: Documents that started matching. exited: type: integer format: int64 minimum: 0 description: Documents that stopped matching. JobEventData: type: object required: - job - url properties: job: $ref: '#/components/schemas/Job' description: The job as `GET /jobs/{id}` returns it. url: type: string format: uri description: That route. NamespaceBudgetData: type: object required: - namespace - reason - month - budget_usd - spend_usd properties: namespace: $ref: '#/components/schemas/NamespaceName' reason: type: string description: >- `budget`: the namespace's monthly budget. `quickstart`: `default/quickstart`'s monthly limit. enum: - budget - quickstart month: type: string pattern: ^[0-9]{4}-(0[1-9]|1[0-2])$ description: The calendar month, such as `2026-09`. budget_usd: type: number minimum: 0 description: The budget, or quickstart's limit, in US dollars. spend_usd: type: number minimum: 0 description: Judging this month, in US dollars. WebhookTestData: type: object required: - endpoint - message properties: endpoint: $ref: '#/components/schemas/WebhookEndpointId' message: type: string ContextRecipe: type: object description: | What the engine sees. A create without it is refused with a suggested recipe, unless `use_context` asks for the suggestion or the whole `state` (D-v15-612); a version with no recipe sends the whole `state`, subject to the engine limit. additionalProperties: false properties: fields: type: array description: Paths included verbatim, in order. items: type: string pattern: ^(state|attributes)(\.[^.]+)+$ last_n: type: object description: Per array path, keep the last n elements. additionalProperties: type: integer minimum: 1 window: type: object description: >- Per array path whose elements have a timestamp field `at`, keep elements within the window. additionalProperties: $ref: '#/components/schemas/Duration' max_tokens: type: integer minimum: 1 description: Hard cap; the compiler truncates from the end of the last field. related: type: object description: | Entities, E1 (§6.5.2). Up to 4 relations, by name: documents in the same namespace that point at the judged one or, from E3a, the one document it points at. Each renders as a `related.` entry after `fields`, and a write that changes what it renders makes the judged document's answer `pending` until it is judged again (for a referenced document, only inside the re-judge scope; outside it, the answer is `stale` with `stale_reason` `referenced_changed`). minProperties: 1 maxProperties: 4 propertyNames: $ref: '#/components/schemas/Name' additionalProperties: $ref: '#/components/schemas/Relation' EventId: type: string description: Also the `webhook-id` header. pattern: ^evt_[0-9a-z]{26}$ Timestamp: type: string format: date-time description: RFC 3339, UTC. SubscriptionRef: type: object required: - id - name - defined_on properties: id: $ref: '#/components/schemas/SubscriptionId' name: $ref: '#/components/schemas/Name' defined_on: $ref: '#/components/schemas/NamespaceOrTemplate' description: >- Where the subscription was created, such as the template prefix `acme/prod/*`. NamespaceName: type: string description: >- Up to 256 bytes. `/` separates levels of the hierarchy, as in `acme/prod/tenant_123`. pattern: ^[A-Za-z0-9._:/-]+$ minLength: 1 maxLength: 256 SubscriptionDocument: type: object required: - id - incarnation properties: id: $ref: '#/components/schemas/DocumentId' revision: $ref: '#/components/schemas/Revision' description: Absent when the document was deleted. incarnation: $ref: '#/components/schemas/Revision' description: >- Tells two lives of one id apart, as when a document is deleted and written again. updated_at: $ref: '#/components/schemas/Timestamp' description: Absent when the document was deleted. Attributes: type: object description: >- Flat, filterable and sortable. At most 64 keys. Not sent to engines unless a context recipe names them. maxProperties: 64 propertyNames: $ref: '#/components/schemas/Name' additionalProperties: $ref: '#/components/schemas/AttributeValue' Answers: type: object description: Answers keyed by judgment name. propertyNames: $ref: '#/components/schemas/Name' additionalProperties: $ref: '#/components/schemas/Answer' Job: type: object description: | A job row (§7.10.5). A `shadow` job comes from activating a differing version without `force` (§6.9). It starts in `awaiting_confirm` and runs its sample while it waits; `report` fills in when the sample is done. `confirm` activates `version` and the job ends `done`; `cancel` leaves the active version unchanged. On a template prefix, `namespace` is the prefix. A `reference_index` job (entities, E1) builds the index an entity judgment's relations need (§6.5.2); it starts `running`, and `attribute` names the attribute it indexes. A `fanout` job (entities, E3a) re-judges the judged documents one referenced change touches (§6.9): `referenced` names the change and `estimate` its cost. It waits in `awaiting_confirm` unless the judgment's `fanout.confirm_jobs` is `false`. `confirm` is `budget_exceeded` when the estimate does not fit what is left of the namespace's budget, and `cancel` leaves the in-scope answers `stale`. An `evaluation_export` job (§9 Plans) starts `running`, counts evaluations written in `progress.documents_done`, never spends, and has `export`. required: - id - type - status - namespace - spend_usd - progress - created_at - updated_at properties: id: type: string type: type: string enum: - backfill - shadow - periodic - namespace_delete - reference_index - fanout - evaluation_export status: type: string enum: - estimating - awaiting_confirm - running - paused - done - failed - cancelled namespace: $ref: '#/components/schemas/NamespaceOrTemplate' judgment: $ref: '#/components/schemas/Name' estimate: oneOf: - $ref: '#/components/schemas/Estimate' - type: 'null' spend_usd: type: number minimum: 0 description: Judging billed by this job so far, in US dollars. progress: type: object required: - documents_done properties: documents_done: type: integer format: int64 minimum: 0 description: | Documents done so far. A `backfill` counts the documents it has judged, and only those in its scope (the judgment's `applies_to` and the backfill's `filters`), so the count climbs to `estimate.documents`. It passes over the rest without counting them. estimated_completion_at: oneOf: - $ref: '#/components/schemas/Timestamp' - type: 'null' error: type: - string - 'null' attribute: type: - string - 'null' pattern: ^attributes\.[A-Za-z0-9_-]+$ description: >- Entities, E1 (§6.5.2). `reference_index` jobs only, as `attributes.`. The attribute whose reference index the job builds. Null for other jobs. referenced: description: Entities, E3a (§6.9). `fanout` jobs only; null for other jobs. oneOf: - $ref: '#/components/schemas/ReferencedChange' - type: 'null' version: $ref: '#/components/schemas/JudgmentVersionNumber' description: Shadow jobs only. The version being activated. report: description: | Shadow jobs only. Null while the sample runs; `progress` counts the sampled documents. A composite version's job has a `CompositeShadowReport` (§6.5.1), and when it has too few labels it fails with an `error` that starts with `insufficient_labels`. oneOf: - $ref: '#/components/schemas/ShadowReport' - $ref: '#/components/schemas/CompositeShadowReport' - type: 'null' export: $ref: '#/components/schemas/EvaluationExport' description: >- `evaluation_export` jobs only (§9 Plans): the range, the evaluations written, and once done the download links. created_at: $ref: '#/components/schemas/Timestamp' updated_at: $ref: '#/components/schemas/Timestamp' WebhookEndpointId: type: string pattern: ^we_[0-9a-z]{26}$ Duration: type: string description: A whole number and a unit (`s`, `m`, `h` or `d`), such as `7d` or `1h`. pattern: ^[1-9][0-9]*[smhd]$ Relation: type: object description: | Entities, E1 (§6.5.2). Documents that point at the judged document. They are ordered newest `created_at` first: `window` keeps those created within the window, then `last_n` keeps the newest n. At least one of `last_n` and `window` is required, and at least one of `fields` and `aggregate`. A relation reads at most the newest 1,000 documents. Entities, E3a: with `join: {theirs: "id", mine: "attributes."}` the relation reads the one document the judged document points at, its referenced document, and takes neither `last_n` nor `window`. additionalProperties: false required: - match - join allOf: - description: A relation renders fields or aggregates. if: not: required: - fields then: required: - aggregate - description: | A relation over the documents that point at the judged one is bounded by `last_n` or `window`; one that reads the referenced document takes neither. if: properties: join: type: object properties: mine: const: id then: anyOf: - required: - last_n - required: - window else: not: anyOf: - required: - last_n - required: - window properties: match: $ref: '#/components/schemas/AttributeFilter' description: Which documents the relation reads. join: $ref: '#/components/schemas/RelationJoin' last_n: type: integer minimum: 1 maximum: 1000 description: >- Keep the newest n. Not on a relation that reads a referenced document. window: $ref: '#/components/schemas/Duration' description: >- Keep documents created within this long before the later of the judged document's newest write and its newest related creation, at the answer's watermark. An edit to an old related document never moves the window. Not on a relation that reads a referenced document. fields: type: array description: >- Paths to include from each document, rendered under `records`, newest first. From entities E3a, an entry may be a `BandedField`, which renders a number as its band. minItems: 1 items: oneOf: - $ref: '#/components/schemas/RelationField' - $ref: '#/components/schemas/BandedField' aggregate: $ref: '#/components/schemas/Aggregate' SubscriptionId: type: string pattern: ^sub_[0-9a-z]{26}$ Name: type: string description: >- A judgment, attribute or threshold name. Names are path segments in field references, so they never contain `.`. pattern: ^[A-Za-z0-9_-]+$ minLength: 1 maxLength: 128 NamespaceOrTemplate: type: string description: | A namespace name, or a template prefix: a namespace path ending in `/*`, such as `acme/prod/*`, which every namespace under `acme/prod/` inherits judgments from (§7.7). Up to 256 bytes. pattern: ^[A-Za-z0-9._:/-]+(/\*)?$ minLength: 1 maxLength: 256 DocumentId: type: string description: Up to 128 bytes. pattern: ^[A-Za-z0-9._:/-]+$ minLength: 1 maxLength: 128 Revision: type: integer format: int64 minimum: 0 AttributeValue: description: A string, number, boolean, string array or null. type: - string - number - boolean - array - 'null' items: type: string Answer: type: object description: | The current answer of one judgment for one document, discriminated on `type`. Only `type` and `freshness` are always present: numbers and provenance are absent when no evaluation has succeeded. oneOf: - $ref: '#/components/schemas/BoolAnswer' - $ref: '#/components/schemas/ChoiceAnswer' - $ref: '#/components/schemas/ScoreAnswer' discriminator: propertyName: type mapping: bool: $ref: '#/components/schemas/BoolAnswer' choice: $ref: '#/components/schemas/ChoiceAnswer' score: $ref: '#/components/schemas/ScoreAnswer' Estimate: type: object required: - documents - tokens - judgment_units - cost_usd - duration_s properties: documents: type: integer format: int64 minimum: 0 tokens: type: integer format: int64 minimum: 0 judgment_units: type: integer format: int64 minimum: 0 description: >- Estimated judgment units (§9), from a sample of up to 1,000 documents. cost_usd: type: number minimum: 0 description: >- What the backfill is expected to cost you, the judgment units at the graduated prices of the tiers your organization will be in, counting what it has already used this month (§9). duration_s: type: integer format: int64 minimum: 0 description: >- Expected wall-clock seconds at the bulk pool's current share of the engine's rate limit. ReferencedChange: type: object description: | Entities, E3a (§6.9). The change a `fanout` job fans out: a write to the referenced document that changed what the relation renders. While the job waits for confirm, a later change to the same document moves `revision` to it, and one confirm covers them all. required: - relation - document_id - revision properties: relation: $ref: '#/components/schemas/Name' document_id: $ref: '#/components/schemas/DocumentId' revision: $ref: '#/components/schemas/Revision' description: >- The referenced document's revision at the change. Every judged document the job re-judges gets an answer with a `watermark` at or above it. JudgmentVersionNumber: type: integer format: int32 minimum: 1 ShadowReport: type: object description: | What activating `version` would change, from a shadow sample of 1,000 random documents, or every document when there are fewer (§7.3.11). Shadow evaluations are in the evaluation log with `shadow: true` and never produce answers. `current` is the answers of the active version for the sampled documents, and `candidate` the shadow evaluations. required: - documents - current - candidate - threshold_flips - recompute properties: documents: type: integer format: int64 minimum: 0 description: Documents in the sample. current: $ref: '#/components/schemas/ShadowSide' candidate: $ref: '#/components/schemas/ShadowSide' threshold_flips: type: object description: | Per named threshold, the sampled documents whose threshold boolean would change. The candidate side uses the thresholds that apply once the version is active. propertyNames: $ref: '#/components/schemas/Name' additionalProperties: $ref: '#/components/schemas/ThresholdFlips' recompute: $ref: '#/components/schemas/Estimate' description: >- What a backfill of every document under the new version would cost, as a backfill estimate. CompositeShadowReport: type: object description: | The shadow report of a composite version (§6.5.1). The job judged up to 1,000 documents with outcomes for this judgment and fitted the combiner with 5-fold cross-validation, so every document is scored by a fit that never saw its label. The baseline gets the same treatment: its cut-off is fitted by the same folds on the same outcomes, never the raw 0.5, so splitting gets no credit for what a fitted threshold alone would give. `verdict` is `better` only when the composite's accuracy beats the baseline's by more than the interval of the difference. Otherwise it is `not_better`, and the threshold recommender on the baseline is the cheaper fix. `confirm` activates the version either way, with the combiner fitted on all the documents. required: - type - documents - labels - composite - baseline - difference - parts - verdict - recompute properties: type: const: composite documents: type: integer format: int64 minimum: 0 description: Labelled documents judged and scored. labels: type: object description: The documents by their outcome. required: - 'true' - 'false' properties: 'true': type: integer format: int64 minimum: 0 'false': type: integer format: int64 minimum: 0 composite: $ref: '#/components/schemas/CrossValidated' baseline: $ref: '#/components/schemas/Baseline' difference: type: object description: >- The composite's cross-validated accuracy minus the baseline's, with its 95% interval over the documents (paired). required: - accuracy - interval properties: accuracy: type: number minimum: -1 maximum: 1 interval: $ref: '#/components/schemas/DifferenceInterval' parts: type: array description: >- Each part's weight in the combiner fitted on all the documents, on standardised log-odds, so sizes compare. A negative weight means a yes points to `false`. items: $ref: '#/components/schemas/PartWeight' features: type: array description: >- Entities, E1 (§6.5.1). Each feature's weight in the same fit, on the same standardised scale as the parts'. items: $ref: '#/components/schemas/FeatureWeight' verdict: type: string enum: - better - not_better recompute: $ref: '#/components/schemas/Estimate' description: >- What a backfill of every document under the new version would cost, as a backfill estimate. Each part is billed as a judgment. EvaluationExport: type: object description: | An `evaluation_export` job's range and files (§9 Plans). `files` is empty until the job is `done`; then `GET /jobs/{id}` links each file until `expires_at`, when the first file is deleted. Other routes that return the job leave `files` empty. When links can't be made right now, the files are listed with `url` null and `links_unavailable` says so: read the job again later. required: - since - until - judgments - evaluations - files - expires_at - links_unavailable properties: since: $ref: '#/components/schemas/Timestamp' until: $ref: '#/components/schemas/Timestamp' judgments: description: The judgments exported; null for every judgment. oneOf: - type: array items: $ref: '#/components/schemas/Name' - type: 'null' evaluations: type: integer format: int64 minimum: 0 description: Evaluations written so far, the same as `progress.documents_done`. files: type: array items: $ref: '#/components/schemas/EvaluationExportFile' expires_at: description: >- When the files are deleted, 7 days after the first was written. Null until the job is done, and when it wrote no file. oneOf: - $ref: '#/components/schemas/Timestamp' - type: 'null' links_unavailable: type: - string - 'null' description: >- Why some or all of `files` have no `url` right now, with a request id for support. Null when every file has its link. The job and its files are unaffected; read the job again later for new links. AttributeFilter: type: object description: | Entities, E1 (§6.5). Which documents something applies to, by their attributes. Each key is an attribute path; a value is equality, and a list of values is `In`, as in query filters. Keys combine with `And`. minProperties: 1 maxProperties: 8 propertyNames: type: string pattern: ^attributes\.[A-Za-z0-9_-]+$ additionalProperties: oneOf: - $ref: '#/components/schemas/AttributeFilterValue' - type: array minItems: 1 items: $ref: '#/components/schemas/AttributeFilterValue' RelationJoin: description: | How a related document is matched to the judged one: one side is `id`, the other an attribute holding an id. A join with an attribute on both sides, documents sharing a key, is E3b and `invalid_request`. oneOf: - $ref: '#/components/schemas/ReferringJoin' - $ref: '#/components/schemas/ReferencedJoin' RelationField: type: string description: A path in a related document, `id`, `created_at` or `updated_at`. pattern: ^((state|attributes)(\.[^.]+)+|id|created_at|updated_at)$ BandedField: type: object description: | Entities, E3a (§6.5.2). A number rendered as its band rather than its value, in any relation. A number renders as the label whose position is how many cut points are at or below it: with `bands: [0.3, 0.7]`, 0.29 is `labels[0]`, 0.3 is `labels[1]` and 0.7 or more `labels[2]`. A value that is not a number renders unchanged, and a missing one is left out. Only a move to another band changes the context, so a move inside one neither fans out nor costs an evaluation. `bands` must be strictly increasing and `labels` must have exactly one more entry; the server refuses anything else with `invalid_request`. additionalProperties: false required: - path - bands - labels properties: path: type: string description: A `state` or `attributes` path. pattern: ^(state|attributes)(\.[^.]+)+$ bands: type: array description: Cut points, strictly increasing. minItems: 1 maxItems: 9 items: type: number labels: type: array description: One label per band, lowest first, so one more than `bands`. minItems: 2 maxItems: 10 items: type: string minLength: 1 maxLength: 64 Aggregate: type: object description: | Numbers over the relation's documents, after `match`, `window` and `last_n`, rendered as keys of its `related.` entry, such as `count` and `sum(state.amount)`. At most 8 paths in total. `sum`, `min` and `max` skip values that are not numbers; `sum` of nothing is 0, and `min` and `max` of nothing are null. `latest` is the value in the newest document that has one, of any JSON type. additionalProperties: false minProperties: 1 properties: count: const: true description: How many documents the relation selected. sum: $ref: '#/components/schemas/AggregatePaths' min: $ref: '#/components/schemas/AggregatePaths' max: $ref: '#/components/schemas/AggregatePaths' latest: $ref: '#/components/schemas/AggregatePaths' BoolAnswer: type: object description: A bool answer has no `value`; booleans come only from named thresholds. required: - type allOf: - $ref: '#/components/schemas/AnswerCommon' properties: type: const: bool p: $ref: '#/components/schemas/Probability' description: >- The engine's probability, or for a composite judgment the probability its combiner gives the parts (§6.5.1). Thresholds, filters and ranking use it. calibrated: $ref: '#/components/schemas/BoolCalibration' parts: type: object description: Composite judgments only. Each part's raw `p` from the engine. propertyNames: $ref: '#/components/schemas/Name' additionalProperties: $ref: '#/components/schemas/Probability' features: $ref: '#/components/schemas/FeatureValues' combiner: $ref: '#/components/schemas/CombinerUse' ChoiceAnswer: type: object required: - type allOf: - $ref: '#/components/schemas/AnswerCommon' properties: type: const: choice value: type: string description: The most probable option; ties go to the earlier option. dist: $ref: '#/components/schemas/Distribution' escape_p: $ref: '#/components/schemas/Probability' description: The probability of `none_of_the_above`. calibrated: $ref: '#/components/schemas/ChoiceCalibration' ScoreAnswer: type: object required: - type allOf: - $ref: '#/components/schemas/AnswerCommon' properties: type: const: score score: type: number description: The probability-weighted mean of level values. dist: $ref: '#/components/schemas/Distribution' calibrated: $ref: '#/components/schemas/ScoreCalibration' ShadowSide: type: object description: One side of a shadow report, discriminated on the judgment `type`. oneOf: - $ref: '#/components/schemas/ShadowValues' - $ref: '#/components/schemas/ShadowOptions' discriminator: propertyName: type mapping: bool: $ref: '#/components/schemas/ShadowValues' score: $ref: '#/components/schemas/ShadowValues' choice: $ref: '#/components/schemas/ShadowOptions' ThresholdFlips: type: object required: - to_true - to_false properties: to_true: type: integer format: int64 minimum: 0 description: Documents false under the active version and true under the new one. to_false: type: integer format: int64 minimum: 0 description: Documents true under the active version and false under the new one. CrossValidated: type: object description: Out-of-fold scores over the report's documents. required: - accuracy - accuracy_interval - auc properties: accuracy: $ref: '#/components/schemas/Probability' description: >- Share of documents whose out-of-fold probability is on the right side of 0.5. accuracy_interval: $ref: '#/components/schemas/MetricInterval' auc: $ref: '#/components/schemas/Probability' description: ROC AUC of the out-of-fold probabilities. Baseline: type: object description: >- The fair baseline, scored like the composite. The active version (`version`), or for a first version the composite's `question` asked on its own (`version` null). required: - version - threshold - accuracy - accuracy_interval - auc allOf: - $ref: '#/components/schemas/CrossValidated' properties: version: oneOf: - $ref: '#/components/schemas/JudgmentVersionNumber' - type: 'null' threshold: description: >- The baseline's fitted cut-off on its `p`, fitted on all the documents. Null when a higher `p` does not mean `true` more often. oneOf: - $ref: '#/components/schemas/Probability' - type: 'null' DifferenceInterval: type: object required: - lower - upper properties: lower: type: number minimum: -1 maximum: 1 upper: type: number minimum: -1 maximum: 1 PartWeight: type: object required: - name - weight properties: name: $ref: '#/components/schemas/Name' weight: type: number FeatureWeight: type: object required: - name - weight properties: name: $ref: '#/components/schemas/FeatureRef' weight: type: number EvaluationExportFile: type: object description: One gzipped JSON Lines file of an evaluation export. required: - url - url_expires_at - evaluations - bytes properties: url: description: >- A signed download link. It needs no API key and works until `url_expires_at`. Null when the link can't be made right now (`links_unavailable` says why). oneOf: - type: string format: uri - type: 'null' url_expires_at: description: When `url` stops working; null with it. oneOf: - $ref: '#/components/schemas/Timestamp' - type: 'null' evaluations: type: integer format: int64 minimum: 1 description: Lines in the file. bytes: type: integer format: int64 minimum: 1 description: The file's size, compressed. AttributeFilterValue: type: - string - number - boolean ReferringJoin: type: object description: | Entities, E1 (§6.5.2). The documents that point at the judged one: a related document belongs to the judged document whose `id` equals its `theirs` attribute. additionalProperties: false required: - theirs - mine properties: theirs: type: string description: >- The related document's attribute that holds the judged document's id, such as `attributes.parent_id`. pattern: ^attributes\.[A-Za-z0-9_-]+$ mine: const: id ReferencedJoin: type: object description: | Entities, E3a (§6.5.2). The one document the judged document points at, its referenced document: the document whose `id` equals the judged document's `mine` attribute. A change to what the relation renders of it re-judges the judged documents that point at it and are inside `freshness.fanout.scope`. additionalProperties: false required: - theirs - mine properties: theirs: const: id mine: type: string description: >- The judged document's attribute that holds the referenced document's id, such as `attributes.group_id`. pattern: ^attributes\.[A-Za-z0-9_-]+$ AggregatePaths: type: array minItems: 1 maxItems: 8 uniqueItems: true items: type: string description: A `state` or `attributes` path. Keys in it cannot contain `(` or `)`. pattern: ^(state|attributes)(\.[^.()]+)+$ AnswerCommon: type: object required: - freshness properties: freshness: $ref: '#/components/schemas/Freshness' stale_reason: $ref: '#/components/schemas/StaleReason' description: Present exactly when `freshness` is `stale`. revision: $ref: '#/components/schemas/Revision' description: The document revision the numbers were computed for. watermark: $ref: '#/components/schemas/Revision' description: | Entity judgments only (entities, E1, §6.6): the log position the context was read at. Every write with a revision at or below it, to the document or its related documents, is reflected; a later one that touched the document makes the answer `pending` or `stale`. For a relation that reads a referenced document (entities, E3a), it also says which version of that document the answer read. A later write to it that changes what the relation renders makes the answer `pending` when the document is inside the re-judge scope, and `stale` (`referenced_changed`) when it is outside, so a referenced document with a `revision` above the watermark shows the answer read an earlier version of it. Absent on a `failed` answer: its numbers are the last good ones, and the failed attempt's position does not describe them. referenced_changes: type: array description: | Entities, E3a (§6.6). Get only, never in query rows. For a judgment with a relation that reads a referenced document: for each such relation, the newest write to the document the judged document points at now that changed what the relation renders, whether or not the document is inside the re-judge scope for it. A relation whose referenced document has never changed is left out, and so is the key when none has. A change at or below `watermark` is reflected in the answer. One above it makes the answer `pending` or `stale` when the document is inside the scope, and `stale` (`referenced_changed`) when it is outside. items: $ref: '#/components/schemas/AnswerReferencedChange' judgment_version: $ref: '#/components/schemas/JudgmentVersionNumber' engine: type: string engine_version: type: string evaluation_id: type: string description: | The evaluation that computed these numbers. When a change leaves the compiled context the same, the answer is reused for the new revision without calling the engine, and this still names the earlier evaluation, whose `revision` (and, for an entity judgment, `watermark`) is older than the answer's. evaluated_at: $ref: '#/components/schemas/Timestamp' thresholds: type: object description: >- The judgment's current thresholds (its setting, not the answer's version), evaluated at read time against the raw fields. propertyNames: $ref: '#/components/schemas/Name' additionalProperties: type: boolean Probability: type: number minimum: 0 maximum: 1 description: | Answers store probabilities as 32-bit floats and return them as the shortest decimal that stands for the stored value, so an engine's 0.01 reads 0.01. Thresholds, query filters and threshold recommendations compare that decimal (§7.1.6). BoolCalibration: type: object required: - p allOf: - $ref: '#/components/schemas/CalibrationCommon' properties: p: $ref: '#/components/schemas/Probability' FeatureValues: type: object description: >- Composite judgments with features only (entities, E1, §6.5.1). Each feature's raw value, as rendered in the context; null when missing. propertyNames: $ref: '#/components/schemas/FeatureRef' additionalProperties: type: - number - 'null' CombinerUse: type: object description: | Composite judgments only: the combiner `p` came from, fitted on the judgment's outcomes per version and engine epoch (§6.5.1). A composite's answer has no `calibrated` object. Being fitted to outcomes makes the combined `p` close to calibrated on data like its labels, not certainly; the calibration report's reliability curve is how to check. required: - outcomes - from_previous_epoch properties: outcomes: type: integer format: int64 minimum: 50 description: The labelled documents the combiner was fitted on. from_previous_epoch: type: boolean description: >- True after a Jev drift, while the answer's epoch has fewer than 50 outcomes, so the previous epoch's combiner applies. Distribution: type: object description: Probability per option or level value. Sums to 1. additionalProperties: $ref: '#/components/schemas/Probability' ChoiceCalibration: type: object required: - value - dist - escape_p allOf: - $ref: '#/components/schemas/CalibrationCommon' properties: value: type: string description: The most probable option under the calibrated distribution. dist: $ref: '#/components/schemas/Distribution' escape_p: $ref: '#/components/schemas/Probability' ScoreCalibration: type: object required: - score - dist allOf: - $ref: '#/components/schemas/CalibrationCommon' properties: score: type: number description: >- The probability-weighted mean of level values under the calibrated distribution. dist: $ref: '#/components/schemas/Distribution' ShadowValues: type: object description: The mean and histogram of `p` (bool) or `score` (score). required: - type - mean - histogram properties: type: type: string enum: - bool - score mean: type: number histogram: type: array description: >- Ten equal-width bins, from 0 to 1 for `p` and from the lowest to the highest level for `score`, lowest first. minItems: 10 maxItems: 10 items: $ref: '#/components/schemas/HistogramBin' ShadowOptions: type: object description: >- The mean probability per option, so the per-option shift is `candidate.dist` minus `current.dist`. required: - type - dist properties: type: const: choice dist: $ref: '#/components/schemas/Distribution' MetricInterval: type: object required: - lower - upper properties: lower: $ref: '#/components/schemas/Probability' upper: $ref: '#/components/schemas/Probability' FeatureRef: type: string description: | Entities, E1 (§6.5.1). An aggregate the recipe declares, as `.count` or `.()`, such as `tickets.count` or `invoices.sum(state.amount)`. pattern: >- ^[A-Za-z0-9_-]+\.(count|(sum|min|max|latest)\((state|attributes)(\.[^.()]+)+\))$ Freshness: type: string description: | Derived at read time (§7.1.6). `pending`: an evaluation is on its way, including for a document never judged whose `on_change` judgment has yet to reach its write. `stale` and `failed` still carry the last good numbers and the `revision` they were computed for; a `stale` answer says why in `stale_reason`. A `failed` answer for a document never judged successfully has no numbers. `unavailable`: never judged in this incarnation, and nothing will judge it on its own. enum: - fresh - pending - stale - failed - unavailable StaleReason: type: string description: | Why an answer is `stale` (§7.1.6). `document_changed`: the document (or something that touches it, for a judgment with related documents) changed, and the policy is `on_read`, `manual` or `periodic`. `judging_paused`: it changed while judging is paused, by the namespace's budget or the organization's unpaid charge. `fanout_declined`: a referenced document changed inside the re-judge scope and that change's `fanout` job was cancelled. `referenced_changed`: a referenced document the answer read changed, and the document is outside the re-judge scope, so nothing re-judges it. `interval_elapsed`: a `periodic` answer older than its interval. enum: - document_changed - judging_paused - fanout_declined - referenced_changed - interval_elapsed AnswerReferencedChange: type: object description: | Entities, E3a (§6.6, §7.1.14). A referenced change row: the newest write to a referenced document that changed what a relation renders. required: - relation - document_id - revision - at - declined properties: relation: $ref: '#/components/schemas/Name' document_id: $ref: '#/components/schemas/DocumentId' revision: $ref: '#/components/schemas/Revision' description: The referenced document's revision at the write. at: $ref: '#/components/schemas/Timestamp' description: The write's `updated_at`, which the re-judge scope counts back from. declined: type: boolean description: >- The change's `fanout` job was cancelled, so its in-scope answers read `stale` until the judged or referenced document next changes (§6.9). CalibrationCommon: type: object description: > The answer's numbers under the judgment's calibration (§6.6, §6.10), computed at read time from the current fit for the answer's judgment version and engine epoch. Present once that fit rests on at least 100 outcomes with at least 20 of each class (`true` and `false` for a bool; two values with 20 each for a choice or score), and beat the raw answers on held-out outcomes (the report's `held_out`). The raw fields never change meaning and are never replaced. Calibration never reverses the order of answers. See [calibration](/concepts/calibration). required: - method - outcomes - from_previous_epoch - extrapolated properties: method: $ref: '#/components/schemas/CalibrationMethod' outcomes: type: integer format: int64 minimum: 100 description: The number of outcomes the calibration was fitted on. from_previous_epoch: type: boolean description: | True after a Jev drift, while the answer's epoch has no fit of its own yet: the calibration is the previous epoch's. A replay of earlier outcomes usually fits the new epoch within the hour, and the previous epoch's fit is used for at most 30 days from the day the epoch began. extrapolated: type: boolean description: | True when the answer's raw value (`p`, or the probability of the most probable option or level) lies outside the range of raw values the fit was made on. The calibrated numbers are then a guess; post outcomes for documents like this one to cover it. HistogramBin: type: object description: Values in `[lower, upper)`; the last bin includes `upper`. required: - lower - upper - count properties: lower: type: number upper: type: number count: type: integer format: int64 minimum: 0 CalibrationMethod: type: string description: >- `platt`: from 100 outcomes, Platt scaling for `bool` and temperature scaling over the distribution for `choice` and `score`. `isotonic`: isotonic regression, from 1,000. enum: - platt - isotonic headers: RateLimitLimit: description: Requests allowed per second for this key. schema: type: integer minimum: 0 RateLimitRemaining: description: Requests left in the current window. schema: type: integer minimum: 0 RetryAfter: description: Seconds to wait before retrying. schema: type: integer minimum: 0 responses: InvalidRequest: description: '`invalid_request`: the request is malformed or breaks a rule of §6.' content: application/json: schema: $ref: '#/components/schemas/Error' Unauthorized: description: '`unauthorized`: the key is missing, unknown or revoked.' content: application/json: schema: $ref: '#/components/schemas/Error' Forbidden: description: '`forbidden`: the key''s role or namespace prefix does not allow this.' content: application/json: schema: $ref: '#/components/schemas/Error' TooLarge: description: >- `too_large`: the request body (on any route), a document, or the estimated query scan is over its limit. `details` carries the estimate for queries. It also answers a write that would take `default/quickstart` past its size cap (10,000 documents or 100 MB), with `details.max_documents` and `details.max_bytes`. content: application/json: schema: $ref: '#/components/schemas/Error' RateLimited: description: >- `rate_limited`: slow down. `Retry-After` says for how long, except when a daily allowance is used up: then `details.resets_at` says when it renews, and there is no `Retry-After`. headers: X-RateLimit-Limit: $ref: '#/components/headers/RateLimitLimit' X-RateLimit-Remaining: $ref: '#/components/headers/RateLimitRemaining' Retry-After: $ref: '#/components/headers/RetryAfter' content: application/json: schema: $ref: '#/components/schemas/Error' Internal: description: >- `internal`: something failed on our side. Nothing partial is returned, and a request never commits part of itself. Most `internal` errors committed nothing. A write can instead fail with a message saying it may have been committed, when we could not tell whether its batch landed (§7.2.10): it then committed all of it or none of it. Retrying a write is safe either way, because writes are idempotent. The message is generic and carries a request id, also in `details.request_id`, which support uses to find the cause. content: application/json: schema: $ref: '#/components/schemas/Error' securitySchemes: apiKey: type: http scheme: bearer description: | An organization API key. Keys carry a role (`read_write` or `read_only`) and may be restricted to a namespace prefix such as `acme/*`, or to one namespace such as `acme/prod/tenant_1`. A prefix matches on a `/` boundary: `acme/prod/tenant_1*` covers `acme/prod/tenant_1` and everything under `acme/prod/tenant_1/`, never `acme/prod/tenant_12`. ````