> ## Documentation Index > Fetch the complete documentation index at: https://docs.vainona.ai/llms.txt > Use this file to discover all available pages before exploring further. # Draw documents to label > Draws up to `count` documents to label (§6.10), a random sample stratified across the 10 probability bands: `count` is split evenly over the bands with documents to give, and what a band cannot fill goes to the others. Within a band, documents come in an order fixed by `seed`, so the same state and seed give the same items. Documents already labelled from the queue for their current answer, and those leased to someone else, are skipped. Each draw leases its documents for 7 days, so two people are not given the same one. Post each label as an ordinary outcome with the item's `queue_item_id`: it counts as `source: queue`. Once an epoch's queue labels alone meet the calibration minimums, the epoch is fitted on them, each weighted by its band's share of the answers, so the rare bands the queue oversamples count for what they are in real traffic. Drawing is part of the learning loop, on the Team plan and above: below it, `plan_required` (§9). The preview (`GET`) is on every plan. A judgment with a non-zero `horizon` has no queue (`invalid_request`). ## OpenAPI ````yaml /api-reference/openapi.yaml post /namespaces/{ns}/judgments/{name}/labelling-queue openapi: 3.1.0 info: title: Vainona API version: '1' license: name: Proprietary identifier: LicenseRef-Proprietary description: | The `/v1` contract from main spec §6. It is locked: changes within `/v1` are additive only, and anything breaking is `/v2`. - **Base URL.** `https://api.vainona.ai/v1`. - **Evolution.** Response objects may gain fields and enums may gain values. Clients must ignore what they do not know. Request objects reject unknown fields. - **Path segments.** Namespace names (`{ns}`) and document ids (`{id}`) may contain `/`. Send it percent-encoded as `%2F`, so that `acme/prod/tenant_123` is `acme%2Fprod%2Ftenant_123`. - **Templates (v1.5).** The judgment routes also take a namespace prefix ending in `/*`, such as `acme%2Fprod%2F*`. A judgment created there is a template: every namespace under the prefix inherits it, including ones created later. The most specific matching prefix wins (§7.7). - **Thresholds are settings (v1.5).** Named thresholds belong to the judgment, not to a version. They are evaluated at read time against the raw fields, so a change applies at once to every answer, with no recompute and no new version (§6.5). - **Entity judgments (entities, E1).** A judgment can read the documents that point at the judged one, through `context.related`, and apply only to documents matching `applies_to` (§6.5.2). Its answers carry a `watermark`, and creating one that runs `on_change` returns a replay estimate of its monthly cost until you confirm (§6.9). - **Referenced documents (entities, E3a).** A relation can also read the one document the judged document points at, through `join: {theirs: "id", mine: "attributes."}`. A change to what the relation renders of that document re-judges the judged documents that point at it and are inside `freshness.fanout.scope`: a fan-out. A large fan-out is a `fanout` job that waits for confirm (§6.5.2, §6.9). - **Idempotency.** Every write operation is idempotent by construction: a retried append adds nothing while its values are still in the array, and a retried upsert or patch sets the same content again (§6.4). `Idempotency-Key` is accepted on every mutating route and, while the node still caches the original response, returns it verbatim. - **Errors.** Every error is the `Error` envelope; the HTTP status follows the code (see each response). - **Rate limits.** Every response carries `X-RateLimit-Limit` and `X-RateLimit-Remaining`; a `rate_limited` response adds `Retry-After`, except a daily allowance that is used up, which says when it renews in `details.resets_at` instead. - **Usage.** Every response that bills carries `usage`. Judging is billed in engine-neutral judgment units: one judgment answered, per started 1,000 tokens of its compiled context and its question together, at least 1 (§9). No response ever carries an engine's price. servers: - url: https://api.vainona.ai/v1 security: - apiKey: [] tags: - name: Namespaces description: Namespaces, their settings and cache warming (§6.2, §6.11). - name: Documents description: Writes, point reads and queries (§6.3 to §6.8). - name: Judgments description: >- Judgment definitions, versions, settings, activation, backfill and template detach (§6.5, §6.9, §7.7). - name: Outcomes and calibration description: | Post what actually happened to judged documents, and read how well answers match it: the calibration report, calibrated answers and threshold recommendations (§6.10). See [calibration](/concepts/calibration). - name: Jobs description: >- Backfill, shadow, periodic, deletion, reference index and fan-out jobs (§6.2, §6.9, §7.10.5). - name: Engines description: The engine registry (§6.5, §7.4.5). - name: Organization description: | Settings of the whole organization: prefixes marked non-production, which the tenant fee and template calibration pools leave out (§9). paths: /namespaces/{ns}/judgments/{name}/labelling-queue: parameters: - $ref: '#/components/parameters/Namespace' - $ref: '#/components/parameters/JudgmentName' post: tags: - Outcomes and calibration summary: Draw documents to label description: | Draws up to `count` documents to label (§6.10), a random sample stratified across the 10 probability bands: `count` is split evenly over the bands with documents to give, and what a band cannot fill goes to the others. Within a band, documents come in an order fixed by `seed`, so the same state and seed give the same items. Documents already labelled from the queue for their current answer, and those leased to someone else, are skipped. Each draw leases its documents for 7 days, so two people are not given the same one. Post each label as an ordinary outcome with the item's `queue_item_id`: it counts as `source: queue`. Once an epoch's queue labels alone meet the calibration minimums, the epoch is fitted on them, each weighted by its band's share of the answers, so the rare bands the queue oversamples count for what they are in real traffic. Drawing is part of the learning loop, on the Team plan and above: below it, `plan_required` (§9). The preview (`GET`) is on every plan. A judgment with a non-zero `horizon` has no queue (`invalid_request`). operationId: drawLabellingQueue requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/LabellingDrawRequest' responses: '200': description: The items drawn and leased; none when nothing is available. headers: X-RateLimit-Limit: $ref: '#/components/headers/RateLimitLimit' X-RateLimit-Remaining: $ref: '#/components/headers/RateLimitRemaining' content: application/json: schema: $ref: '#/components/schemas/LabellingDraw' '400': $ref: '#/components/responses/InvalidRequest' '401': $ref: '#/components/responses/Unauthorized' '402': $ref: '#/components/responses/PlanRequired' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '409': $ref: '#/components/responses/Conflict' '429': $ref: '#/components/responses/RateLimited' '500': $ref: '#/components/responses/Internal' components: parameters: Namespace: name: ns in: path required: true description: The namespace name, with any `/` sent as `%2F`. schema: $ref: '#/components/schemas/NamespaceName' JudgmentName: name: name in: path required: true schema: $ref: '#/components/schemas/Name' schemas: LabellingDrawRequest: type: object additionalProperties: false properties: count: type: integer minimum: 1 maximum: 200 default: 50 description: How many documents to hand out at most. seed: type: integer format: int64 minimum: 0 default: 0 description: >- Fixes each band's random order. The same state and seed give the same items. LabellingDraw: type: object description: Documents drawn from the labelling queue and leased (§6.10). required: - judgment - version - engine - engine_version - seed - lease_expires_at - items properties: judgment: $ref: '#/components/schemas/Name' version: $ref: '#/components/schemas/JudgmentVersionNumber' engine: type: - string - 'null' engine_version: type: - string - 'null' description: The epoch drawn from. Null while the version has no answers. seed: type: integer format: int64 minimum: 0 lease_expires_at: description: >- When the items go back into the queue if not labelled. Null when nothing was drawn. oneOf: - $ref: '#/components/schemas/Timestamp' - type: 'null' items: type: array maxItems: 200 description: Lowest band first; within a band, in the seed's order. items: $ref: '#/components/schemas/LabellingQueueItem' NamespaceName: type: string description: >- Up to 256 bytes. `/` separates levels of the hierarchy, as in `acme/prod/tenant_123`. pattern: ^[A-Za-z0-9._:/-]+$ minLength: 1 maxLength: 256 Name: type: string description: >- A judgment, attribute or threshold name. Names are path segments in field references, so they never contain `.`. pattern: ^[A-Za-z0-9_-]+$ minLength: 1 maxLength: 128 JudgmentVersionNumber: type: integer format: int32 minimum: 1 Timestamp: type: string format: date-time description: RFC 3339, UTC. LabellingQueueItem: type: object required: - queue_item_id - document_id - band - probability - value - evaluation_id - document properties: queue_item_id: type: string description: Post it with the label, on the outcome. document_id: $ref: '#/components/schemas/DocumentId' band: $ref: '#/components/schemas/ProbabilityBand' probability: type: number minimum: 0 maximum: 1 description: >- The answer's raw value, `p` or the most probable option's or level's probability. value: description: Choice and score, the most probable option or level; null for bool. type: - string - integer - 'null' evaluation_id: type: - string - 'null' description: >- The evaluation the answer came from; its context is at `GET /namespaces/{ns}/evaluations/{id}`. document: type: object description: The document to show the labeller. required: - id - revision - attributes - state properties: id: $ref: '#/components/schemas/DocumentId' revision: $ref: '#/components/schemas/Revision' attributes: $ref: '#/components/schemas/Attributes' state: $ref: '#/components/schemas/State' Error: type: object required: - error properties: error: type: object required: - code - message properties: code: $ref: '#/components/schemas/ErrorCode' message: type: string details: type: object description: >- Code-specific detail, such as the scan estimate on a refused query. Every `internal` error has `request_id`. DocumentId: type: string description: Up to 128 bytes. pattern: ^[A-Za-z0-9._:/-]+$ minLength: 1 maxLength: 128 ProbabilityBand: type: object description: >- One of the labelling queue's 10 equal-width bands of raw value; the last includes 1. required: - index - lower - upper properties: index: type: integer minimum: 0 maximum: 9 lower: type: number minimum: 0 maximum: 1 description: Inclusive. upper: type: number minimum: 0 maximum: 1 description: Exclusive, except for the last band. Revision: type: integer format: int64 minimum: 0 Attributes: type: object description: >- Flat, filterable and sortable. At most 64 keys. Not sent to engines unless a context recipe names them. maxProperties: 64 propertyNames: $ref: '#/components/schemas/Name' additionalProperties: $ref: '#/components/schemas/AttributeValue' State: type: object description: >- The JSON content judgments are made about. At most 1 MB. Never filterable. ErrorCode: type: string description: | HTTP status by code: `invalid_request` 400, `unauthorized` 401, `budget_exceeded` 402, `plan_required` 402, `forbidden` 403, `not_found` 404, `conflict` 409, `too_large` 413, `engine_version_unavailable` 422, `insufficient_labels` 422, `rate_limited` 429, `internal` 500, `engine_unavailable` 503. enum: - invalid_request - unauthorized - forbidden - not_found - conflict - too_large - rate_limited - budget_exceeded - plan_required - engine_unavailable - engine_version_unavailable - insufficient_labels - internal AttributeValue: description: A string, number, boolean, string array or null. type: - string - number - boolean - array - 'null' items: type: string headers: RateLimitLimit: description: Requests allowed per second for this key. schema: type: integer minimum: 0 RateLimitRemaining: description: Requests left in the current window. schema: type: integer minimum: 0 RetryAfter: description: Seconds to wait before retrying. schema: type: integer minimum: 0 responses: InvalidRequest: description: '`invalid_request`: the request is malformed or breaks a rule of §6.' content: application/json: schema: $ref: '#/components/schemas/Error' Unauthorized: description: '`unauthorized`: the key is missing, unknown or revoked.' content: application/json: schema: $ref: '#/components/schemas/Error' PlanRequired: description: >- `plan_required`: the organization's plan does not include this (§9). `details.capability` names what it needs (such as `templates`), `details.required_plan` the cheapest plan that includes it, and `details.plan` the plan the organization is on. An owner changes the plan in the dashboard; retrying does not help. content: application/json: schema: $ref: '#/components/schemas/Error' Forbidden: description: '`forbidden`: the key''s role or namespace prefix does not allow this.' content: application/json: schema: $ref: '#/components/schemas/Error' NotFound: description: '`not_found`: the namespace, document, judgment or job does not exist.' content: application/json: schema: $ref: '#/components/schemas/Error' Conflict: description: >- `conflict`: the namespace is being deleted, an object a read needs was compacted away and the read (or the query, without its cursor) must restart (§7.5.11), or the judgment is inherited from a template and this namespace cannot create, activate or delete it (§7.7). content: application/json: schema: $ref: '#/components/schemas/Error' RateLimited: description: >- `rate_limited`: slow down. `Retry-After` says for how long, except when a daily allowance is used up: then `details.resets_at` says when it renews, and there is no `Retry-After`. headers: X-RateLimit-Limit: $ref: '#/components/headers/RateLimitLimit' X-RateLimit-Remaining: $ref: '#/components/headers/RateLimitRemaining' Retry-After: $ref: '#/components/headers/RetryAfter' content: application/json: schema: $ref: '#/components/schemas/Error' Internal: description: >- `internal`: something failed on our side. Nothing partial is returned, and a request never commits part of itself. Most `internal` errors committed nothing. A write can instead fail with a message saying it may have been committed, when we could not tell whether its batch landed (§7.2.10): it then committed all of it or none of it. Retrying a write is safe either way, because writes are idempotent. The message is generic and carries a request id, also in `details.request_id`, which support uses to find the cause. content: application/json: schema: $ref: '#/components/schemas/Error' securitySchemes: apiKey: type: http scheme: bearer description: | An organization API key. Keys carry a role (`read_write` or `read_only`) and may be restricted to a namespace prefix such as `acme/*`, or to one namespace such as `acme/prod/tenant_1`. A prefix matches on a `/` boundary: `acme/prod/tenant_1*` covers `acme/prod/tenant_1` and everything under `acme/prod/tenant_1/`, never `acme/prod/tenant_12`. ````