Export audit and evaluation history

Every evaluation Vainona makes is kept, with the compiled context the engine read. On Scale you can export that history for your compliance, audit or data teams: every evaluation in a time range, with its context, its answer, the engine version it was made in and the outcomes joined to it, as files you download and keep. The dashboard exports the audit log as CSV. Exports are included in Scale and aren’t billed: no judgment units, no bytes scanned, and the files aren’t stored bytes. Below Scale, starting an export returns plan_required (HTTP 402).

Export evaluations

An export is a job. Start it for a namespace, then poll the job until it is done:

const ns = db.namespace("acme/prod");
const { job_id } = await ns.exportEvaluations({
  since: "2026-07-01T00:00:00Z",
  until: "2026-10-01T00:00:00Z",
  judgments: ["needs_escalation"],
});

let job = await db.jobs.get(job_id);
while (job.status === "running") {
  await new Promise((resolve) => setTimeout(resolve, 5_000));
  job = await db.jobs.get(job_id);
}
if (job.export?.links_unavailable) throw new Error(job.export.links_unavailable); // try again later
for (const file of job.export?.files ?? []) {
  if (!file.url) continue;
  const response = await fetch(file.url); // no API key: the link is signed
  // ...save response.body, a gzipped JSON Lines file of file.evaluations lines
}

import time

import httpx

ns = db.namespace("acme/prod")
job_id = ns.export_evaluations(
    since="2026-07-01T00:00:00Z",
    until="2026-10-01T00:00:00Z",
    judgments=["needs_escalation"],
)["job_id"]

job = db.jobs.get(job_id)
while job["status"] == "running":
    time.sleep(5)
    job = db.jobs.get(job_id)
if job["export"]["links_unavailable"]:
    raise RuntimeError(job["export"]["links_unavailable"])  # try again later
for file in job["export"]["files"]:
    if file["url"] is None:
        continue
    with httpx.stream("GET", file["url"]) as response:  # no API key: the link is signed
        ...  # save the gzipped JSON Lines file of file["evaluations"] lines

curl -X POST "https://api.vainona.ai/v1/namespaces/acme%2Fprod/evaluations/exports" \
  -H "Authorization: Bearer $VAINONA_API_KEY" -H 'content-type: application/json' \
  -d '{"since": "2026-07-01T00:00:00Z", "until": "2026-10-01T00:00:00Z"}'
# {"job_id": "job_01j9c4t6w8y0a2c4e6g8j0m2p4"}
curl "https://api.vainona.ai/v1/jobs/job_01j9c4t6w8y0a2c4e6g8j0m2p4" \
  -H "Authorization: Bearer $VAINONA_API_KEY"

POST /namespaces/{ns}/evaluations/exports takes:

since, the earliest created_at exported, inclusive. It defaults to 30 days before until.
until, the end of the range, exclusive. It defaults to now, and a later time is taken as now.
judgments, to export only these. Leave it out for every judgment the namespace has evaluations of, deleted judgments included.

{} exports the last 30 days of every judgment. One export covers at most 366 days: export a longer history in several. Starting one needs a read_write key whose scope covers the namespace, and is in your audit log as job.create. The job is GET /jobs/{id} with type: "evaluation_export". While it runs, progress.documents_done and export.evaluations count the evaluations written so far. You can pause, resume and cancel it like any job. When it is done, export.files lists its files:

{
  "id": "job_01j9c4t6w8y0a2c4e6g8j0m2p4",
  "type": "evaluation_export",
  "status": "done",
  "namespace": "acme/prod",
  "export": {
    "since": "2026-07-01T00:00:00Z",
    "until": "2026-10-01T00:00:00Z",
    "judgments": ["needs_escalation"],
    "evaluations": 214380,
    "files": [
      {"url": "https://storage.googleapis.com/...", "url_expires_at": "2026-10-01T10:15:00Z", "evaluations": 100000, "bytes": 41873920},
      {"url": "https://storage.googleapis.com/...", "url_expires_at": "2026-10-01T10:15:00Z", "evaluations": 100000, "bytes": 41602311},
      {"url": "https://storage.googleapis.com/...", "url_expires_at": "2026-10-01T10:15:00Z", "evaluations": 14380, "bytes": 6011458}
    ],
    "expires_at": "2026-10-08T09:02:41Z",
    "links_unavailable": null
  }
}

Each link works for an hour and needs no API key, so hand it to whatever downloads the file. Read the job again for new links.
If links can’t be made right now, the job still reads as done with its files listed, but their url and url_expires_at are null, and links_unavailable says so, with a request id to quote to support. The files are unaffected: read the job again later.
Files are kept 7 days after they are written. expires_at is when the first one is deleted; after it the job lists no files, and you start a new export.
Files hold up to 100,000 evaluations, and an export writes at most 1,000 files. One that needs more fails with an error that says so: narrow the range, or export fewer judgments at a time.
An export runs in the background and resumes where it stopped if it is interrupted, so a large one can take a while. Deleting the namespace deletes its exports too.

What a file holds

Each file is gzipped JSON Lines: one evaluation per line, as GET /namespaces/{ns}/evaluations/{id} returns it, plus the outcomes joined to it. Most tools read it as it is: BigQuery, Snowflake and Athena load .jsonl.gz directly, and gunzip -c 00001.jsonl.gz | jq . shows it.

{"id": "ev_01j8zq3k5v9w2x4y6z8a0b1c2d", "document_id": "t_123", "revision": 43, "incarnation": 7,
 "judgment": "needs_escalation", "judgment_version": 3, "engine": "jev", "engine_version": "current+2026-09-24.1",
 "context_hash": "sha256:9f86d0...", "context_tokens": 1830, "context_truncated": false,
 "output": {"p": 0.91}, "status": "success", "error": null, "shadow": false, "replay_of": null,
 "created_at": "2026-09-23T12:00:02Z", "latency_ms": 412,
 "context": {"state": {"subject": "Charged twice", "body": "I was charged twice for my order..."}},
 "raw": {"p": 0.91},
 "outcomes": [{"value": true, "observed_at": "2026-09-23T15:40:00Z", "source": "posted"}]}

context is the exact compiled context the engine read, and related_documents, for a judgment with relations, what it read of other documents. raw is the engine’s response.
output is the answer’s raw numbers: p, value, dist, escape_p, score or parts. Thresholds and calibration are applied when answers are read, so the export holds what the engine said.
engine_version is the epoch the evaluation was made in: calibration fits each epoch on its own (see when the engine changes).
outcomes are the outcomes calibration joins to this evaluation: posted outcomes, those your outcome rules derived (source: "rule") and labelling-queue labels ("queue"). Implicit negatives are calibration’s inference rather than outcomes, so they aren’t listed. It is empty when none joins.
Every evaluation is included: failed ones with their error, shadow evaluations from shadow reports marked shadow: true, and replays after an engine change with replay_of set and context: null (the context is the evaluation it names). Evaluations of documents deleted since are included too.
Files aren’t in any order. Sort by created_at if you need to. Rarely, an evaluation appears twice, when the history was reorganized while the export ran: id is unique, so keep one line per id.

Export the audit log

The dashboard’s Audit page lists your organization’s audit events on every plan. On Scale, Export CSV downloads the events that match its filters: each event’s time, actor, action, target, and the values before and after.

After a downgrade

Below Scale, new exports are refused with plan_required, and the audit log’s CSV export goes. An export’s files stay downloadable until they are deleted, and the audit log stays in the dashboard. A downgrade takes effect on the 1st of a month at least 30 days after you ask (changing plan).

Get started

Before you build

Concepts

Guides

Engines

Reference

Export audit and evaluation history

Export evaluations

What a file holds

Export the audit log

After a downgrade

​Export evaluations

​What a file holds

​Export the audit log

​After a downgrade

Export evaluations

What a file holds

Export the audit log

After a downgrade