> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vainona.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Keep your data in sync

> Copy every change from your database with whole-document upserts, never miss one with an outbox, and check the two agree.

export const productName = "Vainona";

Your database stays the system of record. {productName} holds a copy of the records you judge, and you keep that copy current through the [write API](/concepts/documents#writing). There is no automatic sync from your database yet. This guide is the pattern that keeps the copy correct: send whole documents after each change, make sure no change is missed, and check now and then that both sides agree.

For the first load of existing records, see [import existing data](/guides/import-existing-data).

## Send the whole document

After your database commits a change, upsert the record's current version.

```json theme={null}
{
  "upsert": [
    {
      "id": "order_line:5521",
      "attributes": {"kind": "order_line", "product_id": "product:77", "status": "open", "source_version": 1747},
      "state": {"note": "Customer asked to swap the size.", "quantity": 2}
    }
  ]
}
```

* **Upsert whole records.** An upsert replaces the whole document, so the result never depends on what was there before, and sending it twice is harmless. A patch merges fields into the stored document, so the result depends on what it lands on, and nothing sent later repairs a field it did not name.
* **Read the record when you send it,** not when it changed. If two changes to the same record are sent out of order, the later write wins, so a write built from an old event can put stale data back. Sending the record as it is now avoids that.
* **Write after your own commit,** never before it. Otherwise a change your database rolled back can reach {productName}.
* **Record your own version** in an attribute, such as `source_version` (a version number or your `updated_at`). {productName}'s `revision` and `updated_at` say when it received a write; `source_version` tells you which of your versions it holds, which is what the [check](#check-that-both-sides-agree) compares.
* **Delete with `delete`.** When you delete a record, send its id in `{"delete": ["order_line:5521"]}`.
* **Batch when you can.** One write takes up to 1,000 documents, so a sync that sends changes every few seconds costs far fewer requests than one per change.

Resending a document that has not changed costs only its write bytes. A judgment whose compiled context did not change keeps its answer and is not billed.

## Never miss a change

Writing to {productName} right after your commit is simplest, but a crash or a network error between the two loses that change. If every change must arrive, use an outbox:

1. In the same transaction as the change, insert a row into an outbox table: the record's id and kind.
2. A worker reads unsent outbox rows in order, loads each record as it is now, and upserts them in batches (or deletes the ones that no longer exist).
3. When the write succeeds, it marks those rows sent. On an error it retries: every write is idempotent, so a retry is always safe.

If several rows name the same record, send it once. Keep one worker per namespace, or split records between workers by id, so the same document is never in two writes at once.

## Check that both sides agree

Even a careful sync can drift: a bug, a restore from backup, a record changed by hand. Run a check on a schedule, nightly for most teams.

1. **Find the records changed recently** in your database, say in the last two days, with their versions.

2. **Read the same ids** from {productName} with a [query](/concepts/answers#querying-answers) that returns only `source_version`:

   ```json theme={null}
   {
     "filters": ["id", "In", ["order_line:5521", "order_line:5522"]],
     "include": {"attributes": ["source_version"], "state": false},
     "top_k": 1000
   }
   ```

3. **Upsert every record** that is missing or has an older `source_version`.

4. **Find deletions** less often, weekly say: page through the namespace's ids with `rank_by: ["id", "asc"]`, a `Glob` filter on the id prefix to keep each scan small, and `next_cursor`, and delete any id your database no longer has.

Queries are billed by the bytes they scan, and one that includes no `state` scans little.

## When to use patch or append

`patch` and `append` still have their place when you do not hold the whole record, such as an event stream that only knows the new message. [Append](/concepts/documents#writing) skips an element identical to one of the array's last 64, so it is safe to retry. Concurrent patches and appends to one document are all applied, in the order they commit (see [system behavior](/behavior#your-data)). What they cannot do is correct a field that an out-of-order write left stale; a whole record sent from your system of record, or the scheduled check, does that.