Skip to main content
Your database stays the system of record. Vainona holds a copy of the records you judge, and you keep that copy current through the write API. There is no automatic sync from your database yet. This guide is the pattern that keeps the copy correct: send whole documents after each change, make sure no change is missed, and check now and then that both sides agree. For the first load of existing records, see import existing data.

​
Send the whole document

After your database commits a change, upsert the record’s current version.
  • Upsert, don’t patch. An upsert replaces the whole document, so the result never depends on what was there before, and sending it twice is harmless. A patch merges fields into the stored document, and in rare cases one of two patches to the same document at the same moment can be lost (see system behavior). An upsert cannot be.
  • Read the record when you send it, not when it changed. If two changes to the same record are sent out of order, the later write wins, so a write built from an old event can put stale data back. Sending the record as it is now avoids that.
  • Write after your own commit, never before it. Otherwise a change your database rolled back can reach Vainona.
  • Record your own version in an attribute, such as source_version (a version number or your updated_at). Vainona’s revision and updated_at say when it received a write; source_version tells you which of your versions it holds, which is what the check compares.
  • Delete with delete. When you delete a record, send its id in {"delete": ["order_line:5521"]}.
  • Batch when you can. One write takes up to 1,000 documents, so a sync that sends changes every few seconds costs far fewer requests than one per change.
Resending a document that has not changed costs only its write bytes. A judgment whose compiled context did not change keeps its answer and is not billed.

​
Never miss a change

Writing to Vainona right after your commit is simplest, but a crash or a network error between the two loses that change. If every change must arrive, use an outbox:
  1. In the same transaction as the change, insert a row into an outbox table: the record’s id and kind.
  2. A worker reads unsent outbox rows in order, loads each record as it is now, and upserts them in batches (or deletes the ones that no longer exist).
  3. When the write succeeds, it marks those rows sent. On an error it retries: every write is idempotent, so a retry is always safe.
If several rows name the same record, send it once. Keep one worker per namespace, or split records between workers by id, so the same document is never in two writes at once.

​
Check that both sides agree

Even a careful sync can drift: a bug, a restore from backup, a record changed by hand. Run a check on a schedule, nightly for most teams.
  1. Find the records changed recently in your database, say in the last two days, with their versions.
  2. Read the same ids from Vainona with a query that returns only source_version:
  3. Upsert every record that is missing or has an older source_version.
  4. Find deletions less often, weekly say: page through the namespace’s ids with rank_by: ["id", "asc"], a Glob filter on the id prefix to keep each scan small, and next_cursor, and delete any id your database no longer has.
Queries are billed by the bytes they scan, and one that includes no state scans little.

​
When to use patch or append

patch and append still have their place when you do not hold the whole record, such as an event stream that only knows the new message. Append skips an element identical to one of the array’s last 64, so it is safe to retry. Both carry the rare loss described in system behavior, and a lost change stays lost until the next upsert, so if several writers patch or append to the same document, send the whole record from one place instead, or let the scheduled check repair it.