frustrated or legal_threat, a judgment on the account can read those answers the way it reads any field of a related document, beside the account’s own facts. The account’s answer stays current as the conversations’ answers change, and each evaluation says which answer of which conversation it read.
Without it, you copy each conversation’s answer into the account and rewrite the account whenever one changes: a pipeline. With it, Vainona re-judges the account only when a conversation’s answer moves in a way the account can see.
Roll-ups are about keeping that current without a pipeline, not about accuracy. In our test, a judgment reading a roll-up of child answers did no better than one reading the children’s text, and neither beat plain counts (what we measured).
This guide builds on judging a document with its related documents. Read that first: roll-ups add answers to what a relation can show, and everything else about relations stays the same.
Define one
A relation reads a judgment’s current answer atanswers.<judgment>.<field>, the same paths a query filters on: .p, .value, .score, .dist.<option> and .thresholds.<name>.
frustrated answer as a word, how many of them are both frustrated and a legal threat, and whether any of them is a legal threat at all:
count_where.
What the context shows is not necessarily what the judgment weighs. In our test the engine’s answers barely moved with the counts rendered into its context. When a number should decide the answer, such as how many conversations were frustrated, give it to a composite judgment, which fits a relation’s aggregates as features against your outcomes, rather than relying on the engine to read it.
Every rendered answer is stable
A probability moves a little every time a conversation is judged again. If the account saw the raw number, it would be judged again every time too, and nothing would ever be a dedup hit. So what a relation shows of an answer only changes when the answer crosses a line you chose:sum over an answer is refused because a sum changes on every evaluation, and even a banded sum can cross a band when no single conversation’s answer did. A named threshold means what you decided it means from your outcomes; an inline cut is for when you have not set one, and is just as stable.
A banded max says whether any of the documents reaches a band, so over many documents it almost always does. In our test it read the top band for 99% of users once a relation held 50 or more documents, and told the judgment nothing. Keep max and min to a short relation, such as the last ten, and use count_where for many.
What a judgment can read
The judgment whose answers a relation reads must be:- plain: its own context reads no related documents and it is not a composite, so one hop stays one hop and nothing reads an answer that moves without being judged;
- active and
on_change, so its answers stay current as they are read; - in the same namespace or template;
- answering every document the relation selects: its
applies_tocovers the relation’smatch.
invalid_request, and details.path names the path.
Once another judgment reads it, the judgment has readers, and its GET lists them. Deleting or deactivating it, switching it to a policy other than on_change, or activating a version that is not plain or whose applies_to no longer covers a reader’s match is refused with conflict, naming the readers in details.readers.
When the account is judged again
A conversation’s new answer re-judges the account only when it changes what one of the account’s relations shows of that conversation: its band, whether it matches a cut or threshold incount_where or match, the band it gives a min or max, or its latest value. A conversation judged a thousand times with p moving inside a band costs the account nothing.
From there it is an ordinary change to a related document: the account’s debounce_ms and its ceiling apply, so a burst of conversation answers costs one evaluation of the account.
A conversation deleted and written again with the same id is a new conversation: its first answer re-judges the account even when the number is the same as before.
Selecting by answer. A match on an answer, such as only the frustrated conversations, keeps the documents whose answer holds before last_n keeps the newest. Membership changes only when a conversation crosses the cut. To find them, the relation reads at most 4 × last_n of the newest documents that match on attributes; if it reads that many before last_n qualify, the relation shows capped: true. Ordering by an answer is not available.
The document it points at. A relation that reads its referenced document can read that document’s answers the same way. A change to what it shows is a referenced change, and fans out to the documents that point at it under the same limits.
Pending conversations
The account reads each conversation’s newest answer. When a conversation has been written and not judged again yet, the account reads its last answer, and the account’s evaluation sayschild_answers_stale: true. The conversation’s new answer re-judges the account anyway if it changes what the account shows, so this is a flag to wait out, not to act on.
The audit
An account’s evaluation lists every conversation it read inrelated_documents, and for each one whose answers it read, evaluations names the evaluation behind each answer: {"frustrated": "ev_…", "legal_threat": "ev_…"}. Evaluations are kept forever, so you can always follow an account’s answer back to the conversation answers it read. answers_generation says which answers the context read, beside watermark for the documents.
What it costs
A conversation’s answer moving is a cost to every judgment that reads it. So the three requests that move conversation answers, or change what an account shows of them, say what they cost the readers before you confirm, asdownstream, one line per reader:
- a backfill estimate of
frustrated: the accounts it re-judges, in the month it runs; - the shadow report of a new
frustratedversion: the accounts its answers would re-judge, from the sample; - a threshold change on
frustratedthat a reader’s relation names (below).
frustrated for its own sake, reading it adds nothing on the conversation side: its answers exist anyway, and the account’s re-judgments are ones it would have had from the conversations’ writes. So read judgments you use yourself. Running a judgment only to feed another means paying for every child evaluation too, and in our test that bought no accuracy.
Changing a threshold a reader reads
A threshold is a setting: changing one applies to every answer at once, with no evaluation. When a reader’s relation names that threshold, such asanswers.legal_threat.thresholds.any above, the change moves what the reader shows with no answer to re-judge it by. So the PATCH returns the downstream estimate and changes nothing, the rest of the request included:
confirm: true the thresholds apply to every read at once, as ever, and a resync job starts. It goes through legal_threat’s answers once and re-judges every account whose conversation now reads differently. Until the job reaches an account, that account’s answer stays fresh under the old threshold. The response has job_id and the same downstream lines: if you are applying a threshold recommendation, this is often where you learn that another judgment reads it.
A threshold no reader names changes at once, as it always has. An inline cut lives in the reader’s own definition, so changing it is a new version of the reader, with its own backfill estimate.
Two levels, not three
“Message, conversation, account” is not a chain of roll-ups: a judgment can read a plain judgment’s answers, and a judgment that reads related documents is not plain. It works when the leaf carries both keys, which most schemas do. A message withattributes.conversation_id and attributes.account_id can be read by a judgment on its conversation and by a judgment on its account, each one hop:
So judge each message (angry), read answers.angry from the conversation, and read answers.angry again from the account, not the conversation’s answer. If your messages carry only their conversation’s id, write the account’s id on them too.
What we measured
We tested roll-ups on a public Q&A dataset: will a user who posted at least 50 times in their first six months still be active six months later? That is the case where a user’s posts no longer fit in one context, so a roll-up could have helped. It did not:
The roll-up scored at chance, below the text, and neither added anything to the counts. The numbers it showed did carry some signal, but the engine’s answers barely moved with them, and each
max read the top band for almost every user. So this guide makes no claim that roll-ups are more accurate than text or than counts. Use them to judge an account from what its conversations were judged to be, kept current without a pipeline, and put the account’s own facts, such as counts, beside them. Measure your own judgment against your outcomes (see measure, improve and tune).