How it works

1. Publish a record

A record has a kind and a JSON body. The kind identifies the record's role, such as document or summary. The body contains application data.

{ "kind": "document",
  "body": { "type": "pdf",
            "classification": "public",
            "url": "https://example.com/filing.pdf" } }

The record has no destination, queue name or worker identifier. Eligible workers find it by matching its kind and body.

The runtime assigns authoritative metadata, including creation time, creator identity and a body hash. Client-supplied metadata remains separate and cannot override those fields.

2. Match an eligible worker

A summarizer publishes the pattern of records it handles. The same pattern language is used for queries and claims:

{ kind: "document", match: { type: "pdf" } }

Patterns support equality, ranges, membership, existence tests, explicit array predicates and boolean combinations. They do not contain executable expressions or regular expressions.

Keeping patterns as data allows the runtime to validate, store, authorize and inspect them. It also avoids executing participant-supplied predicates inside the runtime.

Workers publish their active interests as records. A dry run can therefore report which live workers would receive a proposed record before the record is written.

Each kind declares the body paths that patterns may use. A pattern that names another path is rejected. These declarations define the fields available to routing and content-scoped grants; undeclared fields can hold data that the runtime never examines. For example, the analysis application indexes dataset and digest fields for content-keyed work, while the chat leaves message prose undeclared so the application can encrypt it.

3. Claim the record under a lease

Claims are competitive. If four summarizers request the same document, one receives it and the others receive no claim.

The claim includes a lease with a deadline and fencing epoch. The worker renews the lease while it is active.

4. Recover from a worker failure

A worker may exit without releasing its claim because of a deployment, process failure or unavailable host.

When the lease expires, the record becomes claimable again and another worker can take it. Work performed before the failure may be repeated.

A slow worker can also lose its lease. Another worker may then claim the same record while the first is still running. The fencing epoch ensures that only the current lease holder can settle the record.

Delivery is at least once. Workers that perform external effects, such as sending email, must make those effects idempotent or deduplicate them at the external boundary.

5. Settle the claim with a result

A successful acknowledgement consumes the document and can create a summary record in the same transaction. The summary names the document as a parent.

The summary is available to queries, watches and downstream claims. A new downstream worker can begin processing summaries without changes to the summarizer.

After the configured maximum number of attempts, a repeatedly failing record is dead-lettered. Operators can inspect and requeue it after correcting the cause.

6. Inspect the resulting history

Each mutation appends an event in the same database transaction. The event log therefore agrees with committed record and lease state.

Watches use the log to wake workers. Parent links provide lineage from a result back to its inputs, while event queries show how each record moved through its runtime states.

Finalized events can be sealed into a hash chain. Each seal covers the previous link, the event and the referenced record's content hash. Later database edits break verification.

Store large payloads as artifacts

PDFs, images, audio and other large byte payloads are stored as artifacts. An artifact record carries the content digest, media type and size; the bytes live in the configured blob store.

Keeping bytes outside record bodies bounds matching and query costs. It also allows the bytes to be shredded while the artifact record, digest, lineage and event history remain.

The digest remains after erasure. This is suitable for high-entropy documents but does not hide a short, guessable secret from someone who can test candidate hashes.

Content digests can also key derived work. The analysis example uses input and code digests to find completed stages and recompute only affected descendants. Applications can store encryption keys in shreddable artifacts to erase encrypted body fields while retaining record history.

Choose a storage adapter

Runtime instances own coordination and authorization. Clients and workers use the HTTP API; they do not access storage directly.

The storage contract suite runs against all three adapters. CI also exercises PostgreSQL-only contention cases that embedded adapters cannot represent.