Capture first, shape later

An event you didn’t store can’t be analyzed later. Shape decisions change, but a lost event stays lost. Event recording stores everything it is allowed to store. Queries select and interpret the stored fields when they run.

Cloudflare Basin makes this concrete. Events that do not match a structured stream’s schema “are accepted during ingestion but will be dropped during processing,” the docs say. They also say “schema modifications are not supported after stream creation.”

An unstructured stream instead accepts “any valid JSON without validation” into “a single value column.” Manage streams

Snowplow reached the same rule years earlier. Its pipelines are “non-lossy,” it says. When validation or enrichment fails, payloads are “stored into a bad rows storage solution … instead of being discarded.” snowplow-event-recovery

In rat-stack, the packages/events cartridge36 owns event recording and its resources. It sends one JSON event record per request to an unstructured Basin stream. A pipeline copies it unchanged (SELECT *) into an Iceberg table, and queries select and interpret its fields later.

Size and rate limits still apply. The stream also requires the { "value": <event> } row wrapper; omitting it drops rows during processing. Analytics18 documents that boundary.

Capture first keeps only permitted request facts. The event record holds method, path, status, and referrer origin and path. Its query string has credential, email, code, and session keys removed.

The request body is never read. The IP is stored only as a salted hash, computed with an additional secret value.

Visitor identity is a setting:

ModeIdentity
dailyHash of IP, user agent, and day; no cookie.
persistentOne first-party id per browser.

The server sets the persistent id. WebKit “deletes all cookies created in JavaScript and all other script-writeable storage after 7 days of no user interaction.” Tracking Prevention in WebKit

Segment’s identify guidance joins people later by a stable database id. It advises against using an email address. It gives the reason: “because database IDs never change.” Spec: Identify

Property tests18 check these rules. See property-based testing18 and model-based testing14.

Sources

  1. Manage streams. Cloudflare docs. Stream validation rules; used for capture-first unstructured ingestion. Accessed 2026-10-01.
  2. GitHub - snowplow-incubator/snowplow-event-recovery · GitHub. GitHub snowplow-incubator/snowplow-event-recovery. Event recovery design; used for retaining failed payloads rather than discarding them. Accessed 2026-10-01.
  3. Spec: Identify. Segment docs. Identify guidance; used for joining people with stable database ids. Accessed 2026-10-01.
  4. Tracking Prevention in WebKit. WebKit. Storage expiry rules; used for the server-set persistent visitor cookie. Accessed 2026-10-01.
  5. rat-stack/packages/events at main · joelhooks/rat-stack · GitHub. GitHub joelhooks/rat-stack. Events implementation; used for the envelope, redaction, and identity modes. Accessed 2026-10-01.

Last changed 2026-10-07 in e73c9d9. Source on GitHub. Change log.

Linked from

  1. Agent guide. An Effect stack so pure (aspirational) Kit Langton will blush.

    Capture first, shape later — Store events without rejecting them for an evolving event shape, then decide their shape when you query.

  2. Analytics. One event per HTTP request, stored unchanged for queries to select and interpret later.

    It follows capture first, shape later.

  3. Change log. What changed in the files served here, newest first.

    Capture first, shape later — edited: feat(learn): teach the next twenty cards with plain lines, diagrams and snippets

  4. Full agent guide. An Effect stack so pure (aspirational) Kit Langton will blush.
  5. Property-based testing. State a rule that must hold for every input, then let a generator hunt for the smallest input that breaks it.

    The capture-first events cartridge is tested this way.

  6. Rat Stack lore | rat-stack. Short, source-grounded notes on the ideas and decisions behind rat-stack.

    Capture first, shape later — Store events without rejecting them for an evolving event shape, then decide their shape when you query.

  7. Source change history (log.md). The generated change log as Markdown.

    Capture first, shape later — edited: feat(learn): teach the next twenty cards with plain lines, diagrams and snippets