Analytics
What it does
The packages/events cartridge36 is a package for one job, with its implementation and required infrastructure.
It records one JSON event for each HTTP request when capture is enabled.¶
It follows capture first, shape later7. Each event is stored unchanged. Queries select and interpret its fields later.¶
A stream receives events. A pipeline copies them to a sink, the destination that writes the stored files.¶
withEventCapturewraps the Worker’s fetch handler.¶- It sends the event in the background after building the response.¶
- A Worker binding23, a declared connection to a resource, sends the event to an unstructured Cloudflare Pipelines stream.¶
- The pipeline runs
INSERT INTO <sink> SELECT * FROM <stream>.¶ - A Basin Catalog sink writes Parquet files to the Iceberg table
default.events_rawin R2.¶
R2 is Cloudflare’s object storage. Parquet stores data by column in files. Iceberg organizes those files as a table. Basin Catalog makes the table queryable with SQL.¶
Sending events through the public HTTP endpoint is disabled. The sink writes files every 300 seconds by default. See Basin Catalog sink.¶
Each event holds:¶
- A visitor id and UUIDv7 message id. UUIDv7 is a time-ordered identifier format.¶
- A server block: host, received time, salted IP hash, and user agent. The hash combines the IP with a secret salt to produce a one-way value.¶
- Coarse geography from Cloudflare’s
request.cf: continent, country, EU membership, region and region code, city, and time zone. It also includes the Cloudflare data center, the network’s autonomous system number, and its organization. The autonomous system number identifies the network.¶ - A request block: method, path, status, duration, accept type, and content type.¶
- Whether the request carries a signed agent header.¶
- The referrer’s origin and path.¶
- The query string, with credential, email, code, and session keys removed.¶
The standard
- Each HTTP response gets exactly one event when capture is enabled.¶
- Capture preserves the response body and status. Persistent HTML responses may add a visitor cookie.¶
- The HTTP request body never reaches an event.¶
- Credential, email, code, and session query keys are removed.¶
- Basin stores an IP address only as a salted hash.¶
- Geography comes from
request.cf, which Cloudflare derives in its data centers. The capture code never derives it from the IP.¶ - Latitude, longitude, postal code, and metro code are never stored. The Mischief Worker sets
captureCityto store city.¶ - A missing or malformed
request.cffield is left out; it never fails the request.¶ - In
persistentmode, the server sets a first-party HttpOnly cookie, which browser scripts cannot read. A browser keeps one id across visits.¶ - In
dailymode, no cookie is set. An id never repeats on a later day.¶ - Each row is
{ "value": <event> }. The unstructured stream requires thevaluecolumn.¶ - Ingest accepts events without
value, but the pipeline drops them asmissing_field.¶ - The sink token is read only during deployment. It is never bound into the Worker.¶
Worker logs
The three HTTP Worker declarations share privateObservability in apps/site/src/observability.ts:¶
The policy disables per-request invocation logs and traces. Application console and Effect error logs remain enabled. Effect is the library that describes work with typed failures36 and required services.
A service is a named interface for one job.¶
500s log their cause with an incident id.¶
Cloudflare invocation logs contain request, response, and related metadata. They are separate from custom application messages. Alchemy cannot set redactQueryString, so invocation logs would store confirm tokens and device codes from query strings.
Mischief therefore keeps invocation logs off and writes one HTTP request log per request with method, path without query, status, and milliseconds. joinInterest logs one outcome label (accepted, busy, saved_mail_delayed, not_open, or failed) and nothing else.
See Workers Logs.¶
Client IPs reach Basin only as salted hashes. The Worker policy prevents automatic request-header dumps in stored invocation logs. Application logs must not print raw IPs or request headers.¶
These settings apply after deployment. They do not erase stored logs from earlier deployments.¶
Crash archive
Mischief declares MischiefCrashTail as an Alchemy tailConsumers entry. registerCrashTail registers a native tail listener through the host Worker’s listen method.
Returning a tail method from the Effect Worker shape does not register this listener.¶
The tail Worker receives execution traces after Mischief finishes an invocation. It records failed outcomes and traces with exceptions, including exceptions after a successful response.¶
Successful traces without exceptions produce no object. Initialization exceptions keep their event time and script version even when the trace has no fetch event.¶
The events cartridge36 owns the CrashArchive port and its R2 adapter.
A private MischiefCrashBucket holds one immutable JSON object per failed trace.
Keys use crashes/<UTC-day>/<random-id>.json.
Separate objects prevent concurrent deliveries from overwriting a daily file.¶
There is no daily aggregate or second store.¶
Each record contains only:¶
- The outcome.¶
- Exception names, sanitized messages, and stack locations.¶
- Console levels and sanitized text from traces with exceptions or an exception outcome.¶
- The script version id, when Cloudflare supplies it.¶
- The event time, or
nullwhen absent.¶ - The URL path without its query string, or
nullwithout a fetch event.¶
Exception messages keep ordinary diagnostic text, including quoted property names.
Redaction removes credential-shaped values, emails, IP addresses, and URL query strings.
Credential rules cover bearer tokens, key assignments, known token prefixes, JWTs, private keys, and long opaque values.
Custom exception names become [redacted name].¶
Credential rules apply even when Cloudflare removes sensitive header values before delivery. They cannot identify every secret written as ordinary prose.¶
Stack locations retain a source filename, line, and column. They exclude function names, directory paths, URLs, and the first error line.¶
Known credential-header, cookie, and query values are removed from exception and console text at token boundaries. Ordinary header values do not participate in redaction. Paths use credential-shape rules instead of substitutions from unrelated headers or query values.¶
Console records keep the level and redacted string arguments for exception outcomes or traces with exceptions.
Structured arguments and serialized request payloads are omitted.
Missing levels stay null; older records can have no console field.¶
The archive excludes request headers, bodies, and diagnostic-channel payloads. Redaction removes recognized private values from diagnostic text.¶
The tail Worker has no public HTTP route and disables its own observability.
Mischief keeps invocation logs and traces disabled.
Crash capture operates independently of EVENTS_ENABLED.¶
R2 permits direct object reads without a catalog, pipeline, or sink token. The archive supplies best-effort evidence. Missing records do not prove health. Cloudflare delivery limits, missing stack data, and archive failures can leave gaps.¶
Null fields stay unknown; they do not acquire guessed values. Replayed deliveries can create duplicate records with different object keys. The bucket retains its objects when removed from the Stack.¶
Read crashes without a dashboard
Use an R2 read-only S3 profile for the archive.
Get the physical bucket name from the production plan for MischiefCrashBucket.
Use a UTC-day prefix to bound the listing.
The following commands need AWS CLI and an authenticated R2 S3 profile.¶
aws --profile <r2-read-profile> \
--endpoint-url https://<account-id>.r2.cloudflarestorage.com \
s3api list-objects-v2 --bucket <crash-bucket> \
--prefix crashes/2026-10-08/ --max-keys 20 --no-paginate Read Contents[].Key from the result.
If IsTruncated is true, pass NextContinuationToken with --continuation-token for the next page.
Fetch one exact key; do not sync the entire archive.¶
aws --profile <r2-read-profile> \
--endpoint-url https://<account-id>.r2.cloudflarestorage.com \
s3api get-object --bucket <crash-bucket> \
--key <listed-key> crash.json Compare eventTime, path, and scriptVersion with the failed deployment probe.
Read stack locations against that script version’s source.
A Cloudflare ray id is not stored, so this comparison is a correlation rather than an exact join.
Fix the observed cause in a separate change.¶
Before deployment, run pnpm --filter @rat-stack/events exec vitest run test/crash.test.ts.
Run pnpm --filter @rat-stack/site exec vitest run test/crash-tail.test.ts to exercise Alchemy’s exported tail dispatcher.
After deployment, verify the tail consumer attachment and read an actual failed trace from R2.
Local tests prove projection and archive writes; they do not prove Cloudflare delivery.¶
Deleting stored events
A delete from events_raw writes a new Iceberg snapshot. Older snapshots keep their Parquet files, and those files still hold the deleted rows.¶
Snapshot expiry and compaction remove the old files. Cloudflare runs this maintenance on its own schedule. In one test run, no snapshot expired within 38 minutes.¶
The cartridge does not prove that a delete physically erases a row by a known time. Consider this before you enable capture for visitors who can request erasure, for example under GDPR in the EU or UK:¶
persistentmode sets a cookie that lasts 400 days. Some jurisdictions require consent for it.¶dailymode sets no cookie, and its visitor ids do not repeat on a later day.¶- The IP hash salt does not rotate. The same IP gives the same hash in both modes.¶
EVENTS_ENABLED=falserecords nothing.¶
Switches
| Setting | Default | What it does |
|---|---|---|
EVENTS_ENABLED | false | Off retains the bucket, stream, and salt, and records nothing. On adds the catalog, sink, and pipeline. |
EVENTS_SINK_TOKEN | unset | Requires R2 Admin Read & Write when capture is on. Export it in the deploy shell. |
EVENTS_IDENTITY_MODE | persistent in the site schema | persistent or daily visitor ids. The Worker falls back to daily when absent. |
The sink token is read from the process environment. A value in .env or an --env-file is not seen.
The deploy credential cannot create API tokens, so the sink token is supplied separately.
See Basin Catalog sink.¶
How to check
- Check an HTML response in
persistentmode without arat_vidcookie. The response must set one.¶ - Check
pipelinesUserErrorsAdaptiveGroupsfor zero pipeline errors.¶ - Check
pipelinesSinkAdaptiveGroupsfor records and files written. See Metrics and analytics.¶ - Find the queryable catalog with
wrangler basin catalog get <bucket>.¶ - Query it with
wrangler basin sql query <warehouse> "<sql>". The command calls the catalog a warehouse. Wrangler readsWRANGLER_BASIN_SQL_AUTH_TOKEN. See Query data.¶ - Run
pnpm --filter @rat-stack/site exec vitest run test/worker-observability.test.ts.¶ - Review the deployed Worker settings before claiming the invocation-log policy is active.¶
A cookie proves that the persistent capture wrapper is enabled. It does not prove that an event is stored in the table.¶
SELECT json_get_str(value, 'body', 'path') AS path, count(*) AS requests
FROM default.events_raw
GROUP BY path
ORDER BY requests DESC
LIMIT 20; The query reads each path from JSON in the value column.¶
Queries use JSON functions such as json_get_str.
See Scalar functions.¶
Tests
The cartridge uses property-based testing18 and model-based testing14.¶
Properties cover:¶
- One event per request.¶
- Preserved response body and status.¶
- Persistent cookies.¶
- No HTTP request body in events.¶
- No sensitive query keys.¶
The visitor model follows ids across devices, cookies, and days.
The observability test checks every application Worker declaration against a named privacy policy.
HTTP Workers use privateObservability; the tail Worker uses the stricter silentObservability.¶
Removing it
The README’s Keep or cut table lists every file and setting under “No analytics.”¶
Sources
- Tail Workers. Cloudflare docs. Tail consumers receive producer execution traces, including uncaught exceptions. Accessed 2026-10-08.
- AWS CLI with R2. Cloudflare docs. S3-compatible object listing and reads through an authenticated R2 endpoint. Accessed 2026-10-08.
- Manage streams. Cloudflare docs. Stream modes; used for unstructured JSON ingestion. Accessed 2026-10-01.
- Basin Catalog. Cloudflare docs. Sink configuration; used for the R2 Data Catalog sink and its permissions. Accessed 2026-10-01.
- Metrics and analytics. Cloudflare docs. Pipeline metrics; used to check dropped records and sink writes. Accessed 2026-10-01.
- Query data. Cloudflare docs. Query commands; used to verify rows land in the warehouse. Accessed 2026-10-01.
- Scalar functions. Cloudflare docs. JSON scalar functions; used to read request paths from the value column. Accessed 2026-10-01.
- Workers Logs. Cloudflare docs. Invocation logs contain request metadata; invocation_logs disables them without disabling custom logs. Accessed 2026-10-02.
- rat-stack/packages/events at main · joelhooks/rat-stack · GitHub. GitHub joelhooks/rat-stack. Events cartridge; used for the capture envelope, identity modes, and tests. Accessed 2026-10-01.
Last changed 2026-10-12 in 4533d33. Source on GitHub. Change log.
Linked from
- A clean playground. rat-stack does real work with packages that stay easy to add and remove.
Analytics supplies another working package, with an explicit removal checklist.
- Agent guide. An Effect stack so pure (aspirational) Kit Langton will blush.
Analytics — One event per HTTP request, stored unchanged for queries to select and interpret later.
- Agent rules for rat-stack (AGENTS.md). What you may change, which commands to run, and which changes need approval.
- An agent applied through Markdown and HTTP. One observed application used the Markdown page and llms.txt, then one POST; confirmation followed 88 seconds later.
This is one counts-only observation from our request analytics.
- Build and run rat-stack (README.md). What is in the repo, how the example works, and how to run it.
- Capture first, shape later. Store events without rejecting them for an evolving event shape, then decide their shape when you query.
Analytics documents that boundary.¶
- Cartridges. A cartridge owns one job and brings the infrastructure its service needs.
Analytics is a second running example: packages/events owns capture and its Basin resources.
- Change log. What changed in the files served here, newest first.
Analytics — edited: feat: log one path-only request line per Mischief request and joinInterest outcome
- Feature flags. Typed feature flags with owners, removal dates, and stable subject rollouts.
See Analytics for capture and identity behavior.¶
- Full agent guide. An Effect stack so pure (aspirational) Kit Langton will blush.
- Interest signup. Agent-only applications start email confirmation through interfaces owned by core and delivery implemented in a separate package.
Request analytics, trace spans, and logs never receive those values.
- One capability, every surface. A schema-typed contract and one handler feed every interface rat-stack keeps.
Analytics shows a separate infrastructure cartridge; interest signup shows core ports with a delivery adapter outside core.
- Property-based testing. State a rule that must hold for every input, then let a generator hunt for the smallest input that breaks it.
Analytics documents cookie behavior and what the checks establish about storage in Basin.¶
- Purpose and boundaries of rat-stack (VISION.md). What this starter is for and what a useful copy should keep.
Request analytics runs with a server-set persistent rat_vid cookie; event bodies exclude request bodies and sensitive query keys.
- Rat Stack systems | rat-stack. The systems rat-stack ships, the standard each keeps, and how to check it.
Analytics — One event per HTTP request, stored unchanged for queries to select and interpret later.
- rat-stack. An Effect stack so pure (aspirational) Kit Langton will blush.
- Source change history (log.md). The generated change log as Markdown.
Analytics — edited: feat: log one path-only request line per Mischief request and joinInterest outcome
- Trace operations where they mean something. Name useful operations, add safe context, and distinguish creating a span from delivering it to a backend.
Request analytics captures request events through its own sink.