Changes touching this path
- Scaffold site/ + wire Vercel #2, loot. subdomain, install proxy (#257)
Thin end-to-end tracer for loot.millerbyte.com (spec docs/specs/loot-site.md
SS1-3): site/ = TanStack Start SSG consuming published @millerbyte/ui@^0.1.0
(theme.css at the root), five placeholder surface routes prerendered, deployed
on Vercel project #2 (root site/) at https://loot.millerbyte.com.
Install proxy: the spec's vercel.json external-rewrite mechanism was falsified
live (Vercel passes GitHub's releases/latest 302 through to the client), so
/install.sh and /install.ps1 are Start server routes that fetch upstream and
stream the bytes back 200 - spec SS2 amended in place.
site/** explicitly public in .lootattributes; .vercelignore allowlists site/
so .loot/ keys and docs/pitch/ can never reach the Vercel builder; site build
artifacts lootignored.
1931146c · dbf3dbe6… - primary catch-up: checkout at the #291 squash (site/ scaffold) folded for the drift reconcile (#297)
e6f7079a · dbf3dbe6…diff - Scaffold site/ + wire Vercel #2, loot. subdomain, install proxy (#257) (#291)
* Scaffold site/ + wire Vercel #2, loot. subdomain, install proxy (#257)
Thin end-to-end tracer for loot.millerbyte.com (spec docs/specs/loot-site.md
SS1-3): site/ = TanStack Start SSG consuming published @millerbyte/ui@^0.1.0
(theme.css at the root), five placeholder surface routes prerendered, deployed
on Vercel project #2 (root site/) at https://loot.millerbyte.com.
Install proxy: the spec's vercel.json external-rewrite mechanism was falsified
live (Vercel passes GitHub's releases/latest 302 through to the client), so
/install.sh and /install.ps1 are Start server routes that fetch upstream and
stream the bytes back 200 - spec SS2 amended in place.
site/** explicitly public in .lootattributes; .vercelignore allowlists site/
so .loot/ keys and docs/pitch/ can never reach the Vercel builder; site build
artifacts lootignored.
Landed via break-glass git branch: the loot-native lane land was blocked by
two live shared-store bugs (review-ferry folds a described WIP into a
reconcile finalize; describe/status disagree on the working change under
concurrent sessions) - tickets to follow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Apply #291 review findings
Cache only successful upstream responses in the installer proxy (a transient
GitHub failure must not be CDN-cached for 5 minutes); extract the install
one-liners to one shared module (three copies drifted apart otherwise); fix
the vite.config "fully static" comment and spec SS0 framing to name the two
server-route exceptions; tighten the .lootattributes rationale (the interim
manual deploy is gated by .vercelignore, not the GitHub projection); drop the
cargo-culted overrides block (its pins targeted server deps the site lacks).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Git-Author: Connor Miller <53197564+Connor-Miller@users.noreply.github.com>
231eb5f8diff - the published index outflanks the metadata gate for exactly the published facts, and the site grows the anonymous read tier that serves it (#749)
8ed81d63 · dbf3dbe6…diff - the browser mints the seed and the account binds the key: Clerk in the site's own routes, variant-B onboarding, and the account tier's one producer (#753)
Migration 0007 adds account_seed (the passphrase-wrapped envelope, opaque
ciphertext the server cannot open) and forge_identity, a NOLOGIN role scoped
to exactly the identity tier -- SELECT/INSERT on tenant, SELECT/INSERT plus a
column-scoped UPDATE of retired_at on account_key, the full row on
account_seed, and nothing else (pinned live by
forge_identity_is_scoped_to_the_identity_tier).
The producer lives in site/src/server/identity/ behind a verified Clerk
session (verifyToken in the site's own route handlers, reusing the millerbyte
Clerk instance and pattern, never the gateway service). Registration is one
transaction: upsert tenant on the verified clerk_user_id, add the key, store
the envelope, retire what rotation names -- and never zero live keys. The
Rust service still writes none of the tier; tests/account_tier.rs now guards
that boundary instead of a vacancy.
Onboarding is #487's variant B, the two-column contract: the app owns the 32
bytes (crypto.getRandomValues -> 24-word BIP39 over the ENTROPY, never the
64-byte derivation -> ed25519 pubkey via @noble/curves, held to the engine's
from_seed by the pinned RFC 8032 vector), a WebCrypto PBKDF2+AES-GCM envelope,
a .txt download that excludes the passphrase, and two consent checkboxes that
no download waives. Clerk mounts only under /account, which joins /k/ on the
prerender deny-list; every static surface stays static.
Follow-ons filed rather than folded: #926 (CLI raw-seed import, so BIP39
recovery reaches the CLI) and #927 (account-aware may_push, burn entitlement
via account, and the #506 historical-key 301s the anonymous read tier
deliberately cannot serve today).
278e1886 · dbf3dbe6…diff - the override was holding esbuild on the vulnerable branch: site npm audit goes to zero, in range (#916)
Four advisories, all four in-range, and the interesting one was ours. `npm audit
fix` cleared js-yaml 4.3.0 -> 4.3.1, nanoid 3.3.16 -> 3.3.18 and postcss 8.5.19
-> 8.5.26, then went on reporting esbuild vulnerable with a fix supposedly
available via itself — because site/package.json OWN overrides block pinned
`esbuild: ^0.27.0`, the vulnerable branch. Every declarer accepts `^0.28.0`
(vite 8 peer `^0.27.0 || ^0.28.0`, vite 7 the same, unplugin `*`), so moving the
override to `^0.28.2` violates no constraint and needs no resolver force.
That override block is how the last three of these were fixed (path-to-regexp,
undici); it is the mechanism, and it needs re-reading at each bump or it becomes
the thing holding the vulnerability in place — which is what it was doing here.
The lockfile diff is 30 entries and all of them are these four packages: esbuild
and its 26 per-platform binaries, plus js-yaml, nanoid, postcss. No package
added, none removed, no unrelated resolver churn, package.json otherwise
untouched. `npm ci` succeeds in site/ now, so the #894 install-repair hazard did
not ride along.
Verified: `npm audit` 0 vulnerabilities, `npm test` 75 passed | 23 skipped,
`npm run build` typechecks and prerenders all 8 pages.
Reachability, since the ticket asked whether the two highs are urgent: all four
are build-time only — esbuild and postcss through vite, nanoid through postcss,
js-yaml through xmlbuilder2 inside @tanstack/start-plugin-core. None can reach a
served response. No audit gate: it is clean today, but its trigger is a third
party publishing an advisory, so a gate breaks unrelated builds on a schedule
nobody here sets — the #642 permanently-yellow objection with the yellow
arriving later. sdk/ is worse (7, one needing a vitest major) and is filed
as #951.
f77ce77e · dbf3dbe6…diff - the browser opens what the forge cannot: own-private read ships an owner read tier, a once-per-device unlock, and a grant mailbox that never acks (#1043)
c8dd4da1 · dbf3dbe6…diff - a land builds the site and weighs the eager bytes of every published surface, so a redesign cannot spend the byte lead in silence, and the four places that called site/ ungated now say what is true (#1241)
674361ad · dbf3dbe6…diff - the site detector pinned the Node whose npm cannot read this repo lockfile, so 38 runs refused at npm ci without once reaching the gate they exist to re-run (#1371)
ed71532c · dbf3dbe6…diff - the lockfile is regenerated by an npm every other npm can read, because pinning CI to the same major was a guess and the runner bundles npm 12 where the land gate bundles 11.6 (#1371)
782fc0f5 · dbf3dbe6…diff - the shared classifier lands, and the review caught me pinning the exact defect 1625 filed a hard constraint against: I wrote curl into the BOTS list and asserted it IS a bot, which would report ZERO INSTALLS while installs were happening - a curl of install.sh is, per 1621, the only observable event in the middle island of the funnel. the cause was a research failure rather than a coding one: I built the ticket from 1625 resolution and the map memory and never read the comment carrying its hard constraint, so my own ticket body omitted the carve-out too.
the rule is therefore keyed on WHAT WAS REQUESTED and not on who requested it. a scripted client on install.sh, install.ps1 or dl is the product working and classifies unknown; the SAME agent on docs is a crawler and is filtered, which is the control that proves the carve-out is scoped rather than a blanket exemption. a real crawler on install.sh is still a crawler, because the carve-out covers scripted clients and not everything. the signature had to widen to take the path, which log-line now passes from the line it already holds - a classifier given only the user agent cannot express the constraint at all.
the tests caught a second bug that was mine: powershell anchored with word boundaries never matches WindowsPowerShell, since there is no boundary inside it, and that failure direction is the dangerous one - a missed scripted client is counted as a bot, which is the funnel deletion itself. the match is a substring now and the looser direction is argued rather than assumed.
agent becomes a family name, because isbotMatch returns Google for googlebot but the WHOLE user agent for bingbot and gptbot - storing that would put a raw UA in a column and make every per-family chart ungroupable. it is non-null exactly when class is bot, which is the privacy boundary: a crawler is not a natural person and which crawler is the analytic value, but the same derivation on an unmatched request keeps a piece of a real visitor UA under another column name. the supplement carries the path rules; its own-agents half is deliberately empty and says why, since the loot CLI sends no user agent at all and the install detector is excluded at the edge.
standards review landed four real ones: Verdict collided with the CONTEXT glossary term for a reconciliation outcome and becomes Classification; the header claimed three consumers in the present tense while none are wired, which is this repo own recurring doc defect written by me again; the classifier shape was declared three times and now lives once in the leaf that both importers name; and isbot was called twice where isbotMatch answers both questions (#1775)
8b1746c6 · dbf3dbe6…diff - the shipper gets the entry point a timer will run, and the reviews caught three defects in it that no gate could: an unreadable log read as a CLEAN RUN, because open swallowed every error and the caller skipped the file - so an EACCES on www-data adm 0640, which is the precise hazard the whole runtime-home decision exists to handle, produced scanned zero, failed false, exit zero, indistinguishable from a fresh box. only absence is swallowed now and a permission error is rethrown, with a control asserting a genuinely missing file is still skipped, since every box before its first rotation has no .1.
the second was a rotation-sized race in the middle of the file source: stat by name then open by name lets logrotate rename between the two calls, so the read lands on the NEW inode bytes and they are committed under the OLD inode position - a double count and a misattribution at once, and scheduled rather than theoretical since rotation is unconditional and daily. a handle is bound to an inode rather than a name, so the source opens once and measures and reads through that same handle, which makes the race unrepresentable instead of unlikely.
the third would have broken every land and every deploy: build:shipper invokes esbuild, which was in overrides and in NEITHER dependency list, resolving only as vite transitive binary - and putting that step inside build made the gate depend on an undeclared one. declared now, matching the override range rather than pinned exact, which npm refuses as a conflict.
found by RUNNING the bundle rather than building it: bundled to esm it compiles clean and dies at startup on Dynamic require of pg, which is CommonJS - and external is no escape because the image has no node_modules at all since nitro bundles its own into server libs. it is cjs, pinned four ways including that the step runs AFTER vite build, since vite writes .output and an earlier step would have its artifact deleted silently.
two doc claims are narrowed rather than left standing, both mine and both the catalogued class: the entry said what the timer runs in the present tense with no timer anywhere, and credentials.ts said the site has no one entry point this module owns, which was true of the serving process and became false of the deployment the moment a one-shot binary existed. and two of my own assertions are DELETED rather than kept, because they were strict consequences of the set-equality above them and could not fail while it passed - decoration that read like coverage (#1756)
0c0385a8 · dbf3dbe6…diff - the retention number becomes an OUTCOME: a daily rollup writes the marginal totals that survive forever and only then drops raw partitions that are both rolled up and past the window
loot#1759, decided by loot#1622 and amended by loot#1625. Until this, thirty
days was a sentence: nothing dropped a row and the raw table grew without bound
on the Postgres volume the relay, the forge and the site share.
The drop is gated on the watermark, never on time alone, and that is the
ticket whole thesis. event_drop_expired() in migration 0003 takes NO argument:
it reads rolled_up_through from the table, uses its own UTC clock, and applies
a retention CONSTANT - so the rollup role cannot widen the drop by lying to it,
and with no watermark it drops nothing. A partition is due only when its day is
at-or-below the watermark AND its whole day is outside the window, so a stalled
rollup ACCUMULATES partitions, which is the observable failure mode we want,
rather than deleting a day nobody aggregated.
The ticket acceptance test is pinned as written: freeze the watermark and
nothing is dropped however old the partitions are. The mutation that makes the
rule time-based reddens exactly that pin and its BOTH-conditions sibling and
leaves every time-only test green, which is the discrimination the ticket asked
for. A day that fails to roll up STOPS the run with the watermark at the last
success; skipping it would advance past a day with no aggregates, the silent
loss the watermark exists to prevent.
Every surviving table is MARGINAL and a test PARSES the CREATE TABLE statements
to prove it: at most one dimension beyond (day, class), never two
visitor-descriptive columns together, never one with path, and no table carries
visitor_key, account_ref or telemetry_id - which is what makes an erasure
complete once the raw rows go. class rides every aggregate and agent gets its
own per-family table (loot#1625); rule_versions reaches every row so rule
boundaries outlive the raw window. No country table, because nothing produces
country.
A THIRD role, metrics_rollup: SELECT on event, which the writer may never have;
write on the rollup tables and the watermark, which the reader may never have;
NO grant on daily_salt; and DDL only through two SECURITY DEFINER doors. The
partition door moves a day rows out of event_default in one transaction with an
explicit UTC bound - p_day::timestamptz would have taken the session time zone
and filed one row under two days depending on who created the partition.
The TypeScript twin of the drop rule PREDICTS what the database will drop and
the summary reports a disagreement loudly with a non-zero exit, because two
implementations of a retention rule parting silently is exactly how a retention
promise stops being true while every timer reads healthy. The bundle was RUN,
not just built: with no credential it starts, reaches main and fails naming the
cause, so the cjs/pg hazard the shipper found is absent here.
The notice moved with it, and a test that asserted the page must NOT claim a
database bound is now inverted: it must claim it, from the shared constant,
and must say the deletion is gated on the totals - a page saying only deleted
after 30 days would describe the time-based expiry loot#1622 rejected. The
account-reference sentences from loot#1758 shortened from as long as the
account to at most 30 days, which is stricter and simpler. Rendered and read.
One header claiming nothing installs a shipper timer yet, a week after one was
installed, is corrected while the rollup was given the same shape.
587 site tests, budget green with the notice raise on the record.
50fe5557 · dbf3dbe6…diff
Renames are not followed. loot's tree maps a path to an address, so a rename is a delete and an add. This list is the history of the name, not of the bytes.