Changes touching this path

  • the shipper gets its writer, its position and the two boundary conditions that would lose data silently, and the position lives in the SAME database as the rows for a reason the code now keeps: a crash between an insert and a position update double counts, silently, because the re-read yields perfectly valid rows - so commitBatch writes both in one transaction rather than the two calls I first wrote, which would have left my own migration comment saying committed alongside them while the code did no such thing. two stores can disagree after a crash; one cannot disagree with itself. the last line is consumed only if it is COMPLETE. nginx appends, so a run landing mid-write sees half a line, and parsing it would refuse it as malformed AND advance past it, so the finished line would never be read - a permanent silent loss of one request that no error would report. everything after the final newline is left for the next run. the boundary is found in the BUFFER before anything is decoded, because the position is a byte offset and user agents are routinely non-ascii, so measuring in characters would shift every offset after a multibyte line; the test carries its own control asserting the fixture really is multibyte, since otherwise it would pass under either implementation. the position is keyed by INODE and not by name, because names rotate - todays access log is tomorrows .log.1 and 1641 made rotation daily and unconditional - and a file SHORTER than our recorded offset was replaced under a reused inode, which logrotate create does routinely, so it is re-read from zero rather than resumed past its opening lines. a refused line still advances the position, or one unparseable line would wedge the shipper behind it forever, and that stall looks exactly like a site with no traffic. and the collision I predicted before starting gets a guard rather than a note: migration numbers are a shared sequential resource and NOTHING was checking them - land refuses a duplicate ADR number under the harbor lock, which is what kept 0074 and 0075 apart when two sessions wrote them the same night, but it knows nothing about site/migrations. two lanes taking 0002 would collide silently and the loser would never run, because the runner records by name and the winner already recorded it. six directory assertions now cover uniqueness, gaplessness, zero padding, and that each file records itself under its OWN name - the copy-paste mistake that makes a table quietly not exist under a green deploy - proved red by planting a duplicate 0002 (#1756) 823ebe57 · dbf3dbe6…
  • the shipper stops being able to ingest without a classifier, and the requirement is the mechanism rather than a note on a ticket: a row can NEVER be reclassified, because 1620 keeps no raw user agent so a later rule cannot revisit it - revisability is bought with provenance and not evidence, and history is never restated. so every row ingested before the shared classify module exists is permanently unjudged, and 1625 default view of class not in bot or internal would count that whole era of bot traffic as ordinary visitors, forever, with nothing in the data saying so. waiting costs NO data and that is what settles it: the logs are retained 30 days, so the bytes sit on disk either way. shipping early does not gain data, it converts recoverable data into rows nobody can ever classify. the type makes omission a compile error and the runtime check makes it an error for a caller that is not typescript, because this guard has to survive the bundler, the entry point, and whoever wires it up months from now - and the refusal says why waiting is free rather than only that it refused, since a message that explains the cost is the difference between someone fixing it and someone deleting the check. the boundary tests pass a real classifier through one helper rather than each supplying its own, so the requirement is stated once and those tests stay about what they are about. two new assertions cover the refusal and its reasoning, both proved red by disabling the guard. this lands the collector core complete and DORMANT: no timer is installed, so nothing runs, and 1756 stays open deliberately rather than being closed over something that does not execute (#1756) c2f0bc87 · dbf3dbe6…diff
  • the shared classifier lands, and the review caught me pinning the exact defect 1625 filed a hard constraint against: I wrote curl into the BOTS list and asserted it IS a bot, which would report ZERO INSTALLS while installs were happening - a curl of install.sh is, per 1621, the only observable event in the middle island of the funnel. the cause was a research failure rather than a coding one: I built the ticket from 1625 resolution and the map memory and never read the comment carrying its hard constraint, so my own ticket body omitted the carve-out too. the rule is therefore keyed on WHAT WAS REQUESTED and not on who requested it. a scripted client on install.sh, install.ps1 or dl is the product working and classifies unknown; the SAME agent on docs is a crawler and is filtered, which is the control that proves the carve-out is scoped rather than a blanket exemption. a real crawler on install.sh is still a crawler, because the carve-out covers scripted clients and not everything. the signature had to widen to take the path, which log-line now passes from the line it already holds - a classifier given only the user agent cannot express the constraint at all. the tests caught a second bug that was mine: powershell anchored with word boundaries never matches WindowsPowerShell, since there is no boundary inside it, and that failure direction is the dangerous one - a missed scripted client is counted as a bot, which is the funnel deletion itself. the match is a substring now and the looser direction is argued rather than assumed. agent becomes a family name, because isbotMatch returns Google for googlebot but the WHOLE user agent for bingbot and gptbot - storing that would put a raw UA in a column and make every per-family chart ungroupable. it is non-null exactly when class is bot, which is the privacy boundary: a crawler is not a natural person and which crawler is the analytic value, but the same derivation on an unmatched request keeps a piece of a real visitor UA under another column name. the supplement carries the path rules; its own-agents half is deliberately empty and says why, since the loot CLI sends no user agent at all and the install detector is excluded at the edge. standards review landed four real ones: Verdict collided with the CONTEXT glossary term for a reconciliation outcome and becomes Classification; the header claimed three consumers in the present tense while none are wired, which is this repo own recurring doc defect written by me again; the classifier shape was declared three times and now lives once in the leaf that both importers name; and isbot was called twice where isbotMatch answers both questions (#1775) 8b1746c6 · dbf3dbe6…diff

Renames are not followed. loot's tree maps a path to an address, so a rename is a delete and an add. This list is the history of the name, not of the bytes.