Changes touching this path
- the forge keeps a bounded local disk cache of object bytes in front of R2 and overlaps 64 blob reads instead of 16, and a /fetch that runs out of budget says how far it got: blob_cache.rs CachedBlobStore writes through on put, reads through on get, removes the copy on delete and drops a fill a delete overtook, checks a blake3 checksum over every cached byte and fetches a copy that fails it again rather than serving it, and evicts the least recently read past LOOT_FORGE_BLOB_CACHE_BYTES (1 GiB by default, 0 for none) in LOOT_FORGE_BLOB_CACHE_DIR (default loot-forge-blob-cache under the temp dir, /tmp in the image); Forge::from_env puts it in front of the S3 tier, and exists and presign still ask the bucket. the budget refusal keeps its leading words and adds K of N objects read and the width, with the elapsed time to a tenth of a second. MAX_CONCURRENT_GETS is 64 because live, 16 in flight spent the 15 s budget on the 5,484 ticket files, a floor of about 44 ms per R2 GET, and 64 fits them up to about 175 ms a GET. it corrects mmxkront claims: the 2.9 ms R2 pace and the R2 latency test name, the stale bytes fetch comment, a budget check restored before each published key lookup, the fetch_bytes stop wording, the first failure recorded is the one reported rather than the lowest thread, and the mid fetch budget test no longer rides a 40 ms budget. pinned: at 44 ms a get a cold fetch of 5,484 objects answers in about 4.3 s and a warm one in about 0.07 s with no tier get and the same bytes. red under named mutations, each restored: over the 14 blob_cache tests, a hit that asks the tier (9 passed and 5 failed), a miss that keeps nothing (12 and 2), no eviction (12 and 2), no refresh on read (13 and 1), no checksum (13 and 1), a delete that leaves the copy (13 and 1), a fill that ignores a delete (13 and 1), an open that indexes nothing (13 and 1), a zero cap left on (13 and 1); over the 5 fetch_latency tests, 16 in flight (4 and 1), the refusal without its progress (4 and 1), the last failure reported (4 and 1), and the budget asked once before the fan-out (3 and 2). cargo test green, 4777 passed over 144 binaries with 13 ignored. owes a forge deploy, and no env is required; it also carries the new ticket rxxtvlvl and a comment on mmxkront (rxxtvlvl)
a0025c9d · dbf3dbe6… - the forge answers where a request spent its time: the new loot_forge::timing records a span per step, each ended where its step ends so the spans sum to the total, the last named answer or until_error, plus counts beside them, and a repo-scoped route whose steps report returns them as a Server-Timing header and writes the same text as one INFO line with the repo, endpoint and status, names durations and counts only. /ingest (ppwukxrx) and /stow name their handler and prepare steps, the ancestry check, publish, proposals, attestations, jobs, and on Postgres each statement family inside the transaction from store.open to store.heads_read, then commit and wake, and count the tree_entry rows written and the parent tree rows read. /fetch (zxkpzxvz) names the serve walk steps from a new Phase enum whose words the budget refusals now use, and counts each object read as a cache_hit or cache_miss with summed times, the time keeping copies took and the width, tallied per thread through the new BlobStore::get_with_source and reported on a refused walk too. CORS exposes Server-Timing and each answer carries Timing-Allow-Origin *, as the CORS layer already answers every origin. measured on a local Postgres 18 of 7.6M tree_entry rows, the ingest of one change onto a 7,240-path tree took 192 to 566 ms, 55 to 79 percent of it the 7,240 tree_entry rows written whole (106 to 447 ms), and a Tickets-shaped walk of 5,463 wants spent 77 to 81 ms outside the blob reads; against a plain-HTTP stand-in bucket at 50 ms a get the real S3 client read 2,048 objects in 6.54 s at 16 wide and 1.66 s at 64. sweep 2 fix-ups of rxxtvlvl: only the serving process opens the blob cache (Forge::serving_from_env; Forge::from_env, and so --reap-runner-deposits, reads the bucket bare) and an open removes no temp file younger than STALE_TEMP; the integrity sweep reads the bucket under the cache through blob::tier_of; the budget phase test covers every Phase by construction, not four by hand; and the MAX_CONCURRENT_GETS doc no longer calls 44 ms a floor. the Dockerfile makes /var/cache/loot-forge, owned by the forge user, for a cache volume. red under named mutations, each restored: spans measured from the start (6 passed and 1 failed over the timing tests, 0 and 2 over the two http tests), every read tallied a miss (0 and 1, and 0 and 1 over http), reads reported only when the walk answered (0 and 1), the sweep reading through the cache (0 and 1), an open removing fresh temp files (0 and 1), a refusal naming the span token (0 and 1), Server-Timing not exposed (0 and 1), no header on a refusal (0 and 1), held manifests counted as tree rows (0 and 1, against a throwaway Postgres 18). cargo test green, 4800 passed over 145 binaries with 13 ignored, and loot-forge green at 696 passed against a throwaway Postgres 18. owes a forge deploy; it also carries comments on vrxsvlsy, zkkkyqun and rxxtvlvl, the new tickets zxkpzxvz and uqooxqvx, and the resolution of zykuxznx, whose gzip change is scripts 3a21f24 (ppwukxrx)
2aacc76d · dbf3dbe6…diff - the Tickets view /fetch wants lane no longer falls to the residue query: since vrxsvlsy it names its heads as have, so every want reached repo_referenced_objects, whose plans on Postgres read every tree_entry row naming a want, every manifest of the repo, or the whole table; bundle_within now asks the new MetadataStore::referenced_by_versions of the manifests of the tips of what the caller named as held (serve::held_tips, tips because a CLI pull declares its closure) before the residue query, on Postgres pg::meta::REFERENCED_BY_VERSIONS, read one named version at a time and plan-pinned in pg::tests, and the answer is the same repo-scoped set. measured against a throwaway Postgres 18 with 2,200 ticket files of a 7,240-path head as wants and the head as have, the wants span went from 21.8-103.6 ms to 7.1-7.5 ms on the perf hunt fixture and from 575-1,820 ms to 11.4-15.2 ms on a repo whose ticket files sit in up to 1,000 manifests (the ADR 0075 zsxqmrtm amendment, and a note in ADR 0098). a /offer or /fetch step that fails now ends its own span before until_error; a budget refusal says how long since the walk began, keeping its leading words, and its objects-read count is taken once the gets in flight end, so it equals the cache_hit and cache_miss tallies summed; the fetch_latency cold test runs under an unspendable budget and asserts the fit from the width the tier saw instead of the wall clock. text: the timing module and CONTEXT.md say what a fact is and where facts are named instead of listing them (timing::count is timing::fact, Got.keeping is keep_time), the phases doc says the runner walk laps nothing and tallies no reads, and the lib.rs cache sentence names only the serving process. judgement calls: the route and the Tickets view build the ticket file list with the new site lib/workbench/ticket-files.ts and compare it by content, manifest.ts shares one private read, the wasm JS names are readTicketList and readWholeTickets, and ADR 0002 and the spike-crdt ignore reason say --include-ignored. red under named mutations, each restored: over the 16 offer_cost tests, no heads step (13 passed and 3 failed) and every named version read (15 and 1); the memory store reading every version (1 and 1 over the 2 conformance runs); the plan pin without OFFSET 0 (2 and 1); over the 27 serve tests, a failing step not lapped (26 and 1), the count taken at the refusal (26 and 1), the old spent wording (25 and 2), 16 in flight (26 and 1); the site ticket file compare as JSON text (2 and 1). the Postgres statement without its repo_id condition stayed green, as the RLS repo binding hides other repos. cargo test green, 4811 passed over 145 test result lines with 14 ignored, loot-forge green at 702 passed against a throwaway Postgres 18, the site gate green at 889 passed and its pg suites at 85. owes a forge deploy and a site deploy; it also carries the new ticket zsxqmrtm and comments on kwlxstko and zxkpzxvz (zsxqmrtm)
ed102f51 · dbf3dbe6…diff - the forge deletes the bytes of a burned object: the serving process runs the burn delete job #482 section 5 designed and nothing built, purge::delete_burned_bytes, once at start and then every 60 s, deleting from the blob tier each burned address that the new burn_bytes_deleted table of migration 0029 holds no row for and recording each delete that succeeded, so the job deletes only for a tombstone it reads as committed, a failed delete stays owed and is tried again on the next run, a missing object counts as deleted, a tombstone the backup replay inserts is owed like any other, and the first run deletes the bytes of every burn already on file; the delete goes through the blob cache, so it also takes the local copy off the disk. the owed set is read off the tombstones rather than kept as a queue of its own. loot burn no longer prints destroyed on the forge now, which was untrue when printed: the forge line says the forge deletes its copy of the bytes when a signed purge reaches it on a sync. pinned in the conformance cases a_burn_owes_the_delete_of_its_bytes_until_one_is_recorded and the_burn_job_deletes_the_bytes_and_asks_again_for_a_failed_delete over memory and Postgres 18, the blob suite case the_burn_job_leaves_the_tier_no_bytes_of_a_burned_address over the memory tier and the cache (not run against an S3 bucket, its test environment unset), the_burn_job_takes_a_burned_copy_off_the_disk, a_serving_forge_deletes_the_bytes_a_burn_owes, an_honored_burn_has_its_bytes_deleted_and_a_refused_one_does_not, the pg test a_burn_owes_its_delete_only_once_it_commits, and the disclosure test a_forge_line_says_the_forge_deletes_the_bytes_when_the_purge_arrives, which went 1 failed and 14 passed over the disclosure tests before the new wording. red under named mutations, each restored, over the 12 selected job tests on Postgres 18: no job in the serving process (1 failed and 11 passed), a failed delete recorded as done (2 and 10), the run stopped at a failed delete (2 and 10), the Postgres owed read without its recorded-delete clause (2 and 10), the memory store recording a delete for an unburned address (1 and 11), the owed set read off object rows (10 and 2), a cache delete that keeps its copy (2 and 10). cargo test green, 4863 passed over 145 test result lines with 15 ignored, and the loot-forge suite green against a throwaway Postgres 18. CONTEXT.md, ADR 0038 and ADR 0046 say so, with what the backup retention window still holds. owes a forge deploy, migration 0029 first, and a scripts change to the forge-restore.js restore note (wosupkwz)
dcae12fa · dbf3dbe6…diff - a push no longer fails past what the Postgres lock table holds, and the puts beside each other earn their deadline together: burned_under_lock took one advisory lock per published key of an ingest, and on a throwaway Postgres 18 at its defaults 13,000 such locks in one transaction were granted and 15,000 refused with out of shared memory, so a push of that many keys failed on every retry (live since uzkouqqx, on forge deploy.8); every address now maps onto one of pg::meta::BURN_LOCKS, 64, under a class of its own in the upper key bits, and record_burn takes the one its address maps to. each put of a stow or ingest earns its S3 deadline from the bytes of all the puts beside it, BlobStore::put_sharing over the new loot_s3 S3Client::put_sharing and handed on by the blob cache, where up to 64 puts shared the link with deadlines each earned by its own body. also built from xyqwqnnv: prepare owes the delete of a burned address it put back again, owe_burn_deletes, before its own delete, so a failed delete there no longer strands bytes the burn job had recorded deleted; the burn job warns once a run with a count, and the server log names a failed record as well as a failed read; the loot burn forge lines say the forge deletes its copy of the bytes when it honors the purge, and that the backups retain the rows naming the object rather than a copy; ADR 0046 section 2 no longer says forge-restore.js prints the re-delete as a step for the operator. the put_cost width is asserted from a hold the stand-in tier keeps until the bound is in flight, not from the clock, and the rounds assertion is gone; the CONTEXT.md Forge phase timing entry says the transaction writes the object rows again; the tree.ts collation comment is rewrapped. pinned: a_push_with_more_keys_than_the_lock_table_holds_commits over 25,600 keys on Postgres 18, each_put_earns_its_deadline_from_the_bytes_sharing_its_link, a_put_sharing_its_link_is_held_to_the_deadline_the_shared_bytes_earn, a_put_hands_the_tier_the_bytes_sharing_its_link, bytes_put_back_after_the_job_ran_are_owed_again over memory and Postgres, the conformance case a_recorded_burn_delete_can_be_owed_again, and a_run_warns_once_however_many_deletes_fail. red under named mutations, each restored: a lock per address (1 failed and 1 passed, out of shared memory), put_blobs calling put (1 and 24), the cache not handing sharing on (1 and 24), the S3 client ignoring sharing (1 and 19), half the put width (1 and 4), one past it (1 and 4), prepare not owing the delete again (2 and 2, over memory and Postgres), a Postgres owe that deletes nothing (2 and 2), a memory owe that removes nothing (2 and 2), a warning per failed address (1 and 15), and the old media copy wording (1 and 32). cargo test green, 4872 passed over 145 test result lines with 15 ignored, the loot-forge suite green against a throwaway Postgres 18 at 739 passed, and the site gate green at 915 passed. it also carries a comment on wosupkwz. owes a forge deploy and no migration; the burn line rides the next release (qptwwkqx)
7daffdfd · dbf3dbe6…diff
Renames are not followed. loot's tree maps a path to an address, so a rename is a delete and an add. This list is the history of the name, not of the bytes.