fix(filler): defer the eager missing-mint reconciliation while the reader is catching up#69
Conversation
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Defers the expensive “missing mint number reconciliation” work during startup when the reader is far behind chain head, avoiding long startup stalls and potential crash-loops from DB statement_timeout.
Changes:
- Adds a reader-lag gate by comparing the persisted reader position (
contract_readers) tochainInfo.head_block_num. - Skips eager mint reconciliation when the reader is >10k blocks behind head, logging the deferral.
- Avoids querying
atomicassets_configentirely when reconciliation is skipped.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const blocksBehindHead = chainInfo.head_block_num - (readerPositionQuery.rows[0] ? Number(readerPositionQuery.rows[0].block_num) : 0); | ||
| const skipMintReconciliation = blocksBehindHead > 10_000; |
| const blocksBehindHead = chainInfo.head_block_num - (readerPositionQuery.rows[0] ? Number(readerPositionQuery.rows[0].block_num) : 0); | ||
| const skipMintReconciliation = blocksBehindHead > 10_000; |
| const contractsQuery = skipMintReconciliation | ||
| ? { rows: [] as any[] } | ||
| : await this.connection.database.query('SELECT * FROM atomicassets_config'); |
…ches up AtomicAssetsHandler.init() loops `CALL update_atomicassets_mints` (bounded 50k/ call) + a full atomicassets_assets COUNT until the missing-mint backlog is under 50k. While the reader catches up the update_atomicassets_mints job is deferred (shouldDeferDrain), so the backlog grows to millions of rows on large chains (observed ~8M on WAX). Running the reconciliation at boot then blocks the reader from ever starting AND busts the connection statement_timeout on the COUNT — so any filler restart mid-catchup becomes an init crash-loop (the reader never starts; the process sits idle). Gate it on the reader's distance from head. shouldDeferDrain() is unusable here because reader.blocksUntilHead is still 0 before the reader starts, so compare the reader's persisted contract_readers.block_num to chain head directly and skip the reconciliation while far behind. The deferred update_atomicassets_mints job fills the backlog once the reader reaches head. Loop body unchanged.
0fc2d79 to
8c4ea75
Compare
robrigo
left a comment
There was a problem hiding this comment.
Review (FACINGS) — LGTM.
Verified chainInfo (get_info()), this.filler.reader.name, and positiveIntEnv all exist. The gate fails closed — unknown/non-numeric position → null → skip — which is the safe direction (never falls open into the expensive reconciliation during catch-up). Patch applies cleanly onto feat/atomicassets-v2.
Note: Copilot's three inline comments were on an earlier commit — all three are already addressed in the current head: the threshold is the env-tunable MINT_RECONCILIATION_MAX_LAG_BLOCKS constant, Number.isFinite + clamp handle the NaN case, and the skip branch uses a plain [] rows array (not {rows: [] as any[]}).
AtomicAssets v2 (dual ownership/renting, mutable templates, schema media types, collection author swaps) was originally authored by Fabian Emilius in the WAX-OIG eosio-contract-api fork; this repo carries it forward as a reconstruction onto 1.7.2. Record the original authorship and co-attribute it. Also credits @igorls for the v2 catch-up/upgrade-safety fixes (#68/#69/#70). Co-authored-by: Fabian Emilius <20770096+fabian-emilius@users.noreply.github.com> Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
|
Rechecked against current |
Problem
AtomicAssetsHandler.init()loopsCALL update_atomicassets_mints(bounded 50k/call) + a fullatomicassets_assetsCOUNTuntil the missing-mint backlog is under 50k. While the reader is catching up theupdate_atomicassets_mintsjob is deferred (shouldDeferDrain), so the backlog grows large — ~8M rows observed on WAX mainnet.Running this reconciliation at boot then:
statement_timeouton the 466M-rowCOUNT,so any filler restart mid-catchup becomes an init crash-loop — the init promise rejects and the reader never starts (the process stays up but idle).
Fix
Gate the reconciliation on the reader's distance from head.
shouldDeferDrain()can't be used here becausereader.blocksUntilHeadis still0before the reader starts, so compare the reader's persistedcontract_readers.block_numto chain head directly and skip while far behind (>10k blocks). The deferredupdate_atomicassets_mintsjob fills the backlog once the reader reaches head. The reconciliation loop body is unchanged.Validation
pnpm check-typesclean.Reader is ~N blocks behind head - skipping eager missing-mint reconciliationand the reader resumes immediately instead of crash-looping.