Skip to content

feat: component-polish batch — network backpressure/rate-limit, client convergence/offline-parity, react useSyncExternalStore - #79

Merged
ivkan merged 2 commits into
mainfrom
feat/cleanup-batch-network-client-react-todo509-510-500-501-515
Jun 21, 2026
Merged

feat: component-polish batch — network backpressure/rate-limit, client convergence/offline-parity, react useSyncExternalStore#79
ivkan merged 2 commits into
mainfrom
feat/cleanup-batch-network-client-react-todo509-510-500-501-515

Conversation

@ivkan

@ivkan ivkan commented Jun 21, 2026

Copy link
Copy Markdown
Member

Summary

Component-polish cleanup batch ahead of core freeze — three independent groups, each fixing a G9 depth-audit finding with a behavioral test + a negative control (the bug reproduces before the fix).

Group 1 — Network / transport (DEPTH_NETWORK_TRANSPORT F4/F5)

  • TODO-509 — slow-consumer no longer diverges silently. ConnectionHandle::try_send_broadcast replaces the silent try_send drop on the live-push paths (CRDT delta, query update, pub/sub). A full outbound channel still drops the event (memory stays bounded) but advances a per-connection consecutive-drop counter; once it reaches slow_consumer_drop_threshold (default 256) the client connection is cancelled to force reconnect + Merkle resync instead of diverging silently. Adds a cumulative dropped_broadcasts metric; the counter resets on any successful send; cluster peers are exempt (the failure detector owns their liveness).
  • TODO-510 — per-connection data-plane rate limit. A TokenBucket in the WS Phase-2 read loop caps inbound op rate per connection (default 20 000 ops/s sustained, 40 000 burst). A single peer's OpBatch flood is throttled with a best-effort 429 back-off without tearing the connection down ("отбой, не падение"); op-cost is charged per op so packing many ops into one batch cannot evade it.

Group 2 — Client SDK (DEPTH_CLIENT_SDK F8/F11)

  • TODO-500 — optimistic re-stamp convergence. New LWWMap.adoptServerEcho() reconciles the server's arrival-order re-stamp of our own optimistic write: when LWW rejects the echo because our clock briefly led the server, it adopts the server's authoritative timestamp only when the value is unchanged (value-equality guarded → no data loss) so memory, disk, and the Merkle tree converge with the server. SyncEngine.applyServerEvent now gates the storage put on merge/adopt instead of the unconditional put that created the memory/disk HLC skew.
  • TODO-501 — offline query parity. runLocalQuery now applies sort + limit offline (sharing one comparator with the hybrid path) and explicitly rejects cursor/groupBy/aggregations rather than silently dropping them; the live-query pre-load degrades gracefully on reject.

Group 3 — React bindings (DEPTH_REACT F1/F2/F5/F6; folds TODO-516)

  • TODO-515 — useSyncExternalStore migration. All 12 subscription hooks move to useSyncExternalStore (with a <18 shim) for synchronous cached snapshots (no first-render loading flash), tearing-freedom under concurrent rendering, and removal of the isMounted anti-pattern. useMap/useORMap drive re-renders via a stable version-counter snapshot while still returning the live mutable map; handle lookups are memoized by name (render purity, F5). QueryHandle/HybridQueryHandle/SearchHandle/HybridSearchHandle gain cached, referentially-stable getSnapshot() accessors. useTopic no longer re-subscribes on inline-callback identity change (TODO-516 folded in).

Cross-vendor adversarial review (GLM-5.2)

Ran on the correctness-critical diffs (509/510 + 500). Acted on the real findings: the rate-limit 429 is now best-effort try_send (no read-loop stall on a slow client); added a recursion-depth guard to the value comparator and a timestamp-precondition guard to adoptServerEcho; documented the relaxed-ordering disconnect as a safe (resync, no-data-loss) failure mode. Bounded/non-issues (Batch undercount, async-put ordering, sort insertion order) evaluated and dismissed with rationale.

Test plan / gates (all green locally)

  • @topgunbuild/core: 2074 passed (+ new adoptServerEcho convergence/no-data-loss/tombstone/depth tests)
  • @topgunbuild/client: 660 passed (+ SyncEngine put-gating, QueryManager offline sort/limit/reject)
  • @topgunbuild/react: 203 passed (+ useSyncExternalStore migration suite, useTopic subscribe-count/StrictMode)
  • topgun-server lib: 1425 passed (+ slow-consumer 7, TokenBucket 7, inbound_op_cost)
  • cargo clippy --all-targets --all-features -- -D warnings: clean · cargo fmt --check: clean
  • pnpm test:sim: 4 passed · pnpm format:check: clean · pnpm lint: 0 errors
  • pnpm test:integration-rust (blocking gate): see CI

🤖 Generated with Claude Code

…t convergence/offline-parity, react useSyncExternalStore

Network (DEPTH_NETWORK_TRANSPORT F4/F5):
- TODO-509: slow-consumer broadcasts no longer drop silently. ConnectionHandle
  gains try_send_broadcast: a full outbound channel drops the event (bounded
  memory) and advances a per-connection consecutive-drop counter; past
  slow_consumer_drop_threshold (default 256) the client connection is cancelled
  to force reconnect + Merkle resync instead of diverging silently. Adds a
  cumulative dropped_broadcasts metric; counter resets on any successful send;
  cluster peers exempt (failure-detector owns their liveness). CRDT/query/
  pubsub live-push paths routed through it.
- TODO-510: per-connection inbound op-rate TokenBucket on the WS data plane
  (default 20k ops/s sustained, 40k burst). One peer's OpBatch flood is
  throttled with a 429 back-off without tearing the connection down; op-cost
  counts batch ops so packing cannot evade the cap.

Client (DEPTH_CLIENT_SDK F8/F11):
- TODO-500: LWWMap.adoptServerEcho reconciles the server's arrival-order
  re-stamp of our own optimistic write (value-equality guarded → no data loss)
  so memory/disk/Merkle converge with the server. SyncEngine.applyServerEvent
  now gates the storage put on merge/adopt instead of persisting the rejected
  echo unconditionally (the memory/disk HLC skew bug).
- TODO-501: offline runLocalQuery now applies sort+limit (shared comparator with
  the hybrid path) and explicitly rejects cursor/groupBy/aggregations rather
  than silently dropping them; live pre-load degrades gracefully on reject.

React (DEPTH_REACT F1/F2/F5/F6, folds TODO-516):
- TODO-515: migrate the 12 subscription hooks to useSyncExternalStore (with a
  <18 shim) for synchronous cached snapshots (no first-render loading flash),
  tearing-freedom, and no isMounted anti-pattern. useMap/useORMap drive
  re-renders via a stable version-counter snapshot while still returning the
  live map; handle lookups memoized by name (render-purity, F5). QueryHandle/
  HybridQueryHandle/SearchHandle/HybridSearchHandle gain cached, referentially-
  stable getSnapshot accessors. useTopic no longer re-subscribes on inline
  callback identity change (TODO-516 folded).

Tests: connection slow-consumer (7, incl. draining/reset/cluster-peer negative
controls) + TokenBucket (7) + inbound_op_cost; LWWMap adoptServerEcho +
SyncEngine put-gating (incl. divergence + no-data-loss controls); QueryManager
offline sort/limit + unsupported-clause rejection; react useSyncExternalStore
migration (8) + useTopic subscribe-count/StrictMode.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 21, 2026

Copy link
Copy Markdown

Deploying topgun with  Cloudflare Pages  Cloudflare Pages

Latest commit: c918a71
Status: ✅  Deploy successful!
Preview URL: https://f3c94abb.topgun-f45.pages.dev
Branch Preview URL: https://feat-cleanup-batch-network-c.topgun-f45.pages.dev

View logs

- websocket: 429 rate-limit reply is best-effort try_send, not await — a
  flooding client is behind on its outbound channel, so awaiting would stall
  the read loop on the very connection being throttled.
- LWWMap.valuesEqual: depth guard (>64) so a pathological/cyclic in-memory
  value can't stack-overflow the server-echo reconcile path.
- LWWMap.adoptServerEcho: precondition rejecting a strictly-newer server
  record (plain merge already wins it) — never downgrade through the echo path.
- connection: document relaxed-ordering safe-failure on the broadcast drop path.
@ivkan
ivkan merged commit 4ad0417 into main Jun 21, 2026
18 checks passed
@ivkan
ivkan deleted the feat/cleanup-batch-network-client-react-todo509-510-500-501-515 branch June 21, 2026 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant