Skip to content

[Backport 1.3] Fix write retries, Pipe resource cleanup, and snapshot recovery#18218

Merged
jt2594838 merged 1 commit into
apache:dev/1.3from
Caideyipi:cp-18180-dev-1.3
Jul 16, 2026
Merged

[Backport 1.3] Fix write retries, Pipe resource cleanup, and snapshot recovery#18218
jt2594838 merged 1 commit into
apache:dev/1.3from
Caideyipi:cp-18180-dev-1.3

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Backport of #18180 (8b60437) to dev/1.3.

This adapts the fixes that apply to the 1.3 codebase:

  • rebuild ConfigNode Pipe listener references and replace stale snapshot-derived state;
  • clean up generated Pipe/subscription TsFiles and async transfer resources on failures;
  • reset listening queues safely during snapshot deserialization;
  • preserve maximum recovered flush times, clear last-flush memory accounting, and report rejected tablet ranges;
  • roll back aligned-row memory accounting when the following non-aligned reservation fails.

The 2.x-only table-model Pipe builders and consensus-subscription frontier/commit-progress implementation are not present on dev/1.3, so those parts of the original commit are intentionally omitted.

Verification

  • mvn spotless:apply -pl iotdb-core/node-commons,iotdb-core/datanode,iotdb-core/confignode
  • mvn -o -nsu -Ddevelocity.off=true test-compile -DskipTests -pl iotdb-core/node-commons,iotdb-core/datanode,iotdb-core/confignode
  • Targeted tests for:
    • ConcurrentIterableLinkedQueueTest
    • AbstractPipeListeningQueueTest
    • PipeTransferTsFileHandlerCleanupTest
    • LastFlushTimeMapTest
    • SubscriptionPipeTsFileEventBatchCleanupTest
    • PipeConfigRegionListenerSnapshotTest
    • QuotaInfoTest
    • TemplatePreSetTableTest
    • TemplateTableTest

…snapshot recovery (apache#18180)

* Fix write retry, pipe fallback, and snapshot recovery edge cases

* Fix additional pipe, write, and snapshot recovery failures

* Added fixes

* Harden subscription commit progress deserialization

* Fix consensus subscription initial frontier synchronization
@jt2594838 jt2594838 merged commit 8ea23e5 into apache:dev/1.3 Jul 16, 2026
16 checks passed
@jt2594838 jt2594838 deleted the cp-18180-dev-1.3 branch July 16, 2026 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants