Skip to content

fix(storage): consume server responses before finishing stream in AsyncWriter Close/Finalize#16270

Open
kalragauri wants to merge 3 commits into
googleapis:mainfrom
kalragauri:feat/close-hangs
Open

fix(storage): consume server responses before finishing stream in AsyncWriter Close/Finalize#16270
kalragauri wants to merge 3 commits into
googleapis:mainfrom
kalragauri:feat/close-hangs

Conversation

@kalragauri

@kalragauri kalragauri commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

This PR fixes an issue where calling AsyncWriter::Close() or Finalize() hangs indefinitely when the GCS service returns intermediate response messages before closing the stream.

Under the hood, Close() sends a kFlushAndClose request, which prompts GCS to respond. Previously, AsyncWriterConnectionImpl called Finish() on the underlying gRPC stream immediately after writing without consuming any remaining server responses. This unread data on the stream caused Finish() to block waiting for response consumption, leading to a deadlock.

The PR also updates unit tests that mock the upload stream to expect the additional Read() call (returning EOF) and adjust the sequencer expectations.

@product-auto-label product-auto-label Bot added the api: storage Issues related to the Cloud Storage API. label Jul 21, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors OnClose and OnFinalUpload in AsyncWriterConnectionImpl to use a recursive ConsumeLoop that reads and processes all intermediate server responses before finalizing the upload, preventing hangs when multiple responses are sent before EOF. Corresponding unit tests are updated and a new test is added to verify this behavior. The review feedback correctly identifies compilation errors in both refactored methods where the undefined variable self is referenced inside lambdas capturing [this], and provides code suggestions to resolve these issues by calling Finish() directly.

Comment thread google/cloud/storage/internal/async/writer_connection_impl.cc Outdated
Comment thread google/cloud/storage/internal/async/writer_connection_impl.cc Outdated
@kalragauri
kalragauri force-pushed the feat/close-hangs branch 4 times, most recently from 6d2b7b3 to cc63d2d Compare July 21, 2026 10:00
@kalragauri
kalragauri marked this pull request as ready for review July 21, 2026 10:22
@kalragauri
kalragauri requested review from a team as code owners July 21, 2026 10:22
@kalragauri
kalragauri force-pushed the feat/close-hangs branch 2 times, most recently from a90a56f to 28689f7 Compare July 22, 2026 12:51
@kalragauri
kalragauri requested a review from rajeevpodar July 23, 2026 04:08
@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.35760% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.29%. Comparing base (a9d896c) to head (85a4af4).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...d/storage/internal/async/writer_connection_impl.cc 93.75% 2 Missing ⚠️
...rage/internal/async/writer_connection_impl_test.cc 99.48% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##             main   #16270    +/-   ##
========================================
  Coverage   92.29%   92.29%            
========================================
  Files        2221     2221            
  Lines      207372   207619   +247     
========================================
+ Hits       191386   191629   +243     
- Misses      15986    15990     +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: storage Issues related to the Cloud Storage API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant