fix(benchmark): use fsspec.open instead of shared IPC instance to avoid multiprocessing timeout

[yuxin00j]

Fix multiprocessing deadlock in open microbenchmark

Problem
The test_open_multi_process microbenchmark was timing out due to a deadlock. All 64 subprocesses were inadvertently sharing the same gRPC HTTP/2 connection instantiated in the parent process, which caused stream exhaustion and crashed the asyncio loop.

Changes
Swapped the shared gcs.open(...) call with with fsspec.open(...) in the _open_op worker process. This forces fsspec to dynamically resolve and cache a fresh, dedicated filesystem connection per worker process.

[gemini-code-assist]

Code Review

This pull request updates the _open_op function in test_open.py to use fsspec.open with a context manager instead of calling gcs.open directly. The review feedback recommends adding an explanatory comment inside the function to document why the gcs parameter is left unused, which will help prevent future maintainers from reverting this change and reintroducing multiprocessing timeouts.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.68%. Comparing base (e6c107b) to head (e43be0b).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #988   +/-   ##
=======================================
  Coverage   89.68%   89.68%           
=======================================
  Files          16       16           
  Lines        3579     3579           
=======================================
  Hits         3210     3210           
  Misses        369      369           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Yonghui-Lee]

Other multi process micro benchmarks also have this problem. The reason they did not deadlock as frequently as test_open_multi_process is primarily due to lower concurrency and longer-running operations which are less likely to overwhelm the connection pool or trigger stream exhaustion.

Maybe we can discard connection sharing for all multi-process benchmarks by dynamically instantiating the GCS filesystem inside the child process.