feat/rayon_parallel_wasm by hitchho · Pull Request #2658 · penumbra-zone/web

hitchho · 2025-11-29T17:20:42Z

adds rayon-based parallel proof generation for transaction building. enables multi-threaded wasm execution using wasm-bindgen-rayon, significantly improving transaction build times on multi-core systems. working prax demo / video / praxPR.

rayon parallel proof generation (`packages/wasm/crate`)

added build_parallel_native function that uses rayon's par_iter() to build all transaction actions concurrently in wasm. the parallel build leverages all available cpu cores through web workers spawned
by wasm-bindgen-rayon.

src/build.rs - new build_parallel_native entry point
src/lib.rs - feature-gated parallel module export
Cargo.toml - added rayon, wasm-bindgen-rayon dependencies with parallel feature

build scripts and config (`packages/wasm/crate`)

created build infrastructure for compiling wasm with atomics and shared memory support required for rayon threading.

scripts/build-wasm-parallel.sh - build script with proper rustflags
.cargo/config.toml - target-specific linker configuration
package.json - added compile:parallel script with --max-memory=4294967296 for 4gb heap

wasm initialization (`packages/wasm/src`)

added explicit initialization module that creates sharedarraybuffer-backed memory and initializes the rayon thread pool before use.

src/init.ts - initWasmWithParallel(numThreads) function
src/build.ts - re-exports parallel build functions

services integration (`packages/services`)

integrated parallel building into the view service with optimistic authorization flow and smooth progress animation using a logarithmic curve.

src/view-service/util/build-tx-parallel.ts - async generator with progress updates
src/offscreen-client.ts - buildParallelWithRayon rpc method
packages/types/src/internal-msg/offscreen.ts - message types for parallel builds

compatibility and fallback

automatic runtime detection with graceful fallback:

checks SharedArrayBuffer availability at runtime
if unavailable, falls back to original js worker build (1 worker per action)
chrome extensions always get rayon (privileged context)
web apps need COOP/COEP headers for rayon, otherwise fallback
no regression - fallback is the original working implementation

progress bar animation

two-phase smooth progress animation:

auth phase (5% → 20%): animates while user approves
build phase (20% → 90%): slower 3s curve for wasm init + proof generation
fixes race condition where fast builds would skip animation

Add wasm-bindgen-rayon integration for multi-threaded WASM execution. This enables parallel proving across CPU cores using rayon's thread pool. - Add rayon and wasm-bindgen-rayon dependencies behind 'parallel' feature - Export initThreadPool for rayon thread pool initialization - Add build_parallel_native function for parallel action proving - Requires SharedArrayBuffer and cross-origin isolation headers

- Add .cargo/config.toml with SIMD128 target features - Add build-wasm-parallel.sh for nightly toolchain builds - Add compile:parallel npm script with atomics and 4GB memory limit - Export wasm-parallel directory in package.json

- Add init.ts with initWasmWithParallel for SharedArrayBuffer setup - Create shared memory with 4GB max for rayon thread pool - Add buildWithRayon wrapper in build.ts for parallel proving - Support dynamic import to avoid bundling in restricted contexts

- Add buildParallelWithRayon to offscreen-client for worker communication - Add optimisticParallelBuild generator in build-tx-parallel.ts - Update build-tx.ts to use parallel path when SharedArrayBuffer available - Add ParallelBuildRequest/Response types for offscreen messaging - Default to parallel build, fallback to JS worker approach

two-phase progress animation: - auth phase (5% -> 20%): slow rise while user approves tx - build phase (20% -> 90%): slower 3s curve for wasm init + proof fixes race condition where build could complete before animation starts. minimum 500ms animation ensures users see smooth progress.

changeset-bot · 2025-11-29T17:20:45Z

⚠️ No Changeset found

Latest commit: 283437d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Add a comprehensive context document for expert review of PR #2658, which proposes adding Rayon-based WASM threading alongside the existing JavaScript-level parallelization for transaction building.

…dule)

With wasm-pack --target web, the WASM module needs explicit initialization before use. Updated all exported functions to call initWasm() automatically. - init.ts: initWasm() now actually initializes the regular wasm module - keys.ts: All functions now async with auto-init - address.ts: All functions now async with auto-init - build.ts: All functions now async with auto-init 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Updated test files and view-server.ts to handle async WASM functions. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Check that Address has inner bytes or alt_bech32m string before calling isControlledAddress, to avoid "Address proto has neither inner nor alt_bech32m fields set" error. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

to not break service worker for penumbra syncing block

Fix/wasm compile bundler

feat: add compute_effect_hash WASM function for airgap signing

Bumps next, @next/bundle-analyzer, eslint-config-next to 15.5.18 and regenerates the lockfile so this branch can't ship the vulnerable next@15.3.1 runtime. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

This reverts commit 8785019e7e2d. The lockfile regeneration silently failed on this branch due to a stale workspace structure (packages/storage depends on @penumbra-zone/wasm@workspace:*, which no longer exists in the workspace). The previous commit only updated package.json, leaving the lockfile at next@15.3.1 — recreating the exact mismatch we were trying to fix. Revert to keep the branch consistent. The Next.js bump on this branch needs a workspace fix first (or rebase onto patched main). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…ename The wasm package was renamed to @rotko/penumbra-wasm in commits 0f20b93 / 705b4a9 / 4b68bbd, but the 3 consumer packages (storage, services, perspective) plus 30 TS imports still asked for @penumbra-zone/wasm@workspace:*. pnpm therefore refused to install on this branch, which blocked any lockfile-touching security patch. This commit finishes the rename across consumers + TS imports + the wasm package's own JSDoc, so pnpm install resolves the workspace again. With the workspace healthy, also bumps apps/veil to next@15.5.18 (closing the App Router RCE, same as main) and regenerates the lockfile. Parallel wasm build and rotko-specific WASM functions are unaffected — this only changes the package identifier and consumer imports. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

hitchho added 6 commits November 29, 2025 12:41

build(wasm): add parallel build scripts and config

4b68bbd

- Add .cargo/config.toml with SIMD128 target features - Add build-wasm-parallel.sh for nightly toolchain builds - Add compile:parallel npm script with atomics and 4GB memory limit - Export wasm-parallel directory in package.json

build(wasm): add gitignore for parallel WASM output

afebc8b

hitchho mentioned this pull request Nov 29, 2025

feat: rayon parallel wasm prax-wallet/prax#404

Draft

hitchho and others added 14 commits December 3, 2025 23:00

fix: should work any chrome/chromium based

94e91d0

fix: use --target web for wasm compile (required for wasm_bindgen::mo…

9f1ea78

…dule)

fix(wasm): update tests and view-server for async functions

48c66c9

Updated test files and view-server.ts to handle async WASM functions. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

fix(types): update ViewServerInterface.isControlledAddress to async

2b15957

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

change main wasm compile to --target bundler

705b4a9

to not break service worker for penumbra syncing block

bump version rotko/penumbra-wasm -> 55.0.1

0f20b93

Merge pull request #1 from rotkonetworks/fix/wasm-compile-bundler

1527d64

Fix/wasm compile bundler

feat: add compute_effect_hash WASM function for airgap signing

49ee258

Merge pull request #2 from rotkonetworks/fix/wasm-compile-bundler

610bc98

feat: add compute_effect_hash WASM function for airgap signing

fix(security): bump next to 15.5.18 to close App Router RCE

8785019

Bumps next, @next/bundle-analyzer, eslint-config-next to 15.5.18 and regenerates the lockfile so this branch can't ship the vulnerable next@15.3.1 runtime. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat/rayon_parallel_wasm - #2658

feat/rayon_parallel_wasm#2658
hitchho wants to merge 20 commits into
penumbra-zone:mainfrom
rotkonetworks:feat/rayon-parallel-wasm

hitchho commented Nov 29, 2025 •

edited

Loading

Uh oh!

changeset-bot Bot commented Nov 29, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

hitchho commented Nov 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

rayon parallel proof generation (packages/wasm/crate)

build scripts and config (packages/wasm/crate)

wasm initialization (packages/wasm/src)

services integration (packages/services)

compatibility and fallback

progress bar animation

Uh oh!

changeset-bot Bot commented Nov 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

hitchho commented Nov 29, 2025 •

edited

Loading

rayon parallel proof generation (`packages/wasm/crate`)

build scripts and config (`packages/wasm/crate`)

wasm initialization (`packages/wasm/src`)

services integration (`packages/services`)

changeset-bot Bot commented Nov 29, 2025 •

edited

Loading