Skip to content

Optimize intensity matching#236

Open
minnerbe wants to merge 9 commits into
newsolverfrom
optimize-intensity-matching
Open

Optimize intensity matching#236
minnerbe wants to merge 9 commits into
newsolverfrom
optimize-intensity-matching

Conversation

@minnerbe

Copy link
Copy Markdown
Collaborator

This PR slightly shifts the level of caching from raw tiles to downscaled and filtered or even fully rendered tiles. The first one is a zero-tradeoff improvement: instead of the full-scale tile + mask, the downsampled versions are stored, reducing per-tile effort and cache pressure. Therefore, it is the new default. The second one trades memory for computational resources: the full rendered bounding box is stored, making the first render somewhat expensive, and the following ones essentially free. It should therefore be used cautiously (especially for ic3d processes, where the full tile needs to be rendered anyway).

@trautmane This introduces a hand-rolled cache that was modeled after ImageProcessorCache. If you think it's worth doing that, I'm happy to try to consolidate all the different caching mechanisms. Also, imglib2 images are used instead of ImageJ image processors to enable zero-copy views for the fully-cached implementation.

Previous state

IntensityMatcher rendered both tiles of every overlapping pair from scratch, on every call, via Render.render(...) into ImageJ FloatProcessor/ColorProcessor rasters sized to just the overlap box. The only cache in the path was a shared ImageProcessorCache of raw, full-resolution source images (pre-filter, pre-downsample, pre-mesh) — so filtering, downsampling, and mesh-mapping were all redone for every pair a tile appears in, even though a tile typically overlaps many same-layer and cross-layer neighbors.

Two options now

Rendering is factored into TileRenderer, split into two independent halves:

  • loadSource — pair-independent: load the full-res source, apply the tile's filter, downsample to its mipmap level → DownsampledSource.
  • renderBox — pair-dependent: transform the downsampled source into the requested world-space box (using a mesh transform) → RenderedTile (image / weight / coefficient-label rasters).

Two strategies decide what to cache between calls, selected by the new --cacheRenderedTiles flag:

TightBoxTileRenderer (default) CachedTileRenderer (--cacheRenderedTiles)
Caches DownsampledSource per tile full-tile RenderedTile (whole bounding box) per tile
Per-pair cost re-mesh the box every time zero-copy crop (Views.interval) of the cached render
Tradeoff less memory, more compute more memory, less compute

Both share a small generic LRU, TileCache<T>, keyed by tile spec and weight-bounded in kilobytes by the existing --maxPixelCacheGb (adapted from ImageProcessorCache's eviction logic). As before, same-layer and cross-layer matching share one renderer/cache when their render scales are equal.

New / modified classes

New:

  • TileRenderer — abstract base defining the loadSource / renderBox split and the shared mesh-mapping logic.
  • TightBoxTileRenderer — caches only the downsampled source; re-meshes each overlap box.
  • CachedTileRenderer — caches the full per-tile render; serves overlaps as zero-copy crops.
  • TileCache<T> — generic per-tile-id, kilobyte-bounded LRU cache used by both renderer strategies.
  • DownsampledSource — the cacheable pair-independent render result (filtered/downsampled image, mask, mipmap level).
  • RenderedTile — the image/weight/coefficient rasters as imglib2 views, replacing the previous raw ImageJ processors.

Modified:

  • IntensityMatcher — no longer renders directly; picks a same-/cross-layer TileRenderer and iterates the resulting imglib2 rasters in lockstep instead of indexing ImageJ processors by hand.
  • AffineIntensityCorrectionBlockWorker — replaced the single shared ImageProcessorCache with a kilobyte budget passed down to IntensityMatcher, which now owns its own per-scale TileCache instances.
  • FIBSEMIntensityCorrectionParameters — carries the new cacheRenderedTiles flag through to the matcher.
  • AlgorithmicIntensityAdjustParameters — adds the --cacheRenderedTiles CLI flag.

Validation and benchmarking

The three methods give exactly the same results, see results obtained with the:

Runtimes with the processes can be seen below, where 10 layers of the w61_s109 multi-sem stack were corrected with ic3d. Matching times were recorded, clustered into two clusters (roughly cached and uncached), and the median of both clusters is shown, as well as overall runtime and cache miss overhead. The first image shows the situation with an overly large cache (100GB), the second with a somewhat tight cache (1GB). It looks like 10GB should be sufficient for multi-sem stacks for all three methods. In the case where the cache is sufficiently large, the new methods speed up the process by ~3x and ~5x, respectively.

image image

FYI @StephanPreibisch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant