fix(store,logger,scan): concurrency-adjacent races and correctness fixes#343
Open
TBX3D wants to merge 4 commits into
Open
fix(store,logger,scan): concurrency-adjacent races and correctness fixes#343TBX3D wants to merge 4 commits into
TBX3D wants to merge 4 commits into
Conversation
CreateFile/Write kept the '/' from a target's url path when building the log filename, so a target like http://host:port/ tried to open <dir>/host:port/.log whose parent directory never existed. os.OpenFile failed and scanTarget aborted the whole target before any scanning ran. fold every '/' and '\\' run in the sanitized url into a single '_' via a shared logPath helper so CreateFile and Write always resolve to the same flat file, and mkdir the parent defensively in getWriter as a second line of defense.
subdomaintakeover, cloudstorage, and the next.js framework detector printed their status/error lines with a bare fmt.Println straight to os.Stdout, bypassing the output package's sink. under -concurrency>1 only the sink is lock-wrapped, so these lines could interleave or tear against lines other targets write concurrently. swap them for output.ScanStart / output.Error, matching every other scanner in the package.
sanitize() folds every separator run (and a literal '_') to one '_', so distinct targets like https://a.com/x and https://a.com//x, or host:8443/path and host_8443_path, sanitized to the identical string and shared one snapshot file - the second target's Save silently clobbered the first's baseline. Save's os.WriteFile also wasn't atomic, so two targets racing on a collided path (or -concurrency>1 in general) could interleave partial writes. pathFor now appends 16 hex chars of the target's sha256 to the sanitized name, so distinct targets never collide, and Save writes through a temp file in the same dir followed by os.Rename so a reader always sees a complete snapshot or the previous one, never a partial write. this changes the on-disk snapshot filename scheme: existing users' -diff baselines under the old sanitize()-only names won't be found and the next run re-baselines from scratch (every finding reports as newly added once). noting this as a deliberate tradeoff for a local hardening pass rather than silently orphaning them.
github-actions
Bot
added
size/l
pr summary9 files changed (+379 -15)
|
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #343 +/- ##
=======================================
Coverage ? 54.79%
=======================================
Files ? 81
Lines ? 6908
Branches ? 0
=======================================
Hits ? 3785
Misses ? 2848
Partials ? 275 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
TBX3D
force-pushed
the
harden-concurrency-portable
branch
from
July 9, 2026 21:52
31b29c2 to
7c479ed
Compare
golangci-lint (errcheck) flagged the bare defer os.Remove(tmpPath) in writeFileAtomic; wrap it so the discard is explicit.
TBX3D
force-pushed
the
harden-concurrency-portable
branch
from
July 9, 2026 23:52
7c479ed to
f5388d8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
a handful of fixes surfaced while working on -concurrency: two targets
whose sanitized names collided (differing only in separator punctuation)
could silently overwrite each other's state snapshot, so the snapshot
path now appends a hash of the full, un-sanitized target to stay
injective, and Save writes through a temp file + rename so a reader never
observes a partially-written snapshot. logger.CreateFile/Write now flatten
a URL's path into the filename instead of keeping the '/' verbatim, which
previously tried (and failed) to open a file under a directory that was
never created. a couple of scan/module status lines that bypassed the
output sink via a raw fmt.Println are routed through it, since only the
sink is lock-wrapped under -concurrency>1.