Rework slskd search + download reliability#212
Open
dammitjeff wants to merge 2 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds some reliability fixes and file matching that VASTLY improves download success, and improves auto-tagging for downloaded tracks! A 25 track playlist will have 25/25 tracks consistently being ripped now with the correct tags, unless the song literally does NOT exist on slskd!
slskd.go
•
searchStatuswas a recursive retry that bailed before slskd had even finished searching.SLSKD_RETRY=2is only 30s, but slskd routinely takes 60s+ to wrap up a search, so we were giving up on tracks that were about to return results. Instead of waiting around for a set time, it now polls on a deadline and actually wait for the search to reportIsComplete. This one change took me from ~9/25 tracks, to 12/25 downloading by itself.• Added
searchQueries, instead of searching one and done, it tries a few combinations in order:So now there's more than one chance to actually pick up a song, and I set them up so 99% of tracks get picked up by the end of the list.
•
GetTracknow retries the /responses endpoint a few times. It lags behind the search summary, so we were occasionally reading an empty response and marking a track dead right when the files were about to show up.CollectFilesis where most of the matching improvements are:• Artist matching is now token based, so a file credited to just one artist of a collab still matches (ex.
Moon - Daniel Caesar feat. Bon Ivernow matches a file likeDaniel Caesar - Mooninstead of getting skipped for not having both names)• Titles are now matched against the files base name, instead of the entire path, so a title word can't accidentally be marked as a successful grab incorrectly.
• Files from busy peers are kept as fallbacks now instead of being thrown away/skipped
monitor.go
•
monitor.goused to just skip a track the moment its download errored. Now it tries the next candidate source viaRetryDownloadand only gives up once there's literally NONE that match. One dead download no longer kills the whole track!downloader.go + metadata.go
• I merged the finalize path into
FinalizeDownloadindownloader.goso tags get written regardless if files are migrated. I saw some people having issues with that too.• Downloaded tracks have their albums grouped under the real lead artist now, so they don't end being imported as "Various Artists". That bug is effectively dead now :)
SO many changes I know, but after all of this, almost all of the playlists I tested ACTUALLY came back with 25/25 tracks, with their proper artist info tagged and everything.
heads up most of these metadata changes only apply with Auto-tag songs is enabled in settings! But EVERY playlist now benefits from these changes, LB, Apple Music AND Spotify. :) Lmk if I can answer any questions. Theres a second PR that will be pushed soon that improves tagging and searching for custom playlists soon. I just split these both up because otherwise it'd be a massive PR lol.