Skip to content

Rework slskd search + download reliability#212

Open
dammitjeff wants to merge 2 commits into
LumePart:devfrom
dammitjeff:feat/download-reliability
Open

Rework slskd search + download reliability#212
dammitjeff wants to merge 2 commits into
LumePart:devfrom
dammitjeff:feat/download-reliability

Conversation

@dammitjeff

Copy link
Copy Markdown
Contributor

This PR adds some reliability fixes and file matching that VASTLY improves download success, and improves auto-tagging for downloaded tracks! A 25 track playlist will have 25/25 tracks consistently being ripped now with the correct tags, unless the song literally does NOT exist on slskd!

slskd.go
searchStatus was a recursive retry that bailed before slskd had even finished searching. SLSKD_RETRY=2 is only 30s, but slskd routinely takes 60s+ to wrap up a search, so we were giving up on tracks that were about to return results. Instead of waiting around for a set time, it now polls on a deadline and actually wait for the search to report IsComplete. This one change took me from ~9/25 tracks, to 12/25 downloading by itself.
• Added searchQueries , instead of searching one and done, it tries a few combinations in order:

title - <artist>
title - <each individual artist>
title - wildcard artist
etc.

So now there's more than one chance to actually pick up a song, and I set them up so 99% of tracks get picked up by the end of the list.

GetTrack now retries the /responses endpoint a few times. It lags behind the search summary, so we were occasionally reading an empty response and marking a track dead right when the files were about to show up.

CollectFiles is where most of the matching improvements are:
• Artist matching is now token based, so a file credited to just one artist of a collab still matches (ex. Moon - Daniel Caesar feat. Bon Iver now matches a file like Daniel Caesar - Moon instead of getting skipped for not having both names)
• Titles are now matched against the files base name, instead of the entire path, so a title word can't accidentally be marked as a successful grab incorrectly.
• Files from busy peers are kept as fallbacks now instead of being thrown away/skipped

monitor.go
monitor.go used to just skip a track the moment its download errored. Now it tries the next candidate source via RetryDownload and only gives up once there's literally NONE that match. One dead download no longer kills the whole track!

downloader.go + metadata.go
• I merged the finalize path into FinalizeDownload in downloader.go so tags get written regardless if files are migrated. I saw some people having issues with that too.
• Downloaded tracks have their albums grouped under the real lead artist now, so they don't end being imported as "Various Artists". That bug is effectively dead now :)

SO many changes I know, but after all of this, almost all of the playlists I tested ACTUALLY came back with 25/25 tracks, with their proper artist info tagged and everything.

image

heads up most of these metadata changes only apply with Auto-tag songs is enabled in settings! But EVERY playlist now benefits from these changes, LB, Apple Music AND Spotify. :) Lmk if I can answer any questions. Theres a second PR that will be pushed soon that improves tagging and searching for custom playlists soon. I just split these both up because otherwise it'd be a massive PR lol.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant