- add a shared shell client and root /status query
- attach shell status to the TanStack root context for React routes
- keep the shell bridge types and test setup aligned with the new status data
- make listening status prefetch optional so /stats still renders on status failures
- normalize no-stats-yet cache responses into the existing empty stats state
- restore streaming fallback when library track resolution errors
- add route and API regression coverage for the review fixes
- mark stats as a completed React-owned route in the migration overview
- capture the stats migration outcome and cleanup status in its route plan
- add guidance for future migrations to watch for shared UI reuse opportunities
- replace Cell-based pie slice coloring with data-driven fill props
- keep the stats genre and database charts visually unchanged
- remove the deprecation warning from the stats route
- rename the Playwright smoke suite to reflect shell-wide route coverage
- share nav highlight expectations through the route manifest helpers
- cover React route activation and canonical stats URL behavior
- remove the stats page timeout and library pre-navigation hack
- hand artist detail opening directly to the legacy shell bridge
- add a route test covering the direct artist navigation bridge call
- move stats route legacy handoffs onto explicit SoulSyncWebShellBridge methods\n- stop relying on ad hoc window globals from React code for artist navigation and playback\n- update shell bridge tests and route test doubles to enforce the expanded bridge contract
- delete the old stats page HTML, JS, and CSS now that the React route owns the experience
- preserve helper/tour selectors by exposing the legacy stats ids from the React page
- move shared track playback fallback into library code
- move the stats route onto the React shell with Recharts-based visualizations
- remove the global Chart.js include and add a local stats seed script for easier testing
- keep parity coverage with route, API, and helper tests while preserving the legacy page layout
Keep WebUI migration plans next to the frontend code so route work
has one predictable home.
Standardize the set around one page migration overview plus
per-route migration plan docs for future tasks.
- convert the sidebar nav to real links with URL-driven state
- intercept left-clicks so internal navigation stays in-app while preserving native browser link behavior
- keep artist-detail transitions param-aware and update route tests
Three related improvements to the now-playing media player and the
"add to wishlist" / "download missing" modals.
1. Play buttons across track-list modals
Every track row in the download-missing modals (Spotify, Tidal,
YouTube, services, artist album, wishlist download-missing) and
the add-to-wishlist modal now carries a play button. Click runs
playTrackFromLibraryOrStream:
- If the track has a local file_path → playLibraryTrack
- Else POST /api/stats/resolve-track to find it in the library
by title + artist → playLibraryTrack
- Else fall back to _gsPlayTrack streaming
Backend ownership response gains track_id / title / file_path so
the wishlist modal's owned tracks can hand the right metadata
to the player without an extra round trip.
The add-to-wishlist modal previously showed the play button only
on owned tracks; now the button is unconditional so the streaming
fallback can take over for unowned ones (matches the standard
pattern from the rest of the app).
2. Clean media-player display titles
YouTube / Tidal / Qobuz / torrent / usenet plugins encode their
source-side identifier into the filename field as
<source_id>||<display> so download() can recover it later. The
media player's track-title renderer never knew about this
convention and showed strings like
"wvgFsXoGFnQ||Sometimes I Cry When I'm Alone" verbatim in the
now-playing UI. extractTrackTitle and setTrackInfo now strip the
<id>|| prefix defensively so any path into the player gets a
clean display.
Local library playback also fetches canonical metadata from
/api/stats/resolve-track when track.id is present so title /
artist / album / album art come straight from the SoulSync DB
instead of whatever the caller passed in. Falls back silently
to caller values on any error so playback never blocks on the
metadata fetch.
3. Lyrics panel + View Artist close
New collapsed lyrics panel between the playback controls and
queue panel. POST /api/lyrics/fetch (new backend endpoint)
prefers the local .lrc / .txt sidecar files SoulSync writes
during post-processing so downloaded tracks resolve lyrics with
zero network hits; falls back to LRClib exact-match (when album
+ duration are available) then to LRClib search.
Synced LRC results are parsed (handles multi-stamp lines for
repeated choruses), and the active line highlights + smooth-
scrolls into the middle of the viewport on every audio
timeupdate. Plain-text results render without highlighting.
Per-track cache prevents re-fetching when the user revisits the
same track. Lyrics fetch is fire-and-forget — failure shows
"No lyrics found" without ever blocking playback.
View Artist on the expanded player now calls
closeNowPlayingModal before navigating; the modal was previously
sitting open over the artist page, hiding it. Handler is bound
once and is a no-op when no artist_id is attached.
CSS additions are additive (new .modal-track-play-btn and
.np-lyrics-* rules); no existing styles touched. Backend endpoint
returns 200-with-success-false on any miss so callers can render
"no lyrics" without treating it as an error.
WHATS_NEW updated under 2.5.9 with two entries (lyrics + View
Artist close).
Reset profile PIN dialog controls each time it opens so stale profile-specific event listeners cannot submit an admin PIN against a previously selected profile.
Keep failed PIN attempts retryable and restrict launch-lock verification to the admin profile PIN only, so non-admin profile PINs cannot mark the admin lock as verified.
Teach the orphan file detector to match tracked files by both track artist and album artist. This prevents Picard-style albumartist/album (year)/track layouts from being reported as orphans when the DB track artist differs from the album artist.
Also check file albumartist tags during fallback matching and add a regression test for the reported Picard folder layout.
Update Import Music album and queue artwork fallbacks to use the shipped /static/placeholder-album.png asset instead of the nonexistent /static/placeholder.png path.
Replace the remaining static UI fallback to the missing placeholder path and add a regression test that fails if static JS references it again.
Refresh registry-backed download plugins when settings are saved so cached Prowlarr clients pick up new indexer credentials immediately. This preserves active download state by reloading existing plugin instances instead of rebuilding the registry.
Add regression coverage for orchestrator reload fanout and the Usenet plugin's cached ProwlarrClient refresh path.
Add a debug log in the exception handler that occurs while inferring legacy HiFi track manifest extensions. Previously exceptions were silently ignored; this change records the error message via logger.debug to aid debugging without changing behavior.
Update project version and release notes for 2.5.9. Changes: update .github/workflows/docker-publish.yml default/version_tag prompt to 2.5.9, bump _SOULSYNC_BASE_VERSION in web_server.py to 2.5.9, and replace the WHATS_NEW entry in webui/static/helper.js with detailed 2.5.9 release notes and a new VERSION_MODAL_SECTIONS entry. Also update the helper.js fallback for latest whats-new version to 2.5.9.
Add fallback support for public hifi-api instances that expose playback through /track/ instead of /trackManifests/. The capability checker now accepts either manifest shape, and downloads can use direct URLs decoded from the legacy base64 manifest.
Tests cover legacy instance capability detection and download-manifest fallback while preserving the newer trackManifests path.
Probe public HiFi instances with the same trackManifests endpoint used by real downloads instead of the legacy /track endpoint. This prevents compatible instances from being falsely labeled search-only in Settings.
Centralize HiFi instance capability checks in HiFiClient and reuse manifest URI parsing with the download path.
Tests cover manifest-based capability detection, no legacy /track probe, and limited instances without a manifest URI.
Keep full refresh moving when post-clear VACUUM hits a transient disk I/O error, and retry clear_server_data once when the clear step itself sees the same transient SQLite failure.
Retry metadata cache maintenance writes once on transient disk I/O errors so first-attempt cache jobs do not fail when an immediate retry would succeed.
Tests cover best-effort VACUUM, clear retry behavior, and cache maintenance retry behavior.
Ensure upgraded databases have the tracks.file_size and albums.api_track_count columns after all legacy migrations run. Add defensive repair paths for Jellyfin track imports and album track-count caching so stale schemas self-heal instead of dropping full-refresh track imports.
Tests cover legacy schema repair and api_track_count self-repair.
Reported bug: filling Jamiroquai's "Light Years" single pulled in
Gut's "Light Years" album tracks (different artist, completely
different genre — track titles like "Wound Fuck" and "Eat My Cum"
made the contamination obvious). The Album Completeness auto-fill
was the only file-copying path with a loose 0.50 SequenceMatcher
artist gate, which let unrelated candidates through whenever the
title matched well.
Two-stage defense now sits on the only album-fill code path
(_fix_incomplete_album in core/repair_worker.py):
- Stage 1 — _album_fill_target_artist_allows_track. Pre-search
gate: before doing any library lookup for a missing track,
refuse to operate if the missing track's source artist(s)
don't match the target album's artist. Compilation albums
(album_artist in {'various artists', 'various', 'soundtrack'})
bypass the gate so legitimate VA releases still work. Empty
source-artist metadata also bypasses for backward compat with
older missing-track records that don't carry per-track artist.
- Stage 2 — _album_fill_artist_names_match. Replaces the old
0.50 SequenceMatcher with an alias-aware 0.82 threshold that
uses core.matching.artist_aliases when available (handles
diacritic variants like Beyoncé/Beyonce and known stage names)
with a normalized-similarity fallback if the aliases module
isn't importable. Skipped candidates are logged at debug so a
later support ticket can show what was rejected and why.
Tests in tests/test_repair_worker_album_fill.py reproduce the
exact reported scenario: target album "Light Years" by Gut +
missing track from a Jamiroquai source → skipped with a logged
warning, no copy attempted, wishlist not poisoned. Second test
covers Stage 2 directly with a wrong-artist library candidate.
Existing test_perform_album_fill_copy_branch still passes.
Note: this fix prevents NEW cross-artist contamination via
Album Completeness. It does not clean up the data anomaly that
made Gut's library entry appear to have a "Light Years" album
in the first place — that's a separate data-quality issue worth
investigating if it recurs.
Adds the PR description for the torrent/usenet release-staging feature and updates drifted tests for the new plugin registry entries and search exclude_sources signature.
Verified with the focused pytest command covering cancellation and default registry source registration.
Clarifies album-bundle progress text in the download modal and active downloads panel so release-first downloads read as downloading a release, then matching tracks after staging.
Adds waiting-state copy and tooltips for rows blocked on release staging, plus source-specific library history badge styling for Torrent, Usenet, Staging, and Auto-Import.
Adds torrent/usenet as release-oriented download sources with album-bundle staging, live progress reporting, and post-processing that selects the requested audio file from completed releases instead of blindly importing the first file.
Keeps album-bundle behavior gated to single-source torrent/usenet album downloads, excludes release sources from hybrid album per-track searches, and allows hybrid non-album tracks to use release results safely.
Improves staged-release matching for featured/bonus track filenames while preserving version mismatches, records torrent/usenet provenance in library history, and updates service/status UI labels.
Covers the flow with focused lifecycle, status, staging, validation, task worker, post-processing, and import side-effect tests.
Route torrent and Usenet album bundles through private per-batch staging so Auto-Import cannot race public staging or duplicate imports.
Expose album-bundle progress in batch status and render it on the Downloads page while the external client is still downloading.
Tighten release handoff safety by rejecting archive path traversal, ignoring torrent candidates without a usable URL, and skipping Soulseek source reuse for torrent/Usenet batches.
Tests: .venv/bin/python -m pytest tests/downloads/test_downloads_status.py tests/test_album_bundle_dispatch.py tests/downloads/test_downloads_staging.py tests/test_torrent_usenet_plugins.py
Wraps up the code-review refactor pass.
- config/settings.py: ``download_source`` defaults gain
``album_bundle_poll_interval_seconds`` (default 2s) and
``album_bundle_timeout_seconds`` (default 6h, was a hard-coded
``6 * 60 * 60`` magic constant in torrent.py). The plugin reads
these via ``album_bundle.get_poll_interval`` /
``get_poll_timeout`` with safe fallback to the defaults when the
config value is missing / non-numeric. ``mode`` doc-comment
extended to list ``torrent`` and ``usenet``.
- core/downloads/validation.py: comment block above the album-name
fallback rewritten to document when the fallback actually runs
now — single-track hybrid downloads only, because the album-
bundle gate handles single-source mode and the hybrid chain
filter strips torrent / usenet from album batches. Code path
unchanged; just clarifies the contract for the next reader.
- webui/static/helper.js: WHATS_NEW entry summarising the refactor
pass (helper extraction, dispatch lift, staging deps injection,
atomic copy, configurable timeout, test additions).
The /loop of: extract → inject → test was sweep enough to drop the
gate code's coupling to 2-3 modules and put 49 unit tests behind
the new boundaries. Code-review feedback addressed:
1. album_bundle.py extracted ✓
2. Dispatch lifted out of master.py ✓
3. staging.py decoupled from runtime_state ✓
4. Validation fallback scope documented ✓
5. Poll timeout config-driven ✓
6. ``amazon`` provenance owned in a prior commit ✓
7. End-to-end-shaped tests added (test_album_bundle_dispatch.py)
8. Auto-Import race closed via atomic copy ✓
Per code review: the album-bundle provenance override added in an
earlier commit reached into ``core.runtime_state.download_batches``
directly from inside the staging matcher. Sibling modules
shouldn't import each other's globals — the existing StagingDeps
pattern is the canonical way to inject everything else this helper
needs.
- core/downloads/staging.py: new optional ``get_batch_field``
callable on ``StagingDeps`` (defaults to None for backward compat
with any caller that doesn't know about it yet). The inline
``from core.runtime_state import download_batches`` is gone; the
helper now calls ``deps.get_batch_field(batch_id,
'album_bundle_source')`` and falls back to 'staging' when None
is returned. Accessor exceptions are swallowed with a debug log
so a deleted batch mid-process can't break the staging match.
- web_server.py: ``_build_staging_deps`` injects a small
``_staging_get_batch_field`` helper that wraps the tasks_lock +
download_batches dict access. Centralises the lock semantics in
one place — the staging module no longer needs to know about
the lock or the dict.
- tests/test_staging_album_provenance.py: 5 new tests covering the
full matrix — torrent override applied, usenet override applied,
no override falls back to 'staging', missing accessor (default
None) falls back to 'staging', accessor raising falls back to
'staging'. Each test seeds + cleans a synthetic task in
runtime_state so the test doesn't bleed state across the suite.
Per code review: ~90 lines of inline gate logic in
``run_full_missing_tracks_process`` was inflating an already-580-
line worker function and was non-testable in isolation. Lifted to
``core/downloads/album_bundle_dispatch.py`` with two entry points:
- ``is_eligible(mode, is_album, album_name, artist_name)`` — pure
predicate, no side effects, easy to assert against. Splits the
gate decision from the resolution + run step so tests can pin
the gate semantics without standing up a plugin.
- ``try_dispatch(...)`` — full flow. Returns True iff the master
worker should stop (gate fired + failed); False = engaged-and-
succeeded OR didn't engage, both fall through to per-track.
State access is now decoupled from ``runtime_state``:
- New ``BatchStateAccess`` Protocol with two methods
(``update_fields``, ``mark_failed``).
- Concrete impl ``_BatchStateAccessImpl`` lives in master.py and
wraps the tasks_lock + dict ops the original inline code did.
- Injected via parameter so the dispatch module never imports
``download_batches`` / ``tasks_lock`` directly.
Same goes for the plugin resolver and config getter — both
injected, so the dispatcher works against any orchestrator /
config implementation (including the in-test fakes).
Behavior unchanged. The master worker call site is now 11 lines
of boilerplate instead of 90 lines of inline conditional. Plugin
contract (``download_album_to_staging`` return dict shape)
unchanged.
- core/downloads/album_bundle_dispatch.py: new module owning the
gate + execution. ~150 lines including docstrings and the
Protocol definition.
- core/downloads/master.py: gate call site shrunk to a single
``if _album_bundle_dispatch.try_dispatch(...): return``. New
``_BatchStateAccessImpl`` class implements the Protocol against
the existing ``download_batches`` dict + ``tasks_lock`` so the
dispatcher gets injected access instead of importing them.
- tests/test_album_bundle_dispatch.py: 16 new tests covering the
pure predicate (album-required, mode allowlist, name validation,
case insensitivity), the resolver-failure fall-through
(plugin missing, plugin lacks method, resolver raises), the
success path returning False so per-track flows, the failure
path returning True with state.mark_failed called, plugin-raise
treated as a normal failure, whitespace stripping on names,
and progress-callback mirroring lifecycle events into batch
state.
Per code review: the album-bundle helpers (release picker + staging
collision suffix) were defined as private symbols in torrent.py and
imported by usenet.py through ``from core.download_plugins.torrent
import _pick_best_album_release, _unique_staging_path``. Sibling
plugins shouldn't reach into each other's private surface — leaky
module boundary, and the underscore prefix says don't import.
Also addressed two latent issues at the same time:
- The Auto-Import sweep race: my plugin copied audio files into
staging via plain ``shutil.copy2``, which exposes a partial file
at the audio extension for the duration of the copy. The Auto-
Import worker filters by audio extension when scanning Staging
(AUDIO_EXTENSIONS in core/auto_import_worker.py), so a mid-flight
scan could pick up a truncated file. Fix: copy to a
``.tmp.<random>`` sidecar first, then atomically rename via
``Path.replace`` (which is ``os.replace`` — atomic on the same
filesystem). Auto-Import sees the file either at its final name
or not at all.
- The 6-hour poll timeout was a hard-coded magic constant. Users
with slow private trackers or large box sets would silently time
out after 6h. Both the timeout and the poll interval are now
read from config (``download_source.album_bundle_timeout_seconds``
/ ``..._poll_interval_seconds``) with safe fallback to the
existing defaults when unset / non-numeric.
- core/download_plugins/album_bundle.py: new module owns the
shared surface — ``pick_best_album_release`` (with quality_guess
passed in as a parameter to avoid the circular import that would
result from this module trying to know about torrent.py's title
parser), ``unique_staging_path``, ``atomic_copy_to_staging``,
``copy_audio_files_atomically``, ``get_poll_interval``,
``get_poll_timeout``. Module-level size constants and quality
weights live here too. Usenet's grabs-as-popularity-proxy is
built into the picker so both plugins get the right behavior
without divergent local logic.
- core/download_plugins/torrent.py: drops the local helpers + the
hard-coded poll constants, imports from album_bundle. Per-track
download flow still uses module-level ``_POLL_TIMEOUT_SECONDS``
/ ``_POLL_INTERVAL_SECONDS`` aliases (read from config once at
import time, same as before from a per-track perspective).
- core/download_plugins/usenet.py: drops the imports of the
torrent.py private helpers; everything goes through album_bundle
now. Stops the cross-plugin private-import leak that started
this whole refactor.
- tests/test_album_bundle.py: 23 new tests covering the picker
heuristic (empty input, singleton drop, FLAC preference, grabs
fallback for usenet, size-floor / ceiling boundaries), the
collision-suffix logic, the atomic-copy invariant (concurrent
scanner thread asserts it never observes a partial audio file
during five sequential copies), the failure-skip behavior of the
batch copier, and the config-driven poll cadence including
garbage-input fallback.
- tests/test_torrent_usenet_plugins.py: existing picker tests
updated to call the new module-level helpers instead of the
former torrent.py privates.
When a user picks Hybrid mode AND downloads an album, the per-track
search loop fires once per track. Torrent / usenet are release-level
sources — Prowlarr returns album torrents, none of which score
meaningfully against an individual track title. Without filtering,
every track triggered a redundant Prowlarr search, qBit rejected
duplicate hashes after the first, and the run only worked at all
because Auto-Import swept Staging behind the scenes. Confusing
logs, wasted searches, brittle timing.
Fix: thread an optional ``exclude_sources`` parameter through
``DownloadOrchestrator.search``. When the per-track worker detects
that the active batch is an album AND mode is hybrid, it passes
``['torrent', 'usenet']`` so the hybrid chain skips them and falls
through to per-track-compatible sources (Soulseek / streaming).
Gate is narrow on purpose:
- Hybrid + album → skip torrent / usenet (THIS fix)
- Single-source torrent / usenet + album → album-bundle flow on
the master worker (already shipped)
- Hybrid + single-track batch (basic search / wishlist / playlist
of singles) → torrent / usenet still tried, validation.py's
album-name fallback gives them a shot
Excluded list logged at INFO when applied so the behavior is
visible in logs ("Hybrid search: excluding ['torrent', 'usenet']
for this query"). Default ``exclude_sources=None`` keeps every
non-task-worker caller (basic search, stream search, search-and-
download-best, automation handlers) on the original code path.
The download history modal was tagging every torrent / usenet
album-bundle download as 'Soulseek FLAC 24bit' because:
- core/imports/side_effects.py's source_service dict didn't have
entries for 'staging', 'torrent', or 'usenet' usernames. The
staging matcher in core/downloads/staging.py sets
download_tasks[task_id]['username'] = 'staging', which fell
through to the dict's default and got recorded as 'soulseek'
in the track download provenance row. Same fate for any
amazon or other source that wasn't whitelisted.
- The album-bundle flow specifically wants to be labeled as
'torrent' or 'usenet' (where the bytes actually came from),
not 'staging' (the intermediate). The plugin already stashes
the source on the batch state as ``album_bundle_source`` for
the Downloads-page status card; provenance recording can
read the same field.
Fixes:
- core/downloads/staging.py: when marking a task post_processing
after a staging match, check the batch's album_bundle_source
override and use that for username instead of 'staging' when
set. Falls back to 'staging' when no override exists
(manual file-drop case).
- core/imports/side_effects.py: source_service map gets entries
for 'staging', 'torrent', 'usenet', and the previously-missing
'amazon' (which was also falling through to 'soulseek').
- webui/static/library.js: the redownload modal's serviceLabels
/ serviceIcons dicts extended to cover lidarr, amazon,
soundcloud, auto_import, staging, torrent, usenet so badges
render the correct name instead of either the raw source_service
string or no badge at all.
- webui/static/wishlist-tools.js: history-source-chip color
palette extended for the new source labels (Torrent sky-blue,
Usenet violet, Staging / Auto-Import neutral grey).
Note: existing tracks in the DB still carry the wrong 'soulseek'
label — only NEW downloads after this fix get the right label.
A future migration could rewrite historical rows but it's
cosmetic and the underlying audio + metadata are correct.
Fixes the core architectural mismatch between indexer-based sources
and the per-track search-and-pick contract every other download
plugin satisfies. Prowlarr returns release-level torrents and NZBs;
searching for "Luther (with SZA)" against the GNX album torrent
scores near-zero on track-title similarity. Per-track candidate
validation rejects every result, every track in the batch flips
to not_found. The album-name fallback added in an earlier commit
papers over it for some cases but doesn't fix the fundamental
behavior: the user wanted the whole album.
New album-bundle flow does what the user actually wanted:
1. Gate fires inside core/downloads/master.py BEFORE the per-track
analysis loop, strictly when the batch has an album context AND
download_source.mode is 'torrent' or 'usenet' (single-source —
hybrid stays per-track to preserve fallback to Soulseek / etc.).
2. Plugin's new download_album_to_staging method searches Prowlarr
ONCE for the album as a whole ('<artist> <album>'), filters to
the right protocol, runs results through _pick_best_album_release.
3. Picker prefers seeded FLAC over low-seeded MP3, drops single-
track torrents that snuck in via the 40 MB size floor (single
tracks are typically ~10 MB), falls back to most-seeded when
every candidate is below the floor.
4. Picked release goes to the active adapter (qBit / Transmission /
Deluge for torrent; SAB / NZBGet for usenet). Polls until
complete with progress mirrored into the batch state so the
Downloads page can show meaningful status.
5. On completion the existing archive_pipeline walks the save dir
(extracting archives if any), every audio file gets copied into
the staging folder via _unique_staging_path so concurrent batches
don't collide.
6. Gate exits, master worker continues into the normal per-track
flow. Each track task hits try_staging_match early in the worker
and finds its file by fuzzy title match — no Prowlarr search
ever fires per-track, no candidate rejection, files flow through
the existing post-processing pipeline (tags, AcoustID, library
import).
Gate is strictly opt-in. Three orthogonal conditions must all hold:
batch_is_album, mode in ('torrent', 'usenet'), and the plugin must
expose download_album_to_staging. Any other source / hybrid mode /
non-album batch flows through the master worker unchanged. The
existing per-track torrent path still works for basic-search
single-track grabs.
- core/download_plugins/torrent.py: download_album_to_staging plus
_pick_best_album_release and _unique_staging_path helpers (shared
with the usenet plugin). _poll_album_download mirrors the existing
poll loop with progress callback emission.
- core/download_plugins/usenet.py: parallel implementation reusing
the picker + staging helpers. Different state set ('failed' vs
'error') from the usenet adapter contract.
- core/downloads/master.py: ~90-line gate right after batch context
loading. Mirrors plugin lifecycle into batch state under
``album_bundle_*`` keys so the Downloads page can render progress
while the torrent/usenet job runs (per-track tasks don't exist
yet during this phase). Failed bundle download fails the batch
with a meaningful error; missing plugin / context falls back to
the per-track flow with a warning.
- tests/test_torrent_usenet_plugins.py: 5 new tests pinning the
album picker preferences (FLAC over MP3 with comparable size +
better seeders, size floor drops singles, fallback when all
small), staging-path collision suffix, and the not-configured
short-circuit.
Live-test bug: Spotify-flow downloads with Torrent Only as the
active source produced 'download_failed' for every track. Searches
hit Prowlarr fine but no candidate ever got picked. Root cause was
in core/downloads/validation.py's get_valid_candidates:
- The streaming-source allowlist for the structured-metadata path
didn't include 'torrent' / 'usenet', so torrent results fell into
the Soulseek matching branch.
- Soulseek matching parses ``candidate.filename`` as a slskd-style
``Artist/Album/Track.flac`` path. Torrent / usenet filenames are
encoded as ``<download_url>||<display_name>`` so the orchestrator
can recover the URL — splitting that string on slashes produced
garbage path segments that never matched the expected artist,
every candidate failed the artist-folder gate, returned [], track
status flipped to 'not_found'.
Fixes:
- _streaming_sources now includes 'torrent' and 'usenet'. They take
the structured-metadata scoring path that reads r.title / r.artist
directly (the projection layer pre-fills both correctly).
- Artist gate skipped for torrent/usenet, same as YouTube. Album-
level releases legitimately don't expose per-track artist — the
projection falls back to the indexer name as the 'artist' field,
which would otherwise fail the gate against every Spotify artist.
- New album-name fallback scoring: for torrent/usenet only, the
candidate title is ALSO scored against the wanted track's
spotify_track.album field, and the max of (track-title score,
album-title score) wins. This makes a candidate titled
"GNX (2024) [FLAC]" match every track on the GNX album rather
than scoring near zero against a specific track title like
"Luther (with SZA)". match_type 'album_release' for visibility.
All 9 existing validation tests still pass.
Live testing surfaced: every download attempt failed with
'Torrent client refused the URL', but qBit was actually accepting
the add request fine. The bug was in our hash-lookup strategy.
qBittorrent's /api/v2/torrents/add returns 200 'Ok.' regardless
of whether the URL was actually valid / accepted / registered.
The previous code then queried /torrents/info?category=soulsync
to find the just-added torrent — but qBit hadn't categorised
the new torrent yet on the first poll, AND a fresh install has
no 'soulsync' category configured, so the lookup returned empty
and the adapter reported failure for every working download.
New strategy:
- Snapshot every torrent hash qBit currently tracks BEFORE
posting to /add.
- POST /add, accept its (uninformative) 200 OK.
- Poll the all-torrents list for up to 5 seconds, looking for a
hash that wasn't present in the before-snapshot. First new
hash wins.
The diff strategy works the same for /add with urls= (HTTP URL /
magnet) and /add with files= (raw .torrent upload), so both
paths now share the same _all_hashes + _poll_for_new_hash
helpers. Adds a warning log when qBit returns an unexpected
body and an error log when no new hash appears (so future
investigation has breadcrumbs).
Real-world test surfaced the bug — torrent results displayed
'by download?apikey=c15d6f69...&link=...' as the uploader / artist
in the basic search UI. The cause is TrackResult.__post_init__:
when artist is None it runs parse_filename_metadata on the bare
filename, and our filename starts with the indexer's download URL
(needed so download() can recover the URL later). The auto-parser
treats the URL as 'artist' and ships it to the UI.
Fix:
- core/download_plugins/torrent.py: new _parse_release_title()
splits 'Artist - Title' / 'Artist - Album' out of the release
title and strips trailing [FLAC] / (2016) tags. Falls back to
('', cleaned_title) when no dash is found, and explicitly
rejects URL-looking strings as an extra defence. The projection
pre-fills both artist and title on TrackResult, so __post_init__
skips the auto-parse entirely. When the release title has no
dash, artist defaults to the indexer name so the UI shows
'by Indexer' instead of a URL.
- core/download_plugins/usenet.py: imports the new helper and
applies the same fix.
- tests/test_torrent_usenet_plugins.py: 5 tests for the new
helper (dash split, trailing-tag stripping, no-dash fallback,
multiple-dash preservation, URL-prefix rejection). Existing
projection tests updated to assert artist + title come through
parsed correctly, plus a new test pinning the indexer-name
fallback for titles without a dash so the URL-leak regression
can't return.
Refines the filesystem-access guidance after realising the
simplest setup is to skip the per-protocol folder split entirely
— point Soulseek + qBit + SAB / NZBGet at the same download
folder and SoulSync reads one place.
- webui/index.html: warning card tone shifted from 'this is a
caveat' to 'here's the easiest fix' — leads with the single-
folder recommendation, demotes the per-protocol mount option
to a fallback. Icon swapped from ⚠️ to 💡 to match the
shifted framing.
- docker-compose.yml: comment block restructured. EASIEST SETUP
now leads (reuse the existing ./downloads mount, point every
client there). SEPARATE FOLDERS demoted to a second option
with the same commented placeholders for users who want them.
Torrent and usenet clients each download to their own folders
(not Soulseek's). SoulSync needs read access to those paths to
import the resulting files. Bare-metal setups work without
configuration; Docker setups need volume mounts; remote
downloader hosts need a network mount.
- webui/index.html: orange warning card on the Indexers &
Downloaders hero, listing the three deployment shapes
(bare-metal / Docker / remote) and what each needs.
- webui/static/style.css: ind-hero-warning rule set —
warning-tone palette (amber on dark glass) so the card
reads as advisory, not destructive. Inline ul + code
styling for the bullet list inside.
- docker-compose.yml: commented placeholder mounts under the
existing IMPORTANT block for /downloads/torrents and
/downloads/usenet. Same uncomment-and-edit pattern as the
existing slskd helper block. Documents the in-container path
must match what the torrent / usenet client reports as its
save_path.
The payoff for the previous five commits. Two new download
sources slot into the existing DownloadSourcePlugin contract,
backed by Prowlarr (search) + the torrent or usenet client
adapter (transfer) + archive_pipeline (post-extract walk). They
appear in the Download Source dropdown next to Soulseek / Tidal /
Lidarr / etc. and also participate in hybrid mode.
Pipeline (both plugins, mirror shape):
1. search(query) → ProwlarrClient.search filtered to the right
protocol, projected into TrackResult / AlbumResult shapes the
existing search UI already speaks. Filename field encodes the
indexer's download URL (or magnet URI for torrents) so
download() can recover it later.
2. download() → decodes URL, hands it to the active adapter
(qBittorrent / Transmission / Deluge for torrent; SABnzbd /
NZBGet for usenet), spawns a background poll thread that
tracks progress + reports the adapter-reported save_path.
3. On 'seeding' / 'completed' → archive_pipeline walks the save
directory, extracts any archives the downloader didn't
already unpack, picks the first audio file as the canonical
file_path. Matches the Lidarr client's single-track-pick
contract — picking which specific track to import happens in
post-processing.
- core/download_plugins/torrent.py: TorrentDownloadPlugin +
module-level helpers (_decode_filename, _guess_quality_from_title,
_parse_indexer_id_filter, _adapter_state_to_display, _row_to_status).
Uses get_active_torrent_adapter() so a settings change to the
client type takes effect without restart.
- core/download_plugins/usenet.py: UsenetDownloadPlugin —
parallel shape, reuses the torrent module's helpers. Different
enough states (no seeding, no magnet) to warrant its own class
but cheap to keep in lockstep.
- core/download_plugins/registry.py: register 'torrent' and
'usenet' plugins. Per the registry docstring this is the only
wiring point needed — the orchestrator picks them up
automatically via the iteration helpers.
- webui/index.html: 'Torrent Only (via Prowlarr)' + 'Usenet Only
(via Prowlarr)' added to the Download Source dropdown. New
redirect card (#prowlarr-source-redirect) explains that the
actual config lives on the Indexers & Downloaders tab —
shown whenever torrent or usenet is in the active source set.
- webui/static/settings.js: HYBRID_SOURCES gets two new entries
so hybrid mode can pick them up. updateDownloadSourceUI now
toggles the redirect card based on active sources.
- tests/test_torrent_usenet_plugins.py: 23 tests covering pure
helpers (filename encode/decode round-trip incl. magnet URIs,
quality guesser, state mapping), search projection logic
(protocol filter, drops without URLs, magnet-preferred-over-URL,
filename encoding, neutralised soulseek-specific score fields),
is_configured (both prowlarr + adapter required), finalize
(picks first audio file, errors on empty dir / missing save_path),
clear/get_all lifecycle, DownloadSourcePlugin protocol
conformance, and registry membership.
Shared helper the upcoming torrent and usenet download plugins
both compose against. Narrow surface — no matching, no tagging,
no library import. Just walks audio files and extracts archives
when needed.
Why a separate module: usenet downloaders (SABnzbd, NZBGet)
already auto-extract by default, and Lidarr's import pipeline
extracts before SoulSync sees the files. The only client that
sometimes leaves an archive behind is a torrent client when the
album was packed as a .rar — most music torrents ship loose but
not all. Centralising the walk + extract logic means both new
plugins can do the same thing, and a future direct-archive source
(zip download from a private site, etc.) plugs in for free.
- core/archive_pipeline.py:
- AUDIO_EXTENSIONS / ARCHIVE_EXTENSIONS constants (audio set
matches core/imports/file_ops.py quality_tiers).
- is_archive(path) handles compound extensions (.tar.gz etc).
- walk_audio_files(directory) — recursive, case-insensitive.
- find_archives_in_dir(directory) — top-level only (don't
surprise-extract sample / proof folders inside a torrent).
- extract_archive(archive_path, extract_to=None) — handles
.zip, .tar variants, .rar (optional rarfile dep), .7z
(optional py7zr dep). Optional deps warn-and-skip if absent.
- extract_all_in_dir + collect_audio_after_extraction — the
one-shot helpers the download plugins call after a download
completes.
- Path-traversal protection: every archive member's resolved
path must stay inside the destination — first violator aborts
the extract without writing anything. Applies to zip, tar,
and rar.
- tests/test_archive_pipeline.py: 21 tests covering the walker
(nested dirs, case-insensitive, ignores non-audio), archive
detection (compound extensions, missing files), zip extraction
+ path-traversal rejection, tar.gz + tar path-traversal,
multi-archive directory, mixed-loose-and-archived collection.
- core/torrent_clients/transmission.py: rename unused loop var
`attempt` to `_attempt` in the session-id renegotiation loop
(B007 — loop var not used in body).
- core/image_cache.py: log the cleanup exception instead of
swallowing it silently (S110 — bare try/except/pass). debug
level since a failed tmp unlink is non-fatal; the outer
``raise`` still propagates the original error.
Full ruff sweep clean.
Restructure the Indexers & Downloaders tab to mirror the
Paths & Organization / Post-Processing / Library Preferences
pattern on the Library page — each subsystem (Indexers / Torrent
Client / Usenet Client) gets its own collapsible section header
with a status dot, hint, and animated arrow.
Visual cues borrowed from Lidarr but rendered in SoulSync's
existing dark-glass theme:
- Intro hero card at the top of the tab with a 1-2-3 flow:
Indexers find releases → Downloader fetches → SoulSync imports.
Accent-color stepper pills + sub-copy summarising what's
optional vs required.
- Status dot in each section header — grey 'unknown' before
testing, green after Test Connection succeeds, red on failure.
Driven by _setIndStatusDot() helper called from each test
handler. Soft glow on the active states.
- Per-service service-title color accents matching existing
spotify-title / tidal-title pattern: prowlarr-title (orange,
Prowlarr brand), torrent-title (sky blue, qBit family),
usenet-title (violet).
- Indexer list cards replace the inline-emoji list — proper
protocol badges (Torrent vs Usenet pill), monospace id chip,
privacy tag, dimmed appearance when the indexer is disabled
in Prowlarr.
- Indexers section starts open; Torrent + Usenet start collapsed
since most users only configure one protocol.
No behavior changes — same fields, same endpoints, same save
flow. Pure visual restructure of the panels added in the previous
three commits.
54 mocked unit tests pinning the parse + dispatch behavior of the
new indexer and downloader plumbing. No live services required —
HTTP is mocked at the requests-library boundary, RPC is mocked at
the _rpc_sync helper.
Coverage:
- core/prowlarr_client.py: parse_indexer / parse_result with
category-shape variants, search query encodes repeated
``categories=`` and ``indexerIds=`` keys, check_connection hits
the right endpoint with the right header.
- core/torrent_clients/qbittorrent.py: login sends the Referer
CSRF header, login failure surfaces, parse_status normalises
field names, eta <= 0 becomes None.
- core/torrent_clients/transmission.py: bare host URL is rewritten
to /transmission/rpc, 409 + X-Transmission-Session-Id is
renegotiated and the retry carries the new id, torrent-add
surfaces torrent-duplicate hashes, eta -1 becomes None.
- core/torrent_clients/deluge.py: requires password to be configured,
magnet vs HTTP URL hit different RPC methods, progress is
normalised from 0-100 to 0-1.
- core/usenet_clients/sabnzbd.py: parse_timeleft handles HH:MM:SS
and the MM:SS fallback, queue + history merge into a single
get_all, addurl vs addfile are dispatched on the input type.
- core/usenet_clients/nzbget.py: requires URL + username + password,
mb_value prefers the 64-bit size split over the legacy MB field,
add_nzb base64-encodes raw bytes, GroupFinalDelete vs GroupDelete
is picked by the delete_files flag, non-numeric job IDs fail fast.
- state mapping tables for all five adapters get explicit assertions
so future refactors can't silently lose a native state value.
WHATS_NEW entry covers the test addition; no VERSION_MODAL_SECTIONS
entry — internal infrastructure, not user-facing.
Third commit in the torrent + usenet rollout. SoulSync now also
speaks the two big usenet downloaders through a sibling adapter
contract that mirrors the torrent adapter set. All three layers are
now stood up — Prowlarr finds releases, the torrent adapter and the
usenet adapter each know how to ship work to the underlying client.
A later commit wires Prowlarr search results through the adapters
and through the archive-extract-match pipeline.
- core/usenet_clients/base.py: UsenetClientAdapter Protocol +
UsenetStatus dataclass. Uniform state set covers usenet-specific
phases (queued / downloading / extracting / verifying / repairing /
completed / failed / paused).
- core/usenet_clients/__init__.py: adapter_for_type factory +
get_active_adapter that reads usenet_client.type each call.
- core/usenet_clients/sabnzbd.py: REST adapter. ?apikey=... auth,
mode=addurl and mode=addfile (multipart) for add_nzb. Reads both
the active queue and the recent history so completed / failed
jobs surface in get_all. Parses SAB's HH:MM:SS ``timeleft`` into
seconds.
- core/usenet_clients/nzbget.py: JSON-RPC adapter. HTTP Basic auth,
``append`` method for add_nzb (auto-detects URL vs base64 NZB),
``editqueue`` with GroupPause/GroupResume/GroupDelete/GroupFinalDelete
for state changes. Reads NZBGet's 64-bit split size fields
(FileSizeHi + FileSizeLo) preferentially over the legacy
FileSizeMB aggregate.
- core/connection_test.py: 'usenet_client' branch picks the right
adapter, runs check_connection, surfaces per-client error
messages (different credentials needed).
- config/settings.py: usenet_client.{type, url, api_key, username,
password, category} defaults + both api_key and password marked
encrypted-at-rest.
- web_server.py: 'usenet_client' added to the /api/settings POST
allow-list.
- webui/index.html: new Usenet Client panel on the Indexers &
Downloaders tab. Type picker swaps the credential fields between
API-key (SABnzbd) and username+password (NZBGet).
- webui/static/settings.js: load/save wiring, updateUsenetClientUI
for the credential field swap, testUsenetClientConnection.
- webui/static/helper.js: WHATS_NEW + VERSION_MODAL_SECTIONS entry.
Second commit in the torrent + usenet rollout. SoulSync now speaks
three different BitTorrent client APIs through one uniform adapter
contract — picks the active client by config and dispatches the same
verbs to whichever backend the user uses. Each adapter handles its
own auth quirk (qBit cookie + CSRF Referer, Transmission session-id
renegotiation, Deluge JSON-RPC session) and maps native state
strings onto a shared 7-value set so the rest of the app stays
client-agnostic.
- core/torrent_clients/base.py: TorrentClientAdapter Protocol +
TorrentStatus dataclass. Eight verbs: is_configured, check_connection,
add_torrent (URL/magnet), add_torrent_file (raw bytes), get_status,
get_all, remove, pause, resume.
- core/torrent_clients/__init__.py: adapter_for_type factory +
get_active_adapter that reads torrent_client.type each call so
settings changes take effect without restart.
- core/torrent_clients/qbittorrent.py: WebUI v2 adapter. Cookie auth
via /api/v2/auth/login, transparent 403 re-login, Referer header
to satisfy qBit's CSRF guard. add_torrent returns the just-added
hash via /torrents/info sort=added_on (qBit's add endpoint doesn't
echo the hash).
- core/torrent_clients/transmission.py: RPC adapter. Auto-resolves
bare host URLs to /transmission/rpc, handles the 409 + new
X-Transmission-Session-Id renegotiation transparently, accepts
HTTP basic auth. add_torrent_file base64-encodes payload per spec.
- core/torrent_clients/deluge.py: Deluge 2.x JSON-RPC adapter.
Password-only auth, distinguishes magnet vs HTTP URL at the RPC
method layer, applies category via Label plugin (best-effort —
label plugin is optional).
- core/connection_test.py: 'torrent_client' branch picks the right
adapter, runs check_connection, surfaces a per-client error
message.
- config/settings.py: torrent_client.{type, url, username, password,
category, save_path} defaults + torrent_client.password in the
encrypted-at-rest secrets list.
- web_server.py: 'torrent_client' added to the /api/settings POST
allow-list so saved config persists.
- webui/index.html: new Torrent Client panel on the Indexers &
Downloaders tab — client-type dropdown, URL, username, password,
category, optional save path, Test Connection.
- webui/static/settings.js: load/save wiring + testTorrentClientConnection.
- webui/static/helper.js: WHATS_NEW + VERSION_MODAL_SECTIONS entry.
First commit toward torrent and usenet download sources. Prowlarr is
the indexer manager component of the *arr stack — it exposes Usenet
and torrent indexers behind a single Newznab-style API so SoulSync
doesn't have to integrate each indexer individually. This commit
wires up Prowlarr as a search-only source; the torrent and usenet
download client adapters land in the next commits and plug into
this search surface.
- core/prowlarr_client.py: sync-backed async client. is_configured,
check_connection, get_indexers, search by Newznab category. Music
category constants (3000 all / 3010 MP3 / 3040 lossless / etc.).
- core/connection_test.py: 'prowlarr' branch hits /api/v1/system/status
for the Test Connection button.
- web_server.py: GET /api/prowlarr/indexers returns the live indexer
list (id, name, protocol, enabled, privacy). Settings POST allow-list
now accepts 'prowlarr' so saved config persists.
- config/settings.py: prowlarr.{url, api_key, indexer_ids} defaults
plus prowlarr.api_key in the encrypted-at-rest secrets list.
- webui/index.html: new "Indexers & Downloaders" tab on Settings with
the Prowlarr panel (URL, API key, Test, Refresh Indexer List,
optional indexer-ID allowlist).
- webui/static/settings.js: load/save wiring, testProwlarrConnection,
loadProwlarrIndexers (HTML-escapes user-supplied indexer names).
- webui/static/helper.js: WHATS_NEW 2.6.0 unreleased block plus a
curated VERSION_MODAL_SECTIONS entry.
Antti Kettunen
BoulderBadgeDad
Broque Thomas