mirror of https://github.com/Nezreka/SoulSync.git
dev
video
main
fix/disable-beatport-features
johnbaumb-discover-redesign
1.0
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2.0
2.1
2.2
2.3
2.4.0
2.4.1
2.4.2
2.5.0
2.5.1
2.5.2
2.5.3
2.5.4
2.5.5
2.5.6
2.5.7
2.5.9
2.6.0
2.6.1
2.6.2
2.6.3
2.6.4
2.6.5
2.6.6
2.6.7
2.6.8
2.6.9
2.7.0
2.7.1
2.7.2
2.7.3
2.7.4
2.7.5
2.7.6
2.7.7
2.7.8
2.7.9
v0.65
${ noResults }
21 Commits (adbdda7b0eeecaaabce5f5b2fa52c026ff84400a)
| Author | SHA1 | Message | Date |
|---|---|---|---|
BoulderBadgeDad
|
95d6ad4bc9 |
Fix: torrent/usenet album bundle hard-fails on 'no results' instead of falling back
A torrent-first (or usenet-first) hybrid download would freeze at
"Torrent searching for release 0%" and never move to the next source when
Prowlarr returned no results for the album. Reported by Cezar:
[Album Bundle] torrent flow failed for '...': No torrent results found
Cause: the album-bundle dispatch only returns to the per-track flow (which,
in hybrid mode, tries the next configured source) when the plugin's failure
outcome carries fallback=True; otherwise it marks the batch failed and stops.
Both plugins set fallback=True on their 'results found but none matched the
album' branch, but the adjacent 'no results at all' branch set only an error
and no fallback flag -- so zero results hard-failed while wrong results fell
back. Backwards, and soulseek's plugin already defaults fallback=True for
exactly this reason.
Fix: set fallback=True on the no-results branch in torrent.py and usenet.py.
The dispatch's fallback handling (return False -> per-track flow) was already
correct and is unchanged.
The only consumer of download_album_to_staging is the dispatch, which reads
the result via .get('fallback'), so the change is additive and locally
contained.
Tests: new test_torrent_album_to_staging_no_results_flags_fallback and
test_usenet_album_to_staging_no_results_flags_fallback assert the plugins now
emit fallback=True on an empty search; the existing torrent no-results test is
extended with the same assertion. Existing dispatch tests already pin
fallback=True -> per-track flow. Full downloads/plugins/adapters sweep: 690
passed.
|
4 weeks ago |
|
BoulderBadgeDad
|
cc433fad37 |
Album picker #730: add word-boundary full-phrase bonus (from PR #731 review)
Compared my #730 fix against contributor PR #731 (same independent design). Grafted their good idea — a confidence bonus when the album's full core phrase appears intact in the release title (rescues long multi-word names whose token coverage gets diluted) — and kept my accent-folding, which #731 lacks (their normalize drops accented chars: Bjork -> 'bj rk'). IMPORTANT: implemented the phrase bonus WORD-BOUNDARY anchored, not as a raw substring. My first cut used 'phrase in norm_title' (matching #731) and it immediately reintroduced the substring bug #730 exists to fix — 'heroes' matched 'superheroes' and the wrong album scored 0.9/passed. PR #731 has this latent flaw. The regex anchors the phrase to word boundaries so the bonus fires for real matches only. Verified: substring trap (Superheroes/Scary Monsters) rejected; edition suffixes + intact-phrase albums kept. +1 phrase-bonus test (incl. the word-boundary guard). 126 plugin tests pass; ruff clean. Co-authored-by: Tyler Richardson-LaPlume <170156756+IamGroot60@users.noreply.github.com> |
4 weeks ago |
|
BoulderBadgeDad
|
1c2efbb15c |
Album picker #730: drop the unused artist_name param (review cleanup)
Review caught that artist_name was added to pick_best_album_release's signature and threaded through both call sites but never actually used — dead, misleading code. Removed it from the helper + both callers. Artist-aware gating would be a deliberate future feature (titles carry the artist inconsistently, so a hard artist gate would risk the same false-negative class I just fixed); the album relevance gate already resolves the reported wrong-release bug. No behavior change. 127 plugin tests pass; compile + ruff clean. |
4 weeks ago |
|
BoulderBadgeDad
|
78c6f09e13 |
Album picker #730: don't reject the right album over an edition suffix
Self-review found a false-negative in the title-relevance gate I just added:
it scored 'fraction of the ALBUM-NAME's words present in the title', so a
stored album name with an edition/remaster suffix the torrent lacks
('Currents (Deluxe)', 'Heroes (2017 Remaster)') scored BELOW the 0.6 floor and
the correct release was wrongly refused -> fell back to per-track. The very
first issue example ('Heroes 2017 Remaster') would have regressed.
Fix: strip edition/format/year NOISE words (deluxe, remaster, edition, flac,
years, bitrates, ...) before scoring, via _significant_words(), with a fallback
to the raw words so an album literally named '1989' or 'Deluxe' isn't emptied
to match-everything. Verified both directions: edition suffixes now KEPT, while
the wrong-album rejection (Scary Monsters for a Heroes request, Superheroes)
still scores 0.
Tests: +2 regression tests (edition-suffix kept; noise/number-only album name).
125 album-bundle/dispatch/plugin tests pass; compile + ruff clean.
|
4 weeks ago |
|
BoulderBadgeDad
|
95f4f41c50 |
Album bundle: gate Prowlarr release picker by album-title relevance (#730)
Reporter (IamGroot60): requesting an album via a Prowlarr-backed source (Usenet/Torrent) could download a DIFFERENT album — e.g. asking for Bowie's 'Heroes' downloaded 'Scary Monsters' because the picker ranked purely by seeders/grabs -> quality -> size with NO title check, and the wrong album had ~16x the grabs. (Confirmed the old picker chose the wrong release on exactly this scenario.) Fix (the reporter's proposal): - album_title_relevance(candidate_title, album_name): word-coverage match, accent-folded (Bjork != bj rk) and WORD-BOUNDARY (Heroes != Superheroes), so a wrong album that shares no title words scores 0. - pick_best_album_release gains album_name/artist_name params and a relevance gate (floor 0.6) applied BEFORE the seeders/quality/size ranking. When album_name is given and NOTHING clears the floor, returns None. - torrent.py + usenet.py call sites pass album_name/artist_name and set result['fallback'] = True on None, so the dispatcher (source-agnostic fallback routing) hands off to the per-track flow instead of grabbing a wrong album. Matches what Soulseek already did via its preflight scorer. No album_name -> no gating (old behavior preserved for callers without a title). Tests: 9 new in test_album_bundle.py (relevance math incl. the substring trap + accent fold, the exact Bowie refuse-and-fallback scenario, None-when-no-match, and no-gate-without-name). 125 album-bundle/dispatch/plugin tests pass; compile + ruff clean. |
4 weeks ago |
Tyler Richardson-LaPlume
|
0b325da3e9 |
Usenet bundle: writable staging dir + client→local path resolution (#721)
Follow-up to the poll fix, covering the two things that blocked a
successful end-to-end album import once the poll itself stopped
freezing:
1. Staging dir permissions
The album-bundle private staging path defaults to
'storage/album_bundle_staging' -> /app/storage, but /app/storage was
never created or chowned by the image (unlike /app/Staging,
/app/Transfer, etc.), and /app is root-owned. The copy failed with
"[Errno 13] Permission denied: 'storage'" under the non-root soulsync
UID. Added /app/storage to the Dockerfile build-time mkdir+chown and
the entrypoint PUID/PGID chown, exactly like the sibling runtime dirs.
2. Client->local path resolution
Usenet/torrent clients report save paths from inside THEIR OWN
container (e.g. SAB '/data/downloads/music/<album>'); SoulSync often
mounts the same files at a different point ('/app/downloads/<album>').
Feeding the client path straight to the audio walker yields
"No audio files found" though the files are physically present.
New resolve_reported_save_path():
a. use the reported path as-is if readable (mirrored mounts),
b. apply explicit download_source.usenet_path_mappings
({from,to}, Sonarr/Radarr-style) for non-shared layouts,
c. basename fallback under SoulSync's own download roots —
zero-config for the standard shared-volume arr setup.
Wired into both call sites in usenet.py AND torrent.py
(download_album_to_staging + _finalize_download), logging any
translation and including the resolved path in the no-audio error.
Tests: resolver verbatim / explicit-mapping / basename-fallback /
priority / not-found / empty / mapping-miss-then-basename. ruff +
compileall + pytest green (645 in the download suites).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
1 month ago |
|
Tyler Richardson-LaPlume
|
b8384beef9 |
Fix: Usenet bundle stuck at 99%/100% — SAB reports post-processing in History as non-terminal (#721)
The earlier #721 fix tolerated a ~10s "completed but no save_path" window, but the real production stall sits upstream of that: SABnzbd removes a finished download from the queue and runs par2 verify / repair / unpack *in History*, exposing the live stage in the slot `status` ('Verifying' / 'Repairing' / 'Extracting' / 'Moving' / ...) with `storage` empty until the final move. `_parse_history_slot` mapped EVERY non-'Failed' status to 'completed', so a still-extracting 1.7 GB FLAC album looked "completed with no save_path" the instant download hit 100%. The poll burned its completed-no-path budget mid-PP and bailed, freezing the UI on the last download emit (the stuck-at-99%/100% signature). SAB then finished fine — which is why the job shows Completed in History but SoulSync never staged it. Root fix - `_parse_history_slot` routes `status` through `_map_state`, so PP stages stay NON-terminal: the poll keeps waiting (as 'downloading') for as long as post-processing takes and only a real 'Completed' flips to terminal success. `save_path` is trusted only on true completion (mid-PP path fields may point at the incomplete dir). Supporting / defensive - `UsenetStatus.incomplete_path`: surfaced separately from save_path (SAB `incomplete_path`) and used by the poll loops as a LAST RESORT after the completed-no-path window, to recover the case where `storage` never lands but the files are physically on disk. - `poll_album_download`: dedicated, configurable completed-no-path window (~120s via `download_source.album_bundle_completed_no_path_seconds`) decoupled from the ~10s transient-miss window; incomplete_path fallback; a 30s heartbeat log so the previously-silent poll loop is diagnosable. - `usenet.py` `_download_thread`: per-track parity — it was erroring immediately on the first completed-no-path read. - `album_bundle_dispatch.py` / `status.py` / `monitor.py`: use the project `get_logger` so download-flow logs land in app.log under the `soulsync.*` namespace (they were console-only before, which hid the `[Album Bundle] flow failed` line during triage). Tests - PP-history state mapping; end-to-end Hunky Dory PP regression (download -> Verifying/Extracting in History past both budgets -> Completed+storage -> success); completed-no-path window + incomplete_path fallback; per-track thread parity. ruff + compileall + pytest all green (the only local failures are environmental: missing tzdata + local tools/ffmpeg.exe, neither present on CI). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
1 month ago |
Broque Thomas
|
df675c7c9f |
Fix: Usenet bundle stuck on "downloading release" when SAB History flips before storage lands (#721)
Follow-up to the 2.6.3 queue→history handoff fix (#706). User @IamGroot60 reported in #721 that on 2.6.3 the bundle still gets stuck mid-flight: SoulSync UI sits on "Usenet downloading release 61%" forever, SAB History shows the job as Completed 2+ minutes ago, files are physically present in the slskd downloads folder but never copied into ``storage/album_bundle_staging/<batch>/``. Root cause: a second-stage gap in the SAB pipeline. SAB flips a job's ``status`` to ``Completed`` in History as soon as par2 + unrar finish, but its post-processing pipeline writes the final ``storage`` field a few seconds LATER (the move-to-final step). ``poll_album_download`` saw the first ``Completed`` read with ``save_path=None`` and bailed: if status.state in complete_states: return last_save_path # ← None at this point ``download_album_to_staging`` got ``save_path=None``, set ``result['error']`` and returned. The bundle was marked failed but the LAST progress emit before the failure was ``downloading progress=0.61``, so the UI froze on "61%" — the terminal ``failed`` emit never registered on the user's screen because the renderer holds the last-known progress. Fix - ``poll_album_download`` now tracks a separate transient counter for "complete state seen, save_path not yet set." Up to ``transient_miss_threshold`` (default 5) consecutive reads in that state are tolerated before the poll bails. SAB writes the ``storage`` field within 2-10 seconds of the History flip in practice — the default 5 × 2s = 10s window covers it. - When save_path eventually lands, return it normally. - When the threshold is exhausted with save_path still empty, emit terminal ``failed`` with an explicit message pointing at the missing save_path field — no more 6-hour silent spin. - Earlier ``downloading`` reads with a non-empty ``save_path`` (qBit / Transmission set this from the start of the download) remain "sticky" — if the eventual ``completed`` read has empty save_path, the cached one applies. So torrent flows aren't affected by the retry path. SAB adapter (``_parse_history_slot``) - Widened the save_path field fallback chain: storage → path → download_path → dirname → incomplete_path Covers SAB version differences (older builds populated ``path``) and forks that expose ``download_path`` or ``dirname``. ``incomplete_path`` is the last resort — SAB's in-progress dir before the final move — so the bundle plugin at least has a path to scan when nothing else lands. - Whitespace-only values are skipped. - Loud debug log when none of the known fields land — users on SAB versions / forks with novel field names need to see this in logs so we can grow ``_HISTORY_SAVE_PATH_KEYS``. Tests - ``test_album_bundle.py`` (3 new): - tolerates_completed_with_late_save_path_arrival — the #721 scenario; first Completed read has no save_path, third has it; poll returns the path normally - gives_up_when_completed_with_no_save_path_persists — past the threshold the poll fails loudly instead of silent-spinning - uses_save_path_from_earlier_downloading_emit_if_completed_lacks_one — sticky save_path keeps torrent flows working - ``test_usenet_client_adapters.py`` (6 new): - falls back to ``path`` when ``storage`` empty - falls back to ``download_path`` - prefers ``storage`` when multiple fields present - returns ``None`` when all fields empty (the #721 gap window) - ignores whitespace-only values - uses ``incomplete_path`` as last resort 132 album-bundle + usenet tests pass. Branch is on dev parented at 2.6.3 — user @IamGroot60 offered to test on dev, so this is a candidate cherry-pick for either a 2.6.4 hotfix or merge straight into dev for the next release. |
1 month ago |
|
Broque Thomas
|
e2d45c51e5 |
Address kettui-flagged items on usenet poll fix (#706)
Follow-up to
|
1 month ago |
|
Broque Thomas
|
f13d339584 |
Usenet album poll: tolerate SAB queue→history handoff, emit terminal failure (#706)
User reported usenet album downloads getting stuck on "downloading
release" while SABnzbd reported the job as complete. Container restart
did not help; reproducible on every usenet album download.
Three independent issues all causing the same symptom — the download
modal freezes mid-flow with no error surfaced to the user:
1. SAB queue → history transition window
SAB removes a slot from its queue BEFORE adding it to the history,
and on a busy server (par2 verify, unrar, multi-file move) that
window can span several poll iterations. The poll treated a single
None status as terminal failure ("disappeared from client") and
gave up. Now the poll tolerates up to ~10s of consecutive misses
(5 polls at the default 2s interval) before declaring the job gone.
2. SAB queue states like `Pp` were unmapped
`_SAB_QUEUE_STATE_MAP` didn't cover SAB's `Pp` (post-processing
summary), `Unpacking`, `Trying`, `Deleted`, or the `Prop_paused`
/ `Prop_failed` variants. Unmapped states fell through to the
default-'error' fallback, and the poll loop only treated explicit
'failed' / 'completed' as terminal — 'error' was neither, so the
loop spun until the 6-hour timeout. Map now covers every Status
value from SAB's `sabnzbd/api.py`, and the poll treats the default-
'error' fallback as a transient miss (warn-logged, retry within
the same tolerance window) so a brand-new unmapped state can't
infinite-loop the way `Pp` did here.
3. No terminal failure emit
The poll only logged on failure / timeout / disappeared — never
called the progress callback with 'failed', so the download modal
stayed at the last 'downloading' emit forever. Plumb a 'failed'
emit through every failure exit path so the UI flips out of the
downloading state when the poll gives up.
Plus:
4. SAB direct nzo_ids lookup instead of paging all-history
`_get_status_sync` was fetching the latest 50 history entries on
every poll and iterating to find the target nzo_id. On busy
servers (many recent downloads), the target job could roll past
the 50-entry window and look like a "disappeared" job. Replaced
with a targeted `mode=queue&nzo_ids=<id>` → `mode=history&nzo_ids=<id>`
chain. Falls back to the bulk path for SAB versions that pre-date
the nzo_ids filter — the transient-miss tolerance covers any
short-lived gap there too.
Implementation:
Lifted the album-bundle poll loop out of `usenet.py` and `torrent.py`
into `core/download_plugins/album_bundle.py:poll_album_download` —
near-duplicate implementations are now a single function with deps
injected so it's testable in isolation (kettui's extract-don't-AST-parse
standard; can't unit-test a `time.sleep` loop inside a plugin method).
The lifted helper takes:
- `get_status` callable bound to job_id, so the same loop works for
usenet UsenetStatus and torrent TorrentStatus shapes
- `complete_states` set so torrent's `{'seeding', 'completed'}` and
usenet's `{'completed'}` both Just Work
- `failed_states` set so torrent's `{'error'}` is terminal while
usenet's default-'error' fallback is transient
- `transient_miss_threshold` (default 5 ≈ 10s at 2s poll)
- `sleep` / `monotonic` injectables for deterministic tests
Per-track flows in both plugins gained the same transient-miss
tolerance inline — they don't use the emit pattern (update an
`active_downloads[id]` row dict via lock instead), so reusing the
helper would have required threading a no-op emit through. Inline
fix is small enough.
Tests:
- 11 new tests in `tests/test_album_bundle.py:poll_album_download`
cover the happy path, transient-miss tolerance with recovery,
hard-failure threshold, explicit-failed surface, timeout-emit,
default-'error' transient treatment, shutdown clean exit,
torrent's `seeding`-counts-as-complete, save_path captured across
iterations, and adapter-exception treated as transient miss.
- 521 download-suite tests pass (33 in test_album_bundle, others
pin existing torrent + usenet contracts).
- Ruff clean.
Closes #706.
|
1 month ago |
|
Broque Thomas
|
6c9b43225a |
Add torrent and usenet release staging support
Adds torrent/usenet as release-oriented download sources with album-bundle staging, live progress reporting, and post-processing that selects the requested audio file from completed releases instead of blindly importing the first file. Keeps album-bundle behavior gated to single-source torrent/usenet album downloads, excludes release sources from hybrid album per-track searches, and allows hybrid non-album tracks to use release results safely. Improves staged-release matching for featured/bonus track filenames while preserving version mismatches, records torrent/usenet provenance in library history, and updates service/status UI labels. Covers the flow with focused lifecycle, status, staging, validation, task worker, post-processing, and import side-effect tests. |
1 month ago |
|
Broque Thomas
|
8b0de9eb76 |
fix(downloads): harden album bundle staging
Route torrent and Usenet album bundles through private per-batch staging so Auto-Import cannot race public staging or duplicate imports. Expose album-bundle progress in batch status and render it on the Downloads page while the external client is still downloading. Tighten release handoff safety by rejecting archive path traversal, ignoring torrent candidates without a usable URL, and skipping Soulseek source reuse for torrent/Usenet batches. Tests: .venv/bin/python -m pytest tests/downloads/test_downloads_status.py tests/test_album_bundle_dispatch.py tests/downloads/test_downloads_staging.py tests/test_torrent_usenet_plugins.py |
1 month ago |
|
Broque Thomas
|
670a2db95e |
refactor(downloads): extract album_bundle shared helpers + atomic copy
Per code review: the album-bundle helpers (release picker + staging collision suffix) were defined as private symbols in torrent.py and imported by usenet.py through ``from core.download_plugins.torrent import _pick_best_album_release, _unique_staging_path``. Sibling plugins shouldn't reach into each other's private surface — leaky module boundary, and the underscore prefix says don't import. Also addressed two latent issues at the same time: - The Auto-Import sweep race: my plugin copied audio files into staging via plain ``shutil.copy2``, which exposes a partial file at the audio extension for the duration of the copy. The Auto- Import worker filters by audio extension when scanning Staging (AUDIO_EXTENSIONS in core/auto_import_worker.py), so a mid-flight scan could pick up a truncated file. Fix: copy to a ``.tmp.<random>`` sidecar first, then atomically rename via ``Path.replace`` (which is ``os.replace`` — atomic on the same filesystem). Auto-Import sees the file either at its final name or not at all. - The 6-hour poll timeout was a hard-coded magic constant. Users with slow private trackers or large box sets would silently time out after 6h. Both the timeout and the poll interval are now read from config (``download_source.album_bundle_timeout_seconds`` / ``..._poll_interval_seconds``) with safe fallback to the existing defaults when unset / non-numeric. - core/download_plugins/album_bundle.py: new module owns the shared surface — ``pick_best_album_release`` (with quality_guess passed in as a parameter to avoid the circular import that would result from this module trying to know about torrent.py's title parser), ``unique_staging_path``, ``atomic_copy_to_staging``, ``copy_audio_files_atomically``, ``get_poll_interval``, ``get_poll_timeout``. Module-level size constants and quality weights live here too. Usenet's grabs-as-popularity-proxy is built into the picker so both plugins get the right behavior without divergent local logic. - core/download_plugins/torrent.py: drops the local helpers + the hard-coded poll constants, imports from album_bundle. Per-track download flow still uses module-level ``_POLL_TIMEOUT_SECONDS`` / ``_POLL_INTERVAL_SECONDS`` aliases (read from config once at import time, same as before from a per-track perspective). - core/download_plugins/usenet.py: drops the imports of the torrent.py private helpers; everything goes through album_bundle now. Stops the cross-plugin private-import leak that started this whole refactor. - tests/test_album_bundle.py: 23 new tests covering the picker heuristic (empty input, singleton drop, FLAC preference, grabs fallback for usenet, size-floor / ceiling boundaries), the collision-suffix logic, the atomic-copy invariant (concurrent scanner thread asserts it never observes a partial audio file during five sequential copies), the failure-skip behavior of the batch copier, and the config-driven poll cadence including garbage-input fallback. - tests/test_torrent_usenet_plugins.py: existing picker tests updated to call the new module-level helpers instead of the former torrent.py privates. |
1 month ago |
|
Broque Thomas
|
c990ce079d |
feat(downloads): album-bundle flow for torrent/usenet single-source mode
Fixes the core architectural mismatch between indexer-based sources
and the per-track search-and-pick contract every other download
plugin satisfies. Prowlarr returns release-level torrents and NZBs;
searching for "Luther (with SZA)" against the GNX album torrent
scores near-zero on track-title similarity. Per-track candidate
validation rejects every result, every track in the batch flips
to not_found. The album-name fallback added in an earlier commit
papers over it for some cases but doesn't fix the fundamental
behavior: the user wanted the whole album.
New album-bundle flow does what the user actually wanted:
1. Gate fires inside core/downloads/master.py BEFORE the per-track
analysis loop, strictly when the batch has an album context AND
download_source.mode is 'torrent' or 'usenet' (single-source —
hybrid stays per-track to preserve fallback to Soulseek / etc.).
2. Plugin's new download_album_to_staging method searches Prowlarr
ONCE for the album as a whole ('<artist> <album>'), filters to
the right protocol, runs results through _pick_best_album_release.
3. Picker prefers seeded FLAC over low-seeded MP3, drops single-
track torrents that snuck in via the 40 MB size floor (single
tracks are typically ~10 MB), falls back to most-seeded when
every candidate is below the floor.
4. Picked release goes to the active adapter (qBit / Transmission /
Deluge for torrent; SAB / NZBGet for usenet). Polls until
complete with progress mirrored into the batch state so the
Downloads page can show meaningful status.
5. On completion the existing archive_pipeline walks the save dir
(extracting archives if any), every audio file gets copied into
the staging folder via _unique_staging_path so concurrent batches
don't collide.
6. Gate exits, master worker continues into the normal per-track
flow. Each track task hits try_staging_match early in the worker
and finds its file by fuzzy title match — no Prowlarr search
ever fires per-track, no candidate rejection, files flow through
the existing post-processing pipeline (tags, AcoustID, library
import).
Gate is strictly opt-in. Three orthogonal conditions must all hold:
batch_is_album, mode in ('torrent', 'usenet'), and the plugin must
expose download_album_to_staging. Any other source / hybrid mode /
non-album batch flows through the master worker unchanged. The
existing per-track torrent path still works for basic-search
single-track grabs.
- core/download_plugins/torrent.py: download_album_to_staging plus
_pick_best_album_release and _unique_staging_path helpers (shared
with the usenet plugin). _poll_album_download mirrors the existing
poll loop with progress callback emission.
- core/download_plugins/usenet.py: parallel implementation reusing
the picker + staging helpers. Different state set ('failed' vs
'error') from the usenet adapter contract.
- core/downloads/master.py: ~90-line gate right after batch context
loading. Mirrors plugin lifecycle into batch state under
``album_bundle_*`` keys so the Downloads page can render progress
while the torrent/usenet job runs (per-track tasks don't exist
yet during this phase). Failed bundle download fails the batch
with a meaningful error; missing plugin / context falls back to
the per-track flow with a warning.
- tests/test_torrent_usenet_plugins.py: 5 new tests pinning the
album picker preferences (FLAC over MP3 with comparable size +
better seeders, size floor drops singles, fallback when all
small), staging-path collision suffix, and the not-configured
short-circuit.
|
1 month ago |
|
Broque Thomas
|
478fd25dd6 |
fix(downloads): pre-fill artist/title so search UI doesn't show download URL
Real-world test surfaced the bug — torrent results displayed
'by download?apikey=c15d6f69...&link=...' as the uploader / artist
in the basic search UI. The cause is TrackResult.__post_init__:
when artist is None it runs parse_filename_metadata on the bare
filename, and our filename starts with the indexer's download URL
(needed so download() can recover the URL later). The auto-parser
treats the URL as 'artist' and ships it to the UI.
Fix:
- core/download_plugins/torrent.py: new _parse_release_title()
splits 'Artist - Title' / 'Artist - Album' out of the release
title and strips trailing [FLAC] / (2016) tags. Falls back to
('', cleaned_title) when no dash is found, and explicitly
rejects URL-looking strings as an extra defence. The projection
pre-fills both artist and title on TrackResult, so __post_init__
skips the auto-parse entirely. When the release title has no
dash, artist defaults to the indexer name so the UI shows
'by Indexer' instead of a URL.
- core/download_plugins/usenet.py: imports the new helper and
applies the same fix.
- tests/test_torrent_usenet_plugins.py: 5 tests for the new
helper (dash split, trailing-tag stripping, no-dash fallback,
multiple-dash preservation, URL-prefix rejection). Existing
projection tests updated to assert artist + title come through
parsed correctly, plus a new test pinning the indexer-name
fallback for titles without a dash so the URL-leak regression
can't return.
|
1 month ago |
|
Broque Thomas
|
080b1aa1b4 |
feat(downloads): wire torrent + usenet as live download sources
The payoff for the previous five commits. Two new download sources slot into the existing DownloadSourcePlugin contract, backed by Prowlarr (search) + the torrent or usenet client adapter (transfer) + archive_pipeline (post-extract walk). They appear in the Download Source dropdown next to Soulseek / Tidal / Lidarr / etc. and also participate in hybrid mode. Pipeline (both plugins, mirror shape): 1. search(query) → ProwlarrClient.search filtered to the right protocol, projected into TrackResult / AlbumResult shapes the existing search UI already speaks. Filename field encodes the indexer's download URL (or magnet URI for torrents) so download() can recover it later. 2. download() → decodes URL, hands it to the active adapter (qBittorrent / Transmission / Deluge for torrent; SABnzbd / NZBGet for usenet), spawns a background poll thread that tracks progress + reports the adapter-reported save_path. 3. On 'seeding' / 'completed' → archive_pipeline walks the save directory, extracts any archives the downloader didn't already unpack, picks the first audio file as the canonical file_path. Matches the Lidarr client's single-track-pick contract — picking which specific track to import happens in post-processing. - core/download_plugins/torrent.py: TorrentDownloadPlugin + module-level helpers (_decode_filename, _guess_quality_from_title, _parse_indexer_id_filter, _adapter_state_to_display, _row_to_status). Uses get_active_torrent_adapter() so a settings change to the client type takes effect without restart. - core/download_plugins/usenet.py: UsenetDownloadPlugin — parallel shape, reuses the torrent module's helpers. Different enough states (no seeding, no magnet) to warrant its own class but cheap to keep in lockstep. - core/download_plugins/registry.py: register 'torrent' and 'usenet' plugins. Per the registry docstring this is the only wiring point needed — the orchestrator picks them up automatically via the iteration helpers. - webui/index.html: 'Torrent Only (via Prowlarr)' + 'Usenet Only (via Prowlarr)' added to the Download Source dropdown. New redirect card (#prowlarr-source-redirect) explains that the actual config lives on the Indexers & Downloaders tab — shown whenever torrent or usenet is in the active source set. - webui/static/settings.js: HYBRID_SOURCES gets two new entries so hybrid mode can pick them up. updateDownloadSourceUI now toggles the redirect card based on active sources. - tests/test_torrent_usenet_plugins.py: 23 tests covering pure helpers (filename encode/decode round-trip incl. magnet URIs, quality guesser, state mapping), search projection logic (protocol filter, drops without URLs, magnet-preferred-over-URL, filename encoding, neutralised soulseek-specific score fields), is_configured (both prowlarr + adapter required), finalize (picks first audio file, errors on empty dir / missing save_path), clear/get_all lifecycle, DownloadSourcePlugin protocol conformance, and registry membership. |
1 month ago |
|
Broque Thomas
|
fa73c41ef6 |
Wire Amazon Music as a first-class download source
Follows the exact same standard as Tidal, Qobuz, HiFi, and Deezer. registry.py — import + register AmazonDownloadClient as 'amazon'. amazon_download_client.py — read amazon_download.quality / allow_fallback from config on init; pass quality as preferred_codec to AmazonClient; _download_sync codec waterfall respects allow_fallback flag. download_orchestrator.py — reload_settings() updates preferred_codec + allow_fallback on the live client after a settings save. 'amazon' added to _streaming_sources so search_and_download_best routes it correctly. api_call_tracker.py — 'amazon' registered in RATE_LIMITS (120/min), SERVICE_LABELS, and SERVICE_ORDER so API call monitoring shows Amazon. web_server.py — 'amazon_download' added to the settings service loop. 'amazon' added to serverless_sources (no slskd probe needed). Streaming file-finder extended to handle amazon username + ||asin||title encoding (extension-less fuzzy match, same as Tidal/Qobuz/HiFi). New endpoint: GET /api/amazon/test-connection → checks T2Tunes proxy status. webui/index.html — amazon-download-settings-container: quality dropdown (flac/opus/eac3), allow-fallback checkbox, test-connection button. webui/static/settings.js — 'Amazon Music' added to HYBRID_SOURCES, _hybridSourceEnabled, allSources mode list, loadSettings(), saveSettings() payload, updateDownloadSourceUI() show/hide + auto-test. New testAmazonConnection() function. |
1 month ago |
|
Broque Thomas
|
2c0a0da9ea |
Address Copilot doc-drift review
Four stale doc/comment references caught by Copilot's pass:
- core/download_plugins/base.py: TYPE_CHECKING comment said the
shared dataclasses lived in core.soulseek_client. They were moved
to core.download_plugins.types in this PR. Comment updated.
- core/qobuz_client.py: reload_credentials docstring still referenced
soulseek_client.client('qobuz') after the global rename to
download_orchestrator. Updated to download_orchestrator.client(...).
- webui/static/helper.js: the older WHATS_NEW entries for the plugin
contract + engine refactor still claimed backward-compat
self.<source> attributes were preserved. Followup commits in the
same PR removed them. Each entry now flags the followup explicitly
and points at the "Drop Backward-Compat Per-Source Attrs" entry
above it so the changelog is internally consistent.
- docs/download-engine-refactor-plan.md: Compatibility commitments
section listed orchestrator.<source> attribute preservation as a
guarantee. Cin's review pass removed those attrs (and renamed the
global handle from soulseek_client to download_orchestrator) — both
are breaking changes for in-tree callers (which were migrated) and
in-flight branches (which will need to update). Section rewritten
to document the actual outcome.
|
2 months ago |
|
Broque Thomas
|
d17365296a |
Lift shared download dataclasses + boot via singleton factory
Two architectural cleanups on top of the download engine refactor. (1) Shared dataclasses move to neutral plugin package. TrackResult, AlbumResult, DownloadStatus, SearchResult lived in core/soulseek_client.py for historical reasons — every other plugin imported them from the soulseek module just to satisfy the contract, coupling 8 clients to a sibling source for type imports only. Moved them to the new core/download_plugins/types.py module and updated all 14 import sites across the deezer/hifi/lidarr/qobuz/soundcloud/tidal/ youtube clients, the engine, matching engine, redownload helper, and tests. Clean break, no backward-compat re-export. (2) web_server.py boots the orchestrator via the singleton factory. After construction it now calls set_download_orchestrator(...) so get_download_orchestrator() returns the same instance the global handle points at instead of lazily building a separate orchestrator. Matches the get_metadata_engine() pattern. |
2 months ago |
|
Broque Thomas
|
ea654f664e |
Cin-1: Make DownloadSourcePlugin inheritance explicit on every client
Cin's review feedback: the plugin contract was discoverable only from the registry, not from the client files themselves. Reading `youtube_client.py` cold gave no signal that the class participates in the DownloadSourcePlugin contract. Every download client class now inherits DownloadSourcePlugin explicitly: - SoulseekClient(DownloadSourcePlugin) - YouTubeClient(DownloadSourcePlugin) - TidalDownloadClient(DownloadSourcePlugin) - QobuzClient(DownloadSourcePlugin) - HiFiClient(DownloadSourcePlugin) - DeezerDownloadClient(DownloadSourcePlugin) - SoundcloudClient(DownloadSourcePlugin) - LidarrDownloadClient(DownloadSourcePlugin) Adjustments: - core/download_plugins/base.py — moved TrackResult/AlbumResult/ DownloadStatus imports under TYPE_CHECKING since they're only used in type annotations. Without this, clients inheriting the contract create a circular import. - core/download_plugins/__init__.py — drops DownloadPluginRegistry re-export. Importing the package no longer triggers the registry's eager client imports (which would also be circular for clients that import from the package). Callers that need the registry import it directly: `from core.download_plugins.registry import DownloadPluginRegistry`. Suite still green (335 download tests). |
2 months ago |
|
Broque Thomas
|
19fbcf267d |
Add DownloadSourcePlugin contract + registry
`core/download_plugins/` defines the canonical interface every download source must satisfy and the registry that holds them. Single source of truth replacing the orchestrator's hardcoded `[self.soulseek, self.youtube, ...]` lists scattered across 6+ dispatch sites. Pure additive — no consumers wired through the registry yet. |
2 months ago |