11 Commits (dev)

Author	SHA1	Message	Date
Broque Thomas	6c9b43225a	Add torrent and usenet release staging support ... Adds torrent/usenet as release-oriented download sources with album-bundle staging, live progress reporting, and post-processing that selects the requested audio file from completed releases instead of blindly importing the first file. Keeps album-bundle behavior gated to single-source torrent/usenet album downloads, excludes release sources from hybrid album per-track searches, and allows hybrid non-album tracks to use release results safely. Improves staged-release matching for featured/bonus track filenames while preserving version mismatches, records torrent/usenet provenance in library history, and updates service/status UI labels. Covers the flow with focused lifecycle, status, staging, validation, task worker, post-processing, and import side-effect tests.	5 days ago
Broque Thomas	8b0de9eb76	fix(downloads): harden album bundle staging ... Route torrent and Usenet album bundles through private per-batch staging so Auto-Import cannot race public staging or duplicate imports. Expose album-bundle progress in batch status and render it on the Downloads page while the external client is still downloading. Tighten release handoff safety by rejecting archive path traversal, ignoring torrent candidates without a usable URL, and skipping Soulseek source reuse for torrent/Usenet batches. Tests: .venv/bin/python -m pytest tests/downloads/test_downloads_status.py tests/test_album_bundle_dispatch.py tests/downloads/test_downloads_staging.py tests/test_torrent_usenet_plugins.py	5 days ago
Broque Thomas	670a2db95e	refactor(downloads): extract album_bundle shared helpers + atomic copy ... Per code review: the album-bundle helpers (release picker + staging collision suffix) were defined as private symbols in torrent.py and imported by usenet.py through ``from core.download_plugins.torrent import _pick_best_album_release, _unique_staging_path``. Sibling plugins shouldn't reach into each other's private surface — leaky module boundary, and the underscore prefix says don't import. Also addressed two latent issues at the same time: - The Auto-Import sweep race: my plugin copied audio files into staging via plain ``shutil.copy2``, which exposes a partial file at the audio extension for the duration of the copy. The Auto- Import worker filters by audio extension when scanning Staging (AUDIO_EXTENSIONS in core/auto_import_worker.py), so a mid-flight scan could pick up a truncated file. Fix: copy to a ``.tmp.<random>`` sidecar first, then atomically rename via ``Path.replace`` (which is ``os.replace`` — atomic on the same filesystem). Auto-Import sees the file either at its final name or not at all. - The 6-hour poll timeout was a hard-coded magic constant. Users with slow private trackers or large box sets would silently time out after 6h. Both the timeout and the poll interval are now read from config (``download_source.album_bundle_timeout_seconds`` / ``..._poll_interval_seconds``) with safe fallback to the existing defaults when unset / non-numeric. - core/download_plugins/album_bundle.py: new module owns the shared surface — ``pick_best_album_release`` (with quality_guess passed in as a parameter to avoid the circular import that would result from this module trying to know about torrent.py's title parser), ``unique_staging_path``, ``atomic_copy_to_staging``, ``copy_audio_files_atomically``, ``get_poll_interval``, ``get_poll_timeout``. Module-level size constants and quality weights live here too. Usenet's grabs-as-popularity-proxy is built into the picker so both plugins get the right behavior without divergent local logic. - core/download_plugins/torrent.py: drops the local helpers + the hard-coded poll constants, imports from album_bundle. Per-track download flow still uses module-level ``_POLL_TIMEOUT_SECONDS`` / ``_POLL_INTERVAL_SECONDS`` aliases (read from config once at import time, same as before from a per-track perspective). - core/download_plugins/usenet.py: drops the imports of the torrent.py private helpers; everything goes through album_bundle now. Stops the cross-plugin private-import leak that started this whole refactor. - tests/test_album_bundle.py: 23 new tests covering the picker heuristic (empty input, singleton drop, FLAC preference, grabs fallback for usenet, size-floor / ceiling boundaries), the collision-suffix logic, the atomic-copy invariant (concurrent scanner thread asserts it never observes a partial audio file during five sequential copies), the failure-skip behavior of the batch copier, and the config-driven poll cadence including garbage-input fallback. - tests/test_torrent_usenet_plugins.py: existing picker tests updated to call the new module-level helpers instead of the former torrent.py privates.	5 days ago
Broque Thomas	c990ce079d	feat(downloads): album-bundle flow for torrent/usenet single-source mode ... Fixes the core architectural mismatch between indexer-based sources and the per-track search-and-pick contract every other download plugin satisfies. Prowlarr returns release-level torrents and NZBs; searching for "Luther (with SZA)" against the GNX album torrent scores near-zero on track-title similarity. Per-track candidate validation rejects every result, every track in the batch flips to not_found. The album-name fallback added in an earlier commit papers over it for some cases but doesn't fix the fundamental behavior: the user wanted the whole album. New album-bundle flow does what the user actually wanted: 1. Gate fires inside core/downloads/master.py BEFORE the per-track analysis loop, strictly when the batch has an album context AND download_source.mode is 'torrent' or 'usenet' (single-source — hybrid stays per-track to preserve fallback to Soulseek / etc.). 2. Plugin's new download_album_to_staging method searches Prowlarr ONCE for the album as a whole ('<artist> <album>'), filters to the right protocol, runs results through _pick_best_album_release. 3. Picker prefers seeded FLAC over low-seeded MP3, drops single- track torrents that snuck in via the 40 MB size floor (single tracks are typically ~10 MB), falls back to most-seeded when every candidate is below the floor. 4. Picked release goes to the active adapter (qBit / Transmission / Deluge for torrent; SAB / NZBGet for usenet). Polls until complete with progress mirrored into the batch state so the Downloads page can show meaningful status. 5. On completion the existing archive_pipeline walks the save dir (extracting archives if any), every audio file gets copied into the staging folder via _unique_staging_path so concurrent batches don't collide. 6. Gate exits, master worker continues into the normal per-track flow. Each track task hits try_staging_match early in the worker and finds its file by fuzzy title match — no Prowlarr search ever fires per-track, no candidate rejection, files flow through the existing post-processing pipeline (tags, AcoustID, library import). Gate is strictly opt-in. Three orthogonal conditions must all hold: batch_is_album, mode in ('torrent', 'usenet'), and the plugin must expose download_album_to_staging. Any other source / hybrid mode / non-album batch flows through the master worker unchanged. The existing per-track torrent path still works for basic-search single-track grabs. - core/download_plugins/torrent.py: download_album_to_staging plus _pick_best_album_release and _unique_staging_path helpers (shared with the usenet plugin). _poll_album_download mirrors the existing poll loop with progress callback emission. - core/download_plugins/usenet.py: parallel implementation reusing the picker + staging helpers. Different state set ('failed' vs 'error') from the usenet adapter contract. - core/downloads/master.py: ~90-line gate right after batch context loading. Mirrors plugin lifecycle into batch state under ``album_bundle_*`` keys so the Downloads page can render progress while the torrent/usenet job runs (per-track tasks don't exist yet during this phase). Failed bundle download fails the batch with a meaningful error; missing plugin / context falls back to the per-track flow with a warning. - tests/test_torrent_usenet_plugins.py: 5 new tests pinning the album picker preferences (FLAC over MP3 with comparable size + better seeders, size floor drops singles, fallback when all small), staging-path collision suffix, and the not-configured short-circuit.	5 days ago
Broque Thomas	478fd25dd6	fix(downloads): pre-fill artist/title so search UI doesn't show download URL ... Real-world test surfaced the bug — torrent results displayed 'by download?apikey=c15d6f69...&link=...' as the uploader / artist in the basic search UI. The cause is TrackResult.__post_init__: when artist is None it runs parse_filename_metadata on the bare filename, and our filename starts with the indexer's download URL (needed so download() can recover the URL later). The auto-parser treats the URL as 'artist' and ships it to the UI. Fix: - core/download_plugins/torrent.py: new _parse_release_title() splits 'Artist - Title' / 'Artist - Album' out of the release title and strips trailing [FLAC] / (2016) tags. Falls back to ('', cleaned_title) when no dash is found, and explicitly rejects URL-looking strings as an extra defence. The projection pre-fills both artist and title on TrackResult, so __post_init__ skips the auto-parse entirely. When the release title has no dash, artist defaults to the indexer name so the UI shows 'by Indexer' instead of a URL. - core/download_plugins/usenet.py: imports the new helper and applies the same fix. - tests/test_torrent_usenet_plugins.py: 5 tests for the new helper (dash split, trailing-tag stripping, no-dash fallback, multiple-dash preservation, URL-prefix rejection). Existing projection tests updated to assert artist + title come through parsed correctly, plus a new test pinning the indexer-name fallback for titles without a dash so the URL-leak regression can't return.	5 days ago
Broque Thomas	080b1aa1b4	feat(downloads): wire torrent + usenet as live download sources ... The payoff for the previous five commits. Two new download sources slot into the existing DownloadSourcePlugin contract, backed by Prowlarr (search) + the torrent or usenet client adapter (transfer) + archive_pipeline (post-extract walk). They appear in the Download Source dropdown next to Soulseek / Tidal / Lidarr / etc. and also participate in hybrid mode. Pipeline (both plugins, mirror shape): 1. search(query) → ProwlarrClient.search filtered to the right protocol, projected into TrackResult / AlbumResult shapes the existing search UI already speaks. Filename field encodes the indexer's download URL (or magnet URI for torrents) so download() can recover it later. 2. download() → decodes URL, hands it to the active adapter (qBittorrent / Transmission / Deluge for torrent; SABnzbd / NZBGet for usenet), spawns a background poll thread that tracks progress + reports the adapter-reported save_path. 3. On 'seeding' / 'completed' → archive_pipeline walks the save directory, extracts any archives the downloader didn't already unpack, picks the first audio file as the canonical file_path. Matches the Lidarr client's single-track-pick contract — picking which specific track to import happens in post-processing. - core/download_plugins/torrent.py: TorrentDownloadPlugin + module-level helpers (_decode_filename, _guess_quality_from_title, _parse_indexer_id_filter, _adapter_state_to_display, _row_to_status). Uses get_active_torrent_adapter() so a settings change to the client type takes effect without restart. - core/download_plugins/usenet.py: UsenetDownloadPlugin — parallel shape, reuses the torrent module's helpers. Different enough states (no seeding, no magnet) to warrant its own class but cheap to keep in lockstep. - core/download_plugins/registry.py: register 'torrent' and 'usenet' plugins. Per the registry docstring this is the only wiring point needed — the orchestrator picks them up automatically via the iteration helpers. - webui/index.html: 'Torrent Only (via Prowlarr)' + 'Usenet Only (via Prowlarr)' added to the Download Source dropdown. New redirect card (#prowlarr-source-redirect) explains that the actual config lives on the Indexers & Downloaders tab — shown whenever torrent or usenet is in the active source set. - webui/static/settings.js: HYBRID_SOURCES gets two new entries so hybrid mode can pick them up. updateDownloadSourceUI now toggles the redirect card based on active sources. - tests/test_torrent_usenet_plugins.py: 23 tests covering pure helpers (filename encode/decode round-trip incl. magnet URIs, quality guesser, state mapping), search projection logic (protocol filter, drops without URLs, magnet-preferred-over-URL, filename encoding, neutralised soulseek-specific score fields), is_configured (both prowlarr + adapter required), finalize (picks first audio file, errors on empty dir / missing save_path), clear/get_all lifecycle, DownloadSourcePlugin protocol conformance, and registry membership.	6 days ago
Broque Thomas	fa73c41ef6	Wire Amazon Music as a first-class download source ... Follows the exact same standard as Tidal, Qobuz, HiFi, and Deezer. registry.py — import + register AmazonDownloadClient as 'amazon'. amazon_download_client.py — read amazon_download.quality / allow_fallback from config on init; pass quality as preferred_codec to AmazonClient; _download_sync codec waterfall respects allow_fallback flag. download_orchestrator.py — reload_settings() updates preferred_codec + allow_fallback on the live client after a settings save. 'amazon' added to _streaming_sources so search_and_download_best routes it correctly. api_call_tracker.py — 'amazon' registered in RATE_LIMITS (120/min), SERVICE_LABELS, and SERVICE_ORDER so API call monitoring shows Amazon. web_server.py — 'amazon_download' added to the settings service loop. 'amazon' added to serverless_sources (no slskd probe needed). Streaming file-finder extended to handle amazon username + ||asin||title encoding (extension-less fuzzy match, same as Tidal/Qobuz/HiFi). New endpoint: GET /api/amazon/test-connection → checks T2Tunes proxy status. webui/index.html — amazon-download-settings-container: quality dropdown (flac/opus/eac3), allow-fallback checkbox, test-connection button. webui/static/settings.js — 'Amazon Music' added to HYBRID_SOURCES, _hybridSourceEnabled, allSources mode list, loadSettings(), saveSettings() payload, updateDownloadSourceUI() show/hide + auto-test. New testAmazonConnection() function.	1 week ago
Broque Thomas	2c0a0da9ea	Address Copilot doc-drift review ... Four stale doc/comment references caught by Copilot's pass: - core/download_plugins/base.py: TYPE_CHECKING comment said the shared dataclasses lived in core.soulseek_client. They were moved to core.download_plugins.types in this PR. Comment updated. - core/qobuz_client.py: reload_credentials docstring still referenced soulseek_client.client('qobuz') after the global rename to download_orchestrator. Updated to download_orchestrator.client(...). - webui/static/helper.js: the older WHATS_NEW entries for the plugin contract + engine refactor still claimed backward-compat self.<source> attributes were preserved. Followup commits in the same PR removed them. Each entry now flags the followup explicitly and points at the "Drop Backward-Compat Per-Source Attrs" entry above it so the changelog is internally consistent. - docs/download-engine-refactor-plan.md: Compatibility commitments section listed orchestrator.<source> attribute preservation as a guarantee. Cin's review pass removed those attrs (and renamed the global handle from soulseek_client to download_orchestrator) — both are breaking changes for in-tree callers (which were migrated) and in-flight branches (which will need to update). Section rewritten to document the actual outcome.	3 weeks ago
Broque Thomas	d17365296a	Lift shared download dataclasses + boot via singleton factory ... Two architectural cleanups on top of the download engine refactor. (1) Shared dataclasses move to neutral plugin package. TrackResult, AlbumResult, DownloadStatus, SearchResult lived in core/soulseek_client.py for historical reasons — every other plugin imported them from the soulseek module just to satisfy the contract, coupling 8 clients to a sibling source for type imports only. Moved them to the new core/download_plugins/types.py module and updated all 14 import sites across the deezer/hifi/lidarr/qobuz/soundcloud/tidal/ youtube clients, the engine, matching engine, redownload helper, and tests. Clean break, no backward-compat re-export. (2) web_server.py boots the orchestrator via the singleton factory. After construction it now calls set_download_orchestrator(...) so get_download_orchestrator() returns the same instance the global handle points at instead of lazily building a separate orchestrator. Matches the get_metadata_engine() pattern.	3 weeks ago
Broque Thomas	ea654f664e	Cin-1: Make DownloadSourcePlugin inheritance explicit on every client ... Cin's review feedback: the plugin contract was discoverable only from the registry, not from the client files themselves. Reading `youtube_client.py` cold gave no signal that the class participates in the DownloadSourcePlugin contract. Every download client class now inherits DownloadSourcePlugin explicitly: - SoulseekClient(DownloadSourcePlugin) - YouTubeClient(DownloadSourcePlugin) - TidalDownloadClient(DownloadSourcePlugin) - QobuzClient(DownloadSourcePlugin) - HiFiClient(DownloadSourcePlugin) - DeezerDownloadClient(DownloadSourcePlugin) - SoundcloudClient(DownloadSourcePlugin) - LidarrDownloadClient(DownloadSourcePlugin) Adjustments: - core/download_plugins/base.py — moved TrackResult/AlbumResult/ DownloadStatus imports under TYPE_CHECKING since they're only used in type annotations. Without this, clients inheriting the contract create a circular import. - core/download_plugins/__init__.py — drops DownloadPluginRegistry re-export. Importing the package no longer triggers the registry's eager client imports (which would also be circular for clients that import from the package). Callers that need the registry import it directly: `from core.download_plugins.registry import DownloadPluginRegistry`. Suite still green (335 download tests).	3 weeks ago
Broque Thomas	19fbcf267d	Add DownloadSourcePlugin contract + registry ... `core/download_plugins/` defines the canonical interface every download source must satisfy and the registry that holds them. Single source of truth replacing the orchestrator's hardcoded `[self.soulseek, self.youtube, ...]` lists scattered across 6+ dispatch sites. Pure additive — no consumers wired through the registry yet.	3 weeks ago