8 Commits (50fe4bec97757646884b0459547c190849706dfd)

Author	SHA1	Message	Date
Broque Thomas	d8437c87c6	Fix Album Completeness Auto-Fill on Docker / shared-library setups (#476) ... GitHub issue #476 (gabistek, Docker on Arch host): "Auto-Fill" / "Fix Selected" on the Album Completeness findings page returned "Could not determine album folder from existing tracks" for every album. Reproduces on any setup where the media-server library lives outside the SoulSync transfer/download folders — Docker is the headline case but native installs that point Plex at a NAS via SMB hit it too. Root cause: `core/repair_worker.py:_resolve_file_path` only probed the transfer + download folders. Docker users have their Plex/Jellyfin library bind-mounted at /music (or similar) — neither configured in SoulSync. Every existing track got silently treated as missing, so `album_folder` stayed None and the fix workflow bailed. The same incomplete logic was duplicated four more times in the repair_jobs/ modules, all with the same bug. Album Completeness was just the most user-visible — the same setups were also producing false "missing file" findings from Dead File Cleaner, silent skips in MBID Mismatch Detector, etc. The web server already had the correct logic at `web_server.py:_resolve_library_file_path` (probes transfer + download + Plex-reported library locations + user-configured library.music_paths). The repair workers had never been updated to match. Fix: - New `core/library/path_resolver.py` extracts the union logic into a single shared function `resolve_library_file_path()`. Probes (in order, deduped): explicit transfer/download kwargs, config-derived soulseek.transfer_path/download_path, Plex-reported library locations (when a plex_client is passed), user-configured library.music_paths. Each defensive: malformed config or a flaky Plex client degrades to the dirs that did succeed. - `core/repair_worker.py:_resolve_file_path` becomes a delegating wrapper preserving the legacy signature, with a new `config_manager` kwarg. All 15 in-tree call sites updated to thread `self._config_manager` through. - `core/repair_jobs/dead_file_cleaner.py`, `mbid_mismatch_detector.py`, and `lossy_converter.py` get the same treatment: duplicate function replaced with a thin wrapper, call sites pass `context.config_manager`. - `core/repair_jobs/acoustid_scanner.py` and `unknown_artist_fixer.py` (which used to import from repair_worker) now call the shared resolver directly with `context.config_manager`. Side benefit: every other repair job (Dead File Cleaner, MBID Mismatch Detector, Lossy Converter, AcoustID Scanner, Unknown Artist Fixer) also stops missing files in the media-server library mount. Single fix unblocks five user-visible features. Tests: `tests/library/test_path_resolver.py` — 20 cases covering all four base-dir sources, suffix-walk algorithm, dedup, defensive paths (None plex client, malformed config entries, raising config_manager.get, broken plex attribute access), Docker path translation. Full suite 1677 passed locally. WHATS_NEW entry under '2.4.2' dev cycle.	3 weeks ago
Broque Thomas	24c2d75c6d	Make extract_external_ids recognize all source-tagging conventions ... Smoke-testing the just-merged provenance PR against live logs revealed the new ID-match block was silently no-opping: no [ExtID Match] / [Provenance Match] log lines despite the code path being live. Tracing revealed two related gaps in extract_external_ids' source detection: 1. **Underscore-prefixed key.** Deezer / Discogs / Hydrabase clients tag normalized track dicts with ``_source`` (underscore prefix — convention used in 8+ places across core/). The extractor only looked for ``provider`` and ``source``, so Deezer-sourced tracks silently returned no IDs. 2. **No provider field at all.** Spotify and iTunes raw API responses carry ``id`` but no provider/source key of any kind. The extractor couldn't disambiguate the native ``id``, so Spotify-primary scans would have hit the same silent miss once the user switched primary sources. Two-part fix: - ``extract_external_ids`` now recognizes ``_source`` as another candidate provider field. - New optional ``source_hint`` parameter lets the caller supply the configured primary source as a fallback when the track dict has no provider field of its own. Track-side provider field still wins when present (defensive against a wrong hint). Watchlist scanner now passes ``get_primary_source()`` as the hint so both naming conventions (Deezer-style _source, Spotify-style no-tag) get handled uniformly. 6 new regression tests cover: - _source recognized for Deezer - _source recognized for Hydrabase (cross-provider mapping) - _source recognized for Discogs (no library column — verifies graceful no-crash) - source_hint disambiguates raw tracks for spotify/itunes/deezer - track-side provider takes precedence over hint - None hint defaults safely Full pytest 1630 passed; ruff clean. After this lands and the server restarts, watchlist scans should produce [ExtID Match] / [Provenance Match] log lines for tracks already on disk regardless of which metadata source the user has configured as primary.	3 weeks ago
Broque Thomas	34ba26f5c8	Persist source IDs at download time + backfill onto tracks on sync ... Followup to fix/watchlist-external-id-match. The companion PR closed the demand side — the watchlist scanner asks for tracks by external IDs before falling back to fuzzy. But for users on Plex / Jellyfin / Navidrome the supply side was still broken: tracks.spotify_track_id (and the other ID columns) only got populated by the asynchronous enrichment workers, sometimes hours after the file was actually written. During that window the ID match fell through to fuzzy and the bug returned. We were already collecting every ID during post-processing — they live in the `pp` dict in core/metadata/source.py:embed_source_ids and get embedded into file tags. We just dropped the in-memory copy afterwards. This PR persists them and uses them: - Schema migration adds spotify_track_id / itunes_track_id / deezer_track_id / tidal_track_id / qobuz_track_id / musicbrainz_recording_id / audiodb_id / soul_id / isrc columns + indexes to the existing track_downloads table (already keyed by file_path). - core/metadata/source.py:embed_source_ids exposes pp["id_tags"] and the resolved ISRC back to the import context as _embedded_id_tags / _isrc. - core/imports/side_effects.py:record_download_provenance reads those context fields and passes them to db.record_track_download, which now accepts the new ID kwargs and persists them. - New db.get_provenance_by_file_path with exact + basename-suffix fallback (handles container mount-root differences between download-time path and media-server-reported path). - New db.backfill_track_external_ids_from_provenance copies IDs from track_downloads onto a tracks row idempotently — COALESCE on every column preserves any value the enrichment worker already wrote (enrichment is more authoritative for late binding). - database/music_database.py:insert_or_update_media_track (the single insertion point used by every Plex / Jellyfin / Navidrome sync) calls the backfill immediately after each INSERT/UPDATE. - New core/library/track_identity.py:find_provenance_by_external_id used as a second-tier fallback in watchlist_scanner.is_track_missing _from_library — catches the window between download and media-server sync. Caller checks os.path.exists on the provenance file_path before treating it as "already in library" so a deleted file doesn't prevent re-download. Effect: freshly downloaded files become ID-recognizable to the watchlist on the very next scan, no enrichment-wait window. 19 regression tests in tests/test_provenance_id_persistence.py: - Schema migration adds expected columns + indexes - record_track_download persists every ID kwarg - record_track_download backward-compat (old kwargs still work) - get_provenance_by_file_path: exact match, basename fallback for mount-root differences, multi-record latest-wins, defensive None - backfill: copies all IDs, preserves existing via COALESCE, no-op when no provenance exists - find_provenance_by_external_id: per-ID lookup, ISRC cross-bridge, OR semantics, latest-wins on multiple matches Out of scope: backfilling provenance for files downloaded BEFORE this PR (their track_downloads rows don't carry the new IDs). Those continue to wait for enrichment. Acceptable — only affects historical files; new downloads benefit immediately. Full pytest 1625 passed; ruff clean.	3 weeks ago
Broque Thomas	ecb8939c80	Match library tracks by external IDs before fuzzy in watchlist scan ... Reported case (CAL): a track already on disk got re-downloaded by the watchlist scanner on every scan. Library DB had stale album metadata for the file (track tagged on album "Left Alone") while the metadata source reported it on a different album ("NPC" single). The title+artist+album fuzzy block correctly said the album names didn't match and declared the track missing — but the file's stable external IDs (Spotify ID, ISRC, etc.) unambiguously identified it as the same recording. The earlier compilation-album fix (PR #461) handled qualifier drift ("OST" vs "Music From The Motion Picture"). This case is two genuinely different album names referring to the same song. Fix: provider-neutral external-ID short-circuit before the fuzzy block in `is_track_missing_from_library`. Pulls every recognized ID off the source track (Spotify / iTunes / Deezer / Tidal / Qobuz / MusicBrainz / AudioDB / Hydrabase / ISRC), runs a single SELECT against the indexed external-ID columns on the `tracks` table, and treats any hit as "track exists in library — don't re-download". If no IDs are available (older imports without enrichment, library scans that didn't populate external IDs), falls through to the existing fuzzy logic so the safety net stays intact. New `core/library/track_identity.py` module with two helpers: - `extract_external_ids(track)`: handles dict and object-style track shapes, direct-field aliases (spotify_id / spotify_track_id / SPOTIFY_TRACK_ID), and provider-disambiguated native `id` fields (when track has `provider='deezer'` and `id='X'`, treats X as a Deezer ID). - `find_library_track_by_external_id(db, external_ids, server_source)`: builds an OR of indexed column matches with IS NOT NULL guards, optional server_source filter that also passes legacy NULL rows, single-row LIMIT. ISRC bridges across providers — a library track imported via Deezer can be matched against a Spotify scan when both sides carry the same ISRC. 43 regression tests in `tests/test_library_track_identity.py`: - 9 ID-extraction tests for direct fields (Spotify / iTunes / Deezer / ISRC / MBID / AudioDB / Hydrabase) - 8 ID-extraction tests via the provider field (8 providers + source alias + missing-provider-ignored) - 7 mixed/defensive tests (multiple IDs, object-style, empty strings, None track, numeric coercion) - 8 lookup tests (per-provider + ISRC cross-bridge) - 3 OR-semantics tests - 4 server_source filter tests - 2 ID-column-map sanity tests Full pytest 1606 passed; ruff clean.	3 weeks ago
Broque Thomas	b395e33820	Lift redownload_start to core/library/redownload.py ... Body byte-identical to the original. Spotify proxy via registry, iTunes/Deezer client shims wrap registry helpers, _resolve_library_file_path, _attempt_download_with_candidates, and missing_download_executor are injected via init() right after _init_wishlist_failed where all three deps are already defined. web_server.py: 35239 → 35063 (-176 lines).	4 weeks ago
Broque Thomas	8299dc211e	Lift _run_duplicate_cleaner to core/library/duplicate_cleaner.py ... Body byte-identical to the original. The shared state dict, lock, docker_resolve_path helper, and automation engine are injected via init() at the lift point, where all four originals are already defined. web_server.py: 37015 → 36833 (-182 lines).	4 weeks ago
Broque Thomas	dae7f21265	Lift _search_service to core/library/service_search.py ... Lifts _search_service and its _detect_provider helper. Both bodies are byte-identical to the originals. The nine enrichment worker handles (spotify/itunes/mb/lastfm/genius/tidal/qobuz/discogs/audiodb) are injected via init() right after qobuz is constructed, which is the last worker to come up — and well before Flask starts accepting requests, so the route handlers never see unbound workers. web_server.py: 37245 → 37015 (-230 lines).	4 weeks ago
Broque Thomas	3a6597561a	Lift _execute_retag to core/library/retag.py ... Pulls the 258-line retag worker out of `web_server.py` into a new `core/library/` package. Pure 1:1 lift — wrapper keeps the original entry-point name so the retag-trigger endpoint continues to work without changes. What `execute_retag` does: 1. Fetch album + track metadata for the new `album_id` (Spotify or iTunes — the Spotify client transparently falls back). 2. Load existing files in the retag group from the DB. 3. Match each existing track to a new Spotify track: - Priority 1: same disc + track number. - Priority 2: title similarity >= 0.6 (SequenceMatcher). 4. For each matched pair: - Re-write metadata tags via `_enhance_file_metadata`. - Compute the new path via `_build_final_path_for_track` and move the audio file (plus .lrc / .txt sidecars) if the path changes. - Drop an orphaned cover.jpg if it's left in an empty directory. - Clean up empty parent directories left behind. - Download the new cover art into the new album dir. 5. Update the retag group record with new artist / album / image / total_tracks / release_date and the appropriate Spotify-or-iTunes album ID (numeric → iTunes, alphanumeric → Spotify). 6. Mark the retag state 'finished' (or 'error' on exception). Strict 1:1 byte parity: The original mutated `retag_state` as a module global (the function declared `global retag_state` even though it only mutates in place). Here `retag_state` is exposed through the `RetagDeps` proxy as a Python property so the lifted body keeps `name[key] = value` / `name.update(...)` syntax. The property setter rebinds the web_server.py reference if the function ever reassigns it (currently it doesn't, but the setter is wired for parity with the watchlist lift). Diff vs original after `deps.X` → global X normalization is **zero differences** apart from the dropped `global retag_state` decl and the inline `from database.music_database import get_database` (replaced by deps.get_database()). 258 lines orig = 258 lines lifted, byte-identical body otherwise. Dependencies injected via `RetagDeps` (13 fields) — config_manager, retag_lock, spotify_client, plus 8 callable helpers (get_audio_quality_string, enhance_file_metadata, build_final_path_for_track, safe_move_file, cleanup_empty_directories, download_cover_art, docker_resolve_path, get_database) and 2 property delegates (_get_retag_state / _set_retag_state). Tests: 11 new under tests/library/test_retag.py covering setup error paths (no album data, no album tracks, no existing tracks), track-number priority match, title-similarity fallback, no-match skip, missing file skip, file move when path changes, group record update (spotify vs iTunes ID branching by alphanumeric vs numeric album_id), multi-disc total_discs computation. Full suite: 1330 passing (was 1319). Ruff clean.	4 weeks ago